This post introduces s-xml-feedparse-http-fixes:
So what do these changes slash "HTTP fixes" consist of, more precisely? Two pieces of code are being repaired -- one of the repairs being required by Feedbot, while the other is more or less an opportunity for making the code a tad clea[nr]er. The latter (simpler) one goes as follows:
+(defvar *http-request-timeout* 60)
...
- (parse-feed-string (http-request-with-timeout url 60)))
+ (parse-feed-string (http-request-with-timeout url *http-request-timeout*)))
In other words, we're calling http-request-with-timeout with an
operator-configurable timeout setting -- as it should have been in the
first place, but let's not go into too many details, lest I end up
burning myself and applying ice on the wound.
The former (more important) change is along the lines of:
+(defun http-request-thread (url timeout)
+ (handler-case
+ (drakma:http-request url :connection-timeout timeout)
+ ;; Note: this will have to be revisited once we have a sane view of
+ ;; all the conditions returned by drakma.
+ (t (c)
+ (return-from http-request-thread (values nil c)))))
... and a bit later:
(let ((req-thread (sb-thread:make-thread
- #'(lambda (url timeout)
- (drakma:http-request url :connection-timeout timeout))
+ #'http-request-thread
This means that the thread we're creating now runs the code in
http-request-thread instead of an anonymous function; but more
importantly, that the call to Drakma's http-request is now
guarded by a handler-case that, long story short, acts
as a wildcard exception catcher.
Feedbot benefits from this change by making sure that all HTTP requests -- that, as we know from above, occur on a separate thread -- are deterministic, i.e. the CL run-time doesn't end up throwing exceptions at us if somehow a request times out or who knows what other condition out of the operator's control. I know, this isn't the most fortunate of implementation choices, but well... if I do the right thing(tm) and fix Drakma at this very moment rather than later, and then that other thing, and then the other one, then I might get to publish Feedbot in a decade or so. Let's say that the debt is manageable -- for now, but not for long.
Up next: Feedbot, part the first.