The Tar Pit

The following is a V patch on Diana's Keccak V tree: v_fixes_vpatch_sync.vpatch (seal). Additionally, as per ye mirroring discussion, my WWW contains a full mirror of the Keccak V tree. In addition to that, I'm also putting my seal on previous V patches, namely:

• v_mod6_genesis.vpatch.spyked.sig;
• v_mod6_99993.vpatch.spyked.sig; and
• v_keccak_vtools.vpatch.spyked.sig.

Getting back to my V patch, there's the following question: why the hell would I try to "fix" a fundamental Republican tool which as far as everyone knows works perfectly fine?

The answer'd then be, because there's one piece there that doesn't.

I use v.pl's "init"¹ command quite a bit when setting up new stuff, given that downloading V trees on multiple machines is reduced by this to one "v.pl init". However, while preparing the MP-WP V patches and trying to test a clean sync and press, I noticed the following:

$ ls -a .
.  ..  vk.pl  .wot
$ ls .wot
billymg.asc  hanbot.asc  spyked.asc
$ grep '^my $version' vk.pl
my $version  = "99992 K ";
$ ksum vk.pl
115b06a799eae048ffbd18642234dc234e9bf879ebdedf0ed9b4be69a410187537b618d2b4b581954b89d1f63ef0925f9ff1ae26f0882c28f05d3a1fcdb1a732 vk.pl
$ ./vk.pl init http://lucian.mogosanu.ro/src/mp-wp
Full vpatch sync complete to "/tmp/mp-wp/patches"
Seal sync complete to "/tmp/mp-wp/.seals"
$ ls patches/
 mp-wp_add-footnotes-and-textselectionjs.vpatch
 mp-wp_genesis.vpatch
'mp-wp_genesis.vpatch?'
'mp-wp_genesis.vpatch?ajax=photo_thickbox&i='''%20+%20encodeURIComponent(img.src)%20+%20'''&u=%3C?php%20echo%20urlencode($url);%20?%3E&height=400&width=500'
 mp-wp_html-comments-regrind.vpatch
 mp-wp_markdown-plugin.vpatch
 mp-wp_remove-tinymce-and-other-crud.vpatch
'mp-wp_remove-tinymce-and-other-crud.vpatch?ajax=photo_thickbox&i='''%20+%20encodeURIComponent(img.src)%20+%20'''&u=%3C?php%20echo%20urlencode($url);%20?%3E&height=400&width=500'
 mp-wp_thetarpit-theme.vpatch
 mp-wp_tools-ttp2mpwp.vpatch
 mp-wp_update-image-references-to-svg.vpatch

What the fuck is this, then? I'm used to this kind of mind-boggling weirdness by now, so I went and investigated. The first observation was that the "?ajax=photowhatever" text is part of the MP-WP genesis V patch itself -- and so is all the rest of that weird. The second observation was obtained by poking at the code itself: "init" calls sync_all_vpatches and sync_seals; and moreover there's a sync_vpatches that works similarly. Let's reproduce sync_all_vpatches (at 99992K) here, because it's precisely the code that produces the output above:

sub sync_all_vpatches {
  my ($URL, $out) = @_;
  if(!-d $out) { `mkdir -p $out`; }
  my $wget = "wget -q -r -nd -N --no-parent " .
  "--reject "index.html*"  $URL/v/patches/ -P $out";
  `$wget`;
  print "Full vpatch sync complete to "$out"n";
}

We observe that at the core of this syncing code lies a wget command that's run with some particular parameters, e.g. quiet, recursive, skip grabbing the parent etc., see the manpage for more. Also, running the command as given produces the same output as above, so we've isolated the weird to this very particular piece of code.

The problem... well, I'm not sure it's a problem per se, but at least in our naive interpretation, the problem with wget's recursive pagegrabbing algorithm is that it's a mindless brute that doesn't care whether the grabbed item is an actual web page or a text file or whatever. So given a URI, it'll download it and attempt to recursively grab links inside the downloaded file, irrespective of the URI in question pointing to a V patch or something else.

So then, assuming that the hierarchy under the "patches" and "seals" directories is flat, i.e. "patches" contains only regular .vpatch files, not also .vpatch under subdirectories, and that "seals" contains only regular .sig files, my solution was to limit the recursion depth to 1, i.e. by adding the -l 1 option to the wget command. Problem solved.

Now, it could be that my assumption is wrong, although in my experience there's no observed instances that break it. Thus comments are more than welcome. Moreover, given that the V patch discussed here indeed brings changes to a fundamental Republican item, I'm encouraging whoever uses this to triple-check the code before running it. Who knows what other creatures might lurk in there, and the last thing you'd like is screw up your V.

Filed under: computing.
RSS 2.0 feed. Comment. Send trackback.