A journey through the Gales installation process

January 31, 2020 by Lucian Mogosanu

Gales is... well, better to let previous writings to speak for themselves; more precisely, JFW's introduction to Gales Linux; the initial release; and Bvt's installation report are so far the foremost pieces on the subject, as far as I'm aware.

So at the end of last year, following my foray into Trinque's Cuntoo, I decided that I should also give Gales a look, given that it stems from principles that on a first glance looked very much aligned with Republican values. It took a while to get started, but last weekend I finally fired up my setup and did it, or at least most of it, in about six hours, the remaining work taking about another five or so, which puts the time spent on this about on par with the average Debian1 install.

Speaking of setups, the machine that I used for bootstrapping was pretty much the same as the one previously employed for Cuntoo, i.e. a QEMU virtual machine with about 3GB of RAM, a 20GB disk, a serial port and a VNC display, booting a disk with a Debian operating system. I started by grabbing the Gales release and syncing all the sources needed for bootstrapping. Then I went and read BUILD, which contains an exhaustive guide of how to get everything -- by and large, a compiler, the kernel and the root filesystem, this last piece containing all the base system utilities among others -- prepared for deployment to a new machine. Then finally, I went through INSTALL and adapted it to my own installation process, i.e. installing the root filesystem, bootloader etc. on an emulated external drive, which would afterwards be used as the boot medium. After a couple of hiccups2, this resulted in a working Gales, on which I built and installed and configured an OpenSSH, which to my eye represents the minimum required for a test machine.

The whole thing went quite smoothly from start to end, so I don't see any need to go into details here -- the annotated commands in BUILD certainly helped, as did my previous head-on encounter with Cuntoo. Anyway, I do have some comments to make, point by point:

Since the bootstrapping process is deterministic for the most part, it wouldn't hurt to at least give a rough estimation of the total space required for installation. I'm not asking for this out of nowhere, I find it genuinely useful to have an idea of the resources needed to do something beforehand, at least I remember PC computer games used to include such specs up until 2000-something, which was useful, as I could at least refrain from downloading a game when I found myself too poor to get it. Sure, there's:

The build tree can be anywhere, but for reproducibility the standard is /var/build/gales. It'll need ~3GB of disk space.

but for some reason, the estimated 3GB actually turned out to be around 10. Maybe I did something wrong?

Moving on, the bootstrapper uses tar with the --sort flag, which was introduced in version 1.28 and thus is unavailable on older systems... which unfortunately was my case. I'm not sure going through the lengths of sticking to some so-called "standard" would help too much here; actually, I'm not sure there is any standard for system utilities such as tar to begin with. Still, getting files sorted in the archive is a requirement for reproducibility, so some portable alternative wouldn't hurt, unless this also turns out to be a can of worms, in which case... well.

I guess in the end the standard will be TMSR-OS itself, which brings me to my following point: is Gales self-bootstrappable? I haven't tried going there yet, but looking for example at the Busybox tar included in Gales, this probably isn't the case. Don't get me wrong, I kinda like Debian Wheezy, but that's gonna go away at some point if it hasn't already; and even assuming it won't, if the need for (a) binary distribution ever arises, I'd rather get a signed blob from the WoT rather than sink into the pits of heathendom.

As for the minor nitpicks: in step 8 of BUILD, the guide guides the user to use mkinstaller to generate the initramfs; however, this depends on gen_init_cpio, which I built manually, i.e. by hitting make in the appropriate directory. Did I forget something there, or should it be built this way? If the latter, perhaps this should be part of the guide.

And finally, speaking of RAM filesystems, the initramfs thingamajigg didn't do anything for me, nor did I see its usefulness in this particular case. Since I went through the trouble of installing a Debian as a bootstrapper system, I used the very same environment for installation, but maybe there are use cases for booting the initramfs that I haven't seen.

Finally finally, getting OpenSSH up was all a matter of issuing a few gports commands to get the couple of dependencies there, then for the SSH package itself, then well, it turns out I was clueless about daemontools, which I proceeded to fix.

Overall, the Gales installer comes with a whole fucking lot of that clarity which other operating systems lack, which to be entirely honest made it a real pleasure to work through. Moreover, the BUILD document is merely a bootstrapping guide at the moment, and that's all good; but I think that in all this lies a palpable opportunity to build a full specification for a Republican OS. Even proceeding from POSIX doesn't sound like such a bad idea to me, and then throwing it away if its warts prove themselves unfixable. Either way, this would put some much needed cement on the idea of "TMSR-OS" and we'd all finally understand each other with respect to what the name means, which to me looks like a big step towards having the actual thing built.

Well, at least that's what I think. What do you?

  1. I don't know about you, reader, but I took my first steps into the world of GNU slash Linux on Red Hat -- which gave me very little knowledge about what a "Linux" means, to be honest; I was but a seventh grade lad back then, what can I say -- then messed around with Slackware for a bit, started growing up on Gentoo, then finally matured on Debian. And as far as the smoothness of the so-called "installation process" goes, Debian was by far the "easiest" and fastest experience.

    Sure, meanwhile they ruined that one, but this doesn't keep me from deploying the old ones, e.g. Lenny or Wheezy, for some quick testing and whatnot. Not like there's other operating systems to choose from, I'll confidently venture to guess that Diana's CentOS ain't that much different. 

  2. One of them related to disk space: the Debian operating system leaves more than enough space for installing a few programs starting from a 10GB disk. Compiling GCC and Linux, however, is more demanding, which had me going through the somewhat painful and error-prone process of resizing disks, partitions and so on. Well, it all went fine in the end.

    The other hiccup was related to my using the wrong kernel config, which was quite easy to remedy by drawing from previous experience with Cuntoo

Filed under: computing.
RSS 2.0 feed. Comment. Send trackback.

6 Responses to “A journey through the Gales installation process”

  1. #1:
    Diana Coman says:

    Re tar version, I bumped into that on CentOS 6 indeed, as reported.

    Re gen_init_cpio iirc there's a setting in the kernel config so possibly you didn't have that option set so it wasn't built (some old notes of mine say CONFIG_BLK_DEV_INITRD fwiw).

  2. #2:
    Mircea Popescu says:

    > Then I went and read BUILD

    Is it just me that's meanwhile developed a serious issue reading anything not in standard republican format ? I came to this point where I loathe having to go through man for instance, because fuck that scrolling style and where's the footnotes and the links and wtf is this.

    For eons I had been doing x --help | grep q, x --help | grep r, s, t, u, v in endless succession, and it always sucked, but by now it's also become intolerable.

    > Moving on, the bootstrapper uses tar with the --sort flag,

    Why, incidentally ?

    > which unfortunately was my case.

    It is almost universally the case, I for instance generally have 1.22

    > unless this also turns out to be a can of worms

    Piping should do this just fine, I'd think.

    > Even proceeding from POSIX doesn't sound like such a bad idea to me

    I don't think maintaining posix is avoidable. Though as recently alluded on trinque's site, it'll prolly be a standard matured off what the heathens call "199209" (as opposed to "200112"). In which vein lettuce never forget

    23 Sort does not sort in normal order!

    Why is sort not sorting correctly?

    Some years ago at the time of transition to locale aware libc routines this question arose very often. With the passage of time most people have learned about this behavior and workarounds are quite well known. But the question is still raised often. It is most often due to having LANG set to ‘en_US.UTF-8’ set in your user environment. At that point sort appears broken because case is folded and punctuation is ignored because ‘en_US.UTF-8’ specifies this behavior. Once specifically requested by LANG and other LC_* variables, sort and other locale knowledgeable programs must respect that setting and sort according to the operating system locale tables. That is why this is not a bug in sort, sort is doing the right thing, even if the behavior is generally undesirable. Most of the language specific locales have tables that specify the sort behavior to ignore punctuation and to fold case. This is counter intuitive to most long time computer users!

    Of course locales are a blessing to non-english speaking computer users. Many languages use non-ASCII fonts and character sets. The POSIX standards and the GNU utilities support those by using the installed system library sorting routines. By using locales languages can specify a specific ordering. This can be done by a translator well after the program has been written and by translators not known by the program author. It is a dynamic change to the program. However, when those tables exhibit an undesired behavior then it can also break an otherwise correct program. If locale tables force a locale specific collation sequence when a standard collation sequence was desired then this is most noticeable with the sort command and so it bears the full force of the problem.

    This locale sorting behavior is configured in your environment. This is due to the fact that you or your vendor have set environment variables that direct the program to use locale specific sorting tables which do not sort as you expect. You or your vendor have probably set environment variables such as LANG, LC_COLLATE, or LC_ALL to ‘en_US.UTF-8’. The locale collating table selected by the locale setting is not part of the GNU program but part of your vendor’s system release.

    To select a standard sort ordering simply set LC_ALL to ‘C’ or ‘POSIX’.

    Fuck the fucking "non-english speaking" morons. They have no business touching the machines, off to the ovens with them, what the fuck, C programs written in arabic ? It ain't ever gonna happen, nor is the dumbphone ruminant ever touching anything to do with computers.

    Nobody, let me say it again, NOBODY was yet helped in any way to any degree by the attempts to make computers moron-tolerant. The intended market still didn't get on computers, thirty years later, but found yet-another sort of amiga to mindlessly & uncomprehendingly mash the buttons off of. None of the retards on fetlife misrepresenting themselves through stolen pictures ever sorted anything ; for lack of enough mental power as'd support the sort of curiosity directed towards the outside world such as would be capable of producing large enough collections for sorting to be even meaningful to them as an activity. Watch them "work", they do it "manually" -- like everything else in their bovine, inconsequential lives. Because that's what they fucking are. Libc ain't about to "change them", no matter what it does, just like the women that spawn them ain't gonna "change" anything.

  3. #3:
    Jacob Welsh says:

    Sorry about the tar --sort troubles; silly me, thinking that since it was in my local man page and not noted as "oh yeah we just cooked this up the day before you built your box" it was therefore a thing.

    > Why, incidentally ?

    @Mircea Popescu: the dream is to have installer binaries that come out the same no matter what system you bootstrap from. Left to its own devices tar enumerates directories in arbitrary (presumably filesystem) order.

    > Piping should do this just fine, I'd think.

    Recalling that tar takes file names through argv, which has a limited length, but tar output is concatenative, I come up with:

    find DIR -type f -o -type l -print0 | LC_ALL=C sort -z | xargs -0 tar -cf - | gzip > DIR.tar.gz

    This misses empty directories. But see also http://fixpoint.welshcomputing.com/2019/gales-linux-initial-release/#comment-182 .

    The localization thing is one drawback to musl - though its implementation is limited, they see this as a problem and have been expanding it. Not that we have to import their decisions past any given point, but it could complicate reviewing their changes for possibly-important fixes.

    > Since I went through the trouble of installing a Debian as a bootstrapper system, I used the very same environment for installation, but maybe there are use cases for booting the initramfs that I haven't seen.

    @spyked: nothing wrong with doing it that way i.e. installing to a secondary drive (virtual or physical). I find the initramfs approach nicely flexible though in that nothing needs to be mounted after boot. For instance, with a freestanding install environment you can install to a single-drive system while fully repartitioning it, if you're careful. It can also provide a recovery boot option in the event you hose your root partition in some way.

    Of course an initramfs with the appropriate utilities & scripts also enables root fs on fancier devices like lvm, LUKS, software RAID or network. Not sure how much that matters.

    Sounds like @Diana Coman has it re building gen_init_cpio.

    > which brings me to my following point: is Gales self-bootstrappable?

    Yes, except for the tar thing (and one can simply omit the --sort for now).

    Sorry the disk usage note was anti-helpful, I'll have to take a closer look on the next pass. I didn't pull the 3G out of nowhere but perhaps the final build tree size is lower than some intermediate step.

  4. #4:
    spyked says:

    @Diana Coman: Huh, it seems I missed your report re. Gales, my bad. Thank you for sharing it!

    Re. gen_init_cpio, I'm building a pretty barebones kernel indeed, without an init ramdisk. Building it from hand was effortless though, so no real problem there.

    @Mircea Popescu:

    > I came to this point where I loathe having to go through man for instance

    Not only this, but there's plenty of real documentation missing from them or otherwise misplaced, e.g. constants to system calls and the likes. And even when said #defines are there, the descriptions are so vague that it's near damn impossible not to botch some piece of code proceeding from them.

    > Why, incidentally ?

    Filesystems don't provide any guarantees regarding the order of entries in a directory; nor do I think that they should, but this brings forth the necessity of some way to sort them.

    > In which vein lettuce never forget

    Fucking hell, yes, there's nothing there worth using besides LC_ALL=C.

    @Jacob Welsh:

    > Sorry about the tar --sort troubles

    For the record, I don't think that's my fault, nor mine, but of those who wrote the manpages mentioned above. Even the muchly contempted PHP states explicitly all the versions implementing some functionality, whereas tar, a "standard" system utility has nothing of the sort, even the change history keeps being buried under piles of new "versions".

    So IMHO it's worth stating in the documentation that some command is expected to have certain flags available, which in the end would lead us to a complete enumeration of system utilities and the functionality they provide, which IMHO would be worth at least as much as the current POSIX spec.

    > The localization thing is one drawback to musl

    For the record, I wholeheartedly agree with signalling the Republican point of view.

    > Not sure how much that matters.

    Since people do boot their stuff off the network, then I suppose network mounts aren't all that irrelevant either. I'm not worrying about that for now though.

    > Yes, except for the tar thing (and one can simply omit the --sort for now).

    That's great news, actually! I'll give self-bootstrapping a spin sometime and report back.

  5. [...] of delightfully small size and clear setup that works reportedly ~everywhere, from Panama to the tar pit of many virtual machines and the backtrace of various network [...]

  6. [...] and invest the proceeds expanding the salt farms. [↩]Reviewed to date by bvt (WoT : bvt) and Lucian Mogosanu (WoT : spyked) and on which Mircea Popescu (WoT : mircea_popescu) noted [...]

Leave a Reply