Moving on, let’s see how well ZOL takes to Voidlinux. I am targeting a workstation setup to replace my aging Archlinux mdraid/lvm daily driver that’s become a sketchy proposition since Archlinux adopted systemd.

As of 2.02-beta3 Grub2 sports full ZFS feature flag support for ZOL 0.6.5. Opinions are still divided regarding use of separate /boot and / partitions but I’m going to live dangerously and do a single ZFS pool for root/boot for this setup.

Voidlinux is not exactly mainstream and I erroneously presumed I would have to roll my own ZOL dkms package from source. Turns out it’s been available in xbps since August 2014. Void takes a minimalist approach, however, and there isn’t enough free space provided by the stock live image to install ZFS. Boo Hiss! What’s a body to do? Well, you can either spin up your own using void-mklive or save yourself some work and grab the ZOL enhanced image I spun up for this project. ^[3]

Make a bootable usb key. Substitute for sdX as appropriate.

Now git yerself booted into that puppy. I like to ssh in from my main squeeze so I have ready access to my favorite tools. Void enables ssh out of the box. No additional muss nor fuss. Me likey. ^[4]

That was easy ;D Now that we’re jacked into our target install box, let’s verify our hard disk info:

Time to partition our drives. I am going to set this rig up with a raid1 mirror using sda and sdb. This box is capable of BIOS booting from GPT partitioned disks. Lucky me (may the Saints preserve us from UEFI hell). I prefer to use GPT over MSDOS style partitions for the following reasons:

As such, the iso I built includes gptfdisk. Sweet ;D Let’s Git-R-Done!.

Okay, what was that all about, eh? Lemme 'splain…​.

The first partition, partition #2, is the BIOS boot partition. If grub-install finds a partition type ef02, it installs itself there. While this partition only requires on the order of 31KiB, it’s commonly made 1MiB for rounding and to allow for future growth. As UEFI uptake by hardware manufacturers "progresses" I suspect GRUB’s future is less than bright but I double that nonetheless as my way of making a snarky comment about GRUB bloatware.

The second partiton, partition #1, will host our ZFS pool. Various how-to’s around the 'net use bf00, bf01, bf05, or whatever. ZFS doesn’t actually care what type this partition is. Labeling it something 'Solaris-Y' is just a crutch for us humanoids to help us remember it’s being used for a ZFS pool. I’ve chosen Solaris root here to help remind me that this is a root zpool.

Okay, what’s up with the +/- 128M thangs? I doubt I’ll ever use these disks with Mac OSX but predicting the future is tricky at best. Reportedly, some tools can also hork things up if they don’t have sufficient breathing room. ^[5] Hence, I suggest following Apple’s recommendation of leaving 128MiB of free space after each partition. Leaving 128MiB free space at the end also provides a bit of insurance against manufacturer variance should you ever need to replace one of your drives Moreover, 256MiB is insignificant on modern drives.

Finally, what is up with my crazy numbering? Well, again, it doesn’t really matter what number you give your partitions. We could have just as easily used 3 and 7. Indeed, ZOL on whole disk defaults to using 1 and 9. Designating the partiton we can boot from as partition #1 is just another memory crutch in the event something goes awry and we need to invoke GRUB command line commando mode.

With all that 'splainin out of the way, let’s clone that table onto our second device, sdb.

Being an exact copy, we now have two drives with identical GUID’s. Not good. So let’s give sdb a new GUID and confirm that sda and sdb are indeed unique:

Nice. We have at least one 7 in each GUID (I changed one of them behind the scenes ;D)

We are almost ready to create our ZFS pool. ZFS performs best when its block size matches your drives' sector size. ZFS automagically detects this and sets an alignment shift value as appropriate. Nevertheless, I prefer setting ashift explicitly. Read on, dear reader, for enlightenment as to why.

Modern Advanced Format (AF) disks use 4K. Some disks, particularly early AF disks, lied about this and reported 512 byte logical sector sizes in order to maintain compatability with Windows XP. Legacy operating systems notwithstanding, does anybody trust hardware manufacturers to tell the truth? Nope, didn’t think so. They’re in the marketing business, fer' chrissakes! So let’s double check it. ^[6]

I like to use good quality drives. These are older WD RE2’s, which employ 512 bytes for both physical and logical sector size. Best practices suggest setting ashift of 12 (2^12 = 4096 = 4K) regardless since if a drive fails the replacement will almost certainly use AF. Well, if either of these drives fail both will be replaced and I’ll just clone the data over to a new pool. In the meantime, using 4K on 512 byte drives wastes space so I will be creating my pools using ashift=9 (2^9 = 512).

ZFS on Linux supports several options for selecting /dev names when creating a pool. For testing purposes top level dev names, e.g. /dev/sda1, are the default for consistency with other ZFS implementations. I’m going to use 'by-id' for this set up. This decision will cause us some pain points due to a GRUB bug that’s been outstanding since the last POTUS election, but these will be minor enough to overcome that the benefits justify the costs. Your mileage may vary. ^[7]

We’re almost ready to create our pool but not quite. Commercial software frequently utilizes "hostid" to deter license/subscription abuse. ZFS pools use "hostid" as a precaution to keep pools from inadvertenly being imported by the wrong system. This is more of a potential issue in datacenter environments than home but I still like to follow best practices. Voidlinux, being a true FOSS project, has no need to set a hostid and falls back to a default of 00000000.

So we need to give it one. You can choose whatever you like for this but I will be basing mine on the MAC address of the first ethernet device. ^[8]

Now let’s write that to /etc/hostid. If you’re doing the copy pasta polka, don’t forget to strip out the colons.

Alright then. Let’s create that pool. While the name of the pool can be anything (w/in limits of naming restrictions) many guides around the net use "rpool" as their defacto default top level pool name. Some platforms automatically import anything so named. Not my cup of tea. Instead, I prefer to base my top level names on hostname. ^[9]

Let’s check out our handiwork.

Lord have mercy! As of yet our pool doesn’t have any filesystems/datasets.

Let’s make some. But what?

I like using high level "containers" to separate out stuff that changes often from that which does not. Ditto essential platform bits from non essential stuff like package distfiles. This sets me up for being able to snapshot the important things without wasting space on bits that are readily replaceable.

Opinions are varied on how to handle logs. I prefer mine to be comprehensive. Should I roll back a snapshot, I don’t want my logs to go back in time as well and loose information. Additionally, archived logs are usually compressed by log rotating daemons. ZFS features built in compression at the filesystem level. Kind of silly to be compressing twice, eh? I may decide to let ZFS handle the compression at the dataset level and not have my logging daemon compress logs at all. No zgrep’ing. I haven’t decided yet but breaking out logs to a dedicated data set provides the flexibility to season to taste.

I also like to separate out stuff that is not necessarily tied to a particular host or operating system, e.g. user directories and application data directories. Other candidates include things you may want to share out to other hosts, e.g. a directory of flac audio.

One can spend many hours of analysis paralysis scratching their noggin over such considerations. You may want to consult the Linux Filesystem Hierarchy Standard for insights. The good news is that ZFS makes it damn easy to change your mind later, so getting it perfect out of the gate isn’t do or die. Give it your best shot and get on with it. ^[10]

I will use "ROOT" as a container for essential platform system bits that I want to snapshot and possibly clone. ^[11]

Let’s give my users some love.

"VOID" will be a container for stuff that I don’t want/need to keep synchronized with my platform bits, e.g. logs, and/or can be readily replaced, e.g. package manager and source distfiles. I will also use this for locally installed software, i.e. things I build from source that are not under the Voidlinux umbrella.

All together now:

ZFS automatically mounts filesystems, creating any directories as needed. Of course ZFS also suports legacy mounts via /etc/fstab. This is not an either, or, and you are free to mix and match should you have need to embrace your inner lunatic. Yep. You guessed it. With the exception of swap space, I do not use legacy mounts. ^[12]

Properties set at pool creation time are inherited by child datasets.

As noted above I have yet to decide how I am going to handle my logs on Void. Should I so desire, turning off compression is a simple matter, e.g.

Just a small taste of the awesome sauce that is ZFS. See man zfs(8) for the full smorgasboard.

Void’s official repositories are signed with RSA keys. I’ll be using the glibc repo. Other repositories, e.g. musl, will have a different signature. You should confirm signatures as appropriate prior to proceeding. ^[13]

After doing its thing xbps-install exits with something similar to above.

Next stop on our journey is to get chroot’d into our fresh Void bits and do some final configuration.

We need copy the hostid file we created above into our new world.

Mount up! We ride!

Ready to chroot.

Yep. We’re in like flint. Check yer' saddle and confirm your mounts if you like.

Everything looks just dandy. Give your box a hostname.

Void’s BSD roots are evidenced in it’s use of rc.conf. I’ll set some basic stuff here. You can peruse the commented file for more suggestions.

Presuming you’re using glibc, configure your locale. If you’re a musl user, omit this step.

Confirm that the ZFS modules are indeed loaded.

Yippie yi yo kiyah!

Once upon a time there was talk of ditching ZOL’s use of a cachefile and you may have seen dated how-to’s that omit this. That plan was scraped, however, so we need to set our zpool.cache.

Set up some swap. You should match your system’s page size. This will most likely be 4096 but it’s always wise to confirm.

Edit your /etc/fstab using your editor of choice to end up with something that looks like this:

GRUB needs to know which dataset to boot from:

Let’s confirm this:

We also need to tune our kernel command for ZOL. Nowadays this is done via knobs in /etc/default/grub.

The Linux kernel attempts to optimize disk I/O via the use of various "elevator" schemes. We want to let ZFS handle it’s own I/O optimizations so we’ll use the noop elevator, which is basically a FIFO queue sans any extra logic.

Linux also presumes that the presence of a swap file means we’re super psyched about using power management features like suspend. Unlike traditional dedicated swap partitions, however, our ZOL swap device isn’t actually available for such. Hence, during bootup our kernel hangs on in quiet desperation for 120 seconds, hoping said missing swap dev will magically appear and it can resume being politically, if not technically, correct. This is a drag. So we’ll just give it a little spanking and tell it to get on with it by passing "noresume".

I prefer text to graphic terminals on boot. This is also a dedicated Linux workstation and I don’t want to screw around probing for other operating systems, so I’ll tweak those knobs as well. Fire up your editor of choice and caress your keyboard lovingly until you have something similar to what I’ve concocted below:

Remember those minor pain points about using disk by-id I alluded to when creating our pool? Well, they’ve just come to roost. Turns out grub-probe/install is too brain dead to actually recognize any of the /dev/disk/by-* options. Instead, it probes zpool status and prepends "/dev" to the devices returned thereupon, and pukes up the infamous "failed to get canonical path of /dev/ata-driveid-partX" error message.

So we’ll outfool the hapless fool by grabbing our vdevs:

And creating the requisite symlinks: ^[14]

grub-probe should now be able to deduce we’re using ZFS:

We’re finally ready to install GRUB:

Voidlinux uses dracut. We need to tune a couple things for ZOL. I am new to dracut and there may well be things I missed. See man dracut(8) and/or man dracut.conf(5) for enlightenment.

Setting "hostonly" tells dracut to skip all the extraneous needful things bundled into a generic kernel and generate an initramfs trimmed for booting our host’s particular hardware potpouri.

ZOL doesn’t use fsck, so that bit should be self explanatory.

I don’t think the add_dracutmodules line is required in light of our "hostonly" but it doesn’t hurt to explicitly ensure inclusion of the zfs and spl modules, especially since on this rig they will be essential for booting. As this is my first tour with dracut I’ll error on the side of caution.

Alas, for reasons that elude me, Void devs have included btrfs (shudder…​) in their default kernel. Oh well, nobody’s perfect. We shall endeavor to reform their evil ways. In the meantime, I am unsure whether including "btrfs" in the "omit_dracutmodules" line has any effect after having specified "hostonly", but explicitly excluding the mess that is btrfs makes me feel better and certainly doesn’t hurt.^[15]

Recall that 120s resume hang I mentioned? Omitting the resume module from our configuration is another solution to this issue and provides an additional benefit of shaving a few more KiB from our initramfs.

Sometimes I have fat fingers so I like to confirm I done did what I thought I did…​

We’re ready to regenerate our kernel and initramfs. As of this writing, Voidlinux is on a 4.6 kernel. As of your reading, you may not be. Beware of mindless copy pasta. Let 'er buck! ^[16]

Awesome ;D

Note that hook that just reconfigured GRUB and generated your grub.cfg. Interested parties can review my my grub.cfg for troubleshooting purposes. Heck, feel free to go for it anyway just because it’s so darn memezmerizng ;D

We’re almost ready to reboot into our shinny new ZOL workstation, but let’s do a couple final sanity checks first.

Cool. Our initramfs does indeed include the requisite zfs dkms. Let’s double check our zpool.cache.

We need to set a root password before bailing out of our chroot environment.

And umount some mounts just to keep things copacetic…​

Export our pool.

And reboot into our new rig!

After rebooting into our shinny new Voidlinux box, we need to enable some basic networking services. I’ll be using dhcp. I also like being able to ssh into my boxes. Season runit to taste and create symlinks as appropriate.

Now the real journey begins. Before venturing too far afield, you may want to take a snapshot of your fresh tableau noir.

Awesome sauce, indeed!

Congratulations! Welcome to the Void!

Stay tuned for exciting adventures in fleshing this puppy out ;D

I captured two terminal recordings that closely follow the steps outlined above. Exceptions are noted below.

Is Voidlinux for you? If you read my "The Challenge To Be Free" article you no doubt realize that Voidlinux ticks a lot of my boxes. So what’s not to like? Well, candidly speaking, documentation is minimal, at best. Be prepared to do your homework, be self reliant, and spend some effort troubleshooting. In addition to the wiki, Void sports a forum and a small but active irc channel on freenode. In the weeks I have been hanging out on #voidlinux, I have noted an influx of new users, many of whom seem to be looking to escape the tyranny of systemd. The other main incoming cohort tend to be from the embeded device camp. Some of these folks are quite frankly on the bleeding edge of newbie and invested minimal to zero effort reading documentation. I’m truly impressed by the patience of Void’s elder statesmen and like to take this opportunity to give them a big shout out! You know who you are. I have also noted some of these new users giving back and the wiki getting some love. Kudos to you as well! I think there is opportunity for a healthy community here. Don’t be a taker and exploit it, lest we burn helpers out. ^[17]

As for Gentoo? It’s still on my short, short list and I may end up there yet. I’m going to spend some time exploring Voidlinux before I make that call.