Dumb Partition Scheme

I've written before about what I think is a good way to layout your partitions. Now I want to show a bad way! This is from an actual, FreeBSD, dedicated server that someone I know rented. (The provider shall go unnamed ... to protect the guilty.) Let's look:

$ df -h -t ufs
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ad0s1a    6.6G     79M    6.0G     1%    /
/dev/ad0s1e    965M     20K    888M     0%    /tmp
/dev/ad0s1f    9.5G    2.1G    6.6G    24%    /usr
/dev/ad0s1d    6.6G     63M    6.0G     1%    /var
/dev/ad0s2     9.0G    6.1G    2.2G    73%    /usr/home

The big problems here are wasting all that space in the root partition and then dividing the /usr partition. (I won't complain about the size of /var, since it was meant to be a server; or having a separate /tmp since that is still a common practice.) But, the icing on this cr4p cake is the home partition. It wasn't bad enough to just put it on another disklabel partition, let's put it on it's very own slice! (a.k.a. DOS partition)

I'm still awfully glad when any place offers FreeBSD as an option on the dedicated servers. I don't know if this place does any better laying out the disks for the other OS's. But, it's often useful (and sometimes amusing) to have a bad example. :)

How many disklabel partitions should I have?

In a previous post, I talked about my typical disk layout. I don't do anything radical. From the standard, auto-defaults I eliminate the "/tmp" partition, linking it back to "/var/tmp", and then tweak the sizes of the partitions, based on factors like the machine's future role and total space available. To be clear, I'm just talking about the file system partitions here, not the swap partition(s).

Why change the layouts, why does it matter? If you've got a big disk, why shouldn't you make lots of partitions?

The problem with lots of partitions is that it creates bottlenecks. Rather than having your entire disk available, you really only have the space in the partition that is being used at the time. Exceed that and you'll get the dreaded "warning: filesystem full" message and probably application errors. So, in the absence of any other information, the best partition layout is one big partition for the entire disk. That's our starting point.

Right away, however, there is another partition that we should separate from the single mega-partition: we should have a separate root partition ("/"). The first reason for this is that FreeBSD is laid out in such a way that the essential files and utilities will normally end up in the root partition. "/etc", "/bin", and "/sbin" should all be under the root partition and then they will be easily available for booting, for use in repairing the other filesystems, and what have you. Historically, part of the idea was that perhaps you couldn't even mount the other filesystems before you'd used the various utilities on the root filesystem to repair them. That may not really apply in the same way now that we have background fsck. There've also been concerns about enabling soft updates on the root partition. See here, for example -- and even that (the FAQ) discounts those concerns somewhat.

So, what's the bottom line? Upon examination, none of the issues with a single, large, filesystem partition appear to be critical -- you probably could lay out your disk that way and it probably would work fine. But, I haven't tried that, and I maintain that having a separate root partition is still the best way to go, even if it's less crucial than it once was. The important thing is you have nothing to lose by making a separate "/" partition and several things to gain: faster booting and better recovery and repair, for example.

On all of my machines I will also add a partition for "/var". The rationale for this is precisely that it is a bottle neck! I wish to isolate the logs and mail spool and other write heavy applications from the rest of my disk so that if(/when) things go wrong, it won't fill the entire disk. This is a double-edged sword, of course. If you stuck with just two partitions (root and everything else) you'd have a lot longer to notice a run away log. But, I've found BSD to be very resilient to a full "/var" partition. (I've also found that it's good to have some alerting for high utilization. :) )

Another rationale for the separate "/var" partition is to allow FreeBSD to optimize the I/O of those files. Chapter 2 of the Handbook puts it like this:

"Putting these [/var] files on another filesystem allows FreeBSD to optimize the access of these files without affecting other files in other directories that do not have the same access pattern."

... Which makes sense since "/var" will presumably have many small writes, unlike the other file systems.

And, voilà, the rest of the disk should be dedicated to the "/usr" partition. Since "/home" is linked there and all third-party software is installed there by default, you're probably going to need all that space.

Install tip: disklabel partitions

I've got a quick tip regarding how you partition your FreeBSD slice when you are installing. I'm not going to address the D0S partitions (or disk slices as they're usually called in FreeBSD), but the FreeBSD partitions that will be put within the slice (or slices) that you install FreeBSD into. This is covered in the Handbook in the section "2.6.5 Creating Partitions Using Disklabel" found here (as of this writing).

What I suggest is that you alter the default/auto partition recommendations of sysinstall by, at least, eliminating the "/tmp" partition, doubling (at least) the "/var" partition, and then sym-linking "/tmp" to "/var/tmp".

To do this is easy enough. During the install, when I get to the label editor screen I first press the 'A' to see the auto-defaults. Make a quick note of the order and sizes if you need it (I do) and then arrow down and delete all of the proposed partitions. (Remember they haven't actually been created yet.) Then re-add them in the same order, entering your new sizes (and skipping "/tmp"). I usually make "/" a little bigger, too. I would add "/" at 256M, swap at 1xRAM (assuming you've got plenty of RAM, 2xRAM otherwise), "/var" at 1G (for a workstation, more for a server), and the rest to "/usr". Then (important!), remember to create the sym-link as soon as you've rebooted. As root, these commands should do it:

cd /
mv tmp/.* var/tmp/
mv tmp/* var/tmp/
rmdir tmp
ln -s /var/tmp

That's it. You're done!

I first got the idea to modify the default partitions from Greg Lehey's book, "The Complete FreeBSD." (A good book and freely available on-line now. See here.) The crucial point he made is that having many partitions was largely a defense against unreliable filesystems. But, filesystems have gotten much better in recent years. Given that, you shouldn't break up your disk into so many pieces without good reasons -- it creates unnecessary bottlenecks ("warning: filesystem full"). His specific recommendation was pretty radical, though: a single, large "/" partition of 4-6 GB, and the remainder of the space in a "/home" partition. (He was assuming a 20GB disk in his example.) I see the advantages of this, especially if you do a lot of work in your home directory that will use space. But, this recommendation has not stood the test of time, as my "/usr" partition is very often much larger than 6GB (to say nothing of when I must extract and compile a large port). Granted, disks (and software) are much larger than when Greg first wrote that, but it still fails to allow for the considerable wildcard of how much third-party software you may eventually end up installing. Also, if you let "/usr" and "/var" ride on "/" you've lost the advantages of the soft-updates for that IO ... or you've enabled soft-updates on your "/" partition and I don't like either of those options.

This article has become longer than I expected. I will follow-up on it soon with my thoughts on why and when you should throw in an extra partition.

Beat the rush! cvsup before the release

If you're planning to do a source based upgrade of FreeBSD after the next release, here's an idea: beat the rush and cvsup to the new release branch before the release date. If you look at the 6.2 release schedule here, you'll see that the new branch for this release was created on November the 15th. It has a cvs tag of 'RELENG_6_2'. This will be the same tag that you might use later to keep up with security fixes via source. But, it's already been branched and most of the coming changes are already there. If you cvsup it now you can get the great majority of the changes into your system source tree now and then after the release, when the servers are suddenly very busy, you'll only be getting a few more small changes to be fully up to date. This makes life better for everyone.

There is a whole other question about whether one should update the system via source/cvsup or binary upgrades. That's a tough question and depends on your own priorities. Bandwidth is a consideration for me. To do a binary upgrade to the next major version would require downloading the CD image, and then only after the release. With cvsup, you're using a lot less total bandwidth because you're only downloading what has changed (and much of that you can get a little early). The freebsd-update tool (in ports, under security) is great for binary patches, but only within the same release branch.

There's one caveat I can think of. It's possible that an important security patch for 6.1 might be released between the time you update your source to 6.2-RCx and the time you get that last update and start rebuilding 6.2-RELEASE. This would be an annoyance, but no cause to panic. You might be able to use the freebsd-update tool to apply a binary update for the running 6.1. You could choose to cvsup 'RELENG_6_2' again and then rebuild to the 6.2-RCx (it should get equivalent patches at the same time, if they're even applicable). You could just cvsup your system source back to 'RELENG_6_1' and rebuild that (or effected portions thereof). Lots of ways to skin that cat. In any case, I think it's a fairly unlikely scenario if you make sure to make the switch as close to the release date as possible. The 6.2 branch is already at RC2 as I write. From the schedule, the 6.2 release announcement is due on January the 10th, but the cvs servers may start getting busy a day or two early (as they should already be showing the release version).

Don't forget to keep the "Rebuilding World" chapter handy.