27 August 2006

A short ports primer

This short article is mainly to support the article "Top ten rules for using ports system," which contains some terminology that may be confusing to someone who isn't already familiar with the FreeBSD ports system.

What's a port? In FreeBSD, a port usually refers to third party software that may be installed via the ports system.

What's the ports tree? It's a large directory tree usually located at "/usr/ports". It contains many category subdirectories, each of which contains many port subdirectories. Each port subdirectory contains a Makefile, a text file with a short description of the port ("pkg-descr"), maybe some FreeBSD specific patches, and a few other items useful for installing or maintaining the port.

What's a package? A package is just a precompiled binary version of some port. These are widely available for each new release. They are not widely available for updated versions of the software that may come out between FreeBSD releases (for that use the port). Once installed, a package is no different from the same version port that was compiled with the same options. Either would be recorded in the system's package database the same way. Therefore, statements about one often apply to the other.

How do I install a port? The typical way to install a port is to go to that ports directory in the tree and then type "make install" as the root user. For example, to install the firefox web browser, you would go to the directory "/usr/ports/www/firefox". The ports system will automatically identify and attempt to install any other required software and then install firefox. When using this method the ports system will take several steps for each port, the most notable of which are: download the source code, compile software, copy the new binaries to the appropriate locations, and record details of the installation in the system's package database.

How do I install a package? Download the package then, as user root, type "pkg_add filename". Alternatively, you can have pkg_add attempt to download the file for you first by typing "pkg_add -r portname" (as user root).

How do I delete a port or package? By using the "pkg_delete fullportname" command (as root). In this case, it's not enough to just know the name of the port or its directory in the tree, you want to use the full name of the installed port, which will include version numbers at the end. For example, "firefox-,1". A good way to find out this full name is with the "pkg_info" command which will give you a list of all installed ports/packages. (You could also use a wildcard, e.g. "pkg_delete firefox-*", but be careful!)

How do I update the ports tree? There are two main tools used to update the ports tree: cvsup and portsnap. cvsup is itself a port. portsnap is the newer tool and is included in the base system in recent versions of FreeBSD. Type "man portsnap" to view the manual. ... Also, read the "Top ten rules ..." for some thoughts on when you should update your ports tree.

[ tags: , ]


25 August 2006

Top ten rules for using ports system

(This article assumes a basic knowledge of the FreeBSD ports system. You may want to read this article first if you're unfamiliar with the ports system.)

10. Decide up front whether you really need or want to update your ports tree every day. Or every week. Or every month. Unless you're learning/playing/experimenting I recommend that you do not schedule updates to the ports tree, but only update it when you know that there is something newer that you want in the newer tree. This saves a huge amount of time in possibly updating ports that are working just fine. And, it will avoid the (mercifully rare) broken, out of date dependencies.

9. When it is time to update your ports tree use portsnap. This utility is more efficent than cvsup for small, frequent updates, and more secure in any case. It does take up a little disk space for its compressed snapshots, though. It's a part of the base system starting at version 6.0 and above (and also in 5.5).

8. Preview which ports could be updated. The best way to do this is probably with "pkg_version -v", which will output a list of all your installed ports along with whether they are up-to-date with what's in your updated ports tree and, if not, what the newer version number (in the tree) is. Here's a snip of example output:

alienblaster-1.1.0_1                =   up-to-date with port
allegro-4.2.0 < needs updating (port has 4.2.1)
apache+mod_ssl-1.3.37+2.8.28 = up-to-date with port
aspell-0.60.4_4 < needs updating (port has 0.60.5)

Another way to do something similar is with the command "portupgrade -n -a" ("-n" is dry run mode). But, the output is not as clean as that of pkg_version.

7. When it is time to update the ports themselves use portupgrade (sysutils/portupgrade). portupgrade is a great utility that will allow you to update any one port without necessarily updating its dependencies or the ports that depend on it. It then updates all the dependencies in the package database. Or, with the '-R' option you can upgrade a port and all of its dependencies as well. Of course, you can also use it to just ...

6. When it is time to update the ports themselves, just go ahead and update all of them. There is a little bit of work associated with keeping your ports updated. The most efficient thing is to just wait until there's a new port or new version of a port that you want and then update everything that you have installed at once. Or, just set aside some time and update all of your installed ports every month or two.

(When doing a mass update with portupgrade, you should seriously consider specifying "hold" packages in "pkgtools.conf" (mine is located in /usr/local/etc/). Java, for example, is a port that takes a long time to compile and requires manual downloading of the source and I have a hold on it. (News on Java.) The same for the nvidia drivers, which are sensitive and get special attention (such as a back-up, and a reboot between de-install and new install).)

There is a recurring theme here about when to update your tree and when to update your ports. When your installed ports are not in sync with your ports tree it can cause problems. These problems are rare and usually correctable, but they can be vexing. For example, portupgrade may try to update an installed port's dependency that has changed or changed location or been removed. Then you'll need to sort it out and run a 'pkgdb -F' to get things happy again. If this sounds imposing, it's really not so bad. I can't stress enough that these cases are very rare. In the worst case, you could just uninstall problem ports and start clean. But, first read on.

5. For critical ports, back them up before upgrading. A simple "pkg_create -b portname" will create a compressed, binary package for any installed port. This is important because once you've updated your tree, you may be unable to roll back to the earlier version in the event that the new version does not behave as expected. portupgrade also has an option to do this automatically. That's actually a pretty good idea to use that backup option for all ports. But, such is my faith in the ports system that I choose to do it myself and only for important and/or difficult ports.

4. Consider the really lazy method of never updating your tree and using all packages! Wow. It's like heresy. But, it can work. If you get no thrill from watching a port compile and feel no particular need to have the latest and greatest software then -- don't update that ports tree! Many newcomers to FreeBSD just start updating their tree and their ports because, well, that's what you do. But, if you leave your tree as it was on the install then it will always match the release packages and you can install software quickly with "pkg_add -r pkg-name". You don't even need the version number/full package name! Just the basename, like "firefox" or "python". Then, when you upgrade the system itself, you can grab the new release ports tarball, replace your ports tree in toto and start again. You can always use a "pkg_add -r", even if you've updated your tree. But, if you're updating your tree regularly it gets more and more out of sync with the various versions of the release packages and has the potential to cause problems. There are, of course, many ports that are not available as a package, so you'll still get in the occasional "make install clean".

I don't use this "lazy" system myself. If you are installing a new FreeBSD system you need to consider that you're starting from scratch anyways and, even shortly after the release, there are normally many updates in the most current ports tree that aren't in the release ports tree. So, my preference is to install practically no packages during the install, then update the ports tree, then start adding ports. (And, yes, I do get a thrill from watching ports compile.) I do usually install X packages during the install (they take so long to compile). But, do not try to configure X during the system install!

3. Read /usr/ports/UPDATING. This is where all the gotchas are supposed to be documented. How many times have I had a problem and then gone back and read UPDATING to find the solution? (Too many times.) But, that's the point of this article. I've made the mistakes so you don't have to.

2. Go under the hood and have fun. That's what great about ports. You have a lot of control. Some of the basic "make" targets that will allow you to check out a port before installing are "fetch-list", "fetch-recursive-list", "config", "depends-list", "build-depends-list", and "run-depends-list". These and many others are documented in the ports manual ("man ports"). (But, not the last three "*depends-list" targets for some reason.) Running a "make extract" and then looking over the documentation included with source itself is often very useful (found in the "work" subdirectory after extraction). If you're going to do a mass portupgrade and then go to lunch, you may be irritated to come back and find it stopped on a config options dialog. But, by running a "make config" on everything first you can get your options chosen and saved and then take a nice long lunch. "make config" is often a good idea anyway because if this is an upgrade and the available options have not changed, your prior install's options will be used without asking. But, you may have learned more and want to change them after all. You can also check out the "Makefile" in a port's tree directory to see what options are available.

1. Use the FreshPorts site (link) to see what's new without changing your local tree. I've said several times that I don't recommend random updating of the ports tree. It can be a lot of fun when you're learning. But, it ends up being make-work to update ports all the time. The FreshPorts site is a convenient way to browse the latest in the ports tree, see comments on what's changed, see vulnerabilities, even sign up for watch lists. This post describes a browser search plugin for FreshPorts that I find very useful.

[ tags: , ]