Filesystem Compression

One of the neat things that Windows has provided for quite some time is the ability to compress filesystems (using LZW encoding) yet still allow them to be usable filesystems. At the expense of some CPU power and usually a small performance hit, DriveSpace could be used to reduce the disk space occupied by data and applications, allowing more to be shoehorned onto the system.

Windows NT's support for filesystem compression is even more flexible, allowing compression attributes to be set on a per-object basis; for example, allowing most of a drive to be compressed, apart from areas such as TEMP directories which will be frequently written to (since writing compressed files is normally very much slower than writing uncompressed files).

UNIX and Filesystem Compression

This sort of functionality is something that's been almost entirely absent from UNIX operating systems to date, since writing to a compressed filesystem doesn't fit terribly well with UNIX filesystem semantics.

Mac OS X's support for virtual filesystems (or 'loopback' file systems in conventional UNIX terminology) includes a format that behaves like HFS+, but in a compressed image file. Apple's Disk Copy utility can be used to create and mount such a compressed image. Sounds ideal, doesn't it?

There is, of course, a catch. These filesystems are read-only. So while it does mean that greater compression can be achieved, and no space is wasted by fragmentation, you can't simply convert your entire drive to a compressed format without sacrificing the ability to, say, save your data.

This doesn't mean to say, of course, that it's useless. Thanks to the nifty way that Mac OS X is structured into 'domains', there's a fairly clear separation between things which need to be read and written (for example, user documents) and things which are generally just read (program files and constant data). This makes it easy to select large chunks of stuff to compress.

What Colin Did

The most obvious chunk of stuff to compress, or so it seemed to me, would be the Applications folder. My Applications folder weighed in at about 1.15Gb, including all the standard OS X applications, VirtualPC, Photoshop, and Office X. To see how much I could potentially save by compressing this, I created a compressed image from the Applications folder using Disk Copy. The resulting image was about 550Mb in size, meaning that I could save myself about 600Mb of disk space if I could use the compressed image. Turns out that, after a fashion, with a few 'ifs' and 'buts' I can. So here's what we'll do.

Create a test image

Some applications will be perfectly happy to run from a read-only filesystem, and some won't. The best way to determine which class an application falls into is to try it out.

  1. Put the kettle on
  2. Start Disk Copy
  3. Select 'File ⇒ New ⇒ Image from Folder or Volume'
  4. Select your Applications folder as the folder to image
  5. Save the image in 'Compressed' format in the root directory of your hard disk.
  6. The kettle might have boiled by now, so make yourself some tea while your computer creates the image file. It's going to take a while.

When the image file is read, open it to mount it on the desktop. Now we're ready to...

Test and segregate applications

As I said above, most applications are perfectly happy to run from a compressed filesystem. Some, on the other hand, aren't. The object of this exercise is to find those that can run from a compressed filesystem and compress them. So what we'll do is separate applications into compressible and uncompressible categories

  1. Create a new folder beside your Applications folder, and call it Applications_compressed, or something similar.
  2. For each of your applications in the compressed image, try running it from the compressed image. Remember to try things like saving preferences and saving game positions, that sort of thing. If the applications seems happy, move the original copy from Applications into Applications_compressed. Applications_compressed will be the folder we create the compressed image from.

Note that you should keep Disk Copy and System Preferences in the Applications directory. You may need Disk Copy to mount the image file!

After trawling through my own applications, the only ones I found that didn't like being compressed were

Compress and replace!

Compress the Applications_compressed folder to an image file, just as we did earlier with Applications. Save the image file as Applications_compressed.dmg in the root directory of your disk.

Now, drag the original Applications_compressed to the wastebasket, take a deep breath, and empty the wastebasket.

You may also want to laugh maniacally for a moment or so at this point.

Mount the compressed Applications_compressed.dmg by double-clicking it. Create aliases in Applications for each application in the compressed folder (Select all the applications and drag to Applications folder while holding down Command and Option/Alt). Do the same thing with the Utilities folder in Applications (the original should still be there because you've left in Disk Copy,.. haven't you?).

And that's pretty much it. If you log out or unmount the compressed image, using one of the aliases in the Applications folder should cause the compressed image to be mounted automatically, and the application started. (Personally, I think this is pretty damn nifty, but then, I'm easily impressed).

Maintenance

When you're installing new applications, these will by default go into Applications, so they won't be compressed. To incorporate new applications into the compressed image, you'll have to recreate the uncompressed applications directory by copying the applications out of the compressed image, before adding the new application to the Applications_compressed folder and compressing that.

Since this needs some additional hard disk space to store the uncompressed applications, an external disk drive or a fileserver with an AFS filesystem is very useful.