Essentially, a file system contained in a single file. Compound files are similar to archive formats such as zip or rar, and may/may not feature compression or other useful features. Every compound file implementation that I've come across has featured directories, though I would assume that these are not always necessary.
Compound files have found much use, especially as a way to keep grubby hands from screwing up sensitive data. Quake's pak files are an example of compound files, and OLE structured storage makes use of them too, to the point where every MS Office document is a compound file.
In actuality, a compound file needn't contain files. Rather, objects or state information (see finite state machine) could also be kept in a compound file, which leads into the topic of serialization.