tmpfs is Sun's name for a very curious filesystem. Essentially, it's a RAM disk. But since it lives on a UN*X system (Solaris, to be precise), it doesn't need to run out of space: it uses the system's virtual memory. As the name hints, tmpfs is intended to be used to mount a filesystem for temporary files. Typically, the filesystem is mounted on /tmp.
Many programs require temporary files. Compilers are a particularly notorious example. A compiler will often run numerous phases; the phases communicate by temporary files. (Seeks are necessary on these files, which is why pipes cannot be used.) Temporary files need to show quick access times: almost by definition, they contain nothing of interest to the user. Mounting /tmp on a local disk (as opposed to accessing it by NFS) is a good idea; Sun claims that keeping your data in RAM and using tmpfs is an even better idea.
So tmpfs is a filesystem which uses RAM instead of disk. Nothing new here -- that's just a RAM disk. Which is why it should be fast: data transfer from physical memory is always faster. But to use a RAM disk, you must set aside a portion of your physical memory (think of it as a partition of your RAM). Do that, and you have less RAM, and everything slows down again. But the "not enough RAM for all processes" argument is exactly what leads to virtual memory. Why not store your "RAM" disk in virtual memory?
So the tmpfs filesystem shows filesystem semantics to programs and users, but uses plain ordinary UN*X virtual memory to store data. Of course, the OS doesn't need to page this filesystem -- the only way to access it is to go through the ordinary paging channels. So this interface between two portions of the OS is eliminated. Better yet, the filesystem only gets full when you run out of virtual memory. (Of course, things slow down when you run out of physical memory. But that cannot be helped: If your /tmp lived on disk and you ran out of physical memory, you'd still have to page out parts of /tmp. The difference is that while you have enough physical memory, tmpfs gets better treatment than a real disk.
An important point is that tmpfs is very temporary: after rebooting, it's wiped clean. In a mixed environment with other Unices, this can surprise users. At least they spend their whining time on their precious temporary files, not on their compilation times.
Still, there are problems. If you fill up all your computer's swap space, you'll be in big trouble (much worse than just filling up your /tmp). An option exists to specify the maximum size of the filesystem; you'd be advised to use it. (Also, in Solaris tmpfs, the number of files you can create is determined only by physical memory. This limitation might be difficult to overcome if needed.)
As of 2.4.x versions, Linux too has tmpfs. Other UN*Xoids might have one, too.
tmpfs is simultaneously a beautiful extension of the RAM disk concept and a hideous kluge stemming from the ridiculous reliance on files in modern software engineering practices.