An ancient horror left to haunt us still.
Back in the early days of MS-DOS, people had one disk drive. Luckiest had two. Later, these "winchester disk" things got popular.
MS-DOS originally identified drives with letter and colon (:) - This was the CP/M behavior. I believe this was originally a feature of QDOS and it stuck. The primary floppy drive was called A: and the second floppy drive B:. You could refer to some file as A:FILE.EXT, or just FILE.EXT if A: was your active drive.
DOS 2.0 introduced subdirectories - so you could refer to files with A:\DRCTRY\FILE.EXT, or A:FILE.EXT (in case \DRCTRY was the current working directory of a:), or FILE.EXT (if you were in A:\DRCTRY).
Then came the hard drives - hard disk partitions were named C:, D:, and so on.
In Windows 9x series, like in DOS since the beginning of the time, the drive letters are always ordered so that C: is the first partition in the first IDE drive, then come other partitions on that drive, then on secondary IDE drive, and so on.
With the advent of MSCDEX, CD-ROM drives also appeared as one drive letter.
MS has refused to move away from this scheme, for historical reasons.
The problem with this was, of course, simple: If you moved a drive from one place on IDE bus to another, or changed partitioning of some drive by adding or removing a FAT partition, all "following" disk drive letters changed place.
This is a particularly big headache if the drive letter of some partition you have installed stuff to/from changes (a good example would be a CD-ROM drive location).
Personally, I added a new hard disk and put a new FAT32 partition to it. As a result, my CD-ROM drive moved from D: to E:. My next project is to reinstall every game I have, because all of them seem to store installation CD-ROM drive to some place not visible to us mere mortals. (Is it really that hard to ask the OS where our CD-ROM drive is, and if there's a specific CD-ROM in the drive?) I have also been told some apps, even some of those from Microsoft themselves, will not even uninstall if drive letter ordering changes...
Of course, in Windows NT series, it is possible to rearrange disk letters, but we "dumb consumers" who use 9x aren't expected to add new hard disks or something...
(UNIXes, in comparision, have a single root directory under which all partitions can be mounted to any place - and drives are always referred to by their physical names.)