c:\windows
This is where Windows lives. Everything contained within this directory is pretty much what makes up the Windows OS. All system files are stored here, which is why by default it is closed to view when you click on it in windows explorer.
All the .dll's(dynamic linked libraries), or shared object's, go in the subdirectory 'system' or 'system32' under the c:\windows directory. Although both are used to store dll's, Windows 9x/ME systems tend to store the majority of it's dll's in c:\windows\system, whereas Windows NT based systems store them under c:\windows\system32. This is due to the fact that Windows 9x systems, while capable of running 32-bit applications, is really a 16-bit operating system(so the 16-bit libraries go into 'system'), while Windows NT systems are truly 32-bit(as are it's libraries, so they go in 'system32').
On the MS-DOS based Windows 9x systems, the primary executable to start Windows is located in this directory. Vital system executable files are also in this directory.
On WindowsNT based systems c:\windows can also be called c:\winnt, although this is only true of WindowsNT and Windows 2000. Windows XP, although based on Windows NT/2000, uses c:\windows.
This style of layout is very different from Unix-based systems. On *nix systems, the system libraries and executables are distributed on /(the root directory). Executables/binaries go in /bin, libraries in /lib, and so on.(as a Linux newbie I had a hell of a time figuring out what went where)
There are a lot more files and directories under c:\windows, but it's pretty much more of the same.
On a side note, if you've been considering trying an alternative OS, try Linux. RedHat and Mandrake are great newbie distro's.