PCI, or Peripheral Component Interconnect, is a currently the most common system bus used in personal computers and low-end servers. It connects the primary CPU and memory subsystems to various peripherals which provide all kinds of useful services. It was designed in the early 1990s at Intel to replace the ISA bus, with its paltry bandwidth of 9 megabytes / second. For a long time ISA and PCI coexisted, with PCI being used for new or higher speed devices like graphics cards, with ISA being slowly phased out. As late as 2000 people such as myself were using ISA modems (in my particular case, to avoid having to deal with Winmodems). A particularly interesting use of PCI is the SunPCi card, which lets you run Windows on your SPARC-based workstation.
The common variant of PCI available on most systems is a 32-bit bus running at 33.33 MHz, providing a theoretical bandwidth of 133 MB/s. However, due to bus overhead, it is essentially impossible to actually reach this in practice. At the time of it's introduction this performance was quite good, especially in comparison to ISA and EISA, but PCI has a hard time keeping up today with high-end devices, such as gigabit Ethernet and larger RAID setups. In recognition of this, PCI has extensions providing a 64-bit bus and a 66.66 MHz clock speed, giving up to 533 MB/s. Both of these extensions are backwards compatible; a 66 MHz card is visually indistinguishable from a 33 MHz card, and will downclock itself if plugged into a 33 MHz bus. The 64-bit variants use more pins (184, instead of the 124 used by 32-bit cards), but is also backwards compatible with 32-bit PCI. If plugged into a 32-bit slot, some of the pins are unused, and it pretends to be a 32-bit card. Similarly, you can plug a 32-bit card into a 64-bit slot without any trouble, though this can adversely affect overall performance, as the bus is usually shared among all the cards in the system, and adding a single 32-bit card might well force all the other cards to operate in 32-bit mode.
In addition to the obvious PCI slots on a motherboard, PCI is often used to access the onboard features of the board. These depend on the specific configuration, but often include IDE, SCSI, and USB controllers, sound, and Ethernet. This is mostly because it is very cheap, and it's easy to basically just take a chip that was formerly on a PCI board and stick it directly onto the motherboard. Some newer and much faster interconnects, like AMD's Hypertransport, are beginning to make inroads in this area, but PCI is still very common, especially on low to medium range systems.
While the PCI bus originated on Intel x86 class machines, it has become widely used on a variety of systems, including modern Apple systems, low-end Sun workstations like the Ultra 5, and the gone but not forgotten Alpha. In the form of Cardbus and Mini-PCI it has also be made suitable for laptops and other systems that would not accept a normally sized PCI card. These are identical to the normal PCI you find in a PC, just with a different form factor. AGP is essentially a variation of PCI; the first version was basically 66 MHz PCI with some extra bus commands, though later versions have dramatically increased the clock rate (currently up to 533 MHz) and a better transfer/clock ratio; the current version of AGP, with about 2 Gb/s of bandwidth, blows PCI out of the water, but is basically dead thanks to PCI-E.
PCI has been extended in a mildly incompatible way with PCI-X, and will eventually be phased out in favor of PCI-E, but that is not expected to complete for at least 5 more years, due to the huge installed userbase of PCI.