There are several vegetables which may naturally feature the color purple: kohlrabi, carrots, peppers, potatoes, yams, asparagus, cabbage, cauliflower, chard, onions, beets, radicchio, and eggplant. The "common" varities of these vegetables are typically not purple, with the exception of eggplant.
The distinctive hue of purple vegetables is due to an exceptionally high concentration of pigment in the plant. For most purple vegetables, anthocyanin is the pigment in question. Anthocyanin acts as an indicator of how little sunlight a plant receives. Beets are unique in that their purple-red coloration is due to the presence of the pigment betacyanin. If a person ingests an excess of betacyanins, their body may exhibit unusual reactions.
Even though the differences in taste between purple vegetables and their conventional counterparts are negligible, purple vegetables make for a more potent medicine than other varities. The medicinal benefits are due to the strong detoxifying effects of the anthocyanin, which acts as an antioxidant. As with any medicinal substance, an excess may be harmful. In the case of purple vegetables such as beets, an excess of betacyanin may overload the liver with toxins purged from other parts of the body. This is mostly a danger for those with a liver or kidney malfunction. A person with a healthy endocrine system need not worry about such toxicity.
Purple vegetables may be found nearly anywhere quality produce may be had. But more often they are not carried at the average grocery and may be had from a health food store or farmers market or an ethnic grocer. You can, of course, always try to grow your own.
Thanks to Oolong for help with this node. Oolong also notes that "anthocyanin is strongly pH-dependent - pretty much any of the *other* purple fruit (aside from beets) and veg will go an orangey red colour if you add lemon juice, for example, or an even bluer purple if you add sodium bicarbonate."