A matronym is a name derived from the name of a mother or maternal ancestor. (It is sometimes spelled "metronym.")
The word comes from the Latin root metr- (mother) and the Greek -onym (name, word). Some matronyms are obvious to any speakers of the language ("Yocheved Bat Miriam" to a Hebrew speaker); however, as an SCA page points out: "A matronym may not (indeed, it often does not) take the same form as the given name from which it is derived. Indeed, Lorraine may very well be a matronym derived from the given name Lora or something very similar."
A matronym can be a given name passed on (naming a child after her mother, for example), a nickname (such as Henry II of England being known in his youth as "Henry FitzEmpress"), or a family name. In most cultures, matronyms have been much less likely to become surnames than patronyms, those derived from the father or paternal ancestors. In some cases, they were used only for illegitimate children, where a patronym could not be applied. However, some cultures such as the Lobi of Burkina Faso and Cote d'Ivoire, do use matronyms as surnames in exactly the same way English speakers usually use patronyms: "Each child, whether girl or boy, receives at birth a matronym, the name of the "matri-clan" (tyar) of the mother, certifying kinship with a common female ancestor. The name is passed on from mother to daughter. Each son bears it but does not pass it on to his children. Thus maternal kinship is lost in the male line." (However, this same group passes down sacred secret clan names through the father's line.)
Matronymic family names were not everday tradition but happened sometimes among Eastern European Jews; a page on Jews in what is now Belarus notes that "The Jews of Eastern Europe crowned the Jewish woman, who was a role model. She went out and earned a living to support her family and educate her children while her husband sat in shul and studied Torah. As a result, many times the husband and her children were called by her name and sometimes even her son-in-law." Matrilineal cultures are more likely to use matronyms culture-wide, but many formerly matrilineal societies have adopted more common patrilineal customs of descent and naming.
Other societies use both: Spanish and Portuguese speakers commonly use both a patronym and matronym after the person's given name, though Spanish speakers put the father's family name first and then the mother's (so when John Smith marries Jane Doe, their children will all be given name Smith Doe) and Portuguese speakers put the mother's name first (thus the children of the above pair would be given name Doe Smith). In either case, through, the patronym is considered the family name and is what both sons and daughters would pass on to a third generation. Giving a child the mother's maiden name as a middle name used to be quite common in parts of the United States, and still is in some places, and using one's mother's maiden name as part of a pseudonym is also fairly common. A few American families of Lucy Stone-esque beliefs choose to pass on the mother's surname to some or all of their kids. However, the most common use of the mother's surname for children in the U.S. is when the children's parents never married or have divorced, though at least as many children in these situations have their fathers' surnames.
Sources:
http://www.wordsmith.org/awad/archives/0503
http://www.sandiegohistory.org/journal/73summer/cabrillo.htm
http://lists.village.virginia.edu/~spoons/archive_links/deleuze-guattari.archive/d-g_1994/deleuze_Jun.94
http://www.sca.org/heraldry/loar/1998/05/lar.html
http://ccat.sas.upenn.edu/jwst/WomJewLit.pdf
http://perso.wanadoo.fr/africart/pages/lobetud.htm
http://www.jewishgen.org/Yizkor/lida/lid075.html