Common descent is, IMHO, the only 'real' problem most people really have with the theory of evolution.
Basically put, evolutionary theory as commonly taught has a few basic tenets:
First, Survival of the Fittest - not too controversial, except for semantic quibbles on the meaning of 'fit' - which basically says that things that have an advantage of some kind survive better than those that don't. This is actually kind of self-evident.
Second, that this process causes the different kinds of these "fittest" to differentiate into separate species - mildly controversial, though I understand it's been documented in laboratories. (At least, that's what I glean from talk.origins.)
Third, Common Descent, as a sort of logical walking backwards, says that all species now extant must therefore be descended from one common ancestor. Most controversial, because it claims humans to have descended from an ape-like creature, which many find repellent; and because it reintroduces a need for the long-discredited theory of abiogenesis aka spontaneous generation, which many find untidy; and most importantly because it conflicts with many creation stories, which causes many to find it heretical. Has some backing in the fossil record, but any creation story worth its salt can explain that.
I think that people who have been given "evolution" as a whole take-it-or-leave-it bundle and left it, have done so because of common descent. It's also been my experience that people who have trouble analyzing evolution into its parts try to abstractly disprove the other two bits as well, the results of which usually aren't pretty.