When people are miscast, they are said to be acting in a play, television series or film in roles that are totally wrong for them. They are seen to be out of their depth, and generally accused of bringing down the quality of the work they are participating in.
An egregious example of someone totally miscast is, in this writer's opinion, Keanu Reeves in the film Much Ado About Nothing.