Jail. The penitentiary. The lock-up. The hoosegow. Up the river. A place to be avoided, if possible.

This probably evolved from criminal slang, and especially, prisoner slang. When you spend a lot of time in prison, you have two houses: the little house with wives and kids and freedom, where you'd prefer to be, and the big house, where the food sucks, you can't see the sky, and everyone seems to want to stick a shiv in your neck. Gimme the little house any day of the week...