While fidgeting may be a major part of one’s resting energy use, thinking uses up energy, too. The neurons in the brain aren’t very good at holding their own energy for long, so glial cells in the brain constantly transfer energy to the brain cells, energy taken from the blood. So if you keep your body steady but try solving a differential equation, you’ll probably find yourself losing more energy than while sitting and doing nothing at all.

Of-course, TV excites neurons in the primary visual areas, but perhaps that’s, on average, as far as it goes ;). We may lose less calories watching TV because we're not using our brains to think, only to watch...