display | more...

Everyone with at least a Primary School education should recognize that the answers you get depend very heavily on the questions you ask. This remains true even if the questions are asked for the purposes of a poll to determine what most people think about an issue of the day. Polls cannot divine "what the people are thinking" - but they can get a bunch of answers to the pollster's actual questions.

Unfortunately, like many truths we all should know, this one generally gets a complete bludgeoning during your modern war, as it does at other times when "majority" and "opinion" become slippery, changing things.

The liberal/right-wing media (sorry, having trouble deciding these days) are mostly to blame, of course. To use an example from the time of this writing:

The question actually being asked an awful lot at the height of the War on Iraq was something like "Do you support the troops?", but since that couldn't be neatly conjoined to the daily soundbite of Blush (my pet name for Tony'n'George) talking about the certainty of uncertainty, it was changed in the graphic to "Do you support the war?". Which, as we all know, is pretty much exactly the same anyway.

There was a classic example of this on the screen here in Australia, with a twist. The newsreader introduced the item as "A majority now support the war". But when the graphic actually came up, with the numbers clearly within the margin of error (54% for, 46% against, plus or minus 5%) the smallprint question at the bottom of the screen was as follows: "Now that troops are committed in the Gulf should they enjoy the full support of the public?", or something very similar. Pure "survey-speak".

The other main problem is sometimes the message that the person being polled wants to send is completely different from the questions they are being asked, but they are forced into a simplistic Yes or No. In a case like this the question may actually be "Do you support the war?" The person being polled really wants to say "I have reservations about the war but I support the troops." But in a Yes or No situation, their "reservations" lose out to a simple "Yes" answer,

Another good example of this happened in the New South Wales state election here in Australia, concurrent with the War on Iraq. In the days leading up to polling day, people were ringing talkback radio in droves wanting to know if they wrote "No War" on their ballot papers, but otherwise filled it out correctly, would their votes count? Of course, under Australian law, as long as their voting intention is clear, they can write "Votecounters suck!" and they still have a valid vote. It was pointed out, several times, that the only people who would see this scrawl were the votecounters, and the number of people who did it would never be counted or known, but that didn't seem to matter. People just wanted to send a message to someone, anyone, anyhow.

As they often did with political footballs, the writers of Yes Minister neatly gamed this out years ago in a brilliantly funny episode called The Ministerial Broadcast (the second episode of the Yes Prime Minister series of Yes Minister - it's a British show, ok?). For those not familiar with the show, it's essentially a too-true behind the scenes look at how our political masters really run things, set in the hallowed halls of Westminster and Whitehall. In this scene, Sir Humphrey (the old master of realpolitik) is showing young Bernard Woolley (the novice), exactly how these polls work.

Sir Humphrey: "You know what happens: nice young lady comes up to you. Obviously you want to create a good impression, you don't want to look a fool, do you? So she starts asking you some questions: Mr. Woolley, are you worried about the number of young people without jobs?"
Bernard Woolley: "Yes"
Sir Humphrey: "Are you worried about the rise in crime among teenagers?"
Bernard Woolley: "Yes"
Sir Humphrey: "Do you think there is a lack of discipline in our Comprehensive schools?"
Bernard Woolley: "Yes"
Sir Humphrey: "Do you think young people welcome some authority and leadership in their lives?"
Bernard Woolley: "Yes"
Sir Humphrey: "Do you think they respond to a challenge?"
Bernard Woolley: "Yes"
Sir Humphrey: "Would you be in favour of reintroducing National Service?"
Bernard Woolley: "Oh...well, I suppose I might be."
Sir Humphrey: "Yes or no?"
Bernard Woolley: "Yes"
Sir Humphrey: "Of course you would, Bernard. After all you told you can't say no to that. So they don't mention the first five questions and they publish the last one."
Bernard Woolley: "Is that really what they do?"
Sir Humphrey: "Well, not the reputable ones no, but there aren't many of those. So alternatively the young lady can get the opposite result."
Bernard Woolley: "How?"
Sir Humphrey: "Mr. Woolley, are you worried about the danger of war?"
Bernard Woolley: "Yes"
Sir Humphrey: "Are you worried about the growth of armaments?"
Bernard Woolley: "Yes"
Sir Humphrey: "Do you think there is a danger in giving young people guns and teaching them how to kill?"
Bernard Woolley: (alarmed) "Yes!"
Sir Humphrey: "Do you think it is wrong to force people to take up arms against their will?"
Bernard Woolley: "Yes"
Sir Humphrey: (smoothly, with no change in tone) "Would you oppose the reintroduction of National Service?"
Bernard Woolley: (without even pausing) "Yes"
Sir Humphrey: "There you are, you see Bernard. The perfect balanced sample."

So next time you hear the phrase "The majority of X support Y" you can, if you disagree with the view the majority is allegedly expressing, remember this node. Or alternatively, if you agree with the view the majority is clearly expressing, you can feel lovely and smug.


Thanks to http://www.yes-minister.com/ for the excerpt. Yes Prime Minister was created by Antony Jay and Jonathan Lynn. The episode quoted above was written by Jonathan Lynn. This writeup is CST Approved.

Log in or register to write something here or to contact authors.