Margins of Error

What does it really mean when the news anchor says: "The latest polls show Johnson with 51 percent of the vote and Smith with 49 percent, with a 3 percent margin of error"? If there is a 3 percent margin of error, and Johnson leads Smith by only two percentage points, then isn't the poll useless? Isn't it equally possible that Smith is winning by one point?

The margin of error is one of the least understood aspects of political polling. The confusion begins with the name itself. The official name of the margin of error is the margin of sampling error (MOSE). The margin of sampling error is a statistically proven number based on the size of the sample group [source: American Association for Public Opinion Research]. It has nothing to do with the accuracy of the poll itself. The true margin of error of a political poll is impossible to measure, because there are so many different things that could alter the accuracy of a poll: biased questions, poor analysis, simple math mistakes.

Instead, the MOSE is a straightforward equation based solely on the size of the sample group (assuming that the total population is 10,000 or greater) [source: AAPOR]. As a rule, the larger the sample group, the smaller the margin of error. For example, a sample size of 100 respondents has a MOSE of +/- 10 percentage points, which is pretty huge. A sample of 1,000 respondents, however, has a MOSE of +/- 3 percentage points. To achieve a MOSE of +/- 1 percentage point, you need a sample of at least 5,000 respondents [source: AAPOR]. Most political polls aim for 1,000 respondents, because it delivers the most accurate results with the fewest calls.

Let's get back to our tight political race between Johnson and Smith. Does a 2-percent lead mean anything in a poll with a 3 percent margin of sampling error? Not really. In fact, it's worse than you think. The margin of error applies to each candidate independently [source: Zukin]. When the poll says that Johnson has 51 percent of the vote, it really means that he has anywhere between 48 and 54 percent of the vote. Likewise, Smith's 49 percent really means that he has between 46 and 52 percent of the vote. So the poll could just as likely have Smith winning 52 to 48.

Next we'll look at one of the most important factors that determine the accuracy of a political poll: the wording of the questions and answers.