"Margin of Error", When Used by Pollsters, Is Widely Misunderstood and Confuses Most...

A Large Majority Believes That Calculations of "Margin of Error" Should Include All Sources of Error, Not Just "Sampling Error"

ROCHESTER, N.Y. -- Many media and pollsters, when releasing new poll results, include statements such as "the margin of error for this

survey is +/- 3 percent". A new Harris Poll was developed to measure the public's understanding, or misunderstanding, of the phrase "margin of error" when used to describe opinion polls. It found that these words are misunderstood by most people. Arguably they confuse more people than they enlighten, and they suggest a level of accuracy that no statistician could justify.

These conclusions are based on a Harris Poll of 1,052 U.S. adults surveyed by telephone between October 16 and 23, 2007 by Harris Interactive[R].

This number is actually a purely theoretical calculation of what the likely maximum error (at a 95% confidence level) would be if the survey had used a pure probability sample with a response rate of 100% and there were no other possible sources of error. In the real world of polling there are several other sources of error that may sometimes be larger than this theoretical calculation of sampling error, and there is no good way to calculate them. However, a new Harris Poll shows that most people do not understand this.

There are a number of other possible sources of error in any poll which include:

* Non-response errors - Pollsters often do not complete interviews with most of the people they intend to survey because they are not available or refuse to be interviewed;

* Errors due to question wording or question order. The answers to questions are sometimes influenced by such things as how the questions are posed, what questions were asked earlier in the survey, or which responses are presented to the respondent, among other things;

* Errors due to interviewers. Interviewers sometimes influence, often unconsciously, the answers given by the people they survey (e.g. social desirability, acquiescence bias, researcher expectancy effects, etc.);

* Weighting errors - Most polls are "weighted" statistically to compensate for demographic and other biases in the survey sample; this is an imperfect process. Weighting the data can cause errors in the results.