SlideLegend

Slot R-04

Margin of Error Explained Simply: Confidence Intervals in Polling

Margin of error explained simply, with real numbers: why ±3.1 points needs 1,000 people, why a poll's lead needs double that margin, and what it cannot see.

Narrated lesson · R-04

Listen to it 2:22

Plus or minus three point one points.

Read the transcript

Plus or minus three point one points. That is the margin of error a poll of a thousand carries, and it is one of the most printed, and most misread, numbers in any survey-based report. Here is what it actually promises. Imagine the true share of something, say, support for a candidate, sits at exactly fifty two percent, a number no single poll can ever see directly. Draw a random sample of a thousand and ask them. That sample lands close to fifty two, almost never exactly on it, somewhere in a range the margin describes in advance. Run the same poll a hundred times, each with a thousand new people, and roughly ninety five of those hundred polls land within three point one points of the truth. That is the confidence interval a ninety five percent margin promises: not that one poll is correct, but that ninety five in a hundred would be. Sample size decides how wide that band has to be. A poll of a thousand carries a margin near three point one points. Cut the sample to four hundred and the margin grows to four point nine. Cut it further, to a hundred, and it nearly triples to nine point eight. Precision gets expensive fast, since the margin shrinks only with the square root of the sample size, not the size itself. A margin most people misapply sits inside close races. A poll showing fifty one to forty nine carries a margin of three point one on each candidate's own number, but the gap between them needs roughly double that, about six point two, since the gap is the difference of two numbers moving in opposite directions. A two point lead inside that margin is not really a lead; in a hundred re-runs of a race genuinely at fifty one to forty nine, the trailing candidate still leads or ties in close to a third. And the margin only ever measures one kind of error: the luck of the random draw. It says nothing about who chose to answer. An opt-in online poll where one side is even slightly more likely to respond can cluster eight or nine points from the truth every time, while still printing the same three point one margin, because that number never accounted for who showed up. A seeded, interactive simulation walks through this step by step, and AAPOR has its own statement distinguishing a real margin of sampling error from the credibility interval an opt-in panel reports instead.

Plus or minus three point one points. That is the margin of error a poll of a thousand people typically carries, printed at the bottom of nearly every election survey, and it is one of the most reported and most misread numbers in any report built on a poll. Margin of error explained simply comes down to one question: how far can a single sample drift from the truth, purely by the luck of who got asked.

Margin of error explained: what the number is actually promising

Somewhere out there sits a true value the poll is trying to measure, a share of an electorate, an opinion held by some fraction of a population, a number no single poll can ever observe directly. A pollster draws a random sample instead, asks the same question of everyone in it, and reports a share. That one sample almost never lands exactly on the true value. It lands somewhere close, and the margin of error describes, in advance, how close “close” tends to be.

Run the same poll a hundred times, each time with a thousand new randomly chosen people, and roughly ninety-five of those hundred samples will land within about three point one points of the true value. That is the confidence interval a ninety-five percent margin actually promises: not that any single reported number is correct, but that roughly ninety-five in a hundred such intervals, built the same way, would contain the truth. It is a statement about the reliability of the method across many repeats, not a guarantee attached to the one poll sitting in front of a reader.

Confidence interval, in plain terms

A confidence interval is simply the poll’s reported share with the margin added and subtracted on either side, paired with the confidence level, almost always ninety-five percent, used to calculate it. A poll reporting fifty-two percent with a margin of three point one points is reporting an interval running from roughly forty-nine to fifty-five, and the ninety-five percent confidence level describes how often intervals built this exact way would actually capture the true value, across an imagined series of repeated polls, rather than describing how likely this specific interval is to be right.

Sample size vs precision: why smaller polls need bigger margins

Sample size decides how wide the margin has to be, and the relationship is unforgiving. A poll of one thousand people carries a margin of about three point one points, calculated from the worst-case fifty-fifty split pollsters generally publish, since that split produces the widest possible margin for any given sample size. Cut the sample to four hundred people and the margin grows to about four point nine points. Cut it further, to a hundred people, and the margin nearly triples from the thousand-person figure, reaching about nine point eight points. Precision is expensive because the margin shrinks only with the square root of the sample size rather than the sample size itself: cutting a sample to a quarter of its size roughly doubles the margin, not quadruples it, but the cost still adds up fast at the smaller end.

That square-root relationship is worth sitting with for a moment, because it explains a pattern that otherwise looks strange: pollsters rarely survey more than about a thousand or fifteen hundred people, even for a national poll covering hundreds of millions. Doubling a sample from a thousand to two thousand only shrinks the margin from about three point one points to roughly two point two, a real but modest gain that usually is not worth the doubled cost of fielding the survey. Below a few hundred respondents, though, every additional person bought matters far more, which is exactly why a poll of a hundred people, the kind a local newsroom or a small campaign might actually be able to afford, carries a margin wide enough to make most close results essentially unreadable.

Margin of error explained for a lead, not just a single share

Where this number gets misapplied most often is inside a close race. A poll reporting fifty-one percent to forty-nine carries a margin of about three point one points on each candidate’s own share, and it is tempting to read a two-point lead as real given that margin. But the lead is the difference between two numbers, each with its own sampling error moving in opposite directions, so the margin on that gap runs roughly double the margin on either single share, about six point two points rather than three point one. Re-run a poll like that a hundred times on a race that genuinely sits at fifty-one to forty-nine, and the candidate shown trailing still leads or ties in close to a third of those hundred re-polls. A two-point lead inside a six-point margin is not really a lead in any meaningful sense, even though headlines built on it rarely say so.

What a margin of error cannot see

The margin only ever measures one kind of error: the luck of a random draw. It has nothing to say about who actually chose to answer. An opt-in online panel where one side’s supporters are even modestly more likely to participate can produce a result that clusters several points away from the truth in the same direction every single time, while the printed margin stays exactly the same, since that number was never built to account for who showed up to answer in the first place. That distinction matters enough that the American Association for Public Opinion Research maintains a formal statement on it: a genuine margin of sampling error applies to a real random probability sample, and an opt-in panel, not drawn that way, reports a model-based credibility interval instead, a related idea built on different assumptions rather than a stricter version of the same thing.

An interactive margin of error explainer walks through this entire progression step by step, from one sample to a hundred repeated samples to the effect of an opt-in panel, using a seeded simulation so the same numbers appear on every visit. Pew Research Center has its own accessible explanation covering much of the same ground, including the doubled margin on a lead between two candidates. None of it changes the underlying lesson: a margin of error is a real, useful, precisely calculated number, and it answers a much narrower question than most headlines built on top of it assume it does.

Reading a poll’s margin of error is really one instance of a broader habit covered in how to read a report: checking what a number can and cannot support before treating it as settled. A margin of error can tell a reader exactly how much random scatter to expect from a well-run random sample. It cannot tell a reader whether the sample was random to begin with, whether the question was worded fairly, or whether the people who declined to answer differ in some systematic way from the people who did. Those questions sit in the methodology section of whatever report the poll appears in, not in the plus-or-minus figure printed underneath it, and a reader who checks only the margin has checked the easiest part of the number, not the most important one.

Explainer

Questions

Margin of error explained simply: what does ±3 points actually mean?

It means that if the same poll were run many times on the same population, roughly 95 out of 100 of those polls would land within 3 points of the true value. It is a statement about how repeatable the result is, not a guarantee that any one poll is exactly correct.

What is a confidence interval in polling?

A confidence interval is the range a poll's result plus its margin of error covers, paired with a confidence level, usually 95%, stating how often intervals built this way would actually contain the true value across many repeated samples of the same size.

Why does a smaller poll have a bigger margin of error?

Margin of error shrinks with the square root of the sample size, not the sample size itself, so cutting a sample to a quarter of its size roughly doubles the margin. A poll of 1,000 people typically carries about ±3.1 points; a poll of 100 carries closer to ±9.8.

Why does a poll's lead need a bigger margin than either candidate's own number?

A lead is the difference between two shares, each carrying its own sampling error moving in opposite directions, so the margin on that difference is roughly double the margin on either individual share. A poll with a ±3.1 margin on each candidate needs about ±6.2 on the gap between them.

Can a poll's margin of error catch bias from who answered?

No. The margin only accounts for random sampling luck, the natural scatter from drawing a different sample each time. It says nothing about whether one group was more likely to respond than another, which is a separate problem entirely, sometimes large enough to move a result by several points in one direction every time.

What is the difference between a margin of sampling error and a credibility interval?

A margin of sampling error applies to a genuine random probability sample, per AAPOR's own definition. An opt-in online panel, which is not drawn randomly, cannot claim a true margin of sampling error, and instead reports a model-based credibility interval, a related but distinct measure with its own assumptions.

Slide check · 5 questions

Check yourself

Question 01 of 05

What does a ±3.1-point margin of error on a poll of 1,000 people actually promise?

Show the answer

B · That about 95 out of 100 such polls would land within 3.1 points of the true valueA 95% confidence level means roughly 95 of 100 repeated samples would produce a margin that captures the true value, not that any single poll is exactly right.

Question 02 of 05

How does margin of error change as sample size shrinks from 1,000 to 100 people?

Show the answer

B · It grows from about ±3.1 points to about ±9.8 pointsBecause the margin shrinks only with the square root of sample size, cutting a sample of 1,000 down to 100 roughly triples the margin, from about ±3.1 to about ±9.8 points.

Question 03 of 05

Why does a poll's lead (the gap between two candidates) need a wider margin than each candidate's individual share?

Show the answer

B · The lead is the difference of two moving numbers, so its margin is roughly double the margin on either share aloneSince both candidates' shares carry their own sampling error moving in opposite directions, the margin on the gap between them is roughly twice the margin on either individual share.

Question 04 of 05

What can a printed margin of error NOT detect?

Show the answer

B · Bias from who chose to respond, such as an opt-in panel skewed toward one sideThe margin of error measures only random sampling luck. An opt-in panel where one side is more likely to respond can be biased by many points while still printing an unchanged margin.

Question 05 of 05

According to AAPOR, what is different about the interval an opt-in online poll reports?

Show the answer

B · It is typically a model-based credibility interval, not a true margin of sampling error, since opt-in panels aren't random probability samplesAAPOR ties a genuine margin of sampling error to probability-based samples; opt-in online panels report a model-based credibility interval instead, a related but distinct concept.