的 2020 总统选举 polls were far from perfect. They had President-elect Joe Biden winning the election, which is worth something. 然而, nationally and in every single one of the 18 states where the final polling average was within 10 points in either direction, Biden’s standing relative to President Donald Trump was overestimated.
对于一些, this might mean we shouldn’t pay attention to polling.
对我来说, to do so would be a mistake. Polling is still one of the best tools for understanding public opinion. While pollsters likely have some work to do to improve accuracy, those who ask for precision from polling are simply asking too much.
A look at the long arc of history suggests that the polling errors of the last few years should not be too surprising. In some ways, you would be right to call them average.
Let’s start with what I think is the most important error of the 2020 选举: the national polling. 是, elections are ultimately state affairs, but most polls we use are national surveys. They help us understand where Americans stand on electoral issues and well as those outside of this realm.
Different national 轮询 aggregates had Biden ahead of Trump by anywhere from a little more than 7 points to 8.5 点数. Biden’s final margin in the popular vote was 4.5 点数.
That’s a big enough difference for the national polling average to be off by 3 至 4 点数. The national polling average 在 2016 overestimated Hillary Clinton’s margin by less than 2 点数, so the polling this year was worse.
明显, we’d prefer the polling to be better. It’s not good that the polling in two elections in a row underestimated Trump.
But we should be clear about the extent of the error we’re talking about here. The polls (when allocating undecideds evenly and concentrating on the major party candidates) had Biden winning approximately 54% 至 46%. He won on the order of 52.25% 至 47.75%.
In close elections where the margin between the candidates is within 5 要么 10 点数, which all nationally have been in the last 32 年份, a 4-point error can make a big difference.
Most issues, 然而, are not so evenly divided.
We can still feel quite confident that when polling finds that more than 55% of Americans agree on something, then a majority almost certainly do. (Note: additional potential errors from question wording and other areas where polling has erred such as volunteer work can add even more uncertainty.)
确实, approximation is what polling should generally be used for. I noted throughout this past election cycle that Biden and Trump were within the “true margin of error” of each other. 那是, the error necessary for Trump to win has occurred more than 5% of the time in past elections.
We may have gotten spoiled in some recent years with how close polling came to the outcome. The national polls differed from the final outcome by about a point in both 2004 和 2008.
历史上, 然而, polls can be much further off. A look at the final public polling aggregates 以来 1936 show that the polling errors have averaged about 4 点数, or about what they were this year.
If you were to construct a 95% confidence interval for the “true margin of error” 以来 1936, it would be about +/- 10 点数. The national error this year wasn’t anywhere close to that.
历史上, polling errors were greatest from 1936 至 1952, when scientific polling was first getting started. The polls also seemed to consistently overestimate one side in that era (共和党人).
We’ve obviously come a long way since then. That said, pollsters have new challenges. Some are specific to 2020 (e.g. the pandemic). Others are applicable to other cycles (e.g. declining response rates and the large number of pollsters using newer techniques such as online polling).
Pollsters probably need to and will likely figure out how exactly to deal with those issues.
目前, analysts, journalists and consumers have to recognize that merely repeating the sampling margin of error produced on major polls likely isn’t likely going to cut it. There are other sources of error not incorporated by the sampling margin of error that can rear their head.
This is especially true on the state level where the state polling left much to be desired in 2020, as it did four years ago.
的 FiveThirtyEight polling average, 例如, was off by about 5.5 points in the states where the final polling had the race within 10 点数, after being off by 4.4 points in 2016.Since 1972, the average error was 3.3 points in states within 10 points in the final polls. (不幸, we don’t have state polling since 1936 to compare it with. You could bet the historical error rate would be higher if we could go further back in time.)
的 2020 state polling was nearly the worst on record since 1972. 只要 1980 with an average error of 5.6 points was worse.
More worrisome, the direction of the errors (Republican underestimation) was the same in both 2016 和 2020.
This is absolutely something that must be examined more, and it’s likely to be by pollsters and their critics.
仍然, these errors shouldn’t be shocking.
以来 1972, 的 95% confidence interval for state polling averages in competitive states has been a little more than +/- 9 点数. We had two competitive states (Montana and Utah) exceed that, and both by less than 2 点数. We should have had about one based on history.
So what does this mean for consumers?
When one side is ahead of another by more than a few points, that advantage is probably real, even on the state level. It’s not until that lead is closer to double-digits that we know that edge is real outside of rare circumstances.
Perhaps the best way to think about the state polling (and polling at large) is to borrow a phrase from the Trump era: take it seriously, but not too literally.
