Tiny Modelby Around The CounterThere are no good polling models
Hello, and welcome to Tiny Model. Tiny Model is a collaboration between not-so-expert but still-quite-expert statistician and software engineer Alex Clemmer and Molly Jones. Over the next few weeks between now and the 2026 Midterm election on November 03, 2026 we are going to build an election model and attempt to predict the results across the U.S. And we want to bring you along with us! Whether you are a seasoned statistician or someone who has never heard the term “the generic ballot”, we’ve got something for you along the way. Welcome, we’re delighted you’re here.
There are no good polling models
(And we mean it!)
Every election cycle, the polling models show up (e.g., in the NYT, The Economist, 538 (RIP 🪦), Silver Bulletin etc.) to produce a fresh crop of election predictions. And every election cycle, the public incorrectly understands this to mean that it is possible to predict elections.
It is NOT POSSIBLE to predict elections. In some election cycles (like 2004, 2014, or 2016), the polls are simply incorrect. Because of this, 0 polling models get all of these cycles right. Not 538, not NYT, not The Economist. Not even during back-testing. “Back-testing” just means “we looked at how our model did in previous election cycles.”.
And when you think about it, this makes sense. You are building a polling model to predict election outcomes, and the data is completely wrong! How can you make accurate predictions if the data is completely wrong?
To get an intuition for this, we’ll build a tiny polling model and let you interactively tweak it to see exactly how hard it is to get any model to correctly predict these “outlier” cycles.
What goes into a polling model?
The goal of a polling model is to predict an election outcome. To predict an outcome, we’ve collected a whole bunch of data. Mostly polls, but also some information about the polls themselves that will determine how much “weight” we should give any individual poll.
Example polls
Weights for Siena/NYT
In other words, a poll gets more “weight” if it comes from a pollster with a long track record of polls that ended up being very close to the eventual outcome, is recent, has a large number of responses (i.e., thousands or more), and is a live-caller poll. As of writing, a large, recent poll from Siena/NYT will generally be given a lot of “weight” in this model. The Siena/NYT poll is largely considered the best pollster because they are very transparent about their methodology, conduct polls with massive numbers of people surveyed, and generally work hard to explain their outcomes and own their faults. All of which we love.
We also want to include data about other, miscellaneous things that we know tend to sway elections. Stuff like:
- How the economy is doing. A good economy tends to benefit incumbents.
- What people think of the president. If the president is, say, a Democrat that everyone hates, Democrats will tend to do poorly in the next House and the Senate races.
- Whether or not it’s a midterm election. The incumbent President’s party tends to do worse overall in midterms.
- Historical, correlated polling errors. If the polls systematically under- or over-estimate non-college white voters, then Pennsylvania, Michigan, and Wisconsin are all likely to have significant polling error.
All of this data is useful as we attempt to predict what the polls will do!
The simplest possible polling model
As this series progresses towards the November election, we’ll develop increasingly sophisticated uses of this data for election prediction and bring you along with us. But first, let’s start with the simplest possible model: the generic-ballot polling average. This model asks:
If the election were held today, would the public generally support Democrats or Republicans? e.g., would the electorate be D+2, or R+2, or what?
This kind of model does not answer questions like “how many house seats will Republicans win”, or “who will win the presidency”, or anything like that. In fact, it actually doesn’t technically predict anything at all! Notice that the question specifically asks “If the election were held today.”
And this question is especially important because what it does is tell politicians exactly how people feel today. And that today is doing a lot of heavy lifting. Many people read a poll and instantly think “the Democrats will win the election because of these poll results.” And that’s not correct. If the election were happening this very instant, perhaps the Democrats would win. But it’s almost never happening on the day and time the poll is in the field, so even when you’re reading the polling results, you should know that those are out of date results and any election that’s upcoming will likely be different from what you are reading.
Of course, we will develop models that can answer all of these questions and more. But we wanted to start with something simple and build into something more complex. Our goal is to bring you along with us as we develop from something naive to something complex. Through this process, we hope that you learn and grow with us!
How it works: the stupid version
One simple thing we could do to test how the electorate is leaning is to simply average the margins of all the polls we have. For example, if we have 5 polls D+5, R+1, D+3, R+2, D+4, the average un-weighted margin is (5-1+3-2+4) / 5 = D+1.8.
In other words, this “model” “predicts” that the electorate will generically lean Democratic by 1.8 points.
A simple polling average
Example polls
YouGov
Campaign internal
Siena/NYT
Local Research Co.
Ipsos
Average of 1 poll: R+1.0
How it works: the less stupid version
Of course, this almost certainly is not a very good model. In the first place, not all polls are equally good. So we want some polls to count for more aka have more “weight”. We also want to punish bad polls like internal campaign polls that skew towards a candidate and other pollsters who have bad methodology and are using polling as a means to influence voter or donor behavior.
So, to start, because of historical data, we know the following about polls:
- Bigger polls are (mostly) better. The more people you ask, the less chance there is that any individual mistake you make seriously impacts the outcome.
- Recent polls are (mostly) better. The electorate can (and does!) change their opinion frequently! Especially after a big event, like a debate (RIP Joe Biden).
- Some pollsters are better than others. NYT/Siena is excellent, and its long track record should count for more than newcomers.
- Partisan polls and internal polls are (mostly) worse. If a poll is conducted by a campaign, that should count for less than a serious, non-partisan pollster (see NYT/Sienna).
- Many polls in quick succession should not tilt the average too much. Outfits like YouGov sometimes run many, many polls at once, and if we counted them all at full weight, they’d overwhelm the average.
Now, let’s go back to our example. Suppose that each poll now also has a “weight”, like a grade, which is a combination of the above factors. An excellent poll might get a 100% or A+ grade, while an abysmal poll might get a 20% or F grade. In other words, the F poll here counts for 20% of what the A+ grade counts for.
So let’s give each poll a weight (or “grade”) and compute the weighted average:
- D+5 gets 20% (an F)
- R+1 gets 80% (a B-)
- D+3 gets 85% (a B)
- R+2 gets an 82% (a B-)
- D+4 gets 60% (a D-)
A weighted polling average
Weighted polls
Campaign internal
YouGov
Siena/NYT
Local Research Co.
Ipsos
Weighted average of 1 poll: D+1.0
In other words:
If we consider all polls equal, this polling model tells us Democrats are ahead, at D+1.8
If we consider some polls to be better than others, this polling model tells us Democrats are well within the margin of error, at D+0.7
This is a huge difference. It’s the difference between a relatively close victory and a nailbiter. And we get it by incorporating a little bit of historical data into the polling model!
It is impossible to predict elections
Now that we have a nice sense of how the models work, let’s look at an actual model, back-tested against the 538 archives.
Here is the formula we use to calculate the polling average. It roughly does what we talked about in the last section, assigning each poll a weight (“grade”).
average margin =
sum(poll weight * poll margin) / sum(poll weight)
poll weight =
recency
* sample size weight
* pollster quality weight
* population weight
* anti-flooding weight
* partisan/internal weight
This UI lets you change the weight we use for each factor, and dynamically shows you how those weights would work in previous election cycles. The “outlier” cycles like 2016 and 2014 are highlighted. Play around with it! Notice how hard it is to get ANY combination of weights to accurately predict all outlier years. *If you DO get all weights to accurately predict outlier years - email us! We want to know what you did.
Notice also that if you do get a model that predicts all outlier cycles, it is weird: it has weights that seem unlikely to represent the real world. We would call these models statistically implausible.
Prototype playground
Try to save the outlier years.
This is the interactive shape of the final backtest section. The numbers are illustrative for now; the important behavior is that “reasonable” settings can improve normal years while still struggling with outlier cycles.
Outlier error
2.1
Normal error
1.0
Plausibility
92
So, yes. Impossible.
So far in this series we’ve covered the simplest possible polling model. We’ve established how hard it is to predict the outlier cycles.
In the next article we’ll show a slightly more complicated model, (and we’ll start to << I FORGET OUR ROADMAP LOL >>
—> Email 2: Why We Built "Tiny Model" (When 538/NYTimes/Silver Bulletin Already Exists)
- The Wonk Concept: Tiny Model is intentionally naive and simple. While big aggregators use complex algorithms, stripping a model down to its bare bones shows how polling logic actually functions.
- Cooking & Cocktail Analogy: Learning to scramble a basic egg or make a simple 2-ingredient highball before trying complex gourmet recipes is a requisite to understanding exactly how these things operate. If you don’t understand it at its simplest, can you ever really hope to understand it at its more complex?
- Utah Seasoning: Utah is a polling desert compared to battlegrounds like Pennsylvania. A simple model highlights how we process limited state data
- ATC Focus: Joy. Translating PhD speak to Kitchen Counter conversations)
and we’ll lay out why we decided to even create our own model when there are some people whose entire job it is to create models like these. Why us, why now, and why a Utah focused publication? See you next week for more. :)