The Deluxe version of the FiveThirtyEight midterm forecast just shifted slightly, but it’s not because of anything that happened in politics. Rather, it’s because we identified — and fixed — an error related to missing data that was leading to incorrect odds for several Senate, House and gubernatorial races. We wanted to take some time to explain what happened, how we identified it and the effect the correction had.
On Thursday, Sabato’s Crystal Ball — one of three election forecasters whose race ratings factor into the Deluxe version of the FiveThirtyEight midterm forecast — moved the Ohio 9th Congressional District from “toss-up” to “leans Democratic.” When we went to input this change into our forecast, though, we found that our spreadsheet of race ratings was missing any Crystal Ball rating for the Ohio 9th whatsoever.
Upon closer inspection, we discovered we were actually missing Crystal Ball ratings for dozens of races. We used to have this data, but a look back at old files shows it was accidentally deleted on June 30. (Ratings from the other expert race raters, Inside Elections and the Cook Political Report, were unaffected.)
On Friday, we re-collected the Crystal Ball ratings data and added it back into the model — and the Deluxe version of the forecast has changed slightly as a result. (The Lite and Classic versions of the forecast, which don’t incorporate expert race ratings, remain unchanged.) Democrats’ overall odds of winning the House increased by 0.7 percentage points to 32-in-100, but their odds of winning the Senate decreased by 0.5 points to 70-in-100. More importantly, though, some individual races did shift significantly. Here’s every race whose odds changed by at least 1 percentage point.
Several races shifted after we fixed a bug in our forecast
Every race where the odds of winning shifted by 1 percentage point or more after we added missing Sabato’s Crystal Ball ratings to the FiveThirtyEight Deluxe forecast
| Race | Before | After | Change |
|---|---|---|---|
| TX-15 | 41.3% | 55.9% | +14.7 |
| NC-13 | 45.6 | 32.4 | -13.2 |
| MI-03 | 81.7 | 68.9 | -12.8 |
| IA-03 | 34.2 | 46.1 | +11.9 |
| CO-08 | 44.7 | 33.0 | -11.7 |
| WA-08 | 75.8 | 85.9 | +10.1 |
| NV-03 | 64.9 | 74.6 | +9.6 |
| OR-05 | 38.7 | 46.6 | +7.9 |
| CA-13 | 69.5 | 77.3 | +7.9 |
| NY-19 | 43.3 | 50.9 | +7.6 |
| NH-01 | 77.2 | 84.8 | +7.6 |
| CA-45 | 38.0 | 30.4 | -7.6 |
| IN-01 | 79.3 | 86.3 | +7.0 |
| TX-28 | 76.8 | 83.0 | +6.2 |
| AK-AL | 35.2 | 41.2 | +6.1 |
| NY-18 | 62.5 | 68.5 | +6.0 |
| RI-02 | 76.8 | 82.3 | +5.5 |
| NJ-07 | 43.0 | 37.7 | -5.3 |
| IL-13 | 86.2 | 90.6 | +4.4 |
| MD-06 | 66.8 | 62.5 | -4.4 |
| CA-09 | 77.8 | 82.0 | +4.2 |
| NY-03 | 77.2 | 73.3 | -3.9 |
| CT-05 | 77.6 | 81.5 | +3.9 |
| NY-22 | 54.2 | 50.5 | -3.7 |
| CT-02 | 87.6 | 84.3 | -3.4 |
| NE-02 | 17.1 | 14.0 | -3.1 |
| TX-34 | 81.1 | 78.1 | -3.0 |
| CO-07 | 88.1 | 91.0 | +2.8 |
| NJ-03 | 92.3 | 89.5 | -2.8 |
| MI-07 | 78.8 | 76.1 | -2.7 |
| OH-09 | 72.4 | 74.8 | +2.4 |
| CA-41 | 11.4 | 13.7 | +2.4 |
| OR-06 | 72.7 | 74.9 | +2.2 |
| MI-08 | 83.1 | 81.0 | -2.1 |
| WI Gov. | 67.8 | 65.8 | -2.0 |
| WI Sen. | 42.0 | 39.9 | -2.0 |
| NY-01 | 33.2 | 31.2 | -2.0 |
| AZ-04 | 91.9 | 93.8 | +2.0 |
| IA-02 | 13.0 | 11.1 | -1.9 |
| NY-17 | 89.1 | 87.2 | -1.9 |
| IL-14 | 91.0 | 89.2 | -1.8 |
| IA-01 | 11.4 | 9.9 | -1.5 |
| IL-11 | 96.3 | 94.9 | -1.5 |
| GA-02 | 93.8 | 95.2 | +1.3 |
| NC-14 | 97.4 | 96.0 | -1.3 |
| CA-25 | 96.6 | 95.3 | -1.3 |
| VA-02 | 50.6 | 49.3 | -1.3 |
| NJ-05 | 96.5 | 95.2 | -1.3 |
| WA-03 | 2.9 | 4.2 | +1.3 |
| WI-03 | 26.8 | 25.6 | -1.2 |
| AZ-02 | 50.8 | 49.6 | -1.2 |
| TN-05 | 3.8 | 2.7 | -1.1 |
| CA-49 | 93.7 | 92.6 | -1.1 |
| NH-02 | 95.1 | 94.0 | -1.0 |
| NM-03 | 97.2 | 96.2 | -1.0 |
We know how much readers rely on our forecast to be as accurate as possible, and we’re aware that, for some people — politicians, campaign staffers and voters alike — the forecast represents something more than just a number on a page. So we’re taking steps to ensure this error doesn’t recur. We usually use an automated process to collect and update race ratings, but we’ve turned that off so that we can manually check all future ratings changes from all three election forecasters. And while we’re still unsure how the data got deleted, we are actively investigating to ensure we patch up any systematic problems in our workflow. We hope you’ll continue to use and trust our forecast.
CORRECTION (Sept. 23, 2022, 8:36 p.m.): An earlier version of this article transposed Democrats’ chances of winning the House and Senate. Democrats have a 32-in-100 chance of winning the House and a 70-in-100 chance of winning the Senate, not the other way around.
