Carl Allen

1/x: re: sports data, politics data, and my predictive models I've noticed that most of the arguments/debates I've had with people, mostly about sports (but elections too) have a major common underlying theme: That past results are the best predictor of future performance.

This is some outstanding work which gets into the REAL spot the (real) analytics folks should be doing: Probabilistic distribution of polling errors. Eg 90% chance polling error was at least 1% 70% chance at least 2% 50% chance at least 3% Etc. This is how it works irl

Consider this race: median forecasted result Flanagan by 6 You observe 45-43. Undecideds likely lean D But as a pollster, you're judged by how closely poll margin = election margin So...do honest work - get called bad Fudge your numbers - get called accurate Healthy science

And to be clear, Trafalgar flooded poll avgs very close to the election, and FiveThirtyEight (under Silver, and still under Morris) reward them for being most recent And note the grade given to Trafalgar. These guys literally think that getting lucky in 16 and 20 is predictive

Relevant: In my book, I introduced "ideal polls" A poll whose only source of error is the MOE itself It's intended to be a "zero friction" benchmark against which real-world polls can be measured It gives a theoretical upper bound of precision Accuracy by his metric is luck

Reminder that comparing poll margin to election margin to determine "most accurate" pollster is statistical incompetence It operationalizes the assumption that undecideds split evenly, and disregards the high probability that even a perfectly accurate poll won't hit the margin

And the people who calculate "poll accuracy" assume unconditionally across all election types and state makeup that undecideds will always split evenly to the major candidates.

The tldr version is exactly correct as you summarized Assuming a precise and unconditional resolution of undecided (not to mention third-party mind changers) across all election types, states, and various other dynamics is such a basic violation of statistical reasoning lmao

Assuming the polls are "biased" because they don't predict the election margin is like saying this photograph was biased because it didn't predict the margin of victory The statement itself requires a fundamental misunderstanding of what the data means in the first place

Made this with conversations like this in mind

Every person that studies poll data must be able to answer the question below It's extremely basic And yet, with exactly zero exceptions in the literature, they all get it wrong. Before my work, there was NO framework to explain what the right answer is, or how to find it.

"What is your book about, why does it matter?" As short as I can: The consensus of experts in the field believe that polls, if accurate, will predict both the election outcome and its margin...and that's incorrect. In my book, I build a proper foundation for analyzing poll accuracy, starting with the painfully simple (but necessary) "what do polls measure?" and introduce some new concepts that should be adopted by stats classes and experts alike. Then, with a proper foundation, I contrast my work to that of the field's established methods (Spread Method in the US, Proportional Method most other places) and outline in detail why they're wrong...but why they're appealing. After that, I introduce some nonpolitical polling examples to illustrate some other concepts...ideal polls, compensating error, poll masking, changing attitudes, undecided ratio...that also exist in political polls Finally, I tackle head-on some real election data, 2016 US and 2015/17 UK and Brexit, analyzing them according to a more proper scientific standard... opposed to the current polls-as-predictions one.

In all seriousness (I know I'm being an asshole) This practice of comparing polls to elections is not valid The analysis of what a poll says based on "margin" is also not valid But this is how it has always been done. They don't know better because they don't know better.

Correct If someone says "smoking causes cancer" and insists that this is a scientific truth that everyone must accept Then it's pretty important that they can define the terms @zaelefty et al being unable/unwilling to define what it means for a body to be "organized around" 🚩

Just asking ignorant ideologues like @zaelefty to adhere to standards of basic science and provide objective, falsifiable criteria to support their position Last time he tried, I called him out on it and he ran away like a bitch And he's too big of a coward to try me