Mystery Pollster analyzes the official report of the Election Night exit-poll debacle; Mickey Kaus summarizes his conclusions thusly:
Mystery Pollster’s analysis of the offical Mitofsky exit poll screw-up report confirms that, as suspected, the problem was not bloggers, and not some esoteric technical bias it takes an advanced degree to figure out. The problem is that Mitofsky [the exit polling organization] had built a cheesy, dime-store organization that relied crucially on poorly-trained young people at the bottom. … [T]here was a lot less going on behind the curtain than Mitofsky’s arrogant professionalism would lead you to expect.
And yes, I’m quoting a summary of the analysis of the report because I’m far too lazy to read the primary or even the secondary source material. :)
UPDATE: Okay, okay, I’m reading Mystery Pollster’s actual post now. This is particularly interesting:
The problem was not in the selection of the sample precincts — it was that the data in the chosen precincts was not representative of the actual voting at those precincts. … The authors…found higher rates of “within precinct error” favoring Kerry in precincts with the following characteristics:
* An interviewer age 35 or lower
* An interviewer with a graduate degree
* A larger number of voters, where a smaller proportion were selected
* An interviewer with less experience
* An interviewer who had been hired a week or less prior to the election
* An interviewer who said they had been trained “somewhat or not very well.”
* In cities and suburbs
* In swing states
* Where Bush ran stronger
* Interviewers had to stand far from the exits
* Interviewers could not approach every voter
* Polling place officials were not cooperative
* Voters were not cooperative
* Poll-watchers or lawyers interfered with interviewing
* Weather affected interviewing…Unfortunately, none of the characteristics above, by itself, “proves” the Kerry supporters were more likely than Bush supporters to participate in the poll. However, it is not hard to see the underlying attitudes and behaviors at work might create and exacerbate the within-precinct bias.
Consider age, for example. What assumptions might a voter make about a college student approaching with a clipboard? Would it be crazy to assume that student was a Kerry supporter? If you were a Bush voter already suspicious of the media, might the appearance of such an interviewer make you just a bit more likely to say no, or to walk briskly in the other direction? Would it be easier to avoid that interviewer if they were standing farther away? What if the interviewer were forced to stand 100 feet away, among a group of electioneering Democrats - would the Bush voter be more likely to avoid the whole group?
Sounds pretty damn plausible to me.
P.S. More here, including a mention of similar problems in past elections, and a quote that could be applied to many, many things in life:
So many who are considering the exit poll problem yearn for simple, tidy answers that can be easily proved or dismissed: It was fraud! It was incompetence! Someone is lying! Unfortunately, this is one of those problems for which simple answers are elusive.
|
Categories: Election 2004
|
January 25th, 2005 at 11:17:23 pm
And what kind of person would they choose to interview? A volvo driving person that looks like their neighbors, or a guy in a “F’em all let god sort it out” hat and a bumper sticker that says “I’d rather be bowhunting.”
January 25th, 2005 at 11:26:13 pm
And what kind of person would they choose to interview?
Not so fast, Andrew. That form of “selection bias” is just a wee bit too obvious not to have been taken into consideration by the pollsters. According to the rules that all interviewers are required to follow, they are supposed to interview every “nth” person who walks out of the polling place (i.e., every eighth person or every eleventh person, whatever), regardless of what they look like or what kind of car they’re driving. They’re not supposed to “choose” at all.
That said, because many of the interviewers were young and poorly trained, what you are suggesting may very well be true, though in a considerably less widespread way than you’re envisioning. But then, it’s “bias at the edges” that we’re looking for, after all. MP addresses your point specifically:
The problem may have been as innocent as an inexperienced interviewer feeling too intimidated to follow the procedure and approach every “nth” voter and occasionally skipping over those who “looked” hostile, choosing instead those who seemed more friendly and approachable.
And, as you say, the guy with the “f**k the terrorists” hat would surely “look” a lot more hostile to a liberal college student than the Volvo driver drive. But it’s a matter of “occasionally” breaking the rules, not rampantly “choosing” people who look liberal. That’s simply not how it works, and no way is the answer that simple or obvious.
Another possible example of occasional bias on the edges:
Suppose the randomly selected voter [a Bush voter, suspicious of college students with clipboards] did not want to participate but his wife - a Kerry supporter - eagerly volunteers to take his place. Would the less experienced interviewer be more likely to bend the selection rules so she could take the poll?
So you might very well be right to some extent, but be careful about the assumptions you make. As Mystery Pollster says, this is NOT a simple problem.
January 26th, 2005 at 12:05:53 am
I say f’ em all and let God sort ‘em out!
Love,
The 20-something conservative population who looks like your neighbor, is highly educated, and democrats can’t admit exist.
PS - And we’ve never been bowhunting!
January 26th, 2005 at 1:12:27 am
But bowhunting is necessary to maintain the proper balance of nature. Otherwise, the bow population explodes, and their prey species are devastated…
January 26th, 2005 at 2:09:22 am
Amen. BK.
January 26th, 2005 at 10:51:17 am
Reminds me of the Simpsons last night:
Poachers are natures way of evening out the balance…
January 27th, 2005 at 4:07:52 pm
Waw haw, Mike! Again we bow to your wisdom. :)
The 2000 Florida debacle, whose venom has so poisoned our national bloodstream, was triggered, and greatly exacerbated, by erroneous interpretation of flawed exit-poll data by the late unlamented Voter News Service.
The vaunted Mitofsky Cure turns out to be worse (in scope anyway) than the Disease. Thankfully the Media, rightly Gun-shy after triggerhappily shooting themselves in the Butt four years ago, didn’t Project prematurely in the exit-poll-misallocated Battlegrounds. But the EP’s leaked (duh!), causing Repubs to Panic (until Rove Rode to their Rescue with his late-afternoon “don’t give up the ship” emails :) & Dems to Gloat & Teddy Kennedy to practically appoint the Kerry Cabinet on CNN. :) Then when Reality set in — again — presto!, we get new Grist for the Paranoid Fraudmania mill, yeah like any more was Needed. Bah.
Did I dream, or did I read somewhere, that They are considering going back to the Golden Oldy system :) — of projections based solely on actual (albeit unofficial) election results from the microcosmically-representative Key Precincts? / If I only Dreamed it, They ought to wake up and just Do it. Screw the exit polls. (Whose ONLY benefit — so called — is greater Speed. Which, being forever the sworn enemy of Accuracy, is in fact a Detriment accordingly.)
Of course the Key Precincts Returns-Only method can — did — will — also Err sometimes. (Poor precinct selection, tally errors by media “stringers” and/or officials, incomplete results [e.g. no ABs], etc.) But at least it won’t be because of punks polling the wrong voters. :) By definition, the Voters who Voted in the precinct — not a Sample of them, All (or nearly all) of them — are the right voters. / And: if there are no exit polls, there will be no exit-poll Leaks. :)
(Better yet of course: defenestrate the Projections altogether. No, really. Patience is a Virtue. The only decisive votes are The Votes. Let us Hurry up & Wait. :)
January 27th, 2005 at 4:27:32 pm
“Screw the exit polls. (Whose ONLY benefit — so called — is greater Speed.”
Actually, that’s not true at all. Without exit polls, we would have no data on who voted for whom and why. We wouldn’t have any info about whether voters were thinking about the economy, Iraq, or which candidate looks better in a suit. We wouldn’t know how blacks voted, how Hispanics voted, how gays voted, how Evangelicals voted. Granted, we don’t truly know these things now, but after all the corrections are made to the “raw” exit poll data, we have reasonably decent estimates. Throw out the exit polls, and all that information disappears completely, much to the dismay of everyone from political pundits and campaign strategists to social scientists and historians.
January 27th, 2005 at 5:17:13 pm
“…all that information disappears completely, much to the dismay of everyone from political pundits and campaign strategists to social scientists and historians.”
Yes, so, what’s yer Point? :)
WAW haw haw haw NO no no no… :)
OKOK. So I Forgot about that little matter. :) What can I yell ya, I’m just an Elections Bureaucrat. Whaddoo I looklike here, Per-fesser Arglebargle of the Sociology Dept.? Don’t answer that. :)
Actually we can (and we Used to, Sonny :) estimate Black & Hispanic & some Other ethnic voting patterns fairly decently from actual Returns in de-facto-segregated precincts. BUT yes, that gets increasingly Flawed as such populations increasingly Melt throughout the Pot (and Change their Politics concomitantly. ) As for Gays, that method wouldn’t have worked even if it had been so applied Back in the Day, because the Closeted Diaspora was always so widespread that you couldn’t reasonably Extrapolate the vote from the few known isolated small enclaves like the City of San Francisco. :) Similarly, Evangelicals & North Georgia. Hi Charles, you old Basque Baptist, you. :)
And re the Why, & the What (if at All) do you Think oh sovereign Voter? — right. No Data.
All right — so have Somebody :) DO the exit polls for those reasons; but not give ‘em to the Media, or to ANYone, until (say) three days after the election. By then the Turmoil & the Shouting will have (usually) Died; and in any case it’ll be effectively too late to use the junk for Projections, since the few remaining Too-close-to-Call states will have nearly-complete Unofficial tallies anyway and nobody’s gonna then Allocate such Real-vote squeakyclose ones based on polls.
Best of Both worlds, see? Perfect. Cake Had & Eaten too. :) Hm? Who’s going to PAY for such an EP operation, you ask, since the Deeppockets Tee Vee people can’t use it on Election Night? Details, details, don’t Bother me with the Trivia, I’m busy with the Bigthink Gnosis, here.