Definitions: March 2008 Archives

"Margin of error" is a measurement of how confident you can be of a statistical result.  My stats books have long since been relegated to a box in the basement, but if you want more specifics, try this link (http://whyfiles.org/009poll/math_primer.html).

The primary influence on the margin of error is the size of the sample, but methodology can also have an impact, although it generally isn't measurable until you discover just how wrong you were after the fact.  That's what happened in the famous Dewey vs. Truman example (they polled via telephone, and people who had telephones at the time turned out to be a rather different group of people than the general population).  Methodology problems also explain what went wrong with the predictions of who was going to win the 2000 presidential election in Florida, and why web-based polls (sell-selecting sample) are completely bogus.  Of course, if enough people start refusing to answer pollsters, all polls will be self-selecting and invalid.

The problem is that, although pollsters report margin of error, newspapers seldom give them more than a footnote.  So this year we've seen a lot of headlines like "XXX Takes the Lead" followed by an article that tells how the candidate is now two percentage points ahead, followed by something (buried deep inside) mentioning that the margin of error is plus or minus three percentage points.  Let's be perfectly clear.  If the difference in the polls is within the margin of error, then there is no difference.  None.  Any headline to the contrary is making up something that doesn't exist.  If you did the poll again it might well show XXX losing by two points--and that wouldn't mean anything either.

But of course, margin of error in polls is a pretty trivial problem compared to margin of error in vote taking machinery.  The reports I've seen (sorry, I've lost the reference) say that these lovely punch card machines are, when working absolutely optimally with absolutely wonderfully punched cards, 99.9% accurate.  That sounds nice, until we consider that the machines in use throughout polling machines in this country are almost certainly not in optimal condition, and we know for a fact that the cards weren't well punched either.  On top of it all, 99.9% sounds nice, but the results of the Florida elections were so close that the difference between the candidates was less than .1%.  Or in other words, the difference was within the margin of error of the counting machines.  Which means, just like in polling, the difference is statistically insignificant.  It doesn't mean anything.  Do it again and you might get a different result. However, in this case the errors tend towards a particular type.  Punch card voting machines have trouble with partially poked holes--they think nobody voted.  This isn't necessarily because the voter screwed up either--you can poke the things and the chad still might not separate properly. So that biases the results in a particular direction.

So, we have a vote that is so close that it is statistically a tie.  You can either treat it that way, or you can try and decrease the margin of error.  That's what hand counting is all about.  The real good news is you don't have to hand count every single vote.  All you need to count are the ones that the voting machine rejected as being possibly in error.  Voting machines don't tend to count unpunched holes as punched (and if they did, you'd get multiple votes--so those will be flagged as errors anyway).  So by counting just a few (relatively speaking) votes, you can greatly increase the accuracy of the count, and thus decrease the margin of error.  Meaning that even if the results are still off by a few hundred votes--now you have confidence that those votes actually represent the will of the people.

BTW.  If punch cards are so inaccurate, how come computers used them so easily?  Two reasons.  One, you didn't punch computer punch cards by hand--you used a special machine.  You'd enter the data for a card, and then it punches out the holes.  Secondly, computer punch cards, in recognition of the fact that you can have errors, tended to use a checksum to help ensure that if something did go wrong with a card, you'd notice.  (An example checksum might be a parity bit--where a hole would be punched at the end if the result of adding the numbers on the card was even, and not if it was odd.  If a hole became clogged, or the chad fell out, the checksum would not be likely to match, and the card would be rejected.)

So, what was that about instant gratification?

Some people (primarily pundits and candidates) have been making much of urgency to bring the election to resolution as soon as possible.  This sounds reasonable because we are used to our election results being announced nearly as soon as the polls close (and sometimes before).  But in fact, the whole concept is silly.  The elections are not over until the electoral college has made its decision.  But even before that, as people are finally seeing in the Florida situation, there are three levels of "results" from an election.  There are the polls, there are the unofficially tallies (often made by some organization jointly formed by the two major parties), and then there are the official tallies that the state certifies.  You virtually never see the official tallies.  That's particularly frustrating if you are trying to form a viable third party, since the unofficial tallies often don't even bother to count minor candidates.  (Quick, how many votes did the Socialist Party get nationwide?)  A few years ago I voted for a third party candidate for governor and I was never able to find out how many votes he received. By the time the official results were available, nobody considered it news--so the press never reported it.  So far this election has taken no longer than any other election has ever taken.  It's just that this time people actually want to know what the real results are, not just the fast food version.