I was playing around with a really cool and easy to use analysis tool for Excel, Analyse It. And I thought I'd show how easy it is by running an old data set through it. So here is some quick analysis of an old political campaign I worked on. Seriously, this took about 30 minutes total.
We won the race overwhelmingly. In fact we won every precinct in the city. One thing the tool allows you to do is compare groups of data. Here is our vote share by city ward, with each blob representing a precinct:
This type of chart can be created with about three button presses. It shows we won big in our home ward (yay!). It also shows the distribution of precincts. They were pretty bunched together, but it might be interesting to look at the outliers. Why were they so different?
We did an ok job of predicting where turnout was going to come from. I can tell this by running a regression of our predicted turnout by precinct and the actual turnout by precinct. The tool does this very simply, with no specialist knowledge required:
This graph also takes about three button presses to generate, and comes with a neat set of stats telling us that we did an ok (not great) job of predicting turnout by precinct. But then we should have done, since historical turnout by precinct was widely available, and the best predictor of future behaviour is past behaviour.
Now here is the interesting bit: we were crap at predicting how people would actually vote. I should stress that this was a Democrat versus Democrat election, so it is a notoriously difficult thing to call. But either way, we didn't do a good job! Here is our categorisation of precincts in to how we thought they would vote in advance versus how they actually voted:
You have to say that's pretty bad. In the precincts we called for our opponent, we actually did better than those that we thought were too close to call. Its good to know that we did better in the precincts we thought would support our candidate. But in those precincts there is a really wide range of support. They could probably have been better broken out in to different categories.
We did try to break them out further, but without much luck. Here is a more detailed look at what vote we thought we would get versus what we actually got:
We did a good job of predicting the very best precincts, but beyond that group, all of the other five groups we decided on behaved very similarly.
We also broke the precincts out that we thought our opponent would do well in. That also didn't turn out to be a good predictor of the actual vote:
Pretty depressing really!
Data and targeting has played a huge part in many political campaigns. That is particularly clear from '08 and organisations like Catalist have played a huge part in this. In politics I had the pleasure of working with a fantastic group of people who would honestly evaluate their own work, but ...
... usually after elections finish, candidates' political campaigns close their doors and everyone goes off to the next campaign, tired and often ready to forget about the campaign. There is no will to evaluate what worked and no money to fund evaluation. And besides, the people who did the analysis often don't want to evaluate for fear that their analysis that they were paid handsomely for wasn't actually helpful after all. This is a tragedy.
Even if predictions aren't that helpful, having lots of data on a campaign makes people believe. Everyone has confidence in a well presented graph and they feel that those extra phone calls and those extra door knocks are being well targeted. In reality, I often believe that when you are lost, any old map will do.
However if there were more evaluation after the effect, perhaps campaign analysis would improve quicker and it would more often actually help the campaign to win, rather than just boosting morale.
Add a comment