So Nate Silver has been making the news lately, because his statistical analysis so perfectly (and humbly) predicted the election. Meanwhile, pundits everywhere did their usual entrails-reading analysis, and refused to eat the crow even after they were proven disastrously wrong.
And that’s all I’ll say about the election in this article.
But you can read an excerpt from his new book, as well as a telling NBC interview, here: Inside the mind of the man who predicted Obama’s win
The thing is, I’m not even sure my mom knows I studied econometrics in college, which was the application of statistical analysis to economic theory. So this perked up my ears on a personal level.
I didn’t pursue it because economic theory was just so much entrails at the time I was in college I felt that pursuing it any further would just dumb myself down. But Nate Silver helps to not only “make math cool” as the talking heads like to put it, he also proves that you can develop solid theories first, and then apply numerical analysis to those theories to make some pretty damn accurate predictions.
So that’s heartening to me. Maybe I still have some science career ahead of me. But that depends whether I can find a place in academia, even as a maverick. Because there are a few things that need to happen in the social sciences before statistical analysis can properly be elevated from entrails-reading to a diligent science.
And I’ll explain it with a bit of philosophy. Before you can make a quantitative analysis of anything, you first need an accepted qualitative theory. Examples: before you can accurately chart and predict the movements of the planets, you need Newton’s universal law of gravitation. Before you can split an atom, you need E=mc2. Before you can refine oil, you need laws of chemistry. These are obvious now, but they weren’t always given. Planetary movement seemed odd and random to us, back when our astronomical theory was based on heavenly circular movement. And so on.
In the field of economics and politcs it becomes doubly difficult, because we are what we’re studying – we’re no longer watching an object through a looking glass like we watch the planets. On a higher level, this means a different philosophy towards the science, but on a basic level, these sciences are so politicized that they believe their own bullshit and lies. They can’t predict their own crises? Of course not! They’re too busy selling overinflated stock to ever entertain the notion that it’s overvalued and can collapse.
But guess what? Karl Marx made some really accurate qualitative theories about boom and bust cycles, as well as how capital accumulates, and needs to accumulate. It’s so precise it predicts not only crises, but trade wars, and real wars. It’s too bad that its revolutionary implications mean that academia, which seeks to find a role within society, tends to ignore it completely, throwing out the baby with the bathwater.
There is plenty of promise in the field of statistical analysis in the social sciences. But there’s also a lot of work to be done to make it legitimate. Work which I’m not sure the universities are up to, and work that I’m not willing to do on my own, especially if it means arguing against a bunch of loudmouth dipshits.
There are good professors out there who offer a proper qualitative understanding of contemporary capitalism. This illustrated David Harvey lecture offers a great framework for analysis. The question for faculties out there is whether they make the courageous decisions to hire more professors like this, and fewer apologists, who can write enormous amounts of shiny garbage but it all amounts to wasted time and money.
To close, I wanted to quote Nate Silver’s book, where he talks about the explosion in information that the internet provides: “This exponential growth in information is sometimes seen as a cure-all, as computers were in the 1970s. Chris Anderson, the editor of Wired magazine, wrote in 2008 that the sheer volume of data would obviate the need for theory, and even the scientific method.”
This is nothing new. I came of age when Michael Crichton wrote Jurrasic Park, which popularized chaos theory and ridiculed this same notion. It’s too bad that he only offers the unknowableness of nature and a superstitious reverence for it as an alternative. Because chaos theory actually says we overcomplicate things, and that behind seeming chaos and disorder isn’t unknowability, but a very simple equation with a profoundly different philosophy. Understanding that equation at the heart of systems has enabled us to develop more effective predictors for everything from hurricanes to elections. We can only expand on that.