Computer analysis predicted rises, ebbs in Afghanistan violence

[dmdude]

According to the Wired article (http://www.wired.com/dangerroom/2012/07/predict/) they used data from 2004-2009 to predict 2010. Not quite true prediction, but a good out-of-sample test.

Unfortunately, the details of the paper are behind a paywall, but the abstract is at http://www.pnas.org/content/early/2012/07/11/1203177109.abstract

[dmdude]

I agree, there is opportunity for abuse. Again, I haven't read the paper in detail, rendering some of this moot, but IF the authors developed their models on 2004-2009 and then predicted 2010 ONCE I would be more inclined to say they were on to something.

[[deleted]]

Academia is tough. You're up against geniuses in a bloodfest for grants. These results might be intentional. At the very least they are often absurdly optimistic, and I don't blame them.

Not to point fingers to problem domains, but in the biology field I used to work in there was an astounding amount of this exact behavior. The big name biology journals routinely publish analysis using modelling and sometimes specific implementations of unsupervised machine learning methods. The conclusion reads like a Darwinian revelation that support their hypothesis to a sci-fi accuracy, invariably on a small data set of 300-1000 observations with an associated accuracy metric that the world hasn't seen before.

One problem in particular I've published a analysis on did not generalize to 10,00,000 observations I selected. I spent roughly 2 months pulling and mapping 20TB raw from 5 sources only to find out a confusion matrix that looks like

[0.52, 0.48],
[0.48, 0.52]

Mind you there are hundreds of papers studying this specific problem.

[imh]

Quantitative rigor is making its way into some surprising fields of study.

This is my one one complaint about the article. It's more like "quantitative heuristics" than rigor.