Odds and ends

10/21/12

Odds and ends - FT.com

UNDERCOVER ECONOMIST

October 12, 2012 9:03 pm

Odds and ends Review by Tim Harford

An investigation into the art of forecasting argues for a pick-and-mix approach

©Rex

A contestant on the television gameshow ‘Let’s Make a Deal’ tries to predict w hich box contains a prize

T

he Signal and the Noise: The Art and Science of Prediction, by Nate Silver, Allen Lane RRP£25/Penguin Press, RRP$27.95, 544 pages

When it comes to soothsaying, there seem to be two types of person: those who will gladly and glibly opine on anything from the chance of rain tomorrow to the chance of Mitt Romney winning the presidency, and those who think the forecasting game is all but impossible, the exclusive preserve of fools and frauds. This state of affairs is not good enough, says Nate Silver, a statistician most celebrated for his New York Times blog, FiveThirtyEight, and its forecasts of election results in the US. While prediction is indeed a difficult affair, it is not hopeless. Thoughtful people with serious theories and mathematical www.f t.com/cms/s/2/19c5b98a-1151-11e2-8d5f -00144f eabdc0.html#axzz29wcVRO00

1/4

10/21/12


nous should get involved, argues Silver in The Signal and the Noise, not only because they have a good chance of raising the bar for forecasters but because prediction is the acid test of their expertise. Although Silver is a numbers man, and his book is seasoned with graphs, tables and the occasional equation, he advocates a pick-and-mix approach to forecasting. A statistical model may or may not beat expert judgment, and a computer may or may not out-forecast a human, but a judicious combination of approaches will generally outperform any particular method.

The Signal and the Noise Nate Silver Best Price £13.93 or Buy New £17.50

Privacy Information

What works in forecasting is not always effective when it comes to writing a book. Each chapter picks a different forecasting problem, from terrorism to baseball to climate change. These individual chapters are strong, and some of them are outstanding. The analysis of the subprime crisis is as lucid as any I have read, I was hooked by his account of computer chess and the explanation of weather forecasting is a revelation. The whole is less than the sum of the parts, alas. A chapter on poker seems to be there because Silver was once a professional poker player; the chess chapter, while brilliant, seems barely relevant. His pen portraits do not satisfy (do we care about Robert Daum, whose research is briefly cited, now that we know he is “a doctor’s doctor, with a dignified voice, a beard, and a detached sense of humor”?) and at times his efforts to document his extensive research descend into name dropping. “I was told by Tim Berners-Lee,” he writes, of some fact or other. After the first 400 pages of this kind of thing one begins to wish that Silver’s editor had been more assertive. Despite these frustrations, there is a great deal to admire in the book. It defies easy summary but at its heart is the admonition that we should all think more like Thomas Bayes, an 18thcentury minister and mathematician, nonconformist in both roles. Bayes’ theorem, published posthumously, tells us how to combine our pre-existing view of the world with new information in a rational way. Bayes’ theorem can produce some counterintuitive results. My colleague John Kay once published a pair of columns about the classic game show “Let’s Make A Deal”, in which a grand prize lurks in one of three boxes. The contestant provisionally chooses a box; the host of the show opens a different box to reveal no prize; then the contestant must decide whether to open her chosen box, or to switch at the last minute and open the only alternative. Bayes’ theorem demonstrates quite clearly that the contestant should switch, but Kay’s postbag was testimony to the fact that few people believe this conclusion. Silver explains Bayes’ theorem with a dark example: the attacks on the World Trade Center. www.f t.com/cms/s/2/19c5b98a-1151-11e2-8d5f -00144f eabdc0.html#axzz29wcVRO00

2/4

10/21/12


When the first plane hit the tower, horrified observers instinctively updated the possibility of a terrorist attack that day from “barely thinkable” to “distinctly possible”, although at that stage an accident could not be discounted. Bayes’ theorem shows that when the second plane hit, the chance of terrorism could be updated again, from “distinctly possible” to “all but certain”. There is no need for a mathematical analysis to tell us that, but Silver argues convincingly that Bayes’ theorem is an important reality check on our efforts to forecast the future. How, for instance, should we reconcile a large body of theory and evidence predicting global warming with the fact that there has been no warming trend over the last decade or so? Sceptics react with glee, while true believers dismiss the new information. A better response is to use Bayes’ theorem: the lack of recent warming is evidence against recent global warming predictions, but it is weak evidence. This is because there is enough variability in global temperatures to make such an outcome unsurprising. The new information should reduce our confidence in our models of global warming – but only a little. The same approach can be used in anything from an economic forecast to a hand of poker, and while Bayes’ theorem can be a formal affair, Bayesian reasoning also works as a rule of thumb. We tend to either dismiss new evidence, or embrace it as though nothing else matters. Bayesians try to weigh both the old hypothesis and the new evidence in a sensible way. This is good advice, and less technical than it might sound. Despite its flaws, The Signal and the Noise is a book worth reading. It says something new in a crowded field, it is fun to read, and it’s full of facts you will remember. There is some noise here, but Silver has also produced a signal that is a pleasure to follow. Tim Harford is an FT columnist and author of ‘Adapt: Why Success Always Starts with Failure’ (Abacus)

You may be interested in Investor frustration grows at State Street Financial ecosystems can be vulnerable too Early roles can cast a giant shadow Biology takes a quantum leap Housing pickup unlikely to fuel growth

Printed from: http://www.f t.com/cms/s/2/19c5b98a-1151-11e2-8d5f -00144f eabdc0.html

www.f t.com/cms/s/2/19c5b98a-1151-11e2-8d5f -00144f eabdc0.html#axzz29wcVRO00

3/4

10/21/12


Print a single copy of this article f or personal use. Contact us if y ou wish to print more to distribute to others. © THE FINANCIAL TIMES LTD 2012 FT and ‘Financial Times’ are trademarks of The Financial Times Ltd.

www.f t.com/cms/s/2/19c5b98a-1151-11e2-8d5f -00144f eabdc0.html#axzz29wcVRO00

4/4