Tim Harford je zelo lepo opisal problem empiričnih raziskav na problemu financ. Celo stoletje raziskovalci iščejo sveti gral oziroma spoznanje, kaj vpliva na borzne cene. Začeli so iz predpostavke, da se borzne cene gibljejo kot random walk in jih je torej nemogoče predvideti. Nato pa je lanski nobelovec Eugene Fama leta 1992 izpustil duha iz steklenice z raziskavo, ki je ugotavljala, da na donos finančnega portfelja vplivajo trije dejavniki. Temu je sledila plejada raziskav, ki so iskale vedno nove dejavnike, ki naj bi tudi vplivali na cene. Skupaj so jih identificirali kar 316. No, nekdanji urednik top akademske revije Journal of Finance Campbell Harvey je z dvema kolegoma pokazal, da gre pri teh raziskavah za absurd oziroma za t.i. jelly bean problem: preveč možnih primerjav, pri čemer za njimi ni nobene solidne teorije.
Kar pomeni, da je večina teh znanstvenih raziskav, čeprav objavljenih v top revijah, milo rečeno vprašljiva.
Discomfiting news: most of those financial strategies that claim to beat the market don’t. Even more surprising, many of the financial research papers that claim to have found patterns in financial markets haven’t.
Don’t take my word for it: this is the conclusion of three US-based academics, Campbell Harvey, Yan Liu and Heqing Zhu. What is particularly striking about the way they’ve lobbed a hand grenade into the finance research literature is that Campbell Harvey isn’t some heterodox radical. He’s the former editor of the leading journal in the field, The Journal of Finance.
What’s going on?
The finance literature has looked at far more than 20 possibilities. Harvey, Liu and Zhu scrutinise 316 different factors that have been explored by a selection of reputable research studies, of which 296 are statistically significant by conventional standards. That’s just a subset of the factors that have been examined in minor journals, or not published at all because the results were too boring.
For example, a paper might try to explain stock market returns as a function of media coverage of companies; of corporate debt; of momentum in previous returns; or of the volume of trades.
With 316 factors – and probably many more – under investigation, using a 5 per cent significance standard is absurd. Harvey and his colleagues suggest that after trying to correct for the jelly bean problem (more technically known as the multiple-comparisons problem), more than half the 296 statistically significant variables might have to be discarded. They suggest higher and more discerning statistical hurdles in future, not to mention a more explicit role for variables with some theory behind them, rather than variables that have happened to stick after the entire statistical fruit salad has been hurled at the wall.
None of this should astonish us. In 2005 an epidemiologist called John Ioannidis published a research paper that has become famous. It has the self-explanatory title “Why Most Published Research Findings Are False”. The reason is partly the multiple comparisons problem, and partly publication bias: a tendency on the part of researchers and journal editors alike to publish surprising findings and leave dull ones to languish in desk drawers.
Harvey and his colleagues have shown that the Ioannidis critique applies in the finance research literature too. No doubt it applies far more strongly in the advertisements we’re shown for financial products. We should have always been on the lookout for intriguing patterns in the data. But if we’re not careful, our analysis will produce plenty of flukes. And in finance, flukes are just as marketable as the truth.
Vir: Tim Harford