[Editorial note: During the next few weeks, each of the editors of the Mathematical Investor will provide, in an essay format, some personal background explaining the origins of their interest and work in this area. This is a perspective essay by David H. Bailey.]
Early interest in economics and finance
Although I only recently began to delve into the world of financial mathematics in any technical depth, I have been interested in (indeed, fascinated by) the world of economics and finance for many years. During my senior year at Brigham Young University, I very much enjoyed a course in economics that I took there. I may have freaked out my professor when, on the final exam, I invoked measure theory to answer an exam question. And I still remember, in my 20s, watching “Wall Street Week” with Louis Rukeyser on PBS television, wondering how I would invest money if I had any (which, as a poor mathematician, I didn’t).
Mathematicians and hedge funds
As early as the late 1970s, when I was completing my Ph.D. degree in mathematics at Stanford, I learned that some mathematicians (both recent graduates and established academics) were going to work for Wall Street firms. In 1982, for example, James H. Simons, a distinguished mathematician who had done prize-winning work in multidimensional geometry, left his position as Chair of the Mathematics Department at SUNY Stony Brook to form an investment fund based on sophisticated mathematical algorithms. Among these techniques, as I later learned, was the application of hidden Markov models to uncover trends in stock or commodity market data.
Today, as is well known, Simons’ Renaissance Technologies is one of the world’s largest hedge fund management firms, operating at least four funds with combined assets exceeding $15 billion. At least 30% of its staff of 275 employees have Ph.D. degrees in a variety of fields, including mathematics, physics, astrophysics and statistics. Renaissance’s Medallion Fund has returned to its investors an average return exceeding 30% since its founding in 1989.
Simons retired in 2009, after amassing considerable personal wealth (approximately $12 billion), which currently places him #31 on the Forbes list of U.S. billionaires. He has been involved in a number of mathematics-related charitable efforts, including, for example, providing major funding to the Mathematical Sciences Research Institute in Berkeley, California. Recently he established the Simons Foundation, which provides funding for mathematical researchers and operates the Quanta online magazine, a high-quality source of news in the arena of mathematics, physics and computer science.
Similarly, I know well the hedge fund operated by computer scientist David E. Shaw. This fund, as far as I am aware, employs a strategy of determining models of stock prices, based on massive statistical analysis of stock market data, then trading securities when their market price exceeds or lags the model price. Shaw’s fund has been highly successful, and Shaw, like Simons, is very wealthy, currently ranking #131 on the Forbes list of U.S. billionaires.
Shaw has been quite active in promoting science and technology. Among other things, he has served on the Presidential Council of Advisors on Science and Technology under both Clinton and Obama. More recently, he established D. E. Shaw Research, a research institute in high-performance computing. His research team and their “Anton” computer system, which is dedicated to performing protein folding computations, has received several awards, including Best Paper at the 2011 Supercomputing Conference. I personally met Shaw at this meeting and have chatted with him once or twice since then.
Recent research work
Given what I have learned about mathematical finance, I realized long ago that it is hopeless for an individual investor such as myself to think that he/she can compete with the likes of the Medallion Fund or the D. E. Shaw Fund, with their large teams of very bright analysts employing extremely sophisticated mathematical and computational techniques. So I have invested savings mostly in broad-market mutual funds or exchange-traded funds, and only occasionally change my holdings. The only time I departed from this strategy was to invest in technology-heavy funds during the 1998-2000 boom. The ensuing fall was a hard lesson on the consequences of not sticking to a disciplined strategy!
Two or three years ago, based on my continuing interest in economics and finance, I embarked on a very fruitful research collaboration with Marcos Lopez de Prado, who is one of the co-editors of this blog. We analyzed investment risk management strategy techniques, using methods of mathematical statistics. This collaboration resulted in several research papers, including for example, Balanced baskets: A new approach to trading and hedging risks.
The investment enigma
Recently I retired from my position at the Lawrence Berkeley National Laboratory. As I approached retirement, and even more since retiring, I have paid careful attention to financial news, in part to ensure the continuing health of my personal savings. In this process, I have read numerous financial news and analysis articles that seemed to me to make little sense, based on what I have learned over the years. For example, some would offer confident predictions that the market would go up, or go down, or go up then down in the coming weeks. I have also read articles presenting detailed analyses, filled with charts and graphs, that, upon careful reading, I did not understand. What was the solid scientific basis for these analyses? They did not seem to be as rigorous or as scientific as that being done by leading hedge funds.
After thinking about these matters, on June 10, 2013 I sent my colleague Marcos Lopez de Prado an email note with a “dumb” question. Noting that the stock market contains, almost by definition, the consensus of all available information, reacting very rapidly to current events, and given that thousands of very bright mathematicians and computer scientists working for investment funds are employing very sophisticated methods to ferret out any conceivable advantage that could be had by data analysis alone, how can the financial analysts I was reading possibly be offering scientific advice?
It is true that markets are not always rational. But the articles I had read did not seem to be presenting assessments of business conditions, economic outlook, market psychology or the merits of any individual securities. Instead they were offering advice based on data analyses. But as I pointed out in my note, “if these [analysts] really had a substantive, scientifically defensible basis for making such predictions, they wouldn’t be selling their advice online, but instead would be making millions advising institutional clients with many billions of dollars.”
A scientific response
Lopez de Prado responded that this is not a “dumb question” at all, but is something that he and fellow researchers in the field of finance and investment have wrestled with for years: a significant portion of financial analysis in the public arena today is not well-grounded in rigorous mathematics and statistics, and remains to be proven effective over the long haul.
As we further discussed this over the intervening months, we resolved to highlight inaccuracies and provide a scientifically defensible alternative. In other words, our intent is not to debunk so much as to educate and do research in the field, publishing results that help both mathematicians and financial practitioners better understand the issues.
After some discussions, we teamed with my long-time collaborator Prof. Jonathan Borwein of the University of Newcastle, Australia, and a new colleague, Prof. Qiji Jim Zhu of Western Michigan University. This collaboration led to two research papers, Pseudo-mathematics and financial charlatanism: The effects of backtest over fitting on out-of-sample performance and The probability of backtest overfitting. It also led to the construction of this website and blog.
The rest, as they say, is history.