Backtest overfitting and the post-hoc probability fallacy

13-card hand dealt “at random”

Introduction

In several articles on this site (see, for instance, A and B), we have commented on the dangers of backtest overfitting in finance.

By backtest overfitting, we mean the usage of historical market data to develop an investment model, strategy or fund, where many variations are tried on the same fixed dataset. Backtest overfitting, a form of selection bias under multiple testing, has long plagued the field of finance and is now thought to be the leading reason why investments that look great when designed often disappoint when actually fielded to investors.

Continue reading Backtest overfitting and the post-hoc probability fallacy

How backtest overfitting in finance leads to false discoveries

The present author, together with Marcos López de Prado, has just published the article How backtest overfitting in finance leads to false discoveries in Significance, a journal of the British Statistical Society. The published article is now available at the Significance (Wiley) website.

This article is condensed from the following manuscript, which is freely available from SSRN: Finance is Not Excused: Why Finance Should Not Flout Basic Principles of Statistics.

This paper introduces the problem of backtest overfitting in finance to the general reader who may be trained in the basics of statistics but not necessarily familiar with the application

Continue reading How backtest overfitting in finance leads to false discoveries

The brave new world of probability and statistics

Block function approximation to normal distribution

Introduction

Today, arguably more than ever before, the world is governed by the science of probability and statistics. “Big data” is now the norm in scientific research, with terabytes of data streaming into research centers from satellites and experimental facilities, analyzed by supercomputers. “Data mining” is now an essential part of mathematical finance and business management. Numerous public opinion polls, expertly analyzed, guide the political arena. Covid-19 infection rates, immunization levels and r0 factors are a staple of nightly newscasts.

Yet the public at large remains mostly ignorant of the basic principles

Continue reading The brave new world of probability and statistics

Can astrology predict financial markets??

Venice astrological circle; credit Wikimedia

Astrology in finance?

In a previous MathInvestor article, we mentioned how absurd it would be if someone offered predictions of stock or bond prices or cryptocurrency rates based on astrological signs.

Consider for a moment that financial market prices are based on a confluence of many thousands of factors worldwide, including developments in science and technology, changes in consumer sentiment and preferences, changes in prices of production, public health emergencies (e.g., Covid-19), political developments, competition with other financial instruments and even changes in weather. These prices are negotiated electronically, on behalf of millions of

Continue reading Can astrology predict financial markets??

The failure of anomaly indicators in finance

A black swan; credit: Wikimedia

The replicability crisis in science

Recent public reports have underscored a crisis of replicability in numerous fields of science:

In 2012, Amgen researchers reported that they were able to replicate fewer than 10 of 53 cancer studies. In March 2014, physicists announced with fanfare that they had detected evidence of gravitational waves from the “inflation” epoch of the big bang. However, other researchers were unable to verify this conclusion. The current consensus is that the twisting patterns in the data are due to dust in the Milky Way, not inflation. In 2015, in a

Continue reading The failure of anomaly indicators in finance

Another miserable year for market forecasters

Venice astrological circle; credit Wikimedia

Pseudoscience and forecasting

Suppose, during a nightly TV weather broadcast, that a reporter presented forecasts by persons, with no credentials in mathematical meteorology, who based their analysis on eyeballing a few charts and graphs. If anyone took such amateur forecasts seriously, when a severe storm was approaching, rather than relying on the consensus of qualified scientists assisted by state-of-the-art supercomputer models, they would risk disaster.

Or suppose that someone suggested that “biorhythms” (a person’s presumed daily, monthly and yearly cycles that supposedly start in sync at birth) could be used to predict the performance

Continue reading Another miserable year for market forecasters

López de Prado on machine learning in finance

Marcos López de Prado

Introduction

Marcos López de Prado, whom we have featured in previous Math Scholar articles (see Article A, Article B and Article C), has been invited to present a keynote presentation at the ACM Conference on Artificial Intelligence in Finance, to be conducted virtually October 14-16, 2020.

López de Prado is a faculty member of Cornell University and also CEO of True Positive Technologies, LP, a private firm that provides machine learning techniques techniques for finance applications. He is also the author of two books in the field: Advances in Financial Machine Learning, published by Wiley

Continue reading López de Prado on machine learning in finance

Day-trading in the age of coronavirus

Amateur pseudoscience

Suppose, during a TV weather broadcast, that the reporter presented “forecasts” from several private persons, wholly unqualified in meteorology, who base their forecasts on eyeballing a handful of charts and graphs. Most of us would never rely on such clearly amateurish methods ourselves, and would reject outright any such forecasts presented by others. If anyone took such forecasts seriously, when a severe storm was approaching, rather than relying on the consensus of qualified government scientists assisted by state-of-the-art supercomputer models, they would risk disaster, including the potential loss of life.

Similarly, suppose that after an examination at

Continue reading Day-trading in the age of coronavirus

Covid-19 and the worth of a human life

The scales of justice (courtesy Wikimedia)

Covid-19’s grim toll

The statistics are staggering: As of 1 June 2020, according to the Johns Hopkins University database, the U.S. had logged over 1.811 million confirmed cases of Covid-19 and over 105,000 deaths. The U.K. was next, with over 277,000 confirmed cases and over 38,000 deaths. Worldwide, over 6.3 million cases had been confirmed, with more than 376,000 deaths. If current trends continue, the U.S. death toll alone will soon exceed that of all wars in its history except for the Civil War and World War II.

The economic costs have been

Continue reading Covid-19 and the worth of a human life

Exit strategies for Covid-19

Marcos Lopez de Prado, the financial mathematician whom we have mentioned several times before in this venue (see, for example, Marcos Lopez de Prado testifies before U.S. Congress), has co-written an article entitled Exit Strategies for COVID-19: An Application of the K-SEIR Model. The article, actually a 52-page presentation file, is co-authored by Lopez de Prado (Cornell University) and Alexander Lipton (Hebrew University of Jerusalem and MIT).

In this presentation, the authors argue that universal lockdowns are blunt tools that should be employed only for brief periods of time. They then present a new mathematical model (K-SIER), which permits one

Continue reading Exit strategies for Covid-19