Lessons from the “Flash Crash” regulatory fiasco

On April 21, 2015, the U.S. Department of Justice announced that it would press criminal charges against Mr. Navinder Singh Sarao, a 36-year-old small-time British day-trader. He is being blamed for nothing less than causing the “Flash Crash” of May 6, 2010, the second largest point swing (1010.14 points) and the biggest one-day point decline (998.5 points) in the history of the Dow Jones Industrial Average. This shocking revelation is just the most recent turn in a surreal story that exposes the technical inadequacy of present-day market regulatory agencies.

A bit of history

Following that infamous day, observers were quick to offer explanations for the flash crash, from a “fat-finger trade” in the “Procter and Gamble” stock, changes in the U.S. Dollar/Japanese Yen exchange rate, to a large purchase of put options by the hedge fund “Universa Investments.” On September 30, 2010, after almost 5 months of analyses, the CFTC-SEC issued its final official report of the crash. This report blamed the crash on a low-frequency trader (p.2):

At 2:32 p.m., against this backdrop of unusually high volatility and thinning liquidity, a large fundamental trader (a mutual fund complex) initiated a sell program to sell a total of 75,000 E-Mini contracts (valued at approximately $4.1 billion) as a hedge to an existing equity position.

The report rightly pointed that the “flash crash” was the result of a liquidity crisis. However, it made the mistake of blaming a low-frequency trader for causing that crisis. On November 19, 2010, a new study by Easley, Lopez de Prado and O’Hara (published in the Journal of Portfolio Management) explained that liquidity crises are a structural part of our new, high frequency world. A mutual fund was not the culprit. The market structure, sanctioned by the 2007 RegNMS legislation, set the conditions for an accident to happen. In particular, this Easley et al. study showed how order imbalance reached toxic levels one hour before the crash.

Order Imbalance (dotted red line) reached toxic levels one hour before the crash. See video here

 

It was order imbalance…

To understand the crash, we have to explain how modern markets work:

1. A transaction occurs when two opposite sides (a buyer and a seller) agree on a price and an amount.
2. One side provides liquidity and the other takes liquidity. The liquidity provider posts an order, and waits for a liquidity taker to match it.
3. Liquidity providers are “sitting ducks.” They are afraid of offering too many securities at a wrong price (selling too low or buying too high). They avoid this by monitoring order imbalance. Sellers will provide liquidity at a higher price if they see that buying orders are more prevalent than selling orders. Buyer will provide liquidity at a lower price if they see that selling orders are more prevalent than buying orders.

These operations were done mostly by humans until around 2008, when High Frequency Trading (HFT) became the most common way of transacting. HFT is the rapid exchange of securities enabled by sophisticated computer algorithms and advanced communications technology. Unlike humans, algorithms operate in a volume clock. This means that rather than analyzing order imbalance every hour, or every minute, HFT algorithms analyze order imbalance in volume buckets. When activity spikes, they will make even more decisions in less time, which will trigger even more activity.

This sort of snowballing effect is characteristic of HFT’s volume clock paradigm. Spoofers know this and exploit it to their advantage. A “spoofer” is a trader that generates one-sided orders in high volume with the intention of scaring liquidity providers of the opposite side.

The SEC-CFTC finally see the light

Every financial outlet seemed shocked last Tuesday (April 21, 2015) to hear the new explanation pursued by our regulators. Here is a Reuters article on the announcement (underline is ours):

“His conduct was at least significantly responsible for the order imbalance that in turn was one of the conditions that led to the flash crash,” said Aitan Goelman, head of enforcement at the Commodity Futures Trading Commission, which filed parallel civil charges against Sarao on Tuesday.

Yes, you read the Reuters article well. There is no mention of a 75,000-contracts sell order originated from a mutual fund, just as if that never happened, similar to the fate of Nikolai Yezhov, who, after his fall from power under Stalin, was executed and excised from public photos and records (see above). Instead, all of the sudden, Regulators have admitted that the Flash Crash was due to order imbalance, as Easley et al. had explained almost five years earlier. Here is a quote from a Bloomberg News article:

Tuesday’s developments fly in the face of the prevailing narratives of what happened. Regulators initially concluded that a mutual fund company — said to be Waddell and Reed Financial Inc. of Overland Park, Kansas — played a leading role. Many in the industry countered that a confluence of several forces, including high-frequency trading, was probably behind the crash.

Why did it take five years?

As evidenced by the work of Easley et al., the conflation of HFT, order imbalance and spoofing was known five years ago. Then, why did it take five years to find Mr. Sarao? Two reasons.

First, Easley et al. pointed out the mechanism of the crash. However, they could not pursue the particular actors, because that would have required access to FIX-Tag-50 data. This is the data that identifies the person that generated each order. Section 8 of the Commodity Exchange Act bars the CFTC from disclosing this data.

Second, regulators appear to have been extremely satisfied with the official culprit: A mutual fund. The alternative explanation, that the post-RegNMS market structure exacerbates snowballing, was more inconvenient. Regulators preside over this market structure, and blaming a system prone to failures makes them partly responsible.

How did the whistleblower gain access to this confidential Section 8 data? This data appears to have been leaked in the past, and perhaps this was the source used by the whistleblower. What it is clear is that the SEC and CFTC did a poor job. In the words of a Bloomberg report:

 [Regulators’] analysis was upended Tuesday with the arrest of Navinder Singh Sarao — a U.K.-based trader accused by U.S. authorities of abusive algorithmic trading dating back to 2009. Even that action was spurred not by regulators’ own analysis but by that of a whistle-blower who studied the crash, according to Shayne Stevenson, a Seattle lawyer representing the person who reported the conduct. The episode shows fundamental cracks in the way some of the world’s most important markets are regulated, from the exchanges that get to police themselves to government agencies that complain they don’t have adequate resources to do their jobs.

Is Sarao responsible for the flash crash?

This is, of course, a pointless question. Proving Mr. Sarao’s guilt would require replaying the events of May 6, 2010 in absence of Sarao’s spoofing, only to find that no crash occurred. This is an exercise of fiction. And even if that was possible, it would only prove what we already know: The current market structure is an accident waiting to happen. We can blame Mr. Sarao as much as we could blame a massive accident on a Mini-Cooper’s reckless driver crashing on a road poorly maintained by the Department of Transportation. Yes, someone triggered the crash, and that unfortunate random individual may have been Mr. Sarao.

The real villain

The real villain of this story is Section 8 of the Commodity Exchange Act. If this data had been made available, spoofers would have been outed and charged back in 2010. Make no mistake: It is this law that protects spoofers, and should be blamed for the next flash crash. Any experienced data scientist would have found Mr. Sarao in minutes. His footprint was unmissable to trained eyes. Think of it, 99% cancellation rates from the same FIX-Tag-50, and regulators didn’t see that? It takes a simple query of sell orders, grouped by FIX-Tag-50, to solve this case. After all, we are talking about a ridiculously small dataset, compared to the datasets used in Big Data analyses every day, which include trillions of records.

Lessons for the next flash crash

There are several lessons to be learned:

1. The market structure is more complex now than it was five years ago. Regulators do not appear to be willing to discuss the actual root of the problem. More flash crashes have and will continue to occur.
2. The SEC-CFTC explain that it took five years because of the magnitude of the data. Such comments expose a certain naivety regarding data: a few billion records is small data. The problem is, the regulatory agencies are not staffed with data scientists but instead are populated by Excel users (at best!). This is not a matter of budget or resources. All it took to catch Sarao was one competent person with access to confidential data. Perhaps the SEC-CFTC should transform a lawyer-centered agency into something closer to the NSA. Or they could simply get out of the way and let private citizens do the research for them, by divulging the Section 8 data.
3. Since regulators failed to identify someone as careless as Mr. Sarao, most likely they will not be more successful in chasing the real pros working at cutting-edge HFT firms. There are thousands of spoofers out there, far more sophisticated than Sarao. Perhaps regulators do not go after high-frequency firms, because billionaires are a tougher fight than a 36-year-old small-time British day-trader.