Crop Circles in Cyberspace
© Jeffrey DonovanThe trading pattern identified by software engineer Jeffrey Donovan as "Mountain Range"

On 6 May this year, the financial markets freaked. The Dow Jones index (which tracks the overall value of key stocks on the US market) crashed by 1,000 points in just a few minutes, creating what has become known as the "Flash Crash" and wiping billions of dollars off the value of the New York stock market. Now most people's picture of a stock exchange involves a bunch of pushy geezers in red braces shouting and waving their arms about, and while that might have been the case in the 1980s, today's markets are very different. These days, stock markets are high tech and virtual. Trading is carried out by carefully-honed, esoteric computer programs that can make thousands of trades a second using automated bots. Finance companies vie to locate their server farms as close to the trading floors as possible because these things trade so fast that even the speed of light (at which the data moves through the cables) is a factor in the calculations as to whether a successful trade is pulled off or not. It's no longer the Gordon Geckos of this world who make the running, but PhD physicists who have defected to hedge funds, where their skill at creating algorithms to mimic physical processes is put to work modell­ing financial markets in order to pull off these light-speed trades.

Because of this, various companies have grown up with that record and analyse the digital traffic of the trades in order to better understand how the market is working, and when one of these, a data services company called Nanex, turned its attention to the 6 May "Flash Crash", they turned up something very weird indeed. Normally, trades are looked at on a minute-by-minute basis, but Nanex's Software Engineer Jeffrey Donovan decided it might be instructive to bore down into the data and look at what was happening on a millisecond scale - and what he found stopped him in his tracks. A pattern was visible, which Donovan dubbed 'The Knife'.

This was clearly a pattern generated by a trading algorithm, but not one that seemed to make any sense in terms of actual trading. So he looked further, and the more he looked, the more of these strange millisecond scale patterns he found. According to Donovan, "we probably get 10 stocks in any 10 minutes where we see something like this."[1] So he started collecting them, gathering them together on his webpage.[2]

Essentially, what is supposed to happen is that people offer to sell shares at a certain price while others indicate that they want to buy them at a certain price, and the traders match these up and try to buy cheaper than they sell, creaming off a profit. To do this, quotes are sought from various sources, and the algorithms are designed to seek these, match clients and help execute profitable trades; all of which happens in a fraction of a second via these automated systems. In the case of these mysterious patt­erns, what is happening is that the algorithms are bombarding the system with demands for quotes, sending the exchanges thousands of orders a second with no intent of actually trading, often with prices so awry from market reality there is no possibility of them being part of a transaction, and they are doing it on a time-scale that means their activity would never show up in any normal market analysis. For example, there's the 'Blue Bandsaw' (a burst of 10,000 oscillating priced quotes asked for over a 12-second period), the 'Ask Mountain' (56,000 quotes in 10 seconds, all with the same ask price and the ask size increasing or decreasing by one) and the 'BATS 60-Step' (which takes 60 steps up at a penny a time, then one step down to the original level, resets and does it all over again, 700 times per second). And so on - there are literally dozens of examples.

What isn't clear, though, is why these things are there and what, if anything, they are meant to do. The process that they carry out has been dubbed 'Quote Stuffing' and Nanex's report on the Flash Crash seems to think this might have had something to do with the events of 6 May; whether or not this is the case is far from clear, and even if it was, was this intentional or accidental?

There are several theories as to what these things might be and why they are appearing in trading patterns. When Donovan first spotted them, he assumed they were being generated deliberately, that they were a new kind of trading algorithm, and that their essential purpose was to spoof the market in some way. Potentially, a trader deploying a bot that generated these nonsensical demands could use them to fractionally slow down the programs of rival traders. If they programmed their own systems to ignore the bot's behaviour, dealing with its actions could take up rivals' processing time, while they got on with the trades, which could, potentially, give them a fractional advantage in a market where even light speed counts. More paranoid commentators have considered the possibility that they might be external attacks on the market, designed to increase instability and foster events like the Flash Crash, in which case their source could perhaps be Russia, or more likely, China, given that it is known to be actively attempting to compromise the US digital infrastructure. China allegedly already has the ability to fatally collapse the USA's electricity grid using trojan horses and viruses already indelibly embedded in the system, so it seems plausible that it would want to infiltrate the stock markets as well. However, there's not a shred of evidence that these patterns are the result of cyberwarfare, so this theory has to be consigned to the outer limits of probability as a cause.

Most intriguing, though, is the idea that no one is responsible - that instead, the systems are doing it for themselves. The financial markets' digital systems are now so complex, and the number of organisations using bespoke trading bots so huge, that the interaction between the programs could well be generating emergent phenomena spontaneously. That is, these patterns are just weird side effects that come about in much the same way that fract­als come about as a result of chaotic phenomena, emerging fully formed from the ether with no design or overall purpose. But they are nonetheless there, asking for quotes and behaving in ways that every so often have an influence on trading. New behaviours could be evolving from the combination of, and interaction between, the behaviours that the programmers intended. The question remains, though: how long has this been going on? After all, no one thought to look before May of this year. There already seems to be a huge amount of this kind of activity taking place, and as the markets increase in size and complexity, how will the behaviour of these patterns, now dubbed 'Algorithmic Crop Circles', develop? And what will the outcome be? Will they eventually disrupt electronic trading to the point where it is no longer viable? Will they remain creatures of the financial undergrowth, lurking out of sight and rarely interacting with the real world? Or perhaps the rolling financial crisis will eventually render them extinct? It's not often you get a completely new phenomenon emerging, and I wouldn't say that the financial markets are the first place I'd have looked for one. It'll be interesting to see how this one shapes up.

With thanks to @GreatDismal, whose tweets put me on the trail.

Notes
1. The Atlantic, 4 Aug 2010.
2. 'Crop Circle of the Day', Nanex.