How’s this for a headline: Dormant computer system awakens and causes major meltdown!
Sounds like a new sci-fi armageddon doomsday scenario film. Or the latest Official Excuse, like ‘leaves on the line’ and ‘the wrong kind of snow’.
However, perfectly seriously, it seems that a dormant system sprang to life and went on a share trading binge which clocked up over £280 million worth of debt in just 45 minutes for Knight Capital.
The US company was almost bankrupted in that short time, and in early August admitted to discovering the problem was on the New York Stock Exchange and was caused by ‘old software being accidentally reactivated when a new program was installed’.
Perhaps the idea of a terror film was not so wide of the mark. The baddie is down and everyone assumes he is dead, when suddenly he grabs the ankle of the innocent and unleashes a fresh wave of hideous torment.
Old Software Never Dies
In the same way, people assume that deactivated software is dead and gone. But clearly it isn’t. Just as computers never forget anything, so old software can still sting.
The US Securities and Exchange Commission controversially forced Knight Capital to honour most of the trades, even though they have been rescued by rivals (at a high price) and their share price has collapsed 75% so far.
For at least ten years the accumulation of dormant faults have been recognised as potential threats to any fault-intolerant system, especially where fault intolerance is based on the single-fault assumption. IEEE Xplore published in the early 2000s proposals for transparent online testing as fault dormancy ‘can degrade a system by several orders of magnitude’.
There is a growing market for devising the enabling of dormant computer hardware resources in computer systems. Customer requests for dormant hardware resources that they can install themselves are multiplying. They want order and payment facilities, contract terms, order fulfilment.
They want ordering through configuration rules, information management and interactive interfaces. But what they don’t want is dormant software that malfunctions.
To the non-technically minded user of computers, all that may start to sound a bit too much to follow. But to the suspicious minded, it offers a field day.
Why do corporate systems crash, freeze, leak or become confused? Why does an update release previous errors?
The answer is, according to conspiracy theories that somebody, somewhere stands to gain from it. It’s usually money they hope to acquire.
So, in the best traditions of detective films/fiction, follow the money trail back and there will be the driving force. Who would gain from a bank database, social security or tax system collapsing into gibberish?
Well paid experts called in to put it right?
Outright criminals/fraudsters who can syphon off data from a system that is failing?
But let’s go back to that Knight Capital business. Financial Adviser magazine shed some light on the mystery:
There has been no official explanation from the company, only from two people who spoke anonymously. An outside investigator is to be hired. Staff evidently searched eight sets of software before coming up with the dormant theory as new programs were installed.
The dormant system then created stock trades multiplied by a thousand, in effect ‘bombarding the market with unintended orders’ which drained its capital, necessitating a $400 million rescue that ceded most of the company to investors led by Jefferies Group Inc.
What is really confusing is the 45-minute delay in shutting down the malfunction. Experts believe such trading programs can be disabled instantly. So was it kept going deliberately?
Most observers confine themselves to asking about the ability of ‘arcane, outdated and nonstandard’ patchwork systems evolved over years to deliver in today’s high speed trading wars, given our over-reliance on computers nowadays.
Those who minded to fear the worst, think there may be more sinister explanations and questions. So, pick your viewpoint!
FA-Mag, 14 August 2012
Image: Chad fitz