...but luckily he's overstretched and has underestimated the job of keeping track of us all
By Phil Hendren
27/05/08 "The Times" -- 22/05/08 - -The Government is planning to introduce a giant database that will hold the details of every phone call we have made, every e-mail we have sent and every webpage we have visited in the past 12 months. This is needed to fight crime and terrorism, the Government claims.
The Orwellian nature of this proposal cannot be overstated. However, there is one saving grace for people who fear for their civil liberties. The probability of the project ever seeing the light of day is close to zero. This proposal - like so many grandiose government IT schemes before it - is technologically unfeasible.
The current levels of traffic on the internet alone (including e-mail) would require storage volumes of astronomical proportions - and internet use by the public is still growing rapidly. Meanwhile, the necessary processing capabilities to handle such a relentless torrent of information do not bear thinking about. Modern computer processors are fast, but writing data to disks will always be a serious bottleneck.
Take a quick sample from the London Internet Exchange, the UK's hub and one of world's largest points at which each ISP exchanges traffic. Yearly LINX carries at the very least 365 petabytes of data - that is the equivalent of the contents of about 26 million iPod Nanos that have the capacity to hold nearly 2,000 songs each. There is no commercial technology that is capable of writing at those kinds of speeds.
It's not just writing that would be problematic, but the reading of the data too. It would be immensely difficult to pinpoint in such a massive database an e-mail sent by a particular person at a particular time.
It's all too familiar in large-scale government projects that the technological expectations of civil servants gallop far ahead of reality. The Ministry of Defence's requirements for the Nimrod radar project was a classic example of overspecification. The result was a system that was unable to process data because the technology Whitehall assumed would exist in the future, when the planes would finally take to the skies, simply never materialised. The planes, after hundreds of millions were spent, had to revert to the traditional Awacs system instead. The men who gave us the new NHS database, likewise, severely underestimated operational realities.
The good news is that we will not be robbed of our privacy by this latest database because it will remain just a pipedream. We taxpayers will, however, be robbed of billions of pounds as the IT consultancies draw up their bids to design and deliver the undeliverable.
Phil Hendren is a Unix systems administrator. He blogs at www.dizzythinks.net