Database State - Executive Summary

This significant report was published in 2009. It reviews 46 databases used across government and observes that:

 "All of these systems had a rationale and purpose. But this report shows how, in too many cases, the public are neither served nor protected by the increasingly complex and intrusive holdings of personal information invading every aspect of our lives."

Of course since this was published Edward Snowden's revelations have been revealed to a surprisingly unknowing citizenery.  But it is worth relooking at the findings by Ross Anderson and his colleagues in the 2009 report in the light of the Snowden revelations and my earlier posting about the NHS centralised data opt-out possibility. They found that:

• A quarter of the public-sector databases reviewed are almost certainly illegal under human rights or data protection law; they should be scrapped or substantially redesigned. More than half have significant problems with privacy or effectiveness and could fall foul of a legal challenge. 
• Fewer than 15% of the public databases assessed in this report are effective, proportionate and necessary, with a proper legal basis for any privacy intrusions. Even so, some of them still have operational problems. 
• Britain is out of line with other developed countries, where records on sensitive matters like
  healthcare and social services are held locally. In Britain, data is increasingly centralised, and shared between health and social services, the police, schools, local government and the taxman.
• The benefits claimed for data sharing are often illusory. Sharing can harm the vulnerable, not least by leading to discrimination and stigmatisation. 
• The UK public sector spends over £16 billion a year on IT. Over £100 billion in spending is planned for the next five years, and even the Government cannot provide an accurate figure for cost of its ‘Transformational Government’ programme. Yet only about 30% of government IT projects succeed.

So as to better appreciate the extent of on-going systemic failure in this field it is worth reading Geoffrey Sampson's article:

Whistleblowing for health
Geoffrey Sampson
The University of South Africa
This paper offers a concise account of what was possibly the largest-scale failure ever in the
history of software engineering, namely the British National Health Service “Connecting for
Health” project. The failure offers important lessons about the rôle of professional expertise
and the impotence of authority in the software development process.

Word has it that the English government is at long last about to scrap the NPfIT.