The ABS #censusfail: Gazing into chaos.

I would not trust today’s ABS with a pocket calculator let alone my own personal data given the way their biggest project to date has gone.

A big shout out to Patrick Gray from Risky Business for a bulk of the information that I based this post on.

Here’s what unfolded last week as Australians went to complete the Census on Tuesday, 9th August 2016.

censusfailMost of the points below were confirmed by industry sources and the public perception of the ABS census is severely tarnished. IBM has been put on notice by the Australian Treasury that compensation will be sought if IBM is found to be irresponsible or negligent, Turnbull has stated that “heads will roll” and privacy advocates are even more fervent in their opposition to the 2016 Census. It would be fair to appraise the exercise (if you will pardon my language) a shit-show.

  • Data stored at contractor (IBM) controlled facility was not encrypted to allow reloading of interrupted forms/sessions. Even though TLS was used to secure online forms, the data was stored on IBM servers in plain text to allow reloading of partially completed forms. A classic case of convenience at odds with security as a foreign company had access to unencrypted, personally identifiable data on incomplete submissions to save people from starting over if their connection is interrupted. This raises questions regarding the Patriot Act and another potential point of breach.
  • IBM/ABS failed to employ DDoS mitigation service (Cloudflare is a popular service for this) which could have been a simple oversight to an attempt to shave dollars of the project. In my opinion this is where things started to go superbly pear-shaped on Census night. At best this shows inept planning and at worst a blatant attempt by a techno-phobic bureaucrat deciding “we don’t need that” because they did not understand what DDoS was.
  • The system was serviced by a pair of routers acting to share the load between them. One of the routers did not have the configuration (rules) saved and rebooted into a state that did not match the previous setup resulting in the other router shouldering the entire load. This is where things turned into a real debacle and sent everything into a tailspin.
  • Intrusion detection services tripped with a false positive prompting ABS admins to shut down the service as a precaution. These messages were a result of normal use. The reactions by staff were not the bad part here but the intrusion detection systems really failed by false positives. Your system is only as good as the feedback it gives. If you are given incorrect data, you are going to make mistakes no matter how much of a gun operation you are.
  • Meanwhile ABS had an automated Twitter-bot auto-replying to tweets to the ABS twitter account to remind people to go online even though the service was offline. End users do not respond well to mixed messages and the #censusfail hashtag spun out of control.
  • There is no evidence to support claims of any significant DDoS or other denial of service attack. This is supported by DDoS monitoring tools. There were small DDoS attacks prior but the DDoS mitigation service (that was not implemented) would have easily dealt with these and not even registered as a blip on the radar. The Australian Signals Directorate has confirmed that no breach has occurred.
  • ABS did not allow use of VPN services to access the online census. This rules out the possibility of a foreign DDoS attack but also compromises security practices of visitors that would rather use a VPN to access a service meant to carry sensitive data. For the people that are worried about data security, this is not how you win allies.
  • Outage was poorly handled at all levels with speculation allowed to run wild and ABS seemed unwilling to accept Ockham’s Razor that the site was poorly implemented and was crushed under the weight of legitimate requests that would have peaked around the same time in each timezone (between 7 and 9PM). Simply stating that the service was unable to meet demand that exceeded expectations and that an investigation was underway although a bit of a canned response rolled out with every other technical hitch is a much better option than the creative approach taken on by the ABS PR team with blame ranging from foreign hackers to a Telstra failure.

If the data was anonomised from the beginning chances of this causing a negative PR impact would have been much less. People would have been more forgiving as the stakes would have been much lower. To keep personally identifiable information in the first big tilt at an online service was too much to ask of the public as well as the ABS.

This should not impact chances of electronic voting as that is anonomised and if approached as a secure distributed model should be a much simpler implementation based on existing technology.

This Censusfail will cost dearly to resolve and has the potential to delay the next census while the ABS recovers from this botched project. It is a political tool that holds great value to the government but the double edged sword that is online security we saw things spiral out of control to a point where the government of the day may be forced to amputate perfectly good sections of the public service while political players manage to protect their own interests.

I can’t help but wonder that this entire project saw smart IT and data security people would have voiced opposition to the way things were going and in some cases would have left the project in protest to avoid their reputation become tied to this shambolic nightmare. As a member of the tech community I would want to be as far away as possible from anything relating to ABS and the Census right now.

3 thoughts on “The ABS #censusfail: Gazing into chaos.

Comments are closed.