By Karl J. Paloucek
It’s so simple, that we often take it for granted. When we turn on the taps in our homes, our businesses, or anywhere, we’re certain that a rush of water
will flow forth, for the purpose we intend. As members of InfraGard, we like to think that we’re mindful of threats to every sector of our critical infrastructure, but water is so essential even to other sectors we aim to protect that we may not prioritize the safety of our water systems as highly as we should.
In fact, the under-prioritization of protecting our water was cited as a chief concern by the National Infrastructure Advisory Council (NIAC) as published in its report on water sector resilience this past June. Water was shown to be inadequately prioritized by public officials and sectors dependent on water for disaster planning, prevention and response, when in fact it’s an absolutely critical sector that needs protection.
Because of the lack of understanding by state and local leaders of the impact of an extended disruption to water service, water utility employees frequently lack the priority access they need to restore damaged assets in a disaster. Compounding the problem, while water plant operators and emergency personnel may be aware of how essential electricity, fuel for backup and transportation, and water treatment chemicals are, these dependencies may not be known to those in sectors responsible for supplying them. As a result, these dependencies may not be adequately addressed in exercises or response planning where supply chains are traced across dependent sectors.
But the news is not all discouraging. Earlier this year, the Indiana chapter of InfraGard helped to organize CritEx — a full-day tabletop exercise that took place at the Muscatatuck Urban Training Center. John Lucas, Sector Chief of Indiana’s Water Sector was instrumental in evolving this exercise and pulling together the people and resources necessary to replicate exactly what could happen in the event of a full-scale cyberattack on a water plant.
“It was similar to the attack that happened on the electric company in the Ukraine, in that we started out with an attack on the electric companies, and that over a period of about two days, it shut down the electric companies in the state of Indiana,” Lucas recalls. “We then started to work with the cascading effect of what would happen with, in this case, water? We had a group of electric companies who went through how they would handle recovery. We dealt with cross-sector issues, and a lot of the business recovery plans for electric — they assume they would be able to get into their buildings, but they hadn’t taken into account, in many cases, the fact of not having potable water; therefore some of their plans didn’t include port-a-potties, water trucks, or other things that they needed. So it was a great opportunity for people to look at their plans, their communications and all of that from a multi-sector perspective.”
Perhaps the greatest opportunity was in the Muscatatuck facility itself. Originally a state mental hospital, the location featured its own working water company, sewer plant and steam plant. “They had a sub station,” Lucas explains, “so they had electric, but everything else — they were a self-contained city.” When the facility closed and the state took it over, it became an urban training facility for military, SWAT and police. “We took it a step farther and said, ‘Why can’t it be a cyber facility, because it’s a self-contained city?’” he says.
This was the basis and the premise for CritEx — to use the onsite water plant to explore the attack strategies of potential threat actors and how they might conceivably disable it in spite of the operators’ best efforts. The goal was to educate the plant operators about the seriousness of the risks to their day-to-day operations, but also to alert those in the security community about the ease with which their own sectors could be compromised by taking this one vital link in the chain for granted.
“This was a working water company that produced 2 million gallons a day, peak output,” Lucas says. “So the water company and the water company SCADA (Supervisory Control And Data Acquisition) systems that operated it and the controls were consistent with a rural water company — one of 25,000 or more in the U.S. This was not a large municipal system. This represented what 850 water companies in Indiana look like. [The plant operators] would walk into the water plant, and they would go, ‘Ah — this looks just like mine!’”
Six different instances of attacks were explored during the event — not all of which can be discussed in these pages — but the dynamics and severity of the attacks were serious enough to get the attention of everyone observing these exercises, which included many of Indiana’s top utility companies, the FBI, DHS, FEMA — District V, and the Indiana Department of Homeland Security, which sponsored the tabletop exercise. Rockwell and Cisco also were in attendance. “We probably had 150 people who observed [the exercise] over the two days, and it was a phenomenal, mind-boggling, tiring, exhausting experience,” Lucas says. “But we actually did live attacks, compromised SCADA systems, broke through firewalls, got in and made changes. We actually created signatures that we provided the FBI, the CS-CERT, so that this type of attack couldn’t happen in the future.”
The first attack was of the honeypot variety. “We did it live,” Lucas explains. “Somebody went to what they thought was a real weather site, and was checking out the weather coming forward, and while they did that, they actually downloaded some malware. Then we exploited the malware and did a live pivot from the non-SCADA side, through a controller, through a router, into the SCADA side, and then at that point, took control of the SCADA system. So that one was live, looking at a SCADA system where they were separate from one to the other, but operating at the same network, but not on the same, say, subnetwork.”