Security Firms Scramble For SCADA Talent After Stuxnet

Three months after the world first learned of the sophisticated Stuxnet worm, insiders say that there’s a scramble to find and hire engineers with knowledge of both security and the industrial control systems that were Stuxnet’s intended target. 

Three months after the world first learned of the sophisticated Stuxnet worm, insiders say that there’s a scramble to find and hire engineers with knowledge of both security and the industrial control systems that were Stuxnet’s intended target. 

Anti virus companies admit their research teams were ill prepared for Stuxnet and are still coming up to speed on the functioning of Siemens industrial control systems and programmable logic controllers that Stuxnet infected. At the same time, the companies are searching high and low for technical talent with knowledge of the kinds of systems that run power plants, factories and industrial machinery — preparing for a future in which malicious hackers increasingly put critical infrastructure and an Internet of things in the crosshairs. 
Speaking at the Virus Bulletin Conference in Vancouver last week, Symantec researcher Liam O’Murchu said he and his colleagues had to teach themselves about both the Siemens SIMATIC programmable logic controllers targeted by Stuxnet worked in order to understand what it was the virus was doing. 
Reverse engineering new threats is nothing new for virus researchers, O’Murchu said, but Stuxnet’s focus on SCADA systems exposed a missing area of expertise on Symantec’s research staff, which is accustomed to analyzing malicious software designed to infect desktop computers, servers as well as mobile devices. 
“We realize we need new knowledge, but not new skills,” O’ Murchu said. “Its not like Stuxnet changes how AV researchers work, but new fields of expertise are needed. This is an area we’re not well equipped for.” 
In a lab that’s used to processing hundreds of thousands of new software threats a year, the analysis of Stuxnet is three months old – and counting, he said. 
Stuxnet has set other, major anti malware and security software firms scrambling for experienced engineers who are familiar with how SCADA and industrial control systems work.
“I’ve ran across a ton of people who talk the talk, but can’t walk the walk,” wrote a manager of product security at a major anti malware firm who asked not to be named because he had not been given permission to speak with the press. 
Engineers capable of programming PLCs and with a deep, technical understanding large scale SCADA networks are now in hot demand by security firm – even those without a firm grounding in computer security. 
If nothing else, Stuxnet proved that, as a practical matter the worlds of PC security and critical infrastructure security are now overlapping and comingled – not separate. 
“There’s no security in obscurity with Windows,” said Costin Raiu, Director of Global Research at Kaspersky Lab. 
“People are asking ‘Why would you run nuclear reactors on Windows?’ The answer is that they’re not running on Windows directly, but Windows PCs are, at some point, connected to the programmable logic controllers (that run the nuclear reactors). So we’ve got this complex Windows/machinery hybrid.” 
However, in the marketplace, the worlds of traditional network computing and industrial control systems areas are still separate. 
“There’s just not a lot of intersection between the security and the SCADA folks. You have folks who do network operations and understand how to do policy and operational security, and then you’ve got the types of people who design SCADA networks, and the two just don’t hang out that much,” said the manager of product security. 
That’s likely to change in the coming months and years, as  security firms scoop up top engineering talent with experience working on industrial control systems and as the companies that make the machinery and the software that runs it come to grips with the fact that they’re now in the crosshairs of organized criminal groups, international terrorists and state sponsored hackers. 
The lack of security readiness came to light early on in the Stuxnet saga, when it was revealed that the worm targeted a hard-coded password in the WinCC SCADA software – a serious breach of secure coding practice. Siemens pain was compounded when the company had to admit that it could not change or disable the password without adversely affecting the WinCC systems. (https://threatpost.com/more-secure-software-needed-utilities-nerc-cso-says-100710/)
“It’s like having a very valuable house with the door wide open,” said Raiu. “From the financial point of view, security companies realize there’s a good opportunity here and will be looking for experts, but the right way to handle the situation is to get Siemens and other firms like it to fix the underlying bugs,” he said. 
That’s a position that has been echoed recently by the top security officer at the North American Electric Reliability Corporation (NERC), who was quoted in a published report saying that the industry needs better and more secure development practices that produce more relient applications. (https://threatpost.com/more-secure-software-needed-utilities-nerc-cso-says-100710/)

Anti virus companies admit their research teams were ill prepared for Stuxnet and are still coming up to speed on the functioning of Siemens industrial control systems and programmable logic controllers that Stuxnet infected. At the same time, the companies are searching high and low for technical talent with knowledge of the kinds of systems that run power plants, factories and industrial machinery — preparing for a future in which malicious hackers increasingly put critical infrastructure and an Internet of things in the cross hairs. 

Speaking at the Virus Bulletin Conference in Vancouver last week, Symantec researcher Liam O’Murchu said he and his colleagues had to teach themselves about both the Siemens SIMATIC programmable logic controllers targeted by Stuxnet worked in order to understand what it was the virus was doing. 

Reverse engineering new threats is nothing new for virus researchers, O’Murchu said, but Stuxnet’s focus on SCADA systems exposed a missing area of expertise on Symantec’s research staff, which is accustomed to analyzing malicious software designed to infect desktop computers, servers as well as mobile devices. 

“We realize we need new knowledge, but not new skills,” O’ Murchu said. “Its not like Stuxnet changes how AV researchers work, but new fields of expertise are needed. This is an area we’re not well equipped for.” 

Symantec and other anti malware research labs process hundreds of thousands of unique malware a month, but the analysis of Stuxnet is three months old – and counting, he said. 

Stuxnet has set off a rush for engineering expertise that spans industrial control and automation systems and security, experts interviewed by Threatpost agree. The list of interested parties includes major anti malware and security software firms, as well as critical infrastructure firms and OEMs (original equipment manufacturers) that create the components that power power plants, refineries, water treatment plants and other critical infrastructure. 

“I’ve ran across a ton of people who talk the talk, but can’t walk the walk,” wrote a manager of product security at a major anti malware firm who asked not to be named because he had not been given permission to speak with the press. 

Engineers capable of programming PLCs and with a deep, technical understanding large scale SCADA networks are now in hot demand by security firm – even those without a firm grounding in computer security. 

“There’s definitely a shortage of people with the skills to understand security and process control and automation,” said Walter Sikora, Vice President of Security Solutions at Industrial Defender, Inc., a firm based in Foxborough, Massachusetts.

Sikora’s firm hires industrial control and automation engineers that act as security-focused consultants. It has been deluged with requests since Stuxnet was first identified in July from firms in the chemical, manufacturing, water, pipeline and power generation, as well as systems vendors, he said.

If nothing else, Stuxnet proved that, as a practical matter, the worlds of PC security and critical infrastructure security are now overlapping and comingled – not separate. 

“There’s no security in obscurity with Windows,” said Costin Raiu, Director of Global Research at Kaspersky Lab. “People are asking ‘Why would you run nuclear reactors on Windows?’ The answer is that they’re not running on Windows directly, but Windows PCs are, at some point, connected to the programmable logic controllers (that run the nuclear reactors). So we’ve got this complex Windows/machinery hybrid.” 

However, in the marketplace, the worlds of traditional network computing and industrial control systems areas are still separate. 

“There’s just not a lot of intersection between the security and the SCADA folks. You have folks who do network operations and understand how to do policy and operational security, and then you’ve got the types of people who design SCADA networks, and the two just don’t hang out that much,” said the manager of product security. 

That’s likely to change in the coming months and years, as  security firms look for top engineering talent with experience working on industrial control and automation systems and as the companies that make the machinery and the software that run those systems wrestle with how to create applications that are more resilient to attack and compromise. 

The lack of security readiness came to light early on in the Stuxnet saga, when it was revealed that the worm targeted a hard-coded password in the WinCC SCADA software – a serious breach of secure coding practice. Siemens pain was compounded when the company had to admit that it could not change or disable the password without adversely affecting the WinCC systems

“It’s like having a very valuable house with the door wide open,” said Raiu. “From the financial point of view, security companies realize there’s a good opportunity here and will be looking for experts, but the right way to handle the situation is to get Siemens and other firms like it to fix the underlying bugs,” he said. 

That’s a position that has been echoed recently by the top security officer th e North American Electric Reliability Corporation (NERC), who was quoted in a published report saying that the industry needs better and more secure development practices that produce more reliant applications.

However, experts like Sikora at Industrial Defender caution against expecting too much overlap – critical infrastructure verticals are still highly specialized, and the expertise needed to write applications for, say, energy generation can’t readily be transferred to water or chemical manufacturing, he notes. The systems and applications used in each vertical also differ from vendor to vendor.

Still, Sikora and others agree that Stuxnet was a turning point, demonstrating the need for industrial control and automation vendors to pay more attention to application security and resilience as well as protection against external threats that are now very real. 

“Stuxnet shows that its no longer just a Windows, Apple or Adobe thing. It can happen – its been shown. Now all those things we didn’t want to have to do on control systems we have to do and there’s a lot of work to be done,” Sikora said.

Suggested articles

Discussion

  • n3td3v Security on

    "Motivation behind Stuxnet." BP lobbied for the release of the Lockerbie bomber, and the people responsible for Stuxnet wanted to make sure they paid. To make sure the oil deal from releasing the bomber, BP couldn't make a profit from. Stuxnet targeted the oil well. There were a lot of unhappy people after the release of Abdelbaset Ali al-Megrahi. Abdelbaset Ali al-Megrahi was convicted for blowing up Pan Am Flight 103 over Lockerbie, Scotland, on December, 21, 1988. He was freed on compassionate grounds by the Scottish government on August, 20, 2009. The claim was he had terminal prostate cancer and was expected to have less than three months to live. It was a lie and he is still alive living the life of riley in Libya. Originally posted by me at http://www.schneier.com/blog/archives/2010/10/stuxnet.html#c467887
  • n3td3v Security on

    I'm just an asshole who likes to pretend that I know things. In reality? I don't.

  • n3td3v Security on

    Actually, I'm a clueless retard.  I can't back up anything that I say, and continue to post mindless rants all over the internet without any facts.

  • Anonymous on

    Actually, I have been an asshat all of my life, it is a terminal condition.

  • JoeFam on

    "I've ran across a ton of people who talk the talk, but can't walk the walk," wrote a manager of product security at a major anti malware firm who asked not to be named because he had not been given permission to speak with the press. This is a quote from above. The past participle should be: "I have run." No big deal, right? Well, maybe. But if one cannot master the American (note, not English) language, one's other skills should come into question...
  • n3td3v Security on

    Re: the fake comments above

    It was getting close to our budget review for defence cuts, Stuxnet was convenient and easy and the Israeli's get the blame.

    This is what Stuxnet was all about for us https://sites.google.com/site/n3td3v/latest/defencecutscouldharmcybercapability

    Andrew

  • csworks on

    This move from security companies makes sense. For a long time, SCADA admins and developers pretended to live in a parallel universe and had a luxury to be delusional about "air gaps" between control system and SCADA, and to practice "security by obscurity". Not anymore. Stuxnet was a wake up call for the industrial automation industry - it's time to look around and start leveraging the experience that humankind has accumulated fighting security threats in the last decades. The following article gives some recommendations based on the excellent Stuxnet analysis paper from Symantec:

    http://www.controlsystemworks.com/blogengine/post/Post-Stuxnet-industrial-automation-systems.aspx

     

  • AnonymousDCS on

    The issue with "the company had to admit that it could not change or disable the password without adversely affecting the WinCC systems. " is that this is a major supplier of critical infrastructure that has hashed together purchased pieces for a working, supportable system that has fundamental holes.  

    The USA pharma industry has insisted on better core practices from the control system suppliers that work with them.  So there are systems with better core features to support best practices, but it is still work to reliably implement best practices reliably.  

    I do not find public documentation of these core features,  just indirect mentions of why come to private conferences where discussions reference support materials accessible by customers with support relationships with the supplier.   

Subscribe to our newsletter, Threatpost Today!

Get the latest breaking news delivered daily to your inbox.