Connectivity & Cybersecurity Part 1 of 3: IT-OT Convergence

In their quest to optimize operations, manufacturing companies in recent years have begun discovering the value of operational technology (OT) data. When this data is accessed by information technology (IT) storage and computing resources, it can more effectively be transformed into valuable information. IT-OT convergence, flattened architectures, and machine-as-a-service concepts are worthy additions for businesses, but each of these industrial internet of things (IIoT) technologies introduces new cybersecurity concerns. Integration between IT and OT has been proceeding rapidly and accelerating for several years.

In this three-part series, we’ll examine connectivity and cybersecurity issues related to IT-OT convergence, an appropriate manufacturing plant security system, and the security aspects of interconnecting many different and distributed IIoT devices to achieve advanced functionality. As a collaborative effort to bring insights from different viewpoints, this article is written from the perspective of two experienced experts: one with over 26 years in IT security and the other with over 26 years in OT and industrial automation.

IT-OT convergence

IT-OT convergence is becoming a best practice in most manufacturing organizations because of the benefits found from accessing and acting on data from a variety of sources. It is implemented to:

• Provide ease of deployment and scalability when using new technologies.

• Enable new types of business processes.

• Increase organizational agility.

IT-OT convergence provides these and other benefits, but cybersecurity concerns must be addressed for OT assets. These assets often are vulnerable because of:

• Old operating systems.

• Lack of built-in security features in software.

• Complex topologies.

• Critical assets with built-in vendor support using virtual private networks (VPNs) to provide backdoor access.

• Insufficient inventory of existing assets.

• Inadequate details regarding where and how these assets are used.

New devices and business models introduced into manufacturing organizations—such as smart devices, cloud connectivity, control system remote access, and IIoT devices—add to the complexity. Personnel in many manufacturing organizations are often overwhelmed with the volume of security data, visibility gaps, and lack of staff with cybersecurity skills. Another challenge facing many manufacturing organizations relates to processes and production which can be stopped or paused rarely, if at all. Production downtime is often measured in a cost-per-minute loss of revenue.

For critical assets, the security team must be capable of identifying if even one device is infected, and then evaluate whether it involves a safety system that could put individuals or facilities at risk, or what kind of production impact it could cause. The team needs a procedure for quarantining infected devices until they can be safely and efficiently returned to the ecosystem.

The threat landscapes

Unfortunately, there have been many examples of security flaws in the OT world. The old adage stating a chain is only as strong as the weakest link holds painfully true for cybersecurity. In one case, a vendor supplying RS-485 to Ethernet converters that allowed OT assets to access the internet experienced a security breach with their devices. If a certain sequence of network bits were transmitted, it was possible to bypass authentication and provide unauthorized access to the OT assets. Another example occurs when third-party providers bring in unsecured assets, enabling rogue access points.

This means the asset is installed on a secure network without explicit authorization from the security team or network administrators. Unsecured assets installed without authorization on a manufacturing organization’s network can cause damage, production delays, monetary loss, and in some cases a safety hazard. In the OT environment, it is exceedingly common to bring in third-party assets such as programmable logic controllers (PLCs), human-machine interfaces (HMIs), cameras, or other specialized sensors or controllers used for production equipment. Almost all modern equipment assets come with some combination of digital controllers, networking, and/or embedded software.

Stack-based vulnerabilities are sometimes forgotten by vendors during quality assurance and system integration processes. Stacks are the locations in memory where functions are executed. Improper vender coding can allow for attackers to overflow buffers in the stack and take control of the application. For example, the Triton malware that shut down Saudi Arabia’s Red Sea Refinery targeted the code of the Triconex line of safety systems. The rogue code disabled safety systems designed to prevent industrial accidents, which could have resulted in an explosion from the high pressure and temperature operation.

There happened to be a flaw in the code that did not allow the perpetrator to remotely access the industrial assets, and instead a trigger was released from the Triconex safety systems to shut down the plant. These incidents are occurring at a time when industrial facilities are connecting many kinds of intelligent equipment and providing remote workers with access to industrial assets, highlighting the fact that cybersecurity must be paramount in IT-OT convergence.

Architectures

There are many architectures for improved protection of both IT and OT networks, with each based on the topography of corporate environment and the company’s security goals. Implementations to securely protect OT systems can vary widely based on the industry type, which may be as varied as pharmaceutical manufacturing, electrical grid utility operators, oil and gas platforms, and more. Cyber-physical systems encompass both the digital and physical world, and protecting cyber-physical systems while keeping production moving is the main architecture principle guiding a practical security plan. Organizations must find the right balance among the confidentiality, integrity, and availability (CIA) triad.

Availability is paramount in OT systems because an OT network outage has a direct impact on production and an immediate revenue impact. However, IT networking and security professionals have historically placed confidentiality and integrity as top network security requirements, with availability further down the requirements list. A converged IT-OT system demands a new look at the CIA goal. Security implementation plan requirements must evaluate the nuances between an IT network outage, which may only result in calls to a helpdesk, and an OT network outage, which can severely impact bottom line revenue. For this discussion, we will assume that IT networks are already protected against external OT-sourced threats.

Looking the other direction, several potential architectures exist to protect the OT network from the IT network:

Network barriers

Network barriers such as firewalls, implemented between the IT and OT networks, should be considered when designing a security architecture for a converged IT-OT network environment (Figure 1). Only those systems on the IT network side that manage the OT assets should be allowed access to the OT network. Protection provided by a network barrier confers many benefits, such as keeping the new security associate who runs a network test tool from inadvertently crashing the OT network environment.

VPN barriers

A more sophisticated network barrier option is a VPN between the IT and OT environments (Figure 2) . While VPN technology is typically used to secure access to the IT network from the internet, the same technology can be used to validate specific system privileges for allowing IT access to the OT network environment.

Intermediate servers

There are several other network barrier setup variations. Many use reverse proxy servers, with or without multifactor authentication, to protect very sensitive networks. Proxy servers replace perennially vulnerable vendor authentication systems. Proxy servers shield the full attack surface of vendor systems. When designing a security architecture to protect the OT environment, it is important to remember that several network barrier setups can be combined.

Static network protections should be considered for fixed assets, combined with a VPN or reverse proxy servers with authentication for access for mobile assets without a fixed IP address. Company laptops and remote administrators on mobile devices are prime examples of systems related to IT and OT, but without a fixed IP address.

Best practices

A key challenge is gaining visibility into OT networks without negatively impacting operations. Several best practices tools and methodologies have evolved since Alan Boulanger’s 1998 IBM Systems Journal ₁ article. However, his fundamental methodology regarding network intrusions still applies today, providing understanding of hackers’ mentality. In the IT-OT convergence, here are some recommended best practices for protecting against unwanted intrusions:

• Create and maintain a complete asset inventory of both hardware and software. OT legacy protocols and communication channels should be documented, and existing OT networks should be reviewed for vulnerabilities. Regular security audits should be conducted to eliminate vulnerabilities in the system. If the industrial asset—such  as an operating system or application software—can be safely updated, for example with a security patch, it should be on a regular basis, as with IT assets.

• Determine which processes cannot fail, specifically, those capable of causing safety, operational, or environmental damage. These should be protected by securing connections, monitoring for anomalies, and preparing incident responses. The philosophy should be zero trust access for the systems controlling these processes.

• Protect OT devices that cannot defend themselves or are unable to host advanced security controls.

• Regularly backup systems and assets to ensure quick recovery from ransomware and other attacks. Another strategy is to automate software deployment and updates with centralized, multi-tenant management.

• Use good password policies that are not easy to guess, weak, or hardcoded. Controlling the use of administrative privileges is also important. When reviewing password policy, it is important to also review the credentials of third-party software, backdoors in assets, and embedded credentials.

• Use security solutions for OT and IT endpoints

• Enable automated threat detection.

• Monitor all assets entering the network including third-party laptops, tablets, or mobile devices. These devices should have security services installed.

• Have an incident response plan in place. If a security breach occurs, an action plan needs to be in place to minimize damages as quickly as possible. Security and response plans should be continually updated based on intrusions, attacker’s activities, and other security breach tactics.

Many best practices resources are available, such as:

• The Center for Internet Security₂.

• Open Web Application Security Project (OWASP)₃.

• Cybersecurity & Infrastructure Security Agency₄.

It is also prudent to check notices from the ICS Threat Intelligence Reporting Service ₅ as they become available.

Confidently converging IT-OT

IT-OT convergence is fundamental for realizing many benefits in manufacturing organizations and for enabling digital transformation journeys. When utilizing a security plan or tool, it is important to prove the concept on the IT network where conditions may be more forgiving, before deploying to the OT network where uptime is paramount. Common sense informs us that an email system outage is a hassle, but a manufacturing environment failure can easily lead to lost revenue, wasted product, damaged equipment, or worse. IT-OT convergence security plans must keep in mind the specific needs for manufacturing systems. As organizations evolve into these new capabilities and business models, security must be at the forefront of all decision-making, along with alignment of IT and OT objectives.

See part two of this article series. See part three of this article series. Figures all courtesy of ADISRA.

References

1. Boulanger, A. “Catapults and Grappling Hooks: The Tools and Techniques of Information Warfare.” IBM Syst. J. 37 (1998): 106-114. https://www.semanticscholar.org/paper/Catapults-and-Grappling-Hooks%3A-The-Tools-and-of-Boulanger/02911e156feb1f973422f6079a64dfa94b717fda

2. The Center for Internet Security, Cybersecurity Best Practices. (n.d.). Retrieved January 28, 2021, from https://www.cisecurity.org/cybersecurity-best-practices/

3. Open Web Application Security Project (OWASP), OWASP Top Ten. (n.d.). Retrieved January 28, 2021, from https://owasp.org/www-project-top-ten/

4. Cybersecurity & Infrastructure Security Agency, Cybersecurity Best Practices for Industrial Control Systems. (n.d.). Retrieved January 28, 2021, from https://www.cisa.gov/publication/cybersecurity-best-practices-for-industrial-control-systems

5. ICS Cert Website: https://us-cert.cisa.gov/ics