- By John S. Rinaldi
- September 15, 2021
- Real Time Automation, Inc.
The search for reliable, practical information on building EtherNet/IP subnetworks on Ethernet for factory floor manufacturing systems is over. This article was originally published in the Automation 2021: Control Systems Ebook.
“I’ll never allow Ethernet onto our factory floor.” That was a common refrain for many factory managers in the earliest days of industrial Ethernet. Now it’s such a universal technology that we expect wireless Ethernet service in the same fashion that we expect water, electricity, and a public toilet. Not only is Ethernet pervasive in our homes and street corners, but it’s nearly universal in our factories with some believing that it should be the one and only factory floor communication media.
The most pervasive technology in North America for supporting factory floor manufacturing systems is EtherNet/IP. EtherNet/IP is an Ethernet application layer network technology promoted by ODVA. It is the foundational architecture for manufacturers in industries as diverse asautomotive, pharmaceuticals, and wastewater. In fact, it is hard to find an industry in North America that doesn’t rely on EtherNet/IP.
What’s been missing from the EtherNet/IP discussion is reliable and practical information on building EtherNet/IP subnetworks on Ethernet. This article attempts to fill that gap.
There is a context and a set of key principles that provide the foundation for the recommendations that follow:
- EtherNet/IP devices are certified by the ODVA to conform to both the CIP and EtherNet/IP specifications.
- EtherNet/IP networks mean EtherNet/IP networks. Blended networks, networks containing non-EtherNet/IP devices, are not EtherNet/IP networks.
- Blended networks are strongly discouraged for machine control systems. Adding non-EtherNet/IP devices to an EtherNet/IP network to create a blended network eventually leads to process issues no matter how much bandwidth is available on the network.
- Most of the traffic for devices in an EtherNet/IP control network must be EtherNet/IP control or traffic in support of the control traffic. Devices with no control traffic should not be connected to the control (EtherNet/IP) network.
- A CIP device is not necessarily an EtherNet/IP device. CIP messages can be transferred over CAN, USB, RS-232 serial, Bluetooth, and other communication mechanisms. The recommendations that follow refer specifically to EtherNet/IP devices using CIP explicit and implicit messaging in an EtherNet/IP network as defined above.
- To address security concerns, IP addresses for EtherNet/IP networks should be plant or company private.
Recommendation 1: Use fully switched, full-duplex Ethernet
Control system architects should ensure an EtherNet/IP network is a fully switched and full duplex network wherever and whenever possible.
Using a fully switched architecture and connecting EtherNet/ IP-capable end devices to Ethernet switches, gives an end device exclusive access to the rest of the EtherNet/IP network. Because control system messages must be delivered as quickly as possible, the switches supporting the EtherNet/IP-capable end devices shouldn’t be contending with each other when forwarding those messages along the path to their destinations.
Using full-duplex communications mode for each point-to-point link within a switched EtherNet/IP network doubles the potential bandwidth of a cable and eliminates the possibility of collisions on that link (Figure 1).
Consistency in the configuration of the device duplex settings is important. The devices at both ends of a point-to-point Ethernet network link need to use the same duplex settings to have the link operate in a full-duplex manner. Monitoring the operational behavior of the network interfaces of the devices is the best way to ensure Ethernet communication on each link is optimal. Many managed switches offer features that can detect and alert on collisions and/or duplex mismatches.
The primary exception to using fully switched and full-duplex behavior in an EtherNet/IP network arises when sending EtherNet/IP traffic across a wireless Ethernet portion of an EtherNet/IP network. Wireless media is inherently a shared communication media. Wireless transmissions are always potentially subject to remote interference.
Recommendation 2: Control system messages get priority, period
An EtherNet/IP network is a control system network with an underlying Ethernet network. The most important traffic it conveys is control signal traffic. Control traffic should pre-empt any non-control traffic. Non-control traffic—even network management traffic— should defer to control system traffic on a control system network. In a temporary congestion situation, control traffic should unconditionally be preferentially handled over any other traffic on the network. The potential impact of even momentary traffic congestion on an EtherNet/ IP network should be minimized. One way to do that is to prioritize the control signal traffic.
Most Ethernet managed switches have queues where messages are temporarily stored when an outgoing port is busy. There can be a number of these queues, some for high priority messages and other for lower priority messages. The priority queuing mechanisms they employ for each of the different queues are often configurable.
The priority of an Ethernet message is indicated by the value of the priority field in the Ethernet frame. There are eight possible priority values, with 0 indicating the lowest priority and 7 indicating the highest priority. These priorities are mapped to specific message queues. Multiple priorities are often mapped to the same queue.
Regardless of how many queues and queuing mechanisms are supported, every switch that supports prioritized transmission queues supports the ability to implement an exclusive priority mechanism. The traffic in the lowest priority transmission queue will only be sent if there is no traffic to send in any higher priority transmission queue for a port.
Two priority queues are sufficient for EtherNet/IP networks: high and low priority. EtherNet/IP implicit message traffic should get assigned to the highest priority queue. All other traffic should be assigned to the low priority queue. No exceptions.
End devices that properly implement the EtherNet/IP specifications are able to explicitly tag traffic with a priority value. For end devices that don’t support that feature, there is sophisticated infrastructure equipment able to preferentially process EtherNet/IP implicit message traffic. These devices can be configured to assign implicit traffic (UDP protocol traffic using port address 2222) to the high priority outgoing message transmission queue.
Recommendation 3: Use unicast communications
Architect EtherNet/IP networks to use unicast communications wherever and whenever possible.
The EtherNet/IP specification allow for both unicast and multicast delivery of “real-time” data, but it was several years into its initial deployment before EtherNet/IP programmable controllers fully embraced and implemented unicast delivery of real-time control and safety traffic.
Since then, only a small minority of applications on EtherNet/ IP networks are architected to use multicast communications. The advantage of multicast traffic is that it can be used to replicate traffic to tens, hundreds, or thousands of destinations. There are a minority of EtherNet/IP networks where this is desirable. Overall, EtherNet/IP multicast Ethernet communications should be discouraged due to its many disadvantages including extra complexity, more complicated and expensive infrastructure equipment, and the inability to be routed.
Today, two EtherNet/IP capable devices can exchange unicast I/O messages across different subnets using a line speed Layer 3 switch. These EtherNet/IP capable devices can exchange unicast I/O messages clear across a factory by routing that traffic over the company network just as if they belonged to the same EtherNet/IP network and were physically located adjacent to each other.
Routing of unicast messages is underappreciated. In fact, too few control engineers take advantage of the ability to route unicast messages (Figure 2).
Recommendation 4: Right-size your network
It is important to architect an EtherNet/IP network for the optimal size of the control system it supports. The broadcast domain for that network must not cover the entirety of the Ethernet network, but only cover the EtherNet/IP network. To do that, right-size your EtherNet/IP network.
An EtherNet/IP network is an IP sub-network superimposed upon an Ethernet network. The definition of what constitutes an Ethernet network has changed over time. The original Ethernet network transformed from a single cable network with a shared media access control mechanism to the advanced networks of today. Modern Ethernet network collision domains have shrunk to the size of a single cable connecting a device to a port on a switch. Full duplex operation on all the cables of a switched Ethernet network have eliminated the possibility of a collision on that Ethernet network.
It was the promise of a low-cost, high-speed and high-bandwidth, collision-free Ethernet network that led to the development of the EtherNet/IP specification. But beware, there is nothing in the EtherNet/ IP specification that prevents you from using it on a shared media Ethernet network. There is nothing in the EtherNet/IP specification that prevents you from putting non-EtherNet/IP-capable devices on an EtherNet/IP network. It is your responsibility as an EtherNet/IP network designer to ensure the Ethernet network foundation for your EtherNet/ IP-capable devices can support the control system applications you plan to run on them.
The EtherNet/IP specification is a tool. It’s up to you to ensure that the tool is used properly and safely.
In a programmable logic controller (PLC)-centric control system architecture, the best way right-size your EtherNet/IP network is to divide a potentially excessively large EtherNet/IP network into smaller networks at a point where the control signal interactions between networked devices only involve horizontal control signal communications (EtherNet/ IP scanner/adapter communications) between peer-to-peer controllers.
To right size your network, adopt the rule that a 254-node EtherNet/IP device capable network will be the largest EtherNet/IP network you will deploy. If you have an EtherNet/IP network with more than 220 devices, separate that network into two EtherNet/IP networks of 200 end devices or less. The technique used to divide an excessively large EtherNet/IP network into two smaller networks is the same one recommended for separating a large switch network into multiple switches: Cluster the devices into groups that minimize the need to exchange control signal traffic between them.
Recommendation 5: Define a well-architected address space
The day may come when it is possible and advantageous to provide a globally uniquely address for every device on an EtherNet/IP network, but presently, there is little to no advantage to do so. In fact, it is much more advantageous to use a private, segmented addressing scheme that can be easily understood, implemented, and maintained by the team members of your controls department (Figure 3).
In fact, that is the definition of a well-architected address space. It is an address space easily understood and explainable to plant team members. It is an address space easily implemented without extraordinarily complicated router and switch configurations. And it is an address space easily maintained by your controls department.
Architecting an address space for an EtherNet/IP network is both an extremely important and complex endeavor. A well-architected address scheme should meet the following conditions:
- It should take advantage of the company-wide private address space provided by IANA. Such an address space protects the EtherNet/ IP network from unwanted messages from external networks and external networks from unwanted EtherNet/IP messages.
- It should be consistent across the plant.
- It should take advantage of switches supporting static routing.
- It should be limited in scope. The number of devices in the network should be within the limits of what the least capable EtherNet/IP device can withstand in worst-case broadcast conditions.
- It should be consistent with the requirements of your IT organization.
These conditions are not absolutes. Designers can have a wellarchitected EtherNet/IP address space that, for specific technology or organization reasons, violates one or more of these tenets of good addressing. But, in general, these tenets are the foundations for easily understood, easily implemented, and well-maintained EtherNet/IP networks.
Consider a few of these conditions in more detail.
Use of private address space: Designers should choose an EtherNet/ IP address space that ensures a separation between the EtherNet/IP controls network and the corporate network and the Internet. This means taking advantage of the IANA non-routable, address ranges. Those ranges ensure your routers will not route any of these private addresses to a corporate network or the Internet and that no unwanted messages from the Internet will be routed to your control networks.
Use of static routing: Static routing and factory preconfigured I/O addresses (192.168.1.xxx) provide the designer with a host of advantages. This is especially true in situations where machines or machine sections are duplicated or PLC programs are often copied from one PLC to another. Operationally, it simplifies installation and maintenance for the trades people supporting the network as device replacement becomes a matter of setting three switches.
Consistency across the plant: It is advantageous to have specific address ranges for your switches, controllers, and I/O devices, and insist on consistency as new machines and devices are added to your plant floor. And it means all EtherNet/IP controllers use the same set of address ranges for device networks. Designers should duplicate EtherNet/IP I/O network addressing as much as possible across the plant. Addressing consistency of this type vastly improves understanding of the architecture by controls staff, enhances troubleshooting, and makes copying/pasting of PLC programs from one machine to another much less cumbersome and complicated.
Vendors of EtherNet/IP devices often advise users on building EtherNet/ IP control systems. While they know nearly everything about the EtherNet/IP specification and much about how messages transfer between EtherNet/IP scanners and adapters, they know little about building practical, reliable EtherNet/IP networks in the field.
This article proposes a small set of philosophies that can serve as guideposts for control engineers tasked with architecting their next EtherNet/IP system.
Claim your free copy of the book, “The Smart Product Manager’s Guide to Industrial Automation Connectivity,” by visiting this site. This book covers many of the topics in the article in much deeper details and helps define the many options to implement these technologies.
This article was originally published in the Automation 2021: Control Systems Ebook.
Did you enjoy this great article?
Check out our free e-newsletters to read more great articles..Subscribe