Technical Deep Dive: Multi-core Industrial Controllers

  • April 11, 2016
  • Beckhoff Automation
  • Feature

By Bill Lydon, Editor,

Achieving Industry 4.0 and Industrial Internet of Things (IIoT) goals requires pushing more computer power to the edge of automation networks and today’s multicore processors provide a cost-effective way to accomplish this.  Multi-core processors have become ubiquitous in computers, tablets, and cell phones we use every day, and powerful industrial controllers sometimes referred to as Programmable Automation Controllers (PACs) are delivering high performance computing for industrial automation.   These new breeds of powerful industrial controllers are the platform for more powerful automation and control functions that have traditionally been supplied with middleware software. This includes real-time historians, optimization, predictive maintenance, analytics, web Servers, email clients and web services communicating directly with enterprise business systems.  These more powerful industrial controllers are becoming automation computing engines that are starting to collapse the classical 5 level manufacturing and process automation model. This is resulting in increased flexibility, responsiveness, and higher system reliability.

Exploiting Multicore Industrial Controllers

Simply using a multi-core processor to boost the performance of an industrial controller is like using a bigger hammer.  Fully exploiting this computing power requires a multi-core software architecture and user tools. There are only a handful of industrial automation vendors that have developed architectures to take advantage of this powerful technology.   

Beckhoff Automation Multi-core Industrial Controllers

Beckhoff Automation has been offering multi-core industrial controllers since 2006 with users benefiting from the advantages that this increased performance delivers.   Rather than “reinventing the wheel” Beckhoff Automation leverages the Microsoft Visual Studio® development platform.  I discussed multi-core technology with Beckhoff Automation’s Daymon Thompson, Automation Specialist.

Question: How long has Beckhoff Automation been providing multi-core controllers and what are the smallest and largest controllers with multi-core CPUs?

Beckhoff began offering industrial controllers with multiple cores starting with the Intel® Core 2 Duo™ chipset in 2006. With that chipset, TwinCAT, the dynamic automation software from Beckhoff, could handle demanding control tasks even more efficiently. Currently, the smallest multi-core PC-based controller from Beckhoff is the CX5100 series Embedded PC. The CX5130 variant has an E3827 Intel® Atom™ processor, operating at 1.75 GHz with 2 cores. This controller is suitable for compact motion control and PLC applications for machines and plants. The largest multi-core controller Beckhoff offers is actually in the new, high-performance control category, referred to as “many-core control”. This extremely powerful industrial server is equipped with two Intel® Xeon® processors (E5-2699 v3), which offer 2.3 GHz and 18 cores per processor, for a total of 36 available cores on one device. Many-core control is intended for control of machines with highly CPU-intensive computational algorithms such as condition monitoring, data analytics, or complex machine vision, or to control multiple machines, and even complete manufacturing lines in smart factory applications.

Question: Beckhoff people refer to embedded PCs, what does this mean?

True, “Embedded PC” (ePC) can refer to different device formats depending on the vendor. From the Beckhoff point of view, the Embedded PC is a DIN-rail mountable PC-based controller that offers a form-factor similar to a traditional PLC, but with far more power and flexibility. Embedded PCs from Beckhoff are oftentimes fanless, utilizing passive cooling methods and “mobile” versions of CPU chipsets as they are optimized to use less power and generate less heat than their equivalent desktop chipsets. The Embedded product range also all utilizes flash-based storage rather than rotating media such as traditional hard drives. Additionally, to increase industrial ruggedness, our Embedded PCs use a direct board-mounted RAM and CPU rather than the traditional socket and clip mounting system. This means the ePCs are better suited to industrial environments and are highly resistant to factors such as vibration. Specifically on fanless models (which describes most Beckhoff ePCs), there are no moving parts in the device, enabling a bare minimum of hardware maintenance. ePCs can also utilize Intel multi-core processors (even up to quad core Intel® Core™ i7) for computing power that was strictly in the range of large, cabinet-mounted IPCs just five years ago. 

Question: Simply having a multi-core processor for increased computing power and speed certainly can be an advantage but truly leveraging this technology can be challenging. How does Beckhoff deal with this?

Recognizing this challenge, Beckhoff developed TwinCAT 3 automation software that enables users to leverage the power of multi-core processors. The software provides a framework where each real-time control element becomes an independent software ‘runtime module’. These modules can contain a number of functions including a PLC controller, C++ project, motion controller, kinematic transformations, or even a MATLAB®/Simulink® module, compiled to run in one universal TwinCAT runtime. No distinction is made between representation of the modules as basic functions of an automation system, such as real-time tasks, fieldbus drivers or a PLC runtime system, or as user and application-specific algorithms for the control or regulation of a machine unit. Application complexity of these modules varies, therefore it makes no difference whether something like a small filter module is generated, or a complete PLC program is contained within a single module. In fact, without any additional action from the user, each PLC program is automatically packed into a module. PLCs can utilize all IEC 61131-3 languages, including all of the object-oriented extensions found in the 3rd edition of this standard. Implemented within the IEC 61131-3 programing languages, more industry standardization is available within the PLC, as is demonstrated by the availability of the PLCopen functions. Additionally, UML class diagrams can be used within the programming environment to build and show software engineering relationships between objects, as UML state chart diagrams can be used for logical state development and then compiled into runtime code.

Question: What is the universal TwinCAT runtime?

The universal TwinCAT runtime describes how this PC-based automation software can run across the Beckhoff product spectrum, from low end ARM-based controllers, up to the very top of the line many-core device. This ensures that TwinCAT users are not required to move to a completely different software platform if they use an Embedded PC on one machine, a cabinet-mounted Industrial PC on another machine, and a pole/arm-mounted Panel PC with integrated display on a third machine. Same goes for any combination of devices from our expansive I/O, drive and motor offerings. To us, this is the definition of “universal”. Of course, application and code execution needs should be taken into account when selecting controller hardware to ensure great machine performance. For example, we would never recommend you try implementing kinematics or CNC on the smallest controller in the lineup. Similarly, a quad-core Industrial PC is typically overkill for small, local control applications. However, these hardware selection considerations are trivial when compared with the real need to juggle completely different software platforms within another vendor’s system architecture when users must use different controllers for different machines, or migrate from one to another if system design changes necessitate it.

Question: Why is this approach required in a controller?

In order to design modern machines that are far more feature-filled, higher performance, and, at the same time, less expensive to engineer, the trend moves toward modular control software. Individual functions, assemblies, or machine units can be regarded as modules, which should each be as independent as possible and structured hierarchically. Best practices in programming format necessitate that the lowest modules in the hierarchy be the easiest to use, consisting of highly reusable basic elements. With the implementation of standardized interfaces, software modules can be combined with higher-level modules to create more complex machine units up to a complete machine controlled from a single CPU core. Ideally, the individual modules can be put into operation, extended, scaled and reused independently.

Question: How can the power of multicore be leveraged to the greatest advantage?

Individual TwinCAT 3 modules can be allocated to individual cores of a CPU. This is often referred to as “core isolation”. Multiple software-based PLCs can run on the same Industrial PC, along with motion control, with each function allocated to its own CPU core. Each of these PLCs is capable of cyclic updates for various code segments at many predetermined updates rates via configured ‘tasks’. CPU cores can also be ‘isolated’ from Windows to be fully utilized by the TwinCAT runtime. Multi-core, multi-task programming in a more traditional computer science framework can involve very complex code, paying special attention to maintaining data integrity of information shared between cores and tasks. With the open protocol and inherent message routing engine in TwinCAT software, the ‘automation device specification’ (ADS), linking data between modules and tasks from different cores becomes very easy for the programmer, as they are abstracted from the underlying complexities of multi-core programming. With this same abstraction philosophy, it is also very simple to select which cores to use for the real-time, and which tasks are assigned to which cores.

With these efficient engineering tools, the impressive performance of the newest multi-core Industrial and Embedded PCs can be fully leveraged.

Figure 1- TwinCAT 3 Multi-core Architecture: Multiple software-based PLCs can run on the same Industrial PC.

Question: How do all these software modules communicate with each other?

There are a few different methods for communicating between the modules. The first and simplest is via input and output mappings. In the same manner that PLC or C++ variables can be linked to input data from any I/O or fieldbus, one module’s variables can be linked to another module’s variables. This data is then updated automatically at the update rate of the module. The second method uses the Automation Device Specification (ADS) protocol developed by Beckhoff. ADS is a device- and fieldbus-independent protocol and establishes an interface to all Beckhoff runtimes and hardware. ADS is fully open and documented, with open sources drivers available for other operating systems such as Linux. Messages between objects are exchanged through a consistent ADS interface by the "message router". The message router is a "soft ADS router" that manages and distributes all the messages in the Beckhoff system and over other physical connections such as TCP/IP connections. TwinCAT message routers exist on every PC running TwinCAT and enables communication to every aspect of the TwinCAT architecture. The third and fourth methods include more the traditional Windows programming methodologies of using memory pointers or module interfaces. Data pointers consist of the modules sharing memory address information where then the modules can read values directly out of the other modules memory space. This permits immediate access to the values, but the user may also have to ensure data consistency. Interfaces, such as those known from the object-oriented programming (OOP) methodology, can also be created in each module. These interfaces can include methods that can be called from other modules.

Question: Did Beckhoff develop its own Integrated Development Environment?

One of the main approaches of TwinCAT 3 is to simplify the engineering of software. Instead of developing one’s own variation of standalone tools, it makes far more sense to integrate into common and existing software development environments. For TwinCAT 3, this is integrated into one of the most widely used development environments (IDEs) in the world, Microsoft Visual Studio®. By integrating TwinCAT 3 as an extension into Visual Studio®, we provide the user an expandable and forward-compatible automation programming platform.

There are several different possibilities that demonstrate how integration into Microsoft Visual Studio® can be done:

1.       If the traditional PLC programmer does not have full Microsoft Visual Studio® previously installed, the TwinCAT 3 setup automatically installs the necessary Visual Studio® Shell, in addition to PLC programming languages such as Structured Text, SFC, ladder logic, etc., as well as modules created for PLC runtime, NC, CNC or safety applications.

Figure 2- TwinCAT stand-alone tools

2.       If a full version of the Microsoft Visual Studio® is already installed on the engineering PC, the TwinCAT 3 extensions will be integrated into the existing Visual Studio® installation. With Visual Studio® installed, TwinCAT can now access the Microsoft C compiler through which real-time applications in C, C++ or MATLAB®/Simulink® can be programmed or generated. This is in addition to the previously mentioned classic PLC, fieldbus, and safety functions.

Figure 3- TwinCAT extended toolset within Microsoft Visual Studio®

Further examples of openness and the use of standards can be found in the C++ implementation. TwinCAT uses the standard Microsoft Visual Studio® C/C++ compiler, which is based on the standards – C: ISO/IEC 9899 and C++: IEC 14882. Although programming is accomplished through the Visual Studio® compiler, the C++ project remains within the TwinCAT project, and engineering efficiencies are again implemented in TwinCAT by extending the existing C++ debug functions. TwinCAT C++ provides helpful mechanisms for debugging the TwinCAT C++ modules running in a real-time context. Most of them are similar or comparable to more traditional C++ development; however, the domain of automation requires additional debugging mechanisms not traditionally found in real-time C++ programming.

The most common way to debug C++ programs is to set breakpoints and step through the code while reviewing variables, pointers, and other necessary components. TwinCAT 3 provides capabilities to set breakpoints and inspect the real time-executed code as part of the Visual Studio® debugging environment. However, during engineering and development of machines, it is not always reasonable to halt the system using a breakpoint, as this influences the behavior of the running machine. Also with most traditional PLC development environments, TwinCAT PLC projects provide online view and manipulation of variables during a RUN state without interrupting the real-time. This functionality has been implemented within TwinCAT C++ to provide similar functionality for reading and writing real-time values for C++ code via the TwinCAT “Live Watch” window. This is a unique and powerful feature of the real-time TwinCAT C++ offering.

Figure 4 - Common IDE for both PLC languages as well as real-time C++. Here on the left, a real-time C++ program, on the right, standard PLC programming with any IEC 61131-3 language, in this example, Function Block Diagram.

Question: Do applications on cores share physical network ports and can a port be dedicated to a core?

Ports on the Industrial PC (for example, a Network Interface Card – NIC) are better utilized by assigning the NIC to the user’s protocol or fieldbus of choice, rather than assigning the NIC to an individual core. In this manner, the communications protocol and fieldbus data can be shared among various modules (which may be running on different cores). As an example, a large machine might have complex motion algorithms and kinematics being calculated on core 1, while the PLC or C++ code runs on core 2. The motion controller will need cyclic data from the servo drives on the EtherCAT network, while the PLC will need cyclic updates from the I/O or other devices on that same EtherCAT network. The data being read and written via the fieldbus can therefore be linked to the particular resource that requires the data, regardless of which core it is running on, allowing a single network to achieve multi-core processing of the data.

This also abstracts the program code from the specific network being used. If an algorithm expects an analog value as an input, and a Boolean variable as an output, this code should be able to be reused on a future project that utilizes a different network protocol with different hardware, making it possible to supply the analog and Boolean values without having to change the real-time code. Programmers need only repoint the real-time program (i.e. PLC) to the appropriate network and data point from which to read and write the specific input and output.

Related Articles

Learn More

Did you enjoy this great article?

Check out our free e-newsletters to read more great articles..