Solving Communication Roadblocks with Immutable Data Records

Summary

Companies in industries ranging from manufacturing to oil and gas often face the same problem–they need to transfer data between two components, and they need those data points to both stay together as a group and remain unchanged. When assets are distributed, communications are often inefficient and slow. The polling process for remote assets can take hours, offering only a few snapshots of how the assets are performing throughout the day and making it impossible to receive continuous feedback from the field.
 
For years data has been created in PLCs, RTUs and HMIs that represent a record of information. With the limitations of register-based communication protocols, this information has been clumsily handled as consecutive blocks of registers. This method runs the risk of not keeping all the data together as a record event when a partial subset of the registers gets overwritten with a newer set of information.
 
Today with the power of MQTT, records of data can be published, on event, as an immutable data record which automatically builds and populates a table within a database. An immutable data record is a set of values that has a security checksum wrapped around it for protection and validation. The value of an immutable data record is defined when it is created, and the security checksum can verify it has not been modified.
 
Cirrus Link has added the definition of DRecords for MQTT Sparkplug, an immutable data record to transfer which can be audited by an authorized consumer as unchanged from source. The value is these records can be sent securely, contained as one snapshot, and given a timestamp and an object for validation.
 
The DRecords definition is especially unique since it is open source. Companies who have solved their communication problems with immutable data records usually do it with a lot of custom code and proprietary technologies. Protocols don’t typically have data objects, so enabling open source DRecords for MQTT Sparkplug is a game-changer for the industry.
 
DRecords enabled by MQTT Sparkplug can be used to improve asset communications in the following industries:

• Oil & Gas: EFM Quantitative Transaction Records, EFM Alarms, EFM Events, pump cards for pump-off controllers

• Pharmaceutical: Batch records for drug manufacturing

• Transportation: Locomotive engine management

• Any industry: Ticket information to record a transaction event

 
The technology will be added to the Sparkplug Eclipse Tahu project that defines how to use MQTT in a mission-critical, real-time environment. The DRecord definition is so new it hasn’t been added yet. This paper will explain the benefits of DRecords exposed by MQTT Sparkplug as we work to get it included in the official open-source specification.
 

Oil and gas company enables DRecords to monitor pumps

Let’s look at an example of how MQTT Sparkplug, and specifically DRecords, helped one company solve their communications bottleneck.  Cirrus Link recently worked with this oil exploration and production company to help them more efficiently gather data from oil wells in the field – however the solution can be applied in other industries to help anyone collect asset data to create a real-time picture of operations.
 
In this use case, the company has 10,000 oil wells in the field, each with a pump-off controller. Pump-off controllers create P-cards measuring various data points at the well for how efficiently the pump is working. The operations team has asked the company’s lead information analyst to find a way to do chemical surveillance on the wells. The company injects a chemical into each well with every stroke to inhibit corrosion in the pump while it goes up and down. Sometimes the pump gets clogged, sometimes someone doesn’t come by to fill up the chemical.  Either way, they need to monitor the level of chemical in the tank for maintenance and operations.
 
The request from operations to monitor the chemical exposed the actual larger goal–to figure out how to get secure communications from the edge into their on-premise operations systems in real-time. As they tried to do the chemical surveillance with their legacy infrastructure, they quickly realized that although they had a poll/response, serial communications system for collecting data, they couldn’t get the information in real-time and they were always worrying about someone being able to access the data.
 
The customer was reading a P-card once or twice a day with 400 plus data points for each pump stroke, which honestly was not doing anything for them.  All they knew was the well was still there and the rod didn’t fall off the pump. However, with MQTT and DRecords we showed them how they could send all 400 data points as one immutable object directly into a database to create a far more complete and real-time data picture.
 
The work we did with DRecords allowed the customer to package up one P-card with all 400 data points and publish that one record to their enterprise service bus hub so it is then available to operations or anyone in the enterprise. Instead of publishing hundreds of tags, they packaged them and sent one DRecord.  They also published other well measurements upon change instead of wasting bandwidth by polling continuously.
 
To solve the original problem of monitoring the chemical tank we took this project one step further by allowing the customer to include data as a picture or short video as well. They can take a picture of the chemical tank. If the file is too large for transmission the system will break it up into smaller messages to send to the MQTT broker, then on the other side it rebuilds those and puts them into a file location as shown in Figure 1. So, without having to compromise security, they access the highest level of MQTT security and lockdown the HTTP piece so no one has access, and can still get a short video from the edge to their on-premise systems. As a result, they can use those wellsite pictures in a machine learning algorithm to improve performance and maintenance.

Cost is an important consideration–so with this project we are working to get the price point for the 10,000 wells to around $2000 for a radio, transmitters on the chemical tank, temperature probe on carrier line, hardware and the MQTT solution. With thousands of distributed assets, the combination of great cutting-edge technology with a reasonable cost of ownership is key.
 

How to implement MQTT Sparkplug DRecords

DRecords is something the industry has never been able to do–get one single record per stroke. Consider the implications for any industry. DRecords can provide real-time data from edge assets to on-premise operations systems, create a database with a complete data picture, and ultimately improve efficiency and operations. There is a huge return on investment to be gained from being able to keep data together to analyze any particular action or circumstance at an asset.
 

Sending Sparkplug DRecords is extremely simple, with the relevant modules installed and a folder of Ignition tags configured as a DRecord folder, which generates a publish tag that when activated will send the DRecord. For a full technical tutorial see here.
 
Today the power of MQTT Sparkplug is in publishing this data as an immutable record that automatically builds and populates a table within a database.
 

How to send files using MQTT

The ability to take a picture, video or any file and send it through MQTT is a game changer for most companies and it’s also simple with MQTT. With the relevant module’s setup in Ignition, select the MQTT Transmission configuration and select the “Files” tab. Here you will define a directory path on the edge devices hard drive to look in for files.  With the configuration set to auto, any file that appears in that folder will automatically be transmitted via the MQTT Broker to MQTT Engine and stored in the specified directory path, along with metrics to monitor performance at each end. For a full technical tutorial on setting this up see here.