- By Josh Eastburn
- May 19, 2021
- Opto 22
- Feature
Summary
New hand sanitizer packaging plant uses unique automation architecture to run at the pace of modern business. This article is from the Ebook Automation 2021: OT/ICS Cybersecurity.
With demand for hand sanitizer more than doubling across the U.S. and continuing annual growth of more than 20 percent expected in coming years, 2020 presented the cleaning industry with a big financial opportunity. Emerald 66 Enterprises (E66) mobilized resources to meet this need, setting up shop in an empty denim processing plant in Seminole, Okla., U.S.
In only three months, E66 had three automated packaging lines producing up to 1 million bottles of hand sanitizer a week in a cGMP-compliant facility, and it continues to expand its core process capabilities at a rapid clip. “When we say we do stuff quick,” said E66 chief of operations Robert Bodnar. “We’re talking days, not weeks or months.”
Let’s step back to the summer of 2020 and examine the technologies and techniques Emerald 66 and system integrator Northeast Automation Company, Inc. (NACI) used to achieve competitive advantage in a challenging market environment.
Getting down to business
When E66 hired NACI to develop its bottling and packaging process, the company understood it needed to move fast. It was competing against low-paid, high-volume workforces operating manually and believed it could use technology to do more with a smaller, better-paid workforce.
“Each piece of equipment needs to be intelligent . . . because management is so keen on information,” explained Thomas Coombs, principal engineer at NACI. “We’re going to make every conveyor and every device smart.”
Coombs hoped to use industrial edge computing—a design technique that adds general-purpose data processing and connectivity capabilities to traditional real-time control and sensing applications—to build an information management system at the same time that he scaled up production capabilities. But E66 had also determined that the quickest way to build a new packaging process was by acquiring a variety of equipment at auction. Therefore, the state of equipment on arrival varied widely, and NACI had to get creative in order to design a cohesive system at the speed that Emerald 66 needed.
Layered distributed control
To address the different circumstances the team faced, NACI employed a unique architecture that enabled separate control systems to function together. It also laid a foundation for E66’s data acquisition goals.
NACI used an edge programmable industrial controller (EPIC) to establish a primary control network. EPICs combine programmable logic controller (PLC) control with other automation capabilities, particularly connectivity options like OPC UA and secure gateway server functions. The EPIC was tasked with supervising the packaging lines, connecting disparate devices through REST APIs, and integrating any equipment that arrived with a defunct control system through remote I/O.
Any functional controllers, on the other hand, were left in place but loosely coupled to the main process using edge I/O modules. Edge I/O communicates directly with connected systems independently of any controller. These particular modules provided multi-signal I/O channels and could be powered over Ethernet (PoE), allowing NACI to quickly scale the network. NACI placed a module in each piece of equipment upon arrival, connected any I/O wires, and identified the types of signals the equipment provided. These signals were then integrated into the EPIC’s Ethernet network, while the existing PLC I/O connections were allowed to function independently (Figure 1).
Coombs noted, “The ease with which you can do this, you know, you’re talking about a half-hour of wiring. Your biggest problem is finding the documentation from the original manufacturer [of the equipment].”
NACI also engineered an additional layer of control independent of the EPIC by adding limited local control to each edge I/O module. Living up to the ambition to make every device smart, NACI added motors, photo eyes, load cells, and other instrumentation to many pieces of semi-automated and dumb equipment and connected these to the network of edge I/O modules. Each module was programmed with embedded logic, using IBM’s open-source Internet of Things (IoT) language, Node-RED, to make them work together and report process data up to the supervisory level.
A veteran engineer himself, E66’s Bodnar was involved in much of this design. He explained, “If you have a conveyor coming in and you have a conveyor going out, they may not be running at the same speed . . . You’ve got cappers that are running at different speeds. You’ve got label printers and all these different components all kind of running at different speeds, so it’s kinda neat to be able to say, okay, what if we use the [edge modules] to control just the conveyors and the belts and case packers and things like that? . . . We’re doing just little simple stuff: turning on and off motors to run a line and trying to match up to how fast a filling machine is spinning. They’re kind of little islands of point automation.”
This loosely coupled, distributed architecture allowed NACI to assemble the packaging line without modifying any of the existing control systems that came as part of the purchased equipment. This strategy ultimately saved them development time, and in three months, Coombs and his team had 15 pieces of equipment up and running across three packaging lines, including conveyance, filling, capping, printing, labeling, packing, and palletizing operations.
But this would not be their last challenge.
The big pivot
Because it had built its business around packaging and distribution for a single large purchaser, Emerald 66’s original process was designed to maximize throughput. But the situation changed significantly when their customer suffered a financial setback and had to close production. Then, the whole business had to pivot to allow E66 to become a multi-product facility.
Automation grew from processing a high volume of singleformulation, one-gallon containers to working with a variety of sanitizer chemistries in different batch sizes and packaging form factors: from small two-, four-, six-, and eight-ounce containers, hand pumps, and spray bottles, to large jugs in excess of one gallon.
Fortunately, NACI’s decision to use a loosely coupled production line made it easy to modify individual segments without interrupting data collection and process integration. In combination with on-site panel building and 3D printing for rapid prototyping of new mechanical components, E66’s investment in edge-oriented automation allowed it to retool very quickly and break even on its initial investment within six months.
“There are a lot of good things that you can do in the industrial world that people might [comment on],” added Bodnar. “You know, ‘You’ve got good quality. You’ve got good throughput,’ and whatever, but they usually don’t say that you’re nimble. It’s not usually the term people think of . . . But it is fun to be nimble, and on the industrial side, I think that’s one of the things [edge computing has] really helped us with.”
Data-ready automation
As Emerald 66’s process expands, so do its ambitions for building out a data infrastructure. In addition to using Node-RED to augment the production process, E66 has begun tapping into the core connectivity functions of the platform to move data from each edge device into relational and time-series databases, financial software, and other connected systems. And since Node-RED is a free, open-source application, E66 is adding it to its back-end systems as well, so it can push data down into the control system.
“You might want to trip a lot number forward on a device or increment something based on a date that’s somewhat arbitrary, based on a business event, not necessarily on a machine event,” Bodnar explained. “You can send that over [to the control system] fairly easily, or just get summarized data [from the control system] and publish that back to your business systems . . . It gives you a very lightweight way to run an operational bus or hub.”
For example, Emerald 66 purchased a standalone pallet-wrapping machine that used a proprietary circuit board design with only a limited operator interface. However, by adding load cells, connecting them to an edge I/O module, and feeding that data to Node-RED, E66 could verify the weight based on the known pallet contents and communicate the pallet number, lot number, shipping weight, and date to its central database to create bills of lading automatically.
Bodnar, who describes his career as back and forth between giant enterprise application integration (EAI) companies, like Oracle, and manufacturing and consumer products companies, was impressed by the ease with which E66 could integrate business and operations data.
“I’ve done smart grid stuff for a bunch of cities and things like that on the big integration systems . . . We would have multiple [enterprise data] buses for operational data, reporting data, back-office data, or for communicating with [different kinds of devices] . . . I was thinking about how insanely hard that was . . . It would have been so much easier with Node-RED in an [edge] device.”
More to come
As demand grows, Emerald 66 continues to expand its automation and diversify its business, rapidly adapting to accommodate new functions. Previously, for example, E66 received hand sanitizer in bulk 330 gallon totes or by transferring it directly from tanker trucks into its filling system. Now, it is experimenting with blending down concentrated product to reduce shipping costs while also tripling production capacity for its largest customers. The company also has plans to introduce equipment for onsite container molding and automating quality control procedures through additional Node-RED integration.
“We are scaling rapidly,” said Bodnar. “[I’ve got customers] calling me at 9:00 at night saying, ‘Hey, let’s double it again! Let’s double it again!’ So in this environment, especially knowing that [the market] might recede again, automation is really the only answer.”
But Bodnar says he is defying the industry norm of investing in a multi-million dollar build for an automated facility and instead “putting in just enough automation to double my business, literally, which is going to make the company’s year. And then I can grow into that. And I’m not going to have to go back and rip and replace everything . . . We’ll tweak things, but I’m not going to have to throw away anything that I do because I can scale it.”
Recently, E66 has also spun up a lucrative technology division that is using edge devices to develop filling equipment for overseas export. In one example, NACI took an inline mixer design from concept to implementation in about five days, including a mobile operator interface built using the EPIC’s embedded human-machine interface (HMI) server. By NACI’s estimate, they implemented the controls in about four hours for a system that E66 will sell for $50,000.
“The fluidity and dynamics of modern manufacturing requires extremely fast response to changing market demands,” adds NACI’s Thomas Coombs. “[Edge computing] puts dynamic manufacturing data at the edge of the production line and into enterprise systems simultaneously in real-time.” For more information on Emerald 66 and Northeast Automation Company, Inc., visit their websites.
This article comes from the Ebook Automation 2021: OT/ICS Cybersecurity.
About The Author
Josh Eastburn is director of technical marketing at Opto 22. After 12 years as an automation engineer working in the semiconductor, petrochemical, food and beverage, and life sciences industries, Eastburn now works with the engineers at Opto 22 to understand the needs of tomorrow’s customers. He is a contributing writer at blog.opto22.com.
Did you enjoy this great article?
Check out our free e-newsletters to read more great articles..
Subscribe