Simplifying Integration Logic Development

Simplifying Integration Logic Development

by Joel Nash
As One Technologies
 
Introduction
 
As business operations evolve they require richer and more widespread inter-process integration. Because each case involves a unique combination of technologies and business rules, shrink-wrapped MES packages cannot implement the rich integration that is needed. The requirement goes well beyond communication pipes and protocols, integration requires developing custom software. The problem is that custom software is difficult to develop and maintain.  These problems were highlighted by Kevin Harding with Wyeth Pharmaceuticals at a February 2008 ARC conference, who said that “they would rather compromise on their requirements than write custom software.”
 

  
Integration requires logic, and it must be customized to address the unique combinations of technologies and business rules. The problem is that with today’s development tools this logic requires “software craftsmen.” This situation is analogous to craftsmen hand-building cars before Henry Ford developed the assembly line. As One Technologies’ Catalyst software provides the basis for “software factories” that simplify integration logic by allowing software to be assembled from pre-built components. 
 
Computing Paradigms Primer
 
Within the world of business operations a number of computing paradigms are utilized. 
 
 
 
Scan-based computing, used in plant-floor technologies such as PLCs and SCADA (supervisory control and data acquisition) is optimized for problems that require guaranteed response times.  One example might be stopping a motor within 100msec of a switch being turned off. Some characteristics of scan-based computing include:
  • Guaranteed performance
  • Logic must be relatively simple (otherwise there could be no scan time guarantee)
  • Scalability must be limited (otherwise there could be no scan time guarantee)
  • Collaboration with other technologies is limited to passing bits and bytes
Service Oriented Architectures (SOAs) “is a computer systems architectural style for creating and using business processes, packaged as services, throughout their lifecycle... These services communicate with each other by passing data from one service to another, or by coordinating an activity between two or more services… SOAs build applications out of software services. Services are intrinsically unassociated units of functionality, which have no calls to each other embedded in them.” (Wikipedia). SOA has become a popular style for creating and integrating back-office applications. With SOA, applications are structured as ad-hoc assemblies of re-usable services, with the overall application being uncompiled. 
 
While SOA is agnostic regarding the structure of applications, SOA implementations are generally based on a synchronous client-server model, with a high degree of centralization of data and/or orchestration logic. With the client-server model there is a clear distinction between the client that initiates a request and the server that: accepts the request, processes it, and returns the requested information to the client.   This approach lends itself well to many applications but is not well-suited for event-driven applications in which large numbers of concurrent and asynchronous events are generated by diverse and distributed sources. 
 
SOA addresses the service as a contract between multiple units of functionality, but does not specify a model for the units of functionality themselves, which implement the services. This being the case, there are potentially many ways to implement functionality that supports the SOA model. This also means that SOA does not have to follow the synchronous (typically client/server) model; asynchronous, event-driven relationships between units of functionality can also adhere to SOA principles of loose-coupling and assembled applications. 
 
The more closely an architecture models the real-world that it serves the more appropriate it is in solving real-world problems. Since the world of operations is distributed, asynchronous and event-driven, it follows that a distributed, asynchronous and event-driven architecture has advantages over synchronous (such as client/server) architectures. 
 
In a manufacturing environment every piece of data generated by people, equipment and applications could be treated as an event and any event can have wide-ranging repercussions. For example, a custom order impacts many production and business processes. Likewise, a problem with one piece of equipment may impact many other processes. It may be necessary to automatically and dynamically change the production schedule when a problem or a parts shortage occurs.   The better events are handled, the better the factory can optimize the pull of goods (products being manufactured) and services (production labor, maintenance, etc) throughout the enterprise.
 
In a commodity trading environment, prices are affected by events from many sources. Events could include inventories, demand, weather forecasts, political activities, currency valuations and flow of money into or from another market (e.g. stocks). The faster and more accurately the events can be processed the more money the trader will make. 
 
The real world is event-driven and asynchronous; it is As One’s contention that software / information systems should also be event-driven and asynchronous as well. Such systems will be easier to develop and maintain.
 
Event-Driven SOA - As One Catalyst Concepts
 
As One’s Catalyst Software constitutes a event-driven form of SOA by combining a “reactive agent” component model with the concepts of re-use, loose-coupling and assembled applications.  Each component (reactive agent) is an autonomous object that perceives the context in which it operates and reacts to events (incoming messages) appropriately. Each component exposes one or many ports, each port implementing a mono-directional (either input or output) asynchronous service that either sends events (messages) to, or receives events from another component(s). A message path between interconnected components is called a “link.” As with SOA the linked components are loosely coupled; they have no knowledge of each other, links are uncompiled and there are no “coded” relationships between components.
 
As One’s Catalyst automates both the creation of new components and the assembly of “solutions” of interconnected components. With Catalyst’s IDE (Integrated Development Environment) a non-programmer can visually mix and match pre-built components from libraries to assemble semi-custom solutions (applications). Note that the IDE is not a code generator, rather it produces XML metadata.
 
 
Catalyst has been proven to dramatically improve the productivity of developers; software is developed faster and at lower cost. By simplifying integration logic it becomes simpler to integrate and reduce the cost of operations. 
 
As One Software Factories
 
Catalyst is a generic system development tool.  When Catalyst is combined (productized) with a domain specific component library it constitutes a “Software Factory.” This is analogous to the world of manufacturing. A cell-phone factory assembles cell-phone components to create cell-phones; an automobile factory assembles automobile parts. In both cases the manufacturing process is dramatically simplified by assembling pre-built components. Craftsmen develop the components but craftsmen are not required to assemble the components into a finished product. 
 
Similarly, Catalyst compliant component libraries can be developed for a wide range of domains. A library for ERP integration would be very different from a library for military command and control. In either case software craftsmen create component libraries and less-skilled domain experts assemble them into semi-custom systems. 
 
 

 
As One’s business model is to put the Catalyst tools into the hands of OEMs (including ISVs) or integration firms (including large end users with internal integration capabilities). These partners may productize their own software factory(ies) by packaging As One Tools (Catalyst) with components libraries that they develop for their domain. 
 
Overcoming the Problems
 
As One Technologies has been developing component-based software since 1994 and agent-based software since 1997. Catalyst represents our second generation of agent-based development tools. Over the years we have proven our tools while providing large-scale systems to Fortune 100 manufacturers. In so doing we have encountered and overcome a number of hurdles. 
 
The first hurdle is manageability. With fine-grained components, a system may be comprised of thousands of components deployed across dozens or hundreds of nodes (computers with our runtime). The challenge of managing large systems is addressed in a number of ways:
  • Components can be combined and nested into re-usable “composite components”
  • The primary deployment module is called a “Solution,” which is a collection of interconnected components. A system is typically comprised of multiple interconnected Solutions
    • Each runtime can host multiple Solutions
    • Components are provided for intra-solution communication
 
The second hurdle is to automate development tasks and simplify maintenance.
  • There is no need to write “plumbing” code. All threading, lifecycle management and scheduling code is provided by the tools. 
  • The IDE is not a code generator, rather it generates XML that can be organized and managed in a number of ways.
  • Loosely coupled (uncompiled) components make changes easier.
  • Intuitive, visual representations of Solutions are stored with the XML Metadata for each Solution, making future maintenance much simpler than maintaining hundreds of lines of code (as is the case with other technologies).
  • The IDE includes a built-in debugger.
The third hurdle is to provide programmability. No matter how extensive the component libraries, there will always be a need for new logic. We address the programmability issue with “dual mode components.”  Components have two types of behavior: behavior that executes in design mode (within the IDE) and behavior that executes in runtime mode. In design mode, components can expose functionality that is as simple as property configuration or as complex as “logic editor” functionality. The resulting logic (created during the editing session) is encapsulated within the component that created it and will execute in runtime mode. At the present time Catalyst includes the following programmable components:
  • State Machine Engine
  • Workflow Engine
  • Rules Engine
  • Scripting Engine
The fourth hurdle is to provide components that are applicable for multiple domains. In addition to the programmable components (above) we provide components for:
  • File and database manipulation
  • Web-services
  • Networking, messaging, industrial IO
  • Enterprise Busses
  • Logic (gates, timers, counters, etc)
Summary
 
There is a large need to simplify integration logic for operations. This need will only grow as businesses seek to become more integrated and demand-driven. Today’s integration logic, developed by software craftsmen, is expensive and fragile. As One Technologies simplifies the development of integration logic (and process automation logic). We do so with a model for software factories that simplifies the development process for event-driven software. In so doing we lower the bar, reduce cost and improve flexibility of system development.  For more information on As One Technologies or their products, please visit www.as1tech.com.