- January 13, 2014
By Bill Lydon, Editor
Big data has been a hot topic in the IT world for a few years and now industrial automation vendors are talking about this as well. I attended The Big Data Conference on October 22 ‚Äì 23, 2013 in Chicago to learn more.
By Bill Lydon, Editor
Big data has been a hot topic in the IT world for a few years and now industrial automation vendors are talking about this as well. I attended The Big Data Conference on October 22-23, 2013 in Chicago to learn more. The event was targeted towards business and technology professionals charged with developing their company's big data strategies.
Big data momentum and investment is growing. Wikibon says the total Big Data market reached $11.59 billion in 2012, ahead of their 2011 forecast. The Big Data market is projected to reach $18.1 billion in 2013, an annual growth of 61%. This puts it on pace to exceed $47 billion by 2017.
Analise Polsky, Thought Leader at SaS, provided some insights in her presentation called "Going Big - the Analytics and Visualization." She recommended thinking carefully about how the information will be deployed and emphasized that design matters. Polsky discussed presenting information in graphical ways that enable visual discovery, since 70% of all sensory receptors are in the eyes. Visual discovery allows users to quickly see patterns to questions or confirm hypothesis.
Oracle's Peter Filice, Group Vice President IoT (Internet of Things) and Architecture, made a presentation called "Big Data and The Internet of Things" about the IoT value chain from devices to enterprise applications for optimization of business processes in real time. This is accomplished using big data analytics based on data from a wide range of sources including enterprise applications, Internet, human-generated sources, and machined-generated sources. He noted that machine-generated data constitutes the most recent generation of information sources. He believes the challenges for gaining the benefits of the Internet of Things are extreme scalability, real-time event handling, and lowering the time-to-insight. Using big data and analytics with all this information allows businesses to create and apply predictive models for decision-making to meet business performance goals. He noted these systems should be closed-loop and continuously adapting based on feedback to improve results.
John Burke, Principal Research Analyst and CIO at Nemertes Research, noted that big data generally pushes folks to parallelize with Hadoop, the most common platform for accomplishing this. Hadoop
Hadoop is making big data applications practical for collecting and analyzing huge amounts of data to improve business performance. These large amounts of data are measured in petabytes (symbol: PB) which is 1015 bytes of digital information. Onepetabyte is equal to one quadrillion bytes.
Hadoop is software designed to solve problems based on huge amounts of data. It is used to run analytics that are deep and computationally intensive like Google does when indexing the web and examining user behavior to improve performance algorithms. Hadoop is designed to run on a large number of machines that don’t share any memory or disks. That means many commodity servers can be deployed with Hadoop software on each one. The Hadoop software fragments large amounts of data into pieces that it spreads across multiple servers. Hadoop keeps track of where the data resides. By contrast, in a centralized database system, there is one big disk connected to multiple processors that limits computing power. (Editor’s note: A centralized database system is the configuration of many industrial automation system architectures.)
In a Hadoop cluster, every server has multiple CPUs spreading out the computing to multiple servers, with each operating on its own little piece of the data. Results are then delivered back as a unified whole. Hadoop provides reliability and redundancy to ensure no loss of data. Architecturally, the system is able to deal with lots of data because Hadoop spreads it out. And the complicated computational questions can be answered quickly because all of these processors are working in parallel. For example, I just Googled “Hadoop,” and it delivered answers ranked by relevance in 280 milliseconds.
Major companies are applying big data technology to handle large amounts of data. In a June 2013 eBay presentation, they noted that they service more than 120 million active users, or 300 million searches, every single day, and they host more than 350,000,000 available items. Amazon handles millions of back-end operations every day, as well as queries from more than half a million third-party sellers.
Thoughts & Observations Relative to Industrial Automation
The tremendous growth of IT investments is accelerating and creating a range of off-the-shelf software for tapping data sources, analyzing big data and closing the loop to optimize business operations and processes, including manufacturing. The overall goal of closing the entire loop for business operations through manufacturing enabled by the Internet of Things may well be the next force driving the integration of IT and automation. Business requirements including supply chain integration, real-time purchasing and other factors are driving enterprise business systems to become real-time transaction processors. Traditionally, the plant historian has been the single location to store large amounts of data and is the source for reporting and analytic tools. In contrast, big data concepts are knitting together silos of data to more holistically improve business operations. The plant historian will be just another data source along with distributed data located in automation controllers and devices.
There are a couple of Big Data initiatives in industrial automation.
GE Intelligent Platforms recently announced their Proficy Monitoring & Analysis Software Suite to deal with Big Data. I interviewed Brian Courtney, General Manger Industrial Data Intelligence at GE IP and wrote an article about that conversation.
Is this the next big step that will merge industrial automation information sources with enterprise IT?
- Time for a new automation architecture?
- Manufacturing Internet of Things (MIoT)
- Simplifying Automation System Hierarchies
- Intelligence-based Manufacturing
Hadoop Basics Resources
Did you enjoy this great article?
Check out our free e-newsletters to read more great articles..Subscribe