Introduction to Industrial Augmented Reality

February 07, 2009
News

Summary

1.1 Introduction to Unifeye PlannerToday’s factory planning needs to be fast and flexible to meet the increasing demands for product varieties and custom design. Therefore, reduction of planning errors is one major goal to achieve increased planning reliability as well as a reduction of planning time and costs. To ensure this process, up-to-date and complete planning data is required.The digital factory offers a large database for factory planning. However, difficulties arise when the real world does not correspond to the available digital representation, or when no digital representation is given at all. As the factory is a dynamic environment, existing layouts quickly become outdated. Therefore, continuous consistency assurance of reality and virtual planning data is needed.Augmented reality, a technology, which combines real and virtual data, becomes a beneficial interface between the real shop floors and the digital factory.metaio’s AR-based factory planning tool, Unifeye Planner (based on metaio’s Unifeye SDK), offers this the interface between real and digital factory by providing the needed functionality for easy and fast analysis. The comprehensive toolbox of Unifeye Planner offers flexible usage for a variety of factory planning related application:- Interfering edge analysis, to identify collisions and bottlenecks between real and virtual factory- Concept planning, to evaluate new factory layouts in a real environment- Planning workshops, for interactive discussion of new products- Part verification, to check the correct fit of virtual parts with their real connecting elements - Variance comparison, to identify discrepancies between reality and its digital representation1.2 Setting the augmented reality sceneIndependent from the concrete factory planning scenario, AR-based analysis requires correct overlay of virtual content (e.g. a virtual car body) with a real factory environment (e.g. a bottleneck in the production line).Scene analysisWhen an AR-scene is created in the Unifeye Planner application different tools can be used for analysis. The so called clipping planes support the process of collision detection and interfering edge analysis. Different views of the virtual model (visible, invisible, wireframe, occlusion) provide help for variance comparison or part verification. In addition, 3D distance measurements between virtual models or clipping planes allow identifying sufficient clearance. Referencing aspectsA crucial point for successful factory planning is correct and accurate referencing of the real and virtual world. Marker targets are used for tracking and can be detected in the image data of the real environment. However, in most cases, the markers cannot be attached at the desired location and an additional offset is needed to reference the virtual model with the marker position.Unifeye Planner therefore offers a variety of tools to support the referencing process depending on available data, tools and knowledge.Accuracy aspectsFinally, accuracy plays an important role in AR-based factory planning. The reliability of the analysis results depends on the quality of the virtual overlay with respect to the real environment. To assure high precision overlays, Unifeye Planner is based on high-resolution images with high-precision camera calibration. Still, uncertainty is given in the process of positioning the virtual overlay. In detail tracking, calibration, offset referencing as well as the virtual data are subject to uncertainty and must therefore be considered when evaluating the preciseness of an AR-overlay.To provide a quality statement for a given AR-scene, Unifeye Planner offers an Accuracy Calculation process, which combines uncertainty factors throughout the pipeline and summarizes them in error and standard deviation values for a given point in the scene. This accuracy information is an indicator for the reliability of the planning result.1.3 Step-by-step factory planningPreparationThe first step for preparation is the collection of information on the given planning case. What is the goal of the analysis? Which locations need to be evaluated and who needs to be present? Which virtual data is involved (e.g. different variants or different development stages of a virtual product)? Based on the initial information collection, data needs to be acquired. This includes pictures or sketches of the real environment, CAD data of the virtual content as well as of relevant parts of the real environment. A detailed description of the needed CAD data is of great importance, to ensure correct and suitable data (needed objects, level of detail, development stages, file format …).Then the planning of the execution phase can be done. On one hand the date and time must be set and coordinated with the factory. On the other hand the attachment of the markers for the tracking must be prepared. Favourable locations need to offer easy attachment possibilities as well as easy identification in the digital data, to reference real and virtual information correctly. Drill-holes for instance are a good referencing location. The markers can be attached rigidly by using special adapters and the holes are usually clearly detectable in digital planning data.ExecutionThe execution phase includes the discussion of the analysis with the responsible people on site, as well as the concrete collection of image data. For each examination point, the concrete problems must be identified, positioning of markers and the camera must be clarified and the required analysis results after the evaluation phase must be specified. Based on this information, the actual collection of image data can be done. It includes- the attachment of markers at the needed locations

at least one marker needs to reference the virtual content later on
other markers can be used to represent interesting objects in the scene and possible points of collision- the creation of an examination protocol
one per scene
detailed description of the scene (sketch, used markers, object descriptions and numbers, position in the factory, offsets, relevant digital models …)- acquisition of image data
image acquisition in correspondence with the examination protocol
assurance of optimal image acquisition conditions (line of sight to relevant markers, illumination …)
image acquisition from several perspectives to have sufficient data for later evaluation
image acquisition in series for moving objectsEvaluationIn the evaluation phase the collected data is combined to concrete AR-scenes, which aare then loaded in Unifeye Planner and used for the evaluation of the identified problems. First, the model data is prepared, to assure correct development stage, model color for optimal recognition in the AR-scene and adequate model size for the evaluation. The image data is checked for illumination and a high-precision camera calibration is calculated. Finally, the referencing offsets need to be determined, to bring marker coordinate system and virtual model coordinate system in accordance.For the evaluation, AR-scenes are created for the different problem cases. Each scene consists of image data, virtual model data and a tracking configuration, which contains the marker information and the needed offsets.Using the Unifeye Planner toolbox, analysis can be performed by applying clipping planes, distance measures or different model views. Each scene setup can be stored separately to document every important step in the evaluation process. Finally, quality statements for important analysis locations can be calculated to ensure reliable and sufficient clearance.1.4 SummaryCurrently Unifeye Planner has been successfully applied to various planning scenarios in different industrial environments. Based on the Unifeye SDK, the system proves to be a helpful and valuable planning tool, as it allows verifying and validating planning processes in an early stage. Through the easy photo-based analysis approach, results can be achieved very quickly, offering a reliable visual decision base early in the planning process.2 Robot AR Sales ToolIn the field of product presentation metaio has developed offline aAugmented Rreality applications together with partners from various industries. One of the biggest advantages of augmented reality product presentation concepts is putting the product into the problem domain of the end user or customer and offering the user interaction with the product in a very intuitive way. The Robot AR Sales Tool is used to support the marketing and selling of industrial robots. A unique visualization option is given to the potential customer to help determine the perfect placement of the robots he is intending to buy. The only preparation step needed is placing a paper marker in the area the robots are to be installed, taking a photo of the scene and loading it into the application. Within the application, virtual models can be selected from a wide range of different robots. The chosen model will be superimposed into the photo in accurate scale and perspective. The user can then interact with the virtual robot in its real future environment in real time: move it around in all three dimensions, rotate it, adjust each of its arms independently and equip it with different tools.In addition, the possibility to display each robot’s working envelope enables the user to quickly judge the effectiveness of a robot’s particular positioning, as well as detect potential risks of robots being too close to each other or to existing objects.When satisfied with the arrangement of the robots, a screenshot of the final scene can be saved along with a PDF file including detailed information on each selected robot.About metaioAs a pioneer in the area of augmented reality technology, metaio develops software products for systems driven by visual interaction in both real and virtual worlds. Our "Unifeye" software platform not only lets you place 3D animations directly into live video streams, but also supports the seamless integration of images from the external user environment. Founded in February 2003, metaio has 45 employees at three locations by now. Headquarter is in Munich, subsidiary companies are metaio Inc. in San Francisco and metaio Asia in Seoul. At present, metaio operates worldwide for more than 140 renowned clients from various branches and domains. Among them are BMW, Toyota, Peugeot, EADS, Siemens, Bertelsmann, LEGO, KUKA and Volkswagen.

Learn More

Did you enjoy this great article?

Check out our free e-newsletters to read more great articles..