Insight into an augmented factory

New AR software will help process engineers with commissioning, maintenance, inventory and more, as Nicola Brittain discovers.

Augmented reality was the stuff of science fiction for decades – The Terminator had visual access to swathes of data, as did the cyborgs in Bladerunner 2049. Augmented (AR) tools in the process and manufacturing industry now very much resemble these early sci-fi representations.

AR technology relies on IoT sensors and computer vision to overlay the real world with digital information, thereby enhancing the viewer’s perception of their surroundings. Now that so many elements of a factory infrastructure contain IoT sensors, AR is fast becoming the next step towards digitalising industry.

The World Economic Forum issued a white paper in 2022 that identified the manufacturer’s relationship with technology as one that had moved on from automation to augmenting the capabilities of the workforce. Similarly Gartner, in its group outlook for 2024, identified the ‘augmented connected workforce’ as one of ten key strategic trends of note.

AR is just one element of the digital revolution (along with robotics, automation and more) that will help factory operators and engineers streamline their processes.

SICK’s move into AR

Germany-headquartered technology company SICK recently launched an augmented reality assistant called SARA (the SICK Augmented Reality Assistant) that it expects to be particularly useful for process engineers and manufacturing plant operators. Originally a hardware manufacturer, the company moved into sensors then digitalisation, and now AR, as digital solutions consultant for SICK Charlie Walker explains.

SARA explained

SARA is the company’s first augmented reality tool. It is a programmable software package that retrieves data from sensors or PLC gateways and presents it to operators in a readable format. The sensor might be a Siemens PLC, a Rockwell PLC or an app detailing weather channel information for example.

All factories or warehouses now contain intelligent sensors systems and analysers that provide raw data to gateways. This data is converted to smart information via the cloud and on premises digital solutions.

As Walker explains, “getting data is no longer an issue, it’s how best to use it that has many engineers and operators thinking.”

And SICK’s engineers and operators thought very hard about how make factory data available to its customers.

As such, the SARA engine converts raw data into the standard OASIS IoT MQTT format which Walker describes as “a very manageable protocol” then provides information where it’s needed via tablets and smart phones as long as they have an Android or Apple operating system and camera.

A device agnostic tool

Walker makes it very clear that the data doesn’t need to come from IoT-enabled SICK devices, “although of course it can,” he says.

Once installed, an operator will use their device to view information related to a factory’s IoT connected inventory (pipes, robots, valves or stock) via diagnostic information that will appear as lines, arrows and labelled lines; a 3D point cloud; warning triangles; images; cubic volumes; or hyperlinks to sites and documents.

Technicians will be immediately alerted to any issues within their plant or factory as well as what is causing them via an easy-to-read display.

Walker argues that the provision of data in a real environment will lead to reduced downtime and make equipment more available leading to cost savings. “The service will also provide a better understanding of the machines and general infrastructure,” he explains.

The MQTT format explained

MQTT is an OASIS standard messaging protocol for IoT-connected devices. A deliberately lightweight publish/subscribe messaging transport system, it is ideal for connecting remote devices with a small code footprint and minimal network bandwidth. MQTT is used in a variety of industries, such as automotive, manufacturing, telecommunications and oil and gas.

How will the tool help process engineers?

As Walker explained the product was originally designed as a LiDAR (light detection and ranging tool) for autonomous vehicles since it can help to make these sometimes complex, error prone systems accessible to the workforce without the need for robotics experts. However, the SICK engineers quickly realised that the technology’s application was broader than this.

For example the tool can help plug skill shortages. “If there is one engineer and lots of technicians, as well as software and a little bit of programming, a senior engineer can pour their understanding into the software to make it available for the whole team,” Walker explains. “The skills shortage has been particularly acute in the UK since Brexit and so we expect this to be a key driver for uptake of our product.”

Inaccessible or unsafe areas

SARA can also be useful for checking inaccessible, unsafe or restricted areas or equipment since a phone or tablet can be pointed at a sensor from a distance. More specifically SARA might be used to test a motor’s temperature, to check a PLC load, to analyse vibration, or to measure liquid/gas levels and flow. In addition, an array of drives, motors, and pumps can be inspected in a short period of time.

SARA will also help with maintenance and commissioning since it can link to URLs or PDFs providing pictorial or step-by-step guides. It can also be used to order new stock when there is a fault.

Avoiding costly mistakes

Manufacturing processes require precision and attention to detail, and even a small mistake can result in significant losses for the company.

The technology can help operators avoid this by providing easy to follow instructions to employees without the need for additional explanatatory material.

Inventory feedback is another potential use case for the product. As long as a high shelf containing stock can be pointed at, the operator doing so can easily collate stock data.

Walker concludes: “Wherever equipment or stock is IoT enabled, SARA can provide insight. The number of potential use cases mean we expect to see a wide range of industries making use of the technology in the near future.”