Industrial Augmented Reality As An Approach For Device Identification Within A Manufacturing Plant For Property Alteration Purposes

Abstract

The introduction of 3D computer graphics has led to an increase in the processing capacity of the computational units monumentally, along with speed, memory and transmission bandwidth. Augmented Reality (AR) has modelled remarkable progress towards real-world consumer applications. Considering the fact that mass production occurs daily in the manufacturing plants with large sums of wastage, caused either by human error, load-shedding (power outage), machine malfunction, or the time it takes the engineers to identify and fix the problem, are observed in high volumes. Therefore, the need to identify strategies and solutions to reduce such problems on-site with accurate data, rather than outsourcing or depending solely on the Supervisory Control and Data Acquisition (SCADA) system data, which might damage the integrity and economy of the manufacturing plant, needs to be developed and implemented. In a controlled network, identification and detection of a component in the process are difficult without prior knowledge and background in the design and implementation process. Thus, the concept of device identification with the aid of augmented reality, utilising markerless identifiers, such as machine vision, other than Quick Response Codes (QR codes) or Radio Frequency Identification (RFID), needs to be investigated. It is because of such reasons that the deployment of new types of technologies, such as “augmented reality” and “machine vision” need to further be investigated to obtain the device details, based on their positions and features within the indoor manufacturing plant to procure and commercialise this solution technology. This study proposes an optimal and efficient model, utilising machine vision application to detect and identify devices, based on their positions and features within the manufacturing plant with the aid of an augmented reality application for extending the device details. The study has outlined a machine vision application developed for object detection, based on colour and shape. Additionally, another method based on the augmented reality application was developed for the identification and augmentation of device details, based on the feature and position of the device within the indoor manufacturing plant. The study proved to be very successful in the identification and detection of objects, making use of machine vision algorithms, namely colour, shape and Canny Edge detection and the identification of devices (robotic arm and motors), based on their features and position within an indoor manufacturing environment set-up. For the optimal efficiency of this model, the Simultaneous Localisation and Mapping (SLAM) algorithm (ORB-SLAM) was used, in conjunction with the bundle adjustment algorithm as an alternative solution in the absence of the user built-in maps for the calculation of the device positions, based on the uncertainties of the exact locations within the indoor manufacturing environment set-up. However, some of the shortcomings were identified and addressed, such as the communication speed and the room’s light conditions, which impacted the sensing of the camera to detect the correct objects. These shortcomings were, however, addressed by conducting two studies, namely the day and night study to compare the best light settings and also to reduce the distance between the devices and the AR application to compensate for the communication speed issues. The scientific contribution of this study is the recognition of components by means of vision identification within such a process within an indoor manufacturing set-up. By means of identification, the user will have the capability to view and adjust the parameters of the process in a scaled plant. This contribution makes use of a modelled JPEG image. An AR image that the user can identify the devices apart from, relying on the SCADA system alone, was physically modelled on Blender3D for utilisation in Unity3D, as opposed to utilisation of any image and referencing it which would make the process tedious and reduce the processing speed. Subsequently, it has been depicted as part of a new knowledge contribution, that the identification of the devices can be achieved by placing the smartphone at any angle of the device (robotic arm or motor), and the detection and augmentation will be achieved without any change in the settings. As part of result validation, a video was taken and uploaded on YouTube to receive a user perspective on the developed AR application. After the video upload, a survey was shared with 20 individuals, together with the YouTube link to indicate a broader base evaluation. However, the results came back positive with the majority of the sample individuals recommending the adoption of the application and its utilisation in the scaled manufacturing plant. In addition to the results verification, a SCADA model was developed in National InstrumentsTM LabviewTM and was integrated with the AR application for evaluation purposes. The results showed that the AR application doesn’t require any alteration, despite utilising a different SCADA model in different software applications, provided that the array index is the same. Only when the array index differs, is it that alterations are necessary utilising the AR application in order to have the same array elements and avoid having a null index that might cause the application to crash or not to debug. It is therefore noted that the AR application is compatible and reliable for integration with other SCADA models without alteration requirements. The entire work outlined in this thesis was validated by two sets of physical experiments, namely GPS-based detection, and the ORB-SLAM, integrated with the Bundle Adjustment algorithm for feature and position detection. However, despite the prior knowledge of the GPS's inconsistent operation within a scaled indoor environment, it was necessary to perform the test to obtain more insight into this inconsistency and inaccurate data results.