This poster was presented at Computer Graphics & Visual Computing (CGVC24) at City St George's, University of London.
Mokhtar Salem Ba Wahal
Digital twins are virtual replicas of physical assets capable of mimicking their behavior through bidirectional communication in real-time, to better assess and understand complex cyber-physical systems (CPS) operation under both hypothetical and real scenarios. As CPS grow to become more complex and advanced, the need for enhanced visualization and interaction arises to ease dealing with such complex systems. Supplementing digital twins with augmented and virtual reality (VR) capabilities is a step towards easing human interaction by elevating the situational awareness of the user through high-fidelity immersive experiences. In this paper, we present a comprehensive framework for integrating VR with Dockerized private cloud infrastructure, to benefit from the minimized communication latency between both the physical/digital assets and the VR headset being utilized. The integration with the private cloud is ROS-based to ease the integration with robotic applications, make use of the extensive ROS libraries for robotics control and navigation, and to support the wide community of ROS. In addition, the modular development approach of ROS enhances scalability and reduces engineering efforts when incorporating new features to the system. Furthermore, the VR capabilities include innovative techniques for improving human interaction such as surrounding the digital asset in the simulation environment with a proximity heat map and adding VR-based hand-gesture control. A widely used research robotic kit, Turtlebot3, with a 5-degrees-of-freedom robotic arm have been used to conduct user studies on the aforementioned techniques to assess the efficacy of the added features compared to traditional techniques such as web-based control dashboards. This comprehensive integration of VR with Dockerized private cloud infrastructure facilitated by ROS and novel human interaction techniques presents a promising avenue for advancing cyber-physical systems.