Online Virtual Reality Networked Control Laboratory Applied in Control Engineering Education

Summary

This study describes a WebVR-based online virtual reality (VR) laboratory system that provides users with immersive and interactive experimentation capabilities supported by VR devices. The proposed system not only helps to enhance the realism of user participation in online experiments but is also applicable to a wide range of online laboratory frameworks.

Abstract

Online laboratories play an important role in engineering education. This work discusses a WebVR-based virtual laboratory system. The user enters the simulated laboratory environment through a virtual reality (VR) device and interacts with the experimental equipment, similar to hands-on experiments in a physical laboratory. In addition, the proposed system allows users to design their own control algorithms and observe the effects of different control parameters to enhance their understanding of the experiment. To illustrate the features of the proposed virtual laboratory, an example is provided in this paper, which is an experiment on a double inverted pendulum system. The experimental results show that the proposed system allows users to conduct experiments in an immersive and interactive manner and provides users with a complete experimental process from principal design to experimental operation. A solution is also provided to change any virtual laboratory into a WebVR-based virtual laboratory for education and training.

Introduction

With the advancement of the Internet and the popularity of mobile devices, the demand for online education is increasing¹. In particular, during periods of widespread epidemics, traditional educational institutions often face challenges in conducting in-person instruction effectively, which highlights the importance of online education as an important pedagogical approach². Theoretical courses are relatively easy to transfer to online platforms. They can be conducted with the help of tools such as remote video conferencing software and massive open online courses (MOOCs)³. However, practical courses face greater challenges as they require users to perform hands-on experiments in traditional laboratories.

Researchers have made significant contributions to addressing the challenge of making experimental equipment available online. Over the past two decades, extensive studies have been conducted on the concepts and technologies of online laboratories^4,5. Online laboratories typically encompass remote laboratories⁶, virtual laboratories⁷, and hybrid laboratories⁸. These online laboratory approaches have found widespread application in various engineering disciplines, including control engineering⁹, mechanical engineering¹⁰, and software engineering¹¹.

While significant progress has been made in terms of the convenience of experimental operations in online laboratories¹², users still perceive a lack of realism and similar hands-on practical operations compared to traditional laboratory environments, which affects their overall experience¹³. This discrepancy in user experience motivates further research and development efforts to enhance realism and engagement in online laboratory environments.

To solve the above problems, virtual reality (VR) technology has been applied in virtual laboratories¹⁴ to improve the immersiveness and interactivity of virtual laboratories¹⁵. VR-based virtual laboratories provide users with a close-to-realistic experimental experience. Users can complete group assignments in the architectural education process through avatars¹⁶, performing the architectural surveying process together immersively, just as they would in a traditional classroom environment. Furthermore, the VR-based virtual laboratories allow users to enter the immersive environment of virtual laboratories and interact with virtual experimental equipment by wearing VR headsets and handles¹⁷, improving users' hands-on abilities¹⁸. For different educational purposes, we can design different virtual environments. For example, VR can be combined with gamification theory to enhance engineering education for the general public and to improve the efficiency of disseminating difficult-to-understand knowledge such as sustainable development¹⁹.

Similar to online laboratories, particularly virtual laboratories, WebVR-based virtual laboratories have many advantages. Firstly, they break through the time and space limitations of traditional laboratories, and users can conduct experiments anytime and anywhere²⁰. Secondly, online laboratories can provide a safer experimental environment to avoid possible dangers and accidents in experimental operations²¹. Thirdly, virtual laboratories can also provide more experimental resources and simulation situations to extend users' experimental scope and experience²². Most importantly, WebVR-based virtual laboratories can stimulate users' learning interest and initiative and enhance their experimental experience and participation²³.

Compared with other VR-based virtual laboratories, WebVR-based virtual laboratory seamlessly combines the merits of VR-based virtual laboratories with web-based online laboratories. Virtual Instrument Systems in Reality (VISIR)²⁴ builds a basic analog electronic remote laboratory by constructing real circuit boards. Users can perform simulated experiments on the web interface to complete real circuit board experiments. Weblab-Deusto⁸ builds the water tank Field Programmable Gate Array (FPGA) laboratory where users can interact with the three-dimensional (3D) model of the water tank in the web platform without relying on other plug-ins. The system proposed in this paper introduces the capability to seamlessly integrate WebVR as a modular component into the existing virtual laboratory infrastructure. This integration can be achieved without destroying the original architectural framework of the laboratory, thus preserving the basic structure and function of the laboratory. This integration is also applicable to the framework of an online laboratory with separate front end and back end.

The system proposed in this paper is implemented based on Networked Control System Laboratory (NCSLab)²⁵, which inherits the flexibility, interactivity, modularity, and cross-platform features of the NCSLab system. Users can conduct experiments according to different modules and can also customize algorithms and configuration interfaces, providing users with enough space for self-realization. Online experiments are driven in real-time according to the algorithms run by the user. Users can interact with the virtual model to change the inputs of the experimental algorithm when conducting VR experiments and can even change the parameters of the control algorithm through the components so that users can experience the principle of the control algorithm more realistically.

WebVR-based virtual laboratories bring great potential for online education. It can provide an immersive experimental experience, overcome the limitations of traditional laboratories, and promote hands-on practical skills and innovative thinking among users.

Protocol

This study met the guidelines of the Human Research Ethics Committee at Wuhan University, and informed consent was obtained for all experimental data. In this paper, the experimental steps for the double-inverted pendulum system are discussed, and all the steps are performed in the WebVR-based NCSLab.

1. Access WebVR-based NCSLab system

1. Open a web browser that supports WebVR. Enter the Uniform Resource Locator (URL) of the WebVR-based NCSLab to access the system.
2. Click the Start Experiment button to log into the NCSLab system. If it is the first-time logging into the system, do an account registration.
3. Log into the NCSLab system, select different experiments from the left menu bar, and choose the double inverted pendulum experiment in this case.
4. Access the 3D sub-page on the main page.
   NOTE: There are five sub-pages on the main page, beginning with the first one, which is the introduction of the equipment model. It contains a 3D model animation as well as documentation. By visiting this page, users can grasp the principle of the double inverted pendulum system, enabling convenient execution of subsequent experiments.
5. Apply for experiment control by clicking the Request Control button to ensure efficient use of resources. This will grant users 30 min of experiment time.
   NOTE: For virtual experiments, 500 users can be allowed to conduct experiments at the same time.
6. Enter the plant information sub-page to gain access to comprehensive details regarding the experimental apparatus. This encompasses information on equipment that is currently in use, equipment that remains unused, and maintenance-related equipment.
7. Choose the system default control algorithm to download on the Experimental Algorithm sub-page. Alternatively, proceed to the Algorithm Design sub-page to design a different algorithm.
   1. To design a new control algorithm, click the Create New Model button on the algorithm design sub-page to enter the design interface.
      NOTE: The process of algorithm design closely mirrors that of MATLAB/Simulink, whereby users construct the control algorithm block diagram through an intuitive drag-and-drop approach, employing various modules to craft the desired control logic.
   2. Build the complete control algorithm block diagram, as depicted in Figure 1, and follow the steps described below.
   3. Select the Double Inverted Pendulum System Module from the device model on the left.
   4. Choose the Gain Module to design the feedback matrix for the Linear Quadratic Regulator (LQR) controller.
   5. Select the Step Signal as input and add other modules. Double-click the module to view detailed information and modify the parameter configuration. For example, double-click the Constant Signal Module to modify the value of the constant signal.
8. Click the Start Simulation button upon completing the control algorithm design. Upon completion of the simulation, observe the control effectiveness of the designed algorithm. If unsatisfied with the simulation results, fine-tune the parameters of the LQR controllers until a control algorithm with improved performance is achieved.
9. Click the Compile button to generate the control algorithm. After compilation, the algorithm is stored in the private algorithm area of the experimental algorithm sub-page and the algorithm design sub-page.
10. Download the control algorithm on the experimental algorithm sub-page by clicking the Download Algorithm button located on the right side of the control algorithm section.
11. Select an experimental configuration and conduct experiments on the Monitoring Configuration sub-page. The system provides a pre-defined configuration to meet the general experimental requirements of users.
    NOTE: Users have the flexibility to click the Create New monitor button to craft a customized monitoring setup tailored to their specific experimental demands.
12. Customize the monitoring configuration and choose from a variety of components available in the editing interface of the monitoring configuration sub-page, as depicted in Figure 2. These components include input variable components, output variable curve display components, and 3D model components.
13. For VR experiments, select the 3D model component. The 3D model component allows users to integrate a 3D model into the monitoring configuration.
14. To facilitate parameter configuration, adjust the parameters for each component, which are directly linked to system parameter variables. Double-click on a component and access the window to select the relevant optional parameters within the experimental system.
15. Users have the flexibility to optimize the layout of the monitoring configuration by resizing components. To do this, drag the edges of the respective components to the desired dimensions.
16. Click the Save button to save the designed monitoring configuration for future use in subsequent experiments, saving time and effort for setting up the monitoring system repeatedly.
    NOTE: The monitoring configuration can only be performed after the control algorithm has been correctly downloaded.
17. Click the Start Experiment button on the window to initiate the experiment. Click the VR Button in the bottom right corner of the 3D model component to launch the VR experiment.
    NOTE: The VR experiment is embedded on the web page. When users use it for the first time, the browser may prompt them in the top left corner to allow the browser to use VR functionality, select Allow to proceed.

2. Selecting the access method

1. Use a WebVR emulator extension. To engage in experimentation using this method, install the WebVR emulator extension, which is readily available for search and download from the browser's extension store.
   NOTE: The WebVR emulator extension helps users run WebVR content in a web browser and provides the virtual VR headset and the handles controller environment without the need to use the real VR device.
2. Use VR devices that support WebVR. If VR devices are used for the first time, the basic environment configuration is needed. First, turn ON the power of the headset and controller to start the system. Set up the initial ROOM program in the headset. Following the visual cues displayed on the headset screen, use the handle controllers to carefully calibrate the boundaries and orientation of the virtual space environment. Finally, establish a streaming connection between the headset and the computer.
   NOTE: This is the second method to access the proposed system. VR devices generally include a headset and a pair of handle controllers. VR devices have built-in stores where users can download WebVR-enabled browsers. Alternatively, users can use the built-in browser, which generally supports WebVR. It is noteworthy that various VR devices may employ distinct methods for connectivity.

3. Experimental procedure

1. Adjust the perspective to find the optimal position for conducting the double inverted pendulum system experiment.
   1. For users utilizing the WebVR emulator extension, open the Developer Tools, locate the WebVR extension, and manipulate the virtual VR device using the mouse to adjust the perspective, as shown in Figure 3.
   2. For users employing VR devices, immerse in the virtual experimental environment and ascertain the optimal experimental position through physical movements.
2. Interact with the double inverted pendulum system using the handle controller as described below.
   1. Move the handle closer to the cube. Press the Trigger button to pick up the cube and the double-inverted pendulum system will stop moving.
   2. By moving the handle, control the position of the cube. Release the cube once it is in the desired position by releasing the trigger button. The position is now designated as the subsequent setpoint for the cart, as depicted in Figure 3.
3. Observe the motion process of the double inverted pendulum system. By manipulating the Alternating Current (AC) servo motor, set the belt in motion. Under the impetus of the belt, the inverted pendulum can move along the guide rail, The system structure of the double inverted pendulum is elucidated in Figure 4. Eventually, the double inverted pendulum will stabilize at the setpoint.
4. Encourage users to iteratively manipulate the cube's position, continuously adjust the cart's setpoint, and meticulously observe the dynamic behavior of the double inverted pendulum system.

Results

The VR experiment system presented provides users with the capability to engage in immersive experiments using VR devices, thereby enhancing the interaction between users and the experimental equipment. Furthermore, the system is web-based, eliminating the need for users to configure local environments. This design allows for the system's scalability, making it suitable for large-scale applications and training and educational purposes.

In traditional laboratory environments, users are req...

Discussion

The presented protocol describes a virtual laboratory system that enables users to conduct VR experiments online but also uses a low-cost PC controller²⁸, which is conducive to large-scale application promotion. Users can gain knowledge about the entire experimental process, from principles and algorithms to practical experimental operations. This system allows users to immerse themselves in the experiments, eliminating the reliance on traditional mouse and keyboard input. This system provides an ...

Materials

Name	Company	Catalog Number	Comments
3DS Max	Autodesk		3ds Max professional 3D modeling, rendering, and animation software enables you to create expansive worlds and premium designs. https://www.autodesk.com/ca-en/products/3ds-max/overview
Meta Quest 2	Meta Platforms	10036728220341	meta quest 2 is a standalone virtual reality headset that allows users to experience WebVR content. https://www.meta.com/it/quest/products/quest-2/
Unity	Unity Technologies		Unity is the platform for real-time 3D interactive content creation and operation. All creators, including game developers, artists, architects, automotive designers, film and television, use Unity to bring their ideas to life. The Unity platform offers a complete suite of software solutions for creating, operating, and realizing any real-time interactive 2D and 3D content on cell phones, tablets, PCs, game consoles, augmented reality, and virtual reality devices. https://unity.com/cn