October 23, 2020 by Biljana Ognenova
Virtual reality and augmented reality (or VR and AR) are just two examples of human-machine interfaces (HMIs) that are integral to immersive learning. Using industry-focused simulations, such HMIs can educate both prospective and experienced electrical engineers for practical work—and ultimately help companies to safely and affordably take on engineering graduates.
Many talented EEs are fortunate to enrol in world-leading universities and have access to reputable research labs. However, blended learning, computer-based platforms, and other e-learning—i.e. electronics-based learning—opportunities are limited for EE (electrical engineering) students and recent graduates.
Technologies, after all, advance faster in an industrial environment as opposed to an educational one. On top of this, the many technologies (and other engineering resources) that can inform and enhance e-learning are often shielded from non-professionals by their respective industries anyway. (The reasons for this include confidentiality and public safety.)
As a consequence of such obstacles, many prospective and up-and-coming engineers may lack access, particularly to industry-focused equipment and a hands-on research environment with all the latest trends. Nevertheless, many employers still expect qualified graduates to be ready to work immediately after graduation.
Engineers are often abstract thinkers and problem solvers, but qualities such as these can not be fully honed unless engineering education transforms intangible academic concepts into tangible—and therefore practical—projects. Indeed, if more electrical engineering (and other technically-focused) education were to adopt digital immersion into their syllabi, there will be excellent potential for overcoming these challenges.
A labelled continuum of real, mixed, and virtual reality concepts, ranging (left to right) from ‘real reality’ to virtual reality (with ‘mediated reality’ in the middle). Image Credit: IEEE Xplore.
Immersive learning is an educational system in which the user delves into a simulation or an artificial representation of reality to gain knowledge through immersion. The transition from reality to virtuality is on a continuum, as depicted on the above diagram (from the IEEE), which covers progress from ‘real reality’ through to virtual reality, or VR.
Immersive learning is based on extended reality (XR). XR is an immersion concept that envelopes all realities mentioned on the above continuum, including:
Although there are indeed gradients on which users immerse themselves within a simulated environment, immersive learning generally assumes using a human-machine interface (HMI) as part of the learning curriculum.
An HMI is, after all, loosely defined as any user interface that connects a human to whichever technology applies. And accordingly, almost everyone potentially immerses themselves within virtual reality to some degree today. However, for electrical and electronics engineers, the benefits of being granted a hands-on approach to their immediate work environment are multifold: the potential for better grades, improved knowledge retention, the bridging of the research-practice gap, enhanced design thinking (particularly for solving end-user problems)—and, ultimately, better grades and even job prospects.
On top of this, when you consider the restrictions of taking an e-learning course in solely engineering theory, or studying in a laboratory with limited scope for research and practical applications, there are few, if any, substitutes for gaining real-time feedback from a product and its users. Indeed, giving full attention to a product—while being virtually absorbed in the ideal environment to observe how it will actually work—is a prolific learning curve for engineers: it has the potential to completely transform how they learn.
Two engineers monitor part of an oil refinery by using a tablet and a mixed reality system. Image Credit: Research Gate.
On top of the above, immersive education also provides and accommodates new educational grounds, particularly given the unprecedented progress already made in cloud computing, AI, sensor technologies, and wireless networks.
Cloud computing, for instance, gives better data storage and networking capabilities, while AI—particularly as it merges with AR—enhances user experiences in healthcare, consumer electronics, retail, automotive industries, and more. Simultaneously, immersive learning gives employers a secure space to test new concepts in a safe environment and provide on-the-job-training to freshly-graduated EE students.
A lot of research carried out in the last decade, such as the studies mentioned below, has already indicated the potential of immersive education. Such R&D includes plenty of innovations, such as interactive training tools, smartphone immersion, and electronics design simulators, such as the AR-focused iCircuit app.
Desktop PCs may be the first HMIs that come to mind when talking about digital immersion. Nevertheless, mobile devices, e.g. smartphones, tablets, and wearable tech, should also come into consideration (despite their computational constraints when compared to PCs).
After all, alongside their lightness and portability, mobile devices have many marked advantages: the first and foremost perhaps being that they can incorporate a variety of sensors that may be remotely assisted, particularly for the more computationally-intensive tasks. This is just one example of what’s known as mobile augmented reality.
Mobile augmented reality (MAR) is an HMI system that combines virtual objects in a real environment, provides real-time interaction, registers and aligns real and virtual objects, and fundamentally runs an augmented view on a mobile device.
A diagram that represents some of the components of a mobile augmented reality system: eye tracking technology, remote servers, smartphone sensors, and more. Image Credit: IEEE Xplore.
An eye tracking device is an integral part of many MAR human-machine interfaces, especially as it offers students hands-free immersion due to its ability to track the user’s eye movements rather than gestures. The above diagram depicts an eye tracking device along with a cloud server (which is used for storing virtual, 3D objects for users to interact with). Alongside these components, the smart device’s integrated sensors (namely the GPS, gyroscope, compass, and accelerometer sensors) gather the input from the user’s environment so that he or she can engage with the AR simulation in real time.
As well as its relevance to training applications, the comfort of device mobility makes MAR systems suitable for such areas as tourism, navigation, architecture and construction, entertainment and advertising, industrial assembly and maintenance, and information management.
While not all of these industries carry an immediate association with education, many of them are at least indirectly linked to life-long learning opportunities for engineers and engineering students.
A report by professional services network Deloitte explores several case studies about AR applications in nuclear power plants, including communication between workers and engineers, the implementation of computer-based procedures, and the superimposition of radiation and plant information.
Furthermore, the AR-based maintenance of nuclear power plants (using information supplied through legacy paper documents) highlights faults and helps with plant repairs. A mobile augmented reality system for augmented assembly can also provide AR tips for real-world assembly tasks. Such a MAR system is based on a client-server architecture that consists of a PC for storing complex model information and a mobile device with a camera that acts as a thin client.
AR and VR have a propensity to be playful and creative, and it’s largely thanks to their mixture of study and play that such technologies are attractive to young students. As the next section reflects, starting with immersive learning as early as at the primary school age may well be a significant push towards STEM diversity.
An engineer inspects workshop machinery through a virtual reality headset. Image Credit: UL.
Evidence that low grades may not be the result of student aptitude but rather a complex interaction of factors—particularly how they are taught—comes from authors Rubina Freitas and Pedro F. Campos. Freitos and Campos have researched AR-based education through SMART (a SysteM of Augmented Reality for Teaching), which has been aimed at schoolchildren aged seven to eight (but similar concepts have also been applied to preschool teachers and in kindergartens).
In the SMART study, the researchers developed a manipulation system that helps children better grasp semantic concepts like means of transportation or types of animals. The students use a set of racquets to manipulate a television-style game with 3D models of cars, motorcycles, boats, and aeroplanes that are superimposed over a real-time video of the class. The research indicated that students can indeed improve their grades with help from the new learning system.
A typical example of an AR application in engineering education is hazard reduction. We are all more prone to make mistakes when we lack experience. Simulated errors are easy to correct and don’t cause casualties. Exploring high-risk environments (an oil refinery) through MAR systems has been proposed by two researchers affiliated with the University of Helsinki, Marjaana Träskbäck and Michael Haller who replaced traditional training with a virtual classroom, following a plant closure caused by safety considerations.
Apart from industrial product design and usability evaluation, 3D prototyping can find its extensive use in student motivation and engagement when combined with immersive CAVE displays. According to a paper on immersive engineering education and project-based learning, virtual reality and 3D software-focused digital prototyping assist students in becoming better communicators and problem solvers—and altogether help them improve their learning outcomes.
An engineer works on a holographic display using a zSpace stylus, a six-degree pen-like device. Image Credit: Oregon State University.
zSpace creates mixed reality (or MR) tools for students starting from kindergarten all the way up to university. Computer science and automotive are two exciting study areas supported by zSpace that could be of interest to anyone considering advanced engineering education institutions.
Immersive learning, despite its above research, development, and opportunities, is not without its challenges. Concerns related to social acceptance, data management, privacy and security, and less-than-perfect user experiences/user interfaces remain significant impediments to commercially viable extended reality solutions.
Digitally-simulated environments, nevertheless, are here to stay. Immersive technologies open up the chance to overcome many industry challenges, owing to their ability to combine the fresh, creative juices of students and graduates with years of hands-on experience.
