adaVR

The Adaptive Digital Assistance in Virtual Reality project (adaVR for short) explores how virtual environments in Virtual Reality (VR) can adapt to their users.

Why adapt?

One major problem with VR is that it is still a comparatively new medium. Many users need some time to get used to the technology. And even then, the mental load is often so high that even "simple" interactions can become overwhelming. Users may then forget how to open a door or pick up a cup. Yet one of the key advantages of VR is supposed to be that interactions are far more natural and direct than on traditional PCs. What could a solution to this problem look like? Often, we have to rely on tutorials and tips to give users important information exactly when they need it.

At the same time, there are also users who already have experience with VR, for example people who own a VR headset and use it regularly. These users quickly get bored by too much assistance, which can have a negative effect on user satisfaction. Although this group makes up only a small fraction of learners (according to a survey by the gaming platform Steam, about 1.5% of Steam users currently own a VR headset), even inexperienced users differ greatly in how quickly they adopt the technology.

This is where adaVR comes in: it tries to "guess" from sensor and behavioral data how well a user is coping with specific VR content. A key advantage of VR here is, for example, the eye-tracking and pose data that modern VR headsets can record. Combined with data on task performance, such as how many mistakes are made or how close a user is to the next correct step of a task, these data can allow conclusions about mental load and orientation.

In adaVR, a machine learning algorithm captures these data and feeds them into an adaptive assistance system. It stays in the background for people who are coping well, while offering more support to those who are making less progress. In this way, the VR environment becomes intelligent and can adapt better to the needs of its users. We hope this will remove significant barriers and make VR in teaching a little better for everyone.

Many thanks to the Stiftung Innovation in der Hochschullehre, which funds the project through its Freiraum2025 program.