The Future of Immersive Entertainment has Arrived
What if virtual worlds were more than just a set of screens in front of your eyes? What if these worlds could understand your needs and change without your doing?
Imagine having a home that “knows” you and transforms into what you need. Sometimes it could be having a cozy fireplace; others, a relaxing beach house or a snowy cabin in the middle of a mountaintop because that is what your mood feels like today. This is what physiological user-interfaces will be able to do: VR worlds will be able to understand the user and adapt the experience to provide new ways of interactivity.
For example, in the near future it will be possible to have a virtual manual that explains how to solve a certain task and, by knowing your attention and workload, the system will give you just the right amount of hints you need to fulfill the task without unnecessary disruptions. Or imagine a game where your magical powers are actually dependent on your own mental abilities.
Physiological user interfaces will allow just that; machines will be able to read a mental state and create implicit control over the virtual world, augmenting and adapting it to one’s desire without having to voice or command any changes at all.
Even though the research exists, the devices are not user friendly yet and the software and ideas are just starting to develop. Some of them can be used, like the modification for the VR game Skyrim VR Real Virtual Magic, which links the in-game magical powers to your brain. Real Virtual Magic uses the Muse S meditation headband, which has physiological sensors, but it doesn’t play perfectly together with VR.
The OpenBCI Galea is an interesting new integrated device that may be a significant step forward for these kind of user interfaces: It is an augmentation of the Varjo Aero VR headset that has 8 dry electroencephalography (EEG — brain waves) channels and also measures PPG (heart rate) and EDA (skin conductance), which are great measurements of stress. It even has facial EMG (muscle activity), which may allow emotion recognition. In addition, it has the eye tracking sensors that the Varjo Aero already has, and eyes are a great way to measure the mind!
From a research point of view, low-density dry electrodes mean that no spatial filters for in-depth analysis and source localization can be applied and the signal quality is unclear, especially if people move around. This isn’t to say the device is not good, but it is a different approach than trying to get the highest-end brain readings of a human. Instead, the multimodal data may allow imaging the mind of the users in a more multifaceted way — not only via the brain, but also other means we can use.
On the flip side, just like the signal quality is not to be compared with research-grade EEG, neither is the usability of the device: the Galea may allow users to set up and measure physiological data themselves without guidance, and this could open up entirely new ways to research and to create softwares for gaming and other applications. The native integration with BrainFlow also allows creating one’s own software for it.
Zander Labs is currently moving towards this direction, starting with the Real Virtual Magic modification for Skyrim VR and an upcoming interface to other game engines.
New and exciting times are ahead of us! The way we interact with virtual worlds, gaming or other, will have to be rethought, and we are proud to be part of this movement!
