Virtual reality: The next frontier

A research assistant assesses the terrain

Scientists are challenging our perceptions of who we really are with their work on a new immersive virtual reality.

Read my recent piece in The FT Weekend Magazine and cover story in below or by following this link.

It was one of three stories about tech in that edition – one by the amazing Douglas Coupland, another by the FT’s US editor and then mine! It shared the front cover with Douglas Coupland’s story.

am standing on a ledge situated 30 storeys up on the side of a skyscraper. One in 10 people has the guts to jump off but, I am rapidly discovering, I am not among them. This may seem sensible but it is, in fact, embarrassing because neither the ledge nor the skyscraper nor the city around me actually exist. They are all part of a virtual world whose graphics aren’t as good as Call of Duty and which I can only see and hear by wearing a large headset while standing in a groundbreaking Oxford university laboratory with a researcher holding the heavy operating leads. And yet I still can’t jump because, even though I know it’s not real, a voice in my head keeps saying, “Just suppose you are wrong . . . ”Situated in the grounds of Oxford’s Warneford Hospital, which specialises in mental health issues, this virtual reality laboratory is run by Professor Daniel Freeman, an MRC senior clinical fellow in Oxford’s department of psychiatry. He is pioneering the use of virtual reality to address paranoia, and the ledge simulation helps to show how real virtual reality can feel — important if you are going to use it as a diagnostic and therapeutic tool. When I take the headset off, the laboratory has plain walls, almost no furniture and mysterious lines marked out with duct tape on the floor, along with two young researchers in front of computer screens.

Now imagine a computer-generated world where if you did step off that ledge, you would feel something close to actual pain when your body hits the concrete. While this isn’t yet reality, it isn’t complete science fiction either. An increasing number of experiments conclude that we will be able to experience physical shocks like this as we develop virtual realities. Researchers reassuringly describe this kind of pain as merely simulated yet, under close questioning, it is clear that the boundary between simulated and real pain is rather blurred.

Immersive virtual reality is the vision of a computer-generated reality that looks, sounds, smells and feels so authentic that it psychologically tricks our brain into accepting that it is real. In this world we would feel actual pain — and pleasure — as we would identify so strongly with our virtual body (in whatever form we want that to take) that our brain would think it was our body. We would walk and talk with the freedom we feel walking to the pub. It would be an experience uncompromised by an awareness of any of the data-collection or display technologies necessary to create it.

For Professor Robert J Stone, one of the UK’s original VR pioneers and director of the human interface technologies team at the University of Birmingham, the “benefits of a truly immersive virtual reality would be limitless. It would help to break down the barriers between cultures and religions, the old and the young. It could revolutionise healthcare, and what about learning? Think about the ultimate virtual field trip, especially for inner-city schoolkids, or being able to teach history via a convincing form of ‘virtual time travel’, and extending geography to include the study of outer and inner space.”

Stone’s own recent research includes a command-and-control system for British Aerospace that uses virtual reality to help monitor and direct emergency operations taking place anywhere in the world. Another project uses a virtual cockpit that allows a pilot’s instrumentation panel to be customised to meet the demands of each mission.

A virtual reality headset©Daniel Stier

A virtual reality headset

We haven’t reached Stone’s vision yet. At the moment we make do with headsets such as the Oculus Rift, which, however sophisticated, are recognisable descendants of the clunky headsets and gloves with sensors that many of us experienced virtual reality with for the first time — at the London Trocadero in Piccadilly Circus in the 1990s in my case. Even if the graphics and sound of these new headsets are better, they still can’t trick our sense of taste and touch or give us the freedom of movement that would bring our virtual experience closer to that of Captain Kirk on the holodeck.

However, these technologies do give us experiences with different degrees of immersion. When we play Call of Duty for long enough, we start to lean with our body, like we are looking round a door. A headset can trick our brain into thinking that we shouldn’t jump off a virtual ledge. In a laboratory in Barcelona scientists are stabbing people’s virtual hands to see how much simulated pain their body actually feels. The same researchers have found that changing a virtual body’s skin colour can inform us about our attitudes to other races. It’s not hard to imagine that somewhere deep in the Pentagon they are testing virtual reality as a tool for torture that leaves only psychological scars. In Las Vegas they are no doubt exploring it for something far more pleasurable.

“The difference between watching a video game and being in a truly immersive reality would be like the difference between watching the Iraq war on TV and the feeling of being in it,” says Professor Mel Slater, ICREA research professor at the University of Barcelona and UCL, who has been involved in virtual reality research since the early 1990s. Currently, he is studying the longer-term effects of virtual reality.

Robert J Stone©Daniel Stier

Robert J Stone (pictured in his laboratory at the University of Birmingham): believable experiences are a “very long way off”

“Fully immersive virtual reality would ideally involve all your senses but this doesn’t exist today. What is missing particularly is the haptic, or force feedback, sense. I feel my elbows brushing a wall in the real world. In the virtual, I don’t. At the moment it is not really clear how this feedback might be delivered — whether it is through a whole-body suit that could give you all kinds of tactile sensations or even a giant robot ready to push you.”

Robert J Stone puts it more bluntly. “A VR system that delivers the freedom to explore and interact via believable experiences of sight, sound, touch, smell and, possibly, taste is the Holy Grail, as without any one of these then it’s not immersion, presence . . . or whatever you want to call it. We are a very, very long way off this right now no matter what the marketing hype says. Sure, the new generation of headsets is lighter and cheaper than what we had to endure in the 1990s, with slightly better displays, optics and head-tracking capabilities but, frankly, we really haven’t progressed that much over the past two decades.”

For Stone, all the hype about the “gizmos” distracts from the power of content, which is what audiences relate to when attempting to create a sense of engagement. “For example, when we show villagers on Dartmoor a high-quality virtual reconstruction of their long-closed local railway on a typical high-definition TV, allowing them to explore using a Microsoft Xbox controller, they become fully engaged with a scene that is on their doorstep and instantly forget their issues or fears with the hardware technologies.”

Military VR simulation©Daniel Stier

Military simulation

Key to the usefulness of immersive virtual realities for video games, whether in training soldiers to fight, doctors to deal with patients better or veterans to overcome their post-traumatic stress disorders, is the concept of body identification. Body identification is a phenomenon triggered, says Mel Slater, when you are in an immersive virtual reality “and you look down to see that your actual body has been replaced by a virtual one, or you look in a virtual mirror and see the virtual you staring back . . . It is a perceptual illusion.”

This was most recently demonstrated when Slater and his colleagues measured the reactions of people who were stabbed in their virtual hand while in an immersive virtual reality. Their automatic brain response matched what would have happened in the real world. In another experiment by the team, when the virtual bodies of adult volunteers were replaced with those of children, they started seeing the world as much larger than it is; yet when they were embodied in a child-sized adult body, their perception didn’t change to anywhere near the same extent. When white volunteers’ virtual bodies were replaced with black ones, they found that their unconscious racial bias had declined after the experiment was over. More recent evidence suggests that this change lasts at least a week.

Back in Oxford, Daniel Freeman’s research in the virtual reality lab is demonstrating a here-and-now application for immersive technologies. Within a few minutes of refusing to jump off the ledge, the computer had been rebooted and I was plunged into the depths of a virtual London Underground. Walking through a carriage with all the sounds of the Tube in my ears, I passed 10 human avatars: a young woman, a businessman, a student with a backpack… all their behaviour had been programmed to be neutral in order to assess my levels of paranoia. Yet all I could think was, why are they all looking at me?

This simulation was created by Freeman and his team to demonstrate how low self-esteem can cause paranoid thinking. Shrinking a person’s height (I am 6ft 2in) in the virtual reality simulation can make them feel inferior, weak, incompetent and more fearful that others are trying to harm them. His research has also started to show that this can work in reverse, by making people taller in a Tube train or a lift (another simulation they are experimenting with), scientists may be able to boost their confidence in the virtual world, which will help in real situations.

“We have been able to give people virtually exactly the same social situation that they are scared of and to assess how hostile they feel the virtual people are,” says Freeman. “Where we are heading right now is using it for treatment, and we have seen some people’s fears that other people are about to attack them out in the real world reduce from 100 per cent to 10 to 20 per cent after half an hour of use.”

Close-up of a VR gun©Daniel Stier

Close-up of a VR gun

There are still many problems associated with virtual reality. These include so-called “cybersickness” and even symptom flashbacks for some people who use head-mounted displays for too long. On the horizon, Stone says, are bigger issues such as “the potential future abuse of VR for brainwashing, particularly if you could target people with latent psychological problems or extreme tendencies. It may even be used as a form of torture by putting people into nightmarish environments that they would find physically and psychologically disturbing.”

Slater believes that, as with any other new technology, “it is for all of us to decide where the limits to its use should be set”. What he is most worried about is the ensuing moral panic if there is a high-school massacre and a head-mounted display is found in the killer’s bedroom. “The headlines will be ‘virtual reality turned my son into a killer’.” Full immersion will come, he says, “in about 50 years and it may only happen if we can connect a computer direct to our nervous system.”

Freeman, however, believes that the positive effects of immersive virtual realities could be felt much sooner “if every mental health ward had a VR set-up, as patients would then be able to practise going back into the social situations they had found difficult and so recover the confidence to leave the ward to live back in the community again”. And this wouldn’t be virtual, it would be real.

Leave a Reply