Journey down the rabbit hole of the technology behind VR



A highly debatable and contentious question amongst the VR community "What is VR?". Closest to the actual definition propagated from purists, is MXR Lab Creative Director David Nelson "Virtual Reality’s one of a variety of disciplines of immersive technology that include Virtual Reality, Augmented Reality and Mixed Reality. So Virtual Reality is typically defined as a completely synthetic environment where you have no view other than that environment. It’s typically experienced through a head-mounted display, where you have a visual display and some lenses that create a stereoscopic image. Your head is tracked, and sometimes your body is tracked. The environment is usually somewhat interactive or responsive to your actions". Nelson also acknowledged 360 video "Expect researchers in the field would say that 360 degree video is not Virtual Reality at all. There’s a difference of opinion there, and it is certainly a lot of people’s first experience with Virtual Reality. But based on the criteria, which is defined as it being an interactive, responsive environment, a lot of 360 video is really just that."


Michael Mortimer from the VR Lab tends to agree "we consider it as a subset of VR, the reason why we put 360 videos on the VR spectrum is because the user is being completely immersed, and they see nothing but the recorded video." 


Most stakeholders within the industry tend to hold a similar view stating that 360 video viewed in a headset is an immersive experience that fools multiple senses into simulating a virtual environment. 

The Nuts & Bolts

In order to apply best practice when using VR, it is important to understand exactly how the technology works. VR, in essence, is a sensory trick, so how do these VR systems simulate another environment and transfer the user to another "reality"?


In order to convince the brain you are experiencing a 3D environment, how images are displayed is highly important. Firstly HMD's (Head Mounted Display) achieve this with a stereoscopic display. This works by displaying two slightly different angles of the same scene to each eye, thus creating depth. Michael Mortimer from the VR lab explained the mechanisms best "So the visual aspect is actually done by presenting the user two images – a left and a right eye –  and that’s to deal with the separation between your two eyes. We actually do two things when we try to judge a distance, we will do parallax and convergence on accommodation. Convergence is when we try to use our two eyes to pinpoint a single object and make a connection between the two eyes, so it’s the same object, allowing us to reference a distance using the distance between our two eyes. Accommodation is like the focal point - light being reflected off a physical object. That’s something that we can’t, at this point, simulate, which is why some of the early VR might become a little bit difficult for viewers. But the stereoscopic aspect deals with the convergence of the two eyes. So we just present two images to each eye, and it gives them a sense of depth."


Latency or display lag is another contributing factor to good or bad VR experiences. Latency is the measure of how quickly a pixel in an image can refresh and switch. An acceptable maximum rate for VR experiences is 20 milliseconds, anything over this can result in a less than pleasant experience and can lead to motion sickness. Several variables can contribute to latency and they can be the CPU, GPU & Screen. With this many variables, each able to add 3-4ms, it is easy to get close to the max 20ms and therefore reiterates the importance of making sure appropriate hardware is being used to view VR.


Motion tracking is another highly important aspect to take VR to a more immersive encounter and there are different levels of tracking available. The amount of movement available in a VR environment is called "degrees of freedom".


3 degrees of freedom (3DoF) - This is exactly how it sounds as it allows 3 directional types of movement as listed in the image below. These are achieved due to, in large part, the technology inside our smartphones - accelerometers, gyroscopes, and magnetometers. 3DoF is the level of immersion that is found in most smartphone based headsets (Samsung Gear, Google daydream, Google Cardboard etc) and can be achieved somewhat with 360 video when viewed in these headsets.

6 degrees of freedom (6DoF) - This adds 3 additional possible movements to the above. This where true VR and immersion takes place with the ability to move around in a computer generated space. This requires a number of senses to track the movements of the subject.

He goes on to explain the importance of audio "Audio’s come a long way. Traditionally audio might’ve been mono or stereo, just single left and right. However, now we have systems that can represent seven channels, allowing two rear, two side, two forward and back."


Effective audio in a VR experience will provide the user with a surround experience and it can also play a vital role in 360 video by helping dictate a users actions by drawing their attention to a specific piece of action taking place in the spherical environment.

take a look around

360 video is in a large part responsible for the uptake of VR and its rise in mainstream popularity. Companies such as YouTube and Facebook embraced 360 video by enabling it on their platforms and positioning it as an engaging piece of content. It is a large component of this project as it is by far the most accessible and widely used VR content within the tourism industry. 


360 degree video or also known as spherical video, is video shot simultaneously on a multi-directional camera or multiple cameras. The footage is then stitched together to create a moving image of an entire scene. With the ability of smartphones and HMD's offering 3DoF, a 360 degree video becomes an immersive experience where the user can dictate the field of view. 


The advance in this technology has seen multiple companies such as GoPro, Samsung and Nokia release 360 degree cameras which are available from $100 right through to $60,000. The consumer/prosumer market has seen significant growth and will play a major part in how tourism as an industry can embrace VR and immersive technologies.