Serious VR: – Cinematic VR for the Simulations and Training Industry:
VR has always been the domain of well funded private and govt. organizations, and the military. However, now that it is within reach of storytellers, at RealVision, we find the first person POV that training simulations are based on, makes for a perfect vehicle to craft engaging user experiences.
We call it Serious VR!
Hybrid VR Cinematography for compelling Simulations:
Cinematic VR capture of a world, either the real world or a make believe one, is quite different from a real-time CG generated world when viewed via an HMD. Both have their pros and cons. To go over a few:
- In Cinematic (video based) VR, the real world is captured photographically, in motion, but there is no ‘positional tracking’ possible, i.e the viewer has 3 degrees of freedom to ‘experience’ the virtual world.
- A CG generated world can either be computer generated – or more recently – be photo-grammatically captured if trying to recreate the real world, but offers 6 degrees of freedom (translation as well as rotation). There are a few caveats with a CG only representation of the real-world, such as not being able to recreate the entire 360 field of view in motion. However, lightfield tech and cameras might soon make this possible in full 360.
What we believe will be a happy medium for now, will be the creative use of CG/Photogrammetry/Lidar (laser scanning) and traditional stereoscopic 360 video capture, to craft a Virtual Reality world that draws the user in, and doesn’t break their suspension of disbelief.
The image above is from a Faro Laser scan of my old workplace. With the megapixel color overlay photo that gets overlayed on top of this dense point-cloud, what we end up with is a digital replica of a real world location – that I can ‘visit’ at anytime – down to millimeter accuracy.
However, it’s a freeze-frame of real life, and lidar cannot capture 30 fps datasets. What you do get is a 6 DOF Digital Set in stereoscopic 360 with no “stitch lines” to ever worry about.
This is where hybrid VR cinematography comes in, particularly the creative use of Green screen to composite in action. At this time it is a challenge to use such techniques to re-create outdoor locations, But with careful planning… it just might, depending on the scene. Take a look at the image below. If this were a realworld structure of some ancient ruins, using a laser scanner, a skilled artist/compositor who knows Foliage generating software and green screening for equirectangular compositing in stereo, could create this scene and it would make for a very immersive experience.
Which brings us back to Serious VR, Documenting the Real World, and Simulations:
The first video in this article is a raw, unprocessed almost 2 million point cloud data-set of an existing Desalination plant. While it is a man made structure, and of not much cultural or heritage value, imagine what VR Filmmakers would be doing for the humanities, by documenting real world heritage locations, that while entertaining audiences also serve the dual purpose of digitally documenting our world. These digital ‘assets’ will become one of our most prized possessions to visit in our VR future.
As filmmakers, entering a new era in storytelling we are witnessing how VR excels at giving us the power to put the audience directly in charge of the camera and not have to ride behind a proxy protagonist. We see Cinematic VR being of use to industries beyond media and entertainment, and in our limited experience, Serious VR is already finding a ready audience in the Medical and Engineering domains.
For the technically inclined:
- This is an approx 2million point cloud un-processed scan of a Desalination Plant and being used as part of an orientation training project for a client who is looking at VR training for employees in a related engineering industry.
- The experience is running off an Asus Rog Laptop, that was simultaneously, in the background, also recording the video output (in stereoscopic 3d) to an h.264 file.
- Chroma-key artifacts visible because the key was pulled off of a normal white office wall during a demo at the client’s office.
- Video resolution is further reduced, due to stretching the side-by-side stereoscopic captured video stream to mono 720p.