(Image credit: lenslord.com)

Directing Audience attention in a Narrative 360 VR movie:

The biggest problem a Director or Virtual Reality Cinematographer might face in creating a narrative piece in Virtual Reality will be – how does one direct the audience’s attention to the “frame”.

There is no frame in the traditional sense, in an immersive 360 movie.

Part 1 on the Language of Visual Storytelling in 360 VR explains some real options to retain audience attention. In this article, let’s look at an anomaly in 360 Spherical capture that could work to our advantage as a Narrative storytelling tool in look-around VR movies.

There is emphasis on Narrative, as this method might not be suitable for Docu styled VR films, where audiences could be encouraged to look around.

So what is this anomaly? – Take a look at the youtube video above. This is not a true multi cam Stereoscopic 3D Pano rig, but after I carefully extracted sequential frames from the video, I got in touch with Jack from Lenslord.com, and I’m grateful he volunteered over two days, sending various shots which I requested he take, using his Ricoh Theta panoramic camera, and the cha-cha stereoscopic method.  The title image of this article is the result.  You can use red-cyan glasses to view in 3d. Download details for the Oculus Rift at the end.

What is of interest to note is:

  • There is a rapid fall-off of stereo as one looks away from the car toward the telephone pole in the center of the image.
  • Like wise, in the second half (the “back” of the 360 pano) the same effect is noticeable on the house-roofs as the eye travels toward the phone pole.

Inflating and Deflating “depth” in 3D movies: A visual Storytelling  tool:

Stereographers are known to inflate and deflate the Z axis or “the Depth channel” in a 3D film, and it goes un-noticed to the average movie goer. The theory is one can create a feeling of claustrophobia, by compressing depth or indeed, create a feeling of elation by opening up depth in a scene. Read the free chapter from “Think in 3D” via Amazon’s look inside feature, for more on the topic.

Stereo Fall-off  (SFO) in 360 VR: – A narrative storytelling tool?

Taking a cue from this use of depth as a visual storytelling aid in 3D movies, we can use this aspect of Stereoscopic 3D to subliminally guide the VR audience where to look in a 360 movie. For instance:

  • In the title image, there is full rounded Stereoscopic depth on the house. If this were a scene in a movie we could imagine an actor coming out of the house…
  • Now, while he might be making his way to his car (which is behind the audience; the wearer of a VR headset) … the audience might be tempted to look around the scene…
  • If there is a graduated SFO (stereo fall-off) employed where the depth channel tapers off at the outer field-of-view of the VR headset, this can have an effect much in the same way depth of field has in 2D movie making.
  • Of -course, we should not forget, that if the Director intends to track with the actor walking to his car which is behind the audience, the camera will follow (unless skilfully executed, a tracking shot in VR might induce nausea in audiences, we have learnt) cut to / re-frame… and so will the full rounded 3D depth- and now, the car won’t be “behind” the audience anymore.

Meanwhile, if the audience – the wearer of the VR headset – think of looking around, the graduated fall-off in Stereo (SFO) can act as a tool to “rubberband” the wearer to turn back to where the action is. Should the audience decide to continue to turn beyond this point… Depth is inflated again, and the back of the scene is in full rounded stereoscopic 3D.

The key here is, we use the anomaly of a full fish-eye lens, where stereoscopic depth tapers off as we move away from the center of a (stereoscopic) fisheye lens… to our advantage, as a Narrative Storytelling tool.

To see this effect, download the Oculus Rift scene. Use the free VRplayer and included json scene file settings for the scene to experience the concepts discussed in this article.

Side Notes to Cinematographers on Camera Motivation in 360 films:

Keystone and the Corner Pin:

Important: Frames created with any kind of 360 camera rig; either with Multicam GoPros or through one-shot Spherical lens methods, should be processed to rid the final output off stereo anomalies caused by un-intentional keystoning, and lens distortions. When creating traditional 2 camera stereoscopic movies, it was hard enough to get a matched set of optics.

That need created an entire industry for matched lenses – Angenieux Optomo 3D is one such dedicated offering. Trying to get matching lenses for a 12 cam GoPro 360 VR stereo rig will be a task indeed. Then there’s the issue of true “sync” Nothing short of true Genlock at scan-line level (if CMOS sensor) should be acceptable. Read a related article: Stereoscopic 3D Corner Pinning.

(Video: The importance of Genlock for Stereo 3D. Credit: unknown, will be updated)

VR Fatigue on the Oculus Rift and Sony Morpheus:

One of the contributors to eye fatigue and nausea on VR headwear such as the Oculus Rift, Sony Morpheus and similar VR eyewear will be from not adhering to, or being unaware of Stereoscopic VR production best practices.

This will result in Eyeball gymnastics for the audience; overtaxing their eyes to “fuse” badly created 3D VR imagery. If the previous paragraphs sounds daunting… it’s one of the reasons to investigate a slim profile, spherical rig as a viable solution, at least for narrative driven VR movies.

Suspension of Disbelief in VR films:

Two things that can’t be cheated in Stereoscopic 3D and VR presentations are – depth, and scale.

  • In the scene being created above, what is the motivation for placing the camera approximately 7 feet high? If this were a first person Pov scene, would the actor/character really be that tall? In 3D VR scale can’t be cheated. Audiences viewing such a scene would feel they are towering over the actress in the shot.
  • Or… looking at the title image in this article via the Oculus Rift, we feel like a giant looking down on the house, street and car – Why is that?

The reason is, the camera was placed at approx 12 feet. Camera Motivation in 360 VR films is crucial, if we are to maintain the suspension of disbelief in the audience. Unmotivated pedestal and crane shots will pull an audience out of the scene. This is not to say that it cannot be done, but the Director and Cinematographer are encouraged to step back at every scene and evaluate the pros ans cons of a seemingly no brainer 2D film-making camera move, to weigh it’s effectiveness in Immersive Virtual Reality.

Interaxial and Scale: OVR neck height.

Using 12 Go-Pros in a 3D printed or fancy rig, does not guarantee an immersive 360 VR film in 3D. The interaxial used in the title image is approximately 2 1/2 to 3 inches, and yet gigantism (the VR headgear wearer feels like a giant) is clearly felt in the scene. Unmotivated camera placement – in this case at 12 feet, combined with a large interaxial (camera base spacing) can lead to these unwanted results in a VR film. If creating an immersive  movie for the Oculus Rift – we need to think about OVR camera “neck height” when placing the camera, either in a CG or live action 360 scene.

The Virtual Reality Director is encouraged to “Think in 360!”

 Notes:

  • The Ricoh Theta is not a video camera, but is a one shot full spherical camera, which after investigating, looks promising to use for my self-funded Motion Comic project – MAYA. 
  • Years ago, Ipix had similar solutions for still as well as Video 360. They still do. 
  • Professional Stereo Pano practitioners might not condone using one shot (parabolic mirrors) or extreme fish eye lenses for stereo 360 video panoramas. As a stereo purist myself, I find it intriguing as a narrative storytelling tool as described in this article, and I would not hesitate to explore such as camera’s use in a VR filmmaking. The biggest draw is the lack of possible motion artifacts when stitching 360 stereoscopic video footage versus multicam setups. 
  • The Ricoh Theta does not provide a preview when taking a shot. Combined with the processing in photoshop to flip the stereo, readers are requested to excuse the quality of the end image when viewing through an Oculus Rift. However, the image serves well to illustrate points touched on in this article. Download the scene for the Oculus Rift.  Settings for VRPlayer are included in the zip.

 

 

  • Dirrogate

    As the downloadable demo for the Rift has been hosted on the official Oculus Rift website, people have asked how the final image was done.
    A few insights:

    The theta is a one-shot full sphere camera.
    Step 1: Take a picture with it
    2) move the cam 1 1/2 to 2inches to the right.
    3) take another shot.

    Ideally nothing should move between shots or you get what’s called “retinal rivalry” in stereoscopic 3D parlance.

    4) One half of the pano will have inverted Stereo (pseudo stereo). In photoshop or aftereffects the right half of the first shot and the right half of the second shot needs to be swapped (i.e right half of the second shot needs to become right half of the first shot and vice versa). This makes both shots show proper stereo. – No more pseudo stereo.
    5) Create a new image canvas in photoshop / aftereffects, Put the first shot (now called left eye image) on top, and second shot (right eye image) below.
    6) View in VR player to view on the Oculus Rift. Adjust VR player parameters/projection matrix to taste.

    For Unity 3D (theory)

    1) Don’t do step 5 above.
    2) Create two inverted spheres – these will be our skyboxes.
    3) Map left eye image to one sphere, map the right eye image to other.
    4) Make left sphere invisible to OVR camera right and vice versa (using culling for the cameras)

    —-

    All this is well and good for a still image background…what about Video?
    I’m talking to a few spherical cam manufactures to:

    a) Create cameras with slim profiles so two full sphere cameras can be placed side by side.
    b) Implement scanline level sync between cameras (I’ve not yet seen conclusive tests on Gopros so called stereo sync..yet)

    What needs further experimentation is: to see how much of a distraction the extreme periphery would be if in a side by side camera setup, the other cameras spherical “lens” is seen by the other camera.

    I don’t suspect it would be a problem in a narrative film, because the field-of-interest (FOI) usually follows the central talent around.
    Yet…need to experiment more.

    Advantages over multicam stitching rigs:

    a) No conclusive evidence that a 12 pair GO-pro rig is in genlock sync
    b) No possibility of stitching artifacts on action scenes as witnessed in multicam stitching
    c) advantage over 3 cam Ixicam spherical stereo – the vertical FOV is not optimal.

    More experiments to come.