Adobe recently showed off “CLOVER VR” a kind of Premier Pro for editing VR video, in VR. The concept and the need is very cool and much needed at the same time.
While watching the demonstrator showing a cut, I immediately thought of an older article I’d written (copied and adapted below) on depth continuity and how it’s as relevant to “VR” video as it is to stereoscopic 3D films. Why VR in quotes? Rather than replicate – it might be worthwhile to skim through to SKILL 1 in a previous article.
If the demo of the CLOVER VR were to have shown stereoscopic VR scene cutting, then editors working with a VR headset would quickly experience, first hand, the importance of depth continuity in a VR film. Particularly a narrative VR film.
(image copyright: the movie, Haunted 3D. used for educative purposes only. click each for larger)
Stereoscopic 3D Depth Continuity in a VR film:
In traditional 2D films the task of maintaining continuity from scene to scene is that of the Script Supervisor (and Director). The same goes for “depth” in a 3D or VR film. For instance, in a 3D movie, a scene features a person walking down a road into the distance, the next camera angle can’t show the person to have traveled half way down the road in a matter of a couple of seconds. Unless, this is done intentionally as part of an effect of speeding time.
But in 2D movies, it’s easy to cheat depth. That’s one of the ways those stunts involving a person crossing a railroad track at seemingly the very last second while a train approaches, works. The use of long lenses compresses depth and makes it look like the train is a lot closer than it actually is.
In Stereoscopic 3D and Stereoscopic 360 or “VR video films”, this cheating of depth is very near impossible to do. The one thing that stereoscopic 3D gives us, is the ability to record and present “Spatial information” or unlock the depth channel in a scene.
Take a look at the two images above. They are from (imo) a well made 3D Bollywood movie, Haunted 3D. The stereographer, Brent Robinson has done an excellent job throughout the film in maintaining depth continuity along with other S3D best practices. While it can be said that he has been conservative with the depth budget throughout the movie, in hind sight, the movie worked well without causing any harm to the audience.
S3D Depth Continuity is more than just Cut matching:
In the scene above, both the shots “match” in depth continuity. The actual “deepness” of the corridor matches in both shots shown at two different times during the movie. One way to ensure this, is of course to shoot all scenes involving a particular set, at the same time. However, there are other things to take note of to maintain depth continuity:
1) Ensuring the lens settings are the same (in a stereoscopic movie). In a VR movie lenses are usually fixed fish eyes.
3) The Interaxial and /or toe-in (convergence) if any, is noted and re-applied when shooting at the same location. – For a 360 VR movie, it would not be kosher to set interaxial, on set… (it could be done, but that’s a whole different article!)..so for VR it is usually limited to camera placement, if executing a hard cut.
4) That the time taken to travel the distance is approx the same depending if the character or camera is previously shown as running or walking.
The next example might not be as relevant to VR Depth continuity, as it is to stereoscopic 3D filmmaking. Take a look at the images below:
(image copyright: the movie, Dark Country. used for educative purposes only. click each for larger)
The image above is a severe breach of Depth Continuity in a single take in a traditional stereoscopic movie. It is not a cut but an actual tracking shot around the main characters, but the depth abruptly bounces due to an interaxial change. Thus the entire depth of the scene gets disturbingly changed. (This can be executed in VR in post, but unless you are an experienced Stereographer and have mastered timing, the general advice is: Dont! or you’ll make your audience hurl)
Another example of a depth mismatch is in cuts. In the pictures above, the first shows the same scene with one 3D depth setting, this then cuts to a solo shot of one of the characters speaking, and then cuts to the second image above, showing a completely different depth. This is another example of no Depth Continuity.
VR Depth Continuity – The Script Supervisor’s Job? The DPs or the Stereographer?
That question is open to debate at this stage of VR movie making. As time goes by and more experience is gained, it’s natural that the DP would want to take charge of the entire composition of the VR”frame” (Yes, yes… there is such a thing in narrative VR filmmaking). Frame is in quotes because today in VR filmmaking, a lot of established terminology from regular fimmaking, are open to new interpretation.
Getting back…should the script supervisor be in-charge of the depth script as well as the movie script? or should the depth-script then be in the hands of the DP, and can he/she also pay attention to maintaining depth continuity in addition to his/her other tasks.
As VR movie making evolves, one debate that will soon be over is whether there is a need for a stereographer or not. I would argue yes, there is a need. For the very same reason there is a need for a Best Boy and Gaffer… for the smooth running of VR film shoot, a stereographer is certainly recommended for this one reason alone: taking care of depth continuity during shoot while consulting with the Cinematographer and in post, with the editor.
Can these jobs be offloaded to the script supervisor or DP? Yes, in indie productions one person fills many shoes, but in a well planned (and financed) VR film, Depth Continuity could be better handled by a Stereographer.
(image copyright: the movie, Drive Angry 3D. used for educative purposes only. click each for larger)
Motivated Camera Placement and Depth Continuity:
In the shots above, taken from the movie Drive Angry, the image on the left shows a related version of Depth Continuity mismatch: Perspective mismatch.
The creative framing of the scene would suggest a person’s POV who is much taller than the two characters in front, but more reasonably, it shows a between-the-shoulders objective framing. In a normal 2D movie (not a 2D 360 movie) this framing would look just fine, suggesting the car is far away and thus tiny in size.
Can you guess why not in a 2d 360 movie? -Because when viewed in a headset, those heads and shoulders would tower above you!
In a Virtual Reality movie, immersion is best served with proper scale.
When wearing a headset, you are ‘in’ the movie.
Getting back…in this scene it looks perspectively wrong in stereoscopic 3D. The scale doesn’t add up. The scene looks like the two characters are perched either on higher ground, or on top of a vehicle’s hood, looking down at an approaching car that’s far away. Yet in the very next cut (in this particular movie), we get the POV of the driver of the car looking at the roadbock with the two previous characters now shown standing on the ground.
In fact, nothing in the Geography of the scene suggest that the driver is driving as uphill as is made out to be, as shown in the previous shot. Depth continuity does not match, because in 3D 360 (let’s call it video based VR) the terrain – the ground, is very easily accessed by the audience.
In VR movie editing, the editor now has to learn a couple more aspects of shot and scene continuity; that of camera motivation, perspective, and depth matching between cuts. A stereographer, with experience and depth intuition would keep a lookout and red-flag such events during principal photography or at least during edits if he/she is present during post production of a VR movie.
It is not the intention to critique the two movies mentioned above simply for the sake of critique. To be fair, the movie Dark Country was pioneering in it’s usage of a maneuverable stereo camera rig and the film was made many years ago. Some un-related errors found, thanks to the luxury of a pause button of the BluRay player:
Pic 1: 2D shot inserted into the 3D film. There are a couple more such 2D shots noticed. In a VR movie, this could be an immersion breaker – instantly pulling an audience ‘out’ of the moment.
Has any recent VR film been guilty of this? – There are a few pointed out in the review of the big budget, episodic VR production, Invisible.
Pic 2: Rotational and/or vertical mis-align error. (noticeable on the actresses teeth) – Worth keeping in mind even in VR films where Camera Rig vibration or mismatched lens pairs might call for a post-Vfx fixes.
Image Copyrights: All copyrights are respected and acknowledged as belonging to their respective owners. It is only our intention to use the images above for educative purposes and to further the cause of stereoscopic best practices in movie making.