dan_equirectangular_Dirrogate

Compositing elements in Virtual Reality:

The image above, is the result of over 3 weeks of research and testing in Cinematic VR film-making. While it is not perfect, we are in the midst of stirring up some secret sauce to enable a smooth workflow, for Maya the 360 motion-comic.

Compositing in 360 space is not trivial. In fact, as of today, NO mainstream compositing software or solution exists for compositing in 360, in 2d. In Stereoscopic 360, compositing is a whole different beast. This knowledgebase will be updated during production of the VR novel.

For now, a few pointers to keep in mind:

Dirrogate_Airport_Stereoscopic_360_VR
(click for larger)

  • In S3D, you can’t cheat Scale. – If you’re shooting a scene from the point-of-view of an adult, make sure the 360 camera rig (or Stereo 360 rig) is at the eye height of an adult.
  • If you download the above image and view it via a VR HMD, notice how you will be looking “up” at the characters. Good for a seated VR experience, but if this were a scene where the Director would like ‘you’ to be a voyeur, listening in on a private conversation between two people at an airport… the vantage point would be suspect. It would be strange for you to be at crotch level!
    (Trivia: this scene was shot Guerrilla VR style at an airport in Stereo 360. Compositing in AfterEffects)
  • Try to match your CG camera as near as possible to the lens that shoots the 360 video/panorama. For the image above, the CG camera that shoots the girl is set at a 14mm focal length.
  • Pay no heed so-called VR Gurus. There’s a few of them cropping up who claim to know everything about Cinematic VR production. While best practices are great to adhere to, it’s still an open playground, and the Language of 360 VR storytelling is still being written. There are no experts, yet.

** Update** Aug1st 2015

As this article has been receiving hits from many AAA studios (Hollywood, Asia and Australia)… We’ve decided to make known some of the sauce currently in use, and other solutions for compositing in Equirectangular space

1) Sat Domes’ Camsphere

2) Mettle’s Skybox

3) Exclusively for Mac (in beta) : Dashwood Cinema’s VR toolkit

4) The not-so-cheap but very effective – Navegar Full Dome (also does equirectangular) plugin for After Effects.

5) Soon to be released for Nuke: see video below

Observations:

  • In a VR film, if seen from a first person P.O.V, a unique problem arises – if the wearers “Body” is superimposed or composited in place under the actual 360 camera, what happens if the wearer turns around? VR exorcism!
  • One possible solution to the problem above, is to have a dynamic “headless avatar” generated by VR movie playback software. This avatar could then respond to head-tracking and accelerometer/compass heading.

If you’re interested in some of the results of compositing in stereoscopic VR, grab a copy of the Stereoscopic 360 teaser of Dirrogate: Memories with Maya, and avail of an early access discount of 50% on the final VR novel when it launches.

  • Have you had any luck working out a rig inside of After Effects to create stereo 360.. I keep looking at Mettle Skybox as a potential way, by mixing two 360 camera rigs (in After Effects) with an offset but need to figure out a way so they would rotate their position with an offset between the two so as to create a true 3D stereoscopic result.. the two images could then be layered on top of one another and composited so as to be compatible for YouTubes ‘VR Video’ Stereo 360 format that it’s accept since November. Kind Regards Jon at Stormy Studio.

    • Hello Jon.
      You’re right in the approach of offsetting and rotating, as thats the only way to get true 360 stereo,
      You can ‘get away’ with using SkyBox to do a ‘correct’ stereoscopic shot, but will have to be creative about it.As you know, by simply rendering a left and right equirectangular image via Skybox, you’ll get around 160 degrees good stereo in front which then gradually tapers off to zero and gets completely inverted behind.

      Getting creative would be to make sure the there’s no closeup objects at around the 180 degree angle, and then swapping the image pair to correct for background pseudo stereo.

      Sadly, at this point there is no correct stereo rig for AE 360.
      You’re in luck though!
      Nuke just announced their beta today… you might want to take a look at CARAVR https://www.thefoundry.co.uk/about-us/news-awards/caravr-open-beta-announcement/

      best wishes

  • Pingback: Panasonic Professional 360 Real-Time 4K Camera: Hands-On – Found It Elsewhere()