2D to 3D Movie Conversion Software Tools:

At Realvision, we are not big fans of 2d to 3D conversions. There are exceptions, typically for content that has already been shot such as older blockbuster movies and TV commercials. Some touch-up work in scenes that have been shot with true stereoscopic 3D camera rigs and exhibit unwanted errors, are also acceptable candidates for a 2D to 3D Conversion. The process is supposed to be treated as a VFX or Special Effect (SFX) and not abused for the entire duration of a feature length movie as can be seen by the recent outcry from Hollywood itself to such movies such as Clash of the Titans.

While there are many tools in the arsenal of 2D to 3D Conversion houses, and many claims of proprietary software or methods – the so called “secret sauce” of 2D to 3D conversion studios to help them set themselves apart from this lucrative industry ( estimated at US$1200 per second of conversion), there are common methods and software that can assist in this process.

Nuke’s Ocula and Disparity Map:

There are many ways to start off on a 2D to 3D movie conversion project. One such way is to obtain a “Depth Map” of a scene. A depth map would show the suggested or exact location of each pixel as mapped on a “depth volume”. In common language this would mean what parts of a scene are in front, and what parts are behind. With this depth map, software can then “render” a second point of view – The missing second camera view of a scene.

Missing pixel information (occlusion) is filled in by algorithms in the software, and thus two views are generated – the left and right views that make up a stereoscopic 3D movie. Nuke’s Ocula plugin suite is actually capable of generating a kind of depth map, or, Disparity Map from actual left – and right views of a scene for further manipulation by other tools in the Ocula suite of plugins. Some of these other plugins can do things like generate “new” camera positions etc. One thing in common that all the other plugins rely on heavily is this Disparity Map.

One trick is to use the Pulfrich principles of Stereoscopy (time multiplexed stereo) to extract a left and right eye view from a single 2D scene. This works of course only if there is no Vertical movement of subjects in the scene between successive frames of the 2D video, but there has to be horizontal movement of either the camera or the subjects.

Once this “stereo pair” has been extracted, it can be passed through Ocula’s plugins to generate either a Disparity field or Depth map which is then used further down the pipe-line in the software to generate the second camera view needed.  It is beyond the scope of this article to explain a complete workflow of this process. A list of all the plugins that comprise the Ocula suite of plugins for Nuke is here.

Adobe After Effects CS5′s Roto Brush for 2D to 3D Conversions:

The other method of doing 2D to 3D conversions, is with banks of animators and rotoscopy artists working under a supervisor  and via use of semi-automated and manual intervention. This involves assigning a “depth” to each object in a scene, then either / or rotoscoping the objects out and re-layering them, whilst filling in the “holes” between left and right eye offsets, or also “mapping” the rotoscoped objects over crude polygons modelled to represent the scene. Again this is beyond the scope of this article to show the details, but as can be seen – this is the reason why the entire process is so cost prohibitive and begs the question… Why can’t movie makers plan for a 3D shoot from the beginning? The excuse of bulky camera rigs is a thing of the past now.  Stereo 3D Camera rigs shooting RAW Digital video are very lightweight and small. What needs to change is the mind-set of 2D film makers.

Getting back to the task of rotoscoping, there are already many rotoscopy software on the market such as Mokey, Mocha, Nuke has them as well. What is exciting is the affordable yet powerful ROTOBRUSH that is built into the new upcoming release of Adobe AfterEffects CS5. This will offer quick and easy rotoscoping of objects in a scene and from the video here, it seems to be able to do this effortlessly.

Combined with the ability to be key-framed and use all of AfterEffects other tools, it will make 2D to 3D conversions a bit more within the reach of smaller studios and independent professionals.

Depth of Field – An artifact of 2D cinematography:

In this video, pay attention to what is being demonstrated around 3 minutes into the video. This is an important consideration with creating 3D movies. Depth of Field – a 2D cinematic technique is very much used by Cinematographers to convey a sense of depth to the otherwise flat 2D world, and also as a creative tool for instance to “introduce” new characters or areas of interest in a scene. While this works well and has been a staple in 2D cinematography it fails in almost all cases in 3D cinematography.

The reason is, people will try to “fuse” slightly out-of-focus areas of the scene, and will strain their eyes / brain doing this. The aftereffects tutorial while not intending to address this need, is exactly what can be done to completely de-focus a background and keep the subject in focus.