Virtual Cinematography Reloaded

There’s no denying it, Avatar the movie has caused a revolution not only in film making, but has affected other spheres of life as well, almost to the point of it becoming an inspiration and a way of life for some. The strong emotional bond that it created with audiences can also be attributed in a way to the immersion that only Stereoscopic 3D movies can, as in people’s opinion the story line was not something so unique, so it could be the 3D effect that created the bond.

There was a similar phenomenon that happened a few years ago with another movie that reached cult status and inspired a whole legion of followers to the very futurist ideas and thinking it provoked in those who understood the deep meaning that the movie presented.  - The Matrix. In fact the storyline behind the Matrix was complex and thought provoking enough to actually warrant more than a few repeat viewings, and unlike Avatar, for which repeat viewings are for the exquisite CGI eye-candy presented in stereoscopic 3D, The Matrix was a 2D movie and the reasons for going back to watch it was almost always because people went back home and had that question in their heads “what is the Matrix”.

The reason to compare the two movies in this article however is not for philosophical reasons, but because they had one thing in common, though it may not be very obvious at first glance in the latter movie – Virtual Cinematography.

Above: the VCam Virtual Camera Tracking System from Intersense

What is Virtual Cinematography?

An in-depth answer to this could actually take up the space of a book, so to give a generic overview of it and in context to this article, it can be defined as the creation of a CG “set” and the choreographing of CGI  (Computer Generated Imagery) actors that are animated or their performances motion captured  and then directed by the director who with a  ”CG camera” produces moving pictures that vary from cartoon like renditions, to highly realistic renderings that may be indistinguishable from real world live action scenery and human talent (actors). This is a simplistic overview of the topic.

Directors or Cinematographers have always been used to a physical camera in their hands, much like DJs are used to the feel of Vinyl and a turntable. But that is soon changing… just as DJs have moved from Vinyl to CD and now you see the familiar Apple logo back-light on a laptop in a nightclub, so too have Directors slowly evolved their film making style to using Computer technology and “virtual assets” in movie making. One of the biggest breakthroughs however, is the creation of a physical camera rig that can be “tracked” in 3 dimensional space i.e on X,Y and Z axes along with other parameters such as zoom setting and acceleration and this tracked data of the physical camera can then drive in real-time, a computer “camera” in the rendering software. Thus a Director can hold in his/her hand, something tangible that they have come to rely on to create computer generated cinematographic sequences – Virtual Cinematography.

Virtual Cinematography – Matrix versus Avatar:

In the movie Matrix Reloaded, which was produced in 2003, there was some excellent use of Virtual Cinematography but perhaps without the glamor of having a Director on set with a Physical “rig” directing the action. The virtual cinematography is so amazing and well done, that unless pointed out, not many even know that one of the best scenes in the movie, dubbed “The Burly Brawl” was the turning point in Virtual Cinematography, and re-defined  fast action cinematography itself.

Watch on Youtube

(the scene above is completely Computer generated after the first minute of conversation between the two actors – Everything including the buildings and realistic skins of the actors)

Quoting from this in-depth article from Wired Magazine : …” But by the time the melee is in full effect, everyone and everything on the screen is computer-generated – including the perspective of the camera itself, steering at 2,000 miles per hour and screaming through arcs that would tear any physical camera apart. This is virtual cinematography, but the most impressive thing about the Burly Brawl is that it doesn’t look virtual at all. The digital faces of Reeves and Weaving could get past a flank of security guards, and the buildings surrounding the courtyard look dreary and lived-in – the grimy, unmistakable patina of the real.”…

The credit for this Virtual Cinematography has to be acknowledged to the genius of John Gaeta who conceived this game changing way that virtual cinematography was done. The sad part is, it was probably ahead of it’s time! When you compare the realism of that scene, even though it does not have as many close-ups of actors as Avatar, from reading these references, and the in-depth article on Wired,  you realize the amount of work that went into creating the realism of skin, muscles, facial capture and texturing. The Avatar characters themselves (in my opinion) do not have as much realism and look more clay like, probably not helped much with the blue skin, but they would look just as clay-like if they had more normal Caucasian skin tones. Maybe this is what the Director wanted, but it puts an ever so subtle block to the suspension-of-disbelief and breaks the emotional bond between an audience and the on-screen actors. (Again, just a personal point of view)

We also have to remember that Matrix Reloaded was completed in 2003 – and according to many of the quotes floating around on the internet – The “vision” behind Avatar could not be completed earlier than 2009 because technology had not caught up. This is the reason to draw parallels between the technology prowess used in virtual cinematography in the Matrix and how good it was. Physical camera tracking, so called “simul-cams” could be tracked even back then as is evident in this other system

There is un-deniably a lot of rich and exquisite eye-candy in Avatar, and helped even more by the fact that it was rendered in stereoscopic 3D which adds a whole different level of realism and immersion to wow audiences. While this is not an article to be-little the effort that went into making this true block buster, it is hoped that aspiring Cinematographers and Directors are not intimidated by the huge budgets and arguably, some hype that went about in the way Virtual Cinematography was done with glossy terms such as the “Simul-Cam” (physical tracked camera) and the use of a multi-million dollar 3D rig to shoot the film.

Technology cost and innovation changes in a matter of months if not a year – and what used to fill a truck load or a full height rack frame of CG equipment, can now be layed out on a desk or two to create Virtual Cinematography.

Box Office Marketing: Avatar – The right place at the right time:

Traditional Advertising agencies are soon realizing that conventional advertising is dying out, and media platforms such as Twitter, Face book, Social Networks and viral marketing is the way a message is spread. It is no mystery why Avatar was also so successful – It has Stereoscopic 3D, a huge much anticipated build up and these tools to spread the word like wild-fire that what was going to be released was an entertainment extravaganza unlike anything before.

In almost all aspects it was! – The only exceptions to take note of are the claims (if any) of ground breaking use of Virtual Cinematography and what we call Hybrid Cinematography . Again to re-emphasize, this is not to belittle what has been achieved with Avatar, it is a spectacular feature film that showcases the best of technology in cinematography and is produced in stereoscopic 3D. This article is to primarily make aspiring Cinematographers and up coming movie makers aware that Avatar like budgets are not needed to do Hybrid movies or get into Virtual Cinematography.

It’s possible to have super-computer like performance on a desktop today, at such a low cost, that it would be un-imaginable to think of such things just 2 to 3 years ago. Clever programmers, cheaper motion capture equipment and companies such as Nvidia and their Tesla GPUs (Graphics processing Units) are replacing banks of equipment that were both cumbersome and expensive before.

The Stereoscopic 3D Connection?

As mentioned, the entire fight scene in the Matrix Reloaded was computer generated. If indeed no shortcuts were used (i.e flat billboard 3D models) and the scene was actually modeled in 3D, wouldn’t it be great to dust off the old hard drives from the Zion mainframe (the computer system actually named by the production team) and re-render with a second CG camera … so as to capitalize on additional revenue from the recent 3D craze?

Imagine a blue ray 3D “Single” release,  of the Burly Brawl in 3D – beats the hell out of a 2d to 3D converted sequence.