Shoot first think about it after is a simple way to explain it, but it seems to be what Lytro Cinema offers. Forget about focusing and camera angles, because you’ll be able to do it in post-production. Is this the future of video? Is Lytro Cinema, “the ultimate creative tool”?
Well, according to some it may well be. The first short film produced with Lytro Cinema, “Life”, will be premiered April 19 at NAB 2016, produced with Academy Award Winners Robert Stromberg, DGA and David Stump, ASC in Association with The Virtual Reality Company.
According to the information provided by Lytro, it unlocks “a new level of creative freedom and flexibility for filmmakers with the introduction of Lytro Cinema, the world’s first Light Field solution for film and television. The breakthrough capture system enables the complete virtualization of the live action camera — transforming creative camera controls from fixed on set decisions to computational post-production processes — and allows for historically impossible shots.”
With Lytro Cinema, every frame of a live action scene becomes a 3D model: every pixel has color and directional and depth properties bringing the control and creative flexibility of computer generated VFX to real world capture. The system opens up new creative avenues for the integration of live action footage and visual effects with capabilities like Light Field Camera Tracking and Lytro Depth Screen — the ability to accurately key green screens for every object and space in the scene without the need for a green screen.
By capturing the entire high resolution Light Field, Lytro Cinema is the first system able to produce a Light Field Master. The richest dataset in the history of the medium, the Light Field Master enables creators to render content in multiple formats — including IMAX, RealD and traditional cinema and broadcast at variable frame rates and shutter angles.
“We are in the early innings of a generational shift from a legacy 2D video world to a 3D volumetric Light Field world,” said Jason Rosenthal, CEO of Lytro. “Lytro Cinema represents an important step in that evolution. We are excited to help usher in a new era of cinema technology that allows for a broader creative palette than has ever existed before.”
Designed for cutting edge visual effects (VFX), Lytro Cinema represents a complete paradigm shift in the integration of live action footage and computer generated (CG) visual effects. The rich dataset captured by the system produces a Light Field master that can be rendered in any format in post-production and enables a whole range of creative possibilities that have never before existed.
“Lytro Cinema defies traditional physics of on-set capture allowing filmmakers to capture shots that have been impossible up until now,” said Jon Karafin, Head of Light Field Video at Lytro. “Because of the rich data set and depth information, we’re able to virtualize creative camera controls, meaning that decisions that have traditionally been made on set, like focus position and depth of field, can now be made computationally. We’re on the cutting edge of what’s possible in film production.”
Technically the idea sound interesting, but one should remember that it also sounds very much like the promise of Lytro photo cameras, especially on the aspects of not having to care about focus and depth of field… because it can be selected in post-production. The problem with the technology is that although it sounded revolutionary – and it probably is – it never worked in the real world, and the results were not good enough to keep Lytro into the photographic market. So Lytro confirmed last week that it had put an end to its line of photo cameras.
One week later it announces the Lytro Cinema, which continues the exploration of tools for cinema, after a first note from November 2015, when Lytro presented Lytro Immerge, the world’s first professional Light Field powered solution to provide lifelike presence in VR with Six Degrees of Freedom (6DoF), addressing the challenges of live action immersive storytelling.
Lytro Cinema sounds very much like the virtual cameras used in video games, which follow a narrative very close to the models of cinema, but are not bound by the usual limitations of placing physical cameras around the set. In that sense, what Lytro Cinema promises is exciting, and can/could really solve many problems, but I am not so sure the possibilities it offers will allow for a “broader creative palette than has ever existed before” through “transforming creative camera controls from fixed on set decisions to computational post-production processes”. What this means is that, if I get it right, the style of different authors like Stanley Kubrick, J J Abrams, Wes Anderson or Ingmar Bergman, to mention a few, would be lost in a Lytro Cinema film, because the director’s options at the time of taking can be completely changed in post. In fact, the way Lytro puts it – as they did for the photos from the Lytro cameras – is that everything can be changed in post.
According to information on the company website, post-capture refocus and extended depth of field are not set in stone with Light Field cinematography, as “creative camera controls transform into flexible post-production processes, liberating shots from restrictive on-set decisions”. This means it is possible to “focus the camera near or far, control the size of the aperture, and create a shot as if that exact decision was made on set.” The Lytro Cinema also allows users to “adjust frame rates and shutter angles in post-production, so motion blur becomes a flexible creative decision” and break away from camera settings that are usually “locked” on set and optimize renders for multiple distribution channels.
For me, as a photographer, this sound frightening. I sincerely believe that when we photograph or create a video we take options that reflect both our style and the theme we’re working on. Lytro Cinema suggests, apparently, that everything can be changed in post-production, meaning that anyone could pick one Wes Anderson movie and alter it to be… something else. Is this the future of cinema?
Fix it in post?
To some extent we’ve seen this same idea of “fix it in post” with 4K and beyond, from people who say that with more pixels we can just reframe areas of the 4K image (or 8K or 16K, in fact) to create multiple FullHD (or 4K from 8K and 8K from 16K…) perspectives. While in practical terms it may sound like a good working solution, it completely destroys the idea we’ve that different directors have a unique view which makes part of the appeal we feel when we see their movies. Not to mention the fact that with Lytro Cinema the focus-puller profession is going to become a thing of the past.
Maybe I am wrong, but these are the thoughts that first crossed my mind reading about the Lytro Cinema project. I was very critical of the Lytro light field photo cameras when they arrived, although some magazines and websites wrote, at the time, that with these cameras we could forget all we knew about photography. Well, time confirmed I was right. The market does not want, at least for now, cameras that allow you to focus where you want, after the shot was taken. Do cinema directors want a video camera or system that allows anyone to change their options in terms of framing, point of focus and depth of field?
Whether you like the idea or not, the Lytro Cinema will be on show at NAB 2016, so it you’re going to Las Vegas it may be a good idea to visit Lytro’s booth. The system comprises a camera, server array for storage and processing, which can also be done in the cloud, and software to edit Light Field data. The entire system integrates into existing production and post-production workflows, working in tandem with popular industry standard tools.
Lytro Cinema represents, also, a step function increase in terms of raw data capture and optical performance. It uses “the highest resolution video sensor ever designed, 755 RAW megapixels at up to 300 FPS”, offers “up to 16 stops of dynamic range and wide color gamut” and has “integrated high resolution active scanning”.
“Lytro has always been a company thinking about what the future of imaging will be,” said Ted Schilowitz, Futurist at FOX Studios. “There are a lot of companies that have been applying new technologies and finding better ways to create cinematic content, and they are all looking for better ways and better tools to achieve live action highly immersive content. Lytro is focusing on getting a much bigger, better and more sophisticated cinematography-level dataset that can then flow through the VFX pipeline and modernize that world.”
“Life” the first short produced with Lytro Cinema in association with The Virtual Reality Company (VRC) will premiere at the National Association of Broadcasters (NAB) conference on Tuesday, April 19 at 4 p.m. PT at the Las Vegas Convention Center in Room S222. “Life” was directed by Academy Award winner Robert Stromberg, Chief Creative Officer at VRC and shot by David Stump, Chief Imaging Scientist at VRC.
The Lytro Cinema system will be available for production in Q3 2016 to exclusive partners on a subscription basis. For more information, visit Lytro Cinema.