An important element in Alex Buono’s Visual Storytelling 2 Tour was a quick but in-depth color grading session, which just goes to show how the walls that used to exist between production and post continue to come down. Alex mentioned a few color correction tools, but a casual comment about a product called LiveGrain caught my attention. He said it completely changed the opinions some big name directors had in regards to film stock. And these were the sorts of people who maintained that the “look” of film would never be able to be replicated with video.
We had an in-depth conversation about what makes LiveGrain different, how he’s seen the product impact projects, how he sees LiveGrain influence the “film vs video” debate and plenty more.
ProVideo Coalition: Briefly, can you tell us what LiveGrain does?
Suny Behar: LiveGrain is a real-time texturing tool. The tool’s design and purpose is to replicate the three dimensional texturing that occurs naturally and organically when shooting with motion picture film.
The LiveGrain interface
How is it different from other services that are out there?
Up till now, every effort to replicate the texture found when shooting film fell mostly into one of two camps.
In Camp 1, film grain particles were generated digitally, and then scattered through some randomizing algorithm on top of your source image by means of some method of overlay. In Camp 2, you would take a piece of film, scan it, and overlay it on top of the source image. In both these instances, we felt the true nature of films’ texture was not represented.
Film grain does not float above an otherwise clean image. It is deeply embedded in the creation of that image. All of the layers in the film emulsion carry different size silver particles that leave behind a particular imprint when washed in the bath. Those silver particles act as light receptors and respond differently to color and light input. Through our patent pending process, LiveGrain was designed to fully represent the entire characteristic curve of a film stock by analyzing the source digital image and creating a map to the full dynamics of a film stock for all exposures and colors.
What was the genesis of the service? Did it come from a need you identified, or a technique that you were able to develop?
I got my Masters Degree in Film and Television Production from UCLA a few years back, with a concentration on Directing and Cinematography. At that time, we were only allowed to shoot film. Video was starting to creep up, but its quality was vastly inferior to that of film. Early on, I realized that I was quite drawn to the textural imprint of film. It seemed to have a direct emotional impact on the story for me. However, I never quite realized what it would mean to lose that entirely until I started shooting more digital projects than film projects.
I think we are spoiled with some amazing digital cinema cameras today. They shoot incredible images. This might sound like heresy, but personally, I am not one of those people who is deeply attached to the color palette of film or its dynamic range (I understand and respect those who are, but I don’t fall into that camp). I feel like digital cameras today have an ever expanding color gamut that gives them the ability to capture a vast array of colors, and from a dynamics perspective, one could argue that, especially in the shadow areas, film has already been outdone. The ARRI Alexa camera for example, has the ability to see deeper into the shadows and with more accurate colorimetry than film which tends to become monochromatic in the toe of the curve. I recently did tests on the new Canon Log2 Gamma curve coming out in the new Canon C300 Mark II this Fall, and was able to read information 8 stops under key exposure. It was quite remarkable and way beyond what is possible on film.
Here’s the rub though: when you increase signal-to-noise ratio to achieve higher resolution and more dynamics you also end up with an image that is very sharp, and most importantly, very “quiet”. What I mean by that, is an image void of any organic movement or “life” in it. Akin to the repetition of a clean still image, rather than an image in constant motion, like that of motion picture film.
It is this loss that I was hoping to address with LiveGrain. Cameras should get sharper, and should have increasing dynamic range but I should not as a filmmaker be robbed of my ability to lace the image with textural definition that can further the sub-text of the story line. With film, you could swap stocks, pull and push process, cross process, flash, etc. While all these processes affected color and contrast, they also gave you control of tone and texture, and used wisely, could greatly impact the emotional arc of the story. I can still remember today the texture of films such as “Babel” and “21 Grams”. Rodrigo Prieto ASC, is a master of using texture to tell stories and I’m a huge fan of his work.
I have attempted to achieve this effect by using gain settings in digital cameras (under-exposing and pushing) but because most modern cameras use CMOS chips, you mostly end up with fixed pattern noise or multicolor video noise scatter. In my view, neither of those represents an improvement to the underlying image.
LiveGrain was used on Season 7 of HBO’s “True Blood”
On a creative level, what can LiveGrain do for and to a project?
I have been continually surprised at what we’ve been able to do for certain projects. From a simplistic perspective, it feels like we automatically seem to add “million dollar production value” to the projects. I am quoting this because it was said to me many, many times.
From a deeper perspective I think that the dithering that occurs from applying film grain seems to smooth out costumes, production design, hair and make-up without softening the image. Film grain cores fine detail, while accentuating other elements. Faces, for example, have been a wonderful surprise. LiveGrain seems to fill in the fine lines and wrinkles all while sharpening the eyes by contouring the eyelashes and eyes.
My wife is a costume designer and cringes when she sees her work and that of her fellow designers shot with some of the sharper cameras on the market. They are quite unforgiving. Every seam, every stitch, every single loose hair is visible and in focus, yet that is not how our human visual system behaves. When you stare into someone’s eyes close up, the rest of the world fades a bit. You do not focus on every pore of their skin or every stray hair of their eyebrow, you simply see their eyes and a softened version of the features of their face. Yet, with digital cinema cameras today, a close up, shot on a long lens, with no diffusion in the matte box, and displayed on a 60FT screen, will reveal all the harsh realities of being a human being and having an epidermis.
How have you seen the product push or impact the look of project?
We got a chance to work with DP Jas Shelton on a film that made its debut at this year’s Sundance Film Festival called “The Stanford Prison Experiment”. The film recreates a famous experiment conducted at Stanford University. The film is built in a structure that has the tension slowly rise through the piece leading you to the climactic resolution at the end, yet, it all takes place in the same location, with the same actors, wearing the same costumes the entire film. To compound things, the prison has white walls everywhere.
Jas contacted me and said that if he could shoot it on film, he would probably start with 3 perf 35mm, then subtly he would switch to 2 perf 35mm, then for the climax go to 2 perf Push 1 stop, subliminally introducing grittiness and tension into the scenes. Since their budget was very constrained, film was not an option, but Jas knew that we could recreate that progression faithfully using LiveGrain since he had used LiveGrain on the entire first Season of HBO’s show “Togetherness”, written and directed by the Duplass brothers. Ironically, I believe It was a good thing that they could not afford film on that particular show because if he had baked the negative as described above, he would have to pray that editorial didn’t move scenes around. Taking a scene designed for the ending and move it to the beginning, or turning a scene into a flashback, would have severely damaged the planned textural progression of the story.
With LiveGrain, you know exactly what your movie looks like. You have a complete and final version of the story, and you get to design a textural arc that fits your final film. That is what we did on Jas’s film and I felt after watching it again in the theater a few months later that it had an important emotional impact on the story.
LiveGrain was used on HBO’s “Togetherness”
Is this a tool designed for colorists? Or editors? Or both?
This tool is specifically designed for colorists. It is neither an editorial tool nor an engineering tool. It is designed to allow the DP and director to have a meaningful conversation with the colorist on what the appropriate textural feel of their story should be, and then to empower the colorist to implement that vision in an intuitive and interactive manner.
What sort of feedback have you gotten from your customers?
Knock on wood, we have only had great responses so far. Most of our customers are repeat customers or referrals from customers. We do not spend any money advertising as you can tell by our website. Every penny goes straight into development and making the product better every day.
Our biggest source of pride to date, I feel, is working with Martin Scorsese and Rodrigo Prieto ASC on the new HBO show “Vinyl” coming out soon. Rodrigo did a very extensive test shooting and comparing 35mm, 16mm, and several different digital cameras (with LiveGrain applied), and when offered their choice of medium to shoot the pilot they choose to go digital with LiveGrain.
That moment made the hundreds and hundreds of hours of development and countless sleepless nights all worth it. To have a masterful filmmaker such as Scorsese along with one of the most talented DPs alive today chose to use our product to tell their story was truly humbling.
Have there been any major surprises as you’ve gone about this process?
Yes there have been. Many people say you have to be young to make movies because as you get older, you know how bloody hard it is, and know too much for your own good. You psych yourself out before even starting. I would say the same for software development. Although I have an undergraduate degree with lots of computer science in it, creating a full-fledged living and breathing piece of software is a different story. I don’t know that I would have proceeded with this project if I had known just how many 20-hour days were involved.
Our software relies on intricate algorithms that have to perform millions of calculations per second. We analyze every pixel, of every frame, and have to perform many layers of transforms and return it to the frame buffer in real-time. This requires an enormous amount of processing power and a very clever development team to manage GPU, RAM and CPU resources. We have even developed our own visually lossless custom codec specifically designed to handle grain texture, that decomposes in parallel entirely in GPU, and can be read natively on Mac, Windows and Linux OS’s with no re-wrapping. If all that sounds like Greek to you, you would be extremely bored at our Dev meetings. It’s nothing but Greek.
I’m still with you, but the visually lossless custom codec that you developed is what grabbed my attention. That’s kind of a big deal, isn’t it?
It is a rather big deal indeed. We are dealing with something that is very fragile and delicate when handling film grain, yet uncompressed grain plates can be as large as 55MB/frame for an uncompressed 4K Full Frame image. Since our process involves using many different layers of texture all cycled in real-time, we had to come up with a very efficient way of encoding our assets, and yet, not cause any visual harm to the texture itself.
Preserving an analog film look is key to the authenticity of our texture application. When you combine that with the need for accelerated GPU processing and the need to be able to play our assets natively on Mac, Windows and Linux with no re-wrapping, writing our own custom codec specifically designed to handle texture became a necessity for us to achieve the quality we were after.
What’s the future of the product? Are you planning to one day have it at a price point that will make it available to much smaller productions?
It’s impossible to tell what the future holds. For now, there is no plan to make a consumer facing plugin for Avid, FCP or Premiere. Our hardware requirements are simply too onerous to be able to support that environment adequately.
The second part of that is quality control. We built this tool for artists. As such, I have a keen interest in placing it in the hands of artists and letting them produce incredible content with it. By focusing on working with established post houses, we can focus our LiveGrain training on professional and experienced colorists who we can then trust to adequately represent the capabilities of our product, and do some incredible work for DPs and directors out there.
From a pricing perspective, we represent a small fraction of what it would have cost to shoot on film. All our clients have saved between 70% and 80% of their film production budgets by using LiveGrain instead. Given our commitment and elevated current levels of continued development we need to maintain pricing that will allow us to keep improving and building on our product. I have high hopes that when our development costs soften a bit we’ll be able to find a way to be even more affordable to smaller productions.
So right now, this is a product that really only makes sense for large-scale film and TV productions?
Not necessarily. We have done mid size commercials, as well as low budget Independent Feature films. We do offer discounts to low budget features on a case-by-case basis.
What we have found is that the application of LiveGrain seems to add a considerable amount of production value to the final production whereby even independent feature productions find the cost well worth the results.
Does this mean all the arguments about how we’ll never be able to create the “look” of film have finally been settled?
With LiveGrain, we were not looking to address color nor dynamics. If those are the foundations of the film “look” then we have done nothing to settle that argument. If however, like me, the element of film you miss the most was your ability to control texture and nuance your story through the use of texture, then I feel we have made a significant dent in that argument.
This is was just a brief overview around what LiveGrain has to offer, but you can get in touch with Suny and the rest of his team to learn more.