Stephen Lawes, creative director and co-owner of Cantina Creative, Venti Hristova, graphics and HUD supervisor, Wes Sewell, stereo supervisor, and Jayse Hansen, freelance screen designer/animator, had their work cut out for them on Marvel's The Avengers, which premiered May 4th and has already passed the $1 billion mark. The movie, directed by Joss Whedon, features Marvel's most popular superheroes: Iron Man, The Incredible Hulk, Thor, Captain America, Hawkeye, and Black Widow. For Lawes, Hristova, and Hansen, the most challenging on-screen graphics job was the creation of the heads-up displays (HUDs) in the film, used for two versions of Iron Man's suits. The team also created the on-screen graphics to fill in countless digital computer screens throughout the movie. Here, Lawes and Hansen talk about using Adobe Creative Suite Production Premium for the project.
Copyright 2012 Marvel Studios.
On-screen graphics set the stage
Adobe: First, tell us about the movie. Did you have any idea it would be such a huge success?
Lawes: With a “superheroes save the world” plot, a great director and cast, there were all the ingredients for a huge movie, but the visual effects and action are particularly epic. We knew we were working on something special, but when we hear things like it scored the biggest Saturday gross of all time and has become the third biggest grossing movie in history – it's pretty amazing.
Adobe: What was Cantina Creative's role on the film?
Lawes: Cantina was responsible for the on-screen graphics in the NASA JDEM Lab, S.H.I.E.L.D. Helicarrier bridge, science lab sequences, Stark Devices, and for Iron Man's HUDs. Heading up the team was Venti Hristova, VFX supervisor, Sean Cushing, VFX Producer, and myself as Creative Director. We were initially hired on The Avengers during preproduction to help design and animate all the practical monitor graphics which would be used onset in the Helicarrier. We worked with the production designer James Chinlund and the production VFX Supervisor Janek Sirrs in order to establish a visual language for the Shield graphics. We later animated a wide variety of practical monitors for other environments as well-all with their own design sensibility.
After production, we continued by designing and animating the glass monitor graphics on the Helicarrier and any green screen story point graphics that were required. Simultaneously we created the two Iron Man HUDs (the Mark 6 and the Mark 7) and we ended by animating and compositing all the final 2D and stereo comps for the HUD interior shots and the POVs. Screen designer Jayse Hansen worked with us on the Hill/Fury screen designs as well as led the Mark 7 HUD design.
Adobe: How did you approach the on-screen graphics?
Hansen: For the Helicarrier screens we did lots of research on real aircraft carrier instruments and panel layouts. Not just the software, but the hardware as well. For my work on Hill and Fury's glass multi-panel-screens, I was specifically interested in command-center dashboard-style layouts such as those used in the space shuttle. I began to think of the Helicarrier, in its base sense, as a military combination of a helicopter and a boat, so the design integrated elements common to both. I was already familiar with naval interfaces from my work on 2012, but these screens presented a lot of new challenges. I've never seen a film with so many screens occupying so much screen-time. Venti (Hristova) and Lily (Shapiro) did an amazing job with the monumental task of just keeping all of them organized and on track. Basically, in 70% to 80% of film you are seeing our designs in the foreground, background and sometimes even between the actors' faces as they have conversations. A lot of the computer screens were filmed as simple blank glass screens. We then filled them with appropriate graphics designed in Adobe Illustrator and animated in Adobe After Effects. In many cases, the actors would improvise interactions with the screens and we had the challenge of designing content that corresponded to their actions. Luckily, we had some great talent to help make it happen.
Copyright 2012 Marvel Studios.
The HUD takes center stage
Adobe: So the HUDs were particularly ambitious. Tell us more.
Hansen: Yes, we had quite a number of unique story-points to tell within extremely quick shots. The HUD was a character, Tony's computer named Jarvis, and each shot had a specific message to deliver. We designed new elements for each of these. Whedon also wanted to have a lot of new bells and whistles incorporated into Iron Man's new Mark 7 suit. That meant designing a new HUD that would have several modes – Diagnostic, Battle, and Flight, along with some rather intricate new widgets like a 2D radar-map that could unfold into a 3D representation of his environment, trajectories and potential threats.
Adobe: What were the instructions for the new HUD?
Hansen: The new HUD had to look and feel like previous Iron Man HUDs, but with amplified sophistication and detail. I filled three notebooks with sketches and concepts that I showed to Hristova, Lawes, and Cushing. Their input was essential in keeping it within the Iron Man universe since they had worked on the previous Iron Man films. I designed all the widgets, icons and panels in Illustrator, converted them to layered Adobe Photoshop files and brought them into After Effects for animation. After the initial tracking was done, we could then deliver a fully composited HUD, sometimes within a day or two, to get Joss and Marvel's feedback.
Adobe: How did you come up with the basic design concepts?
Hansen: I basically taught myself how to fly. I used flight simulators, videos, books and the help of an A-10 fighter pilot. I asked him how he would want his “Ultimate HUD” to work in various situations. That information helped inform the design, and he explained things like relations between true air speed, ground speed, and vertical velocity indicators. I used that as my functional foundational, but added my flair for untraditional interface design and research into new fields like network-complexity mapping to come up with something unique, yet still very plausible, for Tony Stark's most advanced suit to date.
Copyright 2012 Marvel Studios.
Adobe: What was the workflow?
Lawes: Both Venti and I visited the Albuquerque set a handful of times to assist with the on-set playback and HUD shoots. The HUD's were shot entirely on an ARRI Alexa with a Frazier lens, and once edited, the footage was sent to us to start designing and animating. The footage was also sent to StereoD for the stereo conversion of Robert Downey Jr. (RDJ)'s head. We had devised a system and rig on Iron Man 2 that we upgraded for stereo within After Effects. It utilized a virtual representation of RDJ's head in a 3D environment surrounded by the HUD graphics. Once we got the footage, we tracked it in 2D using the point tracker in After Effects. We put point trackers on the inside corners of both of his eyes and the end of his nose. From that, we averaged those tracks and shifted out the average in z space to get rotational values.
Adobe: What were some of the main challenges?
Hansen: The Avengers was the first time the HUD had been done in stereo. On the previous Iron Man films, there was a lot of blurred graphics for depth, but in stereo depth of field doesn't work as well. So we quickly realized that the amount of detail in these HUDs needed to hold up to close scrutiny. We focused a lot of time on how widgets and graphics would actually function because everything was clearly readable. Everything in the HUD, even down to the tiny micro-text, relates precisely to the current story-point.
Lawes: A challenging HUD design was for a scene when Iron Man is about to attack an army. Joss had the idea of making it spin from one mode to another to make it more dramatic. So Jayse created a rig variation that could spin the entire HUD 180 degrees. In a sense it was two HUDs in one: A standard mode and a battle mode. We used numerous Expressions in After Effects to get everything to tie together.
The eye reflections were another challenge. They are a small but incredibly important detail to sell the idea of the virtual space around the RDJ. They were especially challenging in stereo, because they needed to live on RDJ's eyes, yet they couldn't have their own stereo depth in the true sense of the word. If they did, they would have 3D depth inside them and would make RDJ's eyes seem artificially volumetric. We solved this by having the left and right plates use only the left-eye graphics. This meant the renders for the right eye composites were twice as long as the left, as After Effects had to compute one set of graphics for the eyes and facial highlights, and another set for what's actually shown around him.
Since the graphics are duplicated for eye reflections and facial interactive lighting, the elements on screen numbered in the tens of thousands of individual pieces. In fact variations of the Mark 7 suit diagnostic contained more than 1,000 elements alone. We pushed After Effects hard, and it rose to the task.
At the end of the day, designing the stereo HUD opened up a whole new realm of exciting opportunities. It made for an incredibly fun experience.
Watch this silent video to see a build of how the graphics for Robert Downey Jr.'s Iron Man heads-up displays (HUDs) were made. Copyright 2012 Marvel Studios.
Production Premium drives efficiency, innovation
Adobe: Compared to other tools, how did Production Premium allow you to accomplish your work better?
Lawes: There was a definite consideration on whether to use Nuke on this job, but when you analyze what type of work you end up doing, it is definitely biased to animation/motion graphics rather than strictly VFX compositing. After Effects has the best of both worlds and with a great connection to Cinema 4D, which was also part of that pipeline, we had ourselves a great and consistent pipeline to produce content.
Hansen: For interface design for film, I design everything in Illustrator. It's simply the most flexible in terms of generating original icons, complex radial panels and grid-based layouts. There is simply no match for these tools. With After Effects we were able to go straight from design to animation, integrate cameras and nulls from Cinema 4D, and quickly turn around complex shots and revisions.
Adobe: Was there something you would not have been able to accomplish without the help of Adobe solutions?
Lawes: To be fair, no. There are plenty of great tools out there than can do a similar job. The key, however, is the direct connection between each of the Adobe tools that makes for an incredibly efficient workflow. That, and they are based squarely in the realm of being creative. It just makes it more fun.
Hansen: With the rapid pace of turning around shots and shot revisions, I wouldn't want to imagine doing it without Adobe's core design and animation tools.
Copyright 2012 Marvel Studios.
Adobe: We understand you've moved to CS6 Production Premium. What are your impressions?
Lawes: The 3D tracking, mask updates, and persistent cache in After Effects CS6 are all really valuable additions. I think also the connections between Adobe Premiere Pro and After Effects seem to be maturing well, which will help us re-think our pipeline setup. I'm quite intrigued with the revised SpeedGrade too, and seeing where that is headed.
Hansen: There is a ton in After Effects CS6 that I'm super excited about. I think the new 3D tracking and 3D layer engine will dramatically change the way I work and design. The persistent Global Performance Cache is a dream come true. I currently use a dedicated SSD drive for proxies, and I plan to make it the cache drive as well. And, it may seem like a small thing, but I'm very happy about the dark interfaces for Photoshop and Illustrator. In Illustrator, the new pattern generator looks really cool, as I actually create a lot of grid-tech patterns in Illustrator for easy re-use on screen design projects. The new searchable layer systems are also welcome – as my screen design layer-stacks get very dense, very quickly.
Adobe: Did any of the new features in CS6 Production Premium open up new creative possibilities or inspire ideas for new projects/viewer experiences?
Lawes: You never really know how these new features will eventually find there way into the work we do. The technical and creative aspects of this are so intrinsically intertwined it's always a fun journey figuring out how to utilize a new feature in an unusual or unexpected way, so only time will tell.
Hansen: The ability to bend layers and extrude shapes was the one thing I really would have loved to have had on The Avengers– particularly for the HUDs. It will be getting lots of use in subtle ways going forward in my future designs. I think there's a lot of new toys to play with in this release- many of which I'm sure we will soon be wondering how we ever lived without.
Learn more about Adobe CS6 Production Premium and Adobe Creative Cloud
Download a free trial of Adobe CS6 Production Premium (includes After Effects, Adobe Premiere Pro, Photoshop, Illustrator, and more)
Go watch the movie
All images and video are copyright of Marvel Studios 2012.