Robert Fisher Jr. most recently served as lead editor on Sony Pictures Animation’s computer-animated Spider-Man: Into the Spider-Verse, now in theaters. Robert has been a member of the Sony Pictures Animation family since 2009 having worked on both Cloudy with a Chance of Meatballs films and the short, The Smurfs: A Christmas Carol.

Prior to joining Sony Pictures Animation, Fisher served in editorial roles on Disney’s Aladdin and the King of Thieves, The Tigger Movie, and Beauty and the Beast: The Enchanted Christmas. His additional credits include serving as executive producer/writer/editor on Murder in Small Town X, an unscripted drama series for Fox Television and Fox Broadcasting. He was also an editor/story editor/director/producer during the first four seasons of MTV’s ground-breaking reality television series, The Real World. Robert began his career as an associate producer and post-production supervisor for many television and feature projects.

(This interview was transcribed with SpeedScriber. Thanks to Martin Baker at Digital Heaven)

HULLFISH: I’m assuming — having done a little bit of animation editing myself — that this is a typical animated film instead of more of a live action type edit where you are working at the end of the process. You’re editing at the beginning of the process before the animation is occurring.

FISHER: That’s correct. Most animation — well most modern animation anyway, it was different earlier — is that as animation editors, we’re involved from the very, very beginning — including helping to develop the story. Some live action editors don’t really understand what an animation editor does. The analogous situation in live action would be: if you were invited to the first story discussions with a director and writer and then to a table read where you would have input as an editor and helping to define moments and maybe add ideas for additional scenes or deletions or something into a table read and on through the whole process. Some are invited to do that in a live action film. With an animated film — especially the last few that I’ve been involved with — you’re in very close relationship with the directors, producers, writers, storyboard artists. We get together from the very beginning. As the script comes in, it comes in in pieces or sometimes sequences are storyboarded without the benefit of a script. It’s based on an idea. And we’ll continue that throughout the process by adding scratch dialogue, and in some cases just actors around the office that we would use for temporary dialogue and then refine the idea. So I think in animation it’s an ongoing process of refining and revising and trying new things and different things in different ways so that a cut of the movie may evolve over a much longer period of time than you would have in a live action film.

HULLFISH: You even get to speak into shots that you want, so as you’re watching storyboards, if you feel like, “I really need a big close up of his hand or his eyes” or something, you can ask for that.

FISHER: Correct. It’s analogous to having an editor on the set when you’re shooting a live action film. It’s not even as we’re watching a scene. Sometimes we’ll do that as we’re being pitched a scene so that the artist would basically walk us through a sequence and say this character walks from the door to here where he pulls a magazine out of the kitchen drawer and writes down a number very quickly. So we’d say, “I think we need an insert of that and I need a single on his face to understand why this is important and then perhaps an insert of what he’s written down.” So that could have happened at almost any stage in the process, including up to where the shots are being defined in layout. You could say, “This scene really isn’t working. It was envisioned as a oner, but it really isn’t communicating what the story needs of that sequence are.” So then we would talk with the directors and storyboard artists and come up with solutions that we could try.

HULLFISH: Tell me a little bit about the idea of the editor as a pitchman. Because when you’re cutting these scenes together even something like you describe where it’s a oner, that oner could be in the editing stage of storyboards you could be editing 100 storyboards into that oner, correct?.

FISHER: Easily. Before modern animation editing really started it was typical to have very few board panels per minute – which was a very slow editorial pace – so it communicated the basic ideas of the story, but not too much concerning pacing and camera coverage. That was all handled later in the process, in terms of layout and final shot selection and those things. But now, because storyboards are becoming more and more complete and more complicated — they’re also easier to produce with Photoshop and other programs, so now artists put maybe five boards describing key poses and certain actions where in the past maybe only one. So a minute-long one-shot scene that has a lot of action, I’ve had boards that have gone up to 500 for a minute sequence.

HULLFISH: Holy cow! That’s like a board every three frames!

FISHER: A few years back — not on this movie — I had 1200 panels describe a big battle sequence, and it was camera moves, it was all connected to other camera moves and it was extremely well drawn and well designed, but it took a long time to find the editorial pace that would support what the artist was trying to do and what the director was intending. So it has to be a combination of what I can bring to the table so I have to ask, “Do I really need 1200 boards to describe this action?” Or maybe I could simplify that. Maybe I could do fewer number panels so, in concert with the directors and storyboard people, we would come up with a cut that we would refine — and perhaps maybe the oner didn’t exactly work out the way that they had intended. So as an editor you can also help define solutions to the problem and say, “Why isn’t this working? Where’s the emphasis? Why did we lose the character in all this activity? What are the character moments that we’re kind of skipping? And those are the very interesting things that you only get as an animation editor.

HULLFISH: Could you tell me a little bit about your background.

FISHER: My checkered past… I started out in college as a theater major and then sort of migrated to film: camera and editing and lighting and I just stayed with it and got involved in all aspects of making films whether as a cameraman as an editor, director. I’ve done documentaries, commercials, and television and somehow – through all that – I fell into editing episodic television animation which was a whole different thing for me. It was really a process of assembly more than it was creative. We had to format the show and try to get it to time without taking out story elements and trying to figure out how to rearrange it and maybe tell a scene a little bit better than it was originally conceived. But from that, I just fell in love with having the ability to work in animation from the ground up.

My first feature was the sequel to Aladdin and I just enjoyed it and fell in love with the process. I really enjoyed the craft of it and it took a longer period of time to do so I could devote more thought to the day today: “Can I come up with a solution that I could pitch to a director? What’s wrong with the first part of the movie?” And so I kept returning to animation and did a few at Disney and then have been involved with Sony for the last 10 years or so.

There are places that you can go back in and look at the story reels for early animated films and it’s amazing how much different it is today. There were very, very few sound effects and it was pretty much dialogue and picture and occasionally some music; because that was all cut on film and nobody wanted to go to 64 tracks of audio at the time. (laughs) Now producers and directors are really dependent upon an editorial team to really put together almost a finished looking and sounding movie. Success in animation editing is really having a collaborative and talented team all on the same page. Ready to cut in dialogue, temp sound effects, and music as well as help take care of the myriad details and procedures in evolving the film, from boards to finished animation, from temp sound to the finished mix. I’ve been lucky to have such a talented and dedicated team on this movie.

HULLFISH: Let’s talk a little bit about that process. What was the schedule like?

FISHER: To date, our schedule was about two and a half years. We had a script and they were just barely into the storyboarding process when we started. At the time I had my assistant Sarah Cole and associates Andy Leviton and Vic Sharma and we put the first cut of the movie together in about 10 weeks with director Bob Persichetti. And as we did that, we definitely saw that we had a quite a story. It’s long but we have a very interesting character and a very interesting premise. So I think that powered everyone through even though we knew that at the time the movie is going to evolve and we looked forward to saying, “Here’s where we think we can do better.”

As we went along the screenplay was evolving. Some of the storyboard artists would invent scenes and sequences and show them to the directors and get a pitch or we would come up with an idea and talk about it with the directors and say, “Let’s give it a try.”

HULLFISH: Editors, of course, think of ourselves as storytellers and animation editing is the epitome of that really.

FISHER: I think it is. I’ve cut live action and I’ve cut documentary. With animation editing, what I find is especially rewarding is the relationships that you have with the writers and with the directors, producers, storyboard artists. With a good relationship, you feel free to make a pitch that is not going to impinge on anybody’s territory. It’s additive. And if it gets rejected it’s rejected it’s because it’s not the BEST idea. But sometimes it’s totally embraced because we’re part of a team.

All of us realize that it’s a collaborative effort. It’s an effort that takes really the best of all the departments and puts them together. Editorial is where you assemble all those elements and show the first cut of a reel that is the result of the efforts of everyone involved; here’s what we’ve come up with, and now let’s see how we can make it better.”

HULLFISH: As you’re constructing a scene in animation are you trying to build it from an audio bed and laying picture over it? Are you doing the other way around? Tell me a little bit about the process of moving through a scene from a rough to a more complete version.

FISHER: Well I don’t think there’s a single answer to that question about how it’s approached because it’s really different in a lot of different cases. Sometimes you start with a blank screen and you have scratch audio from your co-workers and other office people who like to do voices. And that’s on a script that may or may not get recorded as written. It may be ad lib. So you’re going to be trying to go through the best takes and put together a radio play of a particular scene.

At that time, if you come up with something that the directors think is good, you take that and maybe give it to a storyboard artist so they can sort of follow the idea in audio and they would give you storyboards that more or less would match what you’ve cut in the radio play.

But there are times where the storyboard artist would say, “Wow. What if we interrupted this radio play with this little bit of action or this little interlude?” Sometimes that happens. In a lot of cases it is cutting the audio first and sometimes it’s getting boards that have no corresponding audio.

It’s a slow, evolving process, but at some point, collectively, we may say, “That’s good. Let’s record that. Or let’s re-write that.” Then the writers and directors take that into a re-write and we cut it again. At some point, it all goes to recording. If there’s dialogue in the scene, then that’s cut against picture – or picture against dialogue – until you finally come up with a coherent sequence that hopefully gets across the story points the movie needs.

HULLFISH: Talk to me a little bit about pace. When I’m cutting something as a live-action editor, I have some control of pace, but a lot of the pace is dependent on the camera moves or the action that’s on screen or the pace of the dialogue and the acting. For you, as an animation editor, you have complete control over how fast any action or scene actually happens.

FISHER: Not entirely.

HULLFISH: OK, but you’ve got to use your imagination to decide how fast an action has to happen.

FISHER: Yes. A lot of times what we’re trying to do as animation editors is — by a certain number of panels you can predict what the shot will look like when it goes through layout and animation and finished lighting. If I have 10 panels that describe a certain action of Spiderman leaping into the air and shooting his web and taking off, maybe I have 6 7 8 10 panels that I’m trying to figure out: “Is this a leisurely activity that he’s doing? Is he in a hurry? Is he angry?” All that comes into account what you’re trying to cut the boards to a certain pace, and that’s just one shot in a sequence, so then you ask: Can I pick up the pace by maybe eliminating some panels? Or do I want to slow it down a little bit? Do I want to maybe increase the pace throughout the sequence by maybe dropping a shot that the artist and director’s had put in there and said, “we don’t really need it.” Let’s accelerate this. Or instead of waiting for an action to completely resolve in the frame, maybe I’ll cut out a little earlier and give the animators a guide to how I would envision the speed of the action.

So it is a great deal of using your imagination and then trying to communicate that to other people. Firstly to your directors and to the producers and also to people who are going to be handling layout. Then ultimately to the people who are going to be animating it because they’ll say, “OK I get it, because the way it’s put together with the sound and dialogue, effects, music, is all communicating a certain thing. We believe that we understand the shot from beginning to end and think we’re on board with the action taking place in this amount of time.” I don’t think any editors can be 100 percent successful in describing the pace of a shot in all cases. The animators are experts in character timing. They may ask for an extra foot or so in bringing the action to life. So we try it. Again it’s collaborative and it’s a back and forth process.

HULLFISH: The film has a really interesting look. It’s 3D but it’s not 3-D. It transitions between somewhat photorealistic backgrounds and stylized 3D then almost like comic book graphic style. Talk to me a little bit about the look of the film and how it was achieved.

https://www.youtube.com/watch?v=pSvblThigz4

FISHER: What the producers and directors wanted was an animated movie using the best 3-D CGI animation that we can, but also not ignoring what’s available in 2D and maybe we can also use some techniques and effects from comic books themselves like etch lines on a face and burst cards that come on and say “BLAM!” Thought bubbles and those things or title cards that would come on to say “Meanwhile in Queens.” These are things seen all the time in comic books but aren’t seen very often in film. Every department contributed — our VisDev department, our fabulous production design, and art team, animators, everybody. What would it look like — as a train is bearing down on Miles — to put six frames of what looks like a comic book illustration that interrupts the action? What would that feel like? They give me the frames, I give it a try and we go back and forth and we finally come up with a moment that I don’t think I’ve seen that a feature film before. So these techniques that are being used help tell the story, emphasizing a moment or heightening an action.

HULLFISH: So it sounds like everybody kind of threw their ideas into the hat and you would need to kind of do a proof-of-concept in the edit room?

FISHER: I think that’s what we did. I think most of those ideas for comic book illustrative techniques or what they called burst cards or what would be in the background of a fistfight like what a comic book would do. Those things I think primarily came from the directors, producers, and our story team. And then when the shot would come to me, I was a participant to say something like, “I think this is disruptive of the mood we’ve created here” or fully embrace it and say, “Let’s stop the action for a fraction of a second and maybe give us a visual jolt or unexpected event.”

I don’t think anything here was done just because we could do it. It was done really to enhance the storytelling and to make people feel like they’re involved in a comic book, even though they are watching an animated movie. It’s kind of a different concept but I enjoyed it.

HULLFISH: Were you cutting this in Avid or something else?

FISHER: I was in Avid.

HULLFISH: I’m sure that your timeline has multiple layers as you get through each process: storyboard, then layout, then rough animation, then lit animation. Can you kind of describe that process of turn-overs and sending one iteration off and then getting information back and putting it in your timeline. How does it progress?

FISHER: I actually like to limit the video tracks to as few as possible. A lot of animation editors do carry every version of that sequence or scene in their timeline even though it’s irrelevant to what they’re currently cutting.

My assistants and associates and I have developed a system that would archive cuts that are easily retrievable so that we could go back to older versions, but we try to keep the timeline as small as possible when we’re going through and doing storyboards.

Instead of an 8 layer or 10 layer elaborate effect, we would build that as a test and say, “That looks pretty good. What happens if we collapse that down?” Maybe we’d even finish it in After Effects. We can put it back into the timeline. All the while archiving that 8 or 10 layer effects sequence somewhere so we can get at it, but trying to understand that we have to move quickly and we the faster that we can get to the relevant picture materials the better, so we use just a few video layers for storyboards and as they get older we replace them.

So in that way we’re trying to stay as current as we can but still have some access to archived material very quickly so we can go back and say, “Remember that shot that we did a year ago?” Well, the system that we’ve designed we can go back and get that rather quickly and incorporate it into the new material.

So once that process is done and you come up with your final storyboard shots approved by the directors and producers, then we would basically divide that into shots and go through the movie or go through the sequence with the director and divide the sequence into shots. During this part of the process, we can decide to eliminate or add shots. Always revising and fine-tuning to make the storytelling better.

Launching into layout is a complicated process that probably no one would really be interested in, but it’s a process of turning it over to camera layout so they understand where the shots are (where each shot begins and ends). When it comes back, it’s basically block figures — Lego-like figures describing what actions are and the basic camera set-ups. We go through several iterations of that, so that’s several layers of layout, but we remove the older layers and put the newer ones in. We can easily go backward.

Then finally it goes into animation where you get rough animation or intermediate animation then it goes into lighting and each of those requires another video layer or two that you’re trying not to keep a full archive in the timeline. Trying to represent the various stages so we could always go back and say, “What did this shot look like in layout?” Or what did it look like in boards? The layout crew often created something better than what we ended boards. In some cases, our layout team created shots without the benefit of storyboards. That’s the basic fundamental process. Eventually, you have a complete movie that’s in lighting and then it’s turned over to finishing people — in this case, it’s E-Film, and we would turn over Avid cut and the studio would deliver the final files and that would be conformed for the DI where the color grading would take place.

HULLFISH: When you go from storyboards to the layout some of the energy or art or emotion is lost between the storyboard process and the layout process. That is a dangerous place for an animation editor, I think, because you’ve lost the emotion and you might succumb to tightening or feeling like something’s not moving you anymore. Do you have to fight against that?

FISHER: Personally I don’t because all the way I’m imagining what the shot would look like and sometimes I’m leaving time at the end of a sentence for the look to change to make the nuance in delivery of that line. All of that intention is lost in layout which is why — once I look at layout — I rarely look at the layout as a sequence. I would always go back to the boards until it’s more complete and then once it comes into animation I rarely look at the boards.

Layout is very rudimentary and you do lose a lot of intention in layout. But that’s not the intention of layout. The purpose of layout is to describe the shot in terms of camera and blocking. This shot uses a certain lens. The camera is a certain height, it’s moving left to right, tilting, booming down, etc. All that is occurring in layout. The acting in layout is secondary.

That being said, layout artists are getting better and better all the time and using the tools that they’re given. At times, it’s starting to merge with rough animation.

HULLFISH: Layout is also concerned with the characters and other objects getting from point A to point B in 3D space, right? Sometimes something that’s drawn in a board just can’t happen in 3D space.

FISHER: Right. Sometimes the layout artist will call you and tell you that they’re having a really hard time getting this character across the room or to do whatever they’re asked to do in the script in the time that you’ve allocated. Then you go look at the shots with the layout artist and say, “Let me give you another few seconds or another couple of feet to do what you need to” and then you go back into the sequence. But I have found that changing the length of one shot usually changes the pace of the cut, sometimes for several shots around it. At times, you may have to go back to the three or four shots around it and try to figure out how to keep a certain editorial rhythm while giving the animators and layout people enough time to accomplish the action.

HULLFISH: I wanted to touch on the fact that you needed to leave space for a look. Obviously, you’re not going to see that look – the actual motion and emotion – until much later in the animation process. So much of the pace – or the amount of time a live-action editor leaves at the end of a shot – is based on the motion within the frame – eyes turning, an expression revealed, a turn… but you are having to imagine that and trust that in the end, it will be there to sustain that pacing decision.

FISHER: Yes. What I try to do is imagine what it would be like in a live action film. In my mind’s eye I can see that this needs a little extra room after the sentence before the cut. I’m trying to lead people in a natural way to say, “if you were a full participant in that scene, where would your eye naturally go? There’s a lot of experimentation, which I really enjoy, and I think it helps to tell a better story and get a smoother performance.

HULLFISH: I love the idea that a live-action editor can pull a reaction from a different place in the scene – or even another scene – to illustrate an emotion or thought, but you’re actually being even more intentional than that because you can actually determine the exact expression or motion that reveals the expression or helps the story.

FISHER: I think so, because I’m not reliant on what was shot. I’m only reliant on the intention of the scene and trying to tell that the best way I can – imagining the most perfect reaction shot that’s ever been, and then it’s up to the animators to create the moment.

HULLFISH: Thinking about ego – do you save a place for a reaction at the end of a line and then the director says, “Tighten up the ending.” Do you have to pitch the director on the fact that your hold at the end has a purpose? Or is the director imagining along with you?

FISHER: It works differently with every director that I’ve worked with. Some – as you say – are imagining along with me and can picture it – they aren’t seeing what is in front of them, they’re seeing the final animation in their head. They’re listening to the rhythm of the speech and in their head, they’re already considering what they want to direct the animators to do or to add to that – or not. Sometimes it just doesn’t fit with the way they see the pace of the scene, and in that case, it’s analogous to live action, where every editor can probably cut the same scene very different ways and still be successful. But the director may look at it and say, ‘That’s not how I imagined it or where the emphasis needs to be. It’s a collaborative process and if I feel I think it will help communicate my idea, I’ll say, “I want to wait until he drops his eyes and looks away.” And if the director is open to it, I’ll ask the storyboard artists to give me another panel or two to represent that pause or that action.

HULLFISH: It’s got such an interesting look. Was it all animated in Maya in 3D or what’s the production technique that I’m seeing that makes this look so original?

FISHER: It was animated in Maya and various other things. But it’s really in their treatment of it where the differences are coming out because Bob Persichetti – one of our directors – was very involved in handling animation and coming up with a style with our production designer and art director that he liked. He provided a vision of what he wanted it to look like – animating on twos in most cases, where every two frames there’s a change (essentially animating and playing back at 12fps instead of 24fps) But perhaps mixing it up in the middle of the scene – starting animating at twos and going to ones, or maybe the background animation is on ones and the characters are on twos, making it look a little more stop-motioney. Put us in that world of comics rather than live action even though the surfacing of the characters and the buildings and everything may be very photorealistic in a scene. But the way that the animation is handled never takes as far from the world that we’re trying to stay in. It’s a comic book world.

Most of it is really handled in the way that things are treated. along with the texture and comic book bubble patterns and etchings on the face. They’re actually going in and hand-painting or hand-finishing on the character frames, so that there are etchings on the face to describe browlines and things that would give the character face more emotion but not the typical CGI movie.

HULLFISH: Interesting. So they’re drawing directly on the 3D model.

FISHER: Exactly. How it was accomplished is beyond me, but I think that was very effective. As we were lighting shots and they were being finished, we could go back in and see what it looked like without the line etching or sometimes there’s very very little motion blur. That blur would be represented in different ways, for example, we would use four poses in two frames or one frame. When you look at it in motion it looks similar to motion blur but in this case, it’s totally unique to this movie.

HULLFISH: Let’s talk about temp music.

FISHER: Music we temped for the first year or so with scores from different movies and songs that the directors would bring in. Let’s try some old-school hip-hop or something more contemporary. Then they brought in a music editor – Katie Greathouse and a music supervisor who worked closely with the producers and went through the process of – like – what would happen if we replaced the temp orchestral score with a song? What will be the dynamic? It is best to remember that temp score is temp. We use it to try and demonstrate an idea of how the final score can contribute to the emotional/action through-line in the scene. That process was all part of the last year. It was an iterative process.

HULLFISH: All editing is an iterative process, right?

FISHER: Always. We’re always looking at the best way to tell the story. It’s not just to make it different looking or sound different but to push the envelope a little bit to see what other kinds of tools that we can use to get inside the characters, or get inside the emotion of the sequence. Maybe coming at it from a non-traditional angle but achieves similar results in putting the audience in the characters’ situation or appreciating what’s happening to the characters. Cheering them on. We wanted to attempt to do things maybe a little nontraditional but that wasn’t the first intention. The first intention was to try to illuminate the character’s emotion and tell the story along the way.

HULLFISH: With the exception of some movies, the sound design and mix is only turned over at the very end. How is it on an animated movie? Or on THIS animated movie?

FISHER: Well most times we had a very complete track in the Avid. And as soon as we got close to the first preview we turned over all our temp sound effects and dialogue. Then our sound designers and our sound supervisor – Geoff Rubay – said “Okay, now let’s put some development into these sounds and make them unique to this movie. What does Miles’ web shooter sound like? Different than the first Spiderman’s? Or is it similar but has a little bit of difference? And the fantastical things that are done in this movie, what are they going to sound like? How can sound help push the story?”

I think they’ve come up with some very unique sounds and those were developed throughout the whole process. They’ve really created a soundscape that enhanced the movie through five or six temp previews and then onto the final mix.

HULLFISH: From those screenings, did stuff change much?

FISHER: I think the process of having quite a few preview screenings helped – producers, directors, writers – everybody in terms of how a real audience is going to see something this different. The first ones were very challenging to present because they were mostly storyboards that in very, very few cases would we ever show a non-professional audience. So our attempt was to say “Here’s a full movie but it’s in storyboard, so it’s very different than what a lot of the recruited audiences were expecting.” But even with that, I think we were all gratified that the movie played as well as it did in early previews and that would give us an idea of what scenes are working and what actions weren’t explained properly.

HULLFISH: I really appreciate you giving me so much of your wisdom in time today.

FISHER: Thank you for your time, too.

To read more interviews in the Art of the Cut series, check out THIS LINK and follow me on Twitter @stevehullfish

The first 50 interviews in the series provided the material for the book, “Art of the Cut: Conversations with Film and TV Editors.” This is a unique book that breaks down interviews with many of the world’s best editors and organizes it into a virtual roundtable discussion centering on the topics editors care about. It is a powerful tool for experienced and aspiring editors alike. Cinemontage and CinemaEditor magazine both gave it rave reviews. No other book provides the breadth of opinion and experience. Combined, the editors featured in the book have edited for over 1,000 years on many of the most iconic, critically acclaimed and biggest box office hits in the history of cinema.