VoCo is, probably, the most disturbing piece of software presented at Adobe MAX. With it you can transform the words of a voiceover artist, even if the artist is dead, into… other words. Imagine the potential… and dangers.
I must confess that when I first saw the demonstration video for “Photoshopping Voiceovers”, a technology known inside Adobe under the name “VoCo” I considered it fun but nothing to think much about: only a lab experience that will eventually become real in a very distant future. Here at ProVideo Coalition we commented about the program and I considered the options given by VoCo dangerous, something that the demo video from Adobe also suggests. In it words are put into the mouth of television personality and comedian Jordan Peele, who hosted the show together with Adobe’s Community Engagement Manager Kim Chambers.
Imagine being able to change the words someone says, making them say something else, and I guess you’ll agree with me: there may be some causes for concern. Similar projects are in development, like Google’s WaveNet, which is said to be “able to generate speech which mimics any human voice and which sounds more natural than the best existing Text-to-Speech systems, reducing the gap with human performance by over 50% “.
Google indicates, though, that the processing power needed for WaveNet makes it difficult to bring it to the market, while Adobe already suggests that VoCo, which can be considered a “Photoshop for words”, can be used for multiple things. In Adobe’s own words, “when recording voiceovers, dialogue, and narration, wouldn’t you love the option to edit or insert a few words without the hassle of recreating the recording environment or bringing the voiceover artist in for another session? VoCo allows you to change words in a voiceover simply by typing new words.”
Independently of how dangerous the technology might be if it falls in the wrong hands, the fact is that there are some – good – uses for VoCo. Yes, I believe you can see the potential for this. Adobe even goes to the extent of showing a scenario: “You send your client a completed video project and they ask you to make a last minute change to the voiceover…but the voiceover artist is already on a plane to Hawaii. Well, thanks to the technology behind ‘Photoshopping Voiceovers’, this soon may no longer be an issue.”
You’ll have to watch the full video to see how it works. The demonstration was performed by Adobe Creative Technologies Lab intern Zeyu Jin, who at a certain point says “we’ve already revolutionized photo editing, now it is time to do the same with words”. While it may sound fantastic, we always have to remember the negative tones that the “made with Photoshop” brings about, from time to time. And the problems created by being able to us put different words in someone’s mouth.
On one hand, the idea one gets from Adobe is that the technology may never be available to the public, as so many other things that are shown at similar conferences. But on the other hand, during the demonstration some possible uses for VoCo are suggested, and Adobe even writes that “while these technologies are not yet part of Creative Cloud, many Sneaks from previous years have later been incorporated into our products”. Now, imagine you’ve a way to fully integrate VoCo with the timeline of Adobe Premiere Pro, being able to edit in realtime not only images and sound, but also voice…
VoCo reminded me of one thing: a sequence in episode 5 of the recent TV show Westworld where a lab technician, Felix, shows one robot, Maeve, how her thoughts and words are created, in real time, by a text to speech program running on a tablet. Although the situations are different, there are some analogies in the reality from Adobe – see the sequence where words in text are changed – and the Westworld clip where Feliz shows Maeve how he controls her words with his tablet.
While VoCo was, probably, the most disturbing presentation from Adobe MAX 2016, other moments during the show revealed interesting new technologies developed by the engineers at Adobe Creative Technologies Lab. Some of them are quite interesting for both cinematographers and photographers, suggesting new directons the technology is taking us to.
CloverVR is such a technology. With VR and 360° video becoming common, new tools for editing are needed and that’s what Clover VR does. Because you can’t edit 360° video while immersed in 360° video, the current process for editing VR footage is tedious and clumsy, requiring ad-hoc workflows and desktop plugins. CloverVR aims to solve this problem by providing editors dedicated tools to view and edit directly in VR.
Syncmaster is another technology presented at the conference that we might see coming to reality. Syncmaster helps video editors and motion designers to sync a key moment in a video to the perfect moment in a song. Another name to retain from the event is LoopWelder, which turns videos into perfect loops that you can use to create engaging content. With the explosion of digital and social content, you need more than a static image to make your post standout, and that’s where LoopWelder comes in.
Other technologies presented this November at Adobe MAX 2016 introduce new solutions for graphic artists. Adobe is also promoting new tools that relate directly to Adobe Stock, which they want to make more popular among different segments of the market. For photographers there were some new technologies that might well appear in future editions of Photoshop. Things like SkyReplace, a tool that makes it easy to choose from a series of skies the one that better fits in a photograph shot in a dull day. For reasons I will never understand, changing skies continues to be a hot topic when it comes to photography, so this SkyReplace may well find its way in the desktop of many photographers.
Adobe presents SkyReplace saying this: “Have you ever taken an awesome vacation selfie but wished you could change the sky to be as amazing as the foreground? Well, now you can. SkyReplace allows you to quickly swap out the sky in your digital photos.”
I must admit I am puzzled, though. I understand a graphic artist may need to change a sky. But vacation snaps? It must be me, but I always thought photographs, special those from vacation periods, were supposed to be memories of the moment. So, instead of saying “see this picture of me at the beach in Cuba late afternoon on a dull day” you’ll talk about the “glorious sunset picture” of you, knowing full well it’s a stock sky taken by someone else…