The TrueDepth camera from the iPhone X is finally used to more than animate Animojis in Facetime. It can produce fast cartoon animations that can be used in everything from Facebook to broadcast.
Introduced in the iPhone X by the end of 2017, the TrueDepth camera used for Apple’s Face ID is able to capture accurate face data by projecting and analyzing over 30,000 invisible dots to create a depth map of your face and also captures an infrared image of your face. Its first innovation was also included in the smartphone: new animated icons called Animojis, which are nothing more than a new generation of emojis used with the iPhone X TrueDepth camera to “bring emoji to life in a fun new way”.
For a while the iPhone X’s exclusive face tracking power was only known for usage with Animojis in Facetime, what could be considered a waste of a promising technology. Now, with Reallusion’s LIVE FACE app, users can unlock its full potential by live data streaming facial mocap data to any desktop client.
The new LIVE FACE app live captures facial performances via an iPhone X to create 2D facial animations with CrazyTalk Animator 3 (CTA 3) on a Mac or PC, turning an iPhone X into a powerful production tool that anyone can use. With the iPhone X’s TrueDepth camera, and the LIVE FACE app for CrazyTalk Animator 3, users can instantly capture facial performances to animate illustrative or photo characters to produce fast cartoon animations, also for broadcast shows on TV, YouTube, Twitch, Facebook or any other live service.
Jesse Griffith, Chief Animator at the Jimmy Kimmel Live said that “LIVE FACE by Reallusion made my jaw drop, both literally and figuratively when I used it. LIVE FACE is the game changer I’ve been waiting for to execute funny, quick animations utilizing the iPhone X for live TV.”
“Thousands of dollars of cumbersome equipment and connections are being whittled down into an app that students can easily use on an iPhone X they already have, on or off campus”, according to John Blalock, Digital Applications Supervisor at the Chapman University, who added this to his comment: “Not only does LIVE FACE simplify the entire educational process, but it looks to bridge the significant gap that students face once they graduate and try to incorporate technology into their own pipelines.”
LIVE FACE for iPhone X can track up to 57 face signals through Wi-Fi, to drive character facial expressions including; head angle, brows, eyeball rotation, as well as controlling talking mouth shapes. Users can also import custom facial animation sprites, or convert expression sets directly from Photoshop layers. You can further add different levels of Free Form Deformations (FFD) to smoothly alter each facial feature.
Flexibility in terms of capture is an important element and it is present here as, no matter where you place your iPhone, the unique Zero Pose design will quickly recalibrate the angle offset of your face, and accurately standardize the facial tracking in one-click. CrazyTalk Animator 3, the tool you need for all this to work, will automatically choose the appropriate face profile, for front-facing or angled-facing characters. Mocap facial profiles provide optimized facial mocap for both sprite-based and image-based characters.
This complete solution to add virtual hosts to live broadcasts works with major livestreaming tools including OBS, Xsplit, and FFSplit to superimpose talking characters on top of major broadcasting platforms like Twitch, YouTube, Mixer and Facebook. It is also possible to apply green screen backgrounds to realtime character performances for easy color keying.
LIVE FACE app is a free download, now available on the App Store. CrazyTalk Animator 3 is available from Reallusion.com, or on the Mac App Store, and all users can now download the extended trial version of CTA 3.3 pipeline version ($299 value) for free until July 31st, 2018. You’ll need to have access to an iPhone X to make the magic happen!