Not all videos are destined to be broadcast on TV, or even viewed on a computer screen.  While building projections have continued to get bigger, brighter and bolder there are many other options for capturing the interest of the general public.  Here are three case studies of video productions  where the clients have gone beyond simply producing big videos, and have used other techniques to create an immersive experience for the viewer.

Video production is a broad term that covers many different markets and industries, but despite the huge variety in people and companies making videos, it’s a pretty safe bet that none of them go out of their way to make something that’s boring and forgettable.

In many cases, agencies and their clients are always looking for new and innovative ways to grab people’s attention, and (as one example) over the past few years building projections have become an increasingly popular way to make an impact. Since they first started appearing around 2009, they’ve become bigger and bigger – and the Vivid festival in Sydney has become the largest lighting and projection show in the world, running for three weeks every year.

When the Vivid festival is in full swing it’s pretty hard to miss, but having already posted several articles on building projections, I thought it would be worth taking a look at a few other projects I’ve worked on that have used different approaches to capture the interest of the viewer. And while building projections have only been with us for a few years, one of the largest projects I’ve ever worked on relied on an old theatre trick dating back to the 16th century.

Xian: The song of everlasting regret
A 1,300 year old story and a 400 year old theatre trick.

It’s not uncommon to find videos as part of museum exhibits, and there are companies around that specialize in producing videos and interactive kiosks for museums. However it’s less common to find a museum that’s been entirely designed and built to show one video – and that’s what makes the Xian project unique.

“The song of everlasting regret” is one of the most significant stories in Chinese culture, and as a mark of its traditional importance a new museum was constructed in China’s Xian province as a permanent exhibit for the tale. While the “song of everlasting regret” is as well known in China as the story of Cinderella is in the west, the full title was a bit of a mouthful so we generally just referred to it as the Xian project.

The museum under construction in the Xian province.

The story itself was adapted into a short film, and the production was handled by Sydney company Spinifex. Although the original story (actually a poem) is over 1,300 years old, almost the entire film was shot against greenscreen- with all sets, backgrounds and many props being created using CG. In all, over 300 shots were composited for the 26 minute film.

I often use the term “Hollywood level” as an indication of quality even though I admit it’s pretty meaningless, it just sounds good, but in this case it’s no exaggeration to say the level of technical quality was beyond what is typical for current feature films. To help create a fully immersive experience, the film was made at a very wide aspect ratio, which in turn meant very high resolutions. Each scene was a slightly different size and aspect, but the two scenes I worked on had a pixel size of 6156 x 1080 – so the level of detail was exceptional.

Multiple video projectors being aligned to project the ultra-widescreen image.

Compositing examples from the scenes I worked on – all completed within After Effects.  Overall resolution was 6156 x 1080.

The story was broken down into seven scenes, and the museum was built with 4 different levels – normally I’d say the museum had 4 stories, but in this case the term ‘story’ is ambiguous, so I’ll use levels instead to avoid confusion. The public would begin on the ground level with the opening two scenes, and move up to the next level as the story progressed.

As you might expect from a museum specifically built to tell a story, the intention was always to create an immersive, encompassing experience for the viewer. A huge amount of effort went in to recreating the period details in the architecture, props, costumes and overall setting.

Stepping into the museum was stepping back in time by 1,300 years.

But not only was the video being displayed in a custom space which immersed the viewer into the same world as the film, it adapted a old stage technique from the 16th century to make the film even more special.

Commonly referred to as “Pepper’s ghost” after a famous demonstration by John Pepper in 1862, the actual technique has been used for several hundred years to create illusions for theatres and sideshows. Recently, however, the same basic technique has been used to great effect and marketed as a “hologram”, even though it isn’t really.  A real hologram drives around in a glowing Lamborghini Countach.

Originally, the Pepper’s ghost effect used an angled sheet of glass to reflect an object or actor off-stage. By controlling the lighting, the reflection would appear or disappear – giving the audience the effect that the object or actor itself was appearing or disappearing – just like a ghost.

More recently, the same basic approach has been used to project “holograms” – which sounds impressive but is really just video being carefully projected onto a transparent material called ‘scrim’. Just like the theatre illusion that’s hundreds of years old, by carefully controlling the lighting the video being projected can appear and disappear.

To give you an idea of scrim in action, I’m actually going to jump to a clip from a completely different production called ‘Clusters of Light’, which also used scrim to great effect. You’ll see here that simply by changing the lighting the material can either appear opaque or transparent. In this case the scrim is used to conceal and reveal performers on stage, but the basic principal is the same.

Scrim in action – different show, same technique.  Changing the lighting makes the scrim visible or invisible.

In the Xian museum, the scrim was used in a number of different ways, but in a general sense it was used to give the film a sense of depth, by separating the scenes into several layers, and projecting them onto staggered screens.

It’s difficult to exactly explain what this means, but it’s a way of producing a 3D effect without having to wear glasses. Basically, instead of one video being projected onto one screen (as in a cinema, for comparison), the video was separated out into three layers, and there were three screens positioned in front of each other at different depths. Depending on the lighting, the back layers of video were still visible through the front layers – and the overall impression to the viewers is roughly comparable to watching a 3D film.

Here you can see the museum under construction, and see some of the screens being installed. This is the important shot, showing the scrim material being installed. I don’t know exactly what this is made of but I’ve heard it’s very expensive.

In addition to simply creating a sense of depth, the multiple layers were especially useful to add atmospheric effects. In this battle scene, for example, the fire elements you see are on a totally different plane to the main video, and the burning embers are separated out again.

The action scene is projected onto the rear screen.  The fire effect is projected onto a separate, closer screen and dust, atmos and burning embers are projected onto a 3rd screen again.

Overall, ‘The song of everlasting regret’ was a massive undertaking. The total duration of the film was over 26 minutes, and not only was the resolution higher than a standard Hollywood film, but final result was delivered in multiple layers.

The end result, however, was nothing short of spectacular and when viewed within the context of the purpose-built museum the multiple layers of video helped to create a powerful and immersive experience – thanks to the ultra-wide aspect ratios used and a theatre trick that’s hundreds of years old.
 

Cisco
Stereoscopic 3D for total immersion

While the Xian museum projected multiple layers of video onto staggered screens to create a faux-3D effect, Cisco went one step further and created a true stereoscopic 3D environment for a corporate presentation starring British legend Stephen Fry.

Cisco built an entire conference environment for the London Olympics.  This included lifting a London Tube carriage onto the top of the builidng.

As part of a corporate exhibition space built for the duration of the London Olympics, Cisco built a small theatre that looked just like a train carriage – or a tube carriage since it was in London.

The opening and closing scenes of the video were set on the platform of a tube station. Audience members would put on 3D glasses and then go and sit in the tube carriage. As they looked out the window they would see the station platform in true stereoscopic 3D.

As the video played, the train travelled to outer-space as well as other digitally created environments, all in stereoscopic 3D. While technically demanding – I composited the entire video in After Effects – the end result was incredibly immersive and very well received by the client and the viewers.

What was a fairly typical corporate presentation was made interesting, engaging and immersive by using stereoscopic 3D and Stephen Fry.  Who makes pretty much anything awesome.

Acura
Glasses-free 3D

The problem with true stereoscopic 3D has always been the need for the audience to wear glasses. For several years there have been a number of glasses-free techniques available, and luxury carmaker Acura used one for an interactive kiosk at various international car shows.

There are several different brands of glasses-free 3D monitors around, and their displays use miniature lenticular lenses to show a different image to each eye, and create the illusion of true 3D.  Most of these systems require multiple outputs from a 3D program, and some need up to 9 different render passes (each one from a unique camera angle) to be compiled together for the system to work. While the end result can be  impressive, it’s important not to underestimate the 3D rendering time needed to produce content for such screens.

The Dimenco system we used for Acura, however, worked with both conventional stereoscopic 3D renders, and also with a single 2D render and a z-depth pass.

This meant we could use a conventional stereoscopic 3D workflow to produce content that was eye catching and immersive, even in an environment full of eye catching and impressive displays.  If you’ve ever been to one of the major car shows around the world you will know how difficult it is to stand out and grab the public’s attention.

Unfortunately I don’t have any images or footage of the final kiosk in action, but since the impressive part was the glasses-free 3D I’m not sure a photo would do the final installation justice.

As a consolation you’ll just have to watch a flat 2D preview and imagine it’s jumping out of the screen…

Here’s a short section of video in regular 2D. Trust me that it looks really cool in 3D, especially without glasses on.

The Xian project, Cisco House & the Acura kiosk are all examples of immersive video experiences that do more than simply play a video.  By using a combination of setting and different techiques to add depth to the vision, the viewer comes away with an overall experience that’s much more than they’ll get from simply watching a clip on YouTube.

With a new generation of virtual reality equipment about to be launched to the general public, it’s worth knowing that you don’t need a full headset and a virtual reality rig for an immersive experience.