VR technology is here…sorta. Storytelling, though, has taken a back seat to technology. That’s a big problem.
Recently I attended the Silicon Valley Virtual Reality (SVVR) Expo at the San Jose Convention Center. While most things related to entertainment exist in places like LA and NY, my stomping grounds—the San Francisco Bay Area—are where it’s at for new computer technology.
Apparently that includes VR.
I wasn’t sure what I’d find at the expo. I’d been turned on to it by someone I know locally who is way more dialed in to the VR scene than I am. He encouraged me to check out the show floor, and for $75 for a one-day pass I couldn’t refuse. (“Don’t pay $750 for the week,” he told me. “It’s not all that.”)
My one-day five-hour pass didn’t get me into any of the seminars, many of which looked interesting. (The adult video seminar caught my eye, but not for the obvious reason—if nothing else, I’d get a chance to see where VR is headed as porn tends to lead the pack when it comes to media distribution. It’s safe to say that, if it wasn’t for streaming internet porn, we might still be waiting for Netflix to move away from its DVD distribution business.) I learned enough from wandering through that one hall, though, to give me a good idea of VR’s state of the art.
Most of what I saw was game-related. I didn’t get a chance to see any demos through VR goggles as many of those booths had long lines that formed as soon as the hall doors opened, but nearby 2D screens gave me an idea of what was happening. Most looked like your classic first-person shooter games, which is cool as I own more than a couple and enjoy playing them when I have time. What I couldn’t get a sense of, though, is how the game used the full 360 degrees of VR space. One thing I’ve noticed is that most VR presentations are missed opportunities when it comes to using the environment to its fullest: most of the action takes place in a “TV safe” window directly in front of the viewer, and the rest of the environment is simply window dressing.
One company offered a ten minute “cinematic experience” that claimed to show off the storytelling aspects of VR to their fullest. This intrigued me as, so far, I’d not seen much in the way of storytelling beyond what one finds in video games.
Without mentioning the name of the game, I will say that the “cinematic” intro was fun. I found myself in a spaceship with a view of Earth dead ahead. As I ventured closer another spaceship appeared and emitted a beam of magenta light that raked the Earth’s surface until it found something interesting and designated a landing spot. After a trip through a somewhat realistic meteorite shower I found myself plunging through the atmosphere and into the ocean, where my spaceship sank into the depths. Upon landing on the bottom, a cute little salamander-like creature climbed up the windshield, looked at me, smiled, and swam away. That was my cue to follow.
I’d been handed a game controller, and instructions appeared in front of me that showed how to move forward and stop. I steered by tilting my head left and right, and I was able to gain some altitude by drifting across hot underwater vents. I explored a little, but as it was clear that I was meant to follow this little creature I did exactly that, looking forward to the story that was about to envelope me.
Then the image faded out. Someone took off my headset and asked me if I enjoyed the experience. “Where’s the story?” I asked.
The gentleman looked perturbed. “Oh, we’ve got that all worked out,” he told me. “This is just a proof of concept.” And he moved on to the next person, who had also seen their sign and stopped for a cinematic story-telling demo.
And that was about it. That’s where storytelling in VR is right now. It doesn’t seem to exist. Right now, VR is mostly about experiences—and that’s completely valid, but it seems there’s no place for cinematographers at the moment.
One company showed off a camera meant to capture sports from the perspective of someone standing immediately at the edge of the action. The demo consistent of 360-degree footage of a basketball game, and yes, I felt like I was right there at the edge of the court watching a game. I asked the gentleman manning the booth what kind of crew they sent out with the camera. “A software engineer and two techs,” he told me.
“So… no one with any photographic experience?” I asked.
“Nope,” he said. And he smiled. This actually made him happy.
“That’s not unusual,” my friend in the know told me later. “VR currently exists in the realm of technicians and software developers. There aren’t many filmmakers involved yet, and that’s what is desperately needed.” I agree. Of the many uses I saw for VR at this expo—physical therapy, interior design pre-visualization, game play, live event participation—almost no one was telling stories. The stories that were being told were not terribly sophisticated.
The Nokia Ozo looks like one of the more technically mature VR cameras around at the moment, but the demo material doesn’t reflect this. Here’s one of the demo films that Nokia showed (view the videos in Chrome see them in 360):
And behind-the-scenes:
https://www.youtube.com/watch?v=86M-FJypoPs
It’s an interesting proof-of-concept piece, but it doesn’t compare experientially to another company’s demo where a VR helmet was strapped to the head of a professional skier. That was a wild ride, especially when I hit the 40′ ski jump (someone at the booth held on to me for that part), and for a 90 second experience it was a lot of fun. That company’s camera system was a bit more primitive than Nokia’s in that I could see, under certain circumstances, the stitch lines between cameras, but at least the experience was fun. Nokia’s presentation was technically excellent but the bear attack felt extraordinarily fake. It didn’t help that the shadow of the camera featured so prominently on the ground. (It may be that VR will force cinematographers to do something we regularly resist: shoot at noon.)
The best and worst storytelling examples I’ve seen so far come from Jaunt (not at the expo), who seems to have drifted from camera manufacturing into becoming a VR distributor. This video is one of the first VR experiences I’ve had:
https://www.youtube.com/watch?v=mNVPckKOrdo
It’s fun, but… nothing really happens. Okay, an alien DJ plays some tunes, but once the FBI arrives and draws their guns (a traditionally American response to anything different or unusual) they end up being the ones to freeze. Down the other alley a similar car full of agents does the same thing, only at a distance. The area behind the VR camera, opposite the alien, isn’t used at all. You’d think someone would spray paint some cryptic symbols up there, along with a crude alien drawing and some indication of what its schedule was, as the viewer could explore the environment and assemble a little bit of a storyline: “Oh, the alien has been here before!” Instead… nothing. 90% of the VR frame is wasted.
This VR film has me baffled:
https://www.youtube.com/watch?v=J28ojfmF0rI
If I’m supposed to be immersed the scene, why am I in the middle of the zombie apocalypse? Shouldn’t I be near the car, watching these people defend us/me? Why am I standing in the open, being ignored by zombies?
Here’s another demo:
https://www.youtube.com/watch?v=gR7fOrCo3xo
Once again, most of the action takes place within a “TV safe” window directly ahead of me. I can watch all of this on Youtube and, with barely any panning or tilting, see all the action I need to see. There’s literally nothing else outside of that frame that contributes to the story.
This is not unusual. While enjoying the alien journey demo I mentioned at the beginning of this article, I spent a short amount of time looking around the cockpit of my spaceship to see if there was anything interesting going on outside of my “TV safe” window. There were controls on either side of me but their displays were on loops, so clearly just filler material.
The best example of VR storytelling is a very early film from Jaunt that doesn’t seem available online other than as part of VR film reviews. Here’s one of them:
https://www.youtube.com/watch?v=vZ8t6IgPd88
The acting is good, there’s actually a bit of dramatic lighting going on that helps both with mood and transitions, and the story, while vague, has been blocked out for maximum effect. The interesting thing about this film is that the sets were clearly just slapped together—no art directors or art departments involved here—but it totally worked. The garage looked about as messy as a garage should look, and in VR that was exactly right. It was an interesting revelation that, while there are some things that VR doesn’t allow us to get away with, there are other things that it does.
What’s common about all these storytelling and experiential pieces is that they are very short. I’ve been told that 15 minutes is the average length of time the average person can remain in a VR environment without starting to feel some sort of motion sickness. IMAX films are never more than 60 minutes as the overwhelming size and resolution of the screen has a tendency to stress and drain audiences over time. I’ll be curious to see if this is an effect that diminishes over time as the population at large adapts to VR, or if VR stories will—of necessity—be told in short but very realistic episodes.
It’s clear that VR requires a rethinking of how we tell stories. In the game arena Myst-like games could do very well, where the viewer explores an environment and discovers the story rather than experiencing it directly. When it comes to live action, the non-VR example I keep coming back to is the play “Sleep No More,” where parallel storylines occur in different areas of a building and the story you experience changes depending on where you are and how many times you return to experience the play from different vantage points. This could be an idea application for VR.
I was told by several people, who know the VR playing field better than I, that live action VR won’t come into its own until until light field cameras become common. They tell me that light field capture will allow viewers to walk around a set and look behind things, as if they were actually in that space. Right now live action consists primarily of “brain in a jar” experiences, as one reviewer put it, and while I think there’s room in that jar for storytelling it will be a while before real filmmakers develop the new language that will allow dramatic and sophisticated VR stories to be told. Rather than relying on framing they’ll have to experiment with blocking, creative lighting (which is almost unknown in live action VR at the moment, but based on what I saw above in Black Mass it can add an amazing amount of depth to the experience), visual depth and sound cues, and even dialog to direct the viewer’s attention toward the key elements of the story. And even then, they’ll have to deal with people like me who deliberately look everywhere else but where we’re supposed to in order to take in the full VR landscape.
I think there’s a future here, but it’s a future in search of a reason to exist. Based on what I saw at SVVR, there are lots of people searching for it—and they have lots of money. The show floor was swarming with venture capitalists looking for the next hot thing, and most of the booths consisted of companies who were almost literally throwing their ideas against a virtual wall and seeing which ones would stick.
None stuck for me, but some slid down the wall slower than others. Here’s hoping that, sooner than later, we’ll get some real storytellers and cinematographers behind the camera (or under it, as that’s where the blind spot is in VR) who can do something compelling with this medium, instead of simply capturing experiences or framing shots with 270 degrees of padding around them.