I have always been fascinated by Virtual Reality. I remember the earliest incarnations of VR from my childhood – those giant ‘cinema 180’ screens, which convinced my brain that I was on a roller-coaster, despite just standing in front of a huge screen in a tent at a fair. Science fiction promised me so much – Star Trek: The Next Generation’s holodecks, Ready Player One’s OASIS, even 1992’s The Lawnmower Man – but Virtual Reality never materialised.
The truth was that the technology simply was not there to bring it to a mass audience – VR required too much processing power and there were just too many technological problems to overcome. The flame of VR, however, was kept flickering by NASA and the military who foresaw useful applications. Recently, with the advent of faster working chips and the wider availability of much greater computing power, companies have been pouring money into the format – not least the Facebook-owned Oculus Rift, Microsoft and Samsung. Rumours abound that Apple are also planning to release something as early as next year, which could be a real game changer.
The considerable advances in technology and the interest shown by story-tellers has meant that, in the last two years, VR has begun to emerge as a whole new artistic medium. At the forefront in training in the new format is the brilliant Raindance. I was lucky enough to attend their ‘Hands-On Virtual Reality Weekend’ last year and thoroughly enjoyed myself shooting a couple of short films in 360. I discovered the joys and the limitations of the format – learning about what works in 360 film and what doesn’t work.
It was Raindance who, in association with Facebook 360 and Blend Media, hosted the recent ‘Raindance VRX summit’, so that we could hear about the latest developments by some impressive creators. The day went something like this:
Virtual Reality, A Live Performance by Sid Phoenix
Things kicked off with an impassioned monologue by the actor, Sid Phoenix, who is currently investigating the interaction which is possible between VR and live performance. Such was the intensity of the piece, if Sid had dissolved into millions of pixels at the conclusion of the piece, I wouldn’t have been unduly surprised.
Welcome Note
The inspiring Raindance Founder, the ever dark-spectacled Elliot Grove, welcomed us. Elliot is a Canadian film producer who is passionate about independent filmmaking. I’ve enjoyed a number of his lectures on low budget film-making over the years and I really admire his current commitment to evangelising VR.
Bringing Immersive Media to 2 Billion People
The keynote speech was by Chetan Gupta, Product Manager for 360 Media, Facebook. Facebook has a global audience of 2 billion people and they are leading the way in new VR technology, as well as educating their audience. Panoramic 360 photos, which you can take on your smartphone and upload to Facebook are most people’s introduction to VR. Gupta explained that VR presented new problems for content providers to over come – just explaining how to view the format had proved challenging. For example, to encourage its users to pan around in a 360 photograph required Facebook to create new logos to instruct newbies that here was something a bit different. At first, the new users were amazed, posting questions asking what witchcraft this was that allowed them to change the viewing angle by moving their phone but gradually no holiday was complete without a 360 panoramic shot of the beach or the mountains. Soon, 360 filming on your smart phone will become as common as taking a selfie and Facebook will have led that charge.
Facebook have developed some brilliant new tech to assist VR film makers:
‘Dynamic streaming’ – Facebook quickly discovered that maintaining a high resolution 360 film took an enormous amount of computing power, which would render the films incapable of being played back on a smartphone. They realised, however, that much of the 360 film would not be seen, the viewer only being able to look in one direction at any one time and not having eyes in the back of his/her head. What, they asked, if everything the viewer looked at was in high definition but, when they weren’t looking at something, the resolution was diminished? This could save an enormous amount of memory. Thus, dynamic streaming was invented to reduce the bandwidth by decreasing the resolution of content, which is not being directly observed.
‘Author guidance’ – one of the difficulties I found working with 360 film was how to encourage my audience to look in a particular direction, to follow the story that I wanted to tell. With standard film (‘flatty’!) film-making, you just point your camera at what you want your audience to look at but, with a 360 film, your viewers could be looking off in the opposite direction. Facebook developed ‘author guidance’ to allow a director to easily highlight points of interest. This is a little compass in the corner of the screen showing which way the viewer is looking and, at certain points, which way they should be looking. More subtle guidance can be achieved by the use of things like 360 ambient sound – whereby a sound effect (e.g. a gun shot, a scream, etc) coming from a specific area serves to draw the viewer’s attention. Facebook have developed free software to assist low budget film-makers with this art too.
‘Heat maps’ – as a director, how can you discover where the average audience member focuses their attention in your 360 film? Facebook created ‘heat maps’ to track where viewers look and highlight it on to the film. This helps directors discover if their vision is being achieved, or if something is being missed, so that they can adjust their edit accordingly. This also assists directors in placing on-screen text where it is most likely to be seen.
Pushing the Boundaries: An Exploration of Narrative & Storytelling in 360°
This was a panel chaired by Damian Collier, Founder and CEO of Blend Media. Panellists included Alexandra Ruhl, David Betteridge, Chloe Thomas and Sam Weston. The panel expounded upon the virtues of having some background in ‘theatre in the round’, which most resembles 360 films. They discussed the question of whether the viewer is an unseen observer or a character within the film. The panel noted that actors had appreciated the freedom, which a director was forced to give them, since the director either could not be physically present at all, or was often hiding, silently, beneath the camera tripod. This presented some difficulties in adjusting the blocking, however, as action often needs to be in a golden area (not to close, not too far away) from the static camera.
Such difficulties were familiar from my own experiments in 360 film. It was surprising to discover that, most of the time, action works much better when the focus of the viewer is in one particular direction and you resign yourself to the fact that the 360 nature of the piece just encourages a sense of immersion. As I have, Ruhl, had experimented with a conversation between two people with the camera between them, necessitating that the viewer looks at one person, or the other, but she found, as I did, that it doesn’t really work! Perhaps this technique will work better in time, when audiences are more familiar with 360, but, at the moment, it just feels uncomfortable and we are left, as a viewer, feeling that we are missing something or looking at the ‘wrong’ thing (not to mention a potentially sore ‘tennis match neck’!).
Munduruku: Crafting VR Experiences for Social Impact
Pete Speller (Greenpeace), Grace Boyle (Multisensory Director) and James Manisty (VR Director) spoke about creating Greenpeace’s multisensory virtual reality experience Munduruku: The Fight to Defend the Heart of the Amazon, which went on to win Raindance’s VRX Awards ‘Best Social Impact Experience’ that evening.
Speller (Greenpeace) produced some impressive statistics regarding the audience’s feelings towards the indigenous people depicted and expressed how impressed he was by the impact, which the film obviously had upon people. VR really does seem to create empathy in a uniquely powerful way.
Boyle (Multisensory) discussed the creation of special viewing pods (see photo above), which utilised various senses to augment the film’s impact, by creating bespoke immersive spaces. Viewings of the film in these pods are enhanced by the use of various sensory materials, including site specific smells (a perfumer visited and then recreated forest scents for them), an infrasonic frequency track and the creation of a humid atmosphere.
Manistry (Director) spoke about the difficulties of lugging filming equipment into isolated parts of the Amazon and having to beg use of the tribe’s only generator to edit footage late into the night. He discussed the use of the camera as a character – the tribe being told to treat the camera as they would a guest – which created considerable empathy in the viewer, something with which Greenpeace were delighted.
[I’ve written a bit more about the film itself in another blog entry, the link is here – Raindance Film Festival – Virtual Reality Arcade]
Dear Angelica: Drawing Story in VR
Wesley Allsbrook, Script Writer, Art Director and Quill Artist spoke about her creation of the hit Oculus VR experience, Dear Angelica. Nominated for the Raindance VRX Awards Best Animation Experience, Dear Angelica is the world’s first hand-illustrated 360 short, created entirely inside VR with the programme Oculus Quill.
This extraordinary film shows how artists can create entire new worlds in VR. The joyful feeling which you get as a viewer – that you could just reach out and touch the moving paint – is a difficult one to describe without witnessing it first hand. Suffice to say, I highly recommend it.
Treehugger: Can VR Reconnect Us to Nature?
Barnaby Steel, the founder of Marshmallow Laser Feast, a London-based design studio, spoke about his VR experiences and his VR film Treehugger. MLF have been experimenting with 360 immersive experiences since their foundation in 2011 and they specialise in the use of drones, spectacular light shows and lasers.
Their commitment to bridging the gap between science and art, creating genuine moments of wonder for their audience is commendable. Their extraordinary film, Treehugger, lets participants explore the forest as various different creatures – a midge, a dragonfly, an owl and a frog.
Ray: Unlocking Empathy Through Reactive Content
Rafael Pavon, Director of Ray, spoke about his new interactive VR narrative that allows the experiencer to use their voice or hands to interact with a ray of light. Ray received its world premiere at the Raindance VR Arcade. Rafael also directed Snatch VR Heist Experience, starring Harry Potter’s Rupert Grint, which won Raindance’s VRX Awards ‘Best Branded Experience’ that evening.
Pavon spoke of the unscripted moment in Ray, where the actor looked into the camera, suddenly acknowledging the audience as being physically present, perhaps even as a character in the room. Difficulties such as having a mirror in the room (how do you hide the reflection of the camera?) were overcome by deciding that the central character was dreaming – so, if, when you watch the film, you look into the mirror, you can see the character asleep in her bed. Pavon opined the benefits of using a set and related just how beautiful you can make a CGI ceiling.
[I’ve written a bit more about the film itself in another blog entry, the link is here – Raindance Film Festival – Virtual Reality Arcade]
If you have a VR headset, try it for yourself (below):
Alteration: Shaking Up Cinematic VR Narratives With AI
Antoine Cayrol (OKIO Studio), executive producer, spoke about his creation, Alteration, which went on to win Raindance VRX Awards ‘Best Cinematic Experience’ that evening.
Alteration deals with its central character’s mind being taken over by an Artificial Intelligence and, at an early stage of development, Cayrol discussed with Facebook the possibility of using Facebook’s AI algorithms to allow the viewers to experience precisely what it’s like to be hacked by an AI. Uncertain of how this might look, Cayrol prepared an alternative plan but, ultimately, was so impressed by the AI version that he retained it. I agree with him – the results are truly stunning and this was my favourite film at Raindance’s VR Arcade.
[I’ve written a bit more about the film itself & you can see the trailer in another blog entry, the link is here – Raindance Film Festival – Virtual Reality Arcade]
Sensual Experiences in VR: Opportunities, Challenges and Ethics
It wasn’t going to be long before pornographic films grasped the possibilities of VR – they already thrive on the idea of the ‘viewer as voyeur’, watching a couple’s most private acts – so doesn’t VR, with its ability to put its viewers even more into the scene, sound like it would be the perfect new format?
What the directors of the 360 films on this panel had done, however, was also to recognise the increase in empathy, which VR engenders. Thus, the directors of erotic films, whom are keener to explore intimacy, rather than simply show sex acts, can be well-served by the format.This section of the day was a panel discussion, with Catherine Allen in conversation with:
Jane Gauntlett, Director of In My Shoes: Intimacy, a first-person sensual documentary designed for two people to experience three encounters from six very different perspectives. Gauntlett expressed her ultimate desire to encourage two simultaneously-watching viewers, who were strangers to each other, to touch one another, as a result of watching her films.
Jennifer Lyon Bell, Director of Blue Artichoke, who produce artistic, unusual erotic films that portray sexuality in an emotionally realistic way. Her 360 film, Second Date, set on an Amsterdam houseboat, is a lighthearted, unscripted Virtual Reality portrait of two young people fumbling towards ecstasy. Trying to find common ground, their conversation veers from clumsy to joyous and back again – until they finally start connecting for real. Bell seemed fascinated by the future possibilities of VR, although she hoped that PornHub’s interest in the format would not have the same effect that it had on the erotic film industry – reducing, in her opinion, the ‘art’ of the early high quality pornographic cinefilm-making to amateurish, VHS tread-milled fare.
Anne De Winter, Actress in Second Date, discussed the freedom she felt on set and the naturalism possible on a 360 film.
First Impressions: The Evolving Landscape of Non-Fictional VR Storytelling
Nicole Jackson and Anetta Jones, the Guardian’s VR team, discussed the Guardian’s forays into 360 film-making.
Their film 6×9 impressed many with its virtual experience of solitary confinement. Again, one is impressed by the empathy 360 films can engender in the audience – if you feel cramped and uncomfortable in the few minutes you spend watching it, imagine what it feels like to be confined in a 6×9 cell for 22 hours a day, with little or no human contact for days, even decades? How, then, does that make you feel afterwards about the 80,000 people currently confined like that in the US?
If you have a VR headset, try it for yourself (below):
They also spoke about their latest project, First Impressions, which used the latest research in neural development and colour vision in infants to create a film, which allows you to experience and interact with the world from the point of view of a baby.
If you have a VR headset, try it for yourself (below):
NEST: Blending Cinematic Storytelling with Augmented Reality
Indie game developer and filmmaker, Duncan Walker, screened the world’s first Augmented Reality experimental short film, created with ARKit.
Augmented Reality really hit the public imagination with the release of the Pokemon Go game – suddenly people were glued to their smartphones seeing Pokemon characters which had been brought to virtual life in their local environment. Grown adults were overjoyed at seeing Pikachu scampering along the street outside their home. Walker showed us how effectively digital characters could be super-imposed on to the real world.
Thus, we saw Walker’s footage of robotic police officers patrolling the streets of London and alien creatures horribly morphing in alley ways – not the products of hundreds of hours of CGI animation but an evening with ARKit (you can see the impressive footage for yourself, below).
If the film Monsters showed us that Hollywood-style special effects could be achieved by low budget films (the Director of Monsters, Gareth Edwards, created all the visual effects himself using his home computer and store-bought visual effects programmes), Walker showed us that Hollywood-style special effects were within the grasp of zero budget films too, with the creative use of AR. Something to keep an eye on, certainly…