Beyond Reality | #ICPTalks
The #ICPtalks “Telling Stories Through Technology: 3D, VR, and Photography in Today’s Image Making” on November 18 turned around different realities: worlds beyond the ones we know through direct experience, photography, or even film. Virtual, augmented, or multi-dimensionally recorded, these new experiences are enabled by a combination of cutting-edge technology and an open mind geared towards its possibilities. The central question: How will these new technologies change the way we tell stories?
Part of a programming season focused on multimedia storytelling and new narrative tools, and presented by Studio 55, this talk presented the viewpoints of three innovators working at the juncture of stories and tech: Caitlin Burns of Datavized, Mattieu Lorrain of Google, and Tatjana Dzambazova with Autodesk. The conversation was moderated by Julia Sourikoff of The Future of Storytelling (FoST).
A co-founder and COO of the VR and data visualization company Datavized, Burns also defines herself as a “transmedia producer,” someone who can envision and translate a story across three or more platforms. Her resume includes work on large-scale movie productions like The Pirates of the Caribbean, Avatar, Teenage Mutant Ninja Turtles, and Disney’s The Descendants, but also social media-linked wearable technology projects for Unicef and virtual reality platforms in development.
“When we’re working with these big projects,” she explained, “we’re looking not just at the business and the product rollout but how the story world is going to be experienced on each one of those different platforms… whether that’s a game on a console, a feature film, television, or web video experience—or holo-deck live-action craziness.”
It’s a careful balance, Burns indicated. “You’re trying to strike a balance between the needs of art; the needs of the design of the experience that each platform requires; and the business, the resources and actual distribution needs that are going to be coming into play: image rights, distribution platforms, different legal constraints for different platforms—all play into the decisions you make about how you’re going to visualize the work.” For multi-platform storytelling, as well as more complex virtual reality set ups, she said, the format and the platform have to be considered with care. “When you’re trying to get a point across, when you’re building an infrastructure around the business of storytelling, each one of these [tools] had its own place, but you want to choose the right ones.”
When integrated with the right platform, virtual reality is not far removed from one of Burns’ very early passions: theater. They have in common the total immersion in a different world. “There’s nothing quite as cool as putting someone in a physical place,” she said. “The only thing I can think of is putting someone into a virtual space. And of course, I’m not the only one who thinks so.”
Lorrain’s interest in technology and the potentials of virtual reality also grew out of a real-life experience. In his case, though, it lay outside any institutional performative cadre: his developed via the “freeparty” movement that brought techno parties to the remote corner of the French Alps where he grew up. In the relatively analog 1990s, these underground techno music parties, outside of any regulation or social control, would create “a different reality,” said Lorrain. “It had a very big impact and influence on my work. Because of freeparties I started creating technology.”
Today Lorrain works with Google, leading interactive experiences through the development of new models for interactive storytelling, virtual and augmented reality, and spatial narratives. These include explorations both playful and educational, such as an iteration of Google Street View that include fantastical creatures (picture rhinos traipsing down Times Square, or Japanese cherry blossoms on the Champs-Élysées in Paris), or virtual reality journeys that allow a user to experience the solar system up close. Lorrain is enthusiastic about the many possibilities that VR technology offers. “You can travel to impossible places with virtual reality,” he said. “You can see microscopic things, you can travel inside the human body and be immersed. You can go to places that are forbidden or dangerous.”
But where Lorrain sees some of the greatest potential for virtual reality is empathy. He showed a video from MIT project’s “The Machine to be Another,” a performatic experience in which two people stand opposite each other, both wearing Oculus Rift headsets and headphones that virtually place them in the place of the other. “I tried it,” Lorrain explained. “I tried it with my spouse and it was just mind-blowing to be able to see, from the eyes of my wife, see her body, see her hands, her arms, and see the perspective that she has of the world. I think that’s something very powerful to think about, when we think about VR, that we can now transport you to the eyes and the body of someone else. That’s something you couldn’t really do before.”
The possibilities for real empathy have also drawn Dzambazova into explorations of virtual reality and multi-sensorial experience recording. Trained in architecture, she now works as the engineering and design company Autodesk. She is convinced that a 360-degree-view and surround-sound recording of her experience visiting Virunga National Park, in DR Congo, would have awakened her friends’ interest much more than her “stupid photos and videos.” “They look and they pretend to be interested and yet they have no way to understand it,” she said. If she had been able to really capture the environment around her and return to San Francisco, if “they put on the goggles and now they're walking with me… all together in Congo, and they can actually see it, and explore it—that is where real global empathy can happen.” For Dzambazova, this kind of technology represents “the future of journalism.” She feels strongly that it’s also “the future of understanding what the heck is going on in this world.”
Other projects she presented also dealt with transfer, though less that of humans being transported to a different reality, and rather objects moving in and out of the physical and digital realms. Dzambazova has been exploring the possibilities for art, conservation, and education offered by 3D scanning and printing. She presented a recorded demo of Memento, a new Autodesk product that facilitates the creation of a 3D model based on a series of images made with iPhone cameras. She offered case studies of people and teams whose work has benefited from this kind of technology, like artists, for instance, who are harnessing the technology’s potential for their work, notably in sculpture. Another impressive collaboration has been with the Smithsonian Museum, which is digitizing all objects in their collection to create high-definition, interactive 3D models that can be looked at online, measured, and explored from all angles.
“This was for us a prototype of how museums in the future will look like, and how can we combine digital media and allow for storytelling online without programming. And you can combine music and video and audio and 2D and 3D and live tweets and Google Earth, etc. to tell a story,” she said with excitement. This technology also allows museums to easily create replicas of valuable objects. Dzambazova declared: “This is the end of no touch in museums, you can touch now!”
In the conversation that ensued among the three panelists, the possibilities for virtual and augmented reality were further explored. Lorrain described the current moment in the development of these technologies as the “train arriving in the station,” referring to the first snippet of documentary film made in 1895, when the film industry had developed its first, most basic tool but had not arrived at telling stories yet. “Virtual reality is in the same space: we can show you very cool, immersive images. We can transport you to space. But the question is: What stories are we going to be able to tell?’” said Lorrain.
In a discussion about user experience, or UX, Dzambazova underscored the need for tech tools to be user friendly. “It’s unpardonable that we are surrounded with technologies that frustrate us,” she said. “It’s important to understand that almost all users would rather have a less potent tool that they can learn and they can use rather than a computer that asks you to think like a computer.” When thinking about the user, Burns called attention to the fact that their conversation generally assumed a single-user environment. For her, the question is “how you articulate each one of these creative components when you’re dealing with multiple users, multiple players, each with their own perspective.”
New technologies and immersive experiences also command a different type of attention on the part of the user. As Lorrain pointed out, “with virtual reality, you can’t really multitask. It’s both a challenge and an opportunity: A challenge because people will have higher expectations, but in the meantime it’s a great opportunity… with VR people will have to fully engage with the story to fully experience it,” he said. To which Burns responded with a safety caution: “We don’t have a full sense of where the human body is going to operate with these new hardware technologies,” she said. “If you’re going to be walking around the virtual world, you’ll need to have some sort of secure paddock in your living room.”
Burns’ point was acutely illustrated by a recent VR experience that Dzambazova described: she was standing in a room with a headset on, but in the VR world, she was walking to the edge of a cliff. “I promise you, it didn’t matter what the brain was telling me, I lost forces in my knees. It really messes with your brain, you really think you’re on a cliff and that you’re going to fall, and that’s very strange. You cannot control your emotions, you cannot control your mind.” The discussion began to highlight some of the potential perils of virtual and augmented reality, and the questions from the audience picked up from here: Can it be used effectively as a tool in the service of journalism? How can the technology developed today protect itself from obsolescence? What about the potentials for abuse and manipulation of such immersive technology? Here, Sourikoff, the moderator, stepped in: “I think this ethical conversation is raised every time a new technology is introduced and made available to the masses. We saw this with 3D printing and the fear that people were going to start printing firearms. I think it’s a matter of faith in humanity.”
I’m sure this point is up for debate, or could perhaps inform a future conversation grappling with the ethical and human implications of virtual and augmented reality, or the multi-dimensional recording of reality without as much of an interlocutor who could be held responsible. In many ways, the conversation between Burns, Lorrain, and Dzambazova served as a metaphor for the field in general. As Lorrain pointed out, we are just at the start, at the point of “the train entering the station.” The possibilities and intricacies of these types of technologies are just starting to be worked out.
Paula Kupfer is a writer and editor based in New York. She works as photo editor for The California Sunday Magazine, and was previously the managing editor of Aperture magazine.
