Jim Malcolm, General Manager, North America, Humaneyes Technologies
Srinivas Krishna, Founder and CEO, Geogram and AWE Company Ltd.
Ethan Shaftel, Director/Writer of Film, EasyAction
Matthew Celia, Creative Director, Light Sail VR
James DeRuvo, Editor-in-Chief, DoddleNEWS
Larry Jordan: Tonight on the Buzz, we are looking at virtual reality and augmented reality. We start with Jim Malcolm, general manager for North America for Humaneyes. They make the Vuze VR camera. Jim shares his thoughts on where VR is heading, and how we can start experimenting with it today.
Larry Jordan: Next we look at a specific example of an AR application, Geogram, developed by the AWE Company. CEO Srinivas Krishna explains the difference between VR and AR and showcases the areas where AR is most likely to be successful initially.
Larry Jordan: Ethan Shaftel is a writer and director of immersive experiences and VR storytelling. He shares his thoughts on how VR differs from traditional filmmaking and describes his process and software that he uses to create two recent VR films.
Larry Jordan: Matthew Celia is the creative director of Light Sail VR. Tonight he talks about the challenges in creating his latest VR project, Speak of the Devil, along with the gear that he used.
Larry Jordan: All this, plus James DeRuvo with our weekly doddleNEWS update. The Buzz starts now.
Announcer: Since the dawn of digital filmmaking, Authoritative: One show serves worldwide network of media professionals. Current: Uniting industry experts. Production: Filmmakers. Post-production: And content creators around the planet. Distribution: From the media capital of the world, in Los Angeles, California, the Digital Production Buzz goes live now.
Larry Jordan: Welcome to the Digital Production Buzz; the world’s longest running podcast for the creative content industry, covering media production, post-production and marketing around the world.
Larry Jordan: Hi, my name is Larry Jordan. Tonight we’re revisiting VR and adding a look at AR which is short for augmented reality. We’ve avoided AR in the past because it tends to require less filmmaking and more development and engineering. However, tonight we learn about an application that is releasing this week that combines AR with filmmaking. You’ll be interested in our interview with Srinivas Krishna as he tells us more about it. I’ll have more on this in my just thinking segment at the end of the show, but as you listen to tonight’s interviews, ask yourself, “How can exploring VR enable me to add more services to offer to my current or potential clients?” Jim Malcolm’s thoughts on an inexpensive way to start experimenting with VR can be very useful in your thinking. This will be an interesting show.
Larry Jordan: Before we start though, I want to invite you to subscribe to our free weekly show newsletter at digitalproductionbuzz.com. Every issue, every week, provides quick links to the different segments on the show, plus articles of interest to filmmakers. Best of all, every issue is free, and comes out every Saturday.
Larry Jordan: Now it’s time for our doddleNEWS update with James DeRuvo. Hello James.
James DeRuvo: Hello Larry. This is your one month call to NAB.
Larry Jordan: I’m already starting to get press releases of people saying, “Here’s what we’re going to announce at NAB,” so I think other people are aware of that fact too.
James DeRuvo: As the news guy for doddle, I’m more than happy about that because it’s been a dry month.
Larry Jordan: February is always hard because nobody wants to talk about anything. They’re gearing up for NAB.
James DeRuvo: I’m going to be busy for the next month or so so that’s going to be fun.
Larry Jordan: It will be fun, I’m looking forward to all the news and all the new products. That gets us to the news this week. What’s happening?
James DeRuvo: There’s an indication that Canon may be shifting focus to mirrorless cameras for their consumer DSLR camera production. Based on an interview with Canon president of marketing, Masahiro Sakata, Canon may be shifting focus of their consumer division away from DSLRs and to mirrorless designs and the M50 which we talked about last week, may be their first step. Sales and rental trends show that users tend to prefer the smaller form factor of a mirrorless camera, so it looks like Canon is tired of playing second fiddle to Sony and Panasonic.
Larry Jordan: Well hasn’t Canon built their reputation on DSLRs which are, inherently, mirrored cameras?
James DeRuvo: That’s why I kind of think that they’re not going to forsake their DSLR centric clientele just to focus solely on mirrorless designs moving forward. But it could mean that because the fact that mirrorless cameras are so popular, they’re reacting finally to that trend and they’re going to start focusing more of their design improvements and testing them out in mirrorless consumer cameras first before they get seated down into the DSLRs.
Larry Jordan: That’s news from Canon, what else is happening?
James DeRuvo: You may remember last week we talked about ProGrade, the new media card company that was started to fill the gap left by Lexar closing not too long ago? They have announced that they’re not going to be supporting XQD media cards for both Nikon and Sony. They want to push those manufacturers to go for the newer Compact Flash Express format, which they say will be backwards compatible with XQD but will have faster file transfer speeds.
Larry Jordan: James, back up a step. Tell me what ProGrade is and then tell me what the significance of this news is.
James DeRuvo: ProGrade was a company that was started by former executives of both Lexar and SanDisk who wanted to start a company to fill the gap left when Lexar was closed. If you look at their line of SD cards and CFast cards, they look exactly like a Lexar, right on down to the color scheme. So they want to create high performing media cards and it sounds to me like these guys are now free to push for their preferred formats. Clearly they support the upcoming Compact Flash Express format over XQD but the only hiccup is that they have to convince Nikon and Sony to enable support for Compact Flash Express with a future firmware update and until then, they’re going to be focusing on producing CFast 2 and SD cards in the near future.
Larry Jordan: That’s ProGrade. What’s our third story this week?
James DeRuvo: We just had the Oscars as everybody knows and it was very exciting to see Roger Deakins finally get his due. But if you look at how many cameras were used in all the Oscar nominated and winning films, it looks like Arri once again was the big winner. Arri scored the most nominations and wins amongst features at this year’s Academy Awards. Meanwhile, film is firmly back as Steven Spielberg’s The Post and Paul Thomas Anderson’s Phantom Thread were both nominated for best picture, and they were shot using Panavision’s Panaflex XL2.
Larry Jordan: Arri’s been around for a long time. What makes this announcement interesting to you?
James DeRuvo: Well what I find most interesting wasn’t that Arri cleaned up at the awards, they usually do every year but that Red was practically nonexistent. Even in nominations. Red continues to push the state of the art though and that makes everything move. I just thought it was interesting that they weren’t around.
Larry Jordan: What other stories are you following this week?
James DeRuvo: Other stories we’re following include there are some great DIY solutions this week, including a $20 shock mount that looks and works great. Pomfort goes into beta with their latest live grade color correction software, and Amazon is going after YouTube in the streaming wars.
Larry Jordan: Where can people go on the web to learn about these and other stories?
James DeRuvo: All these stories and more can be found at doddlenews.com.
Larry Jordan: James DeRuvo is the Editor in Chief of doddlenews.com and joins us every week. James, as always, thank you and we’ll talk to you next Thursday.
James DeRuvo: See you next Thursday.
Female voice: Starting Monday April 9, join the Digital Production Buzz at the 2018 NAB show in Las Vegas, Nevada. Larry Jordan and the Buzz team are taking their microphones on the road to cover the latest news and announcements from the largest media show in the world. Every hour of every day, the Buzz is live on the trade show floor. More than 100 interviews creating 27 new shows in four days. The Buzz has webcast directly from NAB for ten years and our coverage is legendary, heard in more than 195 countries around the world. If you’re attending the show, visit us at booth SL10 527 and say “Hello.” If you can’t attend, visit NABshowbuzz.com for a schedule of shows and guests. That’s NABshowbuzz.com and join the Buzz at NAB.
Larry Jordan: Jim Malcolm is the general manager North America for Humaneyes Technologies. He’s responsible for managing the Vuze VR camera product eco system as well as all the other Humaneyes products. He also sits on the board of directors for the Consumer Technology Association and is the immediate past president of the Imaging Alliance, and before joining Humaneyes, he was president and chief marketing officer of Ricoh imaging America’s Corporation. Hello Jim, welcome.
Jim Malcolm: Hi, how are you today?
Larry Jordan: How would you describe Humaneyes?
Jim Malcolm: Humaneyes is an innovative technology company that really has its roots in graphic arts and computer animation. But started to see the real benefit and future in virtual reality and has applied its technology and patents to really defining the future of virtual reality.
Larry Jordan: One of the ways you’re defining it is a product called the Vuze VR camera. What’s this?
Jim Malcolm: So the Vuze camera is really a very powerful camera system, self contained, that basically takes two series of video. One for your left eye, and one for your right eye, and automatically creates what we call VR video. This is something that you can play back in a VR headset, and see everything around you in full 3D with ambisonic audio.
Larry Jordan: I hate to break it to you, but most of us would call this stereoscopic 3D and you’re calling it VR. How come?
Jim Malcolm: Because it’s the intent of the camera, in fact not just our camera, but the category is to play the product back in a headset. And unlike when you look at 3D stereoscopic on a movie screen, or on a flat 2D environment, the intent of VR video is to be able to allow you to look everywhere and add interactivity to that video as well.
Larry Jordan: So are you shooting a 360 degree image off the left eye, and a separate 360 degree image off the right eye?
Jim Malcolm: That’s correct. So the camera systems actually have, in our world, a series of eight cameras. We use four of those to create a 4K spherical video for your left eye, the other four for 4K video for your right eye. We put those together in software, sync them and time them and make sure the color matches, so when you put it into the headset, you have this seamless 360 degree 3D world, really captured from two different viewing perspectives.
Larry Jordan: How do you deal with issues like nausea and disorientation, because a lot of people would not be used to viewing this?
Jim Malcolm: You know, it’s a really big challenge. One of the biggest complaints about VR is “Wow, it makes me feel nauseous.” I’ll go back to the early days of motion pictures, back when people sat in movie theaters for the first time, and they felt nauseous inside of a movie theater. The brain wasn’t used to the experience. So there’s a couple of things that happen, one is the technology’s improved which makes things a little bit easier inside the headset. Frame rates and color and syncing between each eye. But also very important to it, is creators are learning special techniques or discipline techniques in order to shoot in a way that you don’t make your viewer sick.
Larry Jordan: For instance?
Jim Malcolm: In the past when people first started shooting with VR, maybe they would do a handheld shot across a room. The problem with that is regardless of how steady you are, you start getting a little bit of shake to the movement or it goes up and down. Your brain’s not used to dealing with this, so either a camera placement that’s stationary and move your action around it, or a stable camera platform that moves the camera very uniformly is a lot better way to keep people in a headset without feeling nauseous.
Larry Jordan: Is it not only an issue of camera stabilization but also resolution? Does higher resolution improve the viewing experience?
Jim Malcolm: It’s a tough one to answer. The reason I hesitate is, I’ve been in experiences that are pixilated and have terrible image quality but they don’t make you sick. So in and of itself, it’s not the resolving power. In fact, all of these VR headsets today really don’t have the greatest image because they don’t really have the highest resolution. What’s more important than the resolution is the actual frame rates. So if you take the same experience and view it in a mobile VR like on a Gear VR headset that plays back in 30 frames per second, you might feel a little bit more nauseous or a little more uncomfortable, let’s not even call it nauseous, than if you took that same video and played it back on something like an HTC Vive, which the computer takes that video and rezzes it up to 90 frames per second. That higher frame rate eliminates pulse when you’re looking around quickly in that headset and help gives you a more comfortable experience.
Larry Jordan: One of the things that we’re all looking at in VR is how it breaks out of being a niche product into a mass market product, and it seems to me that the big gating factor is this headset that we have to wear. As opposed to AR which can be seen on say a mobile device. Do you think it’s possible for VR to achieve mainstream status or is the experience itself because it needs a headset going to be limited to more specific applications?
Jim Malcolm: VR is going to make mainstream status, and it’s going to first start probably at your workplace, whether it’s training, education, remote conferencing. There’s a lot of very practical applications for wearing a headset. It’s those kind of defined disciplines that are going to allow us then to open the door towards a social VR and VR for entertainment and some of the other components. It’s interesting right now in the news and in the press, you see a lot of discussion around the entertainment properties. There was this movie, or there was this event or whatever, the Olympics were just shot in VR, and people are trying to understand what is the value to that? You can take that same technology, that same capability, and apply it to how to inspect an oil rig before you fly somebody out to that platform for example, and you can deliver a very rich experience, play all kinds of value to it and people don’t think about the entertainment, they think about the practical part. We will start to incorporate VR into our everyday workplace and then we’ll start to bring it into our everyday social worlds.
Larry Jordan: It seems to me that the initial foray with VR is going to be experiences and education because I think narrative’s going to take us a while to develop. Would you agree with that?
Jim Malcolm: I think that’s true. We’re already seeing it, and what’s interesting from an education point of view, which I didn’t expect, originally the educators were looking at how do I share my course, my class, or my discussion in virtual reality? That’s kind of plateaued a little bit but we see this huge spike at learning institutions themselves, adopting the Vuze camera in order to teach the principles of VR. The reason they use it as a teaching tool is, they can teach virtual reality without having to make the students computer programmers which really opens up a much larger audience to VR, including journalism. The list goes on and on, education, government, law enforcement, where you can have a class on just about anything, and people can start to create content without having to become a programmer.
Larry Jordan: If you were to advise a filmmaker, not necessarily to create narrative fiction because for me the jury is still out on that, but the idea of experiences in education makes perfect sense. If you were to advise them on how to create content for 360 VR, what are some of the top rules they need to keep in mind as they’re starting their own production experience?
Jim Malcolm: I’ll first say that I hate rules but you do need some rules or guidelines to get started, because the principles of storytelling remain the same. I think my biggest frustration to us as an industry is that because a lot of people haven’t figured out how to tell a narrative in a VR headset, then the whole narrative story gets a bum rap.
Larry Jordan: And you don’t think it should?
Jim Malcolm: I don’t think that it should because the opportunity to tell a story, regardless of the medium, is there. And it is so much more immersive and so much more impactful, but again the problem that we have is that too many people have tried to simply replace a camera placement or replace a production that they used to do with a 2D or a fixed view camera, with a virtual reality camera with mixed results. But if you actually start by creating the script and the dialog from the very beginning as a VR narrative, how you bring that story to life, it makes sense that it’s in a headset. It makes it have a purpose and I think that’s the biggest gap. I guess my recommendation if you’re starting to dabble with, explore, quite frankly even deliver for your clients, make sure whatever you make has got a story. The same way that you would do with any other video or film equipment.
Larry Jordan: One of the ideas that I found very helpful in understanding VR was to think of it not as the typical filmmaking experience but more like creating live theater where you still have creative control but the process is different between the two. Would you think that’s a good analogy?
Jim Malcolm: I actually think that’s a great analogy and the reason that I like that analogy is, when we think about directing an existing film, you’ve got your craft cart and the sound guy and the lighting guys, and you’re working in a confined space. When you do look at that whole stage and you know that your audience is there and you know that you’ve got to keep all of your support off or out of view, I think it makes a lot of sense. I think that the challenge of the director who’s used to filming in 2D traditional film, maybe has a much more difficult time getting that in the head than somebody who’s directing stage performances. So I hadn’t thought about that, but I think live stage performance, that kind of core competency that it takes to be able to create that, is absolutely the base foundation to build virtual reality on.
Larry Jordan: So let’s get back to those rules that you hate. You’ve already told us that we want to make sure the camera is really stable, that we don’t do hand held shots. It’s also because we’re seeing in 360, we can’t suddenly cut to a tight shot because that means we’re going to a wide shot at a different angle, so what other suggestions do you have us consider as we play with this, to make ourselves feel like we’re being successful?
Jim Malcolm: There’s a couple of things that you can do and I wouldn’t say that you can’t go from a wide shot to a tight shot, because what you have to start thinking about is camera placement. Because basically what you’re doing is you’re taking the viewer, and you’re transporting them, or teleporting them to a new direction. So you need some sort of an event that triggers a logical reason why you’re moving. Maybe it’s a fade to black, maybe it’s a sound, maybe it’s a bright light, maybe it’s tracking some motion. Then you have to have a reason for why that camera shows up at a new location.
Jim Malcolm: So the example I’ll give you is out of a series called Invisible, and spoiler alert, but there’s a section where you are this third person and you’re looking over in this room and then it immediately cuts to this scene where the camera is placed on the ground. It is the most uncomfortable feeling you’ve ever had, because it almost feels like you’ve been decapitated, sorry to sound so negative, and it’s very uncomfortable because we don’t sit on the ground. But when you turn around and you look behind you, you’re actually sitting in a pool of blood with a dead body. So as the movie unfolds and there’s a reason for you to have this impact and end up on the floor, the impact of that transition is huge. When done right it is impactful, it’s memorable and helps further advance the story.
Larry Jordan: There’s a lot of different VR cameras out there. Why should somebody consider Vuze?
Jim Malcolm: The short answer is cost. The second part of it is ease of use. What we’ve done is, we’ve made two different versions of the camera. One is an $800 product, the other is 1200. Basically the difference between the two is the $1200 camera actually allows you to do live streaming, and to be able to capture video directly to a hard drive on a computer. At those price points, with a system that can render out as high as 4096 by 4096, we can create content that exceeds the playback capability of VR headsets today. For not a lot of money, any filmmaker who has the bug or the curiosity to figure out, “How do I start to tell a story in VR?” can create some amazing content today.
Larry Jordan: For people that want more information, where can they go on the web?
Jim Malcolm: The easiest place to go is to vuze.camera, there is no dot com in there, but just vuze.camera. That’ll bring you to our Vuze camera website, there are samples that can be downloaded, you can learn about the product and you can purchase right on that site.
Larry Jordan: Jim Malcolm is the general manager for North America for Humaneyes Technology, the folks that make the Vuze camera, and Jim, thanks for joining us today.
Jim Malcolm: Thank you.
Larry Jordan: Srinivas Krishna is the founder and CEO of the pioneering AR VR studio called AWE Company Limited. For the past six years he’s been engaging mobile users with ground breaking AR and VR experiences. Now he and his team are releasing a new mobile app that lets people create augmented reality videos using only their smart phones. It’s called Geogram. Hello Srinivas, welcome.
Srinivas Krishna: Thank you Larry, glad to be here.
Larry Jordan: How would you describe what AWE Company does?
Srinivas Krishna: AWE Company, for the past six years or so, has been creating technologies and experiences for mobile users of augmented and virtual reality experiences. We’re a studio that serves businesses and their users essentially.
Larry Jordan: Why this? Why VR?
Srinivas Krishna: I had been making films for 20 years, and not long after the smartphone came out, I had a kid. And when I saw him engage with an ipad which I didn’t know how to turn on, I knew that life was really going to change, and I had to face the future head on. So I really did fall into a rabbit hole. I started exploring augmented reality and what I could do with mobile phones, and devices, and there was no looking back.
Larry Jordan: Tell us about this new thing you’ve got called Geogram. What is it?
Srinivas Krishna: Geogram is a mobile app that lets you share your memories where they happen. Today we have social media that essentially allows us to create a post about our experience at a place, it’s about a place but really the place itself is disassociated from the story or the memory or the video. What Geogram does is it allows us to mark our presence at a place with an augmented reality video that we can record and leave there and communicate our presence from that location and share it. You can share it across your normal social media channels, but if you have the app it transports you to that place in virtual reality where you can see that video. If you’re at the location, you can see the video there, playing for you and any other videos that go alongside it.
Larry Jordan: I had the chance to watch the story that NBC News did on you where they profiled Geogram, but give us an example that NBC talked about, of how this is used in real life.
Srinivas Krishna: That was almost the first use case of the product, and we were still building it at the time. We had a client in Orlando who was building a memorial in honor of all those who’d been impacted by the tragedy of the shooting at the Pulse nightclub. We had told them about this app that we were building that might be just more useful and easier to use than the augmented reality projects that we’d been doing till then. He was very excited about it so we released an early version of it and asked him if he would like to share it with people in his community. What happened was that people who were impacted by this tragedy started going to the location of the nightclub and leaving their sentiments, their statements for the people that they had lost, at the location, much like a digital memento. I don’t know how else to describe it, it was a digital memento, it was like a virtual shrine at the location. Extremely powerful experience for us and it’s really what got us down this road to say there’s something very powerful at the core of this technology and what we’re able to enable with it, and we really felt compelled to develop it and release it.
Larry Jordan: What do you see as the differences between augmented reality, AR and virtual reality, VR?
Srinivas Krishna: Augmented reality marries digital content with the world. So it can super impose or imbed a 3D object, a virtual object, or a video or a picture, anything you want, into the world that you’re looking at through the camera on your device. Virtual reality is an immersive experience that is 360, and completely envelops you inside its world. The result is they have different use cases, augmented reality is very much something that you might do on the go. It’s more active, you can be in motion. You can be walking around. Virtual reality, because it’s so immersive, requires you often to sit down, be more passive, not as active in the experience of it.
Larry Jordan: Do you think VR’s success is limited by the fact that every viewer needs to wear a headset, where with AR we don’t?
Srinivas Krishna: I think they’re different kinds of experiences and they have different ways that we measure their success. For example, movies have a certain kind of experience. They’re successful, you have to go to the theater to have that theatrical experience. In the same way, I think virtual reality, we just need to look at the success of it differently. It’s the kind of thing that maybe belongs in a basement or a man cave, you’re at home, it’s passive. While AR is really much more ubiquitous, it gives you information about objects, about the world. With Geogram it can give you more than just information, it can give you stories and memories and emotions.
Larry Jordan: With VR, we’ve been hearing the buzz about VR for a long time, but AR has sort of been flying below the radar. What’s holding the technology of AR back?
Srinivas Krishna: Great question. What’s been holding it back is really just that it needs to get on a great number of devices. I think that’s what’s changing this year is that both Apple and android are embedding AR technologies inside their operating systems. So that now it’s much easier to create content and consume it in a more exciting way. So the basic technologies that underlie AR are the ability to superimpose or position digital content realistically in the real world through a camera. Sounds easy the way I say it, but it’s actually a lot harder than it sounds. The other piece of technology that is important for AR is the ability to share a piece of content between two or more people. So that they’re each able to experience the same digital content in the world, each from their own perspectives. These are now starting to become more and more commonplace, because the big Apple and Google and Facebook and the big platforms are really investing in it and making it possible.
Larry Jordan: What do you see as the low hanging fruit for AR? What apps, and I use apps in the broader context, what uses will we see first?
Srinivas Krishna: I think a really great use case is way finding, helping you get around. A lot of people just have trouble with maps. I do, I look at maps and I say “I’d rather just ask someone the directions.” I’m that kind of person. So, now with AR what you can do, is you can point your phone at the world, it can recognize where you are and you can tell it where you want to go and it’ll give you directions in real time.
Srinivas Krishna: The other thing is you can get information about the world very easily just by looking at objects or buildings and your environment. In a sense, that’s what Geogram enables you to do, it enables you to experience the stories of a place. You go to a location, and it used to be in fact that to learn any kind of story you had to go to the place where the story happened. This was a long time ago but that’s where you’d go and people in the community would tell you the stories. Today much of our stories are disconnected from places altogether. They live in screens, they live in print, they don’t really have any physical bearing with that location any more. What augmented reality does is it allows you then to go back to that place of stories residing in a place which Geogram enables for example. There’ll be others that will do it over time. That’s a remarkable thing. It gives stories context. It gives them a setting. So these I see are the low hanging fruit, they’re things that anyone can do and I think we’ll start seeing that rolling out pretty quickly.
Larry Jordan: Has Geogram been released and if so, what’s the cost?
Srinivas Krishna: We are releasing Geogram in fact this week.
Larry Jordan: Congratulations.
Srinivas Krishna: It’s live in the app stores. We’re building new features every day so we’d love people to start using this, especially content makers and creators because when you look through, there will not be many Geograms in the world so we’re starting in a sense from scratch. So we’d really love people to start sharing their memories and their experiences and whatever you see or experience that is really interesting to you and to other people on the platform.
Larry Jordan: What does it cost?
Srinivas Krishna: Geogram you’ll be happy to know, is absolutely free, and we hope to keep it free for individual users forever.
Larry Jordan: For people who want more information about it, where can they go on the web?
Srinivas Krishna: They can find us at Geogram.xyz.
Larry Jordan: That website is geogram.xyz. Not dot com, geogram.xyz. Srinivas Krishna is the founder and CEO of the company that created it, which is AWE Company, and Srinivas, thanks for joining us today.
Srinivas Krishna: Larry it’s been my pleasure. Thank you so much.
Larry Jordan: Ethan Shaftel directs film, VR, immersive media and interactive projects. His background is in spatial media and video installation projects. In fact he’s created media for multi screen pop concerts and music festivals as well as animation wrapped floats for Disney. Hello Ethan, welcome.
Ethan Shaftel: Hi Larry, I’m delighted to be here, thanks.
Larry Jordan: Today we’re talking about VR. What got you interested in VR in the first place?
Ethan Shaftel: I love interactive storytelling. I’ve always loved film, but I was never much of a gamer, but I loved Mist when it came out and even though it was a game, you were solving puzzles. Mist the computer game was so immersive and so story driven, that I felt like it was a great example of what could be done, but I felt like when VR came back on the scene about three years ago that it was a wide open platform, a wide open media that could be defined in any way we want, and so a interactive narrative of VR experience, doesn’t need any more definition than just VR. In a sense it’s new, we can make it whatever we want.
Larry Jordan: Is VR better suited for experiences as you describe, or better suited for education or narrative story telling?
Ethan Shaftel: I think it’s very suited to all these things. The power of the medium fully, we know the things that grab us instantly, about VR the first time we put on a headset, and that’s really the immersion, that sense of presence. But that’s only the first step. What do we do with that? We transported an audience member or viewer somewhere, that’s the first step of VR. Then what do we do? VR games are really obvious successes. It makes a ton of sense to then have that viewer shoot zombies or play tennis. That’s going to work really well, those are obvious successes, and I don’t mean to denigrate that when I say obvious. But that’s the low hanging fruit of VR. But then what else can we do? What type of other experiential or narrative experiences? Those require some more thought and those are very much still in development right now.
Larry Jordan: Tell me about some of the projects that you’re working on.
Ethan Shaftel: When I got into VR I was doing prototypes and experiments and things on the side with my existing collaborators, a lot of people involved in animation and motion graphics. The first stand alone piece that we decided to make and try to get into festivals, was a mix of animation and live action called Extravaganza. That was lucky enough to get into Tribeca last year and premier there in April. That’s opened a lot of doors subsequent to that, and we’ve been very fortunate about getting Extravaganza out there. So now I’ve been working on a variety of things, most recently a piece for Red Bull. Our VR version of the classic bonus stage from Street Fighter 2.
Larry Jordan: Films are generally viewed as part of a group, in a theater. How do you answer the charge that VR by isolating each viewer in their own headset, is an isolating experience?
Ethan Shaftel: That’s a really interesting topic. We participate in different media in different ways. I don’t think the idea that VR replaces or represents an evolution of film is accurate. There is huge overlap between VR and film in certain visual and storytelling ways. I think that overlap will continue to be something we explore but it’s not the same. VR in that sense, at least the pieces that we see out there in the narrative space, and the things that I’ve made and things like Extravaganza, they’re more of an experience similar to something like … We don’t tend to read novels out loud to each other or even in a room full of people, it is something that is a single person experience.
Ethan Shaftel: A word that comes up a lot with great literature is immersion. You sink into it, you feel that you’re there. A little bit different from the literal immersion of VR, but similar and overlapping. So there’s a place for single experiences, or experiences that are primarily immersed in by an individual. What I think we see is that how we connect to great literature, becomes really important to us and becomes a way of actually making social connections. On your dating profile you say your favorite book and if someone else has that favorite book, that’s really meaningful. That means that you guys should probably be talking, you should probably go have coffee. Going in deeply into an experience, it’s when you come out and you start talking about it with others, sharing your experience, comparing and contrasting, that’s what makes art social. Even if it’s not in a room full of people.
Larry Jordan: When you’re creating your own VR experiences, what tools, what software do you use?
Ethan Shaftel: The first software that me and my collaborators were really using to experiment was Cinema 40. Largely that’s because it was already an animation software that we were using for traditional cinema projects, motion graphics, animation that were not … C40 had already some really good tools and the ability to go into existing animations, maybe you’ve already done, and then used the virtual VR camera to export them and view them in a headset? And then learn of course what falls flat. How you need to rework staging and you need to think about space in a different way, and that was really the key. You’ve got to start playing with it if you want to do it at all. Then as we grew, the software grew as well. The VR features are much more robust now than they were then, and that’s really exciting to see in concert with all of our tests and experimentation.
Larry Jordan: There’s stereoscopic 3D, there’s 2D, there’s 360 degrees, versus 180 degrees. How do you pick the right format for a VR project?
Ethan Shaftel: That’s a great question. When I use the term VR and even just in this conversation, I am very inclusive with that. I think that when the primary aspect or at least that initial aspect of the piece of art regardless of what it is, transportation to a new world, replacing what the viewer sees and hears in their living room with something else, then that counts as VR, virtual reality. So something like Extravaganza, my initial VR movie, is stereoscopic 360 video. It means that it’s three degrees of freedom, you can look around but you can’t get up and walk across the room. It’s not interactive, you know, it’s a linear piece of media that’s going to play from the beginning to end and aside from the ability to look around at any location, and any place in that world, from your locked point of view, there is no interactivity. So I think that’s very much VR, but of course the creator has to understand the technological limitations of that choice of what kind of flavor of VR you’re going to use.
Ethan Shaftel: So with Extravaganza, making it work with the storyline, the viewer becomes a puppet locked into a VR puppet show. You have a body that is interacting with other characters, you see it moving, but someone else is controlling you. That kind of storytelling works well with 360 video because it leans into that idea of lack of agency and the lack of control you have except for what you’re seeing. But that wouldn’t have been a good choice at all for the Street Fighter game. That needed to have a much higher level of agency because it really is a game. You’re running, you’re punching, you’re kicking, you’re moving, and you’re scoring more or less points depending on the damage you do. So for that, we did it on a game engine with a six degree of freedom platform. I don’t think that 360 ever hurts. If you can do it, and make a wider field of exploration for the viewer, then I think that’s a benefit and it makes you think about attention and the audience’s view of the world in a really good way. So I don’t think it’s meaningful to cut that down arbitrarily. However the tenable side of it, there’s a lot of advantages to only shooting 180. You have an off camera, you have the ability to hide equipment and use lighting in a different way. Plus it can be less expensive.
Larry Jordan: Ethan, for people that want to be able to take a look at the projects you’re creating, where can they go on the web?
Ethan Shaftel: You can find me at ethanshaftel.com.
Larry Jordan: That’s all one word, ethanshaftel.com. His website is excellent. Make a point to go visit, and Ethan, thanks for joining us today.
Ethan Shaftel: Thank you so much. I was delighted to sit down and talk.
Larry Jordan: Here’s another website I want to introduce you to. doddlenews.com. doddleNEWS gives you a portal into the broadcast, video and film industries. It’s a leading online resource, presenting news, reviews and products for the film and video industry. doddleNEWS also offers a resource guide and crew management platforms specifically designed for production. These digital call sheets, along with their app, directory and premium listings, provide in depth organizational tools for busy production professionals. doddleNEWS is a part of the Thalo Arts Community, a worldwide community of artists, film makers and story tellers. From photography to film making, performing arts to fine arts, and everything in between, Thalo is filled with resources you need to succeed. Whether you want the latest industry news, need to network with other creative professionals or require state of the art online tools to manage your next project, there’s only one place to go. doddlenews.com.
Larry Jordan: Matthew Celia is the managing director and creative director of Light Sail VR. This is a company that creates engaging VR content. Hello Matt, welcome back.
Matthew Celia: Hi, thanks for having me.
Larry Jordan: How would you describe Light Sail VR?
Matthew Celia: Light Sail VR is a Hollywood based VR production company, and we are pioneering the evolving field of cinematic virtual reality storytelling. Our focus, being from a film background, is on bringing compelling narratives and characters and cool visuals to every project we touch. So far we’ve made our name producing high profile pieces for Google, ABC, GoPro, Paramount to name a few. But we’ve just recently released our first original project called Speak of the Devil.
Larry Jordan: Well before we get to Speak of the Devil, isn’t cinematic VR storytelling an oxymoron?
Matthew Celia: Oh I disagree. No I think you can always put a story into anything. It’s just how you execute it and make it work for the …
Larry Jordan: We’re going to talk a lot about how you execute it. Tell us what Speak of the Devil is before we talk about how you created it.
Matthew Celia: Sure, Speak of the Devil at its heart is a horror film that takes place in the forest. It’s about this woman who accidentally summons a demon but the unique thing about it is that it is an interactive live action film much in the style of an immersive theater piece. That was a unique challenge in VR.
Larry Jordan: What made it a unique challenge?
Matthew Celia: Well what we had to do is we had to put together this story that made sense and figure out this logic programming, so that the audience could explore it at their own pace and go wherever they wanted to go but we weren’t going to lose that storytelling, that narrative aspect.
Larry Jordan: But wait a minute, this is a horror film, and people are exploring a horror film under their own initiative? Horror I’m afraid is not my favorite subject, but this strikes me as something that people would not want to do.
Matthew Celia: Right, but you know, horror theme houses and haunted houses are very popular and escape rooms and all that stuff. So we’re tapping into that. We’re tapping into that thrill and what we do is we drop you into the forest and you meet these characters and you go off searching and things are just not right. And eventually we’re ramping up that tension, just as you would in the traditional screenplay. We really broke it down into looking at here’s the setup, the theme, the inciting incidents, the all is lost moments, the final climactic end. We took that structure and we just started parceling it together.
Larry Jordan: I want to come back to writing the script in just a minute, but before we do, what’s the status of Speak of the Devil now?
Matthew Celia: Speak of the Devil has just been released a few weeks ago. It is available on the Google Daydream store and on the Gear VR, the Oculus store. It’s for sale for like three bucks, so forego that one cup of coffee and check it out.
Larry Jordan: Hope you sell tens of thousands of them, I wish you great success.
Matthew Celia: Thank you.
Larry Jordan: Earlier tonight, Ethan Shaftel mentioned that writing a VR script is different from writing a normal film script. Would you agree or disagree? And if you agree, what makes writing a VR script different?
Matthew Celia: I agree and I disagree. I don’t agree that you need to write a VR script where you describe the quadrants which is very popular for some reason, which say in front of us, there’s this, to left of us, this. That’s the director’s job. It’s a director’s job just like in a traditional screenplay you wouldn’t put every single camera angle. But it is different because I think you need to think carefully about how characters are entering and exiting a scene. I think you need to think about more descriptive places and in this latest project that we did, we had to think about how is the logic of an interactive story going to work so that it makes sense no matter how people experience it?
Larry Jordan: One of the things that I think you and I talked about the last time you were on which was in November, and if not, take the credit for it anyway, because I found it incredibly brilliant, is that you made the analogy that VR is much more like staging a theater play than staging a traditional film. Would you still agree with that?
Matthew Celia: Absolutely. I think every project we do we just kind of reinforce that and it just proves it to us that VR narrative cinematic storytelling is like immersive interactive theater.
Larry Jordan: I love that phrase. Alright, I’m going to steal that one from you too.
Matthew Celia: Cool.
Larry Jordan: When you were shooting this, what kind of camera gear were you working with? What was your production hardware?
Matthew Celia: We shot with the GoPro Odyssey. We were so lucky and fortunate to be in the Google Jumpstart program which is this amazing program from Google that gives indie filmmakers the camera and then access to the post production stitching software gratis, totally free, no strings attached. It’s incredible, and we were able to shoot it at 5.7K, left eye and right eye, so that’s quite a bit of data, at 60 frames a second. It really is like the pinnacle of quality for stereoscopic live action VR right now.
Larry Jordan: How did you edit it?
Matthew Celia: We edited it, and this is kind of funny actually, we used Premiere Pro and we broke every scene down into its own sequence, and we were editing each part, each scene individually because it’s a narrative that has multiple options. But we found that we really couldn’t experience it or play it back or understand the audience is going to view it until we put it into a game engine. So we used Unity to actually do the playback of it so that we could write all the logic code that would tell us when we could see each scene.
Larry Jordan: Did you shoot 360 or 180 and do you have an opinion on which format is better?
Matthew Celia: We shot 360 and we’re actually about to do our first 180 piece in a couple of weeks here. I think the formats are very different. I think 180 for me, you lose some of that immersion. You lose some of that sense of presence that you get with 360, because if you’re turning your head to the left in 180 you’re going to see a black line. But 180 is a lot easier to make, a lot easier to produce, the quality of the stereoscopic imagery and the resolution is a lot higher. And right now when we’re in these early days when there’s not really much of a monetization path and things still take so much longer, and the dollars aren’t there, you need to come up with concepts and ideas that work not only creatively but financially. And I think VR 180 is a really strong step in that direction.
Larry Jordan: So maybe you think VR 180 is an interim step before we have the equipment and the budgets for 360?
Matthew Celia: Absolutely. And I think it works really well for point of view stuff, where you’re talking to a person directly. But I think eventually it’s all going to end up in 360 because you really do want that full presence I find.
Larry Jordan: Well this I think gets to a bigger question, can VR really become mainstream when we all need to wear a headset to see it?
Matthew Celia: You know, I think right now our headsets remind me of the Gordon Gecko cell phone. They’re big, they’re clunky, people are like, “Oh that’s never going to take off. Who’s going to have a cell phone in your car?” But now look at where we’re at. When you’re starting to wear glasses with augmented reality where the lenses become fully opaque, that’s virtual reality. And I think that there is location based experiences that are really popular, and theme park experiences that are really popular. Will it be as popular and ubiquitous as the television? I don’t know, but I really think there’s a strong market and a strong group of people who enjoy escaping from their reality for a few minutes a day.
Larry Jordan: So it sounds like overall you’re pretty optimistic?
Matthew Celia: I am, especially with the Oculus Go’s coming out later this year. This is a $200 stand alone headset. You don’t need a Samsung phone or a special phone, you don’t need anything. It’s very affordably priced, and it has access to all of the amazing content on the Oculus store. I think that the cost right now is still too high, and when that hits the market, I think we’re going to start seeing consumers adopt in even bigger ways.
Larry Jordan: You’ve done a number of VR projects, and now that you’ve got some experience under your belt, what’s the number one piece of advice you’d give a producer who’s starting to experiment with it for the first time?
Matthew Celia: Know your whole workflow. Know it from beginning all the way to distribution. A lot of these tools are really new and they’re always in beta. A lot of the distribution platforms are always in beta, so things are changing a lot. So you need to work with a partner who is up on it, knows the whole pipeline from A to Z, because they will help you achieve a better quality product because they know all the little … and every step of the way that segmenting it out you might miss.
Larry Jordan: For people that want more information about the kind of creative projects you’re coming up with, where can they go on the web?
Matthew Celia: You can check us out at www.lightsailvr.com.
Larry Jordan: That’s all one word, lightsailvr.com and the managing director and creative director of Light Sail VR is Matthew Celia, and Matt, thanks for joining us today.
Matthew Celia: Thanks Larry, appreciate you having me back.
Larry Jordan: Oh, always a pleasure. Take care, bye bye.
Larry Jordan: I was just thinking, most of us look at the world based on what we know. For example, most of us are filmmakers in some form, and look at the world through a media lens. This often colors how we view new technology such as VR and AR that we’re talking about tonight. Jim Malcolm made a good comment this evening that helped me understand this better. When he said that VR is not a replacement for traditional filmmaking, any more than TV was a replacement for film. Years ago when I first heard of VR I was puzzled, because I couldn’t see how it would benefit narrative storytelling or traditional filmmaking. What I’ve learned since is that I’m trying to put a square peg in a round hole. VR isn’t filmmaking. It’s its own thing.
Larry Jordan: A better way to think of VR is that we’re creating the digital equivalent of live theater. Theater can be just as involving emotional and uplifting as the best films, but it uses entirely different techniques and technology to tell its stories. The same with VR. The more we think of it as filmmaking, the more frustrated we’ll become. That’s why experimenting with this format is so important. We need to try it for ourselves, see what works and what existing skills we need to unlearn.
Larry Jordan: VR provides us, especially those of us with theater backgrounds, with an opportunity to provide expanded services to our clients. The world of VR is still young, there’s room for more people like us to explore and expand the medium, and along the way, it provides those who do with the ability to generate more revenue and stand out from the increasingly crowded filmmaking pack. Because at the end of the day, there’s nothing wrong with paying the rent and sleeping soundly at night. VR gives us the opportunity to explore something new and make money with it at the same time. Just something I’m thinking about.
Larry Jordan: I want to thank our guests for this week, Jim Malcolm with Humaneyes, Srinivas Krishna with the AWE Company, VR filmmaker Ethan Shaftel, VR filmmaker Matthew Celia, and James DeRuvo doddlenews.com.
Larry Jordan: There’s a lot of history in our industry and it’s all posted to our website, at digitalproductionbuzz.com. Here you’ll find thousands of interviews, all online and all available to you today. Remember to sign up for our free weekly show newsletter that comes out every Saturday.
Larry Jordan: Our theme music is composed by Nathan Dugi-Turner with additional music provided by Smartsound.com. Text transcripts are provided by Take1 Transcription. Visit Take1.tv to learn how they can help you.
Larry Jordan: Our producer is Debbie Price, my name is Larry Jordan, and thanks for listening to The Digital Production Buzz.
Larry Jordan: The Digital Production Buzz is copyright 2018 by Thalo LLC.