Since the age of 18, multidisciplinary artist and director Jake Oleson has been directing documentaries, short films, and commercials for brands such as Airbnb, Facebook, P&G, and Gatorade – garnering numerous awards and Vimeo Staff Picks along the way. Lately, he has also been fusing his cinematic work with his music in live audio-visual shows. His recent project, "Given Again," performed at Siggraph and won silver at Runway ML's AI Film Festival.
As part of the launch of Vimeo's new app for Apple Vision Pro, we asked him to create an original short film for us that was shot entirely in spatial format. Armed with a single Canon camera with a dual fisheye lens, Jake and his team flew out to Vietnam to create "currents," the story of a young Vietnamese woman named Lynn, and her experience traveling to and arriving in Ho Chi Minh City for the first time.
During a recent visit to our New York office, Jake sat down with us to discuss the unique experience of shooting in spatial and his thoughts on the format in general.
For people who have never seen a spatial video or understand the technology, how would you describe it to them? What makes viewing this content with an Apple Vision Pro so remarkable?
Spatial video capture is a step in a step closer to the way that we see the world. So, instead of capturing video from a single perspective, we're capturing video from a stereoscopic perspective, with two lenses placed exactly or approximately as far as our two eyes are apart and then synced together to watch with a VR headset like the Apple Vision Pro.
What makes the Apple Vision Pro so remarkable? Its fidelity, among many other things, but for me, it's its fidelity. It's 8K per eye at 90 frames a second. Our film was only 60 frames, but it still is fluid enough to begin to trick the brain into thinking you're actually in the room with the character.
As an artist with a background in traditional directing and more experiential work, what was it like shooting for spatial for the first time?
Working in immersive, VR, and spatial for the first time was a huge learning curve that was intimidating but also really exciting because it's different from traditional filmmaking in ways that can be a bit of a puzzle to work out sometimes. Whether it's moving the camera in a certain way that's careful to not make people sick or just understanding that a closeup feels very different than a traditional closeup in a two-dimensional film.
Luckily, we did have a period of time before going to Vietnam to test everything. So I was shooting as much as possible and testing it in the headset and understanding what kind of camera moves feel good, what camera moves don't feel good. It's similar to traditional film, but it's not the same. It really does capture this sort of presence the same way that it feels to sit in a room with you now. It feels like putting on the Apple Vision Pro and witnessing these characters.
You had to keep in mind that if something's happening to the right or left in the space, it's part of the story and it might be something that the viewer would focus on. How did that impact your process?
I think the main difference, the learning curve for me with immersive is understanding that the viewer is really the cinematographer and where they choose to look can be anywhere in the full stereo view. That means that in terms of directing attention, you need to take a few extra steps to understand where we're drawn to look when you're cutting.
So for instance, there's a shot in the film where Lynn, our main character first arrives in Saigon, and we cut to this apartment complex and the camera is suspended above, just in between these two buildings. And immediately everyone looks down because it really feels like you are floating almost. And the following cut is this kid hitting a soccer ball, like kicking a soccer ball in the street. And it's this intentional cut where you need to anticipate where people are going to look and where their attention is going to go.
We tried to make sure that all of the action and the majority of the storytelling beats happen within the center of the frame. And that auxiliary details that complimented the story existed beyond this center action. So you could see the woman that she's living with, if you were to look out the window to the right or the people that she's working with in the market, and all these details are a part of it, and you could take all of that in, but the majority of Lynn's story is unfolding right in front of you
You really need to distill it down to its essence. So I thought, okay, we have a character that's arriving to the city for the first time, and we have people that are experiencing the city for the first time, experiencing spatial for the first time and keeping it simple, following her progression from working in her job in the marketplace to making the first friends.
For me, that was a huge lesson in understanding the potential of spatial, even just holding on shots. There were earlier edits where we would cut after seven seconds or so, which is a long shot. But in spatial, it feels very quick because you're taking all of this new world in. It's so much new information that if you don't spend enough time in it, it could feel like you're being rushed through the story.
I love the parallel of you as a filmmaker, bringing the audience into a space for the first time, and the principal character in your story is also going on that journey.
Yeah, I mean, when you're watching traditional films, there's a barrier between you and the character, and so much of the director and the writer and the team's job and energy is spent developing the relationship between the viewer and the main character in the story and gaining that trust.
With immersive filmmaking, you're really dissolving that barrier and you're not watching a story, you're witnessing a story, you're witnessing a character. And for me, witnessing means to be sitting in the room and seeing them as a human being and really feeling their aura and their presence in a way that I personally haven't experienced. In traditional film, you get glimpses of it, but still, you're always sitting in a theater or your room watching through this screen, through this window. I didn't realize the kind of impact that certain shots would have and the level of trust and strength between the viewer and the character that can be established in a short period of time.
Filmmaking is this empathy machine. As Robert Roger Ebert says, we have this beautiful mechanism to be able to garner empathy with people in a short period of time, and immersive filmmaking is closing that gap even further. And so I think this opens up new paths for telling different kinds of stories, more challenging stories, more complex narratives that would be difficult for a viewer to wrap their heads around that we now can arrive there or at least begin to explore that landscape because you're physically within the space with that character.
If you had to distill your learnings into a couple tips someone should know that wants to start shooting something using spatial immersive video, what would those be?
Yeah. For me, it's prioritize experimentation, and lean into the way your body is feeling when witnessing these characters, because it's different from the two dimensional traditional filmmaking process. Making "currents" was about opening my mind to the possibilities of immersive filmmaking and not getting caught up in my idea of the way a film should be, because this is much more of an experience that's washing over people than it is watching a traditional film.
With this new level of immersion, with immersive filmmaking and spatial, because you're bringing people into the scene, you don't want to whip them around. And so moving the camera is a very fast way of making people uneasy and making yourself uneasy when revisiting the footage. You feel like you're in the scene. And because of that, we're getting that much closer to the story and that much closer to the character, but also opening up this entirely different world of pace. Things take longer to sink in, and stimulation is a lot higher when it's all washing over you at once.
