While many of the virtual reality experiences being shown off at the 2017 Tribeca Film Festival are pushing boundaries in VR story-telling techniques, “Hallelujah” is perhaps most noteworthy because of the cutting-edge tech used to shoot the experience and the story behind it.
Lytro has been around for 11 years; raised more than $210 million from Silicon Valley’s top investors, including Andreessen Horowitz, Greylock Partners and NEA; and yet, this VR music video represents the startup’s public-facing square one as it executes on a pivot it’s been building up for the last two years. It’s all designed around Lytro’s core technology, called “light field capture.”
True light field capture technology is confusing to pin down with a simple definition, but one of the things it can enable is a bit easier to visualize: realistic live action spaces that users wearing virtual reality headsets can explore. You can do some of this in today’s game engine-built VR titles, but with light field tech, the real world can be captured in 3D space as it moves, with a realism that can’t be oversold.
It’s a decidedly empowering experience that will likely soon make Lytro’s VR product, Immerge, among the most-desired tools for creatives looking to create high-end VR content.
The technology behind the Within-produced “Hallelujah” is a technical feat for the Mountain View-based Lytro, but it’s perhaps also proof that the venture-drenched startup hasn’t been all that unhinged in betting so heavily on light field tech.
Lytro was founded more than a decade ago, and has, for much of that time, been an ahead-of-its time technology in search of a problem to solve, all while burning through quite a bit of cash.
The startup focused the bulk of its early efforts on consumer-focused light field cameras that allowed users to — among other things — retroactively change the focus of their shots. The cameras were interesting and undoubtedly innovative, but were hampered by their niche audience, price and the free-falling nature of the larger consumer camera market. In 2015, the company announced it was laying off 25-50 of its 130 employees and was abandoning consumers to pivot to VR.
Fast-forward a couple of years and the company is still taking risks as it looks to find the future its technology fits into. A few months after announcing development on its VR-centric camera, the company announced Lytro Cinema, an almost comically large filmmaking camera that sports the highest-resolution video sensor ever made, capturing as much as 400 gigabytes per second.
The virtual reality rig used to shoot “Hallelujah” also comes with a massive form factor; Lytro’s VR Immerge camera is a hulking, data-hungry, hexagonal device made up of an array of 95 individual cameras. The massive size of the hardware is directly proportional to the amount of space that users can walk around. Essentially, if the camera can’t see the perspective, neither can the viewer.
Unlike most VR cameras, it isn’t a spherical device but instead captures “wedges” of an environment. The shots can later be stitched together into a spherical 10K-per-eye package.
The last I saw Lytro’s Immerge tech, they were showing off “Moon,” a fun little short on the faking of the moon landing that sought to integrate computer graphics and light field into a single project. It’s been less than a year in-between my viewings of “Moon” and Hallelujah,” and the capture quality has increased immensely, with the only limiting factor now appearing to be the display resolutions of today’s VR headsets.
Within’s “Hallelujah” is a brilliant, albeit brief, piece that gives viewers the soundtrack of the late Leonard Cohen’s formidable classic to guide them through a digital experience that brings the light field ghosts of the real world right in front of them. It’s the first of many projects, according to the company, which hopes to collaborate on 5-10 more projects this year using the tech.
It’s admittedly a bit unusual to see a company with such substantial backing pushing most of its chips in on 3D 360 video, but as headset-makers like Magic Leap (another a16z bet) and others focus their energies on consumer smartglasses powered by light field displays, startups like Lytro are able to see a potential world where its technology could be indispensable.
While the broader AR/VR headset markets have yet to fully cement their staying power, the quality of Lytro’s latest effort shows the fruits of its core light field tech and its VR pivot are starting to be realized.
Comment