Lytro Immerge – The World’s First Light Field Solution for Cinematic VR

News

by Nic DivischekNovember 6^th, 2015

The idea of capturing light rays instead of a flat image is an interesting concept. This will allow users  to change things like focal range in post. Lytro have taken it a step further and developed Lytro Immerge – the world’s first light field solution for cinematic VR.

https://vimeo.com/144034085

Lytro burst onto the scene three years ago with its first light-field camera. The company’s imaging technology has always been groundbreaking. They brought still cameras onto the market that allow you to capture a shot, then in post let you decide where to focus and how to adjust the depth-of-field. The usual cameraman’s term “fix it in post” can now be applied to things that were never before possible in still photography and now cinematography. However, Lytro’s first two cameras were far from good. Its recent Lytro Illum looked like a DSLR with a hefty pricetage of $1600, but could neither capture great image quality, nor could it shoot video, despite its innovative refocusing technology.

Lytro captures Light rays instead of a flat image

A Lytro camera that shot video was inevitable. Lytro has set its sights on capturing VR video, and the company is announcing a new end-to-end system that could radically change the possibilities for VR viewers and filmmakers alike.

The Lytro Immerge system has been in development for about a year and a half. It’s the first Lytro product intended for professional video production.

In an interview between Wired and Lytro CEO Jason Rosenthal, Rosenthal says the sensors and the system were designed completely from scratch. Its centrepiece is the Immerge camera, a five-ring globe that captures what Lytro is calling a “light-field volume.” Unlike existing VR camera rigs, which capture a static 360-degree image, the Immerge camera will let you move around within the scene.

“Imagine a camera staying stationary, but being able to move your head around and getting further and closer away from an object in a scene,” Rosenthal explains. “Having the reflection and the light rays adjusting accordingly. What the light field volume represents is, we’re densely capturing all the rays in a given geometric volume, and then we’ve built software that lets us play back those rays at very high frame rates and at high resolution. It gives you the perfect recreation of the actual world you’re capturing.”

That capability alone is a mind-boggling leap for VR filmmaking, and the Immerge announcement is timed well in preparation for the next generation of VR hardware. Positional-tracking headsets such as the Oculus Rift and Sony PlayStation VR are all slated for release next year. The Lytro’s video will be compatible with all those platforms.

Lytro intends Immerge to be an end-to-end workflow system. In order to save the amounts of high-speed, high-resolution data, Lytro developed their own server. Each holds about an hour of 360-degree light-field video. It is designed to work with production tools like Nuke, Final Cut Pro, Adobe Premiere, and Avid Media Composer. There will be plug-ins for those programs and other industry-standard software that support importing and working with Immerge video.

What is exciting about the future of this technology is the ability to refocus within a scene based on a headset’s eye-tracking, instantaneous 3D-modeling, and realistically blending CG content with live video.

“We’re capturing all the depth and the 3D geometry of the real world, so compositing computer-generated objects into that with the right depth and the right shading and shadows and lighting, that all becomes much easier than it’s ever been before,” Rosenthal explains. “Imagine a video game where instead of having the weird motion-captured, CG-rendered people, you can actually have photo-realistic people. That starts to become possible.”

The Lytro will be available in the second half of 2016 for a couple hundred thousand dollars. Obviously these are numbers that are out of reach for most professionals, however rental plans are being drafted, which will allow you to have it for a mere couple of thousand dollars a day.

For more information, visit Lytro’s website.

What do you think of this new VR capturing Technology? Would you like to try and use Lytro Immerge on your set? Let us know in the comments below.