HomeTechnologyThis room-sized LED egg captures amazing 3D models of the people inside...

This room-sized LED egg captures amazing 3D models of the people inside it

Capturing human performances in high-definition 3D is a complicated proposition, and among the many challenges is getting the lighting right. This impressive new project from Google researchers puts the subject in the center of what can only be described as an prismatic LED egg, but the resulting 3D models are remarkable — and more importantly, relightable.

What’s called volumetric capture uses multiple cameras in a 360-degree setup to capture what can look like a photorealistic representation of a subject, including all the little details like clothing deformation, hair movement, and so on. It has two serious weaknesses: First, it’s more like a 3D movie than a model, since you can’t pose the person or change their attributes or clothing; The second is an extension of the first, in that you can’t change the way the person is lit — whatever lighting they had when you captured them, that’s what you get.

“The Relightables” is an attempt by a team at Google AI to address this second issue, since the first is pretty much baked in. Their system not only produces a highly detailed 3D model of a person in motion, but allows that model to be lit realistically by virtual light sources, making it possible to place it in games, movies, and other situations where lighting can change.

Images from the Google AI paper that show the capture process and resulting 3D model alone and in a lighted virtual environment.

It’s all thanks to the aforementioned prismatic egg (and a couple lines of code, of course). The egg is lined with 331 LED lights that can produce any color, and as the person is being captured, those LEDs shift in a special structured pattern that produces a lighting-agnostic model.

The resulting models can be placed in any virtual environment and will reflect not the lighting they were captured in but the lighting of that little world. The examples in the video below aren’t exactly Hollywood-level quality, but you can see the general idea of what they’re going for.

The limitations of volumetric capture make it unsuitable for many uses in film, but being relightable brings these performances a lot closer to ordinary 3D models than they were before. Of course, you still have to do all your acting inside a giant egg.

“The Relightables” will be presented by the team at SIGGRAPH Asia.

(techcrunch)

Colin Ingram
Colin Ingram
Colin Ingram, a graduate in journalism from Tshwane University of Technology, is presently working as a senior reporter with NYK Daily. He loves his gadgets and his cat. (although the cat, being a cat, doesn’t let his gadgets stay in one piece within a few blinks.) He also loves trekking and the wildlife.
RELATED ARTICLES

Was it worth reading? Let us know.

Most Popular

Recent Comments

Sreerama. t.v,70+ years old,ex manager BEL-BENGALURU-560013/24-11-2021 on Celebrating The Most Learned Man on Earth: Adi Shankaracharya
Sid Martonn on YOUTH AGAINST RAPE 
Nikhil Chandwani on Why is Navratri Celebrated?
Utkarsh Shukla on Ghanta Engineering!