The State of the Art in Real-time Relighting for Augmented Reality
ETH Zurich Visual Computing Talk
Abstract
With the high availability of powerful mobile devices a renewed interest in augmented reality and the plausible fusion of virtual objects into real environments emerged. This in turn has recently lead to new research seeking to adapt real-time global illumination algorithms for AR relighting.
Because augmenting objects are directly exposed to the physical surrounding, discrepancies to reality, errors in the shading model and inconsistencies in the mutual interaction of real and virtual light are easily spotted by human observers. The problem of shading virtual surfaces according to real world illumination has been attacked before with image-based lighting, but the proper fusion of both virtual and real spaces requires physically plausible illumination exchange on both ends. True immersiveness additionally requires that any solution operates on a tight per-frame budget.
This talk will introduce the relighting problem, cover the current state of the art in global illumination relighting in AR and will conclude with an analysis of the different methods currently available - from Instant Radiosity to Irradiance Caching to my own research, which seeks to solve the problem of simulating glossy reflections off real surfaces with varying degrees of roughness - and their practical application in modern AR renderers.