Authors
Zhe Chen, Shohei Nobuhara, Ko Nishino
Kyoto University
Portals
Abstract
We introduce a novel neural network-based BRDF model and a Bayesian framework for object inverse rendering, i.e., joint estimation of reflectance and natural illumination from a single image of an object of known geometry. The BRDF is expressed with an invertible neural network, namely, normalizing flow, which provides the expressive power of a high-dimensional representation, computational simplicity of a compact analytical model, and physical plausibility of a real-world BRDF. We extract the latent space of real-world reflectance by conditioning this model, which directly results in a strong reflectance prior. We refer to this model as the invertible neural BRDF model (iBRDF). We also devise a deep illumination prior by leveraging the structural bias of deep neural networks. By integrating this novel BRDF model and reflectance and illumination priors in a MAP estimation formulation, we show that this joint estimation can be computed efficiently with stochastic gradient descent. We experimentally validate the accuracy of the invertible neural BRDF model on a large number of measured data and demonstrate its use in object inverse rendering on a number of synthetic and real images. The results show new ways in which deep neural networks can help solve challenging radiometric inverse problems.
Contribution
- a novel BRDF model with the expressiveness of a nonparametric representation and computational simplicity of an analytical distribution model
- a reflectance prior based on the embedding of this novel BRDF model
- an illumination prior leveraging architectural bias of neural networks
- and a fully differentiable joint estimation framework for reflectance and illumination based on these novel models and priors
Related Works
Reflectance Models; Reflectance Estimation; Illumination Estimation; Joint Estimation of Reflectance and Illumination