Authors
Gilles Rainer, Abhijeet Ghosh, Wenzel Jakob, Tim Weyrich
University College London; Imperial College London; Ecole Polytechnique Fédérale de Lausanne (EPFL)
Portals
Summary
We train a network to encode/decode BTF texels using 77 materials from the Bonn BTF database. Each texel’s appearance is projected into a shared, 32-dimensional encoding space. We show renderings for unseen material from each of the 7 classes in the database (bottom row of the database), rendered directly from the latent projection.
Abstract
Realistic rendering using discrete reflectance measurements is challenging, because arbitrary directions on the light and view hemispheres are queried at render time, incurring large memory requirements and the need for interpolation. This explains the desire for compact and continuously parametrized models akin to analytic BRDFs; however, fitting BRDF parameters to complex data such as BTF texels can prove challenging, as models tend to describe restricted function spaces that cannot encompass real-world behavior. Recent advances in this area have increasingly relied on neural representations that are trained to reproduce acquired reflectance data. The associated training process is extremely costly and must typically be repeated for each material. Inspired by autoencoders, we propose a unified network architecture that is trained on a variety of materials, and which projects reflectance measurements to a shared latent parameter space. Similarly to SVBRDF fitting, real-world materials are represented by parameter maps, and the decoder network is analog to the analytic BRDF expression (also parametrized on light and view directions for practical rendering application). With this approach, encoding and decoding materials becomes a simple matter of evaluating the network. We train and validate on BTF datasets of the University of Bonn, but there are no prerequisites on either the number of angular reflectance samples, or the sample positions. Additionally, we show that the latent space is well-behaved and can be sampled from, for applications such as mipmapping and texture synthesis.
Contribution
- In this paper, we present our new unified architecture that is trained on a wide range of BTF texels and projects them all to a common latent space, and investigate the flexibility, stability and robustness of such encoding.
Related Works
Fitting parametric models; Latent spaces of appearance; Neural encoding of appearance
Comparisons
Rainer et al, full BTF
Overview
Our architecture is conceptually an autoencoder for BTF texels, that works for any angular sampling resolution and pattern. It encodes input ABRDFs of arbitrary ordering and length n to a low-dimensional latent vector. Using an MLP that predicts weights from angles, we build a weights matrix that the expanded RGB measurements are multiplied with. Averaging across the vertical dimension allows us to recover a 3-by-m feature matrix for any input, satisfying the BTF sampling invariance criterion. Intuitively this is equivalent to discrete angular integration of the product of reflectance signal with angular filters. The remainder of our encoding architecture consists of standard fully-connected networks with ReLU activations