AI Is Learning How to Perfect Lighting in Photos
Authors:
(1) Hoon Kim, Beeble AI, and contributed equally to this work;
(2) Minje Jang, Beeble AI, and contributed equally to this work;
(3) Wonjun Yoon, Beeble AI, and contributed equally to this work;
(4) Jisoo Lee, Beeble AI, and contributed equally to this work;
(5) Donghyun Na, Beeble AI, and contributed equally to this work;
(6) Sanghyun Woo, New York University, and contributed equally to this work.
Editor’s Note: This is Part 5 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below.
Table of Links
Appendix
3.3. Architecture
Illum Net. The network infers the lighting conditions in the given image captured in an HDRI format. Specifically, it computes the convolved HDRIs:
The network employs a cross-attention mechanism at its core, where predefined Phong reflectance lobes serve as queries, and the original image acts as both keys and values. Within this setup, the convolved HDRI maps are synthesized by integrating image information into the Phong reflectance lobe representation. Specifically, our model utilizes bottleneck features from the Normal Net as a compact image representation. Our approach simplifies the complex task of HDRI reconstruction by instead focusing on estimating interactions with known surface reflective properties.
We have empirically validated that it significantly enhances albedo prediction across a range of real-world scenarios.
Specular Net. The network infers surface attributes associated with the Cook-Torrance specular elements, specifically, the roughness α and Fresnel reflectivity f0. It uses a source image, predicted normal, and albedo maps as inputs.
