Single Image Based Automatic Portrait Photography Relighting System

Open Access
Author:
Wang, Kun
Area of Honors:
Computer Engineering
Degree:
Bachelor of Science
Document Type:
Thesis
Thesis Supervisors:
  • James Z. Wang, Thesis Supervisor
  • John Morgan Sampson, Honors Advisor
Keywords:
  • 3D face reconstruction
  • Face Relighting
  • Albedo Estimation
Abstract:
This thesis research proposes a novel intelligent face relighting framework to help amateur photographers take high-quality portraits. Face relighting is a challenging problem when there is only a single image of the face available. To solve this problem, we often assume that human faces obey Lambert’s law. Recovering the shape, reflectance and illuminance of a face is essential to relight the face. The proposed framework can decompose a single face image into a normal map (shape), an albedo map (reflectance), and lighting coefficients (illuminance). This framework incorporates the state-of-the-art face landmark detection algorithm, the 3D morphable face model fitting framework, and the spherical-harmonics-base face albedo estimation algorithm to relight a face from a single image. Our proposed framework first detects the face landmarks in a portrait. Then, it fits a 3D morphable face model to the face with detected landmark positions. Next, the framework calculates the vertex normal vectors of the reconstructed 3D model and renders the normal map. In order to obtain the albedo map and illumination coefficients, it solves a linear equation system iteratively. Finally, with normal map and albedo map, the framework allows the user to apply different portrait lighting styles to the input image by changing the illumination coefficients, which are learned from artistic portrait photos. When the input face is under a general smooth lighting, the experiment results are realistic and reliable under the subjective evaluation.