Exploiting Light Polarization for Deep HDR Imaging from a Single Exposure

Information

• Publication Type: Journal Paper (without talk)
• Workgroup(s)/Project(s):
  • Rendering and Modeling
• Date: June 2023
• Article Number: 5370
• DOI: 10.3390/s23125370
• ISSN: 1424-8220
• Journal: Sensors
• Number: 12
• Open Access: yes
• Pages: 23
• Volume: 23
• Publisher: MDPI
• Keywords: PFA camera, deep learning, high dynamic range imaging, polarimetric imaging

Abstract

In computational photography, high dynamic range (HDR) imaging refers to the family of techniques used to recover a wider range of intensity values compared to the limited range provided by standard sensors. Classical techniques consist of acquiring a scene-varying exposure to compensate for saturated and underexposed regions, followed by a non-linear compression of intensity values called tone mapping. Recently, there has been a growing interest in estimating HDR images from a single exposure. Some methods exploit data-driven models trained to estimate values outside the camera’s visible intensity levels. Others make use of polarimetric cameras to reconstruct HDR information without exposure bracketing. In this paper, we present a novel HDR reconstruction method that employs a single PFA (polarimetric filter array) camera with an additional external polarizer to increase the scene’s dynamic range across the acquired channels and to mimic different exposures. Our contribution consists of a pipeline that effectively combines standard HDR algorithms based on bracketing and data-driven solutions designed to work with polarimetric images. In this regard, we present a novel CNN (convolutional neural network) model that exploits the underlying mosaiced pattern of the PFA in combination with the external polarizer to estimate the original scene properties, and a second model designed to further improve the final tone mapping step. The combination of such techniques enables us to take advantage of the light attenuation given by the filters while producing an accurate reconstruction. We present an extensive experimental section in which we validate the proposed method on both synthetic and real-world datasets specifically acquired for the task. Quantitative and qualitative results show the effectiveness of the approach when compared to state-of-the-art methods. In particular, our technique exhibits a PSNR (peak signal-to-noise ratio) on the whole test set equal to 23 dB, which is 18% better with respect to the second-best alternative.

Additional Files and Images

Weblinks

• Entry in reposiTUm (TU Wien Publication Database)
• DOI: 10.3390/s23125370

BibTeX

@article{pistellato-2023-elp,
  title =      "Exploiting Light Polarization for Deep HDR Imaging from a
               Single Exposure",
  author =     "Mara Pistellato and Tehreem Fatima and Michael Wimmer",
  year =       "2023",
  abstract =   "In computational photography, high dynamic range (HDR)
               imaging refers to the family of techniques used to recover a
               wider range of intensity values compared to the limited
               range provided by standard sensors. Classical techniques
               consist of acquiring a scene-varying exposure to compensate
               for saturated and underexposed regions, followed by a
               non-linear compression of intensity values called tone
               mapping. Recently, there has been a growing interest in
               estimating HDR images from a single exposure. Some methods
               exploit data-driven models trained to estimate values
               outside the camera’s visible intensity levels. Others make
               use of polarimetric cameras to reconstruct HDR information
               without exposure bracketing. In this paper, we present a
               novel HDR reconstruction method that employs a single PFA
               (polarimetric filter array) camera with an additional
               external polarizer to increase the scene’s dynamic range
               across the acquired channels and to mimic different
               exposures. Our contribution consists of a pipeline that
               effectively combines standard HDR algorithms based on
               bracketing and data-driven solutions designed to work with
               polarimetric images. In this regard, we present a novel CNN
               (convolutional neural network) model that exploits the
               underlying mosaiced pattern of the PFA in combination with
               the external polarizer to estimate the original scene
               properties, and a second model designed to further improve
               the final tone mapping step. The combination of such
               techniques enables us to take advantage of the light
               attenuation given by the filters while producing an accurate
               reconstruction. We present an extensive experimental section
               in which we validate the proposed method on both synthetic
               and real-world datasets specifically acquired for the task.
               Quantitative and qualitative results show the effectiveness
               of the approach when compared to state-of-the-art methods.
               In particular, our technique exhibits a PSNR (peak
               signal-to-noise ratio) on the whole test set equal to 23 dB,
               which is 18% better with respect to the second-best
               alternative.",
  month =      jun,
  articleno =  "5370",
  doi =        "10.3390/s23125370",
  issn =       "1424-8220",
  journal =    "Sensors",
  number =     "12",
  pages =      "23",
  volume =     "23",
  publisher =  "MDPI",
  keywords =   "PFA camera, deep learning, high dynamic range imaging,
               polarimetric imaging",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2023/pistellato-2023-elp/",
}