Quantifying the Error of Light Transport Algorithms

Adam Celarek, Wenzel Jakob, Michael Wimmer, Jaakko Lehtinen
Quantifying the Error of Light Transport Algorithms
Computer Graphics Forum, 38(4):111-121, July 2019. [paper_preprint] [Git repository]

Information

Abstract

This paper proposes a new methodology for measuring the error of unbiased physically based rendering algorithms. The current state of the art includes mean squared error (MSE) based metrics and visual comparisons of equal-time renderings of competing algorithms. Neither is satisfying as MSE does not describe behavior and can exhibit significant variance, and visual comparisons are inherently subjective. Our contribution is two-fold: First, we propose to compute many short renderings instead of a single long run and use the short renderings to estimate MSE expectation and variance as well as per-pixel standard deviation. An algorithm that achieves good results in most runs, but with occasional outliers is essentially unreliable, which we wish to quantify numerically. We use per-pixel standard deviation to identify problematic lighting effects of rendering algorithms. The second contribution is the error spectrum ensemble (ESE), a tool for measuring the distribution of error over frequencies. The ESE serves two purposes: It reveals correlation between pixels and can be used to detect outliers, which offset the amount of error substantially.

Additional Files and Images

Additional images and videos

image: Error spectrum ensemble image: Error spectrum ensemble
teaser: Teaser teaser: Teaser

Additional files

paper_preprint: Paper paper_preprint: Paper
presentation: Slides from the presentation at EGSR presentation: Slides from the presentation at EGSR
supplemental_material: Additional examples, experiments and similar. supplemental_material: Additional examples, experiments and similar.

Weblinks

BibTeX

@article{celarek_adam-2019-qelta,
  title =      "Quantifying the Error of Light Transport Algorithms",
  author =     "Adam Celarek and Wenzel Jakob and Michael Wimmer and Jaakko
               Lehtinen",
  year =       "2019",
  abstract =   "This paper proposes a new methodology for measuring the
               error of unbiased physically based rendering algorithms. The
               current state of the art includes mean squared error (MSE)
               based metrics and visual comparisons of equal-time
               renderings of competing algorithms. Neither is satisfying as
               MSE does not describe behavior and can exhibit significant
               variance, and visual comparisons are inherently subjective.
               Our contribution is two-fold: First, we propose to compute
               many short renderings instead of a single long run and use
               the short renderings to estimate MSE expectation and
               variance as well as per-pixel standard deviation. An
               algorithm that achieves good results in most runs, but with
               occasional outliers is essentially unreliable, which we wish
               to quantify numerically. We use per-pixel standard deviation
               to identify problematic lighting effects of rendering
               algorithms. The second contribution is the error spectrum
               ensemble (ESE), a tool for measuring the distribution of
               error over frequencies. The ESE serves two purposes: It
               reveals correlation between pixels and can be used to detect
               outliers, which offset the amount of error substantially.",
  month =      jul,
  doi =        "10.1111/cgf.13775",
  journal =    "Computer Graphics Forum",
  number =     "4",
  volume =     "38",
  publisher =  "The Eurographics Association and John Wiley & Sons Ltd.",
  pages =      "111--121",
  keywords =   "measuring error, light transport, global illumination",
  URL =        "/research/publications/2019/celarek_adam-2019-qelta/",
}