Photorealistic Material Editing Through Direct Image Manipulation

Information

  • Publication Type: Journal Paper with Conference Talk
  • Workgroup(s)/Project(s):
  • Date: June 2020
  • Call for Papers: Call for Paper
  • Date (from): 29. June 2020
  • Date (to): 3. July 2020
  • Event: Eurographics Symposium on Rendering 2020
  • ISSN: 1467-8659
  • Journal: Computer Graphics Forum
  • Lecturer: Károly Zsolnai-Fehér
  • Location: London, UK
  • Number: 4
  • Open Access: yes
  • Pages (from): to appear
  • Volume: 39
  • Keywords: neural rendering, neural networks, photorealistic rendering, photorealistic material editing

Abstract

Creating photorealistic materials for light transport algorithms requires carefully fine-tuning a set of material properties to achieve a desired artistic effect. This is typically a lengthy process that involves a trained artist with specialized knowledge. In this work, we present a technique that aims to empower novice and intermediate-level users to synthesize high-quality photorealistic materials by only requiring basic image processing knowledge. In the proposed workflow, the user starts with an input image and applies a few intuitive transforms (e.g., colorization, image inpainting) within a 2D image editor of their choice, and in the next step, our technique produces a photorealistic result that approximates this target image. Our method combines the advantages of a neural network-augmented optimizer and an encoder neural network to produce high-quality output results within 30 seconds. We also demonstrate that it is resilient against poorly-edited target images and propose a simple extension to predict image sequences with a strict time budget of 1-2 seconds per image.

Video: https://www.youtube.com/watch?v=8eNHEaxsj18

Additional Files and Images

Additional images and videos

Additional files

Weblinks

BibTeX

@article{zsolnaifeher-2020-pme,
  title =      "Photorealistic Material Editing Through Direct Image
               Manipulation",
  author =     "K\'{a}roly Zsolnai-Feh\'{e}r and Peter Wonka and Michael
               Wimmer",
  year =       "2020",
  abstract =   "Creating photorealistic materials for light transport
               algorithms requires carefully fine-tuning a set of material
               properties to achieve a desired artistic effect. This is
               typically a lengthy process that involves a trained artist
               with specialized knowledge. In this work, we present a
               technique that aims to empower novice and intermediate-level
               users to synthesize high-quality photorealistic materials by
               only requiring basic image processing knowledge. In the
               proposed workflow, the user starts with an input image and
               applies a few intuitive transforms (e.g., colorization,
               image inpainting) within a 2D image editor of their choice,
               and in the next step, our technique produces a
               photorealistic result that approximates this target image.
               Our method combines the advantages of a neural
               network-augmented optimizer and an encoder neural network to
               produce high-quality output results within 30 seconds. We
               also demonstrate that it is resilient against poorly-edited
               target images and propose a simple extension to predict
               image sequences with a strict time budget of 1-2 seconds per
               image.  Video: https://www.youtube.com/watch?v=8eNHEaxsj18",
  month =      jun,
  issn =       "1467-8659",
  journal =    "Computer Graphics Forum",
  number =     "4",
  volume =     "39",
  pages =      "to appear--",
  keywords =   "neural rendering, neural networks, photorealistic rendering,
               photorealistic material editing",
  URL =        "/research/publications/2020/zsolnaifeher-2020-pme/",
}