Image-based Reprojection Using a Non-local Means Algorithm

Information

• Publication Type: Technical Report
• Workgroup(s)/Project(s):
  • Rendering and Modeling
• Date: April 2015
• Number: TR-186-2-15-02
• Keywords: optical flow, offline rendering, image reprojection, temporal upsampling, image-based rendering

Abstract

We introduce an image-based approach to increase the framerate of image sequences generated with offline rendering algorithms. Our method handles in most cases reflections and refractions better than existing image-based temporal coherence techniques. The proposed technique is also more accurate than some image-based upsampling methods, because it calculates an individual result for each pixel.

Our proposed algorithm takes a pair of frames and generates motion vectors for each pixel. This allows for adding a new frame between that pair and thus increasing the framerate. To find the motion vectors, we utilize the non-local means denoising algorithm, which determines the similarity of two pixels by their surrounding and reinterpret that similarity as the likelihood of movement from one pixel to the other. This is similar to what it is done in video encoding to reduce file size, but in our case is done for each pixel individually instead of a block-wise approach, making our technique more accurate. Our method also improves on work in the field of real-time rendering. Such techniques use motion vectors, which are generated through knowledge about the movement of objects within the scene. This can lead to problems when the optical flow in an image sequence is not coherent with the objects movement. Our method avoids those problems. Furthermore, previous work has shown, that the non-local means algorithm can be optimized for parallel execution, which signicantly reduces the time to execute our proposed technique as well.

Additional Files and Images

Weblinks

No further information available.

BibTeX

@techreport{ROEGNER-2015-IBR,
  title =      "Image-based Reprojection Using a Non-local Means Algorithm",
  author =     "Clemens R\"{o}gner and Michael Wimmer and Johannes Hanika
               and Carsten Dachsbacher",
  year =       "2015",
  abstract =   "We introduce an image-based approach to increase the
               framerate of image sequences generated with offline
               rendering algorithms. Our method handles in most cases
               reflections and refractions better than existing image-based
               temporal coherence techniques. The proposed technique is
               also more accurate than some image-based upsampling methods,
               because it calculates an individual result for each pixel. 
               Our proposed algorithm takes a pair of frames and generates
               motion vectors for each pixel. This allows for adding a new
               frame between that pair and thus increasing the framerate.
               To find the motion vectors, we utilize the non-local means
               denoising algorithm, which determines the similarity of two
               pixels by their surrounding and reinterpret that similarity
               as the likelihood of movement from one pixel to the other.
               This is similar to what it is done in video encoding to
               reduce file size, but in our case is done for each pixel
               individually instead of a block-wise approach, making our
               technique more accurate. Our method also improves on work in
               the field of real-time rendering. Such techniques use motion
               vectors, which are generated through knowledge about the
               movement of objects within the scene. This can lead to
               problems when the optical flow in an image sequence is not
               coherent with the objects movement. Our method avoids those
               problems. Furthermore, previous work has shown, that the
               non-local means algorithm can be optimized for parallel
               execution, which signicantly reduces the time to execute our
               proposed technique as well. ",
  month =      apr,
  number =     "TR-186-2-15-02",
  address =    "Favoritenstrasse 9-11/E193-02, A-1040 Vienna, Austria",
  institution = "Institute of Computer Graphics and Algorithms, Vienna
               University of Technology ",
  note =       "human contact: technical-report@cg.tuwien.ac.at",
  keywords =   "optical flow, offline rendering, image reprojection,
               temporal upsampling, image-based rendering",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2015/ROEGNER-2015-IBR/",
}