Improving Real-Time Rendering Quality and Efficiency using Variable Rate Shading on Modern Hardware

Stefan Stappen
Improving Real-Time Rendering Quality and Efficiency using Variable Rate Shading on Modern Hardware
[poster] [thesis]

Information

Abstract

With the NVIDIA Turing graphics card micro-architecture released in 2018, not only performance in terms of operations per second is increased but also new hardware features are introduced, like Variable Rate Shading (VRS). VRS allows focussing the processing power by dividing the framebuffer into tiles and dynamically controlling the resolution of each tile. To be precise, the screen is partitioned into tiles of 16x16 pixels and for each tile, it can be specified how often the fragment shader shall be executed. It is both possible, to have fewer fragment shader invocations than there are fragments, or more fragment shader invocations than there are fragments. This allows individually defining lower sampling rates or supersampling for regions of the screen. Regions of less interest or with less visual details can be assigned less computational power in terms of shader executions while regions that should provide high fidelity can be supersampled. The challenges here are to find and distinguish these regions in a dynamic scene, like it is the case for games, and how this technique integrates with commonly used techniques in the industry, like deferred shading. NVIDIA already proposed some strategies on how these regions can be distinguished and how the shading rate can be selected. Among these strategies are Content-Adaptive Shading and Motion-Adaptive Shading. Content-Adaptive Shading varies the shading rate according to the current content of a frame and does not take temporal coherence into account. Motion-Adaptive Shading adapts the shading rate according to the changes in the scene. Stable regions, like for example the horizon and the car in a driving simulation, will be rendered with higher quality. In contrast, moving regions like the street will be rendered more coarsely because the viewer cannot focus on these regions anyway. Another approach for selecting the shading rate is to adapt the resolution to the viewer’s focus. This can be done in combination with an eye-tracking device and is called foveated rendering. We invented a novel approach that utilizes data from temporal anti-aliasing techniques to detect under- and oversampled regions and select the appropriate shading rate for these regions. We developed five algorithms, edge-based and texel-differential based Content-Adaptive Shading, Motion-Adaptive Shading integrating the motion over multiple frames, single-pass foveated rendering and TAA-Adaptive Shading. The applicability of each algorithm to modern renderer architectures with forward and deferred shading and anti-aliasing post-processing has been evaluated. The major advantage of our VRS techniques is that some of them enable up to 4x higher rendering resolution with the same performance or up to 4x better performance at the same resolution.

Additional Files and Images

Additional images and videos

image
image

Additional files

poster
poster
thesis
thesis

Weblinks

No further information available.

BibTeX

@mastersthesis{stappen-2019-vrs,
  title =      "Improving Real-Time Rendering Quality and Efficiency using
               Variable Rate Shading on Modern Hardware",
  author =     "Stefan Stappen",
  year =       "2019",
  abstract =   "With the NVIDIA Turing graphics card micro-architecture
               released in 2018, not only performance in terms of
               operations per second is increased but also new hardware
               features are introduced, like Variable Rate Shading (VRS).
               VRS allows focussing the processing power by dividing the
               framebuffer into tiles and dynamically controlling the
               resolution of each tile. To be precise, the screen is
               partitioned into tiles of 16x16 pixels and for each tile, it
               can be specified how often the fragment shader shall be
               executed. It is both possible, to have fewer fragment shader
               invocations than there are fragments, or more fragment
               shader invocations than there are fragments. This allows
               individually defining lower sampling rates or supersampling
               for regions of the screen. Regions of less interest or with
               less visual details can be assigned less computational power
               in terms of shader executions while regions that should
               provide high fidelity can be supersampled. The challenges
               here are to find and distinguish these regions in a dynamic
               scene, like it is the case for games, and how this technique
               integrates with commonly used techniques in the industry,
               like deferred shading. NVIDIA already proposed some
               strategies on how these regions can be distinguished and how
               the shading rate can be selected. Among these strategies are
               Content-Adaptive Shading and Motion-Adaptive Shading.
               Content-Adaptive Shading varies the shading rate according
               to the current content of a frame and does not take temporal
               coherence into account. Motion-Adaptive Shading adapts the
               shading rate according to the changes in the scene. Stable
               regions, like for example the horizon and the car in a
               driving simulation, will be rendered with higher quality. In
               contrast, moving regions like the street will be rendered
               more coarsely because the viewer cannot focus on these
               regions anyway. Another approach for selecting the shading
               rate is to adapt the resolution to the viewer’s focus.
               This can be done in combination with an eye-tracking device
               and is called foveated rendering. We invented a novel
               approach that utilizes data from temporal anti-aliasing
               techniques to detect under- and oversampled regions and
               select the appropriate shading rate for these regions. We
               developed five algorithms, edge-based and texel-differential
               based Content-Adaptive Shading, Motion-Adaptive Shading
               integrating the motion over multiple frames, single-pass
               foveated rendering and TAA-Adaptive Shading. The
               applicability of each algorithm to modern renderer
               architectures with forward and deferred shading and
               anti-aliasing post-processing has been evaluated. The major
               advantage of our VRS techniques is that some of them enable
               up to 4x higher rendering resolution with the same
               performance or up to 4x better performance at the same
               resolution.",
  month =      dec,
  address =    "Favoritenstrasse 9-11/E193-02, A-1040 Vienna, Austria",
  school =     "Research Unit of Computer Graphics, Institute of Visual
               Computing and Human-Centered Technology, Faculty of
               Informatics, TU Wien ",
  keywords =   "Variable Rate Shading",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2019/stappen-2019-vrs/",
}