Introduction

We implemented a visualization tool for rendering and analyzing video sequences based on [1]. The authors propose to „employ volume visualisation techniques for "summarising" video sequences and to render video volumes into appropriate visual representations.“ The application of various statistical difference metrics show activity and therefore changes in the data. Transfer functions are applied to the difference volume to highlight these changes. Thus meaningful information is extracted from the video, which can be used to support decision making processes.

The authors of [1] use a slice-based approach for rendering the volume. We chose to go with a modern raycast-based approach implemented via OpenGL shaders. [1] implement different volume shapes in order to reduce to problem of occlusions. We render the volume as a fixed cuboid and allow the user to rotate and zoom the volume. Furthermore, we allow the user to ,browse' through the volume, only rendering local areas (employing a focus+context approach). We belive this is superior to the original implementation.

Implementation

We implemented our project in C# with Visual Studio 2010 Professional. For rendering the volume we used GPU shaders and the OpenGL wrapper OpenTK. Our project consists auf two parts:

• HighGL: This library wraps OpenTK functionality to simplify the handling of it. It enables us to write several shaders, which are linked together at compile time and pre-compiles them in order to check for syntax errors, unavailable uniforms, etc.
• VideoDVR: A windows forms application that uses the HighGL-Classes for rendering the volume, difference metrics and transfer functions.

We chose to implement as much as possible ourselves, in order to learn modern OpenGL. The only library we use is OpenTK, which makes OpenGL functions available from C#. In order to run the program, you have to download and install OpenTK on your computer first. Then download the program and extract it to a new folder. Run VideoDVR.exe.

Manual

As of the current version, our application is able to read .jpg and .png image sequences, that represent frames of a video. To generate a new visualization, click the Open directory button or hit the key O.
Note: See all available shortcuts in the About window (heart button).

After choosing the data source, you specify settings for the image sequence like the downsampling factor or the frame rate (i.e. every n-th frame, which should be processed).

The rendered video sequence is displayed in the main window. You can rotate the volume with the mouse and zoom in and out with the mouse wheel. The Slice Selector window enables the user to browse through the volume by selecting a slice with the slider. By changing the Context, you can define, how many percent of the volume are displayed in both directions of the selected slice.

By clicking the Display Video Volume button you can switch between two different transparency modes. In Constant Transparency mode, the volume is displayed with alpha = 0.5. When selecting Hue-Based Transparency, you can specify a hue-based transfer function in a seperate window. By changing the Hue and Range sliders, the range of the rendered hue is defined by: [hue - range/2, hue + range/2]. The maximum value for alpha in the rendering can be changed as well.

Figure 1: Shows the application of hue-based transparency to a volume.

Difference metrics

In order to show changes/activity in the video, you can apply difference metrics on the volume data. You can choose between the difference to a reference frame or frame difference in the video. For both modes exist three different metrics:

• Y-DIF: This simple difference metric calculates the linear distance between the Y-values of two corresponding pixels (in two images).
• Y-MSE: Instead of computing the linear distance, the squared distance is calculated.
• UV-DIF: This colour difference metric calculates the angle between the UV vectors of two corresponding pixels (in two images). The corresponding pixel in the output is set to the angle.

Transfer Functions

In order to visualize the changes calculated with the difference metrics, you can define transfer functions for the rendering of a difference volume. In the Transfer Function window you can specify a new function by clicking into the grid. You specify a new color and alpha value for the input alpha values. The x-axis represents the input alpha values and the y-axis the new values. The resulting transfer function is shown in the right panel and rendered immediately.

Figure 2: Shows the application of a transfer function to a frame difference volume with Y-DIF metric.

Graphs

When rendering difference metrics, the user can obtain detailed information about the intensities in the difference volume. By clicking the Show Graph button, a graph with the mean pixel intensities per slice is displayed. If you find a region of interest, you can choose a slice by clicking into the graph. The Slice Selector automatically browses to the selected slice in the volume. The graphs also act as supporting tool for the definition of transfer functions.