Visual Decoding – Deriving Visually Encoded Information from Static Images

Type: 
BA/PR/DA
Persons: 
1

Description

While a static image of a visualization or a plot captures and conveys information about the underlying data, the actual data is often not accessible in any other form – the data is literally captured in the image. This makes tasks, such as comparing the underlying data across two plots cumbersome and inaccurate.

The goal of this project is to create a semi-automatic tool that decodes the visually encoded information within a plot, thus re-capturing the underlying raw data – and enabling various operations on it, e.g., filtering, aggregating, comparing. The tool should be able to identify the plotted visual primitives, e.g., points, circles, bars, the axes and axis values of a plot, as well as the values encoded within the visual primitives, e.g., size, position, color. Derived attributes from each visual primitive should then be stored and processed, e.g., by re-encoding them within a new plot alongside additional data (that could have been derived from another image).

The motivation behind this project comes from a use case in the context of environmental history, where information is collected from a broad range of sources where the underlying data is often unavailable in a (re)usable form. Thus the manual derivation of the encoded information is required – a laborious and error prone task. The goal of this project is to reduce the amount of manual work as well as the risk of errors that is involved in manual data collection/annotation.

Tasks

Your task will be the implementation of the described features within a browser-based tool. This includes but is not limited to implementing support for: 

  • (user assisted) feature detection on images for different types of visual primitives (see above)
  • dynamic creation of web based visualizations based on the decoded data
  • export of the data into an expanding data base of decoded information

The complexity of these tasks can be scaled up or down, depending on whether the implementation is carried out within the scope of a bachelor or master thesis.

Requirements

  • Knowledge of the English language (the final report should be in English)
  • Interest in (web based) data visualization
  • Programming skills (optimally in javascript)
  • Basic knowledge in image processing (feature detection)

Environment

This project will be realized in a collaboration between the Institute of Visual Computing &
Human-Centered Technology (formerly Institute of Computer Graphics and Algorithms) at the TU Wien and the Complexity Science Hub (CSH) Vienna.

You will be mentored during regular meetings at the CSH. The duration of the implementation and writeup of the thesis depends on the prior knowledge and commitment of the student, but the estimated duration is one semester for a bachelor thesis and two semesters for a master thesis.

Please contact sorger@csh.ac.at if you are interested in joining this project.

You can find more information on the Complexity Science Hub on the official webpage:
https://www.csh.ac.at