Anime Deep Learning

Type: 
DA
Persons: 
1-3
Workgroup: 

Description

We are looking for students to join our team exploring the usage of neural networks on anime imagery. Students will work with our in-house anime dataset, featuring over 60 million images from about 40.000 different shows/movies, and are expected to develop new image processing and/or deep learning methods targeted for it.

We are exploring what tasks can and cannot be done using deep learning techniques on production ready animation frames. Long term, our goal is to develop tools that could be used by consumers and studios alike. For example:

  • Search and classification tools, such as being able to estimate genre, studio, art direction or even series from images.
  • Drawing tools, such as automatically generating character features (ex: eyes, hair) from rough drafts.

Topics and Tasks

Multiple possible topics are currently open for applicants:

  • Metadata Estimation from Single Image: The student will explore classifying series, genre, studio, etc from a single user inputted image. taken
  • Frame Relevancy Estimation: Not all frames from a show are necessarily as characteristic as others (ex: black screen). The student will explore classifying the importance of images in the dataset, allowing for the most characteristic images from a show to be identified, and for irrelevant ones to be ignored in other learning algorithms. taken
  • Contour Reconstruction: Due to the nature of 2D animation, using the raw production images might not be the optimal input for learning. The student will explore alternative input representations, such as identifying the most important contour lines in a given frame. taken
  • Feature Extraction: Before we can develop any drawing tools, we need to be able to extract specific features from images in the dataset. The student will explore developing networks capable of identifying such features.

Topic will be chosen after discussion with the applicant according to his/her qualifications and interests. The exact tasks will depend on the topic, but generally all involve working on Image Processing, Deep Learning and with our dataset.
 

Requirements

  • Fluent in English, spoken and written (supervisor speaks English, code and reports must be written in English).
  • Familiar with Machine Learning and/or Image Processing (eg: taken a course in the subject).
  • Proficiency with Python.
  • Basic understanding of the media domain in question.
  • Available to meet weekly Mondays at 2:30pm.

Preferred

  • Proficiency in either Matlab or Javascript.

Contact

For more information please contact or Joao Afonso Cardoso (jaliborc@cg.tuwien.ac.at).