We are looking for students to join our team exploring the usage of neural networks on anime imagery. Students will work with our in-house anime dataset, featuring over 90 million images from about 40.000 different shows/movies, and are expected to develop new image processing and/or deep learning methods targeted for it.
We are exploring what tasks can and cannot be done using deep learning techniques on production ready animation frames. Long term, our goal is to develop tools that could be used by consumers and studios alike. For example:
- Search and classification tools, such as being able to estimate genre, studio, art direction or even series from images.
- Drawing tools, such as automatically generating character features (ex: eyes, hair) from rough drafts.
Topics and Tasks
Multiple possible topics are currently open for applicants:
- Metadata Estimation from Single Image: The student will explore classifying series, genre, studio, etc from a single user inputted image.
- Frame Relevancy Estimation: Not all frames from a show are necessarily as characteristic as others (ex: black screen). The student will explore classifying the importance of images in the dataset, allowing for the most characteristic images from a show to be identified, and for irrelevant ones to be ignored in other learning algorithms.
- Optimized Input: Due to the nature of 2D animation, using the raw production images might not be the optimal input for learning. The student will explore alternative input representations, such as contour lines, luminance and hue-shift.
- Feature Extraction: Before we can develop any drawing tools, we need to be able to extract specific features from images in the dataset. The student will explore developing networks capable of identifying such features.
- Feature Translation: Character and scene features are often drawn with different qualities depending on factors such as being in focus or distance. The student will explore translating drawings between these two levels of quality.
- In-Between Generation: In betweens are the frames drawn as an interpolaion of the key frames. While the latter are generally drawn by more senior people and drive the art of the scene, in betweens are more tedious work (and poorly paid) normally done by more junior artists. The student will explore using networks to draw in between given pairs of keyframes.
- Bring your Own: If you have an idea of your own related to image deep learning you'd like to pursue, feel free to contact me with it.
Topic will be chosen after discussion with the applicant according to his/her qualifications and interests. The exact tasks will depend on the topic, but generally all involve working on Image Processing, Deep Learning and with our dataset.
- Fluent in English, spoken and written (supervisor speaks English, code and reports must be written in English).
- Familiar with Machine Learning and/or Image Processing (eg: taken a course in the subject) or proved ability to learn new concepts on their own.
- Proficiency with Python.
- Basic understanding of the media domain in question.
- Available to meet bi-weekly with group.
- Previous Machine Learning work.