Motivation
We want to classify objects in scenes that contain many similar objects (e.g., in a factory hall, warehouse, or office). However, especially for non-standard objects, training these specific classes requires a lot of effort to create enough ground truth. Instead, we aim at classifying yet unseen objects automatically and label them just with weak supervision, i.e., a human-in-the-loop being queried for unknown classes for training on-the-fly.
Description
Unsupervised object detection [1] can also be done using open vocabulary object detection [2], or zero-shot learning [3]. Objects can be divided into classes by k-means clustering, see this BA for a simple 2-means image classification [4]. Here is a survey in weakly supervised object detection [5].
[2] https://arxiv.org/abs/2207.
[3] https://arxiv.org/abs/2104.
[4] https://www.cg.tuwien.ac.at/
[5] https://arxiv.org/pdf/2104.
Tasks (extent depends on PR/BA/DA)
-
Segment objects: apply a suitable ML algorithm
-
Cluster similar objects: apply a suiting ML algorithm
-
Label objects in clusters with human-in-the-loop and adjust clusters
Requirements
Python and C++ programming skills and interest in geometry processing and machine learning.
Environment
Platform-independent.
A bonus of €500/1,000 if completed to satisfaction within an agreed time frame of 6/12 months (PR/BA or DA).