A common way to explain what an AI model has learned is to feed it with a lot of input samples (e.g., images) and to visualize how similar these samples are with respect to the features the model has learned. In the best case, these similarities reflect what the user considers to be similar. A simple example is Tensorflow’s embedding projector: similar images are tightly grouped together, and the color illustrates what the users consider to be similar (i.e., the ground truth labels). This works well if the number of samples is low and, especially, if the number of ground truth classes is so low that each class can be assigned a distinct color. However, more than 12 colors are hard to discriminate for humans. In addition, the similarity visualization requires a dimensionality reduction from the high-dimensional feature space to 2D, which can lead to considerable projection errors - and in the worst case to incorrect conclusions.
This work should combine topological analysis of the high-dimensional space with hierarchical aggregation of the class labels to compute an “agreement graph” between human and machine. Through the hierarchical aggregation, a visualization of this graph could give a visual overview of agreements and disagreements between user and machine for much more complex data sets, like ImageNet with 1,000 classes. In the evaluation step, the student shall investigate if the visualized disagreements indeed are indicative of classification errors.
The preferred target platform is the web using visualization platforms like d3.js or three.js with a Python backend. Code to organize class labels hierarchically and to organize the high-dimensional space is available. The implementation should target at pre-trained CNN models, such as Inception trained on ImageNet, with large-scale image validation datasets.