If we do a 3D scan an interior room and repeat this after some time, we want to be able to detect which objects have been removed/added or simply been moved around. In many environments, it is useful to observe such changes, e.g. for inventarization of offices, warehouses or urban spaces. Furthermore, this information can be used to train deep learning algorithms for semantic understanding of such environments.
Since 3D scans can be done in real-time using Kinect or smartphones, we also want immediate results of the change detection, unlike previous work which may take seconds or even minutes. To achieve this, we will exploit a hierarchical data structure for the scanned data so that the change detection can be run in parallel. If we consider the known sensor device noise properties, we can also get robust results. Therefore, we will augment the data with the uncertainty from the 3D scanner so that we know how far the samples are allowed to diverge.
Tasks (extent depends on PR/BA/DA)
- Use a Kinect 3D scanner with the Infinitam software to scan an office room at two different times, resulting in an octree with data in its nodes
- Hand-align scans with Meshlab or register them using ICP (http://pointclouds.org/documentation/tutorials/iterative_closest_point…) so that they correspond spatially
- Modify Infinitam such that uncertainty (as an ellipsoid around the point) is stored in the octree nodes along with the scanned points
- Compare all octree nodes between the old and new model, based on the permitted distance (points + uncertainty ellipsoids), to classify into changed/unchanged geometry
- Evaluate the robustness of the algorithm by using a Blensor virtual scan of a ground truth 3D model
- Classify changes with CNN as which objects and whether added/removed/moved
- Fast parallel implementation in CUDA and compare run-time to the state-of-the-art
C++ programming skills and interest in geometry processing. Experience in geometry processing and 3D data structures such as octrees, point clouds, or CUDA will speed up the development tasks.
A bonus of €500/1,000 if completed to satisfaction within an agreed time-frame of 6/12 months (PR/BA or DA).