Cycling in the city risks close passes and dangerous situations with other vehicles, especially cars. Cameras, RGB-D or LIDAR sensors can measure the distance to parked and moving vehicles to project their trajectories. This permits analysis of imminent collision risks and subsequent alerting of both cyclists and drivers. [Image: Movie "Premium Rush"]
Acquire a stream of registered point clouds or camera images, similar to the KITTI benchmark suite (http://www.cvlibs.net/datasets/kitti/). Detect and track vehicles, and compute their speed and trajectory. Classify behaviors of vehicle drivers such as passing, turning, and crossing. Train a neural network based on environmental information, e.g. street boundaries and intersection locations, in order to predict hazardous situations.
Tasks (depending on PR/BA/DA and number of students)
Note that due to the large number of tasks, this can also be done as a group project, or else work on a subset!
- Construct a battery-powered sensor package (Kinect or LIDAR, e.g., Velodyne H800,+RGB camera, GPS) mounted at bicycle handlebars [taken]
- Record depth image stream together with color video (for ground truth) onto storage media or smartphone [taken]
- Register the recorded point clouds/camera images with GPS and IMU or smartphone acceleration/gyro sensors
- Based on measured vehicle poses, compute speed, trajectories, and project potential future paths [taken]
- Detect and track vehicles on a smartphone (light-weight)
- Reconstruct the surface of vehicles, compute the distance to vehicles, and fit curves to tracked paths to predict trajectories on a smartphone (light-weight)
- Classify vehicle driver behavior based on relative trajectories and verify with ground truth [taken]
- Train a neural network with trajectories and point clouds to detect potential collisions [taken]
- Record localized statistics on collision distances, multi-vehicle/street layout configurations
- use OpenStreetMap data and point cloud information to localize the bicycle and vehicles more precisely
- Knowledge of English language (source code comments and final report have to be in English)
- Knowledge of C++, python, or deep learning is advantageous, but not necessary
- Ability to ride a bicycle
The project should be implemented platform independent (Linux, Windows).
A bonus of €500/€1000 if completed to satisfaction within an agreed time-frame of 6/12 months (PR/BA or DA)