Motivation
Cycling in the city risks close passes and dangerous situations with other vehicles, especially cars. Cameras, RGB-D or LIDAR sensors can measure the distance to parked and moving vehicles to project their trajectories. This permits analysis of imminent collision risks and subsequent alerting of both cyclists and drivers. [Image: Movie "Premium Rush"]
Description
Acquire a stream of registered point clouds or camera images, similar to the KITTI benchmark suite (http://www.cvlibs.net/datasets/kitti/). Detect and track vehicles, and compute their speed and trajectory. Classify behaviors of vehicle drivers such as passing, turning, and crossing. Train a neural network based on environmental information, e.g. street boundaries and intersection locations, in order to predict hazardous situations.
Tasks (depending on PR/BA/DA and number of students)
Note that due to the large number of tasks, this can also be done as a group project, or else work on a subset!
- Construct a battery-powered sensor package (Kinect or LIDAR, e.g., Velodyne H800,+RGB camera, GPS) mounted at bicycle handlebars
- Record depth image stream together with color video (for ground truth) onto storage media or smartphone
- Convert RGB from cameras to depth images using depth estimation with machine learning, e.g., LiteDepth https://arxiv.org/abs/2209.00961
- Register the recorded point clouds/camera images with GPS and IMU or smartphone acceleration/gyro sensors
- Detect and track vehicles with existing algorithms such as YOLOv4 and DeepSORT (https://github.com/LeonLok/Deep-SORT-YOLOv4)
- Based on measured vehicle poses, compute speed, trajectories, and project potential future paths
- Detect and track vehicles on a smartphone (light-weight)
- Reconstruct the surface of vehicles, compute the distance to vehicles, and fit curves to tracked paths to predict trajectories on a smartphone (light-weight)
- Classify vehicle driver behavior based on relative trajectories and verify with ground truth
- Train a neural network with trajectories and point clouds to detect potential collisions
- Record localized statistics on collision distances, multi-vehicle/street layout configurations
- use OpenStreetMap data and point cloud information to localize the bicycle and vehicles more precisely
- train a neural network with features (traffic light states, participant trajectories, street markings) to recognize/collect situations which are both violating traffic laws AND dangerous to others
Requirements
- Knowledge of English language (source code comments and final report have to be in English)
- Knowledge of C++, python, or deep learning is advantageous, but not necessary
- Ability to ride a bicycle
Environment
The project should be implemented platform independent (Linux, Windows).
A bonus of €500/€1000 if completed to satisfaction within an agreed time-frame of 6/12 months (PR/BA or DA)
Responsible
For more information please contact Stefan Ohrhallinger.