5LSH0 (2018) - Computer Vision and 3D Image Processing

Goal:
Learn the main components of the video content analysis, such as feature detection and description, object segmentation, object detection and tracking. Learn advanced object classification techniques based on the Deep Learning concept. Learn the basics of 3D multi-view geometry, 3D sensing principles and 3D model reconstruction. Learn the practical aspects of implementing the above described methods, by programming (C++, python, TensorFlow) a surveillance application, using a UAV drone as a data capturing device.

Content:
Content of the course is divided globally into two areas.

  1. Techniques for object detection and recognition, feature extraction and analysis, like SIFT and HOG. Furthermore, semantic level processing for understanding events and scenes, including human behavior. Furthermore, classification techniques for understanding objects and events. Modern classification like K-means and SVM (support vector machine) algorithms, evolving into basics of learning with neural nets. This part will gradually evolve to fundamentals and practical applications of Deep Learning.
  2. 3D processing based on the camera pinhole model, multi-view processing and calibration. Also registration of 3D datasets, 3D reconstruction models with TSDF, introduction to SLAM, RGB-Depth processing and specific algorithms like G2.0 and bundle adjustment. Finally, the 3D processing modules end with plane/object segmentation in 3D.

The programming assignments aim at applying the knowledge and algorithms (or parts of them) to provide the student a framework for experiments with video content understanding and 3D image-based modeling for surveillance applications.
The assignments are based on C++ / Python / TensorFlow programming.

Preknowledge
Advised: 5LSE0 Multimedia video coding and architectures

Schedule and location: 
All lectures in a single week at the end of August, about 40% is dedicated to computer exercises.
Full schedule: 
Date Time Room
27 aug 09.30-17.30 Flux 1.07
28 aug 09.30-17.30 Flux 1.07
29 aug 19.30-17.30 Flux 1.07
30 aug 09.30-17.30 Flux 1.07
31 aug 09.30-17.30 Flux 1.07
Slides: 
Module 00: Introduction 5LSH0
Module 01: Feature extraction and matching
Module 02: Segmentation
Module 03: Camera model
Module 04: 3D reconstruction, data fusion, SLAM
Module 05: 3D structures from motion
Module 06: Classification - clustering
Module 07: Classification - supervised
Module 08: Introduction to deep learning
Module 09: Convolutional features
Module 10: Classification - CNN
Module 11: Segmentation using CNNs
Module 12: Object tracking
Module 13: Person re-identification
Module 14: Behavior Analysis RNNs
Exercises: 
Introduction
Feature point detection
Clustering
Retrain a neural network