# 3D Scanning & Motion Capture

 Lecturer Studies Master Informatics Time, Place Tuesdays, 14:00-16:00, Room:  02.13.010Wednesdays, 10:00-12:00, Room:  02.13.010 Begin October 17., 2017 Prerequisites Introduction to Informatics I, Analysis, Linear Algebra, Computer Graphics, C++

# Content

3D reconstruction, RGB-D scanning (Kinect, Tango, RealSense), ICP, camera tracking, sensor calibration, VolumetricFusion, Non-Rigid Registration, PoseTracking, Motion Capture, Body-, Face-, and Hand-Tracking, 3D DeepLearning, selected optimization techniques to solve the problem statements (GN, LM, gradient descent).

Syllabus:

Lecture Content

• Requirements
• C++ is a must
• Profound knowledge of linear algebra
• Basic concepts of graphics and 3D
• Basic concepts of geometry
• Meshes (polygonal), Point Clouds, Pixels & Voxels
• RGB and Depth Cameras
• Extrinsics and Intrinsics
• Capture devices
• RGB and Multi-view
• RGB-D cameras
• Stereo
• Time of Flight (ToF)
• Structured Light
• Laser Scanner, Lidar
• Surface Representations
• Polygonal meshes (trimeshes, etc.)
• Parametric surfaces: splines, nurbs
• Implicit surfaces
• Ridge-based surfaces
• Signed distance functions (volumetric, Curless & Levoy)
• Indicator function (Poisson Surface Reconstruction)
• More general: level sets
• Marching cubes
• High-level overview of reconstructions
• Structure from Motion (SfM)
• Multi-view Stereo (MVS)
• SLAM
• Optimization
• Non-linear least squares
• Gauss-Newton LM
• Examples in Ceres
• Symbolic diff vs auto-diff
• Auto-diff with dual numbers
• Rigid Surface Tracking & Reconstruction
• Pose alignment
• ICP (point cloud alignment; depth-to-model alignment; rigid ‘fitting’)
• Online surface reconstruction pipeline: KinectFusion
• Scalable surface representations: VoxelHashing, OctTrees
• Loop closures and global optimization
• Robust optimization
• Non-rigid Surface Tracking & Reconstruction
• Surface deformation for modeling
• Regularizers: ARAP, ED, etc.
• Non-rigid surface fitting: e.g., non-rigid ICP
• Non-rigid reconstruction: DynamicFusion/VolumeDeform/KillingFusion
• Body Tracking & Reconstruction
• Skeleton Tracking and Inverse Kinematics
• Learning-based approaches from RGB and RGB-D
• Face Tracking & Reconstruction
• Keypoint detection & tracking
• Parametric / Statistical Models -> BlendShapes
• Hand Tracking & Reconstruction
• Parametric Models
• Some DeepLearning-based things
• Discriminative vs generative tracking
• Random forests
• Motion Capture in Movies
• Marker-based motion capture
• LightStage -> movies
• BRDF and Material Capture
• Open research questions
• Useful tools:
• Eigen
• Ceres

# References

D. A. Forsyth and J. Ponce. Computer Vision: A Modern Approach (2nd Edition). Prentice Hall, 2011.

R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, 2003.

Botsch, Kobbelt, Pauly, Alliez, Levy: Polygon Mesh Processing, AK Peters, 2010

# News

Matthias Niessner, our new Professor from Stanford University, offers a number of interesting topics for  master theses.

PhD positions on   Computational Fabrication and 3D Printing and  Photorealistic Rendering for Deep Learning and Online Reconstruction are available at the Computer Graphics & Visualization group.