3D Scenes: Understanding and Rendering

The general availability of depth sensors makes it easier and faster to acquire 3D data of a real-world scene. Together with neural networks, recent advances in deep learning in 3D has paved the way for better understanding our world in 3D. In this talk, I am going to share the experience of my journey in this direction, framing it from an end-to-end perspective. I will start with various stages in dataset making from building tools for data acquisition to data annotation. I will then highlight recent works in 3D scene understanding including designing convolution and neural networks for point clouds classification and post-processing semantic segmentation with conditional random fields. In addition, I will briefly explain the rendering of 3D scenes, and discuss potential future research in both directions.