Mark Pauly, Digital Humans - Will 3D Avatars Change the Way We Communicate in the Future?

16 November 2015 | Colloquium

ABSTRACT: Virtual environments offer a fascinating playground for developing new forms of human communication. I will discuss recent advances in real-time face and hand tracking and show how such technology can be integrated to build new avatar-based online communication systems.

A user is tracked with a commodity sensor in a natural environment. Her expressions and motions are transferred in realtime to a 3D avatar in an arbitrary virtual world, such as a game, a conference room, or any other online social interaction space. The avatar can be a faithful digital representation of the user herself or an entirely different virtual character, allowing new applications of online role-play.

Operating such a system in uncontrolled consumer environments puts high demands on robustness, computational efficiency, and ease of use. I will show how these requirements can be met through a careful design of algorithms specifically targeted to recent consumer-level depth sensing devices. These algorithms are based on research a the interface of computer graphics, computer vision, and machine learning. Leveraging statistical priors on human facial expressions and hand poses, we achieve real-time performance at low latency, which is essential for creating a strong sense of immersion and control of the 3D avatars.

SHORT BIO: Mark Pauly is an associate professor at the School of Computer and Communication Sciences at EPFL. Prior to joining EPFL, he was assistant professor at the CS department of ETH Zurich from April 2005. From August 2003 to March 2005 he was a postdoctoral scholar at Stanford University, where he also held a position as visiting assistant professor during the summer of 2005. He received his PhD (with distinction) in 2003 from ETH Zurich and his MSc degree (with highest honors) in 1999 from TU Kaiserslautern. His research interests include computer graphics and animation, geometry processing, shape modeling and analysis, and computational geometry. He was a recipient of a full-time scholarship of the German National Merit Foundation, received the ETH medal for his outstanding dissertation, and was awarded the Eurographics Young Researcher Award in 2006.

Date 16 November 2015
Time 16:15 - 17:15
Speaker Mark Pauly, École Polytechnique Fédérale de Lausanne
Language English
Area of expertise Computer Sciences
Host Dep. Informatik, Hosts: Olga Sorkine, Wenzel Jakob
Include this event in your calendar (ICS, 5kB) 
Page URL:
Fri Jul 28 15:10:01 CEST 2017
© 2017 Eidgenössische Technische Hochschule Zürich