Prof. Marc Pollefeys: Computer Vision

Marc Pollefeys has been Full Professor in the Institute for Computational Science at ETH Zurich since 2007 where he leads the Computer Vision and Geometry Lab. Between 2002 and 2005 he was Assistant Professor at the Department of Computer Science of the University of North Carolina at Chapel Hill. In 2005 he became Associate Professor. Before, he was a postdoctoral researcher at the Katholieke Universiteit Leuven in Belgium where he received his M.S. and Ph.D. degrees in 1994 and 1999, respectively. His main area of research is Computer Vision. Marc Pollefeys has received several prizes and awards for his research, including a Marr prize, an NSF CAREER award, a Packard Fellowship and most recently a European Research Council grant. An interview with Katja Abrahams.

June 2008

What is your main research area?
One of the main research goals of the Computer Vision and Geometry Group is to capture visual representations of the real world. We take images of objects, scenes and events from different view points and analyze them with the computer, trying to rebuild their three-dimensional structure. This may, e.g. be done by a number of cameras mounted on a vehicle to measure the environment by capturing images. A good example of such an application is Google Earth where you need three-dimensional models of the city so that people can explore them virtually. Another example consists of robot navigation where our technology allows robots to map their environment and to localize themselves in it. We are currently planning collaborations with Roland Siegwart from the Department of Mechanical Engineering on autonomous flying micro-aerial vehicles.  We are also setting up a collaboration with Honda to enable their humanoid robot ASIMO to explore environments on its own.

Besides static scenes, the other main research direction in my group consists of recording dynamic events where multiple people interact.  Our main goal in this area is to recover visually immersive spatio-temporal (4D) representations of interacting people or events taking place in large indoor or outdoor environments. We have just received a large European Research Council grant to do research in this area over the next five years.  More in general, Computer Vision is an exciting field with applications in many different areas, ranging from archaeology, forensics, terrain modeling to human-computer interaction, entertainment, surveillance and medicine.

What are currently the most challenging issues in Computer Vision?
The biggest challenge in Computer Vision actually is the complexity of the real world. It is very hard for the computer to recognize and reconstruct certain objects, for example a chair as a chair with all the different types that exist. As humans we are very good at judging what is going on in a scene. For the computer, however, recovering the different shapes in a non-static scene where people interact is very difficult.

Which courses are you teaching this semester and what will you teach next semester?
This semester I am teaching the class Computational Photography & Video for master students and advanced bachelor students. In this class we look at how the advent of digital technology has allowed to change photography. At first, digital photography was just developed to mimic analog photography. Nowadays, the progress in technology, storage capacity and processing gives us a lot more options. Instead of doing as well as analog photography we can really create the pictures we want. By taking multiple pictures and combining the elements we like best, we get quite satisfactory results. A typical problem we can solve this way is a family picture: you will never find everybody with their nicest smile at the same time in the same picture. So, just take a few shots and pick the best smiles! Some companies, such as Adobe, are already quite active in this field.

Next semester I will co-teach one of the first semester classes, e.g. linear algebra,  together with Daniel Kresner, a young professor from mathematics. Mathematics in general and linear algebra in particular are  actually tremendously useful in my research area. And it is easier for students to internalize the concepts when they get to know areas where the theory can be applied.

Will you collaborate with other people at D-INFK?
I hope to set up a lot of interesting collaborations in the years to come.  Obviously, I expect to collaborate with my colleagues working on Visual Computing, in particular Markus Gross, Mark Pauly and Joachim Buhmann.  As an example Mark Pauly, who heads the Applied Geometry Group, is interested in the dynamic 3D shapes we can capture to analyze how they deform and also how they can be rendered. But I also expect to collaborate with other members of the department. For example, my group develops technology to generate 3D movies from 2D videos. Thus, a collaboration with Cary Kornfeld who teaches the Stereoscopic Imaging class where students make 3D movies would make a lot of sense. Currently, spectators get tired just after watching short 3D films. In the future we might be able to provide technologies where people do not need to wear special glasses any more to see 3D effects without limiting the artistic choices of the directors.

What do you like about teaching?
I like the interaction and bringing over concepts students will be able to use throughout their careers. Seeing their evolution from bachelor to master students to PhD students and may be afterwards to researchers is very rewarding.

Why did you become a computer scientist?
In Leuven, my specialty was taught at the Electrical Engineering Department. However, Computer Vision is actually a field at the boundary between Electrical Engineering and Computer Science. Historically, signal processing and image processing belong more to Electrical Engineering. In CS, image processing evolved more out of Artificial Intelligence. The trend is more tending towards CS because the signal processing aspects are losing importance whereas the algorithmic concepts are gaining ground. Synergies between both departments should of course be used.

What do you think of ETH as a research center?
The University of North Carolina at Chapel Hill where I used to work and ETH are both excellent places. As a European I was happy to come to Zurich since it is in the heart of the continent and offers a very good quality of life. Besides, there is a lot of flexibility and freedom in research here at ETH. Also, there are many excellent colleagues both in our department as well as in other departments and this provides excellent opportunities for collaborations.

Is there anything special you would like to say to our students or future students?
I am looking forward to work with them, in classes as well as in research!

For further information, see http://www.inf.ethz.ch/personal/pomarc/research.html