Where Computer Graphics, Computer Vision and AI Meet

Lingjie Liu, an assistant professor in the Department of Computer and Information Science in the School of Engineering and Applied Science, uses AI both as a subject of research and as a core methodology for building new 3D visual computing systems. Her work resides at the interface of computer graphics, computer vision, and AI, where she focuses on learning-based methods for representing, reconstructing, and generating 3D humans and scenes from visual observations.

“More broadly, I am interested in how AI can help us move from pixels to structured 3D understanding: recovering geometry, appearance, motion, and dynamics in ways that are not only visually compelling but also controllable and useful for downstream applications,” she says.

Liu is intrigued by a new genre of 3D reconstruction and rendering algorithms for human characters and general scenes. She says this area is especially exciting because it sits right between strong mathematical rules about geometric structures and the flexibility of modern machine learning.

Classical computer graphics provides powerful built-in rules regarding geometry, rendering, and animation, while deep learning is able to handle the complexity, ambiguity, and scale of real-world data, she explains. So the ability to combine them is particularly important for rendering realistic human characters and general scenes, which are “highly dynamic, visually complex, and difficult to model with purely hand-engineered pipelines.”

Recently, Liu has been especially excited about projects that push 3D vision and generation beyond visual realism toward more physically accurate motion. For example, in the PhysCtrl project, Liu and researchers explored how to make video generation more physically grounded (i.e., consistent with physics) and controllable.

Read More at Penn Today