Visualizing the Future of Robotics

On a sunny summer day in 2000, Camillo Jose Taylor and several of his students visited a U.S. Army training site in Fort Benning, Georgia. Soon after they arrived, three helicopters suddenly appeared above, and soldiers started to rappel down to the ground. Then they heard gunfire and became enveloped by plumes of smoke from simulated grenades that landed nearby. In the midst of the mayhem, the students remained focused on their one mission: to program a team of robots to diligently collect data about the scene. "It was not the usual environment for successful programming, but the students performed very well," says Taylor, professor in Computer and Information Science (CIS) at Penn.

The training operation illustrates one way that robots may be used in the future, according to Taylor’s grand vision. By assembling themselves into communication networks, teams of robots can relay messages to one another to optimally position themselves for efficient data collection in dangerous places. "It’s hard for individual robots to understand everything about the surrounding scene. But when they form social networks or swarms and start gossiping with each other, they can develop a better understanding of the environment to solve complicated problems," Taylor says.

As a member of Penn’s General Robotics, Automation, Sensing and Perception (GRASP) Laboratory, Taylor has set his sights on making robots smarter. He is developing algorithms that will allow robots to efficiently recognize objects so that they can respond to commands such as "fetch the keys on the table" and dexterously manipulate different items coming down a conveyer belt in a factory. "Currently, robots are good at not bumping into things, but they can’t tell the difference between a garbage can and a person," Taylor notes. :For a long time, computer vision researchers were trying to figure out how to program robots to avoid obstacles. Now we’re starting to ask the next level of questions to allow them to build a detailed 3D representation of the environment."

Taylor is also using his expertise in computer vision to build compelling virtual experiences. Working with Ph.D. student Ryan Kennedy, he has developed an extremely fast algorithm to create 3D models of objects that have been videotaped with a webcam. Additional algorithms make sense of building interiors through the construction of high-resolution 3D models from image data and the interpretation of data from depth cameras such as the Kinect™ sensor.

Ultimately, Taylor envisions combining these approaches with virtual reality technology to allow users to visualize and edit complex 3D shapes and environments. This kind of technology could be used to speed the design of complex shapes such as dental molds or medical implants, which could be sculpted in a virtual environment and then manufactured on a 3D printer. “With the current generation of systems, it’s very tough to visualize what’s happening in 3D,” Taylor reports. “In the long run, we hope to produce systems that are a lot more intuitive and simpler to use.”

Sharing the Spirit of Fun

One of Taylor’s earliest robotics experiences set the stage for his current research interests. As a Ph.D. student working on a project with Daniel Koditschek, who was then at Yale but is now the Alfred Fitler Moore Professor in Electrical and Systems Engineering at Penn, Taylor developed algorithms that allowed juggling robots to track Ping-Pong balls. “That project got me thinking about the problems of robotics and computer vision, and inspired me to focus my work in those areas,” Taylor recalls.

Toward that end, Taylor was enticed to join Penn in 1997, in large part because of the world-class GRASP Lab and its top-notch faculty. Now as a professor, he has brought the spirit of fun from his graduate school years into the classroom. For example, he teaches the C programming language to students taking CIS 240: Introduction to Computer Architecture by having them program video games such as Tetris, Space Invaders and Frogger. “Professor Taylor likes to trick his students into learning something,” says David Mally (CIS’15), who was a teaching assistant for the course. “He not only effectively communicates the information he intends in lecture, but is also incredibly witty and funny along the way.”

This sense of humor has even spread to exam time. Some of Taylor’s most memorable test questions have asked students to respond to the harebrained schemes of their cousin “Crazy Eddie,” a feckless computer scientist who does things the quick and dirty way. “Crazy Eddie typically makes a big mess, which you are left to clean up,” Mally says. “Professor Taylor definitely enjoys being more than a little evil in his classes. Once, he came to the midterm exam dressed as the Grim Reaper, complete with a scythe, and menacingly walked up and down the rows of Towne 100.”

What stands out to Evelyn Yeung (CIS’16) as she reminisces about taking CIS 240 are Taylor’s creative analogies, such as, “C is like a chainsaw. You can cut through much more, but if you don’t know what you’re doing, you’ll end up in a world of hurt.” Taylor makes the course material come to life through a refreshing combination of energy, humor and wit, notes Yeung. “He strikes a great balance between detailed explanation and relevant humor, making him approachable.”

Promoting Progress at Penn

Through engaging lectures in the classroom and patient explanations during office hours, Taylor has earned the Christian R. and Mary F. Lindback Award for Distinguished Teaching at Penn. He has also helped to develop two new courses: Engineering and Applied Science (EAS) 105: Introduction to Scientific Computing, which focuses on programming and computational thinking and has been adopted by many departments; and EAS 205: Applications of Scientific Computation, which aims to provide a solid foundation in both the theory of linear algebra and its application to important problems in robotics, computer vision, graphics and machine learning.

When he is not developing course material or conducting research, Taylor stays busy with leadership activities. He was the undergraduate chair of the Computer and Information Science department, and now he directs the GRASP Lab’s master’s program in Robotics and is a member of the faculty oversight committee for the Computer Engineering program. Taylor, who was born in Jamaica, also serves as a member of the faculty diversity committee at the School of Engineering and Applied Science.

Through his interactions with other faculty involved in shared leadership activities and research projects, Taylor is reminded of what originally lured him to Penn. “It’s a great place to work because I know that if I have a question or an idea, my colleagues are always happy to talk with me,” Taylor says. “We benefit a lot from sharing different ways of thinking between disciplines, and with this wonderful mix of ideas, there’s no problem we can’t attack.”

View the original article in Penn Engineering magazine "Visualizing the Future" by Janelle Weaver.

Return to News Features