Building Foundation Models for Generalist Humanoid Robots

In an era of rapid AI progress, leveraging large-scale computing and data has unlocked new possibilities for developing generalist AI models. As AI systems like ChatGPT achieve remarkable performance in the digital realm, we are compelled to ask: Can we replicate these breakthroughs in the physical world — to create autonomous robots capable of performing everyday tasks? In this talk, I will discuss data-centric research principles and methodologies for building generalist humanoid robots. I will discuss the sciences and techniques of leveraging diverse data sources for training robot foundation models.

Bio

Yuke Zhu is an Associate Professor in the Computer Science Department of UT-Austin, where he directs the Robot Perception and Learning (RPL). He is also a Director and Distinguished Research Scientist at NVIDIA Research, where he co-leads the Lab. I am also a senior research scientist at NVIDIA Research, where I co-lead the Generalist Embodied Agent Research (GEAR) group. He focuses on developing intelligent algorithms for generalist robots and embodied agents to reason about and interact with the real world. He obtained his Master's and Ph.D. degrees from Stanford University. He received the NSF CAREER Award, the IEEE RAS Early Academic Career Award, and various faculty fellowships and research awards from Amazon, JP Morgan, and Sony Research.