EE - From Natural Language to Safe Task and Motion Planning (ITMP)
This project aims to build robots that can understand everyday natural-language instructions and turn them into safe, verifiable behavior in complex environments. The research focuses on an Integrated Temporal Task and Motion Planning (ITMP) architecture that connects language, temporal logic, perception, scene-graph belief, task planning, motion optimization, and execution into one coherent pipeline. Students will work with techniques such as translating natural-language commands into finite-trace temporal logic, compiling these formulas into deterministic finite automata for task monitoring, building and querying 3D dynamic scene graphs for persistent semantic perception, and performing optimization-based task and motion planning using convex-region graphs. They will also help design and analyze runtime monitoring, failure classification, and abstraction refinement so that the robot can detect when plans are unsafe or mismatched to reality and repair the underlying models rather than silently failing. Undergraduates will participate by implementing and testing these components in simulation, integrating modules into the full pipeline, and running experiments that evaluate task success, contract satisfaction, safety, and recovery on realistic household-like scenarios.
This lab is a dynamic place to work because it sits right at the frontier of embodied, safe AI for robots: connecting natural language, formal logic, rich 3D perception, and optimization-based motion planning in a single system that can be tested on realistic tasks and environments. The project is important because future service robots must do more than “follow a script”. Instead, they need explicit, checkable contracts for goals and safety constraints so that long-horizon behaviors like “deliver objects while avoiding certain rooms and hazards” can be verified and monitored rather than trusted blindly. Alongside this ITMP effort, the group is also working on integrated perception–planning–control for ground robots, physics-informed control and learning for Physical AI, and safety-aware planning and execution, giving students exposure to multiple cutting-edge directions in robotics and trustworthy AI within one research environment.