Research
Featured
Reasoning about people’s mental states based on their behaviors observed from multimodal inputs.
• Self-Supervised Online ToM Reasoning (ICML 2026)
• Automated ToM reasoning (NeurIPS 2025) (Spotlight)
• Multimodal Multiagent ToM (AAAI 2025) (Oral)
• Multimodal ToM QA (ACL 2024) (Outstanding Paper Award)
Assisting humans with physical actions and verbal communication in complex embodied settings.
• Goal-Oriented Mental Alignment (IROS 2024)
• Neurally-guided Online Probabilistic Assistance (ICRA 2023)
• Watch-And-Help (ICLR 2021)
Learning human preferences from rich human feedback.
• User Thoughts in LLM Interactions (preprint)
• RL from User Conversations (preprint)
• Pragmatic Feature Preferences (ICML 2024)
• Graph-based Active Reward Learning (ICML 2022)
A world simulator for scaling photorealistic multi-agent interactions.
• SimWorld (NeurIPS 2025) (Spotlight)
• SimWorld-Robotics (NeurIPS 2025)
More
This tutorial aims to summarize and present a unified view of connecting WMs and LMs and highlight the various opportunities for improved machine reasoning and planning based on (or even beyond) large LMs through world modeling.
An open-sourced mulit-agent household embodied simulator.