embodied_ai.md

Embodied AI

• Embodied AI
  • Survey
  • Embodied AI
  • Robotics
  • Humanoids
  • Projects
  • Misc

Survey

• Neural Fields in Robotics: A Survey, arXiv, 2410.20220, arxiv, pdf, cication: -1

  Muhammad Zubair Irshad, Mauro Comi, Yen-Chen Lin, ..., Zsolt Kira, Jonathan Tremblay · (robonerf.github)

Embodied AI

• OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints, arXiv, 2501.03841, arxiv, pdf, cication: -1

  Mingjie Pan, Jiyao Zhang, Tianshu Wu, ..., Wenlong Gao, Hao Dong · (omnimanip.github)

• Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Grounding, arXiv, 2501.04693, arxiv, pdf, cication: -1

  Joshua Jones, Oier Mees, Carmelo Sferrazza, ..., Pieter Abbeel, Sergey Levine

• 🌟 EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation, arXiv, 2501.01895, arxiv, pdf, cication: -1

  Siyuan Huang, Liliang Chen, Pengfei Zhou, ..., Maoqing Yao, Guanghui Ren · (sites.google)

• Towards Generalist Robot Policies: What Matters in Building Vision-Language-Action Models, arXiv, 2412.14058, arxiv, pdf, cication: -1

  Xinghang Li, Peiyan Li, Minghuan Liu, ..., Hanbo Zhang, Huaping Liu

• Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection, arXiv, 2412.04455, arxiv, pdf, cication: -1

  Enshen Zhou, Qi Su, Cheng Chi, ..., Lu Sheng, He Wang · (zhoues.github)

• Moto: Latent Motion Token as the Bridging Language for Robot Manipulation, arXiv, 2412.04445, arxiv, pdf, cication: -1

  Yi Chen, Yuying Ge, Yizhuo Li, ..., Ying Shan, Xihui Liu · (chenyi99.github) · (Moto - TencentARC)

• 🌟 Unraveling the Complexity of Memory in RL Agents: an Approach for Classification and Evaluation, arXiv, 2412.06531, arxiv, pdf, cication: -1

  Egor Cherepanov, Nikita Kachaev, Artem Zholus, ..., Alexey K. Kovalev, Aleksandr I. Panov

• Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making, arXiv, 2410.07166, arxiv, pdf, cication: 1

  Manling Li, Shiyu Zhao, Qineng Wang, ..., Jiayuan Mao, Jiajun Wu · (embodied-agent-interface.github) · (embodied-agent-eval - embodied-agent-eval) · (𝕏)

• foundation models for the physical world.

• DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution, arXiv, 2411.02359, arxiv, pdf, cication: -1

  Yang Yue, Yulin Wang, Bingyi Kang, ..., Jiashi Feng, Gao Huang · (DeeR-VLA - yueyang130)

• DynaMem: Online Dynamic Spatio-Semantic Memory for Open World Mobile Manipulation, arXiv, 2411.04999, arxiv, pdf, cication: -1

  Peiqi Liu, Zhanqiu Guo, Mohit Warke, ..., Nur Muhammad Mahi Shafiullah, Lerrel Pinto · (dynamem.github)

• A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks, arXiv, 2410.22391, arxiv, pdf, cication: -1

  Thomas Schmied, Thomas Adler, Vihang Patil, ..., Razvan Pascanu, Sepp Hochreiter · (arxiv) · (huggingface) · (LRAM - ml-jku)

• VLMimic: Vision Language Models are Visual Imitation Learner for Fine-grained Actions, arXiv, 2410.20927, arxiv, pdf, cication: -1

  Guanyan Chen, Meiling Wang, Te Cui, ..., Yi Yang, Yufeng Yue

• VidEgoThink: Assessing Egocentric Video Understanding Capabilities for Embodied AI, arXiv, 2410.11623, arxiv, pdf, cication: -1

  Sijie Cheng, Kechen Fang, Yangyang Yu, ..., Lei Han, Yang Liu

Robotics

• CS 294-277, Robots That Learn (Fall 2024)

• Introducing AgiBot World, the first large-scale robotic learning dataset designed to advance multi-purpose robotic policies.

  · (Agibot-World - OpenDriveLab)

• openpilot - commaai

• flow_matching - HRI-EU

• fastest, most precise, and most capable hand control setup 𝕏

• GRAPE: Generalizing Robot Policy via Preference Alignment, arXiv, 2411.19309, arxiv, pdf, cication: -1

  Zijian Zhang, Kaiyuan Zheng, Zhaorun Chen, ..., Dieter Fox, Huaxiu Yao · (grape-vla.github)

• Soft Robotic Dynamic In-Hand Pen Spinning, arXiv, 2411.12734, arxiv, pdf, cication: -1

  Yunchao Yao, Uksang Yoo, Jean Oh, ..., Christopher G. Atkeson, Jeffrey Ichnowski · (soft-spin.github)

• trained surgical robots through video-based imitation learning to match human surgeon skill levels 𝕏

• DEEP Robotics shared a new video of their new commercially available robot dog 𝕏

• HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots, arXiv, 2410.21229, arxiv, pdf, cication: -1

  Tairan He, Wenli Xiao, Toru Lin, ..., Linxi Fan, Yuke Zhu · (x) · (hover-versatile-humanoid.github)

• DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning, arXiv, 2410.24185, arxiv, pdf, cication: -1

  Zhenyu Jiang, Yuqi Xie, Kevin Lin, ..., Linxi Fan, Yuke Zhu · (dexmimicgen.github) · (twitter)

• Advancing embodied AI through progress in touch perception, dexterity, and human-robot interaction

  · (x)

• what is the largest public dataset of robot expert demonstrations across diverse manipulations tasks, constrained to a single form factor? 𝕏

• AI-assisted multi-arm robot for apple picking

  · (v.redd)

• moss-robot-arms - jess-moss

• Language-Model-Assisted Bi-Level Programming for Reward Learning from Internet Videos, arXiv, 2410.09286, arxiv, pdf, cication: -1

  Harsh Mahesheka, Zhixian Xie, Zhaoran Wang, ..., Wanxin Jin

Humanoids

• unitree_rl_gym - unitreerobotics

• Boston Dynamics showcased Atlas autonomously transferring engine covers using ML vision for object detection and localization 𝕏

• EngineAI unveiled SE01, a humanoid robot achieving natural walking through advanced joint modules and neural networks. 𝕏

• Introducing Torso, a bimanual android actuated with artificial muscles. 𝕏

• CooHOI, a learning-based framework designed for the cooperative transportation of objects by multiple humanoid robots. 𝕏

• CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics, arXiv, 2406.14558, arxiv, pdf, cication: -1

  Jiawei Gao, Ziqin Wang, Zeqi Xiao, ..., Jifeng Dai, Jiangmiao Pang · (gao-jiawei)

Projects

• Genesis - Genesis-Embodied-AI

Misc