🤹‍♂️ Auro Puppeteer

An application leveraging Apple Vsion Pro and RGB camera for showcasing and teleoperation of robots.

⚙️ Install

Auro Puppeteer could be installed with following commands.

# Clone repo
git clone [email protected]:Auromix/auro_puppeteer.git
# Update submodules
cd auro_puppeteer
git submodule update --init --recursive

# Download latest assets to top directory of project
xdg-open https://drive.google.com/drive/folders/1bhe3dcdHjzunkdBjNsx2vrv3O-L8qrY_?usp=drive_link

# Create conda environment
conda create -n auro_puppeteer python=3.10 -y
conda activate auro_puppeteer

# Install dex-retargeting
cd auro_puppeteer/auro_puppeteer/libs/dex-retargeting/
pip  install -e ".[example]"

# Check GLIBC>2.34
ldd --version
# Install isaac sim in conda
pip install isaacsim==4.1.0.0 --extra-index-url https://pypi.nvidia.com
pip install isaacsim-extscache-physics==4.1.0.0 isaacsim-extscache-kit==4.1.0.0 isaacsim-extscache-kit-sdk==4.1.0.0 --extra-index-url https://pypi.nvidia.com

# Install auro_puppeteer
cd auro_puppeteer
pip install -e .

🔥 Quickstart

You can find detailed examples in auro_puppeteer/examples.

RGB Camera teleoperation

For RGB camera dual hand teleoperation, run:

python rgb_camera_dual_hand_teleoperation.py

🙋 Troubleshooting

For any issues, questions, or contributions related to this package, please contact maintainers of this repository.

🧑‍💻 Future Development Plans

• Native Support for Apple Vision Pro

Integrate native support for Apple Vision Pro to leverage advanced visual capabilities.

• Data Collection and Synthetic Data Synthesis

Implement functionality for data collection and synthesis of simulated data to enhance training and operation.

🏆 Contributing

Contributions are welcome! Please read the contributing guidelines before submitting a pull request.

⚛️ Acknowledgments

This project draws inspiration and references from the following projects:

• AnyTeleop
• DexRetargeting
• OpenTeleVision
• RoboCasa
• MimicGen

📜 License

Copyright 2023-2024 Herman Ye @Auromix
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.