diff --git a/README.md b/README.md index 300fa1b78..c9c0bb322 100644 --- a/README.md +++ b/README.md @@ -33,53 +33,100 @@ ## Why dora-rs? -In 2023, AI is booming! Robotic framework however hasn't changed much in years... This is why we create dora-rs! dora-rs is a new robotic framework that brings modernity into robotic application. +In 2024, AI is booming! Robotic framework however hasn't changed much in years... This is why we create dora-rs! dora-rs main objectives is: -dora-rs can already show impressive performance, up to 17x speedup compared to ROS2 in Python! This is the result of using our own shared memory server and Apache Arrow to achieve zero copy! +- Make creation of robotic applications fast and simple +- Dead easy integration with latest technologies (e.g. DL models, Cloud, DBs ...) -Those performance improvements make a world of difference for beginners, AI practitioners, and weekend hobbyists who have been limited by the lack of support for Python in this field! +dora-rs is still experimental and you might experience bugs, but we're working very hard to make it stable as possible. -But, dora-rs is still experimental and we're still working on many features such as: +## Arrow Message Passing -- [dora-ros2-bridge](https://github.com/dora-rs/dora-ros2-bridge) to bridge dora and ros 2! -- dora-rs for Deep Learning applications, to make it easy to integrate state of the art DL model and hopefully train them within the framework. +dora uses Apache Arrow message format to pas data within the dataflow. -Feel free to reach out if you have any questions! +[Apache Arrow](https://github.com/apache/arrow-rs) defines a language-independent columnar memory format for flat and hierarchical data. The Arrow memory format also supports zero-copy reads for lightning-fast data access without serialization overhead. -

- - -

+Using Apache Arrow, all messages shares a unified memory layout that can be used by many of your favorite libraries. + + + +## Performance + +dora-rs can already show impressive performance, up to 17x speedup compared to ROS2 in Python! This is the result of using our own shared memory server and Apache Arrow to achieve zero copy data passing. + + + +> See: https://github.com/dora-rs/dora-benchmark/tree/main for reproduction. +> Note that performs can vary depending on platform but should still be significantly better. + +## Opentelemetry + +dora-rs uses Opentelemetry to record all your logs, metrics and traces and link them all through a shared abstraction. + +[Opentelemetry](https://opentelemetry.io/) is a high-quality, ubiquitous, and portable telemetry to enable effective observability. + +Opentelemetry is language independent, backend agnostic, and distributable making it usable anywhere you need it. We also link all the data recorded by [dora-record](libraries/extensions/dora-record) point to the Opentelemetry traces, going further than the initial premise of Opentelemetry. + + + +## Hot-Reloading + +Hot-Reloading makes it possible to change code at runtime in Python while keeping state. + +We knows that it might be tiresome to switch off and on a robot or a simulation, so we made it possible using the feature flag: `--attach --hot-reload` to watch for code change and reload nodes that has been modified. + +keep in mind that we keep the current class state to not create weird state reset. + +This features comes with fail-safe mecanism. See: https://github.com/dora-rs/dora/pull/239 + + +demo + + +## Self-Coding Robot: Code RAG (WIP) + +Hot-reloading can be combined with a Retrieval Augmented Generation (RAG), so that you can modify the robot source code using natural language at runtime. + +See:[examples/python-operator-dataflow](examples/python-operator-dataflow) + + + +Self-Coding Robot is just the tip of the iceberg of robotics combined with llm. but, there is so much more that we haven't explored yet like: + +- [self-debugging](https://arxiv.org/pdf/2304.05128.pdf) +- [memory](https://github.com/cpacker/MemGPT) +- [function calling](https://github.com/ShishirPatil/gorilla) +- ... ## Installation Quickest way: ```bash -cargo install dora-cli # In case of issues, you can try to add `--locked` +cargo install dora-cli --locked pip install dora-rs # For Python API dora --help ``` -For more installation guideline, check out our installation guide here: https://dora.carsmos.ai/docs/guides/Installation/installing +For more info on installation, check out [our guide](https://dora.carsmos.ai/docs/guides/Installation/installing). ## Getting Started 1. Install the example python dependencies: ```bash -pip install -r https://raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/requirements.txt +pip install -r raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/requirements.txt ``` 2. Get some example operators: ```bash -wget https://raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/webcam.py -wget https://raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/plot.py -wget https://raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/utils.py -wget https://raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/object_detection.py -wget https://raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/dataflow.yml +wget raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/webcam.py +wget raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/plot.py +wget raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/utils.py +wget raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/object_detection.py +wget raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/dataflow.yml ``` 3. Start the dataflow @@ -95,9 +142,54 @@ To stop your dataflow, you can use ctrl+c To go further, you can add a yolov8 operator, check out our getting started here: https://dora.carsmos.ai/docs/guides/getting-started/yolov8/ +## ROS2 Bridge + +- Compilation Free Message passing to ROS 2 +- Automatic conversion ROS 2 Message <-> Arrow Array + +```python +import random +import pyarrow as pa + +# Configuration Boilerplate... +turtle_twist_writer = ... + +## Arrow Based ROS2 Twist Message +## which does not requires ROS2 import +message = pa.array([{ + "linear": { + "x": 1, + }, + "angular": { + "z": 1 + }, + }]) + +turtle_twist_writer.publish(message) +``` + +> You might want to use ChatGPT to write the Arrow Formatting: https://chat.openai.com/share/4eec1c6d-dbd2-46dc-b6cd-310d2895ba15 + +## Hardwares + +Cool hardware that we think might be good fit to try out dora-rs 🙋 We are not sponsored by manufacturers: + +| | Price | Open Source | Github | type | Dora Project | +| --------------------------------- | ----- | ------------------ | ---------------------------------------------------- | ---------- | ------------------------------------------------------- | +| DJI Robomaster S1 | 550$ | SDK | https://github.com/dji-sdk/RoboMaster-SDK | Rover | https://huggingface.co/datasets/dora-rs/dora-robomaster | +| DJI Robomaster EP Core | 950$ | SDK | https://github.com/dji-sdk/RoboMaster-SDK | Rover, Arm | | +| DJI Tello | 100$ | | | Drone | | +| BitCraze Crazyflies | 225$ | Firmware, Lib, SDK | https://github.com/bitcraze | Drone | | +| AlexanderKoch-Koch/low_cost_robot | 250$ | Everything | https://github.com/AlexanderKoch-Koch/low_cost_robot | Arm | | +| xArm 1S | 200$ | | | Arm | | +| Wavego | 250$ | | | Quadruplet | | +| AINex | 800$ | | | Humanoid | | + +> For more: https://docs.google.com/spreadsheets/d/1YYeW2jfOIWDVgdEgqnMvltonHquQ7K8OZCrnJRELL6o/edit#gid=0 + ## Documentation -The full documentation is available on our website: https://dora.carsmos.ai +The full documentation is available on [our website](https://dora.carsmos.ai) ## Discussions @@ -110,6 +202,23 @@ Feel free to reach out on any topic, issues or ideas. We also have [a contributing guide](CONTRIBUTING.md). +## Support Matrix + +| | dora-rs | Hoped for | +| --------------------------------- | --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | +| **Tier 1 Support** | Python, Rust | C, C++, ROS 2 | +| **Tier 2 Support** | C, C++, ROS2 | +| **Hot-reloading** | Python | Rust (https://github.com/orgs/dora-rs/discussions/360) | +| **Message Format** | Arrow | Native | +| **Local Communication** | Shared Memory | Custom Middleware, [zero-copy GPU IPC](https://arrow.apache.org/docs/python/api/cuda.html), intra-process channel communication | +| **Remote Communication** | TCP (See: https://github.com/dora-rs/dora/issues/459) | Custom Middleware, [Zenoh](https://zenoh.io/) | +| **Metrics, Tracing, and Logging** | Opentelemetry | Native logging libraries into Opentelemetry | +| **Data archives** | Parquet ([dora-record](libraries/extensions/dora-record)) | +| **Visualization and annotation** | OpenCV | [rerun.io](rerun.io) | +| **Supported Platforms (x86)** | Windows, macOS, Linux | +| **Supported Platforms (ARM)** | macOS, Linux | +| **Configuration** | YAML | + ## License This project is licensed under Apache-2.0. Check out [NOTICE.md](NOTICE.md) for more information.