Skip to content

Commit

Permalink
Refactor README
Browse files Browse the repository at this point in the history
  • Loading branch information
@haixuanTao
haixuanTao committed Apr 9, 2024
1 parent 9508fbb commit c3d4cac
Showing 1 changed file with 129 additions and 20 deletions.
149 changes: 129 additions & 20 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,53 +33,100 @@

## Why dora-rs?

In 2023, AI is booming! Robotic framework however hasn't changed much in years... This is why we create dora-rs! dora-rs is a new robotic framework that brings modernity into robotic application.
In 2024, AI is booming! Robotic framework however hasn't changed much in years... This is why we create dora-rs! dora-rs main objectives is:

dora-rs can already show impressive performance, up to 17x speedup compared to ROS2 in Python! This is the result of using our own shared memory server and Apache Arrow to achieve zero copy!
- Make creation of robotic applications fast and simple
- Dead easy integration with latest technologies (e.g. DL models, Cloud, DBs ...)

Those performance improvements make a world of difference for beginners, AI practitioners, and weekend hobbyists who have been limited by the lack of support for Python in this field!
dora-rs is still experimental and you might experience bugs, but we're working very hard to make it stable as possible.

But, dora-rs is still experimental and we're still working on many features such as:
## Arrow Message Passing

- [dora-ros2-bridge](https://github.com/dora-rs/dora-ros2-bridge) to bridge dora and ros 2!
- dora-rs for Deep Learning applications, to make it easy to integrate state of the art DL model and hopefully train them within the framework.
dora uses Apache Arrow message format to pas data within the dataflow.

Feel free to reach out if you have any questions!
[Apache Arrow](https://github.com/apache/arrow-rs) defines a language-independent columnar memory format for flat and hierarchical data. The Arrow memory format also supports zero-copy reads for lightning-fast data access without serialization overhead.

<p align="center">
<img src="./docs/src/latency.png" width="600">

</p>
Using Apache Arrow, all messages shares a unified memory layout that can be used by many of your favorite libraries.

<img align="center" src="https://github.com/dora-rs/dora-rs.github.io/blob/main/static/img/arrow.png?raw=true" width="600">

## Performance

dora-rs can already show impressive performance, up to 17x speedup compared to ROS2 in Python! This is the result of using our own shared memory server and Apache Arrow to achieve zero copy data passing.

<img src="./docs/src/latency.png" align="center" width="600">

> See: https://github.com/dora-rs/dora-benchmark/tree/main for reproduction.
> Note that performs can vary depending on platform but should still be significantly better.
## Opentelemetry

dora-rs uses Opentelemetry to record all your logs, metrics and traces and link them all through a shared abstraction.

[Opentelemetry](https://opentelemetry.io/) is a high-quality, ubiquitous, and portable telemetry to enable effective observability.

Opentelemetry is language independent, backend agnostic, and distributable making it usable anywhere you need it. We also link all the data recorded by [dora-record](libraries/extensions/dora-record) point to the Opentelemetry traces, going further than the initial premise of Opentelemetry.

<img src="https://github.com/dora-rs/dora-rs.github.io/blob/main/static/img/opentelemetry.png?raw=true" align="center" width="600">

## Hot-Reloading

Hot-Reloading makes it possible to change code at runtime in Python while keeping state.

We knows that it might be tiresome to switch off and on a robot or a simulation, so we made it possible using the feature flag: `--attach --hot-reload` to watch for code change and reload nodes that has been modified.

keep in mind that we keep the current class state to not create weird state reset.

This features comes with fail-safe mecanism. See: https://github.com/dora-rs/dora/pull/239

<a href="http://www.youtube.com/watch?v=NvvTEP8Jak8">
<img align="center" width="600" alt="demo" src=http://img.youtube.com/vi/NvvTEP8Jak8/0.jpg>
</a>

## Self-Coding Robot: Code RAG (WIP)

Hot-reloading can be combined with a Retrieval Augmented Generation (RAG), so that you can modify the robot source code using natural language at runtime.

See:[examples/python-operator-dataflow](examples/python-operator-dataflow)

<img src="https://github.com/dora-rs/dora-rs.github.io/blob/main/static/img/RAG.svg?raw=true" align="center" width="600">

Self-Coding Robot is just the tip of the iceberg of robotics combined with llm. but, there is so much more that we haven't explored yet like:

- [self-debugging](https://arxiv.org/pdf/2304.05128.pdf)
- [memory](https://github.com/cpacker/MemGPT)
- [function calling](https://github.com/ShishirPatil/gorilla)
- ...

## Installation

Quickest way:

```bash
cargo install dora-cli # In case of issues, you can try to add `--locked`
cargo install dora-cli --locked
pip install dora-rs # For Python API

dora --help
```

For more installation guideline, check out our installation guide here: https://dora.carsmos.ai/docs/guides/Installation/installing
For more info on installation, check out [our guide](https://dora.carsmos.ai/docs/guides/Installation/installing).

## Getting Started

1. Install the example python dependencies:

```bash
pip install -r https://raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/requirements.txt
pip install -r raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/requirements.txt
```

2. Get some example operators:

```bash
wget https://raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/webcam.py
wget https://raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/plot.py
wget https://raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/utils.py
wget https://raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/object_detection.py
wget https://raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/dataflow.yml
wget raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/webcam.py
wget raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/plot.py
wget raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/utils.py
wget raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/object_detection.py
wget raw.githubusercontent.com/dora-rs/dora/v0.3.2/examples/python-operator-dataflow/dataflow.yml
```

3. Start the dataflow
Expand All @@ -95,9 +142,54 @@ To stop your dataflow, you can use <kbd>ctrl</kbd>+<kbd>c</kbd>

To go further, you can add a yolov8 operator, check out our getting started here: https://dora.carsmos.ai/docs/guides/getting-started/yolov8/

## ROS2 Bridge

- Compilation Free Message passing to ROS 2
- Automatic conversion ROS 2 Message <-> Arrow Array

```python
import random
import pyarrow as pa

# Configuration Boilerplate...
turtle_twist_writer = ...

## Arrow Based ROS2 Twist Message
## which does not requires ROS2 import
message = pa.array([{
"linear": {
"x": 1,
},
"angular": {
"z": 1
},
}])

turtle_twist_writer.publish(message)
```

> You might want to use ChatGPT to write the Arrow Formatting: https://chat.openai.com/share/4eec1c6d-dbd2-46dc-b6cd-310d2895ba15
## Hardwares

Cool hardware that we think might be good fit to try out dora-rs 🙋 We are not sponsored by manufacturers:

| | Price | Open Source | Github | type | Dora Project |
| --------------------------------- | ----- | ------------------ | ---------------------------------------------------- | ---------- | ------------------------------------------------------- |
| DJI Robomaster S1 | 550$ | SDK | https://github.com/dji-sdk/RoboMaster-SDK | Rover | https://huggingface.co/datasets/dora-rs/dora-robomaster |
| DJI Robomaster EP Core | 950$ | SDK | https://github.com/dji-sdk/RoboMaster-SDK | Rover, Arm | |
| DJI Tello | 100$ | | | Drone | |
| BitCraze Crazyflies | 225$ | Firmware, Lib, SDK | https://github.com/bitcraze | Drone | |
| AlexanderKoch-Koch/low_cost_robot | 250$ | Everything | https://github.com/AlexanderKoch-Koch/low_cost_robot | Arm | |
| xArm 1S | 200$ | | | Arm | |
| Wavego | 250$ | | | Quadruplet | |
| AINex | 800$ | | | Humanoid | |

> For more: https://docs.google.com/spreadsheets/d/1YYeW2jfOIWDVgdEgqnMvltonHquQ7K8OZCrnJRELL6o/edit#gid=0
## Documentation

The full documentation is available on our website: https://dora.carsmos.ai
The full documentation is available on [our website](https://dora.carsmos.ai)

## Discussions

Expand All @@ -110,6 +202,23 @@ Feel free to reach out on any topic, issues or ideas.

We also have [a contributing guide](CONTRIBUTING.md).

## Support Matrix

| | dora-rs | Hoped for |
| --------------------------------- | --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| **Tier 1 Support** | Python, Rust | C, C++, ROS 2 |
| **Tier 2 Support** | C, C++, ROS2 |
| **Hot-reloading** | Python | Rust (https://github.com/orgs/dora-rs/discussions/360) |
| **Message Format** | Arrow | Native |
| **Local Communication** | Shared Memory | Custom Middleware, [zero-copy GPU IPC](https://arrow.apache.org/docs/python/api/cuda.html), intra-process channel communication |
| **Remote Communication** | TCP (See: https://github.com/dora-rs/dora/issues/459) | Custom Middleware, [Zenoh](https://zenoh.io/) |
| **Metrics, Tracing, and Logging** | Opentelemetry | Native logging libraries into Opentelemetry |
| **Data archives** | Parquet ([dora-record](libraries/extensions/dora-record)) |
| **Visualization and annotation** | OpenCV | [rerun.io](rerun.io) |
| **Supported Platforms (x86)** | Windows, macOS, Linux |
| **Supported Platforms (ARM)** | macOS, Linux |
| **Configuration** | YAML |

## License

This project is licensed under Apache-2.0. Check out [NOTICE.md](NOTICE.md) for more information.

0 comments on commit c3d4cac

Please sign in to comment.