Dora v0.3.8

Breaking Change

Inputs are now schedule fairly meaning that they will be now be received equally and not necessarily in chronological order. This enables to always be able to refresh input with the least latency between input.

Hardwares

• CUDA zero copy feature
• Intel Realsense
• Agilex Piper ams
• Astra Orbbec Cameras

Models

• Adding support for RoboticDiffusionTransformers model

Scheduler

• Made the node scheduler fair between input and respect queue size compared to previously where the queue size was not respected

Bug Fix

• Fix MacOS not using the right python version
• Fix python not flushing out print in python by removing buffering function.

v0.3.8 (2024-12-06)

• Make node hub CI/CD cross platform by @haixuanTao in #714
• Make node hub CI/CD cross architecture by @haixuanTao in #716
• Make list an available type for metadata by @haixuanTao in #721
• Add stdout logging by @haixuanTao in #720
• Add an error when a node fails when using dora run by @haixuanTao in #719
• Add pyarrow cuda zero copy helper by @haixuanTao in #722
• feat: Add Dora-kit car Control in node-hub by @LyonRust in #715
• Add yuv420 encoding to opencv-video-capture by @haixuanTao in #725
• Change macOS CI runner to macos-13 by @phil-opp in #729
• Add eyre to pyo3 node by @haixuanTao in #730
• Moving queue size and making node flume queue bigger by @haixuanTao in #724
• Make python default for macos by @haixuanTao in #731
• Modify the node queue Scheduler to make it able to schedule input fairly by @haixuanTao in #728

Full Changelog: v0.3.7...v0.3.8

0.3.8-rc

Bump to 0.3.8-rc version

Dora v0.3.7

Making Dora simpler

• dora run command now removes the need for dora up and removes the need for background process: #703

dora run examples/camera/dataflow.yml

• You can now build, run and start URL dataflow. #682

dora build/start https://raw.githubusercontent.com/dora-rs/dora/refs/heads/main/examples/camera/dataflow.yml

• You can now pip install the dora cli for Linux, as well as all rust written nodes. #706

pip install dora-rs-cli
pip install dora-rerun

Releasing additional nodes

• Vision Language Model (VLM): Qwenvl2 with data recording and finetuning with llama factory

dora build examples/vlm/dataflow.yml
dora run examples/vlm/dataflow.yml

Screencast.from.2024-11-18.04-23-10.webm

• Audio: Whisper Turbo, Sidero Voice Activity Detection(VAD), Opus translation

speech-to-text example:

dora build examples/speech-to-text/dataflow.yml
dora run examples/speech-to-text/dataflow.yml

Screencast.from.2024-11-18.04-28-33.webm

translation example:

dora build examples/translation/dataflow_en_fr.yml
dora run examples/translation/dataflow_en_fr.yml

• visualisation: Dora rerun with 3D visualisation. Time series and now in pip.

dora build examples/piper/play_dummy_inference.yml
dora run examples/piper/play_dummy_inference.yml

Screencast.from.2024-11-18.05-30-19.webm

Please checkout piper examples.

• Hardware: Agilex Piper sdk, Astra camera with PyOrbbeckSdk and tracer mobile base

Known Issue still present in 0.3.7

• Python dataflow does not stop gracefully #625
• Queue_size is not properly respected #652

What's Changed

• Post release 0.3.6 small fix by @haixuanTao in #638
• Changes to template by @XxChang in #639
• Add appending to PATH instruction inside installation script by @Hennzau in #641
• Make the benchmark run in release and at full speed by @Hennzau in #644
• Use the new node syntax for examples dataflow by @Hennzau in #643
• Improve beginner experience by @Hennzau in #645
• improve node-hub pytest by @haixuanTao in #640
• Fix not-null terminated string print within C template by @haixuanTao in #654
• Raise error if dora-coordinator is not connected when calling dora destroy by @haixuanTao in #655
• Coordinator stopped on bad control command by @Hennzau in #650
• Add support for Qwenvl2 by @haixuanTao in #646
• Fix distributed node by @haixuanTao in #658
• Small install script update for bash by @haixuanTao in #657
• Add additional image encoding by @haixuanTao in #661
• dora-echo replicate the topic received with the topic send by @haixuanTao in #663
• Update dependencies by @renovate in #656
• Bump pyo3 and arrow versions by @haixuanTao in #667
• Fix ros2 bridge incompatibility with CI Ubuntu 24 and with pyo3 22 by @haixuanTao in #670
• Add transformers version pinning for qwenvl2 by @haixuanTao in #665
• Remove cli dataflow path check by @haixuanTao in #662
• Better error handling for unknown output by @haixuanTao in #675
• Fix llama recorder multi image recorder by @haixuanTao in #677
• Dora openai server example by @haixuanTao in #676
• Update dependencies by @renovate in #674
• Create Rust-based openai api proxy server in node hub by @phil-opp in #678
• Update dependencies by @renovate in #679
• Update Rust crate hyper to v0.14.30 by @renovate in #680
• Fix hanged coordinator when failing to connect to the daemon on destroy command by @haixuanTao in #664
• Small example improvement using pyarrow assertion by @haixuanTao in #669
• Fix dora list listing twice a stopping dataflow when using multiple daemon. by @haixuanTao in #668
• Add package flake by @Ben-PH in #685
• Add jpeg format to qwenvl2 by @haixuanTao in #684
• Enable downloading remote dataflow by @haixuanTao in #682
• Enable multiline build for better packaging of dora node. by @haixuanTao in #683
• Bump rerun version to 0.18 by @haixuanTao in #686
• Temporary fix qwenvl2 queue error by @haixuanTao in #688
• Make daemon loop over coordinator connection to make it possible to create a system service awaiting coordinator connection by @haixuanTao in #689
• Add translation example from chinese, french to english by @haixuanTao in #681
• Update dependencies by @renovate in #690
• Fix macos 14 yolo error by @haixuanTao in #696
• Update dependencies by @renovate in #692
• Publish rust project on pip to make it simpler to deploy dora node on different machine without requiring installing cargo by @haixuanTao in #695
• Docs: README by @Radovenchyk in #697
• Update README.md by @pucedoteth in #705
• Bump rust toolchains 1.81 by @haixuanTao in #707
• Make dora cli pip installable by @haixuanTao in #706
• Add urdf visualization in rerun by @haixuanTao in #704
• Fix child process receiving ctrl-c by setting own process group by @haixuanTao in #712
• Move more types from dora-core to dora-message to avoid dependency by @phil-opp in #711
• Implement dora run command by @phil-opp in #703
• Adding Agilex Piper node, PyOrbbeckSDK node, Agilex UGV node by @haixuanTao in #709
• Make the node hub CI/CD parallel for faster testing as well as having more granular integration control by @haixuanTao in #710
• Add time series to dora rerun by @haixuanTao in #713

New Contributors

• @Ben-PH made their first contribution in #685
• @Radovenchyk made their first contribution in #697
• @pucedoteth made their first contribution in #705

v0.3.7rc2

Fix piper version

v0.3.7rc1

Fix CI/CD for pip release

v0.3.7rc0

What's Changed

• Post release 0.3.6 small fix by @haixuanTao in #638
• Changes to template by @XxChang in #639
• Add appending to PATH instruction inside installation script by @Hennzau in #641
• Make the benchmark run in release and at full speed by @Hennzau in #644
• Use the new node syntax for examples dataflow by @Hennzau in #643
• Improve beginner experience by @Hennzau in #645
• improve node-hub pytest by @haixuanTao in #640
• Fix not-null terminated string print within C template by @haixuanTao in #654
• Raise error if dora-coordinator is not connected when calling dora destroy by @haixuanTao in #655
• Coordinator stopped on bad control command by @Hennzau in #650
• Add support for Qwenvl2 by @haixuanTao in #646
• Fix distributed node by @haixuanTao in #658
• Small install script update for bash by @haixuanTao in #657
• Add additional image encoding by @haixuanTao in #661
• dora-echo replicate the topic received with the topic send by @haixuanTao in #663
• Update dependencies by @renovate in #656
• Bump pyo3 and arrow versions by @haixuanTao in #667
• Fix ros2 bridge incompatibility with CI Ubuntu 24 and with pyo3 22 by @haixuanTao in #670
• Add transformers version pinning for qwenvl2 by @haixuanTao in #665
• Remove cli dataflow path check by @haixuanTao in #662
• Better error handling for unknown output by @haixuanTao in #675
• Fix llama recorder multi image recorder by @haixuanTao in #677
• Dora openai server example by @haixuanTao in #676
• Update dependencies by @renovate in #674
• Create Rust-based openai api proxy server in node hub by @phil-opp in #678
• Update dependencies by @renovate in #679
• Update Rust crate hyper to v0.14.30 by @renovate in #680
• Fix hanged coordinator when failing to connect to the daemon on destroy command by @haixuanTao in #664
• Small example improvement using pyarrow assertion by @haixuanTao in #669
• Fix dora list listing twice a stopping dataflow when using multiple daemon. by @haixuanTao in #668
• Add package flake by @Ben-PH in #685
• Add jpeg format to qwenvl2 by @haixuanTao in #684
• Enable downloading remote dataflow by @haixuanTao in #682
• Enable multiline build for better packaging of dora node. by @haixuanTao in #683
• Bump rerun version to 0.18 by @haixuanTao in #686
• Temporary fix qwenvl2 queue error by @haixuanTao in #688
• Make daemon loop over coordinator connection to make it possible to create a system service awaiting coordinator connection by @haixuanTao in #689
• Add translation example from chinese, french to english by @haixuanTao in #681

New Contributors

• @Ben-PH made their first contribution in #685

Full Changelog: v0.3.6...v0.3.7rc0

v0.3.6

Dora v0.3.6 Release

New Website Frontpage!

We've been doing a lot of demo at dora, and it was about time to put them in front so that people get a better understanding of what dora-rs stand for

https://dora-rs.ai/

Preview

Quicker installer!

As we're adding support for more platform such as linux arm musl, it was time to make installation simpler. So we copied other project installation script so that people can install dora in one line:

curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/dora-rs/dora/main/install.sh | bash

Shipping the Node-Hub!

We have released a node-hub, that should make it a lot easier to get started with dora.

The idea is that, we will be able to directly pull pre-packaged node from pip and cargo and use them in our own project.

For example to get started with a video stream, instead of writing code or copying code from the dora project you can just write:

nodes:
  - id: camera
    build: pip install opencv-video-capture
    path: opencv-video-capture
    inputs:
      tick: dora/timer/millis/20
    outputs:
      - image
    env:
      CAPTURE_PATH: 0
      IMAGE_WIDTH: 640
      IMAGE_HEIGHT: 480

  - id: plot
    build: pip install opencv-plot
    path: opencv-plot
    inputs:
      image:
        source: camera/image
        queue_size: 1

Then, use:

dora build dataflow.yml
dora up
dora start dataflow.yml

This should reduce the necessity to copy paste code from one place to another as well as reduce the risk of path issues.

You can copy our CI/CD github worklfow : https://github.com/dora-rs/dora/blob/main/.github/workflows/node-hub-ci-cd.yml so that your nodes within your node-hub folder are tested and published on cargo and pip.

Dora Node should now be compatible between version from 0.3.6 forward!

Thanks to @phil-opp, we know have an independant dora-message crate that will be responsible for the communication of messages, and should be able to handle dora nodes from multiple version as long as dora-message version are compatible.

C++ Documentation :)

Thanks to @starlitxiling, we now have a first experimental C++ documentation to make it easier for people to get started in C++

https://dora-rs.ai/cpp/cpp-api.html

Preview

Fix dynamic node initialisation issues

We have noticed that dora sometimes had issues with initializing dynamic node and this release should fix this issue.

What's Changed

• Update dependencies by @renovate in #579
• Don't wait for non-started dynamic nodes on stop by @phil-opp in #583
• add a comment on read_dora_input_id by @XxChang in #580
• Update dependencies by @renovate in #584
• Update dependencies by @renovate in #585
• Add domain unix socket supports by @XxChang in #594
• Check build for cross-compiled targets on CI by @phil-opp in #597
• Test pip release creation as part of normal CI by @phil-opp in #596
• Add-armv7-musleabihf-prebuilt-release by @haixuanTao in #578
• Update dependencies by @renovate in #602
• Delay dropping of DoraNode in Python until all event data is freed by @phil-opp in #601
• Add install script by @haixuanTao in #600
• Nodes hub to store and reuse commonly used node by @haixuanTao in #569
• Ros2-bridge action attempt by @starlitxiling in #567
• Update dependencies by @renovate in #605
• Add a CI/CD for the node-hub by @haixuanTao in #604
• Update dependencies by @renovate in #608
• Remove dynamic node from pending nodes before starting a dataflow by @haixuanTao in #606
• Fix alignment of atomics in shared memory communication channel by @phil-opp in #612
• Update dependencies by @renovate in #622
• Refactor: Move message definitions to dora-message crate by @phil-opp in #613
• Update README.md by @heyong4725 in #623
• Update Rust crate serde to v1.0.207 by @renovate in #624
• fix clippy warnings by @Michael-J-Ward in #626
• Update dependencies by @renovate in #629
• Update dependencies by @renovate in #630
• Update Rust crate reqwest to v0.12.7 by @renovate in #632
• Fix typos by @omahs in #633
• Fix errors reported in #628 by @phil-opp in #631
• Update dependencies by @renovate in #636
• Make dora versions compatible by using separate versioning for dora-message crate by @phil-opp in #614
• Adding additional node in the node-hub by @haixuanTao in #620
• Bump version v0.3.6 by @haixuanTao in #637

New Contributors

• @starlitxiling made their first contribution in #567
• @heyong4725 made their first contribution in #623
• @omahs made their first contribution in #633

Full Changelog: v0.3.5...v0.3.6

v0.3.6-rc0

What's Changed

• Update dependencies by @renovate in #579
• Don't wait for non-started dynamic nodes on stop by @phil-opp in #583
• add a comment on read_dora_input_id by @XxChang in #580
• Update dependencies by @renovate in #584
• Update dependencies by @renovate in #585
• Add domain unix socket supports by @XxChang in #594
• Check build for cross-compiled targets on CI by @phil-opp in #597
• Test pip release creation as part of normal CI by @phil-opp in #596
• Add-armv7-musleabihf-prebuilt-release by @haixuanTao in #578
• Update dependencies by @renovate in #602
• Delay dropping of DoraNode in Python until all event data is freed by @phil-opp in #601
• Add install script by @haixuanTao in #600
• Nodes hub to store and reuse commonly used node by @haixuanTao in #569
• Ros2-bridge action attempt by @starlitxiling in #567
• Update dependencies by @renovate in #605
• Add a CI/CD for the node-hub by @haixuanTao in #604
• Update dependencies by @renovate in #608
• Remove dynamic node from pending nodes before starting a dataflow by @haixuanTao in #606
• Fix alignment of atomics in shared memory communication channel by @phil-opp in #612
• Update dependencies by @renovate in #622
• Refactor: Move message definitions to dora-message crate by @phil-opp in #613
• Update README.md by @heyong4725 in #623
• Update Rust crate serde to v1.0.207 by @renovate in #624
• fix clippy warnings by @Michael-J-Ward in #626

New Contributors

• @starlitxiling made their first contribution in #567
• @heyong4725 made their first contribution in #623

Full Changelog: v0.3.5-fix7...v0.3.5.6-rc0

v0.3.5-fix7

Add installl shell

v0.3.5-fix6

Zip only binary