-
Notifications
You must be signed in to change notification settings - Fork 110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Nodes hub to store and reuse commonly used node #569
Conversation
I'm currently working on this PR on the latest version of Dora (0.3.5), so I bumped it from rc0 to release. Structure
Each python node respects the following structure:
The - id: object-detection
build: pip install ../../node-hub/ultralytics-yolo
path: ultralytics-yolo Usagecd examples/python-dataflow
dora build ./dataflow.yml (or dora build ./dataflow_dynamic.yml)
dora up
dora start ./dataflow.yml (or dora start ./dataflow_dynamic.yml) Note: if you're running the dynamic dataflow, you will need to start manually the ultralytics-yolo node: # activate your virtual environment in another terminal
python ultralytics-yolo --name object-detection --model yolov5n.pt |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great!
I made couple of small comments.
FYI, in a follow up PR we can look at: https://arrow.apache.org/docs/python/extending_types.html#custom-scalar-conversion
We can create a new project called something like pyarrow-extension-types
and define some standard format such as ImageRGB
, BboxXYXY
... So that, the abstraction of types is more solid than dictionary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
Sorry I looked more into details and found more things to update.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please squash the commits a bit before merging, we don't need all those "Fix CI" commits in our git history.
07229c4
to
6e508ec
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Everything is good for me. Maybe we should think about the ZeroCopy dictionary issue for a next PR (or this one), tell me your ideas when you are working on it 😄
.idea .idea README.md and better outputs/inputs README.md and ultralytics-yolo fix queue_size and dynamic README.md Fix CI Fix CI Fix CI Fix CI Fix CI Fix CI Fix CI Fix CI Fix CI
Co-authored-by: Philipp Oppermann <[email protected]>
fix typo
…encoding to support multiple encoding
…etadata as well as enable more flexibility in managing metadata
…inition to make our input more generalistic. Rewrite README documentation to reflect metadata changes
This PR creates a node hub to store commonly used nodes so that we can make it extremely easy for people to get started with dora.