Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add filter to combine pipeline results into a single multi-domain dataset #1417

Open
cyrush opened this issue Nov 6, 2024 · 0 comments
Open

Comments

@cyrush
Copy link
Member

cyrush commented Nov 6, 2024

interface strawman:

-
  action: add_pipelines
    p1:
        f1:
           ... my filter, translate dataset, etc
    p2: 
        f1:
           ... my filter, translate dataset, etc
    my_pipeline:
         pipelines: 
              - p1
              - p2
         my_filter:
               type: "combine_pipelines"

Considerations:

  • Underlying datasets can be zero copied.
  • We will have duplicate domain ids, so domains will need to be renumbered.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant