Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow custom run_id #18

Closed
waleedsamy opened this issue Apr 29, 2021 · 2 comments
Closed

allow custom run_id #18

waleedsamy opened this issue Apr 29, 2021 · 2 comments

Comments

@waleedsamy
Copy link

waleedsamy commented Apr 29, 2021

With apache/airflow#6317, TriggerDagRunOperator changed to drop the use of DagRunOrder and you thankfully adapted their way. However I think we are following bad decision by allowing TriggerMultiDagRunOperator to use unidentifiable run_id

I'd like to discuss the possibility of restoring the old behavior where TriggerMultiDagRunOperator#python_callable can return DagRunOrder, so a user can provide custom+dynamic run_id.

In my case, I use TriggerMultiDagRunOperator to trigger thousands of dagruns and I assign for every one of those an identifiable run_id that can distinguish every dagrun from the other. surrendering indefinable run_id with gabbrish one is not wise at all.
WDYT?

@waleedsamy waleedsamy changed the title alow custom run_id allow custom run_id Apr 29, 2021
@waleedsamy
Copy link
Author

Created an airflow issue apache/airflow#17438

@mastak
Copy link
Owner

mastak commented Aug 17, 2021

Hi @waleedsamy
Added it in the new version 2.2.0. Here you can find an example:
https://github.com/mastak/airflow_multi_dagrun/blob/v2.2.0/examples/trigger_with_custom_trigger_id.py#L12

Be careful, run_id has to be unique.

@mastak mastak closed this as completed Aug 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants