Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add Support for Custom Map Indexing with map_index_template #302

Closed
su-four opened this issue Dec 1, 2024 · 0 comments · Fixed by #314
Closed

[Feature] Add Support for Custom Map Indexing with map_index_template #302

su-four opened this issue Dec 1, 2024 · 0 comments · Fixed by #314
Assignees
Labels
enhancement New feature or request

Comments

@su-four
Copy link

su-four commented Dec 1, 2024

Description

Airflow provides the ability to override the default integer index for mapped tasks in the UI by assigning a custom name based on the task's input. This is achieved using the map_index_template parameter, which allows a Jinja template to dynamically generate the index based on the expanded task's inputs.

.expand(my_param=["value1", "value2", "value3"], map_index_template="{{ my_param }}")  

In this configuration, the task instances would appear in the UI with names based on the my_param values (e.g., value1, value2, value3) instead of default numerical indices (0, 1, 2).(how it is done in airflow)

Use case/motivation

Add support in dag-factory to define and utilize map_index_template in the YAML configuration. This would allow users to customize the indexing of mapped tasks for better readability and debugging in the Airflow UI.

Benefits:

  1. Improves clarity and debugging by displaying meaningful indices in the Airflow UI.
  2. Aligns dag-factory with Airflow's native features for dynamic task mapping.
  3. Enhances usability for workflows where task parameters carry significant context.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants