Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get_all_executions_in_stage(stage) in time order #222

Open
atripathy86 opened this issue Dec 16, 2024 · 0 comments
Open

get_all_executions_in_stage(stage) in time order #222

atripathy86 opened this issue Dec 16, 2024 · 0 comments

Comments

@atripathy86
Copy link
Contributor

When running cmflib/commands/artifact/pull.py , we are fetching all executions of a stage for a pipeline. Is there a guarantee that the executions returned will be in time order. Perhaps we should introduce an option to return a time ordered list. For example:

get_all_executions_in_stage(stage, ordered=True)

This becomes important if there are multiple executions (which have produced a given artifact) and the latest one needs to be pulled/returned. At this point all are returned, but which is the one that will be written to the remote filesystem? If there are multiple file hashes corresponding to a named file, its a matter of chance, what is pulled last. (Without time-ordering)

From pull.py run()

executions = query.get_all_executions_in_stage(stage)
            # check if stage has executions
            if len(executions) > 0:
                 # converting it to dictionary
                dict_executions = executions.to_dict("dict")
                # append id's of executions inside identifiers
                for id in dict_executions["id"].values():
                    identifiers.append(id)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant