[Question] What happens if job dies between push commands?

[jernsting]

Taken from the processing scripts:

# push result file content first - does not need a lock, no interaction with Git
datalad push --to output-storage
# and the output branch next - needs a lock to prevent concurrency issues
flock --verbose $DSLOCKFILE git push outputstore

This makes totally sense, but what happens if the datalad push is successful and the job dies (e.g. due to time limit) before the git push command is executed?

[adswa]

Sorry for the late response! The data will be pushed and in the output store, but there won't be any Git information about it or its generation, and also no branch that could be merged in the end. As the latter push is at least as important, it wouldn't be good if the job dies in between, the data would essentially be dangling disconnected. However, unless the lock takes unusually long to acquire, the git push operation is typically done very fast afterwards, as it also shouldn't take much time to complete