Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix collection state updates by setting collection update time #11849

Merged
merged 3 commits into from
Apr 19, 2021
Merged

Fix collection state updates by setting collection update time #11849

merged 3 commits into from
Apr 19, 2021

Conversation

mvdbeek
Copy link
Member

@mvdbeek mvdbeek commented Apr 15, 2021 •
edited
Loading

What did you do?

  • Restore update_time query dropped in 33883fe, hopefully now without deadlocks. The new query locks all rows to update using WITH FOR UPDATE SKIP LOCKED, if possible, and else uses SERIALIZABLE isolation level.
  • Apply update_time filter against HistoryDatasetCollectionAssociation instead of DatasetCollection

Why did you make this change?

Needed for updating the collection state in the new history panel (#11824).

How to test the changes?

(select the most appropriate option; if the latter, provide steps for testing below)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    Run a collection job in the new history panel and see that the collection eventually turns green.

License

  • I agree to license these contributions under Galaxy's current license.
  • I agree to allow the Galaxy committers to license these and all my past contributions to the core galaxy codebase under the MIT license. If this condition is an issue, uncheck and just let us know why with an e-mail to [email protected].

For UI Components

  • I've included a screenshot of the changes

@mvdbeek
Restore setting hdca.update_time
44b988c 
and fix the subcontainer query to filter the update time on the hdca,
not on the DatasetCollection. This is necessary for HDCA state changes
in the new history panel.
Using the `with_for_update` should work around the observed deadlocks,
and `skip_locked` should prevent any slowdown when many processes
finish jobs belonging to the same implicit collection.
@mvdbeek mvdbeek force-pushed the restore_update_time_query branch from e849ccc to ec50ade Compare April 15, 2021 19:54
@Nerdinacan Nerdinacan mentioned this pull request Apr 15, 2021
Merged
6 tasks
@mvdbeek mvdbeek changed the title Restore setting hdca.update_time Fix collection state updates by setting collection update time Apr 16, 2021
@mvdbeek mvdbeek marked this pull request as ready for review April 16, 2021 11:41
@github-actions github-actions bot added this to the 21.05 milestone Apr 16, 2021
@mvdbeek mvdbeek force-pushed the restore_update_time_query branch 2 times, most recently from 8e567c0 to c1377f4 Compare April 16, 2021 13:30
@mvdbeek mvdbeek force-pushed the restore_update_time_query branch from c1377f4 to 9c379cd Compare April 16, 2021 14:28
@mvdbeek mvdbeek mentioned this pull request Apr 16, 2021
Closed
12 tasks
@mvdbeek mvdbeek requested a review from jmchilton April 16, 2021 16:43
@jmchilton jmchilton merged commit d50579d into galaxyproject:dev Apr 19, 2021
@jmchilton
Copy link
Member

Nice hand-crafted queries. Seems about as clean as we can hope for I think. I hope it works!

@nsoranzo nsoranzo deleted the restore_update_time_query branch April 19, 2021 19:13
@mvdbeek mvdbeek mentioned this pull request Jun 15, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jmchilton jmchilton Awaiting requested review from jmchilton

Assignees
No one assigned
Labels
area/jobs area/testing/api area/testing kind/bug kind/enhancement
Projects
None yet
Milestone
21.05
Development

Successfully merging this pull request may close these issues.
2 participants
@mvdbeek @jmchilton