Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send HighestVersion attribute to Inflator #102

Merged
merged 2 commits into from
Nov 28, 2019

Conversation

HebaruSan
Copy link
Member

@HebaruSan HebaruSan commented Nov 25, 2019

Motivation

Now that #101 has given us the ability to compare module versions in Python the same as in CKAN-Core, this is the Scheduler + Webhook part of KSP-CKAN/CKAN#2824, which aims to reduce the need for manual maintenance of x_netkan_epoch properties.

Changes

A new CkanGroup class represents all of a given module's .ckan files in a CKAN-meta repo. This class provides a highest_version function that parses the .ckan files and compares their version properties to find the highest one, and returns it.

The Ckan.sqs_message function now accepts a CkanGroup parameter and passes it to a new sqs_message_attribs function to generate the MessageAttributes property of the message. Naturally, this uses CkanGroup.highest_version to set HighestVersion.

If x_netkan_allow_out_of_order is set to true in a netkan, this means auto-epoching is turned off, so the HighestVersion property will not be set and its .ckan files will not be parsed.

The Scheduler now checks out CKAN-meta and uses it to pass CkanGroup objects to Ckan.sqs_message. The Webhooks also pass CkanGroup objects to Ckan.sqs_message.

Fixes #7.

@HebaruSan HebaruSan added Enhancement New feature or request Webhooks Listens for notifications and triggers actions via queues Scheduler Adds netkans to the queue to be inflated labels Nov 25, 2019
@HebaruSan HebaruSan requested a review from techman83 November 25, 2019 02:33
@HebaruSan HebaruSan force-pushed the feature/send-highest-version branch from 7e45b46 to 38a1f58 Compare November 25, 2019 07:08
Copy link
Member

@techman83 techman83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This technically isn't related to the scheduling component, but what's this going to look like on the indexing side of things? I'll refresh myself with the other PR.

@HebaruSan
Copy link
Member Author

The Indexer will not need any changes for this; it's solely a Scheduler+Inflator change.

@techman83
Copy link
Member

Yep, after refreshing myself I realised we could re-utilise the staging logic without any changes to the indexer. Which is rather neat.

Copy link
Member

@techman83 techman83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome. Love your work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request Scheduler Adds netkans to the queue to be inflated Webhooks Listens for notifications and triggers actions via queues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Support HighestVersion attribute for auto epoching
2 participants