Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SAP: Introduce external scheduler filter #502

Merged
merged 1 commit into from
Jan 20, 2025
Merged

SAP: Introduce external scheduler filter #502

merged 1 commit into from
Jan 20, 2025

Conversation

auhlig
Copy link
Member

@auhlig auhlig commented Jul 9, 2024 •
edited by PhilippMatthes
Loading

SAP: Introduce an external scheduler filter

Provides support for an external service that can perform advanced scheduling
decisions. SchedulerManager now calls a function call_external_scheduler_api
after executing filters and weighers. This function sends the hosts and their
current weights to an external scheduling service. This service then performs
advanced scheduling decisions and returns the sorted list of host names, from
highest priority to lowest priority. To toggle this feature, provide
external_scheduler_api_url to Nova's configuration.

@auhlig auhlig requested a review from fwiesel July 9, 2024 13:45
joker-at-work
joker-at-work requested changes Jul 10, 2024
View reviewed changes
Copy link

@joker-at-work joker-at-work left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

implement using BaseFilter

I think it should be documented (at least in the commit message), why we do not use the usual approach of BaseHostFilter.

nova/scheduler/filters/external_scheduler_filter.py Outdated Show resolved Hide resolved
nova/scheduler/filters/external_scheduler_filter.py Outdated Show resolved Hide resolved
nova/scheduler/filters/external_scheduler_filter.py Outdated Show resolved Hide resolved
nova/scheduler/filters/external_scheduler_filter.py Outdated Show resolved Hide resolved
@auhlig auhlig force-pushed the feat/external-scheduler-filter branch from 0ee9d1d to 8fc7c36 Compare July 11, 2024 10:01
fwiesel
fwiesel requested changes Jul 11, 2024
View reviewed changes
Copy link
Member

@fwiesel fwiesel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also rewrite the history to a single commit please?

@auhlig auhlig force-pushed the feat/external-scheduler-filter branch from c897817 to bedc095 Compare July 11, 2024 11:23
@fwiesel
Copy link
Member

fwiesel commented Jul 11, 2024

Sorry for the piece-meal: Could you also please prefix the commit title with "SAP: "?

@auhlig auhlig force-pushed the feat/external-scheduler-filter branch from bedc095 to 0ea42f9 Compare July 11, 2024 11:29
@fwiesel
Copy link
Member

fwiesel commented Jul 11, 2024

And maybe a slightly more elaborate commit-message as @joker-at-work already asked for. I'd suggest:

SAP: Introduce an external scheduler filter

This allows us to rely on an external http service to do the filtering,
on a potentially more complex rule-set.
Instead of filtering each host one-by-one, we want to filter them in
a single requests and therefor override the `BaseFilter.filter_all`

@auhlig auhlig force-pushed the feat/external-scheduler-filter branch 3 times, most recently from a94f213 to ec91341 Compare July 11, 2024 13:12
@auhlig auhlig changed the title Introduce external scheduler filter SAP: Introduce external scheduler filter Jul 11, 2024
@auhlig auhlig force-pushed the feat/external-scheduler-filter branch 2 times, most recently from 43eaccc to b52d5c7 Compare July 11, 2024 14:18
@auhlig auhlig force-pushed the feat/external-scheduler-filter branch from b52d5c7 to 0b35204 Compare July 11, 2024 16:07
fwiesel
fwiesel previously approved these changes Jul 25, 2024
View reviewed changes
Copy link
Member

@fwiesel fwiesel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I still would prefer the oslo_versionedobject serialization, but I am okay with this change.

@fwiesel fwiesel requested a review from joker-at-work July 25, 2024 07:39
@auhlig
Copy link
Member Author

auhlig commented Jul 25, 2024

Personally, I still would prefer the oslo_versionedobject serialization, but I am okay with this change.

I tried that. However, the nested python structures aren't serialized correctly. Thus I implemented the current solution.

joker-at-work
joker-at-work previously requested changes Aug 21, 2024
View reviewed changes
nova/conf/scheduler.py Outdated Show resolved Hide resolved
nova/scheduler/filters/external_scheduler_filter.py Outdated Show resolved Hide resolved
nova/scheduler/filters/external_scheduler_filter.py Outdated Show resolved Hide resolved
nova/scheduler/filters/external_scheduler_filter.py Outdated Show resolved Hide resolved
nova/tests/unit/scheduler/filters/test_external_scheduler_filter.py Outdated Show resolved Hide resolved
nova/tests/unit/scheduler/filters/test_external_scheduler_filter.py Outdated Show resolved Hide resolved
@PhilippMatthes PhilippMatthes self-assigned this Jan 6, 2025
@PhilippMatthes PhilippMatthes force-pushed the feat/external-scheduler-filter branch from 27311c3 to 384d788 Compare January 8, 2025 13:56
@PhilippMatthes PhilippMatthes dismissed stale reviews from joker-at-work and fwiesel January 8, 2025 15:45

Review comments were addressed

@fwiesel
Copy link
Member

fwiesel commented Jan 16, 2025

@PhilippMatthes Could you squash the history and force-push it please? And update the PR that way each time?
I'd like to avoid having a historic changelog but rather a logical one.

@PhilippMatthes PhilippMatthes force-pushed the feat/external-scheduler-filter branch from d73d9c1 to d53c934 Compare January 17, 2025 08:23
@PhilippMatthes
Copy link
Member

@fwiesel sure, I squashed the commits in d53c934

@auhlig
Copy link
Member Author

auhlig commented Jan 17, 2025

Thanks for taking this further @fwiesel, @joker-at-work, @PhilippMatthes! ❤️

Does the PR look reasonably good to you from Nova perspective to get it merged and deployed to QA?
It's currently a major blocker for us and we'd like to make some progress on our initial placement and scheduling side.

joker-at-work
joker-at-work reviewed Jan 20, 2025
View reviewed changes
Copy link

@joker-at-work joker-at-work left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think that my comments are blockers for this PR. They could happen in a follow-up, if you need this to merge fast. Please have a look and tell me whether to approve now.

nova/scheduler/external.py Show resolved Hide resolved
nova/scheduler/external.py Outdated Show resolved Hide resolved
nova/scheduler/external.py Outdated Show resolved Hide resolved
@PhilippMatthes PhilippMatthes force-pushed the feat/external-scheduler-filter branch 2 times, most recently from 43f07a5 to 4819887 Compare January 20, 2025 11:12
joker-at-work
joker-at-work reviewed Jan 20, 2025
View reviewed changes
nova/scheduler/external.py Outdated Show resolved Hide resolved
nova/scheduler/external.py Outdated Show resolved Hide resolved
@PhilippMatthes
SAP: Introduce external scheduler filter
f76efba 
Provides support for an external service that can perform advanced
scheduling decisions.

SchedulerManager now calls a function call_external_scheduler_api
after executing filters and weighers. This function sends the
hosts and their current weights to an external scheduling service.
This service then performs advanced scheduling decisions and
returns the sorted list of host names, from highest priority to
lowest priority. To toggle this feature, provide
external_scheduler_api_url to Nova's configuration.
@PhilippMatthes PhilippMatthes force-pushed the feat/external-scheduler-filter branch from 4819887 to f76efba Compare January 20, 2025 13:33
@PhilippMatthes PhilippMatthes merged commit cdc6aa3 into stable/xena-m3 Jan 20, 2025
2 checks passed
@PhilippMatthes PhilippMatthes deleted the feat/external-scheduler-filter branch January 20, 2025 13:47
@PhilippMatthes PhilippMatthes mentioned this pull request Jan 24, 2025
Merged
joker-at-work pushed a commit that referenced this pull request Feb 4, 2025
@auhlig @PhilippMatthes @joker-at-work
SAP: Introduce external scheduler filter (#502)
521e94e 
Provides support for an external service that can perform advanced
scheduling decisions.

SchedulerManager now calls a function call_external_scheduler_api
after executing filters and weighers. This function sends the
hosts and their current weights to an external scheduling service.
This service then performs advanced scheduling decisions and
returns the sorted list of host names, from highest priority to
lowest priority. To toggle this feature, provide
external_scheduler_api_url to Nova's configuration.

Change-Id: Ibf7d3238c77fa73383ae7d0368a03cc3044409f8
Co-authored-by: PhilippMatthes <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PhilippMatthes PhilippMatthes PhilippMatthes left review comments

@fwiesel fwiesel fwiesel approved these changes

@joker-at-work joker-at-work joker-at-work approved these changes

Assignees

@PhilippMatthes PhilippMatthes

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@auhlig @fwiesel @PhilippMatthes @joker-at-work