Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Allow modules to run only on specific worker #11845

Closed
wants to merge 3 commits into from

Conversation

dklimpel
Copy link
Contributor

Fixes: #10701

This is not the requested solution of #10701
I am not sure if a module should decide independently where it runs.

This makes it possible to configure the workers at a central location (homerserver.yaml).
I am not sure if a config value is needed to run a module on all servers with the same configuration.

Pull Request Checklist

  • Pull request is based on the develop branch
  • Pull request includes a changelog file. The entry should:
    • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
    • Use markdown where necessary, mostly for code blocks.
    • End with either a period (.) or an exclamation mark (!).
    • Start with a capital letter.
    • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
  • Pull request includes a sign off
  • Code style is correct
    (run the linters)

Signed-off-by: Dirk Klimpel [email protected]

@dklimpel dklimpel requested a review from a team as a code owner January 27, 2022 14:13
@babolivier
Copy link
Contributor

babolivier commented Jan 27, 2022

I am not sure if a module should decide independently where it runs.

This is kind of already what we do with ModuleApi.looping_background_call, where we allow the module to decide whether to run its background process on all of the processes or just the one in charge of background jobs.

Personally I'm more keen to delegate the logic of figuring out what to run on which worker. The module interface rewrite was in part motivated by the fact that the old module system would force you to write split your module if you wanted part of it to implement interface X and part of it to implement interface Y (which meant the only way to implement both at once was to get really hacky, e.g. sharing state via global variables, etc).
It feels like going backwards to me to force module developers to split their module just because they don't want to run method A and method B on the same worker. I think the modules themselves should have the possibility to decide which worker each functionality runs on, just like they have the possibility to decide which callback categories they want to implement.

@anoadragon453
Copy link
Member

anoadragon453 commented Jan 27, 2022

Additionally, this solution as it stands only allows for running a module on a single worker, whereas there's often cases where modules should run across a set of multiple workers.

@dklimpel
Copy link
Contributor Author

Additionally, this solution as it stands only allows for running a module on a single worker, whereas there's often cases where modules should run across a set of multiple workers.

IMO you can run with this solution a module on multiple workers:
e.g.:

modules:
  - module: my_super_module.MySuperClass
    worker_name: Worker1
  - module: my_super_module.MySuperClass
    worker_name: Worker2

There were a chance to improve this solution:

  • set worker_name to None run on all workers incl. main
  • set worker_name to main run only on main

@babolivier babolivier closed this Jan 27, 2022
@babolivier babolivier reopened this Jan 27, 2022
@babolivier
Copy link
Contributor

(sorry, I missclicked)

@dklimpel
Copy link
Contributor Author

I close the PR and wait for another solution. If this kind of solution has a chance, I can reopen it.

@dklimpel dklimpel closed this Jan 27, 2022
@dklimpel dklimpel deleted the modules_worker branch January 28, 2022 08:11
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow modules to run only on specific worker(s)
3 participants