Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make SpecCluster declarative #4606

Open
jacobtomlinson opened this issue Mar 18, 2021 · 0 comments
Open

Make SpecCluster declarative #4606

jacobtomlinson opened this issue Mar 18, 2021 · 0 comments

Comments

@jacobtomlinson
Copy link
Member

SpecCluster and other components within distributed like distributed.cli.dask_spec aim to create clusters and workers according to a specification.

Currently these operations are performed imperatively. The SpecCluster class is given a spec and a resource is created to match the spec.

I would like to move things towards a declarative approach instead. This would mean the class is given a spec and it ensures that a resource exists to match that spec. If one doesn't exist it will create it, same as today. But if it already exists it will do nothing.

This slight difference of ensuring a resource instead of creating one will enable much more flexibility when it comes to recreating objects to represent existing resources. This ties into the lifecycle work I've been doing as part of #4263.

Opening this issue to signal my intent to work on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant