Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expose endpoint to control provisioning #426

Closed
Tracked by #300
morgsmccauley opened this issue Nov 22, 2023 · 2 comments
Closed
Tracked by #300

Expose endpoint to control provisioning #426

morgsmccauley opened this issue Nov 22, 2023 · 2 comments
Labels
component: Runner

Comments

@morgsmccauley
Copy link
Collaborator

morgsmccauley commented Nov 22, 2023 •
edited
Loading

We want to be able to control provisioning from within Coordinator. As provisioning currently exists within Runner, we should expose endpoints to control provisioning of Indexers:

  • is indexer provisioned
  • provision indexer
@morgsmccauley morgsmccauley mentioned this issue Nov 22, 2023
Closed
@morgsmccauley
Copy link
Collaborator Author

Having a separate endpoint will provide more control over this process - but as a first stage it may be better to just integrate this in to the executors endpoint, i.e. setting provision: true when starting an Executor will trigger a provisioning cycle

This was referenced Jan 8, 2024
Merged
Closed
morgsmccauley added a commit that referenced this issue Jan 11, 2024
@morgsmccauley
feat: Create initial Coordinator V2 service (#444)
b76fac0 
This PR adds the initial Coordinator V2 service, which acts as the main
driver for the Control Plane.

## Coordinator Overview
Coordinator V2 will exist as a new standalone service with it's primary
goal being ensuring the current registry configuration is mapped to the
system. It's core logic is just an infinite loop which reads the
registry, and sends necessary requests to the Block Streamer and Runner
services to synchronise that config.

## Block Streamer Changes
Some changes have been made to Block Streamer to enable the above:
1. `version` and `redis_stream` have been added to the proto so that
Coordinator can configure them.
2. Support for `ActionFunctionCall` has been added - Initially I thought
only `ActionAny` allowed, but the current registry has
`ActionFunctionCall` and therefore needs to be supported.
3. `last_published_block` is now written to Redis to enable "Start from
interruption"

## What's not been done
I wanted to limit the scope of this PR as it was starting to get big,
I'll address these tasks in follow up PRs:
- Provisioning - Coordinator should check the status of provisioning,
and act when the state isn't as expected. This can be done after
#426 is implemented.
- Retry recoverable errors - Any error will be propagated cause the
error to exit, this includes connection errors to the Block Streamer and
Runner services. As these are very likely to occur (across restarts) we
should retry these errors.
- Environment configuration - There are many hard coded values
(endpoints, registry contract, etc.) which should be configurable via
the environment.
- Logging and Metrics
@darunrs darunrs mentioned this issue Jan 29, 2024
Closed
@pkudinov pkudinov mentioned this issue Feb 20, 2024
Open
@pkudinov
Copy link
Collaborator

Duplicates #759

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
component: Runner
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pkudinov @morgsmccauley @darunrs