Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Remote Store] Add support to restore only unassigned shards #8836

Closed
sachinpkale opened this issue Jul 24, 2023 · 0 comments · Fixed by #8792
Closed

[Remote Store] Add support to restore only unassigned shards #8836

sachinpkale opened this issue Jul 24, 2023 · 0 comments · Fixed by #8792
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.10.0

Comments

@sachinpkale
Copy link
Member

Is your feature request related to a problem? Please describe.

  • Currently, remote store restore API restores all the shards of a given index irrespective of the shard state.
  • This approach is not optimized for scenarios where only few of the shards are unassigned. Restore will still download data for all the shards in order to restore them.
  • We also need to close the index before calling remote store restore API. This impacts the index availability during the restore.

Describe the solution you'd like

  • Ideally, we should be able to restore only those shards that are unassigned and that too without closing the index.
  • In restore API execution, while marking shards to be restored, we will filter shards based on existing ShardRoutingState.
  • We pick only shards that are in UNASSIGNED state as shards to be restored.
  • User do not need to provide shard info in the API request. API signature would remain same which will accept index/indices to be restored. API implementation internally picks up only unassigned shards to restore.
  • We will also provide a query parameter restore_all_shards which, as the name suggests, will restore all the shards of the given index. As some of the shards can be active, to avoid an inconsistent state, when restore_all_shards=true is passed, index needs to be in closed state.
@sachinpkale sachinpkale added enhancement Enhancement or improvement to existing feature or request untriaged Storage:Durability Issues and PRs related to the durability framework v2.10.0 and removed untriaged labels Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.10.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant