Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide an access function to map _id to shard number. #88661

Closed
linker-c opened this issue Jul 20, 2022 · 5 comments
Closed

Provide an access function to map _id to shard number. #88661

linker-c opened this issue Jul 20, 2022 · 5 comments
Labels
:Distributed Indexing/CRUD A catch all label for issues around indexing, updating and getting a doc by id. Not search. >enhancement Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. team-discuss

Comments

@linker-c
Copy link

Description

Please provide an API to translate document ID (_id) to the corresponding shard location.
This is very useful when I am reading a document directly instead of searching. The "getResponse" should be able to provide a function like GetShard(int _id) that returns the shard this document resides.

There are some application side optimizations we are thinking about doing only if we know the shard of every document read.
As of today, I can't find an easy way to make this association.

@linker-c linker-c added >enhancement needs:triage Requires assignment of a team area label labels Jul 20, 2022
@pugnascotia pugnascotia added the :Distributed Indexing/CRUD A catch all label for issues around indexing, updating and getting a doc by id. Not search. label Jul 21, 2022
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@elasticsearchmachine elasticsearchmachine added Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. and removed needs:triage Requires assignment of a team area label labels Jul 21, 2022
@linker-c
Copy link
Author

@DaveCTurner
Copy link
Contributor

We (the @elastic/es-distributed team) discussed this idea today. We wouldn't want to add an API which exposes this calcluation, because to do so would be to commit to having this notion of an _id which maps to a shard number that we may want to relax in future versions (see e.g. #48699). We also feel it doesn't make sense to do this calculation via a remote call to an API on Elasticsearch. Instead, since it's just for an optimization, you can just do the calculation within your client. I'm linking to a discuss forum thread from another user that was trying to do the same sort of thing, where we went into some depth about how the calculation is done today.

Since there's no action to take on the Elasticsearch side here, I'm closing this.

@linker-c
Copy link
Author

linker-c commented Aug 2, 2022

Thanks for the feedback @DaveCTurner.
But can it at least be an option for direct access to a document? You won't be exposing the algorithm. You are simply displaying which shard a document is stored.

@DaveCTurner
Copy link
Contributor

I'll probably not reply to this issue again now it's closed. If you need further help, the community discussion forum is the best place to ask. Hopefully someone there will be able to help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Indexing/CRUD A catch all label for issues around indexing, updating and getting a doc by id. Not search. >enhancement Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. team-discuss
Projects
None yet
Development

No branches or pull requests

5 participants