Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the primary_key parameter of SecondaryIndex::GetSecondary{KeyPrefix,Value} #13207

Closed
wants to merge 1 commit into from

Conversation

ltamasi
Copy link
Contributor

@ltamasi ltamasi commented Dec 13, 2024

The patch tweaks the new SecondaryIndex interface a bit by removing the primary_key parameter of GetSecondaryKeyPrefix and GetSecondaryValue. This parameter is currently unused by existing implementations and it actually does not make sense to have the secondary index prefix depend on the primary key since it would lead to potential chicken-and-egg problems at query time.

Differential Revision: D67184936

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D67184936

@ltamasi ltamasi marked this pull request as draft December 13, 2024 05:56
@ltamasi ltamasi added the WIP Work in progress label Dec 13, 2024
ltamasi added a commit to ltamasi/rocksdb that referenced this pull request Dec 13, 2024
Summary: Pull Request resolved: facebook#13207

Differential Revision: D67184936
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D67184936

@wanghenshui
Copy link

great work!

ltamasi added a commit to ltamasi/rocksdb that referenced this pull request Dec 13, 2024
Summary: Pull Request resolved: facebook#13207

Differential Revision: D67184936
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D67184936

ltamasi added a commit to ltamasi/rocksdb that referenced this pull request Dec 13, 2024
Summary: Pull Request resolved: facebook#13207

Differential Revision: D67184936
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D67184936

…refix,Value} (facebook#13207)

Summary:
The patch tweaks the new `SecondaryIndex` interface a bit by removing the `primary_key` parameter of `GetSecondaryKeyPrefix` and `GetSecondaryValue`. This parameter is currently unused by existing implementations and it actually does not make sense to have the secondary index prefix depend on the primary key since it would lead to potential chicken-and-egg problems at query time.


Differential Revision: D67184936
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D67184936

@ltamasi ltamasi changed the title Support KNN searches for FAISS IVF indices Remove the primary_key parameter of SecondaryIndex::GetSecondary{KeyPrefix,Value} Dec 27, 2024
@ltamasi ltamasi removed the WIP Work in progress label Dec 27, 2024
@ltamasi ltamasi requested a review from jaykorean December 27, 2024 22:20
@ltamasi ltamasi marked this pull request as ready for review December 27, 2024 22:21
ltamasi added a commit to ltamasi/rocksdb that referenced this pull request Dec 28, 2024
…refix,Value} (facebook#13207)

Summary:
The patch tweaks the new `SecondaryIndex` interface a bit by removing the `primary_key` parameter of `GetSecondaryKeyPrefix` and `GetSecondaryValue`. This parameter is currently unused by existing implementations and it actually does not make sense to have the secondary index prefix depend on the primary key since it would lead to potential chicken-and-egg problems at query time.


Differential Revision: D67184936
ltamasi added a commit to ltamasi/rocksdb that referenced this pull request Dec 28, 2024
…refix,Value} (facebook#13207)

Summary:
The patch tweaks the new `SecondaryIndex` interface a bit by removing the `primary_key` parameter of `GetSecondaryKeyPrefix` and `GetSecondaryValue`. This parameter is currently unused by existing implementations and it actually does not make sense to have the secondary index prefix depend on the primary key since it would lead to potential chicken-and-egg problems at query time.


Differential Revision: D67184936
ltamasi added a commit to ltamasi/rocksdb that referenced this pull request Dec 28, 2024
…refix,Value} (facebook#13207)

Summary:
The patch tweaks the new `SecondaryIndex` interface a bit by removing the `primary_key` parameter of `GetSecondaryKeyPrefix` and `GetSecondaryValue`. This parameter is currently unused by existing implementations and it actually does not make sense to have the secondary index prefix depend on the primary key since it would lead to potential chicken-and-egg problems at query time.


Differential Revision: D67184936
ltamasi added a commit to ltamasi/rocksdb that referenced this pull request Dec 28, 2024
…refix,Value} (facebook#13207)

Summary:
The patch tweaks the new `SecondaryIndex` interface a bit by removing the `primary_key` parameter of `GetSecondaryKeyPrefix` and `GetSecondaryValue`. This parameter is currently unused by existing implementations and it actually does not make sense to have the secondary index prefix depend on the primary key since it would lead to potential chicken-and-egg problems at query time.


Differential Revision: D67184936
ltamasi added a commit to ltamasi/rocksdb that referenced this pull request Dec 28, 2024
…refix,Value} (facebook#13207)

Summary:
The patch tweaks the new `SecondaryIndex` interface a bit by removing the `primary_key` parameter of `GetSecondaryKeyPrefix` and `GetSecondaryValue`. This parameter is currently unused by existing implementations and it actually does not make sense to have the secondary index prefix depend on the primary key since it would lead to potential chicken-and-egg problems at query time.


Differential Revision: D67184936
ltamasi added a commit to ltamasi/rocksdb that referenced this pull request Dec 28, 2024
…refix,Value} (facebook#13207)

Summary:
The patch tweaks the new `SecondaryIndex` interface a bit by removing the `primary_key` parameter of `GetSecondaryKeyPrefix` and `GetSecondaryValue`. This parameter is currently unused by existing implementations and it actually does not make sense to have the secondary index prefix depend on the primary key since it would lead to potential chicken-and-egg problems at query time.


Differential Revision: D67184936
ltamasi added a commit to ltamasi/rocksdb that referenced this pull request Dec 28, 2024
…refix,Value} (facebook#13207)

Summary:
The patch tweaks the new `SecondaryIndex` interface a bit by removing the `primary_key` parameter of `GetSecondaryKeyPrefix` and `GetSecondaryValue`. This parameter is currently unused by existing implementations and it actually does not make sense to have the secondary index prefix depend on the primary key since it would lead to potential chicken-and-egg problems at query time.

Pull Request resolved: facebook#13207

Differential Revision: D67184936
@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 3570e4f.

ltamasi added a commit to ltamasi/rocksdb that referenced this pull request Jan 22, 2025
Summary: There are actually some use cases which would benefit from the ability to use the primary key when forming the secondary key prefix or value. One such use case, which is demonstrated using a unit test, is building a secondary index on non-initial part(s) of the primary key. The patch adds back this ability, which was was removed in facebook#13207, with a twist: the earlier `GetSecondaryKeyPrefix` is essentially split into two parts, with `GetSecondaryKeyPrefix` now being responsible only for computing whatever the secondary index is built on (let's call this "index function result") and a new `FinalizeSecondaryKeyPrefix` method having the responsibility of dealing with serialization concerns like adding a length indicator for disambiguation. This also means a slight change for the `SecondaryIndexIterator` class: it now treats its `Seek` argument as an "index function result" and thus only calls the new `FinalizeSecondaryKeyPrefix` on it (but not `GetSecondaryKeyPrefix`).

Differential Revision: D68514201
ltamasi added a commit to ltamasi/rocksdb that referenced this pull request Jan 22, 2025
…es (facebook#13324)

Summary:

There are actually some use cases which would benefit from the ability to use the primary key when forming the secondary key prefix or value. One such use case, which is demonstrated using a unit test, is building a secondary index on non-initial part(s) of the primary key. The patch adds back this ability, which was was removed in facebook#13207, with a twist: the earlier `GetSecondaryKeyPrefix` is essentially split into two parts, with `GetSecondaryKeyPrefix` now being responsible only for computing whatever the secondary index is built on (let's call this "index function result") and a new `FinalizeSecondaryKeyPrefix` method having the responsibility of dealing with serialization concerns like adding a length indicator for disambiguation. This also means a slight change for the `SecondaryIndexIterator` class: it now treats its `Seek` argument as an "index function result" and thus only calls the new `FinalizeSecondaryKeyPrefix` on it (but not `GetSecondaryKeyPrefix`).

Differential Revision: D68514201
facebook-github-bot pushed a commit that referenced this pull request Jan 22, 2025
…es (#13324)

Summary:
Pull Request resolved: #13324

There are actually some use cases which would benefit from the ability to use the primary key when forming the secondary key prefix or value. One such use case, which is demonstrated using a unit test, is building a secondary index on non-initial part(s) of the primary key. The patch adds back this ability, which was was removed in #13207, with a twist: the earlier `GetSecondaryKeyPrefix` is essentially split into two parts, with `GetSecondaryKeyPrefix` now being responsible only for computing whatever the secondary index is built on (let's call this "index function result") and a new `FinalizeSecondaryKeyPrefix` method having the responsibility of dealing with serialization concerns like adding a length indicator for disambiguation. This also means a slight change for the `SecondaryIndexIterator` class: it now treats its `Seek` argument as an "index function result" and thus only calls the new `FinalizeSecondaryKeyPrefix` on it (but not `GetSecondaryKeyPrefix`).

Reviewed By: jaykorean

Differential Revision: D68514201

fbshipit-source-id: d3750d049b0aee37e6c20edc19f5e4a0d3fce91e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants