Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CLN] Cleanup hnsw provider to not know about segments #3044

Merged
merged 5 commits into from
Nov 14, 2024
Merged

[CLN] Cleanup hnsw provider to not know about segments #3044

merged 5 commits into from
Nov 14, 2024

Conversation

sanketkedia
Copy link
Contributor

@sanketkedia sanketkedia commented Oct 31, 2024 •
edited
Loading

Description of changes

Summarize the changes made by this PR.

  • Improvements & Bug fixes
    • Leaky abstraction. The Hnsw provider knew about segments and was parsing it to extract stuff like index params. The distributed hnsw segment writer and reader should instead be doing that and provider should not know about it.

Test plan

How are these changes tested?

  • Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Documentation Changes

None

@github-actions GitHub Actions
Copy link

Please tag your PR title with one of: [ENH | BUG | DOC | TST | BLD | PERF | TYP | CLN | CHORE]. See https://docs.trychroma.com/contributing#contributing-code-and-ideas

@github-actions GitHub Actions
Copy link

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

  • Can you think of any use case in which the code does not behave as intended? Have they been tested?
  • Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
  • If appropriate, are there adequate property based tests?
  • If appropriate, are there adequate unit tests?
  • Should any logging, debugging, tracing information be added or removed?
  • Are error messages user-friendly?
  • Have all documentation changes needed been made?
  • Have all non-obvious changes been commented?

System Compatibility

  • Are there any potential impacts on other parts of the system or backward compatibility?
  • Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

  • Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

@sanketkedia sanketkedia changed the title Cleanup hnsw provider to not know about segments [CLN] Cleanup hnsw provider to not know about segments Oct 31, 2024
@sanketkedia Graphite App
Copy link
Contributor Author

sanketkedia commented Oct 31, 2024 •
edited
Loading

This stack of pull requests is managed by Graphite. Learn more about stacking.

This was referenced Oct 31, 2024
Merged
Merged
@sanketkedia sanketkedia requested a review from HammadB October 31, 2024 22:04
@sanketkedia sanketkedia marked this pull request as ready for review October 31, 2024 22:04
HammadB
HammadB reviewed Oct 31, 2024
View reviewed changes
rust/index/src/hnsw.rs Outdated Show resolved Hide resolved
This was referenced Nov 6, 2024
Merged
Merged
@sanketkedia sanketkedia force-pushed the 10-29-writer.get_support branch from 6dd7191 to 8c3ddcc Compare November 11, 2024 02:50
@sanketkedia sanketkedia force-pushed the 10-31-cleanup_hnsw_provider_to_not_know_about_segments branch from 00c5f8c to f0ad6a4 Compare November 11, 2024 02:50
@sanketkedia sanketkedia force-pushed the 10-29-writer.get_support branch from 8c3ddcc to bc48745 Compare November 11, 2024 06:32
@sanketkedia sanketkedia force-pushed the 10-31-cleanup_hnsw_provider_to_not_know_about_segments branch 2 times, most recently from 175edc0 to 2d374da Compare November 11, 2024 06:43
@sanketkedia sanketkedia force-pushed the 10-29-writer.get_support branch from c2b4b85 to f70ef34 Compare November 13, 2024 21:25
@sanketkedia sanketkedia force-pushed the 10-31-cleanup_hnsw_provider_to_not_know_about_segments branch 2 times, most recently from cd1753d to 8ecadb4 Compare November 13, 2024 21:28
@sanketkedia sanketkedia force-pushed the 10-29-writer.get_support branch 2 times, most recently from fb17ddf to 997c9ac Compare November 13, 2024 22:11
@sanketkedia sanketkedia force-pushed the 10-31-cleanup_hnsw_provider_to_not_know_about_segments branch from 8ecadb4 to 64f6165 Compare November 13, 2024 22:11
@sanketkedia sanketkedia changed the base branch from 10-29-writer.get_support to graphite-base/3044 November 13, 2024 22:14
@sanketkedia sanketkedia force-pushed the 10-31-cleanup_hnsw_provider_to_not_know_about_segments branch from 64f6165 to 4a4d501 Compare November 13, 2024 22:15
@sanketkedia sanketkedia changed the base branch from graphite-base/3044 to main November 13, 2024 22:15
@sanketkedia sanketkedia force-pushed the 10-31-cleanup_hnsw_provider_to_not_know_about_segments branch from 4a4d501 to 12db19e Compare November 13, 2024 22:15
@sanketkedia Graphite App
Copy link
Contributor Author

sanketkedia commented Nov 14, 2024 •
edited
Loading

Merge activity

  • Nov 14, 1:46 AM EST: A user started a stack merge that includes this pull request via Graphite.
  • Nov 14, 1:46 AM EST: A user merged this pull request with Graphite.

@sanketkedia sanketkedia merged commit 3f56aeb into main Nov 14, 2024
70 checks passed
This was referenced Nov 19, 2024
Merged
Merged
Merged
Merged
This was referenced Nov 29, 2024
Merged
Merged
Merged
Merged
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rescrv rescrv rescrv left review comments

@HammadB HammadB HammadB approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@sanketkedia @rescrv @HammadB