Code

Team/hypothesis tests#474

Code

Merged

HammadB merged 197 commits intomainchroma-core/chroma:mainfrom team/hypothesis-testschroma-core/chroma:team/hypothesis-testsCopy head branch name to clipboardMay 6, 2023

+3,222-556

Commits

Commits on Apr 6, 2023

• initial cut of hypothesis-based property tests

  levand

  committed
  0f80148

Commits on Apr 7, 2023

• Allow capital letters in collection names

  levand

  committed
  d3e17ea

Commits on Apr 10, 2023

• WIP on collection state machine test

  levand

  committed
  b45e2cf

• add clean failing minimal examples

  levand

  committed
  217502b

Commits on Apr 11, 2023

• fix incorrect test logic

  levand

  committed
  86888f9

• Fix collection name validation

  levand

  committed
  785d3c1

• only construct default embedding function once

  levand

  committed
  f48a07a

• update metadata when doing 'upsert' on collection

  levand

  committed
  69c6822

• re-enable all test api fixtures

  levand

  committed
  9e9e97c

• Merge pull request #302 from chroma-core/lukev/fix-collection-name-validation

  Show description for 470c0cb
  levand

  authored
  470c0cb

• Update docstrings to reflect metadata upsert behavior

  levand

  authored
  02bb481

Commits on Apr 12, 2023

• Revert "only construct default embedding function once".

  Show description for f23ba4b
  levand

  committed
  f23ba4b

• Use class var to store SentenceTransformer instances

  Show description for 34006bc
  levand

  committed
  34006bc

• Minimal ANN Accuracy Invariant and Collection.add() (#329)

  Show description for 9b4f003
  

  atroyn

  and

  levand

  authored
  9b4f003

Commits on Apr 13, 2023

• Merge pull request #328 from chroma-core/lukev/fix-collections

  Show description for 2558481
  levand

  authored
  2558481

• Merge branch 'team/hypothesis-tests' into lukev/collection-state-machine-tests

  levand

  committed
  d6a308a

• Merge pull request #324 from chroma-core/lukev/collection-state-machine-tests

  Show description for 214c06c
  levand

  authored
  214c06c

Commits on Apr 15, 2023

• state machine tests for embeddings

  levand

  committed
  6e20759

• remember to reset before each unit test

  levand

  committed
  7fd4233

• if creation fails, finish step

  levand

  committed
  3707a35

• temporarily generate IDs that we know won't cause SQL issues

  levand

  committed
  7b213ac

• Merge branch 'lukev/hypothesis-test-fixes' into lukev/embeddings-stateful-tests

  levand

  committed
  682445d

Commits on Apr 16, 2023

• add failing tests for duplicate embeddings

  levand

  committed
  5e7940d

Commits on Apr 17, 2023

• add update to embedding stateful tests

  levand

  committed
  f7e3874

• valiation to prevent dup ID inserts

  levand

  committed
  d036595

• add JS validation & tests

  levand

  committed
  1a46040

• use unique IDs in unit tests

  levand

  committed
  8671504

• fix js test to handle local validation

  levand

  committed
  c5b096e

• ensure that documents are populated for updates

  levand

  committed
  25d451f

• clean unused code

  levand

  committed
  87802f0

• Revert "fix js test to handle local validation"

  Show description for e31d240
  levand

  committed
  e31d240

• Revert "add JS validation & tests"

  Show description for b136933
  levand

  committed
  b136933

• don't convert existing IDs to set

  levand

  committed
  c43051a

• use and check for specific error types

  levand

  committed
  d6fd0c6

• avoid operator overloading

  levand

  committed
  693a53a

• Merge pull request #363 from chroma-core/lukev/validate_add

  Show description for 252e56d
  levand

  authored
  252e56d

• avoid shadowing name from strategies module

  levand

  committed
  e02deb5

• remove extra type annotations to avoid confusion

  levand

  committed
  ff66ce3

• simplify code as discussed in review

  levand

  committed
  6b54ca9

• remove precondition; start updating right away

  levand

  committed
  5139c39

• all invariants in their own module

  levand

  committed
  4b338fd

Commits on Apr 18, 2023

• Added xfail overrides on tests expected to be failing.

  atroyn

  committed
  554c003

• Merge pull request #355 from chroma-core/lukev/embeddings-stateful-tests

  Show description for eb57b56
  levand

  authored
  eb57b56

• fix updates by ensuring correct ID ordering

  levand

  committed
  a2e219f

• Revert "Revert "add JS validation & tests""

  Show description for b377d7f
  levand

  committed
  b377d7f

• Revert "Revert "fix js test to handle local validation""

  Show description for a5cef8a
  levand

  committed
  a5cef8a

• remove stub test we don't plan on implementing

  levand

  committed
  aa305c9

• enable CI for team/hypothesis-tests branch

  levand

  committed
  0733afd

• Whitespace change to test CI.

  levand

  authored
  12f83da

• Whitespace change to provoke CI

  levand

  authored
  bd1a871

• Upsert test

  

  atroyn

  authored and

  levand

  committed
  e5d6090

• More info about the test

  

  atroyn

  authored and

  levand

  committed
  498b5bb

• Cleaned up what I actually meant in the test

  

  atroyn

  authored and

  levand

  committed
  ddc4aef

• API and tests

  

  atroyn

  authored and

  levand

  committed
  d2d454c

• Collection and APIs

  

  atroyn

  authored and

  levand

  committed
  b4c74b7

• Pytest on by default in vscode

  

  atroyn

  authored and

  levand

  committed
  cc91113

• Updated tests

  

  atroyn

  authored and

  levand

  committed
  93d9dde

• Removed prints

  

  atroyn

  authored and

  levand

  committed
  615a5b0

• factor out dup code in add/update/upsert

  levand

  committed
  1bbc2b2

• clean up docstrings

  levand

  committed
  e7e4fab

• fix invalid regex

  levand

  committed
  a0d017a

• fix argument order

  levand

  committed
  f75abf1

• updates do not require embeddings

  levand

  committed
  f7a6636

• add upsert to js client

  levand

  committed
  5109895

• fix occasional test failure

  levand

  committed
  391c4b2

• remove stub test we don't plan on implementing

  levand

  committed
  088fa88

• enable CI for team/hypothesis-tests branch

  levand

  committed
  2acf3cd

• Upsert test

  

  atroyn

  authored and

  levand

  committed
  4ffeee2

• More info about the test

  

  atroyn

  authored and

  levand

  committed
  8f456a3

• Cleaned up what I actually meant in the test

  

  atroyn

  authored and

  levand

  committed
  667c595

• API and tests

  

  atroyn

  authored and

  levand

  committed
  9394cab

• Collection and APIs

  

  atroyn

  authored and

  levand

  committed
  eaf8de2

• Pytest on by default in vscode

  

  atroyn

  authored and

  levand

  committed
  2c6878f

• Updated tests

  

  atroyn

  authored and

  levand

  committed
  1af761b

• Removed prints

  

  atroyn

  authored and

  levand

  committed
  12d666a

• factor out dup code in add/update/upsert

  levand

  committed
  6f6f697

• clean up docstrings

  levand

  committed
  be7ed6d

• fix invalid regex

  levand

  committed
  a236b73

• fix argument order

  levand

  committed
  da8d6b4

• updates do not require embeddings

  levand

  committed
  bbbe737

Commits on Apr 19, 2023

• Basic Persistence Tests (#372)

  Show description for 25870d7
  HammadB

  authored
  25870d7

• state machine tests for upsert

  levand

  committed
  58e88fd

• Merge branch 'team/hypothesis-tests' into lukev/anton-upsert

  levand

  authored
  dbd7358

• Add explanatory comment.

  levand

  authored
  1af0f09

• Merge pull request #375 from chroma-core/lukev/fix-update-ids

  Show description for 0ad2b40
  levand

  authored
  0ad2b40

• add unit test to cover case removed from state machine

  levand

  committed
  be1ee89

• unit test duplicate checks in upsert

  levand

  committed
  653184f

• fix: call right method

  levand

  committed
  21a8722

• rename argument for clarity

  levand

  committed
  867ca06

• Merge pull request #379 from chroma-core/lukev/fix-coll-upsert-failure

  Show description for 29dff1a
  levand

  authored
  29dff1a

• use generators to allow cleanup for fixtures

  levand

  committed
  869996b

• fixtures for local fastapi server

  levand

  committed
  d1d7c4c

• propagate HTTP errors as correct type

  levand

  committed
  48d7296

Commits on Apr 20, 2023

• Merge branch 'team/hypothesis-tests' into lukev/anton-upsert

  levand

  authored
  b8f4db4

• update .dockerignore to improve build times

  levand

  committed
  d461c9b

• run integration tests only from bin/integration-tests

  levand

  committed
  aed8fde

• parameterize SQL

  levand

  committed
  de31abe

• add integration tests to hypothesis fixtures

  levand

  committed
  e0c1fd6

• Merge pull request #385 from chroma-core/lukev/anton-upsert

  Show description for d205f87
  levand

  authored
  d205f87

• Merge branch 'team/hypothesis-tests' into lukev/upsert-js

  levand

  committed
  196959d

• Delete clickhouse-run

  levand

  authored
  81cb8a3

Commits on Apr 21, 2023

• remove unused function

  levand

  committed
  4301535

Commits on Apr 22, 2023

• clean up and simplify strategies

  levand

  committed
  32b3957

• update tests to use new strategies

  levand

  committed
  dd3beb6

Commits on Apr 24, 2023

• Cross-version persistence tests (#386)

  Show description for a42fefd
  HammadB

  authored
  a42fefd

• remove not doing TODOs

  HammadB

  committed
  abff57f

Commits on Apr 25, 2023

• Generalized ANN Tests (#414)

  Show description for 887d466
  atroyn

  authored
  887d466

• WIP unsatisfiable errors

  levand

  committed
  2468acc

• cleanup & tweaks to avoid UnsatisfiableErrors

  levand

  committed
  c088086

• Merge branch 'team/hypothesis-tests' into lukev/collection-contents-strategy

  levand

  committed
  1265a6d

• update persist tests to use new strategies

  levand

  committed
  c9faacb

• WIP filtering

  levand

  committed
  0fc6775

• where-clause filtering working

  levand

  committed
  d6d4a35

• combo with id-based filter

  levand

  committed
  4317c85

• add doc generation and keyword filtering

  levand

  committed
  86c4680

• add flag to omit filter data unless needed

  levand

  committed
  4336f3d

Commits on Apr 26, 2023

• move module fixtures to conftest level

  levand

  committed
  2f2f579

• Persist state machine (#401)

  Show description for d183872
  HammadB

  authored
  d183872

• use common fixtures for all tests

  levand

  committed
  512d3f8

• type hints on fixture generators

  levand

  committed
  84cf64c

• Merge branch 'team/hypothesis-tests' into lukev/hypothesis-integration-tests

  levand

  committed
  b2fc5a6

• cleanup whitespace

  levand

  authored
  683b601

• restrict tests & enable full logging

  levand

  committed
  0f062e7

• Seperate integration tests into their own github actions (#427)

  Show description for c866e55
  HammadB

  authored
  c866e55

• split out test matrix

  levand

  committed
  4f6b82b

• Merge branch 'team/hypothesis-tests' into lukev/hypothesis-integration-tests

  levand

  authored
  65e8cf7

Commits on Apr 27, 2023

• add explicit timeout to avoid timeout cache bug

  levand

  committed
  309edf4

• cleanup assertion messages

  levand

  committed
  820bb5d

• updates in response to PR feedback

  levand

  committed
  096f273

• only query for a fraction of results

  levand

  committed
  57c8695

• Incorporate tweaks from PR feedback

  levand

  committed
  bf83b07

• Merge pull request #425 from chroma-core/lukev/hypothesis-filtering

  Show description for 84802d7
  levand

  authored
  84802d7

• cleanup based on PR feadback

  levand

  committed
  0e1cbf6

• Merge branch 'team/hypothesis-tests' into lukev/collection-contents-strategy

  levand

  authored
  2cf7363

• small change to test CI

  levand

  committed
  b3acc46

• demonstrate a bug with ANN accuracy

  levand

  committed
  9459baf

• Combine fixtures (#431)

  Show description for 816a3ad
  

  HammadB

  and

  atroyn

  authored
  816a3ad

• Merge branch 'team/hypothesis-tests' into lukev/hypothesis-integration-tests

  levand

  committed
  5d229bd

• Merge branch 'team/hypothesis-tests' into lukev/collection-contents-strategy

  levand

  authored
  a429db3

Commits on Apr 28, 2023

• ensure ann results are sorted (#434)

  HammadB

  authored
  3bb54aa

• Increase teset hnsw settings

  HammadB

  committed
  3fec500

• merge

  HammadB

  committed
  db36086

• Merge branch 'lukev/ann_invariant_bug_demo' into lukev/collection-contents-strategy

  levand

  committed
  2c6cee5

• Merge branch 'lukev/ann_invariant_bug_demo' into lukev/collection-contents-strategy

  levand

  committed
  25096ed

• Merge branch 'lukev/ann_invariant_bug_demo' into lukev/hypothesis-integration-tests

  levand

  committed
  8b5f62a

• Ann invariant increase hnsw params (#446)

  Show description for 551f69a
  

  HammadB

  and

  levand

  authored
  551f69a

• clean up configurations

  levand

  committed
  e05e7b5

• remove verbose logging

  levand

  committed
  2d23fa7

• more type hints

  levand

  committed
  89aeab7

• Merge branch 'team/hypothesis-tests' into lukev/collection-contents-strategy

  levand

  committed
  f07abc0

• Merge pull request #423 from chroma-core/lukev/collection-contents-strategy

  Show description for b042279
  levand

  authored
  b042279

• Update test_persist.py

  levand

  authored
  48b6143

• Merge branch 'team/hypothesis-tests' into lukev/hypothesis-integration-tests

  levand

  committed
  df1cc76

• add filtering tests to CI

  levand

  committed
  932567c

• fix merge errors

  levand

  committed
  feda987

Commits on Apr 30, 2023

• add workaround for FastAPI quirk

  levand

  committed
  748c9cc

Commits on May 1, 2023

• fix filter semantics in ClickHouse

  levand

  committed
  7ed736e

• update strategies and invariants to handle unwrapped values

  levand

  committed
  f28fcc2

• don't convert empty dicts to None

  levand

  committed
  dd350b8

• expose invariant wrapping for consistency

  levand

  committed
  ced7357

• add standardized mechanism for cross-version changes

  levand

  committed
  1161a7e

• normalize embeddings before adding IDs to bundle

  levand

  committed
  4f4e600

• require key to be present for all operators

  levand

  committed
  a8afde6

• require key to be present for implicit ops too

  levand

  committed
  20583c0

• make guard logic compose within OR clauses

  levand

  committed
  d7cab42

• cleanup simple duplicative merge

  HammadB

  committed
  5906935

Commits on May 2, 2023

• constrain JSONd values to float32 for Clickhouse compatibility

  levand

  committed
  e3ba284

• Merge branch 'team/hypothesis-tests' into lukev/hypothesis-integration-tests

  levand

  authored
  0a0f66e

• Revert "constrain JSONd values to float32 for Clickhouse compatibility"

  Show description for 53e93d2
  levand

  committed
  53e93d2

• prevent generation of subnormal flaots

  levand

  committed
  2e43c5f

• parallelize integration tests using same approach as unit tests

  levand

  committed
  f3e818b

• Split apart tests to match what's currently in main

  Show description for 5cd22fb
  levand

  committed
  5cd22fb

• factor out upsert tests to their own file

  levand

  committed
  016315c

• Merge pull request #398 from chroma-core/lukev/hypothesis-integration-tests

  Show description for f732ede
  levand

  authored
  f732ede

• Merge branch 'team/hypothesis-tests' into lukev/test-unwrapped-values

  levand

  authored
  746ce9e

• Merge pull request #451 from chroma-core/lukev/test-unwrapped-values

  Show description for 7ca1e7b
  levand

  authored
  7ca1e7b

• Merge branch 'team/hypothesis-tests' into lukev/validate-add-js

  levand

  committed
  a1b9347

• Merge branch 'team/hypothesis-tests' into lukev/upsert-js

  levand

  committed
  f39f049

• fix bug with intended test partition; actually exclude prop tests

  levand

  committed
  caa03d0

• poke CI

  levand

  committed
  824a406

Commits on May 3, 2023

• Merge pull request #454 from chroma-core/lukev/fix-glob-in-ci-config

  Show description for 5c929c9
  levand

  authored
  5c929c9

• Merge branch 'team/hypothesis-tests' into lukev/upsert-js

  levand

  committed
  e1de81f

• Merge branch 'team/hypothesis-tests' into lukev/validate-add-js

  levand

  committed
  5904549

• python version matrix (#448)

  Show description for 487a48e
  HammadB

  authored
  487a48e

• Query filtering (#453)

  Show description for 9500e2a
  HammadB

  authored
  9500e2a

• Merge pull request #377 from chroma-core/lukev/validate-add-js

  Show description for 3fdd908
  levand

  authored
  3fdd908

• Merge branch 'team/hypothesis-tests' into lukev/upsert-js

  levand

  authored
  dec24e7

• Merge pull request #399 from chroma-core/lukev/upsert-js

  Show description for be04dca
  levand

  authored
  be04dca

Commits on May 4, 2023

• Add support for multiple spaces (#457)

  Show description for 000a9d3
  HammadB

  authored
  000a9d3

Commits on May 5, 2023

• PR checklist (#459)

  Show description for 833b89a
  HammadB

  authored
  833b89a

• Fix PR review checklist

  HammadB

  committed
  cfdf89c

• Test embedding functions (#466)

  Show description for 891f637
  atroyn

  authored
  891f637

• merge main into team hypothesis test

  HammadB

  committed
  8dfb223

• Merge branch 'team/merge_team_hypothesis' into team/hypothesis-tests

  HammadB

  committed
  2f52675

Commits on May 6, 2023

• merge main

  HammadB

  committed
  a19af5c

• inaccurate log

  HammadB

  committed
  fefab56

• Add epsilon for norms in cosine per hnswli

  HammadB

  committed
  08d4fc0