e2e testing for mongo

[fnikolai]

Description

Add end-to-end testing for mongo.

The test_cdc is falling.
This is probably due to a false assumption that the database is being reset on every iteration. However, this is not the case for integration testing, and the test_index is increasingly populated, resulting in failing assertions.

For instance:

>       raise Exception(exc_msg)
E       Exception: assert 55 == 1
E        +  where 55 = len(<superduperdb.vector_search.interface.FastVectorSearcher object at 0x7f1bba7bd650>)

test/integration/test_cdc.py:93: Exception
====================================================================== short test summary info =======================================================================
FAILED test/integration/test_cdc.py::test_vector_database_sync - Exception: assert 55 == 1
========================================================================= 1 failed in 1.67s ==========================================================================

Related Issues

Checklist

• Is this code covered by new or existing unit tests or integration tests?
• Did you run make test successfully?
• Do new classes, functions, methods and parameters all have docstrings?
• Were existing docstrings updated, if necessary?
• Was external documentation updated, if necessary?

Additional Notes or Comments

[blythed]

Description

Add end-to-end testing for mongo.

The test_cdc is falling. This is probably due to a false assumption that the database is being reset on every iteration. However, this is not the case for integration testing, and the test_index is increasingly populated, resulting in failing assertions.

For instance:

>       raise Exception(exc_msg)
E       Exception: assert 55 == 1
E        +  where 55 = len(<superduperdb.vector_search.interface.FastVectorSearcher object at 0x7f1bba7bd650>)

test/integration/test_cdc.py:93: Exception
====================================================================== short test summary info =======================================================================
FAILED test/integration/test_cdc.py::test_vector_database_sync - Exception: assert 55 == 1
========================================================================= 1 failed in 1.67s ==========================================================================

Related Issues

Checklist

• Is this code covered by new or existing unit tests or integration tests?
• Did you run make test successfully?
• Do new classes, functions, methods and parameters all have docstrings?
• Were existing docstrings updated, if necessary?
• Was external documentation updated, if necessary?

Additional Notes or Comments

This is the line where the database is being dropped between tests.
If this is not working, it may be to do with the user/ permissions? Have these been changed.

https://github.com/SuperDuperDB/superduperdb/blob/f007384854ce6c91fa4ecea48cf8c761312ed6f5/test/conftest.py#L69

The integration tests, all call this function, since they call this fixture:

@pytest.fixture
def database_with_default_encoders_and_model(test_db):
    ...

[codecov-commenter]

Codecov Report

Attention: 164 lines in your changes are missing coverage. Please review.

Comparison is base (34830a7) 80.33% compared to head (479396e) 64.14%.
Report is 41 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1277       +/-   ##
===========================================
- Coverage   80.33%   64.14%   -16.20%     
===========================================
  Files          95      103        +8     
  Lines        6602     6941      +339     
===========================================
- Hits         5304     4452      -852     
- Misses       1298     2489     +1191     

Files	Coverage Δ
superduperdb/__init__.py	100.00% <100.00%> (ø)
superduperdb/backends/mongodb/cdc/base.py	77.77% <100.00%> (ø)
superduperdb/backends/mongodb/cdc/listener.py	82.78% <100.00%> (ø)
superduperdb/base/config.py	100.00% <100.00%> (ø)
superduperdb/ext/torch/model.py	39.75% <100.00%> (-37.05%)	⬇️
superduperdb/misc/download.py	37.44% <100.00%> (-37.92%)	⬇️
superduperdb/misc/serialization.py	82.60% <100.00%> (-2.34%)	⬇️
superduperdb/misc/superduper.py	45.00% <ø> (-31.25%)	⬇️
superduperdb/vector_search/in_memory.py	84.21% <100.00%> (-1.76%)	⬇️
superduperdb/vector_search/lance.py	77.77% <100.00%> (+0.41%)	⬆️
... and 18 more

... and 38 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[blythed]

Should this all be commented out?