Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Pinecone tests #3555

Merged
merged 8 commits into from
Nov 14, 2022
Merged

refactor: Pinecone tests #3555

merged 8 commits into from
Nov 14, 2022

Conversation

masci
Copy link
Contributor

@masci masci commented Nov 11, 2022

Related Issues

  • fixes n/a

Proposed Changes:

  • Implement Pinecone tests as a subclass of DocumentStoreBaseTestAbstract
  • Fix Label metadata management: it was ignored before, probably a sign that's not a feature Pinecone users rely on very much, still had to fix it in order for the tests to pass.

How did you test it?

pytest -m"document_store and integration" test/document_stores/test_pinecone.py

Notes for the reviewer

Tests run on Linux, Win and MacOS because the storage is mocked. Mock can be disabled passing --mock-pinecone=false so the test suite is still to be considered of kind "integration".

Checklist

@masci masci marked this pull request as ready for review November 11, 2022 16:04
@masci masci requested a review from a team as a code owner November 11, 2022 16:04
@masci masci requested review from julian-risch and removed request for a team November 11, 2022 16:04
Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very good to me! 👍 The only change I would request is to remove one unnecessary import of Label and Answer in test/document_stores/test_pinecone.py. It doesn't seem to be needed there if I am not mistaken. Otherwise it's all ready to merge.
Out of curiosity I would like to better understand the reason behind changing [""] to .get("") to prevent KeyErrors I assume in test_base.py 🙂

@@ -318,7 +318,7 @@ def test_duplicate_documents_skip(self, ds, documents):

ds.write_documents(updated_docs, duplicate_documents="skip")
for d in ds.get_all_documents():
assert d.meta["name"] != "Updated"
assert d.meta.get("name") != "Updated"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that this change ensure that no KeyError is raised if "name" is not part of d.meta and instead Null is returned. My question is if that was an issue before? The documents all have meta data with a name set, no?
There is nothing wrong about this change. I just would like to understand it a bit better. 🙂

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pinecone tests have their own set of documents provided by the documents fixture, and some of them don't have any metadata attached. I think all the document stores should be able to pass tests on the same set of documents, but I didn't want to change too much functionality in this PR so I left pinecone tests with their custom set of docs, which requires this change.

test/document_stores/test_pinecone.py Outdated Show resolved Hide resolved
@pytest.mark.integration
def test_simplified_filters(self, ds, documents):
pass

# NOTE: Pinecone does not support dates, so it can't do lte or gte on date fields. When a new release introduces this feature,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that we only have this comment once now. 👍

test/document_stores/test_pinecone.py Outdated Show resolved Hide resolved
Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! 👍 Thanks for taking the time to explain the reason behind changing [""] to .get("").

@masci masci merged commit 057a8c0 into main Nov 14, 2022
@masci masci deleted the massi/pinecone branch November 14, 2022 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants