Skip to content

Commit

Permalink
Harmonize meta data handling across doc stores (#214)
Browse files Browse the repository at this point in the history
  • Loading branch information
tholor authored Jul 9, 2020
1 parent c45d549 commit 549f3a1
Showing 1 changed file with 9 additions and 3 deletions.
12 changes: 9 additions & 3 deletions haystack/database/elasticsearch.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,16 +118,22 @@ def write_documents(self, documents: List[dict]):
:param documents: List of dictionaries.
Default format: {"name": "<some-document-name>, "text": "<the-actual-text>"}
Optionally: You can add more key-value-pairs, that will be indexed as fields in
Elasticsearch and can be accessed later for filtering or shown in the responses of the Finder
Optionally: Include meta data via {"name": "<some-document-name>,
"text": "<the-actual-text>", "meta":{"author": "somebody", ...}}
It can be used for filtering and is accessible in the responses of the Finder.
Advanced: If you are using your own Elasticsearch mapping, the key names in the dictionary
should be changed to what you have set for self.text_field and self.name_field .
:return: None
"""
for doc in documents:
doc["_op_type"] = "create"
doc["_index"] = self.index

# In order to have a flat structure in elastic + similar behaviour to the other DocumentStores,
# we "unnest" all value within "meta"
if "meta" in doc.keys():
for k, v in doc["meta"].items():
doc[k] = v
del doc["meta"]
bulk(self.client, documents, request_timeout=300)

def get_document_count(self) -> int:
Expand Down

0 comments on commit 549f3a1

Please sign in to comment.