fix: Send batches of query-doc pairs to inference_from_objects

[bogdankostic]

Related Issues

• n/a

Proposed Changes:

This PR adds the parameter preprocessing_batch_size to FARMReader, which is used to limit the number of query-doc pairs passed to the QAInferencer's inference_from_objects method. This is needed because when passing all query-doc pairs to inference_from_objects, the QAInferencer will tokenize and featurize all inputs which might cause the RAM to go out of memory if the number of inputs is large.

How did you test it?

I added two unit tests.

Notes for the reviewer

I moved get_batches_from_generator from haystack.document_stores.base to haystack.utils.batching as this utility method is not only useful for document stores.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[coveralls]

Pull Request Test Coverage Report for Build 5353938751

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 199 unchanged lines in 2 files lost coverage.
• Overall coverage increased (+0.5%) to 43.056%

---

Files with Coverage Reduction	New Missed Lines	%
document_stores/base.py	62	44.76%
nodes/reader/farm.py	137	39.9%

Totals
Change from base Build 5353925852:	0.5%
Covered Lines:	9688
Relevant Lines:	22501

---

💛 - Coveralls

[masci]

Nothing to say, LGTM. Thanks for the "notes for the reviewer" helping me understand why we moved the method under utils.