feat: Add ContextRelevanceEvaluator component

[julian-risch]

Related Issues

• fixes Custom LLM-based evaluator in Haystack core #7022

Proposed Changes:

• Add new ContextRelevanceEvaluator component

How did you test it?

• Added new unit tests
• Ran the docstring example locally
• I also ran the following example with a pipeline containing two evaluators locally:

from haystack import Pipeline
from haystack.components.evaluators import ContextRelevanceEvaluator, FaithfulnessEvaluator

pipeline = Pipeline()
context_relevance_evaluator = ContextRelevanceEvaluator()
faithfulness_evaluator = FaithfulnessEvaluator()
pipeline.add_component("context_relevance_evaluator", context_relevance_evaluator)
pipeline.add_component("faithfulness_evaluator", faithfulness_evaluator)

questions = ["Who created the Python language?"]
contexts = [
    [
        "Python, created by Guido van Rossum in the late 1980s, is a high-level general-purpose programming language. Its design philosophy emphasizes code readability, and its language constructs aim to help programmers write clear, logical code for both small and large-scale software projects."
    ],
]
responses = ["Python is a high-level general-purpose programming language that was created by George Lucas."]


result = pipeline.run(
		{
			"context_relevance_evaluator": {"questions": questions, "contexts": contexts},
	    "faithfulness_evaluator": {"questions": questions, "contexts": contexts, "responses": responses}
    }
)

for evaluator in result:
    print(result[evaluator]["individual_scores"])
# [1.0]
# [0.5]
for evaluator in result:
    print(result[evaluator]["score"])
# 1.0
# 0.5

Notes for the reviewer

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[julian-risch]

Hi @shadeMe I addressed all your change requests. In addition I added the new evaluator to the api reference config that was merged earlier today.
I will open a separate PR to update the FaithfulnessEvaluator. -> #7548

[coveralls]

Pull Request Test Coverage Report for Build 8664013887

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.04%) to 90.105%

---

Totals
Change from base Build 8663731064:	0.04%
Covered Lines:	6283
Relevant Lines:	6973

---

💛 - Coveralls