Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: FaithfulnessEvaluator specifies inputs explicitly #7548

Merged
merged 2 commits into from
Apr 22, 2024

Conversation

julian-risch
Copy link
Member

Related Issues

None. Came up in the implementation of ContextRelevanceEvaluator

Proposed Changes:

  • FaithfulnessEvaluator specifies run method inputs explicitly
  • Move out examples
  • Add integration test
  • Extend docstring to explain that default examples are used if none are provided

How did you test it?

existing unit tests, new integration test

Notes for the reviewer

Checklist

@julian-risch julian-risch added the ignore-for-release-notes PRs with this flag won't be included in the release notes. label Apr 12, 2024
@julian-risch julian-risch requested a review from a team as a code owner April 12, 2024 15:13
@julian-risch julian-risch requested review from vblagoje and shadeMe and removed request for a team and vblagoje April 12, 2024 15:13
@github-actions github-actions bot added topic:tests 2.x Related to Haystack v2.0 type:documentation Improvements on the docs labels Apr 12, 2024
@coveralls
Copy link
Collaborator

coveralls commented Apr 12, 2024

Pull Request Test Coverage Report for Build 8783754562

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.002%) to 90.065%

Totals Coverage Status
Change from base Build 8663731064: 0.002%
Covered Lines: 6255
Relevant Lines: 6945

💛 - Coveralls

haystack/components/evaluators/faithfulness.py Outdated Show resolved Hide resolved
@julian-risch julian-risch enabled auto-merge (squash) April 22, 2024 12:37
@julian-risch julian-risch merged commit d7638cf into main Apr 22, 2024
17 of 20 checks passed
@julian-risch julian-risch deleted the refactor-faithfulness-eval branch April 22, 2024 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.x Related to Haystack v2.0 ignore-for-release-notes PRs with this flag won't be included in the release notes. topic:eval topic:tests type:documentation Improvements on the docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants