[BUG] Evaluator respond with Nan values as the first token is a text #3522
Labels
bug
Something isn't working
no-recent-activity
There has been no recent activity on this issue/pull request
Describe the bug
Similarity Evaluator responds with a response Nan as first token in response is 'Text" and promptflow is not able to convert text to integer.
How To Reproduce the bug
Steps to reproduce the behavior, how frequent can you experience the bug:
Expected behavior
A clear and concise description of what you expected to happen.
It should respond with a number 1 to 5 as per the prompty - https://github.com/microsoft/promptflow/blob/main/src/promptflow-evals/promptflow/evals/evaluators/_similarity/similarity.prompty
Screenshots
If applicable, add screenshots to help explain your problem.
Running Information(please complete the following information):
pf -v
: [e.g. 0.0.102309906] :{
"promptflow": "1.12.0",
"promptflow-azure": "1.13.0",
"promptflow-core": "1.12.0",
"promptflow-devkit": "1.12.0",
"promptflow-evals": "0.3.0",
"promptflow-tracing": "1.12.0"
}
python --version
: [e.g. python==3.10.12]Python (Windows) 3.11.9 (tags/v3.11.9:de54cf5, Apr 2 2024, 10:12:12) [MSC v.1938 64 bit (AMD64)]
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: