You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There is some bug,Exception raised in Job[1]: ValidationError(1 validation error for ChatMessage role Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
#1782
Open
Terran0629 opened this issue
Dec 21, 2024
· 0 comments
[ ] I have checked the documentation and related resources and couldn't resolve my bug.
Describe the bug
When I tried to use Ragas to evaluate and test a piece of text, I encountered the following error message
Exception raised in Job[1]: ValidationError(1 validation error for ChatMessage role Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
Evaluating: 0%| | 0/4 [00:00<?, ?it/s]Exception raised in Job[3]: ValidationError(1 validation error for ChatMessage
role
Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.10/v/string_type)
Evaluating: 25%|█████████████████████████████ | 1/4 [00:04<00:14, 4.73s/it]Exception raised in Job[2]: ValidationError(1 validation error for ChatMessage
role
Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.10/v/string_type)
Evaluating: 50%|██████████████████████████████████████████████████████████ | 2/4 [00:05<00:04, 2.32s/it]Exception raised in Job[0]: ValidationError(1 validation error for ChatMessage
role
Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.10/v/string_type)
Evaluating: 75%|███████████████████████████████████████████████████████████████████████████████████████ | 3/4 [00:06<00:01, 1.66s/it]Exception raised in Job[1]: ValidationError(1 validation error for ChatMessage
role
Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.10/v/string_type)
Evaluating: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:06<00:00, 1.67s/it]
Traceback (most recent call last):
File "d:\Visual Studio Code\LLM\_LLM05期资料\17-langchain和RAG进阶-24.11.13-于老师\01-简单RAG.py", line 79, in <module>
evaluate_result = evaluate(
File "D:\Anaconda\envs\DL_env\lib\site-packages\ragas\_analytics.py", line 205, in wrapper
result = func(*args, **kwargs)
File "D:\Anaconda\envs\DL_env\lib\site-packages\ragas\evaluation.py", line 333, in evaluate
result = EvaluationResult(
File "<string>", line 10, in __init__
File "D:\Anaconda\envs\DL_env\lib\site-packages\ragas\dataset_schema.py", line 410, in __post_init__
self.traces = parse_run_traces(self.ragas_traces, run_id)
File "D:\Anaconda\envs\DL_env\lib\site-packages\ragas\callbacks.py", line 167, in parse_run_traces
"output": prompt_trace.outputs.get("output", {})[0],
KeyError: 0
Additional context
Preliminary judgment is that it is a version issue. When I use Python version 3.10 and the following library versions, the code can run normally:
langchain==0.2.5
langchain-community==0.2.5
langchain-core==0.2.7
langchain-openai==0.1.8
langchain-text-splitters==0.2.1
langdetect==1.0.9
langsmith==0.1.77
pydantic==2.0.2
pydantic_core==2.1.2
ragas==0.1.9
The text was updated successfully, but these errors were encountered:
Terran0629
changed the title
Exception raised in Job[1]: ValidationError(1 validation error for ChatMessage role Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
There is some bug,Exception raised in Job[1]: ValidationError(1 validation error for ChatMessage role Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
Dec 21, 2024
[ ] I have checked the documentation and related resources and couldn't resolve my bug.
Describe the bug
When I tried to use Ragas to evaluate and test a piece of text, I encountered the following error message
Exception raised in Job[1]: ValidationError(1 validation error for ChatMessage role Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
Ragas version: >2.6.2
Python version: 3.10
Code to Reproduce
Error trace
Additional context
Preliminary judgment is that it is a version issue. When I use Python version 3.10 and the following library versions, the code can run normally:
langchain==0.2.5
langchain-community==0.2.5
langchain-core==0.2.7
langchain-openai==0.1.8
langchain-text-splitters==0.2.1
langdetect==1.0.9
langsmith==0.1.77
pydantic==2.0.2
pydantic_core==2.1.2
ragas==0.1.9
The text was updated successfully, but these errors were encountered: