You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
I customized a variant through the SDK and generated a test set in the playground. When I perform an evaluation, I encounter the following error message:
Traceback (most recent call last):
File "/app/agenta_backend/tasks/evaluations.py", line 297, in evaluate
type="text", value=app_output.result.value["data"]
KeyError: 'data'
variant project code
from pydantic import BaseModel, Field
import agenta as ag
class MyConfig(BaseModel):
prompt: str = Field(default="nihao")
The code you are using looks good to me. I will try to reproduce the error. Can you please make sure to upgrade agenta locally to the latest version: pip install -U agenta
then check which version are you using before serving the app again:
agenta --version agenta variant serve app.py
Please let me know if that does not resolve the issue.
If the case persist:
Can you please share with me the response of the application that you have created when called from the playground (you can go to the developer tools in your browser, and check the response to the request POST /generate).
Can you use the application in the playground, or the issue is solely within evaluation.
As soon as we have that, and the problem is still unresolved, I will try to reproduce it from our side.
mmabrouk
changed the title
[Bug]
[Bug] Running custom workflows in evaluation
Jan 12, 2025
Describe the bug
I customized a variant through the SDK and generated a test set in the playground. When I perform an evaluation, I encounter the following error message:
Traceback (most recent call last):
File "/app/agenta_backend/tasks/evaluations.py", line 297, in evaluate
type="text", value=app_output.result.value["data"]
KeyError: 'data'
variant project code
from pydantic import BaseModel, Field
import agenta as ag
class MyConfig(BaseModel):
prompt: str = Field(default="nihao")
@ag.route("/", config_schema=MyConfig)
def generate_text(input_text: str):
config = ag.ConfigManager.get_from_route(schema=MyConfig)
# Function implementation
evaluate config
The text was updated successfully, but these errors were encountered: