Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test PR 2 #2

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open

Test PR 2 #2

wants to merge 52 commits into from

Conversation

HarshTrivedi
Copy link
Member

No description provided.

HarshTrivedi and others added 14 commits November 26, 2024 18:58
@HarshTrivedi
add bundle test
8445a97
@HarshTrivedi
minor
50c2f3c
@HarshTrivedi
Merge branch 'main' of https://github.com/stonybrooknlp/appworld-lead…
bc99ef7 
…erboard into test-pr
@HarshTrivedi
Merge branch 'main' of https://github.com/stonybrooknlp/appworld-lead…
64e3279 
…erboard into test-pr
@HarshTrivedi
Merge branch 'main' of https://github.com/stonybrooknlp/appworld-lead…
f3f5524 
…erboard into test-pr
@HarshTrivedi
Merge branch 'main' of https://github.com/stonybrooknlp/appworld-lead…
6d0f7ca 
…erboard into test-pr
@HarshTrivedi
minor
4c43b27
@HarshTrivedi
Merge branch 'main' of https://github.com/stonybrooknlp/appworld-lead…
f4c202f 
…erboard into test-pr
@HarshTrivedi
Merge branch 'main' of https://github.com/stonybrooknlp/appworld-lead…
441aa72 
…erboard into test-pr
@HarshTrivedi
Merge branch 'main' of https://github.com/stonybrooknlp/appworld-lead…
cbd8268 
…erboard into test-pr
@HarshTrivedi
Merge branch 'main' of https://github.com/stonybrooknlp/appworld-lead…
a3ae098 
…erboard into test-pr
@HarshTrivedi
Merge branch 'main' of https://github.com/stonybrooknlp/appworld-lead…
305f1b5 
…erboard into test-pr
@github-actions
add leaderboard entry
84efdb2
@HarshTrivedi
Merge branch 'test-pr' of https://github.com/stonybrooknlp/appworld-l…
6fb030f 
…eaderboard into test-pr
@HarshTrivedi
Copy link
Member Author

/add-to-leaderboard --python 3.11 --appworld 0.1.3 --experiment-prefix temp_test

@github-actions GitHub Actions
Copy link

Latest Leaderboard Entry

{
    "id": "5509efd4-3e4bf4e2",
    "method": {
        "name": "temp",
        "tooltip": "temp"
    },
    "llm": {
        "name": "temp",
        "tooltip": "temp"
    },
    "url": "temp",
    "date": "2024-11-27",
    "test_normal": {
        "all": {
            "task_goal_completion": 9.5,
            "scenario_goal_completion": 7.1,
            "interactions": 3.32
        },
        "level 1": {
            "task_goal_completion": 28.1,
            "scenario_goal_completion": 21.1,
            "interactions": 3.32
        },
        "level 2": {
            "task_goal_completion": 0,
            "scenario_goal_completion": 0,
            "interactions": 3.32
        },
        "level 3": {
            "task_goal_completion": 0,
            "scenario_goal_completion": 0,
            "interactions": 3.32
        }
    },
    "test_challenge": {
        "all": {
            "task_goal_completion": 5.5,
            "scenario_goal_completion": 4.3,
            "interactions": 4.39
        },
        "level 1": {
            "task_goal_completion": 26.4,
            "scenario_goal_completion": 20.8,
            "interactions": 4.39
        },
        "level 2": {
            "task_goal_completion": 2.7,
            "scenario_goal_completion": 2,
            "interactions": 4.39
        },
        "level 3": {
            "task_goal_completion": 0,
            "scenario_goal_completion": 0,
            "interactions": 4.39
        }
    }
}

@HarshTrivedi
Copy link
Member Author

/remove-from-entry 5509efd4-3e4bf4e2

@HarshTrivedi
Copy link
Member Author

/remove-from-leaderboard 5509efd4-3e4bf4e2

@HarshTrivedi
Copy link
Member Author

/remove-from-leaderboard 5509efd4-3e4bf4e2

1 similar comment
@HarshTrivedi
Copy link
Member Author

/remove-from-leaderboard 5509efd4-3e4bf4e2

@HarshTrivedi
Copy link
Member Author

/remove-from-leaderboard --entry-id 5509efd4-3e4bf4e2

4 similar comments
@HarshTrivedi
Copy link
Member Author

/remove-from-leaderboard --entry-id 5509efd4-3e4bf4e2

@HarshTrivedi
Copy link
Member Author

/remove-from-leaderboard --entry-id 5509efd4-3e4bf4e2

@HarshTrivedi
Copy link
Member Author

/remove-from-leaderboard --entry-id 5509efd4-3e4bf4e2

@HarshTrivedi
Copy link
Member Author

/remove-from-leaderboard --entry-id 5509efd4-3e4bf4e2

@github-actions GitHub Actions
Copy link

Removed the following entry from the leaderboard:

{
  "id": "5509efd4-3e4bf4e2",
  "method": {
    "name": "temp",
    "tooltip": "temp"
  },
  "llm": {
    "name": "temp",
    "tooltip": "temp"
  },
  "url": "temp",
  "date": "2024-11-27",
  "test_normal": {
    "all": {
      "task_goal_completion": 9.5,
      "scenario_goal_completion": 7.1,
      "interactions": 3.32
    },
    "level 1": {
      "task_goal_completion": 28.1,
      "scenario_goal_completion": 21.1,
      "interactions": 3.32
    },
    "level 2": {
      "task_goal_completion": 0,
      "scenario_goal_completion": 0,
      "interactions": 3.32
    },
    "level 3": {
      "task_goal_completion": 0,
      "scenario_goal_completion": 0,
      "interactions": 3.32
    }
  },
  "test_challenge": {
    "all": {
      "task_goal_completion": 5.5,
      "scenario_goal_completion": 4.3,
      "interactions": 4.39
    },
    "level 1": {
      "task_goal_completion": 26.4,
      "scenario_goal_completion": 20.8,
      "interactions": 4.39
    },
    "level 2": {
      "task_goal_completion": 2.7,
      "scenario_goal_completion": 2,
      "interactions": 4.39
    },
    "level 3": {
      "task_goal_completion": 0,
      "scenario_goal_completion": 0,
      "interactions": 4.39
    }
  }
}

@HarshTrivedi
Copy link
Member Author

/add-to-leaderboard --python 3.11 --appworld 0.1.3 temp_test

1 similar comment
@HarshTrivedi
Copy link
Member Author

/add-to-leaderboard --python 3.11 --appworld 0.1.3 temp_test

@HarshTrivedi
Copy link
Member Author

/add-to-leaderboard --python 3.11 --appworld 0.1.3 temp_test

@github-actions GitHub Actions
Copy link

Added leaderboard entry

{
    "id": "5509efd4-3e4bf4e2",
    "method": {
        "name": "temp",
        "tooltip": "temp"
    },
    "llm": {
        "name": "temp",
        "tooltip": "temp"
    },
    "url": "temp",
    "date": "2024-11-29",
    "test_normal": {
        "all": {
            "task_goal_completion": 9.5,
            "scenario_goal_completion": 7.1,
            "interactions": 3.32
        },
        "level 1": {
            "task_goal_completion": 28.1,
            "scenario_goal_completion": 21.1,
            "interactions": 3.32
        },
        "level 2": {
            "task_goal_completion": 0,
            "scenario_goal_completion": 0,
            "interactions": 3.32
        },
        "level 3": {
            "task_goal_completion": 0,
            "scenario_goal_completion": 0,
            "interactions": 3.32
        }
    },
    "test_challenge": {
        "all": {
            "task_goal_completion": 5.5,
            "scenario_goal_completion": 4.3,
            "interactions": 4.39
        },
        "level 1": {
            "task_goal_completion": 26.4,
            "scenario_goal_completion": 20.8,
            "interactions": 4.39
        },
        "level 2": {
            "task_goal_completion": 2.7,
            "scenario_goal_completion": 2,
            "interactions": 4.39
        },
        "level 3": {
            "task_goal_completion": 0,
            "scenario_goal_completion": 0,
            "interactions": 4.39
        }
    }
}

@HarshTrivedi
Copy link
Member Author

/remove-from-leaderboard 5509efd4-3e4bf4e2

2 similar comments
@HarshTrivedi
Copy link
Member Author

/remove-from-leaderboard 5509efd4-3e4bf4e2

@HarshTrivedi
Copy link
Member Author

/remove-from-leaderboard 5509efd4-3e4bf4e2

@HarshTrivedi
Copy link
Member Author

/add-to-leaderboard --python 3.11 --appworld 0.1.3 temp_test

@github-actions GitHub Actions
Copy link

Added to leaderboard:

[
{
"id": "5509efd4-3e4bf4e2",
"method": {
"name": "temp",
"tooltip": "temp"
},
"llm": {
"name": "temp",
"tooltip": "temp"
},
"url": "temp",
"date": "2024-11-29",
"test_normal": {
"all": {
"task_goal_completion": 9.5,
"scenario_goal_completion": 7.1,
"interactions": 3.32
},
"level 1": {
"task_goal_completion": 28.1,
"scenario_goal_completion": 21.1,
"interactions": 3.32
},
"level 2": {
"task_goal_completion": 0.0,
"scenario_goal_completion": 0.0,
"interactions": 3.32
},
"level 3": {
"task_goal_completion": 0.0,
"scenario_goal_completion": 0.0,
"interactions": 3.32
}
},
"test_challenge": {
"all": {
"task_goal_completion": 5.5,
"scenario_goal_completion": 4.3,
"interactions": 4.39
},
"level 1": {
"task_goal_completion": 26.4,
"scenario_goal_completion": 20.8,
"interactions": 4.39
},
"level 2": {
"task_goal_completion": 2.7,
"scenario_goal_completion": 2.0,
"interactions": 4.39
},
"level 3": {
"task_goal_completion": 0.0,
"scenario_goal_completion": 0.0,
"interactions": 4.39
}
}
}
]

@HarshTrivedi
Copy link
Member Author

/add-to-leaderboard --python 3.11 --appworld 0.1.3 temp_test

@github-actions GitHub Actions
Copy link

Added to leaderboard:

[
    {
        "id": "5509efd4-3e4bf4e2",
        "method": {
            "name": "temp",
            "tooltip": "temp"
        },
        "llm": {
            "name": "temp",
            "tooltip": "temp"
        },
        "url": "temp",
        "date": "2024-11-29",
        "test_normal": {
            "all": {
                "task_goal_completion": 9.5,
                "scenario_goal_completion": 7.1,
                "interactions": 3.32
            },
            "level 1": {
                "task_goal_completion": 28.1,
                "scenario_goal_completion": 21.1,
                "interactions": 3.32
            },
            "level 2": {
                "task_goal_completion": 0.0,
                "scenario_goal_completion": 0.0,
                "interactions": 3.32
            },
            "level 3": {
                "task_goal_completion": 0.0,
                "scenario_goal_completion": 0.0,
                "interactions": 3.32
            }
        },
        "test_challenge": {
            "all": {
                "task_goal_completion": 5.5,
                "scenario_goal_completion": 4.3,
                "interactions": 4.39
            },
            "level 1": {
                "task_goal_completion": 26.4,
                "scenario_goal_completion": 20.8,
                "interactions": 4.39
            },
            "level 2": {
                "task_goal_completion": 2.7,
                "scenario_goal_completion": 2.0,
                "interactions": 4.39
            },
            "level 3": {
                "task_goal_completion": 0.0,
                "scenario_goal_completion": 0.0,
                "interactions": 4.39
            }
        }
    }
]

@HarshTrivedi
Copy link
Member Author

/add-to-leaderboard --python 3.11 --appworld 0.1.3 temp_test

@HarshTrivedi
Copy link
Member Author

/add-to-leaderboard --python 3.11 --appworld 0.1.3 temp_test

@HarshTrivedi
Copy link
Member Author

/add-to-leaderboard --python 3.11 --appworld 0.1.3 temp_test

@github-actions GitHub Actions
Copy link

Added to leaderboard:

[
    {
        "id": "5509efd4-3e4bf4e2",
        "method": {
            "name": "temp",
            "tooltip": "temp"
        },
        "llm": {
            "name": "temp",
            "tooltip": "temp"
        },
        "url": "temp",
        "date": "2024-11-29",
        "test_normal": {
            "all": {
                "task_goal_completion": 9.5,
                "scenario_goal_completion": 7.1,
                "interactions": 3.32
            },
            "level 1": {
                "task_goal_completion": 28.1,
                "scenario_goal_completion": 21.1,
                "interactions": 3.32
            },
            "level 2": {
                "task_goal_completion": 0.0,
                "scenario_goal_completion": 0.0,
                "interactions": 3.32
            },
            "level 3": {
                "task_goal_completion": 0.0,
                "scenario_goal_completion": 0.0,
                "interactions": 3.32
            }
        },
        "test_challenge": {
            "all": {
                "task_goal_completion": 5.5,
                "scenario_goal_completion": 4.3,
                "interactions": 4.39
            },
            "level 1": {
                "task_goal_completion": 26.4,
                "scenario_goal_completion": 20.8,
                "interactions": 4.39
            },
            "level 2": {
                "task_goal_completion": 2.7,
                "scenario_goal_completion": 2.0,
                "interactions": 4.39
            },
            "level 3": {
                "task_goal_completion": 0.0,
                "scenario_goal_completion": 0.0,
                "interactions": 4.39
            }
        }
    }
]

@HarshTrivedi
Copy link
Member Author

/add-to-leaderboard --python 3.11 --appworld 0.1.3 temp_test

@github-actions GitHub Actions
Copy link

Added to leaderboard:

[
    {
        "id": "5509efd4-3e4bf4e2",
        "method": {
            "name": "temp",
            "tooltip": "temp"
        },
        "llm": {
            "name": "temp",
            "tooltip": "temp"
        },
        "url": "temp",
        "date": "2024-11-29",
        "test_normal": {
            "all": {
                "task_goal_completion": 9.5,
                "scenario_goal_completion": 7.1,
                "interactions": 3.32
            },
            "level 1": {
                "task_goal_completion": 28.1,
                "scenario_goal_completion": 21.1,
                "interactions": 3.32
            },
            "level 2": {
                "task_goal_completion": 0.0,
                "scenario_goal_completion": 0.0,
                "interactions": 3.32
            },
            "level 3": {
                "task_goal_completion": 0.0,
                "scenario_goal_completion": 0.0,
                "interactions": 3.32
            }
        },
        "test_challenge": {
            "all": {
                "task_goal_completion": 5.5,
                "scenario_goal_completion": 4.3,
                "interactions": 4.39
            },
            "level 1": {
                "task_goal_completion": 26.4,
                "scenario_goal_completion": 20.8,
                "interactions": 4.39
            },
            "level 2": {
                "task_goal_completion": 2.7,
                "scenario_goal_completion": 2.0,
                "interactions": 4.39
            },
            "level 3": {
                "task_goal_completion": 0.0,
                "scenario_goal_completion": 0.0,
                "interactions": 4.39
            }
        }
    }
]

@HarshTrivedi
Copy link
Member Author

/add-to-leaderboard --python 3.11 --appworld 0.1.3 temp_test

@github-actions GitHub Actions
Copy link

Added to leaderboard:

[
    {
        "id": "5509efd4-3e4bf4e2",
        "method": {
            "name": "temp",
            "tooltip": "temp"
        },
        "llm": {
            "name": "temp",
            "tooltip": "temp"
        },
        "url": "temp",
        "date": "2024-11-29",
        "test_normal": {
            "all": {
                "task_goal_completion": 9.5,
                "scenario_goal_completion": 7.1,
                "interactions": 3.32
            },
            "level 1": {
                "task_goal_completion": 28.1,
                "scenario_goal_completion": 21.1,
                "interactions": 3.32
            },
            "level 2": {
                "task_goal_completion": 0.0,
                "scenario_goal_completion": 0.0,
                "interactions": 3.32
            },
            "level 3": {
                "task_goal_completion": 0.0,
                "scenario_goal_completion": 0.0,
                "interactions": 3.32
            }
        },
        "test_challenge": {
            "all": {
                "task_goal_completion": 5.5,
                "scenario_goal_completion": 4.3,
                "interactions": 4.39
            },
            "level 1": {
                "task_goal_completion": 26.4,
                "scenario_goal_completion": 20.8,
                "interactions": 4.39
            },
            "level 2": {
                "task_goal_completion": 2.7,
                "scenario_goal_completion": 2.0,
                "interactions": 4.39
            },
            "level 3": {
                "task_goal_completion": 0.0,
                "scenario_goal_completion": 0.0,
                "interactions": 4.39
            }
        }
    }
]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@HarshTrivedi