Code

Add CRAG benchmark#88

Code

Merged

minmin-intel merged 20 commits intoopea-project:mainopea-project/GenAIEval:mainfrom minmin-intel:crag-evalminmin-intel/GenAIEval:crag-evalCopy head branch name to clipboardSep 10, 2024

+649-18

Commits

Commits on Aug 22, 2024

• add crag eval first pass code

  minmin-intel

  committed
  be54803

  Copy full SHA for be54803

Commits on Aug 23, 2024

• add first pass llm eval code

  minmin-intel

  committed
  6da78fe

  Copy full SHA for 6da78fe

Commits on Aug 26, 2024

• fix answer correctness code

  Show description for d57f9d5
  minmin-intel

  committed
  d57f9d5

  Copy full SHA for d57f9d5

• docker container for crag eval

  minmin-intel

  committed
  a61efa8

  Copy full SHA for a61efa8

• sample data for testing

  minmin-intel

  committed
  914cdc2

  Copy full SHA for 914cdc2

• docker compose for tgi gaudi

  Show description for 1b49b1b
  minmin-intel

  committed
  1b49b1b

  Copy full SHA for 1b49b1b

• fix tgi gaudi docker compose for llama3 70b

  minmin-intel

  committed
  7867461

  Copy full SHA for 7867461

• update llm eval code

  Show description for 8e999dc
  minmin-intel

  committed
  8e999dc

  Copy full SHA for 8e999dc

Commits on Aug 27, 2024

• allow per sample grading

  Show description for 7b9b9b2
  minmin-intel

  committed
  7b9b9b2

  Copy full SHA for 7b9b9b2

• save graded scores

  Show description for 5c58b72
  minmin-intel

  committed
  5c58b72

  Copy full SHA for 5c58b72

• ipdate readme

  Show description for d043dde
  minmin-intel

  committed
  d043dde

  Copy full SHA for d043dde

• Merge branch 'main' into crag-eval

  minmin-intel

  authored
  ba55652

  Copy full SHA for ba55652

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 9556f9a
  pre-commit-ci[bot]

  committed
  9556f9a

  Copy full SHA for 9556f9a

• update readme and test all commands

  minmin-intel

  committed
  90e855b

  Copy full SHA for 90e855b

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 153e30f
  pre-commit-ci[bot]

  committed
  153e30f

  Copy full SHA for 153e30f

Commits on Sep 9, 2024

• mv crag_eval to agent_eval

  Show description for d51c083
  minmin-intel

  committed
  d51c083

  Copy full SHA for d51c083

• Merge branch 'main' into crag-eval

  minmin-intel

  authored
  e77e58c

  Copy full SHA for e77e58c

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for e4cd9c6
  pre-commit-ci[bot]

  committed
  e4cd9c6

  Copy full SHA for e4cd9c6

• update test case col names in grade_answer.py

  Show description for 69f87c2
  minmin-intel

  committed
  69f87c2

  Copy full SHA for 69f87c2

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 82894aa
  pre-commit-ci[bot]

  committed
  82894aa

  Copy full SHA for 82894aa