Evaluate prompts for topic classification

[InMatrix]

• Create an eval prototype with dummy data (tag)
• Create an evaluation dataset
  • Use the metadata of the 50 papers retrieved from 2024-09-20
  • Update eval config file with the actual topics of interest
  • Create golden responses through manually selecting papers that match the topics of interest.
  • Save the golden results
• Use the promptfoo eval tool to compute prediction metrics such as recall, precision, and f1.
• Realign the prompt developed through the eval process and the prompt in production