Code

Add new datasets from ir_datasets#2

Code

Merged

bpiwowar merged 22 commits intoexperimaestro:masterexperimaestro/datamaestro_text:masterfrom MathVast:add-new-datasets-from-ir-datasetsCopy head branch name to clipboardOct 27, 2023

+204-3

Commits

Commits on Jul 20, 2023

• Add BEIR NQ dataset

  

  mat_vast

  committed
  aa89d8d

  Copy full SHA for aa89d8d

• Add BEIR NFCorpus dataset (still need to find a workaround for the queries with the train/dev/test sub-version)

  

  mat_vast

  committed
  c211a0b

  Copy full SHA for c211a0b

Commits on Jul 24, 2023

• Change the content of the method get_text for TrecTopic

  

  mat_vast

  committed
  6363ebe

  Copy full SHA for 6363ebe

Commits on Jul 26, 2023

• Add another MsMarco and TREC-Covid format for MLM

  

  mat_vast

  committed
  680d6a1

  Copy full SHA for 680d6a1

• Fix the format for CordFullTextDocument

  

  mat_vast

  committed
  6cc3dc3

  Copy full SHA for 6cc3dc3

• Add new formats to account for nfcorpus outside of the scope of BEIR

  

  mat_vast

  committed
  8d9e548

  Copy full SHA for 8d9e548

Commits on Jul 28, 2023

• fit the ir_dataset new id msmarco-passagev2/dedup which has no description yet

  yzong12138

  committed
  612cc7a

  Copy full SHA for 612cc7a

• add the data structure for robust-04

  yzong12138

  committed
  923797a

  Copy full SHA for 923797a

• Add Trec-News compatibility

  

  mat_vast

  committed
  b581e3f

  Copy full SHA for b581e3f

Commits on Sep 20, 2023

• Update formats and data

  

  mat_vast

  committed
  adb9e3f

  Copy full SHA for adb9e3f

• Revert "Update formats and data"

  Show description for 46b93d6
  

  mat_vast

  committed
  46b93d6

  Copy full SHA for 46b93d6

• add for the dataset tweet13

  yzong12138

  committed
  d604058

  Copy full SHA for d604058

• fix

  yzong12138

  committed
  64169c8

  Copy full SHA for 64169c8

Commits on Sep 25, 2023

• Merge remote-tracking branch 'origin/master' into add-new-datasets-from-ir-datasets

  

  mat_vast

  committed
  1ed07fb

  Copy full SHA for 1ed07fb

• Merge branch 'add-new-datasets-from-ir-datasets' of https://github.com/MathVast/datamaestro_text into add-new-datasets-from-ir-datasets

  

  mat_vast

  committed
  373f377

  Copy full SHA for 373f377

Commits on Sep 27, 2023

• Merge remote-tracking branch 'origin/master' into add-new-datasets-from-ir-datasets

  

  mat_vast

  committed
  20efd05

  Copy full SHA for 20efd05

Commits on Oct 6, 2023

• Merge branch 'experimaestro:master' into add-new-datasets-from-ir-datasets

  yzong12138

  authored
  89f1717

  Copy full SHA for 89f1717

Commits on Oct 25, 2023

• Small fix of the StoreTrainingTripletDocumentAdapter iter function

  

  mat_vast

  committed
  9c5482a

  Copy full SHA for 9c5482a

• Merge origin/master into fork's branch

  

  mat_vast

  committed
  39cc51d

  Copy full SHA for 39cc51d

• Merge branch 'add-new-datasets-from-ir-datasets' of https://github.com/MathVast/datamaestro_text into add-new-datasets-from-ir-datasets

  

  mat_vast

  committed
  9f87c6a

  Copy full SHA for 9f87c6a

• Update datasets.py

  

  mat_vast

  committed
  975af79

  Copy full SHA for 975af79

Commits on Oct 26, 2023

• Delete the __main__ part

  

  mat_vast

  committed
  f02f55e

  Copy full SHA for f02f55e