Skip to content

Commit

Permalink
Merge branch 'embeddings-benchmark:main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
@imenelydiaker
imenelydiaker authored May 21, 2024
2 parents fbf2fb7 + 08eb58b commit 71b601a
Show file tree
Hide file tree
Showing 285 changed files with 352,038 additions and 13,563 deletions.
2 changes: 1 addition & 1 deletion .github/pull_request_template.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Reason for dataset addition:


- [ ] I have tested that the dataset runs with the `mteb` package.
- [ ] I have run the following models on the task (adding the results to the pr). These can be run using the `mteb run -m {model_name} -t {task_name}` command.
- [ ] I have run the following models on the task (adding the results to the pr). These can be run using the `mteb -m {model_name} -t {task_name}` command.
- [ ] `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2`
- [ ] `intfloat/multilingual-e5-small`
- [ ] I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
Expand Down
4 changes: 4 additions & 0 deletions docs/mmteb/points.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,13 +71,17 @@ Please also add your first name and last name are as you want them to appear in
| Akash190104 | Akash | Kundu | [email protected] |~Akash_Kundu2 | Heritage Institute of Technology, Kolkata && Apart Research |
| dwzhu-pku | Dawei | Zhu | [email protected] | ~Dawei_Zhu2 | Peking University |
| ljvmiranda921 | Lester James | Miranda | [email protected] | ~Lester_James_Validad_Miranda1 | Allen Institute for AI |
| Sakshamrzt | Saksham | Thakur | [email protected] | ~Saksham_Thakur1 | N/A |
| Andrian0s | Andrianos | Michail | [email protected] | ~Andrianos_Michail1 | University of Zurich|
| simon-clematide | Simon | Clematide | [email protected] | ~Simon_Clematide1 | University of Zurich|
| SaitejaUtpala | Saiteja | Utpala | [email protected] | ~Saiteja_Utpala1 | Microsoft Research|
| mmhamdy | Mohammed | Hamdy | [email protected] | ~Mohammed_Hamdy1 | Cohere For AI Community|
| jupyterjazz | Saba | Sturua | [email protected] | ~Saba_Sturua1 | Jina AI |
| Ruqyai | Ruqiya | Bin Safi | [email protected] | ~Ruqiya_Bin_Safi1 | LIBF : The London Institute of Banking & Finance |
| kranthigv | Kranthi Kiran | GV | [email protected] | ~Kranthi_Kiran_GV1 | New York University|
| jupyterjazz | Saba | Sturua | [email protected] | ~Saba_Sturua1 | Jina AI
| shreeya-dhakal | Shreeya | Dhakal | [email protected] | | Individual Contributor |
| dipam7 | Dipam | Vasani | [email protected] | ~Dipam_Vasani1 | Individual Contributor |
| Art3mis07 | Gayatri | K | [email protected] | ~Gayatri_K1 | R. V. College of Engineering, Bengaluru |
| jankounchained | Jan | Kostkan | [email protected] | ~Jan_Kostkan1 | Aarhus University, Denmark |
| bp-high | Bhavish | Pahwa | [email protected] | ~Bhavish_Pahwa1 | Microsoft Research |
3 changes: 3 additions & 0 deletions docs/mmteb/points/512.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"GitHub": "shreeya-dhakal", "New dataset": 2}
{"GitHub": "asparius", "Review PR": 2}
{"GitHub": "KennethEnevoldsen", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/546.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "dokato", "New dataset": 2}
{"GitHub": "isaac-chung", "Review PR": 2}
3 changes: 3 additions & 0 deletions docs/mmteb/points/588.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"GitHub": "shreeya-dhakal", "New dataset": 2}
{"GitHub": "isaac-chung", "Review PR": 2}
{"GitHub": "dipam7", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/610.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "Sakshamrzt", "New dataset": 2}
{"GitHub": "imenelydiaker", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/635.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "loicmagne", "Bug fixes": 20}
{"GitHub": "KennethEnevoldsen", "Review PR": 2}
1 change: 1 addition & 0 deletions docs/mmteb/points/661.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"GitHub": "awinml", "New dataset": 20}
1 change: 1 addition & 0 deletions docs/mmteb/points/665.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"GitHub": "awinml", "New dataset": 20}
5 changes: 5 additions & 0 deletions docs/mmteb/points/674.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{"GitHub": "Ruqyai", "New dataset": 2}
{"GitHub": "imenelydiaker", "Review PR": 2}
{"GitHub": "mrshu", "Review PR": 2}
{"GitHub": "KranthiGV", "Review PR": 2}
{"GitHub": "KennethEnevoldsen", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/678.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "awinml", "New dataset": 20}
{"GitHub": "KennethEnevoldsen", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/695.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "mrshu", "New dataset": 8}
{"GitHub": "x-tabdeveloping", "Review PR": 2}
3 changes: 3 additions & 0 deletions docs/mmteb/points/697.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"GitHub": "mrshu", "New dataset": 4}
{"GitHub": "x-tabdeveloping", "Review PR": 2}
{"GitHub": "KennethEnevoldsen", "Review PR": 2}
4 changes: 4 additions & 0 deletions docs/mmteb/points/714.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{"GitHub": "vaibhavad", "New dataset": 2}
{"GitHub": "vaibhavad", "Bug fixes": 4}
{"GitHub": "KennethEnevoldsen", "Review PR": 2}
{"GitHub": "orionw", "Review PR": 2}
3 changes: 3 additions & 0 deletions docs/mmteb/points/716.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"GitHub": "mrshu", "New dataset": 22}
{"GitHub": "KranthiGV", "Review PR": 2}
{"GitHub": "isaac-chung", "Review PR": 2}
3 changes: 3 additions & 0 deletions docs/mmteb/points/724.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"GitHub": "Art3mis07", "New dataset": 12}
{"GitHub": "imenelydiaker", "Review PR": 2}
{"GitHub": "kranthigv", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/727.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "mrshu", "New dataset": 2}
{"GitHub": "KennethEnevoldsen", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/729.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "isaac-chung", "New dataset": 4}
{"GitHub": "KennethEnevoldsen", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/730.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "wissam-sib", "New dataset": 2}
{"GitHub": "imenelydiaker", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/731.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "malteos", "New dataset": 2}
{"GitHub": "KennethEnevoldsen", "Review PR": 2}
3 changes: 3 additions & 0 deletions docs/mmteb/points/733.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"GitHub": "bp-high", "New dataset": 26}
{"GitHub": "isaac-chung", "Review PR": 2}
{"GitHub": "KranthiGV", "Review PR": 2}
3 changes: 3 additions & 0 deletions docs/mmteb/points/735.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"GitHub": "malteos", "New dataset": 2}
{"GitHub": "isaac-chung", "Review PR": 2}
{"GitHub": "Ruqyai", "Review PR": 2}
3 changes: 3 additions & 0 deletions docs/mmteb/points/736.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"GitHub": "wissam-sib", "New dataset": 6}
{"GitHub": "isaac-chung", "Review PR": 2}
{"GitHub": "kranthigv", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/740.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "shreeya-dhakal", "New dataset": 4}
{"GitHub": "isaac-chung", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/745.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "isaac-chung", "New dataset": 2, "Bug fixes": 2}
{"GitHub": "x-tabdeveloping", "Review PR": 2}
3 changes: 3 additions & 0 deletions docs/mmteb/points/750.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"GitHub": "xhluca", "New dataset": 2}
{"GitHub": "isaac-chung", "Review PR": 2}
{"GitHub": "wissam-sib", "Review PR": 2}
4 changes: 4 additions & 0 deletions docs/mmteb/points/757.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{"GitHub": "isaac-chung", "Bug fixes": 8}
{"GitHub": "KennethEnevoldsen", "Review PR": 2}
{"GitHub": "Ruqyai", "Review PR": 2}
{"GitHub": "x-tabdeveloping", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/758.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "isaac-chung", "Review PR": 2}
{"GitHub": "mrshu", "Bug fixes": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/759.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "KennethEnevoldsen", "Bug fixes": 6}
{"GitHub": "x-tabdeveloping", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/767.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "isaac-chung", "Bug fixes": 2}
{"GitHub": "x-tabdeveloping", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/768.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "awinml", "New dataset": 8}
{"GitHub": "isaac-chung", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/769.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "awinml", "New dataset": 2}
{"GitHub": "isaac-chung", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/770.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "KennethEnevoldsen", "Bug fixes": 6}
{"GitHub": "x-tabdeveloping", "Review PR": 2}
2 changes: 2 additions & 0 deletions docs/mmteb/points/772.jsonl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"GitHub": "isaac-chung", "New dataset": 2, "Bug fixes": 2}
{"GitHub": "KennethEnevoldsen", "Review PR": 2}
Loading

0 comments on commit 71b601a

Please sign in to comment.