support deberta v3

Tiiiger · Dec 10, 2021 · 0603cbd · 0603cbd
1 parent 8d44ab3
commit 0603cbd
Show file tree

Hide file tree

Showing 2 changed files with 9 additions and 1 deletion.
diff --git a/README.md b/README.md
@@ -8,7 +8,9 @@
 Automatic Evaluation Metric described in the paper [BERTScore: Evaluating Text Generation with BERT](https://arxiv.org/abs/1904.09675) (ICLR 2020). We now support about 130 models (see this [spreadsheet](https://docs.google.com/spreadsheets/d/1RKOVpselB98Nnh_EOC4A2BYn8_201tmPODpNWu4w7xI/edit?usp=sharing) for their correlations with human evaluation). Currently, the best model is `microsoft/deberta-xlarge-mnli`, please consider using it instead of the default `roberta-large` in order to have the best correlation with human evaluation.
 
 #### News:
-- Features to appear in the next version (currently in the master branch):
+<!-- - Features to appear in the next version (currently in the master branch): -->
+- Updated to version 0.3.11
+  - Support 6 DeBERTa v3 models
   - Support 3 ByT5 models
 - Updated to version 0.3.10
   - Support 8 SimCSE models

diff --git a/bert_score/utils.py b/bert_score/utils.py
@@ -176,6 +176,12 @@
     'google/byt5-small': 1, # 0.5100025975052146
     'google/byt5-base': 17, # 0.5810347173565313
     'google/byt5-large': 30, # 0.6151895697554877
+    'microsoft/deberta-v3-xsmall': 10, # 0.6941803815412021
+    'microsoft/deberta-v3-small': 4, # 0.6651551203179679
+    'microsoft/deberta-v3-base': 9, # 0.7261586651018335
+    'microsoft/mdeberta-v3-base': 10, # 0.6778713684091584
+    'microsoft/deberta-v3-large': 12, # 0.6927693082293821
+    'khalidalt/DeBERTa-v3-large-mnli': 18, # 0.7428756686018376
 }