Klue with RobertaGCN

Result

Not fold valid.

Model	KLUE-ynat
RoBERTaGCN	87.25

5-fold Avg. (date. 2023/04/09)

Model	KLUE-ynat
BERT	86.31
RoBERTa	85.90
BertGCN	86.39
RoBERTaGCN	86.25
BertGCN_TBGC	86.99
RoBERTaGCN_TBGC	86.72

How to Use

Model

1. Run klue_data_convert.py

2. Run build_graph.py

3. Run robertagcn_klue.py

Requirements

• dgl-cu113 == 0.9.1.post1
• ignite == 1.1.0
• python == 3.6.9
• torch == 1.10.0+cu113
• scikit-learn =< 0.24.2
• transformers =< 4.18.0
• numpy =< 1.19.5
• networkx == 2.5.1

Problem

When constructing an adjacency matrix, there is a data leakage problem. (The paperwithcode doesn't seem to consider that issue.)

Reference

Backbone Model

• klue/roberta-base

Dataset

• KLUE-ynat

Citation

• BertGCN: Transductive Text Classification by Combining GCN and BERT
• KLUE: Korean Language Understanding Evaluation

Workers

노영준

• Change build graph method
• Preprocessing

박정원

• Finetune RoBERTaGCN