We provide training code, commands and requirements to enable running various sprase algorithms on SMC-Bench here. Please check INSTALL.md for installation instructions first.
We heavily rely on the awesome repo fairseq from FACEBOOK for Commonsense Reasoning.
The full instructions for CommonsenseQA can be found here.
More specifially, we need to
- cd SMC-Bench/Commonsense_Reasoning/
- pip install --editable ./
- Download CommonsenseQA dataset: bash examples/roberta/commonsense_qa/download_cqa_data.sh
- Download roberta.large from here.
- Densely or sparsely finetuning with the commands provided here.
The dense training on WinoGrande dataset can be found here. To enable sparse training, or pruning, just follow the commands of CommonsenseQA.
The dense training on WinoGrande dataset can be found here. To enable sparse training, or pruning, just follow the commands of CommonsenseQA.