BART fine-tuning

[minji-o-j]

best checkpoint기준 실험 (멈춘것 -3)
**bart finetuning**시 batch 또 조정해줘야함

QKV 학습 버전 (tag: QKVo_#num)

• 1번 서버 5 gpu 01

accelerate launch run_textbox.py --model=PTG --dataset=pc --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_task1 --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-pc-2023-Jul-26_05-16-58_QKVo_1/checkpoint_epoch-11 --train_batch_size=4 --accumulation_steps=24

• cross_task1-_17-03-16
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_task-pc/runs/o7eq711a?workspace=user-minji913

• 5번 서버 5 gpu23

accelerate launch run_textbox.py --model=PTG --dataset=pc --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset2 --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-pc-2023-Jul-26_04-56-03_QKVo_5/checkpoint_epoch-3 --train_batch_size=4 --accumulation_steps=24

• cross_dataset2--59-40
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-pc/runs/bgw7rj4b?workspace=user-minji913

• 9번 서버 5

accelerate launch run_textbox.py --model=PTG --dataset=pc --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset3_paper --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-pc-2023-Jul-26_04-59-25_QKVo_9/checkpoint_epoch-5 --train_batch_size=4 --accumulation_steps=24

• cross_dataset3_paper--33-22
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-pc/runs/oxxervv3?workspace=user-minji913

• 6번 서버 3 gpu 23

accelerate launch run_textbox.py --model=PTG --dataset=dd  --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset2 --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-dd-2023-Jul-26_04-59-17_QKVo_6/checkpoint_epoch-4 --train_batch_size=2 --accumulation_steps=48

• cross_dataset2--24-06
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-dd/runs/s5pcvxk8?workspace=user-minji913

• 10번 서버 3 gpu 23

accelerate launch run_textbox.py --model=PTG --dataset=dd --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset4_paper --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-dd-2023-Jul-26_12-31-16_QKVo_10/checkpoint_epoch-1 --train_batch_size=2 --accumulation_steps=48

• cross_dataset4_paper--22-23
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-dd/runs/jzh9mxwh?workspace=user-minji913

[minji-o-j]

QKV 학습X 버전 (tag: QKVx_#num)

• 1번 서버 5

accelerate launch run_textbox.py --model=PTG --dataset=pc --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_task1 --training_option=BART-finetuning --QKV_training=False --learning_rate=3e-5 --attention_path=PTG-pc-2023-Jul-28_14-55-31_QKVx_1/checkpoint_epoch-3 --train_batch_size=4 --accumulation_steps=24

• cross_task1-_05-06-02
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_task-pc/runs/l7s7irry?workspace=user-minji913

• 5번 서버 5 gpu 01

accelerate launch run_textbox.py --model=PTG --dataset=pc --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset2 --training_option=BART-finetuning --QKV_training=False --learning_rate=3e-5 --attention_path=PTG-pc-2023-Jul-26_18-38-20_QKVx_5/checkpoint_epoch-2 --train_batch_size=4 --accumulation_steps=24

• cross_dataset2--48-31
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-pc/runs/ny9czl0d?workspace=user-minji913

• 9번 서버 5 23

accelerate launch run_textbox.py --model=PTG --dataset=pc --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset3_paper  --training_option=BART-finetuning --QKV_training=False --learning_rate=3e-5 --attention_path=PTG-pc-2023-Jul-28_08-42-47_QKVx_9/checkpoint_epoch-13 --train_batch_size=4 --accumulation_steps=24

• cross_dataset3_paper--48-36
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-pc/runs/ci1ozmv2?workspace=user-minji913

• 6번 서버 5 gpu 5,6

accelerate launch run_textbox.py --model=PTG --dataset=dd  --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset2 --training_option=BART-finetuning --QKV_training=False --learning_rate=3e-5 --attention_path=PTG-dd-2023-Jul-27_10-17-18_QKVx_6/checkpoint_epoch-10 --train_batch_size=2 --accumulation_steps=48

• cross_dataset2--13-36
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-dd/runs/xsnmbefv?workspace=user-minji913

• 10번 서버5 gpu 2,3

accelerate launch run_textbox.py --model=PTG --dataset=dd --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset4_paper --training_option=BART-finetuning --QKV_training=False --learning_rate=3e-5 --attention_path=PTG-dd-2023-Jul-29_13-34-25_QKVx_10/checkpoint_epoch-1 --train_batch_size=2 --accumulation_steps=48

• cross_dataset4_paper--38-57
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-dd/runs/n0khes8g?workspace=user-minji913

[minji-o-j]

• 4번 서버5 gpu0123 진행중

accelerate launch run_textbox.py --model=PTG --dataset=xsum --model_path=facebook/bart-large --gpu_id=0,1,2,3 --find_unused_parameters=true --source_task=cross_dataset1 --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-xsum-2023-Aug-14_03-53-23_QKVo_4/checkpoint_epoch-8 --train_batch_size=2 --accumulation_steps=48 --eval_batch_size=8

• 8번 서버5 gpu 4567

accelerate launch run_textbox.py --model=PTG --dataset=xsum --model_path=facebook/bart-large --gpu_id=0,1,2,3 --find_unused_parameters=true --source_task=cross_dataset2_paper --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-xsum-2023-Aug-14_03-53-29_QKVo_8/checkpoint_epoch-1 --train_batch_size=2 --accumulation_steps=48 --eval_batch_size=8

• cross_dataset2_paper--33-22
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-xsum/runs/unv3l5o8?workspace=user-minji913

8번 prompt1 말고
accelerate launch run_textbox.py --model=PTG --dataset=xsum --model_path=facebook/bart-large --gpu_id=0,1,2,3 --find_unused_parameters=true --source_task=cross_dataset2_paper --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-xsum-2023-Aug-14_03-53-29_QKVo_8/checkpoint_epoch-4 --train_batch_size=2 --accumulation_steps=48 --eval_batch_size=8