when resume from peft checkpoint, the model should be trainable

[sywangyi]

model.training is false after model.load_adapter of peft. see https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L402, default value of is_trainable is False

• trainer: @sgugger

[sywangyi]

@younesbelkada please help review

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[younesbelkada]

Thank you for adding this, this looks good, however I am not sure whether we should merge this as in any case it'll lead to the same behavior (i.e. I don't think the PR is technically fixing an issue). the impact of is_trainable is to activate dropout: https://github.com/huggingface/peft/blob/8af8dbd2ec9b4b8f664541e9625f898db7c7c78f/src/peft/peft_model.py#L513 which can be activated by just doing model.train() which is done anyway by the trainer here: https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py#L2632

cc @pacman100 as well

EDIT: this seems great, some PEFT adapters have inference_mode in the config that freezes the adapter layers: https://github.com/huggingface/peft/blob/8af8dbd2ec9b4b8f664541e9625f898db7c7c78f/src/peft/peft_model.py#L197 So this PR definitely fixes an issue IMO

[younesbelkada]

This looks great IMO, updated my comment above! Thanks!

[sgugger]

Thanks for the fix!