freeze_backbone and freeze_decoder in Trainers

[isaaccorley]

This PR adds a freeze_backbone arg to all trainers and a freeze_decoder arg to the SemanticSegmentationTask and PixelwiseRegressionTask. This is particularly useful if wanting to fine-tune (aka linear probe) only the classifier head of a pretrained model or only the decoder or segmentation head in an encoder/decoder architecture.

[adamjstewart]

Can you do other trainers too? For this paper, all of our tasks will be semantic segmentation, but might as well do all trainers if it's easy.

[calebrob6]

sick

[adamjstewart]

Not sure about the parameter names. Also not sure if freezing the ResNet encoder in a U-Net counts as linear probing or not. Same with classification/regression. Might be easier just to freeze the entire model and add a single layer MLP to the end.

[adamjstewart]

Not documented.

[isaaccorley]

Can you reference some papers that freeze the entire UNet and add fine-tune an MLP on top? I have never seen this before. Also it would make more sense to fine tune a conv layer not a MLP.

Could you also suggest better parameter names?

[adamjstewart]

I'm only talking about classification/regression. No idea what the usual practice is for linear probing of semantic segmentation.

Will think about parameter names, don't have a better name off the top of my head. I still kind of like linear_probing because it makes it clear why you would want to use the parameter.

[isaaccorley]

Disagree about the naming. Freeze_backbone is the action we are performing. Linear probing is just the downstream task a user may choose to perform with the classifier with a frozen backbone. There may be other potential reasons for freezing the backbone other than linear probing.