AttributeError: 'SGD' object has no attribute 'train'

[SangbumChoi]

System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

• transformers version: 4.46.0.dev0

• Platform: Linux-5.15.0-100-generic-x86_64-with-glibc2.35

• Python version: 3.10.14

• Huggingface_hub version: 0.24.7

• Safetensors version: 0.4.5

• Accelerate version: 0.33.0

• Accelerate config: not found

• PyTorch version (GPU?): 2.2.2 (True)

• Tensorflow version (GPU?): not installed (NA)

• Flax version (CPU?/GPU?/TPU?): not installed (NA)

• Jax version: not installed

• JaxLib version: not installed

• Using distributed or parallel set-up in script?:

• Using GPU in script?:

• GPU type: NVIDIA A100 80GB PCIe

• Accelerate version: 0.33.0

• Platform: Linux-5.15.0-100-generic-x86_64-with-glibc2.35

• accelerate bash location: /opt/conda/bin/accelerate

• Python version: 3.10.14

• Numpy version: 1.26.4

• PyTorch version (GPU?): 2.2.2 (True)

• PyTorch XPU available: False

• PyTorch NPU available: False

• PyTorch MLU available: False

• PyTorch MUSA available: False

• System RAM: 755.86 GB

• GPU type: NVIDIA A100 80GB PCIe

• Accelerate default config:
  Not found

Who can help?

@muellerzr @SunMarc

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

run_object_decection.py

Expected behavior

After adding

        if hasattr(self.optimizer, "train") and callable(self.optimizer.train):
            self.optimizer.train()

It returns the following error when the input of optimizers is raw pytorch optimizer

    optimizer = torch.optim.AdamW(
        parameter,
        lr=cfg.lr,
        weight_decay=cfg.weight_decay,
    )

[LysandreJik]

Also cc @qubvel regarding object detction

[SangbumChoi]

I assume that this error will not be stricted to vision area, but not sure

[SangbumChoi]

https://github.com/SangbumChoi/transformers/blob/e783f12f204ead2906ecd9c8c1196650b4c4adad/src/transformers/trainer.py#L354

[SangbumChoi]

potential related issue
huggingface/accelerate#2631
#30079

[qubvel]

Hi @SangbumChoi, I also faced this issue with the outdated version of Accelerate, updating it should help!

[SangbumChoi]

@qubvel Can you tell me which version should I use? I am using 0.33.0

[qubvel]

Could you try with 0.34?

[SangbumChoi]

@qubvel Thanks it works :)