Model loading extremely slow after upgrading from 0.31.0 to 0.32.0

[chenyuteng01]

System Info

- `Accelerate` version: 0.32.0
- Platform: Linux-6.5.0-28-generic-x86_64-with-glibc2.35
- `accelerate` bash location: /home/chenyuteng/workspace/stt/.venv/bin/accelerate
- Python version: 3.10.12
- Numpy version: 1.26.4
- PyTorch version (GPU?): 2.3.1+cu121 (True)
- PyTorch XPU available: False
- PyTorch NPU available: False
- PyTorch MLU available: False
- System RAM: 62.47 GB
- GPU type: NVIDIA GeForce RTX 3090
- `Accelerate` default config:
        Not found

Information

• The official example scripts
• My own modified scripts

Tasks

• One of the scripts in the examples/ folder of Accelerate or an officially supported no_trainer script in the examples folder of the transformers repo (such as run_no_trainer_glue.py)
• My own task or dataset (give details below)

Reproduction

import torch
import transformers
import librosa

class STT:
    model: transformers.WhisperForConditionalGeneration
    processor: transformers.WhisperProcessor

    def __init__(self, model_path):
        self.model = transformers.WhisperForConditionalGeneration.from_pretrained(model_path, torch_dtype=torch.bfloat16, device_map="auto")
        self.processor = transformers.WhisperProcessor.from_pretrained(model_path)

s = STT("path_to_the_model")

Expected behavior

Before upgrading, my accelerate version is 0.31.0 and this code will finish within 5 seconds. But now I have to wait for more than 30s. If I downgrade to 0.31.0, this problem will be fixed.

[chenyuteng01]

My package version as follows:
accelerate==0.32.0
transformers==4.42.3
torch==2.3.1

[muellerzr]

Thanks! We’ve released a patch fixing this

[chenyuteng01]

Thanks! We’ve released a patch fixing this

Thanks!