[Core] Add lazy import structure to imports

[patrickvonplaten]

What does this PR do?

Importing model classes from transformers has become quite slow. This is mainly because we link more and more directories and files to modeling_utils.py and generation_utils.py both files which are imported almost by every model (e.g. as soon as PreTrainedModel is imported).

At the moment we're always importing all third party libraries:

    from .integration_utils import (
        INTEGRATION_TO_CALLBACK,
        AzureMLCallback,
        ClearMLCallback,
        CodeCarbonCallback,
        CometCallback,
        DagsHubCallback,
        FlyteCallback,
        MLflowCallback,
        NeptuneCallback,
        NeptuneMissingConfiguration,
        TensorBoardCallback,
        WandbCallback,
        get_available_reporting_integrations,
        get_reporting_integration_callbacks,
        hp_params,
        is_azureml_available,
        is_clearml_available,
        is_codecarbon_available,
        is_comet_available,
        is_dagshub_available,
        is_fairscale_available,
        is_flyte_deck_standard_available,
        is_flytekit_available,
        is_mlflow_available,
        is_neptune_available,
        is_optuna_available,
        is_ray_available,
        is_ray_tune_available,
        is_sigopt_available,
        is_tensorboard_available,
        is_wandb_available,
        rewrite_logs,
        run_hp_search_optuna,
        run_hp_search_ray,
        run_hp_search_sigopt,
        run_hp_search_wandb,
    )

which makes import very slow. This PR adds the lazy import structure to the integrations init file so that we only import all third-party libraries when actually needed.

For environments where e.g. tensorflow is installed we now see a significant speed-up (if we make sure to remove accelerate before as accelerate is really slow to import.

To reproduce:

1. Make sure accelerate is not installed, but tensorflow is installed.

2. Run time python -c "from transformers import CLIPTextModel" # or any other model import

Before PR:

2023-09-11 11:07:52.010179: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
python3 -c "from transformers import CLIPTextModel"  3.31s user 3.06s system 220% cpu 2.893 total

After PR:

python3 -c "from transformers import CLIPTextModel"  1.70s user 1.49s system 220% cpu 1.447 total

=> We're getting a 2x speed-up.

We should also make sure that importing accelerate is sped up as it become a utility library for more and more libraries (@muellerzr @SunMarc)

[patrickvonplaten]

Also cc @DN6 - this should then also improve importing from diffusers:

time python -c "from diffusers import StableDiffusionPipeline" is pretty slow at the moment because of accelerate and this

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[patrickvonplaten]

Failing tests seem unrelated - ready for a review.

[amyeroberts]

Thank you for updating this! 🙏