Error loading text generation pipeline: Exception: Python patch version not an integer

[morrisalp]

System Info

Platform: Ubuntu 20.04.5, Jupyter Lab 3.5.2, dockerized
Python version: 3.8.13

pip freeze output:

absl-py==1.2.0
accelerate==0.15.0
aiohttp==3.8.3
aiosignal==1.3.1
alabaster==0.7.12
anyio==3.6.1
apex==0.1
appdirs==1.4.4
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens @ file:///home/conda/feedstock_root/build_artifacts/asttokens_1660605382950/work
async-timeout==4.0.2
attrs==22.1.0
audioread==3.0.0
Babel==2.10.3
backcall @ file:///home/conda/feedstock_root/build_artifacts/backcall_1592338393461/work
backports.functools-lru-cache @ file:///home/conda/feedstock_root/build_artifacts/backports.functools_lru_cache_1618230623929/work
beautifulsoup4 @ file:///home/conda/feedstock_root/build_artifacts/beautifulsoup4_1649463573192/work
bitsandbytes==0.35.4
bleach==5.0.1
blis @ file:///home/conda/feedstock_root/build_artifacts/cython-blis_1656314523915/work
brotlipy @ file:///home/conda/feedstock_root/build_artifacts/brotlipy_1648854175163/work
cachetools==5.2.0
catalogue @ file:///home/conda/feedstock_root/build_artifacts/catalogue_1661366525041/work
certifi==2022.9.24
cffi @ file:///home/conda/feedstock_root/build_artifacts/cffi_1656782821535/work
chardet @ file:///home/conda/feedstock_root/build_artifacts/chardet_1656142044710/work
charset-normalizer @ file:///home/conda/feedstock_root/build_artifacts/charset-normalizer_1655906222726/work
click @ file:///home/conda/feedstock_root/build_artifacts/click_1651215152883/work
cloudpickle==2.2.0
codecov==2.1.12
colorama @ file:///home/conda/feedstock_root/build_artifacts/colorama_1655412516417/work
conda-package-handling @ file:///home/conda/feedstock_root/build_artifacts/conda-package-handling_1663583601093/work
contourpy==1.0.5
coverage==6.5.0
cryptography @ file:///home/conda/feedstock_root/build_artifacts/cryptography_1665535545125/work
cuda-python @ file:///rapids/cuda_python-11.7.0%2B0.g95a2041.dirty-cp38-cp38-linux_x86_64.whl
cudf @ file:///rapids/cudf-22.8.0a0%2B304.g6ca81bbc78.dirty-cp38-cp38-linux_x86_64.whl
cugraph @ file:///rapids/cugraph-22.8.0a0%2B132.g2daa31b6.dirty-cp38-cp38-linux_x86_64.whl
cuml @ file:///rapids/cuml-22.8.0a0%2B52.g73b8d00d0.dirty-cp38-cp38-linux_x86_64.whl
cupy-cuda118 @ file:///rapids/cupy_cuda118-11.0.0-cp38-cp38-linux_x86_64.whl
cycler==0.11.0
cymem @ file:///home/conda/feedstock_root/build_artifacts/cymem_1636053152744/work
Cython==0.29.32
dask @ file:///rapids/dask-2022.7.1-py3-none-any.whl
dask-cuda @ file:///rapids/dask_cuda-22.8.0a0%2B36.g9860cad-py3-none-any.whl
dask-cudf @ file:///rapids/dask_cudf-22.8.0a0%2B304.g6ca81bbc78.dirty-py3-none-any.whl
dataclasses @ file:///home/conda/feedstock_root/build_artifacts/dataclasses_1628958434797/work
debugpy==1.6.3
decorator @ file:///home/conda/feedstock_root/build_artifacts/decorator_1641555617451/work
defusedxml==0.7.1
diffusers==0.11.0
distributed @ file:///rapids/distributed-2022.7.1-py3-none-any.whl
docutils==0.17.1
entrypoints==0.3
et-xmlfile==1.1.0
executing @ file:///home/conda/feedstock_root/build_artifacts/executing_1665301981797/work
expecttest==0.1.3
fastjsonschema==2.16.2
fastrlock==0.8
filelock @ file:///home/conda/feedstock_root/build_artifacts/filelock_1660129891014/work
flake8==3.7.9
Flask==2.2.2
fonttools==4.37.4
frozenlist==1.3.3
fsspec==2022.8.2
functorch==0.3.0a0
future==0.18.2
glob2==0.7
google-auth==2.12.0
google-auth-oauthlib==0.4.6
graphsurgeon @ file:///workspace/TensorRT-8.5.0.12/graphsurgeon/graphsurgeon-0.4.6-py2.py3-none-any.whl
grpcio==1.49.1
HeapDict==1.0.1
huggingface-hub==0.11.0
hypothesis==4.50.8
idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1642433548627/work
imagesize==1.4.1
importlib-metadata==5.0.0
importlib-resources==5.10.0
iniconfig==1.1.1
iopath==0.1.10
ipykernel==6.16.0
ipython @ file:///home/conda/feedstock_root/build_artifacts/ipython_1662481517711/work
ipython-genutils==0.2.0
ipywidgets==8.0.2
itsdangerous==2.1.2
jedi @ file:///home/conda/feedstock_root/build_artifacts/jedi_1659959867326/work
Jinja2 @ file:///home/conda/feedstock_root/build_artifacts/jinja2_1654302431367/work
joblib==1.2.0
json5==0.9.10
jsonschema==4.16.0
jupyter-core==4.11.1
jupyter-server==1.21.0
jupyter-tensorboard @ git+https://github.com/cliffwoolley/jupyter_tensorboard.git@ffa7e26138b82549453306e06b535a9ac36db17a
jupyter_client==7.4.2
jupyterlab==2.3.2
jupyterlab-pygments==0.2.2
jupyterlab-server==1.2.0
jupyterlab-widgets==3.0.3
jupytext==1.14.1
kiwisolver==1.4.4
langcodes @ file:///home/conda/feedstock_root/build_artifacts/langcodes_1636741340529/work
libarchive-c @ file:///home/conda/feedstock_root/build_artifacts/python-libarchive-c_1649436017468/work
librosa==0.9.2
lightning-utilities==0.4.2
llvmlite==0.39.1
lmdb==1.3.0
locket==1.0.0
Markdown==3.4.1
markdown-it-py==2.1.0
MarkupSafe @ file:///home/conda/feedstock_root/build_artifacts/markupsafe_1648737563195/work
matplotlib==3.6.2
matplotlib-inline @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-inline_1660814786464/work
mccabe==0.6.1
mdit-py-plugins==0.3.1
mdurl==0.1.2
mistune==2.0.4
mock @ file:///home/conda/feedstock_root/build_artifacts/mock_1648992799371/work
msgpack==1.0.4
multidict==6.0.3
murmurhash @ file:///home/conda/feedstock_root/build_artifacts/murmurhash_1636019583024/work
nbclassic==0.4.5
nbclient==0.7.0
nbconvert==7.2.1
nbformat==5.7.0
nest-asyncio==1.5.6
networkx==2.6.3
nltk==3.7
notebook==6.4.10
notebook-shim==0.1.0
numba==0.56.2
numpy @ file:///home/conda/feedstock_root/build_artifacts/numpy_1643958805350/work
nvidia-dali-cuda110==1.18.0
nvidia-pyindex==1.0.9
nvtx==0.2.5
oauthlib==3.2.1
onnx @ file:///opt/pytorch/pytorch/third_party/onnx
openpyxl==3.0.10
packaging @ file:///home/conda/feedstock_root/build_artifacts/packaging_1637239678211/work
pandas==1.4.4
pandocfilters==1.5.0
parso @ file:///home/conda/feedstock_root/build_artifacts/parso_1638334955874/work
partd==1.3.0
pathy @ file:///home/conda/feedstock_root/build_artifacts/pathy_1656568808184/work
pexpect @ file:///home/conda/feedstock_root/build_artifacts/pexpect_1602535608087/work
pickleshare @ file:///home/conda/feedstock_root/build_artifacts/pickleshare_1602536217715/work
Pillow @ file:///tmp/pillow-simd
pkginfo @ file:///home/conda/feedstock_root/build_artifacts/pkginfo_1654782790443/work
pkgutil_resolve_name==1.3.10
pluggy==1.0.0
polygraphy==0.42.1
pooch==1.6.0
portalocker==2.5.1
preshed @ file:///home/conda/feedstock_root/build_artifacts/preshed_1636077712344/work
prettytable==3.4.1
prometheus-client==0.15.0
prompt-toolkit @ file:///home/conda/feedstock_root/build_artifacts/prompt-toolkit_1662384672173/work
protobuf==3.20.1
psutil @ file:///home/conda/feedstock_root/build_artifacts/psutil_1662356143277/work
ptyprocess @ file:///home/conda/feedstock_root/build_artifacts/ptyprocess_1609419310487/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl
pure-eval @ file:///home/conda/feedstock_root/build_artifacts/pure_eval_1642875951954/work
py==1.11.0
pyarrow @ file:///rapids/pyarrow-8.0.0-cp38-cp38-linux_x86_64.whl
pyasn1==0.4.8
pyasn1-modules==0.2.8
pybind11==2.10.0
pycocotools @ git+https://github.com/nvidia/cocoapi.git@142b17a358fdb5a31f9d5153d7a9f3f1cd385178#subdirectory=PythonAPI
pycodestyle==2.5.0
pycosat @ file:///home/conda/feedstock_root/build_artifacts/pycosat_1649384811940/work
pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1636257122734/work
pydantic @ file:///home/conda/feedstock_root/build_artifacts/pydantic_1636021149719/work
pydot==1.4.2
pyflakes==2.1.1
Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1660666458521/work
pylibcugraph @ file:///rapids/pylibcugraph-22.8.0a0%2B132.g2daa31b6.dirty-cp38-cp38-linux_x86_64.whl
pynvml==11.4.1
pyOpenSSL @ file:///home/conda/feedstock_root/build_artifacts/pyopenssl_1643496850550/work
pyparsing @ file:///home/conda/feedstock_root/build_artifacts/pyparsing_1652235407899/work
pyrsistent==0.18.1
PySocks @ file:///home/conda/feedstock_root/build_artifacts/pysocks_1661604839144/work
pytest==6.2.5
pytest-cov==4.0.0
pytest-pythonpath==0.7.4
python-dateutil==2.8.2
python-hostlist==1.22
python-nvd3==0.15.0
python-slugify==6.1.2
pytorch-lightning==1.8.5.post0
pytorch-quantization==2.1.2
pytz @ file:///home/conda/feedstock_root/build_artifacts/pytz_1664798238822/work
PyYAML @ file:///home/conda/feedstock_root/build_artifacts/pyyaml_1648757091578/work
pyzmq==24.0.1
raft @ file:///rapids/raft-22.8.0a0%2B70.g9070c30.dirty-cp38-cp38-linux_x86_64.whl
regex==2022.9.13
requests @ file:///home/conda/feedstock_root/build_artifacts/requests_1656534056640/work
requests-oauthlib==1.3.1
resampy==0.4.2
revtok @ git+git://github.com/jekbradbury/revtok.git@f1998b72a941d1e5f9578a66dc1c20b01913caab
rmm @ file:///rapids/rmm-22.8.0a0%2B62.gf6bf047.dirty-cp38-cp38-linux_x86_64.whl
rsa==4.9
ruamel-yaml-conda @ file:///home/conda/feedstock_root/build_artifacts/ruamel_yaml_1653464386701/work
sacremoses==0.0.53
safetensors==0.2.6
scikit-learn @ file:///rapids/scikit_learn-0.24.2-cp38-cp38-manylinux2010_x86_64.whl
scipy @ file:///home/conda/feedstock_root/build_artifacts/scipy_1619561901336/work
seaborn==0.12.1
Send2Trash==1.8.0
shellingham @ file:///home/conda/feedstock_root/build_artifacts/shellingham_1659638615822/work
six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work
smart-open @ file:///home/conda/feedstock_root/build_artifacts/smart_open_1630238320325/work
sniffio==1.3.0
snowballstemmer==2.2.0
sortedcontainers==2.4.0
soundfile==0.11.0
soupsieve @ file:///home/conda/feedstock_root/build_artifacts/soupsieve_1658207591808/work
spacy @ file:///home/conda/feedstock_root/build_artifacts/spacy_1644657943105/work
spacy-legacy @ file:///home/conda/feedstock_root/build_artifacts/spacy-legacy_1660748275723/work
spacy-loggers @ file:///home/conda/feedstock_root/build_artifacts/spacy-loggers_1661365735520/work
Sphinx==5.2.3
sphinx-glpi-theme==0.3
sphinx-rtd-theme==1.0.0
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
srsly @ file:///home/conda/feedstock_root/build_artifacts/srsly_1638879568141/work
stack-data @ file:///home/conda/feedstock_root/build_artifacts/stack_data_1664126450622/work
tabulate==0.9.0
tblib==1.7.0
tensorboard==2.10.1
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorboardX==2.5.1
tensorrt @ file:///workspace/TensorRT-8.5.0.12/python/tensorrt-8.5.0.12-cp38-none-linux_x86_64.whl
terminado==0.16.0
text-unidecode==1.3
thinc @ file:///home/conda/feedstock_root/build_artifacts/thinc_1638980259098/work
threadpoolctl==3.1.0
tinycss2==1.1.1
tokenizers==0.13.2
toml @ file:///home/conda/feedstock_root/build_artifacts/toml_1604308577558/work
tomli==2.0.1
toolz @ file:///home/conda/feedstock_root/build_artifacts/toolz_1657485559105/work
torch==1.13.0a0+d0d6b1f
torch-tensorrt @ file:///opt/pytorch/torch_tensorrt/py/dist/torch_tensorrt-1.3.0a0-cp38-cp38-linux_x86_64.whl
torchinfo==1.7.1
torchmetrics==0.11.0
torchtext==0.11.0a0
torchvision @ file:///opt/pytorch/vision
tornado==6.2
tqdm==4.64.1
traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1663005918942/work
transformer-engine @ file:///tmp/te_wheel/transformer_engine-0.1.0-cp38-cp38-linux_x86_64.whl
transformers==4.25.1
treelite @ file:///rapids/treelite-2.4.0-py3-none-manylinux2014_x86_64.whl
treelite-runtime @ file:///rapids/treelite_runtime-2.4.0-py3-none-manylinux2014_x86_64.whl
typer @ file:///home/conda/feedstock_root/build_artifacts/typer_1657029164904/work
typing_extensions @ file:///home/conda/feedstock_root/build_artifacts/typing_extensions_1665144421445/work
ucx-py @ file:///rapids/ucx_py-0.27.0a0%2B29.ge9e81f8-cp38-cp38-linux_x86_64.whl
uff @ file:///workspace/TensorRT-8.5.0.12/uff/uff-0.6.9-py2.py3-none-any.whl
urllib3 @ file:///home/conda/feedstock_root/build_artifacts/urllib3_1658789158161/work
wasabi @ file:///home/conda/feedstock_root/build_artifacts/wasabi_1658931821849/work
wcwidth @ file:///home/conda/feedstock_root/build_artifacts/wcwidth_1600965781394/work
webencodings==0.5.1
websocket-client==1.4.1
Werkzeug==2.2.2
widgetsnbextension==4.0.3
xgboost @ file:///rapids/xgboost-1.6.1-cp38-cp38-linux_x86_64.whl
yarl==1.8.2
zict==2.2.0
zipp==3.9.0

Who can help?

@ArthurZucker @younesbelkada @gante @Narsil

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

from transformers import pipeline
generator = pipeline('text-generation', model='gpt2')

Output:

Downloading:   0%|          | 0.00/665 [00:00<?, ?B/s]
Downloading:   0%|          | 0.00/548M [00:00<?, ?B/s]
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In [2], line 1
----> 1 generator = pipeline('text-generation', model='gpt2')

File /storage/morrisalper/notebooks/envs/notebook_env/lib/python3.8/site-packages/transformers/pipelines/__init__.py:724, in pipeline(task, model, config, tokenizer, feature_extractor, framework, revision, use_fast, use_auth_token, device, device_map, torch_dtype, trust_remote_code, model_kwargs, pipeline_class, **kwargs)
    720 # Infer the framework from the model
    721 # Forced if framework already defined, inferred if it's None
    722 # Will load the correct model if possible
    723 model_classes = {"tf": targeted_task["tf"], "pt": targeted_task["pt"]}
--> 724 framework, model = infer_framework_load_model(
    725     model,
    726     model_classes=model_classes,
    727     config=config,
    728     framework=framework,
    729     task=task,
    730     **hub_kwargs,
    731     **model_kwargs,
    732 )
    734 model_config = model.config
    735 hub_kwargs["_commit_hash"] = model.config._commit_hash

File /storage/morrisalper/notebooks/envs/notebook_env/lib/python3.8/site-packages/transformers/pipelines/base.py:257, in infer_framework_load_model(model, config, model_classes, task, framework, **model_kwargs)
    251     logger.warning(
    252         "Model might be a PyTorch model (ending with `.bin`) but PyTorch is not available. "
    253         "Trying to load the model with Tensorflow."
    254     )
    256 try:
--> 257     model = model_class.from_pretrained(model, **kwargs)
    258     if hasattr(model, "eval"):
    259         model = model.eval()

File /storage/morrisalper/notebooks/envs/notebook_env/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py:463, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    461 elif type(config) in cls._model_mapping.keys():
    462     model_class = _get_model_class(config, cls._model_mapping)
--> 463     return model_class.from_pretrained(
    464         pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
    465     )
    466 raise ValueError(
    467     f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
    468     f"Model type should be one of {', '.join(c.__name__ for c in cls._model_mapping.keys())}."
    469 )

File /storage/morrisalper/notebooks/envs/notebook_env/lib/python3.8/site-packages/transformers/modeling_utils.py:2230, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   2227 if from_pt:
   2228     if not is_sharded and state_dict is None:
   2229         # Time to load the checkpoint
-> 2230         state_dict = load_state_dict(resolved_archive_file)
   2232     # set dtype to instantiate the model under:
   2233     # 1. If torch_dtype is not None, we use that dtype
   2234     # 2. If torch_dtype is "auto", we auto-detect dtype from the loaded state_dict, by checking its first
   2235     #    weights entry that is of a floating type - we assume all floating dtype weights are of the same dtype
   2236     # we also may have config.torch_dtype available, but we won't rely on it till v5
   2237     dtype_orig = None

File /storage/morrisalper/notebooks/envs/notebook_env/lib/python3.8/site-packages/transformers/modeling_utils.py:386, in load_state_dict(checkpoint_file)
    381 """
    382 Reads a PyTorch checkpoint file, returning properly formatted errors if they arise.
    383 """
    384 if checkpoint_file.endswith(".safetensors") and is_safetensors_available():
    385     # Check format of the archive
--> 386     with safe_open(checkpoint_file, framework="pt") as f:
    387         metadata = f.metadata()
    388     if metadata.get("format") not in ["pt", "tf", "flax"]:

Exception: Python patch version not an integer

Expected behavior

Should not output an exception. E.g. this code runs as-is (after pip install transformers) in Google Colab.

[Narsil]

This is very odd.

Could you share maybe a bit more about your environment so we could reproduce ?
It seems like the way Python itself is installed is odd (I'm purely inferring from the error message), maybe ?

Is it possible to provide a way to reproduce maybe ? Like a docker image or something ?

It does seem to work on colab so it's hard to know what is wrong with the enviroment. It also seems like there's a mix of conda and pip install which might be at play (both link to different things, so maybe the linker is confused somehow ?)
I tried googling your error message but nothing came up..

[Codle]

I met the same problem and fixed it by degrading the transformers version like 4.22.0 or others.

[SimonLinnebjerg]

Experienced the same problem. I also downgraded to make it work.

i dont know what the _commit_hash variable is used for, but removing the line in transformers/pipelines/init.py also seems to work.

this line hub_kwargs["_commit_hash"] = model.config._commit_hash

A fix for this would be very appreciated

[Codle]

I think it was related to this issue. All PyTorch container images of NVIDIA NGC have alpha version tags for PyTorch. cc @Narsil

[Narsil]

Thanks @Codle .

It seems to be indeed the issue. Releasing a new version soon so everyone has access.

[Narsil]

Should be fixed with new version (0.2.8), could you confirm ?

[Codle]

Hi @Narsil, sorry for my late response. After updating safetensors to 0.2.8, it works fine for me.

[Narsil]

Closing this. Thank you for sharing !

[morrisalp]

Updating safetensors solved it for me too. Thanks!

[surya-narayanan]

despite downgrading my safetensors, I get the following error

Traceback (most recent call last):
  File "/home/suryahari/Vornoi/QA.py", line 5, in <module>
    model = AutoModelForQuestionAnswering.from_pretrained("deepset/roberta-base-squad2")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/suryahari/miniconda3/envs/diffusers/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 493, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/suryahari/miniconda3/envs/diffusers/lib/python3.11/site-packages/transformers/modeling_utils.py", line 2629, in from_pretrained
    state_dict = load_state_dict(resolved_archive_file)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/suryahari/miniconda3/envs/diffusers/lib/python3.11/site-packages/transformers/modeling_utils.py", line 447, in load_state_dict
    with safe_open(checkpoint_file, framework="pt") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: No such device (os error 19)