Exception: ModuleNotFoundError: No module named '_cffi_backend'

[mebibou]

• Package Name: azure-blob-storage
• Package Version: 12.8.1
• Operating System: Azure Function App
• Python Version: 3.6

Describe the bug

When running the function app, this error is thrown:

Result: Failure
Exception: ModuleNotFoundError: No module named '_cffi_backend'. Troubleshooting Guide: https://aka.ms/functions-modulenotfound
Stack:   File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/dispatcher.py", line 309, in _handle__function_load_request
    func_request.metadata.entry_point)
  File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 42, in call
    raise extend_exception_message(e, message)
  File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 40, in call
    return func(*args, **kwargs)
  File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/loader.py", line 83, in load_function
    mod = importlib.import_module(fullmodname)
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/site/wwwroot/Calculate/__init__.py", line 5, in <module>
    from azure.storage.blob import BlobServiceClient
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/__init__.py", line 10, in <module>
    from ._blob_client import BlobClient
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/_blob_client.py", line 27, in <module>
    from ._shared.encryption import generate_blob_encryption_data
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/_shared/encryption.py", line 19, in <module>
    from cryptography.hazmat.primitives.padding import PKCS7
  File "/home/site/wwwroot/.python_packages/lib/site-packages/cryptography/hazmat/primitives/padding.py", line 11, in <module>
    from cryptography.hazmat.bindings._padding import lib

To Reproduce
I have set this in requirements.txt:

azure-functions
azure-storage-blob

I am deploying the function via github Actions like this:

name: Deploy Function

on: [push]

env:
  AZURE_FUNCTIONAPP_NAME: xxx
  AZURE_FUNCTIONAPP_PACKAGE_PATH: '.'
  PYTHON_VERSION: '3.6'

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    - name: 'Checkout GitHub Action'
      uses: actions/checkout@v2

    - name: Setup Python ${{ env.PYTHON_VERSION }} Environment
      uses: actions/setup-python@v1
      with:
        python-version: ${{ env.PYTHON_VERSION }}

    - name: 'Resolve Project Dependencies Using Pip'
      shell: bash
      run: |
        pushd './${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}'
        python -m pip install --upgrade pip
        pip install -r requirements.txt --target=".python_packages/lib/site-packages"
        popd

    - name: 'Run Azure Functions Action'
      uses: Azure/functions-action@v1
      id: fa
      with:
        app-name: ${{ env.AZURE_FUNCTIONAPP_NAME }}
        package: ${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}
        publish-profile: ${{ secrets.AZURE_FUNCTIONAPP_PUBLISH_PROFILE }}

Strangely, if I deploy the function from my mac using func azure functionapp publish xxx it does work, even though it seems to be deploying in Python 3.6 as well:

Downloading and extracting 'nodejs' version '14.17.4' to '/tmp/oryx/platforms/nodejs/14.17.4'...
Downloaded in 1 sec(s).
Verifying checksum...
Extracting contents...
Done in 3 sec(s).

Downloading and extracting 'python' version '3.6.12' to '/tmp/oryx/platforms/python/3.6.12'...
Downloaded in 0 sec(s).
Verifying checksum...
Extracting contents...
Done in 3 sec(s).

Expected behavior

It should not crash whether I deploy locally or from Github

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context

Deployment logs on Github Actions:

Run Azure/functions-action@v1
  with:
    app-name: xxx
    package: .
    publish-profile: ***
    respect-pom-xml: false
    respect-funcignore: false
    scm-do-build-during-deployment: false
    enable-oryx-build: false
  env:
    AZURE_FUNCTIONAPP_NAME: xxx
    AZURE_FUNCTIONAPP_PACKAGE_PATH: .
    PYTHON_VERSION: 3.6
    pythonLocation: /opt/hostedtoolcache/Python/3.6.14/x64
Successfully parsed SCM credential from old publish-profile format.
Using SCM credential for authentication, GitHub Action will not perform resource validation.
Successfully acquired app settings from function app (with SCM credential)!
Will archive . into /home/runner/work/_temp/temp_web_package_8622910865863398.zip as function app content
Will use Kudu https://<scmsite>/api/zipdeploy to deploy since publish-profile is detected.
Setting SCM_DO_BUILD_DURING_DEPLOYMENT in Kudu container to false
Update using Client.updateAppSettingViaKudu
Response with status code 204
App setting SCM_DO_BUILD_DURING_DEPLOYMENT propagated to Kudu container
Setting ENABLE_ORYX_BUILD in Kudu container to false
Update using Client.updateAppSettingViaKudu
Response with status code 204
App setting ENABLE_ORYX_BUILD propagated to Kudu container
Package deployment using ZIP Deploy initiated.
Deploy logs can be viewed at https://xxx
Successfully deployed web package to App Service.
Successfully updated deployment History at https://xxx
Response with status code 204
Response with status code 204
Response with status code 204

[YalinLi0312]

Thanks for your feedback, we'll investigate asap.

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.

Issue Details

---

• Package Name: azure-blob-storage
• Package Version: 12.8.1
• Operating System: Azure Function App
• Python Version: 3.6

Describe the bug

When running the function app, this error is thrown:

Result: Failure
Exception: ModuleNotFoundError: No module named '_cffi_backend'. Troubleshooting Guide: https://aka.ms/functions-modulenotfound
Stack:   File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/dispatcher.py", line 309, in _handle__function_load_request
    func_request.metadata.entry_point)
  File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 42, in call
    raise extend_exception_message(e, message)
  File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 40, in call
    return func(*args, **kwargs)
  File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/loader.py", line 83, in load_function
    mod = importlib.import_module(fullmodname)
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/site/wwwroot/Calculate/__init__.py", line 5, in <module>
    from azure.storage.blob import BlobServiceClient
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/__init__.py", line 10, in <module>
    from ._blob_client import BlobClient
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/_blob_client.py", line 27, in <module>
    from ._shared.encryption import generate_blob_encryption_data
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/_shared/encryption.py", line 19, in <module>
    from cryptography.hazmat.primitives.padding import PKCS7
  File "/home/site/wwwroot/.python_packages/lib/site-packages/cryptography/hazmat/primitives/padding.py", line 11, in <module>
    from cryptography.hazmat.bindings._padding import lib

To Reproduce
I have set this in requirements.txt:

azure-functions
azure-storage-blob

I am deploying the function via github Actions like this:

name: Deploy Function

on: [push]

env:
  AZURE_FUNCTIONAPP_NAME: xxx
  AZURE_FUNCTIONAPP_PACKAGE_PATH: '.'
  PYTHON_VERSION: '3.6'

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    - name: 'Checkout GitHub Action'
      uses: actions/checkout@v2

    - name: Setup Python ${{ env.PYTHON_VERSION }} Environment
      uses: actions/setup-python@v1
      with:
        python-version: ${{ env.PYTHON_VERSION }}

    - name: 'Resolve Project Dependencies Using Pip'
      shell: bash
      run: |
        pushd './${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}'
        python -m pip install --upgrade pip
        pip install -r requirements.txt --target=".python_packages/lib/site-packages"
        popd

    - name: 'Run Azure Functions Action'
      uses: Azure/functions-action@v1
      id: fa
      with:
        app-name: ${{ env.AZURE_FUNCTIONAPP_NAME }}
        package: ${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}
        publish-profile: ${{ secrets.AZURE_FUNCTIONAPP_PUBLISH_PROFILE }}

Strangely, if I deploy the function from my mac using func azure functionapp publish xxx it does work, even though it seems to be deploying in Python 3.6 as well:

Downloading and extracting 'nodejs' version '14.17.4' to '/tmp/oryx/platforms/nodejs/14.17.4'...
Downloaded in 1 sec(s).
Verifying checksum...
Extracting contents...
Done in 3 sec(s).

Downloading and extracting 'python' version '3.6.12' to '/tmp/oryx/platforms/python/3.6.12'...
Downloaded in 0 sec(s).
Verifying checksum...
Extracting contents...
Done in 3 sec(s).

Expected behavior

It should not crash whether I deploy locally or from Github

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context

Deployment logs on Github Actions:

Run Azure/functions-action@v1
  with:
    app-name: xxx
    package: .
    publish-profile: ***
    respect-pom-xml: false
    respect-funcignore: false
    scm-do-build-during-deployment: false
    enable-oryx-build: false
  env:
    AZURE_FUNCTIONAPP_NAME: xxx
    AZURE_FUNCTIONAPP_PACKAGE_PATH: .
    PYTHON_VERSION: 3.6
    pythonLocation: /opt/hostedtoolcache/Python/3.6.14/x64
Successfully parsed SCM credential from old publish-profile format.
Using SCM credential for authentication, GitHub Action will not perform resource validation.
Successfully acquired app settings from function app (with SCM credential)!
Will archive . into /home/runner/work/_temp/temp_web_package_8622910865863398.zip as function app content
Will use Kudu https://<scmsite>/api/zipdeploy to deploy since publish-profile is detected.
Setting SCM_DO_BUILD_DURING_DEPLOYMENT in Kudu container to false
Update using Client.updateAppSettingViaKudu
Response with status code 204
App setting SCM_DO_BUILD_DURING_DEPLOYMENT propagated to Kudu container
Setting ENABLE_ORYX_BUILD in Kudu container to false
Update using Client.updateAppSettingViaKudu
Response with status code 204
App setting ENABLE_ORYX_BUILD propagated to Kudu container
Package deployment using ZIP Deploy initiated.
Deploy logs can be viewed at https://xxx
Successfully deployed web package to App Service.
Successfully updated deployment History at https://xxx
Response with status code 204
Response with status code 204
Response with status code 204

Author:	mebibou
Assignees:	-
Labels:	Storage, question, Service Attention, customer-reported, needs-triage
Milestone:	-

[xiafu-msft]

Hi @mebibou

Thanks for reaching out. It looks like cffi==1.14.6 wasn't installed successfully though I don't know why, I'm pretty sure there's no problem when we use blob package locally. can you please try to list cffi==1.14.6 specifically in requirements.txt and see if it can help?

[mebibou]

@xiafu-msft yes thank you that works

[KevinTechExec]

Having similar problem as @mebibou - tried solution proposed by @xiafu-msft with no result. Code works perfectly when run under WSL connecting to a ADLS gen 2 storage account using the following Python version and Azure Function Core Tools

Found Python version 3.8.10 (python3).

Azure Functions Core Tools
Core Tools Version:       3.0.3568 Commit hash: e30a0ede85fd498199c28ad699ab2548593f759b  (64-bit)
Function Runtime Version: 3.0.15828.0

however fails with

Result: Failure
Exception: ModuleNotFoundError: No module named '_cffi_backend'. Troubleshooting Guide: https://aka.ms/functions-modulenotfound
Stack:   File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 305, in _handle__function_load_request
    func = loader.load_function(
  File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 42, in call
    raise extend_exception_message(e, message)
  File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 40, in call
    return func(*args, **kwargs)
  File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/loader.py", line 83, in load_function
    mod = importlib.import_module(fullmodname)
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/site/wwwroot/mail-collector/__init__.py", line 10, in <module>
    from shared.writeadls2 import uploaddata
  File "/home/site/wwwroot/shared/writeadls2.py", line 11, in <module>
    from azure.storage.filedatalake import DataLakeFileClient, DataLakeServiceClient
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/filedatalake/__init__.py", line 7, in <module>
    from ._download import StorageStreamDownloader
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/filedatalake/_download.py", line 8, in <module>
    from ._deserialize import from_blob_properties
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/filedatalake/_deserialize.py", line 15, in <module>
    from ._models import FileProperties, DirectoryProperties, LeaseProperties, DeletedPathProperties, StaticWebsite, \
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/filedatalake/_models.py", line 11, in <module>
    from azure.storage.blob import LeaseProperties as BlobLeaseProperties
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/__init__.py", line 10, in <module>
    from ._blob_client import BlobClient
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/_blob_client.py", line 27, in <module>
    from ._shared.encryption import generate_blob_encryption_data
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/_shared/encryption.py", line 19, in <module>
    from cryptography.hazmat.primitives.padding import PKCS7
  File "/home/site/wwwroot/.python_packages/lib/site-packages/cryptography/hazmat/primitives/padding.py", line 11, in <module>
    from cryptography.hazmat.bindings._padding import lib

when run from Azure Function in cloud (sdk version azurefunctions: 3.1.4.0)

requirements.txt

azure-core
azure-functions
azure-storage-blob
azure-storage-file-datalake
certifi
cffi==1.14.6
charset-normalizer
cryptography
idna
isodate
msal
msrest
oauthlib
pycparser
PyJWT
requests
requests-oauthlib
six
urllib3

Finally - here is the log from the install phase of the pipeline - everything looks to be installed properly

Starting: Install application dependencies
==============================================================================
Task         : Bash
Description  : Run a Bash script on macOS, Linux, or Windows
Version      : 3.189.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
==============================================================================
Generating script.
Script contents:
pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
========================== Starting Command Output ===========================
/usr/bin/bash --noprofile --norc /home/vsts/work/_temp/1134e58f-087d-4f9d-b62f-3231bf25c7bd.sh
Collecting azure-core
  Downloading azure_core-1.18.0-py2.py3-none-any.whl (166 kB)
Collecting azure-functions
  Downloading azure_functions-1.7.2-py3-none-any.whl (137 kB)
Collecting azure-storage-blob
  Downloading azure_storage_blob-12.8.1-py2.py3-none-any.whl (345 kB)
Collecting azure-storage-file-datalake
  Downloading azure_storage_file_datalake-12.4.0-py2.py3-none-any.whl (206 kB)
Collecting certifi
  Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
Collecting cffi==1.14.6
  Downloading cffi-1.14.6-cp38-cp38-manylinux1_x86_64.whl (411 kB)
Collecting charset-normalizer
  Downloading charset_normalizer-2.0.4-py3-none-any.whl (36 kB)
Collecting cryptography
  Downloading cryptography-3.4.8-cp36-abi3-manylinux_2_24_x86_64.whl (3.0 MB)
Collecting idna
  Downloading idna-3.2-py3-none-any.whl (59 kB)
Collecting isodate
  Downloading isodate-0.6.0-py2.py3-none-any.whl (45 kB)
Collecting msal
  Downloading msal-1.14.0-py2.py3-none-any.whl (75 kB)
Collecting msrest
  Downloading msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting oauthlib
  Downloading oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
Collecting pycparser
  Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting PyJWT
  Downloading PyJWT-2.1.0-py3-none-any.whl (16 kB)
Collecting requests
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting requests-oauthlib
  Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting six
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting urllib3
  Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
Collecting azure-storage-blob
  Downloading azure_storage_blob-12.9.0b1-py2.py3-none-any.whl (356 kB)
Installing collected packages: urllib3, idna, charset-normalizer, certifi, six, requests, pycparser, oauthlib, requests-oauthlib, isodate, cffi, PyJWT, msrest, cryptography, azure-core, azure-storage-blob, msal, azure-storage-file-datalake, azure-functions
Successfully installed PyJWT-2.1.0 azure-core-1.18.0 azure-functions-1.7.2 azure-storage-blob-12.8.1 azure-storage-file-datalake-12.4.0 certifi-2021.5.30 cffi-1.14.6 charset-normalizer-2.0.4 cryptography-3.4.8 idna-3.2 isodate-0.6.0 msal-1.14.0 msrest-0.6.21 oauthlib-3.1.1 pycparser-2.20 requests-2.26.0 requests-oauthlib-1.3.0 six-1.16.0 urllib3-1.26.6
Finishing: Install application dependencies

[amishra-dev]

@xiafu-msft can you look at this?

[xiafu-msft]

Hi @KevinTechExec

Can you specify the package version specifically and see if it helps?
Here's the package version I got from my virtualenv

azure-core==1.19.0
azure-storage-blob==12.9.0
azure-storage-file-datalake==12.4.0
certifi==2021.5.30
cffi==1.14.6
charset-normalizer==2.0.6
cryptography==35.0.0
idna==3.2
isodate==0.6.0
msrest==0.6.21
oauthlib==3.1.1
pycparser==2.20
requests==2.26.0
requests-oauthlib==1.3.0
six==1.16.0
urllib3==1.26.7

Let me know if it still doesn't work.

[KevinTechExec]

@xiafu-msft thanks very much! This appears to be working now in azure function space. For reference - here is the change:

Prior version requirements.txt that was not working

azure-functions
azure-storage-file-datalake
cffi==1.14.6
msal

Current updated version of requirements.txt that is working

azure-core==1.19.0
azure-storage-blob==12.9.0
azure-storage-file-datalake==12.4.0
certifi==2021.5.30
cffi==1.14.6
charset-normalizer==2.0.6
cryptography==35.0.0
idna==3.2
isodate==0.6.0
msrest==0.6.21
oauthlib==3.1.1
pycparser==2.20
requests==2.26.0
requests-oauthlib==1.3.0
six==1.16.0
urllib3==1.26.7
msal

[ghost]

Hi @mebibou. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

[pharrukh]

/unresolve

[ghost]

Hi pharrukh, only the original author of the issue can ask that it be unresolved. Please open a new issue with your scenario and details if you would like to discuss this topic with the team.

[ghost]

Hi @mebibou, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.

[rikjansen-hu]

For future readers, I had this same issue when using azure.keyvault.keys.crypto.CryptographyClient. Adding the dependencies above solved the errors on provisioning, but popped up again during invocation. Turned out I forgot to install build extensions during code deploy:

      steps:
      - bash: |
          if [ -f extensions.csproj ]
          then
              dotnet build extensions.csproj --runtime ubuntu.16.04-x64 --output ./bin
          fi
        workingDirectory: $(workingDirectory)
        displayName: 'Build extensions'

Once done, you don't need install any of the dependencies listen above.