Extension add fails due to "libffi.so.5: cannot open shared object file: No such file or directory"

[benc-uk]

Extension name (the extension in question)

webapp

Description of issue (in as much detail as possible)

Running az extension add --name webapp results in an error. "An error occurred. Pip failed with status code 1. Use --debug for more information." When run with --debug the problem seems to be related to importing libffi.so.5.
I am running WSL on Windows 10 (1709) with latest version of CLI (2.0.27)

Full output of command with --debug (click to expand)

$ az extension add --name webapp --debug
Command arguments: ['extension', 'add', '--name', 'webapp', '--debug']
Event: Cli.PreExecute []
Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f56a70142f0>, <function OutputProducer.on_global_arguments at 0x7f56a6b21ea0>, <function CLIQuery.on_global_arguments at 0x7f56a66b09d8>]
Event: CommandInvoker.OnPreCommandTableCreate [<function CLILogging.remove_logger_flags at 0x7f56a7014378>]
Installed command modules ['acr', 'acs', 'advisor', 'appservice', 'backup', 'batch', 'batchai', 'billing', 'cdn', 'cloud', 'cognitiveservices', 'configure', 'consumption', 'container', 'cosmosdb', 'dla', 'dls', 'eventgrid', 'extension', 'feedback', 'find', 'interactive', 'iot', 'keyvault', 'lab', 'monitor', 'network', 'profile', 'rdbms', 'redis', 'reservations', 'resource', 'role', 'servicefabric', 'sql', 'storage', 'vm']
Loaded module 'acr' in 0.016 seconds.
Loaded module 'acs' in 0.012 seconds.
Loaded module 'advisor' in 0.011 seconds.
Loaded module 'appservice' in 0.016 seconds.
Loaded module 'backup' in 0.011 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'batch' in 0.031 seconds.
Loaded module 'batchai' in 0.012 seconds.
Loaded module 'billing' in 0.011 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'cdn' in 0.013 seconds.
Loaded module 'cloud' in 0.012 seconds.
Loaded module 'cognitiveservices' in 0.009 seconds.
Loaded module 'configure' in 0.007 seconds.
Loaded module 'consumption' in 0.018 seconds.
Loaded module 'container' in 0.012 seconds.
Loaded module 'cosmosdb' in 0.015 seconds.
Loaded module 'dla' in 0.015 seconds.
Loaded module 'dls' in 0.011 seconds.
Loaded module 'eventgrid' in 0.012 seconds.
Loaded module 'extension' in 0.006 seconds.
Loaded module 'feedback' in 0.007 seconds.
Loaded module 'find' in 0.008 seconds.
Loaded module 'interactive' in 0.002 seconds.
Loaded module 'iot' in 0.015 seconds.
Loaded module 'keyvault' in 0.018 seconds.
Loaded module 'lab' in 0.017 seconds.
Loaded module 'monitor' in 0.020 seconds.
Loaded module 'network' in 0.037 seconds.
Loaded module 'profile' in 0.013 seconds.
Loaded module 'rdbms' in 0.014 seconds.
Loaded module 'redis' in 0.012 seconds.
Loaded module 'reservations' in 0.013 seconds.
Loaded module 'resource' in 0.015 seconds.
Loaded module 'role' in 0.012 seconds.
Loaded module 'servicefabric' in 0.009 seconds.
Loaded module 'sql' in 0.015 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'storage' in 0.044 seconds.
Loaded module 'vm' in 0.025 seconds.
Loaded all modules in 0.544 seconds. (note: there's always an overhead with the first module loaded)
Extensions directory: '/home/ben/.azure/cliextensions'
Event: CommandInvoker.OnPreCommandTableTruncate []
Event: CommandInvoker.OnPostCommandTableCreate [<function add_id_parameters at 0x7f56a48569d8>]
Event: CommandInvoker.OnCommandTableLoaded []
Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x7f56a4695268>]
Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7f56a6b21f28>, <function CLIQuery.handle_query_parameter at 0x7f56a66b0a60>]
Extensions directory: '/home/ben/.azure/cliextensions'
urllib3.connectionpool : Starting new HTTPS connection (1): aka.ms
urllib3.connectionpool : https://aka.ms:443 "GET /azure-cli-extension-index-v1 HTTP/1.1" 301 186
urllib3.connectionpool : Starting new HTTPS connection (1): azurecliextensionsync.blob.core.windows.net
urllib3.connectionpool : https://azurecliextensionsync.blob.core.windows.net:443 "GET /index1/index.json HTTP/1.1" 200 22317
Candidates ['webapp-0.0.8-py2.py3-none-any.whl']
Candidates ['webapp-0.0.8-py2.py3-none-any.whl']
Candidates ['webapp-0.0.8-py2.py3-none-any.whl']
Candidates ['webapp-0.0.8-py2.py3-none-any.whl']
Choosing the latest of the remaining candidates.
Chosen {'filename': 'webapp-0.0.8-py2.py3-none-any.whl', 'sha256Digest': '4800c51978f7801b613f93afc62367c6646839dc4fd99c7673f4e217ba96cc58', 'downloadUrl': 'https://github.com/panchagnula/azure-cli-extensions/raw/sisirap-extensions-whl/dist/webapp-0.0.8-py2.py3-none-any.whl', 'metadata': {'classifiers': ['Development Status :: 4 - Beta', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'Programming Language :: Python', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', 'License :: OSI Approved :: MIT License'], 'extensions': {'python.details': {'contacts': [{'email': '[email protected]', 'name': 'Sisira Panchagnula', 'role': 'author'}], 'document_names': {'description': 'DESCRIPTION.rst'}, 'project_urls': {'Home': 'https://github.com/Azure/azure-cli-extensions'}}}, 'generator': 'bdist_wheel (0.29.0)', 'license': 'MIT', 'metadata_version': '2.0', 'name': 'webapp', 'summary': 'An Azure CLI Extension to manage appservice resources', 'version': '0.0.8'}}
Extension source is url? True
Extensions directory: '/home/ben/.azure/cliextensions'
Downloading https://github.com/panchagnula/azure-cli-extensions/raw/sisirap-extensions-whl/dist/webapp-0.0.8-py2.py3-none-any.whl to /tmp/tmp4ly546cw/webapp-0.0.8-py2.py3-none-any.whl
urllib3.connectionpool : Starting new HTTPS connection (1): github.com
urllib3.connectionpool : https://github.com:443 "GET /panchagnula/azure-cli-extensions/raw/sisirap-extensions-whl/dist/webapp-0.0.8-py2.py3-none-any.whl HTTP/1.1" 302 194
urllib3.connectionpool : Starting new HTTPS connection (1): raw.githubusercontent.com
urllib3.connectionpool : https://raw.githubusercontent.com:443 "GET /panchagnula/azure-cli-extensions/sisirap-extensions-whl/dist/webapp-0.0.8-py2.py3-none-any.whl HTTP/1.1" 200 8686
Downloaded to /tmp/tmp4ly546cw/webapp-0.0.8-py2.py3-none-any.whl
Validating the extension /tmp/tmp4ly546cw/webapp-0.0.8-py2.py3-none-any.whl
Checksum of /tmp/tmp4ly546cw/webapp-0.0.8-py2.py3-none-any.whl is OK
Extension compatibility result: is_compatible=True cli_core_version=2.0.27 min_required=None max_required=None
Validation successful on /tmp/tmp4ly546cw/webapp-0.0.8-py2.py3-none-any.whl
Executing pip with args: ['install', '--target', '/home/ben/.azure/cliextensions/webapp', '/tmp/tmp4ly546cw/webapp-0.0.8-py2.py3-none-any.whl']
Running: ['/opt/az/bin/python3', '-m', 'pip', 'install', '--target', '/home/ben/.azure/cliextensions/webapp', '/tmp/tmp4ly546cw/webapp-0.0.8-py2.py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/opt/az/lib/python3.6/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/opt/az/lib/python3.6/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/opt/az/lib/python3.6/site-packages/pip/__init__.py", line 28, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/opt/az/lib/python3.6/site-packages/pip/vcs/mercurial.py", line 9, in <module>
    from pip.download import path_to_url
  File "/opt/az/lib/python3.6/site-packages/pip/download.py", line 36, in <module>
    from pip.utils.glibc import libc_ver
  File "/opt/az/lib/python3.6/site-packages/pip/utils/glibc.py", line 4, in <module>
    import ctypes
  File "/opt/az/lib/python3.6/ctypes/__init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array
ImportError: libffi.so.5: cannot open shared object file: No such file or directory

Command '['/opt/az/bin/python3', '-m', 'pip', 'install', '--target', '/home/ben/.azure/cliextensions/webapp', '/tmp/tmp4ly546cw/webapp-0.0.8-py2.py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 1.
Pip failed so deleting anything we might have installed at /home/ben/.azure/cliextensions/webapp
An error occurred. Pip failed with status code 1. Use --debug for more information.

-----

[derekbekoe]

@benc-uk

I suspect you’re on 16.04, you should have the ~xenial version. I suspect you have the ~wheezy version as I reproduced this with those steps.
You can use ‘dpkg -s azure-cli’ to check yourself.

dpkg -s azure-cli
Package: azure-cli
Status: install ok installed
Priority: extra
Section: python
Installed-Size: 262909
Maintainer: Azure Python CLI Team <[email protected]>
Architecture: all
Version: 2.0.27-1~wheezy

To get the appropriate package for your distribution, remove the current version of the CLI (apt-get remove azure-cli) and follow the steps on our install page.

We recently updated the install page to add this step (AZ_REPO=$(lsb_release -cs)) which will get the correct package for your distro.
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-apt

Once done, this is what you should see:

# dpkg -s azure-cli
Package: azure-cli
Status: install ok installed
Priority: extra
Section: python
Installed-Size: 284525
Maintainer: Azure Python CLI Team <[email protected]>
Architecture: all
Version: 2.0.27-1~xenial

The extension installs fine after that.

[benc-uk]

Ahhh ha! Yes this was the problem I had. Reinstalling the CLI with the "AZ_REPO=$(lsb_release -cs)" stuff fixed it and I'm now on the xenial version.

Aren't a lot of people that have installed the CLI on WSL longer than 2-3 weeks ago, going to hit this issue? (and others presumably!) by being locked into the wrong branch in their apt sources?

[derekbekoe]

Yes potentially. The install page is up-to-date though so following those instructions would resolve this.

We needed to create more .deb packages due to Azure/azure-cli#3720 where users could not even run az login on some distros and we weren't supporting the latest versions of Ubuntu and Debian.