Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ModuleNotFoundError for keyring.util.escape in Docker image (poetry 1.0.0 and 1.0.0b9) #1719

Closed
3 tasks done
TheButlah opened this issue Dec 12, 2019 · 14 comments · Fixed by #1788
Closed
3 tasks done
Labels
area/docs/faq Frequently duplicated/potential addition to FAQ kind/bug Something isn't working as expected

Comments

@TheButlah
Copy link
Contributor

TheButlah commented Dec 12, 2019

  • I am on the latest Poetry version.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
  • OS version and name: MacOS 10.14.6, running docker. Docker image is nvidia/cuda:10.2-cudnn7-devel-ubuntu18.04
  • Poetry version: Both 1.0.0b9 and 1.0.0

Issue

Attempted an install of poetry in my docker file. The install succeeds, but running any poetry commands gives a strange error. I've shown the docker build (trimmed down) along with the error below.

Docker Build output (Trimmed down for brevity):

Setting up container for folder or workspace: /Users/ryan.butler/Programming/robotics
Run: docker build -f /Users/ryan.butler/Programming/robotics/Dockerfile -t vsc-robotics-14e00cfc95af169a4eb8b5156cec6517 /Users/ryan.butler/Programming/robotics
Sending build context to Docker daemon  115.4MB
Step 1/44 : FROM nvidia/cuda:10.2-cudnn7-devel-ubuntu18.04
 ---> 81c7dbc267d8
....................................
Step 11/44 : RUN  update-alternatives --install /usr/bin/python python /usr/bin/python3 10   && update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 10
 ---> Using cache
 ---> 0cb8d16b47b8
.....................................
Step 18/44 : USER ${USERNAME}
 ---> Using cache
 ---> e18c506cbcad
......................................
Step 27/44 : SHELL ["/bin/zsh", "-c"]
 ---> Using cache
 ---> 426fe465c653
.......................................
Step 33/44 : ENV PY_USER_BIN="/home/${USERNAME}/.local/bin"
 ---> Using cache
 ---> 28e257b22543
Step 34/44 : RUN echo "PY_USER_BIN=$PY_USER_BIN"
 ---> Using cache
 ---> 9f22bf26abcc
Step 35/44 : ENV PATH=$PY_USER_BIN:$PATH
 ---> Using cache
 ---> ef605d0626e3
Step 36/44 : RUN pip list | grep keyring
 ---> Running in 9528402aa43c
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
keyring (10.6.0)
keyrings.alt (3.0)
Removing intermediate container 9528402aa43c
 ---> 294fa116e934
Step 37/44 : ENV PATH="/home/${USERNAME}/.poetry/bin:$PATH"
 ---> Running in b1b79a218bcc
Removing intermediate container b1b79a218bcc
 ---> 596f1ba898f6
Step 38/44 : RUN which python && which pip
 ---> Running in a5f223ed10f2
/usr/bin/python
/usr/bin/pip
Removing intermediate container a5f223ed10f2
 ---> 6fd46b40f5ee
Step 39/44 : RUN curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python
 ---> Running in eb812cbd3c07
Retrieving Poetry metadata

# Welcome to Poetry!

This will download and install the latest version of Poetry,
a dependency and package manager for Python.
.................................................

Poetry (1.0.0) is installed now. Great!

To get started you need Poetry's bin directory ($HOME/.poetry/bin) in your `PATH`
environment variable. Next time you log in this will be done
automatically.

To configure your current shell run `source $HOME/.poetry/env`

Removing intermediate container eb812cbd3c07
 ---> b0cfd3ab6a72
Step 40/44 : RUN which poetry
 ---> Running in 459a6af62814
/home/saicny/.poetry/bin/poetry
Removing intermediate container 459a6af62814
 ---> 3e2a9b083675
Step 41/44 : RUN poetry
 ---> Running in be08c33c0095
Traceback (most recent call last):
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/backend.py", line 203, in _load_plugins
    init_func = ep.load()
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/importlib_metadata/__init__.py", line 92, in load
    module = import_module(match.group('module'))
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/keyrings/alt/Windows.py", line 9, in <module>
    from . import file_base
  File "/usr/lib/python3/dist-packages/keyrings/alt/file_base.py", line 13, in <module>
    from keyring.util.escape import escape as escape_for_ini
ModuleNotFoundError: No module named 'keyring.util.escape'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/saicny/.poetry/bin/poetry", line 12, in <module>
    from poetry.console import main
  File "/home/saicny/.poetry/lib/poetry/console/__init__.py", line 1, in <module>
    from .application import Application
  File "/home/saicny/.poetry/lib/poetry/console/application.py", line 5, in <module>
    from .commands.about import AboutCommand
  File "/home/saicny/.poetry/lib/poetry/console/commands/__init__.py", line 4, in <module>
    from .check import CheckCommand
  File "/home/saicny/.poetry/lib/poetry/console/commands/check.py", line 1, in <module>
    from poetry.factory import Factory
  File "/home/saicny/.poetry/lib/poetry/factory.py", line 17, in <module>
    from .packages.dependency import Dependency
  File "/home/saicny/.poetry/lib/poetry/packages/__init__.py", line 7, in <module>
    from .dependency import Dependency
  File "/home/saicny/.poetry/lib/poetry/packages/dependency.py", line 10, in <module>
    from poetry.utils.helpers import canonicalize_name
  File "/home/saicny/.poetry/lib/poetry/utils/helpers.py", line 11, in <module>
    from keyring import delete_password
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/__init__.py", line 1, in <module>
    from .core import (
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/core.py", line 192, in <module>
    init_backend()
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/core.py", line 96, in init_backend
    filter(limit, backend.get_all_keyring()),
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/util/__init__.py", line 22, in wrapper
    func.always_returns = func(*args, **kwargs)
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/backend.py", line 216, in get_all_keyring
    _load_plugins()
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/backend.py", line 207, in _load_plugins
    log.exception("Error initializing plugin %s." % ep)
TypeError: not all arguments converted during string formatting
The command '/bin/zsh -c poetry' returned a non-zero code: 1
Failed: Building an image from the ../Dockerfile (this can take a while).
Command failed: docker build -f /Users/ryan.butler/Programming/robotics/Dockerfile -t vsc-robotics-14e00cfc95af169a4eb8b5156cec6517 /Users/ryan.butler/Programming/robotics
@TheButlah TheButlah added the kind/bug Something isn't working as expected label Dec 12, 2019
@TheButlah
Copy link
Contributor Author

Just realized that poetry 1.0.0 released this past hour, will update the issue once testing with the non-prerelease version.

@TheButlah TheButlah changed the title ModuleNotFoundError for keyring.util.escape in Docker image (poetry 1.0.0b9) ModuleNotFoundError for keyring.util.escape in Docker image (poetry 1.0.0 and 1.0.0b9) Dec 12, 2019
@TheButlah
Copy link
Contributor Author

Updated the issue, confirmed that the bug is present in 1.0.0 and 1.0.0b9

@TheButlah
Copy link
Contributor Author

TheButlah commented Dec 12, 2019

Possibly related, may describe proper solutions:
https://bugs.launchpad.net/usd-importer/+bug/1794041
https://stackoverflow.com/questions/53164278/missing-dependencies-causing-keyring-error-when-opening-spyder3-on-ubuntu18
vmware-archive/vcd-cli#235

It appears as though there is a bug with the system-installed version of keyring (10.6.0) or keyrings.alt (3.0). I've updated the error message to print out the version of keyring installed when poetry is installed. It appears as though poetry vendors the keyring library, but not the keyrings.alt library. Unsure if this matters.

@TheButlah
Copy link
Contributor Author

TheButlah commented Dec 12, 2019

I've tried upgrading keyrings.alt before poetry is installed, and the error changes:

Step 36/45 : RUN pip install -U keyring keyrings.alt
 ---> Running in 96a385e7336e
Collecting keyring
  Downloading https://files.pythonhosted.org/packages/a0/7b/77364ff41dda99cb9621c8a087ff38ffaddbe0a72ad776905a423858a47a/keyring-20.0.0-py2.py3-none-any.whl
Collecting keyrings.alt
  Downloading https://files.pythonhosted.org/packages/0c/3e/a39c3fb722ffdee50db5dff1e794a01cab7e0408bba1aa0e02e28a87667d/keyrings.alt-3.4.0-py2.py3-none-any.whl
...............................................
Successfully installed cffi-1.13.2 cryptography-2.8 importlib-metadata-1.3.0 jeepney-0.4.1 keyring-20.0.0 keyrings.alt-3.4.0 more-itertools-8.0.2 pycparser-2.19 secretstorage-3.1.1 six-1.13.0 zipp-0.6.0
Removing intermediate container 96a385e7336e
 ---> 139664e28eab
Step 37/45 : RUN pip list | grep keyring
 ---> Running in 6702023bd31c
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
keyring (20.0.0)
keyrings.alt (3.4.0)
Removing intermediate container 6702023bd31c
 ---> 134615591246
Step 38/45 : ENV PATH="/home/${USERNAME}/.poetry/bin:$PATH"
 ---> Running in 550219612ef1
Removing intermediate container 550219612ef1
 ---> 15ed24ddf21d
Step 39/45 : RUN which python && which pip
 ---> Running in c9e71ac87719
/usr/bin/python
/usr/bin/pip
Removing intermediate container c9e71ac87719
 ---> 7828a4af3068
Step 40/45 : RUN curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python
 ---> Running in dd4747c47683
Retrieving Poetry metadata

# Welcome to Poetry!

This will download and install the latest version of Poetry,
a dependency and package manager for Python.
..........................................
Poetry (1.0.0) is installed now. Great!

To get started you need Poetry's bin directory ($HOME/.poetry/bin) in your `PATH`
environment variable. Next time you log in this will be done
automatically.

To configure your current shell run `source $HOME/.poetry/env`

Removing intermediate container dd4747c47683
 ---> 88240b98122a
Step 41/45 : RUN which poetry
 ---> Running in 596dfddfb1e9
/home/saicny/.poetry/bin/poetry
Removing intermediate container 596dfddfb1e9
 ---> 9530a6d7ebe2
Step 42/45 : RUN poetry
 ---> Running in aa0fdbe1b237
Poetry version 1.0.0

USAGE

[UnicodeEncodeError]
'ascii' codec can't encode character '\xa0' in position 22: ordinal not in range(128)
The command '/bin/zsh -c poetry' returned a non-zero code: 1
Failed: Building an image from the ../Dockerfile (this can take a while).
Command failed: docker build -f /Users/ryan.butler/Programming/robotics/Dockerfile -t vsc-robotics-14e00cfc95af169a4eb8b5156cec6517 /Users/ryan.butler/Programming/robotics

@finswimmer
Copy link
Member

Hello @TheButlah ,

is this related: #221 ?

fin swimmer

@TheButlah
Copy link
Contributor Author

TheButlah commented Dec 13, 2019

No clue, but note that I'm not using poetry to install a project, but rather just trying to run any poetry command after installing the poetry tool itself.

That is, given the dockerfile setup I posted, the second of the two commands triggers the error:

curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python
poetry (any command here)

@abn
Copy link
Member

abn commented Dec 13, 2019

@TheButlah the issue that you are hitting here is the same as the root cause of #1427, which is described in #1427 (comment).

TLDR version is to add the following to your Dockerfile.

ENV LANG C.UTF-8

@TheButlah
Copy link
Contributor Author

TheButlah commented Dec 13, 2019

After adding

ENV LANG=C.UTF-8
RUN apt-get update && apt-get install -y language-pack-en && dpkg-reconfigure locales

to the top of my dockerfile, I get the same error when not upgrading keyrings.alt:

Step 36/45 : RUN echo $LANG
[7406 ms]  ---> Running in 4c1b696ef8c0
[8105 ms] C.UTF-8
[8976 ms] Removing intermediate container 4c1b696ef8c0
 ---> fb0d17af851c
Step 37/45 : ENV PATH="/home/${USERNAME}/.poetry/bin:$PATH"
[9073 ms]  ---> Running in fef81c248e8a
[9284 ms] Removing intermediate container fef81c248e8a
 ---> 34c17277f556
[9284 ms] Step 38/45 : RUN curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py |   python && [ -d "/home/${USERNAME}/.poetry" ]
[9451 ms]  ---> Running in 9968d27a7ead
[15691 ms] Retrieving Poetry metadata
...................................................................
Poetry (1.0.0) is installed now. Great!

To get started you need Poetry's bin directory ($HOME/.poetry/bin) in your `PATH`
environment variable. Next time you log in this will be done
automatically.

To configure your current shell run `source $HOME/.poetry/env`

[18063 ms] Removing intermediate container 9968d27a7ead
 ---> b5c22a0ce404
Step 39/45 : RUN poetry config virtualenvs.create false
[18223 ms]  ---> Running in c98de45552f6
[19215 ms] Traceback (most recent call last):
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/backend.py", line 203, in _load_plugins
    init_func = ep.load()
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/importlib_metadata/__init__.py", line 92, in load
    module = import_module(match.group('module'))
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/keyrings/alt/Windows.py", line 9, in <module>
    from . import file_base
 
[19215 ms]  File "/usr/lib/python3/dist-packages/keyrings/alt/file_base.py", line 13, in <module>
    from keyring.util.escape import escape as escape_for_ini
ModuleNotFoundError: No module named 'keyring.util.escape'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/saicny/.poetry/bin/poetry", line 12, in <module>
    from poetry.console import main
  File "/home/saicny/.poetry/lib/poetry/console/__init__.py", line 1, in <module>
    from .application import Application
  File "/home/saicny/.poetry/lib/poetry/console/application.py", line 5, in <module>
    from .commands.about import AboutCommand
  File "/home/saicny/.poetry/lib/poetry/console/commands/__init__.py", line 4, in <module>
    from .check import CheckCommand
  File "/home/saicny/.poetry/lib/poetry/console/commands/check.py", line 1, in <module>
    from poetry.factory import Factory
  File "/home/saicny/.poetry/lib/poetry/factory.py", line 17, in <module>
    from .pack
[19215 ms] ages.dependency import Dependency
  File "/home/saicny/.poetry/lib/poetry/packages/__init__.py", line 7, in <module>
    from .dependency import Dependency
  File "/home/saicny/.poetry/lib/poetry/packages/dependency.py", line 10, in <module>
    from poetry.utils.helpers import canonicalize_name
  File "/home/saicny/.poetry/lib/poetry/utils/helpers.py", line 11, in <module>
    from keyring import delete_password
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/__init__.py", line 1, in <module>
    from .core import (
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/core.py", line 192, in <module>
    init_backend()
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/core.py", line 96, in init_backend
    filter(limit, backend.get_all_keyring()),
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/util/__init__.py", line 22, in wrapper
    func.always_returns = func(*args, **kwargs)
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/backe
[19215 ms] nd.py", line 216, in get_all_keyring
    _load_plugins()
  File "/home/saicny/.poetry/lib/poetry/_vendor/py3.6/keyring/backend.py", line 207, in _load_plugins
    log.exception("Error initializing plugin %s." % ep)
TypeError: not all arguments converted during string formatting

[19880 ms] The command '/bin/zsh -c poetry config virtualenvs.create false' returned a non-zero code: 1

But, after adding pip install -U keyrings.alt before installing poetry, the error goes away:

Step 36/46 : RUN echo $LANG
[7457 ms]  ---> Running in 891cf7a6bda2
[8325 ms] C.UTF-8
[9150 ms] Removing intermediate container 891cf7a6bda2
 ---> 16fa9c6b579f
Step 37/46 : RUN pip install -U keyrings.alt
............................................................
[11420 ms] Successfully installed keyrings.alt-3.4.0
[12381 ms] Removing intermediate container ddaa87f1684e
[12381 ms]  ---> aca1f62904b0
[12381 ms] Step 38/46 : ENV PATH="/home/${USERNAME}/.poetry/bin:$PATH"
[12381 ms] 
[12514 ms]  ---> Running in 5acde667861e
[12753 ms] Removing intermediate container 5acde667861e
[12753 ms]  ---> 4f4f587ea812
Step 39/46 : RUN curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py |   python && [ -d "/home/${USERNAME}/.poetry" ]
[12892 ms]  ---> Running in b57574ff25e6
[19585 ms] Retrieving Poetry metadata
..........................................................
Poetry (1.0.0) is installed now. Great!

To get started you need Poetry's bin directory ($HOME/.poetry/bin) in your `PATH`
environment variable. Next time you log in this will be done
automatically.

To configure your current shell run `source $HOME/.poetry/env`

[21942 ms] Removing intermediate container b57574ff25e6
[21942 ms]  ---> 3b26468a1bb0
Step 40/46 : RUN poetry config virtualenvs.create false
[22074 ms]  ---> Running in d84fb08daafa
[24395 ms] Removing intermediate container d84fb08daafa
 ---> 00984bde2505

TLDR: upgrading keyrings.alt fixes the ModuleNotFound error, and LANG=C.UTF-8 fixes the unicode error.

@TheButlah
Copy link
Contributor Author

TheButlah commented Dec 13, 2019

I believe this indicates that poetry needs to vendor keyrings.alt, or at least update the required version of it, in order to fix this bug. A temporary workaround is to run pip install -U keyrings.alt before installing poetry, but ultimately this is something that should be fixed in poetry itself.

@abn
Copy link
Member

abn commented Dec 15, 2019

@TheButlah sorry, I should have been clearer on my comment. I was talking about the unicode issue only.

As for the issue regarding keyrings.alt, I am not sure what the best solution is. It is important to note that keyrings.alt package is not required for poetry's use of keyring. And as per the keyring package documentation it is an opt-in third-party backend.

While explicitly locking the minimum version will resolve the problem for this particular distribution but could open up concerns on other platforms. From the package documentation:

Keyrings in this package may have security risks or other implications. These backends were extracted from the main keyring project to make them available for those who wish to employ them, but are discouraged for general production use. Include this module and use its backends at your own risk.

The root issue (for the crash at least), however, seems like is a problem in the keyring backends error handling. And since all available (even unused) backends are dynamically loaded, the error is triggered. If the logging was fixed, it would simply log an exception and continue execution.

  File "/root/.poetry/lib/poetry/_vendor/py3.6/keyring/backend.py", line 208, in _load_plugins
    log.exception("Error initializing plugin %s." % ep)
TypeError: not all arguments converted during string formatting

Personally, I think poetry should not specify the package as a dependency as it does not explicitly use them.

@TheButlah
Copy link
Contributor Author

Ah, ok. I understand and agree on the rationale for not providing the backend as a dependency as it is system dependent and dynamically loaded. What then is the recommended course of action right now? Is this a bug in keyring and therefore I should file a bug report, or is the recommended course of action to upgrade keyrings.alt manually in my Dockerfile? If the latter, should this be added to poetry's documentation in a troubleshooting section?

@abn
Copy link
Member

abn commented Dec 15, 2019

@TheButlah I have submitted a PR for fixing the logging issue in keyring (jaraco/keyring#417).

Good questions regarding the next steps. I'd be inclined to say that this is a troubleshooting entry for poetry project if anything.

Hopefully, the distro bug will be resolved with an update and a new container image will be published with that for downstream consumption.

@TheButlah
Copy link
Contributor Author

TheButlah commented Dec 16, 2019

@abn To clarify, the PR you submitted was for the Unicode error or the ModuleNotFound error?

@abn abn added the area/docs/faq Frequently duplicated/potential addition to FAQ label Aug 15, 2020
Copy link

github-actions bot commented Mar 3, 2024

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/docs/faq Frequently duplicated/potential addition to FAQ kind/bug Something isn't working as expected
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants