Incompatibility between requests 2.29.0 and/or urllib3 2.0.0 and vendored Docker SDK for Python code #611

felixfontein · 2023-04-28T09:51:29Z

SUMMARY

Currently there are a lot of CI failures. The problem seems to be urllib3 2.0.0, which has trouble with some requests versions: docker/docker-py#3113

ISSUE TYPE

Bug Report

COMPONENT NAME

various

felixfontein · 2023-04-28T09:55:07Z

02:33   File "/tmp/ansible_docker_container_copy_into_payload_xvaobvmd/ansible_docker_container_copy_into_payload.zip/ansible_collections/community/docker/plugins/module_utils/_api/api/client.py", line 236, in _put
02:33   File "/usr/local/lib/python3.10/dist-packages/requests/sessions.py", line 647, in put
02:33     return self.request("PUT", url, data=data, **kwargs)
02:33   File "/usr/local/lib/python3.10/dist-packages/requests/sessions.py", line 587, in request
02:33     resp = self.send(prep, **send_kwargs)
02:33   File "/usr/local/lib/python3.10/dist-packages/requests/sessions.py", line 701, in send
02:33     r = adapter.send(request, **kwargs)
02:33   File "/usr/local/lib/python3.10/dist-packages/requests/adapters.py", line 487, in send
02:33     resp = conn.urlopen(
02:33   File "/usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py", line 703, in urlopen
02:33     httplib_response = self._make_request(
02:33   File "/usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py", line 396, in _make_request
02:33     conn.request_chunked(method, url, **httplib_request_kw)
02:33 AttributeError: 'UnixHTTPConnection' object has no attribute 'request_chunked'
02:33 fatal: [testhost]: FAILED! => {
02:33     "changed": false,
02:33     "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 141, in <module>\n  File \"<stdin>\", line 133, in _ansiballz_main\n  File \"<stdin>\", line 81, in invoke_module\n  File \"/usr/lib/python3.10/runpy.py\", line 224, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib/python3.10/runpy.py\", line 96, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_docker_container_copy_into_payload_xvaobvmd/ansible_docker_container_copy_into_payload.zip/ansible_collections/community/docker/plugins/modules/docker_container_copy_into.py\", line 870, in <module>\n  File \"/tmp/ansible_docker_container_copy_into_payload_xvaobvmd/ansible_docker_container_copy_into_payload.zip/ansible_collections/community/docker/plugins/modules/docker_container_copy_into.py\", line 820, in main\n  File \"/tmp/ansible_docker_container_copy_into_payload_xvaobvmd/ansible_docker_container_copy_into_payload.zip/ansible_collections/community/docker/plugins/modules/docker_container_copy_into.py\", line 736, in copy_content_into_container\n  File \"/tmp/ansible_docker_container_copy_into_payload_xvaobvmd/ansible_docker_container_copy_into_payload.zip/ansible_collections/community/docker/plugins/module_utils/copy.py\", line 197, in put_file_content\n    d[types.FunctionType] = _deepcopy_atomic\n  File \"/tmp/ansible_docker_container_copy_into_payload_xvaobvmd/ansible_docker_container_copy_into_payload.zip/ansible_collections/community/docker/plugins/module_utils/copy.py\", line 40, in _put_archive\n    \n  File \"/tmp/ansible_docker_container_copy_into_payload_xvaobvmd/ansible_docker_container_copy_into_payload.zip/ansible_collections/community/docker/plugins/module_utils/_api/utils/decorators.py\", line 58, in inner\n  File \"/tmp/ansible_docker_container_copy_into_payload_xvaobvmd/ansible_docker_container_copy_into_payload.zip/ansible_collections/community/docker/plugins/module_utils/_api/api/client.py\", line 236, in _put\n  File \"/usr/local/lib/python3.10/dist-packages/requests/sessions.py\", line 647, in put\n    return self.request(\"PUT\", url, data=data, **kwargs)\n  File \"/usr/local/lib/python3.10/dist-packages/requests/sessions.py\", line 587, in request\n    resp = self.send(prep, **send_kwargs)\n  File \"/usr/local/lib/python3.10/dist-packages/requests/sessions.py\", line 701, in send\n    r = adapter.send(request, **kwargs)\n  File \"/usr/local/lib/python3.10/dist-packages/requests/adapters.py\", line 487, in send\n    resp = conn.urlopen(\n  File \"/usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py\", line 703, in urlopen\n    httplib_response = self._make_request(\n  File \"/usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py\", line 396, in _make_request\n    conn.request_chunked(method, url, **httplib_request_kw)\nAttributeError: 'UnixHTTPConnection' object has no attribute 'request_chunked'\n",
02:33     "module_stdout": "",
02:33     "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
02:33     "rc": 1
02:33 }

(https://dev.azure.com/ansible/community.docker/_build/results?buildId=75569&view=logs&j=7ae541fe-2c6c-5974-1d05-caf0c4a75c23&t=371a43b6-c3e8-552e-f689-7437df699ff8)

The full traceback is:
  File "/tmp/ansible_docker_container_info_payload_0cvgp4p3/ansible_docker_container_info_payload.zip/ansible_collections/community/docker/plugins/module_utils/common_api.py", line 117, in __init__
    super(AnsibleDockerClientBase, self).__init__(**self._connect_params)
  File "/tmp/ansible_docker_container_info_payload_0cvgp4p3/ansible_docker_container_info_payload.zip/ansible_collections/community/docker/plugins/module_utils/_api/api/client.py", line 188, in __init__
    self._version = self._retrieve_server_version()
  File "/tmp/ansible_docker_container_info_payload_0cvgp4p3/ansible_docker_container_info_payload.zip/ansible_collections/community/docker/plugins/module_utils/_api/api/client.py", line 212, in _retrieve_server_version
    raise DockerException(
fatal: [testhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_version": "auto",
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "debug": false,
            "docker_host": "unix://var/run/docker.sock",
            "name": "4eb7ab2918506c1974bd8c42c9150da028bff5b8e33df9f4c3347f77552e59f4",
            "ssl_version": null,
            "timeout": 60,
            "tls": false,
            "tls_hostname": null,
            "use_ssh_client": false,
            "validate_certs": false
        }
    },
    "msg": "Error connecting: Error while fetching server API version: request() got an unexpected keyword argument 'chunked'"
}

(https://github.com/ansible-collections/community.docker/actions/runs/4828909386/jobs/8603313113)

urllib3 2.0 (<https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html>) breaks <ansible-collections/community.docker#611> via <docker/docker-py#3113>.

felixfontein · 2023-04-28T10:44:36Z

After looking at things a bit more, I think the problem isn't really requests + urllib3, but a specific requests 2.29.0 feature: psf/requests#6226. Since Docker SDK for Python - and now us - define some HTTP adapters for Unix connections etc. that don't provide the native chunking abilities that urllib3 provides, they probably cause the failures.

felixfontein · 2023-04-28T10:51:00Z

The following excerpt of the first stack trace highlights this:

02:33   File "/usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py", line 396, in _make_request
02:33     conn.request_chunked(method, url, **httplib_request_kw)
02:33 AttributeError: 'UnixHTTPConnection' object has no attribute 'request_chunked'

Also it looks like urllib3 < 2.0.0 is installed in the AZP run that fails. (The GHA run does have urllib3 2.0.0.)

felixfontein · 2023-04-28T12:47:03Z

I created #612 to restrict requests to < 2.29.0 for now.

ntimo · 2023-04-29T16:00:46Z

@felixfontein any news when a new release of the collection will be published that contains the workarround

felixfontein · 2023-04-29T18:59:13Z

@ntimo do you have this problem with an EE? If not, this is nothing a new release of the collection will fix, since then you are responsible yourself to install requests.

fcecagno · 2023-04-30T19:53:34Z

In order to properly install the dependencies for the docker module, I had to do this:

- name: Install docker
  become: yes
  pip:
    name:
      - websocket-client==0.59.0
      - requests==2.28.2
      - urllib3==1.26.15
      - docker
      - docker-compose
    state: present

Is there a better way to do so?
websocket-client<1 was required for docker-compose.

felixfontein · 2023-04-30T20:05:18Z

@fcecagno right now there isn't (at least not that I'm aware of).

felixfontein · 2023-05-01T08:29:04Z

community.docker 3.4.4 is out which restricts requests and urllib3 to <2.29.0 and <2.0.0, respectively, until this is properly fixed.

See also related bug report for more details: ansible-collections/community.docker#611

fcecagno · 2023-05-02T00:36:19Z

@felixfontein should we wait for an Ansible release with community.docker 3.4.4? I'm not aware of the process, if you could please clarify.

felixfontein · 2023-05-02T10:11:13Z

@fcecagno please note that 3.4.4 only helps you if a) you read the docs, or b) you create an Execution Environment. For EEs you don't need Ansible, and for a) you don't need it either (https://github.com/ansible-collections/community.docker#collection-documentation). So there's no need to wait for another Ansible release for 3.4.4.

felixfontein · 2023-05-02T13:33:37Z

#613 hopefully resolves the requests 2.29.0 incompatibility.

felixfontein · 2023-05-02T14:02:35Z

Would be great if some of you have time to test #613, especially for the transports I don't really use (I only use Unix sockets; there are minimal SSH tests in CI, but that's it).

felixfontein · 2023-05-05T19:20:30Z

Docker SDK 6.1.0 has been released which fixes this problem. The main change is basically what we have in #613.

felixfontein · 2023-05-05T20:47:21Z

community.docker 3.4.5 is out and fixes this.

to resolve molecule test failure. More details: ansible-collections/community.docker#611 Signed-off-by: everettraven <[email protected]>

pustovit · 2024-05-09T22:30:09Z

Still, observe this issue on the newly installed ansible via pipx (pipx install --include-deps ansible)

{"changed": false, "msg": "Error connecting: Error while fetching server API version: HTTPConnection.request() got an unexpected keyword argument 'chunked'"}

- name: Run Portainer docker-compose up
  community.docker.docker_compose:
    project_src: /home/user/docker-compose/portainer
    state: present

felixfontein · 2024-05-10T15:52:21Z

@pustovit the docker_compose module (now deprecated) uses a old version of Docker SDK for Python which does not have a fix for this. You have to make sure to install the right version of requests.

It's better if you switch to the docker_compose_v2 module, which no longer uses docker-compose 1.x.y (which has been End of Life for several years by now).

to resolve molecule test failure. More details: ansible-collections/community.docker#611 Signed-off-by: everettraven <[email protected]>

felixfontein changed the title ~~CI: incompatibility between requests and urllib3~~ Incompatibility between requests 2.29.0 and vendored Docker SDK for Python code Apr 28, 2023

felixfontein mentioned this issue Apr 28, 2023

Restrict requests to < 2.29.0 #612

Merged

felixfontein changed the title ~~Incompatibility between requests 2.29.0 and vendored Docker SDK for Python code~~ Incompatibility between requests 2.29.0 and/or urllib3 2.0.0 and vendored Docker SDK for Python code May 1, 2023

felixfontein mentioned this issue May 1, 2023

Release plan #4

Open

rafaelgieschke pushed a commit to emulation-as-a-service/eaas-ansible that referenced this issue May 1, 2023

Fix incompatible dependencies for Docker's Python bindings

b338741

See also related bug report for more details: ansible-collections/community.docker#611

felixfontein mentioned this issue May 2, 2023

Make compatible with requests 2.29.0 and urllib3 2.0 #613

Merged

felixfontein mentioned this issue May 2, 2023

urllib3 v2 incompatibility docker/docker-py#3113

Closed

saltydk added a commit to saltyorg/Saltbox that referenced this issue May 3, 2023

docker: add temp workaround for ansible-collections/community.docker#611

ae83468

saltydk added a commit to saltyorg/Saltbox that referenced this issue May 3, 2023

docker: add temp workaround for ansible-collections/community.docker#611

06393f8

fosterseth mentioned this issue May 4, 2023

fix molecule tests by pinning docker ansible/awx-operator#1394

Closed

rooftopcellist mentioned this issue May 4, 2023

Fix molecule test by pinning community.docker for requests issue ansible/awx-operator#1396

Merged

Pterosaur mentioned this issue May 5, 2023

Cannot add-topo due to request() got an unexpected keyword argument 'chunked' sonic-net/sonic-mgmt#8221

Open

felixfontein closed this as completed May 5, 2023

knx-am mentioned this issue May 9, 2023

Support for wildcard ssl certificates garutilorenzo/docker-swarm-ingress#3

Closed

felixfontein mentioned this issue May 22, 2023

Get an error when trying to use community.docker.docker_container_info with latest version requests #635

Closed

ngnix mentioned this issue May 24, 2023

Update community.docker collection for fix issue with request lib. ansible-community/molecule-plugins#160

Closed

macfreek mentioned this issue Oct 29, 2023

Error connecting: Error while fetching server API version: HTTPConnection.request() got an unexpected keyword argument 'chunked' macfreek/empty-container#1

Closed

everettraven added a commit to everettraven/ansible-operator-plugins that referenced this issue Jan 12, 2024

bump community.docker to 3.4.5

8aad7a9

to resolve molecule test failure. More details: ansible-collections/community.docker#611 Signed-off-by: everettraven <[email protected]>

chiragkyal pushed a commit to chiragkyal/ansible-operator-plugins that referenced this issue Aug 8, 2024

bump community.docker to 3.4.5

7370398

to resolve molecule test failure. More details: ansible-collections/community.docker#611 Signed-off-by: everettraven <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Incompatibility between requests 2.29.0 and/or urllib3 2.0.0 and vendored Docker SDK for Python code #611

Incompatibility between requests 2.29.0 and/or urllib3 2.0.0 and vendored Docker SDK for Python code #611

felixfontein commented Apr 28, 2023

felixfontein commented Apr 28, 2023

felixfontein commented Apr 28, 2023

felixfontein commented Apr 28, 2023

felixfontein commented Apr 28, 2023

ntimo commented Apr 29, 2023

felixfontein commented Apr 29, 2023

fcecagno commented Apr 30, 2023

felixfontein commented Apr 30, 2023

felixfontein commented May 1, 2023

fcecagno commented May 2, 2023 •

edited

Loading

felixfontein commented May 2, 2023

felixfontein commented May 2, 2023

felixfontein commented May 2, 2023

felixfontein commented May 5, 2023

felixfontein commented May 5, 2023

pustovit commented May 9, 2024

felixfontein commented May 10, 2024

Incompatibility between requests 2.29.0 and/or urllib3 2.0.0 and vendored Docker SDK for Python code #611

Incompatibility between requests 2.29.0 and/or urllib3 2.0.0 and vendored Docker SDK for Python code #611

Comments

felixfontein commented Apr 28, 2023

SUMMARY

ISSUE TYPE

COMPONENT NAME

felixfontein commented Apr 28, 2023

felixfontein commented Apr 28, 2023

felixfontein commented Apr 28, 2023

felixfontein commented Apr 28, 2023

ntimo commented Apr 29, 2023

felixfontein commented Apr 29, 2023

fcecagno commented Apr 30, 2023

felixfontein commented Apr 30, 2023

felixfontein commented May 1, 2023

fcecagno commented May 2, 2023 • edited Loading

felixfontein commented May 2, 2023

felixfontein commented May 2, 2023

felixfontein commented May 2, 2023

felixfontein commented May 5, 2023

felixfontein commented May 5, 2023

pustovit commented May 9, 2024

felixfontein commented May 10, 2024

fcecagno commented May 2, 2023 •

edited

Loading