CI: TypeError: mux_client_read_packet: read header failed: Broken pipe

[ricolin]

I found this issue in https://github.com/vexxhost/atmosphere/actions/runs/6045081164/job/16404592822?pr=551

but it's also reproducible within my local environment.
when executing Install all CRDs from roles/kube_prometheus_stack/tasks/main.yml
following error happens:

OpenSSH_8.9p1 Ubuntu-3ubuntu0.3, OpenSSL 3.0.2 15 Mar 2022\r
debug1: Reading configuration data /etc/ssh/ssh_config\r
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r
debug1: /etc/ssh/ssh_config line 21: Applying options for *\r
debug2: resolve_canonicalize: hostname 38.108.68.104 is address\r
debug3: expanded UserKnownHostsFile '/.ssh/known_hosts' -> '/home/rlin/.ssh/known_hosts'\r
debug3: expanded UserKnownHostsFile '/.ssh/known_hosts2' -> '/home/rlin/.ssh/known_hosts2'\r
debug1: auto-mux: Trying existing master\r
debug2: fd 3 setting O_NONBLOCK\r
debug2: mux_client_hello_exchange: master version 4\r
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r
debug3: mux_client_request_session: entering\r
debug3: mux_client_request_alive: entering\r
debug3: mux_client_request_alive: done pid = 613800\r
debug3: mux_client_request_session: session request sent\r
debug1: mux_client_request_session: master session id: 2\r
Traceback (most recent call last):
File "", line 107, in
File "", line 99, in _ansiballz_main
File "", line 47, in invoke_module
File "/usr/lib/python3.8/runpy.py", line 207, in run_module
return _run_module_code(code, init_globals, run_name, mod_spec)
File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/tmp/ansible_kubernetes.core.k8s_payload_v8tr1o5c/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s.py", line 486, in
File "/tmp/ansible_kubernetes.core.k8s_payload_v8tr1o5c/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s.py", line 482, in main
File "/tmp/ansible_kubernetes.core.k8s_payload_v8tr1o5c/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s.py", line 459, in execute_module
File "/tmp/ansible_kubernetes.core.k8s_payload_v8tr1o5c/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/common.py", line 807, in execute_module
File "/tmp/ansible_kubernetes.core.k8s_payload_v8tr1o5c/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/common.py", line 1114, in perform_action
File "/tmp/ansible_kubernetes.core.k8s_payload_v8tr1o5c/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8sdynamicclient.py", line 45, in apply
File "/tmp/ansible_kubernetes.core.k8s_payload_v8tr1o5c/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/apply.py", line 136, in k8s_apply
File "/usr/local/lib/python3.8/dist-packages/kubernetes/dynamic/client.py", line 164, in server_side_apply
return self.request('patch', path, body=body, force_conflicts=force_conflicts, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/kubernetes/dynamic/client.py", line 55, in inner
resp = func(self, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/kubernetes/dynamic/client.py", line 270, in request
api_response = self.client.call_api(
File "/usr/local/lib/python3.8/dist-packages/kubernetes/client/api_client.py", line 348, in call_api
return self.__call_api(resource_path, method,
File "/usr/local/lib/python3.8/dist-packages/kubernetes/client/api_client.py", line 180, in __call_api
response_data = self.request(
File "/usr/local/lib/python3.8/dist-packages/kubernetes/client/api_client.py", line 407, in request
return self.rest_client.PATCH(url,
File "/usr/local/lib/python3.8/dist-packages/kubernetes/client/rest.py", line 296, in PATCH
return self.request("PATCH", url,
File "/usr/local/lib/python3.8/dist-packages/kubernetes/client/rest.py", line 168, in request
request_body = json.dumps(body)
File "/usr/lib/python3.8/json/init.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/usr/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.class.name} '
TypeError: Object of type bytes is not JSON serializable
debug3: mux_client_read_packet: read header failed: Broken pipe\r
debug2: Received exit status from master 1\r
",
"module_stdout": "",
"msg": "MODULE FAILURE
See stdout/stderr for the exact error",
"rc": 1
}

[ricolin]

looks like it's related to 219f92c

[ricolin]

so I hitting this ansible-collections/kubernetes.core#548

[ricolin]

solution: bump kubernetes.core to 2.4.0 in vexxhost/ansible-collection-kubernetes and atmosphere to get kubernetes.core fix

[mnaser]

fixed by #510