common: k8s module import fails due to upstream Ansible bug #162

amin224 · 2021-07-05T09:57:37Z

ISSUE TYPE

Bug Report

COMPONENT NAME

kubernetes.core.k8s_info

ENVIRONMENT

```
ansible --version
```

ansible 2.9.6
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/amin/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.8.10 (default, Jun  2 2021, 10:49:15) [GCC 9.4.0]

Operating System: Ubuntu 20.04
ansible-galaxy collection install kubernetes.core

SUMMARY

I am testing the Kubernetes module by creating a simple namespace resource but I am getting this Error and I cannot find a similar topic elsewhere.

ansible-playbook -i inventory.yml testingmain.yml --limit server1

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ValueError: ansible_collections.kubernetes.core.plugins.module_utils.k8sdynamicclient.__spec__ is None
fatal: [192.]: FAILED! => {"msg": "Unexpected failure during module execution.", "stdout": ""}

testingmain.yml

- name: Deploy Kubernetes resources in Ansible Playbook
  hosts: all
  gather_facts: true
  become: true

  tasks:

    - name: Ensure the myapp Namespace exists.
      kubernetes.core.k8s_info:
        kubeconfig: '~/.kube/config'
        api_version: v1
        kind: Namespace
        name: amintestingansible
        #state: present
      register: namespace

    - debug: msg= "{{ namespace.stdout.split('\n') }}"

while using verbose:

The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/ansible/executor/task_executor.py", line 146, in run
    res = self._execute()
  File "/usr/lib/python3/dist-packages/ansible/executor/task_executor.py", line 645, in _execute
    result = self._handler.run(task_vars=variables)
  File "/home/amin/.ansible/collections/ansible_collections/kubernetes/core/plugins/action/k8s.py", line 242, in run
    module_return = self._execute_module(module_name=self._task.action, module_args=new_module_args, task_vars=task_vars)
  File "/usr/lib/python3/dist-packages/ansible/plugins/action/__init__.py", line 815, in _execute_module
    (module_style, shebang, module_data, module_path) = self._configure_module(module_name=module_name, module_args=module_args, task_vars=task_vars)
  File "/usr/lib/python3/dist-packages/ansible/plugins/action/__init__.py", line 204, in _configure_module
    (module_data, module_style, module_shebang) = modify_module(module_name, module_path, module_args, self._templar,
  File "/usr/lib/python3/dist-packages/ansible/executor/module_common.py", line 1255, in modify_module
    (b_module_data, module_style, shebang) = _find_module_utils(module_name, b_module_data, module_path, module_args, task_vars, templar, module_compression,
  File "/usr/lib/python3/dist-packages/ansible/executor/module_common.py", line 1094, in _find_module_utils
    recursive_finder(module_name, remote_module_fqn, b_module_data, py_module_names,
  File "/usr/lib/python3/dist-packages/ansible/executor/module_common.py", line 885, in recursive_finder
    recursive_finder(py_module_file[-1], next_fqn, py_module_cache[py_module_file][0],
  File "/usr/lib/python3/dist-packages/ansible/executor/module_common.py", line 730, in recursive_finder
    module_info = CollectionModuleInfo(py_module_name[-idx],
  File "/usr/lib/python3/dist-packages/ansible/executor/module_common.py", line 650, in __init__
    self.get_source()
  File "/usr/lib/python3/dist-packages/ansible/executor/module_common.py", line 666, in get_source
    data = pkgutil.get_data(to_native(self._package_name), to_native(self._mod_name + '.py'))
  File "/usr/lib/python3.8/pkgutil.py", line 619, in get_data
    spec = importlib.util.find_spec(package)
  File "/usr/lib/python3.8/importlib/util.py", line 114, in find_spec
    raise ValueError('{}.__spec__ is None'.format(name))
ValueError: ansible_collections.kubernetes.core.plugins.module_utils.k8sdynamicclient.__spec__ is None
fatal: [192.]: FAILED! => {
    "msg": "Unexpected failure during module execution.",
    "stdout": ""
}

The text was updated successfully, but these errors were encountered:

Akasurde · 2021-07-05T10:22:22Z

@amin224 Thanks for reporting this issue. Could you please update Ansible to 2.9.17 or latest and retry?

amin224 · 2021-07-05T12:23:28Z

@Akasurde Please check below results as you requested. Unfortunately I am getting the same error when running the playbook.

$sudo -H pip3 install ansible==2.9.17

[sudo] password for user: 
Collecting ansible==2.9.17
  Downloading ansible-2.9.17.tar.gz (14.3 MB)
     |████████████████████████████████| 14.3 MB 104 kB/s 
Requirement already satisfied: PyYAML in /usr/lib/python3/dist-packages (from ansible==2.9.17) (5.3.1)
Requirement already satisfied: cryptography in /usr/lib/python3/dist-packages (from ansible==2.9.17) (2.8)
Collecting jinja2
  Downloading Jinja2-3.0.1-py3-none-any.whl (133 kB)
     |████████████████████████████████| 133 kB 79 kB/s 
Collecting MarkupSafe>=2.0
  Downloading MarkupSafe-2.0.1-cp36-cp36m-manylinux2010_x86_64.whl (30 kB)
Building wheels for collected packages: ansible
  Building wheel for ansible (setup.py) ... done
  Created wheel for ansible: filename=ansible-2.9.17-py3-none-any.whl size=16199629 sha256=e67e78c97b48dc62053fc1ce21bb053
  Stored in directory: /root/.cache/pip/wheels/98/4c/fa/e2e437f7591e9e5916a53194c7
Successfully built ansible
Installing collected packages: MarkupSafe, jinja2, ansible
  Attempting uninstall: MarkupSafe
    Found existing installation: MarkupSafe 1.1.0
    Not uninstalling markupsafe at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'MarkupSafe'. No files were found to uninstall.
Successfully installed MarkupSafe-2.0.1 ansible-2.9.17 jinja2-3.0.1

$ ansible --version

ansible 2.9.17
  config file = None
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.6/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.6.14 (default, Jul  3 2021, 18:10:19) [GCC 9.3.0]

ansible-galaxy collection install kubernetes.core

Process install dependency map
Starting collection install process
Installing 'kubernetes.core:2.1.1' to '/home/user/.ansible/collections/ansible_collections/kubernetes/core'

$ ansible-doc kubernetes.core.k8s


> K8S    (/home/user/.ansible/collections/ansible_collections/kubernetes/core/plugins/modules/k8s.py)

        Use the Kubernetes Python client to perform CRUD operations on K8s objects. Pass the object definition from a source file or inline. See examples for
        reading files and using Jinja templates or vault-encrypted files. Access to the full range of K8s APIs. Use the [kubernetes.core.k8s_info] module to
        obtain a list of items about an object of type `kind' Authenticate using either a config file, certificates, password or token. Supports check mode.

  * This module is maintained by The Ansible Community
  * note: This module has a corresponding action plugin.

amin224 · 2021-07-05T12:59:23Z

ansible/ansible#68361 (comment)

My issue is similar to this output, and they are suggesting to upgrade to 2.10 I will try and share results.

ValueError: ansible_collections.freeipa.ansible_freeipa.plugins.module_utils.ansible_ipa_server.__spec__ is None
fatal: [cmds1-dev.tensor-tech.io]: FAILED! => {
    "msg": "Unexpected failure during module execution.",
    "stdout": ""

Akasurde · 2021-07-05T13:13:15Z

The issue you connected is not working. Could you please correct the link?

amin224 · 2021-07-05T13:18:50Z

Apologies. Fixed

Akasurde · 2021-07-05T13:22:26Z

@amin224 Is it possible for you to upgrade to 2.10 or the latest?

amin224 · 2021-07-05T13:24:57Z

Yes I will install ansible 2.10 and share results hopefully it works.

Akasurde · 2021-07-05T13:37:58Z

I am trying to reproduce this issue on 2.9 with the Kubernetes collection.

To workaround Ansible upstream bug, import k8sdynamicclient directly. Fixes: ansible-collections#162 Signed-off-by: Abhijeet Kasurde <[email protected]>

Akasurde · 2021-07-06T05:54:30Z

@amin224 Could you please try PR #163 and let me know if it works for you? Thanks in advance.

amin224 · 2021-07-06T09:09:50Z

@Akasurde Good day,

I tested both versions of ansible 2.9.6 and ansible 2.10.11 in 2 different VMs

In ansible version 2.9.6 I edited the lines of code in the path /home/amin/.ansible/collections/ansible_collections/kubernetes/core/plugins/module_utils/common.py

    try:
        client = k8sdynamicclient(kubernetes.client.ApiClient(configuration), discoverer=LazyDiscoverer)
        #client = K8SDynamicClient(kubernetes.client.ApiClient(configuration), discoverer=LazyDiscoverer)
    except Exception as err:
        _raise_or_fail(err, 'Failed to get client due to %s')

    get_api_client._pool[digest] = client
    return client

IMP_K8S_CLIENT = None
try:
    from ansible_collections.kubernetes.core.plugins.module_utils import k8sdynamicclient
   # from ansible_collections.kubernetes.core.plugins.module_utils.k8sdynamicclient import K8SDynamicClient
    from ansible_collections.kubernetes.core.plugins.module_utils.client.discovery import LazyDiscoverer
    IMP_K8S_CLIENT = True

After running the ansible-playbook ansible-playbook -i inventory.yml testingmain.yml --ask-become-pass --limit server1 -vvv
I am getting this error

The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_kubernetes.core.k8s_payload_gnbmr_pi/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/common.py", line 92, in <module>
    from kubernetes.dynamic.resource import ResourceInstance
ModuleNotFoundError: No module named 'kubernetes'
fatal: [192.]: FAILED! => {
    "changed": false,
    "error": "No module named 'kubernetes'",
    "invocation": {
        "module_args": {
            "api_key": null,
            "api_version": "v1",
            "append_hash": false,
            "apply": false,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "context": null,
            "continue_on_error": false,
            "delete_options": null,
            "force": false,
            "host": null,
            "kind": "Namespace",
            "kubeconfig": "/root/.kube/config",
            "merge_type": null,
            "name": "amintestingansible",
            "namespace": null,
            "password": null,
            "persist_config": null,
            "proxy": null,
            "proxy_headers": null,
            "resource_definition": null,
            "src": null,
            "state": "present",
            "template": null,
            "username": null,
            "validate": null,
            "validate_certs": null,
            "wait": false,
            "wait_condition": null,
            "wait_sleep": 5,
            "wait_timeout": 120
        }
    },
    "msg": "Failed to import the required Python library (kubernetes) on ubunt's Python /usr/bin/python. Please read module documentation and install in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"
}

In ansible version 2.10.11, without changing the common.py file I received the following error as well. I will try to troubleshoot the problem and share results.

The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_kubernetes.core.k8s_payload_huhjlej8/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/common.py", line 91, in <module>
    from kubernetes.dynamic.resource import ResourceInstance
ModuleNotFoundError: No module named 'kubernetes'
fatal: [local]: FAILED! => {
    "changed": false,
    "error": "No module named 'kubernetes'",
    "invocation": {
        "module_args": {
            "api_key": null,
            "api_version": "v1",
            "append_hash": false,
            "apply": false,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "context": null,
            "continue_on_error": false,
            "delete_options": null,
            "force": false,
            "host": null,
            "kind": "Namespace",
            "kubeconfig": "/home/user/.kube/config",
            "merge_type": null,
            "name": "amintestingansible",
            "namespace": null,
            "password": null,
            "persist_config": null,
            "proxy": null,
            "proxy_headers": null,
            "resource_definition": null,
            "src": null,
            "state": "present",
            "template": null,
            "username": null,
            "validate": null,
            "validate_certs": null,
            "wait": false,
            "wait_condition": null,
            "wait_sleep": 5,
            "wait_timeout": 120
        }
    },
    "msg": "Failed to import the required Python library (kubernetes) on ubunt's Python /usr/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"
}

Akasurde · 2021-07-06T09:44:55Z

@amin224 Please copy exactly as per the following -

diff --git a/plugins/module_utils/common.py b/plugins/module_utils/common.py
index 48e55ec..844d20a 100644
--- a/plugins/module_utils/common.py
+++ b/plugins/module_utils/common.py
@@ -53,7 +53,7 @@ except ImportError as e:

 IMP_K8S_CLIENT = None
 try:
-    from ansible_collections.kubernetes.core.plugins.module_utils.k8sdynamicclient import K8SDynamicClient
+    from ansible_collections.kubernetes.core.plugins.module_utils import k8sdynamicclient
     from ansible_collections.kubernetes.core.plugins.module_utils.client.discovery import Lazy
Discoverer
     IMP_K8S_CLIENT = True
 except ImportError as e:
@@ -195,7 +195,7 @@ def get_api_client(module=None, **kwargs):
         return client

     try:
-        client = K8SDynamicClient(kubernetes.client.ApiClient(configuration), discoverer=LazyD
iscoverer)
+        client = k8sdynamicclient.K8SDynamicClient(kubernetes.client.ApiClient(configuration),
 discoverer=LazyDiscoverer)

and install kubernetes Client Python library on ubunt machine using

pip install kubernetes

or use the package manager to install it.

amin224 · 2021-07-06T10:45:03Z

@Akasurde Thank you for your time and much appreciated help!
After editing the common.py file as per requested and installing the Kubernetes module on the remote machine using

pip install kubernetes

There are no errors when deploying Kubernetes resources using ansible playbooks.

Please note that this issue is related to 2.9.x versions of ansible, but in version 2.10.11 I did not have to edit the common.py file.

Akasurde · 2021-07-06T11:36:38Z

@amin224 Sweet. Thanks for the feedback.

Akasurde added a commit to Akasurde/kubernetes.core that referenced this issue Jul 6, 2021

common: import k8sdynamicclient directly

2c490c7

To workaround Ansible upstream bug, import k8sdynamicclient directly. Fixes: ansible-collections#162 Signed-off-by: Abhijeet Kasurde <[email protected]>

Akasurde mentioned this issue Jul 6, 2021

common: import k8sdynamicclient directly #163

Merged

Akasurde self-assigned this Jul 6, 2021

Akasurde added has_pr type/bug Something isn't working verified The issue is reproduced labels Jul 6, 2021

Akasurde changed the title ~~ValueError: ansible_collections.kubernetes.core.plugins.module_utils.k8sdynamicclient.__spec__ is None~~ common: k8s module import fails due to upstream Ansible bug Jul 6, 2021

Akasurde closed this as completed in #163 Jul 9, 2021

This was referenced Sep 16, 2021

Update dependency kubernetes.core to v2.2.1 metal3-io/metal3-dev-env#795

Merged

Update dependency kubernetes.core to v2.2.3 - autoclosed Sunnatillo/metal3-dev-env#5

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

common: k8s module import fails due to upstream Ansible bug #162

common: k8s module import fails due to upstream Ansible bug #162

amin224 commented Jul 5, 2021

Akasurde commented Jul 5, 2021 •

edited

Loading

amin224 commented Jul 5, 2021

amin224 commented Jul 5, 2021 •

edited

Loading

Akasurde commented Jul 5, 2021

amin224 commented Jul 5, 2021

Akasurde commented Jul 5, 2021

amin224 commented Jul 5, 2021

Akasurde commented Jul 5, 2021

Akasurde commented Jul 6, 2021

amin224 commented Jul 6, 2021

Akasurde commented Jul 6, 2021

amin224 commented Jul 6, 2021

Akasurde commented Jul 6, 2021

common: k8s module import fails due to upstream Ansible bug #162

common: k8s module import fails due to upstream Ansible bug #162

Comments

amin224 commented Jul 5, 2021

Akasurde commented Jul 5, 2021 • edited Loading

amin224 commented Jul 5, 2021

amin224 commented Jul 5, 2021 • edited Loading

Akasurde commented Jul 5, 2021

amin224 commented Jul 5, 2021

Akasurde commented Jul 5, 2021

amin224 commented Jul 5, 2021

Akasurde commented Jul 5, 2021

Akasurde commented Jul 6, 2021

amin224 commented Jul 6, 2021

Akasurde commented Jul 6, 2021

amin224 commented Jul 6, 2021

Akasurde commented Jul 6, 2021

Akasurde commented Jul 5, 2021 •

edited

Loading

amin224 commented Jul 5, 2021 •

edited

Loading