k8s_info returned is successful == true when the api-server was not reachable.

[wmlynch]

SUMMARY

The k8s_info module will return successful == true after the resource cache has been established during periods where communication to the api-server is not possible. The Recreate steps listed below simulate situations where a playbook contains a series of k8s* module tasks. After a handful of successful k8s* tasks are run, the communication to the api-server becomes problematic due to temporary/intermittent availability/communication issues. During this api-server "problematic" phase, the k8s_info based tasks continue to return successful == true with an empty resources list.

If kubectl get ... would fail due to an api-server with intermittent availability/communication problems, so should k8s_info.

ISSUE TYPE

• Bug Report

COMPONENT NAME

• The k8s_info module. Possibly other kubernetes.core modules experience the same behavior

ANSIBLE VERSION

ansible --version
ansible [core 2.13.3]
  config file = /Users/wmlynch/armada-dev/src/[redacted]/ansible.cfg
  configured module search path = ['/Users/wmlynch/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/wmlynch/armada-dev/src/[redacted]/.venv/lib/python3.9/site-packages/ansible
  ansible collection location = /Users/wmlynch/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/wmlynch/armada-dev/src/[redacted]/.venv/bin/ansible
  python version = 3.9.13 (v3.9.13:6de2ca5339, May 17 2022, 11:23:25) [Clang 6.0 (clang-600.0.57)]
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION

ansible-galaxy collection list

# /Users/wmlynch/armada-dev/src/[redacted]/.venv/lib/python3.9/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    3.2.0  
ansible.netcommon             3.0.1  
ansible.posix                 1.4.0  
ansible.utils                 2.6.1  
ansible.windows               1.10.0 
arista.eos                    5.0.1  
awx.awx                       21.0.0 
azure.azcollection            1.12.0 
check_point.mgmt              2.3.0  
chocolatey.chocolatey         1.2.0  
cisco.aci                     2.2.0  
cisco.asa                     3.0.0  
cisco.dnac                    6.4.0  
cisco.intersight              1.0.19 
cisco.ios                     3.0.0  
cisco.iosxr                   3.0.0  
cisco.ise                     2.4.1  
cisco.meraki                  2.6.2  
cisco.mso                     2.0.0  
cisco.nso                     1.0.3  
cisco.nxos                    3.0.0  
cisco.ucs                     1.8.0  
cloud.common                  2.1.1  
cloudscale_ch.cloud           2.2.2  
community.aws                 3.2.1  
community.azure               1.1.0  
community.ciscosmb            1.0.5  
community.crypto              2.3.2  
community.digitalocean        1.19.0 
community.dns                 2.1.1  
community.docker              2.6.0  
community.fortios             1.0.0  
community.general             5.0.2  
community.google              1.0.0  
community.grafana             1.4.0  
community.hashi_vault         3.0.0  
community.hrobot              1.3.1  
community.libvirt             1.1.0  
community.mongodb             1.4.0  
community.mysql               3.2.1  
community.network             4.0.1  
community.okd                 2.2.0  
community.postgresql          2.1.5  
community.proxysql            1.4.0  
community.rabbitmq            1.2.1  
community.routeros            2.1.0  
community.sap                 1.0.0  
community.sap_libs            1.1.0  
community.skydive             1.0.0  
community.sops                1.2.2  
community.vmware              2.5.0  
community.windows             1.10.0 
community.zabbix              1.7.0  
containers.podman             1.9.3  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.14 
dellemc.enterprise_sonic      1.1.1  
dellemc.openmanage            5.4.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.17.0 
fortinet.fortimanager         2.1.5  
fortinet.fortios              2.1.6  
frr.frr                       2.0.0  
gluster.gluster               1.0.2  
google.cloud                  1.0.2  
hetzner.hcloud                1.6.0  
hpe.nimble                    1.1.4  
ibm.qradar                    2.0.0  
infinidat.infinibox           1.3.3  
infoblox.nios_modules         1.2.2  
inspur.sm                     2.0.0  
junipernetworks.junos         3.0.1  
kubernetes.core               2.3.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.17.0
netapp.elementsw              21.7.0 
netapp.ontap                  21.19.1
netapp.storagegrid            21.10.0
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.3.0  
netbox.netbox                 3.7.1  
ngine_io.cloudstack           2.2.4  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.1  
openstack.cloud               1.8.0  
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   2.0.4  
purestorage.flasharray        1.13.0 
purestorage.flashblade        1.9.0  
sensu.sensu_go                1.13.1 
servicenow.servicenow         1.0.6  
splunk.es                     2.0.0  
t_systems_mms.icinga_director 1.29.0 
theforeman.foreman            3.4.0  
vmware.vmware_rest            2.1.5  
vyos.vyos                     3.0.1  
wti.remote                    1.0.3  

# /Users/wmlynch/.ansible/collections/ansible_collections
Collection       Version
---------------- -------
community.docker 2.2.0  

CONFIGURATION

ansible-config dump --only-changed
ANSIBLE_PIPELINING(/Users/wmlynch/armada-dev/src/[redacted]/ansible.cfg) = True
CALLBACKS_ENABLED(/Users/wmlynch/armada-dev/src/[redacted]/ansible.cfg) = ['timer', 'profile_roles']
DEFAULT_STDOUT_CALLBACK(/Users/wmlynch/armada-dev/src/[redacted]/ansible.cfg) = yaml

OS / ENVIRONMENT

sw_vers
ProductName:	macOS
ProductVersion:	12.4
BuildVersion:	21F79

STEPS TO REPRODUCE

I used kind to recreate but any kubernetes cluster will work.

1. kind create cluster --name test --kubeconfig /tmp/kind.kubeconfig --image kindest/node:v1.24.4
2. kubectl --kubeconfig /tmp/kind.kubeconfig create secret generic my-secret --from-literal=foo=bar
3. cp /tmp/kind.kubeconfig /tmp/botched.kubeconfig
4. Edit /tmp/botched.kubeconfig and remove the "certificate-authority-data:" line from the file.
5. ansible-playbook recreate-k8s-info-error.yml -vvv

# PLAYBOOK recreate-k8s-info-error.yml 

---
- hosts: localhost
  connection: local
  tasks:
  - name: Check for existing cluster secret with good kubeconfig
    k8s_info:
      api_version: v1
      kind: Secret
      name: 'my-secret'
      namespace: 'default'
      kubeconfig: '/tmp/kind.kubeconfig'
    register: _secret_data_a

# The expectation is that this will result in a failed task.
# However, this will return as a successful task.
  - name: Check for existing cluster secret with bad kubeconfig
    k8s_info:
      api_version: v1
      kind: Secret
      name: 'my-secret'
      namespace: 'default'
      kubeconfig: '/tmp/botched.kubeconfig'
    register: _secret_data_b

EXPECTED RESULTS

I expect that the "Check for existing cluster secret with bad kubeconfig" task to fail.

ACTUAL RESULTS

ansible-playbook recreate-k8s-info-error.yml -v
No config file found; using defaults
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] ***************************************************************************************************************************************************************************************************

TASK [Gathering Facts] *********************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Check for existing cluster secret with good kubeconfig] ******************************************************************************************************************************************************
ok: [localhost] => {"api_found": true, "changed": false, "resources": [{"apiVersion": "v1", "data": {"foo": "YmFy"}, "kind": "Secret", "metadata": {"creationTimestamp": "2022-09-07T03:51:50Z", "managedFields": [{"apiVersion": "v1", "fieldsType": "FieldsV1", "fieldsV1": {"f:data": {".": {}, "f:foo": {}}, "f:type": {}}, "manager": "kubectl-create", "operation": "Update", "time": "2022-09-07T03:51:50Z"}], "name": "my-secret", "namespace": "default", "resourceVersion": "1009", "uid": "8b96bebd-8d4d-46b7-9645-5abd85fc25d3"}, "type": "Opaque"}]}

TASK [Check for existing cluster secret with bad kubeconfig] *******************************************************************************************************************************************************
ok: [localhost] => {"api_found": true, "changed": false, "msg": "Exception 'HTTPSConnectionPool(host='127.0.0.1', port=55002): Max retries exceeded with url: /api/v1/namespaces/default/secrets/my-secret?fieldSelector=&labelSelector= (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))' raised while trying to get resource using {'name': 'my-secret', 'namespace': 'default', 'label_selector': '', 'field_selector': ''}", "resources": []}

PLAY RECAP *********************************************************************************************************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

[gravesm]

@wmlynch Thanks for reporting this. It should definitely fail if the cluster is unreachable at any point.