/bin/sh not supporting "set -o pipefail" option

[miron9]

Environment:

• DigitalOcean droplet 2Gb RAM, 2 vCPU

• OS:

root@master-0:~# uname -srm
Linux 5.4.0-51-generic x86_64
root@master-0:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

• Version of Ansible (ansible --version):

ansible 2.9.0
  config file = None
  configured module search path = ['/home/mirek/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/mirek/Projects/nonus.co.uk/ns2rent-project/env/lib/python3.8/site-packages/ansible
  executable location = /home/mirek/Projects/nonus.co.uk/ns2rent-project/env/bin/ansible
  python version = 3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0]

• Version of Python (python --version):

Python 3.8.5

Kubespray version (commit) (git rev-parse --short HEAD):
8ee1b70

Network plugin used:
Calico/Default

Full inventory with variables (ansible -i inventory/sample/inventory.ini all -m debug -a "var=hostvars[inventory_hostname]"):

master-0 | SUCCESS => {
    "hostvars[inventory_hostname]": {
        "ansible_check_mode": false,
        "ansible_diff_mode": false,
        "ansible_facts": {},
        "ansible_forks": 5,
        "ansible_host": "192.168.128.1",
        "ansible_inventory_sources": [
            "/tmp/kubespray_inventory.yaml"
        ],
        "ansible_playbook_python": "/home/mirek/Projects/nonus.co.uk/ns2rent-project/env/bin/python",
        "ansible_ssh_common_args": "-C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no",
        "ansible_user": "root",
        "ansible_verbosity": 0,
        "ansible_version": {
            "full": "2.9.0",
            "major": 2,
            "minor": 9,
            "revision": 0,
            "string": "2.9.0"
        },
        "group_names": [
            "etcd",
            "k8s-cluster",
            "kube-master"
        ],
        "groups": {
            "all": [
                "master-0",
                "node-0"
            ],
            "calico-rr": [],
            "etcd": [
                "master-0"
            ],
            "k8s-cluster": [
                "master-0",
                "node-0"
            ],
            "kube-master": [
                "master-0"
            ],
            "kube-node": [
                "node-0"
            ],
            "ungrouped": []
        },
        "inventory_dir": "/tmp",
        "inventory_file": "/tmp/kubespray_inventory.yaml",
        "inventory_hostname": "master-0",
        "inventory_hostname_short": "master-0",
        "ip": "192.168.128.1",
        "omit": "__omit_place_holder__0477c12aeb1d644db699c9d221d47597fbfc59e9",
        "playbook_dir": "/home/mirek/Projects/nonus.co.uk/ns2rent-project"
    }
}
node-0 | SUCCESS => {
    "hostvars[inventory_hostname]": {
        "ansible_check_mode": false,
        "ansible_diff_mode": false,
        "ansible_facts": {},
        "ansible_forks": 5,
        "ansible_host": "192.168.128.2",
        "ansible_inventory_sources": [
            "/tmp/kubespray_inventory.yaml"
        ],
        "ansible_playbook_python": "/home/mirek/Projects/nonus.co.uk/ns2rent-project/env/bin/python",
        "ansible_ssh_common_args": "-C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no",
        "ansible_user": "root",
        "ansible_verbosity": 0,
        "ansible_version": {
            "full": "2.9.0",
            "major": 2,
            "minor": 9,
            "revision": 0,
            "string": "2.9.0"
        },
        "group_names": [
            "k8s-cluster",
            "kube-node"
        ],
        "groups": {
            "all": [
                "master-0",
                "node-0"
            ],
            "calico-rr": [],
            "etcd": [
                "master-0"
            ],
            "k8s-cluster": [
                "master-0",
                "node-0"
            ],
            "kube-master": [
                "master-0"
            ],
            "kube-node": [
                "node-0"
            ],
            "ungrouped": []
        },
        "inventory_dir": "/tmp",
        "inventory_file": "/tmp/kubespray_inventory.yaml",
        "inventory_hostname": "node-0",
        "inventory_hostname_short": "node-0",
        "ip": "192.168.128.2",
        "omit": "__omit_place_holder__0477c12aeb1d644db699c9d221d47597fbfc59e9",
        "playbook_dir": "/home/mirek/Projects/nonus.co.uk/ns2rent-project"
    }
}

Command used to invoke ansible:
ansible-playbook ansible/kubespray/remove-node.yml --private-key=assets/ssh/ns2retn -i /tmp/kubespray_inventory.yaml --extra-vars "node=node-0" -v

Output of ansible run:

[redacted as all task ran fine until this one]
...
TASK [remove-node/pre-remove : remove-node | Identify nodes to drain, ignore non-cluster nodes] ****************************************************************************************************************
failed: [master-0 -> 192.168.128.1] (item=node-0) => {"ansible_loop_var": "item", "changed": false, "cmd": "set -o pipefail\n/usr/local/bin/kubectl get nodes -o json              | jq .items[].metadata.name              | jq \"select(. | test(\\\"^node-0$\\\"))\"\n", "delta": "0:00:00.003542", "end": "2021-03-08 15:07:29.550398", "item": "node-0", "msg": "non-zero return code", "rc": 2, "start": "2021-03-08 15:07:29.546856", "stderr": "/bin/sh: 1: set: Illegal option -o pipefail", "stderr_lines": ["/bin/sh: 1: set: Illegal option -o pipefail"], "stdout": "", "stdout_lines": []}

NO MORE HOSTS LEFT *********************************************************************************************************************************************************************************************

PLAY RECAP *****************************************************************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
master-0                   : ok=15   changed=0    unreachable=0    failed=1    skipped=36   rescued=0    ignored=0   

Anything else do we need to know:
The command is executed on the master node with Shell (/bin/sh) which is not supporting the "pipefail" option. the Task works if /bin/bash is specified as the executable for "shell" block lie presented below.

In roles/remove-node/pre-remove/tasks/main.yml

- name: remove-node | Identify nodes to drain, ignore non-cluster nodes
  shell: |
    set -o pipefail
    {{ bin_dir }}/kubectl get nodes -o json \
                  | jq .items[].metadata.name \
                  | jq "select(. | test(\"^{{ hostvars[item]['kube_override_hostname']|default(item) }}$\"))"
  loop: "{{ node.split(',') | default(groups['kube-node']) }}"
  register: nodes
  delegate_to: "{{ groups['kube-master']|first }}"
  changed_when: false
  run_once: true
  args:
    executable: bash

So fat I had a problem with this one call to shell only so not making any other suggestion as just to change this particular shell to BASH.

[sathieu]

This bug is marked as fixed while there are other instances of set -o pipefail in the code.

SOmeone can reopen?

[champtar]

The other instances have executable set to /bin/bash