$HOME and ~ point to different locations

[yajo]

By running bash in the container, you can get to this session:

bash-4.4# echo ~root
/root
bash-4.4# whoami
root
bash-4.4# echo ~
/home/runner
bash-4.4# echo $HOME
/home/runner

This discrepancy makes makes that i.e. ~/.ssh and $HOME/.ssh are different. As that folder is so important for Ansible, you can imagine the problems this supposes.

FTR ssh seems to seek into ~root, not into ~:

ssh $args -vv
[...]
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/id_rsa
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Trying private key: /root/.ssh/id_ecdsa
debug1: Trying private key: /root/.ssh/id_ed25519
debug1: Trying private key: /root/.ssh/id_xmss
debug2: we did not send a packet, disable method
debug1: Next authentication method: password
[...]

Workaround: run container with -e HOME=/root.

@moduon MT-136

[ssbarnea]

Yep, I can confirm that and to be it looks like a read bug caused by base ansible-runner base image, where PWD !- HOME, something I cannot see any reason for.

$ docker run -it quay.io/ansible/ansible-runner:latest /bin/bash
bash-4.4# echo $HOME
/home/runner
bash-4.4# pwd
/runner
bash-4.4# ls
artifacts  env  inventory  project
bash-4.4# 

I am going to ask ansible-runner folks about this...

[nitzmahone]

Ah I see someone's already hit this: ansible/ansible-runner#1024 - yeah, this problem totally breaks async tasks locally, and will probably break become in some cases, among other things. It "helps" that container user is root, so even though it's trying to splat files/dirs in the wrong places, it at least has permissions to do so, but any dissociated case where those two values need to line up will be broken.

[ssbarnea]

We really need to find a fix for this ASAP. Thanks for raising ansible/ansible-runner#1027 (review)

[jtrutwin]

Just hit this today:

TASK [Wait for instance(s) creation to complete] *******************************
failed: [localhost] (item={'failed': 0, 'started': 1, 'finished': 0, 'ansible_job_id': '633121066512.658', 'results_file': '/home/runner/.ansible_async/633121066512.658', 'changed': True, 'item': {'command': ''
, 'image': '/rhel7-ansible-molecule:latest', 'name': 'aws-rhel7-gold-ami', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']}, 'ansible_lo
op_var': 'item'}) => {"ansible_job_id": "633121066512.658", "ansible_loop_var": "item", "attempts": 1, "changed": false, "finished": 1, "item": {"ansible_job_id": "633121066512.658", "ansible_loop_var": "item",
"changed": true, "failed": 0, "finished": 0, "item": {"command": "", "image": "/rhel7-ansible-molecule:latest", "name": "aws-rhel7-gold-ami", "pre_build_image": true, "privileged":
true, "volumes": ["/sys/fs/cgroup:/sys/fs/cgroup:ro"]}, "results_file": "/home/runner/.ansible_async/633121066512.658", "started": 1}, "msg": "could not find job", "results_file": "/root/.ansible_async/6331210
66512.658", "started": 1, "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}

Not sure if there's a workaround?
Thank you

[IncredibleLou]

Any news about this bug?

I fixed it with a custom entrypoint described here: https://github.com/ansible/ansible-runner/pull/1027/files#diff-4d820c4986f8adb686a55c9ce8318cc2f6658479815a40c4ac7e9af28f7aed95

But I would like to remove the workaround and go back to the official image.

[ssbarnea]

We are still waiting for the official source images to be updated and it might take some more time. AFAIK the build process is completely changed and once done we should also get other problems sorted, like the lack of an arm64 image.

[ssbarnea]

Since yesterday, this is fixed as we no longer use ansible-builder.