Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add private CAs to the containers #376

Closed
suukit opened this issue Jun 7, 2021 · 21 comments · Fixed by #401
Closed

Add private CAs to the containers #376

suukit opened this issue Jun 7, 2021 · 21 comments · Fixed by #401
Assignees
Labels
type:bug Something isn't working

Comments

@suukit
Copy link
Contributor

suukit commented Jun 7, 2021

Hi,
to allow access to TLS sites using private CAs we need to add CA certificates to the AWX containers. Is there a native way to do so using AWX operator?

I used extra_volumes/ee_extra_volume_mounts to get crt files to /etc/pki/ca-trust/source/anchors/ but a run of update-ca-trust is missing. Is there a native way to accomplish adding own CAs?

Currently we got two use cases for that:

  1. fetching projects from GIT
  2. using the "uri" module in roles
    thanx in advance
    Max
@AndrewSav
Copy link

This comment describes a couple of way to customize an execution environment. I do not have a first hand experience with the ansible-builder but simply deriving a docker image from the "official" EE (in combination with extra volume mounting) works for me.

@tchellomello
Copy link
Contributor

@tchellomello tchellomello added state:in_progress type:bug Something isn't working labels Jun 10, 2021
@tchellomello tchellomello self-assigned this Jun 10, 2021
@tchellomello
Copy link
Contributor

tchellomello commented Jun 11, 2021

@suukit could you please give a try with the changes noted on this branch https://github.com/ansible/awx-operator/compare/devel...tchellomello:custom-ca?expand=1

To make it easier for you, I've published this testing image at https://quay.io/repository/tchellomello/awx-operator?tab=tags quay.io/tchellomello/awx-operator:custom-ca

So basically you can do the following steps:

  1. Update your awx-operator using this testing POC (see https://gist.github.com/tchellomello/e38c71248591034f8a7cc28421fe2245)
$ kubectl apply -f https://gist.githubusercontent.com/tchellomello/e38c71248591034f8a7cc28421fe2245/raw/b8c1d657553d33d8ba75bb077b5960bb5abbca3c/awx-operator.yml
  1. Create a secret with all the bundle certificate authorities. See my example below:

note: the key must be bundle-ca.crt

$ cat Toca_ROOT_CA.crt  Toca_Intermediate_CA.crt  > /tmp/bundle-ca.crt
$ kubectl create secret generic  awx-ssl-ca-custom  --from-file=bundle-ca.crt=/tmp/bundle-ca.crt
  1. Once the operator gets updated, modify your awx kind to map the new secret
apiVersion: awx.ansible.com/v1beta1
kind: AWX
....
spec:
  bundle_cacert_secret: awx-ssl-ca-custom
....

So before applying this patch, you should see:

image

$  openssl  s_client -connect git.tatu.home:443 
CONNECTED(00000003)
[...SNIP...]

    SRP username: None
    TLS session ticket lifetime hint: 604800 (seconds)
    TLS session ticket:
    0000 - 18 71 bb 56 c4 6d 89 64-d0 df ac 2d fa cc 45 1e   .q.V.m.d...-..E.
    0010 - c0 71 c4 ba 50 ee 91 90-da d5 fe 8e 5e d1 a1 00   .q..P.......^...
    0020 - 57 8c 77 3b 09 e9 d5 fe-25 24 d5 bf d7 fd 76 bc   W.w;....%$....v.
    0030 - 1e a5 77 1b bd 3c bb 9b-25 df 48 a5 07 91 40 3b   ..w..<..%.H...@;
    0040 - d0 28 de e7 c6 4c 3c 12-51 d8 a0 0f ae 38 7a 44   .(...L<.Q....8zD
    0050 - 65 03 9a ac a7 82 e6 6f-be 2f 68 6c 6e 4e 11 55   e......o./hlnN.U
    0060 - d9 a6 85 9a ee 81 cd 63-51 65 58 8a 38 30 61 c8   .......cQeX.80a.
    0070 - d0 91 0c 1a 96 2b 1d 6c-c4 67 2a cf a2 05 a1 a0   .....+.l.g*.....
    0080 - ad                                                .

    Start Time: 1623388039
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)   <--- oops
    Extended master secret: no
    Max Early Data: 0

After this patch, you should see:

awx-ssl-ca-6cccf6577d-jzrk9   0/4     Pending             0          0s      <none>          <none>   <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   0/4     Pending             0          0s      <none>          p70      <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   0/4     Init:0/1            0          0s      <none>          p70      <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   0/4     Init:0/1            0          1s      10.233.64.98    p70      <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   0/4     PodInitializing     0          2s      10.233.64.98    p70      <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   4/4     Running             0          4s      10.233.64.98    p70      <none>           <none>

image

Furthermore, checking the container you should see the custom ca listed as trusted

(py39) mdemello@storm ~> kubectl iexec awx /bin/bash                                                                                                                                                                                     00:53:40
Namespace: default | Pod: ✔ awx-ssl-ca-6cccf6577d-jzrk9
Container: ✔ awx-ssl-ca-task
bash-4.4$ ls -la /etc/pki/ca-trust/source/anchors/bundle-ca.crt 
-rw-r--r--. 1 root root 4086 Jun 11 04:51 /etc/pki/ca-trust/source/anchors/bundle-ca.crt
bash-4.4$ trust list | grep -i toca
    label: TOCA ROOT CA
    label: Toca Intermediate Certificate Authority
bash-4.4$ openssl  s_client -connect git.tatu.home:443 
CONNECTED(00000003)
[...SNIP...]

    SRP username: None
    TLS session ticket lifetime hint: 604800 (seconds)
    TLS session ticket:
    0000 - 18 71 bb 56 c4 6d 89 64-d0 df ac 2d fa cc 45 1e   .q.V.m.d...-..E.
    0010 - 00 b2 67 b8 66 db 91 57-f8 85 92 e3 ef 61 4e 3f   ..g.f..W.....aN?
    0020 - 66 e2 64 01 45 b8 ab 7f-f8 84 7f 5e f6 2d e2 56   f.d.E......^.-.V
    0030 - d3 2c 4b 19 cb 93 19 74-c7 0b e3 7d 76 d8 cd f7   .,K....t...}v...
    0040 - 30 5a 87 23 27 34 d7 47-8e f5 c3 6c 41 81 7d 18   0Z.#'4.G...lA.}.
    0050 - 13 96 4e e7 76 3b 50 f0-fb 8d 9d df 4a 51 9d 36   ..N.v;P.....JQ.6
    0060 - 0e a9 1a 57 26 62 51 eb-f2 ec 24 56 93 5f 01 73   ...W&bQ...$V._.s
    0070 - 67 f5 a1 a7 38 e1 dc 5e-27 65 c6 24 f5 ff 2c dc   g...8..^'e.$..,.
    0080 - eb                                                .

    Start Time: 1623387247
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)    <----- yes
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK

bash-4.4$ git clone https://git.tatu.home/mmello/test-ansible.git
Cloning into 'test-ansible'...
remote: Enumerating objects: 10, done.
remote: Counting objects: 100% (10/10), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 10 (delta 3), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (10/10), 1.81 KiB | 928.00 KiB/s, done.

Please let me know if that worked for you.

@suukit
Copy link
Contributor Author

suukit commented Jun 11, 2021

@tchellomello : thank you, i'll give it a try

@suukit
Copy link
Contributor Author

suukit commented Jun 22, 2021

Works fine here, sorry for late feedback!

@derhoeppi
Copy link

Does this solution work for WinRM CA certificates? I open an issue to awx, because i don't know if it is an operator or awx problem.
ansible/awx#10884

@AntMCE
Copy link

AntMCE commented Oct 15, 2022

@tchellomello I'm not sure I get the full concept.
I have created de bundle certificate, I added it to my awx.yaml under "spec".
of course I made sure the secret was generetaed inside the awx namespace.
However in still unable to synchronize my git project.
You said to update awx operator in the first place with a version of yours. why should we need to update awx-operator in the first place?
I installed awx on minikube following the official documentation :https://github.com/ansible/awx-operator

what am I missing here?

am I suppose to see new container after applying the freshly updated awx.yaml?

Thanks for your help

@eselvam
Copy link

eselvam commented Jan 5, 2023

I tried above step in the solution however it is not working for me.
I edited the spec and added the cabundle and secret
kubectl apply
however the operator is not picking up the change.

Kindly help me how to add cabundle for existing and running awx instance. Thanks.

@eselvam
Copy link

eselvam commented Jan 5, 2023

@suukit could you please give a try with the changes noted on this branch https://github.com/ansible/awx-operator/compare/devel...tchellomello:custom-ca?expand=1

To make it easier for you, I've published this testing image at https://quay.io/repository/tchellomello/awx-operator?tab=tags quay.io/tchellomello/awx-operator:custom-ca

So basically you can do the following steps:

1. Update your `awx-operator` using this testing POC (see https://gist.github.com/tchellomello/e38c71248591034f8a7cc28421fe2245)
$ kubectl apply -f https://gist.githubusercontent.com/tchellomello/e38c71248591034f8a7cc28421fe2245/raw/b8c1d657553d33d8ba75bb077b5960bb5abbca3c/awx-operator.yml
2. Create a secret with all the bundle certificate authorities. See my example below:

note: the key must be bundle-ca.crt

$ cat Toca_ROOT_CA.crt  Toca_Intermediate_CA.crt  > /tmp/bundle-ca.crt
$ kubectl create secret generic  awx-ssl-ca-custom  --from-file=bundle-ca.crt=/tmp/bundle-ca.crt
3. Once the operator gets updated, modify your `awx` kind to map the new secret
apiVersion: awx.ansible.com/v1beta1
kind: AWX
....
spec:
  bundle_cacert_secret: awx-ssl-ca-custom
....

So before applying this patch, you should see:

image

$  openssl  s_client -connect git.tatu.home:443 
CONNECTED(00000003)
[...SNIP...]

    SRP username: None
    TLS session ticket lifetime hint: 604800 (seconds)
    TLS session ticket:
    0000 - 18 71 bb 56 c4 6d 89 64-d0 df ac 2d fa cc 45 1e   .q.V.m.d...-..E.
    0010 - c0 71 c4 ba 50 ee 91 90-da d5 fe 8e 5e d1 a1 00   .q..P.......^...
    0020 - 57 8c 77 3b 09 e9 d5 fe-25 24 d5 bf d7 fd 76 bc   W.w;....%$....v.
    0030 - 1e a5 77 1b bd 3c bb 9b-25 df 48 a5 07 91 40 3b   ..w..<..%.H...@;
    0040 - d0 28 de e7 c6 4c 3c 12-51 d8 a0 0f ae 38 7a 44   .(...L<.Q....8zD
    0050 - 65 03 9a ac a7 82 e6 6f-be 2f 68 6c 6e 4e 11 55   e......o./hlnN.U
    0060 - d9 a6 85 9a ee 81 cd 63-51 65 58 8a 38 30 61 c8   .......cQeX.80a.
    0070 - d0 91 0c 1a 96 2b 1d 6c-c4 67 2a cf a2 05 a1 a0   .....+.l.g*.....
    0080 - ad                                                .

    Start Time: 1623388039
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)   <--- oops
    Extended master secret: no
    Max Early Data: 0

After this patch, you should see:

awx-ssl-ca-6cccf6577d-jzrk9   0/4     Pending             0          0s      <none>          <none>   <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   0/4     Pending             0          0s      <none>          p70      <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   0/4     Init:0/1            0          0s      <none>          p70      <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   0/4     Init:0/1            0          1s      10.233.64.98    p70      <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   0/4     PodInitializing     0          2s      10.233.64.98    p70      <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   4/4     Running             0          4s      10.233.64.98    p70      <none>           <none>

image

Furthermore, checking the container you should see the custom ca listed as trusted

(py39) mdemello@storm ~> kubectl iexec awx /bin/bash                                                                                                                                                                                     00:53:40
Namespace: default | Pod: ✔ awx-ssl-ca-6cccf6577d-jzrk9
Container: ✔ awx-ssl-ca-task
bash-4.4$ ls -la /etc/pki/ca-trust/source/anchors/bundle-ca.crt 
-rw-r--r--. 1 root root 4086 Jun 11 04:51 /etc/pki/ca-trust/source/anchors/bundle-ca.crt
bash-4.4$ trust list | grep -i toca
    label: TOCA ROOT CA
    label: Toca Intermediate Certificate Authority
bash-4.4$ openssl  s_client -connect git.tatu.home:443 
CONNECTED(00000003)
[...SNIP...]

    SRP username: None
    TLS session ticket lifetime hint: 604800 (seconds)
    TLS session ticket:
    0000 - 18 71 bb 56 c4 6d 89 64-d0 df ac 2d fa cc 45 1e   .q.V.m.d...-..E.
    0010 - 00 b2 67 b8 66 db 91 57-f8 85 92 e3 ef 61 4e 3f   ..g.f..W.....aN?
    0020 - 66 e2 64 01 45 b8 ab 7f-f8 84 7f 5e f6 2d e2 56   f.d.E......^.-.V
    0030 - d3 2c 4b 19 cb 93 19 74-c7 0b e3 7d 76 d8 cd f7   .,K....t...}v...
    0040 - 30 5a 87 23 27 34 d7 47-8e f5 c3 6c 41 81 7d 18   0Z.#'4.G...lA.}.
    0050 - 13 96 4e e7 76 3b 50 f0-fb 8d 9d df 4a 51 9d 36   ..N.v;P.....JQ.6
    0060 - 0e a9 1a 57 26 62 51 eb-f2 ec 24 56 93 5f 01 73   ...W&bQ...$V._.s
    0070 - 67 f5 a1 a7 38 e1 dc 5e-27 65 c6 24 f5 ff 2c dc   g...8..^'e.$..,.
    0080 - eb                                                .

    Start Time: 1623387247
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)    <----- yes
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK

bash-4.4$ git clone https://git.tatu.home/mmello/test-ansible.git
Cloning into 'test-ansible'...
remote: Enumerating objects: 10, done.
remote: Counting objects: 100% (10/10), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 10 (delta 3), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (10/10), 1.81 KiB | 928.00 KiB/s, done.

Please let me know if that worked for you.

I followed your instruction however after kubectl apply awx.yaml, I did not see the change and the cert is not populated in containers. Am I missing any thing here. Kindly help. Thanks.

@eselvam
Copy link

eselvam commented Apr 4, 2023

Could some one guide me here.

@Klaas-
Copy link

Klaas- commented Apr 5, 2023

Could some one guide me here.

https://github.com/ansible/awx-operator#trusting-a-custom-certificate-authority

add the secret

....

secretGenerator:
  - name: <resourcename>-custom-certs
    files:
      - bundle-ca.crt=<path+filename>
    options:
      disableNameSuffixHash: true
      
...

add the spec change:

---
spec:
  ...
  bundle_cacert_secret: <resourcename>-custom-certs

delete your awx instance, let the operator recreate it with the updated values

@eselvam
Copy link

eselvam commented Apr 5, 2023 via email

@yuanyuefeng
Copy link

'delete your awx instance' . Would u pls give exactly steps for this operation ?

@eselvam
Copy link

eselvam commented May 17, 2023

I tried it but no luck.

I created the resource file named awx.yaml with below content:

spec:
...
bundle_cacert_secret: awx-custom-certs =>awx is the resource name in my kubectl get awx/awx. hence used that name to prefix.

When I checked the operator log, it is running the specific Ansible task without any error, however Operator is not triggering the pod deployment for awx with changes.

Kindly advice.

@kaziislam
Copy link

@suukit could you please give a try with the changes noted on this branch https://github.com/ansible/awx-operator/compare/devel...tchellomello:custom-ca?expand=1

To make it easier for you, I've published this testing image at https://quay.io/repository/tchellomello/awx-operator?tab=tags quay.io/tchellomello/awx-operator:custom-ca

So basically you can do the following steps:

  1. Update your awx-operator using this testing POC (see https://gist.github.com/tchellomello/e38c71248591034f8a7cc28421fe2245)
$ kubectl apply -f https://gist.githubusercontent.com/tchellomello/e38c71248591034f8a7cc28421fe2245/raw/b8c1d657553d33d8ba75bb077b5960bb5abbca3c/awx-operator.yml
  1. Create a secret with all the bundle certificate authorities. See my example below:

note: the key must be bundle-ca.crt

$ cat Toca_ROOT_CA.crt  Toca_Intermediate_CA.crt  > /tmp/bundle-ca.crt
$ kubectl create secret generic  awx-ssl-ca-custom  --from-file=bundle-ca.crt=/tmp/bundle-ca.crt
  1. Once the operator gets updated, modify your awx kind to map the new secret
apiVersion: awx.ansible.com/v1beta1
kind: AWX
....
spec:
  bundle_cacert_secret: awx-ssl-ca-custom
....

So before applying this patch, you should see:

image

$  openssl  s_client -connect git.tatu.home:443 
CONNECTED(00000003)
[...SNIP...]

    SRP username: None
    TLS session ticket lifetime hint: 604800 (seconds)
    TLS session ticket:
    0000 - 18 71 bb 56 c4 6d 89 64-d0 df ac 2d fa cc 45 1e   .q.V.m.d...-..E.
    0010 - c0 71 c4 ba 50 ee 91 90-da d5 fe 8e 5e d1 a1 00   .q..P.......^...
    0020 - 57 8c 77 3b 09 e9 d5 fe-25 24 d5 bf d7 fd 76 bc   W.w;....%$....v.
    0030 - 1e a5 77 1b bd 3c bb 9b-25 df 48 a5 07 91 40 3b   ..w..<..%.H...@;
    0040 - d0 28 de e7 c6 4c 3c 12-51 d8 a0 0f ae 38 7a 44   .(...L<.Q....8zD
    0050 - 65 03 9a ac a7 82 e6 6f-be 2f 68 6c 6e 4e 11 55   e......o./hlnN.U
    0060 - d9 a6 85 9a ee 81 cd 63-51 65 58 8a 38 30 61 c8   .......cQeX.80a.
    0070 - d0 91 0c 1a 96 2b 1d 6c-c4 67 2a cf a2 05 a1 a0   .....+.l.g*.....
    0080 - ad                                                .

    Start Time: 1623388039
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)   <--- oops
    Extended master secret: no
    Max Early Data: 0

After this patch, you should see:

awx-ssl-ca-6cccf6577d-jzrk9   0/4     Pending             0          0s      <none>          <none>   <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   0/4     Pending             0          0s      <none>          p70      <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   0/4     Init:0/1            0          0s      <none>          p70      <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   0/4     Init:0/1            0          1s      10.233.64.98    p70      <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   0/4     PodInitializing     0          2s      10.233.64.98    p70      <none>           <none>
awx-ssl-ca-6cccf6577d-jzrk9   4/4     Running             0          4s      10.233.64.98    p70      <none>           <none>

image

Furthermore, checking the container you should see the custom ca listed as trusted

(py39) mdemello@storm ~> kubectl iexec awx /bin/bash                                                                                                                                                                                     00:53:40
Namespace: default | Pod: ✔ awx-ssl-ca-6cccf6577d-jzrk9
Container: ✔ awx-ssl-ca-task
bash-4.4$ ls -la /etc/pki/ca-trust/source/anchors/bundle-ca.crt 
-rw-r--r--. 1 root root 4086 Jun 11 04:51 /etc/pki/ca-trust/source/anchors/bundle-ca.crt
bash-4.4$ trust list | grep -i toca
    label: TOCA ROOT CA
    label: Toca Intermediate Certificate Authority
bash-4.4$ openssl  s_client -connect git.tatu.home:443 
CONNECTED(00000003)
[...SNIP...]

    SRP username: None
    TLS session ticket lifetime hint: 604800 (seconds)
    TLS session ticket:
    0000 - 18 71 bb 56 c4 6d 89 64-d0 df ac 2d fa cc 45 1e   .q.V.m.d...-..E.
    0010 - 00 b2 67 b8 66 db 91 57-f8 85 92 e3 ef 61 4e 3f   ..g.f..W.....aN?
    0020 - 66 e2 64 01 45 b8 ab 7f-f8 84 7f 5e f6 2d e2 56   f.d.E......^.-.V
    0030 - d3 2c 4b 19 cb 93 19 74-c7 0b e3 7d 76 d8 cd f7   .,K....t...}v...
    0040 - 30 5a 87 23 27 34 d7 47-8e f5 c3 6c 41 81 7d 18   0Z.#'4.G...lA.}.
    0050 - 13 96 4e e7 76 3b 50 f0-fb 8d 9d df 4a 51 9d 36   ..N.v;P.....JQ.6
    0060 - 0e a9 1a 57 26 62 51 eb-f2 ec 24 56 93 5f 01 73   ...W&bQ...$V._.s
    0070 - 67 f5 a1 a7 38 e1 dc 5e-27 65 c6 24 f5 ff 2c dc   g...8..^'e.$..,.
    0080 - eb                                                .

    Start Time: 1623387247
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)    <----- yes
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK

bash-4.4$ git clone https://git.tatu.home/mmello/test-ansible.git
Cloning into 'test-ansible'...
remote: Enumerating objects: 10, done.
remote: Counting objects: 100% (10/10), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 10 (delta 3), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (10/10), 1.81 KiB | 928.00 KiB/s, done.

Please let me know if that worked for you.

I am running into this issue. My setup is AWX K3S on Ubuntu server. This setup works fine but when I move this server to secure network environment I get "SSL: Certificate error" while I create a project and point it to get the project as ZIP file.

First I tried to follow this document https://github.com/kurokobo/awx-on-k3s/blob/main/tips/trust-custom-ca.md and I tried the RootCaCert as .crt and .pem extension and still seeing same error. Any suggestion would be really appreciated. I tried copying root cert manually to awx-task pod but no good either.

@abctaylor
Copy link

I think the docs at https://ansible.readthedocs.io/projects/awx-operator/en/latest/user-guide/advanced-configuration/trusting-a-custom-certificate-authority.html are pretty thin, at best, and need much more guidance on this topic.

@abctaylor
Copy link

I hope the below helps anyone else who is new to Kubernetes. I'll be extra verbose below.

Step 1
Get a cert bundle in PEM format. Put it somewhere on one of your Kubernetes control plane servers where you run kubectl commands from. This should have your custom enterprise cert inside amongst other certs like Verisign, Thawte etc. For example on my control plane server, all is rosy in /etc/ssl/certs/ca-bundle.crt so I'm just gonna lift that.

Also, just get the enterprise root CA on its own for the LDAP part, again in PEM format. I'll call it ldap-ca.crt.

We will use these later as secrets, called awx-custom-cert-bundle and awx-custom-cert-ldap.

Step 2
Create two Kubernetes secrets storing this cert bundle and the LDAP cert.
kubectl create secret generic awx-custom-cert-bundle -n awx --from-file=bundle-ca.crt=/etc/ssl/certs/ca-bundle.crt

kubectl create secret generic awx-custom-cert-ldap -n awx --from-file=ldap-ca.crt=ldap-ca.crt

Make sure ldap-ca.crt is in the directory you're running the command from. Same for the bundle, if you're not specifying an absolute path like I did above.

Also note I am specifying the namespace with -n awx.

Step 3
Make a yaml file for the patch - I've called it cacert-patch.yaml. I'm giving full syntax not the irritating ...s you see elsewhere:

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
  namespace: awx
spec:
  ldap_cacert_secret: awx-custom-cert-bundle
  bundle_cacert_secret: awx-custom-cert-ldap

Step 4
You should be good to apply this now: kubectl apply -f cacert-patch.yaml.

Step 5
Get pods in the awx namespace and kill them:
kubectl delete pod -n awx awx-operator-controller-manager-7d849d77f8-czpx8 awx-task-64f7fbcd89-ns95z awx-web-77c6b6cf87-46z29

Step 6
Restart the sync job in the AWX console (or whatever else you were trying to do):
image

@Exerti0n
Copy link

Exerti0n commented Mar 6, 2024

I hope the below helps anyone else who is new to Kubernetes. I'll be extra verbose below.

Step 1 Get a cert bundle in PEM format. Put it somewhere on one of your Kubernetes control plane servers where you run kubectl commands from. This should have your custom enterprise cert inside amongst other certs like Verisign, Thawte etc. For example on my control plane server, all is rosy in /etc/ssl/certs/ca-bundle.crt so I'm just gonna lift that.

Also, just get the enterprise root CA on its own for the LDAP part, again in PEM format. I'll call it ldap-ca.crt.

We will use these later as secrets, called awx-custom-cert-bundle and awx-custom-cert-ldap.

Step 2 Create two Kubernetes secrets storing this cert bundle and the LDAP cert. kubectl create secret generic awx-custom-cert-bundle -n awx --from-file=bundle-ca.crt=/etc/ssl/certs/ca-bundle.crt

kubectl create secret generic awx-custom-cert-ldap -n awx --from-file=ldap-ca.crt=ldap-ca.crt

Make sure ldap-ca.crt is in the directory you're running the command from. Same for the bundle, if you're not specifying an absolute path like I did above.

Also note I am specifying the namespace with -n awx.

Step 3 Make a yaml file for the patch - I've called it cacert-patch.yaml. I'm giving full syntax not the irritating ...s you see elsewhere:

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
  namespace: awx
spec:
  ldap_cacert_secret: awx-custom-cert-bundle
  bundle_cacert_secret: awx-custom-cert-ldap

Step 4 You should be good to apply this now: kubectl apply -f cacert-patch.yaml.

Step 5 Get pods in the awx namespace and kill them: kubectl delete pod -n awx awx-operator-controller-manager-7d849d77f8-czpx8 awx-task-64f7fbcd89-ns95z awx-web-77c6b6cf87-46z29

Step 6 Restart the sync job in the AWX console (or whatever else you were trying to do): image

I just want to say a big big thank you for taking the time/effort to describe this process.
I was not sure on the exact process but you made it very easy.

Thank you again!

@shvarsha
Copy link

shvarsha commented Mar 8, 2024

hi, does AWX operator support self-signed certificates ?

@Exerti0n
Copy link

Exerti0n commented Mar 8, 2024 via email

@moskey71
Copy link

I hope the below helps anyone else who is new to Kubernetes. I'll be extra verbose below.

Step 1 Get a cert bundle in PEM format. Put it somewhere on one of your Kubernetes control plane servers where you run kubectl commands from. This should have your custom enterprise cert inside amongst other certs like Verisign, Thawte etc. For example on my control plane server, all is rosy in /etc/ssl/certs/ca-bundle.crt so I'm just gonna lift that.

Also, just get the enterprise root CA on its own for the LDAP part, again in PEM format. I'll call it ldap-ca.crt.

We will use these later as secrets, called awx-custom-cert-bundle and awx-custom-cert-ldap.

Step 2 Create two Kubernetes secrets storing this cert bundle and the LDAP cert. kubectl create secret generic awx-custom-cert-bundle -n awx --from-file=bundle-ca.crt=/etc/ssl/certs/ca-bundle.crt

kubectl create secret generic awx-custom-cert-ldap -n awx --from-file=ldap-ca.crt=ldap-ca.crt

Make sure ldap-ca.crt is in the directory you're running the command from. Same for the bundle, if you're not specifying an absolute path like I did above.

Also note I am specifying the namespace with -n awx.

Step 3 Make a yaml file for the patch - I've called it cacert-patch.yaml. I'm giving full syntax not the irritating ...s you see elsewhere:

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
  namespace: awx
spec:
  ldap_cacert_secret: awx-custom-cert-bundle
  bundle_cacert_secret: awx-custom-cert-ldap

Step 4 You should be good to apply this now: kubectl apply -f cacert-patch.yaml.

Step 5 Get pods in the awx namespace and kill them: kubectl delete pod -n awx awx-operator-controller-manager-7d849d77f8-czpx8 awx-task-64f7fbcd89-ns95z awx-web-77c6b6cf87-46z29

Step 6 Restart the sync job in the AWX console (or whatever else you were trying to do): image

Thanks for posting this solution. One quick correction... the bundle and ldap certs need swapped to look like so...

spec:
  ldap_cacert_secret: awx-custom-cert-ldap
  bundle_cacert_secret: awx-custom-cert-bundle

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.