salt,tests: Allow to resolve registry endpoint from containers

In some cases, we may want to get some "stuff" from the internal registry
from inside a container (for example an operator that try to retrieve
some information from the registry).

Since we use a "fake" endpoint in containerd config file to reach the
registries, this commit make this "fake" endpoint also available when
resolving name with CoreDNS as we `rewrite` the "fake" endpoint name to
resolve to a non-headless service that points to the registry.

NOTE: We need to use a non-headless service as we want to forward the
port so that we have the same behavior as the fake endpoint configured
in containerd

CHANGELOG.md

@@ -13,6 +13,9 @@
 - Bump CoreDNS version to [v1.8.6](https://github.com/coredns/coredns/releases/tag/v1.8.6)
   (PR[#3634](https://github.com/scality/metalk8s/pull/3634))
 
+- Allow to resolve the registry endpoint from inside containers using CoreDNS
+  (PR[#3690](https://github.com/scality/metalk8s/pull/3690))
+
 ## Release 2.11.1 (in development)
 ### Enhancements
 

salt/metalk8s/kubernetes/coredns/deployed.sls

@@ -1,5 +1,6 @@
 {%- from "metalk8s/map.jinja" import coredns with context %}
 {%- from "metalk8s/map.jinja" import metalk8s with context %}
+{%- from "metalk8s/map.jinja" import repo with context %}
 
 {%- set cluster_dns_ip = salt.metalk8s_network.get_cluster_dns_ip() %}
 
@@ -28,6 +29,7 @@ Create coredns ConfigMap:
                   lameduck 5s
                 }
                 ready
+                rewrite name {{ repo.registry_endpoint }} repositories-internal.kube-system.svc.{{ coredns.cluster_domain }}
                 kubernetes {{ coredns.cluster_domain }} {{ coredns.reverse_cidrs }} {
                   pods insecure
                   fallthrough in-addr.arpa ip6.arpa

salt/metalk8s/repo/deployed.sls

@@ -24,3 +24,23 @@ Deploy repo service object:
           selector:
             app.kubernetes.io/name: repositories
           type: ClusterIP
+
+Deploy repo internal service object:
+  metalk8s_kubernetes.object_present:
+    - manifest:
+        apiVersion: v1
+        kind: Service
+        metadata:
+          name: repositories-internal
+          namespace: kube-system
+          labels:
+            app.kubernetes.io/name: repositories
+        spec:
+          ports:
+          - name: http
+            port: 80
+            protocol: TCP
+            targetPort: http
+          selector:
+            app.kubernetes.io/name: repositories
+          type: ClusterIP

tests/post/features/registry.feature

@@ -6,6 +6,11 @@ Feature: Registry is up and running
     When we pull metalk8s utils image from node 'bootstrap'
     Then pull succeeds
 
+  Scenario: We can reach registry from a container
+    Given the Kubernetes API is available
+    And pods with label 'app.kubernetes.io/name=repositories' are 'Ready'
+    Then we can reach registry from inside a container
+
   @registry_ha
   Scenario: Pull container image from registry (HA)
     Given the Kubernetes API is available

tests/post/steps/test_registry.py

@@ -41,6 +41,11 @@ def test_pull_registry(host):
     pass
 
 
+@scenario("../features/registry.feature", "We can reach registry from a container")
+def test_get_registry_from_container(host):
+    pass
+
+
 @scenario("../features/registry.feature", "Pull container image from registry (HA)")
 def test_pull_registry_ha(host, teardown):
     pass
@@ -158,4 +163,16 @@ def pull_fails(context):
     )
 
 
+@then("we can reach registry from inside a container")
+def reach_registry(utils_pod, host, registry_address):
+    output = utils.kubectl_exec(
+        host=host, pod=utils_pod, command=["curl", "--fail", registry_address]
+    )
+
+    assert output.exit_status == 0, (
+        f"Unable to reach '{registry_address}' from container:"
+        f"\nout: {output.stdout}\nerr: {output.stderr}"
+    )
+
+
 # }}}