ingress on k3s not full working

[eslizn]

i use k3s to deploy acorn and encounter the following problems：

• IngressCapability false Ingress not ready (test timed out after 1 minute)
• app endpoint：http://[Pending Ingress] => default:80
• ssl error：TRAEFIK DEFAULT CERT

k3s install scripts：

curl -sfL https://get.k3s.io | sh -s - \
  --disable traefik \
  --disable servicelb \
  --advertise-address [server ip] \
  --disable-cloud-controller \
  --cluster-init \
  --cluster-domain=[mydomain]

traefik install

helm repo add traefik https://helm.traefik.io/traefik
helm repo update
helm install -n kube-system traefik traefik/traefik
kubectl patch svc traefik -n kube-system -p '{"spec": {"type": "ClusterIP", "externalIPs":["server ip"]}}'

acorn info：

client:
  version:
    commit: 1970d72e4c7bed86c118c7ae6cf49edba896c42f
    tag: v0.4.1-rc3
namespace:
  publicKeys:
  - keyID: ******
server:
  apiServerImage: ghcr.io/acorn-io/acorn:v0.4.1-rc3
  config:
    acornDNS: disabled
    acornDNSEndpoint: https://alpha-dns.acrn.io/v1
    autoUpgradeInterval: 5m
    clusterDomains:
    - .******
    defaultPublishMode: defined
    ingressClassName: traefik
    internalClusterDomain: svc.cluster.local
    letsEncrypt: enabled
    letsEncryptEmail: ******
    letsEncryptTOSAgree: true
    podSecurityEnforceProfile: baseline
    setPodSecurityEnforceProfile: true
  controllerImage: ghcr.io/acorn-io/acorn:v0.4.1-rc3
  dirty: false
  gitCommit: 1970d72e4c7bed86c118c7ae6cf49edba896c42f
  letsEncryptCertificate: enabled, pending (secrets "acorn-tls" not found)
  tag: v0.4.1-rc3
  userConfig:
    acornDNS: disabled
    acornDNSEndpoint: null
    autoUpgradeInterval: null
    clusterDomains:
    - .******
    defaultPublishMode: ""
    ingressClassName: traefik
    internalClusterDomain: ""
    letsEncrypt: enabled
    letsEncryptEmail: ******
    letsEncryptTOSAgree: true
    podSecurityEnforceProfile: ""
    setPodSecurityEnforceProfile: null
  version: v0.4.1-rc3+1970d72e

[cjellick]

Can you deploy any ingress successfully on this cluster? I suspect Traefik is misconfigured. We ran into a similar problem awhile ago with Civo's treafik install being misconfigured: obot-platform/obot#494

[eslizn]

Can you deploy any ingress successfully on this cluster? I suspect Traefik is misconfigured. We ran into a similar problem awhile ago with Civo's treafik install being misconfigured: #494

Tks，but ssl still doesn't work

ssl error：TRAEFIK DEFAULT CERT

[cjellick]

I don't think that is pertinent yet. I think we need to ensure your ingress is working before we debug ths SSL error.

If you don't want to setup a non-acorn service to test ingress, you could do the following:

1. Launch our hello-world:

acorn run ghcr.io/library/acorn-io/hello-world

2. Paste the output of acorn ps here
3. Paste the output of kubectl get -A ingress -o yaml here

I suspect that the ingress object won't have load balancer information in the status field. If that's the case, it means the ingress can't "solve" the ingress and in my experience with traefik, it often points to it being misconfigured?

A different thing you could try to debug: rather than disable k3s's built-in ingress and deploy your own, just leave the built-in one enabled and see if it works.

[eslizn]

acorn run Acornfile

containers: {
 default: {
  image: "nginx"
  ports: publish: "80/http"
  files: {
   // Simple index.html file
   "/usr/share/nginx/html/index.html": "<h1>My First Acorn!</h1>"
  }
 }
}

acorn apps

NAME        IMAGE          HEALTHY   UP-TO-DATE   CREATED   ENDPOINTS                                    MESSAGE
long-surf   ebab9158abb7   1         1            26h ago   http://long-surf.*** => default:80   OK

kubectl get -A ingress -o yaml

apiVersion: v1
items:
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      acorn.io/targets: '{"long-surf.***":{"port":80,"service":"default"}}'
      apply.acorn.io/applied: ***
      apply.acorn.io/owner-gvk: internal.acorn.io/v1, Kind=AppInstance
      apply.acorn.io/owner-name: long-surf
      apply.acorn.io/owner-namespace: acorn
      apply.acorn.io/owner-sub-context: ""
    creationTimestamp: "2022-12-01T04:05:20Z"
    generation: 1
    labels:
      acorn.io/app-name: long-surf
      acorn.io/app-namespace: acorn
      acorn.io/managed: "true"
      acorn.io/service-name: default
      apply.acorn.io/hash: cf7c1cf1bde17225e183bcf6acfb16153501c491
    name: default
    namespace: long-surf-f69261d6-35c
    resourceVersion: "110211"
    uid: 481f8496-260b-4b1f-8e31-a20527c777b0
  spec:
    ingressClassName: traefik
    rules:
    - host: long-surf.***
      http:
        paths:
        - backend:
            service:
              name: default
              port:
                number: 80
          path: /
          pathType: Prefix
  status:
    loadBalancer:
      ingress:
      - hostname: ***
        ip: [server ip]
kind: List
metadata:
  resourceVersion: ""

curl http://long-surf.***

<h1>My First Acorn!</h1>

curl https://long-surf.***

curl: (60) Issuer certificate is invalid.
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

[cjellick]

Thanks for that. And the above is with your custom install of Traefik?

How are you expecting the cert to be provisioned? Have you installed cert-manager yourself or did you turn on our Let's Encrypt integration? (https://docs.acorn.io/installation/options#tls-via-lets-encrypt)

[eslizn]

i use acorn let's encrypt integration：

acorn install --acorn-dns disabled   --cluster-domain ***   --ingress-class-name traefik   --lets-encrypt enabled   --lets-encrypt-tos-agree=true   --lets-encrypt-email ***

[cjellick]

ah, ok. I see the issue... the one piece of our TLS integration that we are missing is suppoting it for custom domains specified via: acorn install --cluster-domain ....

I am hoping we get that out this month. Here is the tracking issue: obot-platform/obot#206 (comment)

I'll add this issue to the milestone and track it along side the others, so that when this feature is complete, you'll get notified on this issue.

[cjellick]

Addressed in v0.5.0. But note that for some ingress controllers (specifically observed with nginx on AKS), it wasn't working properly (see: obot-platform/obot#1135). That'll be fixed in v0.5.1, which is releasing next week.