Karpenter controller & webhook pods logs in json format

[rdpravin1895]

Is there a way to set the format of controller & webhook pods in json format? Currently its like this:

2022-04-06T12:29:38.013Z	DEBUG	controller.provisioning	Discovered EC2 instance types zonal offerings	{"commit": "f78fa16", "provisioner": "default"}

But we are looking to push the logs to elasticsearch through fluentd to keep track of the node scaling events, and this requires the logs to be in json format.

[bwagner5]

Here's a related issue: #1465

You can configure the zap logging via the config-logging config-map:

kubectl get configmap config-logging -n karpenter

In the zap-logger-config section, you can change encoding from console to json.

apiVersion: v1
 data:
   zap-logger-config: |
     {
       "level": "debug",
       "development": false,
       "disableStacktrace": true,
       "disableCaller": true,
       "sampling": {
         "initial": 100,
         "thereafter": 100
       },
       "outputPaths": ["stdout"],
       "errorOutputPaths": ["stderr"],
       "encoding": "json",
       "encoderConfig": {
         "timeKey": "time",
         "levelKey": "level",
         "nameKey": "logger",
         "callerKey": "caller",
         "messageKey": "message",
         "stacktraceKey": "stacktrace",
         "levelEncoder": "capital",
         "timeEncoder": "iso8601"
       }
     }
 kind: ConfigMap
 metadata:
   annotations:
     meta.helm.sh/release-name: karpenter
     meta.helm.sh/release-namespace: karpenter
   creationTimestamp: "2022-03-21T15:18:55Z"
   labels:
     app.kubernetes.io/instance: karpenter
     app.kubernetes.io/managed-by: Helm
     app.kubernetes.io/name: karpenter
     app.kubernetes.io/version: 0.8.1
     helm.sh/chart: karpenter-0.8.1
   name: config-logging
   namespace: karpenter

https://github.com/aws/karpenter/blob/main/charts/karpenter/templates/configmap-logging.yaml#L26

and then your logs will look like this:

karpenter-8646894756-2vssm controller {"level":"DEBUG","time":"2022-04-07T15:31:31.048Z","logger":"controller.provisioning","message":"Discovered EC2 instance types zonal offerings","commit":"60c029e","provisioner":"default"}
karpenter-8646894756-2vssm controller {"level":"INFO","time":"2022-04-07T15:31:31.063Z","logger":"controller.provisioning","message":"Waiting for unschedulable pods","commit":"60c029e","provisioner":"default"}

[rdpravin1895]

@bwagner5 Thanks for the pointer. This is great, but from the yaml file, I see that the encoding field is not configurable through custom helm values.

https://github.com/aws/karpenter/blob/main/charts/karpenter/templates/configmap-logging.yaml#L26

Are there plans to make this configurable? Or the other option that we currently have is to clone the repo and maintain a custom version of the chart with this change in the configmap.

[bwagner5]

Good suggestion, it has been merged! :D should be released next week