Checksum generation from templated ConfigMap

[deblaci]

Hello,

I have ytt template what generate one microservice k8s objects in a list. It can happen, one microservice has one or more configmap. If it has configmap, configmap will be added with overlay feature, usually these configmaps contains templates.
How can I generate one ultimate ( or one by configmap) checksum annotation from the templated configmaps in pod template of Deployment?

Templated output:
`

apiVersion: v1
items:

• kind: ServiceAccount
  ...
• kind: Deployment
  ...
• kind: ConfigMap
  name: nameone
  ...
• kind: ConfigMap
  name: nametwo
  ...
  `

My idea was to create function what collect all the templated configmap content into a variable then generate a checksum. But I couldn't figure out how I can get the templated content, only the raw.

functions.star: def getConfigMapsContent(data): configMapFilesContent = "" for fileName in [ fileName for fileName in data.list() if fileName.endswith(".yaml") ]: fileContent = data.read(fileName) if "kind: ConfigMap" in fileContent: configMapFilesContent += fileContent end end return configMapFilesContent end

templte.yaml: #@ load("@ytt:data", "data") #@ load("@ytt:sha256", "sha256") #@ load("functions.star", "fc") ... template: metadata: annotations: configMapChecksum: #@ sha256.sum(fc.getConfigMapsContent(data))

Could you please help me how I can fix it? Or suggest new "design" to handle configmap templating and checksum generation?

[prembhaskal]

will check today.

[prembhaskal]

@deblaci
so based on my understanding i tried something here https://carvel.dev/ytt/#gist:https://gist.github.com/prembhaskal/b20e0aae9051e380f236b053842231aa
and it is generating the sha256 of configmap.
I have put the raw data too in output to illustrate that it is reading the right data, but you can ofcourse remove it.

Check if this helps.

[deblaci]

@prembhaskal
Thanks for your help. Your example working well if there is no template in configmaps. In my case there are template and text template in configmap. Here you can see an example configmap.
I created an example repo: https://github.com/deblaci/ytt-configmap-checksum

I would like to generate the checksum for the templated configmaps, so if data value change in dev.yaml(example: "color: Yellow") I would like to get changed checksum.

One more thing what is not releated to this issue, I had to give explicitly the files and they order to ytt to run without error: ytt -f template/config.yaml -f template/functions.star -f template/schema.yaml -f dev.yaml -f nginx/values.yaml -f nginx/configmaps/

But I think ytt -f template -f dev.yaml -f nginx -f nginx/configmaps/ should be work, but got

ytt: Error: 
- cannot load functions.star: Expected to find file 'functions.star' (hint: only files included via -f flag are available)
    in <toplevel>
      configmaps/game-config.yaml:3 | #@ load("functions.star", "f")

[prembhaskal]

@deblaci
ok so i tried some experiments with ytt, specifically to load the results of template in a variable using overlay but it does not look so straightforward. One thing which you can try is do ytt twice.
1st ytt:
do the template and generate each document in a separate file using --output-file argument of ytt. like below command.

% ytt -f functions.star -f values.yaml -f dev.yaml  -f app.yaml -f cm1.yaml -f cm2.yaml -f schema.yaml --output-files output/
creating: output/app.yaml
creating: output/cm1.yaml
creating: output/cm2.yaml

2nd ytt:
then run an overlay to match and update the key with SHA of the config map files.

Please try this and see if it helps. if you are unable to do it, I can come up with an example by tomorrow.

[deblaci]

@prembhaskal
Thanks, I thought the above workaround solution ;) , but I wanted to template it in a single command.
So I found a solution to do it and I updated my repo, please check it.
I created a function what returns with the templated data of configMap. After this I could create an overlay to insert a configmap to the "items" list and also create an overlay to insert a checksum annotation to the pod template.

This is a little bit dirty I had to split the configmap structure, I don't know how I can do this more "clean".

[prembhaskal]

I think it is much better than templating into 2 steps. Based on my understanding of the tool, I don't think it can get more cleaner than this.

[deblaci]

Okey, thanks your help!