Updating templates to use etcd3

[skloeckner]

Getting these errors when just attempting to change the templates to just "etcd":

May 11 23:00:40 k8s-master.local coreos-cloudinit[926]: 2017/05/11 23:00:40 Wrote file to "/home/core/cloud-config.yml"
May 11 23:00:40 k8s-master.local coreos-cloudinit[926]: 2017/05/11 23:00:40 Wrote file /home/core/cloud-config.yml to filesystem
May 11 23:00:40 k8s-master.local coreos-cloudinit[926]: 2017/05/11 23:00:40 Writing unit "coreos-bootstrap.service" to filesystem
May 11 23:00:40 k8s-master.local coreos-cloudinit[926]: 2017/05/11 23:00:40 Writing file to "/etc/systemd/system/coreos-bootstrap.service"
May 11 23:00:40 k8s-master.local coreos-cloudinit[926]: 2017/05/11 23:00:40 Wrote file to "/etc/systemd/system/coreos-bootstrap.service"
May 11 23:00:40 k8s-master.local coreos-cloudinit[926]: 2017/05/11 23:00:40 Wrote unit "coreos-bootstrap.service"
May 11 23:00:40 k8s-master.local coreos-cloudinit[926]: 2017/05/11 23:00:40 Ensuring runtime unit file "etcd.service" is unmasked
May 11 23:00:40 k8s-master.local coreos-cloudinit[926]: 2017/05/11 23:00:40 Ensuring runtime unit file "etcd2.service" is unmasked
May 11 23:00:40 k8s-master.local coreos-cloudinit[926]: 2017/05/11 23:00:40 Ensuring runtime unit file "fleet.service" is unmasked
May 11 23:00:40 k8s-master.local coreos-cloudinit[926]: 2017/05/11 23:00:40 Ensuring runtime unit file "locksmithd.service" is unmasked
May 11 23:00:40 k8s-master.local systemd[1]: Reloading.
May 11 23:00:40 k8s-master.local systemd[1]: Starting Garbage Collection for rkt...
May 11 23:00:40 k8s-master.local coreos-cloudinit[926]: 2017/05/11 23:00:40 Calling unit command "start" on "coreos-bootstrap.service"
May 11 23:00:40 k8s-master.local systemd[1]: Starting Install coreos to disk...
May 11 23:00:40 k8s-master.local systemd[1]: Started Garbage Collection for rkt.
May 11 23:00:40 k8s-master.local coreos-install[1090]: 2017/05/11 23:00:40 Checking availability of "local-file"
May 11 23:00:40 k8s-master.local coreos-install[1090]: 2017/05/11 23:00:40 Fetching user-data from datasource of type "local-file"
May 11 23:00:40 k8s-master.local coreos-install[1090]: 2017/05/11 23:00:40 line 17: warning: deprecated key "listen_client_urls" (etcd2 options no longer work for etcd)
May 11 23:00:40 k8s-master.local coreos-install[1090]: 2017/05/11 23:00:40 line 18: warning: deprecated key "advertise_client_urls" (etcd2 options no longer work for etcd)
May 11 23:00:40 k8s-master.local coreos-install[1090]: 2017/05/11 23:00:40 line 19: warning: deprecated key "initial_cluster" (etcd2 options no longer work for etcd)
May 11 23:00:40 k8s-master.local coreos-install[1090]: 2017/05/11 23:00:40 line 20: warning: deprecated key "proxy" (etcd2 options no longer work for etcd)
May 11 23:00:40 k8s-master.local coreos-install[1090]: /usr/bin/coreos-install: Cloud config file (/home/core/cloud-config.yml) is not valid.
May 11 23:00:40 k8s-master.local systemd[1]: coreos-bootstrap.service: Main process exited, code=exited, status=1/FAILURE
May 11 23:00:40 k8s-master.local systemd[1]: Failed to start Install coreos to disk.
May 11 23:00:40 k8s-master.local systemd[1]: coreos-bootstrap.service: Unit entered failed state.
May 11 23:00:40 k8s-master.local coreos-cloudinit[926]: 2017/05/11 23:00:40 Result of "start" on "coreos-bootstrap.service": failed

And here is when I changed it to "etcd3"

May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Fetching meta-data from datasource of type "proc-cmdline"
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Parsing user-data as cloud-config
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Merging cloud-config from meta-data and user-data
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Authorized SSH keys for core user
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Writing file to "/home/core/cloud-config.yml"
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Wrote file to "/home/core/cloud-config.yml"
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Wrote file /home/core/cloud-config.yml to filesystem
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Writing unit "coreos-bootstrap.service" to filesystem
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Writing file to "/etc/systemd/system/coreos-bootstrap.service"
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Wrote file to "/etc/systemd/system/coreos-bootstrap.service"
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Wrote unit "coreos-bootstrap.service"
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Ensuring runtime unit file "etcd.service" is unmasked
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Ensuring runtime unit file "etcd2.service" is unmasked
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Ensuring runtime unit file "fleet.service" is unmasked
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Ensuring runtime unit file "locksmithd.service" is unmasked
May 11 23:23:25 k8s-master.local systemd[1]: Reloading.
May 11 23:23:25 k8s-master.local systemd[1]: Starting Garbage Collection for rkt...
May 11 23:23:25 k8s-master.local systemd-udevd[687]: Invalid rule /usr/lib64/udev/rules.d/73-seat-late.rules:15: RUN{builtin}: 'uaccess' unknown
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Calling unit command "start" on "coreos-bootstrap.service"
May 11 23:23:25 k8s-master.local systemd[1]: Starting Install coreos to disk...
May 11 23:23:25 k8s-master.local systemd[1]: Started Garbage Collection for rkt.
May 11 23:23:25 k8s-master.local coreos-install[1079]: 2017/05/11 23:23:25 Checking availability of "local-file"
May 11 23:23:25 k8s-master.local coreos-install[1079]: 2017/05/11 23:23:25 Fetching user-data from datasource of type "local-file"
May 11 23:23:25 k8s-master.local coreos-install[1079]: 2017/05/11 23:23:25 line 16: warning: unrecognized key "etcd3"
May 11 23:23:25 k8s-master.local coreos-install[1079]: /usr/bin/coreos-install: Cloud config file (/home/core/cloud-config.yml) is not valid.
May 11 23:23:25 k8s-master.local systemd[1]: coreos-bootstrap.service: Main process exited, code=exited, status=1/FAILURE
May 11 23:23:25 k8s-master.local systemd[1]: Failed to start Install coreos to disk.
May 11 23:23:25 k8s-master.local systemd[1]: coreos-bootstrap.service: Unit entered failed state.
May 11 23:23:25 k8s-master.local coreos-cloudinit[952]: 2017/05/11 23:23:25 Result of "start" on "coreos-bootstrap.service": failed

[skloeckner]

I don't think etcd3 is what the service shows up as. I believe etcd2 is maintained just for backwards compatibility for the moment as the errors in the first seem to suggest that options used in the template are no longer supported.

I am using CoreOS 1353.7.0.

[skloeckner]

So I guess cloud-configis just deprecated now. I wonder why they even continue to keep it in Coreos if that's the case. So much for being lightweight :/

I'm going to look at ignition. May just need to rewrite templates for Foreman in that case.

Either way, I learned stuff along the way.

[johscheuer]

Did you try to validate your rendered config with: https://coreos.com/validate ? Maybe just a missing space or something?

It looks like they named the etcd3 service etcd again... https://coreos.com/etcd/docs/latest/getting-started-with-etcd.html probalby not the best idea but okay :D

[skloeckner]

Line: 17deprecated key "listen_client_urls" (etcd2 options no longer work for etcd)
Line: 18deprecated key "advertise_client_urls" (etcd2 options no longer work for etcd)
Line: 19deprecated key "initial_cluster" (etcd2 options no longer work for etcd)
Line: 20deprecated key "proxy" (etcd2 options no longer work for etcd)

Same errors I saw in the journal. I imagine since the github repo for cloudconfig says its deprecated that there may not be an alternative for those options. I'll re-open this for now until I can confirm.

[johscheuer]

I investigated some time to understand the changes they made. From my perspective there are the following changes:

• The etcd/etcd2 section in cloud-config is deprecated
• There is now a two step approach: https://coreos.com/os/docs/latest/overview-of-ct.html At first create a container linux config then use the ct tool to generate an ignition file (ctwill do an simple syntax and sanity check). The generated ignition file can be used to bootstrap a new Container Linux instance at boot time.
• etcd will be per default started as rkt container (the etcd-wrapper will automatically generate the systemd file for you)
• Here are the docs for running etcd: https://coreos.com/etcd/docs/latest/getting-started-with-etcd.html

So from my perspective we should move from cloud-config to ignition (since this is the supported and preferred).

[skloeckner]

So, theoretically we should be able to use this ct tool to convert the existing cloudconfigs to Ignition? Then convert that to a template in Foreman?

I searched but I can't seem to find anymore reference to the CT tool itself.

[mimmus]

No, ct is not a tool to convert from cloudconfig to Ignition but a tool to convert from an "intermediate", human-friendly YAML format to a machine-readable JSON config file.

https://coreos.com/os/docs/latest/overview-of-ct.html
https://github.com/coreos/container-linux-config-transpiler/