networking-tools: Add nmstate

[qinqon]

As discussed the nmstatectl tool is going to be added to FCOS, it will allow to declare the network configuration at boot time

[qinqon]

/cc @cybertron

[bgilbert]

I believe you'll need to manually add nmstate to the manifest-lock files.

It'd also be good to add a kola test that configures the network by writing an nmstate config from a Butane config.

[cathay4t]

@qinqon Please also include nmstate-libs in case someone want a C library instead of CLI.

[bgilbert]

If nmstate-libs isn't needed by nmstate, we probably shouldn't include the library package. We don't encourage users to install custom software in the host, and so we don't ship libraries that aren't needed by an OS component.

[qinqon]

@cathay4t @bgilbert
This would be needed if one of the os tools use nmstate programatically linking dynamically to it right ?

For static linking the nmstate-lib should be at the dev/build env but not af the target system.

[qinqon]

Looks like nmstatectl service do not support nmpolicy

Mar 02 13:59:48 qemu0 kola-runext-test.sh[1762]: [2023-03-02T13:59:48Z ERROR nmstatectl::service] Failed to apply state file /etc/nmstate/br-ex-policy.yml: serde_yaml::Error: Unsupported keys found: ["capture", "desiredState"]

nmstatectl apply is fine

[bgilbert]

This would be needed if one of the os tools use nmstate programatically linking dynamically to it right ?

In that case, presumably the RPM of the tool would have Requires: nmstate-libs, right? So we shouldn't need to explicitly add it here.

[cathay4t]

No. The nmstate does not depend on nmstate-libs.

[bgilbert]

Great. So it sounds as though we don't need nmstate-libs.

[qinqon]

Have reconstructed the PR and now there are two tests one configuring a normal state and the other using a policy, so we exercise the two main modes to configure networking with nmstate

[qinqon]

Great. So it sounds as though we don't need nmstate-libs.

[qinqon]

@bgilbert looks like nmstate.service is running before NetworkManger is ready, this is the systemd unit:

[Unit]
Description=Apply nmstate on-disk state
Documentation=man:nmstate.service(8) https://www.nmstate.io
After=NetworkManager.service
Requires=NetworkManager.service

[Service]
Type=oneshot
ExecStart=/usr/bin/nmstatectl service

[Install]
WantedBy=multi-user.target

Logs

Mar  6 07:50:05.269335 systemd[1]: Starting nmstate.service - Apply nmstate on-disk state...
Mar  6 07:50:05.277998 systemd[1]: Finished systemd-homed-activate.service - Home Area Activation.  
Mar  6 07:50:05.278000 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-homed-activate comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' 
Mar  6 07:50:05.280000 audit: BPF prog-id=67 op=LOAD                            
Mar  6 07:50:05.281000 audit: BPF prog-id=68 op=LOAD                            
Mar  6 07:50:05.281000 audit: BPF prog-id=69 op=LOAD                            
Mar  6 07:50:05.290390 systemd[1]: Starting systemd-hostnamed.service - Hostname Service...
Mar  6 07:50:05.294788 systemd[1]: Started chronyd.service - NTP client/server. 
Mar  6 07:50:05.298000 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=chronyd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Mar  6 07:50:05.308385 nmstatectl[1263]: [2023-03-06T07:50:05Z WARN  nmstate::nm::nm_dbus::error] Unknown DBUS error MethodError("org.freedesktop.DBus.Error.UnknownMethod", Some("Object does not exist at path “/org/freedesktop/NetworkManager”"), Msg { type: Error, sender: ":1.3", reply-serial: 2, body: Signature: [
Mar  6 07:50:05.308385 nmstatectl[1263]:             s (115),                    
Mar  6 07:50:05.308385 nmstatectl[1263]:     ] })                                
Mar  6 07:50:05.310309 nmstatectl[1263]: [2023-03-06T07:50:05Z ERROR nmstatectl::service] Failed to apply state file /etc/nmstate/br-ex.yml: NmstateError: Bug: DbusConnectionError: org.freedesktop.DBus.Error.UnknownMethod: Object does not exist at path “/org/freedesktop/NetworkManager”
Mar  6 07:50:05.310867 systemd[1]: nmstate.service: Deactivated successfully.    

[cathay4t]

Above service dependency issue will be fixed by nmstate/nmstate#2265

[bgilbert]

I commented in the PR, but in short, I don't think that PR is the right fix.

[cathay4t]

@bgilbert I have update the PR with new approach (retry on daemon not start failure) as NM team confirmed this is a bug of NM.

[qinqon]

tests are failing.

also can we get someone to answer the questions and try to get the fedora-release PR merged?

looks like qemu is running as ipv6 single stack at CI

[dustymabe]

looks like qemu is running as ipv6 single stack at CI

I'd be surprised if that was the case.

[dustymabe]

looks like qemu is running as ipv6 single stack at CI

I'd be surprised if that was the case.

Maybe try adding https://github.com/coreos/fedora-coreos-config/pull/2269/files#r1143551266 to both your tests.

[bgilbert]

@qinqon I'm confused by nmstate/nmstate#2283. Does that mean nmstate.service is failing in a default install today? CI in this PR should have failed if that's the case.

[bgilbert]

I've confirmed that nmstate.service is not failing if /etc/nmstate doesn't exist, so I'm not sure what nmstate/nmstate#2283 is doing.

[cathay4t]

I tried with old nmstatectl. The systemd start nmstate.service will fail on missing /etc/nmstate folder with error:

Mar 22 14:27:19 c9s nmstatectl[4929]: std::io::Error: No such file or directory (os error 2)

Anyway, PR nmstate/nmstate#2283 should fixed in next fedora update.

[cathay4t]

The detail rpm based install will have /etc/nmstate with README file in it, hence no failure on default install.

[travier]

Shouldn't this PR close coreos/fedora-coreos-tracker#1175?

[bgilbert]

No, the remaining steps are in coreos/fedora-coreos-tracker#1175 (comment).