[smf] Allow Crucible Zones to be self-assembling #498
Conversation
|
oxidecomputer/omicron#1902 shows what the updates look like on the sled agent side |
|
Does this change also need oxidecomputer/omicron#1902 to go in before it will work? |
Yeah, the two need to land together to work. If both are approved, I would merge this, then update the crucible rev while merging 1902 in omicron. |
There was a problem hiding this comment.
Broadly looks good! Thanks for doing this. A few nits and small changes, but otherwise I think pretty much ready to go! I looked at this in tandem with the other PR: oxidecomputer/omicron#1902
| ipadm show-addr "$DATALINK/ll" || ipadm create-addr -t -T addrconf "$DATALINK/ll" | ||
| ipadm show-addr "$DATALINK/omicron6" || ipadm create-addr -t -T static -a "$LISTEN_ADDR" "$DATALINK/omicron6" | ||
| route get -inet6 default -inet6 "$GATEWAY" || route add -inet6 default -inet6 "$GATEWAY" |
There was a problem hiding this comment.
We should really move this network initialisation into a separate service that's uniform in all of the zones, I think. If we're happy with the shape of this (i.e., this sequence of commands with this set of input variables) we could even include it in the brand as effectively a stable interface. You'd be able to just turn it on and specify the datalink/address/gateway in the site profile and that'd be it. We could make svc:/milestone/network depend on it, and all our services could just depend on that.
We can also do that later, of course, if you want to get moving now.
There was a problem hiding this comment.
I'm totally on-board with doing this , and agree that it would be better served as a service. Is https://github.com/oxidecomputer/helios-omicron-brand a better spot for this? Where should it live in there?
(I do plan on moving forward with this implemented manually, but I'd be happy to switch over to another service if I could get some guidance on setting up the service within the brand)
| DATALINK="$(svcprop -c -p config/datalink "${SMF_FMRI}")" | ||
| GATEWAY="$(svcprop -c -p config/gateway "${SMF_FMRI}")" |
There was a problem hiding this comment.
I think we should probably explicitly check that the DATALINK and GATEWAY values here are not set to the unknown default value?
After L3-4, but before L6, we could insert:
. /lib/svc/share/smf_include.shThen, if we detect missing configuration, we can exit $SMF_EXIT_ERR_CONFIG to have SMF flag a permanent configuration error; e.g.,
if [[ $DATALINK == unknown ]] || [[ $GATEWAY == unknown ]]; then
printf 'ERROR: missing datalink or gateway\n' >&2
exit "$SMF_EXIT_ERR_CONFIG"
fiThere was a problem hiding this comment.
Sure, updated.
Couple things:
- Is there a standard for checking "empty values" of properties in illumos? I noticed that I could either...
- (1) totally delete the property, which obfuscates the value by removing it from the manifest.xml file
- (2) set it to
'', but then it appears to be set in the shell to\"\", rather than an actual empty string, which is kinda odd. - (3) use this quirky naming scheme, at least for
astringvalues, of setting it to some reserved sentinel value (likedefault).
I ask because if we generalize some of this stuff to an extra service, I don't think all these values actually need to be set, so some could be only supplied optionally. For example, I think the underlay address we're supplying might not always be available when the zone starts (such as within the switch zone).
…1902) Part of #1898 Relies on oxidecomputer/crucible#498 Converts Crucible, Cockroach, and Clickhouse to be (mostly) self-assembling. I'm happy to proceed and convert the rest of the zones we're launching using a similar format, if we like how this looks. Fixes #2886
Make the "crucible agent" self-assembling.
This intends to replace the following parts of the sled agent in Omicron:
Part of oxidecomputer/omicron#1898