switch to stable toolchain

[davepacheco]

This does not build yet. Known blockers:

• diesel-dtrace needs to drop use of #![feature(asm)]
• slog-dtrace needs to drop use of #![feature(asm)]
• use of async_closure (@smklein says he's removing it)

Other changes in this PR:

• We had some drift in the way our dropshot deps were declared which results in two copies in Cargo.lock. This prevents the use of cargo update -p dropshot --precise. I suspect that's why we haven't seen dependabot updates to dropshot in a couple of weeks. (note: I'm doing this in normalize dropshot dependencies #917 as well)

[smklein]

Link to async_closure removal: https://github.com/oxidecomputer/omicron/pull/904/files#diff-47887367c1e0816a1e9405c9e748d167c55abd5743fd02c55246ff0bd76e9ff9L7

[leftwo]

Omicron, does it need to compile propolis (and therefore Crucible) to build, or does it now use something different? Or I'm just remembering wrong and that was never required?

[bnaecker]

I think we need to decide on macOS support at this point. If we want to support it, this issue is blocked by the stabilization of the asm_sym feature. There is continuing progress on that, but even if it were to land tomorrow, that means at least 3 months until it's on stable.

At that point, there is still a good bit of work to get the usdt crate and all its consumers in the Omicron build graph up to date. That's not hard work, but it is not trivial. (Part of that is that I believe we should remove the feature flags from the usdt crate entirely. If consumers want to provide their own feature flags to enable/disable the probes, that's fine, but the feature flag in usdt itself was a mistake in retrospect.)

[bnaecker]

@jmpesp Omicron requires the propolis API crate, but otherwise pulls both the propolis and crucible crates pre-built from their CI artifacts.

[davepacheco]

I think we need to decide on macOS support at this point. If we want to support it, this issue is blocked by the stabilization of the asm_sym feature. There is continuing progress on that, but even if it were to land tomorrow, that means at least 3 months until it's on stable.

Personally, I'd be game to drop it. I'd like input from the folks that use it, of course!

At that point, there is still a good bit of work to get the usdt crate and all its consumers in the Omicron build graph up to date. That's not hard work, but it is not trivial. (Part of that is that I believe we should remove the feature flags from the usdt crate entirely. If consumers want to provide their own feature flags to enable/disable the probes, that's fine, but the feature flag in usdt itself was a mistake in retrospect.)

I think there's a lot of value to be on "stable":

• being on nightly makes it more costly to update because (1) you have to find a working nightly and (2) as we've learned the hard way, nightly makes no compatibility guarantees, so you can wind up having to make a bunch of source changes
• so (empirically) we stay further behind. Being out of date causes us to accumulate debt that eventually has to be paid by the poor soul who needs to update for some reason.
• staying on "nightly" when we could switch makes it easier to accidentally introduce new deps on "nightly"

@bnaecker is it easy to inventory the work you're describing? If folks are on board, maybe we can spread it around.

[bnaecker]

Personally, I'd be game to drop it. I'd like input from the folks that use it, of course!

I'm also on board :)

I think there's a lot of value to be on "stable":

Definitely! I got bit by the stabilization of asm too, and just don't want people to have to worry about it.

@bnaecker is it easy to inventory the work you're describing? If folks are on board, maybe we can spread it around.

I think the work-list would roughly as follows. I'll update this if I think of more.

• Remove the usdt feature flag "asm". Instead of feature-flags, it should only decide what code is generated inside the probe macros on the basis of the target OS. This means usdt itself would still require nightly on macOS, but if we drop it for Omicron, that's not at issue here.
• Publish a new version of usdt (0.4.0), as the above change is breaking (change to feature flags).
• For each consuming crate (currently dropshot, slog-dtrace, diesel-dtrace, propolis, and crucible): Update to the latest version of usdt. If the crate chooses, it can expose its own feature flag to conditionally enable and include the usdt probes. (Dropshot is a good example. We might still want to support building on stable on macOS, meaning you don't get probes.) If a crate chooses to do this, it would need a cfg(my_usdt_feature_flag) directive conditionally compiling the usdt probe macros.
• Bump each crate's version on crates.io, if applicable.
• Update the omicron dependency of each crate exposing usdt probes, possibly enabling those probes if the crate now exposes a flag.

I'd of course welcome help, but as long as everyone's on board with dropping macOS support, then I'd be happy to just spend a day and churn through it.

[david-crespo]

I'm fine with it, assuming it would be relatively easy to run Nexus + simulated Sled Agent in a Docker container or whatever for local dev. We do most console dev using a mock API server that runs in the browser, but we do definitely sometimes dev against real Nexus.

[bnaecker]

I talked with @pfmooney after the huddle this morning. He's planning to update usdt to automatically select the no-op probe implementation on macOS platforms, if one is using a stable toolchain. If using nightly, or on another supported platform and using stable, the appropriate implementation for the platform will be chosen. (Real probes on illumos, no-op on Linux, etc.)

In addition, there'll be a feature flag called something like strict-features. This will override the above selection of the implementation, and always use the correct one for the platform. This will cause a compiler error if using a stable toolchain on macOS, for example. Consuming crates can thus ensure that they're built with probes if possible, rather than silently building without them, if they choose.

[davepacheco]

I updated this PR to see how far we are. Right now the problem we run into is that illumos-sys-hdrs (from OPTE) uses the unstable extern_types feature.

[davepacheco]

It looks like opte has already been updated to avoid this without the "kernel" feature (which we don't seem to use in Omicron's build), but omicron is using a fairly old version of opte and related crates (git commit oxidecomputer/opte@23fdf). I'm not sure what's involved in updating it further. Maybe that's covered by #1904?

[luqmana]

It looks like opte has already been updated to avoid this without the "kernel" feature (which we don't seem to use in Omicron's build), but omicron is using a fairly old version of opte and related crates (git commit oxidecomputer/opte@23fdf). I'm not sure what's involved in updating it further. Maybe that's covered by #1904?

opte has been updated so give this another shot and it should work!

[ahl]

@mkeeter and/or @jgallagher can you confirm that false is what we want here?

[jgallagher]

That does maintain the current behavior, yes. Probably we want to extend SpComponentFirmwareSlot to add a boolean to pass here, but that can be done later.

[ahl]

Perfect; thank you

[mkeeter]

uhhhhh

false means that we will not persistently select that boot slot. It is a policy question about what the sp_component_active_slot_set should be doing, and I'm not sure of the answer.

When updating similar APIs, I've added a persist: bool to the function signature, to force callers to make a choice about it; we may want to do the same to sp_component_active_slot_set here.

[jgallagher]

When updating similar APIs, I've added a persist: bool to the function signature, to force callers to make a choice about it; we may want to do the same to sp_component_active_slot_set here.

Yep, this is what I meant by extending SpComponentFirmwareSlot, which is coming from MGS's client here.

[mkeeter]

Github strikes again – I had the page open before you commented, so didn't see it until my comment was already submitted!

[ahl]

let's get that money