Confirm that it's safe to reboot with transient VNICs

[smklein]

Original title: Use non-transient VNICs.

As we move to using self-assembling zones #1898 and also work to make it possible to re-initialize services after reboot #725 , we should confirm: Can the system re-initialize itself if all VNICs are destroyed?

I think the answer is: "Yes", but this means that a reboot -- either of the sled agent, or a true system reboot -- will fail all running instances.

We may want to be more cautious with treating VNICs as transient and deletable on reboot, but this requires more investigation / testing.

[morlandi7]

closing per control plane meeting 8/15/2023