[Merged by Bors] - Migrate engine to Schedule v3

[alice-i-cecile]

Huge thanks to @maniwani, @devil-ira, @hymm, @cart, @superdump and @jakobhellermann for the help with this PR.

Objective

• Followup [Merged by Bors] - Add bevy_ecs::schedule_v3 module #6587.
• Minimal integration for the Stageless Scheduling RFC: Stageless: a complete scheduling overhaul rfcs#45

Solution

• Remove old scheduling module
• Migrate new methods to no longer use extension methods
• Fix compiler errors
• Fix benchmarks
• Fix examples
• Fix docs
• Fix tests

Changelog

Added

• a large number of methods on App to work with schedules ergonomically
• the CoreSchedule enum
• the private prepare_view_uniforms system now has a public system set for scheduling purposes, called ViewSet::PrepareUniforms

Removed

• stages, and all code that mentions stages
• states have been dramatically simplified, and no longer use a stack
• RunCriteriaLabel
• AsSystemLabel trait
• on_hierarchy_reports_enabled run criteria (now just uses an ad hoc resource checking run condition)
• systems in RenderSet/Stage::Extract no longer warn when they do not read data from the main world
• RunCriteriaLabel
• transform_propagate_system_set: this was a nonstandard pattern that didn't actually provide enough control. The systems are already pub: the docs have been updated to ensure that the third-party usage is clear.

Changed

• System::default_labels is now System::default_system_sets.
• App::add_default_labels is now App::add_default_sets
• CoreStage and StartupStage enums are now CoreSet and StartupSet
• App::add_system_set was renamed to App::add_systems
• The StartupSchedule label is now defined as part of the CoreSchedules enum
• .label(SystemLabel) is now referred to as .in_set(SystemSet)
• SystemLabel trait was replaced by SystemSet
• SystemTypeIdLabel<T> was replaced by SystemSetType<T>
• The ReportHierarchyIssue resource now has a public constructor (new), and implements PartialEq
• Fixed time steps now use a schedule (CoreSchedule::FixedTimeStep) rather than a run criteria.
• Adding rendering extraction systems now panics rather than silently failing if no subapp with the RenderApp label is found.
• SceneSpawnerSystem now runs under CoreSet::Update, rather than CoreStage::PreUpdate.at_end().
• bevy_pbr::add_clusters is no longer an exclusive system
• the top level bevy_ecs::schedule module was replaced with bevy_ecs::scheduling
• tick_global_task_pools_on_main_thread is no longer run as an exclusive system. Instead, it has been replaced by tick_global_task_pools, which uses a NonSend resource to force running on the main thread.

Migration Guide

• Calls to .label(MyLabel) should be replaced with .in_set(MySet)
• Stages have been removed. Replace these with system sets, and then add command flushes using the apply_system_buffers exclusive system where needed.
• The CoreStage, StartupStage, RenderStageandAssetStage enums have been replaced withCoreSet, StartupSet, RenderSet and AssetSet. The same scheduling guarantees have been preserved.
  • Systems are no longer added to CoreSet::Update by default. Add systems manually if this behavior is needed, although you should consider adding your game logic systems to CoreSchedule::FixedTimestep instead for more reliable framerate-independent behavior.
  • Similarly, startup systems are no longer part of StartupSet::Startup by default. In most cases, this won't matter to you.
  • For example, add_system_to_stage(CoreStage::PostUpdate, my_system) should be replaced with
  • add_system(my_system.in_set(CoreSet::PostUpdate)
• When testing systems or otherwise running them in a headless fashion, simply construct and run a schedule using Schedule::new() and World::run_schedule rather than constructing stages
• Run criteria have been renamed to run conditions. These can now be combined with each other and with states.
• Looping run criteria and state stacks have been removed. Use an exclusive system that runs a schedule if you need this level of control over system control flow.
• For app-level control flow over which schedules get run when (such as for rollback networking), create your own schedule and insert it under the CoreSchedule::Outer label.
• Fixed timesteps are now evaluated in a schedule, rather than controlled via run criteria. The run_fixed_timestep system runs this schedule between CoreSet::First and CoreSet::PreUpdate by default.
• Command flush points introduced by AssetStage have been removed. If you were relying on these, add them back manually.
• the calculate_bounds system, with the CalculateBounds label, is now in CoreSet::Update, rather than in CoreSet::PostUpdate before commands are applied. You may need to order your movement systems to occur before this system in order to avoid system order ambiguities in culling behavior.
• the RenderLabel AppLabel was renamed to RenderApp for clarity
• App::add_state now takes 0 arguments: the starting state is set based on the Default impl.
• Instead of creating SystemSet containers for systems that run in stages, simply use .on_enter::<State::Variant>() or its on_exit or on_update siblings.
• SystemLabel derives should be replaced with SystemSet. You will also need to add the Debug, PartialEq, Eq, and Hash traits to satisfy the new trait bounds.
• with_run_criteria has been renamed to run_if. Run criteria have been renamed to run conditions for clarity, and should now simply return a bool.
• States have been dramatically simplified: there is no longer a "state stack". To queue a transition to the next state, call NextState::set
• Strings can no longer be used as a SystemLabel or SystemSet. Use a type, or use the system function instead.

TODO

• remove dead methods on App and World
• add App::add_system_to_schedule and App::add_systems_to_schedule
• avoid adding the default system set at inappropriate times
• remove any accidental cycles in the default plugins schedule
• migrate benchmarks
• expose explicit labels for the built-in command flush points
• migrate engine code
• remove all mentions of stages from the docs
• verify docs for States
• fix uses of exclusive systems that use .end / .at_start / .before_commands
• migrate RenderStage and AssetStage
• migrate examples
• ensure that transform propagation is exported in a sufficiently public way (the systems are already pub)
• ensure that on_enter schedules are run at least once before the main app
• re-enable opt-in to execution order ambiguities
• revert change to update_bounds to ensure it runs in PostUpdate
• test all examples
  • unbreak directional lights
  • unbreak shadows (see 3d_scene, 3d_shape, lighting, transparaency_3d examples)
  • game menu example shows loading screen and menu simultaneously
  • display settings menu is a blank screen
  • without_winit example panics
• ensure all tests pass
  • SubApp doc test fails
  • runs_spawn_local tasks fails
  • Fix panic_when_hierachy_cycle test hanging

Points of Difficulty and Controversy

Reviewers, please give feedback on these and look closely

1. Default sets, from the RFC, have been removed. These added a tremendous amount of implicit complexity and result in hard to debug scheduling errors. They're going to be tackled in the form of "base sets" by @cart in a followup.
2. The outer schedule controls which schedule is run when App::update is called.
3. I implemented Label for Box` for our label types. This enables us to store schedule labels in concrete form, and then later run them. I ran into the same set of problems when working with one-shot systems. We've previously investigated this pattern in depth, and it does not appear to lead to extra indirection with nested boxes.
4. SubApp::update simply runs the default schedule once. This sucks, but this whole API is incomplete and this was the minimal changeset.
5. time_system and tick_global_task_pools_on_main_thread no longer use exclusive systems to attempt to force scheduling order
6. Implemetnation strategy for fixed timesteps
7. AssetStage was migrated to AssetSet without reintroducing command flush points. These did not appear to be used, and it's nice to remove these bottlenecks.
8. Migration of bevy_render/lib.rs and pipelined rendering. The logic here is unusually tricky, as we have complex scheduling requirements.

Future Work (ideally before 0.10)

• Rename schedule_v3 module to schedule or scheduling
• Add a derive macro to states, and likely a EnumIter trait of some form
• Figure out what exactly to do with the "systems added should basically work by default" problem
• Improve ergonomics for working with fixed timesteps and states
• Polish FixedTime API to match Time
• Rebase and merge [Merged by Bors] - Stageless: add a method to scope to always run a task on the scope thread #7415
• Resolve all internal ambiguities (blocked on better tools, especially ScheduleBuildSettings should have a report_sets option #7442)
• Add "base sets" to replace the removed default sets.

[DasLixou]

[ ] add IterableEnum macro and trait (different PR please)

one question about it: does that mean you want a derive macro which generates a iterator over an enum where fields don't have any types?
Like

#[derive(IterableEnum)]
enum Alphabet {
   A,
   B,
   C,
}

let mut iter = Alphabet::enum_iter();
assert_eq!(iter.next(), Some(Alphabet::A));
assert_eq!(iter.next(), Some(Alphabet::B));
assert_eq!(iter.next(), Some(Alphabet::C));
assert_eq!(iter.next(), None);

or do i get it wrong? just asking out of interest 😅

[alice-i-cecile]

one question about it: does that mean you want a derive macro which generates a iterator over an enum where fields don't have any types?

Yes, precisely. See https://github.com/Leafwing-Studios/Emergence/blob/main/emergence_macros/src/iterable_enum.rs for example, although strum has one too.

[alice-i-cecile]

bors try

[mockersf]

bors r+

[bors]

Pull request successfully merged into main.

Build succeeded:

• build-and-install-on-iOS
• build-android
• build (macos-latest)
• build (ubuntu-latest)
• build-wasm
• build (windows-latest)
• build-without-default-features (bevy)
• build-without-default-features (bevy_ecs)
• build-without-default-features (bevy_reflect)
• check-compiles
• check-doc
• check-missing-examples-in-docs
• ci
• markdownlint
• msrv
• run-examples
• run-examples-on-wasm
• run-examples-on-windows-dx12

[bors]

try

Build succeeded:

• build-without-default-features (bevy)
• build-without-default-features (bevy_reflect)
• check-doc
• build-and-install-on-iOS
• build (windows-latest)
• build (ubuntu-latest)
• build (macos-latest)
• build-wasm
• build-android
• markdownlint
• run-examples
• run-examples-on-wasm
• check-missing-examples-in-docs
• ci
• check-compiles
• run-examples-on-windows-dx12
• build-without-default-features (bevy_ecs)
• msrv

[JoJoJet]

Spotted a small mistake in the migration guide:

• Instead of creating SystemSet containers for systems that run in stages states, simply use .on_enter::<State::Variant>() or its on_exit or on_update siblings.

I believe this is only true for on_update. "on enter" or "on exit" systems are added using App::add_system_to_schedule.