Track source location in change detection

[aevyrie]

Objective

• Make it possible to know what changed your component or resource.
• Common need when debugging, when you want to know the last code location that mutated a value in the ECS.
• This feature would be very useful for the editor alongside system stepping.

Solution

• Adds the caller location to column data.
• Mutations now track_caller all the way up to the public API.
• Commands that invoke these functions immediately call Location::caller, and pass this into the functions, instead of the functions themselves attempting to get the caller. This would not work for commands which are deferred, as the commands are executed by the scheduler, not the user's code.

Testing

• The component_change_detection example now shows where the component was mutated:

2024-07-28T06:57:48.946022Z  INFO component_change_detection: Entity { index: 1, generation: 1 }: New value: MyComponent(0.0)
2024-07-28T06:57:49.004371Z  INFO component_change_detection: Entity { index: 1, generation: 1 }: New value: MyComponent(1.0)
2024-07-28T06:57:49.012738Z  WARN component_change_detection: Change detected!
        -> value: Ref(MyComponent(1.0))
        -> added: false
        -> changed: true
        -> changed by: examples/ecs/component_change_detection.rs:36:23

• It's also possible to inspect change location from a debugger:

---

Changelog

• Added source locations to ECS change detection behind the track_change_detection flag.

Migration Guide

• Added changed_by field to many internal ECS functions used with change detection when the track_change_detection feature flag is enabled. Use Location::caller() to provide the source of the function call.

[cart]

Mut is one of the hottest paths in Bevy. I'm concerned about the performance implications of introducing anything (even the cheapest ops) to it. I'd like to either see compelling evidence that this won't meaningfully affect performance, especially given that this is a "debugging tool". I'd feel much more comfortable if this was behind a feature flag.

[alice-i-cecile]

I really like this feature and think it'll be very useful, but I'll second the request for putting it behind an off-by-default feature flag.

[aevyrie]

Yup, I assumed as much. I was hoping to get a read on whether this is even an acceptable approach to take for the feature. Before going down the path of adding feature flags.

[Brezak]

Is there a reason you're de referencing the static reference you get from Location::caller? Couldn't you just pass the reference around?

[BD103]

Is there a reason you're de referencing the static reference you get from Location::caller? Couldn't you just pass the reference around?

Location is two u32s and one &str, making it 16 bytes on 64-bit platforms. Storing a reference instead would bring it down to 8 bytes, which might have some mild performance gains because it would fit in a register.

Furthermore, why do you use core::panic::Location as compared with the std variant?

[aevyrie]

• static reference: I started with this, but was having trouble with the deref making it hard to update the field, so I switched to an owned value to get it working. I agree it should be fixed if this is going to be merged.
• using core: because it can't hurt? I wasn't sure if bevy_ecs was ever intended to be used in a no_std context, and figured it would be safer to use core for a type so ubiquitous.

The answer to most questions is probably going to be "because this is a proof of concept". 😄
I wasn't sure if this was going to be possible or if it would return any useful information, so I'm mostly just happy to see that it's doable. The useful findings from this are:

1. It's possible to get line and column level tracing for change detection, which is super useful
2. Changes from commands completely break this approach because commands are deferred, and the call stack can't be traced to the calling location by the time the command is actually executed.

For 2, it seems like it should be possible to get information from tracing to figure out what system the commands are being executed for. Not quite sure how to do that though, might need to replace Location with a new struct that also contains the Location as well as the current system span.

[BD103]

2. Changes from commands completely break this approach because commands are deferred, and the call stack can't be traced to the calling location by the time the command is actually executed.

Is it possible to find the Location in Commands::insert, then pass that forward to the actual command?

[alice-i-cecile]

using core: because it can't hurt? I wasn't sure if bevy_ecs was ever intended to be used in a no_std context, and figured it would be safer to use core for a type so ubiquitous.

I would like this, one day :) Probably core + alloc, but still! Not a priority, but might as well do it when there's no cost to it.

[SpecificProtagonist]

Is it possible to find the Location in Commands::insert, then pass that forward to the actual command?

Yep, that would look like this (you'd also need to do the same for init_resource, insert_resource, spawn_batch etc).

[aevyrie]

Good call! That looks pretty straightforward.

[slyedoc]

Spent far to long trying to figure out why a camera transform from glb was getting reset, was correct on spawn, next frame its Identity, this found the issue in seconds. Was calling remove_parent_in_place before TransformPropagate had ran. Huge help.

[NthTensor]

I like this and I want it. Should Commands::add and Command::push be track_caller so that custom user commands can provide call tracing? Or would that be something user should do on the methods of their extension trait.

It looks like this fully covers commands as well as direct world access. Except for the special paths that let you avoid change detection, is there anything that this can't track?

[BD103]

I like this and I want it. Should Commands::add and Command::push be track_caller so that custom user commands can provide call tracing? Or would that be something user should do on the methods of their extension trait.

If we do want to support custom commands like this, it would need more documentation. I feel like this is probably follow-up PR territory.

[NthTensor]

follow-up PR territory.

Fair enough, agreed.

[workingjubilee]

Is there a reason you're de referencing the static reference you get from Location::caller? Couldn't you just pass the reference around?

Location is two u32s and one &str, making it 16 bytes on 64-bit platforms. Storing a reference instead would bring it down to 8 bytes, which might have some mild performance gains because it would fit in a register.

at the risk of sounding incredibly pedantic: 24 bytes. &str is the pointer and the length, so it is already 16 bytes.

remember that this is a pretty magical struct that gets willed into existence by the compiler during codegen. I kinda expect it to have somewhat nonintuitive codegen properties because inspecting Location::caller() kinda... creates the thing as much as it returns a reference.

[SpecificProtagonist]

Should Commands::add and Command::push be track_caller so that custom user commands can provide call tracing?

At best that would mean storing the caller location and changing Commands::apply to take a &Location, but custom commands can already just store the location themselves in their constructor.

In either case the problem for custom commands is that insert_with_caller & friends aren't pub. So the actual question is: Do we want to make them pub?

[SpecificProtagonist]

This and insert_non_send_by_id are the only pub functions that take Locations (and also have a unused #[track_caller]); they should be spit into a pub version with #[track_caller] and a pub(crate) one that takes the caller.

[aevyrie]

insert_with_caller & friends aren't pub. So the actual question is: Do we want to make them pub?

I think so. This will be needed for custom impls, as you said. We can document the context around this on the with_caller methods. I wish there was a way to make this pattern nicer, but alas.

[SpecificProtagonist]

self.set_changed already updates the location

[BD103]

at the risk of sounding incredibly pedantic: 24 bytes. &str is the pointer and the length, so it is already 16 bytes.

Hah, I forgot about wide pointers. Thanks!

[alice-i-cecile]

Thank you to everyone involved with the authoring or reviewing of this PR! This work is relatively important and needs release notes! Head over to bevyengine/bevy-website#1666 if you'd like to help out.