Snapbox updated the same inline snapshots shared between different tests

[weihanglo]

When an inline snapshot is shared between different #[test], it could be updated multiple times, causing the final updated snapshot being wrong.

• Discussion
  • https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/Sharing.20inline.20snapshot.20between.20different.20.60.23.5Btest.5D.60.20cases
  • test: migrate registry and registry_auth to snapbox rust-lang/cargo#14149 (comment)
• Package Version: snapbox@=0.6.10
• Cargo version: cargo 1.81.0-nightly (bc89bffa5 2024-06-22)

Here is a reproducible case using Cargo -Zscript:

---
[dependencies]
snapbox = "=0.6.10"
---

pub fn shared_snapshots() {
    snapbox::assert_data_eq!("Bad hobbits!", snapbox::str![]);
    snapbox::assert_data_eq!("My precious!\n", snapbox::str![]);
}

#[test]
fn frodo_baggins() {
    shared_snapshots();
}
#[test]
fn samwise_gamgee() {
    shared_snapshots();
}
#[test]
fn peregrin_took() {
    shared_snapshots();
}
#[test]
fn meriadoc_brandybuck() {
    shared_snapshots();
}

You can found snapbox has overwritten the expected string multiple times:

$ SNAPSHOTS=overwrite cargo +nightly -Zscript test --manifest-path lib.rs
...
...

Fixing: 
---- expected: /home/user/.cargo/target/69/8aed57ccbe65fe/lib.rs:8:48
++++ actual:   In-memory
        1 + My precious!

Fixing: 
---- expected: /home/user/.cargo/target/69/8aed57ccbe65fe/lib.rs:8:48
++++ actual:   In-memory
        1 + My precious!

Fixing: 
---- expected: /home/user/.cargo/target/69/8aed57ccbe65fe/lib.rs:8:48
++++ actual:   In-memory
        1 + My precious!

Fixing: 
---- expected: /home/user/.cargo/target/69/8aed57ccbe65fe/lib.rs:8:48
++++ actual:   In-memory
        1 + My precious!

test frodo_baggins ... ok
test meriadoc_brandybuck ... ok
test samwise_gamgee ... ok
test peregrin_took ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

$ cat /home/user/.cargo/target/69/8aed57ccbe65fe/lib.rs
pub fn shared_snapshots() {
    snapbox::assert_data_eq!("Bad hobbits!", snapbox::str!["Bad hobbits!""Bad hobbits!""Bad hobbits!""Bad hobbits!"]);
    snapbox::assert_data_eq!("My precious!\n", snapbox::str![[r#"
My precious!

"#][r#"
My precious!

"#][r#"
My precious!

"#][r#"
My precious!

"#]]);
}

#[test]
fn frodo_baggins() {
    shared_snapshots();
}
#[test]
fn samwise_gamgee() {
    shared_snapshots();
}
#[test]
fn peregrin_took() {
    shared_snapshots();
}
#[test]
fn meriadoc_brandybuck() {
    shared_snapshots();
}

[weihanglo]

I am not sure if this is a thing worth fixing in snapbox. To me an inline snapshot should not be shared. Who knows when those tests start diverging from each other.

[epage]

For some of the tests I ran into this, it was intentional that the output is the same (post redactions) so I think its reasonable to fix.