Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iterative test case with multiple Snapshot.Match fails in 0.5.4 but worked in 0.5.2 #98

Closed
moredatapls opened this issue Jul 3, 2020 · 3 comments · Fixed by #99
Closed
Assignees
Labels
bug Something isn't working

Comments

@moredatapls
Copy link

Describe the bug

We have a test case that consists of a for loop. In each iteration, an operation is performed and its result is compared to a snapshot. The name of the snapshot is dynamically created in each loop.

[Fact]
public void IterativeTest()
{
    const string nameBase = "TestCase";

    for (var i = 0; i < 2; i++)
    {
        var contents = $"this is iteration = {i}";

        var name = $"{nameBase}_{i}";

        Snapshot.Match(contents, Snapshot.FullName(SnapshotNameExtension.Create(name)));
    }
}

When I run this test with version 0.5.2, the files are created and compared as expected. When I upgrade to 0.5.4, the test case fails because in the second iteration, the contents are compared to the file of the first iteration.

In 0.5.2

  • i = 0: name = "TestCase_0", contents are compared to TestCase_0.snap -> successful
  • i = 1: name = "TestCase_1", contents are compared to TestCase_1.snap -> successful
  • -> test succeeds

In 0.5.4

  • i = 0: name = "TestCase_0", contents are compared to TestCase_0.snap -> successful
  • i = 1: name = "TestCase_1", contents are compared to TestCase_0.snap -> fails
  • -> test fails

To Reproduce

Steps to reproduce the behavior:

  1. Add the test case I posted above
  2. Run with 0.5.2, will work
  3. Run with 0.5.4, will fail

Expected behavior

The test case should not fail.

Desktop (please complete the following information):

  • OS: macOS Catalina
  • Version: 10.15.5
@michaelstaib
Copy link
Member

Hi,

I have introduced this little bug with a feature that I have added to support some cases with test helpers.

We will introduce a new test context and with that fix this issue.

Thanks for reporting this.

@michaelstaib michaelstaib self-assigned this Jul 3, 2020
@michaelstaib michaelstaib added the bug Something isn't working label Jul 3, 2020
@jannickj
Copy link

maybe remove the bug until feature is ready?, I just spent 20 min debugging my code :(

@RohrerF
Copy link
Member

RohrerF commented Jul 27, 2020

this issue should be fixed in version 0.5.5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants