Where to go from the POC?

[egil]

The current iteration makes it fairly easy to test the initially generated HTML, in a fairly stable manner (spacing ignored, order of css classes doesnt matter, etc.).

However, a few shortcomings exists currently:

• The builder-based syntax for creating components and sub components/fragments works, but ideally we could write the SUT as razor code.
• Defining the expected output HTML in a string works OK but would be nicer in a file (e.g. .razor) where the editor understands HTML and can provide assistance.
• Missing a way to trigger various on-events (e.g. onclick) on the created SUT component and verify outcome.
• Missing a way to update a SUT component after first render and verify outcome.

Idears:
I like the idea of defining component tests in a .razor file, that inherits from a ComponentTestFixture class. ComponentTestFixture would be responsible for having the correct unit testing framework annotations, render the SUT component and grap the expected HTML.

It could also support various steps, where each step matches a step in a components life-cycle (i.e. first render, users interaction, parameter changed, disposed). E.g. something like this:

@inherits ComponentTestFixture

<SUT>
    <Alert color="primary"></Alert>
</SUT>

<ExpectedHtml>
    <div class="alert fade show alert-primary" role="alert"></div>
</ExpectedHtml>

[egil]

An alternative to defining tests in Razor files would be to define SUT components as strings in normal test methods, and somehow trigger the razor compiler. Its still not ideal to write html/razor in strings, but I prefer it to using the builder the library have now.

[chanan]

I would also suggest removing xunit into its own lib, then we can add other libs for MSTest and nUnit. Although, my next paragraph (if possible) could remove all of these options.

Still looking at it to figure out what is going on exactly, but I wouldnt move to strings quite yet, I wonder if there are other options like using "real" razor files. Here is what I am thinking (and i have no idea of it is even possible to do half of the things I am thinking of - which is kind of how Jest works):

Have a test project that in it there are a bunch of razor file that are the subject of the test. When the test project run (somehow) - it will execute all the test files and store the strings as files in the test project as .snapshot files with a known name. Next time you run the project it then does the compare to the .snapshot file.

What are your thoughts on that, assuming it can even be done?

The running the test project part could maybe be a dotnet global tool?

[egil]

The library itself does not have a dependency on xUnit. It does have one on XMLUnit, but that is needed to compare the HTML snippets. Shouldly, which it does have a dependency on, as well as Moq, could quite easily be removed. They are my preferred libraries, but not required.

I haven't played with Jest in a long time, so not entirely up-to-date on the functionality. I personally like to do TDD, and then snapshot testing isn't needed. That said, I do not think it will be hard to make that work. Here is a rough outline of what would be needed (assuming I understand your idea/requirements correctly):

Snapshot testing

Solution setup

• A component library with real production components (SUT components) - e.g. Alert.razor.
• A component test library with test.razor files containing various usages examples of the SUT components. E.g. Alert-without-dismiss-button.razor and Alert-with-dismiss-button.razor.

Testing Library

Since all modern testing libraries support data driven tests (Theories in xUnit), we can utilize that to create a custom test data source, let us call that RazorSnapshotTestDataSource.

RazorSnapshotTestDataSource will have the responsibility of finding all our test razor components, which is defined in our test library, and their (latest) snapshot files content. It should probably be configurable to e.g. control what happens if a snapshot doesn't exist already (generate new or just fail test).

With that, a data driven test will be as simple as this:

public class RazorComponentSnapshotTests : RazorComponentFixture
{
    [Theory]
    [RazorSnapshotTestDataSource]
    public void SnapshotTest(Type testComponent, string snapshot)
    {
        var expectedHtml = snapshot;
        var result = Component(testComponent).Render();
        result.ShouldBe(expectedHtml);
    }
}

RazorSnapshotTestDataSource will provide a pair of (Type testComponent, string snapshot) for each test that is defined in .razor files in the test project to the SnapshotTest method. That will run the test code for each pair, and should it fail, it will be printed out using the normal test output from e.g. xUnit.

The method Component(Type) used in the unit test doesn't exist in the framework right now, but it is trivial to create.

Considerations

1. This will work with simple tests, where there are no need to provide different services or control how they behave. Things like setting up a custom JSRuntime mock that does a certain thing is not included in the example above. To make that possible, we need to figure out a way for each test defined in a .razor file to specify custom behavior of services. Maybe custom attributes in the .razor file that we inspect using reflection?

2. This still only covers comparison of the initial render of a component, and does not seem to help testing how a component changes/behaves when a user triggers an 'on' event (e.g. onclick) on the component.

Improvements

It would be quite awesome if the test razor components (razor files in test project) could both include the test razor markup and the snapshot/expected html in the same file. Then it is much easier to inspect a test and see everything together.

Let me experiment a bit with the above and see if I can hack together a PoC.

[chanan]

I am working on the TestRunner idea in the meantime (I guess you would say the non TDD version of this), I will let you know if I get anywhere with it - so I dont think I need to change the base lib, we can have both versions working.

[egil]

I think we are working on the same thing to be honest. My approach would just be to utilize the visual studio test framework to execute a test runner. For my initial PoC that will be the xunit test runner.

With that approach we will reuse all the existing tooling around unittest runners.

[chanan]

Yes, I understand that is what you are doing, I am doing something slightly different, using the same library, I am making a console app (that could turn into a dotnet global tool maybe) that would load all Components from the library being tested and snapshot them. Both ways are useful and shouldnt conflict with each other

[egil]

OK. I agree, both are useful. Let me know when you have something.

Should we set up a Gitter channel to chat about this?

[egil]

I got defining tests in a razor file working locally.

Currently, this is what a test can look like (ComponentTest1.razor):

@using Microsoft.Extensions.DependencyInjection;

@code {
    ServiceCollection TestServices
    {
        get
        {
            var res = new ServiceCollection();
            // Add mocked services etc. here, e.g.: res.AddSingleton(x => Mock.Of<IJSRuntime>()); 
            return res;
        }
    }
}

<Fact DisplayName="Component renders without params 2" Services=TestServices>
    <Setup>
        <Component1 />
    </Setup>
    <ExpectedOutput>
        <div class="my-component xxx">
            This Blazor component is defined in the <strong>ComponentLib</strong> package.
        </div>
    </ExpectedOutput>
</Fact>
<Fact>
    <Setup>
        <Component1 />
    </Setup>
    <ExpectedOutput>
        <div class="my-component">
            This Blazor component is defined in the <strong>ComponentLib</strong> package.
        </div>
    </ExpectedOutput>
</Fact>

I have a special template component named Fact that allows you to specify a service collection to pass to the component under test (specified in ).

If you do not need to pass a custom service collection to component under test, a test razor component can be as simple as this, where one or more Fact components is included:

<Fact DisplayName="Component renders without params 2">
    <Setup><Component1 /></Setup>
    <ExpectedOutput>
        <div class="my-component xxx">
            This Blazor component is defined in the <strong>ComponentLib</strong> package.
        </div>
    </ExpectedOutput>
</Fact>

To run the tests, another class is needed, that XUnit will run. Right now it is enough to have a minimal class that just inherits from the RazorComponentTestFixture class, e.g.:

public class Component1Tests : RazorComponentTestFixture { }

How do you like this syntax compared to the original? I like it quite a lot, because it makes it so much nicer to Setup the component under test with params and everything and also specify expected output, since VS will provide intellisense, and we dont have to escape quotes in strings, etc.

No snapshotting yet, but it should be possible. My idea here is to allow the original .razor file to be modified and have its <ExpectedOutput> generated, if they are not provided, and a parameter (GenerateSnapshot) has been set on the <Fact> component. That does require knowledge at runtime about the location of the .razor files, but that should be doable.

[egil]

I have pushed my current progress to the egil-snapshot-testing branch. Check out the code in: https://github.com/egil/razor-component-testing-library/tree/egil-snapshot-testing/sample

[egil]

OK guys, think I have a pretty good solution for defining unit tests in razor files.

It solves a few of the issues mentioned originally, namely:

1. Being able to define test component in razor
2. Being able to define expected html in html/razor
3. Being able to access the rendered components, allowing us to assert against its state

Currently, all relevant code is in this folder: https://github.com/egil/razor-component-testing-library/tree/egil-snapshot-testing/sample/ComponentLibTests

Example:

Simple Test.Razor file that just compares rendered component and to expected html:

@inherits RazorComponentTest
<Fact>
    <Setup>
        <Component1 />
    </Setup>
    <ExpectedOutput>
        <div class="my-component">
            This Blazor component is defined in the <strong>ComponentLib</strong> package.
        </div>
    </ExpectedOutput>
</Fact>

Another example that shows of a few other possiblities:

@inherits RazorComponentTest
@using Xunit
@using Moq
@using Shouldly
@using Microsoft.Extensions.DependencyInjection
@using Microsoft.JSInterop

@code{
    // Override AddServices to provide services used when renderinger
    protected override void AddServices(IServiceCollection services)
    {
        services.AddScoped<IJSRuntime>(_ => Mock.Of<IJSRuntime>());
    }

    // Use @ref to get a reference to the tested element
    Component1 sut1;

    // Add more than the default input/output html test in @code sections

    [Fact]
    public void CanReadPropFromSut()
    {
        sut1.AccessibleProp.ShouldBeTrue();
    }
}
<Fact>
    <Setup><Component1 @ref="@sut1" /></Setup>
    <ExpectedOutput>
        <div class="my-component">
            This Blazor component is defined in the <strong>ComponentLib</strong> package.
        </div>
    </ExpectedOutput>
</Fact>

@code {
    // Access rendered result of Fact elements in the RenderResults list.
    [Fact]
    public void AnotherTest()
    {
        var expectedText = "This Blazor component is defined in the ComponentLib package.";

        RenderResults[1].RenderedHtml.InnerText.Trim().ShouldBe(expectedText);
    }
}
<Fact>
    <Setup><Component1 /></Setup>
    <ExpectedOutput>
        <div class="my-component">
            This Blazor component is defined in the <strong>ComponentLib</strong> package.
        </div>
    </ExpectedOutput>
</Fact>

TODO:

• Enable snapshotting - i.e. generating expected html if non is provided. Perhaps only when a specific @atttribute has been added to the .razor file?
• Create custom xunit runners/test discoverers, such that individual <Fact /> components are reported in the Test explorer and to console runner as individual tests.
• Figure out how to reuse RazorComponentFixture between all unit tests in a single test class (.razor file). That will save us for re-rendering the entire .razor file for each test.

Snapshotting

I prefer if we update the original Test.razor and simply add the missing <ExpectedOutput/> to it, if snapshotting is enabled and requested. Generating a separate .snapshot file would probably best be placed next to the original Test.razor file anyway, and it helps with readability to have them together.

Two challenges with this:

1. How to get the location of the Test.razor file. This problem seems very environment dependent, e.g. the .razor files might be in different locations depending on the environment (local dev computer, build server, etc.). We need a good strategy here!

2. Parse and replace the proper sections of the Test.razor. This part doesnt seem hard, using some search and replace for the <Fact /> element.

Please provide some feedback to the above!

[egil]

I'll close this issue, and create a separate for snapshotting.

[egil]

Snapshotting moved to its own issue: #1