Testing

Tools

Framework

• XUnit is used in most of the projects any new test projects created should utilize XUnit.
• NUnit is used in some legacy projects, we tend to migrate all tests to run under XUnit so any new test projects created should utilize XUnit.
• xharness is a dotnet tool and TestRunner library that makes running unit tests in mobile platforms easier.

Project Test Types

• Unit tests - These are tests that will not run on a device. This is useful for testing device independent logic.

├── Controls 
│   ├── test
│   │   ├── Controls.Core.UnitTests
│   │   ├── Controls.Core.XamlUnitTests
├── Core 
│   ├── test
│   │   ├── Core.UnitTests
├── Essentials 
│   ├── test
│   │   ├── Essentials.UnitTests
├── Graphics
│   ├── test
│   │   ├── Graphics.Tests
├── SingleProject
│   ├── test
│   │   ├── Resizetizer.UnitTests

• Device tests - These are tests that will run on an actual device or simulator

├── Controls 
│   ├── test
│   │   ├── Controls.DeviceTests
├── Core 
│   ├── test
│   │   ├── Core.DeviceTests
├── Essentials 
│   ├── test
│   │   ├── Essentials.DeviceTests
├── Graphics
│   ├── test
│   │   ├── Graphics.DeviceTests
├── BlazorWebView 
│   ├── test
│   │   ├── MauiBlazorWebView.DeviceTests

• UI tests - These are tests that will run on an actual device or simulator and interact with an application

├── Controls 
│   ├── test
│   │   ├── Controls.AppiumTests

• Controls.UITests: .NET MAUI Controls Visual Runner for running device based xunit tests. This is useful for tests that require XAML features

Test count

	25/01/2022	20/09/2023
Controls.Core.UnitTests	4670	5498
Controls.Core.XamlUnitTests	989	1037
Controls.DeviceTests.ios	198	257
Controls.DeviceTests.android	214	305
Controls.DeviceTests.maccatalyst	199	212
Controls.DeviceTests.windows	230	0
Controls.UITests.ios	0	38
Controls.UITests.android	0	38
Controls.UITests.maccatalyst	0	28
Controls.UITests.windows	0	25
Core.UnitTests	475	655
Core.DeviceTests.ios	2782	1034
Core.DeviceTests.android	599	1193
Core.DeviceTests.maccatalyst	592	956
Core.DeviceTests.windows	440	810
Essentials.UnitTests	989	283
Essentials.DeviceTests.ios	280	120
Essentials.DeviceTests.android	278	134
Essentials.DeviceTests.maccatalyst	280	123
Essentials.DeviceTests.windows	287	133
MauiBlazorWebView.DeviceTests	1	0
MauiBlazorWebView.DeviceTests.ios	0	3
MauiBlazorWebView.DeviceTests.android	0	3
MauiBlazorWebView.DeviceTests.maccatalyst	0	3
MauiBlazorWebView.DeviceTests.windows	0	3
Graphics.UnitTests	84	316
Graphics.DeviceTests	3	0
Graphics.DeviceTests.ios	0	8
Graphics.DeviceTests.android	0	8
Graphics.DeviceTests.maccatalyst	0	8
Graphics.DeviceTests.windows	0	8
Resizetizer.UnitTests	0	543
Integration.UnitTests	0	36
Total	13572	16661

Running Tests

We use xunit.runner.visualstudio and nunittestadapter for test discoverability and running the tests. This allows us to view the tests from within Visual Studio via the Test Explorer tool pane and utilizing vstest.console.exe <AssemblyPath> from a developer command prompt.

Writing tests

Controls unit tests

The tests on this project are written using XUnit. Add your class and use BaseTestFixture as your base class. You can also add a [Category("Layout")] . You might need to create a Stub for your test you can add that to the TestClasses folder

public class CarouselViewTests : BaseTestFixture
{
	[Fact]
	public void TestPositionChangedCommand()
	{
		var source = new List<string> { "1", "2", "3" };
		var carouselView = new CarouselView
		{
			ItemsSource = source
		};
		int countFired = 0;
		carouselView.PositionChangedCommand = new Command(() => countFired = countFired + 1;);
		Assert.Same(source, carouselView.ItemsSource);
		carouselView.Position = 1;
		Assert.True(countFired == 1);
	}
}

Xaml unit tests

The tests on this project are written using NUnit. To create a new test start by adding a new Xaml ContentPage to the Controls.Core.XamlUnitTests project. In the code behind add your NUnit TestFixture. Make sure to specify a 2nd actor for two TestCase for True/False so that the test runs with and without XAMLC enabled. Also take note that the default actor will not be called 50% of the times.

<ContentPage 
    xmlns="http://schemas.microsoft.com/dotnet/2021/maui"
    xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
    x:Class="Microsoft.Maui.Controls.Xaml.UnitTests.AutoMergedResourceDictionaries">
    <ContentPage.Resources>
        <ResourceDictionary>
            <Color x:Key="notpink">Purple</Color>
            <Color x:Key="pink">Pink</Color>
            <ResourceDictionary Source="AppResources/Colors.xaml" />
        </ResourceDictionary>
    </ContentPage.Resources>
    <Label x:Name="label" TextColor="{StaticResource notpink}" BackgroundColor="{StaticResource Primary}"/>
</ContentPage>

public partial class AutoMergedResourceDictionaries : ContentPage
{
	public AutoMergedResourceDictionaries()
	{
		InitializeComponent();
	}

	public AutoMergedResourceDictionaries(bool useCompiledXaml)
	{
		//this stub will be replaced at compile time
	}

	[TestFixture]
	public class Tests
	{
		[TestCase(false)]
		[TestCase(true)]
		public void AutoMergedRd(bool useCompiledXaml)
		{
			var layout = new AutoMergedResourceDictionaries(useCompiledXaml);
			Assert.That(layout.label.TextColor, Is.EqualTo(Colors.Purple));
			Assert.That(layout.label.BackgroundColor, Is.EqualTo(Color.FromArgb("#FF96F3")));
		}
	}
}

Controls UI tests

First, add your sample page in Controls.Sample.UITests. Then, include a test in Controls.AppiumTests. The tests on this project are written using XUnit and Appium. Add your class and use _IssuesUITest as your base class.

[Issue(IssueTracker.Github, 14257, "VerticalStackLayout inside Scrollview: Button at the bottom not clickable on IOS", PlatformAffected.iOS)]
public class Issue14257 : TestContentPage
{
	protected override void Init()
	{
		var scrollView = new ScrollView();
		var layout = new VerticalStackLayout() { Margin = new Microsoft.Maui.Thickness(10, 40) };

		var description = new Label { Text = "Tap the Resize button; this will force the Test button off the screen. Then tap the Test button; if a Label with the text \"Success\" appears, the test has passed." };

		var resizeButton = new Button() { Text = "Resize", AutomationId = "Resize" };
		var layoutContent = new Label() { Text = "Content", HeightRequest = 50 };
		var testButton = new Button() { Text = "Test", AutomationId = "Test" };
		var resultLabel = new Label() { AutomationId = "Result" };

		layout.Add(description);
		layout.Add(resizeButton);
		layout.Add(layoutContent);
		layout.Add(resultLabel);
		layout.Add(testButton);

		scrollView.Content = layout;
		Content = scrollView;

		resizeButton.Clicked += (sender, args) =>
		{
			// Resize the ScrollView content so the test button will be off the screen
			// If the bug is present, this will make the button untappable
			layoutContent.HeightRequest = 1000;
		};

		// Show the Success label if the button is tapped, so we can verify the bug is not present
		testButton.Clicked += (sender, args) => { resultLabel.Text = "Success"; };
	}
}

public class Issue14257 : _IssuesUITest
{
	public Issue14257(TestDevice device) : base(device) { }

	public override string Issue => "VerticalStackLayout inside Scrollview: Button at the bottom not clickable on IOS";

	[Test]
	public void ResizeScrollViewAndTapButtonTest()
	{
		// Tapping the Resize button will change the height of the ScrollView content
		App.Click("Resize");

		// Scroll down to the Test button. When the bug is present, the button cannot be tapped.
		App.ScrollTo("Test");

		App.WaitForElement("Test");
		App.Click("Test");

		// If we can successfully tap the button, the Success label will be displayed
		Assert.IsTrue(App.WaitForTextToBePresentInElement("Result", "Success"));
	}
}

To run the tests on iOS or Catalyst, you'll need a Mac. To run the tests on Windows, you'll need a Windows machine. Android tests can be run from either platform. You can review the prerequisites and how to run the tests here.