diff --git a/docs/feature/ios_test_plans.md b/docs/feature/ios_test_plans.md new file mode 100644 index 0000000000..62cd5a9131 --- /dev/null +++ b/docs/feature/ios_test_plans.md @@ -0,0 +1,187 @@ +# Flow + +Flow starts by parsing .xctestrun file. +Search for: `__xctestrun_metadata__` key. + +```xml +__xctestrun_metadata__ + + FormatVersion + 1 + +``` + +- **FormatVersion: `1` -** old version of .xctestrun +- **FormatVersion: `2` -** the newest version with test plans +- If format is different than 1 or 2 throw an error. + +--- + +### FormatVersion: 1 + +Any other key than metadata should have corresponding **TestTarget** dictionary. In example below `EarlGreyExampleSwiftTests` has a **TestTarget** dictionary. + +```xml + + + EarlGreyExampleSwiftTests + + BlueprintName + EarlGreyExampleSwiftTests + ... + + __xctestrun_metadata__ + + FormatVersion + 1 + + + +``` + +### FormatVersion: 2 + +In this version, XML contains two keys: `TestConfigurations` and `TestPlan` in addition to `__xctestrun_metadata__`. + +`TestPlan` is just dictionary containing basic informations about current **TestPlan.** We can ignore it. + +`TestConfigurations` is an array of different test configurations. Test configuration contains name property and array of TestTargets. + +```xml + + + Name + pl + TestTargets + + + BlueprintName + UITests + + + + BlueprintName + SecondUITests + + + + + +``` + +Each configuration may contain different Environment Variables, languages, regions or any other properties. Those properties are stored under TestTarget. + +Currently **FTL** doesn't support specifying TestConfiguration for test execution. + +If there is more than one configuration FTL will probably choose one arbitrarily. + +For now Flank will allow specifying which test configuration should run with `only-test-configuration` argument. + +--- + +# Running test plan locally + +## Build Xcode project + +To build example project run command below. + +```bash +xcodebuild build-for-testing \ +-allowProvisioningUpdates \ +-project "FlankMultiTestTargetsExample.xcodeproj" \ +-scheme "AllTests" \ #Scheme should have test plans enabled +-derivedDataPath "build_testplan_device" \ +-sdk iphoneos | xcpretty +``` + +This command will generate directory: **Debug-iphoneos** containing binaries and .xctestrun file for each TestPlan. + +In this example scheme `AllTests` has have only one test plan: **AllTests** with two test configurations: `pl` and `en`. + +**Test Plan** contains two **Test Targets: `UITests` and `SecondUITests`** +Outputted .xctestrun should looks like this: + +```xml + + + TestConfigurations + + + Name + en + TestTargets + + + BlueprintName + UITests + TestLanguage + en + TestRegion + GB + + + + BlueprintName + SecondUITests + TestLanguage + en + TestRegion + GB + + + + + + Name + pl + TestTargets + + + BlueprintName + UITests + TestLanguage + pl + TestRegion + PL + + + + BlueprintName + SecondUITests + TestLanguage + pl + TestRegion + PL + + + + + + TestPlan + + IsDefault + + Name + AllTests + + __xctestrun_metadata__ + + FormatVersion + 2 + + + +``` + +## Running tests on a local device + +After generating binaries and .xctestrun file we can run tests using command. + +```bash +xcodebuild test-without-building \ +-xctestrun "build_testplan_device/Build/Products/testrun.xctestrun" \ +-destination "platform=iOS,id=00008030-000209DC1A50802E" \ +-only-test-configuration pl | xcpretty +``` + +Option: `-only-test-configuration pl` allows to specify which test configuration should Xcode run. \ No newline at end of file