Added additional wiki content - especially testing

.vscode/settings.json

@@ -29,5 +29,13 @@
     "powershell.codeFormatting.useCorrectCasing": true,
     "powershell.codeFormatting.whitespaceBetweenParameters": true,
     "yaml.format.singleQuote": true,
-    "markdown.extension.tableFormatter.enabled": false
+    "markdown.extension.tableFormatter.enabled": false,
+    "spellright.language": [
+        "en"
+    ],
+    "spellright.documentTypes": [
+        "markdown",
+        "latex",
+        "plaintext"
+    ]
 }

docs/wiki/GettingStarted.md

@@ -45,7 +45,6 @@ To ensure you can use all the content in this repostiroy you'd want to install
 
 Depending on how you want to use this repositories content you may go down different paths to get started.
 
-
 ### **Option 1**: Use it as a basis to set up your own inner-source project
 
 The repository is set up in a way that you can essentially create your own private 1:1 copy and would be able to re-use the same concepts and functionality in your own environment like GitHub. This set up is a 2-step process. First, you have to either 'Form' the repository to you own GitHub account, or move it to your desired location manually. And second you have to configure the environment, that is, you have to update all references to the original source respository to your own and also set up several secrets to point to the Azure environment of your choice.
@@ -64,6 +63,14 @@ To fork the repostory you can simply click on the `Fork` button on the top right
 
 Once forked, make sure you update all references to the original repository like for example any link that points to the original location.
 
+#### Service Names
+On of the most important actions you should take from the beginning is to update the parameter files in your module. Though you may not want to use all modules that are available, even the subset you use may currently be configured with resource names that must be globally unique and are already taken. For those it is recommended that you define your own unique naming schema (for example a special prefix) and update the resource names accordingly.
+
+Please refer to [this list][AzureNames] to check which services have a global scope and must be updated.
+
+#### Dependencies
+As the modules we test often times have dependencies to other servies, we created a pipeline to deploys several standard services like VirtualNetworks and KeyVaults (alongside dummy secrets) for the modules to use. This _dependency_ pipeline should be prepared and executed before you start running any pipelines on your own. In case you need to rename any services there (for example because a certain globally unique resource name was already taken) make sure to update any references to this name in the module parameter files. You can find further details about this pipeline [here](.\TestingDesign.md#Module-Dependencies).
+
 #### GitHub-specific prerequisites
 In case you want to not only leverage the module templates but actually re-use the implemented pipelines & testing framework as well, you need to set up a few additional secrets in your GitHub environment:
 
@@ -117,3 +124,4 @@ Last but not least, instead of fetching your own copy of the repository you can
 
 <!-- Docs -->
 [PowerShellDocs]: <https://docs.microsoft.com/en-us/powershell/>
+[AzureNames]: <https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/resource-name-rules>

docs/wiki/Home.md

@@ -15,8 +15,8 @@ This wiki describes the content of this repository, the modules, pipelines, poss
   - [Design](./TestingDesign)
   - [Usage](./TestingUsage)
 - [Pipelines](./Pipelines)
-  - [Design](./PipelineDesign)
-  - [Usage](./PipelineUsage)
+  - [Design](./PipelinesDesign)
+  - [Usage](./PipelinesUsage)
 - [Contribution Guide](./ContributionGuide)
 - [Known Issues](./KnownIssues)
 

docs/wiki/KnownIssues.md

@@ -1,3 +1,31 @@
 # Known Issues
 
-This section gives you an overview of the repositories known issues.
+This section gives you an overview of the repositories larger known issues. We are actively working on resolving these problems and track them as issues.
+
+
+---
+### _Navigation_
+- [Module specific](#module-specific)
+- [Testing specific](#testing-specific)
+- [Pipeline specific](#pipeline-specific)
+---
+
+## Module specific
+This section outlines known issues that currently affect our modules.
+
+### Consistency
+While we work hard on aligning all modules to a common standard, we're currently missing consistency in how we're handling child-resources across all modules (e.g. Container for a Storage Account). Ref issue #236.
+
+## Testing specific
+This section outlines known issues that currently affect our testing.
+
+### Not all modules are removed after their test deployment
+In general, the current approach works for about 80-90% of the modules. That said, there are several different reasons why some of the modules are currently not auto-removed:
+- The module does not support tags: The current test implementation heavily relies on tags to find and remove a test-deployed resource. However, as not all resources support tags (for example Role Assignments), these must be removed in a different way that is yet to be implemented.
+- The module's resources must be removed in a specific order: Some resources like Azure NetAppFiles & VirtualWan, while supporting tags, cannot be removed as is. Due to their nature they deploy several child-resources that must be removed first. However, the current implementation of the removal is not aware of these limitations.
+
+### Limited parameter file set
+We have yet to implement the full set of parameter files we need in order to test all possible scenarios. The most important first step will be a 'minimum-set' parameter file vs. a 'maximum-set' parameter file for each module, followed by parameter files for specific scenarios
+
+## Pipeline specific
+This section outlines known issues that currently affect our pipelines.

docs/wiki/TestingDesign.md

@@ -4,22 +4,52 @@ This section gives you an overview of the design principals the testing follows.
 
 ---
 ### _Navigation_
+- [Design Overview](#Design-Overview)
+  - [Unit Tests](#unit-tests)
+  - [Template Validation Tests](#template-validation-tests)
+  - [Template Deployment Tests](#template-deployment-tests)
 - [Module Dependencies](#Module-Dependencies)
 ---
 
 
-##  Approach
-...
+## Design Overview  Approach
 
-### Module Dependencies
-In order to successfully deploy and test all Modules in your desired environment some Modules have to have resources deployed beforehand.
+Our lifecycle management & testing framework runs 3 types of tests on all its modules:
+- Pester-driven Unit Tests
+- Template Validation Tests
+- Template Deployment Tests
+All tests are executed as part of the individual module pipelines, run each time any module code was altered, and ensure that only modules that pass each test successfully are published. If a test fails, it tells you in the pipeline log exactly what went wrong and in most cases gives you recommendations what to do to resolve the problem.
 
-Of course it is obvious and by default one should know which Azure Service needs specific resources to be deployed beforehand but here is the full list of Modules which have dependencies on other Services.
+The general idea is that you should fail as early as possible to allow for minimal wasted time and a fast response time.
+
+> ***Note:*** Both the Template Validation and Template Deployment tests are only as good as their parameter files. Hence you should make sure that you test at least a minimum set of parameters and a maximum set of parameters. Furthermore it makes sense to have different parameter files for different scenarios to test each variant.
+
+### Unit Tests
+Our Unit Tests are configured in the `global.module.tests.ps1` script and execute static tests across several different area to ensure that our modules are configured correctly, documentation is up to date, and modules don't turn stale.
+We can categorize these tests into a few different categories:
+
+- **File & folder tests:** These tests validate that the module folder structure is set up in the intended way. For example, we test that each module should contain a parameters folder with at least one parameter file in it that follows a specific naming convention.
+- **ReadMe tests:** These tests ensure that a module's readme contains all required sections, that for example the documented parameters match the ones in the template, and that a consistent format is applied.
+- **Deployment template tests:** These tests check the template's structure and elements for errors as well as consistency matters. For example, we test that names are set up in a certain way, that if specific resources are contained in the template that they are set up in the format we want them in, and for example that a minimum set of outputs are always returned.
+- **Api version tests:** These tests make sure that the API versions applied to resources are recent. For example, the test checks that an applied API version should not be older than the five latest (non-preview) versions.
+
+### Template Validation Tests
+The template validation tests execute a dry-run with each parameter file provided & configured for a module. For example, if you have two parameter files for a module, one with the minimum set of parameters, one with the maximum, the tests will run an `Test-AzDeployment` (_- the command may vary based on the template schema_) with each of the two parameter files to see if the template would be able to be deployed with them. This test could fail either because the template is invalid, or because any of the parameter files is configured incorrectly.
+
+### Template Deployment Tests
+If all other tests passed, the deployment tests are the ultimate module validation. Using the available & configured parameter files for a module, each is deployed to Azure (in parallel) and verifies if the deployment works end to end.
+
+Most of the resources are deleted by default after their deployment, to keep costs down and to be able to retest resource modules from scratch in the next run. However, the removal step can be skipped in case further investigation on the deployed resource is needed. For further details, please refer to the (.\PipelinesUsage.md) section.
+
+## Module Dependencies
+In order to successfully deploy and test all modules in your desired environment some modules have to have resources deployed beforehand.
+
+Of course it is obvious and by default one should know which Azure Service needs specific resources to be deployed beforehand but here is the full list of modules which have dependencies on other Services.
 
 > **Note**<br>
-If we speak from **Modules** in this context we mean the **Services** which get created from these Modules.
+If we speak from **modules** in this context we mean the **Services** which get created from these modules.
 
-#### Services (in order)
+### Services (in order)
 1. VirtualNetwork
 1. StorageAccounts
 1. KeyVault
@@ -34,8 +64,8 @@ If we speak from **Modules** in this context we mean the **Services** which get
 1. SharedImageGallery
 1. ApplicationSecurityGroups
 1. NetworkSecurityGroups
-1. WvdHostPool
-1. WvdAppliccationGroups
+1. AvdHostPool
+1. AvdAppliccationGroups
 1. Managed Service Identity
 1. Deployment Scripts
 
@@ -95,10 +125,10 @@ If we speak from **Modules** in this context we mean the **Services** which get
   - KeyVault / Secret (vpnSharedKey)
 - WebApp
   - AppServicePlan
-- WvdApplicationGroups
-  - WvdHostPool
-- WvdApplications
-  - WvdAppliccationGroups
+- AvdApplicationGroups
+  - AvdHostPool
+- AvdApplications
+  - AvdAppliccationGroups
 
 #### Required Secrets and Keys