az policy auditIfNotExists vnetRouteAllEnabled. Compliance scan fails when WEBSITE_VNET_ROUTE_ALL=1

[DanielLarsenNZ]

Resource Provider

Microsoft.Web/sites

Description of Feature or Work Requested

An ARM Policy deployed with az policy definition create and az policy assignment create with an effect of auditIfNotExists that tests for the presence of Microsoft.Web/sites/config/vnetRouteAllEnabled = true will report failed when App Services App Setting WEBSITE_VNET_ROUTE_ALL=1.

Background

In Azure App Services, you can force a Web App to route its outbound traffic through a VNet (as opposed to Internet) by setting an App Setting (environment variable) WEBSITE_VNET_ROUTE_ALL: Regional VNet Integration.

App Services team have submitted a PR to expose this property via ARM: Add vnetRouteAllEnabled and vnetPrivatePortsCount to Microsoft.Web/sites/config swagger #11152.

We now need this property to be effective for ARM Policy compliance scans.

Minimum API Version Required

Not sure.

Swagger Link

Add vnetRouteAllEnabled and vnetPrivatePortsCount to Microsoft.Web/sites/config swagger #11152

Target Date

ASAP.

[yonzhan]

policy

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @aperezcloud, @kenieva.

Issue Details

---

Resource Provider

Microsoft.Web/sites

Description of Feature or Work Requested

An ARM Policy deployed with az policy definition create and az policy assignment create with an effect of auditIfNotExists that tests for the presence of Microsoft.Web/sites/config/vnetRouteAllEnabled = true will report failed when App Services App Setting WEBSITE_VNET_ROUTE_ALL=1.

Background

In Azure App Services, you can force a Web App to route its outbound traffic through a VNet (as opposed to Internet) by setting an App Setting (environment variable) WEBSITE_VNET_ROUTE_ALL: Regional VNet Integration.

App Services team have submitted a PR to expose this property via ARM: Add vnetRouteAllEnabled and vnetPrivatePortsCount to Microsoft.Web/sites/config swagger #11152.

We now need this property to be effective for ARM Policy compliance scans.

Minimum API Version Required

Not sure.

Swagger Link

Add vnetRouteAllEnabled and vnetPrivatePortsCount to Microsoft.Web/sites/config swagger #11152

Target Date

ASAP.

Author:	DanielLarsenNZ
Assignees:	zhoxing-ms
Labels:	Policy, Service Attention, feature-request
Milestone:	S181

[kenieva]

Investigating..

[kenieva]

Hi @DanielLarsenNZ, it seems that this is a known issue by that RP: https://github.com/Azure/azure-policy#resource-type-query-results-incomplete-missing-or-non-standard-format

[DanielLarsenNZ]

Yes that is right :D And on that page is a link to open a support ticket with the Web team, which is how we started with this case in the first place. This scenario is very important to this customer.

[kenieva]

If they have added the change into swagger, the only action on the policy side would be to create aliases. But per our discussion, it seems that the aliases are exposed.

[DanielLarsenNZ]

Here is where the property was added to rest-api-specs by the Web team. Apparently it's available as of version 2020-09-01.

Azure/azure-rest-api-specs@7c3af69

[kenieva]

Policy operates on the latest version so the property should have been added. So the alias has been updated and exposed, but is it still not working as required?

[DanielLarsenNZ]

Hi @kenieva, I have just tested this again and it is still not working from my perspective. Here are my steps to reproduce:

1. Run deploy-vnet-route-all.ps1

This will deploy a test App Service with the WEBSITE_VNET_ROUTE_ALL=1 App Setting configured. The app attempts to create a connection to https://www.dropbox.com/ but is prevented from connecting because traffic is being forced into the VNet and is being blocked by an NSG.

1. Run deploy-policy-azcli.ps1

This will deploy the policy to the resource group that the App Service is deployed into. When the scan completes* you will see that the audit policy is non compliant. My expectation is that the policy would report the resource as compliant.

*You can trigger the scan on the Resource Group with this command: az policy state trigger-scan -g 'hellovnetrouteall-rg'

[kenieva]

Hi Daniel, from this policy: https://github.com/DanielLarsenNZ/azure-policy/blob/master/samples/WebApp/web-app-vnet-route-all-enabled/azurepolicy.json the existence condition reads if the Microsoft.Web/sites/config/vnetRouteAllEnabled equals true then audit as non-compliant. I believe the logic you are looking for is if Microsoft.Web/sites/config/vnetRouteAllEnabled equals false.

[DanielLarsenNZ]

Hi @kenieva, this is the rule I am using (the logic is identical): azurepolicy.rules.json.

The effect is auditIfNotExists: If any matching related resource evaluates to true, the effect is satisfied and doesn't trigger the audit.

Never the less, I tried your suggestion and the policy does go compliant, but it is also compliant when the WEBSITE_VNET_ROUTE_ALL=1 setting is missing, so it is still not working - note in my screen shot above, the Current value is false which is not correct.

[DanielLarsenNZ]

Thanks to @kahawai-sre I finally got this to work. The vnetRouteAllEnabled property must be set using ARM. Policy will not be effected by the App Setting (WEBSITE_VNET_ROUTE_ALL).

For this policy to take effect:

1. App Service must be deployed using an ARM API Version >= 2019-04-01
2. App Service (Microsoft.Web/sites) property siteConfig > vnetRouteAllEnabled must be set to true using an ARM template, e.g.

"resources": [
    {
      "type": "Microsoft.Web/sites",
      "apiVersion": "2020-09-01",
      "name": "[parameters('appName')]",
      "location": "[parameters('location')]",
      "kind": "app",
      "dependsOn": [
        
      ],
      "properties": {
        "serverFarmId": "[resourceId('Microsoft.Web/serverFarms', parameters('appServicePlanName'))]",
        "siteConfig": {
          "vnetRouteAllEnabled": true
        }
      }
    }
  ]