├── .gitignore ├── docs ├── github-fork.png ├── test-pyramid.png ├── github-build-status.png ├── github-run-workflow.png ├── pester-test-results.png ├── azure-powershell-debug.png └── azure-management-groups.png ├── .github ├── ISSUE_TEMPLATE │ ├── custom.md │ ├── feature_request.md │ └── bug_report.md └── workflows │ └── test-policies.yml ├── .vscode └── launch.json ├── LICENSE ├── .azure-pipelines └── test-policies.yml ├── tests ├── Modify-RouteTable-NextHopVirtualAppliance.Tests.ps1 ├── Audit-Route-NextHopVirtualAppliance.Tests.ps1 └── Deny-Route-NextHopVirtualAppliance.Tests.ps1 ├── CODE_OF_CONDUCT.md ├── utils ├── Rest.Utils.psm1 ├── Test.Utils.psm1 ├── RouteTable.Utils.psm1 └── Policy.Utils.psm1 ├── policies ├── Audit-Route-NextHopVirtualAppliance.json ├── Modify-RouteTable-NextHopVirtualAppliance.json └── Deny-Route-NextHopVirtualAppliance.json └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | tests/coverage.xml 2 | tests/testResults.xml 3 | -------------------------------------------------------------------------------- /docs/github-fork.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fawohlsc/azure-policy-testing/HEAD/docs/github-fork.png -------------------------------------------------------------------------------- /docs/test-pyramid.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fawohlsc/azure-policy-testing/HEAD/docs/test-pyramid.png -------------------------------------------------------------------------------- /docs/github-build-status.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fawohlsc/azure-policy-testing/HEAD/docs/github-build-status.png -------------------------------------------------------------------------------- /docs/github-run-workflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fawohlsc/azure-policy-testing/HEAD/docs/github-run-workflow.png -------------------------------------------------------------------------------- /docs/pester-test-results.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fawohlsc/azure-policy-testing/HEAD/docs/pester-test-results.png -------------------------------------------------------------------------------- /docs/azure-powershell-debug.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fawohlsc/azure-policy-testing/HEAD/docs/azure-powershell-debug.png -------------------------------------------------------------------------------- /docs/azure-management-groups.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fawohlsc/azure-policy-testing/HEAD/docs/azure-management-groups.png -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/custom.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Custom issue template 3 | about: Describe this issue template's purpose here. 4 | title: '' 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | 11 | -------------------------------------------------------------------------------- /.vscode/launch.json: -------------------------------------------------------------------------------- 1 | { 2 | // Use IntelliSense to learn about possible attributes. 3 | // Hover to view descriptions of existing attributes. 4 | // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 5 | "version": "0.2.0", 6 | "configurations": [ 7 | { 8 | "name": "PowerShell: Interactive Session", 9 | "type": "PowerShell", 10 | "request": "launch", 11 | "cwd": "" 12 | } 13 | ] 14 | } -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for this project 4 | title: '' 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Is your feature request related to a problem? Please describe.** 11 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 12 | 13 | **Describe the solution you'd like** 14 | A clear and concise description of what you want to happen. 15 | 16 | **Describe alternatives you've considered** 17 | A clear and concise description of any alternative solutions or features you've considered. 18 | 19 | **Additional context** 20 | Add any other context or screenshots about the feature request here. 21 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | title: '' 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Describe the bug** 11 | A clear and concise description of what the bug is. 12 | 13 | **To Reproduce** 14 | Steps to reproduce the behavior: 15 | 1. Go to '...' 16 | 2. Click on '....' 17 | 3. Scroll down to '....' 18 | 4. See error 19 | 20 | **Expected behavior** 21 | A clear and concise description of what you expected to happen. 22 | 23 | **Screenshots** 24 | If applicable, add screenshots to help explain your problem. 25 | 26 | **Desktop (please complete the following information):** 27 | - OS: [e.g. iOS] 28 | - Browser [e.g. chrome, safari] 29 | - Version [e.g. 22] 30 | 31 | **Smartphone (please complete the following information):** 32 | - Device: [e.g. iPhone6] 33 | - OS: [e.g. iOS8.1] 34 | - Browser [e.g. stock browser, safari] 35 | - Version [e.g. 22] 36 | 37 | **Additional context** 38 | Add any other context about the problem here. 39 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Fabian Wohlschläger 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /.github/workflows/test-policies.yml: -------------------------------------------------------------------------------- 1 | name: test-policies 2 | on: 3 | push: 4 | branches: 5 | - main 6 | paths: 7 | - ".github/workflows/**" 8 | - "policies/**" 9 | - "tests/**" 10 | - "utils/**" 11 | workflow_dispatch: 12 | inputs: 13 | remarks: 14 | description: "Reason for triggering the workflow run" 15 | required: false 16 | default: "Testing Azure Policies..." 17 | jobs: 18 | test-policies: 19 | runs-on: ubuntu-latest 20 | steps: 21 | - name: Checkout repository 22 | uses: actions/checkout@v2 23 | - name: Install PowerShell modules 24 | shell: pwsh 25 | run: | 26 | Install-Module -Name "Az" -RequiredVersion "4.7.0" -Force -Scope CurrentUser -ErrorAction Stop 27 | - name: Login to Azure 28 | uses: azure/login@v1 29 | with: 30 | creds: ${{secrets.AZURE_CREDENTIALS}} 31 | enable-AzPSSession: true 32 | - name: Create or update Azure Policies 33 | shell: pwsh 34 | run: | 35 | Get-ChildItem -Path "./policies" | ForEach-Object { 36 | New-AzDeployment -Location "northeurope" -TemplateFile $_.FullName 37 | } 38 | # Logout/Login to Azure to ensure that the latest policies are applied 39 | - name: Logout of Azure 40 | shell: pwsh 41 | run: | 42 | # Suppress printing out client secret in clear text by sending output to $null. 43 | # See also: https://github.com/Azure/azure-powershell/issues/14208 44 | Disconnect-AzAccount > $null 45 | - name: Login to Azure 46 | uses: azure/login@v1 47 | with: 48 | creds: ${{secrets.AZURE_CREDENTIALS}} 49 | enable-AzPSSession: true 50 | - name: Test Azure Policies 51 | shell: pwsh 52 | run: | 53 | Invoke-Pester -Output Detailed -CI 54 | -------------------------------------------------------------------------------- /.azure-pipelines/test-policies.yml: -------------------------------------------------------------------------------- 1 | parameters: 2 | - name: "Reason" 3 | displayName: "Reason for triggering the pipeline run" 4 | type: string 5 | default: "Testing Azure Policies..." 6 | 7 | trigger: 8 | branches: 9 | include: 10 | - main 11 | paths: 12 | include: 13 | - ".github/workflows/**" 14 | - "policies/**" 15 | - "tests/**" 16 | - "utils/**" 17 | 18 | pool: 19 | vmImage: ubuntu-latest 20 | 21 | steps: 22 | - checkout: self 23 | displayName: "Checkout repository" 24 | - pwsh: | 25 | Install-Module -Name "Az" -RequiredVersion "4.7.0" -Force -Scope CurrentUser -ErrorAction Stop 26 | Import-Module -Name "Az" -RequiredVersion "4.7.0" -Force 27 | displayName: "Install and import PowerShell modules" 28 | workingDirectory: $(Build.SourcesDirectory) 29 | - pwsh: | 30 | $password = ConvertTo-SecureString $env:AZURE_SUBSCRIPTION_CLIENT_SECRET -AsPlainText -Force 31 | $credential = New-Object System.Management.Automation.PSCredential($env:AZURE_SUBSCRIPTION_CLIENT_ID, $password) 32 | Connect-AzAccount -Subscription $env:AZURE_SUBSCRIPTION_ID -Tenant $env:AZURE_SUBSCRIPTION_TENANT_ID -ServicePrincipal -Credential $credential -Scope CurrentUser > $null 33 | displayName: "Login to Azure" 34 | env: 35 | AZURE_SUBSCRIPTION_CLIENT_SECRET: $(AZURE_SUBSCRIPTION_CLIENT_SECRET) 36 | - pwsh: | 37 | Get-ChildItem -Path "./policies" | ForEach-Object { 38 | New-AzDeployment -Location "northeurope" -TemplateFile $_.FullName 39 | } 40 | displayName: "Create or update Azure Policies" 41 | workingDirectory: $(Build.SourcesDirectory) 42 | # Logout/Login to Azure to ensure that the latest policies are applied 43 | - pwsh: | 44 | # Suppress printing out client secret in clear text by sending output to $null. 45 | # See also: https://github.com/Azure/azure-powershell/issues/14208 46 | Disconnect-AzAccount > $null 47 | displayName: "Logout of Azure" 48 | - pwsh: | 49 | $password = ConvertTo-SecureString $env:AZURE_SUBSCRIPTION_CLIENT_SECRET -AsPlainText -Force 50 | $credential = New-Object System.Management.Automation.PSCredential($env:AZURE_SUBSCRIPTION_CLIENT_ID, $password) 51 | Connect-AzAccount -Subscription $env:AZURE_SUBSCRIPTION_ID -Tenant $env:AZURE_SUBSCRIPTION_TENANT_ID -ServicePrincipal -Credential $credential -Scope CurrentUser > $null 52 | displayName: "Login to Azure" 53 | env: 54 | AZURE_SUBSCRIPTION_CLIENT_SECRET: $(AZURE_SUBSCRIPTION_CLIENT_SECRET) 55 | - pwsh: | 56 | Invoke-Pester -Output Detailed -CI 57 | displayName: "Test Azure Policies" 58 | - pwsh: | 59 | Clear-AzContext -Scope CurrentUser -Force -ErrorAction SilentlyContinue | Out-Null 60 | displayName: "Cleanup cached Azure credentials" 61 | condition: always() 62 | -------------------------------------------------------------------------------- /tests/Modify-RouteTable-NextHopVirtualAppliance.Tests.ps1: -------------------------------------------------------------------------------- 1 | Import-Module -Name Az.Network 2 | Import-Module -Name Az.Resources 3 | Import-Module "$($PSScriptRoot)/../utils/Policy.Utils.psm1" -Force 4 | Import-Module "$($PSScriptRoot)/../utils/Rest.Utils.psm1" -Force 5 | Import-Module "$($PSScriptRoot)/../utils/RouteTable.Utils.psm1" -Force 6 | Import-Module "$($PSScriptRoot)/../utils/Test.Utils.psm1" -Force 7 | 8 | Describe "Testing policy 'Modify-RouteTable-NextHopVirtualAppliance'" -Tag "modify-routetable-nexthopvirtualappliance" { 9 | # Create or update route tables is actually the same PUT request, hence testing create covers update as well. 10 | # PATCH requests are currently not supported in Network Resource Provider. 11 | # See also: https://docs.microsoft.com/en-us/rest/api/virtualnetwork/routetables/createorupdate 12 | Context "When route table is created or updated" -Tag "modify-routetable-nexthopvirtualappliance-routetable-create-update" { 13 | It "Should add missing route 0.0.0.0/0 pointing to the virtual appliance" -Tag "modify-routetable-nexthopvirtualappliance-routetable-create-update-10" { 14 | AzTest -ResourceGroup { 15 | param($ResourceGroup) 16 | 17 | $routeTable = New-AzRouteTable ` 18 | -Name "route-table" ` 19 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 20 | -Location $ResourceGroup.Location 21 | 22 | # Verify that route 0.0.0.0/0 was added by policy. 23 | $routeTable 24 | | Test-RouteNextHopVirtualAppliance 25 | | Should -BeTrue 26 | } 27 | } 28 | } 29 | 30 | Context "When route is deleted" -Tag "modify-routetable-nexthopvirtualappliance-route-delete" { 31 | It "Should remediate missing route 0.0.0.0/0 pointing to the virtual appliance" -Tag "modify-routetable-nexthopvirtualappliance-route-delete-10" { 32 | AzTest -ResourceGroup { 33 | param($ResourceGroup) 34 | 35 | $routeTable = New-AzRouteTable ` 36 | -Name "route-table" ` 37 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 38 | -Location $ResourceGroup.Location 39 | 40 | # Get route 0.0.0.0/0 pointing to the virtual appliance, which was added by policy. 41 | $route = Get-RouteNextHopVirtualAppliance -RouteTable $routeTable 42 | 43 | # Remove-AzRouteConfig/Set-AzRouteTable will issue a PUT request for routeTables and hence policy might kick in. 44 | # In order to delete the route without policy interfering, directly call the REST API by issuing a DELETE request for route. 45 | $routeTable | Invoke-RouteDelete -Route $route 46 | 47 | # Remediate route table by policy and wait for completion. 48 | $routeTable | Complete-PolicyRemediation -PolicyDefinitionName "Modify-RouteTable-NextHopVirtualAppliance" -CheckDeployment 49 | 50 | # Verify that route 0.0.0.0/0 was added by policy remediation. 51 | Get-AzRouteTable -ResourceGroupName $routeTable.ResourceGroupName -Name $routeTable.Name 52 | | Test-RouteNextHopVirtualAppliance 53 | | Should -BeTrue 54 | } 55 | } 56 | } 57 | } -------------------------------------------------------------------------------- /tests/Audit-Route-NextHopVirtualAppliance.Tests.ps1: -------------------------------------------------------------------------------- 1 | Import-Module -Name Az.Network 2 | Import-Module -Name Az.Resources 3 | Import-Module "$($PSScriptRoot)/../utils/Policy.Utils.psm1" -Force 4 | Import-Module "$($PSScriptRoot)/../utils/Rest.Utils.psm1" -Force 5 | Import-Module "$($PSScriptRoot)/../utils/RouteTable.Utils.psm1" -Force 6 | Import-Module "$($PSScriptRoot)/../utils/Test.Utils.psm1" -Force 7 | 8 | Describe "Testing policy 'Audit-Route-NextHopVirtualAppliance'" -Tag "audit-route-nexthopvirtualappliance" { 9 | Context "When auditing route tables" { 10 | It "Should mark route table as compliant with route 0.0.0.0/0 pointing to virtual appliance." -Tag "audit-route-nexthopvirtualappliance-compliant" { 11 | AzTest -ResourceGroup { 12 | param($ResourceGroup) 13 | 14 | # Create compliant route table. 15 | $route = New-AzRouteConfig ` 16 | -Name "default" ` 17 | -AddressPrefix "0.0.0.0/0" ` 18 | -NextHopType "VirtualAppliance" ` 19 | -NextHopIpAddress (Get-VirtualApplianceIpAddress -Location $ResourceGroup.Location) 20 | 21 | $routeTable = New-AzRouteTable ` 22 | -Name "route-table" ` 23 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 24 | -Location $ResourceGroup.Location ` 25 | -Route $Route 26 | 27 | # Trigger compliance scan for resource group and wait for completion. 28 | $ResourceGroup | Complete-PolicyComplianceScan 29 | 30 | # Verify that route table is compliant. 31 | $routeTable 32 | | Get-PolicyComplianceState -PolicyDefinitionName "Audit-Route-NextHopVirtualAppliance" 33 | | Should -BeTrue 34 | } 35 | } 36 | 37 | It "Should mark route table as incompliant without route 0.0.0.0/0 pointing to virtual appliance." -Tag "audit-route-nexthopvirtualappliance-incompliant" { 38 | AzTest -ResourceGroup { 39 | param($ResourceGroup) 40 | 41 | # Create incompliant route table by deleting route 0.0.0.0/0 pointing to the virtual appliance. 42 | $routeTable = New-AzRouteTable ` 43 | -Name "route-table" ` 44 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 45 | -Location $ResourceGroup.Location ` 46 | -Route $Route 47 | 48 | # Get route 0.0.0.0/0 pointing to the virtual appliance, which was added by policy. 49 | $route = Get-RouteNextHopVirtualAppliance -RouteTable $RouteTable 50 | 51 | # Remove-AzRouteConfig/Set-AzRouteTable will issue a PUT request for routeTables and hence policy might kick in. 52 | # In order to delete the route without policy interfering, directly call the REST API by issuing a DELETE request for route. 53 | $routeTable | Invoke-RouteDelete -Route $route 54 | 55 | # Trigger compliance scan for resource group and wait for completion. 56 | $ResourceGroup | Complete-PolicyComplianceScan 57 | 58 | # Verify that route table is incompliant. 59 | $routeTable 60 | | Get-PolicyComplianceState -PolicyDefinitionName "Audit-Route-NextHopVirtualAppliance" 61 | | Should -BeFalse 62 | } 63 | } 64 | } 65 | } 66 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, sex characteristics, gender identity and expression, 9 | level of experience, education, socio-economic status, nationality, personal 10 | appearance, race, religion, or sexual identity and orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the project team at . All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 71 | available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html 72 | 73 | [homepage]: https://www.contributor-covenant.org 74 | 75 | For answers to common questions about this code of conduct, see 76 | https://www.contributor-covenant.org/faq 77 | -------------------------------------------------------------------------------- /utils/Rest.Utils.psm1: -------------------------------------------------------------------------------- 1 | Import-Module -Name Az.Resources 2 | 3 | <# 4 | .SYNOPSIS 5 | Await an asynchronous operation against the Azure REST API. 6 | 7 | .DESCRIPTION 8 | Helper method to await an asynchronous operation against the Azure REST API. 9 | 10 | .PARAMETER HttpResponse 11 | The HTTP response returned from the asynchronous operation. 12 | 13 | .PARAMETER MaxRetries 14 | The maximum retries to monitor the status of the asynchronous operation (Default: 100 times). 15 | 16 | .EXAMPLE 17 | if ($httpResponse.StatusCode -eq 202) { 18 | $asyncOperation = $httpResponse | Wait-AsyncOperation 19 | if ($asyncOperation.Status -ne "Succeeded") { 20 | throw "Asynchronous operation failed with message: '$($asyncOperation)'" 21 | } 22 | } 23 | 24 | .LINK 25 | https://github.com/Azure/azure-powershell/issues/13293 26 | 27 | .LINK 28 | https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/async-operations#status-codes-for-asynchronous-operations 29 | 30 | .LINK 31 | https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/async-operations#url-to-monitor-status 32 | #> 33 | function Wait-AsyncOperation { 34 | param ( 35 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 36 | [ValidateNotNull()] 37 | [Microsoft.Azure.Commands.Profile.Models.PSHttpResponse]$HttpResponse, 38 | [Parameter()] 39 | [ValidateRange(1, [uint32]::MaxValue)] 40 | [uint32]$MaxRetries = 100 41 | ) 42 | 43 | # Asynchronous operations either return HTTP status code 201 (Created) or 202 (Accepted). 44 | # See also: https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/async-operations#status-codes-for-asynchronous-operations 45 | if ($HttpResponse.StatusCode -notin @(201, 202)) { 46 | throw "HTTP response status code must be either '201' or '202' to indicate an asynchronous operation." 47 | } 48 | 49 | # Extracting retry after from HTTP Response Headers. 50 | # See also: https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/async-operations#url-to-monitor-status 51 | $retryAfter = $HttpResponse 52 | | Get-HttpResponseHeaderValues -HeaderName "Retry-After" 53 | | Select-Object -First 1 54 | 55 | # Extracting status URL from HTTP Response Headers. 56 | $statusUrl = $HttpResponse 57 | | Get-HttpResponseHeaderValues -HeaderName "Azure-AsyncOperation" 58 | | Select-Object -First 1 59 | 60 | if ($null -eq $statusUrl) { 61 | $statusUrl = $HttpResponse 62 | | Get-HttpResponseHeaderValues -HeaderName "Location" 63 | | Select-Object -First 1 64 | } 65 | 66 | if ($null -eq $statusUrl) { 67 | throw "HTTP response does not contain any header 'Azure-AsyncOperation' or 'Location' containing the URL to monitor the status of the asynchronous operation." 68 | } 69 | 70 | # Convert status URL to path. 71 | $statusPath = $statusUrl.Replace("https://management.azure.com", "") 72 | 73 | # Monitor status of asynchronous operation. 74 | $httpResponse = $null 75 | $retries = 0 76 | do { 77 | $asyncOperation = Invoke-AzRestMethod -Path $statusPath -Method "GET" 78 | | Select-Object -ExpandProperty Content 79 | | ConvertFrom-Json 80 | 81 | if ($asyncOperation.Status -in @("Succeeded", "Failed", "Canceled")) { 82 | break 83 | } 84 | else { 85 | Start-Sleep -Second $retryAfter 86 | $retries++ 87 | } 88 | } until ($retries -gt $MaxRetries) # Prevent endless loop, just defensive programming. 89 | 90 | if ($retries -gt $MaxRetries) { 91 | throw "Status of asynchronous operation '$($statusPath)' could not be retrieved even after $($MaxRetries) retries." 92 | } 93 | 94 | return $asyncOperation 95 | } 96 | 97 | <# 98 | .SYNOPSIS 99 | Gets HTTP header values from a HTTP response. 100 | 101 | .DESCRIPTION 102 | Helper method to extract HTTP header values from a HTTP response. 103 | 104 | .PARAMETER HttpResponse 105 | The HTTP response. 106 | 107 | .PARAMETER HeaderName 108 | The name of the HTTP header. 109 | 110 | .EXAMPLE 111 | $statusUrl = $HttpResponse | Get-HttpResponseHeaderValues -HeaderName "Azure-AsyncOperation" | Select-Object -First 1 112 | #> 113 | function Get-HttpResponseHeaderValues { 114 | param ( 115 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 116 | [ValidateNotNull()] 117 | [Microsoft.Azure.Commands.Profile.Models.PSHttpResponse]$HttpResponse, 118 | [Parameter(Mandatory = $true)] 119 | [ValidateNotNullOrEmpty()] 120 | [string]$HeaderName 121 | ) 122 | 123 | $headerValues = New-Object System.Collections.Generic.List[string] 124 | $httpResponse.Headers.TryGetValues($HeaderName, [ref] $headerValues) > $null 125 | return $headerValues 126 | } -------------------------------------------------------------------------------- /policies/Audit-Route-NextHopVirtualAppliance.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#", 3 | "contentVersion": "1.0.0.0", 4 | "variables": { 5 | "policyName": "Audit-Route-NextHopVirtualAppliance", 6 | "policyDescription": "Audit route tables for route with address prefix 0.0.0.0/0 pointing to the virtual appliance.", 7 | "policyCategory": "Network", 8 | "policyAssignmentParameters": { 9 | "routeTableSettings": { 10 | "value": { 11 | "northeurope": { 12 | "virtualApplianceIpAddress": "10.0.0.23" 13 | }, 14 | "westeurope": { 15 | "virtualApplianceIpAddress": "10.1.0.23" 16 | }, 17 | "disabled": { 18 | "virtualApplianceIpAddress": "" 19 | } 20 | } 21 | } 22 | } 23 | }, 24 | "resources": [ 25 | { 26 | "name": "[variables('policyName')]", 27 | "type": "Microsoft.Authorization/policyDefinitions", 28 | "apiVersion": "2020-03-01", 29 | "properties": { 30 | "policyType": "Custom", 31 | "mode": "All", 32 | "displayName": "[variables('policyName')]", 33 | "description": "[variables('policyDescription')]", 34 | "metadata": { 35 | "category": "[variables('policyCategory')]" 36 | }, 37 | "parameters": { 38 | "routeTableSettings": { 39 | "type": "Object", 40 | "metadata": { 41 | "displayName": "Route Table Settings", 42 | "description": "Location-specific settings for route tables." 43 | } 44 | } 45 | }, 46 | "policyRule": { 47 | "if": { 48 | "allOf": [ 49 | { 50 | "field": "type", 51 | "equals": "Microsoft.Network/routeTables" 52 | }, 53 | { 54 | "count": { 55 | "field": "Microsoft.Network/routeTables/routes[*]", 56 | "where": { 57 | "allOf": [ 58 | { 59 | "field": "Microsoft.Network/routeTables/routes[*].addressPrefix", 60 | "equals": "0.0.0.0/0" 61 | }, 62 | { 63 | "field": "Microsoft.Network/routeTables/routes[*].nextHopType", 64 | "equals": "VirtualAppliance" 65 | }, 66 | { 67 | "field": "Microsoft.Network/routeTables/routes[*].nextHopIpAddress", 68 | "equals": "[[parameters('routeTableSettings')[field('location')].virtualApplianceIpAddress]" 69 | } 70 | ] 71 | } 72 | }, 73 | "equals": 0 74 | } 75 | ] 76 | }, 77 | "then": { 78 | "effect": "audit" 79 | } 80 | } 81 | } 82 | }, 83 | { 84 | "name": "[uniqueString(variables('policyName'))]", 85 | "type": "Microsoft.Authorization/policyAssignments", 86 | "apiVersion": "2020-03-01", 87 | "properties": { 88 | "displayName": "[variables('policyName')]", 89 | "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions', variables('policyName'))]", 90 | "parameters": "[variables('policyAssignmentParameters')]", 91 | "description": "[variables('policyDescription')]", 92 | "metadata": { 93 | "category": "[variables('policyCategory')]" 94 | }, 95 | "enforcementMode": "Default" 96 | }, 97 | "dependsOn": [ 98 | "[resourceId('Microsoft.Authorization/policyDefinitions', variables('policyName'))]" 99 | ] 100 | } 101 | ] 102 | } -------------------------------------------------------------------------------- /policies/Modify-RouteTable-NextHopVirtualAppliance.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#", 3 | "contentVersion": "1.0.0.0", 4 | "variables": { 5 | "policyName": "Modify-RouteTable-NextHopVirtualAppliance", 6 | "policyDescription": "Adds route with address prefix 0.0.0.0/0 pointing to the virtual appliance in case there is none. Best combined with policy deny-route-nexthopvirtualappliance to ensure the correct IP address of the virtual appliance.", 7 | "policyCategory": "Network", 8 | // Built-in role 'Network Contributor'. 9 | "policyRoleDefinitionId": "/providers/microsoft.authorization/roleDefinitions/4d97b98b-1d4f-4787-a291-c67834d212e7", 10 | "policyAssignmentName": "[uniqueString(variables('policyName'))]", 11 | "policyAssignmentLocation": "northeurope", 12 | "policyAssignmentParameters": { 13 | "routeTableSettings": { 14 | "value": { 15 | "northeurope": { 16 | "virtualApplianceIpAddress": "10.0.0.23" 17 | }, 18 | "westeurope": { 19 | "virtualApplianceIpAddress": "10.1.0.23" 20 | }, 21 | "disabled": { 22 | "virtualApplianceIpAddress": "" 23 | } 24 | } 25 | } 26 | } 27 | }, 28 | "resources": [ 29 | { 30 | "name": "[variables('policyName')]", 31 | "type": "Microsoft.Authorization/policyDefinitions", 32 | "apiVersion": "2020-03-01", 33 | "properties": { 34 | "policyType": "Custom", 35 | "mode": "All", 36 | "displayName": "[variables('policyName')]", 37 | "description": "[variables('policyDescription')]", 38 | "metadata": { 39 | "category": "[variables('policyCategory')]" 40 | }, 41 | "parameters": { 42 | "routeTableSettings": { 43 | "type": "Object", 44 | "metadata": { 45 | "displayName": "Route Table Settings", 46 | "description": "Location-specific settings for route tables." 47 | } 48 | } 49 | }, 50 | "policyRule": { 51 | "if": { 52 | "allOf": [ 53 | { 54 | "field": "type", 55 | "equals": "Microsoft.Network/routeTables" 56 | }, 57 | { 58 | "count": { 59 | "field": "Microsoft.Network/routeTables/routes[*]", 60 | "where": { 61 | "field": "Microsoft.Network/routeTables/routes[*].addressPrefix", 62 | "equals": "0.0.0.0/0" 63 | } 64 | }, 65 | "equals": 0 66 | } 67 | ] 68 | }, 69 | "then": { 70 | "effect": "modify", 71 | "details": { 72 | "roleDefinitionIds": [ 73 | "[variables('policyRoleDefinitionId')]" 74 | ], 75 | "conflictEffect": "audit", 76 | "operations": [ 77 | { 78 | "operation": "add", 79 | "field": "Microsoft.Network/routeTables/routes[*]", 80 | "value": { 81 | "name": "default", 82 | "properties": { 83 | "addressPrefix": "0.0.0.0/0", 84 | "nextHopType": "VirtualAppliance", 85 | "nextHopIpAddress": "[[parameters('routeTableSettings')[field('location')].virtualApplianceIpAddress]" 86 | } 87 | } 88 | } 89 | ] 90 | } 91 | } 92 | } 93 | } 94 | }, 95 | { 96 | "name": "[variables('policyAssignmentName')]", 97 | "type": "Microsoft.Authorization/policyAssignments", 98 | "apiVersion": "2020-03-01", 99 | "properties": { 100 | "displayName": "[variables('policyName')]", 101 | "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions', variables('policyName'))]", 102 | "parameters": "[variables('policyAssignmentParameters')]", 103 | "description": "[variables('policyDescription')]", 104 | "metadata": { 105 | "category": "[variables('policyCategory')]" 106 | }, 107 | "enforcementMode": "Default" 108 | }, 109 | "location": "[variables('policyAssignmentLocation')]", 110 | "identity": { 111 | "type": "SystemAssigned" 112 | }, 113 | "dependsOn": [ 114 | "[resourceId('Microsoft.Authorization/policyDefinitions', variables('policyName'))]" 115 | ] 116 | }, 117 | { 118 | "name": "[guid(variables('policyAssignmentName'))]", 119 | "type": "Microsoft.Authorization/roleAssignments", 120 | "apiVersion": "2020-04-01-preview", 121 | "properties": { 122 | "principalType": "ServicePrincipal", 123 | "roleDefinitionId": "[variables('policyRoleDefinitionId')]", 124 | // Get the identifier of the managed identity created for the policy assignment. 125 | "principalId": "[reference(variables('policyAssignmentName'), '2020-03-01', 'Full').identity.principalId]" 126 | }, 127 | "dependsOn": [ 128 | "[resourceId('Microsoft.Authorization/policyAssignments', variables('policyAssignmentName'))]" 129 | ] 130 | } 131 | ] 132 | } -------------------------------------------------------------------------------- /policies/Deny-Route-NextHopVirtualAppliance.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#", 3 | "contentVersion": "1.0.0.0", 4 | "variables": { 5 | "policyName": "Deny-Route-NextHopVirtualAppliance", 6 | "policyDescription": "Deny route with address prefix 0.0.0.0/0 not pointing to the virtual appliance. Both creating routes as a standalone resource or nested within their parent resource route table are considered.", 7 | "policyCategory": "Network", 8 | "policyAssignmentParameters": { 9 | "routeTableSettings": { 10 | "value": { 11 | "northeurope": { 12 | "virtualApplianceIpAddress": "10.0.0.23" 13 | }, 14 | "westeurope": { 15 | "virtualApplianceIpAddress": "10.1.0.23" 16 | }, 17 | "disabled": { 18 | "virtualApplianceIpAddress": "" 19 | } 20 | } 21 | } 22 | } 23 | }, 24 | "resources": [ 25 | { 26 | "name": "[variables('policyName')]", 27 | "type": "Microsoft.Authorization/policyDefinitions", 28 | "apiVersion": "2020-03-01", 29 | "properties": { 30 | "policyType": "Custom", 31 | "mode": "All", 32 | "displayName": "[variables('policyName')]", 33 | "description": "[variables('policyDescription')]", 34 | "metadata": { 35 | "category": "[variables('policyCategory')]" 36 | }, 37 | "parameters": { 38 | "routeTableSettings": { 39 | "type": "Object", 40 | "metadata": { 41 | "displayName": "Route Table Settings", 42 | "description": "Location-specific settings for route tables." 43 | } 44 | } 45 | }, 46 | "policyRule": { 47 | "if": { 48 | "anyOf": [ 49 | { 50 | "allOf": [ 51 | { 52 | "field": "type", 53 | "equals": "Microsoft.Network/routeTables" 54 | }, 55 | { 56 | "count": { 57 | "field": "Microsoft.Network/routeTables/routes[*]", 58 | "where": { 59 | "allOf": [ 60 | { 61 | "field": "Microsoft.Network/routeTables/routes[*].addressPrefix", 62 | "equals": "0.0.0.0/0" 63 | }, 64 | { 65 | "anyOf": [ 66 | { 67 | "field": "Microsoft.Network/routeTables/routes[*].nextHopType", 68 | "notEquals": "VirtualAppliance" 69 | }, 70 | { 71 | "field": "Microsoft.Network/routeTables/routes[*].nextHopIpAddress", 72 | "notEquals": "[[parameters('routeTableSettings')[field('location')].virtualApplianceIpAddress]" 73 | } 74 | ] 75 | } 76 | ] 77 | } 78 | }, 79 | "greater": 0 80 | } 81 | ] 82 | }, 83 | { 84 | "allOf": [ 85 | { 86 | "field": "type", 87 | "equals": "Microsoft.Network/routeTables/routes" 88 | }, 89 | { 90 | "field": "Microsoft.Network/routeTables/routes/addressPrefix", 91 | "equals": "0.0.0.0/0" 92 | }, 93 | { 94 | "anyOf": [ 95 | { 96 | "field": "Microsoft.Network/routeTables/routes/nextHopType", 97 | "notEquals": "VirtualAppliance" 98 | }, 99 | { 100 | "field": "Microsoft.Network/routeTables/routes/nextHopIpAddress", 101 | "notEquals": "[[parameters('routeTableSettings')[field('location')].virtualApplianceIpAddress]" 102 | } 103 | ] 104 | } 105 | ] 106 | } 107 | ] 108 | }, 109 | "then": { 110 | "effect": "deny" 111 | } 112 | } 113 | } 114 | }, 115 | { 116 | "name": "[uniqueString(variables('policyName'))]", 117 | "type": "Microsoft.Authorization/policyAssignments", 118 | "apiVersion": "2020-03-01", 119 | "properties": { 120 | "displayName": "[variables('policyName')]", 121 | "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions', variables('policyName'))]", 122 | "parameters": "[variables('policyAssignmentParameters')]", 123 | "description": "[variables('policyDescription')]", 124 | "metadata": { 125 | "category": "[variables('policyCategory')]" 126 | }, 127 | "enforcementMode": "Default" 128 | }, 129 | "dependsOn": [ 130 | "[resourceId('Microsoft.Authorization/policyDefinitions', variables('policyName'))]" 131 | ] 132 | } 133 | ] 134 | } -------------------------------------------------------------------------------- /utils/Test.Utils.psm1: -------------------------------------------------------------------------------- 1 | Import-Module -Name Az.Resources 2 | 3 | <# 4 | .SYNOPSIS 5 | Cleans up any Azure resources created during the test. 6 | 7 | .DESCRIPTION 8 | Cleans up any Azure resources created during the test. If any clean-up operation fails, the whole test will fail. 9 | 10 | .PARAMETER CleanUp 11 | The script block specifying the clean-up operations. 12 | 13 | .EXAMPLE 14 | AzCleanUp { 15 | Remove-AzResourceGroup -Name $ResourceGroup.ResourceGroupName -Force 16 | } 17 | #> 18 | function AzCleanUp { 19 | param ( 20 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 21 | [ValidateNotNull()] 22 | [ScriptBlock] $CleanUp 23 | ) 24 | 25 | try { 26 | # Remember $ErrorActionPreference. 27 | $errorAction = $ErrorActionPreference 28 | 29 | # Stop clean-up on errors, since $ErrorActionPreference defaults to 'Continue' in PowerShell. 30 | $ErrorActionPreference = "Stop" 31 | 32 | # Execute clean-up script. 33 | $CleanUp.Invoke() 34 | 35 | # Reset $ErrorActionPreference to previous value. 36 | $ErrorActionPreference = $errorAction 37 | } 38 | catch { 39 | throw "Clean-up failed with message: '$($_)'" 40 | } 41 | } 42 | 43 | <# 44 | .SYNOPSIS 45 | Retries the test on transient errors. 46 | 47 | .DESCRIPTION 48 | Retries the script block when a transient errors occurs during test execution. 49 | 50 | .PARAMETER Retry 51 | The script block specifying the test. 52 | 53 | .PARAMETER MaxRetries 54 | The maximum amount of retries in case of transient errors (Default: 3 times). 55 | 56 | .EXAMPLE 57 | AzRetry { 58 | # When a dedicated resource group should be created for the test 59 | if ($ResourceGroup) { 60 | try { 61 | $resourceGroup = New-ResourceGroupTest 62 | Invoke-Command -ScriptBlock $Test -ArgumentList $resourceGroup 63 | } 64 | finally { 65 | # Stops on failures during clean-up 66 | CleanUp { 67 | Remove-AzResourceGroup -Name $ResourceGroup.ResourceGroupName -Force -AsJob 68 | } 69 | } 70 | } 71 | else { 72 | Invoke-Command -ScriptBlock $Test 73 | } 74 | } 75 | #> 76 | function AzRetry { 77 | param ( 78 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 79 | [ValidateNotNull()] 80 | [ScriptBlock] $Retry, 81 | [Parameter()] 82 | [ValidateRange(1, [ushort]::MaxValue)] 83 | [ushort]$MaxRetries = 3 84 | ) 85 | 86 | $retries = 0 87 | do { 88 | try { 89 | $Retry.Invoke() 90 | 91 | # Exit loop when no exception was thrown. 92 | break 93 | } 94 | catch { 95 | # Determine root cause exception. 96 | $innermostException = Get-InnermostException $_.Exception 97 | 98 | # Rethrow exception when maximum retries are reached. 99 | if ($retries -ge $MaxRetries) { 100 | throw (New-Object System.Management.Automation.RuntimeException("Test failed even after $($MaxRetries) retries.", $_.Exception)) 101 | } 102 | # Retry when exception is caused by a transient error. 103 | elseif ($innermostException -is [System.Threading.Tasks.TaskCanceledException]) { 104 | Write-Host "Test failed due to a transient error. Retrying..." 105 | $retries++ 106 | continue 107 | } 108 | # Rethrow exception when it is caused by a non-transient error. 109 | else { 110 | throw $_.Exception 111 | } 112 | } 113 | } while ($retries -le $MaxRetries) # Prevent endless loop, just defensive programming. 114 | } 115 | 116 | <# 117 | .SYNOPSIS 118 | Wraps a test targeting Azure. 119 | 120 | .DESCRIPTION 121 | Wraps a test targeting Azure. Also retries the test on transient errors. 122 | 123 | .PARAMETER Test 124 | The script block specifying the test. 125 | 126 | .PARAMETER ResourceGroup 127 | Creates a dedicated resource group for the test, which is automatically cleaned up afterwards. 128 | 129 | .EXAMPLE 130 | AzTest -ResourceGroup { 131 | param($ResourceGroup) 132 | 133 | # Your test code leveraging the resource group, which is automatically cleaned up afterwards. 134 | } 135 | 136 | .EXAMPLE 137 | AzTest { 138 | try { 139 | # Your test code 140 | } 141 | finally { 142 | # Don't forget to wrap your clean-up operations in AzCleanUp, otherwise failures during clean-up might remain unnoticed. 143 | AzCleanUp { 144 | # Your clean-up code 145 | } 146 | } 147 | } 148 | #> 149 | function AzTest { 150 | param ( 151 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 152 | [ValidateNotNull()] 153 | [ScriptBlock] $Test, 154 | [Parameter()] 155 | [Switch] $ResourceGroup 156 | ) 157 | 158 | # Retries the test on transient errors. 159 | AzRetry { 160 | # When a dedicated resource group should be created for the test. 161 | if ($ResourceGroup) { 162 | try { 163 | $resourceGroup = New-ResourceGroupTest 164 | Invoke-Command -ScriptBlock $Test -ArgumentList $resourceGroup 165 | } 166 | finally { 167 | # Stops on failures during clean-up. 168 | AzCleanUp { 169 | Remove-AzResourceGroup -Name $ResourceGroup.ResourceGroupName -Force -AsJob 170 | } 171 | } 172 | } 173 | else { 174 | Invoke-Command -ScriptBlock $Test 175 | } 176 | } 177 | } 178 | 179 | <# 180 | .SYNOPSIS 181 | Gets the innermost exception. 182 | 183 | .DESCRIPTION 184 | Gets the innermost exception or root cause. 185 | 186 | .PARAMETER Exception 187 | The exception. 188 | 189 | .EXAMPLE 190 | $innermostException = Get-InnermostException $_.Exception 191 | 192 | .EXAMPLE 193 | $innermostException = Get-InnermostException -Exception $_.Exception 194 | #> 195 | function Get-InnermostException { 196 | param ( 197 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 198 | [ValidateNotNull()] 199 | [System.Exception] $Exception 200 | ) 201 | 202 | # Innermost exceptions do not have an inner exception. 203 | if ($null -eq $Exception.InnerException) { 204 | return $Exception 205 | } 206 | else { 207 | return Get-InnermostException $Exception.InnerException 208 | } 209 | } 210 | 211 | <# 212 | .SYNOPSIS 213 | Gets the default Azure region. 214 | 215 | .DESCRIPTION 216 | Gets the default Azure region, e.g. northeurope. 217 | 218 | .EXAMPLE 219 | $location = Get-ResourceLocationDefault 220 | #> 221 | function Get-ResourceLocationDefault { 222 | return "northeurope" 223 | } 224 | 225 | <# 226 | .SYNOPSIS 227 | Create a dedicated resource group for an automated test case. 228 | 229 | .DESCRIPTION 230 | Create a dedicated resource group for an automated test case. The resource group name will be a GUID to avoid naming collisions. 231 | 232 | .PARAMETER Location 233 | The Azure region where the resource group is created, e.g. northeurope. When no location is provided, the default location is retrieved by using Get-ResourceLocationDefault. 234 | 235 | .EXAMPLE 236 | $resourceGroup = New-ResourceGroupTest 237 | 238 | .EXAMPLE 239 | $resourceGroup = New-ResourceGroupTest -Location "westeurope" 240 | #> 241 | function New-ResourceGroupTest { 242 | param ( 243 | [Parameter()] 244 | [ValidateNotNullOrEmpty()] 245 | [string]$Location = (Get-ResourceLocationDefault) 246 | ) 247 | 248 | $resourceGroup = New-AzResourceGroup -Name (New-Guid).Guid -Location $Location 249 | return $resourceGroup 250 | } -------------------------------------------------------------------------------- /tests/Deny-Route-NextHopVirtualAppliance.Tests.ps1: -------------------------------------------------------------------------------- 1 | Import-Module -Name Az.Network 2 | Import-Module -Name Az.Resources 3 | Import-Module "$($PSScriptRoot)/../utils/Policy.Utils.psm1" -Force 4 | Import-Module "$($PSScriptRoot)/../utils/Rest.Utils.psm1" -Force 5 | Import-Module "$($PSScriptRoot)/../utils/RouteTable.Utils.psm1" -Force 6 | Import-Module "$($PSScriptRoot)/../utils/Test.Utils.psm1" -Force 7 | 8 | Describe "Testing policy 'Deny-Route-NextHopVirtualAppliance'" -Tag "deny-route-nexthopvirtualappliance" { 9 | # Create or update route is actually the same PUT request, hence testing create covers update as well. 10 | # PATCH requests are currently not supported in Network Resource Provider. 11 | # See also: https://docs.microsoft.com/en-us/rest/api/virtualnetwork/routes/createorupdate 12 | Context "When route is created or updated" -Tag "deny-route-nexthopvirtualappliance-route-create-update" { 13 | It "Should deny incompliant route 0.0.0.0/0 with next hop type 'None'" -Tag "deny-route-nexthopvirtualappliance-route-create-update-10" { 14 | AzTest -ResourceGroup { 15 | param($ResourceGroup) 16 | 17 | $routeTable = New-AzRouteTable ` 18 | -Name "route-table" ` 19 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 20 | -Location $ResourceGroup.Location 21 | 22 | # Should be disallowed by policy, so exception should be thrown. 23 | { 24 | # Directly calling REST API with PUT routes, since New-AzRouteConfig/Set-AzRouteTable will issue PUT routeTables. 25 | $routeTable | Invoke-RoutePut ` 26 | -Name "default" ` 27 | -AddressPrefix "0.0.0.0/0" ` 28 | -NextHopType "None" # Incompliant. 29 | } | Should -Throw "*RequestDisallowedByPolicy*Deny-Route-NextHopVirtualAppliance*" 30 | } 31 | } 32 | 33 | It "Should deny incompliant route 0.0.0.0/0 with next hop IP address '10.10.10.10'" -Tag "deny-route-nexthopvirtualappliance-route-create-update-20" { 34 | AzTest -ResourceGroup { 35 | param($ResourceGroup) 36 | 37 | $routeTable = New-AzRouteTable ` 38 | -Name "route-table" ` 39 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 40 | -Location $ResourceGroup.Location 41 | 42 | # Should be disallowed by policy, so exception should be thrown. 43 | { 44 | # Directly calling REST API with PUT routes, since New-AzRouteConfig/Set-AzRouteTable will issue PUT routeTables. 45 | $routeTable | Invoke-RoutePut ` 46 | -Name "default" ` 47 | -AddressPrefix "0.0.0.0/0" ` 48 | -NextHopType "VirtualAppliance" ` 49 | -NextHopIpAddress "10.10.10.10" # Incompliant. 50 | } | Should -Throw "*RequestDisallowedByPolicy*Deny-Route-NextHopVirtualAppliance*" 51 | } 52 | } 53 | 54 | It "Should allow compliant route route 0.0.0.0/0" -Tag "deny-route-nexthopvirtualappliance-route-create-update-30" { 55 | AzTest -ResourceGroup { 56 | param($ResourceGroup) 57 | 58 | $routeTable = New-AzRouteTable ` 59 | -Name "route-table" ` 60 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 61 | -Location $ResourceGroup.Location 62 | 63 | # Should be allowed by policy, so no exception should be thrown. 64 | { 65 | # Directly calling REST API with PUT routes, since New-AzRouteConfig/Set-AzRouteTable will issue PUT routeTables. 66 | $routeTable | Invoke-RoutePut ` 67 | -Name "default" ` 68 | -AddressPrefix "0.0.0.0/0" ` 69 | -NextHopType "VirtualAppliance" ` 70 | -NextHopIpAddress (Get-VirtualApplianceIpAddress -Location $routeTable.Location) # Compliant. 71 | } | Should -Not -Throw 72 | } 73 | } 74 | } 75 | 76 | # Create or update route tables is actually the same PUT request, hence testing create covers update as well. 77 | # PATCH requests are currently not supported in Network Resource Provider. 78 | # See also: https://docs.microsoft.com/en-us/rest/api/virtualnetwork/routetables/createorupdate 79 | Context "When route table is created or updated" -Tag "deny-route-nexthopvirtualappliance-routetable-create-update" { 80 | It "Should deny route table containing incompliant route 0.0.0.0/0 with next hop type 'None'" -Tag "deny-route-nexthopvirtualappliance-routetable-create-update-10" { 81 | AzTest -ResourceGroup { 82 | param($ResourceGroup) 83 | 84 | # Create route table. 85 | $route = New-AzRouteConfig ` 86 | -Name "virtual-appliance" ` 87 | -AddressPrefix "0.0.0.0/0" ` 88 | -NextHopType "None" # Incompliant. 89 | 90 | # Should be disallowed by policy, so exception should be thrown. 91 | { 92 | New-AzRouteTable ` 93 | -Name "route-table" ` 94 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 95 | -Location $ResourceGroup.Location ` 96 | -Route $route ` 97 | -ErrorAction Stop # Otherwise no exception would be thrown, since $ErrorActionPreference defaults to 'Continue' in PowerShell. 98 | } | Should -Throw "*RequestDisallowedByPolicy*Deny-Route-NextHopVirtualAppliance*" 99 | } 100 | } 101 | 102 | It "Should deny route table containing incompliant route 0.0.0.0/0 with next hop IP address '10.10.10.10'" -Tag "deny-route-nexthopvirtualappliance-routetable-create-update-20" { 103 | AzTest -ResourceGroup { 104 | param($ResourceGroup) 105 | 106 | # Create route table. 107 | $route = New-AzRouteConfig ` 108 | -Name "virtual-appliance" ` 109 | -AddressPrefix "0.0.0.0/0" ` 110 | -NextHopType "VirtualAppliance" ` 111 | -NextHopIpAddress "10.10.10.10" # Incompliant. 112 | 113 | # Should be disallowed by policy, so exception should be thrown. 114 | { 115 | New-AzRouteTable ` 116 | -Name "route-table" ` 117 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 118 | -Location $ResourceGroup.Location ` 119 | -Route $route ` 120 | -ErrorAction Stop # Otherwise no exception would be thrown, since $ErrorActionPreference defaults to 'Continue' in PowerShell. 121 | } | Should -Throw "*RequestDisallowedByPolicy*Deny-Route-NextHopVirtualAppliance*" 122 | } 123 | } 124 | 125 | It "Should allow route table containing compliant route 0.0.0.0/0" -Tag "deny-route-nexthopvirtualappliance-route-routetable-update-30" { 126 | AzTest -ResourceGroup { 127 | param($ResourceGroup) 128 | 129 | # Create route table 130 | $route = New-AzRouteConfig ` 131 | -Name "virtual-appliance" ` 132 | -AddressPrefix "0.0.0.0/0" ` 133 | -NextHopType "VirtualAppliance" ` 134 | -NextHopIpAddress (Get-VirtualApplianceIpAddress -Location $ResourceGroup.Location) # Compliant. 135 | 136 | # Should be allowed by policy, so no exception should be thrown. 137 | { 138 | New-AzRouteTable ` 139 | -Name "route-table" ` 140 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 141 | -Location $ResourceGroup.Location ` 142 | -Route $route ` 143 | -ErrorAction Stop # Otherwise no exception would be thrown, since $ErrorActionPreference defaults to 'Continue' in PowerShell. 144 | } | Should -Not -Throw 145 | } 146 | } 147 | } 148 | } -------------------------------------------------------------------------------- /utils/RouteTable.Utils.psm1: -------------------------------------------------------------------------------- 1 | Import-Module -Name Az.Network 2 | Import-Module -Name Az.Resources 3 | Import-Module "$($PSScriptRoot)/Rest.Utils.psm1" -Force 4 | Import-Module "$($PSScriptRoot)/Test.Utils.psm1" -Force 5 | 6 | <# 7 | .SYNOPSIS 8 | Gets route 0.0.0.0/0 pointing to the virtual appliance. 9 | 10 | .DESCRIPTION 11 | Gets route 0.0.0.0/0 pointing to the virtual appliance which is provisioned as part of the landing zone. 12 | 13 | .PARAMETER RouteTable 14 | The route table containing the route 0.0.0.0/0 pointing to the virtual appliance. 15 | 16 | .EXAMPLE 17 | $route = $RouteTable | Get-RouteNextHopVirtualAppliance 18 | 19 | .EXAMPLE 20 | $route = Get-RouteNextHopVirtualAppliance -RouteTable $routeTable 21 | #> 22 | function Get-RouteNextHopVirtualAppliance { 23 | param ( 24 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 25 | [ValidateNotNull()] 26 | [Microsoft.Azure.Commands.Network.Models.PSRouteTable]$RouteTable 27 | ) 28 | 29 | $nextHopIpAddress = Get-VirtualApplianceIpAddress -Location $RouteTable.Location 30 | 31 | $route = $RouteTable.Routes | Where-Object { 32 | ($_.AddressPrefix -eq "0.0.0.0/0") -and 33 | ($_.NextHopType -eq "VirtualAppliance") -and 34 | ($_.NextHopIpAddress -eq $nextHopIpAddress) 35 | } | Select-Object -First 1 # Address prefixes are unique within a route table. 36 | 37 | return $route 38 | } 39 | 40 | <# 41 | .SYNOPSIS 42 | Gets the IP address of the virtual appliance. 43 | 44 | .DESCRIPTION 45 | Gets the IP address of the virtual appliance for the respective Azure region. The test environment is based on a hub/spoke network topology, with a virtual appliance deployed in each hub and each hub provisioned per Azure region. When no location is provided, the default location is retrieved by using Get-ResourceLocationDefault. 46 | 47 | .PARAMETER Location 48 | The Azure region where the virtual appliance is deployed to, e.g. northeurope. 49 | 50 | .EXAMPLE 51 | $nextHopIpAddress = Get-VirtualApplianceIpAddress -Location $RouteTable.Location 52 | 53 | .EXAMPLE 54 | $nextHopIpAddress = Get-VirtualApplianceIpAddress 55 | #> 56 | function Get-VirtualApplianceIpAddress { 57 | param ( 58 | [Parameter()] 59 | [ValidateNotNullOrEmpty()] 60 | [string]$Location = (Get-ResourceLocationDefault) 61 | ) 62 | 63 | $virtualApplianceIpAddress > $null 64 | switch ($Location) { 65 | "northeurope" { $virtualApplianceIpAddress = "10.0.0.23"; break } 66 | "westeurope" { $virtualApplianceIpAddress = "10.1.0.23"; break } 67 | default { throw "Location '$($Location)' not handled." } 68 | } 69 | 70 | return $virtualApplianceIpAddress 71 | } 72 | 73 | <# 74 | .SYNOPSIS 75 | Deletes a route in a route table. 76 | 77 | .DESCRIPTION 78 | Deletes a route in a route table by directly invoking the Azure REST API. 79 | 80 | .PARAMETER RouteTable 81 | The route table containing the route to be deleted. 82 | 83 | .PARAMETER Route 84 | The route to be deleted. 85 | 86 | .EXAMPLE 87 | $routeTable | Invoke-RouteDelete -Route $route 88 | 89 | .LINK 90 | https://docs.microsoft.com/en-us/rest/api/virtualnetwork/routes/delete 91 | #> 92 | function Invoke-RouteDelete { 93 | param ( 94 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 95 | [ValidateNotNull()] 96 | [Microsoft.Azure.Commands.Network.Models.PSRouteTable]$RouteTable, 97 | [Parameter(Mandatory = $true)] 98 | [ValidateNotNull()] 99 | [Microsoft.Azure.Commands.Network.Models.PSRoute]$Route 100 | ) 101 | 102 | $httpResponse = Invoke-AzRestMethod ` 103 | -ResourceGroupName $RouteTable.ResourceGroupName ` 104 | -ResourceProviderName "Microsoft.Network" ` 105 | -ResourceType @("routeTables", "routes") ` 106 | -Name @($RouteTable.Name, $Route.Name) ` 107 | -ApiVersion "2020-05-01" ` 108 | -Method "DELETE" 109 | 110 | # Handling the HTTP status codes returned by the DELETE request for route 111 | # See also: https://docs.microsoft.com/en-us/rest/api/virtualnetwork/routes/delete 112 | # Accepted. 113 | if ($httpResponse.StatusCode -eq 200) { 114 | # All good, do nothing 115 | } 116 | # Accepted and the operation will complete asynchronously. 117 | elseif ($httpResponse.StatusCode -eq 202) { 118 | # Invoke-AzRestMethod currently does not support awaiting asynchronous operations 119 | # See also: https://github.com/Azure/azure-powershell/issues/13293 120 | $asyncOperation = $httpResponse | Wait-AsyncOperation 121 | if ($asyncOperation.Status -ne "Succeeded") { 122 | throw "Asynchronous operation failed with message: '$($asyncOperation)'" 123 | } 124 | } 125 | # Route was deleted or not found. 126 | elseif ($httpResponse.StatusCode -eq 204) { 127 | # All good, do nothing. 128 | } 129 | # Error response describing why the operation failed. 130 | else { 131 | throw "Operation failed with message: '$($httpResponse.Content)'" 132 | } 133 | } 134 | 135 | <# 136 | .SYNOPSIS 137 | Creates or updates a route in a route table. 138 | 139 | .DESCRIPTION 140 | Creates or updates a route in a route table by directly invoking the Azure REST API. 141 | 142 | .PARAMETER RouteTable 143 | The route table containing the route to be created or updated. 144 | 145 | .PARAMETER Name 146 | The name of the route. 147 | 148 | .PARAMETER AddressPrefix 149 | The destination CIDR to which the route applies. 150 | 151 | .PARAMETER NextHopType 152 | The type of Azure hop the packet should be sent to. 153 | 154 | .PARAMETER NextHopIpAddress 155 | The IP address packets should be forwarded to. Next hop values are only allowed in routes where the next hop type is VirtualAppliance. 156 | 157 | .EXAMPLE 158 | $routeTable | Invoke-RoutePut -Name "container-registry" -AddressPrefix "13.69.227.80/29" -NextHopType "Internet" 159 | 160 | .LINK 161 | https://docs.microsoft.com/en-us/rest/api/virtualnetwork/routes/createorupdate 162 | #> 163 | function Invoke-RoutePut { 164 | param ( 165 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 166 | [ValidateNotNull()] 167 | [Microsoft.Azure.Commands.Network.Models.PSRouteTable]$RouteTable, 168 | [Parameter(Mandatory = $true)] 169 | [ValidateNotNullOrEmpty()] 170 | [string]$Name, 171 | [Parameter(Mandatory = $true)] 172 | [ValidateNotNullOrEmpty()] 173 | [string]$AddressPrefix, 174 | [Parameter(Mandatory = $true)] 175 | [ValidateNotNullOrEmpty()] 176 | [string]$NextHopType, 177 | [Parameter()] 178 | [ValidateNotNullOrEmpty()] 179 | [string]$NextHopIpAddress 180 | ) 181 | 182 | $payload = @" 183 | { 184 | "properties": { 185 | "addressPrefix": "$($AddressPrefix)", 186 | "nextHopType": "$($NextHopType)", 187 | "nextHopIpAddress": "$($NextHopIpAddress)" 188 | } 189 | } 190 | "@ 191 | 192 | $httpResponse = Invoke-AzRestMethod ` 193 | -ResourceGroupName $RouteTable.ResourceGroupName ` 194 | -ResourceProviderName "Microsoft.Network" ` 195 | -ResourceType @("routeTables", "routes") ` 196 | -Name @($RouteTable.Name, $Name) ` 197 | -ApiVersion "2020-05-01" ` 198 | -Method "PUT" ` 199 | -Payload $payload 200 | 201 | # Handling the HTTP status codes returned by the PUT request for route. 202 | # See also: https://docs.microsoft.com/en-us/rest/api/virtualnetwork/routes/createorupdate 203 | # Update successful. The operation returns the resulting Route resource. 204 | if ($httpResponse.StatusCode -eq 200) { 205 | # All good, do nothing. 206 | } 207 | # Create successful. The operation returns the resulting Route resource. 208 | elseif ($httpResponse.StatusCode -eq 201) { 209 | # Invoke-AzRestMethod currently does not support awaiting asynchronous operations. 210 | # See also: https://github.com/Azure/azure-powershell/issues/13293 211 | $asyncOperation = $httpResponse | Wait-AsyncOperation 212 | if ($asyncOperation.Status -ne "Succeeded") { 213 | throw "Asynchronous operation failed with message: '$($asyncOperation)'" 214 | } 215 | } 216 | # Error response describing why the operation failed. 217 | else { 218 | throw "Operation failed with message: '$($httpResponse.Content)'" 219 | } 220 | } 221 | 222 | <# 223 | .SYNOPSIS 224 | Tests whether a route table contains the route 0.0.0.0/0 pointing to the virtual appliance. 225 | 226 | .DESCRIPTION 227 | Tests whether a route table contains the route 0.0.0.0/0 pointing to the virtual appliance, which is provisioned as part of the landing zone. 228 | 229 | .PARAMETER RouteTable 230 | The route table to be tested. 231 | 232 | .EXAMPLE 233 | $routeTable | Test-RouteNextHopVirtualAppliance | Should -BeTrue 234 | #> 235 | function Test-RouteNextHopVirtualAppliance { 236 | param ( 237 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 238 | [ValidateNotNull()] 239 | [Microsoft.Azure.Commands.Network.Models.PSRouteTable]$RouteTable 240 | ) 241 | 242 | $route = Get-RouteNextHopVirtualAppliance -RouteTable $RouteTable 243 | 244 | return $null -ne $route 245 | } -------------------------------------------------------------------------------- /utils/Policy.Utils.psm1: -------------------------------------------------------------------------------- 1 | Import-Module -Name Az.Resources 2 | 3 | <# 4 | .SYNOPSIS 5 | Completes a policy compliance scan. 6 | 7 | .DESCRIPTION 8 | Starts a policy compliance scan and awaits it's completion. In case of a failure, the policy compliance scan is retried (Default: 3 times). 9 | 10 | .PARAMETER ResourceGroup 11 | The resource group to be scanned for policy compliance. 12 | 13 | .PARAMETER MaxRetries 14 | The maximum amount of retries in case of failures (Default: 3 times). 15 | 16 | .EXAMPLE 17 | $ResourceGroup | Complete-PolicyComplianceScan 18 | #> 19 | function Complete-PolicyComplianceScan { 20 | param ( 21 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 22 | [ValidateNotNull()] 23 | [Microsoft.Azure.Commands.ResourceManager.Cmdlets.SdkModels.PSResourceGroup]$ResourceGroup, 24 | [Parameter()] 25 | [ValidateRange(1, [ushort]::MaxValue)] 26 | [ushort]$MaxRetries = 3 27 | ) 28 | 29 | # Policy compliance scan might fail, hence retrying to avoid flaky tests. 30 | $retries = 0 31 | do { 32 | $job = Start-AzPolicyComplianceScan -ResourceGroupName $ResourceGroup.ResourceGroupName -PassThru -AsJob 33 | $succeeded = $job | Wait-Job | Receive-Job 34 | 35 | if ($succeeded) { 36 | break 37 | } 38 | # Failure: Retry policy compliance scan when still below maximum retries. 39 | elseif ($retries -le $MaxRetries) { 40 | Write-Host "Policy compliance scan for resource group '$($ResourceGroup.ResourceId)' failed. Retrying..." 41 | $retries++ 42 | continue # Not required, just defensive programming. 43 | } 44 | # Failure: Policy compliance scan is still failing after maximum retries. 45 | else { 46 | throw "Policy compliance scan for resource group '$($ResourceGroup.ResourceId)' failed even after $($MaxRetries) retries." 47 | } 48 | } while ($retries -le $MaxRetries) # Prevent endless loop, just defensive programming. 49 | } 50 | 51 | <# 52 | .SYNOPSIS 53 | Completes a policy remediation. 54 | 55 | .DESCRIPTION 56 | Starts a remediation for a policy and awaits it's completion. In case of a failure, the policy remediation is retried (Default: 3 times). 57 | 58 | .PARAMETER Resource 59 | The resource to be remediated. 60 | 61 | .PARAMETER PolicyDefinitionName 62 | The name of the policy definition. 63 | 64 | .PARAMETER CheckDeployment 65 | The switch to determine if a deployment is expected. If a deployment is expected but did not happen during policy remediation, the policy remediation is retried. 66 | 67 | .PARAMETER MaxRetries 68 | The maximum amount of retries in case of failures (Default: 3 times). 69 | 70 | .EXAMPLE 71 | $routeTable | Complete-PolicyRemediation -PolicyDefinition "Modify-RouteTable-NextHopVirtualAppliance" -CheckDeployment 72 | #> 73 | function Complete-PolicyRemediation { 74 | param ( 75 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 76 | [ValidateNotNull()] 77 | [Microsoft.Azure.Commands.Network.Models.PSChildResource]$Resource, 78 | [Parameter(Mandatory = $true)] 79 | [ValidateNotNullOrEmpty()] 80 | [string]$PolicyDefinitionName, 81 | [Parameter()] 82 | [switch]$CheckDeployment, 83 | [Parameter()] 84 | [ValidateRange(1, [ushort]::MaxValue)] 85 | [ushort]$MaxRetries = 3 86 | ) 87 | 88 | # Determine policy assignment id- 89 | $scope = "/subscriptions/$((Get-AzContext).Subscription.Id)" 90 | $policyAssignmentId = (Get-AzPolicyAssignment -Scope $scope 91 | | Select-Object -Property PolicyAssignmentId -ExpandProperty Properties 92 | | Where-Object { $_.PolicyDefinitionId.EndsWith($PolicyDefinitionName) } 93 | | Select-Object -Property PolicyAssignmentId -First 1 94 | ).PolicyAssignmentId 95 | 96 | if ($null -eq $policyAssignmentId) { 97 | throw "Policy '$($PolicyDefinitionName)' is not assigned to scope '$($scope)'." 98 | } 99 | 100 | # Remediation might be started before all previous changes on the resource in scope are completed. 101 | # This race condition could lead to a successful remediation without any deployment being triggered. 102 | # When a deployment is expected, it might be required to retry remediation to avoid flaky tests. 103 | $retries = 0 104 | do { 105 | # Trigger and wait for remediation. 106 | $job = Start-AzPolicyRemediation ` 107 | -Name "$($Resource.Name)-$([DateTimeOffset]::Now.ToUnixTimeSeconds())" ` 108 | -Scope $Resource.Id ` 109 | -PolicyAssignmentId $policyAssignmentId ` 110 | -ResourceDiscoveryMode ReEvaluateCompliance ` 111 | -AsJob 112 | $remediation = $job | Wait-Job | Receive-Job 113 | 114 | # Check remediation provisioning state and deployment when required . 115 | $succeeded = $remediation.ProvisioningState -eq "Succeeded" 116 | if ($succeeded) { 117 | if ($CheckDeployment) { 118 | $deployed = $remediation.DeploymentSummary.TotalDeployments -gt 0 119 | 120 | # Success: Deployment was triggered. 121 | if ($deployed) { 122 | break 123 | } 124 | # Failure: No deployment was triggered, so retry when still below maximum retries. 125 | elseif ($retries -le $MaxRetries) { 126 | Write-Host "Policy '$($PolicyDefinitionName)' succeeded to remediated resource '$($Resource.Id)', but no deployment was triggered. Retrying..." 127 | $retries++ 128 | continue # Not required, just defensive programming. 129 | } 130 | # Failure: No deployment was triggered even after maximum retries. 131 | else { 132 | throw "Policy '$($PolicyDefinitionName)' succeeded to remediated resource '$($Resource.Id)', but no deployment was triggered even after $($MaxRetries) retries." 133 | } 134 | } 135 | # Success: No deployment need to checked, hence no retry required. 136 | else { 137 | break 138 | } 139 | } 140 | # Failure: Remediation failed, so retry when still below maximum retries. 141 | elseif ($retries -le $MaxRetries) { 142 | Write-Host "Policy '$($PolicyDefinitionName)' failed to remediate resource '$($Resource.Id)'. Retrying..." 143 | $retries++ 144 | continue # Not required, just defensive programming. 145 | } 146 | # Failure: Remediation failed even after maximum retries. 147 | else { 148 | throw "Policy '$($PolicyDefinitionName)' failed to remediate resource '$($Resource.Id)' even after $($MaxRetries) retries." 149 | } 150 | } while ($retries -le $MaxRetries) # Prevent endless loop, just defensive programming. 151 | } 152 | 153 | <# 154 | .SYNOPSIS 155 | Gets the policy compliance state of a resource. 156 | 157 | .DESCRIPTION 158 | Gets the policy compliance state of a resource. In case of a failure, getting the policy compliance state is retried (Default: 30 times) after a few seconds of waiting (Default: 60s). 159 | 160 | .PARAMETER Resource 161 | The resource to get the policy compliance state for. 162 | 163 | .PARAMETER PolicyDefinitionName 164 | The name of the policy definition. 165 | 166 | .PARAMETER WaitSeconds 167 | The duration in seconds to wait between retries in case of failures (Default: 60s). 168 | 169 | .PARAMETER MaxRetries 170 | The maximum amount of retries in case of failures (Default: 3 times). 171 | 172 | .EXAMPLE 173 | $networkSecurityGroup | Get-PolicyComplianceState -PolicyDefinition "OP-Audit-NSGAny" | Should -BeFalse 174 | #> 175 | function Get-PolicyComplianceState { 176 | param ( 177 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 178 | [ValidateNotNull()] 179 | [Microsoft.Azure.Commands.Network.Models.PSChildResource]$Resource, 180 | [Parameter(Mandatory = $true)] 181 | [ValidateNotNullOrEmpty()] 182 | [string]$PolicyDefinitionName, 183 | [Parameter()] 184 | [ValidateRange(1, [ushort]::MaxValue)] 185 | [ushort]$WaitSeconds = 60, 186 | [Parameter()] 187 | [ValidateRange(1, [ushort]::MaxValue)] 188 | [ushort]$MaxRetries = 30 189 | ) 190 | 191 | # Policy compliance scan might be completed, but policy compliance state might still be null due to race conditions. 192 | # Hence waiting a few seconds and retrying to get the policy compliance state to avoid flaky tests. 193 | $retries = 0 194 | do { 195 | $isCompliant = (Get-AzPolicyState ` 196 | -PolicyDefinitionName $PolicyDefinitionName ` 197 | -Filter "ResourceId eq '$($Resource.Id)'" ` 198 | ).IsCompliant 199 | 200 | # Success: Policy compliance state is not null. 201 | if ($null -ne $isCompliant) { 202 | break 203 | } 204 | # Failure: Policy compliance state is null, so wait a few seconds and retry when still below maximum retries. 205 | elseif ($retries -le $MaxRetries) { 206 | Write-Host "Policy '$($PolicyDefinitionName)' completed compliance scan for resource '$($Resource.Id)', but policy compliance state is null. Retrying..." 207 | Start-Sleep -Seconds $WaitSeconds 208 | $retries++ 209 | continue # Not required, just defensive programming. 210 | } 211 | # Failure: Policy compliance state still null after maximum retries. 212 | else { 213 | throw "Policy '$($PolicyDefinitionName)' completed compliance scan for resource '$($Resource.Id)', but policy compliance state is null even after $($MaxRetries) retries." 214 | } 215 | } while ($retries -le $MaxRetries) # Prevent endless loop, just defensive programming. 216 | 217 | return $isCompliant 218 | } -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![test-policies](https://github.com/fawohlsc/azure-policy-testing/workflows/test-policies/badge.svg) 2 | 3 | # Testing Azure Policy 4 | 5 | ## Introduction 6 | This repository outlines an automated testing approach for Azure Policies. The approach is fundamentally based on behavior-driven development (BDD) to improve communication between developers, security experts and compliance officers. The PowerShell testing framework Pester, Azure PowerShell and GitHub Actions are used in conjunction to automate the tests and run them as part of a DevOps pipeline. After the problem statement, the solution is described in more detail including how to set it up in your Azure environment. 7 | 8 | ## Problem Statement 9 | Let's start simple: Why should you test Azure Policy in the first place? It's just configuration not code. This is a fair statement, but any configuration changes on Azure Policies can be highly impactful and when done wrong even lead to production outages. Just see this example for an Azure Policy, which is quite common for enterprise customers adopting Azure (See: [Hub-spoke network topology in Azure](https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/hybrid-networking/hub-spoke)). In a nutshell, whenever a route table is created, a user-defined route (UDR) should be added to route all internet traffic to a virtual appliance hosted centrally in the hub virtual network for outbound traffic inspection: 10 | 11 | ```json 12 | { 13 | "name": "[variables('policyName')]", 14 | "type": "Microsoft.Authorization/policyDefinitions", 15 | "apiVersion": "2020-03-01", 16 | "properties": { 17 | "policyType": "Custom", 18 | "mode": "All", 19 | "displayName": "[variables('policyName')]", 20 | "description": "[variables('policyDescription')]", 21 | "metadata": { 22 | "category": "[variables('policyCategory')]" 23 | }, 24 | "parameters": { 25 | "routeTableSettings": { 26 | "type": "Object", 27 | "metadata": { 28 | "displayName": "Route Table Settings", 29 | "description": "Location-specific settings for route tables." 30 | } 31 | } 32 | }, 33 | "policyRule": { 34 | "if": { 35 | "allOf": [ 36 | { 37 | "field": "type", 38 | "equals": "Microsoft.Network/routeTables" 39 | }, 40 | { 41 | "count": { 42 | "field": "Microsoft.Network/routeTables/routes[*]", 43 | "where": { 44 | "field": "Microsoft.Network/routeTables/routes[*].addressPrefix", 45 | "equals": "0.0.0.0/0" 46 | } 47 | }, 48 | "equals": 0 49 | } 50 | ] 51 | }, 52 | "then": { 53 | "effect": "modify", 54 | "details": { 55 | "roleDefinitionIds": [ 56 | "[variables('policyRoleDefinitionId')]" 57 | ], 58 | "conflictEffect": "audit", 59 | "operations": [ 60 | { 61 | "operation": "add", 62 | "field": "Microsoft.Network/routeTables/routes[*]", 63 | "value": { 64 | "name": "default", 65 | "properties": { 66 | "addressPrefix": "0.0.0.0/0", 67 | "nextHopType": "VirtualAppliance", 68 | "nextHopIpAddress": "[[parameters('routeTableSettings')[field('location')].virtualApplianceIpAddress]" 69 | } 70 | } 71 | } 72 | ] 73 | } 74 | } 75 | } 76 | } 77 | } 78 | ``` 79 | Now just imagine, we configured the wrong IP addresses for our virtual appliance or we forgot to add this route at all. Depending on the scope of the policy assignment, a lot of applications might stop working. So we should test it, since any change might have an high impact on production environments. 80 | 81 | ## Solution 82 | 83 | ### Test Pyramid 84 | So how can we test Azure Policy? Which kind of tests should we perform? Martin Fowler listed these test categories in his definition of a [test pyramid](https://martinfowler.com/bliki/TestPyramid.html): 85 | 86 | ![Martin Fowler's definition of a test pyramid](./docs/test-pyramid.png) 87 | 88 | **UI tests** typically are recording the interaction of the user with the user interface. When it comes to testing Azure Policy, a lot of people heavily rely on UI Tests using the Azure Portal. Basically, clicking through the Azure Portal and document each step. While this seems fine to get started with testing policies, its also very slow since a human being has to click through the portal and document each step. Since multiple policies can be assigned to the same scope or inherited from parent scopes (Management Group, Subscription and Resource Group), introducing a new policy can lead to regression bugs when overlapping with other existing policies (See: [Layering policy definitions 89 | ](https://docs.microsoft.com/en-us/azure/governance/policy/concepts/effects#layering-policy-definitions)). Just think of an example with two policies overlapping and both of them allowing different Azure regions for deployment - no deployment at all will be possible. Basically, this requires a lot of additional regression testing and further slows down the testing process. So, what about these handy tools for UI automation testing? Both automated and manual UI tests will be hard to maintain, since Azure is rapidly evolving and so is the Azure Portal. Additionally, browser caching issues might lead to false positives during UI testing. To cut a long story short, it is not possible to use manual or automated UI tests to validate Azure Policy in an effective and scalable manner, they are time consuming to run and hard to maintain. 90 | 91 | **Service tests** or **API tests** are actual code targeting the API layer. In the context of Azure Policy, you could actually call the [Azure REST API](https://docs.microsoft.com/en-us/rest/api/azure/) to perform tests, which is a much more stable and versioned contract to test against than the UI. Also, regression testing can be done way easier by just running all the test scripts either manually triggered or even better by performing [continuous integration](https://martinfowler.com/articles/continuousIntegration.html) within your DevOps pipeline of choice, i.e. [GitHub Actions](https://github.com/features/actions). Finally, since the tests are written as code, parallelization techniques can be applied to speed up the tests. Taking into consideration that performing compliance scans and remediation with Azure Policy can take a few minutes per test, parallelization helps to scale the test suite to potentially hundreds of tests. Going forward, we will prefer the term *API tests* instead of *service tests* since much more applicable when testing policies. 92 | 93 | **Unit tests** are the fastest and cheapest way to gain feedback to verify that a single unit of code, e.g. a class or a method, are actually working as expected. Typically, unit tests are focused on testing business logic. Unfortunately, they also require to have the code under test available. This does not apply to Azure Policy, since the policy engine itself is not available to the public. If this ever changes, most of your test suite should become unit tests as indicated by the different sizes for the surface areas in the test pyramid. Additionally, service tests might be used to validate policy remediation, which cannot be done with unit tests. Finally, if UI tests are used at all, they might be just limited to smoke testing. 94 | 95 | In summary, **UI tests** are not suitable for testing Azure Policy at scale and **Unit tests** are not doable without the policy engine being available to the public. Hence, we will focus on testing Azure Policy with **API tests** going forward. 96 | 97 | ### API Tests 98 | For our API Tests, we will use **Azure PowerShell** to call the Azure REST API. Hence the question, why are we not calling the API directly? 99 | 100 | First, Azure PowerShell handles a lot of low-level details which you would be exposed to when directly calling the API. For instance, for long-running operations, the HTTP status code 202 (Accepted) and an URL for the status update to determine when the operation is completed is returned. This basically means that you have to perform [busy waiting](https://en.wikipedia.org/wiki/Busy_waiting) and periodically call the URL for the status update to wait for the operations to complete. Later is important for policy remediation and compliance scans, which can take a few minutes to complete. All this is already handled for you in Azure PowerShell (See: [LongRunningOperationHelper.cs](https://github.com/Azure/azure-powershell/blob/1bcbe7b1f7a3323ac98f7754ba03eeb6b45e79f2/src/Resources/ResourceManager/Components/LongRunningOperationHelper.cs#L139)). Just see this sample code written in Azure PowerShell, which is easy to understand even without a lot of explanations: 101 | 102 | ```powershell 103 | # Create route table. 104 | $routeTable = New-AzRouteTable ` 105 | -Name "route-table" ` 106 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 107 | -Location $ResourceGroup.Location 108 | 109 | # Verify that route 0.0.0.0/0 was added by policy. 110 | $routeTable 111 | | Test-RouteNextHopVirtualAppliance 112 | | Should -BeTrue 113 | ``` 114 | 115 | Second, Azure PowerShell also allows you to fallback and conveniently directly call the Azure REST API when needed: 116 | 117 | ```powershell 118 | $httpResponse = Invoke-AzRestMethod ` 119 | -ResourceGroupName $RouteTable.ResourceGroupName ` 120 | -ResourceProviderName "Microsoft.Network" ` 121 | -ResourceType @("routeTables", "routes") ` 122 | -Name @($RouteTable.Name, $Route.Name) ` 123 | -ApiVersion "2020-05-01" ` 124 | -Method "DELETE" 125 | 126 | # Handling the HTTP status codes returned by the DELETE request for route. 127 | # See also: https://docs.microsoft.com/en-us/rest/api/virtualnetwork/routes/delete 128 | # Accepted. 129 | if ($httpResponse.StatusCode -eq 200) { 130 | # All good, do nothing. 131 | } 132 | # Accepted and the operation will complete asynchronously. 133 | elseif ($httpResponse.StatusCode -eq 202) { 134 | # Invoke-AzRestMethod currently does not support awaiting asynchronous operations. 135 | # See also: https://github.com/Azure/azure-powershell/issues/13293 136 | $asyncOperation = $httpResponse | Wait-AsyncOperation 137 | if ($asyncOperation.Status -ne "Succeeded") { 138 | throw "Asynchronous operation failed with message: '$($asyncOperation)'" 139 | } 140 | } 141 | # Route was deleted or not found. 142 | elseif ($httpResponse.StatusCode -eq 204) { 143 | # All good, do nothing 144 | } 145 | # Error response describing why the operation failed. 146 | else { 147 | throw "Operation failed with message: '$($httpResponse.Content)'" 148 | } 149 | ``` 150 | 151 | Additionally, you can view any underlying HTTP request initiated by Azure PowerShell and the corresponding HTTP response even when using the high-level methods using the ```-Debug``` flag: 152 | 153 | ```powershell 154 | Get-AzResourceGroup -Debug 155 | ``` 156 | ![Detailed output of Azure PowerShell when using -Debug flag](./docs/azure-powershell-debug.png) 157 | 158 | Third, Azure Policy is well supported and documented in Azure PowerShell. Just see this more complex example to trigger a long-running policy remediation including an upfront compliance scan for a policy with [Modify](https://docs.microsoft.com/en-us/azure/governance/policy/concepts/effects#modify) effect: 159 | 160 | ```powershell 161 | # Trigger and wait for remediation. 162 | $job = Start-AzPolicyRemediation ` 163 | -Name "$($Resource.Name)-$([DateTimeOffset]::Now.ToUnixTimeSeconds())" ` 164 | -Scope $Resource.Id ` 165 | -PolicyAssignmentId $policyAssignmentId ` 166 | -ResourceDiscoveryMode ReEvaluateCompliance ` 167 | -AsJob 168 | $remediation = $job | Wait-Job | Receive-Job 169 | 170 | # Check remediation provisioning state and deployment when required. 171 | $succeeded = $remediation.ProvisioningState -eq "Succeeded" 172 | ``` 173 | 174 | When using Azure PowerShell or PowerShell in general, you can also make use of its powerful test framework [Pester](https://pester.dev/docs/quick-start). Pester is based on [Behavior-driven Development](https://en.wikipedia.org/wiki/Behavior-driven_development) (BDD), a software development approach that has evolved from [Test-driven Development](https://en.wikipedia.org/wiki/Test-driven_development) (TDD). It differs by being written in a shared [Domain-specific Language](https://en.wikipedia.org/wiki/Domain-specific_language) (DSL), which improves communication between tech and non-tech teams and stakeholders, i.e. developers creating Azure Policies and compliance officers and security experts defining their requirements. In both development approaches, tests are written ahead of the code, but in BDD, tests are more user-focused and based on the system’s behavior (See: [Powershell BDD with Pester](https://www.netscylla.com/blog/2019/04/28/Powershell-BDD-with-Pester.html)). In the context of Azure Policy, a test written in Pester might look like this: 175 | 176 | ```powershell 177 | Context "When route table is created or updated" -Tag "modify-routetable-nexthopvirtualappliance-routetable-create-update" { 178 | It "Should add missing route 0.0.0.0/0 pointing to the virtual appliance" -Tag "modify-routetable-nexthopvirtualappliance-routetable-create-update-10" { 179 | AzTest -ResourceGroup { 180 | param($ResourceGroup) 181 | 182 | $routeTable = New-AzRouteTable ` 183 | -Name "route-table" ` 184 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 185 | -Location $ResourceGroup.Location 186 | 187 | # Verify that route 0.0.0.0/0 was added by policy. 188 | $routeTable 189 | | Test-RouteNextHopVirtualAppliance 190 | | Should -BeTrue 191 | } 192 | } 193 | } 194 | ``` 195 | 196 | Also, the test results are very easy to grasp, by just looking at the detailed output of [Invoke-Pester](https://github.com/pester/Pester#simple-and-advanced-interface): 197 | ![Test results generated by Pester](./docs/pester-test-results.png) 198 | 199 | Finally, Pester tests can also run during [continuous integration](https://martinfowler.com/articles/continuousIntegration.html) as part of your DevOps pipeline. Following an example using [GitHub Actions](https://github.com/features/actions) (See: [test-policies.yml](./.github/workflows/test-policies.yml)): 200 | 201 | ```yaml 202 | - name: Test Azure Policies 203 | shell: pwsh 204 | run: | 205 | Invoke-Pester -Output Detailed -CI 206 | ``` 207 | 208 | As you can see, we can combine Pester and Azure PowerShell to conveniently test Azure Policy. But how does this look in more detail? How to test the different [Azure Policy effects](https://docs.microsoft.com/en-us/azure/governance/policy/concepts/effects)? Let's put them into buckets to ease the conversation: 209 | - **Synchronously** evaluated 210 | - **Asynchronously** evaluated 211 | - **Asynchronously** evaluated with **remediation task** support 212 | 213 | Policy effects, which are **synchronously** evaluated are *Append*, *Deny* and *Modify* (See: [Deny-Route-NextHopVirtualAppliance](./policies/Deny-Route-NextHopVirtualAppliance.json)). Basically, the policies already take effect during the PATCH/PUT request. Testing them with Azure PowerShell is quiet straightforward and basically just performing a PATCH/PUT request like creating a route (See: [Deny-Route-NextHopVirtualAppliance.Tests.ps1](./tests/Deny-Route-NextHopVirtualAppliance.Tests.ps1)): 214 | 215 | ```powershell 216 | Context "When route is created or updated" -Tag "deny-route-nexthopvirtualappliance-route-create-update" { 217 | It "Should deny incompliant route 0.0.0.0/0 with next hop type 'None'" -Tag "deny-route-nexthopvirtualappliance-route-create-update-10" { 218 | AzTest -ResourceGroup { 219 | param($ResourceGroup) 220 | 221 | $routeTable = New-AzRouteTable ` 222 | -Name "route-table" ` 223 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 224 | -Location $ResourceGroup.Location 225 | 226 | # Should be disallowed by policy, so exception should be thrown. 227 | { 228 | # Directly calling REST API with PUT routes, since New-AzRouteConfig/Set-AzRouteTable will issue PUT routeTables. 229 | $routeTable | Invoke-RoutePut ` 230 | -Name "default" ` 231 | -AddressPrefix "0.0.0.0/0" ` 232 | -NextHopType "None" # Incompliant. 233 | } | Should -Throw "*RequestDisallowedByPolicy*Deny-Route-NextHopVirtualAppliance*" 234 | } 235 | } 236 | } 237 | ``` 238 | 239 | For reusability reasons, the utility methods like ```AzTest``` were moved into dedicated PowerShell Modules (See: [Test.Utils.psm1](./utils/Test.Utils.psm1)): 240 | 241 | ```powershell 242 | function AzTest { 243 | param ( 244 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 245 | [ValidateNotNull()] 246 | [ScriptBlock] $Test, 247 | [Parameter()] 248 | [Switch] $ResourceGroup 249 | ) 250 | 251 | # Retries the test on transient errors. 252 | AzRetry { 253 | # When a dedicated resource group should be created for the test. 254 | if ($ResourceGroup) { 255 | try { 256 | $resourceGroup = New-ResourceGroupTest 257 | Invoke-Command -ScriptBlock $Test -ArgumentList $resourceGroup 258 | } 259 | finally { 260 | # Stops on failures during clean-up. 261 | AzCleanUp { 262 | Remove-AzResourceGroup -Name $ResourceGroup.ResourceGroupName -Force -AsJob 263 | } 264 | } 265 | } 266 | else { 267 | Invoke-Command -ScriptBlock $Test 268 | } 269 | } 270 | } 271 | ``` 272 | 273 | **Asynchronously** evaluated policy effects are *Audit* and *AuditIfNotExists*. The PATCH/PUT request just triggers a compliance scan, but the evaluation happens asynchronously in the background, e.g. [Audit-Route-NextHopVirtualAppliance](./policies/Audit-Route-NextHopVirtualAppliance.json). As it turns out, we can manually trigger a compliance scan and wait for its completion by using Azure PowerShell (See: [Audit-Route-NextHopVirtualAppliance.Tests.ps1](./tests/Audit-Route-NextHopVirtualAppliance.Tests.ps1) and [Policy.Utils.psm1](./utils/Policy.Utils.psm1)): 274 | 275 | ```powershell 276 | Context "When auditing route tables" { 277 | It "Should mark route table as compliant with route 0.0.0.0/0 pointing to virtual appliance." -Tag "audit-route-nexthopvirtualappliance-compliant" { 278 | AzTest -ResourceGroup { 279 | param($ResourceGroup) 280 | 281 | # Create compliant route table. 282 | $route = New-AzRouteConfig ` 283 | -Name "default" ` 284 | -AddressPrefix "0.0.0.0/0" ` 285 | -NextHopType "VirtualAppliance" ` 286 | -NextHopIpAddress (Get-VirtualApplianceIpAddress -Location $ResourceGroup.Location) 287 | 288 | $routeTable = New-AzRouteTable ` 289 | -Name "route-table" ` 290 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 291 | -Location $ResourceGroup.Location ` 292 | -Route $Route 293 | 294 | # Trigger compliance scan for resource group and wait for completion. 295 | $ResourceGroup | Complete-PolicyComplianceScan 296 | 297 | # Verify that route table is compliant. 298 | $routeTable 299 | | Get-PolicyComplianceState -PolicyDefinitionName "Audit-Route-NextHopVirtualAppliance" 300 | | Should -BeTrue 301 | } 302 | } 303 | } 304 | 305 | function Complete-PolicyComplianceScan { 306 | param ( 307 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 308 | [ValidateNotNull()] 309 | [Microsoft.Azure.Commands.ResourceManager.Cmdlets.SdkModels.PSResourceGroup]$ResourceGroup, 310 | [Parameter()] 311 | [ValidateRange(1, [ushort]::MaxValue)] 312 | [ushort]$MaxRetries = 3 313 | ) 314 | 315 | # Policy compliance scan might fail, hence retrying to avoid flaky tests. 316 | $retries = 0 317 | do { 318 | $job = Start-AzPolicyComplianceScan -ResourceGroupName $ResourceGroup.ResourceGroupName -PassThru -AsJob 319 | $succeeded = $job | Wait-Job | Receive-Job 320 | 321 | if ($succeeded) { 322 | break 323 | } 324 | # Failure: Retry policy compliance scan when still below maximum retries. 325 | elseif ($retries -le $MaxRetries) { 326 | Write-Host "Policy compliance scan for resource group '$($ResourceGroup.ResourceId)' failed. Retrying..." 327 | $retries++ 328 | continue # Not required, just defensive programming. 329 | } 330 | # Failure: Policy compliance scan is still failing after maximum retries. 331 | else { 332 | throw "Policy compliance scan for resource group '$($ResourceGroup.ResourceId)' failed even after $($MaxRetries) retries." 333 | } 334 | } while ($retries -le $MaxRetries) # Prevent endless loop, just defensive programming. 335 | } 336 | 337 | function Get-PolicyComplianceState { 338 | param ( 339 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 340 | [ValidateNotNull()] 341 | [Microsoft.Azure.Commands.Network.Models.PSChildResource]$Resource, 342 | [Parameter(Mandatory = $true)] 343 | [ValidateNotNullOrEmpty()] 344 | [string]$PolicyDefinitionName, 345 | [Parameter()] 346 | [ValidateRange(1, [ushort]::MaxValue)] 347 | [ushort]$WaitSeconds = 60, 348 | [Parameter()] 349 | [ValidateRange(1, [ushort]::MaxValue)] 350 | [ushort]$MaxRetries = 30 351 | ) 352 | 353 | # Policy compliance scan might be completed, but policy compliance state might still be null due to race conditions. 354 | # Hence waiting a few seconds and retrying to get the policy compliance state to avoid flaky tests. 355 | $retries = 0 356 | do { 357 | $isCompliant = (Get-AzPolicyState ` 358 | -PolicyDefinitionName $PolicyDefinitionName ` 359 | -Filter "ResourceId eq '$($Resource.Id)'" ` 360 | ).IsCompliant 361 | 362 | # Success: Policy compliance state is not null. 363 | if ($null -ne $isCompliant) { 364 | break 365 | } 366 | # Failure: Policy compliance state is null, so wait a few seconds and retry when still below maximum retries. 367 | elseif ($retries -le $MaxRetries) { 368 | Write-Host "Policy '$($PolicyDefinitionName)' completed compliance scan for resource '$($Resource.Id)', but policy compliance state is null. Retrying..." 369 | Start-Sleep -Seconds $WaitSeconds 370 | $retries++ 371 | continue # Not required, just defensive programming. 372 | } 373 | # Failure: Policy compliance state still null after maximum retries. 374 | else { 375 | throw "Policy '$($PolicyDefinitionName)' completed compliance scan for resource '$($Resource.Id)', but policy compliance state is null even after $($MaxRetries) retries." 376 | } 377 | } while ($retries -le $MaxRetries) # Prevent endless loop, just defensive programming. 378 | 379 | return $isCompliant 380 | } 381 | ``` 382 | 383 | Last but not least, the **asynchronously** evaluated policy effects with **remediation task** support are *DeployIfNotExists* and *Modify*. Just like the asynchronously evaluated policies, the compliance scan happens in the background. Additionally, non-compliant resources can be remediated with a remediation task. When testing these kind of policy effects, the easiest way is to just start a remediation task including an upfront compliance scan (See: [Modify-RouteTable-NextHopVirtualAppliance.Tests.ps1](./tests/Modify-RouteTable-NextHopVirtualAppliance.Tests.ps1) and [Policy.Utils.psm1](./utils/Policy.Utils.psm1)): 384 | 385 | ```powershell 386 | Context "When route is deleted" -Tag "modify-routetable-nexthopvirtualappliance-route-delete" { 387 | It "Should remediate missing route 0.0.0.0/0 pointing to the virtual appliance" -Tag "modify-routetable-nexthopvirtualappliance-route-delete-10" { 388 | AzTest -ResourceGroup { 389 | param($ResourceGroup) 390 | 391 | $routeTable = New-AzRouteTable ` 392 | -Name "route-table" ` 393 | -ResourceGroupName $ResourceGroup.ResourceGroupName ` 394 | -Location $ResourceGroup.Location 395 | 396 | # Get route 0.0.0.0/0 pointing to the virtual appliance, which was added by policy. 397 | $route = Get-RouteNextHopVirtualAppliance -RouteTable $routeTable 398 | 399 | # Remove-AzRouteConfig/Set-AzRouteTable will issue a PUT request for routeTables and hence policy might kick in. 400 | # In order to delete the route without policy interfering, directly call the REST API by issuing a DELETE request for route. 401 | $routeTable | Invoke-RouteDelete -Route $route 402 | 403 | # Remediate route table by policy and wait for completion. 404 | $routeTable | Complete-PolicyRemediation -PolicyDefinitionName "Modify-RouteTable-NextHopVirtualAppliance" -CheckDeployment 405 | 406 | # Verify that route 0.0.0.0/0 was added by policy remediation. 407 | Get-AzRouteTable -ResourceGroupName $routeTable.ResourceGroupName -Name $routeTable.Name 408 | | Test-RouteNextHopVirtualAppliance 409 | | Should -BeTrue 410 | } 411 | } 412 | } 413 | 414 | function Complete-PolicyRemediation { 415 | param ( 416 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 417 | [ValidateNotNull()] 418 | [Microsoft.Azure.Commands.Network.Models.PSChildResource]$Resource, 419 | [Parameter(Mandatory = $true)] 420 | [ValidateNotNullOrEmpty()] 421 | [string]$PolicyDefinitionName, 422 | [Parameter()] 423 | [switch]$CheckDeployment, 424 | [Parameter()] 425 | [ValidateRange(1, [ushort]::MaxValue)] 426 | [ushort]$MaxRetries = 3 427 | ) 428 | 429 | # Determine policy assignment id. 430 | $scope = "/subscriptions/$((Get-AzContext).Subscription.Id)" 431 | $policyAssignmentId = (Get-AzPolicyAssignment -Scope $scope 432 | | Select-Object -Property PolicyAssignmentId -ExpandProperty Properties 433 | | Where-Object { $_.PolicyDefinitionId.EndsWith($PolicyDefinitionName) } 434 | | Select-Object -Property PolicyAssignmentId -First 1 435 | ).PolicyAssignmentId 436 | 437 | if ($null -eq $policyAssignmentId) { 438 | throw "Policy '$($PolicyDefinitionName)' is not assigned to scope '$($scope)'." 439 | } 440 | 441 | # Remediation might be started before all previous changes on the resource in scope are completed. 442 | # This race condition could lead to a successful remediation without any deployment being triggered. 443 | # When a deployment is expected, it might be required to retry remediation to avoid flaky tests. 444 | $retries = 0 445 | do { 446 | # Trigger and wait for remediation. 447 | $job = Start-AzPolicyRemediation ` 448 | -Name "$($Resource.Name)-$([DateTimeOffset]::Now.ToUnixTimeSeconds())" ` 449 | -Scope $Resource.Id ` 450 | -PolicyAssignmentId $policyAssignmentId ` 451 | -ResourceDiscoveryMode ReEvaluateCompliance ` 452 | -AsJob 453 | $remediation = $job | Wait-Job | Receive-Job 454 | 455 | # Check remediation provisioning state and deployment when required. 456 | $succeeded = $remediation.ProvisioningState -eq "Succeeded" 457 | if ($succeeded) { 458 | if ($CheckDeployment) { 459 | $deployed = $remediation.DeploymentSummary.TotalDeployments -gt 0 460 | 461 | # Success: Deployment was triggered. 462 | if ($deployed) { 463 | break 464 | } 465 | # Failure: No deployment was triggered, so retry when still below maximum retries. 466 | elseif ($retries -le $MaxRetries) { 467 | Write-Host "Policy '$($PolicyDefinitionName)' succeeded to remediated resource '$($Resource.Id)', but no deployment was triggered. Retrying..." 468 | $retries++ 469 | continue # Not required, just defensive programming. 470 | } 471 | # Failure: No deployment was triggered even after maximum retries. 472 | else { 473 | throw "Policy '$($PolicyDefinitionName)' succeeded to remediated resource '$($Resource.Id)', but no deployment was triggered even after $($MaxRetries) retries." 474 | } 475 | } 476 | # Success: No deployment need to checked, hence no retry required. 477 | else { 478 | break 479 | } 480 | } 481 | # Failure: Remediation failed, so retry when still below maximum retries. 482 | elseif ($retries -le $MaxRetries) { 483 | Write-Host "Policy '$($PolicyDefinitionName)' failed to remediate resource '$($Resource.Id)'. Retrying..." 484 | $retries++ 485 | continue # Not required, just defensive programming. 486 | } 487 | # Failure: Remediation failed even after maximum retries. 488 | else { 489 | throw "Policy '$($PolicyDefinitionName)' failed to remediate resource '$($Resource.Id)' even after $($MaxRetries) retries." 490 | } 491 | } while ($retries -le $MaxRetries) # Prevent endless loop, just defensive programming. 492 | } 493 | ``` 494 | 495 | >Please note, that *Modify* can be evaluated both **synchronously** and **asynchronously**. 496 | 497 | As you can see, the combination of Pester, Azure PowerShell and GitHub Actions is quiet powerful and convenient for testing Azure Policy. In the next chapter, we will describe how to setup this repository with your GitHub account using your Azure environment, so you can further explore it. 498 | 499 | ## Setup 500 | ### Folder Structure 501 | Before going into the steps to setup this repository with your GitHub account using your Azure environment, it is important to understand how the folders in this repository are structured (generated by using the [tree](http://mama.indstate.edu/users/ice/tree/) command): 502 | 503 | ```bash 504 | . 505 | ├── .azure-pipelines 506 | ├── .github 507 | ├── docs 508 | ├── policies 509 | ├── tests 510 | └── utils 511 | ``` 512 | 513 | - **.azure-pipelines**: Leverage the [Azure YAML pipeline](https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema) when you want to deploy and test policies in your subscription using Azure DevOps. 514 | - **.github**: Leverage the [GitHub Actions workflow](https://github.com/features/actions) when you want to deploy and test policies in your subscription using GitHub Actions. 515 | - **docs**: The Markdown files and images used for documentation purposes are placed in this folder, except the **README.md** at the root, which serves as the entry point. 516 | - **policies**: All the policy definitions and assignments are placed here. Each policy is wrapped in an [ARM template](https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/overview) to ease deployment, e.g.: 517 | ```powershell 518 | Get-ChildItem -Path "./policies" | ForEach-Object { 519 | New-AzDeployment -Location "northeurope" -TemplateFile $_.FullName 520 | } 521 | ``` 522 | - **tests**: This is were all the magic happens. Each policy is tested by a corresponding PowerShell script. 523 | - **utils**: For reusability reasons, the utility methods are moved into dedicated PowerShell modules. 524 | 525 | ### Step Guide 526 | 1. **Prerequisite:** You should have installed Azure CLI on your local machine to run the command or use the Azure CloudShell in the Azure portal. To install Azure CLI, follow [Install Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest). To use Azure CloudShell, follow [Quickstart for Bash in Azure Cloud Shell](https://docs.microsoft.com/en-us/azure/cloud-shell/quickstart). 527 | 2. **Prerequisite:** Verify that [jq](https://stedolan.github.io/jq/) is installed on your system by running ```jq --version```. It should already come pre-installed in Azure CloudShell. If you run the Azure CLI commands locally you might have to install it, e.g. Ubuntu: 528 | 529 | ```bash 530 | sudo apt-get install jq 531 | ``` 532 | 533 | 3. Fork this repository (See: [Fork a repo](https://docs.github.com/en/free-pro-team@latest/github/getting-started-with-github/fork-a-repo)) 534 | 535 | ![Create a repository fork in GitHub](./docs/github-fork.png) 536 | 537 | 4. Create a [GitHub Secret](https://www.edwardthomson.com/blog/github_actions_11_secrets.html) named ```AZURE_SUBSCRIPTION_ID``` with the value being your Azure Subscription ID. You can retrieve the ID using the Azure CLI: 538 | 539 | ```bash 540 | az account show | jq -r '.id' 541 | ``` 542 | 5. Create a [GitHub Secret](https://www.edwardthomson.com/blog/github_actions_11_secrets.html) named ```AZURE_CREDENTIALS``` with the value being the JSON object outputted by this Azure CLI command: 543 | 544 | ```bash 545 | az ad sp create-for-rbac --name "azure-policy-testing" --role "Owner" \ 546 | --scopes /subscriptions/{YOUR AZURE SUBSCRIPTION ID} \ 547 | --sdk-auth 548 | ``` 549 | 550 | 3. Change the ```README.md``` to represent your build status: 551 | 552 | ```markdown 553 | ![test-policies](https://github.com/{YOUR GITHUB HANDLE}/azure-policy-testing/workflows/test-policies/badge.svg) 554 | ``` 555 | 556 | 4. Manually run the ```test-policies``` GitHub workflow and wait for it to complete successfully: 557 | 558 | ![Run the GitHub workflow](./docs/github-run-workflow.png) 559 | 560 | 5. Alternatively, you can perform a code change on either the GitHub workflow, policies, tests, or utils in the main branch to trigger the workflow by continuous integration. 561 | 562 | 6. Anyways, the build status should be reflected in your repository as well: 563 | 564 | ![GitHub build status is passing](./docs/github-build-status.png) 565 | 566 | 7. **Congrats, you are done!** Your feedback is very much appreciated, either by starring this repository, opening a pull request or by raising an issues. Many thanks upfront! 567 | 568 | > The setup focuses on GitHub Actions only. For Azure DevOps, you need to create these variables: ```AZURE_SUBSCRIPTION_CLIENT_ID```, ```AZURE_SUBSCRIPTION_CLIENT_SECRET```, ```AZURE_SUBSCRIPTION_ID```, and ```AZURE_SUBSCRIPTION_TENANT_ID```. Make ```AZURE_SUBSCRIPTION_CLIENT_SECRET``` a secret by selecting the checkbox *Keep this value secret* during variable creation. 569 | 570 | ## FAQ 571 | ### What should we consider when designing tests for policies? 572 | There are many different 1st and 3rd party tools to provision resources in Azure e.g. ARM templates, Azure PowerShell, and Terraform. Under the hood, all of them are calling the Azure REST API. Hence, it makes sense to carefully study the [Azure REST API reference](https://docs.microsoft.com/en-us/rest/api/azure/) and [Azure REST API guidelines](https://github.com/microsoft/api-guidelines/blob/vNext/azure/Guidelines.md) when designing tests for policies. Especially consider: 573 | - Structure your test cases around Azure REST API calls consider i.e., PUT, PATCH and DELETE requests. Basically, any request which can can lead to your resources being incompliant. 574 | - When the [resource provider](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/resource-providers-and-types) does not support PATCH requests, you do not need separate test cases for creating and updating resources since they both result in the same PUT request. 575 | - Also consider that some properties are optional, so they might not be sent as part of the PUT requests. You can leverage the [Azure REST API reference](https://docs.microsoft.com/en-us/rest/api/azure/) to check if a property is optional or required. In case the policy alias you are using is referring to an optional property, you should create a dedicated test case to validate the behavior of your policy. 576 | - Some child resources e.g., [route](https://docs.microsoft.com/en-us/rest/api/virtualnetwork/routes/createorupdate), can be created standalone or wrapped as inline property and created with their their parent resource e.g., [route table](https://docs.microsoft.com/en-us/rest/api/virtualnetwork/routetables/createorupdate#request-body). Keep that in mind when designing and testing polices e.g., policy [Deny-Route-NextHopVirtualAppliance.json](./policies/Deny-Route-NextHopVirtualAppliance.json) and the corresponding tests [Deny-Route-NextHopVirtualAppliance.Tests.ps1](./tests/Deny-Route-NextHopVirtualAppliance.Tests.ps1). 577 | - Policies currently do not trigger on DELETE only PUT and PATCH requests. Hence deleted resources can only be remediated asynchronously by using a remediation task. 578 | - Accessing shared resources during your tests can cause race conditions, e.g. parallel test runs. Consider creating a dedicated resource group per test case to be a best practice. [AzTest](./utils/Test.Utils.psm1) can automatically create and delete a resource group for you: 579 | 580 | ```powershell 581 | It "..." -Tag "..." { 582 | AzTest -ResourceGroup { 583 | param($ResourceGroup) 584 | 585 | # ... 586 | } 587 | } 588 | ``` 589 | 590 | ### Can we execute the tests on our local machines? 591 | Just like in your DevOps pipeline of choice, you can execute the tests on your local machines as well, e.g.: 592 | 593 | ```powershell 594 | Invoke-Pester -Output Detailed 595 | ``` 596 | 597 | ### Is it possible to execute just a subset of the tests? 598 | You can leverage tags to execute just a subset of the tests, e.g.: 599 | 600 | ```powershell 601 | Invoke-Pester -Output Detailed -Tags "tag" 602 | 603 | It "..." -Tag "tag" { 604 | # ... 605 | } 606 | ``` 607 | 608 | ### Is it possible to execute the tests under a different user? 609 | Yes you can. Just use different ```AZURE_CREDENTIALS``` to login before you execute the tests. Additionally, you can tag the tests by user and select them accordingly when running them in your DevOps pipeline or locally. 610 | 611 | ```yaml 612 | - name: Login to Azure 613 | uses: azure/login@v1 614 | with: 615 | creds: ${{secrets.AZURE_CREDENTIALS}} 616 | enable-AzPSSession: true 617 | - name: Test Azure Policies 618 | shell: pwsh 619 | run: | 620 | Invoke-Pester -Output Detailed -CI -Tags "user" 621 | ``` 622 | 623 | ```powershell 624 | It "..." -Tag "user" { 625 | # ... 626 | } 627 | ``` 628 | 629 | > Do not mix testing Azure Policy and RBAC. If you need to test RBAC e.g., to validate custom roles, create dedicated PowerShell scripts in the [tests](./tests/) folder. This separation helps you to keep your test maintainable by not mixing different concerns. 630 | 631 | ### Can we pass parameters to our tests? 632 | Yes you can. Starting with [Pester 5.1.0-beta2](https://www.powershellgallery.com/packages/Pester/) passing parameters is supported (See: [GitHub Issue #1485](https://github.com/pester/Pester/issues/1485)): 633 | 634 | ```powershell 635 | $container = @( 636 | (New-TestContainer -Path $file -Data @{ Value = 1 }) 637 | (New-TestContainer -Path $file -Data @{ Value = 2 }) 638 | ) 639 | $r = Invoke-Pester -Container $container -PassThru 640 | ``` 641 | 642 | ### The tests take a long time to complete, can we speed things up? 643 | Currently, Pester itself does not natively support it (See [GitHub Issue #1270](https://github.com/pester/Pester/issues/1270)), but you can achieve parallelization by invoking Pester multiple times: 644 | 645 | ```powershell 646 | $job = Get-ChildItem -Path "./tests" 647 | | ForEach-Object -Parallel { 648 | Invoke-Pester -Path $_ -Output None -PassThru -CI 649 | } -ThrottleLimit 10 -AsJob 650 | $testResults = $job | Wait-Job | Receive-Job 651 | ``` 652 | 653 | Please consider above as sample code to give you an idea how to parallelize your tests. Parallelization is a future topic to cover in case there is enough community interest. Just as a side note, each job in a GitHub workflow can run for up to 6 hours of execution time. Following, your tests should finish before that or you can split them into multiple jobs, since GitHub workflows can run up to 72 hours. Avoid accessing shared resources when parallelizing test execution to avoid race conditions. Instead create a dedicated resource group per test case i.e., [AzTest](./utils/Test.Utils.psm1) can automatically create and delete a resource group for you: 654 | 655 | ```powershell 656 | It "..." -Tag "..." { 657 | AzTest -ResourceGroup { 658 | param($ResourceGroup) 659 | 660 | # ... 661 | } 662 | } 663 | ``` 664 | 665 | 666 | ### Should we execute the tests to validate a pull request? 667 | Executing the tests can take a few minutes up to some hours. The long duration is mainly caused by waiting for policy compliance scans and remediations to complete. So while you certainly can execute the tests to validate your pull request, it is not advisable since a pull request should provide your developers feedback in just a couple of minutes to reduce their unproductive waiting time. That being said, executing them as part of your [continuous integration](https://martinfowler.com/articles/continuousIntegration.html) on the main branch is what you should aim for. Alternatively, it might be just good enough to schedule a test run once a day. 668 | 669 | ### Why did you assign the policies to subscription and not management group scope? 670 | Mainly to reduce complexity when explaining the approach and to ease setting it up in your Azure environment. But the approach can easily be scaled towards supporting management groups i.e., by adding support for management groups to [Policy.Utils.psm1](./utils/Policy.Utils.psm1)). If you want to learn more about managing Azure at scale, checkout [Enterprise Scale](https://github.com/Azure/Enterprise-Scale). 671 | 672 | ### Can we scale this testing approach towards a complex management group hierarchy? 673 | You can try to scale towards a more complex management group hierarchy like this (See: [Enterprise Scale](https://github.com/Azure/Enterprise-Scale/blob/main/docs/reference/adventureworks/README.md)): 674 | 675 | ![Complex management group hierarchy](./docs/azure-management-groups.png) 676 | 677 | An idea would be to create an Azure subscription for testing per leaf management group, so referring to the example management group hierarchy: Management, Connectivity, Identity, Corp and Online. For each of this subscriptions you would run a set of tests. Since each policy is tested by a dedicated PowerShell script, you could reuse them across subscriptions. Also keep in mind that you have to add support for management groups to [Policy.Utils.psm1](./utils/Policy.Utils.psm1). While scaling towards a complex management group hierarchy certainly improves test coverage by also considering policy layering (See: [Layering policy definitions 678 | ](https://docs.microsoft.com/en-us/azure/governance/policy/concepts/effects#layering-policy-definitions)), it also increases the test duration and complexity a lot. If you are just interested in validating the logic of a single policy, this might be overkill. 679 | --------------------------------------------------------------------------------