├── .gitattributes ├── .github ├── CONTRIBUTING.md ├── ISSUE_TEMPLATE.md └── PULL_REQUEST_TEMPLATE.md ├── .gitignore ├── .vscode ├── launch.json └── tasks.json ├── CHANGELOG.md ├── LICENSE ├── OperationValidation ├── Diagnostics │ ├── Comprehensive │ │ └── PSGallery.Comprehensive.Tests.ps1 │ └── Simple │ │ └── PSGallery.Simple.Tests.ps1 ├── OperationValidation.Format.ps1xml ├── OperationValidation.psd1 ├── OperationValidation.psm1 ├── Private │ ├── Convert-TestResult.ps1 │ ├── Get-ModuleList.ps1 │ ├── Get-TestCaseNamesFromAst.ps1 │ ├── Get-TestFromAst.ps1 │ ├── Get-TestFromScript.ps1 │ ├── Get-TestName.ps1 │ ├── New-OperationValidationFailure.ps1 │ ├── New-OperationValidationInfo.ps1 │ ├── New-OperationValidationResult.ps1 │ └── Parse-Psd1.ps1 └── Public │ ├── Get-OperationValidation.ps1 │ └── Invoke-OperationValidation.ps1 ├── README.md ├── TestArtifacts ├── Example.WindowsSearch │ └── Diagnostics │ │ └── Simple │ │ └── WindowsSearch.Simple.Tests.ps1 ├── SingleTest.tests.ps1 └── VersionedModule │ ├── 1.0.0 │ ├── Diagnostics │ │ └── Simple │ │ │ └── PSGallery.Simple.Tests.ps1 │ └── VersionedModule.psd1 │ └── 2.0.0 │ ├── Diagnostics │ └── Simple │ │ └── PSGallery.Simple.Tests.ps1 │ └── VersionedModule.psd1 ├── Tests ├── Meta │ ├── Help.tests.ps1 │ ├── Manifest.tests.ps1 │ ├── Meta.tests.ps1 │ └── MetaFixers.psm1 ├── OperationValidation.Tests.ps1 └── Unit │ ├── Get-OperationValidation.tests.ps1 │ ├── Invoke-OperationValidation.tests.ps1 │ └── Module.tests.ps1 ├── appveyor.yml ├── azure-pipelines.yml ├── build.ps1 ├── docs └── en-US │ ├── Get-OperationValidation.md │ ├── Invoke-OperationValidation.md │ └── OperationValidation.md ├── psakeFile.ps1 └── requirements.psd1 /.gitattributes: -------------------------------------------------------------------------------- 1 | # Set the default behavior, in case people don't have core.autocrlf set. 2 | * text=auto 3 | * -crlf -------------------------------------------------------------------------------- /.github/CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # How to contribute 2 | Contributions to OperationValidation are highly encouraged and desired. Below are some guidelines that will help make the process as smooth as possible. 3 | 4 | # Getting Started 5 | * Make sure you have a [GitHub account](https://github.com/signup/free) 6 | * Submit a new issue, assuming one does not already exist. 7 | * Clearly describe the issue including steps to reproduce when it is a bug. 8 | * Make sure you fill in the earliest version that you know has the issue. 9 | * Fork the repository on GitHub 10 | 11 | # Suggesting Enhancements 12 | We want to know what you think is missing from OperationValidation and how it can be made better. 13 | * When submitting an issue for an enhancement, please be as clear as possible about why you think the enhancement is needed and what the benefit of 14 | it would be. 15 | 16 | # Making Changes 17 | * From your fork of the repository, create a topic branch where work on your change will take place. 18 | * To quickly create a topic branch based on master; `git checkout -b my_contribution master`. Please avoid working directly on the `master` branch. 19 | * Make commits of logical units. 20 | * Check for unnecessary whitespace with `git diff --check` before committing. 21 | * Please follow the prevailing code conventions in the repository. Differences in style make the code harder to understand for everyone. 22 | * Make sure your commit messages are in the proper format. 23 | ```` 24 | Add more cowbell to Get-Something.ps1 25 | 26 | The functionaly of Get-Something would be greatly improved if there was a little 27 | more 'pizzazz' added to it. I propose a cowbell. Adding more cowbell has been 28 | shown in studies to both increase one's mojo, and cement one's status 29 | as a rock legend. 30 | ```` 31 | 32 | * Make sure you have added all the necessary Pester tests for your changes. 33 | * Run _all_ PESTER tests in the module to assure nothing else was accidentally broken. 34 | 35 | # Documentation 36 | We are infallible and as such my documenation needs no corectoin. In the highly 37 | unlikely event that that is _not_ the case, commits to update or add documentation 38 | are highly apprecaited. 39 | 40 | # Submitting Changes 41 | * Push your changes to a topic branch in your fork of the repository. 42 | * Submit a pull request to the main repository. 43 | * Once the pull request has been reviewed and accepted, it will be merged with the master branch. 44 | * Celebrate 45 | 46 | # Additional Resources 47 | * [General GitHub documentation](https://help.github.com/) 48 | * [GitHub forking documentation](https://guides.github.com/activities/forking/) 49 | * [GitHub pull request documentation](https://help.github.com/send-pull-requests/) 50 | * [GitHub Flow guide](https://guides.github.com/introduction/flow/) 51 | * [GitHub's guide to contributing to open source projects](https://guides.github.com/activities/contributing-to-open-source/) 52 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## Current Behavior 4 | 5 | 6 | 7 | ## Expected Behavior 8 | 9 | 10 | 11 | ## Possible Solution 12 | 13 | 14 | 15 | ## Steps to Reproduce (for bugs) 16 | 17 | 18 | 1. 19 | 2. 20 | 3. 21 | 4. 22 | 23 | ## Context 24 | 25 | 26 | 27 | ## Your Environment 28 | 29 | * Module version used: 30 | * Operating System and PowerShell version: 31 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## Description 4 | 5 | 6 | ## Related Issue 7 | 8 | 9 | 10 | 11 | 12 | ## Motivation and Context 13 | 14 | 15 | ## How Has This Been Tested? 16 | 17 | 18 | 19 | 20 | ## Screenshots (if appropriate): 21 | 22 | ## Types of changes 23 | 24 | - [ ] Bug fix (non-breaking change which fixes an issue) 25 | - [ ] New feature (non-breaking change which adds functionality) 26 | - [ ] Breaking change (fix or feature that would cause existing functionality to change) 27 | 28 | ## Checklist: 29 | 30 | 31 | - [ ] My code follows the code style of this project. 32 | - [ ] My change requires a change to the documentation. 33 | - [ ] I have updated the documentation accordingly. 34 | - [ ] I have read the **CONTRIBUTING** document. 35 | - [ ] I have added tests to cover my changes. 36 | - [ ] All new and existing tests passed. 37 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | out/* 2 | output/* -------------------------------------------------------------------------------- /.vscode/launch.json: -------------------------------------------------------------------------------- 1 | { 2 | // Use IntelliSense to learn about possible attributes. 3 | // Hover to view descriptions of existing attributes. 4 | // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 5 | "version": "0.2.0", 6 | "configurations": [ 7 | 8 | { 9 | "type": "PowerShell", 10 | "request": "launch", 11 | "name": "PowerShell Launch Current File", 12 | "script": "${file}", 13 | "args": [], 14 | "cwd": "${file}" 15 | }, 16 | { 17 | "type": "PowerShell", 18 | "request": "launch", 19 | "name": "PowerShell Launch Current File in Temporary Console", 20 | "script": "${file}", 21 | "args": [], 22 | "cwd": "${file}", 23 | "createTemporaryIntegratedConsole": true 24 | }, 25 | { 26 | "type": "PowerShell", 27 | "request": "launch", 28 | "name": "PowerShell Launch Current File w/Args Prompt", 29 | "script": "${file}", 30 | "args": [ 31 | "${command:SpecifyScriptArgs}" 32 | ], 33 | "cwd": "${file}" 34 | }, 35 | { 36 | "type": "PowerShell", 37 | "request": "attach", 38 | "name": "PowerShell Attach to Host Process", 39 | "processId": "${command:PickPSHostProcess}", 40 | "runspaceId": 1 41 | }, 42 | { 43 | "type": "PowerShell", 44 | "request": "launch", 45 | "name": "PowerShell Interactive Session", 46 | "cwd": "" 47 | } 48 | ] 49 | } -------------------------------------------------------------------------------- /.vscode/tasks.json: -------------------------------------------------------------------------------- 1 | { 2 | // See https://go.microsoft.com/fwlink/?LinkId=733558 3 | // for the documentation about the tasks.json format 4 | "version": "0.1.0", 5 | "showOutput": "always", 6 | 7 | // Start PowerShell 8 | "windows": { 9 | "command": "${env:windir}\\sysnative\\WindowsPowerShell\\v1.0\\powershell.exe", 10 | "args": [ "-NoProfile", "-ExecutionPolicy", "Bypass" ] 11 | }, 12 | "linux": { 13 | "command": "/usr/bin/powershell", 14 | "args": [ "-NoProfile" ] 15 | }, 16 | "osx": { 17 | "command": "/usr/local/bin/powershell", 18 | "args": [ "-NoProfile" ] 19 | }, 20 | 21 | "tasks": [ 22 | { 23 | "taskName": "Test", 24 | "suppressTaskName": true, 25 | "showOutput": "always", 26 | "isTestCommand": true, 27 | "args": [ 28 | "./build.ps1 -Task Test" 29 | ] 30 | }, 31 | { 32 | "taskName": "Lint", 33 | "suppressTaskName": true, 34 | "showOutput": "always", 35 | "isTestCommand": false, 36 | "args":[ 37 | "./build.ps1 -Task Analyze" 38 | ] 39 | } 40 | ] 41 | } -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Change Log 2 | 3 | All notable changes to this project will be documented in this file. 4 | 5 | The format is based on [Keep a Changelog](http://keepachangelog.com/) 6 | and this project adheres to [Semantic Versioning](http://semver.org/). 7 | 8 | ## [1.3.0] - Unreleased 9 | 10 | ### 11 | 12 | - [**#38**](https://github.com/PowerShell/Operation-Validation-Framework/pull/38) - Add progress bar during test execution (via [@My-Random-Thoughts](https://github.com/My-Random-Thoughts)) 13 | 14 | ## [1.2.1] - 2019-02-15 15 | 16 | ### Fixed 17 | 18 | - [**#36**](https://github.com/PowerShell/Operation-Validation-Framework/pull/36) - Fixed PowerShell version check logic (via [@miketheitguy](https://github.com/miketheitguy)) 19 | 20 | ## [1.2.0] - 2018-10-18 21 | 22 | ### Added 23 | 24 | - Enable cross platform support 25 | 26 | ## [1.1.0] - 2018-03-13 27 | 28 | ### Added 29 | 30 | - PR12 - Ability to specify parameter overrides to Pester tests (via @devblackops) 31 | - PR16 - Ability to specify Pester tests to execute based on tag(s) (via @devblackops) 32 | - PR25 - Support for parsing Pester tests when the -Fixture parameter is used (via @GitAMBS) 33 | - PR26 - Support for -Path and -LiteralPath parameters to module directories (via @devblackops) 34 | 35 | ### Fixed 36 | 37 | - PR25 - Discover Pester 'Describe' block names when the -Name parameter is used (via @GitAMBS) 38 | - PR29 - Remove Pester 4.0+ warning by using -Show parameter instead of -Quiet. Add -UseBasicParsing parameter to Invoke-WebRequest in tests (via @larssb) 39 | - Fixed #3 where an error would be generated when using Invoke-OperationValidation with the -TestFilePath parameter 40 | against a Pester test file with a short path. 41 | 42 | ## [1.0.1] - 2015-12-05 43 | 44 | ### Changed 45 | 46 | - Handle version specific directories. 47 | 48 | ## [1.0.0] - Unreleased 49 | 50 | ### Added 51 | 52 | - Initial release 53 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Microsoft Corporation 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /OperationValidation/Diagnostics/Comprehensive/PSGallery.Comprehensive.Tests.ps1: -------------------------------------------------------------------------------- 1 | 2 | Describe 'E2E validation of PSGallery' -Fixture { 3 | BeforeAll { 4 | $repository = 'PSGallery' 5 | $moduleName = 'FormatTools' 6 | $version = '0.5.0' 7 | if (Get-Module -Name $moduleName -ListAvailable) { 8 | # the module is already installed 9 | $PSDefaultParameterValues['It:skip'] = $true 10 | } 11 | } 12 | 13 | # It 'should return the same number of modules via cmdlets and website' { 14 | # $galleryUrl = 'https://www.powershellgallery.com' 15 | # # timing window here - between these two operations, modules list may change 16 | # $wc = New-Object System.Net.WebClient 17 | # $modules = Find-Module -Repository $repository -ErrorAction SilentlyContinue 18 | # $page = $wc.downloadstring("${galleryUrl}/packages").replace("`n", '') 19 | # $expectedCount = $page -replace ".*There are (\d+) modules.*", '$1' 20 | # $modules.Count | Should be $expectedCount 21 | # } 22 | It -skip:$false 'Should be possible to find a known module' { 23 | $myModule = Find-Module -Repository $repository -Name $moduleName -RequiredVersion $version 24 | $myModule.Name | Should be $moduleName 25 | $myModule.Version | Should be $version 26 | } 27 | It 'Should be possible to install and import a known module' { 28 | Install-Module -Force -Name $moduleName -RequiredVersion $version -Repository $repository -Scope CurrentUser 29 | $m = Import-Module $moduleName -PassThru 30 | $m.ModuleBase.IndexOf($HOME) | Should be 0 31 | } 32 | 33 | AfterAll { 34 | if ($PSDefaultParameterValues['It:skip'] -ne $true) { 35 | Uninstall-Module -Force -RequiredVersion $version -Name $ModuleName -ErrorAction SilentlyContinue 36 | } 37 | } 38 | } 39 | 40 | -------------------------------------------------------------------------------- /OperationValidation/Diagnostics/Simple/PSGallery.Simple.Tests.ps1: -------------------------------------------------------------------------------- 1 | Describe -Name 'Simple Validation of PSGallery' { 2 | It 'The PowerShell Gallery should be responsive' { 3 | $response = Invoke-WebRequest -Uri 'https://www.powershellgallery.com' 4 | $response.StatusCode | Should be 200 5 | } 6 | } 7 | -------------------------------------------------------------------------------- /OperationValidation/OperationValidation.Format.ps1xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | OperationValidationResultTable 5 | 6 | OperationValidationResult 7 | 8 | 9 | Module 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | Result 21 | Name 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | OperationValidation 30 | 31 | OperationValidationInfo 32 | 33 | 34 | 35 | 36 | 37 | 38 | Module: 39 | 40 | 4 41 | 42 | 43 | ModuleName 44 | 45 | 46 | 47 | 48 | 49 | Version: 50 | 51 | 4 52 | 53 | 54 | Version 55 | 56 | 57 | 58 | 59 | 60 | Type: 61 | 62 | 4 63 | 64 | 65 | Type 66 | 67 | 68 | 69 | 70 | 71 | Tags: 72 | 73 | 4 74 | 75 | 76 | Tags 77 | 78 | 79 | 80 | 81 | 82 | File: 83 | 84 | 4 85 | 86 | 87 | File 88 | 89 | 90 | 91 | 92 | 93 | FilePath: 94 | 95 | 4 96 | 97 | 98 | FilePath 99 | 100 | 101 | 102 | 103 | 104 | Name: 105 | 106 | 4 107 | 108 | 109 | 110 | 4 111 | 112 | 113 | Name 114 | 115 | 116 | 117 | 118 | 119 | 120 | $_ 121 | 122 | 123 | 124 | 125 | 126 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 135 | 136 | 137 | 138 | 139 | 140 | -------------------------------------------------------------------------------- /OperationValidation/OperationValidation.psd1: -------------------------------------------------------------------------------- 1 | # 2 | # Module manifest for module 'OperationValidation' 3 | # 4 | # Generated by: jimtru 5 | # 6 | # Generated on: 3/18/2015 7 | # 8 | 9 | @{ 10 | 11 | # Script module or binary module file associated with this manifest. 12 | RootModule = 'OperationValidation.psm1' 13 | # ModuleToProcess = 'OperationValidation.psm1' 14 | 15 | # Version number of this module. 16 | ModuleVersion = '1.3.0' 17 | 18 | # ID used to uniquely identify this module 19 | GUID = '25bd9e34-bff9-4552-a23d-854857b42462' 20 | 21 | # Author of this module 22 | Author = 'jimtru' 23 | 24 | # Company or vendor of this module 25 | CompanyName = 'Unknown' 26 | 27 | # Copyright statement for this module 28 | Copyright = '(c) 2015 jimtru. All rights reserved.' 29 | 30 | # Description of the functionality provided by this module 31 | Description = 'A set of tools for executing validation of the operation of a system. It provides a way to organize and execute Pester tests which are written to validate operation (rather than limited feature tests)' 32 | 33 | # Minimum version of the Windows PowerShell engine required by this module 34 | # PowerShellVersion = '' 35 | 36 | # Name of the Windows PowerShell host required by this module 37 | # PowerShellHostName = '' 38 | 39 | # Minimum version of the Windows PowerShell host required by this module 40 | # PowerShellHostVersion = '' 41 | 42 | # Minimum version of Microsoft .NET Framework required by this module 43 | # DotNetFrameworkVersion = '' 44 | 45 | # Minimum version of the common language runtime (CLR) required by this module 46 | # CLRVersion = '' 47 | 48 | # Processor architecture (None, X86, Amd64) required by this module 49 | # ProcessorArchitecture = '' 50 | 51 | # Modules that must be imported into the global environment prior to importing this module 52 | RequiredModules = @('Pester') 53 | 54 | # Assemblies that must be loaded prior to importing this module 55 | # RequiredAssemblies = @() 56 | 57 | # Script files (.ps1) that are run in the caller's environment prior to importing this module. 58 | # ScriptsToProcess = @() 59 | 60 | # Type files (.ps1xml) to be loaded when importing this module 61 | # TypesToProcess = @() 62 | 63 | # Format files (.ps1xml) to be loaded when importing this module 64 | FormatsToProcess = @('OperationValidation.Format.ps1xml') 65 | 66 | # Modules to import as nested modules of the module specified in RootModule/ModuleToProcess 67 | # NestedModules = @() 68 | 69 | # Functions to export from this module 70 | FunctionsToExport = @( 71 | 'Get-OperationValidation' 72 | 'Invoke-OperationValidation' 73 | ) 74 | 75 | # Cmdlets to export from this module 76 | # CmdletsToExport = '*' 77 | 78 | # Variables to export from this module 79 | # VariablesToExport = '*' 80 | 81 | # Aliases to export from this module 82 | # AliasesToExport = '*' 83 | 84 | # DSC resources to export from this module 85 | # DscResourcesToExport = @() 86 | 87 | # List of all modules packaged with this module 88 | # ModuleList = @() 89 | 90 | # List of all files packaged with this module 91 | FileList = @( 92 | 'OperationValidation.Format.ps1xml' 93 | 'OperationValidation.psd1' 94 | 'OperationValidation.psm1' 95 | 'Diagnostics\Comprehensive\PSGallery.Comprehensive.Tests.ps1' 96 | 'Diagnostics\Simple\PSGallery.Simple.Tests.ps1' 97 | 'Private\Convert-TestResult.ps1' 98 | 'Private\Get-ModuleList.ps1' 99 | 'Private\Get-TestCaseNamesFromAst.ps1' 100 | 'Private\Get-TestFromAst.ps1' 101 | 'Private\Get-TestFromScript.ps1' 102 | 'Private\Get-TestName.ps1' 103 | 'Private\New-OperationValidationFailure.ps1' 104 | 'Private\New-OperationValidationInfo.ps1' 105 | 'Private\New-OperationValidationInfo.ps1' 106 | 'Private\New-OperationValidationResult.ps1' 107 | 'Private\Parse-Psd1.ps1' 108 | 'Public\Get-OperationValidation.ps1' 109 | 'Public\Invoke-OperationValidation.ps1' 110 | ) 111 | 112 | # Private data to pass to the module specified in RootModule/ModuleToProcess. This may also contain a PSData hashtable with additional module metadata used by PowerShell. 113 | PrivateData = @{ 114 | 115 | PSData = @{ 116 | 117 | # Tags applied to this module. These help with module discovery in online galleries. 118 | Tags = @('Operation', 'Validation', 'Infrastructure', 'Testing', 'Pester', 'OVF', 'PSEdition_Desktop', 'PSEdition_Core') 119 | 120 | # A URL to the license for this module. 121 | LicenseUri = 'https://raw.githubusercontent.com/PowerShell/Operation-Validation-Framework/master/LICENSE' 122 | 123 | # A URL to the main website for this project. 124 | ProjectUri = 'https://github.com/PowerShell/Operation-Validation-Framework' 125 | 126 | # A URL to an icon representing this module. 127 | # IconUri = '' 128 | 129 | # ReleaseNotes of this module 130 | ReleaseNotes = 'https://raw.githubusercontent.com/PowerShell/Operation-Validation-Framework/master/CHANGELOG.md' 131 | 132 | } # End of PSData hashtable 133 | 134 | } # End of PrivateData hashtable 135 | 136 | # HelpInfo URI of this module 137 | # HelpInfoURI = '' 138 | 139 | # Default prefix for commands exported from this module. Override the default prefix using Import-Module -Prefix. 140 | # DefaultCommandPrefix = '' 141 | 142 | } 143 | 144 | -------------------------------------------------------------------------------- /OperationValidation/OperationValidation.psm1: -------------------------------------------------------------------------------- 1 | 2 | $script:pathSeparator = [IO.Path]::PathSeparator 3 | 4 | # Dot source public/private functions 5 | $public = @( Get-ChildItem -Path ([IO.Path]::Combine($PSScriptRoot, 'Public', '*.ps1')) -Recurse -ErrorAction SilentlyContinue ) 6 | $private = @( Get-ChildItem -Path ([IO.Path]::Combine($PSScriptRoot, 'Private', '*.ps1')) -Recurse -ErrorAction SilentlyContinue ) 7 | foreach($import in @($public + $private)) { 8 | try { 9 | . $import.FullName 10 | } catch { 11 | throw "Unable to dot source [$($import.FullName)]" 12 | } 13 | } 14 | 15 | Export-ModuleMember -Function $public.Basename 16 | -------------------------------------------------------------------------------- /OperationValidation/Private/Convert-TestResult.ps1: -------------------------------------------------------------------------------- 1 | 2 | # Emit an object which can be used in reporting 3 | Function Convert-TestResult { 4 | param ( 5 | [Parameter(Mandatory)] 6 | $result, 7 | 8 | [string]$ModuleName 9 | ) 10 | 11 | foreach ($testResult in $result.TestResult) { 12 | $testError = $null 13 | if ($testResult.Result -eq 'Failed') { 14 | Write-Verbose -message 'Creating error object' 15 | $testError = New-OperationValidationFailure -Stacktrace $testResult.StackTrace -FailureMessage $testResult.FailureMessage 16 | } 17 | 18 | $TestName = '{0}:{1}:{2}' -f $testResult.Describe, $testResult.Context, $testResult.Name 19 | 20 | $newOVResultParams = @{ 21 | Name = $TestName 22 | FileName = $result.path 23 | Result = $testresult.result 24 | RawResult = $testResult 25 | Error = $TestError 26 | } 27 | if (-not [string]::IsNullOrEmpty($ModuleName)) { 28 | $newOVResultParams.Module = $ModuleName 29 | } 30 | New-OperationValidationResult @newOVResultParams 31 | } 32 | } 33 | -------------------------------------------------------------------------------- /OperationValidation/Private/Get-ModuleList.ps1: -------------------------------------------------------------------------------- 1 | function Get-ModuleList { 2 | [cmdletbinding(DefaultParameterSetName = 'Name')] 3 | param ( 4 | [Parameter( 5 | Mandatory, 6 | ParameterSetName = 'Name' 7 | )] 8 | [string[]]$Name, 9 | 10 | [Parameter( 11 | Mandatory, 12 | ParameterSetName = 'Path' 13 | )] 14 | [string[]]$Path, 15 | 16 | [version]$Version 17 | ) 18 | 19 | if ($PSCmdlet.ParameterSetName -eq 'Name') { 20 | $pathsToSearch = $env:PSModulePath.Trim($script:pathSeparator).Split($script:pathSeparator) 21 | } elseIf ($PSCmdlet.ParameterSetName -eq 'Path') { 22 | $pathsToSearch = $Path 23 | } 24 | 25 | foreach($p in $pathsToSearch) { 26 | if (Test-Path -Path $p) { 27 | foreach($modDir in Get-ChildItem -Path $p -Directory) { 28 | Write-Debug "Checking for OVF in [$modDir]" 29 | 30 | if ($PSCmdlet.ParameterSetName -eq 'Path') { 31 | $Name = $modDir.Name 32 | } 33 | 34 | foreach ($n in $Name) { 35 | if ($modDir.Name -like $n) { 36 | # now determine if there's a diagnostics directory, or a version 37 | if (Test-Path -Path (Join-Path -Path $modDir.FullName -ChildPath 'Diagnostics')) { 38 | # Did we specify a specific version to find? 39 | if ($PSBoundParameters.ContainsKey('Version')) { 40 | $manifestFile = Get-ChildItem -Path $modDir.FullName -Filter "$($modDir.Name).psd1" | Select-Object -First 1 41 | if ($manifestFile -and (Test-Path -Path $manifestFile.FullName)) { 42 | Write-Verbose $manifestFile 43 | $manifest = Test-ModuleManifest -Path $manifestFile.FullName -Verbose:$false 44 | if ($manifest.Version -eq $Version) { 45 | $modDir.FullName 46 | break 47 | } 48 | } 49 | } else { 50 | $modDir.FullName 51 | break 52 | } 53 | } 54 | 55 | # Get latest version if no specific version specified 56 | if ($PSBoundParameters.ContainsKey('Version')) { 57 | $versionDirectories = Get-Childitem -Path $modDir.FullName -Directory | 58 | Where-Object { $_.name -as [version] -and $_.Name -eq $Version } 59 | } else { 60 | $versionDirectories = Get-Childitem -Path $modDir.FullName -Directory | 61 | Where-Object { $_.name -as [version] } 62 | } 63 | 64 | $potentialDiagnostics = $versionDirectories | Where-Object { 65 | Test-Path -Path (Join-Path -Path $_.FullName -ChildPath 'Diagnostics') 66 | } 67 | # Now select the most recent module path which has diagnostics 68 | $DiagnosticDir = $potentialDiagnostics | 69 | Sort-Object {$_.name -as [version]} | 70 | Select-Object -Last 1 71 | if ($DiagnosticDir) { 72 | $DiagnosticDir.FullName 73 | break 74 | } 75 | } 76 | } 77 | } 78 | } else { 79 | Write-Error -Message "Could not access [$p]. Does it exist?" 80 | } 81 | } 82 | } 83 | -------------------------------------------------------------------------------- /OperationValidation/Private/Get-TestCaseNamesFromAst.ps1: -------------------------------------------------------------------------------- 1 | 2 | function Get-TestCaseNamesFromAst { 3 | param( 4 | $ast 5 | ) 6 | 7 | $eb = $ast.EndBlock 8 | foreach($statement in $eb.Statements) { 9 | if ($statement -isnot 'System.Management.Automation.Language.PipelineAst') { 10 | continue 11 | } 12 | $commandAst = $statement.PipelineElements[0].CommandElements[0] 13 | 14 | if ($commandAst.Value -eq 'It') { 15 | Get-TestName $CommandAst 16 | } 17 | } 18 | } 19 | -------------------------------------------------------------------------------- /OperationValidation/Private/Get-TestFromAst.ps1: -------------------------------------------------------------------------------- 1 | 2 | function Get-TestFromAst { 3 | param( 4 | $ast 5 | ) 6 | 7 | $eb = $ast.EndBlock 8 | foreach($statement in $eb.Statements) { 9 | if ($statement -isnot 'System.Management.Automation.Language.PipelineAst') { 10 | continue 11 | } 12 | $commandAst = $statement.PipelineElements[0].CommandElements[0] 13 | 14 | if ($commandAst.Value -eq 'Describe') { 15 | Get-TestName $commandAst 16 | } 17 | } 18 | } 19 | -------------------------------------------------------------------------------- /OperationValidation/Private/Get-TestFromScript.ps1: -------------------------------------------------------------------------------- 1 | 2 | function Get-TestFromScript { 3 | param ( 4 | [parameter(Mandatory)] 5 | [string]$ScriptPath 6 | ) 7 | 8 | $text = Get-Content -Path $ScriptPath -Raw 9 | $tokens = $null 10 | $errors = $null 11 | $describes = [Management.Automation.Language.Parser]::ParseInput($text, [ref]$tokens, [ref]$errors). 12 | FindAll([Func[Management.Automation.Language.Ast,bool]]{ 13 | param ($ast) 14 | $ast.CommandElements -and 15 | $ast.CommandElements[0].Value -eq 'describe' 16 | }, $true) | 17 | ForEach-Object { 18 | # This is the name of the 'describe' block 19 | for ($x = 0; $x -lt $_.CommandElements.Count; $x++) { 20 | # Name parameter is named 21 | if ($_.CommandElements[$x] -is [System.Management.Automation.Language.CommandParameterAst] -and $_.CommandElements[$x].ParameterName -eq 'Name') { 22 | $describeName = $_.CommandElements[$x + 1].value 23 | } elseIf (($_.CommandElements[$x] -is [System.Management.Automation.Language.StringConstantExpressionAst]) -and ($_.CommandElements[$x - 1] -is [System.Management.Automation.Language.StringConstantExpressionAst])) { 24 | # if we have a string without a parameter name, return first hit. Name parameter is at position 0. 25 | $describeName = $_.CommandElements[$x].value 26 | } 27 | } 28 | $item = [pscustomobject][ordered]@{ 29 | Name = $describeName 30 | Tags = @() 31 | } 32 | 33 | # Get any tags defined 34 | $tagIndex = $_.CommandElements.IndexOf(($_.CommandElements | Where-Object ParameterName -eq 'Tag')) + 1 35 | if ($tagIndex -and $tagIndex -lt $_.CommandElements.Count) { 36 | $tagExtent = $_.CommandElements[$tagIndex].Extent 37 | 38 | $tagAST = [System.Management.Automation.Language.Parser]::ParseInput($tagExtent, [ref]$null, [ref]$null) 39 | 40 | # Try to get the tags as an array 41 | $tagElements = $tagAST.FindAll({$args[0] -is [System.Management.Automation.Language.ArrayLiteralAst]}, $true) 42 | if ($tagElements) { 43 | $item.Tags = $tagElements.SafeGetValue() 44 | } else { 45 | # Try to get the tag as a string 46 | $tagElements = $tagAST.FindAll({$args[0] -is [System.Management.Automation.Language.StringConstantExpressionAst]}, $true) 47 | if ($tagElements) { 48 | $item.Tags = @($tagElements.SafeGetValue()) 49 | } 50 | } 51 | } 52 | $item 53 | } 54 | $describes 55 | } 56 | -------------------------------------------------------------------------------- /OperationValidation/Private/Get-TestName.ps1: -------------------------------------------------------------------------------- 1 | 2 | function Get-TestName { 3 | param( 4 | $ast 5 | ) 6 | 7 | for($i = 1; $i -lt $ast.Parent.CommandElements.Count; $i++) { 8 | if ($ast.Parent.CommandElements[$i] -is 'System.Management.Automation.Language.CommandParameterAst') { 9 | $i++; continue 10 | } 11 | if ($ast.Parent.CommandElements[$i] -is 'System.Management.Automation.Language.ScriptBlockExpressionAst') { 12 | continue 13 | } 14 | if ($ast.Parent.CommandElements[$i] -is 'System.Management.Automation.Language.StringConstantExpressionAst') { 15 | return $ast.Parent.CommandElements[$i].Value 16 | } 17 | } 18 | 19 | throw 'Could not determine test name' 20 | } 21 | -------------------------------------------------------------------------------- /OperationValidation/Private/New-OperationValidationFailure.ps1: -------------------------------------------------------------------------------- 1 | 2 | function New-OperationValidationFailure { 3 | [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseShouldProcessForStateChangingFunctions', '', Scope='Function', Target='*')] 4 | param ( 5 | [Parameter(Mandatory)] 6 | [string]$StackTrace, 7 | 8 | [Parameter(Mandatory)] 9 | [string]$FailureMessage 10 | ) 11 | 12 | $o = [pscustomobject]@{ 13 | PSTypeName = 'OperationValidationFailure' 14 | StackTrace = $StackTrace 15 | FailureMessage = $FailureMessage 16 | } 17 | $toString = { return $this.StackTrace } 18 | Add-Member -Inputobject $o -MemberType ScriptMethod -Name ToString -Value $toString -Force 19 | $o 20 | } 21 | -------------------------------------------------------------------------------- /OperationValidation/Private/New-OperationValidationInfo.ps1: -------------------------------------------------------------------------------- 1 | 2 | function New-OperationValidationInfo { 3 | [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseShouldProcessForStateChangingFunctions', '', Scope='Function', Target='*')] 4 | param ( 5 | [Parameter(Mandatory)] 6 | [string]$File, 7 | 8 | [Parameter(Mandatory)] 9 | [string]$FilePath, 10 | 11 | [Parameter(Mandatory)] 12 | [string[]]$Name, 13 | 14 | [string[]]$TestCases, 15 | 16 | [Parameter(Mandatory)] 17 | [ValidateSet('None', 'Simple', 'Comprehensive')] 18 | [string]$Type, 19 | 20 | [string]$Modulename, 21 | 22 | [string[]]$Tags, 23 | 24 | [Version]$Version, 25 | 26 | [hashtable]$Parameters 27 | ) 28 | 29 | $o = [pscustomobject]@{ 30 | PSTypeName = 'OperationValidationInfo' 31 | File = $File 32 | FilePath = $FilePath 33 | Name = $Name 34 | TestCases = $testCases 35 | Type = $type 36 | ModuleName = $Modulename 37 | Tags = $Tags 38 | Version = $Version 39 | ScriptParameters = $Parameters 40 | } 41 | $toString = { return ('{0} ({1}): {2}' -f $this.testFile, $this.Type, ($this.TestCases -join ',')) } 42 | Add-Member -InputObject $o -MemberType ScriptMethod -Name ToString -Value $toString -Force 43 | $o 44 | } 45 | -------------------------------------------------------------------------------- /OperationValidation/Private/New-OperationValidationResult.ps1: -------------------------------------------------------------------------------- 1 | 2 | function New-OperationValidationResult { 3 | [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseShouldProcessForStateChangingFunctions', '', Scope='Function', Target='*')] 4 | param ( 5 | [Parameter(Mandatory)] 6 | [string]$FileName, 7 | 8 | [Parameter(Mandatory)] 9 | [string]$Name, 10 | 11 | [Parameter(Mandatory)] 12 | [string]$Result, 13 | 14 | [string]$Module, 15 | 16 | [object]$RawResult, 17 | 18 | [object]$Error 19 | ) 20 | 21 | $o = [pscustomobject]@{ 22 | PSTypeName = 'OperationValidationResult' 23 | Module = $Module 24 | FileName = $FileName 25 | ShortName = ([IO.Path]::GetFileName($FileName)) 26 | Name = $Name 27 | Result = $Result 28 | Error = $Error 29 | RawResult = $RawResult 30 | } 31 | $toString = { return ('{0} ({1}): {2}' -f $this.Module, $this.FileName, $this.Name) } 32 | Add-Member -InputObject $o -MemberType ScriptMethod -Name ToString -Value $toString -Force 33 | $o 34 | } 35 | -------------------------------------------------------------------------------- /OperationValidation/Private/Parse-Psd1.ps1: -------------------------------------------------------------------------------- 1 | 2 | function Parse-Psd1 { 3 | [CmdletBinding()] 4 | Param ( 5 | [Parameter(Mandatory)] 6 | [Microsoft.PowerShell.DesiredStateConfiguration.ArgumentToConfigurationDataTransformation()] 7 | [hashtable] $data 8 | ) 9 | $data 10 | } 11 | -------------------------------------------------------------------------------- /OperationValidation/Public/Get-OperationValidation.ps1: -------------------------------------------------------------------------------- 1 | function Get-OperationValidation { 2 | <# 3 | .SYNOPSIS 4 | Retrieve the operational tests from modules 5 | 6 | .DESCRIPTION 7 | Modules which include a Diagnostics directory are inspected for 8 | Pester tests in either the "Simple" or "Comprehensive" subdirectories. 9 | If files are found in those directories, they will be inspected to determine 10 | whether they are Pester tests. If Pester tests are found, the 11 | test names in those files will be returned. 12 | 13 | The module structure required is as follows: 14 | 15 | ModuleBase\ 16 | Diagnostics\ 17 | Simple # simple tests are held in this location 18 | (e.g., ping, serviceendpoint checks) 19 | Comprehensive # comprehensive scenario tests should be placed here 20 | 21 | .PARAMETER Name 22 | One or more module names to inspect and return if they adhere to the OVF Pester test structure. 23 | 24 | By default this is [*] which will inspect all modules in $env:PSModulePath. 25 | 26 | .PARAMETER Path 27 | One or more paths to search for OVF modules in. This bypasses searching the directories contained in $env:PSModulePath. 28 | 29 | .PARAMETER LiteralPath 30 | One or more literal paths to search for OVF modules in. This bypasses searching the directories contained in $env:PSModulePath. 31 | 32 | Unlike the Path parameter, the value of LiteralPath is used exactly as it is typed. 33 | No characters are interpreted as wildcards. If the path includes escape characters, enclose it in single quotation marks. Single quotation 34 | marks tell PowerShell not to interpret any characters as escape sequences. 35 | 36 | .PARAMETER TestType 37 | The type of tests to retrieve, this may be either "Simple", "Comprehensive", or Both ("Simple,Comprehensive"). 38 | "Simple, Comprehensive" is the default. 39 | 40 | .PARAMETER Version 41 | The version of the module to retrieve. If not specified, the latest version 42 | of the module will be retured. 43 | 44 | .PARAMETER Tag 45 | Executes tests with specified tag parameter values. Wildcard characters and tag values that include spaces 46 | or whitespace characters are not supported. 47 | 48 | When you specify multiple tag values, Get-OperationValidation executes tests that have any of the 49 | listed tags. If you use both Tag and ExcludeTag, ExcludeTag takes precedence. 50 | 51 | .PARAMETER ExcludeTag 52 | Omits tests with the specified tag parameter values. Wildcard characters and tag values that include spaces 53 | or whitespace characters are not supported. 54 | 55 | When you specify multiple ExcludeTag values, Get-OperationValidation omits tests that have any 56 | of the listed tags. If you use both Tag and ExcludeTag, ExcludeTag takes precedence. 57 | 58 | .EXAMPLE 59 | PS> Get-OperationValidation -Name OVF.Windows.Server 60 | 61 | Module: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2 62 | Version: 1.0.2 63 | Type: Simple 64 | Tags: {} 65 | File: LogicalDisk.tests.ps1 66 | FilePath: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2\Diagnostics\Simple\LogicalDisk.tests.ps1 67 | Name: 68 | Logical Disks 69 | 70 | 71 | Module: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2 72 | Version: 1.0.2 73 | Type: Simple 74 | Tags: {} 75 | File: Memory.tests.ps1 76 | FilePath: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2\Diagnostics\Simple\Memory.tests.ps1 77 | Name: 78 | Memory 79 | 80 | 81 | Module: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2 82 | Version: 1.0.2 83 | Type: Simple 84 | Tags: {} 85 | File: Network.tests.ps1 86 | FilePath: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2\Diagnostics\Simple\Network.tests.ps1 87 | Name: 88 | Network Adapters 89 | 90 | 91 | Module: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2 92 | Version: 1.0.2 93 | Type: Simple 94 | Tags: {} 95 | File: Services.tests.ps1 96 | FilePath: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2\Diagnostics\Simple\Services.tests.ps1 97 | Name: 98 | Operating System 99 | 100 | .EXAMPLE 101 | PS> $tests = Get-OperationValidation 102 | 103 | Search in all modules found in $env:PSModulePath for OVF tests. 104 | 105 | .EXAMPLE 106 | PS> $tests = Get-OperationValidation -Path C:\MyTests 107 | 108 | Search for OVF modules under c:\MyTests 109 | 110 | .EXAMPLE 111 | PS> $simpleTests = Get-OperationValidation -ModuleName OVF.Windows.Server -TypeType Simple 112 | 113 | Get just the simple tests in the OVF.Windows.Server module. 114 | 115 | .EXAMPLE 116 | $tests = Get-OperationValidation -ModuleName OVF.Windows.Server -Version 1.0.2 117 | 118 | Get all the tests from version 1.0.2 of the OVF.Windows.Server module. 119 | 120 | .EXAMPLE 121 | $storageTests = Get-OperationValidation -Tag Storage 122 | 123 | Search in all modules for OVF tests that include the tag Storage. 124 | 125 | .EXAMPLE 126 | $tests = Get-OperationValidation -ExcludeTag memory 127 | 128 | Search for OVF tests that don't include the tag Memory 129 | 130 | .LINK 131 | Invoke-OperationValidation 132 | 133 | #> 134 | [CmdletBinding(DefaultParameterSetName = 'ModuleName')] 135 | param ( 136 | [Parameter( 137 | ParameterSetName = 'ModuleName', 138 | Position = 0 139 | )] 140 | [Alias('ModuleName')] 141 | [string[]]$Name = '*', 142 | 143 | [parameter( 144 | Mandatory, 145 | ParameterSetName = 'Path', 146 | Position = 0, 147 | ValueFromPipeline, 148 | ValueFromPipelineByPropertyName 149 | )] 150 | [ValidateNotNullOrEmpty()] 151 | [SupportsWildcards()] 152 | [string[]]$Path, 153 | 154 | [parameter( 155 | Mandatory, 156 | ParameterSetName = 'LiteralPath', 157 | Position = 0, 158 | ValueFromPipelineByPropertyName 159 | )] 160 | [ValidateNotNullOrEmpty()] 161 | [Alias('PSPath')] 162 | [string[]]$LiteralPath, 163 | 164 | # [Parameter(ParameterSetName = 'ModuleName')] 165 | # [Parameter(ParameterSetName = 'Path')] 166 | # [Parameter(ParameterSetName = 'LiteralPath')] 167 | [ValidateSet('Simple', 'Comprehensive')] 168 | [string[]]$TestType = @('Simple', 'Comprehensive'), 169 | 170 | [Version]$Version, 171 | 172 | [string[]]$Tag, 173 | 174 | [string[]]$ExcludeTag 175 | ) 176 | 177 | PROCESS { 178 | Write-Progress -Activity 'Inspecting Modules' -Status ' ' 179 | 180 | # Resolve module list either by module name, path, or literalpath 181 | $modListParams = @{} 182 | switch ($PSCmdlet.ParameterSetName) { 183 | 'ModuleName' { 184 | $modListParams.Name = $Name 185 | break 186 | } 187 | 'Path' { 188 | $paths = Resolve-Path -Path $Path | Select-Object -ExpandProperty Path 189 | $modListParams.Path = $paths 190 | } 191 | 'LiteralPath' { 192 | $paths = Resolve-Path -LiteralPath $LiteralPath | Select-Object -ExpandProperty Path 193 | $modListParams.Path = $paths 194 | } 195 | } 196 | 197 | if ($PSBoundParameters.ContainsKey('Version')) { 198 | $modListParams.Version = $Version 199 | $moduleCollection = @(Get-ModuleList -Name $Name -Version $Version) 200 | } 201 | $moduleCollection = @(Get-ModuleList @modListParams) 202 | 203 | $count = 1 204 | $moduleCount = $moduleCollection.Count 205 | Write-Debug -Message "Found [$moduleCount] OVF modules" 206 | foreach($modulePath in $moduleCollection) { 207 | Write-Progress -Activity ("Searching for Diagnostics in $modulePath") -PercentComplete ($count++/$moduleCount*100) -status ' ' 208 | $diagnosticsDir = Join-Path -Path $modulePath -ChildPath 'Diagnostics' 209 | 210 | # Get the module manifest so we can pull out the version 211 | $modName = Split-Path -Path $modulePath -Leaf 212 | $manifestFile = Get-ChildItem -Path $modulePath -Filter "$($modName).psd1" 213 | if (-not $manifestFile) { 214 | if ("$modName" -as [version]) { 215 | # We are in a "version" directory so get the actual module name from the parent directory 216 | $parent = Split-Path -Path (Split-Path -Path $modulePath -Parent) -Leaf 217 | $manifestFile = Get-ChildItem -Path $modulePath -Filter "$($parent).psd1" 218 | } 219 | } 220 | 221 | # Some OVF modules might not have a manifest (.psd1) file. 222 | if ($manifestFile) { 223 | if ($PSVersionTable.PSVersion.Major -ge 5) { 224 | $manifest = Import-PowerShellDataFile -Path $manifestFile.FullName 225 | } else { 226 | $manifest = Parse-Psd1 $manifestFile.FullName 227 | } 228 | } else { 229 | $manifest = $null 230 | } 231 | 232 | if (Test-Path -Path $diagnosticsDir) { 233 | foreach($dir in $testType) { 234 | $testDir = Join-Path -Path $diagnosticsDir -ChildPath $dir 235 | if (-not (Test-Path -Path $testDir)) { 236 | continue 237 | } 238 | foreach($file in (Get-ChildItem -Path $testDir | Where-Object {$_.Name -like '*.tests.ps1'})) { 239 | # Pull out parameters to Pester script if they exist 240 | $script = Get-Command -Name $file.fullname 241 | $parameters = $script.Parameters 242 | if ($parameters.Keys.Count -gt 0) { 243 | Write-Debug -Message 'Test script has overrideable parameters' 244 | Write-Debug -Message "`n$($parameters.Keys | Out-String)" 245 | } 246 | 247 | $tests = @(Get-TestFromScript -ScriptPath $file.FullName) 248 | foreach ($test in $tests) { 249 | # Only return tests that match the tag filter(s) 250 | if ($Tag -and @(Compare-Object -ReferenceObject $Tag -DifferenceObject $test.Tags -IncludeEqual -ExcludeDifferent).count -eq 0) { continue } 251 | if ($ExcludeTag -and @(Compare-Object -ReferenceObject $ExcludeTag -DifferenceObject $test.Tags -IncludeEqual -ExcludeDifferent).count -gt 0) { continue } 252 | 253 | $modInfoParams = @{ 254 | FilePath = $file.Fullname 255 | File = $file.Name 256 | Type = $dir 257 | Name = $test.Name 258 | ModuleName = $modulePath 259 | Tags = $test.Tags 260 | Version = if ($manifest.ModuleVersion) { [version]$manifest.ModuleVersion } else { $null } 261 | Parameters = $parameters 262 | } 263 | New-OperationValidationInfo @modInfoParams 264 | } 265 | } 266 | } 267 | } 268 | } 269 | } 270 | } 271 | -------------------------------------------------------------------------------- /OperationValidation/Public/Invoke-OperationValidation.ps1: -------------------------------------------------------------------------------- 1 | 2 | function Invoke-OperationValidation { 3 | <# 4 | .SYNOPSIS 5 | Invoke the operational tests from modules 6 | 7 | .DESCRIPTION 8 | Modules which include Diagnostics tests are executed via this cmdlet 9 | 10 | .PARAMETER TestFilePath 11 | The path to a diagnostic test to execute. By default all discoverable diagnostics will be invoked 12 | 13 | .PARAMETER TestInfo 14 | The type of tests to invoke, this may be either "Simple", "Comprehensive" 15 | or Both ("Simple,Comprehensive"). "Simple,Comprehensive" is the default. 16 | 17 | .PARAMETER ModuleName 18 | By default this is * which will retrieve and execute all OVF modules in $env:psmodulepath 19 | Additional module directories may be added. If you wish to check both 20 | $env:psmodulepath and your own specific locations, use 21 | *, 22 | 23 | .PARAMETER Path 24 | One or more paths to search for OVF modules in. This bypasses searching the directories contained in $env:PSModulePath. 25 | 26 | .PARAMETER LiteralPath 27 | One or more literal paths to search for OVF modules in. This bypasses searching the directories contained in $env:PSModulePath. 28 | 29 | Unlike the Path parameter, the value of LiteralPath is used exactly as it is typed. 30 | No characters are interpreted as wildcards. If the path includes escape characters, enclose it in single quotation marks. Single quotation 31 | marks tell PowerShell not to interpret any characters as escape sequences. 32 | 33 | .PARAMETER TestType 34 | The type of tests to execute, this may be either "Simple", "Comprehensive" 35 | or Both ("Simple,Comprehensive"). "Simple,Comprehensive" is the default. 36 | 37 | .PARAMETER IncludePesterOutput 38 | Include the Pester output when execute the tests. 39 | 40 | .PARAMETER Version 41 | The version of the module to retrieve. If the specified, the latest version 42 | of the module will be retured. 43 | 44 | .PARAMETER Overrides 45 | If the Pester test(s) include script parameters, those parameters can be overridden by 46 | specifying a hashtable of values. The key(s) in the hashtable must match the parameter 47 | names in the Pester test. 48 | 49 | For example, if the Pester test includes a parameter block like the following, one or more of 50 | these parameters can be overriden using values from the hashtable passed to the -Overrides parameter. 51 | 52 | Pester test script: 53 | param( 54 | [int]$SomeValue = 100 55 | [bool]$ExtraChecks = $false 56 | ) 57 | 58 | Overrides the default parameter values: 59 | Invoke-OperationValidation -ModuleName MyModule -Overrides @{ SomeValue = 500; ExtraChecks = $true } 60 | 61 | .PARAMETER Tag 62 | Executes tests with specified tag parameter values. Wildcard characters and tag values that include spaces 63 | or whitespace characters are not supported. 64 | 65 | When you specify multiple tag values, Invoke-OperationValidation executes tests that have any of the 66 | listed tags. If you use both Tag and ExcludeTag, ExcludeTag takes precedence. 67 | 68 | .PARAMETER ExcludeTag 69 | Omits tests with the specified tag parameter values. Wildcard characters and tag values that include spaces 70 | or whitespace characters are not supported. 71 | 72 | When you specify multiple ExcludeTag values, Get-OperationValidation omits tests that have any 73 | of the listed tags. If you use both Tag and ExcludeTag, ExcludeTag takes precedence. 74 | 75 | .EXAMPLE 76 | PS> Get-OperationValidation -ModuleName OperationValidation | Invoke-OperationValidation -IncludePesterOutput 77 | Describing Simple Test Suite 78 | [+] first Operational test 20ms 79 | [+] second Operational test 19ms 80 | [+] third Operational test 9ms 81 | Tests completed in 48ms 82 | Passed: 3 Failed: 0 Skipped: 0 Pending: 0 83 | Describing Scenario targeted tests 84 | Context The RemoteAccess service 85 | [+] The service is running 37ms 86 | Context The Firewall Rules 87 | [+] A rule for TCP port 3389 is enabled 1.19s 88 | [+] A rule for UDP port 3389 is enabled 11ms 89 | Tests completed in 1.24s 90 | Passed: 3 Failed: 0 Skipped: 0 Pending: 0 91 | 92 | 93 | Module: OperationValidation 94 | 95 | Result Name 96 | ------- -------- 97 | Passed Simple Test Suite::first Operational test 98 | Passed Simple Test Suite::second Operational test 99 | Passed Simple Test Suite::third Operational test 100 | Passed Scenario targeted tests:The RemoteAccess service:The service is running 101 | Passed Scenario targeted tests:The Firewall Rules:A rule for TCP port 3389 is enabled 102 | Passed Scenario targeted tests:The Firewall Rules:A rule for UDP port 3389 is enabled 103 | 104 | .LINK 105 | Get-OperationValidation 106 | #> 107 | 108 | [CmdletBinding(SupportsShouldProcess = $true, DefaultParameterSetName = 'FileAndTest')] 109 | param ( 110 | [Parameter( 111 | ParameterSetName = 'TestFile', 112 | ValueFromPipelineByPropertyName = $true 113 | )] 114 | [string[]]$TestFilePath, 115 | 116 | [Parameter( 117 | ParameterSetName = 'FileAndTest', 118 | ValueFromPipeline 119 | )] 120 | [pscustomobject[]]$TestInfo, 121 | 122 | [Parameter(ParameterSetName = 'UseGetOperationTest')] 123 | [string[]]$ModuleName, 124 | 125 | [parameter( 126 | Mandatory, 127 | ParameterSetName = 'Path', 128 | Position = 0, 129 | ValueFromPipeline, 130 | ValueFromPipelineByPropertyName 131 | )] 132 | [ValidateNotNullOrEmpty()] 133 | [SupportsWildcards()] 134 | [string[]]$Path, 135 | 136 | [parameter( 137 | Mandatory, 138 | ParameterSetName = 'LiteralPath', 139 | Position = 0, 140 | ValueFromPipelineByPropertyName 141 | )] 142 | [ValidateNotNullOrEmpty()] 143 | [Alias('PSPath')] 144 | [string[]]$LiteralPath, 145 | 146 | [Parameter(ParameterSetName = 'Path')] 147 | [Parameter(ParameterSetName = 'LiteralPath')] 148 | [Parameter(ParameterSetName = 'UseGetOperationTest')] 149 | [ValidateSet('Simple', 'Comprehensive')] 150 | [string[]]$TestType = @('Simple', 'Comprehensive'), 151 | 152 | [switch]$IncludePesterOutput, 153 | 154 | [Parameter(ParameterSetName = 'Path')] 155 | [Parameter(ParameterSetName = 'LiteralPath')] 156 | [Parameter(ParameterSetName = 'UseGetOperationTest')] 157 | [Version]$Version, 158 | 159 | [Parameter(ParameterSetName = 'Path')] 160 | [Parameter(ParameterSetName = 'LiteralPath')] 161 | [Parameter(ParameterSetName = 'FileAndTest')] 162 | [Parameter(ParameterSetName = 'UseGetOperationTest')] 163 | [hashtable]$Overrides, 164 | 165 | [Parameter(ParameterSetName = 'Path')] 166 | [Parameter(ParameterSetName = 'LiteralPath')] 167 | [Parameter(ParameterSetName = 'UseGetOperationTest')] 168 | [string[]]$Tag, 169 | 170 | [Parameter(ParameterSetName = 'Path')] 171 | [Parameter(ParameterSetName = 'LiteralPath')] 172 | [Parameter(ParameterSetName = 'UseGetOperationTest')] 173 | [string[]]$ExcludeTag 174 | ) 175 | 176 | begin { 177 | $pesterMod = Get-Module -Name Pester 178 | if (-not $pesterMod) { 179 | if (Get-Module -Name Pester -ListAvailable) { 180 | $pesterMod = Import-Module -Name Pester -Verbose:$false -PassThru 181 | } else { 182 | Throw "Cannot load Pester module" 183 | } 184 | } 185 | 186 | if ($PSCmdLet.ParameterSetName -eq 'UseGetOperationTest') { 187 | if ([string]::IsNullOrEmpty($ModuleName)) { 188 | $ModuleName = '*' 189 | } 190 | } 191 | 192 | $resolveOvfTestParameterSetNames = 'UseGetOperationTest', 'Path', 'LiteralPath' 193 | } 194 | 195 | process { 196 | if ($PSCmdlet.ParameterSetName -in $resolveOvfTestParameterSetNames) { 197 | $getOvfParams = @{ 198 | TestType = $TestType 199 | } 200 | if ($PSBoundParameters.ContainsKey('Version')) { 201 | $getOvfParams.Version = $Version 202 | } 203 | if ($PSBoundParameters.ContainsKey('Tag')) { 204 | $getOvfParams.Tag = $Tag 205 | } 206 | if ($PSBoundParameters.ContainsKey('ExcludeTag')) { 207 | $getOvfParams.ExcludeTag = $ExcludeTag 208 | } 209 | 210 | if ($PSCmdlet.ParameterSetName -eq 'Path') { 211 | $getOvfParams.Path = $Path 212 | } elseIf ($PSCmdlet.ParameterSetName -eq 'LiteralPath') { 213 | $getOvfParams.LiteralPath = $LiteralPath 214 | } elseIf ($PSCmdLet.ParameterSetName -eq 'UseGetOperationTest') { 215 | $getOvfParams.ModuleName = $ModuleName 216 | } 217 | 218 | $testInfo = Get-OperationValidation @getOvfParams 219 | } 220 | 221 | if ($testInfo) { 222 | # first check to be sure all of the TestInfos are sane 223 | foreach($ti in $testinfo) { 224 | if (-not ($ti.FilePath -and $ti.Name)) { 225 | throw "TestInfo must contain the path and the list of tests" 226 | } 227 | } 228 | 229 | # first check to be sure all of the TestInfos are sane 230 | foreach($ti in $testinfo) { 231 | if (-not ($ti.FilePath -and $ti.Name)) { 232 | throw "TestInfo must contain the path and the list of tests" 233 | } 234 | } 235 | 236 | [int]$testCount = 0 237 | Write-Verbose -Message ("EXECUTING: {0} [{1}]" -f $ti.FilePath,($ti.Name -join ",")) 238 | foreach($ti in $testinfo) { 239 | Write-Progress -Activity "Executing: $($ti.Name)" -PercentComplete ($testCount++ / $($testinfo.Count) * 100) 240 | 241 | $pesterParams = @{ 242 | TestName = $ti.Name 243 | PassThru = $true 244 | Verbose = $false 245 | } 246 | 247 | # Pester 4.0.0 deprecated the 'Quiet' parameter in favor of 'Show' 248 | if ($pesterMod.Version -ge '4.0.0') { 249 | if ($IncludePesterOutput) { 250 | $pesterParams.Show = 'All' 251 | } else { 252 | $pesterParams.Show = 'None' 253 | } 254 | } else { 255 | $pesterParams.Quiet = -not $IncludePesterOutput 256 | } 257 | 258 | if ($ti.ScriptParameters) { 259 | Write-Debug -Message 'Test has script parameters' 260 | if ($PSBoundParameters.ContainsKey('Overrides')) { 261 | Write-Verbose -Message "Overriding with parameters:`n$($Overrides | Format-Table -Property Key, Value | Out-String)" 262 | $pesterParams.Script = @{ 263 | Path = $ti.FilePath 264 | Parameters = $Overrides 265 | } 266 | } else { 267 | Write-Debug -Message 'Using default parameters for test' 268 | $pesterParams.Path = $ti.FilePath 269 | } 270 | } else { 271 | $pesterParams.Path = $ti.FilePath 272 | } 273 | 274 | if ($PSBoundParameters.ContainsKey('Tag')) { 275 | $pesterParams.Tag = $Tag 276 | } 277 | 278 | if ($PSBoundParameters.ContainsKey('ExcludeTag')) { 279 | $pesterParams.ExcludeTag = $ExcludeTag 280 | } 281 | 282 | if ($PSCmdlet.ShouldProcess("$($ti.Name) [$($ti.FilePath)]")) { 283 | $testResult = Invoke-Pester @pesterParams 284 | if ($testResult) { 285 | Add-member -InputObject $testResult -MemberType NoteProperty -Name Path -Value $ti.FilePath 286 | Convert-TestResult -Result $testResult -ModuleName $ti.ModuleName 287 | } 288 | } 289 | } 290 | return 291 | } 292 | 293 | if ($TestFilePath) { 294 | $pesterParams = @{ 295 | PassThru = $true 296 | Verbose = $false 297 | } 298 | 299 | # Pester 4.0.0 deprecated the 'Quiet' parameter in favor of 'Show' 300 | if ($pesterMod.Version -ge '4.0.0') { 301 | if ($IncludePesterOutput) { 302 | $pesterParams.Show = 'All' 303 | } else { 304 | $pesterParams.Show = 'None' 305 | } 306 | } else { 307 | $pesterParams.Quiet = -not $IncludePesterOutput 308 | } 309 | 310 | foreach($filePath in $TestFilePath) { 311 | write-progress -Activity "Invoking tests in $filePath" 312 | if ($PSCmdlet.ShouldProcess($filePath)) { 313 | $testResult = Invoke-Pester $filePath @pesterParams 314 | Add-Member -InputObject $testResult -MemberType NoteProperty -Name Path -Value $filePath 315 | Convert-TestResult -Result $testResult 316 | } 317 | } 318 | } 319 | } 320 | } 321 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # Operation-Validation-Framework 3 | 4 | | Azure Pipelines | AppVeyor | PS Gallery | License 5 | |-----------------|----------|------------|---------| 6 | [![Azure Pipelines Build Status][azure-pipeline-badge]][azure-pipeline-build] | [![AppVeyor Build Status][appveyor-badge]][appveyor-build] | [![PowerShell Gallery][psgallery-badge]][psgallery] | [![License][license-badge]][license] 7 | 8 | A set of tools for executing validation of the operation of a system. 9 | It provides a way to organize and execute Pester tests which are written 10 | to validate operation (rather than limited feature tests) 11 | 12 | Modules which include a Diagnostics directory are inspected for 13 | Pester tests in either the "Simple" or "Comprehensive" directories. 14 | If files are found in those directories, they will be inspected to determine 15 | whether they are Pester tests. If Pester tests are found, the 16 | test names in those files will be returned. 17 | 18 | The module structure required is as follows: 19 | 20 | * ModuleBase\ 21 | * Diagnostics\ 22 | * Simple *simple tests are held in this location (e.g., ping, serviceendpoint checks)* 23 | * Comprehensive *comprehensive scenario tests should be placed here* 24 | 25 | 26 | It supplies two cmdlets: 27 | ``` 28 | PS# get-help *operationvalidation 29 | 30 | Name Category Synopsis 31 | ---- -------- -------- 32 | Get-OperationValidation Function Retrieve the operational tests from modules 33 | Invoke-OperationValidation Function Invoke the operational tests from modules 34 | ``` 35 | 36 | ## Examples 37 | ``` 38 | PS> Get-OperationValidation -ModuleName C:\temp\modules\AddNumbers 39 | 40 | 41 | Type: Simple 42 | File: addnum.tests.ps1 43 | FilePath: C:\temp\modules\AddNumbers\Diagnostics\Simple\addnum.tests.ps1 44 | Name: 45 | Add-Em 46 | Subtract em 47 | Add-Numbers 48 | Type: Comprehensive 49 | File: Comp.Adding.Tests.ps1 50 | FilePath: C:\temp\modules\AddNumbers\Diagnostics\Comprehensive\Comp.Adding.Tests.ps1 51 | Name: 52 | Comprehensive Adding Tests 53 | Comprehensive Subtracting Tests 54 | Comprehensive Examples 55 | 56 | 57 | PS> Invoke-OperationValidation -IncludePesterOutput 58 | 59 | Describing Simple Test Suite 60 | [+] first Operational test 20ms 61 | [+] second Operational test 19ms 62 | [+] third Operational test 9ms 63 | Tests completed in 48ms 64 | Passed: 3 Failed: 0 Skipped: 0 Pending: 0 65 | Describing Scenario targeted tests 66 | Context The RemoteAccess service 67 | [+] The service is running 37ms 68 | Context The Firewall Rules 69 | [+] A rule for TCP port 3389 is enabled 1.19s 70 | [+] A rule for UDP port 3389 is enabled 11ms 71 | Tests completed in 1.24s 72 | Passed: 3 Failed: 0 Skipped: 0 Pending: 0 73 | 74 | 75 | Module: OperationValidation 76 | 77 | Result Name 78 | ------- -------- 79 | Passed Simple Test Suite::first Operational test 80 | Passed Simple Test Suite::second Operational test 81 | Passed Simple Test Suite::third Operational test 82 | Passed Scenario targeted tests:The RemoteAccess service:The service is running 83 | Passed Scenario targeted tests:The Firewall Rules:A rule for TCP port 3389 is enabled 84 | Passed Scenario targeted tests:The Firewall Rules:A rule for UDP port 3389 is enabled 85 | 86 | ``` 87 | [azure-pipeline-badge]: https://dev.azure.com/devblackops/Operation-Validation-Framework/_apis/build/status/PowerShell.Operation-Validation-Framework?branchName=master 88 | [azure-pipeline-build]: https://dev.azure.com/devblackops/Operation-Validation-Framework/_build/latest?definitionId=4&branchName=master 89 | [appveyor-badge]: https://ci.appveyor.com/api/projects/status/rvbve3ajjtn4m0n2?svg=true 90 | [appveyor-build]: https://ci.appveyor.com/project/devblackops/operation-validation-framework-a635v 91 | [psgallery-badge]: https://img.shields.io/powershellgallery/dt/operationvalidation.svg 92 | [psgallery]: https://www.powershellgallery.com/packages/operationvalidation 93 | [license-badge]: https://img.shields.io/github/license/powerShell/operation-validation-framework.svg 94 | [license]: https://raw.githubusercontent.com/PowerShell/Operation-Validation-Framework/master/LICENSE -------------------------------------------------------------------------------- /TestArtifacts/Example.WindowsSearch/Diagnostics/Simple/WindowsSearch.Simple.Tests.ps1: -------------------------------------------------------------------------------- 1 | Describe 'Windows Search finds the proper application' { 2 | It 'Finds notepad' { 3 | $con = New-Object -com ADODB.Connection 4 | $rs = New-Object -com ADODB.RecordSet 5 | $con.Open("Provider=Search.CollatorDSO;Extended properties='Application=Windows';") 6 | $rs.Open("Select System.ItemPathDisplay FROM SYSTEMINDEX Where System.FileName = 'notepad.exe'", $con) 7 | $rs.MoveFirst() 8 | $notepads = @() 9 | while (-not $rs.EOF) { 10 | $notepads += $rs.Fields['System.ItemPathDisplay'].Value 11 | $rs.MoveNext() 12 | } 13 | $rs.Close() 14 | $con.Close() 15 | $notepadPath = 'C:\Windows\System32\notepad.exe' 16 | $notepads -eq $notepadpath | Should be $notepadpath 17 | } 18 | } 19 | -------------------------------------------------------------------------------- /TestArtifacts/SingleTest.tests.ps1: -------------------------------------------------------------------------------- 1 | 2 | Describe 'Single Test' { 3 | it 'should work' { 4 | $true | should be $true 5 | } 6 | } -------------------------------------------------------------------------------- /TestArtifacts/VersionedModule/1.0.0/Diagnostics/Simple/PSGallery.Simple.Tests.ps1: -------------------------------------------------------------------------------- 1 | param( 2 | [string]$WebsiteUrl = 'https://www.powershellgallery.com', 3 | [string]$StatusCode = 200 4 | ) 5 | 6 | Describe 'Simple Validation of PSGallery' -Tag 'AAABBBCCC' { 7 | It 'The PowerShell Gallery should be responsive' { 8 | $response = Invoke-WebRequest -Uri $WebsiteUrl -UseBasicParsing 9 | $response.StatusCode | Should Be $StatusCode 10 | } 11 | 12 | it 'Has correct test parameters' { 13 | $WebsiteUrl | Should Be 'https://www.powershellgallery.com' 14 | $StatusCode | Should Be 200 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /TestArtifacts/VersionedModule/1.0.0/VersionedModule.psd1: -------------------------------------------------------------------------------- 1 | # 2 | # Module manifest for module 'VersionedModule' 3 | # 4 | # Generated by: Brandon Olin 5 | # 6 | # Generated on: 3/23/2017 7 | # 8 | 9 | @{ 10 | 11 | # Script module or binary module file associated with this manifest. 12 | # RootModule = '' 13 | 14 | # Version number of this module. 15 | ModuleVersion = '1.0.0' 16 | 17 | # Supported PSEditions 18 | # CompatiblePSEditions = @() 19 | 20 | # ID used to uniquely identify this module 21 | GUID = 'ccb1afd1-7ed8-4097-87d2-d7a375b04528' 22 | 23 | # Author of this module 24 | Author = 'Brandon Olin' 25 | 26 | # Company or vendor of this module 27 | CompanyName = 'Community' 28 | 29 | # Copyright statement for this module 30 | Copyright = '(c) 2017 Brandon Olin. All rights reserved.' 31 | 32 | # Description of the functionality provided by this module 33 | Description = 'Test OVF module' 34 | 35 | # Minimum version of the Windows PowerShell engine required by this module 36 | # PowerShellVersion = '' 37 | 38 | # Name of the Windows PowerShell host required by this module 39 | # PowerShellHostName = '' 40 | 41 | # Minimum version of the Windows PowerShell host required by this module 42 | # PowerShellHostVersion = '' 43 | 44 | # Minimum version of Microsoft .NET Framework required by this module. This prerequisite is valid for the PowerShell Desktop edition only. 45 | # DotNetFrameworkVersion = '' 46 | 47 | # Minimum version of the common language runtime (CLR) required by this module. This prerequisite is valid for the PowerShell Desktop edition only. 48 | # CLRVersion = '' 49 | 50 | # Processor architecture (None, X86, Amd64) required by this module 51 | # ProcessorArchitecture = '' 52 | 53 | # Modules that must be imported into the global environment prior to importing this module 54 | # RequiredModules = @() 55 | 56 | # Assemblies that must be loaded prior to importing this module 57 | # RequiredAssemblies = @() 58 | 59 | # Script files (.ps1) that are run in the caller's environment prior to importing this module. 60 | # ScriptsToProcess = @() 61 | 62 | # Type files (.ps1xml) to be loaded when importing this module 63 | # TypesToProcess = @() 64 | 65 | # Format files (.ps1xml) to be loaded when importing this module 66 | # FormatsToProcess = @() 67 | 68 | # Modules to import as nested modules of the module specified in RootModule/ModuleToProcess 69 | # NestedModules = @() 70 | 71 | # Functions to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no functions to export. 72 | FunctionsToExport = @() 73 | 74 | # Cmdlets to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no cmdlets to export. 75 | CmdletsToExport = @() 76 | 77 | # Variables to export from this module 78 | VariablesToExport = '*' 79 | 80 | # Aliases to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no aliases to export. 81 | AliasesToExport = @() 82 | 83 | # DSC resources to export from this module 84 | # DscResourcesToExport = @() 85 | 86 | # List of all modules packaged with this module 87 | # ModuleList = @() 88 | 89 | # List of all files packaged with this module 90 | # FileList = @() 91 | 92 | # Private data to pass to the module specified in RootModule/ModuleToProcess. This may also contain a PSData hashtable with additional module metadata used by PowerShell. 93 | PrivateData = @{ 94 | 95 | PSData = @{ 96 | 97 | # Tags applied to this module. These help with module discovery in online galleries. 98 | # Tags = @() 99 | 100 | # A URL to the license for this module. 101 | # LicenseUri = '' 102 | 103 | # A URL to the main website for this project. 104 | # ProjectUri = '' 105 | 106 | # A URL to an icon representing this module. 107 | # IconUri = '' 108 | 109 | # ReleaseNotes of this module 110 | # ReleaseNotes = '' 111 | 112 | } # End of PSData hashtable 113 | 114 | } # End of PrivateData hashtable 115 | 116 | # HelpInfo URI of this module 117 | # HelpInfoURI = '' 118 | 119 | # Default prefix for commands exported from this module. Override the default prefix using Import-Module -Prefix. 120 | # DefaultCommandPrefix = '' 121 | 122 | } 123 | 124 | -------------------------------------------------------------------------------- /TestArtifacts/VersionedModule/2.0.0/Diagnostics/Simple/PSGallery.Simple.Tests.ps1: -------------------------------------------------------------------------------- 1 | Describe 'Simple Validation of PSGallery' -Tag 'AAABBBCCC' { 2 | It 'The PowerShell Gallery should be responsive' { 3 | $response = Invoke-WebRequest -Uri 'https://www.powershellgallery.com' -UseBasicParsing 4 | $response.StatusCode | Should be 200 5 | } 6 | } 7 | 8 | Describe 'Simple Validation of Microsoft' -Tag 'AAABBBCCC', 'XXXYYYZZZ' { 9 | It 'Microsoft should be responsive' { 10 | $response = Invoke-WebRequest -Uri 'https://www.microsoft.com' -UseBasicParsing 11 | $response.StatusCode | Should be 200 12 | } 13 | } 14 | 15 | 16 | Describe 'Simple Validation of Github' -Tag 'JJJKKKLLL' { 17 | It 'GitHub should be responsive' { 18 | [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 19 | $response = Invoke-WebRequest -Uri 'https://www.github.com' -UseBasicParsing 20 | $response.StatusCode | Should be 200 21 | } 22 | } 23 | -------------------------------------------------------------------------------- /TestArtifacts/VersionedModule/2.0.0/VersionedModule.psd1: -------------------------------------------------------------------------------- 1 | # 2 | # Module manifest for module 'VersionedModule' 3 | # 4 | # Generated by: Brandon Olin 5 | # 6 | # Generated on: 3/23/2017 7 | # 8 | 9 | @{ 10 | 11 | # Script module or binary module file associated with this manifest. 12 | # RootModule = '' 13 | 14 | # Version number of this module. 15 | ModuleVersion = '2.0.0' 16 | 17 | # Supported PSEditions 18 | # CompatiblePSEditions = @() 19 | 20 | # ID used to uniquely identify this module 21 | GUID = 'ccb1afd1-7ed8-4097-87d2-d7a375b04528' 22 | 23 | # Author of this module 24 | Author = 'Brandon Olin' 25 | 26 | # Company or vendor of this module 27 | CompanyName = 'Community' 28 | 29 | # Copyright statement for this module 30 | Copyright = '(c) 2017 Brandon Olin. All rights reserved.' 31 | 32 | # Description of the functionality provided by this module 33 | Description = 'Test OVF module' 34 | 35 | # Minimum version of the Windows PowerShell engine required by this module 36 | # PowerShellVersion = '' 37 | 38 | # Name of the Windows PowerShell host required by this module 39 | # PowerShellHostName = '' 40 | 41 | # Minimum version of the Windows PowerShell host required by this module 42 | # PowerShellHostVersion = '' 43 | 44 | # Minimum version of Microsoft .NET Framework required by this module. This prerequisite is valid for the PowerShell Desktop edition only. 45 | # DotNetFrameworkVersion = '' 46 | 47 | # Minimum version of the common language runtime (CLR) required by this module. This prerequisite is valid for the PowerShell Desktop edition only. 48 | # CLRVersion = '' 49 | 50 | # Processor architecture (None, X86, Amd64) required by this module 51 | # ProcessorArchitecture = '' 52 | 53 | # Modules that must be imported into the global environment prior to importing this module 54 | # RequiredModules = @() 55 | 56 | # Assemblies that must be loaded prior to importing this module 57 | # RequiredAssemblies = @() 58 | 59 | # Script files (.ps1) that are run in the caller's environment prior to importing this module. 60 | # ScriptsToProcess = @() 61 | 62 | # Type files (.ps1xml) to be loaded when importing this module 63 | # TypesToProcess = @() 64 | 65 | # Format files (.ps1xml) to be loaded when importing this module 66 | # FormatsToProcess = @() 67 | 68 | # Modules to import as nested modules of the module specified in RootModule/ModuleToProcess 69 | # NestedModules = @() 70 | 71 | # Functions to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no functions to export. 72 | FunctionsToExport = @() 73 | 74 | # Cmdlets to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no cmdlets to export. 75 | CmdletsToExport = @() 76 | 77 | # Variables to export from this module 78 | VariablesToExport = '*' 79 | 80 | # Aliases to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no aliases to export. 81 | AliasesToExport = @() 82 | 83 | # DSC resources to export from this module 84 | # DscResourcesToExport = @() 85 | 86 | # List of all modules packaged with this module 87 | # ModuleList = @() 88 | 89 | # List of all files packaged with this module 90 | # FileList = @() 91 | 92 | # Private data to pass to the module specified in RootModule/ModuleToProcess. This may also contain a PSData hashtable with additional module metadata used by PowerShell. 93 | PrivateData = @{ 94 | 95 | PSData = @{ 96 | 97 | # Tags applied to this module. These help with module discovery in online galleries. 98 | # Tags = @() 99 | 100 | # A URL to the license for this module. 101 | # LicenseUri = '' 102 | 103 | # A URL to the main website for this project. 104 | # ProjectUri = '' 105 | 106 | # A URL to an icon representing this module. 107 | # IconUri = '' 108 | 109 | # ReleaseNotes of this module 110 | # ReleaseNotes = '' 111 | 112 | } # End of PSData hashtable 113 | 114 | } # End of PrivateData hashtable 115 | 116 | # HelpInfo URI of this module 117 | # HelpInfoURI = '' 118 | 119 | # Default prefix for commands exported from this module. Override the default prefix using Import-Module -Prefix. 120 | # DefaultCommandPrefix = '' 121 | 122 | } 123 | 124 | -------------------------------------------------------------------------------- /Tests/Meta/Help.tests.ps1: -------------------------------------------------------------------------------- 1 | # Taken with love from @juneb_get_help (https://raw.githubusercontent.com/juneb/PesterTDD/master/Module.Help.Tests.ps1) 2 | 3 | $moduleName = $env:BHProjectName 4 | $moduleManifest = Join-Path -Path $env:BHModulePath -ChildPath "$($moduleName).psd1" 5 | $testModuleDir = (Resolve-Path -Path (Join-Path -Path $env:BHProjectPath -ChildPath TestArtifacts)).Path 6 | 7 | # Get module commands 8 | # Remove all versions of the module from the session. Pester can't handle multiple versions. 9 | $pathSeparator = [IO.Path]::PathSeparator 10 | $savedModulePath = $env:PSModulePath 11 | if ($env:PSModulePath.split($pathSeparator) -notcontains $testModuleDir) { 12 | $env:PSModulePath += ($pathSeparator + $testModuleDir) 13 | } 14 | if ($env:PSModulePath.Split($pathSeparator) -notcontains $env:BHModulePath) { 15 | $env:PSModulePath += ($pathSeparator + $env:BHProjectPath) 16 | } 17 | Remove-Module Microsoft.PowerShell.Operation.Validation -Force -ErrorAction SilentlyContinue -Verbose:$false 18 | Import-Module $env:BHModulePath -Force -Verbose:$false 19 | 20 | $moduleVersion = (Test-ModuleManifest $moduleManifest -Verbose:$false | Select-Object -ExpandProperty Version).ToString() 21 | $ms = [Microsoft.PowerShell.Commands.ModuleSpecification]@{ ModuleName = $moduleName; RequiredVersion = $moduleVersion } 22 | $commands = Get-Command -FullyQualifiedModule $ms -CommandType Cmdlet, Function, Workflow # Not alias 23 | 24 | ## When testing help, remember that help is cached at the beginning of each session. 25 | ## To test, restart session. 26 | 27 | Describe 'Module help' { 28 | 29 | foreach ($command in $commands) { 30 | $commandName = $command.Name 31 | 32 | # The module-qualified command fails on Microsoft.PowerShell.Archive cmdlets 33 | $help = Get-Help $commandName -ErrorAction SilentlyContinue -Verbose:$false 34 | 35 | Describe "[$commandName]" { 36 | 37 | # If help is not found, synopsis in auto-generated help is the syntax diagram 38 | It "Is not auto-generated" { 39 | $help.Synopsis | Should Not BeLike '*`[``]*' 40 | } 41 | 42 | # Should be a description for every function 43 | It "Has description" { 44 | $help.Description | Should Not BeNullOrEmpty 45 | } 46 | 47 | # Should be at least one example 48 | It "Has example code" { 49 | ($help.Examples.Example | Select-Object -First 1).Code | Should Not BeNullOrEmpty 50 | } 51 | 52 | # Should be at least one example description 53 | It "Has example help" { 54 | ($help.Examples.Example.Remarks | Select-Object -First 1).Text | Should Not BeNullOrEmpty 55 | } 56 | 57 | Context 'Parameters' { 58 | 59 | $common = 'Debug', 'ErrorAction', 'ErrorVariable', 'InformationAction', 'InformationVariable', 'OutBuffer', 'OutVariable', 60 | 'PipelineVariable', 'Verbose', 'WarningAction', 'WarningVariable', 'Confirm', 'Whatif' 61 | 62 | $parameters = $command.ParameterSets.Parameters | Sort-Object -Property Name -Unique | Where-Object { $_.Name -notin $common } 63 | $parameterNames = $parameters.Name 64 | 65 | ## Without the filter, WhatIf and Confirm parameters are still flagged in "finds help parameter in code" test 66 | $helpParameters = $help.Parameters.Parameter | Where-Object { $_.Name -notin $common } | Sort-Object -Property Name -Unique 67 | $helpParameterNames = $helpParameters.Name 68 | 69 | foreach ($parameter in $parameters) { 70 | $parameterName = $parameter.Name 71 | $parameterHelp = $help.parameters.parameter | Where-Object Name -EQ $parameterName 72 | 73 | # Should be a description for every parameter 74 | It "$parameterName`: Has description" { 75 | $parameterHelp.Description.Text | Should Not BeNullOrEmpty 76 | } 77 | 78 | # Required value in Help should match IsMandatory property of parameter 79 | It "$parameterName`: IsMandatory is correct" { 80 | $codeMandatory = $parameter.IsMandatory.toString() 81 | $parameterHelp.Required | Should Be $codeMandatory 82 | } 83 | 84 | # Parameter type in Help should match code 85 | # It "help for $commandName has correct parameter type for $parameterName" { 86 | # $codeType = $parameter.ParameterType.Name 87 | # # To avoid calling Trim method on a null object. 88 | # $helpType = if ($parameterHelp.parameterValue) { $parameterHelp.parameterValue.Trim() } 89 | # $helpType | Should be $codeType 90 | # } 91 | } 92 | 93 | context 'Help parameters' { 94 | foreach ($helpParm in $HelpParameterNames) { 95 | # Shouldn't find extra parameters in help. 96 | It "[$helpParm] found in parameter help" { 97 | $helpParm -in $parameterNames | Should Be $true 98 | } 99 | } 100 | } 101 | } 102 | 103 | if ($help.relatedLinks.navigationLink.uri) { 104 | Context "Help Links" { 105 | foreach ($link in $help.relatedLinks.navigationLink.uri) { 106 | # Should have a valid uri if one is provided. 107 | it "[$link] should have 200 Status Code for $commandName" { 108 | $Results = Invoke-WebRequest -Uri $link -UseBasicParsing 109 | $Results.StatusCode | Should Be '200' 110 | } 111 | } 112 | } 113 | } 114 | } 115 | } 116 | } 117 | 118 | $env:PSModulePath = $savedModulePath 119 | Remove-Module OperationValidation 120 | -------------------------------------------------------------------------------- /Tests/Meta/Manifest.tests.ps1: -------------------------------------------------------------------------------- 1 | $moduleName = $env:BHProjectName 2 | $changelogPath = Join-Path -Path $env:BHProjectPath -Child "CHANGELOG.md" 3 | 4 | Describe 'Module manifest' { 5 | Context 'Validation' { 6 | 7 | $script:manifest = $null 8 | 9 | It "has a valid manifest" { 10 | { 11 | $script:manifest = Test-ModuleManifest -Path $env:BHPSModuleManifest -ErrorAction Stop -WarningAction SilentlyContinue -Verbose:$false 12 | } | Should Not Throw 13 | } 14 | 15 | It "has a valid name in the manifest" { 16 | $script:manifest.Name | Should Be $env:BHProjectName 17 | } 18 | 19 | It 'has a valid root module' { 20 | $script:manifest.RootModule | Should Be "$moduleName.psm1" 21 | } 22 | 23 | It "has a valid version in the manifest" { 24 | $script:manifest.Version -as [Version] | Should Not BeNullOrEmpty 25 | } 26 | 27 | It 'has a valid description' { 28 | $script:manifest.Description | Should Not BeNullOrEmpty 29 | } 30 | 31 | It 'has a valid author' { 32 | $script:manifest.Author | Should Not BeNullOrEmpty 33 | } 34 | 35 | It 'has a valid guid' { 36 | { 37 | [guid]::Parse($script:manifest.Guid) 38 | } | Should Not throw 39 | } 40 | 41 | It 'has a valid copyright' { 42 | $script:manifest.CopyRight | Should Not BeNullOrEmpty 43 | } 44 | 45 | $script:changelogVersion = $null 46 | It "has a valid version in the changelog" { 47 | foreach ($line in (Get-Content $changelogPath)) { 48 | if ($line -match "^##\s\[(?(\d+\.){1,3}\d+)\]") { 49 | $script:changelogVersion = $matches.Version 50 | break 51 | } 52 | } 53 | $script:changelogVersion | Should Not BeNullOrEmpty 54 | $script:changelogVersion -as [Version] | Should Not BeNullOrEmpty 55 | } 56 | 57 | It "changelog and manifest versions are the same" { 58 | $script:changelogVersion -as [Version] | Should be ($script:manifest.Version -as [Version]) 59 | } 60 | 61 | if (Get-Command -Name 'git.exe' -ErrorAction SilentlyContinue) { 62 | $script:tagVersion = $null 63 | It "is tagged with a valid version" -skip { 64 | $thisCommit = git.exe log --decorate --oneline HEAD~1..HEAD 65 | 66 | if ($thisCommit -match 'tag:\s*(\d+(?:\.\d+)*)') { 67 | $script:tagVersion = $matches[1] 68 | } 69 | 70 | $script:tagVersion | Should Not BeNullOrEmpty 71 | $script:tagVersion -as [Version] | Should Not BeNullOrEmpty 72 | } 73 | 74 | It "all versions are the same" { 75 | $script:changelogVersion -as [Version] | Should be ($script:manifest.Version -as [Version]) 76 | } 77 | } 78 | } 79 | } 80 | -------------------------------------------------------------------------------- /Tests/Meta/Meta.tests.ps1: -------------------------------------------------------------------------------- 1 | 2 | Set-StrictMode -Version latest 3 | 4 | # Make sure MetaFixers.psm1 is loaded - it contains Get-TextFilesList 5 | Import-Module -Name (Join-Path -Path $PSScriptRoot -ChildPath 'MetaFixers.psm1') -Verbose:$false -Force 6 | 7 | $projectRoot = $ENV:BHProjectPath 8 | if(-not $projectRoot) { 9 | $projectRoot = $PSScriptRoot 10 | } 11 | 12 | Describe 'Text files formatting' { 13 | 14 | $allTextFiles = Get-TextFilesList $projectRoot 15 | 16 | Context 'Files encoding' { 17 | It "Doesn't use Unicode encoding" { 18 | $unicodeFilesCount = 0 19 | $allTextFiles | ForEach-Object { 20 | if (Test-FileUnicode $_) { 21 | $unicodeFilesCount += 1 22 | Write-Warning "File $($_.FullName) contains 0x00 bytes. It's probably uses Unicode and need to be converted to UTF-8. Use Fixer 'Get-UnicodeFilesList `$pwd | ConvertTo-UTF8'." 23 | } 24 | } 25 | $unicodeFilesCount | Should Be 0 26 | } 27 | } 28 | 29 | Context 'Indentations' { 30 | It 'Uses spaces for indentation, not tabs' { 31 | $totalTabsCount = 0 32 | $allTextFiles | ForEach-Object { 33 | $fileName = $_.FullName 34 | (Get-Content $_.FullName -Raw) | Select-String "`t" | % { 35 | Write-Warning "There are tab in $fileName. Use Fixer 'Get-TextFilesList `$pwd | ConvertTo-SpaceIndentation'." 36 | $totalTabsCount++ 37 | } 38 | } 39 | $totalTabsCount | Should Be 0 40 | } 41 | } 42 | } 43 | -------------------------------------------------------------------------------- /Tests/Meta/MetaFixers.psm1: -------------------------------------------------------------------------------- 1 | 2 | # Taken with love from https://github.com/PowerShell/DscResource.Tests/blob/master/MetaFixers.psm1 3 | 4 | <# 5 | This module helps fix problems, found by Meta.Tests.ps1 6 | #> 7 | 8 | $ErrorActionPreference = 'stop' 9 | Set-StrictMode -Version latest 10 | 11 | function ConvertTo-UTF8() { 12 | [CmdletBinding()] 13 | [OutputType([void])] 14 | param( 15 | [Parameter(ValueFromPipeline=$true, Mandatory=$true)] 16 | [System.IO.FileInfo]$FileInfo 17 | ) 18 | 19 | process { 20 | $content = Get-Content -Raw -Encoding Unicode -Path $FileInfo.FullName 21 | [System.IO.File]::WriteAllText($FileInfo.FullName, $content, [System.Text.Encoding]::UTF8) 22 | } 23 | } 24 | 25 | function ConvertTo-SpaceIndentation() { 26 | [CmdletBinding()] 27 | [OutputType([void])] 28 | param( 29 | [Parameter(ValueFromPipeline=$true, Mandatory=$true)] 30 | [System.IO.FileInfo]$FileInfo 31 | ) 32 | 33 | process { 34 | $content = (Get-Content -Raw -Path $FileInfo.FullName) -replace "`t",' ' 35 | [System.IO.File]::WriteAllText($FileInfo.FullName, $content) 36 | } 37 | } 38 | 39 | function Get-TextFilesList { 40 | [CmdletBinding()] 41 | [OutputType([System.IO.FileInfo])] 42 | param( 43 | [Parameter(Mandatory=$true)] 44 | [string]$root 45 | ) 46 | Get-ChildItem -File -Recurse $root | Where-Object { @('.gitignore', '.gitattributes', '.ps1', '.psm1', '.psd1', '.json', '.xml', '.cmd', '.mof') -contains $_.Extension } 47 | } 48 | 49 | function Test-FileUnicode { 50 | [CmdletBinding()] 51 | [OutputType([bool])] 52 | param( 53 | [Parameter(ValueFromPipeline=$true, Mandatory=$true)] 54 | [System.IO.FileInfo]$fileInfo 55 | ) 56 | 57 | process { 58 | $path = $fileInfo.FullName 59 | $bytes = [System.IO.File]::ReadAllBytes($path) 60 | $zeroBytes = @($bytes -eq 0) 61 | [bool]$zeroBytes.Length 62 | } 63 | } 64 | 65 | function Get-UnicodeFilesList() { 66 | [CmdletBinding()] 67 | [OutputType([System.IO.FileInfo])] 68 | param( 69 | [Parameter(Mandatory=$true)] 70 | [string]$root 71 | ) 72 | 73 | Get-TextFilesList $root | ? { Test-FileUnicode $_ } 74 | } 75 | -------------------------------------------------------------------------------- /Tests/OperationValidation.Tests.ps1: -------------------------------------------------------------------------------- 1 | 2 | $testModuleDir = (Resolve-Path -Path (Join-Path -Path $env:BHProjectPath -ChildPath TestArtifacts)).Path 3 | 4 | Describe 'OperationValidation Module Tests' { 5 | 6 | BeforeAll { 7 | $pathSeparator = [IO.Path]::PathSeparator 8 | $SavedModulePath = $env:PSModulePath 9 | if ($env:PSModulePath.split($pathSeparator) -notcontains $testModuleDir) { 10 | $env:PSModulePath += ($pathSeparator + $testModuleDir) 11 | } 12 | if ($env:PSModulePath.Split($pathSeparator) -notcontains $env:BHModulePath) { 13 | $env:PSModulePath += ($pathSeparator + $env:BHProjectPath) 14 | } 15 | Remove-Module Microsoft.PowerShell.Operation.Validation -Force -ErrorAction SilentlyContinue 16 | Import-Module $env:BHModulePath -Force 17 | $commands = Get-Command -Module OperationValidation | Sort-object Name 18 | } 19 | 20 | AfterAll { 21 | $env:PSModulePath = $savedModulePath 22 | Remove-Module OperationValidation 23 | } 24 | 25 | Context 'Get-OperationValidation parameters' { 26 | It 'ModuleName parameter is proper type' { 27 | $commands[0].Parameters['Name'].ParameterType | Should be ([System.String[]]) 28 | } 29 | It 'Version parameter is proper type' { 30 | $commands[0].Parameters['Version'].ParameterType | Should be ([System.Version]) 31 | } 32 | It 'TestType parameter is proper type' { 33 | $commands[0].Parameters['TestType'].ParameterType | Should be ([System.String[]]) 34 | } 35 | It 'Tag parameter is property type' { 36 | $commands[0].Parameters['Tag'].ParameterType | Should be ([System.String[]]) 37 | } 38 | It 'ExcludeTag parameter is property type' { 39 | $commands[0].Parameters['Tag'].ParameterType | Should be ([System.String[]]) 40 | } 41 | It 'TestType parameter has proper constraints' { 42 | $Commands[0].Parameters['TestType'].Attributes.ValidValues.Count | should be 2 43 | $Commands[0].Parameters['TestType'].Attributes.ValidValues -eq 'Simple' | Should be 'Simple' 44 | $Commands[0].Parameters['TestType'].Attributes.ValidValues -eq 'Comprehensive' | Should be 'Comprehensive' 45 | } 46 | } 47 | Context 'Invoke-OperationValidation parameters' { 48 | It 'TestFilePath parameter is proper type' { 49 | $commands[1].Parameters['TestFilePath'].ParameterType | Should be ([System.String[]]) 50 | } 51 | It 'TestInfo parameter is proper type' { 52 | $commands[1].Parameters['TestInfo'].ParameterType | Should be ([System.Management.Automation.PSObject[]]) 53 | } 54 | It 'ModuleName parameter is proper type' { 55 | $commands[1].Parameters['ModuleName'].ParameterType | Should be ([System.String[]]) 56 | } 57 | It 'Version parameter is proper type' { 58 | $commands[1].Parameters['Version'].ParameterType | Should be ([System.Version]) 59 | } 60 | It 'Overrides parameter is proper type' { 61 | $commands[1].Parameters['Overrides'].ParameterType | Should be ([System.Collections.Hashtable]) 62 | } 63 | It 'IncludePesterOutput is proper type' { 64 | $commands[1].Parameters['IncludePesterOutput'].ParameterType | Should be ([System.Management.Automation.SwitchParameter]) 65 | } 66 | It 'TestType parameter is proper type' { 67 | $commands[1].Parameters['TestType'].ParameterType | Should be ([System.String[]]) 68 | } 69 | It 'Tag parameter is proper type' { 70 | $commands[1].Parameters['Tag'].ParameterType | Should be ([System.String[]]) 71 | } 72 | It 'ExcludeTag parameter is proper type' { 73 | $commands[1].Parameters['ExcludeTag'].ParameterType | Should be ([System.String[]]) 74 | } 75 | It 'TestType parameter has proper constraints' { 76 | $Commands[1].Parameters['TestType'].Attributes.ValidValues.Count | should be 2 77 | $Commands[1].Parameters['TestType'].Attributes.ValidValues -eq 'Simple' | Should be 'Simple' 78 | $Commands[1].Parameters['TestType'].Attributes.ValidValues -eq 'Comprehensive' | Should be 'Comprehensive' 79 | } 80 | } 81 | } 82 | -------------------------------------------------------------------------------- /Tests/Unit/Get-OperationValidation.tests.ps1: -------------------------------------------------------------------------------- 1 | 2 | $testModuleDir = (Resolve-Path -Path (Join-Path -Path $env:BHProjectPath -ChildPath TestArtifacts)).Path 3 | 4 | Describe 'Get-OperationValidation' { 5 | 6 | BeforeAll { 7 | $pathSeparator = [IO.Path]::PathSeparator 8 | $savedModulePath = $env:PSModulePath 9 | if ($testModuleDir -notin $env:PSModulePath.split($pathSeparator)) { 10 | $env:PSModulePath += ($pathSeparator + $testModuleDir) 11 | } 12 | if ($env:BHProjectPath -notin $env:PSModulePath.Split($pathSeparator)) { 13 | $env:PSModulePath += ($pathSeparator + $env:BHProjectPath) 14 | } 15 | Remove-Module Microsoft.PowerShell.Operation.Validation -Force -ErrorAction SilentlyContinue -Verbose:$false 16 | Import-Module $env:BHModulePath -Force -Verbose:$false 17 | } 18 | 19 | AfterAll { 20 | $env:PSModulePath = $savedModulePath 21 | Remove-Module OperationValidation -Verbose:$false 22 | } 23 | 24 | Context "Finds proper tests" { 25 | It "Can find its own tests" { 26 | $tests = Get-OperationValidation -Path (Split-Path -Path $env:BHModulePath -Parent) 27 | 28 | $tests.Count | Should be 2 29 | $tests.File -eq "PSGallery.Simple.Tests.ps1" | Should be "PSGallery.Simple.Tests.ps1" 30 | $tests.File -eq "PSGallery.Comprehensive.Tests.ps1" | Should be "PSGallery.Comprehensive.Tests.ps1" 31 | } 32 | It "Can find tests which don't have an actual module" { 33 | $tests = Get-OperationValidation -moduleName Example.WindowsSearch 34 | @($tests).Count | Should be 1 35 | $tests.File | should be WindowsSearch.Simple.Tests.ps1 36 | } 37 | It "Can find a specific version of a module" { 38 | $v1Tests = @(Get-OperationValidation -ModuleName 'VersionedModule' -Version '1.0.0') 39 | $v1Tests.Count | Should be 1 40 | $v1Tests.File | Should be 'PSGallery.Simple.Tests.ps1' 41 | 42 | $v2Tests = @(Get-OperationValidation -ModuleName 'VersionedModule' -Version '2.0.0') 43 | $v2Tests.Count | Should be 3 44 | $v2Tests[0].File | Should be 'PSGallery.Simple.Tests.ps1' 45 | $v2Tests[1].File | Should be 'PSGallery.Simple.Tests.ps1' 46 | } 47 | It "Can get the latest version of a module if no version is specified" { 48 | $tests = Get-OperationValidation -ModuleName VersionedModule 49 | $tests[0].Version | Should be ([Version]'2.0.0') 50 | $tests[1].Version | Should be ([Version]'2.0.0') 51 | $tests[2].Version | Should be ([Version]'2.0.0') 52 | } 53 | It "Can get tests with a tag" { 54 | $tests = Get-OperationValidation -Tag 'AAABBBCCC' 55 | $tests.Count | should be 2 56 | $tests[0].Tags[0] | Should be 'AAABBBCCC' 57 | $tests[1].Tags[0] | Should be 'AAABBBCCC' 58 | $tests[0].Name | Should be 'Simple Validation of PSGallery' 59 | $tests[1].Name | Should be 'Simple Validation of Microsoft' 60 | } 61 | It "Can get tests with multiple tags" { 62 | $tests = Get-OperationValidation -Tag 'AAABBBCCC', 'XXXYYYZZZ' 63 | $tests.Count | Should be 2 64 | @($tests | Where-Object {'AAABBBCCC' -in $_.Tags}).Count | Should be 2 65 | @($tests | Where-Object {'XXXYYYZZZ' -in $_.Tags}).Count | Should be 1 66 | } 67 | It "Can exclude modules with a tag" { 68 | $tests = Get-OperationValidation -ExcludeTag 'AAABBBCCC' 69 | $myTest = $tests | Where-Object {$_.Tags -Contains 'AAABBBCCC'} 70 | $myTest | Should BeNullOrEmpty 71 | } 72 | It "Can exclude modules with multiple tags" { 73 | $tests = Get-OperationValidation -ExcludeTag 'AAABBBCCC', 'XXXYYYZZZ' 74 | $myTest = $tests | Where-Object {('AAABBBCCC' -in $_.Tags) -or ('XXXYYYZZZ' -in $_.Tags)} 75 | $myTest | Should BeNullOrEmpty 76 | } 77 | It "Formats the output appropriately" { 78 | $output = Get-OperationValidation -Modulename OperationValidation | Out-String -Stream -Width 210 | Where-Object {$_} 79 | $expected = ".*Module: .*OperationValidation", 80 | "Version: *" 81 | "Type: Simple", 82 | "Tags: {}", 83 | "File: PSGallery.Simple.Tests.ps1", 84 | "FilePath: .*PSGallery.Simple.Tests.ps1", 85 | "Name:", 86 | "Simple Validation of PSGallery", 87 | "" 88 | "Module: .*OperationValidation", 89 | "Type: Comprehensive", 90 | "Tags: {}", 91 | "File: PSGallery.Comprehensive.Tests.ps1", 92 | "FilePath: .*PSGallery.Comprehensive.Tests.ps1", 93 | "Name:", 94 | " E2E validation of PSGallery" 95 | for($i = 0; $i -lt $expected.Count; $i++) 96 | { 97 | $output[$i] | Should match $expected[$i] 98 | } 99 | } 100 | It 'Can get tests by Path' { 101 | $tests = $testModuleDir | Get-OperationValidation 102 | $tests.Count | should be 4 103 | } 104 | 105 | It 'Can get tests by LiteralPath' { 106 | $tests = Get-OperationValidation -LiteralPath $testModuleDir 107 | $tests.Count | should be 4 108 | } 109 | } 110 | } -------------------------------------------------------------------------------- /Tests/Unit/Invoke-OperationValidation.tests.ps1: -------------------------------------------------------------------------------- 1 | 2 | $testModuleDir = (Resolve-Path -Path (Join-Path -Path $env:BHProjectPath -ChildPath TestArtifacts)).Path 3 | 4 | Describe 'Invoke-OperationValidation' { 5 | 6 | BeforeAll { 7 | $pathSeparator = [IO.Path]::PathSeparator 8 | $savedModulePath = $env:PSModulePath 9 | if ($testModuleDir -notin $env:PSModulePath.split($pathSeparator)) { 10 | $env:PSModulePath += ($pathSeparator + $testModuleDir) 11 | } 12 | if ($env:BHProjectPath -notin $env:PSModulePath.Split($pathSeparator)) { 13 | $env:PSModulePath += ($pathSeparator + $env:BHProjectPath) 14 | } 15 | Remove-Module Microsoft.PowerShell.Operation.Validation -Force -ErrorAction SilentlyContinue -Verbose:$false 16 | Import-Module $env:BHModulePath -Force -Verbose:$false 17 | } 18 | 19 | AfterAll { 20 | $env:PSModulePath = $savedModulePath 21 | Remove-Module OperationValidation -Verbose:$false 22 | } 23 | 24 | Context "Passes override parameters" { 25 | $tests = Get-OperationValidation -ModuleName VersionedModule -Version '1.0.0' 26 | 27 | It "No override parameters supplied" { 28 | $results = $tests | Invoke-OperationValidation 29 | $results[0].Result | Should be 'Passed' 30 | $results[1].Result | Should be 'Passed' 31 | } 32 | 33 | It "Override parameters supplied" { 34 | $results = $tests | Invoke-OperationValidation -Overrides @{ WebsiteUrl = 'https://www.microsoft.com'} 35 | $results[0].Result | Should be 'Passed' 36 | $results[1].Result | Should be 'Failed' 37 | } 38 | } 39 | 40 | Context "Runs tests based on tags" { 41 | It "Can run tests with certain tag" { 42 | $results = Invoke-OperationValidation -Tag 'AAABBBCCC' 43 | $results[0].Result | Should be 'Passed' 44 | $results[1].Result | Should be 'Passed' 45 | } 46 | 47 | It "Can run tests excluding a tag" { 48 | $results = Invoke-OperationValidation -Modulename VersionedModule -ExcludeTag 'AAABBBCCC' 49 | $results.Result | Should be 'Passed' 50 | 51 | $results = Invoke-OperationValidation -Modulename VersionedModule -ExcludeTag 'XXXYYYZZZ' 52 | $results.Count | Should be 2 53 | } 54 | } 55 | 56 | Context 'Accepts Path and LiteralPath' { 57 | It 'Can run tests by Path' { 58 | $results = $testModuleDir | Invoke-OperationValidation 59 | $results.Count | should be 4 60 | } 61 | 62 | It 'Can run tests by LiteralPath' { 63 | $results = Invoke-OperationValidation -LiteralPath $testModuleDir 64 | $results.Count | should be 4 65 | } 66 | } 67 | 68 | Context 'Single Files' { 69 | it 'Can run a single Pester script' { 70 | $results = Invoke-OperationValidation -TestFilePath (Join-Path -Path $testModuleDir -ChildPath 'SingleTest.tests.ps1') 71 | $results.Result | Should Be 'Passed' 72 | } 73 | } 74 | } -------------------------------------------------------------------------------- /Tests/Unit/Module.tests.ps1: -------------------------------------------------------------------------------- 1 | 2 | Describe 'Module' { 3 | 4 | BeforeAll { 5 | $pathSeparator = [IO.Path]::PathSeparator 6 | $savedModulePath = $env:PSModulePath 7 | if ($env:PSModulePath.split($pathSeparator) -notcontains $testModuleDir) { 8 | $env:PSModulePath += ($pathSeparator + $testModuleDir) 9 | } 10 | if ($env:PSModulePath.Split($pathSeparator) -notcontains $env:BHModulePath) { 11 | $env:PSModulePath += ($pathSeparator + $env:BHProjectPath) 12 | } 13 | Remove-Module Microsoft.PowerShell.Operation.Validation -Force -ErrorAction SilentlyContinue 14 | Import-Module $env:BHModulePath -Force 15 | } 16 | 17 | AfterAll { 18 | $env:PSModulePath = $savedModulePath 19 | Remove-Module OperationValidation 20 | } 21 | 22 | Context "Exported Commands" { 23 | 24 | $commands = Get-Command -Module OperationValidation | Sort-object Name 25 | 26 | It "Exports 2 commands" { 27 | $commands.Count | Should be 2 28 | } 29 | It "The command names are correct" { 30 | $commands[0].Name | Should be "Get-OperationValidation" 31 | $commands[1].Name | Should be "Invoke-OperationValidation" 32 | } 33 | } 34 | 35 | } -------------------------------------------------------------------------------- /appveyor.yml: -------------------------------------------------------------------------------- 1 | version: 1.1.0.{build} 2 | image: 3 | - Visual Studio 2015 4 | - Visual Studio 2017 5 | - Ubuntu 6 | - Ubuntu1804 7 | skip_commits: 8 | message: /updated readme.*|update readme.*s/ 9 | 10 | build: off 11 | 12 | for: 13 | - 14 | matrix: 15 | only: 16 | - image: Ubuntu 17 | 18 | test_script: 19 | - ps: ./build.ps1 -Task Test -Bootstrap 20 | 21 | - 22 | matrix: 23 | only: 24 | - image: Ubuntu1804 25 | 26 | test_script: 27 | - ps: ./build.ps1 -Task Test -Bootstrap 28 | 29 | #Kick off the CI/CD pipeline 30 | test_script: 31 | - pwsh: ./build.ps1 -Task Test -Bootstrap 32 | -------------------------------------------------------------------------------- /azure-pipelines.yml: -------------------------------------------------------------------------------- 1 | jobs: 2 | - job: Build_PS_Win2016 3 | pool: 4 | vmImage: vs2017-win2016 5 | steps: 6 | - powershell: | 7 | ./build.ps1 -Task Test -Bootstrap -Verbose 8 | displayName: 'Build and Test' 9 | - task: PublishTestResults@2 10 | inputs: 11 | testRunner: 'NUnit' 12 | testResultsFiles: '**/out/testResults*.xml' 13 | testRunTitle: 'PS_Win2016' 14 | displayName: 'Publish Test Results' 15 | 16 | - job: Build_PSCore_Ubuntu1604 17 | pool: 18 | vmImage: ubuntu-16.04 19 | steps: 20 | - script: | 21 | pwsh -c './build.ps1 -Task Test -Bootstrap -Verbose' 22 | displayName: 'Build and Test' 23 | - task: PublishTestResults@2 24 | inputs: 25 | testRunner: 'NUnit' 26 | testResultsFiles: '**/out/testResults*.xml' 27 | testRunTitle: 'PSCore_Ubuntu1604' 28 | displayName: 'Publish Test Results' 29 | 30 | - job: Build_PSCore_MacOS1013 31 | pool: 32 | vmImage: xcode9-macos10.13 33 | steps: 34 | - script: | 35 | pwsh -c './build.ps1 -Task Test -Bootstrap -Verbose' 36 | displayName: 'Build and Test' 37 | - task: PublishTestResults@2 38 | inputs: 39 | testRunner: 'NUnit' 40 | testResultsFiles: '**/out/testResults*.xml' 41 | testRunTitle: 'PSCore_MacOS1013' 42 | displayName: 'Publish Test Results' -------------------------------------------------------------------------------- /build.ps1: -------------------------------------------------------------------------------- 1 | [cmdletbinding(DefaultParameterSetName = 'Task')] 2 | param( 3 | # Build task(s) to execute 4 | [parameter(ParameterSetName = 'task', position = 0)] 5 | [string[]]$Task = 'default', 6 | 7 | # Optional properties to pass to psake 8 | [hashtable]$Properties, 9 | 10 | # Bootstrap dependencies 11 | [switch]$Bootstrap, 12 | 13 | # List available build tasks 14 | [parameter(ParameterSetName = 'Help')] 15 | [switch]$Help 16 | ) 17 | 18 | $ErrorActionPreference = 'Stop' 19 | 20 | # Bootstrap dependencies 21 | if ($Bootstrap.IsPresent) { 22 | Get-PackageProvider -Name Nuget -ForceBootstrap | Out-Null 23 | Set-PSRepository -Name PSGallery -InstallationPolicy Trusted 24 | if (-not (Get-Module -Name PSDepend -ListAvailable)) { 25 | Install-module -Name PSDepend -Repository PSGallery -Scope CurrentUser -Force 26 | } 27 | Import-Module -Name PSDepend -Verbose:$false 28 | Invoke-PSDepend -Path './requirements.psd1' -Install -Import -Force -WarningAction SilentlyContinue 29 | } 30 | 31 | # Execute psake task(s) 32 | $psakeFile = './psakeFile.ps1' 33 | if ($PSCmdlet.ParameterSetName -eq 'Help') { 34 | Get-PSakeScriptTasks -buildFile $psakeFile | 35 | Format-Table -Property Name, Description, Alias, DependsOn 36 | } else { 37 | Set-BuildEnvironment -Force 38 | Invoke-psake -buildFile $psakeFile -taskList $Task -nologo -properties $Properties 39 | exit ( [int]( -not $psake.build_success ) ) 40 | } -------------------------------------------------------------------------------- /docs/en-US/Get-OperationValidation.md: -------------------------------------------------------------------------------- 1 | --- 2 | external help file: OperationValidation-help.xml 3 | Module Name: OperationValidation 4 | online version: 5 | schema: 2.0.0 6 | --- 7 | 8 | # Get-OperationValidation 9 | 10 | ## SYNOPSIS 11 | Retrieve the operational tests from modules 12 | 13 | ## SYNTAX 14 | 15 | ### ModuleName (Default) 16 | ``` 17 | Get-OperationValidation [[-Name] ] [-TestType ] [-Version ] [-Tag ] 18 | [-ExcludeTag ] [] 19 | ``` 20 | 21 | ### Path 22 | ``` 23 | Get-OperationValidation [-Path] [-TestType ] [-Version ] [-Tag ] 24 | [-ExcludeTag ] [] 25 | ``` 26 | 27 | ### LiteralPath 28 | ``` 29 | Get-OperationValidation [-LiteralPath] [-TestType ] [-Version ] [-Tag ] 30 | [-ExcludeTag ] [] 31 | ``` 32 | 33 | ## DESCRIPTION 34 | Modules which include a Diagnostics directory are inspected for 35 | Pester tests in either the "Simple" or "Comprehensive" subdirectories. 36 | If files are found in those directories, they will be inspected to determine 37 | whether they are Pester tests. 38 | If Pester tests are found, the 39 | test names in those files will be returned. 40 | 41 | The module structure required is as follows: 42 | 43 | ModuleBase\ 44 | Diagnostics\ 45 | Simple # simple tests are held in this location 46 | (e.g., ping, serviceendpoint checks) 47 | Comprehensive # comprehensive scenario tests should be placed here 48 | 49 | ## EXAMPLES 50 | 51 | ### EXAMPLE 1 52 | ``` 53 | Get-OperationValidation -Name OVF.Windows.Server 54 | ``` 55 | 56 | Module: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2 57 | Version: 1.0.2 58 | Type: Simple 59 | Tags: {} 60 | File: LogicalDisk.tests.ps1 61 | FilePath: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2\Diagnostics\Simple\LogicalDisk.tests.ps1 62 | Name: 63 | Logical Disks 64 | 65 | 66 | Module: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2 67 | Version: 1.0.2 68 | Type: Simple 69 | Tags: {} 70 | File: Memory.tests.ps1 71 | FilePath: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2\Diagnostics\Simple\Memory.tests.ps1 72 | Name: 73 | Memory 74 | 75 | 76 | Module: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2 77 | Version: 1.0.2 78 | Type: Simple 79 | Tags: {} 80 | File: Network.tests.ps1 81 | FilePath: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2\Diagnostics\Simple\Network.tests.ps1 82 | Name: 83 | Network Adapters 84 | 85 | 86 | Module: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2 87 | Version: 1.0.2 88 | Type: Simple 89 | Tags: {} 90 | File: Services.tests.ps1 91 | FilePath: C:\Program Files\WindowsPowerShell\Modules\OVF.Windows.Server\1.0.2\Diagnostics\Simple\Services.tests.ps1 92 | Name: 93 | Operating System 94 | 95 | ### EXAMPLE 2 96 | ``` 97 | $tests = Get-OperationValidation 98 | ``` 99 | 100 | Search in all modules found in $env:PSModulePath for OVF tests. 101 | 102 | ### EXAMPLE 3 103 | ``` 104 | $tests = Get-OperationValidation -Path C:\MyTests 105 | ``` 106 | 107 | Search for OVF modules under c:\MyTests 108 | 109 | ### EXAMPLE 4 110 | ``` 111 | $simpleTests = Get-OperationValidation -ModuleName OVF.Windows.Server -TypeType Simple 112 | ``` 113 | 114 | Get just the simple tests in the OVF.Windows.Server module. 115 | 116 | ### EXAMPLE 5 117 | ``` 118 | $tests = Get-OperationValidation -ModuleName OVF.Windows.Server -Version 1.0.2 119 | ``` 120 | 121 | Get all the tests from version 1.0.2 of the OVF.Windows.Server module. 122 | 123 | ### EXAMPLE 6 124 | ``` 125 | $storageTests = Get-OperationValidation -Tag Storage 126 | ``` 127 | 128 | Search in all modules for OVF tests that include the tag Storage. 129 | 130 | ### EXAMPLE 7 131 | ``` 132 | $tests = Get-OperationValidation -ExcludeTag memory 133 | ``` 134 | 135 | Search for OVF tests that don't include the tag Memory 136 | 137 | ## PARAMETERS 138 | 139 | ### -Name 140 | One or more module names to inspect and return if they adhere to the OVF Pester test structure. 141 | 142 | By default this is \[*\] which will inspect all modules in $env:PSModulePath. 143 | 144 | ```yaml 145 | Type: String[] 146 | Parameter Sets: ModuleName 147 | Aliases: ModuleName 148 | 149 | Required: False 150 | Position: 1 151 | Default value: * 152 | Accept pipeline input: False 153 | Accept wildcard characters: False 154 | ``` 155 | 156 | ### -Path 157 | One or more paths to search for OVF modules in. 158 | This bypasses searching the directories contained in $env:PSModulePath. 159 | 160 | ```yaml 161 | Type: String[] 162 | Parameter Sets: Path 163 | Aliases: 164 | 165 | Required: True 166 | Position: 1 167 | Default value: None 168 | Accept pipeline input: True (ByPropertyName, ByValue) 169 | Accept wildcard characters: True 170 | ``` 171 | 172 | ### -LiteralPath 173 | One or more literal paths to search for OVF modules in. 174 | This bypasses searching the directories contained in $env:PSModulePath. 175 | 176 | Unlike the Path parameter, the value of LiteralPath is used exactly as it is typed. 177 | No characters are interpreted as wildcards. 178 | If the path includes escape characters, enclose it in single quotation marks. 179 | Single quotation 180 | marks tell PowerShell not to interpret any characters as escape sequences. 181 | 182 | ```yaml 183 | Type: String[] 184 | Parameter Sets: LiteralPath 185 | Aliases: PSPath 186 | 187 | Required: True 188 | Position: 1 189 | Default value: None 190 | Accept pipeline input: True (ByPropertyName) 191 | Accept wildcard characters: False 192 | ``` 193 | 194 | ### -TestType 195 | The type of tests to retrieve, this may be either "Simple", "Comprehensive", or Both ("Simple,Comprehensive"). 196 | "Simple, Comprehensive" is the default. 197 | 198 | ```yaml 199 | Type: String[] 200 | Parameter Sets: (All) 201 | Aliases: 202 | 203 | Required: False 204 | Position: Named 205 | Default value: @('Simple', 'Comprehensive') 206 | Accept pipeline input: False 207 | Accept wildcard characters: False 208 | ``` 209 | 210 | ### -Version 211 | The version of the module to retrieve. 212 | If not specified, the latest version 213 | of the module will be retured. 214 | 215 | ```yaml 216 | Type: Version 217 | Parameter Sets: (All) 218 | Aliases: 219 | 220 | Required: False 221 | Position: Named 222 | Default value: None 223 | Accept pipeline input: False 224 | Accept wildcard characters: False 225 | ``` 226 | 227 | ### -Tag 228 | Executes tests with specified tag parameter values. 229 | Wildcard characters and tag values that include spaces 230 | or whitespace characters are not supported. 231 | 232 | When you specify multiple tag values, Get-OperationValidation executes tests that have any of the 233 | listed tags. 234 | If you use both Tag and ExcludeTag, ExcludeTag takes precedence. 235 | 236 | ```yaml 237 | Type: String[] 238 | Parameter Sets: (All) 239 | Aliases: 240 | 241 | Required: False 242 | Position: Named 243 | Default value: None 244 | Accept pipeline input: False 245 | Accept wildcard characters: False 246 | ``` 247 | 248 | ### -ExcludeTag 249 | Omits tests with the specified tag parameter values. 250 | Wildcard characters and tag values that include spaces 251 | or whitespace characters are not supported. 252 | 253 | When you specify multiple ExcludeTag values, Get-OperationValidation omits tests that have any 254 | of the listed tags. 255 | If you use both Tag and ExcludeTag, ExcludeTag takes precedence. 256 | 257 | ```yaml 258 | Type: String[] 259 | Parameter Sets: (All) 260 | Aliases: 261 | 262 | Required: False 263 | Position: Named 264 | Default value: None 265 | Accept pipeline input: False 266 | Accept wildcard characters: False 267 | ``` 268 | 269 | ### CommonParameters 270 | This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable, -InformationAction, -InformationVariable, -OutVariable, -OutBuffer, -PipelineVariable, -Verbose, -WarningAction, and -WarningVariable. For more information, see [about_CommonParameters](http://go.microsoft.com/fwlink/?LinkID=113216). 271 | 272 | ## INPUTS 273 | 274 | ## OUTPUTS 275 | 276 | ## NOTES 277 | 278 | ## RELATED LINKS 279 | 280 | [Invoke-OperationValidation]() 281 | 282 | -------------------------------------------------------------------------------- /docs/en-US/Invoke-OperationValidation.md: -------------------------------------------------------------------------------- 1 | --- 2 | external help file: OperationValidation-help.xml 3 | Module Name: OperationValidation 4 | online version: 5 | schema: 2.0.0 6 | --- 7 | 8 | # Invoke-OperationValidation 9 | 10 | ## SYNOPSIS 11 | Invoke the operational tests from modules 12 | 13 | ## SYNTAX 14 | 15 | ### FileAndTest (Default) 16 | ``` 17 | Invoke-OperationValidation [-TestInfo ] [-IncludePesterOutput] [-Overrides ] [-WhatIf] 18 | [-Confirm] [] 19 | ``` 20 | 21 | ### TestFile 22 | ``` 23 | Invoke-OperationValidation [-TestFilePath ] [-IncludePesterOutput] [-WhatIf] [-Confirm] 24 | [] 25 | ``` 26 | 27 | ### UseGetOperationTest 28 | ``` 29 | Invoke-OperationValidation [-ModuleName ] [-TestType ] [-IncludePesterOutput] 30 | [-Version ] [-Overrides ] [-Tag ] [-ExcludeTag ] [-WhatIf] [-Confirm] 31 | [] 32 | ``` 33 | 34 | ### Path 35 | ``` 36 | Invoke-OperationValidation [-Path] [-TestType ] [-IncludePesterOutput] 37 | [-Version ] [-Overrides ] [-Tag ] [-ExcludeTag ] [-WhatIf] [-Confirm] 38 | [] 39 | ``` 40 | 41 | ### LiteralPath 42 | ``` 43 | Invoke-OperationValidation [-LiteralPath] [-TestType ] [-IncludePesterOutput] 44 | [-Version ] [-Overrides ] [-Tag ] [-ExcludeTag ] [-WhatIf] [-Confirm] 45 | [] 46 | ``` 47 | 48 | ## DESCRIPTION 49 | Modules which include Diagnostics tests are executed via this cmdlet 50 | 51 | ## EXAMPLES 52 | 53 | ### EXAMPLE 1 54 | ``` 55 | Get-OperationValidation -ModuleName OperationValidation | Invoke-OperationValidation -IncludePesterOutput 56 | ``` 57 | 58 | Describing Simple Test Suite 59 | \[+\] first Operational test 20ms 60 | \[+\] second Operational test 19ms 61 | \[+\] third Operational test 9ms 62 | Tests completed in 48ms 63 | Passed: 3 Failed: 0 Skipped: 0 Pending: 0 64 | Describing Scenario targeted tests 65 | Context The RemoteAccess service 66 | \[+\] The service is running 37ms 67 | Context The Firewall Rules 68 | \[+\] A rule for TCP port 3389 is enabled 1.19s 69 | \[+\] A rule for UDP port 3389 is enabled 11ms 70 | Tests completed in 1.24s 71 | Passed: 3 Failed: 0 Skipped: 0 Pending: 0 72 | 73 | 74 | Module: OperationValidation 75 | 76 | Result Name 77 | ------- -------- 78 | Passed Simple Test Suite::first Operational test 79 | Passed Simple Test Suite::second Operational test 80 | Passed Simple Test Suite::third Operational test 81 | Passed Scenario targeted tests:The RemoteAccess service:The service is running 82 | Passed Scenario targeted tests:The Firewall Rules:A rule for TCP port 3389 is enabled 83 | Passed Scenario targeted tests:The Firewall Rules:A rule for UDP port 3389 is enabled 84 | 85 | ## PARAMETERS 86 | 87 | ### -TestFilePath 88 | The path to a diagnostic test to execute. 89 | By default all discoverable diagnostics will be invoked 90 | 91 | ```yaml 92 | Type: String[] 93 | Parameter Sets: TestFile 94 | Aliases: 95 | 96 | Required: False 97 | Position: Named 98 | Default value: None 99 | Accept pipeline input: True (ByPropertyName) 100 | Accept wildcard characters: False 101 | ``` 102 | 103 | ### -TestInfo 104 | The type of tests to invoke, this may be either "Simple", "Comprehensive" 105 | or Both ("Simple,Comprehensive"). 106 | "Simple,Comprehensive" is the default. 107 | 108 | ```yaml 109 | Type: PSObject[] 110 | Parameter Sets: FileAndTest 111 | Aliases: 112 | 113 | Required: False 114 | Position: Named 115 | Default value: None 116 | Accept pipeline input: True (ByValue) 117 | Accept wildcard characters: False 118 | ``` 119 | 120 | ### -ModuleName 121 | By default this is * which will retrieve and execute all OVF modules in $env:psmodulepath 122 | Additional module directories may be added. 123 | If you wish to check both 124 | $env:psmodulepath and your own specific locations, use 125 | *,\ 126 | 127 | ```yaml 128 | Type: String[] 129 | Parameter Sets: UseGetOperationTest 130 | Aliases: 131 | 132 | Required: False 133 | Position: Named 134 | Default value: None 135 | Accept pipeline input: False 136 | Accept wildcard characters: False 137 | ``` 138 | 139 | ### -Path 140 | One or more paths to search for OVF modules in. 141 | This bypasses searching the directories contained in $env:PSModulePath. 142 | 143 | ```yaml 144 | Type: String[] 145 | Parameter Sets: Path 146 | Aliases: 147 | 148 | Required: True 149 | Position: 1 150 | Default value: None 151 | Accept pipeline input: True (ByPropertyName, ByValue) 152 | Accept wildcard characters: True 153 | ``` 154 | 155 | ### -LiteralPath 156 | One or more literal paths to search for OVF modules in. 157 | This bypasses searching the directories contained in $env:PSModulePath. 158 | 159 | Unlike the Path parameter, the value of LiteralPath is used exactly as it is typed. 160 | No characters are interpreted as wildcards. 161 | If the path includes escape characters, enclose it in single quotation marks. 162 | Single quotation 163 | marks tell PowerShell not to interpret any characters as escape sequences. 164 | 165 | ```yaml 166 | Type: String[] 167 | Parameter Sets: LiteralPath 168 | Aliases: PSPath 169 | 170 | Required: True 171 | Position: 1 172 | Default value: None 173 | Accept pipeline input: True (ByPropertyName) 174 | Accept wildcard characters: False 175 | ``` 176 | 177 | ### -TestType 178 | The type of tests to execute, this may be either "Simple", "Comprehensive" 179 | or Both ("Simple,Comprehensive"). 180 | "Simple,Comprehensive" is the default. 181 | 182 | ```yaml 183 | Type: String[] 184 | Parameter Sets: UseGetOperationTest, Path, LiteralPath 185 | Aliases: 186 | 187 | Required: False 188 | Position: Named 189 | Default value: @('Simple', 'Comprehensive') 190 | Accept pipeline input: False 191 | Accept wildcard characters: False 192 | ``` 193 | 194 | ### -IncludePesterOutput 195 | Include the Pester output when execute the tests. 196 | 197 | ```yaml 198 | Type: SwitchParameter 199 | Parameter Sets: (All) 200 | Aliases: 201 | 202 | Required: False 203 | Position: Named 204 | Default value: False 205 | Accept pipeline input: False 206 | Accept wildcard characters: False 207 | ``` 208 | 209 | ### -Version 210 | The version of the module to retrieve. 211 | If the specified, the latest version 212 | of the module will be retured. 213 | 214 | ```yaml 215 | Type: Version 216 | Parameter Sets: UseGetOperationTest, Path, LiteralPath 217 | Aliases: 218 | 219 | Required: False 220 | Position: Named 221 | Default value: None 222 | Accept pipeline input: False 223 | Accept wildcard characters: False 224 | ``` 225 | 226 | ### -Overrides 227 | If the Pester test(s) include script parameters, those parameters can be overridden by 228 | specifying a hashtable of values. 229 | The key(s) in the hashtable must match the parameter 230 | names in the Pester test. 231 | 232 | For example, if the Pester test includes a parameter block like the following, one or more of 233 | these parameters can be overriden using values from the hashtable passed to the -Overrides parameter. 234 | 235 | Pester test script: 236 | param( 237 | \[int\]$SomeValue = 100 238 | \[bool\]$ExtraChecks = $false 239 | ) 240 | 241 | Overrides the default parameter values: 242 | Invoke-OperationValidation -ModuleName MyModule -Overrides @{ SomeValue = 500; ExtraChecks = $true } 243 | 244 | ```yaml 245 | Type: Hashtable 246 | Parameter Sets: FileAndTest, UseGetOperationTest, Path, LiteralPath 247 | Aliases: 248 | 249 | Required: False 250 | Position: Named 251 | Default value: None 252 | Accept pipeline input: False 253 | Accept wildcard characters: False 254 | ``` 255 | 256 | ### -Tag 257 | Executes tests with specified tag parameter values. 258 | Wildcard characters and tag values that include spaces 259 | or whitespace characters are not supported. 260 | 261 | When you specify multiple tag values, Invoke-OperationValidation executes tests that have any of the 262 | listed tags. 263 | If you use both Tag and ExcludeTag, ExcludeTag takes precedence. 264 | 265 | ```yaml 266 | Type: String[] 267 | Parameter Sets: UseGetOperationTest, Path, LiteralPath 268 | Aliases: 269 | 270 | Required: False 271 | Position: Named 272 | Default value: None 273 | Accept pipeline input: False 274 | Accept wildcard characters: False 275 | ``` 276 | 277 | ### -ExcludeTag 278 | Omits tests with the specified tag parameter values. 279 | Wildcard characters and tag values that include spaces 280 | or whitespace characters are not supported. 281 | 282 | When you specify multiple ExcludeTag values, Get-OperationValidation omits tests that have any 283 | of the listed tags. 284 | If you use both Tag and ExcludeTag, ExcludeTag takes precedence. 285 | 286 | ```yaml 287 | Type: String[] 288 | Parameter Sets: UseGetOperationTest, Path, LiteralPath 289 | Aliases: 290 | 291 | Required: False 292 | Position: Named 293 | Default value: None 294 | Accept pipeline input: False 295 | Accept wildcard characters: False 296 | ``` 297 | 298 | ### -WhatIf 299 | Shows what would happen if the cmdlet runs. 300 | The cmdlet is not run. 301 | 302 | ```yaml 303 | Type: SwitchParameter 304 | Parameter Sets: (All) 305 | Aliases: wi 306 | 307 | Required: False 308 | Position: Named 309 | Default value: None 310 | Accept pipeline input: False 311 | Accept wildcard characters: False 312 | ``` 313 | 314 | ### -Confirm 315 | Prompts you for confirmation before running the cmdlet. 316 | 317 | ```yaml 318 | Type: SwitchParameter 319 | Parameter Sets: (All) 320 | Aliases: cf 321 | 322 | Required: False 323 | Position: Named 324 | Default value: None 325 | Accept pipeline input: False 326 | Accept wildcard characters: False 327 | ``` 328 | 329 | ### CommonParameters 330 | This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable, -InformationAction, -InformationVariable, -OutVariable, -OutBuffer, -PipelineVariable, -Verbose, -WarningAction, and -WarningVariable. For more information, see [about_CommonParameters](http://go.microsoft.com/fwlink/?LinkID=113216). 331 | 332 | ## INPUTS 333 | 334 | ## OUTPUTS 335 | 336 | ## NOTES 337 | 338 | ## RELATED LINKS 339 | 340 | [Get-OperationValidation]() 341 | 342 | -------------------------------------------------------------------------------- /docs/en-US/OperationValidation.md: -------------------------------------------------------------------------------- 1 | --- 2 | Module Name: OperationValidation 3 | Module Guid: 25bd9e34-bff9-4552-a23d-854857b42462 4 | Download Help Link: {{ Update Download Link }} 5 | Help Version: {{ Please enter version of help manually (X.X.X.X) format }} 6 | Locale: en-US 7 | --- 8 | 9 | # OperationValidation Module 10 | ## Description 11 | {{ Fill in the Description }} 12 | 13 | ## OperationValidation Cmdlets 14 | ### [Get-OperationValidation](Get-OperationValidation.md) 15 | {{ Fill in the Description }} 16 | 17 | ### [Invoke-OperationValidation](Invoke-OperationValidation.md) 18 | {{ Fill in the Description }} 19 | 20 | -------------------------------------------------------------------------------- /psakeFile.ps1: -------------------------------------------------------------------------------- 1 | properties { 2 | 3 | } 4 | 5 | task Default -depends Test 6 | 7 | task Test -FromModule PowerShellBuild -Version 0.3.0 -------------------------------------------------------------------------------- /requirements.psd1: -------------------------------------------------------------------------------- 1 | @{ 2 | PSDependOptions = @{ 3 | Target = 'CurrentUser' 4 | } 5 | BuildHelpers = 'latest' 6 | Pester = @{ 7 | Version = 'latest' 8 | Parameters = @{ 9 | SkipPublisherCheck = $true 10 | } 11 | } 12 | PowerShellBuild = '0.3.0' 13 | psake = 'latest' 14 | PSScriptAnalyzer = 'latest' 15 | } --------------------------------------------------------------------------------