├── README.md ├── resources ├── functions.ps1 └── scripts │ ├── New-CodeQLScan.ps1 │ └── Set-CommitStatusChecks.ps1 └── vars └── codeqlScan.groovy /README.md: -------------------------------------------------------------------------------- 1 | # Overview 2 | During a presentation at the PowerShell Summit, it was mentioned that [with the onset of PowerShell Core](https://devblogs.microsoft.com/powershell/powershell-core-now-available-as-a-snap-package/) PowerShell aims to be _ubiquitous_. This really stuck with me, and is a primary reason why I think PowerShell is 👑. Regardless, there is no reason that CodeQL should also not be _ubiquitous_. Let's see how deep we can get in the pocket by wrapping the needed integration steps with PowerShell functions 💪. 3 | 4 | While there has historically always been the ability to leverage CodeQL within 3rd party CI/CD systems, the CodeQL runner [has been deprecated](https://github.blog/changelog/2021-09-21-codeql-runner-deprecation/). This repository aims to create, to the greatest extent possible, an environment agnostic (e.g., Azure DevOps, Jenkins, or even your local machine) means to to use CodeQL. 5 | 6 | --- 7 | 8 | - [Requirements](#requirements) 9 | - [Approach](#approach) 10 | - [Assumptions](#assumptions) 11 | - [Examples](#examples) 12 | - [Local Machine](#local-machine) 13 | - [Azure DevOps](#azure-devops) 14 | - [Jenkins](#jenkins) 15 | - [Interpreted Languages](#interpreted-languages) 16 | - [Compiled Languages](#compiled-languages) 17 | - [C / C++](#c--c) 18 | - [C#](#c) 19 | - [Java](#java) 20 | - [Go](#go) 21 | - [Build Script](#build-script) 22 | - [Dot Sourcing](#dot-sourcing) 23 | 24 | # Requirements 25 | - PowerShell Core (i.e., [PowerShell 7](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell?view=powershell-7.2)) 26 | 27 | That's pretty much it... 28 | 29 | # Approach 30 | 1. Download the latest CodeQL bundle for the detected OS 31 | 2. Detect the languages of a given repository to determine which ones are supported by CodeQL 32 | 3. Create CodeQL databases, analyze, and upload results for each supported language 33 | 34 | # Assumptions 35 | - GitHub is used as a source control platform 36 | - The repository is either _public_, or GitHub Advanced Security is enabled if _private_ 37 | - The working directory is a `git` repository 38 | - The Personal Access Token that is used has the ability to write `security_events` for a given repository 39 | - We don't use the `codeql` CLI to upload SARIF results because it currently (❓) doesn't support the ability to record the length of time a given scan takes. This leaves an undesirable _Unknown_ in the Duration field of the banner that is above the alerts table in the Code scanning section of a given repository. 40 | 41 | # Examples 42 | This section aims to demonstrate the portability of `codeql-anywhere`. 43 | 44 | ## Local Machine 45 | The following examples assume that a GitHub Personal Access Token (PAT) is saved to the `$GITHUB_TOKEN` environment variable. 46 | 47 | Ensure that the functions in the `functions.ps1` file of this repository are loaded into memory. Do this by either copy / pasting them into a PowerShell session, or [dot source](#dot-sourcing) them. The following example specifies to retain the SARIF files in the source root for further exploration. 48 | 49 | ```powershell 50 | New-CodeQLScan -token $env:GITHUB_TOKEN -keepSarif 51 | ``` 52 | 53 | Alternatively you could create the CodeQL databases in a temporary directory. 54 | ```powershell 55 | $activeTempRoot = (Get-PSDrive | Where-Object {$_.name -like 'Temp'}).Root 56 | $codeQLDatabaseDirectory = Join-Path -Path $activeTempRoot 'codeql/databases' 57 | New-CodeQLScan -token $env:GITHUB_TOKEN -codeQLDatabaseDirectory $codeQLDatabaseDirectory 58 | ``` 59 | 60 | Specify an alternative query suite. 61 | ```powershell 62 | New-CodeQLScan -token $env:GITHUB_TOKEN -querySuite 'security-and-quality' 63 | ``` 64 | 65 | ## Azure DevOps 66 | Using the [PowerShell task](https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/powershell?view=azure-devops) in Azure DevOps, we can call the `New-CodeQLScan.ps1` script. Note that the `GITHUB_TOKEN` environment variable must be set to pass in the secret value defined in the `$(GITHUB_TOKEN)` pipeline variable. 67 | 68 | ```yaml 69 | jobs: 70 | - job: Job_1 71 | displayName: CodeQL Scan 72 | pool: 73 | name: Default 74 | steps: 75 | - task: PowerShell@2 76 | displayName: 'Checkout codeql-anywhere' 77 | inputs: 78 | targetType: inline 79 | script: | 80 | if (Test-Path './.git/modules/codeql-anywhere') {rm -rf './.git/modules/codeql-anywhere'} 81 | git submodule add https://github.com/david-wiggs/codeql-anywhere.git 82 | pwsh: true 83 | - task: PowerShell@2 84 | displayName: 'Run New-CodeQLScan.ps1' 85 | inputs: 86 | targetType: filePath 87 | filePath: 'codeql-anywhere/resources/scripts/New-CodeQLScan.ps1' 88 | pwsh: true 89 | env: 90 | GITHUB_TOKEN: $(GITHUB_TOKEN) 91 | ``` 92 | 93 | ## Jenkins 94 | This repository has the folder structure such that it can be consumed as a Jenkins [Shared Library](https://www.jenkins.io/doc/book/pipeline/shared-libraries/). Additionally, it contains the needed groovy script to call the `New-CodeQLScan.ps1` PowerShell script. Below is an example of a Jenkins Pipeline script that could be used. It assumes that there is a Jenkins credential named `codeql-token` of type _Secret Text_ that is the GitHub Personal Access Token to be used that uploads CodeQL results. 95 | 96 | ### Interpreted language example 97 | The below example demonstrates how to execute the `codeqlScan()` function included as a shared library. In this example, it is assumed that only interpreted languages are being scanned. 98 | 99 | ```groovy 100 | @Library('codeql-anywhere') _ 101 | pipeline { 102 | agent any 103 | environment { 104 | GITHUB_TOKEN = credentials('codeql-token') 105 | } 106 | stages { 107 | stage('Checkout') { 108 | steps { 109 | git branch: 'main', 110 | url: 'https://github.com/david-wiggs/codeql-testing.git' 111 | } 112 | } 113 | stage('Execute CodeQL Scan') { 114 | steps { 115 | codeqlScan() 116 | } 117 | } 118 | } 119 | } 120 | ``` 121 | 122 | ### Compiled language example 123 | The below example demonstrates how to pass the simple build command of `maven clean` (assuming a Java / maven project) so that CodeQL is able to successfully extract the source into a database 124 | 125 | ```groovy 126 | @Library('codeql-anywhere') _ 127 | pipeline { 128 | agent any 129 | environment { 130 | GITHUB_TOKEN = credentials('codeql-token') 131 | } 132 | stages { 133 | stage('Checkout') { 134 | steps { 135 | git branch: 'main', 136 | url: 'https://github.com/david-wiggs/codeql-testing.git' 137 | } 138 | } 139 | stage('Execute CodeQL Scan') { 140 | steps { 141 | codeqlScan() { 142 | maven clean 143 | } 144 | } 145 | } 146 | } 147 | } 148 | ``` 149 | 150 | # Interpreted Languages 151 | CodeQL supports the following interpreted languages (i.e., languages that don't require compilation): 152 | - Python 153 | - JavaScript 154 | - TypeScript 155 | - Ruby 156 | 157 | As such, if any of these languages is found to be present in a given repository, CodeQL will execute queries to analyze that language, respectively. 158 | 159 | # Compiled Languages 160 | The CodeQL currently supports [_autobuild_ features](https://codeql.github.com/docs/codeql-cli/creating-codeql-databases/#detecting-the-build-system). However, when complex combinations of compiled languages are passed in when using the `--db-cluster` parameters, sometimes the CodeQL cli will fail to build applications correctly, even if default application files, directories, an frameworks are leveraged. Because of this the `New-CodeQLScan` function attempts to build each compiled language, in serial, as described below. 161 | 162 | ## C / C++ 163 | In an attempt to cover both C and C++ the `New-CodeQLScan` functions assumes a `MAKEFILE` exists in the source root. A simple `make` is issued to compile source code. 164 | 165 | ## C# 166 | Assuming that the appropriate version of .NET is installed, `New-CodeQLScan` will attempt to execute `dotnet build /p:UseSharedCompilation=false /t:rebuild` in the source root. Note that the `/p:UseSharedCompilation=false` flag is required per [CodeQL documentation](https://codeql.github.com/docs/codeql-cli/creating-codeql-databases/#specifying-build-commands). It is also recommended that the `/t:rebuild` flag be used to ensure that the entire build process is captured. 167 | 168 | ## Java 169 | Assuming that Java is installed and the `$JAVA_HOME` environment variable is set appropriately, the `New-CodeQLScan` function will attempt to determine the appropriate build method. A general assumption is that if Maven or Gradle is used, that a respective wrapper file(s) exist (e.g., mvnw, or gradlew) somewhere in the working repository. 170 | 171 | ### Maven 172 | Assuming that Maven is installed, if a file that matches `*mvn*` is present in the repository, then an attempt to build the Java code will be made by executing `mvn clean --file $($pomFile.FullName) install` where `$($pomFile.FullName)` is the full path to `pom.xml` (assuming it exists). 173 | 174 | ### Gradle 175 | Assuming that Gradle is installed, if a file that matches `*gradle*` is present in the repository, then an attempt to build the Java code will be made by executing `gradle --no-daemon clean test`. 176 | 177 | ### Ant 178 | Assuming that Ant is installed, if a file that matches `*build.xml` is present in the repository, then an attempt to build the Java code will be made by executing `ant -f $antBuildFile`. 179 | 180 | ## Go 181 | The `New-CodeQLScan` function leverages the `CODEQL_EXTRACTOR_GO_BUILD_TRACING=on` environment variable in an attempt to automatically build Go source code. The first directory that contains a `.go` file is used to execute the build commands. If a specific build script is needed to compile the Go source code, consider leveraging the `-pathToBuildScript` parameter to specify the path to a file that gives general steps (e.g., scripts/build.sh). 182 | 183 | ## Build Script 184 | If the automated build logic for compiled languages does not fit your application, consider creating a file that defines the required build steps (e.g., commands). If there are multiple compiled languages in your application, this file should contain the needed steps to compile all languages, as only a single build file can be specified. 185 | 186 | Specify a build script file. 187 | ```powershell 188 | New-CodeQLScan -token $env:GITHUB_TOKEN -pathToBuildScript 'scripts/build.sh' 189 | ``` 190 | 191 | ## Dot Sourcing 192 | Execute the below in a PowerShell session to dot source the functions in the `functions.ps1` file in the `main` branch. 193 | 194 | ```powershell 195 | function Get-GitHubRepositoryFileContent { 196 | [CmdletBinding()] 197 | Param 198 | ( 199 | [Parameter(Mandatory = $True)] [string] $gitHubRepository, 200 | [Parameter(Mandatory = $True)] [string] $path, 201 | [Parameter(Mandatory = $True)] [string] $branch, 202 | [Parameter(Mandatory = $False)] [string] $token 203 | ) 204 | $uri = "https://api.github.com/repos/$gitHubRepository/contents/$path`?ref=$branch" # Need to escape the ? that indicates an http query 205 | $uri = [uri]::EscapeUriString($uri) 206 | $splat = @{ 207 | Method = 'Get' 208 | Uri = $uri 209 | Headers = $headers 210 | ContentType = 'application/json' 211 | } 212 | if ($PSBoundParameters.ContainsKey('token')) { 213 | $headers = @{'Authorization' = "token $token"} 214 | $splat.Add('Headers', $headers) 215 | } 216 | 217 | try { 218 | $fileData = Invoke-RestMethod @splat 219 | [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($fileData.content)) | Out-File -FilePath $(Split-Path $path -Leaf) -Force 220 | Get-Item -Path $(Split-Path $path -Leaf) 221 | } catch { 222 | Write-Warning "Unable to get file content." 223 | $ErrorMessage = $_.Exception.Message 224 | Write-Warning "$ErrorMessage" 225 | break 226 | } 227 | } 228 | 229 | function Get-DotSourceFileFromGitHub { 230 | [CmdletBinding()] 231 | Param 232 | ( 233 | [Parameter(Mandatory = $True)] [string] $gitHubRepository, 234 | [Parameter(Mandatory = $True)] [string] $path, 235 | [Parameter(Mandatory = $True)] [string] $branch, 236 | [Parameter(Mandatory = $False)] [string] $token 237 | ) 238 | 239 | $splat = @{ 240 | gitHubRepository = $gitHubRepository 241 | path = $path 242 | branch = $branch 243 | } 244 | if ($PSBoundParameters.ContainsKey('token')) {$splat.Add('token', $token)} 245 | $dotSourcefile = Get-GitHubRepositoryFileContent @splat 246 | $content = Get-Content -Path $dotSourcefile.FullName 247 | $content.Replace('function ', 'function Global:') | Out-File $dotSourceFile.FullName -Force 248 | . $dotSourcefile.FullName 249 | Remove-Item -Path $dotSourcefile.FullName -Force 250 | } 251 | 252 | $splat = @{ 253 | gitHubRepository = 'david-wiggs/codeql-anywhere' 254 | path = 'resources/functions.ps1' 255 | branch = 'main' 256 | } 257 | Get-DotSourceFileFromGitHub @splat 258 | ``` 259 | -------------------------------------------------------------------------------- /resources/functions.ps1: -------------------------------------------------------------------------------- 1 | function Get-LatestCodeQLBundle { 2 | if ($PSVersionTable.OS -like '*windows*') {$os = 'windows'} elseif ($PSVersionTable.OS -like '*linux*') {$os = 'linux'} elseif ($PSVersionTable.OS -like 'darwin*') {$os = 'macos'} else {Write-Error "Could not determine OS."; break} 3 | 4 | $splat = @{ 5 | Method = 'Get' 6 | Uri = 'https://api.github.com/repos/github/codeql-action/releases/latest' 7 | ContentType = 'application/json' 8 | } 9 | $codeQlLatestVersion = Invoke-RestMethod @splat 10 | 11 | if ($os -like 'linux') { 12 | $bundleName = 'codeql-bundle-linux64.tar.gz' 13 | } elseif ($os -like 'macos') { 14 | $bundleName = 'codeql-bundle-osx64.tar.gz' 15 | } elseif ($os -like 'windows') { 16 | $bundleName = 'codeql-bundle-win64.tar.gz' 17 | } 18 | 19 | $splat = @{ 20 | Method = 'Get' 21 | Uri = "https://github.com/github/codeql-action/releases/download/$($codeQlLatestVersion.tag_name)/$bundleName" 22 | ContentType = 'application/zip' 23 | } 24 | $activeTempRoot = (Get-PSDrive | Where-Object {$_.name -like 'Temp'}).Root 25 | if (Test-Path -Path "$activeTempRoot/codeql") {Remove-Item -Path "$activeTempRoot/codeql" -Recurse -Force} 26 | Invoke-RestMethod @splat -OutFile "$activeTempRoot/$bundleName" 27 | $oldLocation = Get-Location 28 | Set-Location -Path $activeTempRoot 29 | tar -xzf $bundleName 30 | Set-Location -Path $oldLocation.Path 31 | "$activeTempRoot" + "codeql" 32 | } 33 | 34 | function Get-GitHubRepositoryLanguages { 35 | [CmdletBinding()] 36 | Param 37 | ( 38 | [Parameter(Mandatory = $False)] [string] $token, 39 | [Parameter(Mandatory = $True)] [string] $owner, 40 | [Parameter(Mandatory = $True)] [string] $repositoryName 41 | ) 42 | 43 | $uri = "https://api.github.com/repos/$owner/$repositoryName" 44 | $splat = @{ 45 | Method = 'Get' 46 | Uri = $uri 47 | ContentType = 'application/json' 48 | } 49 | if ($PSBoundParameters.ContainsKey('token')) { 50 | $headers = @{'Authorization' = "token $token"} 51 | $splat.Add('Headers', $headers) 52 | } 53 | $repository = Invoke-RestMethod @splat 54 | $splat.Uri = $repository.languages_url 55 | (Invoke-RestMethod @splat | Select-Object -First 1 | Get-Member -MemberType NoteProperty).Name 56 | } 57 | 58 | function Get-GitHubRepositorySupportedCodeQLLanguages { 59 | [CmdletBinding()] 60 | Param 61 | ( 62 | [Parameter(Mandatory = $False)] [string] $token, 63 | [Parameter(Mandatory = $True)] [string] $owner, 64 | [Parameter(Mandatory = $True)] [string] $repositoryName 65 | ) 66 | 67 | $splat = @{ 68 | repositoryName = $repositoryName 69 | owner = $owner 70 | } 71 | if ($PSBoundParameters.ContainsKey('token')) { 72 | $splat.Add('token', $token) 73 | } 74 | [array]$repositoryLanguages = Get-GitHubRepositoryLanguages @splat 75 | $supportedCodeQLLanguages = @('c++', 'c', 'c#', 'go', 'java', 'javascript', 'python', 'ruby', 'typescript') 76 | $repositorySupportedCodeQLLanguages = $repositoryLanguages | Where-Object {$_ -in $supportedCodeQLLanguages} 77 | foreach ($language in $repositorySupportedCodeQLLanguages) { 78 | if ($language -like 'c++' -or $language -like 'c') { 79 | $prettyLanguage = 'cpp' 80 | } elseif ($language -like 'c#') { 81 | $prettyLanguage = 'csharp' 82 | } elseif ($language -like 'typescript') { 83 | $prettyLanguage = 'javascript' 84 | } else { 85 | $prettyLanguage = $language.ToLower() 86 | } 87 | [array]$returnLanguages += $prettyLanguage 88 | } 89 | $returnLanguages 90 | } 91 | 92 | function Set-GZipFile([ValidateScript({Test-Path $_})][string]$File){ 93 | 94 | $srcFile = Get-Item -Path $File 95 | $newFileName = "$($srcFile.FullName).gz" 96 | try { 97 | $srcFileStream = New-Object System.IO.FileStream($srcFile.FullName,([IO.FileMode]::Open),([IO.FileAccess]::Read),([IO.FileShare]::Read)) 98 | $dstFileStream = New-Object System.IO.FileStream($newFileName,([IO.FileMode]::Create),([IO.FileAccess]::Write),([IO.FileShare]::None)) 99 | $gzip = New-Object System.IO.Compression.GZipStream($dstFileStream,[System.IO.Compression.CompressionMode]::Compress) 100 | $srcFileStream.CopyTo($gzip) 101 | } catch { 102 | Write-Host "$_.Exception.Message" -ForegroundColor Red 103 | } finally { 104 | $gzip.Dispose() 105 | $srcFileStream.Dispose() 106 | $dstFileStream.Dispose() 107 | } 108 | } 109 | 110 | function Set-GitHubRepositorySarifResults { 111 | [CmdletBinding()] 112 | Param ( 113 | [Parameter(Mandatory = $True)] [string] $token , 114 | [Parameter(Mandatory = $True)] [string] $owner, 115 | [Parameter(Mandatory = $True)] [string] $repository, 116 | [Parameter(Mandatory = $True)] [string] $ref, 117 | [Parameter(Mandatory = $True)] [string] $commitSha, 118 | [Parameter(Mandatory = $True)] [string] $startedAt, 119 | [Parameter(Mandatory = $True)] [string] $pathToSarif, 120 | [Parameter(Mandatory = $True)] [string] $checkoutUri, 121 | [Parameter(Mandatory = $True)] [string] $toolName 122 | ) 123 | $headers = @{ 124 | Authorization = "token $token" 125 | Accept = 'application/vnd.github+json' 126 | } 127 | 128 | # Creates a based64 encoded string of the gzip compressed .sarif file 129 | Set-GZipFile -File $pathToSarif 130 | $gZipSarifFile = Get-Item -Path "$pathToSarif.gz" 131 | $bytes = [System.IO.File]::ReadAllBytes($gZipSarifFile.FullName) 132 | $base64Sarif = [System.Convert]::ToBase64String($bytes, [System.Base64FormattingOptions]::None) 133 | $uri = "https://api.github.com/repos/$owner/$repository/code-scanning/sarifs" 134 | try { 135 | $splat = @{ 136 | Method = 'Post' 137 | Uri = $uri 138 | Headers = $headers 139 | Body = @{ 140 | commit_sha = $commitSha 141 | ref = $ref 142 | sarif = $base64Sarif 143 | checkout_uri = $checkoutUri 144 | tool_name = $toolName 145 | started_at = $startedAt 146 | } | ConvertTo-Json 147 | ContentType = 'application/json' 148 | } 149 | Invoke-RestMethod @splat 150 | } catch { 151 | Write-Warning "Unable to upload SARIF results." 152 | $ErrorMessage = $_.Exception.Message 153 | Write-Warning "$ErrorMessage" 154 | } 155 | } 156 | 157 | function New-CodeQLScan { 158 | [CmdletBinding()] 159 | Param 160 | ( 161 | [Parameter(Mandatory = $False)] [array] $languages, 162 | [Parameter(Mandatory = $False)] [string] $token, 163 | [Parameter(Mandatory = $False)] [string] $codeQLDatabaseDirectoryPath = 'codeql/databases', 164 | [Parameter(Mandatory = $False)] [string] $pathToBuildScript, 165 | [Parameter(Mandatory = $False)] [string] $buildCmd, 166 | [Parameter(Mandatory = $False)] [switch] $keepSarif, 167 | [Parameter(Mandatory = $False)] [switch] $preventUploadResultsToGitHubCodeScanning, 168 | [Parameter(Mandatory = $False)] [string] [ValidateSet('code-scanning', 'security-extended', 'security-and-quality')] $querySuite = 'code-scanning' 169 | ) 170 | 171 | $originUrl = git remote get-url origin 172 | Write-Host "Origin URL is $originUrl." 173 | $owner = $originUrl.Split('/')[-2] 174 | Write-Host "Repository owner is $owner." 175 | $repositoryName = $originUrl.Split('/')[-1].Split('.')[0] 176 | Write-Host "Repository name is $repositoryName." 177 | $codeQLDatabaseDirectoryPath = 'codeql/databases' 178 | Write-Host "CodeQL database(s) directory is $codeQLDatabaseDirectoryPath." 179 | Write-Host "Query suite it $querySuite." 180 | if ($PSVersionTable.OS -like "*windows*") {$codeQlCmd = 'codeql.exe'} else {$codeQlCmd = 'codeql'} 181 | Write-Host "CodeQL executable is $codeqlCmd." 182 | if ($null -eq $env:CODEQL_LOCATION) { 183 | Write-Host "Getting latest CodeQL bundle." 184 | $codeQlDirectory = Get-LatestCodeQLBundle 185 | [System.Environment]::SetEnvironmentVariable('CODEQL_LOCATION',$codeQlDirectory,'Process') 186 | } else { 187 | $codeQlDirectory = $env:CODEQL_LOCATION 188 | } 189 | Write-Host "CodeQL directory is $codeqlDirectory." 190 | $sourceRoot = (Get-Location).Path 191 | 192 | $splat = @{ 193 | owner = $owner 194 | repositoryName = $repositoryName 195 | } 196 | if ($PSBoundParameters.ContainsKey('token')) {$splat.Add('token', $token)} 197 | Write-Host "Detecting repository languages supported by CodeQL." 198 | [array]$repositoryCodeQLSupportedLaguages = Get-GitHubRepositorySupportedCodeQLLanguages @splat | Select-Object -Unique 199 | if ($null -ne $repositoryCodeQLSupportedLaguages) { 200 | Write-Host "The following languages that are supported by CodeQL were detected: $($repositoryCodeQLSupportedLaguages -join ', ')." 201 | if (Test-Path $codeQLDatabaseDirectoryPath) {Remove-Item -Path $codeQLDatabaseDirectoryPath -Recurse -Force} 202 | $codeQLDatabaseDirectory = (New-Item -Path $codeQLDatabaseDirectoryPath -ItemType Directory).FullName 203 | New-Item -ItemType Directory -Path "$codeQLDatabaseDirectoryPath/inpterpretted" | Out-Null 204 | New-Item -ItemType Directory -Path "$codeQLDatabaseDirectoryPath/compiled" | Out-Null 205 | } else { 206 | Write-Warning "The repository, $owner/$repository does not contain any languages that are supported by CodeQL." 207 | break 208 | } 209 | $startedAt = (Get-date).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:ssZ") 210 | 211 | [array]$nonCompiledLanguages = $repositoryCodeQLSupportedLaguages | Where-Object {$_ -like 'javascript' -or $_ -like 'python' -or $_ -like 'ruby'} 212 | if ($null -ne $nonCompiledLanguages) { 213 | Write-Host "Creating CodeQL databases for non-compiled languages." 214 | if ($nonCompiledLanguages.Count -gt 1) { 215 | Invoke-Expression -Command "$(Join-Path -Path $codeQlDirectory $codeQlCmd) database create --language=$($nonCompiledLanguages -join ',') --source-root . --db-cluster $codeQLDatabaseDirectory/inpterpretted" 216 | } elseif ($nonCompiledLanguages.Count -eq 1) { 217 | Invoke-Expression -Command "$(Join-Path -Path $codeQlDirectory $codeQlCmd) database create --language=$($nonCompiledLanguages[0]) --source-root . $codeQLDatabaseDirectory/inpterpretted/$($nonCompiledLanguages[0])" 218 | } 219 | } 220 | 221 | [array]$compiledLanguages = $repositoryCodeQLSupportedLaguages | Where-Object {$_ -like 'cpp' -or $_ -like 'java' -or $_ -like 'csharp' -or $_ -like 'go'} 222 | if ($null -ne $compiledLanguages) { 223 | Write-Host "Creating CodeQL databases for compiled languages." 224 | if ($PSBoundParameters.ContainsKey('pathToBuildScript')) { 225 | Write-Host "Using build script at $pathToBuildScript to build all detected compiled lanuages." 226 | try { 227 | Invoke-Expression -Command "$(Join-Path -Path $codeQlDirectory $codeQlCmd) database create --language=$($compiledLanguages -join ',') --source-root . $codeQLDatabaseDirectory/compiled --command='$((Get-Item $pathToBuildScript).FullName)'" 228 | } 229 | catch { 230 | Write-Error "Unable able to build compiled language project(s) and create a CodeQL database(s)." 231 | } 232 | } elseif ($PSBoundParameters.ContainsKey('buildCmd')) { 233 | Write-Host "Using build cmd '$buildCmd' to build all detected compiled lanuages." 234 | try { 235 | Invoke-Expression -Command "$(Join-Path -Path $codeQlDirectory $codeQlCmd) database create --language=$($compiledLanguages -join ',') --source-root . $codeQLDatabaseDirectory/compiled --command='$buildCmd'" 236 | } 237 | catch { 238 | Write-Error "Unable able to build compiled language project(s) and create a CodeQL database(s)." 239 | } 240 | } else { 241 | foreach ($language in $compiledLanguages) { 242 | if ($language -like 'cpp') { 243 | try { 244 | Write-Host "Attempting to build C / C++ project with Makefile." 245 | Invoke-Expression -Command "$(Join-Path -Path $codeQlDirectory $codeQlCmd) database create --language=cpp --source-root . $codeQLDatabaseDirectory/compiled/cpp --command=make" 246 | } 247 | catch { 248 | Write-Error "Unable able to autobuild C / C++ project and create a CodeQL database." 249 | Write-Error "Consider supplying a path to a build script with the -pathToBuildScript parameter." 250 | } 251 | } elseif ($language -like 'csharp') { 252 | if ((Test-Path "*.csproj") -or (Test-Path "*.sln")) { 253 | try { 254 | Write-Host "Attempting to build C# project with dotnet build." 255 | Invoke-Expression -Command "$(Join-Path -Path $codeQlDirectory $codeQlCmd) database create --language=csharp --source-root . $codeQLDatabaseDirectory/compiled/csharp --command='dotnet build /p:UseSharedCompilation=false /t:rebuild'" 256 | } 257 | catch { 258 | Write-Error "Unable able to autobuild C# project and create a CodeQL database." 259 | Write-Error "Consider supplying a path to a build script with the -pathToBuildScript parameter." 260 | } 261 | } else { 262 | Write-Error "Unable able to autobuild C# project and create a CodeQL database." 263 | Write-Error "Detected C# and did not find a .sln or .csproj file present in the root directory of the repository." 264 | } 265 | 266 | } elseif ($language -like 'java') { 267 | $antBuildFile = Get-ChildItem -Path $sourceRoot -Recurse -Filter "*build.xml" | Select-Object -First 1 268 | if ($null -ne $antBuildFile) { 269 | $antBuildFile = $antBuildFile.FullName 270 | } 271 | 272 | if ($null -ne (Get-ChildItem -Path $sourceRoot -Recurse -Filter "*mvn*")) { 273 | Write-Host "Detected Maven files." 274 | $pomFile = Get-ChildItem -Recurse -Include 'pom.xml' 275 | Write-Host "Attempting to build Java project with Maven." 276 | if (Test-Path $pomFile.FullName) { 277 | try { 278 | Invoke-Expression -Command "$(Join-Path -Path $codeQlDirectory $codeQlCmd) database create --language=java --source-root . $codeQLDatabaseDirectory/compiled/java --command='mvn clean --file $($pomFile.FullName) install -DskipTests'" 279 | } 280 | catch { 281 | Write-Error "Unable able to autobuild Java project." 282 | Write-Error "Consider supplying a path to a build script with the -pathToBuildScript parameter." 283 | } 284 | } else { 285 | Write-Error "Unable able to autobuild Java project and create a CodeQL database." 286 | Write-Error "Detected Maven files but did not find a pom.xml file." 287 | } 288 | } elseif ($null -ne (Get-ChildItem -Path $sourceRoot -Recurse -Filter "*gradle*")) { 289 | Write-Host "Detected Gradle files." 290 | Write-Host "Attempting to build Java project with Gradle." 291 | try { 292 | # need to set working directory? 293 | Invoke-Expression -Command "$(Join-Path -Path $codeQlDirectory $codeQlCmd) database create --language=java --source-root . $codeQLDatabaseDirectory/compiled/java --command='gradle --no-daemon clean test'" 294 | } 295 | catch { 296 | Write-Error "Unable able to autobuild Java project and create a CodeQL database." 297 | Write-Error "Consider supplying a path to a build script with the -pathToBuildScript parameter." 298 | } 299 | } elseif ($null -ne $antBuildFile) { 300 | Write-Host "Detected Ant build files." 301 | Write-Host "Attempting to build Java project with Ant." 302 | try { 303 | Invoke-Expression -Command "$(Join-Path -Path $codeQlDirectory $codeQlCmd) database create --language=java --source-root . $codeQLDatabaseDirectory/compiled/java --command='ant -f $antBuildFile'" 304 | } 305 | catch { 306 | Write-Error "Unable able to autobuild Java project and create a CodeQL database." 307 | Write-Error "Consider supplying a path to a build script with the -pathToBuildScript parameter." 308 | } 309 | } 310 | } elseif ($language -like 'go') { 311 | Write-Host "Attempting to build Go project with CODEQL_EXTRACTOR_GO_BUILD_TRACING = 'on'." 312 | try { 313 | $goSourceRoot = Split-Path (Get-ChildItem -Recurse -Include '*.go' | Select-Object -First 1) -Parent 314 | $env:CODEQL_EXTRACTOR_GO_BUILD_TRACING = 'on' 315 | Invoke-Expression -Command "$(Join-Path -Path $codeQlDirectory $codeQlCmd) database create --language=go --source-root $sourceRoot $codeQLDatabaseDirectory/compiled/go --working-dir $goSourceRoot" 316 | } 317 | catch { 318 | Write-Error "Unable able to autobuild Go project and create a CodeQL database." 319 | Write-Error "Consider supplying a path to a build script with the -pathToBuildScript parameter." 320 | } 321 | } 322 | } 323 | } 324 | } 325 | 326 | Write-Host "Analyzing CodeQL databases." 327 | [array]$codeQLDatabases = Get-ChildItem -Path "$codeQLDatabaseDirectory/inpterpretted" -Directory -Exclude log, working 328 | [array]$codeQLDatabases += Get-Item -Path "$codeQLDatabaseDirectory/compiled" 329 | foreach ($database in $codeQLDatabases) { 330 | $language = $database.Name 331 | $queries = Get-ChildItem -Recurse -Filter "*$language-$querySuite.qls" 332 | Write-Host "Analyzing $language database." 333 | Invoke-Expression -Command "$(Join-Path -Path $codeQlDirectory $codeQlCmd) database analyze $($database.FullName) $($queries.FullName) --format=sarifv2.1.0 --output=$language-results.sarif --sarif-category=$language" 334 | if (-not $preventUploadResultsToGitHubCodeScanning) { 335 | if ($null -ne $env:BUILD_SOURCEBRANCH) {$ref = $env:BUILD_SOURCEBRANCH} else {$ref = $(git symbolic-ref HEAD)} 336 | if ($null -ne $env:BUILD_SOURCEVERSION) {$sha = $env:BUILD_SOURCEVERSION} else {$sha = $(git rev-parse --verify HEAD)} 337 | $splat = @{ 338 | owner = $owner 339 | repository = $repositoryName 340 | ref = $ref 341 | startedAt = $startedAt 342 | commitSha = $sha 343 | pathToSarif = "$language-results.sarif" 344 | checkoutUri = $sourceRoot 345 | toolName = 'CodeQL' 346 | } 347 | if ($PSBoundParameters.ContainsKey('token')) {$splat.Add('token', $token)} 348 | Write-Host "Uploading SARIF results for $owner / $repositoryName for $language to GitHub Code Scanning." 349 | Set-GitHubRepositorySarifResults @splat 350 | } 351 | Get-ChildItem -Path $sourceRoot -Filter "*-results.sarif.gz*" | Remove-Item -Force 352 | if (-not $keepSarif) {Get-ChildItem -Path $sourceRoot -Filter "*-results.sarif" | Remove-Item -Force} 353 | } 354 | Remove-Item -Path (Split-Path $codeQLDatabaseDirectory -Parent) -Recurse -Force 355 | } 356 | 357 | function Set-GitHubRepositoryDispatch { 358 | [CmdletBinding()] 359 | Param 360 | ( 361 | [Parameter(Mandatory = $False)] [string] $token, 362 | [Parameter(Mandatory = $True)] [string] $owner, 363 | [Parameter(Mandatory = $True)] [string] $repositoryName, 364 | [Parameter(Mandatory = $True)] [string] $branch 365 | ) 366 | 367 | $uri = "https://api.github.com/repos/$owner/$repositoryName/dispatches" 368 | $body = [PSCustomObject]@{ 369 | event_type = 'compliance-check' 370 | client_payload = @{ 371 | head_branch = $branch 372 | } 373 | } | ConvertTo-Json -Depth 10 374 | 375 | $splat = @{ 376 | Method = 'Post' 377 | Uri = $uri 378 | ContentType = 'application/json' 379 | Body = $body 380 | 381 | } 382 | if ($PSBoundParameters.ContainsKey('token')) { 383 | $headers = @{'Authorization' = "token $token"} 384 | $splat.Add('Headers', $headers) 385 | } 386 | Invoke-RestMethod @splat 387 | } 388 | -------------------------------------------------------------------------------- /resources/scripts/New-CodeQLScan.ps1: -------------------------------------------------------------------------------- 1 | [CmdletBinding()] 2 | Param 3 | ( 4 | [Parameter(Mandatory = $False)] [string] $token, 5 | [Parameter(Mandatory = $False)] [string] $codeQLDatabaseDirectoryPath, 6 | [Parameter(Mandatory = $False)] [string] $buildCmd, 7 | [Parameter(Mandatory = $False)] [string] $pathToBuildScript, 8 | [Parameter(Mandatory = $False)] [string] [ValidateSet('code-scanning', 'security-extended', 'security-and-quality')] $querySuite 9 | ) 10 | 11 | function Get-GitHubRepositoryFileContent { 12 | [CmdletBinding()] 13 | Param 14 | ( 15 | [Parameter(Mandatory = $True)] [string] $gitHubRepository, 16 | [Parameter(Mandatory = $True)] [string] $path, 17 | [Parameter(Mandatory = $True)] [string] $branch, 18 | [Parameter(Mandatory = $False)] [string] $token 19 | ) 20 | $uri = "https://api.github.com/repos/$gitHubRepository/contents/$path`?ref=$branch" # Need to escape the ? that indicates an http query 21 | $uri = [uri]::EscapeUriString($uri) 22 | $splat = @{ 23 | Method = 'Get' 24 | Uri = $uri 25 | Headers = $headers 26 | ContentType = 'application/json' 27 | } 28 | if ($PSBoundParameters.ContainsKey('token')) { 29 | $headers = @{'Authorization' = "token $token"} 30 | $splat.Add('Headers', $headers) 31 | } 32 | 33 | try { 34 | $fileData = Invoke-RestMethod @splat 35 | [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($fileData.content)) | Out-File -FilePath $(Split-Path $path -Leaf) -Force 36 | Get-Item -Path $(Split-Path $path -Leaf) 37 | } catch { 38 | Write-Warning "Unable to get file content." 39 | $ErrorMessage = $_.Exception.Message 40 | Write-Warning "$ErrorMessage" 41 | break 42 | } 43 | } 44 | 45 | function Get-DotSourceFileFromGitHub { 46 | [CmdletBinding()] 47 | Param 48 | ( 49 | [Parameter(Mandatory = $True)] [string] $gitHubRepository, 50 | [Parameter(Mandatory = $True)] [string] $path, 51 | [Parameter(Mandatory = $True)] [string] $branch, 52 | [Parameter(Mandatory = $False)] [string] $token 53 | ) 54 | 55 | $splat = @{ 56 | gitHubRepository = $gitHubRepository 57 | path = $path 58 | branch = $branch 59 | } 60 | if ($PSBoundParameters.ContainsKey('token')) {$splat.Add('token', $token)} 61 | $dotSourcefile = Get-GitHubRepositoryFileContent @splat 62 | $content = Get-Content -Path $dotSourcefile.FullName 63 | $content.Replace('function ', 'function Global:') | Out-File $dotSourceFile.FullName -Force 64 | . $dotSourcefile.FullName 65 | Remove-Item -Path $dotSourcefile.FullName -Force 66 | } 67 | 68 | $splat = @{ 69 | gitHubRepository = 'david-wiggs/codeql-anywhere' 70 | path = 'resources/functions.ps1' 71 | branch = 'main' 72 | } 73 | Get-DotSourceFileFromGitHub @splat 74 | 75 | $splat = @{} 76 | if ($null -ne $($env:GITHUB_TOKEN)) {$splat.Add('token', $env:GITHUB_TOKEN)} elseif ($PSBoundParameters.ContainsKey('token')) {$splat.Add('token', $token)} 77 | if ($null -ne $codeQLDatabaseDirectoryPath) {$splat.Add('codeQLDatabaseDirectoryPath', $codeQLDatabaseDirectoryPath)} 78 | if ($pathToBuildScript -ne '') {$splat.Add('pathToBuildScript', $pathToBuildScript)} 79 | if ($buildCmd -ne '') {$splat.Add('buildCmd', $buildCmd)} 80 | if ($PSBoundParameters.ContainsKey('querySuite')) {$splat.Add('querySuite', $querySuite)} 81 | 82 | New-CodeQLScan @splat -------------------------------------------------------------------------------- /resources/scripts/Set-CommitStatusChecks.ps1: -------------------------------------------------------------------------------- 1 | [CmdletBinding()] 2 | Param 3 | ( 4 | [Parameter(Mandatory = $False)] [string] $token 5 | ) 6 | 7 | function Get-GitHubRepositoryFileContent { 8 | [CmdletBinding()] 9 | Param 10 | ( 11 | [Parameter(Mandatory = $True)] [string] $gitHubRepository, 12 | [Parameter(Mandatory = $True)] [string] $path, 13 | [Parameter(Mandatory = $True)] [string] $branch, 14 | [Parameter(Mandatory = $False)] [string] $token 15 | ) 16 | $uri = "https://api.github.com/repos/$gitHubRepository/contents/$path`?ref=$branch" # Need to escape the ? that indicates an http query 17 | $uri = [uri]::EscapeUriString($uri) 18 | $splat = @{ 19 | Method = 'Get' 20 | Uri = $uri 21 | Headers = $headers 22 | ContentType = 'application/json' 23 | } 24 | if ($PSBoundParameters.ContainsKey('token')) { 25 | $headers = @{'Authorization' = "token $token"} 26 | $splat.Add('Headers', $headers) 27 | } 28 | 29 | try { 30 | $fileData = Invoke-RestMethod @splat 31 | [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($fileData.content)) | Out-File -FilePath $(Split-Path $path -Leaf) -Force 32 | Get-Item -Path $(Split-Path $path -Leaf) 33 | } catch { 34 | Write-Warning "Unable to get file content." 35 | $ErrorMessage = $_.Exception.Message 36 | Write-Warning "$ErrorMessage" 37 | break 38 | } 39 | } 40 | 41 | function Get-DotSourceFileFromGitHub { 42 | [CmdletBinding()] 43 | Param 44 | ( 45 | [Parameter(Mandatory = $True)] [string] $gitHubRepository, 46 | [Parameter(Mandatory = $True)] [string] $path, 47 | [Parameter(Mandatory = $True)] [string] $branch, 48 | [Parameter(Mandatory = $False)] [string] $token 49 | ) 50 | 51 | $splat = @{ 52 | gitHubRepository = $gitHubRepository 53 | path = $path 54 | branch = $branch 55 | } 56 | if ($PSBoundParameters.ContainsKey('token')) {$splat.Add('token', $token)} 57 | $dotSourcefile = Get-GitHubRepositoryFileContent @splat 58 | $content = Get-Content -Path $dotSourcefile.FullName 59 | $content.Replace('function ', 'function Global:') | Out-File $dotSourceFile.FullName -Force 60 | . $dotSourcefile.FullName 61 | Remove-Item -Path $dotSourcefile.FullName -Force 62 | } 63 | 64 | $splat = @{ 65 | gitHubRepository = 'david-wiggs/codeql-anywhere' 66 | path = 'resources/functions.ps1' 67 | branch = 'main' 68 | } 69 | Get-DotSourceFileFromGitHub @splat 70 | 71 | $splat = @{} 72 | if ($null -ne $($env:GITHUB_TOKEN)) {$splat.Add('token', $env:GITHUB_TOKEN)} elseif ($PSBoundParameters.ContainsKey('token')) {$splat.Add('token', $token)} 73 | $originUrl = git remote get-url origin 74 | $splat.Add('owner', $originUrl.Split('/')[-2]) 75 | $splat.Add('repositoryName', $originUrl.Split('/')[-1].Split('.')[0]) 76 | $splat.Add('branch', $(git symbolic-ref HEAD).split('/')[-1]) 77 | Set-GitHubRepositoryDispatch @splat 78 | -------------------------------------------------------------------------------- /vars/codeqlScan.groovy: -------------------------------------------------------------------------------- 1 | def call (Map params, Closure closure = null) { 2 | def languages = processLanguages(params['languages']) 3 | def ref = processRef(params['ref']) 4 | def ram = defaultIfNullOrEmtpy(params['ram'] as Integer, 4000) 5 | def threads = defaultIfNullOrEmtpy(params['threads'] as Integer, 1) 6 | def verbosity = defaultIfNullOrEmtpy(params['verbosity'], 'errors') 7 | def querySuite = defaultIfNullOrEmtpy(params['querySuite'], 'code-scanning') 8 | def origin = pwsh(script: 'git config --get remote.origin.url', returnStdout: true).trim() 9 | def org = origin.tokenize('/')[-2] 10 | def repo = origin.tokenize('/')[-1].tokenize('.')[0] 11 | def commit = pwsh(script: 'git rev-parse --verify HEAD', returnStdout: true).trim() 12 | 13 | createCodeqlFolders() 14 | codeqlInstall() 15 | 16 | def tmp = getCodeqlTempFolder() 17 | def databasesCompiled = "${WORKSPACE}/${tmp}/databases-compiled" 18 | def databasesInterpreted = "${WORKSPACE}/${tmp}/databases-interpreted" 19 | def supportedInterpretedLanguages = getInterpretedLanguages() 20 | def detectedInterpretedLanguages = supportedInterpretedLanguages.intersect(languages) 21 | def supportedCompiledLanguages = getCompiledLangauges() 22 | def detectedCompiledLanguages = supportedCompiledLanguages.intersect(languages) 23 | 24 | if (detectedCompiledLanguages && closure == null){ 25 | logWarn('CodeQL requires the build to be placed inside a closure for compiled language identifiers! Compiled languages will not be scanned.') 26 | detectedCompiledLanguages = [] 27 | } 28 | 29 | if (detectedCompiledLanguages && detectedInterpretedLanguages) { 30 | createTracedDatabses([codeqlDatabase: databasesCompiled, languages: detectedCompiledLanguages, ram: ram, threads: threads, verbosity: verbosity], closure) 31 | createStandardDatabases([codeqlDatabase: databasesInterpreted, languages:detectedInterpretedLanguages, ram: ram, threads: threads, verbosity: verbosity]) 32 | } else if (detectedCompiledLanguages) { 33 | createdTracedDatabases([codeqlDatabase: databasesCompiled, languages: detectedCompiledLanguages, ram: ram, threads: threads, verbosity: verbosity], closure) 34 | } else { 35 | createStandardDatabases([codeqlDatabase: databasesInterpreted, languages:detectedInterpretedLanguages, ram: ram, threads: threads, verbosity: verbosity]) 36 | } 37 | 38 | def detectedLanguages = detectedCompiledLanguages + detectedInterpretedLanguages 39 | for (language in detectedLanguages) { 40 | def codeqlDatabase 41 | if (detectedCompiledLanguages.contains(language)) { 42 | codeqlDatabase = databasesCompiled 43 | } else { 44 | codeqlDatabase = databasesInterpreted 45 | } 46 | 47 | codeqlDatabase = "${codeqlDatabase}/${language}" 48 | def sarifResults = "${WORKSPACE}/${tmp}/results/${language}-results.sarif" 49 | analyze([codeqlDatabase: codeqlDatabase, querySuite: querySuite, category: language, sarifResults: sarifResults, ram: ram, threads: threads, verbosity: verbosity]) 50 | uploadScanResults([sarifResults: sarifResults, org: org, repo: repo, ref: ref, commit: commit, verbosity: verbosity]) 51 | } 52 | 53 | pwsh("Remove-Item ${WORKSPACE}/${tmp} -Recurse -Force") 54 | } 55 | 56 | def defaultIfNullOrEmtpy(val, defaultVal) { 57 | if (val == null || val == "") { 58 | return defaultVal 59 | } 60 | return val 61 | } 62 | 63 | def getCodeqlTempFolder() { 64 | return '.codeql-tmp' 65 | } 66 | 67 | def createCodeqlFolders() { 68 | def tmp = getCodeqlTempFolder() 69 | pwsh(""" 70 | if (Test-Path -Path ${WORKSPACE}/${tmp}) {Remove-Item ${WORKSPACE}/${tmp} -Recurse -Force | Out-Null} 71 | New-Item -ItemType Directory -Path ${WORKSPACE}/${tmp}/database | Out-Null 72 | New-Item -ItemType Directory -Path ${WORKSPACE}/${tmp}/results | Out-Null 73 | New-Item -ItemType Directory -Path ${WORKSPACE}/${tmp}/codeql | Out-Null 74 | """) 75 | } 76 | 77 | def getCodeqlExecutable() { 78 | def tmp = getCodeqlTempFolder() 79 | return "${WORKSPACE}/${tmp}/codeql/codeql" 80 | } 81 | 82 | def codeqlInstall() { 83 | def tmp = getCodeqlTempFolder() 84 | withEnv(["tmp=${tmp}"]) { 85 | pwsh(''' 86 | if ($PSVersionTable.OS -like '*windows*') {$os = 'windows'} elseif ($PSVersionTable.OS -like '*linux*') {$os = 'linux'} elseif ($PSVersionTable.OS -like 'darwin*') {$os = 'macos'} else {Write-Error "Could not determine OS."; break} 87 | 88 | $splat = @{ 89 | Method = 'Get' 90 | Uri = 'https://api.github.com/repos/github/codeql-action/releases/latest' 91 | ContentType = 'application/json' 92 | } 93 | $codeQlLatestVersion = Invoke-RestMethod @splat 94 | 95 | if ($os -like 'linux') { 96 | $bundleName = 'codeql-bundle-linux64.tar.gz' 97 | } elseif ($os -like 'macos') { 98 | $bundleName = 'codeql-bundle-osx64.tar.gz' 99 | } elseif ($os -like 'windows') { 100 | $bundleName = 'codeql-bundle-win64.tar.gz' 101 | } 102 | 103 | $splat = @{ 104 | Method = 'Get' 105 | Uri = "https://github.com/github/codeql-action/releases/download/$($codeQlLatestVersion.tag_name)/$bundleName" 106 | ContentType = 'application/zip' 107 | } 108 | 109 | Invoke-RestMethod @splat -OutFile $env:WORKSPACE/$env:tmp/$bundleName 110 | tar -xzf $env:WORKSPACE/$env:tmp/$bundleName -C $env:WORKSPACE/$env:tmp 111 | Remove-Item $env:WORKSPACE/$env:tmp/$bundleName -Force 112 | ''') 113 | } 114 | } 115 | 116 | def getCompiledLangauges() { 117 | return ['cpp', 'csharp', 'go', 'java'] 118 | } 119 | 120 | def getInterpretedLanguages() { 121 | return ['javascript', 'python', 'ruby'] 122 | } 123 | 124 | def getSupportedLanguages() { 125 | def compiledLanguages = getCompiledLangauges() 126 | def interpretedLanguages = getInterpretedLanguages() 127 | return compiledLanguages + interpretedLanguages 128 | } 129 | 130 | def processLanguages(List suppliedLanguages) { 131 | if (suppliedLanguages.isEmpty()) { 132 | logAndRaiseError("No languages were supplied in the 'languages' parameter. You must provide a list of languages.") 133 | } 134 | 135 | def languages = suppliedLanguages.collect { it.toLowerCase() } 136 | def supportedLanguages = getSupportedLanguages() 137 | def isEveryLanguageSupported = languages.every { supportedLanguages.contains(it) } 138 | 139 | if (!isEveryLanguageSupported) { 140 | logAndRaiseError("The following languages were specified: ${languages}. CodeQL currently supports ${supportedLanguages}") 141 | } 142 | 143 | return languages 144 | } 145 | 146 | def createTracedDatabases(Map params, Closure closure) { 147 | def codeqlDatabase = params['codeqlDatabase'] 148 | def languages = params['languages'] 149 | def ram = params['ram'] 150 | def threads = params['threads'] 151 | def verbosity = params['verbosity'] 152 | def listOfLanguages = languages.join(",") 153 | def codeql = getCodeqlExecutable() 154 | 155 | pwsh(""" 156 | ${codeql} database init ${codeqlDatabase} \ 157 | --source-root=. \ 158 | --language=${listOfLanguages} \ 159 | --begin-tracing \ 160 | --db-cluster \ 161 | --overwrite \ 162 | --verbosity=${verbosity} 163 | """) 164 | 165 | def codeqlTracingScript = "${codeqlDatabase}/temp/tracingEnvironment/start-tracing.sh" 166 | def tracingScriptContent = pwsh(script: "Get-Content -Path ${codeqlTracingScript}", returnStdout: true).trim() 167 | def effectiveOverrides = tracingScriptContent 168 | .split('\n') 169 | .collect { 170 | line -> line.replace("export ", "") 171 | .replace("\'", "") 172 | .replace("\"", "") 173 | } 174 | 175 | withEnv(effectiveOverrides) { 176 | closure.call() 177 | } 178 | 179 | pwsh(""" 180 | ${codeql} database finalize ${codeqlDatabase} \ 181 | --db-cluster \ 182 | --ram=${ram} \ 183 | --threads=${threads} \ 184 | --verbosity=${verbosity} 185 | """) 186 | } 187 | 188 | def createStandardDatabases(Map params) { 189 | def codeqlDatabase = params['codeqlDatabase'] 190 | def languages = params['languages'] 191 | def ram = params['ram'] 192 | def threads = params['threads'] 193 | def verbosity = params['verbosity'] 194 | def listOfLanguages = languages.join(",") 195 | def codeql = getCodeqlExecutable() 196 | 197 | pwsh(""" 198 | ${codeql} database create ${codeqlDatabase} \ 199 | --language=${listOfLanguages} \ 200 | --db-cluster \ 201 | --overwrite \ 202 | --ram=${ram} \ 203 | --threads=${threads} \ 204 | --verbosity=${verbosity} 205 | """) 206 | } 207 | 208 | def analyze(Map params) { 209 | def codeqlDatabase = params['codeqlDatabase'] 210 | def querySuite = params['querySuite'] 211 | def category = params['category'] 212 | def sarifResults = params['sarifResults'] 213 | def ram = params['ram'] 214 | def threads = params['threads'] 215 | def verbosity = params['verbosity'] 216 | def codeql = getCodeqlExecutable() 217 | def queries = pwsh(script:"(Get-ChildItem -Recurse -Filter '*${category}-${querySuite}.qls').FullName", returnStdout: true).trim() 218 | 219 | pwsh(""" 220 | ${codeql} database analyze ${codeqlDatabase} ${queries} \ 221 | --format=sarif-latest \ 222 | --sarif-category=${category} \ 223 | --output=${sarifResults} \ 224 | --ram=${ram} \ 225 | --threads=${threads} \ 226 | --verbosity=${verbosity} 227 | """) 228 | } 229 | 230 | def uploadScanResults(Map params) { 231 | def sarifResults = params['sarifResults'] 232 | def org = params['org'] 233 | def repo = params['repo'] 234 | def ref = params['ref'] 235 | def commit = params['commit'] 236 | def verbosity = params['verbosity'] 237 | def codeql = getCodeqlExecutable() 238 | def repository = org + '/' + repo 239 | 240 | dir("${WORKSPACE}") { 241 | pwsh(""" 242 | ${codeql} github upload-results \ 243 | --sarif=${sarifResults} \ 244 | --ref=${ref} \ 245 | --repository=${repository} \ 246 | --commit=${commit} \ 247 | --verbosity=${verbosity}""" 248 | ) 249 | } 250 | } 251 | 252 | def processRef(String suppliedRef) { 253 | def ref 254 | 255 | if (suppliedRef) { 256 | if (suppliedRef ==~ /(^refs\/(heads|tags)\/.*)|(^refs\/pull\/\d+\/(merge|head))/) { 257 | ref = suppliedRef 258 | } else { 259 | logAndRaiseError("Supplied ref '${suppliedRef}' does not match expected formats:\n'refs/heads/'\n'refs/tags/'\n'refs/pull//merge'\n'refs/pull//head'") 260 | } 261 | } else { 262 | ref = pwsh(script:'git symbolic-ref HEAD', returnStdout: true).trim() 263 | } 264 | 265 | return ref 266 | } 267 | 268 | def logAndRaiseError(message, String ... parameters) { 269 | def messageParameters = ['ERROR'] 270 | messageParameters.addAll(parameters) 271 | def sanitizedInput = messageParameters.collect { it.replaceAll("%","%%") } 272 | error(sprintf("[%s] ${message}", sanitizedInput)) 273 | } 274 | --------------------------------------------------------------------------------