├── .github └── CODEOWNERS ├── .gitignore ├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md ├── SECURITY.md ├── SUPPORT.md ├── bin ├── codeql-analyze ├── codeql-init ├── codeql-upload └── codeql-utils └── gh-codeql-scan /.github/CODEOWNERS: -------------------------------------------------------------------------------- 1 | # This project is maintained with love by: 2 | 3 | * @geekmasher 4 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | 3 | .database/ 4 | .results/ 5 | 6 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | We as members, contributors, and leaders pledge to make participation in our 6 | community a harassment-free experience for everyone, regardless of age, body 7 | size, visible or invisible disability, ethnicity, sex characteristics, gender 8 | identity and expression, level of experience, education, socio-economic status, 9 | nationality, personal appearance, race, religion, or sexual identity 10 | and orientation. 11 | 12 | We pledge to act and interact in ways that contribute to an open, welcoming, 13 | diverse, inclusive, and healthy community. 14 | 15 | ## Our Standards 16 | 17 | Examples of behavior that contributes to a positive environment for our 18 | community include: 19 | 20 | * Demonstrating empathy and kindness toward other people 21 | * Being respectful of differing opinions, viewpoints, and experiences 22 | * Giving and gracefully accepting constructive feedback 23 | * Accepting responsibility and apologizing to those affected by our mistakes, 24 | and learning from the experience 25 | * Focusing on what is best not just for us as individuals, but for the 26 | overall community 27 | 28 | Examples of unacceptable behavior include: 29 | 30 | * The use of sexualized language or imagery, and sexual attention or 31 | advances of any kind 32 | * Trolling, insulting or derogatory comments, and personal or political attacks 33 | * Public or private harassment 34 | * Publishing others' private information, such as a physical or email 35 | address, without their explicit permission 36 | * Other conduct which could reasonably be considered inappropriate in a 37 | professional setting 38 | 39 | ## Enforcement Responsibilities 40 | 41 | Community leaders are responsible for clarifying and enforcing our standards of 42 | acceptable behavior and will take appropriate and fair corrective action in 43 | response to any behavior that they deem inappropriate, threatening, offensive, 44 | or harmful. 45 | 46 | Community leaders have the right and responsibility to remove, edit, or reject 47 | comments, commits, code, wiki edits, issues, and other contributions that are 48 | not aligned to this Code of Conduct, and will communicate reasons for moderation 49 | decisions when appropriate. 50 | 51 | ## Scope 52 | 53 | This Code of Conduct applies within all community spaces, and also applies when 54 | an individual is officially representing the community in public spaces. 55 | Examples of representing our community include using an official e-mail address, 56 | posting via an official social media account, or acting as an appointed 57 | representative at an online or offline event. 58 | 59 | ## Enforcement 60 | 61 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 62 | reported to the community leaders responsible for enforcement at 63 | opensource@github.com. 64 | All complaints will be reviewed and investigated promptly and fairly. 65 | 66 | All community leaders are obligated to respect the privacy and security of the 67 | reporter of any incident. 68 | 69 | ## Enforcement Guidelines 70 | 71 | Community leaders will follow these Community Impact Guidelines in determining 72 | the consequences for any action they deem in violation of this Code of Conduct: 73 | 74 | ### 1. Correction 75 | 76 | **Community Impact**: Use of inappropriate language or other behavior deemed 77 | unprofessional or unwelcome in the community. 78 | 79 | **Consequence**: A private, written warning from community leaders, providing 80 | clarity around the nature of the violation and an explanation of why the 81 | behavior was inappropriate. A public apology may be requested. 82 | 83 | ### 2. Warning 84 | 85 | **Community Impact**: A violation through a single incident or series 86 | of actions. 87 | 88 | **Consequence**: A warning with consequences for continued behavior. No 89 | interaction with the people involved, including unsolicited interaction with 90 | those enforcing the Code of Conduct, for a specified period of time. This 91 | includes avoiding interactions in community spaces as well as external channels 92 | like social media. Violating these terms may lead to a temporary or 93 | permanent ban. 94 | 95 | ### 3. Temporary Ban 96 | 97 | **Community Impact**: A serious violation of community standards, including 98 | sustained inappropriate behavior. 99 | 100 | **Consequence**: A temporary ban from any sort of interaction or public 101 | communication with the community for a specified period of time. No public or 102 | private interaction with the people involved, including unsolicited interaction 103 | with those enforcing the Code of Conduct, is allowed during this period. 104 | Violating these terms may lead to a permanent ban. 105 | 106 | ### 4. Permanent Ban 107 | 108 | **Community Impact**: Demonstrating a pattern of violation of community 109 | standards, including sustained inappropriate behavior, harassment of an 110 | individual, or aggression toward or disparagement of classes of individuals. 111 | 112 | **Consequence**: A permanent ban from any sort of public interaction within 113 | the community. 114 | 115 | ## Attribution 116 | 117 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], 118 | version 2.0, available at 119 | https://www.contributor-covenant.org/version/2/0/code_of_conduct.html. 120 | 121 | Community Impact Guidelines were inspired by [Mozilla's code of conduct 122 | enforcement ladder](https://github.com/mozilla/diversity). 123 | 124 | [homepage]: https://www.contributor-covenant.org 125 | 126 | For answers to common questions about this code of conduct, see the FAQ at 127 | https://www.contributor-covenant.org/faq. Translations are available at 128 | https://www.contributor-covenant.org/translations. 129 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 GitHub Advanced Security 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 |
3 | 4 |

gh-codeql-scan

5 | 6 | [![GitHub](https://img.shields.io/badge/github-%23121011.svg?style=for-the-badge&logo=github&logoColor=white)][github] 7 | [![GitHub Issues](https://img.shields.io/github/issues/advanced-security/gh-codeql-scan?style=for-the-badge)][github-issues] 8 | [![GitHub Stars](https://img.shields.io/github/stars/advanced-security/gh-codeql-scan?style=for-the-badge)][github] 9 | [![License](https://img.shields.io/github/license/advanced-security/gh-codeql-scan?style=for-the-badge)][license] 10 | 11 |
12 | 13 | 14 | [GitHub CLI CodeQL Scan Extension][github] to help abstract [CodeQL][codeql] away from users. 15 | 16 |
17 | Motivation 18 | 19 | This project was created to make the lives of users that use CodeQL simpiler. 20 | CodeQL outside of GitHub Actions can be complicated but this projects aim is to make it as simple as possible. 21 | 22 |
23 | 24 | ## Requirements 25 | 26 | - [GitHub CLI](https://cli.github.com/) 27 | - [CodeQL GH Extension][gh-codeql] (optional) 28 | 29 | ## Install and Setup 30 | 31 | This installs CodeQL and this scan tool: 32 | 33 | ```bash 34 | gh extensions install github/gh-codeql 35 | gh extensions install advanced-security/gh-codeql-scan 36 | 37 | gh codeql-scan --help 38 | ``` 39 | 40 |
41 | CLI Help 42 | 43 | 
 44 | GitHub CodeQL Scan tool
 45 | 
 46 | gh codeql-scan {MODE} {ARGS}
 47 | 
 48 | # Modes
 49 | 
 50 | gh codeql-scan              # default: "scan"
 51 | gh codeql-scan init         # initialise the scan 
 52 | gh codeql-scan analyze      # run the analysis
 53 | gh codeql-scan upload       # upload present SARIF files
 54 | gh codeql-scan scan         # full end-to-end scan 
 55 | 
 56 | # Arguments
 57 | 
 58 | > All arguments can be set with enviroment variables
 59 | 
 60 | -h|--help               # Print help
 61 | --debug                 # Enable debugging
 62 |     
 63 | -r=*|--repo=*           # GitHub Respository (OWNER/NAME)
 64 | -i=*|--instance=*       # GitHub Instance (github.com or Enterprise Server)
 65 | 
 66 | -l=*|--language=*       # Set language to scan
 67 | --auto-detect           # Auto-detect languages
 68 | 
 69 | -s=*|--suite=*          # Query Suite to use
 70 | -d=*|--databases=*      # Location of the databases to store
 71 | -b=*|--binary=*         # Path to the CodeQL Binary
 72 | -w=*|--workspace=*      # Workspace for the souce code
 73 | 
 74 | -c=*|--command=*        # Set the build comment (compiled languages)
 75 | -m=*|--mode=*           # Build mode (autobuild | none)
 76 | --buildless             # Enable buildless / build mode none
 77 | 
 78 | --view-in-vscode        # Auto-open the results in VSCode
 79 | 
 80 | --disable-tracing       # Disable Build Tracing
 81 | --disable-trap-caching  # Disable Trap file caching
 82 | --disable-upload        # Disable Uploading SARIF to GitHub
 83 | --disable-banner        # Disable printing banner
 84 |
85 | 86 |
87 | 88 | ### Alias / Stub 89 | 90 | A couple of tips and tricks: 91 | 92 | ```bash 93 | # Create an alias to make things even easier 94 | alias codeql-scan="gh codeql-scan" 95 | ``` 96 | 97 | ## Usage 98 | 99 | The main use of the script is to automatically run CodeQL in a number of modes. 100 | 101 | ```bash 102 | # End-to-end analysis and upload results 103 | gh codeql-scan 104 | ``` 105 | 106 | #### Initialise with language 107 | 108 | Automatically detect languages or manually set the language to create an initial CodeQL database. 109 | 110 | ```bash 111 | gh codeql-scan init --auto-detect 112 | # or manually set language 113 | gh codeql-scan init -l=java 114 | ``` 115 | 116 | #### Scan without build 117 | 118 | This will scan your code in build mode `none`. 119 | 120 | ```bash 121 | gh codeql-scan -m="none" 122 | # or simply 123 | gh codeql-scan --buildless 124 | ``` 125 | 126 | #### Scan with Build Command 127 | 128 | Pass in the build command for a compiled language and it will be run along with CodeQL. 129 | 130 | ```bash 131 | gh codeql-scan -c "mvn build ..." 132 | ``` 133 | 134 | #### Indirect build tracing 135 | 136 | For Compiled languages, complicated build process using indirect build tracing 137 | 138 | ```bash 139 | gh codeql-scan init 140 | echo "password=$password" > settings.xml 141 | mvn build --random-custom=flags 142 | gh codeql-scan analyze 143 | ``` 144 | 145 | #### Running analysis 146 | 147 | Run query-suites on an existing database (auto-detects databases) 148 | 149 | ```bash 150 | gh codeql-scan analyze 151 | ``` 152 | 153 | #### Uploading results to GitHub 154 | 155 | The `upload` mode will upload all SARIF files for you to a repository 156 | 157 | ```bash 158 | gh codeql-scan upload 159 | ``` 160 | 161 | ## Maintainers 162 | 163 | - @GeekMasher 164 | 165 | ## Support / Maintainance 166 | 167 | Support is via [GitHub Issues][github-issues] 168 | 169 | ## License 170 | 171 | This project is licensed under the terms of the MIT open source license. 172 | Please refer to [MIT][license] for the full terms. 173 | 174 | 175 | 176 | [license]: ./LICENSE 177 | [github]: https://github.com/advanced-security/gh-codeql-scan 178 | [github-issues]: https://github.com/advanced-security/gh-codeql-scan/issues 179 | [codeql]: https://codeql.github.com/ 180 | [gh-codeql]: https://github.com/github/gh-codeql 181 | 182 | -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | # Security Policy 2 | 3 | GitHub takes the security of our software products and services seriously, including all of the open source code repositories managed through our GitHub organizations, such as [GitHub](https://github.com/GitHub). 4 | 5 | Even though [open source repositories are outside of the scope of our bug bounty program](https://bounty.github.com/index.html#scope) and therefore not eligible for bounty rewards, we will ensure that your finding gets passed along to the appropriate maintainers for remediation. 6 | 7 | 8 | ## Supported Versions 9 | 10 | Use this section to tell people about which versions of your project are 11 | currently being supported with security updates. 12 | 13 | | Version | Supported | 14 | | ------- | ------------------ | 15 | | 0.1.0 | :white_check_mark: | 16 | 17 | 18 | ## Reporting Security Issues 19 | 20 | If you believe you have found a security vulnerability in any GitHub-owned repository, please report it to us through coordinated disclosure. 21 | 22 | **Please do not report security vulnerabilities through public GitHub issues, discussions, or pull requests.** 23 | 24 | Instead, please send an email to opensource-security[@]github.com. 25 | 26 | Please include as much of the information listed below as you can to help us better understand and resolve the issue: 27 | 28 | * The type of issue (e.g., buffer overflow, SQL injection, or cross-site scripting) 29 | * Full paths of source file(s) related to the manifestation of the issue 30 | * The location of the affected source code (tag/branch/commit or direct URL) 31 | * Any special configuration required to reproduce the issue 32 | * Step-by-step instructions to reproduce the issue 33 | * Proof-of-concept or exploit code (if possible) 34 | * Impact of the issue, including how an attacker might exploit the issue 35 | 36 | This information will help us triage your report more quickly. 37 | 38 | -------------------------------------------------------------------------------- /SUPPORT.md: -------------------------------------------------------------------------------- 1 | # Support 2 | 3 | ## How to file issues and get help 4 | 5 | This project uses GitHub issues to track bugs and feature requests. Please search the existing issues before filing new issues to avoid duplicates. For new issues, file your bug or feature request as a new issue. 6 | -------------------------------------------------------------------------------- /bin/codeql-analyze: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | source $EXTENSION_LOCATION/bin/codeql-utils 4 | 5 | CODEQL_DATABASE_PATHS=$(cat $CODEQL_DATABASE_PATHS_FILE) 6 | 7 | export CODEQL_SUITE="${CODEQL_SUITE:=}" 8 | CODEQL_ADD_HELP="--sarif-add-query-help" 9 | CODEQL_ADD_SNIPPETS="--sarif-add-snippets" 10 | 11 | 12 | for i in "$@"; do 13 | case $i in 14 | --ignore-sarif-query-help) 15 | CODEQL_ADD_HELP="" 16 | shift 17 | ;; 18 | --ignore-snippets) 19 | CODEQL_ADD_SNIPPETS="" 20 | shift 21 | ;; 22 | -s=*|--suite=*) 23 | export CODEQL_SUITE="${i#*=}" 24 | shift 25 | ;; 26 | -p=*|--path=*) 27 | CODEQL_DATABASE_PATHS="${i#*=}" 28 | shift 29 | ;; 30 | *) 31 | ;; 32 | esac 33 | done 34 | 35 | if [[ -z $CODEQL_DATABASE_PATHS ]]; then 36 | error "No CodeQL Databases found" 37 | exit 1 38 | fi 39 | 40 | info "Analyzing CodeQL Databases :: $CODEQL_DATABASE_PATHS" 41 | 42 | 43 | for CODEQL_DATABASE in $CODEQL_DATABASE_PATHS ; do 44 | CODEQL_DATABASE_NAME="$(basename $CODEQL_DATABASE)" 45 | CODEQL_LANGUAGE=$(cat $CODEQL_DATABASE/codeql-database.yml | grep primaryLanguage | cut -d ":" -f 2 | tr -d ' ' | tr -d '"') 46 | CODEQL_SARIF="${CODEQL_RESULTS}/${CODEQL_DATABASE_NAME}.sarif" 47 | 48 | info "CodeQL Database Path :: $CODEQL_DATABASE" 49 | info "CodeQL Database Name :: $CODEQL_DATABASE_NAME" 50 | info "CodeQL Language :: $CODEQL_LANGUAGE" 51 | 52 | # For tracing 53 | debug "Finalizing CodeQL Database" 54 | $CODEQL_BINARY database finalize --finalize-dataset $CODEQL_DATABASE 2>/dev/null | true 55 | 56 | # The --sarif-category must be set in case of multiple databases 57 | $CODEQL_BINARY database analyze \ 58 | --format="sarif-latest" \ 59 | --sarif-category="codeql-scan:${CODEQL_LANGUAGE}" \ 60 | --output=$CODEQL_SARIF \ 61 | -j=0 -M="$(memory)" \ 62 | $CODEQL_ADD_HELP $CODEQL_ADD_SNIPPETS \ 63 | ${CODEQL_DATABASE} ${CODEQL_SUITE} 64 | 65 | info "CodeQL SARIF Output :: $CODEQL_SARIF" 66 | done 67 | -------------------------------------------------------------------------------- /bin/codeql-init: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | source $EXTENSION_LOCATION/bin/codeql-utils 4 | 5 | while [[ "$#" -gt 0 ]]; do 6 | case $1 in 7 | --auto-detect) 8 | GITHUB_AUTO_DETECT="1" 9 | ;; 10 | --disable-tracing) 11 | export CODEQL_TRACING=0 12 | ;; 13 | -l=*|--language=*) 14 | export CODEQL_LANGUAGES="${1#*=}" 15 | ;; 16 | -c=*|--command=*) 17 | export CODEQL_BUILD_COMMAND="${1#*=}" 18 | export CODEQL_TRACING="1" # enable tracing 19 | ;; 20 | -m=*|--mode=*) 21 | export CODEQL_BUILD_MODE="${1#*=}" 22 | ;; 23 | --disable-trap-caching) 24 | export CODEQL_TRAP_CACHING=0 25 | ;; 26 | *) 27 | ;; 28 | esac 29 | shift 30 | done 31 | 32 | 33 | if [ ${GITHUB_UPLOAD} == 0 ] && [ -z ${GITHUB_REPOSITORY+x} ]; then 34 | # If the repo isn't set 35 | export GITHUB_REPOSITORY="$(basename $PWD)" 36 | 37 | elif [ ${GITHUB_UPLOAD} == 1 ] && [ -z ${GITHUB_REPOSITORY+x} ]; then 38 | error "GITHUB_REPOSITORY must be set" 39 | exit 1 40 | fi 41 | 42 | 43 | if [[ ! -z ${CODEQL_LANGUAGES+x} && "$CODEQL_LANGUAGES" != "" ]] ; then 44 | info "Manually set languages :: $CODEQL_LANGUAGES" 45 | 46 | # auto-detect languages 47 | elif [[ "$GITHUB_AUTO_DETECT" = "1" ]]; then 48 | if ! command -v gh &> /dev/null ; then 49 | error "GitHub CLI is not installed and can't auto-detect languages" 50 | exit 1 51 | fi 52 | 53 | info "Detecting Repository Language(s)" 54 | # Get the data from GitHub 55 | # https://github.com/advanced-security/advanced-security-material/blob/main/code-scanning-workflows/reusable_code_scanning-00.yml#L19-L27 56 | # JSON array 57 | language_data=$(gh api repos/${GITHUB_REPOSITORY}/languages -q '[{"C":"cpp", "C++":"cpp", "C#":"csharp", "Go":"go", "Java":"java", "JavaScript":"javascript", "TypeScript":"javascript", "Python":"python", "Ruby":"ruby"}[keys[]]] | unique -[null]') 58 | # Remove unneeded chars 59 | CODEQL_LANGUAGES=$(echo $language_data | tr -d '"[]') 60 | fi 61 | 62 | 63 | if [ -z $CODEQL_LANGUAGES ] ; then 64 | error "No CodeQL languages were set" 65 | exit 1 66 | fi 67 | 68 | MEMORY=$(memory) 69 | info "Memory :: $MEMORY (MB)" 70 | 71 | # comma seperated list (`java,javascript`) 72 | for CODEQL_LANGUAGE in ${CODEQL_LANGUAGES//,/ } ; do 73 | debug "CodeQL Language :: $CODEQL_LANGUAGE" 74 | 75 | CODEQL_GITHUB_REPOSITORY=$(echo $GITHUB_REPOSITORY | tr '/' '_') 76 | CODEQL_DATABASE="$CODEQL_DATABASES/${CODEQL_LANGUAGE}-${CODEQL_GITHUB_REPOSITORY}" 77 | 78 | info "CodeQL Database Name :: $CODEQL_DATABASE" 79 | 80 | if [ -d $CODEQL_DATABASES ] ; then 81 | info "Deleting old database :: $CODEQL_DATABASE" 82 | rm -rf $CODEQL_DATABASE 83 | fi 84 | 85 | debug "CodeQL Binary :: $CODEQL_BINARY" 86 | debug "Tracing :: $CODEQL_TRACING" 87 | 88 | CODEQL_CREATE="$CODEQL_BINARY" 89 | 90 | if [ "$CODEQL_BUILD_MODE" = "none" ] && [[ "$CODEQL_LANGUAGES" =~ ^(csharp|java)$ ]]; then 91 | info "CodeQL build mode is set to none" 92 | 93 | CODEQL_CREATE="$CODEQL_CREATE database init --language=$CODEQL_LANGUAGE --build-mode=none" 94 | 95 | elif [ "$CODEQL_TRACING" = "1" ] && [[ "$CODEQL_LANGUAGE" =~ ^(cpp|csharp|java|go)$ ]]; then 96 | debug "Swap to a init command and add --begin-tracing" 97 | 98 | # check if the OS is Mac 99 | if [[ "$OSTYPE" == "darwin"* ]]; then 100 | warning "Tracing on MacOS can have issues with the CodeQL CLI" 101 | warning "See https://docs.github.com/en/enterprise-cloud@latest/code-security/codeql-cli/getting-started-with-the-codeql-cli/preparing-your-code-for-codeql-analysis#using-indirect-build-tracing" 102 | fi 103 | 104 | # https://codeql.github.com/docs/codeql-cli/creating-codeql-databases/#using-indirect-build-tracing 105 | CODEQL_CREATE="$CODEQL_CREATE database init --begin-tracing --language=$CODEQL_LANGUAGE" 106 | 107 | elif [[ ! -z $CODEQL_BUILD_COMMAND ]]; then 108 | debug "Enabling CodeQL manual build" 109 | CODEQL_CREATE="$CODEQL_CREATE --command \"${CODEQL_BUILD_COMMAND}\"" 110 | else 111 | debug "Append only Database" 112 | debug "CodeQL auto-build is enabled" 113 | CODEQL_CREATE="$CODEQL_CREATE database create -j=0 -M=$MEMORY --language=$CODEQL_LANGUAGE" 114 | fi 115 | 116 | if [[ ! -z "${WORKSPACE}" ]]; then 117 | debug "Setting source-root to workspace" 118 | CODEQL_CREATE="$CODEQL_CREATE --source-root=$WORKSPACE" 119 | fi 120 | 121 | # TRAP File caching (C/C++ only for now) 122 | if [ "$CODEQL_TRAP_CACHING" = "1" ] && [[ "$CODEQL_LANGUAGE" = "cpp" ]]; then 123 | info "CodeQL TRAP Caching is enabled" 124 | info "CodeQL TRAP Caching Dir :: $CODEQL_TRAP_CACHING_DIR" 125 | 126 | CODEQL_TRAP_CACHING_PROJECT="$CODEQL_TRAP_CACHING_DIR/${CODEQL_LANGUAGE}-${CODEQL_GITHUB_REPOSITORY}" 127 | # Make sure the directory exists 128 | mkdir -p $CODEQL_TRAP_CACHING_PROJECT 129 | 130 | # Set the environment variable 131 | export CODEQL_EXTRACTOR_CPP_TRAP_CACHING=true 132 | 133 | CODEQL_CREATE="$CODEQL_CREATE -O=cpp.trap.cache.dir=$CODEQL_TRAP_CACHING_PROJECT -O=cpp.trap.cache.bound=1024 -O=cpp.trap.cache.write=true" 134 | fi 135 | 136 | # Add the database to the command 137 | CODEQL_CREATE="$CODEQL_CREATE $CODEQL_DATABASE" 138 | 139 | info "Runnning CodeQL database init/create command..." 140 | debug "CodeQL Create Command :: $CODEQL_CREATE" 141 | 142 | eval $CODEQL_CREATE 143 | 144 | if [ "$CODEQL_TRACING" = "1" ] && [[ "$CODEQL_LANGUAGE" =~ ^(cpp|csharp|java|go)$ ]]; then 145 | info "Starting CodeQL Tracing..." 146 | # Note: This only supports one language at a time to be intercepted using tracing 147 | source $CODEQL_DATABASE/temp/tracingEnvironment/start-tracing.sh 148 | 149 | debug "CodeQL Tracer Languages :: $CODEQL_TRACER_LANGUAGES" 150 | fi 151 | 152 | if [ "$CODEQL_BUILD_MODE" = "none" ] && [[ "$CODEQL_LANGUAGE" =~ ^(csharp|java)$ ]]; then 153 | info "Running buildless mode..." 154 | 155 | $CODEQL_BINARY database trace-command --use-build-mode $CODEQL_DATABASE 156 | 157 | debug "Finalizing CodeQL Database" 158 | $CODEQL_BINARY database finalize $CODEQL_DATABASE | true 159 | 160 | elif [ ! -z ${CODEQL_BUILD_COMMAND+x} ] && [[ "$CODEQL_LANGUAGE" =~ ^(cpp|csharp|java|go)$ ]] ; then 161 | info "Running build command :: '$CODEQL_BUILD_COMMAND'" 162 | 163 | eval ${CODEQL_BUILD_COMMAND} 164 | 165 | debug "Finalizing CodeQL Database" 166 | $CODEQL_BINARY database finalize $CODEQL_DATABASE | true 167 | 168 | debug "Finished running build command..." 169 | fi 170 | 171 | # Add CodeQL DB to paths 172 | CODEQL_DATABASE_PATHS="$CODEQL_DATABASE_PATHS $CODEQL_DATABASE" 173 | done 174 | 175 | debug "Storing CodeQL Database Paths" 176 | echo $CODEQL_DATABASE_PATHS > $CODEQL_DATABASE_PATHS_FILE 177 | -------------------------------------------------------------------------------- /bin/codeql-upload: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | source $EXTENSION_LOCATION/bin/codeql-utils 5 | 6 | export CODEQL_DATABASE_PATHS=$(cat $CODEQL_DATABASE_PATHS_FILE) 7 | 8 | 9 | if [[ $CODEQL_DATABASE_PATHS != "" ]]; then 10 | debug "Database paths found :: $CODEQL_DATABASE_PATHS" 11 | FILES=$(get-sarif-files $CODEQL_DATABASE_PATHS_FILE) 12 | else 13 | info "Uploading all SARIF files found in :: $CODEQL_RESULTS" 14 | FILES=$(find $CODEQL_RESULTS -type f -name "*.sarif") 15 | fi 16 | 17 | info "GitHub Instance :: $GITHUB_INSTANCE" 18 | 19 | 20 | # Upload results for each SARIF results file found 21 | for SARIF_FILE in $FILES; do 22 | if [ ! -f $SARIF_FILE ]; then 23 | error "SARIF file not found :: $SARIF_FILE" 24 | exit 1 25 | fi 26 | 27 | info "SARIF File uploading :: $SARIF_FILE" 28 | 29 | debug "GitHub Instance :: $GITHUB_INSTANCE" 30 | debug "GitHub Repository :: $GITHUB_REPOSITORY" 31 | 32 | gh codeql github upload-results \ 33 | --sarif=$SARIF_FILE \ 34 | --github-url=$GITHUB_INSTANCE \ 35 | --repository=$GITHUB_REPOSITORY \ 36 | --ref=$GIT_REF \ 37 | --commit=$GIT_HASH 38 | 39 | debug "Finished uploading SARIF file :: $SARIF_FILE" 40 | 41 | done 42 | -------------------------------------------------------------------------------- /bin/codeql-utils: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | export QL_LANGUAGES="cpp,csharp,go,java,javascript,python,ruby" 6 | 7 | # https://unix.stackexchange.com/questions/9957/how-to-check-if-bash-can-print-colors 8 | if test -t 1; then 9 | ncolors=$(tput colors) 10 | if test -n "$ncolors" && test $ncolors -ge 8; then 11 | bold="$(tput bold)" 12 | underline="$(tput smul)" 13 | standout="$(tput smso)" 14 | normal="$(tput sgr0)" 15 | black="$(tput setaf 0)" 16 | red="$(tput setaf 1)" 17 | green="$(tput setaf 2)" 18 | yellow="$(tput setaf 3)" 19 | blue="$(tput setaf 4)" 20 | magenta="$(tput setaf 5)" 21 | cyan="$(tput setaf 6)" 22 | white="$(tput setaf 7)" 23 | fi 24 | else 25 | bold="" 26 | underline="" 27 | standout="" 28 | normal="" 29 | black="" 30 | red="" 31 | green="" 32 | yellow="" 33 | blue="" 34 | magenta="" 35 | cyan="" 36 | white="" 37 | fi 38 | 39 | display-banner() { 40 | if [ ! -z ${CODEQL_SCAN_BANNER+x} ] && [ "$CODEQL_SCAN_BANNER" = "0" ]; then 41 | return 42 | fi 43 | echo "${green} _____ _ _____ _ _____" 44 | echo " / __ \ | | | _ | | / ___|" 45 | echo " | / \/ ___ __| | ___ | | | | | \ \`--. ___ __ _ _ __" 46 | echo " | | / _ \ / _\` \|/ _ | | | | | \`--. \/ __/ _\` | '_ \\" 47 | echo " | \__/\ (_) | (_| | __/\ \/' / |____ /\__/ / (_| (_| | | | |" 48 | echo " \____/\___/ \__,_|\___| \_/\_\_____/ \____/ \___\__,_|_| |_|${normal}" 49 | echo " v${underline}$CODEQL_SCAN_VERSION${normal} - by ${bold}${red}GitHub Field Security Specialist team${normal}" 50 | echo "" 51 | } 52 | 53 | codeql-scan-help() { 54 | cat < All arguments can be set with enviroment variables 70 | 71 | -h|--help # Print help 72 | --debug # Enable debugging 73 | 74 | -r=*|--repo=* # GitHub Respository (OWNER/NAME) 75 | -i=*|--instance=* # GitHub Instance (github.com or Enterprise Server) 76 | 77 | -l=*|--language=* # Set language to scan 78 | --auto-detect # Auto-detect languages 79 | 80 | -s=*|--suite=* # Query Suite to use 81 | -d=*|--databases=* # Location of the databases to store 82 | -b=*|--binary=* # Path to the CodeQL Binary 83 | -w=*|--workspace=* # Workspace for the souce code 84 | 85 | -c=*|--command=* # Set the build comment (compiled languages) 86 | -m=*|--mode=* # Build mode (autobuild | none) 87 | --buildless # Enable buildless / build mode none 88 | 89 | --view-in-vscode # Auto-open the results in VSCode 90 | 91 | --disable-tracing # Disable Build Tracing 92 | --disable-trap-caching # Disable Trap file caching 93 | --disable-upload # Disable Uploading SARIF to GitHub 94 | --disable-banner # Disable printing banner 95 | 96 | EOF 97 | } 98 | 99 | # Printing methods for stdout 100 | 101 | info() { 102 | echo "${green}[+]${normal} $@" 103 | } 104 | debug() { 105 | if [ ! -z ${DEBUG+x} ]; then 106 | echo "${blue}[#] DEBUG: $@ ${normal}" 107 | fi 108 | } 109 | warning() { 110 | echo "${yellow}[!] WARNING: $@ ${normal}" 111 | } 112 | error() { 113 | echo "${red}[*] ERROR: $@${normal}" 114 | } 115 | 116 | 117 | # CodeQL methods 118 | 119 | codeql-version() { 120 | # return the version of CodeQL installed 121 | CODEQL_VERSION=$($CODEQL_BINARY version --format=terse 2>/dev/null) 122 | echo "$CODEQL_VERSION" 123 | } 124 | 125 | codeql-install() { 126 | # https://github.com/GeekMasher/.dotfiles/blob/d08a1525c624e88b4d686cf70da349616d2b8aa4/codeql/.local/codeql-update 127 | info "Installing CodeQL via gh-cli..." 128 | 129 | gh extensions install github/gh-codeql 130 | gh codeql set-version latest 131 | 132 | # install new packs for each language 133 | for lang in $(echo $QL_LANGUAGES | sed "s/,/ /g"); do 134 | gh codeql pack download "codeql/$lang-queries" 135 | done 136 | 137 | info "CodeQL installed!" 138 | } 139 | 140 | # Get 80% the memory of the system in MB 141 | memory() { 142 | MEMORY=0 143 | # Get the memory of the system 144 | if [[ "$OSTYPE" == "darwin"* ]]; then 145 | # MacOS 146 | MEMORY=$(sysctl -n hw.memsize) 147 | else 148 | # Linux 149 | MEMORY=$(free -m | awk 'NR==2{print $2}') 150 | fi 151 | 152 | # 80% of the memory 153 | MEMORY=$(echo "0.8 * $MEMORY" | bc | cut -d'.' -f1) 154 | echo $MEMORY 155 | } 156 | 157 | # Get SARIF files based on database names 158 | get-sarif-files() { 159 | # return a list of SARIF files based on database names 160 | DATABASE_PATHS=$(cat $1) 161 | for CODEQL_DATABASE_PATH in $DATABASE_PATHS; do 162 | SARIF_FILE="$CODEQL_RESULTS/$(basename $CODEQL_DATABASE_PATH).sarif" 163 | debug "Possible SARIF file :: $SARIF_FILE" 164 | if [ -f $SARIF_FILE ]; then 165 | echo "$SARIF_FILE" 166 | fi 167 | done 168 | } 169 | -------------------------------------------------------------------------------- /gh-codeql-scan: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -eu 3 | 4 | export EXTENSION_LOCATION="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)" 5 | 6 | source $EXTENSION_LOCATION/bin/codeql-utils 7 | 8 | # >> CodeQL Scan 9 | export CODEQL_SCAN_VERSION="0.8.0" 10 | # enabled by default 11 | export CODEQL_SCAN_BANNER=1 12 | 13 | # >> CodeQL 14 | export CODEQL_DATABASES="${CODEQL_DATABASES:=$HOME/.codeql/databases}" 15 | # Override the database if it exists 16 | export CODEQL_DATABASES_OVERRIDE=1 17 | export CODEQL_RESULTS="${CODEQL_RESULTS:=$HOME/.codeql/results}" 18 | export CODEQL_BINARY="${CODEQL_BINARY:=gh codeql}" 19 | # `code-scanning` (default), `security-extended`, or `security-and-quality` 20 | export CODEQL_SUITE="${CODEQL_SUITE:=}" 21 | # Tracing is disabled by default 22 | export CODEQL_TRACING=0 23 | export CODEQL_LANGUAGES= 24 | # this variable is used to know while databases need to be analyzed 25 | export CODEQL_DATABASE_PATHS_FILE="${CODEQL_DATABASE_PATHS_FILE:=/tmp/codeql-database-paths.txt}" 26 | export CODEQL_BUILD_COMMAND="${CODEQL_BUILD_COMMAND:=}" 27 | export CODEQL_BUILD_MODE="${CODEQL_BUILD_MODE:=}" 28 | # TRAP Caching 29 | export CODEQL_TRAP_CACHING=1 30 | export CODEQL_TRAP_CACHING_DIR="${CODEQL_TRAP_CACHING_DIR:=$HOME/.codeql/trap-caching}" 31 | 32 | export WORKSPACE="${WORKSPACE:=$PWD}" 33 | 34 | # >> GitHub 35 | export GITHUB_INSTANCE=${GITHUB_INSTANCE:="https://github.com"} 36 | # upload by default 37 | export GITHUB_UPLOAD=${GITHUB_UPLOAD:=1} 38 | # auto-detect languages 39 | export GITHUB_AUTO_DETECT=${GITHUB_AUTO_DETECT:=1} 40 | # vscode 41 | export CODEQL_VIEW_IN_VSCODE=${CODEQL_VIEW_IN_VSCODE:=0} 42 | 43 | # Git Based metadata 44 | export GIT_HASH=$(git rev-parse HEAD) 45 | # Assumes that this is scanning a branch, not a Pull Request 46 | # https://docs.github.com/en/code-security/secure-coding/configuring-codeql-code-scanning-in-your-ci-system#scanning-pull-requests 47 | export GIT_REF="refs/heads/$(git branch --show-current)" 48 | 49 | 50 | # default: scan 51 | MODE="scan" 52 | if [ -z ${1+x} ]; then 53 | debug "Using default mode" 54 | elif [[ "${1}" = "--help" ]]; then 55 | codeql-scan-help 56 | exit 0 57 | elif [[ "${1}" =~ ^(init|analyze|scan|upload) ]]; then 58 | MODE="$1" 59 | debug "Using mode :: $MODE" 60 | shift 61 | fi 62 | 63 | while [[ "$#" -gt 0 ]]; do 64 | case $1 in 65 | -h|--help) 66 | codeql-scan-help 67 | exit 0 68 | ;; 69 | --debug) 70 | export DEBUG=1 71 | ;; 72 | -r=*|--repo=*) 73 | export GITHUB_REPOSITORY="${1#*=}" 74 | ;; 75 | --auto-detect) 76 | export GITHUB_AUTO_DETECT=1 77 | ;; 78 | --disable-tracing) 79 | export CODEQL_TRACING=0 80 | ;; 81 | --disable-upload) 82 | export GITHUB_UPLOAD=0 83 | ;; 84 | --disable-banner) 85 | export CODEQL_SCAN_BANNER=0 86 | ;; 87 | -l=*|--language=*) 88 | export CODEQL_LANGUAGES="${1#*=}" 89 | ;; 90 | -c=*|--command=*) 91 | export CODEQL_BUILD_COMMAND="${1#*=}" 92 | export CODEQL_TRACING="1" # enable tracing 93 | ;; 94 | -m=*|--mode=*) 95 | export CODEQL_BUILD_MODE="${1#*=}" 96 | ;; 97 | -d=*|--databases=*) 98 | export CODEQL_DATABASES="${1#*=}" 99 | ;; 100 | -s=*|--suite=*) 101 | export CODEQL_SUITE="${1#*=}" 102 | ;; 103 | -b=*|--binary=*|--codeql-bin=*) 104 | export CODEQL_BINARY="${1#*=}" 105 | ;; 106 | -i=*|--instance=*) 107 | export GITHUB_INSTANCE="${1#*=}" 108 | ;; 109 | -w=*|--workspace=*) 110 | export WORKSPACE="${1#*=}" 111 | ;; 112 | --view-in-vscode) 113 | export CODEQL_VIEW_IN_VSCODE=1 114 | ;; 115 | --disable-trap-caching) 116 | export CODEQL_TRAP_CACHING=0 117 | ;; 118 | --buildless) 119 | export CODEQL_BUILD_MODE="none" 120 | ;; 121 | *) 122 | echo "Unknown option $i" 123 | exit 1 124 | ;; 125 | esac 126 | shift 127 | done 128 | 129 | 130 | display-banner 131 | 132 | debug "Checking CodeQL installation :: $CODEQL_BINARY" 133 | CODEQL_VERSION=$(codeql-version) 134 | debug "CodeQL Original version :: $CODEQL_VERSION" 135 | 136 | if [ "$CODEQL_VERSION" = "" ]; then 137 | warning "CodeQL is not installed, now installing automatically..." 138 | 139 | codeql-install 140 | # re-export the gh cli codeql extension 141 | export CODEQL_BINARY="gh codeql" 142 | export CODEQL_VERSION=$(codeql-version) 143 | fi 144 | 145 | 146 | info "Using CodeQL version :: $CODEQL_VERSION" 147 | 148 | # Output dirs 149 | info "Storing CodeQL databases :: $CODEQL_DATABASES" 150 | if [ ! -d $CODEQL_DATABASES ]; then 151 | mkdir -p $CODEQL_DATABASES 152 | fi 153 | info "Storing CodeQL results :: $CODEQL_RESULTS" 154 | if [ ! -d $CODEQL_RESULTS ]; then 155 | mkdir -p $CODEQL_RESULTS 156 | fi 157 | 158 | 159 | # Auto detect the repository owner and name 160 | if [ -z ${GITHUB_REPOSITORY+x} ]; then 161 | REMOTE=$(git config --get remote.origin.url) 162 | if [[ $REMOTE == git@* ]]; then 163 | debug "Using SSH remote :: $REMOTE" 164 | # only works for github.com repositories 165 | export GITHUB_REPOSITORY=$(echo $REMOTE | sed -e 's/.*github.com[:\/]\(.*\)\.git/\1/') 166 | elif [[ $REMOTE == https://* ]]; then 167 | debug "Using HTTPS remote :: $REMOTE" 168 | export GITHUB_REPOSITORY=${REMOTE#*://*/} && export GITHUB_REPOSITORY=${GITHUB_REPOSITORY%.git} 169 | else 170 | error "Unable to auto-detect the repository owner and name" 171 | exit 1 172 | fi 173 | info "Using remote repository :: $GITHUB_REPOSITORY" 174 | fi 175 | 176 | # Running different modes 177 | if [ $MODE = "init" ]; then 178 | info "Running Init mode..." 179 | 180 | $EXTENSION_LOCATION/bin/codeql-init $@ 181 | 182 | elif [ $MODE = "analyze" ]; then 183 | info "Running Analyze mode..." 184 | 185 | $EXTENSION_LOCATION/bin/codeql-analyze $@ 186 | 187 | elif [ $MODE = "upload" ]; then 188 | info "Uploading results to GitHub..." 189 | 190 | $EXTENSION_LOCATION/bin/codeql-upload $@ 191 | 192 | info "Find Code Scanning results at: $GITHUB_INSTANCE/$GITHUB_REPOSITORY/security/code-scanning" 193 | 194 | elif [ $MODE = "scan" ]; then 195 | info "Running Scan mode..." 196 | 197 | $EXTENSION_LOCATION/bin/codeql-init $@ 198 | 199 | $EXTENSION_LOCATION/bin/codeql-analyze $@ 200 | 201 | if [ "$GITHUB_UPLOAD" = "1" ]; then 202 | info "Uploading results to GitHub..." 203 | $EXTENSION_LOCATION/bin/codeql-upload $@ 204 | 205 | info "Find Code Scanning results at: $GITHUB_INSTANCE/$GITHUB_REPOSITORY/security/code-scanning" 206 | else 207 | debug "Results are not uploaded to GitHub" 208 | fi 209 | fi 210 | 211 | 212 | if [ "$CODEQL_VIEW_IN_VSCODE" = "1" ]; then 213 | info "Opening results in VSCode..." 214 | code -r $(get-sarif-files $CODEQL_DATABASE_PATHS_FILE) 215 | fi 216 | 217 | info "Completed CodeQL Scan!" 218 | --------------------------------------------------------------------------------