├── .github ├── ISSUE_TEMPLATE │ └── config.yml └── workflows │ └── label-issue.yml ├── README.md ├── LICENSE.md └── CHANGELOG.md /.github/ISSUE_TEMPLATE/config.yml: -------------------------------------------------------------------------------- 1 | blank_issues_enabled: false 2 | contact_links: 3 | - name: Report an issue or ask a question about CodeQL 4 | url: https://github.com/github/codeql/issues/new/choose 5 | about: Please create issues and ask questions in the `github/codeql` repository. 6 | -------------------------------------------------------------------------------- /.github/workflows/label-issue.yml: -------------------------------------------------------------------------------- 1 | name: Label issue 2 | on: 3 | issues: 4 | types: [opened] 5 | 6 | jobs: 7 | label: 8 | name: Label issue 9 | runs-on: ubuntu-slim 10 | permissions: 11 | issues: write 12 | steps: 13 | - name: Label issue 14 | run: gh issue edit "$NUMBER" --add-label "$LABELS" 15 | env: 16 | GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} 17 | GH_REPO: ${{ github.repository }} 18 | NUMBER: ${{ github.event.issue.number }} 19 | LABELS: CLI 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # CodeQL CLI 2 | 3 | This repo holds binaries for the CodeQL CLI. 4 | 5 | [CodeQL overview](https://securitylab.github.com/tools/codeql/) | [CodeQL CLI Documentation](https://codeql.github.com/docs/codeql-cli/) 6 | 7 | # Getting started 8 | 9 | 1. Go to the [Releases page](https://github.com/github/codeql-cli-binaries/releases). 10 | 2. Find the latest release, select Assets, and download the zip file containing the CLI. 11 | 3. You'll also want to clone https://github.com/github/codeql to get the CodeQL queries and 12 | libraries. Please take note of the 13 | [set-up instructions](https://codeql.github.com/docs/codeql-cli/getting-started-with-the-codeql-cli/) 14 | for placing it in a location where the CLI can find it. 15 | 4. Read the rest of the [CodeQL CLI documentation](https://codeql.github.com/docs/codeql-cli/). 16 | 17 | # Found a bug or have a question? 18 | 19 | Please raise an issue in the [github/codeql](https://github.com/github/codeql/issues/new/choose) repository. 20 | 21 | # License 22 | 23 | By downloading, you agree to the [GitHub CodeQL Terms & Conditions](https://securitylab.github.com/tools/codeql/license/). 24 | 25 | GitHub CodeQL can only be used on codebases that are released under an OSI-approved open source license, or to perform academic research. It can't be used to generate CodeQL databases for or during automated analysis, continuous integration or continuous delivery, whether as part of normal software engineering processes or otherwise. For these uses, [contact the sales team](https://enterprise.github.com/contact). 26 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | GitHub CodeQL Terms and Conditions 2 | ================================== 3 | These GitHub CodeQL Terms and Conditions ("**Terms**") are a legal 4 | agreement between you (either as an individual or on behalf of an 5 | entity) and GitHub, Inc. regarding your use of the GitHub CodeQL 6 | software and associated documentation (collectively, the 7 | "**Software**"). By using the Software, you accept these Terms. 8 | **Please read all of these Terms;** in many cases, provisions set 9 | forth later in the Terms limit and qualify provisions set forth 10 | earlier in the Terms. If you do not accept these Terms, do not 11 | download, install, use, or copy the Software. 12 | 13 | Definitions 14 | ----------- 15 | 16 | In these Terms: 17 | 18 | * "OSI-approved License" means an Open Source Initiative 19 | (OSI)-approved open source software license. 20 | 21 | * "Open Source Codebase" means a codebase that is released under an 22 | OSI-approved License. 23 | 24 | Use Rights; Scope of License 25 | ---------------------------- 26 | 27 | The Software is licensed on a per user basis. Here's what you may do 28 | with the Software, but subject to License Restrictions provisions 29 | below: 30 | 31 | * Use the Software to perform academic research. 32 | 33 | * Use the Software to demonstrate the Software. 34 | 35 | * Test CodeQL queries that are released under an OSI-approved 36 | Licence to confirm that new versions of those queries continue to 37 | find the right vulnerabilities. 38 | 39 | Here's what you may also do with the Software, but only with an Open 40 | Source Codebase and subject to the License Restrictions provisions 41 | below: 42 | 43 | * Perform analysis on the Open Source Codebase. 44 | 45 | * If the Open Source Codebase is hosted and maintained on 46 | GitHub.com, generate CodeQL databases for or during automated 47 | analysis, CI, or CD. 48 | 49 | License Restrictions 50 | -------------------- 51 | 52 | These Terms do not authorize, and the Software may not be used for any 53 | purpose not expressly set forth above, including: 54 | 55 | * To otherwise or in any other context generate any CodeQL database 56 | for or during automated analysis, CI or CD, whether as part of 57 | normal engineering processes or another context. 58 | 59 | * To otherwise or in any other context use the Software in 60 | connection with any codebase that is not an Open Source Codebase 61 | (e.g., code in a private repo in GitHub). 62 | 63 | _**Please note:** if your use of the Software is under a paid customer 64 | license for GitHub Advanced Security, the restrictions with respect to 65 | automated analysis, CI, and CD and use in connection with non-Open 66 | Source Codebases do not apply._ 67 | 68 | At all times, except (and only to the extent) permitted by applicable 69 | law or applicable third-party license, you will not (and have no right 70 | to): 71 | 72 | * work around any technical limitations in the Software that only 73 | allow you to use it in certain ways; 74 | 75 | * reverse engineer, decompile or disassemble the Software; 76 | 77 | * remove, minimize, block, or modify any notices of GitHub or its 78 | suppliers in the Software; 79 | 80 | * use the Software in any way that is against the law; or 81 | 82 | * share, publish, distribute or lend the Software, provide or make 83 | available the Software as a hosted solution (whether on a 84 | standalone basis or combined, incorporated or integrated with 85 | other software or services) for others to use, or transfer the 86 | Software or these Terms to any third party. 87 | 88 | The Software is licensed, not sold. GitHub reserves all rights not 89 | expressly granted in these Terms. 90 | 91 | Open Source Software 92 | -------------------- 93 | 94 | The Software may include components licensed under open source 95 | software licenses. Any such licenses are included in the "Open Source 96 | Notices" documentation that is included with the Software. Such 97 | documentation also includes copies of all applicable open source 98 | licenses. 99 | 100 | To the extent the terms of the licenses applicable to open source 101 | components require GitHub to make an offer to provide source code in 102 | connection with the Software, such offer is hereby made, and you may 103 | exercise it by contacting GitHub: https://github.com/contact. 104 | 105 | Unless otherwise agreed to in writing with GitHub, your agreement with 106 | GitHub will always include, at a minimum, these Terms. Open source 107 | software licenses for the Software's source code constitute separate 108 | written agreements. To the limited extent that any open source 109 | software license expressly supersedes these Terms, such open source 110 | license governs your use of the applicable component(s) of the 111 | Software subject to such license. 112 | 113 | GitHub Trademarks 114 | ----------------- 115 | 116 | These Terms do not grant any right or license to use any of GitHub's 117 | trademarks or logos, including, without limitation, the names GitHub 118 | and CodeQL and any Software logo designs in the "logos" folder of the 119 | Software. You agree not to display or use any of these trademarks or 120 | logos in any manner without GitHub's prior written permission, except 121 | as allowed by GitHub's Logos and Usage Policy located at 122 | https://github.com/logos. GitHub reserves all right, title and 123 | interest in and to all GitHub trademarks and logos. 124 | 125 | Additional Services 126 | ------------------- 127 | 128 | Auto-Updates: The Software may include an auto-update service. If the 129 | Software automatically enables such service (or, if it is not 130 | automatically enabled and you choose to use it), GitHub will 131 | automatically update the Software when a new version is available. 132 | 133 | Support 134 | ------- 135 | 136 | Because the Software is "as-is," GitHub may not provide support for it. 137 | 138 | Export Control 139 | -------------- 140 | 141 | Customer will comply with all applicable export and import laws and 142 | regulations that apply to the Software. 143 | 144 | Disclaimer; Limitations of Liability 145 | ------------------------------------ 146 | 147 | THE SOFTWARE, INCLUDING ANY ADDITIONAL SERVICES, IS PROVIDED ON AN 148 | "AS-IS" BASIS, AND GITHUB GIVES NO EXPRESS WARRANTIES, GUARANTEES OR 149 | CONDITIONS. TO THE EXTENT PERMITTED BY APPLICABLE LAW, GITHUB 150 | DISCLAIMS THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 151 | PARTICULAR PURPOSE AND NON-INFRINGEMENT. YOUR USE OF THE SOFTWARE IS 152 | AT YOUR SOLE RISK. 153 | 154 | TO THE EXTENT PERMITTED BY APPLICABLE LAW, YOU EXPRESSLY UNDERSTAND 155 | AND AGREE THAT (1) YOU CAN RECOVER DIRECT DAMAGES RELATING TO THE 156 | SOFTWARE, INCLUDING ANY ADDITIONAL SERVICES, UP TO U.S. $5.00 FROM 157 | GITHUB AND ITS SUPPLIERS, AND (2) GITHUB WILL NOT BE LIABLE FOR ANY 158 | INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL OR EXEMPLARY DAMAGES, 159 | INCLUDING, WITHOUT LIMITATION, ANY DAMAGES FOR LOSS OF PROFITS, 160 | GOODWILL, USE, OR DATA OR OTHER INTANGIBLE LOSSES (EVEN IF GITHUB HAS 161 | BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES) RELATING TO THE 162 | SOFTWARE, INCLUDING ANY ADDITIONAL SERVICES. 163 | 164 | Miscellaneous 165 | ------------- 166 | 167 | _No Waiver._ The failure of GitHub to exercise or enforce any right or 168 | provision of these Terms will not constitute a waiver of such right or 169 | provision. 170 | 171 | _Entire Agreement._ These Terms, together with any open source 172 | software licenses referenced above, constitutes the entire agreement 173 | between you and GitHub regarding your use of the Software, superseding 174 | any prior agreements between you and GitHub (including, but not 175 | limited to, any prior versions of these Terms) regarding such use. 176 | 177 | _Governing Law._ You agree that these Terms and your use of the 178 | Software are governed by the laws of the State of California and any 179 | dispute relating to the Software or your use thereof must be brought 180 | in a tribunal of competent jurisdiction located in or near San 181 | Francisco, California. 182 | 183 | _Modifications._ These Terms may only be modified by a written 184 | amendment signed by an authorized representative of GitHub, or by the 185 | posting by GitHub of a revised version. 186 | 187 | _Contact Us._ Questions about these Terms? Contact us at 188 | https://support.github.com/contact. 189 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # CodeQL CLI changelog 2 | 3 | 19 | ## Release 2.23.8 (2025-12-10) 20 | 21 | This release contains no CLI changes. 22 | 23 | ## Release 2.23.7 (2025-12-05) 24 | 25 | ### Deprecations 26 | 27 | - The `--save-cache` flag to `codeql database run-queries` and other commands that execute queries has been deprecated. This flag previously instructed the evaluator to aggressively write intermediate results to the disk cache, but now has no effect. 28 | 29 | ## Release 2.23.6 (2025-11-24) 30 | 31 | ### Breaking changes 32 | 33 | - The LGTM results format for uploading to LGTM has been removed. 34 | 35 | ## Release 2.23.5 (2025-11-13) 36 | 37 | ### Breaking changes 38 | 39 | - In order to make a `@kind path-problem` query diff-informed, the `getASelectedSourceLocation` and `getASelectedSinkLocation` predicates in the dataflow configuration now need to be overridden to always return the location of the source/sink _in addition to_ any other locations that are selected by the query. See the [QLdoc](https://github.com/github/codeql/blob/d122534398c5eb9182a23a9ad65caa5937d627b5/shared/dataflow/codeql/dataflow/DataFlow.qll#L474) for more details. 40 | 41 | ## Release 2.23.4 42 | 43 | This release was skipped. 44 | 45 | ## Release 2.23.3 (2025-10-17) 46 | 47 | ### Breaking changes 48 | 49 | - The `--permissive` command line option has been removed from the C/C++ extractor, 50 | and passing the option will make the extractor fail. The option was introduced to 51 | make the extractor accept the following invalid code, which is accepted by gcc with 52 | the `-fpermissive` flag: 53 | 54 | ```cpp 55 | void f(char*); 56 | void g() { 57 | const char* str = "string"; 58 | f(str); 59 | } 60 | ``` 61 | 62 | The `--permissive` option was removed, as under some circumstances it would break the extractor's ability to parse valid C++ code. When calling the extractor directly, 63 | `--permissive` should no longer be passed. The above code will fail to parse, and we 64 | recommend the code being made `const`-correct. 65 | 66 | ### Bugs fixed 67 | 68 | - Fixed a bug that made many `codeql` subcommands fail with the 69 | message `not in while, until, select, or repeat loop` on Linux or 70 | macOS systems where `/bin/sh` is `zsh`. 71 | 72 | ## Release 2.23.2 (2025-10-02) 73 | 74 | ### New features 75 | 76 | - CodeQL Go analysis now supports the "Git Source" type for [private package registries](https://docs.github.com/en/code-security/securing-your-organization/enabling-security-features-in-your-organization/giving-org-access-private-registries). This is in addition to the existing support for the "GOPROXY server" type. 77 | 78 | ### Bugs Fixed 79 | 80 | - The `codeql generate query-help` command now prepends the query's name (taken from the `.ql` file) as a level-one heading when processing markdown query help, for consistency with help generated from a `.qhelp` file. 81 | 82 | ## Release 2.23.1 (2025-09-23) 83 | 84 | ### New features 85 | 86 | - CodeQL now adds the sources and sinks of path alerts to the `relatedLocations` 87 | property of SARIF results if they are not included as the primary location or 88 | within the alert message. This means that path alerts will show on PRs if a 89 | source or sink is added or modified, even for queries that don't follow the 90 | common convention of selecting the sink as the primary location and mentioning 91 | the source in the alert message. 92 | 93 | - CodeQL now populates file coverage information for GitHub Actions on 94 | [the tool status page for code scanning](https://docs.github.com/en/code-security/code-scanning/managing-your-code-scanning-configuration/about-the-tool-status-page#viewing-the-tool-status-page-for-a-repository). 95 | 96 | ## Release 2.23.0 (2025-09-04) 97 | 98 | ### Miscellaneous 99 | 100 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL 101 | CLI has been updated to version 21.0.8. 102 | 103 | ## Release 2.22.4 (2025-08-21) 104 | 105 | - There are no user-facing changes in this release. 106 | 107 | ## Release 2.22.3 (2025-08-06) 108 | 109 | ### New features 110 | 111 | - The `codeql database cleanup` command now takes the `--cache-cleanup=overlay` 112 | option, which trims the cache to just the data that will be useful when 113 | evaluating against an overlay. 114 | 115 | ## Release 2.22.2 (2025-07-29) 116 | 117 | ### Bug fix 118 | 119 | - Fixes a bug in query suites where the `version` property of an `import` instruction was ignored. Previously, the following query suite would _not_ resolve to `v1.0.19` of `codeql/csharp-queries`. Instead it would resolve to the latest version. This is now fixed and the resolve pack version would be `v1.0.19`. 120 | ``` 121 | - from: codeql/csharp-queries 122 | import: codeql-suites/csharp-security-and-quality.qls 123 | version: 1.0.19 124 | ``` 125 | 126 | ## Release 2.22.1 (2025-06-26) 127 | 128 | ### New features 129 | 130 | - Rust language support is now in public preview. 131 | 132 | ### Miscellaneous 133 | 134 | - The version of `jgit` used by the CodeQL CLI has been updated to `6.10.1.202505221210-r`. 135 | 136 | ## Release 2.22.0 (2025-06-11) 137 | 138 | ### Breaking changes 139 | 140 | - A number of breaking changes have been made to the C and C++ CodeQL test 141 | environment as used by `codeql test run`: 142 | - Options starting with a `/` are no longer supported by 143 | `semmle-extractor-options`. Any option starting with a `/` should be 144 | replaced by the equivalent option starting with a `-`, e.g., `/D` should be 145 | replaced by `-D`. 146 | - Preprocessor command line options of the form `-D#` are no 147 | longer supported by `semmle-extractor-options`. `-D=` should be 148 | used instead. 149 | - The `/Fp` and `-o` options are no longer supported by 150 | `semmle-extractor-options`. The options should be omitted. 151 | - The `-emit-pch`, `-include-pch`, `/Yc`, and `/Yu` options, and the 152 | `--preinclude` option taking a pre-compiled header as its argument, are no 153 | longer supported by `semmle-extractor-options`. Any test that makes use of 154 | this should be replaced by a test that invokes the CodeQL CLI with the 155 | `create database` option and that runs the relevant queries on the created 156 | database. 157 | 158 | ## Release 2.21.4 (2025-06-02) 159 | 160 | ### Deprecations 161 | 162 | - The `clang_vector_types`, `clang_attributes`, and `flax-vector-conversions` command 163 | line options have been removed from the C/C++ extractor. These options were introduced 164 | as workarounds to frontend limitations in earlier versions of the extractor and are 165 | no longer needed when calling the extractor directly. 166 | 167 | ### Miscellaneous 168 | 169 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL 170 | CLI has been updated to version 21.0.7. 171 | 172 | ## Release 2.21.3 (2025-05-15) 173 | 174 | ### Miscellaneous 175 | 176 | - Windows binaries for the CodeQL CLI are now built with `/guard:cf`, enabling [Control Flow Guard](https://learn.microsoft.com/en-us/windows/win32/secbp/control-flow-guard). 177 | 178 | ## Release 2.21.2 (2025-05-01) 179 | 180 | ### Bugs fixed 181 | 182 | - `codeql generate log-summary` now correctly includes `dependencies` 183 | maps in predicate events for `COMPUTED_EXTENSIONAL` predicates. 184 | 185 | ## Release 2.21.1 (2025-04-22) 186 | 187 | ### Bugs fixed 188 | 189 | - Fixed a bug in CodeQL analysis for GitHub Actions in the presence 190 | of a code scanning configuration file containing `paths-ignore` 191 | exclusion patterns but not `paths` inclusion patterns. 192 | Previously, such a configuration incorrectly led to all YAML, HTML, 193 | JSON, and JS source files being extracted, 194 | except for those filtered by `paths-ignore`. 195 | This in turn led to performance issues on large codebases. 196 | Now, only workflow and Action metadata YAML files relevant to the 197 | GitHub Actions analysis will be extracted, 198 | except for those filtered by `paths-ignore`. 199 | This matches the default behavior when no configuration file 200 | is provided. 201 | The handling of `paths` inclusion patterns is unchanged: 202 | if provided, only those paths will be considered, 203 | except for those filtered by `paths-ignore`. 204 | 205 | ## Release 2.21.0 (2025-04-03) 206 | 207 | ### Miscellaneous 208 | 209 | - On macOS the `CODEQL_TRACER_RELOCATION_EXCLUDE` environment variable can now be used to exclude certain paths from the 210 | tracer relocation and tracing process. This environment variable accepts newline-separated regex patterns of binaries 211 | to be excluded. 212 | 213 | ## Release 2.20.7 (2025-03-18) 214 | 215 | - There are no user-facing changes in this release. 216 | 217 | ## Release 2.20.6 (2025-03-06) 218 | 219 | ### Miscellaneous 220 | 221 | - The CodeQL XML extractor is now able to parse documents in a wider array of 222 | character sets. 223 | 224 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL 225 | CLI has been updated to version 21.0.6. 226 | 227 | ## Release 2.20.5 (2025-02-20) 228 | 229 | ### Breaking changes 230 | 231 | - Removed support for `QlBuiltins::BigInt`s in the `avg()` aggregate. 232 | 233 | - A number of breaking changes have been made to the C and C++ CodeQL test environment as used by `codeql test run`: 234 | - The `-Xclang-only=` option is no longer supported by `semmle-extractor-options`. Instead, when either `--clang` or `--clang_version` is specified the option should be replaced by `` only, otherwise the option should be omitted. 235 | - The `--sys_include ` and `--preinclude ` options are no longer supported by `semmle-extractor-options`. Instead, `--edg --edg ` should be specified. 236 | - The `-idirafter ` option is no longer supported by `semmle-extractor-options`. Instead, `--edg --sys_include --edg ` should be specified. 237 | - The `-imacros ` option is no longer supported by `semmle-extractor-options`. Instead, `--edg --preinclude_macros --edg ` should be specified. 238 | - The `/FI ` option is no longer supported by `semmle-extractor-options`. Instead, `--edg --preinclude --edg ` should be specified. 239 | - The `-Wreserved-user-defined-literal`, `-Wno-reserved-user-defined-literal`, `-fwritable-strings`, `/Zc:rvalueCast`, `/Zc:rvalueCast-`, and `/Zc:wchar_t-` options are no longer supported by `semmle-extractor-options`. Instead, `--edg --reserved_user_defined_literal`, `--edg --no-reserved_user_defined_literal`, `--edg --no_const_string_literals`, `--edg --no_preserve_lvalues_with_same_type_casts`, `--edg --preserve_lvalues_with_same_type_casts`, and `--edg --no_wchar_t_keyword` should be specified, respectively. 240 | - The `/Fo ` option is no longer supported by `semmle-extractor-options`. The option should be omitted. 241 | 242 | ## Release 2.20.4 (2025-02-06) 243 | 244 | ### New features 245 | 246 | - Using the `actions` language (for analysis of GitHub Actions workflows) no longer requires 247 | the `CODEQL_ENABLE_EXPERIMENTAL_FEATURES` environment variable to be set. Support for analysis 248 | of GitHub Actions workflows remains in public preview. 249 | 250 | ### Bugs fixed 251 | 252 | - Fixed a bug where CodeQL for Java would fail with an SSL exception while trying to download `maven`. 253 | 254 | ### Miscellaneous 255 | 256 | - The build of the [logback-core](https://logback.qos.ch/) library that is used for logging in the CodeQL CLI has been updated to version 1.3.15. 257 | 258 | ## Release 2.20.3 (2025-01-24) 259 | 260 | ### Security Updates 261 | 262 | - Resolves a security vulnerability where CodeQL databases or logs produced by the CodeQL CLI may contain the environment variables from the time of 263 | database creation. This includes any secrets stored in an environment variables. For more information, see the 264 | [CodeQL CLI security advisory](https://github.com/github/codeql-cli-binaries/security/advisories/GHSA-gqh3-9prg-j95m). 265 | 266 | All users of CodeQL should follow the advice in the CodeQL advisory mentioned above or upgrade to this version or a later version of CodeQL. 267 | 268 | If you are using the CodeQL Action, also see the related [CodeQL Action security advisory](https://github.com/github/codeql-action/security/advisories/GHSA-vqf5-2xx6-9wfm). 269 | 270 | ## Release 2.20.2 (2025-01-22) 271 | 272 | ### Improvements 273 | 274 | - `codeql database create` and `codeql database finalize` now write relations to disk in a new, compressed format. As a result, databases will generally take up less space on disk, whether zipped or unzipped. Note that databases in this format can only be read and analyzed using CodeQL version 2.20.1 onwards. Attempting to analyze such a database with CodeQL version 2.20.0 or older will fail, with an error message like the following: 275 | ``` 276 | UnsortedExtensionalError: Tuples that were assumed to be in order are not: [123456777, 777654321, 123456777]<[777654321, 123456777, 777654321] 277 | ``` 278 | 279 | ### Enhancements 280 | 281 | - Added the `.bitLength()` method to `QlBuiltins::BigInt`. 282 | 283 | ### Bugs Fixed 284 | 285 | - Fixed a bug where CodeQL would crash on rare occasions while merging SARIF files before uploading results. 286 | 287 | ## Release 2.20.1 (2025-01-09) 288 | 289 | ### Improvements 290 | 291 | - Automatic installation of dependencies for C++ autobuild is now supported on Ubuntu 24.04. 292 | 293 | - The CLI will now warn if it detects that it is installed in a 294 | location where it is likely to cause performance issues. This 295 | includes: user home, desktop, downloads, or the file system root. 296 | 297 | You can avoid this warning by setting the `CODEQL_ALLOW_INSTALLATION_ANYWHERE` 298 | environment variable to `true`. 299 | 300 | ## Release 2.20.0 (2024-12-09) 301 | 302 | ### Known issues 303 | 304 | - The Windows executable for this release is labeled with an incorrect version number 305 | within its properties: the version number should be 2.20.0 rather than 2.19.4. 306 | `codeql version` reports the correct version number. 307 | 308 | ### New features 309 | 310 | - The [`QlBuiltins::BigInt` type](https://codeql.github.com/docs/ql-language-reference/modules/#bigint) of 311 | arbitrary precision integers is generally available and no longer hidden behind the 312 | `--allow-experimental=bigint` CLI feature flag. 313 | 314 | ### Miscellaneous 315 | 316 | - Backslashes are now escaped when writing output in the Graphviz DOT format (`--format=dot`). 317 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL CLI has been updated to version 318 | 21.0.5. 319 | 320 | ## Release 2.19.4 (2024-12-02) 321 | 322 | ### Improvements 323 | 324 | - CodeQL now supports passing values containing the equals character (`=`) to extractor options via 325 | the `--extractor-option` flag. This allows cases like `--extractor-option opt=key=value`, which 326 | sets the extractor option `opt` to hold the value `key=value`, whereas previously that would have 327 | been rejected with an error. 328 | - The `codeql pack bundle` command now sets the numeric user and group IDs of entries in the generated 329 | `tar` archive to `0`. This avoids failures like `IllegalArgumentException: user id '7111111' is too big ( > 2097151 )` 330 | when the numeric user ID is too large. 331 | 332 | ### Bugs fixed 333 | 334 | - On MacOS, `arch -arm64` commands no longer fail when they are executed via `codeql database create --command`, 335 | via `codeql database trace-command`, or are run after `codeql database init --begin-tracing`. Note 336 | that build commands invoked this way still will not normally be traced, so this is useful only for 337 | running ancillary commands which are incidental to building your code. 338 | - Fixed a bug where `codeql test run` would not preserve test 339 | databases on disk after a test failed. 340 | 341 | ## Release 2.19.3 (2024-11-07) 342 | 343 | ### Bugs fixed 344 | 345 | - Fixed a bug where using `codeql database import` to combine multiple non-empty 346 | databases may produce a corrupted database. (The bug does not affect using 347 | `codeql database finalize --additional-dbs` to combine multiple databases.) 348 | 349 | - Fixed a bug where uses of a `QlBuiltins::ExtensionId` variable that was not 350 | bound to a value could be incorrectly accepted in some cases. In many cases, 351 | this would result in a crash. 352 | 353 | - CodeQL would sometimes refuse to run with more than around 1,500 GB of RAM 354 | available, complaining that having so much memory was "unrealistic". The 355 | amount of memory CodeQL is able to make any meaningful use of still tops out 356 | at about that value, but it will now gracefully accept that so large 357 | computers do in fact exist. 358 | 359 | - Fixed a bug in command-line parsing where a misspelled option could sometimes 360 | be misinterpreted as, e.g., the name of a query to run. Now every command-line 361 | argument that begins with a dash is assumed to be intended as an option 362 | (unless it comes after the `--` separator), and an appropriate error is 363 | emitted if that is not a recognized one. 364 | 365 | The build command in `codeql database trace-command` is exempted from this for 366 | historical reasons, but we strongly recommend putting a `--` before the entire 367 | build command there, in case a future `codeql` version starts recognizing 368 | options that you intended to be part of the build command. 369 | 370 | ### Miscellaneous 371 | 372 | - The CodeQL Bundle is now available as an artifact that is compressed using 373 | [Zstandard](https://en.wikipedia.org/wiki/Zstd). This artifact is 374 | smaller and faster to decompress than the original, gzip-compressed bundle. The CodeQL bundle 375 | is a tar archive containing tools, scripts, and various CodeQL-specific files. 376 | 377 | If you are currently using the CodeQL Bundle, you may want to consider switching to the 378 | Zstandard variant of the bundle. You can download the new form of the CodeQL Bundle from the 379 | [codeql-action releases page](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.19.3) 380 | by selecting the appropriate bundle with the `.zst` extension. The gzip-compressed bundles will 381 | continue to be available for backwards compatibility. 382 | 383 | ## Release 2.19.2 (2024-10-21) 384 | 385 | ### Potentially breaking changes 386 | 387 | - The Python extractor will no longer extract the standard library by default, relying instead on models of the standard library. This should result in significantly faster extraction and analysis times, while the effect on alerts should be minimal. It will for a while be possible to force extraction of the standard library by setting the environment variable `CODEQL_EXTRACTOR_PYTHON_EXTRACT_STDLIB` to `1`. 388 | 389 | ### Bugs fixed 390 | 391 | - The 2.19.1 release contained a bug in the query evaluator that under rare conditions could lead to wrong alerts or resource exhaustion. Although we have never seen the problem outside of internal testing, we encourage users on 2.19.1 to upgrade to 2.19.2. 392 | 393 | ### Miscellaneous 394 | 395 | - The database relation `sourceLocationPrefix` is changed for databases created with 396 | `codeql test run`. Instead of containing the path of the enclosing qlpack, it now 397 | contains the actual path of the test, similar to if one had run `codeql database create` 398 | on the test folder. For example, for a test such as 399 | `/cpp/ql/test/query-tests/Security/CWE/CWE-611/XXE.qlref` we now populate 400 | `sourceLocationPrefix` with `/cpp/ql/test/query-tests/Security/CWE/CWE-611/` 401 | instead of `/cpp/ql/test/`. This change typically impacts calls to 402 | `File.getRelativePath()`, and may as a result change the expected test output. 403 | 404 | ## Release 2.19.1 (2024-10-04) 405 | 406 | ### New Features 407 | 408 | - The command `codeql generate query-help` now supports Markdown help files. 409 | The Markdown help format is commonly used in custom CodeQL query packs. This new 410 | feature allows us to generate SARIF reporting descriptors for CodeQL queries that 411 | include Markdown help directly from a query Markdown help file. 412 | 413 | - Added a new command, `codeql resolve packs`. This command shows each step in the 414 | pack search process, including what packs were found in each step. With the 415 | `--show-hidden-packs` option, it can also show details on which packs were hidden 416 | by packs found earlier in the search sequence. `codeql resolve packs` is intended 417 | as a replacement for most uses of `codeql resolve qlpacks`, whose output is both 418 | less detailed and less accurate. 419 | 420 | ## Release 2.19.0 (2024-09-18) 421 | 422 | ### Improvements 423 | 424 | - `codeql database analyze` and `codeql database interpret-results` now support 425 | the `--sarif-run-property` option. You can provide this option when using a SARIF 426 | output format to add a key-value pair to the property bag of the run object. 427 | 428 | ### Miscellaneous 429 | 430 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL 431 | CLI has been updated to version 21.0.4. 432 | 433 | ## Release 2.18.4 (2024-09-12) 434 | 435 | ### New Features 436 | 437 | - C# support for `build-mode: none` is now out of beta, and generally available. 438 | - Go 1.23 is now supported. 439 | 440 | ## Release 2.18.3 (2024-08-28) 441 | 442 | - There are no user-facing changes in this release. 443 | 444 | ## Release 2.18.2 (2024-08-13) 445 | 446 | ### Deprecations 447 | 448 | - Swift analysis on Ubuntu is no longer supported. Please migrate to macOS if this affects you. 449 | 450 | ### Miscellaneous 451 | 452 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL 453 | CLI has been updated to version 21.0.3. 454 | 455 | ## Release 2.18.1 (2024-07-25) 456 | 457 | ### Security Updates 458 | 459 | - Resolves CVE-2023-4759, an arbitrary file overwrite in Eclipse JGit 460 | that can be triggered when using untrusted third-party queries from a 461 | git repository. See the 462 | [security advisory](https://github.com/github/codeql-cli-binaries/security/advisories/GHSA-x4gx-f2xv-6wj9) 463 | for more information. 464 | - The following dependencies have been updated. These updates include 465 | security fixes in the respective libraries that prevent 466 | out-of-bounds accesses or denial-of-service in scenarios where 467 | untrusted files are processed. These scenarios are not likely to be 468 | encountered in most uses of CodeQL and code scanning, and only 469 | apply to advanced use cases where precompiled query packs, 470 | database ZIP files, or database TRAP files are obtained from 471 | untrusted sources and then processed on a trusted machine. 472 | - airlift/aircompressor is updated to version 0.27. 473 | - Apache Ant is updated to version 1.10.11. 474 | - Apache Commons Compress is updated to version 1.26.0. 475 | - Apache Commons IO is updated to version 2.15.1. 476 | - Apache Commons Lang3 is updated to version 3.14.0. 477 | - jsoup is updated to version 1.15.3. 478 | - Logback is updated to version 1.2.13. 479 | - Snappy is updated to version 0.5. 480 | 481 | ### New features 482 | 483 | - The *experimental* type `QlBuiltins::BigInt` of arbitrary-precision integers 484 | has been introduced. To opt in to this API, compile your queries with 485 | `--allow-experimental=bigint`. Big integers can be constructed using the 486 | `.toBigInt()` methods of `int` and `string`. The built-in operations are: 487 | - comparisons: `=`, `!=`, `<`, `<=`, `>`, `>=`, 488 | - conversions: `.toString()`, `.toInt()`, 489 | - arithmetic: binary `+`, `-`, `*`, `/`, `%`, unary `-`, 490 | - bitwise operations: `.bitAnd(BigInt)`, `.bitOr(BigInt)`, 491 | `.bitXor(BigInt)`, `.bitShiftLeft(int)`, `.bitShiftRightSigned(int)`, 492 | `.bitNot()`, 493 | - aggregates: `min`, `max`, (`strict`)`sum`, (`strict`)`count`, `avg`, 494 | `rank`, `unique`, `any`. 495 | - other: `.pow(int)`, `.abs()`, `.gcd(BigInt)`, `.minimum(BigInt)`, 496 | `.maximum(BigInt)`. 497 | - `codeql test run` now supports postprocessing of test results. When .qlref 498 | files specify a path to a `postprocess` query, then this is evaluated after 499 | the test query to transform the test outputs prior to concatenating them into 500 | the `actual` results. 501 | 502 | ### Improvements 503 | 504 | - The 30% QL query compilation slowdown noted in 2.18.0 has been fixed. 505 | 506 | ## Release 2.18.0 (2024-07-11) 507 | 508 | ### Breaking changes 509 | 510 | - A number of breaking changes have been made to the C and C++ CodeQL 511 | test environment as used by `codeql test run`: 512 | - The test environment no longer defines any GNU-specific builtin 513 | macros. If these macros are still needed by a test, please define 514 | them via `semmle-extractor-options`. 515 | - The `--force-recompute` option is no longer directly supported by 516 | `semmle-extractor-options`. Instead, `--edg --force-recompute` 517 | should be specified. 518 | - The `--gnu_version` and `--microsoft_version` options that can be 519 | specified via `semmle-extractor-options` are now synonyms, and only 520 | one should be specified as part of `semmle-extractor-options`. 521 | Furthermore, is also no longer possible to specify these options 522 | via the following syntax. 523 | 524 | - `--edg --gnu_version --edg `, and 525 | - `--edg --microsoft_version --edg ` 526 | 527 | The shorter `--gnu_version ` and 528 | `--microsoft_version ` should be used. 529 | - The `--build_error_dir` and `--predefined_macros` command line options 530 | have been removed from the C/C++ extractor. It has never been possible 531 | to pass these options through the CLI, but some customers with advanced 532 | setups may have been passing them through internal undocumented interfaces. 533 | Passing the option `--build_error_dir` did not have any effect, and it 534 | is safe to remove the option. The `--predefined_macros` option should 535 | have been unnecessary, as long as the extractor was invoked with the 536 | `--mimic` option. 537 | 538 | ### Regressions 539 | 540 | - Compilation of QL queries is about 30% slower than in previous releases. This only affects users who write custom queries, and only at compilation time, not at run time. This regression will be fixed in the upcoming 2.18.1 release. 541 | 542 | ### Improvements 543 | 544 | - Introduced the `--include-logs` option to the `codeql database bundle` 545 | command. This new feature allows users to include logs in the generated 546 | database bundle, allowing for a more complete treatment of the bundle, and 547 | bringing the tool capabilities up-to-speed with the documentation. 548 | - `codeql database init` and `codeql database create` now support the 549 | `--force-overwrite` option. When this option is specified, the command will 550 | delete the specified database directory even if it does not look like a 551 | database directory. This option is only recommended for automation. For 552 | directcommand line commands, it is recommended to use the `--overwrite` 553 | option, which includes extra protection and will refuse to delete a 554 | directory that does not look like a database directory. 555 | - Extract `.xsaccess`, `*.xsjs` and `*.xsjslib` files for SAP HANA XS as 556 | Javascript. 557 | - We have updated many compiler error messages and warnings to improve their 558 | readability and standardize their grammar. 559 | Where necessary, please use the `--learn` option for the `codeql test run` 560 | command. 561 | 562 | ### Bugs fixed 563 | 564 | - Where a MacOS unsigned binary cannot be signed, CodeQL will now continue 565 | trying to trace compiler invocations created by that process and its 566 | children. In particular this means that Bazel builds on MacOS are now 567 | traceable. 568 | - Fixed a bug where test discovery would fail if there is a syntax error in a 569 | qlpack file. Now, a warning message will be printed and discovery will 570 | continue. 571 | 572 | ## Release 2.17.6 (2024-06-27) 573 | 574 | ### New features 575 | 576 | - Beta support is now available for analyzing C# codebases without needing a working build. To use 577 | this, pass the `--build-mode none` option to `codeql database create`. 578 | 579 | ### Improvements 580 | 581 | - The `--model-packs` option is now publicly available. This option allows commands like `codeql database analyze` 582 | to accept a list of model packs that are used to augment the analysis of all queries involved in the analysis. 583 | 584 | ## Release 2.17.5 (2024-06-12) 585 | 586 | ### Breaking changes 587 | 588 | - All the commands that output SARIF will output a minified version to reduce the size. 589 | The `codeql database analyze`, `codeql database interpret-results`, `codeql generate query-help`, and `codeql bqrs interpret` commands support the option `--no-sarif-minify` to output a pretty printed SARIF file. 590 | 591 | - A number of breaking changes have been made to the `semmle-extractor-options` 592 | functionality available for C and C++ CodeQL tests. 593 | 594 | - The Arm, Intel, and CodeWarrior compilers are no longer supported and the 595 | `--armcc`, `--intel`, `--codewarrior` flags are now ignored, as are all the 596 | flags that only applied to those compilers. 597 | - The `--threads` and `-main-file-name` options, which did not have any effect 598 | on tests, are now ignored. Any specification of these options as part of 599 | `semmle-extractor-options` should be removed. 600 | - Support for `--linker`, all flags that would only invoke the preprocessor, 601 | and the `/clr` flag have been removed, as those flags would never produce any 602 | usable test output. 603 | - Support for the `--include_path_environment` flag has been removed. All include 604 | paths should directly be specified as part of `semmle-extractor-options`. 605 | - Microsoft C/C++ compiler response files specified via `@some_file_name` are 606 | now ignored. Instead, all options should directly be specified as part of 607 | `semmle-extractor-options`. 608 | - Support for Microsoft `#import` preprocessor directive has been removed, as 609 | support depends on the availability of the Microsoft C/C++ compiler, and 610 | availability cannot be guaranteed on all platforms while executing tests. 611 | - Support for the Microsoft `/EHa`, `/EHs`, `/GX`, `/GZ`, `/Tc`, `/Tp`, and `/Zl` 612 | flags, and all `/RTC` flags have been removed. Any specification of these 613 | options as part of `semmle-extractor-options` should be removed. 614 | - Support for the Apple-specific `-F` and `-iframework` flags has been removed. 615 | The `-F` flag can still be used by replacing `-F ` by 616 | `--edg -F --edg `. Any occurrence of `-iframework ` should be 617 | replaced by `--edg --sys_framework --edg `. 618 | - Support for the `/TC`, `/TP`, and `-x` flags has been removed. Please ensure 619 | all C, respectively C++, source files have a `.c`, respectively `.cpp`, 620 | extension. 621 | - The `--build_error_dir`, `-db`, `--edg_base_dir`, `--error_limit`, 622 | `--src_archive`, `--trapfolder`, and `--variadic_macros` flags are now ignored. 623 | 624 | The above changes do not affect the creation of databases through the CodeQL CLI, 625 | or when calling the C/C++ extractor directly with the `--mimic` or `--linker` flags. 626 | Similar functionality continues to be supported in those scenarios, except for 627 | CodeWarrior and the `--edg_base_dir`, `--include_path_environment`, `/Tc`, and `/Tp` 628 | flags, which were never supported. 629 | 630 | ### Improvements 631 | 632 | - `codeql generate log-summary` now reports completed pipeline runs that 633 | are part of an incomplete recursive predicate. 634 | 635 | ### Miscellaneous 636 | 637 | - The OWASP Java HTML Sanitizer library used by the CodeQL CLI for internal 638 | documentation generation commands has been updated to version 639 | [20240325.1](https://github.com/OWASP/java-html-sanitizer/releases/tag/release-20240325.1). 640 | 641 | ## Release 2.17.4 (2024-06-03) 642 | 643 | ### New features 644 | 645 | - CodeQL package management is now generally available, and all GitHub-produced 646 | CodeQL packages have had their version numbers increased to 1.0.0. 647 | 648 | ## Release 2.17.3 (2024-05-17) 649 | 650 | ### Improvements 651 | 652 | - The language server that our IDE integration is built on now defaults 653 | to fine-grained dependency tracking for incremental error-checking 654 | after file changes. This slightly improves the latency of refreshing 655 | errors after local source code edits and will enable significant 656 | speedups in the future. 657 | - We now properly handle globs (such as `folder/**/*.py`) in `paths` configuration 658 | to specify what files to include for Python analysis (see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#specifying-directories-to-scan). 659 | - TRAP import (a part of `codeql database create` and `codeql database finalize`) 660 | now supports allocating 2^32 IDs during the import process. The previous limit 661 | was 2^31 IDs. 662 | 663 | ## Release 2.17.2 (2024-05-07) 664 | 665 | ### Known issues 666 | 667 | - The beta support for analyzing Swift in this release and all 668 | previous releases requires `g++-13` when running on Linux. Users 669 | analyzing Swift using the `ubuntu-latest`, `ubuntu-22.04`, or 670 | `ubuntu-20.04` runner images for GitHub Actions should update their 671 | workflows to install `g++-13`. For more information, see [the runner 672 | images 673 | announcement](https://github.com/actions/runner-images/issues/9679). 674 | 675 | ### Improvements 676 | 677 | - When uploading a SARIF file to GitHub using `codeql github 678 | upload-results`, the CodeQL CLI now waits for the file to be 679 | processed by GitHub. If any errors occurred during processing of the 680 | analysis results, the command will log these and return a non-zero 681 | exit code. To disable this behaviour, pass the 682 | `--no-wait-for-processing` flag. 683 | 684 | By default, the command will wait for the SARIF file to be processed 685 | for a maximum of 2 minutes, however this is configurable with the 686 | `--wait-for-processing-timeout` option. 687 | - The build tracer is no longer enabled when using the [`none` build 688 | mode](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#codeql-build-modes) 689 | to analyze a compiled language, thus improving performance. 690 | 691 | ## Release 2.17.1 (2024-04-24) 692 | 693 | ### Deprecations 694 | 695 | - The `--mode` option and `-m` alias to `codeql database create`, 696 | `codeql database cleanup`, and `codeql dataset cleanup` has been 697 | deprecated. Instead, use the new `--cache-cleanup` option, which has 698 | identical behavior. 699 | 700 | ### Improvements 701 | 702 | - Improved the diagnostic message produced when no code is processed 703 | when creating a database. If a build mode was specified using 704 | `--build-mode`, the message is now tailored to your build mode. 705 | 706 | ### Miscellaneous 707 | 708 | - The `scc` tool used by the CodeQL CLI to calculate source code baseline 709 | information has been updated to version [3.2.0](https://github.com/boyter/scc/releases/tag/v3.2.0). 710 | 711 | ## Release 2.17.0 (2024-04-04) 712 | 713 | ### Deprecations 714 | 715 | - The `--[no-]analysis-summary-v2` and `--[no-]new-analysis-summary` options 716 | that were used to enable (or disable) improved summary information printed at 717 | the end of a `codeql database analyze` invocation are no longer supported. 718 | Improved summary information is now enabled for all invocations. 719 | - Support for overwriting default CodeQL SARIF run properties using the 720 | `--sarif-run-property` command line option has been removed. This removes the 721 | ability to overwrite the `semmle.formatSpecifier`, `metricResults`, and 722 | `codeqlConfigSummary` properties in the SARIF run file. 723 | 724 | ### Improvements 725 | 726 | - TRAP import (a part of `codeql database create` and `codeql database finalize`) 727 | now performs better in low-memory situations. (Put another way, it now needs 728 | less RAM to achieve the same performance as before.) 729 | 730 | - The worst-case performance of transitive closure computation (using 731 | the `+` or `*` postfix operators or the `fastTC` higher-order 732 | primitive in QL) has been greatly improved. 733 | 734 | ### Miscellaneous 735 | 736 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL 737 | CLI has been updated to version 21.0.2. 738 | 739 | ## Release 2.16.6 (2024-03-26) 740 | 741 | ### Bugs fixed 742 | 743 | - Fixes a bug where extractor logs would be output at a lower than expected 744 | verbosity level when using the `codeql database create` command. 745 | 746 | ## Release 2.16.5 (2024-03-21) 747 | 748 | ### New features 749 | 750 | - Beta support has been added for analyzing Java codebases without needing a working build. To enable 751 | this, pass the `--build-mode none` option to `codeql database create`. 752 | 753 | ## Release 2.16.4 (2024-03-11) 754 | 755 | ### Potentially breaking changes 756 | 757 | - A number of internal command line options (`--builtin_functions_file`, `--clang_builtin_functions`, 758 | `--disable-objc-default-synthesize-properties`, `--list_builtin_functions`, `--memory-limit-bytes`, 759 | `--mimic_config`, and `--objc`) has been removed from the C/C++ extractor. It has never been 760 | possible to pass these options through the CLI itself, but some customers with advanced setups may 761 | have been passing them through internal undocumented interfaces. All of the removed options were 762 | already no-ops, and will now generate errors. 763 | 764 | The `--verbosity` command line option has also been removed. The option was an alias for 765 | `--codeql-verbosity`, which should be used instead. 766 | 767 | ### Improvements 768 | 769 | - The frontend of the C/C++ extractor has been updated, improving the 770 | extractor's reliability and increasing its ability to extract source code. 771 | 772 | ### Bugs fixed 773 | 774 | - When parsing user-authored YAML files such as `codeql-pack.yml`, 775 | `qlpack.yml`, `codeql-workspace.yml`, and any YAML file defining a data 776 | extension, unquoted string values starting with a `*` character are now 777 | correctly interpreted as YAML aliases. Previously, they were interpreted 778 | as strings, but with the first character skipped. 779 | 780 | If you see a parse error similar to `while scanning an alias... unexpected` 781 | `character found *(42)`,it likely means that you need to add quotes around 782 | the indicated string value. The most common cause is unquoted glob patterns 783 | that start with `*`, such as `include: **/*.yml`, which will need to be 784 | quoted as `include: "**/*.yml"`. 785 | 786 | ## Release 2.16.3 (2024-02-22) 787 | 788 | ### Security patches 789 | 790 | - Fixes CVE-2024-25129, a limited data exfiltration vulnerability that 791 | could be triggered by untrusted databases or QL packs. See the 792 | [security advisory](https://github.com/github/codeql-cli-binaries/security/advisories/GHSA-gf8p-v3g3-3wph) 793 | for more information. 794 | 795 | ### New Features 796 | 797 | - A new extractor option has been added to the Python extractor: 798 | `python_executable_name`. You can use this option to override the default 799 | process the extractor uses to find and select a Python executable. Pass one of 800 | `--extractor-option python_executable_name=py` or `--extractor-option 801 | python_executable_name=python` or `--extractor-option 802 | python_executable_name=python3` to commands that run the extractor, for 803 | example: `codeql database create`. 804 | 805 | On Windows machines, the Python extractor will expect to find `py.exe` on the 806 | system `PATH` by default. If the Python executable has a different name, you 807 | can set the new extractor option to override this value and look for 808 | `python.exe` or `python3.exe`. 809 | 810 | For more information about using the extractor option with the CodeQL CLI, see 811 | [Extractor 812 | options](https://docs.github.com/en/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/extractor-options). 813 | 814 | ### Bugs fixed 815 | 816 | - Fixed a bug where CodeQL may produce an invalid database when it exhausts 817 | all available ID numbers. Now it detects the condition and reports an 818 | error instead. 819 | 820 | ## Release 2.16.2 (2024-02-12) 821 | 822 | - There are no user-facing changes in this release. 823 | 824 | ## Release 2.16.1 (2024-01-25) 825 | 826 | ### Improvements 827 | 828 | - When executing the `codeql database init` command, the CodeQL runner 829 | executable path is now stored in the `CODEQL_RUNNER` environment variable. 830 | Users of indirect tracing on MacOS with System Integrity Protection enabled 831 | who previously had trouble with indirect tracing should prefix their build 832 | command with this path. For example, `$CODEQL_RUNNER build.sh`. 833 | 834 | ### QL language improvements 835 | 836 | - Name clashes between weak aliases (i.e. aliases that are not final aliases of 837 | non-final entities) of the same target no longer cause ambiguity errors. 838 | 839 | ## Release 2.16.0 (2024-01-16) 840 | 841 | ### New Features 842 | 843 | - Users specifying extra tracing configurations may now use the `GetRegisteredMatchers(languageId)` Lua function to retrieve the existing table of matchers registered to a given language. 844 | 845 | ### Improvements 846 | 847 | - The `Experimental` flag has been removed from all packaging and related commands. 848 | - The RA pretty-printer omits names of internal RA nodes and pretty-prints 849 | binary unions with nested internal unions as n-ary unions. VS Code extension 850 | v1.11.0 or newer is required to compute join order badness metrics in VS Code 851 | for the new RA format. 852 | 853 | 854 | ### Potentially breaking changes 855 | 856 | - The Python extractor will no longer extract dependencies by default. See https://github.blog/changelog/2023-07-12-code-scanning-with-codeql-no-longer-installs-python-dependencies-automatically-for-new-users/ for more context. In versions until 2.17.0, it will be possible to restore the old behavior by setting `CODEQL_EXTRACTOR_PYTHON_FORCE_ENABLE_LIBRARY_EXTRACTION_UNTIL_2_17_0=1`. 857 | - The `--ram` option to `codeql database run-queries` and other 858 | commands that execute queries is now interpreted more strictly. 859 | Previously it was mostly a rough hint for how much memory to use, 860 | and the actual memory footprint of the CodeQL process could be 861 | hundreds of megabytes higher. From this release, CodeQL tries harder 862 | to keep its _total_ memory consumption during evaluation below the 863 | given limit. 864 | 865 | The new behavior yields more predictable memory use, but since it 866 | works by allocating less RAM, it can lead to more use of _disk_ 867 | storage for intermediate results compared to earlier releases with 868 | the same `--ram` value, and consequently a slight performance 869 | loss. In rare cases, for large databases, analysis may fail with a 870 | Java `OutOfMemoryError`. 871 | 872 | The cure for this is to increase `--ram` to be closer to the amount 873 | of memory actually available for CodeQL. As a rule of thumb, it will 874 | usually be possible to increase the value of `--ram` by 700 MB or 875 | more, without actually using more resources than release 2.15.x 876 | would with the old setting. An exact amount cannot stated, however, 877 | since the actual memory footprint in earlier releases depended on 878 | factors such as the size of the databases that were not fully taken 879 | into account. 880 | 881 | If you use the CodeQL Action, you do not need to do anything unless 882 | you have manually overridden the Action's RAM setting. The Action 883 | will automatically select a `--ram` setting that matches the version 884 | of the CLI it uses. 885 | 886 | ## Release 2.15.5 (2023-12-20) 887 | 888 | ### New features 889 | 890 | - A new extractor option has been added to the JavaScript/TypeScript extractor. 891 | Set the environment variable `CODEQL_EXTRACTOR_JAVASCRIPT_OPTION_SKIP_TYPES` 892 | to `true` to skip the extraction of types in TypeScript files. 893 | Use this to speed up extraction if your codebase has a high volume of 894 | TypeScript type information that causes a noticeable bottleneck for 895 | TypeScript extraction. The majority of analysis results should be preserved 896 | even when no types are extracted. 897 | 898 | ### Bugs fixed 899 | 900 | - Fixed an issue where CodeQL would sometimes incorrectly report that no files 901 | were scanned when running on Windows. 902 | This affected the human-readable summary produced by `codeql database analyze` 903 | and `codeql database interpret-results`, but did not impact the file coverage 904 | information produced in the SARIF output and displayed on the tool status page. 905 | - When analyzing Swift codebases, CodeQL build tracing will now ignore the 906 | `codesign` tool. This prevents errors in build commands or workflows on macOS 907 | that include both CodeQL and code signing. 908 | 909 | ## Release 2.15.4 (2023-12-11) 910 | 911 | ### New features 912 | 913 | - Java 21 is now fully supported, including support for new language features such as pattern switches and record patterns. 914 | 915 | ### Improvements 916 | 917 | - Parallelism in the evaluator has been improved, resulting in faster analysis when 918 | running with many threads, particularly for large databases. 919 | 920 | ## Release 2.15.3 (2023-11-22) 921 | 922 | ### New features 923 | 924 | - `codeql database analyze` now defaults to include markdown query help for all custom 925 | queries with help files available. To change the default behaviour you can pass the 926 | new flag `--sarif-include-query-help`, which provides the options `always` (which 927 | includes query help for all queries), `custom_queries_only` (the default) and `never` 928 | (which does not include query help for any query). The existing flag 929 | `--sarif-add-query-help` has been deprecated and will be removed in a future release. 930 | - The new (advanced) command-line option `--[no-]linkage-aware-import` disables the 931 | linkage-awareness phase of `codeql dataset import`, as a quick fix (at the expense of 932 | database completeness) for C++ projects where this part of database creation consumes 933 | too much memory. This option is available in the commands `database create`, 934 | `database finalize`, `database import`, `dataset import`, `test extract`, and 935 | `test run`. 936 | - The CodeQL language server now provides basic support for Rename, and you can now use 937 | the Rename Symbol functionality in Visual Studio Code for CodeQL. The current Rename 938 | support is less a refactoring tool and more a labor-saving device. You may have to 939 | perform some manual edits after using Rename, but it should still be faster and less 940 | work than renaming a symbol manually. 941 | 942 | ### Improvements 943 | 944 | - The Find References feature in the CodeQL language server now supports all CodeQL 945 | identifiers and offers improved performance compared to CodeQL CLI 2.14 releases. 946 | - The compiler generates shorter human-readable DIL and RA relation names. Due to use 947 | of an extended character set, full VS Code support for short relation names requires 948 | VS Code extension 1.9.4 or newer. 949 | - `codeql database create` and `codeql database finalize` now log more diagnostic 950 | information during database finalization, including the size of each relation, their 951 | total size, and the rate at which they were written to disk. 952 | 953 | ### Bugs fixed 954 | 955 | - Fixed an internal error in the compiler when arguments to the `codePointCount` string 956 | primitive were not bound. 957 | - Fixed a bug where `codeql database finalize` would fail if a database under construction 958 | was moved between machines between `codeql database init` and `codeql database finalize`. 959 | This should now work, as long as both commands are run by the same _release_ of the 960 | CodeQL CLI and the extractors used are the ones bundled with the CLI. 961 | - Fixed a bug where `codeql database run-queries` would fail in some circumstances when 962 | the database path included an `@`. 963 | 964 | ## Release 2.15.2 (2023-11-13) 965 | 966 | ### Breaking changes 967 | 968 | - C++ extraction has been updated to output more accurate C++ value categories. 969 | This may cause unexpected alerts on databases extracted with an up-to-date CodeQL 970 | when the queries are part of a query pack that was compiled with an earlier CodeQL. 971 | To resolve this, please recompile the query pack with the latest CodeQL. 972 | 973 | ### New features 974 | 975 | - `codeql database analyze` and `codeql database interpret-results` can now 976 | output human-readable analysis summaries in a new format. This format provides file coverage 977 | information and improves the way that diagnostic messages are displayed. The new format also includes a link to the tool status page when the `GITHUB_SERVER_URL` and `GITHUB_REPOSITORY` environment variables are set. Note that that page only exists on GitHub.com, or in GitHub Enterprise Server 978 | version 3.9.0 or later. To enable this new format, pass the `--analysis-summary-v2` flag. 979 | - CodeQL now supports 980 | distinguishing file coverage information between related languages C and C++, Java and Kotlin, 981 | and JavaScript and TypeScript. By default, file coverage information for each 982 | of these pairs of languages is grouped together. To enable specific file coverage information for these languages, pass the 983 | `--sublanguage-file-coverage` flag when initializing the database (with `codeql database create` or `codeql database init`) and when analyzing the database (with `codeql database analyze` or `codeql database interpret-results`). If you are uploading results to a GitHub instance, this flag requires GitHub.com or GitHub Enterprise Server version 3.12 or later. 984 | - All CLI commands now support `--common-caches`, which controls the location of the 985 | cached data that is persisted between several runs of the CLI, such as downloaded QL packs 986 | and compiled query plans. 987 | 988 | ### Improvements 989 | 990 | - Model packs that are used in an analysis will now be included in an output SARIF results file. All model packs now include the `isCodeQLModelPack: true` property in their tool component property bag. 991 | - The default formatting of DIL now more closely resembles equivalent QL code. 992 | 993 | ### Bugs fixed 994 | 995 | - Fixed a bug where `codeql github upload-results` would report a 403 error when attempting to upload to a GitHub Enterprise Server instance. 996 | - Fixed a bug in Python extraction where UTF-8 characters would cause 997 | logging to fail on systems with non-UTF-8 default system encoding (for example, Windows systems). 998 | - The `resolve qlpacks --kind extension` command no longer resolves 999 | extensions packs from the search path. This matches the behavior of 1000 | `resolve extensions-by-pack` and will ensure that extensions which are 1001 | resolved by `resolve qlpacks --kind extension` can also be resolved by 1002 | `resolve extensions-by-pack`. 1003 | 1004 | ## Release 2.15.1 (2023-10-19) 1005 | 1006 | ### Potentially Breaking Changes 1007 | 1008 | - The query server's `evaluation/trimCache` command was previously equivalent to the `codeql database cleanup --mode=gentle` CLI command, but is now equivalent to using `--mode=normal`. The new meaning of the command is to clear the entire evaluation cache of a database except for predicates annotated with the `cached` keyword. 1009 | 1010 | ### Deprecations 1011 | 1012 | - The accepted values of the `--mode` option for `codeql database cleanup` have been renamed to bring them in line with what they are called in the VSCode extension and the query server: 1013 | - `--mode=brutal` is now `--mode=clear`. 1014 | - `--mode=normal` is now `--mode=trim`. 1015 | - `--mode=light` is now `--mode=fit`. 1016 | - The old names are deprecated, but will be accepted for backwards-compatibility reasons until further notice. 1017 | 1018 | ### Improvements 1019 | 1020 | - The list of failed tests at the end of a `codeql test run` is now sorted lexicographically. 1021 | - The syntax of DIL now more closely resembles the QL source code that it is 1022 | compiled from. In particular, conjunctions and disjunctions now use the 1023 | familiar `and` and `or` keywords, and clauses are enclosed in curly braces. 1024 | 1025 | ### Bugs fixed 1026 | 1027 | - Fixed a bug where the `$CODEQL_JAVA_HOME` environment variable was 1028 | erroneously ignored for certain subsidiary Java processes started by 1029 | `codeql`. 1030 | - Fixed a bug in the CodeQL build tracer on Apple Silicon machines that prevented database creation if System Integrity Protection was disabled. 1031 | 1032 | ## Release 2.15.0 (2023-10-11) 1033 | 1034 | ### Deprecations 1035 | 1036 | - `pragma[assume_small_delta]` is now deprecated. The pragma has no effect and 1037 | should be removed. 1038 | 1039 | - Missing override annotations on class fields now raise errors rather than 1040 | warnings. This is to avoid confusion with the shadowing behavior in the 1041 | presence of final fields. 1042 | 1043 | - The CodeQL CLI no longer supports ML-powered alerts. For more information, 1044 | including details of our work in the AI-powered security technology space, 1045 | see 1046 | "[CodeQL code scanning deprecates ML-powered alerts](https://github.blog/changelog/2023-09-29-codeql-code-scanning-deprecates-ml-powered-alerts/)." 1047 | 1048 | ### New Features 1049 | 1050 | - The output of `codeql version --format json` now includes a `features` 1051 | property. Each key in the map identifies a feature of the CodeQL CLI. The 1052 | value for a key is always `true`. Going forward, whenever a significant new 1053 | feature is added to the CodeQL CLI, a corresponding entry will be added to the 1054 | `features` map. This is intended to make it easier for tools that invoke the 1055 | CodeQL CLI to know if the particular version of the CLI they are invoking 1056 | supports a given feature, without having to know exactly what CLI version 1057 | introduced that feature. 1058 | 1059 | ### Improvements 1060 | 1061 | - You can now specify the CodeQL languages C/C++, Java/Kotlin, and 1062 | JavaScript/TypeScript using `--language c-cpp`, `--language java-kotlin`, and 1063 | `--language javascript-typescript` respectively. These new CodeQL language 1064 | names convey more clearly what languages each CodeQL language will analyze. 1065 | 1066 | You can also reference these CodeQL languages via their secondary language 1067 | names (C/C++ via `--language c` or `--language cpp`, Java/Kotlin via 1068 | `--language java` or `--language kotlin`, and JavaScript/TypeScript via 1069 | `--language javascript` or `--language typescript`), however we recommend you 1070 | refer to them via the new primary CodeQL language names for improved clarity. 1071 | 1072 | - CodeQL now respects custom home directories set by the `$HOME` environment 1073 | variable on MacOS and Linux and `%USERPROFILE%` on Windows. When set, CodeQL 1074 | will use the variable's value to change the default location of downloaded 1075 | packages and the global compilation cache. 1076 | 1077 | - This release improves the quality of 1078 | [file coverage information](https://docs.github.com/en/code-security/code-scanning/managing-your-code-scanning-configuration/about-the-tool-status-page#using-the-tool-status-page) 1079 | for repositories that vendor their dependencies. This is currently supported 1080 | for Go and JavaScript projects. 1081 | 1082 | ### Bugs fixed 1083 | 1084 | - Fixed an issue with analyzing Python projects using Python 3.12. 1085 | 1086 | ### QL language improvements 1087 | 1088 | - The QL language now has two new methods `codePointAt` and `codePointCount` on 1089 | the `string` type. The methods both return integers and act the same as the 1090 | similarly named Java methods on strings. For example, `"abc".codePointAt(2)` 1091 | is `99` and `("a" + 128512.toUnicode() + "c").codePointAt(1)` is a `128512`. 1092 | 1093 | ## Release 2.14.6 (2023-09-26) 1094 | 1095 | ### Bugs fixed 1096 | 1097 | - The tracking of RAM usage has been improved. This fixes some cases 1098 | where CodeQL uses more RAM than requested. 1099 | 1100 | ## Release 2.14.5 (2023-09-14) 1101 | 1102 | ### Bugs fixed 1103 | 1104 | - Fixed a JavaScript extractor crash that was introduced in 2.14.4. 1105 | 1106 | ## Release 2.14.4 (2023-09-12) 1107 | 1108 | ### Potentially breaking changes 1109 | 1110 | - The CodeQL CLI no longer supports the `SEMMLE_JAVA_ARGS` environment variable. 1111 | All previous versions of the CodeQL CLI perform command substitution on the 1112 | `SEMMLE_JAVA_ARGS` value (for example, replacing `'$(echo foo)'` with `'foo'`) 1113 | when starting a new Java virtual machine, which, depending on the execution 1114 | environment, may have security implications. Users are advised to check their 1115 | environments for possible `SEMMLE_JAVA_ARGS` misuse. 1116 | 1117 | ### New Features 1118 | 1119 | - The Java extractor now supports files that use Lombok. 1120 | 1121 | ### Bugs fixed 1122 | 1123 | - `codeql database init` (and `github/codeql-action/init@v2` on GitHub Actions) 1124 | should no longer hang or crash for traced languages on 64-bit Windows machines 1125 | when certain antivirus software is installed. 1126 | - During `codeql pack create` and `codeql pack publish`, a source version of a 1127 | pack coming from `--additional-packs` can explicitly be used to override a 1128 | requested pack version even if this source version is incompatible with the 1129 | requested version in the pack file. Previously, this would fail with a 1130 | confusing error message. 1131 | - Fixed a bug where `codeql database interpret-results` hangs when a path query 1132 | produces a result that has no paths from source to sink. 1133 | 1134 | ### Miscellaneous 1135 | 1136 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1137 | CLI has been updated to version 17.0.8. 1138 | 1139 | ## Release 2.14.3 (2023-08-25) 1140 | 1141 | ### Breaking changes 1142 | 1143 | - The `.tool.extensions` property in the SARIF generated by `codeql database analyze` now contains the 1144 | following packs: 1145 | 1146 | - The containing query pack for each query that was evaluated. 1147 | - Each model pack that was specified via the `--model-packs` option, regardless of whether 1148 | that model pack affected any of the evaluated queries. 1149 | 1150 | Library packs are no longer included in the list. 1151 | 1152 | Previously, this property contained every query and library pack that was available on the search 1153 | path, regardless of whether that pack was used during the evaluation. 1154 | 1155 | ### Miscellaneous 1156 | 1157 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1158 | CLI has been updated to version 17.0.8. 1159 | 1160 | - When `codeql test` generates `.actual` files, they will in some cases 1161 | list the query predicates in a different order than past versions. 1162 | There is no need to update `.expected` files, as `codeql test` sorts 1163 | their results accordingly before diffing. 1164 | However, when there are genuine changes in expected results, the 1165 | generated `.actual` file can show additional changes against the 1166 | `.expected` due to the reordering. 1167 | 1168 | ## Release 2.14.2 (2023-08-11) 1169 | 1170 | ### Breaking changes 1171 | 1172 | - The functionality provided by the `codeql execute query-server` subcommand 1173 | has been removed. The subcommand now responds to all JSON RPC requests with 1174 | an error response. Correspondingly, this release is no longer compatible with 1175 | versions of the CodeQL extension for Visual Studio Code prior to 1.7.6. 1176 | 1177 | This change also breaks third-party CodeQL IDE integrations that still rely 1178 | on the `codeql execute query-server` subcommand. Maintainers of such CodeQL 1179 | IDE integrations should migrate to the `codeql execute query-server2` 1180 | subcommand at the earliest opportunity. 1181 | 1182 | ### Improvements 1183 | 1184 | - Switched from prefix filtering of autocomplete suggestions in the language 1185 | server to client-side filtering. This improves autocomplete suggestions in 1186 | contexts with an autocompletion prefix. 1187 | 1188 | - The CodeQL language server now checks query metadata for errors. This allows 1189 | Visual Studio Code users to see errors in their query metadata without needing 1190 | to compile the query. 1191 | 1192 | ### Bugs fixed 1193 | 1194 | - Fixed bug that made the `--warnings=hide` option do nothing in 1195 | `codeql database analyze` and other commands that _evaluate_ queries. 1196 | 1197 | ## Release 2.14.1 (2023-07-27) 1198 | 1199 | - There are no user-facing changes in this release. 1200 | 1201 | ## Release 2.14.0 (2023-07-13) 1202 | 1203 | ### Potentially breaking changes 1204 | 1205 | - The legacy option `--search-path` will now be used, if provided, when 1206 | searching for the dependencies of packages that have no lock file. 1207 | - CodeQL query packs that specify their dependencies using the legacy 1208 | `libraryPathDependencies` property in `qlpack.yml`/`codeql-pack.yml` 1209 | files are no longer permitted to contain a `codeql-pack.lock.yml` lock file. 1210 | This will lead to a compilation error. This change is intended to prevent 1211 | confusing behavior arising from a mix of legacy (unversioned) and modern 1212 | (versioned) package dependencies. To fix this error, either delete the lock 1213 | file, or convert `libraryPathDependencies` to `dependencies`. 1214 | - CodeQL CLI commands that create packages or update package lock files, such 1215 | as `codeql pack publish` and `codeql pack create`, will no longer work on 1216 | query packs that specify their dependencies using the legacy 1217 | `libraryPathDependencies` property. To fix this error, convert 1218 | `libraryPathDependencies` to `dependencies`. 1219 | 1220 | ### Deprecations 1221 | 1222 | - Missing override annotations on class member predicates now raise 1223 | errors rather than warnings. This is to avoid confusion with the 1224 | shadowing behaviour in the presence of final member predicates. 1225 | ```ql 1226 | class Foo extends Base { 1227 | final predicate foo() { ... } 1228 | 1229 | predicate bar() { ... } 1230 | } 1231 | 1232 | class Bar extends Foo { 1233 | // This method shadows Foo::foo. 1234 | predicate foo() { ... } 1235 | 1236 | // This used to override Foo::bar with a warning, now raises error. 1237 | predicate bar() { ... } 1238 | } 1239 | ``` 1240 | 1241 | ### Improvements 1242 | 1243 | - Unqualified imports can now be marked as deprecated to indicate that the 1244 | import may be removed in the future. Usage of names only reachable through 1245 | deprecated imports will generate deprecation warnings. 1246 | - Classes declared inside a parameterized modules can final extend 1247 | parameters of the module as well as types that are declared outside 1248 | the parameterized module. 1249 | - Fields are fully functional when extending types from within a module 1250 | instantiation. 1251 | - Files with a `.yaml` extension will now be included in compiled 1252 | CodeQL packs. Previously, files with this extension were excluded 1253 | even though `.yml` files were included. 1254 | - When interpreting results (e.g., using `bqrs interpret` or 1255 | `database interpret-results`), extra placeholders in alert messages are 1256 | treated as normal text. Previously, results with more placeholders than 1257 | placeholder values were skipped. 1258 | - Windows users of the CodeQL extension for VS Code will see faster start times. 1259 | - In VS Code, errors in the current file are rechecked when dependencies change. 1260 | - In VS Code, autocomplete in large QL files is now faster. 1261 | - Member predicates can shadow final member predicates of the same arity even 1262 | when the signatures are not fully matching. 1263 | 1264 | ### Bugs fixed 1265 | 1266 | - Fixed super calls on final base classes (or final aliases) so that they 1267 | are now dispatched the same way as super calls on instanceof supertypes. 1268 | - Fixed a bug where running `codeql database finalize` with a large number of 1269 | threads would fail due to running out of file descriptors. 1270 | - Fixed a bug where `codeql database create --overwrite` would not work 1271 | with database clusters. 1272 | - Fixed a bug where the CodeQL documentation coverage statistics were 1273 | incorrect. 1274 | - Fixed a bug where the generated CodeQL libarary documentation could 1275 | generate invalid uris on windows. 1276 | 1277 | ## Release 2.13.5 (2023-07-05) 1278 | 1279 | ### New Features 1280 | 1281 | - The Swift extractor now supports Swift 5.8.1. 1282 | 1283 | ## Release 2.13.4 (2023-06-19) 1284 | 1285 | ### New features 1286 | 1287 | - Temporary files and folders created by the CodeQL CLI will now be cleaned up 1288 | when each CLI command (and its internal JVM) shuts down normally. 1289 | 1290 | ### Bugs fixed 1291 | 1292 | - Fixed an issue where indirect build tracing did not work in Azure DevOps 1293 | pipeline jobs in Windows containers. To use indirect build tracing in such 1294 | environments, ensure both the `--begin-tracing` and 1295 | `--trace-process-name=CExecSvc.exe` arguments are passed to 1296 | `codeql database init`. 1297 | - Improved the error message for the `codeql pack create` command when the pack 1298 | being published has a dependency with no scope in its name. 1299 | 1300 | ## Release 2.13.3 (2023-05-31) 1301 | 1302 | ### New features 1303 | 1304 | - This release enhances our preliminary Swift support, setting the stage for the upcoming public beta. 1305 | 1306 | - The `codeql database bundle` command now supports the `--[no]-include-temp` 1307 | option. When enabled, this option will include the `temp` folder of the 1308 | database directory in the zip file of the bundled database. This folder 1309 | includes generated packages and queries, and query suites. 1310 | 1311 | - The structured log produced by `codeql generate log-summary` now includes a Boolean `isCached` 1312 | field for predicate events, where a `true` value indicates the predicate is a wrapper implementing 1313 | the `cached` annotation on another predicate. The wrapper depends on the underlying predicate that 1314 | the annotation was found on, and will usually have the same name, but it has a separate `raHash`. 1315 | 1316 | ### Bugs fixed 1317 | 1318 | - Fixed a bug that could cause the compiler to infer incorrect binding sets for 1319 | non-direct calls to overriding member predicates that have stronger binding sets 1320 | than their root definitions. 1321 | 1322 | - Fixed a bug that could have caused the compiler to incorrectly infer that a 1323 | class matched a type signature. The bug only affected classes with overriding 1324 | member predicates that had stronger binding sets than their root definitions. 1325 | 1326 | - Fixed a bug where a query could not be run from VS Code 1327 | when there were packs nested within sibling directories 1328 | of the query. 1329 | 1330 | ## Release 2.13.2 1331 | 1332 | This release was skipped. 1333 | 1334 | ## Release 2.13.1 (2023-05-03) 1335 | 1336 | ### Bugs fixed 1337 | 1338 | - Fixed a bug in `codeql database upload-results` where the subcommand 1339 | would fail with "A fatal error occurred: Invalid SARIF.", reporting 1340 | an `InvalidDefinitionException`. This issue occurred when the SARIF 1341 | file contained certain kinds of diagnostic information. 1342 | 1343 | ### Miscellaneous 1344 | 1345 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1346 | CLI has been updated to version 17.0.7. 1347 | 1348 | ## Release 2.13.0 (2023-04-20) 1349 | 1350 | ### Known issues 1351 | 1352 | - We recommend that customers using the CodeQL CLI in a third party CI 1353 | system do not upgrade to this release, due to an issue with `codeql 1354 | github upload-results`. Instead, please use CodeQL 2.12.5, or, when 1355 | available, CodeQL 2.12.7 or 2.13.1. For more information, see the 1356 | "Known issues" section for CodeQL 2.12.6. 1357 | 1358 | ### Potentially breaking changes 1359 | 1360 | - In `codeql pack add`, the dependency that is added to the `qlpack.yml` file will now allow any 1361 | version of the pack that is compatible with the specified version (`^version`) in the following 1362 | cases: 1363 | - When no version is specified (`codeql pack add codeql/cpp-all`). 1364 | - When the version is specified as `latest` (`codeql pack add codeql/cpp-all@latest`). 1365 | - When a single version is specified (`codeql pack add codeql/cpp-all@1.0.0`). 1366 | 1367 | The `^version` dependency allows any version of that pack with no breaking changes since `version`. 1368 | For example, `^1.2.3` would allow versions `1.2.3`, `1.2.5`, and `1.4.0`, but not `2.0.0`, because 1369 | changing the major version number to `2` indicates a breaking change. 1370 | 1371 | Using `^version` ensures that the added pack is not needlessly constrained to an exact version by default. 1372 | 1373 | - Upper-case variable names are no longer accepted by the QL compiler. 1374 | 1375 | Such variable names have produced a deprecation warning since 1376 | release 2.9.2 (released 2022-05-16), so QL code that compiles 1377 | without warnings with a recent release of the CLI should still work. 1378 | 1379 | ### New features 1380 | 1381 | - `codeql database analyze` and related commands now export file 1382 | coverage information by default. GHAS customers using CodeQL in 1383 | third-party CI systems will now see file coverage information on the 1384 | [tool status page](https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-the-tool-status-page) 1385 | without needing to modify their CI workflows. 1386 | 1387 | ### Deprecations 1388 | 1389 | - The possibility to omit `override` annotations on class member 1390 | predicates that override a base class predicate has been deprecated. 1391 | This is to avoid confusion with shadowing behaviour in the 1392 | presence of final member predicates. 1393 | 1394 | ```ql 1395 | class Foo extends Base { 1396 | final predicate foo() { ... } 1397 | 1398 | predicate bar() { ... } 1399 | 1400 | predicate baz() { ... } 1401 | } 1402 | 1403 | class Bar extends Foo { 1404 | // This method shadows Foo::foo. 1405 | predicate foo() { ... } 1406 | 1407 | // This used to override Foo::bar with a warning, is now deprecated. 1408 | predicate bar() { ... } 1409 | 1410 | // This correctly overrides Foo::baz 1411 | override predicate baz() { ... } 1412 | } 1413 | ``` 1414 | 1415 | ## Release 2.12.7 (2023-04-18) 1416 | 1417 | ### Bugs fixed 1418 | 1419 | - Fixed a bug in `codeql database upload-results` where the subcommand 1420 | would fail with "A fatal error occurred: Invalid SARIF.", reporting 1421 | an `InvalidDefinitionException`. This issue occurred when the SARIF 1422 | file contained certain kinds of diagnostic information. 1423 | 1424 | ## Release 2.12.6 (2023-04-04) 1425 | 1426 | ### Known issues 1427 | 1428 | - We recommend that customers using the CodeQL CLI in a third party CI 1429 | system do not upgrade to this release, due to an issue with `codeql 1430 | github upload-results`. Instead, please use CodeQL 2.12.5, or, when 1431 | available, CodeQL 2.12.7 or 2.13.1. 1432 | 1433 | This issue occurs when uploading certain kinds of diagnostic information and 1434 | causes the subcommand to fail with "A fatal error occurred: Invalid 1435 | SARIF.", reporting an `InvalidDefinitionException`. 1436 | 1437 | Customers who wish to use CodeQL 2.12.6 or 2.13.0 can 1438 | work around the problem by passing `--no-sarif-include-diagnostics` 1439 | to any invocations of `codeql database analyze` or `codeql database 1440 | interpret-results`. 1441 | 1442 | ### New features 1443 | 1444 | - Several experimental subcommands have been added in support of the 1445 | new [code scanning tool status page](https://github.blog/changelog/2023-03-28-code-scanning-shows-the-health-of-tools-enabled-on-a-repository/). 1446 | These include `codeql database add-diagnostic`, 1447 | `codeql database export-diagnostics`, and the 1448 | `codeql diagnostic add` and `codeql diagnostic export` plumbing 1449 | subcommands. 1450 | 1451 | ### Bugs fixed 1452 | 1453 | - Fixed a bug in `codeql database analyze` and related commands 1454 | where the `--max-paths` option was not respected correctly when 1455 | multiple alerts with the same primary code location were grouped together. 1456 | (This grouping is the default behavior unless the `--no-group-alerts` 1457 | option is passed.) 1458 | This bug caused some SARIF files produced by CodeQL to exceed the limits 1459 | on the number of paths (`threadFlows`) accepted by code scanning, 1460 | leading to errors when uploading results. 1461 | 1462 | ## Release 2.12.5 (2023-03-21) 1463 | 1464 | ### New features 1465 | 1466 | - The `codeql pack install` command now accepts a `--additional-packs` 1467 | option. This option takes a list of directories to search for 1468 | locally available packs when resolving which packs to install. Any 1469 | pack that is found locally through `--additional-packs` will 1470 | override any other version of a pack found in the package registry. 1471 | Locally resolved packs are not added to the lock file. 1472 | 1473 | Because the use of `--additional-packs` when running 1474 | `codeql pack install` makes running queries dependent on the local 1475 | state of the machine initially invoking `codeql pack install`, a 1476 | warning is emitted if any pack is found outside of the package 1477 | registry. This warning can be suppressed by using the 1478 | `--no-strict-mode` option. 1479 | 1480 | ### Bugs fixed 1481 | 1482 | - Fix a bug in `codeql query run` where queries whose path contain 1483 | colons cannot be run. 1484 | 1485 | ## Release 2.12.4 (2023-03-09) 1486 | 1487 | ### Breaking changes 1488 | 1489 | - The default value of the `--mode` switch to `codeql pack install` has changed. The default is now `--mode minimal-update`. 1490 | Previously, it was `use-lock`. 1491 | 1492 | ### New features 1493 | 1494 | - The per-pack compilation cache has been replaced with a global compilation cache 1495 | found within `~/.codeql`. 1496 | - `codeql pack install` now uses a new algorithm to determine which versions of 1497 | the pack's dependencies to use, based on the [PubGrub](https://nex3.medium.com/pubgrub-2fb6470504f) 1498 | algorithm. The new algorithm is able to find a solution for many cases that 1499 | the previous algorithm would fail to solve. When the new algorithm is unable 1500 | to find a valid solution, it generates a detailed error message explaining 1501 | why there is no valid solution. 1502 | - Added a new command, `codeql pack upgrade`. This command is similar to `codeql pack install`, 1503 | except that it ignores any existing lock file, installs the latest compatible version of each 1504 | dependency, and writes a new lock file. This is equivalent to `codeql pack install --mode update`. 1505 | Note that the `--mode` switch to `codeql pack install` is now deprecated. 1506 | - Added a new command, `codeql pack ci`. This command is similar to `codeql pack install`, 1507 | except if the existing lock file is missing, or if it conflicts with the version constraints in 1508 | the `qlpack.yml` file, the command generates an error. This is equivalent to 1509 | `codeql pack install --mode verify`. Note that the `--mode` switch to `codeql pack install` is now 1510 | deprecated. 1511 | 1512 | ### Deprecations 1513 | 1514 | - The `--freeze` switch for `codeql pack create`, `codeql pack bundle`, and `codeql pack publish` 1515 | is now deprecated and ignored, as there is no longer a cache within a pack. 1516 | - The `--mode update` switch to `codeql pack resolve-dependencies` is now deprecated. Instead, use 1517 | the new `--mode upgrade` switch, which has identical behavior. 1518 | - The `--mode` switch to `codeql pack install` is now deprecated. 1519 | - Instead of `--mode update`, use `codeql pack upgrade`. 1520 | - Instead of `--mode verify`, use `codeql pack ci`. 1521 | 1522 | ## Release 2.12.3 (2023-02-23) 1523 | 1524 | ### New features 1525 | 1526 | - The CodeQL compiler now produces better error messages when it is unable 1527 | to find a QL library that the query being evaluated depends on. 1528 | 1529 | ### Bugs fixed 1530 | 1531 | - Fixed a bug where the CLI would refuse to complete database creation 1532 | if the OS reports less than about 1.5 GB of physical memory. Now an 1533 | attempt will be made even on low-memory systems (but it might still 1534 | run out of memory unless there's swap space available). 1535 | 1536 | ## Release 2.12.2 (2023-02-07) 1537 | 1538 | ### Bugs fixed 1539 | 1540 | - Fixed a QL evaluator bug introduced in release 2.12.1 which could in 1541 | certain rare cases lead to wrong analysis results. 1542 | 1543 | - Fixed handling of `-Xclang ` arguments passed to the `clang` compiler which 1544 | could cause missing extractions for C++ code bases. 1545 | 1546 | - Fixed a bug where the `--overwrite` option was failing for database clusters. 1547 | 1548 | ### Miscellaneous 1549 | 1550 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1551 | CLI has been updated to version 17.0.6. 1552 | 1553 | ## Release 2.12.1 (2023-01-23) 1554 | 1555 | ### New features 1556 | 1557 | - Added a new command-line flag `--expect-discarded-cache`, which gives a hint 1558 | to the evaluator that the evaluation cache will be discarded after analysis 1559 | completes. This allows it to avoid some unnecessary writes to the cache, for 1560 | predicates that aren't needed by the query/suite being evaluated. 1561 | 1562 | ## Release 2.12.0 (2023-01-10) 1563 | 1564 | ### Breaking changes 1565 | 1566 | - The `--[no-]count-lines` option to `codeql database create` and related commands that was 1567 | deprecated in 2.11.1 has been removed. Users of this option should instead pass 1568 | `--[no-]calculate-baseline`. 1569 | 1570 | ### New features 1571 | 1572 | - Query packs created by `codeql pack create`, `codeql pack bundle`, and `codeql pack release` now 1573 | contain precompiled queries in a new format that aims to be compatible with future (and, to a 1574 | certain extent, past) releases of the CodeQL CLI. Previously the precompiled queries were in a 1575 | format specific to each CLI release, and all other releases would need to re-compile queries. 1576 | 1577 | Published packs contain precompiled queries in files with a `.qlx` extension located next to each 1578 | query's `.ql` source file. In case of differences between the `.ql` and `.qlx` files, the `.qlx` 1579 | file takes priority when evaluating queries from the command line, so if you need to modify a 1580 | published pack, be sure to delete the `.qlx` files first. 1581 | 1582 | A new `--precompile` flag to `codeql query compile` can be used to construct `*.qlx` file 1583 | explicitly, but in all usual cases it should be enough to rely on `codeql pack create` doing the 1584 | right thing. 1585 | - The `codeql database init` command now accepts a PAT that allows you to download queries from 1586 | external, private repositories when using the `--codescanning-config ` option. For 1587 | example, you can specify the following queries block in the config file, which will checkout the main 1588 | branch of the `codeql-test/my-private-repository` repository and evaluate any queries found in that 1589 | repository: 1590 | 1591 | ```yaml 1592 | queries: 1593 | - codeql-test/my-private-repository@main 1594 | ``` 1595 | 1596 | If the repository is private, you can add a `--external-repository-token-stdin` option and supply a 1597 | PAT with appropriate permissions via standard input. For more information on queries and external 1598 | repositories in Code Scanning, see [Using queries in QL packs](https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs). 1599 | - The baseline information produced by `codeql database init` and 1600 | `codeql database create` now accounts for 1601 | [`paths` and `paths-ignore` configuration](https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#specifying-directories-to-scan). 1602 | - In the VS Code extension, recursive calls will be marked with inlay 1603 | hints. These can be disabled with the global inlay hints setting 1604 | (`editor.inlayHints.enabled`). If you just want to disable them for 1605 | codeql the settings can be scoped to just codeql files (language id is `ql`). 1606 | See [Language Specific Editor Settings](https://code.visualstudio.com/docs/getstarted/settings#_language-specific-editor-settings) 1607 | in the VS Code documentation for more information. 1608 | - The CLI now gives a more helpful error message when asked to run queries on a 1609 | database that has not been finalized. 1610 | 1611 | ### Bugs fixed 1612 | 1613 | - Fixed a bug where the `codeql pack install` command would fail if 1614 | a [CodeQL configuration file](https://codeql.github.com/docs/codeql-cli/specifying-command-options-in-a-codeql-configuration-file/#using-a-codeql-configuration-file) 1615 | is used and the `--additional-packs` option is specified. 1616 | 1617 | ## Release 2.11.6 (2022-12-13) 1618 | 1619 | ### Breaking changes 1620 | 1621 | - Java and Kotlin analyses in this release of the CLI and all earlier 1622 | releases are incompatible with Kotlin 1.7.30 and later. To prevent 1623 | code scanning alerts being spuriously dismissed, Java and Kotlin 1624 | analyses will now fail when using Kotlin 1.7.30 or later. 1625 | 1626 | If you are unable to use Kotlin 1.7.29 or earlier, you can disable 1627 | Kotlin support by setting 1628 | `CODEQL_EXTRACTOR_JAVA_AGENT_DISABLE_KOTLIN` to `true` in the 1629 | environment. 1630 | 1631 | ### Bugs fixed 1632 | 1633 | - Fixed a bug where it was not possible to run queries in CodeQL query 1634 | packs for C# that use the legacy `libraryPathDependencies` property 1635 | in their `qlpack.yml` file. The associated error message complained 1636 | about undefined extensional predicates. 1637 | 1638 | ## Release 2.11.5 (2022-12-07) 1639 | 1640 | ### Bugs Fixed 1641 | 1642 | - Fixed a bug that could cause log summary generation to fail in vscode. 1643 | 1644 | ## Release 2.11.4 (2022-11-24) 1645 | 1646 | ### New features 1647 | 1648 | - Kotlin support is now in beta. This means that Java analyses will also 1649 | include Kotlin code by default. Kotlin support can be disabled by 1650 | setting `CODEQL_EXTRACTOR_JAVA_AGENT_DISABLE_KOTLIN` to `true` in 1651 | the environment. 1652 | 1653 | ### Potentially breaking changes 1654 | 1655 | - CodeQL 2.11.1 to 2.11.3 contained a bug in [indirect build tracing](https://codeql.github.com/docs/codeql-cli/creating-codeql-databases/#using-indirect-build-tracing) on Windows 1656 | when using `codeql database init` with the [`--trace-process-level`](https://codeql.github.com/docs/codeql-cli/manual/database-init/#cmdoption-codeql-database-init-trace-process-level) flag. 1657 | In these versions, when `--trace-process-level` was set to a value greater than zero, 1658 | (or left at the default value of 1), CodeQL attempted to inject its build tracer 1659 | at a higher level in the process tree than the requested process level. 1660 | This could lead to errors of the form "No source code found" or 1661 | "Process tree ended before reaching required level". 1662 | From 2.11.4 onwards, the CodeQL build tracer is injected at the requested process level. 1663 | 1664 | ### Deprecations 1665 | 1666 | - The `--[no-]fast-compilation` option to `codeql test run` is now 1667 | deprecated. 1668 | 1669 | ## Release 2.11.3 (2022-11-11) 1670 | 1671 | ### Breaking changes 1672 | 1673 | - The `codeql pack ls --format json` deep plumbing command now returns 1674 | only the `name` and `version` properties for each found pack. 1675 | 1676 | ### Potentially breaking changes 1677 | 1678 | - `codeql pack download`, `codeql pack install`, and `codeql pack add` 1679 | will ignore CodeQL packs with pre-release versions, unless the 1680 | `--allow-prerelease` option is passed to the command. This brings 1681 | these commands into alignment with `codeql pack publish` that will 1682 | avoid publishing CodeQL packs with pre-release versions unless the 1683 | `--allow-prerelease` option is specified. Pre-release versions have 1684 | the following format: `X.Y.Z-qualifier` where `X`, `Y`, and `Z` are 1685 | respectively the major, minor, and patch number. `qualifier` is the 1686 | pre-release version. For more information about pre-releases, see 1687 | the 1688 | [Semantic Versioning specification](https://semver.org/#spec-item-9). 1689 | 1690 | ### Deprecations 1691 | 1692 | - The `--[no-]fast-compilation` option to `codeql query compile` is 1693 | now deprecated. 1694 | 1695 | ### New features 1696 | 1697 | - `codeql resolve files` and `codeql database index-files` have a new 1698 | `--find-any` option, which finds at most one match. 1699 | 1700 | ### Miscellaneous 1701 | 1702 | - The build of Apache Commons Text that is bundled with the CodeQL CLI 1703 | has been updated to version 1.10.0. While previous releases shipped 1704 | with version 1.6 of the library, no part of the CodeQL CLI 1705 | references the `StringSubstitutor` class that the recently disclosed 1706 | [CVE-2022-42889](https://github.com/advisories/GHSA-599f-7c49-w659) 1707 | vulnerability applies to. We therefore do not believe that running 1708 | previous releases of CodeQL exposes users to this vulnerability. 1709 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1710 | CLI has been updated to version 17.0.5. 1711 | 1712 | ## Release 2.11.2 (2022-10-25) 1713 | 1714 | ### Breaking changes 1715 | 1716 | - Bundling and publishing a CodeQL pack will no longer include nested 1717 | CodeQL packs. If you want to include a nested pack in your published pack, 1718 | then you must explicitly include it using the `include` property in the 1719 | top-level `qlpack.yml` file. 1720 | 1721 | For example, if your package structure looks like this: 1722 | 1723 | ```text 1724 | qlpack.yml 1725 | nested-pack 1726 | ∟ qlpack.yml 1727 | query.ql 1728 | ``` 1729 | 1730 | then the contents of `nested-pack` will not be included by default within 1731 | the published package. To include `nested-pack`, add an entry like this 1732 | to the top level `qlpack.yml` file: 1733 | 1734 | ```yml 1735 | include: 1736 | - nested-pack/** 1737 | ``` 1738 | 1739 | ### Bugs fixed 1740 | 1741 | - Using the `--codescanning-config=` option in 1742 | `codeql database init` will now correctly process the `paths` and 1743 | `pathsIgnore` properties of the configuration file in a way that is 1744 | identical to the behavior of the `codeql-action`. Previously, `paths` 1745 | or `pathsIgnore` entries that end in `/**` or start with `/` were 1746 | incorrectly rejected by the CLI. 1747 | 1748 | - Fixed a bug where the `--compilation-cache` option to 1749 | `codeql pack publish` and `codeql pack create` was being ignored 1750 | when creating a query pack. Now, the indicated cache is used 1751 | when pre-compiling the queries in it. 1752 | 1753 | - Fixed a bug that would make the "Show DIL" command in the VSCode 1754 | extension display nothing. 1755 | 1756 | ### Other changes 1757 | 1758 | - Emit a detailed warning if package resolution fails, the legacy 1759 | `--search-path` option is provided, _and_ there is at least one 1760 | referenced pack that does not use legacy package resolution. 1761 | In this case, `--additional-packs` should be used to extend the 1762 | search to additional directories, instead of `--search-path`. 1763 | 1764 | ## Release 2.11.1 (2022-10-11) 1765 | 1766 | ### Breaking changes 1767 | 1768 | - Pack installation using the CodeQL Packaging beta will now fail if a 1769 | compatible version cannot be found. This replaces the previous 1770 | behavior where `codeql pack download` and related commands would 1771 | instead install the latest version of the pack in this situation. 1772 | 1773 | ### Deprecations 1774 | 1775 | - The `--[no-]count-lines` option to `codeql database create` and 1776 | related commands is now deprecated and will be removed in a future 1777 | release of the CodeQL CLI (earliest 2.12.0). It is replaced by 1778 | `--[no-]calculate-baseline` to reflect the additional baseline 1779 | information that is now captured as of this release. 1780 | 1781 | ### New features 1782 | 1783 | - Subcommands that compile QL accept a new `--no-release-compatibility` 1784 | option. It does nothing for now, but in the future it will be used 1785 | to control a trade-off between query performance and compatibility 1786 | with older/newer releases of the QL evaluator. 1787 | - `codeql database analyze` and related commands now support absolute 1788 | paths containing the `@` or `:` characters when specifying which queries 1789 | to run. To reference a query file, directory, or suite whose path contains 1790 | a literal `@` or `:`, prefix the query specifier with `path:`, for example: 1791 | ```shell 1792 | codeql database analyze --format=sarif-latest --output=results path:C:/Users/ci/workspace@2/security/query.ql 1793 | ``` 1794 | ### Bugs fixed 1795 | 1796 | - It is no longer an error to call `codeql pack create ` with a `` 1797 | option pointing to a file name. The CLI will walk up the directory tree and 1798 | run the command in the first directory containing the `qlpack.yml` or `codeql-pack.yml` file. 1799 | - Fixed a concurrency error observed when using `codeql database import` or 1800 | `codeql database finalize` with multiple threads and multiple additional 1801 | databases on a C++ codebase. 1802 | 1803 | ## Release 2.11.0 (2022-09-28) 1804 | 1805 | ### Deprecation 1806 | 1807 | - The CodeQL CLI now uses Python 3 to extract both Python 2 and Python 3 1808 | databases. Correspondingly, support for using Python 2 to extract 1809 | Python databases is now deprecated. Starting with version 2.11.3, you 1810 | will need to install Python 3 to extract Python databases. 1811 | 1812 | ### Miscellaneous 1813 | 1814 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1815 | CLI has been updated to version 17.0.4. 1816 | 1817 | ## Release 2.10.5 (2022-09-13) 1818 | 1819 | ### New features 1820 | 1821 | - You can now define which registries should be used for downloading and publishing CodeQL packs on a 1822 | per-workspace basis by creating a `codeql-workspace.yml` file and adding a `registries` block. For 1823 | more infomation, see [About CodeQL Workspaces](https://codeql.github.com/docs/codeql-cli/about-codeql-workspaces/). 1824 | 1825 | ## Release 2.10.4 (2022-08-31) 1826 | 1827 | - The bundled extractors are updated to match the versions currently 1828 | used on LGTM.com. These are newer than the last release (1.30) of 1829 | LGTM Enterprise. If you plan to upload databases to an LGTM 1830 | Enterprise 1.30 instance, you need to create them with release 1831 | 2.7.6. 1832 | 1833 | - This release does not include any user-facing changes. 1834 | 1835 | ## Release 2.10.3 (2022-08-15) 1836 | 1837 | - The bundled extractors are updated to match the versions currently 1838 | used on LGTM.com. These are newer than the last release (1.30) of 1839 | LGTM Enterprise. If you plan to upload databases to an LGTM 1840 | Enterprise 1.30 instance, you need to create them with release 1841 | 2.7.6. 1842 | 1843 | ### New features 1844 | 1845 | - When called with `--start-tracing`, the `codeql database init` command 1846 | now accepts extractor options for the indirect tracing environment via 1847 | `--extractor-option`. Users should continue to specify extractor options 1848 | for direct tracing environments by passing them to 1849 | `codeql database trace-command` invocations. 1850 | 1851 | ### Other changes 1852 | 1853 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1854 | CLI has been updated to version 17.0.4. 1855 | 1856 | ## Release 2.10.2 (2022-08-02) 1857 | 1858 | - The bundled extractors are updated to match the versions currently 1859 | used on LGTM.com. These are newer than the last release (1.30) of 1860 | LGTM Enterprise. If you plan to upload databases to an LGTM 1861 | Enterprise 1.30 instance, you need to create them with release 1862 | 2.7.6. 1863 | 1864 | ### Breaking change 1865 | 1866 | - The option `--compiler-spec` to `codeql database create` (and 1867 | `codeql database trace-command`) no longer works. It is replaced by 1868 | `--extra-tracing-config`, which accepts a tracer configuration file 1869 | in the new, Lua-based tracer configuration format instead. See 1870 | `tools/tracer/base.lua` for the precise API available. If you need 1871 | help help porting your existing compiler specification files, please 1872 | file a public issue in https://github.com/github/codeql-cli-binaries, 1873 | or open a private ticket with GitHub support and request an 1874 | escalation to engineering. 1875 | 1876 | ### Potentially breaking changes 1877 | 1878 | - Versions of the CodeQL extension for Visual Studio Code released 1879 | before February 2021 may not work correctly with this CLI, in 1880 | particular if database upgrades are necessary. We recommend keeping 1881 | your VS Code extension up-to-date. 1882 | 1883 | ### Deprecation 1884 | 1885 | - The experimental `codeql resolve ml-models` command has been 1886 | deprecated. Advanced users calling this command should use the new 1887 | `codeql resolve extensions` command instead. 1888 | 1889 | ### New features 1890 | 1891 | - The `codeql github upload-results` command now supports a `--merge` 1892 | option. If this option is provided, the command will accept the paths 1893 | to multiple SARIF files, and will merge those files before uploading 1894 | them as a single analysis. This option is recommended _only_ for 1895 | backwards compatibility with old analyses produced by the CodeQL 1896 | Runner, which combined the results for multiple languages into a 1897 | single analysis. 1898 | 1899 | ## Release 2.10.1 (2022-07-19) 1900 | 1901 | - The bundled extractors are updated to match the versions currently 1902 | used on LGTM.com. These are newer than the last release (1.30) of 1903 | LGTM Enterprise. If you plan to upload databases to an LGTM 1904 | Enterprise 1.30 instance, you need to create them with release 1905 | 2.7.6. 1906 | 1907 | ### New features 1908 | 1909 | - Improved error message from `codeql database analyze` when a query is 1910 | missing `@id` or `@kind` query metadata. 1911 | 1912 | ## Release 2.10.0 (2022-06-27) 1913 | 1914 | - The bundled extractors are updated to match the versions currently 1915 | used on LGTM.com. These are newer than the last release (1.30) of 1916 | LGTM Enterprise. If you plan to upload databases to an LGTM 1917 | Enterprise 1.30 instance, you need to create them with release 1918 | 2.7.6. 1919 | 1920 | ### Breaking changes 1921 | 1922 | - The `--format=stats` option of `codeql generate log-summary` has been 1923 | renamed to `--format=overall`. It now produces a richer JSON object 1924 | that, in addition to the previous statistics about the run (which can 1925 | be found in the `stats` property) also records the most expensive 1926 | predicates in the evaluation run. 1927 | 1928 | ### Potentially breaking changes 1929 | 1930 | - The `codeql resolve ml-model` command now requires one or more query 1931 | specifications as command line arguments in order to determine the set 1932 | of starting packs from which to initiate the resolution process. The 1933 | command will locate all ML models in any qlpack that is a transitive 1934 | dependency of any of the starting packs. Also, the output of the 1935 | command has been expanded to include for each model the containing 1936 | package's name, version, and path. 1937 | 1938 | - The `buildMetadata` inside of compiled CodeQL packs no longer contains 1939 | a `creationTime` property. This was removed in order to ensure that 1940 | the content of a CodeQL pack is identical when it is re-compiled. 1941 | 1942 | - The `codeql pack download` command, when used with the `--dir` option, 1943 | now downloads requested packs in directories corresponding to their 1944 | version numbers. Previously, 1945 | `codeql pack download --dir ./somewhere codeql/java-queries@0.1.2` 1946 | would download the pack into the `./somewhere/codeql/java-queries` 1947 | directory. Now, it will download the pack into the 1948 | `./somewhere/codeql/java-queries/0.1.2` directory. This allows you to 1949 | download multiple versions of the same pack using a single command. 1950 | 1951 | ### New features 1952 | 1953 | - You can now include diagnostic messages in the summary produced by 1954 | the `--print-diagnostics-summary` option of the 1955 | `codeql database interpret-results` and `codeql database analyze` 1956 | commands by running these commands at high verbosity levels. 1957 | 1958 | ### Bugs fixed 1959 | 1960 | - Fixed a bug where `codeql pack download`, when used with the `--dir` 1961 | option, would not download a pack that is in the global package cache. 1962 | 1963 | - Fixed a bug where some versions of a CodeQL package could not be 1964 | downloaded if there are more than 100 versions of this package in the 1965 | package registry. 1966 | 1967 | - Fixed a bug where the `--also-match` option for `codeql resolve files` 1968 | and `codeql database index-files` does not work with relative paths. 1969 | 1970 | - Fixed a bug that caused `codeql query decompile` to ignore the 1971 | `--output` option when producing bytecode output (`--kind=bytecode`), 1972 | writing only to `stdout`. 1973 | 1974 | ## Release 2.9.4 (2022-06-20) 1975 | 1976 | - The bundled extractors are updated to match the versions currently 1977 | used on LGTM.com. These are newer than the last release (1.30) of 1978 | LGTM Enterprise. If you plan to upload databases to an LGTM 1979 | Enterprise 1.30 instance, you need to create them with release 1980 | 2.7.6. 1981 | 1982 | ### New features 1983 | 1984 | - Users of CodeQL Packaging Beta can now optionally authenticate to 1985 | Container registries on GitHub Enterprise Server (GHES) versions 3.6 1986 | and later using standard input instead of the `CODEQL_REGISTRIES_AUTH` 1987 | environment variable. To authenticate via standard input, pass 1988 | `--registries-auth-stdin`. The value you provide will override the 1989 | value of the `CODEQL_REGISTRIES_AUTH` environment variable. 1990 | 1991 | ## Release 2.9.3 (2022-05-31) 1992 | 1993 | - The bundled extractors are updated to match the versions currently 1994 | used on LGTM.com. These are newer than the last release (1.30) of 1995 | LGTM Enterprise. If you plan to upload databases to an LGTM 1996 | Enterprise 1.30 instance, you need to create them with release 1997 | 2.7.6. 1998 | 1999 | ### New features 2000 | 2001 | - Users can now use CodeQL Packaging Beta to publish and download 2002 | CodeQL packs on GitHub Enterprise Server (GHES) versions 3.6 and 2003 | later. 2004 | 2005 | To authenticate to a package registry on GHES 3.6+, first create a 2006 | `~/.codeql/qlconfig.yml` file. For example, the following file 2007 | specifies that all CodeQL packages should be uploaded to the GHES 2008 | instance with the hostname `GHE_HOSTNAME`: 2009 | 2010 | ```yml 2011 | registries: 2012 | - packages: '*' 2013 | url: https://containers.GHE_HOSTNAME/v2/ 2014 | ``` 2015 | 2016 | You can now download public packages from GHES using 2017 | `codeql pack download`. 2018 | 2019 | To publish any package or download private packages, authenticate to 2020 | GHES by specifying registry/token pairs in the 2021 | `CODEQL_REGISTRIES_AUTH` environment variable. You can authenticate 2022 | using either a GitHub Apps token or a personal access token. For 2023 | example, 2024 | `https://containers.GHEHOSTNAME1/v2/=TOKEN1,https://containers.GHEHOSTNAME2/v2/=TOKEN2` 2025 | will authenticate the CLI to the `GHEHOSTNAME1` and `GHEHOSTNAME2` 2026 | GHES instances. 2027 | 2028 | ### Bugs Fixed 2029 | 2030 | - Fixed a bug where precompiled CodeQL packages in the CodeQL bundle were 2031 | being recompiled if they were in a read-only directory. 2032 | 2033 | - Fixed a bug where new versions of the VS Code extension wouldn't run two 2034 | queries in parallel against one database. 2035 | 2036 | ## Release 2.9.2 (2022-05-16) 2037 | 2038 | - The bundled extractors are updated to match the versions currently 2039 | used on LGTM.com. These are newer than the last release (1.30) of 2040 | LGTM Enterprise. If you plan to upload databases to an LGTM 2041 | Enterprise 1.30 instance, you need to create them with release 2042 | 2.7.6. 2043 | 2044 | ### Features removed 2045 | 2046 | - The table printed by `codeql database analyze` to summarize the 2047 | results of metric queries that were part of the analysis now reports 2048 | a single row per metric name independently of the verbosity level of 2049 | the command. Previously, at higher verbosity levels, this table 2050 | would contain multiple rows for metric names with multiple values. 2051 | 2052 | ### New features 2053 | 2054 | - The tables produced by `codeql database analyze` summarizing the 2055 | results of any diagnostic and metric queries that were run now 2056 | exclude the results of queries tagged `telemetry`. 2057 | 2058 | - Uploading SARIF results using the `codeql github upload-results` 2059 | command now has a timeout of 5 minutes. 2060 | 2061 | - Downloading CodeQL packs using the `codeql pack download`, 2062 | `codeql pack install` and related commands now have a timeout of 2063 | 5 minutes and will retry 3 times before failing. Similar behavior 2064 | has been added to the `codeql pack publish` command. 2065 | 2066 | - The `codeql generate log-summary` command will now print progress 2067 | updates to `stderr`. 2068 | 2069 | ### Bugs fixed 2070 | 2071 | - Fixed a bug that could make it unpredictable whether the QL compiler 2072 | reports problems about query metadata tags, and thereby make `codeql 2073 | test run` fail spuriously in some cases. 2074 | 2075 | ## Release 2.9.1 (2022-05-05) 2076 | 2077 | - The bundled extractors are updated to match the versions currently 2078 | used on LGTM.com. These are newer than the last release (1.30) of 2079 | LGTM Enterprise. If you plan to upload databases to an LGTM 2080 | Enterprise 1.30 instance, you need to create them with release 2081 | 2.7.6. 2082 | 2083 | ## Release 2.9.0 (2022-04-26) 2084 | 2085 | - The bundled extractors are updated to match the versions currently 2086 | used on LGTM.com. These are newer than the last release (1.30) of 2087 | LGTM Enterprise. If you plan to upload databases to an LGTM 2088 | Enterprise 1.30 instance, you need to create them with release 2089 | 2.7.6. 2090 | 2091 | ### New features 2092 | 2093 | - `codeql database create` now supports the `--[no-]-count-lines` 2094 | option, which was previously only available with `codeql database 2095 | init`. 2096 | 2097 | - `codeql resolve files` and `codeql database index-files` has a new 2098 | `--also-match` option, which allows users to specify glob patterns 2099 | that are applied in conjunction with the existing `--include` 2100 | option. 2101 | 2102 | ### New language features 2103 | 2104 | - This release introduces experimental support for parameterized QL 2105 | modules. This language feature is still subject to change and should 2106 | not be used in production yet. 2107 | 2108 | ### Bugs fixed 2109 | 2110 | - Fixed a bug that would prevent resolution of a query suite in a 2111 | published CodeQL query pack that has a reference to the pack itself. 2112 | 2113 | - Fixed inaccurate documentation of what the `--include-extension` 2114 | option to `codeql resolve files` and `codeql database index-files` 2115 | does. The actual behavior is unchanged. 2116 | 2117 | ## Release 2.8.5 (2022-04-07) 2118 | 2119 | - The bundled extractors are updated to match the versions currently 2120 | used on LGTM.com. These are newer than the last release (1.30) of 2121 | LGTM Enterprise. If you plan to upload databases to an LGTM 2122 | Enterprise 1.30 instance, you need to create them with release 2123 | 2.7.6. 2124 | 2125 | - There are no user-facing changes in this release. 2126 | 2127 | ## Release 2.8.4 (2022-03-29) 2128 | 2129 | - The bundled extractors are updated to match the versions currently 2130 | used on LGTM.com. These are newer than the last release (1.29) of 2131 | LGTM Enterprise. If you plan to upload databases to an LGTM 2132 | Enterprise 1.29 instance, you need to create them with release 2133 | 2.6.3. 2134 | 2135 | ### Bugs fixed 2136 | 2137 | - Fixed an error where running out of memory during query evaluation 2138 | would cause `codeql` to exit with status 34 instead of the 99 that 2139 | is documented for this condition. 2140 | 2141 | - Fixed a bug in our handling of Clang's header maps, which caused 2142 | missing files for Xcode-based projects on macOS (e.g. WebKit). 2143 | 2144 | ## Release 2.8.3 (2022-03-14) 2145 | 2146 | - This release of CodeQL (and all future ones) will not include the 2147 | CodeQL runner, which is now deprecated. For more information, and 2148 | instructions on how to migrate to using the CodeQL CLI, see 2149 | [CodeQL runner deprecation][11]. 2150 | 2151 | [11]: https://github.blog/changelog/2021-09-21-codeql-runner-deprecation/ 2152 | 2153 | - The bundled extractors are updated to match the versions currently 2154 | used on LGTM.com. These are newer than the last release (1.29) of 2155 | LGTM Enterprise. If you plan to upload databases to an LGTM 2156 | Enterprise 1.29 instance, you need to create them with release 2157 | 2.6.3. 2158 | 2159 | ### New features 2160 | 2161 | - Executable binaries for Windows are now digitally signed by a GitHub 2162 | certificate. 2163 | 2164 | ### Other changes 2165 | 2166 | - The evaluator logs produced by `--evaluator-log` now default to the 2167 | maximum verbosity level and will therefore contain more information 2168 | (and, accordingly, grow larger). The verbosity level can still be 2169 | configured with `--evaluator-log-level`. In particular, 2170 | `--evaluator-log-level=1` will restore the previous default behavior. 2171 | 2172 | ## Release 2.8.2 (2022-02-28) 2173 | 2174 | - The bundled extractors are updated to match the versions currently 2175 | used on LGTM.com. These are newer than the last release (1.29) of 2176 | LGTM Enterprise. If you plan to upload databases to an LGTM 2177 | Enterprise 1.29 instance, you need to create them with release 2178 | 2.6.3. 2179 | 2180 | ### Breaking change 2181 | 2182 | - The support for the output formats SARIF v1.0.0 and SARIF v2.0.0 2183 | (Committee Specification Draft 1) that were deprecated in 2.7.1 has 2184 | been removed. If you need this functionality, please file a public 2185 | issue against https://github.com/github/codeql-cli-binaries, or open 2186 | a private ticket with GitHub Support and request an escalation to 2187 | engineering. 2188 | 2189 | ### New Features 2190 | 2191 | - The CodeQL CLI is now compatible with Windows 11 and Windows Server 2192 | 2022, including building databases for compiled languages. 2193 | 2194 | ## Release 2.8.1 (2022-02-15) 2195 | 2196 | - The bundled extractors are updated to match the versions currently 2197 | used on LGTM.com. These are newer than the last release (1.29) of 2198 | LGTM Enterprise. If you plan to upload databases to an LGTM 2199 | Enterprise 1.29 instance, you need to create them with release 2200 | 2.6.3. 2201 | 2202 | ### New Features 2203 | 2204 | - Commands that find or run queries now allow you to refer to queries 2205 | within a named CodeQL pack. For example: 2206 | 2207 | ```sh 2208 | # Analyze a database using all queries in the experimental/Security folder within the codeql/cpp-queries 2209 | # CodeQL query pack. 2210 | codeql database analyze --format=sarif-latest --output=results \ 2211 | codeql/cpp-queries:experimental/Security 2212 | 2213 | # Analyse using only the RedundantNullCheckParam.ql query in the codeql/cpp-queries CodeQL query pack. 2214 | codeql database analyze --format=sarif-latest --output=results \ 2215 | 'codeql/cpp-queries:experimental/Likely Bugs/RedundantNullCheckParam.ql' 2216 | 2217 | # Analyse using the cpp-security-and-quality.qls query suite in the codeql/cpp-queries CodeQL query pack. 2218 | codeql database analyze --format=sarif-latest --output=results \ 2219 | 'codeql/cpp-queries:codeql-suites/cpp-security-and-quality.qls' 2220 | 2221 | # Analyse using the cpp-security-and-quality.qls query suite from a version of the codeql/cpp-queries pack 2222 | # that is >= 0.0.3 and < 0.1.0 (the highest compatible version will be chosen). 2223 | # All valid semver ranges are allowed. See https://docs.npmjs.com/cli/v6/using-npm/semver#ranges 2224 | codeql database analyze --format=sarif-latest --output=results \ 2225 | 'codeql/cpp-queries@~0.0.3:codeql-suites/cpp-security-and-quality.qls' 2226 | ``` 2227 | 2228 | The complete way to specify a set of queries is in the form 2229 | `scope/name@range:path`, where: 2230 | 2231 | - `scope/name` is the qualified name of a CodeQL pack. 2232 | - `range` is a [semver range][10]. 2233 | - `path` is a file system path 2234 | 2235 | If a `scope/name` is specified, the `range` and `path` are 2236 | optional. A missing `range` implies the latest version of the 2237 | specified pack. A missing `path` implies the default query suite 2238 | of the specified pack. 2239 | 2240 | The `path` can be one of a `*.ql` query file, a directory 2241 | containing one or more queries, or a `.qls` query suite file. If 2242 | there is no pack name specified, then a `path` must be provided, 2243 | and will be interpreted relative to the current working directory 2244 | of the current process. 2245 | 2246 | If a `scope/name` and `path` are specified, then the `path` cannot 2247 | be absolute. It is considered relative to the root of the CodeQL 2248 | pack. 2249 | 2250 | The relevant commands are: 2251 | - `codeql database analyze` 2252 | - `codeql database run-queries` 2253 | - `codeql execute queries` 2254 | - `codeql resolve queries` 2255 | 2256 | [10]: https://docs.npmjs.com/cli/v6/using-npm/semver#ranges 2257 | 2258 | ### Bugs fixed 2259 | 2260 | - Fixed a bug that would sometimes lead to query evaluation on 2261 | M1-based Macs to crash with `Did not preallocate enough memory` 2262 | error. 2263 | 2264 | ## Release 2.8.0 (2022-02-04) 2265 | 2266 | - The bundled extractors are updated to match the versions currently 2267 | used on LGTM.com. These are newer than the last release (1.29) of 2268 | LGTM Enterprise. If you plan to upload databases to an LGTM 2269 | Enterprise 1.29 instance, you need to create them with release 2270 | 2.6.3. 2271 | 2272 | ### Breaking change 2273 | 2274 | - The CodeQL Action versions up to and including version 1.0.22 are 2275 | not compatible with the CodeQL CLI 2.8.0 and later. The CLI 2276 | will emit an error if it detects that it is being used by an 2277 | incompatible version of the codeql-action. 2278 | 2279 | ### New features 2280 | 2281 | - A new extractor option has been added to the Java extractor. The 2282 | flag `--extractor-option exclude=''` allows specifying a glob 2283 | that describes which paths need to be excluded from extraction but 2284 | still need to be compiled. This is useful when some files are necessary 2285 | for a successful build but are uninteresting for analysis. 2286 | 2287 | See also: https://codeql.github.com/docs/codeql-cli/extractor-options/ 2288 | 2289 | - Summary metrics can now associate messages with their results, for 2290 | instance to report the name and number of uses of a particular API 2291 | endpoint within a repository. To associate messages with summary 2292 | metrics, define a query with `@kind metric` and `@tags summary` metadata 2293 | and use either the `location, message, value` or the `message, value` 2294 | results pattern. 2295 | 2296 | ### Bug fixed 2297 | 2298 | - Fixed a bug where `codeql resolve upgrades` ignores the 2299 | `--target-dbscheme` option. 2300 | 2301 | ## Release 2.7.6 (2022-01-24) 2302 | 2303 | - The bundled extractors are updated to match the versions currently 2304 | used on LGTM.com. These are newer than the last release (1.28) of 2305 | LGTM Enterprise. If you plan to upload databases to an LGTM 2306 | Enterprise 1.28 instance, you need to create them with release 2307 | 2.5.9. 2308 | 2309 | ### Bug fixed 2310 | 2311 | - A bug where creation of a CodeQL database could sometimes fail with 2312 | a `NegativeArraySizeException` has now been fixed. 2313 | 2314 | ### New feature 2315 | 2316 | - The CLI and evaluator contain a number of new features in support of 2317 | internal machine learning experiments. This includes an experimental 2318 | `resolve ml-models` subcommand and new `mlModels` metadata in pack 2319 | definition files. As these new features are not yet ready for general 2320 | use, they should be ignored by external CodeQL users. 2321 | 2322 | ## Release 2.7.5 (2022-01-17) 2323 | 2324 | - The bundled extractors are updated to match the versions currently 2325 | used on LGTM.com. These are newer than the last release (1.28) of 2326 | LGTM Enterprise. If you plan to upload databases to an LGTM 2327 | Enterprise 1.28 instance, you need to create them with release 2328 | 2.5.9. 2329 | 2330 | ### Deprecation 2331 | 2332 | - The CodeQL Action versions up to and including version 1.0.22 are 2333 | now deprecated for use with CodeQL CLI 2.7.5 and later. The CLI 2334 | will emit a warning if it detects that it is being used by a 2335 | deprecated version of the codeql-action. This warning will become a 2336 | fatal error with version 2.8.0 of the CLI. 2337 | 2338 | ### New feature 2339 | 2340 | - The `codeql github upload-results` command will now print the API 2341 | response body in JSON format if a `--format=json` flag is 2342 | given. Otherwise the command will print the URL of the SARIF 2343 | upload. This URL can be used to get status information for the 2344 | upload. 2345 | 2346 | See also: https://docs.github.com/en/rest/reference/code-scanning 2347 | 2348 | ### Documentation fixes 2349 | 2350 | - The documentation for the `--trace-process-level` flag of `codeql 2351 | database init` (which is used with indirect build tracing on 2352 | Windows) was erroneous. 2353 | 2354 | The help text previously claimed that `--trace-process-level=1` 2355 | would inject CodeQL's build tracer into the calling process. This is 2356 | actually what `--trace-process-level=0` achieves. The help text has 2357 | now been corrected to match the actual (unchanged) behavior. 2358 | 2359 | Also, some log messages incorrectly stated which process CodeQL was 2360 | injected into. These have also been corrected. 2361 | 2362 | ### Other changes 2363 | 2364 | - For commands that run queries, the `--timeout` option now controls 2365 | the maximal time it may take to evaluate a "layer" of a query rather 2366 | than a "stage". There are usually many "layers" in each "stage", 2367 | but it is usually a single one of the layers in a stage that uses 2368 | most of the time, so there is no need to reduce existing timeout 2369 | values as a result of this change. 2370 | 2371 | ## Release 2.7.4 2372 | 2373 | This release was skipped. 2374 | 2375 | ## Release 2.7.3 (2021-12-06) 2376 | 2377 | - The bundled extractors are updated to match the versions currently 2378 | used on LGTM.com. These are newer than the last release (1.28) of 2379 | LGTM Enterprise. If you plan to upload databases to an LGTM 2380 | Enterprise 1.28 instance, you need to create them with release 2381 | 2.5.9. 2382 | 2383 | ### Potentially breaking changes 2384 | 2385 | - The experimental command-line option `--ml-model-path` that was 2386 | introduced to support internal experiments has been removed. 2387 | 2388 | ### Bugs fixed 2389 | 2390 | - Editing support (content assist, code navigation, etc.) in files 2391 | under the `.github` directory will now work properly. This is 2392 | because files under the `.github` directory will now be indexed and 2393 | processed by the CodeQL language server. Other hidden directories 2394 | that start with `.` will remain un-indexed. This affects the 2395 | vscode-codeql extension and any other IDE extension that uses 2396 | the CodeQL language server. 2397 | 2398 | - Fixed authentication with GitHub package registries via the 2399 | `GITHUB_TOKEN` environment variable and the `--github-auth-stdin` 2400 | flag when downloading and publishing packs. 2401 | 2402 | - Fixed an incompatibility with glibc version 2.34 on Linux, where 2403 | build tracing failed with an error message. 2404 | 2405 | - Fixed a bug where `codeql generate log-summary` could sometimes fail 2406 | with a `JsonMappingException`. 2407 | 2408 | ### New features 2409 | 2410 | - The CodeQL CLI for Mac OS now ships with a native Java virtual machine for M1 Macs, 2411 | and this will be used by default where applicable to run the CodeQL 2412 | engine, thus improving performance. 2413 | [Rosetta 2](https://support.apple.com/en-us/HT211861) is still 2414 | required as not all components of the CodeQL CLI are natively compiled. 2415 | 2416 | - Commands that execute queries will now exit with status code 34 if 2417 | certain errors that prevent the evaluation of one or more 2418 | individual queries are detected. Previously some of these errors 2419 | would crash the evaluator and exit with status code 100. 2420 | 2421 | (This is currently used for "external predicate not found" errors). 2422 | 2423 | ## Release 2.7.2 (2021-11-22) 2424 | 2425 | - The bundled extractors are updated to match the versions currently 2426 | used on LGTM.com. These are newer than the last release (1.28) of 2427 | LGTM Enterprise. If you plan to upload databases to an LGTM 2428 | Enterprise 1.28 instance, you need to create them with release 2429 | 2.5.9. 2430 | 2431 | ### Potentially breaking changes 2432 | 2433 | - The Java extractor now defaults to extracting all XML documents under 2434 | 10MB in size, a change from the previous default of only extracting 2435 | documents with particular well-known names (e.g. `pom.xml`). However, 2436 | if the source tree contains more than 50MB of XML in total, it prints 2437 | a warning and falls back to the old default behaviour. 2438 | Set the environment variable `LGTM_INDEX_XML_MODE` to `byname` to get 2439 | the old default behaviour, or `all` to extract all documents under 2440 | 10MB regardless of total size. 2441 | 2442 | - The experimental command-line option `--native-library-path` that was 2443 | introduced to support internal experiments has been removed. 2444 | 2445 | - The beta `codeql pack publish` command will now prevent accidental 2446 | publishing of packages with pre-release version qualifiers. Prerelease 2447 | versions are those that include a `-` after the major, minor, and patch 2448 | versions such as `1.2.3-dev`. To avoid this change, use the 2449 | `--allow-prerelease` option. 2450 | 2451 | ### Bugs fixed 2452 | 2453 | - Fixed an issue when using the `--evaluator-log` option where a 2454 | `NullPointerException` could sometimes occur non-deterministically. 2455 | 2456 | - Fixed bugs observed when using indirect build tracing using a CodeQL 2457 | distribution unpacked to a path containing spaces or on Arch Linux. 2458 | 2459 | ### New features 2460 | 2461 | - CodeQL databases now contain metadata about how and when they were 2462 | created. This can be found in the `creationMetadata` field of the 2463 | `codeql-database.yml` file within the CodeQL database directory. More 2464 | information may be added to this field in future releases. 2465 | 2466 | ## Release 2.7.1 (2021-11-15) 2467 | 2468 | - The bundled extractors are updated to match the versions currently 2469 | used on LGTM.com. These are newer than the last release (1.28) of 2470 | LGTM Enterprise. If you plan to upload databases to an LGTM 2471 | Enterprise 1.28 instance, you need to create them with release 2472 | 2.5.9. 2473 | 2474 | ### Potentially breaking changes 2475 | 2476 | - Previously, `codeql test run` would fall back to looking for an 2477 | accompanying `queries.xml` file if it found a `qlpack.yml` that did 2478 | not declare an extractor to use when extracting a test database. 2479 | This has been removed because the internal use case that 2480 | neccessitated the fallback are now removed. If you suddenly 2481 | encounter errors that complain of missing extractor declarations, 2482 | check whether you had a `queries.xml` you were inadvertently relying 2483 | on. 2484 | 2485 | - When queries are specified by naming a directory to scan for `*.ql` 2486 | files, subdirectories named `.codeql` will now be ignored. The new 2487 | QL packaging support uses subdirectories with this name of various 2488 | scratch and caching purposes, so they may contain `*.ql` files that 2489 | are not intended to be directly user-visible. 2490 | 2491 | - When copying dependencies for CodeQL packages into a query pack 2492 | bundle, `*.ql` files in these dependencies will now be included 2493 | inside of the query pack's `.codeql` directory. 2494 | 2495 | - The tables printed by `codeql database analyze` to summarize the 2496 | results of diagnostic and metric queries that were part of the 2497 | analysis have a new format and contains less (but hopefully more 2498 | pertinent) information. We recommend against attempting to parse 2499 | this human-readable output programmatically. Instead, use the 2500 | `runs[].tool.driver.invocations[].toolExecutionNotifications` 2501 | property in the SARIF output. 2502 | 2503 | - The experimental plumbing command `codeql pack packlist` has a new 2504 | format for its JSON results. Previously, the results were a list of 2505 | paths. Now, the results are an object with a single property `paths` 2506 | that contains the list of paths. 2507 | 2508 | - The internal `qlpacks` directory of the CodeQL bundle available on the 2509 | [CodeQL Action releases page](https://github.com/github/codeql-action/releases/) 2510 | has a new structure. This directory is internal to the CLI and can change without 2511 | notice in future releases. 2512 | 2513 | The currently-shipped `qlpacks` directory mirrors the structure of [CodeQL package](https://github.blog/changelog/2021-07-28-introducing-the-codeql-package-manager-public-beta/) caches and looks like this: 2514 | 2515 | ```text 2516 | qlpacks 2517 | - codeql 2518 | - {lang}-all 2519 | - {version} 2520 | - qlpack contents 2521 | - {lang}-examples 2522 | - {version} 2523 | - qlpack contents 2524 | - {lang}-queries 2525 | - {version} 2526 | - qlpack contents 2527 | - {lang}-upgrades 2528 | - {version} 2529 | - qlpack contents 2530 | - ... and so on for all languages 2531 | ``` 2532 | 2533 | ### Deprecations 2534 | 2535 | - The output formats SARIF v1.0.0 and SARIF v2.0.0 (Committee 2536 | Specification Draft 1) have been deprecated. They will be removed 2537 | in a later version (earliest 2.8.0). If you need this 2538 | functionality, please file a public issue against 2539 | https://github.com/github/codeql-cli-binaries, or open a private 2540 | ticket with GitHub Support and request an escalation to engineering. 2541 | 2542 | - The `qlpack:` instruction in query suite definitions has been 2543 | deprecated due to uncertainty about whether it is intended to 2544 | include _all_ the `*.ql` files in the named pack, or only the pack's 2545 | "default query suite". The behavior of the instruction is 2546 | determined by whether the named pack declares any default query 2547 | suite, but this means that a pack _starting_ to declare such a suite 2548 | may break the behavior of existing query suites that reference the 2549 | pack from outside. 2550 | 2551 | We recommend replacing `qlpack:` by one of 2552 | ```yaml 2553 | - queries: '.' # import all *.ql files 2554 | from: some/pack-name 2555 | version: 1.2.3 # optional 2556 | ``` 2557 | or 2558 | ```yaml 2559 | - import: path/to/actual/suite.ql # just that suite 2560 | from: some/pack-name 2561 | version: 1.2.3 # optional 2562 | ``` 2563 | 2564 | A warning will now be printed when a `qlpack:` instruction resolves 2565 | to a default suite, because that is the case where the effect may 2566 | not be what the query suite author intended. 2567 | 2568 | ### Bugs fixed 2569 | 2570 | - Fixed a bug where the `paths` and `paths-ignore` properties of a 2571 | Code Scanning config file specified using `--codescanning-config` 2572 | were being interpreted the wrong way around. 2573 | 2574 | - Fixed a bug where queries specified using the 2575 | `--codescanning-config` option could not be run after an explicit 2576 | call to `codeql database finalize`. 2577 | 2578 | - Fixed a bug where `-J` options would erroneously be recognized even 2579 | after `--` on the command line. 2580 | 2581 | - When running `codeql database analyze` and `codeql database 2582 | interpret-results` without the `--sarif-group-rules-by-pack` flag, 2583 | the SARIF output did not include baseline lines-of-code counts. This 2584 | is now fixed. 2585 | 2586 | - Fixed a bug where expansion of query suites would sometimes fail if 2587 | a query suite in a compiled query pack referenced that pack itself 2588 | explicitly. 2589 | 2590 | ### New language features 2591 | 2592 | - [Set literal expressions][9] can now optionally contain a trailing 2593 | comma after the last element. 2594 | 2595 | [9]: https://codeql.github.com/docs/ql-language-reference/expressions/#set-literal-expressions 2596 | 2597 | ### New features 2598 | 2599 | - Beta support for database creation on Apple Silicon has been added. 2600 | It depends on the following requirements: 2601 | 2602 | - [Rosetta 2][8] needs to be installed 2603 | 2604 | [8]: https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment 2605 | 2606 | - Developer tools need to be installed. CodeQL requires the `lipo`, 2607 | `codesign`, and `install_name_tool` tools to be present. 2608 | 2609 | - Build systems invoking `csh` may experience [intermittent 2610 | crashes][7]. 2611 | 2612 | [7]: https://openradar.appspot.com/radar?id=4936797431791616 2613 | 2614 | - `codeql database analyze` can now include query-specific help texts 2615 | for alerts in the SARIF output (for SARIF v2.1.0 or later). The help 2616 | text must be located in an `.md` file next to (and with the same 2617 | basename as) the `.ql` file for each query. Since this can 2618 | significantly increase SARIF file size, the feature is not enabled 2619 | by default; give a `--sarif-add-query-help` option to enable it. 2620 | 2621 | - The query metadata validator now knows about queries that produce 2622 | alert scores, so these queries no longer need to be run with a 2623 | `--no-metadata-verification` flag. 2624 | 2625 | - `codeql database create` and `codeql-finalize` have a new flag 2626 | `--skip-empty` that will cause a language with no extracted source 2627 | code to be ignored with a warning instead of treated like a fatal 2628 | error. This can be useful with `--db-cluster` where not all of the 2629 | languages may exist in the source tree. It will not be possible to 2630 | run queries against the skipped database. 2631 | 2632 | - `codeql resolve extractor` and `codeql resolve languages` now 2633 | support an extended output format `--format=betterjson` wich 2634 | includes information about each extractor's language-specific 2635 | options. 2636 | 2637 | - This release introduces rudimentary support for parallelizing 2638 | database creation by importing unfinished databases (or database 2639 | clusters) into another unfinished database (or cluster) under 2640 | creation. This is implemented by the new flag `--additional-dbs` for 2641 | `codeql database finalize`, or the new plumbing command `codeql 2642 | database import`. 2643 | 2644 | - `codeql database create`, `codeql database index-files`, and `codeql 2645 | database trace-command` support a [unified syntax for passing 2646 | language-specific options][6] to the extractor with the new 2647 | `--extractor-option` and `--extractor-options-file` options. 2648 | (The extractors do not make use of this yet, though). 2649 | 2650 | [6]: https://codeql.github.com/docs/codeql-cli/extractor-options 2651 | 2652 | ## Release 2.7.0 (2021-10-27) 2653 | 2654 | - The extractor for Ruby is now included. CodeQL analysis for Ruby is 2655 | currently in beta. During the beta, analysis of Ruby will not be as 2656 | comprehensive as CodeQL analysis of other languages. The source code 2657 | of the extractor and the queries can be found in the 2658 | [`github/codeql`](https://github.com/github/codeql) repository. 2659 | 2660 | - The bundled extractors are updated to match the versions currently 2661 | used on LGTM.com. These are newer than the last release (1.28) of 2662 | LGTM Enterprise. If you plan to upload databases to an LGTM 2663 | Enterprise 1.28 instance, you need to create them with release 2664 | 2.5.9. 2665 | 2666 | ### Bugs fixed 2667 | 2668 | - Fixed a bug where indirect tracing would sometimes not manage to 2669 | observe build processes if certain environment variables were unset 2670 | during the build. 2671 | 2672 | ## Release 2.6.3 (2021-10-06) 2673 | 2674 | - The bundled extractors are updated to match the versions currently 2675 | used on LGTM.com. These are newer than the last release (1.28) of 2676 | LGTM Enterprise. If you plan to upload databases to an LGTM 2677 | Enterprise 1.28 instance, you need to create them with release 2678 | 2.5.9. 2679 | 2680 | ### Potentially breaking changes 2681 | 2682 | - The option `--compiler-spec` accepted by some subcommands of `codeql 2683 | database` is deprecated. It will be removed in a later version 2684 | (earliest 2.7.0). If you need this option, please file a public 2685 | issue in https://github.com/github/codeql-cli-binaries, or open a 2686 | private ticket with GitHub support and request an escalation to 2687 | engineering. 2688 | 2689 | - By default, databases created using the CodeQL CLI will now have 2690 | their underlying datasets finalized, meaning that no further data 2691 | can be subsequently imported into them. This change should not 2692 | affect most users. 2693 | 2694 | - The `codeql resolve qlref` command will now throw an error when the 2695 | target is ambiguous. The qlref resolution rules are now as follows: 2696 | 2697 | 1. If the target of a qlref is in the same qlpack, then that target 2698 | is always returned. 2699 | 2700 | 2. If multiple targets of the qlref are found in dependent packs, 2701 | this is an error. 2702 | 2703 | Previously, the command would have arbitrarily chosen one of the targets and ignored any 2704 | ambiguities. 2705 | 2706 | ### Bugs fixed 2707 | 2708 | - Linux/MacOS: When tracing a build that involves an 2709 | `execvp`/`execvpe` (Linux-only)/`posix_spawnp` syscall where `PATH` 2710 | was not set in the environment, CodeQL sometimes would break the 2711 | build. Now, CodeQL uses the correct, platform-specific fallback for 2712 | `PATH` instead. 2713 | 2714 | - Linux/MacOS: When tracing a build that involves an `execvpe` 2715 | (Linux-only)/`posix_spawnp` syscall, the `PATH` lookup of the 2716 | executable wrongly took place in the environment provided via 2717 | `envp`, instead of the environment of the process calling 2718 | `execvpe`/`posix_spawnp`. Now, the correct environment is used for 2719 | the `PATH` lookup. 2720 | 2721 | - A bug where query compilation would sometimes fail with a 2722 | `StackOverflowError` when compiling a query that uses `instanceof` 2723 | has now been fixed. 2724 | 2725 | ### New features 2726 | 2727 | - The `codeql query compile` command now accepts a `--keep-going` or 2728 | `-k` option, which indicates that the compiler should continue 2729 | compiling queries even if one of the queries has a compile error in 2730 | it. 2731 | 2732 | - CLI commands now run default queries if none are specified. If no 2733 | queries are specified, the `codeql database analyze`, `codeql 2734 | database run-queries`, and `codeql database interpret-results` 2735 | commands will now run the default suite for the language being 2736 | analyzed. 2737 | 2738 | - `codeql pack publish` now copies the published package to the local 2739 | package cache. In addition to publishing to a remote repository, the 2740 | `codeql pack publish` command will also copy the published package 2741 | to the local package cache. 2742 | 2743 | ## Release 2.6.2 (2021-09-21) 2744 | 2745 | - CodeQL CLI 2.6.2 includes the same functionality as **the CodeQL 2746 | runner**, which is being deprecated. For more information, see 2747 | [CodeQL runner deprecation][5]. 2748 | 2749 | [5]: https://github.blog/changelog/2021-09-21-codeql-runner-deprecation/ 2750 | 2751 | - The bundled extractors are updated to match the versions currently 2752 | used on LGTM.com. These are newer than the last release (1.28) of 2753 | LGTM Enterprise. If you plan to upload databases to an LGTM 2754 | Enterprise 1.28 instance, you need to create them with release 2755 | 2.5.9. 2756 | 2757 | ### Bugs fixed 2758 | 2759 | - A bug where `codeql generate log-summary` would sometimes crash with 2760 | a `JsonMappingException` has been fixed. 2761 | 2762 | ### New features 2763 | 2764 | - The CodeQL CLI now counts the lines of code found under 2765 | `--source-root` when `codeql database init` or `codeql database 2766 | create` is called. This information can be viewed later by either 2767 | the new `codeql database print-baseline` command or the new 2768 | `--print-baseline-loc` argument to `codeql database 2769 | interpret-results`. 2770 | 2771 | - `qlpack.yml` files now support an additional field `include` in 2772 | which glob patterns of additional files that should be included (or 2773 | excluded) when creating a given CodeQL pack can be specified. 2774 | 2775 | - QL packs created by the experimental `codeql pack create` command 2776 | will now include some information about the build in a new 2777 | `buildMetadata` field of their `qlpack.yml` file. 2778 | 2779 | - `codeql database create` now supports the same flags as `codeql 2780 | database init` for automatically recognizing the languages present 2781 | in checkouts of GitHub repositories: 2782 | 2783 | - `--github-url` accepts the URL of a custom GitHub instance 2784 | (previously only `github.com` was supported). 2785 | 2786 | - `--github-auth-stdin` allows a personal access token to be 2787 | provided through standard input (previously only the 2788 | `GITHUB_TOKEN` environment variable was supported). 2789 | 2790 | ### Notable documentation changes 2791 | 2792 | - Documentation has been added detailing how to use the "indirect 2793 | build tracing" feature, which is enabled by using the 2794 | `--begin-tracing` flag provided by `codeql database init`. The new 2795 | documentation can be found [here][4]. This feature was temporarily 2796 | described as "sandwiched tracing" in the 2.6.0 release notes. 2797 | 2798 | [4]: https://aka.ms/codeql-docs/indirect-tracing 2799 | 2800 | ## Release 2.6.1 (2021-09-07) 2801 | 2802 | - The bundled extractors are updated to match the versions currently 2803 | used on LGTM.com. These are newer than the last release (1.28) of 2804 | LGTM Enterprise. If you plan to upload databases to an LGTM 2805 | Enterprise 1.28 instance, you need to create them with release 2806 | 2.5.9. 2807 | 2808 | ### Potentially breaking changes 2809 | 2810 | - The `codeql resolve qlref` command will now throw an error when the 2811 | target is ambiguous. 2812 | 2813 | The qlref resolution rules are now as follows: 2814 | 2815 | 1. If the target of a qlref is in the same qlpack, then that target is 2816 | always returned. 2817 | 2818 | 2. If multiple targets of the qlref are found in dependent packs, this 2819 | is an error. 2820 | 2821 | Previously, the command would have arbitrarily chosen one of the targets 2822 | and ignored any ambiguities. 2823 | 2824 | - The `qlpack` directive in query suites has its semantics changed. 2825 | Previously, this directive would return all queries in the 2826 | qlpack. Now, the directive returns only those queries matched by the 2827 | `defaultSuite` directive in the query pack. Here is an example: 2828 | 2829 | Consider a `qlpack.yml` like the following: 2830 | 2831 | ```yml 2832 | name: codeql/my-qlpack 2833 | version: 0.0.1 2834 | defaultSuite: 2835 | queries: standard 2836 | ``` 2837 | 2838 | And the directory structure is the following: 2839 | 2840 | ```text 2841 | qlpack.yml 2842 | standard/ 2843 | a.ql 2844 | experimental/ 2845 | b.ql 2846 | ``` 2847 | 2848 | A query suite `suite.qls` like this: 2849 | 2850 | ```yml 2851 | - qlpack: codeql/my-qlpack 2852 | ``` 2853 | 2854 | Previously, would return all the queries in all subdirectories (i.e, 2855 | `standard/a.ql` and `experimental/b.ql`). Now, it only returns 2856 | `standard/a.ql`, since that is the only query matched by its default 2857 | suite. 2858 | 2859 | If you want to have the same behavior as before, you must update your 2860 | query suites to use the `queries` directive with a `from` attribute, 2861 | like this: 2862 | 2863 | ```yml 2864 | - queries: . 2865 | from: codeql/my-qlpack 2866 | ``` 2867 | 2868 | ### New features 2869 | 2870 | - Commands that evaluate CodeQL queries now support an additional 2871 | option `--evaluator-log=path/to/log.json` that will result in the 2872 | evaluator producing a structured log (in JSON format) of events that 2873 | occurred during evaluation in order to aid debugging of query 2874 | performance. The format of these logs will be subject to change with 2875 | no notice as we make modifications to the evaluator. 2876 | 2877 | There is also a new CLI command `codeql generate log-summary` that 2878 | will produce a summary of the predicates that were evaluated from 2879 | these event logs. We will aim to keep this summary format more 2880 | stable, although it is also subject to change. Unless you have a 2881 | good reason to use the event logs directly, it is strongly 2882 | recommended you use this command to produce summary logs and use 2883 | these instead. 2884 | 2885 | For further information on these new logs and additional options to 2886 | configure their format and verbosity, please refer to the CLI 2887 | documentation. 2888 | 2889 | ### New language features 2890 | 2891 | - QL classes can now be non-extending subtypes via the `instanceof` 2892 | keyword, allowing for a form of private subtyping that is not 2893 | visible externally. Methods of the supertype are accessible from 2894 | within a non-extending subtype class through extended semantics of 2895 | the `super` keyword. 2896 | 2897 | ``` 2898 | class Foo instanceof int { 2899 | Foo() { this in [1 .. 10] } 2900 | string toString() { result = "foo" + super.toString() } 2901 | } 2902 | ``` 2903 | 2904 | ## Release 2.6.0 (2021-08-24) 2905 | 2906 | - The bundled extractors are updated to match the versions currently 2907 | used on LGTM.com. These are newer than the last release (1.28) of 2908 | LGTM Enterprise. If you plan to upload databases to an LGTM 2909 | Enterprise 1.28 instance, you need to create them with release 2910 | 2.5.9. 2911 | 2912 | ### Bugs fixed 2913 | 2914 | - The `physicalLocation.artifactLocation.uri` fields in SARIF output 2915 | are now properly encoded as specified by RFC 3986. 2916 | 2917 | - The `--include-extension` option to the `codeql database 2918 | index-files` command no longer includes directories that are named 2919 | with the provided extension. For example, if the option 2920 | `--include-extension=.rb` is provided, then a directory named 2921 | `foo.rb/` will be excluded from the indexing. 2922 | 2923 | ### New features 2924 | 2925 | - A new `codeql database unbundle` subcommand performs the reverse of 2926 | `codeql database bundle` and extracts a CodeQL database from an 2927 | archive. 2928 | 2929 | - The CLI now understands per-codebase configuration files in [the 2930 | format already supported by the CodeQL Action][3]. The 2931 | configuration file must be given in a `--codescanning-config` option 2932 | to `codeql database create` or `codeql database init`. For some 2933 | languages, this configuration can contain pathname filters that 2934 | control which parts of the codebase is analysed; the configuration 2935 | file is the only way this functionality is exposed. The 2936 | configuration file can also control which queries are run, including 2937 | custom queries from repositories that must first be downloaded. To 2938 | actually use those queries, run `codeql database analyze` without 2939 | any query-selection arguments. 2940 | 2941 | [3]: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#example-configuration-files 2942 | 2943 | - The CLI now supports the "sandwiched tracing" feature that has 2944 | previously only been offered through the separate CodeQL Runner. 2945 | This feature is intended for use with CI systems that cannot be 2946 | configured to wrap build actions with `codeql database 2947 | trace-command`. Instead the CI system must be able to set custom 2948 | environment variables for each build action; the required 2949 | environment variables are output by `codeql database init` when 2950 | given a `--begin-tracing` argument. 2951 | 2952 | On Windows, `codeql database init --begin-tracing` will also inject 2953 | build-tracing code into the calling process or an ancestor; there 2954 | are additional options to control this. 2955 | 2956 | - This version contains _beta_ support for a new packaging and 2957 | publishing system for third-party QL queries and libraries. It 2958 | comprises the following new commands: 2959 | 2960 | - `codeql pack init`: Creates an empty CodeQL pack from a template. 2961 | 2962 | - `codeql pack add`: Adds a dependency to a CodeQL pack. 2963 | 2964 | - `codeql pack install`: Installs all pack dependencies specified in 2965 | the `qlpack.yml` file. 2966 | 2967 | - `codeql pack download`: Downloads one or more pack dependencies 2968 | into the global package cache. 2969 | 2970 | - `codeql pack publish`: Publishes a package to the GitHub Container 2971 | Registry. 2972 | 2973 | - (Plumbing) `codeql pack bundle`: Builds a `.zip` file for a CodeQL 2974 | query or library pack from sources. Used by `codeql pack publish`. 2975 | 2976 | - (Plumbing) `codeql pack create`: Creates a compiled CodeQL query 2977 | or library pack from sources. Used by `codeql pack bundle`. 2978 | 2979 | - (Plumbing) `codeql pack packlist`: Lists all files in a local 2980 | CodeQL pack that will be included in the pack's bundle. Used by 2981 | `codeql pack create`. 2982 | 2983 | - (Plumbing) `codeql pack resolve-dependencies`: Resolves all 2984 | transitive dependencies of a local CodeQL pack. Used by `codeql 2985 | pack install`. 2986 | 2987 | ## Release 2.5.9 (2021-08-09) 2988 | 2989 | This release corresponds to release 1.28.x of LGTM Enterprise, and 2990 | should be used when creating databases that will be uploaded to it. 2991 | Future CLI releases (numbered 2.6.x) may produce databases that are not 2992 | backwards compatible with this version of LGTM Enterprise. 2993 | 2994 | ## Release 2.5.8 (2021-07-26) 2995 | 2996 | - The bundled extractors are updated to match the versions currently 2997 | used on LGTM.com. These are newer than the last release (1.27) of 2998 | LGTM Enterprise. If you plan to upload databases to an LGTM 2999 | Enterprise 1.27 instance, you need to create them with release 3000 | 2.4.6. 3001 | 3002 | ### Potentially breaking changes 3003 | 3004 | - The QL compiler now verifies that `@security-severity` query 3005 | metadata is numeric. You can disable this verification by passing 3006 | the `--no-metadata-verification` flag. 3007 | 3008 | ### New features 3009 | 3010 | - The `database index-files` and `database trace-command` CLI commands 3011 | now support `--threads` and `--ram` options, which are passed to 3012 | extractors as suggestions. 3013 | 3014 | - The `database finalize` CLI command now supports the `--ram` option, 3015 | which controls memory usage for finalization. 3016 | 3017 | - The `database create` CLI command now supports the `--ram` option, 3018 | which controls memory usage for database creation. - The `generate 3019 | query-help` CLI command now support rendering query help in SARIF 3020 | format. 3021 | 3022 | ## Release 2.5.7 (2021-07-02) 3023 | 3024 | - The bundled extractors are updated to match the versions currently 3025 | used on LGTM.com. These are newer than the last release (1.27) of 3026 | LGTM Enterprise. If you plan to upload databases to an LGTM 3027 | Enterprise 1.27 instance, you need to create them with release 3028 | 2.4.6. 3029 | 3030 | ### New features 3031 | 3032 | - `codeql database create` and `codeql database init` can now 3033 | automatically recognise the languages present in checkouts of GitHub 3034 | repositories by making an API call to the GitHub server. This 3035 | requires a PAT token to either be set in the `GITHUB_TOKEN` 3036 | environment variable, or passed by stdin with the 3037 | `--github-auth-stdin` argument. 3038 | 3039 | - Operations that make outgoing HTTP calls (that is, `codeql github 3040 | upload-results` and the language-detection feature described above) 3041 | now support the use of HTTP proxies. To use a proxy, specify an 3042 | `$https_proxy` environment variable for HTTPS requests or a 3043 | `$http_proxy` environment variable for HTTP requests. If the 3044 | `$no_proxy` variable is also set, these variables will be ignored 3045 | and requests will be made without a proxy. 3046 | 3047 | ### New language features 3048 | 3049 | - The QL language now has a new method `toUnicode` on the `int` 3050 | type. This method converts Unicode codepoint to a one-character 3051 | string. For example, `65.toUnicode() = "A"`, `128512.toUnicode()` 3052 | results in a smiley, and `any(int i | i.toUnicode() = "A") = 65`. 3053 | 3054 | ## Release 2.5.6 (2021-06-22) 3055 | 3056 | - The bundled extractors are updated to match the versions currently 3057 | used on LGTM.com. These are newer than the last release (1.27) of 3058 | LGTM Enterprise. If you plan to upload databases to an LGTM 3059 | Enterprise 1.27 instance, you need to create them with release 3060 | 2.4.6. 3061 | 3062 | ### Features added 3063 | 3064 | - `codeql database create` (and the plumbing commands it comprises) 3065 | now supports creating databases for a source tree with several 3066 | languages while tracing a single build. This is enabled by a new 3067 | `--db-cluster` option. Once created, the multiple databases must be 3068 | _analyzed_ one by one. 3069 | 3070 | - `codeql database create` and `codeql database init` now accept an 3071 | `--overwrite` argument which will lead existing CodeQL databases to 3072 | be overwritten. 3073 | 3074 | - `codeql database analyze` now supports "diagnostic" queries (tagged 3075 | `@kind diagnostic`), which are intended to report information about 3076 | the analysis process itself rather than problems with the analyzed 3077 | code. The results of these queries will be summarized in a table 3078 | printed to the terminal when `codeql database analyze` finishes. 3079 | 3080 | They are also included in the analysis results in SARIF output 3081 | formats as [notification objects][2] so they can be displayed by 3082 | subsequent tooling such as the Code Scanning user interface. 3083 | 3084 | - For SARIF v2.1.0, a reporting descriptor object for each 3085 | diagnostic query is output to output to 3086 | `runs[].tool.driver.notifications`, or 3087 | `runs[].tool.extensions[].notifications` if running with 3088 | `--sarif-group-rules-by-pack`. A rule object for each diagnostic 3089 | query is output to `runs[].resources[].rules` for SARIF v2, or to 3090 | `runs[].rules` for SARIF v1. 3091 | 3092 | - Results of diagnostic queries are exported to the 3093 | `runs[].invocations[].toolExecutionNotifications` property in 3094 | SARIF v2.1.0, the `runs[].invocations[].toolNotifications` 3095 | property in SARIF v2, and the `runs[].toolNotifications` property 3096 | in SARIF v1. 3097 | 3098 | SARIF v2.1.0 output will now also contain version information for 3099 | query packs in `runs[].tool.extensions[].semanticVersion`, if the 3100 | Git commit the queries come from is known. 3101 | 3102 | [2]: https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317894 3103 | 3104 | - `codeql github upload-results` has a `--checkout-path` option which 3105 | will attempt to automatically configure upload target parameters. 3106 | When this is given, the `--commit` option will be taken from the 3107 | HEAD of the checkout Git repository, and if there is precisely one 3108 | remote configured in the local repository, the `--repository` and 3109 | `--github-url` options will also be automatically configured. 3110 | 3111 | - The CodeQL C++ extractor includes beta support for C++20. 3112 | This is only available when building codebases with GCC on Linux. 3113 | C++20 modules are **not** supported. 3114 | 3115 | ## Release 2.5.5 (2021-05-17) 3116 | 3117 | - The bundled extractors are updated to match the versions currently 3118 | used on LGTM.com. These are newer than the last release (1.27) of 3119 | LGTM Enterprise. If you plan to upload databases to an LGTM 3120 | Enterprise 1.27 instance, you need to create them with release 3121 | 2.4.6. 3122 | 3123 | ### Potentially breaking changes 3124 | 3125 | - When scanning the disk for QL packs and extractors, directories of 3126 | the form `.../SOMETHING/SOMETHING.testproj` (where the two 3127 | `SOMETHING` are identical) will now be ignored. Names of this form 3128 | are used by `codeql test run` for ephemeral test databases, which 3129 | can sometimes contain files that confuse QL compilations. 3130 | 3131 | ### Features added 3132 | 3133 | - Query writers can now optionally use `@severity` in place of 3134 | `@problem.severity` in the metadata for alert queries. SARIF 3135 | consumers should continue to consume this severity information using 3136 | the `rule.defaultConfiguration.level` property for SARIF v2.1.0, and 3137 | corresponding properties for other versions of SARIF. They should 3138 | not depend on the value stored in the `rule.properties` property 3139 | bag, since this will contain either `@problem.severity` or 3140 | `@severity` based on exactly what was written in the query metadata. 3141 | 3142 | - When exporting analysis results to SARIF v2.1.0, results and metric 3143 | results now contain a [reporting descriptor reference object][1] 3144 | that specifies the rule that produced them. For metric results, this 3145 | new property replaces the `metric` property. 3146 | 3147 | [1]: https://docs.oasis-open.org/sarif/sarif/v2.1.0/csprd01/sarif-v2.1.0-csprd01.html#_Toc10541300 3148 | 3149 | - `codeql database analyze` now outputs a table that summarizes the 3150 | results of metric queries that were part of the analysis. This can 3151 | be suppressed by passing the `--no-print-metrics-summary` flag. 3152 | 3153 | ### Bugs fixed 3154 | 3155 | - When using the `--sarif-group-rules-by-pack` flag to place the SARIF 3156 | rule object for each query underneath its corresponding query pack 3157 | in `runs[].tool.extensions`, the `rule` property of result objects 3158 | can now be used to look up the rule within the `rules` property of 3159 | the appropriate query pack in `runs[].tool.extensions`. Previously, 3160 | rule lookup for result objects in the SARIF output was not 3161 | well-defined when the `--sarif-group-rules-by-pack` flag was passed. 3162 | 3163 | ## Release 2.5.4 (2021-05-03) 3164 | 3165 | - This release is identical to release 2.5.3, except that 3166 | `codeql database analyze` no longer produces a generated 3167 | `automationDetails.id` field when the `--sarif-category` 3168 | is not explicitly provided. Previously, the 3169 | `--sarif-category` was autogenerated if not present. 3170 | - Code Scanning users should upgrade to this version and 3171 | avoid 2.5.3. 3172 | 3173 | ## Release 2.5.3 (2021-04-30) 3174 | 3175 | - The bundled extractors are updated to match the versions currently 3176 | used on LGTM.com. These are newer than the last release (1.27) of 3177 | LGTM Enterprise. If you plan to upload databases to an LGTM 3178 | Enterprise 1.27 instance, you need to create them with release 3179 | 2.4.6. 3180 | 3181 | ### Next release: Features added 3182 | 3183 | - When tracing a C/C++ build, the C compiler entries in compiler-settings 3184 | must now specify `order compiler,extractor`. The default configuration 3185 | already does this, so no change is necessary if using the default 3186 | configuration. 3187 | 3188 | - `codeql database analyze` and `codeql database interpret-results` 3189 | now report the results of summary metric queries in the 3190 | `.properties.metricResults` property of the SARIF output. 3191 | Summary metric queries describe metrics about the code analyzed by 3192 | CodeQL. They are identified by the query metadata `@kind metric` and 3193 | `@tag summary`. 3194 | For example, see the [lines of code summary metric query for 3195 | C++](https://github.com/github/codeql/blob/main/cpp/ql/src/Summary/LinesOfCode.ql). 3196 | 3197 | - `codeql database analyze` and `codeql database interpret-results` 3198 | now calculate an 3199 | [automation ID](https://docs.oasis-open.org/sarif/sarif/v2.1.0/cs01/sarif-v2.1.0-cs01.html#_Toc16012482) 3200 | and add it to the resulting SARIF. In SARIF v2.1.0, this field is 3201 | `runs[].automationDetails.id`. In SARIF v2, this field is 3202 | `runs[].automationLogicalId`. In SARIF v1, this field is 3203 | `runs[].automationId`. By default, this automation ID will be 3204 | derived from the database language and the operating system of the 3205 | machine that performed the run. It can be set explicitly using a new 3206 | `--sarif-category` option. 3207 | 3208 | - In query metadata, `@kind alert` and `@kind path-alert` are now 3209 | recognized as (more accurate) synonyms of `@kind problem` and 3210 | `@kind path-problem`, respectively. 3211 | 3212 | - Diagnostic queries are now permitted by the metadata verifier. They 3213 | are identified by `@kind diagnostic` metadata. Currently the result 3214 | patterns of diagnostic queries are not verified. This will change in 3215 | a future CLI release. 3216 | 3217 | ### Bugs fixed 3218 | 3219 | - Ensure the correct URL is generated during `codeql github upload-results` 3220 | for GitHub Enterprise Server. 3221 | 3222 | ## Release 2.5.2 (2021-04-21) 3223 | 3224 | This release is identical to release 2.5.1, except that an internal 3225 | incompatibility with the CodeQL action (and the codeql-runner that 3226 | some customers use for CI integrations) has been fixed. 3227 | 3228 | The fix does not affect any use cases where the CLI is downloaded from 3229 | github/codeql-cli-binaries, so if you're seeing this release there, 3230 | there's no need to upgrade from 2.5.1. 3231 | 3232 | ## Release 2.5.1 (2021-04-19) 3233 | 3234 | - The bundled extractors are updated to match the versions currently 3235 | used on LGTM.com. These are newer than the last release (1.27) of 3236 | LGTM Enterprise. If you plan to upload databases to an LGTM 3237 | Enterprise 1.27 instance, you need to create them with release 3238 | 2.4.6. 3239 | 3240 | ### Potentially breaking changes 3241 | 3242 | - The QL compiler will now reject queries where the query metadata (if 3243 | present) at the top of the `.ql` file is inconsistent with the 3244 | output format of the query. This check can be disabled by giving 3245 | the `--no-metadata-verification` flag. (The flag already existed 3246 | but has not had any effect until now.) 3247 | 3248 | ### Bugs fixed 3249 | 3250 | - Environment variables required for Java extraction are now 3251 | propagated by the tracer. This may resolve issues with tracing and 3252 | extraction in the context of certain build systems such as Bazel. 3253 | 3254 | - A number of `--check-CONDITION` options to `codeql database 3255 | finalize` and `codeql dataset import` designed to look for 3256 | consistency errors in the intermediate "TRAP" output from extractors 3257 | erroneously did nothing. They will now actually print warnings if 3258 | errors are found. The warnings become fatal errors if the new 3259 | `--fail-on-trap-errors` option is also given. 3260 | 3261 | ### Features added 3262 | 3263 | - `codeql resolve qlref` is a new command that takes in a `.qlref` 3264 | file for a CodeQL test case and returns the path of the `.ql` file 3265 | it references. 3266 | 3267 | - `codeql database analyze` and `codeql database interpret-results` 3268 | have a new `--sarif-group-rules-by-pack` option which will place the 3269 | SARIF rule object for each query underneath its corresponding query 3270 | pack in `runs[].tool.extensions`. 3271 | 3272 | - `codeql database finalize` and `codeql dataset import` have a new 3273 | `--fail-on-trap-errors` option that will make database creation fail 3274 | if extractors produce ill-formatted "TRAP" data for inclusion into a 3275 | database. This is not enabled by default because some of the 3276 | existing extractors have minor output bugs that cause the check to 3277 | fail. 3278 | 3279 | - `codeql database finalize` and `codeql dataset import` have a new 3280 | `--check-undefined-labels` option that enables stricter consistency 3281 | checks on the "TRAP" output from extractors. 3282 | 3283 | ### QL language improvements 3284 | 3285 | - `super` may now be used unqualified, e.g. `super.predicateName()`, 3286 | when the declaring class has multiple super types, as long as the 3287 | call itself is unambiguous. 3288 | 3289 | ## Release 2.5.0 (2021-03-26) 3290 | 3291 | - The bundled extractors are updated to match the versions currently 3292 | used on LGTM.com. These are newer than the last release (1.27) of 3293 | LGTM Enterprise. If you plan to upload databases to an LGTM 3294 | Enterprise 1.27 instance, you need to create them with release 3295 | 2.4.6. 3296 | 3297 | ### Potentially breaking changes 3298 | 3299 | - By default, `codeql test` now performs additional compiler 3300 | checks when extracting test code written in Java. 3301 | Existing Java tests that previously passed may therefore fail due 3302 | to this change, if they do not compile using the `javac` compiler. 3303 | To allow time to migrate existing tests, the new behavior can be 3304 | disabled by setting the environment variable 3305 | `CODEQL_EXTRACTOR_JAVA_FLOW_CHECKS=false`. 3306 | 3307 | ### Features added 3308 | 3309 | - Log files that contain output from build processes will now prefix 3310 | it with `[build-stdout]` and `[build-stderr]` instead of `[build]` 3311 | and `[build-err]`. In particular the latter sometimes caused 3312 | confusion. 3313 | 3314 | ### QL language improvements 3315 | 3316 | - The QL language now recognizes new `pragma[only_bind_into](...)` and 3317 | `pragma[only_bind_out](...)` annotations on expressions. Advanced users 3318 | may use these annotations to provide hints to the compiler to influence 3319 | binding behavior and thus indirectly performance. 3320 | 3321 | ## Release 2.4.6 (2021-03-19) 3322 | 3323 | This release corresponds to release 1.27.x of LGTM Enterprise, and 3324 | should be used when creating databases that will be uploaded to it. 3325 | Future CLI releases (numbered 2.5.x) may produce databases that are not 3326 | backwards compatible with this version of LGTM Enterprise. 3327 | 3328 | - Fixed a bug in `codeql test run` that causes tests to fail messily 3329 | if the freshly-extracted test database needed to be upgraded in 3330 | order to be compatible with the QL source under test. This would 3331 | happen more often at the _end_ of a release cycle, after updates to 3332 | the QL repository had happened. 3333 | 3334 | - `codeql github upload-results` should now work correctly against 3335 | GitHub Enterprise Server instances that are configured with a path 3336 | prefix. 3337 | 3338 | ## Release 2.4.5 (2021-03-08) 3339 | 3340 | - The bundled extractors are updated to match the versions currently 3341 | used on LGTM.com. These are newer than the last release (1.26) of 3342 | LGTM Enterprise. If you plan to upload databases to an LGTM 3343 | Enterprise 1.26 instance, you need to create them with release 3344 | 2.3.4. 3345 | 3346 | - The C/C++ extractor can now parse more Microsoft language extensions when in 3347 | C++14 and C++17 mode. 3348 | 3349 | - `codeql database analyze` now reports the name and version of each 3350 | QL pack used by the analysis. You can find this information in the 3351 | SARIF output. In particular, the `runs[0].tool.extensions` property 3352 | contains an object for each QL pack used by the analysis. Each 3353 | object contains the `name` and `semanticVersion` of the 3354 | corresponding QL pack, if such information is available. 3355 | 3356 | - `codeql github upload-results` is a new command that uploads a SARIF file 3357 | generated by CodeQL to GitHub's Code Scanning. 3358 | 3359 | ## Release 2.4.4 (2021-02-12) 3360 | 3361 | - The bundled extractors are updated to match the versions currently 3362 | used on LGTM.com. These are newer than the last release (1.26) of 3363 | LGTM Enterprise. If you plan to upload databases to an LGTM 3364 | Enterprise 1.26 instance, you need to create them with release 3365 | 2.3.4. 3366 | 3367 | ### Potentially breaking changes 3368 | 3369 | - The `name` property in `qlpack.yml` must now meet the following requirements: 3370 | - Only lowercase ASCII letters, ASCII digits, and hyphens (`-`) are allowed. 3371 | - A hyphen is not allowed as the first or last character of the name. 3372 | - The name must be at least one character long, and no longer than 128 characters. 3373 | 3374 | ### New features 3375 | 3376 | - Alert and path queries can now give a score to each alert they 3377 | produce. You can incorporate alert scores in an alert or path query 3378 | by first adding the `@scored` property to the query metadata. You 3379 | can then introduce a new numeric column at the end of the `select` 3380 | statement structure to represent the score of each alert. 3381 | Alert scores are exposed in the SARIF output of commands like 3382 | `codeql database analyze` as the `score` property in the property 3383 | bags of result objects. 3384 | 3385 | ### Bugs fixed 3386 | 3387 | - The default value of the `--working-dir` options for the 3388 | `index-files` and `trace-command` subcommands of `codeql database` 3389 | has been fixed to match the documentation; previously, it would 3390 | erroneously use the process' current working directory rather than 3391 | the database source root. 3392 | 3393 | - `codeql test run` will not crash if database extraction in a test 3394 | directory fails. Instead only the tests in that directory will be 3395 | marked as failing, and tests in other directories will continue 3396 | executing. 3397 | 3398 | ## Release 2.4.3 (2021-01-29) 3399 | 3400 | Fixes several bugs introduced in 2.4.2, related to searching the disk for 3401 | QL packs: 3402 | 3403 | - In many cases the search would scan through more of the file system 3404 | than it should. Often the only effect of this was that the scan would 3405 | take longer (sometimes significantly longer) but in some corner 3406 | cases it could lead to packs being found that _shouldn't_ be found, 3407 | which could lead to compilation failure if different versions of the same 3408 | pack exist on disk. 3409 | 3410 | - The search would terminate a fatal error if it met a directory without 3411 | read permission. 3412 | 3413 | - A `provide` entry in `.codeqlmanifest.json` that ended with `*` would 3414 | erroneously not match a `.codeqlmanifest.json` in a subdirectory. 3415 | 3416 | As a consequence of the latter fix, the semantics of 3417 | `.codeqlmanifest.json` files has changed slightly: Directory names 3418 | that start with a dot used to not be matched by the pattern elements 3419 | `*` and `**`, whereas now even dotted directories match such a pattern 3420 | element. The previous behavior was never documented, and only very few 3421 | users have `.codeqlmanifest.json` files of their own in the first 3422 | place, so this change is expected to have minimal practical effect. 3423 | 3424 | ## Release 2.4.2 (2021-01-22) 3425 | 3426 | - The bundled extractors are updated to match the versions currently 3427 | used on LGTM.com. These are newer than the last release (1.26) of 3428 | LGTM Enterprise. If you plan to upload databases to an LGTM 3429 | Enterprise 1.26 instance, you need to create them with release 3430 | 2.3.4. 3431 | 3432 | ## Release 2.4.1 (2020-12-19) 3433 | 3434 | - The bundled extractors are updated to match the versions currently 3435 | used on LGTM.com. These are newer than the last release (1.26) of 3436 | LGTM Enterprise. If you plan to upload databases to an LGTM 3437 | Enterprise 1.26 instance, you need to create them with release 3438 | 2.3.4. 3439 | 3440 | ### Features added 3441 | 3442 | - `codeql query format` now checks all files rather than stopping 3443 | after the first failure when the `--check-only` option is given. 3444 | 3445 | - `codeql resolve database` will produce a `languages` key giving the 3446 | language the database was created for. This can be useful in IDEs to 3447 | help describe the database and suggest default actions or queries. 3448 | For databases created by earlier versions, the result will be a 3449 | best-effort guess. 3450 | 3451 | - `codeql database interpret-results` can now produce Graphviz `.dot` 3452 | files from queries with `@kind graph`. 3453 | 3454 | ### Features removed 3455 | 3456 | - `codeql test run` had some special compatibility support for running 3457 | unit tests for the "code duplication" extractor features of certain 3458 | discontinued Semmle products. Those tests have since been removed 3459 | from the [public QL repository](https://github.com/github/codeql), 3460 | so the compatibility support for them has been removed. This should 3461 | not affect any external users (since the extractor feature in 3462 | question was never supported by `codeql database create` anyway), 3463 | but if you run `codeql test run` against the unit tests belonging to 3464 | an _old_ checkout of the repository, you may now see some failures 3465 | among `Metrics` tests. 3466 | 3467 | ## Release 2.3.4 (2020-12-15) 3468 | 3469 | This release corresponds to release 1.26.x of LGTM Enterprise, and 3470 | should be used when creating databases that will be uploaded to it. 3471 | Future CLI releases (numbered 2.4.x) may produce databases that are not 3472 | backwards compatible with this version of LGTM Enterprise. 3473 | 3474 | For all purposes other than creating databases for LGTM Enterprise we 3475 | recommend that you upgrade to CLI releases numbered 2.4.x or later. 3476 | 3477 | ## Release 2.4.0 (2020-11-25) 3478 | 3479 | - The bundled extractors are updated to match the versions currently 3480 | used on LGTM.com. These are newer than the last release (1.25) of 3481 | LGTM Enterprise. If you plan to upload databases to an LGTM 3482 | Enterprise 1.25 instance, you need to create them with release 3483 | 2.2.6. 3484 | 3485 | - Much of the work done by `codeql database upgrade` now happens 3486 | implicitly (and reversibly) as part of ordinary query evaluation. 3487 | This should make the need to explicitly run `codeql database 3488 | upgrade` much less common. However there are still some corner cases 3489 | that will require it, particularly for very old databases. 3490 | 3491 | - `codeql test run` with a `--threads` argument will now _compile_ 3492 | test queries in parallel even if they belong to the same single 3493 | test directory. This can speed up localized testing considerably. 3494 | 3495 | ## Release 2.3.3 (2020-11-06) 3496 | 3497 | - The bundled extractors are updated to match the versions currently 3498 | used on LGTM.com. These are newer than the last release (1.25) of 3499 | LGTM Enterprise. If you plan to upload databases to an LGTM 3500 | Enterprise 1.25 instance, you need to create them with release 3501 | 2.2.6. 3502 | 3503 | - Fixed bug in `codeql test run` where the `--format` and 3504 | `--failing-exitcode` options would not work reliably when `--ram` 3505 | was also given 3506 | 3507 | - The `$CODEQL_JAVA_HOME` environment variable will now be passed to 3508 | extractors such that extractors implemented in Java can be affected 3509 | too. Beware that this variable will override the JVM that executes 3510 | the main `codeql` process. It should not normally be set explicitly. 3511 | 3512 | ## Release 2.3.2 (2020-10-27) 3513 | 3514 | - The bundled extractors are updated to match the versions currently 3515 | used on LGTM.com. These are newer than the last release (1.25) of 3516 | LGTM Enterprise. If you plan to upload databases to an LGTM 3517 | Enterprise 1.25 instance, you need to create them with release 3518 | 2.2.6. 3519 | 3520 | ## Release 2.3.1 (2020-10-15) 3521 | 3522 | - The bundled extractors are updated to match the versions currently 3523 | used on LGTM.com. These are newer than the last release (1.25) of 3524 | LGTM Enterprise. If you plan to upload databases to an LGTM 3525 | Enterprise 1.25 instance, you need to create them with release 3526 | 2.2.6. 3527 | 3528 | ### Features added 3529 | 3530 | - `codeql database create` now accepts a `--working-dir` option, which 3531 | allows the working directory for extractor scripts to differ from 3532 | the source root. This is useful in some specialized integration 3533 | situations. 3534 | 3535 | - `codeql database create` will now pass a `--compiler-spec` option on 3536 | to `codeql database trace-command`. This allows adapting the build 3537 | tracing process when unusual compiler toolchains are used. 3538 | 3539 | - `codeql database init` accepts an `--allow-missing-source-root` 3540 | option, which is useful in some specialized integration situations. 3541 | 3542 | ## Release 2.3.0 (2020-09-28) 3543 | 3544 | - The bundled extractors are updated to match the versions currently 3545 | used on LGTM.com. These are newer than the last release (1.25) of 3546 | LGTM Enterprise. If you plan to upload databases to an LGTM 3547 | Enterprise 1.25 instance, you need to create them with release 3548 | 2.2.6. 3549 | 3550 | ### Potentially breaking changes 3551 | 3552 | - The Java extractor no longer supports builds running on a Java 6 3553 | JRE. The minimum supported version is Java 7. 3554 | 3555 | - The interpretation of binding set annotations in QL has changed 3556 | subtly. In rare cases, existing QL code that contains explicit 3557 | binding set annotations on overriding class predicates may now be 3558 | rejected with errors of the form "... is not bound to a value". You 3559 | can fix this by adding explicit binding sets to the overridden 3560 | predicate, or to the abstract class itself in the case of the 3561 | characteristic predicate. For more information about binding sets, 3562 | see [Annotations](https://help.semmle.com/QL/ql-handbook/annotations.html#binding-sets) 3563 | in the QL language reference. 3564 | 3565 | ### QL language improvements 3566 | 3567 | - You can now use binding sets on class bodies. This lets you 3568 | explicitly annotate dynamically dispatched characteristic 3569 | predicates. 3570 | 3571 | ### New features 3572 | 3573 | - Query authors can use the new subcommand `codeql generate query-help` to 3574 | validate query help files and render the files as Markdown. For more information, 3575 | see [Testing query help files](https://help.semmle.com/codeql/codeql-cli/procedures/testing-query-help-files.html). 3576 | 3577 | - The new subcommand `codeql bqrs hash` computes a stable hash of a 3578 | BQRS file. 3579 | 3580 | - `codeql query decompile` now accepts a `--kind` flag. This allows 3581 | advanced users to choose which intermediate representation to show 3582 | for a compiled QL query. `--kind dil` shows the Datalog 3583 | representation while `--kind ra` shows the relational algebra 3584 | representation used by the evaluator. 3585 | 3586 | ## Release 2.2.6 (2020-09-11) 3587 | 3588 | This release corresponds to release 1.25.x of LGTM Enterprise, and 3589 | should be used when creating databases that will be uploaded to it. 3590 | Future CLI releases (numbered 2.3.x) may produce databases that are not 3591 | backwards compatible with this version of LGTM Enterprise. 3592 | 3593 | For all purposes other than creating databases for LGTM Enterprise we 3594 | recommend that you continue upgrading to newer CLI releases as they 3595 | become available. 3596 | 3597 | ## Release 2.2.5 (2020-08-21) 3598 | 3599 | - The bundled extractors are updated to match the versions currently 3600 | used on LGTM.com. These are newer than the last release (1.24) of 3601 | LGTM Enterprise. If you plan to upload databases to an LGTM 3602 | Enterprise 1.24 instance, you need to create them with release 3603 | 2.1.4. 3604 | 3605 | - Updated license terms with a rewritten description of what is and 3606 | is not allowed. No substantive changes are intended, but the new 3607 | text is hopefully easier to understand. 3608 | 3609 | ### New features 3610 | 3611 | - The CLI can now execute queries that use QL's `external predicate` 3612 | feature. All subcommands that execute queries have a new 3613 | `--external` option to specify the value set for those predicates. 3614 | 3615 | - A new `codeql bqrs diff` command can be used to compute the 3616 | difference between two binary query result sets. 3617 | 3618 | - `codeql test run` has some new options to improve support for 3619 | testing of extractors: 3620 | - `--check-databases` which will run `codeql dataset check` on 3621 | every test database produced during a run. 3622 | - `--consistency-queries` which will run a set of additional 3623 | queries over _all_ the test databases produced during a run. 3624 | - `--show-extractor-output` 3625 | 3626 | ## Release 2.2.4 (2020-06-29) 3627 | 3628 | - The bundled extractors are updated to match the versions currently 3629 | used on LGTM.com. These are newer than the last release (1.24) of 3630 | LGTM Enterprise. If you plan to upload databases to an LGTM 3631 | Enterprise 1.24 instance, you need to create them with release 3632 | 2.1.4. 3633 | 3634 | ### Bugs fixed 3635 | 3636 | - QL packs found through the `--search-path` option, or in a sibling 3637 | directory to the unpacked CLI would erroneously take precedence over 3638 | the content of the workspace when using the CodeQL extension for 3639 | Visual Studio Code. This is now fixed such that the workspace 3640 | takes priority. 3641 | 3642 | - Two command-line options that control the amount of disk space that 3643 | the QL evaluator will try to keep free of disk cache are now called 3644 | `--min-disk-free` and `--min-disk-free-pct`. Previously they were 3645 | called `--max-disk-free` instead, which made no sense. The old names 3646 | are still recognized such as not to break existing scripts, but are 3647 | now undocumented and deprecated. 3648 | 3649 | ## Release 2.2.3 (2020-06-15) 3650 | 3651 | CodeQL CLI 2.2.3 is the same as version 2.2.2, but re-released with a new 3652 | version number because the `v2.2.2` folder on the download site 3653 | originally contained the 2.2.0 binaries instead of the correct 2.2.2 3654 | ones. 3655 | 3656 | If you have downloaded release 2.2.2, and `codeql --version` correctly 3657 | identifies itself as being that version, you don't need to upgrade to 3658 | 2.2.3. 3659 | 3660 | ## Release 2.2.2 (2020-06-12) 3661 | 3662 | - The bundled extractors are updated to match the versions currently 3663 | used on LGTM.com. These are newer than the last release (1.24) of 3664 | LGTM Enterprise. If you plan to upload databases to an LGTM 3665 | Enterprise 1.24 instance, you need to create them with release 3666 | 2.1.4. 3667 | 3668 | ### Improvements 3669 | 3670 | - Query evaluations that time out due to a `--timeout` option are no 3671 | longer silently discarded. Instead `codeql` will terminate with exit 3672 | code 33. Commands that evaluate multiple queries will produce as 3673 | much output as they can even if one of the queries times out. 3674 | 3675 | ## Release 2.2.1 3676 | 3677 | There is no CodeQL CLI version 2.2.1. This version number was used 3678 | internally to work around restrictions in the CodeQL for VS Code 3679 | extension. 3680 | 3681 | ## Release 2.2.0 (2020-05-29) 3682 | 3683 | - The bundled extractors are updated to match the versions currently 3684 | used on LGTM.com. These are newer than the last release (1.24) of 3685 | LGTM Enterprise. If you plan to upload databases to an LGTM 3686 | Enterprise 1.24 instance, you need to create them with release 3687 | 2.1.4. 3688 | - Starting with this release, the CodeQL CLI can be downloaded either 3689 | as a single `codeql.zip` file containing the CLI for all supported 3690 | platforms, or as a `codeql-PLATFORM.zip` that contains the files for 3691 | just one platform. The single-platform zips are faster to download. 3692 | 3693 | ### QL language improvement 3694 | 3695 | - QL now supports the definition of new types as type unions. This 3696 | feature currently allows unions of branches from an already existing 3697 | algebraic data type and unions of database types. 3698 | 3699 | ## Release 2.1.4 (2020-05-26) 3700 | 3701 | This release corresponds to release 1.24.x of LGTM Enterprise, and 3702 | should be used when creating databases that will be uploaded to it. 3703 | Future CLI releases (numbered 2.2.x) may produce databases that are not 3704 | backwards compatible with this version of LGTM Enterprise. 3705 | 3706 | For all purposes other than creating databases for LGTM Enterprise we 3707 | recommend that you continue upgrading to newer CLI releases as they 3708 | become available. 3709 | 3710 | ### Features added 3711 | 3712 | - A new `codeql query format` command exposes the QL autoformatter for 3713 | use on the command line. 3714 | 3715 | ### Bugs fixed 3716 | 3717 | - `-J` command-line options that contain spaces now ought to work on 3718 | Windows. They still do not work reliably on Linux or MacOS, though. 3719 | 3720 | ## Release 2.1.3 (2020-05-13) 3721 | 3722 | ### Bugs fixed 3723 | 3724 | - Fixes a bug in `codeql execute cli-server` (a helper used by the VS 3725 | Code extension) which would sometimes cause query compilation to 3726 | fail until the extension was restarted. 3727 | - Fixes a bug in `codeql database upgrade` which could lead to 3728 | performance losses if the upgraded database was subsequently used 3729 | with LGTM or the legacy Semmle Core product. 3730 | - Fixes a bug in the QL evaluator that would sometimes lead to crashes 3731 | for queries that use the new `unique` aggregate added in release 3732 | 2.1.0. 3733 | - The value of the `--compilation-cache-size` option is now correctly 3734 | interpreted as a number of megabytes rather than a number of bytes. 3735 | 3736 | ## Release 2.1.2 (2020-05-06) 3737 | 3738 | - Updated license terms to allow CI use with GitHub Actions for 3739 | open-source software. 3740 | 3741 | ### Potentially breaking changes 3742 | 3743 | - In [query suite definitions](https://help.semmle.com/codeql/codeql-cli/procedures/query-suites.html), filter 3744 | instructions that filter on the `query path` pseudo-tag will now 3745 | always see the relative path to the query expressed with `/` as a 3746 | directory separator, independently of the platform. Previously they 3747 | erroneously used the platform's directory separator, meaning that 3748 | query suites developed on Windows would not work correctly on Unix 3749 | systems (and vice versa) if they used `query path`. Existing suite 3750 | definitions developed on Windows may need to be updated to match the 3751 | new behavior. 3752 | 3753 | ### Features added 3754 | 3755 | - A new `codeql test accept` subcommand helps automate updating the 3756 | expected output for unit tests after a desired change in query 3757 | behavior. This can also be done by the new `--learn` option for 3758 | `codeql test run`. 3759 | 3760 | ### Bugs fixed 3761 | 3762 | - `codeql database create` will now report an explicit error if given 3763 | a `--command` argument that specifies an empty string. Previously 3764 | this would be accepted initially, leading to confusing failures 3765 | later. 3766 | 3767 | ## Release 2.1.1 (2020-04-20) 3768 | 3769 | - The bundled extractors are updated to match the versions currently 3770 | used on LGTM.com. 3771 | 3772 | ### Features added 3773 | 3774 | - `codeql resolve queries` accepts a `--format=bylanguage` option. 3775 | This is used to help automated workflows determine which languages 3776 | to create databases for, from the queries that are available to run. 3777 | - It is now possible to attempt to execute `.ql` files that are not in 3778 | a QL pack. This is used by a few specialized internal workflows. 3779 | However, standalone queries cannot import any of the dependencies 3780 | that you would usually declare in a `qlpack.yml` file, so will not 3781 | be useful in most cases. 3782 | 3783 | ## Release 2.1.0 (2020-03-27) 3784 | 3785 | - The bundled extractors are updated to match the versions currently 3786 | used on LGTM.com. These are newer than the last release (1.23) of 3787 | LGTM Enterprise. If you plan to upload databases to an LGTM 3788 | Enterprise 1.23 instance, you need to create them with release 3789 | 2.0.1. For more information, see [Preparing CodeQL databases to 3790 | upload to 3791 | LGTM](https://help.semmle.com/lgtm-enterprise/admin/help/prepare-database-upload.html) 3792 | in the LGTM admin help. 3793 | 3794 | ### Potentially breaking changes 3795 | 3796 | - If you pass a directory name as a command-line argument to 3797 | `codeql test run`, it will now consider all `.ql` or `.qlref` files 3798 | found under that directory to be test queries, even if they have no 3799 | accompanying `.expected` file. Tests that lack an `.expected` file 3800 | will fail, but will generate an `.actual` file that you can rename 3801 | to `.expected` if you want to use the results. 3802 | 3803 | The goal of this change is to support existing workflows of 3804 | experienced CodeQL users, and also to provide clear error 3805 | indications if an `.expected` file is accidentally lost, renamed, or 3806 | misspelled. 3807 | 3808 | However, if you invoke `codeql test run` on a directory tree that 3809 | contains both tests and non-test queries, you will now encounter 3810 | errors if any of the `.ql` files can't be processed as test queries. 3811 | If you're affected by this change, you can suppress these errors by: 3812 | 3813 | - Adding a `tests` property to this QL pack to define specify 3814 | which directories contain only test queries and associated test 3815 | code. For more information, see 3816 | [About QL packs](https://help.semmle.com/codeql/codeql-cli/reference/qlpack-overview.html). 3817 | - Running `codeql test run` with a new `--strict-test-discovery` 3818 | option. 3819 | 3820 | In the longer term, we recommend that you reorganize the queries so 3821 | that test queries are stored in a directory tree that's separate 3822 | from actual queries. 3823 | 3824 | - `codeql database create` and `codeql database finalize` will no 3825 | longer recognize a `--no-duplicate-code` option. This option has 3826 | never had any effect, and its positive variant `--duplicate-code` 3827 | previously led to a fatal error. 3828 | 3829 | ### Features added 3830 | 3831 | - A new XML extractor is included. It is not intended to be used as a 3832 | stand-alone extractor, but rather to augment the data produced by 3833 | other extractors. In particular, the C\# and Java extractors invoke 3834 | it during database creation to include information relevant to the 3835 | analysis of those languages, much like LGTM.com does. 3836 | - Two new plumbing commands `codeql database index-files` and 3837 | `codeql resolve files` have been added for support of invoking the 3838 | XML extractor support. These commands are generally only of interest 3839 | for extractor authors. 3840 | - Two new plumbing commands have been added to `codeql dataset`. The 3841 | `measure` subcommand can be used to collect size information from a 3842 | dataset, and the `check` subcommand can scan a dataset for database 3843 | inconsistencies. These commands are useful when developing a new 3844 | CodeQL extractor. 3845 | - The QL evaluator contains a number of features in support of an 3846 | internal experiment with using machine-learning techniques to 3847 | identify functions in unknown codebases as sources or sinks of 3848 | taint. This includes new command-line options `--ml-model-path` and 3849 | `--native-library-path` to several subcommands. As the new features 3850 | are not yet ready for general use, these new options should be 3851 | ignored by external CodeQL users. 3852 | 3853 | ### Bugs fixed 3854 | 3855 | - Fixes a bug that could result in empty databases for C/C++. 3856 | Previously, extraction would mistakenly be skipped for source files 3857 | compiled with the Clang compiler, if the `-fintegrated-cc1` option 3858 | was specified. 3859 | - `codeql database create` and `codeql database init` will now, as 3860 | they have always been documented, refuse to create a database whose 3861 | parent directory doesn't already exist. 3862 | - `codeql test run` will no longer leave `.actual` files from previous 3863 | runs in the file system after a test passes. 3864 | 3865 | ### QL language improvements 3866 | 3867 | - QL now supports set literals, and the QL extractor can identify them 3868 | with the `SetLiteral` class. For more information, see [Set literal 3869 | expressions](https://help.semmle.com/QL/ql-handbook/expressions.html#set-literal-expressions) 3870 | in the QL language reference. 3871 | - QL now supports a uniqueness aggregate. This can express constraints 3872 | that there is precisely one value. The syntax is taken from previous 3873 | aggregates such as `min` and `max`. 3874 | 3875 | ``` {.sourceCode .ql} 3876 | unique(int x | x = 4 or x = 2 * 2 | x) 3877 | ``` 3878 | 3879 | ## Release 2.0.6 (2020-03-16) 3880 | 3881 | ### Bugs fixed 3882 | 3883 | - Fixes a problem preventing `codeql database create` from working 3884 | with Python 3 on macOS. 3885 | - Fixes a problem preventing `codeql database create` from finding 3886 | locally installed Python packages. 3887 | 3888 | ## Release 2.0.5 (2020-03-13) 3889 | 3890 | - The bundled extractors (which are responsible for converting source 3891 | code to databases for each supported language) are updated to match 3892 | the versions currently used on LGTM.com. These are newer than the 3893 | last release of LGTM Enterprise, so this release should not be used 3894 | if you plan to upload databases to an LGTM Enterprise instance. For 3895 | more information, see [Preparing CodeQL databases to upload to 3896 | LGTM](https://help.semmle.com/lgtm-enterprise/admin/help/prepare-database-upload.html) 3897 | in the LGTM admin help. 3898 | 3899 | ### Features added 3900 | 3901 | - `codeql test run` has a new `--slice` option that can be used to 3902 | parallelize tests over more machines. 3903 | 3904 | ## Release 2.0.4 (2020-02-21) 3905 | 3906 | - The bundled extractors (which are responsible for converting source 3907 | code to databases for each supported language) are updated to match 3908 | the versions currently used on LGTM.com. These are newer than the 3909 | last release of LGTM Enterprise, so this release should not be used 3910 | if you plan to upload databases to an LGTM Enterprise instance. For 3911 | more information, see [Preparing CodeQL databases to upload to 3912 | LGTM](https://help.semmle.com/lgtm-enterprise/admin/help/prepare-database-upload.html) 3913 | in the LGTM admin help. 3914 | 3915 | ### Features added 3916 | 3917 | - Subcommands that execute queries (such as `codeql database analyze`) 3918 | now have a `--timeout` option that can be used to set a timeout to 3919 | automatically cancel query evaluations that appear to diverge. 3920 | - A new plumbing command `codeql query decompile` can display the DIL 3921 | intermediate representations that is included in the output of 3922 | `codeql query compile --dump-qlo --include-dil-in-qlo`. This is 3923 | useful mainly for certain internal workflows; the information 3924 | produced is the same as what `codeql query compile --dump-dil` 3925 | already outputs. 3926 | 3927 | ### Bugs fixed 3928 | 3929 | - The `--debug` and `--tuple-counting` options to 3930 | `codeql test run` erroneously had no effect. Now they ought to work. 3931 | 3932 | ## Release 2.0.3 (2020-02-12) 3933 | 3934 | ### Bugs fixed 3935 | 3936 | - Fixes a bug where `codeql test run` would fail with the 3937 | message 3938 | `CatastrophicError: There should be a --library-path option for com.semmle.cli2.LibraryPathOptions.libraryPath but we didn't find it` 3939 | when running tests against the `master` branch of the CodeQL 3940 | libraries for certain languages. 3941 | - Otherwise identical to release 2.0.2. 3942 | 3943 | ## Release 2.0.2 (2020-02-05) 3944 | 3945 | - The bundled extractors (which are responsible for converting source 3946 | code to databases for each supported language) are updated to match 3947 | the versions currently used on LGTM.com. These are newer than the 3948 | last release of LGTM Enterprise, so this release should not be used 3949 | if you plan to upload databases to an LGTM Enterprise instance. For 3950 | more information, see [Preparing CodeQL databases to upload to 3951 | LGTM](https://help.semmle.com/lgtm-enterprise/admin/help/prepare-database-upload.html) 3952 | in the LGTM admin help. 3953 | - The parent and sibling directories of the unpacked CLI are no longer 3954 | searched recursively for QL packs. QL packs will only be found if 3955 | there's a `qlpack.yml` or `.codeqlmanifest.json` directly in a 3956 | parent or sibling directory. This should eliminate the very long 3957 | disk-scanning delays experienced by users who unpacked earlier 3958 | versions of the CLI in their home directory. 3959 | - Parent and sibling directories of the unpacked CLI will now be 3960 | searched for QL packs as a last resort, even if you give an explicit 3961 | `--search-path` option. This means, for example, that you can define 3962 | a search path in the [per-user configuration file](https://help.semmle.com/codeql/codeql-cli/reference/configuration-overview.html) without it depending on 3963 | where the CLI is unpacked. In particular, the setting can now be 3964 | meaningfully used by users who let the CodeQL for VS Code extension 3965 | manage the downloading and unpacking of the CLI. 3966 | 3967 | ### Security updates 3968 | 3969 | - The `codeql database create` command and its relatives will no 3970 | longer attempt to find extractors located in the parent and sibling 3971 | directories of the unpacked CLI. This closes a security risk for 3972 | users who unpacked the CodeQL CLI in their home directory. This 3973 | could've resulted in arbitrary code execution if the user unpacked a 3974 | file archive containing a malicious extractor anywhere in the home 3975 | directory. Extractors will now only be found within the unpacked CLI 3976 | itself, or in directories explicitly listed in the `--search-path`. 3977 | It is expected that users will only point `--search-path` to 3978 | locations they trust at least as much as the CLI download itself. 3979 | 3980 | ### Features added 3981 | 3982 | - This release supports executing query regression tests using the 3983 | `codeql test` command. For further information, see 3984 | [Testing custom queries](https://help.semmle.com/codeql/codeql-cli/procedures/test-queries.html). 3985 | - The error message if you try executing a query against a database 3986 | that needs to be upgraded (which can happen routinely if you're 3987 | using a fresh `master` checkout of the CodeQL libraries with the 3988 | bundled extractors) will now explicitly suggest a 3989 | `codeql database update` command to run. The database is not 3990 | automatically upgraded, as this may make it irreversibly 3991 | incompatible with older versions of the CodeQL libraries. This 3992 | allows users who want to compare behavior of different versions of 3993 | the libraries against the same database to make a copy before they 3994 | upgrade it. 3995 | 3996 | ## Release 2.0.1 (2019-12-17) 3997 | 3998 | - Corresponds to LGTM Enterprise release 1.23. 3999 | - The bundled extractors (which are responsible for converting source 4000 | code to databases for each supported language) are updated to match 4001 | the extractor versions used in LGTM Enterprise. 4002 | - No other changes to the core CLI. 4003 | 4004 | ## Release 2.0.0 (2019-11-14) 4005 | 4006 | - First public release. 4007 | 4008 | --------------------------------------------------------------------------------