├── .github ├── ISSUE_TEMPLATE │ └── config.yml └── workflows │ └── label-issue.yml ├── CHANGELOG.md ├── LICENSE.md └── README.md /.github/ISSUE_TEMPLATE/config.yml: -------------------------------------------------------------------------------- 1 | blank_issues_enabled: false 2 | contact_links: 3 | - name: Report an issue or ask a question about CodeQL 4 | url: https://github.com/github/codeql/issues/new/choose 5 | about: Please create issues and ask questions in the `github/codeql` repository. 6 | -------------------------------------------------------------------------------- /.github/workflows/label-issue.yml: -------------------------------------------------------------------------------- 1 | name: Label issue 2 | on: 3 | issues: 4 | types: [opened] 5 | 6 | jobs: 7 | label: 8 | name: Label issue 9 | runs-on: ubuntu-latest 10 | if: github.event.action == 'opened' 11 | steps: 12 | - name: Label issue 13 | env: 14 | GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} 15 | run: | 16 | echo '{"labels": ["CLI"]}' | gh api repos/${{ github.repository }}/issues/${{ github.event.issue.number }}/labels --input - 17 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # CodeQL CLI changelog 2 | 3 | 19 | 20 | ## Release 2.21.4 (2025-06-02) 21 | 22 | ### Deprecations 23 | 24 | - The `clang_vector_types`, `clang_attributes`, and `flax-vector-conversions` command 25 | line options have been removed from the C/C++ extractor. These options were introduced 26 | as workarounds to frontend limitations in earlier versions of the extractor and are 27 | no longer needed when calling the extractor directly. 28 | 29 | ### Miscellaneous 30 | 31 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL 32 | CLI has been updated to version 21.0.7. 33 | 34 | ## Release 2.21.3 (2025-05-15) 35 | 36 | ### Miscellaneous 37 | 38 | - Windows binaries for the CodeQL CLI are now built with `/guard:cf`, enabling [Control Flow Guard](https://learn.microsoft.com/en-us/windows/win32/secbp/control-flow-guard). 39 | 40 | ## Release 2.21.2 (2025-05-01) 41 | 42 | ### Bugs fixed 43 | 44 | - `codeql generate log-summary` now correctly includes `dependencies` 45 | maps in predicate events for `COMPUTED_EXTENSIONAL` predicates. 46 | 47 | ## Release 2.21.1 (2025-04-22) 48 | 49 | ### Bugs fixed 50 | 51 | - Fixed a bug in CodeQL analysis for GitHub Actions in the presence 52 | of a code scanning configuration file containing `paths-ignore` 53 | exclusion patterns but not `paths` inclusion patterns. 54 | Previously, such a configuration incorrectly led to all YAML, HTML, 55 | JSON, and JS source files being extracted, 56 | except for those filtered by `paths-ignore`. 57 | This in turn led to performance issues on large codebases. 58 | Now, only workflow and Action metadata YAML files relevant to the 59 | GitHub Actions analysis will be extracted, 60 | except for those filtered by `paths-ignore`. 61 | This matches the default behavior when no configuration file 62 | is provided. 63 | The handling of `paths` inclusion patterns is unchanged: 64 | if provided, only those paths will be considered, 65 | except for those filtered by `paths-ignore`. 66 | 67 | ## Release 2.21.0 (2025-04-03) 68 | 69 | ### Miscellaneous 70 | 71 | - On macOS the `CODEQL_TRACER_RELOCATION_EXCLUDE` environment variable can now be used to exclude certain paths from the 72 | tracer relocation and tracing process. This environment variable accepts newline-separated regex patterns of binaries 73 | to be excluded. 74 | 75 | ## Release 2.20.7 (2025-03-18) 76 | 77 | - There are no user-facing changes in this release. 78 | 79 | ## Release 2.20.6 (2025-03-06) 80 | 81 | ### Miscellaneous 82 | 83 | - The CodeQL XML extractor is now able to parse documents in a wider array of 84 | character sets. 85 | 86 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL 87 | CLI has been updated to version 21.0.6. 88 | 89 | ## Release 2.20.5 (2025-02-20) 90 | 91 | ### Breaking changes 92 | 93 | - Removed support for `QlBuiltins::BigInt`s in the `avg()` aggregate. 94 | 95 | - A number of breaking changes have been made to the C and C++ CodeQL test environment as used by `codeql test run`: 96 | - The `-Xclang-only=` option is no longer supported by `semmle-extractor-options`. Instead, when either `--clang` or `--clang_version` is specified the option should be replaced by `` only, otherwise the option should be omitted. 97 | - The `--sys_include ` and `--preinclude ` options are no longer supported by `semmle-extractor-options`. Instead, `--edg --edg ` should be specified. 98 | - The `-idirafter ` option is no longer supported by `semmle-extractor-options`. Instead, `--edg --sys_include --edg ` should be specified. 99 | - The `-imacros ` option is no longer supported by `semmle-extractor-options`. Instead, `--edg --preinclude_macros --edg ` should be specified. 100 | - The `/FI ` option is no longer supported by `semmle-extractor-options`. Instead, `--edg --preinclude --edg ` should be specified. 101 | - The `-Wreserved-user-defined-literal`, `-Wno-reserved-user-defined-literal`, `-fwritable-strings`, `/Zc:rvalueCast`, `/Zc:rvalueCast-`, and `/Zc:wchar_t-` options are no longer supported by `semmle-extractor-options`. Instead, `--edg --reserved_user_defined_literal`, `--edg --no-reserved_user_defined_literal`, `--edg --no_const_string_literals`, `--edg --no_preserve_lvalues_with_same_type_casts`, `--edg --preserve_lvalues_with_same_type_casts`, and `--edg --no_wchar_t_keyword` should be specified, respectively. 102 | - The `/Fo ` option is no longer supported by `semmle-extractor-options`. The option should be omitted. 103 | 104 | ## Release 2.20.4 (2025-02-06) 105 | 106 | ### New features 107 | 108 | - Using the `actions` language (for analysis of GitHub Actions workflows) no longer requires 109 | the `CODEQL_ENABLE_EXPERIMENTAL_FEATURES` environment variable to be set. Support for analysis 110 | of GitHub Actions workflows remains in public preview. 111 | 112 | ### Bugs fixed 113 | 114 | - Fixed a bug where CodeQL for Java would fail with an SSL exception while trying to download `maven`. 115 | 116 | ### Miscellaneous 117 | 118 | - The build of the [logback-core](https://logback.qos.ch/) library that is used for logging in the CodeQL CLI has been updated to version 1.3.15. 119 | 120 | ## Release 2.20.3 (2025-01-24) 121 | 122 | ### Security Updates 123 | 124 | - Resolves a security vulnerability where CodeQL databases or logs produced by the CodeQL CLI may contain the environment variables from the time of 125 | database creation. This includes any secrets stored in an environment variables. For more information, see the 126 | [CodeQL CLI security advisory](https://github.com/github/codeql-cli-binaries/security/advisories/GHSA-gqh3-9prg-j95m). 127 | 128 | All users of CodeQL should follow the advice in the CodeQL advisory mentioned above or upgrade to this version or a later version of CodeQL. 129 | 130 | If you are using the CodeQL Action, also see the related [CodeQL Action security advisory](https://github.com/github/codeql-action/security/advisories/GHSA-vqf5-2xx6-9wfm). 131 | 132 | ## Release 2.20.2 (2025-01-22) 133 | 134 | ### Improvements 135 | 136 | - `codeql database create` and `codeql database finalize` now write relations to disk in a new, compressed format. As a result, databases will generally take up less space on disk, whether zipped or unzipped. Note that databases in this format can only be read and analyzed using CodeQL version 2.20.1 onwards. Attempting to analyze such a database with CodeQL version 2.20.0 or older will fail, with an error message like the following: 137 | ``` 138 | UnsortedExtensionalError: Tuples that were assumed to be in order are not: [123456777, 777654321, 123456777]<[777654321, 123456777, 777654321] 139 | ``` 140 | 141 | ### Enhancements 142 | 143 | - Added the `.bitLength()` method to `QlBuiltins::BigInt`. 144 | 145 | ### Bugs Fixed 146 | 147 | - Fixed a bug where CodeQL would crash on rare occasions while merging SARIF files before uploading results. 148 | 149 | ## Release 2.20.1 (2025-01-09) 150 | 151 | ### Improvements 152 | 153 | - Automatic installation of dependencies for C++ autobuild is now supported on Ubuntu 24.04. 154 | 155 | - The CLI will now warn if it detects that it is installed in a 156 | location where it is likely to cause performance issues. This 157 | includes: user home, desktop, downloads, or the file system root. 158 | 159 | You can avoid this warning by setting the `CODEQL_ALLOW_INSTALLATION_ANYWHERE` 160 | environment variable to `true`. 161 | 162 | ## Release 2.20.0 (2024-12-09) 163 | 164 | ### Known issues 165 | 166 | - The Windows executable for this release is labeled with an incorrect version number 167 | within its properties: the version number should be 2.20.0 rather than 2.19.4. 168 | `codeql version` reports the correct version number. 169 | 170 | ### New features 171 | 172 | - The [`QlBuiltins::BigInt` type](https://codeql.github.com/docs/ql-language-reference/modules/#bigint) of 173 | arbitrary precision integers is generally available and no longer hidden behind the 174 | `--allow-experimental=bigint` CLI feature flag. 175 | 176 | ### Miscellaneous 177 | 178 | - Backslashes are now escaped when writing output in the Graphviz DOT format (`--format=dot`). 179 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL CLI has been updated to version 180 | 21.0.5. 181 | 182 | ## Release 2.19.4 (2024-12-02) 183 | 184 | ### Improvements 185 | 186 | - CodeQL now supports passing values containing the equals character (`=`) to extractor options via 187 | the `--extractor-option` flag. This allows cases like `--extractor-option opt=key=value`, which 188 | sets the extractor option `opt` to hold the value `key=value`, whereas previously that would have 189 | been rejected with an error. 190 | - The `codeql pack bundle` command now sets the numeric user and group IDs of entries in the generated 191 | `tar` archive to `0`. This avoids failures like `IllegalArgumentException: user id '7111111' is too big ( > 2097151 )` 192 | when the numeric user ID is too large. 193 | 194 | ### Bugs fixed 195 | 196 | - On MacOS, `arch -arm64` commands no longer fail when they are executed via `codeql database create --command`, 197 | via `codeql database trace-command`, or are run after `codeql database init --begin-tracing`. Note 198 | that build commands invoked this way still will not normally be traced, so this is useful only for 199 | running ancillary commands which are incidental to building your code. 200 | - Fixed a bug where `codeql test run` would not preserve test 201 | databases on disk after a test failed. 202 | 203 | ## Release 2.19.3 (2024-11-07) 204 | 205 | ### Bugs fixed 206 | 207 | - Fixed a bug where using `codeql database import` to combine multiple non-empty 208 | databases may produce a corrupted database. (The bug does not affect using 209 | `codeql database finalize --additional-dbs` to combine multiple databases.) 210 | 211 | - Fixed a bug where uses of a `QlBuiltins::ExtensionId` variable that was not 212 | bound to a value could be incorrectly accepted in some cases. In many cases, 213 | this would result in a crash. 214 | 215 | - CodeQL would sometimes refuse to run with more than around 1,500 GB of RAM 216 | available, complaining that having so much memory was "unrealistic". The 217 | amount of memory CodeQL is able to make any meaningful use of still tops out 218 | at about that value, but it will now gracefully accept that so large 219 | computers do in fact exist. 220 | 221 | - Fixed a bug in command-line parsing where a misspelled option could sometimes 222 | be misinterpreted as, e.g., the name of a query to run. Now every command-line 223 | argument that begins with a dash is assumed to be intended as an option 224 | (unless it comes after the `--` separator), and an appropriate error is 225 | emitted if that is not a recognized one. 226 | 227 | The build command in `codeql database trace-command` is exempted from this for 228 | historical reasons, but we strongly recommend putting a `--` before the entire 229 | build command there, in case a future `codeql` version starts recognizing 230 | options that you intended to be part of the build command. 231 | 232 | ### Miscellaneous 233 | 234 | - The CodeQL Bundle is now available as an artifact that is compressed using 235 | [Zstandard](https://en.wikipedia.org/wiki/Zstd). This artifact is 236 | smaller and faster to decompress than the original, gzip-compressed bundle. The CodeQL bundle 237 | is a tar archive containing tools, scripts, and various CodeQL-specific files. 238 | 239 | If you are currently using the CodeQL Bundle, you may want to consider switching to the 240 | Zstandard variant of the bundle. You can download the new form of the CodeQL Bundle from the 241 | [codeql-action releases page](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.19.3) 242 | by selecting the appropriate bundle with the `.zst` extension. The gzip-compressed bundles will 243 | continue to be available for backwards compatibility. 244 | 245 | ## Release 2.19.2 (2024-10-21) 246 | 247 | ### Potentially breaking changes 248 | 249 | - The Python extractor will no longer extract the standard library by default, relying instead on models of the standard library. This should result in significantly faster extraction and analysis times, while the effect on alerts should be minimal. It will for a while be possible to force extraction of the standard library by setting the environment variable `CODEQL_EXTRACTOR_PYTHON_EXTRACT_STDLIB` to `1`. 250 | 251 | ### Bugs fixed 252 | 253 | - The 2.19.1 release contained a bug in the query evaluator that under rare conditions could lead to wrong alerts or resource exhaustion. Although we have never seen the problem outside of internal testing, we encourage users on 2.19.1 to upgrade to 2.19.2. 254 | 255 | ### Miscellaneous 256 | 257 | - The database relation `sourceLocationPrefix` is changed for databases created with 258 | `codeql test run`. Instead of containing the path of the enclosing qlpack, it now 259 | contains the actual path of the test, similar to if one had run `codeql database create` 260 | on the test folder. For example, for a test such as 261 | `/cpp/ql/test/query-tests/Security/CWE/CWE-611/XXE.qlref` we now populate 262 | `sourceLocationPrefix` with `/cpp/ql/test/query-tests/Security/CWE/CWE-611/` 263 | instead of `/cpp/ql/test/`. This change typically impacts calls to 264 | `File.getRelativePath()`, and may as a result change the expected test output. 265 | 266 | ## Release 2.19.1 (2024-10-04) 267 | 268 | ### New Features 269 | 270 | - The command `codeql generate query-help` now supports Markdown help files. 271 | The Markdown help format is commonly used in custom CodeQL query packs. This new 272 | feature allows us to generate SARIF reporting descriptors for CodeQL queries that 273 | include Markdown help directly from a query Markdown help file. 274 | 275 | - Added a new command, `codeql resolve packs`. This command shows each step in the 276 | pack search process, including what packs were found in each step. With the 277 | `--show-hidden-packs` option, it can also show details on which packs were hidden 278 | by packs found earlier in the search sequence. `codeql resolve packs` is intended 279 | as a replacement for most uses of `codeql resolve qlpacks`, whose output is both 280 | less detailed and less accurate. 281 | 282 | ## Release 2.19.0 (2024-09-18) 283 | 284 | ### Improvements 285 | 286 | - `codeql database analyze` and `codeql database interpret-results` now support 287 | the `--sarif-run-property` option. You can provide this option when using a SARIF 288 | output format to add a key-value pair to the property bag of the run object. 289 | 290 | ### Miscellaneous 291 | 292 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL 293 | CLI has been updated to version 21.0.4. 294 | 295 | ## Release 2.18.4 (2024-09-12) 296 | 297 | ### New Features 298 | 299 | - C# support for `build-mode: none` is now out of beta, and generally available. 300 | - Go 1.23 is now supported. 301 | 302 | ## Release 2.18.3 (2024-08-28) 303 | 304 | - There are no user-facing changes in this release. 305 | 306 | ## Release 2.18.2 (2024-08-13) 307 | 308 | ### Deprecations 309 | 310 | - Swift analysis on Ubuntu is no longer supported. Please migrate to macOS if this affects you. 311 | 312 | ### Miscellaneous 313 | 314 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL 315 | CLI has been updated to version 21.0.3. 316 | 317 | ## Release 2.18.1 (2024-07-25) 318 | 319 | ### Security Updates 320 | 321 | - Resolves CVE-2023-4759, an arbitrary file overwrite in Eclipse JGit 322 | that can be triggered when using untrusted third-party queries from a 323 | git repository. See the 324 | [security advisory](https://github.com/github/codeql-cli-binaries/security/advisories/GHSA-x4gx-f2xv-6wj9) 325 | for more information. 326 | - The following dependencies have been updated. These updates include 327 | security fixes in the respective libraries that prevent 328 | out-of-bounds accesses or denial-of-service in scenarios where 329 | untrusted files are processed. These scenarios are not likely to be 330 | encountered in most uses of CodeQL and code scanning, and only 331 | apply to advanced use cases where precompiled query packs, 332 | database ZIP files, or database TRAP files are obtained from 333 | untrusted sources and then processed on a trusted machine. 334 | - airlift/aircompressor is updated to version 0.27. 335 | - Apache Ant is updated to version 1.10.11. 336 | - Apache Commons Compress is updated to version 1.26.0. 337 | - Apache Commons IO is updated to version 2.15.1. 338 | - Apache Commons Lang3 is updated to version 3.14.0. 339 | - jsoup is updated to version 1.15.3. 340 | - Logback is updated to version 1.2.13. 341 | - Snappy is updated to version 0.5. 342 | 343 | ### New features 344 | 345 | - The *experimental* type `QlBuiltins::BigInt` of arbitrary-precision integers 346 | has been introduced. To opt in to this API, compile your queries with 347 | `--allow-experimental=bigint`. Big integers can be constructed using the 348 | `.toBigInt()` methods of `int` and `string`. The built-in operations are: 349 | - comparisons: `=`, `!=`, `<`, `<=`, `>`, `>=`, 350 | - conversions: `.toString()`, `.toInt()`, 351 | - arithmetic: binary `+`, `-`, `*`, `/`, `%`, unary `-`, 352 | - bitwise operations: `.bitAnd(BigInt)`, `.bitOr(BigInt)`, 353 | `.bitXor(BigInt)`, `.bitShiftLeft(int)`, `.bitShiftRightSigned(int)`, 354 | `.bitNot()`, 355 | - aggregates: `min`, `max`, (`strict`)`sum`, (`strict`)`count`, `avg`, 356 | `rank`, `unique`, `any`. 357 | - other: `.pow(int)`, `.abs()`, `.gcd(BigInt)`, `.minimum(BigInt)`, 358 | `.maximum(BigInt)`. 359 | - `codeql test run` now supports postprocessing of test results. When .qlref 360 | files specify a path to a `postprocess` query, then this is evaluated after 361 | the test query to transform the test outputs prior to concatenating them into 362 | the `actual` results. 363 | 364 | ### Improvements 365 | 366 | - The 30% QL query compilation slowdown noted in 2.18.0 has been fixed. 367 | 368 | ## Release 2.18.0 (2024-07-11) 369 | 370 | ### Breaking changes 371 | 372 | - A number of breaking changes have been made to the C and C++ CodeQL 373 | test environment as used by `codeql test run`: 374 | - The test environment no longer defines any GNU-specific builtin 375 | macros. If these macros are still needed by a test, please define 376 | them via `semmle-extractor-options`. 377 | - The `--force-recompute` option is no longer directly supported by 378 | `semmle-extractor-options`. Instead, `--edg --force-recompute` 379 | should be specified. 380 | - The `--gnu_version` and `--microsoft_version` options that can be 381 | specified via `semmle-extractor-options` are now synonyms, and only 382 | one should be specified as part of `semmle-extractor-options`. 383 | Furthermore, is also no longer possible to specify these options 384 | via the following syntax. 385 | 386 | - `--edg --gnu_version --edg `, and 387 | - `--edg --microsoft_version --edg ` 388 | 389 | The shorter `--gnu_version ` and 390 | `--microsoft_version ` should be used. 391 | - The `--build_error_dir` and `--predefined_macros` command line options 392 | have been removed from the C/C++ extractor. It has never been possible 393 | to pass these options through the CLI, but some customers with advanced 394 | setups may have been passing them through internal undocumented interfaces. 395 | Passing the option `--build_error_dir` did not have any effect, and it 396 | is safe to remove the option. The `--predefined_macros` option should 397 | have been unnecessary, as long as the extractor was invoked with the 398 | `--mimic` option. 399 | 400 | ### Regressions 401 | 402 | - Compilation of QL queries is about 30% slower than in previous releases. This only affects users who write custom queries, and only at compilation time, not at run time. This regression will be fixed in the upcoming 2.18.1 release. 403 | 404 | ### Improvements 405 | 406 | - Introduced the `--include-logs` option to the `codeql database bundle` 407 | command. This new feature allows users to include logs in the generated 408 | database bundle, allowing for a more complete treatment of the bundle, and 409 | bringing the tool capabilities up-to-speed with the documentation. 410 | - `codeql database init` and `codeql database create` now support the 411 | `--force-overwrite` option. When this option is specified, the command will 412 | delete the specified database directory even if it does not look like a 413 | database directory. This option is only recommended for automation. For 414 | directcommand line commands, it is recommended to use the `--overwrite` 415 | option, which includes extra protection and will refuse to delete a 416 | directory that does not look like a database directory. 417 | - Extract `.xsaccess`, `*.xsjs` and `*.xsjslib` files for SAP HANA XS as 418 | Javascript. 419 | - We have updated many compiler error messages and warnings to improve their 420 | readability and standardize their grammar. 421 | Where necessary, please use the `--learn` option for the `codeql test run` 422 | command. 423 | 424 | ### Bugs fixed 425 | 426 | - Where a MacOS unsigned binary cannot be signed, CodeQL will now continue 427 | trying to trace compiler invocations created by that process and its 428 | children. In particular this means that Bazel builds on MacOS are now 429 | traceable. 430 | - Fixed a bug where test discovery would fail if there is a syntax error in a 431 | qlpack file. Now, a warning message will be printed and discovery will 432 | continue. 433 | 434 | ## Release 2.17.6 (2024-06-27) 435 | 436 | ### New features 437 | 438 | - Beta support is now available for analyzing C# codebases without needing a working build. To use 439 | this, pass the `--build-mode none` option to `codeql database create`. 440 | 441 | ### Improvements 442 | 443 | - The `--model-packs` option is now publicly available. This option allows commands like `codeql database analyze` 444 | to accept a list of model packs that are used to augment the analysis of all queries involved in the analysis. 445 | 446 | ## Release 2.17.5 (2024-06-12) 447 | 448 | ### Breaking changes 449 | 450 | - All the commands that output SARIF will output a minified version to reduce the size. 451 | The `codeql database analyze`, `codeql database interpret-results`, `codeql generate query-help`, and `codeql bqrs interpret` commands support the option `--no-sarif-minify` to output a pretty printed SARIF file. 452 | 453 | - A number of breaking changes have been made to the `semmle-extractor-options` 454 | functionality available for C and C++ CodeQL tests. 455 | 456 | - The Arm, Intel, and CodeWarrior compilers are no longer supported and the 457 | `--armcc`, `--intel`, `--codewarrior` flags are now ignored, as are all the 458 | flags that only applied to those compilers. 459 | - The `--threads` and `-main-file-name` options, which did not have any effect 460 | on tests, are now ignored. Any specification of these options as part of 461 | `semmle-extractor-options` should be removed. 462 | - Support for `--linker`, all flags that would only invoke the preprocessor, 463 | and the `/clr` flag have been removed, as those flags would never produce any 464 | usable test output. 465 | - Support for the `--include_path_environment` flag has been removed. All include 466 | paths should directly be specified as part of `semmle-extractor-options`. 467 | - Microsoft C/C++ compiler response files specified via `@some_file_name` are 468 | now ignored. Instead, all options should directly be specified as part of 469 | `semmle-extractor-options`. 470 | - Support for Microsoft `#import` preprocessor directive has been removed, as 471 | support depends on the availability of the Microsoft C/C++ compiler, and 472 | availability cannot be guaranteed on all platforms while executing tests. 473 | - Support for the Microsoft `/EHa`, `/EHs`, `/GX`, `/GZ`, `/Tc`, `/Tp`, and `/Zl` 474 | flags, and all `/RTC` flags have been removed. Any specification of these 475 | options as part of `semmle-extractor-options` should be removed. 476 | - Support for the Apple-specific `-F` and `-iframework` flags has been removed. 477 | The `-F` flag can still be used by replacing `-F ` by 478 | `--edg -F --edg `. Any occurrence of `-iframework ` should be 479 | replaced by `--edg --sys_framework --edg `. 480 | - Support for the `/TC`, `/TP`, and `-x` flags has been removed. Please ensure 481 | all C, respectively C++, source files have a `.c`, respectively `.cpp`, 482 | extension. 483 | - The `--build_error_dir`, `-db`, `--edg_base_dir`, `--error_limit`, 484 | `--src_archive`, `--trapfolder`, and `--variadic_macros` flags are now ignored. 485 | 486 | The above changes do not affect the creation of databases through the CodeQL CLI, 487 | or when calling the C/C++ extractor directly with the `--mimic` or `--linker` flags. 488 | Similar functionality continues to be supported in those scenarios, except for 489 | CodeWarrior and the `--edg_base_dir`, `--include_path_environment`, `/Tc`, and `/Tp` 490 | flags, which were never supported. 491 | 492 | ### Improvements 493 | 494 | - `codeql generate log-summary` now reports completed pipeline runs that 495 | are part of an incomplete recursive predicate. 496 | 497 | ### Miscellaneous 498 | 499 | - The OWASP Java HTML Sanitizer library used by the CodeQL CLI for internal 500 | documentation generation commands has been updated to version 501 | [20240325.1](https://github.com/OWASP/java-html-sanitizer/releases/tag/release-20240325.1). 502 | 503 | ## Release 2.17.4 (2024-06-03) 504 | 505 | ### New features 506 | 507 | - CodeQL package management is now generally available, and all GitHub-produced 508 | CodeQL packages have had their version numbers increased to 1.0.0. 509 | 510 | ## Release 2.17.3 (2024-05-17) 511 | 512 | ### Improvements 513 | 514 | - The language server that our IDE integration is built on now defaults 515 | to fine-grained dependency tracking for incremental error-checking 516 | after file changes. This slightly improves the latency of refreshing 517 | errors after local source code edits and will enable significant 518 | speedups in the future. 519 | - We now properly handle globs (such as `folder/**/*.py`) in `paths` configuration 520 | to specify what files to include for Python analysis (see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#specifying-directories-to-scan). 521 | - TRAP import (a part of `codeql database create` and `codeql database finalize`) 522 | now supports allocating 2^32 IDs during the import process. The previous limit 523 | was 2^31 IDs. 524 | 525 | ## Release 2.17.2 (2024-05-07) 526 | 527 | ### Known issues 528 | 529 | - The beta support for analyzing Swift in this release and all 530 | previous releases requires `g++-13` when running on Linux. Users 531 | analyzing Swift using the `ubuntu-latest`, `ubuntu-22.04`, or 532 | `ubuntu-20.04` runner images for GitHub Actions should update their 533 | workflows to install `g++-13`. For more information, see [the runner 534 | images 535 | announcement](https://github.com/actions/runner-images/issues/9679). 536 | 537 | ### Improvements 538 | 539 | - When uploading a SARIF file to GitHub using `codeql github 540 | upload-results`, the CodeQL CLI now waits for the file to be 541 | processed by GitHub. If any errors occurred during processing of the 542 | analysis results, the command will log these and return a non-zero 543 | exit code. To disable this behaviour, pass the 544 | `--no-wait-for-processing` flag. 545 | 546 | By default, the command will wait for the SARIF file to be processed 547 | for a maximum of 2 minutes, however this is configurable with the 548 | `--wait-for-processing-timeout` option. 549 | - The build tracer is no longer enabled when using the [`none` build 550 | mode](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#codeql-build-modes) 551 | to analyze a compiled language, thus improving performance. 552 | 553 | ## Release 2.17.1 (2024-04-24) 554 | 555 | ### Deprecations 556 | 557 | - The `--mode` option and `-m` alias to `codeql database create`, 558 | `codeql database cleanup`, and `codeql dataset cleanup` has been 559 | deprecated. Instead, use the new `--cache-cleanup` option, which has 560 | identical behavior. 561 | 562 | ### Improvements 563 | 564 | - Improved the diagnostic message produced when no code is processed 565 | when creating a database. If a build mode was specified using 566 | `--build-mode`, the message is now tailored to your build mode. 567 | 568 | ### Miscellaneous 569 | 570 | - The `scc` tool used by the CodeQL CLI to calculate source code baseline 571 | information has been updated to version [3.2.0](https://github.com/boyter/scc/releases/tag/v3.2.0). 572 | 573 | ## Release 2.17.0 (2024-04-04) 574 | 575 | ### Deprecations 576 | 577 | - The `--[no-]analysis-summary-v2` and `--[no-]new-analysis-summary` options 578 | that were used to enable (or disable) improved summary information printed at 579 | the end of a `codeql database analyze` invocation are no longer supported. 580 | Improved summary information is now enabled for all invocations. 581 | - Support for overwriting default CodeQL SARIF run properties using the 582 | `--sarif-run-property` command line option has been removed. This removes the 583 | ability to overwrite the `semmle.formatSpecifier`, `metricResults`, and 584 | `codeqlConfigSummary` properties in the SARIF run file. 585 | 586 | ### Improvements 587 | 588 | - TRAP import (a part of `codeql database create` and `codeql database finalize`) 589 | now performs better in low-memory situations. (Put another way, it now needs 590 | less RAM to achieve the same performance as before.) 591 | 592 | - The worst-case performance of transitive closure computation (using 593 | the `+` or `*` postfix operators or the `fastTC` higher-order 594 | primitive in QL) has been greatly improved. 595 | 596 | ### Miscellaneous 597 | 598 | - The build of Eclipse Temurin OpenJDK that is used to run the CodeQL 599 | CLI has been updated to version 21.0.2. 600 | 601 | ## Release 2.16.6 (2024-03-26) 602 | 603 | ### Bugs fixed 604 | 605 | - Fixes a bug where extractor logs would be output at a lower than expected 606 | verbosity level when using the `codeql database create` command. 607 | 608 | ## Release 2.16.5 (2024-03-21) 609 | 610 | ### New features 611 | 612 | - Beta support has been added for analyzing Java codebases without needing a working build. To enable 613 | this, pass the `--build-mode none` option to `codeql database create`. 614 | 615 | ## Release 2.16.4 (2024-03-11) 616 | 617 | ### Potentially breaking changes 618 | 619 | - A number of internal command line options (`--builtin_functions_file`, `--clang_builtin_functions`, 620 | `--disable-objc-default-synthesize-properties`, `--list_builtin_functions`, `--memory-limit-bytes`, 621 | `--mimic_config`, and `--objc`) has been removed from the C/C++ extractor. It has never been 622 | possible to pass these options through the CLI itself, but some customers with advanced setups may 623 | have been passing them through internal undocumented interfaces. All of the removed options were 624 | already no-ops, and will now generate errors. 625 | 626 | The `--verbosity` command line option has also been removed. The option was an alias for 627 | `--codeql-verbosity`, which should be used instead. 628 | 629 | ### Improvements 630 | 631 | - The frontend of the C/C++ extractor has been updated, improving the 632 | extractor's reliability and increasing its ability to extract source code. 633 | 634 | ### Bugs fixed 635 | 636 | - When parsing user-authored YAML files such as `codeql-pack.yml`, 637 | `qlpack.yml`, `codeql-workspace.yml`, and any YAML file defining a data 638 | extension, unquoted string values starting with a `*` character are now 639 | correctly interpreted as YAML aliases. Previously, they were interpreted 640 | as strings, but with the first character skipped. 641 | 642 | If you see a parse error similar to `while scanning an alias... unexpected` 643 | `character found *(42)`,it likely means that you need to add quotes around 644 | the indicated string value. The most common cause is unquoted glob patterns 645 | that start with `*`, such as `include: **/*.yml`, which will need to be 646 | quoted as `include: "**/*.yml"`. 647 | 648 | ## Release 2.16.3 (2024-02-22) 649 | 650 | ### Security patches 651 | 652 | - Fixes CVE-2024-25129, a limited data exfiltration vulnerability that 653 | could be triggered by untrusted databases or QL packs. See the 654 | [security advisory](https://github.com/github/codeql-cli-binaries/security/advisories/GHSA-gf8p-v3g3-3wph) 655 | for more information. 656 | 657 | ### New Features 658 | 659 | - A new extractor option has been added to the Python extractor: 660 | `python_executable_name`. You can use this option to override the default 661 | process the extractor uses to find and select a Python executable. Pass one of 662 | `--extractor-option python_executable_name=py` or `--extractor-option 663 | python_executable_name=python` or `--extractor-option 664 | python_executable_name=python3` to commands that run the extractor, for 665 | example: `codeql database create`. 666 | 667 | On Windows machines, the Python extractor will expect to find `py.exe` on the 668 | system `PATH` by default. If the Python executable has a different name, you 669 | can set the new extractor option to override this value and look for 670 | `python.exe` or `python3.exe`. 671 | 672 | For more information about using the extractor option with the CodeQL CLI, see 673 | [Extractor 674 | options](https://docs.github.com/en/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/extractor-options). 675 | 676 | ### Bugs fixed 677 | 678 | - Fixed a bug where CodeQL may produce an invalid database when it exhausts 679 | all available ID numbers. Now it detects the condition and reports an 680 | error instead. 681 | 682 | ## Release 2.16.2 (2024-02-12) 683 | 684 | - There are no user-facing changes in this release. 685 | 686 | ## Release 2.16.1 (2024-01-25) 687 | 688 | ### Improvements 689 | 690 | - When executing the `codeql database init` command, the CodeQL runner 691 | executable path is now stored in the `CODEQL_RUNNER` environment variable. 692 | Users of indirect tracing on MacOS with System Integrity Protection enabled 693 | who previously had trouble with indirect tracing should prefix their build 694 | command with this path. For example, `$CODEQL_RUNNER build.sh`. 695 | 696 | ### QL language improvements 697 | 698 | - Name clashes between weak aliases (i.e. aliases that are not final aliases of 699 | non-final entities) of the same target no longer cause ambiguity errors. 700 | 701 | ## Release 2.16.0 (2024-01-16) 702 | 703 | ### New Features 704 | 705 | - Users specifying extra tracing configurations may now use the `GetRegisteredMatchers(languageId)` Lua function to retrieve the existing table of matchers registered to a given language. 706 | 707 | ### Improvements 708 | 709 | - The `Experimental` flag has been removed from all packaging and related commands. 710 | - The RA pretty-printer omits names of internal RA nodes and pretty-prints 711 | binary unions with nested internal unions as n-ary unions. VS Code extension 712 | v1.11.0 or newer is required to compute join order badness metrics in VS Code 713 | for the new RA format. 714 | 715 | 716 | ### Potentially breaking changes 717 | 718 | - The Python extractor will no longer extract dependencies by default. See https://github.blog/changelog/2023-07-12-code-scanning-with-codeql-no-longer-installs-python-dependencies-automatically-for-new-users/ for more context. In versions until 2.17.0, it will be possible to restore the old behavior by setting `CODEQL_EXTRACTOR_PYTHON_FORCE_ENABLE_LIBRARY_EXTRACTION_UNTIL_2_17_0=1`. 719 | - The `--ram` option to `codeql database run-queries` and other 720 | commands that execute queries is now interpreted more strictly. 721 | Previously it was mostly a rough hint for how much memory to use, 722 | and the actual memory footprint of the CodeQL process could be 723 | hundreds of megabytes higher. From this release, CodeQL tries harder 724 | to keep its _total_ memory consumption during evaluation below the 725 | given limit. 726 | 727 | The new behavior yields more predictable memory use, but since it 728 | works by allocating less RAM, it can lead to more use of _disk_ 729 | storage for intermediate results compared to earlier releases with 730 | the same `--ram` value, and consequently a slight performance 731 | loss. In rare cases, for large databases, analysis may fail with a 732 | Java `OutOfMemoryError`. 733 | 734 | The cure for this is to increase `--ram` to be closer to the amount 735 | of memory actually available for CodeQL. As a rule of thumb, it will 736 | usually be possible to increase the value of `--ram` by 700 MB or 737 | more, without actually using more resources than release 2.15.x 738 | would with the old setting. An exact amount cannot stated, however, 739 | since the actual memory footprint in earlier releases depended on 740 | factors such as the size of the databases that were not fully taken 741 | into account. 742 | 743 | If you use the CodeQL Action, you do not need to do anything unless 744 | you have manually overridden the Action's RAM setting. The Action 745 | will automatically select a `--ram` setting that matches the version 746 | of the CLI it uses. 747 | 748 | ## Release 2.15.5 (2023-12-20) 749 | 750 | ### New features 751 | 752 | - A new extractor option has been added to the JavaScript/TypeScript extractor. 753 | Set the environment variable `CODEQL_EXTRACTOR_JAVASCRIPT_OPTION_SKIP_TYPES` 754 | to `true` to skip the extraction of types in TypeScript files. 755 | Use this to speed up extraction if your codebase has a high volume of 756 | TypeScript type information that causes a noticeable bottleneck for 757 | TypeScript extraction. The majority of analysis results should be preserved 758 | even when no types are extracted. 759 | 760 | ### Bugs fixed 761 | 762 | - Fixed an issue where CodeQL would sometimes incorrectly report that no files 763 | were scanned when running on Windows. 764 | This affected the human-readable summary produced by `codeql database analyze` 765 | and `codeql database interpret-results`, but did not impact the file coverage 766 | information produced in the SARIF output and displayed on the tool status page. 767 | - When analyzing Swift codebases, CodeQL build tracing will now ignore the 768 | `codesign` tool. This prevents errors in build commands or workflows on macOS 769 | that include both CodeQL and code signing. 770 | 771 | ## Release 2.15.4 (2023-12-11) 772 | 773 | ### New features 774 | 775 | - Java 21 is now fully supported, including support for new language features such as pattern switches and record patterns. 776 | 777 | ### Improvements 778 | 779 | - Parallelism in the evaluator has been improved, resulting in faster analysis when 780 | running with many threads, particularly for large databases. 781 | 782 | ## Release 2.15.3 (2023-11-22) 783 | 784 | ### New features 785 | 786 | - `codeql database analyze` now defaults to include markdown query help for all custom 787 | queries with help files available. To change the default behaviour you can pass the 788 | new flag `--sarif-include-query-help`, which provides the options `always` (which 789 | includes query help for all queries), `custom_queries_only` (the default) and `never` 790 | (which does not include query help for any query). The existing flag 791 | `--sarif-add-query-help` has been deprecated and will be removed in a future release. 792 | - The new (advanced) command-line option `--[no-]linkage-aware-import` disables the 793 | linkage-awareness phase of `codeql dataset import`, as a quick fix (at the expense of 794 | database completeness) for C++ projects where this part of database creation consumes 795 | too much memory. This option is available in the commands `database create`, 796 | `database finalize`, `database import`, `dataset import`, `test extract`, and 797 | `test run`. 798 | - The CodeQL language server now provides basic support for Rename, and you can now use 799 | the Rename Symbol functionality in Visual Studio Code for CodeQL. The current Rename 800 | support is less a refactoring tool and more a labor-saving device. You may have to 801 | perform some manual edits after using Rename, but it should still be faster and less 802 | work than renaming a symbol manually. 803 | 804 | ### Improvements 805 | 806 | - The Find References feature in the CodeQL language server now supports all CodeQL 807 | identifiers and offers improved performance compared to CodeQL CLI 2.14 releases. 808 | - The compiler generates shorter human-readable DIL and RA relation names. Due to use 809 | of an extended character set, full VS Code support for short relation names requires 810 | VS Code extension 1.9.4 or newer. 811 | - `codeql database create` and `codeql database finalize` now log more diagnostic 812 | information during database finalization, including the size of each relation, their 813 | total size, and the rate at which they were written to disk. 814 | 815 | ### Bugs fixed 816 | 817 | - Fixed an internal error in the compiler when arguments to the `codePointCount` string 818 | primitive were not bound. 819 | - Fixed a bug where `codeql database finalize` would fail if a database under construction 820 | was moved between machines between `codeql database init` and `codeql database finalize`. 821 | This should now work, as long as both commands are run by the same _release_ of the 822 | CodeQL CLI and the extractors used are the ones bundled with the CLI. 823 | - Fixed a bug where `codeql database run-queries` would fail in some circumstances when 824 | the database path included an `@`. 825 | 826 | ## Release 2.15.2 (2023-11-13) 827 | 828 | ### Breaking changes 829 | 830 | - C++ extraction has been updated to output more accurate C++ value categories. 831 | This may cause unexpected alerts on databases extracted with an up-to-date CodeQL 832 | when the queries are part of a query pack that was compiled with an earlier CodeQL. 833 | To resolve this, please recompile the query pack with the latest CodeQL. 834 | 835 | ### New features 836 | 837 | - `codeql database analyze` and `codeql database interpret-results` can now 838 | output human-readable analysis summaries in a new format. This format provides file coverage 839 | information and improves the way that diagnostic messages are displayed. The new format also includes a link to the tool status page when the `GITHUB_SERVER_URL` and `GITHUB_REPOSITORY` environment variables are set. Note that that page only exists on GitHub.com, or in GitHub Enterprise Server 840 | version 3.9.0 or later. To enable this new format, pass the `--analysis-summary-v2` flag. 841 | - CodeQL now supports 842 | distinguishing file coverage information between related languages C and C++, Java and Kotlin, 843 | and JavaScript and TypeScript. By default, file coverage information for each 844 | of these pairs of languages is grouped together. To enable specific file coverage information for these languages, pass the 845 | `--sublanguage-file-coverage` flag when initializing the database (with `codeql database create` or `codeql database init`) and when analyzing the database (with `codeql database analyze` or `codeql database interpret-results`). If you are uploading results to a GitHub instance, this flag requires GitHub.com or GitHub Enterprise Server version 3.12 or later. 846 | - All CLI commands now support `--common-caches`, which controls the location of the 847 | cached data that is persisted between several runs of the CLI, such as downloaded QL packs 848 | and compiled query plans. 849 | 850 | ### Improvements 851 | 852 | - Model packs that are used in an analysis will now be included in an output SARIF results file. All model packs now include the `isCodeQLModelPack: true` property in their tool component property bag. 853 | - The default formatting of DIL now more closely resembles equivalent QL code. 854 | 855 | ### Bugs fixed 856 | 857 | - Fixed a bug where `codeql github upload-results` would report a 403 error when attempting to upload to a GitHub Enterprise Server instance. 858 | - Fixed a bug in Python extraction where UTF-8 characters would cause 859 | logging to fail on systems with non-UTF-8 default system encoding (for example, Windows systems). 860 | - The `resolve qlpacks --kind extension` command no longer resolves 861 | extensions packs from the search path. This matches the behavior of 862 | `resolve extensions-by-pack` and will ensure that extensions which are 863 | resolved by `resolve qlpacks --kind extension` can also be resolved by 864 | `resolve extensions-by-pack`. 865 | 866 | ## Release 2.15.1 (2023-10-19) 867 | 868 | ### Potentially Breaking Changes 869 | 870 | - The query server's `evaluation/trimCache` command was previously equivalent to the `codeql database cleanup --mode=gentle` CLI command, but is now equivalent to using `--mode=normal`. The new meaning of the command is to clear the entire evaluation cache of a database except for predicates annotated with the `cached` keyword. 871 | 872 | ### Deprecations 873 | 874 | - The accepted values of the `--mode` option for `codeql database cleanup` have been renamed to bring them in line with what they are called in the VSCode extension and the query server: 875 | - `--mode=brutal` is now `--mode=clear`. 876 | - `--mode=normal` is now `--mode=trim`. 877 | - `--mode=light` is now `--mode=fit`. 878 | - The old names are deprecated, but will be accepted for backwards-compatibility reasons until further notice. 879 | 880 | ### Improvements 881 | 882 | - The list of failed tests at the end of a `codeql test run` is now sorted lexicographically. 883 | - The syntax of DIL now more closely resembles the QL source code that it is 884 | compiled from. In particular, conjunctions and disjunctions now use the 885 | familiar `and` and `or` keywords, and clauses are enclosed in curly braces. 886 | 887 | ### Bugs fixed 888 | 889 | - Fixed a bug where the `$CODEQL_JAVA_HOME` environment variable was 890 | erroneously ignored for certain subsidiary Java processes started by 891 | `codeql`. 892 | - Fixed a bug in the CodeQL build tracer on Apple Silicon machines that prevented database creation if System Integrity Protection was disabled. 893 | 894 | ## Release 2.15.0 (2023-10-11) 895 | 896 | ### Deprecations 897 | 898 | - `pragma[assume_small_delta]` is now deprecated. The pragma has no effect and 899 | should be removed. 900 | 901 | - Missing override annotations on class fields now raise errors rather than 902 | warnings. This is to avoid confusion with the shadowing behavior in the 903 | presence of final fields. 904 | 905 | - The CodeQL CLI no longer supports ML-powered alerts. For more information, 906 | including details of our work in the AI-powered security technology space, 907 | see 908 | "[CodeQL code scanning deprecates ML-powered alerts](https://github.blog/changelog/2023-09-29-codeql-code-scanning-deprecates-ml-powered-alerts/)." 909 | 910 | ### New Features 911 | 912 | - The output of `codeql version --format json` now includes a `features` 913 | property. Each key in the map identifies a feature of the CodeQL CLI. The 914 | value for a key is always `true`. Going forward, whenever a significant new 915 | feature is added to the CodeQL CLI, a corresponding entry will be added to the 916 | `features` map. This is intended to make it easier for tools that invoke the 917 | CodeQL CLI to know if the particular version of the CLI they are invoking 918 | supports a given feature, without having to know exactly what CLI version 919 | introduced that feature. 920 | 921 | ### Improvements 922 | 923 | - You can now specify the CodeQL languages C/C++, Java/Kotlin, and 924 | JavaScript/TypeScript using `--language c-cpp`, `--language java-kotlin`, and 925 | `--language javascript-typescript` respectively. These new CodeQL language 926 | names convey more clearly what languages each CodeQL language will analyze. 927 | 928 | You can also reference these CodeQL languages via their secondary language 929 | names (C/C++ via `--language c` or `--language cpp`, Java/Kotlin via 930 | `--language java` or `--language kotlin`, and JavaScript/TypeScript via 931 | `--language javascript` or `--language typescript`), however we recommend you 932 | refer to them via the new primary CodeQL language names for improved clarity. 933 | 934 | - CodeQL now respects custom home directories set by the `$HOME` environment 935 | variable on MacOS and Linux and `%USERPROFILE%` on Windows. When set, CodeQL 936 | will use the variable's value to change the default location of downloaded 937 | packages and the global compilation cache. 938 | 939 | - This release improves the quality of 940 | [file coverage information](https://docs.github.com/en/code-security/code-scanning/managing-your-code-scanning-configuration/about-the-tool-status-page#using-the-tool-status-page) 941 | for repositories that vendor their dependencies. This is currently supported 942 | for Go and JavaScript projects. 943 | 944 | ### Bugs fixed 945 | 946 | - Fixed an issue with analyzing Python projects using Python 3.12. 947 | 948 | ### QL language improvements 949 | 950 | - The QL language now has two new methods `codePointAt` and `codePointCount` on 951 | the `string` type. The methods both return integers and act the same as the 952 | similarly named Java methods on strings. For example, `"abc".codePointAt(2)` 953 | is `99` and `("a" + 128512.toUnicode() + "c").codePointAt(1)` is a `128512`. 954 | 955 | ## Release 2.14.6 (2023-09-26) 956 | 957 | ### Bugs fixed 958 | 959 | - The tracking of RAM usage has been improved. This fixes some cases 960 | where CodeQL uses more RAM than requested. 961 | 962 | ## Release 2.14.5 (2023-09-14) 963 | 964 | ### Bugs fixed 965 | 966 | - Fixed a JavaScript extractor crash that was introduced in 2.14.4. 967 | 968 | ## Release 2.14.4 (2023-09-12) 969 | 970 | ### Potentially breaking changes 971 | 972 | - The CodeQL CLI no longer supports the `SEMMLE_JAVA_ARGS` environment variable. 973 | All previous versions of the CodeQL CLI perform command substitution on the 974 | `SEMMLE_JAVA_ARGS` value (for example, replacing `'$(echo foo)'` with `'foo'`) 975 | when starting a new Java virtual machine, which, depending on the execution 976 | environment, may have security implications. Users are advised to check their 977 | environments for possible `SEMMLE_JAVA_ARGS` misuse. 978 | 979 | ### New Features 980 | 981 | - The Java extractor now supports files that use Lombok. 982 | 983 | ### Bugs fixed 984 | 985 | - `codeql database init` (and `github/codeql-action/init@v2` on GitHub Actions) 986 | should no longer hang or crash for traced languages on 64-bit Windows machines 987 | when certain antivirus software is installed. 988 | - During `codeql pack create` and `codeql pack publish`, a source version of a 989 | pack coming from `--additional-packs` can explicitly be used to override a 990 | requested pack version even if this source version is incompatible with the 991 | requested version in the pack file. Previously, this would fail with a 992 | confusing error message. 993 | - Fixed a bug where `codeql database interpret-results` hangs when a path query 994 | produces a result that has no paths from source to sink. 995 | 996 | ### Miscellaneous 997 | 998 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 999 | CLI has been updated to version 17.0.8. 1000 | 1001 | ## Release 2.14.3 (2023-08-25) 1002 | 1003 | ### Breaking changes 1004 | 1005 | - The `.tool.extensions` property in the SARIF generated by `codeql database analyze` now contains the 1006 | following packs: 1007 | 1008 | - The containing query pack for each query that was evaluated. 1009 | - Each model pack that was specified via the `--model-packs` option, regardless of whether 1010 | that model pack affected any of the evaluated queries. 1011 | 1012 | Library packs are no longer included in the list. 1013 | 1014 | Previously, this property contained every query and library pack that was available on the search 1015 | path, regardless of whether that pack was used during the evaluation. 1016 | 1017 | ### Miscellaneous 1018 | 1019 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1020 | CLI has been updated to version 17.0.8. 1021 | 1022 | - When `codeql test` generates `.actual` files, they will in some cases 1023 | list the query predicates in a different order than past versions. 1024 | There is no need to update `.expected` files, as `codeql test` sorts 1025 | their results accordingly before diffing. 1026 | However, when there are genuine changes in expected results, the 1027 | generated `.actual` file can show additional changes against the 1028 | `.expected` due to the reordering. 1029 | 1030 | ## Release 2.14.2 (2023-08-11) 1031 | 1032 | ### Breaking changes 1033 | 1034 | - The functionality provided by the `codeql execute query-server` subcommand 1035 | has been removed. The subcommand now responds to all JSON RPC requests with 1036 | an error response. Correspondingly, this release is no longer compatible with 1037 | versions of the CodeQL extension for Visual Studio Code prior to 1.7.6. 1038 | 1039 | This change also breaks third-party CodeQL IDE integrations that still rely 1040 | on the `codeql execute query-server` subcommand. Maintainers of such CodeQL 1041 | IDE integrations should migrate to the `codeql execute query-server2` 1042 | subcommand at the earliest opportunity. 1043 | 1044 | ### Improvements 1045 | 1046 | - Switched from prefix filtering of autocomplete suggestions in the language 1047 | server to client-side filtering. This improves autocomplete suggestions in 1048 | contexts with an autocompletion prefix. 1049 | 1050 | - The CodeQL language server now checks query metadata for errors. This allows 1051 | Visual Studio Code users to see errors in their query metadata without needing 1052 | to compile the query. 1053 | 1054 | ### Bugs fixed 1055 | 1056 | - Fixed bug that made the `--warnings=hide` option do nothing in 1057 | `codeql database analyze` and other commands that _evaluate_ queries. 1058 | 1059 | ## Release 2.14.1 (2023-07-27) 1060 | 1061 | - There are no user-facing changes in this release. 1062 | 1063 | ## Release 2.14.0 (2023-07-13) 1064 | 1065 | ### Potentially breaking changes 1066 | 1067 | - The legacy option `--search-path` will now be used, if provided, when 1068 | searching for the dependencies of packages that have no lock file. 1069 | - CodeQL query packs that specify their dependencies using the legacy 1070 | `libraryPathDependencies` property in `qlpack.yml`/`codeql-pack.yml` 1071 | files are no longer permitted to contain a `codeql-pack.lock.yml` lock file. 1072 | This will lead to a compilation error. This change is intended to prevent 1073 | confusing behavior arising from a mix of legacy (unversioned) and modern 1074 | (versioned) package dependencies. To fix this error, either delete the lock 1075 | file, or convert `libraryPathDependencies` to `dependencies`. 1076 | - CodeQL CLI commands that create packages or update package lock files, such 1077 | as `codeql pack publish` and `codeql pack create`, will no longer work on 1078 | query packs that specify their dependencies using the legacy 1079 | `libraryPathDependencies` property. To fix this error, convert 1080 | `libraryPathDependencies` to `dependencies`. 1081 | 1082 | ### Deprecations 1083 | 1084 | - Missing override annotations on class member predicates now raise 1085 | errors rather than warnings. This is to avoid confusion with the 1086 | shadowing behaviour in the presence of final member predicates. 1087 | ```ql 1088 | class Foo extends Base { 1089 | final predicate foo() { ... } 1090 | 1091 | predicate bar() { ... } 1092 | } 1093 | 1094 | class Bar extends Foo { 1095 | // This method shadows Foo::foo. 1096 | predicate foo() { ... } 1097 | 1098 | // This used to override Foo::bar with a warning, now raises error. 1099 | predicate bar() { ... } 1100 | } 1101 | ``` 1102 | 1103 | ### Improvements 1104 | 1105 | - Unqualified imports can now be marked as deprecated to indicate that the 1106 | import may be removed in the future. Usage of names only reachable through 1107 | deprecated imports will generate deprecation warnings. 1108 | - Classes declared inside a parameterized modules can final extend 1109 | parameters of the module as well as types that are declared outside 1110 | the parameterized module. 1111 | - Fields are fully functional when extending types from within a module 1112 | instantiation. 1113 | - Files with a `.yaml` extension will now be included in compiled 1114 | CodeQL packs. Previously, files with this extension were excluded 1115 | even though `.yml` files were included. 1116 | - When interpreting results (e.g., using `bqrs interpret` or 1117 | `database interpret-results`), extra placeholders in alert messages are 1118 | treated as normal text. Previously, results with more placeholders than 1119 | placeholder values were skipped. 1120 | - Windows users of the CodeQL extension for VS Code will see faster start times. 1121 | - In VS Code, errors in the current file are rechecked when dependencies change. 1122 | - In VS Code, autocomplete in large QL files is now faster. 1123 | - Member predicates can shadow final member predicates of the same arity even 1124 | when the signatures are not fully matching. 1125 | 1126 | ### Bugs fixed 1127 | 1128 | - Fixed super calls on final base classes (or final aliases) so that they 1129 | are now dispatched the same way as super calls on instanceof supertypes. 1130 | - Fixed a bug where running `codeql database finalize` with a large number of 1131 | threads would fail due to running out of file descriptors. 1132 | - Fixed a bug where `codeql database create --overwrite` would not work 1133 | with database clusters. 1134 | - Fixed a bug where the CodeQL documentation coverage statistics were 1135 | incorrect. 1136 | - Fixed a bug where the generated CodeQL libarary documentation could 1137 | generate invalid uris on windows. 1138 | 1139 | ## Release 2.13.5 (2023-07-05) 1140 | 1141 | ### New Features 1142 | 1143 | - The Swift extractor now supports Swift 5.8.1. 1144 | 1145 | ## Release 2.13.4 (2023-06-19) 1146 | 1147 | ### New features 1148 | 1149 | - Temporary files and folders created by the CodeQL CLI will now be cleaned up 1150 | when each CLI command (and its internal JVM) shuts down normally. 1151 | 1152 | ### Bugs fixed 1153 | 1154 | - Fixed an issue where indirect build tracing did not work in Azure DevOps 1155 | pipeline jobs in Windows containers. To use indirect build tracing in such 1156 | environments, ensure both the `--begin-tracing` and 1157 | `--trace-process-name=CExecSvc.exe` arguments are passed to 1158 | `codeql database init`. 1159 | - Improved the error message for the `codeql pack create` command when the pack 1160 | being published has a dependency with no scope in its name. 1161 | 1162 | ## Release 2.13.3 (2023-05-31) 1163 | 1164 | ### New features 1165 | 1166 | - This release enhances our preliminary Swift support, setting the stage for the upcoming public beta. 1167 | 1168 | - The `codeql database bundle` command now supports the `--[no]-include-temp` 1169 | option. When enabled, this option will include the `temp` folder of the 1170 | database directory in the zip file of the bundled database. This folder 1171 | includes generated packages and queries, and query suites. 1172 | 1173 | - The structured log produced by `codeql generate log-summary` now includes a Boolean `isCached` 1174 | field for predicate events, where a `true` value indicates the predicate is a wrapper implementing 1175 | the `cached` annotation on another predicate. The wrapper depends on the underlying predicate that 1176 | the annotation was found on, and will usually have the same name, but it has a separate `raHash`. 1177 | 1178 | ### Bugs fixed 1179 | 1180 | - Fixed a bug that could cause the compiler to infer incorrect binding sets for 1181 | non-direct calls to overriding member predicates that have stronger binding sets 1182 | than their root definitions. 1183 | 1184 | - Fixed a bug that could have caused the compiler to incorrectly infer that a 1185 | class matched a type signature. The bug only affected classes with overriding 1186 | member predicates that had stronger binding sets than their root definitions. 1187 | 1188 | - Fixed a bug where a query could not be run from VS Code 1189 | when there were packs nested within sibling directories 1190 | of the query. 1191 | 1192 | ## Release 2.13.2 1193 | 1194 | This release was skipped. 1195 | 1196 | ## Release 2.13.1 (2023-05-03) 1197 | 1198 | ### Bugs fixed 1199 | 1200 | - Fixed a bug in `codeql database upload-results` where the subcommand 1201 | would fail with "A fatal error occurred: Invalid SARIF.", reporting 1202 | an `InvalidDefinitionException`. This issue occurred when the SARIF 1203 | file contained certain kinds of diagnostic information. 1204 | 1205 | ### Miscellaneous 1206 | 1207 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1208 | CLI has been updated to version 17.0.7. 1209 | 1210 | ## Release 2.13.0 (2023-04-20) 1211 | 1212 | ### Known issues 1213 | 1214 | - We recommend that customers using the CodeQL CLI in a third party CI 1215 | system do not upgrade to this release, due to an issue with `codeql 1216 | github upload-results`. Instead, please use CodeQL 2.12.5, or, when 1217 | available, CodeQL 2.12.7 or 2.13.1. For more information, see the 1218 | "Known issues" section for CodeQL 2.12.6. 1219 | 1220 | ### Potentially breaking changes 1221 | 1222 | - In `codeql pack add`, the dependency that is added to the `qlpack.yml` file will now allow any 1223 | version of the pack that is compatible with the specified version (`^version`) in the following 1224 | cases: 1225 | - When no version is specified (`codeql pack add codeql/cpp-all`). 1226 | - When the version is specified as `latest` (`codeql pack add codeql/cpp-all@latest`). 1227 | - When a single version is specified (`codeql pack add codeql/cpp-all@1.0.0`). 1228 | 1229 | The `^version` dependency allows any version of that pack with no breaking changes since `version`. 1230 | For example, `^1.2.3` would allow versions `1.2.3`, `1.2.5`, and `1.4.0`, but not `2.0.0`, because 1231 | changing the major version number to `2` indicates a breaking change. 1232 | 1233 | Using `^version` ensures that the added pack is not needlessly constrained to an exact version by default. 1234 | 1235 | - Upper-case variable names are no longer accepted by the QL compiler. 1236 | 1237 | Such variable names have produced a deprecation warning since 1238 | release 2.9.2 (released 2022-05-16), so QL code that compiles 1239 | without warnings with a recent release of the CLI should still work. 1240 | 1241 | ### New features 1242 | 1243 | - `codeql database analyze` and related commands now export file 1244 | coverage information by default. GHAS customers using CodeQL in 1245 | third-party CI systems will now see file coverage information on the 1246 | [tool status page](https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-the-tool-status-page) 1247 | without needing to modify their CI workflows. 1248 | 1249 | ### Deprecations 1250 | 1251 | - The possibility to omit `override` annotations on class member 1252 | predicates that override a base class predicate has been deprecated. 1253 | This is to avoid confusion with shadowing behaviour in the 1254 | presence of final member predicates. 1255 | 1256 | ```ql 1257 | class Foo extends Base { 1258 | final predicate foo() { ... } 1259 | 1260 | predicate bar() { ... } 1261 | 1262 | predicate baz() { ... } 1263 | } 1264 | 1265 | class Bar extends Foo { 1266 | // This method shadows Foo::foo. 1267 | predicate foo() { ... } 1268 | 1269 | // This used to override Foo::bar with a warning, is now deprecated. 1270 | predicate bar() { ... } 1271 | 1272 | // This correctly overrides Foo::baz 1273 | override predicate baz() { ... } 1274 | } 1275 | ``` 1276 | 1277 | ## Release 2.12.7 (2023-04-18) 1278 | 1279 | ### Bugs fixed 1280 | 1281 | - Fixed a bug in `codeql database upload-results` where the subcommand 1282 | would fail with "A fatal error occurred: Invalid SARIF.", reporting 1283 | an `InvalidDefinitionException`. This issue occurred when the SARIF 1284 | file contained certain kinds of diagnostic information. 1285 | 1286 | ## Release 2.12.6 (2023-04-04) 1287 | 1288 | ### Known issues 1289 | 1290 | - We recommend that customers using the CodeQL CLI in a third party CI 1291 | system do not upgrade to this release, due to an issue with `codeql 1292 | github upload-results`. Instead, please use CodeQL 2.12.5, or, when 1293 | available, CodeQL 2.12.7 or 2.13.1. 1294 | 1295 | This issue occurs when uploading certain kinds of diagnostic information and 1296 | causes the subcommand to fail with "A fatal error occurred: Invalid 1297 | SARIF.", reporting an `InvalidDefinitionException`. 1298 | 1299 | Customers who wish to use CodeQL 2.12.6 or 2.13.0 can 1300 | work around the problem by passing `--no-sarif-include-diagnostics` 1301 | to any invocations of `codeql database analyze` or `codeql database 1302 | interpret-results`. 1303 | 1304 | ### New features 1305 | 1306 | - Several experimental subcommands have been added in support of the 1307 | new [code scanning tool status page](https://github.blog/changelog/2023-03-28-code-scanning-shows-the-health-of-tools-enabled-on-a-repository/). 1308 | These include `codeql database add-diagnostic`, 1309 | `codeql database export-diagnostics`, and the 1310 | `codeql diagnostic add` and `codeql diagnostic export` plumbing 1311 | subcommands. 1312 | 1313 | ### Bugs fixed 1314 | 1315 | - Fixed a bug in `codeql database analyze` and related commands 1316 | where the `--max-paths` option was not respected correctly when 1317 | multiple alerts with the same primary code location were grouped together. 1318 | (This grouping is the default behavior unless the `--no-group-alerts` 1319 | option is passed.) 1320 | This bug caused some SARIF files produced by CodeQL to exceed the limits 1321 | on the number of paths (`threadFlows`) accepted by code scanning, 1322 | leading to errors when uploading results. 1323 | 1324 | ## Release 2.12.5 (2023-03-21) 1325 | 1326 | ### New features 1327 | 1328 | - The `codeql pack install` command now accepts a `--additional-packs` 1329 | option. This option takes a list of directories to search for 1330 | locally available packs when resolving which packs to install. Any 1331 | pack that is found locally through `--additional-packs` will 1332 | override any other version of a pack found in the package registry. 1333 | Locally resolved packs are not added to the lock file. 1334 | 1335 | Because the use of `--additional-packs` when running 1336 | `codeql pack install` makes running queries dependent on the local 1337 | state of the machine initially invoking `codeql pack install`, a 1338 | warning is emitted if any pack is found outside of the package 1339 | registry. This warning can be suppressed by using the 1340 | `--no-strict-mode` option. 1341 | 1342 | ### Bugs fixed 1343 | 1344 | - Fix a bug in `codeql query run` where queries whose path contain 1345 | colons cannot be run. 1346 | 1347 | ## Release 2.12.4 (2023-03-09) 1348 | 1349 | ### Breaking changes 1350 | 1351 | - The default value of the `--mode` switch to `codeql pack install` has changed. The default is now `--mode minimal-update`. 1352 | Previously, it was `use-lock`. 1353 | 1354 | ### New features 1355 | 1356 | - The per-pack compilation cache has been replaced with a global compilation cache 1357 | found within `~/.codeql`. 1358 | - `codeql pack install` now uses a new algorithm to determine which versions of 1359 | the pack's dependencies to use, based on the [PubGrub](https://nex3.medium.com/pubgrub-2fb6470504f) 1360 | algorithm. The new algorithm is able to find a solution for many cases that 1361 | the previous algorithm would fail to solve. When the new algorithm is unable 1362 | to find a valid solution, it generates a detailed error message explaining 1363 | why there is no valid solution. 1364 | - Added a new command, `codeql pack upgrade`. This command is similar to `codeql pack install`, 1365 | except that it ignores any existing lock file, installs the latest compatible version of each 1366 | dependency, and writes a new lock file. This is equivalent to `codeql pack install --mode update`. 1367 | Note that the `--mode` switch to `codeql pack install` is now deprecated. 1368 | - Added a new command, `codeql pack ci`. This command is similar to `codeql pack install`, 1369 | except if the existing lock file is missing, or if it conflicts with the version constraints in 1370 | the `qlpack.yml` file, the command generates an error. This is equivalent to 1371 | `codeql pack install --mode verify`. Note that the `--mode` switch to `codeql pack install` is now 1372 | deprecated. 1373 | 1374 | ### Deprecations 1375 | 1376 | - The `--freeze` switch for `codeql pack create`, `codeql pack bundle`, and `codeql pack publish` 1377 | is now deprecated and ignored, as there is no longer a cache within a pack. 1378 | - The `--mode update` switch to `codeql pack resolve-dependencies` is now deprecated. Instead, use 1379 | the new `--mode upgrade` switch, which has identical behavior. 1380 | - The `--mode` switch to `codeql pack install` is now deprecated. 1381 | - Instead of `--mode update`, use `codeql pack upgrade`. 1382 | - Instead of `--mode verify`, use `codeql pack ci`. 1383 | 1384 | ## Release 2.12.3 (2023-02-23) 1385 | 1386 | ### New features 1387 | 1388 | - The CodeQL compiler now produces better error messages when it is unable 1389 | to find a QL library that the query being evaluated depends on. 1390 | 1391 | ### Bugs fixed 1392 | 1393 | - Fixed a bug where the CLI would refuse to complete database creation 1394 | if the OS reports less than about 1.5 GB of physical memory. Now an 1395 | attempt will be made even on low-memory systems (but it might still 1396 | run out of memory unless there's swap space available). 1397 | 1398 | ## Release 2.12.2 (2023-02-07) 1399 | 1400 | ### Bugs fixed 1401 | 1402 | - Fixed a QL evaluator bug introduced in release 2.12.1 which could in 1403 | certain rare cases lead to wrong analysis results. 1404 | 1405 | - Fixed handling of `-Xclang ` arguments passed to the `clang` compiler which 1406 | could cause missing extractions for C++ code bases. 1407 | 1408 | - Fixed a bug where the `--overwrite` option was failing for database clusters. 1409 | 1410 | ### Miscellaneous 1411 | 1412 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1413 | CLI has been updated to version 17.0.6. 1414 | 1415 | ## Release 2.12.1 (2023-01-23) 1416 | 1417 | ### New features 1418 | 1419 | - Added a new command-line flag `--expect-discarded-cache`, which gives a hint 1420 | to the evaluator that the evaluation cache will be discarded after analysis 1421 | completes. This allows it to avoid some unnecessary writes to the cache, for 1422 | predicates that aren't needed by the query/suite being evaluated. 1423 | 1424 | ## Release 2.12.0 (2023-01-10) 1425 | 1426 | ### Breaking changes 1427 | 1428 | - The `--[no-]count-lines` option to `codeql database create` and related commands that was 1429 | deprecated in 2.11.1 has been removed. Users of this option should instead pass 1430 | `--[no-]calculate-baseline`. 1431 | 1432 | ### New features 1433 | 1434 | - Query packs created by `codeql pack create`, `codeql pack bundle`, and `codeql pack release` now 1435 | contain precompiled queries in a new format that aims to be compatible with future (and, to a 1436 | certain extent, past) releases of the CodeQL CLI. Previously the precompiled queries were in a 1437 | format specific to each CLI release, and all other releases would need to re-compile queries. 1438 | 1439 | Published packs contain precompiled queries in files with a `.qlx` extension located next to each 1440 | query's `.ql` source file. In case of differences between the `.ql` and `.qlx` files, the `.qlx` 1441 | file takes priority when evaluating queries from the command line, so if you need to modify a 1442 | published pack, be sure to delete the `.qlx` files first. 1443 | 1444 | A new `--precompile` flag to `codeql query compile` can be used to construct `*.qlx` file 1445 | explicitly, but in all usual cases it should be enough to rely on `codeql pack create` doing the 1446 | right thing. 1447 | - The `codeql database init` command now accepts a PAT that allows you to download queries from 1448 | external, private repositories when using the `--codescanning-config ` option. For 1449 | example, you can specify the following queries block in the config file, which will checkout the main 1450 | branch of the `codeql-test/my-private-repository` repository and evaluate any queries found in that 1451 | repository: 1452 | 1453 | ```yaml 1454 | queries: 1455 | - codeql-test/my-private-repository@main 1456 | ``` 1457 | 1458 | If the repository is private, you can add a `--external-repository-token-stdin` option and supply a 1459 | PAT with appropriate permissions via standard input. For more information on queries and external 1460 | repositories in Code Scanning, see [Using queries in QL packs](https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs). 1461 | - The baseline information produced by `codeql database init` and 1462 | `codeql database create` now accounts for 1463 | [`paths` and `paths-ignore` configuration](https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#specifying-directories-to-scan). 1464 | - In the VS Code extension, recursive calls will be marked with inlay 1465 | hints. These can be disabled with the global inlay hints setting 1466 | (`editor.inlayHints.enabled`). If you just want to disable them for 1467 | codeql the settings can be scoped to just codeql files (language id is `ql`). 1468 | See [Language Specific Editor Settings](https://code.visualstudio.com/docs/getstarted/settings#_language-specific-editor-settings) 1469 | in the VS Code documentation for more information. 1470 | - The CLI now gives a more helpful error message when asked to run queries on a 1471 | database that has not been finalized. 1472 | 1473 | ### Bugs fixed 1474 | 1475 | - Fixed a bug where the `codeql pack install` command would fail if 1476 | a [CodeQL configuration file](https://codeql.github.com/docs/codeql-cli/specifying-command-options-in-a-codeql-configuration-file/#using-a-codeql-configuration-file) 1477 | is used and the `--additional-packs` option is specified. 1478 | 1479 | ## Release 2.11.6 (2022-12-13) 1480 | 1481 | ### Breaking changes 1482 | 1483 | - Java and Kotlin analyses in this release of the CLI and all earlier 1484 | releases are incompatible with Kotlin 1.7.30 and later. To prevent 1485 | code scanning alerts being spuriously dismissed, Java and Kotlin 1486 | analyses will now fail when using Kotlin 1.7.30 or later. 1487 | 1488 | If you are unable to use Kotlin 1.7.29 or earlier, you can disable 1489 | Kotlin support by setting 1490 | `CODEQL_EXTRACTOR_JAVA_AGENT_DISABLE_KOTLIN` to `true` in the 1491 | environment. 1492 | 1493 | ### Bugs fixed 1494 | 1495 | - Fixed a bug where it was not possible to run queries in CodeQL query 1496 | packs for C# that use the legacy `libraryPathDependencies` property 1497 | in their `qlpack.yml` file. The associated error message complained 1498 | about undefined extensional predicates. 1499 | 1500 | ## Release 2.11.5 (2022-12-07) 1501 | 1502 | ### Bugs Fixed 1503 | 1504 | - Fixed a bug that could cause log summary generation to fail in vscode. 1505 | 1506 | ## Release 2.11.4 (2022-11-24) 1507 | 1508 | ### New features 1509 | 1510 | - Kotlin support is now in beta. This means that Java analyses will also 1511 | include Kotlin code by default. Kotlin support can be disabled by 1512 | setting `CODEQL_EXTRACTOR_JAVA_AGENT_DISABLE_KOTLIN` to `true` in 1513 | the environment. 1514 | 1515 | ### Potentially breaking changes 1516 | 1517 | - CodeQL 2.11.1 to 2.11.3 contained a bug in [indirect build tracing](https://codeql.github.com/docs/codeql-cli/creating-codeql-databases/#using-indirect-build-tracing) on Windows 1518 | when using `codeql database init` with the [`--trace-process-level`](https://codeql.github.com/docs/codeql-cli/manual/database-init/#cmdoption-codeql-database-init-trace-process-level) flag. 1519 | In these versions, when `--trace-process-level` was set to a value greater than zero, 1520 | (or left at the default value of 1), CodeQL attempted to inject its build tracer 1521 | at a higher level in the process tree than the requested process level. 1522 | This could lead to errors of the form "No source code found" or 1523 | "Process tree ended before reaching required level". 1524 | From 2.11.4 onwards, the CodeQL build tracer is injected at the requested process level. 1525 | 1526 | ### Deprecations 1527 | 1528 | - The `--[no-]fast-compilation` option to `codeql test run` is now 1529 | deprecated. 1530 | 1531 | ## Release 2.11.3 (2022-11-11) 1532 | 1533 | ### Breaking changes 1534 | 1535 | - The `codeql pack ls --format json` deep plumbing command now returns 1536 | only the `name` and `version` properties for each found pack. 1537 | 1538 | ### Potentially breaking changes 1539 | 1540 | - `codeql pack download`, `codeql pack install`, and `codeql pack add` 1541 | will ignore CodeQL packs with pre-release versions, unless the 1542 | `--allow-prerelease` option is passed to the command. This brings 1543 | these commands into alignment with `codeql pack publish` that will 1544 | avoid publishing CodeQL packs with pre-release versions unless the 1545 | `--allow-prerelease` option is specified. Pre-release versions have 1546 | the following format: `X.Y.Z-qualifier` where `X`, `Y`, and `Z` are 1547 | respectively the major, minor, and patch number. `qualifier` is the 1548 | pre-release version. For more information about pre-releases, see 1549 | the 1550 | [Semantic Versioning specification](https://semver.org/#spec-item-9). 1551 | 1552 | ### Deprecations 1553 | 1554 | - The `--[no-]fast-compilation` option to `codeql query compile` is 1555 | now deprecated. 1556 | 1557 | ### New features 1558 | 1559 | - `codeql resolve files` and `codeql database index-files` have a new 1560 | `--find-any` option, which finds at most one match. 1561 | 1562 | ### Miscellaneous 1563 | 1564 | - The build of Apache Commons Text that is bundled with the CodeQL CLI 1565 | has been updated to version 1.10.0. While previous releases shipped 1566 | with version 1.6 of the library, no part of the CodeQL CLI 1567 | references the `StringSubstitutor` class that the recently disclosed 1568 | [CVE-2022-42889](https://github.com/advisories/GHSA-599f-7c49-w659) 1569 | vulnerability applies to. We therefore do not believe that running 1570 | previous releases of CodeQL exposes users to this vulnerability. 1571 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1572 | CLI has been updated to version 17.0.5. 1573 | 1574 | ## Release 2.11.2 (2022-10-25) 1575 | 1576 | ### Breaking changes 1577 | 1578 | - Bundling and publishing a CodeQL pack will no longer include nested 1579 | CodeQL packs. If you want to include a nested pack in your published pack, 1580 | then you must explicitly include it using the `include` property in the 1581 | top-level `qlpack.yml` file. 1582 | 1583 | For example, if your package structure looks like this: 1584 | 1585 | ```text 1586 | qlpack.yml 1587 | nested-pack 1588 | ∟ qlpack.yml 1589 | query.ql 1590 | ``` 1591 | 1592 | then the contents of `nested-pack` will not be included by default within 1593 | the published package. To include `nested-pack`, add an entry like this 1594 | to the top level `qlpack.yml` file: 1595 | 1596 | ```yml 1597 | include: 1598 | - nested-pack/** 1599 | ``` 1600 | 1601 | ### Bugs fixed 1602 | 1603 | - Using the `--codescanning-config=` option in 1604 | `codeql database init` will now correctly process the `paths` and 1605 | `pathsIgnore` properties of the configuration file in a way that is 1606 | identical to the behavior of the `codeql-action`. Previously, `paths` 1607 | or `pathsIgnore` entries that end in `/**` or start with `/` were 1608 | incorrectly rejected by the CLI. 1609 | 1610 | - Fixed a bug where the `--compilation-cache` option to 1611 | `codeql pack publish` and `codeql pack create` was being ignored 1612 | when creating a query pack. Now, the indicated cache is used 1613 | when pre-compiling the queries in it. 1614 | 1615 | - Fixed a bug that would make the "Show DIL" command in the VSCode 1616 | extension display nothing. 1617 | 1618 | ### Other changes 1619 | 1620 | - Emit a detailed warning if package resolution fails, the legacy 1621 | `--search-path` option is provided, _and_ there is at least one 1622 | referenced pack that does not use legacy package resolution. 1623 | In this case, `--additional-packs` should be used to extend the 1624 | search to additional directories, instead of `--search-path`. 1625 | 1626 | ## Release 2.11.1 (2022-10-11) 1627 | 1628 | ### Breaking changes 1629 | 1630 | - Pack installation using the CodeQL Packaging beta will now fail if a 1631 | compatible version cannot be found. This replaces the previous 1632 | behavior where `codeql pack download` and related commands would 1633 | instead install the latest version of the pack in this situation. 1634 | 1635 | ### Deprecations 1636 | 1637 | - The `--[no-]count-lines` option to `codeql database create` and 1638 | related commands is now deprecated and will be removed in a future 1639 | release of the CodeQL CLI (earliest 2.12.0). It is replaced by 1640 | `--[no-]calculate-baseline` to reflect the additional baseline 1641 | information that is now captured as of this release. 1642 | 1643 | ### New features 1644 | 1645 | - Subcommands that compile QL accept a new `--no-release-compatibility` 1646 | option. It does nothing for now, but in the future it will be used 1647 | to control a trade-off between query performance and compatibility 1648 | with older/newer releases of the QL evaluator. 1649 | - `codeql database analyze` and related commands now support absolute 1650 | paths containing the `@` or `:` characters when specifying which queries 1651 | to run. To reference a query file, directory, or suite whose path contains 1652 | a literal `@` or `:`, prefix the query specifier with `path:`, for example: 1653 | ```shell 1654 | codeql database analyze --format=sarif-latest --output=results path:C:/Users/ci/workspace@2/security/query.ql 1655 | ``` 1656 | ### Bugs fixed 1657 | 1658 | - It is no longer an error to call `codeql pack create ` with a `` 1659 | option pointing to a file name. The CLI will walk up the directory tree and 1660 | run the command in the first directory containing the `qlpack.yml` or `codeql-pack.yml` file. 1661 | - Fixed a concurrency error observed when using `codeql database import` or 1662 | `codeql database finalize` with multiple threads and multiple additional 1663 | databases on a C++ codebase. 1664 | 1665 | ## Release 2.11.0 (2022-09-28) 1666 | 1667 | ### Deprecation 1668 | 1669 | - The CodeQL CLI now uses Python 3 to extract both Python 2 and Python 3 1670 | databases. Correspondingly, support for using Python 2 to extract 1671 | Python databases is now deprecated. Starting with version 2.11.3, you 1672 | will need to install Python 3 to extract Python databases. 1673 | 1674 | ### Miscellaneous 1675 | 1676 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1677 | CLI has been updated to version 17.0.4. 1678 | 1679 | ## Release 2.10.5 (2022-09-13) 1680 | 1681 | ### New features 1682 | 1683 | - You can now define which registries should be used for downloading and publishing CodeQL packs on a 1684 | per-workspace basis by creating a `codeql-workspace.yml` file and adding a `registries` block. For 1685 | more infomation, see [About CodeQL Workspaces](https://codeql.github.com/docs/codeql-cli/about-codeql-workspaces/). 1686 | 1687 | ## Release 2.10.4 (2022-08-31) 1688 | 1689 | - The bundled extractors are updated to match the versions currently 1690 | used on LGTM.com. These are newer than the last release (1.30) of 1691 | LGTM Enterprise. If you plan to upload databases to an LGTM 1692 | Enterprise 1.30 instance, you need to create them with release 1693 | 2.7.6. 1694 | 1695 | - This release does not include any user-facing changes. 1696 | 1697 | ## Release 2.10.3 (2022-08-15) 1698 | 1699 | - The bundled extractors are updated to match the versions currently 1700 | used on LGTM.com. These are newer than the last release (1.30) of 1701 | LGTM Enterprise. If you plan to upload databases to an LGTM 1702 | Enterprise 1.30 instance, you need to create them with release 1703 | 2.7.6. 1704 | 1705 | ### New features 1706 | 1707 | - When called with `--start-tracing`, the `codeql database init` command 1708 | now accepts extractor options for the indirect tracing environment via 1709 | `--extractor-option`. Users should continue to specify extractor options 1710 | for direct tracing environments by passing them to 1711 | `codeql database trace-command` invocations. 1712 | 1713 | ### Other changes 1714 | 1715 | - The build of Eclipse Temurin OpenJDK that is bundled with the CodeQL 1716 | CLI has been updated to version 17.0.4. 1717 | 1718 | ## Release 2.10.2 (2022-08-02) 1719 | 1720 | - The bundled extractors are updated to match the versions currently 1721 | used on LGTM.com. These are newer than the last release (1.30) of 1722 | LGTM Enterprise. If you plan to upload databases to an LGTM 1723 | Enterprise 1.30 instance, you need to create them with release 1724 | 2.7.6. 1725 | 1726 | ### Breaking change 1727 | 1728 | - The option `--compiler-spec` to `codeql database create` (and 1729 | `codeql database trace-command`) no longer works. It is replaced by 1730 | `--extra-tracing-config`, which accepts a tracer configuration file 1731 | in the new, Lua-based tracer configuration format instead. See 1732 | `tools/tracer/base.lua` for the precise API available. If you need 1733 | help help porting your existing compiler specification files, please 1734 | file a public issue in https://github.com/github/codeql-cli-binaries, 1735 | or open a private ticket with GitHub support and request an 1736 | escalation to engineering. 1737 | 1738 | ### Potentially breaking changes 1739 | 1740 | - Versions of the CodeQL extension for Visual Studio Code released 1741 | before February 2021 may not work correctly with this CLI, in 1742 | particular if database upgrades are necessary. We recommend keeping 1743 | your VS Code extension up-to-date. 1744 | 1745 | ### Deprecation 1746 | 1747 | - The experimental `codeql resolve ml-models` command has been 1748 | deprecated. Advanced users calling this command should use the new 1749 | `codeql resolve extensions` command instead. 1750 | 1751 | ### New features 1752 | 1753 | - The `codeql github upload-results` command now supports a `--merge` 1754 | option. If this option is provided, the command will accept the paths 1755 | to multiple SARIF files, and will merge those files before uploading 1756 | them as a single analysis. This option is recommended _only_ for 1757 | backwards compatibility with old analyses produced by the CodeQL 1758 | Runner, which combined the results for multiple languages into a 1759 | single analysis. 1760 | 1761 | ## Release 2.10.1 (2022-07-19) 1762 | 1763 | - The bundled extractors are updated to match the versions currently 1764 | used on LGTM.com. These are newer than the last release (1.30) of 1765 | LGTM Enterprise. If you plan to upload databases to an LGTM 1766 | Enterprise 1.30 instance, you need to create them with release 1767 | 2.7.6. 1768 | 1769 | ### New features 1770 | 1771 | - Improved error message from `codeql database analyze` when a query is 1772 | missing `@id` or `@kind` query metadata. 1773 | 1774 | ## Release 2.10.0 (2022-06-27) 1775 | 1776 | - The bundled extractors are updated to match the versions currently 1777 | used on LGTM.com. These are newer than the last release (1.30) of 1778 | LGTM Enterprise. If you plan to upload databases to an LGTM 1779 | Enterprise 1.30 instance, you need to create them with release 1780 | 2.7.6. 1781 | 1782 | ### Breaking changes 1783 | 1784 | - The `--format=stats` option of `codeql generate log-summary` has been 1785 | renamed to `--format=overall`. It now produces a richer JSON object 1786 | that, in addition to the previous statistics about the run (which can 1787 | be found in the `stats` property) also records the most expensive 1788 | predicates in the evaluation run. 1789 | 1790 | ### Potentially breaking changes 1791 | 1792 | - The `codeql resolve ml-model` command now requires one or more query 1793 | specifications as command line arguments in order to determine the set 1794 | of starting packs from which to initiate the resolution process. The 1795 | command will locate all ML models in any qlpack that is a transitive 1796 | dependency of any of the starting packs. Also, the output of the 1797 | command has been expanded to include for each model the containing 1798 | package's name, version, and path. 1799 | 1800 | - The `buildMetadata` inside of compiled CodeQL packs no longer contains 1801 | a `creationTime` property. This was removed in order to ensure that 1802 | the content of a CodeQL pack is identical when it is re-compiled. 1803 | 1804 | - The `codeql pack download` command, when used with the `--dir` option, 1805 | now downloads requested packs in directories corresponding to their 1806 | version numbers. Previously, 1807 | `codeql pack download --dir ./somewhere codeql/java-queries@0.1.2` 1808 | would download the pack into the `./somewhere/codeql/java-queries` 1809 | directory. Now, it will download the pack into the 1810 | `./somewhere/codeql/java-queries/0.1.2` directory. This allows you to 1811 | download multiple versions of the same pack using a single command. 1812 | 1813 | ### New features 1814 | 1815 | - You can now include diagnostic messages in the summary produced by 1816 | the `--print-diagnostics-summary` option of the 1817 | `codeql database interpret-results` and `codeql database analyze` 1818 | commands by running these commands at high verbosity levels. 1819 | 1820 | ### Bugs fixed 1821 | 1822 | - Fixed a bug where `codeql pack download`, when used with the `--dir` 1823 | option, would not download a pack that is in the global package cache. 1824 | 1825 | - Fixed a bug where some versions of a CodeQL package could not be 1826 | downloaded if there are more than 100 versions of this package in the 1827 | package registry. 1828 | 1829 | - Fixed a bug where the `--also-match` option for `codeql resolve files` 1830 | and `codeql database index-files` does not work with relative paths. 1831 | 1832 | - Fixed a bug that caused `codeql query decompile` to ignore the 1833 | `--output` option when producing bytecode output (`--kind=bytecode`), 1834 | writing only to `stdout`. 1835 | 1836 | ## Release 2.9.4 (2022-06-20) 1837 | 1838 | - The bundled extractors are updated to match the versions currently 1839 | used on LGTM.com. These are newer than the last release (1.30) of 1840 | LGTM Enterprise. If you plan to upload databases to an LGTM 1841 | Enterprise 1.30 instance, you need to create them with release 1842 | 2.7.6. 1843 | 1844 | ### New features 1845 | 1846 | - Users of CodeQL Packaging Beta can now optionally authenticate to 1847 | Container registries on GitHub Enterprise Server (GHES) versions 3.6 1848 | and later using standard input instead of the `CODEQL_REGISTRIES_AUTH` 1849 | environment variable. To authenticate via standard input, pass 1850 | `--registries-auth-stdin`. The value you provide will override the 1851 | value of the `CODEQL_REGISTRIES_AUTH` environment variable. 1852 | 1853 | ## Release 2.9.3 (2022-05-31) 1854 | 1855 | - The bundled extractors are updated to match the versions currently 1856 | used on LGTM.com. These are newer than the last release (1.30) of 1857 | LGTM Enterprise. If you plan to upload databases to an LGTM 1858 | Enterprise 1.30 instance, you need to create them with release 1859 | 2.7.6. 1860 | 1861 | ### New features 1862 | 1863 | - Users can now use CodeQL Packaging Beta to publish and download 1864 | CodeQL packs on GitHub Enterprise Server (GHES) versions 3.6 and 1865 | later. 1866 | 1867 | To authenticate to a package registry on GHES 3.6+, first create a 1868 | `~/.codeql/qlconfig.yml` file. For example, the following file 1869 | specifies that all CodeQL packages should be uploaded to the GHES 1870 | instance with the hostname `GHE_HOSTNAME`: 1871 | 1872 | ```yml 1873 | registries: 1874 | - packages: '*' 1875 | url: https://containers.GHE_HOSTNAME/v2/ 1876 | ``` 1877 | 1878 | You can now download public packages from GHES using 1879 | `codeql pack download`. 1880 | 1881 | To publish any package or download private packages, authenticate to 1882 | GHES by specifying registry/token pairs in the 1883 | `CODEQL_REGISTRIES_AUTH` environment variable. You can authenticate 1884 | using either a GitHub Apps token or a personal access token. For 1885 | example, 1886 | `https://containers.GHEHOSTNAME1/v2/=TOKEN1,https://containers.GHEHOSTNAME2/v2/=TOKEN2` 1887 | will authenticate the CLI to the `GHEHOSTNAME1` and `GHEHOSTNAME2` 1888 | GHES instances. 1889 | 1890 | ### Bugs Fixed 1891 | 1892 | - Fixed a bug where precompiled CodeQL packages in the CodeQL bundle were 1893 | being recompiled if they were in a read-only directory. 1894 | 1895 | - Fixed a bug where new versions of the VS Code extension wouldn't run two 1896 | queries in parallel against one database. 1897 | 1898 | ## Release 2.9.2 (2022-05-16) 1899 | 1900 | - The bundled extractors are updated to match the versions currently 1901 | used on LGTM.com. These are newer than the last release (1.30) of 1902 | LGTM Enterprise. If you plan to upload databases to an LGTM 1903 | Enterprise 1.30 instance, you need to create them with release 1904 | 2.7.6. 1905 | 1906 | ### Features removed 1907 | 1908 | - The table printed by `codeql database analyze` to summarize the 1909 | results of metric queries that were part of the analysis now reports 1910 | a single row per metric name independently of the verbosity level of 1911 | the command. Previously, at higher verbosity levels, this table 1912 | would contain multiple rows for metric names with multiple values. 1913 | 1914 | ### New features 1915 | 1916 | - The tables produced by `codeql database analyze` summarizing the 1917 | results of any diagnostic and metric queries that were run now 1918 | exclude the results of queries tagged `telemetry`. 1919 | 1920 | - Uploading SARIF results using the `codeql github upload-results` 1921 | command now has a timeout of 5 minutes. 1922 | 1923 | - Downloading CodeQL packs using the `codeql pack download`, 1924 | `codeql pack install` and related commands now have a timeout of 1925 | 5 minutes and will retry 3 times before failing. Similar behavior 1926 | has been added to the `codeql pack publish` command. 1927 | 1928 | - The `codeql generate log-summary` command will now print progress 1929 | updates to `stderr`. 1930 | 1931 | ### Bugs fixed 1932 | 1933 | - Fixed a bug that could make it unpredictable whether the QL compiler 1934 | reports problems about query metadata tags, and thereby make `codeql 1935 | test run` fail spuriously in some cases. 1936 | 1937 | ## Release 2.9.1 (2022-05-05) 1938 | 1939 | - The bundled extractors are updated to match the versions currently 1940 | used on LGTM.com. These are newer than the last release (1.30) of 1941 | LGTM Enterprise. If you plan to upload databases to an LGTM 1942 | Enterprise 1.30 instance, you need to create them with release 1943 | 2.7.6. 1944 | 1945 | ## Release 2.9.0 (2022-04-26) 1946 | 1947 | - The bundled extractors are updated to match the versions currently 1948 | used on LGTM.com. These are newer than the last release (1.30) of 1949 | LGTM Enterprise. If you plan to upload databases to an LGTM 1950 | Enterprise 1.30 instance, you need to create them with release 1951 | 2.7.6. 1952 | 1953 | ### New features 1954 | 1955 | - `codeql database create` now supports the `--[no-]-count-lines` 1956 | option, which was previously only available with `codeql database 1957 | init`. 1958 | 1959 | - `codeql resolve files` and `codeql database index-files` has a new 1960 | `--also-match` option, which allows users to specify glob patterns 1961 | that are applied in conjunction with the existing `--include` 1962 | option. 1963 | 1964 | ### New language features 1965 | 1966 | - This release introduces experimental support for parameterized QL 1967 | modules. This language feature is still subject to change and should 1968 | not be used in production yet. 1969 | 1970 | ### Bugs fixed 1971 | 1972 | - Fixed a bug that would prevent resolution of a query suite in a 1973 | published CodeQL query pack that has a reference to the pack itself. 1974 | 1975 | - Fixed inaccurate documentation of what the `--include-extension` 1976 | option to `codeql resolve files` and `codeql database index-files` 1977 | does. The actual behavior is unchanged. 1978 | 1979 | ## Release 2.8.5 (2022-04-07) 1980 | 1981 | - The bundled extractors are updated to match the versions currently 1982 | used on LGTM.com. These are newer than the last release (1.30) of 1983 | LGTM Enterprise. If you plan to upload databases to an LGTM 1984 | Enterprise 1.30 instance, you need to create them with release 1985 | 2.7.6. 1986 | 1987 | - There are no user-facing changes in this release. 1988 | 1989 | ## Release 2.8.4 (2022-03-29) 1990 | 1991 | - The bundled extractors are updated to match the versions currently 1992 | used on LGTM.com. These are newer than the last release (1.29) of 1993 | LGTM Enterprise. If you plan to upload databases to an LGTM 1994 | Enterprise 1.29 instance, you need to create them with release 1995 | 2.6.3. 1996 | 1997 | ### Bugs fixed 1998 | 1999 | - Fixed an error where running out of memory during query evaluation 2000 | would cause `codeql` to exit with status 34 instead of the 99 that 2001 | is documented for this condition. 2002 | 2003 | - Fixed a bug in our handling of Clang's header maps, which caused 2004 | missing files for Xcode-based projects on macOS (e.g. WebKit). 2005 | 2006 | ## Release 2.8.3 (2022-03-14) 2007 | 2008 | - This release of CodeQL (and all future ones) will not include the 2009 | CodeQL runner, which is now deprecated. For more information, and 2010 | instructions on how to migrate to using the CodeQL CLI, see 2011 | [CodeQL runner deprecation][11]. 2012 | 2013 | [11]: https://github.blog/changelog/2021-09-21-codeql-runner-deprecation/ 2014 | 2015 | - The bundled extractors are updated to match the versions currently 2016 | used on LGTM.com. These are newer than the last release (1.29) of 2017 | LGTM Enterprise. If you plan to upload databases to an LGTM 2018 | Enterprise 1.29 instance, you need to create them with release 2019 | 2.6.3. 2020 | 2021 | ### New features 2022 | 2023 | - Executable binaries for Windows are now digitally signed by a GitHub 2024 | certificate. 2025 | 2026 | ### Other changes 2027 | 2028 | - The evaluator logs produced by `--evaluator-log` now default to the 2029 | maximum verbosity level and will therefore contain more information 2030 | (and, accordingly, grow larger). The verbosity level can still be 2031 | configured with `--evaluator-log-level`. In particular, 2032 | `--evaluator-log-level=1` will restore the previous default behavior. 2033 | 2034 | ## Release 2.8.2 (2022-02-28) 2035 | 2036 | - The bundled extractors are updated to match the versions currently 2037 | used on LGTM.com. These are newer than the last release (1.29) of 2038 | LGTM Enterprise. If you plan to upload databases to an LGTM 2039 | Enterprise 1.29 instance, you need to create them with release 2040 | 2.6.3. 2041 | 2042 | ### Breaking change 2043 | 2044 | - The support for the output formats SARIF v1.0.0 and SARIF v2.0.0 2045 | (Committee Specification Draft 1) that were deprecated in 2.7.1 has 2046 | been removed. If you need this functionality, please file a public 2047 | issue against https://github.com/github/codeql-cli-binaries, or open 2048 | a private ticket with GitHub Support and request an escalation to 2049 | engineering. 2050 | 2051 | ### New Features 2052 | 2053 | - The CodeQL CLI is now compatible with Windows 11 and Windows Server 2054 | 2022, including building databases for compiled languages. 2055 | 2056 | ## Release 2.8.1 (2022-02-15) 2057 | 2058 | - The bundled extractors are updated to match the versions currently 2059 | used on LGTM.com. These are newer than the last release (1.29) of 2060 | LGTM Enterprise. If you plan to upload databases to an LGTM 2061 | Enterprise 1.29 instance, you need to create them with release 2062 | 2.6.3. 2063 | 2064 | ### New Features 2065 | 2066 | - Commands that find or run queries now allow you to refer to queries 2067 | within a named CodeQL pack. For example: 2068 | 2069 | ```sh 2070 | # Analyze a database using all queries in the experimental/Security folder within the codeql/cpp-queries 2071 | # CodeQL query pack. 2072 | codeql database analyze --format=sarif-latest --output=results \ 2073 | codeql/cpp-queries:experimental/Security 2074 | 2075 | # Analyse using only the RedundantNullCheckParam.ql query in the codeql/cpp-queries CodeQL query pack. 2076 | codeql database analyze --format=sarif-latest --output=results \ 2077 | 'codeql/cpp-queries:experimental/Likely Bugs/RedundantNullCheckParam.ql' 2078 | 2079 | # Analyse using the cpp-security-and-quality.qls query suite in the codeql/cpp-queries CodeQL query pack. 2080 | codeql database analyze --format=sarif-latest --output=results \ 2081 | 'codeql/cpp-queries:codeql-suites/cpp-security-and-quality.qls' 2082 | 2083 | # Analyse using the cpp-security-and-quality.qls query suite from a version of the codeql/cpp-queries pack 2084 | # that is >= 0.0.3 and < 0.1.0 (the highest compatible version will be chosen). 2085 | # All valid semver ranges are allowed. See https://docs.npmjs.com/cli/v6/using-npm/semver#ranges 2086 | codeql database analyze --format=sarif-latest --output=results \ 2087 | 'codeql/cpp-queries@~0.0.3:codeql-suites/cpp-security-and-quality.qls' 2088 | ``` 2089 | 2090 | The complete way to specify a set of queries is in the form 2091 | `scope/name@range:path`, where: 2092 | 2093 | - `scope/name` is the qualified name of a CodeQL pack. 2094 | - `range` is a [semver range][10]. 2095 | - `path` is a file system path 2096 | 2097 | If a `scope/name` is specified, the `range` and `path` are 2098 | optional. A missing `range` implies the latest version of the 2099 | specified pack. A missing `path` implies the default query suite 2100 | of the specified pack. 2101 | 2102 | The `path` can be one of a `*.ql` query file, a directory 2103 | containing one or more queries, or a `.qls` query suite file. If 2104 | there is no pack name specified, then a `path` must be provided, 2105 | and will be interpreted relative to the current working directory 2106 | of the current process. 2107 | 2108 | If a `scope/name` and `path` are specified, then the `path` cannot 2109 | be absolute. It is considered relative to the root of the CodeQL 2110 | pack. 2111 | 2112 | The relevant commands are: 2113 | - `codeql database analyze` 2114 | - `codeql database run-queries` 2115 | - `codeql execute queries` 2116 | - `codeql resolve queries` 2117 | 2118 | [10]: https://docs.npmjs.com/cli/v6/using-npm/semver#ranges 2119 | 2120 | ### Bugs fixed 2121 | 2122 | - Fixed a bug that would sometimes lead to query evaluation on 2123 | M1-based Macs to crash with `Did not preallocate enough memory` 2124 | error. 2125 | 2126 | ## Release 2.8.0 (2022-02-04) 2127 | 2128 | - The bundled extractors are updated to match the versions currently 2129 | used on LGTM.com. These are newer than the last release (1.29) of 2130 | LGTM Enterprise. If you plan to upload databases to an LGTM 2131 | Enterprise 1.29 instance, you need to create them with release 2132 | 2.6.3. 2133 | 2134 | ### Breaking change 2135 | 2136 | - The CodeQL Action versions up to and including version 1.0.22 are 2137 | not compatible with the CodeQL CLI 2.8.0 and later. The CLI 2138 | will emit an error if it detects that it is being used by an 2139 | incompatible version of the codeql-action. 2140 | 2141 | ### New features 2142 | 2143 | - A new extractor option has been added to the Java extractor. The 2144 | flag `--extractor-option exclude=''` allows specifying a glob 2145 | that describes which paths need to be excluded from extraction but 2146 | still need to be compiled. This is useful when some files are necessary 2147 | for a successful build but are uninteresting for analysis. 2148 | 2149 | See also: https://codeql.github.com/docs/codeql-cli/extractor-options/ 2150 | 2151 | - Summary metrics can now associate messages with their results, for 2152 | instance to report the name and number of uses of a particular API 2153 | endpoint within a repository. To associate messages with summary 2154 | metrics, define a query with `@kind metric` and `@tags summary` metadata 2155 | and use either the `location, message, value` or the `message, value` 2156 | results pattern. 2157 | 2158 | ### Bug fixed 2159 | 2160 | - Fixed a bug where `codeql resolve upgrades` ignores the 2161 | `--target-dbscheme` option. 2162 | 2163 | ## Release 2.7.6 (2022-01-24) 2164 | 2165 | - The bundled extractors are updated to match the versions currently 2166 | used on LGTM.com. These are newer than the last release (1.28) of 2167 | LGTM Enterprise. If you plan to upload databases to an LGTM 2168 | Enterprise 1.28 instance, you need to create them with release 2169 | 2.5.9. 2170 | 2171 | ### Bug fixed 2172 | 2173 | - A bug where creation of a CodeQL database could sometimes fail with 2174 | a `NegativeArraySizeException` has now been fixed. 2175 | 2176 | ### New feature 2177 | 2178 | - The CLI and evaluator contain a number of new features in support of 2179 | internal machine learning experiments. This includes an experimental 2180 | `resolve ml-models` subcommand and new `mlModels` metadata in pack 2181 | definition files. As these new features are not yet ready for general 2182 | use, they should be ignored by external CodeQL users. 2183 | 2184 | ## Release 2.7.5 (2022-01-17) 2185 | 2186 | - The bundled extractors are updated to match the versions currently 2187 | used on LGTM.com. These are newer than the last release (1.28) of 2188 | LGTM Enterprise. If you plan to upload databases to an LGTM 2189 | Enterprise 1.28 instance, you need to create them with release 2190 | 2.5.9. 2191 | 2192 | ### Deprecation 2193 | 2194 | - The CodeQL Action versions up to and including version 1.0.22 are 2195 | now deprecated for use with CodeQL CLI 2.7.5 and later. The CLI 2196 | will emit a warning if it detects that it is being used by a 2197 | deprecated version of the codeql-action. This warning will become a 2198 | fatal error with version 2.8.0 of the CLI. 2199 | 2200 | ### New feature 2201 | 2202 | - The `codeql github upload-results` command will now print the API 2203 | response body in JSON format if a `--format=json` flag is 2204 | given. Otherwise the command will print the URL of the SARIF 2205 | upload. This URL can be used to get status information for the 2206 | upload. 2207 | 2208 | See also: https://docs.github.com/en/rest/reference/code-scanning 2209 | 2210 | ### Documentation fixes 2211 | 2212 | - The documentation for the `--trace-process-level` flag of `codeql 2213 | database init` (which is used with indirect build tracing on 2214 | Windows) was erroneous. 2215 | 2216 | The help text previously claimed that `--trace-process-level=1` 2217 | would inject CodeQL's build tracer into the calling process. This is 2218 | actually what `--trace-process-level=0` achieves. The help text has 2219 | now been corrected to match the actual (unchanged) behavior. 2220 | 2221 | Also, some log messages incorrectly stated which process CodeQL was 2222 | injected into. These have also been corrected. 2223 | 2224 | ### Other changes 2225 | 2226 | - For commands that run queries, the `--timeout` option now controls 2227 | the maximal time it may take to evaluate a "layer" of a query rather 2228 | than a "stage". There are usually many "layers" in each "stage", 2229 | but it is usually a single one of the layers in a stage that uses 2230 | most of the time, so there is no need to reduce existing timeout 2231 | values as a result of this change. 2232 | 2233 | ## Release 2.7.4 2234 | 2235 | This release was skipped. 2236 | 2237 | ## Release 2.7.3 (2021-12-06) 2238 | 2239 | - The bundled extractors are updated to match the versions currently 2240 | used on LGTM.com. These are newer than the last release (1.28) of 2241 | LGTM Enterprise. If you plan to upload databases to an LGTM 2242 | Enterprise 1.28 instance, you need to create them with release 2243 | 2.5.9. 2244 | 2245 | ### Potentially breaking changes 2246 | 2247 | - The experimental command-line option `--ml-model-path` that was 2248 | introduced to support internal experiments has been removed. 2249 | 2250 | ### Bugs fixed 2251 | 2252 | - Editing support (content assist, code navigation, etc.) in files 2253 | under the `.github` directory will now work properly. This is 2254 | because files under the `.github` directory will now be indexed and 2255 | processed by the CodeQL language server. Other hidden directories 2256 | that start with `.` will remain un-indexed. This affects the 2257 | vscode-codeql extension and any other IDE extension that uses 2258 | the CodeQL language server. 2259 | 2260 | - Fixed authentication with GitHub package registries via the 2261 | `GITHUB_TOKEN` environment variable and the `--github-auth-stdin` 2262 | flag when downloading and publishing packs. 2263 | 2264 | - Fixed an incompatibility with glibc version 2.34 on Linux, where 2265 | build tracing failed with an error message. 2266 | 2267 | - Fixed a bug where `codeql generate log-summary` could sometimes fail 2268 | with a `JsonMappingException`. 2269 | 2270 | ### New features 2271 | 2272 | - The CodeQL CLI for Mac OS now ships with a native Java virtual machine for M1 Macs, 2273 | and this will be used by default where applicable to run the CodeQL 2274 | engine, thus improving performance. 2275 | [Rosetta 2](https://support.apple.com/en-us/HT211861) is still 2276 | required as not all components of the CodeQL CLI are natively compiled. 2277 | 2278 | - Commands that execute queries will now exit with status code 34 if 2279 | certain errors that prevent the evaluation of one or more 2280 | individual queries are detected. Previously some of these errors 2281 | would crash the evaluator and exit with status code 100. 2282 | 2283 | (This is currently used for "external predicate not found" errors). 2284 | 2285 | ## Release 2.7.2 (2021-11-22) 2286 | 2287 | - The bundled extractors are updated to match the versions currently 2288 | used on LGTM.com. These are newer than the last release (1.28) of 2289 | LGTM Enterprise. If you plan to upload databases to an LGTM 2290 | Enterprise 1.28 instance, you need to create them with release 2291 | 2.5.9. 2292 | 2293 | ### Potentially breaking changes 2294 | 2295 | - The Java extractor now defaults to extracting all XML documents under 2296 | 10MB in size, a change from the previous default of only extracting 2297 | documents with particular well-known names (e.g. `pom.xml`). However, 2298 | if the source tree contains more than 50MB of XML in total, it prints 2299 | a warning and falls back to the old default behaviour. 2300 | Set the environment variable `LGTM_INDEX_XML_MODE` to `byname` to get 2301 | the old default behaviour, or `all` to extract all documents under 2302 | 10MB regardless of total size. 2303 | 2304 | - The experimental command-line option `--native-library-path` that was 2305 | introduced to support internal experiments has been removed. 2306 | 2307 | - The beta `codeql pack publish` command will now prevent accidental 2308 | publishing of packages with pre-release version qualifiers. Prerelease 2309 | versions are those that include a `-` after the major, minor, and patch 2310 | versions such as `1.2.3-dev`. To avoid this change, use the 2311 | `--allow-prerelease` option. 2312 | 2313 | ### Bugs fixed 2314 | 2315 | - Fixed an issue when using the `--evaluator-log` option where a 2316 | `NullPointerException` could sometimes occur non-deterministically. 2317 | 2318 | - Fixed bugs observed when using indirect build tracing using a CodeQL 2319 | distribution unpacked to a path containing spaces or on Arch Linux. 2320 | 2321 | ### New features 2322 | 2323 | - CodeQL databases now contain metadata about how and when they were 2324 | created. This can be found in the `creationMetadata` field of the 2325 | `codeql-database.yml` file within the CodeQL database directory. More 2326 | information may be added to this field in future releases. 2327 | 2328 | ## Release 2.7.1 (2021-11-15) 2329 | 2330 | - The bundled extractors are updated to match the versions currently 2331 | used on LGTM.com. These are newer than the last release (1.28) of 2332 | LGTM Enterprise. If you plan to upload databases to an LGTM 2333 | Enterprise 1.28 instance, you need to create them with release 2334 | 2.5.9. 2335 | 2336 | ### Potentially breaking changes 2337 | 2338 | - Previously, `codeql test run` would fall back to looking for an 2339 | accompanying `queries.xml` file if it found a `qlpack.yml` that did 2340 | not declare an extractor to use when extracting a test database. 2341 | This has been removed because the internal use case that 2342 | neccessitated the fallback are now removed. If you suddenly 2343 | encounter errors that complain of missing extractor declarations, 2344 | check whether you had a `queries.xml` you were inadvertently relying 2345 | on. 2346 | 2347 | - When queries are specified by naming a directory to scan for `*.ql` 2348 | files, subdirectories named `.codeql` will now be ignored. The new 2349 | QL packaging support uses subdirectories with this name of various 2350 | scratch and caching purposes, so they may contain `*.ql` files that 2351 | are not intended to be directly user-visible. 2352 | 2353 | - When copying dependencies for CodeQL packages into a query pack 2354 | bundle, `*.ql` files in these dependencies will now be included 2355 | inside of the query pack's `.codeql` directory. 2356 | 2357 | - The tables printed by `codeql database analyze` to summarize the 2358 | results of diagnostic and metric queries that were part of the 2359 | analysis have a new format and contains less (but hopefully more 2360 | pertinent) information. We recommend against attempting to parse 2361 | this human-readable output programmatically. Instead, use the 2362 | `runs[].tool.driver.invocations[].toolExecutionNotifications` 2363 | property in the SARIF output. 2364 | 2365 | - The experimental plumbing command `codeql pack packlist` has a new 2366 | format for its JSON results. Previously, the results were a list of 2367 | paths. Now, the results are an object with a single property `paths` 2368 | that contains the list of paths. 2369 | 2370 | - The internal `qlpacks` directory of the CodeQL bundle available on the 2371 | [CodeQL Action releases page](https://github.com/github/codeql-action/releases/) 2372 | has a new structure. This directory is internal to the CLI and can change without 2373 | notice in future releases. 2374 | 2375 | The currently-shipped `qlpacks` directory mirrors the structure of [CodeQL package](https://github.blog/changelog/2021-07-28-introducing-the-codeql-package-manager-public-beta/) caches and looks like this: 2376 | 2377 | ```text 2378 | qlpacks 2379 | - codeql 2380 | - {lang}-all 2381 | - {version} 2382 | - qlpack contents 2383 | - {lang}-examples 2384 | - {version} 2385 | - qlpack contents 2386 | - {lang}-queries 2387 | - {version} 2388 | - qlpack contents 2389 | - {lang}-upgrades 2390 | - {version} 2391 | - qlpack contents 2392 | - ... and so on for all languages 2393 | ``` 2394 | 2395 | ### Deprecations 2396 | 2397 | - The output formats SARIF v1.0.0 and SARIF v2.0.0 (Committee 2398 | Specification Draft 1) have been deprecated. They will be removed 2399 | in a later version (earliest 2.8.0). If you need this 2400 | functionality, please file a public issue against 2401 | https://github.com/github/codeql-cli-binaries, or open a private 2402 | ticket with GitHub Support and request an escalation to engineering. 2403 | 2404 | - The `qlpack:` instruction in query suite definitions has been 2405 | deprecated due to uncertainty about whether it is intended to 2406 | include _all_ the `*.ql` files in the named pack, or only the pack's 2407 | "default query suite". The behavior of the instruction is 2408 | determined by whether the named pack declares any default query 2409 | suite, but this means that a pack _starting_ to declare such a suite 2410 | may break the behavior of existing query suites that reference the 2411 | pack from outside. 2412 | 2413 | We recommend replacing `qlpack:` by one of 2414 | ```yaml 2415 | - queries: '.' # import all *.ql files 2416 | from: some/pack-name 2417 | version: 1.2.3 # optional 2418 | ``` 2419 | or 2420 | ```yaml 2421 | - import: path/to/actual/suite.ql # just that suite 2422 | from: some/pack-name 2423 | version: 1.2.3 # optional 2424 | ``` 2425 | 2426 | A warning will now be printed when a `qlpack:` instruction resolves 2427 | to a default suite, because that is the case where the effect may 2428 | not be what the query suite author intended. 2429 | 2430 | ### Bugs fixed 2431 | 2432 | - Fixed a bug where the `paths` and `paths-ignore` properties of a 2433 | Code Scanning config file specified using `--codescanning-config` 2434 | were being interpreted the wrong way around. 2435 | 2436 | - Fixed a bug where queries specified using the 2437 | `--codescanning-config` option could not be run after an explicit 2438 | call to `codeql database finalize`. 2439 | 2440 | - Fixed a bug where `-J` options would erroneously be recognized even 2441 | after `--` on the command line. 2442 | 2443 | - When running `codeql database analyze` and `codeql database 2444 | interpret-results` without the `--sarif-group-rules-by-pack` flag, 2445 | the SARIF output did not include baseline lines-of-code counts. This 2446 | is now fixed. 2447 | 2448 | - Fixed a bug where expansion of query suites would sometimes fail if 2449 | a query suite in a compiled query pack referenced that pack itself 2450 | explicitly. 2451 | 2452 | ### New language features 2453 | 2454 | - [Set literal expressions][9] can now optionally contain a trailing 2455 | comma after the last element. 2456 | 2457 | [9]: https://codeql.github.com/docs/ql-language-reference/expressions/#set-literal-expressions 2458 | 2459 | ### New features 2460 | 2461 | - Beta support for database creation on Apple Silicon has been added. 2462 | It depends on the following requirements: 2463 | 2464 | - [Rosetta 2][8] needs to be installed 2465 | 2466 | [8]: https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment 2467 | 2468 | - Developer tools need to be installed. CodeQL requires the `lipo`, 2469 | `codesign`, and `install_name_tool` tools to be present. 2470 | 2471 | - Build systems invoking `csh` may experience [intermittent 2472 | crashes][7]. 2473 | 2474 | [7]: https://openradar.appspot.com/radar?id=4936797431791616 2475 | 2476 | - `codeql database analyze` can now include query-specific help texts 2477 | for alerts in the SARIF output (for SARIF v2.1.0 or later). The help 2478 | text must be located in an `.md` file next to (and with the same 2479 | basename as) the `.ql` file for each query. Since this can 2480 | significantly increase SARIF file size, the feature is not enabled 2481 | by default; give a `--sarif-add-query-help` option to enable it. 2482 | 2483 | - The query metadata validator now knows about queries that produce 2484 | alert scores, so these queries no longer need to be run with a 2485 | `--no-metadata-verification` flag. 2486 | 2487 | - `codeql database create` and `codeql-finalize` have a new flag 2488 | `--skip-empty` that will cause a language with no extracted source 2489 | code to be ignored with a warning instead of treated like a fatal 2490 | error. This can be useful with `--db-cluster` where not all of the 2491 | languages may exist in the source tree. It will not be possible to 2492 | run queries against the skipped database. 2493 | 2494 | - `codeql resolve extractor` and `codeql resolve languages` now 2495 | support an extended output format `--format=betterjson` wich 2496 | includes information about each extractor's language-specific 2497 | options. 2498 | 2499 | - This release introduces rudimentary support for parallelizing 2500 | database creation by importing unfinished databases (or database 2501 | clusters) into another unfinished database (or cluster) under 2502 | creation. This is implemented by the new flag `--additional-dbs` for 2503 | `codeql database finalize`, or the new plumbing command `codeql 2504 | database import`. 2505 | 2506 | - `codeql database create`, `codeql database index-files`, and `codeql 2507 | database trace-command` support a [unified syntax for passing 2508 | language-specific options][6] to the extractor with the new 2509 | `--extractor-option` and `--extractor-options-file` options. 2510 | (The extractors do not make use of this yet, though). 2511 | 2512 | [6]: https://codeql.github.com/docs/codeql-cli/extractor-options 2513 | 2514 | ## Release 2.7.0 (2021-10-27) 2515 | 2516 | - The extractor for Ruby is now included. CodeQL analysis for Ruby is 2517 | currently in beta. During the beta, analysis of Ruby will not be as 2518 | comprehensive as CodeQL analysis of other languages. The source code 2519 | of the extractor and the queries can be found in the 2520 | [`github/codeql`](https://github.com/github/codeql) repository. 2521 | 2522 | - The bundled extractors are updated to match the versions currently 2523 | used on LGTM.com. These are newer than the last release (1.28) of 2524 | LGTM Enterprise. If you plan to upload databases to an LGTM 2525 | Enterprise 1.28 instance, you need to create them with release 2526 | 2.5.9. 2527 | 2528 | ### Bugs fixed 2529 | 2530 | - Fixed a bug where indirect tracing would sometimes not manage to 2531 | observe build processes if certain environment variables were unset 2532 | during the build. 2533 | 2534 | ## Release 2.6.3 (2021-10-06) 2535 | 2536 | - The bundled extractors are updated to match the versions currently 2537 | used on LGTM.com. These are newer than the last release (1.28) of 2538 | LGTM Enterprise. If you plan to upload databases to an LGTM 2539 | Enterprise 1.28 instance, you need to create them with release 2540 | 2.5.9. 2541 | 2542 | ### Potentially breaking changes 2543 | 2544 | - The option `--compiler-spec` accepted by some subcommands of `codeql 2545 | database` is deprecated. It will be removed in a later version 2546 | (earliest 2.7.0). If you need this option, please file a public 2547 | issue in https://github.com/github/codeql-cli-binaries, or open a 2548 | private ticket with GitHub support and request an escalation to 2549 | engineering. 2550 | 2551 | - By default, databases created using the CodeQL CLI will now have 2552 | their underlying datasets finalized, meaning that no further data 2553 | can be subsequently imported into them. This change should not 2554 | affect most users. 2555 | 2556 | - The `codeql resolve qlref` command will now throw an error when the 2557 | target is ambiguous. The qlref resolution rules are now as follows: 2558 | 2559 | 1. If the target of a qlref is in the same qlpack, then that target 2560 | is always returned. 2561 | 2562 | 2. If multiple targets of the qlref are found in dependent packs, 2563 | this is an error. 2564 | 2565 | Previously, the command would have arbitrarily chosen one of the targets and ignored any 2566 | ambiguities. 2567 | 2568 | ### Bugs fixed 2569 | 2570 | - Linux/MacOS: When tracing a build that involves an 2571 | `execvp`/`execvpe` (Linux-only)/`posix_spawnp` syscall where `PATH` 2572 | was not set in the environment, CodeQL sometimes would break the 2573 | build. Now, CodeQL uses the correct, platform-specific fallback for 2574 | `PATH` instead. 2575 | 2576 | - Linux/MacOS: When tracing a build that involves an `execvpe` 2577 | (Linux-only)/`posix_spawnp` syscall, the `PATH` lookup of the 2578 | executable wrongly took place in the environment provided via 2579 | `envp`, instead of the environment of the process calling 2580 | `execvpe`/`posix_spawnp`. Now, the correct environment is used for 2581 | the `PATH` lookup. 2582 | 2583 | - A bug where query compilation would sometimes fail with a 2584 | `StackOverflowError` when compiling a query that uses `instanceof` 2585 | has now been fixed. 2586 | 2587 | ### New features 2588 | 2589 | - The `codeql query compile` command now accepts a `--keep-going` or 2590 | `-k` option, which indicates that the compiler should continue 2591 | compiling queries even if one of the queries has a compile error in 2592 | it. 2593 | 2594 | - CLI commands now run default queries if none are specified. If no 2595 | queries are specified, the `codeql database analyze`, `codeql 2596 | database run-queries`, and `codeql database interpret-results` 2597 | commands will now run the default suite for the language being 2598 | analyzed. 2599 | 2600 | - `codeql pack publish` now copies the published package to the local 2601 | package cache. In addition to publishing to a remote repository, the 2602 | `codeql pack publish` command will also copy the published package 2603 | to the local package cache. 2604 | 2605 | ## Release 2.6.2 (2021-09-21) 2606 | 2607 | - CodeQL CLI 2.6.2 includes the same functionality as **the CodeQL 2608 | runner**, which is being deprecated. For more information, see 2609 | [CodeQL runner deprecation][5]. 2610 | 2611 | [5]: https://github.blog/changelog/2021-09-21-codeql-runner-deprecation/ 2612 | 2613 | - The bundled extractors are updated to match the versions currently 2614 | used on LGTM.com. These are newer than the last release (1.28) of 2615 | LGTM Enterprise. If you plan to upload databases to an LGTM 2616 | Enterprise 1.28 instance, you need to create them with release 2617 | 2.5.9. 2618 | 2619 | ### Bugs fixed 2620 | 2621 | - A bug where `codeql generate log-summary` would sometimes crash with 2622 | a `JsonMappingException` has been fixed. 2623 | 2624 | ### New features 2625 | 2626 | - The CodeQL CLI now counts the lines of code found under 2627 | `--source-root` when `codeql database init` or `codeql database 2628 | create` is called. This information can be viewed later by either 2629 | the new `codeql database print-baseline` command or the new 2630 | `--print-baseline-loc` argument to `codeql database 2631 | interpret-results`. 2632 | 2633 | - `qlpack.yml` files now support an additional field `include` in 2634 | which glob patterns of additional files that should be included (or 2635 | excluded) when creating a given CodeQL pack can be specified. 2636 | 2637 | - QL packs created by the experimental `codeql pack create` command 2638 | will now include some information about the build in a new 2639 | `buildMetadata` field of their `qlpack.yml` file. 2640 | 2641 | - `codeql database create` now supports the same flags as `codeql 2642 | database init` for automatically recognizing the languages present 2643 | in checkouts of GitHub repositories: 2644 | 2645 | - `--github-url` accepts the URL of a custom GitHub instance 2646 | (previously only `github.com` was supported). 2647 | 2648 | - `--github-auth-stdin` allows a personal access token to be 2649 | provided through standard input (previously only the 2650 | `GITHUB_TOKEN` environment variable was supported). 2651 | 2652 | ### Notable documentation changes 2653 | 2654 | - Documentation has been added detailing how to use the "indirect 2655 | build tracing" feature, which is enabled by using the 2656 | `--begin-tracing` flag provided by `codeql database init`. The new 2657 | documentation can be found [here][4]. This feature was temporarily 2658 | described as "sandwiched tracing" in the 2.6.0 release notes. 2659 | 2660 | [4]: https://aka.ms/codeql-docs/indirect-tracing 2661 | 2662 | ## Release 2.6.1 (2021-09-07) 2663 | 2664 | - The bundled extractors are updated to match the versions currently 2665 | used on LGTM.com. These are newer than the last release (1.28) of 2666 | LGTM Enterprise. If you plan to upload databases to an LGTM 2667 | Enterprise 1.28 instance, you need to create them with release 2668 | 2.5.9. 2669 | 2670 | ### Potentially breaking changes 2671 | 2672 | - The `codeql resolve qlref` command will now throw an error when the 2673 | target is ambiguous. 2674 | 2675 | The qlref resolution rules are now as follows: 2676 | 2677 | 1. If the target of a qlref is in the same qlpack, then that target is 2678 | always returned. 2679 | 2680 | 2. If multiple targets of the qlref are found in dependent packs, this 2681 | is an error. 2682 | 2683 | Previously, the command would have arbitrarily chosen one of the targets 2684 | and ignored any ambiguities. 2685 | 2686 | - The `qlpack` directive in query suites has its semantics changed. 2687 | Previously, this directive would return all queries in the 2688 | qlpack. Now, the directive returns only those queries matched by the 2689 | `defaultSuite` directive in the query pack. Here is an example: 2690 | 2691 | Consider a `qlpack.yml` like the following: 2692 | 2693 | ```yml 2694 | name: codeql/my-qlpack 2695 | version: 0.0.1 2696 | defaultSuite: 2697 | queries: standard 2698 | ``` 2699 | 2700 | And the directory structure is the following: 2701 | 2702 | ```text 2703 | qlpack.yml 2704 | standard/ 2705 | a.ql 2706 | experimental/ 2707 | b.ql 2708 | ``` 2709 | 2710 | A query suite `suite.qls` like this: 2711 | 2712 | ```yml 2713 | - qlpack: codeql/my-qlpack 2714 | ``` 2715 | 2716 | Previously, would return all the queries in all subdirectories (i.e, 2717 | `standard/a.ql` and `experimental/b.ql`). Now, it only returns 2718 | `standard/a.ql`, since that is the only query matched by its default 2719 | suite. 2720 | 2721 | If you want to have the same behavior as before, you must update your 2722 | query suites to use the `queries` directive with a `from` attribute, 2723 | like this: 2724 | 2725 | ```yml 2726 | - queries: . 2727 | from: codeql/my-qlpack 2728 | ``` 2729 | 2730 | ### New features 2731 | 2732 | - Commands that evaluate CodeQL queries now support an additional 2733 | option `--evaluator-log=path/to/log.json` that will result in the 2734 | evaluator producing a structured log (in JSON format) of events that 2735 | occurred during evaluation in order to aid debugging of query 2736 | performance. The format of these logs will be subject to change with 2737 | no notice as we make modifications to the evaluator. 2738 | 2739 | There is also a new CLI command `codeql generate log-summary` that 2740 | will produce a summary of the predicates that were evaluated from 2741 | these event logs. We will aim to keep this summary format more 2742 | stable, although it is also subject to change. Unless you have a 2743 | good reason to use the event logs directly, it is strongly 2744 | recommended you use this command to produce summary logs and use 2745 | these instead. 2746 | 2747 | For further information on these new logs and additional options to 2748 | configure their format and verbosity, please refer to the CLI 2749 | documentation. 2750 | 2751 | ### New language features 2752 | 2753 | - QL classes can now be non-extending subtypes via the `instanceof` 2754 | keyword, allowing for a form of private subtyping that is not 2755 | visible externally. Methods of the supertype are accessible from 2756 | within a non-extending subtype class through extended semantics of 2757 | the `super` keyword. 2758 | 2759 | ``` 2760 | class Foo instanceof int { 2761 | Foo() { this in [1 .. 10] } 2762 | string toString() { result = "foo" + super.toString() } 2763 | } 2764 | ``` 2765 | 2766 | ## Release 2.6.0 (2021-08-24) 2767 | 2768 | - The bundled extractors are updated to match the versions currently 2769 | used on LGTM.com. These are newer than the last release (1.28) of 2770 | LGTM Enterprise. If you plan to upload databases to an LGTM 2771 | Enterprise 1.28 instance, you need to create them with release 2772 | 2.5.9. 2773 | 2774 | ### Bugs fixed 2775 | 2776 | - The `physicalLocation.artifactLocation.uri` fields in SARIF output 2777 | are now properly encoded as specified by RFC 3986. 2778 | 2779 | - The `--include-extension` option to the `codeql database 2780 | index-files` command no longer includes directories that are named 2781 | with the provided extension. For example, if the option 2782 | `--include-extension=.rb` is provided, then a directory named 2783 | `foo.rb/` will be excluded from the indexing. 2784 | 2785 | ### New features 2786 | 2787 | - A new `codeql database unbundle` subcommand performs the reverse of 2788 | `codeql database bundle` and extracts a CodeQL database from an 2789 | archive. 2790 | 2791 | - The CLI now understands per-codebase configuration files in [the 2792 | format already supported by the CodeQL Action][3]. The 2793 | configuration file must be given in a `--codescanning-config` option 2794 | to `codeql database create` or `codeql database init`. For some 2795 | languages, this configuration can contain pathname filters that 2796 | control which parts of the codebase is analysed; the configuration 2797 | file is the only way this functionality is exposed. The 2798 | configuration file can also control which queries are run, including 2799 | custom queries from repositories that must first be downloaded. To 2800 | actually use those queries, run `codeql database analyze` without 2801 | any query-selection arguments. 2802 | 2803 | [3]: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#example-configuration-files 2804 | 2805 | - The CLI now supports the "sandwiched tracing" feature that has 2806 | previously only been offered through the separate CodeQL Runner. 2807 | This feature is intended for use with CI systems that cannot be 2808 | configured to wrap build actions with `codeql database 2809 | trace-command`. Instead the CI system must be able to set custom 2810 | environment variables for each build action; the required 2811 | environment variables are output by `codeql database init` when 2812 | given a `--begin-tracing` argument. 2813 | 2814 | On Windows, `codeql database init --begin-tracing` will also inject 2815 | build-tracing code into the calling process or an ancestor; there 2816 | are additional options to control this. 2817 | 2818 | - This version contains _beta_ support for a new packaging and 2819 | publishing system for third-party QL queries and libraries. It 2820 | comprises the following new commands: 2821 | 2822 | - `codeql pack init`: Creates an empty CodeQL pack from a template. 2823 | 2824 | - `codeql pack add`: Adds a dependency to a CodeQL pack. 2825 | 2826 | - `codeql pack install`: Installs all pack dependencies specified in 2827 | the `qlpack.yml` file. 2828 | 2829 | - `codeql pack download`: Downloads one or more pack dependencies 2830 | into the global package cache. 2831 | 2832 | - `codeql pack publish`: Publishes a package to the GitHub Container 2833 | Registry. 2834 | 2835 | - (Plumbing) `codeql pack bundle`: Builds a `.zip` file for a CodeQL 2836 | query or library pack from sources. Used by `codeql pack publish`. 2837 | 2838 | - (Plumbing) `codeql pack create`: Creates a compiled CodeQL query 2839 | or library pack from sources. Used by `codeql pack bundle`. 2840 | 2841 | - (Plumbing) `codeql pack packlist`: Lists all files in a local 2842 | CodeQL pack that will be included in the pack's bundle. Used by 2843 | `codeql pack create`. 2844 | 2845 | - (Plumbing) `codeql pack resolve-dependencies`: Resolves all 2846 | transitive dependencies of a local CodeQL pack. Used by `codeql 2847 | pack install`. 2848 | 2849 | ## Release 2.5.9 (2021-08-09) 2850 | 2851 | This release corresponds to release 1.28.x of LGTM Enterprise, and 2852 | should be used when creating databases that will be uploaded to it. 2853 | Future CLI releases (numbered 2.6.x) may produce databases that are not 2854 | backwards compatible with this version of LGTM Enterprise. 2855 | 2856 | ## Release 2.5.8 (2021-07-26) 2857 | 2858 | - The bundled extractors are updated to match the versions currently 2859 | used on LGTM.com. These are newer than the last release (1.27) of 2860 | LGTM Enterprise. If you plan to upload databases to an LGTM 2861 | Enterprise 1.27 instance, you need to create them with release 2862 | 2.4.6. 2863 | 2864 | ### Potentially breaking changes 2865 | 2866 | - The QL compiler now verifies that `@security-severity` query 2867 | metadata is numeric. You can disable this verification by passing 2868 | the `--no-metadata-verification` flag. 2869 | 2870 | ### New features 2871 | 2872 | - The `database index-files` and `database trace-command` CLI commands 2873 | now support `--threads` and `--ram` options, which are passed to 2874 | extractors as suggestions. 2875 | 2876 | - The `database finalize` CLI command now supports the `--ram` option, 2877 | which controls memory usage for finalization. 2878 | 2879 | - The `database create` CLI command now supports the `--ram` option, 2880 | which controls memory usage for database creation. - The `generate 2881 | query-help` CLI command now support rendering query help in SARIF 2882 | format. 2883 | 2884 | ## Release 2.5.7 (2021-07-02) 2885 | 2886 | - The bundled extractors are updated to match the versions currently 2887 | used on LGTM.com. These are newer than the last release (1.27) of 2888 | LGTM Enterprise. If you plan to upload databases to an LGTM 2889 | Enterprise 1.27 instance, you need to create them with release 2890 | 2.4.6. 2891 | 2892 | ### New features 2893 | 2894 | - `codeql database create` and `codeql database init` can now 2895 | automatically recognise the languages present in checkouts of GitHub 2896 | repositories by making an API call to the GitHub server. This 2897 | requires a PAT token to either be set in the `GITHUB_TOKEN` 2898 | environment variable, or passed by stdin with the 2899 | `--github-auth-stdin` argument. 2900 | 2901 | - Operations that make outgoing HTTP calls (that is, `codeql github 2902 | upload-results` and the language-detection feature described above) 2903 | now support the use of HTTP proxies. To use a proxy, specify an 2904 | `$https_proxy` environment variable for HTTPS requests or a 2905 | `$http_proxy` environment variable for HTTP requests. If the 2906 | `$no_proxy` variable is also set, these variables will be ignored 2907 | and requests will be made without a proxy. 2908 | 2909 | ### New language features 2910 | 2911 | - The QL language now has a new method `toUnicode` on the `int` 2912 | type. This method converts Unicode codepoint to a one-character 2913 | string. For example, `65.toUnicode() = "A"`, `128512.toUnicode()` 2914 | results in a smiley, and `any(int i | i.toUnicode() = "A") = 65`. 2915 | 2916 | ## Release 2.5.6 (2021-06-22) 2917 | 2918 | - The bundled extractors are updated to match the versions currently 2919 | used on LGTM.com. These are newer than the last release (1.27) of 2920 | LGTM Enterprise. If you plan to upload databases to an LGTM 2921 | Enterprise 1.27 instance, you need to create them with release 2922 | 2.4.6. 2923 | 2924 | ### Features added 2925 | 2926 | - `codeql database create` (and the plumbing commands it comprises) 2927 | now supports creating databases for a source tree with several 2928 | languages while tracing a single build. This is enabled by a new 2929 | `--db-cluster` option. Once created, the multiple databases must be 2930 | _analyzed_ one by one. 2931 | 2932 | - `codeql database create` and `codeql database init` now accept an 2933 | `--overwrite` argument which will lead existing CodeQL databases to 2934 | be overwritten. 2935 | 2936 | - `codeql database analyze` now supports "diagnostic" queries (tagged 2937 | `@kind diagnostic`), which are intended to report information about 2938 | the analysis process itself rather than problems with the analyzed 2939 | code. The results of these queries will be summarized in a table 2940 | printed to the terminal when `codeql database analyze` finishes. 2941 | 2942 | They are also included in the analysis results in SARIF output 2943 | formats as [notification objects][2] so they can be displayed by 2944 | subsequent tooling such as the Code Scanning user interface. 2945 | 2946 | - For SARIF v2.1.0, a reporting descriptor object for each 2947 | diagnostic query is output to output to 2948 | `runs[].tool.driver.notifications`, or 2949 | `runs[].tool.extensions[].notifications` if running with 2950 | `--sarif-group-rules-by-pack`. A rule object for each diagnostic 2951 | query is output to `runs[].resources[].rules` for SARIF v2, or to 2952 | `runs[].rules` for SARIF v1. 2953 | 2954 | - Results of diagnostic queries are exported to the 2955 | `runs[].invocations[].toolExecutionNotifications` property in 2956 | SARIF v2.1.0, the `runs[].invocations[].toolNotifications` 2957 | property in SARIF v2, and the `runs[].toolNotifications` property 2958 | in SARIF v1. 2959 | 2960 | SARIF v2.1.0 output will now also contain version information for 2961 | query packs in `runs[].tool.extensions[].semanticVersion`, if the 2962 | Git commit the queries come from is known. 2963 | 2964 | [2]: https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317894 2965 | 2966 | - `codeql github upload-results` has a `--checkout-path` option which 2967 | will attempt to automatically configure upload target parameters. 2968 | When this is given, the `--commit` option will be taken from the 2969 | HEAD of the checkout Git repository, and if there is precisely one 2970 | remote configured in the local repository, the `--repository` and 2971 | `--github-url` options will also be automatically configured. 2972 | 2973 | - The CodeQL C++ extractor includes beta support for C++20. 2974 | This is only available when building codebases with GCC on Linux. 2975 | C++20 modules are **not** supported. 2976 | 2977 | ## Release 2.5.5 (2021-05-17) 2978 | 2979 | - The bundled extractors are updated to match the versions currently 2980 | used on LGTM.com. These are newer than the last release (1.27) of 2981 | LGTM Enterprise. If you plan to upload databases to an LGTM 2982 | Enterprise 1.27 instance, you need to create them with release 2983 | 2.4.6. 2984 | 2985 | ### Potentially breaking changes 2986 | 2987 | - When scanning the disk for QL packs and extractors, directories of 2988 | the form `.../SOMETHING/SOMETHING.testproj` (where the two 2989 | `SOMETHING` are identical) will now be ignored. Names of this form 2990 | are used by `codeql test run` for ephemeral test databases, which 2991 | can sometimes contain files that confuse QL compilations. 2992 | 2993 | ### Features added 2994 | 2995 | - Query writers can now optionally use `@severity` in place of 2996 | `@problem.severity` in the metadata for alert queries. SARIF 2997 | consumers should continue to consume this severity information using 2998 | the `rule.defaultConfiguration.level` property for SARIF v2.1.0, and 2999 | corresponding properties for other versions of SARIF. They should 3000 | not depend on the value stored in the `rule.properties` property 3001 | bag, since this will contain either `@problem.severity` or 3002 | `@severity` based on exactly what was written in the query metadata. 3003 | 3004 | - When exporting analysis results to SARIF v2.1.0, results and metric 3005 | results now contain a [reporting descriptor reference object][1] 3006 | that specifies the rule that produced them. For metric results, this 3007 | new property replaces the `metric` property. 3008 | 3009 | [1]: https://docs.oasis-open.org/sarif/sarif/v2.1.0/csprd01/sarif-v2.1.0-csprd01.html#_Toc10541300 3010 | 3011 | - `codeql database analyze` now outputs a table that summarizes the 3012 | results of metric queries that were part of the analysis. This can 3013 | be suppressed by passing the `--no-print-metrics-summary` flag. 3014 | 3015 | ### Bugs fixed 3016 | 3017 | - When using the `--sarif-group-rules-by-pack` flag to place the SARIF 3018 | rule object for each query underneath its corresponding query pack 3019 | in `runs[].tool.extensions`, the `rule` property of result objects 3020 | can now be used to look up the rule within the `rules` property of 3021 | the appropriate query pack in `runs[].tool.extensions`. Previously, 3022 | rule lookup for result objects in the SARIF output was not 3023 | well-defined when the `--sarif-group-rules-by-pack` flag was passed. 3024 | 3025 | ## Release 2.5.4 (2021-05-03) 3026 | 3027 | - This release is identical to release 2.5.3, except that 3028 | `codeql database analyze` no longer produces a generated 3029 | `automationDetails.id` field when the `--sarif-category` 3030 | is not explicitly provided. Previously, the 3031 | `--sarif-category` was autogenerated if not present. 3032 | - Code Scanning users should upgrade to this version and 3033 | avoid 2.5.3. 3034 | 3035 | ## Release 2.5.3 (2021-04-30) 3036 | 3037 | - The bundled extractors are updated to match the versions currently 3038 | used on LGTM.com. These are newer than the last release (1.27) of 3039 | LGTM Enterprise. If you plan to upload databases to an LGTM 3040 | Enterprise 1.27 instance, you need to create them with release 3041 | 2.4.6. 3042 | 3043 | ### Next release: Features added 3044 | 3045 | - When tracing a C/C++ build, the C compiler entries in compiler-settings 3046 | must now specify `order compiler,extractor`. The default configuration 3047 | already does this, so no change is necessary if using the default 3048 | configuration. 3049 | 3050 | - `codeql database analyze` and `codeql database interpret-results` 3051 | now report the results of summary metric queries in the 3052 | `.properties.metricResults` property of the SARIF output. 3053 | Summary metric queries describe metrics about the code analyzed by 3054 | CodeQL. They are identified by the query metadata `@kind metric` and 3055 | `@tag summary`. 3056 | For example, see the [lines of code summary metric query for 3057 | C++](https://github.com/github/codeql/blob/main/cpp/ql/src/Summary/LinesOfCode.ql). 3058 | 3059 | - `codeql database analyze` and `codeql database interpret-results` 3060 | now calculate an 3061 | [automation ID](https://docs.oasis-open.org/sarif/sarif/v2.1.0/cs01/sarif-v2.1.0-cs01.html#_Toc16012482) 3062 | and add it to the resulting SARIF. In SARIF v2.1.0, this field is 3063 | `runs[].automationDetails.id`. In SARIF v2, this field is 3064 | `runs[].automationLogicalId`. In SARIF v1, this field is 3065 | `runs[].automationId`. By default, this automation ID will be 3066 | derived from the database language and the operating system of the 3067 | machine that performed the run. It can be set explicitly using a new 3068 | `--sarif-category` option. 3069 | 3070 | - In query metadata, `@kind alert` and `@kind path-alert` are now 3071 | recognized as (more accurate) synonyms of `@kind problem` and 3072 | `@kind path-problem`, respectively. 3073 | 3074 | - Diagnostic queries are now permitted by the metadata verifier. They 3075 | are identified by `@kind diagnostic` metadata. Currently the result 3076 | patterns of diagnostic queries are not verified. This will change in 3077 | a future CLI release. 3078 | 3079 | ### Bugs fixed 3080 | 3081 | - Ensure the correct URL is generated during `codeql github upload-results` 3082 | for GitHub Enterprise Server. 3083 | 3084 | ## Release 2.5.2 (2021-04-21) 3085 | 3086 | This release is identical to release 2.5.1, except that an internal 3087 | incompatibility with the CodeQL action (and the codeql-runner that 3088 | some customers use for CI integrations) has been fixed. 3089 | 3090 | The fix does not affect any use cases where the CLI is downloaded from 3091 | github/codeql-cli-binaries, so if you're seeing this release there, 3092 | there's no need to upgrade from 2.5.1. 3093 | 3094 | ## Release 2.5.1 (2021-04-19) 3095 | 3096 | - The bundled extractors are updated to match the versions currently 3097 | used on LGTM.com. These are newer than the last release (1.27) of 3098 | LGTM Enterprise. If you plan to upload databases to an LGTM 3099 | Enterprise 1.27 instance, you need to create them with release 3100 | 2.4.6. 3101 | 3102 | ### Potentially breaking changes 3103 | 3104 | - The QL compiler will now reject queries where the query metadata (if 3105 | present) at the top of the `.ql` file is inconsistent with the 3106 | output format of the query. This check can be disabled by giving 3107 | the `--no-metadata-verification` flag. (The flag already existed 3108 | but has not had any effect until now.) 3109 | 3110 | ### Bugs fixed 3111 | 3112 | - Environment variables required for Java extraction are now 3113 | propagated by the tracer. This may resolve issues with tracing and 3114 | extraction in the context of certain build systems such as Bazel. 3115 | 3116 | - A number of `--check-CONDITION` options to `codeql database 3117 | finalize` and `codeql dataset import` designed to look for 3118 | consistency errors in the intermediate "TRAP" output from extractors 3119 | erroneously did nothing. They will now actually print warnings if 3120 | errors are found. The warnings become fatal errors if the new 3121 | `--fail-on-trap-errors` option is also given. 3122 | 3123 | ### Features added 3124 | 3125 | - `codeql resolve qlref` is a new command that takes in a `.qlref` 3126 | file for a CodeQL test case and returns the path of the `.ql` file 3127 | it references. 3128 | 3129 | - `codeql database analyze` and `codeql database interpret-results` 3130 | have a new `--sarif-group-rules-by-pack` option which will place the 3131 | SARIF rule object for each query underneath its corresponding query 3132 | pack in `runs[].tool.extensions`. 3133 | 3134 | - `codeql database finalize` and `codeql dataset import` have a new 3135 | `--fail-on-trap-errors` option that will make database creation fail 3136 | if extractors produce ill-formatted "TRAP" data for inclusion into a 3137 | database. This is not enabled by default because some of the 3138 | existing extractors have minor output bugs that cause the check to 3139 | fail. 3140 | 3141 | - `codeql database finalize` and `codeql dataset import` have a new 3142 | `--check-undefined-labels` option that enables stricter consistency 3143 | checks on the "TRAP" output from extractors. 3144 | 3145 | ### QL language improvements 3146 | 3147 | - `super` may now be used unqualified, e.g. `super.predicateName()`, 3148 | when the declaring class has multiple super types, as long as the 3149 | call itself is unambiguous. 3150 | 3151 | ## Release 2.5.0 (2021-03-26) 3152 | 3153 | - The bundled extractors are updated to match the versions currently 3154 | used on LGTM.com. These are newer than the last release (1.27) of 3155 | LGTM Enterprise. If you plan to upload databases to an LGTM 3156 | Enterprise 1.27 instance, you need to create them with release 3157 | 2.4.6. 3158 | 3159 | ### Potentially breaking changes 3160 | 3161 | - By default, `codeql test` now performs additional compiler 3162 | checks when extracting test code written in Java. 3163 | Existing Java tests that previously passed may therefore fail due 3164 | to this change, if they do not compile using the `javac` compiler. 3165 | To allow time to migrate existing tests, the new behavior can be 3166 | disabled by setting the environment variable 3167 | `CODEQL_EXTRACTOR_JAVA_FLOW_CHECKS=false`. 3168 | 3169 | ### Features added 3170 | 3171 | - Log files that contain output from build processes will now prefix 3172 | it with `[build-stdout]` and `[build-stderr]` instead of `[build]` 3173 | and `[build-err]`. In particular the latter sometimes caused 3174 | confusion. 3175 | 3176 | ### QL language improvements 3177 | 3178 | - The QL language now recognizes new `pragma[only_bind_into](...)` and 3179 | `pragma[only_bind_out](...)` annotations on expressions. Advanced users 3180 | may use these annotations to provide hints to the compiler to influence 3181 | binding behavior and thus indirectly performance. 3182 | 3183 | ## Release 2.4.6 (2021-03-19) 3184 | 3185 | This release corresponds to release 1.27.x of LGTM Enterprise, and 3186 | should be used when creating databases that will be uploaded to it. 3187 | Future CLI releases (numbered 2.5.x) may produce databases that are not 3188 | backwards compatible with this version of LGTM Enterprise. 3189 | 3190 | - Fixed a bug in `codeql test run` that causes tests to fail messily 3191 | if the freshly-extracted test database needed to be upgraded in 3192 | order to be compatible with the QL source under test. This would 3193 | happen more often at the _end_ of a release cycle, after updates to 3194 | the QL repository had happened. 3195 | 3196 | - `codeql github upload-results` should now work correctly against 3197 | GitHub Enterprise Server instances that are configured with a path 3198 | prefix. 3199 | 3200 | ## Release 2.4.5 (2021-03-08) 3201 | 3202 | - The bundled extractors are updated to match the versions currently 3203 | used on LGTM.com. These are newer than the last release (1.26) of 3204 | LGTM Enterprise. If you plan to upload databases to an LGTM 3205 | Enterprise 1.26 instance, you need to create them with release 3206 | 2.3.4. 3207 | 3208 | - The C/C++ extractor can now parse more Microsoft language extensions when in 3209 | C++14 and C++17 mode. 3210 | 3211 | - `codeql database analyze` now reports the name and version of each 3212 | QL pack used by the analysis. You can find this information in the 3213 | SARIF output. In particular, the `runs[0].tool.extensions` property 3214 | contains an object for each QL pack used by the analysis. Each 3215 | object contains the `name` and `semanticVersion` of the 3216 | corresponding QL pack, if such information is available. 3217 | 3218 | - `codeql github upload-results` is a new command that uploads a SARIF file 3219 | generated by CodeQL to GitHub's Code Scanning. 3220 | 3221 | ## Release 2.4.4 (2021-02-12) 3222 | 3223 | - The bundled extractors are updated to match the versions currently 3224 | used on LGTM.com. These are newer than the last release (1.26) of 3225 | LGTM Enterprise. If you plan to upload databases to an LGTM 3226 | Enterprise 1.26 instance, you need to create them with release 3227 | 2.3.4. 3228 | 3229 | ### Potentially breaking changes 3230 | 3231 | - The `name` property in `qlpack.yml` must now meet the following requirements: 3232 | - Only lowercase ASCII letters, ASCII digits, and hyphens (`-`) are allowed. 3233 | - A hyphen is not allowed as the first or last character of the name. 3234 | - The name must be at least one character long, and no longer than 128 characters. 3235 | 3236 | ### New features 3237 | 3238 | - Alert and path queries can now give a score to each alert they 3239 | produce. You can incorporate alert scores in an alert or path query 3240 | by first adding the `@scored` property to the query metadata. You 3241 | can then introduce a new numeric column at the end of the `select` 3242 | statement structure to represent the score of each alert. 3243 | Alert scores are exposed in the SARIF output of commands like 3244 | `codeql database analyze` as the `score` property in the property 3245 | bags of result objects. 3246 | 3247 | ### Bugs fixed 3248 | 3249 | - The default value of the `--working-dir` options for the 3250 | `index-files` and `trace-command` subcommands of `codeql database` 3251 | has been fixed to match the documentation; previously, it would 3252 | erroneously use the process' current working directory rather than 3253 | the database source root. 3254 | 3255 | - `codeql test run` will not crash if database extraction in a test 3256 | directory fails. Instead only the tests in that directory will be 3257 | marked as failing, and tests in other directories will continue 3258 | executing. 3259 | 3260 | ## Release 2.4.3 (2021-01-29) 3261 | 3262 | Fixes several bugs introduced in 2.4.2, related to searching the disk for 3263 | QL packs: 3264 | 3265 | - In many cases the search would scan through more of the file system 3266 | than it should. Often the only effect of this was that the scan would 3267 | take longer (sometimes significantly longer) but in some corner 3268 | cases it could lead to packs being found that _shouldn't_ be found, 3269 | which could lead to compilation failure if different versions of the same 3270 | pack exist on disk. 3271 | 3272 | - The search would terminate a fatal error if it met a directory without 3273 | read permission. 3274 | 3275 | - A `provide` entry in `.codeqlmanifest.json` that ended with `*` would 3276 | erroneously not match a `.codeqlmanifest.json` in a subdirectory. 3277 | 3278 | As a consequence of the latter fix, the semantics of 3279 | `.codeqlmanifest.json` files has changed slightly: Directory names 3280 | that start with a dot used to not be matched by the pattern elements 3281 | `*` and `**`, whereas now even dotted directories match such a pattern 3282 | element. The previous behavior was never documented, and only very few 3283 | users have `.codeqlmanifest.json` files of their own in the first 3284 | place, so this change is expected to have minimal practical effect. 3285 | 3286 | ## Release 2.4.2 (2021-01-22) 3287 | 3288 | - The bundled extractors are updated to match the versions currently 3289 | used on LGTM.com. These are newer than the last release (1.26) of 3290 | LGTM Enterprise. If you plan to upload databases to an LGTM 3291 | Enterprise 1.26 instance, you need to create them with release 3292 | 2.3.4. 3293 | 3294 | ## Release 2.4.1 (2020-12-19) 3295 | 3296 | - The bundled extractors are updated to match the versions currently 3297 | used on LGTM.com. These are newer than the last release (1.26) of 3298 | LGTM Enterprise. If you plan to upload databases to an LGTM 3299 | Enterprise 1.26 instance, you need to create them with release 3300 | 2.3.4. 3301 | 3302 | ### Features added 3303 | 3304 | - `codeql query format` now checks all files rather than stopping 3305 | after the first failure when the `--check-only` option is given. 3306 | 3307 | - `codeql resolve database` will produce a `languages` key giving the 3308 | language the database was created for. This can be useful in IDEs to 3309 | help describe the database and suggest default actions or queries. 3310 | For databases created by earlier versions, the result will be a 3311 | best-effort guess. 3312 | 3313 | - `codeql database interpret-results` can now produce Graphviz `.dot` 3314 | files from queries with `@kind graph`. 3315 | 3316 | ### Features removed 3317 | 3318 | - `codeql test run` had some special compatibility support for running 3319 | unit tests for the "code duplication" extractor features of certain 3320 | discontinued Semmle products. Those tests have since been removed 3321 | from the [public QL repository](https://github.com/github/codeql), 3322 | so the compatibility support for them has been removed. This should 3323 | not affect any external users (since the extractor feature in 3324 | question was never supported by `codeql database create` anyway), 3325 | but if you run `codeql test run` against the unit tests belonging to 3326 | an _old_ checkout of the repository, you may now see some failures 3327 | among `Metrics` tests. 3328 | 3329 | ## Release 2.3.4 (2020-12-15) 3330 | 3331 | This release corresponds to release 1.26.x of LGTM Enterprise, and 3332 | should be used when creating databases that will be uploaded to it. 3333 | Future CLI releases (numbered 2.4.x) may produce databases that are not 3334 | backwards compatible with this version of LGTM Enterprise. 3335 | 3336 | For all purposes other than creating databases for LGTM Enterprise we 3337 | recommend that you upgrade to CLI releases numbered 2.4.x or later. 3338 | 3339 | ## Release 2.4.0 (2020-11-25) 3340 | 3341 | - The bundled extractors are updated to match the versions currently 3342 | used on LGTM.com. These are newer than the last release (1.25) of 3343 | LGTM Enterprise. If you plan to upload databases to an LGTM 3344 | Enterprise 1.25 instance, you need to create them with release 3345 | 2.2.6. 3346 | 3347 | - Much of the work done by `codeql database upgrade` now happens 3348 | implicitly (and reversibly) as part of ordinary query evaluation. 3349 | This should make the need to explicitly run `codeql database 3350 | upgrade` much less common. However there are still some corner cases 3351 | that will require it, particularly for very old databases. 3352 | 3353 | - `codeql test run` with a `--threads` argument will now _compile_ 3354 | test queries in parallel even if they belong to the same single 3355 | test directory. This can speed up localized testing considerably. 3356 | 3357 | ## Release 2.3.3 (2020-11-06) 3358 | 3359 | - The bundled extractors are updated to match the versions currently 3360 | used on LGTM.com. These are newer than the last release (1.25) of 3361 | LGTM Enterprise. If you plan to upload databases to an LGTM 3362 | Enterprise 1.25 instance, you need to create them with release 3363 | 2.2.6. 3364 | 3365 | - Fixed bug in `codeql test run` where the `--format` and 3366 | `--failing-exitcode` options would not work reliably when `--ram` 3367 | was also given 3368 | 3369 | - The `$CODEQL_JAVA_HOME` environment variable will now be passed to 3370 | extractors such that extractors implemented in Java can be affected 3371 | too. Beware that this variable will override the JVM that executes 3372 | the main `codeql` process. It should not normally be set explicitly. 3373 | 3374 | ## Release 2.3.2 (2020-10-27) 3375 | 3376 | - The bundled extractors are updated to match the versions currently 3377 | used on LGTM.com. These are newer than the last release (1.25) of 3378 | LGTM Enterprise. If you plan to upload databases to an LGTM 3379 | Enterprise 1.25 instance, you need to create them with release 3380 | 2.2.6. 3381 | 3382 | ## Release 2.3.1 (2020-10-15) 3383 | 3384 | - The bundled extractors are updated to match the versions currently 3385 | used on LGTM.com. These are newer than the last release (1.25) of 3386 | LGTM Enterprise. If you plan to upload databases to an LGTM 3387 | Enterprise 1.25 instance, you need to create them with release 3388 | 2.2.6. 3389 | 3390 | ### Features added 3391 | 3392 | - `codeql database create` now accepts a `--working-dir` option, which 3393 | allows the working directory for extractor scripts to differ from 3394 | the source root. This is useful in some specialized integration 3395 | situations. 3396 | 3397 | - `codeql database create` will now pass a `--compiler-spec` option on 3398 | to `codeql database trace-command`. This allows adapting the build 3399 | tracing process when unusual compiler toolchains are used. 3400 | 3401 | - `codeql database init` accepts an `--allow-missing-source-root` 3402 | option, which is useful in some specialized integration situations. 3403 | 3404 | ## Release 2.3.0 (2020-09-28) 3405 | 3406 | - The bundled extractors are updated to match the versions currently 3407 | used on LGTM.com. These are newer than the last release (1.25) of 3408 | LGTM Enterprise. If you plan to upload databases to an LGTM 3409 | Enterprise 1.25 instance, you need to create them with release 3410 | 2.2.6. 3411 | 3412 | ### Potentially breaking changes 3413 | 3414 | - The Java extractor no longer supports builds running on a Java 6 3415 | JRE. The minimum supported version is Java 7. 3416 | 3417 | - The interpretation of binding set annotations in QL has changed 3418 | subtly. In rare cases, existing QL code that contains explicit 3419 | binding set annotations on overriding class predicates may now be 3420 | rejected with errors of the form "... is not bound to a value". You 3421 | can fix this by adding explicit binding sets to the overridden 3422 | predicate, or to the abstract class itself in the case of the 3423 | characteristic predicate. For more information about binding sets, 3424 | see [Annotations](https://help.semmle.com/QL/ql-handbook/annotations.html#binding-sets) 3425 | in the QL language reference. 3426 | 3427 | ### QL language improvements 3428 | 3429 | - You can now use binding sets on class bodies. This lets you 3430 | explicitly annotate dynamically dispatched characteristic 3431 | predicates. 3432 | 3433 | ### New features 3434 | 3435 | - Query authors can use the new subcommand `codeql generate query-help` to 3436 | validate query help files and render the files as Markdown. For more information, 3437 | see [Testing query help files](https://help.semmle.com/codeql/codeql-cli/procedures/testing-query-help-files.html). 3438 | 3439 | - The new subcommand `codeql bqrs hash` computes a stable hash of a 3440 | BQRS file. 3441 | 3442 | - `codeql query decompile` now accepts a `--kind` flag. This allows 3443 | advanced users to choose which intermediate representation to show 3444 | for a compiled QL query. `--kind dil` shows the Datalog 3445 | representation while `--kind ra` shows the relational algebra 3446 | representation used by the evaluator. 3447 | 3448 | ## Release 2.2.6 (2020-09-11) 3449 | 3450 | This release corresponds to release 1.25.x of LGTM Enterprise, and 3451 | should be used when creating databases that will be uploaded to it. 3452 | Future CLI releases (numbered 2.3.x) may produce databases that are not 3453 | backwards compatible with this version of LGTM Enterprise. 3454 | 3455 | For all purposes other than creating databases for LGTM Enterprise we 3456 | recommend that you continue upgrading to newer CLI releases as they 3457 | become available. 3458 | 3459 | ## Release 2.2.5 (2020-08-21) 3460 | 3461 | - The bundled extractors are updated to match the versions currently 3462 | used on LGTM.com. These are newer than the last release (1.24) of 3463 | LGTM Enterprise. If you plan to upload databases to an LGTM 3464 | Enterprise 1.24 instance, you need to create them with release 3465 | 2.1.4. 3466 | 3467 | - Updated license terms with a rewritten description of what is and 3468 | is not allowed. No substantive changes are intended, but the new 3469 | text is hopefully easier to understand. 3470 | 3471 | ### New features 3472 | 3473 | - The CLI can now execute queries that use QL's `external predicate` 3474 | feature. All subcommands that execute queries have a new 3475 | `--external` option to specify the value set for those predicates. 3476 | 3477 | - A new `codeql bqrs diff` command can be used to compute the 3478 | difference between two binary query result sets. 3479 | 3480 | - `codeql test run` has some new options to improve support for 3481 | testing of extractors: 3482 | - `--check-databases` which will run `codeql dataset check` on 3483 | every test database produced during a run. 3484 | - `--consistency-queries` which will run a set of additional 3485 | queries over _all_ the test databases produced during a run. 3486 | - `--show-extractor-output` 3487 | 3488 | ## Release 2.2.4 (2020-06-29) 3489 | 3490 | - The bundled extractors are updated to match the versions currently 3491 | used on LGTM.com. These are newer than the last release (1.24) of 3492 | LGTM Enterprise. If you plan to upload databases to an LGTM 3493 | Enterprise 1.24 instance, you need to create them with release 3494 | 2.1.4. 3495 | 3496 | ### Bugs fixed 3497 | 3498 | - QL packs found through the `--search-path` option, or in a sibling 3499 | directory to the unpacked CLI would erroneously take precedence over 3500 | the content of the workspace when using the CodeQL extension for 3501 | Visual Studio Code. This is now fixed such that the workspace 3502 | takes priority. 3503 | 3504 | - Two command-line options that control the amount of disk space that 3505 | the QL evaluator will try to keep free of disk cache are now called 3506 | `--min-disk-free` and `--min-disk-free-pct`. Previously they were 3507 | called `--max-disk-free` instead, which made no sense. The old names 3508 | are still recognized such as not to break existing scripts, but are 3509 | now undocumented and deprecated. 3510 | 3511 | ## Release 2.2.3 (2020-06-15) 3512 | 3513 | CodeQL CLI 2.2.3 is the same as version 2.2.2, but re-released with a new 3514 | version number because the `v2.2.2` folder on the download site 3515 | originally contained the 2.2.0 binaries instead of the correct 2.2.2 3516 | ones. 3517 | 3518 | If you have downloaded release 2.2.2, and `codeql --version` correctly 3519 | identifies itself as being that version, you don't need to upgrade to 3520 | 2.2.3. 3521 | 3522 | ## Release 2.2.2 (2020-06-12) 3523 | 3524 | - The bundled extractors are updated to match the versions currently 3525 | used on LGTM.com. These are newer than the last release (1.24) of 3526 | LGTM Enterprise. If you plan to upload databases to an LGTM 3527 | Enterprise 1.24 instance, you need to create them with release 3528 | 2.1.4. 3529 | 3530 | ### Improvements 3531 | 3532 | - Query evaluations that time out due to a `--timeout` option are no 3533 | longer silently discarded. Instead `codeql` will terminate with exit 3534 | code 33. Commands that evaluate multiple queries will produce as 3535 | much output as they can even if one of the queries times out. 3536 | 3537 | ## Release 2.2.1 3538 | 3539 | There is no CodeQL CLI version 2.2.1. This version number was used 3540 | internally to work around restrictions in the CodeQL for VS Code 3541 | extension. 3542 | 3543 | ## Release 2.2.0 (2020-05-29) 3544 | 3545 | - The bundled extractors are updated to match the versions currently 3546 | used on LGTM.com. These are newer than the last release (1.24) of 3547 | LGTM Enterprise. If you plan to upload databases to an LGTM 3548 | Enterprise 1.24 instance, you need to create them with release 3549 | 2.1.4. 3550 | - Starting with this release, the CodeQL CLI can be downloaded either 3551 | as a single `codeql.zip` file containing the CLI for all supported 3552 | platforms, or as a `codeql-PLATFORM.zip` that contains the files for 3553 | just one platform. The single-platform zips are faster to download. 3554 | 3555 | ### QL language improvement 3556 | 3557 | - QL now supports the definition of new types as type unions. This 3558 | feature currently allows unions of branches from an already existing 3559 | algebraic data type and unions of database types. 3560 | 3561 | ## Release 2.1.4 (2020-05-26) 3562 | 3563 | This release corresponds to release 1.24.x of LGTM Enterprise, and 3564 | should be used when creating databases that will be uploaded to it. 3565 | Future CLI releases (numbered 2.2.x) may produce databases that are not 3566 | backwards compatible with this version of LGTM Enterprise. 3567 | 3568 | For all purposes other than creating databases for LGTM Enterprise we 3569 | recommend that you continue upgrading to newer CLI releases as they 3570 | become available. 3571 | 3572 | ### Features added 3573 | 3574 | - A new `codeql query format` command exposes the QL autoformatter for 3575 | use on the command line. 3576 | 3577 | ### Bugs fixed 3578 | 3579 | - `-J` command-line options that contain spaces now ought to work on 3580 | Windows. They still do not work reliably on Linux or MacOS, though. 3581 | 3582 | ## Release 2.1.3 (2020-05-13) 3583 | 3584 | ### Bugs fixed 3585 | 3586 | - Fixes a bug in `codeql execute cli-server` (a helper used by the VS 3587 | Code extension) which would sometimes cause query compilation to 3588 | fail until the extension was restarted. 3589 | - Fixes a bug in `codeql database upgrade` which could lead to 3590 | performance losses if the upgraded database was subsequently used 3591 | with LGTM or the legacy Semmle Core product. 3592 | - Fixes a bug in the QL evaluator that would sometimes lead to crashes 3593 | for queries that use the new `unique` aggregate added in release 3594 | 2.1.0. 3595 | - The value of the `--compilation-cache-size` option is now correctly 3596 | interpreted as a number of megabytes rather than a number of bytes. 3597 | 3598 | ## Release 2.1.2 (2020-05-06) 3599 | 3600 | - Updated license terms to allow CI use with GitHub Actions for 3601 | open-source software. 3602 | 3603 | ### Potentially breaking changes 3604 | 3605 | - In [query suite definitions](https://help.semmle.com/codeql/codeql-cli/procedures/query-suites.html), filter 3606 | instructions that filter on the `query path` pseudo-tag will now 3607 | always see the relative path to the query expressed with `/` as a 3608 | directory separator, independently of the platform. Previously they 3609 | erroneously used the platform's directory separator, meaning that 3610 | query suites developed on Windows would not work correctly on Unix 3611 | systems (and vice versa) if they used `query path`. Existing suite 3612 | definitions developed on Windows may need to be updated to match the 3613 | new behavior. 3614 | 3615 | ### Features added 3616 | 3617 | - A new `codeql test accept` subcommand helps automate updating the 3618 | expected output for unit tests after a desired change in query 3619 | behavior. This can also be done by the new `--learn` option for 3620 | `codeql test run`. 3621 | 3622 | ### Bugs fixed 3623 | 3624 | - `codeql database create` will now report an explicit error if given 3625 | a `--command` argument that specifies an empty string. Previously 3626 | this would be accepted initially, leading to confusing failures 3627 | later. 3628 | 3629 | ## Release 2.1.1 (2020-04-20) 3630 | 3631 | - The bundled extractors are updated to match the versions currently 3632 | used on LGTM.com. 3633 | 3634 | ### Features added 3635 | 3636 | - `codeql resolve queries` accepts a `--format=bylanguage` option. 3637 | This is used to help automated workflows determine which languages 3638 | to create databases for, from the queries that are available to run. 3639 | - It is now possible to attempt to execute `.ql` files that are not in 3640 | a QL pack. This is used by a few specialized internal workflows. 3641 | However, standalone queries cannot import any of the dependencies 3642 | that you would usually declare in a `qlpack.yml` file, so will not 3643 | be useful in most cases. 3644 | 3645 | ## Release 2.1.0 (2020-03-27) 3646 | 3647 | - The bundled extractors are updated to match the versions currently 3648 | used on LGTM.com. These are newer than the last release (1.23) of 3649 | LGTM Enterprise. If you plan to upload databases to an LGTM 3650 | Enterprise 1.23 instance, you need to create them with release 3651 | 2.0.1. For more information, see [Preparing CodeQL databases to 3652 | upload to 3653 | LGTM](https://help.semmle.com/lgtm-enterprise/admin/help/prepare-database-upload.html) 3654 | in the LGTM admin help. 3655 | 3656 | ### Potentially breaking changes 3657 | 3658 | - If you pass a directory name as a command-line argument to 3659 | `codeql test run`, it will now consider all `.ql` or `.qlref` files 3660 | found under that directory to be test queries, even if they have no 3661 | accompanying `.expected` file. Tests that lack an `.expected` file 3662 | will fail, but will generate an `.actual` file that you can rename 3663 | to `.expected` if you want to use the results. 3664 | 3665 | The goal of this change is to support existing workflows of 3666 | experienced CodeQL users, and also to provide clear error 3667 | indications if an `.expected` file is accidentally lost, renamed, or 3668 | misspelled. 3669 | 3670 | However, if you invoke `codeql test run` on a directory tree that 3671 | contains both tests and non-test queries, you will now encounter 3672 | errors if any of the `.ql` files can't be processed as test queries. 3673 | If you're affected by this change, you can suppress these errors by: 3674 | 3675 | - Adding a `tests` property to this QL pack to define specify 3676 | which directories contain only test queries and associated test 3677 | code. For more information, see 3678 | [About QL packs](https://help.semmle.com/codeql/codeql-cli/reference/qlpack-overview.html). 3679 | - Running `codeql test run` with a new `--strict-test-discovery` 3680 | option. 3681 | 3682 | In the longer term, we recommend that you reorganize the queries so 3683 | that test queries are stored in a directory tree that's separate 3684 | from actual queries. 3685 | 3686 | - `codeql database create` and `codeql database finalize` will no 3687 | longer recognize a `--no-duplicate-code` option. This option has 3688 | never had any effect, and its positive variant `--duplicate-code` 3689 | previously led to a fatal error. 3690 | 3691 | ### Features added 3692 | 3693 | - A new XML extractor is included. It is not intended to be used as a 3694 | stand-alone extractor, but rather to augment the data produced by 3695 | other extractors. In particular, the C\# and Java extractors invoke 3696 | it during database creation to include information relevant to the 3697 | analysis of those languages, much like LGTM.com does. 3698 | - Two new plumbing commands `codeql database index-files` and 3699 | `codeql resolve files` have been added for support of invoking the 3700 | XML extractor support. These commands are generally only of interest 3701 | for extractor authors. 3702 | - Two new plumbing commands have been added to `codeql dataset`. The 3703 | `measure` subcommand can be used to collect size information from a 3704 | dataset, and the `check` subcommand can scan a dataset for database 3705 | inconsistencies. These commands are useful when developing a new 3706 | CodeQL extractor. 3707 | - The QL evaluator contains a number of features in support of an 3708 | internal experiment with using machine-learning techniques to 3709 | identify functions in unknown codebases as sources or sinks of 3710 | taint. This includes new command-line options `--ml-model-path` and 3711 | `--native-library-path` to several subcommands. As the new features 3712 | are not yet ready for general use, these new options should be 3713 | ignored by external CodeQL users. 3714 | 3715 | ### Bugs fixed 3716 | 3717 | - Fixes a bug that could result in empty databases for C/C++. 3718 | Previously, extraction would mistakenly be skipped for source files 3719 | compiled with the Clang compiler, if the `-fintegrated-cc1` option 3720 | was specified. 3721 | - `codeql database create` and `codeql database init` will now, as 3722 | they have always been documented, refuse to create a database whose 3723 | parent directory doesn't already exist. 3724 | - `codeql test run` will no longer leave `.actual` files from previous 3725 | runs in the file system after a test passes. 3726 | 3727 | ### QL language improvements 3728 | 3729 | - QL now supports set literals, and the QL extractor can identify them 3730 | with the `SetLiteral` class. For more information, see [Set literal 3731 | expressions](https://help.semmle.com/QL/ql-handbook/expressions.html#set-literal-expressions) 3732 | in the QL language reference. 3733 | - QL now supports a uniqueness aggregate. This can express constraints 3734 | that there is precisely one value. The syntax is taken from previous 3735 | aggregates such as `min` and `max`. 3736 | 3737 | ``` {.sourceCode .ql} 3738 | unique(int x | x = 4 or x = 2 * 2 | x) 3739 | ``` 3740 | 3741 | ## Release 2.0.6 (2020-03-16) 3742 | 3743 | ### Bugs fixed 3744 | 3745 | - Fixes a problem preventing `codeql database create` from working 3746 | with Python 3 on macOS. 3747 | - Fixes a problem preventing `codeql database create` from finding 3748 | locally installed Python packages. 3749 | 3750 | ## Release 2.0.5 (2020-03-13) 3751 | 3752 | - The bundled extractors (which are responsible for converting source 3753 | code to databases for each supported language) are updated to match 3754 | the versions currently used on LGTM.com. These are newer than the 3755 | last release of LGTM Enterprise, so this release should not be used 3756 | if you plan to upload databases to an LGTM Enterprise instance. For 3757 | more information, see [Preparing CodeQL databases to upload to 3758 | LGTM](https://help.semmle.com/lgtm-enterprise/admin/help/prepare-database-upload.html) 3759 | in the LGTM admin help. 3760 | 3761 | ### Features added 3762 | 3763 | - `codeql test run` has a new `--slice` option that can be used to 3764 | parallelize tests over more machines. 3765 | 3766 | ## Release 2.0.4 (2020-02-21) 3767 | 3768 | - The bundled extractors (which are responsible for converting source 3769 | code to databases for each supported language) are updated to match 3770 | the versions currently used on LGTM.com. These are newer than the 3771 | last release of LGTM Enterprise, so this release should not be used 3772 | if you plan to upload databases to an LGTM Enterprise instance. For 3773 | more information, see [Preparing CodeQL databases to upload to 3774 | LGTM](https://help.semmle.com/lgtm-enterprise/admin/help/prepare-database-upload.html) 3775 | in the LGTM admin help. 3776 | 3777 | ### Features added 3778 | 3779 | - Subcommands that execute queries (such as `codeql database analyze`) 3780 | now have a `--timeout` option that can be used to set a timeout to 3781 | automatically cancel query evaluations that appear to diverge. 3782 | - A new plumbing command `codeql query decompile` can display the DIL 3783 | intermediate representations that is included in the output of 3784 | `codeql query compile --dump-qlo --include-dil-in-qlo`. This is 3785 | useful mainly for certain internal workflows; the information 3786 | produced is the same as what `codeql query compile --dump-dil` 3787 | already outputs. 3788 | 3789 | ### Bugs fixed 3790 | 3791 | - The `--debug` and `--tuple-counting` options to 3792 | `codeql test run` erroneously had no effect. Now they ought to work. 3793 | 3794 | ## Release 2.0.3 (2020-02-12) 3795 | 3796 | ### Bugs fixed 3797 | 3798 | - Fixes a bug where `codeql test run` would fail with the 3799 | message 3800 | `CatastrophicError: There should be a --library-path option for com.semmle.cli2.LibraryPathOptions.libraryPath but we didn't find it` 3801 | when running tests against the `master` branch of the CodeQL 3802 | libraries for certain languages. 3803 | - Otherwise identical to release 2.0.2. 3804 | 3805 | ## Release 2.0.2 (2020-02-05) 3806 | 3807 | - The bundled extractors (which are responsible for converting source 3808 | code to databases for each supported language) are updated to match 3809 | the versions currently used on LGTM.com. These are newer than the 3810 | last release of LGTM Enterprise, so this release should not be used 3811 | if you plan to upload databases to an LGTM Enterprise instance. For 3812 | more information, see [Preparing CodeQL databases to upload to 3813 | LGTM](https://help.semmle.com/lgtm-enterprise/admin/help/prepare-database-upload.html) 3814 | in the LGTM admin help. 3815 | - The parent and sibling directories of the unpacked CLI are no longer 3816 | searched recursively for QL packs. QL packs will only be found if 3817 | there's a `qlpack.yml` or `.codeqlmanifest.json` directly in a 3818 | parent or sibling directory. This should eliminate the very long 3819 | disk-scanning delays experienced by users who unpacked earlier 3820 | versions of the CLI in their home directory. 3821 | - Parent and sibling directories of the unpacked CLI will now be 3822 | searched for QL packs as a last resort, even if you give an explicit 3823 | `--search-path` option. This means, for example, that you can define 3824 | a search path in the [per-user configuration file](https://help.semmle.com/codeql/codeql-cli/reference/configuration-overview.html) without it depending on 3825 | where the CLI is unpacked. In particular, the setting can now be 3826 | meaningfully used by users who let the CodeQL for VS Code extension 3827 | manage the downloading and unpacking of the CLI. 3828 | 3829 | ### Security updates 3830 | 3831 | - The `codeql database create` command and its relatives will no 3832 | longer attempt to find extractors located in the parent and sibling 3833 | directories of the unpacked CLI. This closes a security risk for 3834 | users who unpacked the CodeQL CLI in their home directory. This 3835 | could've resulted in arbitrary code execution if the user unpacked a 3836 | file archive containing a malicious extractor anywhere in the home 3837 | directory. Extractors will now only be found within the unpacked CLI 3838 | itself, or in directories explicitly listed in the `--search-path`. 3839 | It is expected that users will only point `--search-path` to 3840 | locations they trust at least as much as the CLI download itself. 3841 | 3842 | ### Features added 3843 | 3844 | - This release supports executing query regression tests using the 3845 | `codeql test` command. For further information, see 3846 | [Testing custom queries](https://help.semmle.com/codeql/codeql-cli/procedures/test-queries.html). 3847 | - The error message if you try executing a query against a database 3848 | that needs to be upgraded (which can happen routinely if you're 3849 | using a fresh `master` checkout of the CodeQL libraries with the 3850 | bundled extractors) will now explicitly suggest a 3851 | `codeql database update` command to run. The database is not 3852 | automatically upgraded, as this may make it irreversibly 3853 | incompatible with older versions of the CodeQL libraries. This 3854 | allows users who want to compare behavior of different versions of 3855 | the libraries against the same database to make a copy before they 3856 | upgrade it. 3857 | 3858 | ## Release 2.0.1 (2019-12-17) 3859 | 3860 | - Corresponds to LGTM Enterprise release 1.23. 3861 | - The bundled extractors (which are responsible for converting source 3862 | code to databases for each supported language) are updated to match 3863 | the extractor versions used in LGTM Enterprise. 3864 | - No other changes to the core CLI. 3865 | 3866 | ## Release 2.0.0 (2019-11-14) 3867 | 3868 | - First public release. 3869 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | GitHub CodeQL Terms and Conditions 2 | ================================== 3 | These GitHub CodeQL Terms and Conditions ("**Terms**") are a legal 4 | agreement between you (either as an individual or on behalf of an 5 | entity) and GitHub, Inc. regarding your use of the GitHub CodeQL 6 | software and associated documentation (collectively, the 7 | "**Software**"). By using the Software, you accept these Terms. 8 | **Please read all of these Terms;** in many cases, provisions set 9 | forth later in the Terms limit and qualify provisions set forth 10 | earlier in the Terms. If you do not accept these Terms, do not 11 | download, install, use, or copy the Software. 12 | 13 | Definitions 14 | ----------- 15 | 16 | In these Terms: 17 | 18 | * "OSI-approved License" means an Open Source Initiative 19 | (OSI)-approved open source software license. 20 | 21 | * "Open Source Codebase" means a codebase that is released under an 22 | OSI-approved License. 23 | 24 | Use Rights; Scope of License 25 | ---------------------------- 26 | 27 | The Software is licensed on a per user basis. Here's what you may do 28 | with the Software, but subject to License Restrictions provisions 29 | below: 30 | 31 | * Use the Software to perform academic research. 32 | 33 | * Use the Software to demonstrate the Software. 34 | 35 | * Test CodeQL queries that are released under an OSI-approved 36 | Licence to confirm that new versions of those queries continue to 37 | find the right vulnerabilities. 38 | 39 | Here's what you may also do with the Software, but only with an Open 40 | Source Codebase and subject to the License Restrictions provisions 41 | below: 42 | 43 | * Perform analysis on the Open Source Codebase. 44 | 45 | * If the Open Source Codebase is hosted and maintained on 46 | GitHub.com, generate CodeQL databases for or during automated 47 | analysis, CI, or CD. 48 | 49 | License Restrictions 50 | -------------------- 51 | 52 | These Terms do not authorize, and the Software may not be used for any 53 | purpose not expressly set forth above, including: 54 | 55 | * To otherwise or in any other context generate any CodeQL database 56 | for or during automated analysis, CI or CD, whether as part of 57 | normal engineering processes or another context. 58 | 59 | * To otherwise or in any other context use the Software in 60 | connection with any codebase that is not an Open Source Codebase 61 | (e.g., code in a private repo in GitHub). 62 | 63 | _**Please note:** if your use of the Software is under a paid customer 64 | license for GitHub Advanced Security, the restrictions with respect to 65 | automated analysis, CI, and CD and use in connection with non-Open 66 | Source Codebases do not apply._ 67 | 68 | At all times, except (and only to the extent) permitted by applicable 69 | law or applicable third-party license, you will not (and have no right 70 | to): 71 | 72 | * work around any technical limitations in the Software that only 73 | allow you to use it in certain ways; 74 | 75 | * reverse engineer, decompile or disassemble the Software; 76 | 77 | * remove, minimize, block, or modify any notices of GitHub or its 78 | suppliers in the Software; 79 | 80 | * use the Software in any way that is against the law; or 81 | 82 | * share, publish, distribute or lend the Software, provide or make 83 | available the Software as a hosted solution (whether on a 84 | standalone basis or combined, incorporated or integrated with 85 | other software or services) for others to use, or transfer the 86 | Software or these Terms to any third party. 87 | 88 | The Software is licensed, not sold. GitHub reserves all rights not 89 | expressly granted in these Terms. 90 | 91 | Open Source Software 92 | -------------------- 93 | 94 | The Software may include components licensed under open source 95 | software licenses. Any such licenses are included in the "Open Source 96 | Notices" documentation that is included with the Software. Such 97 | documentation also includes copies of all applicable open source 98 | licenses. 99 | 100 | To the extent the terms of the licenses applicable to open source 101 | components require GitHub to make an offer to provide source code in 102 | connection with the Software, such offer is hereby made, and you may 103 | exercise it by contacting GitHub: https://github.com/contact. 104 | 105 | Unless otherwise agreed to in writing with GitHub, your agreement with 106 | GitHub will always include, at a minimum, these Terms. Open source 107 | software licenses for the Software's source code constitute separate 108 | written agreements. To the limited extent that any open source 109 | software license expressly supersedes these Terms, such open source 110 | license governs your use of the applicable component(s) of the 111 | Software subject to such license. 112 | 113 | GitHub Trademarks 114 | ----------------- 115 | 116 | These Terms do not grant any right or license to use any of GitHub's 117 | trademarks or logos, including, without limitation, the names GitHub 118 | and CodeQL and any Software logo designs in the "logos" folder of the 119 | Software. You agree not to display or use any of these trademarks or 120 | logos in any manner without GitHub's prior written permission, except 121 | as allowed by GitHub's Logos and Usage Policy located at 122 | https://github.com/logos. GitHub reserves all right, title and 123 | interest in and to all GitHub trademarks and logos. 124 | 125 | Additional Services 126 | ------------------- 127 | 128 | Auto-Updates: The Software may include an auto-update service. If the 129 | Software automatically enables such service (or, if it is not 130 | automatically enabled and you choose to use it), GitHub will 131 | automatically update the Software when a new version is available. 132 | 133 | Support 134 | ------- 135 | 136 | Because the Software is "as-is," GitHub may not provide support for it. 137 | 138 | Export Control 139 | -------------- 140 | 141 | Customer will comply with all applicable export and import laws and 142 | regulations that apply to the Software. 143 | 144 | Disclaimer; Limitations of Liability 145 | ------------------------------------ 146 | 147 | THE SOFTWARE, INCLUDING ANY ADDITIONAL SERVICES, IS PROVIDED ON AN 148 | "AS-IS" BASIS, AND GITHUB GIVES NO EXPRESS WARRANTIES, GUARANTEES OR 149 | CONDITIONS. TO THE EXTENT PERMITTED BY APPLICABLE LAW, GITHUB 150 | DISCLAIMS THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 151 | PARTICULAR PURPOSE AND NON-INFRINGEMENT. YOUR USE OF THE SOFTWARE IS 152 | AT YOUR SOLE RISK. 153 | 154 | TO THE EXTENT PERMITTED BY APPLICABLE LAW, YOU EXPRESSLY UNDERSTAND 155 | AND AGREE THAT (1) YOU CAN RECOVER DIRECT DAMAGES RELATING TO THE 156 | SOFTWARE, INCLUDING ANY ADDITIONAL SERVICES, UP TO U.S. $5.00 FROM 157 | GITHUB AND ITS SUPPLIERS, AND (2) GITHUB WILL NOT BE LIABLE FOR ANY 158 | INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL OR EXEMPLARY DAMAGES, 159 | INCLUDING, WITHOUT LIMITATION, ANY DAMAGES FOR LOSS OF PROFITS, 160 | GOODWILL, USE, OR DATA OR OTHER INTANGIBLE LOSSES (EVEN IF GITHUB HAS 161 | BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES) RELATING TO THE 162 | SOFTWARE, INCLUDING ANY ADDITIONAL SERVICES. 163 | 164 | Miscellaneous 165 | ------------- 166 | 167 | _No Waiver._ The failure of GitHub to exercise or enforce any right or 168 | provision of these Terms will not constitute a waiver of such right or 169 | provision. 170 | 171 | _Entire Agreement._ These Terms, together with any open source 172 | software licenses referenced above, constitutes the entire agreement 173 | between you and GitHub regarding your use of the Software, superseding 174 | any prior agreements between you and GitHub (including, but not 175 | limited to, any prior versions of these Terms) regarding such use. 176 | 177 | _Governing Law._ You agree that these Terms and your use of the 178 | Software are governed by the laws of the State of California and any 179 | dispute relating to the Software or your use thereof must be brought 180 | in a tribunal of competent jurisdiction located in or near San 181 | Francisco, California. 182 | 183 | _Modifications._ These Terms may only be modified by a written 184 | amendment signed by an authorized representative of GitHub, or by the 185 | posting by GitHub of a revised version. 186 | 187 | _Contact Us._ Questions about these Terms? Contact us at 188 | https://support.github.com/contact. 189 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # CodeQL CLI 2 | 3 | This repo holds binaries for the CodeQL CLI. 4 | 5 | [CodeQL overview](https://securitylab.github.com/tools/codeql/) | [CodeQL CLI Documentation](https://codeql.github.com/docs/codeql-cli/) 6 | 7 | # Getting started 8 | 9 | 1. Go to the [Releases page](https://github.com/github/codeql-cli-binaries/releases). 10 | 2. Find the latest release, select Assets, and download the zip file containing the CLI. 11 | 3. You'll also want to clone https://github.com/github/codeql to get the CodeQL queries and 12 | libraries. Please take note of the 13 | [set-up instructions](https://codeql.github.com/docs/codeql-cli/getting-started-with-the-codeql-cli/) 14 | for placing it in a location where the CLI can find it. 15 | 4. Read the rest of the [CodeQL CLI documentation](https://codeql.github.com/docs/codeql-cli/). 16 | 17 | # Found a bug or have a question? 18 | 19 | Please raise an issue in the [github/codeql](https://github.com/github/codeql/issues/new/choose) repository. 20 | 21 | # License 22 | 23 | By downloading, you agree to the [GitHub CodeQL Terms & Conditions](https://securitylab.github.com/tools/codeql/license/). 24 | 25 | GitHub CodeQL can only be used on codebases that are released under an OSI-approved open source license, or to perform academic research. It can't be used to generate CodeQL databases for or during automated analysis, continuous integration or continuous delivery, whether as part of normal software engineering processes or otherwise. For these uses, [contact the sales team](https://enterprise.github.com/contact). 26 | --------------------------------------------------------------------------------