├── .github ├── cloudformation │ ├── README.md │ ├── oidc.yaml │ └── token.yaml ├── dependabot.yml └── workflows │ ├── makefile-test.yaml │ ├── publish.yaml │ ├── release-brew.yaml │ ├── release-pypi.yaml │ ├── release.yaml │ └── summarize-test.yaml ├── .gitignore ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── Makefile ├── NOTICE ├── README.md ├── docs ├── .gitignore ├── Makefile ├── README.md ├── book.toml └── src │ ├── README.md │ ├── SUMMARY.md │ ├── contributing │ └── README.md │ ├── installation │ └── README.md │ ├── reference-manual │ ├── README.md │ ├── cbmc-starter-kit-migrate-license.md │ ├── cbmc-starter-kit-setup-ci.md │ ├── cbmc-starter-kit-setup-proof.md │ ├── cbmc-starter-kit-setup.md │ └── cbmc-starter-kit-update.md │ ├── resources │ └── README.md │ └── tutorial │ ├── FreeRTOSConfig.h │ ├── Makefile │ ├── README.md │ ├── pvPortMalloc_harness.c │ ├── pvPortMalloc_harness1.c │ └── report │ └── html │ └── index.html ├── pyproject.toml ├── setup.cfg ├── src └── cbmc_starter_kit │ ├── Makefile │ ├── __init__.py │ ├── arguments.py │ ├── ctagst.py │ ├── etc │ └── bash_completion.d │ │ └── cbmc-starter-kit.sh │ ├── migrate_license.py │ ├── repository.py │ ├── setup.py │ ├── setup_ci.py │ ├── setup_proof.py │ ├── template-for-ci-workflow │ ├── proof_ci.yaml │ └── proof_ci_resources │ │ └── config.yaml │ ├── template-for-proof │ ├── FUNCTION_harness.c │ ├── Makefile │ ├── README.md │ ├── cbmc-proof.txt │ └── cbmc-viewer.json │ ├── template-for-repository │ ├── .gitignore │ ├── include │ │ └── README.md │ ├── negative_tests │ │ ├── README.md │ │ ├── assert │ │ │ ├── Makefile │ │ │ └── assert_harness.c │ │ ├── bounds_check │ │ │ ├── Makefile │ │ │ └── bounds_check_harness.c │ │ ├── conversion_check │ │ │ ├── Makefile │ │ │ └── conversion_check_harness.c │ │ ├── div_by_zero_check │ │ │ ├── Makefile │ │ │ └── div_by_zero_check_harness.c │ │ ├── float_overflow_check │ │ │ ├── Makefile │ │ │ └── float_overflow_check_harness.c │ │ ├── float_underflow_check │ │ │ ├── Makefile │ │ │ └── float_underflow_check_harness.c │ │ ├── nan_check │ │ │ ├── Makefile │ │ │ └── nan_check_harness.c │ │ ├── pointer_check │ │ │ ├── Makefile │ │ │ └── pointer_check_harness.c │ │ ├── pointer_overflow_check │ │ │ ├── Makefile │ │ │ └── pointer_overflow_check_harness.c │ │ ├── pointer_primitive_check │ │ │ ├── Makefile │ │ │ └── pointer_primitive_check_harness.c │ │ ├── pointer_underflow_check │ │ │ ├── Makefile │ │ │ └── pointer_underflow_check_harness.c │ │ ├── signed_overflow_check │ │ │ ├── Makefile │ │ │ └── signed_overflow_check_harness.c │ │ ├── signed_underflow_check │ │ │ ├── Makefile │ │ │ └── signed_underflow_check_harness.c │ │ ├── undefined_shift_check │ │ │ ├── Makefile │ │ │ └── undefined_shift_check_harness.c │ │ ├── unsigned_overflow_check │ │ │ ├── Makefile │ │ │ └── unsigned_overflow_check_harness.c │ │ └── unsigned_underflow_check │ │ │ ├── Makefile │ │ │ └── unsigned_underflow_check_harness.c │ ├── proofs │ │ ├── Makefile-project-defines │ │ ├── Makefile-project-targets │ │ ├── Makefile-project-testing │ │ ├── Makefile.common │ │ ├── README.md │ │ ├── lib │ │ │ ├── __init__.py │ │ │ ├── print_tool_versions.py │ │ │ └── summarize.py │ │ └── run-cbmc-proofs.py │ ├── sources │ │ └── README.md │ └── stubs │ │ └── README.md │ ├── update.py │ ├── util.py │ └── version.py ├── test ├── repo │ └── Makefile └── summarize_test │ ├── sample_run.json │ └── summarize_test.py └── training-material ├── CODING-FOR-VERIFICATION.md ├── DEBUG-CBMC.md ├── FAQ.md ├── PLANNING.md ├── PROOF-WRITING.md ├── README.md └── checklist.md /.github/cloudformation/README.md: -------------------------------------------------------------------------------- 1 | These are 2 | [AWS CloudFormation templates](https://aws.amazon.com/cloudformation/resources/templates/) 3 | for maintaining pypi and homebrew credentials used to publish the CBMC starter kit. 4 | -------------------------------------------------------------------------------- /.github/cloudformation/oidc.yaml: -------------------------------------------------------------------------------- 1 | Description: 2 | Register the GitHub identity provider with the AWS security token service. 3 | 4 | Resources: 5 | GithubIdentityProvider: 6 | Type: AWS::IAM::OIDCProvider 7 | Properties: 8 | Url: 9 | # The GitHub identity provider supporting OIDC 10 | https://token.actions.githubusercontent.com 11 | ThumbprintList: 12 | # The GitHub certification authority (the signature of its certificate) 13 | - 6938fd4d98bab03faadb97b34396831e3780aea1 14 | ClientIdList: 15 | # The AWS security token service 16 | - sts.amazonaws.com 17 | 18 | 19 | Outputs: 20 | GithubIdentityProvider: 21 | Value: !Ref GithubIdentityProvider 22 | Export: 23 | Name: GithubIdentityProvider 24 | -------------------------------------------------------------------------------- /.github/cloudformation/token.yaml: -------------------------------------------------------------------------------- 1 | Description: > 2 | Enable storage of access tokens in AWS Secrets Manager and access to the PAT 3 | from the GitHub workflows in model-checking/cbmc-starter-kit. 4 | 5 | Parameters: 6 | GithubRepoOrganization: 7 | Type: String 8 | Description: GitHub organization for the CBMC starter kit 9 | Default: model-checking 10 | CbmcStarterKitRepoName: 11 | Type: String 12 | Description: GitHub repository for CBMC starter kit 13 | Default: cbmc-starter-kit 14 | CbmcStarterKitPublicationTag: 15 | Type: String 16 | Description: GitHub tag for CBMC starter kit triggering the GitHub publication workflow 17 | Default: starterkit-* 18 | 19 | Resources: 20 | 21 | BrewBotEmail: 22 | Type: AWS::SecretsManager::Secret 23 | Properties: 24 | Name: BOT_EMAIL 25 | Description: > 26 | The email address to use with Homebrew commits. 27 | 28 | BrewToken: 29 | Type: AWS::SecretsManager::Secret 30 | Properties: 31 | Name: RELEASE_CI_ACCESS_TOKEN 32 | Description: > 33 | GitHub access token. 34 | 35 | PypiToken: 36 | Type: AWS::SecretsManager::Secret 37 | Properties: 38 | Name: PYPI_ACCESS_TOKEN 39 | Description: > 40 | Pypi access token. 41 | 42 | PublisherTokenReader: 43 | Type: AWS::IAM::Role 44 | Properties: 45 | RoleName: PublisherTokenReader 46 | Description: > 47 | This role can retrieve the personal access token for the model 48 | checking publisher in the Microsoft Marketplace. 49 | 50 | AssumeRolePolicyDocument: 51 | Version: "2012-10-17" 52 | Statement: 53 | - Effect: Allow 54 | Principal: 55 | Federated: !ImportValue GithubIdentityProvider 56 | Action: sts:AssumeRoleWithWebIdentity 57 | Condition: 58 | StringEquals: 59 | token.actions.githubusercontent.com:aud: sts.amazonaws.com 60 | StringLike: 61 | token.actions.githubusercontent.com:sub: 62 | !Sub repo:${GithubRepoOrganization}/${CbmcStarterKitRepoName}:ref:refs/tags/${CbmcStarterKitPublicationTag} 63 | 64 | Policies: 65 | - PolicyName: PublisherTokenAccess 66 | PolicyDocument: 67 | Version: "2012-10-17" 68 | Statement: 69 | - Effect: Allow 70 | Action: secretsmanager:GetSecretValue 71 | Resource: !Ref BrewBotEmail 72 | - Effect: Allow 73 | Action: secretsmanager:GetSecretValue 74 | Resource: !Ref BrewToken 75 | - Effect: Allow 76 | Action: secretsmanager:GetSecretValue 77 | Resource: !Ref PypiToken 78 | -------------------------------------------------------------------------------- /.github/dependabot.yml: -------------------------------------------------------------------------------- 1 | # https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file 2 | 3 | version: 2 4 | updates: 5 | - package-ecosystem: "github-actions" 6 | directory: "/" 7 | schedule: 8 | interval: "monthly" 9 | -------------------------------------------------------------------------------- /.github/workflows/makefile-test.yaml: -------------------------------------------------------------------------------- 1 | name: Test CBMC starter-kit by using coreHTTP 2 | on: 3 | pull_request: 4 | types: [opened, synchronize, reopened, labeled, unlabeled] 5 | push: 6 | branches: [ master ] 7 | 8 | jobs: 9 | run-tests: 10 | if: "!contains(github.event.pull_request.labels.*.name, 'no-test')" 11 | runs-on: ubuntu-latest 12 | strategy: 13 | matrix: 14 | python-version: [ '3.8', '3.9', '3.10' ] 15 | name: Python ${{ matrix.python-version }} 16 | steps: 17 | - uses: actions/checkout@v4 18 | - name: Set up Python 19 | uses: actions/setup-python@v5 20 | with: 21 | python-version: ${{ matrix.python-version }} 22 | - name: Install latest CBMC 23 | id: install_latest_cbmc 24 | shell: bash 25 | run: | 26 | # Search within 5 most recent releases for latest available package 27 | CBMC_REL="https://api.github.com/repos/diffblue/cbmc/releases?page=1&per_page=5" 28 | CBMC_DEB=$(curl -s $CBMC_REL | jq -r '.[].assets[].browser_download_url' | grep -e 'ubuntu-20.04' | head -n 1) 29 | CBMC_ARTIFACT_NAME=$(basename $CBMC_DEB) 30 | curl -o $CBMC_ARTIFACT_NAME -L $CBMC_DEB 31 | sudo dpkg -i $CBMC_ARTIFACT_NAME 32 | rm ./$CBMC_ARTIFACT_NAME 33 | - name: Install latest Litani 34 | id: install_latest_litani 35 | shell: bash 36 | run: | 37 | # Search within 5 most recent releases for latest available package 38 | LITANI_REL="https://api.github.com/repos/awslabs/aws-build-accumulator/releases?page=1&per_page=5" 39 | LITANI_DEB=$(curl -s $LITANI_REL | jq -r '.[].assets[0].browser_download_url' | head -n 1) 40 | DBN_PKG_FILENAME=$(basename $LITANI_DEB) 41 | curl -L $LITANI_DEB -o $DBN_PKG_FILENAME 42 | sudo apt-get update 43 | sudo apt-get install --no-install-recommends --yes ./$DBN_PKG_FILENAME 44 | rm ./$DBN_PKG_FILENAME 45 | - name: Install dependencies 46 | run: python3 -m pip install pyyaml jinja2 cbmc-viewer 47 | - name: Run test target in Makefile 48 | run: | 49 | cd test/repo 50 | make test 51 | -------------------------------------------------------------------------------- /.github/workflows/publish.yaml: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT 3 | 4 | name: Publish cbmc-starter-kit documentation 5 | on: 6 | push: 7 | branches: 8 | - 'documentation' 9 | 10 | jobs: 11 | publish: 12 | runs-on: macos-latest 13 | steps: 14 | - name: Checkout repository 15 | uses: actions/checkout@v4 16 | 17 | - name: Install mdbook 18 | run: brew install mdbook 19 | 20 | - name: Build documentation 21 | run: cd docs && mdbook build && touch book/.nojekyll 22 | 23 | - name: Publish documentation 24 | uses: JamesIves/github-pages-deploy-action@v4.7.3 25 | with: 26 | branch: gh-pages 27 | folder: docs/book/ 28 | 29 | # This conditional might be useful on the publish step in the future 30 | # if: ${{ github.event_name == 'push' && startsWith('refs/heads/master', github.ref) }} 31 | -------------------------------------------------------------------------------- /.github/workflows/release-brew.yaml: -------------------------------------------------------------------------------- 1 | # See following documentations for more information - 2 | 3 | # Creation and maintenance of personal taps: 4 | # https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap 5 | 6 | # More information about bottles: 7 | # https://docs.brew.sh/Bottles 8 | 9 | # Create a tap with automatic tests and bottle creation workflows: 10 | # https://brew.sh/2020/11/18/homebrew-tap-with-bottles-uploaded-to-github-releases/ 11 | 12 | # This workflow can be used to perform certain tasks from main repository rather 13 | # than a Tap - 14 | # - Formula Syntax Check 15 | # - Run brew test-bot for testing and bottle creation 16 | # (and upload bottles to release which triggered this workflow) 17 | # - Add commit on top of PR created by brew bump-formla-pr to add generate 18 | # bottle DSL to formula 19 | 20 | # USAGE: 21 | # Copy this workflow to .github/workflows of your repository and update 22 | # following variables in env below - 23 | # - FORMULA 24 | # - TAP 25 | # - BOT_USER 26 | # - BOT_EMAIL 27 | # - BOT_TOKEN 28 | 29 | # NOTE: 30 | # In case any other changes are required in the formula such as - 31 | # - Updating python dependencies 32 | # - Updating brew dependencies 33 | # - Updating test 34 | # - Updating Install block etc 35 | # Make sure to merge those changes before this workflow gets triggered as this 36 | # workflow will bump the formula and also create bottles right away 37 | # automatically. 38 | 39 | 40 | name: Release to brew 41 | on: 42 | release: 43 | types: [created] 44 | 45 | env: 46 | FORMULA: cbmc-starter-kit 47 | TAP: aws/tap 48 | BOT_USER: cbmc-starter-kit-release-ci 49 | RELEASE_TAG: ${GITHUB_REF/refs\/tags\/} # GITHUB_REF = refs/tags/STRING-MAJOR.MINOR 50 | VERSION: $(echo $GITHUB_REF | cut -d "/" -f 3 | cut -d "-" -f 2) 51 | AWS_ROLE: arn:aws:iam::${{secrets.AWS_ACCOUNT}}:role/PublisherTokenReader 52 | AWS_REGION: us-west-2 53 | 54 | jobs: 55 | homebrew-pr: 56 | name: Homebrew Bump Formula PR 57 | runs-on: macos-latest 58 | permissions: 59 | id-token: write 60 | steps: 61 | - name: Authenticate GitHub workflow to AWS 62 | uses: aws-actions/configure-aws-credentials@v4 63 | with: 64 | role-to-assume: ${{ env.AWS_ROLE }} 65 | aws-region: ${{ env.AWS_REGION }} 66 | 67 | - name: Fetch secrets 68 | run: | 69 | bot_email=$(aws secretsmanager get-secret-value --secret-id BOT_EMAIL | jq -r '.SecretString') 70 | echo "::add-mask::$bot_email" 71 | echo "BOT_EMAIL=$bot_email" >> $GITHUB_ENV 72 | homebrew_github_api_token=$(aws secretsmanager get-secret-value --secret-id RELEASE_CI_ACCESS_TOKEN | jq -r '.SecretString') 73 | echo "::add-mask::$homebrew_github_api_token" 74 | echo "HOMEBREW_GITHUB_API_TOKEN=$homebrew_github_api_token" >> $GITHUB_ENV 75 | 76 | - name: Configure git user name and email 77 | run: | 78 | git config --global user.name ${{ env.BOT_USER }} 79 | git config --global user.email ${{ env.BOT_EMAIL }} 80 | 81 | - name: Create homebrew PR 82 | run: | 83 | brew tap ${{ env.TAP }} 84 | brew update-reset 85 | brew bump-formula-pr --tag "${{ env.RELEASE_TAG }}" --revision "${{ github.sha }}" ${{ env.TAP }}/${{ env.FORMULA }} --force 86 | 87 | build-bottle: 88 | needs: homebrew-pr 89 | strategy: 90 | matrix: 91 | os: [ubuntu-latest, macos-latest] 92 | runs-on: ${{ matrix.os }} 93 | permissions: 94 | id-token: write 95 | contents: write 96 | steps: 97 | - name: Set up Homebrew 98 | id: set-up-homebrew 99 | uses: Homebrew/actions/setup-homebrew@master 100 | 101 | - name: Authenticate GitHub workflow to AWS 102 | uses: aws-actions/configure-aws-credentials@v4 103 | with: 104 | role-to-assume: ${{ env.AWS_ROLE }} 105 | aws-region: ${{ env.AWS_REGION }} 106 | 107 | - name: Fetch secrets 108 | run: | 109 | fork_repo="https://$(aws secretsmanager get-secret-value --secret-id RELEASE_CI_ACCESS_TOKEN | jq -r '.SecretString')@github.com/${{ env.BOT_USER }}/homebrew-$(echo ${{ env.TAP }} |cut -d / -f 2).git" 110 | echo "::add-mask::$fork_repo" 111 | echo "FORK_REPO=$fork_repo" >> $GITHUB_ENV 112 | github_token="$(aws secretsmanager get-secret-value --secret-id RELEASE_CI_ACCESS_TOKEN | jq -r '.SecretString')" 113 | echo "::add-mask::$github_token" 114 | echo "GITHUB_TOKEN=$github_token" >> $GITHUB_ENV 115 | 116 | - name: Set up Python 117 | # The GitHub action actions/setup-python@v4 installs CPython 3.10 118 | run: | 119 | brew install --overwrite python@3.10 120 | 121 | - name: Checkout PR 122 | run: | 123 | brew tap ${{ env.TAP }} 124 | brew update-reset 125 | cd $(brew --repo ${{ env.TAP }}) 126 | git remote add fork-repo ${{ env.FORK_REPO }} 127 | git fetch fork-repo 128 | git checkout -B bump-${{ env.FORMULA }}-${{ env.VERSION }} fork-repo/bump-${{ env.FORMULA }}-${{ env.VERSION }} 129 | 130 | - name: Tap Syntax 131 | run: | 132 | brew audit --online --git --skip-style ${{ env.TAP }}/${{ env.FORMULA }} 133 | brew style ${{ env.TAP }}/${{ env.FORMULA }} 134 | 135 | - name: Build bottle 136 | run: | 137 | brew test-bot --tap ${{ env.TAP }} --testing-formulae ${{ env.TAP }}/${{ env.FORMULA }} --only-formulae --root-url=https://github.com/${{ github.repository }}/releases/download/${{ env.RELEASE_TAG }} 138 | 139 | - name: Get Package Path 140 | id: get_package_path 141 | run: | 142 | echo "bottle_name=$(ls *.tar.gz)" >> $GITHUB_OUTPUT 143 | 144 | - name: Upload bottles as artifact 145 | uses: actions/upload-artifact@v4 146 | with: 147 | name: bottle-${{ matrix.os }} 148 | path: '*.bottle.*' 149 | 150 | - name: Upload release binary 151 | uses: softprops/action-gh-release@v2 152 | with: 153 | files: ${{ steps.get_package_path.outputs.bottle_name }} 154 | 155 | update-pr: 156 | needs: build-bottle 157 | runs-on: macos-latest 158 | permissions: 159 | id-token: write 160 | contents: write 161 | steps: 162 | - uses: actions/download-artifact@v4 163 | with: 164 | pattern: bottle-* 165 | 166 | - name: Authenticate GitHub workflow to AWS 167 | uses: aws-actions/configure-aws-credentials@v4 168 | with: 169 | role-to-assume: ${{ env.AWS_ROLE }} 170 | aws-region: ${{ env.AWS_REGION }} 171 | 172 | - name: Fetch secrets 173 | run: | 174 | bot_email="$(aws secretsmanager get-secret-value --secret-id BOT_EMAIL | jq -r '.SecretString')" 175 | echo "::add-mask::$bot_email" 176 | echo "BOT_EMAIL=$bot_email" >> $GITHUB_ENV 177 | fork_repo="https://$(aws secretsmanager get-secret-value --secret-id RELEASE_CI_ACCESS_TOKEN | jq -r '.SecretString')@github.com/${{ env.BOT_USER }}/homebrew-$(echo ${{ env.TAP }} |cut -d / -f 2).git" 178 | echo "::add-mask::$fork_repo" 179 | echo "FORK_REPO=$fork_repo" >> $GITHUB_ENV 180 | 181 | - name: Configure git user name and email 182 | run: | 183 | git config --global user.name ${{ env.BOT_USER }} 184 | git config --global user.email ${{ env.BOT_EMAIL }} 185 | 186 | - name: Checkout PR 187 | run: | 188 | brew tap ${{ env.TAP }} 189 | brew update-reset 190 | cd $(brew --repo ${{ env.TAP }}) 191 | git remote add fork-repo ${{ env.FORK_REPO }} 192 | git fetch fork-repo 193 | git checkout -B bump-${{ env.FORMULA }}-${{ env.VERSION }} fork-repo/bump-${{ env.FORMULA }}-${{ env.VERSION }} 194 | 195 | - name: Generate and merge bottle DSL 196 | run: | 197 | brew bottle --merge --write $(ls bottle-*/*.json) 198 | cd $(brew --repo ${{ env.TAP }}) 199 | git push fork-repo bump-${{ env.FORMULA }}-${{ env.VERSION }} 200 | -------------------------------------------------------------------------------- /.github/workflows/release-pypi.yaml: -------------------------------------------------------------------------------- 1 | name: Release to PyPi 2 | on: 3 | release: 4 | types: [created] 5 | 6 | env: 7 | AWS_ROLE: arn:aws:iam::${{secrets.AWS_ACCOUNT}}:role/PublisherTokenReader 8 | AWS_REGION: us-west-2 9 | 10 | jobs: 11 | upload-to-pypi: 12 | name: Upload to PyPi 13 | runs-on: ubuntu-20.04 14 | permissions: 15 | id-token: write 16 | contents: write 17 | steps: 18 | - uses: actions/checkout@v4 19 | - name: Install dependencies 20 | run: python3 -m pip install --upgrade pip build setuptools wheel 21 | - name: Build pip package 22 | run: python3 -m build 23 | - name: Authenticate GitHub workflow to AWS 24 | uses: aws-actions/configure-aws-credentials@v4 25 | with: 26 | role-to-assume: ${{ env.AWS_ROLE }} 27 | aws-region: ${{ env.AWS_REGION }} 28 | - name: Fetch secrets 29 | run: | 30 | github_token="$(aws secretsmanager get-secret-value --secret-id RELEASE_CI_ACCESS_TOKEN | jq -r '.SecretString')" 31 | echo "::add-mask::$github_token" 32 | echo "GITHUB_TOKEN=$github_token" >> $GITHUB_ENV 33 | - name: Get Package Name 34 | id: get_package_name 35 | run: | 36 | echo "package_name=$(ls dist/*.whl | cut -d / -f 2)" >> $GITHUB_OUTPUT 37 | - name: Upload release binary 38 | uses: softprops/action-gh-release@v2 39 | with: 40 | files: dist/${{ steps.get_package_name.outputs.package_name }} 41 | - name: Publish package distributions to PyPI 42 | uses: pypa/gh-action-pypi-publish@release/v1 43 | -------------------------------------------------------------------------------- /.github/workflows/release.yaml: -------------------------------------------------------------------------------- 1 | name: CBMC starter kit release 2 | on: 3 | push: 4 | tags: 5 | - starterkit-* 6 | env: 7 | AWS_ROLE: arn:aws:iam::${{secrets.AWS_ACCOUNT}}:role/PublisherTokenReader 8 | AWS_REGION: us-west-2 9 | 10 | jobs: 11 | Release: 12 | name: CBMC starter kit release 13 | runs-on: ubuntu-20.04 14 | permissions: 15 | id-token: write 16 | contents: write 17 | steps: 18 | - name: Checkout code 19 | uses: actions/checkout@v4 20 | - name: Get version 21 | run: | 22 | # The environment variable GITHUB_REF is refs/tags/starterkit-* 23 | echo "SETUP_VERSION=$(python3 -c "import configparser; config = configparser.ConfigParser(); config.read('setup.cfg'); print(config['metadata']['version'])")" >> $GITHUB_ENV 24 | echo "SOURCE_VERSION=$(python3 -c "import src.cbmc_starter_kit.version; print(src.cbmc_starter_kit.version.NUMBER)")" >> $GITHUB_ENV 25 | echo "TAG_VERSION=$(echo ${{ github.ref }} | cut -d "/" -f 3 | cut -d "-" -f 2)" >> $GITHUB_ENV 26 | - name: Version check 27 | run: | 28 | if [[ ${{ env.SETUP_VERSION }} != ${{ env.TAG_VERSION }} ]] || [[ ${{ env.SOURCE_VERSION }} != ${{ env.TAG_VERSION }} ]]; then 29 | echo "Setup and source versions ${{env.SETUP_VERSION}} and ${{env.SOURCE_VERSION}} did not match tag version ${{env.TAG_VERSION}}" 30 | exit 1 31 | fi 32 | - name: Authenticate GitHub workflow to AWS 33 | uses: aws-actions/configure-aws-credentials@v4 34 | with: 35 | role-to-assume: ${{ env.AWS_ROLE }} 36 | aws-region: ${{ env.AWS_REGION }} 37 | - name: Fetch secrets 38 | run: | 39 | github_token="$(aws secretsmanager get-secret-value --secret-id RELEASE_CI_ACCESS_TOKEN | jq -r '.SecretString')" 40 | echo "::add-mask::$github_token" 41 | echo "GITHUB_TOKEN=$github_token" >> $GITHUB_ENV 42 | - name: Create release 43 | uses: actions/create-release@v1 44 | with: 45 | tag_name: starterkit-${{ env.TAG_VERSION }} 46 | release_name: starterkit-${{ env.TAG_VERSION }} 47 | body: | 48 | This is the CBMC Starter Kit version ${{ env.TAG_VERSION }}. 49 | 50 | The [CBMC starter kit](https://model-checking.github.io/cbmc-starter-kit/) makes it easy to add [CBMC](https://github.com/diffblue/cbmc) verification to a software project. See the [starter kit tutorial](https://model-checking.github.io/cbmc-training/starter-kit/overview/index.html) for an example of how to use the starter kit. 51 | 52 | On MacOS, you can install with brew. Install with 53 | ``` 54 | brew tap aws/tap 55 | brew install cbmc-starter-kit 56 | ``` 57 | and upgrade to the latest version with 58 | ``` 59 | brew upgrade cbmc-starter-kit 60 | ``` 61 | 62 | On all machines, you can install with pip. Install with 63 | ``` 64 | python3 -m pip install cbmc-starter-kit 65 | ``` 66 | and upgrade to the latest version with 67 | ``` 68 | python3 -m pip install --upgrade cbmc-starter-kit 69 | ``` 70 | draft: false 71 | prerelease: false 72 | -------------------------------------------------------------------------------- /.github/workflows/summarize-test.yaml: -------------------------------------------------------------------------------- 1 | name: Unit tests for summarize module 2 | on: 3 | pull_request: 4 | types: [opened, synchronize, reopened, labeled, unlabeled] 5 | push: 6 | branches: [ master ] 7 | 8 | jobs: 9 | run-tests: 10 | if: "!contains(github.event.pull_request.labels.*.name, 'no-test')" 11 | runs-on: ubuntu-latest 12 | strategy: 13 | matrix: 14 | python-version: [ '3.8', '3.9', '3.10' ] 15 | name: Python ${{ matrix.python-version }} 16 | steps: 17 | - name: Check out repository 18 | uses: actions/checkout@v4 19 | - name: Set up Python 20 | uses: actions/setup-python@v5 21 | with: 22 | python-version: ${{ matrix.python-version }} 23 | - name: Run unit test for the 'summarize' module 24 | run: | 25 | cd test/summarize_test 26 | python summarize_test.py 27 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__/ 2 | *~ 3 | *.egg-info 4 | dist/ 5 | build/ 6 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *master* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | 61 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. 62 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of this 4 | software and associated documentation files (the "Software"), to deal in the Software 5 | without restriction, including without limitation the rights to use, copy, modify, 6 | merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 7 | permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 10 | INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 11 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 12 | HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 13 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 14 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | # See https://packaging.python.org/en/latest/tutorials/installing-packages/ 5 | # See https://packaging.python.org/tutorials/packaging-projects/ 6 | # python3 -m ensurepip 7 | # python3 -m pip install --upgrade pip setuptools wheel 8 | # python3 -m pip install --upgrade build 9 | # python3 -m pip install --upgrade pylint 10 | 11 | default: 12 | @echo Nothing to make 13 | 14 | ################################################################ 15 | # Run pylint over the package 16 | 17 | pylint: 18 | make -C src/cbmc_starter_kit pylint 19 | 20 | ################################################################ 21 | # Build the distribution package 22 | 23 | build: 24 | python3 -m build 25 | 26 | unbuild: 27 | $(RM) -r dist 28 | 29 | ################################################################ 30 | # Install the package into a virtual environment in development mode 31 | # 32 | # Note: Editable installs from pyproject.toml require at least pip 21.3 33 | 34 | VENV = /tmp/cbmc-starter-kit 35 | develop: 36 | python3 -m venv $(VENV) 37 | $(VENV)/bin/python3 -m pip install --upgrade pip 38 | $(VENV)/bin/python3 -m pip install --editable . 39 | @ echo 40 | @ echo "Package installed into virtual environment at $(VENV)." 41 | @ echo "Activate virtual environment with 'source $(VENV)/bin/activate'" 42 | @ echo "(or add it to PATH with 'export PATH=\$$PATH:$(VENV)/bin')." 43 | @ echo 44 | 45 | undevelop: 46 | $(RM) -r $(VENV) 47 | 48 | ################################################################ 49 | # Install the package 50 | # 51 | # Note: This requires write permission (sudo): It updates the system 52 | # site-packages directory. 53 | 54 | 55 | install: 56 | python3 -m pip install --verbose . 57 | 58 | uninstall: 59 | python3 -m pip uninstall --verbose --yes cbmc-starter-kit 60 | 61 | ################################################################ 62 | # Clean up after packaging and installation 63 | 64 | clean: 65 | $(RM) *~ 66 | $(RM) *.pyc 67 | $(RM) -r __pycache__ 68 | 69 | veryclean: clean unbuild undevelop 70 | 71 | ################################################################ 72 | # Test uploading package to test.pypi.org 73 | 74 | twine: 75 | $(RM) -r dist 76 | python3 -m build 77 | python3 -m twine upload --repository testpypi dist/* 78 | 79 | ################################################################ 80 | 81 | .PHONY: build clean default develop install pylint twine 82 | .PHONY: unbuild undevelop uninstall veryclean 83 | -------------------------------------------------------------------------------- /NOTICE: -------------------------------------------------------------------------------- 1 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # CBMC starter kit 2 | 3 | This is a starter kit for writing CBMC proofs. 4 | 5 | [CBMC](https://github.com/diffblue/cbmc) is a model checker for 6 | C. This means that CBMC will explore all possible paths through your code 7 | on all possible inputs, and will check that all assertions in your code are 8 | true. 9 | CBMC can also check for the possibility of 10 | memory safety errors (like buffer overflow) and for instances of 11 | undefined behavior (like signed integer overflow). 12 | CBMC is a bounded model checker, however, which means that the set of all 13 | possible inputs may have to be restricted to all inputs of some bounded size. 14 | 15 | The [starter kit overview](https://model-checking.github.io/cbmc-training/starter-kit/overview/index.html) 16 | gives a fairly complete example of how to use the starter kit to add 17 | CBMC verification to an existing software project. 18 | 19 | The [starter kit wiki](https://github.com/awslabs/aws-templates-for-cbmc-proofs/wiki) 20 | is currently the primary user guide for the starter kit. 21 | 22 | ## Installation 23 | 24 | The starter kit is distributed as both a brew package and a pip package, and the 25 | [release page](https://github.com/model-checking/cbmc-starter-kit/releases/latest) 26 | gives installation instructions that we repeat here. 27 | 28 | ### brew installation 29 | 30 | On MacOS, we recommend using brew to install the starter kit with 31 | ``` 32 | brew tap aws/tap 33 | brew install cbmc-starter-kit 34 | ``` 35 | and upgrade to the latest version with 36 | ``` 37 | brew upgrade cbmc-starter-kit 38 | ``` 39 | In these instructions, the first line taps an AWS repository that hosts the starter kit. 40 | The [brew home page](https://brew.sh/) gives instructions for installing brew. 41 | 42 | ### pip installation 43 | 44 | On any operating system with python installed, use pip to install the 45 | starter kit with 46 | ``` 47 | python3 -m pip install cbmc-starter-kit 48 | ``` 49 | and upgrade to the latest version with 50 | ``` 51 | python3 -m pip install --upgrade cbmc-starter-kit 52 | ``` 53 | The [python download page](https://www.python.org/downloads/) gives instructions 54 | for installing python. 55 | 56 | ## Security 57 | 58 | See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. 59 | 60 | ## License 61 | 62 | This project is licensed under the Apache-2.0 License. 63 | -------------------------------------------------------------------------------- /docs/.gitignore: -------------------------------------------------------------------------------- 1 | book 2 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | build: 2 | mdbook build 3 | 4 | browse: 5 | mdbook serve --open 6 | 7 | clean: 8 | $(RM) books 9 | -------------------------------------------------------------------------------- /docs/README.md: -------------------------------------------------------------------------------- 1 | This directory contains our documentation on how to use cbmc-viewer. 2 | 3 | The [`mdbook` documentation](https://rust-lang.github.io/mdBook/) 4 | explains how to use `mdbook` to generate documentation. 5 | * The file `book.toml` is the configuration file used by `mdbook` to 6 | build the documentation. 7 | * The directory `src` is the tree of markdown files that generate the 8 | "book" containing the documentation. 9 | * The file `src/SUMMARY.md` is the list of "chapters" and 10 | "subchapters" that will appear in the book. The only pages that 11 | will appear in the book are those generated from the markdown files 12 | listed in this summary. 13 | * The directory `book` will contain the html making up the book. 14 | 15 | You can build and browse the documentation by running the following 16 | commands in this directory: 17 | ```bash 18 | mdbook build 19 | mdbook serve --open 20 | ``` 21 | -------------------------------------------------------------------------------- /docs/book.toml: -------------------------------------------------------------------------------- 1 | [book] 2 | authors = ["Mark R. Tuttle"] 3 | language = "en" 4 | multilingual = false 5 | src = "src" 6 | title = "CBMC starter kit" 7 | -------------------------------------------------------------------------------- /docs/src/README.md: -------------------------------------------------------------------------------- 1 | # Getting started with the CBMC starter kit 2 | 3 | The [CBMC starter kit](https://github.com/model-checking/cbmc-starter-kit) makes 4 | it easy to add CBMC verification to an existing software project. 5 | 6 | [CBMC](https://github.com/diffblue/cbmc) is a model checker for 7 | C. This means that CBMC will explore all possible paths through your code 8 | on all possible inputs, and will check that all assertions in your code are 9 | true. 10 | CBMC can also check for the possibility of 11 | memory safety errors (like buffer overflow) and for instances of 12 | undefined behavior (like signed integer overflow). 13 | CBMC is a bounded model checker, however, which means that the set of all 14 | possible inputs may have to be restricted to all inputs of some bounded size. 15 | 16 | For a quick start on using the starter kit, see 17 | 18 | * [Installation](installation) 19 | * [Tutorial](tutorial) -------------------------------------------------------------------------------- /docs/src/SUMMARY.md: -------------------------------------------------------------------------------- 1 | # Summary 2 | 3 | * [Getting started](README.md) 4 | * [Installation](installation/README.md) 5 | * [Tutorial](tutorial/README.md) 6 | * [User manual]() 7 | * [Reference manual](reference-manual/README.md) 8 | * [cbmc-starter-kit-setup](reference-manual/cbmc-starter-kit-setup.md) 9 | * [cbmc-starter-kit-setup-proof](reference-manual/cbmc-starter-kit-setup-proof.md) 10 | * [cbmc-starter-kit-setup-ci](reference-manual/cbmc-starter-kit-setup-ci.md) 11 | * [cbmc-starter-kit-update](reference-manual/cbmc-starter-kit-update.md) 12 | * [cbmc-starter-kit-migrate-license](reference-manual/cbmc-starter-kit-migrate-license.md) 13 | * [Frequently asked questions]() 14 | * [Resources](resources/README.md) 15 | * [Contributing](contributing/README.md) 16 | -------------------------------------------------------------------------------- /docs/src/contributing/README.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | This material is a work in progress. If you have suggestions, 4 | corrections, or questions, contact us by submitting a 5 | [GitHub issue](https://github.com/model-checking/cbmc-starter-kit/issues). 6 | If you have some examples of your own that you would like to contribute, 7 | submit your contributions as a 8 | [GitHub pull request](https://github.com/model-checking/cbmc-starter-kit/pulls). 9 | -------------------------------------------------------------------------------- /docs/src/installation/README.md: -------------------------------------------------------------------------------- 1 | # Installation 2 | 3 | The starter kit is a command line tool distributed as both a brew package and a pip package. 4 | The [starter kit release page](https://github.com/model-checking/cbmc-starter-kit/releases/latest) 5 | gives installation instructions that we repeat here. 6 | 7 | Note: The starter kit used to be distributed as a git repository that you submoduled into 8 | your own repository. The starter kit is now distributed as a command line tool. Follow 9 | the [update instructions](#updating-to-the-command-line-tool) below to upgrade your project 10 | from using the submodule to using the command line tool. 11 | 12 | ## Installing with brew 13 | 14 | On MacOS, we recommend using brew to install the starter kit: 15 | ``` 16 | brew tap aws/tap 17 | brew install cbmc-starter-kit cbmc-viewer litani 18 | ``` 19 | 20 | The [brew home page](https://brew.sh/) gives instructions for installing brew. 21 | The command `brew tap aws/tap` taps the AWS repository that contains the brew packages. 22 | The [cbmc-viewer](https://github.com/model-checking/cbmc-viewer) 23 | and [litani](https://github.com/awslabs/aws-build-accumulator) packages are tools 24 | used by the starter kit. 25 | See the [cbmc-viewer release page](https://github.com/model-checking/cbmc-viewer/releases/latest) 26 | and the [litani release page](https://github.com/awslabs/aws-build-accumulator/releases/latest) 27 | for other ways to install them if you don't want to use brew. 28 | 29 | ## Installing with pip 30 | 31 | On any machine, you can use pip to install the starter kit: 32 | ``` 33 | python3 -m pip install cbmc-starter-kit cbmc-viewer 34 | ``` 35 | 36 | The [python download page](https://www.python.org/downloads/) 37 | gives instructions for installing python and pip. 38 | The [cbmc-viewer](https://github.com/model-checking/cbmc-viewer) 39 | and [litani](https://github.com/awslabs/aws-build-accumulator) packages are tools 40 | used by the starter kit. 41 | See the [cbmc-viewer release page](https://github.com/model-checking/cbmc-viewer/releases/latest) 42 | and the [litani release page](https://github.com/awslabs/aws-build-accumulator/releases/latest) 43 | for installation instructions (we used pip to install `cbmc-viewer` in the command above). 44 | 45 | ## Installing for developers 46 | 47 | Developers can install the package in "editable mode" which makes 48 | it possible to modify the code in the source tree and then run the command 49 | from the command line as usual to test the changes. 50 | First, install `cbmc-viewer` and `litani` as described above if you haven't already. 51 | Then 52 | 53 | * Clone the repository and install dependencies with 54 | ``` 55 | git clone https://github.com/model-checking/cbmc-starter-kit cbmc-starter-kit 56 | python3 -m pip install virtualenv gitpython 57 | ``` 58 | * Install into a virtual environment with 59 | ``` 60 | cd cbmc-starter-kit 61 | make develop 62 | ``` 63 | At this point you can either activate the virtual environment with 64 | ``` 65 | source /tmp/cbmc-starter-kit/bin/activate 66 | ``` 67 | or simply add the virtual environment to your path with 68 | ``` 69 | export PATH=/tmp/cbmc-starter-kit/bin:$PATH 70 | ``` 71 | 72 | * Uninstall with 73 | ``` 74 | cd cbmc-starter-kit 75 | make undevelop 76 | ``` 77 | ## Updating 78 | 79 | To update to the the latest version of the starter kit: 80 | * Update the starter kit itself with 81 | ``` 82 | brew upgrade cbmc-starter-kit 83 | ``` 84 | or 85 | ``` 86 | python3 -m pip install cbmc-starter-kit --upgrade 87 | ``` 88 | * Update your repository by changing to the `cbmc` directory that contains the `proofs` 89 | directory installed by the starter kit and running the update script: 90 | ``` 91 | cd cbmc 92 | cbmc-starter-kit-update 93 | ``` 94 | This will overwrite `Makefile.common` and `run-cbmc-proofs.py` in the `proofs` directory 95 | with the latest versions. 96 | 97 | ## Updating to the command line tool 98 | 99 | The starter kit used to be distributed as a git repository that you submoduled into your 100 | own repository. The same used to be true for litani, the build tool used by the starter kit. 101 | Now both are distributed a command line tools. 102 | We recommend that you migrate to using these command line tools. 103 | 104 | * Install the starter kit and litani as command line tools 105 | * On MacOS 106 | ``` 107 | brew install cbmc-starter-kit litani 108 | ``` 109 | * On Ubuntu, first download the litani debian package from the 110 | [litani release page](https://github.com/awslabs/aws-build-accumulator/releases/latest) then 111 | ``` 112 | python3 -m pip install cbmc-starter-kit 113 | apt install ./litani-*.deb 114 | ``` 115 | * Change to the `cbmc` directory that contains the `proofs` directory installed by the 116 | starter kit, and run the update script 117 | ``` 118 | cd cbmc 119 | cbmc-starter-kit-update --remove-starter-kit-submodule --remove-litani-submodule 120 | ``` 121 | This will overwrite `Makefile.common` and `run-cbmc-proofs.py` in the `proofs` directory 122 | with the latest versions, and remove the submodules for the starter kit and litani 123 | if they are present. It will also perform some cleanup actions, such as replacing symbolic 124 | links into the starter kit submodule with copies of the files being linked to, and removing 125 | a set of negative tests that are rarely used. 126 | 127 | See [cbmc-starter-kit-update](../reference-manual/cbmc-starter-kit-update.md) 128 | for more information. 129 | 130 | While you are at it, we have changed the starter kit license from Apache 2.0 to MIT-0, and 131 | you might want to run a script to change copyright headers from Apache to MIT in files 132 | under the `cbmc` directory: 133 | ``` 134 | cd cbmc 135 | cbmc-starter-kit-migrate-license --remove 136 | ``` 137 | See [cbmc-starter-kit-migrate-license](../reference-manual/cbmc-starter-kit-migrate-license.md) 138 | for more information. 139 | 140 | ## Running CBMC proofs as part of CI 141 | 142 | The starter kit offers GitHub repositories with the ability to run CBMC proofs 143 | in GitHub Actions. If the proofs are unable to run in 144 | [GitHub's standard Runner](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources), 145 | you may need to constrain some proofs' parallelism with the 146 | [`EXPENSIVE`](https://model-checking.github.io/cbmc-starter-kit/tutorial/index.html#the-makefile) 147 | setting, or [create a Large Runner for your repository](https://docs.github.com/en/actions/using-github-hosted-runners/using-larger-runners). 148 | 149 | See [cbmc-starter-kit-setup-ci](../reference-manual/cbmc-starter-kit-setup-ci.md) 150 | for more information. 151 | 152 | ## Installation notes 153 | 154 | If you have difficulty installing these tools, please let us know by 155 | submitting a [GitHub issue](https://github.com/model-checking/cbmc-starter-kit/issues). 156 | -------------------------------------------------------------------------------- /docs/src/reference-manual/README.md: -------------------------------------------------------------------------------- 1 | # Reference manual 2 | 3 | The CBMC Starter Kit makes it easy to add CBMC verification to a 4 | software project. 5 | 6 | * [cbmc-starter-kit-setup](cbmc-starter-kit-setup.md) 7 | * [cbmc-starter-kit-setup-proof](cbmc-starter-kit-setup-proof.md) 8 | * [cbmc-starter-kit-setup-ci](cbmc-starter-kit-setup-ci.md) 9 | * [cbmc-starter-kit-update](cbmc-starter-kit-update.md) 10 | * [cbmc-starter-kit-migrate-license](cbmc-starter-kit-migrate-license.md) 11 | -------------------------------------------------------------------------------- /docs/src/reference-manual/cbmc-starter-kit-migrate-license.md: -------------------------------------------------------------------------------- 1 | # Reference manual 2 | 3 | ## Name 4 | 5 | `cbmc-starter-kit-migrate-license` - Remove references to Apache license from CBMC starter kit. 6 | 7 | ## Synopsis 8 | 9 | ``` 10 | cbmc-starter-kit-migrate-license [-h] [--proofdir PROOFDIR] [--remove] 11 | [--verbose] [--debug] [--version] 12 | ``` 13 | ## Description 14 | 15 | This script is used to remove references to the Apache license installed 16 | by early versions of the starter kit. 17 | 18 | The CBMC starter kit was originally released under the Apache 19 | license. All files in the starter kit contained references to the 20 | Apache license. The starter kit installation scripts copied files from 21 | the stater kit into the project repository. This became an issue when 22 | the project repository was released under a more permissive 23 | license. This script removes all references to the Apache license from 24 | the files copied into the project repository from the starter kit, and 25 | uses the MIT-0 license instead. 26 | 27 | ## Options 28 | 29 | `--proofdir PROOFDIR` 30 | 31 | * Root of the proof subtree (default: .) 32 | 33 | `--remove` 34 | 35 | * Remove Apache references from files under PROOFDIR (otherwise just list them) 36 | 37 | `--help, -h` 38 | 39 | `--verbose` 40 | 41 | * Verbose output. 42 | 43 | `--debug` 44 | 45 | * Debugging output. 46 | 47 | `--version` 48 | 49 | * Display version number and exit. 50 | -------------------------------------------------------------------------------- /docs/src/reference-manual/cbmc-starter-kit-setup-ci.md: -------------------------------------------------------------------------------- 1 | # Reference manual 2 | 3 | ## Name 4 | 5 | `cbmc-starter-kit-setup-ci` - Add GitHub Actions workflow to run proofs as part of CI 6 | 7 | ## Synopsis 8 | 9 | ``` 10 | cbmc-starter-kit-setup-ci [-h] 11 | --github-actions-runner 12 | [--cbmc ] 13 | [--cbmc-viewer ] 14 | [--litani ] 15 | [--kissat ] 16 | [--cadical ] 17 | [--verbose] [--debug] [--version] 18 | ``` 19 | 20 | ## Description 21 | 22 | This script will copy a GitHub Action workflow to `.github/workflows` of your 23 | repository. The workflow will runs CBMC proofs on every push event. 24 | 25 | The workflow must be executed in a GitHub-hosted Ubuntu 20.04 runner. If the 26 | proofs are unable to run in 27 | [GitHub's standard Runner](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources), 28 | you may need to constrain some proofs' parallelism with the 29 | [`EXPENSIVE`](https://model-checking.github.io/cbmc-starter-kit/tutorial/index.html#the-makefile) 30 | setting, or [create a Large Runner for your repository](https://docs.github.com/en/actions/using-github-hosted-runners/using-larger-runners). 31 | 32 | The script offers users with the option of specifying custom versions and tags 33 | for all tools being used inside of the CI. 34 | 35 | For a given invocation of the GitHub Actions workflow, the results of the 36 | execution of all CBMC proofs will be presented in a tabular format in the logs 37 | of the GitHub Actions step called "CBMC proof results". 2 tables are being 38 | printed out that summarize the number of proofs that succeeded/failed as well 39 | as individual statuses per CBMC proof. Aside for the logs, these tables are also available at the "Summary" page for this particular invocation workflow. Finally, 40 | a zip artifact, which contains all logs and artifacts pertaining to this 41 | execution of CBMC proofs, is available to be downloaded at this "Summary" page. 42 | 43 | If a CBMC proof fails, the GitHub Actions workflow will fail at the 44 | "CBMC proof results" step. 45 | -------------------------------------------------------------------------------- /docs/src/reference-manual/cbmc-starter-kit-setup-proof.md: -------------------------------------------------------------------------------- 1 | # Reference manual 2 | 3 | ## Name 4 | 5 | `cbmc-starter-kit-setup-proof` - Set up CBMC proof infrastructure for a proof 6 | 7 | ## Synopsis 8 | 9 | ``` 10 | cbmc-starter-kit-setup-proof [-h] [--verbose] [--debug] [--version] 11 | ``` 12 | 13 | ## Description 14 | 15 | This script sets up the CBMC proof infrastructure for an individual proof. 16 | It asks for the name of the function under test. 17 | It then searches the repository for source files that define a 18 | function with that name, and asks you to select the file giving the 19 | implementation you want to test. 20 | If none of the files listed is the correct file, you can give the path 21 | to the correct source file yourself. 22 | Finally, the script creates a directory with the name of the function 23 | and copies into that directory some files to simplify getting started 24 | with the verification of that function. Most important, it copies 25 | a skeleton of a Makefile and skeleton of a proof harness that you can 26 | edit to get started. 27 | 28 | This script is usually run in the CBMC verification root 29 | (the directory you created to hold the CBMC verification work, 30 | and in which you ran the `cbmc-stater-kit-setup` script). 31 | This script can, however, be run in any subdirectory of the 32 | CBMC verification root. In this way, you can group verification of 33 | similar functions together in a hierarchy of subdirectories. 34 | 35 | This script will need to be run once for each function you want 36 | to verify. 37 | 38 | ## Options 39 | 40 | `--verbose` 41 | 42 | * Verbose output. 43 | 44 | `--debug` 45 | 46 | * Debugging output. 47 | 48 | `--version` 49 | 50 | * Display version number and exit. 51 | 52 | `--help, -h` 53 | 54 | * Print the help message and exit. 55 | -------------------------------------------------------------------------------- /docs/src/reference-manual/cbmc-starter-kit-setup.md: -------------------------------------------------------------------------------- 1 | # Reference manual 2 | 3 | ## Name 4 | 5 | `cbmc-starter-kit-setup` - Set up CBMC proof infrastructure for a repository 6 | 7 | ## Synopsis 8 | 9 | ``` 10 | cbmc-starter-kit-setup [-h] [--verbose] [--debug] [--version] 11 | ``` 12 | 13 | ## Description 14 | 15 | This script sets up the CBMC proof infrastructure for a repository. 16 | It locates the root of the repository, it asks for a name to use 17 | for the CBMC verification activity (any name will do, and the name can be 18 | changed at any time), and it copies into the current directory a collection 19 | of files that simplify getting started with CBMC. 20 | 21 | We recommend that you create a directory with a name like `cbmc` somewhere 22 | within the repository to hold the CBMC verification work. This script 23 | assumes that it is running in this verification directory, and assumes 24 | that this directory is under the root of a git repository. 25 | 26 | This script needs to be run only one time to prepare the repository for 27 | CBMC verification. 28 | 29 | ## Options 30 | 31 | `--verbose` 32 | 33 | * Verbose output. 34 | 35 | `--debug` 36 | 37 | * Debugging output. 38 | 39 | `--version` 40 | 41 | * Display version number and exit. 42 | 43 | `--help, -h` 44 | 45 | * Print the help message and exit. 46 | -------------------------------------------------------------------------------- /docs/src/reference-manual/cbmc-starter-kit-update.md: -------------------------------------------------------------------------------- 1 | # Reference manual 2 | 3 | ## Name 4 | 5 | `cbmc-starter-kit-update` - Update CBMC starter kit in a CBMC proof repository 6 | 7 | ## Synopsis 8 | 9 | ``` 10 | cbmc-starter-kit-update [-h] [--cbmc-root CBMC] 11 | [--starter-kit-root STARTER_KIT] [--no-migrate] 12 | [--no-test-removal] [--no-update] 13 | [--remove-starter-kit-submodule] 14 | [--remove-litani-submodule] [--verbose] 15 | [--debug] [--version] 16 | ``` 17 | 18 | ## Description 19 | 20 | This script is used to update the CBMC starter kit installed in your 21 | repository to the lastest version. It copies (overwrites) two files 22 | into your repository (`Makefile.common` and `run-cbmc-proofs.py`). 23 | These are the two files in the starter kit that encode our best 24 | practices for how to use CBMC in a software verification project. 25 | 26 | This script will also migrate your repository from early versions of 27 | the starter kit and litani (the build system used by the starter kit) 28 | to modern versions. Early versions of the starter kit and litani were 29 | distributed as as git repositories that you submoduled into your 30 | repository. The starter kit also installed symbolic links from your 31 | repository into the starter kit submodule. This script will remove the 32 | symbolic links (replace them with the files they are linking to) and will 33 | also remove the starter kit and litani submodules from your repository 34 | if you give the `--remove-stater-kit-submodule` and 35 | `--remove-litani-submodule` flags (flags you almost certainly want to 36 | use during the migration). Finally, it will remove some regression tests 37 | that were distributed with early versions of the starter kit that most people 38 | don't use. 39 | 40 | ## Options 41 | 42 | `--cbmc-root CBMC` 43 | 44 | * Root of CBMC proof infrastructure (default: ".") 45 | 46 | `--starter-kit-root STARTER_KIT` 47 | 48 | * Root of CBMC starter kit submodule (default: None or 49 | root of starter kit submodule installed in repository 50 | containing CBMC) 51 | 52 | `--no-migrate` 53 | 54 | * Do not remove symlinks under CBMC. Normally remove 55 | symlinks under CBMC to files under STARTER_KIT. 56 | 57 | `--no-test-removal` 58 | 59 | * Do not remove negative tests in CBMC/negative_tests. 60 | Normally remove the directory CBMC/negative_tests 61 | since most projects don't use these tests. 62 | 63 | `--no-update` 64 | 65 | * Do not update Makefile.common and run-cbmc-proofs.py 66 | under CBMC/proofs. Normally update these files with 67 | the versions in the starter kit package. 68 | 69 | `--remove-starter-kit-submodule` 70 | 71 | * Remove the starter kit submodule if it is present. 72 | Normally just recommend removal. 73 | 74 | `--remove-litani-submodule` 75 | 76 | * Remove the litani submodule and update the definition 77 | of LITANI in Makefile-template-defines if the litani 78 | submodule is present and the litani command is in 79 | PATH. Normally just recommend removal. 80 | 81 | `--verbose` 82 | 83 | * Verbose output 84 | 85 | `--debug` 86 | 87 | * Debug output 88 | 89 | `--version` 90 | 91 | * Display version and exit 92 | 93 | `--help, -h` 94 | 95 | * Print the help message and exit. 96 | -------------------------------------------------------------------------------- /docs/src/resources/README.md: -------------------------------------------------------------------------------- 1 | # CBMC starter kit resources 2 | 3 | * CBMC (the model checker used by the starter kit) 4 | * [Documentation](http://www.cprover.org/cprover-manual) and [tutorial](https://github.com/diffblue/cbmc/blob/develop/doc/cprover-manual/cbmc-tutorial.md) 5 | * [Source code](https://github.com/diffblue/cbmc) 6 | 7 | * Litani (the build system used by the starter kit) 8 | * [Documentation](https://awslabs.github.io/aws-build-accumulator/) 9 | * [Source code](https://github.com/awslabs/aws-build-accumulator) 10 | 11 | * CBMC starter kit 12 | * [Source code](https://github.com/model-checking/cbmc-starter-kit) 13 | -------------------------------------------------------------------------------- /docs/src/tutorial/FreeRTOSConfig.h: -------------------------------------------------------------------------------- 1 | /* 2 | * FreeRTOS V202112.00 3 | * Copyright (C) 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | * 5 | * Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | * this software and associated documentation files (the "Software"), to deal in 7 | * the Software without restriction, including without limitation the rights to 8 | * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | * the Software, and to permit persons to whom the Software is furnished to do so, 10 | * subject to the following conditions: 11 | * 12 | * The above copyright notice and this permission notice shall be included in all 13 | * copies or substantial portions of the Software. 14 | * 15 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 17 | * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 18 | * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 19 | * IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 20 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 21 | * 22 | * https://www.FreeRTOS.org 23 | * https://github.com/FreeRTOS 24 | * 25 | */ 26 | #ifndef FREERTOS_CONFIG_H 27 | #define FREERTOS_CONFIG_H 28 | 29 | /*----------------------------------------------------------- 30 | * Application specific definitions. 31 | * 32 | * These definitions should be adjusted for your particular hardware and 33 | * application requirements. 34 | * 35 | * THESE PARAMETERS ARE DESCRIBED WITHIN THE 'CONFIGURATION' SECTION OF THE 36 | * FreeRTOS API DOCUMENTATION AVAILABLE ON THE FreeRTOS.org WEB SITE. See 37 | * https://www.FreeRTOS.org/a00110.html 38 | *----------------------------------------------------------*/ 39 | 40 | #define configUSE_PREEMPTION 1 41 | #define configUSE_PORT_OPTIMISED_TASK_SELECTION 0 42 | #define configUSE_IDLE_HOOK 1 43 | #define configUSE_TICK_HOOK 1 44 | #define configUSE_DAEMON_TASK_STARTUP_HOOK 1 45 | #define configTICK_RATE_HZ ( 1000 ) /* In this non-real time simulated environment the tick frequency has to be at least a multiple of the Win32 tick frequency, and therefore very slow. */ 46 | #define configMINIMAL_STACK_SIZE ( ( unsigned short ) 70 ) /* In this simulated case, the stack only has to hold one small structure as the real stack is part of the win32 thread. */ 47 | #define configTOTAL_HEAP_SIZE ( ( size_t ) ( 1 * 128 ) ) 48 | #define configMAX_TASK_NAME_LEN ( 12 ) 49 | #define configUSE_TRACE_FACILITY 1 50 | #define configUSE_16_BIT_TICKS 0 51 | #define configIDLE_SHOULD_YIELD 1 52 | #define configUSE_MUTEXES 1 53 | #define configCHECK_FOR_STACK_OVERFLOW 0 54 | #define configUSE_RECURSIVE_MUTEXES 1 55 | #define configQUEUE_REGISTRY_SIZE 20 56 | #define configUSE_APPLICATION_TASK_TAG 1 57 | #define configUSE_COUNTING_SEMAPHORES 1 58 | #define configUSE_ALTERNATIVE_API 0 59 | #define configUSE_QUEUE_SETS 1 60 | #define configUSE_TASK_NOTIFICATIONS 1 61 | #define configSUPPORT_STATIC_ALLOCATION 1 62 | 63 | /* Software timer related configuration options. The maximum possible task 64 | * priority is configMAX_PRIORITIES - 1. The priority of the timer task is 65 | * deliberately set higher to ensure it is correctly capped back to 66 | * configMAX_PRIORITIES - 1. */ 67 | #define configUSE_TIMERS 1 68 | #define configTIMER_TASK_PRIORITY ( configMAX_PRIORITIES - 1 ) 69 | #define configTIMER_QUEUE_LENGTH 20 70 | #define configTIMER_TASK_STACK_DEPTH ( configMINIMAL_STACK_SIZE * 2 ) 71 | 72 | #define configMAX_PRIORITIES ( 7 ) 73 | 74 | /* Run time stats gathering configuration options. */ 75 | unsigned long ulGetRunTimeCounterValue( void ); /* Prototype of function that returns run time counter. */ 76 | void vConfigureTimerForRunTimeStats( void ); /* Prototype of function that initialises the run time counter. */ 77 | #define configGENERATE_RUN_TIME_STATS 1 78 | 79 | /* Co-routine related configuration options. */ 80 | #define configUSE_CO_ROUTINES 0 81 | #define configMAX_CO_ROUTINE_PRIORITIES ( 2 ) 82 | 83 | /* This demo can use of one or more example stats formatting functions. These 84 | * format the raw data provided by the uxTaskGetSystemState() function in to human 85 | * readable ASCII form. See the notes in the implementation of vTaskList() within 86 | * FreeRTOS/Source/tasks.c for limitations. */ 87 | #define configUSE_STATS_FORMATTING_FUNCTIONS 0 88 | 89 | /* Enables the test whereby a stack larger than the total heap size is 90 | * requested. */ 91 | #define configSTACK_DEPTH_TYPE uint32_t 92 | 93 | /* Set the following definitions to 1 to include the API function, or zero 94 | * to exclude the API function. In most cases the linker will remove unused 95 | * functions anyway. */ 96 | #define INCLUDE_vTaskPrioritySet 1 97 | #define INCLUDE_uxTaskPriorityGet 1 98 | #define INCLUDE_vTaskDelete 1 99 | #define INCLUDE_vTaskCleanUpResources 0 100 | #define INCLUDE_vTaskSuspend 1 101 | #define INCLUDE_vTaskDelayUntil 1 102 | #define INCLUDE_vTaskDelay 1 103 | #define INCLUDE_uxTaskGetStackHighWaterMark 1 104 | #define INCLUDE_uxTaskGetStackHighWaterMark2 1 105 | #define INCLUDE_xTaskGetSchedulerState 1 106 | #define INCLUDE_xTimerGetTimerDaemonTaskHandle 1 107 | #define INCLUDE_xTaskGetIdleTaskHandle 1 108 | #define INCLUDE_xTaskGetHandle 1 109 | #define INCLUDE_eTaskGetState 1 110 | #define INCLUDE_xSemaphoreGetMutexHolder 1 111 | #define INCLUDE_xTimerPendFunctionCall 1 112 | #define INCLUDE_xTaskAbortDelay 1 113 | 114 | #define configINCLUDE_MESSAGE_BUFFER_AMP_DEMO 0 115 | #if ( configINCLUDE_MESSAGE_BUFFER_AMP_DEMO == 1 ) 116 | extern void vGenerateCoreBInterrupt( void * xUpdatedMessageBuffer ); 117 | #define sbSEND_COMPLETED( pxStreamBuffer ) vGenerateCoreBInterrupt( pxStreamBuffer ) 118 | #endif /* configINCLUDE_MESSAGE_BUFFER_AMP_DEMO */ 119 | 120 | extern void vAssertCalled( const char * const pcFileName, 121 | unsigned long ulLine ); 122 | 123 | 124 | 125 | /* networking definitions */ 126 | #define configMAC_ISR_SIMULATOR_PRIORITY ( configMAX_PRIORITIES - 1 ) 127 | 128 | /* Prototype for the function used to print out. In this case it prints to the 129 | * console before the network is connected then a UDP port after the network has 130 | * connected. */ 131 | extern void vLoggingPrintf( const char * pcFormatString, 132 | ... ); 133 | 134 | /* Set to 1 to print out debug messages. If ipconfigHAS_DEBUG_PRINTF is set to 135 | * 1 then FreeRTOS_debug_printf should be defined to the function used to print 136 | * out the debugging messages. */ 137 | #define ipconfigHAS_DEBUG_PRINTF 1 138 | #if ( ipconfigHAS_DEBUG_PRINTF == 1 ) 139 | #define FreeRTOS_debug_printf( X ) vLoggingPrintf X 140 | #endif 141 | 142 | /* Set to 1 to print out non debugging messages, for example the output of the 143 | * FreeRTOS_netstat() command, and ping replies. If ipconfigHAS_PRINTF is set to 1 144 | * then FreeRTOS_printf should be set to the function used to print out the 145 | * messages. */ 146 | #define ipconfigHAS_PRINTF 0 147 | #if ( ipconfigHAS_PRINTF == 1 ) 148 | #define FreeRTOS_printf( X ) vLoggingPrintf X 149 | #endif 150 | #endif /* FREERTOS_CONFIG_H */ 151 | -------------------------------------------------------------------------------- /docs/src/tutorial/Makefile: -------------------------------------------------------------------------------- 1 | HARNESS_ENTRY = harness 2 | HARNESS_FILE = pvPortMalloc_harness 3 | 4 | # This should be a unique identifier for this proof, and will appear on the 5 | # Litani dashboard. It can be human-readable and contain spaces if you wish. 6 | PROOF_UID = pvPortMalloc 7 | 8 | DEFINES += 9 | INCLUDES += -I$(PROOFDIR) 10 | INCLUDES += -I$(SRCDIR)/include 11 | INCLUDES += -I$(SRCDIR)/portable/ThirdParty/GCC/Posix 12 | 13 | REMOVE_FUNCTION_BODY += 14 | UNWINDSET += 15 | 16 | PROOF_SOURCES += $(PROOFDIR)/$(HARNESS_FILE).c 17 | PROJECT_SOURCES += $(SRCDIR)/portable/MemMang/heap_5.c 18 | 19 | # If this proof is found to consume huge amounts of RAM, you can set the 20 | # EXPENSIVE variable. With new enough versions of the proof tools, this will 21 | # restrict the number of EXPENSIVE CBMC jobs running at once. See the 22 | # documentation in Makefile.common under the "Job Pools" heading for details. 23 | # EXPENSIVE = true 24 | 25 | include ../Makefile.common 26 | -------------------------------------------------------------------------------- /docs/src/tutorial/README.md: -------------------------------------------------------------------------------- 1 | # CBMC starter kit tutorial 2 | 3 | The [CBMC starter kit](https://github.com/model-checking/cbmc-starter-kit) 4 | makes it easy to add CBMC verification to an existing software project. 5 | 6 | In this tutorial, we show how to begin proving the memory safety of 7 | a memory allocator that comes with the 8 | [FreeRTOS Kernel](https://github.com/FreeRTOS/FreeRTOS-Kernel). 9 | The kernel comes with five allocators, and we look at the simplest one. 10 | It allocates blocks from a region of memory set aside for the heap. 11 | It maintains a linked list of free blocks and allocates a block from 12 | the first block in the free list that is big enough to satisfy the request. 13 | When the block is freed, it is added back to the free list and merged with 14 | adjacent free blocks already in the free list. 15 | The function we want to prove memory safe is the allocator 16 | [`pvPortMalloc`](https://github.com/FreeRTOS/FreeRTOS-Kernel/blob/main/portable/MemMang/heap_5.c#L155) 17 | in the source file 18 | [portable/MemMang/heap_5.c](https://github.com/FreeRTOS/FreeRTOS-Kernel/blob/main/portable/MemMang/heap_5.c). 19 | 20 | This tutorial is actually a slightly cleaned-up version of the work done by two 21 | developers who used the starter kit to begin verification of `pvPortMalloc`. 22 | Two developers who had little more hand-on experience that the demonstration of 23 | CBMC running on a few simple examples were able to use the starter kit 24 | to begin verification in about ten or fifteen minutes, and were able to 25 | breathe some real life into the proof within a few more hours. 26 | 27 | Using the starter kit consists of five steps 28 | * [Clone the source repository](#clone-the-source-repository) 29 | * [Configure the repository](#configure-the-repository) 30 | * [Configure the proof](#configure-the-proof) 31 | * [Write the proof](#write-the-proof) 32 | * [Run the proof](#run-the-proof) 33 | 34 | ## Clone the source repository 35 | 36 | The first step is to clone the FreeRTOS Kernel repository. 37 | ``` 38 | git clone https://github.com/FreeRTOS/FreeRTOS-Kernel.git kernel 39 | cd kernel 40 | git submodule update --init --checkout --recursive 41 | ``` 42 | The first line clones the repository into the directory `kernel`. 43 | The remaining lines step into the directory `kernel` 44 | and clone the kernel's submodules. 45 | 46 | ## Configure the repository 47 | 48 | The next step is to configure the repository for CBMC verification. 49 | ``` 50 | mkdir cbmc 51 | cd cbmc 52 | cbmc-starter-kit-setup 53 | ``` 54 | ``` 55 | What is the project name? Kernel 56 | ``` 57 | The first line create a `cbmc` directory to hold everything 58 | related to CBMC verification. The last line runs a setup script 59 | to configure the repository for CBMC verification. 60 | It examines the layout of the repository and asks for a name to use for 61 | the CBMC verification project. We use the project name `Kernel`. 62 | 63 | Looking at the `cbmc` directory, we see that some infrastructure has 64 | been installed: 65 | ``` 66 | ls 67 | ``` 68 | ``` 69 | include proofs sources stubs 70 | ``` 71 | We see directories for holding header files, source files, and stubs written 72 | for the verification work. Examples of useful stubs for a verification project 73 | are `send` and `receive` methods for a physical communication network that 74 | isn't being explicitly modeled. 75 | 76 | The most important directory here is the `proofs` directory that will hold 77 | the CBMC proofs themselves: 78 | ``` 79 | ls proofs 80 | ``` 81 | ``` 82 | Makefile-project-defines Makefile.common 83 | Makefile-project-targets README.md 84 | Makefile-project-testing run-cbmc-proofs.py 85 | Makefile-template-defines 86 | ``` 87 | The most important files here are 88 | * `Makefile.common` This makefile 89 | implements our best practices for CBMC verification: 90 | our best practices for building code for CBMC, our best practices 91 | for running CBMC, and for building a report of CBMC results in a form 92 | that makes it easy to debug issues found by CBMC. 93 | * `run-cbmc-proofs.py` This python script runs all of the CBMC proofs in 94 | the `proofs` directory with maximal concurrency, 95 | and builds a dashboard of the results. 96 | This script is invoked by continuous integration to recheck the proofs on 97 | changes proposed in a pull request. 98 | 99 | The remaining Makefiles are just hooks for describing project-specific 100 | modifications or definitions. For example, within `Makefile-project-defines` 101 | you can define the `INCLUDES` variable to set the search path for the header 102 | files needed to build the project functions being verified. 103 | 104 | ## Configure the proof 105 | 106 | The next step is to configure CBMC verification of the memory allocator 107 | [`pvPortMalloc`](https://github.com/FreeRTOS/FreeRTOS-Kernel/blob/main/portable/MemMang/heap_5.c#L155) 108 | in the source file 109 | [portable/MemMang/heap_5.c](https://github.com/FreeRTOS/FreeRTOS-Kernel/blob/main/portable/MemMang/heap_5.c). 110 | ``` 111 | cd proofs 112 | cbmc-starter-kit-setup-proof 113 | ``` 114 | ``` 115 | What is the function name? pvPortMalloc 116 | These source files define a function 'pvPortMalloc': 117 | 0 ../../portable/ARMv8M/secure/heap/secure_heap.c 118 | 1 ../../portable/GCC/ARM_CM23/secure/secure_heap.c 119 | 2 ../../portable/GCC/ARM_CM33/secure/secure_heap.c 120 | 3 ../../portable/IAR/ARM_CM23/secure/secure_heap.c 121 | 4 ../../portable/IAR/ARM_CM33/secure/secure_heap.c 122 | 5 ../../portable/MemMang/heap_1.c 123 | 6 ../../portable/MemMang/heap_2.c 124 | 7 ../../portable/MemMang/heap_3.c 125 | 8 ../../portable/MemMang/heap_4.c 126 | 9 ../../portable/MemMang/heap_5.c 127 | 10 ../../portable/WizC/PIC18/port.c 128 | 11 The source file is not listed here 129 | Select a source file (the options are 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11): 9 130 | ``` 131 | This runs a setup script for the proof that first asks for the name 132 | of the function being verified. We give it the name `pvPortMalloc`. 133 | It then examines the source code in the repository and lists all of 134 | the source files that define a function named `pvPortMalloc`. It lists 135 | these files and asks us to chose the file with the implementation we 136 | want to verify. We choose source file number 9. 137 | 138 | The `proofs` directory now contains a subdirectory `pvPortMalloc` 139 | for verification of the memory allocator `pvPortMalloc`. 140 | ``` 141 | cd pvPortMalloc 142 | ls 143 | ``` 144 | ``` 145 | Makefile cbmc-proof.txt pvPortMalloc_harness.c 146 | README.md cbmc-viewer.json 147 | ``` 148 | The most important files in this directory are 149 | * `Makefile` This is a skeleton of a Makefile to build and run the proof. 150 | * `pvPortMalloc_harness.c` This is a skeleton of a proof harness 151 | for `pvPortMalloc`. 152 | 153 | ## Write the proof 154 | 155 | ### The Makefile 156 | 157 | The Makefile is very simple. 158 | ``` 159 | HARNESS_ENTRY = harness 160 | HARNESS_FILE = pvPortMalloc_harness 161 | 162 | # This should be a unique identifier for this proof, and will appear on the 163 | # Litani dashboard. It can be human-readable and contain spaces if you wish. 164 | PROOF_UID = pvPortMalloc 165 | 166 | DEFINES += 167 | INCLUDES += 168 | 169 | REMOVE_FUNCTION_BODY += 170 | UNWINDSET += 171 | 172 | PROOF_SOURCES += $(PROOFDIR)/$(HARNESS_FILE).c 173 | PROJECT_SOURCES += $(SRCDIR)/portable/MemMang/heap_5.c 174 | 175 | # If this proof is found to consume huge amounts of RAM, you can set the 176 | # EXPENSIVE variable. With new enough versions of the proof tools, this will 177 | # restrict the number of EXPENSIVE CBMC jobs running at once. See the 178 | # documentation in Makefile.common under the "Job Pools" heading for details. 179 | # EXPENSIVE = true 180 | 181 | include ../Makefile.common 182 | ``` 183 | You can see that it identifies the the function `pvPortMalloc`, it 184 | identifies the source file `portable/MemMang/heap_5.c`, and it includes 185 | the `Makefile.common` describing our best practices for using CBMC. 186 | 187 | It also gives us the option of defining `INCLUDES` to set the include path 188 | for header files. We do need a few header files to build `pvPortMalloc`, 189 | so let us update the Makefile with 190 | ``` 191 | INCLUDES += -I$(PROOFDIR) 192 | INCLUDES += -I$(SRCDIR)/include 193 | INCLUDES += -I$(SRCDIR)/portable/ThirdParty/GCC/Posix 194 | ``` 195 | 196 | The final [`Makefile`](Makefile) is: 197 | ``` 198 | HARNESS_ENTRY = harness 199 | HARNESS_FILE = pvPortMalloc_harness 200 | 201 | # This should be a unique identifier for this proof, and will appear on the 202 | # Litani dashboard. It can be human-readable and contain spaces if you wish. 203 | PROOF_UID = pvPortMalloc 204 | 205 | DEFINES += 206 | INCLUDES += -I$(PROOFDIR) 207 | INCLUDES += -I$(SRCDIR)/include 208 | INCLUDES += -I$(SRCDIR)/portable/ThirdParty/GCC/Posix 209 | 210 | REMOVE_FUNCTION_BODY += 211 | UNWINDSET += 212 | 213 | PROOF_SOURCES += $(PROOFDIR)/$(HARNESS_FILE).c 214 | PROJECT_SOURCES += $(SRCDIR)/portable/MemMang/heap_5.c 215 | 216 | # If this proof is found to consume huge amounts of RAM, you can set the 217 | # EXPENSIVE variable. With new enough versions of the proof tools, this will 218 | # restrict the number of EXPENSIVE CBMC jobs running at once. See the 219 | # documentation in Makefile.common under the "Job Pools" heading for details. 220 | # EXPENSIVE = true 221 | 222 | include ../Makefile.common 223 | ``` 224 | 225 | ### The proof harness 226 | 227 | The proof harness is also very simple (especially after omitting some 228 | comments at the top of the file). 229 | ``` 230 | /** 231 | * @file pvPortMalloc_harness.c 232 | * @brief Implements the proof harness for pvPortMalloc function. 233 | */ 234 | 235 | void harness() 236 | { 237 | 238 | /* Insert argument declarations */ 239 | 240 | pvPortMalloc( /* Insert arguments */ ); 241 | } 242 | ``` 243 | 244 | We need to add the main header file `FreeRTOS.h` for the FreeRTOS project, 245 | and we need to add a function prototype for `pvPortMalloc` saying that it 246 | takes a size and returns a pointer. 247 | ``` 248 | #include 249 | #include "FreeRTOS.h" 250 | void *pvPortMalloc(size_t size); 251 | ``` 252 | 253 | All that is left is to declare an unconstrained size of type `size_t` and 254 | pass it to `pvPortMalloc.` 255 | ``` 256 | size_t size; 257 | pvPortMalloc(size); 258 | ``` 259 | 260 | The final [`pvPortMalloc_harness.c`](pvPortMalloc_harness.c) is: 261 | ``` 262 | /** 263 | * @file pvPortMalloc_harness.c 264 | * @brief Implements the proof harness for pvPortMalloc function. 265 | */ 266 | 267 | #include 268 | #include "FreeRTOS.h" 269 | void *pvPortMalloc(size_t size); 270 | 271 | void harness() 272 | { 273 | size_t size; 274 | pvPortMalloc(size); 275 | } 276 | ``` 277 | 278 | And that is it, with the exception of one last detail. Building FreeRTOS 279 | requires a configuration file that sets all the parameters used to define 280 | a system configuration. This kernel repository does not contain a 281 | configuration file, so let us use a simplified 282 | configuration file from a demonstration in another repository. Let us add 283 | [`FreeRTOSConfig.h`](FreeRTOSConfig.h) to the `pvPortMalloc` directory. 284 | 285 | ## Run the proof 286 | 287 | Finally, we can run the proof: 288 | ``` 289 | make 290 | ``` 291 | This builds a report of the results that we can open in a browser 292 | ``` 293 | open report/html/index.html 294 | ``` 295 | Examining [the report](report/html/index.html), we see a list of coverage results, a list of 296 | warnings, and a list of errors or issues found by CBMC. In this report, 297 | there are no errors, but the coverage is *terrible*: only 40% of the lines 298 | in the function are exercised by CBMC! 299 | 300 | Further thought makes it clear that we haven't set up the heap 301 | that the allocator is supposed to be using. We have invoked 302 | an allocator to allocate space on a heap, but we haven't allocated 303 | or initialized the heap itself yet! 304 | 305 | At this point it is interesting to see what the developers 306 | did to breathe some life into this verification effort. 307 | Their proof harness looked something like this 308 | [`pvPortMalloc_harness.c`](pvPortMalloc_harness1.c): 309 | ``` 310 | /** 311 | * @file pvPortMalloc_harness.c 312 | * @brief Implements the proof harness for pvPortMalloc function. 313 | */ 314 | 315 | #include 316 | #include "FreeRTOS.h" 317 | void * pvPortMalloc( size_t xWantedSize ); 318 | 319 | void harness() 320 | { 321 | /* allocate heap */ 322 | uint8_t app_heap[ configTOTAL_HEAP_SIZE ]; 323 | 324 | /* initialize heap */ 325 | HeapRegion_t xHeapRegions[] = 326 | { 327 | { ( unsigned char * ) app_heap, sizeof( app_heap ) }, 328 | { NULL, 0 } 329 | }; 330 | vPortDefineHeapRegions( xHeapRegions ); 331 | 332 | /* mess up heap */ 333 | size_t xWantedSize1, xWantedSize2, xWantedSize3; 334 | void* pv1 = pvPortMalloc( xWantedSize1 ); 335 | void* pv2 = pvPortMalloc( xWantedSize2 ); 336 | void* pv3 = pvPortMalloc( xWantedSize3 ); 337 | vPortFree( pv2 ); 338 | 339 | size_t xWantedSize; 340 | pvPortMalloc( xWantedSize ); 341 | } 342 | ``` 343 | You can see that they allocate the heap, they initialize the heap 344 | data structures, and then they mess up the heap a bit by allocating 345 | three chunks of unconstrained size and freeing the middle one. 346 | Then they invoke `pvPortMalloc` with an unconstrained size. 347 | Doing this was enough to get complete code coverage and uncover a 348 | few minor instances of integer overflow. 349 | 350 | Of course, this is not a complete proof of memory safety. This is a proof that 351 | if you allocate a heap consisting of a single chunk of memory of a size 352 | fixed by the configuration, and if you allocate three chunks of unconstrained 353 | size and free the middle one, then `pvPortMalloc` will exhibit no memory 354 | safety errors or other undefined behaviors. But it is an elegant example 355 | of how quickly developers were able to get started doing real work. 356 | Good for them! And, soon, good for you. 357 | -------------------------------------------------------------------------------- /docs/src/tutorial/pvPortMalloc_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file pvPortMalloc_harness.c 3 | * @brief Implements the proof harness for pvPortMalloc function. 4 | */ 5 | 6 | #include 7 | #include "FreeRTOS.h" 8 | void *pvPortMalloc(size_t size); 9 | 10 | void harness() 11 | { 12 | size_t size; 13 | pvPortMalloc(size); 14 | } 15 | -------------------------------------------------------------------------------- /docs/src/tutorial/pvPortMalloc_harness1.c: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | /* 5 | * Insert copyright notice 6 | */ 7 | 8 | /** 9 | * @file pvPortMalloc_harness.c 10 | * @brief Implements the proof harness for pvPortMalloc function. 11 | */ 12 | 13 | /* 14 | * Insert project header files that 15 | * - include the declaration of the function 16 | * - include the types needed to declare function arguments 17 | */ 18 | 19 | #include 20 | #include "FreeRTOS.h" 21 | 22 | void * pvPortMalloc( size_t xWantedSize ); 23 | 24 | void harness() 25 | { 26 | /* allocate heap */ 27 | uint8_t app_heap[ configTOTAL_HEAP_SIZE ]; 28 | 29 | /* initialize heap */ 30 | HeapRegion_t xHeapRegions[] = 31 | { 32 | { ( unsigned char * ) app_heap, sizeof( app_heap ) }, 33 | { NULL, 0 } 34 | }; 35 | vPortDefineHeapRegions( xHeapRegions ); 36 | 37 | /* mess up heap */ 38 | size_t xWantedSize1, xWantedSize2, xWantedSize3; 39 | void* pv1 = pvPortMalloc( xWantedSize1 ); 40 | void* pv2 = pvPortMalloc( xWantedSize2 ); 41 | void* pv3 = pvPortMalloc( xWantedSize3 ); 42 | vPortFree( pv2 ); 43 | 44 | size_t xWantedSize; 45 | pvPortMalloc( xWantedSize ); 46 | } 47 | -------------------------------------------------------------------------------- /docs/src/tutorial/report/html/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | CBMC 6 | 7 | 8 | 9 | 10 |

CBMC report

11 | 12 |
13 |

Coverage

14 | 15 | 16 | 17 | 18 |

19 | Coverage: 0.31 (reached 35 of 114 reachable lines) 20 |

21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
CoverageFunctionFile
0.81 (29/36)vPortDefineHeapRegionsportable/MemMang/heap_5.c
0.43 (6/14)harnesscbmc/proofs/pvPortMalloc/pvPortMalloc_harness.c
0.00 (0/17)prvInsertBlockIntoFreeListportable/MemMang/heap_5.c
0.00 (0/33)pvPortMallocportable/MemMang/heap_5.c
0.00 (0/14)vPortFreeportable/MemMang/heap_5.c
63 |
64 | 65 | 66 | 67 | 68 |
69 |

Warnings

70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | None 80 | 81 | 82 |
83 |

Errors

84 | 85 | 86 | 87 |
    88 |
  • Loop unwinding failures 89 | 100 |
    • 101 |
102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 |
111 | 112 | 113 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | # See https://packaging.python.org/en/latest/tutorials/packaging-projects 2 | 3 | [build-system] 4 | requires = [ 5 | "setuptools>=42", 6 | "wheel", 7 | "build" 8 | ] 9 | build-backend = "setuptools.build_meta" 10 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | # See https://packaging.python.org/en/latest/tutorials/packaging-projects 2 | 3 | [metadata] 4 | name = cbmc-starter-kit 5 | version = 2.11 6 | author = Mark R. Tuttle 7 | author_email = mrtuttle@amazon.com 8 | description = CBMC starter kit makes it easy to add CBMC verification to a software project 9 | long_description = file: README.md 10 | long_description_content_type = text/markdown 11 | url = https://github.com/model-checking/cbmc-starter-kit 12 | license = Apache License 2.0 13 | classifiers = 14 | Programming Language :: Python :: 3 15 | License :: OSI Approved :: Apache Software License 16 | Operating System :: OS Independent 17 | 18 | [options] 19 | package_dir = 20 | = src 21 | packages = find: 22 | include_package_data = True 23 | install_requires = 24 | gitpython 25 | setuptools 26 | python_requires = >=3.8 27 | 28 | [options.packages.find] 29 | where = src 30 | 31 | [options.package_data] 32 | # We want to two subtrees of data, but static configuration files do 33 | # not appear to give us an easy way to do this. 34 | cbmc_starter_kit = 35 | etc/* 36 | etc/bash_completion.d/* 37 | template-for-ci-workflow/* 38 | template-for-ci-workflow/*/* 39 | template-for-ci-workflow/*/*/* 40 | template-for-ci-workflow/*/*/*/* 41 | template-for-proof/* 42 | template-for-proof/*/* 43 | template-for-proof/*/*/* 44 | template-for-proof/*/*/*/* 45 | template-for-repository/* 46 | template-for-repository/*/* 47 | template-for-repository/*/*/* 48 | template-for-repository/*/*/*/* 49 | template-for-repository/.gitignore 50 | 51 | [options.entry_points] 52 | console_scripts = 53 | cbmc-starter-kit-setup = cbmc_starter_kit.setup:main 54 | cbmc-starter-kit-setup-proof = cbmc_starter_kit.setup_proof:main 55 | cbmc-starter-kit-setup-ci = cbmc_starter_kit.setup_ci:main 56 | cbmc-starter-kit-update = cbmc_starter_kit.update:main 57 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/Makefile: -------------------------------------------------------------------------------- 1 | default: 2 | @echo Nothing to do 3 | 4 | pylint: 5 | pylint \ 6 | --disable=missing-module-docstring \ 7 | --disable=missing-function-docstring \ 8 | --disable=duplicate-code \ 9 | --module-rgx '[\w-]+' \ 10 | *.py \ 11 | template-for-repository/proofs/*.py \ 12 | template-for-repository/proofs/lib/*.py 13 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/model-checking/cbmc-starter-kit/95804ece4ae1de908ced2b57d9dfb8e0b57214c2/src/cbmc_starter_kit/__init__.py -------------------------------------------------------------------------------- /src/cbmc_starter_kit/arguments.py: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | """Methods for common command-line argument parsing.""" 5 | 6 | import argparse 7 | import logging 8 | 9 | from cbmc_starter_kit import version as starter_kit_version 10 | 11 | def create_parser(options=None, description=None, epilog=None, formatter_class=argparse.HelpFormatter): 12 | """Create a parser for command line arguments.""" 13 | 14 | options = options or [] 15 | description = description or "" 16 | 17 | flags = [option.get('flag') for option in options] 18 | if '--verbose' not in flags: 19 | options.append({'flag': '--verbose', 'action': 'store_true', 'help': 'Verbose output'}) 20 | if '--debug' not in flags: 21 | options.append({'flag': '--debug', 'action': 'store_true', 'help': 'Debug output'}) 22 | if '--version' not in flags: 23 | options.append({'flag': '--version', 24 | 'action': 'version', 'version': starter_kit_version.version(), 25 | 'help': 'Display version and exit'}) 26 | 27 | parser = argparse.ArgumentParser( 28 | description=description, 29 | epilog=epilog, 30 | formatter_class=formatter_class) 31 | for option in options: 32 | flag = option.pop('flag') 33 | parser.add_argument(flag, **option) 34 | return parser 35 | 36 | def configure_logging(args): 37 | """Configure logging level based on command line arguments.""" 38 | 39 | # Logging is configured by first invocation of basicConfig 40 | fmt = '%(levelname)s: %(message)s' 41 | if args.debug: 42 | logging.basicConfig(level=logging.DEBUG, format=fmt) 43 | return 44 | if args.verbose: 45 | logging.basicConfig(level=logging.INFO, format=fmt) 46 | return 47 | logging.basicConfig(format=fmt) 48 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/ctagst.py: -------------------------------------------------------------------------------- 1 | """Ctags support for locating symbol definitions""" 2 | 3 | from pathlib import Path 4 | import json 5 | import logging 6 | import subprocess 7 | import sys 8 | 9 | ################################################################ 10 | # This popen method is used to subprocess-out the invocation of ctags. 11 | # This method duplicates code in other modules to make this ctags 12 | # module a stand-alone module that can be copied and reused in other 13 | # projects. 14 | 15 | def popen(cmd, cwd=None, stdin=None, encoding=None): 16 | """Run a command with string stdin on stdin, return stdout and stderr.""" 17 | 18 | cmd = [str(word) for word in cmd] 19 | kwds = {'cwd': cwd, 20 | 'text': True, 21 | 'stdin': subprocess.PIPE, 22 | 'stdout': subprocess.PIPE, 23 | 'stderr': subprocess.PIPE} 24 | if sys.version_info >= (3, 6): # encoding is new in Python 3.6 25 | kwds['encoding'] = encoding or 'utf-8' 26 | try: 27 | logging.debug('Popen command: "%s"', ' '.join(cmd)) 28 | logging.debug('Popen stdin: "%s"', stdin) 29 | with subprocess.Popen(cmd, **kwds) as pipe: 30 | stdout, stderr = pipe.communicate(input=stdin) 31 | logging.debug('Popen stdout: "%s"', stdout) 32 | logging.debug('Popen stderr: "%s"', stderr) 33 | if pipe.returncode: 34 | logging.debug('Popen command failed: "%s"', ' '.join(cmd)) 35 | logging.debug('Popen return code: "%s"', pipe.returncode) 36 | raise UserWarning(f"Failed to run command: {' '.join(cmd)}") 37 | return stdout, stderr 38 | except FileNotFoundError as error: 39 | logging.debug("FileNotFoundError: command '%s': %s", ' '.join(cmd), error) 40 | raise UserWarning(f"Failed to run command: {' '.join(cmd)}") from error 41 | 42 | ################################################################ 43 | 44 | def ctags(root, files): 45 | """List symbols defined in files under root.""" 46 | 47 | root = Path(root) 48 | files = [str(file_) for file_ in files] 49 | return (universal_ctags(root, files) or 50 | exhuberant_ctags(root, files) or 51 | legacy_ctags(root, files) or 52 | []) 53 | 54 | ################################################################ 55 | 56 | def universal_ctags(root, files): 57 | """Use universal ctags to list symbols defined in files under root.""" 58 | 59 | # See universal ctags man page at https://docs.ctags.io/en/latest/man/ctags.1.html 60 | cmd = [ 61 | 'ctags', 62 | '-L', '-', # read files from standard input, one file per line 63 | '-f', '-', # write tags to standard output, one tag per line 64 | '--output-format=json', # each tag is a one-line json blob 65 | '--fields=FNnK' # json blob is {"name": symbol, "path": file, "line": line, "kind": kind} 66 | ] 67 | try: 68 | logging.info("Running universal ctags") 69 | stdout, _ = popen(cmd, cwd=root, stdin='\n'.join(files)) 70 | strings = stdout.splitlines() 71 | except UserWarning: 72 | logging.info("Universal ctags failed") 73 | strings = [] 74 | 75 | return [tag for string in strings for tag in universal_tag(root, string)] 76 | 77 | def universal_tag(root, string): 78 | """Extract tag from universal ctag output.""" 79 | 80 | try: 81 | # universal ctag json output is '{"name": symbol, "path": file, "line": line, "kind": kind}' 82 | blob = json.loads(string) 83 | return [{'symbol': blob['name'], 'file': root/blob['path'], 'line': int(blob['line']), 84 | 'kind': blob['kind']}] 85 | except (json.decoder.JSONDecodeError, # json is unparsable 86 | KeyError, # json key is missing 87 | ValueError) as error: # invalid literal for int() 88 | logging.debug("Bad universal ctag: %s: %s", string, error) 89 | return [] 90 | 91 | ################################################################ 92 | 93 | def exhuberant_ctags(root, files): 94 | """Use exhuberant ctags to list symbols defined in files under root.""" 95 | 96 | # See exhuberant ctags man page at https://linux.die.net/man/1/ctags 97 | cmd = [ 98 | 'ctags', 99 | '-L', '-', # read files from standard input, one file per line 100 | '-f', '-', # write tags to standard output, one tag per line 101 | '-n', # use line numbers (not search expressions) to locate symbol in file 102 | '--fields=K' # include symbol kind among extension fields 103 | ] 104 | try: 105 | logging.info("Running exhuberant ctags") 106 | stdout, _ = popen(cmd, cwd=root, stdin='\n'.join(files)) 107 | strings = stdout.splitlines() 108 | except UserWarning: 109 | logging.info("Exhuberant ctags failed") 110 | strings = [] 111 | 112 | return [tag for string in strings for tag in exhuberant_tag(root, string)] 113 | 114 | def exhuberant_tag(root, string): 115 | """Extract tag from exhuberant ctag output.""" 116 | 117 | try: 118 | # exhuberant ctag output is 'symbolpathline;"kind' 119 | left, right = string.split(';"')[:2] 120 | symbol, path, line = left.split("\t")[:3] 121 | kind = right.split("\t")[1] 122 | return [{'symbol': symbol, 'file': root/path, 'line': int(line), 'kind': kind}] 123 | except (ValueError, IndexError): # not enough values to unpack, invalid literal for int() 124 | logging.debug('Bad exhuberant ctag: "%s"', string) 125 | return [] 126 | 127 | ################################################################ 128 | 129 | def legacy_ctags(root, files): 130 | """Use legacy ctags to list symbols defined in files under root.""" 131 | 132 | # MacOS ships with a legacy ctags from BSD installed in /usr/bin/ctags. 133 | # See the MacOS man page for the documentation used to implement this method. 134 | cmd = ['ctags', 135 | '-x', # write human-readable summary to standard output 136 | *files # legacy ctags cannot read list of files from stdin 137 | ] 138 | try: 139 | logging.info("Running legacy ctags") 140 | stdout, _ = popen(cmd, cwd=root) 141 | strings = stdout.splitlines() 142 | except UserWarning: 143 | logging.info("Legacy ctags failed") 144 | strings = [] 145 | return [tag for string in strings for tag in legacy_tag(root, string)] 146 | 147 | def legacy_tag(root, string): 148 | """Extract tag from legacy ctag output.""" 149 | 150 | try: 151 | # legacy ctag -x output is 'symbol line path source_code_fragment' 152 | symbol, line, path = string.split()[:3] 153 | return [{'symbol': symbol, 'file': root/path, 'line': int(line), 'kind': None}] 154 | except ValueError: # not enough values to unpack, invalid literal for int() 155 | logging.debug('Bad legacy ctag: "%s"', string) 156 | return [] 157 | 158 | ################################################################ 159 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/etc/bash_completion.d/cbmc-starter-kit.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | # SPDX-License-Identifier: MIT-0 5 | 6 | ################################################################ 7 | # Documentation 8 | # 9 | # compgen, complete: 10 | # https://www.gnu.org/software/bash/manual/html_node/Programmable-Completion-Builtins.html 11 | # _filedir: 12 | # https://github.com/nanoant/bash-completion-lib/blob/master/include/_filedir 13 | 14 | ################################################################ 15 | # Options 16 | # 17 | 18 | common_options="--help -h --verbose --debug --version" 19 | 20 | setup_options="" 21 | setup_proof_options="" 22 | update_options="--cbmc-root --starter-kit-root --no-test-removal --no-update" 23 | 24 | _core_autocomplete() 25 | { 26 | local options=$1 27 | local cur=${COMP_WORDS[COMP_CWORD]} 28 | local prev=${COMP_WORDS[COMP_CWORD-1]} 29 | 30 | case "$prev" in 31 | --cbmc-root|--starter-kit-root|--proofdir) 32 | _filedir -d 33 | return 0 34 | ;; 35 | esac 36 | 37 | # all remaining completions satisfy: "$cur" == -* 38 | COMPREPLY=( $( compgen -W "$options $common_options" -- $cur ) ) 39 | return 0 40 | } 41 | 42 | _setup_autocomplete() 43 | { 44 | _core_autocomplete "$setup_options" 45 | } 46 | 47 | _setup_proof_autocomplete() 48 | { 49 | _core_autocomplete "$setup_proof_options" 50 | } 51 | 52 | _update_autocomplete() 53 | { 54 | _core_autocomplete "$update_options" 55 | } 56 | 57 | complete -F _setup_autocomplete cbmc-starter-kit-setup 58 | complete -F _setup_proof_autocomplete cbmc-starter-kit-setup-proof 59 | complete -F _update_autocomplete cbmc-starter-kit-update 60 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/migrate_license.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | # SPDX-License-Identifier: MIT-0 5 | 6 | import logging 7 | import os 8 | import re 9 | import shutil 10 | import subprocess 11 | import sys 12 | 13 | from cbmc_starter_kit import arguments 14 | 15 | ################################################################ 16 | # Shell out commands 17 | 18 | def run(cmd, cwd=None, encoding=None): 19 | """Run a command in a subshell and return the standard output. 20 | 21 | Run the command cmd in the directory cwd and use encoding to 22 | decode the standard output. 23 | """ 24 | 25 | kwds = { 26 | 'cwd': cwd, 27 | 'stdout': subprocess.PIPE, 28 | 'stderr': subprocess.PIPE, 29 | 'text': True, 30 | } 31 | if sys.version_info >= (3, 6): # encoding introduced in Python 3.6 32 | kwds['encoding'] = encoding 33 | 34 | logging.debug('Running "%s" in %s', ' '.join(cmd), cwd) 35 | 36 | result = subprocess.run(cmd, **kwds, check=False) 37 | if result.returncode: 38 | logging.debug('Failed command: %s', ' '.join(cmd)) 39 | logging.debug('Failed return code: %s', result.returncode) 40 | logging.debug('Failed stdout: %s', result.stdout.strip()) 41 | logging.debug('Failed stderr: %s', result.stderr.strip()) 42 | return [] 43 | 44 | # Remove line continuations before splitting stdout into lines 45 | # Running command with text=True converts line endings to \n in stdout 46 | lines = result.stdout.replace('\\\n', ' ').splitlines() 47 | return [strip_whitespace(line) for line in lines] 48 | 49 | def strip_whitespace(string): 50 | return re.sub(r'\s+', ' ', string).strip() 51 | 52 | ################################################################ 53 | 54 | def maybe_a_copied_file(path): 55 | copied_files = [ 56 | 'Makefile', 57 | 'Makefile-project-defines', 58 | 'Makefile-project-targets', 59 | 'Makefile-project-testing', 60 | ] 61 | return os.path.basename(path) in copied_files 62 | 63 | def find_apache_references(proofdir=None): 64 | paths = run(['git', 'grep', '-l', 'Apache', '.'], cwd=proofdir) 65 | paths = [os.path.normpath(os.path.join(proofdir, path)) for path in paths] 66 | paths = [path for path in paths if not os.path.islink(path)] 67 | return sorted(paths) 68 | 69 | def remove_apache_reference(path, extension='backup'): 70 | apache_references = [ 71 | '# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.', 72 | '# SPDX-License-Identifier: Apache-2.0' 73 | ] 74 | backup = path + '.' + extension 75 | shutil.move(path, backup) 76 | with open(backup, encoding='utf-8') as infile, open(path, "w", encoding='utf-8') as outfile: 77 | removed = False 78 | for line in infile: 79 | if strip_whitespace(line) in apache_references: 80 | removed = True 81 | logging.debug('Deleted Apache reference in %s: %s', 82 | path, strip_whitespace(line)) 83 | continue 84 | outfile.write(line) 85 | if removed: 86 | logging.info('Deleted Apache reference in %s', path) 87 | return True 88 | return False 89 | 90 | def remove_apache_references(paths): 91 | removed = False 92 | for path in paths: 93 | if not maybe_a_copied_file(path): 94 | logging.debug('Skipping %s', path) 95 | continue 96 | logging.debug('Updating %s', path) 97 | removed = remove_apache_reference(path) or removed 98 | return removed 99 | 100 | ################################################################ 101 | 102 | def main(): 103 | 104 | desc = "Remove references to Apache license from CBMC starter kit." 105 | 106 | options = [ 107 | { 108 | "flag": "--proofdir", 109 | "help": "Root of the proof subtree (default: %(default)s)", 110 | "default": ".", 111 | }, 112 | { 113 | "flag": "--remove", 114 | "action": "store_true", 115 | "help": "Remove Apache references from files under PROOFDIR (otherwise just list them)" 116 | }, 117 | ] 118 | 119 | epilog = """ 120 | The CBMC starter kit was originally released under the Apache 121 | license. All files in the starter kit contained references to the 122 | Apache license. The starter kit installation scripts copied files 123 | from the starter kit into the project repository. This became an 124 | issue when the project repository was released under a different 125 | license. This script removes all references to the Apache license 126 | from the files copied into the project repository from the starter 127 | kit. 128 | """ 129 | 130 | args = arguments.create_parser( 131 | options=options, 132 | description=desc, 133 | epilog=epilog).parse_args() 134 | arguments.configure_logging(args) 135 | 136 | paths = find_apache_references(args.proofdir) 137 | 138 | if not args.remove: 139 | if paths: 140 | print("The following files contain references to the Apache license:") 141 | for path in paths: 142 | print(f" {path}") 143 | script = os.path.basename(sys.argv[0]) 144 | print(f"Remove Apache references from these files with '{script} --remove'") 145 | sys.exit(0) 146 | 147 | remove_apache_references(paths) 148 | 149 | paths = find_apache_references(args.proofdir) 150 | if paths: 151 | logging.warning("Files left unchanged contain Apache references: %s", ', '.join(paths)) 152 | sys.exit(1) 153 | 154 | sys.exit(0) 155 | 156 | if __name__ == "__main__": 157 | main() 158 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/repository.py: -------------------------------------------------------------------------------- 1 | """Discover repository properties like repository root, proof root, etc.""" 2 | 3 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | # SPDX-License-Identifier: MIT-0 5 | 6 | from pathlib import Path 7 | from subprocess import Popen, PIPE 8 | import logging 9 | 10 | import git 11 | 12 | from cbmc_starter_kit import ctagst 13 | 14 | ################################################################ 15 | # Construct an ascending relative path like "../../.." from a 16 | # directory to an ancestor in the file system. 17 | # 18 | # The Path method dst.relative_to(src) requires that dst is a 19 | # descendant of src and will not produce a path like "../../..". 20 | 21 | def path_to_ancestor(descendant, ancestor): 22 | """Relative path from descendant to ancestor.""" 23 | 24 | descendant = Path(descendant).resolve() 25 | ancestor = Path(ancestor).resolve() 26 | 27 | try: 28 | path = descendant.relative_to(ancestor) 29 | except ValueError: 30 | raise UserWarning(f"{ancestor} is not an ancestor of {descendant}") from ValueError 31 | 32 | return Path(*[Path('..') for part in path.parts]) 33 | 34 | ################################################################ 35 | # Discover the roots of 36 | # * the respository and 37 | # * the proofs subtree installed by the starter kit 38 | # * the directory containing GitHub Actions workflows 39 | 40 | def repository_root(cwd='.', abspath=True): 41 | """Path to root of repository containing current directory. 42 | 43 | Return the absolute path if abspath is True. If abspath is False, 44 | return the relative path from the current directory. If the 45 | current directory is within a submodule of the repository, then 46 | the root of the submodule is returned, not the repository itself. 47 | """ 48 | 49 | try: 50 | root = git.Repo(cwd, search_parent_directories=True).working_dir 51 | return Path(root) if abspath else path_to_ancestor(cwd, root) 52 | except git.InvalidGitRepositoryError: 53 | raise UserWarning(f"No git repository contains {cwd}") from None 54 | 55 | def proofs_root(cwd='.', abspath=True): 56 | """Path to root of proofs subtree installed by starter kit. 57 | 58 | Return an absolute path if abspath is True, and a path relative to 59 | cwd otherwise. Search starts at cwd and ascends until it reaches 60 | the root of the enclosing repository, and raises UserWarning if 61 | there is no enclosing repository.""" 62 | 63 | cwd = Path(cwd).resolve() # Where to start looking 64 | root = repository_root(cwd=cwd, abspath=True) # Where to stop looking 65 | proofs = "proofs" # What to look for 66 | 67 | for path in [cwd, *cwd.parents]: 68 | if path.name == proofs: 69 | return path if abspath else path_to_ancestor(cwd, path) 70 | if path == root: 71 | break 72 | raise UserWarning(f"'{cwd}' has no ancestor named '{proofs}'") 73 | 74 | def _get_directory_paths(start): 75 | """Generate path to subdirectory, only if it is not a git submodule 76 | 77 | This is used in identifying the CBMC proof root subdirectory.""" 78 | for path in start.iterdir(): 79 | if not path.is_dir() or git.repo.fun.is_git_dir(path): 80 | continue 81 | yield path 82 | 83 | def get_abspath_to_proofs_root(start): 84 | """Return absolute path to CBMC proof root.""" 85 | for path in _get_directory_paths(start): 86 | if path.name == "proofs": 87 | return path.absolute() 88 | proofs_dir = get_abspath_to_proofs_root(start=path) 89 | if proofs_dir: 90 | return proofs_dir 91 | 92 | def get_relative_path_from_repository_to_proofs_root(): 93 | """Return str version of absolute path to CBMC proof root. 94 | 95 | This function starts from the root of the repository and traverses all 96 | subdirectories.w""" 97 | repo_root = repository_root(abspath=True) 98 | return str(get_abspath_to_proofs_root(repo_root).relative_to(repo_root)) 99 | 100 | def github_actions_workflows_root(cwd='.', abspath=True): 101 | """Path to directory containing GitHub Actions workflows.""" 102 | 103 | return repository_root(cwd=cwd, abspath=abspath) / ".github" / "workflows" 104 | 105 | ################################################################ 106 | # Discover the set of all source files in the repository that define a 107 | # function named func. 108 | 109 | def run(cmd, cwd=None, stdin=None): 110 | """Run a command with string stdin on stdin, return stdout and stderr.""" 111 | 112 | cmd = [str(word) for word in cmd] 113 | try: 114 | with Popen(cmd, cwd=cwd, text=True, stdin=PIPE, stdout=PIPE, stderr=PIPE) as pipe: 115 | stdout, stderr = pipe.communicate(input=stdin) 116 | if pipe.returncode: 117 | logging.debug("Nonzero return code %s: command: '%s'", pipe.returncode, ' '.join(cmd)) 118 | logging.debug("Nonzero return code %s: stderr: '%s'", pipe.returncode, pipe.stderr) 119 | return None, None 120 | return stdout, stderr 121 | except FileNotFoundError: 122 | logging.debug("FileNotFoundError: command '%s'", ' '.join(cmd)) 123 | return None, None 124 | 125 | def function_tags(repo='.'): 126 | """List of tags for function definitions in respository source files. 127 | 128 | Each tag is a dict '{"name": function, "path": source}' naming a 129 | function and a source file defining the function.""" 130 | 131 | repo = Path(repo).resolve() 132 | 133 | find_cmd = ['find', '.', '-name', '*.c'] 134 | files, _ = run(find_cmd, cwd=repo) 135 | if files is None: # run() logs errors on debug 136 | return [] 137 | 138 | # legacy ctags does not give the kind of a symbol 139 | # assume a symbol is a function if the kind is None 140 | tags = ctagst.ctags(repo, files.split()) 141 | return [tag for tag in tags if tag['kind'] in ['function', None]] 142 | 143 | def function_paths(func, tags): 144 | """Paths to all source files in tags defining a function func.""" 145 | 146 | return sorted([tag['file'] for tag in tags if tag['symbol'] == func]) 147 | 148 | def function_sources(func, cwd='.', repo='.'): 149 | """Paths to all source files in the repository defining a function func. 150 | 151 | Paths are absolute if abspath is True, and relative to cwd otherwise. 152 | 153 | """ 154 | 155 | cwd = Path(cwd).resolve() 156 | repo = Path(repo).resolve() 157 | 158 | tags = function_tags(repo) 159 | sources = function_paths(func, tags) 160 | 161 | assert all(src.is_file() for src in sources) 162 | return sources 163 | 164 | ################################################################ 165 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | # SPDX-License-Identifier: MIT-0 5 | 6 | """Set up the CBMC proof instrastructure.""" 7 | 8 | from pathlib import Path 9 | import os 10 | import shutil 11 | import logging 12 | 13 | from cbmc_starter_kit import arguments 14 | from cbmc_starter_kit import repository 15 | from cbmc_starter_kit import update 16 | from cbmc_starter_kit import util 17 | 18 | ################################################################ 19 | 20 | def parse_arguments(): 21 | desc = "Set up CBMC proof infrastructure for a repository." 22 | options = [] 23 | args = arguments.create_parser( 24 | options=options, 25 | description=desc).parse_args() 26 | arguments.configure_logging(args) 27 | return args 28 | 29 | ################################################################ 30 | 31 | SRCDIR_TEXT = """ 32 | # Absolute path to the root of the source tree. 33 | # 34 | SRCDIR ?= {} 35 | """ 36 | 37 | LITANI_TEXT = """ 38 | # How to invoke litani. 39 | # Use "litani" when litani is present in PATH. 40 | # 41 | LITANI ?= {} 42 | """ 43 | 44 | PROJECT_TEXT = """ 45 | # Name of this proof project, displayed in proof reports. For example, 46 | # "s2n" or "Amazon FreeRTOS". For projects with multiple proof roots, 47 | # this may be overridden on the command-line to Make, for example 48 | # 49 | # make PROJECT_NAME="FreeRTOS MQTT" report 50 | # 51 | PROJECT_NAME = "{}" 52 | """ 53 | 54 | def srcdir_definition(source_root, proof_root): 55 | # Let makefile construct absolute path to source root 56 | srcdir_path = f"$(abspath $(PROOF_ROOT)/{os.path.relpath(source_root, proof_root)})" 57 | return SRCDIR_TEXT.format(srcdir_path) 58 | 59 | def litani_definition(litani, proof_root): 60 | if litani.is_file(): 61 | # Let makefile construct absolute path to litani 62 | litani_path = f"$(abspath $(PROOF_ROOT)/{os.path.relpath(litani, proof_root)})" 63 | return LITANI_TEXT.format(litani_path) 64 | return LITANI_TEXT.format(litani) 65 | 66 | def project_name_definition(project_name): 67 | return PROJECT_TEXT.format(project_name) 68 | 69 | ################################################################ 70 | 71 | def main(): 72 | """Set up the CBMC proof infrastructure.""" 73 | 74 | parse_arguments() # only arguments are --verbose and --debug 75 | 76 | # Gather project-specific definitions 77 | source_root = repository.repository_root() 78 | if shutil.which("litani"): 79 | litani = Path("litani") 80 | elif repository.litani_root() is not None: 81 | litani = repository.litani_root() / "litani" 82 | else: 83 | logging.error("Could not find litani root. See installation instructions at https://github.com/awslabs/aws-build-accumulator/releases/latest.") 84 | raise FileNotFoundError("litani") 85 | 86 | project_name = util.ask_for_project_name() 87 | 88 | # Copy cbmc infrastructure into cbmc directory 89 | cbmc_root = Path.cwd() 90 | shutil.copytree(util.package_repository_template_root(), cbmc_root, dirs_exist_ok=True) 91 | shutil.rmtree(cbmc_root / util.NEGATIVE_TESTS, ignore_errors=True) 92 | shutil.rmtree(cbmc_root / util.PROOF_DIR / "__pycache__", ignore_errors=True) 93 | 94 | # Overwrite Makefile.common and run-cbmc-proofs.py with versioned copies 95 | # Quiet warnings about overwriting files 96 | update.update(cbmc_root, quiet=True) 97 | 98 | # Write project-specific definitions to cbmc/proofs/Makefile-template-defines 99 | proof_root = cbmc_root / util.PROOF_DIR 100 | makefile = proof_root/util.TEMPLATE_DEFINES 101 | with open(makefile, "w", encoding='utf-8') as mkfile: 102 | print(srcdir_definition(source_root, proof_root), file=mkfile) 103 | print(litani_definition(litani, proof_root), file=mkfile) 104 | print(project_name_definition(project_name), file=mkfile) 105 | 106 | if __name__ == "__main__": 107 | main() 108 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/setup_ci.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | # SPDX-License-Identifier: MIT-0 5 | 6 | """Set up the AWS infrastructure to run proofs in CI.""" 7 | 8 | import logging 9 | import shutil 10 | from argparse import RawDescriptionHelpFormatter 11 | 12 | 13 | from cbmc_starter_kit import arguments, repository, util, version 14 | 15 | ################################################################ 16 | 17 | def parse_arguments(): 18 | """Parse arguments for cbmc-starter-kit-setup-ci command""" 19 | desc = """ 20 | Copy a GitHub Action workflow to `.github/workflows` in this repository, 21 | which runs CBMC proofs on every push event. 22 | """ 23 | eplg = """ 24 | The most recently released and available version of a tool (like CBMC, CBMC 25 | viewer, Litani, kissat, cadical) will be used by default. If the latest 26 | version of a tool does not work with some proofs, use one of the flags above 27 | to specify a particular version of the tool to pin to. 28 | 29 | Previous releases for each tool are listed here: 30 | - https://github.com/diffblue/cbmc/releases 31 | - https://github.com/model-checking/cbmc-viewer/releases 32 | - https://github.com/awslabs/aws-build-accumulator/releases 33 | - https://github.com/arminbiere/kissat/tags 34 | - https://github.com/arminbiere/cadical/tags 35 | """ 36 | options = [{ 37 | 'flag': '--github-actions-runner', 38 | 'metavar': '', 39 | 'default': 'ubuntu-20.04', 40 | 'help': """ 41 | The GitHub-hosted runner (operating on Ubuntu 20.04) that will 42 | run CBMC proofs in GitHub Actions. If your repo has a large 43 | runner available, you can specify it here. default: %(default)s""" 44 | }, { 45 | 'flag': '--cbmc', 46 | 'metavar': '', 47 | 'default': 'latest', 48 | 'help': 'Use this version of CBMC in CI. default: %(default)s' 49 | }, { 50 | 'flag': '--cbmc-viewer', 51 | 'metavar': '', 52 | 'default': 'latest', 53 | 'help': 'Use this version of CBMC viewer in CI. default: %(default)s' 54 | }, { 55 | 'flag': '--litani', 56 | 'metavar': '', 57 | 'default': 'latest', 58 | 'help': 'Use this version of Litani in CI. default: %(default)s' 59 | }, { 60 | 'flag': '--kissat', 61 | "metavar": '', 62 | 'default': 'latest', 63 | 'help': 'Use this tag of kissat in CI. default: %(default)s' 64 | }, { 65 | 'flag': '--cadical', 66 | "metavar": '', 67 | 'default': 'latest', 68 | 'help': 'Use this tag of cadical in CI. default: %(default)s' 69 | }] 70 | args = arguments.create_parser( 71 | options=options, 72 | description=desc, 73 | epilog=eplg, 74 | formatter_class=RawDescriptionHelpFormatter).parse_args() 75 | arguments.configure_logging(args) 76 | return args 77 | 78 | ################################################################ 79 | 80 | def _replace_placeholders_in_config_template(lines, replacements): 81 | """Returns a list of new lines, where some lines have had a placeholder 82 | value appropriately updated.""" 83 | buf = [] 84 | for line in lines: 85 | if not line or line[-1] != ">": 86 | buf.append(line) 87 | continue 88 | key_lower_kebab = line.split(":")[0] 89 | key_upper_snake = key_lower_kebab.replace('-', '_').upper() 90 | buf.append(f"{key_lower_kebab}: {replacements[key_upper_snake]}") 91 | return buf 92 | 93 | def patch_proof_ci_config(config, args): 94 | """Patch config for GitHub Actions workflow with appropriate values""" 95 | with open(config, encoding='utf-8') as data: 96 | lines = data.read().splitlines() 97 | proofs_dir = repository.get_relative_path_from_repository_to_proofs_root() 98 | replacements = { 99 | "CADICAL_TAG": args.cadical, 100 | "CBMC_VERSION": args.cbmc, 101 | "CBMC_VIEWER_VERSION": args.cbmc_viewer, 102 | "KISSAT_TAG": args.kissat, 103 | "LITANI_VERSION": args.litani, 104 | "PROOFS_DIR": proofs_dir, 105 | } 106 | new_lines = _replace_placeholders_in_config_template(lines, replacements) 107 | with open(config, "w", encoding='utf-8') as data: 108 | data.write('\n'.join(new_lines) + '\n') 109 | 110 | def patch_proof_ci_workflow(workflow, github_actions_runner): 111 | """Patch GitHub Actions workflow with appropriate GitHub-hosted runner""" 112 | with open(workflow, encoding='utf-8') as data: 113 | lines = data.read().splitlines() 114 | new_lines = [] 115 | for line in lines: 116 | if line.strip() == "runs-on: <__GITHUB_ACTIONS_RUNNER__>": 117 | new_line = line.replace( 118 | "<__GITHUB_ACTIONS_RUNNER__>", github_actions_runner) 119 | new_lines.append(new_line) 120 | continue 121 | new_lines.append(line) 122 | with open(workflow, "w", encoding='utf-8') as data: 123 | data.write('\n'.join(new_lines) + '\n') 124 | 125 | ################################################################ 126 | 127 | def copy_lib_directory(repo_root): 128 | """Will either add or update the "summarize" module located within the CBMC 129 | proof root's lib module""" 130 | logging.debug('Updating CBMC starter kit') 131 | src = util.package_repository_template_root() / util.PROOF_DIR / util.LIB_MODULE 132 | dst = repository.get_abspath_to_proofs_root(repo_root) / util.LIB_MODULE 133 | logging.info('Copying template lib module from %s to %s', src, dst) 134 | assert src.exists() 135 | shutil.copytree(src, dst, dirs_exist_ok=True) 136 | 137 | ################################################################ 138 | 139 | def main(): 140 | """ 141 | Creates a GitHub Actions workflow file based on the provided input. 142 | """ 143 | args = parse_arguments() 144 | 145 | workflows_root = repository.github_actions_workflows_root() 146 | config = workflows_root / "proof_ci_resources" / "config.yaml" 147 | workflow = workflows_root / "proof_ci.yaml" 148 | 149 | shutil.copytree( 150 | util.package_ci_workflow_template_root(), 151 | workflows_root, 152 | dirs_exist_ok=True) 153 | 154 | patch_proof_ci_config(config, args) 155 | patch_proof_ci_workflow(workflow, args.github_actions_runner) 156 | version.copy_with_version(workflow, workflow) 157 | 158 | repo_root = repository.repository_root() 159 | copy_lib_directory(repo_root) 160 | 161 | if __name__ == "__main__": 162 | main() 163 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/setup_proof.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | # SPDX-License-Identifier: MIT-0 5 | 6 | """Set up a CBMC proof.""" 7 | 8 | from pathlib import Path 9 | import os 10 | import shutil 11 | 12 | from cbmc_starter_kit import arguments 13 | from cbmc_starter_kit import repository 14 | from cbmc_starter_kit import util 15 | 16 | ################################################################ 17 | 18 | def parse_arguments(): 19 | desc = "Set up CBMC proof infrastructure for a proof." 20 | options = [] 21 | args = arguments.create_parser( 22 | options=options, 23 | description=desc).parse_args() 24 | arguments.configure_logging(args) 25 | return args 26 | 27 | ################################################################ 28 | 29 | def proof_template_filenames(): 30 | for path in util.package_proof_template_root().iterdir(): 31 | yield path 32 | 33 | def read_proof_template(path): 34 | with open(path, encoding='utf-8') as data: 35 | return data.read().splitlines() 36 | 37 | def write_proof_template(lines, filename, directory): 38 | with open(directory / filename, "w", encoding='utf-8') as data: 39 | data.write('\n'.join(lines) + '\n') 40 | 41 | def rename_proof_harness(function, directory): 42 | shutil.move(directory / "FUNCTION_harness.c", directory / f"{function}_harness.c") 43 | 44 | ################################################################ 45 | 46 | def patch_function_name(lines, function): 47 | return [line.replace("<__FUNCTION_NAME__>", function) for line in lines] 48 | 49 | def patch_path_to_makefile(lines, proof_root, proof_dir): 50 | path = os.path.relpath(proof_root, proof_dir) 51 | return [line.replace("<__PATH_TO_MAKEFILE__>", path) for line in lines] 52 | 53 | def patch_path_to_proof_root(lines, proof_root, source_root): 54 | path = os.path.relpath(proof_root, source_root) 55 | return [line.replace("<__PATH_TO_PROOF_ROOT__>", path) for line in lines] 56 | 57 | def patch_path_to_source_file(lines, source_file, source_root): 58 | path = os.path.relpath(source_file, source_root) 59 | return [line.replace("<__PATH_TO_SOURCE_FILE__>", path) for line in lines] 60 | 61 | ################################################################ 62 | 63 | def main(): 64 | """Set up CBMC proof.""" 65 | 66 | parse_arguments() # only arguments are --verbose and --debug 67 | 68 | function = util.ask_for_function_name() 69 | source_file = util.ask_for_source_file(function) 70 | source_root = repository.repository_root() 71 | proof_root = repository.proofs_root() 72 | 73 | 74 | proof_dir = Path(function) 75 | proof_dir.mkdir() 76 | 77 | for filename in proof_template_filenames(): 78 | lines = read_proof_template(filename) 79 | lines = patch_function_name(lines, function) 80 | lines = patch_path_to_makefile(lines, proof_root, proof_dir) 81 | lines = patch_path_to_proof_root(lines, proof_root, source_root) 82 | lines = patch_path_to_source_file(lines, source_file, source_root) 83 | write_proof_template(lines, filename.name, proof_dir) 84 | 85 | rename_proof_harness(function, proof_dir) 86 | 87 | if __name__ == "__main__": 88 | main() 89 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-ci-workflow/proof_ci.yaml: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | # _CBMC_STARTER_KIT_VERSION_ 4 | name: Run CBMC proofs 5 | on: 6 | push: 7 | branches-ignore: 8 | - gh-pages 9 | pull_request: 10 | branches-ignore: 11 | - gh-pages 12 | workflow_dispatch: 13 | 14 | # USAGE 15 | # 16 | # If you need to use different versions for tools like CBMC, modify this file: 17 | # .github/workflows/proof_ci_resources/config.yaml 18 | # 19 | # If you want the CI to use a different GitHub-hosted runner (which must still 20 | # be running Ubuntu 20.04), modify the value of this key: 21 | # jobs.run_cbmc_proofs.runs-on 22 | 23 | jobs: 24 | run_cbmc_proofs: 25 | runs-on: <__GITHUB_ACTIONS_RUNNER__> 26 | name: run_cbmc_proofs 27 | permissions: 28 | contents: read 29 | id-token: write 30 | pull-requests: read 31 | steps: 32 | - name: Check out repository and submodules recursively 33 | uses: actions/checkout@v3 34 | with: 35 | submodules: 'recursive' 36 | - name: Parse config file 37 | run: | 38 | CONFIG_FILE='.github/workflows/proof_ci_resources/config.yaml' 39 | for setting in cadical-tag cbmc-version cbmc-viewer-version kissat-tag litani-version proofs-dir run-cbmc-proofs-command; do 40 | VAR=$(echo $setting | tr "[:lower:]" "[:upper:]" | tr - _) 41 | echo "${VAR}"=$(yq .$setting $CONFIG_FILE) >> $GITHUB_ENV 42 | done 43 | - name: Ensure CBMC, CBMC viewer, Litani versions have been specified 44 | shell: bash 45 | run: | 46 | should_exit=false 47 | if [ "${{ env.CBMC_VERSION }}" == "" ]; then 48 | echo "You must specify a CBMC version (e.g. 'latest' or '5.70.0')" 49 | should_exit=true 50 | fi 51 | if [ "${{ env.CBMC_VIEWER_VERSION }}" == "" ]; then 52 | echo "You must specify a CBMC viewer version (e.g. 'latest' or '3.6')" 53 | should_exit=true 54 | fi 55 | if [ "${{ env.LITANI_VERSION }}" == "" ]; then 56 | echo "You must specify a Litani version (e.g. 'latest' or '1.27.0')" 57 | should_exit=true 58 | fi 59 | if [[ "$should_exit" == true ]]; then exit 1; fi 60 | - name: Install latest CBMC 61 | if: ${{ env.CBMC_VERSION == 'latest' }} 62 | shell: bash 63 | run: | 64 | # Search within 5 most recent releases for latest available package 65 | CBMC_REL="https://api.github.com/repos/diffblue/cbmc/releases?page=1&per_page=5" 66 | CBMC_DEB=$(curl -s $CBMC_REL | jq -r '.[].assets[].browser_download_url' | grep -e 'ubuntu-20.04' | head -n 1) 67 | CBMC_ARTIFACT_NAME=$(basename $CBMC_DEB) 68 | curl -o $CBMC_ARTIFACT_NAME -L $CBMC_DEB 69 | sudo dpkg -i $CBMC_ARTIFACT_NAME 70 | rm ./$CBMC_ARTIFACT_NAME 71 | - name: Install CBMC ${{ env.CBMC_VERSION }} 72 | if: ${{ env.CBMC_VERSION != 'latest' }} 73 | shell: bash 74 | run: | 75 | curl -o cbmc.deb -L \ 76 | https://github.com/diffblue/cbmc/releases/download/cbmc-${{ env.CBMC_VERSION }}/ubuntu-20.04-cbmc-${{ env.CBMC_VERSION }}-Linux.deb 77 | sudo dpkg -i ./cbmc.deb 78 | rm ./cbmc.deb 79 | - name: Install latest CBMC viewer 80 | if: ${{ env.CBMC_VIEWER_VERSION == 'latest' }} 81 | shell: bash 82 | run: | 83 | CBMC_VIEWER_REL="https://api.github.com/repos/model-checking/cbmc-viewer/releases/latest" 84 | CBMC_VIEWER_VERSION=$(curl -s $CBMC_VIEWER_REL | jq -r .name | sed 's/viewer-//') 85 | pip3 install cbmc-viewer==$CBMC_VIEWER_VERSION 86 | - name: Install CBMC viewer ${{ env.CBMC_VIEWER_VERSION }} 87 | if: ${{ env.CBMC_VIEWER_VERSION != 'latest' }} 88 | shell: bash 89 | run: | 90 | sudo apt-get update 91 | sudo apt-get install --no-install-recommends --yes \ 92 | build-essential universal-ctags 93 | pip3 install cbmc-viewer==${{ env.CBMC_VIEWER_VERSION }} 94 | - name: Install latest Litani 95 | if: ${{ env.LITANI_VERSION == 'latest' }} 96 | shell: bash 97 | run: | 98 | # Search within 5 most recent releases for latest available package 99 | LITANI_REL="https://api.github.com/repos/awslabs/aws-build-accumulator/releases?page=1&per_page=5" 100 | LITANI_DEB=$(curl -s $LITANI_REL | jq -r '.[].assets[0].browser_download_url' | head -n 1) 101 | DBN_PKG_FILENAME=$(basename $LITANI_DEB) 102 | curl -L $LITANI_DEB -o $DBN_PKG_FILENAME 103 | sudo apt-get update 104 | sudo apt-get install --no-install-recommends --yes ./$DBN_PKG_FILENAME 105 | rm ./$DBN_PKG_FILENAME 106 | - name: Install Litani ${{ env.LITANI_VERSION }} 107 | if: ${{ env.LITANI_VERSION != 'latest' }} 108 | shell: bash 109 | run: | 110 | curl -o litani.deb -L \ 111 | https://github.com/awslabs/aws-build-accumulator/releases/download/${{ env.LITANI_VERSION }}/litani-${{ env.LITANI_VERSION }}.deb 112 | sudo apt-get update 113 | sudo apt-get install --no-install-recommends --yes ./litani.deb 114 | rm ./litani.deb 115 | - name: Install ${{ env.KISSAT_TAG }} kissat 116 | if: ${{ env.KISSAT_TAG != '' }} 117 | shell: bash 118 | run: | 119 | if ${{ env.KISSAT_TAG == 'latest' }} 120 | then 121 | KISSAT_REL="https://api.github.com/repos/arminbiere/kissat/releases/latest" 122 | KISSAT_TAG_NAME=$(curl -s $KISSAT_REL | jq -r '.tag_name') 123 | else 124 | KISSAT_TAG_NAME=${{ env.KISSAT_TAG }} 125 | fi 126 | echo "Installing kissat $KISSAT_TAG_NAME" 127 | git clone https://github.com/arminbiere/kissat.git \ 128 | && cd kissat \ 129 | && git checkout $KISSAT_TAG_NAME \ 130 | && ./configure \ 131 | && cd build \ 132 | && make -j; 133 | echo "$(pwd)" >> $GITHUB_PATH 134 | - name: Install ${{ env.CADICAL_TAG }} cadical 135 | if: ${{ env.CADICAL_TAG != '' }} 136 | shell: bash 137 | run: | 138 | if ${{ env.CADICAL_TAG == 'latest' }} 139 | then 140 | CADICAL_REL="https://api.github.com/repos/arminbiere/cadical/releases/latest" 141 | CADICAL_TAG_NAME=$(curl -s $CADICAL_REL | jq -r '.tag_name') 142 | else 143 | CADICAL_TAG_NAME=${{ env.CADICAL_TAG }} 144 | fi 145 | echo "Installing cadical $CADICAL_TAG_NAME" 146 | git clone https://github.com/arminbiere/cadical.git \ 147 | && cd cadical \ 148 | && git checkout $CADICAL_TAG_NAME \ 149 | && ./configure \ 150 | && cd build \ 151 | && make -j; 152 | echo "$(pwd)" >> $GITHUB_PATH 153 | - name: Run CBMC proofs 154 | shell: bash 155 | env: 156 | EXTERNAL_SAT_SOLVER: kissat 157 | working-directory: ${{ env.PROOFS_DIR }} 158 | run: ${{ env.RUN_CBMC_PROOFS_COMMAND }} 159 | - name: Check repository visibility 160 | shell: bash 161 | run: | 162 | VIZ="${{ fromJson(toJson(github.event.repository)).visibility }}"; 163 | echo "REPO_VISIBILITY=${VIZ}" | tee -a "${GITHUB_ENV}"; 164 | - name: Set name for zip artifact with CBMC proof results 165 | id: artifact 166 | if: ${{ env.REPO_VISIBILITY == 'public' }} 167 | run: | 168 | echo "name=cbmc_proof_results_${{ fromJson(toJson(github.event.repository)).name }}_$(date +%Y_%m_%d_%H_%M_%S)" >> $GITHUB_OUTPUT 169 | - name: Create zip artifact with CBMC proof results 170 | if: ${{ env.REPO_VISIBILITY == 'public' }} 171 | shell: bash 172 | run: | 173 | FINAL_REPORT_DIR=$PROOFS_DIR/output/latest/html 174 | pushd $FINAL_REPORT_DIR \ 175 | && zip -r ${{ steps.artifact.outputs.name }}.zip . \ 176 | && popd \ 177 | && mv $FINAL_REPORT_DIR/${{ steps.artifact.outputs.name }}.zip . 178 | - name: Upload zip artifact of CBMC proof results to GitHub Actions 179 | if: ${{ env.REPO_VISIBILITY == 'public' }} 180 | uses: actions/upload-artifact@v3 181 | with: 182 | name: ${{ steps.artifact.outputs.name }} 183 | path: ${{ steps.artifact.outputs.name }}.zip 184 | - name: CBMC proof results 185 | shell: bash 186 | run: | 187 | python3 ${{ env.PROOFS_DIR }}/lib/summarize.py \ 188 | --run-file ${{ env.PROOFS_DIR }}/output/latest/html/run.json 189 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-ci-workflow/proof_ci_resources/config.yaml: -------------------------------------------------------------------------------- 1 | cadical-tag: <__CADICAL_TAG__> 2 | cbmc-version: <__CBMC_VERSION__> 3 | cbmc-viewer-version: <__CBMC_VIEWER_VERSION__> 4 | kissat-tag: <__KISSAT_TAG__> 5 | litani-version: <__LITANI_VERSION__> 6 | proofs-dir: <__PROOFS_DIR__> 7 | run-cbmc-proofs-command: ./run-cbmc-proofs.py 8 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-proof/FUNCTION_harness.c: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | /* 5 | * Insert copyright notice 6 | */ 7 | 8 | /** 9 | * @file <__FUNCTION_NAME__>_harness.c 10 | * @brief Implements the proof harness for <__FUNCTION_NAME__> function. 11 | */ 12 | 13 | /* 14 | * Insert project header files that 15 | * - include the declaration of the function 16 | * - include the types needed to declare function arguments 17 | */ 18 | 19 | /** 20 | * @brief Starting point for formal analysis 21 | * 22 | */ 23 | void harness(void) 24 | { 25 | 26 | /* Insert argument declarations */ 27 | 28 | <__FUNCTION_NAME__>( /* Insert arguments */ ); 29 | } 30 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-proof/Makefile: -------------------------------------------------------------------------------- 1 | HARNESS_ENTRY = harness 2 | HARNESS_FILE = <__FUNCTION_NAME__>_harness 3 | 4 | # This should be a unique identifier for this proof, and will appear on the 5 | # Litani dashboard. It can be human-readable and contain spaces if you wish. 6 | PROOF_UID = <__FUNCTION_NAME__> 7 | 8 | DEFINES += 9 | INCLUDES += 10 | 11 | REMOVE_FUNCTION_BODY += 12 | UNWINDSET += 13 | 14 | PROOF_SOURCES += $(PROOFDIR)/$(HARNESS_FILE).c 15 | PROJECT_SOURCES += $(SRCDIR)/<__PATH_TO_SOURCE_FILE__> 16 | 17 | # If this proof is found to consume huge amounts of RAM, you can set the 18 | # EXPENSIVE variable. With new enough versions of the proof tools, this will 19 | # restrict the number of EXPENSIVE CBMC jobs running at once. See the 20 | # documentation in Makefile.common under the "Job Pools" heading for details. 21 | # You may also choose to set ENABLE_POOLS right here, or else set it on the 22 | # command-line (see Makefile.common). 23 | # EXPENSIVE = true 24 | # ENABLE_POOLS = true 25 | 26 | # If you require access to a file-local ("static") function or object to conduct 27 | # your proof, set the following (and do not include the original source file 28 | # ("<__PATH_TO_SOURCE_FILE__>") in PROJECT_SOURCES). 29 | # REWRITTEN_SOURCES = $(PROOFDIR)/<__SOURCE_FILE_BASENAME__>.i 30 | # include <__PATH_TO_MAKEFILE__>/Makefile.common 31 | # $(PROOFDIR)/<__SOURCE_FILE_BASENAME__>.i_SOURCE = $(SRCDIR)/<__PATH_TO_SOURCE_FILE__> 32 | # $(PROOFDIR)/<__SOURCE_FILE_BASENAME__>.i_FUNCTIONS = foo bar 33 | # $(PROOFDIR)/<__SOURCE_FILE_BASENAME__>.i_OBJECTS = baz 34 | # Care is required with variables on the left-hand side: REWRITTEN_SOURCES must 35 | # be set before including Makefile.common, but any use of variables on the 36 | # left-hand side requires those variables to be defined. Hence, _SOURCE, 37 | # _FUNCTIONS, _OBJECTS is set after including Makefile.common. 38 | 39 | include <__PATH_TO_MAKEFILE__>/Makefile.common 40 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-proof/README.md: -------------------------------------------------------------------------------- 1 | <__FUNCTION_NAME__> proof 2 | ============== 3 | 4 | This directory contains a memory safety proof for <__FUNCTION_NAME__>. 5 | 6 | To run the proof. 7 | ------------- 8 | * Follow these [tool installation instructions](https://github.com/awslabs/aws-templates-for-cbmc-proofs/wiki/Installation) to install cbmc and cbmc-viewer. 9 | * Add `cbmc`, `goto-cc`, `goto-instrument`, `goto-analyzer`, and `cbmc-viewer` 10 | to your path. 11 | * Run `make`. 12 | * Open `report/html/index.html` in a web browser. 13 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-proof/cbmc-proof.txt: -------------------------------------------------------------------------------- 1 | # This file marks this directory as containing a CBMC proof. 2 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-proof/cbmc-viewer.json: -------------------------------------------------------------------------------- 1 | { "expected-missing-functions": 2 | [ 3 | 4 | ], 5 | "proof-name": "<__FUNCTION_NAME__>", 6 | "proof-root": "<__PATH_TO_PROOF_ROOT__>" 7 | } 8 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/.gitignore: -------------------------------------------------------------------------------- 1 | # Emitted when running CBMC proofs 2 | proofs/**/logs 3 | proofs/**/gotos 4 | proofs/**/report 5 | proofs/**/html 6 | proofs/output 7 | 8 | # Emitted by CBMC Viewer 9 | TAGS-* 10 | 11 | # Emitted by litani 12 | .ninja_deps 13 | .ninja_log 14 | .litani_cache_dir 15 | 16 | # These files should be overwritten whenever prepare.py runs 17 | cbmc-batch.yaml 18 | 19 | __pycache__/ 20 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/include/README.md: -------------------------------------------------------------------------------- 1 | CBMC proof include files 2 | ======================== 3 | 4 | This directory contains include files written for CBMC proof. It is 5 | common to write some code to model aspects of the system under test, 6 | and the header files for this code go here. 7 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/README.md: -------------------------------------------------------------------------------- 1 | ## Negative Tests 2 | 3 | This directory contains negative checks to ensure that CBMC CI jobs are run 4 | with the complete set of property-checking flags 5 | (see `CHECKFLAGS` in [Makefile.common](../proofs/Makefile.common)) 6 | which we consider to be part of the best practice. 7 | 8 | To enable these tests in CI jobs, 9 | copy this (`negative_tests`) directory into `../proofs`. 10 | 11 | If a property-checking flag is not used used by your project, 12 | you might want to disable the corresponding negative test. 13 | To do so, simply delete the particular test directory from `../proofs/negative_tests`. 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/assert/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = assert_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/assert/assert_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | /** 7 | * A basic negative assertion should fail 8 | * if CBMC was run at all. 9 | */ 10 | void assert_harness() { 11 | int lhs, rhs; 12 | assert(lhs == rhs); 13 | } 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/bounds_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = bounds_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/bounds_check/bounds_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | #include 7 | 8 | /** 9 | * A negative test for --bounds-check flag 10 | */ 11 | void bounds_check_harness() { 12 | char test[10]; 13 | size_t index; 14 | char ch; 15 | test[index] = ch; 16 | } 17 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/conversion_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = conversion_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/conversion_check/conversion_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | #include 7 | 8 | /** 9 | * A negative test for --conversion-check flag 10 | */ 11 | void conversion_check_harness() { 12 | uint64_t src; 13 | uint32_t dst = src; 14 | } 15 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/div_by_zero_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = div_by_zero_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/div_by_zero_check/div_by_zero_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | /** 7 | * A negative test for --div-by-zero-check flag 8 | */ 9 | void div_by_zero_check_harness() { 10 | int num, den; 11 | int div = num / den; 12 | } 13 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/float_overflow_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = float_overflow_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/float_overflow_check/float_overflow_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | /** 7 | * A negative test for --float-overflow-check flag 8 | */ 9 | void float_overflow_check_harness() { 10 | float overflow, increment; 11 | overflow += increment; 12 | } 13 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/float_underflow_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = float_underflow_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/float_underflow_check/float_underflow_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | /** 7 | * A negative test for --float-overflow-check flag 8 | */ 9 | void float_underflow_check_harness() { 10 | float underflow, increment; 11 | underflow -= increment; 12 | } 13 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/nan_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = nan_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/nan_check/nan_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | /** 7 | * A negative test for --nan-check flag 8 | */ 9 | void nan_check_harness() { 10 | float nan; 11 | nan = nan / nan; 12 | } 13 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/pointer_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = pointer_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/pointer_check/pointer_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | /** 7 | * A negative test for --pointer-check flag 8 | */ 9 | void pointer_check_harness() { 10 | int *src; 11 | int test = *src; 12 | } 13 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/pointer_overflow_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = pointer_overflow_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/pointer_overflow_check/pointer_overflow_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | #include 7 | 8 | /** 9 | * A negative test for --pointer-overflow-check flag 10 | */ 11 | void pointer_overflow_check_harness() { 12 | size_t offset; 13 | char *pointer; 14 | pointer += offset; 15 | } 16 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/pointer_primitive_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | CHECKFLAGS += --pointer-primitive-check 5 | 6 | EXPECTED = FAILED 7 | 8 | HARNESS_ENTRY = pointer_primitive_check_harness 9 | HARNESS_FILE = $(HARNESS_ENTRY).c 10 | 11 | PROOF_SOURCES += $(HARNESS_FILE) 12 | 13 | PROJECT_SOURCES += $(HARNESS_FILE) 14 | 15 | include ../Makefile.common 16 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/pointer_primitive_check/pointer_primitive_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | /** 7 | * A negative test for --pointer-primitive-check flag 8 | */ 9 | void pointer_primitive_check_harness() { 10 | char *pointer; 11 | assert(__CPROVER_r_ok(pointer, 10)); 12 | } 13 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/pointer_underflow_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = pointer_underflow_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/pointer_underflow_check/pointer_underflow_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | #include 7 | 8 | /** 9 | * A negative test for --pointer-overflow-check flag 10 | */ 11 | void pointer_underflow_check_harness() { 12 | size_t offset; 13 | char *pointer; 14 | pointer -= offset; 15 | } 16 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/signed_overflow_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = signed_overflow_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/signed_overflow_check/signed_overflow_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | /** 7 | * A negative test for --signed-overflow-check flag 8 | */ 9 | void signed_overflow_check_harness() { 10 | int overflow, offset; 11 | overflow += offset; 12 | } 13 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/signed_underflow_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = signed_underflow_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/signed_underflow_check/signed_underflow_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | /** 7 | * A negative test for --signed-overflow-check flag 8 | */ 9 | void signed_underflow_check_harness() { 10 | int underflow, offset; 11 | underflow -= offset; 12 | } 13 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/undefined_shift_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = undefined_shift_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/undefined_shift_check/undefined_shift_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | /** 7 | * A negative test for --undefined-shift-check flag 8 | */ 9 | void cbmc_ensure__undefined_shift_check_harness() { 10 | int base, shift; 11 | base <<= shift; 12 | base >>= shift; 13 | } 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/unsigned_overflow_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = unsigned_overflow_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/unsigned_overflow_check/unsigned_overflow_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | /** 7 | * A negative test for --unsigned-overflow-check flag 8 | */ 9 | void unsigned_overflow_check_harness() { 10 | unsigned overflow, offset; 11 | overflow += offset; 12 | } 13 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/unsigned_underflow_check/Makefile: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0. 3 | 4 | EXPECTED = FAILED 5 | 6 | HARNESS_ENTRY = unsigned_underflow_check_harness 7 | HARNESS_FILE = $(HARNESS_ENTRY).c 8 | 9 | PROOF_SOURCES += $(HARNESS_FILE) 10 | 11 | PROJECT_SOURCES += $(HARNESS_FILE) 12 | 13 | include ../Makefile.common 14 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/negative_tests/unsigned_underflow_check/unsigned_underflow_check_harness.c: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | * SPDX-License-Identifier: MIT-0. 4 | */ 5 | 6 | /** 7 | * A negative test for --unsigned-overflow-check flag 8 | */ 9 | void unsigned_underflow_check_harness() { 10 | unsigned underflow, offset; 11 | underflow -= offset; 12 | } 13 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/proofs/Makefile-project-defines: -------------------------------------------------------------------------------- 1 | # -*- mode: makefile -*- 2 | # The first line sets the emacs major mode to Makefile 3 | 4 | ################################################################ 5 | # Use this file to give project-specific definitions of the command 6 | # line arguments to pass to CBMC tools like goto-cc to build the goto 7 | # binaries and cbmc to do the property and coverage checking. 8 | # 9 | # Use this file to override most default definitions of variables in 10 | # Makefile.common. 11 | ################################################################ 12 | 13 | # Flags to pass to goto-cc for compilation (typically those passed to gcc -c) 14 | # COMPILE_FLAGS = 15 | 16 | # Flags to pass to goto-cc for linking (typically those passed to gcc) 17 | # LINK_FLAGS = 18 | 19 | # Flag to pass to goto-cc to make all static symbols globally visible. Leave it 20 | # unset or set it to --export-file-local-symbols to enable this behavior. Else, 21 | # selectively enable access to such symbols via each proof's Makefile. 22 | EXPORT_FILE_LOCAL_SYMBOLS = 23 | 24 | # Preprocessor include paths -I... 25 | # Consider adding 26 | # INCLUDES += -I$(CBMC_ROOT)/include 27 | # You will want to decide what order that comes in relative to the other 28 | # include directories in your project. 29 | # 30 | # INCLUDES = 31 | 32 | # Preprocessor definitions -D... 33 | # DEFINES = 34 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/proofs/Makefile-project-targets: -------------------------------------------------------------------------------- 1 | # -*- mode: makefile -*- 2 | # The first line sets the emacs major mode to Makefile 3 | 4 | ################################################################ 5 | # Use this file to give project-specific targets, including targets 6 | # that may depend on targets defined in Makefile.common. 7 | ################################################################ 8 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/proofs/Makefile-project-testing: -------------------------------------------------------------------------------- 1 | # -*- mode: makefile -*- 2 | # The first line sets the emacs major mode to Makefile 3 | 4 | ################################################################ 5 | # Use this file to define project-specific targets and definitions for 6 | # unit testing or continuous integration that may depend on targets 7 | # defined in Makefile.common 8 | ################################################################ 9 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/proofs/README.md: -------------------------------------------------------------------------------- 1 | CBMC proofs 2 | =========== 3 | 4 | This directory contains the CBMC proofs. Each proof is in its own 5 | directory. 6 | 7 | This directory includes four Makefiles. 8 | 9 | One Makefile describes the basic workflow for building and running proofs: 10 | 11 | * Makefile.common: 12 | * make goto: builds the goto binary 13 | * make result: does cbmc property checking 14 | * make coverage: does cbmc coverage checking 15 | * make report: builds the final report 16 | 17 | Running `make` or `make report` builds the final report. Running 18 | `make report-no-coverage` builds the final report but without checking 19 | coverage. Coverage checking can be slow, and it is usually not 20 | interesting until after the issues raised by property checking have 21 | been resolved. 22 | 23 | Three included Makefiles describe project-specific settings and can override 24 | definitions in Makefile.common: 25 | 26 | * Makefile-project-defines: definitions like compiler flags 27 | required to build the goto binaries, and definitions to override 28 | definitions in Makefile.common. 29 | * Makefile-project-targets: other make targets needed for the project 30 | * Makefile-project-testing: other definitions and targets needed for 31 | unit testing or continuous integration. 32 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/proofs/lib/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/model-checking/cbmc-starter-kit/95804ece4ae1de908ced2b57d9dfb8e0b57214c2/src/cbmc_starter_kit/template-for-repository/proofs/lib/__init__.py -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/proofs/lib/print_tool_versions.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # 3 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | # SPDX-License-Identifier: MIT-0 5 | 6 | 7 | import logging 8 | import pathlib 9 | import shutil 10 | import subprocess 11 | 12 | 13 | _TOOLS = [ 14 | "cadical", 15 | "cbmc", 16 | "cbmc-viewer", 17 | "cbmc-starter-kit-update", 18 | "kissat", 19 | "litani", 20 | ] 21 | 22 | 23 | def _format_versions(table): 24 | lines = [ 25 | "", 26 | '', 27 | ] 28 | for tool, version in table.items(): 29 | if version: 30 | v_str = f'
{version}
' 31 | else: 32 | v_str = 'not found' 33 | lines.append( 34 | f'' 36 | f'') 37 | lines.append("
Tool Versions
{tool}:{v_str}
") 38 | return "\n".join(lines) 39 | 40 | 41 | def _get_tool_versions(): 42 | ret = {} 43 | for tool in _TOOLS: 44 | err = f"Could not determine version of {tool}: " 45 | ret[tool] = None 46 | if not shutil.which(tool): 47 | logging.error("%s'%s' not found on $PATH", err, tool) 48 | continue 49 | cmd = [tool, "--version"] 50 | proc = subprocess.Popen(cmd, text=True, stdout=subprocess.PIPE) 51 | try: 52 | out, _ = proc.communicate(timeout=10) 53 | except subprocess.TimeoutExpired: 54 | logging.error("%s'%s --version' timed out", err, tool) 55 | continue 56 | if proc.returncode: 57 | logging.error( 58 | "%s'%s --version' returned %s", err, tool, str(proc.returncode)) 59 | continue 60 | ret[tool] = out.strip() 61 | return ret 62 | 63 | 64 | def main(): 65 | exe_name = pathlib.Path(__file__).name 66 | logging.basicConfig(format=f"{exe_name}: %(message)s") 67 | 68 | table = _get_tool_versions() 69 | out = _format_versions(table) 70 | print(out) 71 | 72 | 73 | if __name__ == "__main__": 74 | main() 75 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/proofs/lib/summarize.py: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import argparse 5 | import json 6 | import logging 7 | import os 8 | import sys 9 | 10 | 11 | DESCRIPTION = """Print 2 tables in GitHub-flavored Markdown that summarize 12 | an execution of CBMC proofs.""" 13 | 14 | 15 | def get_args(): 16 | """Parse arguments for summarize script.""" 17 | parser = argparse.ArgumentParser(description=DESCRIPTION) 18 | for arg in [{ 19 | "flags": ["--run-file"], 20 | "help": "path to the Litani run.json file", 21 | "required": True, 22 | }]: 23 | flags = arg.pop("flags") 24 | parser.add_argument(*flags, **arg) 25 | return parser.parse_args() 26 | 27 | 28 | def _get_max_length_per_column_list(data): 29 | ret = [len(item) + 1 for item in data[0]] 30 | for row in data[1:]: 31 | for idx, item in enumerate(row): 32 | ret[idx] = max(ret[idx], len(item) + 1) 33 | return ret 34 | 35 | 36 | def _get_table_header_separator(max_length_per_column_list): 37 | line_sep = "" 38 | for max_length_of_word_in_col in max_length_per_column_list: 39 | line_sep += "|" + "-" * (max_length_of_word_in_col + 1) 40 | line_sep += "|\n" 41 | return line_sep 42 | 43 | 44 | def _get_entries(max_length_per_column_list, row_data): 45 | entries = [] 46 | for row in row_data: 47 | entry = "" 48 | for idx, word in enumerate(row): 49 | max_length_of_word_in_col = max_length_per_column_list[idx] 50 | space_formatted_word = (max_length_of_word_in_col - len(word)) * " " 51 | entry += "| " + word + space_formatted_word 52 | entry += "|\n" 53 | entries.append(entry) 54 | return entries 55 | 56 | 57 | def _get_rendered_table(data): 58 | table = [] 59 | max_length_per_column_list = _get_max_length_per_column_list(data) 60 | entries = _get_entries(max_length_per_column_list, data) 61 | for idx, entry in enumerate(entries): 62 | if idx == 1: 63 | line_sep = _get_table_header_separator(max_length_per_column_list) 64 | table.append(line_sep) 65 | table.append(entry) 66 | table.append("\n") 67 | return "".join(table) 68 | 69 | 70 | def _get_status_and_proof_summaries(run_dict): 71 | """Parse a dict representing a Litani run and create lists summarizing the 72 | proof results. 73 | 74 | Parameters 75 | ---------- 76 | run_dict 77 | A dictionary representing a Litani run. 78 | 79 | 80 | Returns 81 | ------- 82 | A list of 2 lists. 83 | The first sub-list maps a status to the number of proofs with that status. 84 | The second sub-list maps each proof to its status. 85 | """ 86 | count_statuses = {} 87 | proofs = [["Proof", "Status"]] 88 | for proof_pipeline in run_dict["pipelines"]: 89 | if proof_pipeline["name"] == "print_tool_versions": 90 | continue 91 | status_pretty_name = proof_pipeline["status"].title().replace("_", " ") 92 | try: 93 | count_statuses[status_pretty_name] += 1 94 | except KeyError: 95 | count_statuses[status_pretty_name] = 1 96 | proofs.append([proof_pipeline["name"], status_pretty_name]) 97 | statuses = [["Status", "Count"]] 98 | for status, count in count_statuses.items(): 99 | statuses.append([status, str(count)]) 100 | return [statuses, proofs] 101 | 102 | 103 | def print_proof_results(out_file): 104 | """ 105 | Print 2 strings that summarize the proof results. 106 | When printing, each string will render as a GitHub flavored Markdown table. 107 | """ 108 | output = "## Summary of CBMC proof results\n\n" 109 | with open(out_file, encoding='utf-8') as run_json: 110 | run_dict = json.load(run_json) 111 | status_table, proof_table = _get_status_and_proof_summaries(run_dict) 112 | for summary in (status_table, proof_table): 113 | output += _get_rendered_table(summary) 114 | 115 | print(output) 116 | sys.stdout.flush() 117 | 118 | github_summary_file = os.getenv("GITHUB_STEP_SUMMARY") 119 | if github_summary_file: 120 | with open(github_summary_file, "a") as handle: 121 | print(output, file=handle) 122 | handle.flush() 123 | else: 124 | logging.warning( 125 | "$GITHUB_STEP_SUMMARY not set, not writing summary file") 126 | 127 | msg = ( 128 | "Click the 'Summary' button to view a Markdown table " 129 | "summarizing all proof results") 130 | if run_dict["status"] != "success": 131 | logging.error("Not all proofs passed.") 132 | logging.error(msg) 133 | sys.exit(1) 134 | logging.info(msg) 135 | 136 | 137 | if __name__ == '__main__': 138 | args = get_args() 139 | logging.basicConfig(format="%(levelname)s: %(message)s") 140 | try: 141 | print_proof_results(args.run_file) 142 | except Exception as ex: # pylint: disable=broad-except 143 | logging.critical("Could not print results. Exception: %s", str(ex)) 144 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/sources/README.md: -------------------------------------------------------------------------------- 1 | CBMC proof source code 2 | ====================== 3 | 4 | This directory contains source code written for CBMC proofs. It is 5 | common to write some code to model aspects of the system under test, 6 | and this code goes here. 7 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/template-for-repository/stubs/README.md: -------------------------------------------------------------------------------- 1 | CBMC proof stubs 2 | ====================== 3 | 4 | This directory contains the stubs written for CBMC proofs. It is 5 | common to stub out functionality like network send and receive methods 6 | when writing a CBMC proof, and the code for these stubs goes here. 7 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/update.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | # SPDX-License-Identifier: MIT-0 5 | 6 | 'Update CBMC starter kit.' 7 | 8 | from pathlib import Path 9 | import logging 10 | import platform 11 | import re 12 | import shutil 13 | 14 | from cbmc_starter_kit import arguments, repository, util, version 15 | 16 | ################################################################ 17 | 18 | def parse_arguments(): 19 | desc = 'Update CBMC starter kit in a CBMC proof repository.' 20 | options = [ 21 | {'flag': '--cbmc-root', 22 | 'type': Path, 23 | 'metavar': 'CBMC', 24 | 'help': 'Root of CBMC proof infrastructure (default: ".")'}, 25 | {'flag': '--no-migrate', 26 | 'action': 'store_true', 27 | 'help': """ 28 | Do not remove symlinks under CBMC. 29 | Normally remove symlinks under CBMC to files under STARTER_KIT."""}, 30 | {'flag': '--no-test-removal', 31 | 'action': 'store_true', 32 | 'help': """ 33 | Do not remove negative tests in CBMC/negative_tests. 34 | Normally remove the directory CBMC/negative_tests since most projects don't 35 | use these tests."""}, 36 | {'flag': '--no-update', 37 | 'action': 'store_true', 38 | 'help': """ 39 | Do not update Makefile.common and run-cbmc-proofs.py under CBMC/proofs. 40 | Normally update these files with the versions in the starter kit package."""} 41 | ] 42 | args = arguments.create_parser( 43 | options=options, 44 | description=desc).parse_args() 45 | arguments.configure_logging(args) 46 | return args 47 | 48 | ################################################################ 49 | 50 | def validate_cbmc_root(args): 51 | args.cbmc_root = args.cbmc_root or Path('.') 52 | args.cbmc_root = args.cbmc_root.resolve() 53 | if not args.cbmc_root.is_dir(): 54 | raise UserWarning(f'CBMC root is not a directory: {args.cbmc_root}') 55 | if not (args.cbmc_root/util.PROOF_DIR).is_dir(): 56 | raise UserWarning(f'CBMC root is missing a {util.PROOF_DIR} subdirectory: {args.cbmc_root}') 57 | logging.debug('CBMC root: %s', args.cbmc_root) 58 | return args 59 | 60 | ################################################################ 61 | 62 | def update_litani_makefile_variable(cbmc_root, path): 63 | """Update LITANI to LITANI?=litani in the makefile at the named path""" 64 | 65 | is_litani_line = lambda line: bool(re.match(r'^\s*LITANI\s*\??=', line)) 66 | 67 | with open(cbmc_root/path, encoding='utf-8') as makefile: 68 | lines = makefile.read().splitlines() 69 | if not any(is_litani_line(line) for line in lines): 70 | logging.debug("Not updating LITANI in makefile: %s", path) 71 | return 72 | 73 | logging.info("Updating LITANI in makefile: %s", path) 74 | with open(cbmc_root/path, 'w', encoding='utf-8') as makefile: 75 | for line in lines: 76 | if is_litani_line(line): 77 | line = 'LITANI ?= litani' 78 | print(line, file=makefile) 79 | return 80 | 81 | ################################################################ 82 | 83 | def remove_negative_tests(cbmc_root): 84 | logging.debug('Removing CBMC starter kit negative tests') 85 | negative_tests = cbmc_root / util.NEGATIVE_TESTS 86 | if negative_tests.is_dir(): 87 | logging.info('Removing: %s', negative_tests) 88 | shutil.rmtree(negative_tests) 89 | 90 | def update(cbmc_root, quiet=False): 91 | logging.debug('Updating CBMC starter kit') 92 | for path in [util.COMMON_MAKEFILE, util.RUN_SCRIPT, util.LIB_MODULE]: 93 | src = util.package_repository_template_root() / util.PROOF_DIR / path 94 | dst = cbmc_root / util.PROOF_DIR / path 95 | (logging.debug if quiet else logging.info)('Copying: %s -> %s', src, dst) 96 | assert src.exists() 97 | if src.is_dir(): 98 | shutil.copytree(src, dst, dirs_exist_ok=True) 99 | else: 100 | version.copy_with_version(src, dst) 101 | 102 | def check_for_litani(cbmc_root): 103 | if shutil.which('litani'): 104 | for fyle in (util.TEMPLATE_DEFINES, util.PROJECT_DEFINES): 105 | update_litani_makefile_variable(cbmc_root, Path(util.PROOF_DIR) / fyle) 106 | return 107 | logging.debug("Found litani command is not installed") 108 | system = platform.system() 109 | if system == "Darwin": 110 | logging.warning("Consider installing the litani command available via " 111 | "'brew install litani'") 112 | if system == "Linux": 113 | logging.warning("Consider installing the litani command available via " 114 | "'apt install litani*.deb'") 115 | logging.warning("Download the litani Debian package from " 116 | "https://github.com/awslabs/aws-build-accumulator/releases/latest") 117 | return 118 | 119 | def check_for_proof_ci_workflow(): 120 | workflow_root = repository.github_actions_workflows_root() 121 | path_to_workflow_in_customer_repo = workflow_root / "proof_ci.yaml" 122 | if path_to_workflow_in_customer_repo.exists(): 123 | version.update_existing_version_in_workflow_file( 124 | path_to_workflow_in_customer_repo) 125 | 126 | ################################################################ 127 | 128 | def main(): 129 | """Update files, that were previously added, during installation of the 130 | CBMC starter kit package.""" 131 | 132 | args = parse_arguments() 133 | logging.debug('args: %s', args) 134 | 135 | args = validate_cbmc_root(args) 136 | 137 | if not args.no_test_removal: 138 | remove_negative_tests(args.cbmc_root) 139 | if not args.no_update: 140 | update(args.cbmc_root) 141 | check_for_litani(args.cbmc_root) 142 | check_for_proof_ci_workflow() 143 | 144 | 145 | ################################################################ 146 | 147 | if __name__ == '__main__': 148 | main() 149 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/util.py: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | """Methods of manipulating the templates repository.""" 5 | 6 | import importlib.util 7 | from pathlib import Path 8 | 9 | from cbmc_starter_kit import repository 10 | 11 | REPOSITORY_TEMPLATES = "template-for-repository" 12 | PROOF_TEMPLATES = "template-for-proof" 13 | CI_WORKFLOW_TEMPLATES = "template-for-ci-workflow" 14 | PROOF_DIR = "proofs" 15 | LIB_MODULE = "lib" 16 | NEGATIVE_TESTS = "negative_tests" 17 | 18 | COMMON_MAKEFILE = "Makefile.common" 19 | TEMPLATE_DEFINES = "Makefile-template-defines" 20 | PROJECT_DEFINES = "Makefile-project-defines" 21 | RUN_SCRIPT = "run-cbmc-proofs.py" 22 | GITHUB_ACTIONS_PROOF_CI = "proof_ci.yaml" 23 | CFN_STACK_OIDC = "proof-ci-Oidc" 24 | CFN_STACK_PIPELINE = "proof-ci-Pipeline" 25 | 26 | ################################################################ 27 | 28 | def package_root(): 29 | """Directory containing the package.""" 30 | 31 | # There are so many incompatible, deprecated ways of doing this 32 | # available in Python version 3.6 through 3.10, and the right 33 | # solution is in question. 34 | # 35 | # The right solution is actually 36 | # 37 | # importlib.resources.files(__package__) 38 | # 39 | # but importlib.resources appears in 3.7 (with a backport 40 | # importlib_resources available before 3.7) and files itself 41 | # appears in 3.9 42 | # 43 | # The next solution is 44 | # 45 | # importlib.resources.path(__package__, resource_name) 46 | # 47 | # but the resource_name must be a string and can't be a directory 48 | # or a path, and this deprecated after 3.9 in favor of files(). 49 | # 50 | # The next solution is 51 | # 52 | # pkgutil.get_data(__package__, resource_path) 53 | # 54 | # but this returns contents and not a path. But, amazing, this 55 | # works even if the package is an egg or a zipfile! 56 | # 57 | # The next solution is 58 | # 59 | # pkg_resources.resource_filename(__package__, resource_path) 60 | # 61 | # but pkg_resources is slow and distributed with setuptools and 62 | # not a standard module. 63 | 64 | # this gives the path to the package __init__.py 65 | init = importlib.util.find_spec(__package__).origin 66 | # this gives the path to the package directory 67 | return Path(init).parent 68 | 69 | def package_repository_template_root(): 70 | return package_root() / REPOSITORY_TEMPLATES 71 | 72 | def package_proof_template_root(): 73 | return package_root() / PROOF_TEMPLATES 74 | 75 | def package_ci_workflow_template_root(): 76 | return package_root() / CI_WORKFLOW_TEMPLATES 77 | 78 | ################################################################ 79 | # Ask the user for set up information 80 | 81 | def ask_for_project_name(): 82 | """Ask user for project name.""" 83 | 84 | return input("What is the project name? ").strip() 85 | 86 | def ask_for_function_name(): 87 | """Ask user for function name.""" 88 | 89 | return input("What is the function name? ").strip() 90 | 91 | def ask_for_source_file(func, cwd=None, repo=None): 92 | """Ask user to select path to source file defining function func.""" 93 | 94 | cwd = Path(cwd or Path.cwd()).resolve() 95 | repo = Path(repo or repository.repository_root(cwd=cwd)).resolve() 96 | sources = repository.function_sources(func, cwd=cwd, repo=repo) 97 | options = sources + ["The source file is not listed here"] 98 | choices = [str(idx) for idx in range(len(options))] 99 | index = choices[-1] 100 | 101 | if sources: 102 | print(f"These source files define a function '{func}':") 103 | for idx, src in enumerate(options): 104 | print(f" {idx:3} {src}") 105 | index = input( 106 | f"Select a source file (the options are {', '.join(choices)}): " 107 | ).strip() or choices[-1] 108 | 109 | if index not in choices: 110 | raise UserWarning(f"{index} is not in {', '.join(choices)}") 111 | if index == choices[-1]: 112 | src = input(f"Enter path to source file defining {func}: ").strip() 113 | else: 114 | src = sources[int(index)] 115 | src = Path(src).expanduser().resolve() 116 | if not src.is_file(): 117 | raise UserWarning(f"Source file '{src}' does not exist") 118 | 119 | return src 120 | 121 | ################################################################ 122 | -------------------------------------------------------------------------------- /src/cbmc_starter_kit/version.py: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | """Version number.""" 5 | 6 | import os 7 | 8 | NAME = "CBMC starter kit" 9 | NUMBER = "2.11" 10 | VERSION = f"{NAME} {NUMBER}" 11 | 12 | REPLACE_TARGET = '_CBMC_STARTER_KIT_VERSION_' 13 | 14 | def version(display=False): 15 | """The version of cbmc viewer.""" 16 | 17 | if display: 18 | print(VERSION) 19 | return VERSION 20 | 21 | def copy_with_version(src, dst): 22 | with open(src, encoding='utf-8') as srcfile: 23 | data = srcfile.read() 24 | with open(dst, 'w', encoding='utf-8') as dstfile: 25 | dstfile.write(data.replace(REPLACE_TARGET, version())) 26 | 27 | def update_existing_version_in_workflow_file(workflow): 28 | tmp_file = f"{workflow}~" 29 | with open(workflow) as src, open(tmp_file, "w") as dst: 30 | for line in src: 31 | if line.startswith(f"# {NAME}"): 32 | print(version(), file=dst) 33 | else: 34 | print(line, file=dst) 35 | os.rename(tmp_file, workflow) 36 | -------------------------------------------------------------------------------- /test/repo/Makefile: -------------------------------------------------------------------------------- 1 | # This test clones a proof repository and compares the proof results 2 | # with the proof results after running cbmc-starter-kit-update. 3 | 4 | PACKAGE_ROOT = ../.. 5 | 6 | REPO = https://github.com/FreeRTOS/coreHTTP.git 7 | REPO_CBMC_ROOT = test/cbmc 8 | REPO_PROOF_ROOT = $(REPO_CBMC_ROOT)/proofs 9 | REPO_PROOFS = --proofs \ 10 | findHeaderFieldParserCallback \ 11 | findHeaderOnHeaderCompleteCallback \ 12 | findHeaderValueParserCallback \ 13 | httpParserOnStatusCallback 14 | 15 | REPO_ROOT = /tmp/repo 16 | CONFIG_ROOT = /tmp/config 17 | REPORT_ROOT = /tmp/reports 18 | REPORT_ROOT1 = $(REPORT_ROOT)1 19 | REPORT_ROOT2 = $(REPORT_ROOT)2 20 | 21 | CONFIG = $(CONFIG_ROOT)/tests/config/config.py 22 | 23 | default: 24 | @ echo "Run test with 'make test'" 25 | 26 | test: clone-repo clone-config run1 install-starter update-starter run2 compare 27 | 28 | clone-repo: 29 | $(RM) -r $(REPO_ROOT) 30 | git clone $(REPO) $(REPO_ROOT) 31 | cd $(REPO_ROOT) && git submodule update --init --checkout --recursive 32 | 33 | # The config.py command is currently in the config branch of a fork 34 | clone-config: 35 | $(RM) -r $(CONFIG_ROOT) 36 | git clone https://github.com/markrtuttle/aws-viewer-for-cbmc $(CONFIG_ROOT) 37 | cd $(CONFIG_ROOT) && git checkout config 38 | 39 | install-starter: 40 | cd $(PACKAGE_ROOT) && make develop 41 | 42 | update-starter: 43 | cd $(REPO_ROOT)/$(REPO_CBMC_ROOT) && /tmp/cbmc-starter-kit/bin/cbmc-starter-kit-update 44 | 45 | run: 46 | $(RM) -r $(REPORT_ROOT) 47 | cd $(REPO_ROOT)/$(REPO_PROOF_ROOT) && litani init --project-name "Report" 48 | cd $(REPO_ROOT)/$(REPO_PROOF_ROOT) && ./run-cbmc-proofs.py --no-standalone $(REPO_PROOFS) 49 | cd $(REPO_ROOT)/$(REPO_PROOF_ROOT) && $(CONFIG) --report-root $(REPORT_ROOT) 50 | cd $(REPO_ROOT)/$(REPO_PROOF_ROOT) && litani run-build 51 | 52 | run1: 53 | $(MAKE) REPORT_ROOT=$(REPORT_ROOT1) run 54 | 55 | run2: 56 | $(MAKE) REPORT_ROOT=$(REPORT_ROOT2) run 57 | 58 | # Need to use the semantic comparison for the json output 59 | # A simple 'diff -rq $(REPORT_ROOT1) $(REPORT_ROOT2)' fails on json property/result output 60 | # For now, just skip comparison of the json output 61 | compare: 62 | @ for html in $$(cd $(REPORT_ROOT1) && find . -name html); do \ 63 | diff -rq $(REPORT_ROOT1)/$$html $(REPORT_ROOT2)/$$html; \ 64 | done 65 | @ for html in $$(cd $(REPORT_ROOT2) && find . -name html); do \ 66 | diff -rq $(REPORT_ROOT1)/$$html $(REPORT_ROOT2)/$$html; \ 67 | done 68 | -------------------------------------------------------------------------------- /test/summarize_test/sample_run.json: -------------------------------------------------------------------------------- 1 | { 2 | "aux": {}, 3 | "project": "example-build", 4 | "version": "1.26.0-rc", 5 | "version_major": 1, 6 | "version_minor": 26, 7 | "version_patch": 0, 8 | "release_candidate": true, 9 | "run_id": "b018ce5a-e34c-45ee-80bc-209c60622e1f", 10 | "stages": [ 11 | "build", 12 | "test", 13 | "report" 14 | ], 15 | "start_time": "2022-05-24T02:48:34Z", 16 | "status": "fail", 17 | "pools": {}, 18 | "parallelism": { 19 | "trace": [ 20 | { 21 | "running": 1, 22 | "finished": 1, 23 | "total": 1, 24 | "time": "2022-05-24T02:51:28.057125Z" 25 | } 26 | ], 27 | "max_parallelism": 1, 28 | "n_proc": 12 29 | }, 30 | "latest_symlink": "/var/folders/bd/w7pff05n5k35735rjvph7dkh0000gs/T/litani/runs/latest", 31 | "end_time": "2022-05-24T02:51:28Z", 32 | "pipelines": [ 33 | { 34 | "name": "pipe-will-fail", 35 | "ci_stages": [ 36 | { 37 | "jobs": [], 38 | "progress": 0, 39 | "complete": true, 40 | "status": "success", 41 | "url": "artifacts/pipe-will-fail/build", 42 | "name": "build" 43 | }, 44 | { 45 | "jobs": [], 46 | "progress": 0, 47 | "complete": true, 48 | "status": "success", 49 | "url": "artifacts/pipe-will-fail/test", 50 | "name": "test" 51 | }, 52 | { 53 | "jobs": [ 54 | { 55 | "wrapper_arguments": { 56 | "subcommand": "exec", 57 | "verbose": false, 58 | "very_verbose": false, 59 | "inputs": null, 60 | "command": "/usr/bin/false", 61 | "outputs": null, 62 | "pipeline_name": "pipe-will-fail", 63 | "ci_stage": "report", 64 | "cwd": null, 65 | "timeout": null, 66 | "timeout_ok": false, 67 | "timeout_ignore": false, 68 | "ignore_returns": null, 69 | "ok_returns": null, 70 | "outcome_table": null, 71 | "interleave_stdout_stderr": false, 72 | "stdout_file": null, 73 | "stderr_file": null, 74 | "pool": null, 75 | "description": null, 76 | "profile_memory": false, 77 | "profile_memory_interval": 10, 78 | "phony_outputs": null, 79 | "tags": null, 80 | "status_file": "/var/folders/bd/w7pff05n5k35735rjvph7dkh0000gs/T/litani/runs/b018ce5a-e34c-45ee-80bc-209c60622e1f/status/0f65a89f-d82a-4a37-8ac6-30172cb8de8b.json", 81 | "job_id": "0f65a89f-d82a-4a37-8ac6-30172cb8de8b" 82 | }, 83 | "complete": true, 84 | "start_time": "2022-05-24T02:51:28Z", 85 | "timeout_reached": false, 86 | "command_return_code": 1, 87 | "memory_trace": {}, 88 | "loaded_outcome_dict": null, 89 | "outcome": "fail", 90 | "wrapper_return_code": 1, 91 | "stdout": [], 92 | "stderr": [], 93 | "end_time": "2022-05-24T02:51:28Z", 94 | "duration_str": "00s", 95 | "duration": 0 96 | } 97 | ], 98 | "progress": 100, 99 | "complete": true, 100 | "status": "fail", 101 | "url": "artifacts/pipe-will-fail/report", 102 | "name": "report" 103 | } 104 | ], 105 | "url": "pipelines/pipe-will-fail", 106 | "status": "fail", 107 | "dependencies_url": "dependencies.svg" 108 | }, 109 | { 110 | "name": "pipe-will-succeed", 111 | "ci_stages": [ 112 | { 113 | "jobs": [], 114 | "progress": 0, 115 | "complete": true, 116 | "status": "success", 117 | "url": "artifacts/pipe-will-succeed/build", 118 | "name": "build" 119 | }, 120 | { 121 | "jobs": [ 122 | { 123 | "wrapper_arguments": { 124 | "subcommand": "exec", 125 | "verbose": false, 126 | "very_verbose": false, 127 | "inputs": null, 128 | "command": "/usr/bin/true", 129 | "outputs": null, 130 | "pipeline_name": "pipe-will-succeed", 131 | "ci_stage": "test", 132 | "cwd": null, 133 | "timeout": null, 134 | "timeout_ok": false, 135 | "timeout_ignore": false, 136 | "ignore_returns": null, 137 | "ok_returns": null, 138 | "outcome_table": null, 139 | "interleave_stdout_stderr": false, 140 | "stdout_file": null, 141 | "stderr_file": null, 142 | "pool": null, 143 | "description": null, 144 | "profile_memory": false, 145 | "profile_memory_interval": 10, 146 | "phony_outputs": null, 147 | "tags": null, 148 | "status_file": "/var/folders/bd/w7pff05n5k35735rjvph7dkh0000gs/T/litani/runs/b018ce5a-e34c-45ee-80bc-209c60622e1f/status/f29d81df-0692-4002-ae1f-5068d684b025.json", 149 | "job_id": "f29d81df-0692-4002-ae1f-5068d684b025" 150 | }, 151 | "complete": true, 152 | "start_time": "2022-05-24T02:49:30Z", 153 | "timeout_reached": false, 154 | "command_return_code": 0, 155 | "memory_trace": {}, 156 | "loaded_outcome_dict": null, 157 | "outcome": "success", 158 | "wrapper_return_code": 0, 159 | "stdout": [], 160 | "stderr": [], 161 | "end_time": "2022-05-24T02:49:30Z", 162 | "duration_str": "00s", 163 | "duration": 0 164 | } 165 | ], 166 | "progress": 100, 167 | "complete": true, 168 | "status": "success", 169 | "url": "artifacts/pipe-will-succeed/test", 170 | "name": "test" 171 | }, 172 | { 173 | "jobs": [], 174 | "progress": 0, 175 | "complete": true, 176 | "status": "success", 177 | "url": "artifacts/pipe-will-succeed/report", 178 | "name": "report" 179 | } 180 | ], 181 | "url": "pipelines/pipe-will-succeed", 182 | "status": "success", 183 | "dependencies_url": "dependencies.svg" 184 | } 185 | ], 186 | "__duration_str": "02m 54s" 187 | } 188 | 189 | -------------------------------------------------------------------------------- /test/summarize_test/summarize_test.py: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import unittest 5 | import unittest.mock 6 | 7 | import pathlib 8 | import sys 9 | 10 | sys.path.append("../../src/cbmc_starter_kit/template-for-repository/proofs/lib") 11 | import summarize 12 | 13 | 14 | class TestSummarizeResults(unittest.TestCase): 15 | def setUp(self): 16 | self.run_file = pathlib.Path().cwd() / "sample_run.json" 17 | 18 | 19 | @unittest.mock.patch('builtins.print') 20 | def test_print_proof_results(self, mock): 21 | with self.assertRaises(SystemExit): 22 | summarize.print_proof_results(self.run_file) 23 | expected_calls = [ 24 | unittest.mock.call( 25 | "## Summary of CBMC proof results\n\n" 26 | "| Status | Count |\n" 27 | "|---------|-------|\n" 28 | "| Fail | 1 |\n" 29 | "| Success | 1 |\n" 30 | "\n" 31 | "| Proof | Status |\n" 32 | "|-------------------|---------|\n" 33 | "| pipe-will-fail | Fail |\n" 34 | "| pipe-will-succeed | Success |\n" 35 | "\n") 36 | ] 37 | self.assertEqual(expected_calls[0], mock.call_args_list[0]) 38 | 39 | 40 | if __name__ == '__main__': 41 | unittest.main() 42 | -------------------------------------------------------------------------------- /training-material/CODING-FOR-VERIFICATION.md: -------------------------------------------------------------------------------- 1 | # CBMC Coding Guidelines 2 | 3 | 4 | 5 | **Table of Contents** 6 | 7 | - [Code Organization to Support Verification](#code-organization-to-support-verification) 8 | - [Improving Verification Performance](#improving-verification-performance) 9 | 10 | 11 | 12 | 13 | The basic principles of coding for verification are similar to those of coding for testability, with some modifications due to the nature of the SAT solver underlying CBMC. 14 | 15 | ## Code Organization to Support Verification 16 | 17 | * Write small functions. Functions should: 18 | * Do exactly one thing. 19 | * Take all their input and outputs through function parameters and rely as little as possible on global state. Where possible, avoid global variables. 20 | * Encapsulate interaction with the environment within a small function. Interaction with the environment includes accessing files, the network, etc. This makes is possible to verify that function independently and then stub it out for the rest of the verification. 21 | * Functions should check their input parameters, and return an error code when they fail to verify. 22 | This makes harnesses much simpler, since any value for the parameters is a valid input to the function. 23 | * Avoid unbounded loops as far as possible, and encapsulate the ones that you need. CBMC does bounded model checking, so we need to be able to compute a bound on the number of iterations of any given loop. Loops that iterate a constant number of times are best. Loops whose iteration depends on input will require making some assumptions about the input. 24 | * Consider defining magic numbers that control loop bounds and buffer sizes in your build system, i.e., `-DBUFFER_SIZE=1024` and similar. This ensures that you can configure this value at build time, and we can also use those values in our proofs. 25 | * Provide an easy way to access static functions and data structures for testing, if you must have them. For example, use a macro that overrides static. 26 | * Make threads independently verifiable. When writing concurrent programs, reduce interaction to well-defined points. This enables verification of each thread in isolation. 27 | 28 | ## Improving Verification Performance 29 | 30 | * Avoid void pointers (`void*`). There are two reasons people use void pointers: 31 | * To hide implementation detail. This use of void pointers is unnecessary, because we can replace `void *bar` with `struct foo *bar` and declare `struct foo` later within the implementation. 32 | * To implement a form of polymorphism. Don't do this for gratuitous reasons (e.g., because it might someday be useful). Void pointers can block constant propagation which can dramatically reduce the size of the formula constructed for the constraint solver. 33 | * Avoid function pointers. When unavoidable, ensure that function pointer types have a unique signature. They really can make the difference between a proof and no proof. When CBMC encounters a function pointer, it has to consider all possibilities for what that function could be, based on loose signature matching. CBMC has to consider possible any function in the entire program whose address is taken with a signature matching the function pointer. So for each function invocation, the symbolic execution of a single function is replaced with the symbolic execution of a collection of functions (including the functions they call), and the combinatorial explosion makes the size of the formula too big for memory. The worst thing you can do is to give your functions the signature `void foo(void *arg)`; see the point above about avoiding `void*`. 34 | * Large (more than several kB in size) arrays can cause trouble. Again, defining the sizes of arrays in the build system means that we can cleanly re-define them to smaller bounds for our proofs. 35 | * Data-structures should explicitly carry their size, as a parameter (e.g., Pascal strings are better than C strings). 36 | * Stay type safe. 37 | * Allocate the correct size of objects. Don't use smaller structs when you're only using some fields. 38 | * Consider encapsulating loops in a function, or even just the loop body. Nested loops can lead to a combinatorial explosion in the size of the formula sent to the constraint solver. Encapsulated loops can be specified and validated in isolation, and the simpler specification can be used in place of the function in the rest of the validation. 39 | * Try to minimize string comparisons 40 | * E.g., instead of making a `string->string` hash table, consider an `enum->string` hash-table. 41 | -------------------------------------------------------------------------------- /training-material/DEBUG-CBMC.md: -------------------------------------------------------------------------------- 1 | # Debugging CBMC issues 2 | 3 | 4 | 5 | **Table of Contents** 6 | 7 | - [I see 12 proof failures. How do I select which one to debug?](#i-see-12-proof-failures-how-do-i-select-which-one-to-debug) 8 | - [How do I debug a proof failure?](#how-do-i-debug-a-proof-failure) 9 | - [Read the trace](#read-the-trace) 10 | - [Add additional information to the trace](#add-additional-information-to-the-trace) 11 | - [Delta debugging](#delta-debugging) 12 | - [Add assertions to check your hypotheses.](#add-assertions-to-check-your-hypotheses) 13 | - [Use `assert(0)` to dump program state leading to a checkpoint](#use-assert0-to-dump-program-state-leading-to-a-checkpoint) 14 | - [Use `assume(...)` to block uninteresting paths](#use-assume-to-block-uninteresting-paths) 15 | - [Consider the possibility it is a fault in the code itself](#consider-the-possibility-it-is-a-fault-in-the-code-itself) 16 | - [How do I improve proofs with low coverage?](#how-do-i-improve-proofs-with-low-coverage) 17 | - [Fix any CBMC errors](#fix-any-cbmc-errors) 18 | - [Check for truly unreachable code.](#check-for-truly-unreachable-code) 19 | - [Check for over-constrained inputs](#check-for-over-constrained-inputs) 20 | - [How can I tell if my proof is over-constrained?](#how-can-i-tell-if-my-proof-is-over-constrained) 21 | - [What should I do if CBMC crashes?](#what-should-i-do-if-cbmc-crashes) 22 | 23 | 24 | 25 | 26 | ## I see 12 proof failures. How do I select which one to debug? 27 | CBMC proof failures seem to come in batches: you run the proof, and see a dozen different errors reported 28 | In many cases, these failures are related: instead of stressing about the number of failures, pick one, debug it, and see if fixing it removes (many of) the others. 29 | Some good heuristics for deciding which failure to investigate: 30 | 31 | 1. **Look for a failure that occurs early on in the proof.** 32 | This will often be the one with the shortest trace [TODO viewer should output this information]. 33 | The shorter the trace leading to the issue, the easier it is to debug. 34 | 1. **Look for a failure in code you understand.** 35 | Some functions are simpler than others: a failure in a simple function is often easier to analyze that one in a complicated function. 36 | And a failure in a function you understand is easier than one in a function you are not familiar with. 37 | 1. **Look for a simple type of failure.** 38 | For example, the trace from a null dereference is often easier to follow than the trace for a use of a DEAD pointer. 39 | But they're normally exactly the same bug! 40 | Since null dereference bugs normally give the simplest traces, start with them first. 41 | Often, resolving the null dereference also fixes the other related bugs. 42 | 43 | ## How do I debug a proof failure? 44 | 45 | There are a number of techniques that have proven useful in debugging proof failures. 46 | 47 | ### Read the trace 48 | 49 | [TODO link to a guide to viewer] 50 | CBMC viewer generates a step-by-step trace that leads to the assertion violation. 51 | This trace details 52 | 53 | * Every line of code executed 54 | * Every function call made 55 | * Every time a variable is assigned 56 | 57 | Essentially, this trace contains everything you would get from attaching a debugger to the program, and single stepping until the violation occurred. 58 | Take a look at the values of the relevant variables right before the assertion violation. 59 | Do they make sense? 60 | If not, figure out where they were assigned. 61 | I often find that `Ctrl-F` is my friend here: I search for either the variable name, or the value it was assigned, and see where it appears in the trace. 62 | 63 | Similarly, look at the set of function calls that led to the error. 64 | Do they make sense? 65 | Are there functions you expect to see there, but don't? 66 | Are there functions you didn't expect to see there, but do? 67 | 68 | ### Add additional information to the trace 69 | The trace has all the information you need to understand the state of program memory at every point during the execution. 70 | But its not always that easy to reconstruct. 71 | In particular, the trace records the value of a variable when it is written to. 72 | But it doesn't record the value of a variable that is only read, or passed along to another function. 73 | 74 | You can solve this by adding "dummy writes" to the program. 75 | For example, let's say you were debugging an error that involved the following function 76 | 77 | ``` 78 | int foo(struct bar* b, int x) { 79 | baz(b->data, x); 80 | } 81 | ``` 82 | 83 | Figuring out the value of `b->data` and `x` are possible given a complete trace, but its difficult. 84 | Any it might harder to figure out the value of `b->size`. 85 | Instead, annotate the code to track those values: 86 | 87 | ``` 88 | int foo(struct bar* b, int x) { 89 | struct bar debug_foo_b = *b; 90 | int debug_foo_x = x; 91 | baz(b->data, x); 92 | } 93 | ``` 94 | 95 | the trace will now contain an assignment to `debug_foo_b`, which will let you see what values each member of the struct had. 96 | 97 | ### Delta debugging 98 | 99 | [Delta debugging](http://web2.cs.columbia.edu/~junfeng/09fa-e6998/papers/delta-debug.pdf) is a powerful technique for localizing faults and creating minimal reproducing test-cases. 100 | Essentially, you modify the program in some way, typically either by removing (commenting out) or modifying code. 101 | You then rerun the verification tool, and see if the results changed. 102 | The goal is to either: 103 | 104 | 1. produce a small program which still displays the bug or 105 | 1. produce a small change between two programs, one of which has the bug, and the other doesn't. 106 | 107 | In case 1, you now have a small program which is hopefully easy to understand; 108 | In case 2, you have a small change which induces the bug, and hopefully leads you toward the root cause. 109 | 110 | ### Add assertions to check your hypotheses. 111 | 112 | For example, consider the case of a null pointer dereference of a pointer `p`. 113 | It is important to distinguish the case where the pointer *must* be null, vs the case where it *may* be null, vs the case where it *is never* null. 114 | You can test for these cases by adding `assert(p)` to the function. 115 | If the can be null, the assertion will trigger. 116 | If it cannot be null, the assertion will succeed. 117 | 118 | Now, check `assert(!p)` instead. 119 | If can be non-null, this assertion will fail. 120 | If it can only be null, this assertion will succeed. 121 | 122 | You now know which one of the three cases is true. 123 | And you can use the trace to see why it can be null/non-null. 124 | 125 | You can do similar things to determine why a branch is reachable, or unreachable. 126 | 127 | ### Use `assert(0)` to dump program state leading to a checkpoint 128 | 129 | Sometimes, you want to know how/whether a particular line of code is reachable. 130 | One easy way to learn that is to put `assert(0)` right before the line. 131 | CBMC will detect the assertion violation, and give a trace explaining how it reached there, and with what values. 132 | If the assertion passes without error, you know that the line is unreachable given the current proof harness. 133 | 134 | ### Use `assume(...)` to block uninteresting paths 135 | 136 | There are often many possible execution paths that reach a given line of code / assertion. 137 | Some of these may reflect cases you are trying to understand, while others do not help with your current debugging plan. 138 | Left to its own devices, CBMC will non-deterministically choose one of those traces, which may not be the one you want. 139 | You can guide CBMC to the trace you want by sprinkling `__CPROVER_assume()` statements within the code. 140 | For example, you might `__CPROVER_assume()` that a function fails with an error code, to test whether the calling function handles that error code correctly. 141 | Or you might `__CPROVER_assume()` that a given variable is null, to simplify you search for the root cause of a null dereference. 142 | 143 | ### Consider the possibility it is a fault in the code itself 144 | 145 | In many cases, the error detected by CBMC represents a true issue within the code itself. 146 | This is particularly common in the case of functions which fail to validate their inputs. 147 | In this case, the fix is either to validate the inputs, and return an error if given invalid inputs, or to document the requirements on the inputs, and state that actions on illegal inputs are undefined behaviour. 148 | Which solution you choose depends on the risk profile of the code. 149 | 150 | It is also common that code being verified has integer-overflows and other errors that only occur in unusual circumstances. 151 | In these cases, the solution is to either guarantee that inputs are sufficiently small to prevent these issues, or to use overflow-safe builtins, such as gcc's `__builtin_mul_overflow` (documented [here](https://gcc.gnu.org/onlinedocs/gcc/Integer-Overflow-Builtins.html)). 152 | 153 | ## How do I improve proofs with low coverage? 154 | 155 | ### Fix any CBMC errors 156 | Make sure that there are no missing function definitions, or property violations. 157 | Both of these errors can affect coverage calculations. 158 | 159 | ### Check for truly unreachable code. 160 | 161 | In some cases, code may be truly unreachable - for example, redundant defensive checks. 162 | Or this may be code which is sometimes reachable, but not in the context of your proof. 163 | For example: 164 | 165 | ``` 166 | int size_from_enum(type_enum t) { 167 | switch (t) { 168 | case BAR: return 1; 169 | case BAZ: return 2; 170 | ... 171 | } 172 | 173 | int function_being_tested() { 174 | return size_from_enum(BAZ); 175 | } 176 | ``` 177 | 178 | In this case, most of the lines in `size_from_enum` will appear to be unreachable, even though the proof has full coverage of all truly reachable paths. 179 | 180 | ### Check for over-constrained inputs 181 | 182 | Consider the case where one side of a branch is not reached, or where execution does not continue past an assumption. 183 | In this case, it is possible that the inputs have been over-constrained 184 | 185 | 186 | ## How can I tell if my proof is over-constrained? 187 | 188 | This will normally appear in coverage - overconstrained proofs will normally have unreachable portions of code. 189 | You can also add a "smoke test", but adding assertions that you expect to fail to the code (which can be as simple as `assert(0)`). 190 | If these assertions do not fail, then sometime is wrong with your proof. 191 | 192 | ## What should I do if CBMC crashes? 193 | 194 | 1. Make a new branch, containing the exact code that caused cbmc to crash. 195 | We recommend giving it a name like `cbmc-crashing-bug-1`. 196 | 1. Push it to public github repo (if possible) 197 | 1. Post a bug report [here](https://github.com/diffblue/cbmc/issues/new), linking to the branch that you pushed containing the bug. 198 | 1. Post a bug report on this repo, linking to the bug that you posted on the main CBMC repo. 199 | -------------------------------------------------------------------------------- /training-material/PLANNING.md: -------------------------------------------------------------------------------- 1 | # CBMC Proof Planning 2 | 3 | 4 | 5 | **Table of Contents** 6 | 7 | - [How to select what proofs to attempt, in what order](#how-to-select-what-proofs-to-attempt-in-what-order) 8 | - [How to get a sense of the work CBMC will involve](#how-to-get-a-sense-of-the-work-cbmc-will-involve) 9 | 10 | 11 | 12 | 13 | ## How to select what proofs to attempt, in what order 14 | 15 | 1. Make a dependency graph of the modules in your program. 16 | There are a number of tools that can help with this, including [doxygen](https://www.doxygen.nl/manual/). 17 | In addition, you can manually determine a good approximation to the dependency graph using the `.h` files. 18 | If a module includes the `.h` file of another module, it likely depends on it. 19 | 2. Select the leaves of the graph - those modules which other modules depend upon, but which do not depend on other modules themselves. 20 | Typically, these include the basic data-structures and algorithms used by the rest of the codebase. 21 | Which one of these you choose is a matter of style: you can use the [guidelines for coding for verification](CODING-FOR-VERIFICATION.md) to help select modules which are likely to be good verification targets. 22 | 3. Inside a given module, select the best initial verification target. 23 | This is often, but not always, one of the simpler functions. 24 | In particular, you are looking for a function which is both easy to verify, and will give good insight into the data-structure invariants of the data-structures used in the given module. 25 | The more a function conforms to our [guidelines for coding for verification](CODING-FOR-VERIFICATION.md), the easiest it will be to verify. 26 | In our experience, it often makes sense to start with allocation or initialization functions (which often have named that end in `_alloc()` or `_init()`. 27 | 4. The first proof for a given module is typically the hardest. 28 | It typically requires the creation of an `_is_valid()` function and an `_ensure_is_allocated()` function. 29 | However, once these have been written once, the remainder of the module becomes much easier. 30 | The amount of time needed to complete a proof can vary significantly, from hours for a simple proof to days for a complex one. 31 | If the function has few dependencies, and conforms to the guidelines for coding for verification, we would expect an initial proof to take perhaps a day's work.[TODO I made up this number. We need data] 32 | If it is taking longer than this, try a different entry-point. 33 | 34 | ## How to get a sense of the work CBMC will involve 35 | 36 | We recommend selecting a few (2-3) modules from the leaves of the dependency graph, and then doing 2-3 proofs from each module. 37 | This will give you a sense of 38 | 39 | 1. How much work the first proof in a new module is 40 | 1. How much work subsequent proofs in that module is, once the `_is_valid()` and `_ensure_is_allocated()` functions are written. 41 | 42 | Predicting precisely how hard a piece of code will be to verify can be difficult. 43 | In general, however, the more code conforms to our [guidelines for coding for verification](CODING-FOR-VERIFICATION.md), the easier it will be to verify. 44 | We recommend trying modules of different verification complexity to get a sense of overall expected effort. 45 | 46 | Particular features to look for are: 47 | 48 | 1. Does the code have loops? 49 | 1. If so, are those loops nested? 50 | Since CBMC unrolls loops before verifying them, nested loops can lead to a quadratic (or worse) increase in the amount of work CBMC will need to perform. 51 | 1. Are they over fixed sizes, or do they vary with the size of inputs? 52 | If loops are of fixed size, it may be hard to simplify the problem if CBMC is having performance issues as the proof is being developed. 53 | On the other hand, once the proof is complete, functions with fixed-sized loops may have higher assurance proofs, since data-structures do not need to be bounded for performance reasons. 54 | 1. Does the code use inductive data-structures (e.g. linked lists, trees)? 55 | Inductive data-structures are much harder to model and verify than linear structures such as arrays. 56 | 1. Does the code have function pointers? 57 | Function pointers are hard to model. 58 | They can also cause performance problems for CBMC. 59 | 1. Does the code have an simple and obvious specification? 60 | One of the main challenges in verification is writing the specification. 61 | The simpler the specification of the code being verified, the easier it is to verify. 62 | Similarly, the better the documentation, the easier it is. 63 | 64 | 65 | -------------------------------------------------------------------------------- /training-material/README.md: -------------------------------------------------------------------------------- 1 | # README 2 | 3 | 4 | 5 | **Table of Contents** 6 | 7 | - [Structure of this repository](#structure-of-this-repository) 8 | 9 | 10 | 11 | ## Structure of this repository 12 | 13 | This repository includes a variety of information to help you verify a C project using CBMC. 14 | 15 | * [How to setup your repository](https://github.com/awslabs/aws-templates-for-cbmc-proofs/blob/master/README.md) 16 | * [How to plan your verification tasks, and estimate effort](PLANNING.md) 17 | * [How to write a good CBMC proof](PROOF-WRITING.md) 18 | * [How to debug a CBMC error report](DEBUG-CBMC.md) 19 | * [How to write code so that it will be easy to verify](CODING-FOR-VERIFICATION.md) 20 | * [A proof checklist for writers and reviewers](checklist.md) 21 | * [A FAQ](FAQ.md) 22 | -------------------------------------------------------------------------------- /training-material/checklist.md: -------------------------------------------------------------------------------- 1 | # A proof checklist for writers and reviewers 2 | 3 | This is a check list intended for 4 | 5 | * proof writers to use before checking a proof into a repository and 6 | * proof reviewers to use during the code review of a pull 7 | request containing a proof. 8 | 9 | This check list is intended to ensure clear answers to two questions: 10 | 11 | * What properties are being checked by the proof? 12 | * What assumptions are being made by the proof? 13 | 14 | and that these answers can be found in one of three places: 15 | 16 | * The proof harness, 17 | * The proof makefiles (and, with the starter kit, these makefiles are 18 | the proof Makefile, the project Makefile-project-defines, 19 | and the project Makefile.common), and perhaps 20 | * The proof readme file. 21 | 22 | The best practices for writing a proof are described 23 | in the section [How to Write a CBMC Proof](PROOF-WRITING.md) 24 | of the [training material](README.md) 25 | in this [starter kit](../README.md). 26 | Reviewers should keep these best practices in mind when reading a proof. 27 | We recommend that any deviations from best practices be explained in the 28 | readme file. 29 | 30 | ## Properties checked 31 | 32 | Check the following: 33 | 34 | * All of the standard property-checking flags are used: 35 | 36 | * --bounds-check 37 | * --conversion-check 38 | * --div-by-zero-check 39 | * --float-overflow-check 40 | * --malloc-fail-null 41 | * --malloc-may-fail 42 | * --nan-check 43 | * --pointer-check 44 | * --pointer-overflow-check 45 | * --pointer-primitive-check 46 | * --signed-overflow-check 47 | * --undefined-shift-check 48 | * --unsigned-overflow-check 49 | 50 | Note that the starter kit uses these flags by default. 51 | Note, however, that a developer may disable any one of these flags 52 | by editing project Makefile.common or 53 | by setting a makefile variable to the empty string 54 | (as in `CBMC_FLAG_MALLOC_MAY_FAIL = `) 55 | in the project Makefile-project-defines or a proof Makefile. 56 | These are the places to look for deviations. 57 | 58 | * All deviations from the standard property-checking flags are documented. 59 | 60 | There are valid reasons to omit flags either for a project or for an 61 | individual proof. But the decision and the reason for the decision 62 | must be documented either in a project readme or a proof readme file. 63 | 64 | CBMC checks assertions in the code. This is understood and need not be 65 | documented. 66 | 67 | ## Assumptions made 68 | 69 | Check the following: 70 | 71 | * All nontrivial data structures have an 72 | [`ensure_allocated` function](PROOF-WRITING.md#the-ensure_allocated-function) 73 | as described in the training material. 74 | 75 | Feel free to use any naming scheme that makes sense for your project --- some 76 | projects use `allocate_X` in place of `ensure_allocated_X` --- but be 77 | consistent. 78 | 79 | * All nontrivial data structures have an 80 | [`is_valid()` predicate](PROOF-WRITING.md#the-is_valid-function) 81 | as described in the training material for every nontrivial data structure. 82 | 83 | * All definitions of `ensure_allocated` functions and `is_valid` predicates 84 | appear in a common location. 85 | 86 | These definitions are most commonly stored in the `proofs/sources` 87 | subdirectory of the starter kit. Definitions are stored here and used 88 | consistently in the proofs. 89 | 90 | * All pointers passed as input are allocated on the heap with `malloc`. 91 | 92 | One common mistake is to allocate a buffer `buf` on the stack and to 93 | pass `&buf` to the function under test in the proof harness. This prevents 94 | the proof from considering the case of a NULL pointer. 95 | 96 | * All instances of `__CPROVER_assume` appear in a proof harness. 97 | 98 | Note that some exceptions are required. For example, it may be necessary 99 | in an `ensure_allocated` to assume `length < CBMC_MAX_OBJECT_SIZE` before 100 | invoking `malloc(length)` to avoid a false positive about malloc'ing a 101 | too-big object. But every instance of `__CPROVER_assume` in supporting code 102 | should be copied into the proof harness. The goal is for all proof 103 | assumptions to be documented in one place. 104 | 105 | * All preprocessor definitions related to bounds on input size or 106 | otherwise related to proof assumptions appear in the proof Makefile. 107 | 108 | In particular, do not embed definitions in the supporting code or header 109 | files. The goal is for all proof assumptions to be documented in one place. 110 | 111 | * Confirm that all stubs used in the proof are acceptable abstractions 112 | of the actual code. 113 | 114 | Acceptable could mean simply that every behavior of the original code 115 | is a behavior of the abstraction. 116 | 117 | ## Results checked 118 | 119 | Look at the report in the checks attached to the pull request. 120 | 121 | * Confirm that the coverage is acceptable and confirm that the readme file 122 | explains the reason for any lines not covered. 123 | 124 | * Confirm that the list of missing functions is acceptable. 125 | 126 | * Confirm that there are no errors reported. 127 | 128 | ## Other things to consider 129 | 130 | * Consider writing function contracts for the function under test as 131 | described in [How to Write a CBMC Proof](PROOF-WRITING.md) 132 | of the [training material](README.md). 133 | The check list above ensures that the properties (including the 134 | assumptions about the input) that must be true before function 135 | invocation are clearly stated in the proof harness. Consider adding 136 | a statement of what properties must be true after function invocation 137 | as assertions at the end of the proof harness. 138 | 139 | * Consider adding the assumptions made by the proof harness for a 140 | function under test to the source code for the function in the form 141 | of assertions in the code. This will validate that the assumptions made 142 | by the proof of a function are satisfied by each invocation of the function 143 | (at least during testing). 144 | --------------------------------------------------------------------------------