├── .github └── workflows │ ├── pull_request-6_3_24.yml │ ├── pull_request-7_1_12.yml │ ├── push-6_3_24.yml │ ├── push-7_1_12.yml │ ├── push-rustdoc-7_1_12.yml │ ├── schedule-6_3_24.yml │ └── schedule-7_1_12.yml ├── .gitignore ├── Cargo.toml ├── LICENSE-APACHE ├── LICENSE-MIT ├── README.md ├── fdb-gen ├── Cargo.toml ├── LICENSE-APACHE ├── LICENSE-MIT ├── README.md ├── include │ ├── 630 │ │ ├── fdb.options │ │ └── version.txt │ └── 710 │ │ ├── fdb.options │ │ └── version.txt └── src │ ├── bin │ └── fdb-options-gen.rs │ └── lib.rs ├── fdb-stacktester ├── fdb-stacktester-630 │ ├── Cargo.toml │ ├── LICENSE-APACHE │ ├── LICENSE-MIT │ ├── README.md │ ├── bindingtester │ │ ├── README.md │ │ ├── bindingtester │ │ │ ├── __init__.py │ │ │ ├── bindingtester.py │ │ │ ├── known_testers.py │ │ │ ├── known_testers.py.orig │ │ │ ├── run_binding_tester.sh │ │ │ ├── run_tester_loop.sh │ │ │ ├── spec │ │ │ │ ├── bindingApiTester.md │ │ │ │ └── directoryLayerTester.md │ │ │ ├── tests │ │ │ │ ├── __init__.py │ │ │ │ ├── api.py │ │ │ │ ├── api.py.orig │ │ │ │ ├── directory.py │ │ │ │ ├── directory_hca.py │ │ │ │ ├── directory_state_tree.py │ │ │ │ ├── directory_util.py │ │ │ │ ├── scripted.py │ │ │ │ ├── test_util.py │ │ │ │ └── tuple.py │ │ │ └── util.py │ │ └── python │ │ │ └── tests │ │ │ ├── cancellation_timeout_tests.py │ │ │ ├── directory_extension.py │ │ │ ├── size_limit_tests.py │ │ │ ├── tester.py │ │ │ └── tuple_tests.py │ └── src │ │ └── main.rs └── fdb-stacktester-710 │ ├── Cargo.toml │ ├── LICENSE-APACHE │ ├── LICENSE-MIT │ ├── README.md │ ├── bindingtester │ ├── README.md │ ├── bindingtester │ │ ├── __init__.py │ │ ├── bindingtester.py │ │ ├── known_testers.py │ │ ├── known_testers.py.orig │ │ ├── run_binding_tester.sh │ │ ├── run_tester_loop.sh │ │ ├── spec │ │ │ ├── bindingApiTester.md │ │ │ ├── directoryLayerTester.md │ │ │ └── tenantTester.md │ │ ├── tests │ │ │ ├── __init__.py │ │ │ ├── api.py │ │ │ ├── api.py.orig │ │ │ ├── directory.py │ │ │ ├── directory_hca.py │ │ │ ├── directory_state_tree.py │ │ │ ├── directory_util.py │ │ │ ├── scripted.py │ │ │ ├── test_util.py │ │ │ └── tuple.py │ │ └── util.py │ └── python │ │ └── tests │ │ ├── cancellation_timeout_tests.py │ │ ├── directory_extension.py │ │ ├── fdbcli_tests.py │ │ ├── size_limit_tests.py │ │ ├── tenant_tests.py │ │ ├── tester.py │ │ └── tuple_tests.py │ └── src │ └── main.rs ├── fdb-sys ├── Cargo.toml ├── LICENSE-APACHE ├── LICENSE-MIT ├── README.md ├── build.rs ├── include │ ├── 630 │ │ ├── fdb_c.h │ │ ├── fdb_c_options.g.h │ │ └── version.txt │ └── 710 │ │ ├── fdb_c.h │ │ ├── fdb_c_options.g.h │ │ ├── fdb_c_types.h │ │ └── version.txt └── src │ └── lib.rs ├── fdb ├── Cargo.toml ├── LICENSE-APACHE ├── LICENSE-MIT ├── README.md ├── build.rs ├── examples │ ├── get_committed_version.rs │ ├── get_mapped_range.rs │ ├── get_range.rs │ ├── get_versionstamp.rs │ ├── hello_world.rs │ ├── open_database.rs │ └── watch.rs ├── sismic │ ├── mapped_range_result_state_machine.yaml │ └── range_result_state_machine.yaml ├── src │ ├── database │ │ ├── fdb_database.rs │ │ ├── mod.rs │ │ └── open_database.rs │ ├── error.rs │ ├── fdb.rs │ ├── future.rs │ ├── key_value.rs │ ├── lib.rs │ ├── mapped_key_value.rs │ ├── mapped_range.rs │ ├── option.rs │ ├── range.rs │ ├── subspace.rs │ ├── tenant │ │ ├── fdb_tenant.rs │ │ ├── mod.rs │ │ ├── tenant_inner.rs │ │ └── tenant_management.rs │ ├── transaction │ │ ├── fdb_transaction.rs │ │ ├── mod.rs │ │ ├── read_transaction.rs │ │ └── transaction.rs │ └── tuple │ │ ├── element.rs │ │ ├── key_util.rs │ │ ├── mod.rs │ │ ├── tuple.rs │ │ └── versionstamp.rs └── tests │ ├── panic_on_select_api_version_twice.rs │ ├── panic_on_select_wrong_api_version.rs │ ├── panic_on_start_network_after_network_start_stop.rs │ ├── panic_on_stop_network_without_start_network.rs │ ├── select_api_version_once.rs │ ├── set_network_option.rs │ └── start_stop_network.rs └── nix └── ci ├── 6.3 ├── pull_request.sh ├── push.sh └── schedule.sh ├── 7.1 ├── pull_request.sh ├── pull_request_lcov.sh ├── push.sh ├── push_lcov.sh ├── push_rustdoc.sh └── schedule.sh ├── DEVELOP.md ├── cargo-llvm-cov └── default.nix ├── fdb-6.3 ├── app │ └── default.nix ├── client-lib │ └── default.nix └── default.nix ├── fdb-7.1 ├── app │ └── default.nix ├── client-lib │ └── default.nix └── default.nix ├── fdb-files-6.3 ├── default.nix ├── fdb.cluster ├── fdbcli.service ├── foundationdb.conf └── foundationdb.service ├── fdb-files-7.1 ├── default.nix ├── fdb.cluster ├── fdbcli.service ├── foundationdb.conf └── foundationdb.service ├── flake.lock ├── flake.nix ├── nss └── default.nix └── systemd ├── default.nix └── unit-files ├── default.target ├── halt.service ├── halt.target ├── nix-daemon.service ├── nix-daemon.socket ├── sysinit.target ├── systemd-journald.service └── systemd-journald.socket /.github/workflows/pull_request-6_3_24.yml: -------------------------------------------------------------------------------- 1 | name: pull_request-6_3_24 2 | 3 | env: 4 | image_version: 6_3_24 5 | image_tag: a941c67da0e8 6 | 7 | on: 8 | pull_request: 9 | branches: ["main", "fdb-*.x"] 10 | 11 | jobs: 12 | job-on-pull_request: 13 | runs-on: ubuntu-20.04 14 | 15 | steps: 16 | - uses: actions/checkout@v2 17 | - 18 | name: Pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} container 19 | run: | 20 | sudo podman pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} 21 | - 22 | name: Start container 23 | run: | 24 | sudo podman run --name fdb --volume ${GITHUB_WORKSPACE}:/home/runner/fdb --rm ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} & 25 | - 26 | name: Run 6.3/pull_request.sh 27 | run: | 28 | sudo podman exec --user runner:docker --workdir /home/runner/fdb/nix/ci --tty fdb nix develop .#pull_request-6_3_24 --command ./6.3/pull_request.sh 29 | - 30 | name: Stop container 31 | run: | 32 | sudo podman stop fdb 33 | -------------------------------------------------------------------------------- /.github/workflows/pull_request-7_1_12.yml: -------------------------------------------------------------------------------- 1 | name: pull_request-7_1_12 2 | 3 | env: 4 | image_version: 7_1_12 5 | image_tag: c72739bee1a9 6 | 7 | on: 8 | pull_request: 9 | branches: ["main", "fdb-*.x"] 10 | 11 | jobs: 12 | job-on-pull_request: 13 | runs-on: ubuntu-20.04 14 | 15 | steps: 16 | - uses: actions/checkout@v2 17 | - 18 | name: Pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} container 19 | run: | 20 | sudo podman pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} 21 | - 22 | name: Start container 23 | run: | 24 | sudo podman run --name fdb --volume ${GITHUB_WORKSPACE}:/home/runner/fdb --rm ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} & 25 | - 26 | name: Run 7.1/pull_request.sh 27 | run: | 28 | sudo podman exec --user runner:docker --workdir /home/runner/fdb/nix/ci --tty fdb nix develop .#pull_request-7_1_12 --command ./7.1/pull_request.sh 29 | - 30 | name: Run 7.1/pull_request_lcov.sh 31 | run: | 32 | sudo podman exec --user runner:docker --workdir /home/runner/fdb/nix/ci --tty fdb nix develop .#pull_request-nightly-7_1_12 --command ./7.1/pull_request_lcov.sh 33 | - 34 | name: Upload coverage to Codecov 35 | uses: codecov/codecov-action@v1 36 | with: 37 | files: lcov.info 38 | fail_ci_if_error: true 39 | - 40 | name: Stop container 41 | run: | 42 | sudo podman stop fdb 43 | -------------------------------------------------------------------------------- /.github/workflows/push-6_3_24.yml: -------------------------------------------------------------------------------- 1 | name: push-6_3_24 2 | 3 | env: 4 | image_version: 6_3_24 5 | image_tag: a941c67da0e8 6 | 7 | on: 8 | push: 9 | branches: ["main", "fdb-*.x"] 10 | 11 | jobs: 12 | job-on-push: 13 | runs-on: ubuntu-20.04 14 | 15 | steps: 16 | - uses: actions/checkout@v2 17 | - 18 | name: Pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} container 19 | run: | 20 | sudo podman pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} 21 | - 22 | name: Start container 23 | run: | 24 | sudo podman run --name fdb --volume ${GITHUB_WORKSPACE}:/home/runner/fdb --rm ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} & 25 | - 26 | name: Run 6.3/push.sh 27 | run: | 28 | sudo podman exec --user runner:docker --workdir /home/runner/fdb/nix/ci --tty fdb nix develop .#push-6_3_24 --command ./6.3/push.sh 29 | - 30 | name: Stop container 31 | run: | 32 | sudo podman stop fdb 33 | -------------------------------------------------------------------------------- /.github/workflows/push-7_1_12.yml: -------------------------------------------------------------------------------- 1 | name: push-7_1_12 2 | 3 | env: 4 | image_version: 7_1_12 5 | image_tag: c72739bee1a9 6 | 7 | on: 8 | push: 9 | branches: ["main", "fdb-*.x"] 10 | 11 | jobs: 12 | job-on-push: 13 | runs-on: ubuntu-20.04 14 | 15 | steps: 16 | - uses: actions/checkout@v2 17 | - 18 | name: Pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} container 19 | run: | 20 | sudo podman pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} 21 | - 22 | name: Start container 23 | run: | 24 | sudo podman run --name fdb --volume ${GITHUB_WORKSPACE}:/home/runner/fdb --rm ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} & 25 | - 26 | name: Run 7.1/push.sh 27 | run: | 28 | sudo podman exec --user runner:docker --workdir /home/runner/fdb/nix/ci --tty fdb nix develop .#push-7_1_12 --command ./7.1/push.sh 29 | - 30 | name: Run 7.1/push_lcov.sh 31 | run: | 32 | sudo podman exec --user runner:docker --workdir /home/runner/fdb/nix/ci --tty fdb nix develop .#push-nightly-7_1_12 --command ./7.1/push_lcov.sh 33 | - 34 | name: Upload coverage to Codecov 35 | uses: codecov/codecov-action@v1 36 | with: 37 | files: lcov.info 38 | fail_ci_if_error: true 39 | - 40 | name: Stop container 41 | run: | 42 | sudo podman stop fdb 43 | -------------------------------------------------------------------------------- /.github/workflows/push-rustdoc-7_1_12.yml: -------------------------------------------------------------------------------- 1 | name: push-rustdoc-7_1_12 2 | 3 | env: 4 | image_version: 7_1_12 5 | image_tag: c72739bee1a9 6 | 7 | on: 8 | push: 9 | branches: ["main"] 10 | 11 | jobs: 12 | job-on-push: 13 | runs-on: ubuntu-20.04 14 | 15 | steps: 16 | - uses: actions/checkout@v2 17 | - 18 | name: Pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} container 19 | run: | 20 | sudo podman pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} 21 | - 22 | name: Start container 23 | run: | 24 | sudo podman run --name fdb --volume ${GITHUB_WORKSPACE}:/home/runner/fdb --rm ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} & 25 | - 26 | name: Run 7.1/push_rustdoc.sh 27 | run: | 28 | sudo podman exec --user runner:docker --workdir /home/runner/fdb/nix/ci --tty fdb nix develop .#push_rustdoc-7_1_12 --command ./7.1/push_rustdoc.sh 29 | - 30 | name: Deploy docs 31 | uses: peaceiris/actions-gh-pages@v3 32 | with: 33 | github_token: ${{ secrets.GITHUB_TOKEN }} 34 | publish_branch: gh-pages 35 | publish_dir: ./target/doc 36 | - 37 | name: Stop container 38 | run: | 39 | sudo podman stop fdb 40 | -------------------------------------------------------------------------------- /.github/workflows/schedule-6_3_24.yml: -------------------------------------------------------------------------------- 1 | name: schedule-6_3_24 2 | 3 | env: 4 | image_version: 6_3_24 5 | image_tag: a941c67da0e8 6 | 7 | # We adjust the number of iterations in `schedule.sh` so that the job 8 | # takes approximately 1 hour to finish. Jobs are scheduled at minute 9 | # 30. 10 | on: 11 | schedule: 12 | - cron: '30 * * * *' 13 | 14 | jobs: 15 | job-on-schedule: 16 | runs-on: ubuntu-20.04 17 | 18 | steps: 19 | - uses: actions/checkout@v2 20 | - 21 | name: Pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} container 22 | run: | 23 | sudo podman pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} 24 | - 25 | name: Start container 26 | run: | 27 | sudo podman run --name fdb --volume ${GITHUB_WORKSPACE}:/home/runner/fdb --rm ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} & 28 | - 29 | name: Run 6.3/schedule.sh 30 | run: | 31 | sudo podman exec --user runner:docker --workdir /home/runner/fdb/nix/ci --tty fdb nix develop .#schedule-6_3_24 --command ./6.3/schedule.sh 32 | - 33 | name: Stop container 34 | run: | 35 | sudo podman stop fdb 36 | -------------------------------------------------------------------------------- /.github/workflows/schedule-7_1_12.yml: -------------------------------------------------------------------------------- 1 | name: schedule-7_1_12 2 | 3 | env: 4 | image_version: 7_1_12 5 | image_tag: c72739bee1a9 6 | 7 | # We adjust the number of iterations in `schedule.sh` so that the job 8 | # takes approximately 1 hour to finish. Jobs are scheduled at minute 9 | # 30. 10 | on: 11 | schedule: 12 | - cron: '30 * * * *' 13 | 14 | jobs: 15 | job-on-schedule: 16 | runs-on: ubuntu-20.04 17 | 18 | steps: 19 | - uses: actions/checkout@v2 20 | - 21 | name: Pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} container 22 | run: | 23 | sudo podman pull ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} 24 | - 25 | name: Start container 26 | run: | 27 | sudo podman run --name fdb --volume ${GITHUB_WORKSPACE}:/home/runner/fdb --rm ghcr.io/fdb-rs/fdb-${{ env.image_version }}:${{ env.image_tag }} & 28 | - 29 | name: Run 7.1/schedule.sh 30 | run: | 31 | sudo podman exec --user runner:docker --workdir /home/runner/fdb/nix/ci --tty fdb nix develop .#schedule-7_1_12 --command ./7.1/schedule.sh 32 | - 33 | name: Stop container 34 | run: | 35 | sudo podman stop fdb 36 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | target 2 | Cargo.lock 3 | *.py[cod] 4 | -------------------------------------------------------------------------------- /Cargo.toml: -------------------------------------------------------------------------------- 1 | [workspace] 2 | members = [ 3 | "fdb", 4 | "fdb-gen", 5 | "fdb-stacktester/fdb-stacktester-630", 6 | "fdb-stacktester/fdb-stacktester-710", 7 | "fdb-sys", 8 | ] 9 | -------------------------------------------------------------------------------- /LICENSE-MIT: -------------------------------------------------------------------------------- 1 | Permission is hereby granted, free of charge, to any 2 | person obtaining a copy of this software and associated 3 | documentation files (the "Software"), to deal in the 4 | Software without restriction, including without 5 | limitation the rights to use, copy, modify, merge, 6 | publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software 8 | is furnished to do so, subject to the following 9 | conditions: 10 | 11 | The above copyright notice and this permission notice 12 | shall be included in all copies or substantial portions 13 | of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF 16 | ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED 17 | TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 18 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT 19 | SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY 20 | CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 21 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR 22 | IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 23 | DEALINGS IN THE SOFTWARE. 24 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # FoundationDB Client API for Tokio 2 | 3 | [![crates.io][crates-badge]][crates-url] 4 | [![codecov][codecov-badge]][codecov-url] 5 | [![build][build-badge]][build-url] 6 | 7 | [crates-badge]: https://img.shields.io/crates/v/fdb.svg 8 | [crates-url]: https://crates.io/crates/fdb 9 | [codecov-badge]: https://codecov.io/gh/fdb-rs/fdb/branch/main/graph/badge.svg?token=3O9VXRK4JU 10 | [codecov-url]: https://codecov.io/gh/fdb-rs/fdb 11 | [build-badge]: https://img.shields.io/github/workflow/status/fdb-rs/fdb/push-6_3_23 12 | [build-url]: https://github.com/fdb-rs/fdb/actions/workflows/push-6_3_23.yml 13 | 14 | ## Getting Help 15 | 16 | First, see if the answer to your question can be found in the 17 | [Guides](https://fdb-rs.github.io/docs/getting-started/introduction/) 18 | or the [API documentation](https://docs.rs/fdb/latest/fdb/). If the 19 | answer is not there, you can [contact 20 | us](https://fdb-rs.github.io/docs/help/contact-us/). We would be happy 21 | to try to answer your question. 22 | 23 | You can find `main` branch API doumentation 24 | [here](https://fdb-rs.github.io/fdb/fdb/). 25 | 26 | ## Supported Rust Versions 27 | 28 | We follow Tokio project 29 | [MSRV](https://github.com/tokio-rs/tokio/#supported-rust-versions) 30 | (minimum supported rust version) policy. The current MSRV is 1.49.0. 31 | 32 | ## License 33 | 34 | Licensed under either of 35 | 36 | * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or 37 | http://www.apache.org/licenses/LICENSE-2.0) 38 | * MIT license ([LICENSE-MIT](LICENSE-MIT) or 39 | http://opensource.org/licenses/MIT) 40 | 41 | at your option. 42 | 43 | ## Contribution 44 | 45 | Unless you explicitly state otherwise, any contribution intentionally 46 | submitted for inclusion in the work by you, as defined in the 47 | Apache-2.0 license, shall be dual licensed as above, without any 48 | additional terms or conditions. 49 | -------------------------------------------------------------------------------- /fdb-gen/Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | name = "fdb-gen" 3 | version = "0.4.0" 4 | edition = "2018" 5 | rust-version = "1.49" 6 | authors = ["fdb-rs Developers"] 7 | description = """ 8 | Binding generation helper for FoundationDB 9 | """ 10 | license = "MIT OR Apache-2.0" 11 | repository = "https://github.com/fdb-rs/fdb" 12 | keywords = ["foundationdb", "tokio"] 13 | categories = ["database", "external-ffi-bindings"] 14 | 15 | [package.metadata.docs.rs] 16 | features = ["fdb-7_1"] 17 | 18 | [features] 19 | default = [] 20 | fdb-6_3 = [] 21 | fdb-7_1 = [] 22 | 23 | [dependencies] 24 | xml-rs = "0.8" 25 | -------------------------------------------------------------------------------- /fdb-gen/LICENSE-MIT: -------------------------------------------------------------------------------- 1 | Permission is hereby granted, free of charge, to any 2 | person obtaining a copy of this software and associated 3 | documentation files (the "Software"), to deal in the 4 | Software without restriction, including without 5 | limitation the rights to use, copy, modify, merge, 6 | publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software 8 | is furnished to do so, subject to the following 9 | conditions: 10 | 11 | The above copyright notice and this permission notice 12 | shall be included in all copies or substantial portions 13 | of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF 16 | ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED 17 | TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 18 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT 19 | SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY 20 | CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 21 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR 22 | IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 23 | DEALINGS IN THE SOFTWARE. 24 | -------------------------------------------------------------------------------- /fdb-gen/README.md: -------------------------------------------------------------------------------- 1 | # C API bindings for FoundationDB 2 | 3 | See [fdb](https://crates.io/crates/fdb) crate for a safe abstraction. 4 | 5 | ## License 6 | 7 | Licensed under either of 8 | 9 | * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or 10 | http://www.apache.org/licenses/LICENSE-2.0) 11 | * MIT license ([LICENSE-MIT](LICENSE-MIT) or 12 | http://opensource.org/licenses/MIT) 13 | 14 | at your option. 15 | 16 | ## Contribution 17 | 18 | Unless you explicitly state otherwise, any contribution intentionally 19 | submitted for inclusion in the work by you, as defined in the 20 | Apache-2.0 license, shall be dual licensed as above, without any 21 | additional terms or conditions. 22 | -------------------------------------------------------------------------------- /fdb-gen/include/630/version.txt: -------------------------------------------------------------------------------- 1 | 6.3.15 2 | -------------------------------------------------------------------------------- /fdb-gen/include/710/version.txt: -------------------------------------------------------------------------------- 1 | 7.1.3 2 | -------------------------------------------------------------------------------- /fdb-gen/src/bin/fdb-options-gen.rs: -------------------------------------------------------------------------------- 1 | fn main() { 2 | let mut code = String::new(); 3 | fdb_gen::emit(&mut code).expect("couldn't generate options.rs code!"); 4 | println!("{}", code); 5 | } 6 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | name = "fdb-stacktester-630" 3 | version = "0.1.0" 4 | edition = "2018" 5 | authors = ["fdb-rs Developers"] 6 | license = "MIT OR Apache-2.0" 7 | 8 | [features] 9 | default = ["fdb/fdb-6_3"] 10 | 11 | [dependencies] 12 | bytes = "1" 13 | dashmap = "4" 14 | fdb = { path = "../../fdb", default-features = false } 15 | fdb-sys = { path = "../../fdb-sys", default-features = false } 16 | itertools = "0.10" 17 | num-bigint = "0.4" 18 | tokio = { version = "1", features = ["full"] } 19 | tokio-stream = "0.1" 20 | uuid = { version = "0.8", features = ["v4"] } 21 | 22 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/LICENSE-MIT: -------------------------------------------------------------------------------- 1 | Permission is hereby granted, free of charge, to any 2 | person obtaining a copy of this software and associated 3 | documentation files (the "Software"), to deal in the 4 | Software without restriction, including without 5 | limitation the rights to use, copy, modify, merge, 6 | publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software 8 | is furnished to do so, subject to the following 9 | conditions: 10 | 11 | The above copyright notice and this permission notice 12 | shall be included in all copies or substantial portions 13 | of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF 16 | ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED 17 | TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 18 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT 19 | SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY 20 | CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 21 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR 22 | IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 23 | DEALINGS IN THE SOFTWARE. 24 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/README.md: -------------------------------------------------------------------------------- 1 | # FoundationDB Binding Tester 2 | 3 | [Binding 4 | tester](https://github.com/apple/foundationdb/blob/6.3.23/bindings/bindingtester/spec/bindingApiTester.md) 5 | implementation for Tokio FoundationDB Client. 6 | 7 | ## License 8 | 9 | Licensed under either of 10 | 11 | * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or 12 | http://www.apache.org/licenses/LICENSE-2.0) 13 | * MIT license ([LICENSE-MIT](LICENSE-MIT) or 14 | http://opensource.org/licenses/MIT) 15 | 16 | at your option. 17 | 18 | ## Contribution 19 | 20 | Unless you explicitly state otherwise, any contribution intentionally 21 | submitted for inclusion in the work by you, as defined in the 22 | Apache-2.0 license, shall be dual licensed as above, without any 23 | additional terms or conditions. 24 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/bindingtester/README.md: -------------------------------------------------------------------------------- 1 | Imported following directories from FDB release 6.3.23 2 | 3 | 1. `bindings/python/tests` 4 | 5 | 2. `bindings/bindingtester` 6 | 7 | Following changes were applied. 8 | 9 | 1. Remove testing for `SET_VERSIONSTAMPED_VALUE` v1 format in 10 | `api.py`. 11 | 12 | 2. Set minimum API version to `630` for `python` and `python3` tester 13 | in `known_testers.py` and added `rust` tester. 14 | 15 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/bindingtester/bindingtester/__init__.py: -------------------------------------------------------------------------------- 1 | # 2 | # __init__.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import math 22 | import sys 23 | import os 24 | 25 | sys.path[:0] = [os.path.join(os.path.dirname(__file__), '..', '..', 'bindings', 'python')] 26 | 27 | import util 28 | 29 | FDB_API_VERSION = 630 30 | 31 | LOGGING = { 32 | 'version': 1, 33 | 'disable_existing_loggers': False, 34 | 'formatters': { 35 | 'simple': { 36 | 'format': '%(message)s' 37 | } 38 | }, 39 | 'handlers': { 40 | 'console': { 41 | 'level': 'NOTSET', 42 | 'class': 'logging.StreamHandler', 43 | 'stream': sys.stdout, 44 | 'formatter': 'simple' 45 | } 46 | }, 47 | 'loggers': { 48 | 'foundationdb.bindingtester': { 49 | 'level': 'INFO', 50 | 'handlers': ['console'] 51 | } 52 | } 53 | } 54 | 55 | 56 | class Result: 57 | def __init__(self, subspace, key, values): 58 | self.subspace_tuple = util.subspace_to_tuple(subspace) 59 | self.key_tuple = subspace.unpack(key) 60 | self.values = values 61 | 62 | def key(self, specification): 63 | return self.key_tuple[specification.key_start_index:] 64 | 65 | @staticmethod 66 | def elements_equal(el1, el2): 67 | if type(el1) != type(el2): 68 | return False 69 | 70 | if isinstance(el1, tuple): 71 | return Result.tuples_match(el1, el2) 72 | 73 | if isinstance(el1, float) and math.isnan(el1): 74 | return math.isnan(el2) 75 | 76 | return el1 == el2 77 | 78 | @staticmethod 79 | def tuples_match(t1, t2): 80 | if len(t1) != len(t2): 81 | return False 82 | 83 | return all([Result.elements_equal(x,y) for x,y in zip(t1, t2)]) 84 | 85 | def matches_key(self, rhs, specification): 86 | if not isinstance(rhs, Result): 87 | return False 88 | 89 | return Result.tuples_match(self.key(specification), rhs.key(specification)) 90 | 91 | def matches(self, rhs, specification): 92 | if not self.matches_key(rhs, specification): 93 | return False 94 | 95 | for value in self.values: 96 | for rValue in rhs.values: 97 | if value == rValue: 98 | return True 99 | 100 | return False 101 | 102 | def matches_global_error_filter(self, specification): 103 | return any([specification.matches_global_error_filter(v) for v in self.values]) 104 | 105 | # A non-unique sequence of numbers used to align results from different testers 106 | def sequence_num(self, specification): 107 | if specification.ordering_index is not None: 108 | return self.key_tuple[specification.ordering_index] 109 | 110 | return None 111 | 112 | def __str__(self): 113 | if len(self.values) == 1: 114 | value_str = repr(self.values[0]) 115 | else: 116 | value_str = repr(self.values) 117 | 118 | return '%s = %s' % (repr(self.subspace_tuple + self.key_tuple), value_str) 119 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/bindingtester/bindingtester/known_testers.py: -------------------------------------------------------------------------------- 1 | # 2 | # known_testers.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import os 22 | 23 | MAX_API_VERSION = 630 24 | COMMON_TYPES = ['null', 'bytes', 'string', 'int', 'uuid', 'bool', 'float', 'double', 'tuple'] 25 | ALL_TYPES = COMMON_TYPES + ['versionstamp'] 26 | 27 | 28 | class Tester: 29 | def __init__(self, name, cmd, max_int_bits=64, min_api_version=0, max_api_version=MAX_API_VERSION, threads_enabled=True, types=COMMON_TYPES, directory_snapshot_ops_enabled=True): 30 | self.name = name 31 | self.cmd = cmd 32 | self.max_int_bits = max_int_bits 33 | self.min_api_version = min_api_version 34 | self.max_api_version = max_api_version 35 | self.threads_enabled = threads_enabled 36 | self.types = types 37 | self.directory_snapshot_ops_enabled = directory_snapshot_ops_enabled 38 | 39 | def supports_api_version(self, api_version): 40 | return api_version >= self.min_api_version and api_version <= self.max_api_version 41 | 42 | @classmethod 43 | def get_test(cls, test_name_or_args): 44 | if test_name_or_args in testers: 45 | return testers[test_name_or_args] 46 | else: 47 | return Tester(test_name_or_args.split(' ')[0], test_name_or_args) 48 | 49 | 50 | def _absolute_path(path): 51 | return os.path.join(os.path.dirname(os.path.realpath(__file__)), '..', path) 52 | 53 | 54 | _java_cmd = 'java -ea -cp %s:%s com.apple.foundationdb.test.' % ( 55 | _absolute_path('java/foundationdb-client.jar'), 56 | _absolute_path('java/foundationdb-tests.jar')) 57 | 58 | # We could set min_api_version lower on some of these if the testers were updated to support them 59 | testers = { 60 | 'python': Tester('python', 'python ' + _absolute_path('python/tests/tester.py'), 2040, 630, MAX_API_VERSION, types=ALL_TYPES), 61 | 'python3': Tester('python3', 'python3 ' + _absolute_path('python/tests/tester.py'), 2040, 630, MAX_API_VERSION, types=ALL_TYPES), 62 | 'ruby': Tester('ruby', _absolute_path('ruby/tests/tester.rb'), 2040, 23, MAX_API_VERSION), 63 | 'rust': Tester('rust', _absolute_path('../../../target/release/fdb-stacktester-630'), 2040, 630, MAX_API_VERSION, types=ALL_TYPES), 64 | 'java': Tester('java', _java_cmd + 'StackTester', 2040, 510, MAX_API_VERSION, types=ALL_TYPES), 65 | 'java_async': Tester('java', _java_cmd + 'AsyncStackTester', 2040, 510, MAX_API_VERSION, types=ALL_TYPES), 66 | 'go': Tester('go', _absolute_path('go/build/bin/_stacktester'), 2040, 200, MAX_API_VERSION, types=ALL_TYPES), 67 | 'flow': Tester('flow', _absolute_path('flow/bin/fdb_flow_tester'), 63, 500, MAX_API_VERSION, directory_snapshot_ops_enabled=False), 68 | } 69 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/bindingtester/bindingtester/known_testers.py.orig: -------------------------------------------------------------------------------- 1 | # 2 | # known_testers.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import os 22 | 23 | MAX_API_VERSION = 630 24 | COMMON_TYPES = ['null', 'bytes', 'string', 'int', 'uuid', 'bool', 'float', 'double', 'tuple'] 25 | ALL_TYPES = COMMON_TYPES + ['versionstamp'] 26 | 27 | 28 | class Tester: 29 | def __init__(self, name, cmd, max_int_bits=64, min_api_version=0, max_api_version=MAX_API_VERSION, threads_enabled=True, types=COMMON_TYPES, directory_snapshot_ops_enabled=True): 30 | self.name = name 31 | self.cmd = cmd 32 | self.max_int_bits = max_int_bits 33 | self.min_api_version = min_api_version 34 | self.max_api_version = max_api_version 35 | self.threads_enabled = threads_enabled 36 | self.types = types 37 | self.directory_snapshot_ops_enabled = directory_snapshot_ops_enabled 38 | 39 | def supports_api_version(self, api_version): 40 | return api_version >= self.min_api_version and api_version <= self.max_api_version 41 | 42 | @classmethod 43 | def get_test(cls, test_name_or_args): 44 | if test_name_or_args in testers: 45 | return testers[test_name_or_args] 46 | else: 47 | return Tester(test_name_or_args.split(' ')[0], test_name_or_args) 48 | 49 | 50 | def _absolute_path(path): 51 | return os.path.join(os.path.dirname(os.path.realpath(__file__)), '..', path) 52 | 53 | 54 | _java_cmd = 'java -ea -cp %s:%s com.apple.foundationdb.test.' % ( 55 | _absolute_path('java/foundationdb-client.jar'), 56 | _absolute_path('java/foundationdb-tests.jar')) 57 | 58 | # We could set min_api_version lower on some of these if the testers were updated to support them 59 | testers = { 60 | 'python': Tester('python', 'python ' + _absolute_path('python/tests/tester.py'), 2040, 23, MAX_API_VERSION, types=ALL_TYPES), 61 | 'python3': Tester('python3', 'python3 ' + _absolute_path('python/tests/tester.py'), 2040, 23, MAX_API_VERSION, types=ALL_TYPES), 62 | 'ruby': Tester('ruby', _absolute_path('ruby/tests/tester.rb'), 2040, 23, MAX_API_VERSION), 63 | 'java': Tester('java', _java_cmd + 'StackTester', 2040, 510, MAX_API_VERSION, types=ALL_TYPES), 64 | 'java_async': Tester('java', _java_cmd + 'AsyncStackTester', 2040, 510, MAX_API_VERSION, types=ALL_TYPES), 65 | 'go': Tester('go', _absolute_path('go/build/bin/_stacktester'), 2040, 200, MAX_API_VERSION, types=ALL_TYPES), 66 | 'flow': Tester('flow', _absolute_path('flow/bin/fdb_flow_tester'), 63, 500, MAX_API_VERSION, directory_snapshot_ops_enabled=False), 67 | } 68 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/bindingtester/bindingtester/run_tester_loop.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | LOGGING_LEVEL=WARNING 4 | 5 | function run() { 6 | echo "Running $1 api" 7 | ./bindingtester.py $1 --test-name api --cluster-file fdb.cluster --compare --num-ops 1000 --logging-level $LOGGING_LEVEL 8 | echo "Running $1 concurrent api" 9 | ./bindingtester.py $1 --test-name api --cluster-file fdb.cluster --num-ops 1000 --concurrency 5 --logging-level $LOGGING_LEVEL 10 | echo "Running $1 directory" 11 | ./bindingtester.py $1 --test-name directory --cluster-file fdb.cluster --compare --num-ops 1000 --logging-level $LOGGING_LEVEL 12 | echo "Running $1 directory hca" 13 | ./bindingtester.py $1 --test-name directory_hca --cluster-file fdb.cluster --num-ops 100 --concurrency 5 --logging-level $LOGGING_LEVEL 14 | } 15 | 16 | function scripted() { 17 | echo "Running $1 scripted" 18 | ./bindingtester.py $1 --test-name scripted --cluster-file fdb.cluster --logging-level $LOGGING_LEVEL 19 | } 20 | 21 | function run_scripted() { 22 | scripted python 23 | scripted python3 24 | scripted ruby 25 | scripted java 26 | scripted java_async 27 | scripted go 28 | scripted flow 29 | } 30 | 31 | run_scripted 32 | 33 | i=1 34 | while `true`; do 35 | echo "Pass $i" 36 | i=$((i+1)) 37 | run python 38 | run python3 39 | run ruby 40 | run java 41 | run java_async 42 | run go 43 | run flow 44 | done 45 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/bindingtester/bindingtester/tests/directory_hca.py: -------------------------------------------------------------------------------- 1 | # 2 | # directory_hca.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import random 22 | 23 | import fdb 24 | 25 | from bindingtester import FDB_API_VERSION 26 | from bindingtester import util 27 | 28 | from bindingtester.tests import Test, Instruction, InstructionSet, ResultSpecification 29 | from bindingtester.tests import test_util, directory_util 30 | 31 | fdb.api_version(FDB_API_VERSION) 32 | 33 | 34 | class DirectoryHcaTest(Test): 35 | def __init__(self, subspace): 36 | super(DirectoryHcaTest, self).__init__(subspace) 37 | self.coordination = subspace['coordination'] 38 | self.prefix_log = subspace['prefix_log'] 39 | self.next_path = 1 40 | 41 | def setup(self, args): 42 | self.random = test_util.RandomGenerator(args.max_int_bits, args.api_version, args.types) 43 | self.transactions = [b'tr%d' % i for i in range(3)] # SOMEDAY: parameterize this number? 44 | self.barrier_num = 0 45 | 46 | self.max_directories_per_transaction = 30 47 | if args.api_version < 300: 48 | if args.concurrency > 8: 49 | raise Exception('Directory HCA test does not support concurrency larger than 8 with API version less than 300') 50 | 51 | self.max_directories_per_transaction = 8.0 / args.concurrency 52 | 53 | def commit_transactions(self, instructions, args): 54 | for tr in self.transactions: 55 | if random.random() < 0.8 or args.api_version < 300: 56 | instructions.push_args(tr) 57 | instructions.append('USE_TRANSACTION') 58 | test_util.blocking_commit(instructions) 59 | 60 | def barrier(self, instructions, thread_number, thread_ending=False): 61 | if not thread_ending: 62 | instructions.push_args(self.coordination[(self.barrier_num + 1)][thread_number].key(), b'') 63 | instructions.append('SET_DATABASE') 64 | instructions.append('WAIT_FUTURE') 65 | 66 | instructions.push_args(self.coordination[self.barrier_num][thread_number].key()) 67 | instructions.append('CLEAR_DATABASE') 68 | instructions.append('WAIT_FUTURE') 69 | instructions.push_args(self.coordination[self.barrier_num].key()) 70 | instructions.append('WAIT_EMPTY') 71 | 72 | self.barrier_num += 1 73 | 74 | def generate(self, args, thread_number): 75 | instructions = InstructionSet() 76 | 77 | instructions.append('NEW_TRANSACTION') 78 | 79 | default_path = 'default%d' % self.next_path 80 | self.next_path += 1 81 | dir_list = directory_util.setup_directories(instructions, default_path, self.random) 82 | num_dirs = len(dir_list) 83 | 84 | instructions.push_args(directory_util.DEFAULT_DIRECTORY_INDEX) 85 | instructions.append('DIRECTORY_CHANGE') 86 | 87 | instructions.setup_complete() 88 | 89 | current_op = 0 90 | 91 | while current_op < args.num_ops: 92 | if args.concurrency > 1: 93 | self.barrier(instructions, thread_number) 94 | 95 | instructions.push_args(random.choice(self.transactions)) 96 | instructions.append('USE_TRANSACTION') 97 | 98 | if thread_number == 0 and args.concurrency > 1: 99 | num_directories = 1 100 | else: 101 | num_directories = int(max(1, pow(random.random(), 4) * min(self.max_directories_per_transaction, args.num_ops - current_op))) 102 | 103 | for i in range(num_directories): 104 | path = (self.random.random_unicode_str(16),) 105 | op_args = test_util.with_length(path) + (b'', None) 106 | directory_util.push_instruction_and_record_prefix(instructions, 'DIRECTORY_CREATE', 107 | op_args, path, num_dirs, self.random, self.prefix_log) 108 | num_dirs += 1 109 | 110 | current_op += num_directories 111 | 112 | if args.concurrency > 1: 113 | self.barrier(instructions, thread_number, thread_ending=(current_op >= args.num_ops)) 114 | 115 | if thread_number == 0: 116 | self.commit_transactions(instructions, args) 117 | 118 | return instructions 119 | 120 | @fdb.transactional 121 | def pre_run(self, tr, args): 122 | if args.concurrency > 1: 123 | for i in range(args.concurrency): 124 | tr[self.coordination[0][i]] = b'' 125 | 126 | def validate(self, db, args): 127 | errors = [] 128 | errors += directory_util.check_for_duplicate_prefixes(db, self.prefix_log) 129 | errors += directory_util.validate_hca_state(db) 130 | 131 | return errors 132 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/bindingtester/bindingtester/tests/directory_util.py: -------------------------------------------------------------------------------- 1 | # 2 | # directory_util.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import random 22 | import struct 23 | 24 | import fdb 25 | 26 | from bindingtester import FDB_API_VERSION 27 | from bindingtester import util 28 | 29 | from bindingtester.tests import test_util 30 | from bindingtester.tests.directory_state_tree import DirectoryStateTreeNode 31 | 32 | fdb.api_version(FDB_API_VERSION) 33 | 34 | DEFAULT_DIRECTORY_INDEX = 4 35 | DEFAULT_DIRECTORY_PREFIX = b'default' 36 | DIRECTORY_ERROR_STRING = b'DIRECTORY_ERROR' 37 | 38 | def setup_directories(instructions, default_path, random): 39 | # Clients start with the default directory layer in the directory list 40 | DirectoryStateTreeNode.reset() 41 | dir_list = [DirectoryStateTreeNode.get_layer(b'\xfe')] 42 | 43 | instructions.push_args(0, b'\xfe') 44 | instructions.append('DIRECTORY_CREATE_SUBSPACE') 45 | dir_list.append(DirectoryStateTreeNode(False, True)) 46 | 47 | instructions.push_args(0, b'') 48 | instructions.append('DIRECTORY_CREATE_SUBSPACE') 49 | dir_list.append(DirectoryStateTreeNode(False, True)) 50 | 51 | instructions.push_args(1, 2, 1) 52 | instructions.append('DIRECTORY_CREATE_LAYER') 53 | dir_list.append(DirectoryStateTreeNode.get_layer(b'\xfe')) 54 | 55 | create_default_directory_subspace(instructions, default_path, random) 56 | dir_list.append(dir_list[0].add_child((default_path,), DirectoryStateTreeNode(True, True, has_known_prefix=True))) 57 | DirectoryStateTreeNode.set_default_directory(dir_list[-1]) 58 | 59 | instructions.push_args(DEFAULT_DIRECTORY_INDEX) 60 | instructions.append('DIRECTORY_SET_ERROR_INDEX') 61 | 62 | return dir_list 63 | 64 | 65 | def create_default_directory_subspace(instructions, path, random): 66 | test_util.blocking_commit(instructions) 67 | instructions.push_args(3) 68 | instructions.append('DIRECTORY_CHANGE') 69 | prefix = random.random_string(16) 70 | instructions.push_args(1, path, b'', b'%s-%s' % (DEFAULT_DIRECTORY_PREFIX, prefix)) 71 | instructions.append('DIRECTORY_CREATE_DATABASE') 72 | 73 | instructions.push_args(DEFAULT_DIRECTORY_INDEX) 74 | instructions.append('DIRECTORY_CHANGE') 75 | 76 | 77 | def push_instruction_and_record_prefix(instructions, op, op_args, path, dir_index, random, subspace): 78 | if not op.endswith('_DATABASE'): 79 | instructions.push_args(1, *test_util.with_length(path)) 80 | instructions.append('DIRECTORY_EXISTS') 81 | 82 | # This op must leave the stack in the state it is in at this point, with the exception 83 | # that it may leave an error on the stack 84 | instructions.push_args(*op_args) 85 | instructions.append(op) 86 | 87 | if not op.endswith('_DATABASE'): 88 | instructions.push_args(dir_index) 89 | instructions.append('DIRECTORY_CHANGE') 90 | 91 | instructions.push_args(1, b'', random.random_string(16), b'') 92 | instructions.append('DIRECTORY_PACK_KEY') 93 | test_util.to_front(instructions, 3) # move the existence result up to the front of the stack 94 | 95 | t = util.subspace_to_tuple(subspace) 96 | instructions.push_args(len(t) + 3, *t) 97 | 98 | instructions.append('TUPLE_PACK') # subspace[][][random.random_string(16)] = b'' 99 | instructions.append('SET') 100 | 101 | instructions.push_args(DEFAULT_DIRECTORY_INDEX) 102 | instructions.append('DIRECTORY_CHANGE') 103 | 104 | 105 | def check_for_duplicate_prefixes(db, subspace): 106 | last_prefix = None 107 | start_key = subspace[0].range().start 108 | 109 | duplicates = set() 110 | count = 0 111 | while True: 112 | prefixes = db.get_range(start_key, subspace[0].range().stop, limit=1000) 113 | if len(prefixes) == 0: 114 | break 115 | 116 | start_key = fdb.KeySelector.first_greater_than(prefixes[-1].key) 117 | 118 | prefixes = [subspace[0].unpack(kv.key)[0] for kv in prefixes] 119 | prefixes = [p for p in prefixes if not (p.startswith(DEFAULT_DIRECTORY_PREFIX) or p == DIRECTORY_ERROR_STRING)] 120 | count += len(prefixes) 121 | 122 | prefixes = [last_prefix] + prefixes 123 | duplicates.update([p for i, p in enumerate(prefixes[1:]) if p == prefixes[i]]) 124 | last_prefix = prefixes[-1] 125 | 126 | util.get_logger().info('Checked %d directory prefixes for duplicates' % count) 127 | return ['The prefix %r was allocated multiple times' % d[:-2] for d in set(duplicates)] 128 | 129 | 130 | def validate_hca_state(db): 131 | hca = fdb.Subspace((b'\xfe', b'hca'), b'\xfe') 132 | counters = hca[0] 133 | recent = hca[1] 134 | 135 | last_counter = db.get_range(counters.range().start, counters.range().stop, limit=1, reverse=True) 136 | [(start, reported_count)] = [(counters.unpack(kv.key)[0], struct.unpack(' reported_count: 140 | return ['The HCA reports %d prefixes allocated in current window, but it actually allocated %d' % (reported_count, actual_count)] 141 | 142 | return [] 143 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/bindingtester/bindingtester/tests/tuple.py: -------------------------------------------------------------------------------- 1 | # 2 | # tuple.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import random 22 | import struct 23 | 24 | import fdb 25 | import fdb.tuple 26 | 27 | from bindingtester import FDB_API_VERSION 28 | from bindingtester import util 29 | from bindingtester.tests import Test, Instruction, InstructionSet, ResultSpecification 30 | from bindingtester.tests import test_util 31 | 32 | fdb.api_version(FDB_API_VERSION) 33 | 34 | class TupleTest(Test): 35 | def __init__(self, subspace): 36 | super(TupleTest, self).__init__(subspace) 37 | self.workspace = self.subspace['workspace'] # The keys and values here must match between subsequent runs of the same test 38 | self.stack_subspace = self.subspace['stack'] 39 | 40 | def setup(self, args): 41 | self.max_int_bits = args.max_int_bits 42 | self.api_version = args.api_version 43 | 44 | def generate(self, args, thread_number): 45 | instructions = InstructionSet() 46 | 47 | min_value = -2**self.max_int_bits+1 48 | max_value = 2**self.max_int_bits-1 49 | 50 | instructions.append('NEW_TRANSACTION') 51 | 52 | # Test integer encoding 53 | mutations = 0 54 | for i in range(0, self.max_int_bits+1): 55 | for sign in [-1, 1]: 56 | sign_str = '' if sign == 1 else '-' 57 | for offset in range(-10, 11): 58 | val = (2**i) * sign + offset 59 | if val >= min_value and val <= max_value: 60 | if offset == 0: 61 | add_str = '' 62 | elif offset > 0: 63 | add_str = '+%d' % offset 64 | else: 65 | add_str = '%d' % offset 66 | 67 | instructions.push_args(1, val) 68 | instructions.append('TUPLE_PACK') 69 | instructions.push_args(self.workspace.pack(('%s2^%d%s' % (sign_str, i, add_str),))) 70 | instructions.append('SET') 71 | mutations += 1 72 | 73 | if mutations >= 5000: 74 | test_util.blocking_commit(instructions) 75 | mutations = 0 76 | 77 | instructions.begin_finalization() 78 | 79 | test_util.blocking_commit(instructions) 80 | instructions.push_args(self.stack_subspace.key()) 81 | instructions.append('LOG_STACK') 82 | 83 | test_util.blocking_commit(instructions) 84 | 85 | return instructions 86 | 87 | def get_result_specifications(self): 88 | return [ 89 | ResultSpecification(self.workspace, global_error_filter=[1007, 1009, 1021]), 90 | ResultSpecification(self.stack_subspace, key_start_index=1, ordering_index=1, global_error_filter=[1007, 1009, 1021]), 91 | ] 92 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/bindingtester/bindingtester/util.py: -------------------------------------------------------------------------------- 1 | # 2 | # util.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import logging 22 | import signal 23 | import os 24 | import glob 25 | 26 | import fdb 27 | 28 | 29 | def initialize_logger_level(logging_level): 30 | logger = get_logger() 31 | 32 | assert logging_level in ['DEBUG', 'INFO', 'WARNING', 'ERROR'] 33 | 34 | if logging_level == 'DEBUG': 35 | logger.setLevel(logging.DEBUG) 36 | elif logging_level == 'INFO': 37 | logger.setLevel(logging.INFO) 38 | elif logging_level == 'WARNING': 39 | logger.setLevel(logging.WARNING) 40 | elif logging_level == 'ERROR': 41 | logger.setLevel(logging.ERROR) 42 | 43 | 44 | def get_logger(): 45 | return logging.getLogger('foundationdb.bindingtester') 46 | 47 | 48 | # Attempts to get the name associated with a process termination signal 49 | def signal_number_to_name(signal_num): 50 | name = [] 51 | for key in signal.__dict__.keys(): 52 | if key.startswith('SIG') and getattr(signal, key) == signal_num: 53 | name.append(key) 54 | if len(name) == 1: 55 | return name[0] 56 | else: 57 | return str(signal_num) 58 | 59 | 60 | def import_subclasses(filename, module_path): 61 | for f in glob.glob(os.path.join(os.path.dirname(filename), '*.py')): 62 | fn = os.path.basename(f) 63 | if fn == '__init__.py': 64 | continue 65 | __import__('%s.%s' % (module_path, os.path.splitext(fn)[0])) 66 | 67 | 68 | # Attempts to unpack a subspace 69 | # This throws an exception if the subspace cannot be unpacked as a tuple 70 | # As a result, the binding tester cannot use subspaces that have non-tuple raw prefixes 71 | def subspace_to_tuple(subspace): 72 | try: 73 | return fdb.tuple.unpack(subspace.key()) 74 | except Exception as e: 75 | get_logger().debug(e) 76 | raise Exception('The binding tester does not support subspaces with non-tuple raw prefixes') 77 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-630/bindingtester/python/tests/size_limit_tests.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # 3 | # size_limit_tests.py 4 | # 5 | # This source file is part of the FoundationDB open source project 6 | # 7 | # Copyright 2013-2019 Apple Inc. and the FoundationDB project authors 8 | # 9 | # Licensed under the Apache License, Version 2.0 (the "License"); 10 | # you may not use this file except in compliance with the License. 11 | # You may obtain a copy of the License at 12 | # 13 | # http://www.apache.org/licenses/LICENSE-2.0 14 | # 15 | # Unless required by applicable law or agreed to in writing, software 16 | # distributed under the License is distributed on an "AS IS" BASIS, 17 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | # See the License for the specific language governing permissions and 19 | # limitations under the License. 20 | # 21 | import fdb 22 | import sys 23 | 24 | if __name__ == '__main__': 25 | fdb.api_version(630) 26 | 27 | @fdb.transactional 28 | def setValue(tr, key, value): 29 | tr[key] = value 30 | 31 | @fdb.transactional 32 | def setValueWithLimit(tr, key, value, limit): 33 | tr.options.set_size_limit(limit) 34 | tr[key] = value 35 | 36 | def test_size_limit_option(db): 37 | value = b'a' * 1024 38 | 39 | setValue(db, b't1', value) 40 | assert(value == db[b't1']) 41 | 42 | try: 43 | db.options.set_transaction_size_limit(1000) 44 | setValue(db, b't2', value) 45 | assert(False) # not reached 46 | except fdb.FDBError as e: 47 | assert(e.code == 2101) # Transaction exceeds byte limit (2101) 48 | 49 | # Per transaction option overrides database option 50 | db.options.set_transaction_size_limit(1000000) 51 | try: 52 | setValueWithLimit(db, b't3', value, 1000) 53 | assert(False) # not reached 54 | except fdb.FDBError as e: 55 | assert(e.code == 2101) # Transaction exceeds byte limit (2101) 56 | 57 | # DB default survives on_error reset 58 | db.options.set_transaction_size_limit(1000) 59 | tr = db.create_transaction() 60 | try: 61 | tr[b't4'] = b'bar' 62 | tr.on_error(fdb.FDBError(1007)).wait() 63 | setValue(tr, b't4', value) 64 | tr.commit().wait() 65 | assert(False) # not reached 66 | except fdb.FDBError as e: 67 | assert(e.code == 2101) # Transaction exceeds byte limit (2101) 68 | 69 | @fdb.transactional 70 | def test_get_approximate_size(tr): 71 | tr[b'key1'] = b'value1' 72 | s1 = tr.get_approximate_size().wait() 73 | 74 | tr[b'key2'] = b'value2' 75 | s2 = tr.get_approximate_size().wait() 76 | assert(s1 < s2) 77 | 78 | tr.clear(b'key3') 79 | s3 = tr.get_approximate_size().wait() 80 | assert(s2 < s3) 81 | 82 | tr.add_read_conflict_key(b'key3+') 83 | s4 = tr.get_approximate_size().wait() 84 | assert(s3 < s4) 85 | 86 | tr.add_write_conflict_key(b'key4') 87 | s5 = tr.get_approximate_size().wait() 88 | assert(s4 < s5) 89 | 90 | # Expect a cluster file as input. This test will write to the FDB cluster, so 91 | # be aware of potential side effects. 92 | if __name__ == '__main__': 93 | clusterFile = sys.argv[1] 94 | db = fdb.open(clusterFile) 95 | db.options.set_transaction_timeout(2000) # 2 seconds 96 | db.options.set_transaction_retry_limit(3) 97 | test_size_limit_option(db) 98 | test_get_approximate_size(db) 99 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | name = "fdb-stacktester-710" 3 | version = "0.1.0" 4 | edition = "2018" 5 | authors = ["fdb-rs Developers"] 6 | license = "MIT OR Apache-2.0" 7 | 8 | [features] 9 | default = ["fdb/fdb-7_1"] 10 | 11 | [dependencies] 12 | bytes = "1" 13 | dashmap = "4" 14 | fdb = { path = "../../fdb", default-features = false } 15 | fdb-sys = { path = "../../fdb-sys", default-features = false } 16 | itertools = "0.10" 17 | num-bigint = "0.4" 18 | tokio = { version = "1", features = ["full"] } 19 | tokio-stream = "0.1" 20 | uuid = { version = "0.8", features = ["v4"] } 21 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/LICENSE-MIT: -------------------------------------------------------------------------------- 1 | Permission is hereby granted, free of charge, to any 2 | person obtaining a copy of this software and associated 3 | documentation files (the "Software"), to deal in the 4 | Software without restriction, including without 5 | limitation the rights to use, copy, modify, merge, 6 | publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software 8 | is furnished to do so, subject to the following 9 | conditions: 10 | 11 | The above copyright notice and this permission notice 12 | shall be included in all copies or substantial portions 13 | of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF 16 | ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED 17 | TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 18 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT 19 | SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY 20 | CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 21 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR 22 | IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 23 | DEALINGS IN THE SOFTWARE. 24 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/README.md: -------------------------------------------------------------------------------- 1 | # FoundationDB Binding Tester 2 | 3 | [Binding 4 | tester](https://github.com/apple/foundationdb/blob/7.1.3/bindings/bindingtester/spec/bindingApiTester.md) 5 | implementation for Tokio FoundationDB Client. 6 | 7 | ## License 8 | 9 | Licensed under either of 10 | 11 | * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or 12 | http://www.apache.org/licenses/LICENSE-2.0) 13 | * MIT license ([LICENSE-MIT](LICENSE-MIT) or 14 | http://opensource.org/licenses/MIT) 15 | 16 | at your option. 17 | 18 | ## Contribution 19 | 20 | Unless you explicitly state otherwise, any contribution intentionally 21 | submitted for inclusion in the work by you, as defined in the 22 | Apache-2.0 license, shall be dual licensed as above, without any 23 | additional terms or conditions. 24 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/bindingtester/README.md: -------------------------------------------------------------------------------- 1 | Imported following directories from FDB release 7.1.3 2 | 3 | 1. `bindings/python/tests` 4 | 5 | 2. `bindings/bindingtester` 6 | 7 | Following changes were applied. 8 | 9 | 1. Remove testing for `SET_VERSIONSTAMPED_VALUE` v1 format in 10 | `api.py`. 11 | 12 | 2. Set minimum API version to `630` for `python` and `python3` tester 13 | in `known_testers.py` and added `rust` tester. 14 | 15 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/bindingtester/bindingtester/__init__.py: -------------------------------------------------------------------------------- 1 | # 2 | # __init__.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import math 22 | import sys 23 | import os 24 | 25 | sys.path[:0] = [os.path.join(os.path.dirname(__file__), '..', '..', 'bindings', 'python')] 26 | 27 | import util 28 | 29 | FDB_API_VERSION = 710 30 | 31 | LOGGING = { 32 | 'version': 1, 33 | 'disable_existing_loggers': False, 34 | 'formatters': { 35 | 'simple': { 36 | 'format': '%(message)s' 37 | } 38 | }, 39 | 'handlers': { 40 | 'console': { 41 | 'level': 'NOTSET', 42 | 'class': 'logging.StreamHandler', 43 | 'stream': sys.stdout, 44 | 'formatter': 'simple' 45 | } 46 | }, 47 | 'loggers': { 48 | 'foundationdb.bindingtester': { 49 | 'level': 'INFO', 50 | 'handlers': ['console'] 51 | } 52 | } 53 | } 54 | 55 | 56 | class Result: 57 | def __init__(self, subspace, key, values): 58 | self.subspace_tuple = util.subspace_to_tuple(subspace) 59 | self.key_tuple = subspace.unpack(key) 60 | self.values = values 61 | 62 | def key(self, specification): 63 | return self.key_tuple[specification.key_start_index:] 64 | 65 | @staticmethod 66 | def elements_equal(el1, el2): 67 | if type(el1) != type(el2): 68 | return False 69 | 70 | if isinstance(el1, tuple): 71 | return Result.tuples_match(el1, el2) 72 | 73 | if isinstance(el1, float) and math.isnan(el1): 74 | return math.isnan(el2) 75 | 76 | return el1 == el2 77 | 78 | @staticmethod 79 | def tuples_match(t1, t2): 80 | if len(t1) != len(t2): 81 | return False 82 | 83 | return all([Result.elements_equal(x,y) for x,y in zip(t1, t2)]) 84 | 85 | def matches_key(self, rhs, specification): 86 | if not isinstance(rhs, Result): 87 | return False 88 | 89 | return Result.tuples_match(self.key(specification), rhs.key(specification)) 90 | 91 | def matches(self, rhs, specification): 92 | if not self.matches_key(rhs, specification): 93 | return False 94 | 95 | for value in self.values: 96 | for rValue in rhs.values: 97 | if value == rValue: 98 | return True 99 | 100 | return False 101 | 102 | def matches_global_error_filter(self, specification): 103 | return any([specification.matches_global_error_filter(v) for v in self.values]) 104 | 105 | # A non-unique sequence of numbers used to align results from different testers 106 | def sequence_num(self, specification): 107 | if specification.ordering_index is not None: 108 | return self.key_tuple[specification.ordering_index] 109 | 110 | return None 111 | 112 | def __str__(self): 113 | if len(self.values) == 1: 114 | value_str = repr(self.values[0]) 115 | else: 116 | value_str = repr(self.values) 117 | 118 | return '%s = %s' % (repr(self.subspace_tuple + self.key_tuple), value_str) 119 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/bindingtester/bindingtester/known_testers.py: -------------------------------------------------------------------------------- 1 | # 2 | # known_testers.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import os 22 | 23 | MAX_API_VERSION = 710 24 | COMMON_TYPES = ['null', 'bytes', 'string', 'int', 'uuid', 'bool', 'float', 'double', 'tuple'] 25 | ALL_TYPES = COMMON_TYPES + ['versionstamp'] 26 | 27 | 28 | class Tester: 29 | def __init__(self, name, cmd, max_int_bits=64, min_api_version=0, max_api_version=MAX_API_VERSION, threads_enabled=True, types=COMMON_TYPES, directory_snapshot_ops_enabled=True, tenants_enabled=False): 30 | self.name = name 31 | self.cmd = cmd 32 | self.max_int_bits = max_int_bits 33 | self.min_api_version = min_api_version 34 | self.max_api_version = max_api_version 35 | self.threads_enabled = threads_enabled 36 | self.types = types 37 | self.directory_snapshot_ops_enabled = directory_snapshot_ops_enabled 38 | self.tenants_enabled = tenants_enabled 39 | 40 | def supports_api_version(self, api_version): 41 | return api_version >= self.min_api_version and api_version <= self.max_api_version 42 | 43 | @classmethod 44 | def get_test(cls, test_name_or_args): 45 | if test_name_or_args in testers: 46 | return testers[test_name_or_args] 47 | else: 48 | return Tester(test_name_or_args.split(' ')[0], test_name_or_args) 49 | 50 | 51 | def _absolute_path(path): 52 | return os.path.join(os.path.dirname(os.path.realpath(__file__)), '..', path) 53 | 54 | 55 | _java_cmd = 'java -ea -cp %s:%s com.apple.foundationdb.test.' % ( 56 | _absolute_path('java/foundationdb-client.jar'), 57 | _absolute_path('java/foundationdb-tests.jar')) 58 | 59 | # We could set min_api_version lower on some of these if the testers were updated to support them 60 | testers = { 61 | 'python': Tester('python', 'python ' + _absolute_path('python/tests/tester.py'), 2040, 710, MAX_API_VERSION, types=ALL_TYPES, tenants_enabled=True), 62 | 'python3': Tester('python3', 'python3 ' + _absolute_path('python/tests/tester.py'), 2040, 710, MAX_API_VERSION, types=ALL_TYPES, tenants_enabled=True), 63 | 'ruby': Tester('ruby', _absolute_path('ruby/tests/tester.rb'), 2040, 23, MAX_API_VERSION), 64 | 'rust': Tester('rust', _absolute_path('../../../target/release/fdb-stacktester-710'), 2040, 710, MAX_API_VERSION, types=ALL_TYPES, tenants_enabled=True), 65 | 'java': Tester('java', _java_cmd + 'StackTester', 2040, 510, MAX_API_VERSION, types=ALL_TYPES, tenants_enabled=True), 66 | 'java_async': Tester('java', _java_cmd + 'AsyncStackTester', 2040, 510, MAX_API_VERSION, types=ALL_TYPES, tenants_enabled=True), 67 | 'go': Tester('go', _absolute_path('go/build/bin/_stacktester'), 2040, 200, MAX_API_VERSION, types=ALL_TYPES), 68 | 'flow': Tester('flow', _absolute_path('flow/bin/fdb_flow_tester'), 63, 500, MAX_API_VERSION, directory_snapshot_ops_enabled=False), 69 | } 70 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/bindingtester/bindingtester/known_testers.py.orig: -------------------------------------------------------------------------------- 1 | # 2 | # known_testers.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import os 22 | 23 | MAX_API_VERSION = 710 24 | COMMON_TYPES = ['null', 'bytes', 'string', 'int', 'uuid', 'bool', 'float', 'double', 'tuple'] 25 | ALL_TYPES = COMMON_TYPES + ['versionstamp'] 26 | 27 | 28 | class Tester: 29 | def __init__(self, name, cmd, max_int_bits=64, min_api_version=0, max_api_version=MAX_API_VERSION, threads_enabled=True, types=COMMON_TYPES, directory_snapshot_ops_enabled=True, tenants_enabled=False): 30 | self.name = name 31 | self.cmd = cmd 32 | self.max_int_bits = max_int_bits 33 | self.min_api_version = min_api_version 34 | self.max_api_version = max_api_version 35 | self.threads_enabled = threads_enabled 36 | self.types = types 37 | self.directory_snapshot_ops_enabled = directory_snapshot_ops_enabled 38 | self.tenants_enabled = tenants_enabled 39 | 40 | def supports_api_version(self, api_version): 41 | return api_version >= self.min_api_version and api_version <= self.max_api_version 42 | 43 | @classmethod 44 | def get_test(cls, test_name_or_args): 45 | if test_name_or_args in testers: 46 | return testers[test_name_or_args] 47 | else: 48 | return Tester(test_name_or_args.split(' ')[0], test_name_or_args) 49 | 50 | 51 | def _absolute_path(path): 52 | return os.path.join(os.path.dirname(os.path.realpath(__file__)), '..', path) 53 | 54 | 55 | _java_cmd = 'java -ea -cp %s:%s com.apple.foundationdb.test.' % ( 56 | _absolute_path('java/foundationdb-client.jar'), 57 | _absolute_path('java/foundationdb-tests.jar')) 58 | 59 | # We could set min_api_version lower on some of these if the testers were updated to support them 60 | testers = { 61 | 'python': Tester('python', 'python ' + _absolute_path('python/tests/tester.py'), 2040, 23, MAX_API_VERSION, types=ALL_TYPES, tenants_enabled=True), 62 | 'python3': Tester('python3', 'python3 ' + _absolute_path('python/tests/tester.py'), 2040, 23, MAX_API_VERSION, types=ALL_TYPES, tenants_enabled=True), 63 | 'ruby': Tester('ruby', _absolute_path('ruby/tests/tester.rb'), 2040, 23, MAX_API_VERSION), 64 | 'java': Tester('java', _java_cmd + 'StackTester', 2040, 510, MAX_API_VERSION, types=ALL_TYPES, tenants_enabled=True), 65 | 'java_async': Tester('java', _java_cmd + 'AsyncStackTester', 2040, 510, MAX_API_VERSION, types=ALL_TYPES, tenants_enabled=True), 66 | 'go': Tester('go', _absolute_path('go/build/bin/_stacktester'), 2040, 200, MAX_API_VERSION, types=ALL_TYPES), 67 | 'flow': Tester('flow', _absolute_path('flow/bin/fdb_flow_tester'), 63, 500, MAX_API_VERSION, directory_snapshot_ops_enabled=False), 68 | } 69 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/bindingtester/bindingtester/run_tester_loop.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | LOGGING_LEVEL=WARNING 4 | 5 | function run() { 6 | echo "Running $1 api" 7 | ./bindingtester.py $1 --test-name api --cluster-file fdb.cluster --compare --num-ops 1000 --logging-level $LOGGING_LEVEL 8 | echo "Running $1 concurrent api" 9 | ./bindingtester.py $1 --test-name api --cluster-file fdb.cluster --num-ops 1000 --concurrency 5 --logging-level $LOGGING_LEVEL 10 | echo "Running $1 directory" 11 | ./bindingtester.py $1 --test-name directory --cluster-file fdb.cluster --compare --num-ops 1000 --logging-level $LOGGING_LEVEL 12 | echo "Running $1 directory hca" 13 | ./bindingtester.py $1 --test-name directory_hca --cluster-file fdb.cluster --num-ops 100 --concurrency 5 --logging-level $LOGGING_LEVEL 14 | } 15 | 16 | function scripted() { 17 | echo "Running $1 scripted" 18 | ./bindingtester.py $1 --test-name scripted --cluster-file fdb.cluster --logging-level $LOGGING_LEVEL 19 | } 20 | 21 | function run_scripted() { 22 | scripted python 23 | scripted python3 24 | scripted ruby 25 | scripted java 26 | scripted java_async 27 | scripted go 28 | scripted flow 29 | } 30 | 31 | run_scripted 32 | 33 | i=1 34 | while `true`; do 35 | echo "Pass $i" 36 | i=$((i+1)) 37 | run python 38 | run python3 39 | run ruby 40 | run java 41 | run java_async 42 | run go 43 | run flow 44 | done 45 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/bindingtester/bindingtester/spec/tenantTester.md: -------------------------------------------------------------------------------- 1 | Overview 2 | -------- 3 | 4 | Tenant testing is an optional extension to the core binding tester that enables 5 | testing of the tenant API. This testing is enabled by adding some additional 6 | instructions and modifying the behavior of some existing instructions. 7 | 8 | Additional State and Initialization 9 | ----------------------------------- 10 | 11 | Your tester should store an additional piece of state tracking the active tenant 12 | that is to be used to create transactions. This tenant must support an unset 13 | state, in which case transactions will be created directly on the database. 14 | 15 | New Instructions 16 | ---------------- 17 | 18 | The tenant API introduces some new operations: 19 | 20 | #### TENANT_CREATE 21 | 22 | Pops the top item off of the stack as TENANT_NAME. Creates a new tenant 23 | in the database with the name TENANT_NAME. May optionally push a future 24 | onto the stack. 25 | 26 | #### TENANT_DELETE 27 | 28 | Pops the top item off of the stack as TENANT_NAME. Deletes the tenant with 29 | the name TENANT_NAME from the database. May optionally push a future onto 30 | the stack. 31 | 32 | #### TENANT_SET_ACTIVE 33 | 34 | Pops the top item off of the stack as TENANT_NAME. Opens the tenant with 35 | name TENANT_NAME and stores it as the active tenant. 36 | 37 | #### TENANT_CLEAR_ACTIVE 38 | 39 | Unsets the active tenant. 40 | 41 | Updates to Existing Instructions 42 | -------------------------------- 43 | 44 | Some existing operations in the binding tester will have slightly modified 45 | behavior when tenants are enabled. 46 | 47 | #### NEW_TRANSACTION 48 | 49 | When creating a new transaction, the active tenant should be used. If no active 50 | tenant is set, then the transaction should be created as normal using the 51 | database. 52 | 53 | #### _TENANT suffix 54 | 55 | Similar to the _DATABASE suffix, an operation with the _TENANT suffix indicates 56 | that the operation should be performed on the current active tenant object. If 57 | there is no active tenant, then the operation should be performed on the database 58 | as if _DATABASE was specified. In any case where the operation suffixed with 59 | _DATABASE is allowed to push a future onto the stack, the same operation suffixed 60 | with _TENANT is also allowed to push a future onto the stack. 61 | 62 | If your binding does not support operations directly on a tenant object, you should 63 | simulate it using an anonymous transaction. Remember that set and clear operations 64 | must immediately commit (with appropriate retry behavior!). 65 | 66 | Operations that can include the _TENANT prefix are: 67 | 68 | GET_TENANT 69 | GET_KEY_TENANT 70 | GET_RANGE_TENANT 71 | GET_RANGE_STARTS_WITH_TENANT 72 | GET_RANGE_SELECTOR_TENANT 73 | SET_TENANT 74 | CLEAR_TENANT 75 | CLEAR_RANGE_TENANT 76 | CLEAR_RANGE_STARTS_WITH_TENANT 77 | ATOMIC_OP_TENANT 78 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/bindingtester/bindingtester/tests/directory_hca.py: -------------------------------------------------------------------------------- 1 | # 2 | # directory_hca.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import random 22 | 23 | import fdb 24 | 25 | from bindingtester import FDB_API_VERSION 26 | from bindingtester import util 27 | 28 | from bindingtester.tests import Test, Instruction, InstructionSet, ResultSpecification 29 | from bindingtester.tests import test_util, directory_util 30 | 31 | fdb.api_version(FDB_API_VERSION) 32 | 33 | 34 | class DirectoryHcaTest(Test): 35 | def __init__(self, subspace): 36 | super(DirectoryHcaTest, self).__init__(subspace) 37 | self.coordination = subspace['coordination'] 38 | self.prefix_log = subspace['prefix_log'] 39 | self.next_path = 1 40 | 41 | def setup(self, args): 42 | self.random = test_util.RandomGenerator(args.max_int_bits, args.api_version, args.types) 43 | self.transactions = [b'tr%d' % i for i in range(3)] # SOMEDAY: parameterize this number? 44 | self.barrier_num = 0 45 | 46 | self.max_directories_per_transaction = 30 47 | if args.api_version < 300: 48 | if args.concurrency > 8: 49 | raise Exception('Directory HCA test does not support concurrency larger than 8 with API version less than 300') 50 | 51 | self.max_directories_per_transaction = 8.0 / args.concurrency 52 | 53 | def commit_transactions(self, instructions, args): 54 | for tr in self.transactions: 55 | if random.random() < 0.8 or args.api_version < 300: 56 | instructions.push_args(tr) 57 | instructions.append('USE_TRANSACTION') 58 | test_util.blocking_commit(instructions) 59 | 60 | def barrier(self, instructions, thread_number, thread_ending=False): 61 | if not thread_ending: 62 | instructions.push_args(self.coordination[(self.barrier_num + 1)][thread_number].key(), b'') 63 | instructions.append('SET_DATABASE') 64 | instructions.append('WAIT_FUTURE') 65 | 66 | instructions.push_args(self.coordination[self.barrier_num][thread_number].key()) 67 | instructions.append('CLEAR_DATABASE') 68 | instructions.append('WAIT_FUTURE') 69 | instructions.push_args(self.coordination[self.barrier_num].key()) 70 | instructions.append('WAIT_EMPTY') 71 | 72 | self.barrier_num += 1 73 | 74 | def generate(self, args, thread_number): 75 | instructions = InstructionSet() 76 | 77 | instructions.append('NEW_TRANSACTION') 78 | 79 | default_path = 'default%d' % self.next_path 80 | self.next_path += 1 81 | dir_list = directory_util.setup_directories(instructions, default_path, self.random) 82 | num_dirs = len(dir_list) 83 | 84 | instructions.push_args(directory_util.DEFAULT_DIRECTORY_INDEX) 85 | instructions.append('DIRECTORY_CHANGE') 86 | 87 | instructions.setup_complete() 88 | 89 | current_op = 0 90 | 91 | while current_op < args.num_ops: 92 | if args.concurrency > 1: 93 | self.barrier(instructions, thread_number) 94 | 95 | instructions.push_args(random.choice(self.transactions)) 96 | instructions.append('USE_TRANSACTION') 97 | 98 | if thread_number == 0 and args.concurrency > 1: 99 | num_directories = 1 100 | else: 101 | num_directories = int(max(1, pow(random.random(), 4) * min(self.max_directories_per_transaction, args.num_ops - current_op))) 102 | 103 | for i in range(num_directories): 104 | path = (self.random.random_unicode_str(16),) 105 | op_args = test_util.with_length(path) + (b'', None) 106 | directory_util.push_instruction_and_record_prefix(instructions, 'DIRECTORY_CREATE', 107 | op_args, path, num_dirs, self.random, self.prefix_log) 108 | num_dirs += 1 109 | 110 | current_op += num_directories 111 | 112 | if args.concurrency > 1: 113 | self.barrier(instructions, thread_number, thread_ending=(current_op >= args.num_ops)) 114 | 115 | if thread_number == 0: 116 | self.commit_transactions(instructions, args) 117 | 118 | return instructions 119 | 120 | @fdb.transactional 121 | def pre_run(self, tr, args): 122 | if args.concurrency > 1: 123 | for i in range(args.concurrency): 124 | tr[self.coordination[0][i]] = b'' 125 | 126 | def validate(self, db, args): 127 | errors = [] 128 | errors += directory_util.check_for_duplicate_prefixes(db, self.prefix_log) 129 | errors += directory_util.validate_hca_state(db) 130 | 131 | return errors 132 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/bindingtester/bindingtester/tests/directory_util.py: -------------------------------------------------------------------------------- 1 | # 2 | # directory_util.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import random 22 | import struct 23 | 24 | import fdb 25 | 26 | from bindingtester import FDB_API_VERSION 27 | from bindingtester import util 28 | 29 | from bindingtester.tests import test_util 30 | from bindingtester.tests.directory_state_tree import DirectoryStateTreeNode 31 | 32 | fdb.api_version(FDB_API_VERSION) 33 | 34 | DEFAULT_DIRECTORY_INDEX = 4 35 | DEFAULT_DIRECTORY_PREFIX = b'default' 36 | DIRECTORY_ERROR_STRING = b'DIRECTORY_ERROR' 37 | 38 | def setup_directories(instructions, default_path, random): 39 | # Clients start with the default directory layer in the directory list 40 | DirectoryStateTreeNode.reset() 41 | dir_list = [DirectoryStateTreeNode.get_layer(b'\xfe')] 42 | 43 | instructions.push_args(0, b'\xfe') 44 | instructions.append('DIRECTORY_CREATE_SUBSPACE') 45 | dir_list.append(DirectoryStateTreeNode(False, True)) 46 | 47 | instructions.push_args(0, b'') 48 | instructions.append('DIRECTORY_CREATE_SUBSPACE') 49 | dir_list.append(DirectoryStateTreeNode(False, True)) 50 | 51 | instructions.push_args(1, 2, 1) 52 | instructions.append('DIRECTORY_CREATE_LAYER') 53 | dir_list.append(DirectoryStateTreeNode.get_layer(b'\xfe')) 54 | 55 | create_default_directory_subspace(instructions, default_path, random) 56 | dir_list.append(dir_list[0].add_child((default_path,), DirectoryStateTreeNode(True, True, has_known_prefix=True))) 57 | DirectoryStateTreeNode.set_default_directory(dir_list[-1]) 58 | 59 | instructions.push_args(DEFAULT_DIRECTORY_INDEX) 60 | instructions.append('DIRECTORY_SET_ERROR_INDEX') 61 | 62 | return dir_list 63 | 64 | 65 | def create_default_directory_subspace(instructions, path, random): 66 | test_util.blocking_commit(instructions) 67 | instructions.push_args(3) 68 | instructions.append('DIRECTORY_CHANGE') 69 | prefix = random.random_string(16) 70 | instructions.push_args(1, path, b'', b'%s-%s' % (DEFAULT_DIRECTORY_PREFIX, prefix)) 71 | instructions.append('DIRECTORY_CREATE_DATABASE') 72 | 73 | instructions.push_args(DEFAULT_DIRECTORY_INDEX) 74 | instructions.append('DIRECTORY_CHANGE') 75 | 76 | 77 | def push_instruction_and_record_prefix(instructions, op, op_args, path, dir_index, random, subspace): 78 | if not op.endswith('_DATABASE'): 79 | instructions.push_args(1, *test_util.with_length(path)) 80 | instructions.append('DIRECTORY_EXISTS') 81 | 82 | # This op must leave the stack in the state it is in at this point, with the exception 83 | # that it may leave an error on the stack 84 | instructions.push_args(*op_args) 85 | instructions.append(op) 86 | 87 | if not op.endswith('_DATABASE'): 88 | instructions.push_args(dir_index) 89 | instructions.append('DIRECTORY_CHANGE') 90 | 91 | instructions.push_args(1, b'', random.random_string(16), b'') 92 | instructions.append('DIRECTORY_PACK_KEY') 93 | test_util.to_front(instructions, 3) # move the existence result up to the front of the stack 94 | 95 | t = util.subspace_to_tuple(subspace) 96 | instructions.push_args(len(t) + 3, *t) 97 | 98 | instructions.append('TUPLE_PACK') # subspace[][][random.random_string(16)] = b'' 99 | instructions.append('SET') 100 | 101 | instructions.push_args(DEFAULT_DIRECTORY_INDEX) 102 | instructions.append('DIRECTORY_CHANGE') 103 | 104 | 105 | def check_for_duplicate_prefixes(db, subspace): 106 | last_prefix = None 107 | start_key = subspace[0].range().start 108 | 109 | duplicates = set() 110 | count = 0 111 | while True: 112 | prefixes = db.get_range(start_key, subspace[0].range().stop, limit=1000) 113 | if len(prefixes) == 0: 114 | break 115 | 116 | start_key = fdb.KeySelector.first_greater_than(prefixes[-1].key) 117 | 118 | prefixes = [subspace[0].unpack(kv.key)[0] for kv in prefixes] 119 | prefixes = [p for p in prefixes if not (p.startswith(DEFAULT_DIRECTORY_PREFIX) or p == DIRECTORY_ERROR_STRING)] 120 | count += len(prefixes) 121 | 122 | prefixes = [last_prefix] + prefixes 123 | duplicates.update([p for i, p in enumerate(prefixes[1:]) if p == prefixes[i]]) 124 | last_prefix = prefixes[-1] 125 | 126 | util.get_logger().info('Checked %d directory prefixes for duplicates' % count) 127 | return ['The prefix %r was allocated multiple times' % d[:-2] for d in set(duplicates)] 128 | 129 | 130 | def validate_hca_state(db): 131 | hca = fdb.Subspace((b'\xfe', b'hca'), b'\xfe') 132 | counters = hca[0] 133 | recent = hca[1] 134 | 135 | last_counter = db.get_range(counters.range().start, counters.range().stop, limit=1, reverse=True) 136 | [(start, reported_count)] = [(counters.unpack(kv.key)[0], struct.unpack(' reported_count: 140 | return ['The HCA reports %d prefixes allocated in current window, but it actually allocated %d' % (reported_count, actual_count)] 141 | 142 | return [] 143 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/bindingtester/bindingtester/tests/tuple.py: -------------------------------------------------------------------------------- 1 | # 2 | # tuple.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import random 22 | import struct 23 | 24 | import fdb 25 | import fdb.tuple 26 | 27 | from bindingtester import FDB_API_VERSION 28 | from bindingtester import util 29 | from bindingtester.tests import Test, Instruction, InstructionSet, ResultSpecification 30 | from bindingtester.tests import test_util 31 | 32 | fdb.api_version(FDB_API_VERSION) 33 | 34 | class TupleTest(Test): 35 | def __init__(self, subspace): 36 | super(TupleTest, self).__init__(subspace) 37 | self.workspace = self.subspace['workspace'] # The keys and values here must match between subsequent runs of the same test 38 | self.stack_subspace = self.subspace['stack'] 39 | 40 | def setup(self, args): 41 | self.max_int_bits = args.max_int_bits 42 | self.api_version = args.api_version 43 | 44 | def generate(self, args, thread_number): 45 | instructions = InstructionSet() 46 | 47 | min_value = -2**self.max_int_bits+1 48 | max_value = 2**self.max_int_bits-1 49 | 50 | instructions.append('NEW_TRANSACTION') 51 | 52 | # Test integer encoding 53 | mutations = 0 54 | for i in range(0, self.max_int_bits+1): 55 | for sign in [-1, 1]: 56 | sign_str = '' if sign == 1 else '-' 57 | for offset in range(-10, 11): 58 | val = (2**i) * sign + offset 59 | if val >= min_value and val <= max_value: 60 | if offset == 0: 61 | add_str = '' 62 | elif offset > 0: 63 | add_str = '+%d' % offset 64 | else: 65 | add_str = '%d' % offset 66 | 67 | instructions.push_args(1, val) 68 | instructions.append('TUPLE_PACK') 69 | instructions.push_args(self.workspace.pack(('%s2^%d%s' % (sign_str, i, add_str),))) 70 | instructions.append('SET') 71 | mutations += 1 72 | 73 | if mutations >= 5000: 74 | test_util.blocking_commit(instructions) 75 | mutations = 0 76 | 77 | instructions.begin_finalization() 78 | 79 | test_util.blocking_commit(instructions) 80 | instructions.push_args(self.stack_subspace.key()) 81 | instructions.append('LOG_STACK') 82 | 83 | test_util.blocking_commit(instructions) 84 | 85 | return instructions 86 | 87 | def get_result_specifications(self): 88 | return [ 89 | ResultSpecification(self.workspace, global_error_filter=[1007, 1009, 1021]), 90 | ResultSpecification(self.stack_subspace, key_start_index=1, ordering_index=1, global_error_filter=[1007, 1009, 1021]), 91 | ] 92 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/bindingtester/bindingtester/util.py: -------------------------------------------------------------------------------- 1 | # 2 | # util.py 3 | # 4 | # This source file is part of the FoundationDB open source project 5 | # 6 | # Copyright 2013-2018 Apple Inc. and the FoundationDB project authors 7 | # 8 | # Licensed under the Apache License, Version 2.0 (the "License"); 9 | # you may not use this file except in compliance with the License. 10 | # You may obtain a copy of the License at 11 | # 12 | # http://www.apache.org/licenses/LICENSE-2.0 13 | # 14 | # Unless required by applicable law or agreed to in writing, software 15 | # distributed under the License is distributed on an "AS IS" BASIS, 16 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | # See the License for the specific language governing permissions and 18 | # limitations under the License. 19 | # 20 | 21 | import logging 22 | import signal 23 | import os 24 | import glob 25 | 26 | import fdb 27 | 28 | 29 | def initialize_logger_level(logging_level): 30 | logger = get_logger() 31 | 32 | assert logging_level in ['DEBUG', 'INFO', 'WARNING', 'ERROR'] 33 | 34 | if logging_level == 'DEBUG': 35 | logger.setLevel(logging.DEBUG) 36 | elif logging_level == 'INFO': 37 | logger.setLevel(logging.INFO) 38 | elif logging_level == 'WARNING': 39 | logger.setLevel(logging.WARNING) 40 | elif logging_level == 'ERROR': 41 | logger.setLevel(logging.ERROR) 42 | 43 | 44 | def get_logger(): 45 | return logging.getLogger('foundationdb.bindingtester') 46 | 47 | 48 | # Attempts to get the name associated with a process termination signal 49 | def signal_number_to_name(signal_num): 50 | name = [] 51 | for key in signal.__dict__.keys(): 52 | if key.startswith('SIG') and getattr(signal, key) == signal_num: 53 | name.append(key) 54 | if len(name) == 1: 55 | return name[0] 56 | else: 57 | return str(signal_num) 58 | 59 | 60 | def import_subclasses(filename, module_path): 61 | for f in glob.glob(os.path.join(os.path.dirname(filename), '*.py')): 62 | fn = os.path.basename(f) 63 | if fn == '__init__.py': 64 | continue 65 | __import__('%s.%s' % (module_path, os.path.splitext(fn)[0])) 66 | 67 | 68 | # Attempts to unpack a subspace 69 | # This throws an exception if the subspace cannot be unpacked as a tuple 70 | # As a result, the binding tester cannot use subspaces that have non-tuple raw prefixes 71 | def subspace_to_tuple(subspace): 72 | try: 73 | return fdb.tuple.unpack(subspace.key()) 74 | except Exception as e: 75 | get_logger().debug(e) 76 | raise Exception('The binding tester does not support subspaces with non-tuple raw prefixes') 77 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/bindingtester/python/tests/size_limit_tests.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # 3 | # size_limit_tests.py 4 | # 5 | # This source file is part of the FoundationDB open source project 6 | # 7 | # Copyright 2013-2019 Apple Inc. and the FoundationDB project authors 8 | # 9 | # Licensed under the Apache License, Version 2.0 (the "License"); 10 | # you may not use this file except in compliance with the License. 11 | # You may obtain a copy of the License at 12 | # 13 | # http://www.apache.org/licenses/LICENSE-2.0 14 | # 15 | # Unless required by applicable law or agreed to in writing, software 16 | # distributed under the License is distributed on an "AS IS" BASIS, 17 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | # See the License for the specific language governing permissions and 19 | # limitations under the License. 20 | # 21 | import fdb 22 | import sys 23 | 24 | if __name__ == '__main__': 25 | fdb.api_version(710) 26 | 27 | @fdb.transactional 28 | def setValue(tr, key, value): 29 | tr[key] = value 30 | 31 | @fdb.transactional 32 | def setValueWithLimit(tr, key, value, limit): 33 | tr.options.set_size_limit(limit) 34 | tr[key] = value 35 | 36 | def test_size_limit_option(db): 37 | value = b'a' * 1024 38 | 39 | setValue(db, b't1', value) 40 | assert(value == db[b't1']) 41 | 42 | try: 43 | db.options.set_transaction_size_limit(1000) 44 | setValue(db, b't2', value) 45 | assert(False) # not reached 46 | except fdb.FDBError as e: 47 | assert(e.code == 2101) # Transaction exceeds byte limit (2101) 48 | 49 | # Per transaction option overrides database option 50 | db.options.set_transaction_size_limit(1000000) 51 | try: 52 | setValueWithLimit(db, b't3', value, 1000) 53 | assert(False) # not reached 54 | except fdb.FDBError as e: 55 | assert(e.code == 2101) # Transaction exceeds byte limit (2101) 56 | 57 | # DB default survives on_error reset 58 | db.options.set_transaction_size_limit(1000) 59 | tr = db.create_transaction() 60 | try: 61 | tr[b't4'] = b'bar' 62 | tr.on_error(fdb.FDBError(1007)).wait() 63 | setValue(tr, b't4', value) 64 | tr.commit().wait() 65 | assert(False) # not reached 66 | except fdb.FDBError as e: 67 | assert(e.code == 2101) # Transaction exceeds byte limit (2101) 68 | 69 | @fdb.transactional 70 | def test_get_approximate_size(tr): 71 | tr[b'key1'] = b'value1' 72 | s1 = tr.get_approximate_size().wait() 73 | 74 | tr[b'key2'] = b'value2' 75 | s2 = tr.get_approximate_size().wait() 76 | assert(s1 < s2) 77 | 78 | tr.clear(b'key3') 79 | s3 = tr.get_approximate_size().wait() 80 | assert(s2 < s3) 81 | 82 | tr.add_read_conflict_key(b'key3+') 83 | s4 = tr.get_approximate_size().wait() 84 | assert(s3 < s4) 85 | 86 | tr.add_write_conflict_key(b'key4') 87 | s5 = tr.get_approximate_size().wait() 88 | assert(s4 < s5) 89 | 90 | # Expect a cluster file as input. This test will write to the FDB cluster, so 91 | # be aware of potential side effects. 92 | if __name__ == '__main__': 93 | clusterFile = sys.argv[1] 94 | db = fdb.open(clusterFile) 95 | db.options.set_transaction_timeout(2000) # 2 seconds 96 | db.options.set_transaction_retry_limit(3) 97 | test_size_limit_option(db) 98 | test_get_approximate_size(db) 99 | -------------------------------------------------------------------------------- /fdb-stacktester/fdb-stacktester-710/bindingtester/python/tests/tenant_tests.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # 3 | # tenant_tests.py 4 | # 5 | # This source file is part of the FoundationDB open source project 6 | # 7 | # Copyright 2013-2022 Apple Inc. and the FoundationDB project authors 8 | # 9 | # Licensed under the Apache License, Version 2.0 (the "License"); 10 | # you may not use this file except in compliance with the License. 11 | # You may obtain a copy of the License at 12 | # 13 | # http://www.apache.org/licenses/LICENSE-2.0 14 | # 15 | # Unless required by applicable law or agreed to in writing, software 16 | # distributed under the License is distributed on an "AS IS" BASIS, 17 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | # See the License for the specific language governing permissions and 19 | # limitations under the License. 20 | # 21 | import fdb 22 | import sys 23 | import json 24 | from fdb.tuple import pack 25 | 26 | if __name__ == '__main__': 27 | fdb.api_version(710) 28 | 29 | def cleanup_tenant(db, tenant_name): 30 | try: 31 | tenant = db.open_tenant(tenant_name) 32 | del tenant[:] 33 | fdb.tenant_management.delete_tenant(db, tenant_name) 34 | except fdb.FDBError as e: 35 | if e.code == 2131: # tenant not found 36 | pass 37 | else: 38 | raise 39 | 40 | def test_tenant_tuple_name(db): 41 | tuplename=(b'test', b'level', b'hierarchy', 3, 1.24, 'str') 42 | cleanup_tenant(db, tuplename) 43 | 44 | fdb.tenant_management.create_tenant(db, tuplename) 45 | 46 | tenant=db.open_tenant(tuplename) 47 | tenant[b'foo'] = b'bar' 48 | 49 | assert tenant[b'foo'] == b'bar' 50 | 51 | del tenant[b'foo'] 52 | fdb.tenant_management.delete_tenant(db, tuplename) 53 | 54 | 55 | def test_tenant_operations(db): 56 | cleanup_tenant(db, b'tenant1') 57 | cleanup_tenant(db, b'tenant2') 58 | 59 | fdb.tenant_management.create_tenant(db, b'tenant1') 60 | fdb.tenant_management.create_tenant(db, b'tenant2') 61 | 62 | tenant1 = db.open_tenant(b'tenant1') 63 | tenant2 = db.open_tenant(b'tenant2') 64 | 65 | db[b'tenant_test_key'] = b'no_tenant' 66 | tenant1[b'tenant_test_key'] = b'tenant1' 67 | tenant2[b'tenant_test_key'] = b'tenant2' 68 | 69 | tenant1_entry = db[b'\xff\xff/management/tenant_map/tenant1'] 70 | tenant1_json = json.loads(tenant1_entry) 71 | prefix1 = tenant1_json['prefix'].encode('utf8') 72 | 73 | tenant2_entry = db[b'\xff\xff/management/tenant_map/tenant2'] 74 | tenant2_json = json.loads(tenant2_entry) 75 | prefix2 = tenant2_json['prefix'].encode('utf8') 76 | 77 | assert tenant1[b'tenant_test_key'] == b'tenant1' 78 | assert db[prefix1 + b'tenant_test_key'] == b'tenant1' 79 | assert tenant2[b'tenant_test_key'] == b'tenant2' 80 | assert db[prefix2 + b'tenant_test_key'] == b'tenant2' 81 | assert db[b'tenant_test_key'] == b'no_tenant' 82 | 83 | tr1 = tenant1.create_transaction() 84 | try: 85 | del tr1[:] 86 | tr1.commit().wait() 87 | except fdb.FDBError as e: 88 | tr.on_error(e).wait() 89 | 90 | assert tenant1[b'tenant_test_key'] == None 91 | assert db[prefix1 + b'tenant_test_key'] == None 92 | assert tenant2[b'tenant_test_key'] == b'tenant2' 93 | assert db[prefix2 + b'tenant_test_key'] == b'tenant2' 94 | assert db[b'tenant_test_key'] == b'no_tenant' 95 | 96 | fdb.tenant_management.delete_tenant(db, b'tenant1') 97 | try: 98 | tenant1[b'tenant_test_key'] 99 | assert False 100 | except fdb.FDBError as e: 101 | assert e.code == 2131 # tenant not found 102 | 103 | del tenant2[:] 104 | fdb.tenant_management.delete_tenant(db, b'tenant2') 105 | 106 | assert db[prefix1 + b'tenant_test_key'] == None 107 | assert db[prefix2 + b'tenant_test_key'] == None 108 | assert db[b'tenant_test_key'] == b'no_tenant' 109 | 110 | del db[b'tenant_test_key'] 111 | 112 | assert db[b'tenant_test_key'] == None 113 | 114 | def test_tenant_operation_retries(db): 115 | cleanup_tenant(db, b'tenant1') 116 | cleanup_tenant(db, b'tenant2') 117 | 118 | # Test that the tenant creation only performs the existence check once 119 | fdb.tenant_management._create_tenant_impl(db, b'tenant1', [], force_existence_check_maybe_committed=True) 120 | 121 | # An attempt to create the tenant again should fail 122 | try: 123 | fdb.tenant_management.create_tenant(db, b'tenant1') 124 | assert False 125 | except fdb.FDBError as e: 126 | assert e.code == 2132 # tenant already exists 127 | 128 | # Using a transaction skips the existence check 129 | tr = db.create_transaction() 130 | fdb.tenant_management.create_tenant(tr, b'tenant1') 131 | 132 | # Test that a concurrent tenant creation doesn't interfere with the existence check logic 133 | tr = db.create_transaction() 134 | existence_check_marker = [] 135 | fdb.tenant_management._create_tenant_impl(tr, b'tenant2', existence_check_marker) 136 | 137 | fdb.tenant_management.create_tenant(db, b'tenant2') 138 | 139 | tr = db.create_transaction() 140 | try: 141 | fdb.tenant_management._create_tenant_impl(tr, b'tenant2', existence_check_marker) 142 | tr.commit().wait() 143 | except fdb.FDBError as e: 144 | tr.on_error(e).wait() 145 | 146 | # Test that tenant deletion only performs the existence check once 147 | fdb.tenant_management._delete_tenant_impl(db, b'tenant1', [], force_existence_check_maybe_committed=True) 148 | 149 | # An attempt to delete the tenant again should fail 150 | try: 151 | fdb.tenant_management.delete_tenant(db, b'tenant1') 152 | assert False 153 | except fdb.FDBError as e: 154 | assert e.code == 2131 # tenant not found 155 | 156 | # Using a transaction skips the existence check 157 | tr = db.create_transaction() 158 | fdb.tenant_management.delete_tenant(tr, b'tenant1') 159 | 160 | # Test that a concurrent tenant deletion doesn't interfere with the existence check logic 161 | tr = db.create_transaction() 162 | existence_check_marker = [] 163 | fdb.tenant_management._delete_tenant_impl(tr, b'tenant2', existence_check_marker) 164 | 165 | fdb.tenant_management.delete_tenant(db, b'tenant2') 166 | 167 | tr = db.create_transaction() 168 | try: 169 | fdb.tenant_management._delete_tenant_impl(tr, b'tenant2', existence_check_marker) 170 | tr.commit().wait() 171 | except fdb.FDBError as e: 172 | tr.on_error(e).wait() 173 | 174 | def test_tenants(db): 175 | test_tenant_tuple_name(db) 176 | test_tenant_operations(db) 177 | test_tenant_operation_retries(db) 178 | 179 | # Expect a cluster file as input. This test will write to the FDB cluster, so 180 | # be aware of potential side effects. 181 | if __name__ == '__main__': 182 | clusterFile = sys.argv[1] 183 | db = fdb.open(clusterFile) 184 | db.options.set_transaction_timeout(2000) # 2 seconds 185 | db.options.set_transaction_retry_limit(3) 186 | 187 | test_tenants(db) 188 | -------------------------------------------------------------------------------- /fdb-sys/Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | name = "fdb-sys" 3 | version = "0.4.0" 4 | edition = "2018" 5 | rust-version = "1.49" 6 | authors = ["fdb-rs Developers"] 7 | description = """ 8 | Binding to the C APIs for FoundationDB 9 | """ 10 | license = "MIT OR Apache-2.0" 11 | repository = "https://github.com/fdb-rs/fdb" 12 | keywords = ["foundationdb", "tokio"] 13 | categories = ["database", "external-ffi-bindings"] 14 | 15 | [package.metadata.docs.rs] 16 | features = ["fdb-7_1"] 17 | 18 | [features] 19 | default = [] 20 | fdb-6_3 = [] 21 | fdb-7_1 = [] 22 | 23 | [dependencies] 24 | 25 | [build-dependencies] 26 | bindgen = "0.59" 27 | -------------------------------------------------------------------------------- /fdb-sys/LICENSE-MIT: -------------------------------------------------------------------------------- 1 | Permission is hereby granted, free of charge, to any 2 | person obtaining a copy of this software and associated 3 | documentation files (the "Software"), to deal in the 4 | Software without restriction, including without 5 | limitation the rights to use, copy, modify, merge, 6 | publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software 8 | is furnished to do so, subject to the following 9 | conditions: 10 | 11 | The above copyright notice and this permission notice 12 | shall be included in all copies or substantial portions 13 | of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF 16 | ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED 17 | TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 18 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT 19 | SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY 20 | CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 21 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR 22 | IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 23 | DEALINGS IN THE SOFTWARE. 24 | -------------------------------------------------------------------------------- /fdb-sys/README.md: -------------------------------------------------------------------------------- 1 | # C API bindings for FoundationDB 2 | 3 | See [fdb](https://crates.io/crates/fdb) crate for a safe abstraction. 4 | 5 | ## License 6 | 7 | Licensed under either of 8 | 9 | * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or 10 | http://www.apache.org/licenses/LICENSE-2.0) 11 | * MIT license ([LICENSE-MIT](LICENSE-MIT) or 12 | http://opensource.org/licenses/MIT) 13 | 14 | at your option. 15 | 16 | ## Contribution 17 | 18 | Unless you explicitly state otherwise, any contribution intentionally 19 | submitted for inclusion in the work by you, as defined in the 20 | Apache-2.0 license, shall be dual licensed as above, without any 21 | additional terms or conditions. 22 | -------------------------------------------------------------------------------- /fdb-sys/build.rs: -------------------------------------------------------------------------------- 1 | // Make `cargo build` panic with the right message without warnings 2 | // when called without ` 3 | #![allow(unreachable_code)] 4 | #![allow(unused_assignments)] 5 | #![allow(unused_variables)] 6 | #![allow(unused_mut)] 7 | 8 | use std::env; 9 | use std::fs::File; 10 | use std::io::prelude::*; 11 | use std::path::PathBuf; 12 | 13 | #[cfg(not(any(feature = "fdb-6_3", feature = "fdb-7_1")))] 14 | const INCLUDE_PATH: &str = ""; 15 | 16 | #[cfg(feature = "fdb-6_3")] 17 | const INCLUDE_PATH: &str = "-I./include/630"; 18 | 19 | #[cfg(feature = "fdb-7_1")] 20 | const INCLUDE_PATH: &str = "-I./include/710"; 21 | 22 | fn main() { 23 | // Link against fdb_c. 24 | println!("cargo:rustc-link-lib=dylib=fdb_c"); 25 | 26 | if let Ok(link_search) = env::var("RUSTC_LINK_SEARCH_FDB_CLIENT_LIB") { 27 | println!("cargo:rustc-link-search=native={}", link_search); 28 | } 29 | 30 | let out_path = PathBuf::from(env::var("OUT_DIR").expect("OUT_DIR is not defined!")); 31 | 32 | // We need to have FDB_API_VERSION set to a constant so that 33 | // bindgen will generate a const value for it. We could try to 34 | // pass -DFDB_TRICKY_VERSION=630 to the driver and then '#define 35 | // FDB_API_VERSION FDB_TRICKY_VERSION', but bindgen isn't smart 36 | // enough to resolve that from the arguments. Instead, write out a 37 | // src/wrapper.h file with the chosen version instead. 38 | let mut api_version = 0; 39 | 40 | #[cfg(not(any(feature = "fdb-6_3", feature = "fdb-7_1")))] 41 | panic!("Please specify fdb-_ feature"); 42 | 43 | #[cfg(feature = "fdb-6_3")] 44 | { 45 | api_version = 630; 46 | } 47 | 48 | #[cfg(feature = "fdb-7_1")] 49 | { 50 | api_version = 710; 51 | } 52 | 53 | // Sigh, bindgen only takes a String for its header path, but 54 | // that's UTF-8 while PathBuf is OS-native... 55 | let wpath = out_path.join("wrapper.h"); 56 | let wrapper_path = wpath 57 | .to_str() 58 | .expect("couldn't convert wrapper PathBuf to String!"); 59 | 60 | let mut wrapper = File::create(wrapper_path).expect("couldn't create wrapper.h!"); 61 | wrapper 62 | .write_all( 63 | format!( 64 | "// This is used as `header_version` in\n\ 65 | // `fdb_select_api_version_impl`\n\ 66 | #define FDB_API_VERSION {}\n", 67 | api_version 68 | ) 69 | .as_bytes(), 70 | ) 71 | .expect("couldn't write wrapper.h!"); 72 | wrapper 73 | .write_all(b"#include \n") 74 | .expect("couldn't write wrapper.h!"); 75 | drop(wrapper); 76 | 77 | // Finish up by writing the actual bindings 78 | let bindings = bindgen::Builder::default() 79 | .clang_arg(INCLUDE_PATH) 80 | .header(wrapper_path) 81 | .generate_comments(true) 82 | .generate() 83 | .expect("Unable to generate FoundationDB bindings"); 84 | bindings 85 | .write_to_file(out_path.join("bindings.rs")) 86 | .expect("Couldn't write bindings!"); 87 | } 88 | -------------------------------------------------------------------------------- /fdb-sys/include/630/version.txt: -------------------------------------------------------------------------------- 1 | 6.3.15 2 | -------------------------------------------------------------------------------- /fdb-sys/include/710/fdb_c_types.h: -------------------------------------------------------------------------------- 1 | /* 2 | * fdb_c_types.h 3 | * 4 | * This source file is part of the FoundationDB open source project 5 | * 6 | * Copyright 2013-2022 Apple Inc. and the FoundationDB project authors 7 | * 8 | * Licensed under the Apache License, Version 2.0 (the "License"); 9 | * you may not use this file except in compliance with the License. 10 | * You may obtain a copy of the License at 11 | * 12 | * http://www.apache.org/licenses/LICENSE-2.0 13 | * 14 | * Unless required by applicable law or agreed to in writing, software 15 | * distributed under the License is distributed on an "AS IS" BASIS, 16 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17 | * See the License for the specific language governing permissions and 18 | * limitations under the License. 19 | */ 20 | 21 | #ifndef FDB_C_TYPES_H 22 | #define FDB_C_TYPES_H 23 | #pragma once 24 | 25 | #ifndef DLLEXPORT 26 | #define DLLEXPORT 27 | #endif 28 | 29 | #ifdef __cplusplus 30 | extern "C" { 31 | #endif 32 | 33 | /* Pointers to these opaque types represent objects in the FDB API */ 34 | typedef struct FDB_future FDBFuture; 35 | typedef struct FDB_result FDBResult; 36 | typedef struct FDB_cluster FDBCluster; 37 | typedef struct FDB_database FDBDatabase; 38 | typedef struct FDB_tenant FDBTenant; 39 | typedef struct FDB_transaction FDBTransaction; 40 | 41 | typedef int fdb_error_t; 42 | typedef int fdb_bool_t; 43 | 44 | #ifdef __cplusplus 45 | } 46 | #endif 47 | #endif 48 | -------------------------------------------------------------------------------- /fdb-sys/include/710/version.txt: -------------------------------------------------------------------------------- 1 | 7.1.3 2 | -------------------------------------------------------------------------------- /fdb-sys/src/lib.rs: -------------------------------------------------------------------------------- 1 | #![allow(clippy::unreadable_literal)] 2 | #![allow(deref_nullptr)] 3 | #![allow(non_camel_case_types)] 4 | #![allow(non_snake_case)] 5 | #![allow(non_upper_case_globals)] 6 | #![allow(unaligned_references)] 7 | include!(concat!(env!("OUT_DIR"), "/bindings.rs")); 8 | -------------------------------------------------------------------------------- /fdb/Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | name = "fdb" 3 | version = "0.4.0" 4 | edition = "2018" 5 | rust-version = "1.49" 6 | authors = ["fdb-rs Developers"] 7 | 8 | description = """ 9 | FoundationDB Client API for Tokio 10 | """ 11 | license = "MIT OR Apache-2.0" 12 | repository = "https://github.com/fdb-rs/fdb" 13 | keywords = ["foundationdb", "tokio"] 14 | categories = ["api-bindings", "database"] 15 | 16 | [package.metadata.docs.rs] 17 | features = ["fdb-7_1"] 18 | 19 | [features] 20 | default = [] 21 | fdb-6_3 = ["fdb-gen/fdb-6_3", "fdb-sys/fdb-6_3"] 22 | fdb-7_1 = ["fdb-gen/fdb-7_1", "fdb-sys/fdb-7_1"] 23 | 24 | [dependencies] 25 | bytes = "1" 26 | fdb-sys = { version = "0.4.0", path = "../fdb-sys", default-features = false } 27 | futures = "0.3" 28 | nom = "7" 29 | num-bigint = "0.4" 30 | num-traits = "0.2" 31 | parking_lot = "0.11" 32 | tokio = { version = "1", features = ["full"] } 33 | tokio-stream = "0.1" 34 | uuid = { version = "0.8", features = ["v4"] } 35 | 36 | [dev-dependencies] 37 | impls = "1" 38 | libc = "0.2" 39 | 40 | [build-dependencies] 41 | fdb-gen = { version = "0.4.0", path = "../fdb-gen", default-features = false } 42 | -------------------------------------------------------------------------------- /fdb/LICENSE-MIT: -------------------------------------------------------------------------------- 1 | Permission is hereby granted, free of charge, to any 2 | person obtaining a copy of this software and associated 3 | documentation files (the "Software"), to deal in the 4 | Software without restriction, including without 5 | limitation the rights to use, copy, modify, merge, 6 | publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software 8 | is furnished to do so, subject to the following 9 | conditions: 10 | 11 | The above copyright notice and this permission notice 12 | shall be included in all copies or substantial portions 13 | of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF 16 | ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED 17 | TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 18 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT 19 | SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY 20 | CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 21 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR 22 | IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 23 | DEALINGS IN THE SOFTWARE. 24 | -------------------------------------------------------------------------------- /fdb/README.md: -------------------------------------------------------------------------------- 1 | # FoundationDB Client API for Tokio 2 | 3 | [![crates.io][crates-badge]][crates-url] 4 | [![codecov][codecov-badge]][codecov-url] 5 | [![build][build-badge]][build-url] 6 | 7 | [crates-badge]: https://img.shields.io/crates/v/fdb.svg 8 | [crates-url]: https://crates.io/crates/fdb 9 | [codecov-badge]: https://codecov.io/gh/fdb-rs/fdb/branch/main/graph/badge.svg?token=3O9VXRK4JU 10 | [codecov-url]: https://codecov.io/gh/fdb-rs/fdb 11 | [build-badge]: https://img.shields.io/github/workflow/status/fdb-rs/fdb/push-6_3_23 12 | [build-url]: https://github.com/fdb-rs/fdb/actions/workflows/push-6_3_23.yml 13 | 14 | ## Getting Help 15 | 16 | First, see if the answer to your question can be found in the 17 | [Guides](https://fdb-rs.github.io/docs/getting-started/introduction/) 18 | or the [API documentation](https://docs.rs/fdb/latest/fdb/). If the 19 | answer is not there, you can [contact 20 | us](https://fdb-rs.github.io/docs/help/contact-us/). We would be happy 21 | to try to answer your question. 22 | 23 | You can find `main` branch API doumentation 24 | [here](https://fdb-rs.github.io/fdb/fdb/). 25 | 26 | ## Supported Rust Versions 27 | 28 | We follow Tokio project 29 | [MSRV](https://github.com/tokio-rs/tokio/#supported-rust-versions) 30 | (minimum supported rust version) policy. The current MSRV is 1.49.0. 31 | 32 | ## License 33 | 34 | Licensed under either of 35 | 36 | * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or 37 | http://www.apache.org/licenses/LICENSE-2.0) 38 | * MIT license ([LICENSE-MIT](LICENSE-MIT) or 39 | http://opensource.org/licenses/MIT) 40 | 41 | at your option. 42 | 43 | ## Contribution 44 | 45 | Unless you explicitly state otherwise, any contribution intentionally 46 | submitted for inclusion in the work by you, as defined in the 47 | Apache-2.0 license, shall be dual licensed as above, without any 48 | additional terms or conditions. 49 | -------------------------------------------------------------------------------- /fdb/build.rs: -------------------------------------------------------------------------------- 1 | use std::env; 2 | use std::fs::File; 3 | use std::io::prelude::*; 4 | use std::path::PathBuf; 5 | 6 | fn main() { 7 | let out_path = PathBuf::from(env::var("OUT_DIR").expect("OUT_DIR is undefined!")); 8 | let option_file = out_path.join("option.rs"); 9 | let mut options = String::new(); 10 | fdb_gen::emit(&mut options).expect("couldn't emit options.rs code!"); 11 | 12 | File::create(option_file) 13 | .expect("couldn't create option.rs!") 14 | .write_all(options.as_bytes()) 15 | .expect("couldn't write option.rs!"); 16 | } 17 | -------------------------------------------------------------------------------- /fdb/examples/get_committed_version.rs: -------------------------------------------------------------------------------- 1 | use bytes::Bytes; 2 | 3 | use fdb::error::FdbResult; 4 | use fdb::transaction::Transaction; 5 | 6 | use tokio::runtime::Runtime; 7 | 8 | use std::env; 9 | use std::error::Error; 10 | 11 | fn main() -> Result<(), Box> { 12 | let fdb_cluster_file = env::var("FDB_CLUSTER_FILE").expect("FDB_CLUSTER_FILE not defined!"); 13 | 14 | unsafe { 15 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 16 | fdb::start_network(); 17 | } 18 | 19 | let fdb_database = fdb::open_database(fdb_cluster_file)?; 20 | 21 | let rt = Runtime::new()?; 22 | 23 | let cloned_fdb_database = fdb_database.clone(); 24 | 25 | rt.block_on(async { 26 | let fdb_database = cloned_fdb_database; 27 | 28 | let committed_version = fdb_database 29 | .run(|tr| async move { 30 | tr.set(Bytes::from("hello"), Bytes::from("world")); 31 | Ok(unsafe { tr.get_committed_version() }) 32 | }) 33 | .await?; 34 | 35 | println!( 36 | "get_commited_version: {:?}", 37 | Into::>::into(committed_version)? 38 | ); 39 | 40 | Result::<(), Box>::Ok(()) 41 | })?; 42 | 43 | drop(fdb_database); 44 | 45 | unsafe { 46 | fdb::stop_network(); 47 | } 48 | 49 | Ok(()) 50 | } 51 | -------------------------------------------------------------------------------- /fdb/examples/get_range.rs: -------------------------------------------------------------------------------- 1 | use bytes::Bytes; 2 | 3 | use fdb::range::{Range, RangeOptions}; 4 | use fdb::transaction::{ReadTransaction, Transaction}; 5 | use fdb::KeySelector; 6 | 7 | use tokio::runtime::Runtime; 8 | use tokio_stream::StreamExt; 9 | 10 | use std::env; 11 | use std::error::Error; 12 | 13 | fn main() -> Result<(), Box> { 14 | let fdb_cluster_file = env::var("FDB_CLUSTER_FILE").expect("FDB_CLUSTER_FILE not defined!"); 15 | 16 | unsafe { 17 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 18 | fdb::start_network(); 19 | } 20 | 21 | let fdb_database = fdb::open_database(fdb_cluster_file)?; 22 | 23 | let rt = Runtime::new()?; 24 | 25 | let cloned_fdb_database = fdb_database.clone(); 26 | 27 | rt.block_on(async { 28 | let fdb_database = cloned_fdb_database; 29 | 30 | // Clear the database. 31 | fdb_database 32 | .run(|tr| async move { 33 | tr.clear_range(Range::new(Bytes::new(), Bytes::from_static(b"\xFF"))); 34 | 35 | Ok(()) 36 | }) 37 | .await?; 38 | 39 | // Set a few key values. 40 | fdb_database 41 | .run(|tr| async move { 42 | tr.set(Bytes::from("apple"), Bytes::from("foo")); 43 | tr.set(Bytes::from("cherry"), Bytes::from("baz")); 44 | tr.set(Bytes::from("banana"), Bytes::from("bar")); 45 | 46 | Ok(()) 47 | }) 48 | .await?; 49 | 50 | println!("non-snapshot range read"); 51 | 52 | fdb_database 53 | .run(|tr| async move { 54 | let mut range_stream = tr.get_range( 55 | KeySelector::first_greater_or_equal(Bytes::new()), 56 | KeySelector::first_greater_or_equal(Bytes::from_static(b"\xFF")), 57 | RangeOptions::default(), 58 | ); 59 | 60 | while let Some(x) = range_stream.next().await { 61 | let (key, value) = x?.into_parts(); 62 | println!( 63 | "{} is {}", 64 | String::from_utf8_lossy(&Bytes::from(key)[..]), 65 | String::from_utf8_lossy(&Bytes::from(value)[..]) 66 | ); 67 | } 68 | 69 | println!(); 70 | 71 | let mut range_stream = Range::new(Bytes::new(), Bytes::from_static(b"\xFF")) 72 | .into_stream(&tr, RangeOptions::default()); 73 | 74 | while let Some(x) = range_stream.next().await { 75 | let (key, value) = x?.into_parts(); 76 | println!( 77 | "{} is {}", 78 | String::from_utf8_lossy(&Bytes::from(key)[..]), 79 | String::from_utf8_lossy(&Bytes::from(value)[..]) 80 | ); 81 | } 82 | 83 | Ok(()) 84 | }) 85 | .await?; 86 | 87 | println!(); 88 | println!("snapshot range read"); 89 | 90 | fdb_database 91 | .read(|tr| async move { 92 | let mut range_stream = tr.get_range( 93 | KeySelector::first_greater_or_equal(Bytes::new()), 94 | KeySelector::first_greater_or_equal(Bytes::from_static(b"\xFF")), 95 | RangeOptions::default(), 96 | ); 97 | 98 | while let Some(x) = range_stream.next().await { 99 | let (key, value) = x?.into_parts(); 100 | println!( 101 | "{} is {}", 102 | String::from_utf8_lossy(&Bytes::from(key)[..]), 103 | String::from_utf8_lossy(&Bytes::from(value)[..]) 104 | ); 105 | } 106 | 107 | println!(); 108 | 109 | let mut range_stream = Range::new(Bytes::new(), Bytes::from_static(b"\xFF")) 110 | .into_stream(&tr, RangeOptions::default()); 111 | 112 | while let Some(x) = range_stream.next().await { 113 | let (key, value) = x?.into_parts(); 114 | println!( 115 | "{} is {}", 116 | String::from_utf8_lossy(&Bytes::from(key)[..]), 117 | String::from_utf8_lossy(&Bytes::from(value)[..]) 118 | ); 119 | } 120 | 121 | Ok(()) 122 | }) 123 | .await?; 124 | 125 | Result::<(), Box>::Ok(()) 126 | })?; 127 | 128 | drop(fdb_database); 129 | 130 | unsafe { 131 | fdb::stop_network(); 132 | } 133 | 134 | Ok(()) 135 | } 136 | -------------------------------------------------------------------------------- /fdb/examples/get_versionstamp.rs: -------------------------------------------------------------------------------- 1 | use bytes::Bytes; 2 | 3 | use fdb::range::RangeOptions; 4 | use fdb::subspace::Subspace; 5 | use fdb::transaction::{MutationType, Transaction}; 6 | use fdb::tuple::{Tuple, Versionstamp}; 7 | 8 | use tokio::runtime::Runtime; 9 | use tokio_stream::StreamExt; 10 | 11 | use std::env; 12 | use std::error::Error; 13 | 14 | fn main() -> Result<(), Box> { 15 | let fdb_cluster_file = env::var("FDB_CLUSTER_FILE").expect("FDB_CLUSTER_FILE not defined!"); 16 | 17 | unsafe { 18 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 19 | fdb::start_network(); 20 | } 21 | 22 | let fdb_database = fdb::open_database(fdb_cluster_file)?; 23 | 24 | let rt = Runtime::new()?; 25 | 26 | let cloned_fdb_database = fdb_database.clone(); 27 | 28 | rt.block_on(async { 29 | let fdb_database = cloned_fdb_database; 30 | 31 | let tr_version = fdb_database 32 | .run(|tr| async move { 33 | let t = { 34 | let mut tup = Tuple::new(); 35 | tup.add_string(String::from("prefix")); 36 | tup.add_versionstamp(Versionstamp::incomplete(0)); 37 | tup 38 | }; 39 | 40 | unsafe { 41 | tr.mutate( 42 | MutationType::SetVersionstampedKey, 43 | t.pack_with_versionstamp(Bytes::new())?, 44 | Bytes::new(), 45 | ); 46 | } 47 | 48 | Ok(unsafe { tr.get_versionstamp() }) 49 | }) 50 | .await? 51 | .get() 52 | .await?; 53 | 54 | let vs = fdb_database 55 | .run(|tr| async move { 56 | let subspace = Subspace::new(Bytes::new()).subspace(&{ 57 | let mut tup = Tuple::new(); 58 | tup.add_string("prefix".to_string()); 59 | tup 60 | }); 61 | 62 | let subspace_range = subspace.range(&Tuple::new()); 63 | 64 | let key = subspace_range 65 | .into_stream(&tr, RangeOptions::default()) 66 | .take(1) 67 | .next() 68 | .await 69 | .unwrap()? 70 | .into_key(); 71 | 72 | Ok(subspace 73 | .unpack(&key.into())? 74 | .get_versionstamp_ref(0)? 75 | .clone()) 76 | }) 77 | .await?; 78 | 79 | assert_eq!(vs, Versionstamp::complete(tr_version, 0)); 80 | 81 | Result::<(), Box>::Ok(()) 82 | })?; 83 | 84 | drop(fdb_database); 85 | 86 | unsafe { 87 | fdb::stop_network(); 88 | } 89 | 90 | Ok(()) 91 | } 92 | -------------------------------------------------------------------------------- /fdb/examples/hello_world.rs: -------------------------------------------------------------------------------- 1 | use bytes::Bytes; 2 | 3 | use fdb::transaction::{ReadTransaction, Transaction}; 4 | 5 | use tokio::runtime::Runtime; 6 | 7 | use std::env; 8 | use std::error::Error; 9 | 10 | fn main() -> Result<(), Box> { 11 | let fdb_cluster_file = env::var("FDB_CLUSTER_FILE").expect("FDB_CLUSTER_FILE not defined!"); 12 | 13 | unsafe { 14 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 15 | fdb::start_network(); 16 | } 17 | 18 | let fdb_database = fdb::open_database(fdb_cluster_file)?; 19 | 20 | let rt = Runtime::new()?; 21 | 22 | let cloned_fdb_database = fdb_database.clone(); 23 | 24 | rt.block_on(async { 25 | let fdb_database = cloned_fdb_database; 26 | 27 | fdb_database 28 | .run(|tr| async move { 29 | tr.set(Bytes::from("hello"), Bytes::from("world")); 30 | Ok(()) 31 | }) 32 | .await?; 33 | 34 | let ret = fdb_database 35 | .read(|rtr| async move { rtr.get(Bytes::from("hello")).await }) 36 | .await? 37 | .expect("Key not found"); 38 | 39 | println!( 40 | "snapshot get: hello, {}", 41 | String::from_utf8_lossy(&Bytes::from(ret)[..]) 42 | ); 43 | 44 | let ret = fdb_database 45 | .run(|tr| async move { tr.get(Bytes::from("hello")).await }) 46 | .await? 47 | .expect("Key not found"); 48 | 49 | println!( 50 | "non-snapshot get: hello, {}", 51 | String::from_utf8_lossy(&Bytes::from(ret)[..]) 52 | ); 53 | 54 | Result::<(), Box>::Ok(()) 55 | })?; 56 | 57 | drop(fdb_database); 58 | 59 | unsafe { 60 | fdb::stop_network(); 61 | } 62 | 63 | Ok(()) 64 | } 65 | -------------------------------------------------------------------------------- /fdb/examples/open_database.rs: -------------------------------------------------------------------------------- 1 | fn main() { 2 | unsafe { 3 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 4 | fdb::start_network(); 5 | } 6 | 7 | // Default cluster file specified by environment variable 8 | // FDB_CLUSTER_FILE is opened. 9 | let fdb_database = fdb::open_database("").unwrap(); 10 | 11 | drop(fdb_database); 12 | 13 | unsafe { 14 | fdb::stop_network(); 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /fdb/examples/watch.rs: -------------------------------------------------------------------------------- 1 | use bytes::Bytes; 2 | 3 | use fdb::transaction::Transaction; 4 | 5 | use tokio::runtime::Runtime; 6 | 7 | use std::env; 8 | use std::error::Error; 9 | 10 | fn main() -> Result<(), Box> { 11 | let fdb_cluster_file = env::var("FDB_CLUSTER_FILE").expect("FDB_CLUSTER_FILE not defined!"); 12 | 13 | unsafe { 14 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 15 | fdb::start_network(); 16 | } 17 | 18 | let fdb_database = fdb::open_database(fdb_cluster_file)?; 19 | 20 | let rt = Runtime::new()?; 21 | 22 | let cloned_fdb_database = fdb_database.clone(); 23 | 24 | rt.block_on(async { 25 | let fdb_database = cloned_fdb_database; 26 | 27 | fdb_database 28 | .run(|tr| async move { 29 | tr.set(Bytes::from("k"), Bytes::from("0")); 30 | Ok(()) 31 | }) 32 | .await?; 33 | 34 | fdb_database 35 | .run(|tr| async move { 36 | // w0 = (grv, k, 0) 37 | let w0 = tr.watch(Bytes::from("k")); 38 | 39 | // The transaction that creates a watch needs to be 40 | // committed in order for the watch to be set 41 | // *globally*. A watch that isn’t committed can only 42 | // be triggered by modifications that happen in the 43 | // same transaction as the watch. 44 | tr.set(Bytes::from("k"), Bytes::from("1")); 45 | 46 | // We are triggering w0 locally. 47 | w0.await?; 48 | 49 | println!("w0 is ready"); 50 | 51 | Ok(()) 52 | }) 53 | .await?; 54 | 55 | // `w1` is a global watch 56 | let w1 = fdb_database 57 | .run(|tr| async move { 58 | // w1 = (grv, k, 1) 59 | let w1 = tr.watch(Bytes::from("k")); 60 | Ok(w1) 61 | }) 62 | .await?; 63 | 64 | // // If we do `w1.await` here now, we will block. 65 | // // 66 | // w1.await?; 67 | 68 | // The transaction below, will resolve the `w1` future as 69 | // we'll be updating `k`. 70 | fdb_database 71 | .run(|tr| async move { 72 | tr.set(Bytes::from("k"), Bytes::from("2")); 73 | Ok(()) 74 | }) 75 | .await?; 76 | 77 | // won't block. 78 | w1.await?; 79 | 80 | println!("w1 is ready"); 81 | 82 | Result::<(), Box>::Ok(()) 83 | })?; 84 | 85 | drop(fdb_database); 86 | 87 | unsafe { 88 | fdb::stop_network(); 89 | } 90 | 91 | Ok(()) 92 | } 93 | -------------------------------------------------------------------------------- /fdb/sismic/mapped_range_result_state_machine.yaml: -------------------------------------------------------------------------------- 1 | # NOTE: Even though there are two final states - Error and Done, 2 | # plantuml shows it as one done state. 3 | statechart: 4 | name: MappedRangeResultStateMachine Statechart 5 | preamble: | 6 | **Context:** Root State 7 | ----------------------- 8 | Immutable: 9 | transaction 10 | snapshot 11 | mode 12 | reverse 13 | mapper 14 | 15 | Mutable: 16 | iteration 17 | limit 18 | begin_sel 19 | end_sel 20 | --------------------------- 21 | **Context:** Fetching State 22 | --------------------------- 23 | Immutable: 24 | fdb_future_key_value_array 25 | ----------------------------------------------- 26 | **Context:** MappedKeyValueArrayAvailable State 27 | ----------------------------------------------- 28 | Immutable: 29 | count 30 | mkvs 31 | more 32 | 33 | Mutable: 34 | index 35 | -------------------------------- 36 | **Context:** Error State (final) 37 | -------------------------------- 38 | Immutable: 39 | fdb_error 40 | root state: 41 | name: MappedRangeResultStateMachine 42 | initial: Fetching 43 | states: 44 | - name: Fetching 45 | transitions: 46 | - event: FetchOk{mkvs, index, count, more} 47 | target: MappedKeyValueArrayAvailable 48 | action: | 49 | update limit, iteration, begin_sel, end_sel 50 | - event: FetchDone 51 | target: Done 52 | - event: FetchError{fdb_error} 53 | target: Error 54 | - name: MappedKeyValueArrayAvailable 55 | transitions: 56 | - event: FetchNextBatch{fdb_future_mapped_key_value_array} 57 | target: Fetching 58 | - event: FetchDone 59 | target: Done 60 | - name: Error 61 | type: final 62 | - name: Done 63 | type: final 64 | -------------------------------------------------------------------------------- /fdb/sismic/range_result_state_machine.yaml: -------------------------------------------------------------------------------- 1 | # NOTE: Even though there are two final states - Error and Done, 2 | # plantuml shows it as one done state. 3 | statechart: 4 | name: RangeResultStateMachine Statechart 5 | preamble: | 6 | **Context:** Root State 7 | ----------------------- 8 | Immutable: 9 | transaction 10 | snapshot 11 | mode 12 | reverse 13 | 14 | Mutable: 15 | iteration 16 | limit 17 | begin_sel 18 | end_sel 19 | --------------------------- 20 | **Context:** Fetching State 21 | --------------------------- 22 | Immutable: 23 | fdb_future_key_value_array 24 | ----------------------------------------- 25 | **Context:** KeyValueArrayAvailable State 26 | ----------------------------------------- 27 | Immutable: 28 | count 29 | kvs 30 | more 31 | 32 | Mutable: 33 | index 34 | -------------------------------- 35 | **Context:** Error State (final) 36 | -------------------------------- 37 | Immutable: 38 | fdb_error 39 | root state: 40 | name: RangeResultStateMachine 41 | initial: Fetching 42 | states: 43 | - name: Fetching 44 | transitions: 45 | - event: FetchOk{kvs, index, count, more} 46 | target: KeyValueArrayAvailable 47 | action: | 48 | update limit, iteration, begin_sel, end_sel 49 | - event: FetchDone 50 | target: Done 51 | - event: FetchError{fdb_error} 52 | target: Error 53 | - name: KeyValueArrayAvailable 54 | transitions: 55 | - event: FetchNextBatch{fdb_future_key_value_array} 56 | target: Fetching 57 | - event: FetchDone 58 | target: Done 59 | - name: Error 60 | type: final 61 | - name: Done 62 | type: final 63 | -------------------------------------------------------------------------------- /fdb/src/database/mod.rs: -------------------------------------------------------------------------------- 1 | //! Provides [`FdbDatabase`] type for working with FDB Database. 2 | //! 3 | //! Clients operating on a [`FdbDatabase`] should, in most cases use the 4 | //! [`run`] method. This implements a proper retry loop around the 5 | //! work that needs to be done and, assure 6 | //! that [`commit`] has returned successfully before returning. 7 | //! 8 | //! [`run`]: FdbDatabase::run 9 | //! [`commit`]: crate::transaction::Transaction::commit 10 | 11 | mod fdb_database; 12 | 13 | #[doc(hidden)] 14 | pub mod open_database; 15 | 16 | pub use crate::option::DatabaseOption; 17 | 18 | pub use fdb_database::FdbDatabase; 19 | -------------------------------------------------------------------------------- /fdb/src/database/open_database.rs: -------------------------------------------------------------------------------- 1 | use std::ffi::CString; 2 | use std::path::Path; 3 | use std::ptr::{self, NonNull}; 4 | use std::sync::Arc; 5 | 6 | use crate::database::FdbDatabase; 7 | use crate::error::{check, FdbError, FdbResult, DATABASE_OPEN}; 8 | 9 | /// Returns [`FdbDatabase`] handle to the FDB cluster identified by 10 | /// the provided cluster file. 11 | /// 12 | /// If no cluster file is passed, FDB automatically [determines a 13 | /// cluster file] with which to connect to a cluster. 14 | /// 15 | /// A single client can use this function multiple times to connect to 16 | /// different clusters simultaneously, with each invocation requiring 17 | /// its own cluster file. 18 | /// 19 | /// # Note 20 | /// 21 | /// The caller *must* ensure that [`FdbDatabase`] stays alive during 22 | /// the lifetime of any transactions or futures that [`FdbDatabase`] 23 | /// creates. 24 | /// 25 | /// [determines a cluster file]: https://apple.github.io/foundationdb/administration.html#specifying-a-cluster-file 26 | pub fn open_database

(cluster_file_path: P) -> FdbResult 27 | where 28 | P: AsRef, 29 | { 30 | let path = CString::new( 31 | cluster_file_path 32 | .as_ref() 33 | .to_str() 34 | .ok_or_else(|| FdbError::new(DATABASE_OPEN))?, 35 | ) 36 | .map_err(|_| FdbError::new(DATABASE_OPEN))?; 37 | 38 | // `path_ptr` is valid till we do `drop(path)`. 39 | let path_ptr = path.as_ptr(); 40 | 41 | let mut v = ptr::null_mut(); 42 | 43 | let err = unsafe { fdb_sys::fdb_create_database(path_ptr, &mut v) }; 44 | 45 | drop(path); 46 | 47 | // At this stage, we either have an error or a valid v. 48 | check(err)?; 49 | 50 | Ok(FdbDatabase::new(Some(Arc::new(NonNull::new(v).expect( 51 | "fdb_create_database returned null, but did not return an error", 52 | ))))) 53 | } 54 | -------------------------------------------------------------------------------- /fdb/src/error.rs: -------------------------------------------------------------------------------- 1 | //! Provides [`FdbError`] type, [`FdbResult`] type alias and error 2 | //! constants. 3 | 4 | use std::convert::TryInto; 5 | use std::error::Error; 6 | use std::fmt::{self, Display}; 7 | 8 | use crate::option::ErrorPredicate; 9 | 10 | /// Error type for this crate. 11 | /// 12 | /// Internally it wraps FDB [Error Codes]. Error codes from 100 thru' 13 | /// 999 is generated by the binding layer and not the C API. 14 | /// 15 | /// [Error Codes]: https://apple.github.io/foundationdb/api-error-codes.html 16 | // 17 | // NOTE: 0 cannot be used for `error_code`. 18 | // 19 | // 100 - `database` module 20 | // 110 - `tuple` module 21 | // 120 - `subspace` module 22 | #[derive(Copy, Clone, Debug, PartialEq)] 23 | pub struct FdbError { 24 | /// FoundationDB error code `fdb_error_t` 25 | error_code: i32, 26 | } 27 | 28 | /// Error occurred while opening database. 29 | pub const DATABASE_OPEN: i32 = 100; 30 | 31 | /// Error occurred while getting a value from the tuple. 32 | pub const TUPLE_GET: i32 = 110; 33 | 34 | /// Error occurred extracting a [`Tuple`] value from [`Bytes`]. 35 | /// 36 | /// [`Tuple`]: crate::tuple::Tuple 37 | /// [`Bytes`]: bytes::Bytes 38 | pub const TUPLE_FROM_BYTES: i32 = 111; 39 | 40 | /// Error occured when trying to pack [`Tuple`] containing an 41 | /// incomplete [`Versionstamp`]. No incomplete [`Versionstamp`] found. 42 | /// 43 | /// [`Tuple`]: crate::tuple::Tuple 44 | /// [`Versionstamp`]: crate::tuple::Versionstamp 45 | pub const TUPLE_PACK_WITH_VERSIONSTAMP_NOT_FOUND: i32 = 112; 46 | 47 | /// Error occured when trying to pack [`Tuple`] containing an 48 | /// incomplete [`Versionstamp`]. Multiple incomplete [`Versionstamp`] 49 | /// found. 50 | /// 51 | /// [`Tuple`]: crate::tuple::Tuple 52 | /// [`Versionstamp`]: crate::tuple::Versionstamp 53 | pub const TUPLE_PACK_WITH_VERSIONSTAMP_MULTIPLE_FOUND: i32 = 113; 54 | 55 | /// Error occurred when calling [`strinc`], as the `prefix` supplied 56 | /// is either empty or contains only `0xFF`. 57 | /// 58 | /// [`strinc`]: crate::tuple::key_util::strinc 59 | pub const TUPLE_KEY_UTIL_STRINC_ERROR: i32 = 114; 60 | 61 | /// Error occured when trying to pack [`Subspace`] containing an 62 | /// incomplete [`Versionstamp`]. Prefix contains an incomplete 63 | /// [`Versionstamp`], which is not allowed. 64 | /// 65 | /// [`Subspace`]: crate::subspace::Subspace 66 | /// [`Versionstamp`]: crate::tuple::Versionstamp 67 | pub const SUBSPACE_PACK_WITH_VERSIONSTAMP_PREFIX_INCOMPLETE: i32 = 120; 68 | 69 | /// Error occured when trying to unpack a key. The provided key is not 70 | /// contained in the [`Subspace`]. 71 | /// 72 | /// [`Subspace`]: crate::subspace::Subspace 73 | pub const SUBSPACE_UNPACK_KEY_MISMATCH: i32 = 121; 74 | 75 | /// Alias for [`Result`]`` 76 | /// 77 | /// [`Result`]: std::result::Result 78 | /// [`FdbError`]: crate::error::FdbError 79 | pub type FdbResult = Result; 80 | 81 | impl FdbError { 82 | /// Create new [`FdbError`] 83 | pub fn new(err: i32) -> FdbError { 84 | FdbError { error_code: err } 85 | } 86 | 87 | /// Returns raw FDB error code 88 | pub fn code(self) -> i32 { 89 | self.error_code 90 | } 91 | 92 | /// Returns `true` if the error indicates the operations in the 93 | /// transactions should be retried because of transient error. 94 | pub fn is_retryable(&self) -> bool { 95 | unsafe { 96 | // `FDB_ERROR_PREDICATE_RETRYABLE` has a value `50000` 97 | // which can safely be converted into an `i32`. 98 | // 99 | // non-zero is `true`. 100 | fdb_sys::fdb_error_predicate( 101 | ErrorPredicate::Retryable.code().try_into().unwrap(), 102 | self.error_code, 103 | ) != 0 104 | } 105 | } 106 | 107 | /// Returns true if the error indicates the transaction may have 108 | /// succeeded, though not in a way the system can verify. 109 | pub fn is_maybe_committed(&self) -> bool { 110 | unsafe { 111 | // `FDB_ERROR_PREDICATE_MAYBE_COMMITTED` has a value 112 | // `50001` which can safely be converted into an `i32`. 113 | // 114 | // non-zero is `true`. 115 | fdb_sys::fdb_error_predicate( 116 | ErrorPredicate::MaybeCommitted.code().try_into().unwrap(), 117 | self.error_code, 118 | ) != 0 119 | } 120 | } 121 | 122 | /// Returns `true` if the error indicates the transaction has not 123 | /// committed, though in a way that can be retried. 124 | pub fn is_retryable_not_committed(&self) -> bool { 125 | unsafe { 126 | // `FDB_ERROR_PREDICATE_RETRYABLE_NOT_COMMITTED` has a 127 | // value `50002` which can safely be converted into an 128 | // `i32`. 129 | // 130 | // non-zero is `true`. 131 | fdb_sys::fdb_error_predicate( 132 | ErrorPredicate::RetryableNotCommitted 133 | .code() 134 | .try_into() 135 | .unwrap(), 136 | self.error_code, 137 | ) != 0 138 | } 139 | } 140 | 141 | /// Returns `true` if the error is from a layer. Returns `false` 142 | /// if the error is a [C binding] error 143 | /// 144 | /// [C binding]: https://apple.github.io/foundationdb/api-error-codes.html 145 | pub(crate) fn layer_error(e: i32) -> bool { 146 | (100..=999).contains(&e) 147 | } 148 | } 149 | 150 | impl Error for FdbError {} 151 | 152 | impl Display for FdbError { 153 | fn fmt<'a>(&self, f: &mut fmt::Formatter<'a>) -> fmt::Result { 154 | write!(f, "{:?}", self) 155 | } 156 | } 157 | 158 | /// Converts `fdb_error_t` to `FdbResult` 159 | pub(crate) fn check(err: fdb_sys::fdb_error_t) -> FdbResult<()> { 160 | if err == 0 { 161 | Ok(()) 162 | } else { 163 | Err(FdbError::new(err)) 164 | } 165 | } 166 | -------------------------------------------------------------------------------- /fdb/src/fdb.rs: -------------------------------------------------------------------------------- 1 | //! Starting point for accessing FoundationDB 2 | use parking_lot::{Once, OnceState}; 3 | 4 | use std::thread::{self, JoinHandle}; 5 | 6 | use crate::error::{check, FdbResult}; 7 | use crate::option::NetworkOption; 8 | 9 | static SELECT_API_VERSION_INIT: Once = Once::new(); 10 | 11 | /// Select the version of the client API. 12 | /// 13 | /// # Panic 14 | /// 15 | /// This will panic if the version is not supported by the 16 | /// implementation of the API. 17 | /// 18 | /// As only one version can be selected for the lifetime of the 19 | /// process, calling this function more than once will also result in 20 | /// a panic. 21 | /// 22 | /// # Safety 23 | /// 24 | /// This API is part of FDB client setup. 25 | /// 26 | /// # Warning 27 | /// 28 | /// When using the multi-version client API, setting an API version 29 | /// that is not supported by a particular client library will prevent 30 | /// that client from being used to connect to the cluster. 31 | /// 32 | /// In particular, you should not advance the API version of your 33 | /// application after upgrading your client **until** the cluster has 34 | /// also been upgraded. 35 | pub unsafe fn select_api_version(version: i32) { 36 | if SELECT_API_VERSION_INIT.state() == OnceState::New { 37 | SELECT_API_VERSION_INIT.call_once(|| { 38 | // run initialization here 39 | check(fdb_sys::fdb_select_api_version_impl( 40 | version, 41 | // `bindgen` defaults `FDB_API_VERSION` to `u32` 42 | fdb_sys::FDB_API_VERSION as i32, 43 | )) 44 | .unwrap_or_else(|_| { 45 | panic!("Unable to call select_api_version for version {}", version) 46 | }); 47 | }); 48 | } else { 49 | panic!("select_api_version(...) was previously called!"); 50 | } 51 | } 52 | 53 | /// Set global options for the [FDB API]. 54 | /// 55 | /// # Safety 56 | /// 57 | /// This API is part of FDB client setup. 58 | /// 59 | /// [FDB API]: crate 60 | pub unsafe fn set_network_option(option: NetworkOption) -> FdbResult<()> { 61 | option.apply() 62 | } 63 | 64 | static mut FDB_NETWORK_THREAD: Option>> = None; 65 | 66 | // `FDB_NETWORK_STARTED` is set to `true` in `main` thread, and 67 | // `FDB_NETWORK_STOPPED` is set to `true`, in `fdb-network-thread`. 68 | static mut FDB_NETWORK_STARTED: bool = false; 69 | static mut FDB_NETWORK_STOPPED: bool = false; 70 | 71 | /// Initializes FDB network. 72 | /// 73 | /// # Safety 74 | /// 75 | /// This API is part of FDB client setup. 76 | pub unsafe fn start_network() { 77 | if FDB_NETWORK_STOPPED { 78 | panic!("Network has been stopped and cannot be started"); 79 | } 80 | 81 | if FDB_NETWORK_STARTED { 82 | return; 83 | } 84 | 85 | check(fdb_sys::fdb_setup_network()).unwrap_or_else(|e| { 86 | panic!("fdb_sys::fdb_setup_network() failed with error {:?}", e); 87 | }); 88 | 89 | FDB_NETWORK_STARTED = true; 90 | 91 | FDB_NETWORK_THREAD = Some( 92 | thread::Builder::new() 93 | .name("fdb-network-thread".into()) 94 | .spawn(|| { 95 | let res = check(fdb_sys::fdb_run_network()); 96 | FDB_NETWORK_STOPPED = true; 97 | res 98 | }) 99 | .unwrap_or_else(|e| { 100 | panic!("unable to create fdb-network-thread: error = {}", e); 101 | }), 102 | ); 103 | } 104 | 105 | /// Stops the FDB networking engine. 106 | /// 107 | /// # Safety 108 | /// 109 | /// This API is part of FDB client setup. 110 | pub unsafe fn stop_network() { 111 | if !FDB_NETWORK_STARTED { 112 | panic!("Trying to stop the network, before network has been started"); 113 | }; 114 | 115 | check(fdb_sys::fdb_stop_network()).unwrap_or_else(|e| { 116 | panic!("fdb_sys::fdb_stop_network() failed with error {:?}", e); 117 | }); 118 | 119 | FDB_NETWORK_THREAD 120 | .take() 121 | .unwrap() 122 | .join() 123 | .unwrap_or_else(|e| { 124 | panic!("failed to join on fdb-network-thread: error {:?}", e); 125 | }) 126 | .unwrap_or_else(|e| { 127 | panic!("fdb_sys::fdb_run_network() failed with error {:?}", e); 128 | }); 129 | } 130 | -------------------------------------------------------------------------------- /fdb/src/key_value.rs: -------------------------------------------------------------------------------- 1 | //! Key, value and selector types used to access FoundationDB 2 | use bytes::Bytes; 3 | 4 | /// [`Key`] represents a FDB key, a lexicographically-ordered sequence 5 | /// of bytes. 6 | /// 7 | /// [`Key`] can be converted from and into [`Bytes`]. 8 | #[derive(Clone, Debug, PartialEq)] 9 | pub struct Key(Bytes); 10 | 11 | impl From for Key { 12 | fn from(b: Bytes) -> Key { 13 | Key(b) 14 | } 15 | } 16 | 17 | impl From for Bytes { 18 | fn from(k: Key) -> Bytes { 19 | k.0 20 | } 21 | } 22 | 23 | /// [`Value`] represents a value of an FDB [`Key`] and is a sequence 24 | /// of bytes. 25 | /// 26 | /// [`Value`] can be converted from and into [`Bytes`]. 27 | #[derive(Clone, Debug, PartialEq)] 28 | pub struct Value(Bytes); 29 | 30 | impl From for Value { 31 | fn from(b: Bytes) -> Value { 32 | Value(b) 33 | } 34 | } 35 | 36 | impl From for Bytes { 37 | fn from(v: Value) -> Bytes { 38 | v.0 39 | } 40 | } 41 | 42 | /// A key/value pair. 43 | /// 44 | /// Range read operations on FDB return [`KeyValue`]s. 45 | #[derive(Clone, Debug)] 46 | pub struct KeyValue { 47 | key: Key, 48 | value: Value, 49 | } 50 | 51 | impl KeyValue { 52 | /// Gets a reference to [`Key`] from [`KeyValue`]. 53 | pub fn get_key_ref(&self) -> &Key { 54 | &self.key 55 | } 56 | 57 | /// Gets a reference to [`Value`] from [`KeyValue`]. 58 | pub fn get_value_ref(&self) -> &Value { 59 | &self.value 60 | } 61 | 62 | /// Extract [`Key`] from [`KeyValue`]. 63 | pub fn into_key(self) -> Key { 64 | self.key 65 | } 66 | 67 | /// Extract [`Value`] from [`KeyValue`]. 68 | pub fn into_value(self) -> Value { 69 | self.value 70 | } 71 | 72 | /// Extract [`Key`] and [`Value`] from [`KeyValue`]. 73 | pub fn into_parts(self) -> (Key, Value) { 74 | (self.key, self.value) 75 | } 76 | 77 | pub(crate) fn new(key: Key, value: Value) -> KeyValue { 78 | KeyValue { key, value } 79 | } 80 | } 81 | 82 | /// [`KeySelector`] identifies a particular key in the database. 83 | /// 84 | /// FDB's lexicographically ordered data model permits finding keys 85 | /// based on their order (for example, finding the first key in the 86 | /// database greater than a given key). Key selectors represent a 87 | /// description of a key in the database that could be resolved to an 88 | /// actual key by transaction's [`get_key`] or used directly as the 89 | /// beginning or end of a range in transaction's [`get_range`]. 90 | /// 91 | /// For more about how key selectors work in practive, see the [`key 92 | /// selector`] documentation. Note that the way key selectors are 93 | /// resolved is somewhat non-intuitive, so users who wish to use a key 94 | /// selector other than the default ones described below should 95 | /// probably consult that documentation before proceeding. 96 | /// 97 | /// Generally one of the following methods should be used to construct 98 | /// a [`KeySelector`]. 99 | /// - [`last_less_than`] 100 | /// - [`last_less_or_equal`] 101 | /// - [`first_greater_than`] 102 | /// - [`first_greater_or_equal`] 103 | /// 104 | /// This is an *immutable* type. The `add(i32)` call does not modify 105 | /// internal state, but returns a new value. 106 | /// 107 | /// [`get_key`]: crate::transaction::ReadTransaction::get_key 108 | /// [`get_range`]: crate::transaction::ReadTransaction::get_range 109 | /// [`key selector`]: https://apple.github.io/foundationdb/developer-guide.html#key-selectors 110 | /// [`last_less_than`]: KeySelector::last_less_than 111 | /// [`last_less_or_equal`]: KeySelector::last_less_or_equal 112 | /// [`first_greater_than`]: KeySelector::first_greater_than 113 | /// [`first_greater_or_equal`]: KeySelector::first_greater_or_equal 114 | #[derive(Clone, Debug)] 115 | pub struct KeySelector { 116 | key: Key, 117 | or_equal: bool, 118 | offset: i32, 119 | } 120 | 121 | impl KeySelector { 122 | /// Create a new [`KeySelector`] from the given parameters. 123 | pub fn new(key: impl Into, or_equal: bool, offset: i32) -> KeySelector { 124 | KeySelector { 125 | key: key.into(), 126 | or_equal, 127 | offset, 128 | } 129 | } 130 | 131 | /// Returns a new [`KeySelector`] offset by a given number of keys 132 | /// from this one. 133 | #[allow(clippy::should_implement_trait)] 134 | pub fn add(self, offset: i32) -> KeySelector { 135 | KeySelector::new(self.key, self.or_equal, self.offset + offset) 136 | } 137 | 138 | /// Creates a [`KeySelector`] that picks the first key greater 139 | /// than or equal to the parameter. 140 | pub fn first_greater_or_equal(key: impl Into) -> KeySelector { 141 | KeySelector::new(key, false, 1) 142 | } 143 | 144 | /// Creates a [`KeySelector`] that picks the first key greater 145 | /// than or equal to the parameter. 146 | pub fn first_greater_than(key: impl Into) -> KeySelector { 147 | KeySelector::new(key, true, 1) 148 | } 149 | 150 | /// Returns a reference to the key that serves as the anchor for 151 | /// this [`KeySelector`]. 152 | pub fn get_key(&self) -> &Key { 153 | &self.key 154 | } 155 | 156 | /// Returns the key offset parameter for this [`KeySelector`]. 157 | pub fn get_offset(&self) -> i32 { 158 | self.offset 159 | } 160 | 161 | /// Creates a [`KeySelector`] that picks the last key less than or 162 | /// equal to the parameter. 163 | pub fn last_less_or_equal(key: impl Into) -> KeySelector { 164 | KeySelector::new(key, true, 0) 165 | } 166 | 167 | /// Creates a [`KeySelector`] that picks the last key less than the parameter. 168 | pub fn last_less_than(key: impl Into) -> KeySelector { 169 | KeySelector::new(key, false, 0) 170 | } 171 | 172 | pub(crate) fn deconstruct(self) -> (Key, bool, i32) { 173 | let KeySelector { 174 | key, 175 | or_equal, 176 | offset, 177 | } = self; 178 | (key, or_equal, offset) 179 | } 180 | } 181 | -------------------------------------------------------------------------------- /fdb/src/lib.rs: -------------------------------------------------------------------------------- 1 | #![warn( 2 | missing_debug_implementations, 3 | missing_docs, 4 | rust_2018_idioms, 5 | unreachable_pub 6 | )] 7 | 8 | //! FoundationDB Client API for Tokio 9 | //! 10 | //! Guide level documentation is on our [website]. You will find API 11 | //! level documentation here. 12 | //! 13 | //! [website]: https://fdb-rs.github.io/docs/getting-started/introduction/ 14 | //! 15 | //! A minimal example to get started. 16 | //! 17 | //! ```no_run 18 | //! use tokio::runtime::Runtime; 19 | //! 20 | //! use std::env; 21 | //! use std::error::Error 22 | //! 23 | //! fn main() -> Result<(), Box> { 24 | //! let fdb_cluster_file = env::var("FDB_CLUSTER_FILE").expect("FDB_CLUSTER_FILE not defined!"); 25 | //! 26 | //! unsafe { 27 | //! fdb::select_api_version(710); 28 | //! fdb::start_network(); 29 | //! } 30 | //! 31 | //! let fdb_database = fdb::open_database(fdb_cluster_file)?; 32 | //! 33 | //! let rt = Runtime::new()?; 34 | //! 35 | //! let cloned_fdb_database = fdb_database.clone(); 36 | //! 37 | //! rt.block_on(async { 38 | //! let fdb_database = cloned_fdb_database; 39 | //! 40 | //! // your main async app here 41 | //! 42 | //! Result::<(), Box>::Ok(()) 43 | //! })?; 44 | //! 45 | //! drop(fdb_database); 46 | //! 47 | //! unsafe { 48 | //! fdb::stop_network(); 49 | //! } 50 | //! 51 | //! Ok(()) 52 | //! } 53 | //! ``` 54 | 55 | mod fdb; 56 | mod key_value; 57 | mod option; 58 | 59 | #[cfg(feature = "fdb-7_1")] 60 | mod mapped_key_value; 61 | 62 | #[cfg(feature = "fdb-7_1")] 63 | mod mapped_range; 64 | 65 | pub mod database; 66 | pub mod error; 67 | pub mod future; 68 | pub mod range; 69 | pub mod subspace; 70 | pub mod transaction; 71 | pub mod tuple; 72 | 73 | #[cfg(feature = "fdb-7_1")] 74 | pub mod tenant; 75 | 76 | /// Maximum API version supported by the client 77 | pub use fdb_sys::FDB_API_VERSION; 78 | 79 | pub use crate::fdb::{select_api_version, set_network_option, start_network, stop_network}; 80 | 81 | pub use crate::key_value::{Key, KeySelector, KeyValue, Value}; 82 | 83 | pub use crate::database::open_database::open_database; 84 | 85 | pub use crate::option::NetworkOption; 86 | 87 | #[cfg(feature = "fdb-7_1")] 88 | pub use crate::mapped_key_value::{MappedKeyValue, Mapper}; 89 | 90 | #[cfg(feature = "fdb-7_1")] 91 | pub use crate::tenant::tenant_inner::Tenant; 92 | -------------------------------------------------------------------------------- /fdb/src/mapped_key_value.rs: -------------------------------------------------------------------------------- 1 | //! Mapper and mapped key-value types 2 | 3 | use crate::range::Range; 4 | use crate::tuple::Tuple; 5 | use crate::KeyValue; 6 | 7 | /// [`Mapper`] represents the behaviour of a mapped range read. 8 | /// 9 | /// [`Mapper`] can be converted from and into [`Tuple`]. 10 | #[derive(Clone, Debug, PartialEq)] 11 | pub struct Mapper(Tuple); 12 | 13 | impl From for Mapper { 14 | fn from(t: Tuple) -> Mapper { 15 | Mapper(t) 16 | } 17 | } 18 | 19 | impl From for Tuple { 20 | fn from(m: Mapper) -> Tuple { 21 | m.0 22 | } 23 | } 24 | 25 | /// A mapped key/value pair. 26 | /// 27 | /// Mapped range read operations on FDB return [`MappedKeyValue`]. 28 | #[derive(Clone, Debug)] 29 | pub struct MappedKeyValue { 30 | key_value: KeyValue, 31 | range: Range, 32 | range_result: Vec, 33 | } 34 | 35 | impl MappedKeyValue { 36 | /// Gets a reference to [`KeyValue`] from [`MappedKeyValue`]. 37 | pub fn get_key_value_ref(&self) -> &KeyValue { 38 | &self.key_value 39 | } 40 | 41 | /// Gets a reference to [`Range`] from [`MappedKeyValue`]. 42 | pub fn get_range_ref(&self) -> &Range { 43 | &self.range 44 | } 45 | 46 | /// Gets a reference to [`Vec`] from [`MappedKeyValue`]. 47 | pub fn get_range_result_ref(&self) -> &Vec { 48 | &self.range_result 49 | } 50 | 51 | /// Extract [`KeyValue`] from [`MappedKeyValue`]. 52 | pub fn into_key_value(self) -> KeyValue { 53 | self.key_value 54 | } 55 | 56 | /// Extract [`Range`] from [`MappedKeyValue`]. 57 | pub fn into_range(self) -> Range { 58 | self.range 59 | } 60 | 61 | /// Extract [`Vec`] from [`MappedKeyValue`]. 62 | pub fn into_range_result(self) -> Vec { 63 | self.range_result 64 | } 65 | 66 | /// Extract [`KeyValue`], [`Range`] and [`Vec`] from 67 | /// [`MappedKeyValue`]. 68 | pub fn into_parts(self) -> (KeyValue, Range, Vec) { 69 | (self.key_value, self.range, self.range_result) 70 | } 71 | 72 | pub(crate) fn new( 73 | key_value: KeyValue, 74 | range: Range, 75 | range_result: Vec, 76 | ) -> MappedKeyValue { 77 | MappedKeyValue { 78 | key_value, 79 | range, 80 | range_result, 81 | } 82 | } 83 | } 84 | -------------------------------------------------------------------------------- /fdb/src/option.rs: -------------------------------------------------------------------------------- 1 | include!(concat!(env!("OUT_DIR"), "/option.rs")); 2 | -------------------------------------------------------------------------------- /fdb/src/tenant/mod.rs: -------------------------------------------------------------------------------- 1 | //! Provides [`FdbTenant`] type for working with FDB Tenants. 2 | //! 3 | //! Clients operating on a [`FdbTenant`] should, in most cases use the 4 | //! [`run`] method. This implements a proper retry loop around the 5 | //! work that needs to be done and, assure that [`commit`] has 6 | //! returned successfully before returning. 7 | //! 8 | //! [`run`]: FdbTenant::run 9 | //! [`commit`]: crate::transaction::Transaction::commit 10 | 11 | mod fdb_tenant; 12 | mod tenant_management; 13 | 14 | pub(crate) mod tenant_inner; 15 | 16 | pub use fdb_tenant::FdbTenant; 17 | pub use tenant_management::TenantManagement; 18 | -------------------------------------------------------------------------------- /fdb/src/tenant/tenant_inner.rs: -------------------------------------------------------------------------------- 1 | // We name this module `tenant_inner` in order to avoid 2 | // `module_inception` lint. We need to do this because we want to 3 | // expose `Tenant` type as `crate::Tenant`. 4 | 5 | use bytes::Bytes; 6 | 7 | /// [`Tenant`] is a named key-space within a database. 8 | /// 9 | /// [`Tenant`] can be converted from and into [`Bytes`]. 10 | /// 11 | /// **Note:** Tenant should not begin with `\xFF`. We do not enforce 12 | /// this check when creating a value of type [`Tenant`]. If you create 13 | /// a value of [`Tenant`] that starts with `\xFF`, you can expect 14 | /// tentant operations to fail with an [`FdbError`] code of 2134 15 | /// (`invalid_tenant_name`). 16 | /// 17 | /// [`FdbError`]: crate::error::FdbError 18 | #[derive(Clone, Debug, PartialEq)] 19 | pub struct Tenant(Bytes); 20 | 21 | impl From for Tenant { 22 | fn from(b: Bytes) -> Tenant { 23 | Tenant(b) 24 | } 25 | } 26 | 27 | impl From for Bytes { 28 | fn from(t: Tenant) -> Bytes { 29 | t.0 30 | } 31 | } 32 | -------------------------------------------------------------------------------- /fdb/src/tenant/tenant_management.rs: -------------------------------------------------------------------------------- 1 | use bytes::{BufMut, Bytes, BytesMut}; 2 | 3 | use std::sync::atomic::{AtomicBool, Ordering}; 4 | 5 | use crate::database::FdbDatabase; 6 | use crate::error::{FdbError, FdbResult}; 7 | use crate::transaction::{FdbTransaction, ReadTransaction, Transaction, TransactionOption}; 8 | use crate::Tenant; 9 | 10 | const TENANT_MAP_PREFIX: &[u8; 25] = b"\xFF\xFF/management/tenant_map/"; 11 | 12 | /// The FDB API includes function to manage the set of tenants in a 13 | /// cluster. 14 | #[derive(Debug)] 15 | pub struct TenantManagement; 16 | 17 | impl TenantManagement { 18 | /// Creates a new tenant in the cluster using a transaction 19 | /// created on the specified [`FdbDatabase`]. 20 | pub async fn create_tenant_db( 21 | db: &FdbDatabase, 22 | tenant_name: impl Into, 23 | ) -> FdbResult<()> { 24 | let checked_existence = AtomicBool::new(false); 25 | let key = { 26 | let mut b = BytesMut::new(); 27 | b.put(TENANT_MAP_PREFIX.as_ref()); 28 | b.put(Into::::into(tenant_name.into())); 29 | Into::::into(b) 30 | }; 31 | 32 | let checked_existence_ref = &checked_existence; 33 | let key_ref = &key; 34 | 35 | db.run(|tr| async move { 36 | tr.set_option(TransactionOption::SpecialKeySpaceEnableWrites)?; 37 | 38 | if checked_existence_ref.load(Ordering::SeqCst) { 39 | tr.set(key_ref.clone(), Bytes::new()); 40 | Ok(()) 41 | } else { 42 | let maybe_key = tr.get(key_ref.clone()).await?; 43 | 44 | checked_existence_ref.store(true, Ordering::SeqCst); 45 | 46 | match maybe_key { 47 | None => { 48 | tr.set(key_ref.clone(), Bytes::new()); 49 | Ok(()) 50 | } 51 | Some(_) => { 52 | // `tenant_already_exists` error 53 | Err(FdbError::new(2132)) 54 | } 55 | } 56 | } 57 | }) 58 | .await 59 | } 60 | 61 | /// Creates a new tenant in the cluster. 62 | pub fn create_tenant_tr(tr: &FdbTransaction, tenant_name: impl Into) -> FdbResult<()> { 63 | tr.set_option(TransactionOption::SpecialKeySpaceEnableWrites)?; 64 | 65 | tr.set( 66 | { 67 | let mut b = BytesMut::new(); 68 | b.put(TENANT_MAP_PREFIX.as_ref()); 69 | b.put(Into::::into(tenant_name.into())); 70 | Into::::into(b) 71 | }, 72 | Bytes::new(), 73 | ); 74 | 75 | Ok(()) 76 | } 77 | 78 | /// Deletes a tenant from the cluster using a transaction created 79 | /// on the specified [`FdbDatabase`]. 80 | pub async fn delete_tenant_db( 81 | db: &FdbDatabase, 82 | tenant_name: impl Into, 83 | ) -> FdbResult<()> { 84 | let checked_existence = AtomicBool::new(false); 85 | let key = { 86 | let mut b = BytesMut::new(); 87 | b.put(TENANT_MAP_PREFIX.as_ref()); 88 | b.put(Into::::into(tenant_name.into())); 89 | Into::::into(b) 90 | }; 91 | 92 | let checked_existence_ref = &checked_existence; 93 | let key_ref = &key; 94 | 95 | db.run(|tr| async move { 96 | tr.set_option(TransactionOption::SpecialKeySpaceEnableWrites)?; 97 | 98 | if checked_existence_ref.load(Ordering::SeqCst) { 99 | tr.clear(key_ref.clone()); 100 | Ok(()) 101 | } else { 102 | let maybe_key = tr.get(key_ref.clone()).await?; 103 | 104 | checked_existence_ref.store(true, Ordering::SeqCst); 105 | 106 | match maybe_key { 107 | None => { 108 | // `tenant_not_found` error 109 | Err(FdbError::new(2131)) 110 | } 111 | Some(_) => { 112 | tr.clear(key_ref.clone()); 113 | Ok(()) 114 | } 115 | } 116 | } 117 | }) 118 | .await 119 | } 120 | 121 | /// Deletes a tenant from the cluster. 122 | pub fn delete_tenant_tr(tr: &FdbTransaction, tenant_name: impl Into) -> FdbResult<()> { 123 | tr.set_option(TransactionOption::SpecialKeySpaceEnableWrites)?; 124 | 125 | tr.clear({ 126 | let mut b = BytesMut::new(); 127 | b.put(TENANT_MAP_PREFIX.as_ref()); 128 | b.put(Into::::into(tenant_name.into())); 129 | Into::::into(b) 130 | }); 131 | 132 | Ok(()) 133 | } 134 | } 135 | -------------------------------------------------------------------------------- /fdb/src/transaction/mod.rs: -------------------------------------------------------------------------------- 1 | //! Provides types and traits for working with FDB Transactions and 2 | //! Snapshots. 3 | 4 | mod fdb_transaction; 5 | mod read_transaction; 6 | 7 | // We do this in order to preserve consistency with Java and Go 8 | // bindings. 9 | #[allow(clippy::module_inception)] 10 | mod transaction; 11 | 12 | pub use crate::option::MutationType; 13 | pub use crate::option::TransactionOption; 14 | 15 | pub use fdb_transaction::{ 16 | CommittedVersion, FdbReadTransaction, FdbTransaction, TransactionVersionstamp, 17 | }; 18 | 19 | pub use read_transaction::ReadTransaction; 20 | pub use transaction::Transaction; 21 | -------------------------------------------------------------------------------- /fdb/src/transaction/transaction.rs: -------------------------------------------------------------------------------- 1 | use bytes::Bytes; 2 | 3 | use crate::error::FdbResult; 4 | use crate::future::{FdbFutureI64, FdbFutureUnit}; 5 | use crate::range::Range; 6 | use crate::transaction::{ 7 | CommittedVersion, MutationType, ReadTransaction, TransactionVersionstamp, 8 | }; 9 | use crate::{Key, Value}; 10 | 11 | /// A [`Transaction`] represents a FDB database transaction. 12 | /// 13 | /// All operations on FDB take place, explicity or implicity, through 14 | /// a [`Transaction`]. 15 | /// 16 | /// In FDB, a transaction is a mutable snapshot of a database. All 17 | /// read and write operations on a transaction see and modify an 18 | /// otherwise-unchanging version of the database and only change the 19 | /// underlying database if and when the transaction is committed. Read 20 | /// operations do see the effects of previous write operations on the 21 | /// same transactions. Committing a transaction usually succeeds in 22 | /// the absence of [conflicts]. 23 | /// 24 | /// Transactions group operations into a unit with the properties of 25 | /// atomicity, isolation, and durability. Transactions also provide 26 | /// the ability to maintain an application's invariants or integrity 27 | /// constraints, supporting the property of consistency. Together 28 | /// these properties are known as [ACID]. 29 | /// 30 | /// Transactions are also causally consistent: once a transaction has 31 | /// been successfully committed, all subsequently created transactions 32 | /// will see the modifications made by it. The most convenient way for 33 | /// a developer to manage the lifecycle and retrying of a transaction 34 | /// is to use [`run`] method on [`FdbDatabase`]. Otherwise, the client 35 | /// must have retry logic for fatal failures, failures to commit, and 36 | /// other transient errors. 37 | /// 38 | /// Keys and values in FDB are byte arrays. To encode other data 39 | /// types, see the [tuple layer] documentation. 40 | /// 41 | /// **Note**: All keys with first byte `0xff` are reserved for 42 | /// internal use. 43 | /// 44 | /// [conflicts]: https://apple.github.io/foundationdb/developer-guide.html#developer-guide-transaction-conflicts 45 | /// [ACID]: https://apple.github.io/foundationdb/developer-guide.html#acid 46 | /// [`FdbDatabase`]: crate::database::FdbDatabase 47 | /// [tuple layer]: crate::tuple 48 | /// [`run`]: crate::database::FdbDatabase::run 49 | // 50 | // NOTE: Unlike Java API, `Transaction` does not extend (i.e., is a 51 | // subtrait of) `TransactionContext` (There is no 52 | // `TransactionContext` in our case). Also `onError` method is 53 | // on `ReadTransaction` as we need it to implement the retry 54 | // loop in `FdbDatabase::read`. There is no `getDatabase` 55 | // method, as we don't implement `Database` interface/trait. 56 | pub trait Transaction: ReadTransaction { 57 | /// Adds a key to the transaction's read conflict ranges as if you 58 | /// had read the key. 59 | fn add_read_conflict_key(&self, key: impl Into) -> FdbResult<()>; 60 | 61 | /// Adds a range of keys to the transaction's read conflict ranges 62 | /// as if you had read the range. 63 | fn add_read_conflict_range(&self, range: Range) -> FdbResult<()>; 64 | 65 | /// Adds a key to the transaction's write conflict ranges as if 66 | /// you had written the key. 67 | fn add_write_conflict_key(&self, key: impl Into) -> FdbResult<()>; 68 | 69 | /// Adds a range of keys to the transaction's write conflict 70 | /// ranges as if you had cleared the range. 71 | fn add_write_conflict_range(&self, range: Range) -> FdbResult<()>; 72 | 73 | /// Cancels the [`Transaction`]. 74 | /// 75 | /// # Safety 76 | /// 77 | /// See [C API] for more details. 78 | /// 79 | /// [C API]: https://apple.github.io/foundationdb/api-c.html#c.fdb_transaction_cancel 80 | unsafe fn cancel(&self); 81 | 82 | /// Clears a given key from the database. 83 | fn clear(&self, key: impl Into); 84 | 85 | /// Clears a range of keys from the database. 86 | fn clear_range(&self, range: Range); 87 | 88 | /// Commit this [`Transaction`]. 89 | /// 90 | /// Equivalent to: 91 | /// 92 | /// ```ignore 93 | /// async unsafe fn commit(&self) -> FdbResult<()> 94 | /// ``` 95 | /// # Safety 96 | /// 97 | /// See [C API] for more details. 98 | /// 99 | /// [C API]: https://apple.github.io/foundationdb/api-c.html#c.fdb_transaction_commit 100 | unsafe fn commit(&self) -> FdbFutureUnit; 101 | 102 | /// Returns a future that will contain the approximated size of 103 | /// the commit, which is the summation of mutations, read conflict 104 | /// ranges, and write conflict ranges. 105 | /// 106 | /// ```ignore 107 | /// async fn get_approximate_size(&self) -> FdbResult 108 | /// ``` 109 | fn get_approximate_size(&self) -> FdbFutureI64; 110 | 111 | /// Gets the version number at which a successful commit modified 112 | /// the database. 113 | /// 114 | /// # Safety 115 | /// 116 | /// See [C API] for more details. 117 | /// 118 | /// [C API]: https://apple.github.io/foundationdb/api-c.html#c.fdb_transaction_get_committed_version 119 | unsafe fn get_committed_version(&self) -> CommittedVersion; 120 | 121 | /// Returns [`TransactionVersionstamp`] from which you can get the 122 | /// versionstamp which was used by any versionstamp operations in 123 | /// this transaction. 124 | /// 125 | /// # Safety 126 | /// 127 | /// See [C API] for more details. 128 | /// 129 | /// [C API]: https://apple.github.io/foundationdb/api-c.html#c.fdb_transaction_get_versionstamp 130 | unsafe fn get_versionstamp(&self) -> TransactionVersionstamp; 131 | 132 | /// An atomic operation is a single database command that carries 133 | /// out several logical steps: reading the value of a key, 134 | /// performing a transformation on that value, and writing the 135 | /// result. 136 | /// 137 | /// # Safety 138 | /// 139 | /// See the warning for [`MutationType::AppendIfFits`] variant. 140 | unsafe fn mutate(&self, optype: MutationType, key: impl Into, param: Bytes); 141 | 142 | /// Reset the [`Transaction`]. 143 | /// 144 | /// # Safety 145 | /// 146 | /// See [C API] for more details. 147 | /// 148 | /// [C API]: https://apple.github.io/foundationdb/api-c.html#c.fdb_transaction_cancel 149 | unsafe fn reset(&self); 150 | 151 | /// Sets the value for a given key. 152 | fn set(&self, key: impl Into, value: impl Into); 153 | 154 | /// Creates a watch that will become ready when it reports a 155 | /// change to the value of the specified key. 156 | /// 157 | /// A watch's behavior is relative to the transaction that created 158 | /// it. 159 | fn watch(&self, key: impl Into) -> FdbFutureUnit; 160 | } 161 | -------------------------------------------------------------------------------- /fdb/src/tuple/mod.rs: -------------------------------------------------------------------------------- 1 | //! Provides a set of utilities for serializing and deserializing 2 | //! typed data for use in FDB. 3 | //! 4 | //! When data is packed together into a [`Tuple`] it is suitable for 5 | //! use as an index or organizational structure within FDB. See 6 | //! [general Tuple documentation] for more information about how 7 | //! [`Tuple`] sort and can be uset to efficiently model data. 8 | //! 9 | //! [general Tuple documentation]: https://apple.github.io/foundationdb/data-modeling.html#data-modeling-tuples 10 | mod element; 11 | mod versionstamp; 12 | 13 | // We do this in order to preserve consistency with Java and Go 14 | // bindings. 15 | #[allow(clippy::module_inception)] 16 | mod tuple; 17 | 18 | pub mod key_util; 19 | 20 | pub use tuple::Tuple; 21 | pub use versionstamp::Versionstamp; 22 | -------------------------------------------------------------------------------- /fdb/tests/panic_on_select_api_version_twice.rs: -------------------------------------------------------------------------------- 1 | #[test] 2 | #[should_panic] 3 | fn panic_on_select_api_version_twice() { 4 | unsafe { 5 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 6 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 7 | } 8 | } 9 | -------------------------------------------------------------------------------- /fdb/tests/panic_on_select_wrong_api_version.rs: -------------------------------------------------------------------------------- 1 | #[test] 2 | #[should_panic] 3 | fn panic_on_select_wrong_api_version() { 4 | unsafe { 5 | fdb::select_api_version(i32::MAX); 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /fdb/tests/panic_on_start_network_after_network_start_stop.rs: -------------------------------------------------------------------------------- 1 | #[test] 2 | #[should_panic] 3 | fn panic_on_start_network_after_network_start_stop() { 4 | unsafe { 5 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 6 | 7 | fdb::start_network(); 8 | fdb::stop_network(); 9 | 10 | fdb::start_network(); 11 | } 12 | } 13 | -------------------------------------------------------------------------------- /fdb/tests/panic_on_stop_network_without_start_network.rs: -------------------------------------------------------------------------------- 1 | #[test] 2 | #[should_panic] 3 | fn panic_on_stop_network_without_start_network() { 4 | unsafe { 5 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 6 | fdb::stop_network(); 7 | } 8 | } 9 | -------------------------------------------------------------------------------- /fdb/tests/select_api_version_once.rs: -------------------------------------------------------------------------------- 1 | #[test] 2 | fn select_api_version_once() { 3 | unsafe { 4 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 5 | } 6 | } 7 | -------------------------------------------------------------------------------- /fdb/tests/set_network_option.rs: -------------------------------------------------------------------------------- 1 | use fdb::NetworkOption; 2 | 3 | #[test] 4 | fn set_network_option() { 5 | unsafe { 6 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 7 | fdb::set_network_option(NetworkOption::ExternalClientDirectory(String::from( 8 | "/home/montavista/fdb/fdb-client-lib", 9 | ))) 10 | .unwrap(); 11 | } 12 | } 13 | -------------------------------------------------------------------------------- /fdb/tests/start_stop_network.rs: -------------------------------------------------------------------------------- 1 | #[test] 2 | fn start_stop_network() { 3 | unsafe { 4 | fdb::select_api_version(fdb::FDB_API_VERSION as i32); 5 | fdb::start_network(); 6 | fdb::stop_network(); 7 | } 8 | } 9 | -------------------------------------------------------------------------------- /nix/ci/6.3/pull_request.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | set -u 5 | set -o pipefail 6 | 7 | cp /opt/fdb/conf/fdb.cluster /home/runner/fdb.cluster 8 | 9 | echo "+--------------------------+" 10 | echo "| Build and test fdb crate |" 11 | echo "+--------------------------+" 12 | 13 | cd ../../fdb || { echo "cd failure"; exit 1; } 14 | 15 | cargo build --lib --features=fdb-6_3 16 | 17 | cargo build --example get_committed_version --features=fdb-6_3 18 | cargo build --example get_range --features=fdb-6_3 19 | cargo build --example get_versionstamp --features=fdb-6_3 20 | cargo build --example hello_world --features=fdb-6_3 21 | cargo build --example open_database --features=fdb-6_3 22 | cargo build --example watch --features=fdb-6_3 23 | 24 | RUSTDOCFLAGS="--deny warnings" cargo doc --lib --features=fdb-6_3 25 | 26 | cargo test --lib --tests --features=fdb-6_3 27 | 28 | echo "" 29 | echo "+-------------------------------------------+" 30 | echo "| Check workspace formatting and run clippy |" 31 | echo "+-------------------------------------------+" 32 | 33 | cd ../ || { echo "cd failure"; exit 1; } 34 | 35 | cargo fmt --all --check 36 | 37 | cd fdb-gen || { echo "cd failure"; exit 1; } 38 | cargo clippy --lib --bins --examples --tests --features=fdb-6_3 -- --deny warnings 39 | cd ../ || { echo "cd failure"; exit 1; } 40 | 41 | cd fdb-sys || { echo "cd failure"; exit 1; } 42 | cargo clippy --lib --bins --examples --tests --features=fdb-6_3 -- --deny warnings 43 | cd ../ || { echo "cd failure"; exit 1; } 44 | 45 | cd fdb || { echo "cd failure"; exit 1; } 46 | cargo clippy --lib --bins --tests --features=fdb-6_3 -- --deny warnings 47 | 48 | cargo clippy --example get_committed_version --features=fdb-6_3 -- --deny warnings 49 | cargo clippy --example get_range --features=fdb-6_3 -- --deny warnings 50 | cargo clippy --example get_versionstamp --features=fdb-6_3 -- --deny warnings 51 | cargo clippy --example hello_world --features=fdb-6_3 -- --deny warnings 52 | cargo clippy --example open_database --features=fdb-6_3 -- --deny warnings 53 | cargo clippy --example watch --features=fdb-6_3 -- --deny warnings 54 | cd ../ || { echo "cd failure"; exit 1; } 55 | 56 | cd fdb-stacktester/fdb-stacktester-630 || { echo "cd failure"; exit 1; } 57 | cargo clippy --bins -- --deny warnings 58 | cd ../../ || { echo "cd failure"; exit 1; } 59 | 60 | echo "" 61 | echo "+------------------------------+" 62 | echo "| Setup and run binding tester |" 63 | echo "+------------------------------+" 64 | 65 | cd fdb-stacktester/fdb-stacktester-630 || { echo "cd failure"; exit 1; } 66 | 67 | cargo build --bin fdb-stacktester-630 --release 68 | 69 | pip install foundationdb==6.3.24 70 | 71 | # Run `scripted` test once. This is similar to how it is done in 72 | # `run_tester_loop.sh`. 73 | ./bindingtester/bindingtester/bindingtester.py rust --test-name scripted --logging-level WARNING 74 | 75 | # Some test with binding tester with 10 iterations. Exhaustive testing 76 | # happens using cron. 77 | 78 | START=1 79 | END=10 80 | 81 | for i in $(eval echo "{$START..$END}") 82 | do 83 | echo "Running interation $i" 84 | 85 | ./bindingtester/bindingtester/bindingtester.py rust --test-name api --compare --num-ops 1000 --logging-level WARNING 86 | 87 | ./bindingtester/bindingtester/bindingtester.py rust --test-name api --num-ops 1000 --concurrency 5 --logging-level WARNING 88 | done 89 | -------------------------------------------------------------------------------- /nix/ci/6.3/push.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | set -u 5 | set -o pipefail 6 | 7 | cp /opt/fdb/conf/fdb.cluster /home/runner/fdb.cluster 8 | 9 | echo "+--------------------------+" 10 | echo "| Build and test fdb crate |" 11 | echo "+--------------------------+" 12 | 13 | cd ../../fdb || { echo "cd failure"; exit 1; } 14 | 15 | cargo build --lib --features=fdb-6_3 16 | 17 | cargo build --example get_committed_version --features=fdb-6_3 18 | cargo build --example get_range --features=fdb-6_3 19 | cargo build --example get_versionstamp --features=fdb-6_3 20 | cargo build --example hello_world --features=fdb-6_3 21 | cargo build --example open_database --features=fdb-6_3 22 | cargo build --example watch --features=fdb-6_3 23 | 24 | RUSTDOCFLAGS="--deny warnings" cargo doc --lib --features=fdb-6_3 25 | 26 | cargo test --lib --tests --features=fdb-6_3 27 | 28 | echo "" 29 | echo "+-------------------------------------------+" 30 | echo "| Check workspace formatting and run clippy |" 31 | echo "+-------------------------------------------+" 32 | 33 | cd ../ || { echo "cd failure"; exit 1; } 34 | 35 | cargo fmt --all --check 36 | 37 | cd fdb-gen || { echo "cd failure"; exit 1; } 38 | cargo clippy --lib --bins --examples --tests --features=fdb-6_3 -- --deny warnings 39 | cd ../ || { echo "cd failure"; exit 1; } 40 | 41 | cd fdb-sys || { echo "cd failure"; exit 1; } 42 | cargo clippy --lib --bins --examples --tests --features=fdb-6_3 -- --deny warnings 43 | cd ../ || { echo "cd failure"; exit 1; } 44 | 45 | cd fdb || { echo "cd failure"; exit 1; } 46 | cargo clippy --lib --bins --tests --features=fdb-6_3 -- --deny warnings 47 | 48 | cargo clippy --example get_committed_version --features=fdb-6_3 -- --deny warnings 49 | cargo clippy --example get_range --features=fdb-6_3 -- --deny warnings 50 | cargo clippy --example get_versionstamp --features=fdb-6_3 -- --deny warnings 51 | cargo clippy --example hello_world --features=fdb-6_3 -- --deny warnings 52 | cargo clippy --example open_database --features=fdb-6_3 -- --deny warnings 53 | cargo clippy --example watch --features=fdb-6_3 -- --deny warnings 54 | cd ../ || { echo "cd failure"; exit 1; } 55 | 56 | cd fdb-stacktester/fdb-stacktester-630 || { echo "cd failure"; exit 1; } 57 | cargo clippy --bins -- --deny warnings 58 | cd ../../ || { echo "cd failure"; exit 1; } 59 | 60 | echo "" 61 | echo "+------------------------------+" 62 | echo "| Setup and run binding tester |" 63 | echo "+------------------------------+" 64 | 65 | cd fdb-stacktester/fdb-stacktester-630 || { echo "cd failure"; exit 1; } 66 | 67 | cargo build --bin fdb-stacktester-630 --release 68 | 69 | pip install foundationdb==6.3.24 70 | 71 | # Run `scripted` test once. This is similar to how it is done in 72 | # `run_tester_loop.sh`. 73 | ./bindingtester/bindingtester/bindingtester.py rust --test-name scripted --logging-level WARNING 74 | 75 | # Some test with binding tester with 10 iterations. Exhaustive testing 76 | # happens using cron. 77 | 78 | START=1 79 | END=10 80 | 81 | for i in $(eval echo "{$START..$END}") 82 | do 83 | echo "Running interation $i" 84 | 85 | ./bindingtester/bindingtester/bindingtester.py rust --test-name api --compare --num-ops 1000 --logging-level WARNING 86 | 87 | ./bindingtester/bindingtester/bindingtester.py rust --test-name api --num-ops 1000 --concurrency 5 --logging-level WARNING 88 | done 89 | -------------------------------------------------------------------------------- /nix/ci/6.3/schedule.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | set -u 5 | set -o pipefail 6 | 7 | cp /opt/fdb/conf/fdb.cluster /home/runner/fdb.cluster 8 | 9 | echo "" 10 | echo "+------------------------------+" 11 | echo "| Setup and run binding tester |" 12 | echo "+------------------------------+" 13 | 14 | cd ../../fdb-stacktester/fdb-stacktester-630 || { echo "cd failure"; exit 1; } 15 | 16 | cargo build --bin fdb-stacktester-630 --release 17 | 18 | pip install foundationdb==6.3.24 19 | 20 | # Run `scripted` test once. This is similar to how it is done in 21 | # `run_tester_loop.sh`. 22 | ./bindingtester/bindingtester/bindingtester.py rust --test-name scripted --logging-level WARNING 23 | 24 | # Adjust the number of iterations so it takes approximately an hour to 25 | # finish the run. 26 | 27 | START=1 28 | END=400 29 | 30 | for i in $(eval echo "{$START..$END}") 31 | do 32 | echo "Running interation $i" 33 | 34 | ./bindingtester/bindingtester/bindingtester.py rust --test-name api --compare --num-ops 1000 --logging-level WARNING 35 | 36 | ./bindingtester/bindingtester/bindingtester.py rust --test-name api --num-ops 1000 --concurrency 5 --logging-level WARNING 37 | done 38 | -------------------------------------------------------------------------------- /nix/ci/7.1/pull_request.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | set -u 5 | set -o pipefail 6 | 7 | cp /opt/fdb/conf/fdb.cluster /home/runner/fdb.cluster 8 | 9 | echo "+--------------------------+" 10 | echo "| Build and test fdb crate |" 11 | echo "+--------------------------+" 12 | 13 | cd ../../fdb || { echo "cd failure"; exit 1; } 14 | 15 | cargo build --lib --features=fdb-7_1 16 | 17 | cargo build --example get_committed_version --features=fdb-7_1 18 | cargo build --example get_mapped_range --features=fdb-7_1 19 | cargo build --example get_range --features=fdb-7_1 20 | cargo build --example get_versionstamp --features=fdb-7_1 21 | cargo build --example hello_world --features=fdb-7_1 22 | cargo build --example open_database --features=fdb-7_1 23 | cargo build --example watch --features=fdb-7_1 24 | 25 | RUSTDOCFLAGS="--deny warnings" cargo doc --lib --features=fdb-7_1 26 | 27 | cargo test --lib --tests --features=fdb-7_1 28 | 29 | echo "" 30 | echo "+-------------------------------------------+" 31 | echo "| Check workspace formatting and run clippy |" 32 | echo "+-------------------------------------------+" 33 | 34 | cd ../ || { echo "cd failure"; exit 1; } 35 | 36 | cargo fmt --all --check 37 | 38 | cd fdb-gen || { echo "cd failure"; exit 1; } 39 | cargo clippy --lib --bins --examples --tests --features=fdb-7_1 -- --deny warnings 40 | cd ../ || { echo "cd failure"; exit 1; } 41 | 42 | cd fdb-sys || { echo "cd failure"; exit 1; } 43 | cargo clippy --lib --bins --examples --tests --features=fdb-7_1 -- --deny warnings 44 | cd ../ || { echo "cd failure"; exit 1; } 45 | 46 | cd fdb || { echo "cd failure"; exit 1; } 47 | cargo clippy --lib --bins --tests --features=fdb-7_1 -- --deny warnings 48 | 49 | cargo clippy --example get_committed_version --features=fdb-7_1 -- --deny warnings 50 | cargo clippy --example get_mapped_range --features=fdb-7_1 -- --deny warnings 51 | cargo clippy --example get_range --features=fdb-7_1 -- --deny warnings 52 | cargo clippy --example get_versionstamp --features=fdb-7_1 -- --deny warnings 53 | cargo clippy --example hello_world --features=fdb-7_1 -- --deny warnings 54 | cargo clippy --example open_database --features=fdb-7_1 -- --deny warnings 55 | cargo clippy --example watch --features=fdb-7_1 -- --deny warnings 56 | cd ../ || { echo "cd failure"; exit 1; } 57 | 58 | cd fdb-stacktester/fdb-stacktester-710 || { echo "cd failure"; exit 1; } 59 | cargo clippy --bins -- --deny warnings 60 | cd ../../ || { echo "cd failure"; exit 1; } 61 | 62 | echo "" 63 | echo "+------------------------------+" 64 | echo "| Setup and run binding tester |" 65 | echo "+------------------------------+" 66 | 67 | cd fdb-stacktester/fdb-stacktester-710 || { echo "cd failure"; exit 1; } 68 | 69 | cargo build --bin fdb-stacktester-710 --release 70 | 71 | pip install foundationdb==7.1.12 72 | 73 | # Run `scripted` test once. This is similar to how it is done in 74 | # `run_tester_loop.sh`. 75 | ./bindingtester/bindingtester/bindingtester.py rust --test-name scripted --logging-level WARNING 76 | 77 | # Some test with binding tester with 10 iterations. Exhaustive testing 78 | # happens using cron. 79 | 80 | START=1 81 | END=10 82 | 83 | for i in $(eval echo "{$START..$END}") 84 | do 85 | echo "Running interation $i" 86 | 87 | ./bindingtester/bindingtester/bindingtester.py rust --test-name api --compare --num-ops 1000 --logging-level WARNING 88 | 89 | ./bindingtester/bindingtester/bindingtester.py rust --test-name api --num-ops 1000 --concurrency 5 --logging-level WARNING 90 | done 91 | -------------------------------------------------------------------------------- /nix/ci/7.1/pull_request_lcov.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | set -u 5 | set -o pipefail 6 | 7 | cp /opt/fdb/conf/fdb.cluster /home/runner/fdb.cluster 8 | 9 | echo "+-------------------------------+" 10 | echo "| Generate Code Coverage Report |" 11 | echo "+-------------------------------+" 12 | 13 | cd ../../fdb || { echo "cd failure"; exit 1; } 14 | 15 | mkdir -p lcov 16 | 17 | # Run unit tests and integration tests 18 | 19 | cargo llvm-cov --lib --tests --features=fdb-7_1 --lcov --output-path lcov/tests.info 20 | 21 | # Run examples 22 | 23 | /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 24 | 25 | cargo llvm-cov run --example get_committed_version --features=fdb-7_1 --lcov --output-path lcov/get_committed_version.info 26 | 27 | # # There seems to be a bug [1] that is causing this `run` to fail. Add 28 | # # it back once the bug is resolved. 29 | # # 30 | # # [1] https://forums.foundationdb.org/t/everything-about-getmappedrange/3280/3 31 | 32 | # /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 33 | 34 | # cargo llvm-cov run --example get_mapped_range --features=fdb-7_1 --lcov --output-path lcov/get_mapped_range.info 35 | 36 | /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 37 | 38 | cargo llvm-cov run --example get_range --features=fdb-7_1 --lcov --output-path lcov/get_range.info 39 | 40 | /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 41 | 42 | cargo llvm-cov run --example get_versionstamp --features=fdb-7_1 --lcov --output-path lcov/get_versionstamp.info 43 | 44 | /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 45 | 46 | cargo llvm-cov run --example hello_world --features=fdb-7_1 --lcov --output-path lcov/hello_world.info 47 | 48 | /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 49 | 50 | cargo llvm-cov run --example open_database --features=fdb-7_1 --lcov --output-path lcov/open_database.info 51 | 52 | /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 53 | 54 | cargo llvm-cov run --example watch --features=fdb-7_1 --lcov --output-path lcov/watch.info 55 | 56 | # Need to add `--add-tracefile lcov/get_mapped_range.info` once the 57 | # bug mentioned above is resolved. 58 | 59 | lcov --add-tracefile lcov/tests.info --add-tracefile lcov/get_committed_version.info --add-tracefile lcov/get_range.info --add-tracefile lcov/get_versionstamp.info --add-tracefile lcov/hello_world.info --add-tracefile lcov/open_database.info --add-tracefile lcov/watch.info --output-file ../lcov.info 60 | -------------------------------------------------------------------------------- /nix/ci/7.1/push.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | set -u 5 | set -o pipefail 6 | 7 | cp /opt/fdb/conf/fdb.cluster /home/runner/fdb.cluster 8 | 9 | echo "+--------------------------+" 10 | echo "| Build and test fdb crate |" 11 | echo "+--------------------------+" 12 | 13 | cd ../../fdb || { echo "cd failure"; exit 1; } 14 | 15 | cargo build --lib --features=fdb-7_1 16 | 17 | cargo build --example get_committed_version --features=fdb-7_1 18 | cargo build --example get_mapped_range --features=fdb-7_1 19 | cargo build --example get_range --features=fdb-7_1 20 | cargo build --example get_versionstamp --features=fdb-7_1 21 | cargo build --example hello_world --features=fdb-7_1 22 | cargo build --example open_database --features=fdb-7_1 23 | cargo build --example watch --features=fdb-7_1 24 | 25 | RUSTDOCFLAGS="--deny warnings" cargo doc --lib --features=fdb-7_1 26 | 27 | cargo test --lib --tests --features=fdb-7_1 28 | 29 | echo "" 30 | echo "+-------------------------------------------+" 31 | echo "| Check workspace formatting and run clippy |" 32 | echo "+-------------------------------------------+" 33 | 34 | cd ../ || { echo "cd failure"; exit 1; } 35 | 36 | cargo fmt --all --check 37 | 38 | cd fdb-gen || { echo "cd failure"; exit 1; } 39 | cargo clippy --lib --bins --examples --tests --features=fdb-7_1 -- --deny warnings 40 | cd ../ || { echo "cd failure"; exit 1; } 41 | 42 | cd fdb-sys || { echo "cd failure"; exit 1; } 43 | cargo clippy --lib --bins --examples --tests --features=fdb-7_1 -- --deny warnings 44 | cd ../ || { echo "cd failure"; exit 1; } 45 | 46 | cd fdb || { echo "cd failure"; exit 1; } 47 | cargo clippy --lib --bins --tests --features=fdb-7_1 -- --deny warnings 48 | 49 | cargo clippy --example get_committed_version --features=fdb-7_1 -- --deny warnings 50 | cargo clippy --example get_mapped_range --features=fdb-7_1 -- --deny warnings 51 | cargo clippy --example get_range --features=fdb-7_1 -- --deny warnings 52 | cargo clippy --example get_versionstamp --features=fdb-7_1 -- --deny warnings 53 | cargo clippy --example hello_world --features=fdb-7_1 -- --deny warnings 54 | cargo clippy --example open_database --features=fdb-7_1 -- --deny warnings 55 | cargo clippy --example watch --features=fdb-7_1 -- --deny warnings 56 | cd ../ || { echo "cd failure"; exit 1; } 57 | 58 | cd fdb-stacktester/fdb-stacktester-710 || { echo "cd failure"; exit 1; } 59 | cargo clippy --bins -- --deny warnings 60 | cd ../../ || { echo "cd failure"; exit 1; } 61 | 62 | echo "" 63 | echo "+------------------------------+" 64 | echo "| Setup and run binding tester |" 65 | echo "+------------------------------+" 66 | 67 | cd fdb-stacktester/fdb-stacktester-710 || { echo "cd failure"; exit 1; } 68 | 69 | cargo build --bin fdb-stacktester-710 --release 70 | 71 | pip install foundationdb==7.1.12 72 | 73 | # Run `scripted` test once. This is similar to how it is done in 74 | # `run_tester_loop.sh`. 75 | ./bindingtester/bindingtester/bindingtester.py rust --test-name scripted --logging-level WARNING 76 | 77 | # Some test with binding tester with 10 iterations. Exhaustive testing 78 | # happens using cron. 79 | 80 | START=1 81 | END=10 82 | 83 | for i in $(eval echo "{$START..$END}") 84 | do 85 | echo "Running interation $i" 86 | 87 | ./bindingtester/bindingtester/bindingtester.py rust --test-name api --compare --num-ops 1000 --logging-level WARNING 88 | 89 | ./bindingtester/bindingtester/bindingtester.py rust --test-name api --num-ops 1000 --concurrency 5 --logging-level WARNING 90 | done 91 | -------------------------------------------------------------------------------- /nix/ci/7.1/push_lcov.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | set -u 5 | set -o pipefail 6 | 7 | cp /opt/fdb/conf/fdb.cluster /home/runner/fdb.cluster 8 | 9 | echo "+-------------------------------+" 10 | echo "| Generate Code Coverage Report |" 11 | echo "+-------------------------------+" 12 | 13 | cd ../../fdb || { echo "cd failure"; exit 1; } 14 | 15 | mkdir -p lcov 16 | 17 | # Run unit tests and integration tests 18 | 19 | cargo llvm-cov --lib --tests --features=fdb-7_1 --lcov --output-path lcov/tests.info 20 | 21 | # Run examples 22 | 23 | /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 24 | 25 | cargo llvm-cov run --example get_committed_version --features=fdb-7_1 --lcov --output-path lcov/get_committed_version.info 26 | 27 | # # There seems to be a bug [1] that is causing this `run` to fail. Add 28 | # # it back once the bug is resolved. 29 | # # 30 | # # [1] https://forums.foundationdb.org/t/everything-about-getmappedrange/3280/3 31 | 32 | # /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 33 | 34 | # cargo llvm-cov run --example get_mapped_range --features=fdb-7_1 --lcov --output-path lcov/get_mapped_range.info 35 | 36 | /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 37 | 38 | cargo llvm-cov run --example get_range --features=fdb-7_1 --lcov --output-path lcov/get_range.info 39 | 40 | /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 41 | 42 | cargo llvm-cov run --example get_versionstamp --features=fdb-7_1 --lcov --output-path lcov/get_versionstamp.info 43 | 44 | /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 45 | 46 | cargo llvm-cov run --example hello_world --features=fdb-7_1 --lcov --output-path lcov/hello_world.info 47 | 48 | /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 49 | 50 | cargo llvm-cov run --example open_database --features=fdb-7_1 --lcov --output-path lcov/open_database.info 51 | 52 | /opt/fdb/cli/7.1.12/fdbcli -C /home/runner/fdb.cluster --exec "writemode on; clearrange \x00 \xff" 53 | 54 | cargo llvm-cov run --example watch --features=fdb-7_1 --lcov --output-path lcov/watch.info 55 | 56 | # Need to add `--add-tracefile lcov/get_mapped_range.info` once the 57 | # bug mentioned above is resolved. 58 | 59 | lcov --add-tracefile lcov/tests.info --add-tracefile lcov/get_committed_version.info --add-tracefile lcov/get_range.info --add-tracefile lcov/get_versionstamp.info --add-tracefile lcov/hello_world.info --add-tracefile lcov/open_database.info --add-tracefile lcov/watch.info --output-file ../lcov.info 60 | -------------------------------------------------------------------------------- /nix/ci/7.1/push_rustdoc.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | set -u 5 | set -o pipefail 6 | 7 | echo "+---------------------+" 8 | echo "| Build documentation |" 9 | echo "+---------------------+" 10 | 11 | cd ../../ || { echo "cd failure"; exit 1; } 12 | 13 | cd fdb-gen || { echo "cd failure"; exit 1; } 14 | 15 | cargo doc --lib --no-deps --features=fdb-7_1 16 | 17 | cd ../ || { echo "cd failure"; exit 1; } 18 | 19 | cd fdb-sys || { echo "cd failure"; exit 1; } 20 | 21 | cargo doc --lib --no-deps --features=fdb-7_1 22 | 23 | cd ../ || { echo "cd failure"; exit 1; } 24 | 25 | cd fdb || { echo "cd failure"; exit 1; } 26 | 27 | cargo doc --lib --no-deps --features=fdb-7_1 28 | 29 | cd ../ || { echo "cd failure"; exit 1; } 30 | 31 | -------------------------------------------------------------------------------- /nix/ci/7.1/schedule.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | set -u 5 | set -o pipefail 6 | 7 | cp /opt/fdb/conf/fdb.cluster /home/runner/fdb.cluster 8 | 9 | echo "" 10 | echo "+------------------------------+" 11 | echo "| Setup and run binding tester |" 12 | echo "+------------------------------+" 13 | 14 | cd ../../fdb-stacktester/fdb-stacktester-710 || { echo "cd failure"; exit 1; } 15 | 16 | cargo build --bin fdb-stacktester-710 --release 17 | 18 | pip install foundationdb==7.1.12 19 | 20 | # Run `scripted` test once. This is similar to how it is done in 21 | # `run_tester_loop.sh`. 22 | ./bindingtester/bindingtester/bindingtester.py rust --test-name scripted --logging-level WARNING 23 | 24 | # Adjust the number of iterations so it takes approximately an hour to 25 | # finish the run. 26 | 27 | START=1 28 | END=100 29 | 30 | for i in $(eval echo "{$START..$END}") 31 | do 32 | echo "Running interation $i" 33 | 34 | ./bindingtester/bindingtester/bindingtester.py rust --test-name api --compare --num-ops 1000 --logging-level WARNING 35 | 36 | ./bindingtester/bindingtester/bindingtester.py rust --test-name api --num-ops 1000 --concurrency 5 --logging-level WARNING 37 | done 38 | -------------------------------------------------------------------------------- /nix/ci/DEVELOP.md: -------------------------------------------------------------------------------- 1 | # CI Development Notes 2 | 3 | The CI system is designed around NixOS containers and runs using 4 | `podman` on GitHub. It makes a number of assumptions that is 5 | documented here. If you make changes, please update the new 6 | assumptions here. 7 | 8 | 1. FoundationDB processes are run with a uid/gid of `4059` and 9 | username/groupname of `fdb`. 10 | 11 | 2. Within the container, we use uid/gid of `1001/121` to run workflow 12 | steps. This maps to username/groupname `runner/docker`. The uid/gid 13 | and username/groupname is the same on both the host ubuntu virtual 14 | machine and the container. 15 | 16 | 3. `GITHUB_WORKSPACE` is bind-mounted to `/home/runner/fdb`. 17 | -------------------------------------------------------------------------------- /nix/ci/cargo-llvm-cov/default.nix: -------------------------------------------------------------------------------- 1 | { stdenv 2 | , autoPatchelfHook 3 | , fetchurl 4 | }: 5 | 6 | stdenv.mkDerivation rec { 7 | pname = "cargo-llvm-cov"; 8 | 9 | version = "0.4.9"; 10 | 11 | src = fetchurl { 12 | url = "https://github.com/taiki-e/cargo-llvm-cov/releases/download/v${version}/cargo-llvm-cov-x86_64-unknown-linux-gnu.tar.gz"; 13 | sha256 = "sha256-Gg24R33D1IrG0jX35FyD4sz5vJlpEyrY7Zzo9FKEW+A="; 14 | }; 15 | 16 | unpackPhase = ":"; 17 | 18 | nativeBuildInputs = [ 19 | autoPatchelfHook 20 | ]; 21 | 22 | installPhase = '' 23 | tar zxvf $src 24 | 25 | mkdir -p $out/bin 26 | 27 | cp cargo-llvm-cov $out/bin/cargo-llvm-cov 28 | ''; 29 | } 30 | -------------------------------------------------------------------------------- /nix/ci/fdb-6.3/app/default.nix: -------------------------------------------------------------------------------- 1 | # This derivation can be used to extract out 2 | # - `fdbbackup` 3 | # - `fdbcli` 4 | # - `fdbmonitor` 5 | # - `fdbserver` 6 | # from GitHub and patch the app using `autoPatchelfHook`. 7 | # 8 | # In this derivation, we don't care where the app will eventually get 9 | # placed in the file system. 10 | { stdenv 11 | , autoPatchelfHook 12 | , fetchurl 13 | , name 14 | , sha256 15 | , version 16 | }: 17 | 18 | stdenv.mkDerivation { 19 | pname = "fdb-${name}"; 20 | 21 | inherit version; 22 | 23 | src = fetchurl { 24 | url = "https://github.com/apple/foundationdb/releases/download/${version}/fdb${name}.x86_64"; 25 | inherit sha256; 26 | }; 27 | 28 | nativeBuildInputs = [ 29 | autoPatchelfHook 30 | ]; 31 | 32 | unpackPhase = ":"; 33 | 34 | installPhase = '' 35 | mkdir $out 36 | cp $src $out/fdb${name} 37 | chmod 755 $out/fdb${name} 38 | ''; 39 | } 40 | -------------------------------------------------------------------------------- /nix/ci/fdb-6.3/client-lib/default.nix: -------------------------------------------------------------------------------- 1 | # This derivation just extracts out `libfdb_c_X.Y.Z.so` from GitHub 2 | # and patches the library using `autoPatchelfHook`. 3 | # 4 | # In this derivation we don't care where `libfdb_c_X.Y.Z.so` will 5 | # eventually get placed in the file system. 6 | # 7 | # If `isDir` is `false`, then we don't setup symlink from 8 | # `libfdb_c.so` to `libfdb_c_X.Y.Z.so`. We would not want the symlink 9 | # in case we are building `EXTERNAL_CLIENT_DIRECTORY`. 10 | { stdenv 11 | , autoPatchelfHook 12 | , fetchurl 13 | , isDir 14 | , lib 15 | , sha256 16 | , version 17 | }: 18 | 19 | stdenv.mkDerivation { 20 | pname = with lib; concatStrings [ "fdb-client-lib" (if isDir then "-dir" else "") ]; 21 | 22 | inherit version; 23 | 24 | src = fetchurl { 25 | url = "https://github.com/apple/foundationdb/releases/download/${version}/libfdb_c.x86_64.so"; 26 | inherit sha256; 27 | }; 28 | 29 | nativeBuildInputs = [ 30 | autoPatchelfHook 31 | ]; 32 | 33 | unpackPhase = ":"; 34 | 35 | installPhase = with lib; concatStrings [ 36 | '' 37 | mkdir $out 38 | cp $src $out/libfdb_c.so.${version} 39 | chmod 555 $out/libfdb_c.so.${version} 40 | '' 41 | ( 42 | if isDir then "" else '' 43 | ln -s $out/libfdb_c.so.${version} $out/libfdb_c.so 44 | '' 45 | ) 46 | ]; 47 | } 48 | -------------------------------------------------------------------------------- /nix/ci/fdb-6.3/default.nix: -------------------------------------------------------------------------------- 1 | { pkgs }: 2 | let 3 | opt-fdb-client-lib = { version, fdb-client-lib }: 4 | pkgs.runCommand "opt-lib-client-lib-${version}" { } '' 5 | mkdir -p $out/opt/fdb/client-lib 6 | ln -s ${fdb-client-lib}/libfdb_c.so $out/opt/fdb/client-lib/libfdb_c.so 7 | ln -s ${fdb-client-lib}/libfdb_c.so.${version} $out/opt/fdb/client-lib/libfdb_c.so.${version} 8 | ''; 9 | 10 | opt-fdb-client-lib-dir = { version, fdb-client-lib-dir }: 11 | pkgs.runCommand "opt-lib-client-lib-dir-${version}" { } '' 12 | mkdir -p $out/opt/fdb/client-lib-dir 13 | ln -s ${fdb-client-lib-dir}/libfdb_c.so.${version} $out/opt/fdb/client-lib-dir/libfdb_c.so.${version} 14 | ''; 15 | 16 | opt-fdb-monitor = { version, fdb-monitor }: 17 | pkgs.runCommand "opt-fdb-monitor-${version}" { } '' 18 | mkdir -p $out/opt/fdb/monitor 19 | ln -s ${fdb-monitor}/fdbmonitor $out/opt/fdb/monitor/fdbmonitor 20 | ''; 21 | 22 | opt-fdb-server = { version, fdb-server }: 23 | pkgs.runCommand "opt-fdb-server-${version}" { } '' 24 | mkdir -p $out/opt/fdb/server/${version} 25 | ln -s ${fdb-server}/fdbserver $out/opt/fdb/server/${version}/fdbserver 26 | ''; 27 | 28 | opt-fdb-cli = { version, fdb-cli }: 29 | pkgs.runCommand "opt-fdb-cli-${version}" { } '' 30 | mkdir -p $out/opt/fdb/cli/${version} 31 | ln -s ${fdb-cli}/fdbcli $out/opt/fdb/cli/${version}/fdbcli 32 | ''; 33 | in 34 | [ 35 | ( 36 | let 37 | version = "6.3.24"; 38 | sha256 = "sha256-avg6auM2Vqu00+xsdA+brTB7GX0o3BZvEChnEOErMJk="; 39 | isDir = false; 40 | 41 | fdb-client-lib = pkgs.callPackage ./client-lib { inherit version sha256 isDir; }; 42 | in 43 | opt-fdb-client-lib { inherit version fdb-client-lib; } 44 | ) 45 | 46 | ( 47 | let 48 | version = "6.3.24"; 49 | sha256 = "sha256-avg6auM2Vqu00+xsdA+brTB7GX0o3BZvEChnEOErMJk="; 50 | isDir = true; 51 | 52 | fdb-client-lib-dir = pkgs.callPackage ./client-lib { inherit version sha256 isDir; }; 53 | in 54 | opt-fdb-client-lib-dir { inherit version fdb-client-lib-dir; } 55 | ) 56 | 57 | ( 58 | let 59 | name = "monitor"; 60 | sha256 = "sha256-+hiG+YMt1w6mRBnwV3WmBKhgA7mo/t7qstf8pVPlP1k="; 61 | version = "6.3.24"; 62 | 63 | fdb-monitor = pkgs.callPackage ./app { inherit name sha256 version; }; 64 | in 65 | opt-fdb-monitor { inherit version fdb-monitor; } 66 | ) 67 | 68 | ( 69 | let 70 | name = "server"; 71 | sha256 = "sha256-ogMPAuDkhuyBNIDpEKDsCKpYZtK2Ik7NcQm93gxWFho="; 72 | version = "6.3.24"; 73 | 74 | fdb-server = pkgs.callPackage ./app { inherit name sha256 version; }; 75 | in 76 | opt-fdb-server { inherit version fdb-server; } 77 | ) 78 | 79 | ( 80 | let 81 | name = "cli"; 82 | sha256 = "sha256-zKDCdDfkIwnHCZb3kWHCsqtXVKBJ9serY61jpNAOHzg="; 83 | version = "6.3.24"; 84 | 85 | fdb-cli = pkgs.callPackage ./app { inherit name sha256 version; }; 86 | in 87 | opt-fdb-cli { inherit version fdb-cli; } 88 | ) 89 | ] 90 | -------------------------------------------------------------------------------- /nix/ci/fdb-7.1/app/default.nix: -------------------------------------------------------------------------------- 1 | # This derivation can be used to extract out 2 | # - `fdbbackup` 3 | # - `fdbcli` 4 | # - `fdbmonitor` 5 | # - `fdbserver` 6 | # from GitHub and patch the app using `autoPatchelfHook`. 7 | # 8 | # In this derivation, we don't care where the app will eventually get 9 | # placed in the file system. 10 | { stdenv 11 | , autoPatchelfHook 12 | , fetchurl 13 | , name 14 | , sha256 15 | , version 16 | }: 17 | 18 | stdenv.mkDerivation { 19 | pname = "fdb-${name}"; 20 | 21 | inherit version; 22 | 23 | src = fetchurl { 24 | url = "https://github.com/apple/foundationdb/releases/download/${version}/fdb${name}.x86_64"; 25 | inherit sha256; 26 | }; 27 | 28 | nativeBuildInputs = [ 29 | autoPatchelfHook 30 | ]; 31 | 32 | unpackPhase = ":"; 33 | 34 | installPhase = '' 35 | mkdir $out 36 | cp $src $out/fdb${name} 37 | chmod 755 $out/fdb${name} 38 | ''; 39 | } 40 | -------------------------------------------------------------------------------- /nix/ci/fdb-7.1/client-lib/default.nix: -------------------------------------------------------------------------------- 1 | # This derivation just extracts out `libfdb_c_X.Y.Z.so` from GitHub 2 | # and patches the library using `autoPatchelfHook`. 3 | # 4 | # In this derivation we don't care where `libfdb_c_X.Y.Z.so` will 5 | # eventually get placed in the file system. 6 | # 7 | # If `isDir` is `false`, then we don't setup symlink from 8 | # `libfdb_c.so` to `libfdb_c_X.Y.Z.so`. We would not want the symlink 9 | # in case we are building `EXTERNAL_CLIENT_DIRECTORY`. 10 | { stdenv 11 | , autoPatchelfHook 12 | , fetchurl 13 | , isDir 14 | , lib 15 | , sha256 16 | , version 17 | }: 18 | 19 | stdenv.mkDerivation { 20 | pname = with lib; concatStrings [ "fdb-client-lib" (if isDir then "-dir" else "") ]; 21 | 22 | inherit version; 23 | 24 | src = fetchurl { 25 | url = "https://github.com/apple/foundationdb/releases/download/${version}/libfdb_c.x86_64.so"; 26 | inherit sha256; 27 | }; 28 | 29 | nativeBuildInputs = [ 30 | autoPatchelfHook 31 | ]; 32 | 33 | unpackPhase = ":"; 34 | 35 | installPhase = with lib; concatStrings [ 36 | '' 37 | mkdir $out 38 | cp $src $out/libfdb_c.so.${version} 39 | chmod 555 $out/libfdb_c.so.${version} 40 | '' 41 | ( 42 | if isDir then "" else '' 43 | ln -s $out/libfdb_c.so.${version} $out/libfdb_c.so 44 | '' 45 | ) 46 | ]; 47 | } 48 | -------------------------------------------------------------------------------- /nix/ci/fdb-7.1/default.nix: -------------------------------------------------------------------------------- 1 | { pkgs }: 2 | let 3 | opt-fdb-client-lib = { version, fdb-client-lib }: 4 | pkgs.runCommand "opt-lib-client-lib-${version}" { } '' 5 | mkdir -p $out/opt/fdb/client-lib 6 | ln -s ${fdb-client-lib}/libfdb_c.so $out/opt/fdb/client-lib/libfdb_c.so 7 | ln -s ${fdb-client-lib}/libfdb_c.so.${version} $out/opt/fdb/client-lib/libfdb_c.so.${version} 8 | ''; 9 | 10 | opt-fdb-client-lib-dir = { version, fdb-client-lib-dir }: 11 | pkgs.runCommand "opt-lib-client-lib-dir-${version}" { } '' 12 | mkdir -p $out/opt/fdb/client-lib-dir 13 | ln -s ${fdb-client-lib-dir}/libfdb_c.so.${version} $out/opt/fdb/client-lib-dir/libfdb_c.so.${version} 14 | ''; 15 | 16 | opt-fdb-monitor = { version, fdb-monitor }: 17 | pkgs.runCommand "opt-fdb-monitor-${version}" { } '' 18 | mkdir -p $out/opt/fdb/monitor 19 | ln -s ${fdb-monitor}/fdbmonitor $out/opt/fdb/monitor/fdbmonitor 20 | ''; 21 | 22 | opt-fdb-server = { version, fdb-server }: 23 | pkgs.runCommand "opt-fdb-server-${version}" { } '' 24 | mkdir -p $out/opt/fdb/server/${version} 25 | ln -s ${fdb-server}/fdbserver $out/opt/fdb/server/${version}/fdbserver 26 | ''; 27 | 28 | opt-fdb-cli = { version, fdb-cli }: 29 | pkgs.runCommand "opt-fdb-cli-${version}" { } '' 30 | mkdir -p $out/opt/fdb/cli/${version} 31 | ln -s ${fdb-cli}/fdbcli $out/opt/fdb/cli/${version}/fdbcli 32 | ''; 33 | in 34 | [ 35 | ( 36 | let 37 | version = "7.1.12"; 38 | sha256 = "sha256-5KeYLcy22eYWuQVUMIlrAP90h0crAzvkrcl/ADZ5yCE="; 39 | isDir = false; 40 | 41 | fdb-client-lib = pkgs.callPackage ./client-lib { inherit version sha256 isDir; }; 42 | in 43 | opt-fdb-client-lib { inherit version fdb-client-lib; } 44 | ) 45 | 46 | ( 47 | let 48 | version = "7.1.12"; 49 | sha256 = "sha256-5KeYLcy22eYWuQVUMIlrAP90h0crAzvkrcl/ADZ5yCE="; 50 | isDir = true; 51 | 52 | fdb-client-lib-dir = pkgs.callPackage ./client-lib { inherit version sha256 isDir; }; 53 | in 54 | opt-fdb-client-lib-dir { inherit version fdb-client-lib-dir; } 55 | ) 56 | 57 | ( 58 | let 59 | name = "monitor"; 60 | version = "7.1.12"; 61 | sha256 = "sha256-meuNIjt6xhkuTM1AiF8fvtFM2SnM16MutNPsHH58gz8="; 62 | 63 | fdb-monitor = pkgs.callPackage ./app { inherit name sha256 version; }; 64 | in 65 | opt-fdb-monitor { inherit version fdb-monitor; } 66 | ) 67 | 68 | ( 69 | let 70 | name = "server"; 71 | version = "7.1.12"; 72 | sha256 = "sha256-FQzCcIAeFLfGszkJ61BJqYRlq2ev/fMxA93Lz6qkRJg="; 73 | 74 | fdb-server = pkgs.callPackage ./app { inherit name sha256 version; }; 75 | in 76 | opt-fdb-server { inherit version fdb-server; } 77 | ) 78 | 79 | ( 80 | let 81 | name = "cli"; 82 | version = "7.1.12"; 83 | sha256 = "sha256-JGHdRcAXii+hiOamrmy6GA5LG4a4aCfbF4o1LoHC4p0="; 84 | 85 | fdb-cli = pkgs.callPackage ./app { inherit name sha256 version; }; 86 | in 87 | opt-fdb-cli { inherit version fdb-cli; } 88 | ) 89 | ] 90 | -------------------------------------------------------------------------------- /nix/ci/fdb-files-6.3/default.nix: -------------------------------------------------------------------------------- 1 | { pkgs, version }: 2 | { 3 | conf = [ 4 | ( 5 | pkgs.writeTextDir "opt/fdb/conf/foundationdb.conf" 6 | ( 7 | builtins.readFile 8 | ( 9 | pkgs.substituteAll 10 | { 11 | src = ./foundationdb.conf; 12 | inherit version; 13 | } 14 | ) 15 | ) 16 | ) 17 | ( 18 | pkgs.writeTextDir "opt/fdb/conf/fdb.cluster" 19 | ( 20 | builtins.readFile ./fdb.cluster 21 | ) 22 | ) 23 | ]; 24 | 25 | systemd_units = { 26 | foundationdb_service = pkgs.writeTextDir "etc/systemd/system/foundationdb.service" 27 | ( 28 | builtins.readFile ./foundationdb.service 29 | ); 30 | 31 | fdbcli_service = pkgs.writeTextDir "etc/systemd/system/fdbcli.service" 32 | ( 33 | builtins.readFile 34 | ( 35 | pkgs.substituteAll 36 | { 37 | src = ./fdbcli.service; 38 | inherit version; 39 | } 40 | ) 41 | ); 42 | }; 43 | } 44 | -------------------------------------------------------------------------------- /nix/ci/fdb-files-6.3/fdb.cluster: -------------------------------------------------------------------------------- 1 | github_ci:fDbR00kS@127.0.0.1:4500 2 | -------------------------------------------------------------------------------- /nix/ci/fdb-files-6.3/fdbcli.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Setup FDB cluster 3 | After=foundationdb.service 4 | 5 | [Service] 6 | User=fdb 7 | Type=oneshot 8 | RemainAfterExit=yes 9 | ExecStart=/opt/fdb/cli/@version@/fdbcli -C /opt/fdb/conf/fdb.cluster --exec "configure new single memory" 10 | 11 | [Install] 12 | WantedBy=default.target 13 | -------------------------------------------------------------------------------- /nix/ci/fdb-files-6.3/foundationdb.conf: -------------------------------------------------------------------------------- 1 | [fdbmonitor] 2 | user = fdb 3 | group = fdb 4 | 5 | [general] 6 | cluster_file = /opt/fdb/conf/fdb.cluster 7 | restart_delay = 60 8 | 9 | [fdbserver] 10 | command = /opt/fdb/server/@version@/fdbserver 11 | public_address = auto:$ID 12 | listen_address = public 13 | datadir = /opt/fdb/data/$ID 14 | logdir = /opt/fdb/log 15 | 16 | [fdbserver.4500] 17 | -------------------------------------------------------------------------------- /nix/ci/fdb-files-6.3/foundationdb.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=FoundationDB Key-Value Store 3 | 4 | [Service] 5 | User=fdb 6 | ExecStart=/opt/fdb/monitor/fdbmonitor --conffile /opt/fdb/conf/foundationdb.conf 7 | 8 | [Install] 9 | WantedBy=default.target 10 | -------------------------------------------------------------------------------- /nix/ci/fdb-files-7.1/default.nix: -------------------------------------------------------------------------------- 1 | { pkgs, version }: 2 | { 3 | conf = [ 4 | ( 5 | pkgs.writeTextDir "opt/fdb/conf/foundationdb.conf" 6 | ( 7 | builtins.readFile 8 | ( 9 | pkgs.substituteAll 10 | { 11 | src = ./foundationdb.conf; 12 | inherit version; 13 | } 14 | ) 15 | ) 16 | ) 17 | ( 18 | pkgs.writeTextDir "opt/fdb/conf/fdb.cluster" 19 | ( 20 | builtins.readFile ./fdb.cluster 21 | ) 22 | ) 23 | ]; 24 | 25 | systemd_units = { 26 | foundationdb_service = pkgs.writeTextDir "etc/systemd/system/foundationdb.service" 27 | ( 28 | builtins.readFile ./foundationdb.service 29 | ); 30 | 31 | fdbcli_service = pkgs.writeTextDir "etc/systemd/system/fdbcli.service" 32 | ( 33 | builtins.readFile 34 | ( 35 | pkgs.substituteAll 36 | { 37 | src = ./fdbcli.service; 38 | inherit version; 39 | } 40 | ) 41 | ); 42 | }; 43 | } 44 | -------------------------------------------------------------------------------- /nix/ci/fdb-files-7.1/fdb.cluster: -------------------------------------------------------------------------------- 1 | github_ci:fDbR00kS@127.0.0.1:4500 2 | -------------------------------------------------------------------------------- /nix/ci/fdb-files-7.1/fdbcli.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Setup FDB cluster 3 | After=foundationdb.service 4 | 5 | [Service] 6 | User=fdb 7 | Type=oneshot 8 | RemainAfterExit=yes 9 | ExecStart=/opt/fdb/cli/@version@/fdbcli -C /opt/fdb/conf/fdb.cluster --exec "configure new single memory tenant_mode=optional_experimental" 10 | 11 | [Install] 12 | WantedBy=default.target 13 | -------------------------------------------------------------------------------- /nix/ci/fdb-files-7.1/foundationdb.conf: -------------------------------------------------------------------------------- 1 | [fdbmonitor] 2 | user = fdb 3 | group = fdb 4 | 5 | [general] 6 | cluster_file = /opt/fdb/conf/fdb.cluster 7 | restart_delay = 60 8 | 9 | [fdbserver] 10 | command = /opt/fdb/server/@version@/fdbserver 11 | public_address = auto:$ID 12 | listen_address = public 13 | datadir = /opt/fdb/data/$ID 14 | logdir = /opt/fdb/log 15 | 16 | [fdbserver.4500] 17 | -------------------------------------------------------------------------------- /nix/ci/fdb-files-7.1/foundationdb.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=FoundationDB Key-Value Store 3 | 4 | [Service] 5 | User=fdb 6 | ExecStart=/opt/fdb/monitor/fdbmonitor --conffile /opt/fdb/conf/foundationdb.conf 7 | 8 | [Install] 9 | WantedBy=default.target 10 | -------------------------------------------------------------------------------- /nix/ci/flake.lock: -------------------------------------------------------------------------------- 1 | { 2 | "nodes": { 3 | "flake-utils": { 4 | "locked": { 5 | "lastModified": 1656065134, 6 | "narHash": "sha256-oc6E6ByIw3oJaIyc67maaFcnjYOz1mMcOtHxbEf9NwQ=", 7 | "owner": "numtide", 8 | "repo": "flake-utils", 9 | "rev": "bee6a7250dd1b01844a2de7e02e4df7d8a0a206c", 10 | "type": "github" 11 | }, 12 | "original": { 13 | "owner": "numtide", 14 | "repo": "flake-utils", 15 | "type": "github" 16 | } 17 | }, 18 | "nixpkgs": { 19 | "locked": { 20 | "lastModified": 1657654771, 21 | "narHash": "sha256-uIGbt+rU89aTlSx8+4g9vKioWc8nmCCcaaFCNPHM07c=", 22 | "owner": "NixOS", 23 | "repo": "nixpkgs", 24 | "rev": "4aceab3cadf9fef6f70b9f6a9df964218650db0a", 25 | "type": "github" 26 | }, 27 | "original": { 28 | "id": "nixpkgs", 29 | "ref": "nixos-22.05", 30 | "type": "indirect" 31 | } 32 | }, 33 | "nixpkgs_2": { 34 | "locked": { 35 | "lastModified": 1656401090, 36 | "narHash": "sha256-bUS2nfQsvTQW2z8SK7oEFSElbmoBahOPtbXPm0AL3I4=", 37 | "owner": "NixOS", 38 | "repo": "nixpkgs", 39 | "rev": "16de63fcc54e88b9a106a603038dd5dd2feb21eb", 40 | "type": "github" 41 | }, 42 | "original": { 43 | "owner": "NixOS", 44 | "ref": "nixpkgs-unstable", 45 | "repo": "nixpkgs", 46 | "type": "github" 47 | } 48 | }, 49 | "root": { 50 | "inputs": { 51 | "nixpkgs": "nixpkgs", 52 | "rust-overlay": "rust-overlay" 53 | } 54 | }, 55 | "rust-overlay": { 56 | "inputs": { 57 | "flake-utils": "flake-utils", 58 | "nixpkgs": "nixpkgs_2" 59 | }, 60 | "locked": { 61 | "lastModified": 1657767064, 62 | "narHash": "sha256-Mp7LmSPnRfWqX7OElXr4HKNbTiDCXLaxijp23xQlDJk=", 63 | "owner": "oxalica", 64 | "repo": "rust-overlay", 65 | "rev": "db9443ca1384f94c0aa63f4e74115f7c0632a8e6", 66 | "type": "github" 67 | }, 68 | "original": { 69 | "owner": "oxalica", 70 | "repo": "rust-overlay", 71 | "type": "github" 72 | } 73 | } 74 | }, 75 | "root": "root", 76 | "version": 7 77 | } 78 | -------------------------------------------------------------------------------- /nix/ci/nss/default.nix: -------------------------------------------------------------------------------- 1 | { pkgs }: 2 | [ 3 | # For FoundationDB it looks [1] like the recommended uid/gid is `4059` 4 | # and username/groupname is `fdb`. 5 | # 6 | # GitHub runs jobs in a ubuntu virtual machine using the user 7 | # `runner` whose `uid` is `1001`. The user `runner` primary group is 8 | # `docker`, whose `gid` is `121`. The path where the repository is 9 | # checked out is given by the environment variable 10 | # `GITHUB_WORKSPACE` [2] . We will assume the container is started 11 | # by bind-mounting `GITHUB_WORKSPACE` to `/home/runner/fdb`. 12 | # 13 | # `nixbld` group is required in order to prevent warning from `nix` 14 | # command. `nixbldX` users for `nix` to run in daemon mode. 15 | # 16 | # Also see `DEVELOP.md`. 17 | # 18 | # [1]: https://github.com/FoundationDB/fdb-kubernetes-operator/blob/v0.51.0/Dockerfile#L61-L66 19 | # [2]: https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables 20 | ( 21 | pkgs.writeTextDir "etc/passwd" '' 22 | root:x:0:0:root user:/var/empty:/bin/sh 23 | runner:x:1001:121:runner user:/home/runner:/bin/sh 24 | fdb:x:4059:4059:fdb user:/var/empty:/bin/sh 25 | nixbld1:x:30001:30000:Nix build user 1:/var/empty:/bin/nologin 26 | nixbld2:x:30002:30000:Nix build user 2:/var/empty:/bin/nologin 27 | nixbld3:x:30003:30000:Nix build user 3:/var/empty:/bin/nologin 28 | nixbld4:x:30004:30000:Nix build user 4:/var/empty:/bin/nologin 29 | nixbld5:x:30005:30000:Nix build user 5:/var/empty:/bin/nologin 30 | nixbld6:x:30006:30000:Nix build user 6:/var/empty:/bin/nologin 31 | nixbld7:x:30007:30000:Nix build user 7:/var/empty:/bin/nologin 32 | nixbld8:x:30008:30000:Nix build user 8:/var/empty:/bin/nologin 33 | nobody:x:65534:65534:nobody:/var/empty:/bin/nologin 34 | '' 35 | ) 36 | ( 37 | pkgs.writeTextDir "etc/group" '' 38 | root:x:0: 39 | docker:x:121: 40 | fdb:x:4059: 41 | nixbld:x:30000:nixbld1,nixbld2,nixbld3,nixbld4,nixbld5,nixbld6,nixbld7,nixbld8 42 | nobody:x:65534: 43 | '' 44 | ) 45 | ( 46 | pkgs.writeTextDir "etc/nsswitch.conf" '' 47 | hosts: files dns 48 | '' 49 | ) 50 | ( 51 | pkgs.runCommand "var-empty" { } '' 52 | mkdir -p $out/var/empty 53 | '' 54 | ) 55 | ] 56 | 57 | -------------------------------------------------------------------------------- /nix/ci/systemd/default.nix: -------------------------------------------------------------------------------- 1 | { pkgs }: 2 | rec { 3 | default_target = pkgs.writeTextDir "etc/systemd/system/default.target" 4 | ( 5 | builtins.readFile 6 | ( 7 | pkgs.substituteAll 8 | { 9 | src = ./unit-files/default.target; 10 | sysinit_target_name = sysinit_target.name; 11 | systemd_journald_service_name = systemd_journald_service.name; 12 | } 13 | ) 14 | ); 15 | 16 | halt_target = pkgs.writeTextDir "etc/systemd/system/halt.target" 17 | ( 18 | builtins.readFile 19 | ( 20 | pkgs.substituteAll 21 | { 22 | src = ./unit-files/halt.target; 23 | halt_service_name = halt_service.name; 24 | } 25 | ) 26 | ); 27 | 28 | halt_service = pkgs.writeTextDir "etc/systemd/system/halt.service" 29 | ( 30 | builtins.readFile 31 | ( 32 | pkgs.substituteAll 33 | { 34 | src = ./unit-files/halt.service; 35 | systemd = pkgs.systemd; 36 | } 37 | ) 38 | ); 39 | 40 | sysinit_target = pkgs.writeTextDir "etc/systemd/system/sysinit.target" 41 | ( 42 | builtins.readFile ./unit-files/sysinit.target 43 | ); 44 | 45 | systemd_journald_socket = pkgs.writeTextDir "etc/systemd/system/systemd-journald.socket" 46 | ( 47 | builtins.readFile ./unit-files/systemd-journald.socket 48 | ); 49 | 50 | systemd_journald_service = pkgs.writeTextDir "etc/systemd/system/systemd-journald.service" 51 | ( 52 | builtins.readFile 53 | ( 54 | pkgs.substituteAll 55 | { 56 | src = ./unit-files/systemd-journald.service; 57 | systemd = pkgs.systemd; 58 | systemd_journald_socket_name = systemd_journald_socket.name; 59 | } 60 | ) 61 | ); 62 | 63 | nix_daemon_socket = pkgs.writeTextDir "etc/systemd/system/nix-daemon.socket" 64 | ( 65 | builtins.readFile ./unit-files/nix-daemon.socket 66 | ); 67 | 68 | nix_daemon_service = pkgs.writeTextDir "etc/systemd/system/nix-daemon.service" 69 | ( 70 | builtins.readFile 71 | ( 72 | pkgs.substituteAll 73 | { 74 | src = ./unit-files/nix-daemon.service; 75 | nix = pkgs.nixUnstable; 76 | nix_daemon_socket_name = nix_daemon_socket.name; 77 | cacert = pkgs.cacert; 78 | } 79 | ) 80 | ); 81 | } 82 | -------------------------------------------------------------------------------- /nix/ci/systemd/unit-files/default.target: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Default target 3 | Requires=@sysinit_target_name@ @systemd_journald_service_name@ 4 | -------------------------------------------------------------------------------- /nix/ci/systemd/unit-files/halt.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Halt service 3 | DefaultDependencies=no 4 | 5 | [Service] 6 | ExecStart=@systemd@/bin/systemctl --force halt 7 | -------------------------------------------------------------------------------- /nix/ci/systemd/unit-files/halt.target: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Halt target 3 | Requires=@halt_service_name@ 4 | -------------------------------------------------------------------------------- /nix/ci/systemd/unit-files/nix-daemon.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Nix Daemon 3 | ConditionPathIsReadWrite=/nix/var/nix/daemon-socket 4 | DefaultDependencies=no 5 | 6 | [Service] 7 | ExecStart=@@nix@/bin/nix-daemon nix-daemon --daemon 8 | Sockets=@nix_daemon_socket_name@ 9 | KillMode=process 10 | Environment="NIX_SSL_CERT_FILE=@cacert@/etc/ssl/certs/ca-bundle.crt" 11 | -------------------------------------------------------------------------------- /nix/ci/systemd/unit-files/nix-daemon.socket: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Nix Daemon Socket 3 | DefaultDependencies=no 4 | ConditionPathIsReadWrite=/nix/var/nix/daemon-socket 5 | 6 | [Socket] 7 | ListenStream=/nix/var/nix/daemon-socket/socket 8 | 9 | [Install] 10 | WantedBy=default.target 11 | -------------------------------------------------------------------------------- /nix/ci/systemd/unit-files/sysinit.target: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Sysinit target 3 | -------------------------------------------------------------------------------- /nix/ci/systemd/unit-files/systemd-journald.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Journal service 3 | DefaultDependencies=no 4 | 5 | [Service] 6 | ExecStart=@systemd@/lib/systemd/systemd-journald 7 | Sockets=@systemd_journald_socket_name@ 8 | -------------------------------------------------------------------------------- /nix/ci/systemd/unit-files/systemd-journald.socket: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Journal socket 3 | DefaultDependencies=no 4 | 5 | [Socket] 6 | ListenStream=/run/systemd/journal/stdout 7 | ListenDatagram=/run/systemd/journal/socket 8 | --------------------------------------------------------------------------------