├── .github ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── ISSUE_TEMPLATE.md ├── PULL_REQUEST_TEMPLATE.md ├── stale.yml └── workflows │ └── ci.yml ├── .gitignore ├── .travis.yml ├── CERTIFICATE ├── CHANGELOG.md ├── Cargo.toml ├── LICENSE-APACHE ├── LICENSE-MIT ├── README.md ├── benches └── sync.rs ├── fuzz ├── .gitignore ├── Cargo.toml └── fuzz_targets │ └── fuzz_target_1.rs ├── rustfmt.toml ├── src └── lib.rs └── tests ├── model.rs ├── regression.rs └── test.rs /.github/CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, gender identity and expression, level of 9 | experience, 10 | education, socio-economic status, nationality, personal appearance, race, 11 | religion, or sexual identity and orientation. 12 | 13 | ## Our Standards 14 | 15 | Examples of behavior that contributes to creating a positive environment 16 | include: 17 | 18 | - Using welcoming and inclusive language 19 | - Being respectful of differing viewpoints and experiences 20 | - Gracefully accepting constructive criticism 21 | - Focusing on what is best for the community 22 | - Showing empathy towards other community members 23 | 24 | Examples of unacceptable behavior by participants include: 25 | 26 | - The use of sexualized language or imagery and unwelcome sexual attention or 27 | advances 28 | - Trolling, insulting/derogatory comments, and personal or political attacks 29 | - Public or private harassment 30 | - Publishing others' private information, such as a physical or electronic 31 | address, without explicit permission 32 | - Other conduct which could reasonably be considered inappropriate in a 33 | professional setting 34 | 35 | 36 | ## Our Responsibilities 37 | 38 | Project maintainers are responsible for clarifying the standards of acceptable 39 | behavior and are expected to take appropriate and fair corrective action in 40 | response to any instances of unacceptable behavior. 41 | 42 | Project maintainers have the right and responsibility to remove, edit, or 43 | reject comments, commits, code, wiki edits, issues, and other contributions 44 | that are not aligned to this Code of Conduct, or to ban temporarily or 45 | permanently any contributor for other behaviors that they deem inappropriate, 46 | threatening, offensive, or harmful. 47 | 48 | ## Scope 49 | 50 | This Code of Conduct applies both within project spaces and in public spaces 51 | when an individual is representing the project or its community. Examples of 52 | representing a project or community include using an official project e-mail 53 | address, posting via an official social media account, or acting as an appointed 54 | representative at an online or offline event. Representation of a project may be 55 | further defined and clarified by project maintainers. 56 | 57 | ## Enforcement 58 | 59 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 60 | reported by contacting the project team at yoshuawuyts@gmail.com, or through 61 | IRC. All complaints will be reviewed and investigated and will result in a 62 | response that is deemed necessary and appropriate to the circumstances. The 63 | project team is obligated to maintain confidentiality with regard to the 64 | reporter of an incident. 65 | Further details of specific enforcement policies may be posted separately. 66 | 67 | Project maintainers who do not follow or enforce the Code of Conduct in good 68 | faith may face temporary or permanent repercussions as determined by other 69 | members of the project's leadership. 70 | 71 | ## Attribution 72 | 73 | This Code of Conduct is adapted from the Contributor Covenant, version 1.4, 74 | available at 75 | https://www.contributor-covenant.org/version/1/4/code-of-conduct.html 76 | -------------------------------------------------------------------------------- /.github/CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | Contributions include code, documentation, answering user questions, running the 3 | project's infrastructure, and advocating for all types of users. 4 | 5 | The project welcomes all contributions from anyone willing to work in good faith 6 | with other contributors and the community. No contribution is too small and all 7 | contributions are valued. 8 | 9 | This guide explains the process for contributing to the project's GitHub 10 | Repository. 11 | 12 | - [Code of Conduct](#code-of-conduct) 13 | - [Bad Actors](#bad-actors) 14 | - [Developer Certificate of Origin](#developer-certificate-of-origin) 15 | 16 | ## Code of Conduct 17 | The project has a [Code of Conduct][./CODE_OF_CONDUCT.md] that *all* 18 | contributors are expected to follow. This code describes the *minimum* behavior 19 | expectations for all contributors. 20 | 21 | As a contributor, how you choose to act and interact towards your 22 | fellow contributors, as well as to the community, will reflect back not only 23 | on yourself but on the project as a whole. The Code of Conduct is designed and 24 | intended, above all else, to help establish a culture within the project that 25 | allows anyone and everyone who wants to contribute to feel safe doing so. 26 | 27 | Should any individual act in any way that is considered in violation of the 28 | [Code of Conduct][./CODE_OF_CONDUCT.md], corrective actions will be taken. It is 29 | possible, however, for any individual to *act* in such a manner that is not in 30 | violation of the strict letter of the Code of Conduct guidelines while still 31 | going completely against the spirit of what that Code is intended to accomplish. 32 | 33 | Open, diverse, and inclusive communities live and die on the basis of trust. 34 | Contributors can disagree with one another so long as they trust that those 35 | disagreements are in good faith and everyone is working towards a common 36 | goal. 37 | 38 | ## Bad Actors 39 | All contributors to tacitly agree to abide by both the letter and 40 | spirit of the [Code of Conduct][./CODE_OF_CONDUCT.md]. Failure, or 41 | unwillingness, to do so will result in contributions being respectfully 42 | declined. 43 | 44 | A *bad actor* is someone who repeatedly violates the *spirit* of the Code of 45 | Conduct through consistent failure to self-regulate the way in which they 46 | interact with other contributors in the project. In doing so, bad actors 47 | alienate other contributors, discourage collaboration, and generally reflect 48 | poorly on the project as a whole. 49 | 50 | Being a bad actor may be intentional or unintentional. Typically, unintentional 51 | bad behavior can be easily corrected by being quick to apologize and correct 52 | course *even if you are not entirely convinced you need to*. Giving other 53 | contributors the benefit of the doubt and having a sincere willingness to admit 54 | that you *might* be wrong is critical for any successful open collaboration. 55 | 56 | Don't be a bad actor. 57 | 58 | ## Developer Certificate of Origin 59 | All contributors must read and agree to the [Developer Certificate of 60 | Origin (DCO)](../CERTIFICATE). 61 | 62 | The DCO allows us to accept contributions from people to the project, similarly 63 | to how a license allows us to distribute our code. 64 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | 7 | 8 | **Choose one:** is this a 🐛 bug report, a 🙋 feature request, or a 🔦 question? 9 | 10 | 11 | 12 | ## Expected Behavior 13 | 14 | 15 | 16 | 17 | ## Current Behavior 18 | 19 | 20 | 21 | 22 | ## Possible Solution 23 | 24 | 25 | 26 | ## Context 27 | 28 | 29 | 30 | ## Code Sample 31 | 32 | 33 | ## Your Environment 34 | 35 | 36 | | Software | Version(s) | 37 | | ---------------- | ---------- | 38 | | Package | 39 | | Runtime | 40 | | Package Manager | 41 | | Operating System | 42 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | **Choose one:** is this a 🐛 bug fix, a 🙋 feature, or a 🔦 documentation change? 8 | 9 | 10 | 11 | ## Checklist 12 | 13 | - [ ] tests pass 14 | - [ ] tests and/or benchmarks are included 15 | - [ ] documentation is changed or added 16 | 17 | ## Context 18 | 19 | 20 | ## Semver Changes 21 | 22 | -------------------------------------------------------------------------------- /.github/stale.yml: -------------------------------------------------------------------------------- 1 | # Configuration for probot-stale - https://github.com/probot/stale 2 | 3 | daysUntilStale: 90 4 | daysUntilClose: 7 5 | exemptLabels: 6 | - pinned 7 | - security 8 | exemptProjects: false 9 | exemptMilestones: false 10 | staleLabel: wontfix 11 | markComment: > 12 | This issue has been automatically marked as stale because it has not had 13 | recent activity. It will be closed if no further activity occurs. Thank you 14 | for your contributions. 15 | unmarkComment: false 16 | closeComment: false 17 | limitPerRun: 30 18 | -------------------------------------------------------------------------------- /.github/workflows/ci.yml: -------------------------------------------------------------------------------- 1 | name: 'CI' 2 | on: 3 | pull_request: 4 | push: 5 | branches: 6 | - master 7 | 8 | env: 9 | RUST_BACKTRACE: 1 10 | CARGO_TERM_COLOR: always 11 | 12 | jobs: 13 | ci-pass: 14 | name: CI is green 15 | runs-on: ubuntu-latest 16 | needs: 17 | - test-linux 18 | - test-windows 19 | - test-macos 20 | - build-extra 21 | - lint 22 | steps: 23 | - run: exit 0 24 | 25 | test-linux: 26 | runs-on: ubuntu-latest 27 | 28 | steps: 29 | - uses: actions/checkout@v3 30 | - uses: dtolnay/rust-toolchain@stable 31 | with: 32 | components: rustfmt 33 | - name: Run tests 34 | run: | 35 | cargo check 36 | cargo test 37 | 38 | test-windows: 39 | runs-on: windows-latest 40 | 41 | steps: 42 | - uses: actions/checkout@v3 43 | - uses: dtolnay/rust-toolchain@stable 44 | with: 45 | components: rustfmt 46 | - name: Run tests 47 | run: | 48 | cargo check 49 | cargo test 50 | 51 | test-macos: 52 | runs-on: macos-latest 53 | 54 | steps: 55 | - uses: actions/checkout@v3 56 | - uses: dtolnay/rust-toolchain@stable 57 | with: 58 | components: rustfmt 59 | - name: Run tests 60 | run: | 61 | cargo check 62 | cargo test 63 | 64 | build-extra: 65 | runs-on: ubuntu-latest 66 | 67 | steps: 68 | - uses: actions/checkout@v3 69 | - uses: dtolnay/rust-toolchain@stable 70 | with: 71 | targets: wasm32-unknown-unknown 72 | - name: Build WASM 73 | run: | 74 | cargo build --target=wasm32-unknown-unknown 75 | - name: Build benches 76 | run: | 77 | cargo build --benches 78 | - name: Build release 79 | run: | 80 | cargo build --release 81 | 82 | lint: 83 | runs-on: ubuntu-latest 84 | 85 | steps: 86 | - uses: actions/checkout@v3 87 | - uses: dtolnay/rust-toolchain@stable 88 | with: 89 | components: rustfmt, clippy 90 | - uses: actions-rs/clippy-check@v1 91 | with: 92 | token: ${{ secrets.GITHUB_TOKEN }} 93 | - name: Format check 94 | run: | 95 | cargo fmt -- --check 96 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | node_modules/ 2 | coverage/ 3 | tmp/ 4 | dist/ 5 | npm-debug.log* 6 | .DS_Store 7 | .nyc_output 8 | target/ 9 | Cargo.lock 10 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: rust 2 | rust: 3 | - stable 4 | 5 | before_script: | 6 | rustup component add rustfmt && 7 | rustup component add clippy && 8 | rustup update nightly 9 | 10 | script: | 11 | cargo +nightly check --all-targets --verbose && 12 | cargo fmt -- --check && 13 | cargo clippy -- -D clippy::all && 14 | cargo build --verbose && 15 | cargo test --verbose 16 | cache: cargo 17 | -------------------------------------------------------------------------------- /CERTIFICATE: -------------------------------------------------------------------------------- 1 | Developer Certificate of Origin 2 | Version 1.1 3 | 4 | Copyright (C) 2004, 2006 The Linux Foundation and its contributors. 5 | 1 Letterman Drive 6 | Suite D4700 7 | San Francisco, CA, 94129 8 | 9 | Everyone is permitted to copy and distribute verbatim copies of this 10 | license document, but changing it is not allowed. 11 | 12 | 13 | Developer's Certificate of Origin 1.1 14 | 15 | By making a contribution to this project, I certify that: 16 | 17 | (a) The contribution was created in whole or in part by me and I 18 | have the right to submit it under the open source license 19 | indicated in the file; or 20 | 21 | (b) The contribution is based upon previous work that, to the best 22 | of my knowledge, is covered under an appropriate open source 23 | license and I have the right under that license to submit that 24 | work with modifications, whether created in whole or in part 25 | by me, under the same open source license (unless I am 26 | permitted to submit under a different license), as indicated 27 | in the file; or 28 | 29 | (c) The contribution was provided directly to me by some other 30 | person who certified (a), (b) or (c) and I have not modified 31 | it. 32 | 33 | (d) I understand and agree that this project and the contribution 34 | are public and that a record of the contribution (including all 35 | personal information I submit with it, including my sign-off) is 36 | maintained indefinitely and may be redistributed consistent with 37 | this project or the open source license(s) involved. 38 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | ## 2023-04-12, Version 3.0.0 2 | ### Commits 3 | - [[`ef600832dd`](https://github.com/datrs/random-access-memory/commit/ef600832ddc13d2ba90250ef710a9e0b142e4bad)] Release 3.0.0 (Timo Tiuraniemi) 4 | - [[`e6d7e97ed1`](https://github.com/datrs/random-access-memory/commit/e6d7e97ed1ca4e953a8279e79b614db15562b999)] Switch from Travis to GHA (Timo Tiuraniemi) 5 | - [[`08a7da6cdd`](https://github.com/datrs/random-access-memory/commit/08a7da6cdd016c4fc8961849de9ec716a18e2a19)] Use released 5.0.0 of random-access-storage (Timo Tiuraniemi) 6 | - [[`d69cde848c`](https://github.com/datrs/random-access-memory/commit/d69cde848c3d9591c818739065c122a019f862f4)] Switch from quickcheck to proptest (Timo Tiuraniemi) 7 | - [[`27f5c02748`](https://github.com/datrs/random-access-memory/commit/27f5c0274806b793bb3fa1101bd948ee3046db35)] Use beta.0 as the version until dependencies are bumped (Timo Tiuraniemi) 8 | - [[`2c74bccf2c`](https://github.com/datrs/random-access-memory/commit/2c74bccf2c4922ac7507616e66e0dce5af3d0bd8)] Set version to 3.0.0, bump edition to 2021 (Timo Tiuraniemi) 9 | - [[`c47c2db567`](https://github.com/datrs/random-access-memory/commit/c47c2db5674a327d4397b72d24f13c3fbef273f9)] Fix clippy issues, forbid unsafe code (Timo Tiuraniemi) 10 | - [[`fcd398ada1`](https://github.com/datrs/random-access-memory/commit/fcd398ada15ed9f50a0483884a98cfd94b9ae700)] Complete documentation (Timo Tiuraniemi) 11 | - [[`5dd26f3165`](https://github.com/datrs/random-access-memory/commit/5dd26f316551dd343dff81ffed9b1e64229a4162)] Use a single OutOfBounds error for more convenient error matching (Timo Tiuraniemi) 12 | - [[`9ebea000d6`](https://github.com/datrs/random-access-memory/commit/9ebea000d63f63ad5b0532ef2cb37c870ea0d754)] Switch from anyhow to thiserror and RandomAccessError (Timo Tiuraniemi) 13 | - [[`ecf536f285`](https://github.com/datrs/random-access-memory/commit/ecf536f2852d436f018f286a40a9fedd8774a3af)] Enforce delete offset and truncate when larger (Timo Tiuraniemi) 14 | - [[`606b1e33eb`](https://github.com/datrs/random-access-memory/commit/606b1e33eb291b5d87673fa8414b916ef43fa890)] Migrate bench from libtest to criterion, add del test (Timo Tiuraniemi) 15 | - [[`b73a3c653e`](https://github.com/datrs/random-access-memory/commit/b73a3c653e99a1b3b97a28f87446f35d57ce5816)] Use tokio-compatible RandomAccessStorage (Timo Tiuraniemi) 16 | - [[`3e90f7cc41`](https://github.com/datrs/random-access-memory/commit/3e90f7cc41ef565bd496678dca515041f8a6ed9e)] Fix len() missing &mut (Timo Tiuraniemi) 17 | - [[`5b064d7e71`](https://github.com/datrs/random-access-memory/commit/5b064d7e7174d344592e0da2e1d2a1dce4960865)] feat: del and truncate using intmap (Timo Tiuraniemi) 18 | - [[`6c9761f53a`](https://github.com/datrs/random-access-memory/commit/6c9761f53a3da5fed7b0400cf949958bb3fe5f24)] Update changelog (Bruno Tavares) 19 | 20 | ### Stats 21 | ```diff 22 | .github/workflows/ci.yml | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 23 | CHANGELOG.md | 20 +++++++++++++++ 24 | Cargo.toml | 26 +++++++++++-------- 25 | README.md | 25 +++++------------- 26 | benches/sync.rs | 59 +++++++++++++++++++++++++++--------------- 27 | src/lib.rs | 287 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------------------------- 28 | tests/model.rs | 77 +++++++++++++++++++++++++++++++++++++------------------ 29 | tests/test.rs | 96 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 30 | 8 files changed, 526 insertions(+), 159 deletions(-) 31 | ``` 32 | 33 | 34 | ## 2020-03-03, Version 2.0.0 35 | ### Commits 36 | - [[`cab67711c4`](https://github.com/datrs/random-access-memory/commit/cab67711c42f6229fcf01fac0bfae0b5d0a3fe86)] (cargo-release) version 2.0.0 (Bruno Tavares) 37 | - [[`fc07e9cfef`](https://github.com/datrs/random-access-memory/commit/fc07e9cfefdc808f950fada80df7bfe697eaad6d)] Implement async API for random-access-storage (#26) (Bruno Tavares) 38 | - [[`c10203a920`](https://github.com/datrs/random-access-memory/commit/c10203a9206c6e5f93f31b34998096fa1828d947)] Update changelog (Bruno Tavares) 39 | 40 | ### Stats 41 | ```diff 42 | CHANGELOG.md | 13 +++++++++++++ 43 | Cargo.toml | 7 +++++-- 44 | README.md | 8 ++++---- 45 | benches/sync.rs | 39 +++++++++++++++++++++++---------------- 46 | src/lib.rs | 28 ++++++++++++++++++---------- 47 | tests/model.rs | 52 +++++++++++++++++++++++++++------------------------- 48 | tests/regression.rs | 36 ++++++++++++++++++------------------ 49 | tests/test.rs | 52 ++++++++++++++++++++++++++-------------------------- 50 | 8 files changed, 134 insertions(+), 101 deletions(-) 51 | ``` 52 | 53 | 54 | ## 2020-03-03, Version 1.2.0 55 | ### Commits 56 | - [[`d9fbf73cf1`](https://github.com/datrs/random-access-memory/commit/d9fbf73cf182e6f02e5adc4c016afe38e8650a88)] (cargo-release) version 1.2.0 (Bruno Tavares) 57 | - [[`0b845fbcc7`](https://github.com/datrs/random-access-memory/commit/0b845fbcc747093d8c9eae972e92c8d4a692a208)] Move from failure to use stderr (#25) (Bruno Tavares) 58 | 59 | ### Stats 60 | ```diff 61 | Cargo.toml | 4 ++-- 62 | src/lib.rs | 24 +++++++++++++----------- 63 | 2 files changed, 15 insertions(+), 13 deletions(-) 64 | ``` 65 | 66 | 67 | ## 2019-04-24, Version 1.0.0 68 | ### Commits 69 | - [[`a7ff8ea564`](https://github.com/datrs/random-access-memory/commit/a7ff8ea564f11673b3e39d605befe9b25fc58574)] (cargo-release) version 1.0.0 (Yoshua Wuyts) 70 | - [[`76c0ebfe1b`](https://github.com/datrs/random-access-memory/commit/76c0ebfe1ba046871430f03fe5876a4e27554fb6)] Update random-access-storage to 2.0.0 (#12) (Jack Jennings) 71 | - [[`defaf55071`](https://github.com/datrs/random-access-memory/commit/defaf55071de68206eada89a74616d26603e2faa)] Update quickcheck requirement from 0.7.1 to 0.8.1 (dependabot[bot]) 72 | - [[`42023a8700`](https://github.com/datrs/random-access-memory/commit/42023a870050fdbc196c6f7d2ffd3e7c36797fac)] Update rand requirement from 0.5.5 to 0.6.0 (dependabot[bot]) 73 | - [[`9202fe0308`](https://github.com/datrs/random-access-memory/commit/9202fe0308151dce780c7721f533bed2e11f7f3c)] Update quickcheck requirement from 0.6.2 to 0.7.1 (#6) (dependabot[bot]) 74 | - [[`eae681682e`](https://github.com/datrs/random-access-memory/commit/eae681682eef6a4fe888f25b8728951873b6f114)] Run clippy on travis (#5) (Szabolcs Berecz) 75 | - [[`df0755085c`](https://github.com/datrs/random-access-memory/commit/df0755085c1e472b084f9598aaa2ac0d4fb04f10)] update changelog (Yoshua Wuyts) 76 | 77 | ### Stats 78 | ```diff 79 | .travis.yml | 4 +++- 80 | CHANGELOG.md | 20 ++++++++++++++++++++ 81 | Cargo.toml | 7 ++++--- 82 | src/lib.rs | 12 ++++++++++++ 83 | tests/model.rs | 2 ++ 84 | tests/test.rs | 18 ++++++++++++++++++ 85 | 6 files changed, 59 insertions(+), 4 deletions(-) 86 | ``` 87 | 88 | 89 | ## 2018-08-30, Version 0.5.0 90 | ### Commits 91 | - [[`3baa7c2d23`](https://github.com/datrs/random-access-memory/commits/3baa7c2d23dfa774ac5e1d2b38bbb171eaf95bc0)] (cargo-release) version 0.5.0 (Yoshua Wuyts) 92 | - [[`300ead96ab`](https://github.com/datrs/random-access-memory/commits/300ead96ab4eab5b66f786f3b0562ddb29571d27)] Random access always open (#4) (Szabolcs Berecz) 93 | - [[`db908834f8`](https://github.com/datrs/random-access-memory/commits/db908834f8e79dd8a8f98a22e1403da3ebf458da)] update changelog (Yoshua Wuyts) 94 | 95 | ### Stats 96 | ```diff 97 | .travis.yml | 1 +- 98 | CHANGELOG.md | 23 +++++++++++++++++- 99 | Cargo.toml | 4 +-- 100 | benches/sync.rs | 2 +- 101 | src/lib.rs | 74 ++++++++++++++++++++++++------------------------------ 102 | tests/model.rs | 2 +- 103 | tests/regression.rs | 8 ++---- 104 | tests/test.rs | 4 ++- 105 | 8 files changed, 69 insertions(+), 49 deletions(-) 106 | ``` 107 | 108 | 109 | ## 2018-08-23, Version 0.4.0 110 | ### Commits 111 | - [[`7876a1a1ca`](https://github.com/datrs/random-access-memory/commits/7876a1a1ca10913a6126b767f4d07c3bae99dd8e)] (cargo-release) version 0.4.0 (Yoshua Wuyts) 112 | - [[`cc83784775`](https://github.com/datrs/random-access-memory/commits/cc83784775a7cc737c97d514e88cdcb9ca2e718a)] upgrade random-access-storage (#3) 113 | 114 | * upgrade random-access-storage 115 | 116 | * cargo fmt (Yoshua Wuyts) 117 | - [[`fae1a6509b`](https://github.com/datrs/random-access-memory/commits/fae1a6509b3c3825b3c063a627f64f85ab11cf40)] fix rustfmt in travis.yml (Yoshua Wuyts) 118 | - [[`51c944434b`](https://github.com/datrs/random-access-memory/commits/51c944434b46c16ef7ba4028ac31ec10d6d64fe3)] fix benches (Yoshua Wuyts) 119 | - [[`abfd505e04`](https://github.com/datrs/random-access-memory/commits/abfd505e049da526af068f475c1e8730e33238b9)] (cargo-release) start next development iteration 0.3.1-alpha.0 (Yoshua Wuyts) 120 | 121 | ### Stats 122 | ```diff 123 | .travis.yml | 2 +- 124 | Cargo.toml | 4 ++-- 125 | benches/sync.rs | 6 +++--- 126 | src/lib.rs | 23 ++++++++++++++++------- 127 | tests/regression.rs | 4 +--- 128 | 5 files changed, 23 insertions(+), 16 deletions(-) 129 | ``` 130 | 131 | 132 | -------------------------------------------------------------------------------- /Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | name = "random-access-memory" 3 | version = "3.0.0" 4 | license = "MIT OR Apache-2.0" 5 | repository = "https://github.com/datrs/random-access-memory" 6 | documentation = "https://docs.rs/random-access-memory" 7 | description = "Continuously read and write to memory using random offsets and lengths" 8 | authors = ["datrs ", "Timo Tiuraniemi "] 9 | readme = "README.md" 10 | edition = "2021" 11 | 12 | [dependencies] 13 | random-access-storage = "5.0.0" 14 | async-trait = "0.1" 15 | intmap = "2.0.0" 16 | 17 | [dev-dependencies] 18 | proptest = "1.1.0" 19 | proptest-derive = "0.2.0" 20 | async-std = { version = "1.12.0", features = ["attributes"] } 21 | criterion = "0.4" 22 | 23 | [[bench]] 24 | name = "sync" 25 | harness = false 26 | -------------------------------------------------------------------------------- /LICENSE-APACHE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | Copyright 2018 Yoshua Wuyts 179 | 180 | Licensed under the Apache License, Version 2.0 (the "License"); 181 | you may not use this file except in compliance with the License. 182 | You may obtain a copy of the License at 183 | 184 | http://www.apache.org/licenses/LICENSE-2.0 185 | 186 | Unless required by applicable law or agreed to in writing, software 187 | distributed under the License is distributed on an "AS IS" BASIS, 188 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 189 | See the License for the specific language governing permissions and 190 | limitations under the License. 191 | -------------------------------------------------------------------------------- /LICENSE-MIT: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2018 Yoshua Wuyts 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # random-access-memory 2 | [![crates.io version][1]][2] [![build status][3]][4] 3 | [![downloads][5]][6] [![docs.rs docs][7]][8] 4 | 5 | Continuously read and write to memory using random offsets and lengths 6 | using abstract interface defined in 7 | [random-access-storage](https://github.com/datrs/random-access-storage). 8 | 9 | - [Documentation][8] 10 | - [Crate][2] 11 | 12 | ## Installation 13 | ```sh 14 | $ cargo add random-access-memory 15 | ``` 16 | 17 | ## License 18 | [MIT](./LICENSE-MIT) OR [Apache-2.0](./LICENSE-APACHE) 19 | 20 | [1]: https://img.shields.io/crates/v/random-access-memory.svg?style=flat-square 21 | [2]: https://crates.io/crates/random-access-memory 22 | [3]: https://github.com/datrs/random-access-memory/actions/workflows/ci.yml/badge.svg 23 | [4]: https://github.com/datrs/random-access-memory/actions 24 | [5]: https://img.shields.io/crates/d/random-access-memory.svg?style=flat-square 25 | [6]: https://crates.io/crates/random-access-memory 26 | [7]: https://img.shields.io/badge/docs-latest-blue.svg?style=flat-square 27 | [8]: https://docs.rs/random-access-memory 28 | -------------------------------------------------------------------------------- /benches/sync.rs: -------------------------------------------------------------------------------- 1 | use criterion::{criterion_group, criterion_main, Criterion}; 2 | use random_access_memory as ram; 3 | use random_access_storage::RandomAccess; 4 | 5 | fn write_hello_world(c: &mut Criterion) { 6 | c.bench_function("write hello world", |b| { 7 | b.iter(|| { 8 | async_std::task::block_on(async { 9 | let mut file = ram::RandomAccessMemory::default(); 10 | file.write(0, b"hello").await.unwrap(); 11 | file.write(5, b" world").await.unwrap(); 12 | }) 13 | }) 14 | }); 15 | } 16 | 17 | fn read_hello_world(c: &mut Criterion) { 18 | c.bench_function("read hello world", |b| { 19 | async_std::task::block_on(async { 20 | let mut file = ram::RandomAccessMemory::default(); 21 | file.write(0, b"hello").await.unwrap(); 22 | file.write(5, b" world").await.unwrap(); 23 | b.iter(|| { 24 | async_std::task::block_on(async { 25 | let _text = file.read(0, 11).await.unwrap(); 26 | }) 27 | }); 28 | }) 29 | }); 30 | } 31 | 32 | fn read_write_hello_world(c: &mut Criterion) { 33 | c.bench_function("read/write hello world", |b| { 34 | b.iter(|| { 35 | async_std::task::block_on(async { 36 | let mut file = ram::RandomAccessMemory::default(); 37 | file.write(0, b"hello").await.unwrap(); 38 | file.write(5, b" world").await.unwrap(); 39 | let _text = file.read(0, 11).await.unwrap(); 40 | }) 41 | }) 42 | }); 43 | } 44 | 45 | fn write_del_hello_world(c: &mut Criterion) { 46 | c.bench_function("write/del hello world", |b| { 47 | b.iter(|| { 48 | async_std::task::block_on(async { 49 | let mut file = ram::RandomAccessMemory::default(); 50 | file.write(0, b"hello world").await.unwrap(); 51 | file.del(0, 5).await.unwrap(); 52 | file.del(5, 6).await.unwrap(); 53 | }) 54 | }) 55 | }); 56 | } 57 | 58 | criterion_group!( 59 | benches, 60 | write_hello_world, 61 | read_hello_world, 62 | read_write_hello_world, 63 | write_del_hello_world, 64 | ); 65 | criterion_main!(benches); 66 | -------------------------------------------------------------------------------- /fuzz/.gitignore: -------------------------------------------------------------------------------- 1 | 2 | target 3 | corpus 4 | artifacts 5 | -------------------------------------------------------------------------------- /fuzz/Cargo.toml: -------------------------------------------------------------------------------- 1 | 2 | [package] 3 | name = "random-access-memory-fuzz" 4 | version = "0.0.1" 5 | authors = ["Automatically generated"] 6 | publish = false 7 | edition = "2018" 8 | 9 | [package.metadata] 10 | cargo-fuzz = true 11 | 12 | [dependencies.random-access-memory] 13 | path = ".." 14 | [dependencies.libfuzzer-sys] 15 | git = "https://github.com/rust-fuzz/libfuzzer-sys.git" 16 | 17 | # Prevent this from interfering with workspaces 18 | [workspace] 19 | members = ["."] 20 | 21 | [[bin]] 22 | name = "fuzz_target_1" 23 | path = "fuzz_targets/fuzz_target_1.rs" 24 | -------------------------------------------------------------------------------- /fuzz/fuzz_targets/fuzz_target_1.rs: -------------------------------------------------------------------------------- 1 | #![no_main] 2 | 3 | use libfuzzer_sys::fuzz_target; 4 | use random_access_memory as ram; 5 | 6 | fuzz_target!(|data: &[u8]| { 7 | let mut file = ram::Sync::default(); 8 | file.write(0, data).unwrap(); 9 | }); 10 | -------------------------------------------------------------------------------- /rustfmt.toml: -------------------------------------------------------------------------------- 1 | max_width = 80 2 | tab_spaces = 2 3 | -------------------------------------------------------------------------------- /src/lib.rs: -------------------------------------------------------------------------------- 1 | #![forbid(unsafe_code, missing_docs)] 2 | #![cfg_attr(test, deny(warnings))] 3 | #![doc(test(attr(deny(warnings))))] 4 | //! # Continuously read and write to memory using random offsets and lengths 5 | //! [RandomAccessMemory] is a complete implementation of [random-access-storage](https://docs.rs/random-access-storage) 6 | //! for in-memory storage. 7 | //! 8 | //! See also [random-access-disk](https://docs.rs/random-access-disk) for on-disk storage 9 | //! that can be swapped with this. 10 | //! 11 | //! ## Examples 12 | //! 13 | //! Reading, writing, deleting and truncating: 14 | //! 15 | //! ``` 16 | //! # async_std::task::block_on(async { 17 | //! use random_access_storage::RandomAccess; 18 | //! use random_access_memory::RandomAccessMemory; 19 | //! 20 | //! let mut storage = RandomAccessMemory::default(); 21 | //! storage.write(0, b"hello").await.unwrap(); 22 | //! storage.write(5, b" world").await.unwrap(); 23 | //! assert_eq!(storage.read(0, 11).await.unwrap(), b"hello world"); 24 | //! assert_eq!(storage.len().await.unwrap(), 11); 25 | //! storage.del(5, 2).await.unwrap(); 26 | //! assert_eq!(storage.read(5, 2).await.unwrap(), [0, 0]); 27 | //! assert_eq!(storage.len().await.unwrap(), 11); 28 | //! storage.truncate(2).await.unwrap(); 29 | //! assert_eq!(storage.len().await.unwrap(), 2); 30 | //! storage.truncate(5).await.unwrap(); 31 | //! assert_eq!(storage.len().await.unwrap(), 5); 32 | //! assert_eq!(storage.read(0, 5).await.unwrap(), [b'h', b'e', 0, 0, 0]); 33 | //! # }) 34 | //! ``` 35 | //! 36 | //! In order to get benefits from the swappable interface, you will 37 | //! in most cases want to use generic functions for storage manipulation: 38 | //! 39 | //! ``` 40 | //! # async_std::task::block_on(async { 41 | //! use random_access_storage::RandomAccess; 42 | //! use random_access_memory::RandomAccessMemory; 43 | //! use std::fmt::Debug; 44 | //! 45 | //! let mut storage = RandomAccessMemory::default(); 46 | //! write_hello_world(&mut storage).await; 47 | //! assert_eq!(read_hello_world(&mut storage).await, b"hello world"); 48 | //! 49 | //! /// Write with swappable storage 50 | //! async fn write_hello_world(storage: &mut T) 51 | //! where T: RandomAccess + Debug + Send, 52 | //! { 53 | //! storage.write(0, b"hello").await.unwrap(); 54 | //! storage.write(5, b" world").await.unwrap(); 55 | //! } 56 | //! 57 | //! /// Read with swappable storage 58 | //! async fn read_hello_world(storage: &mut T) -> Vec 59 | //! where T: RandomAccess + Debug + Send, 60 | //! { 61 | //! storage.read(0, 11).await.unwrap() 62 | //! } 63 | //! # }) 64 | //! ``` 65 | 66 | pub use intmap::IntMap; 67 | 68 | use random_access_storage::{RandomAccess, RandomAccessError}; 69 | use std::cmp; 70 | 71 | /// In-memory storage for random access 72 | #[derive(Debug)] 73 | pub struct RandomAccessMemory { 74 | /// Length of each buffer 75 | page_size: usize, 76 | 77 | /// Allocated memory 78 | buffers: IntMap>, 79 | 80 | /// Total length of the data 81 | length: u64, 82 | } 83 | 84 | impl Default for RandomAccessMemory { 85 | /// Create a new instance with a 1mb page size. 86 | fn default() -> Self { 87 | RandomAccessMemory::new(1024 * 1024) 88 | } 89 | } 90 | 91 | #[allow(clippy::needless_range_loop)] 92 | impl RandomAccessMemory { 93 | /// Create a new instance with `page_size` in bytes. 94 | pub fn new(page_size: usize) -> Self { 95 | RandomAccessMemory::with_buffers(page_size, IntMap::new()) 96 | } 97 | 98 | /// Create a new instance with `page_size` in bytes, but pass the initial buffers to the constructor. 99 | pub fn with_buffers(page_size: usize, buffers: IntMap>) -> Self { 100 | RandomAccessMemory { 101 | page_size, 102 | buffers, 103 | length: 0, 104 | } 105 | } 106 | 107 | /// Returns the page number and index within that page for a given offset. 108 | /// If `exclusive_end` is true, when hitting the exact border of two pages 109 | /// gives the previous page and page size as index. 110 | fn page_num_and_index( 111 | &self, 112 | offset: u64, 113 | exclusive_end: bool, 114 | ) -> (usize, usize) { 115 | let page_num = (offset / (self.page_size as u64)) as usize; 116 | let page_index = (offset % (self.page_size as u64)) as usize; 117 | if page_index == 0 && exclusive_end { 118 | (if page_num > 0 { page_num - 1 } else { 0 }, self.page_size) 119 | } else { 120 | (page_num, page_index) 121 | } 122 | } 123 | 124 | /// Zero given range 125 | fn zero(&mut self, offset: u64, length: u64) { 126 | let (first_page_num, first_page_start) = 127 | self.page_num_and_index(offset, false); 128 | let (last_page_num, last_page_end) = 129 | self.page_num_and_index(offset + length, true); 130 | 131 | // Check if we need to zero bytes in the first page 132 | if first_page_start > 0 133 | || (first_page_num == last_page_num && last_page_end > 0) 134 | { 135 | if let Some(page) = self.buffers.get_mut(first_page_num as u64) { 136 | // Need to zero part of the first page 137 | let begin_page_end = first_page_start 138 | + cmp::min(length as usize, self.page_size - first_page_start); 139 | for index in first_page_start..begin_page_end { 140 | page[index] = 0; 141 | } 142 | } 143 | } 144 | 145 | // Delete intermediate pages 146 | if last_page_num > first_page_num + 1 147 | || (first_page_start == 0 && last_page_num == first_page_num + 1) 148 | { 149 | let first_page_to_drop = if first_page_start == 0 { 150 | first_page_num 151 | } else { 152 | first_page_num + 1 153 | }; 154 | 155 | for index in first_page_to_drop..last_page_num { 156 | self.buffers.remove(index as u64); 157 | } 158 | } 159 | 160 | // Finally zero the last page 161 | if last_page_num > first_page_num && last_page_end > 0 { 162 | if let Some(page) = self.buffers.get_mut(last_page_num as u64) { 163 | // Need to zero part of the final page 164 | for index in 0..last_page_end { 165 | page[index] = 0; 166 | } 167 | } 168 | } 169 | } 170 | } 171 | 172 | #[async_trait::async_trait] 173 | impl RandomAccess for RandomAccessMemory { 174 | async fn write( 175 | &mut self, 176 | offset: u64, 177 | data: &[u8], 178 | ) -> Result<(), RandomAccessError> { 179 | let new_len = offset + data.len() as u64; 180 | if new_len > self.length { 181 | self.length = new_len; 182 | } 183 | 184 | let mut page_num = (offset / self.page_size as u64) as usize; 185 | let mut page_cursor = 186 | (offset - (page_num * self.page_size) as u64) as usize; 187 | let mut data_cursor = 0; 188 | 189 | // Iterate over data, write to buffers. Subslice if the data is bigger than 190 | // what we can write in a single go. 191 | while data_cursor < data.len() { 192 | let data_bound = data.len() - data_cursor; 193 | let upper_bound = cmp::min(self.page_size, page_cursor + data_bound); 194 | let range = page_cursor..upper_bound; 195 | let range_len = (page_cursor..upper_bound).len(); 196 | 197 | // Allocate buffer if needed. Either append a new buffer to the end, or 198 | // set a buffer in the center. 199 | if self.buffers.get(page_num as u64).is_none() { 200 | let buf = vec![0; self.page_size]; 201 | self.buffers.insert(page_num as u64, buf); 202 | } 203 | 204 | // Copy data from the vec slice. 205 | // TODO: use a batch operation such as `.copy_from_slice()` so it can be 206 | // optimized. 207 | let buffer = &mut self.buffers.get_mut(page_num as u64).unwrap(); 208 | for (index, buf_index) in range.enumerate() { 209 | buffer[buf_index] = data[data_cursor + index]; 210 | } 211 | 212 | page_num += 1; 213 | page_cursor = 0; 214 | data_cursor += range_len; 215 | } 216 | 217 | Ok(()) 218 | } 219 | 220 | async fn sync_all(&mut self) -> Result<(), RandomAccessError> { 221 | Ok(()) 222 | } 223 | 224 | async fn read( 225 | &mut self, 226 | offset: u64, 227 | length: u64, 228 | ) -> Result, RandomAccessError> { 229 | if (offset + length) > self.length { 230 | return Err(RandomAccessError::OutOfBounds { 231 | offset, 232 | end: Some(offset + length), 233 | length: self.length, 234 | }); 235 | }; 236 | 237 | let mut page_num = (offset / self.page_size as u64) as usize; 238 | let mut page_cursor = 239 | (offset - (page_num * self.page_size) as u64) as usize; 240 | 241 | let mut res_buf = vec![0; length as usize]; 242 | let mut res_cursor = 0; // Keep track we read the right amount of bytes. 243 | let res_capacity = length; 244 | 245 | while res_cursor < res_capacity { 246 | let res_bound = res_capacity - res_cursor; 247 | let page_bound = self.page_size - page_cursor; 248 | let relative_bound = cmp::min(res_bound, page_bound as u64); 249 | let upper_bound = page_cursor + relative_bound as usize; 250 | let range = page_cursor..upper_bound; 251 | 252 | // Fill until either we're done reading the page, or we're done 253 | // filling the buffer. Whichever arrives sooner. 254 | match self.buffers.get(page_num as u64) { 255 | Some(buf) => { 256 | for (index, buf_index) in range.enumerate() { 257 | res_buf[res_cursor as usize + index] = buf[buf_index]; 258 | } 259 | } 260 | None => { 261 | for (index, _) in range.enumerate() { 262 | res_buf[res_cursor as usize + index] = 0; 263 | } 264 | } 265 | } 266 | 267 | res_cursor += relative_bound; 268 | page_num += 1; 269 | page_cursor = 0; 270 | } 271 | 272 | Ok(res_buf) 273 | } 274 | 275 | async fn del( 276 | &mut self, 277 | offset: u64, 278 | length: u64, 279 | ) -> Result<(), RandomAccessError> { 280 | if offset > self.length { 281 | return Err(RandomAccessError::OutOfBounds { 282 | offset, 283 | end: None, 284 | length: self.length, 285 | }); 286 | }; 287 | 288 | if length == 0 { 289 | // No-op 290 | return Ok(()); 291 | } 292 | 293 | // Delete is truncate if up to the current length or more is deleted 294 | if offset + length >= self.length { 295 | return self.truncate(offset).await; 296 | } 297 | 298 | // Deleting means zeroing 299 | self.zero(offset, length); 300 | Ok(()) 301 | } 302 | 303 | #[allow(clippy::comparison_chain)] 304 | async fn truncate(&mut self, length: u64) -> Result<(), RandomAccessError> { 305 | let (current_last_page_num, _) = self.page_num_and_index(self.length, true); 306 | 307 | if self.length < length { 308 | let truncate_page_num = (length / self.page_size as u64) as usize; 309 | // Remove all of the pages between the old length and this newer 310 | // length that might have been left behind. 311 | for index in current_last_page_num + 1..truncate_page_num + 1 { 312 | self.buffers.remove(index as u64); 313 | } 314 | } else if self.length > length { 315 | let delete_length = 316 | ((current_last_page_num + 1) * self.page_size) - length as usize; 317 | // Make sure to zero the remainder to not leave anything but 318 | // zeros lying around. 319 | self.zero(length, delete_length as u64); 320 | } 321 | 322 | // Set new length 323 | self.length = length; 324 | 325 | Ok(()) 326 | } 327 | 328 | async fn len(&mut self) -> Result { 329 | Ok(self.length) 330 | } 331 | 332 | async fn is_empty(&mut self) -> Result { 333 | Ok(self.length == 0) 334 | } 335 | } 336 | -------------------------------------------------------------------------------- /tests/model.rs: -------------------------------------------------------------------------------- 1 | use self::Op::*; 2 | use proptest::prelude::*; 3 | use proptest::test_runner::FileFailurePersistence; 4 | use proptest_derive::Arbitrary; 5 | use random_access_memory as ram; 6 | use random_access_storage::RandomAccess; 7 | use std::u8; 8 | 9 | const MAX_FILE_SIZE: u64 = 50000; 10 | 11 | #[derive(Clone, Debug, Arbitrary)] 12 | enum Op { 13 | Read { 14 | #[proptest(strategy(offset_length_strategy))] 15 | offset: u64, 16 | #[proptest(strategy(offset_length_strategy))] 17 | length: u64, 18 | }, 19 | Write { 20 | #[proptest(strategy(offset_length_strategy))] 21 | offset: u64, 22 | #[proptest(regex(data_regex))] 23 | data: Vec, 24 | }, 25 | Delete { 26 | #[proptest(strategy(offset_length_strategy))] 27 | offset: u64, 28 | #[proptest(strategy(offset_length_strategy))] 29 | length: u64, 30 | }, 31 | } 32 | 33 | fn offset_length_strategy() -> impl Strategy { 34 | 0..MAX_FILE_SIZE 35 | } 36 | 37 | fn data_regex() -> &'static str { 38 | // Write 0..5000 byte chunks of ASCII characters as dummy data 39 | "([ -~]{1,1}\n){0,5000}" 40 | } 41 | 42 | proptest! { 43 | #![proptest_config(ProptestConfig { 44 | failure_persistence: Some(Box::new(FileFailurePersistence::WithSource("regressions"))), 45 | ..Default::default() 46 | })] 47 | 48 | #[test] 49 | fn implementation_matches_model(ops: Vec) { 50 | assert!(async_std::task::block_on(async { 51 | let mut implementation = ram::RandomAccessMemory::new(10); 52 | let mut model = vec![]; 53 | 54 | for op in ops { 55 | match op { 56 | Read { offset, length } => { 57 | let end = offset + length; 58 | if model.len() >= end as usize { 59 | assert_eq!( 60 | &*implementation.read(offset, length).await.expect("Reads should be successful."), 61 | &model[offset as usize..end as usize] 62 | ); 63 | } else { 64 | assert!(implementation.read(offset, length).await.is_err()); 65 | } 66 | }, 67 | Write { offset, ref data } => { 68 | let end = offset + data.len() as u64; 69 | if model.len() < end as usize { 70 | model.resize(end as usize, 0); 71 | } 72 | implementation.write(offset, data).await.expect("Writes should be successful."); 73 | model[offset as usize..end as usize].copy_from_slice(data); 74 | }, 75 | Delete { offset, length } => { 76 | if model.len() >= offset as usize { 77 | implementation.del(offset, length).await.expect("Deletes should be successful."); 78 | if offset + length < model.len() as u64 { 79 | model[offset as usize..(offset + length) as usize].fill(0); 80 | } else { 81 | model.resize(offset as usize, 0); 82 | }; 83 | } else { 84 | assert!(implementation.del(offset, length).await.is_err()); 85 | } 86 | } 87 | } 88 | } 89 | true 90 | })); 91 | } 92 | } 93 | -------------------------------------------------------------------------------- /tests/regression.rs: -------------------------------------------------------------------------------- 1 | use random_access_memory as ram; 2 | use random_access_storage::RandomAccess; 3 | 4 | #[async_std::test] 5 | // Postmortem: looks like we were reading out of bounds by accidentally 6 | // adding an offset to the index while reading. 7 | async fn regress_1() { 8 | let mut file = ram::RandomAccessMemory::new(50); 9 | file.write(30, &[30]).await.unwrap(); 10 | file.read(15, 15).await.unwrap(); 11 | } 12 | 13 | #[async_std::test] 14 | // Postmortem: our buffers weren't zero-filled. Intead we were relying on 15 | // uninitialized (but claimed!) memory, which caused all sorts of weirdness. 16 | async fn regress_2() { 17 | let mut file = ram::RandomAccessMemory::new(50); 18 | file.write(22, &[22, 22, 22, 22]).await.unwrap(); 19 | let buf = file.read(1, 4).await.unwrap(); 20 | assert_eq!(buf, vec![0, 0, 0, 0]); 21 | 22 | let mut file = ram::RandomAccessMemory::new(50); 23 | file.write(48, &[48, 48, 48, 48]).await.unwrap(); 24 | let buf = file.read(39, 9).await.unwrap(); 25 | assert_eq!(buf, vec![0, 0, 0, 0, 0, 0, 0, 0, 0]); 26 | } 27 | 28 | #[async_std::test] 29 | // Postmortem: the way we were reading was off. We were messing up both reading 30 | // and writing. We now keep two cursors, and compute the bounds of every loop 31 | // ahead of time. Also simplified our allocation logic. 32 | async fn regress_3() { 33 | let mut file = ram::RandomAccessMemory::new(50); 34 | file.write(45, &[56, 46, 14, 93, 15, 54, 2]).await.unwrap(); 35 | let buf = file.read(42, 10).await.unwrap(); 36 | assert_eq!(buf, vec![0, 0, 0, 56, 46, 14, 93, 15, 54, 2]); 37 | } 38 | 39 | #[async_std::test] 40 | // Postmortem: we were having trouble when we were reading with an index that's 41 | // larger than the page size. Turned out we weren't doing some math properly, 42 | // which caused a cursor to jump. 43 | async fn regress_4() { 44 | let mut file = ram::RandomAccessMemory::new(10); 45 | file.write(44, &[54, 59]).await.unwrap(); 46 | file.read(13, 3).await.unwrap(); 47 | } 48 | -------------------------------------------------------------------------------- /tests/test.rs: -------------------------------------------------------------------------------- 1 | use random_access_memory as ram; 2 | use random_access_storage::RandomAccess; 3 | 4 | #[async_std::test] 5 | async fn can_call_new() { 6 | let _file = ram::RandomAccessMemory::default(); 7 | } 8 | 9 | #[async_std::test] 10 | async fn can_open_buffer() { 11 | let mut file = ram::RandomAccessMemory::default(); 12 | file.write(0, b"hello").await.unwrap(); 13 | } 14 | 15 | #[async_std::test] 16 | async fn can_write() { 17 | let mut file = ram::RandomAccessMemory::default(); 18 | file.write(0, b"hello").await.unwrap(); 19 | file.write(5, b" world").await.unwrap(); 20 | } 21 | 22 | #[async_std::test] 23 | async fn can_read() { 24 | let mut file = ram::RandomAccessMemory::default(); 25 | file.write(0, b"hello").await.unwrap(); 26 | file.write(5, b" world").await.unwrap(); 27 | let text = file.read(0, 11).await.unwrap(); 28 | let text = String::from_utf8(text.to_vec()).unwrap(); 29 | assert_eq!(text, "hello world"); 30 | } 31 | 32 | #[async_std::test] 33 | async fn can_len() { 34 | let mut file = ram::RandomAccessMemory::default(); 35 | assert_eq!(file.len().await.unwrap(), 0); 36 | file.write(0, b"hello").await.unwrap(); 37 | assert_eq!(file.len().await.unwrap(), 5); 38 | file.write(5, b" world").await.unwrap(); 39 | assert_eq!(file.len().await.unwrap(), 11); 40 | } 41 | 42 | #[async_std::test] 43 | async fn can_is_empty() { 44 | let mut file = ram::RandomAccessMemory::default(); 45 | assert!(file.is_empty().await.unwrap()); 46 | file.write(0, b"hello").await.unwrap(); 47 | assert!(!file.is_empty().await.unwrap()); 48 | } 49 | 50 | #[async_std::test] 51 | async fn can_delete() { 52 | assert_delete(100).await; 53 | assert_delete(10).await; 54 | assert_delete(5).await; 55 | assert_delete(2).await; 56 | assert_delete(1).await; 57 | } 58 | 59 | async fn assert_delete(page_size: usize) { 60 | let mut file = ram::RandomAccessMemory::new(page_size); 61 | file.write(0, b"hello").await.unwrap(); 62 | file.write(5, b" world").await.unwrap(); 63 | file.write(11, b" people").await.unwrap(); 64 | assert_eq!(file.len().await.unwrap(), 18); 65 | file.del(6, 2).await.unwrap(); 66 | assert_eq!(file.len().await.unwrap(), 18); 67 | let text = file.read(0, 6).await.unwrap(); 68 | assert_eq!(String::from_utf8(text.to_vec()).unwrap(), "hello "); 69 | let zeros = file.read(6, 2).await.unwrap(); 70 | assert_eq!(zeros, [0, 0]); 71 | let text = file.read(8, 10).await.unwrap(); 72 | assert_eq!(String::from_utf8(text.to_vec()).unwrap(), "rld people"); 73 | file.del(8, 4).await.unwrap(); 74 | assert_eq!(file.len().await.unwrap(), 18); 75 | file.del(10, 8).await.unwrap(); 76 | assert_eq!(file.len().await.unwrap(), 10); 77 | } 78 | 79 | #[async_std::test] 80 | async fn can_truncate_lt() { 81 | assert_truncate_lt(100).await; 82 | assert_truncate_lt(10).await; 83 | assert_truncate_lt(5).await; 84 | assert_truncate_lt(2).await; 85 | assert_truncate_lt(1).await; 86 | } 87 | 88 | async fn assert_truncate_lt(page_size: usize) { 89 | let mut file = ram::RandomAccessMemory::new(page_size); 90 | file.write(0, b"hello").await.unwrap(); 91 | file.write(5, b" world").await.unwrap(); 92 | file.write(11, b" people").await.unwrap(); 93 | assert_eq!(file.len().await.unwrap(), 18); 94 | file.truncate(7).await.unwrap(); 95 | assert_eq!(file.len().await.unwrap(), 7); 96 | let text = file.read(0, 7).await.unwrap(); 97 | assert_eq!(String::from_utf8(text.to_vec()).unwrap(), "hello w"); 98 | if file.read(0, 8).await.is_ok() { 99 | panic!("storage is too big. read past the end should have failed"); 100 | }; 101 | file.write(11, b" people").await.unwrap(); 102 | assert_eq!(file.len().await.unwrap(), 18); 103 | let zeros = file.read(7, 4).await.unwrap(); 104 | assert_eq!(zeros, [0, 0, 0, 0]); 105 | } 106 | 107 | #[async_std::test] 108 | async fn can_truncate_gt() { 109 | assert_truncate_gt(100).await; 110 | assert_truncate_gt(10).await; 111 | assert_truncate_gt(5).await; 112 | assert_truncate_gt(2).await; 113 | assert_truncate_gt(1).await; 114 | } 115 | 116 | async fn assert_truncate_gt(page_size: usize) { 117 | let mut file = ram::RandomAccessMemory::new(page_size); 118 | file.write(0, b"hello").await.unwrap(); 119 | file.write(5, b" world").await.unwrap(); 120 | file.write(11, b" people").await.unwrap(); 121 | assert_eq!(file.len().await.unwrap(), 18); 122 | file.truncate(22).await.unwrap(); 123 | assert_eq!(file.len().await.unwrap(), 22); 124 | let zeros = file.read(18, 4).await.unwrap(); 125 | assert_eq!(zeros, [0, 0, 0, 0]); 126 | file.write(19, &[1]).await.unwrap(); 127 | let written = file.read(18, 4).await.unwrap(); 128 | assert_eq!(written, [0, 1, 0, 0]); 129 | } 130 | 131 | #[async_std::test] 132 | async fn assert_truncate_eq() { 133 | let mut file = ram::RandomAccessMemory::new(5); 134 | file.write(0, b"hello").await.unwrap(); 135 | file.write(5, b" world").await.unwrap(); 136 | file.write(11, b" people").await.unwrap(); 137 | assert_eq!(file.len().await.unwrap(), 18); 138 | file.truncate(18).await.unwrap(); 139 | assert_eq!(file.len().await.unwrap(), 18); 140 | } 141 | --------------------------------------------------------------------------------