├── .cargo └── config ├── .github └── pull_request_template.md ├── .gitignore ├── .gitmodules ├── CHANGELOG.md ├── CONTRIBUTING.md ├── Cargo.toml ├── LICENSE ├── README.md ├── SECURITY-POLICY.md ├── coverage_config_aarch64.json ├── coverage_config_x86_64.json ├── src ├── crc.rs ├── lib.rs ├── primitives.rs └── version_map.rs └── tests └── test.rs /.cargo/config: -------------------------------------------------------------------------------- 1 | # This workaround is needed because the linker is unable to find __addtf3, 2 | # __multf3 and __subtf3. 3 | # Related issue: https://github.com/rust-lang/compiler-builtins/issues/201 4 | [target.aarch64-unknown-linux-musl] 5 | rustflags = [ "-C", "target-feature=+crt-static", "-C", "link-arg=-lgcc"] 6 | 7 | -------------------------------------------------------------------------------- /.github/pull_request_template.md: -------------------------------------------------------------------------------- 1 | ## Reason for This PR 2 | 3 | `[Author TODO: add issue # or explain reasoning.]` 4 | 5 | ## Description of Changes 6 | 7 | `[Author TODO: add description of changes.]` 8 | 9 | ## License Acceptance 10 | 11 | By submitting this pull request, I confirm that my contribution is made under 12 | the terms of the Apache 2.0 license. 13 | 14 | ## PR Checklist 15 | 16 | `[Author TODO: Meet these criteria.]` 17 | `[Reviewer TODO: Verify that these criteria are met. Request changes if not]` 18 | 19 | - [ ] All commits in this PR are signed (`git commit -s`). 20 | - [ ] The reason for this PR is clearly provided (issue no. or explanation). 21 | - [ ] The description of changes is clear and encompassing. 22 | - [ ] Any required documentation changes (code and docs) are included in this PR. 23 | - [ ] Any newly added `unsafe` code is properly documented. 24 | - [ ] Any user-facing changes are mentioned in `CHANGELOG.md`. 25 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Generated by Cargo 2 | # will have compiled files and executables 3 | /target/ 4 | 5 | # Remove Cargo.lock from gitignore if creating an executable, leave it for libraries 6 | # More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html 7 | Cargo.lock 8 | 9 | # These are backup files generated by rustfmt 10 | **/*.rs.bk 11 | -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "rust-vmm-ci"] 2 | path = rust-vmm-ci 3 | url = https://github.com/rust-vmm/rust-vmm-ci.git 4 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # v0.2.0 2 | 3 | - Implement `Versionize` for arrays of arbitrary length, instead of just up to 32. 4 | 5 | # v0.1.10 6 | 7 | - Fixed a possible out of bounds memory access in FamStructWrapper::deserialize 8 | 9 | # v0.1.9 10 | 11 | - Implement Versionize for i128 and u128 12 | 13 | # v0.1.8 14 | 15 | - Fixed VersionMap not implementing Sync + Send in 0.1.7 16 | 17 | # v0.1.7 [yanked] 18 | 19 | - Use caret requirements instead of comparison requirements 20 | for specifying dependencies 21 | - Update vmm-sys-utils to 0.11.0 22 | 23 | # v0.1.6 24 | 25 | - Upgraded vmm-sys-utils to v0.8.0 26 | 27 | # v0.1.5 28 | 29 | - Added more documentation and examples. 30 | 31 | # v0.1.4 32 | 33 | - Removed Versionize proc macro support for unions. Serializing unions can lead to undefined behaviour especially when no 34 | layout guarantees are provided. The Versionize trait can still be implemented but only for repr(C) unions and extensive care 35 | and testing is required from the implementer. 36 | 37 | # v0.1.3 38 | 39 | - Added extra validations in VersionMap::get_type_version(). 40 | 41 | # v0.1.2 42 | 43 | - Improved edge cases handling for Vec serialization and deserialization. 44 | 45 | # v0.1.0 46 | 47 | - "versionize" v0.1.0 first release. 48 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to versionize 2 | 3 | ## Contribution Workflow 4 | 5 | The versionize repository uses the “fork-and-pull” development model. Follow 6 | these steps if you want to merge your changes: 7 | 8 | 1. Within your fork of 9 | [versionize](https://github.com/firecracker-microvm/versionize), create a 10 | branch for your contribution. Use a meaningful name. 11 | 1. Create your contribution, meeting all 12 | [contribution quality standards](#contribution-quality-standards) 13 | 1. [Create a pull request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/) 14 | against the master branch of the versionize repository. 15 | 1. Work with your reviewers to address any comments and obtain a 16 | minimum of 2 approvals, at least one of which must be provided by 17 | [a maintainer](MAINTAINERS.md). 18 | To update your pull request amend existing commits whenever applicable and 19 | then push the new changes to your pull request branch. 20 | 1. Once the pull request is approved, one of the maintainers will merge it. 21 | 22 | ## Request for Comments 23 | 24 | If you just want to receive feedback for a contribution proposal, open an “RFC” 25 | (“Request for Comments”) pull request: 26 | 27 | 1. On your fork of 28 | [versionize](https://github.com/firecracker-microvm/versionize), create a 29 | branch for the contribution you want feedback on. Use a meaningful name. 30 | 1. Create your proposal based on the existing codebase. 31 | 1. [Create a draft pull request](https://github.blog/2019-02-14-introducing-draft-pull-requests/) 32 | against the master branch of the versionize repository. 33 | 1. Discuss your proposal with the community on the pull request page (or on any 34 | other channel). Add the conclusion(s) of this discussion to the pull request 35 | page. 36 | 37 | ## Contribution Quality Standards 38 | 39 | Most quality and style standards are enforced automatically during integration 40 | testing. Your contribution needs to meet the following standards: 41 | 42 | - Separate each **logical change** into its own commit. 43 | - Each commit must pass all unit & code style tests, and the full pull request 44 | must pass all integration tests. 45 | - Unit test coverage must _increase_ the overall project code coverage. 46 | - Document all your public functions. 47 | - Add a descriptive message for each commit. Follow 48 | [commit message best practices](https://github.com/erlang/otp/wiki/writing-good-commit-messages). 49 | - Document your pull requests. Include the reasoning behind each change. 50 | - Acknowledge versionize's [Apache 2.0 license](LICENSE) and certify that no 51 | part of your contribution contravenes this license by signing off on all your 52 | commits with `git -s`. Ensure that every file in your pull request has a 53 | header referring to the repository license file. 54 | -------------------------------------------------------------------------------- /Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | name = "versionize" 3 | version = "0.2.0" 4 | license = "Apache-2.0" 5 | authors = ["Amazon Firecracker team "] 6 | description = "A version tolerant serialization/deserialization framework." 7 | readme = "README.md" 8 | repository = "https://github.com/firecracker-microvm/versionize" 9 | keywords = ["serialization", "version"] 10 | categories = ["encoding"] 11 | 12 | [dependencies] 13 | serde = "1.0.27" 14 | serde_derive = "1.0.27" 15 | bincode = "1.2.1" 16 | versionize_derive = "0.1.6" 17 | crc64 = "2.0.0" 18 | vmm-sys-util = "0.12.1" 19 | 20 | [build-dependencies] 21 | proc-macro2 = "1.0" 22 | quote = "1.0" 23 | syn = { version = "1.0.13", features=["default","full"]} 24 | 25 | [badges] 26 | maintenance = { status = "experimental" } 27 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | **Versionize is a framework for version tolerant serializion/deserialization of 2 | Rust data structures, designed for usecases that need fast deserialization 3 | times and minimal size overhead. It does not aim to be a generic serialization 4 | framework and only the [bincode](https://crates.io/crates/bincode) backend is 5 | supported.** 6 | 7 | ## Important note 8 | 9 | This crate is currently used for cross-version serialization with the 10 | [Firecracker snapshot-restore dev preview][1], but has not been tested for 11 | other use cases. It should be considered **experimental software** outside the 12 | Firecracker context. It’s likely that this crate will see both interface and 13 | implementation changes in the future. 14 | 15 | ## Versionize in action 16 | 17 | ```rust 18 | extern crate versionize; 19 | extern crate versionize_derive; 20 | 21 | use versionize::{VersionMap, Versionize, VersionizeResult}; 22 | use versionize_derive::Versionize; 23 | 24 | // The test structure is at version 3. 25 | #[derive(Debug, PartialEq, Versionize)] 26 | pub struct Test { 27 | a: u32, 28 | #[version(start = 2, end = 3)] 29 | b: u8, 30 | #[version(start = 3, default_fn = "default_c")] 31 | c: String, 32 | } 33 | 34 | impl Test { 35 | // Default value for field `c`. 36 | // The callback is invoked when deserializing an older version 37 | // where the field did not exist. 38 | fn default_c(_source_version: u16) -> String { 39 | "test_string".to_owned() 40 | } 41 | } 42 | 43 | // Memory to hold the serialization output. 44 | let mut mem = vec![0u8; 512]; 45 | // Create a new version map - it will start at app version 1. 46 | let mut version_map = VersionMap::new(); 47 | // Map structure versions to app version. 48 | version_map 49 | .new_version() // App version 2. 50 | .set_type_version(Test::type_id(), 2) // Struct(2) -> App(2). 51 | .new_version() // App version 3. 52 | .set_type_version(Test::type_id(), 3); // Struct(3) -> App(3). 53 | 54 | let test_struct = Test { 55 | a: 1337, 56 | b: 0xFF, 57 | c: "c_value".to_owned(), 58 | }; 59 | 60 | // Serialize to app version 2 - field c will not be serialized. 61 | test_struct 62 | .serialize(&mut mem.as_mut_slice(), &version_map, 2) 63 | .unwrap(); 64 | 65 | // Deserialize from app version 2 - c should contain the default_fn() return value. 66 | let restored_test_struct = Test::deserialize(&mut mem.as_slice(), &version_map, 2).unwrap(); 67 | 68 | assert_eq!( 69 | restored_test_struct, 70 | Test { 71 | a: 1337, 72 | b: 255, 73 | c: "test_string".to_owned() 74 | } 75 | ); 76 | ``` 77 | 78 | [1]: https://github.com/firecracker-microvm/firecracker/tree/v0.24.0 -------------------------------------------------------------------------------- /SECURITY-POLICY.md: -------------------------------------------------------------------------------- 1 | # Security Issue Policy 2 | 3 | If you uncover a security issue with versionize, please write to us on 4 | . 5 | 6 | Once the Firecracker [maintainers](MAINTAINERS.md) become aware (or are made 7 | aware) of a security issue, they will immediately assess it. Based on impact 8 | and complexity, they will determine an embargo period (if externally reported, 9 | the period will be agreed upon with the external party). 10 | 11 | During the embargo period, maintainers will prioritize developing a fix over 12 | other activities. Within this period, maintainers may also notify a limited 13 | number of trusted parties via a pre-disclosure list, providing them with 14 | technical information, a risk assessment, and early access to a fix. 15 | 16 | The external customers are included in this group based on the scale of their 17 | versionize usage in production. The pre-disclosure list may also contain 18 | significant external security contributors that can join the effort to fix the 19 | issue during the embargo period. 20 | 21 | At the end of the embargo period, maintainers will publicly release information 22 | about the security issue together with the versionize patches that mitigate it. 23 | -------------------------------------------------------------------------------- /coverage_config_aarch64.json: -------------------------------------------------------------------------------- 1 | {"coverage_score": 92.4, "exclude_path": "test", "crate_features": ""} 2 | -------------------------------------------------------------------------------- /coverage_config_x86_64.json: -------------------------------------------------------------------------------- 1 | {"coverage_score": 92.4, "exclude_path": "test", "crate_features": ""} 2 | -------------------------------------------------------------------------------- /src/crc.rs: -------------------------------------------------------------------------------- 1 | // Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: Apache-2.0 3 | 4 | //! Implements readers and writers that compute the CRC64 checksum of the bytes 5 | //! read/written. 6 | 7 | use crc64::crc64; 8 | use std::fmt; 9 | use std::io::{Read, Write}; 10 | 11 | /// Computes the CRC64 checksum of the read bytes. 12 | /// 13 | /// ``` 14 | /// use std::io::Read; 15 | /// use versionize::crc::CRC64Reader; 16 | /// 17 | /// let buf = vec![1, 2, 3, 4, 5]; 18 | /// let mut read_buf = Vec::new(); 19 | /// let mut slice = buf.as_slice(); 20 | /// 21 | /// // Create a reader from a slice. 22 | /// let mut crc_reader = CRC64Reader::new(&mut slice); 23 | /// 24 | /// let count = crc_reader.read_to_end(&mut read_buf).unwrap(); 25 | /// assert_eq!(crc_reader.checksum(), 0xFB04_60DE_0638_3654); 26 | /// assert_eq!(read_buf, buf); 27 | /// ``` 28 | pub struct CRC64Reader { 29 | reader: T, 30 | crc64: u64, 31 | } 32 | 33 | impl fmt::Debug for CRC64Reader { 34 | fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { 35 | write!(f, "CRC64Reader {{ .. }}") 36 | } 37 | } 38 | 39 | impl CRC64Reader 40 | where 41 | T: Read, 42 | { 43 | /// Create a new reader. 44 | pub fn new(reader: T) -> Self { 45 | CRC64Reader { crc64: 0, reader } 46 | } 47 | /// Returns the current checksum value. 48 | pub fn checksum(&self) -> u64 { 49 | self.crc64 50 | } 51 | } 52 | 53 | impl Read for CRC64Reader 54 | where 55 | T: Read, 56 | { 57 | fn read(&mut self, buf: &mut [u8]) -> std::io::Result { 58 | let bytes_read = self.reader.read(buf)?; 59 | self.crc64 = crc64(self.crc64, &buf[..bytes_read]); 60 | Ok(bytes_read) 61 | } 62 | } 63 | 64 | /// Computes the CRC64 checksum of the written bytes. 65 | /// 66 | /// ``` 67 | /// use std::io::Write; 68 | /// use versionize::crc::CRC64Writer; 69 | /// 70 | /// let mut buf = vec![0; 16]; 71 | /// let write_buf = vec![123; 16]; 72 | /// let mut slice = buf.as_mut_slice(); 73 | /// 74 | /// // Create a new writer from slice. 75 | /// let mut crc_writer = CRC64Writer::new(&mut slice); 76 | /// 77 | /// crc_writer.write_all(&write_buf.as_slice()).unwrap(); 78 | /// assert_eq!(crc_writer.checksum(), 0x29D5_3572_1632_6566); 79 | /// assert_eq!(write_buf, buf); 80 | /// ``` 81 | pub struct CRC64Writer { 82 | writer: T, 83 | crc64: u64, 84 | } 85 | 86 | impl fmt::Debug for CRC64Writer { 87 | fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { 88 | write!(f, "CRC64Writer {{ .. }}") 89 | } 90 | } 91 | 92 | impl CRC64Writer 93 | where 94 | T: Write, 95 | { 96 | /// Create a new writer. 97 | pub fn new(writer: T) -> Self { 98 | CRC64Writer { crc64: 0, writer } 99 | } 100 | 101 | /// Returns the current checksum value. 102 | pub fn checksum(&self) -> u64 { 103 | self.crc64 104 | } 105 | } 106 | 107 | impl Write for CRC64Writer 108 | where 109 | T: Write, 110 | { 111 | fn write(&mut self, buf: &[u8]) -> std::io::Result { 112 | let bytes_written = self.writer.write(buf)?; 113 | self.crc64 = crc64(self.crc64, &buf[..bytes_written]); 114 | Ok(bytes_written) 115 | } 116 | 117 | fn flush(&mut self) -> std::io::Result<()> { 118 | self.writer.flush() 119 | } 120 | } 121 | 122 | #[cfg(test)] 123 | mod tests { 124 | use super::{CRC64Reader, CRC64Writer, Read, Write}; 125 | 126 | #[test] 127 | fn test_crc_new() { 128 | let buf = vec![1; 5]; 129 | let mut slice = buf.as_slice(); 130 | let crc_reader = CRC64Reader::new(&mut slice); 131 | assert_eq!(crc_reader.crc64, 0); 132 | assert_eq!(crc_reader.reader, &[1; 5]); 133 | assert_eq!(crc_reader.checksum(), 0); 134 | 135 | let mut buf = vec![0; 5]; 136 | let mut slice = buf.as_mut_slice(); 137 | let crc_writer = CRC64Writer::new(&mut slice); 138 | assert_eq!(crc_writer.crc64, 0); 139 | assert_eq!(crc_writer.writer, &[0; 5]); 140 | assert_eq!(crc_writer.checksum(), 0); 141 | } 142 | 143 | #[test] 144 | fn test_crc_read() { 145 | let buf = vec![1, 2, 3, 4, 5]; 146 | let mut read_buf = vec![0; 16]; 147 | 148 | let mut slice = buf.as_slice(); 149 | let mut crc_reader = CRC64Reader::new(&mut slice); 150 | crc_reader.read_to_end(&mut read_buf).unwrap(); 151 | assert_eq!(crc_reader.checksum(), 0xFB04_60DE_0638_3654); 152 | assert_eq!(crc_reader.checksum(), crc_reader.crc64); 153 | } 154 | 155 | #[test] 156 | fn test_crc_write() { 157 | let mut buf = vec![0; 16]; 158 | let write_buf = vec![123; 16]; 159 | 160 | let mut slice = buf.as_mut_slice(); 161 | let mut crc_writer = CRC64Writer::new(&mut slice); 162 | crc_writer.write_all(write_buf.as_slice()).unwrap(); 163 | crc_writer.flush().unwrap(); 164 | assert_eq!(crc_writer.checksum(), 0x29D5_3572_1632_6566); 165 | assert_eq!(crc_writer.checksum(), crc_writer.crc64); 166 | } 167 | } 168 | -------------------------------------------------------------------------------- /src/lib.rs: -------------------------------------------------------------------------------- 1 | // Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: Apache-2.0 3 | #![deny(missing_docs, missing_debug_implementations)] 4 | 5 | //! Defines a generic interface for version tolerant serialization and 6 | //! implements it for primitive data types using `bincode` as backend. 7 | //! 8 | //! The interface has two components: 9 | //! - `Versionize` trait 10 | //! - `VersionMap` helper 11 | //! 12 | //! `VersionMap` maps individual structure/enum versions to a root version 13 | //! (app version). This mapping is required both when serializing or 14 | //! deserializing structures as it needs to know which version of structure 15 | //! to serialize for a given target app version. 16 | //! 17 | //! `Versionize` trait is implemented for the following primitives: 18 | //! u8, u16, u32, u64, usize, i8, i16, i32, i64, isize, char, f32, f64, 19 | //! String, Vec, Arrays up to 32 elements, Box, Wrapping, Option, 20 | //! FamStructWrapper, and (T, U). 21 | //! 22 | //! Known issues and limitations: 23 | //! - Union serialization is not supported via the `Versionize` proc macro. 24 | //! - Implementing `Versionize` for non-repr(C) unions can result in undefined 25 | //! behaviour and MUST be avoided. 26 | //! - Versionize trait implementations for repr(C) unions must be backed by 27 | //! extensive testing. 28 | //! - Semantic serialization and deserialization is available only for 29 | //! structures. 30 | extern crate bincode; 31 | extern crate crc64; 32 | extern crate serde; 33 | extern crate serde_derive; 34 | extern crate versionize_derive; 35 | extern crate vmm_sys_util; 36 | 37 | pub mod crc; 38 | pub mod primitives; 39 | pub mod version_map; 40 | 41 | use std::any::TypeId; 42 | use std::io::{Read, Write}; 43 | pub use version_map::VersionMap; 44 | use versionize_derive::Versionize; 45 | 46 | /// Versioned serialization/deserialization error definitions. 47 | #[allow(clippy::derive_partial_eq_without_eq)] // FIXME: next major release 48 | #[derive(Debug, PartialEq)] 49 | pub enum VersionizeError { 50 | /// An IO error occured. 51 | Io(i32), 52 | /// Generic serialization error. 53 | Serialize(String), 54 | /// Generic deserialization error. 55 | Deserialize(String), 56 | /// Semantic translation/validation error. 57 | Semantic(String), 58 | /// String length exceeded. 59 | StringLength(usize), 60 | /// Vector length exceeded. 61 | VecLength(usize), 62 | } 63 | 64 | impl std::fmt::Display for VersionizeError { 65 | fn fmt(&self, f: &mut std::fmt::Formatter) -> std::result::Result<(), std::fmt::Error> { 66 | use VersionizeError::*; 67 | 68 | match self { 69 | Io(e) => write!(f, "An IO error occured: {}", e), 70 | Serialize(e) => write!(f, "A serialization error occured: {}", e), 71 | Deserialize(e) => write!(f, "A deserialization error occured: {}", e), 72 | Semantic(e) => write!(f, "A user generated semantic error occured: {}", e), 73 | StringLength(bad_len) => write!( 74 | f, 75 | "String length exceeded {} > {} bytes", 76 | bad_len, 77 | primitives::MAX_STRING_LEN 78 | ), 79 | VecLength(bad_len) => write!( 80 | f, 81 | "Vec of length {} exceeded maximum size of {} bytes", 82 | bad_len, 83 | primitives::MAX_VEC_SIZE 84 | ), 85 | } 86 | } 87 | } 88 | 89 | /// Versioned serialization/deserialization result. 90 | pub type VersionizeResult = std::result::Result; 91 | 92 | /// Trait that provides an interface for version aware serialization and 93 | /// deserialization. 94 | /// The [Versionize proc macro][1] can generate an implementation for a given 95 | /// type if generics are not used, otherwise a manual implementation is 96 | /// required. 97 | /// 98 | /// Example implementation 99 | /// ``` 100 | /// extern crate versionize; 101 | /// extern crate versionize_derive; 102 | /// use versionize::{VersionMap, Versionize, VersionizeResult}; 103 | /// use versionize_derive::Versionize; 104 | /// 105 | /// struct MyType(T); 106 | /// 107 | /// impl Versionize for MyType 108 | /// where 109 | /// T: Versionize, 110 | /// { 111 | /// #[inline] 112 | /// fn serialize( 113 | /// &self, 114 | /// writer: &mut W, 115 | /// version_map: &VersionMap, 116 | /// app_version: u16, 117 | /// ) -> VersionizeResult<()> { 118 | /// self.0.serialize(writer, version_map, app_version) 119 | /// } 120 | /// 121 | /// #[inline] 122 | /// fn deserialize( 123 | /// reader: &mut R, 124 | /// version_map: &VersionMap, 125 | /// app_version: u16, 126 | /// ) -> VersionizeResult { 127 | /// Ok(MyType(T::deserialize(reader, version_map, app_version)?)) 128 | /// } 129 | /// 130 | /// fn version() -> u16 { 131 | /// 1 132 | /// } 133 | /// } 134 | /// ``` 135 | /// [1]: https://docs.rs/versionize_derive/latest/versionize_derive/derive.Versionize.html 136 | pub trait Versionize { 137 | /// Serializes `self` to `target_verion` using the specficifed `writer` and 138 | /// `version_map`. 139 | fn serialize( 140 | &self, 141 | writer: &mut W, 142 | version_map: &VersionMap, 143 | target_version: u16, 144 | ) -> VersionizeResult<()>; 145 | 146 | /// Returns a new instance of `Self` by deserializing from `source_version` 147 | /// using the specficifed `reader` and `version_map`. 148 | fn deserialize( 149 | reader: &mut R, 150 | version_map: &VersionMap, 151 | source_version: u16, 152 | ) -> VersionizeResult 153 | where 154 | Self: Sized; 155 | 156 | /// Returns the `Self` type id. 157 | /// The returned ID represents a globally unique identifier for a type. 158 | /// It is required by the `VersionMap` implementation. 159 | fn type_id() -> std::any::TypeId 160 | where 161 | Self: 'static, 162 | { 163 | TypeId::of::() 164 | } 165 | 166 | /// Returns latest `Self` version number. 167 | fn version() -> u16; 168 | } 169 | 170 | #[cfg(test)] 171 | mod tests { 172 | #[test] 173 | fn test_error_debug_display() { 174 | // Validates Debug and Display are implemented. 175 | use VersionizeError::*; 176 | let str = String::from("test"); 177 | format!("{:?}{}", Io(0), Io(0)); 178 | format!("{:?}{}", Serialize(str.clone()), Serialize(str.clone())); 179 | format!("{:?}{}", Deserialize(str.clone()), Deserialize(str.clone())); 180 | format!("{:?}{}", Semantic(str.clone()), Semantic(str)); 181 | } 182 | } 183 | -------------------------------------------------------------------------------- /src/primitives.rs: -------------------------------------------------------------------------------- 1 | // Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: Apache-2.0 3 | //! Serialization support for primitive data types. 4 | #![allow(clippy::float_cmp)] 5 | 6 | use self::super::{VersionMap, Versionize, VersionizeError, VersionizeResult}; 7 | use vmm_sys_util::fam::{FamStruct, FamStructWrapper}; 8 | 9 | /// Maximum allowed string len in bytes (16KB). 10 | /// Calling `serialize()` or `deserialiaze()` will fail beyond this limit. 11 | pub const MAX_STRING_LEN: usize = 16384; 12 | /// Maximum allowed vec size in bytes (10MB). 13 | /// Calling `serialize()` or `deserialiaze()` will fail beyond this limit. 14 | pub const MAX_VEC_SIZE: usize = 10_485_760; 15 | 16 | /// A macro that implements the Versionize trait for primitive types using the 17 | /// serde bincode backed. 18 | macro_rules! impl_versionize { 19 | ($ty:ident) => { 20 | impl Versionize for $ty { 21 | #[inline] 22 | fn serialize( 23 | &self, 24 | writer: &mut W, 25 | _version_map: &VersionMap, 26 | _version: u16, 27 | ) -> VersionizeResult<()> { 28 | bincode::serialize_into(writer, &self) 29 | .map_err(|ref err| VersionizeError::Serialize(format!("{:?}", err)))?; 30 | Ok(()) 31 | } 32 | 33 | #[inline] 34 | fn deserialize( 35 | mut reader: &mut R, 36 | _version_map: &VersionMap, 37 | _version: u16, 38 | ) -> VersionizeResult 39 | where 40 | Self: Sized, 41 | { 42 | bincode::deserialize_from(&mut reader) 43 | .map_err(|ref err| VersionizeError::Deserialize(format!("{:?}", err))) 44 | } 45 | 46 | // Not used. 47 | fn version() -> u16 { 48 | 1 49 | } 50 | } 51 | }; 52 | } 53 | 54 | impl_versionize!(bool); 55 | impl_versionize!(isize); 56 | impl_versionize!(i8); 57 | impl_versionize!(i16); 58 | impl_versionize!(i32); 59 | impl_versionize!(i64); 60 | impl_versionize!(i128); 61 | impl_versionize!(usize); 62 | impl_versionize!(u8); 63 | impl_versionize!(u16); 64 | impl_versionize!(u32); 65 | impl_versionize!(u64); 66 | impl_versionize!(u128); 67 | impl_versionize!(f32); 68 | impl_versionize!(f64); 69 | impl_versionize!(char); 70 | 71 | impl Versionize for String { 72 | #[inline] 73 | fn serialize( 74 | &self, 75 | writer: &mut W, 76 | version_map: &VersionMap, 77 | app_version: u16, 78 | ) -> VersionizeResult<()> { 79 | // It is better to fail early at serialization time. 80 | if self.len() > MAX_STRING_LEN { 81 | return Err(VersionizeError::StringLength(self.len())); 82 | } 83 | 84 | self.len().serialize(writer, version_map, app_version)?; 85 | writer 86 | .write_all(self.as_bytes()) 87 | .map_err(|e| VersionizeError::Io(e.raw_os_error().unwrap_or(0)))?; 88 | Ok(()) 89 | } 90 | 91 | #[inline] 92 | fn deserialize( 93 | mut reader: &mut R, 94 | version_map: &VersionMap, 95 | app_version: u16, 96 | ) -> VersionizeResult { 97 | let len = usize::deserialize(&mut reader, version_map, app_version)?; 98 | // Even if we fail in serialize, we still need to enforce this on the hot path 99 | // in case the len is corrupted. 100 | if len > MAX_STRING_LEN { 101 | return Err(VersionizeError::StringLength(len)); 102 | } 103 | 104 | let mut v = vec![0u8; len]; 105 | reader 106 | .read_exact(v.as_mut_slice()) 107 | .map_err(|e| VersionizeError::Io(e.raw_os_error().unwrap_or(0)))?; 108 | String::from_utf8(v) 109 | .map_err(|err| VersionizeError::Deserialize(format!("Utf8 error: {:?}", err))) 110 | } 111 | 112 | // Not used yet. 113 | fn version() -> u16 { 114 | 1 115 | } 116 | } 117 | 118 | impl Versionize for [T; N] 119 | where 120 | T: Copy + Default + Versionize, 121 | { 122 | #[inline] 123 | fn serialize( 124 | &self, 125 | writer: &mut W, 126 | version_map: &VersionMap, 127 | app_version: u16, 128 | ) -> VersionizeResult<()> { 129 | for element in self { 130 | element.serialize(writer, version_map, app_version)?; 131 | } 132 | 133 | Ok(()) 134 | } 135 | 136 | #[inline] 137 | fn deserialize( 138 | reader: &mut R, 139 | version_map: &VersionMap, 140 | app_version: u16, 141 | ) -> VersionizeResult { 142 | let mut array = [T::default(); N]; 143 | for elem in &mut array { 144 | *elem = T::deserialize(reader, version_map, app_version)?; 145 | } 146 | Ok(array) 147 | } 148 | 149 | // Not used yet. 150 | fn version() -> u16 { 151 | 1 152 | } 153 | } 154 | 155 | impl Versionize for Box 156 | where 157 | T: Versionize, 158 | { 159 | #[inline] 160 | fn serialize( 161 | &self, 162 | writer: &mut W, 163 | version_map: &VersionMap, 164 | app_version: u16, 165 | ) -> VersionizeResult<()> { 166 | self.as_ref().serialize(writer, version_map, app_version) 167 | } 168 | 169 | #[inline] 170 | fn deserialize( 171 | reader: &mut R, 172 | version_map: &VersionMap, 173 | app_version: u16, 174 | ) -> VersionizeResult { 175 | Ok(Box::new(T::deserialize(reader, version_map, app_version)?)) 176 | } 177 | 178 | // Not used yet. 179 | fn version() -> u16 { 180 | 1 181 | } 182 | } 183 | 184 | impl Versionize for std::num::Wrapping 185 | where 186 | T: Versionize, 187 | { 188 | #[inline] 189 | fn serialize( 190 | &self, 191 | writer: &mut W, 192 | version_map: &VersionMap, 193 | app_version: u16, 194 | ) -> VersionizeResult<()> { 195 | self.0.serialize(writer, version_map, app_version) 196 | } 197 | 198 | #[inline] 199 | fn deserialize( 200 | reader: &mut R, 201 | version_map: &VersionMap, 202 | app_version: u16, 203 | ) -> VersionizeResult { 204 | Ok(std::num::Wrapping(T::deserialize( 205 | reader, 206 | version_map, 207 | app_version, 208 | )?)) 209 | } 210 | 211 | // Not used yet. 212 | fn version() -> u16 { 213 | 1 214 | } 215 | } 216 | 217 | impl Versionize for Option 218 | where 219 | T: Versionize, 220 | { 221 | #[inline] 222 | fn serialize( 223 | &self, 224 | writer: &mut W, 225 | version_map: &VersionMap, 226 | app_version: u16, 227 | ) -> VersionizeResult<()> { 228 | // Serialize an Option just like bincode does: u8, T. 229 | match self { 230 | Some(value) => { 231 | 1u8.serialize(writer, version_map, app_version)?; 232 | value.serialize(writer, version_map, app_version) 233 | } 234 | None => 0u8.serialize(writer, version_map, app_version), 235 | } 236 | } 237 | 238 | #[inline] 239 | fn deserialize( 240 | reader: &mut R, 241 | version_map: &VersionMap, 242 | app_version: u16, 243 | ) -> VersionizeResult { 244 | let option = u8::deserialize(reader, version_map, app_version)?; 245 | match option { 246 | 0u8 => Ok(None), 247 | 1u8 => Ok(Some(T::deserialize(reader, version_map, app_version)?)), 248 | value => Err(VersionizeError::Deserialize(format!( 249 | "Invalid option value {}", 250 | value 251 | ))), 252 | } 253 | } 254 | 255 | // Not used yet. 256 | fn version() -> u16 { 257 | 1 258 | } 259 | } 260 | 261 | impl Versionize for Vec 262 | where 263 | T: Versionize, 264 | { 265 | #[inline] 266 | fn serialize( 267 | &self, 268 | mut writer: &mut W, 269 | version_map: &VersionMap, 270 | app_version: u16, 271 | ) -> VersionizeResult<()> { 272 | if self.len() > MAX_VEC_SIZE / std::mem::size_of::() { 273 | return Err(VersionizeError::VecLength(self.len())); 274 | } 275 | // Serialize in the same fashion as bincode: 276 | // Write len. 277 | bincode::serialize_into(&mut writer, &self.len()) 278 | .map_err(|ref err| VersionizeError::Serialize(format!("{:?}", err)))?; 279 | // Walk the vec and write each element. 280 | for element in self { 281 | element.serialize(writer, version_map, app_version)?; 282 | } 283 | Ok(()) 284 | } 285 | 286 | #[inline] 287 | fn deserialize( 288 | mut reader: &mut R, 289 | version_map: &VersionMap, 290 | app_version: u16, 291 | ) -> VersionizeResult { 292 | let mut v = Vec::new(); 293 | let len: usize = bincode::deserialize_from(&mut reader) 294 | .map_err(|ref err| VersionizeError::Deserialize(format!("{:?}", err)))?; 295 | 296 | if len > MAX_VEC_SIZE / std::mem::size_of::() { 297 | return Err(VersionizeError::VecLength(len)); 298 | } 299 | 300 | for _ in 0..len { 301 | let element: T = T::deserialize(reader, version_map, app_version) 302 | .map_err(|ref err| VersionizeError::Deserialize(format!("{:?}", err)))?; 303 | v.push(element); 304 | } 305 | Ok(v) 306 | } 307 | 308 | // Not used yet. 309 | fn version() -> u16 { 310 | 1 311 | } 312 | } 313 | 314 | // Implement versioning for FAM structures by using the FamStructWrapper interface. 315 | impl Versionize for FamStructWrapper 316 | where 317 | ::Entry: Versionize, 318 | T: std::fmt::Debug, 319 | { 320 | #[inline] 321 | fn serialize( 322 | &self, 323 | mut writer: &mut W, 324 | version_map: &VersionMap, 325 | app_version: u16, 326 | ) -> VersionizeResult<()> { 327 | // Write the fixed size header. 328 | self.as_fam_struct_ref() 329 | .serialize(&mut writer, version_map, app_version)?; 330 | // Write the array. 331 | self.as_slice() 332 | .to_vec() 333 | .serialize(&mut writer, version_map, app_version)?; 334 | 335 | Ok(()) 336 | } 337 | 338 | #[inline] 339 | fn deserialize( 340 | reader: &mut R, 341 | version_map: &VersionMap, 342 | app_version: u16, 343 | ) -> VersionizeResult { 344 | let header = T::deserialize(reader, version_map, app_version) 345 | .map_err(|ref err| VersionizeError::Deserialize(format!("{:?}", err)))?; 346 | let entries: Vec<::Entry> = 347 | Vec::deserialize(reader, version_map, app_version) 348 | .map_err(|ref err| VersionizeError::Deserialize(format!("{:?}", err)))?; 349 | 350 | if header.len() != entries.len() { 351 | let msg = format!( 352 | "Mismatch between length of FAM specified in FamStruct header ({}) \ 353 | and actual size of FAM ({})", 354 | header.len(), 355 | entries.len() 356 | ); 357 | 358 | return Err(VersionizeError::Deserialize(msg)); 359 | } 360 | 361 | // Construct the object from the array items. 362 | // Header(T) fields will be initialized by Default trait impl. 363 | let mut object = FamStructWrapper::from_entries(&entries) 364 | .map_err(|ref err| VersionizeError::Deserialize(format!("{:?}", err)))?; 365 | // SAFETY: We check above that the length in the header matches the number of elements 366 | // in the FAM. 367 | unsafe { 368 | // Update Default T with the deserialized header. 369 | *object.as_mut_fam_struct() = header; 370 | } 371 | Ok(object) 372 | } 373 | 374 | // Not used. 375 | fn version() -> u16 { 376 | 1 377 | } 378 | } 379 | 380 | // Manual implementation for tuple of 2 elems. 381 | impl Versionize for (T, U) { 382 | #[inline] 383 | fn serialize( 384 | &self, 385 | writer: &mut W, 386 | version_map: &VersionMap, 387 | app_version: u16, 388 | ) -> VersionizeResult<()> { 389 | self.0.serialize(writer, version_map, app_version)?; 390 | self.1.serialize(writer, version_map, app_version)?; 391 | Ok(()) 392 | } 393 | 394 | #[inline] 395 | fn deserialize( 396 | reader: &mut R, 397 | version_map: &VersionMap, 398 | app_version: u16, 399 | ) -> VersionizeResult { 400 | Ok(( 401 | T::deserialize(reader, version_map, app_version)?, 402 | U::deserialize(reader, version_map, app_version)?, 403 | )) 404 | } 405 | 406 | // Not used yet. 407 | fn version() -> u16 { 408 | 1 409 | } 410 | } 411 | 412 | #[cfg(test)] 413 | mod tests { 414 | #![allow(clippy::undocumented_unsafe_blocks)] 415 | 416 | use super::*; 417 | use super::{VersionMap, Versionize, VersionizeResult}; 418 | 419 | // Generate primitive tests using this macro. 420 | macro_rules! primitive_int_test { 421 | ($ty:ident, $fn_name:ident) => { 422 | #[test] 423 | fn $fn_name() { 424 | let vm = VersionMap::new(); 425 | let mut snapshot_mem = vec![0u8; 64]; 426 | 427 | let store: $ty = std::$ty::MAX; 428 | store 429 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 430 | .unwrap(); 431 | let restore = 432 | <$ty as Versionize>::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 433 | 434 | assert_eq!(store, restore); 435 | } 436 | }; 437 | } 438 | 439 | primitive_int_test!(usize, test_ser_de_usize); 440 | primitive_int_test!(isize, test_ser_de_isize); 441 | primitive_int_test!(u8, test_ser_de_u8); 442 | primitive_int_test!(u16, test_ser_de_u16); 443 | primitive_int_test!(u32, test_ser_de_u32); 444 | primitive_int_test!(u64, test_ser_de_u64); 445 | primitive_int_test!(u128, test_ser_de_u128); 446 | primitive_int_test!(i8, test_ser_de_i8); 447 | primitive_int_test!(i16, test_ser_de_i16); 448 | primitive_int_test!(i32, test_ser_de_i32); 449 | primitive_int_test!(i64, test_ser_de_i64); 450 | primitive_int_test!(i128, test_ser_de_i128); 451 | primitive_int_test!(f32, test_ser_de_f32); 452 | primitive_int_test!(f64, test_ser_de_f64); 453 | primitive_int_test!(char, test_ser_de_char); 454 | 455 | #[test] 456 | fn test_corrupted_string_len() { 457 | let vm = VersionMap::new(); 458 | let mut buffer = vec![0u8; 1024]; 459 | 460 | let string = String::from("Test string1"); 461 | string 462 | .serialize(&mut buffer.as_mut_slice(), &vm, 1) 463 | .unwrap(); 464 | 465 | // Test corrupt length field. 466 | assert_eq!( 467 | ::deserialize( 468 | &mut buffer.as_slice().split_first().unwrap().1, 469 | &vm, 470 | 1 471 | ) 472 | .unwrap_err(), 473 | VersionizeError::StringLength(6052837899185946624) 474 | ); 475 | 476 | // Test incomplete string. 477 | assert_eq!( 478 | ::deserialize(&mut buffer.as_slice().split_at(6).0, &vm, 1) 479 | .unwrap_err(), 480 | VersionizeError::Deserialize( 481 | "Io(Error { kind: UnexpectedEof, message: \"failed to fill whole buffer\" })" 482 | .to_owned() 483 | ) 484 | ); 485 | 486 | // Test NULL string len. 487 | buffer[0] = 0; 488 | assert_eq!( 489 | ::deserialize(&mut buffer.as_slice(), &vm, 1).unwrap(), 490 | String::new() 491 | ); 492 | } 493 | 494 | #[test] 495 | fn test_corrupted_vec_len() { 496 | let vm = VersionMap::new(); 497 | let mut buffer = vec![0u8; 1024]; 498 | 499 | let mut string = String::from("Test string1"); 500 | let vec = unsafe { string.as_mut_vec() }; 501 | vec.serialize(&mut buffer.as_mut_slice(), &vm, 1).unwrap(); 502 | 503 | // Test corrupt length field. 504 | assert_eq!( 505 | as Versionize>::deserialize( 506 | &mut buffer.as_slice().split_first().unwrap().1, 507 | &vm, 508 | 1 509 | ) 510 | .unwrap_err(), 511 | VersionizeError::VecLength(6052837899185946624) 512 | ); 513 | 514 | // Test incomplete Vec. 515 | assert_eq!( 516 | as Versionize>::deserialize(&mut buffer.as_slice().split_at(6).0, &vm, 1) 517 | .unwrap_err(), 518 | VersionizeError::Deserialize( 519 | "Io(Error { kind: UnexpectedEof, message: \"failed to fill whole buffer\" })" 520 | .to_owned() 521 | ) 522 | ); 523 | 524 | // Test NULL Vec len. 525 | buffer[0] = 0; 526 | assert_eq!( 527 | as Versionize>::deserialize(&mut buffer.as_slice(), &vm, 1).unwrap(), 528 | Vec::new() 529 | ); 530 | } 531 | 532 | #[test] 533 | fn test_ser_de_u32_tuple() { 534 | let vm = VersionMap::new(); 535 | let mut snapshot_mem = vec![0u8; 64]; 536 | 537 | let store: (u32, u32) = (std::u32::MIN, std::u32::MAX); 538 | store 539 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 540 | .unwrap(); 541 | let restore = 542 | <(u32, u32) as Versionize>::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 543 | 544 | assert_eq!(store, restore); 545 | } 546 | 547 | #[derive(Debug, serde_derive::Deserialize, PartialEq, serde_derive::Serialize, Versionize)] 548 | enum CompatibleEnum { 549 | A, 550 | B(String), 551 | C(u64, u64, char), 552 | } 553 | 554 | #[derive(Debug, serde_derive::Deserialize, PartialEq, serde_derive::Serialize, Versionize)] 555 | struct TestCompatibility { 556 | _string: String, 557 | _array: [u8; 32], 558 | _u8: u8, 559 | _u16: u16, 560 | _u32: u32, 561 | _u64: u64, 562 | _u128: u128, 563 | _i8: i8, 564 | _i16: i16, 565 | _i32: i32, 566 | _i64: i64, 567 | _i128: i128, 568 | _f32: f32, 569 | _f64: f64, 570 | _usize: usize, 571 | _isize: isize, 572 | _vec: Vec, 573 | _option: Option, 574 | _enums: Vec, 575 | #[allow(clippy::box_collection)] // we want to test boxes explicitly 576 | _box: Box, 577 | } 578 | 579 | #[test] 580 | fn test_bincode_deserialize_from_versionize() { 581 | let mut snapshot_mem = vec![0u8; 4096]; 582 | let vm = VersionMap::new(); 583 | 584 | let test_struct = TestCompatibility { 585 | _string: "String".to_owned(), 586 | _array: [128u8; 32], 587 | _u8: 1, 588 | _u16: 32000, 589 | _u32: 0x1234_5678, 590 | _u64: 0x1234_5678_9875_4321, 591 | _u128: 0x1234_5678_1234_5678_1234_5678_1234_5678, 592 | _i8: -1, 593 | _i16: -32000, 594 | _i32: -0x1234_5678, 595 | _i64: -0x1234_5678_9875_4321, 596 | _i128: -0x1234_5678_9098_7654_3212_3456_7890_9876, 597 | _usize: 0x1234_5678_9875_4321, 598 | _isize: -0x1234_5678_9875_4321, 599 | _f32: 0.123, 600 | _f64: 0.123_456_789_000_000, 601 | _vec: vec![33; 32], 602 | _option: Some(true), 603 | _enums: vec![ 604 | CompatibleEnum::A, 605 | CompatibleEnum::B("abcd".to_owned()), 606 | CompatibleEnum::C(1, 2, 'a'), 607 | ], 608 | _box: Box::new("Box".to_owned()), 609 | }; 610 | 611 | Versionize::serialize(&test_struct, &mut snapshot_mem.as_mut_slice(), &vm, 1).unwrap(); 612 | 613 | let restored_state: TestCompatibility = 614 | bincode::deserialize_from(snapshot_mem.as_slice()).unwrap(); 615 | assert_eq!(test_struct, restored_state); 616 | } 617 | 618 | #[test] 619 | fn test_bincode_serialize_to_versionize() { 620 | let mut snapshot_mem = vec![0u8; 4096]; 621 | let vm = VersionMap::new(); 622 | 623 | let test_struct = TestCompatibility { 624 | _string: "String".to_owned(), 625 | _array: [128u8; 32], 626 | _u8: 1, 627 | _u16: 32000, 628 | _u32: 0x1234_5678, 629 | _u64: 0x1234_5678_9875_4321, 630 | _u128: 0x1234_1234_1234_1234_1234_1234_1234_1234, 631 | _i8: -1, 632 | _i16: -32000, 633 | _i32: -0x1234_5678, 634 | _i64: -0x1234_5678_9875_4321, 635 | _i128: -0x1234_1234_1234_1234_1234_1234_1234_1234, 636 | _usize: 0x1234_5678_9875_4321, 637 | _isize: -0x1234_5678_9875_4321, 638 | _f32: 0.123, 639 | _f64: 0.123_456_789_000_000, 640 | _vec: vec![33; 32], 641 | _option: Some(true), 642 | _enums: vec![ 643 | CompatibleEnum::A, 644 | CompatibleEnum::B("abcd".to_owned()), 645 | CompatibleEnum::C(1, 2, 'a'), 646 | ], 647 | _box: Box::new("Box".to_owned()), 648 | }; 649 | 650 | bincode::serialize_into(snapshot_mem.as_mut_slice(), &test_struct).unwrap(); 651 | 652 | let restored_state: TestCompatibility = 653 | Versionize::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 654 | assert_eq!(test_struct, restored_state); 655 | } 656 | 657 | #[test] 658 | fn test_ser_de_bool() { 659 | let vm = VersionMap::new(); 660 | let mut snapshot_mem = vec![0u8; 64]; 661 | 662 | let store = true; 663 | store 664 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 665 | .unwrap(); 666 | let restore = 667 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 668 | 669 | assert_eq!(store, restore); 670 | } 671 | 672 | #[test] 673 | fn test_ser_de_string() { 674 | let vm = VersionMap::new(); 675 | let mut snapshot_mem = vec![0u8; 64]; 676 | 677 | let store = String::from("test string"); 678 | store 679 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 680 | .unwrap(); 681 | let restore = 682 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 683 | 684 | assert_eq!(store, restore); 685 | } 686 | 687 | #[test] 688 | fn test_ser_de_vec() { 689 | let vm = VersionMap::new(); 690 | let mut snapshot_mem = vec![0u8; 64]; 691 | 692 | let store = vec![ 693 | "test 1".to_owned(), 694 | "test 2".to_owned(), 695 | "test 3".to_owned(), 696 | ]; 697 | 698 | store 699 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 700 | .unwrap(); 701 | let restore = 702 | as Versionize>::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 703 | 704 | assert_eq!(store, restore); 705 | } 706 | 707 | #[test] 708 | fn test_ser_de_option() { 709 | let vm = VersionMap::new(); 710 | let mut snapshot_mem = vec![0u8; 64]; 711 | let mut store = Some("test".to_owned()); 712 | 713 | store 714 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 715 | .unwrap(); 716 | let mut restore = 717 | as Versionize>::deserialize(&mut snapshot_mem.as_slice(), &vm, 1) 718 | .unwrap(); 719 | assert_eq!(store, restore); 720 | 721 | // Check that ser_de also works for `None` variant. 722 | store = None; 723 | store 724 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 725 | .unwrap(); 726 | restore = as Versionize>::deserialize(&mut snapshot_mem.as_slice(), &vm, 1) 727 | .unwrap(); 728 | assert_eq!(store, restore); 729 | 730 | store = Some("test".to_owned()); 731 | store 732 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 733 | .unwrap(); 734 | // Corrupt `snapshot_mem` by changing the most significant bit to a value different than 0 or 1. 735 | snapshot_mem[0] = 2; 736 | let restore_err = 737 | as Versionize>::deserialize(&mut snapshot_mem.as_slice(), &vm, 1) 738 | .unwrap_err(); 739 | assert_eq!( 740 | restore_err, 741 | VersionizeError::Deserialize("Invalid option value 2".to_string()) 742 | ); 743 | // Corrupt `snapshot_mem` by changing the most significant bit from 1 (`Some(type)`) to 0 (`None`). 744 | snapshot_mem[0] = 0; 745 | restore = as Versionize>::deserialize(&mut snapshot_mem.as_slice(), &vm, 1) 746 | .unwrap(); 747 | assert_ne!(store, restore); 748 | } 749 | 750 | #[test] 751 | fn test_ser_de_box() { 752 | let vm = VersionMap::new(); 753 | let mut snapshot_mem = vec![0u8; 64]; 754 | 755 | let store = Box::new("test".to_owned()); 756 | 757 | store 758 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 759 | .unwrap(); 760 | let restore = 761 | as Versionize>::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 762 | 763 | assert_eq!(store, restore); 764 | } 765 | 766 | #[test] 767 | fn test_ser_de_wrapping() { 768 | let vm = VersionMap::new(); 769 | let mut snapshot_mem = vec![0u8; 64]; 770 | 771 | let store = std::num::Wrapping(1337u32); 772 | 773 | store 774 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 775 | .unwrap(); 776 | let restore = as Versionize>::deserialize( 777 | &mut snapshot_mem.as_slice(), 778 | &vm, 779 | 1, 780 | ) 781 | .unwrap(); 782 | 783 | assert_eq!(store, restore); 784 | } 785 | 786 | #[test] 787 | fn test_vec_limit() { 788 | // We need extra 8 bytes for vector len. 789 | let mut snapshot_mem = vec![0u8; MAX_VEC_SIZE + 8]; 790 | let err = vec![123u8; MAX_VEC_SIZE + 1] 791 | .serialize(&mut snapshot_mem.as_mut_slice(), &VersionMap::new(), 1) 792 | .unwrap_err(); 793 | assert_eq!(err, VersionizeError::VecLength(MAX_VEC_SIZE + 1)); 794 | assert_eq!( 795 | format!("{}", err), 796 | "Vec of length 10485761 exceeded maximum size of 10485760 bytes" 797 | ); 798 | } 799 | 800 | #[test] 801 | fn test_string_limit() { 802 | // We need extra 8 bytes for string len. 803 | let mut snapshot_mem = vec![0u8; MAX_STRING_LEN + 8]; 804 | let err = String::from_utf8(vec![123u8; MAX_STRING_LEN + 1]) 805 | .unwrap() 806 | .serialize(&mut snapshot_mem.as_mut_slice(), &VersionMap::new(), 1) 807 | .unwrap_err(); 808 | assert_eq!(err, VersionizeError::StringLength(MAX_STRING_LEN + 1)); 809 | assert_eq!( 810 | format!("{}", err), 811 | "String length exceeded 16385 > 16384 bytes" 812 | ); 813 | } 814 | } 815 | -------------------------------------------------------------------------------- /src/version_map.rs: -------------------------------------------------------------------------------- 1 | // Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: Apache-2.0 3 | 4 | //! A helper to map struct and enum versions to a sequence of root versions. 5 | //! This helper is required to support the versioning of a hierarchy of 6 | //! structures composed of individually versioned structures or enums. 7 | //! 8 | //! ```rust 9 | //! extern crate versionize; 10 | //! extern crate versionize_derive; 11 | //! 12 | //! use versionize::{VersionMap, Versionize, VersionizeResult}; 13 | //! use versionize_derive::Versionize; 14 | //! 15 | //! #[derive(Versionize)] 16 | //! pub struct Struct1 { 17 | //! a: u32, 18 | //! #[version(start = 2)] 19 | //! b: u8, 20 | //! } 21 | //! 22 | //! #[derive(Versionize)] 23 | //! pub struct Struct2 { 24 | //! x: u32, 25 | //! #[version(start = 2)] 26 | //! y: u8, 27 | //! } 28 | //! 29 | //! #[derive(Versionize)] 30 | //! pub struct State { 31 | //! struct1: Struct1, 32 | //! struct2: Struct2, 33 | //! } 34 | //! 35 | //! let mut version_map = VersionMap::new(); // 36 | //! version_map 37 | //! .new_version() 38 | //! .set_type_version(Struct1::type_id(), 2) 39 | //! .new_version() 40 | //! .set_type_version(Struct2::type_id(), 2); 41 | //! 42 | //! // Check that there are 3 root versions. 43 | //! assert_eq!(version_map.latest_version(), 3); 44 | //! 45 | //! // Check that root version 1 has all structs at version 1. 46 | //! assert_eq!(version_map.get_type_version(1, Struct1::type_id()), 1); 47 | //! assert_eq!(version_map.get_type_version(1, Struct2::type_id()), 1); 48 | //! assert_eq!(version_map.get_type_version(1, State::type_id()), 1); 49 | //! 50 | //! // Check that root version 2 has Struct1 at version 2 and Struct2 51 | //! // at version 1. 52 | //! assert_eq!(version_map.get_type_version(2, Struct1::type_id()), 2); 53 | //! assert_eq!(version_map.get_type_version(2, Struct2::type_id()), 1); 54 | //! assert_eq!(version_map.get_type_version(2, State::type_id()), 1); 55 | //! 56 | //! // Check that root version 3 has Struct1 and Struct2 at version 2. 57 | //! assert_eq!(version_map.get_type_version(3, Struct1::type_id()), 2); 58 | //! assert_eq!(version_map.get_type_version(3, Struct2::type_id()), 2); 59 | //! assert_eq!(version_map.get_type_version(3, State::type_id()), 1); 60 | //! ``` 61 | 62 | use std::any::TypeId; 63 | use std::collections::hash_map::HashMap; 64 | use std::fmt::Debug; 65 | use std::sync::Arc; 66 | 67 | const BASE_VERSION: u16 = 1; 68 | 69 | /// Trait to check whether is specific `version` is supported by a `VersionMap`. 70 | pub trait VersionFilter: Debug { 71 | /// Check whether the `version` is supported or not. 72 | fn is_supported(&self, version: u16) -> bool; 73 | } 74 | 75 | impl VersionFilter for () { 76 | fn is_supported(&self, _version: u16) -> bool { 77 | true 78 | } 79 | } 80 | /// 81 | /// The VersionMap API provides functionality to define the version for each 82 | /// type and attach them to specific root versions. 83 | #[derive(Clone, Debug)] 84 | pub struct VersionMap { 85 | versions: Vec>, 86 | filter: Arc, 87 | } 88 | 89 | impl Default for VersionMap { 90 | fn default() -> Self { 91 | VersionMap { 92 | versions: vec![HashMap::new()], 93 | filter: Arc::new(()), 94 | } 95 | } 96 | } 97 | 98 | impl VersionMap { 99 | /// Create a new version map initialized at version 1. 100 | pub fn new() -> Self { 101 | Default::default() 102 | } 103 | 104 | /// Create a new version map with specified version filter. 105 | pub fn with_filter(filter: Arc) -> Self { 106 | VersionMap { 107 | versions: vec![HashMap::new()], 108 | filter, 109 | } 110 | } 111 | 112 | /// Bumps root version by 1 to create a new root version. 113 | pub fn new_version(&mut self) -> &mut Self { 114 | self.versions.push(HashMap::new()); 115 | self 116 | } 117 | 118 | /// Define a mapping between a specific type version and the latest root version. 119 | pub fn set_type_version(&mut self, type_id: TypeId, type_version: u16) -> &mut Self { 120 | // It is safe to unwrap since `self.versions` always has at least 1 element. 121 | self.versions 122 | .last_mut() 123 | .unwrap() 124 | .insert(type_id, type_version); 125 | self 126 | } 127 | 128 | /// Returns the version of `type_id` corresponding to the specified `root_version`. 129 | /// If `root_version` is out of range returns the version of `type_id` at latest version. 130 | pub fn get_type_version(&self, root_version: u16, type_id: TypeId) -> u16 { 131 | let version_space = if root_version > self.latest_version() || root_version == 0 { 132 | self.versions.as_slice() 133 | } else { 134 | self.versions.split_at(root_version as usize).0 135 | }; 136 | 137 | for i in (0..version_space.len()).rev() { 138 | if let Some(version) = version_space[i].get(&type_id) { 139 | return *version; 140 | } 141 | } 142 | 143 | BASE_VERSION 144 | } 145 | 146 | /// Returns the latest version. 147 | pub fn latest_version(&self) -> u16 { 148 | self.versions.len() as u16 149 | } 150 | 151 | /// Check whether the `version` is supported by the version map. 152 | pub fn is_supported(&self, version: u16) -> bool { 153 | if version == 0 || version > self.latest_version() { 154 | false 155 | } else { 156 | self.filter.is_supported(version) 157 | } 158 | } 159 | } 160 | 161 | #[cfg(test)] 162 | mod tests { 163 | use super::{TypeId, VersionMap, BASE_VERSION}; 164 | use std::sync::Arc; 165 | use version_map::VersionFilter; 166 | 167 | pub struct MyType; 168 | pub struct MySecondType; 169 | pub struct MyThirdType; 170 | 171 | #[derive(Debug)] 172 | struct MyFilter; 173 | 174 | impl VersionFilter for MyFilter { 175 | fn is_supported(&self, version: u16) -> bool { 176 | version < 5 177 | } 178 | } 179 | 180 | #[test] 181 | fn test_default_version() { 182 | let vm = VersionMap::new(); 183 | assert_eq!(vm.latest_version(), 1); 184 | } 185 | 186 | #[test] 187 | fn test_new_versions() { 188 | let mut vm = VersionMap::new(); 189 | vm.new_version().new_version(); 190 | assert_eq!(vm.latest_version(), 3); 191 | } 192 | 193 | #[test] 194 | fn test_1_app_version() { 195 | let mut vm = VersionMap::new(); 196 | vm.set_type_version(TypeId::of::(), 1); 197 | vm.set_type_version(TypeId::of::(), 2); 198 | vm.set_type_version(TypeId::of::(), 3); 199 | 200 | assert_eq!(vm.get_type_version(1, TypeId::of::()), 1); 201 | assert_eq!(vm.get_type_version(1, TypeId::of::()), 2); 202 | assert_eq!(vm.get_type_version(1, TypeId::of::()), 3); 203 | } 204 | 205 | #[test] 206 | fn test_100_app_version_full() { 207 | let mut vm = VersionMap::new(); 208 | 209 | for i in 1..=100 { 210 | vm.set_type_version(TypeId::of::(), i) 211 | .set_type_version(TypeId::of::(), i + 1) 212 | .set_type_version(TypeId::of::(), i + 2) 213 | .new_version(); 214 | } 215 | 216 | for i in 1..=100 { 217 | assert_eq!(vm.get_type_version(i, TypeId::of::()), i); 218 | assert_eq!(vm.get_type_version(i, TypeId::of::()), i + 1); 219 | assert_eq!(vm.get_type_version(i, TypeId::of::()), i + 2); 220 | } 221 | } 222 | 223 | #[test] 224 | fn test_version_map_is_send_and_sync() { 225 | fn assert_send_sync() {} 226 | 227 | assert_send_sync::(); 228 | } 229 | 230 | #[test] 231 | fn test_app_versions_with_gap() { 232 | let my_type_id = TypeId::of::(); 233 | let my_second_type_id = TypeId::of::(); 234 | let my_third_type_id = TypeId::of::(); 235 | 236 | let mut vm = VersionMap::new(); 237 | vm.set_type_version(my_type_id, 1); 238 | vm.set_type_version(my_second_type_id, 1); 239 | vm.set_type_version(my_third_type_id, 1); 240 | vm.new_version(); 241 | vm.set_type_version(my_type_id, 2); 242 | vm.new_version(); 243 | vm.set_type_version(my_third_type_id, 2); 244 | vm.new_version(); 245 | vm.set_type_version(my_second_type_id, 2); 246 | 247 | assert_eq!(vm.get_type_version(1, my_type_id), 1); 248 | assert_eq!(vm.get_type_version(1, my_second_type_id), 1); 249 | assert_eq!(vm.get_type_version(1, my_third_type_id), 1); 250 | 251 | assert_eq!(vm.get_type_version(2, my_type_id), 2); 252 | assert_eq!(vm.get_type_version(2, my_second_type_id), 1); 253 | assert_eq!(vm.get_type_version(2, my_third_type_id), 1); 254 | 255 | assert_eq!(vm.get_type_version(3, my_type_id), 2); 256 | assert_eq!(vm.get_type_version(3, my_second_type_id), 1); 257 | assert_eq!(vm.get_type_version(3, my_third_type_id), 2); 258 | 259 | assert_eq!(vm.get_type_version(4, my_type_id), 2); 260 | assert_eq!(vm.get_type_version(4, my_second_type_id), 2); 261 | assert_eq!(vm.get_type_version(4, my_third_type_id), 2); 262 | } 263 | 264 | #[test] 265 | fn test_unset_type() { 266 | let vm = VersionMap::new(); 267 | assert_eq!(vm.get_type_version(1, TypeId::of::()), BASE_VERSION); 268 | } 269 | 270 | #[test] 271 | fn test_invalid_root_version() { 272 | let mut vm = VersionMap::new(); 273 | vm.new_version().set_type_version(TypeId::of::(), 2); 274 | 275 | assert_eq!(vm.get_type_version(0, TypeId::of::()), 2); 276 | 277 | assert_eq!(vm.latest_version(), 2); 278 | assert_eq!(vm.get_type_version(129, TypeId::of::()), 2); 279 | assert_eq!(vm.get_type_version(1, TypeId::of::()), BASE_VERSION); 280 | } 281 | 282 | #[test] 283 | fn test_version_filter() { 284 | let mut vm = VersionMap::default(); 285 | vm.new_version(); 286 | 287 | assert!(!vm.is_supported(0)); 288 | assert!(vm.is_supported(1)); 289 | assert!(vm.is_supported(2)); 290 | assert!(!vm.is_supported(3)); 291 | 292 | let mut vm = VersionMap::with_filter(Arc::new(MyFilter)); 293 | vm.new_version(); 294 | vm.new_version(); 295 | vm.new_version(); 296 | vm.new_version(); 297 | vm.new_version(); 298 | 299 | let vm1 = vm.clone(); 300 | assert!(!vm1.is_supported(0)); 301 | assert!(vm1.is_supported(1)); 302 | assert!(vm1.is_supported(2)); 303 | assert!(vm1.is_supported(3)); 304 | assert!(vm1.is_supported(4)); 305 | assert!(!vm1.is_supported(5)); 306 | assert!(!vm1.is_supported(6)); 307 | assert_eq!(vm.latest_version(), 6); 308 | } 309 | } 310 | -------------------------------------------------------------------------------- /tests/test.rs: -------------------------------------------------------------------------------- 1 | // Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: Apache-2.0 3 | 4 | #![allow(clippy::undocumented_unsafe_blocks)] 5 | #![allow(clippy::missing_safety_doc)] 6 | 7 | extern crate versionize; 8 | extern crate versionize_derive; 9 | extern crate vmm_sys_util; 10 | 11 | use std::fmt::{Debug, Formatter, Result}; 12 | use std::num::Wrapping; 13 | 14 | use vmm_sys_util::fam::{FamStruct, FamStructWrapper}; 15 | use vmm_sys_util::generate_fam_struct_impl; 16 | 17 | use versionize::{VersionMap, Versionize, VersionizeError, VersionizeResult}; 18 | use versionize_derive::Versionize; 19 | 20 | #[derive(Debug, PartialEq, Versionize, Eq)] 21 | pub enum TestState { 22 | Zero, 23 | One(u32), 24 | #[version(start = 2, default_fn = "default_state_two")] 25 | Two(u64), 26 | } 27 | 28 | impl TestState { 29 | fn default_state_two(&self, target_version: u16) -> VersionizeResult { 30 | match target_version { 31 | 1 => Ok(TestState::One(2)), 32 | i => Err(VersionizeError::Serialize(format!( 33 | "Unknown target version: {}", 34 | i 35 | ))), 36 | } 37 | } 38 | } 39 | 40 | #[test] 41 | fn test_hardcoded_struct_deserialization() { 42 | // We are testing representation compatibility between versions, at the `versionize` 43 | // crate level, by checking that only the newly added/removed fields changes between 44 | // versions are reflected in the hardcoded snapshot. 45 | 46 | #[rustfmt::skip] 47 | let v1_hardcoded_snapshot: &[u8] = &[ 48 | // usize field (8 bytes), u16 field (2 bytes) + 49 | 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x00, 50 | // u64 (8 bytes), i8 (1 byte), i32 (4 bytes) + 51 | 0xCD, 0xAB, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xFF, 0x20, 0x00, 0x00, 0x00, 52 | // f32 (4 bytes), f64 (8 bytes), char (1 bytes) + 53 | 0x00, 0x00, 0x00, 0x3F, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0x50, 0x40, 0x61, 54 | // String len (8 bytes) + 55 | 0x0b, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 56 | // actual String (11 bytes in our case) + 57 | 0x73, 0x6F, 0x6D, 0x65, 0x5F, 0x73, 0x74, 0x72, 0x69, 0x6E, 0x67, 58 | // enum variant number (4 bytes) + value of that variant (in this case it is 59 | // of u32 type -> 4 bytes) + 60 | 0x01, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 61 | // Option variant (1 byte) + value of variant (u8 -> 1 byte) + 62 | 0x01, 0x81, 63 | // Box: String len (8 bytes) + actual String (17 bytes in this case). 64 | 0x11, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x73, 0x6F, 0x6D, 0x65, 0x5F, 65 | 0x6F, 0x74, 0x68, 0x65, 0x72, 0x5F, 0x73, 0x74, 0x72, 0x69, 0x6E, 0x67, 66 | ]; 67 | 68 | // At version 2 isize (8 bytes), i64 (8 bytes) and bool (1 byte) fields will be also 69 | // present. At v2 there is also a new variant available for enum, so we can store that in 70 | // memory and it occupies 4 more bytes than the one stored at v1. 71 | #[rustfmt::skip] 72 | let v2_hardcoded_snapshot: &[u8] = &[ 73 | 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 74 | // New isize field. 75 | 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 76 | 0x04, 0x00, 77 | 0xCD, 0xAB, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xFF, 0x20, 0x00, 0x00, 0x00, 78 | // New i64 field. 79 | 0xFF, 0xFF, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 80 | 0x00, 0x00, 0x00, 0x3F, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0x50, 0x40, 0x61, 81 | // New bool field. 82 | 0x01, 83 | 0x0B, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 84 | 0x73, 0x6F, 0x6D, 0x65, 0x5F, 0x73, 0x74, 0x72, 0x69, 0x6E, 0x67, 85 | // New available enum variant. 86 | 0x02, 0x00, 0x00, 0x00, 0x0E, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 87 | 0x01, 0x81, 88 | 0x11, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x73, 0x6F, 0x6D, 0x65, 0x5F, 89 | 0x6F, 0x74, 0x68, 0x65, 0x72, 0x5F, 0x73, 0x74, 0x72, 0x69, 0x6E, 0x67, 90 | ]; 91 | 92 | // At version 3, u64 and i64 disappear (16 bytes) and Vec (8 + 4 = 12 bytes) and Wrapping 93 | // (4 bytes) fields are available. 94 | #[rustfmt::skip] 95 | let v3_hardcoded_snapshot: &[u8] = &[ 96 | 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 97 | 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 98 | 0x04, 0x00, 99 | 0xFF, 0x20, 0x00, 0x00, 0x00, 100 | 0x00, 0x00, 0x00, 0x3F, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0x50, 0x40, 0x61, 101 | 0x01, 102 | 0x0b, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 103 | 0x73, 0x6F, 0x6D, 0x65, 0x5F, 0x73, 0x74, 0x72, 0x69, 0x6E, 0x67, 104 | 0x02, 0x00, 0x00, 0x00, 0x0E, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 105 | 0x01, 0x81, 106 | // Vec len (8 bytes) + actual Vec (4 bytes). 107 | 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x61, 0x61, 0x61, 0x61, 108 | 0x11, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x73, 0x6F, 0x6D, 0x65, 0x5F, 109 | 0x6F, 0x74, 0x68, 0x65, 0x72, 0x5F, 0x73, 0x74, 0x72, 0x69, 0x6E, 0x67, 110 | // Wrapping over an u32 (4 bytes). 111 | 0xFF, 0x00, 0x00, 0x00, 112 | ]; 113 | 114 | // At version 4, isize and Vec disappear (20 bytes): 0x6F - 0x14 = 0x5B. 115 | #[rustfmt::skip] 116 | let v4_hardcoded_snapshot: &[u8] = &[ 117 | 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 118 | 0x04, 0x00, 119 | 0xFF, 0x20, 0x00, 0x00, 0x00, 120 | 0x00, 0x00, 0x00, 0x3F, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0x50, 0x40, 0x61, 121 | 0x01, 122 | 0x0b, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 123 | 0x73, 0x6F, 0x6D, 0x65, 0x5F, 0x73, 0x74, 0x72, 0x69, 0x6E, 0x67, 124 | 0x02, 0x00, 0x00, 0x00, 0x0e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 125 | 0x01, 0x81, 126 | 0x11, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x73, 0x6F, 0x6D, 0x65, 0x5F, 127 | 0x6F, 0x74, 0x68, 0x65, 0x72, 0x5F, 0x73, 0x74, 0x72, 0x69, 0x6E, 0x67, 128 | 0xFF, 0x00, 0x00, 0x00, 129 | ]; 130 | 131 | #[derive(Debug, PartialEq, Versionize)] 132 | pub struct TestStruct { 133 | usize_1: usize, 134 | #[version(start = 2, end = 4, default_fn = "default_isize")] 135 | isize_1: isize, 136 | u16_1: u16, 137 | #[version(end = 3, default_fn = "default_u64")] 138 | u64_1: u64, 139 | i8_1: i8, 140 | #[version(start = 2, end = 2)] 141 | i16_1: i16, 142 | i32_1: i32, 143 | #[version(start = 2, end = 3, default_fn = "default_i64")] 144 | i64_1: i64, 145 | f32_1: f32, 146 | f64_1: f64, 147 | char_1: char, 148 | #[version(start = 2, default_fn = "default_bool")] 149 | bool_1: bool, 150 | string_1: String, 151 | enum_1: TestState, 152 | option_1: Option, 153 | #[version(start = 3, end = 4, default_fn = "default_vec")] 154 | vec_1: Vec, 155 | #[allow(clippy::box_collection)] // we want to explicitly test Box 156 | box_1: Box, 157 | #[version(start = 3)] 158 | wrapping_1: Wrapping, 159 | } 160 | 161 | impl TestStruct { 162 | fn default_isize(_source_version: u16) -> isize { 163 | 12isize 164 | } 165 | 166 | fn default_u64(_source_version: u16) -> u64 { 167 | 0x0Du64 168 | } 169 | 170 | fn default_i64(_source_version: u16) -> i64 { 171 | 0x0Ei64 172 | } 173 | 174 | fn default_bool(_source_version: u16) -> bool { 175 | false 176 | } 177 | 178 | fn default_vec(_source_version: u16) -> Vec { 179 | vec!['v'; 8] 180 | } 181 | } 182 | 183 | let mut vm = VersionMap::new(); 184 | vm.new_version() 185 | .set_type_version(TestStruct::type_id(), 2) 186 | .set_type_version(TestState::type_id(), 2) 187 | .new_version() 188 | .set_type_version(TestStruct::type_id(), 3) 189 | .new_version() 190 | .set_type_version(TestStruct::type_id(), 4); 191 | 192 | let mut snapshot_blob = v1_hardcoded_snapshot; 193 | 194 | let mut restored_state = 195 | ::deserialize(&mut snapshot_blob, &vm, 1).unwrap(); 196 | 197 | // We expect isize, i16, i64, bool, Vec and Wrapping fields to have the default values at v1. 198 | let mut expected_state = TestStruct { 199 | usize_1: 1, 200 | isize_1: 12, 201 | u16_1: 4, 202 | u64_1: 0xABCDu64, 203 | i8_1: -1, 204 | i16_1: 0, 205 | i32_1: 32, 206 | i64_1: 0x0Ei64, 207 | f32_1: 0.5, 208 | f64_1: 64.5, 209 | char_1: 'a', 210 | bool_1: false, 211 | string_1: "some_string".to_owned(), 212 | enum_1: TestState::One(2), 213 | option_1: Some(129), 214 | vec_1: vec!['v'; 8], 215 | box_1: Box::new("some_other_string".to_owned()), 216 | wrapping_1: Wrapping(0u32), 217 | }; 218 | assert_eq!(restored_state, expected_state); 219 | 220 | snapshot_blob = v2_hardcoded_snapshot; 221 | 222 | restored_state = ::deserialize(&mut snapshot_blob, &vm, 2).unwrap(); 223 | 224 | // We expect only i16, Vec and Wrapping fields to have the default values at v2. 225 | expected_state = TestStruct { 226 | usize_1: 1, 227 | isize_1: 2, 228 | u16_1: 4, 229 | u64_1: 0xABCDu64, 230 | i8_1: -1, 231 | i16_1: 0, 232 | i32_1: 32, 233 | i64_1: 0xFFFFi64, 234 | f32_1: 0.5, 235 | f64_1: 64.5, 236 | char_1: 'a', 237 | bool_1: true, 238 | string_1: "some_string".to_owned(), 239 | enum_1: TestState::Two(14), 240 | option_1: Some(129), 241 | vec_1: vec!['v'; 8], 242 | box_1: Box::new("some_other_string".to_owned()), 243 | wrapping_1: Wrapping(0u32), 244 | }; 245 | assert_eq!(restored_state, expected_state); 246 | 247 | snapshot_blob = v3_hardcoded_snapshot; 248 | 249 | restored_state = ::deserialize(&mut snapshot_blob, &vm, 3).unwrap(); 250 | 251 | // We expect u64, i16 and i64 fields to have the default values at v3. 252 | expected_state = TestStruct { 253 | usize_1: 1, 254 | isize_1: 2, 255 | u16_1: 4, 256 | u64_1: 0x0Du64, 257 | i8_1: -1, 258 | i16_1: 0, 259 | i32_1: 32, 260 | i64_1: 0x0Ei64, 261 | f32_1: 0.5, 262 | f64_1: 64.5, 263 | char_1: 'a', 264 | bool_1: true, 265 | string_1: "some_string".to_owned(), 266 | enum_1: TestState::Two(14), 267 | option_1: Some(129), 268 | vec_1: vec!['a'; 4], 269 | box_1: Box::new("some_other_string".to_owned()), 270 | wrapping_1: Wrapping(255u32), 271 | }; 272 | assert_eq!(restored_state, expected_state); 273 | 274 | snapshot_blob = v4_hardcoded_snapshot; 275 | 276 | restored_state = ::deserialize(&mut snapshot_blob, &vm, 4).unwrap(); 277 | 278 | // We expect isize, u64, i16, i64 and Vec fields to have the default values at v4. 279 | expected_state = TestStruct { 280 | usize_1: 1, 281 | isize_1: 12, 282 | u16_1: 4, 283 | u64_1: 0x0Du64, 284 | i8_1: -1, 285 | i16_1: 0, 286 | i32_1: 32, 287 | i64_1: 0x0Ei64, 288 | f32_1: 0.5, 289 | f64_1: 64.5, 290 | char_1: 'a', 291 | bool_1: true, 292 | string_1: "some_string".to_owned(), 293 | enum_1: TestState::Two(14), 294 | option_1: Some(129), 295 | vec_1: vec!['v'; 8], 296 | box_1: Box::new("some_other_string".to_owned()), 297 | wrapping_1: Wrapping(255u32), 298 | }; 299 | assert_eq!(restored_state, expected_state); 300 | } 301 | 302 | #[test] 303 | fn test_hardcoded_enum_deserialization() { 304 | // We are testing separately also hardcoded snapshot deserialization for enums 305 | // as these have a different behavior in terms of serialization/deserialization. 306 | #[rustfmt::skip] 307 | let v1_hardcoded_snapshot: &[u8] = &[ 308 | // Variant number (4 bytes), the first variant lacks a value. 309 | 0x00, 0x00, 0x00, 0x00, 310 | ]; 311 | 312 | #[rustfmt::skip] 313 | let v2_hardcoded_snapshot: &[u8] = &[ 314 | 0x00, 0x00, 0x00, 0x00, 315 | ]; 316 | 317 | #[rustfmt::skip] 318 | let unexpected_v1_hardcoded_snapshot: &[u8] = &[ 319 | // Second variant (4 bytes) + value of that variant (8 bytes). 320 | 0x02, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 321 | ]; 322 | 323 | #[rustfmt::skip] 324 | let invalid_v1_hardcoded_snapshot: &[u8] = &[ 325 | // Invalid enum variant number. 326 | 0x03, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 327 | ]; 328 | 329 | let mut vm = VersionMap::new(); 330 | vm.new_version().set_type_version(TestState::type_id(), 2); 331 | 332 | let mut snapshot_blob = v1_hardcoded_snapshot; 333 | 334 | let mut restored_state = 335 | ::deserialize(&mut snapshot_blob, &vm, 1).unwrap(); 336 | assert_eq!(restored_state, TestState::Zero); 337 | 338 | snapshot_blob = v2_hardcoded_snapshot; 339 | 340 | restored_state = ::deserialize(&mut snapshot_blob, &vm, 2).unwrap(); 341 | assert_eq!(restored_state, TestState::Zero); 342 | 343 | snapshot_blob = unexpected_v1_hardcoded_snapshot; 344 | 345 | // Versioned deserialization is not implemented for enums, so even though we do not have 346 | // `Two` state available at version 2, restoring the data won't fail :(. 347 | // TODO: This must be fixed. 348 | restored_state = ::deserialize(&mut snapshot_blob, &vm, 1).unwrap(); 349 | assert_eq!(restored_state, TestState::Two(5)); 350 | 351 | // This snapshot contains a non-existent enum variant. 352 | snapshot_blob = invalid_v1_hardcoded_snapshot; 353 | 354 | assert_eq!( 355 | ::deserialize(&mut snapshot_blob, &vm, 1).unwrap_err(), 356 | VersionizeError::Deserialize("Unknown variant_index 3".to_owned()) 357 | ); 358 | } 359 | 360 | #[derive(Debug, PartialEq, Eq, Versionize)] 361 | pub struct A { 362 | a: u32, 363 | #[version(start = 1, end = 2)] 364 | b: Option, 365 | #[version(start = 2, default_fn = "default_c")] 366 | c: String, 367 | } 368 | 369 | #[derive(Debug, PartialEq, Eq, Versionize)] 370 | pub struct X { 371 | x: bool, 372 | a_1: A, 373 | #[version(end = 3, default_fn = "default_y")] 374 | y: Box, 375 | #[version(start = 3, default_fn = "default_z")] 376 | z: Vec, 377 | } 378 | 379 | impl A { 380 | fn default_c(_source_version: u16) -> String { 381 | "some_string".to_owned() 382 | } 383 | } 384 | 385 | impl X { 386 | fn default_y(_source_version: u16) -> Box { 387 | Box::from(4) 388 | } 389 | 390 | fn default_z(_source_version: u16) -> Vec { 391 | vec![16, 4] 392 | } 393 | } 394 | 395 | #[test] 396 | fn test_nested_structs_deserialization() { 397 | #[rustfmt::skip] 398 | let v1_hardcoded_snapshot: &[u8] = &[ 399 | // Bool field (1 byte) from X, `a` field from A (4 bytes) + 400 | 0x00, 0x10, 0x00, 0x00, 0x00, 401 | // `b` field from A: Option type (1 byte), inner enum variant number (4 bytes) + 402 | // + value of that variant (4 bytes) + 403 | 0x01, 0x01, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 404 | // `y` field from A (8 bytes). 405 | 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 406 | ]; 407 | 408 | #[rustfmt::skip] 409 | let v2_hardcoded_snapshot: &[u8] = &[ 410 | // Bool field (1 byte) from X, `a` field from A (4 bytes) + 411 | 0x00, 0x10, 0x00, 0x00, 0x00, 412 | // `c` field from X: String len (8 bytes) + actual String; 413 | // the Option field is not available at v2. 414 | 0x06, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 415 | 0x72, 0x61, 0x6E, 0x64, 0x6F, 0x6D, 416 | // `y` field from A (8 bytes). 417 | 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 418 | ]; 419 | 420 | #[rustfmt::skip] 421 | let v3_hardcoded_snapshot: &[u8] = &[ 422 | 0x00, 0x10, 0x00, 0x00, 0x00, 423 | 0x06, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 424 | 0x72, 0x61, 0x6E, 0x64, 0x6F, 0x6D, 425 | // `z` field from A (8 bytes). 426 | 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x18, 0x18, 427 | ]; 428 | 429 | let mut vm = VersionMap::new(); 430 | vm.new_version() 431 | .set_type_version(A::type_id(), 2) 432 | .set_type_version(X::type_id(), 2) 433 | .set_type_version(TestState::type_id(), 2) 434 | .new_version() 435 | .set_type_version(X::type_id(), 3); 436 | 437 | let mut snapshot_blob = v1_hardcoded_snapshot; 438 | 439 | let mut restored_state = ::deserialize(&mut snapshot_blob, &vm, 1).unwrap(); 440 | // We expect `z` and `c` fields to have the default values. 441 | let mut expected_state = X { 442 | x: false, 443 | a_1: A { 444 | a: 16u32, 445 | b: Some(TestState::One(4)), 446 | c: "some_string".to_owned(), 447 | }, 448 | y: Box::from(2), 449 | z: vec![16, 4], 450 | }; 451 | assert_eq!(restored_state, expected_state); 452 | 453 | snapshot_blob = v2_hardcoded_snapshot; 454 | 455 | restored_state = ::deserialize(&mut snapshot_blob, &vm, 2).unwrap(); 456 | 457 | // We expect `b` and `z` fields to have the default values. 458 | expected_state = X { 459 | x: false, 460 | a_1: A { 461 | a: 16u32, 462 | b: None, 463 | c: "random".to_owned(), 464 | }, 465 | y: Box::from(2), 466 | z: vec![16, 4], 467 | }; 468 | assert_eq!(restored_state, expected_state); 469 | 470 | snapshot_blob = v3_hardcoded_snapshot; 471 | 472 | restored_state = ::deserialize(&mut snapshot_blob, &vm, 3).unwrap(); 473 | 474 | // We expect `b` and `y` fields to have the default values. 475 | expected_state = X { 476 | x: false, 477 | a_1: A { 478 | a: 16u32, 479 | b: None, 480 | c: "random".to_owned(), 481 | }, 482 | y: Box::from(4), 483 | z: vec![24; 4], 484 | }; 485 | assert_eq!(restored_state, expected_state); 486 | } 487 | 488 | pub const SIZE: usize = 10; 489 | pub const SIZE_U8: u8 = 15; 490 | 491 | pub mod dummy_mod { 492 | pub const SIZE: usize = 20; 493 | } 494 | 495 | #[test] 496 | fn test_versionize_struct_with_array() { 497 | #[derive(Debug, PartialEq, Versionize)] 498 | struct TestStruct { 499 | a: [u32; SIZE], 500 | b: [u8; dummy_mod::SIZE], 501 | c: Option<[i16; SIZE]>, 502 | d: [u8; SIZE_U8 as usize], 503 | } 504 | 505 | let test_struct = TestStruct { 506 | a: [1; SIZE], 507 | b: [2; dummy_mod::SIZE], 508 | c: Some([3; SIZE]), 509 | d: [4; SIZE_U8 as usize], 510 | }; 511 | 512 | let mut mem = vec![0; 4096]; 513 | let version_map = VersionMap::new(); 514 | 515 | test_struct 516 | .serialize(&mut mem.as_mut_slice(), &version_map, 1) 517 | .unwrap(); 518 | let restored_test_struct = 519 | TestStruct::deserialize(&mut mem.as_slice(), &version_map, 1).unwrap(); 520 | 521 | assert_eq!(restored_test_struct, test_struct); 522 | } 523 | 524 | #[derive(Clone, Debug, PartialEq, Eq, Versionize)] 525 | pub enum DeviceStatus { 526 | Inactive, 527 | Active, 528 | #[version(start = 2, default_fn = "default_is_activating")] 529 | IsActivating(u32), 530 | } 531 | 532 | impl Default for DeviceStatus { 533 | fn default() -> Self { 534 | Self::Inactive 535 | } 536 | } 537 | 538 | #[derive(Clone, Debug, PartialEq, Eq, Versionize)] 539 | pub enum OperationSupported { 540 | Add, 541 | Remove, 542 | RemoveAndAdd(bool), 543 | #[version(start = 2, default_fn = "default_update")] 544 | Update(String), 545 | } 546 | 547 | impl Default for OperationSupported { 548 | fn default() -> Self { 549 | Self::Add 550 | } 551 | } 552 | 553 | impl DeviceStatus { 554 | fn default_is_activating(&self, target_version: u16) -> VersionizeResult { 555 | match target_version { 556 | 1 => Ok(DeviceStatus::Inactive), 557 | i => Err(VersionizeError::Serialize(format!( 558 | "Unknown target version: {}", 559 | i 560 | ))), 561 | } 562 | } 563 | } 564 | 565 | impl OperationSupported { 566 | fn default_update(&self, target_version: u16) -> VersionizeResult { 567 | match target_version { 568 | 1 => Ok(OperationSupported::RemoveAndAdd(true)), 569 | i => Err(VersionizeError::Serialize(format!( 570 | "Unknown target version: {}", 571 | i 572 | ))), 573 | } 574 | } 575 | } 576 | 577 | #[derive(Clone, Debug, PartialEq, Eq, Versionize)] 578 | pub struct Device { 579 | name: String, 580 | id: Wrapping, 581 | #[version(start = 2, ser_fn = "ser_is_activated")] 582 | is_activated: bool, 583 | some_params: Vec, 584 | #[version( 585 | start = 2, 586 | default_fn = "default_ops", 587 | ser_fn = "ser_ops", 588 | de_fn = "de_ops" 589 | )] 590 | operations: Vec, 591 | status: DeviceStatus, 592 | #[version( 593 | start = 2, 594 | default_fn = "default_queues_limit", 595 | ser_fn = "ser_queues_limit" 596 | )] 597 | no_queues_limit: usize, 598 | queues: Vec, 599 | features: u32, 600 | #[version(start = 3, ser_fn = "ser_extra", de_fn = "de_extra")] 601 | extra_features: u64, 602 | } 603 | 604 | impl Device { 605 | fn default_ops(_target_version: u16) -> Vec { 606 | vec![OperationSupported::Add, OperationSupported::Remove] 607 | } 608 | 609 | fn default_queues_limit(_target_version: u16) -> usize { 610 | 2 611 | } 612 | 613 | fn ser_ops(&mut self, target_version: u16) -> VersionizeResult<()> { 614 | // Fail if semantic serialization is called for a version >= 2. 615 | assert!(target_version < 2); 616 | self.features |= 1; 617 | Ok(()) 618 | } 619 | 620 | fn de_ops(&mut self, target_version: u16) -> VersionizeResult<()> { 621 | // Fail if semantic deserialization is called for a version >= 2. 622 | assert!(target_version < 2); 623 | if self.some_params.contains(&"active".to_owned()) { 624 | self.status = DeviceStatus::Active; 625 | } 626 | Ok(()) 627 | } 628 | 629 | fn ser_queues_limit(&mut self, target_version: u16) -> VersionizeResult<()> { 630 | // Fail if semantic serialization is called for a version >= 2. 631 | assert!(target_version < 2); 632 | if self.queues.len() > 2 { 633 | return Err(VersionizeError::Semantic("Too many queues.".to_owned())); 634 | } 635 | Ok(()) 636 | } 637 | 638 | fn ser_is_activated(&mut self, target_version: u16) -> VersionizeResult<()> { 639 | // Fail if semantic serialization is called for a version >= 2. 640 | assert!(target_version < 2); 641 | self.some_params.push("active".to_owned()); 642 | self.some_params.retain(|x| x.clone() != *"inactive"); 643 | Ok(()) 644 | } 645 | 646 | fn ser_extra(&mut self, target_version: u16) -> VersionizeResult<()> { 647 | // Fail if semantic serialization is called for the latest version. 648 | assert!(target_version < 3); 649 | self.some_params.push("extra_features".to_owned()); 650 | Ok(()) 651 | } 652 | 653 | fn de_extra(&mut self, target_version: u16) -> VersionizeResult<()> { 654 | // Fail if semantic deserialization is called for the latest version. 655 | assert!(target_version < 3); 656 | if self.queues.len() > self.no_queues_limit { 657 | return Err(VersionizeError::Semantic("Too many queues.".to_owned())); 658 | } 659 | self.features |= 1u32 << 31; 660 | Ok(()) 661 | } 662 | } 663 | 664 | #[test] 665 | fn test_versionize_struct_with_enums() { 666 | let mut vm = VersionMap::new(); 667 | vm.new_version() 668 | .set_type_version(Device::type_id(), 2) 669 | .set_type_version(DeviceStatus::type_id(), 2) 670 | .new_version() 671 | .set_type_version(Device::type_id(), 3) 672 | .set_type_version(OperationSupported::type_id(), 2); 673 | 674 | let mut state = Device { 675 | name: "block".to_owned(), 676 | id: Wrapping(1u32), 677 | is_activated: true, 678 | some_params: vec!["inactive".to_owned()], 679 | operations: vec![ 680 | OperationSupported::Add, 681 | OperationSupported::Update("random".to_owned()), 682 | ], 683 | status: DeviceStatus::Inactive, 684 | no_queues_limit: 3, 685 | queues: vec![1u8, 2u8], 686 | features: 6u32, 687 | extra_features: 0u64, 688 | }; 689 | 690 | let mut snapshot_mem = vec![0u8; 1024]; 691 | 692 | // Serialize as v1. 693 | state 694 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 695 | .unwrap(); 696 | let mut restored_state = 697 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 698 | 699 | // At v1, all of the semantic functions should be called. 700 | // `operations` and `no_queues_limit` will take the default values (set by `default_fn`s), 701 | // `features` will be modified by `ser_ops` and `de_extra`, `status` will be changed to 702 | // `Active` by `de_ops`, `is_activated` will take the default bool value, `some_params` 703 | // will be also modified and the other fields will take the original values. 704 | let mut expected_state = Device { 705 | name: "block".to_owned(), 706 | id: Wrapping(1u32), 707 | is_activated: false, 708 | some_params: vec!["active".to_owned(), "extra_features".to_owned()], 709 | operations: vec![OperationSupported::Add, OperationSupported::Remove], 710 | status: DeviceStatus::Active, 711 | no_queues_limit: 2, 712 | queues: vec![1u8, 2u8], 713 | features: 0x8000_0007u32, 714 | extra_features: 0u64, 715 | }; 716 | assert_eq!(expected_state, restored_state); 717 | 718 | state 719 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 2) 720 | .unwrap(); 721 | restored_state = 722 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 2).unwrap(); 723 | 724 | // At v2, we expect that only the semantic functions from `extra_features` to be called, 725 | // this means that `features` and `some_params` will take different values than the ones 726 | // at v1. `status` won't be modified anymore, `is_activated` and `no_queues_limit` will 727 | // take this time the original values. `operations` field will contain only the first 728 | // original element, the second one will be modified by `default_update` because at v2, 729 | // `Update` is not available. 730 | expected_state = Device { 731 | name: "block".to_owned(), 732 | id: Wrapping(1u32), 733 | is_activated: true, 734 | some_params: vec!["inactive".to_owned(), "extra_features".to_owned()], 735 | operations: vec![ 736 | OperationSupported::Add, 737 | OperationSupported::RemoveAndAdd(true), 738 | ], 739 | status: DeviceStatus::Inactive, 740 | no_queues_limit: 3, 741 | queues: vec![1u8, 2u8], 742 | features: 0x8000_0006u32, 743 | extra_features: 0u64, 744 | }; 745 | assert_eq!(expected_state, restored_state); 746 | 747 | state 748 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 3) 749 | .unwrap(); 750 | restored_state = 751 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 3).unwrap(); 752 | 753 | // At v3, `Update` variant is available, so it will be deserialized to its original value. 754 | // We expect no semantic function to be called, so `features` and `some_params` will also 755 | // take the original values. 756 | expected_state = Device { 757 | name: "block".to_owned(), 758 | id: Wrapping(1u32), 759 | is_activated: true, 760 | some_params: vec!["inactive".to_owned()], 761 | operations: vec![ 762 | OperationSupported::Add, 763 | OperationSupported::Update("random".to_owned()), 764 | ], 765 | status: DeviceStatus::Inactive, 766 | no_queues_limit: 3, 767 | queues: vec![1u8, 2u8], 768 | features: 6u32, 769 | extra_features: 0u64, 770 | }; 771 | assert_eq!(expected_state, restored_state); 772 | 773 | // Test semantic errors. 774 | state.queues = vec![1u8, 2u8, 3u8, 4u8]; 775 | assert_eq!( 776 | state 777 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 778 | .unwrap_err(), 779 | VersionizeError::Semantic("Too many queues.".to_owned()) 780 | ); 781 | 782 | state 783 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 2) 784 | .unwrap(); 785 | assert_eq!( 786 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 2).unwrap_err(), 787 | VersionizeError::Semantic("Too many queues.".to_owned()) 788 | ); 789 | } 790 | 791 | #[derive(Clone, Debug, PartialEq, Eq, Versionize)] 792 | pub enum State { 793 | Zero, 794 | One(bool), 795 | #[version(start = 2, default_fn = "default_state_two")] 796 | Two(Vec), 797 | #[version(start = 2, default_fn = "default_state_three")] 798 | Three(String), 799 | #[version(start = 3, default_fn = "default_state_four")] 800 | Four(Option), 801 | } 802 | 803 | impl Default for State { 804 | fn default() -> Self { 805 | Self::One(false) 806 | } 807 | } 808 | 809 | impl State { 810 | fn default_state_two(&self, target_version: u16) -> VersionizeResult { 811 | match target_version { 812 | 1 => Ok(State::One(true)), 813 | i => Err(VersionizeError::Serialize(format!( 814 | "Unknown target version: {}", 815 | i 816 | ))), 817 | } 818 | } 819 | 820 | fn default_state_three(&self, target_version: u16) -> VersionizeResult { 821 | match target_version { 822 | 1 => Ok(State::One(false)), 823 | i => Err(VersionizeError::Serialize(format!( 824 | "Unknown target version: {}", 825 | i 826 | ))), 827 | } 828 | } 829 | 830 | fn default_state_four(&self, target_version: u16) -> VersionizeResult { 831 | match target_version { 832 | 2 => Ok(State::Three("abc".to_owned())), 833 | 1 => Ok(State::Zero), 834 | i => Err(VersionizeError::Serialize(format!( 835 | "Unknown target version: {}", 836 | i 837 | ))), 838 | } 839 | } 840 | } 841 | 842 | #[test] 843 | fn test_versionize_enum() { 844 | let mut vm = VersionMap::new(); 845 | vm.new_version() 846 | .set_type_version(State::type_id(), 2) 847 | .new_version() 848 | .set_type_version(State::type_id(), 3); 849 | 850 | let mut snapshot_mem = vec![0u8; 1024]; 851 | 852 | // First we test that serializing and deserializing an enum variant available at the 853 | // target version results in the same variant. 854 | let mut state = State::One(true); 855 | state 856 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 857 | .unwrap(); 858 | let mut restored_state = 859 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 860 | assert_eq!(state, restored_state); 861 | 862 | // Now we test `default_fn`s for serialization of enum variants that don't exist in 863 | // previous versions. 864 | state = State::Four(Some(0x1234_5678_8765_4321u64)); 865 | state 866 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 867 | .unwrap(); 868 | restored_state = 869 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 870 | assert_eq!(restored_state, State::Zero); 871 | 872 | state 873 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 2) 874 | .unwrap(); 875 | restored_state = 876 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 2).unwrap(); 877 | assert_eq!(restored_state, State::Three("abc".to_owned())); 878 | 879 | state = State::Three("some_string".to_owned()); 880 | state 881 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 882 | .unwrap(); 883 | restored_state = 884 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 2).unwrap(); 885 | assert_eq!(restored_state, State::One(false)); 886 | } 887 | 888 | #[derive(Clone, Debug, PartialEq, Versionize)] 889 | pub struct S { 890 | a: f64, 891 | b: i64, 892 | } 893 | 894 | #[derive(Clone, Debug, PartialEq, Versionize)] 895 | pub struct Test { 896 | usize_1: usize, 897 | #[version(start = 2, end = 3, ser_fn = "ser_isize", de_fn = "de_isize")] 898 | isize_1: isize, 899 | #[version(start = 2)] 900 | u8_1: u8, 901 | #[version(end = 4, default_fn = "default_vec")] 902 | vec_1: Vec, 903 | #[version(start = 3)] 904 | wrapping_1: Wrapping, 905 | #[version( 906 | end = 3, 907 | default_fn = "default_u64", 908 | ser_fn = "ser_u64", 909 | de_fn = "de_u64" 910 | )] 911 | u64_1: u64, 912 | #[version(start = 2, ser_fn = "ser_bool")] 913 | bool_1: bool, 914 | enum_1: State, 915 | i8_1: i8, 916 | i16_1: i16, 917 | #[version(start = 3, end = 4)] 918 | i32_1: i32, 919 | #[version(start = 2, default_fn = "default_box", de_fn = "de_box")] 920 | box_1: Box, 921 | #[version(start = 2, end = 3, default_fn = "default_f32")] 922 | f32_1: f32, 923 | char_1: char, 924 | #[version( 925 | end = 3, 926 | default_fn = "default_option", 927 | ser_fn = "ser_option", 928 | de_fn = "de_option" 929 | )] 930 | option_1: Option, 931 | } 932 | 933 | impl Test { 934 | fn default_vec(_target_version: u16) -> Vec { 935 | vec![0x0102u16; 4] 936 | } 937 | 938 | fn default_u64(_target_version: u16) -> u64 { 939 | 0x0102_0102_0102_0102u64 940 | } 941 | 942 | fn default_f32(_target_version: u16) -> f32 { 943 | 0.5 944 | } 945 | 946 | fn default_box(_target_version: u16) -> Box { 947 | Box::new(S { a: 1.5, b: 2 }) 948 | } 949 | 950 | fn default_option(_target_version: u16) -> Option { 951 | Some("something".to_owned()) 952 | } 953 | 954 | fn ser_isize(&mut self, target_version: u16) -> VersionizeResult<()> { 955 | assert_ne!(target_version, 2); 956 | self.vec_1.push(0x0304u16); 957 | if self.i8_1 == -1 { 958 | return Err(VersionizeError::Semantic( 959 | "Unexpected value for `i8` field.".to_owned(), 960 | )); 961 | } 962 | Ok(()) 963 | } 964 | fn ser_u64(&mut self, target_version: u16) -> VersionizeResult<()> { 965 | assert!(target_version >= 3); 966 | self.vec_1.pop(); 967 | if self.u8_1 == 4 { 968 | self.bool_1 = false; 969 | } 970 | Ok(()) 971 | } 972 | 973 | fn ser_bool(&mut self, target_version: u16) -> VersionizeResult<()> { 974 | assert!(target_version < 2); 975 | self.vec_1.push(0x0506u16); 976 | self.vec_1.push(0x0708u16); 977 | Ok(()) 978 | } 979 | 980 | fn ser_option(&mut self, target_version: u16) -> VersionizeResult<()> { 981 | assert!(target_version >= 3); 982 | self.u8_1 += 2; 983 | if self.vec_1.len() == 10 { 984 | return Err(VersionizeError::Semantic("Vec is full.".to_owned())); 985 | } 986 | Ok(()) 987 | } 988 | 989 | fn de_isize(&mut self, target_version: u16) -> VersionizeResult<()> { 990 | assert_ne!(target_version, 2); 991 | self.u8_1 += 3; 992 | Ok(()) 993 | } 994 | 995 | fn de_u64(&mut self, target_version: u16) -> VersionizeResult<()> { 996 | assert!(target_version >= 3); 997 | self.vec_1.push(0x0101u16); 998 | Ok(()) 999 | } 1000 | 1001 | fn de_box(&mut self, target_version: u16) -> VersionizeResult<()> { 1002 | assert!(target_version < 2); 1003 | self.option_1 = Some("box_change".to_owned()); 1004 | if self.vec_1.len() == 3 { 1005 | return Err(VersionizeError::Semantic( 1006 | "Vec len is too small.".to_owned(), 1007 | )); 1008 | } 1009 | Ok(()) 1010 | } 1011 | 1012 | fn de_option(&mut self, target_version: u16) -> VersionizeResult<()> { 1013 | assert!(target_version >= 3); 1014 | self.enum_1 = State::Two(vec![1; 4]); 1015 | Ok(()) 1016 | } 1017 | } 1018 | 1019 | #[test] 1020 | fn test_versionize_struct() { 1021 | let mut vm = VersionMap::new(); 1022 | vm.new_version() 1023 | .set_type_version(Test::type_id(), 2) 1024 | .set_type_version(State::type_id(), 2) 1025 | .new_version() 1026 | .set_type_version(Test::type_id(), 3) 1027 | .set_type_version(State::type_id(), 3); 1028 | 1029 | let mut state = Test { 1030 | usize_1: 0x0102_0304_0506_0708usize, 1031 | isize_1: -0x1122_3344_5566_7788isize, 1032 | u8_1: 4, 1033 | vec_1: vec![0x1122u16; 5], 1034 | wrapping_1: Wrapping(4u32), 1035 | u64_1: 0x0102_0304_0506_0708u64, 1036 | bool_1: false, 1037 | enum_1: State::Four(Some(0x0102_0304_0506_0708u64)), 1038 | i8_1: 8, 1039 | i16_1: -12, 1040 | i32_1: -0x1234_5678, 1041 | box_1: Box::new(S { a: 4.5, b: 4 }), 1042 | f32_1: 1.25, 1043 | char_1: 'c', 1044 | option_1: None, 1045 | }; 1046 | 1047 | let mut snapshot_mem = vec![0u8; 1024]; 1048 | 1049 | // Serialize as v1. 1050 | state 1051 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 1052 | .unwrap(); 1053 | let mut restored_state = 1054 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 1055 | 1056 | let mut expected_state = Test { 1057 | // usize field exists at all versions, will take the original value. 1058 | usize_1: 0x0102_0304_0506_0708usize, 1059 | // isize field will take the default value as it is not available at v1. 1060 | isize_1: 0isize, 1061 | // u8 field doesn't exist at v1, it wll take the default value and then it will be 1062 | // modified by `de_isize`: 0 + 3 = 3. 1063 | u8_1: 3, 1064 | // Vec field will be modified by the semantic fns of the fields that don't exist 1065 | // at v1: `isize_1`, `bool_1`; there will be 3 new elements added in it. 1066 | vec_1: vec![ 1067 | 0x1122u16, 0x1122u16, 0x1122u16, 0x1122u16, 0x1122u16, 0x0304u16, 0x0506u16, 0x0708u16, 1068 | ], 1069 | // We expect here to have the default value. 1070 | wrapping_1: Wrapping(0u32), 1071 | // We expect here to have the original value. 1072 | u64_1: 0x0102_0304_0506_0708u64, 1073 | // We expect here to have the default value. 1074 | bool_1: false, 1075 | // This will take the default value for state `Four` and v1. 1076 | enum_1: State::Zero, 1077 | // i8, i16 fields take the original values. 1078 | i8_1: 8, 1079 | i16_1: -12, 1080 | // i32 field takes the default value. 1081 | i32_1: 0, 1082 | // Box and f32 fields will take the default values set by `default_fn`s. 1083 | box_1: Box::new(S { a: 1.5, b: 2 }), 1084 | f32_1: 0.5, 1085 | // We expect this field to take the original value. 1086 | char_1: 'c', 1087 | // This field will be modified by `de_box`. 1088 | option_1: Some("box_change".to_owned()), 1089 | }; 1090 | assert_eq!(expected_state, restored_state); 1091 | 1092 | // Serialize as v2. 1093 | state 1094 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 2) 1095 | .unwrap(); 1096 | restored_state = 1097 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 2).unwrap(); 1098 | 1099 | // At v2 isize, u8, bool, box and f32 fields will be available, their semantic fns won't 1100 | // be called. 1101 | expected_state = Test { 1102 | usize_1: 0x0102_0304_0506_0708usize, 1103 | isize_1: -0x1122_3344_5566_7788isize, 1104 | u8_1: 4, 1105 | // This should take the original value this time. 1106 | vec_1: vec![0x1122u16, 0x1122u16, 0x1122u16, 0x1122u16, 0x1122u16], 1107 | wrapping_1: Wrapping(0u32), 1108 | u64_1: 0x0102_0304_0506_0708u64, 1109 | bool_1: false, 1110 | // This will take the default value for state `Four` and v2. 1111 | enum_1: State::Three("abc".to_owned()), 1112 | i8_1: 8, 1113 | i16_1: -12, 1114 | i32_1: 0, 1115 | box_1: Box::new(S { a: 4.5, b: 4 }), 1116 | f32_1: 1.25, 1117 | char_1: 'c', 1118 | option_1: None, 1119 | }; 1120 | assert_eq!(expected_state, restored_state); 1121 | 1122 | // Serialize as v3. 1123 | state 1124 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 3) 1125 | .unwrap(); 1126 | restored_state = 1127 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 3).unwrap(); 1128 | 1129 | expected_state = Test { 1130 | usize_1: 0x0102_0304_0506_0708usize, 1131 | isize_1: 0isize, 1132 | // This field will be modified by `de_isize` and `ser_option`: 4 + 2 + 3 = 9. 1133 | u8_1: 9, 1134 | // Vec field will be modified by `ser_isize` (add one elem), `ser_u64` (remove one elem) 1135 | // and `de_64` (add one elem). 1136 | vec_1: vec![ 1137 | 0x1122u16, 0x1122u16, 0x1122u16, 0x1122u16, 0x1122u16, 0x0101u16, 1138 | ], 1139 | wrapping_1: Wrapping(4u32), 1140 | u64_1: 0x0102_0102_0102_0102u64, 1141 | bool_1: false, 1142 | enum_1: State::Two(vec![1; 4]), 1143 | i8_1: 8, 1144 | i16_1: -12, 1145 | i32_1: -0x1234_5678, 1146 | box_1: Box::new(S { a: 4.5, b: 4 }), 1147 | f32_1: 0.5, 1148 | char_1: 'c', 1149 | // We expect this field to take the default value set by its `default_fn`. 1150 | option_1: Some("something".to_owned()), 1151 | }; 1152 | assert_eq!(expected_state, restored_state); 1153 | 1154 | // Test semantic errors. 1155 | state.vec_1 = Vec::new(); 1156 | state 1157 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 1158 | .unwrap(); 1159 | assert_eq!( 1160 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap_err(), 1161 | VersionizeError::Semantic("Vec len is too small.".to_owned()) 1162 | ); 1163 | 1164 | state.vec_1 = vec![0x1122u16; 10]; 1165 | assert_eq!( 1166 | state 1167 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 3) 1168 | .unwrap_err(), 1169 | VersionizeError::Semantic("Vec is full.".to_owned()) 1170 | ); 1171 | 1172 | state.i8_1 = -1; 1173 | assert_eq!( 1174 | state 1175 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 1176 | .unwrap_err(), 1177 | VersionizeError::Semantic("Unexpected value for `i8` field.".to_owned()) 1178 | ); 1179 | state.i8_1 = 0; 1180 | 1181 | // Test serialize and deserialize errors. 1182 | snapshot_mem = vec![0u8; 8]; 1183 | // Serializing `state` will fail due to the small size of `snapshot_mem`. 1184 | assert_eq!( 1185 | state 1186 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 1187 | .unwrap_err(), 1188 | VersionizeError::Serialize( 1189 | "Io(Error { kind: WriteZero, message: \"failed to write whole buffer\" })".to_owned() 1190 | ) 1191 | ); 1192 | snapshot_mem = vec![0u8; 256]; 1193 | 1194 | state 1195 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 1196 | .unwrap(); 1197 | snapshot_mem.truncate(10); 1198 | // Deserialization will fail if we don't use the whole `snapshot_mem` resulted from 1199 | // serialization. 1200 | assert_eq!( 1201 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap_err(), 1202 | VersionizeError::Deserialize( 1203 | "Io(Error { kind: UnexpectedEof, message: \"failed to fill whole buffer\" })" 1204 | .to_owned() 1205 | ) 1206 | ); 1207 | } 1208 | 1209 | #[repr(C)] 1210 | #[derive(Debug, Default, Versionize)] 1211 | struct Message { 1212 | pub len: u32, 1213 | #[version(end = 4)] 1214 | pub padding: u32, 1215 | pub value: u32, 1216 | #[version(start = 2, default_fn = "default_extra_value")] 1217 | pub extra_value: u16, 1218 | #[version(start = 3, end = 4, default_fn = "default_status")] 1219 | pub status: Wrapping, 1220 | pub entries: __IncompleteArrayField, 1221 | } 1222 | 1223 | impl Message { 1224 | fn default_extra_value(_source_version: u16) -> u16 { 1225 | 4 1226 | } 1227 | 1228 | fn default_status(_source_version: u16) -> Wrapping { 1229 | Wrapping(false) 1230 | } 1231 | } 1232 | 1233 | #[repr(C)] 1234 | #[derive(Debug, Default, Versionize)] 1235 | struct Message2 { 1236 | pub len: u32, 1237 | #[version(end = 4)] 1238 | pub padding: u32, 1239 | pub value: u32, 1240 | #[version(start = 2, default_fn = "default_extra_value")] 1241 | pub extra_value: u16, 1242 | #[version(start = 3, end = 4, default_fn = "default_status")] 1243 | pub status: Wrapping, 1244 | pub entries: __IncompleteArrayField, 1245 | } 1246 | 1247 | impl Message2 { 1248 | fn default_extra_value(_source_version: u16) -> u16 { 1249 | 4 1250 | } 1251 | 1252 | fn default_status(_source_version: u16) -> Wrapping { 1253 | Wrapping(false) 1254 | } 1255 | } 1256 | 1257 | generate_fam_struct_impl!(Message, u32, entries, u32, len, 100); 1258 | // Duplicated structure used but with max_len 1 - for negative testing. 1259 | generate_fam_struct_impl!(Message2, u32, entries, u32, len, 1); 1260 | 1261 | #[repr(C)] 1262 | #[derive(Default)] 1263 | pub struct __IncompleteArrayField(::std::marker::PhantomData, [T; 0]); 1264 | 1265 | impl __IncompleteArrayField { 1266 | #[inline] 1267 | pub fn new() -> Self { 1268 | __IncompleteArrayField(::std::marker::PhantomData, []) 1269 | } 1270 | #[inline] 1271 | pub unsafe fn as_ptr(&self) -> *const T { 1272 | self as *const __IncompleteArrayField as *const T 1273 | } 1274 | #[inline] 1275 | pub unsafe fn as_mut_ptr(&mut self) -> *mut T { 1276 | self as *mut __IncompleteArrayField as *mut T 1277 | } 1278 | #[inline] 1279 | pub unsafe fn as_slice(&self, len: usize) -> &[T] { 1280 | ::std::slice::from_raw_parts(self.as_ptr(), len) 1281 | } 1282 | #[inline] 1283 | pub unsafe fn as_mut_slice(&mut self, len: usize) -> &mut [T] { 1284 | ::std::slice::from_raw_parts_mut(self.as_mut_ptr(), len) 1285 | } 1286 | } 1287 | 1288 | impl Debug for __IncompleteArrayField { 1289 | fn fmt(&self, fmt: &mut Formatter<'_>) -> Result { 1290 | fmt.write_str("__IncompleteArrayField") 1291 | } 1292 | } 1293 | 1294 | impl ::std::clone::Clone for __IncompleteArrayField { 1295 | #[inline] 1296 | fn clone(&self) -> Self { 1297 | Self::new() 1298 | } 1299 | } 1300 | 1301 | impl Versionize for __IncompleteArrayField { 1302 | #[inline] 1303 | fn serialize( 1304 | &self, 1305 | _writer: &mut W, 1306 | _version_map: &VersionMap, 1307 | _app_version: u16, 1308 | ) -> VersionizeResult<()> { 1309 | Ok(()) 1310 | } 1311 | 1312 | #[inline] 1313 | fn deserialize( 1314 | _reader: &mut R, 1315 | _version_map: &VersionMap, 1316 | _app_version: u16, 1317 | ) -> VersionizeResult { 1318 | Ok(Self::new()) 1319 | } 1320 | 1321 | // Not used. 1322 | fn version() -> u16 { 1323 | 1 1324 | } 1325 | } 1326 | 1327 | type MessageFamStructWrapper = FamStructWrapper; 1328 | type Message2FamStructWrapper = FamStructWrapper; 1329 | 1330 | #[test] 1331 | fn test_deserialize_famstructwrapper_invalid_len() { 1332 | let mut vm = VersionMap::new(); 1333 | vm.new_version() 1334 | .set_type_version(Message::type_id(), 2) 1335 | .new_version() 1336 | .set_type_version(Message::type_id(), 3) 1337 | .new_version() 1338 | .set_type_version(Message::type_id(), 4); 1339 | 1340 | // Create FamStructWrapper with len 2 1341 | let state = MessageFamStructWrapper::new(0).unwrap(); 1342 | let mut buffer = [0; 256]; 1343 | 1344 | state.serialize(&mut buffer.as_mut_slice(), &vm, 2).unwrap(); 1345 | 1346 | // the `len` field of the header is the first serialized field. 1347 | // Let's corrupt it by making it bigger than the actual number of serialized elements 1348 | buffer[0] = 255; 1349 | 1350 | assert_eq!( 1351 | MessageFamStructWrapper::deserialize(&mut buffer.as_slice(), &vm, 2).unwrap_err(), 1352 | VersionizeError::Deserialize("Mismatch between length of FAM specified in FamStruct header (255) and actual size of FAM (0)".to_string()) 1353 | ); 1354 | } 1355 | 1356 | #[test] 1357 | fn test_versionize_famstructwrapper() { 1358 | let mut vm = VersionMap::new(); 1359 | vm.new_version() 1360 | .set_type_version(Message::type_id(), 2) 1361 | .new_version() 1362 | .set_type_version(Message::type_id(), 3) 1363 | .new_version() 1364 | .set_type_version(Message::type_id(), 4); 1365 | 1366 | let mut state = MessageFamStructWrapper::new(0).unwrap(); 1367 | unsafe { 1368 | state.as_mut_fam_struct().padding = 8; 1369 | state.as_mut_fam_struct().extra_value = 16; 1370 | state.as_mut_fam_struct().status = Wrapping(true); 1371 | } 1372 | 1373 | state.push(1).unwrap(); 1374 | state.push(2).unwrap(); 1375 | 1376 | let mut snapshot_mem = vec![0u8; 256]; 1377 | 1378 | // Serialize as v1. 1379 | state 1380 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 1381 | .unwrap(); 1382 | let mut restored_state = 1383 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 1) 1384 | .unwrap(); 1385 | 1386 | let mut original_values = state.as_slice(); 1387 | let mut restored_values = restored_state.as_slice(); 1388 | assert_eq!(original_values, restored_values); 1389 | assert_eq!( 1390 | restored_values.len(), 1391 | state.as_fam_struct_ref().len as usize 1392 | ); 1393 | 1394 | assert_eq!( 1395 | state.as_fam_struct_ref().padding, 1396 | restored_state.as_fam_struct_ref().padding 1397 | ); 1398 | // `extra_value` and `status` should take the default values set by their corresponding `default_fn`s. 1399 | assert_eq!(4, restored_state.as_fam_struct_ref().extra_value); 1400 | assert_eq!(Wrapping(false), restored_state.as_fam_struct_ref().status); 1401 | 1402 | // Serialize as v2. 1403 | state 1404 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 2) 1405 | .unwrap(); 1406 | restored_state = 1407 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 2) 1408 | .unwrap(); 1409 | 1410 | original_values = state.as_slice(); 1411 | restored_values = restored_state.as_slice(); 1412 | assert_eq!(original_values, restored_values); 1413 | 1414 | assert_eq!( 1415 | state.as_fam_struct_ref().padding, 1416 | restored_state.as_fam_struct_ref().padding 1417 | ); 1418 | // `extra_value` is available at v2, so it will take its original value. 1419 | assert_eq!( 1420 | state.as_fam_struct_ref().extra_value, 1421 | restored_state.as_fam_struct_ref().extra_value 1422 | ); 1423 | assert_eq!(Wrapping(false), restored_state.as_fam_struct_ref().status); 1424 | 1425 | // Serialize as v3. 1426 | state 1427 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 3) 1428 | .unwrap(); 1429 | restored_state = 1430 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 3) 1431 | .unwrap(); 1432 | 1433 | assert_eq!( 1434 | state.as_fam_struct_ref().padding, 1435 | restored_state.as_fam_struct_ref().padding 1436 | ); 1437 | assert_eq!( 1438 | state.as_fam_struct_ref().extra_value, 1439 | restored_state.as_fam_struct_ref().extra_value 1440 | ); 1441 | // At v3, `status` field exists, so it will take its original value. 1442 | assert_eq!(Wrapping(true), restored_state.as_fam_struct_ref().status); 1443 | 1444 | // Serialize as v4. 1445 | state 1446 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 4) 1447 | .unwrap(); 1448 | restored_state = 1449 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 4) 1450 | .unwrap(); 1451 | 1452 | // At v4, `padding` field will take the default u32 value. 1453 | assert_eq!(0, restored_state.as_fam_struct_ref().padding); 1454 | assert_eq!( 1455 | state.as_fam_struct_ref().extra_value, 1456 | restored_state.as_fam_struct_ref().extra_value 1457 | ); 1458 | // `status` is not available anymore, so it will take the default value. 1459 | assert_eq!(Wrapping(false), restored_state.as_fam_struct_ref().status); 1460 | 1461 | snapshot_mem = vec![0u8; 16]; 1462 | 1463 | assert_eq!( 1464 | state 1465 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 1466 | .unwrap_err(), 1467 | VersionizeError::Serialize( 1468 | "Io(Error { kind: WriteZero, message: \"failed to write whole buffer\" })".to_owned() 1469 | ) 1470 | ); 1471 | } 1472 | 1473 | #[derive(Versionize)] 1474 | pub struct FamStructTest { 1475 | some_u8: u8, 1476 | message_box: Box, 1477 | #[version(start = 2, default_fn = "default_option", de_fn = "de_option")] 1478 | some_option: Option, 1479 | #[version(start = 3)] 1480 | some_string: String, 1481 | #[version(end = 3, default_fn = "default_message", de_fn = "de_message")] 1482 | messages: Vec, 1483 | } 1484 | 1485 | impl FamStructTest { 1486 | fn default_message(_target_version: u16) -> Vec { 1487 | let mut f = MessageFamStructWrapper::new(0).unwrap(); 1488 | unsafe { 1489 | f.as_mut_fam_struct().padding = 1; 1490 | f.as_mut_fam_struct().extra_value = 2; 1491 | } 1492 | 1493 | f.push(10).unwrap(); 1494 | f.push(20).unwrap(); 1495 | 1496 | vec![f] 1497 | } 1498 | 1499 | fn default_option(_target_version: u16) -> Option { 1500 | Some(S { a: 0.5, b: 0 }) 1501 | } 1502 | 1503 | fn de_message(&mut self, target_version: u16) -> VersionizeResult<()> { 1504 | // Fail if semantic deserialization is called for v2. 1505 | assert_ne!(target_version, 2); 1506 | self.some_option = None; 1507 | self.some_string = "some_new_string".to_owned(); 1508 | Ok(()) 1509 | } 1510 | 1511 | fn de_option(&mut self, target_version: u16) -> VersionizeResult<()> { 1512 | // Fail if semantic deserialization is called for a version >= 2. 1513 | assert!(target_version < 2); 1514 | 1515 | let mut f = MessageFamStructWrapper::new(0).unwrap(); 1516 | unsafe { 1517 | f.as_mut_fam_struct().padding = 3; 1518 | f.as_mut_fam_struct().extra_value = 4; 1519 | } 1520 | 1521 | f.push(10).unwrap(); 1522 | f.push(20).unwrap(); 1523 | 1524 | self.messages.push(f); 1525 | Ok(()) 1526 | } 1527 | } 1528 | 1529 | #[test] 1530 | fn test_versionize_struct_with_famstructs() { 1531 | let mut vm = VersionMap::new(); 1532 | vm.new_version() 1533 | .set_type_version(FamStructTest::type_id(), 2) 1534 | .set_type_version(Message::type_id(), 2) 1535 | .new_version() 1536 | .set_type_version(FamStructTest::type_id(), 3) 1537 | .set_type_version(Message::type_id(), 3); 1538 | 1539 | let mut snapshot_mem = vec![0u8; 1024]; 1540 | 1541 | let mut f = MessageFamStructWrapper::new(0).unwrap(); 1542 | unsafe { 1543 | f.as_mut_fam_struct().padding = 5; 1544 | f.as_mut_fam_struct().extra_value = 6; 1545 | } 1546 | f.push(10).unwrap(); 1547 | 1548 | let mut f2 = MessageFamStructWrapper::new(0).unwrap(); 1549 | unsafe { 1550 | f2.as_mut_fam_struct().padding = 7; 1551 | f2.as_mut_fam_struct().extra_value = 8; 1552 | } 1553 | f2.push(20).unwrap(); 1554 | 1555 | let state = FamStructTest { 1556 | some_u8: 1, 1557 | messages: vec![f], 1558 | some_string: "some_string".to_owned(), 1559 | message_box: Box::new(f2), 1560 | some_option: None, 1561 | }; 1562 | 1563 | state 1564 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 1565 | .unwrap(); 1566 | let mut restored_state = 1567 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 1568 | 1569 | // At version 1, we expect `de_option` and `de_message` to be called. 1570 | // `some_string` and `some_option` will take the default values. 1571 | assert_eq!(restored_state.some_string, String::default()); 1572 | assert_eq!(restored_state.some_option, Some(S { a: 0.5, b: 0 })); 1573 | let messages = restored_state.messages; 1574 | 1575 | // We expect to have 2 elements in the messages Vec (the one with which it was initialized and 1576 | // the one inserted by `de_option`). 1577 | assert_eq!(messages.len(), 2); 1578 | for message in messages.iter() { 1579 | assert_eq!(message.as_fam_struct_ref().extra_value, 4); 1580 | assert_eq!(message.as_fam_struct_ref().status, Wrapping(false)); 1581 | } 1582 | assert_eq!(messages[0].as_fam_struct_ref().padding, 5); 1583 | assert_eq!(messages[1].as_fam_struct_ref().padding, 3); 1584 | 1585 | // Serialize as v2. 1586 | state 1587 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 2) 1588 | .unwrap(); 1589 | restored_state = 1590 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 2).unwrap(); 1591 | 1592 | assert_eq!(restored_state.some_string, String::default()); 1593 | // `some_option` is available at v2, so it will take the original value. 1594 | assert_eq!(restored_state.some_option, None); 1595 | let messages = restored_state.messages; 1596 | // We expect to have only one element in `messages` as `de_option` shouldn't be called 1597 | // this time. 1598 | assert_eq!(messages.len(), 1); 1599 | 1600 | // Serialize as v3. 1601 | state 1602 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 3) 1603 | .unwrap(); 1604 | restored_state = 1605 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 3).unwrap(); 1606 | 1607 | // `some_string` is also available at v3. 1608 | assert_eq!(restored_state.some_string, "some_new_string".to_owned()); 1609 | assert_eq!(restored_state.some_option, None); 1610 | let messages = restored_state.messages; 1611 | // `messages` field is not available anymore at v3, it will take the default value, 1612 | // set by the corresponding `default_fn`. 1613 | assert_eq!(messages.len(), 1); 1614 | assert_eq!(messages[0].as_fam_struct_ref().padding, 1); 1615 | } 1616 | 1617 | #[derive(Clone, Versionize)] 1618 | pub struct SomeStruct { 1619 | message: MessageFamStructWrapper, 1620 | #[version(start = 2, ser_fn = "ser_u16")] 1621 | some_u16: u16, 1622 | } 1623 | 1624 | impl SomeStruct { 1625 | fn ser_u16(&mut self, target_version: u16) -> VersionizeResult<()> { 1626 | // Fail if semantic serialization is called for the latest version. 1627 | assert!(target_version < 2); 1628 | unsafe { 1629 | self.message.as_mut_fam_struct().padding += 2; 1630 | } 1631 | 1632 | Ok(()) 1633 | } 1634 | } 1635 | 1636 | #[derive(Clone, Versionize)] 1637 | pub struct SomeStruct2 { 1638 | message: Message2FamStructWrapper, 1639 | #[version(start = 2, ser_fn = "ser_u16")] 1640 | some_u16: u16, 1641 | } 1642 | 1643 | impl SomeStruct2 { 1644 | fn ser_u16(&mut self, target_version: u16) -> VersionizeResult<()> { 1645 | // Fail if semantic serialization is called for the latest version. 1646 | assert!(target_version < 2); 1647 | unsafe { 1648 | self.message.as_mut_fam_struct().padding += 2; 1649 | } 1650 | 1651 | Ok(()) 1652 | } 1653 | } 1654 | 1655 | // `Clone` issue fixed: https://github.com/rust-vmm/vmm-sys-util/issues/85. 1656 | // We are keeping this as regression test. 1657 | #[test] 1658 | fn test_famstructwrapper_clone() { 1659 | // Test that having a `FamStructWrapper` in a structure that implements 1660 | // Clone will result in keeping with their original values, only the number 1661 | // of entries and the entries array when serializing. 1662 | let mut vm = VersionMap::new(); 1663 | vm.new_version().set_type_version(SomeStruct::type_id(), 2); 1664 | 1665 | let mut f = MessageFamStructWrapper::new(0).unwrap(); 1666 | unsafe { f.as_mut_fam_struct().padding = 8 }; 1667 | 1668 | f.push(1).unwrap(); 1669 | f.push(2).unwrap(); 1670 | 1671 | let state = SomeStruct { 1672 | message: f, 1673 | some_u16: 2, 1674 | }; 1675 | 1676 | let mut snapshot_mem = vec![0u8; 128]; 1677 | 1678 | // Serialize as v1. 1679 | state 1680 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 1) 1681 | .unwrap(); 1682 | let mut restored_state = 1683 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).unwrap(); 1684 | 1685 | // Negative scenario - FamStruct versionize impl fails due to SizeLimitExceeded. 1686 | assert!( 1687 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 1).is_err() 1688 | ); 1689 | 1690 | let original_values = state.message.as_slice(); 1691 | let restored_values = restored_state.message.as_slice(); 1692 | 1693 | assert_ne!( 1694 | state.message.as_fam_struct_ref().padding, 1695 | restored_state.message.as_fam_struct_ref().padding 1696 | ); 1697 | assert_eq!(original_values, restored_values); 1698 | // `padding` field will have its value serialized (8), and then it will be incremented with 2 1699 | // by `ser_u16`. 1700 | assert_eq!(10, restored_state.message.as_fam_struct_ref().padding); 1701 | 1702 | // Serialize as v2. 1703 | state 1704 | .serialize(&mut snapshot_mem.as_mut_slice(), &vm, 2) 1705 | .unwrap(); 1706 | restored_state = 1707 | ::deserialize(&mut snapshot_mem.as_slice(), &vm, 2).unwrap(); 1708 | 1709 | // Padding is correctly preserved and ser_u16 is not called - it would fail due to ser_u16 1710 | // assert anyways, but we double that check here. 1711 | assert_eq!( 1712 | state.message.as_fam_struct_ref().padding, 1713 | restored_state.message.as_fam_struct_ref().padding 1714 | ); 1715 | // `padding` field will have its value preserved (8). `ser_u16` won't be called at v2. 1716 | assert_eq!(8, restored_state.message.as_fam_struct_ref().padding); 1717 | } 1718 | --------------------------------------------------------------------------------