├── rustfmt.toml ├── .gitignore ├── Cargo.toml ├── README.md ├── tests └── test.rs ├── LICENSE └── src └── lib.rs /rustfmt.toml: -------------------------------------------------------------------------------- 1 | tab_spaces = 2 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | /target 2 | Cargo.lock 3 | -------------------------------------------------------------------------------- /Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | name = "ointers" 3 | version = "5.0.0" 4 | description = "What do you call a pointer we stole the high bits off? An ointer." 5 | keywords = ["pointer", "bits", "intrusive", "tagged"] 6 | categories = [ 7 | "algorithms", 8 | "data-structures", 9 | "embedded", 10 | "memory-management", 11 | "no-std", 12 | ] 13 | authors = ["James Laver "] 14 | homepage = "https://github.com/irrustible/ointers" 15 | repository = "https://github.com/irrustible/ointers" 16 | documentation = "https://docs.rs/ointers" 17 | edition = "2018" # we can't upgrade to 2024 as that would raise MSRV to 1.85 18 | license = "Apache-2.0 WITH LLVM-exception" 19 | readme = "README.md" 20 | rust-version = "1.84.0" 21 | 22 | [features] 23 | default = ["alloc"] 24 | alloc = [] 25 | i_know_what_im_doing = [] 26 | # deprecated: does nothing 27 | sptr = [] 28 | 29 | [dev-dependencies] 30 | rand = "0.8.4" 31 | rayon = "1.5.1" 32 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![License](https://img.shields.io/crates/l/ointers.svg)](https://github.com/irrustible/ointers/blob/main/LICENSE) 2 | [![Package](https://img.shields.io/crates/v/ointers.svg)](https://crates.io/crates/ointers) 3 | [![Documentation](https://docs.rs/ointers/badge.svg)](https://docs.rs/ointers) 4 | 5 | # ointers 6 | 7 | What do you call a pointer with the high bits stolen? An ointer! 8 | 9 | Ointers is a library for representing pointers where some bits have 10 | been stolen so that they may be used by the programmer for something 11 | else. In effect, it's a small amount of free storage 12 | 13 | Fully supports no_std (including no-alloc), small and dependency-free. 14 | 15 | ## Bit sources 16 | 17 | Ointers supports a handful of bit sources. It's up to you to determine 18 | when it is safe to use them. 19 | 20 | ### Alignment bits (const parameter A) 21 | 22 | If we know that a pointer's address will always be aligned to `A` 23 | bytes where A > 1, we can steal log2(A-1) bits. For an 8-byte aligned 24 | value, this provides a modest 3 bits. 25 | 26 | If you have values aligned to some larger width, you could get even 27 | more. It's common in parallel programming to pad data to a cache line 28 | by increasing its alignment requirements in order eliminate false 29 | sharing. Because a cache line on amd64 or aarch64 is effectively 128 30 | bytes thanks to prefetching, you can reclaim a respectable 7 extra 31 | bits. 32 | 33 | If your data is aligned wider still, the sky is the limit, but you 34 | could get an incredible 24 bits if you have 16MB-aligned data! 35 | Remember that the only alignment rust knows about is what is declared 36 | for the type, so you must create a newtype wrapper to take full 37 | advantage of large alignment sizes. 38 | 39 | | Bits | Min alignment | 40 | |------|---------------| 41 | | 1 | 2b | 42 | | 2 | 4b | 43 | | 3 | 8b | 44 | | 4 | 16b | 45 | | 5 | 32b | 46 | | 6 | 64b | 47 | | 7 | 128b | 48 | | 8 | 256b | 49 | | 9 | 512b | 50 | | 10 | 1k | 51 | | 11 | 2k | 52 | | 12 | 4k | 53 | | 13 | 8k | 54 | | 14 | 16k | 55 | | 15 | 32k | 56 | | 16 | 64k | 57 | | 17 | 128k | 58 | | 18 | 256k | 59 | | 19 | 512k | 60 | | 20 | 1m | 61 | | 21 | 2m | 62 | | 22 | 4m | 63 | | 23 | 8m | 64 | | 24 | 16m | 65 | 66 | Stealing bits from alignment is relatively innocuous, but we can only 67 | get the compiler to check it for you in dev mode as things stand in 68 | rust today. 69 | 70 | ### Sign bit (parameter S) 71 | 72 | The most commonly used operating systems arrange memory so that the 73 | high half of virtual memory space is reserved for the kernel and the 74 | low half is given to userspace. 75 | 76 | Looked at as a signed integer, this makes the kernel half of address 77 | space negative and the userspace half positive. 78 | 79 | Most programs do not deal with kernel addresses, thus giving us an 80 | extra bit to play with. 81 | 82 | We can also get this extra bit in kernel mode if we know we will not 83 | be dealing with userspace addresses. We do this by taking a pointer to 84 | a value on the stack and stealing its sign bit. 85 | 86 | If you know you will be dealing with userspace addresses in kernel 87 | space or kernel space addresses in userspace, or you are using or 88 | implementing a kernel which does not follow this convention, you must 89 | set `S` to `false`. 90 | 91 | The S bit is currently only tested with userspace pointers in 92 | userspace. While we think it should work more generally, we currently 93 | haven't got a test rig for other scenarios so we can't promise it does. 94 | 95 | ### Unused virtual address space (V) 96 | 97 | In 64-bit mode, address space is plentiful: nobody has 64 bits' worth 98 | of RAM and even if they did, their processor is unable to address it 99 | all. V is required to be 0 unless compiling for a 64bit target. 100 | 101 | The number of bits that may be safely stolen by this method depends 102 | upon the microarchitecture in question and the page table depth. 103 | 104 | For x86-64 and aarch64, the following sizes apply: 105 | 106 | | Bits | PT depth | Support | 107 | |------|----------|-----------------------------------| 108 | | 25 | 3 | aarch64 only, uncommon, opt-in | 109 | | 16 | 4 | most common default | 110 | | 7 | 5 | some intel only, uncommon, opt-in | 111 | 112 | If you are made of money and need more than 128TB virtual address 113 | space, you should limit yourself to 7 bits for V. Likewise if you know 114 | you'll be on 3-deep page tables, you can steal a whopping 25 bits. But 115 | you're probably limited to 16 bits in general. 116 | 117 | ## Aarch64 notes 118 | 119 | There is as yet no explicit support for any of the optional platform 120 | features which use the 'unused' high pointer bits, such as MTE or 121 | Pointer Authentication. These features are typically only used in 122 | embedded devices, but if you're targeting such a system (and if you 123 | don't know, you almost certainly aren't), any attempt to steal high 124 | bits will break everything, so stick to alignment bits. 125 | 126 | ## Hacking 127 | 128 | Contributions welcome, please be nice. 129 | 130 | The test suite requires std at present (because of rand's 131 | ThreadRng). It takes about 36 seconds on my Ryzen 3900X. If you're on 132 | a slower machine, you might want to reduce the loop iterations. It 133 | should really only take one to shake most bugs out and a handful to 134 | shake out the rest. The million iterations is just overkill to be sure 135 | that's conveniently enabled by the incredible ease-of-use of rayon. 136 | 137 | The tests are extensive. The number of configurations I'm currently 138 | testing against is limited, however: 139 | 140 | * x86-64, linux with 4PT 141 | 142 | We would like to support more. These should be easy: 143 | 144 | * x86, linux with 4PT should be possible through github actions. 145 | * aarch64, linux with 4PT shouldn't be too hard to find access to. 146 | * 32-bit arm (3PT) should be doable as well. 147 | 148 | These will likely be harder: 149 | 150 | * aarch64, linux with 3PT is probably not widely deployed 151 | * Intel's new 5PT is of incredibly niche interest - if you want us to 152 | test it, you'll have to sponsor it because I don't have access to 153 | the hardware. 154 | 155 | ## Changelog 156 | 157 | ### v5.0.0 158 | 159 | New MSRV: Rust 1.84.0 160 | 161 | New features: 162 | 163 | * Support for unsized types/wide pointers (thanks @jalil-salame!) 164 | 165 | Internals: 166 | * Moved from `sptr` to the new stdlib provenance APIs (thanks @jalil-salame!) 167 | * Refactored `Ox` to use `NotNull` internally (thanks @jalil-salame!) 168 | * Refactored to use `*T` instead of `*u8` and `PhantomData` (thanks @jalil-salame!) 169 | * Explanation: Misaligned T pointers are only UB to read/write, not to hold (as in C). 170 | * Added miri support (thanks @jalil-salame!) 171 | 172 | ### v4.0.2 173 | 174 | This is a transitional release to keep support for pre-1.84 rust compilers while 175 | yanking the previous version because of the security fix. V5 is now available 176 | and you should prefer to use it if you are able. 177 | 178 | Security: 179 | * Fixed `Ox` cloning, preventing potential UAF (thanks @jalil-salame!) 180 | 181 | Misc: 182 | * Fixed the build with `--no-default-features` (i.e. use without allocator - thanks @00xc!) 183 | 184 | ### v4.0.1 185 | 186 | * Optional pointer provenance support with the `sptr` feature (thanks @GoldsteinE!) 187 | 188 | ### v4.0.0 189 | 190 | * Fix possible UB when stealing alignment bits with `Ox`/`NotNull`. 191 | * Note: this was overcaution, there was no UB. 192 | 193 | Features: 194 | 195 | * Add method `new_stealing` for all types allowing to create while stealing data. 196 | 197 | ### v3.0.1 198 | 199 | * Add `i_know_what_im_doing` feature to enable stealing from V when 200 | not building for a 64-bit target. 201 | 202 | ### v3.0.0 203 | 204 | * Make `Ointer` and `NotNull` be `Clone + Copy` even if T is not. 205 | 206 | ### v2.0.0 207 | 208 | New APIS: 209 | 210 | * `pack()` - packs a pointer into the low bits of a usize. 211 | * `unpack()` - reverse of pack. 212 | * `asv_mask()` - calculates a mask where the stolen bits are set on by a, s, v. 213 | * `mask()` - calculates a mask where the stolen bits are set on by total bits. 214 | * `Ointer::raw()` - returns the raw data in the ointer (stolen + ptr) as a usize. 215 | * `NotNull::raw()` - same, but for `NotNull`. 216 | 217 | Changes: 218 | 219 | * `Ointer` now uses a usize internally. 220 | 221 | ## Copyright and License 222 | 223 | Copyright (c) 2021 James Laver, ointers contributors. 224 | 225 | [Licensed](LICENSE) under Apache License, Version 2.0 (https://www.apache.org/licenses/LICENSE-2.0), 226 | with LLVM Exceptions (https://spdx.org/licenses/LLVM-exception.html). 227 | 228 | Unless you explicitly state otherwise, any contribution intentionally submitted 229 | for inclusion in the work by you, as defined in the Apache-2.0 license, shall be 230 | licensed as above, without any additional terms or conditions. 231 | -------------------------------------------------------------------------------- /tests/test.rs: -------------------------------------------------------------------------------- 1 | use ointers::*; 2 | use rand::{rngs::ThreadRng, thread_rng, RngCore}; 3 | use rayon::prelude::*; 4 | use std::fmt::Debug; 5 | use std::ptr::NonNull; 6 | 7 | #[cfg(target_pointer_width = "16")] 8 | fn random_usize(rng: &mut ThreadRng) -> usize { 9 | let mut bytes = [0u8; 2]; 10 | rng.fill_bytes(&bytes[..]); 11 | usize::from_native_bytes(bytes) 12 | } 13 | 14 | #[cfg(target_pointer_width = "32")] 15 | fn random_usize(rng: &mut ThreadRng) -> usize { 16 | rng.next_u32() as usize 17 | } 18 | 19 | #[cfg(target_pointer_width = "64")] 20 | fn random_usize(rng: &mut ThreadRng) -> usize { 21 | rng.next_u64() as usize 22 | } 23 | 24 | #[test] 25 | fn can_do_unsized_types() { 26 | const ALIGN: u8 = align_of::().ilog2() as u8; 27 | let mut array = [1, 2, 3]; 28 | let slice: &mut [i32] = &mut array; 29 | let ptr: Ointer<[i32], { ALIGN }, false, 0> = unsafe { Ointer::new(&raw mut *slice) }; 30 | let slice: &mut [i32] = &mut array[..1]; 31 | let ptr2: Ointer<[i32], { ALIGN }, false, 0> = unsafe { Ointer::new(&raw mut *slice) }; 32 | assert_ne!(ptr, ptr2, "different slices shouldn't compare equal"); 33 | } 34 | 35 | fn round_trip_ointer( 36 | ptr: *mut T, 37 | val: T, 38 | theft: usize, 39 | ) { 40 | let mask = mask(A + S as u8 + V); 41 | let stolen = theft & mask; 42 | let o: Ointer = unsafe { Ointer::new(ptr) }; 43 | assert_eq!(o.as_ptr(), ptr); 44 | assert_eq!(unsafe { o.as_ptr().read() }, val); 45 | assert_eq!(o.stolen(), 0); 46 | let p = o.steal(0); 47 | assert_eq!(p.as_ptr(), ptr); 48 | assert_eq!(unsafe { p.as_ptr().read() }, val); 49 | assert_eq!(p.stolen(), 0); 50 | let p = o.steal(theft); 51 | assert_eq!(p.as_ptr(), ptr); 52 | assert_eq!(unsafe { p.as_ptr().read() }, val); 53 | assert_eq!(p.stolen(), stolen); 54 | let p = o.steal(stolen); 55 | assert_eq!(p.as_ptr(), ptr); 56 | assert_eq!(unsafe { p.as_ptr().read() }, val); 57 | assert_eq!(p.stolen(), stolen); 58 | let p = o.steal(0); 59 | assert_eq!(p.as_ptr(), ptr); 60 | assert_eq!(unsafe { p.as_ptr().read() }, val); 61 | assert_eq!(p.stolen(), 0); 62 | let p = o.steal(usize::MAX); 63 | assert_eq!(p.as_ptr(), ptr); 64 | assert_eq!(unsafe { p.as_ptr().read() }, val); 65 | assert_eq!(p.stolen(), mask); 66 | } 67 | 68 | fn round_trip_not_null( 69 | ptr: NonNull, 70 | val: T, 71 | theft: usize, 72 | ) { 73 | let mask = mask(A + S as u8 + V); 74 | let stolen = theft & mask; 75 | let o: NotNull = unsafe { NotNull::new(ptr) }; 76 | assert_eq!(o.as_non_null(), ptr); 77 | assert_eq!(unsafe { o.as_non_null().as_ptr().read() }, val); 78 | assert_eq!(o.stolen(), 0); 79 | let p = o.steal(0); 80 | assert_eq!(p.as_non_null(), ptr); 81 | assert_eq!(unsafe { p.as_non_null().as_ptr().read() }, val); 82 | assert_eq!(p.stolen(), 0); 83 | let p = o.steal(theft); 84 | assert_eq!(p.as_non_null(), ptr); 85 | assert_eq!(unsafe { p.as_non_null().as_ptr().read() }, val); 86 | assert_eq!(p.stolen(), stolen); 87 | let p = o.steal(stolen); 88 | assert_eq!(p.as_non_null(), ptr); 89 | assert_eq!(unsafe { p.as_non_null().as_ptr().read() }, val); 90 | assert_eq!(p.stolen(), stolen); 91 | let p = o.steal(0); 92 | assert_eq!(p.as_non_null(), ptr); 93 | assert_eq!(unsafe { p.as_non_null().as_ptr().read() }, val); 94 | assert_eq!(p.stolen(), 0); 95 | let p = o.steal(usize::MAX); 96 | assert_eq!(p.as_non_null(), ptr); 97 | assert_eq!(unsafe { p.as_non_null().as_ptr().read() }, val); 98 | assert_eq!(p.stolen(), mask); 99 | } 100 | 101 | macro_rules! rt_ointer { 102 | ($value:expr, $ptr:expr, $theft:expr, $type:ty, $v:literal) => { 103 | round_trip_ointer::<$type, 0, false, $v>($ptr, $value, $theft); 104 | round_trip_ointer::<$type, 1, false, $v>($ptr, $value, $theft); 105 | round_trip_ointer::<$type, 2, false, $v>($ptr, $value, $theft); 106 | round_trip_ointer::<$type, 3, false, $v>($ptr, $value, $theft); 107 | round_trip_ointer::<$type, 0, true, $v>($ptr, $value, $theft); 108 | round_trip_ointer::<$type, 1, true, $v>($ptr, $value, $theft); 109 | round_trip_ointer::<$type, 2, true, $v>($ptr, $value, $theft); 110 | round_trip_ointer::<$type, 3, true, $v>($ptr, $value, $theft); 111 | }; 112 | } 113 | 114 | macro_rules! rt_not_null { 115 | ($value:expr, $ptr:expr, $theft:expr, $type:ty, $v:literal) => { 116 | round_trip_not_null::<$type, 0, false, $v>($ptr, $value, $theft); 117 | round_trip_not_null::<$type, 1, false, $v>($ptr, $value, $theft); 118 | round_trip_not_null::<$type, 2, false, $v>($ptr, $value, $theft); 119 | round_trip_not_null::<$type, 3, false, $v>($ptr, $value, $theft); 120 | round_trip_not_null::<$type, 0, true, $v>($ptr, $value, $theft); 121 | round_trip_not_null::<$type, 1, true, $v>($ptr, $value, $theft); 122 | round_trip_not_null::<$type, 2, true, $v>($ptr, $value, $theft); 123 | round_trip_not_null::<$type, 3, true, $v>($ptr, $value, $theft); 124 | }; 125 | } 126 | 127 | fn miri_or_rayon(test_fn: impl Fn(i32) + Sync + Send) { 128 | if cfg!(miri) { 129 | test_fn(0); 130 | } else { 131 | (0..1_000_000).into_par_iter().for_each(test_fn) 132 | } 133 | } 134 | 135 | #[test] 136 | fn round_trip_ointer_locals() { 137 | miri_or_rayon(|_| { 138 | let mut rng = thread_rng(); 139 | let mut a: usize = random_usize(&mut rng); 140 | let theft = random_usize(&mut rng); 141 | rt_ointer!(a, &mut a as &mut usize, theft, usize, 0); 142 | #[cfg(target_pointer_width = "64")] 143 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 1); 144 | #[cfg(target_pointer_width = "64")] 145 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 2); 146 | #[cfg(target_pointer_width = "64")] 147 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 3); 148 | #[cfg(target_pointer_width = "64")] 149 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 4); 150 | #[cfg(target_pointer_width = "64")] 151 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 5); 152 | #[cfg(target_pointer_width = "64")] 153 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 6); 154 | #[cfg(target_pointer_width = "64")] 155 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 7); 156 | #[cfg(target_pointer_width = "64")] 157 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 8); 158 | #[cfg(target_pointer_width = "64")] 159 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 9); 160 | #[cfg(target_pointer_width = "64")] 161 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 10); 162 | #[cfg(target_pointer_width = "64")] 163 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 11); 164 | #[cfg(target_pointer_width = "64")] 165 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 12); 166 | #[cfg(target_pointer_width = "64")] 167 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 13); 168 | #[cfg(target_pointer_width = "64")] 169 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 14); 170 | #[cfg(target_pointer_width = "64")] 171 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 15); 172 | #[cfg(target_pointer_width = "64")] 173 | rt_ointer!(a, &mut a as *mut usize, theft, usize, 16); 174 | }) 175 | } 176 | 177 | #[test] 178 | fn round_trip_ointer_boxes() { 179 | miri_or_rayon(|_| { 180 | let mut rng = thread_rng(); 181 | let a = Box::into_raw(Box::new(random_usize(&mut rng))); 182 | let theft = random_usize(&mut rng); 183 | unsafe { 184 | rt_ointer!(a.read(), a, theft, usize, 0); 185 | #[cfg(target_pointer_width = "64")] 186 | rt_ointer!(a.read(), a, theft, usize, 1); 187 | #[cfg(target_pointer_width = "64")] 188 | rt_ointer!(a.read(), a, theft, usize, 2); 189 | #[cfg(target_pointer_width = "64")] 190 | rt_ointer!(a.read(), a, theft, usize, 3); 191 | #[cfg(target_pointer_width = "64")] 192 | rt_ointer!(a.read(), a, theft, usize, 4); 193 | #[cfg(target_pointer_width = "64")] 194 | rt_ointer!(a.read(), a, theft, usize, 5); 195 | #[cfg(target_pointer_width = "64")] 196 | rt_ointer!(a.read(), a, theft, usize, 6); 197 | #[cfg(target_pointer_width = "64")] 198 | rt_ointer!(a.read(), a, theft, usize, 7); 199 | #[cfg(target_pointer_width = "64")] 200 | rt_ointer!(a.read(), a, theft, usize, 8); 201 | #[cfg(target_pointer_width = "64")] 202 | rt_ointer!(a.read(), a, theft, usize, 9); 203 | #[cfg(target_pointer_width = "64")] 204 | rt_ointer!(a.read(), a, theft, usize, 10); 205 | #[cfg(target_pointer_width = "64")] 206 | rt_ointer!(a.read(), a, theft, usize, 11); 207 | #[cfg(target_pointer_width = "64")] 208 | rt_ointer!(a.read(), a, theft, usize, 12); 209 | #[cfg(target_pointer_width = "64")] 210 | rt_ointer!(a.read(), a, theft, usize, 13); 211 | #[cfg(target_pointer_width = "64")] 212 | rt_ointer!(a.read(), a, theft, usize, 14); 213 | #[cfg(target_pointer_width = "64")] 214 | rt_ointer!(a.read(), a, theft, usize, 15); 215 | #[cfg(target_pointer_width = "64")] 216 | rt_ointer!(a.read(), a, theft, usize, 16); 217 | drop(Box::from_raw(a)); 218 | } 219 | }) 220 | } 221 | 222 | #[test] 223 | fn round_trip_not_null_locals() { 224 | miri_or_rayon(|_| { 225 | let mut rng = thread_rng(); 226 | let mut a: usize = random_usize(&mut rng); 227 | let b = unsafe { NonNull::new_unchecked(&mut a as *mut usize) }; 228 | let theft = random_usize(&mut rng); 229 | rt_not_null!(a, b, theft, usize, 0); 230 | #[cfg(target_pointer_width = "64")] 231 | rt_not_null!(a, b, theft, usize, 1); 232 | #[cfg(target_pointer_width = "64")] 233 | rt_not_null!(a, b, theft, usize, 2); 234 | #[cfg(target_pointer_width = "64")] 235 | rt_not_null!(a, b, theft, usize, 3); 236 | #[cfg(target_pointer_width = "64")] 237 | rt_not_null!(a, b, theft, usize, 4); 238 | #[cfg(target_pointer_width = "64")] 239 | rt_not_null!(a, b, theft, usize, 5); 240 | #[cfg(target_pointer_width = "64")] 241 | rt_not_null!(a, b, theft, usize, 6); 242 | #[cfg(target_pointer_width = "64")] 243 | rt_not_null!(a, b, theft, usize, 7); 244 | #[cfg(target_pointer_width = "64")] 245 | rt_not_null!(a, b, theft, usize, 8); 246 | #[cfg(target_pointer_width = "64")] 247 | rt_not_null!(a, b, theft, usize, 9); 248 | #[cfg(target_pointer_width = "64")] 249 | rt_not_null!(a, b, theft, usize, 10); 250 | #[cfg(target_pointer_width = "64")] 251 | rt_not_null!(a, b, theft, usize, 11); 252 | #[cfg(target_pointer_width = "64")] 253 | rt_not_null!(a, b, theft, usize, 12); 254 | #[cfg(target_pointer_width = "64")] 255 | rt_not_null!(a, b, theft, usize, 13); 256 | #[cfg(target_pointer_width = "64")] 257 | rt_not_null!(a, b, theft, usize, 14); 258 | #[cfg(target_pointer_width = "64")] 259 | rt_not_null!(a, b, theft, usize, 15); 260 | #[cfg(target_pointer_width = "64")] 261 | rt_not_null!(a, b, theft, usize, 16); 262 | }) 263 | } 264 | 265 | #[test] 266 | fn round_trip_not_null_boxes() { 267 | miri_or_rayon(|_| { 268 | let mut rng = thread_rng(); 269 | let a = Box::into_raw(Box::new(random_usize(&mut rng))); 270 | let b = unsafe { NonNull::new_unchecked(a) }; 271 | let theft = random_usize(&mut rng); 272 | unsafe { 273 | rt_not_null!(a.read(), b, theft, usize, 0); 274 | #[cfg(target_pointer_width = "64")] 275 | rt_not_null!(a.read(), b, theft, usize, 1); 276 | #[cfg(target_pointer_width = "64")] 277 | rt_not_null!(a.read(), b, theft, usize, 2); 278 | #[cfg(target_pointer_width = "64")] 279 | rt_not_null!(a.read(), b, theft, usize, 3); 280 | #[cfg(target_pointer_width = "64")] 281 | rt_not_null!(a.read(), b, theft, usize, 4); 282 | #[cfg(target_pointer_width = "64")] 283 | rt_not_null!(a.read(), b, theft, usize, 5); 284 | #[cfg(target_pointer_width = "64")] 285 | rt_not_null!(a.read(), b, theft, usize, 6); 286 | #[cfg(target_pointer_width = "64")] 287 | rt_not_null!(a.read(), b, theft, usize, 7); 288 | #[cfg(target_pointer_width = "64")] 289 | rt_not_null!(a.read(), b, theft, usize, 8); 290 | #[cfg(target_pointer_width = "64")] 291 | rt_not_null!(a.read(), b, theft, usize, 9); 292 | #[cfg(target_pointer_width = "64")] 293 | rt_not_null!(a.read(), b, theft, usize, 10); 294 | #[cfg(target_pointer_width = "64")] 295 | rt_not_null!(a.read(), b, theft, usize, 11); 296 | #[cfg(target_pointer_width = "64")] 297 | rt_not_null!(a.read(), b, theft, usize, 12); 298 | #[cfg(target_pointer_width = "64")] 299 | rt_not_null!(a.read(), b, theft, usize, 13); 300 | #[cfg(target_pointer_width = "64")] 301 | rt_not_null!(a.read(), b, theft, usize, 14); 302 | #[cfg(target_pointer_width = "64")] 303 | rt_not_null!(a.read(), b, theft, usize, 15); 304 | #[cfg(target_pointer_width = "64")] 305 | rt_not_null!(a.read(), b, theft, usize, 16); 306 | drop(Box::from_raw(a)); 307 | } 308 | }) 309 | } 310 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | 203 | 204 | ---- LLVM Exceptions to the Apache 2.0 License ---- 205 | 206 | As an exception, if, as a result of your compiling your source code, portions 207 | of this Software are embedded into an Object form of such source code, you 208 | may redistribute such embedded portions in such Object form without complying 209 | with the conditions of Sections 4(a), 4(b) and 4(d) of the License. 210 | 211 | In addition, if you combine or link compiled forms of this Software with 212 | software that is licensed under the GPLv2 ("Combined Software") and if a 213 | court of competent jurisdiction determines that the patent provision (Section 214 | 3), the indemnity provision (Section 9) or other Section of the License 215 | conflicts with the conditions of the GPLv2, you may retroactively and 216 | prospectively choose to deem waived or otherwise exclude such Section(s) of 217 | the License, but only in their entirety and only with respect to the Combined 218 | Software. 219 | -------------------------------------------------------------------------------- /src/lib.rs: -------------------------------------------------------------------------------- 1 | //! What do you call a pointer we stole the high bits off? An ointer. 2 | //! 3 | //! Ointers is a library for representing pointers where some bits have 4 | //! been stolen so that they may be used by the programmer for something 5 | //! else. In effect, it's a small amount of free storage 6 | //! 7 | //! Fully supports no_std, dependency-free. 8 | //! 9 | //! Ointers supports a handful of bit sources. It's up to you to determine 10 | //! when it is safe to use them. 11 | //! 12 | //! ## Alignment bits (const parameter A) 13 | //! 14 | //! If we know that a pointer's address will always be aligned to `A` 15 | //! bytes where A > 1, we can steal log2(A-1) bits. For an 8-byte 16 | //! aligned value, this provides a modest 3 bits. 17 | //! 18 | //! If you have values aligned to some larger width, you could get even 19 | //! more. It's common in parallel programming to pad data to a cache line 20 | //! by increasing its alignment requirements in order eliminate false 21 | //! sharing. Because a cache line on amd64 or aarch64 is effectively 128 22 | //! bytes thanks to prefetching, you can reclaim a respectable 7 extra 23 | //! bits. 24 | //! 25 | //! If your data is aligned wider still, the sky is the limit, but you 26 | //! could get an incredible 24 bits if you have 16MB-aligned data! 27 | //! Remember that the only alignment rust knows about is what is 28 | //! declared for the type, so you must create a newtype wrapper to 29 | //! take full advantage of large alignment sizes. 30 | //! 31 | //! | Bits | Min alignment | 32 | //! |------|---------------| 33 | //! | 1 | 2b | 34 | //! | 2 | 4b | 35 | //! | 3 | 8b | 36 | //! | 4 | 16b | 37 | //! | 5 | 32b | 38 | //! | 6 | 64b | 39 | //! | 7 | 128b | 40 | //! | 8 | 256b | 41 | //! | 9 | 512b | 42 | //! | 10 | 1k | 43 | //! | 11 | 2k | 44 | //! | 12 | 4k | 45 | //! | 13 | 8k | 46 | //! | 14 | 16k | 47 | //! | 15 | 32k | 48 | //! | 16 | 64k | 49 | //! | 17 | 128k | 50 | //! | 18 | 256k | 51 | //! | 19 | 512k | 52 | //! | 20 | 1m | 53 | //! | 21 | 2m | 54 | //! | 22 | 4m | 55 | //! | 23 | 8m | 56 | //! | 24 | 16m | 57 | //! 58 | //! Stealing bits from alignment is relatively innocuous, but we can only 59 | //! get the compiler to check it for you in dev mode as things stand in 60 | //! rust today. 61 | //! 62 | //! ## Sign bit (parameter S) 63 | //! 64 | //! The most commonly used operating systems arrange memory so that the 65 | //! high half of virtual memory space is reserved for the kernel and the 66 | //! low half is given to userspace. 67 | //! 68 | //! Looked at as a signed integer, this makes the kernel half of address 69 | //! space negative and the userspace half positive. 70 | //! 71 | //! Most programs do not deal with kernel addresses, thus giving us an 72 | //! extra bit to play with. 73 | //! 74 | //! We can also get this extra bit in kernel mode if we know we will not 75 | //! be dealing with userspace addresses. We do this by taking a pointer to 76 | //! a value on the stack and stealing its sign bit. 77 | //! 78 | //! If you know you will be dealing with userspace addresses in kernel 79 | //! space or kernel space addresses in userspace, or you are using or 80 | //! implementing a kernel which does not follow this convention, you must 81 | //! set `S` to `false`. 82 | //! 83 | //! The S bit is currently only tested with userspace pointers in 84 | //! userspace. While we think it should work more generally, we currently 85 | //! haven't got a test rig for other scenarios so we can't promise it does. 86 | //! 87 | //! ## Unused virtual address space (V) 88 | //! 89 | //! In 64-bit mode, address space is plentiful: nobody has 64 bits' worth 90 | //! of RAM and even if they did, their processor is unable to address it 91 | //! all. V is required to be 0 unless compiling for a 64bit target. 92 | //! 93 | //! The number of bits that may be safely stolen by this method depends 94 | //! upon the microarchitecture in question and the page table depth. 95 | //! 96 | //! For x86-64 and aarch64, the following sizes apply: 97 | //! 98 | //! | Bits | PT depth | Support | 99 | //! |------|----------|-----------------------------------| 100 | //! | 25 | 3 | aarch64 only, uncommon, opt-in | 101 | //! | 16 | 4 | most common default | 102 | //! | 7 | 5 | some intel only, uncommon, opt-in | 103 | //! 104 | //! If you are made of money and need more than 128TB virtual address 105 | //! space, you should limit yourself to 7 bits for V. Likewise if you know 106 | //! you'll be on 3-deep page tables, you can steal a whopping 25 bits. But 107 | //! you're probably limited to 16 bits in general. 108 | #![no_std] 109 | #![allow(unstable_name_collisions)] 110 | 111 | #[cfg(feature = "alloc")] 112 | extern crate alloc; 113 | #[cfg(feature = "alloc")] 114 | use alloc::boxed::Box; 115 | 116 | use core::hash::*; 117 | #[cfg(feature = "alloc")] 118 | use core::ops::{Deref, DerefMut}; 119 | use core::ptr::NonNull; 120 | 121 | /// A pointer we stole the high bits off 122 | /// 123 | /// T: type pointed to. 124 | /// A: number of bits to steal based on the alignment requirements of T. 125 | /// S: whether to steal the sign bit. 126 | /// V: number of bits to steal from unused virtual address space. 127 | 128 | #[derive(Debug)] 129 | #[repr(transparent)] 130 | pub struct Ointer { 131 | ptr: *mut T, 132 | } 133 | 134 | impl Hash for Ointer { 135 | fn hash(&self, state: &mut H) { 136 | self.ptr.hash(state) 137 | } 138 | } 139 | 140 | impl PartialEq for Ointer { 141 | #![allow( 142 | ambiguous_wide_pointer_comparisons, 143 | reason = "mirroring the behaviour of rust ptrs" 144 | )] 145 | fn eq(&self, other: &Self) -> bool { 146 | self.ptr == other.ptr 147 | } 148 | } 149 | 150 | impl Eq for Ointer {} 151 | 152 | impl Clone for Ointer { 153 | fn clone(&self) -> Self { 154 | *self 155 | } 156 | } 157 | 158 | impl Copy for Ointer {} 159 | 160 | impl Ointer { 161 | /// Creates a new Ointer from a presumed legitimate pointer. 162 | /// 163 | /// # Safety 164 | /// 165 | /// * T's alignment must enable stealing A bits. 166 | /// * The high bits (sign upwards) must match a stack pointer's high bits. 167 | /// * If compiling for a 64bit arch, V must be at most 25. 168 | /// * If compiling for a non-64bit arch, V must be 0. 169 | /// 170 | /// These invariants are checked with `debug_assert` only, hence 171 | /// `unsafe`. The usual caveats of pointers apply. 172 | pub unsafe fn new(ptr: *mut T) -> Self { 173 | Ointer { 174 | ptr: pack(ptr, A, S, V), 175 | } 176 | } 177 | 178 | /// Constructor that enables stealing bits. 179 | /// 180 | /// # Safety 181 | /// 182 | /// Same as `new` 183 | pub unsafe fn new_stealing(ptr: *mut T, bits: usize) -> Self { 184 | let mask = asv_mask(A, S, V); 185 | let ptr = ptr.with_addr((bits & mask) | (ptr.addr() & !mask)); 186 | Self { ptr } 187 | } 188 | 189 | /// Returns the stolen bits in the high pos. 190 | pub fn stolen(self) -> usize { 191 | self.ptr.addr() & asv_mask(A, S, V) 192 | } 193 | 194 | /// Takes a value from the high bits of the provided usize and 195 | /// steals them from the ointer. 196 | pub fn steal(self, bits: usize) -> Self { 197 | let mask = asv_mask(A, S, V); 198 | let ptr = self 199 | .ptr 200 | .with_addr((bits & mask) | (self.ptr.addr() & !mask)); 201 | Self { ptr } 202 | } 203 | 204 | /// Get the pointer without the stolen bits 205 | pub fn as_ptr(self) -> *mut T { 206 | unsafe { unpack(self.ptr, A, S, V) } 207 | } 208 | 209 | /// Direct access to the underlying data. The pointer it returns 210 | /// may not be valid. 211 | pub fn raw(self) -> usize { 212 | self.ptr.expose_provenance() 213 | } 214 | } 215 | 216 | /// A non-null pointer that we stole the high bits off. 217 | /// 218 | /// T: type pointed to. 219 | /// V: number of bits to steal directly by masking them off. 220 | /// A: number of bits to steal based on the alignment requirements of T. 221 | #[derive(Debug)] 222 | #[repr(transparent)] 223 | pub struct NotNull(NonNull); 224 | 225 | impl Clone for NotNull { 226 | fn clone(&self) -> Self { 227 | *self 228 | } 229 | } 230 | 231 | impl Copy for NotNull {} 232 | 233 | impl PartialEq for NotNull { 234 | #![allow( 235 | ambiguous_wide_pointer_comparisons, 236 | reason = "mirroring the behaviour of NonNull" 237 | )] 238 | fn eq(&self, other: &Self) -> bool { 239 | self.0 == other.0 240 | } 241 | } 242 | 243 | impl Eq for NotNull {} 244 | 245 | impl Hash for NotNull { 246 | fn hash(&self, state: &mut H) { 247 | self.0.hash(state) 248 | } 249 | } 250 | 251 | impl NotNull { 252 | /// Creates a new Ointer from a presumed legitimate pointer. 253 | /// 254 | /// # Safety 255 | /// 256 | /// * T's alignment must enable stealing A bits. 257 | /// * The high bits (sign upwards) must match a stack pointer's high bits. 258 | /// * If compiling for a 64bit arch, V must be at most 25. 259 | /// * If compiling for a non-64bit arch, V must be 0. 260 | /// 261 | /// These invariants are checked with `debug_assert` only, hence 262 | /// `unsafe`. The usual caveats of pointers apply. 263 | pub unsafe fn new(ptr: NonNull) -> Self { 264 | let ptr = pack(ptr.as_ptr(), A, S, V); 265 | NotNull(NonNull::new_unchecked(ptr)) 266 | } 267 | 268 | /// Constructor that enables stealing bits. 269 | /// 270 | /// # Safety 271 | /// 272 | /// Same as `new` 273 | pub unsafe fn new_stealing(ptr: NonNull, bits: usize) -> Self { 274 | let mask = asv_mask(A, S, V); 275 | let ptr = ptr.as_ptr().map_addr(|addr| (bits & mask) | (addr & !mask)); 276 | NotNull(NonNull::new_unchecked(ptr)) 277 | } 278 | 279 | /// Returns the stolen bits in the high pos. 280 | pub fn stolen(self) -> usize { 281 | self.0.as_ptr().addr() & asv_mask(A, S, V) 282 | } 283 | 284 | /// Takes a value from the high bits of the provided usize and 285 | /// steals them from the ointer. 286 | pub fn steal(self, bits: usize) -> Self { 287 | let mask = asv_mask(A, S, V); 288 | let bits = bits & mask; 289 | let ptr = self.0.as_ptr(); 290 | let addr = ptr.addr() & !mask; 291 | Self(unsafe { NonNull::new_unchecked(ptr.with_addr(addr | bits)) }) 292 | } 293 | 294 | /// Get the pointer without the stolen bits 295 | pub fn as_non_null(self) -> NonNull { 296 | unsafe { NonNull::new_unchecked(unpack(self.0.as_ptr(), A, S, V)) } 297 | } 298 | 299 | /// Direct access to the underlying data. The pointer it returns 300 | /// may not be valid. 301 | pub fn raw(self) -> usize { 302 | self.0.as_ptr().expose_provenance() 303 | } 304 | } 305 | 306 | /// A Box that we stole the high bits off. 307 | /// 308 | /// T: type pointed to. 309 | /// V: number of bits to steal directly by masking them off. 310 | /// A: number of bits to steal based on the alignment requirements of T. 311 | #[derive(Debug)] 312 | #[repr(transparent)] 313 | #[cfg(feature = "alloc")] 314 | pub struct Ox(NotNull); 315 | 316 | #[cfg(feature = "alloc")] 317 | impl Clone for Ox 318 | where 319 | Box: Clone, 320 | { 321 | fn clone(&self) -> Self { 322 | // Safety: even though we have mutable access to the ptr, we don't use it as it is ManuallyDrop 323 | let old = core::mem::ManuallyDrop::new(unsafe { Box::from_raw(self.as_ptr()) }); 324 | let boxed = core::mem::ManuallyDrop::into_inner(old.clone()); 325 | // Safety: the stolen bits where taken from an existing Ox (thus safe) 326 | unsafe { Ox::new_stealing(boxed, self.stolen()) } 327 | } 328 | } 329 | 330 | #[cfg(feature = "alloc")] 331 | impl PartialEq for Ox { 332 | fn eq(&self, other: &Self) -> bool { 333 | self.0 == other.0 334 | } 335 | } 336 | 337 | #[cfg(feature = "alloc")] 338 | impl Eq for Ox {} 339 | 340 | #[cfg(feature = "alloc")] 341 | impl Hash for Ox { 342 | fn hash(&self, state: &mut H) { 343 | self.0.hash(state) 344 | } 345 | } 346 | 347 | #[cfg(feature = "alloc")] 348 | impl Ox { 349 | /// Creates a new Ox from a box. 350 | /// 351 | /// # Safety 352 | /// 353 | /// * T's alignment must enable stealing A bits. 354 | /// * The high bits (sign upwards) must match a stack pointer's high bits. 355 | /// * If compiling for a 64bit arch, V must be at most 25. 356 | /// * If compiling for a non-64bit arch, V must be 0. 357 | /// 358 | /// These invariants are checked with `debug_assert` only, hence 359 | /// `unsafe`. The usual caveats of pointers apply. 360 | pub unsafe fn new(boxed: Box) -> Self { 361 | let ptr = Box::into_raw(boxed); 362 | let ptr = pack(ptr, A, S, V); 363 | Ox(NotNull::new(NonNull::new_unchecked(ptr))) 364 | } 365 | 366 | /// Constructor that enables stealing bits. 367 | /// 368 | /// # Safety 369 | /// 370 | /// Same as `new` 371 | pub unsafe fn new_stealing(boxed: Box, bits: usize) -> Self { 372 | let mask = asv_mask(A, S, V); 373 | let orig_ptr = Box::into_raw(boxed); 374 | let ptr = orig_ptr.map_addr(|addr| (bits & mask) | (addr & !mask)); 375 | Self(NotNull::new(NonNull::new_unchecked(ptr))) 376 | } 377 | 378 | /// Returns the stolen bits in the high pos. 379 | pub fn stolen(&self) -> usize { 380 | self.0.stolen() 381 | } 382 | 383 | /// Takes a value from the high bits of the provided usize and 384 | /// steals them from the ox. 385 | pub fn steal(&mut self, bits: usize) { 386 | let mask = asv_mask(A, S, V); 387 | let bits = bits & mask; 388 | let ptr = self.as_ptr().map_addr(|addr| (addr & !mask) | bits); 389 | self.0 = unsafe { NotNull::new(NonNull::new_unchecked(ptr)) }; 390 | } 391 | 392 | /// Get the box back without the stolen bits 393 | pub fn into_box(self) -> Box { 394 | unsafe { Box::from_raw(unpack(self.as_ptr(), A, S, V)) } 395 | } 396 | 397 | /// Get the box back without the stolen bits 398 | pub fn as_ptr(&self) -> *mut T { 399 | self.0.as_non_null().as_ptr() 400 | } 401 | 402 | /// Direct access to the underlying data. The pointer it returns 403 | /// may not be valid. 404 | pub fn raw(&self) -> usize { 405 | self.0.as_non_null().as_ptr().expose_provenance() 406 | } 407 | } 408 | 409 | #[cfg(feature = "alloc")] 410 | impl Deref for Ox { 411 | type Target = T; 412 | fn deref(&self) -> &T { 413 | unsafe { self.0.as_non_null().as_ref() } 414 | } 415 | } 416 | 417 | #[cfg(feature = "alloc")] 418 | impl DerefMut for Ox { 419 | fn deref_mut(&mut self) -> &mut T { 420 | unsafe { self.0.as_non_null().as_mut() } 421 | } 422 | } 423 | 424 | #[cfg(feature = "alloc")] 425 | impl Drop for Ox { 426 | fn drop(&mut self) { 427 | drop(unsafe { Box::from_raw(self.0.as_non_null().as_ptr()) }) 428 | } 429 | } 430 | 431 | /// Packs a pointer into the bottom `sizeof(usize) - (a + s + v)` bits of a usize. 432 | /// 433 | /// # Safety 434 | /// 435 | /// * T's alignment must enable stealing A bits. 436 | /// * The high bits (sign upwards) must match a stack pointer's high bits. 437 | /// * If compiling for a 64bit arch, V must be at most 25. 438 | /// * If compiling for a non-64bit arch, V must be 0. 439 | /// 440 | /// These invariants are checked with `debug_assert` only, hence 441 | /// `unsafe`. The usual caveats of pointers apply. 442 | pub unsafe fn pack(ptr: *mut T, a: u8, s: bool, v: u8) -> *mut T { 443 | let sv = asv_mask(0, s, v); 444 | #[cfg(debug_assertions)] 445 | { 446 | if let Some(p) = ptr.as_ref() { 447 | debug_assert!((1 << a) <= core::mem::align_of_val(p)); 448 | } 449 | #[cfg(all( 450 | not(target_pointer_width = "64"), 451 | not(feature = "i_know_what_im_doing") 452 | ))] 453 | debug_assert!(v == 0); 454 | debug_assert!(v <= 25); 455 | // If S is set, the user has indicated they will never be 456 | // dealing with foreign pointers, so we can check that 457 | // too. We need only really check the sign bit because of 458 | // canonicalisation, but it's actually cheaper to check 459 | // all the high bits. 460 | if s { 461 | // We don't want to take account of A yet as the pointer 462 | // is still in its undoctored state. 463 | let ptr = ptr.addr(); 464 | let stack = (&ptr as *const usize).addr(); 465 | // the top bits should match 466 | debug_assert!((sv & ptr) == (sv & stack)); 467 | } 468 | } 469 | ptr.with_addr((ptr.addr() & !sv) >> a as usize) 470 | } 471 | 472 | /// Turns the `sizeof(usize) - (a + s + v)` bits of a usize (as 473 | /// returned from `pack`) back into a pointer. 474 | /// 475 | /// # Safety 476 | /// 477 | /// The pointer must be of the correct type, otherwise you're 478 | /// basically unsafely casting the pointer. 479 | /// 480 | /// You must use the same settings as you packed the pointer with. The 481 | /// pointer must be packed into the lower bits. Not strictly unsafe, 482 | /// but indicative of a bug in your program. 483 | pub unsafe fn unpack(packed: *mut T, a: u8, s: bool, v: u8) -> *mut T { 484 | // Mask off all the stolen bits to get the pointer data. 485 | let asv = asv_mask(a, s, v); 486 | let masked = packed.addr() & !asv; 487 | // Restore the empty alignment bits 488 | let base = masked << a; 489 | if s { 490 | // Copy the top bits of a stack pointer 491 | let sv = asv_mask(0, s, v); 492 | let base = base & !sv; 493 | let stack = (&base as *const usize).addr() & sv; 494 | packed.with_addr(stack | base) 495 | } else { 496 | // We need to extend the sign bit. 497 | packed.with_addr((((base << v as usize) as isize) >> v as usize) as usize) 498 | } 499 | } 500 | 501 | /// Produces a mask where the stolen bits (at the top) are set 502 | pub const fn asv_mask(a: u8, s: bool, v: u8) -> usize { 503 | mask(a + s as u8 + v) 504 | } 505 | 506 | /// Produces a mask where the stolen bits (at the top) are set 507 | pub const fn mask(bits: u8) -> usize { 508 | (isize::MIN >> (max(bits as usize, 1) - 1)) as usize 509 | } 510 | 511 | // core::cmp::max and usize::max aren't const fns 512 | const fn max(x: usize, y: usize) -> usize { 513 | if x <= y { 514 | y 515 | } else { 516 | x 517 | } 518 | } 519 | --------------------------------------------------------------------------------