├── .gitignore ├── LICENCE ├── README.md ├── example ├── example_premake.lua ├── handle_config.cpp ├── handle_config.h ├── main.cpp └── premake.bat ├── handle.h ├── handle.natvis ├── handle_win32.cpp ├── premake5.exe └── test ├── basics.cpp ├── catch └── catch.hpp ├── largeobjects.cpp ├── main.cpp ├── multithreading.cpp ├── premake.bat ├── test_handle_config.cpp ├── test_handle_config.h └── test_premake.lua /.gitignore: -------------------------------------------------------------------------------- 1 | *.vcxproj 2 | *.vcxproj.* 3 | *.sln 4 | .vs 5 | test/bin 6 | test/obj 7 | example/bin 8 | example/obj 9 | enc_temp_folder -------------------------------------------------------------------------------- /LICENCE: -------------------------------------------------------------------------------- 1 | The MIT License 2 | 3 | Copyright (c) 2018 jlaumon 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Build status](https://ci.appveyor.com/api/projects/status/nntont83nb1uk9q2?svg=true)](https://ci.appveyor.com/project/jlaumon/handle) 2 | 3 | # Handle 4 | 5 | Handles (in general) are a way of referencing an object with an integer. It's a little bit like pointers, except the additional indirection 6 | they introduce can be leveraged to implement a way of determining if the object is still valid before accessing it. 7 | 8 | This can be useful for a variety of reasons, but basically, any time controlling who owns pointers to the object is not possible 9 | or too complicated, a handle system could be useful. 10 | 11 | ```c++ 12 | using EntityID = Handle; 13 | 14 | void SpawnEntity(const SpawnData& data) 15 | { 16 | // Create an entity and return a handle to it 17 | EntityID id = EntityID::Create(data); 18 | 19 | QueueEntityForProcessing(id); 20 | } 21 | 22 | ... 23 | 24 | // Executed potentially much later 25 | void ProcessEntity(EntityID id) 26 | { 27 | if (Entity* entity = EntityID::Get(id)) 28 | { 29 | entity->process(); 30 | } 31 | else 32 | { 33 | // Oh no, the entity was destroyed in the meantime 34 | goHaveADrink(); 35 | } 36 | } 37 | ``` 38 | A more complete example is available [in the example folder](https://github.com/jlaumon/handle/blob/master/example/main.cpp). 39 | 40 | ## Under the hood 41 | 42 | Each handle is made of a number of bits used as an index in an array (that's where the referenced object is), 43 | and some bits acting as a version number. If the version number in the handle matches the one in the array at the same index, the object is valid. 44 | Otherwise it means the object has been destroyed. This technique is also sometimes called generational indices (where the "generation" is the version). 45 | 46 | Now, that's also pretty much what all the other handle implementations do. Here are the advantages of this one: 47 | 48 | ### It's not so opaque 49 | 50 | The provided **natvis** files mean that - if you use Visual Studio - you can see the value behind a handle at anytime in the debugger. 51 | 52 | ![natvis0](https://user-images.githubusercontent.com/2878094/41821247-fba2833a-77dd-11e8-993c-e883f7e146bf.PNG) 53 | 54 | ### It's resizable AND thread-safe 55 | 56 | One very common limitation of this kind of handle system, is that it needs to allocate the array where the objects 57 | are stored during initialization. And to be able to resize this array means you have to be sure no one is reading it, 58 | so it's usually either not thread-safe, or not resizable (or you'd need to lock every time you access any object, but that's not very appealing). 59 | 60 | This implementation uses virtual memory to reserve enough address space to store all the objects you could fit the index bits of the handle, 61 | but only commits the memory that you need to store the current number of objects, and can commit more as needed. It never shrinks though. 62 | 63 | Accessing an object from a handle is lock-free, it doesn't need any synchronization since growing the array does not move existing objects. 64 | It's also very fast since it's just indexing an array. 65 | Creating/destroying handles does use locks however, but they are short enough. 66 | 67 | ### It's stongly typed 68 | 69 | Handles are not typedefs to integers, they are a class, which is great for type-safety. 70 | 71 | The `Tag` template parameter also means you can have two handles to the same object type behave as different types. 72 | 73 | ```c++ 74 | using TextureID = Handle; 75 | using DebugTextureID = Handle; // provided that Debug is a type 76 | // Can't pass a TextureID to a function taking a DebugTextureID. 77 | ``` 78 | 79 | ### It's customizable 80 | 81 | The template parameters can be used to choose the type of integer to use, and the number of bits for the index/version 82 | (although kind of indirectly: `MaxHandles` will influence the number of index bits, and the version will use all the 83 | remaining bits in the choosen integer type). 84 | 85 | ```c++ 86 | using ObjectHandle = Handle; 87 | // ObjectHandles take 2 bytes (they're uint16_t) and there can be only 512 hanles in flight (which means 9 bits of index and 7 bits of version) 88 | ``` 89 | -------------------------------------------------------------------------------- /example/example_premake.lua: -------------------------------------------------------------------------------- 1 | solution "HandleExample" 2 | 3 | platforms { "x64" } 4 | configurations { "Debug", "Release" } 5 | startproject "HandleExample" 6 | 7 | project "HandleExample" 8 | 9 | kind "ConsoleApp" 10 | 11 | files 12 | { 13 | "../*.h", 14 | "../*.cpp", 15 | "../*.natvis", 16 | "**.h", 17 | "**.hpp", 18 | "**.cpp", 19 | } 20 | 21 | includedirs 22 | { 23 | "..", 24 | ".", 25 | } 26 | 27 | defines 28 | { 29 | "HDL_USER_CONFIG=\"handle_config.h\"" 30 | } 31 | 32 | vpaths 33 | { 34 | ["*"] = "../*", 35 | } 36 | -------------------------------------------------------------------------------- /example/handle_config.cpp: -------------------------------------------------------------------------------- 1 | #include "handle_config.h" 2 | #include 3 | #include 4 | 5 | void HandleAssertionFailed(const char* _condition, const char* _file, int _line) 6 | { 7 | fprintf(stderr, 8 | "Assertion failed: %s\n" 9 | "Source: %s, line %d\n", 10 | _condition, _file, _line 11 | ); 12 | } 13 | 14 | void HandleAssertionFailed(const char* _condition, const char* _file, int _line, const char* _msg, ...) 15 | { 16 | va_list args; 17 | va_start(args, _msg); 18 | 19 | char buffer[1024]; 20 | vsnprintf(buffer, sizeof(buffer), _msg, args); 21 | buffer[sizeof(buffer) - 1] = 0; 22 | 23 | va_end(args); 24 | 25 | fprintf(stderr, 26 | "Assertion failed: %s\n" 27 | "Source: %s, line %d\n" 28 | "Message: %s\n", 29 | _condition, _file, _line, buffer 30 | ); 31 | } 32 | -------------------------------------------------------------------------------- /example/handle_config.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | // This is an example of assert macro for Windows. You should replace it with your own. 4 | #define HDL_ASSERT(condition, ...) \ 5 | if (!(condition)) { \ 6 | HandleAssertionFailed(#condition, __FILE__, __LINE__, ##__VA_ARGS__); \ 7 | __debugbreak(); \ 8 | } 9 | 10 | void HandleAssertionFailed(const char* _condition, const char* _file, int _line); 11 | void HandleAssertionFailed(const char* _condition, const char* _file, int _line, const char* msg, ...); 12 | -------------------------------------------------------------------------------- /example/main.cpp: -------------------------------------------------------------------------------- 1 | #include "handle.h" 2 | #include 3 | 4 | // Just a dummy texture class. 5 | class Texture 6 | { 7 | public: 8 | Texture(const char* _path) { m_path = _path; } 9 | private: 10 | std::string m_path; 11 | }; 12 | 13 | int main(int argc, const char* argv[]) 14 | { 15 | // Let's declare a nice name for our new handle type. 16 | // The using is not mandatory, but it is convenient to keep the name short 17 | // (especially if you don't use the default template param values). 18 | using TextureID = Handle; 19 | 20 | // Now, let's create a texture. It will be allocated inside the Handle pool, 21 | // which provides fast and almost contiguous allocations. 22 | TextureID helloTexID = TextureID::Create("hello_world.png"); 23 | // At this point, if you break into Visual Studio's debugger, you should be able to see 24 | // the texture variable when inspecting helloTexID. 25 | HDL_ASSERT(helloTexID != TextureID::kInvalid); 26 | 27 | // To get the texture from the handle, just call get. 28 | Texture* helloTex = TextureID::Get(helloTexID); 29 | HDL_ASSERT(helloTex); 30 | 31 | // And now let's destroy it! 32 | TextureID::Destroy(helloTexID); 33 | 34 | // At this point, the texture was destroyed and the handle has become invalid, 35 | // it is not possible to get a pointer to the texture anymore. 36 | Texture* helloAgain = TextureID::Get(helloTexID); 37 | HDL_ASSERT(helloAgain == nullptr); 38 | 39 | // The value of helloTexID may be re-used later, but not until all the other possible handle values have been used first. 40 | // In this example where TextureID is 32 bits, that's ~4 billions calls to TextureID::Create later, so that should be fine. 41 | // But you can use 64 bits handles if you want. 42 | for (int i = 0; i < 100; ++i) 43 | { 44 | auto id = TextureID::Create("test"); 45 | HDL_ASSERT(id != helloTexID); 46 | } 47 | } -------------------------------------------------------------------------------- /example/premake.bat: -------------------------------------------------------------------------------- 1 | ..\premake5 --file=example_premake.lua vs2017 -------------------------------------------------------------------------------- /handle.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include // std::is_integral/std::is_unsigned/std::forward 4 | 5 | #ifdef HDL_USER_CONFIG 6 | #include HDL_USER_CONFIG 7 | #endif 8 | 9 | #ifndef HDL_ASSERT 10 | #include 11 | #define HDL_ASSERT(condition, ...) assert(condition) 12 | #endif 13 | 14 | #ifndef HDL_DEQUE 15 | #include // VC++ has a very bad deque implementation, prefer switching to eastl::deque or a custom FIFO that does not allocate each elements separately 16 | #define HDL_DEQUE std::deque 17 | #endif 18 | 19 | #ifndef HDL_MUTEX 20 | #include 21 | #define HDL_MUTEX std::mutex 22 | #endif 23 | 24 | template class HandlePool; 25 | 26 | template 30 | class Handle 31 | { 32 | public: 33 | typedef Handle this_type; 34 | typedef IntegerType integer_type; ///< The type of the (unsigned) integer inside the handle. 35 | typedef HandlePool pool_type; ///< The type of the pool managing the elements/handles. 36 | 37 | static const integer_type kInvalid = pool_type::kInvalid; ///< Special value reserved for indicating an invalid handle. 38 | 39 | /// Creates an instance of T and a handle for it. Parameters are forwarded to the element's constructor. 40 | /// @returns The handle pointing to the created element, or kInvalid if the allocation failed (MaxHandles reached or out-of-memory). 41 | template 42 | static this_type Create (Args&&... _args) { return this_type(s_pool.create(std::forward(_args)...)); } 43 | /// Destroys this handle and the pointed element. 44 | /// @returns True if the destruction happened, or false if the handle was not valid (eg. already destroyed). 45 | static bool Destroy (this_type _handle) { return s_pool.destroy(_handle); } 46 | /// Gets the element pointed by the handle. 47 | /// @returns The pointer to the element, or nullptr if the handle was not valid. 48 | static T* Get (this_type _handle) { return s_pool.get(_handle); } 49 | 50 | /// Returns the current number of elements/handles. 51 | static size_t Size () { return s_pool.size(); } 52 | /// Returns the number of elements/handles that can be held in the currently allocated storage. 53 | static size_t Capacity() { return s_pool.capacity(); } 54 | /// Returns the maximum possible number of elements/handles (ie. MaxHandles). 55 | static size_t MaxSize () { return s_pool.max_size(); } 56 | 57 | /// Reserves storage for at least `_newCap` number of elements/handles. 58 | /// @returns The reserve operation success (can fail if _newCap is greater than MaxHandles or if out-of-memory). 59 | static bool Reserve (size_t _newCap) { return s_pool.reserve(_newCap); } 60 | 61 | /// Destoys all the elements, release all the memory. 62 | static void Reset (); 63 | 64 | Handle() : m_intVal(kInvalid) {} 65 | Handle(const this_type& _handle) : m_intVal(_handle.m_intVal) {} 66 | explicit Handle(integer_type _intVal) : m_intVal(_intVal) {} 67 | operator integer_type() const { return m_intVal; } 68 | this_type& operator=(integer_type _intVal) { m_intVal = _intVal; return *this; } 69 | this_type& operator=(const this_type& _handle) { m_intVal = _handle.m_intVal; return *this; } 70 | 71 | private: 72 | integer_type m_intVal; 73 | 74 | static pool_type s_pool; 75 | }; 76 | 77 | namespace HDL 78 | { 79 | namespace VirtualMemory 80 | { 81 | size_t GetPageSize(); 82 | /// Reserves a memory area of at least _size bytes. The memory needs to be committed before being used. 83 | void* Reserve (size_t _size); 84 | /// Releases reserved memory. Also decommits any part that was committed. 85 | /// _address and _size must match the values returned by/passed to the Reserve call that reserved this memory area. 86 | void Release (void* _address, size_t _size); 87 | /// Commits reserved memory. All newly allocated pages will contain zeros. 88 | /// All the pages containing at least one byte in the range _address, _address + _size will be committed. 89 | /// @returns Commit success. 90 | bool Commit (void* _address, size_t _size); 91 | /// Frees committed memory. 92 | /// All the pages containing at least one byte in the range _address, _address + _size will be decommitted. 93 | void Decommit(void* _address, size_t _size); 94 | } 95 | } 96 | 97 | template 98 | void Handle::Reset() 99 | { 100 | // Call the destructor/constructor explicitely to destroy and recreate the pool 101 | s_pool.~HandlePool(); 102 | new (&s_pool) HandlePool(); 103 | } 104 | 105 | template 106 | HandlePool Handle::s_pool; 107 | 108 | template 109 | class HandlePool 110 | { 111 | public: 112 | typedef HandlePool this_type; 113 | typedef IntegerType integer_type; 114 | static const integer_type kInvalid = ~0; 115 | 116 | HandlePool() = default; 117 | ~HandlePool(); 118 | 119 | HandlePool(const this_type&) = delete; 120 | this_type& operator= (this_type&) = delete; 121 | 122 | template 123 | integer_type create (Args&&... _args); 124 | bool destroy (integer_type _handle); 125 | T* get (integer_type _handle); 126 | 127 | size_t size () const { return m_handleCount; } 128 | size_t capacity() const { return MinSizeT(m_nodeBufferCapacityBytes / sizeof(Node), kMaxHandles); } 129 | size_t max_size() const { return kMaxHandles; } 130 | 131 | bool reserve (size_t _newCap); 132 | 133 | static constexpr size_t MinSizeT(size_t _a, size_t _b) { return _a < _b ? _a : _b; } // Don't want to include just for std::min 134 | static constexpr size_t CeilLog2(size_t _x) { return _x < 2 ? 1 : 1 + CeilLog2(_x >> 1); } 135 | 136 | static const size_t kMaxHandles = MaxHandles; 137 | static const size_t kIndexNumBits = CeilLog2(MaxHandles - 1); 138 | static const size_t kIndexMask = ((size_t)1 << kIndexNumBits) - 1; 139 | static const size_t kVersionNumBits = MinSizeT(sizeof(IntegerType) * 8 - kIndexNumBits, sizeof(size_t) * 8 - 1); // The version is stored in a size_t minus 1 bit below. 140 | static const size_t kVersionMask = ((size_t)1 << kVersionNumBits) - 1; 141 | 142 | static_assert(std::is_integral::value && std::is_unsigned::value, "IntegerType must be an unsigned integer type."); 143 | static_assert(kIndexNumBits < sizeof(IntegerType) * 8, "There are not enough bits in IntegerType to store both index and version."); 144 | 145 | // Choose the smallest index type that can fit the wanted number of bits. 146 | typedef typename std::conditional< kIndexNumBits <= 16, uint16_t, 147 | typename std::conditional::type >::type index_type; 148 | 149 | static index_type GetIndex (integer_type _handle); 150 | static size_t GetVersion(integer_type _handle); 151 | static integer_type GetID (index_type _index, size_t _version); 152 | 153 | private: 154 | struct LockGuard 155 | { 156 | HDL_MUTEX& m_mutex; 157 | LockGuard(HDL_MUTEX& _mutex) : m_mutex(_mutex) { m_mutex.lock(); } 158 | ~LockGuard() { m_mutex.unlock(); } 159 | LockGuard& operator=(LockGuard) = delete; 160 | }; 161 | 162 | size_t getNodeBufferSize() const; 163 | bool reserveNoLock(size_t _newCap); 164 | 165 | struct Node 166 | { 167 | size_t m_allocated : 1; 168 | size_t m_version : sizeof(size_t) * 8 - 1; // = 63 bits on 64 bits systems. 169 | T m_value; 170 | }; 171 | 172 | // The max value m_nodeBufferSizeBytes can take to keep its indexable with kIndexNumBits 173 | static const size_t kNodeBufferMaxSizeBytes = (1 << kIndexNumBits) * sizeof(Node); 174 | 175 | Node* m_nodeBuffer = nullptr; 176 | size_t m_nodeBufferSizeBytes = 0; 177 | size_t m_nodeBufferCapacityBytes = 0; 178 | size_t m_handleCount = 0; 179 | HDL_DEQUE m_freeIndices; 180 | HDL_MUTEX m_mutex; 181 | }; 182 | 183 | template 184 | HandlePool::~HandlePool() 185 | { 186 | // Destroy all the allocated nodes 187 | size_t nodeCount = getNodeBufferSize(); 188 | for (size_t i = 0; i < nodeCount; ++i) 189 | { 190 | auto node = m_nodeBuffer + i; 191 | if (node->m_allocated) 192 | node->m_value.~T(); 193 | } 194 | 195 | // Release the reserved memory 196 | if (m_nodeBuffer) 197 | HDL::VirtualMemory::Release(m_nodeBuffer, kMaxHandles * sizeof(Node)); 198 | } 199 | 200 | template 201 | template 202 | IntegerType 203 | HandlePool::create(Args&&... _args) 204 | { 205 | index_type index; 206 | 207 | { 208 | LockGuard guard(m_mutex); 209 | 210 | if (m_handleCount == kMaxHandles) 211 | return kInvalid; 212 | 213 | // If there is enough space in the node buffer, add a node 214 | // Note: use the rest of the buffer before looking for free indices to delay the wrapping of the versions as much as possible 215 | if (m_nodeBufferSizeBytes < kNodeBufferMaxSizeBytes 216 | && (m_nodeBufferSizeBytes + sizeof(Node)) <= m_nodeBufferCapacityBytes) 217 | { 218 | index = (index_type)getNodeBufferSize(); 219 | m_nodeBufferSizeBytes += sizeof(Node); 220 | } 221 | // Otherwise look for free indices 222 | else if (!m_freeIndices.empty()) 223 | { 224 | index = m_freeIndices.front(); 225 | m_freeIndices.pop_front(); 226 | } 227 | // Last option, grow the node buffer 228 | else 229 | { 230 | HDL_ASSERT(m_nodeBufferSizeBytes < kNodeBufferMaxSizeBytes); // At this point, either the freelist should not be empty, 231 | // or we should have reached kMaxHandles and returned kInvalid 232 | 233 | // Increase capacity to store at least one more node. 234 | // FIXME! Not the best idea if nodes are very big, make alloc size customizable? 235 | auto cap = capacity(); 236 | if (!reserveNoLock(cap + 1)) 237 | { 238 | // Reserve failed, probably out-of-memory. 239 | return kInvalid; 240 | } 241 | 242 | index = (index_type)getNodeBufferSize(); 243 | m_nodeBufferSizeBytes += sizeof(Node); 244 | } 245 | 246 | m_handleCount++; 247 | 248 | } // LockGuard end 249 | 250 | auto node = m_nodeBuffer + index; 251 | node->m_allocated = true; 252 | new (&node->m_value) T(std::forward(_args)...); 253 | 254 | return GetID(index, node->m_version); 255 | } 256 | 257 | template 258 | bool 259 | HandlePool::destroy(integer_type _handle) 260 | { 261 | if (_handle == kInvalid) 262 | return false; 263 | 264 | index_type index = GetIndex(_handle); 265 | size_t version = GetVersion(_handle); 266 | 267 | HDL_ASSERT(index < getNodeBufferSize()); 268 | auto node = m_nodeBuffer + index; 269 | 270 | if (node->m_version != version) 271 | return false; // The handle was already destroyed. 272 | 273 | node->m_version++; 274 | // Force the version to wrap around to make sure it doesn't use more than VersionNumBits (otherwise the equality test would fail). 275 | node->m_version &= kVersionMask; 276 | 277 | // Special case for the last index: it cannot use the max version, otherwise the handle would be equal to kInvalid. 278 | // In this case, wrap around sooner. 279 | if (GetID(index, node->m_version) == kInvalid) 280 | node->m_version = 0; 281 | 282 | HDL_ASSERT(node->m_allocated); 283 | node->m_value.~T(); 284 | node->m_allocated = false; 285 | 286 | { 287 | LockGuard guard(m_mutex); 288 | m_handleCount--; 289 | m_freeIndices.push_back(index); 290 | } 291 | 292 | return true; 293 | } 294 | 295 | template 296 | T* 297 | HandlePool::get(integer_type _handle) 298 | { 299 | if (_handle == kInvalid) 300 | return nullptr; 301 | 302 | index_type index = GetIndex(_handle); 303 | size_t version = GetVersion(_handle); 304 | 305 | HDL_ASSERT(index < getNodeBufferSize()); 306 | auto node = m_nodeBuffer + index; 307 | 308 | if (node->m_version != version) 309 | return nullptr; // The handle was already destroyed. 310 | 311 | HDL_ASSERT(node->m_allocated); 312 | return &node->m_value; 313 | } 314 | 315 | template 316 | bool 317 | HandlePool::reserve(size_t _newCap) 318 | { 319 | LockGuard guard(m_mutex); 320 | return reserveNoLock(_newCap); 321 | } 322 | 323 | template 324 | bool 325 | HandlePool::reserveNoLock(size_t _newCap) 326 | { 327 | if (_newCap > max_size()) 328 | return false; 329 | 330 | auto currentCap = capacity(); 331 | 332 | if (_newCap <= currentCap) 333 | return true; // Nothing to do, we already have enough capacity 334 | 335 | // Check how many pages we need to store the additional nodes 336 | size_t freeBytes = m_nodeBufferCapacityBytes - m_nodeBufferSizeBytes; 337 | size_t neededBytes = (_newCap - currentCap) * sizeof(Node) - freeBytes; 338 | auto pageSize = HDL::VirtualMemory::GetPageSize(); 339 | size_t nbPages = 1; 340 | if (neededBytes > pageSize) 341 | nbPages = 1 + neededBytes / pageSize; 342 | 343 | // Reserve the node buffer if it wasn't done yet 344 | if (!m_nodeBuffer) 345 | m_nodeBuffer = (Node*)HDL::VirtualMemory::Reserve(kMaxHandles * sizeof(Node)); 346 | 347 | // Increase capacity by commiting more pages 348 | // Note: The memory allocated by VirtualMemory::Commit is zeroed, so m_version/m_allocated inside the nodes will automatically be initialized to 0 349 | if (!HDL::VirtualMemory::Commit((char*)m_nodeBuffer + m_nodeBufferCapacityBytes, nbPages * pageSize)) 350 | { 351 | // Allocation failed. (Out of memory?) 352 | return false; 353 | } 354 | 355 | m_nodeBufferCapacityBytes += nbPages * pageSize; 356 | 357 | return true; 358 | } 359 | 360 | template 361 | size_t 362 | HandlePool::getNodeBufferSize() const 363 | { 364 | return m_nodeBufferSizeBytes / sizeof(Node); 365 | } 366 | 367 | template 368 | typename HandlePool::index_type 369 | HandlePool::GetIndex(integer_type _handle) 370 | { 371 | // The index is in the low bits of the handle. 372 | return _handle & kIndexMask; 373 | } 374 | 375 | template 376 | size_t 377 | HandlePool::GetVersion(integer_type _handle) 378 | { 379 | // The version is in the high bits of the handle. 380 | // Note: integer_type must be unsigned otherwise this would do an arithmetic shift instead of logical shift. 381 | return _handle >> kIndexNumBits; 382 | } 383 | 384 | template 385 | typename HandlePool::integer_type 386 | HandlePool::GetID(index_type _index, size_t _version) 387 | { 388 | return (integer_type)((_version << kIndexNumBits) + _index); 389 | } -------------------------------------------------------------------------------- /handle.natvis: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | ({ m_intVal, x }) Invalid 8 | 9 | 10 | ({ m_intVal, x }) Destroyed 11 | 12 | 13 | ({ m_intVal, x }) { s_pool.m_nodeBuffer[m_intVal & s_pool.kIndexMask].m_value } 14 | 15 | 16 | m_intVal, x 17 | 18 | m_intVal & s_pool.kIndexMask 19 | 20 | 21 | m_intVal >> s_pool.kIndexNumBits 22 | 23 | 24 | "Destroyed" 25 | 26 | 27 | s_pool.m_nodeBuffer[m_intVal & s_pool.kIndexMask].m_value 28 | 29 | 30 | 31 | 32 | 33 | -------------------------------------------------------------------------------- /handle_win32.cpp: -------------------------------------------------------------------------------- 1 | #include "handle.h" 2 | #include 3 | #define WIN32_LEAN_AND_MEAN 4 | #define NOMINMAX 5 | #include "windows.h" 6 | 7 | namespace HDL 8 | { 9 | namespace VirtualMemory 10 | { 11 | static std::string GetFormattedErrorString(DWORD _errorCode) 12 | { 13 | LPVOID buf; 14 | FormatMessageA( 15 | FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, 16 | NULL, 17 | _errorCode, 18 | MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), // Default language 19 | (LPSTR)&buf, 20 | 0, 21 | NULL 22 | ); 23 | std::string errorString = (char*)buf; 24 | LocalFree(buf); 25 | 26 | return errorString; 27 | } 28 | 29 | size_t GetPageSize() 30 | { 31 | struct PageSizeInitializer 32 | { 33 | size_t m_value; 34 | 35 | PageSizeInitializer() 36 | { 37 | SYSTEM_INFO info = {}; 38 | GetSystemInfo(&info); 39 | 40 | m_value = info.dwPageSize; 41 | } 42 | } static pageSize; 43 | 44 | return pageSize.m_value; 45 | } 46 | 47 | void* Reserve(size_t _size) 48 | { 49 | auto address = VirtualAlloc( 50 | nullptr, // lpAddress 51 | _size, 52 | MEM_RESERVE, 53 | PAGE_READWRITE 54 | ); 55 | 56 | HDL_ASSERT(address != nullptr, GetFormattedErrorString(GetLastError()).c_str()); 57 | 58 | return address; 59 | } 60 | 61 | void Release(void* _address, size_t _size) 62 | { 63 | (void)_size; // When using MEM_RELEASE, the passed size must be 0. The entire region that was reserved will be released. 64 | 65 | auto success = VirtualFree( 66 | _address, 67 | 0, 68 | MEM_RELEASE 69 | ); 70 | 71 | HDL_ASSERT(success, GetFormattedErrorString(GetLastError()).c_str()); 72 | } 73 | 74 | bool Commit(void* _address, size_t _size) 75 | { 76 | auto address = VirtualAlloc( 77 | _address, 78 | _size, 79 | MEM_COMMIT, 80 | PAGE_READWRITE 81 | ); 82 | 83 | HDL_ASSERT(address != nullptr, GetFormattedErrorString(GetLastError()).c_str()); 84 | return address != nullptr; 85 | } 86 | 87 | void Decommit(void* _address, size_t _size) 88 | { 89 | auto success = VirtualFree( 90 | _address, 91 | _size, 92 | MEM_DECOMMIT 93 | ); 94 | 95 | HDL_ASSERT(success, GetFormattedErrorString(GetLastError()).c_str()); 96 | } 97 | } 98 | } 99 | -------------------------------------------------------------------------------- /premake5.exe: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jlaumon/handle/67b1f74addc2c92518baf9a776652ba737e56c14/premake5.exe -------------------------------------------------------------------------------- /test/basics.cpp: -------------------------------------------------------------------------------- 1 | #include "catch/catch.hpp" 2 | #include "handle.h" 3 | #include 4 | #include 5 | 6 | TEST_CASE("basic tests", "[basics]") 7 | { 8 | using IntHandle = Handle; 9 | 10 | IntHandle::Reset(); 11 | 12 | REQUIRE(IntHandle::Size() == 0); 13 | REQUIRE(IntHandle::Capacity() == 0); 14 | REQUIRE(IntHandle::MaxSize() == 10); 15 | 16 | GIVEN("all handles are created") 17 | { 18 | std::vector v; 19 | 20 | for (int i = 0; i < IntHandle::MaxSize(); ++i) 21 | v.push_back(IntHandle::Create(i)); 22 | 23 | REQUIRE(IntHandle::Size() == IntHandle::MaxSize()); 24 | REQUIRE(IntHandle::Capacity() >= IntHandle::Size()); 25 | 26 | THEN("all handles are unique") 27 | { 28 | std::set s; 29 | for (auto h : v) 30 | s.insert(h); 31 | 32 | REQUIRE(s.size() == v.size()); 33 | } 34 | 35 | THEN("Get returns the same values") 36 | { 37 | for (int i = 0; i < IntHandle::MaxSize(); ++i) 38 | { 39 | REQUIRE(*IntHandle::Get(v[i]) == i); 40 | } 41 | } 42 | 43 | WHEN("trying to create more handles") 44 | { 45 | auto h = IntHandle::Create(-1); 46 | auto h2 = IntHandle::Create(-2); 47 | 48 | THEN("the handles are invalid") 49 | { 50 | h = IntHandle::kInvalid; 51 | h2 = IntHandle::kInvalid; 52 | } 53 | } 54 | 55 | WHEN("destroying all handles") 56 | { 57 | auto cap = IntHandle::Capacity(); 58 | 59 | for (auto h : v) 60 | REQUIRE(IntHandle::Destroy(h)); 61 | 62 | THEN("the size of the pool is zero") 63 | { 64 | REQUIRE(IntHandle::Size() == 0); 65 | } 66 | 67 | THEN("the capacity of the pool did not change") 68 | { 69 | REQUIRE(IntHandle::Capacity() == cap); 70 | } 71 | 72 | THEN("Get returns nullptr for those handles") 73 | { 74 | for (auto h : v) 75 | REQUIRE(IntHandle::Get(h) == nullptr); 76 | } 77 | } 78 | } 79 | 80 | WHEN("using all the indices") 81 | { 82 | int numIndices = 1 << IntHandle::pool_type::kIndexNumBits; 83 | std::vector v; 84 | 85 | for (int i = 0; i < numIndices; ++i) 86 | { 87 | auto h = IntHandle::Create(i); 88 | v.push_back(IntHandle::pool_type::GetIndex(h)); 89 | REQUIRE(IntHandle::Destroy(h)); 90 | } 91 | 92 | THEN("all indices are unique") 93 | { 94 | std::set s; 95 | for (auto i : v) 96 | s.insert(i); 97 | 98 | REQUIRE(s.size() == v.size()); 99 | } 100 | 101 | THEN("next handles reuse indices and have a greater version") 102 | { 103 | auto h = IntHandle::Create(-1); 104 | auto h2 = IntHandle::Create(-2); 105 | 106 | REQUIRE(IntHandle::pool_type::GetIndex(h) < numIndices); 107 | REQUIRE(IntHandle::pool_type::GetVersion(h) > 0); 108 | REQUIRE(IntHandle::pool_type::GetIndex(h2) < numIndices); 109 | REQUIRE(IntHandle::pool_type::GetVersion(h2) > 0); 110 | } 111 | } 112 | } 113 | 114 | TEST_CASE("wrapping test", "[basics]") 115 | { 116 | using CharHandle = Handle; 117 | 118 | CharHandle::Reset(); 119 | 120 | REQUIRE(CharHandle::Size() == 0); 121 | REQUIRE(CharHandle::Capacity() == 0); 122 | 123 | int numPossibleHandles = (1 << (sizeof(CharHandle::integer_type) * 8)) - 1; // -1 because one is reserved for kInvalid 124 | 125 | for (int i = 0; i < numPossibleHandles - 1; ++i) 126 | { 127 | auto h = CharHandle::Create('a'); 128 | REQUIRE(h != CharHandle::kInvalid); 129 | CharHandle::Destroy(h); 130 | } 131 | 132 | auto lastHandle = CharHandle::Create('a'); 133 | REQUIRE(lastHandle != CharHandle::kInvalid); 134 | REQUIRE(CharHandle::pool_type::GetVersion(lastHandle) > 0); 135 | 136 | // If we were not careful, this handle would be equal to kInvalid (max index & max version) 137 | // but the version should automatically wrap sooner to avoid this case. 138 | auto earlyWrapHandle = CharHandle::Create('a'); 139 | REQUIRE(earlyWrapHandle != CharHandle::kInvalid); 140 | REQUIRE(CharHandle::pool_type::GetVersion(earlyWrapHandle) == 0); 141 | 142 | // This one should be the one where the normal wrapping happens. 143 | auto wrappingHandle = CharHandle::Create('a'); 144 | REQUIRE(wrappingHandle != CharHandle::kInvalid); 145 | REQUIRE(wrappingHandle == 0); 146 | } -------------------------------------------------------------------------------- /test/largeobjects.cpp: -------------------------------------------------------------------------------- 1 | #include "catch/catch.hpp" 2 | #include "handle.h" 3 | #include 4 | #include 5 | 6 | struct LargeObject 7 | { 8 | char data[10000-8]; 9 | 10 | LargeObject(char v) 11 | { 12 | memset(data, v, sizeof(data)); 13 | } 14 | 15 | bool check(char v) const 16 | { 17 | LargeObject l(v); 18 | return memcmp(l.data, data, sizeof(data)) == 0; 19 | } 20 | }; 21 | 22 | TEST_CASE("objects that are bigger than a memory page", "[largeobjects]") 23 | { 24 | REQUIRE(sizeof(LargeObject) > HDL::VirtualMemory::GetPageSize()); 25 | REQUIRE(sizeof(LargeObject) == sizeof(LargeObject::data)); // no padding 26 | 27 | using LOHandle = Handle; 28 | 29 | LOHandle::Reset(); 30 | 31 | REQUIRE(LOHandle::Size() == 0); 32 | REQUIRE(LOHandle::Capacity() == 0); 33 | 34 | GIVEN("all handles are created") 35 | { 36 | std::vector v; 37 | 38 | for (int i = 0; i < LOHandle::MaxSize(); ++i) 39 | v.push_back(LOHandle::Create('a' + i)); 40 | 41 | THEN("no memory stomping happened") 42 | { 43 | for (int i = 0; i < LOHandle::MaxSize(); ++i) 44 | REQUIRE(LOHandle::Get(v[i])->check('a' + i)); 45 | } 46 | 47 | WHEN("re-creating all handles") 48 | { 49 | for (auto h : v) 50 | REQUIRE(LOHandle::Destroy(h)); 51 | 52 | v.clear(); 53 | 54 | for (int i = 0; i < LOHandle::MaxSize(); ++i) 55 | v.push_back(LOHandle::Create('A' + i)); 56 | 57 | THEN("no memory stomping happened") 58 | { 59 | for (int i = 0; i < LOHandle::MaxSize(); ++i) 60 | REQUIRE(LOHandle::Get(v[i])->check('A' + i)); 61 | } 62 | } 63 | } 64 | } -------------------------------------------------------------------------------- /test/main.cpp: -------------------------------------------------------------------------------- 1 | #define CATCH_CONFIG_MAIN 2 | #include "catch/catch.hpp" -------------------------------------------------------------------------------- /test/multithreading.cpp: -------------------------------------------------------------------------------- 1 | #include "catch/catch.hpp" 2 | #include "handle.h" 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | TEST_CASE("concurrent creation/destruction of handles", "[multithreading]") 10 | { 11 | using IntHandle = Handle; 12 | 13 | std::atomic error = false; 14 | 15 | struct ThreadStats 16 | { 17 | int createSuccess = 0; 18 | int createFail = 0; 19 | int destroyFailIndex = -1; 20 | int badValueIndex = -1; 21 | std::vector handles; 22 | }; 23 | 24 | auto threadFunc = [&](ThreadStats* stats, int id) 25 | { 26 | std::random_device rd; 27 | std::mt19937 randomEngine(rd()); 28 | std::vector& v = stats->handles; 29 | 30 | for (int i = 0; i < 100000; ++i) 31 | { 32 | // If any thread has a problem, stop everything and report. 33 | if (error) 34 | return; 35 | 36 | // Randomy either create a handle or destroy one. 37 | bool create = std::uniform_int_distribution<>(0, 1)(randomEngine) == 0; 38 | if (create || v.empty()) 39 | { 40 | auto h = IntHandle::Create(id); 41 | if (h != IntHandle::kInvalid) 42 | { 43 | v.push_back(h); 44 | stats->createSuccess++; 45 | } 46 | else 47 | { 48 | stats->createFail++; 49 | } 50 | } 51 | else 52 | { 53 | // Pick a random handle to destroy. 54 | int index = std::uniform_int_distribution<>(0, (int)v.size() - 1)(randomEngine); 55 | 56 | // Make sure the value inside the handle is still the same. 57 | auto ptr = IntHandle::Get(v[index]); 58 | if (ptr && *ptr != id) // Note: if ptr is nullptr, it will be reported as a destroy fail below 59 | { 60 | stats->badValueIndex = index; 61 | error = true; 62 | return; 63 | } 64 | if (!IntHandle::Destroy(v[index])) 65 | { 66 | stats->destroyFailIndex = index; 67 | error = true; 68 | return; 69 | } 70 | 71 | // Erase without moving everything, faster. 72 | std::swap(v[index], v.back()); 73 | v.pop_back(); 74 | } 75 | } 76 | 77 | // Before leaving, destroy all the handles this thread created. 78 | for (auto& h : v) 79 | { 80 | // Make sure the value inside the handle is still the same. 81 | auto ptr = IntHandle::Get(h); 82 | if (ptr && *ptr != id) // Note: if ptr is nullptr, it will be reported as a destroy fail below 83 | { 84 | stats->badValueIndex = (int)(&h - v.data()); 85 | error = true; 86 | return; 87 | } 88 | if (!IntHandle::Destroy(h)) 89 | { 90 | stats->destroyFailIndex = (int)(&h - v.data()); 91 | error = true; 92 | return; 93 | } 94 | } 95 | v.clear(); 96 | }; 97 | 98 | std::vector threads; 99 | std::vector stats(10); 100 | for (int i = 0; i < stats.size(); ++i) 101 | threads.push_back(std::thread(threadFunc, &stats[i], i)); 102 | 103 | for (auto& th : threads) 104 | th.join(); 105 | 106 | for (int i = 0; i < stats.size(); ++i) 107 | { 108 | INFO("Thread " << i); 109 | INFO(" createSuccess = " << stats[i].createSuccess); 110 | INFO(" createFail = " << stats[i].createFail); 111 | REQUIRE(stats[i].destroyFailIndex == -1); 112 | REQUIRE(stats[i].badValueIndex == -1); 113 | REQUIRE(stats[i].handles.size() == 0); 114 | } 115 | 116 | REQUIRE_FALSE(error); // Another REQUIRE should already have broken, but just in case. 117 | REQUIRE(IntHandle::Size() == 0); 118 | } -------------------------------------------------------------------------------- /test/premake.bat: -------------------------------------------------------------------------------- 1 | ..\premake5 --file=test_premake.lua vs2017 -------------------------------------------------------------------------------- /test/test_handle_config.cpp: -------------------------------------------------------------------------------- 1 | #include "test_handle_config.h" 2 | #include 3 | #include 4 | #include "catch/catch.hpp" 5 | 6 | 7 | std::string FormatAssertString(const char* _condition) 8 | { 9 | return std::string() + "HDL_ASSERT(" + _condition + ") failed"; 10 | } 11 | 12 | std::string FormatAssertString(const char* _condition, const char* _msg, ...) 13 | { 14 | va_list args; 15 | va_start(args, _msg); 16 | 17 | char buffer[1024]; 18 | vsnprintf(buffer, sizeof(buffer), _msg, args); 19 | buffer[sizeof(buffer) - 1] = 0; 20 | 21 | va_end(args); 22 | 23 | return std::string() + "HDL_ASSERT(" + _condition + ") failed; " + buffer; 24 | } 25 | -------------------------------------------------------------------------------- /test/test_handle_config.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "catch/catch.hpp" 4 | 5 | #define HDL_ASSERT(condition, ...) \ 6 | if (!(condition)) { \ 7 | auto str = FormatAssertString(#condition, ##__VA_ARGS__); \ 8 | FAIL(str); \ 9 | } 10 | 11 | std::string FormatAssertString(const char* _condition); 12 | std::string FormatAssertString(const char* _condition, const char* _msg, ...); 13 | -------------------------------------------------------------------------------- /test/test_premake.lua: -------------------------------------------------------------------------------- 1 | solution "HandleTest" 2 | 3 | platforms { "x64" } 4 | configurations { "Debug", "Release" } 5 | startproject "HandleTest" 6 | 7 | project "HandleTest" 8 | 9 | kind "ConsoleApp" 10 | 11 | files 12 | { 13 | "../*.h", 14 | "../*.cpp", 15 | "../*.natvis", 16 | "**.h", 17 | "**.hpp", 18 | "**.cpp", 19 | } 20 | 21 | includedirs 22 | { 23 | "..", 24 | ".", 25 | } 26 | 27 | defines 28 | { 29 | "HDL_USER_CONFIG=\"test_handle_config.h\"" 30 | } 31 | 32 | debugargs 33 | { 34 | "--break", -- break into debugger on failure 35 | "--wait-for-keypress exit" -- wait for keypress to make sure we can read the output 36 | } 37 | 38 | vpaths 39 | { 40 | ["*"] = "../*", 41 | } 42 | --------------------------------------------------------------------------------