├── .gitignore ├── CONTRIBUTING.md ├── README.md ├── build.bat ├── build_mac.sh ├── build_nix.sh ├── convert ├── convert_int.cpp └── convert_int.h ├── environment ├── environment_appinfo.cpp └── environment_appinfo.h ├── json ├── json_serializer.cpp └── json_serializer.h ├── security ├── security_csprng.cpp └── security_csprng.h ├── sync ├── sync_event.cpp ├── sync_event.h ├── sync_mutex.cpp ├── sync_mutex.h ├── sync_readwritelock.cpp ├── sync_readwritelock.h ├── sync_semaphore.cpp ├── sync_semaphore.h ├── sync_sharedmem.cpp ├── sync_sharedmem.h ├── sync_tls.cpp ├── sync_tls.h ├── sync_util.cpp └── sync_util.h ├── templates ├── cache.cpp ├── cache.h ├── cache_util.h ├── detachable_list.h ├── detachable_list_util.h ├── detachable_ordered_hash.cpp ├── detachable_ordered_hash.h ├── detachable_ordered_hash_util.h ├── detachable_queue.h ├── detachable_queue_util.h ├── fast_find_replace.h ├── fast_find_replace_util.h ├── packed_ordered_hash.cpp ├── packed_ordered_hash.h ├── packed_ordered_hash_util.h ├── shared_lib.h ├── shared_lib_util.h ├── static_2d_array.h ├── static_mixed_var.h └── static_vector.h ├── test.h ├── test_suite.cpp └── utf8 ├── utf8_appinfo.cpp ├── utf8_appinfo.h ├── utf8_file_dir.cpp ├── utf8_file_dir.h ├── utf8_mixed_var.h ├── utf8_util.cpp └── utf8_util.h /.gitignore: -------------------------------------------------------------------------------- 1 | cross_platform/* 2 | *.exe 3 | *.obj 4 | test_suite 5 | samples/* 6 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | Contributing 2 | ============ 3 | 4 | First off, thank you for your interest in contributing to this CubicleSoft project! Sincere help is always appreciated. 5 | 6 | Before you open an issue on the issue tracker, fork the repo, or start a pull request, there are some simple guidelines that help reduce the amount of time required to handle the request and keeps the repository neat and clean: 7 | 8 | * Don't open an issue or pull request just for your personal benefit but rather for the benefit of the larger community. "How can I help other people?" A large percentage of requests are closed because submitters don't follow this guideline. 9 | * Requests will be rejected if they implicitly create a security vulnerability. Ain't nobody got time for that. A surprisingly large percentage of requests are rejected because submitters don't follow this guideline. 10 | * Forking a repository is serious business. If you are making changes to 5 lines of code or fewer, a fork + pull request is a waste of everyone's time, including yours. Just use the issue tracker. 11 | * It would be really nice if all pull requests carefully respected the existing coding style. All of the initial source code here is written in Crimson Editor with: Autoindent turned off, no automatic coding of any sort, trailing spaces/tabs are removed on save, forced UNIX line endings, tab-indented, and carefully written against a set style (e.g. braces on their own line, single space after keywords unless casting, liberal use of parenthesis, etc). 12 | 13 | These guidelines are just that: Guidelines. The purpose of the guidelines is to maintain zero open issues/pull requests across all repositories without going insane. Sanity matters. These guidelines actually apply to every incrementally maintained repository everywhere, not just CubicleSoft. 14 | 15 | Keeping these things in mind, contribute away! 16 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Cross-platform C++ Snippet Library 2 | ================================== 3 | 4 | This is a cross-platform C++ snippet library. Most C++ libraries (wxWidgets, Qt, Poco, etc.) attempt to be the "be all, end all" solution for the project you are building. Those libraries are great for large projects but have the unfortunate tendency to introduce heavy dependencies for little projects, especially when writing third-party libraries for the general user. 5 | 6 | [![Donate](https://cubiclesoft.com/res/donate-shield.png)](https://cubiclesoft.com/donate/) [![Discord](https://img.shields.io/discord/777282089980526602?label=chat&logo=discord)](https://cubiclesoft.com/product-support/github/) 7 | 8 | This Library is Different 9 | ------------------------- 10 | 11 | Yeah, yeah, everyone says that, but it really is! Instead of dropping a whole library into your project, just find and grab the files you need. For example, if you need a cross-platform, cross-process, named mutex, the only files needed are: 12 | 13 | * sync/sync_mutex.h 14 | * sync/sync_mutex.cpp 15 | * sync/sync_util.h 16 | * sync/sync_util.cpp 17 | 18 | Which only add a few kilobytes to the project. Each category (e.g. 'sync') may have its own 'util' files that include common functionality for that category. If you need to know how to use something, crack open the 'test_suite.cpp' file. It contains a bunch of examples. However, the various classes were designed to be intuitive and most IDEs will pick up the comments scattered around the .h files. 19 | 20 | Features 21 | -------- 22 | 23 | * Small object files. Just a few KB each for the most part. 24 | * Very few interdependencies. 25 | * Cross-platform, cross-process, named: Mutex, semaphore, event, and reader-writer objects. 26 | * Cross-platform, thread local temporary memory management via Sync::TLS. Sync::TLS outperforms system malloc()/free()! (See Notes) 27 | * Cross-platform CSPRNG. 28 | * Detachable node queue, linked list, and ordered hash(!) implementations. (See Notes) 29 | * Cache support. A C++ template that implements a partial hash. 30 | * Static vector implementation. 31 | * Integer to string conversion. With file size options as well (i.e. MB, GB, etc). 32 | * Packed ordered hash. Up to three times faster than the detachable node OrderedHash. 33 | * Minimalist Unicode conversion support. Just enough useful logic without having to drag in a multi-MB Unicode support library. 34 | * Cross-platform, UTF-8 file and directory manipulation classes. 35 | * Cross-platform, UTF-8 storage location functions (e.g. a user's home folder). 36 | * Cross-platform, static buffer JSON serializer class. 37 | * Cross-platform shared library loader. No more awkward LoadLibrary/GetProcAddress or dlopen/dlsym calls. 38 | * FastFind and FastReplace templates. Works on any binary data. FastFind probably outperforms std::search (See Notes). FastReplace supports alternate allocators (e.g. Sync::TLS) and comparison functions (e.g. case-insensitive comparison). 39 | * Variable data storage via StaticMixedVar, UTF8::UTF8MixedVar, and Sync::TLS::MixedVar. For when you want lightweight dynamic typing with basic string support or just want to avoid std::string. 40 | * Has a liberal open source license. MIT or LGPL, your choice. 41 | * Designed for relatively painless integration into your project. 42 | * Sits on GitHub for all of that pull request and issue tracker goodness to easily submit changes and ideas respectively. 43 | 44 | Test Suite 45 | ---------- 46 | 47 | The test suite that comes with the library is able to be built using 'build.bat' on Windows and 'build.sh' on other OSes. Running the test suite (e.g. test_suite.exe) without any options will run the basic startup tests and verify that everything is working properly on the target platform. 48 | 49 | The following commands will run various performance benchmarks: 50 | 51 | * test_suite synctls 52 | * test_suite hashkey 53 | * test_suite list 54 | * test_suite hash 55 | * test_suite loop (Helps identify bad benchmarks) 56 | 57 | Output looks like: 58 | 59 | ``` 60 | # test_suite list 61 | List performance benchmark 62 | -------------------------- 63 | Running List speed tests... 64 | Insertion - 23,179,293 nodes added/sec 65 | Detach/attach performance - 64,911,010 nodes/sec 66 | Find performance (1 million nodes) - 17,245 nodes/sec 67 | 68 | 69 | # test_suite hash 70 | Hash performance benchmark 71 | -------------------------- 72 | Running OrderedHash speed tests... 73 | Integer keys, djb2 hash keys - 5,905,388 nodes added/sec 74 | String keys, djb2 hash keys - 3,500,444 nodes added/sec 75 | Integer keys, SipHash hash keys - 2,578,463 nodes added/sec 76 | String keys, SipHash hash keys - 2,180,187 nodes added/sec 77 | Integer keys, detach/attach performance - 19,256,672 nodes/sec 78 | String keys, detach/attach performance - 11,858,135 nodes/sec 79 | Integer keys, find performance (1 million nodes) - 8,277,900 nodes/sec 80 | String keys, find performance (1 million nodes) - 6,764,330 nodes/sec 81 | 82 | Running PackedOrderedHash speed tests... 83 | Integer keys, djb2 hash keys - 17,638,442 nodes added/sec 84 | String keys, djb2 hash keys - 5,592,405 nodes added/sec 85 | Integer keys, SipHash hash keys - 3,154,760 nodes added/sec 86 | String keys, SipHash hash keys - 3,713,171 nodes added/sec 87 | Integer keys, find performance (1 million nodes) - 22,020,249 nodes/sec 88 | String keys, find performance (1 million nodes) - 13,426,850 nodes/sec 89 | ``` 90 | 91 | System used for example output above: Intel i7-6700K @ 4GHz, 32GB RAM, Windows 10 Pro 92 | 93 | Notes 94 | ----- 95 | 96 | Some classes require files in the 'templates' subdirectory. However, again, the number of dependencies is kept to the bare minimum for proper functionality. These templates were written primarily to reduce the overall size of object files, but a few of them introduce several features that are missing in the Standard library. Plus the Standard library templates tend to be rather heavy. 97 | 98 | In testing, Sync::TLS outperformed system malloc()/free() by a factor of 1.8 to 19.0 times on a single thread. Performance varied greatly depending on hardware, OS, and compiler settings. The approach I used appears to be similar to TCMalloc (both utilize Thread Local Storage in a similar manner), but Sync::TLS has a much simpler implementation and is intended for short-lived data that would normally be placed in a fixed-size stack. Multithreading was not tested but there are probably significant additional performance improvements over system malloc()/free() due to the utilization of Thread Local Storage. 99 | 100 | There are three very slow operations in all programs: External data access (e.g. hard drive, network), memory allocations, and system calls - in that order. Detachable nodes in data structures help mitigate the second problem. 101 | 102 | The detachable node ordered hash is similar to PHP 5 arrays. It accepts both integer and string keys in the same hash, has almost constant time insert, lookup, delete, and iteration operations, and, most importantly, maintains the desired order of elements. This is almost the last std::map-like C++ data structure you will ever need. 103 | 104 | The packed ordered hash is similar to PHP 7 arrays. The PackedOrderedHash template implements a hybrid array + hash and accepts both integer and string keys in the same hash but has better performance metrics for the specific but common scenario of inserting new nodes only at the end, frequent key- and index-based lookups, some iteration, and few deletions. Each node only has 24 bytes of overhead instead of the 56 bytes of overhead for OrderedHashNode on 64-bit OSes. Nodes are inline and therefore can't be detached, but they can be overwritten and unset. The tradeoff for inline nodes is reduced memory overhead, generally fewer allocations, and increased performance by leveraging CPU cache lines. The test suite benchmarks show up to a 3x improvement in performance over OrderedHash for the most common hashing use-cases. 105 | 106 | Handling Unicode is HARD. Once upon a time, many years ago, I started writing my own Unicode implementation but eventually gave up. There are three main sections of code plus large lookup tables in a full-blown, up-to-date Unicode implementation: Code point handling (easy-ish), Combining and Precomposed characters, Line Breaking, and Normalization (hard), and finally Case Folding (nearly impossible). Code point handling is all this snippet library offers and so all Unicode strings that you handle should generally be treated as opaque data. If you need something more refined than code points in C++, then there is only one legitimate option, which is the IBM ICU implementation of Unicode but will add ~25MB of dependencies to your project. For some reason I can't find my original software, but I recall getting through the aforementioned Hard bits with around 65KB of lookup tables for common Normalization and the code even supported unlimited combining code points, which was very cool but extremely nerdy. Regardless, 65KB of tables doesn't really work well for this project (i.e. it wouldn't really count as a "snippet"). Therefore, only code point handling makes any sense. Note that applications on Windows that use the UTF-8 code snippets for directory and file management will run a bit slower than their *NIX counterparts due to translating between UTF-8 and UTF-16 with correct surrogate support for the latter, of course. 107 | 108 | Only JSON serialization is supported in this library at this time. There are quite a few JSON parser + serializer libraries for C++ that you should consider if you need a JSON _parser_ ([See these benchmarks](https://github.com/miloyip/nativejson-benchmark) to get started). Most of the JSON libraries out there require a separate library compilation step and do their own memory management. The serialization class here is different than most libraries since it relies on a static buffer that is set by the application via the SetBuffer() call. As a result, only a small static vector depth stack (StaticVector) is allocated by the class and should therefore be very light on RAM and very fast even when generating extremely large multi-GB JSON blobs. At only ~600 lines of code, the snippet here is roughly half the size of [PicoJSON](https://github.com/kazuho/picojson). As far as parsers go, [RapidJSON](https://github.com/Tencent/rapidjson) is the fastest benchmarked library but requires a separate compilation step and is more awkward to use than [JSON for Modern C++](https://github.com/nlohmann/json), which is a single 900KB header file (25K lines of code). JSON for Modern C++ is 25 times larger than PicoJSON but the parser is about 2 times faster. 109 | 110 | FastFind has an average case (and probably worst case) of O(5n) or better, which is implicitly better than the naive std::search() O(m * n). Up to five "points of interest" are located in the pattern - beginning, end, two minimal values, and one midpoint - to minimize the number of compares before performing the full comparison. This approach mitigates security vulnerabilities in naive implementations while simultaneously outperforming most alternate algorithms including Knuth-Morris-Pratt and Boyer-Moore that construct a lookup table (i.e. allocate memory - a rather sluggish operation not needed for the average string search). 111 | -------------------------------------------------------------------------------- /build.bat: -------------------------------------------------------------------------------- 1 | @echo off 2 | cls 3 | 4 | cl /W3 /Ox test_suite.cpp convert/*.cpp security/*.cpp sync/*.cpp templates/*.cpp environment/*.cpp utf8/*.cpp json/*.cpp /link /FILEALIGN:512 /OPT:REF /OPT:ICF /INCREMENTAL:NO advapi32.lib shell32.lib ws2_32.lib /out:test_suite.exe 5 | -------------------------------------------------------------------------------- /build_mac.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | gcc -m64 -std=c++0x -pedantic -Wall -Wextra -Wshadow -Wpointer-arith -Wcast-qual -pthread -O3 convert/*.cpp security/*.cpp sync/*.cpp templates/*.cpp environment/*.cpp utf8/*.cpp json/*.cpp test_suite.cpp -o test_suite -lstdc++ 3 | -------------------------------------------------------------------------------- /build_nix.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | gcc -m64 -std=c++0x -pedantic -Wall -Wextra -Wshadow -Wpointer-arith -Wcast-qual -pthread -O3 convert/*.cpp security/*.cpp sync/*.cpp templates/*.cpp environment/*.cpp utf8/*.cpp json/*.cpp test_suite.cpp -o test_suite -lstdc++ -lrt -ldl 3 | -------------------------------------------------------------------------------- /convert/convert_int.cpp: -------------------------------------------------------------------------------- 1 | // Cross-platform integer conversion functions. 2 | // (C) 2021 CubicleSoft. All Rights Reserved. 3 | 4 | #define _CRT_SECURE_NO_WARNINGS 5 | 6 | #include "convert_int.h" 7 | 8 | #include 9 | 10 | namespace CubicleSoft 11 | { 12 | namespace Convert 13 | { 14 | bool Int::ToString(char *Result, size_t Size, std::uint64_t Num, char Separator, size_t Radix) 15 | { 16 | if (Size < 2 || Radix < 2 || Radix > 36) return false; 17 | 18 | size_t x = Size, y = 0, z; 19 | 20 | Result[--x] = '\0'; 21 | if (!Num) Result[--x] = '0'; 22 | else 23 | { 24 | while (Num && x) 25 | { 26 | if (Separator != '\0' && y && y % 3 == 0) 27 | { 28 | Result[--x] = Separator; 29 | if (!x) return false; 30 | } 31 | 32 | z = Num % Radix; 33 | Result[--x] = (char)(z > 9 ? z - 10 + 'A' : z + '0'); 34 | Num /= Radix; 35 | y++; 36 | } 37 | 38 | if (Num) return false; 39 | } 40 | 41 | memmove(Result, Result + x, Size - x); 42 | 43 | return true; 44 | } 45 | 46 | bool Int::ToString(char *Result, size_t Size, std::int64_t Num, char Separator, size_t Radix) 47 | { 48 | if (Num >= 0) return ToString(Result, Size, (std::uint64_t)Num, Separator, Radix); 49 | 50 | if (Size < 2) return false; 51 | Result[0] = '-'; 52 | 53 | return ToString(Result + 1, Size - 1, (std::uint64_t)-Num, Separator, Radix); 54 | } 55 | 56 | bool Int::ToFilesizeString(char *Result, size_t Size, std::uint64_t Num, size_t NumFrac, char NumSeparator, char DecimalSeparator, char UnitSeparator, const char *BytesUnit) 57 | { 58 | char Trail; 59 | std::uint64_t Divider, Extra; 60 | 61 | if (Size < 5 + strlen(BytesUnit)) return false; 62 | 63 | size_t x = Size, x2, y = 0; 64 | 65 | if (NumFrac) 66 | { 67 | // Windows/web browser format. 68 | if (Num >= 1125899906842624000ULL) 69 | { 70 | Trail = 'E'; 71 | Divider = 1152921504606846976ULL; 72 | } 73 | else if (Num >= 1099511627776000ULL) 74 | { 75 | Trail = 'P'; 76 | Divider = 1125899906842624ULL; 77 | } 78 | else if (Num >= 1073741824000ULL) 79 | { 80 | Trail = 'T'; 81 | Divider = 1099511627776ULL; 82 | } 83 | else if (Num >= 1048576000ULL) 84 | { 85 | Trail = 'G'; 86 | Divider = 1073741824ULL; 87 | } 88 | else if (Num >= 1024000ULL) 89 | { 90 | Trail = 'M'; 91 | Divider = 1048576ULL; 92 | } 93 | else if (Num >= 1024ULL) 94 | { 95 | Trail = 'K'; 96 | Divider = 1024ULL; 97 | NumFrac = 0; 98 | } 99 | else 100 | { 101 | Trail = '\0'; 102 | Divider = 1ULL; 103 | NumFrac = 0; 104 | } 105 | } 106 | else 107 | { 108 | // Strict format. 109 | if (Num >= 1152921504606846976ULL) 110 | { 111 | Trail = 'E'; 112 | Divider = 1152921504606846976ULL; 113 | } 114 | else if (Num >= 1125899906842624ULL) 115 | { 116 | Trail = 'P'; 117 | Divider = 1125899906842624ULL; 118 | } 119 | else if (Num >= 1099511627776ULL) 120 | { 121 | Trail = 'T'; 122 | Divider = 1099511627776ULL; 123 | } 124 | else if (Num >= 1073741824ULL) 125 | { 126 | Trail = 'G'; 127 | Divider = 1073741824ULL; 128 | } 129 | else if (Num >= 1048576ULL) 130 | { 131 | Trail = 'M'; 132 | Divider = 1048576ULL; 133 | } 134 | else if (Num >= 1024ULL) 135 | { 136 | Trail = 'K'; 137 | Divider = 1024ULL; 138 | NumFrac = 0; 139 | } 140 | else 141 | { 142 | Trail = '\0'; 143 | Divider = 1ULL; 144 | NumFrac = 0; 145 | } 146 | } 147 | 148 | if (Trail != '\0') 149 | { 150 | Result[--x] = '\0'; 151 | Result[--x] = 'B'; 152 | Result[--x] = Trail; 153 | } 154 | else 155 | { 156 | x = Size - strlen(BytesUnit) - 1; 157 | strcpy(Result + x, BytesUnit); 158 | } 159 | 160 | if (UnitSeparator != '\0') Result[--x] = UnitSeparator; 161 | 162 | if (NumFrac) 163 | { 164 | if (x < NumFrac + 2) return false; 165 | 166 | Extra = Num % Divider; 167 | for (x2 = 0; x2 < NumFrac; x2++) Extra *= 10ULL; 168 | Extra /= Divider; 169 | 170 | while (Extra && NumFrac && x) 171 | { 172 | Result[--x] = (char)(Extra % 10) + '0'; 173 | Extra /= 10; 174 | NumFrac--; 175 | } 176 | 177 | while (NumFrac && x) 178 | { 179 | Result[--x] = '0'; 180 | NumFrac--; 181 | } 182 | 183 | if (Extra || !x) return false; 184 | 185 | Result[--x] = DecimalSeparator; 186 | 187 | if (!x) return false; 188 | } 189 | 190 | Num /= Divider; 191 | 192 | if (!Num) Result[--x] = '0'; 193 | else 194 | { 195 | while (Num && x) 196 | { 197 | if (NumSeparator != '\0' && y && y % 3 == 0) 198 | { 199 | Result[--x] = NumSeparator; 200 | if (!x) return false; 201 | } 202 | 203 | Result[--x] = (char)(Num % 10) + '0'; 204 | Num /= 10; 205 | y++; 206 | } 207 | 208 | if (Num) return false; 209 | } 210 | 211 | memmove(Result, Result + x, Size - x); 212 | 213 | return true; 214 | } 215 | 216 | bool Int::ToFilesizeString(char *Result, size_t Size, std::int64_t Num, size_t NumFrac, char NumSeparator, char DecimalSeparator, char UnitSeparator, const char *BytesUnit) 217 | { 218 | if (Num >= 0) return ToFilesizeString(Result, Size, (std::uint64_t)Num, NumFrac, NumSeparator, DecimalSeparator, UnitSeparator, BytesUnit); 219 | 220 | if (Size < 2) return false; 221 | Result[0] = '-'; 222 | 223 | return ToFilesizeString(Result + 1, Size - 1, (std::uint64_t)-Num, NumFrac, NumSeparator, DecimalSeparator, UnitSeparator, BytesUnit); 224 | } 225 | } 226 | } 227 | -------------------------------------------------------------------------------- /convert/convert_int.h: -------------------------------------------------------------------------------- 1 | // Cross-platform integer conversion functions. 2 | // (C) 2021 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_CONVERT_INT 5 | #define CUBICLESOFT_CONVERT_INT 6 | 7 | #include 8 | #include 9 | 10 | namespace CubicleSoft 11 | { 12 | namespace Convert 13 | { 14 | class Int 15 | { 16 | public: 17 | static bool ToString(char *Result, size_t Size, std::uint64_t Num, char Separator = '\0', size_t Radix = 10); 18 | static bool ToString(char *Result, size_t Size, std::int64_t Num, char Separator = '\0', size_t Radix = 10); 19 | 20 | static bool ToFilesizeString(char *Result, size_t Size, std::uint64_t Num, size_t NumFrac = 2, char NumSeparator = '\0', char DecimalSeparator = '.', char UnitSeparator = ' ', const char *BytesUnit = "bytes"); 21 | static bool ToFilesizeString(char *Result, size_t Size, std::int64_t Num, size_t NumFrac = 2, char NumSeparator = '\0', char DecimalSeparator = '.', char UnitSeparator = ' ', const char *BytesUnit = "bytes"); 22 | }; 23 | } 24 | } 25 | 26 | #endif 27 | -------------------------------------------------------------------------------- /environment/environment_appinfo.cpp: -------------------------------------------------------------------------------- 1 | // Cross-platform application information. For Unicode application strings, see 'UTF8::AppInfo'. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #include "environment_appinfo.h" 5 | 6 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 7 | #else 8 | #include 9 | #endif 10 | 11 | namespace CubicleSoft 12 | { 13 | namespace Environment 14 | { 15 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 16 | // Windows. 17 | ProcessIDType AppInfo::GetCurrentProcessID() 18 | { 19 | return ::GetCurrentProcessId(); 20 | } 21 | 22 | ThreadIDType AppInfo::GetCurrentThreadID() 23 | { 24 | return ::GetCurrentThreadId(); 25 | } 26 | 27 | std::uint64_t AppInfo::GetUnixMicrosecondTime() 28 | { 29 | FILETIME TempTime; 30 | ULARGE_INTEGER TempTime2; 31 | std::uint64_t Result; 32 | 33 | ::GetSystemTimeAsFileTime(&TempTime); 34 | TempTime2.HighPart = TempTime.dwHighDateTime; 35 | TempTime2.LowPart = TempTime.dwLowDateTime; 36 | Result = TempTime2.QuadPart; 37 | 38 | Result = (Result / 10) - (std::uint64_t)11644473600000000ULL; 39 | 40 | return Result; 41 | } 42 | #else 43 | ProcessIDType AppInfo::GetCurrentProcessID() 44 | { 45 | return getpid(); 46 | } 47 | 48 | // POSIX pthreads. 49 | ThreadIDType AppInfo::GetCurrentThreadID() 50 | { 51 | return pthread_self(); 52 | } 53 | 54 | std::uint64_t AppInfo::GetUnixMicrosecondTime() 55 | { 56 | struct timeval TempTime; 57 | 58 | if (gettimeofday(&TempTime, NULL)) return 0; 59 | 60 | return (std::uint64_t)((std::uint64_t)TempTime.tv_sec * (std::uint64_t)1000000 + (std::uint64_t)TempTime.tv_usec); 61 | } 62 | #endif 63 | } 64 | } 65 | -------------------------------------------------------------------------------- /environment/environment_appinfo.h: -------------------------------------------------------------------------------- 1 | // Cross-platform application information. For Unicode application strings, see 'UTF8::AppInfo'. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_ENVIRONMENT_APPINFO 5 | #define CUBICLESOFT_ENVIRONMENT_APPINFO 6 | 7 | #include 8 | 9 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 10 | #include 11 | #else 12 | #include 13 | #include 14 | #include 15 | #endif 16 | 17 | namespace CubicleSoft 18 | { 19 | namespace Environment 20 | { 21 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 22 | typedef DWORD ProcessIDType; 23 | typedef DWORD ThreadIDType; 24 | #else 25 | typedef pid_t ProcessIDType; 26 | typedef pthread_t ThreadIDType; 27 | #endif 28 | 29 | class AppInfo 30 | { 31 | public: 32 | static ProcessIDType GetCurrentProcessID(); 33 | static ThreadIDType GetCurrentThreadID(); 34 | static std::uint64_t GetUnixMicrosecondTime(); 35 | }; 36 | } 37 | } 38 | 39 | #endif 40 | -------------------------------------------------------------------------------- /json/json_serializer.cpp: -------------------------------------------------------------------------------- 1 | // Cross-platform JSON serialization class. 2 | // (C) 2021 CubicleSoft. All Rights Reserved. 3 | 4 | #define _CRT_SECURE_NO_WARNINGS 5 | 6 | #include "json_serializer.h" 7 | #include 8 | 9 | namespace CubicleSoft 10 | { 11 | namespace JSON 12 | { 13 | Serializer::Serializer(const bool EscapeSlashes, const char *KeySplitter, const char *ValSplitter, size_t MaxDepth) 14 | : MxModes(MaxDepth), MxModeDepth(0), MxBuffer(NULL), MxBufferPos(0), MxBufferSize(0), MxEscapeSlashes(EscapeSlashes), MxKeySplitter(KeySplitter), MxValSplitter(ValSplitter) 15 | { 16 | MxModes[0] = ModeRootFirst; 17 | MxKeySplitterLen = strlen(KeySplitter); 18 | MxValSplitterLen = strlen(ValSplitter); 19 | } 20 | 21 | void Serializer::Reset() 22 | { 23 | MxModes[0] = ModeRootFirst; 24 | MxModeDepth = 0; 25 | MxBufferPos = 0; 26 | } 27 | 28 | void Serializer::SetBuffer(std::uint8_t *Buffer, size_t BufferSize) 29 | { 30 | MxBuffer = Buffer; 31 | MxBufferPos = 0; 32 | MxBufferSize = BufferSize - 1; 33 | } 34 | 35 | bool Serializer::StartObject(const char *Key) 36 | { 37 | if (MxModeDepth + 1 >= MxModes.GetSize() || !InternalAppendNextPrefix(Key, 1)) return false; 38 | 39 | MxBuffer[MxBufferPos++] = '{'; 40 | MxBuffer[MxBufferPos] = '\0'; 41 | 42 | MxModeDepth++; 43 | MxModes[MxModeDepth] = ModeObjectFirst; 44 | 45 | return true; 46 | } 47 | 48 | bool Serializer::EndObject() 49 | { 50 | if (MxModes[MxModeDepth] != ModeObjectFirst && MxModes[MxModeDepth] != ModeObjectNext) return false; 51 | 52 | if (MxBufferPos + 1 > MxBufferSize) return false; 53 | 54 | MxBuffer[MxBufferPos++] = '}'; 55 | MxBuffer[MxBufferPos] = '\0'; 56 | 57 | MxModeDepth--; 58 | 59 | return true; 60 | } 61 | 62 | bool Serializer::StartArray(const char *Key) 63 | { 64 | if (MxModeDepth + 1 >= MxModes.GetSize() || !InternalAppendNextPrefix(Key, 1)) return false; 65 | 66 | MxBuffer[MxBufferPos++] = '['; 67 | MxBuffer[MxBufferPos] = '\0'; 68 | 69 | MxModeDepth++; 70 | MxModes[MxModeDepth] = ModeArrayFirst; 71 | 72 | return true; 73 | } 74 | 75 | bool Serializer::EndArray() 76 | { 77 | if (MxModes[MxModeDepth] != ModeArrayFirst && MxModes[MxModeDepth] != ModeArrayNext) return false; 78 | 79 | if (MxBufferPos + 1 > MxBufferSize) return false; 80 | 81 | MxBuffer[MxBufferPos++] = ']'; 82 | MxBuffer[MxBufferPos] = '\0'; 83 | 84 | MxModeDepth--; 85 | 86 | return true; 87 | } 88 | 89 | bool Serializer::StartStr(const char *Key) 90 | { 91 | if (MxModeDepth + 1 >= MxModes.GetSize() || !InternalAppendNextPrefix(Key, 1)) return false; 92 | 93 | MxBuffer[MxBufferPos++] = '"'; 94 | MxBuffer[MxBufferPos] = '\0'; 95 | 96 | MxModeDepth++; 97 | MxModes[MxModeDepth] = ModeStr; 98 | 99 | return true; 100 | } 101 | 102 | bool Serializer::EndStr() 103 | { 104 | if (MxModes[MxModeDepth] != ModeStr) return false; 105 | 106 | if (MxBufferPos + 1 > MxBufferSize) return false; 107 | 108 | MxBuffer[MxBufferPos++] = '"'; 109 | MxBuffer[MxBufferPos] = '\0'; 110 | 111 | MxModeDepth--; 112 | 113 | return true; 114 | } 115 | 116 | bool Serializer::AppendNull(const char *Key) 117 | { 118 | if (!InternalAppendNextPrefix(Key, 4)) return false; 119 | 120 | strcpy((char *)MxBuffer + MxBufferPos, "null"); 121 | MxBufferPos += 4; 122 | 123 | return true; 124 | } 125 | 126 | bool Serializer::AppendBool(const char *Key, const bool Val) 127 | { 128 | if (!InternalAppendNextPrefix(Key, 5)) return false; 129 | 130 | if (Val) 131 | { 132 | strcpy((char *)MxBuffer + MxBufferPos, "true"); 133 | MxBufferPos += 4; 134 | } 135 | else 136 | { 137 | strcpy((char *)MxBuffer + MxBufferPos, "false"); 138 | MxBufferPos += 5; 139 | } 140 | 141 | return true; 142 | } 143 | 144 | bool Serializer::AppendInt(const char *Key, const std::int64_t Val) 145 | { 146 | char TempBuffer[44]; 147 | if (!IntToString(TempBuffer, sizeof(TempBuffer), Val)) return false; 148 | 149 | size_t x = strlen(TempBuffer); 150 | if (!InternalAppendNextPrefix(Key, x)) return false; 151 | 152 | strcpy((char *)MxBuffer + MxBufferPos, TempBuffer); 153 | MxBufferPos += x; 154 | 155 | return true; 156 | } 157 | 158 | bool Serializer::AppendUInt(const char *Key, const std::uint64_t Val) 159 | { 160 | char TempBuffer[44]; 161 | if (!IntToString(TempBuffer, sizeof(TempBuffer), Val)) return false; 162 | 163 | size_t x = strlen(TempBuffer); 164 | if (!InternalAppendNextPrefix(Key, x)) return false; 165 | 166 | strcpy((char *)MxBuffer + MxBufferPos, TempBuffer); 167 | MxBufferPos += x; 168 | 169 | return true; 170 | } 171 | 172 | bool Serializer::AppendDouble(const char *Key, const double Val, const size_t Precision) 173 | { 174 | char TempBuffer[100]; 175 | #if (defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64)) && defined(_MSC_VER) && _MSC_VER < 1900 176 | _snprintf_s(TempBuffer, sizeof(TempBuffer), _TRUNCATE, "%1.*g", Precision, Val); 177 | TempBuffer[sizeof(TempBuffer) - 1] = '\0'; 178 | #else 179 | snprintf(TempBuffer, sizeof(TempBuffer), "%1.*g", (int)Precision, Val); 180 | #endif 181 | 182 | size_t x = strlen(TempBuffer); 183 | if (!InternalAppendNextPrefix(Key, x)) return false; 184 | 185 | strcpy((char *)MxBuffer + MxBufferPos, TempBuffer); 186 | MxBufferPos += x; 187 | 188 | return true; 189 | } 190 | 191 | bool Serializer::AppendStr(const char *Key, const char *Val) 192 | { 193 | if (Val == NULL) return false; 194 | 195 | size_t x = CalculateStrSize(Val, false); 196 | if ((MxModes[MxModeDepth] != ModeStr && !InternalAppendNextPrefix(Key, x)) || (MxModes[MxModeDepth] == ModeStr && MxBufferPos + x > MxBufferSize)) return false; 197 | 198 | InternalAppendStr(Val); 199 | 200 | return true; 201 | } 202 | 203 | bool Serializer::AppendStr(const char *Key, const char *Val, const size_t Size) 204 | { 205 | if (Val == NULL) return false; 206 | 207 | size_t x = CalculateStrSize(Val, Size, false); 208 | if ((MxModes[MxModeDepth] != ModeStr && !InternalAppendNextPrefix(Key, x)) || (MxModes[MxModeDepth] == ModeStr && MxBufferPos + x > MxBufferSize)) return false; 209 | 210 | InternalAppendStr(Val, Size); 211 | 212 | return true; 213 | } 214 | 215 | bool Serializer::Append(const char *Val, size_t Size) 216 | { 217 | if (MxBufferPos + Size > MxBufferSize || Val == NULL) return false; 218 | 219 | while (Size) 220 | { 221 | MxBuffer[MxBufferPos++] = *Val++; 222 | Size--; 223 | } 224 | 225 | MxBuffer[MxBufferPos] = '\0'; 226 | 227 | return true; 228 | } 229 | 230 | bool Serializer::Finish() 231 | { 232 | while (MxModeDepth) 233 | { 234 | if (MxModes[MxModeDepth] == ModeObjectFirst || MxModes[MxModeDepth] == ModeObjectNext) 235 | { 236 | if (!EndObject()) return false; 237 | } 238 | if (MxModes[MxModeDepth] == ModeArrayFirst || MxModes[MxModeDepth] == ModeArrayNext) 239 | { 240 | if (!EndArray()) return false; 241 | } 242 | else if (MxModes[MxModeDepth] == ModeStr) 243 | { 244 | if (!EndStr()) return false; 245 | } 246 | } 247 | 248 | return true; 249 | } 250 | 251 | size_t Serializer::CalculateStrSize(const char *Val, bool AddKeySplitter) 252 | { 253 | size_t Result = (MxModes[MxModeDepth] != ModeStr ? 2 : 0); 254 | 255 | for (; *Val; Val++) 256 | { 257 | switch (*Val) 258 | { 259 | case '"': 260 | case '\\': 261 | case '\b': 262 | case '\f': 263 | case '\n': 264 | case '\r': 265 | case '\t': 266 | { 267 | Result++; 268 | 269 | break; 270 | } 271 | 272 | case '/': 273 | { 274 | if (MxEscapeSlashes) Result++; 275 | 276 | break; 277 | } 278 | } 279 | 280 | Result++; 281 | } 282 | 283 | if (AddKeySplitter) Result += MxKeySplitterLen; 284 | 285 | return Result; 286 | } 287 | 288 | size_t Serializer::CalculateStrSize(const char *Val, size_t Size, bool AddKeySplitter) 289 | { 290 | size_t Result = (MxModes[MxModeDepth] != ModeStr ? 2 : 0); 291 | 292 | while (Size) 293 | { 294 | switch (*Val) 295 | { 296 | case '"': 297 | case '\\': 298 | case '\b': 299 | case '\f': 300 | case '\n': 301 | case '\r': 302 | case '\t': 303 | { 304 | Result++; 305 | 306 | break; 307 | } 308 | 309 | case '/': 310 | { 311 | if (MxEscapeSlashes) Result++; 312 | 313 | break; 314 | } 315 | } 316 | 317 | Result++; 318 | Size--; 319 | } 320 | 321 | if (AddKeySplitter) Result += MxKeySplitterLen; 322 | 323 | return Result; 324 | } 325 | 326 | bool Serializer::InternalAppendNextPrefix(const char *Key, size_t ExtraSpace) 327 | { 328 | if (MxModes[MxModeDepth] == ModeRootNext || MxModes[MxModeDepth] == ModeStr || ((MxModes[MxModeDepth] == ModeObjectFirst || MxModes[MxModeDepth] == ModeObjectNext) && Key == NULL) || ((MxModes[MxModeDepth] == ModeRootFirst || MxModes[MxModeDepth] == ModeArrayFirst || MxModes[MxModeDepth] == ModeArrayNext) && Key != NULL)) return false; 329 | 330 | size_t x = (MxModes[MxModeDepth] == ModeObjectNext || MxModes[MxModeDepth] == ModeArrayNext ? MxValSplitterLen : 0); 331 | if (Key != NULL) x += CalculateStrSize(Key, true); 332 | 333 | if (MxBufferPos + x + ExtraSpace > MxBufferSize) return false; 334 | 335 | if (MxModes[MxModeDepth] == ModeObjectNext || MxModes[MxModeDepth] == ModeArrayNext) Append(MxValSplitter, MxValSplitterLen); 336 | else if (MxModes[MxModeDepth] == ModeObjectFirst) MxModes[MxModeDepth] = ModeObjectNext; 337 | else if (MxModes[MxModeDepth] == ModeArrayFirst) MxModes[MxModeDepth] = ModeArrayNext; 338 | else if (MxModes[MxModeDepth] == ModeRootFirst) MxModes[MxModeDepth] = ModeRootNext; 339 | 340 | if (Key != NULL) 341 | { 342 | InternalAppendStr(Key); 343 | Append(MxKeySplitter, MxKeySplitterLen); 344 | } 345 | 346 | return true; 347 | } 348 | 349 | // All calculations prior to calling these internal functions must be applied correctly to avoid an overflow. 350 | void Serializer::InternalAppendStr(const char *Val) 351 | { 352 | if (MxModes[MxModeDepth] != ModeStr) MxBuffer[MxBufferPos++] = '"'; 353 | 354 | while (*Val) 355 | { 356 | switch (*Val) 357 | { 358 | case '"': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = '"'; Val++; break; 359 | case '\\': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = '\\'; Val++; break; 360 | case '/': if (MxEscapeSlashes) { MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = '/'; } Val++; break; 361 | case '\b': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = 'b'; Val++; break; 362 | case '\f': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = 'f'; Val++; break; 363 | case '\n': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = 'n'; Val++; break; 364 | case '\r': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = 'r'; Val++; break; 365 | case '\t': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = 't'; Val++; break; 366 | default: MxBuffer[MxBufferPos++] = *Val++; break; 367 | } 368 | } 369 | 370 | if (MxModes[MxModeDepth] != ModeStr) MxBuffer[MxBufferPos++] = '"'; 371 | MxBuffer[MxBufferPos] = '\0'; 372 | } 373 | 374 | void Serializer::InternalAppendStr(const char *Val, size_t Size) 375 | { 376 | if (MxModes[MxModeDepth] != ModeStr) MxBuffer[MxBufferPos++] = '"'; 377 | 378 | while (Size) 379 | { 380 | switch (*Val) 381 | { 382 | case '"': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = '"'; Val++; break; 383 | case '\\': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = '\\'; Val++; break; 384 | case '/': if (MxEscapeSlashes) { MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = '/'; } Val++; break; 385 | case '\b': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = 'b'; Val++; break; 386 | case '\f': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = 'f'; Val++; break; 387 | case '\n': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = 'n'; Val++; break; 388 | case '\r': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = 'r'; Val++; break; 389 | case '\t': MxBuffer[MxBufferPos++] = '\\'; MxBuffer[MxBufferPos++] = 't'; Val++; break; 390 | default: MxBuffer[MxBufferPos++] = *Val++; break; 391 | } 392 | 393 | Size--; 394 | } 395 | 396 | if (MxModes[MxModeDepth] != ModeStr) MxBuffer[MxBufferPos++] = '"'; 397 | MxBuffer[MxBufferPos] = '\0'; 398 | } 399 | 400 | // Swiped and slightly modified from Int::ToString(). 401 | bool Serializer::IntToString(char *Result, size_t Size, std::uint64_t Num) 402 | { 403 | if (Size < 2) return false; 404 | 405 | size_t x = Size; 406 | 407 | Result[--x] = '\0'; 408 | if (!Num) Result[--x] = '0'; 409 | else 410 | { 411 | while (Num && x) 412 | { 413 | Result[--x] = (char)(Num % 10) + '0'; 414 | Num /= 10; 415 | } 416 | 417 | if (Num) return false; 418 | } 419 | 420 | memmove(Result, Result + x, Size - x); 421 | 422 | return true; 423 | } 424 | 425 | bool Serializer::IntToString(char *Result, size_t Size, std::int64_t Num) 426 | { 427 | if (Num >= 0) return IntToString(Result, Size, (std::uint64_t)Num); 428 | 429 | if (Size < 2) return false; 430 | Result[0] = '-'; 431 | 432 | return IntToString(Result + 1, Size - 1, (std::uint64_t)-Num); 433 | } 434 | } 435 | } 436 | -------------------------------------------------------------------------------- /json/json_serializer.h: -------------------------------------------------------------------------------- 1 | // Cross-platform JSON serialization class. 2 | // (C) 2021 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_JSON_SERIALIZER 5 | #define CUBICLESOFT_JSON_SERIALIZER 6 | 7 | #include 8 | #include 9 | #include 10 | #include 11 | 12 | #include "../templates/static_vector.h" 13 | 14 | namespace CubicleSoft 15 | { 16 | namespace JSON 17 | { 18 | class Serializer 19 | { 20 | public: 21 | enum ModeType 22 | { 23 | ModeRootFirst, 24 | ModeRootNext, 25 | ModeObjectFirst, 26 | ModeObjectNext, 27 | ModeArrayFirst, 28 | ModeArrayNext, 29 | ModeStr 30 | }; 31 | 32 | Serializer(const bool EscapeSlashes = false, const char *KeySplitter = ": ", const char *ValSplitter = ", ", size_t MaxDepth = 512); 33 | 34 | void Reset(); 35 | 36 | void SetBuffer(std::uint8_t *Buffer, size_t BufferSize); 37 | inline std::uint8_t *GetBuffer() const { return MxBuffer; } 38 | inline size_t GetCurrPos() const { return MxBufferPos; } 39 | inline size_t GetBufferSize() const { return MxBufferSize; } 40 | inline void ResetPos() { MxBufferPos = 0; } 41 | 42 | inline ModeType GetCurrMode() { return MxModes[MxModeDepth]; } 43 | 44 | inline void SetEscapeSlashes(const bool EscapeSlashes) { MxEscapeSlashes = EscapeSlashes; } 45 | inline void SetKeySplitter(const char *KeySplitter) { MxKeySplitter = KeySplitter; MxKeySplitterLen = strlen(KeySplitter); } 46 | inline void SetValSplitter(const char *ValSplitter) { MxValSplitter = ValSplitter; MxValSplitterLen = strlen(ValSplitter); } 47 | 48 | inline bool IsKeyRequired() { return (MxModes[MxModeDepth] == ModeObjectFirst || MxModes[MxModeDepth] == ModeObjectNext); } 49 | 50 | bool StartObject(const char *Key = NULL); 51 | bool EndObject(); 52 | 53 | bool StartArray(const char *Key = NULL); 54 | bool EndArray(); 55 | 56 | bool StartStr(const char *Key = NULL); 57 | bool EndStr(); 58 | 59 | bool AppendNull(const char *Key = NULL); 60 | bool AppendBool(const char *Key, const bool Val); 61 | bool AppendInt(const char *Key, const std::int64_t Val); 62 | bool AppendUInt(const char *Key, const std::uint64_t Val); 63 | bool AppendDouble(const char *Key, const double Val, const size_t Precision = 100); 64 | bool AppendStr(const char *Key, const char *Val); 65 | bool AppendStr(const char *Key, const char *Val, const size_t Size); 66 | 67 | inline bool AppendBool(const bool Val) { return AppendBool(NULL, Val); } 68 | inline bool AppendInt(const std::int64_t Val) { return AppendInt(NULL, Val); } 69 | inline bool AppendUInt(const std::uint64_t Val) { return AppendUInt(NULL, Val); } 70 | inline bool AppendDouble(const double Val, const size_t Precision = 16) { return AppendDouble(NULL, Val, Precision); } 71 | inline bool AppendStr(const char *Val) { return AppendStr(NULL, Val); } 72 | inline bool AppendStr(const char *Val, const size_t Size) { return AppendStr(NULL, Val, Size); } 73 | 74 | // Intended for appending arbitrary content (e.g. whitespace). 75 | bool Append(const char *Val, size_t Size); 76 | inline bool Append(const char *Val) { return Append(Val, strlen(Val)); }; 77 | 78 | bool Finish(); 79 | 80 | // Calculate the final size of a string including quotes. 81 | size_t CalculateStrSize(const char *Val, bool AddKeySplitter = false); 82 | size_t CalculateStrSize(const char *Val, size_t Size, bool AddKeySplitter = false); 83 | 84 | private: 85 | bool InternalAppendNextPrefix(const char *Key, size_t ExtraSpace); 86 | void InternalAppendStr(const char *Val); 87 | void InternalAppendStr(const char *Val, size_t Size); 88 | static bool IntToString(char *Result, size_t Size, std::uint64_t Num); 89 | static bool IntToString(char *Result, size_t Size, std::int64_t Num); 90 | 91 | StaticVector MxModes; 92 | size_t MxModeDepth; 93 | 94 | std::uint8_t *MxBuffer; 95 | size_t MxBufferPos, MxBufferSize; 96 | 97 | bool MxEscapeSlashes; 98 | const char *MxKeySplitter, *MxValSplitter; 99 | size_t MxKeySplitterLen, MxValSplitterLen; 100 | }; 101 | } 102 | } 103 | 104 | #endif 105 | -------------------------------------------------------------------------------- /security/security_csprng.cpp: -------------------------------------------------------------------------------- 1 | // Cross-platform CSPRNG. 2 | // (C) 2013 CubicleSoft. All Rights Reserved. 3 | 4 | #include "security_csprng.h" 5 | 6 | namespace CubicleSoft 7 | { 8 | namespace Security 9 | { 10 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 11 | // Windows. 12 | CSPRNG::CSPRNG(bool) 13 | { 14 | HMODULE TempModule = ::LoadLibraryA("ADVAPI32.DLL"); 15 | if (TempModule == NULL) MxRtlGenRandom = NULL; 16 | else MxRtlGenRandom = (RtlGenRandomFunc)::GetProcAddress(TempModule, "SystemFunction036"); 17 | } 18 | 19 | CSPRNG::~CSPRNG() 20 | { 21 | } 22 | 23 | bool CSPRNG::GetBytes(std::uint8_t *Result, size_t Size) 24 | { 25 | if (MxRtlGenRandom == NULL) return false; 26 | 27 | return ((*MxRtlGenRandom)(Result, (ULONG)Size) != FALSE); 28 | } 29 | #else 30 | // Linux. 31 | CSPRNG::CSPRNG(bool CryptoSafe) 32 | { 33 | if (CryptoSafe) MxURandom = open("/dev/random", O_RDONLY); 34 | else 35 | { 36 | MxURandom = open("/dev/arandom", O_RDONLY); 37 | if (MxURandom < 0) MxURandom = open("/dev/urandom", O_RDONLY); 38 | } 39 | } 40 | 41 | CSPRNG::~CSPRNG() 42 | { 43 | close(MxURandom); 44 | } 45 | 46 | bool CSPRNG::GetBytes(std::uint8_t *Result, size_t Size) 47 | { 48 | if (MxURandom < 0) return false; 49 | 50 | ssize_t BytesRead; 51 | 52 | do 53 | { 54 | do 55 | { 56 | BytesRead = read(MxURandom, Result, Size); 57 | } while (BytesRead < 0 && errno == EINTR); 58 | } while (BytesRead != (ssize_t)Size); 59 | 60 | if (BytesRead < 0) return false; 61 | 62 | return true; 63 | } 64 | #endif 65 | 66 | // All platforms. 67 | bool CSPRNG::GetInteger(std::uint64_t &Result, std::uint64_t Min, std::uint64_t Max) 68 | { 69 | std::uint64_t Max2, Mask; 70 | 71 | if (Max < Min) 72 | { 73 | Max2 = Min; 74 | Min = Max; 75 | Max = Max2; 76 | } 77 | 78 | Max2 = Max - Min; 79 | Mask = 0; 80 | while (Max2 > Mask) Mask = (Mask << 1) | 0x01; 81 | 82 | // The correct way to get a random integer in a range along bit boundaries 83 | // is to throw away values outside the allowed range. 84 | do 85 | { 86 | if (!GetBytes((std::uint8_t *)&Result, sizeof(std::uint64_t))) return false; 87 | Result &= Mask; 88 | 89 | } while (Result > Max2); 90 | 91 | Result += Min; 92 | 93 | return true; 94 | } 95 | } 96 | } 97 | -------------------------------------------------------------------------------- /security/security_csprng.h: -------------------------------------------------------------------------------- 1 | // Cross-platform CSPRNG. 2 | // (C) 2013 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_SECURITY_CSPRNG 5 | #define CUBICLESOFT_SECURITY_CSPRNG 6 | 7 | #include 8 | #include 9 | 10 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 11 | #include 12 | #else 13 | #include 14 | #include 15 | #include 16 | #endif 17 | 18 | namespace CubicleSoft 19 | { 20 | namespace Security 21 | { 22 | class CSPRNG 23 | { 24 | public: 25 | CSPRNG(bool CryptoSafe); 26 | ~CSPRNG(); 27 | 28 | bool GetBytes(std::uint8_t *Result, size_t Size); 29 | bool GetInteger(std::uint64_t &Result, std::uint64_t Min, std::uint64_t Max); 30 | 31 | private: 32 | // Deny copy constructor and assignment operator. Use a (smart) pointer instead. 33 | CSPRNG(const CSPRNG &); 34 | CSPRNG &operator=(const CSPRNG &); 35 | 36 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 37 | typedef BOOL (APIENTRY *RtlGenRandomFunc)(PVOID, ULONG); 38 | 39 | RtlGenRandomFunc MxRtlGenRandom; 40 | #else 41 | int MxURandom; 42 | #endif 43 | }; 44 | } 45 | } 46 | 47 | #endif 48 | -------------------------------------------------------------------------------- /sync/sync_event.cpp: -------------------------------------------------------------------------------- 1 | // Cross-platform, optionally named (cross-process), event object (e.g. for producer/consumer queues). 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #include "sync_event.h" 5 | 6 | namespace CubicleSoft 7 | { 8 | namespace Sync 9 | { 10 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 11 | // Windows. 12 | Event::Event() : MxWinWaitEvent(NULL) 13 | { 14 | } 15 | 16 | Event::~Event() 17 | { 18 | if (MxWinWaitEvent != NULL) ::CloseHandle(MxWinWaitEvent); 19 | } 20 | 21 | bool Event::Create(const char *Name, bool Manual, bool Prefire) 22 | { 23 | if (MxWinWaitEvent != NULL) ::CloseHandle(MxWinWaitEvent); 24 | 25 | MxWinWaitEvent = NULL; 26 | 27 | SECURITY_ATTRIBUTES SecAttr; 28 | 29 | SecAttr.nLength = sizeof(SecAttr); 30 | SecAttr.lpSecurityDescriptor = NULL; 31 | SecAttr.bInheritHandle = TRUE; 32 | 33 | MxWinWaitEvent = ::CreateEventA(&SecAttr, (BOOL)Manual, (Prefire ? TRUE : FALSE), Name); 34 | 35 | if (MxWinWaitEvent == NULL) return false; 36 | 37 | return true; 38 | } 39 | 40 | bool Event::Wait(std::uint32_t Wait) 41 | { 42 | if (MxWinWaitEvent == NULL) return false; 43 | 44 | DWORD Result = ::WaitForSingleObject(MxWinWaitEvent, Wait); 45 | if (Result != WAIT_OBJECT_0) return false; 46 | 47 | return true; 48 | } 49 | 50 | bool Event::Fire() 51 | { 52 | if (MxWinWaitEvent == NULL) return false; 53 | 54 | if (!::SetEvent(MxWinWaitEvent)) return false; 55 | 56 | return true; 57 | } 58 | 59 | bool Event::Reset() 60 | { 61 | if (MxWinWaitEvent == NULL) return false; 62 | 63 | if (!::ResetEvent(MxWinWaitEvent)) return false; 64 | 65 | return true; 66 | } 67 | #else 68 | // POSIX pthreads. 69 | Event::Event() : MxNamed(false), MxMem(NULL) 70 | { 71 | } 72 | 73 | Event::~Event() 74 | { 75 | if (MxMem != NULL) 76 | { 77 | if (MxNamed) Util::UnmapUnixNamedMem(MxMem, Util::GetUnixEventSize()); 78 | else 79 | { 80 | Util::FreeUnixEvent(MxPthreadEvent); 81 | 82 | delete[] MxMem; 83 | } 84 | } 85 | } 86 | 87 | bool Event::Create(const char *Name, bool Manual, bool Prefire) 88 | { 89 | if (MxMem != NULL) 90 | { 91 | if (MxNamed) Util::UnmapUnixNamedMem(MxMem, Util::GetUnixEventSize()); 92 | else 93 | { 94 | Util::FreeUnixEvent(MxPthreadEvent); 95 | 96 | delete[] MxMem; 97 | } 98 | } 99 | 100 | MxMem = NULL; 101 | 102 | size_t Pos, TempSize = Util::GetUnixEventSize(); 103 | MxNamed = (Name != NULL); 104 | int Result = Util::InitUnixNamedMem(MxMem, Pos, "/Sync_Event", Name, TempSize); 105 | 106 | if (Result < 0) return false; 107 | 108 | Util::GetUnixEvent(MxPthreadEvent, MxMem + Pos); 109 | 110 | // Handle the first time this event has been opened. 111 | if (Result == 0) 112 | { 113 | Util::InitUnixEvent(MxPthreadEvent, MxNamed, Manual, Prefire); 114 | 115 | if (MxNamed) Util::UnixNamedMemReady(MxMem); 116 | } 117 | 118 | return true; 119 | } 120 | 121 | bool Event::Wait(std::uint32_t WaitAmt) 122 | { 123 | if (MxMem == NULL) return false; 124 | 125 | // Wait for the event. 126 | return Util::WaitForUnixEvent(MxPthreadEvent, WaitAmt); 127 | } 128 | 129 | bool Event::Fire() 130 | { 131 | if (MxMem == NULL) return false; 132 | 133 | return Util::FireUnixEvent(MxPthreadEvent); 134 | } 135 | 136 | bool Event::Reset() 137 | { 138 | if (MxMem == NULL) return false; 139 | 140 | return Util::ResetUnixEvent(MxPthreadEvent); 141 | } 142 | #endif 143 | } 144 | } 145 | -------------------------------------------------------------------------------- /sync/sync_event.h: -------------------------------------------------------------------------------- 1 | // Cross-platform, optionally named (cross-process), event object (e.g. for producer/consumer queues). 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_SYNC_EVENT 5 | #define CUBICLESOFT_SYNC_EVENT 6 | 7 | #include "sync_util.h" 8 | 9 | namespace CubicleSoft 10 | { 11 | namespace Sync 12 | { 13 | class Event 14 | { 15 | public: 16 | Event(); 17 | ~Event(); 18 | 19 | bool Create(const char *Name = NULL, bool Manual = false, bool Prefire = false); 20 | 21 | // Wait time is in milliseconds. Granularity is in 5ms intervals on some platforms. 22 | bool Wait(std::uint32_t Wait = INFINITE); 23 | 24 | // Lets a thread through that is waiting. Lets multiple threads through that are waiting if the event is 'manual'. 25 | bool Fire(); 26 | 27 | // Only use when the event is 'manual'. 28 | bool Reset(); 29 | 30 | private: 31 | // Deny copy constructor and assignment operator. Use a (smart) pointer instead. 32 | Event(const Event &); 33 | Event &operator=(const Event &); 34 | 35 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 36 | HANDLE MxWinWaitEvent; 37 | #else 38 | bool MxNamed; 39 | char *MxMem; 40 | Util::UnixEventWrapper MxPthreadEvent; 41 | #endif 42 | }; 43 | } 44 | } 45 | 46 | #endif 47 | -------------------------------------------------------------------------------- /sync/sync_mutex.cpp: -------------------------------------------------------------------------------- 1 | // Cross-platform, optionally named (cross-process), self-counting (recursive) mutex. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #include "sync_mutex.h" 5 | 6 | namespace CubicleSoft 7 | { 8 | namespace Sync 9 | { 10 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 11 | // Windows. 12 | Mutex::Mutex() : MxWinMutex(NULL), MxOwnerID(0), MxCount(0) 13 | { 14 | ::InitializeCriticalSection(&MxWinCritSection); 15 | } 16 | 17 | Mutex::~Mutex() 18 | { 19 | Unlock(true); 20 | 21 | if (MxWinMutex != NULL) ::CloseHandle(MxWinMutex); 22 | ::DeleteCriticalSection(&MxWinCritSection); 23 | } 24 | 25 | bool Mutex::Create(const char *Name) 26 | { 27 | ::EnterCriticalSection(&MxWinCritSection); 28 | 29 | if (MxCount > 0) 30 | { 31 | if (MxOwnerID == Util::GetCurrentThreadID()) 32 | { 33 | ::ReleaseMutex(MxWinMutex); 34 | ::CloseHandle(MxWinMutex); 35 | MxWinMutex = NULL; 36 | MxCount = 0; 37 | MxOwnerID = 0; 38 | } 39 | else 40 | { 41 | ::LeaveCriticalSection(&MxWinCritSection); 42 | 43 | return false; 44 | } 45 | } 46 | 47 | SECURITY_ATTRIBUTES SecAttr; 48 | 49 | SecAttr.nLength = sizeof(SecAttr); 50 | SecAttr.lpSecurityDescriptor = NULL; 51 | SecAttr.bInheritHandle = TRUE; 52 | 53 | MxWinMutex = ::CreateMutexA(&SecAttr, FALSE, Name); 54 | if (MxWinMutex == NULL) 55 | { 56 | ::LeaveCriticalSection(&MxWinCritSection); 57 | 58 | return false; 59 | } 60 | 61 | ::LeaveCriticalSection(&MxWinCritSection); 62 | 63 | return true; 64 | } 65 | 66 | bool Mutex::Lock(uint32_t Wait) 67 | { 68 | ::EnterCriticalSection(&MxWinCritSection); 69 | 70 | // Make sure the mutex exists. 71 | if (MxWinMutex == NULL) 72 | { 73 | ::LeaveCriticalSection(&MxWinCritSection); 74 | 75 | return false; 76 | } 77 | 78 | // Check to see if this is already owned by the calling thread. 79 | if (MxOwnerID == Util::GetCurrentThreadID()) 80 | { 81 | MxCount++; 82 | ::LeaveCriticalSection(&MxWinCritSection); 83 | 84 | return true; 85 | } 86 | 87 | ::LeaveCriticalSection(&MxWinCritSection); 88 | 89 | // Acquire the mutex. 90 | DWORD Result = ::WaitForSingleObject(MxWinMutex, (DWORD)Wait); 91 | if (Result != WAIT_OBJECT_0) return false; 92 | 93 | ::EnterCriticalSection(&MxWinCritSection); 94 | MxOwnerID = Util::GetCurrentThreadID(); 95 | MxCount = 1; 96 | ::LeaveCriticalSection(&MxWinCritSection); 97 | 98 | return true; 99 | } 100 | 101 | bool Mutex::Unlock(bool All) 102 | { 103 | ::EnterCriticalSection(&MxWinCritSection); 104 | 105 | // Make sure the mutex exists and make sure it is owned by the calling thread. 106 | if (MxWinMutex == NULL || MxOwnerID != Util::GetCurrentThreadID()) 107 | { 108 | ::LeaveCriticalSection(&MxWinCritSection); 109 | 110 | return false; 111 | } 112 | 113 | if (All) MxCount = 1; 114 | MxCount--; 115 | if (!MxCount) 116 | { 117 | MxOwnerID = 0; 118 | 119 | // Release the mutex. 120 | ::ReleaseMutex(MxWinMutex); 121 | } 122 | 123 | ::LeaveCriticalSection(&MxWinCritSection); 124 | 125 | return true; 126 | } 127 | #else 128 | // POSIX pthreads. 129 | Mutex::Mutex() : MxNamed(false), MxMem(NULL), MxOwnerID(0), MxCount(0) 130 | { 131 | pthread_mutex_init(&MxPthreadCritSection, NULL); 132 | } 133 | 134 | Mutex::~Mutex() 135 | { 136 | Unlock(true); 137 | 138 | if (MxMem != NULL) 139 | { 140 | if (MxNamed) Util::UnmapUnixNamedMem(MxMem, Util::GetUnixSemaphoreSize()); 141 | else 142 | { 143 | Util::FreeUnixSemaphore(MxPthreadMutex); 144 | 145 | delete[] MxMem; 146 | } 147 | } 148 | 149 | pthread_mutex_destroy(&MxPthreadCritSection); 150 | } 151 | 152 | bool Mutex::Create(const char *Name) 153 | { 154 | if (pthread_mutex_lock(&MxPthreadCritSection) != 0) return false; 155 | 156 | if (MxCount > 0) 157 | { 158 | if (MxOwnerID == Util::GetCurrentThreadID()) 159 | { 160 | if (MxNamed) Util::UnmapUnixNamedMem(MxMem, Util::GetUnixSemaphoreSize()); 161 | else 162 | { 163 | Util::FreeUnixSemaphore(MxPthreadMutex); 164 | 165 | delete[] MxMem; 166 | } 167 | 168 | MxNamed = false; 169 | MxMem = NULL; 170 | MxCount = 0; 171 | MxOwnerID = 0; 172 | } 173 | else 174 | { 175 | pthread_mutex_unlock(&MxPthreadCritSection); 176 | 177 | return false; 178 | } 179 | } 180 | 181 | size_t Pos, TempSize = Util::GetUnixSemaphoreSize(); 182 | MxNamed = (Name != NULL); 183 | int Result = Util::InitUnixNamedMem(MxMem, Pos, "/Sync_Mutex", Name, TempSize); 184 | 185 | if (Result < 0) 186 | { 187 | pthread_mutex_unlock(&MxPthreadCritSection); 188 | 189 | return false; 190 | } 191 | 192 | Util::GetUnixSemaphore(MxPthreadMutex, MxMem + Pos); 193 | 194 | // Handle the first time this mutex has been opened. 195 | if (Result == 0) 196 | { 197 | Util::InitUnixSemaphore(MxPthreadMutex, MxNamed, 1, 1); 198 | 199 | if (MxNamed) Util::UnixNamedMemReady(MxMem); 200 | } 201 | 202 | pthread_mutex_unlock(&MxPthreadCritSection); 203 | 204 | return true; 205 | } 206 | 207 | bool Mutex::Lock(uint32_t Wait) 208 | { 209 | if (pthread_mutex_lock(&MxPthreadCritSection) != 0) return false; 210 | 211 | // Check to see if this mutex is already owned by the calling thread. 212 | if (MxOwnerID == Util::GetCurrentThreadID()) 213 | { 214 | MxCount++; 215 | pthread_mutex_unlock(&MxPthreadCritSection); 216 | 217 | return true; 218 | } 219 | 220 | pthread_mutex_unlock(&MxPthreadCritSection); 221 | 222 | if (!Util::WaitForUnixSemaphore(MxPthreadMutex, Wait)) return false; 223 | 224 | pthread_mutex_lock(&MxPthreadCritSection); 225 | MxOwnerID = Util::GetCurrentThreadID(); 226 | MxCount = 1; 227 | pthread_mutex_unlock(&MxPthreadCritSection); 228 | 229 | return true; 230 | } 231 | 232 | bool Mutex::Unlock(bool All) 233 | { 234 | if (pthread_mutex_lock(&MxPthreadCritSection) != 0) return false; 235 | 236 | // Make sure the mutex is owned by the calling thread. 237 | if (MxOwnerID != Util::GetCurrentThreadID()) 238 | { 239 | pthread_mutex_unlock(&MxPthreadCritSection); 240 | 241 | return false; 242 | } 243 | 244 | if (All) MxCount = 1; 245 | MxCount--; 246 | if (!MxCount) 247 | { 248 | MxOwnerID = 0; 249 | 250 | // Release the mutex. 251 | Util::ReleaseUnixSemaphore(MxPthreadMutex, NULL); 252 | } 253 | 254 | pthread_mutex_unlock(&MxPthreadCritSection); 255 | 256 | return true; 257 | } 258 | #endif 259 | 260 | 261 | Mutex::AutoUnlock::AutoUnlock(Mutex *LockPtr) 262 | { 263 | MxLock = LockPtr; 264 | } 265 | 266 | Mutex::AutoUnlock::~AutoUnlock() 267 | { 268 | MxLock->Unlock(); 269 | } 270 | } 271 | } 272 | -------------------------------------------------------------------------------- /sync/sync_mutex.h: -------------------------------------------------------------------------------- 1 | // Cross-platform, optionally named (cross-process), self-counting (recursive) mutex. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_SYNC_MUTEX 5 | #define CUBICLESOFT_SYNC_MUTEX 6 | 7 | #include "sync_util.h" 8 | 9 | namespace CubicleSoft 10 | { 11 | namespace Sync 12 | { 13 | class Mutex 14 | { 15 | public: 16 | Mutex(); 17 | ~Mutex(); 18 | 19 | bool Create(const char *Name = NULL); 20 | 21 | // Wait is in milliseconds. Granularity is in 5ms intervals on some platforms. 22 | bool Lock(std::uint32_t Wait = INFINITE); 23 | 24 | bool Unlock(bool All = false); 25 | 26 | // Automatically unlock a mutex when leaving the current scope. 27 | class AutoUnlock 28 | { 29 | public: 30 | AutoUnlock(Mutex *LockPtr); 31 | ~AutoUnlock(); 32 | 33 | private: 34 | // Deny copy constructor and assignment operator. Use a (smart) pointer instead. 35 | AutoUnlock(const AutoUnlock &); 36 | AutoUnlock &operator=(const AutoUnlock &); 37 | 38 | Mutex *MxLock; 39 | }; 40 | 41 | private: 42 | // Deny copy constructor and assignment operator. Use a (smart) pointer instead. 43 | Mutex(const Mutex &); 44 | Mutex &operator=(const Mutex &); 45 | 46 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 47 | CRITICAL_SECTION MxWinCritSection; 48 | 49 | HANDLE MxWinMutex; 50 | #else 51 | pthread_mutex_t MxPthreadCritSection; 52 | 53 | bool MxNamed; 54 | char *MxMem; 55 | Util::UnixSemaphoreWrapper MxPthreadMutex; 56 | #endif 57 | 58 | volatile ThreadIDType MxOwnerID; 59 | volatile std::uint32_t MxCount; 60 | }; 61 | } 62 | } 63 | 64 | #endif 65 | -------------------------------------------------------------------------------- /sync/sync_readwritelock.cpp: -------------------------------------------------------------------------------- 1 | // Cross-platform, optionally named (cross-process), reader/writer lock. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #define _CRT_SECURE_NO_WARNINGS 5 | 6 | #include 7 | #include 8 | 9 | #include "sync_readwritelock.h" 10 | 11 | namespace CubicleSoft 12 | { 13 | namespace Sync 14 | { 15 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 16 | // Windows. 17 | ReadWriteLock::ReadWriteLock() : MxWinRSemMutex(NULL), MxWinRSemaphore(NULL), MxWinRWaitEvent(NULL), MxWinWWaitMutex(NULL) 18 | { 19 | } 20 | 21 | ReadWriteLock::~ReadWriteLock() 22 | { 23 | if (MxWinWWaitMutex != NULL) ::CloseHandle(MxWinWWaitMutex); 24 | if (MxWinRWaitEvent != NULL) ::CloseHandle(MxWinRWaitEvent); 25 | if (MxWinRSemaphore != NULL) ::CloseHandle(MxWinRSemaphore); 26 | if (MxWinRSemMutex != NULL) ::CloseHandle(MxWinRSemMutex); 27 | } 28 | 29 | bool ReadWriteLock::Create(const char *Name) 30 | { 31 | if (MxWinWWaitMutex != NULL) ::CloseHandle(MxWinWWaitMutex); 32 | if (MxWinRWaitEvent != NULL) ::CloseHandle(MxWinRWaitEvent); 33 | if (MxWinRSemaphore != NULL) ::CloseHandle(MxWinRSemaphore); 34 | if (MxWinRSemMutex != NULL) ::CloseHandle(MxWinRSemMutex); 35 | 36 | MxWinWWaitMutex = NULL; 37 | MxWinRWaitEvent = NULL; 38 | MxWinRSemaphore = NULL; 39 | MxWinRSemMutex = NULL; 40 | 41 | char *Name2; 42 | SECURITY_ATTRIBUTES SecAttr; 43 | 44 | SecAttr.nLength = sizeof(SecAttr); 45 | SecAttr.lpSecurityDescriptor = NULL; 46 | SecAttr.bInheritHandle = TRUE; 47 | 48 | if (Name == NULL) Name2 = NULL; 49 | else Name2 = new char[strlen(Name) + 20]; 50 | 51 | // Create the mutexes, semaphore, and event objects. 52 | if (Name2 != NULL) sprintf(Name2, "%s-Sync_ReadWrite-0", Name); 53 | MxWinRSemMutex = ::CreateSemaphoreA(&SecAttr, 1, 1, Name2); 54 | if (Name2 != NULL) sprintf(Name2, "%s-Sync_ReadWrite-1", Name); 55 | MxWinRSemaphore = ::CreateSemaphoreA(&SecAttr, LONG_MAX, LONG_MAX, Name2); 56 | if (Name2 != NULL) sprintf(Name2, "%s-Sync_ReadWrite-2", Name); 57 | MxWinRWaitEvent = ::CreateEventA(&SecAttr, TRUE, TRUE, Name2); 58 | if (Name2 != NULL) sprintf(Name2, "%s-Sync_ReadWrite-3", Name); 59 | MxWinWWaitMutex = ::CreateSemaphoreA(&SecAttr, 1, 1, Name2); 60 | 61 | if (Name2 != NULL) delete[] Name2; 62 | 63 | if (MxWinRSemMutex == NULL || MxWinRSemaphore == NULL || MxWinRWaitEvent == NULL || MxWinWWaitMutex == NULL) return false; 64 | 65 | return true; 66 | } 67 | 68 | bool ReadWriteLock::ReadLock(std::uint32_t Wait) 69 | { 70 | std::uint64_t StartTime, CurrTime; 71 | 72 | if (MxWinRSemaphore == NULL || MxWinRSemMutex == NULL || MxWinRWaitEvent == NULL || MxWinWWaitMutex == NULL) return false; 73 | 74 | // Get current time in milliseconds. 75 | StartTime = (Wait == INFINITE ? 0 : Util::GetUnixMicrosecondTime() / 1000000); 76 | 77 | // Acquire the write lock mutex. Guarantees that readers can't starve the writer. 78 | DWORD Result = ::WaitForSingleObject(MxWinWWaitMutex, Wait); 79 | if (Result != WAIT_OBJECT_0) return false; 80 | 81 | // Acquire the semaphore mutex. 82 | CurrTime = (Wait == INFINITE ? 0 : Util::GetUnixMicrosecondTime() / 1000000); 83 | if (Wait < CurrTime - StartTime || ::WaitForSingleObject(MxWinRSemMutex, Wait - (DWORD)(CurrTime - StartTime)) != WAIT_OBJECT_0) 84 | { 85 | ::ReleaseSemaphore(MxWinWWaitMutex, 1, NULL); 86 | 87 | return false; 88 | } 89 | 90 | // Acquire the semaphore. 91 | CurrTime = (Wait == INFINITE ? 0 : Util::GetUnixMicrosecondTime() / 1000000); 92 | if (Wait < CurrTime - StartTime || ::WaitForSingleObject(MxWinRSemaphore, Wait - (DWORD)(CurrTime - StartTime)) != WAIT_OBJECT_0) 93 | { 94 | ::ReleaseSemaphore(MxWinRSemMutex, 1, NULL); 95 | ::ReleaseSemaphore(MxWinWWaitMutex, 1, NULL); 96 | 97 | return false; 98 | } 99 | 100 | // Update the event state. 101 | if (!::ResetEvent(MxWinRWaitEvent)) 102 | { 103 | ::ReleaseSemaphore(MxWinRSemaphore, 1, NULL); 104 | ::ReleaseSemaphore(MxWinRSemMutex, 1, NULL); 105 | ::ReleaseSemaphore(MxWinWWaitMutex, 1, NULL); 106 | 107 | return false; 108 | } 109 | 110 | // Release the mutexes. 111 | ::ReleaseSemaphore(MxWinRSemMutex, 1, NULL); 112 | ::ReleaseSemaphore(MxWinWWaitMutex, 1, NULL); 113 | 114 | return true; 115 | } 116 | 117 | bool ReadWriteLock::WriteLock(std::uint32_t Wait) 118 | { 119 | std::uint64_t StartTime, CurrTime; 120 | 121 | if (MxWinRWaitEvent == NULL || MxWinWWaitMutex == NULL) return false; 122 | 123 | // Get current time in milliseconds. 124 | StartTime = (Wait == INFINITE ? 0 : Util::GetUnixMicrosecondTime() / 1000000); 125 | 126 | // Acquire the write lock mutex. 127 | DWORD Result = ::WaitForSingleObject(MxWinWWaitMutex, Wait); 128 | if (Result != WAIT_OBJECT_0) return false; 129 | 130 | // Wait for readers to reach zero. 131 | CurrTime = (Wait == INFINITE ? 0 : Util::GetUnixMicrosecondTime() / 1000000); 132 | if (Wait < CurrTime - StartTime || ::WaitForSingleObject(MxWinRWaitEvent, Wait - (DWORD)(CurrTime - StartTime)) != WAIT_OBJECT_0) 133 | { 134 | ::ReleaseSemaphore(MxWinWWaitMutex, 1, NULL); 135 | 136 | return false; 137 | } 138 | 139 | return true; 140 | } 141 | 142 | bool ReadWriteLock::ReadUnlock() 143 | { 144 | if (MxWinRSemMutex == NULL || MxWinRSemaphore == NULL || MxWinRWaitEvent == NULL) return false; 145 | 146 | // Acquire the semaphore mutex. 147 | DWORD Result = ::WaitForSingleObject(MxWinRSemMutex, INFINITE); 148 | if (Result != WAIT_OBJECT_0) return false; 149 | 150 | // Release the semaphore. 151 | LONG Val; 152 | if (!::ReleaseSemaphore(MxWinRSemaphore, 1, &Val)) 153 | { 154 | ::ReleaseSemaphore(MxWinRSemMutex, 1, NULL); 155 | 156 | return false; 157 | } 158 | 159 | // Update the event state. 160 | if (Val == LONG_MAX - 1) 161 | { 162 | if (!::SetEvent(MxWinRWaitEvent)) 163 | { 164 | ::ReleaseSemaphore(MxWinRSemMutex, 1, NULL); 165 | 166 | return false; 167 | } 168 | } 169 | 170 | // Release the semaphore mutex. 171 | ::ReleaseSemaphore(MxWinRSemMutex, 1, NULL); 172 | 173 | return true; 174 | } 175 | 176 | bool ReadWriteLock::WriteUnlock() 177 | { 178 | if (MxWinWWaitMutex == NULL) return false; 179 | 180 | // Release the write lock. 181 | ::ReleaseSemaphore(MxWinWWaitMutex, 1, NULL); 182 | 183 | return true; 184 | } 185 | #else 186 | // POSIX pthreads. 187 | ReadWriteLock::ReadWriteLock() : MxNamed(false), MxMem(NULL), MxRCount(NULL) 188 | { 189 | } 190 | 191 | ReadWriteLock::~ReadWriteLock() 192 | { 193 | if (MxMem != NULL) 194 | { 195 | if (MxNamed) Util::UnmapUnixNamedMem(MxMem, Util::GetUnixSemaphoreSize() + Util::AlignUnixSize(sizeof(std::uint32_t)) + Util::GetUnixEventSize() + Util::GetUnixSemaphoreSize()); 196 | else 197 | { 198 | Util::FreeUnixSemaphore(MxPthreadRCountMutex); 199 | Util::FreeUnixEvent(MxPthreadRWaitEvent); 200 | Util::FreeUnixSemaphore(MxPthreadWWaitMutex); 201 | 202 | delete[] MxMem; 203 | } 204 | } 205 | } 206 | 207 | bool ReadWriteLock::Create(const char *Name) 208 | { 209 | if (MxMem != NULL) 210 | { 211 | if (MxNamed) Util::UnmapUnixNamedMem(MxMem, Util::GetUnixSemaphoreSize() + Util::AlignUnixSize(sizeof(std::uint32_t)) + Util::GetUnixEventSize() + Util::GetUnixSemaphoreSize()); 212 | else 213 | { 214 | Util::FreeUnixSemaphore(MxPthreadRCountMutex); 215 | Util::FreeUnixEvent(MxPthreadRWaitEvent); 216 | Util::FreeUnixSemaphore(MxPthreadWWaitMutex); 217 | 218 | delete[] MxMem; 219 | } 220 | } 221 | 222 | MxMem = NULL; 223 | MxRCount = NULL; 224 | 225 | size_t Pos, TempSize = Util::GetUnixSemaphoreSize() + Util::AlignUnixSize(sizeof(std::uint32_t)) + Util::GetUnixEventSize() + Util::GetUnixSemaphoreSize(); 226 | MxNamed = (Name != NULL); 227 | int Result = Util::InitUnixNamedMem(MxMem, Pos, "/Sync_ReadWrite", Name, TempSize); 228 | 229 | if (Result < 0) return false; 230 | 231 | // Load the pointers. 232 | char *MemPtr = MxMem + Pos; 233 | Util::GetUnixSemaphore(MxPthreadRCountMutex, MemPtr); 234 | MemPtr += Util::GetUnixSemaphoreSize(); 235 | 236 | MxRCount = reinterpret_cast(MemPtr); 237 | MemPtr += Util::AlignUnixSize(sizeof(std::uint32_t)); 238 | 239 | Util::GetUnixEvent(MxPthreadRWaitEvent, MemPtr); 240 | MemPtr += Util::GetUnixEventSize(); 241 | 242 | Util::GetUnixSemaphore(MxPthreadWWaitMutex, MemPtr); 243 | 244 | // Handle the first time this reader/writer lock has been opened. 245 | if (Result == 0) 246 | { 247 | Util::InitUnixSemaphore(MxPthreadRCountMutex, MxNamed, 1, 1); 248 | MxRCount[0] = 0; 249 | Util::InitUnixEvent(MxPthreadRWaitEvent, MxNamed, true, true); 250 | Util::InitUnixSemaphore(MxPthreadWWaitMutex, MxNamed, 1, 1); 251 | 252 | if (MxNamed) Util::UnixNamedMemReady(MxMem); 253 | } 254 | 255 | return true; 256 | } 257 | 258 | bool ReadWriteLock::ReadLock(std::uint32_t Wait) 259 | { 260 | std::uint64_t StartTime, CurrTime; 261 | 262 | if (MxMem == NULL) return false; 263 | 264 | // Get current time in milliseconds. 265 | StartTime = (Wait == INFINITE ? 0 : Util::GetUnixMicrosecondTime() / 1000000); 266 | 267 | // Acquire the write lock mutex. Guarantees that readers can't starve the writer. 268 | if (!Util::WaitForUnixSemaphore(MxPthreadWWaitMutex, Wait)) return false; 269 | 270 | // Acquire the counter mutex. 271 | CurrTime = (Wait == INFINITE ? 0 : Util::GetUnixMicrosecondTime() / 1000000); 272 | if (Wait < CurrTime - StartTime || !Util::WaitForUnixSemaphore(MxPthreadRCountMutex, Wait - (CurrTime - StartTime))) 273 | { 274 | Util::ReleaseUnixSemaphore(MxPthreadWWaitMutex, NULL); 275 | 276 | return false; 277 | } 278 | 279 | // Update the event state. 280 | if (!Util::ResetUnixEvent(MxPthreadRWaitEvent)) 281 | { 282 | Util::ReleaseUnixSemaphore(MxPthreadRCountMutex, NULL); 283 | Util::ReleaseUnixSemaphore(MxPthreadWWaitMutex, NULL); 284 | 285 | return false; 286 | } 287 | 288 | // Increment the number of readers. 289 | MxRCount[0]++; 290 | 291 | // Release the mutexes. 292 | Util::ReleaseUnixSemaphore(MxPthreadRCountMutex, NULL); 293 | Util::ReleaseUnixSemaphore(MxPthreadWWaitMutex, NULL); 294 | 295 | return true; 296 | } 297 | 298 | bool ReadWriteLock::WriteLock(std::uint32_t Wait) 299 | { 300 | std::uint64_t StartTime, CurrTime; 301 | 302 | if (MxMem == NULL) return false; 303 | 304 | // Get current time in milliseconds. 305 | StartTime = (Wait == INFINITE ? 0 : Util::GetUnixMicrosecondTime() / 1000000); 306 | 307 | // Acquire the write lock mutex. 308 | if (!Util::WaitForUnixSemaphore(MxPthreadWWaitMutex, Wait)) return false; 309 | 310 | // Wait for readers to reach zero. 311 | CurrTime = (Wait == INFINITE ? 0 : Util::GetUnixMicrosecondTime() / 1000000); 312 | if (Wait < CurrTime - StartTime || !Util::WaitForUnixEvent(MxPthreadRWaitEvent, Wait - (CurrTime - StartTime))) 313 | { 314 | Util::ReleaseUnixSemaphore(MxPthreadWWaitMutex, NULL); 315 | 316 | return false; 317 | } 318 | 319 | return true; 320 | } 321 | 322 | bool ReadWriteLock::ReadUnlock() 323 | { 324 | if (MxMem == NULL) return false; 325 | 326 | // Acquire the counter mutex. 327 | if (!Util::WaitForUnixSemaphore(MxPthreadRCountMutex, INFINITE)) return false; 328 | 329 | // Decrease the number of readers. 330 | if (MxRCount[0]) MxRCount[0]--; 331 | else 332 | { 333 | Util::ReleaseUnixSemaphore(MxPthreadRCountMutex, NULL); 334 | 335 | return false; 336 | } 337 | 338 | // Update the event state. 339 | if (!MxRCount[0] && !Util::FireUnixEvent(MxPthreadRWaitEvent)) 340 | { 341 | Util::ReleaseUnixSemaphore(MxPthreadRCountMutex, NULL); 342 | 343 | return false; 344 | } 345 | 346 | // Release the counter mutex. 347 | Util::ReleaseUnixSemaphore(MxPthreadRCountMutex, NULL); 348 | 349 | return true; 350 | } 351 | 352 | bool ReadWriteLock::WriteUnlock() 353 | { 354 | if (MxMem == NULL) return false; 355 | 356 | // Release the write lock. 357 | Util::ReleaseUnixSemaphore(MxPthreadWWaitMutex, NULL); 358 | 359 | return true; 360 | } 361 | #endif 362 | 363 | 364 | ReadWriteLock::AutoReadUnlock::AutoReadUnlock(ReadWriteLock *LockPtr) 365 | { 366 | MxLock = LockPtr; 367 | } 368 | 369 | ReadWriteLock::AutoReadUnlock::~AutoReadUnlock() 370 | { 371 | MxLock->ReadUnlock(); 372 | } 373 | 374 | 375 | ReadWriteLock::AutoWriteUnlock::AutoWriteUnlock(ReadWriteLock *LockPtr) 376 | { 377 | MxLock = LockPtr; 378 | } 379 | 380 | ReadWriteLock::AutoWriteUnlock::~AutoWriteUnlock() 381 | { 382 | MxLock->WriteUnlock(); 383 | } 384 | } 385 | } 386 | -------------------------------------------------------------------------------- /sync/sync_readwritelock.h: -------------------------------------------------------------------------------- 1 | // Cross-platform, optionally named (cross-process), reader/writer lock. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_SYNC_READWRITELOCK 5 | #define CUBICLESOFT_SYNC_READWRITELOCK 6 | 7 | #include "sync_util.h" 8 | 9 | namespace CubicleSoft 10 | { 11 | namespace Sync 12 | { 13 | class ReadWriteLock 14 | { 15 | public: 16 | ReadWriteLock(); 17 | ~ReadWriteLock(); 18 | 19 | bool Create(const char *Name = NULL); 20 | 21 | // Wait is in milliseconds. Granularity is in 5ms-15ms intervals on some platforms. 22 | bool ReadLock(std::uint32_t Wait = INFINITE); 23 | 24 | // Wait is in milliseconds. Granularity is in 5ms-15ms intervals on some platforms. 25 | bool WriteLock(std::uint32_t Wait = INFINITE); 26 | 27 | bool ReadUnlock(); 28 | bool WriteUnlock(); 29 | 30 | // Automatically unlock a read lock when leaving the current scope. 31 | class AutoReadUnlock 32 | { 33 | public: 34 | AutoReadUnlock(ReadWriteLock *LockPtr); 35 | ~AutoReadUnlock(); 36 | 37 | private: 38 | // Deny copy constructor and assignment operator. Use a (smart) pointer instead. 39 | AutoReadUnlock(const AutoReadUnlock &); 40 | AutoReadUnlock &operator=(const AutoReadUnlock &); 41 | 42 | ReadWriteLock *MxLock; 43 | }; 44 | 45 | // Automatically unlock a write lock when leaving the current scope. 46 | class AutoWriteUnlock 47 | { 48 | public: 49 | AutoWriteUnlock(ReadWriteLock *LockPtr); 50 | ~AutoWriteUnlock(); 51 | 52 | private: 53 | // Deny copy constructor and assignment operator. Use a (smart) pointer instead. 54 | AutoWriteUnlock(const AutoWriteUnlock &); 55 | AutoWriteUnlock &operator=(const AutoWriteUnlock &); 56 | 57 | ReadWriteLock *MxLock; 58 | }; 59 | 60 | private: 61 | // Deny copy constructor and assignment operator. Use a (smart) pointer instead. 62 | ReadWriteLock(const ReadWriteLock &); 63 | ReadWriteLock &operator=(const ReadWriteLock &); 64 | 65 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 66 | HANDLE MxWinRSemMutex, MxWinRSemaphore, MxWinRWaitEvent, MxWinWWaitMutex; 67 | #else 68 | bool MxNamed; 69 | char *MxMem; 70 | Util::UnixSemaphoreWrapper MxPthreadRCountMutex; 71 | volatile std::uint32_t *MxRCount; 72 | Util::UnixEventWrapper MxPthreadRWaitEvent; 73 | Util::UnixSemaphoreWrapper MxPthreadWWaitMutex; 74 | #endif 75 | }; 76 | } 77 | } 78 | 79 | #endif 80 | -------------------------------------------------------------------------------- /sync/sync_semaphore.cpp: -------------------------------------------------------------------------------- 1 | // Cross-platform, optionally named (cross-process), semaphore. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #include "sync_semaphore.h" 5 | 6 | namespace CubicleSoft 7 | { 8 | namespace Sync 9 | { 10 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 11 | // Windows. 12 | Semaphore::Semaphore() : MxWinSemaphore(NULL) 13 | { 14 | } 15 | 16 | Semaphore::~Semaphore() 17 | { 18 | if (MxWinSemaphore != NULL) ::CloseHandle(MxWinSemaphore); 19 | } 20 | 21 | bool Semaphore::Create(const char *Name, int InitialVal) 22 | { 23 | if (MxWinSemaphore != NULL) ::CloseHandle(MxWinSemaphore); 24 | 25 | MxWinSemaphore = NULL; 26 | 27 | SECURITY_ATTRIBUTES SecAttr; 28 | 29 | SecAttr.nLength = sizeof(SecAttr); 30 | SecAttr.lpSecurityDescriptor = NULL; 31 | SecAttr.bInheritHandle = TRUE; 32 | 33 | MxWinSemaphore = ::CreateSemaphoreA(&SecAttr, (LONG)InitialVal, (LONG)InitialVal, Name); 34 | 35 | if (MxWinSemaphore == NULL) return false; 36 | 37 | return true; 38 | } 39 | 40 | bool Semaphore::Lock(std::uint32_t Wait) 41 | { 42 | if (MxWinSemaphore == NULL) return false; 43 | 44 | DWORD Result = ::WaitForSingleObject(MxWinSemaphore, Wait); 45 | if (Result != WAIT_OBJECT_0) return false; 46 | 47 | return true; 48 | } 49 | 50 | bool Semaphore::Unlock(int *PrevCount) 51 | { 52 | if (MxWinSemaphore == NULL) return false; 53 | 54 | LONG Count; 55 | if (!::ReleaseSemaphore(MxWinSemaphore, 1, &Count)) return false; 56 | if (PrevCount != NULL) *PrevCount = (int)Count; 57 | 58 | return true; 59 | } 60 | #else 61 | // POSIX pthreads. 62 | Semaphore::Semaphore() : MxNamed(false), MxMem(NULL) 63 | { 64 | } 65 | 66 | Semaphore::~Semaphore() 67 | { 68 | if (MxMem != NULL) 69 | { 70 | if (MxNamed) Util::UnmapUnixNamedMem(MxMem, Util::GetUnixSemaphoreSize()); 71 | else 72 | { 73 | Util::FreeUnixSemaphore(MxPthreadSemaphore); 74 | 75 | delete[] MxMem; 76 | } 77 | } 78 | } 79 | 80 | bool Semaphore::Create(const char *Name, int InitialVal) 81 | { 82 | if (MxMem != NULL) 83 | { 84 | if (MxNamed) Util::UnmapUnixNamedMem(MxMem, Util::GetUnixSemaphoreSize()); 85 | else 86 | { 87 | Util::FreeUnixSemaphore(MxPthreadSemaphore); 88 | 89 | delete[] MxMem; 90 | } 91 | } 92 | 93 | MxMem = NULL; 94 | 95 | size_t Pos, TempSize = Util::GetUnixSemaphoreSize(); 96 | MxNamed = (Name != NULL); 97 | int Result = Util::InitUnixNamedMem(MxMem, Pos, "/Sync_Semaphore", Name, TempSize); 98 | 99 | if (Result < 0) return false; 100 | 101 | Util::GetUnixSemaphore(MxPthreadSemaphore, MxMem + Pos); 102 | 103 | // Handle the first time this semaphore has been opened. 104 | if (Result == 0) 105 | { 106 | Util::InitUnixSemaphore(MxPthreadSemaphore, MxNamed, InitialVal, InitialVal); 107 | 108 | if (MxNamed) Util::UnixNamedMemReady(MxMem); 109 | } 110 | 111 | return true; 112 | } 113 | 114 | bool Semaphore::Lock(std::uint32_t Wait) 115 | { 116 | if (MxMem == NULL) return false; 117 | 118 | // Wait for the semaphore. 119 | return Util::WaitForUnixSemaphore(MxPthreadSemaphore, Wait); 120 | } 121 | 122 | bool Semaphore::Unlock(int *PrevCount) 123 | { 124 | if (MxMem == NULL) return false; 125 | 126 | std::uint32_t Count; 127 | Util::ReleaseUnixSemaphore(MxPthreadSemaphore, &Count); 128 | if (PrevCount != NULL) *PrevCount = (int)Count; 129 | 130 | return true; 131 | } 132 | #endif 133 | 134 | 135 | Semaphore::AutoUnlock::AutoUnlock(Semaphore *LockPtr) 136 | { 137 | MxLock = LockPtr; 138 | } 139 | 140 | Semaphore::AutoUnlock::~AutoUnlock() 141 | { 142 | MxLock->Unlock(); 143 | } 144 | } 145 | } 146 | -------------------------------------------------------------------------------- /sync/sync_semaphore.h: -------------------------------------------------------------------------------- 1 | // Cross-platform, optionally named (cross-process), semaphore. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_SYNC_SEMAPHORE 5 | #define CUBICLESOFT_SYNC_SEMAPHORE 6 | 7 | #include "sync_util.h" 8 | 9 | namespace CubicleSoft 10 | { 11 | namespace Sync 12 | { 13 | class Semaphore 14 | { 15 | public: 16 | Semaphore(); 17 | ~Semaphore(); 18 | 19 | bool Create(const char *Name = NULL, int InitialVal = 1); 20 | 21 | // Wait is in milliseconds. Granularity is in 5ms intervals on some platforms. 22 | bool Lock(std::uint32_t Wait = INFINITE); 23 | 24 | // Should only be called after a successful Lock(). Undefined behavior otherwise. 25 | bool Unlock(int *PrevCount = NULL); 26 | 27 | // Automatically unlock a semaphore when leaving the current scope. 28 | class AutoUnlock 29 | { 30 | public: 31 | AutoUnlock(Semaphore *LockPtr); 32 | ~AutoUnlock(); 33 | 34 | private: 35 | // Deny copy constructor and assignment operator. Use a (smart) pointer instead. 36 | AutoUnlock(const AutoUnlock &); 37 | AutoUnlock &operator=(const AutoUnlock &); 38 | 39 | Semaphore *MxLock; 40 | }; 41 | 42 | private: 43 | // Deny copy constructor and assignment operator. Use a (smart) pointer instead. 44 | Semaphore(const Semaphore &); 45 | Semaphore &operator=(const Semaphore &); 46 | 47 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 48 | HANDLE MxWinSemaphore; 49 | #else 50 | bool MxNamed; 51 | char *MxMem; 52 | Util::UnixSemaphoreWrapper MxPthreadSemaphore; 53 | #endif 54 | }; 55 | } 56 | } 57 | 58 | #endif 59 | -------------------------------------------------------------------------------- /sync/sync_sharedmem.cpp: -------------------------------------------------------------------------------- 1 | // Cross-platform, named (cross-process), shared memory with two auto events. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #define _CRT_SECURE_NO_WARNINGS 5 | 6 | #include "sync_sharedmem.h" 7 | 8 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 9 | #include 10 | #else 11 | #endif 12 | 13 | namespace CubicleSoft 14 | { 15 | namespace Sync 16 | { 17 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 18 | // Windows. 19 | SharedMem::SharedMem() : MxFirst(false), MxSize(0), MxMem(NULL), MxFile(NULL) 20 | { 21 | } 22 | 23 | SharedMem::~SharedMem() 24 | { 25 | if (MxMem != NULL) ::UnmapViewOfFile(MxMem); 26 | if (MxFile != NULL) ::CloseHandle(MxFile); 27 | } 28 | 29 | bool SharedMem::Create(const char *Name, size_t Size) 30 | { 31 | if (Name == NULL) return false; 32 | 33 | if (MxMem != NULL) ::UnmapViewOfFile(MxMem); 34 | if (MxFile != NULL) ::CloseHandle(MxFile); 35 | 36 | MxMem = NULL; 37 | MxFile = NULL; 38 | MxFirst = false; 39 | 40 | SECURITY_ATTRIBUTES SecAttr; 41 | 42 | SecAttr.nLength = sizeof(SecAttr); 43 | SecAttr.lpSecurityDescriptor = NULL; 44 | SecAttr.bInheritHandle = TRUE; 45 | 46 | char *Name2 = new char[strlen(Name) + 30]; 47 | 48 | // Create the file mapping object backed by the system page file. 49 | sprintf(Name2, "%s-%u-Sync_SharedMem", Name, (unsigned int)Size); 50 | MxFile = ::CreateFileMappingA(INVALID_HANDLE_VALUE, &SecAttr, PAGE_READWRITE, 0, (DWORD)Size, Name2); 51 | if (MxFile == NULL) 52 | { 53 | MxFile = ::OpenFileMappingA(FILE_MAP_ALL_ACCESS, TRUE, Name2); 54 | if (MxFile == NULL) return false; 55 | } 56 | else if (::GetLastError() != ERROR_ALREADY_EXISTS) 57 | { 58 | MxFirst = true; 59 | } 60 | 61 | delete[] Name2; 62 | 63 | MxMem = (char *)::MapViewOfFile(MxFile, FILE_MAP_ALL_ACCESS, 0, 0, (DWORD)Size); 64 | if (MxMem == NULL) return false; 65 | 66 | MxSize = Size; 67 | 68 | return true; 69 | } 70 | 71 | #else 72 | // POSIX pthreads. 73 | SharedMem::SharedMem() : MxFirst(false), MxSize(0), MxMem(NULL), MxMemInternal(NULL) 74 | { 75 | } 76 | 77 | SharedMem::~SharedMem() 78 | { 79 | if (MxMemInternal != NULL) Util::UnmapUnixNamedMem(MxMemInternal, MxSize); 80 | } 81 | 82 | bool SharedMem::Create(const char *Name, size_t Size) 83 | { 84 | if (Name == NULL) return false; 85 | 86 | if (MxMemInternal != NULL) Util::UnmapUnixNamedMem(MxMemInternal, MxSize); 87 | 88 | MxMemInternal = NULL; 89 | MxMem = NULL; 90 | 91 | size_t Pos, TempSize = Size; 92 | int Result = Util::InitUnixNamedMem(MxMemInternal, Pos, "/Sync_SharedMem", Name, TempSize); 93 | 94 | if (Result < 0) return false; 95 | 96 | // Load the pointers. 97 | MxMem = MxMemInternal + Pos; 98 | MxSize = Size; 99 | 100 | // Handle the first time this named memory has been opened. 101 | if (Result == 0) 102 | { 103 | Util::UnixNamedMemReady(MxMemInternal); 104 | 105 | MxFirst = true; 106 | } 107 | 108 | return true; 109 | } 110 | #endif 111 | } 112 | } 113 | -------------------------------------------------------------------------------- /sync/sync_sharedmem.h: -------------------------------------------------------------------------------- 1 | // Cross-platform, named (cross-process), shared memory with two auto events. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_SYNC_SHAREDMEM 5 | #define CUBICLESOFT_SYNC_SHAREDMEM 6 | 7 | #include "sync_util.h" 8 | 9 | namespace CubicleSoft 10 | { 11 | namespace Sync 12 | { 13 | class SharedMem 14 | { 15 | public: 16 | SharedMem(); 17 | ~SharedMem(); 18 | 19 | // For platform consistency, Name + Size is a unique key. size_t is, of course, limited to 4GB RAM on most platforms. 20 | // On some platforms (e.g. Windows), the objects may vanish if all handles are freed. 21 | bool Create(const char *Name, size_t Size); 22 | 23 | // Returns true if Create() created the shared memory object, false otherwise (i.e. opened an existing object). 24 | inline bool First() { return MxFirst; } 25 | 26 | inline size_t GetSize() { return MxSize; } 27 | inline char *RawData() { return MxMem; } 28 | 29 | private: 30 | // Deny copy constructor and assignment operator. Use a (smart) pointer instead. 31 | SharedMem(const SharedMem &); 32 | SharedMem &operator=(const SharedMem &); 33 | 34 | bool MxFirst; 35 | size_t MxSize; 36 | char *MxMem; 37 | 38 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 39 | HANDLE MxFile; 40 | #else 41 | char *MxMemInternal; 42 | #endif 43 | }; 44 | } 45 | } 46 | 47 | #endif 48 | -------------------------------------------------------------------------------- /sync/sync_tls.cpp: -------------------------------------------------------------------------------- 1 | // Cross-platform thread local storage memory allocation class. Built for short-lived, in-thread memory allocations. 2 | // (C) 2013 CubicleSoft. All Rights Reserved. 3 | 4 | #include "sync_tls.h" 5 | #include "../templates/fast_find_replace.h" 6 | #include 7 | #include 8 | #include 9 | #include 10 | 11 | namespace CubicleSoft 12 | { 13 | namespace Sync 14 | { 15 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 16 | // Windows. 17 | TLS::TLS() 18 | { 19 | MxTlsIndex = ::TlsAlloc(); 20 | } 21 | 22 | TLS::~TLS() 23 | { 24 | ::TlsFree(MxTlsIndex); 25 | } 26 | 27 | bool TLS::SetMainPtr(StaticVector> *MainPtr) 28 | { 29 | return (::TlsSetValue(MxTlsIndex, MainPtr) != 0); 30 | } 31 | 32 | StaticVector> *TLS::GetMainPtr() 33 | { 34 | return (StaticVector> *)::TlsGetValue(MxTlsIndex); 35 | } 36 | #else 37 | // POSIX pthreads. 38 | TLS::TLS() 39 | { 40 | pthread_key_create(&MxKey, NULL); 41 | } 42 | 43 | TLS::~TLS() 44 | { 45 | pthread_key_delete(MxKey); 46 | } 47 | 48 | bool TLS::SetMainPtr(StaticVector> *MainPtr) 49 | { 50 | return (pthread_setspecific(MxKey, MainPtr) == 0); 51 | } 52 | 53 | StaticVector> *TLS::GetMainPtr() 54 | { 55 | return (StaticVector> *)pthread_getspecific(MxKey); 56 | } 57 | #endif 58 | 59 | // All platforms. 60 | bool TLS::ThreadInit(size_t MaxCacheBits) 61 | { 62 | StaticVector> *MainPtr = GetMainPtr(); 63 | if (MainPtr != NULL) return true; 64 | 65 | return SetMainPtr(new StaticVector>(MaxCacheBits)); 66 | } 67 | 68 | void *TLS::malloc(size_t Size) 69 | { 70 | if (Size == 0) return NULL; 71 | 72 | StaticVector> *MainPtr = GetMainPtr(); 73 | if (MainPtr == NULL) return NULL; 74 | 75 | void *Data; 76 | size_t Pos = NormalizeBitPosition(Size); 77 | Size++; 78 | if (MainPtr->GetSize() <= Pos) 79 | { 80 | Data = ::malloc(Size); 81 | Data = (std::uint8_t *)Data + 1; 82 | } 83 | else 84 | { 85 | QueueNode *Node = (*MainPtr)[Pos].Shift(); 86 | if (Node == NULL) 87 | { 88 | if (Size < sizeof(QueueNode)) Size = sizeof(QueueNode); 89 | Data = ::malloc(Size); 90 | if (Data == NULL) return NULL; 91 | 92 | Node = (QueueNode *)Data; 93 | } 94 | 95 | Data = ((std::uint8_t *)Node) + 1; 96 | } 97 | 98 | // Store the position. Safe for QueueNode since data storage is after the NextNode pointer. 99 | ((std::uint8_t *)Data)[-1] = (std::uint8_t)Pos; 100 | 101 | return Data; 102 | } 103 | 104 | void *TLS::realloc(void *Data, size_t NewSize, bool Cache) 105 | { 106 | if (NewSize == 0) 107 | { 108 | if (Data != NULL) free(Data, Cache); 109 | 110 | return NULL; 111 | } 112 | 113 | if (Data == NULL) return malloc(NewSize); 114 | 115 | // See if it fits in the current size. 116 | size_t Pos = NormalizeBitPosition(NewSize); 117 | size_t Pos2 = (size_t)((std::uint8_t *)Data)[-1]; 118 | if (Pos <= Pos2) return Data; 119 | 120 | StaticVector> *MainPtr = GetMainPtr(); 121 | if (MainPtr == NULL) return NULL; 122 | 123 | void *Data2; 124 | if (MainPtr->GetSize() <= Pos && MainPtr->GetSize() <= Pos2) 125 | { 126 | Data2 = ::realloc(((std::uint8_t *)Data) - 1, 1 + NewSize); 127 | Data2 = (std::uint8_t *)Data2 + 1; 128 | } 129 | else 130 | { 131 | // Allocate data. 132 | Data2 = malloc(NewSize); 133 | 134 | // Copy the data. 135 | memcpy(Data2, Data, (1 << Pos2)); 136 | 137 | // Free the previous object. 138 | free(Data, Cache); 139 | } 140 | 141 | return Data2; 142 | } 143 | 144 | void *TLS::dup_malloc(void *Data, bool Cache) 145 | { 146 | if (Data == NULL) return NULL; 147 | 148 | // Allocate the appropriate size buffer. 149 | size_t Size = (size_t)(1 << ((size_t)((std::uint8_t *)Data)[-1])); 150 | void *Data2 = ::malloc(Size); 151 | if (Data2 == NULL) return NULL; 152 | 153 | // Copy the data. 154 | memcpy(Data2, Data, Size); 155 | 156 | // Free the previous object. 157 | free(Data, Cache); 158 | 159 | return Data2; 160 | } 161 | 162 | void TLS::free(void *Data, bool Cache) 163 | { 164 | if (Data == NULL) return; 165 | 166 | StaticVector> *MainPtr = GetMainPtr(); 167 | if (MainPtr == NULL) return; 168 | 169 | size_t Pos = (size_t)((std::uint8_t *)Data)[-1]; 170 | if (MainPtr->GetSize() <= Pos) ::free(((std::uint8_t *)Data) - 1); 171 | else 172 | { 173 | QueueNode *Node = (QueueNode *)(((std::uint8_t *)Data) - 1); 174 | 175 | if (!Cache) ::free(Node); 176 | else 177 | { 178 | // Placement new. Instantiates QueueNode. 179 | Node = new(Node) QueueNode; 180 | 181 | (*MainPtr)[Pos].Push(Node); 182 | } 183 | } 184 | } 185 | 186 | bool TLS::GetBucketInfo(size_t Num, size_t &Nodes, size_t &Size) 187 | { 188 | StaticVector> *MainPtr = GetMainPtr(); 189 | if (MainPtr == NULL) return false; 190 | 191 | if (Num >= MainPtr->GetSize()) return false; 192 | else 193 | { 194 | Nodes = (*MainPtr)[Num].GetSize(); 195 | Size = (size_t)(1 << Num); 196 | if (Size < sizeof(QueueNode)) Size = sizeof(QueueNode); 197 | Size *= Nodes; 198 | } 199 | 200 | return true; 201 | } 202 | 203 | bool TLS::ThreadEnd() 204 | { 205 | StaticVector> *MainPtr = GetMainPtr(); 206 | if (MainPtr == NULL) return false; 207 | 208 | SetMainPtr(NULL); 209 | 210 | // Free all cached data. 211 | size_t y = MainPtr->GetSize(); 212 | Queue *RawData = MainPtr->RawData(); 213 | QueueNode *Node; 214 | for (size_t x = 0; x < y; x++) 215 | { 216 | while ((Node = RawData[x].Shift()) != NULL) 217 | { 218 | ::free(Node); 219 | } 220 | } 221 | 222 | delete MainPtr; 223 | 224 | return true; 225 | } 226 | 227 | size_t TLS::NormalizeBitPosition(size_t &Size) 228 | { 229 | size_t Pos = 3; 230 | 231 | while (((size_t)1 << Pos) < Size) Pos++; 232 | Size = ((size_t)1 << Pos); 233 | 234 | return Pos; 235 | } 236 | 237 | 238 | TLS::AutoFree::AutoFree(TLS *TLSPtr, void *Data) 239 | { 240 | MxTLS = TLSPtr; 241 | MxData = Data; 242 | } 243 | 244 | TLS::AutoFree::~AutoFree() 245 | { 246 | MxTLS->free(MxData); 247 | } 248 | 249 | 250 | TLS::MixedVar::MixedVar(TLS *TLSPtr) : MxMode(TMV_None), MxInt(0), MxDouble(0.0), MxStr(NULL), MxStrPos(0), MxTLS(TLSPtr) 251 | { 252 | } 253 | 254 | TLS::MixedVar::~MixedVar() 255 | { 256 | if (MxStr != NULL) MxTLS->free(MxStr); 257 | } 258 | 259 | // Copy constructor. 260 | TLS::MixedVar::MixedVar(const TLS::MixedVar &TempVar) 261 | { 262 | MxTLS = TempVar.MxTLS; 263 | 264 | if (TempVar.MxStr != NULL) SetData(TempVar.MxStr, TempVar.MxStrPos); 265 | else 266 | { 267 | MxStr = NULL; 268 | MxStrPos = 0; 269 | } 270 | 271 | MxMode = TempVar.MxMode; 272 | MxInt = TempVar.MxInt; 273 | MxDouble = TempVar.MxDouble; 274 | } 275 | 276 | // Assignment operator. 277 | TLS::MixedVar &TLS::MixedVar::operator=(const TLS::MixedVar &TempVar) 278 | { 279 | if (this != &TempVar) 280 | { 281 | MxTLS = TempVar.MxTLS; 282 | 283 | if (TempVar.MxStr != NULL) SetData(TempVar.MxStr, TempVar.MxStrPos); 284 | else 285 | { 286 | MxTLS->free(MxStr); 287 | 288 | MxStr = NULL; 289 | MxStrPos = 0; 290 | } 291 | 292 | MxMode = TempVar.MxMode; 293 | MxInt = TempVar.MxInt; 294 | MxDouble = TempVar.MxDouble; 295 | } 296 | 297 | return *this; 298 | } 299 | 300 | void TLS::MixedVar::SetData(const char *str, size_t size) 301 | { 302 | MxMode = TMV_Str; 303 | if (MxStr != NULL) MxTLS->free(MxStr); 304 | MxStr = (char *)MxTLS->malloc(size + 1); 305 | memcpy(MxStr, str, size); 306 | MxStrPos = size; 307 | MxStr[MxStrPos] = '\0'; 308 | } 309 | 310 | void TLS::MixedVar::SetStr(const char *str) 311 | { 312 | SetData(str, strlen(str)); 313 | } 314 | 315 | void TLS::MixedVar::PrependData(const char *str, size_t size) 316 | { 317 | char *str2 = (char *)MxTLS->malloc(size + MxStrPos + 1); 318 | memcpy(str2, str, size); 319 | memcpy(str2 + size, MxStr, MxStrPos); 320 | MxTLS->free(MxStr); 321 | MxStr = str2; 322 | MxStrPos += size; 323 | MxStr[MxStrPos] = '\0'; 324 | } 325 | 326 | void TLS::MixedVar::PrependStr(const char *str) 327 | { 328 | PrependData(str, strlen(str)); 329 | } 330 | 331 | void TLS::MixedVar::PrependInt(const std::int64_t val, size_t radix) 332 | { 333 | char tempbuffer[44]; 334 | if (IntToString(tempbuffer, sizeof(tempbuffer), val, radix)) PrependStr(tempbuffer); 335 | } 336 | 337 | void TLS::MixedVar::PrependUInt(const std::uint64_t val, size_t radix) 338 | { 339 | char tempbuffer[44]; 340 | if (IntToString(tempbuffer, sizeof(tempbuffer), val, radix)) PrependStr(tempbuffer); 341 | } 342 | 343 | void TLS::MixedVar::PrependDouble(const double val, const size_t precision) 344 | { 345 | char tempbuffer[100]; 346 | #if (defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64)) && defined(_MSC_VER) && _MSC_VER < 1900 347 | _snprintf_s(tempbuffer, sizeof(tempbuffer), _TRUNCATE, "%1.*g", precision, val); 348 | tempbuffer[sizeof(tempbuffer) - 1] = '\0'; 349 | #else 350 | snprintf(tempbuffer, sizeof(tempbuffer), "%1.*g", (int)precision, val); 351 | #endif 352 | 353 | PrependStr(tempbuffer); 354 | } 355 | 356 | void TLS::MixedVar::AppendData(const char *str, size_t size) 357 | { 358 | MxStr = (char *)MxTLS->realloc(MxStr, MxStrPos + size + 1); 359 | memcpy(MxStr + MxStrPos, str, size); 360 | MxStrPos += size; 361 | MxStr[MxStrPos] = '\0'; 362 | } 363 | 364 | void TLS::MixedVar::AppendStr(const char *str) 365 | { 366 | AppendData(str, strlen(str)); 367 | } 368 | 369 | void TLS::MixedVar::AppendInt(const std::int64_t val, size_t radix) 370 | { 371 | char tempbuffer[44]; 372 | if (IntToString(tempbuffer, sizeof(tempbuffer), val, radix)) AppendStr(tempbuffer); 373 | } 374 | 375 | void TLS::MixedVar::AppendUInt(const std::uint64_t val, size_t radix) 376 | { 377 | char tempbuffer[44]; 378 | if (IntToString(tempbuffer, sizeof(tempbuffer), val, radix)) AppendStr(tempbuffer); 379 | } 380 | 381 | void TLS::MixedVar::AppendDouble(const double val, const size_t precision) 382 | { 383 | char tempbuffer[100]; 384 | #if (defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64)) && defined(_MSC_VER) && _MSC_VER < 1900 385 | _snprintf_s(tempbuffer, sizeof(tempbuffer), _TRUNCATE, "%1.*g", precision, val); 386 | tempbuffer[sizeof(tempbuffer) - 1] = '\0'; 387 | #else 388 | snprintf(tempbuffer, sizeof(tempbuffer), "%1.*g", (int)precision, val); 389 | #endif 390 | 391 | AppendStr(tempbuffer); 392 | } 393 | 394 | void TLS::MixedVar::AppendChar(char chr) 395 | { 396 | MxStr = (char *)MxTLS->realloc(MxStr, MxStrPos + 2); 397 | MxStr[MxStrPos++] = chr; 398 | MxStr[MxStrPos] = '\0'; 399 | } 400 | 401 | void TLS::MixedVar::AppendMissingChar(char chr) 402 | { 403 | if (!MxStrPos || MxStr[MxStrPos - 1] != chr) AppendChar(chr); 404 | } 405 | 406 | void TLS::MixedVar::SetSize(size_t pos) 407 | { 408 | if (MxStrPos < pos) MxStr = (char *)MxTLS->realloc(MxStr, pos + 1); 409 | 410 | MxStrPos = pos; 411 | MxStr[MxStrPos] = '\0'; 412 | } 413 | 414 | size_t TLS::MixedVar::ReplaceData(const char *src, size_t srcsize, const char *dest, size_t destsize) 415 | { 416 | size_t Num; 417 | 418 | if (srcsize < destsize) 419 | { 420 | char *Result; 421 | size_t ResultSize; 422 | 423 | Num = FastReplaceAlloc::ReplaceAll(Result, ResultSize, MxStr, MxStrPos, src, srcsize, dest, destsize, MxTLS); 424 | 425 | MxTLS->free(MxStr); 426 | MxStr = Result; 427 | MxStrPos = ResultSize; 428 | } 429 | else 430 | { 431 | Num = FastReplaceAlloc::StaticReplaceAll(MxStr, MxStrPos, MxStr, MxStrPos, src, srcsize, dest, destsize, MxTLS); 432 | } 433 | 434 | MxStr[MxStrPos] = '\0'; 435 | 436 | return Num; 437 | } 438 | 439 | size_t TLS::MixedVar::ReplaceStr(const char *src, const char *dest) 440 | { 441 | return ReplaceData(src, strlen(src), dest, strlen(dest)); 442 | } 443 | 444 | // Swiped and slightly modified from Int::ToString(). 445 | bool TLS::MixedVar::IntToString(char *Result, size_t Size, std::uint64_t Num, size_t Radix) 446 | { 447 | if (Size < 2) return false; 448 | 449 | size_t x = Size, z; 450 | 451 | Result[--x] = '\0'; 452 | if (!Num) Result[--x] = '0'; 453 | else 454 | { 455 | while (Num && x) 456 | { 457 | z = Num % Radix; 458 | Result[--x] = (char)(z > 9 ? z - 10 + 'A' : z + '0'); 459 | Num /= Radix; 460 | } 461 | 462 | if (Num) return false; 463 | } 464 | 465 | memmove(Result, Result + x, Size - x); 466 | 467 | return true; 468 | } 469 | 470 | bool TLS::MixedVar::IntToString(char *Result, size_t Size, std::int64_t Num, size_t Radix) 471 | { 472 | if (Num >= 0) return IntToString(Result, Size, (std::uint64_t)Num, Radix); 473 | 474 | if (Size < 2) return false; 475 | Result[0] = '-'; 476 | 477 | return IntToString(Result + 1, Size - 1, (std::uint64_t)-Num, Radix); 478 | } 479 | } 480 | } 481 | -------------------------------------------------------------------------------- /sync/sync_tls.h: -------------------------------------------------------------------------------- 1 | // Cross-platform thread local storage memory allocation class. Built for short-lived, small, in-thread memory allocations. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_SYNC_TLS 5 | #define CUBICLESOFT_SYNC_TLS 6 | 7 | #include "sync_util.h" 8 | #include "../templates/detachable_queue.h" 9 | #include "../templates/static_vector.h" 10 | 11 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 12 | #include 13 | #else 14 | #include 15 | #endif 16 | 17 | namespace CubicleSoft 18 | { 19 | namespace Sync 20 | { 21 | // Only the main thread in the main executable should instantiate this object. 22 | // Pass to other threads and shared objects/DLLs or use globally via shared memory. 23 | class TLS 24 | { 25 | public: 26 | TLS(); 27 | ~TLS(); 28 | 29 | // Intended for use as a large, fixed-sized stack. 30 | class AutoFree 31 | { 32 | public: 33 | AutoFree(TLS *TLSPtr, void *Data); 34 | ~AutoFree(); 35 | 36 | private: 37 | // Deny copy constructor and assignment operator. Use a (smart) pointer instead. 38 | AutoFree(const AutoFree &); 39 | AutoFree &operator=(const AutoFree &); 40 | 41 | TLS *MxTLS; 42 | void *MxData; 43 | }; 44 | 45 | // Initializes the local thread cache to cache allocations (Default is 15, 2 ^ 15 = up to 32K allocations). 46 | // It is highly recommended to surround all code between ThreadInit() and ThreadEnd() with braces, especially when using AutoFree. 47 | // Only call this function once per thread per TLS instance. 48 | bool ThreadInit(size_t MaxCacheBits = 15); 49 | 50 | // Standard malloc()-like call. Do not mix with real malloc/realloc/free! Do not send to other threads. 51 | void *malloc(size_t Size); 52 | 53 | // Standard realloc()-like call. 54 | void *realloc(void *Data, size_t NewSize, bool Cache = true); 55 | 56 | // Duplicates memory using the real malloc() call, shallow copies Data, and free()'s Data for reuse. 57 | // For real new, just use a copy constructor and then free the memory. That way a deep copy can more naturally happen. 58 | void *dup_malloc(void *Data, bool Cache = true); 59 | 60 | // Standard free()-like call. 61 | void free(void *Data, bool Cache = true); 62 | 63 | // Some static versions of the above to be able to pass the class around to other snippet library functions. 64 | inline static void *malloc(void *TLSPtr, size_t Size) 65 | { 66 | return ((TLS *)TLSPtr)->malloc(Size); 67 | } 68 | 69 | inline static void *realloc(void *TLSPtr, void *Data, size_t NewSize) 70 | { 71 | return ((TLS *)TLSPtr)->realloc(Data, NewSize); 72 | } 73 | 74 | inline static void free(void *TLSPtr, void *Data) 75 | { 76 | ((TLS *)TLSPtr)->free(Data); 77 | } 78 | 79 | // Extract stats. 80 | bool GetBucketInfo(size_t Num, size_t &Nodes, size_t &Size); 81 | 82 | // Frees up all resources associated with the local thread cache. 83 | bool ThreadEnd(); 84 | 85 | 86 | // A MixedVar-style class with raw Sync::TLS support. 87 | // Little to no error checking for that awesome raw performance feeling. 88 | enum TLSMixedVarModes 89 | { 90 | TMV_None, 91 | TMV_Bool, 92 | TMV_Int, 93 | TMV_UInt, 94 | TMV_Double, 95 | TMV_Str 96 | }; 97 | 98 | // Designed to be extended but not overridden. 99 | class MixedVar 100 | { 101 | public: 102 | TLSMixedVarModes MxMode; 103 | std::int64_t MxInt; 104 | double MxDouble; 105 | char *MxStr; 106 | size_t MxStrPos; 107 | 108 | protected: 109 | TLS *MxTLS; 110 | 111 | public: 112 | MixedVar(TLS *TLSPtr = NULL); 113 | ~MixedVar(); 114 | 115 | MixedVar(const MixedVar &TempVar); 116 | MixedVar &operator=(const MixedVar &TempVar); 117 | 118 | inline void SetTLS(TLS *TLSPtr) 119 | { 120 | MxTLS = TLSPtr; 121 | } 122 | 123 | inline TLS *GetTLS() 124 | { 125 | return MxTLS; 126 | } 127 | 128 | // Some functions for those who prefer member functions over directly accessing raw class data. 129 | inline bool IsNone() 130 | { 131 | return (MxMode == TMV_None); 132 | } 133 | 134 | inline bool IsBool() 135 | { 136 | return (MxMode == TMV_Bool); 137 | } 138 | 139 | inline bool IsInt() 140 | { 141 | return (MxMode == TMV_Int); 142 | } 143 | 144 | inline bool IsUInt() 145 | { 146 | return (MxMode == TMV_UInt); 147 | } 148 | 149 | inline bool IsDouble() 150 | { 151 | return (MxMode == TMV_Double); 152 | } 153 | 154 | inline bool IsStr() 155 | { 156 | return (MxMode == TMV_Str); 157 | } 158 | 159 | inline bool GetBool() 160 | { 161 | return (MxInt != 0); 162 | } 163 | 164 | inline std::int64_t GetInt() 165 | { 166 | return MxInt; 167 | } 168 | 169 | inline std::uint64_t GetUInt() 170 | { 171 | return (std::uint64_t)MxInt; 172 | } 173 | 174 | inline double GetDouble() 175 | { 176 | return MxDouble; 177 | } 178 | 179 | inline char *GetStr() 180 | { 181 | return MxStr; 182 | } 183 | 184 | inline size_t GetSize() 185 | { 186 | return MxStrPos; 187 | } 188 | 189 | inline void SetBool(bool newbool) 190 | { 191 | MxMode = TMV_Bool; 192 | MxInt = (int)newbool; 193 | } 194 | 195 | inline void SetInt(std::int64_t newint) 196 | { 197 | MxMode = TMV_Int; 198 | MxInt = newint; 199 | } 200 | 201 | inline void SetUInt(std::uint64_t newint) 202 | { 203 | MxMode = TMV_UInt; 204 | MxInt = (std::int64_t)newint; 205 | } 206 | 207 | inline void SetDouble(double newdouble) 208 | { 209 | MxMode = TMV_Double; 210 | MxDouble = newdouble; 211 | } 212 | 213 | void SetData(const char *str, size_t size); 214 | void SetStr(const char *str); 215 | void PrependData(const char *str, size_t size); 216 | void PrependStr(const char *str); 217 | void PrependInt(const std::int64_t val, size_t radix = 10); 218 | void PrependUInt(const std::uint64_t val, size_t radix = 10); 219 | void PrependDouble(const double val, const size_t precision = 16); 220 | void AppendData(const char *str, size_t size); 221 | void AppendStr(const char *str); 222 | void AppendInt(const std::int64_t val, size_t radix = 10); 223 | void AppendUInt(const std::uint64_t val, size_t radix = 10); 224 | void AppendDouble(const double val, const size_t precision = 16); 225 | void AppendChar(char chr); 226 | void AppendMissingChar(char chr); 227 | void SetSize(size_t pos); 228 | size_t ReplaceData(const char *src, size_t srcsize, const char *dest, size_t destsize); 229 | size_t ReplaceStr(const char *src, const char *dest); 230 | 231 | inline bool RemoveTrailingChar(char chr) 232 | { 233 | if (!MxStrPos || MxStr[MxStrPos - 1] != chr) return false; 234 | 235 | MxStr[--MxStrPos] = '\0'; 236 | 237 | return true; 238 | } 239 | 240 | static bool IntToString(char *Result, size_t Size, std::uint64_t Num, size_t Radix = 10); 241 | static bool IntToString(char *Result, size_t Size, std::int64_t Num, size_t Radix = 10); 242 | }; 243 | 244 | private: 245 | // Deny copy constructor and assignment operator. Use a (smart) pointer instead. 246 | TLS(const TLS &); 247 | TLS &operator=(const TLS &); 248 | 249 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 250 | DWORD MxTlsIndex; 251 | #else 252 | pthread_key_t MxKey; 253 | #endif 254 | 255 | bool SetMainPtr(StaticVector> *MainPtr); 256 | StaticVector> *GetMainPtr(); 257 | 258 | static size_t NormalizeBitPosition(size_t &Size); 259 | }; 260 | } 261 | } 262 | 263 | #endif 264 | -------------------------------------------------------------------------------- /sync/sync_util.h: -------------------------------------------------------------------------------- 1 | // Cross-platform, cross-process synchronization utility functions. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_SYNC_UTIL 5 | #define CUBICLESOFT_SYNC_UTIL 6 | 7 | #include 8 | 9 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 10 | #include 11 | #else 12 | #include 13 | #endif 14 | 15 | #ifndef INFINITE 16 | #define INFINITE 0xFFFFFFFF 17 | #endif 18 | 19 | // Cross-platform utility class/functions. 20 | namespace CubicleSoft 21 | { 22 | namespace Sync 23 | { 24 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 25 | typedef DWORD ThreadIDType; 26 | #else 27 | typedef pthread_t ThreadIDType; 28 | #endif 29 | 30 | class Util 31 | { 32 | public: 33 | static ThreadIDType GetCurrentThreadID(); 34 | static std::uint64_t GetUnixMicrosecondTime(); 35 | 36 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 37 | #else 38 | // *NIX OSes require a lot of extra logic for named object support. 39 | // In general, do not directly call these functions. Use the specific classes instead. 40 | 41 | static size_t GetUnixSystemAlignmentSize(); 42 | static size_t AlignUnixSize(size_t Size); 43 | static int InitUnixNamedMem(char *&ResultMem, size_t &StartPos, const char *Prefix, const char *Name, size_t Size); 44 | static void UnixNamedMemReady(char *MemPtr); 45 | static void UnmapUnixNamedMem(char *MemPtr, size_t Size); 46 | 47 | // Some platforms are broken even for unnamed semaphores (e.g. Mac OSX). 48 | // Implements semaphores directly, bypassing POSIX semaphores. 49 | // Semaphores can be used in place of mutexes (i.e. Start = 1, Max = 1). 50 | class UnixSemaphoreWrapper 51 | { 52 | public: 53 | pthread_mutex_t *MxMutex; 54 | volatile std::uint32_t *MxCount; 55 | volatile std::uint32_t *MxMax; 56 | pthread_cond_t *MxCond; 57 | }; 58 | 59 | static size_t GetUnixSemaphoreSize(); 60 | static void GetUnixSemaphore(UnixSemaphoreWrapper &Result, char *Mem); 61 | static void InitUnixSemaphore(UnixSemaphoreWrapper &UnixSemaphore, bool Shared, std::uint32_t Start, std::uint32_t Max); 62 | static bool WaitForUnixSemaphore(UnixSemaphoreWrapper &UnixSemaphore, std::uint32_t Wait); 63 | static bool ReleaseUnixSemaphore(UnixSemaphoreWrapper &UnixSemaphore, std::uint32_t *PrevVal); 64 | static void FreeUnixSemaphore(UnixSemaphoreWrapper &UnixSemaphore); 65 | 66 | // Implements a more efficient (and portable) event object interface than trying to use semaphores. 67 | class UnixEventWrapper 68 | { 69 | public: 70 | pthread_mutex_t *MxMutex; 71 | volatile char *MxManual; 72 | volatile char *MxSignaled; 73 | volatile std::uint32_t *MxWaiting; 74 | pthread_cond_t *MxCond; 75 | }; 76 | 77 | static size_t GetUnixEventSize(); 78 | static void GetUnixEvent(UnixEventWrapper &Result, char *Mem); 79 | static void InitUnixEvent(UnixEventWrapper &UnixEvent, bool Shared, bool Manual, bool Signaled = false); 80 | static bool WaitForUnixEvent(UnixEventWrapper &UnixEvent, std::uint32_t Wait); 81 | static bool FireUnixEvent(UnixEventWrapper &UnixEvent); 82 | static bool ResetUnixEvent(UnixEventWrapper &UnixEvent); 83 | static void FreeUnixEvent(UnixEventWrapper &UnixEvent); 84 | #endif 85 | }; 86 | } 87 | } 88 | 89 | #endif 90 | -------------------------------------------------------------------------------- /templates/cache.cpp: -------------------------------------------------------------------------------- 1 | // Cache. 2 | // (C) 2021 CubicleSoft. All Rights Reserved. 3 | 4 | #include "cache.h" 5 | 6 | #if (defined(__GNUC__) && __GNUC__ >= 7) || (defined(__clang__) && __clang_major__ >= 12) 7 | #define FALL_THROUGH __attribute__ ((fallthrough)) 8 | #else 9 | #define FALL_THROUGH ((void)0) 10 | #endif 11 | 12 | namespace CubicleSoft 13 | { 14 | // Implements djb2. 15 | // WARNING: This algorithm is weak security-wise! http://www.youtube.com/watch?v=R2Cq3CLI6H8 16 | // It is generally okay to use in this instance since the cache template is a partial hash. 17 | // For a more secure hash function, try SipHash: https://github.com/veorq/SipHash 18 | 19 | size_t CacheUtil::GetHashKey(char *Str) 20 | { 21 | size_t Result = 5381, TempChr; 22 | 23 | while ((TempChr = *Str++)) 24 | { 25 | Result = ((Result << 5) + Result) ^ TempChr; 26 | } 27 | 28 | return Result; 29 | } 30 | 31 | size_t CacheUtil::GetHashKey(std::uint8_t *Str, size_t Size) 32 | { 33 | std::uint32_t Result = 5381; 34 | std::uint32_t y; 35 | const size_t NumLeft = Size & 3; 36 | const std::uint8_t *StrEnd = Str + Size - NumLeft; 37 | 38 | while (Str != StrEnd) 39 | { 40 | // Changes the official implementation. 41 | y = *((std::uint32_t *)Str); 42 | 43 | Result = ((Result << 5) + Result) ^ y; 44 | 45 | Str += 4; 46 | } 47 | 48 | switch (NumLeft) 49 | { 50 | case 3: Result = ((Result << 5) + Result) ^ ((std::uint32_t)Str[2]); FALL_THROUGH; 51 | case 2: Result = ((Result << 5) + Result) ^ ((std::uint32_t)Str[1]); FALL_THROUGH; 52 | case 1: Result = ((Result << 5) + Result) ^ ((std::uint32_t)Str[0]); break; 53 | case 0: break; 54 | } 55 | 56 | return (size_t)Result; 57 | } 58 | } 59 | -------------------------------------------------------------------------------- /templates/cache.h: -------------------------------------------------------------------------------- 1 | // Cache. 2 | // (C) 2013 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_CACHE 5 | #define CUBICLESOFT_CACHE 6 | 7 | #include 8 | #include 9 | 10 | namespace CubicleSoft 11 | { 12 | template 13 | class CacheNode 14 | { 15 | public: 16 | CacheNode() : Used(false) 17 | { 18 | } 19 | 20 | bool Used; 21 | XKey Key; 22 | XValue Value; 23 | }; 24 | 25 | // Prime numbers closest to powers of 2 make good hash table sizes (in theory). 26 | // Therefore, the best hash table primes are: 27 | // 3, 5, 11, 23, 47, 97, 191, 383, 769, 1531, 3067, 6143, 12289, 24571, 49157, 98299, 28 | // 196613, 393209, 786431, 1572869, 3145721, 6291449, 12582917, 25165813, 50331653, 29 | // 100663291, 201326611, 402653189, 805306357, 1610612741, 3221225473 30 | 31 | // Cache. A hash of overwritable storage. 32 | #include "cache_util.h" 33 | 34 | // CacheNoCopy. A hash of overwritable storage with a private copy constructor and assignment operator. 35 | #define CUBICLESOFT_CACHE_NOCOPYASSIGN 36 | #include "cache_util.h" 37 | #undef CUBICLESOFT_CACHE_NOCOPYASSIGN 38 | 39 | // Static functions that take a string and convert to integer. 40 | class CacheUtil 41 | { 42 | public: 43 | static size_t GetHashKey(char *Str); 44 | static size_t GetHashKey(std::uint8_t *Str, size_t Size); 45 | }; 46 | } 47 | 48 | #endif 49 | -------------------------------------------------------------------------------- /templates/cache_util.h: -------------------------------------------------------------------------------- 1 | // Cache. 2 | // (C) 2013 CubicleSoft. All Rights Reserved. 3 | 4 | // NOTE: This file is intended to be included from 'cache.h'. 5 | 6 | // Implements a hash for use primarily as a fixed-size cache. 7 | 8 | template 9 | #ifdef CUBICLESOFT_CACHE_NOCOPYASSIGN 10 | class CacheNoCopy 11 | #else 12 | class Cache 13 | #endif 14 | { 15 | public: 16 | #ifdef CUBICLESOFT_CACHE_NOCOPYASSIGN 17 | CacheNoCopy(size_t PrimeNum) : NumNodes(PrimeNum) 18 | #else 19 | Cache(size_t PrimeNum) : NumNodes(PrimeNum) 20 | #endif 21 | { 22 | Nodes = new CacheNode[NumNodes]; 23 | } 24 | 25 | #ifdef CUBICLESOFT_CACHE_NOCOPYASSIGN 26 | ~CacheNoCopy() 27 | #else 28 | ~Cache() 29 | #endif 30 | { 31 | if (Nodes != NULL) delete[] Nodes; 32 | } 33 | 34 | #ifdef CUBICLESOFT_CACHE_NOCOPYASSIGN 35 | CacheNoCopy(const CacheNoCopy &TempCache); 36 | CacheNoCopy &operator=(const CacheNoCopy &TempCache); 37 | #else 38 | Cache(const Cache &TempCache) 39 | { 40 | size_t x; 41 | 42 | NumNodes = TempCache.NumNodes; 43 | Nodes = new CacheNode[NumNodes]; 44 | for (x = 0; x < NumNodes; x++) Nodes[x] = TempCache.Nodes[x]; 45 | } 46 | 47 | Cache &operator=(const Cache &TempCache) 48 | { 49 | if (&TempCache != this) 50 | { 51 | size_t x; 52 | 53 | if (NumNodes != TempCache.NumNodes) 54 | { 55 | if (Nodes != NULL) delete[] Nodes; 56 | 57 | NumNodes = TempCache.NumNodes; 58 | Nodes = new CacheNode[NumNodes]; 59 | } 60 | 61 | for (x = 0; x < NumNodes; x++) Nodes[x] = TempCache.Nodes[x]; 62 | } 63 | 64 | return *this; 65 | } 66 | #endif 67 | 68 | void Empty() 69 | { 70 | size_t x; 71 | 72 | for (x = 0; x < NumNodes; x++) Nodes[x].Used = false; 73 | } 74 | 75 | void Insert(size_t HashKey, const XKey &Key, const XValue &Value) 76 | { 77 | HashKey = HashKey % NumNodes; 78 | 79 | Nodes[HashKey].Used = true; 80 | Nodes[HashKey].Key = Key; 81 | Nodes[HashKey].Value = Value; 82 | } 83 | 84 | bool Remove(size_t HashKey, const XKey &Key) 85 | { 86 | HashKey = HashKey % NumNodes; 87 | if (Nodes[HashKey].Used && Nodes[HashKey].Key == Key) 88 | { 89 | Nodes[HashKey].Used = false; 90 | 91 | return true; 92 | } 93 | 94 | return false; 95 | } 96 | 97 | bool Exists(size_t HashKey, const XKey &Key) 98 | { 99 | HashKey = HashKey % NumNodes; 100 | 101 | return (Nodes[HashKey].Used && Nodes[HashKey].Key == Key); 102 | } 103 | 104 | bool Find(XValue &Result, size_t HashKey, const XKey &Key) 105 | { 106 | HashKey = HashKey % NumNodes; 107 | if (Nodes[HashKey].Used && Nodes[HashKey].Key == Key) 108 | { 109 | Result = Nodes[HashKey].Value; 110 | 111 | return true; 112 | } 113 | 114 | return false; 115 | } 116 | 117 | inline size_t GetSize() { return NumNodes; } 118 | inline CacheNode *RawData() { return Nodes; } 119 | 120 | private: 121 | CacheNode *Nodes; 122 | size_t NumNodes; 123 | }; 124 | -------------------------------------------------------------------------------- /templates/detachable_list.h: -------------------------------------------------------------------------------- 1 | // Double linked list with detachable nodes. 2 | // (C) 2013 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_DETACHABLE_LIST 5 | #define CUBICLESOFT_DETACHABLE_LIST 6 | 7 | namespace CubicleSoft 8 | { 9 | template 10 | class List; 11 | template 12 | class ListNoCopy; 13 | 14 | template 15 | class ListNode 16 | { 17 | friend class List; 18 | friend class ListNoCopy; 19 | 20 | public: 21 | ListNode() : PrevNode(NULL), NextNode(NULL) 22 | { 23 | } 24 | 25 | inline ListNode *Prev() { return PrevNode; } 26 | inline ListNode *Next() { return NextNode; } 27 | 28 | private: 29 | ListNode *PrevNode; 30 | ListNode *NextNode; 31 | 32 | public: 33 | T Value; 34 | }; 35 | 36 | // List. A double linked list. 37 | #include "detachable_list_util.h" 38 | 39 | // ListNoCopy. A double linked list with a private copy constructor and assignment operator. 40 | #define CUBICLESOFT_DETACHABLE_LIST_NOCOPYASSIGN 41 | #include "detachable_list_util.h" 42 | #undef CUBICLESOFT_DETACHABLE_LIST_NOCOPYASSIGN 43 | } 44 | 45 | #endif -------------------------------------------------------------------------------- /templates/detachable_list_util.h: -------------------------------------------------------------------------------- 1 | // Double linked list with detachable nodes. 2 | // (C) 2013 CubicleSoft. All Rights Reserved. 3 | 4 | // NOTE: This file is intended to be included from 'detachable_list.h'. 5 | 6 | // Implements a list that can detach and attach compatible nodes. 7 | template 8 | #ifdef CUBICLESOFT_DETACHABLE_LIST_NOCOPYASSIGN 9 | class ListNoCopy 10 | #else 11 | class List 12 | #endif 13 | { 14 | public: 15 | #ifdef CUBICLESOFT_DETACHABLE_LIST_NOCOPYASSIGN 16 | ListNoCopy() : FirstNode(NULL), LastNode(NULL), NumNodes(0) 17 | #else 18 | List() : FirstNode(NULL), LastNode(NULL), NumNodes(0) 19 | #endif 20 | { 21 | } 22 | 23 | #ifdef CUBICLESOFT_DETACHABLE_LIST_NOCOPYASSIGN 24 | ~ListNoCopy() 25 | #else 26 | ~List() 27 | #endif 28 | { 29 | Empty(); 30 | } 31 | 32 | #ifdef CUBICLESOFT_DETACHABLE_LIST_NOCOPYASSIGN 33 | ListNoCopy(T Value) 34 | #else 35 | List(T Value) 36 | #endif 37 | { 38 | FirstNode = NULL; 39 | LastNode = NULL; 40 | NumNodes = 0; 41 | 42 | InsertBefore(NULL, Value); 43 | } 44 | 45 | #ifdef CUBICLESOFT_DETACHABLE_LIST_NOCOPYASSIGN 46 | private: 47 | ListNoCopy(const ListNoCopy &); 48 | ListNoCopy &operator=(const ListNoCopy &); 49 | 50 | public: 51 | #else 52 | List(const List &TempList) 53 | { 54 | FirstNode = NULL; 55 | LastNode = NULL; 56 | NumNodes = 0; 57 | 58 | ListNode *Node = TempList.FirstNode; 59 | while (Node != NULL) 60 | { 61 | InsertBefore(NULL, Node->Value); 62 | Node = Node->NextNode; 63 | } 64 | } 65 | 66 | List &operator=(const List &TempList) 67 | { 68 | if (&TempList != this) 69 | { 70 | Empty(); 71 | 72 | ListNode *Node = TempList.FirstNode; 73 | while (Node != NULL) 74 | { 75 | InsertBefore(NULL, Node->Value); 76 | Node = Node->NextNode; 77 | } 78 | } 79 | 80 | return *this; 81 | } 82 | #endif 83 | 84 | #ifdef CUBICLESOFT_DETACHABLE_LIST_NOCOPYASSIGN 85 | inline ListNoCopy &operator+=(const T &Value) 86 | #else 87 | inline List &operator+=(const T &Value) 88 | #endif 89 | { 90 | InsertBefore(NULL, Value); 91 | 92 | return *this; 93 | } 94 | 95 | #ifdef CUBICLESOFT_DETACHABLE_LIST_NOCOPYASSIGN 96 | inline ListNoCopy &operator+=(ListNode *Node) 97 | #else 98 | inline List &operator+=(ListNode *Node) 99 | #endif 100 | { 101 | InsertBefore(NULL, Node); 102 | 103 | return *this; 104 | } 105 | 106 | inline ListNode *Push(const T &Value) 107 | { 108 | return InsertBefore(NULL, Value); 109 | } 110 | 111 | // Only use with detached nodes. 112 | inline ListNode *Push(ListNode *Node) 113 | { 114 | return InsertBefore(NULL, Node); 115 | } 116 | 117 | ListNode *Pop() 118 | { 119 | ListNode *Node = LastNode; 120 | Detach(Node); 121 | 122 | return Node; 123 | } 124 | 125 | inline ListNode *Unshift(const T &Value) 126 | { 127 | return InsertAfter(NULL, Value); 128 | } 129 | 130 | // Only use with detached nodes. 131 | inline ListNode *Unshift(ListNode *Node) 132 | { 133 | return InsertAfter(NULL, Node); 134 | } 135 | 136 | ListNode *Shift() 137 | { 138 | ListNode *Node = FirstNode; 139 | Detach(Node); 140 | 141 | return Node; 142 | } 143 | 144 | static inline ListNode *CreateNode() 145 | { 146 | return new ListNode; 147 | } 148 | 149 | static inline ListNode *CreateNode(const T &Value) 150 | { 151 | ListNode *Node = new ListNode; 152 | Node->Value = Value; 153 | 154 | return Node; 155 | } 156 | 157 | ListNode *InsertBefore(ListNode *Next, const T &Value) 158 | { 159 | ListNode *Node; 160 | 161 | Node = new ListNode; 162 | Node->Value = Value; 163 | if (Next == NULL) 164 | { 165 | Node->PrevNode = LastNode; 166 | if (LastNode != NULL) LastNode->NextNode = Node; 167 | LastNode = Node; 168 | if (FirstNode == NULL) FirstNode = LastNode; 169 | } 170 | else 171 | { 172 | Node->NextNode = Next; 173 | Node->PrevNode = Next->PrevNode; 174 | if (Node->PrevNode != NULL) Node->PrevNode->NextNode = Node; 175 | Next->PrevNode = Node; 176 | if (Next == FirstNode) FirstNode = Node; 177 | } 178 | NumNodes++; 179 | 180 | return Node; 181 | } 182 | 183 | // Only use with detached nodes. 184 | ListNode *InsertBefore(ListNode *Next, ListNode *Node) 185 | { 186 | if (Node->NextNode != NULL || Node->PrevNode != NULL) return NULL; 187 | 188 | if (Next == NULL) 189 | { 190 | Node->PrevNode = LastNode; 191 | if (LastNode != NULL) LastNode->NextNode = Node; 192 | LastNode = Node; 193 | if (FirstNode == NULL) FirstNode = LastNode; 194 | } 195 | else 196 | { 197 | Node->NextNode = Next; 198 | Node->PrevNode = Next->PrevNode; 199 | if (Node->PrevNode != NULL) Node->PrevNode->NextNode = Node; 200 | Next->PrevNode = Node; 201 | if (Next == FirstNode) FirstNode = Node; 202 | } 203 | NumNodes++; 204 | 205 | return Node; 206 | } 207 | 208 | ListNode *InsertAfter(ListNode *Prev, const T &Value) 209 | { 210 | ListNode *Node; 211 | 212 | Node = new ListNode; 213 | Node->Value = Value; 214 | if (Prev == NULL) 215 | { 216 | Node->NextNode = FirstNode; 217 | if (FirstNode != NULL) FirstNode->PrevNode = Node; 218 | FirstNode = Node; 219 | if (LastNode == NULL) LastNode = FirstNode; 220 | } 221 | else 222 | { 223 | Node->PrevNode = Prev; 224 | Node->NextNode = Prev->NextNode; 225 | if (Node->NextNode != NULL) Node->NextNode->PrevNode = Node; 226 | Prev->NextNode = Node; 227 | if (Prev == LastNode) LastNode = Node; 228 | } 229 | NumNodes++; 230 | 231 | return Node; 232 | } 233 | 234 | // Only use with detached nodes. 235 | ListNode *InsertAfter(ListNode *Prev, ListNode *Node) 236 | { 237 | if (Node->NextNode != NULL || Node->PrevNode != NULL) return NULL; 238 | 239 | if (Prev == NULL) 240 | { 241 | Node->NextNode = FirstNode; 242 | if (FirstNode != NULL) FirstNode->PrevNode = Node; 243 | FirstNode = Node; 244 | if (LastNode == NULL) LastNode = FirstNode; 245 | } 246 | else 247 | { 248 | Node->PrevNode = Prev; 249 | Node->NextNode = Prev->NextNode; 250 | if (Node->NextNode != NULL) Node->NextNode->PrevNode = Node; 251 | Prev->NextNode = Node; 252 | if (Prev == LastNode) LastNode = Node; 253 | } 254 | NumNodes++; 255 | 256 | return Node; 257 | } 258 | 259 | // Detaches all of the nodes in TempList and appends them to the end of the list. 260 | #ifdef CUBICLESOFT_DETACHABLE_LIST_NOCOPYASSIGN 261 | void DetachAllAndAppend(ListNoCopy &TempList) 262 | #else 263 | void DetachAllAndAppend(List &TempList) 264 | #endif 265 | { 266 | if (TempList.FirstNode == NULL) return; 267 | 268 | if (LastNode != NULL) 269 | { 270 | LastNode->NextNode = TempList.FirstNode; 271 | TempList.FirstNode->PrevNode = LastNode; 272 | LastNode = TempList.LastNode; 273 | } 274 | else 275 | { 276 | FirstNode = TempList.FirstNode; 277 | LastNode = TempList.LastNode; 278 | } 279 | NumNodes += TempList.NumNodes; 280 | 281 | TempList.FirstNode = NULL; 282 | TempList.LastNode = NULL; 283 | TempList.NumNodes = 0; 284 | } 285 | 286 | bool Detach(ListNode *Node) 287 | { 288 | if (Node == NULL) return false; 289 | 290 | // Disconnect the first/last node (if necessary). 291 | if (Node == FirstNode) 292 | { 293 | FirstNode = FirstNode->NextNode; 294 | if (FirstNode != NULL) FirstNode->PrevNode = NULL; 295 | else LastNode = NULL; 296 | } 297 | else if (Node == LastNode) 298 | { 299 | LastNode = LastNode->PrevNode; 300 | if (LastNode != NULL) LastNode->NextNode = NULL; 301 | else FirstNode = NULL; 302 | } 303 | else 304 | { 305 | if (Node->NextNode != NULL) Node->NextNode->PrevNode = Node->PrevNode; 306 | if (Node->PrevNode != NULL) Node->PrevNode->NextNode = Node->NextNode; 307 | } 308 | Node->NextNode = NULL; 309 | Node->PrevNode = NULL; 310 | 311 | NumNodes--; 312 | 313 | return true; 314 | } 315 | 316 | bool Remove(ListNode *Node) 317 | { 318 | if (!Detach(Node)) return false; 319 | 320 | delete Node; 321 | 322 | return true; 323 | } 324 | 325 | void Empty() 326 | { 327 | while (FirstNode != NULL) Remove(FirstNode); 328 | NumNodes = 0; 329 | } 330 | 331 | inline ListNode *First() const { return FirstNode; } 332 | inline ListNode *Last() const { return LastNode; } 333 | inline size_t GetSize() const { return NumNodes; } 334 | 335 | private: 336 | ListNode *FirstNode, *LastNode; 337 | size_t NumNodes; 338 | }; 339 | -------------------------------------------------------------------------------- /templates/detachable_ordered_hash.cpp: -------------------------------------------------------------------------------- 1 | // Ordered Hash Utilities. 2 | // (C) 2014 CubicleSoft. All Rights Reserved. 3 | 4 | #include "detachable_ordered_hash.h" 5 | 6 | #if (defined(__GNUC__) && __GNUC__ >= 7) || (defined(__clang__) && __clang_major__ >= 12) 7 | #define FALL_THROUGH __attribute__ ((fallthrough)) 8 | #else 9 | #define FALL_THROUGH ((void)0) 10 | #endif 11 | 12 | namespace CubicleSoft 13 | { 14 | // Prime numbers closest to powers of 2 make good hash table sizes (in theory). 15 | // Therefore, the best hash table primes are: 16 | const size_t OrderedHashUtil::Primes[31] = { 17 | 3, 5, 11, 23, 47, 97, 191, 383, 769, 1531, 3067, 6143, 12289, 24571, 49157, 98299, 18 | 196613, 393209, 786431, 1572869, 3145721, 6291449, 12582917, 25165813, 50331653, 19 | 100663291, 201326611, 402653189, 805306357, 1610612741, 3221225473 20 | }; 21 | 22 | size_t OrderedHashUtil::GetDJBX33XHashKey(const std::uint8_t *Str, size_t Size, size_t InitVal) 23 | { 24 | std::uint32_t Result = (std::uint32_t)InitVal; 25 | std::uint32_t y; 26 | const size_t NumLeft = Size & 3; 27 | const std::uint8_t *StrEnd = Str + Size - NumLeft; 28 | 29 | while (Str != StrEnd) 30 | { 31 | // Changes the official implementation. 32 | y = *((const std::uint32_t *)Str); 33 | 34 | Result = ((Result << 5) + Result) ^ y; 35 | 36 | Str += 4; 37 | } 38 | 39 | switch (NumLeft) 40 | { 41 | case 3: Result = ((Result << 5) + Result) ^ ((std::uint32_t)Str[2]); FALL_THROUGH; 42 | case 2: Result = ((Result << 5) + Result) ^ ((std::uint32_t)Str[1]); FALL_THROUGH; 43 | case 1: Result = ((Result << 5) + Result) ^ ((std::uint32_t)Str[0]); break; 44 | case 0: break; 45 | } 46 | 47 | return (size_t)Result; 48 | } 49 | 50 | #define ROTL(x, b) (std::uint64_t)(((x) << (b)) | ((x) >> (64 - (b)))) 51 | 52 | #define SIPROUND \ 53 | do { \ 54 | v0 += v1; v1 = ROTL(v1, 13); v1 ^= v0; v0 = ROTL(v0, 32); \ 55 | v2 += v3; v3 = ROTL(v3, 16); v3 ^= v2; \ 56 | v0 += v3; v3 = ROTL(v3, 21); v3 ^= v0; \ 57 | v2 += v1; v1 = ROTL(v1, 17); v1 ^= v2; v2 = ROTL(v2, 32); \ 58 | } while(0) 59 | 60 | std::uint64_t OrderedHashUtil::GetSipHashKey(const std::uint8_t *Str, size_t Size, std::uint64_t Key1, std::uint64_t Key2, size_t cRounds, size_t dRounds) 61 | { 62 | // "somepseudorandomlygeneratedbytes" 63 | std::uint64_t v0 = 0x736f6d6570736575ULL; 64 | std::uint64_t v1 = 0x646f72616e646f6dULL; 65 | std::uint64_t v2 = 0x6c7967656e657261ULL; 66 | std::uint64_t v3 = 0x7465646279746573ULL; 67 | std::uint64_t Result; 68 | std::uint64_t y; 69 | size_t x; 70 | const size_t NumLeft = Size & 7; 71 | const std::uint8_t *StrEnd = Str + Size - NumLeft; 72 | 73 | Result = ((std::uint64_t)Size) << 56; 74 | v3 ^= Key2; 75 | v2 ^= Key1; 76 | v1 ^= Key2; 77 | v0 ^= Key1; 78 | while (Str != StrEnd) 79 | { 80 | // Minor change to the official implementation. (Does endianness actually matter?) 81 | y = *((const std::uint64_t *)Str); 82 | 83 | v3 ^= y; 84 | 85 | for (x = 0; x < cRounds; ++x) SIPROUND; 86 | 87 | v0 ^= y; 88 | 89 | Str += 8; 90 | } 91 | 92 | switch (NumLeft) 93 | { 94 | case 7: Result |= ((std::uint64_t)Str[6]) << 48; FALL_THROUGH; 95 | case 6: Result |= ((std::uint64_t)Str[5]) << 40; FALL_THROUGH; 96 | case 5: Result |= ((std::uint64_t)Str[4]) << 32; FALL_THROUGH; 97 | case 4: Result |= ((std::uint64_t)Str[3]) << 24; FALL_THROUGH; 98 | case 3: Result |= ((std::uint64_t)Str[2]) << 16; FALL_THROUGH; 99 | case 2: Result |= ((std::uint64_t)Str[1]) << 8; FALL_THROUGH; 100 | case 1: Result |= ((std::uint64_t)Str[0]); break; 101 | case 0: break; 102 | } 103 | 104 | v3 ^= Result; 105 | 106 | for (x = 0; x < cRounds; ++x) SIPROUND; 107 | 108 | v0 ^= Result; 109 | v2 ^= 0xff; 110 | 111 | for (x = 0; x < dRounds; ++x) SIPROUND; 112 | 113 | Result = v0 ^ v1 ^ v2 ^ v3; 114 | 115 | return Result; 116 | } 117 | } 118 | -------------------------------------------------------------------------------- /templates/detachable_ordered_hash.h: -------------------------------------------------------------------------------- 1 | // Ordered hash map with integer and string keys with detachable nodes. 2 | // (C) 2014 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_DETACHABLE_ORDEREDHASH 5 | #define CUBICLESOFT_DETACHABLE_ORDEREDHASH 6 | 7 | #include 8 | #include 9 | #include 10 | 11 | namespace CubicleSoft 12 | { 13 | template 14 | class OrderedHash; 15 | template 16 | class OrderedHashNoCopy; 17 | 18 | template 19 | class OrderedHashNode 20 | { 21 | friend class OrderedHash; 22 | friend class OrderedHashNoCopy; 23 | 24 | public: 25 | OrderedHashNode() : PrevHashNode(NULL), NextHashNode(NULL), PrevListNode(NULL), NextListNode(NULL), HashKey(0), IntKey(0), StrKey(NULL) 26 | { 27 | } 28 | 29 | ~OrderedHashNode() 30 | { 31 | if (StrKey != NULL) delete[] StrKey; 32 | } 33 | 34 | // Only copies the data. Pointers will still need to be set. 35 | OrderedHashNode(const OrderedHashNode &TempNode) 36 | { 37 | HashKey = TempNode.HashKey; 38 | IntKey = TempNode.IntKey; 39 | 40 | if (TempNode.StrKey == NULL) StrKey = NULL; 41 | else 42 | { 43 | StrKey = new char[IntKey]; 44 | std::memcpy(StrKey, TempNode.StrKey, IntKey); 45 | } 46 | 47 | Value = TempNode.Value; 48 | } 49 | 50 | OrderedHashNode &operator=(const OrderedHashNode &TempNode) 51 | { 52 | if (&TempNode != this) 53 | { 54 | HashKey = TempNode.HashKey; 55 | IntKey = TempNode.IntKey; 56 | 57 | if (StrKey != NULL) delete[] StrKey; 58 | 59 | if (TempNode.StrKey == NULL) StrKey = NULL; 60 | else 61 | { 62 | StrKey = new char[IntKey]; 63 | std::memcpy(StrKey, TempNode.StrKey, IntKey); 64 | } 65 | 66 | Value = TempNode.Value; 67 | } 68 | 69 | return *this; 70 | } 71 | 72 | inline OrderedHashNode *PrevHash() { return PrevHashNode; } 73 | inline OrderedHashNode *NextHash() { return NextHashNode; } 74 | inline OrderedHashNode *PrevList() { return PrevListNode; } 75 | inline OrderedHashNode *NextList() { return NextListNode; } 76 | inline char *GetStrKey() { return StrKey; } 77 | inline std::int64_t GetIntKey() { return IntKey; } 78 | 79 | // FreeStrKey will delete[] an existing string in this node. 80 | // If you manage your own memory (see SetStrKey), be sure to set it to false. 81 | void SetIntKey(const std::int64_t NewIntKey, const bool FreeStrKey) 82 | { 83 | HashKey = 0; 84 | IntKey = NewIntKey; 85 | if (FreeStrKey && StrKey != NULL) delete[] StrKey; 86 | StrKey = NULL; 87 | } 88 | 89 | // When CopyStr is false (i.e. managing your own memory), 90 | // SetStrKey() only copies the pointer and you are expected to 91 | // later call SetStrKey(NULL, 0, false) on the node. 92 | // Best used with OrderedHashNoCopy to avoid accidental 93 | // assignment/copy constructor issues. 94 | void SetStrKey(const char *NewStrKey, const size_t NewStrLen, const bool CopyStr) 95 | { 96 | HashKey = 0; 97 | 98 | if (!CopyStr) StrKey = (char *)NewStrKey; 99 | else 100 | { 101 | if (StrKey == NULL) StrKey = new char[NewStrLen]; 102 | else if (IntKey < (std::int64_t)NewStrLen) 103 | { 104 | delete[] StrKey; 105 | StrKey = new char[NewStrLen]; 106 | } 107 | 108 | std::memcpy(StrKey, NewStrKey, NewStrLen); 109 | } 110 | 111 | IntKey = (std::int64_t)NewStrLen; 112 | } 113 | 114 | private: 115 | OrderedHashNode *PrevHashNode; 116 | OrderedHashNode *NextHashNode; 117 | OrderedHashNode *PrevListNode; 118 | public: 119 | // NextListNode is public to allow temporary chains of detached nodes to be forged. 120 | OrderedHashNode *NextListNode; 121 | 122 | private: 123 | std::uint64_t HashKey; 124 | 125 | std::int64_t IntKey; 126 | char *StrKey; 127 | 128 | public: 129 | T Value; 130 | }; 131 | 132 | class OrderedHashUtil 133 | { 134 | public: 135 | static const size_t Primes[31]; 136 | static size_t GetDJBX33XHashKey(const std::uint8_t *Str, size_t Size, size_t InitVal); 137 | static std::uint64_t GetSipHashKey(const std::uint8_t *Str, size_t Size, std::uint64_t Key1, std::uint64_t Key2, size_t cRounds, size_t dRounds); 138 | }; 139 | 140 | // OrderedHash. An ordered hash. 141 | #include "detachable_ordered_hash_util.h" 142 | 143 | // OrderedHashNoCopy. An ordered hash with a private copy constructor and assignment operator. 144 | #define CUBICLESOFT_DETACHABLE_ORDEREDHASH_NOCOPYASSIGN 145 | #include "detachable_ordered_hash_util.h" 146 | #undef CUBICLESOFT_DETACHABLE_ORDEREDHASH_NOCOPYASSIGN 147 | } 148 | 149 | #endif -------------------------------------------------------------------------------- /templates/detachable_queue.h: -------------------------------------------------------------------------------- 1 | // Queue with detachable nodes. 2 | // (C) 2013 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_DETACHABLE_QUEUE 5 | #define CUBICLESOFT_DETACHABLE_QUEUE 6 | 7 | #include 8 | 9 | namespace CubicleSoft 10 | { 11 | template 12 | class Queue; 13 | template 14 | class QueueNoCopy; 15 | 16 | template 17 | class QueueNode 18 | { 19 | friend class Queue; 20 | friend class QueueNoCopy; 21 | 22 | public: 23 | QueueNode() : NextNode(NULL) 24 | { 25 | } 26 | 27 | inline QueueNode *Next() { return NextNode; } 28 | 29 | private: 30 | QueueNode *NextNode; 31 | 32 | public: 33 | T Value; 34 | }; 35 | 36 | // Queue. A single linked list. 37 | #include "detachable_queue_util.h" 38 | 39 | // QueueNoCopy. A single linked list with a private copy constructor and assignment operator. 40 | #define CUBICLESOFT_DETACHABLE_QUEUE_NOCOPYASSIGN 41 | #include "detachable_queue_util.h" 42 | #undef CUBICLESOFT_DETACHABLE_QUEUE_NOCOPYASSIGN 43 | } 44 | 45 | #endif -------------------------------------------------------------------------------- /templates/detachable_queue_util.h: -------------------------------------------------------------------------------- 1 | // Queue with detachable nodes. 2 | // (C) 2013 CubicleSoft. All Rights Reserved. 3 | 4 | // NOTE: This file is intended to be included from 'detachable_queue.h'. 5 | 6 | // Implements a queue that can detach and attach compatible nodes. 7 | template 8 | #ifdef CUBICLESOFT_DETACHABLE_QUEUE_NOCOPYASSIGN 9 | class QueueNoCopy 10 | #else 11 | class Queue 12 | #endif 13 | { 14 | public: 15 | #ifdef CUBICLESOFT_DETACHABLE_QUEUE_NOCOPYASSIGN 16 | QueueNoCopy() : FirstNode(NULL), LastNode(NULL), NumNodes(0) 17 | #else 18 | Queue() : FirstNode(NULL), LastNode(NULL), NumNodes(0) 19 | #endif 20 | { 21 | } 22 | 23 | #ifdef CUBICLESOFT_DETACHABLE_QUEUE_NOCOPYASSIGN 24 | ~QueueNoCopy() 25 | #else 26 | ~Queue() 27 | #endif 28 | { 29 | Empty(); 30 | } 31 | 32 | #ifdef CUBICLESOFT_DETACHABLE_QUEUE_NOCOPYASSIGN 33 | QueueNoCopy(T Value) : FirstNode(NULL), LastNode(NULL), NumNodes(0) 34 | #else 35 | Queue(T Value) : FirstNode(NULL), LastNode(NULL), NumNodes(0) 36 | #endif 37 | { 38 | Push(Value); 39 | } 40 | 41 | #ifdef CUBICLESOFT_DETACHABLE_QUEUE_NOCOPYASSIGN 42 | private: 43 | QueueNoCopy(const Queue &); 44 | QueueNoCopy &operator=(const QueueNoCopy &); 45 | 46 | public: 47 | #else 48 | Queue(const Queue &TempQueue) 49 | { 50 | FirstNode = NULL; 51 | LastNode = NULL; 52 | NumNodes = 0; 53 | 54 | QueueNode *Node = TempQueue.FirstNode; 55 | while (Node != NULL) 56 | { 57 | Push(Node->Value); 58 | Node = Node->NextNode; 59 | } 60 | } 61 | 62 | Queue &operator=(const Queue &TempQueue) 63 | { 64 | if (&TempQueue != this) 65 | { 66 | Empty(); 67 | 68 | QueueNode *Node = TempQueue.FirstNode; 69 | while (Node != NULL) 70 | { 71 | Push(Node->Value); 72 | Node = Node->NextNode; 73 | } 74 | } 75 | 76 | return *this; 77 | } 78 | #endif 79 | 80 | #ifdef CUBICLESOFT_DETACHABLE_QUEUE_NOCOPYASSIGN 81 | inline QueueNoCopy &operator+=(const T &Value) 82 | #else 83 | inline Queue &operator+=(const T &Value) 84 | #endif 85 | { 86 | Push(Value); 87 | 88 | return *this; 89 | } 90 | 91 | #ifdef CUBICLESOFT_DETACHABLE_QUEUE_NOCOPYASSIGN 92 | inline QueueNoCopy &operator+=(QueueNode *Node) 93 | #else 94 | inline Queue &operator+=(QueueNode *Node) 95 | #endif 96 | { 97 | Push(Node); 98 | 99 | return *this; 100 | } 101 | 102 | QueueNode *Push(const T &Value) 103 | { 104 | QueueNode *Node; 105 | 106 | Node = new QueueNode; 107 | Node->Value = Value; 108 | 109 | return Push(Node); 110 | } 111 | 112 | // Only use with detached nodes. 113 | QueueNode *Push(QueueNode *Node) 114 | { 115 | if (Node->NextNode != NULL) return NULL; 116 | 117 | if (LastNode != NULL) LastNode->NextNode = Node; 118 | LastNode = Node; 119 | if (FirstNode == NULL) FirstNode = LastNode; 120 | NumNodes++; 121 | 122 | return Node; 123 | } 124 | 125 | QueueNode *Unshift(const T &Value) 126 | { 127 | QueueNode *Node; 128 | 129 | Node = new QueueNode; 130 | Node->Value = Value; 131 | 132 | return Unshift(Node); 133 | } 134 | 135 | // Only use with detached nodes. 136 | QueueNode *Unshift(QueueNode *Node) 137 | { 138 | if (Node->NextNode != NULL) return NULL; 139 | 140 | Node->NextNode = FirstNode; 141 | FirstNode = Node; 142 | if (LastNode == NULL) LastNode = FirstNode; 143 | NumNodes++; 144 | 145 | return Node; 146 | } 147 | 148 | QueueNode *Shift() 149 | { 150 | if (FirstNode == NULL) return NULL; 151 | 152 | QueueNode *Node = FirstNode; 153 | FirstNode = Node->NextNode; 154 | if (FirstNode == NULL) LastNode = NULL; 155 | Node->NextNode = NULL; 156 | NumNodes--; 157 | 158 | return Node; 159 | } 160 | 161 | static inline QueueNode *CreateNode() 162 | { 163 | return new QueueNode; 164 | } 165 | 166 | static inline QueueNode *CreateNode(const T &Value) 167 | { 168 | QueueNode *Node = new QueueNode; 169 | Node->Value = Value; 170 | 171 | return Node; 172 | } 173 | 174 | // Detaches all of the nodes in TempQueue and appends them to the end of the queue. 175 | #ifdef CUBICLESOFT_DETACHABLE_QUEUE_NOCOPYASSIGN 176 | void DetachAllAndAppend(QueueNoCopy &TempQueue) 177 | #else 178 | void DetachAllAndAppend(Queue &TempQueue) 179 | #endif 180 | { 181 | if (TempQueue.FirstNode == NULL) return; 182 | 183 | if (LastNode != NULL) 184 | { 185 | LastNode->NextNode = TempQueue.FirstNode; 186 | LastNode = TempQueue.LastNode; 187 | } 188 | else 189 | { 190 | FirstNode = TempQueue.FirstNode; 191 | LastNode = TempQueue.LastNode; 192 | } 193 | NumNodes += TempQueue.NumNodes; 194 | 195 | TempQueue.FirstNode = NULL; 196 | TempQueue.LastNode = NULL; 197 | TempQueue.NumNodes = 0; 198 | } 199 | 200 | void Empty() 201 | { 202 | QueueNode *Node; 203 | 204 | while (FirstNode != NULL) 205 | { 206 | Node = Shift(); 207 | delete Node; 208 | } 209 | 210 | NumNodes = 0; 211 | } 212 | 213 | inline QueueNode *First() const { return FirstNode; } 214 | inline QueueNode *Last() const { return LastNode; } 215 | inline size_t GetSize() const { return NumNodes; } 216 | 217 | private: 218 | QueueNode *FirstNode, *LastNode; 219 | size_t NumNodes; 220 | }; 221 | -------------------------------------------------------------------------------- /templates/fast_find_replace.h: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cubiclesoft/cross-platform-cpp/c3fbfedc5e98b3a5381c950250db33c99941392d/templates/fast_find_replace.h -------------------------------------------------------------------------------- /templates/fast_find_replace_util.h: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cubiclesoft/cross-platform-cpp/c3fbfedc5e98b3a5381c950250db33c99941392d/templates/fast_find_replace_util.h -------------------------------------------------------------------------------- /templates/packed_ordered_hash.cpp: -------------------------------------------------------------------------------- 1 | // Packed Ordered Hash Utilities. 2 | // (C) 2014 CubicleSoft. All Rights Reserved. 3 | 4 | #include "packed_ordered_hash.h" 5 | 6 | #if (defined(__GNUC__) && __GNUC__ >= 7) || (defined(__clang__) && __clang_major__ >= 12) 7 | #define FALL_THROUGH __attribute__ ((fallthrough)) 8 | #else 9 | #define FALL_THROUGH ((void)0) 10 | #endif 11 | 12 | namespace CubicleSoft 13 | { 14 | size_t PackedOrderedHashUtil::GetDJBX33XHashKey(const std::uint8_t *Str, size_t Size, size_t InitVal) 15 | { 16 | std::uint32_t Result = (std::uint32_t)InitVal; 17 | std::uint32_t y; 18 | const size_t NumLeft = Size & 3; 19 | const std::uint8_t *StrEnd = Str + Size - NumLeft; 20 | 21 | while (Str != StrEnd) 22 | { 23 | // Changes the official implementation. 24 | y = *((const std::uint32_t *)Str); 25 | 26 | Result = ((Result << 5) + Result) ^ y; 27 | 28 | Str += 4; 29 | } 30 | 31 | switch (NumLeft) 32 | { 33 | case 3: Result = ((Result << 5) + Result) ^ ((std::uint32_t)Str[2]); FALL_THROUGH; 34 | case 2: Result = ((Result << 5) + Result) ^ ((std::uint32_t)Str[1]); FALL_THROUGH; 35 | case 1: Result = ((Result << 5) + Result) ^ ((std::uint32_t)Str[0]); break; 36 | case 0: break; 37 | } 38 | 39 | return (size_t)Result; 40 | } 41 | 42 | #define ROTL(x, b) (std::uint64_t)(((x) << (b)) | ((x) >> (64 - (b)))) 43 | 44 | #define SIPROUND \ 45 | do { \ 46 | v0 += v1; v1 = ROTL(v1, 13); v1 ^= v0; v0 = ROTL(v0, 32); \ 47 | v2 += v3; v3 = ROTL(v3, 16); v3 ^= v2; \ 48 | v0 += v3; v3 = ROTL(v3, 21); v3 ^= v0; \ 49 | v2 += v1; v1 = ROTL(v1, 17); v1 ^= v2; v2 = ROTL(v2, 32); \ 50 | } while(0) 51 | 52 | std::uint64_t PackedOrderedHashUtil::GetSipHashKey(const std::uint8_t *Str, size_t Size, std::uint64_t Key1, std::uint64_t Key2, size_t cRounds, size_t dRounds) 53 | { 54 | // "somepseudorandomlygeneratedbytes" 55 | std::uint64_t v0 = 0x736f6d6570736575ULL; 56 | std::uint64_t v1 = 0x646f72616e646f6dULL; 57 | std::uint64_t v2 = 0x6c7967656e657261ULL; 58 | std::uint64_t v3 = 0x7465646279746573ULL; 59 | std::uint64_t Result; 60 | std::uint64_t y; 61 | size_t x; 62 | const size_t NumLeft = Size & 7; 63 | const std::uint8_t *StrEnd = Str + Size - NumLeft; 64 | 65 | Result = ((std::uint64_t)Size) << 56; 66 | v3 ^= Key2; 67 | v2 ^= Key1; 68 | v1 ^= Key2; 69 | v0 ^= Key1; 70 | while (Str != StrEnd) 71 | { 72 | // Minor change to the official implementation. (Does endianness actually matter?) 73 | y = *((const std::uint64_t *)Str); 74 | 75 | v3 ^= y; 76 | 77 | for (x = 0; x < cRounds; ++x) SIPROUND; 78 | 79 | v0 ^= y; 80 | 81 | Str += 8; 82 | } 83 | 84 | switch (NumLeft) 85 | { 86 | case 7: Result |= ((std::uint64_t)Str[6]) << 48; FALL_THROUGH; 87 | case 6: Result |= ((std::uint64_t)Str[5]) << 40; FALL_THROUGH; 88 | case 5: Result |= ((std::uint64_t)Str[4]) << 32; FALL_THROUGH; 89 | case 4: Result |= ((std::uint64_t)Str[3]) << 24; FALL_THROUGH; 90 | case 3: Result |= ((std::uint64_t)Str[2]) << 16; FALL_THROUGH; 91 | case 2: Result |= ((std::uint64_t)Str[1]) << 8; FALL_THROUGH; 92 | case 1: Result |= ((std::uint64_t)Str[0]); break; 93 | case 0: break; 94 | } 95 | 96 | v3 ^= Result; 97 | 98 | for (x = 0; x < cRounds; ++x) SIPROUND; 99 | 100 | v0 ^= Result; 101 | v2 ^= 0xff; 102 | 103 | for (x = 0; x < dRounds; ++x) SIPROUND; 104 | 105 | Result = v0 ^ v1 ^ v2 ^ v3; 106 | 107 | return Result; 108 | } 109 | } 110 | -------------------------------------------------------------------------------- /templates/packed_ordered_hash.h: -------------------------------------------------------------------------------- 1 | // Ordered hash map with integer and string keys in a memory compact format. No detachable nodes but is CPU cache-friendly. 2 | // Primarily useful for array style hashes with lots of insertions (up to 2^31 values), frequent key- and index-based lookups, some iteration, and few deletions. 3 | // (C) 2017 CubicleSoft. All Rights Reserved. 4 | 5 | #ifndef CUBICLESOFT_PACKEDORDEREDHASH 6 | #define CUBICLESOFT_PACKEDORDEREDHASH 7 | 8 | #include 9 | #include 10 | #include 11 | #include 12 | 13 | namespace CubicleSoft 14 | { 15 | template 16 | class PackedOrderedHash; 17 | template 18 | class PackedOrderedHashNoCopy; 19 | 20 | template 21 | class PackedOrderedHashNode 22 | { 23 | friend class PackedOrderedHash; 24 | friend class PackedOrderedHashNoCopy; 25 | 26 | public: 27 | inline std::int64_t GetIntKey() { return IntKey; } 28 | inline char *GetStrKey() { return StrKey; } 29 | inline size_t GetStrLen() { return *(size_t *)(StrKey - sizeof(size_t)); } 30 | 31 | private: 32 | std::uint32_t PrevHashIndex; 33 | std::uint32_t NextHashIndex; 34 | 35 | std::int64_t IntKey; 36 | char *StrKey; 37 | 38 | public: 39 | T Value; 40 | }; 41 | 42 | class PackedOrderedHashUtil 43 | { 44 | public: 45 | static size_t GetDJBX33XHashKey(const std::uint8_t *Str, size_t Size, size_t InitVal); 46 | static std::uint64_t GetSipHashKey(const std::uint8_t *Str, size_t Size, std::uint64_t Key1, std::uint64_t Key2, size_t cRounds, size_t dRounds); 47 | }; 48 | 49 | // PackedOrderedHash. A packed ordered hash. 50 | #include "packed_ordered_hash_util.h" 51 | 52 | // PackedOrderedHashNoCopy. A packed ordered hash with a private copy constructor and assignment operator. 53 | #define CUBICLESOFT_PACKEDORDEREDHASH_NOCOPYASSIGN 54 | #include "packed_ordered_hash_util.h" 55 | #undef CUBICLESOFT_PACKEDORDEREDHASH_NOCOPYASSIGN 56 | } 57 | 58 | #endif -------------------------------------------------------------------------------- /templates/packed_ordered_hash_util.h: -------------------------------------------------------------------------------- 1 | // Ordered hash map with integer and string keys in a memory compact format. No detachable nodes but is CPU cache-friendly. 2 | // (C) 2017 CubicleSoft. All Rights Reserved. 3 | 4 | // NOTE: This file is intended to be included from 'packed_ordered_hash.h'. 5 | 6 | // Implements a packed ordered hash that grows dynamically and uses integer and string keys. 7 | template 8 | #ifdef CUBICLESOFT_PACKEDORDEREDHASH_NOCOPYASSIGN 9 | class PackedOrderedHashNoCopy 10 | #else 11 | class PackedOrderedHash 12 | #endif 13 | { 14 | public: 15 | // Implements djb2 (DJBX33X). 16 | // WARNING: This algorithm is weak security-wise! 17 | // https://www.youtube.com/watch?v=R2Cq3CLI6H8 18 | // https://www.youtube.com/watch?v=wGYj8fhhUVA 19 | // For much better security with a slight performance reduction, use the other constructor, which implements SipHash. 20 | #ifdef CUBICLESOFT_PACKEDORDEREDHASH_NOCOPYASSIGN 21 | PackedOrderedHashNoCopy 22 | #else 23 | PackedOrderedHash 24 | #endif 25 | (size_t EstimatedSize = 8, std::uint64_t HashKey = 5381) : UseSipHash(false), Key1(HashKey), Key2(0), 26 | ArrayNodes(NULL), HashNodes(NULL), Mask(0), NumNodes(0), NextNodePos(0), NumUsed(0), 27 | CompactNumUsed50(0), CompactNextNodePos75(0), CompactNumUsed90(0), ResizeNumUsed40(0) 28 | { 29 | ResizeHash(EstimatedSize); 30 | } 31 | 32 | // Keys are securely hashed via SipHash-2-4. 33 | // Assumes good (CSPRNG generated) inputs for HashKey1 and HashKey2. 34 | #ifdef CUBICLESOFT_PACKEDORDEREDHASH_NOCOPYASSIGN 35 | PackedOrderedHashNoCopy 36 | #else 37 | PackedOrderedHash 38 | #endif 39 | (size_t EstimatedSize, std::uint64_t HashKey1, std::uint64_t HashKey2) : UseSipHash(true), Key1(HashKey1), Key2(HashKey2), 40 | ArrayNodes(NULL), HashNodes(NULL), Mask(0), NumNodes(0), NextNodePos(0), NumUsed(0), 41 | CompactNumUsed50(0), CompactNextNodePos75(0), CompactNumUsed90(0), ResizeNumUsed40(0) 42 | { 43 | ResizeHash(EstimatedSize); 44 | } 45 | 46 | #ifdef CUBICLESOFT_PACKEDORDEREDHASH_NOCOPYASSIGN 47 | ~PackedOrderedHashNoCopy() 48 | #else 49 | ~PackedOrderedHash() 50 | #endif 51 | { 52 | PackedOrderedHashNode *Node = ArrayNodes, *LastNode = ArrayNodes + NextNodePos; 53 | while (Node != LastNode) 54 | { 55 | if (Node->PrevHashIndex != 0xFFFFFFFF) 56 | { 57 | Node->Value.~T(); 58 | 59 | if (Node->StrKey != NULL) delete[] (Node->StrKey - sizeof(size_t)); 60 | } 61 | 62 | Node++; 63 | } 64 | 65 | delete[] (char *)ArrayNodes; 66 | } 67 | 68 | #ifdef CUBICLESOFT_PACKEDORDEREDHASH_NOCOPYASSIGN 69 | PackedOrderedHashNoCopy(const PackedOrderedHashNoCopy &TempHash); 70 | PackedOrderedHashNoCopy &operator=(const PackedOrderedHashNoCopy &TempHash); 71 | #else 72 | PackedOrderedHash(const PackedOrderedHash &TempHash) 73 | { 74 | UseSipHash = TempHash.UseSipHash; 75 | Key1 = TempHash.Key1; 76 | Key2 = TempHash.Key2; 77 | 78 | ArrayNodes = NULL; 79 | NumNodes = 0; 80 | NumUsed = 0; 81 | 82 | InternalResizeHash(TempHash.NumNodes); 83 | NextNodePos = TempHash.NextNodePos; 84 | memcpy(HashNodes, TempHash.HashNodes, sizeof(std::uint32_t) * NumNodes); 85 | NumUsed = TempHash.NumUsed; 86 | 87 | PackedOrderedHashNode *Node = TempHash.ArrayNodes, *Node2 = ArrayNodes, *LastNode = ArrayNodes + NextNodePos; 88 | 89 | while (Node != LastNode) 90 | { 91 | Node2->PrevHashIndex = Node->PrevHashIndex; 92 | Node2->NextHashIndex = Node->NextHashIndex; 93 | Node2->IntKey = Node->IntKey; 94 | 95 | if (Node->StrKey == NULL) Node2->StrKey = NULL; 96 | else 97 | { 98 | size_t StrLen = Node->GetStrLen(); 99 | char *Str = new char[StrLen + sizeof(size_t)]; 100 | *((size_t *)Str) = StrLen; 101 | Str += sizeof(size_t); 102 | memcpy(Str, Node->StrKey, StrLen); 103 | Node2->StrKey = Str; 104 | } 105 | 106 | if (Node->PrevHashIndex != 0xFFFFFFFF) 107 | { 108 | new (&Node2->Value) T(Node->Value); 109 | } 110 | 111 | Node++; 112 | Node2++; 113 | } 114 | } 115 | 116 | PackedOrderedHash &operator=(const PackedOrderedHash &TempHash) 117 | { 118 | if (&TempHash != this) 119 | { 120 | UseSipHash = TempHash.UseSipHash; 121 | Key1 = TempHash.Key1; 122 | Key2 = TempHash.Key2; 123 | 124 | PackedOrderedHashNode *Node = ArrayNodes, *Node2, *LastNode = ArrayNodes + NextNodePos; 125 | while (Node != LastNode) 126 | { 127 | if (Node->PrevHashIndex != 0xFFFFFFFF) 128 | { 129 | Node->Value.~T(); 130 | 131 | if (Node->StrKey != NULL) delete[] (Node->StrKey - sizeof(size_t)); 132 | } 133 | 134 | Node++; 135 | } 136 | 137 | delete[] (char *)ArrayNodes; 138 | 139 | ArrayNodes = NULL; 140 | NumNodes = 0; 141 | NumUsed = 0; 142 | 143 | InternalResizeHash(TempHash.NumNodes); 144 | NextNodePos = TempHash.NextNodePos; 145 | memcpy(HashNodes, TempHash.HashNodes, sizeof(std::uint32_t) * NumNodes); 146 | NumUsed = TempHash.NumUsed; 147 | 148 | Node = TempHash.ArrayNodes; 149 | Node2 = ArrayNodes; 150 | LastNode = ArrayNodes + NextNodePos; 151 | 152 | while (Node != LastNode) 153 | { 154 | Node2->PrevHashIndex = Node->PrevHashIndex; 155 | Node2->NextHashIndex = Node->NextHashIndex; 156 | Node2->IntKey = Node->IntKey; 157 | 158 | if (Node->StrKey == NULL) Node2->StrKey = NULL; 159 | else 160 | { 161 | size_t StrLen = Node->GetStrLen(); 162 | char *Str = new char[StrLen + sizeof(size_t)]; 163 | *((size_t *)Str) = StrLen; 164 | Str += sizeof(size_t); 165 | memcpy(Str, Node->StrKey, StrLen); 166 | Node2->StrKey = Str; 167 | } 168 | 169 | if (Node->PrevHashIndex != 0xFFFFFFFF) 170 | { 171 | new (&Node2->Value) T(Node->Value); 172 | } 173 | 174 | Node++; 175 | Node2++; 176 | } 177 | } 178 | 179 | return *this; 180 | } 181 | #endif 182 | 183 | PackedOrderedHashNode *Set(const std::int64_t IntKey) 184 | { 185 | size_t Pos; 186 | std::uint64_t HashKey = GetHashKey((const std::uint8_t *)&IntKey, sizeof(std::int64_t)); 187 | PackedOrderedHashNode *Node = InternalFind(IntKey, Pos, HashKey); 188 | 189 | if (Node != NULL) return Node; 190 | 191 | // Create a new node. 192 | if (NextNodePos == NumNodes) AutoResizeHash(); 193 | 194 | Node = ArrayNodes + NextNodePos; 195 | new (&Node->Value) T; 196 | Pos = NextNodePos; 197 | NextNodePos++; 198 | 199 | // Attach the node to the start of the hash list. 200 | std::uint32_t HashPos = (std::uint32_t)HashKey & Mask; 201 | Node->PrevHashIndex = 0x80000000 | HashPos; 202 | Node->NextHashIndex = HashNodes[HashPos]; 203 | if (Node->NextHashIndex != 0xFFFFFFFF) ArrayNodes[Node->NextHashIndex].PrevHashIndex = (std::uint32_t)Pos; 204 | HashNodes[HashPos] = (std::uint32_t)Pos; 205 | 206 | Node->IntKey = IntKey; 207 | Node->StrKey = NULL; 208 | 209 | NumUsed++; 210 | 211 | return Node; 212 | } 213 | 214 | inline PackedOrderedHashNode *Set(const std::int64_t IntKey, const T &Value) 215 | { 216 | PackedOrderedHashNode *Node = Set(IntKey); 217 | Node->Value = Value; 218 | 219 | return Node; 220 | } 221 | 222 | PackedOrderedHashNode *Set(const char *StrKey, const size_t StrLen) 223 | { 224 | size_t Pos; 225 | std::uint64_t HashKey = GetHashKey((const std::uint8_t *)StrKey, StrLen); 226 | PackedOrderedHashNode *Node = InternalFind(StrKey, StrLen, Pos, HashKey); 227 | 228 | if (Node != NULL) return Node; 229 | 230 | // Create a new node. 231 | if (NextNodePos == NumNodes) AutoResizeHash(); 232 | 233 | Node = ArrayNodes + NextNodePos; 234 | Pos = NextNodePos; 235 | new (&Node->Value) T; 236 | NextNodePos++; 237 | 238 | // Attach the node to the start of the hash list. 239 | std::uint32_t HashPos = (std::uint32_t)HashKey & Mask; 240 | Node->PrevHashIndex = 0x80000000 | HashPos; 241 | Node->NextHashIndex = HashNodes[HashPos]; 242 | if (Node->NextHashIndex != 0xFFFFFFFF) ArrayNodes[Node->NextHashIndex].PrevHashIndex = (std::uint32_t)Pos; 243 | HashNodes[HashPos] = (std::uint32_t)Pos; 244 | 245 | Node->IntKey = (std::int64_t)HashKey; 246 | 247 | char *Str = new char[StrLen + sizeof(size_t)]; 248 | *((size_t *)Str) = StrLen; 249 | Str += sizeof(size_t); 250 | memcpy(Str, StrKey, StrLen); 251 | Node->StrKey = Str; 252 | 253 | NumUsed++; 254 | 255 | return Node; 256 | } 257 | 258 | inline PackedOrderedHashNode *Set(const char *StrKey, const size_t StrLen, const T &Value) 259 | { 260 | PackedOrderedHashNode *Node = Set(StrKey, StrLen); 261 | Node->Value = Value; 262 | 263 | return Node; 264 | } 265 | 266 | inline bool Unset(const std::int64_t IntKey) 267 | { 268 | return Unset(Find(IntKey)); 269 | } 270 | 271 | inline bool Unset(const char *StrKey, const size_t StrLen) 272 | { 273 | return Unset(Find(StrKey, StrLen)); 274 | } 275 | 276 | bool Unset(PackedOrderedHashNode *Node) 277 | { 278 | if (Node == NULL || Node->PrevHashIndex == 0xFFFFFFFF) return false; 279 | 280 | // Detach the node from the hash list. 281 | if (Node->NextHashIndex != 0xFFFFFFFF) ArrayNodes[Node->NextHashIndex].PrevHashIndex = Node->PrevHashIndex; 282 | 283 | if (Node->PrevHashIndex & 0x80000000) HashNodes[Node->PrevHashIndex & 0x7FFFFFFF] = Node->NextHashIndex; 284 | else ArrayNodes[Node->PrevHashIndex].NextHashIndex = Node->NextHashIndex; 285 | 286 | // Cleanup. 287 | Node->PrevHashIndex = 0xFFFFFFFF; 288 | Node->Value.~T(); 289 | if (Node->StrKey != NULL) 290 | { 291 | delete[] (Node->StrKey - sizeof(size_t)); 292 | Node->StrKey = NULL; 293 | } 294 | 295 | NumUsed--; 296 | 297 | return true; 298 | } 299 | 300 | // Returns the node in the array by index. 301 | inline PackedOrderedHashNode *Get(size_t Pos) 302 | { 303 | return (Pos >= NextNodePos || ArrayNodes[Pos].PrevHashIndex == 0xFFFFFFFF ? NULL : ArrayNodes + Pos); 304 | } 305 | 306 | // Gets the position of the node in the array. 307 | inline size_t GetPos(PackedOrderedHashNode *Node) { return (size_t)(Node - ArrayNodes); } 308 | 309 | // Finds the node in the array via the hash. 310 | inline PackedOrderedHashNode *Find(const std::int64_t IntKey) 311 | { 312 | size_t Pos; 313 | 314 | return Find(IntKey, Pos); 315 | } 316 | 317 | // Finds the node in the array via the hash. 318 | inline PackedOrderedHashNode *Find(const std::int64_t IntKey, size_t &Pos) 319 | { 320 | return InternalFind(IntKey, Pos, GetHashKey((const std::uint8_t *)&IntKey, sizeof(std::int64_t))); 321 | } 322 | 323 | // Finds the node in the array via the hash. 324 | inline PackedOrderedHashNode *Find(const char *StrKey, const size_t StrLen) 325 | { 326 | size_t Pos; 327 | 328 | return Find(StrKey, StrLen, Pos); 329 | } 330 | 331 | // Finds the node in the array via the hash. 332 | inline PackedOrderedHashNode *Find(const char *StrKey, const size_t StrLen, size_t &Pos) 333 | { 334 | return InternalFind(StrKey, StrLen, Pos, GetHashKey((const std::uint8_t *)StrKey, StrLen)); 335 | } 336 | 337 | // Iterates over the array, skipping unset nodes. Initialize the input Pos to GetNextPos() to start at the beginning. 338 | inline PackedOrderedHashNode *Next(size_t &Pos) 339 | { 340 | if (Pos >= NextNodePos) Pos = 0; 341 | else Pos++; 342 | 343 | for (; Pos < NextNodePos; Pos++) 344 | { 345 | if (ArrayNodes[Pos].PrevHashIndex != 0xFFFFFFFF) return (ArrayNodes + Pos); 346 | } 347 | 348 | return NULL; 349 | } 350 | 351 | // Iterates over the array, skipping unset nodes. Initialize the input Pos to GetNextPos() to start at the end. 352 | inline PackedOrderedHashNode *Prev(size_t &Pos) 353 | { 354 | if (Pos > NextNodePos) Pos = NextNodePos; 355 | 356 | while (Pos > 0) 357 | { 358 | Pos--; 359 | 360 | if (ArrayNodes[Pos].PrevHashIndex != 0xFFFFFFFF) return (ArrayNodes + Pos); 361 | } 362 | 363 | Pos = NextNodePos; 364 | 365 | return NULL; 366 | } 367 | 368 | // Call before iterating over the array multiple times. Then call Optimize() if this function returns true. 369 | inline bool ShouldOptimize() 370 | { 371 | return (NumUsed < CompactNumUsed50 && NextNodePos > CompactNextNodePos75); 372 | } 373 | 374 | // Compacts the array by moving elements to unset positions. 375 | bool Optimize() 376 | { 377 | if (NextNodePos == NumUsed) return true; 378 | 379 | // Find the first unset position. 380 | PackedOrderedHashNode *Node = ArrayNodes, *Node2, *LastNode = ArrayNodes + NextNodePos; 381 | while (Node != LastNode && Node->PrevHashIndex != 0xFFFFFFFF) Node++; 382 | 383 | // Copy nodes. 384 | std::uint32_t x = (std::uint32_t)(Node - ArrayNodes); 385 | for (Node2 = Node + 1; Node2 != LastNode; Node2++) 386 | { 387 | if (Node2->PrevHashIndex != 0xFFFFFFFF) 388 | { 389 | // Raw copy node. 390 | memcpy(Node, Node2, sizeof(PackedOrderedHashNode)); 391 | 392 | // Update node hash indexes. 393 | if (Node->PrevHashIndex & 0x80000000) HashNodes[Node->PrevHashIndex & 0x7FFFFFFF] = x; 394 | else ArrayNodes[Node->PrevHashIndex].NextHashIndex = x; 395 | 396 | if (Node->NextHashIndex != 0xFFFFFFFF) ArrayNodes[Node->NextHashIndex].PrevHashIndex = x; 397 | 398 | Node++; 399 | x++; 400 | } 401 | } 402 | 403 | // Cleanup extra nodes. 404 | for (; Node != LastNode; Node++) 405 | { 406 | Node->PrevHashIndex = 0xFFFFFFFF; 407 | Node->StrKey = NULL; 408 | } 409 | 410 | NextNodePos = NumUsed; 411 | 412 | return true; 413 | } 414 | 415 | // Compact when resizing the hash. 416 | 417 | // Performs automatic resizing based on several rules: 418 | // Compact instead of resizing the hash when NumUsed < 90%. 419 | // Shrink when NumUsed < 40% full. 420 | // Grows if NextNodePos is the same as NumNodes. 421 | bool AutoResizeHash() 422 | { 423 | if (NumUsed < CompactNumUsed90) return Optimize(); 424 | if (NumUsed < ResizeNumUsed40) return InternalResizeHash(NumNodes >> 1); 425 | if (NextNodePos == NumNodes) return InternalResizeHash(NumNodes << 1); 426 | 427 | return false; 428 | } 429 | 430 | bool ResizeHash(size_t NewHashSize) 431 | { 432 | size_t NewSize = 1; 433 | while (NewSize < NewHashSize) NewSize <<= 1; 434 | 435 | return InternalResizeHash(NewSize); 436 | } 437 | 438 | inline size_t GetHashSize() { return NumNodes; } 439 | inline size_t GetNextPos() { return NextNodePos; } 440 | inline size_t GetSize() { return NumUsed; } 441 | 442 | private: 443 | inline std::uint64_t GetHashKey(const std::uint8_t *Str, size_t Size) const 444 | { 445 | return (UseSipHash ? PackedOrderedHashUtil::GetSipHashKey(Str, Size, Key1, Key2, 2, 4) : (std::uint64_t)PackedOrderedHashUtil::GetDJBX33XHashKey(Str, Size, (size_t)Key1)); 446 | } 447 | 448 | PackedOrderedHashNode *InternalFind(const std::int64_t IntKey, size_t &Pos, std::uint64_t HashKey) 449 | { 450 | Pos = HashNodes[(std::uint32_t)HashKey & Mask]; 451 | while (Pos != 0xFFFFFFFF) 452 | { 453 | if (ArrayNodes[Pos].IntKey == IntKey && ArrayNodes[Pos].StrKey == NULL) return ArrayNodes + Pos; 454 | 455 | Pos = ArrayNodes[Pos].NextHashIndex; 456 | } 457 | 458 | return NULL; 459 | } 460 | 461 | PackedOrderedHashNode *InternalFind(const char *StrKey, const size_t StrLen, size_t &Pos, std::uint64_t HashKey) 462 | { 463 | std::int64_t IntKey = (std::int64_t)HashKey; 464 | Pos = HashNodes[(std::uint32_t)HashKey & Mask]; 465 | while (Pos != 0xFFFFFFFF) 466 | { 467 | if (ArrayNodes[Pos].IntKey == IntKey && ArrayNodes[Pos].StrKey != NULL && StrLen == ArrayNodes[Pos].GetStrLen() && !memcmp(StrKey, ArrayNodes[Pos].StrKey, StrLen)) return ArrayNodes + Pos; 468 | 469 | Pos = ArrayNodes[Pos].NextHashIndex; 470 | } 471 | 472 | return NULL; 473 | } 474 | 475 | bool InternalResizeHash(size_t NewHashSize) 476 | { 477 | while (NewHashSize < NumUsed) NewHashSize <<= 1; 478 | if (NewHashSize == NumNodes || (NewHashSize < 512 && NewHashSize < NumNodes)) return false; 479 | 480 | PackedOrderedHashNode *ArrayNodes2 = (PackedOrderedHashNode *)(new char[sizeof(PackedOrderedHashNode) * NewHashSize + sizeof(std::uint32_t) * NewHashSize]); 481 | std::uint32_t *HashNodes2 = (std::uint32_t *)(ArrayNodes2 + NewHashSize); 482 | 483 | memset(HashNodes2, 0xFF, sizeof(std::uint32_t) * NewHashSize); 484 | 485 | if (ArrayNodes != NULL) 486 | { 487 | std::uint64_t HashKey; 488 | std::uint32_t HashPos, Mask2, x = 0; 489 | PackedOrderedHashNode *Node = ArrayNodes, *Node2 = ArrayNodes2, *LastNode = ArrayNodes + NextNodePos; 490 | Mask2 = (std::uint32_t)(NewHashSize - 1); 491 | while (Node != LastNode) 492 | { 493 | if (Node->PrevHashIndex != 0xFFFFFFFF) 494 | { 495 | // Raw copy node. 496 | memcpy(Node2, Node, sizeof(PackedOrderedHashNode)); 497 | 498 | HashKey = (Node2->StrKey != NULL ? (std::uint64_t)Node2->IntKey : GetHashKey((const std::uint8_t *)&Node2->IntKey, sizeof(std::int64_t))); 499 | 500 | // Attach the node to the start of the hash list. 501 | HashPos = (std::uint32_t)HashKey & Mask2; 502 | Node2->PrevHashIndex = 0x80000000 | HashPos; 503 | Node2->NextHashIndex = HashNodes2[HashPos]; 504 | if (Node2->NextHashIndex != 0xFFFFFFFF) ArrayNodes2[Node2->NextHashIndex].PrevHashIndex = x; 505 | HashNodes2[HashPos] = x; 506 | 507 | Node2++; 508 | x++; 509 | } 510 | 511 | Node++; 512 | } 513 | } 514 | 515 | ArrayNodes = ArrayNodes2; 516 | HashNodes = HashNodes2; 517 | NumNodes = NewHashSize; 518 | Mask = (std::uint32_t)(NumNodes - 1); 519 | NextNodePos = NumUsed; 520 | 521 | CompactNumUsed50 = (NumNodes >> 1); 522 | CompactNextNodePos75 = NumNodes - (NumNodes >> 3); 523 | CompactNumUsed90 = NumNodes - (NumNodes / 10); 524 | ResizeNumUsed40 = (size_t)((std::uint64_t)(NumNodes << 3) / 10); 525 | 526 | return true; 527 | } 528 | 529 | bool UseSipHash; 530 | std::uint64_t Key1, Key2; 531 | 532 | PackedOrderedHashNode *ArrayNodes; 533 | std::uint32_t *HashNodes; 534 | 535 | std::uint32_t Mask; 536 | size_t NumNodes, NextNodePos, NumUsed; 537 | size_t CompactNumUsed50, CompactNextNodePos75; 538 | size_t CompactNumUsed90, ResizeNumUsed40; 539 | }; 540 | -------------------------------------------------------------------------------- /templates/shared_lib.h: -------------------------------------------------------------------------------- 1 | // Shared library (DLL, so, etc.) loader and function call templates. 2 | // (C) 2021 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_SHARED_LIBRARY 5 | #define CUBICLESOFT_SHARED_LIBRARY 6 | 7 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 8 | #include 9 | #else 10 | #include 11 | #endif 12 | 13 | namespace CubicleSoft 14 | { 15 | namespace SharedLib 16 | { 17 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 18 | 19 | class ModuleUtil 20 | { 21 | public: 22 | inline ModuleUtil(const char *Filename) : MxFilename(Filename), MxModulePtr(NULL) 23 | { 24 | } 25 | 26 | inline ~ModuleUtil() 27 | { 28 | if (MxModulePtr != NULL) ::FreeLibrary(MxModulePtr); 29 | } 30 | 31 | inline HMODULE Load() 32 | { 33 | if (MxModulePtr != NULL) return MxModulePtr; 34 | if (MxFilename == NULL) return NULL; 35 | 36 | MxModulePtr = ::LoadLibraryA(MxFilename); 37 | 38 | MxFilename = NULL; 39 | 40 | return MxModulePtr; 41 | } 42 | 43 | private: 44 | const char *MxFilename; 45 | HMODULE MxModulePtr; 46 | }; 47 | 48 | class FunctionUtil 49 | { 50 | public: 51 | inline FunctionUtil(ModuleUtil &ModuleInst, const char *FuncName) : MxModule(&ModuleInst), MxFuncName(FuncName), MxFuncPtr(NULL) 52 | { 53 | } 54 | 55 | inline FARPROC GetFuncPtr() { return MxFuncPtr; } 56 | 57 | inline FARPROC LoadFuncPtr() 58 | { 59 | if (MxFuncName == NULL) return NULL; 60 | 61 | const char *FuncName = MxFuncName; 62 | MxFuncName = NULL; 63 | 64 | HMODULE ModulePtr = MxModule->Load(); 65 | 66 | return (ModulePtr == NULL ? NULL : (MxFuncPtr = ::GetProcAddress(ModulePtr, FuncName))); 67 | } 68 | 69 | private: 70 | ModuleUtil *MxModule; 71 | const char *MxFuncName; 72 | FARPROC MxFuncPtr; 73 | }; 74 | 75 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX Call 76 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_VOID CallVoid 77 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE CallOnce 78 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE_VOID CallOnceVoid 79 | #define CUBICLESOFT_SHARED_LIBRARY_CALLCONV 80 | #include "shared_lib_util.h" 81 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX 82 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_VOID 83 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE 84 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE_VOID 85 | #undef CUBICLESOFT_SHARED_LIBRARY_CALLCONV 86 | 87 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX Stdcall 88 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_VOID StdcallVoid 89 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE StdcallOnce 90 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE_VOID StdcallOnceVoid 91 | #define CUBICLESOFT_SHARED_LIBRARY_CALLCONV __stdcall 92 | #include "shared_lib_util.h" 93 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX 94 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_VOID 95 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE 96 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE_VOID 97 | #undef CUBICLESOFT_SHARED_LIBRARY_CALLCONV 98 | 99 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX Cdecl 100 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_VOID CdeclVoid 101 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE CdeclOnce 102 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE_VOID CdeclOnceVoid 103 | #define CUBICLESOFT_SHARED_LIBRARY_CALLCONV __cdecl 104 | #include "shared_lib_util.h" 105 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX 106 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_VOID 107 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE 108 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE_VOID 109 | #undef CUBICLESOFT_SHARED_LIBRARY_CALLCONV 110 | 111 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX Fastcall 112 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_VOID FastcallVoid 113 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE FastcallOnce 114 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE_VOID FastcallOnceVoid 115 | #define CUBICLESOFT_SHARED_LIBRARY_CALLCONV __fastcall 116 | #include "shared_lib_util.h" 117 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX 118 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_VOID 119 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE 120 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE_VOID 121 | #undef CUBICLESOFT_SHARED_LIBRARY_CALLCONV 122 | 123 | #else 124 | 125 | class ModuleUtil 126 | { 127 | public: 128 | inline ModuleUtil(const char *Filename) : MxFilename(Filename), MxModulePtr(NULL) 129 | { 130 | } 131 | 132 | inline ~ModuleUtil() 133 | { 134 | if (MxModulePtr != NULL) dlclose(MxModulePtr); 135 | } 136 | 137 | inline void *Load() 138 | { 139 | if (MxModulePtr != NULL) return MxModulePtr; 140 | if (MxFilename == NULL) return NULL; 141 | 142 | MxModulePtr = dlopen(MxFilename, RTLD_LAZY); 143 | 144 | MxFilename = NULL; 145 | 146 | return MxModulePtr; 147 | } 148 | 149 | private: 150 | const char *MxFilename; 151 | void *MxModulePtr; 152 | }; 153 | 154 | class FunctionUtil 155 | { 156 | public: 157 | inline FunctionUtil(ModuleUtil &ModuleInst, const char *FuncName) : MxModule(&ModuleInst), MxFuncName(FuncName), MxFuncPtr(NULL) 158 | { 159 | } 160 | 161 | inline void *GetFuncPtr() { return MxFuncPtr; } 162 | 163 | inline void *LoadFuncPtr() 164 | { 165 | if (MxFuncName == NULL) return NULL; 166 | 167 | const char *FuncName = MxFuncName; 168 | MxFuncName = NULL; 169 | 170 | void *ModulePtr = MxModule->Load(); 171 | 172 | return (ModulePtr == NULL ? NULL : (MxFuncPtr = dlsym(ModulePtr, FuncName))); 173 | } 174 | 175 | private: 176 | ModuleUtil *MxModule; 177 | const char *MxFuncName; 178 | void *MxFuncPtr; 179 | }; 180 | 181 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX Call 182 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_VOID CallVoid 183 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE CallOnce 184 | #define CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE_VOID CallOnceVoid 185 | #define CUBICLESOFT_SHARED_LIBRARY_CALLCONV 186 | #include "shared_lib_util.h" 187 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX 188 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_VOID 189 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE 190 | #undef CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE_VOID 191 | #undef CUBICLESOFT_SHARED_LIBRARY_CALLCONV 192 | 193 | #endif 194 | } 195 | } 196 | 197 | #endif 198 | -------------------------------------------------------------------------------- /templates/shared_lib_util.h: -------------------------------------------------------------------------------- 1 | // Shared library (DLL, so, etc.) loader and function call templates. 2 | // (C) 2021 CubicleSoft. All Rights Reserved. 3 | 4 | // NOTE: This file is intended to be included from 'shared_lib.h'. 5 | 6 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 7 | 8 | template 9 | bool CUBICLESOFT_SHARED_LIBRARY_PREFIX(FunctionUtil &Func, RetType &ReturnVal, Args... FuncArgs) 10 | { 11 | FARPROC FuncAddressPtr; 12 | 13 | if ((FuncAddressPtr = Func.GetFuncPtr()) == NULL && (FuncAddressPtr = Func.LoadFuncPtr()) == NULL) return false; 14 | 15 | ReturnVal = (*((RetType (CUBICLESOFT_SHARED_LIBRARY_CALLCONV *)(Args...))FuncAddressPtr))(FuncArgs...); 16 | 17 | return true; 18 | } 19 | 20 | template 21 | bool CUBICLESOFT_SHARED_LIBRARY_PREFIX_VOID(FunctionUtil &Func, Args... FuncArgs) 22 | { 23 | FARPROC FuncAddressPtr; 24 | 25 | if ((FuncAddressPtr = Func.GetFuncPtr()) == NULL || (FuncAddressPtr = Func.LoadFuncPtr()) == NULL) return false; 26 | 27 | (*((void (CUBICLESOFT_SHARED_LIBRARY_CALLCONV *)(Args...))FuncAddressPtr))(FuncArgs...); 28 | 29 | return true; 30 | } 31 | 32 | // Doesn't call FreeLibrary() to avoid issues with multiple uses of the same DLL. 33 | template 34 | bool CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE(const char *Filename, const char *FuncName, RetType &ReturnVal, Args... FuncArgs) 35 | { 36 | HMODULE ModulePtr; 37 | FARPROC FuncAddressPtr; 38 | 39 | ModulePtr = ::LoadLibraryA(Filename); 40 | if (ModulePtr == NULL) return false; 41 | 42 | FuncAddressPtr = ::GetProcAddress(ModulePtr, FuncName); 43 | if (FuncAddressPtr == NULL) return false; 44 | 45 | ReturnVal = (*((RetType (CUBICLESOFT_SHARED_LIBRARY_CALLCONV *)(Args...))FuncAddressPtr))(FuncArgs...); 46 | 47 | return true; 48 | } 49 | 50 | // Doesn't call FreeLibrary() to avoid issues with multiple uses of the same DLL. 51 | template 52 | bool CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE_VOID(const char *Filename, const char *FuncName, Args... FuncArgs) 53 | { 54 | HMODULE ModulePtr; 55 | FARPROC FuncAddressPtr; 56 | 57 | ModulePtr = ::LoadLibraryA(Filename); 58 | if (ModulePtr == NULL) return false; 59 | 60 | FuncAddressPtr = ::GetProcAddress(ModulePtr, FuncName); 61 | if (FuncAddressPtr == NULL) return false; 62 | 63 | (*((void (CUBICLESOFT_SHARED_LIBRARY_CALLCONV *)(Args...))FuncAddressPtr))(FuncArgs...); 64 | 65 | return true; 66 | } 67 | 68 | #else 69 | 70 | template 71 | bool CUBICLESOFT_SHARED_LIBRARY_PREFIX(FunctionUtil &Func, RetType &ReturnVal, Args... FuncArgs) 72 | { 73 | void *FuncAddressPtr; 74 | 75 | if ((FuncAddressPtr = Func.GetFuncPtr()) == NULL && (FuncAddressPtr = Func.LoadFuncPtr()) == NULL) return false; 76 | 77 | ReturnVal = (*((RetType (CUBICLESOFT_SHARED_LIBRARY_CALLCONV *)(Args...))FuncAddressPtr))(FuncArgs...); 78 | 79 | return true; 80 | } 81 | 82 | template 83 | bool CUBICLESOFT_SHARED_LIBRARY_PREFIX_VOID(FunctionUtil &Func, Args... FuncArgs) 84 | { 85 | void *FuncAddressPtr; 86 | 87 | if ((FuncAddressPtr = Func.GetFuncPtr()) == NULL || (FuncAddressPtr = Func.LoadFuncPtr()) == NULL) return false; 88 | 89 | (*((void (CUBICLESOFT_SHARED_LIBRARY_CALLCONV *)(Args...))FuncAddressPtr))(FuncArgs...); 90 | 91 | return true; 92 | } 93 | 94 | // Doesn't call dlclose() to avoid issues with multiple uses of the same library. 95 | template 96 | bool CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE(const char *Filename, const char *FuncName, RetType &ReturnVal, Args... FuncArgs) 97 | { 98 | void *ModulePtr; 99 | void *FuncAddressPtr; 100 | 101 | ModulePtr = dlopen(Filename, RTLD_LAZY); 102 | if (ModulePtr == NULL) return false; 103 | 104 | FuncAddressPtr = dlsym(ModulePtr, FuncName); 105 | if (FuncAddressPtr == NULL) return false; 106 | 107 | ReturnVal = (*((RetType (CUBICLESOFT_SHARED_LIBRARY_CALLCONV *)(Args...))FuncAddressPtr))(FuncArgs...); 108 | 109 | return true; 110 | } 111 | 112 | // Doesn't call dlclose() to avoid issues with multiple uses of the same library. 113 | template 114 | bool CUBICLESOFT_SHARED_LIBRARY_PREFIX_ONCE_VOID(const char *Filename, const char *FuncName, Args... FuncArgs) 115 | { 116 | void *ModulePtr; 117 | void *FuncAddressPtr; 118 | 119 | ModulePtr = dlopen(Filename, RTLD_LAZY); 120 | if (ModulePtr == NULL) return false; 121 | 122 | FuncAddressPtr = dlsym(ModulePtr, FuncName); 123 | if (FuncAddressPtr == NULL) return false; 124 | 125 | (*((void (CUBICLESOFT_SHARED_LIBRARY_CALLCONV *)(Args...))FuncAddressPtr))(FuncArgs...); 126 | 127 | return true; 128 | } 129 | 130 | #endif 131 | -------------------------------------------------------------------------------- /templates/static_2d_array.h: -------------------------------------------------------------------------------- 1 | // Statically allocated 2D array. No resize. 2 | // (C) 2015 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_STATIC_2D_ARRAY 5 | #define CUBICLESOFT_STATIC_2D_ARRAY 6 | 7 | namespace CubicleSoft 8 | { 9 | template 10 | class Static2DArray 11 | { 12 | public: 13 | Static2DArray(size_t FixedWidth, size_t FixedHeight) 14 | { 15 | baseptr = new T[FixedWidth * FixedHeight]; 16 | ptr = new T *[FixedHeight]; 17 | for (size_t x = 0; x < FixedHeight; x++) ptr[x] = baseptr + (FixedWidth * x); 18 | Width = FixedWidth; 19 | Height = FixedHeight; 20 | } 21 | 22 | ~Static2DArray() 23 | { 24 | if (ptr != NULL) delete[] ptr; 25 | if (baseptr != NULL) delete[] baseptr; 26 | } 27 | 28 | // Copy constructor. 29 | Static2DArray(const Static2DArray &TempArray) 30 | { 31 | Width = TempArray.Width; 32 | Height = TempArray.Height; 33 | 34 | size_t y = Width * Height; 35 | baseptr = new T[y]; 36 | ptr = new T *[Height]; 37 | for (size_t x = 0; x < Height; x++) ptr[x] = baseptr + (Width * x); 38 | 39 | for (size_t x = 0; x < y; x++) baseptr[x] = TempArray.baseptr[x]; 40 | } 41 | 42 | // Assignment operator. 43 | Static2DArray &operator=(const Static2DArray &TempArray) 44 | { 45 | if (this != &TempArray) 46 | { 47 | if (Width != TempArray.Width || Height != TempArray.Height) 48 | { 49 | if (ptr != NULL) delete[] ptr; 50 | if (baseptr != NULL) delete[] baseptr; 51 | 52 | Width = TempArray.Width; 53 | Height = TempArray.Height; 54 | 55 | size_t y = Width * Height; 56 | if (y) 57 | { 58 | baseptr = NULL; 59 | ptr = NULL; 60 | } 61 | else 62 | { 63 | baseptr = new T[y]; 64 | ptr = new T *[Height]; 65 | for (size_t x = 0; x < Height; x++) ptr[x] = baseptr + (Width * x); 66 | } 67 | } 68 | 69 | size_t y = Width * Height; 70 | for (size_t x = 0; x < y; x++) baseptr[x] = TempArray.baseptr[x]; 71 | } 72 | 73 | return *this; 74 | } 75 | 76 | inline size_t GetWidth() const { return Width; } 77 | inline size_t GetHeight() const { return Height; } 78 | 79 | // Lets the caller access a specific data element. 80 | // If the position is out of bounds, then epic fail. 81 | inline T *operator[](size_t Pos) { return ptr[Pos]; } 82 | 83 | // Grants the caller access to the underlying raw data. 84 | inline T **RawData() const { return ptr; } 85 | 86 | // Raw comparison of another Static2DArray with this Static2DArray. 87 | bool operator==(const Static2DArray &TempArray) const 88 | { 89 | if (Width != TempArray.Width || Height != TempArray.Height) return false; 90 | 91 | size_t x, y = Width * Height; 92 | for (x = 0; x < y && baseptr[x] == TempArray.ptr[x]; x++); 93 | if (x < y) return false; 94 | 95 | return true; 96 | } 97 | 98 | // Raw comparison of another Static2DArray with this Static2DArray. 99 | inline bool operator!=(const Static2DArray &TempArray) const 100 | { 101 | return !((*this) == TempArray); 102 | } 103 | 104 | private: 105 | T *baseptr; 106 | T **ptr; 107 | size_t Width, Height; 108 | }; 109 | } 110 | 111 | #endif 112 | -------------------------------------------------------------------------------- /templates/static_mixed_var.h: -------------------------------------------------------------------------------- 1 | // A mixed, flexible variable data type (plain ol' data - POD) with all-public data access. 2 | // (C) 2021 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_STATIC_MIXED_VAR 5 | #define CUBICLESOFT_STATIC_MIXED_VAR 6 | 7 | #include 8 | #include 9 | #include 10 | #include 11 | 12 | namespace CubicleSoft 13 | { 14 | enum StaticMixedVarModes 15 | { 16 | MV_None, 17 | MV_Bool, 18 | MV_Int, 19 | MV_UInt, 20 | MV_Double, 21 | MV_Str 22 | }; 23 | 24 | // Must be used like: StaticMixedVar 25 | // Designed to be extended but not overridden. 26 | template 27 | class StaticMixedVar 28 | { 29 | public: 30 | StaticMixedVarModes MxMode; 31 | std::int64_t MxInt; 32 | double MxDouble; 33 | T MxStr; 34 | size_t MxStrPos; 35 | 36 | StaticMixedVar() : MxMode(MV_None), MxInt(0), MxDouble(0.0), MxStrPos(0) 37 | { 38 | } 39 | 40 | // Some functions for those who prefer member functions over directly accessing raw class data. 41 | inline bool IsNone() 42 | { 43 | return (MxMode == MV_None); 44 | } 45 | 46 | inline bool IsBool() 47 | { 48 | return (MxMode == MV_Bool); 49 | } 50 | 51 | inline bool IsInt() 52 | { 53 | return (MxMode == MV_Int); 54 | } 55 | 56 | inline bool IsUInt() 57 | { 58 | return (MxMode == MV_UInt); 59 | } 60 | 61 | inline bool IsDouble() 62 | { 63 | return (MxMode == MV_Double); 64 | } 65 | 66 | inline bool IsStr() 67 | { 68 | return (MxMode == MV_Str); 69 | } 70 | 71 | inline bool GetBool() 72 | { 73 | return (MxInt != 0); 74 | } 75 | 76 | inline std::int64_t GetInt() 77 | { 78 | return MxInt; 79 | } 80 | 81 | inline std::uint64_t GetUInt() 82 | { 83 | return (std::uint64_t)MxInt; 84 | } 85 | 86 | inline double GetDouble() 87 | { 88 | return MxDouble; 89 | } 90 | 91 | inline char *GetStr() 92 | { 93 | return MxStr; 94 | } 95 | 96 | inline size_t GetSize() 97 | { 98 | return MxStrPos; 99 | } 100 | 101 | inline size_t GetMaxSize() 102 | { 103 | return sizeof(MxStr); 104 | } 105 | 106 | inline void SetBool(const bool newbool) 107 | { 108 | MxMode = MV_Bool; 109 | MxInt = (int)newbool; 110 | } 111 | 112 | inline void SetInt(const std::int64_t newint) 113 | { 114 | MxMode = MV_Int; 115 | MxInt = newint; 116 | } 117 | 118 | inline void SetUInt(const std::uint64_t newint) 119 | { 120 | MxMode = MV_UInt; 121 | MxInt = (std::int64_t)newint; 122 | } 123 | 124 | inline void SetDouble(const double newdouble) 125 | { 126 | MxMode = MV_Double; 127 | MxDouble = newdouble; 128 | } 129 | 130 | void SetStr(const char *str) 131 | { 132 | MxMode = MV_Str; 133 | MxStrPos = 0; 134 | while (MxStrPos < sizeof(MxStr) - 1 && *str) 135 | { 136 | MxStr[MxStrPos++] = *str++; 137 | } 138 | MxStr[MxStrPos] = '\0'; 139 | } 140 | 141 | void SetData(const char *str, size_t size) 142 | { 143 | MxMode = MV_Str; 144 | MxStrPos = 0; 145 | while (MxStrPos < sizeof(MxStr) - 1 && size) 146 | { 147 | MxStr[MxStrPos++] = *str++; 148 | size--; 149 | } 150 | MxStr[MxStrPos] = '\0'; 151 | } 152 | 153 | // Prepend functions only prepend if there is enough space. 154 | void PrependStr(const char *str) 155 | { 156 | size_t y = strlen(str); 157 | if (MxStrPos + y < sizeof(MxStr) - 1) 158 | { 159 | memmove(MxStr + y, MxStr, MxStrPos + 1); 160 | memcpy(MxStr, str, y); 161 | MxStrPos += y; 162 | } 163 | } 164 | 165 | void PrependData(const char *str, size_t size) 166 | { 167 | if (MxStrPos + size < sizeof(MxStr) - 1) 168 | { 169 | memmove(MxStr + size, MxStr, MxStrPos + 1); 170 | memcpy(MxStr, str, size); 171 | MxStrPos += size; 172 | } 173 | } 174 | 175 | void PrependInt(const std::int64_t val, size_t radix = 10) 176 | { 177 | char tempbuffer[44]; 178 | if (IntToString(tempbuffer, sizeof(tempbuffer), val, radix)) PrependStr(tempbuffer); 179 | } 180 | 181 | void PrependUInt(const std::uint64_t val, size_t radix = 10) 182 | { 183 | char tempbuffer[44]; 184 | if (IntToString(tempbuffer, sizeof(tempbuffer), val, radix)) PrependStr(tempbuffer); 185 | } 186 | 187 | void PrependDouble(const double val, const size_t precision = 16) 188 | { 189 | char tempbuffer[100]; 190 | #if (defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64)) && defined(_MSC_VER) && _MSC_VER < 1900 191 | _snprintf_s(tempbuffer, sizeof(tempbuffer), _TRUNCATE, "%1.*g", precision, val); 192 | tempbuffer[sizeof(tempbuffer) - 1] = '\0'; 193 | #else 194 | snprintf(tempbuffer, sizeof(tempbuffer), "%1.*g", (int)precision, val); 195 | #endif 196 | 197 | PrependStr(tempbuffer); 198 | } 199 | 200 | void AppendStr(const char *str) 201 | { 202 | while (MxStrPos < sizeof(MxStr) - 1 && *str) 203 | { 204 | MxStr[MxStrPos++] = *str++; 205 | } 206 | MxStr[MxStrPos] = '\0'; 207 | } 208 | 209 | void AppendData(const char *str, size_t size) 210 | { 211 | while (MxStrPos < sizeof(MxStr) - 1 && size) 212 | { 213 | MxStr[MxStrPos++] = *str++; 214 | size--; 215 | } 216 | MxStr[MxStrPos] = '\0'; 217 | } 218 | 219 | void AppendInt(const std::int64_t val, size_t radix = 10) 220 | { 221 | char tempbuffer[44]; 222 | if (IntToString(tempbuffer, sizeof(tempbuffer), val, radix)) AppendStr(tempbuffer); 223 | } 224 | 225 | void AppendUInt(const std::uint64_t val, size_t radix = 10) 226 | { 227 | char tempbuffer[44]; 228 | if (IntToString(tempbuffer, sizeof(tempbuffer), val, radix)) AppendStr(tempbuffer); 229 | } 230 | 231 | void AppendDouble(const double val, const size_t precision = 16) 232 | { 233 | char tempbuffer[100]; 234 | #if (defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64)) && defined(_MSC_VER) && _MSC_VER < 1900 235 | _snprintf_s(tempbuffer, sizeof(tempbuffer), _TRUNCATE, "%1.*g", precision, val); 236 | tempbuffer[sizeof(tempbuffer) - 1] = '\0'; 237 | #else 238 | snprintf(tempbuffer, sizeof(tempbuffer), "%1.*g", (int)precision, val); 239 | #endif 240 | 241 | AppendStr(tempbuffer); 242 | } 243 | 244 | inline void AppendChar(const char chr) 245 | { 246 | if (MxStrPos < sizeof(MxStr) - 1) 247 | { 248 | MxStr[MxStrPos++] = chr; 249 | MxStr[MxStrPos] = '\0'; 250 | } 251 | } 252 | 253 | inline void AppendMissingChar(const char chr) 254 | { 255 | if ((!MxStrPos || MxStr[MxStrPos - 1] != chr) && MxStrPos < sizeof(MxStr) - 1) 256 | { 257 | MxStr[MxStrPos++] = chr; 258 | MxStr[MxStrPos] = '\0'; 259 | } 260 | } 261 | 262 | inline bool RemoveTrailingChar(const char chr) 263 | { 264 | if (!MxStrPos || MxStr[MxStrPos - 1] != chr) return false; 265 | 266 | MxStr[--MxStrPos] = '\0'; 267 | 268 | return true; 269 | } 270 | 271 | inline void SetSize(const size_t size) 272 | { 273 | if (size < sizeof(MxStr)) 274 | { 275 | MxStrPos = size; 276 | MxStr[MxStrPos] = '\0'; 277 | } 278 | } 279 | 280 | // Swiped and slightly modified from Int::ToString(). 281 | static bool IntToString(char *Result, size_t Size, std::uint64_t Num, size_t Radix = 10) 282 | { 283 | if (Size < 2) return false; 284 | 285 | size_t x = Size, z; 286 | 287 | Result[--x] = '\0'; 288 | if (!Num) Result[--x] = '0'; 289 | else 290 | { 291 | while (Num && x) 292 | { 293 | z = Num % Radix; 294 | Result[--x] = (char)(z > 9 ? z - 10 + 'A' : z + '0'); 295 | Num /= Radix; 296 | } 297 | 298 | if (Num) return false; 299 | } 300 | 301 | memmove(Result, Result + x, Size - x); 302 | 303 | return true; 304 | } 305 | 306 | static bool IntToString(char *Result, size_t Size, std::int64_t Num, size_t Radix = 10) 307 | { 308 | if (Num >= 0) return IntToString(Result, Size, (std::uint64_t)Num, Radix); 309 | 310 | if (Size < 2) return false; 311 | Result[0] = '-'; 312 | 313 | return IntToString(Result + 1, Size - 1, (std::uint64_t)-Num, Radix); 314 | } 315 | }; 316 | } 317 | 318 | #endif 319 | -------------------------------------------------------------------------------- /templates/static_vector.h: -------------------------------------------------------------------------------- 1 | // Statically allocated vector. No resize. 2 | // (C) 2013 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_STATIC_VECTOR 5 | #define CUBICLESOFT_STATIC_VECTOR 6 | 7 | namespace CubicleSoft 8 | { 9 | template 10 | class StaticVector 11 | { 12 | public: 13 | StaticVector(size_t Num) 14 | { 15 | ptr = new T[Num]; 16 | Size = Num; 17 | } 18 | 19 | ~StaticVector() 20 | { 21 | if (ptr != NULL) delete[] ptr; 22 | } 23 | 24 | // Copy constructor. 25 | StaticVector(const StaticVector &TempVector) 26 | { 27 | Size = TempVector.Size; 28 | ptr = new T[Size]; 29 | for (size_t x = 0; x < Size; x++) ptr[x] = TempVector.ptr[x]; 30 | } 31 | 32 | // Assignment operator. 33 | StaticVector &operator=(const StaticVector &TempVector) 34 | { 35 | if (this != &TempVector) 36 | { 37 | if (Size != TempVector.Size) 38 | { 39 | if (ptr != NULL) delete [] ptr; 40 | 41 | Size = TempVector.Size; 42 | if (!Size) ptr = NULL; 43 | else ptr = new T[Size]; 44 | } 45 | 46 | for (size_t x = 0; x < Size; x++) ptr[x] = TempVector.ptr[x]; 47 | } 48 | 49 | return *this; 50 | } 51 | 52 | inline size_t GetSize() const { return Size; } 53 | 54 | // Lets the caller access a specific data element. 55 | // If the position is out of bounds, then epic fail. 56 | inline T &operator[](size_t Pos) { return ptr[Pos]; } 57 | 58 | // Grants the caller access to the underlying raw data. 59 | inline T *RawData() const { return ptr; } 60 | 61 | // Raw comparison of another StaticVector with this StaticVector. 62 | bool operator==(const StaticVector &TempVector) const 63 | { 64 | if (Size != TempVector.Size) return false; 65 | 66 | size_t x; 67 | for (x = 0; x < Size && ptr[x] == TempVector.ptr[x]; x++); 68 | if (x < Size) return false; 69 | 70 | return true; 71 | } 72 | 73 | // Raw comparison of another StaticVector with this StaticVector. 74 | inline bool operator!=(const StaticVector &TempVector) const 75 | { 76 | return !((*this) == TempVector); 77 | } 78 | 79 | private: 80 | T *ptr; 81 | size_t Size; 82 | }; 83 | } 84 | 85 | #endif 86 | -------------------------------------------------------------------------------- /test.h: -------------------------------------------------------------------------------- 1 | // C++ test suite macros. 2 | // (C) 2013 CubicleSoft. All Rights Reserved. 3 | 4 | const char *Global__StartTestStr = "%s (%lu) test started.\n"; 5 | const char *Global__PassedTestStr = "%s (%lu) test %u passed.\n"; 6 | const char *Global__FailedTestStr = "%s (%lu) test %u failed.\n"; 7 | const char *Global__SummaryTestsStr = "%s summary:\n %u tests performed.\n %u tests failed.\n\n"; 8 | const char *Global__SummaryTestsSuccessStr = "%s summary:\n %u tests performed.\n All tests passed.\n\n"; 9 | const char *Global__CustomTestStartStr = "%s (%lu) custom test started.\n"; 10 | const char *Global__CustomTestEndStr = "%s (%lu) custom test ended.\n"; 11 | 12 | #define TEST_FAILED 0 13 | #define TEST_SUCCESS 1 14 | #define TEST_START(arg) const char *Local__FuncName = #arg; \ 15 | int Local__TestCounter = 0; \ 16 | int Local__TestsFailed = 0; \ 17 | fprintf(Testfp, Global__StartTestStr, Local__FuncName, __LINE__); \ 18 | fflush(Testfp) 19 | #define TEST_COMPARE(arg, arg2) do { \ 20 | Local__TestCounter++; \ 21 | if ((arg) == (arg2)) fprintf(Testfp, Global__PassedTestStr, Local__FuncName, __LINE__, Local__TestCounter); \ 22 | else \ 23 | { \ 24 | fprintf(Testfp, Global__FailedTestStr, Local__FuncName, __LINE__, Local__TestCounter); \ 25 | Local__TestsFailed++; \ 26 | } \ 27 | fflush(Testfp); \ 28 | } while (0) 29 | #define TEST_RESULT(arg) do { \ 30 | Local__TestCounter++; \ 31 | if ((arg) == TEST_SUCCESS) fprintf(Testfp, Global__PassedTestStr, Local__FuncName, __LINE__, Local__TestCounter); \ 32 | else \ 33 | { \ 34 | fprintf(Testfp, Global__FailedTestStr, Local__FuncName, __LINE__, Local__TestCounter); \ 35 | Local__TestsFailed++; \ 36 | } \ 37 | fflush(Testfp); \ 38 | } while (0) 39 | #define TEST_CUSTOM_START() do { \ 40 | fprintf(Testfp, Global__CustomTestStartStr, Local__FuncName, __LINE__); \ 41 | fflush(Testfp); \ 42 | } while (0) 43 | #define TEST_CUSTOM_END() do { \ 44 | Local__TestCounter++; \ 45 | fprintf(Testfp, Global__CustomTestEndStr, Local__FuncName, __LINE__); \ 46 | fflush(Testfp); \ 47 | } while (0) 48 | #define TEST_SUMMARY() do { \ 49 | if (Local__TestsFailed) fprintf(Testfp, Global__SummaryTestsStr, Local__FuncName, Local__TestCounter, Local__TestsFailed); \ 50 | else fprintf(Testfp, Global__SummaryTestsSuccessStr, Local__FuncName, Local__TestCounter); \ 51 | fflush(Testfp); \ 52 | } while (0) 53 | #define TEST_RETURN() return !Local__TestsFailed 54 | -------------------------------------------------------------------------------- /utf8/utf8_appinfo.cpp: -------------------------------------------------------------------------------- 1 | // Cross-platform Unicode application information. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #include "utf8_appinfo.h" 5 | 6 | #include 7 | 8 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 9 | #include 10 | #else 11 | #include 12 | #include 13 | #include 14 | 15 | #ifdef __APPLE__ 16 | #include 17 | #endif 18 | #endif 19 | 20 | namespace CubicleSoft 21 | { 22 | namespace UTF8 23 | { 24 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 25 | bool AppInfo::GetExecutableFilename(char *Buffer, size_t &BufferSize, const char *Argv0) 26 | { 27 | size_t z = BufferSize; 28 | size_t z2 = z; 29 | BufferSize = 0; 30 | if (z2) Buffer[0] = '\0'; 31 | 32 | // Retrieve the full EXE path and filename for this process. 33 | WCHAR Filename[8192]; 34 | size_t y = (size_t)::GetModuleFileNameW(NULL, Filename, sizeof(Filename) / sizeof(WCHAR)); 35 | if (y >= sizeof(Filename) / sizeof(WCHAR) - 1) return false; 36 | Filename[y] = L'\0'; 37 | 38 | Util::ConvertToUTF8(Filename, y + 1, sizeof(WCHAR), (std::uint8_t *)Buffer, z); 39 | if (z == z2) return false; 40 | BufferSize = z; 41 | 42 | return true; 43 | } 44 | 45 | bool AppInfo::GetSystemAppStorageDir(char *Buffer, size_t &BufferSize, const char *AppDir) 46 | { 47 | size_t z = BufferSize; 48 | size_t z2 = z; 49 | BufferSize = 0; 50 | if (z2) Buffer[0] = '\0'; 51 | 52 | WCHAR Dirname[8192]; 53 | if (::SHGetFolderPathW(NULL, CSIDL_COMMON_APPDATA | CSIDL_FLAG_CREATE, NULL, 0, Dirname) != S_OK) return false; 54 | 55 | return GetWindowsUTF8StorageDir(Dirname, Buffer, BufferSize, AppDir, z, z2); 56 | } 57 | 58 | bool AppInfo::GetCurrentUserAppStorageDir(char *Buffer, size_t &BufferSize, const char *AppDir) 59 | { 60 | size_t z = BufferSize; 61 | size_t z2 = z; 62 | BufferSize = 0; 63 | if (z2) Buffer[0] = '\0'; 64 | 65 | WCHAR Dirname[8192]; 66 | if (::SHGetFolderPathW(NULL, CSIDL_LOCAL_APPDATA | CSIDL_FLAG_CREATE, NULL, 0, Dirname) != S_OK) return false; 67 | 68 | return GetWindowsUTF8StorageDir(Dirname, Buffer, BufferSize, AppDir, z, z2); 69 | } 70 | 71 | bool AppInfo::GetTempStorageDir(char *Buffer, size_t &BufferSize) 72 | { 73 | size_t z2 = BufferSize; 74 | BufferSize = 0; 75 | if (z2) Buffer[0] = '\0'; 76 | 77 | WCHAR Dirname[8192]; 78 | size_t z = (size_t)::GetTempPathW(sizeof(Dirname) / sizeof(WCHAR), Dirname); 79 | if (!z) return false; 80 | 81 | return GetWindowsUTF8StorageDir(Dirname, Buffer, BufferSize, NULL, z, z2); 82 | } 83 | 84 | // Internal function to standardize the conversion to UTF-8 on Windows. 85 | bool AppInfo::GetWindowsUTF8StorageDir(WCHAR *Dirname, char *Buffer, size_t &BufferSize, const char *AppDir, size_t z, size_t z2) 86 | { 87 | Util::ConvertToUTF8(Dirname, wcslen(Dirname), sizeof(WCHAR), (std::uint8_t *)Buffer, z); 88 | 89 | if (z < z2 && Buffer[z - 1] != '\\') Buffer[z++] = '\\'; 90 | if (AppDir != NULL) 91 | { 92 | while (z < z2 && *AppDir) Buffer[z++] = *AppDir++; 93 | 94 | if (z < z2 && Buffer[z - 1] != '\\') Buffer[z++] = '\\'; 95 | } 96 | if (z < z2) Buffer[z++] = '\0'; 97 | if (z == z2) return false; 98 | BufferSize = z; 99 | 100 | return true; 101 | } 102 | 103 | #else 104 | 105 | bool AppInfo::GetExecutableFilename(char *Buffer, size_t &BufferSize, const char *Argv0) 106 | { 107 | size_t z2 = BufferSize; 108 | BufferSize = 0; 109 | if (z2) Buffer[0] = '\0'; 110 | 111 | // Linux. 112 | ssize_t z = readlink("/proc/self/exe", Buffer, z2); 113 | 114 | // FreeBSD. 115 | if (z == -1) z = readlink("/proc/curproc/file", Buffer, z2); 116 | 117 | // NetBSD. 118 | if (z == -1) z = readlink("/proc/curproc/exe", Buffer, z2); 119 | 120 | // Solaris. 121 | if (z == -1) z = readlink("/proc/self/path/a.out", Buffer, z2); 122 | 123 | // Mac OSX. 124 | #ifdef __APPLE__ 125 | if (z == -1) z = proc_pidpath(getpid(), Buffer, (z2 > PROC_PIDPATHINFO_MAXSIZE ? PROC_PIDPATHINFO_MAXSIZE : z2)); 126 | if (z > -1) z = strlen(Buffer); 127 | #endif 128 | 129 | if (z2) Buffer[z2 - 1] = '\0'; 130 | 131 | char *Filename; 132 | if (z > 0) 133 | { 134 | if ((size_t)z < z2) Buffer[z] = '\0'; 135 | 136 | Filename = realpath(Buffer, NULL); 137 | } 138 | else 139 | { 140 | // Other *NIX-style OS. 141 | if (Argv0 == NULL) return false; 142 | 143 | Filename = realpath(Argv0, NULL); 144 | } 145 | 146 | if (Filename == NULL) return false; 147 | 148 | if (strlen(Filename) >= z2) 149 | { 150 | free(Filename); 151 | 152 | return false; 153 | } 154 | 155 | strcpy(Buffer, Filename); 156 | BufferSize = strlen(Buffer) + 1; 157 | 158 | free(Filename); 159 | 160 | return true; 161 | } 162 | 163 | bool AppInfo::GetSystemAppStorageDir(char *Buffer, size_t &BufferSize, const char *AppDir) 164 | { 165 | size_t z2 = BufferSize; 166 | BufferSize = 0; 167 | if (z2) Buffer[0] = '\0'; 168 | 169 | if (z2 < 10) return false; 170 | 171 | strcpy(Buffer, "/var/lib/"); 172 | size_t z = strlen(Buffer); 173 | 174 | return GetNIXStorageDir(Buffer, BufferSize, NULL, AppDir, z, z2); 175 | } 176 | 177 | bool AppInfo::GetCurrentUserAppStorageDir(char *Buffer, size_t &BufferSize, const char *AppDir) 178 | { 179 | size_t z2 = BufferSize; 180 | BufferSize = 0; 181 | if (z2) Buffer[0] = '\0'; 182 | 183 | const char *HomeDir = getenv("HOME"); 184 | if (HomeDir == NULL) HomeDir = getpwuid(getuid())->pw_dir; 185 | if (HomeDir == NULL) return false; 186 | 187 | if (z2 < strlen(HomeDir)) return false; 188 | 189 | strcpy(Buffer, HomeDir); 190 | size_t z = strlen(HomeDir); 191 | 192 | return GetNIXStorageDir(Buffer, BufferSize, ".", AppDir, z, z2); 193 | } 194 | 195 | bool AppInfo::GetTempStorageDir(char *Buffer, size_t &BufferSize) 196 | { 197 | size_t z2 = BufferSize; 198 | BufferSize = 0; 199 | if (z2) Buffer[0] = '\0'; 200 | 201 | if (z2 < 5) return false; 202 | 203 | strcpy(Buffer, "/tmp/"); 204 | size_t z = 5; 205 | 206 | return GetNIXStorageDir(Buffer, BufferSize, NULL, NULL, z, z2); 207 | } 208 | 209 | // Internal function to standardize a directory on *NIX. 210 | bool AppInfo::GetNIXStorageDir(char *Buffer, size_t &BufferSize, const char *AppDirPrefix, const char *AppDir, size_t z, size_t z2) 211 | { 212 | if (z < z2 && Buffer[z - 1] != '/') Buffer[z++] = '/'; 213 | if (AppDir != NULL) 214 | { 215 | if (AppDirPrefix != NULL) 216 | { 217 | while (z < z2 && *AppDirPrefix) Buffer[z++] = *AppDirPrefix++; 218 | } 219 | 220 | while (z < z2 && *AppDir) Buffer[z++] = *AppDir++; 221 | 222 | if (z < z2 && Buffer[z - 1] != '/') Buffer[z++] = '/'; 223 | } 224 | if (z < z2) Buffer[z++] = '\0'; 225 | if (z == z2) return false; 226 | BufferSize = z; 227 | 228 | return true; 229 | } 230 | 231 | #endif 232 | 233 | bool AppInfo::GetExecutablePath(char *Buffer, size_t &BufferSize, const char *Argv0) 234 | { 235 | if (!GetExecutableFilename(Buffer, BufferSize, Argv0)) return false; 236 | 237 | while (BufferSize && Buffer[BufferSize - 1] != '/' && Buffer[BufferSize - 1] != '\\') BufferSize--; 238 | Buffer[BufferSize++] = '\0'; 239 | 240 | return true; 241 | } 242 | } 243 | } 244 | -------------------------------------------------------------------------------- /utf8/utf8_appinfo.h: -------------------------------------------------------------------------------- 1 | // Cross-platform Unicode application information. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_UTF8_APPINFO 5 | #define CUBICLESOFT_UTF8_APPINFO 6 | 7 | #include "utf8_util.h" 8 | 9 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 10 | #include 11 | #else 12 | #include 13 | #include 14 | #endif 15 | 16 | namespace CubicleSoft 17 | { 18 | namespace UTF8 19 | { 20 | class AppInfo 21 | { 22 | public: 23 | // Retrieves the current executable's full path and filename. 24 | // Argv0 may be NULL and is only used on some platforms. 25 | static bool GetExecutableFilename(char *Buffer, size_t &BufferSize, const char *Argv0 = NULL); 26 | 27 | // Retrieves the current executable's full path without filename. 28 | // Argv0 may be NULL and is only used on some platforms. 29 | static bool GetExecutablePath(char *Buffer, size_t &BufferSize, const char *Argv0 = NULL); 30 | 31 | // Retrieves the appropriate system-level storage directory for the application to store data (e.g. All Users\AppData\AppDir\, /var/lib/AppDir/). 32 | // AppDir, if not NULL, is appended. 33 | static bool GetSystemAppStorageDir(char *Buffer, size_t &BufferSize, const char *AppDir = NULL); 34 | 35 | // Retrieves the current user's application data storage directory (e.g. AppData\Local\AppDir/, /home/user/.AppDir/). 36 | // AppDir, if not NULL, is appended. 37 | static bool GetCurrentUserAppStorageDir(char *Buffer, size_t &BufferSize, const char *AppDir = NULL); 38 | 39 | // Retrieves the temporary file storage directory (e.g. C:\TEMP\, /tmp/). 40 | static bool GetTempStorageDir(char *Buffer, size_t &BufferSize); 41 | 42 | private: 43 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 44 | static bool GetWindowsUTF8StorageDir(WCHAR *Dirname, char *Buffer, size_t &BufferSize, const char *AppDir, size_t z, size_t z2); 45 | #else 46 | static bool GetNIXStorageDir(char *Buffer, size_t &BufferSize, const char *AppDirPrefix, const char *AppDir, size_t z, size_t z2); 47 | #endif 48 | }; 49 | } 50 | } 51 | 52 | #endif 53 | -------------------------------------------------------------------------------- /utf8/utf8_file_dir.h: -------------------------------------------------------------------------------- 1 | // Cross-platform UTF-8, large file (> 2GB) and directory manipulation classes. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_UTF8_FILE 5 | #define CUBICLESOFT_UTF8_FILE 6 | 7 | #ifndef _LARGEFILE64_SOURCE 8 | #define _LARGEFILE64_SOURCE 9 | #endif 10 | 11 | #include "utf8_util.h" 12 | 13 | #include 14 | #include 15 | 16 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 17 | #include 18 | #include 19 | #include 20 | #else 21 | #include 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | 32 | #ifdef __APPLE__ 33 | #define lseek64 lseek 34 | #define open64 open 35 | #define off64_t off_t 36 | #endif 37 | #endif 38 | 39 | #ifndef O_RDONLY 40 | #define O_RDONLY 00 41 | #endif 42 | #ifndef O_WRONLY 43 | #define O_WRONLY 01 44 | #endif 45 | #ifndef O_RDWR 46 | #define O_RDWR 02 47 | #endif 48 | #ifndef O_ACCMODE 49 | #define O_ACCMODE (O_RDONLY | O_WRONLY | O_RDWR) 50 | #endif 51 | 52 | #ifndef O_CREAT 53 | #define O_CREAT 0100 54 | #endif 55 | #ifndef O_EXCL 56 | #define O_EXCL 0200 57 | #endif 58 | #ifndef O_TRUNC 59 | #define O_TRUNC 01000 60 | #endif 61 | #ifndef O_APPEND 62 | #define O_APPEND 02000 63 | #endif 64 | 65 | // O_UNSAFE allows for devices to be opened for raw reading/writing. 66 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 67 | #ifndef O_UNSAFE 68 | #define O_UNSAFE 04000 69 | #endif 70 | #endif 71 | 72 | #ifndef _S_IFLNK 73 | #ifdef S_IFLNK 74 | #define _S_IFLNK S_IFLNK 75 | #else 76 | #define _S_IFLNK 0120000 77 | #endif 78 | #endif 79 | 80 | #ifndef S_ISLNK 81 | #define S_ISLNK(mode) (((mode) & S_IFMT) == _S_IFLNK) 82 | #endif 83 | 84 | 85 | namespace CubicleSoft 86 | { 87 | namespace UTF8 88 | { 89 | class File 90 | { 91 | public: 92 | enum SeekType 93 | { 94 | SeekStart, 95 | SeekEnd, 96 | SeekForward, 97 | SeekBackward 98 | }; 99 | 100 | enum ShareType 101 | { 102 | ShareBoth, 103 | ShareRead, 104 | ShareWrite, 105 | ShareNone 106 | }; 107 | 108 | struct FilenameInfo 109 | { 110 | size_t StartVolume, StartPath, StartFilename, StartExtension, StartLastExtension, Length; 111 | }; 112 | 113 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 114 | typedef struct __stat64 FileStat; 115 | #else 116 | typedef struct stat FileStat; 117 | #endif 118 | 119 | File(); 120 | virtual ~File(); 121 | 122 | virtual bool IsOpen() const; 123 | bool Open(const char *Filename, int Flags, ShareType ShareFlag = ShareBoth, int Mode = 0666); 124 | virtual bool Seek(SeekType whence, std::uint64_t Pos); 125 | virtual bool Read(std::uint8_t *Data, size_t DataSize, size_t &DataRead); 126 | 127 | // Result is not UTF-8 and also not intended for binary data. '\0' replaced with ' ' (space). 128 | // Result will have been terminated by EOF or an ASCII newline '\r', '\n', '\r\n' but trimmed. 129 | // SizeHint helps optimize memory allocation performance. 130 | // Check out Sync::TLS for a high-performance allocator. 131 | virtual char *LineInput(size_t SizeHint = 1024, void *AltMallocManager = NULL, void *(*AltRealloc)(void *, void *, size_t) = NULL); 132 | 133 | virtual bool Write(const char *Data, size_t &DataWritten); 134 | virtual bool Write(const std::uint8_t *Data, size_t DataSize, size_t &DataWritten); 135 | virtual bool Flush(); 136 | inline std::uint64_t GetCurrPos() const { return MxCurrPos; } 137 | inline std::uint64_t GetMaxPos() const { return MxMaxPos; } 138 | virtual bool UpdateMaxPos(); 139 | virtual bool Close(); 140 | 141 | // Some static functions specifically for files. 142 | static bool GetPlatformFilename(char *Result, size_t ResultSize, const char *Filename, bool AllowUnsafe = false); 143 | static bool IsValidFilenameFormat(const char *Filename); 144 | static void GetPlatformFilenameInfo(FilenameInfo &Result, const char *Filename); 145 | static bool GetAbsoluteFilename(char *Result, size_t ResultSize, const char *BaseDir, const char *Filename, bool TrailingSlash = false); 146 | static bool Exists(const char *Filename); 147 | static bool Realpath(char *Result, size_t ResultSize, const char *Filename); 148 | static bool Chmod(const char *Filename, int Mode); 149 | static bool Chown(const char *Filename, const char *Owner); 150 | static bool Chgrp(const char *Filename, const char *Group); 151 | static bool Copy(const char *SrcFilename, const char *DestFilename); 152 | static bool Move(const char *SrcFilename, const char *DestFilename); 153 | static bool Delete(const char *Filename); 154 | static bool Stat(FileStat &Result, const char *Filename, bool Link = false); 155 | static bool Symlink(const char *Src, const char *Dest); 156 | static bool Readlink(char *Result, size_t ResultSize, const char *Filename); 157 | 158 | // For quickly loading relatively small files. 159 | // Check out Sync::TLS for a high-performance allocator. 160 | static bool LoadEntireFile(const char *Filename, char *&Result, size_t &BytesRead, void *AltMallocManager = NULL, void *(*AltMalloc)(void *, size_t) = NULL, void (*AltFree)(void *, void *) = NULL); 161 | 162 | // Timestamps are in Unix microsecond format. 163 | static bool SetFileTimes(const char *Filename, std::uint64_t *Creation, std::uint64_t *LastAccess, std::uint64_t *LastUpdate); 164 | 165 | private: 166 | // Deny copy construction and assignment. Use a (smart) pointer instead. 167 | File(const File &); 168 | File &operator=(const File &); 169 | 170 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 171 | public: 172 | static bool GetWindowsPlatformFilename(LPWSTR Result, size_t ResultSize, const char *Filename, bool AllowUnsafe = false); 173 | static bool GetWindowsFilenameInfo(BY_HANDLE_FILE_INFORMATION *Result, LPWSTR Filename); 174 | private: 175 | static FILETIME ConvertUnixMicrosecondTimeToFILETIME(std::uint64_t TempTime); 176 | static __time64_t ConvertFILETIMEToUnixTime(FILETIME TempTime); 177 | typedef BOOL (APIENTRY *CreateSymbolicLinkWFunc)(LPWSTR, LPWSTR, DWORD); 178 | static bool SetThreadProcessPrivilege(LPCWSTR PrivilegeName, bool Enable); 179 | 180 | HANDLE MxFile; 181 | #else 182 | int MxFile; 183 | bool MxLocked; 184 | #endif 185 | 186 | protected: 187 | bool MxRead, MxWrite; 188 | std::uint64_t MxCurrPos, MxMaxPos; 189 | }; 190 | 191 | class Dir 192 | { 193 | public: 194 | Dir(); 195 | ~Dir(); 196 | 197 | bool Open(const char *Dirname); 198 | bool Read(char *Filename, size_t Size); 199 | bool Close(); 200 | 201 | // Some static functions specifically for directories. Some functions are possibly not thread-safe. 202 | static bool Getcwd(char *Result, size_t ResultSize); 203 | 204 | // Unlike most directory/file calls, UTF8::Dir::Mkdir() is more difficult to use because Dirname must reference an 205 | // absolute path but UTF8::File::Realpath() can't be used on the complete calculation since one or more directories 206 | // likely don't exist (yet). Use UTF8::File::Realpath() to transform the known existing part of the path and then 207 | // carefully concatenate the new path item(s) and call UTF8::Dir::Mkdir(). 208 | static bool Mkdir(const char *Dirname, int Mode = 0777, bool Recursive = false); 209 | 210 | static bool Rmdir(const char *Dirname, bool Recursive = false); 211 | 212 | private: 213 | // Deny copy construction and assignment. Use a (smart) pointer instead. 214 | Dir(const Dir &); 215 | Dir &operator=(const Dir &); 216 | 217 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 218 | static void Rmdir_RecurseInternal(WCHAR *Dirname, size_t Size); 219 | 220 | HANDLE MxDir; 221 | WIN32_FIND_DATAW MxFindData; 222 | #else 223 | static void Rmdir_RecurseInternal(char *Dirname, size_t Size); 224 | 225 | DIR *MxDir; 226 | #endif 227 | }; 228 | } 229 | } 230 | 231 | #endif 232 | -------------------------------------------------------------------------------- /utf8/utf8_mixed_var.h: -------------------------------------------------------------------------------- 1 | // Adds UTF8 support to the MixedVar template. 2 | // (C) 2021 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_UTF8_MIXED_VAR 5 | #define CUBICLESOFT_UTF8_MIXED_VAR 6 | 7 | #include "../templates/static_mixed_var.h" 8 | #include "utf8_util.h" 9 | 10 | #if defined(_WIN32) || defined(WIN32) || defined(_WIN64) || defined(WIN64) 11 | #include 12 | #else 13 | #include 14 | 15 | #ifndef WCHAR 16 | #define WCHAR wchar_t 17 | #endif 18 | #endif 19 | 20 | namespace CubicleSoft 21 | { 22 | namespace UTF8 23 | { 24 | // Must be used like: UTF8MixedVar 25 | // Designed to be extended but not overridden. 26 | template 27 | class UTF8MixedVar : public StaticMixedVar 28 | { 29 | public: 30 | void SetUTF8(const WCHAR *src, size_t srcsize) 31 | { 32 | this->MxMode = MV_Str; 33 | this->MxStrPos = sizeof(this->MxStr); 34 | Util::ConvertToUTF8(src, srcsize, sizeof(WCHAR), (std::uint8_t *)this->MxStr, this->MxStrPos); 35 | if (this->MxStrPos == sizeof(this->MxStr) || (this->MxStrPos && this->MxStr[this->MxStrPos - 1] == '\0')) this->MxStrPos--; 36 | this->MxStr[this->MxStrPos] = '\0'; 37 | } 38 | 39 | inline void SetUTF8(const WCHAR *src) 40 | { 41 | SetUTF8(src, wcslen(src)); 42 | } 43 | 44 | void ConvertFromUTF8(WCHAR *DestData, size_t &DestDataSize) 45 | { 46 | Util::ConvertFromUTF8((std::uint8_t *)this->MxStr, this->MxStrPos + 1, DestData, DestDataSize, sizeof(WCHAR)); 47 | } 48 | 49 | void AppendUTF8(const WCHAR *src) 50 | { 51 | size_t srcsize = wcslen(src); 52 | if (srcsize > sizeof(this->MxStr) - this->MxStrPos) srcsize = sizeof(this->MxStr) - this->MxStrPos; 53 | Util::ConvertToUTF8(src, srcsize, sizeof(WCHAR), (std::uint8_t *)this->MxStr + this->MxStrPos, srcsize); 54 | this->MxStrPos += srcsize; 55 | if (this->MxStrPos == sizeof(this->MxStr) || (this->MxStrPos && this->MxStr[this->MxStrPos - 1] == '\0')) this->MxStrPos--; 56 | this->MxStr[this->MxStrPos] = '\0'; 57 | } 58 | }; 59 | } 60 | } 61 | 62 | #endif -------------------------------------------------------------------------------- /utf8/utf8_util.cpp: -------------------------------------------------------------------------------- 1 | // Cross-platform UTF-8 string conversion and lightweight parser. 2 | // (C) 2013 CubicleSoft. All Rights Reserved. 3 | 4 | #include "utf8_util.h" 5 | 6 | namespace CubicleSoft 7 | { 8 | namespace UTF8 9 | { 10 | char *Util::ConvertToUTF8(const void *SrcData, size_t SrcWidth, size_t *LastPos, void *AltMallocManager, void *(*AltMalloc)(void *, size_t)) 11 | { 12 | size_t x = 0, y; 13 | std::uint8_t *DestData; 14 | 15 | if (SrcWidth == 1) 16 | { 17 | const std::uint8_t *SrcData2 = (const std::uint8_t *)SrcData; 18 | 19 | do 20 | { 21 | x++; 22 | } while (*SrcData2++); 23 | } 24 | else if (SrcWidth == 2) 25 | { 26 | const uint16_t *SrcData2 = (const uint16_t *)SrcData; 27 | 28 | do 29 | { 30 | x++; 31 | } while (*SrcData2++); 32 | } 33 | else if (SrcWidth == 4) 34 | { 35 | const std::uint32_t *SrcData2 = (const std::uint32_t *)SrcData; 36 | 37 | do 38 | { 39 | x++; 40 | } while (*SrcData2++); 41 | } 42 | else return NULL; 43 | 44 | ConvertToUTF8(SrcData, x, SrcWidth, NULL, y, NULL); 45 | DestData = (AltMalloc != NULL ? (std::uint8_t *)AltMalloc(AltMallocManager, y) : new std::uint8_t[y]); 46 | ConvertToUTF8(SrcData, x, SrcWidth, DestData, y, LastPos); 47 | 48 | return (char *)DestData; 49 | } 50 | 51 | void *Util::ConvertFromUTF8(const char *SrcData, size_t DestWidth, void *AltMallocManager, void *(*AltMalloc)(void *, size_t)) 52 | { 53 | size_t x, y; 54 | void *DestData; 55 | 56 | x = strlen(SrcData); 57 | ConvertFromUTF8((const std::uint8_t *)SrcData, x, NULL, y, DestWidth); 58 | if (DestWidth == 1) DestData = (AltMalloc != NULL ? (std::uint8_t *)AltMalloc(AltMallocManager, y) : new std::uint8_t[y]); 59 | else if (DestWidth == 2) DestData = (AltMalloc != NULL ? (std::uint16_t *)AltMalloc(AltMallocManager, y * sizeof(uint16_t)) : new uint16_t[y]); 60 | else if (DestWidth == 4) DestData = (AltMalloc != NULL ? (std::uint32_t *)AltMalloc(AltMallocManager, y * sizeof(uint32_t)) : new std::uint32_t[y]); 61 | else return NULL; 62 | ConvertFromUTF8((const std::uint8_t *)SrcData, x, DestData, y, DestWidth); 63 | 64 | return DestData; 65 | } 66 | 67 | bool Util::AppendUTF8CodePoint(std::uint32_t TempCP, std::uint8_t *DestData, size_t &DestDataSize, size_t MaxDestDataSize) 68 | { 69 | // 0xD800-0xDFFF are for UTF-16 surrogate pairs. Invalid characters. 70 | // 0xFDD0-0xFDEF are non-characters. 71 | // 0x*FFFE and 0x*FFFF are reserved. 72 | if ((TempCP >= 0xD800 && TempCP <= 0xDFFF) || (TempCP >= 0xFDD0 && TempCP <= 0xFDEF) || (TempCP & 0xFFFE) == 0xFFFE) return false; 73 | 74 | // First character can't be a combining code point. 75 | if (!DestDataSize && ((TempCP >= 0x0300 && TempCP <= 0x036F) || (TempCP >= 0x1DC0 && TempCP <= 0x1DFF) || (TempCP >= 0x20D0 && TempCP <= 0x20FF) || (TempCP >= 0xFE20 && TempCP <= 0xFE2F))) return false; 76 | 77 | if (TempCP <= 0x7F) 78 | { 79 | if (DestData == NULL) DestDataSize++; 80 | else if (DestDataSize >= MaxDestDataSize) return false; 81 | else DestData[DestDataSize++] = (std::uint8_t)TempCP; 82 | 83 | return true; 84 | } 85 | else if (TempCP <= 0x07FF) 86 | { 87 | if (DestData == NULL) DestDataSize += 2; 88 | else if (DestDataSize + 1 >= MaxDestDataSize) return false; 89 | else 90 | { 91 | DestData[DestDataSize++] = (std::uint8_t)(0xC0 | (TempCP >> 6)); 92 | DestData[DestDataSize++] = (std::uint8_t)(0x80 | (TempCP & 0x3F)); 93 | } 94 | 95 | return true; 96 | } 97 | else if (TempCP <= 0xFFFF) 98 | { 99 | if (DestData == NULL) DestDataSize += 3; 100 | else if (DestDataSize + 2 >= MaxDestDataSize) return false; 101 | else 102 | { 103 | DestData[DestDataSize++] = (std::uint8_t)(0xE0 | (TempCP >> 12)); 104 | DestData[DestDataSize++] = (std::uint8_t)(0x80 | ((TempCP >> 6) & 0x3F)); 105 | DestData[DestDataSize++] = (std::uint8_t)(0x80 | (TempCP & 0x3F)); 106 | } 107 | 108 | return true; 109 | } 110 | else if (TempCP <= 0x10FFFF) 111 | { 112 | if (DestData == NULL) DestDataSize += 4; 113 | else if (DestDataSize + 3 >= MaxDestDataSize) return false; 114 | else 115 | { 116 | DestData[DestDataSize++] = (std::uint8_t)(0xF0 | (TempCP >> 18)); 117 | DestData[DestDataSize++] = (std::uint8_t)(0x80 | ((TempCP >> 12) & 0x3F)); 118 | DestData[DestDataSize++] = (std::uint8_t)(0x80 | ((TempCP >> 6) & 0x3F)); 119 | DestData[DestDataSize++] = (std::uint8_t)(0x80 | (TempCP & 0x3F)); 120 | } 121 | 122 | return true; 123 | } 124 | 125 | return false; 126 | } 127 | 128 | void Util::ConvertToUTF8(const void *SrcData, size_t SrcDataSize, size_t SrcWidth, std::uint8_t *DestData, size_t &DestDataSize, size_t *LastPos) 129 | { 130 | size_t x, y = DestDataSize; 131 | std::uint32_t TempCP; 132 | 133 | DestDataSize = 0; 134 | 135 | if (SrcWidth == 1) 136 | { 137 | const std::uint8_t *SrcData2 = (const std::uint8_t *)SrcData; 138 | 139 | x = 0; 140 | while (x < SrcDataSize) 141 | { 142 | TempCP = SrcData2[x]; 143 | 144 | if (TempCP <= 0x7F) 145 | { 146 | x++; 147 | if (AppendUTF8CodePoint(TempCP, DestData, DestDataSize, y) && LastPos != NULL) *LastPos = x; 148 | } 149 | else if (x + 1 < SrcDataSize && (TempCP & 0xE0) == 0xC0 && (SrcData2[x + 1] & 0xC0) == 0x80) 150 | { 151 | TempCP = (((TempCP & 0x1F) << 6) | ((std::uint32_t)SrcData2[x + 1] & 0x3F)); 152 | x += 2; 153 | if (AppendUTF8CodePoint(TempCP, DestData, DestDataSize, y) && LastPos != NULL) *LastPos = x; 154 | } 155 | else if (x + 2 < SrcDataSize && (TempCP & 0xF0) == 0xE0 && (SrcData2[x + 1] & 0xC0) == 0x80 && (SrcData2[x + 2] & 0xC0) == 0x80) 156 | { 157 | TempCP = (((TempCP & 0x0F) << 12) | (((std::uint32_t)SrcData2[x + 1] & 0x3F) << 6) | ((std::uint32_t)SrcData2[x + 2] & 0x3F)); 158 | AppendUTF8CodePoint(TempCP, DestData, DestDataSize, y); 159 | x += 3; 160 | } 161 | else if (x + 3 < SrcDataSize && (TempCP & 0xF8) == 0xF0 && (SrcData2[x + 1] & 0xC0) == 0x80 && (SrcData2[x + 2] & 0xC0) == 0x80 && (SrcData2[x + 3] & 0xC0) == 0x80) 162 | { 163 | TempCP = (((TempCP & 0x07) << 18) | (((std::uint32_t)SrcData2[x + 1] & 0x3F) << 12) | (((std::uint32_t)SrcData2[x + 2] & 0x3F) << 6) | ((std::uint32_t)SrcData2[x + 3] & 0x3F)); 164 | if (AppendUTF8CodePoint(TempCP, DestData, DestDataSize, y) && LastPos != NULL) *LastPos = x; 165 | x += 4; 166 | } 167 | else x++; 168 | } 169 | } 170 | else if (SrcWidth == 2) 171 | { 172 | const uint16_t *SrcData2 = (const uint16_t *)SrcData; 173 | 174 | x = 0; 175 | for (x = 0; x < SrcDataSize; x++) 176 | { 177 | TempCP = SrcData2[x]; 178 | if (TempCP < 0xD800 || TempCP > 0xDBFF) 179 | { 180 | if (AppendUTF8CodePoint(TempCP, DestData, DestDataSize, y) && LastPos != NULL) *LastPos = x; 181 | } 182 | else 183 | { 184 | x++; 185 | 186 | if (x < SrcDataSize && SrcData2[x] >= 0xDC00 && SrcData2[x] <= 0xDFFF) 187 | { 188 | TempCP = (((TempCP - 0xD800) << 10) | ((std::uint32_t)SrcData2[x] - 0xDC00)) + 0x010000; 189 | if (AppendUTF8CodePoint(TempCP, DestData, DestDataSize, y) && LastPos != NULL) *LastPos = x; 190 | } 191 | } 192 | } 193 | } 194 | else if (SrcWidth == 4) 195 | { 196 | const std::uint32_t *SrcData2 = (const std::uint32_t *)SrcData; 197 | 198 | for (x = 0; x < SrcDataSize; x++) 199 | { 200 | TempCP = SrcData2[x]; 201 | 202 | if (AppendUTF8CodePoint(TempCP, DestData, DestDataSize, y) && LastPos != NULL) *LastPos = x; 203 | } 204 | } 205 | } 206 | 207 | bool Util::AppendUTFCodePoint(std::uint32_t TempCP, void *DestData, size_t &DestDataSize, size_t DestWidth, size_t MaxDestDataSize) 208 | { 209 | if (DestWidth == 1) return AppendUTF8CodePoint(TempCP, (std::uint8_t *)DestData, DestDataSize, MaxDestDataSize); 210 | 211 | // 0xD800-0xDFFF are for UTF-16 surrogate pairs. Invalid characters. 212 | // 0xFDD0-0xFDEF are non-characters. 213 | // 0x*FFFE and 0x*FFFF are reserved. 214 | // The largest possible character is 0x10FFFF. 215 | if ((TempCP >= 0xD800 && TempCP <= 0xDFFF) || (TempCP >= 0xFDD0 && TempCP <= 0xFDEF) || (TempCP & 0xFFFE) == 0xFFFE || TempCP > 0x10FFFF) return false; 216 | 217 | // First character can't be a combining code point. 218 | if (!DestDataSize && ((TempCP >= 0x0300 && TempCP <= 0x036F) || (TempCP >= 0x1DC0 && TempCP <= 0x1DFF) || (TempCP >= 0x20D0 && TempCP <= 0x20FF) || (TempCP >= 0xFE20 && TempCP <= 0xFE2F))) return false; 219 | 220 | if (DestWidth == 2) 221 | { 222 | uint16_t *DestData2 = (uint16_t *)DestData; 223 | 224 | if (TempCP > 0xFFFF) 225 | { 226 | if (DestData2 == NULL) DestDataSize += 2; 227 | else if (DestDataSize + 1 >= MaxDestDataSize) return false; 228 | else 229 | { 230 | TempCP -= 0x010000; 231 | DestData2[DestDataSize++] = (uint16_t)(((TempCP >> 10) & 0x03FF) + 0xD800); 232 | DestData2[DestDataSize++] = (uint16_t)((TempCP & 0x03FF) + 0xDC00); 233 | } 234 | } 235 | else if (DestData2 == NULL) DestDataSize++; 236 | else if (DestDataSize >= MaxDestDataSize) return false; 237 | else DestData2[DestDataSize++] = (uint16_t)TempCP; 238 | } 239 | else if (DestWidth == 4) 240 | { 241 | std::uint32_t *DestData2 = (std::uint32_t *)DestData; 242 | 243 | if (DestData2 == NULL) DestDataSize++; 244 | else if (DestDataSize >= MaxDestDataSize) return false; 245 | else DestData2[DestDataSize++] = TempCP; 246 | } 247 | 248 | return true; 249 | } 250 | 251 | void Util::ConvertFromUTF8(const std::uint8_t *SrcData, size_t SrcDataSize, void *DestData, size_t &DestDataSize, size_t DestWidth) 252 | { 253 | size_t x, y; 254 | std::uint32_t TempCP; 255 | 256 | y = DestDataSize; 257 | DestDataSize = 0; 258 | 259 | x = 0; 260 | while (x < SrcDataSize && y) 261 | { 262 | TempCP = SrcData[x]; 263 | 264 | if (TempCP <= 0x7F) 265 | { 266 | if (!AppendUTFCodePoint(TempCP, DestData, DestDataSize, DestWidth, y)) return; 267 | x++; 268 | } 269 | else if (x + 1 < SrcDataSize && (TempCP & 0xE0) == 0xC0 && (SrcData[x + 1] & 0xC0) == 0x80) 270 | { 271 | TempCP = (((TempCP & 0x1F) << 6) | ((std::uint32_t)SrcData[x + 1] & 0x3F)); 272 | if (!AppendUTFCodePoint(TempCP, DestData, DestDataSize, DestWidth, y)) return; 273 | x += 2; 274 | } 275 | else if (x + 2 < SrcDataSize && (TempCP & 0xF0) == 0xE0 && (SrcData[x + 1] & 0xC0) == 0x80 && (SrcData[x + 2] & 0xC0) == 0x80) 276 | { 277 | TempCP = (((TempCP & 0x0F) << 12) | (((std::uint32_t)SrcData[x + 1] & 0x3F) << 6) | ((std::uint32_t)SrcData[x + 2] & 0x3F)); 278 | if (!AppendUTFCodePoint(TempCP, DestData, DestDataSize, DestWidth, y)) return; 279 | x += 3; 280 | } 281 | else if (x + 3 < SrcDataSize && (TempCP & 0xF8) == 0xF0 && (SrcData[x + 1] & 0xC0) == 0x80 && (SrcData[x + 2] & 0xC0) == 0x80 && (SrcData[x + 3] & 0xC0) == 0x80) 282 | { 283 | TempCP = (((TempCP & 0x07) << 18) | (((std::uint32_t)SrcData[x + 1] & 0x3F) << 12) | (((std::uint32_t)SrcData[x + 2] & 0x3F) << 6) | ((std::uint32_t)SrcData[x + 3] & 0x3F)); 284 | if (!AppendUTFCodePoint(TempCP, DestData, DestDataSize, DestWidth, y)) return; 285 | x += 4; 286 | } 287 | else x++; 288 | } 289 | } 290 | 291 | size_t Util::strlen(const char *Str) 292 | { 293 | size_t x, y, Result; 294 | std::uint32_t ResultCP, NextCP; 295 | bool HasCombiningCP; 296 | 297 | x = 0; 298 | Result = 0; 299 | HasCombiningCP = false; 300 | while (NextCodePoint(ResultCP, NextCP, Str, x, y, HasCombiningCP)) 301 | { 302 | if (!HasCombiningCP) Result++; 303 | } 304 | if (HasCombiningCP) Result++; 305 | 306 | return Result; 307 | } 308 | 309 | bool Util::FindCodePoint(size_t &ResultPos, std::uint32_t CodePoint, const char *Str, bool AllowCombiningCP) 310 | { 311 | size_t x, y; 312 | std::uint32_t ResultCP, NextCP; 313 | bool HasCombiningCP; 314 | 315 | if (!AllowCombiningCP && IsCombiningCodePoint(CodePoint)) return false; 316 | 317 | x = 0; 318 | while (NextCodePoint(ResultCP, NextCP, Str, x, y, HasCombiningCP)) 319 | { 320 | if (ResultCP == CodePoint && (AllowCombiningCP || !HasCombiningCP)) 321 | { 322 | ResultPos = x; 323 | 324 | return true; 325 | } 326 | } 327 | 328 | return false; 329 | } 330 | 331 | bool Util::NextCodePoint(std::uint32_t &ResultCP, std::uint32_t &NextCP, const char *Str, size_t &Pos, size_t &Size, bool &HasCombiningCP) 332 | { 333 | const std::uint8_t *Str2 = (const std::uint8_t *)Str; 334 | 335 | if (Pos) 336 | { 337 | ResultCP = NextCP; 338 | if (ResultCP == 0) return false; 339 | } 340 | else 341 | { 342 | Size = strlen(Str); 343 | HasCombiningCP = false; 344 | 345 | ResultCP = Str2[0]; 346 | if (ResultCP == 0) return false; 347 | 348 | if (ResultCP <= 0x7F) Pos++; 349 | else if (Pos + 1 < Size && (ResultCP & 0xE0) == 0xC0 && (Str2[Pos + 1] & 0xC0) == 0x80) Pos += 2; 350 | else if (Pos + 2 < Size && (ResultCP & 0xF0) == 0xE0 && (Str2[Pos + 1] & 0xC0) == 0x80 && (Str2[Pos + 2] & 0xC0) == 0x80) Pos += 3; 351 | else if (Pos + 3 < Size && (ResultCP & 0xF8) == 0xF0 && (Str2[Pos + 1] & 0xC0) == 0x80 && (Str2[Pos + 2] & 0xC0) == 0x80 && (Str2[Pos + 3] & 0xC0) == 0x80) Pos += 4; 352 | else return false; 353 | 354 | // First character can't be a combining code point. 355 | if ((ResultCP >= 0x0300 && ResultCP <= 0x036F) || (ResultCP >= 0x1DC0 && ResultCP <= 0x1DFF) || (ResultCP >= 0x20D0 && ResultCP <= 0x20FF) || (ResultCP >= 0xFE20 && ResultCP <= 0xFE2F)) return false; 356 | } 357 | 358 | NextCP = Str2[Pos]; 359 | 360 | if (NextCP != 0) 361 | { 362 | if (NextCP <= 0x7F) Pos++; 363 | else if (Pos + 1 < Size && (NextCP & 0xE0) == 0xC0 && (Str2[Pos + 1] & 0xC0) == 0x80) Pos += 2; 364 | else if (Pos + 2 < Size && (NextCP & 0xF0) == 0xE0 && (Str2[Pos + 1] & 0xC0) == 0x80 && (Str2[Pos + 2] & 0xC0) == 0x80) Pos += 3; 365 | else if (Pos + 3 < Size && (NextCP & 0xF8) == 0xF0 && (Str2[Pos + 1] & 0xC0) == 0x80 && (Str2[Pos + 2] & 0xC0) == 0x80 && (Str2[Pos + 3] & 0xC0) == 0x80) Pos += 4; 366 | else NextCP = 0; 367 | } 368 | 369 | HasCombiningCP = ((NextCP >= 0x0300 && NextCP <= 0x036F) || (NextCP >= 0x1DC0 && NextCP <= 0x1DFF) || (NextCP >= 0x20D0 && NextCP <= 0x20FF) || (NextCP >= 0xFE20 && NextCP <= 0xFE2F)); 370 | 371 | return true; 372 | } 373 | 374 | bool Util::NextASCIICodePoint(char &ResultCP, std::uint32_t &NextCP, const char *Str, size_t &Pos, size_t &Size) 375 | { 376 | bool HasCombiningCP; 377 | std::uint32_t TempCP; 378 | 379 | while (NextCodePoint(TempCP, NextCP, Str, Pos, Size, HasCombiningCP)) 380 | { 381 | if (!HasCombiningCP && TempCP <= 0x7F) 382 | { 383 | ResultCP = (char)TempCP; 384 | 385 | return true; 386 | } 387 | } 388 | 389 | return false; 390 | } 391 | 392 | bool Util::IsCombiningCodePoint(std::uint32_t CodePoint) 393 | { 394 | return ((CodePoint >= 0x0300 && CodePoint <= 0x036F) || (CodePoint >= 0x1DC0 && CodePoint <= 0x1DFF) || (CodePoint >= 0x20D0 && CodePoint <= 0x20FF) || (CodePoint >= 0xFE20 && CodePoint <= 0xFE2F)); 395 | } 396 | } 397 | } 398 | -------------------------------------------------------------------------------- /utf8/utf8_util.h: -------------------------------------------------------------------------------- 1 | // Cross-platform UTF-8 string conversion and lightweight parser utilities. 2 | // (C) 2016 CubicleSoft. All Rights Reserved. 3 | 4 | #ifndef CUBICLESOFT_UTF8_UTIL 5 | #define CUBICLESOFT_UTF8_UTIL 6 | 7 | #include 8 | #include 9 | 10 | namespace CubicleSoft 11 | { 12 | namespace UTF8 13 | { 14 | class Util 15 | { 16 | public: 17 | // Converts wide "characters" (Unicode) to and from UTF-8. 18 | // Strict transformation without Unicode normalization. Some stream support. 19 | // Designed for OS/compiler "wide char" to/from UTF-8 transformations. 20 | // For example, SrcWidth and DestWidth might be sizeof(TCHAR) on Windows for a (TCHAR *) string. 21 | 22 | // Generally avoid these for small buffers. Small, exact memory allocations = bad. 23 | // Check out Sync::TLS for a higher-performance allocator. 24 | static char *ConvertToUTF8(const void *SrcData, size_t SrcWidth, size_t *LastPos = NULL, void *AltMallocManager = NULL, void *(*AltMalloc)(void *, size_t) = NULL); 25 | static void *ConvertFromUTF8(const char *SrcData, size_t DestWidth, void *AltMallocManager = NULL, void *(*AltMalloc)(void *, size_t) = NULL); 26 | 27 | // More efficient for small buffers. 28 | static void ConvertToUTF8(const void *SrcData, size_t SrcDataSize, size_t SrcWidth, std::uint8_t *DestData, size_t &DestDataSize, size_t *LastPos = NULL); 29 | static void ConvertFromUTF8(const std::uint8_t *SrcData, size_t SrcDataSize, void *DestData, size_t &DestDataSize, size_t DestWidth); 30 | 31 | 32 | // Very basic parser functions for UTF-8 strings. 33 | // Literally anything beyond this *requires* a (multi-)MB library such as ICU or Boost.Locale. 34 | // Unicode strings are ideally treated as purely opaque within an application. 35 | // Realistically, some parsing will still take place. 36 | 37 | // Does not count combining code points (good) but not all grapheme clusters such as Hangul (not as good). 38 | static size_t strlen(const char *Str); 39 | 40 | static bool FindCodePoint(size_t &ResultPos, std::uint32_t CodePoint, const char *Str, bool AllowCombiningCP = false); 41 | 42 | // Returns false on invalid characters. Use ConvertToUTF8 to clean up strings. 43 | // NOTE: Pos must be initialized to 0 for the first call or the function will assume a continuation. 44 | static bool NextCodePoint(std::uint32_t &ResultCP, std::uint32_t &NextCP, const char *Str, size_t &Pos, size_t &Size, bool &HasCombiningCP); 45 | 46 | // Finds strict ASCII characters <= 0x7F (no combining code points). 47 | // NOTE: Pos must be initialized to 0 for the first call or the function will assume a continuation. 48 | static bool NextASCIICodePoint(char &ResultCP, std::uint32_t &NextCP, const char *Str, size_t &Pos, size_t &Size); 49 | 50 | static bool IsCombiningCodePoint(std::uint32_t CodePoint); 51 | 52 | private: 53 | static bool AppendUTF8CodePoint(std::uint32_t TempCP, std::uint8_t *DestData, size_t &DestDataSize, size_t MaxDestDataSize); 54 | static bool AppendUTFCodePoint(std::uint32_t TempCP, void *DestData, size_t &DestDataSize, size_t DestWidth, size_t MaxDestDataSize); 55 | }; 56 | } 57 | } 58 | 59 | #endif --------------------------------------------------------------------------------