├── .circleci └── config.yml ├── .gitattributes ├── .github └── ISSUE_TEMPLATE │ └── bug_report.md ├── .gitignore ├── CMakeLists.txt ├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md ├── include └── xxhash.hpp └── test ├── CMakeLists.txt ├── catch.hpp ├── test_main.cpp ├── xxh3.h └── xxhash.h /.circleci/config.yml: -------------------------------------------------------------------------------- 1 | # Use the latest 2.1 version of CircleCI pipeline process engine. 2 | # See: https://circleci.com/docs/2.0/configuration-reference 3 | version: 2.1 4 | 5 | orbs: 6 | win: circleci/windows@4.1 7 | 8 | # Define a job to be invoked later in a workflow. 9 | # See: https://circleci.com/docs/2.0/configuration-reference/#jobs 10 | jobs: 11 | build-test: 12 | executor: win/server-2019 13 | steps: 14 | - checkout 15 | - run: choco install mingw -y 16 | - run: choco install cmake -y 17 | - run: mkdir build 18 | - run: cd build; $env:Path+=";$Env:ProgramFiles\CMake\bin"; cmake -DCMAKE_BUILD_TYPE=Release -DXXH_CPP_USE_AVX2=ON ..; cmake --build . --config Release; ctest --output-on-failure -C Release 19 | 20 | # Invoke jobs via workflows 21 | # See: https://circleci.com/docs/2.0/configuration-reference/#workflows 22 | workflows: 23 | say-hello-workflow: 24 | jobs: 25 | - build-test 26 | -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- 1 | ############################################################################### 2 | # Set default behavior to automatically normalize line endings. 3 | ############################################################################### 4 | * text=auto 5 | 6 | ############################################################################### 7 | # Set default behavior for command prompt diff. 8 | # 9 | # This is need for earlier builds of msysgit that does not have it on by 10 | # default for csharp files. 11 | # Note: This is only used by command line 12 | ############################################################################### 13 | #*.cs diff=csharp 14 | 15 | ############################################################################### 16 | # Set the merge driver for project and solution files 17 | # 18 | # Merging from the command prompt will add diff markers to the files if there 19 | # are conflicts (Merging from VS is not affected by the settings below, in VS 20 | # the diff markers are never inserted). Diff markers may cause the following 21 | # file extensions to fail to load in VS. An alternative would be to treat 22 | # these files as binary and thus will always conflict and require user 23 | # intervention with every merge. To do so, just uncomment the entries below 24 | ############################################################################### 25 | #*.sln merge=binary 26 | #*.csproj merge=binary 27 | #*.vbproj merge=binary 28 | #*.vcxproj merge=binary 29 | #*.vcproj merge=binary 30 | #*.dbproj merge=binary 31 | #*.fsproj merge=binary 32 | #*.lsproj merge=binary 33 | #*.wixproj merge=binary 34 | #*.modelproj merge=binary 35 | #*.sqlproj merge=binary 36 | #*.wwaproj merge=binary 37 | 38 | ############################################################################### 39 | # behavior for image files 40 | # 41 | # image files are treated as binary by default. 42 | ############################################################################### 43 | #*.jpg binary 44 | #*.png binary 45 | #*.gif binary 46 | 47 | ############################################################################### 48 | # diff behavior for common document formats 49 | # 50 | # Convert binary document formats to text before diffing them. This feature 51 | # is only available from the command line. Turn it on by uncommenting the 52 | # entries below. 53 | ############################################################################### 54 | #*.doc diff=astextplain 55 | #*.DOC diff=astextplain 56 | #*.docx diff=astextplain 57 | #*.DOCX diff=astextplain 58 | #*.dot diff=astextplain 59 | #*.DOT diff=astextplain 60 | #*.pdf diff=astextplain 61 | #*.PDF diff=astextplain 62 | #*.rtf diff=astextplain 63 | #*.RTF diff=astextplain 64 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | title: '' 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Describe the bug** 11 | A clear and concise description of what the bug is. 12 | 13 | **To Reproduce** 14 | Steps to reproduce the behavior: 15 | 1. Go to '...' 16 | 2. Click on '....' 17 | 3. Scroll down to '....' 18 | 4. See error 19 | 20 | **Expected behavior** 21 | A clear and concise description of what you expected to happen. 22 | 23 | **Screenshots** 24 | If applicable, add screenshots to help explain your problem. 25 | 26 | **Desktop (please complete the following information):** 27 | - OS: [e.g. iOS] 28 | - Browser [e.g. chrome, safari] 29 | - Version [e.g. 22] 30 | 31 | **Smartphone (please complete the following information):** 32 | - Device: [e.g. iPhone6] 33 | - OS: [e.g. iOS8.1] 34 | - Browser [e.g. stock browser, safari] 35 | - Version [e.g. 22] 36 | 37 | **Additional context** 38 | Add any other context about the problem here. 39 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | ## Ignore Visual Studio temporary files, build results, and 2 | ## files generated by popular Visual Studio add-ons. 3 | 4 | # User-specific files 5 | *.suo 6 | *.user 7 | *.userosscache 8 | *.sln.docstates 9 | 10 | # User-specific files (MonoDevelop/Xamarin Studio) 11 | *.userprefs 12 | 13 | # Build results 14 | [Dd]ebug/ 15 | [Dd]ebugPublic/ 16 | [Rr]elease/ 17 | [Rr]eleases/ 18 | x64/ 19 | x86/ 20 | bld/ 21 | [Bb]in/ 22 | [Oo]bj/ 23 | [Ll]og/ 24 | build/ 25 | 26 | # Visual Studio 2015 cache/options directory 27 | .vs/ 28 | # Uncomment if you have tasks that create the project's static files in wwwroot 29 | #wwwroot/ 30 | 31 | # MSTest test Results 32 | [Tt]est[Rr]esult*/ 33 | [Bb]uild[Ll]og.* 34 | 35 | # NUNIT 36 | *.VisualState.xml 37 | TestResult.xml 38 | 39 | # Build Results of an ATL Project 40 | [Dd]ebugPS/ 41 | [Rr]eleasePS/ 42 | dlldata.c 43 | 44 | # DNX 45 | project.lock.json 46 | project.fragment.lock.json 47 | artifacts/ 48 | 49 | *_i.c 50 | *_p.c 51 | *_i.h 52 | *.ilk 53 | *.meta 54 | *.obj 55 | *.pch 56 | *.pdb 57 | *.pgc 58 | *.pgd 59 | *.rsp 60 | *.sbr 61 | *.tlb 62 | *.tli 63 | *.tlh 64 | *.tmp 65 | *.tmp_proj 66 | *.log 67 | *.vspscc 68 | *.vssscc 69 | .builds 70 | *.pidb 71 | *.svclog 72 | *.scc 73 | 74 | # Chutzpah Test files 75 | _Chutzpah* 76 | 77 | # Visual C++ cache files 78 | ipch/ 79 | *.aps 80 | *.ncb 81 | *.opendb 82 | *.opensdf 83 | *.sdf 84 | *.cachefile 85 | *.VC.db 86 | *.VC.VC.opendb 87 | 88 | # Visual Studio profiler 89 | *.psess 90 | *.vsp 91 | *.vspx 92 | *.sap 93 | 94 | # TFS 2012 Local Workspace 95 | $tf/ 96 | 97 | # Guidance Automation Toolkit 98 | *.gpState 99 | 100 | # ReSharper is a .NET coding add-in 101 | _ReSharper*/ 102 | *.[Rr]e[Ss]harper 103 | *.DotSettings.user 104 | 105 | # JustCode is a .NET coding add-in 106 | .JustCode 107 | 108 | # TeamCity is a build add-in 109 | _TeamCity* 110 | 111 | # DotCover is a Code Coverage Tool 112 | *.dotCover 113 | 114 | # NCrunch 115 | _NCrunch_* 116 | .*crunch*.local.xml 117 | nCrunchTemp_* 118 | 119 | # MightyMoose 120 | *.mm.* 121 | AutoTest.Net/ 122 | 123 | # Web workbench (sass) 124 | .sass-cache/ 125 | 126 | # Installshield output folder 127 | [Ee]xpress/ 128 | 129 | # DocProject is a documentation generator add-in 130 | DocProject/buildhelp/ 131 | DocProject/Help/*.HxT 132 | DocProject/Help/*.HxC 133 | DocProject/Help/*.hhc 134 | DocProject/Help/*.hhk 135 | DocProject/Help/*.hhp 136 | DocProject/Help/Html2 137 | DocProject/Help/html 138 | 139 | # Click-Once directory 140 | publish/ 141 | 142 | # Publish Web Output 143 | *.[Pp]ublish.xml 144 | *.azurePubxml 145 | # TODO: Comment the next line if you want to checkin your web deploy settings 146 | # but database connection strings (with potential passwords) will be unencrypted 147 | #*.pubxml 148 | *.publishproj 149 | 150 | # Microsoft Azure Web App publish settings. Comment the next line if you want to 151 | # checkin your Azure Web App publish settings, but sensitive information contained 152 | # in these scripts will be unencrypted 153 | PublishScripts/ 154 | 155 | # NuGet Packages 156 | *.nupkg 157 | # The packages folder can be ignored because of Package Restore 158 | **/packages/* 159 | # except build/, which is used as an MSBuild target. 160 | !**/packages/build/ 161 | # Uncomment if necessary however generally it will be regenerated when needed 162 | #!**/packages/repositories.config 163 | # NuGet v3's project.json files produces more ignoreable files 164 | *.nuget.props 165 | *.nuget.targets 166 | 167 | # Microsoft Azure Build Output 168 | csx/ 169 | *.build.csdef 170 | 171 | # Microsoft Azure Emulator 172 | ecf/ 173 | rcf/ 174 | 175 | # Windows Store app package directories and files 176 | AppPackages/ 177 | BundleArtifacts/ 178 | Package.StoreAssociation.xml 179 | _pkginfo.txt 180 | 181 | # Visual Studio cache files 182 | # files ending in .cache can be ignored 183 | *.[Cc]ache 184 | # but keep track of directories ending in .cache 185 | !*.[Cc]ache/ 186 | 187 | # Others 188 | ClientBin/ 189 | ~$* 190 | *~ 191 | *.dbmdl 192 | *.dbproj.schemaview 193 | *.jfm 194 | *.pfx 195 | *.publishsettings 196 | node_modules/ 197 | orleans.codegen.cs 198 | 199 | # Since there are multiple workflows, uncomment next line to ignore bower_components 200 | # (https://github.com/github/gitignore/pull/1529#issuecomment-104372622) 201 | #bower_components/ 202 | 203 | # RIA/Silverlight projects 204 | Generated_Code/ 205 | 206 | # Backup & report files from converting an old project file 207 | # to a newer Visual Studio version. Backup files are not needed, 208 | # because we have git ;-) 209 | _UpgradeReport_Files/ 210 | Backup*/ 211 | UpgradeLog*.XML 212 | UpgradeLog*.htm 213 | 214 | # SQL Server files 215 | *.mdf 216 | *.ldf 217 | 218 | # Business Intelligence projects 219 | *.rdl.data 220 | *.bim.layout 221 | *.bim_*.settings 222 | 223 | # Microsoft Fakes 224 | FakesAssemblies/ 225 | 226 | # GhostDoc plugin setting file 227 | *.GhostDoc.xml 228 | 229 | # Node.js Tools for Visual Studio 230 | .ntvs_analysis.dat 231 | 232 | # Visual Studio 6 build log 233 | *.plg 234 | 235 | # Visual Studio 6 workspace options file 236 | *.opt 237 | 238 | # Visual Studio LightSwitch build output 239 | **/*.HTMLClient/GeneratedArtifacts 240 | **/*.DesktopClient/GeneratedArtifacts 241 | **/*.DesktopClient/ModelManifest.xml 242 | **/*.Server/GeneratedArtifacts 243 | **/*.Server/ModelManifest.xml 244 | _Pvt_Extensions 245 | 246 | # Paket dependency manager 247 | .paket/paket.exe 248 | paket-files/ 249 | 250 | # FAKE - F# Make 251 | .fake/ 252 | 253 | # JetBrains Rider 254 | .idea/ 255 | *.sln.iml 256 | 257 | # CodeRush 258 | .cr/ 259 | 260 | # Python Tools for Visual Studio (PTVS) 261 | __pycache__/ 262 | *.pyc 263 | 264 | # Test files 265 | *.o 266 | *.exe 267 | *.vcxproj 268 | *.vcxproj.filters 269 | *.cmake 270 | *.cmake.in 271 | *.sln 272 | /xxhash 273 | /xxhash_dev 274 | /build 275 | /xxhash_dev 276 | *.tcl 277 | CMakeCache.txt 278 | /test/CMakeFiles 279 | /CMakeFiles -------------------------------------------------------------------------------- /CMakeLists.txt: -------------------------------------------------------------------------------- 1 | cmake_minimum_required(VERSION 3.8) 2 | 3 | project(xxhash_cpp 4 | VERSION 0.8.1 5 | LANGUAGES CXX 6 | DESCRIPTION "C++ port of the xxhash library." 7 | HOMEPAGE_URL "https://github.com/RedSpah/xxhash_cpp") 8 | 9 | include(GNUInstallDirs) 10 | 11 | add_library(${PROJECT_NAME} INTERFACE) 12 | 13 | target_compile_features(${PROJECT_NAME} INTERFACE cxx_std_17) 14 | 15 | target_include_directories(${PROJECT_NAME} INTERFACE 16 | $ 17 | $) 18 | 19 | if(CMAKE_SYSTEM_PROCESSOR MATCHES "arm64" AND APPLE) 20 | find_path(SSE2NEON_HEADER sse2neon.h PATHS ${CMAKE_SOURCE_DIR}/../sse2neon) 21 | target_include_directories(${PROJECT_NAME} INTERFACE ${SSE2NEON_HEADER}) 22 | endif() 23 | 24 | if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR) 25 | include(CTest) 26 | endif() 27 | 28 | if(BUILD_TESTING) 29 | add_subdirectory(test) 30 | endif() 31 | 32 | install(TARGETS ${PROJECT_NAME} 33 | EXPORT ${PROJECT_NAME}_Targets 34 | ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} 35 | LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} 36 | RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}) 37 | 38 | include(CMakePackageConfigHelpers) 39 | 40 | write_basic_package_version_file("${PROJECT_NAME}ConfigVersion.cmake" 41 | VERSION ${PROJECT_VERSION} 42 | COMPATIBILITY ExactVersion) 43 | 44 | file(WRITE ${PROJECT_BINARY_DIR}/${PROJECT_NAME}Config.cmake.in 45 | "@PACKAGE_INIT@\n" 46 | "include(\"\${CMAKE_CURRENT_LIST_DIR}/@PROJECT_NAME@Targets.cmake\")\n" 47 | "check_required_components(\"@PROJECT_NAME@\")\n" 48 | ) 49 | 50 | configure_package_config_file( 51 | "${PROJECT_BINARY_DIR}/${PROJECT_NAME}Config.cmake.in" 52 | "${PROJECT_BINARY_DIR}/${PROJECT_NAME}Config.cmake" 53 | INSTALL_DESTINATION 54 | ${CMAKE_INSTALL_DATAROOTDIR}/${PROJECT_NAME}/cmake) 55 | 56 | install(EXPORT ${PROJECT_NAME}_Targets 57 | FILE ${PROJECT_NAME}Targets.cmake 58 | NAMESPACE ${PROJECT_NAME}:: 59 | DESTINATION ${CMAKE_INSTALL_DATAROOTDIR}/${PROJECT_NAME}/cmake) 60 | 61 | install(FILES "${PROJECT_BINARY_DIR}/${PROJECT_NAME}Config.cmake" 62 | "${PROJECT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake" 63 | DESTINATION ${CMAKE_INSTALL_DATAROOTDIR}/${PROJECT_NAME}/cmake) 64 | 65 | install(DIRECTORY ${PROJECT_SOURCE_DIR}/include DESTINATION include) 66 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, sex characteristics, gender identity and expression, 9 | level of experience, education, socio-economic status, nationality, personal 10 | appearance, race, religion, or sexual identity and orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the project team at redspah@gmail.com. All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 71 | available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html 72 | 73 | [homepage]: https://www.contributor-covenant.org 74 | 75 | For answers to common questions about this code of conduct, see 76 | https://www.contributor-covenant.org/faq 77 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 2-Clause License 2 | 3 | Copyright (c) 2012-2020, Yann Collet 4 | Copyright (c) 2017-2020, Red Gavin 5 | All rights reserved. 6 | 7 | Redistribution and use in source and binary forms, with or without 8 | modification, are permitted provided that the following conditions are met: 9 | 10 | * Redistributions of source code must retain the above copyright notice, this 11 | list of conditions and the following disclaimer. 12 | 13 | * Redistributions in binary form must reproduce the above copyright notice, 14 | this list of conditions and the following disclaimer in the documentation 15 | and/or other materials provided with the distribution. 16 | 17 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 18 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 20 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 21 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 23 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 24 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 25 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 26 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 27 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # xxhash_cpp 2 | Port of the xxHash library to C++17. 3 | 4 | [![CircleCI](https://dl.circleci.com/status-badge/img/gh/RedSpah/xxhash_cpp/tree/master.svg?style=shield)](https://dl.circleci.com/status-badge/redirect/gh/RedSpah/xxhash_cpp/tree/master) 5 | 6 | Compatibility 7 | ---- 8 | | Compiler | Min. Version | 9 | |----------------------|:-------------------:| 10 | | MSVC (Visual Studio) | 19.1 (VS 2017.3 P2) | 11 | | clang | 3.9 | 12 | | gcc | 7 | 13 | 14 | Example Usage 15 | ---- 16 | 17 | ```cpp 18 | // standalone hash 19 | std::array input {322, 2137, 42069, 65536}; 20 | xxh::hash_t<32> hash = xxh::xxhash<32>(input); 21 | 22 | // hash streaming 23 | std::array buffer; 24 | xxh::hash_state_t<64> hash_stream; 25 | while (fill_buffer(buffer)) 26 | { 27 | hash_stream.update(buffer); 28 | } 29 | xxh::hash_t<64> final_hash = hash_stream.digest(); 30 | ``` 31 | The template argument specifies whether the algorithm will use the 32 or 64 bit version. Other values are not allowed. Typedefs `hash32_t`, `hash64_t`, `hash_state32_t` and `hash_state64_t` are provided. 32 | 33 | `xxh::xxhash` and `xxh::hash_state_t::update` provide several convenient overloads, all accepting optional `seed` and `endianness` arguments: 34 | * C-style `const void*` + `size_t` pair 35 | * `const std::vector&` 36 | * `const std::basic_string&` 37 | * A pair of templated iterators 38 | * `const std::array&` 39 | * `const std::initializer_list&` 40 | 41 | Build Instructions 42 | ---- 43 | 44 | The library is provided as a single standalone header, for static linking only. No build instructions are nessessary. 45 | 46 | 47 | xxHash - Extremely fast hash algorithm 48 | ====================================== 49 | 50 | xxHash is an Extremely fast Hash algorithm, running at RAM speed limits. 51 | It successfully completes the [SMHasher](http://code.google.com/p/smhasher/wiki/SMHasher) test suite 52 | which evaluates collision, dispersion and randomness qualities of hash functions. 53 | Code is highly portable, and hashes are identical on all platforms (little / big endian). 54 | 55 | 56 | Benchmarks 57 | ------------------------- 58 | 59 | The benchmark uses SMHasher speed test, compiled with Visual 2010 on a Windows Seven 32-bits box. 60 | The reference system uses a Core 2 Duo @3GHz 61 | 62 | 63 | | Name | Speed | Quality | Author | 64 | |---------------|----------|:-------:|------------------| 65 | | [xxHash] | 5.4 GB/s | 10 | Y.C. | 66 | | MurmurHash 3a | 2.7 GB/s | 10 | Austin Appleby | 67 | | SBox | 1.4 GB/s | 9 | Bret Mulvey | 68 | | Lookup3 | 1.2 GB/s | 9 | Bob Jenkins | 69 | | CityHash64 | 1.05 GB/s| 10 | Pike & Alakuijala| 70 | | FNV | 0.55 GB/s| 5 | Fowler, Noll, Vo | 71 | | CRC32 | 0.43 GB/s| 9 | | 72 | | MD5-32 | 0.33 GB/s| 10 | Ronald L.Rivest | 73 | | SHA1-32 | 0.28 GB/s| 10 | | 74 | 75 | [xxHash]: http://www.xxhash.com 76 | 77 | Q.Score is a measure of quality of the hash function. 78 | It depends on successfully passing SMHasher test set. 79 | 10 is a perfect score. 80 | Algorithms with a score < 5 are not listed on this table. 81 | 82 | A more recent version, XXH64, has been created thanks to [Mathias Westerdahl](https://github.com/JCash), 83 | which offers superior speed and dispersion for 64-bits systems. 84 | Note however that 32-bits applications will still run faster using the 32-bits version. 85 | 86 | SMHasher speed test, compiled using GCC 4.8.2, on Linux Mint 64-bits. 87 | The reference system uses a Core i5-3340M @2.7GHz 88 | 89 | | Version | Speed on 64-bits | Speed on 32-bits | 90 | |------------|------------------|------------------| 91 | | XXH64 | 13.8 GB/s | 1.9 GB/s | 92 | | XXH32 | 6.8 GB/s | 6.0 GB/s | 93 | 94 | ### License 95 | 96 | The library file `xxhash.hpp` is BSD licensed. 97 | 98 | 99 | ### Build modifiers 100 | 101 | The following macros influence xxhash behavior. They are all disabled by default. 102 | 103 | - `XXH_FORCE_NATIVE_FORMAT` : on big-endian systems : use native number representation, 104 | resulting in system-specific results. 105 | Breaks consistency with little-endian results. 106 | 107 | - `XXH_CPU_LITTLE_ENDIAN` : if defined to 0, sets the native endianness to big endian, 108 | if defined to 1, sets the native endianness to little endian, 109 | if left undefined, the endianness is resolved at runtime, 110 | before `main` is called, at the cost of endianness not being `constexpr`. 111 | 112 | - `XXH_FORCE_MEMORY_ACCESS` : if defined to 2, enables unaligned reads as an optimization, this is not standard compliant, 113 | if defined to 1, enables the use of `packed` attribute for optimization, only defined for gcc and icc 114 | otherwise, uses the default fallback method (`memcpy`) 115 | 116 | ### Other languages 117 | 118 | Beyond the C reference version, 119 | xxHash is also available on many programming languages, 120 | thanks to great contributors. 121 | They are [listed here](http://www.xxhash.com/#other-languages). 122 | -------------------------------------------------------------------------------- /include/xxhash.hpp: -------------------------------------------------------------------------------- 1 | #pragma once 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | /* 10 | xxHash - Extremely Fast Hash algorithm 11 | Header File 12 | Copyright (C) 2012-2024, Yann Collet. 13 | Copyright (C) 2017-2024, Red Gavin. 14 | All rights reserved. 15 | 16 | BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) 17 | Redistribution and use in source and binary forms, with or without 18 | modification, are permitted provided that the following conditions are 19 | met: 20 | * Redistributions of source code must retain the above copyright 21 | notice, this list of conditions and the following disclaimer. 22 | * Redistributions in binary form must reproduce the above 23 | copyright notice, this list of conditions and the following disclaimer 24 | in the documentation and/or other materials provided with the 25 | distribution. 26 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 27 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 28 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 29 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 30 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 31 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 32 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 33 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 34 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 35 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 36 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 37 | You can contact the author at : 38 | - xxHash source repository : https://github.com/Cyan4973/xxHash 39 | - xxHash C++ port repository : https://github.com/RedSpah/xxhash_cpp 40 | */ 41 | 42 | /* Intrinsics 43 | * Sadly has to be included in the global namespace or literally everything breaks 44 | */ 45 | #if (defined(__ARM_NEON) && defined(__APPLE__)) 46 | #include "sse2neon.h" 47 | #else 48 | #include 49 | #endif 50 | 51 | namespace xxh 52 | { 53 | /* ************************************* 54 | * Versioning 55 | ***************************************/ 56 | 57 | namespace version 58 | { 59 | constexpr int cpp_version_major = 0; 60 | constexpr int cpp_version_minor = 8; 61 | constexpr int cpp_version_release = 1; 62 | } 63 | 64 | constexpr uint32_t version_number() 65 | { 66 | return version::cpp_version_major * 10000 + version::cpp_version_minor * 100 + version::cpp_version_release; 67 | } 68 | 69 | 70 | /* ************************************* 71 | * Basic Types - Predefining uint128_t for intrin 72 | ***************************************/ 73 | 74 | namespace typedefs 75 | { 76 | struct alignas(16) uint128_t 77 | { 78 | uint64_t low64 = 0; 79 | uint64_t high64 = 0; 80 | 81 | bool operator==(const uint128_t & other) 82 | { 83 | return (low64 == other.low64 && high64 == other.high64); 84 | } 85 | 86 | bool operator>(const uint128_t & other) 87 | { 88 | return (high64 > other.high64 || low64 > other.low64); 89 | } 90 | 91 | bool operator>=(const uint128_t & other) 92 | { 93 | return (*this > other || *this == other); 94 | } 95 | 96 | bool operator<(const uint128_t & other) 97 | { 98 | return !(*this >= other); 99 | } 100 | 101 | bool operator<=(const uint128_t & other) 102 | { 103 | return !(*this > other); 104 | } 105 | 106 | bool operator!=(const uint128_t & other) 107 | { 108 | return !(*this == other); 109 | } 110 | 111 | uint128_t(uint64_t low, uint64_t high) : low64(low), high64(high) {} 112 | 113 | uint128_t() {} 114 | }; 115 | 116 | } 117 | 118 | using uint128_t = typedefs::uint128_t; 119 | 120 | 121 | /* ************************************* 122 | * Compiler / Platform Specific Features 123 | ***************************************/ 124 | 125 | namespace intrin 126 | { 127 | /*!XXH_CPU_LITTLE_ENDIAN : 128 | * This is a CPU endian detection macro, will be 129 | * automatically set to 1 (little endian) if it is left undefined. 130 | * If compiling for a big endian system (why), XXH_CPU_LITTLE_ENDIAN has to be explicitly defined as 0. 131 | */ 132 | #ifndef XXH_CPU_LITTLE_ENDIAN 133 | # define XXH_CPU_LITTLE_ENDIAN 1 134 | #endif 135 | 136 | 137 | /* Vectorization Detection 138 | * NOTE: XXH_NEON and XXH_VSX aren't supported in this C++ port. 139 | * The primary reason is that I don't have access to an ARM and PowerPC 140 | * machines to test them, and the secondary reason is that I even doubt anyone writing 141 | * code for such machines would bother using a C++ port rather than the original C version. 142 | */ 143 | #ifndef XXH_VECTOR /* can be predefined on command line */ 144 | # if defined(__AVX512F__) 145 | # define XXH_VECTOR 3 /* AVX512 for Skylake and Icelake */ 146 | # elif defined(__AVX2__) 147 | # define XXH_VECTOR 2 /* AVX2 for Haswell and Bulldozer */ 148 | # elif defined(__SSE2__) || defined(_M_AMD64) || defined(_M_X64) || (defined(_M_IX86_FP) && (_M_IX86_FP == 2)) 149 | # define XXH_VECTOR 1 /* SSE2 for Pentium 4 and all x86_64 */ 150 | # else 151 | # define XXH_VECTOR 0 /* Portable scalar version */ 152 | # endif 153 | #endif 154 | 155 | constexpr int vector_mode = XXH_VECTOR; 156 | 157 | #if XXH_VECTOR == 3 /* AVX512 for Skylake and Icelake */ 158 | constexpr int acc_align = 64; 159 | using avx512_underlying = __m512i; 160 | using avx2_underlying = __m256i; 161 | using sse2_underlying = __m128i; 162 | #elif XXH_VECTOR == 2 /* AVX2 for Haswell and Bulldozer */ 163 | constexpr int acc_align = 32; 164 | using avx512_underlying = void; 165 | using avx2_underlying = __m256i; 166 | using sse2_underlying = __m128i; 167 | #elif XXH_VECTOR == 1 /* SSE2 for Pentium 4 and all x86_64 */ 168 | using avx512_underlying = void; 169 | using avx2_underlying = void; //std::array<__m128i, 2>; 170 | using sse2_underlying = __m128i; 171 | constexpr int acc_align = 16; 172 | #else /* Portable scalar version */ 173 | using avx512_underlying = void; 174 | using avx2_underlying = void; //std::array; 175 | using sse2_underlying = void; //std::array; 176 | constexpr int acc_align = 8; 177 | #endif 178 | 179 | 180 | /* Compiler Specifics 181 | * Defines inline macros and includes specific compiler's instrinsics. 182 | * */ 183 | #ifdef XXH_FORCE_INLINE /* First undefining the symbols in case they're already defined */ 184 | # undef XXH_FORCE_INLINE 185 | #endif 186 | #ifdef XXH_NO_INLINE 187 | # undef XXH_NO_INLINE 188 | #endif 189 | 190 | #ifdef _MSC_VER /* Visual Studio */ 191 | # pragma warning(disable : 4127) 192 | # define XXH_FORCE_INLINE static __forceinline 193 | # define XXH_NO_INLINE static __declspec(noinline) 194 | # include 195 | #elif defined(__GNUC__) /* Clang / GCC */ 196 | # define XXH_FORCE_INLINE static inline __attribute__((always_inline)) 197 | # define XXH_NO_INLINE static __attribute__((noinline)) 198 | #if (defined(__ARM_NEON) && defined(__APPLE__)) 199 | # include "sse2neon.h" 200 | # else 201 | # include 202 | # endif 203 | #else 204 | # define XXH_FORCE_INLINE static inline 205 | # define XXH_NO_INLINE static 206 | #endif 207 | 208 | 209 | /* Prefetch 210 | * Can be disabled by defining XXH_NO_PREFETCH 211 | */ 212 | #if defined(XXH_NO_PREFETCH) 213 | XXH_FORCE_INLINE void prefetch(const void* ptr) {} 214 | #elif defined(_MSC_VER) && (defined(_M_X64) || defined(_M_IX86)) 215 | XXH_FORCE_INLINE void prefetch(const void* ptr) { _mm_prefetch((const char*)(ptr), _MM_HINT_T0); } 216 | #elif defined(__GNUC__) 217 | XXH_FORCE_INLINE void prefetch(const void* ptr) { __builtin_prefetch((ptr), 0, 3); } 218 | #else 219 | XXH_FORCE_INLINE void prefetch(const void* ptr) {} 220 | #endif 221 | 222 | 223 | /* Restrict 224 | * Defines macro for restrict, which in C++ is sadly just a compiler extension (for now). 225 | * Can be disabled by defining XXH_NO_RESTRICT 226 | */ 227 | #ifdef XXH_RESTRICT 228 | # undef XXH_RESTRICT 229 | #endif 230 | 231 | #if (defined(__GNUC__) || defined(_MSC_VER)) && defined(__cplusplus) && !defined(XXH_NO_RESTRICT) 232 | # define XXH_RESTRICT __restrict 233 | #else 234 | # define XXH_RESTRICT 235 | #endif 236 | 237 | 238 | /* Likely / Unlikely 239 | * Defines macros for Likely / Unlikely, which are official in C++20, but sadly this library aims the previous standard. 240 | * Not present on MSVC. 241 | * Can be disabled by defining XXH_NO_BRANCH_HINTS 242 | */ 243 | #if ((defined(__GNUC__) && (__GNUC__ >= 3)) || (defined(__INTEL_COMPILER) && (__INTEL_COMPILER >= 800)) || defined(__clang__)) && !defined(XXH_NO_BRANCH_HINTS) 244 | # define XXH_likely(x) __builtin_expect(x, 1) 245 | # define XXH_unlikely(x) __builtin_expect(x, 0) 246 | #else 247 | # define XXH_likely(x) (x) 248 | # define XXH_unlikely(x) (x) 249 | #endif 250 | 251 | /* _MM_PERM_ENUM type 252 | * Defines the type for the parameters of the shuffle function, in a way that works with -fpedantic while not breaking non-x86 compatibility. 253 | */ 254 | #if defined(__i386__) || defined(__x86_64__) || defined(_M_X64) 255 | using mm_perm_enum_t = _MM_PERM_ENUM; 256 | #else 257 | using mm_perm_enum_t = int; 258 | #endif 259 | 260 | 261 | namespace bit_ops 262 | { 263 | #if defined(_MSC_VER) 264 | static inline uint32_t rotl32(uint32_t x, int32_t r) { return _rotl(x, r); } 265 | static inline uint64_t rotl64(uint64_t x, int32_t r) { return _rotl64(x, r); } 266 | static inline uint32_t rotr32(uint32_t x, int32_t r) { return _rotr(x, r); } 267 | static inline uint64_t rotr64(uint64_t x, int32_t r) { return _rotr64(x, r); } 268 | #else 269 | static inline uint32_t rotl32(uint32_t x, int32_t r) { return ((x << r) | (x >> (32 - r))); } 270 | static inline uint64_t rotl64(uint64_t x, int32_t r) { return ((x << r) | (x >> (64 - r))); } 271 | static inline uint32_t rotr32(uint32_t x, int32_t r) { return ((x >> r) | (x << (32 - r))); } 272 | static inline uint64_t rotr64(uint64_t x, int32_t r) { return ((x >> r) | (x << (64 - r))); } 273 | #endif 274 | 275 | 276 | #if defined(_MSC_VER) /* Visual Studio */ 277 | static inline uint32_t swap32(uint32_t x) { return _byteswap_ulong(x); } 278 | static inline uint64_t swap64(uint64_t x) { return _byteswap_uint64(x); } 279 | #elif defined(__GNUC__) 280 | static inline uint32_t swap32(uint32_t x) { return __builtin_bswap32(x); } 281 | static inline uint64_t swap64(uint64_t x) { return __builtin_bswap64(x); } 282 | #else 283 | static inline uint32_t swap32(uint32_t x) { return ((x << 24) & 0xff000000) | ((x << 8) & 0x00ff0000) | ((x >> 8) & 0x0000ff00) | ((x >> 24) & 0x000000ff); } 284 | static inline uint64_t swap64(uint64_t x) { return ((x << 56) & 0xff00000000000000ULL) | ((x << 40) & 0x00ff000000000000ULL) | ((x << 24) & 0x0000ff0000000000ULL) | ((x << 8) & 0x000000ff00000000ULL) | ((x >> 8) & 0x00000000ff000000ULL) | ((x >> 24) & 0x0000000000ff0000ULL) | ((x >> 40) & 0x000000000000ff00ULL) | ((x >> 56) & 0x00000000000000ffULL); } 285 | #endif 286 | 287 | 288 | #if defined(_MSC_VER) && defined(_M_IX86) // Only for 32-bit MSVC. 289 | XXH_FORCE_INLINE uint64_t mult32to64(uint32_t x, uint32_t y) { return __emulu(x, y); } 290 | #else 291 | XXH_FORCE_INLINE uint64_t mult32to64(uint32_t x, uint32_t y) { return (uint64_t)(uint32_t)(x) * (uint64_t)(uint32_t)(y); } 292 | #endif 293 | 294 | 295 | #if defined(__GNUC__) && !defined(__clang__) && defined(__i386__) 296 | __attribute__((__target__("no-sse"))) 297 | #endif 298 | static inline uint128_t mult64to128(uint64_t lhs, uint64_t rhs) 299 | { 300 | 301 | #if defined(__GNUC__) && !defined(__wasm__) \ 302 | && defined(__SIZEOF_INT128__) \ 303 | || (defined(_INTEGRAL_MAX_BITS) && _INTEGRAL_MAX_BITS >= 128) 304 | 305 | __uint128_t product = (__uint128_t)lhs * (__uint128_t)rhs; 306 | uint128_t r128; 307 | r128.low64 = (uint64_t)(product); 308 | r128.high64 = (uint64_t)(product >> 64); 309 | return r128; 310 | 311 | #elif defined(_M_X64) || defined(_M_IA64) 312 | 313 | #ifndef _MSC_VER 314 | # pragma intrinsic(_umul128) 315 | #endif 316 | uint64_t product_high; 317 | uint64_t const product_low = _umul128(lhs, rhs, &product_high); 318 | uint128_t r128; 319 | r128.low64 = product_low; 320 | r128.high64 = product_high; 321 | return r128; 322 | 323 | #else 324 | uint64_t const lo_lo = bit_ops::mult32to64(lhs & 0xFFFFFFFF, rhs & 0xFFFFFFFF); 325 | uint64_t const hi_lo = bit_ops::mult32to64(lhs >> 32, rhs & 0xFFFFFFFF); 326 | uint64_t const lo_hi = bit_ops::mult32to64(lhs & 0xFFFFFFFF, rhs >> 32); 327 | uint64_t const hi_hi = bit_ops::mult32to64(lhs >> 32, rhs >> 32); 328 | 329 | /* Now add the products together. These will never overflow. */ 330 | uint64_t const cross = (lo_lo >> 32) + (hi_lo & 0xFFFFFFFF) + lo_hi; 331 | uint64_t const upper = (hi_lo >> 32) + (cross >> 32) + hi_hi; 332 | uint64_t const lower = (cross << 32) | (lo_lo & 0xFFFFFFFF); 333 | 334 | uint128_t r128; 335 | r128.low64 = lower; 336 | r128.high64 = upper; 337 | return r128; 338 | #endif 339 | } 340 | } 341 | } 342 | 343 | 344 | /* ************************************* 345 | * Basic Types - Everything else 346 | ***************************************/ 347 | 348 | namespace typedefs 349 | { 350 | /* ************************************* 351 | * Basic Types - Detail 352 | ***************************************/ 353 | 354 | template 355 | struct hash_type 356 | { 357 | using type = void; 358 | }; 359 | 360 | template <> 361 | struct hash_type<32> 362 | { 363 | using type = uint32_t; 364 | }; 365 | 366 | template <> 367 | struct hash_type<64> 368 | { 369 | using type = uint64_t; 370 | }; 371 | 372 | template <> 373 | struct hash_type<128> 374 | { 375 | using type = uint128_t; 376 | }; 377 | 378 | 379 | template 380 | struct vec_type 381 | { 382 | using type = void; 383 | }; 384 | 385 | template <> 386 | struct vec_type<64> 387 | { 388 | using type = uint64_t; 389 | }; 390 | 391 | template <> 392 | struct vec_type<128> 393 | { 394 | using type = intrin::sse2_underlying; 395 | }; 396 | 397 | template <> 398 | struct vec_type<256> 399 | { 400 | using type = intrin::avx2_underlying; 401 | }; 402 | 403 | template <> 404 | struct vec_type<512> 405 | { 406 | using type = intrin::avx512_underlying; 407 | }; 408 | 409 | /* Rationale 410 | * On the surface level uint_type appears to be pointless, 411 | * as it is just a copy of hash_type. They do use the same types, 412 | * that is true, but the reasoning for the difference is aimed at humans, 413 | * not the compiler, as a difference between values that are 'just' numbers, 414 | * and those that represent actual hash values. 415 | */ 416 | template 417 | struct uint_type 418 | { 419 | using type = void; 420 | }; 421 | 422 | template <> 423 | struct uint_type<32> 424 | { 425 | using type = uint32_t; 426 | }; 427 | 428 | template <> 429 | struct uint_type<64> 430 | { 431 | using type = uint64_t; 432 | }; 433 | 434 | template <> 435 | struct uint_type<128> 436 | { 437 | using type = uint128_t; 438 | }; 439 | } 440 | 441 | template 442 | using hash_t = typename typedefs::hash_type::type; 443 | using hash32_t = hash_t<32>; 444 | using hash64_t = hash_t<64>; 445 | using hash128_t = hash_t<128>; 446 | 447 | template 448 | using vec_t = typename typedefs::vec_type::type; 449 | using vec64_t = vec_t<64>; 450 | using vec128_t = vec_t<128>; 451 | using vec256_t = vec_t<256>; 452 | using vec512_t = vec_t<512>; 453 | 454 | template 455 | using uint_t = typename typedefs::uint_type::type; 456 | 457 | 458 | 459 | /* ************************************* 460 | * Bit Operations 461 | ***************************************/ 462 | 463 | namespace bit_ops 464 | { 465 | /* **************************************** 466 | * Bit Operations 467 | ******************************************/ 468 | 469 | template 470 | static inline uint_t rotl(uint_t n, int32_t r) 471 | { 472 | if constexpr (N == 32) 473 | { 474 | return intrin::bit_ops::rotl32(n, r); 475 | } 476 | 477 | if constexpr (N == 64) 478 | { 479 | return intrin::bit_ops::rotl64(n, r); 480 | } 481 | } 482 | 483 | template 484 | static inline uint_t rotr(uint_t n, int32_t r) 485 | { 486 | if constexpr (N == 32) 487 | { 488 | return intrin::bit_ops::rotr32(n, r); 489 | } 490 | 491 | if constexpr (N == 64) 492 | { 493 | return intrin::bit_ops::rotr64(n, r); 494 | } 495 | } 496 | 497 | template 498 | static inline uint_t swap(uint_t n) 499 | { 500 | if constexpr (N == 32) 501 | { 502 | return intrin::bit_ops::swap32(n); 503 | } 504 | 505 | if constexpr (N == 64) 506 | { 507 | return intrin::bit_ops::swap64(n); 508 | } 509 | } 510 | 511 | template 512 | static inline vec_t mul32to64(vec_t x, vec_t y) 513 | { 514 | if constexpr (N == 64) 515 | { 516 | return intrin::bit_ops::mult32to64(static_cast(x), static_cast(y)); 517 | } 518 | else 519 | { 520 | return 0; 521 | } 522 | } 523 | 524 | static inline uint128_t mul64to128(uint64_t x, uint64_t y) 525 | { 526 | return intrin::bit_ops::mult64to128(x, y); 527 | } 528 | 529 | static inline uint64_t mul128fold64(uint64_t x, uint64_t y) 530 | { 531 | uint128_t product = mul64to128(x, y); 532 | 533 | return (product.low64 ^ product.high64); 534 | } 535 | } 536 | 537 | 538 | /* ************************************* 539 | * Memory Functions 540 | ***************************************/ 541 | 542 | namespace mem_ops 543 | { 544 | 545 | /* ************************************* 546 | * Endianness 547 | ***************************************/ 548 | 549 | constexpr bool is_little_endian() 550 | { 551 | return (XXH_CPU_LITTLE_ENDIAN == 1); 552 | } 553 | 554 | 555 | /* ************************************* 556 | * Memory Access 557 | ***************************************/ 558 | 559 | template 560 | static inline uint_t read(const void* memPtr) 561 | { 562 | uint_t val; 563 | 564 | memcpy(&val, memPtr, sizeof(val)); 565 | return val; 566 | } 567 | 568 | template 569 | static inline uint_t readLE(const void* ptr) 570 | { 571 | if constexpr (is_little_endian()) 572 | { 573 | return read(ptr); 574 | } 575 | else 576 | { 577 | return bit_ops::swap(read(ptr)); 578 | } 579 | } 580 | 581 | template 582 | static inline uint_t readBE(const void* ptr) 583 | { 584 | if constexpr (is_little_endian()) 585 | { 586 | return bit_ops::swap(read(ptr)); 587 | } 588 | else 589 | { 590 | return read(ptr); 591 | } 592 | } 593 | 594 | template 595 | static void writeLE(void* dst, uint_t v) 596 | { 597 | if constexpr (!is_little_endian()) 598 | { 599 | v = bit_ops::swap(v); 600 | } 601 | 602 | memcpy(dst, &v, sizeof(v)); 603 | } 604 | } 605 | 606 | 607 | /* ************************************* 608 | * Vector Functions 609 | ***************************************/ 610 | 611 | namespace vec_ops 612 | { 613 | template 614 | XXH_FORCE_INLINE vec_t loadu(const vec_t* input) 615 | { 616 | static_assert(!(N != 128 && N != 256 && N != 64 && N != 512), "Invalid template argument passed to xxh::vec_ops::loadu"); 617 | 618 | if constexpr (N == 128) 619 | { 620 | return _mm_loadu_si128(input); 621 | } 622 | 623 | if constexpr (N == 256) 624 | { 625 | return _mm256_loadu_si256(input); 626 | } 627 | 628 | if constexpr (N == 512) 629 | { 630 | return _mm512_loadu_si512(input); 631 | } 632 | 633 | if constexpr (N == 64) 634 | { 635 | return mem_ops::readLE<64>(input); 636 | } 637 | 638 | } 639 | 640 | 641 | // 'xorv' instead of 'xor' because 'xor' is a weird wacky alternate operator expression thing. 642 | template 643 | XXH_FORCE_INLINE vec_t xorv(vec_t a, vec_t b) 644 | { 645 | static_assert(!(N != 128 && N != 256 && N != 64 && N != 512), "Invalid argument passed to xxh::vec_ops::xorv"); 646 | 647 | if constexpr (N == 128) 648 | { 649 | return _mm_xor_si128(a, b); 650 | } 651 | 652 | if constexpr (N == 256) 653 | { 654 | return _mm256_xor_si256(a, b); 655 | } 656 | 657 | if constexpr (N == 512) 658 | { 659 | return _mm512_xor_si512(a, b); 660 | } 661 | 662 | if constexpr (N == 64) 663 | { 664 | return a ^ b; 665 | } 666 | } 667 | 668 | 669 | template 670 | XXH_FORCE_INLINE vec_t mul(vec_t a, vec_t b) 671 | { 672 | static_assert(!(N != 128 && N != 256 && N != 64 && N != 512), "Invalid argument passed to xxh::vec_ops::mul"); 673 | 674 | if constexpr (N == 128) 675 | { 676 | return _mm_mul_epu32(a, b); 677 | } 678 | 679 | if constexpr (N == 256) 680 | { 681 | return _mm256_mul_epu32(a, b); 682 | } 683 | 684 | if constexpr (N == 512) 685 | { 686 | return _mm512_mul_epu32(a, b); 687 | } 688 | 689 | if constexpr (N == 64) 690 | { 691 | return a * b; 692 | } 693 | } 694 | 695 | 696 | template 697 | XXH_FORCE_INLINE vec_t add(vec_t a, vec_t b) 698 | { 699 | static_assert(!(N != 128 && N != 256 && N != 64 && N != 512), "Invalid argument passed to xxh::vec_ops::add"); 700 | 701 | if constexpr (N == 128) 702 | { 703 | return _mm_add_epi64(a, b); 704 | } 705 | 706 | if constexpr (N == 256) 707 | { 708 | return _mm256_add_epi64(a, b); 709 | } 710 | 711 | if constexpr (N == 512) 712 | { 713 | return _mm512_add_epi64(a, b); 714 | } 715 | 716 | if constexpr (N == 64) 717 | { 718 | return a + b; 719 | } 720 | } 721 | 722 | 723 | template 724 | XXH_FORCE_INLINE vec_t shuffle(vec_t a) 725 | { 726 | static_assert(!(N != 128 && N != 256 && N != 64 && N != 512), "Invalid argument passed to xxh::vec_ops::shuffle"); 727 | 728 | if constexpr (N == 128) 729 | { 730 | return _mm_shuffle_epi32(a, static_cast(_MM_SHUFFLE(S1, S2, S3, S4))); 731 | } 732 | 733 | if constexpr (N == 256) 734 | { 735 | return _mm256_shuffle_epi32(a, static_cast(_MM_SHUFFLE(S1, S2, S3, S4))); 736 | } 737 | 738 | if constexpr (N == 512) 739 | { 740 | return _mm512_shuffle_epi32(a, static_cast(_MM_SHUFFLE(S1, S2, S3, S4))); 741 | } 742 | 743 | if constexpr (N == 64) 744 | { 745 | return a; 746 | } 747 | } 748 | 749 | 750 | template 751 | XXH_FORCE_INLINE vec_t set1(int64_t a) 752 | { 753 | 754 | #if (defined(__ARM_NEON) && defined(__APPLE__)) 755 | static_assert(!(N != 128 && N != 64), "Invalid argument passed to xxh::vec_ops::set1"); 756 | #else 757 | static_assert(!(N != 128 && N != 256 && N != 64 && N != 512), "Invalid argument passed to xxh::vec_ops::set1"); 758 | if constexpr (N == 256) 759 | { 760 | return _mm256_set1_epi32(static_cast(a)); 761 | } 762 | 763 | if constexpr (N == 512) 764 | { 765 | return _mm512_set1_epi32(static_cast(a)); 766 | } 767 | #endif 768 | 769 | if constexpr (N == 128) 770 | { 771 | return _mm_set1_epi32(static_cast(a)); 772 | } 773 | 774 | if constexpr (N == 64) 775 | { 776 | return a; 777 | } 778 | } 779 | 780 | 781 | template 782 | XXH_FORCE_INLINE vec_t srli(vec_t n, int a) 783 | { 784 | static_assert(!(N != 128 && N != 256 && N != 64 && N != 512), "Invalid argument passed to xxh::vec_ops::srli"); 785 | 786 | if constexpr (N == 128) 787 | { 788 | return _mm_srli_epi64(n, a); 789 | } 790 | 791 | if constexpr (N == 256) 792 | { 793 | return _mm256_srli_epi64(n, a); 794 | } 795 | 796 | if constexpr (N == 512) 797 | { 798 | return _mm512_srli_epi64(n, a); 799 | } 800 | 801 | if constexpr (N == 64) 802 | { 803 | return n >> a; 804 | } 805 | } 806 | 807 | 808 | template 809 | XXH_FORCE_INLINE vec_t slli(vec_t n, int a) 810 | { 811 | static_assert(!(N != 128 && N != 256 && N != 64 && N != 512), "Invalid argument passed to xxh::vec_ops::slli"); 812 | 813 | if constexpr (N == 128) 814 | { 815 | return _mm_slli_epi64(n, a); 816 | } 817 | 818 | if constexpr (N == 256) 819 | { 820 | return _mm256_slli_epi64(n, a); 821 | } 822 | 823 | if constexpr (N == 512) 824 | { 825 | return _mm512_slli_epi64(n, a); 826 | } 827 | 828 | if constexpr (N == 64) 829 | { 830 | return n << a; 831 | } 832 | } 833 | } 834 | 835 | /* ************************************* 836 | * Canonical represenation 837 | ***************************************/ 838 | 839 | template 840 | struct canonical_t 841 | { 842 | std::array digest{ 0 }; 843 | 844 | canonical_t(hash_t hash) 845 | { 846 | if constexpr (bit_mode < 128) 847 | { 848 | if (mem_ops::is_little_endian()) 849 | { 850 | hash = bit_ops::swap(hash); 851 | } 852 | 853 | memcpy(digest.data(), &hash, sizeof(canonical_t)); 854 | } 855 | else 856 | { 857 | if (mem_ops::is_little_endian()) 858 | { 859 | hash.low64 = bit_ops::swap<64>(hash.low64); 860 | hash.high64 = bit_ops::swap<64>(hash.high64); 861 | } 862 | 863 | memcpy(digest.data(), &hash.high64, sizeof(hash.high64)); 864 | memcpy(digest.data() + sizeof(hash.high64), &hash.low64, sizeof(hash.low64)); 865 | } 866 | } 867 | 868 | hash_t get_hash() const 869 | { 870 | if constexpr (bit_mode < 128) 871 | { 872 | return mem_ops::readBE(&digest); 873 | } 874 | else 875 | { 876 | return { mem_ops::readBE<64>(&digest[8]), mem_ops::readBE<64>(&digest) }; 877 | } 878 | } 879 | }; 880 | 881 | using canonical32_t = canonical_t<32>; 882 | using canonical64_t = canonical_t<64>; 883 | using canonical128_t = canonical_t<128>; 884 | 885 | template 886 | inline hash_t to_canonical(hash_t hash) 887 | { 888 | static_assert(!(bit_mode != 128 && bit_mode != 64 && bit_mode != 32), "Canonical form can only be obtained from 32, 64 and 128 bit hashes."); 889 | canonical_t canon(hash); 890 | hash_t res; 891 | memcpy(&res, &canon, bit_mode / 8); 892 | 893 | return res; 894 | } 895 | 896 | 897 | /* ************************************* 898 | * Algorithm Implementation - xxhash 899 | ***************************************/ 900 | 901 | namespace detail 902 | { 903 | using namespace mem_ops; 904 | using namespace bit_ops; 905 | 906 | 907 | /* ************************************* 908 | * Constants 909 | ***************************************/ 910 | 911 | constexpr static std::array primes32 = { 2654435761U, 2246822519U, 3266489917U, 668265263U, 374761393U }; 912 | constexpr static std::array primes64 = { 11400714785074694791ULL, 14029467366897019727ULL, 1609587929392839161ULL, 9650029242287828579ULL, 2870177450012600261ULL }; 913 | 914 | template 915 | constexpr uint_t PRIME(uint64_t n) 916 | { 917 | if constexpr (N == 32) 918 | { 919 | return primes32[n - 1]; 920 | } 921 | else 922 | { 923 | return primes64[n - 1]; 924 | } 925 | } 926 | 927 | 928 | /* ************************************* 929 | * Functions 930 | ***************************************/ 931 | 932 | template 933 | XXH_FORCE_INLINE uint_t avalanche(uint_t hash) 934 | { 935 | if constexpr (N == 32) 936 | { 937 | hash ^= hash >> 15; 938 | hash *= PRIME<32>(2); 939 | hash ^= hash >> 13; 940 | hash *= PRIME<32>(3); 941 | hash ^= hash >> 16; 942 | return hash; 943 | } 944 | else if constexpr (N == 64) 945 | { 946 | hash ^= hash >> 33; 947 | hash *= PRIME<64>(2); 948 | hash ^= hash >> 29; 949 | hash *= PRIME<64>(3); 950 | hash ^= hash >> 32; 951 | return hash; 952 | } 953 | else return 0; 954 | } 955 | 956 | template 957 | XXH_FORCE_INLINE uint_t round(uint_t seed, uint_t input) 958 | { 959 | seed += input * PRIME(2); 960 | 961 | if constexpr (N == 32) 962 | { 963 | seed = rotl(seed, 13); 964 | } 965 | else 966 | { 967 | seed = rotl(seed, 31); 968 | } 969 | 970 | seed *= PRIME(1); 971 | return seed; 972 | } 973 | 974 | XXH_FORCE_INLINE uint64_t mergeRound64(hash64_t acc, uint64_t val) 975 | { 976 | val = round<64>(0, val); 977 | acc ^= val; 978 | acc = acc * PRIME<64>(1) + PRIME<64>(4); 979 | return acc; 980 | } 981 | 982 | XXH_FORCE_INLINE void endian_align_sub_mergeround(hash64_t& hash_ret, uint64_t v1, uint64_t v2, uint64_t v3, uint64_t v4) 983 | { 984 | hash_ret = mergeRound64(hash_ret, v1); 985 | hash_ret = mergeRound64(hash_ret, v2); 986 | hash_ret = mergeRound64(hash_ret, v3); 987 | hash_ret = mergeRound64(hash_ret, v4); 988 | } 989 | 990 | template 991 | static inline hash_t endian_align_sub_ending(hash_t hash_ret, const uint8_t* p, const uint8_t* bEnd) 992 | { 993 | if constexpr (N == 32) 994 | { 995 | while ((p + 4) <= bEnd) 996 | { 997 | hash_ret += readLE<32>(p) * PRIME<32>(3); 998 | hash_ret = rotl<32>(hash_ret, 17) * PRIME<32>(4); 999 | p += 4; 1000 | } 1001 | 1002 | while (p < bEnd) 1003 | { 1004 | hash_ret += (*p) * PRIME<32>(5); 1005 | hash_ret = rotl<32>(hash_ret, 11) * PRIME<32>(1); 1006 | p++; 1007 | } 1008 | 1009 | return avalanche<32>(hash_ret); 1010 | } 1011 | else 1012 | { 1013 | while (p + 8 <= bEnd) 1014 | { 1015 | const uint64_t k1 = round<64>(0, readLE<64>(p)); 1016 | 1017 | hash_ret ^= k1; 1018 | hash_ret = rotl<64>(hash_ret, 27) * PRIME<64>(1) + PRIME<64>(4); 1019 | p += 8; 1020 | } 1021 | 1022 | if (p + 4 <= bEnd) 1023 | { 1024 | hash_ret ^= static_cast(readLE<32>(p))* PRIME<64>(1); 1025 | hash_ret = rotl<64>(hash_ret, 23) * PRIME<64>(2) + PRIME<64>(3); 1026 | p += 4; 1027 | } 1028 | 1029 | while (p < bEnd) 1030 | { 1031 | hash_ret ^= (*p) * PRIME<64>(5); 1032 | hash_ret = rotl<64>(hash_ret, 11) * PRIME<64>(1); 1033 | p++; 1034 | } 1035 | 1036 | return avalanche<64>(hash_ret); 1037 | } 1038 | } 1039 | 1040 | template 1041 | static inline hash_t endian_align(const void* input, size_t len, uint_t seed) 1042 | { 1043 | static_assert(!(N != 32 && N != 64), "You can only call endian_align in 32 or 64 bit mode."); 1044 | 1045 | const uint8_t* p = static_cast(input); 1046 | const uint8_t* bEnd = p + len; 1047 | hash_t hash_ret; 1048 | 1049 | if (len >= (N / 2)) 1050 | { 1051 | const uint8_t* const limit = bEnd - (N / 2); 1052 | uint_t v1 = seed + PRIME(1) + PRIME(2); 1053 | uint_t v2 = seed + PRIME(2); 1054 | uint_t v3 = seed + 0; 1055 | uint_t v4 = seed - PRIME(1); 1056 | 1057 | do 1058 | { 1059 | v1 = round(v1, readLE(p)); 1060 | p += (N / 8); 1061 | v2 = round(v2, readLE(p)); 1062 | p += (N / 8); 1063 | v3 = round(v3, readLE(p)); 1064 | p += (N / 8); 1065 | v4 = round(v4, readLE(p)); 1066 | p += (N / 8); 1067 | } 1068 | while (p <= limit); 1069 | 1070 | hash_ret = rotl(v1, 1) + rotl(v2, 7) + rotl(v3, 12) + rotl(v4, 18); 1071 | 1072 | if constexpr (N == 64) 1073 | { 1074 | endian_align_sub_mergeround(hash_ret, v1, v2, v3, v4); 1075 | } 1076 | } 1077 | else 1078 | { 1079 | hash_ret = seed + PRIME(5); 1080 | } 1081 | 1082 | hash_ret += static_cast>(len); 1083 | 1084 | return endian_align_sub_ending(hash_ret, p, bEnd); 1085 | } 1086 | } 1087 | 1088 | 1089 | /* ************************************* 1090 | * Algorithm Implementation - xxhash3 1091 | ***************************************/ 1092 | 1093 | namespace detail3 1094 | { 1095 | using namespace vec_ops; 1096 | using namespace detail; 1097 | using namespace mem_ops; 1098 | using namespace bit_ops; 1099 | 1100 | 1101 | /* ************************************* 1102 | * Enums 1103 | ***************************************/ 1104 | 1105 | enum class vec_mode : uint8_t { scalar = 0, sse2 = 1, avx2 = 2, avx512 = 3 }; 1106 | 1107 | 1108 | /* ************************************* 1109 | * Constants 1110 | ***************************************/ 1111 | 1112 | constexpr uint64_t secret_default_size = 192; 1113 | constexpr uint64_t secret_size_min = 136; 1114 | constexpr uint64_t secret_consume_rate = 8; 1115 | constexpr uint64_t stripe_len = 64; 1116 | constexpr uint64_t acc_nb = 8; 1117 | constexpr uint64_t prefetch_distance = 384; 1118 | constexpr uint64_t secret_lastacc_start = 7; 1119 | constexpr uint64_t secret_mergeaccs_start = 11; 1120 | constexpr uint64_t midsize_max = 240; 1121 | constexpr uint64_t midsize_startoffset = 3; 1122 | constexpr uint64_t midsize_lastoffset = 17; 1123 | 1124 | constexpr vec_mode vector_mode = static_cast(intrin::vector_mode); 1125 | constexpr uint64_t acc_align = intrin::acc_align; 1126 | constexpr std::array vector_bit_width { 64, 128, 256, 512 }; 1127 | 1128 | 1129 | /* ************************************* 1130 | * Defaults 1131 | ***************************************/ 1132 | 1133 | alignas(64) constexpr uint8_t default_secret[secret_default_size] = { 1134 | 0xb8, 0xfe, 0x6c, 0x39, 0x23, 0xa4, 0x4b, 0xbe, 0x7c, 0x01, 0x81, 0x2c, 0xf7, 0x21, 0xad, 0x1c, 1135 | 0xde, 0xd4, 0x6d, 0xe9, 0x83, 0x90, 0x97, 0xdb, 0x72, 0x40, 0xa4, 0xa4, 0xb7, 0xb3, 0x67, 0x1f, 1136 | 0xcb, 0x79, 0xe6, 0x4e, 0xcc, 0xc0, 0xe5, 0x78, 0x82, 0x5a, 0xd0, 0x7d, 0xcc, 0xff, 0x72, 0x21, 1137 | 0xb8, 0x08, 0x46, 0x74, 0xf7, 0x43, 0x24, 0x8e, 0xe0, 0x35, 0x90, 0xe6, 0x81, 0x3a, 0x26, 0x4c, 1138 | 0x3c, 0x28, 0x52, 0xbb, 0x91, 0xc3, 0x00, 0xcb, 0x88, 0xd0, 0x65, 0x8b, 0x1b, 0x53, 0x2e, 0xa3, 1139 | 0x71, 0x64, 0x48, 0x97, 0xa2, 0x0d, 0xf9, 0x4e, 0x38, 0x19, 0xef, 0x46, 0xa9, 0xde, 0xac, 0xd8, 1140 | 0xa8, 0xfa, 0x76, 0x3f, 0xe3, 0x9c, 0x34, 0x3f, 0xf9, 0xdc, 0xbb, 0xc7, 0xc7, 0x0b, 0x4f, 0x1d, 1141 | 0x8a, 0x51, 0xe0, 0x4b, 0xcd, 0xb4, 0x59, 0x31, 0xc8, 0x9f, 0x7e, 0xc9, 0xd9, 0x78, 0x73, 0x64, 1142 | 0xea, 0xc5, 0xac, 0x83, 0x34, 0xd3, 0xeb, 0xc3, 0xc5, 0x81, 0xa0, 0xff, 0xfa, 0x13, 0x63, 0xeb, 1143 | 0x17, 0x0d, 0xdd, 0x51, 0xb7, 0xf0, 0xda, 0x49, 0xd3, 0x16, 0x55, 0x26, 0x29, 0xd4, 0x68, 0x9e, 1144 | 0x2b, 0x16, 0xbe, 0x58, 0x7d, 0x47, 0xa1, 0xfc, 0x8f, 0xf8, 0xb8, 0xd1, 0x7a, 0xd0, 0x31, 0xce, 1145 | 0x45, 0xcb, 0x3a, 0x8f, 0x95, 0x16, 0x04, 0x28, 0xaf, 0xd7, 0xfb, 0xca, 0xbb, 0x4b, 0x40, 0x7e, 1146 | }; 1147 | 1148 | constexpr std::array init_acc = { PRIME<32>(3), PRIME<64>(1), PRIME<64>(2), PRIME<64>(3), PRIME<64>(4), PRIME<32>(2), PRIME<64>(5), PRIME<32>(1) }; 1149 | 1150 | 1151 | /* ************************************* 1152 | * Functions 1153 | ***************************************/ 1154 | 1155 | XXH_FORCE_INLINE hash_t<64> avalanche(hash_t<64> h64) 1156 | { 1157 | constexpr uint64_t avalanche_mul_prime = 0x165667919E3779F9ULL; 1158 | 1159 | h64 ^= h64 >> 37; 1160 | h64 *= avalanche_mul_prime; 1161 | h64 ^= h64 >> 32; 1162 | return h64; 1163 | } 1164 | 1165 | XXH_FORCE_INLINE hash_t<64> rrmxmx(hash_t<64> h64, uint64_t len) 1166 | { 1167 | h64 ^= rotl<64>(h64, 49) ^ rotl<64>(h64, 24); 1168 | h64 *= 0x9FB21C651E98DF25ULL; 1169 | h64 ^= (h64 >> 35) + len; 1170 | h64 *= 0x9FB21C651E98DF25ULL; 1171 | h64 ^= (h64 >> 28); 1172 | return h64; 1173 | } 1174 | 1175 | XXH_FORCE_INLINE void combine_16(void* dest, hash128_t h128) 1176 | { 1177 | writeLE<64>(dest, readLE<64>(dest) ^ h128.low64); 1178 | writeLE<64>((uint8_t*)dest + 8, readLE<64>((uint8_t*)dest + 8) ^ h128.high64); 1179 | } 1180 | 1181 | XXH_FORCE_INLINE void accumulate_512(void* XXH_RESTRICT acc, const void* XXH_RESTRICT input, const void* XXH_RESTRICT secret) 1182 | { 1183 | constexpr uint64_t bits = vector_bit_width[static_cast(vector_mode)]; 1184 | 1185 | using vec_t = vec_t; 1186 | 1187 | alignas(sizeof(vec_t)) vec_t* const xacc = static_cast(acc); 1188 | const vec_t* const xinput = static_cast(input); 1189 | const vec_t* const xsecret = static_cast(secret); 1190 | 1191 | for (size_t i = 0; i < stripe_len / sizeof(vec_t); i++) 1192 | { 1193 | vec_t const data_vec = loadu(xinput + i); 1194 | vec_t const key_vec = loadu(xsecret + i); 1195 | vec_t const data_key = xorv(data_vec, key_vec); 1196 | vec_t product = set1(0); 1197 | 1198 | if constexpr (vector_mode == vec_mode::scalar) 1199 | { 1200 | product = mul32to64(srli(slli(data_key, 32),32), srli(data_key, 32)); 1201 | xacc[i ^ 1] = add(xacc[i ^ 1], data_vec); 1202 | xacc[i] = add(xacc[i], product); 1203 | } 1204 | else 1205 | { 1206 | vec_t const data_key_lo = shuffle(data_key); 1207 | product = mul(data_key, data_key_lo); 1208 | 1209 | vec_t const data_swap = shuffle(data_vec); 1210 | vec_t const sum = add(xacc[i], data_swap); 1211 | xacc[i] = add(sum, product); 1212 | } 1213 | } 1214 | } 1215 | 1216 | XXH_FORCE_INLINE void scramble_acc(void* XXH_RESTRICT acc, const void* XXH_RESTRICT secret) 1217 | { 1218 | constexpr uint64_t bits = vector_bit_width[static_cast(vector_mode)];; 1219 | 1220 | using vec_t = vec_t; 1221 | 1222 | alignas(sizeof(vec_t)) vec_t* const xacc = (vec_t*)acc; 1223 | const vec_t* const xsecret = (const vec_t*)secret; 1224 | 1225 | for (size_t i = 0; i < stripe_len / sizeof(vec_t); i++) 1226 | { 1227 | vec_t const acc_vec = xacc[i]; 1228 | vec_t const shifted = srli(acc_vec, 47); 1229 | vec_t const data_vec = xorv(acc_vec, shifted); 1230 | vec_t const key_vec = loadu(xsecret + i); 1231 | vec_t const data_key = xorv(data_vec, key_vec); 1232 | 1233 | if constexpr (vector_mode == vec_mode::scalar) 1234 | { 1235 | xacc[i] = mul(data_key, set1(PRIME<32>(1))); 1236 | } 1237 | else 1238 | { 1239 | vec_t const prime32 = set1(PRIME<32>(1)); 1240 | vec_t const data_key_hi = shuffle(data_key); 1241 | vec_t const prod_lo = mul(data_key, prime32); 1242 | vec_t const prod_hi = mul(data_key_hi, prime32); 1243 | 1244 | xacc[i] = add(prod_lo, vec_ops::slli(prod_hi, 32)); 1245 | } 1246 | } 1247 | } 1248 | 1249 | XXH_FORCE_INLINE void accumulate(uint64_t* XXH_RESTRICT acc, const uint8_t* XXH_RESTRICT input, const uint8_t* XXH_RESTRICT secret, size_t nbStripes) 1250 | { 1251 | for (size_t n = 0; n < nbStripes; n++) 1252 | { 1253 | const uint8_t* const in = input + n * stripe_len; 1254 | 1255 | intrin::prefetch(in + prefetch_distance); 1256 | accumulate_512(acc, in, secret + n * secret_consume_rate); 1257 | } 1258 | } 1259 | 1260 | XXH_FORCE_INLINE void hash_long_internal_loop(uint64_t* XXH_RESTRICT acc, const uint8_t* XXH_RESTRICT input, size_t len, const uint8_t* XXH_RESTRICT secret, size_t secretSize) 1261 | { 1262 | size_t const nb_rounds = (secretSize - stripe_len) / secret_consume_rate; 1263 | size_t const block_len = stripe_len * nb_rounds; 1264 | size_t const nb_blocks = (len-1) / block_len; 1265 | 1266 | for (size_t n = 0; n < nb_blocks; n++) 1267 | { 1268 | accumulate(acc, input + n * block_len, secret, nb_rounds); 1269 | scramble_acc(acc, secret + secretSize - stripe_len); 1270 | } 1271 | 1272 | /* last partial block */ 1273 | size_t const nbStripes = ((len - 1) - (block_len * nb_blocks)) / stripe_len; 1274 | 1275 | accumulate(acc, input + nb_blocks * block_len, secret, nbStripes); 1276 | 1277 | /* last stripe */ 1278 | const uint8_t* const p = input + len - stripe_len; 1279 | 1280 | accumulate_512(acc, p, secret + secretSize - stripe_len - secret_lastacc_start); 1281 | } 1282 | 1283 | XXH_FORCE_INLINE uint64_t mix_2_accs(const uint64_t* XXH_RESTRICT acc, const uint8_t* XXH_RESTRICT secret) 1284 | { 1285 | return mul128fold64(acc[0] ^ readLE<64>(secret), acc[1] ^ readLE<64>(secret + 8)); 1286 | } 1287 | 1288 | XXH_FORCE_INLINE uint64_t merge_accs(const uint64_t* XXH_RESTRICT acc, const uint8_t* XXH_RESTRICT secret, uint64_t start) 1289 | { 1290 | uint64_t result64 = start; 1291 | 1292 | result64 += mix_2_accs(acc + 0, secret + 0); 1293 | result64 += mix_2_accs(acc + 2, secret + 16); 1294 | result64 += mix_2_accs(acc + 4, secret + 32); 1295 | result64 += mix_2_accs(acc + 6, secret + 48); 1296 | 1297 | return avalanche(result64); 1298 | } 1299 | 1300 | XXH_FORCE_INLINE void init_custom_secret(uint8_t* customSecret, uint64_t seed) 1301 | { 1302 | for (uint64_t i = 0; i < secret_default_size / 16; i++) 1303 | { 1304 | writeLE<64>(customSecret + i * 16, readLE<64>(default_secret + i * 16) + seed); 1305 | writeLE<64>(customSecret + i * 16 + 8, readLE<64>(default_secret + i * 16 + 8) - seed); 1306 | } 1307 | } 1308 | 1309 | template 1310 | XXH_FORCE_INLINE hash_t len_1to3(const uint8_t* input, size_t len, const uint8_t* secret, uint64_t seed) 1311 | { 1312 | if constexpr (N == 64) 1313 | { 1314 | uint8_t const c1 = input[0]; 1315 | uint8_t const c2 = input[len >> 1]; 1316 | uint8_t const c3 = input[len - 1]; 1317 | uint32_t const combined = ((uint32_t)c1 << 16) | (((uint32_t)c2) << 24) | (((uint32_t)c3) << 0) | (((uint32_t)len) << 8); 1318 | uint64_t const bitflip = (readLE<32>(secret) ^ readLE<32>(secret + 4)) + seed; 1319 | uint64_t const keyed = (uint64_t)combined ^ bitflip; 1320 | return detail::avalanche<64>(keyed); 1321 | } 1322 | else 1323 | { 1324 | uint8_t const c1 = input[0]; 1325 | uint8_t const c2 = input[len >> 1]; 1326 | uint8_t const c3 = input[len - 1]; 1327 | uint32_t const combinedl = ((uint32_t)c1 << 16) + (((uint32_t)c2) << 24) + (((uint32_t)c3) << 0) + (((uint32_t)len) << 8); 1328 | uint32_t const combinedh = rotl<32>(swap<32>(combinedl), 13); 1329 | uint64_t const bitflipl = (readLE<32>(secret) ^ readLE<32>(secret + 4)) + seed; 1330 | uint64_t const bitfliph = (readLE<32>(secret + 8) ^ readLE<32>(secret + 12)) - seed; 1331 | uint64_t const keyed_lo = (uint64_t)combinedl ^ bitflipl; 1332 | uint64_t const keyed_hi = (uint64_t)combinedh ^ bitfliph; 1333 | hash128_t const h128 = { detail::avalanche<64>(keyed_lo), detail::avalanche<64>(keyed_hi)}; 1334 | 1335 | return h128; 1336 | } 1337 | } 1338 | 1339 | template 1340 | XXH_FORCE_INLINE hash_t len_4to8(const uint8_t* input, size_t len, const uint8_t* secret, uint64_t seed) 1341 | { 1342 | constexpr uint64_t mix_constant = 0x9FB21C651E98DF25ULL; 1343 | 1344 | seed ^= (uint64_t)swap<32>((uint32_t)seed) << 32; 1345 | 1346 | if constexpr (N == 64) 1347 | { 1348 | uint32_t const input1 = readLE<32>(input); 1349 | uint32_t const input2 = readLE<32>(input + len - 4); 1350 | uint64_t const bitflip = (readLE<64>(secret + 8) ^ readLE<64>(secret + 16)) - seed; 1351 | uint64_t const input64 = input2 + ((uint64_t)input1 << 32); 1352 | uint64_t keyed = input64 ^ bitflip; 1353 | 1354 | return rrmxmx(keyed, len); 1355 | } 1356 | else 1357 | { 1358 | uint32_t const input_lo = readLE<32>(input); 1359 | uint32_t const input_hi = readLE<32>(input + len - 4); 1360 | uint64_t const input_64 = input_lo + ((uint64_t)input_hi << 32); 1361 | uint64_t const bitflip = (readLE<64>(secret + 16) ^ readLE<64>(secret + 24)) + seed; 1362 | uint64_t const keyed = input_64 ^ bitflip; 1363 | uint128_t m128 = mul64to128(keyed, PRIME<64>(1) + (len << 2)); 1364 | 1365 | m128.high64 += (m128.low64 << 1); 1366 | m128.low64 ^= (m128.high64 >> 3); 1367 | m128.low64 ^= (m128.low64 >> 35); 1368 | m128.low64 *= mix_constant; 1369 | m128.low64 ^= (m128.low64 >> 28); 1370 | m128.high64 = avalanche(m128.high64); 1371 | 1372 | return m128; 1373 | } 1374 | } 1375 | 1376 | template 1377 | XXH_FORCE_INLINE hash_t len_9to16(const uint8_t* input, size_t len, const uint8_t* secret, uint64_t seed) 1378 | { 1379 | if constexpr (N == 64) 1380 | { 1381 | uint64_t const bitflip1 = (readLE<64>(secret + 24) ^ readLE<64>(secret + 32)) + seed; 1382 | uint64_t const bitflip2 = (readLE<64>(secret + 40) ^ readLE<64>(secret + 48)) - seed; 1383 | uint64_t const input_lo = readLE<64>(input) ^ bitflip1; 1384 | uint64_t const input_hi = readLE<64>(input + len - 8) ^ bitflip2; 1385 | uint64_t const acc = len + swap<64>(input_lo) + input_hi + mul128fold64(input_lo, input_hi); 1386 | 1387 | return avalanche(acc); 1388 | } 1389 | else 1390 | { 1391 | uint64_t const bitflipl = (readLE<64>(secret + 32) ^ readLE<64>(secret + 40)) - seed; 1392 | uint64_t const bitfliph = (readLE<64>(secret + 48) ^ readLE<64>(secret + 56)) + seed; 1393 | uint64_t const input_lo = readLE<64>(input); 1394 | uint64_t input_hi = readLE<64>(input + len - 8); 1395 | uint128_t m128 = mul64to128(input_lo ^ input_hi ^ bitflipl, PRIME<64>(1)); 1396 | 1397 | m128.low64 += (uint64_t)(len - 1) << 54; 1398 | input_hi ^= bitfliph; 1399 | 1400 | if constexpr (sizeof(void*) < sizeof(uint64_t)) // 32-bit version 1401 | { 1402 | m128.high64 += (input_hi & 0xFFFFFFFF00000000) + mul32to64((uint32_t)input_hi, PRIME<32>(2)); 1403 | } 1404 | else 1405 | { 1406 | m128.high64 += input_hi + mul32to64((uint32_t)input_hi, PRIME<32>(2) - 1); 1407 | } 1408 | 1409 | m128.low64 ^= swap<64>(m128.high64); 1410 | 1411 | hash128_t h128 = mul64to128(m128.low64, PRIME<64>(2)); 1412 | 1413 | h128.high64 += m128.high64 * PRIME<64>(2); 1414 | h128.low64 = avalanche(h128.low64); 1415 | h128.high64 = avalanche(h128.high64); 1416 | 1417 | return h128; 1418 | } 1419 | } 1420 | 1421 | template 1422 | XXH_FORCE_INLINE hash_t len_0to16(const uint8_t* input, size_t len, const uint8_t* secret, uint64_t seed) 1423 | { 1424 | if (XXH_likely(len > 8)) 1425 | { 1426 | return len_9to16(input, len, secret, seed); 1427 | } 1428 | else if (XXH_likely(len >= 4)) 1429 | { 1430 | return len_4to8(input, len, secret, seed); 1431 | } 1432 | else if (len) 1433 | { 1434 | return len_1to3(input, len, secret, seed); 1435 | } 1436 | else 1437 | { 1438 | if constexpr (N == 64) 1439 | { 1440 | return detail::avalanche<64>((seed) ^ (readLE<64>(secret + 56) ^ readLE<64>(secret + 64))); 1441 | } 1442 | else 1443 | { 1444 | uint64_t const bitflipl = readLE<64>(secret + 64) ^ readLE<64>(secret + 72); 1445 | uint64_t const bitfliph = readLE<64>(secret + 80) ^ readLE<64>(secret + 88); 1446 | 1447 | return hash128_t(detail::avalanche<64>(( seed) ^ bitflipl), detail::avalanche<64>(( seed) ^ bitfliph)); 1448 | } 1449 | } 1450 | } 1451 | 1452 | template 1453 | XXH_FORCE_INLINE hash_t hash_long_internal(const uint8_t* XXH_RESTRICT input, size_t len, const uint8_t* XXH_RESTRICT secret = default_secret, size_t secretSize = sizeof(default_secret)) 1454 | { 1455 | alignas(acc_align) std::array acc = init_acc; 1456 | 1457 | if constexpr (N == 64) 1458 | { 1459 | hash_long_internal_loop(acc.data(), input, len, secret, secretSize); 1460 | 1461 | /* converge into final hash */ 1462 | return merge_accs(acc.data(), secret + secret_mergeaccs_start, (uint64_t)len * PRIME<64>(1)); 1463 | } 1464 | else 1465 | { 1466 | hash_long_internal_loop(acc.data(), input, len, secret, secretSize); 1467 | 1468 | /* converge into final hash */ 1469 | uint64_t const low64 = merge_accs(acc.data(), secret + secret_mergeaccs_start, (uint64_t)len * PRIME<64>(1)); 1470 | uint64_t const high64 = merge_accs(acc.data(), secret + secretSize - sizeof(acc) - secret_mergeaccs_start, ~((uint64_t)len * PRIME<64>(2))); 1471 | 1472 | return hash128_t(low64, high64); 1473 | } 1474 | } 1475 | 1476 | XXH_FORCE_INLINE uint64_t mix_16b(const uint8_t* XXH_RESTRICT input, const uint8_t* XXH_RESTRICT secret, uint64_t seed) 1477 | { 1478 | uint64_t const input_lo = readLE<64>(input); 1479 | uint64_t const input_hi = readLE<64>(input + 8); 1480 | 1481 | return mul128fold64(input_lo ^ (readLE<64>(secret) + seed), input_hi ^ (readLE<64>(secret + 8) - seed)); 1482 | } 1483 | 1484 | XXH_FORCE_INLINE uint128_t mix_32b(uint128_t acc, const uint8_t* input1, const uint8_t* input2, const uint8_t* secret, uint64_t seed) 1485 | { 1486 | acc.low64 += mix_16b(input1, secret + 0, seed); 1487 | acc.low64 ^= readLE<64>(input2) + readLE<64>(input2 + 8); 1488 | acc.high64 += mix_16b(input2, secret + 16, seed); 1489 | acc.high64 ^= readLE<64>(input1) + readLE<64>(input1 + 8); 1490 | 1491 | return acc; 1492 | } 1493 | 1494 | template 1495 | XXH_FORCE_INLINE hash_t len_17to128(const uint8_t* XXH_RESTRICT input, size_t len, const uint8_t* XXH_RESTRICT secret, uint64_t seed) 1496 | { 1497 | if constexpr (N == 64) 1498 | { 1499 | hash64_t acc = len * PRIME<64>(1); 1500 | 1501 | if (len > 32) 1502 | { 1503 | if (len > 64) 1504 | { 1505 | if (len > 96) 1506 | { 1507 | acc += mix_16b(input + 48, secret + 96, seed); 1508 | acc += mix_16b(input + len - 64, secret + 112, seed); 1509 | } 1510 | 1511 | acc += mix_16b(input + 32, secret + 64, seed); 1512 | acc += mix_16b(input + len - 48, secret + 80, seed); 1513 | } 1514 | 1515 | acc += mix_16b(input + 16, secret + 32, seed); 1516 | acc += mix_16b(input + len - 32, secret + 48, seed); 1517 | } 1518 | 1519 | acc += mix_16b(input + 0, secret + 0, seed); 1520 | acc += mix_16b(input + len - 16, secret + 16, seed); 1521 | 1522 | return avalanche(acc); 1523 | } 1524 | else 1525 | { 1526 | hash128_t acc = { len * PRIME<64>(1), 0 }; 1527 | 1528 | if (len > 32) 1529 | { 1530 | if (len > 64) 1531 | { 1532 | if (len > 96) 1533 | { 1534 | acc = mix_32b(acc, input + 48, input + len - 64, secret + 96, seed); 1535 | } 1536 | 1537 | acc = mix_32b(acc, input + 32, input + len - 48, secret + 64, seed); 1538 | } 1539 | 1540 | acc = mix_32b(acc, input + 16, input + len - 32, secret + 32, seed); 1541 | } 1542 | 1543 | acc = mix_32b(acc, input, input + len - 16, secret, seed); 1544 | 1545 | uint64_t const low64 = acc.low64 + acc.high64; 1546 | uint64_t const high64 = (acc.low64 * PRIME<64>(1)) + (acc.high64 * PRIME<64>(4)) + ((len - seed) * PRIME<64>(2)); 1547 | 1548 | return { avalanche(low64), (uint64_t)0 - avalanche(high64) }; 1549 | } 1550 | } 1551 | 1552 | template 1553 | XXH_NO_INLINE hash_t len_129to240(const uint8_t* XXH_RESTRICT input, size_t len, const uint8_t* XXH_RESTRICT secret, uint64_t seed) 1554 | { 1555 | if constexpr (N == 64) 1556 | { 1557 | uint64_t acc = len * PRIME<64>(1); 1558 | size_t const nbRounds = len / 16; 1559 | 1560 | for (size_t i = 0; i < 8; i++) 1561 | { 1562 | acc += mix_16b(input + (i * 16), secret + (i * 16), seed); 1563 | } 1564 | 1565 | acc = avalanche(acc); 1566 | 1567 | for (size_t i = 8; i < nbRounds; i++) 1568 | { 1569 | acc += mix_16b(input + (i * 16), secret + ((i - 8) * 16) + midsize_startoffset, seed); 1570 | } 1571 | 1572 | /* last bytes */ 1573 | acc += mix_16b(input + len - 16, secret + secret_size_min - midsize_lastoffset, seed); 1574 | 1575 | return avalanche(acc); 1576 | } 1577 | else 1578 | { 1579 | hash128_t acc; 1580 | uint64_t const nbRounds = len / 32; 1581 | 1582 | acc.low64 = len * PRIME<64>(1); 1583 | acc.high64 = 0; 1584 | 1585 | for (size_t i = 0; i < 4; i++) 1586 | { 1587 | acc = mix_32b(acc, input + (i * 32), input + (i * 32) + 16, secret + (i * 32), seed); 1588 | } 1589 | 1590 | acc.low64 = avalanche(acc.low64); 1591 | acc.high64 = avalanche(acc.high64); 1592 | 1593 | for (size_t i = 4; i < nbRounds; i++) 1594 | { 1595 | acc = mix_32b(acc, input + (i * 32), input + (i * 32) + 16, secret + midsize_startoffset + ((i - 4) * 32), seed); 1596 | } 1597 | 1598 | /* last bytes */ 1599 | acc = mix_32b(acc, input + len - 16, input + len - 32, secret + secret_size_min - midsize_lastoffset - 16, 0ULL - seed); 1600 | 1601 | uint64_t const low64 = acc.low64 + acc.high64; 1602 | uint64_t const high64 = (acc.low64 * PRIME<64>(1)) + (acc.high64 * PRIME<64>(4)) + ((len - seed) * PRIME<64>(2)); 1603 | 1604 | return { avalanche(low64), (uint64_t)0 - avalanche(high64) }; 1605 | } 1606 | 1607 | } 1608 | 1609 | template 1610 | XXH_NO_INLINE hash_t xxhash3_impl(const void* XXH_RESTRICT input, size_t len, hash64_t seed, const void* XXH_RESTRICT secret = default_secret, size_t secretSize = secret_default_size, bool forceSeedUse = false) 1611 | { 1612 | 1613 | alignas(64) uint8_t custom_secret[secret_default_size]; 1614 | 1615 | const void* short_secret = secret; 1616 | 1617 | if (seed != 0 || forceSeedUse) 1618 | { 1619 | init_custom_secret(custom_secret, seed); 1620 | short_secret = default_secret; 1621 | } 1622 | 1623 | if (len <= 16) 1624 | { 1625 | return len_0to16(static_cast(input), len, static_cast(short_secret), seed); 1626 | } 1627 | else if (len <= 128) 1628 | { 1629 | return len_17to128(static_cast(input), len, static_cast(short_secret), seed); 1630 | } 1631 | else if (len <= midsize_max) 1632 | { 1633 | return len_129to240(static_cast(input), len, static_cast(short_secret), seed); 1634 | } 1635 | else 1636 | { 1637 | return hash_long_internal(static_cast(input), len, static_cast(((seed == 0) ? secret : ((secret == default_secret) ? custom_secret : secret))), ((seed == 0) ? secretSize : ((secret == default_secret) ? secret_default_size : secretSize))); 1638 | } 1639 | } 1640 | 1641 | XXH_NO_INLINE void generate_secret(void* secret_buffer, size_t secret_size, const void* custom_seed, size_t seed_size) 1642 | { 1643 | if (seed_size == 0) 1644 | { 1645 | custom_seed = default_secret; 1646 | seed_size = secret_default_size; 1647 | } 1648 | 1649 | size_t pos = 0; 1650 | while (pos < secret_size) 1651 | { 1652 | size_t const copy_len = std::min(secret_size - pos, seed_size); 1653 | memcpy((uint8_t*)secret_buffer + pos, custom_seed, copy_len); 1654 | pos += copy_len; 1655 | } 1656 | 1657 | size_t const nbseg16 = secret_size / 16; 1658 | canonical128_t scrambled(xxhash3_impl<128>(custom_seed, seed_size, 0)); 1659 | for (size_t n = 0; n < nbseg16; n++) 1660 | { 1661 | hash128_t const h128 = xxhash3_impl<128>(&scrambled, sizeof(scrambled), n); 1662 | combine_16((uint8_t*)secret_buffer + n * 16, h128); 1663 | } 1664 | 1665 | combine_16((uint8_t*)secret_buffer + secret_size - 16, scrambled.get_hash()); 1666 | } 1667 | } 1668 | 1669 | 1670 | /* ************************************* 1671 | * Public Access Point - xxhash 1672 | ***************************************/ 1673 | 1674 | template 1675 | inline hash_t xxhash(const void* input, size_t len, uint_t seed = 0) 1676 | { 1677 | static_assert(!(bit_mode != 32 && bit_mode != 64), "xxhash can only be used in 32 and 64 bit modes."); 1678 | return detail::endian_align(input, len, seed); 1679 | } 1680 | 1681 | template 1682 | inline hash_t xxhash(const std::basic_string& input, uint_t seed = 0) 1683 | { 1684 | static_assert(!(bit_mode != 32 && bit_mode != 64), "xxhash can only be used in 32 and 64 bit modes."); 1685 | return detail::endian_align(static_cast(input.data()), input.length() * sizeof(T), seed); 1686 | } 1687 | 1688 | template 1689 | inline hash_t xxhash(ContiguousIterator begin, ContiguousIterator end, uint_t seed = 0) 1690 | { 1691 | static_assert(!(bit_mode != 32 && bit_mode != 64), "xxhash can only be used in 32 and 64 bit modes."); 1692 | using T = typename std::decay_t; 1693 | return detail::endian_align(static_cast(&*begin), (end - begin) * sizeof(T), seed); 1694 | } 1695 | 1696 | template 1697 | inline hash_t xxhash(const std::vector& input, uint_t seed = 0) 1698 | { 1699 | static_assert(!(bit_mode != 32 && bit_mode != 64), "xxhash can only be used in 32 and 64 bit modes."); 1700 | return detail::endian_align(static_cast(input.data()), input.size() * sizeof(T), seed); 1701 | } 1702 | 1703 | template 1704 | inline hash_t xxhash(const std::array& input, uint_t seed = 0) 1705 | { 1706 | static_assert(!(bit_mode != 32 && bit_mode != 64), "xxhash can only be used in 32 and 64 bit modes."); 1707 | return detail::endian_align(static_cast(input.data()), AN * sizeof(T), seed); 1708 | } 1709 | 1710 | template 1711 | inline hash_t xxhash(const std::initializer_list& input, uint_t seed = 0) 1712 | { 1713 | static_assert(!(bit_mode != 32 && bit_mode != 64), "xxhash can only be used in 32 and 64 bit modes."); 1714 | return detail::endian_align(static_cast(input.begin()), input.size() * sizeof(T), seed); 1715 | } 1716 | 1717 | 1718 | /* ************************************* 1719 | * Public Access Point - xxhash3 1720 | ***************************************/ 1721 | 1722 | template 1723 | inline hash_t xxhash3(const void* input, size_t len, uint64_t seed = 0) 1724 | { 1725 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1726 | return detail3::xxhash3_impl(input, len, seed); 1727 | } 1728 | 1729 | template 1730 | inline hash_t xxhash3(const void* input, size_t len, const void* secret, size_t secretSize) 1731 | { 1732 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1733 | return detail3::xxhash3_impl(input, len, 0, secret, secretSize); 1734 | } 1735 | 1736 | template 1737 | inline hash_t xxhash3(const void* input, size_t len, const void* secret, size_t secretSize, uint64_t seed) 1738 | { 1739 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1740 | return detail3::xxhash3_impl(input, len, seed, secret, secretSize, true); 1741 | } 1742 | 1743 | template 1744 | inline hash_t xxhash3(const std::basic_string& input, uint64_t seed = 0) 1745 | { 1746 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1747 | return detail3::xxhash3_impl(static_cast(input.data()), input.length() * sizeof(T), seed); 1748 | } 1749 | 1750 | template 1751 | inline hash_t xxhash3(const std::basic_string& input, const void* secret, size_t secretSize) 1752 | { 1753 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1754 | return detail3::xxhash3_impl(static_cast(input.data()), input.length() * sizeof(T), 0, secret, secretSize); 1755 | } 1756 | 1757 | template 1758 | inline hash_t xxhash3(const std::basic_string& input, const void* secret, size_t secretSize, uint64_t seed) 1759 | { 1760 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1761 | return detail3::xxhash3_impl(static_cast(input.data()), input.length() * sizeof(T), seed, secret, secretSize, true); 1762 | } 1763 | 1764 | template 1765 | inline hash_t xxhash3(ContiguousIterator begin, ContiguousIterator end, uint64_t seed = 0) 1766 | { 1767 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1768 | using T = typename std::decay_t; 1769 | return detail3::xxhash3_impl(static_cast(&*begin), (end - begin) * sizeof(T), seed); 1770 | } 1771 | 1772 | template 1773 | inline hash_t xxhash3(ContiguousIterator begin, ContiguousIterator end, const void* secret, size_t secretSize) 1774 | { 1775 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1776 | using T = typename std::decay_t; 1777 | return detail3::xxhash3_impl(static_cast(&*begin), (end - begin) * sizeof(T), 0, secret, secretSize); 1778 | } 1779 | 1780 | template 1781 | inline hash_t xxhash3(ContiguousIterator begin, ContiguousIterator end, const void* secret, size_t secretSize, uint64_t seed) 1782 | { 1783 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1784 | using T = typename std::decay_t; 1785 | return detail3::xxhash3_impl(static_cast(&*begin), (end - begin) * sizeof(T), seed, secret, secretSize, true); 1786 | } 1787 | 1788 | template 1789 | inline hash_t xxhash3(const std::vector& input, uint64_t seed = 0) 1790 | { 1791 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1792 | return detail3::xxhash3_impl(static_cast(input.data()), input.size() * sizeof(T), seed); 1793 | } 1794 | 1795 | template 1796 | inline hash_t xxhash3(const std::vector& input, const void* secret, size_t secretSize) 1797 | { 1798 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1799 | return detail3::xxhash3_impl(static_cast(input.data()), input.size() * sizeof(T), 0, secret, secretSize); 1800 | } 1801 | 1802 | template 1803 | inline hash_t xxhash3(const std::vector& input, const void* secret, size_t secretSize, uint64_t seed) 1804 | { 1805 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1806 | return detail3::xxhash3_impl(static_cast(input.data()), input.size() * sizeof(T), seed, secret, secretSize, true); 1807 | } 1808 | 1809 | template 1810 | inline hash_t xxhash3(const std::array& input, uint64_t seed = 0) 1811 | { 1812 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1813 | return detail3::xxhash3_impl(static_cast(input.data()), AN * sizeof(T), seed); 1814 | } 1815 | 1816 | template 1817 | inline hash_t xxhash3(const std::array& input, const void* secret, size_t secretSize) 1818 | { 1819 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1820 | return detail3::xxhash3_impl(static_cast(input.data()), AN * sizeof(T), 0, secret, secretSize); 1821 | } 1822 | 1823 | template 1824 | inline hash_t xxhash3(const std::array& input, const void* secret, size_t secretSize, uint64_t seed) 1825 | { 1826 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1827 | return detail3::xxhash3_impl(static_cast(input.data()), AN * sizeof(T), seed, secret, secretSize, true); 1828 | } 1829 | 1830 | template 1831 | inline hash_t xxhash3(const std::initializer_list& input, uint64_t seed = 0) 1832 | { 1833 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1834 | return detail3::xxhash3_impl(static_cast(input.begin()), input.size() * sizeof(T), seed); 1835 | } 1836 | 1837 | template 1838 | inline hash_t xxhash3(const std::initializer_list& input, const void* secret, size_t secretSize) 1839 | { 1840 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1841 | return detail3::xxhash3_impl(static_cast(input.begin()), input.size() * sizeof(T), 0, secret, secretSize); 1842 | } 1843 | 1844 | template 1845 | inline hash_t xxhash3(const std::initializer_list& input, const void* secret, size_t secretSize, uint64_t seed) 1846 | { 1847 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 can only be used in 64 and 128 bit modes."); 1848 | return detail3::xxhash3_impl(static_cast(input.begin()), input.size() * sizeof(T), seed, secret, secretSize, true); 1849 | } 1850 | 1851 | 1852 | /* ************************************* 1853 | * Secret Generation Functions 1854 | ***************************************/ 1855 | 1856 | inline void generate_secret(void* secret_buffer, size_t secret_size, const void* custom_seed = detail3::default_secret, size_t seed_length = 0) 1857 | { 1858 | detail3::generate_secret(secret_buffer, secret_size, custom_seed, seed_length); 1859 | } 1860 | 1861 | template 1862 | inline void generate_secret(void* secret_buffer, size_t secret_size, const std::array& custom_seed) 1863 | { 1864 | detail3::generate_secret(secret_buffer, secret_size, static_cast(custom_seed.data()), AN * sizeof(T)); 1865 | } 1866 | 1867 | template 1868 | inline void generate_secret(void* secret_buffer, size_t secret_size, const std::initializer_list& custom_seed) 1869 | { 1870 | detail3::generate_secret(secret_buffer, secret_size, static_cast(custom_seed.begin()), custom_seed.size() * sizeof(T)); 1871 | } 1872 | 1873 | template 1874 | inline void generate_secret(void* secret_buffer, size_t secret_size, const std::vector& custom_seed) 1875 | { 1876 | detail3::generate_secret(secret_buffer, secret_size, static_cast(custom_seed.data()), custom_seed.size() * sizeof(T)); 1877 | } 1878 | 1879 | template 1880 | inline void generate_secret(void* secret_buffer, size_t secret_size, const std::basic_string& custom_seed) 1881 | { 1882 | detail3::generate_secret(secret_buffer, secret_size, static_cast(custom_seed.data()), custom_seed.length() * sizeof(T)); 1883 | } 1884 | 1885 | template 1886 | inline void generate_secret(void* secret_buffer, size_t secret_size, ContiguousIterator begin, ContiguousIterator end) 1887 | { 1888 | using T = typename std::decay_t; 1889 | detail3::generate_secret(secret_buffer, secret_size, static_cast(&*begin), (end - begin) * sizeof(T)); 1890 | } 1891 | 1892 | inline void generate_secret_from_seed(void* secret_buffer, uint64_t seed = 0) 1893 | { 1894 | alignas(64) uint8_t custom_secret[detail3::secret_default_size]; 1895 | detail3::init_custom_secret(custom_secret, seed); 1896 | memcpy(secret_buffer, custom_secret, detail3::secret_default_size); 1897 | } 1898 | 1899 | 1900 | /* ************************************* 1901 | * Hash streaming - xxhash 1902 | ***************************************/ 1903 | 1904 | template 1905 | class hash_state_t 1906 | { 1907 | uint64_t total_len = 0; 1908 | uint_t v1 = 0, v2 = 0, v3 = 0, v4 = 0; 1909 | std::array, 4> mem = {0, 0, 0, 0}; 1910 | uint32_t memsize = 0; 1911 | 1912 | inline void update_impl(const void* input, size_t length) 1913 | { 1914 | const uint8_t* p = reinterpret_cast(input); 1915 | const uint8_t* const bEnd = p + length; 1916 | 1917 | total_len += length; 1918 | 1919 | if (memsize + length < (bit_mode / 2)) 1920 | { /* fill in tmp buffer */ 1921 | memcpy(reinterpret_cast(mem.data()) + memsize, input, length); 1922 | memsize += static_cast(length); 1923 | return; 1924 | } 1925 | 1926 | if (memsize > 0) 1927 | { /* some data left from previous update */ 1928 | memcpy(reinterpret_cast(mem.data()) + memsize, input, (bit_mode / 2) - memsize); 1929 | 1930 | const uint_t* ptr = mem.data(); 1931 | 1932 | v1 = detail::round(v1, mem_ops::readLE(ptr)); 1933 | ptr++; 1934 | v2 = detail::round(v2, mem_ops::readLE(ptr)); 1935 | ptr++; 1936 | v3 = detail::round(v3, mem_ops::readLE(ptr)); 1937 | ptr++; 1938 | v4 = detail::round(v4, mem_ops::readLE(ptr)); 1939 | 1940 | p += (bit_mode / 2) - memsize; 1941 | memsize = 0; 1942 | } 1943 | 1944 | while (p + (bit_mode / 2) <= bEnd) 1945 | { 1946 | v1 = detail::round(v1, mem_ops::readLE(p)); 1947 | p += (bit_mode / 8); 1948 | v2 = detail::round(v2, mem_ops::readLE(p)); 1949 | p += (bit_mode / 8); 1950 | v3 = detail::round(v3, mem_ops::readLE(p)); 1951 | p += (bit_mode / 8); 1952 | v4 = detail::round(v4, mem_ops::readLE(p)); 1953 | p += (bit_mode / 8); 1954 | } 1955 | 1956 | if (p < bEnd) 1957 | { 1958 | memcpy(mem.data(), p, static_cast(bEnd - p)); 1959 | memsize = static_cast(bEnd - p); 1960 | } 1961 | } 1962 | 1963 | inline hash_t digest_impl() const 1964 | { 1965 | const uint8_t* p = reinterpret_cast(mem.data()); 1966 | const uint8_t* const bEnd = reinterpret_cast(mem.data()) + memsize; 1967 | hash_t hash_ret; 1968 | 1969 | if (total_len >= (bit_mode / 2)) 1970 | { 1971 | hash_ret = bit_ops::rotl(v1, 1) + bit_ops::rotl(v2, 7) + bit_ops::rotl(v3, 12) + bit_ops::rotl(v4, 18); 1972 | 1973 | if constexpr (bit_mode == 64) 1974 | { 1975 | detail::endian_align_sub_mergeround(hash_ret, v1, v2, v3, v4); 1976 | } 1977 | } 1978 | else 1979 | { 1980 | hash_ret = v3 + detail::PRIME(5); 1981 | } 1982 | 1983 | hash_ret += static_cast>(total_len); 1984 | 1985 | return detail::endian_align_sub_ending(hash_ret, p, bEnd); 1986 | } 1987 | 1988 | public: 1989 | 1990 | hash_state_t(uint_t seed = 0) 1991 | { 1992 | static_assert(!(bit_mode != 32 && bit_mode != 64), "xxhash streaming can only be used in 32 and 64 bit modes."); 1993 | v1 = seed + detail::PRIME(1) + detail::PRIME(2); 1994 | v2 = seed + detail::PRIME(2); 1995 | v3 = seed + 0; 1996 | v4 = seed - detail::PRIME(1); 1997 | }; 1998 | 1999 | void reset(uint_t seed = 0) 2000 | { 2001 | memset(this, 0, sizeof(hash_state_t)); 2002 | v1 = seed + detail::PRIME(1) + detail::PRIME(2); 2003 | v2 = seed + detail::PRIME(2); 2004 | v3 = seed + 0; 2005 | v4 = seed - detail::PRIME(1); 2006 | } 2007 | 2008 | void update(const void* input, size_t length) 2009 | { 2010 | return update_impl(input, length); 2011 | } 2012 | 2013 | template 2014 | void update(const std::basic_string& input) 2015 | { 2016 | return update_impl(static_cast(input.data()), input.length() * sizeof(T)); 2017 | } 2018 | 2019 | template 2020 | void update(ContiguousIterator begin, ContiguousIterator end) 2021 | { 2022 | using T = typename std::decay_t; 2023 | return update_impl(static_cast(&*begin), (end - begin) * sizeof(T)); 2024 | } 2025 | 2026 | template 2027 | void update(const std::vector& input) 2028 | { 2029 | return update_impl(static_cast(input.data()), input.size() * sizeof(T)); 2030 | } 2031 | 2032 | template 2033 | void update(const std::array& input) 2034 | { 2035 | return update_impl(static_cast(input.data()), AN * sizeof(T)); 2036 | } 2037 | 2038 | template 2039 | void update(const std::initializer_list& input) 2040 | { 2041 | return update_impl(static_cast(input.begin()), input.size() * sizeof(T)); 2042 | } 2043 | 2044 | hash_t digest() const 2045 | { 2046 | return digest_impl(); 2047 | } 2048 | }; 2049 | 2050 | using hash_state32_t = hash_state_t<32>; 2051 | using hash_state64_t = hash_state_t<64>; 2052 | 2053 | 2054 | /* ************************************* 2055 | * Hash streaming - xxhash3 2056 | ***************************************/ 2057 | 2058 | template 2059 | class alignas(64) hash3_state_t 2060 | { 2061 | constexpr static int internal_buffer_size = 256; 2062 | constexpr static int internal_buffer_stripes = (internal_buffer_size / detail3::stripe_len); 2063 | 2064 | alignas(64) uint64_t acc[8]; 2065 | alignas(64) uint8_t customSecret[detail3::secret_default_size]; /* used to store a custom secret generated from the seed. Makes state larger. Design might change */ 2066 | alignas(64) uint8_t buffer[internal_buffer_size]; 2067 | uint32_t bufferedSize = 0; 2068 | uint32_t nbStripesPerBlock = 0; 2069 | uint32_t nbStripesSoFar = 0; 2070 | uint32_t secretLimit = 0; 2071 | uint32_t reserved32 = 0; 2072 | uint32_t reserved32_2 = 0; 2073 | uint64_t totalLen = 0; 2074 | uint64_t seed = 0; 2075 | bool useSeed = false; 2076 | uint64_t reserved64 = 0; 2077 | const uint8_t* secret = nullptr; /* note : there is some padding after, due to alignment on 64 bytes */ 2078 | 2079 | 2080 | void consume_stripes(uint64_t* acc, uint32_t& nbStripesSoFar, size_t totalStripes, const uint8_t* input) 2081 | { 2082 | if (nbStripesPerBlock - nbStripesSoFar <= totalStripes) /* need a scrambling operation */ 2083 | { 2084 | size_t const nbStripes = nbStripesPerBlock - nbStripesSoFar; 2085 | 2086 | detail3::accumulate(acc, input, secret + (nbStripesSoFar * detail3::secret_consume_rate), nbStripes); 2087 | detail3::scramble_acc(acc, secret + secretLimit); 2088 | detail3::accumulate(acc, input + nbStripes * detail3::stripe_len, secret, totalStripes - nbStripes); 2089 | nbStripesSoFar = (uint32_t)(totalStripes - nbStripes); 2090 | } 2091 | else 2092 | { 2093 | detail3::accumulate(acc, input, secret + (nbStripesSoFar * detail3::secret_consume_rate), totalStripes); 2094 | nbStripesSoFar += (uint32_t)totalStripes; 2095 | } 2096 | } 2097 | 2098 | void update_impl(const void* input_, size_t len) 2099 | { 2100 | const uint8_t* input = static_cast(input_); 2101 | const uint8_t* const bEnd = input + len; 2102 | 2103 | totalLen += len; 2104 | 2105 | if (bufferedSize + len <= internal_buffer_size) 2106 | { /* fill in tmp buffer */ 2107 | memcpy(buffer + bufferedSize, input, len); 2108 | bufferedSize += (uint32_t)len; 2109 | return; 2110 | } 2111 | /* input now > XXH3_INTERNALBUFFER_SIZE */ 2112 | 2113 | if (bufferedSize > 0) 2114 | { /* some input within internal buffer: fill then consume it */ 2115 | size_t const loadSize = internal_buffer_size - bufferedSize; 2116 | 2117 | memcpy(buffer + bufferedSize, input, loadSize); 2118 | input += loadSize; 2119 | consume_stripes(acc, nbStripesSoFar, internal_buffer_stripes, buffer); 2120 | bufferedSize = 0; 2121 | } 2122 | 2123 | /* consume input by full buffer quantities */ 2124 | if (input + internal_buffer_size <= bEnd) 2125 | { 2126 | const uint8_t* const limit = bEnd - internal_buffer_size; 2127 | 2128 | do 2129 | { 2130 | consume_stripes(acc, nbStripesSoFar, internal_buffer_stripes, input); 2131 | input += internal_buffer_size; 2132 | } 2133 | while (input < limit); 2134 | 2135 | memcpy(buffer + sizeof(buffer) - detail3::stripe_len, input - detail3::stripe_len, detail3::stripe_len); 2136 | } 2137 | 2138 | if (input < bEnd) 2139 | { /* some remaining input input : buffer it */ 2140 | memcpy(buffer, input, (size_t)(bEnd - input)); 2141 | bufferedSize = (uint32_t)(bEnd - input); 2142 | } 2143 | } 2144 | 2145 | void digest_long(uint64_t* acc_) 2146 | { 2147 | memcpy(acc_, acc, sizeof(acc)); /* digest locally, state remains unaltered, and can continue ingesting more input afterwards */ 2148 | 2149 | if (bufferedSize >= detail3::stripe_len) 2150 | { 2151 | size_t const totalNbStripes = (bufferedSize - 1) / detail3::stripe_len; 2152 | uint32_t nbStripesSoFar = this->nbStripesSoFar; 2153 | 2154 | consume_stripes(acc_, nbStripesSoFar, totalNbStripes, buffer); 2155 | 2156 | /* one last partial stripe */ 2157 | detail3::accumulate_512(acc_, buffer + bufferedSize - detail3::stripe_len, secret + secretLimit - detail3::secret_lastacc_start); 2158 | } 2159 | else 2160 | { /* bufferedSize < STRIPE_LEN */ 2161 | /* one last stripe */ 2162 | uint8_t lastStripe[detail3::stripe_len]; 2163 | size_t const catchupSize = detail3::stripe_len - bufferedSize; 2164 | memcpy(lastStripe, buffer + sizeof(buffer) - catchupSize, catchupSize); 2165 | memcpy(lastStripe + catchupSize, buffer, bufferedSize); 2166 | detail3::accumulate_512(acc_, lastStripe, secret + secretLimit - detail3::secret_lastacc_start); 2167 | } 2168 | } 2169 | 2170 | void reset_internal(uint64_t seed_reset, const void* secret_reset, size_t secret_size) 2171 | { 2172 | memset(this, 0, sizeof(*this)); 2173 | memcpy(acc, detail3::init_acc.data(), sizeof(detail3::init_acc)); 2174 | seed = seed_reset; 2175 | useSeed = (seed != 0); 2176 | secret = (const uint8_t*)secret_reset; 2177 | secretLimit = (uint32_t)(secret_size - detail3::stripe_len); 2178 | nbStripesPerBlock = secretLimit / detail3::secret_consume_rate; 2179 | } 2180 | 2181 | public: 2182 | 2183 | hash3_state_t operator=(hash3_state_t& other) 2184 | { 2185 | memcpy(this, &other, sizeof(hash3_state_t)); 2186 | } 2187 | 2188 | hash3_state_t(uint64_t seed = 0) 2189 | { 2190 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 streaming can only be used in 64 and 128 bit modes."); 2191 | reset(seed); 2192 | } 2193 | 2194 | hash3_state_t(const void* secret, size_t secretSize) 2195 | { 2196 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 streaming can only be used in 64 and 128 bit modes."); 2197 | reset(secret, secretSize); 2198 | } 2199 | 2200 | hash3_state_t(const void* secret, size_t secretSize, uint64_t seed) 2201 | { 2202 | static_assert(!(bit_mode != 128 && bit_mode != 64), "xxhash3 streaming can only be used in 64 and 128 bit modes."); 2203 | reset(secret, secretSize, seed); 2204 | } 2205 | 2206 | void reset(uint64_t seed = 0) 2207 | { 2208 | reset_internal(seed, detail3::default_secret, detail3::secret_default_size); 2209 | detail3::init_custom_secret(customSecret, seed); 2210 | secret = customSecret; 2211 | } 2212 | 2213 | void reset(const void* secret, size_t secretSize) 2214 | { 2215 | reset_internal(0, secret, secretSize); 2216 | } 2217 | 2218 | void reset(const void* secret, size_t secretSize, uint64_t seed) 2219 | { 2220 | reset_internal(seed, secret, secretSize); 2221 | useSeed = true; 2222 | } 2223 | 2224 | void update(const void* input, size_t len) 2225 | { 2226 | return update_impl(static_cast(input), len); 2227 | } 2228 | 2229 | template 2230 | void update(const std::basic_string& input) 2231 | { 2232 | return update_impl(static_cast(input.data()), input.length() * sizeof(T)); 2233 | } 2234 | 2235 | template 2236 | void update(ContiguousIterator begin, ContiguousIterator end) 2237 | { 2238 | using T = typename std::decay_t; 2239 | return update_impl(static_cast(&*begin), (end - begin) * sizeof(T)); 2240 | } 2241 | 2242 | template 2243 | void update(const std::vector& input) 2244 | { 2245 | return update_impl(static_cast(input.data()), input.size() * sizeof(T)); 2246 | } 2247 | 2248 | template 2249 | void update(const std::array& input) 2250 | { 2251 | return update_impl(static_cast(input.data()), AN * sizeof(T)); 2252 | } 2253 | 2254 | template 2255 | void update(const std::initializer_list& input) 2256 | { 2257 | return update_impl(static_cast(input.begin()), input.size() * sizeof(T)); 2258 | } 2259 | 2260 | hash_t digest() 2261 | { 2262 | if (totalLen > detail3::midsize_max) 2263 | { 2264 | alignas(128) hash64_t acc[detail3::acc_nb]; 2265 | 2266 | digest_long(acc); 2267 | 2268 | if constexpr (bit_mode == 64) 2269 | { 2270 | return detail3::merge_accs(acc, secret + detail3::secret_mergeaccs_start, (uint64_t)totalLen * detail::PRIME<64>(1)); 2271 | } 2272 | else 2273 | { 2274 | uint64_t const low64 = detail3::merge_accs(acc, secret + detail3::secret_mergeaccs_start, (uint64_t)totalLen * detail::PRIME<64>(1)); 2275 | uint64_t const high64 = detail3::merge_accs(acc, secret + secretLimit + detail3::stripe_len - sizeof(acc) - detail3::secret_mergeaccs_start, ~((uint64_t)totalLen * detail::PRIME<64>(2))); 2276 | 2277 | return { low64, high64 }; 2278 | } 2279 | } 2280 | else 2281 | { 2282 | if (useSeed) 2283 | { 2284 | return detail3::xxhash3_impl(buffer, totalLen, seed); 2285 | } 2286 | return detail3::xxhash3_impl(buffer, totalLen, seed, secret, secretLimit + detail3::stripe_len); 2287 | } 2288 | } 2289 | }; 2290 | 2291 | using hash3_state64_t = hash3_state_t<64>; 2292 | using hash3_state128_t = hash3_state_t<128>; 2293 | } 2294 | -------------------------------------------------------------------------------- /test/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | option(XXH_CPP_USE_AVX2 "Use AVX2 instructions for tests" OFF) 2 | 3 | function(xxh_cpp_add_test name xxh_vector) 4 | add_executable(${name} test_main.cpp) 5 | target_link_libraries(${name} PRIVATE xxhash_cpp) 6 | target_compile_definitions(${name} PRIVATE -DXXH_VECTOR=${xxh_vector}) 7 | if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" OR CMAKE_CXX_COMPILER_ID MATCHES "Clang") 8 | target_compile_options(${name} PRIVATE -Wall -Wextra -pedantic -Werror) 9 | elseif(MSVC) 10 | target_compile_options(${name} PRIVATE /W3 /WX) 11 | endif() 12 | if(XXH_CPP_USE_AVX2) 13 | if(MSVC) 14 | target_compile_options(${name} PRIVATE /arch:AVX2) 15 | else() 16 | target_compile_options(${name} PRIVATE -mavx2) 17 | endif() 18 | endif() 19 | add_test(NAME ${name} COMMAND ${name}) 20 | endfunction() 21 | 22 | xxh_cpp_add_test(test_scalar 0) 23 | xxh_cpp_add_test(test_sse2 1) 24 | xxh_cpp_add_test(test_avx2 2) 25 | -------------------------------------------------------------------------------- /test/test_main.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | #include 10 | #include 11 | 12 | #define XXH_STATIC_LINKING_ONLY 13 | 14 | #define XXH_INLINE_ALL 15 | #include "xxh3.h" 16 | #include "xxhash.hpp" 17 | 18 | 19 | #define CATCH_CONFIG_RUNNER 20 | #include "catch.hpp" 21 | 22 | void to_string(uint64_t, uint64_t) {} 23 | 24 | void to_string(xxh::uint128_t) {} 25 | 26 | template 27 | std::string byte_print(T val) 28 | { 29 | constexpr std::array hex_digits = { '0','1','2','3','4','5','6','7','8','9','A','B','C','D','E','F' }; 30 | const uint8_t* inspect_ptr = reinterpret_cast(&val); 31 | std::string output = ""; 32 | for (size_t i = 0; i < sizeof(T); i++) 33 | { 34 | output += std::string("0x") + hex_digits[(*inspect_ptr) >> 4] + hex_digits[((*inspect_ptr) & 0x0F)] + ' '; 35 | inspect_ptr++; 36 | } 37 | return output; 38 | } 39 | 40 | #define STRINGIFY(x) #x 41 | #define TO_STRING(s) STRINGIFY(s) 42 | #define RAW_PRINT(...) std::cout << std::left << std::setw(50) << TO_STRING(__VA_ARGS__) " == " << byte_print((__VA_ARGS__)) << "\n"; 43 | 44 | #define DUMB_REQUIRE(x) {all++; if (x) res++; else std::cout << "Failed: " << TO_STRING(x) << "\n";} 45 | 46 | template 47 | bool cmp(T a, T b) 48 | { 49 | return !(std::memcmp(&a, &b, sizeof(T))); 50 | } 51 | 52 | template 53 | bool cmp(T1 a, T2 b) 54 | { 55 | return !(std::memcmp(&a, &b, sizeof(T1))); 56 | } 57 | 58 | template > 59 | bool cool_cmp(T1 a, T2 b) 60 | { 61 | bool good = true; 62 | for (size_t i = 0; i < sizeof(T1); i++) 63 | { 64 | uint8_t v1 = *(static_cast(static_cast(&a)) + i); 65 | uint8_t v2 = *(static_cast(static_cast(&b)) + i); 66 | if (v1 != v2) 67 | { 68 | std::cout << "CMP Mismatch at " << i << ": " << byte_print(v1) << " | " << byte_print(v2) << std::endl; 69 | good = false; 70 | } 71 | } 72 | return good; 73 | } 74 | 75 | 76 | 77 | std::string pretty_time(uint64_t ns) 78 | { 79 | std::string out = ""; 80 | 81 | if (ns > 1000000000) 82 | { 83 | uint64_t sec = ns / 1000000000; 84 | ns %= 1000000000; 85 | out += std::to_string(sec) + "s "; 86 | } 87 | if (ns > 1000000) 88 | { 89 | uint64_t ms = ns / 1000000; 90 | ns %= 1000000; 91 | out += std::to_string(ms) + "ms "; 92 | } 93 | if (ns > 1000) 94 | { 95 | uint64_t us = ns / 1000; 96 | ns %= 1000; 97 | out += std::to_string(us) + "us "; 98 | } 99 | if (ns > 0) 100 | { 101 | out += std::to_string(ns) + "ns"; 102 | } 103 | 104 | return out; 105 | } 106 | 107 | bool operator == (XXH128_hash_t h1, xxh::hash128_t h2) 108 | { 109 | return h1.high64 == h2.high64 && h1.low64 == h2.low64; 110 | } 111 | 112 | int main(int argc, char** argv) 113 | { 114 | // Dumb but it works: have the tests run beforehand without Catch so that gdb can catch segfaults, and then repeat afterwards with catch for pretty results 115 | 116 | std::cout << "xxhash_cpp compatibility testing, vectorization setting: " << static_cast(xxh::detail3::vector_mode) << std::endl << std::endl; 117 | 118 | std::string test = "test"; 119 | xxh::hash64_t h = xxh::xxhash<64>(test); 120 | xxh::hash64_t h2 = xxh::to_canonical<64>(h); 121 | std::cout << byte_print(h) << std::endl; 122 | std::cout << byte_print(h2) << std::endl; 123 | 124 | int all = 0, res = 0; 125 | 126 | constexpr int32_t test_num = 1024; 127 | 128 | std::minstd_rand rng(static_cast(std::chrono::system_clock::now().time_since_epoch().count())); 129 | std::uniform_int_distribution dist(0, 4294967295U); 130 | 131 | 132 | for (size_t i = 0; i < test_num; i++) 133 | { 134 | size_t test_buf_size = i; 135 | 136 | std::vector input_buffer; 137 | 138 | std::array secret_min_size; 139 | std::array secret_default_size; 140 | std::array secret_plus_size; 141 | 142 | input_buffer.resize(test_buf_size); 143 | std::generate(input_buffer.begin(), input_buffer.end(), [&rng, &dist]() {return dist(rng); }); 144 | std::generate(secret_min_size.begin(), secret_min_size.end(), [&rng, &dist]() {return dist(rng); }); 145 | std::generate(secret_default_size.begin(), secret_default_size.end(), [&rng, &dist]() {return dist(rng); }); 146 | std::generate(secret_plus_size.begin(), secret_plus_size.end(), [&rng, &dist]() {return dist(rng); }); 147 | 148 | uint32_t seed = dist(rng); 149 | 150 | xxh::hash_state32_t hash_state_32_cpp(seed); 151 | xxh::hash_state64_t hash_state_64_cpp(seed); 152 | XXH32_state_t* hash_state_32_c = XXH32_createState(); 153 | XXH64_state_t* hash_state_64_c = XXH64_createState(); 154 | 155 | XXH32_reset(hash_state_32_c, seed); 156 | XXH64_reset(hash_state_64_c, seed); 157 | 158 | 159 | 160 | xxh::hash3_state64_t hash3_state_64_cpp_seed(seed); 161 | xxh::hash3_state128_t hash3_state_128_cpp_seed(seed); 162 | alignas(64) XXH3_state_t hash3_state_64_c_seed; 163 | alignas(64) XXH3_state_t hash3_state_128_c_seed; 164 | 165 | XXH3_64bits_reset_withSeed(&hash3_state_64_c_seed, seed); 166 | XXH3_128bits_reset_withSeed(&hash3_state_128_c_seed, seed); 167 | 168 | 169 | 170 | xxh::hash3_state64_t hash3_state_64_cpp_secdef(secret_default_size.data(), xxh::detail3::secret_default_size); 171 | xxh::hash3_state128_t hash3_state_128_cpp_secdef(secret_default_size.data(), xxh::detail3::secret_default_size); 172 | alignas(64) XXH3_state_t hash3_state_64_c_secdef; 173 | alignas(64) XXH3_state_t hash3_state_128_c_secdef; 174 | 175 | XXH3_64bits_reset_withSecret(&hash3_state_64_c_secdef, secret_default_size.data(), sizeof(secret_default_size)); 176 | XXH3_128bits_reset_withSecret(&hash3_state_128_c_secdef, secret_default_size.data(), sizeof(secret_default_size)); 177 | 178 | 179 | 180 | xxh::hash3_state64_t hash3_state_64_cpp_secplus(secret_plus_size.data(), sizeof(secret_plus_size)); 181 | xxh::hash3_state128_t hash3_state_128_cpp_secplus(secret_plus_size.data(), sizeof(secret_plus_size)); 182 | alignas(64) XXH3_state_t hash3_state_64_c_secplus; 183 | alignas(64) XXH3_state_t hash3_state_128_c_secplus; 184 | 185 | XXH3_64bits_reset_withSecret(&hash3_state_64_c_secplus, secret_plus_size.data(), sizeof(secret_plus_size)); 186 | XXH3_128bits_reset_withSecret(&hash3_state_128_c_secplus, secret_plus_size.data(), sizeof(secret_plus_size)); 187 | 188 | 189 | 190 | xxh::hash3_state64_t hash3_state_64_cpp_secmin(secret_min_size.data(), sizeof(secret_min_size)); 191 | xxh::hash3_state128_t hash3_state_128_cpp_secmin(secret_min_size.data(), sizeof(secret_min_size)); 192 | alignas(64) XXH3_state_t hash3_state_64_c_secmin; 193 | alignas(64) XXH3_state_t hash3_state_128_c_secmin; 194 | 195 | XXH3_64bits_reset_withSecret(&hash3_state_64_c_secmin, secret_min_size.data(), sizeof(secret_min_size)); 196 | XXH3_128bits_reset_withSecret(&hash3_state_128_c_secmin, secret_min_size.data(), sizeof(secret_min_size)); 197 | 198 | 199 | 200 | hash_state_32_cpp.update(input_buffer); 201 | hash_state_64_cpp.update(input_buffer); 202 | 203 | XXH32_update(hash_state_32_c, input_buffer.data(), test_buf_size); 204 | XXH64_update(hash_state_64_c, input_buffer.data(), test_buf_size); 205 | 206 | 207 | hash3_state_64_cpp_seed.update(input_buffer); 208 | hash3_state_128_cpp_seed.update(input_buffer); 209 | 210 | XXH3_64bits_update(&hash3_state_64_c_seed, input_buffer.data(), test_buf_size); 211 | XXH3_128bits_update(&hash3_state_128_c_seed, input_buffer.data(), test_buf_size); 212 | 213 | hash3_state_64_cpp_secdef.update(input_buffer); 214 | hash3_state_128_cpp_secdef.update(input_buffer); 215 | 216 | XXH3_64bits_update(&hash3_state_64_c_secdef, input_buffer.data(), test_buf_size); 217 | XXH3_128bits_update(&hash3_state_128_c_secdef, input_buffer.data(), test_buf_size); 218 | 219 | hash3_state_64_cpp_secplus.update(input_buffer); 220 | hash3_state_128_cpp_secplus.update(input_buffer); 221 | 222 | XXH3_64bits_update(&hash3_state_64_c_secplus, input_buffer.data(), test_buf_size); 223 | XXH3_128bits_update(&hash3_state_128_c_secplus, input_buffer.data(), test_buf_size); 224 | 225 | hash3_state_64_cpp_secmin.update(input_buffer); 226 | hash3_state_128_cpp_secmin.update(input_buffer); 227 | 228 | XXH3_64bits_update(&hash3_state_64_c_secmin, input_buffer.data(), test_buf_size); 229 | XXH3_128bits_update(&hash3_state_128_c_secmin, input_buffer.data(), test_buf_size); 230 | 231 | 232 | 233 | xxh::canonical32_t canonical_32_cpp(xxh::xxhash<32>(input_buffer, seed)); 234 | xxh::canonical64_t canonical_64_cpp(xxh::xxhash<64>(input_buffer, seed)); 235 | 236 | XXH32_canonical_t canonical_32_c; 237 | XXH64_canonical_t canonical_64_c; 238 | 239 | XXH32_canonicalFromHash(&canonical_32_c, XXH32(input_buffer.data(), test_buf_size, seed)); 240 | XXH64_canonicalFromHash(&canonical_64_c, XXH64(input_buffer.data(), test_buf_size, seed)); 241 | 242 | 243 | xxh::canonical64_t canonical3_64_cpp_seed(xxh::xxhash3<64>(input_buffer, seed)); 244 | xxh::canonical128_t canonical3_128_cpp_seed(xxh::xxhash3<128>(input_buffer, seed)); 245 | 246 | XXH64_canonical_t canonical3_64_c_seed; 247 | XXH128_canonical_t canonical3_128_c_seed; 248 | 249 | XXH64_canonicalFromHash(&canonical3_64_c_seed, XXH3_64bits_withSeed(input_buffer.data(), test_buf_size, seed)); 250 | XXH128_canonicalFromHash(&canonical3_128_c_seed, XXH3_128bits_withSeed(input_buffer.data(), test_buf_size, seed)); 251 | 252 | 253 | xxh::canonical64_t canonical3_64_cpp_secdef(xxh::xxhash3<64>(input_buffer, secret_default_size.data(), xxh::detail3::secret_default_size)); 254 | xxh::canonical128_t canonical3_128_cpp_secdef(xxh::xxhash3<128>(input_buffer, secret_default_size.data(), xxh::detail3::secret_default_size)); 255 | 256 | XXH64_canonical_t canonical3_64_c_secdef; 257 | XXH128_canonical_t canonical3_128_c_secdef; 258 | 259 | XXH64_canonicalFromHash(&canonical3_64_c_secdef, XXH3_64bits_withSecret(input_buffer.data(), test_buf_size, secret_default_size.data(), secret_default_size.size())); 260 | XXH128_canonicalFromHash(&canonical3_128_c_secdef, XXH3_128bits_withSecret(input_buffer.data(), test_buf_size, secret_default_size.data(), secret_default_size.size())); 261 | 262 | 263 | xxh::canonical64_t canonical3_64_cpp_secplus(xxh::xxhash3<64>(input_buffer, secret_plus_size.data(), secret_plus_size.size())); 264 | xxh::canonical128_t canonical3_128_cpp_secplus(xxh::xxhash3<128>(input_buffer, secret_plus_size.data(), secret_plus_size.size())); 265 | 266 | XXH64_canonical_t canonical3_64_c_secplus; 267 | XXH128_canonical_t canonical3_128_c_secplus; 268 | 269 | XXH64_canonicalFromHash(&canonical3_64_c_secplus, XXH3_64bits_withSecret(input_buffer.data(), test_buf_size, secret_plus_size.data(), secret_plus_size.size())); 270 | XXH128_canonicalFromHash(&canonical3_128_c_secplus, XXH3_128bits_withSecret(input_buffer.data(), test_buf_size, secret_plus_size.data(), secret_plus_size.size())); 271 | 272 | 273 | xxh::canonical64_t canonical3_64_cpp_secmin(xxh::xxhash3<64>(input_buffer, secret_min_size.data(), secret_min_size.size())); 274 | xxh::canonical128_t canonical3_128_cpp_secmin(xxh::xxhash3<128>(input_buffer, secret_min_size.data(), secret_min_size.size())); 275 | 276 | XXH64_canonical_t canonical3_64_c_secmin; 277 | XXH128_canonical_t canonical3_128_c_secmin; 278 | 279 | XXH64_canonicalFromHash(&canonical3_64_c_secmin, XXH3_64bits_withSecret(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size())); 280 | XXH128_canonicalFromHash(&canonical3_128_c_secmin, XXH3_128bits_withSecret(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size())); 281 | 282 | xxh::hash_state_t<64> hash_state_cmp_test1; 283 | xxh::hash_state_t<64> hash_state_cmp_test2; 284 | 285 | hash_state_cmp_test1.update(std::string("foo")); 286 | 287 | hash_state_cmp_test2 = hash_state_cmp_test1; 288 | 289 | hash_state_cmp_test1.update(std::string("bar")); 290 | hash_state_cmp_test2.update(std::string("bar")); 291 | 292 | DUMB_REQUIRE(XXH32(input_buffer.data(), test_buf_size, seed) == xxh::xxhash<32>(input_buffer, seed)); 293 | DUMB_REQUIRE(XXH64(input_buffer.data(), test_buf_size, seed) == xxh::xxhash<64>(input_buffer, seed)); 294 | 295 | DUMB_REQUIRE(XXH3_64bits_withSeed(input_buffer.data(), test_buf_size, seed) == xxh::xxhash3<64>(input_buffer, seed)); 296 | DUMB_REQUIRE(XXH3_128bits_withSeed(input_buffer.data(), test_buf_size, seed) == xxh::xxhash3<128>(input_buffer, seed)); 297 | 298 | DUMB_REQUIRE(XXH3_64bits_withSecret(input_buffer.data(), test_buf_size, secret_default_size.data(), secret_default_size.size()) == xxh::xxhash3<64>(input_buffer, secret_default_size.data(), secret_default_size.size())); 299 | DUMB_REQUIRE(XXH3_128bits_withSecret(input_buffer.data(), test_buf_size, secret_default_size.data(), secret_default_size.size()) == xxh::xxhash3<128>(input_buffer, secret_default_size.data(), secret_default_size.size())); 300 | 301 | DUMB_REQUIRE(XXH3_64bits_withSecret(input_buffer.data(), test_buf_size, secret_plus_size.data(), secret_plus_size.size()) == xxh::xxhash3<64>(input_buffer, secret_plus_size.data(), secret_plus_size.size())); 302 | DUMB_REQUIRE(XXH3_128bits_withSecret(input_buffer.data(), test_buf_size, secret_plus_size.data(), secret_plus_size.size()) == xxh::xxhash3<128>(input_buffer, secret_plus_size.data(), secret_plus_size.size())); 303 | 304 | DUMB_REQUIRE(XXH3_64bits_withSecret(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size()) == xxh::xxhash3<64>(input_buffer, secret_min_size.data(), secret_min_size.size())); 305 | DUMB_REQUIRE(XXH3_128bits_withSecret(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size()) == xxh::xxhash3<128>(input_buffer, secret_min_size.data(), secret_min_size.size())); 306 | 307 | DUMB_REQUIRE(XXH32_digest(hash_state_32_c) == hash_state_32_cpp.digest()); 308 | DUMB_REQUIRE(XXH64_digest(hash_state_64_c) == hash_state_64_cpp.digest()); 309 | 310 | DUMB_REQUIRE(XXH3_64bits_digest(&hash3_state_64_c_seed) == hash3_state_64_cpp_seed.digest()); 311 | DUMB_REQUIRE(XXH3_128bits_digest(&hash3_state_128_c_seed) == hash3_state_128_cpp_seed.digest()); 312 | 313 | DUMB_REQUIRE(XXH3_64bits_digest(&hash3_state_64_c_secdef) == hash3_state_64_cpp_secdef.digest()); 314 | DUMB_REQUIRE(XXH3_128bits_digest(&hash3_state_128_c_secdef) == hash3_state_128_cpp_secdef.digest()); 315 | 316 | 317 | 318 | DUMB_REQUIRE(XXH3_64bits_digest(&hash3_state_64_c_secplus) == hash3_state_64_cpp_secplus.digest()); 319 | DUMB_REQUIRE(XXH3_128bits_digest(&hash3_state_128_c_secplus) == hash3_state_128_cpp_secplus.digest()); 320 | 321 | DUMB_REQUIRE(XXH3_64bits_digest(&hash3_state_64_c_secmin) == hash3_state_64_cpp_secmin.digest()); 322 | DUMB_REQUIRE(XXH3_128bits_digest(&hash3_state_128_c_secmin) == hash3_state_128_cpp_secmin.digest()); 323 | 324 | 325 | 326 | DUMB_REQUIRE(XXH32_hashFromCanonical(&canonical_32_c) == canonical_32_cpp.get_hash()); 327 | DUMB_REQUIRE(XXH64_hashFromCanonical(&canonical_64_c) == canonical_64_cpp.get_hash()); 328 | 329 | DUMB_REQUIRE(XXH64_hashFromCanonical(&canonical3_64_c_seed) == canonical3_64_cpp_seed.get_hash()); 330 | DUMB_REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_seed).high64 == canonical3_128_cpp_seed.get_hash().high64); 331 | DUMB_REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_seed).low64 == canonical3_128_cpp_seed.get_hash().low64); 332 | 333 | DUMB_REQUIRE(XXH64_hashFromCanonical(&canonical3_64_c_secdef) == canonical3_64_cpp_secdef.get_hash()); 334 | DUMB_REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_secdef).high64 == canonical3_128_cpp_secdef.get_hash().high64); 335 | DUMB_REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_secdef).low64 == canonical3_128_cpp_secdef.get_hash().low64); 336 | 337 | DUMB_REQUIRE(XXH64_hashFromCanonical(&canonical3_64_c_secplus) == canonical3_64_cpp_secplus.get_hash()); 338 | DUMB_REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_secplus).high64 == canonical3_128_cpp_secplus.get_hash().high64); 339 | DUMB_REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_secplus).low64 == canonical3_128_cpp_secplus.get_hash().low64); 340 | 341 | DUMB_REQUIRE(XXH64_hashFromCanonical(&canonical3_64_c_secmin) == canonical3_64_cpp_secmin.get_hash()); 342 | DUMB_REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_secmin).high64 == canonical3_128_cpp_secmin.get_hash().high64); 343 | DUMB_REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_secmin).low64 == canonical3_128_cpp_secmin.get_hash().low64); 344 | 345 | DUMB_REQUIRE(hash_state_cmp_test1.digest() == hash_state_cmp_test2.digest()); 346 | } 347 | 348 | std::cout << "Dumb test results: " << res << " / " << all << " (" << ((float)res * 100) / (float)all << ")\n"; 349 | 350 | return Catch::Session().run(argc, argv); 351 | } 352 | 353 | TEST_CASE("Results are the same as the original implementation for small inputs", "[compatibility]") 354 | { 355 | uint32_t alternating_data[] = { 0x00000000, 0xFFFFFFFF, 0x00000000, 0xFFFFFFFF, 0x00000000, 0xFFFFFFFF, 0x00000000, 0xFFFFFFFF }; 356 | uint32_t zero_data[] = { 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000 }; 357 | uint32_t one_data[] = { 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF }; 358 | uint32_t alternating_data2[] = { 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA }; 359 | 360 | REQUIRE(XXH64(alternating_data, 32, 0) == xxh::xxhash<64>(alternating_data, 32, 0)); 361 | REQUIRE(XXH64(zero_data, 32, 0) == xxh::xxhash<64>(zero_data, 32, 0)); 362 | REQUIRE(XXH64(one_data, 32, 0) == xxh::xxhash<64>(one_data, 32, 0)); 363 | REQUIRE(XXH64(alternating_data2, 32, 0) == xxh::xxhash<64>(alternating_data2, 32, 0)); 364 | 365 | REQUIRE(XXH32(alternating_data, 32, 0) == xxh::xxhash<32>(alternating_data, 32, 0)); 366 | REQUIRE(XXH32(zero_data, 32, 0) == xxh::xxhash<32>(zero_data, 32, 0)); 367 | REQUIRE(XXH32(one_data, 32, 0) == xxh::xxhash<32>(one_data, 32, 0)); 368 | REQUIRE(XXH32(alternating_data2, 32, 0) == xxh::xxhash<32>(alternating_data2, 32, 0)); 369 | 370 | REQUIRE(XXH128(alternating_data, 32, 0).high64 == xxh::xxhash3<128>(alternating_data, 32, 0).high64); 371 | REQUIRE(XXH128(alternating_data, 32, 0).low64 == xxh::xxhash3<128>(alternating_data, 32, 0).low64); 372 | REQUIRE(XXH128(zero_data, 32, 0).high64 == xxh::xxhash3<128>(zero_data, 32, 0).high64); 373 | REQUIRE(XXH128(zero_data, 32, 0).low64 == xxh::xxhash3<128>(zero_data, 32, 0).low64); 374 | REQUIRE(XXH128(one_data, 32, 0).high64 == xxh::xxhash3<128>(one_data, 32, 0).high64); 375 | REQUIRE(XXH128(one_data, 32, 0).low64 == xxh::xxhash3<128>(one_data, 32, 0).low64); 376 | REQUIRE(XXH128(alternating_data2, 32, 0).high64 == xxh::xxhash3<128>(alternating_data2, 32, 0).high64); 377 | REQUIRE(XXH128(alternating_data2, 32, 0).low64 == xxh::xxhash3<128>(alternating_data2, 32, 0).low64); 378 | 379 | REQUIRE(XXH3_64bits_withSeed(alternating_data, 32, 0) == xxh::xxhash3<64>(alternating_data, 32, 0)); 380 | REQUIRE(XXH3_64bits_withSeed(zero_data, 32, 0) == xxh::xxhash3<64>(zero_data, 32, 0)); 381 | REQUIRE(XXH3_64bits_withSeed(one_data, 32, 0) == xxh::xxhash3<64>(one_data, 32, 0)); 382 | REQUIRE(XXH3_64bits_withSeed(alternating_data2, 32, 0) == xxh::xxhash3<64>(alternating_data2, 32, 0)); 383 | } 384 | 385 | TEST_CASE("Results are the same as the original implementation for large, randomly generated inputs", "[compatibility]") 386 | { 387 | constexpr int32_t test_num = 1024; 388 | 389 | std::minstd_rand rng(static_cast(std::chrono::system_clock::now().time_since_epoch().count())); 390 | std::uniform_int_distribution dist(0, 4294967295U); 391 | 392 | 393 | for (size_t i = 0; i < test_num; i++) 394 | { 395 | //std::cout << "starting tests for size: " << i << std::endl; 396 | 397 | size_t test_buf_size = i; 398 | 399 | std::vector input_buffer; 400 | 401 | std::array secret_min_size; 402 | std::array secret_default_size; 403 | std::array secret_plus_size; 404 | 405 | input_buffer.resize(test_buf_size); 406 | std::generate(input_buffer.begin(), input_buffer.end(), [&rng, &dist]() {return dist(rng); }); 407 | std::generate(secret_min_size.begin(), secret_min_size.end(), [&rng, &dist]() {return dist(rng); }); 408 | std::generate(secret_default_size.begin(), secret_default_size.end(), [&rng, &dist]() {return dist(rng); }); 409 | std::generate(secret_plus_size.begin(), secret_plus_size.end(), [&rng, &dist]() {return dist(rng); }); 410 | 411 | uint32_t seed = dist(rng); 412 | 413 | xxh::hash_state32_t hash_state_32_cpp(seed); 414 | xxh::hash_state64_t hash_state_64_cpp(seed); 415 | alignas(64) XXH32_state_t hash_state_32_c; 416 | alignas(64) XXH64_state_t hash_state_64_c; 417 | 418 | XXH32_reset(&hash_state_32_c, seed); 419 | XXH64_reset(&hash_state_64_c, seed); 420 | 421 | 422 | 423 | xxh::hash3_state64_t hash3_state_64_cpp_seed(seed); 424 | xxh::hash3_state128_t hash3_state_128_cpp_seed(seed); 425 | alignas(64) XXH3_state_t hash3_state_64_c_seed; 426 | alignas(64) XXH3_state_t hash3_state_128_c_seed; 427 | 428 | XXH3_64bits_reset_withSeed(&hash3_state_64_c_seed, seed); 429 | XXH3_128bits_reset_withSeed(&hash3_state_128_c_seed, seed); 430 | 431 | 432 | 433 | xxh::hash3_state64_t hash3_state_64_cpp_secdef(secret_default_size.data(), xxh::detail3::secret_default_size); 434 | xxh::hash3_state128_t hash3_state_128_cpp_secdef(secret_default_size.data(), xxh::detail3::secret_default_size); 435 | alignas(64) XXH3_state_t hash3_state_64_c_secdef; 436 | alignas(64) XXH3_state_t hash3_state_128_c_secdef; 437 | 438 | XXH3_64bits_reset_withSecret(&hash3_state_64_c_secdef, secret_default_size.data(), sizeof(secret_default_size)); 439 | XXH3_128bits_reset_withSecret(&hash3_state_128_c_secdef, secret_default_size.data(), sizeof(secret_default_size)); 440 | 441 | 442 | 443 | xxh::hash3_state64_t hash3_state_64_cpp_secplus(secret_plus_size.data(), sizeof(secret_plus_size)); 444 | xxh::hash3_state128_t hash3_state_128_cpp_secplus( secret_plus_size.data(), sizeof(secret_plus_size)); 445 | alignas(64) XXH3_state_t hash3_state_64_c_secplus; 446 | alignas(64) XXH3_state_t hash3_state_128_c_secplus; 447 | 448 | XXH3_64bits_reset_withSecret(&hash3_state_64_c_secplus, secret_plus_size.data(), sizeof(secret_plus_size)); 449 | XXH3_128bits_reset_withSecret(&hash3_state_128_c_secplus, secret_plus_size.data(), sizeof(secret_plus_size)); 450 | 451 | 452 | 453 | xxh::hash3_state64_t hash3_state_64_cpp_secmin(secret_min_size.data(), sizeof(secret_min_size)); 454 | xxh::hash3_state128_t hash3_state_128_cpp_secmin(secret_min_size.data(), sizeof(secret_min_size)); 455 | alignas(64) XXH3_state_t hash3_state_64_c_secmin; 456 | alignas(64) XXH3_state_t hash3_state_128_c_secmin; 457 | 458 | XXH3_64bits_reset_withSecret(&hash3_state_64_c_secmin, secret_min_size.data(), sizeof(secret_min_size)); 459 | XXH3_128bits_reset_withSecret(&hash3_state_128_c_secmin, secret_min_size.data(), sizeof(secret_min_size)); 460 | 461 | 462 | 463 | xxh::hash3_state64_t hash3_state_64_cpp_secdef_seed(secret_default_size.data(), xxh::detail3::secret_default_size, seed); 464 | xxh::hash3_state128_t hash3_state_128_cpp_secdef_seed(secret_default_size.data(), xxh::detail3::secret_default_size, seed); 465 | alignas(64) XXH3_state_t hash3_state_64_c_secdef_seed; 466 | alignas(64) XXH3_state_t hash3_state_128_c_secdef_seed; 467 | 468 | XXH3_64bits_reset_withSecretandSeed(&hash3_state_64_c_secdef_seed, secret_default_size.data(), sizeof(secret_default_size), seed); 469 | XXH3_128bits_reset_withSecretandSeed(&hash3_state_128_c_secdef_seed, secret_default_size.data(), sizeof(secret_default_size), seed); 470 | 471 | 472 | 473 | xxh::hash3_state64_t hash3_state_64_cpp_secplus_seed(secret_plus_size.data(), sizeof(secret_plus_size), seed); 474 | xxh::hash3_state128_t hash3_state_128_cpp_secplus_seed(secret_plus_size.data(), sizeof(secret_plus_size), seed); 475 | alignas(64) XXH3_state_t hash3_state_64_c_secplus_seed; 476 | alignas(64) XXH3_state_t hash3_state_128_c_secplus_seed; 477 | 478 | XXH3_64bits_reset_withSecretandSeed(&hash3_state_64_c_secplus_seed, secret_plus_size.data(), sizeof(secret_plus_size), seed); 479 | XXH3_128bits_reset_withSecretandSeed(&hash3_state_128_c_secplus_seed, secret_plus_size.data(), sizeof(secret_plus_size), seed); 480 | 481 | 482 | 483 | xxh::hash3_state64_t hash3_state_64_cpp_secmin_seed(secret_min_size.data(), sizeof(secret_min_size), seed); 484 | xxh::hash3_state128_t hash3_state_128_cpp_secmin_seed(secret_min_size.data(), sizeof(secret_min_size), seed); 485 | alignas(64) XXH3_state_t hash3_state_64_c_secmin_seed; 486 | alignas(64) XXH3_state_t hash3_state_128_c_secmin_seed; 487 | 488 | XXH3_64bits_reset_withSecretandSeed(&hash3_state_64_c_secmin_seed, secret_min_size.data(), sizeof(secret_min_size), seed); 489 | XXH3_128bits_reset_withSecretandSeed(&hash3_state_128_c_secmin_seed, secret_min_size.data(), sizeof(secret_min_size), seed); 490 | 491 | 492 | 493 | xxh::hash3_state64_t hash3_state_64_cpp_sec_zero_seed(secret_min_size.data(), sizeof(secret_min_size), 0); 494 | xxh::hash3_state128_t hash3_state_128_cpp_sec_zero_seed(secret_min_size.data(), sizeof(secret_min_size), 0); 495 | alignas(64) XXH3_state_t hash3_state_64_c_sec_zero_seed; 496 | alignas(64) XXH3_state_t hash3_state_128_c_sec_zero_seed; 497 | 498 | XXH3_64bits_reset_withSecretandSeed(&hash3_state_64_c_sec_zero_seed, secret_min_size.data(), sizeof(secret_min_size), 0); 499 | XXH3_128bits_reset_withSecretandSeed(&hash3_state_128_c_sec_zero_seed, secret_min_size.data(), sizeof(secret_min_size), 0); 500 | 501 | 502 | 503 | 504 | hash_state_32_cpp.update(input_buffer); 505 | hash_state_64_cpp.update(input_buffer); 506 | 507 | XXH32_update(&hash_state_32_c, input_buffer.data(), test_buf_size); 508 | XXH64_update(&hash_state_64_c, input_buffer.data(), test_buf_size); 509 | 510 | 511 | hash3_state_64_cpp_seed.update(input_buffer); 512 | hash3_state_128_cpp_seed.update(input_buffer); 513 | 514 | XXH3_64bits_update(&hash3_state_64_c_seed, input_buffer.data(), test_buf_size); 515 | XXH3_128bits_update(&hash3_state_128_c_seed, input_buffer.data(), test_buf_size); 516 | 517 | hash3_state_64_cpp_secdef.update(input_buffer); 518 | hash3_state_128_cpp_secdef.update(input_buffer); 519 | 520 | XXH3_64bits_update(&hash3_state_64_c_secdef, input_buffer.data(), test_buf_size); 521 | XXH3_128bits_update(&hash3_state_128_c_secdef, input_buffer.data(), test_buf_size); 522 | 523 | hash3_state_64_cpp_secplus.update(input_buffer); 524 | hash3_state_128_cpp_secplus.update(input_buffer); 525 | 526 | XXH3_64bits_update(&hash3_state_64_c_secplus, input_buffer.data(), test_buf_size); 527 | XXH3_128bits_update(&hash3_state_128_c_secplus, input_buffer.data(), test_buf_size); 528 | 529 | hash3_state_64_cpp_secmin.update(input_buffer); 530 | hash3_state_128_cpp_secmin.update(input_buffer); 531 | 532 | XXH3_64bits_update(&hash3_state_64_c_secmin, input_buffer.data(), test_buf_size); 533 | XXH3_128bits_update(&hash3_state_128_c_secmin, input_buffer.data(), test_buf_size); 534 | 535 | hash3_state_64_cpp_secdef_seed.update(input_buffer); 536 | hash3_state_128_cpp_secdef_seed.update(input_buffer); 537 | 538 | XXH3_64bits_update(&hash3_state_64_c_secdef_seed, input_buffer.data(), test_buf_size); 539 | XXH3_128bits_update(&hash3_state_128_c_secdef_seed, input_buffer.data(), test_buf_size); 540 | 541 | hash3_state_64_cpp_secplus_seed.update(input_buffer); 542 | hash3_state_128_cpp_secplus_seed.update(input_buffer); 543 | 544 | XXH3_64bits_update(&hash3_state_64_c_secplus_seed, input_buffer.data(), test_buf_size); 545 | XXH3_128bits_update(&hash3_state_128_c_secplus_seed, input_buffer.data(), test_buf_size); 546 | 547 | hash3_state_64_cpp_secmin_seed.update(input_buffer); 548 | hash3_state_128_cpp_secmin_seed.update(input_buffer); 549 | 550 | XXH3_64bits_update(&hash3_state_64_c_secmin_seed, input_buffer.data(), test_buf_size); 551 | XXH3_128bits_update(&hash3_state_128_c_secmin_seed, input_buffer.data(), test_buf_size); 552 | 553 | hash3_state_64_cpp_sec_zero_seed.update(input_buffer); 554 | hash3_state_128_cpp_sec_zero_seed.update(input_buffer); 555 | 556 | XXH3_64bits_update(&hash3_state_64_c_sec_zero_seed, input_buffer.data(), test_buf_size); 557 | XXH3_128bits_update(&hash3_state_128_c_sec_zero_seed, input_buffer.data(), test_buf_size); 558 | 559 | 560 | 561 | xxh::canonical32_t canonical_32_cpp(xxh::xxhash<32>(input_buffer, seed)); 562 | xxh::canonical64_t canonical_64_cpp(xxh::xxhash<64>(input_buffer, seed)); 563 | 564 | XXH32_canonical_t canonical_32_c; 565 | XXH64_canonical_t canonical_64_c; 566 | 567 | XXH32_canonicalFromHash(&canonical_32_c, XXH32(input_buffer.data(), test_buf_size, seed)); 568 | XXH64_canonicalFromHash(&canonical_64_c, XXH64(input_buffer.data(), test_buf_size, seed)); 569 | 570 | 571 | xxh::canonical64_t canonical3_64_cpp_seed(xxh::xxhash3<64>(input_buffer, seed)); 572 | xxh::canonical128_t canonical3_128_cpp_seed(xxh::xxhash3<128>(input_buffer, seed)); 573 | 574 | XXH64_canonical_t canonical3_64_c_seed; 575 | XXH128_canonical_t canonical3_128_c_seed; 576 | 577 | XXH64_canonicalFromHash(&canonical3_64_c_seed, XXH3_64bits_withSeed(input_buffer.data(), test_buf_size, seed)); 578 | XXH128_canonicalFromHash(&canonical3_128_c_seed, XXH3_128bits_withSeed(input_buffer.data(), test_buf_size, seed)); 579 | 580 | 581 | xxh::canonical64_t canonical3_64_cpp_secdef(xxh::xxhash3<64>(input_buffer, secret_default_size.data(), xxh::detail3::secret_default_size)); 582 | xxh::canonical128_t canonical3_128_cpp_secdef(xxh::xxhash3<128>(input_buffer, secret_default_size.data(), xxh::detail3::secret_default_size)); 583 | 584 | XXH64_canonical_t canonical3_64_c_secdef; 585 | XXH128_canonical_t canonical3_128_c_secdef; 586 | 587 | XXH64_canonicalFromHash(&canonical3_64_c_secdef, XXH3_64bits_withSecret(input_buffer.data(), test_buf_size, secret_default_size.data(), secret_default_size.size())); 588 | XXH128_canonicalFromHash(&canonical3_128_c_secdef, XXH3_128bits_withSecret(input_buffer.data(), test_buf_size, secret_default_size.data(), secret_default_size.size())); 589 | 590 | 591 | xxh::canonical64_t canonical3_64_cpp_secplus(xxh::xxhash3<64>(input_buffer, secret_plus_size.data(), secret_plus_size.size())); 592 | xxh::canonical128_t canonical3_128_cpp_secplus(xxh::xxhash3<128>(input_buffer, secret_plus_size.data(), secret_plus_size.size())); 593 | 594 | XXH64_canonical_t canonical3_64_c_secplus; 595 | XXH128_canonical_t canonical3_128_c_secplus; 596 | 597 | XXH64_canonicalFromHash(&canonical3_64_c_secplus, XXH3_64bits_withSecret(input_buffer.data(), test_buf_size, secret_plus_size.data(), secret_plus_size.size())); 598 | XXH128_canonicalFromHash(&canonical3_128_c_secplus, XXH3_128bits_withSecret(input_buffer.data(), test_buf_size, secret_plus_size.data(), secret_plus_size.size())); 599 | 600 | 601 | xxh::canonical64_t canonical3_64_cpp_secmin(xxh::xxhash3<64>(input_buffer, secret_min_size.data(), secret_min_size.size())); 602 | xxh::canonical128_t canonical3_128_cpp_secmin(xxh::xxhash3<128>(input_buffer, secret_min_size.data(), secret_min_size.size())); 603 | 604 | XXH64_canonical_t canonical3_64_c_secmin; 605 | XXH128_canonical_t canonical3_128_c_secmin; 606 | 607 | XXH64_canonicalFromHash(&canonical3_64_c_secmin, XXH3_64bits_withSecret(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size())); 608 | XXH128_canonicalFromHash(&canonical3_128_c_secmin, XXH3_128bits_withSecret(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size())); 609 | 610 | REQUIRE(XXH32(input_buffer.data(), test_buf_size, seed) == xxh::xxhash<32>(input_buffer, static_cast(seed))); 611 | REQUIRE(XXH64(input_buffer.data(), test_buf_size, seed) == xxh::xxhash<64>(input_buffer, seed)); 612 | 613 | std::array acc_c; 614 | std::array acc_cpp; 615 | 616 | std::array input; 617 | 618 | std::generate(acc_c.begin(), acc_c.end(), []() {return 999999; }); 619 | std::generate(acc_cpp.begin(), acc_cpp.end(), []() {return 999999; }); 620 | std::generate(input.begin(), input.end(), []() {return 999999; }); 621 | 622 | XXH3_accumulate(acc_c.data(), secret_default_size.data(), secret_min_size.data(), 2); 623 | xxh::detail3::accumulate(acc_cpp.data(), secret_default_size.data(), secret_min_size.data(), 2); 624 | 625 | REQUIRE(acc_c[0] == acc_cpp[0]); 626 | REQUIRE(acc_c[1] == acc_cpp[1]); 627 | REQUIRE(acc_c[2] == acc_cpp[2]); 628 | REQUIRE(acc_c[3] == acc_cpp[3]); 629 | REQUIRE(acc_c[4] == acc_cpp[4]); 630 | REQUIRE(acc_c[5] == acc_cpp[5]); 631 | REQUIRE(acc_c[6] == acc_cpp[6]); 632 | REQUIRE(acc_c[7] == acc_cpp[7]); 633 | 634 | REQUIRE(XXH3_len_9to16_64b((const uint8_t*)acc_c.data(), 16, XXH3_kSecret, seed) == xxh::detail3::len_9to16<64>((const uint8_t*)acc_c.data(), 16, XXH3_kSecret, seed)); 635 | REQUIRE(XXH3_len_4to8_64b((const uint8_t*)acc_c.data(), 8, XXH3_kSecret, seed) == xxh::detail3::len_4to8<64>((const uint8_t*)acc_c.data(), 8, XXH3_kSecret, seed)); 636 | REQUIRE(XXH3_len_1to3_64b((const uint8_t*)acc_c.data(), 3, XXH3_kSecret, seed) == xxh::detail3::len_1to3<64>((const uint8_t*)acc_c.data(), 3, XXH3_kSecret, seed)); 637 | REQUIRE(XXH3_len_0to16_64b((const uint8_t*)acc_c.data(), 0, XXH3_kSecret, seed) == xxh::detail3::len_0to16<64>((const uint8_t*)acc_c.data(), 0, XXH3_kSecret, seed)); 638 | 639 | REQUIRE(XXH3_len_9to16_128b((const uint8_t*)acc_c.data(), 16, XXH3_kSecret, seed) == xxh::detail3::len_9to16<128>((const uint8_t*)acc_c.data(), 16, XXH3_kSecret, seed)); 640 | REQUIRE(XXH3_len_4to8_128b((const uint8_t*)acc_c.data(), 8, XXH3_kSecret, seed) == xxh::detail3::len_4to8<128>((const uint8_t*)acc_c.data(), 8, XXH3_kSecret, seed)); 641 | REQUIRE(XXH3_len_1to3_128b((const uint8_t*)acc_c.data(), 3, XXH3_kSecret, seed) == xxh::detail3::len_1to3<128>((const uint8_t*)acc_c.data(), 3, XXH3_kSecret, seed)); 642 | REQUIRE(XXH3_len_0to16_128b((const uint8_t*)acc_c.data(), 0, XXH3_kSecret, seed) == xxh::detail3::len_0to16<128>((const uint8_t*)acc_c.data(), 0, XXH3_kSecret, seed)); 643 | 644 | XXH3_accumulate_512(acc_c.data(), input.data(), secret_min_size.data()); 645 | xxh::detail3::accumulate_512((void*)acc_cpp.data(), (const void*)input.data(), (const void*)secret_min_size.data()); 646 | 647 | REQUIRE(acc_c[0] == acc_cpp[0]); 648 | REQUIRE(acc_c[1] == acc_cpp[1]); 649 | REQUIRE(acc_c[2] == acc_cpp[2]); 650 | REQUIRE(acc_c[3] == acc_cpp[3]); 651 | REQUIRE(acc_c[4] == acc_cpp[4]); 652 | REQUIRE(acc_c[5] == acc_cpp[5]); 653 | REQUIRE(acc_c[6] == acc_cpp[6]); 654 | REQUIRE(acc_c[7] == acc_cpp[7]); 655 | 656 | XXH3_scrambleAcc(acc_c.data(), secret_min_size.data()); 657 | xxh::detail3::scramble_acc(acc_cpp.data(), secret_min_size.data()); 658 | 659 | REQUIRE(acc_c[0] == acc_cpp[0]); 660 | REQUIRE(acc_c[1] == acc_cpp[1]); 661 | REQUIRE(acc_c[2] == acc_cpp[2]); 662 | REQUIRE(acc_c[3] == acc_cpp[3]); 663 | REQUIRE(acc_c[4] == acc_cpp[4]); 664 | REQUIRE(acc_c[5] == acc_cpp[5]); 665 | REQUIRE(acc_c[6] == acc_cpp[6]); 666 | REQUIRE(acc_c[7] == acc_cpp[7]); 667 | 668 | alignas(64) std::array secret_c; 669 | alignas(64) std::array secret_cpp; 670 | std::generate(secret_c.begin(), secret_c.end(), []() {return 0x7F; }); 671 | std::generate(secret_cpp.begin(), secret_cpp.end(), []() {return 0x7F; }); 672 | 673 | REQUIRE(secret_c[0] == secret_cpp[0]); 674 | REQUIRE(secret_c[1] == secret_cpp[1]); 675 | REQUIRE(secret_c[2] == secret_cpp[2]); 676 | REQUIRE(secret_c[3] == secret_cpp[3]); 677 | REQUIRE(secret_c[4] == secret_cpp[4]); 678 | REQUIRE(secret_c[5] == secret_cpp[5]); 679 | REQUIRE(secret_c[6] == secret_cpp[6]); 680 | REQUIRE(secret_c[7] == secret_cpp[7]); 681 | 682 | REQUIRE(XXH3_64bits_withSeed(input_buffer.data(), test_buf_size, seed) == xxh::xxhash3<64>(input_buffer, seed)); 683 | REQUIRE(XXH3_128bits_withSeed(input_buffer.data(), test_buf_size, seed) == xxh::xxhash3<128>(input_buffer, seed)); 684 | 685 | REQUIRE(XXH3_64bits_withSecret(input_buffer.data(), test_buf_size, secret_default_size.data(), secret_default_size.size()) == xxh::xxhash3<64>(input_buffer, secret_default_size.data(), secret_default_size.size())); 686 | REQUIRE(XXH3_128bits_withSecret(input_buffer.data(), test_buf_size, secret_default_size.data(), secret_default_size.size()) == xxh::xxhash3<128>(input_buffer, secret_default_size.data(), secret_default_size.size())); 687 | 688 | REQUIRE(XXH3_64bits_withSecret(input_buffer.data(), test_buf_size, secret_plus_size.data(), secret_plus_size.size()) == xxh::xxhash3<64>(input_buffer, secret_plus_size.data(), secret_plus_size.size())); 689 | REQUIRE(XXH3_128bits_withSecret(input_buffer.data(), test_buf_size, secret_plus_size.data(), secret_plus_size.size()) == xxh::xxhash3<128>(input_buffer, secret_plus_size.data(), secret_plus_size.size())); 690 | 691 | REQUIRE(XXH3_64bits_withSecret(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size()) == xxh::xxhash3<64>(input_buffer, secret_min_size.data(), secret_min_size.size())); 692 | REQUIRE(XXH3_128bits_withSecret(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size()) == xxh::xxhash3<128>(input_buffer, secret_min_size.data(), secret_min_size.size())); 693 | 694 | REQUIRE(XXH3_64bits_withSecretandSeed(input_buffer.data(), test_buf_size, secret_default_size.data(), secret_default_size.size(), seed) == xxh::xxhash3<64>(input_buffer, secret_default_size.data(), secret_default_size.size(), seed)); 695 | REQUIRE(XXH3_128bits_withSecretandSeed(input_buffer.data(), test_buf_size, secret_default_size.data(), secret_default_size.size(), seed) == xxh::xxhash3<128>(input_buffer, secret_default_size.data(), secret_default_size.size(), seed)); 696 | 697 | REQUIRE(XXH3_64bits_withSecretandSeed(input_buffer.data(), test_buf_size, secret_plus_size.data(), secret_plus_size.size(), seed) == xxh::xxhash3<64>(input_buffer, secret_plus_size.data(), secret_plus_size.size(), seed)); 698 | REQUIRE(XXH3_128bits_withSecretandSeed(input_buffer.data(), test_buf_size, secret_plus_size.data(), secret_plus_size.size(), seed) == xxh::xxhash3<128>(input_buffer, secret_plus_size.data(), secret_plus_size.size(), seed)); 699 | 700 | REQUIRE(XXH3_64bits_withSecretandSeed(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size(), seed) == xxh::xxhash3<64>(input_buffer, secret_min_size.data(), secret_min_size.size(), seed)); 701 | REQUIRE(XXH3_128bits_withSecretandSeed(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size(), seed) == xxh::xxhash3<128>(input_buffer, secret_min_size.data(), secret_min_size.size(), seed)); 702 | 703 | 704 | REQUIRE(XXH32_digest(&hash_state_32_c) == hash_state_32_cpp.digest()); 705 | REQUIRE(XXH64_digest(&hash_state_64_c) == hash_state_64_cpp.digest()); 706 | 707 | REQUIRE(XXH3_64bits_digest(&hash3_state_64_c_seed) == hash3_state_64_cpp_seed.digest()); 708 | REQUIRE(XXH3_128bits_digest(&hash3_state_128_c_seed) == hash3_state_128_cpp_seed.digest()); 709 | 710 | REQUIRE(XXH3_64bits_digest(&hash3_state_64_c_secdef) == hash3_state_64_cpp_secdef.digest()); 711 | REQUIRE(XXH3_128bits_digest(&hash3_state_128_c_secdef) == hash3_state_128_cpp_secdef.digest()); 712 | 713 | REQUIRE(XXH3_64bits_digest(&hash3_state_64_c_secplus) == hash3_state_64_cpp_secplus.digest()); 714 | REQUIRE(XXH3_128bits_digest(&hash3_state_128_c_secplus) == hash3_state_128_cpp_secplus.digest()); 715 | 716 | REQUIRE(XXH3_64bits_digest(&hash3_state_64_c_secmin) == hash3_state_64_cpp_secmin.digest()); 717 | REQUIRE(XXH3_128bits_digest(&hash3_state_128_c_secmin) == hash3_state_128_cpp_secmin.digest()); 718 | 719 | REQUIRE(XXH3_64bits_digest(&hash3_state_64_c_secdef_seed) == hash3_state_64_cpp_secdef_seed.digest()); 720 | REQUIRE(XXH3_128bits_digest(&hash3_state_128_c_secdef_seed) == hash3_state_128_cpp_secdef_seed.digest()); 721 | 722 | REQUIRE(XXH3_64bits_digest(&hash3_state_64_c_secplus_seed) == hash3_state_64_cpp_secplus_seed.digest()); 723 | REQUIRE(XXH3_128bits_digest(&hash3_state_128_c_secplus_seed) == hash3_state_128_cpp_secplus_seed.digest()); 724 | 725 | REQUIRE(XXH3_64bits_digest(&hash3_state_64_c_secmin_seed) == hash3_state_64_cpp_secmin_seed.digest()); 726 | REQUIRE(XXH3_128bits_digest(&hash3_state_128_c_secmin_seed) == hash3_state_128_cpp_secmin_seed.digest()); 727 | 728 | const auto digest_c_64_zero_seed = XXH3_64bits_digest(&hash3_state_64_c_sec_zero_seed); 729 | const auto digest_c_128_zero_seed = XXH3_128bits_digest(&hash3_state_128_c_sec_zero_seed); 730 | const auto digest_cpp_64_zero_seed = hash3_state_64_cpp_sec_zero_seed.digest(); 731 | const auto digest_cpp_128_zero_seed = hash3_state_128_cpp_sec_zero_seed.digest(); 732 | 733 | REQUIRE(digest_c_64_zero_seed == digest_cpp_64_zero_seed); 734 | REQUIRE(digest_c_128_zero_seed == digest_cpp_128_zero_seed); 735 | 736 | REQUIRE(digest_c_64_zero_seed == XXH3_64bits_withSecretandSeed(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size(), 0)); 737 | REQUIRE(digest_cpp_64_zero_seed == xxh::xxhash3<64>(input_buffer, secret_min_size.data(), secret_min_size.size(), 0)); 738 | REQUIRE(digest_c_128_zero_seed.high64 == XXH3_128bits_withSecretandSeed(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size(), 0).high64); 739 | REQUIRE(digest_cpp_128_zero_seed.high64 == xxh::xxhash3<128>(input_buffer, secret_min_size.data(), secret_min_size.size(), 0).high64); 740 | REQUIRE(digest_c_128_zero_seed.low64 == XXH3_128bits_withSecretandSeed(input_buffer.data(), test_buf_size, secret_min_size.data(), secret_min_size.size(), 0).low64); 741 | REQUIRE(digest_cpp_128_zero_seed.low64 == xxh::xxhash3<128>(input_buffer, secret_min_size.data(), secret_min_size.size(), 0).low64); 742 | 743 | REQUIRE(XXH32_hashFromCanonical(&canonical_32_c) == canonical_32_cpp.get_hash()); 744 | REQUIRE(XXH64_hashFromCanonical(&canonical_64_c) == canonical_64_cpp.get_hash()); 745 | 746 | REQUIRE(XXH64_hashFromCanonical(&canonical3_64_c_seed) == canonical3_64_cpp_seed.get_hash()); 747 | REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_seed).high64 == canonical3_128_cpp_seed.get_hash().high64); 748 | REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_seed).low64 == canonical3_128_cpp_seed.get_hash().low64); 749 | 750 | REQUIRE(XXH64_hashFromCanonical(&canonical3_64_c_secdef) == canonical3_64_cpp_secdef.get_hash()); 751 | REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_secdef).high64 == canonical3_128_cpp_secdef.get_hash().high64); 752 | REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_secdef).low64 == canonical3_128_cpp_secdef.get_hash().low64); 753 | 754 | REQUIRE(XXH64_hashFromCanonical(&canonical3_64_c_secplus) == canonical3_64_cpp_secplus.get_hash()); 755 | REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_secplus).high64 == canonical3_128_cpp_secplus.get_hash().high64); 756 | REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_secplus).low64 == canonical3_128_cpp_secplus.get_hash().low64); 757 | 758 | REQUIRE(XXH64_hashFromCanonical(&canonical3_64_c_secmin) == canonical3_64_cpp_secmin.get_hash()); 759 | REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_secmin).high64 == canonical3_128_cpp_secmin.get_hash().high64); 760 | REQUIRE(XXH128_hashFromCanonical(&canonical3_128_c_secmin).low64 == canonical3_128_cpp_secmin.get_hash().low64); 761 | 762 | std::vector secret_buffer_c; 763 | std::vector secret_buffer_cpp; 764 | secret_buffer_c.resize(std::max(static_cast(xxh::detail3::secret_default_size), i)); 765 | secret_buffer_cpp.resize(std::max(static_cast(xxh::detail3::secret_default_size), i)); 766 | 767 | std::vector secret_seed_buffer_c; 768 | std::vector secret_seed_buffer_cpp; 769 | secret_seed_buffer_c.resize(xxh::detail3::secret_default_size); 770 | secret_seed_buffer_cpp.resize(xxh::detail3::secret_default_size); 771 | 772 | XXH_INLINE_XXH3_generateSecret(secret_buffer_c.data(), secret_buffer_c.size(), acc_c.data(), acc_c.size()); 773 | xxh::generate_secret(secret_buffer_cpp.data(), secret_buffer_cpp.size(), acc_c.data(), acc_c.size()); 774 | 775 | XXH_INLINE_XXH3_generateSecret_fromSeed(secret_seed_buffer_c.data(), seed); 776 | xxh::generate_secret_from_seed(secret_seed_buffer_cpp.data(), seed); 777 | 778 | REQUIRE(memcmp(secret_buffer_c.data(), secret_buffer_cpp.data(), secret_buffer_c.size()) == 0); 779 | REQUIRE(memcmp(secret_seed_buffer_c.data(), secret_seed_buffer_cpp.data(), secret_seed_buffer_c.size()) == 0); 780 | 781 | std::string data = "kekl"; 782 | 783 | #define checksum_bits 128 784 | static_assert(checksum_bits == 64 || checksum_bits == 128); 785 | 786 | xxh::hash3_state_t state; 787 | state.update(data.data(), data.size()); 788 | 789 | // convert checksum to canonical byte order 790 | xxh::canonical_t const canonical{ state.digest() }; 791 | auto const hash{ canonical.get_hash() }; 792 | 793 | #if checksum_bits == 128 794 | to_string(hash.low64, hash.high64); 795 | #else 796 | to_string(hash); 797 | #endif 798 | } 799 | } 800 | 801 | 802 | TEST_CASE("Printing results for inter-platform comparison", "[platform]") 803 | { 804 | uint32_t alternating_data[] = { 0x00000000, 0xFFFFFFFF, 0x00000000, 0xFFFFFFFF, 0x00000000, 0xFFFFFFFF, 0x00000000, 0xFFFFFFFF }; 805 | uint32_t zero_data[] = { 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000 }; 806 | uint32_t one_data[] = { 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF }; 807 | uint32_t alternating_data2[] = { 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA }; 808 | 809 | RAW_PRINT(xxh::xxhash<32>(alternating_data, 32, 0)); 810 | RAW_PRINT(xxh::xxhash<64>(alternating_data, 32, 0)); 811 | RAW_PRINT(xxh::xxhash<32>(alternating_data, 32, 65536)); 812 | RAW_PRINT(xxh::xxhash<64>(alternating_data, 32, 65536)); 813 | RAW_PRINT(xxh::xxhash3<64>(alternating_data, 32, 0)); 814 | RAW_PRINT(xxh::xxhash3<64>(alternating_data, 32, 65536)); 815 | RAW_PRINT(xxh::xxhash3<128>(alternating_data, 32, 0)); 816 | RAW_PRINT(xxh::xxhash3<128>(alternating_data, 32, 65536)); 817 | 818 | RAW_PRINT(xxh::xxhash<32>(zero_data, 32, 0)); 819 | RAW_PRINT(xxh::xxhash<64>(zero_data, 32, 0)); 820 | RAW_PRINT(xxh::xxhash<32>(zero_data, 32, 65536)); 821 | RAW_PRINT(xxh::xxhash<64>(zero_data, 32, 65536)); 822 | RAW_PRINT(xxh::xxhash3<64>(zero_data, 32, 0)); 823 | RAW_PRINT(xxh::xxhash3<64>(zero_data, 32, 65536)); 824 | RAW_PRINT(xxh::xxhash3<128>(zero_data, 32, 0)); 825 | RAW_PRINT(xxh::xxhash3<128>(zero_data, 32, 65536)); 826 | 827 | RAW_PRINT(xxh::xxhash<32>(one_data, 32, 0)); 828 | RAW_PRINT(xxh::xxhash<64>(one_data, 32, 0)); 829 | RAW_PRINT(xxh::xxhash<32>(one_data, 32, 65536)); 830 | RAW_PRINT(xxh::xxhash<64>(one_data, 32, 65536)); 831 | RAW_PRINT(xxh::xxhash3<64>(one_data, 32, 0)); 832 | RAW_PRINT(xxh::xxhash3<64>(one_data, 32, 65536)); 833 | RAW_PRINT(xxh::xxhash3<128>(one_data, 32, 0)); 834 | RAW_PRINT(xxh::xxhash3<128>(one_data, 32, 65536)); 835 | 836 | RAW_PRINT(xxh::xxhash<32>(alternating_data2, 32, 0)); 837 | RAW_PRINT(xxh::xxhash<64>(alternating_data2, 32, 0)); 838 | RAW_PRINT(xxh::xxhash<32>(alternating_data2, 32, 65536)); 839 | RAW_PRINT(xxh::xxhash<64>(alternating_data2, 32, 65536)); 840 | RAW_PRINT(xxh::xxhash3<64>(alternating_data2, 32, 0)); 841 | RAW_PRINT(xxh::xxhash3<64>(alternating_data2, 32, 65536)); 842 | RAW_PRINT(xxh::xxhash3<128>(alternating_data2, 32, 0)); 843 | RAW_PRINT(xxh::xxhash3<128>(alternating_data2, 32, 65536)); 844 | } 845 | 846 | TEST_CASE("Checking c++ assignment operator","[c++ compliance]") 847 | { 848 | xxh::hash_state_t<64> hash_state_cmp_test1; 849 | xxh::hash_state_t<64> hash_state_cmp_test2; 850 | 851 | hash_state_cmp_test1.update(std::string("foo")); 852 | 853 | hash_state_cmp_test2 = hash_state_cmp_test1; 854 | 855 | hash_state_cmp_test1.update(std::string("bar")); 856 | hash_state_cmp_test2.update(std::string("bar")); 857 | 858 | REQUIRE(hash_state_cmp_test1.digest() == hash_state_cmp_test2.digest()); 859 | } 860 | -------------------------------------------------------------------------------- /test/xxh3.h: -------------------------------------------------------------------------------- 1 | /* 2 | * xxHash - Extremely Fast Hash algorithm 3 | * Development source file for `xxh3` 4 | * Copyright (C) 2019-2020 Yann Collet 5 | * 6 | * BSD 2-Clause License (https://www.opensource.org/licenses/bsd-license.php) 7 | * 8 | * Redistribution and use in source and binary forms, with or without 9 | * modification, are permitted provided that the following conditions are 10 | * met: 11 | * 12 | * * Redistributions of source code must retain the above copyright 13 | * notice, this list of conditions and the following disclaimer. 14 | * * Redistributions in binary form must reproduce the above 15 | * copyright notice, this list of conditions and the following disclaimer 16 | * in the documentation and/or other materials provided with the 17 | * distribution. 18 | * 19 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 20 | * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 21 | * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 22 | * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 23 | * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 24 | * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 25 | * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 26 | * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 27 | * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 28 | * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | * 31 | * You can contact the author at: 32 | * - xxHash homepage: https://www.xxhash.com 33 | * - xxHash source repository: https://github.com/Cyan4973/xxHash 34 | */ 35 | 36 | /* 37 | * Note: This file used to host the source code of XXH3_* variants. 38 | * during the development period. 39 | * The source code is now properly integrated within xxhash.h. 40 | * 41 | * xxh3.h is no longer useful, 42 | * but it is still provided for compatibility with source code 43 | * which used to include it directly. 44 | * 45 | * Programs are now highly discourage to include xxh3.h. 46 | * Include `xxhash.h` instead, which is the officially supported interface. 47 | * 48 | * In the future, xxh3.h will start to generate warnings, then errors, 49 | * then it will be removed from source package and from include directory. 50 | */ 51 | 52 | /* Simulate the same impact as including the old xxh3.h source file */ 53 | 54 | #define XXH_INLINE_ALL 55 | #include "xxhash.h" 56 | --------------------------------------------------------------------------------