├── .github └── workflows │ └── .github-ci.yml ├── .gitignore ├── GxHash.Benchmarks ├── Benchmarks │ └── ThroughputBenchmark.cs ├── GxHash.Benchmarks.csproj ├── Program.cs ├── SummaryConsoleLogger.cs └── ThroughputColumn.cs ├── GxHash.Tests ├── GxHash.Tests.csproj └── GxHashTests.cs ├── GxHash.Utils ├── GxHash.Utils.csproj ├── Noop.cs ├── QualificationUtils.cs ├── RandomObjectUtils.cs └── UnsafeUtils.cs ├── GxHash.sln ├── GxHash ├── GxHash.cs └── GxHash.csproj ├── LICENSE.txt └── README.md /.github/workflows/.github-ci.yml: -------------------------------------------------------------------------------- 1 | name: Build Test Benchmark 2 | 3 | on: [push] 4 | 5 | jobs: 6 | benchmark: 7 | runs-on: ${{ matrix.os }} 8 | strategy: 9 | matrix: 10 | os: [ 'ubuntu-latest', 'macos-14' ] 11 | steps: 12 | - uses: actions/checkout@v3 13 | - name: Setup .NET Core SDK 14 | uses: actions/setup-dotnet@v3 15 | with: 16 | dotnet-version: '8.0' 17 | - name: Install dependencies 18 | run: dotnet restore 19 | - name: Build 20 | run: dotnet build -c Release --no-restore 21 | - name: Test 22 | run: dotnet test -c Release --no-restore 23 | - name: Benchmark 24 | run: dotnet run -c Release --no-restore --project "GxHash.Benchmarks" -- -f '*' 25 | - name: Upload benchmark results 26 | uses: actions/upload-artifact@v3 27 | with: 28 | name: benchmarks 29 | path: 'benchmarks/results' -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | ## Ignore Visual Studio temporary files, build results, and 2 | ## files generated by popular Visual Studio add-ons. 3 | ## 4 | ## Get latest from https://github.com/github/gitignore/blob/master/VisualStudio.gitignore 5 | 6 | **/.idea 7 | 8 | # User-specific files 9 | *.rsuser 10 | *.suo 11 | *.user 12 | *.userosscache 13 | *.sln.docstates 14 | 15 | # User-specific files (MonoDevelop/Xamarin Studio) 16 | *.userprefs 17 | 18 | # Mono auto generated files 19 | mono_crash.* 20 | 21 | # Build results 22 | [Dd]ebug/ 23 | [Dd]ebugPublic/ 24 | [Rr]elease/ 25 | [Rr]eleases/ 26 | x64/ 27 | x86/ 28 | [Aa][Rr][Mm]/ 29 | [Aa][Rr][Mm]64/ 30 | bld/ 31 | [Bb]in/ 32 | [Oo]bj/ 33 | [Ll]og/ 34 | [Ll]ogs/ 35 | 36 | # Visual Studio 2015/2017 cache/options directory 37 | .vs/ 38 | # Uncomment if you have tasks that create the project's static files in wwwroot 39 | #wwwroot/ 40 | 41 | # Visual Studio 2017 auto generated files 42 | Generated\ Files/ 43 | 44 | # MSTest test Results 45 | [Tt]est[Rr]esult*/ 46 | [Bb]uild[Ll]og.* 47 | 48 | # NUnit 49 | *.VisualState.xml 50 | TestResult.xml 51 | nunit-*.xml 52 | 53 | # Build Results of an ATL Project 54 | [Dd]ebugPS/ 55 | [Rr]eleasePS/ 56 | dlldata.c 57 | 58 | # Benchmark Results 59 | BenchmarkDotNet.Artifacts/ 60 | 61 | # .NET Core 62 | project.lock.json 63 | project.fragment.lock.json 64 | artifacts/ 65 | 66 | # StyleCop 67 | StyleCopReport.xml 68 | 69 | # Files built by Visual Studio 70 | *_i.c 71 | *_p.c 72 | *_h.h 73 | *.ilk 74 | *.meta 75 | *.obj 76 | *.iobj 77 | *.pch 78 | *.pdb 79 | *.ipdb 80 | *.pgc 81 | *.pgd 82 | *.rsp 83 | *.sbr 84 | *.tlb 85 | *.tli 86 | *.tlh 87 | *.tmp 88 | *.tmp_proj 89 | *_wpftmp.csproj 90 | *.log 91 | *.vspscc 92 | *.vssscc 93 | .builds 94 | *.pidb 95 | *.svclog 96 | *.scc 97 | 98 | # Chutzpah Test files 99 | _Chutzpah* 100 | 101 | # Visual C++ cache files 102 | ipch/ 103 | *.aps 104 | *.ncb 105 | *.opendb 106 | *.opensdf 107 | *.sdf 108 | *.cachefile 109 | *.VC.db 110 | *.VC.VC.opendb 111 | 112 | # Visual Studio profiler 113 | *.psess 114 | *.vsp 115 | *.vspx 116 | *.sap 117 | 118 | # Visual Studio Trace Files 119 | *.e2e 120 | 121 | # TFS 2012 Local Workspace 122 | $tf/ 123 | 124 | # Guidance Automation Toolkit 125 | *.gpState 126 | 127 | # ReSharper is a .NET coding add-in 128 | _ReSharper*/ 129 | *.[Rr]e[Ss]harper 130 | *.DotSettings.user 131 | 132 | # TeamCity is a build add-in 133 | _TeamCity* 134 | 135 | # DotCover is a Code Coverage Tool 136 | *.dotCover 137 | 138 | # AxoCover is a Code Coverage Tool 139 | .axoCover/* 140 | !.axoCover/settings.json 141 | 142 | # Visual Studio code coverage results 143 | *.coverage 144 | *.coveragexml 145 | 146 | # NCrunch 147 | _NCrunch_* 148 | .*crunch*.local.xml 149 | nCrunchTemp_* 150 | 151 | # MightyMoose 152 | *.mm.* 153 | AutoTest.Net/ 154 | 155 | # Web workbench (sass) 156 | .sass-cache/ 157 | 158 | # Installshield output folder 159 | [Ee]xpress/ 160 | 161 | # DocProject is a documentation generator add-in 162 | DocProject/buildhelp/ 163 | DocProject/Help/*.HxT 164 | DocProject/Help/*.HxC 165 | DocProject/Help/*.hhc 166 | DocProject/Help/*.hhk 167 | DocProject/Help/*.hhp 168 | DocProject/Help/Html2 169 | DocProject/Help/html 170 | 171 | # Click-Once directory 172 | publish/ 173 | 174 | # Publish Web Output 175 | *.[Pp]ublish.xml 176 | *.azurePubxml 177 | # Note: Comment the next line if you want to checkin your web deploy settings, 178 | # but database connection strings (with potential passwords) will be unencrypted 179 | *.pubxml 180 | *.publishproj 181 | 182 | # Microsoft Azure Web App publish settings. Comment the next line if you want to 183 | # checkin your Azure Web App publish settings, but sensitive information contained 184 | # in these scripts will be unencrypted 185 | PublishScripts/ 186 | 187 | # NuGet Packages 188 | *.nupkg 189 | # NuGet Symbol Packages 190 | *.snupkg 191 | # The packages folder can be ignored because of Package Restore 192 | **/[Pp]ackages/* 193 | # except build/, which is used as an MSBuild target. 194 | !**/[Pp]ackages/build/ 195 | # Uncomment if necessary however generally it will be regenerated when needed 196 | #!**/[Pp]ackages/repositories.config 197 | # NuGet v3's project.json files produces more ignorable files 198 | *.nuget.props 199 | *.nuget.targets 200 | 201 | # Microsoft Azure Build Output 202 | csx/ 203 | *.build.csdef 204 | 205 | # Microsoft Azure Emulator 206 | ecf/ 207 | rcf/ 208 | 209 | # Windows Store app package directories and files 210 | AppPackages/ 211 | BundleArtifacts/ 212 | Package.StoreAssociation.xml 213 | _pkginfo.txt 214 | *.appx 215 | *.appxbundle 216 | *.appxupload 217 | 218 | # Visual Studio cache files 219 | # files ending in .cache can be ignored 220 | *.[Cc]ache 221 | # but keep track of directories ending in .cache 222 | !?*.[Cc]ache/ 223 | 224 | # Others 225 | ClientBin/ 226 | ~$* 227 | *~ 228 | *.dbmdl 229 | *.dbproj.schemaview 230 | *.jfm 231 | *.pfx 232 | *.publishsettings 233 | orleans.codegen.cs 234 | 235 | # Including strong name files can present a security risk 236 | # (https://github.com/github/gitignore/pull/2483#issue-259490424) 237 | #*.snk 238 | 239 | # Since there are multiple workflows, uncomment next line to ignore bower_components 240 | # (https://github.com/github/gitignore/pull/1529#issuecomment-104372622) 241 | #bower_components/ 242 | 243 | # RIA/Silverlight projects 244 | Generated_Code/ 245 | 246 | # Backup & report files from converting an old project file 247 | # to a newer Visual Studio version. Backup files are not needed, 248 | # because we have git ;-) 249 | _UpgradeReport_Files/ 250 | Backup*/ 251 | UpgradeLog*.XML 252 | UpgradeLog*.htm 253 | ServiceFabricBackup/ 254 | *.rptproj.bak 255 | 256 | # SQL Server files 257 | *.mdf 258 | *.ldf 259 | *.ndf 260 | 261 | # Business Intelligence projects 262 | *.rdl.data 263 | *.bim.layout 264 | *.bim_*.settings 265 | *.rptproj.rsuser 266 | *- [Bb]ackup.rdl 267 | *- [Bb]ackup ([0-9]).rdl 268 | *- [Bb]ackup ([0-9][0-9]).rdl 269 | 270 | # Microsoft Fakes 271 | FakesAssemblies/ 272 | 273 | # GhostDoc plugin setting file 274 | *.GhostDoc.xml 275 | 276 | # Node.js Tools for Visual Studio 277 | .ntvs_analysis.dat 278 | node_modules/ 279 | 280 | # Visual Studio 6 build log 281 | *.plg 282 | 283 | # Visual Studio 6 workspace options file 284 | *.opt 285 | 286 | # Visual Studio 6 auto-generated workspace file (contains which files were open etc.) 287 | *.vbw 288 | 289 | # Visual Studio LightSwitch build output 290 | **/*.HTMLClient/GeneratedArtifacts 291 | **/*.DesktopClient/GeneratedArtifacts 292 | **/*.DesktopClient/ModelManifest.xml 293 | **/*.Server/GeneratedArtifacts 294 | **/*.Server/ModelManifest.xml 295 | _Pvt_Extensions 296 | 297 | # Paket dependency manager 298 | .paket/paket.exe 299 | paket-files/ 300 | 301 | # FAKE - F# Make 302 | .fake/ 303 | 304 | # CodeRush personal settings 305 | .cr/personal 306 | 307 | # Python Tools for Visual Studio (PTVS) 308 | __pycache__/ 309 | *.pyc 310 | 311 | # Cake - Uncomment if you are using it 312 | # tools/** 313 | # !tools/packages.config 314 | 315 | # Tabs Studio 316 | *.tss 317 | 318 | # Telerik's JustMock configuration file 319 | *.jmconfig 320 | 321 | # BizTalk build output 322 | *.btp.cs 323 | *.btm.cs 324 | *.odx.cs 325 | *.xsd.cs 326 | 327 | # OpenCover UI analysis results 328 | OpenCover/ 329 | 330 | # Azure Stream Analytics local run output 331 | ASALocalRun/ 332 | 333 | # MSBuild Binary and Structured Log 334 | *.binlog 335 | 336 | # NVidia Nsight GPU debugger configuration file 337 | *.nvuser 338 | 339 | # MFractors (Xamarin productivity tool) working folder 340 | .mfractor/ 341 | 342 | # Local History for Visual Studio 343 | .localhistory/ 344 | 345 | # BeatPulse healthcheck temp database 346 | healthchecksdb 347 | 348 | # Backup folder for Package Reference Convert tool in Visual Studio 2017 349 | MigrationBackup/ 350 | 351 | # Ionide (cross platform F# VS Code tools) working folder 352 | .ionide/ 353 | 354 | # Smart specific 355 | **/build-artifacts 356 | benchmarks/Equativ.Hashing.Qualification/*.jpg 357 | benchmarks/results/* 358 | benchmarks/Equativ.Hashing.Qualification/**/*.jpg 359 | -------------------------------------------------------------------------------- /GxHash.Benchmarks/Benchmarks/ThroughputBenchmark.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using BenchmarkDotNet.Attributes; 3 | using System.Collections.Generic; 4 | using System.IO.Hashing; 5 | using System.Runtime.InteropServices; 6 | using BenchmarkDotNet.Order; 7 | using System.Security.Cryptography; 8 | using GxHash.Utils; 9 | 10 | namespace GxHash.Benchmarks; 11 | 12 | [Orderer(SummaryOrderPolicy.Declared)] 13 | [Throughput] 14 | public class ThroughputBenchmark 15 | { 16 | [Benchmark(Baseline = true)] 17 | public int Marvin() => string.GetHashCode(Value); 18 | 19 | [Benchmark] 20 | public int XxH3() => (int)(long)XxHash3.HashToUInt64(MemoryMarshal.AsBytes(Value.AsSpan()), _seed); 21 | 22 | [Benchmark] 23 | public int GxHash32() => GxHash.Hash32(MemoryMarshal.AsBytes(Value.AsSpan()), _useed); 24 | 25 | private static readonly UInt128 _useed = new UInt128(BitConverter.ToUInt64(RandomNumberGenerator.GetBytes(8)), BitConverter.ToUInt64(RandomNumberGenerator.GetBytes(8))); 26 | private static readonly long _seed = BitConverter.ToInt64(RandomNumberGenerator.GetBytes(8)); 27 | 28 | [ParamsSource(nameof(GetData))] 29 | public string Value; 30 | 31 | public IEnumerable GetData() 32 | { 33 | // Using similar values as what https://github.com/rurban/smhasher uses 34 | for (int i = 1; i < 14; i++) 35 | { 36 | // Testing on aligned buffers (favorable) 37 | int chars = (int)Math.Pow(2, i); 38 | yield return RandomObjectUtils.CreateRandomString( 39 | minSize: chars, 40 | maxSize: chars, 41 | seed: chars, 42 | charSet: "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"); 43 | } 44 | } 45 | } -------------------------------------------------------------------------------- /GxHash.Benchmarks/GxHash.Benchmarks.csproj: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | Exe 5 | net8.0 6 | true 7 | false 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | -------------------------------------------------------------------------------- /GxHash.Benchmarks/Program.cs: -------------------------------------------------------------------------------- 1 | using System.Collections.Generic; 2 | using BenchmarkDotNet.Configs; 3 | using BenchmarkDotNet.Loggers; 4 | using BenchmarkDotNet.Running; 5 | 6 | var config = ManualConfig.Create(DefaultConfig.Instance); 7 | ((List)config.GetLoggers()).Clear(); // BDN api... 🙄 8 | config.AddLogger(new SummaryConsoleLogger()); 9 | 10 | BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args, config); -------------------------------------------------------------------------------- /GxHash.Benchmarks/SummaryConsoleLogger.cs: -------------------------------------------------------------------------------- 1 | using System.Text.RegularExpressions; 2 | using System.Threading; 3 | using BenchmarkDotNet.Loggers; 4 | 5 | /// 6 | /// Only prints errors and end summary table 7 | /// 8 | public class SummaryConsoleLogger : ILogger 9 | { 10 | private readonly ConsoleLogger _logger = new(); 11 | private readonly Regex _regex = new(@"\/\/ \* [A-Za-z]* \*"); 12 | private long _isSummarySection = 0; 13 | 14 | public void Write(LogKind logKind, string text) 15 | { 16 | TryWrite(logKind, text); 17 | } 18 | 19 | public void WriteLine() 20 | { 21 | if (Interlocked.Read(ref _isSummarySection) == 1) 22 | { 23 | _logger.WriteLine(); 24 | } 25 | } 26 | 27 | public void WriteLine(LogKind logKind, string text) 28 | { 29 | if (TryWrite(logKind, text)) 30 | { 31 | _logger.WriteLine(); 32 | } 33 | } 34 | 35 | private bool TryWrite(LogKind logKind, string text) 36 | { 37 | var isSummarySection = _regex.IsMatch(text) ? Interlocked.Exchange(ref _isSummarySection, text == "// * Summary *" ? 1 : 0) : Interlocked.Read(ref _isSummarySection); 38 | 39 | if ((logKind == LogKind.Error && !text.StartsWith("Failed to set up high priority")) 40 | || logKind == LogKind.Help 41 | || (isSummarySection == 1 && logKind == LogKind.Statistic)) 42 | { 43 | _logger.Write(logKind, text); 44 | return true; 45 | } 46 | 47 | //_logger.Write(logKind, "█"); 48 | return false; 49 | } 50 | 51 | public void Flush() => _logger.Flush(); 52 | 53 | public string Id => _logger.Id; 54 | public int Priority => _logger.Priority; 55 | } -------------------------------------------------------------------------------- /GxHash.Benchmarks/ThroughputColumn.cs: -------------------------------------------------------------------------------- 1 | using BenchmarkDotNet.Columns; 2 | using BenchmarkDotNet.Configs; 3 | using BenchmarkDotNet.Reports; 4 | using BenchmarkDotNet.Running; 5 | using System; 6 | using System.Linq; 7 | 8 | namespace GxHash.Benchmarks; 9 | 10 | [AttributeUsage(AttributeTargets.Class)] 11 | public class ThroughputAttribute : Attribute, IConfigSource 12 | { 13 | public IConfig Config { get; } 14 | 15 | public ThroughputAttribute(bool displayGenColumns = true) 16 | { 17 | Config = ManualConfig.CreateEmpty().AddColumn(new ThroughputColumn()); 18 | } 19 | } 20 | 21 | public class ThroughputColumn : IColumn 22 | { 23 | public string Id => "Throughput"; 24 | 25 | public string ColumnName => "Throughput (MiB/s)"; 26 | 27 | public bool AlwaysShow => true; 28 | 29 | public ColumnCategory Category => ColumnCategory.Custom; 30 | 31 | public int PriorityInCategory => 0; 32 | 33 | public bool IsNumeric => true; 34 | 35 | public UnitType UnitType => UnitType.Dimensionless; 36 | 37 | public bool IsAvailable(Summary summary) => true; 38 | 39 | public bool IsDefault(Summary summary, BenchmarkCase benchmarkCase) => false; 40 | 41 | public string GetValue(Summary summary, BenchmarkCase benchmarkCase, SummaryStyle style) 42 | { 43 | return GetValue(summary, benchmarkCase); 44 | } 45 | 46 | public string GetValue(Summary summary, BenchmarkCase benchmarkCase) 47 | { 48 | var report = summary.Reports.FirstOrDefault(x => x.BenchmarkCase == benchmarkCase); 49 | if (report?.ResultStatistics == null) 50 | { 51 | return "N/A"; 52 | } 53 | 54 | int inputSize = 1; 55 | var parameter = benchmarkCase.Parameters.Items.FirstOrDefault(x => x.Value is string); 56 | if (parameter != null && parameter.Value is string str) 57 | { 58 | inputSize = str.Length * sizeof(char); 59 | } 60 | 61 | double coeff = (summary.Style?.TimeUnit?.NanosecondAmount ?? 1d) * 1_000_000_000d / (1024d * 1024d); 62 | coeff *= inputSize; 63 | 64 | double mean = report.ResultStatistics.Mean; 65 | double meanThroughput = coeff / mean; 66 | 67 | double errorThroughput = coeff / report.ResultStatistics.ConfidenceInterval.Lower - coeff / report.ResultStatistics.ConfidenceInterval.Upper; 68 | 69 | return $"{meanThroughput:F2} ± {errorThroughput:F2}"; 70 | } 71 | 72 | public string Legend => "Throughput"; 73 | } 74 | -------------------------------------------------------------------------------- /GxHash.Tests/GxHash.Tests.csproj: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | net8.0 5 | true 6 | false 7 | false 8 | 9 | 10 | 11 | 12 | all 13 | runtime; build; native; contentfiles; analyzers; buildtransitive 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | -------------------------------------------------------------------------------- /GxHash.Tests/GxHashTests.cs: -------------------------------------------------------------------------------- 1 | using NUnit.Framework; 2 | using System; 3 | using System.Collections.Generic; 4 | using System.Linq; 5 | 6 | namespace GxHash.Tests; 7 | 8 | public class GxHashTests 9 | { 10 | [Test] 11 | public void ValuesTest() 12 | { 13 | Assert.AreEqual(456576800, GxHash.Hash(Array.Empty().AsSpan(), 0), "Unexpected hash value"); 14 | Assert.AreEqual(978957914, GxHash.Hash(new byte[1].AsSpan(), 0), "Unexpected hash value"); 15 | Assert.AreEqual(3325885698, GxHash.Hash(new byte[1000].AsSpan(), 0), "Unexpected hash value"); 16 | Assert.AreEqual(1741430579, GxHash.Hash(Enumerable.Repeat((byte)42, 4242).ToArray().AsSpan(), 42), "Unexpected hash value"); 17 | } 18 | 19 | [Test] 20 | public void SanityChecks() 21 | { 22 | HashSet hashes = new HashSet(); 23 | 24 | // Check that zero filled inputs are hashes differently depending on their size 25 | byte[] bytes = new byte[1000]; 26 | for (int i = 0; i < bytes.Length; i++) 27 | { 28 | ReadOnlySpan slice = bytes.AsSpan().Slice(0, i); 29 | long hash = GxHash.Hash64(slice, 42); 30 | Assert.AreNotEqual(0L, hash, "Zero hash!"); 31 | Assert.IsTrue(hashes.Add(hash), "Collision!"); 32 | } 33 | 34 | // Check that zero padding affects output hash 35 | hashes.Clear(); 36 | bytes[0] = 123; 37 | for (int i = 0; i < bytes.Length; i++) 38 | { 39 | ReadOnlySpan slice = bytes.AsSpan().Slice(0, i); 40 | long hash = GxHash.Hash64(slice, 42); 41 | Assert.AreNotEqual(0L, hash, "Zero hash!"); 42 | Assert.IsTrue(hashes.Add(hash), "Collision!"); 43 | } 44 | 45 | // Check that we don't hash beyond input data 46 | Random.Shared.NextBytes(bytes); 47 | for (int i = 0; i < bytes.Length - 100; i++) 48 | { 49 | ReadOnlySpan slice = bytes.AsSpan().Slice(100, i); 50 | long hashBefore = GxHash.Hash64(slice, 42); 51 | // Randomize bytes right before/after input bounds 52 | Random.Shared.NextBytes(bytes.AsSpan().Slice(0, 100)); 53 | Random.Shared.NextBytes(bytes.AsSpan().Slice(100 + i)); 54 | long hashAfter = GxHash.Hash64(slice, 42); 55 | Assert.AreEqual(hashBefore, hashAfter, "Hash depends on out of bounds data!"); 56 | } 57 | } 58 | 59 | [Test] 60 | public void AllBytesAreRead() 61 | { 62 | for (int s = 0; s < 1200; s++) { 63 | byte[] bytes = new byte[s]; 64 | int hash = GxHash.Hash32(bytes, 42); 65 | 66 | for (int i = 0; i < s; i++) { 67 | byte swap = bytes[i]; 68 | bytes[i] = 82; 69 | int newHash = GxHash.Hash32(bytes, 42); 70 | bytes[i] = swap; 71 | 72 | Assert.AreNotEqual(hash, newHash, $"byte {i} not processed for input of size {s}"); 73 | } 74 | } 75 | } 76 | 77 | [TestCase(1, 0, 1)] 78 | [TestCase(1, 0, 16)] 79 | [TestCase(1, 0, 32)] 80 | [TestCase(16, 0, 16)] 81 | [TestCase(16, 0, 32)] 82 | [TestCase(16, 16, 32)] 83 | [TestCase(16, 32, 48)] 84 | [TestCase(32, 0, 32)] 85 | [TestCase(32, 0, 64)] 86 | [TestCase(32, 32, 64)] 87 | [TestCase(32, 64, 96)] 88 | public void BytesOrderMatters(int swapSize, int swapPositionA, int swapPositionB) 89 | { 90 | Random rnd = new Random(123); 91 | byte[] bytes = new byte[255]; 92 | rnd.NextBytes(bytes); 93 | 94 | int hash = GxHash.Hash32(bytes, 0); 95 | 96 | SwapBytes(bytes, swapPositionA, swapPositionB, swapSize); 97 | 98 | int hashAfterSwap = GxHash.Hash32(bytes, 0); 99 | 100 | Assert.AreNotEqual(hash, hashAfterSwap); 101 | } 102 | 103 | private static void SwapBytes(Span span, int pos1, int pos2, int n) 104 | { 105 | // Check if the input parameters are valid 106 | if (pos1 < 0 || pos2 < 0 || n < 0) 107 | { 108 | throw new ArgumentOutOfRangeException("Positions and length must be non-negative."); 109 | } 110 | if (pos1 + n > span.Length || pos2 + n > span.Length) 111 | { 112 | throw new ArgumentOutOfRangeException("Positions and length must be within the span's length."); 113 | } 114 | 115 | // Perform the swap 116 | Span temp = stackalloc byte[n]; 117 | span.Slice(pos1, n).CopyTo(temp); 118 | span.Slice(pos2, n).CopyTo(span.Slice(pos1, n)); 119 | temp.CopyTo(span.Slice(pos2, n)); 120 | } 121 | } 122 | -------------------------------------------------------------------------------- /GxHash.Utils/GxHash.Utils.csproj: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | net8.0 5 | enable 6 | enable 7 | true 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | -------------------------------------------------------------------------------- /GxHash.Utils/Noop.cs: -------------------------------------------------------------------------------- 1 | using System.Runtime.CompilerServices; 2 | 3 | namespace GxHash.Utils; 4 | 5 | public static class No 6 | { 7 | /// 8 | /// Prevents compiler optimizations such as code hoisting, with minimal overhead 9 | /// 10 | /// 11 | /// 12 | [MethodImpl(MethodImplOptions.NoInlining)] 13 | public static void NoOp(this T input) 14 | { 15 | // Noop 16 | } 17 | 18 | /// 19 | /// Prevents compiler optimizations such as code hoisting, with minimal overhead 20 | /// 21 | /// 22 | /// 23 | /// 24 | [MethodImpl(MethodImplOptions.NoInlining)] 25 | public static T NoOpReturn(this T input) 26 | { 27 | return input; 28 | } 29 | } -------------------------------------------------------------------------------- /GxHash.Utils/QualificationUtils.cs: -------------------------------------------------------------------------------- 1 | namespace GxHash.Utils; 2 | 3 | public static class QualificationUtils 4 | { 5 | /// 6 | /// Returns distribution ratio 7 | /// Best case is 0 (perfectly evenly distributed) 8 | /// Worst case is 1 (the worst distribution) 9 | /// 10 | /// 11 | /// 12 | /// 13 | public static unsafe double BitsDistribution(this IEnumerable input) 14 | where T : unmanaged 15 | { 16 | int size = sizeof(T); 17 | int count = 0; 18 | int[] bitBuckets = new int[size * 8]; // Not sponsored... 19 | 20 | foreach (T val in input) 21 | { 22 | count++; 23 | byte* p = (byte*)&val; 24 | for (int b = 0; b < size; b++) 25 | { 26 | for (int k = 0; k < 8; k++) 27 | { 28 | bitBuckets[8 * b + k] += (p[b] >> k) & 1; 29 | } 30 | } 31 | } 32 | 33 | double[] bitBucketsFloating = bitBuckets.Select(x => 1d * x).ToArray(); 34 | double std = StandardDeviation(bitBucketsFloating); 35 | double worstStd = 0.5d * count; // For normalizing std from 0 to 1 36 | 37 | return std / worstStd; 38 | } 39 | 40 | public delegate TResult HashFunction(ReadOnlySpan data) 41 | where TResult : unmanaged; 42 | 43 | /// 44 | /// Compute the avalanche effect of an hashing function. 45 | /// "The strict avalanche criterion (SAC) is a formalization of the avalanche effect. 46 | /// It is satisfied if, whenever a single input bit is complemented, each of the output bits changes with a 50% probability" 47 | /// https://en.wikipedia.org/wiki/Avalanche_effect 48 | /// Best case is 0 (means 50% probability for each bit to be changed on 1 bit changed) 49 | /// Worst case is 1 50 | /// 51 | /// 52 | /// 53 | /// 54 | /// 55 | /// 56 | public static unsafe double Avalanche(this HashFunction hashFunction, ReadOnlySpan input, int inputSizeBytes) 57 | where TResult : unmanaged 58 | { 59 | unchecked 60 | { 61 | int sizeR = sizeof(TResult); 62 | 63 | Span bytesBitChanged = stackalloc byte[inputSizeBytes]; 64 | 65 | int iterations = input.Length / inputSizeBytes; 66 | 67 | double[] results = new double[iterations]; 68 | 69 | for (int i = 0; i < iterations; i++) 70 | { 71 | var slice = input.Slice(i + inputSizeBytes, inputSizeBytes); 72 | 73 | UnsafeUtils.FlipRandomBit(slice, bytesBitChanged); 74 | 75 | TResult v1 = hashFunction(slice); 76 | TResult v2 = hashFunction(bytesBitChanged); 77 | 78 | // Analyze the difference in output hash 79 | byte* pv1 = (byte*)&v1; 80 | byte* pv2 = (byte*)&v2; 81 | int diffs = 0; 82 | for (int b = 0; b < sizeR; b++) 83 | { 84 | int delta = pv1[b] ^ pv2[b]; 85 | for (int k = 0; k < 8; k++) 86 | { 87 | if ((delta & (1 << k)) != 0) 88 | { 89 | diffs++; 90 | } 91 | } 92 | } 93 | 94 | results[i] = 1d * diffs / (sizeR * 8); 95 | } 96 | 97 | return Math.Abs(1d - 2 * results.Average()); 98 | } 99 | } 100 | 101 | /// 102 | /// Returns a collision ratio. 103 | /// Best case is 0 (no collisions) 104 | /// Worst case is 1 (everything is colliding). 105 | /// 106 | /// 107 | /// 108 | public static double Uniqueness(this IEnumerable input) 109 | { 110 | int collisions = 0; 111 | int count = 0; 112 | HashSet set = new HashSet(); 113 | foreach (T val in input) 114 | { 115 | count++; 116 | if (!set.Add(val)) 117 | { 118 | collisions++; 119 | } 120 | } 121 | return 1d * collisions / (count - 1); 122 | } 123 | 124 | /// 125 | /// Returns the standard deviation of a given serie. 126 | /// If average is not given, it will be computed from the input. 127 | /// 128 | /// 129 | /// 130 | /// 131 | public static double StandardDeviation(this IEnumerable input, double? average = null) 132 | { 133 | double avg = average ?? input.Average(); 134 | double sum = input.Sum(i => Math.Pow(i - avg, 2)); 135 | return Math.Sqrt(sum / input.Count()); 136 | } 137 | } -------------------------------------------------------------------------------- /GxHash.Utils/RandomObjectUtils.cs: -------------------------------------------------------------------------------- 1 | namespace GxHash.Utils; 2 | 3 | public static class RandomObjectUtils 4 | { 5 | private static readonly Random _Random = new(); 6 | 7 | public static unsafe T CreateRandom() 8 | where T : unmanaged 9 | { 10 | int size = sizeof(T); 11 | 12 | // Create object on the stack 13 | T val = default; 14 | 15 | // Get span to internal data 16 | byte* p = (byte*)&val; 17 | var bytes = new Span(p, size); 18 | 19 | // Fill with random bytes 20 | _Random.NextBytes(bytes); 21 | 22 | return val; 23 | } 24 | 25 | public static byte[] CreateRandomBytes(int minSize, int maxSize) 26 | { 27 | byte[] bytes = new byte[_Random.Next(minSize, maxSize)]; 28 | _Random.NextBytes(bytes); 29 | return bytes; 30 | } 31 | 32 | public static string CreateRandomString(int minSize = 4, int maxSize = 100, int seed = 0, string charSet = " abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789") 33 | { 34 | Random random = new Random(seed); 35 | return new string(Enumerable 36 | .Repeat(charSet, random.Next(minSize, maxSize + 1)) 37 | .Select(s => s[random.Next(s.Length)]) 38 | .ToArray()); 39 | } 40 | } -------------------------------------------------------------------------------- /GxHash.Utils/UnsafeUtils.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Runtime.CompilerServices; 3 | using System.Runtime.InteropServices; 4 | 5 | namespace GxHash; 6 | 7 | public static class UnsafeUtils 8 | { 9 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 10 | public static unsafe void FlipRandomBit(ReadOnlySpan input, Span output) 11 | { 12 | unchecked 13 | { 14 | input.CopyTo(output); 15 | 16 | int bit = Random.Shared.Next(0, input.Length * 8); 17 | 18 | fixed (byte* p = &MemoryMarshal.GetReference(output)) 19 | { 20 | // Swap 1 random bit 21 | p[bit / 8] ^= (byte)(1 << (bit % 8)); 22 | } 23 | } 24 | } 25 | } -------------------------------------------------------------------------------- /GxHash.sln: -------------------------------------------------------------------------------- 1 |  2 | Microsoft Visual Studio Solution File, Format Version 12.00 3 | # Visual Studio Version 17 4 | VisualStudioVersion = 17.0.32014.148 5 | MinimumVisualStudioVersion = 10.0.40219.1 6 | Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "GxHash", "GxHash\GxHash.csproj", "{665C3432-2C61-4C78-B210-D3813C8A420B}" 7 | EndProject 8 | Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "GxHash.Tests", "GxHash.Tests\GxHash.Tests.csproj", "{1DEFAE60-4C4F-40B9-94AF-7CD207E56B81}" 9 | EndProject 10 | Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "GxHash.Benchmarks", "GxHash.Benchmarks\GxHash.Benchmarks.csproj", "{6BDACBF4-BE0D-4153-871D-680FEFE3D953}" 11 | EndProject 12 | Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "GxHash.Utils", "GxHash.Utils\GxHash.Utils.csproj", "{CD29ABEF-A1A2-4CD1-A9DA-D2E7F0262B2E}" 13 | EndProject 14 | Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "ci-cd", "ci-cd", "{7D5CFAF9-2247-48B5-A60E-AA4BA93ADC09}" 15 | ProjectSection(SolutionItems) = preProject 16 | .github\workflows\.github-ci.yml = .github\workflows\.github-ci.yml 17 | EndProjectSection 18 | EndProject 19 | Global 20 | GlobalSection(SolutionConfigurationPlatforms) = preSolution 21 | Debug|Any CPU = Debug|Any CPU 22 | Release|Any CPU = Release|Any CPU 23 | EndGlobalSection 24 | GlobalSection(ProjectConfigurationPlatforms) = postSolution 25 | {665C3432-2C61-4C78-B210-D3813C8A420B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 26 | {665C3432-2C61-4C78-B210-D3813C8A420B}.Debug|Any CPU.Build.0 = Debug|Any CPU 27 | {665C3432-2C61-4C78-B210-D3813C8A420B}.Release|Any CPU.ActiveCfg = Release|Any CPU 28 | {665C3432-2C61-4C78-B210-D3813C8A420B}.Release|Any CPU.Build.0 = Release|Any CPU 29 | {1DEFAE60-4C4F-40B9-94AF-7CD207E56B81}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 30 | {1DEFAE60-4C4F-40B9-94AF-7CD207E56B81}.Debug|Any CPU.Build.0 = Debug|Any CPU 31 | {1DEFAE60-4C4F-40B9-94AF-7CD207E56B81}.Release|Any CPU.ActiveCfg = Release|Any CPU 32 | {1DEFAE60-4C4F-40B9-94AF-7CD207E56B81}.Release|Any CPU.Build.0 = Release|Any CPU 33 | {6BDACBF4-BE0D-4153-871D-680FEFE3D953}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 34 | {6BDACBF4-BE0D-4153-871D-680FEFE3D953}.Debug|Any CPU.Build.0 = Debug|Any CPU 35 | {6BDACBF4-BE0D-4153-871D-680FEFE3D953}.Release|Any CPU.ActiveCfg = Release|Any CPU 36 | {6BDACBF4-BE0D-4153-871D-680FEFE3D953}.Release|Any CPU.Build.0 = Release|Any CPU 37 | {CD29ABEF-A1A2-4CD1-A9DA-D2E7F0262B2E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 38 | {CD29ABEF-A1A2-4CD1-A9DA-D2E7F0262B2E}.Debug|Any CPU.Build.0 = Debug|Any CPU 39 | {CD29ABEF-A1A2-4CD1-A9DA-D2E7F0262B2E}.Release|Any CPU.ActiveCfg = Release|Any CPU 40 | {CD29ABEF-A1A2-4CD1-A9DA-D2E7F0262B2E}.Release|Any CPU.Build.0 = Release|Any CPU 41 | EndGlobalSection 42 | GlobalSection(SolutionProperties) = preSolution 43 | HideSolutionNode = FALSE 44 | EndGlobalSection 45 | GlobalSection(ExtensibilityGlobals) = postSolution 46 | SolutionGuid = {8FDEB035-9702-486C-8644-867C34629DDF} 47 | EndGlobalSection 48 | EndGlobal 49 | -------------------------------------------------------------------------------- /GxHash/GxHash.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Runtime.CompilerServices; 3 | using System.Runtime.InteropServices; 4 | using System.Runtime.Intrinsics; 5 | using System.Runtime.Intrinsics.Arm; 6 | using ArmAes = System.Runtime.Intrinsics.Arm.Aes; 7 | using X86Aes = System.Runtime.Intrinsics.X86.Aes; 8 | 9 | namespace GxHash; 10 | 11 | public class GxHash 12 | { 13 | // Internal usage only because T cannot be checked at compile time via generic type constrains 14 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 15 | internal static T Hash(ReadOnlySpan bytes, UInt128 seed) { 16 | return Finalize(CompressFast(Compress(bytes), Unsafe.As>(ref seed))) 17 | .As().GetElement(0); 18 | } 19 | 20 | /// 21 | /// Hash a span of bytes into an 32-bit signed integer, using the given seed 22 | /// 23 | /// The input bytes to hash 24 | /// A 128-bit seed 25 | /// 26 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 27 | public static int Hash32(ReadOnlySpan bytes, UInt128 seed) { 28 | return Finalize(CompressFast(Compress(bytes), Unsafe.As>(ref seed))) 29 | .AsInt32().GetElement(0); 30 | } 31 | 32 | /// 33 | /// Hash a span of bytes into an 32-bit signed integer, using the given seed 34 | /// 35 | /// The input bytes to hash 36 | /// A 128-bit seed 37 | /// 38 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 39 | public static uint HashU32(ReadOnlySpan bytes, UInt128 seed) { 40 | return Finalize(CompressFast(Compress(bytes), Unsafe.As>(ref seed))) 41 | .AsUInt32().GetElement(0); 42 | } 43 | 44 | /// 45 | /// Hash a span of bytes into an 64-bit signed integer, using the given seed 46 | /// 47 | /// The input bytes to hash 48 | /// A 128-bit seed 49 | /// 50 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 51 | public static long Hash64(ReadOnlySpan bytes, UInt128 seed) { 52 | return Finalize(CompressFast(Compress(bytes), Unsafe.As>(ref seed))) 53 | .AsInt64().GetElement(0); 54 | } 55 | 56 | /// 57 | /// Hash a span of bytes into an 64-bit unsigned integer, using the given seed 58 | /// 59 | /// The input bytes to hash 60 | /// A 128-bit seed 61 | /// 62 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 63 | public static ulong HashU64(ReadOnlySpan bytes, UInt128 seed) { 64 | return Finalize(CompressFast(Compress(bytes), Unsafe.As>(ref seed))) 65 | .AsUInt64().GetElement(0); 66 | } 67 | 68 | /// 69 | /// Hash a span of bytes into an 128-bit unsigned integer, using the given seed 70 | /// 71 | /// The input bytes to hash 72 | /// A 128-bit seed 73 | /// 74 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 75 | public static UInt128 Hash128(ReadOnlySpan bytes, UInt128 seed) { 76 | Vector128 hash = Finalize(CompressFast(Compress(bytes), Unsafe.As>(ref seed))); 77 | return Unsafe.As, UInt128>(ref hash); 78 | } 79 | 80 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 81 | private static Vector128 Finalize(Vector128 input) { 82 | var keys1 = Vector128.Create(0x713b01d0, 0x8f2f35db, 0xaf163956, 0x85459f85).AsByte(); 83 | var keys2 = Vector128.Create(0x1de09647, 0x92cfa39c, 0x3dd99aca, 0xb89c054f).AsByte(); 84 | var keys3 = Vector128.Create(0xc78b122b, 0x5544b1b7, 0x689d2b7d, 0xd0012e32).AsByte(); 85 | 86 | Vector128 output = input; 87 | 88 | if (ArmAes.IsSupported) { 89 | // For some reasons the ARM Neon intrinsics for AES a very different from the ones for X86, 90 | // so we need these operations below to achieve the same results as for x86 91 | // See https://blog.michaelbrase.com/2018/05/08/emulating-x86-aes-intrinsics-on-armv8-a 92 | output = AdvSimd.Xor(ArmAes.MixColumns(ArmAes.Encrypt(output, Vector128.Zero)), keys1); 93 | output = AdvSimd.Xor(ArmAes.MixColumns(ArmAes.Encrypt(output, Vector128.Zero)), keys2); 94 | output = AdvSimd.Xor(ArmAes.Encrypt(output, Vector128.Zero), keys3); 95 | } else if (X86Aes.IsSupported) { 96 | output = X86Aes.Encrypt(output, keys1); 97 | output = X86Aes.Encrypt(output, keys2); 98 | output = X86Aes.EncryptLast(output, keys3); 99 | } else { 100 | throw new PlatformNotSupportedException(); 101 | } 102 | 103 | return output; 104 | } 105 | 106 | private const int VECTOR_SIZE = 16; 107 | private const int UNROLL_FACTOR = 8; 108 | 109 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 110 | private static Vector128 Compress(ReadOnlySpan bytes) 111 | { 112 | // Get pointer of SIMD vectors from input span 113 | ref var ptr = ref Unsafe.As>(ref MemoryMarshal.GetReference(bytes)); 114 | 115 | int len = bytes.Length; 116 | 117 | if (len <= VECTOR_SIZE) { 118 | // Input fits on a single SIMD vector, however we might read beyond the input message 119 | // Thus we need this safe method that checks if it can safely read beyond or must copy 120 | return GetPartialVector(ref ptr, len); 121 | } 122 | 123 | Vector128 hashVector; 124 | int remainingBytes; 125 | 126 | int extraBytesCount = len % VECTOR_SIZE; 127 | if (extraBytesCount == 0) { 128 | hashVector = ptr; 129 | ptr = ref Unsafe.Add(ref ptr, 1); 130 | remainingBytes = len - VECTOR_SIZE; 131 | } else { 132 | // If the input length does not match the length of a whole number of SIMD vectors, 133 | // it means we'll need to read a partial vector. We can start with the partial vector first, 134 | // so that we can safely read beyond since we expect the following bytes to still be part of 135 | // the input 136 | hashVector = GetPartialVectorUnsafe(ref ptr, extraBytesCount); 137 | ptr = ref Unsafe.AddByteOffset(ref ptr, extraBytesCount); 138 | remainingBytes = len - extraBytesCount; 139 | } 140 | 141 | if (len <= VECTOR_SIZE * 2) { 142 | // Fast path when input length > 16 and <= 32 143 | hashVector = Compress(hashVector, ptr); 144 | } else if (len <= VECTOR_SIZE * 3) { 145 | // Fast path when input length > 32 and <= 48 146 | hashVector = Compress(hashVector, Compress(ptr, Unsafe.Add(ref ptr, 1))); 147 | } else { 148 | // Input message is large and we can use the high ILP loop 149 | hashVector = CompressMany(ref ptr, hashVector, remainingBytes); 150 | } 151 | 152 | return hashVector; 153 | } 154 | 155 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 156 | private static Vector128 CompressMany(ref Vector128 start, Vector128 hashVector, int len) 157 | { 158 | int unrollableBlocksCount = len / (VECTOR_SIZE * UNROLL_FACTOR) * UNROLL_FACTOR; 159 | ref var end2 = ref Unsafe.Add(ref start, unrollableBlocksCount); 160 | 161 | while (Unsafe.IsAddressLessThan(ref start, ref end2)) { 162 | 163 | Vector128 blockHash = start; 164 | blockHash = CompressFast(blockHash, Unsafe.Add(ref start, 1)); 165 | blockHash = CompressFast(blockHash, Unsafe.Add(ref start, 2)); 166 | blockHash = CompressFast(blockHash, Unsafe.Add(ref start, 3)); 167 | blockHash = CompressFast(blockHash, Unsafe.Add(ref start, 4)); 168 | blockHash = CompressFast(blockHash, Unsafe.Add(ref start, 5)); 169 | blockHash = CompressFast(blockHash, Unsafe.Add(ref start, 6)); 170 | blockHash = CompressFast(blockHash, Unsafe.Add(ref start, 7)); 171 | start = ref Unsafe.Add(ref start, UNROLL_FACTOR); 172 | 173 | hashVector = Compress(hashVector, blockHash); 174 | } 175 | 176 | int remainingBlocksCount = len / VECTOR_SIZE - unrollableBlocksCount; 177 | 178 | ref var end = ref Unsafe.Add(ref start, remainingBlocksCount); 179 | 180 | while (Unsafe.IsAddressLessThan(ref start, ref end)) 181 | { 182 | hashVector = Compress(hashVector, start); 183 | start = ref Unsafe.Add(ref start, 1); 184 | } 185 | 186 | return hashVector; 187 | } 188 | 189 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 190 | private static unsafe Vector128 GetPartialVector(ref Vector128 start, int remainingBytes) 191 | { 192 | fixed (Vector128* pin = &start) 193 | { 194 | if (IsReadBeyondSafe(ref start)) 195 | { 196 | return GetPartialVectorUnsafe(ref start, remainingBytes); 197 | } 198 | } 199 | 200 | return GetPartialVectorSafe(ref start, remainingBytes); 201 | } 202 | 203 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 204 | private static Vector128 GetPartialVectorSafe(ref Vector128 start, int remainingBytes) 205 | { 206 | Vector128 input = Vector128.Zero; 207 | ref byte source = ref Unsafe.As, byte>(ref start); 208 | ref byte dest = ref Unsafe.As, byte>(ref input); 209 | Unsafe.CopyBlockUnaligned(ref dest, ref source, (uint)remainingBytes); 210 | return Vector128.Add(input, Vector128.Create((byte)remainingBytes)); 211 | } 212 | 213 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 214 | private static Vector128 GetPartialVectorUnsafe(ref Vector128 start, int remainingBytes) 215 | { 216 | var indices = Vector128.Create(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15); 217 | var mask = Vector128.GreaterThan(Vector128.Create((sbyte)remainingBytes), indices).AsByte(); 218 | Vector128 hashVector = Vector128.BitwiseAnd(mask, start); 219 | return Vector128.Add(hashVector, Vector128.Create((byte)remainingBytes)); 220 | } 221 | 222 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 223 | private static Vector128 Compress(Vector128 a, Vector128 b) 224 | { 225 | var keys1 = Vector128.Create(0xFC3BC28E, 0x89C222E5, 0xB09D3E21, 0xF2784542).AsByte(); 226 | var keys2 = Vector128.Create(0x03FCE279, 0xCB6B2E9B, 0xB361DC58, 0x39136BD9).AsByte(); 227 | 228 | if (ArmAes.IsSupported) 229 | { 230 | b = AdvSimd.Xor(ArmAes.MixColumns(ArmAes.Encrypt(b, Vector128.Zero)), keys1); 231 | b = AdvSimd.Xor(ArmAes.MixColumns(ArmAes.Encrypt(b, Vector128.Zero)), keys2); 232 | return AdvSimd.Xor(ArmAes.Encrypt(a, Vector128.Zero), b); 233 | } 234 | if (X86Aes.IsSupported) 235 | { 236 | b = X86Aes.Encrypt(b, keys1); 237 | b = X86Aes.Encrypt(b, keys2); 238 | return X86Aes.EncryptLast(a, b); 239 | } 240 | 241 | throw new PlatformNotSupportedException(); 242 | } 243 | 244 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 245 | private static Vector128 CompressFast(Vector128 a, Vector128 b) 246 | { 247 | if (ArmAes.IsSupported) 248 | { 249 | return AdvSimd.Xor(ArmAes.MixColumns(ArmAes.Encrypt(a, Vector128.Zero)), b); 250 | } 251 | if (X86Aes.IsSupported) 252 | { 253 | return X86Aes.Encrypt(a, b); 254 | } 255 | 256 | throw new PlatformNotSupportedException(); 257 | } 258 | 259 | /// 260 | /// Returns true if reading the ref value is safe. 261 | /// This is done using the pointer address and making sure we aren't going to 262 | /// read past the end of the current memory page (which could produce segfaults) 263 | /// 264 | /// 265 | /// 266 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 267 | private static unsafe bool IsReadBeyondSafe(ref Vector128 reference) 268 | { 269 | // 4096 bytes is a conservative value for the page size 270 | const int PAGE_SIZE = 0x1000; 271 | IntPtr address = (IntPtr)Unsafe.AsPointer(ref reference); 272 | IntPtr offsetWithinPage = address & (PAGE_SIZE - 1); 273 | return offsetWithinPage < PAGE_SIZE - VECTOR_SIZE; 274 | } 275 | } -------------------------------------------------------------------------------- /GxHash/GxHash.csproj: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | GxHash 5 | Olivier Giniaux 6 | C# version of GxHash, the fastest non-cryptographic algorithm 7 | LICENSE.txt 8 | README.md 9 | 2.0.1 10 | 11 | 2.0.1: Pin the source while we're doing a read beyond safe 12 | 2.0.0: Do partial-first 13 | 1.0.0: First version 14 | 15 | net8.0 16 | true 17 | true 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Olivier Giniaux 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # GxHash 2 | 3 | ![CI](https://github.com/ogxd/gxhash-csharp/actions/workflows/.github-ci.yml/badge.svg) 4 | 5 | GxHash is a blazingly fast and robust non-cryptographic hashing algorithm. 6 | 7 | # Features 8 | 9 | ## Blazingly Fast 🚀 10 | Up to this date, GxHash is the fastest non-cryptographic hashing algorithm of its class, for all input sizes. This performance is possible mostly thanks to heavy usage of SIMD intrinsics, high ILP construction and a small bytecode (easily inlined and cached). 11 | See the benchmarks. 12 | 13 | ## Highly Robust 🗿 14 | GxHash uses several rounds of hardware-accelerated AES block cipher for efficient bit mixing. 15 | Thanks to this, GxHash passes all SMHasher tests, which is the de facto quality benchmark for non-cryptographic hash functions, gathering most of the existing algorithms. GxHash has low collisions, uniform distribution and high avalanche properties. 16 | 17 | # Portability 18 | 19 | ## Architecture Compatibility 20 | GxHash is compatible with: 21 | 22 | - X86 processors with AES-NI & SSE2 intrinsics 23 | - ARM processors with AES & NEON intrinsics 24 | Warning: Other platforms are currently not supported (there is no fallback). GxHash will not build on these platforms. 25 | 26 | ## Hashes Stability 27 | All generated hashes for a given version of GxHash are stable, meaning that for a given input the output hash will be the same across all supported platforms. 28 | 29 | # Benchmarks 30 | 31 | This library is a C# port of [gxhash](https://github.com/ogxd/gxhash). Despite the language difference, performance is really close to the Rust version of the algorithm for a given version of the algorithm. You can run the benchmarks in GxHash.Benchmarks to see for yourself. --------------------------------------------------------------------------------