├── 1MConcurrentTasks ├── .gitignore ├── 1MConcurrentTasks.sln └── src │ ├── 1MConcurrentTasks.csproj │ ├── Program.cs │ └── ShortRunConfig.cs └── README.md /1MConcurrentTasks/.gitignore: -------------------------------------------------------------------------------- 1 | ## Ignore Visual Studio temporary files, build results, and 2 | ## files generated by popular Visual Studio add-ons. 3 | ## 4 | ## Get latest from https://github.com/github/gitignore/blob/main/VisualStudio.gitignore 5 | 6 | # User-specific files 7 | *.rsuser 8 | *.suo 9 | *.user 10 | *.userosscache 11 | *.sln.docstates 12 | 13 | # User-specific files (MonoDevelop/Xamarin Studio) 14 | *.userprefs 15 | 16 | # Mono auto generated files 17 | mono_crash.* 18 | 19 | # Build results 20 | [Dd]ebug/ 21 | [Dd]ebugPublic/ 22 | [Rr]elease/ 23 | [Rr]eleases/ 24 | x64/ 25 | x86/ 26 | [Ww][Ii][Nn]32/ 27 | [Aa][Rr][Mm]/ 28 | [Aa][Rr][Mm]64/ 29 | bld/ 30 | [Bb]in/ 31 | [Oo]bj/ 32 | [Ll]og/ 33 | [Ll]ogs/ 34 | 35 | # Visual Studio 2015/2017 cache/options directory 36 | .vs/ 37 | # Uncomment if you have tasks that create the project's static files in wwwroot 38 | #wwwroot/ 39 | 40 | # Visual Studio 2017 auto generated files 41 | Generated\ Files/ 42 | 43 | # MSTest test Results 44 | [Tt]est[Rr]esult*/ 45 | [Bb]uild[Ll]og.* 46 | 47 | # NUnit 48 | *.VisualState.xml 49 | TestResult.xml 50 | nunit-*.xml 51 | 52 | # Build Results of an ATL Project 53 | [Dd]ebugPS/ 54 | [Rr]eleasePS/ 55 | dlldata.c 56 | 57 | # Benchmark Results 58 | BenchmarkDotNet.Artifacts/ 59 | 60 | # .NET Core 61 | project.lock.json 62 | project.fragment.lock.json 63 | artifacts/ 64 | 65 | # ASP.NET Scaffolding 66 | ScaffoldingReadMe.txt 67 | 68 | # StyleCop 69 | StyleCopReport.xml 70 | 71 | # Files built by Visual Studio 72 | *_i.c 73 | *_p.c 74 | *_h.h 75 | *.ilk 76 | *.meta 77 | *.obj 78 | *.iobj 79 | *.pch 80 | *.pdb 81 | *.ipdb 82 | *.pgc 83 | *.pgd 84 | *.rsp 85 | *.sbr 86 | *.tlb 87 | *.tli 88 | *.tlh 89 | *.tmp 90 | *.tmp_proj 91 | *_wpftmp.csproj 92 | *.log 93 | *.tlog 94 | *.vspscc 95 | *.vssscc 96 | .builds 97 | *.pidb 98 | *.svclog 99 | *.scc 100 | 101 | # Chutzpah Test files 102 | _Chutzpah* 103 | 104 | # Visual C++ cache files 105 | ipch/ 106 | *.aps 107 | *.ncb 108 | *.opendb 109 | *.opensdf 110 | *.sdf 111 | *.cachefile 112 | *.VC.db 113 | *.VC.VC.opendb 114 | 115 | # Visual Studio profiler 116 | *.psess 117 | *.vsp 118 | *.vspx 119 | *.sap 120 | 121 | # Visual Studio Trace Files 122 | *.e2e 123 | 124 | # TFS 2012 Local Workspace 125 | $tf/ 126 | 127 | # Guidance Automation Toolkit 128 | *.gpState 129 | 130 | # ReSharper is a .NET coding add-in 131 | _ReSharper*/ 132 | *.[Rr]e[Ss]harper 133 | *.DotSettings.user 134 | 135 | # TeamCity is a build add-in 136 | _TeamCity* 137 | 138 | # DotCover is a Code Coverage Tool 139 | *.dotCover 140 | 141 | # AxoCover is a Code Coverage Tool 142 | .axoCover/* 143 | !.axoCover/settings.json 144 | 145 | # Coverlet is a free, cross platform Code Coverage Tool 146 | coverage*.json 147 | coverage*.xml 148 | coverage*.info 149 | 150 | # Visual Studio code coverage results 151 | *.coverage 152 | *.coveragexml 153 | 154 | # NCrunch 155 | _NCrunch_* 156 | .*crunch*.local.xml 157 | nCrunchTemp_* 158 | 159 | # MightyMoose 160 | *.mm.* 161 | AutoTest.Net/ 162 | 163 | # Web workbench (sass) 164 | .sass-cache/ 165 | 166 | # Installshield output folder 167 | [Ee]xpress/ 168 | 169 | # DocProject is a documentation generator add-in 170 | DocProject/buildhelp/ 171 | DocProject/Help/*.HxT 172 | DocProject/Help/*.HxC 173 | DocProject/Help/*.hhc 174 | DocProject/Help/*.hhk 175 | DocProject/Help/*.hhp 176 | DocProject/Help/Html2 177 | DocProject/Help/html 178 | 179 | # Click-Once directory 180 | publish/ 181 | 182 | # Publish Web Output 183 | *.[Pp]ublish.xml 184 | *.azurePubxml 185 | # Note: Comment the next line if you want to checkin your web deploy settings, 186 | # but database connection strings (with potential passwords) will be unencrypted 187 | *.pubxml 188 | *.publishproj 189 | 190 | # Microsoft Azure Web App publish settings. Comment the next line if you want to 191 | # checkin your Azure Web App publish settings, but sensitive information contained 192 | # in these scripts will be unencrypted 193 | PublishScripts/ 194 | 195 | # NuGet Packages 196 | *.nupkg 197 | # NuGet Symbol Packages 198 | *.snupkg 199 | # The packages folder can be ignored because of Package Restore 200 | **/[Pp]ackages/* 201 | # except build/, which is used as an MSBuild target. 202 | !**/[Pp]ackages/build/ 203 | # Uncomment if necessary however generally it will be regenerated when needed 204 | #!**/[Pp]ackages/repositories.config 205 | # NuGet v3's project.json files produces more ignorable files 206 | *.nuget.props 207 | *.nuget.targets 208 | 209 | # Microsoft Azure Build Output 210 | csx/ 211 | *.build.csdef 212 | 213 | # Microsoft Azure Emulator 214 | ecf/ 215 | rcf/ 216 | 217 | # Windows Store app package directories and files 218 | AppPackages/ 219 | BundleArtifacts/ 220 | Package.StoreAssociation.xml 221 | _pkginfo.txt 222 | *.appx 223 | *.appxbundle 224 | *.appxupload 225 | 226 | # Visual Studio cache files 227 | # files ending in .cache can be ignored 228 | *.[Cc]ache 229 | # but keep track of directories ending in .cache 230 | !?*.[Cc]ache/ 231 | 232 | # Others 233 | ClientBin/ 234 | ~$* 235 | *~ 236 | *.dbmdl 237 | *.dbproj.schemaview 238 | *.jfm 239 | *.pfx 240 | *.publishsettings 241 | orleans.codegen.cs 242 | 243 | # Including strong name files can present a security risk 244 | # (https://github.com/github/gitignore/pull/2483#issue-259490424) 245 | #*.snk 246 | 247 | # Since there are multiple workflows, uncomment next line to ignore bower_components 248 | # (https://github.com/github/gitignore/pull/1529#issuecomment-104372622) 249 | #bower_components/ 250 | 251 | # RIA/Silverlight projects 252 | Generated_Code/ 253 | 254 | # Backup & report files from converting an old project file 255 | # to a newer Visual Studio version. Backup files are not needed, 256 | # because we have git ;-) 257 | _UpgradeReport_Files/ 258 | Backup*/ 259 | UpgradeLog*.XML 260 | UpgradeLog*.htm 261 | ServiceFabricBackup/ 262 | *.rptproj.bak 263 | 264 | # SQL Server files 265 | *.mdf 266 | *.ldf 267 | *.ndf 268 | 269 | # Business Intelligence projects 270 | *.rdl.data 271 | *.bim.layout 272 | *.bim_*.settings 273 | *.rptproj.rsuser 274 | *- [Bb]ackup.rdl 275 | *- [Bb]ackup ([0-9]).rdl 276 | *- [Bb]ackup ([0-9][0-9]).rdl 277 | 278 | # Microsoft Fakes 279 | FakesAssemblies/ 280 | 281 | # GhostDoc plugin setting file 282 | *.GhostDoc.xml 283 | 284 | # Node.js Tools for Visual Studio 285 | .ntvs_analysis.dat 286 | node_modules/ 287 | 288 | # Visual Studio 6 build log 289 | *.plg 290 | 291 | # Visual Studio 6 workspace options file 292 | *.opt 293 | 294 | # Visual Studio 6 auto-generated workspace file (contains which files were open etc.) 295 | *.vbw 296 | 297 | # Visual Studio 6 auto-generated project file (contains which files were open etc.) 298 | *.vbp 299 | 300 | # Visual Studio 6 workspace and project file (working project files containing files to include in project) 301 | *.dsw 302 | *.dsp 303 | 304 | # Visual Studio 6 technical files 305 | *.ncb 306 | *.aps 307 | 308 | # Visual Studio LightSwitch build output 309 | **/*.HTMLClient/GeneratedArtifacts 310 | **/*.DesktopClient/GeneratedArtifacts 311 | **/*.DesktopClient/ModelManifest.xml 312 | **/*.Server/GeneratedArtifacts 313 | **/*.Server/ModelManifest.xml 314 | _Pvt_Extensions 315 | 316 | # Paket dependency manager 317 | .paket/paket.exe 318 | paket-files/ 319 | 320 | # FAKE - F# Make 321 | .fake/ 322 | 323 | # CodeRush personal settings 324 | .cr/personal 325 | 326 | # Python Tools for Visual Studio (PTVS) 327 | __pycache__/ 328 | *.pyc 329 | 330 | # Cake - Uncomment if you are using it 331 | # tools/** 332 | # !tools/packages.config 333 | 334 | # Tabs Studio 335 | *.tss 336 | 337 | # Telerik's JustMock configuration file 338 | *.jmconfig 339 | 340 | # BizTalk build output 341 | *.btp.cs 342 | *.btm.cs 343 | *.odx.cs 344 | *.xsd.cs 345 | 346 | # OpenCover UI analysis results 347 | OpenCover/ 348 | 349 | # Azure Stream Analytics local run output 350 | ASALocalRun/ 351 | 352 | # MSBuild Binary and Structured Log 353 | *.binlog 354 | 355 | # NVidia Nsight GPU debugger configuration file 356 | *.nvuser 357 | 358 | # MFractors (Xamarin productivity tool) working folder 359 | .mfractor/ 360 | 361 | # Local History for Visual Studio 362 | .localhistory/ 363 | 364 | # Visual Studio History (VSHistory) files 365 | .vshistory/ 366 | 367 | # BeatPulse healthcheck temp database 368 | healthchecksdb 369 | 370 | # Backup folder for Package Reference Convert tool in Visual Studio 2017 371 | MigrationBackup/ 372 | 373 | # Ionide (cross platform F# VS Code tools) working folder 374 | .ionide/ 375 | 376 | # Fody - auto-generated XML schema 377 | FodyWeavers.xsd 378 | 379 | # VS Code files for those working on multiple tools 380 | .vscode/* 381 | !.vscode/settings.json 382 | !.vscode/tasks.json 383 | !.vscode/launch.json 384 | !.vscode/extensions.json 385 | *.code-workspace 386 | 387 | # Local History for Visual Studio Code 388 | .history/ 389 | 390 | # Windows Installer files from build outputs 391 | *.cab 392 | *.msi 393 | *.msix 394 | *.msm 395 | *.msp 396 | 397 | # JetBrains Rider 398 | *.sln.iml -------------------------------------------------------------------------------- /1MConcurrentTasks/1MConcurrentTasks.sln: -------------------------------------------------------------------------------- 1 |  2 | Microsoft Visual Studio Solution File, Format Version 12.00 3 | # Visual Studio Version 17 4 | VisualStudioVersion = 17.0.32112.339 5 | MinimumVisualStudioVersion = 10.0.40219.1 6 | Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "1MConcurrentTasks", "src\1MConcurrentTasks.csproj", "{58967D38-D1EB-466A-8693-F0314C00D6B3}" 7 | EndProject 8 | Global 9 | GlobalSection(SolutionConfigurationPlatforms) = preSolution 10 | Debug|Any CPU = Debug|Any CPU 11 | Release|Any CPU = Release|Any CPU 12 | EndGlobalSection 13 | GlobalSection(ProjectConfigurationPlatforms) = postSolution 14 | {58967D38-D1EB-466A-8693-F0314C00D6B3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 15 | {58967D38-D1EB-466A-8693-F0314C00D6B3}.Debug|Any CPU.Build.0 = Debug|Any CPU 16 | {58967D38-D1EB-466A-8693-F0314C00D6B3}.Release|Any CPU.ActiveCfg = Release|Any CPU 17 | {58967D38-D1EB-466A-8693-F0314C00D6B3}.Release|Any CPU.Build.0 = Release|Any CPU 18 | EndGlobalSection 19 | GlobalSection(SolutionProperties) = preSolution 20 | HideSolutionNode = FALSE 21 | EndGlobalSection 22 | GlobalSection(ExtensibilityGlobals) = postSolution 23 | SolutionGuid = {542DD02E-8751-4155-AF25-25FBF9ED4D9E} 24 | EndGlobalSection 25 | EndGlobal 26 | -------------------------------------------------------------------------------- /1MConcurrentTasks/src/1MConcurrentTasks.csproj: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | Exe 5 | net6.0 6 | enable 7 | enable 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | -------------------------------------------------------------------------------- /1MConcurrentTasks/src/Program.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using BenchmarkDotNet.Attributes; 3 | using BenchmarkDotNet.Running; 4 | using Benchmarks.Config; 5 | 6 | namespace Benchmarks 7 | { 8 | [MemoryDiagnoser] 9 | [ThreadingDiagnoser] 10 | [Config(typeof(ShortRunConfig))] 11 | public class TaskRunOrNot 12 | { 13 | private const int delayMillisec = 10000; 14 | private const int N = 1000000; 15 | 16 | public TaskRunOrNot() {} 17 | 18 | [Benchmark(Baseline = true)] 19 | public async Task Baseline() 20 | { 21 | long start = Stopwatch.GetTimestamp(); 22 | 23 | List tasks = new List(); 24 | for (int i = 0; i < N; i++) 25 | { 26 | Task task = Task.Run(async () => 27 | { 28 | await Task.Delay(TimeSpan.FromMilliseconds(delayMillisec)); 29 | }); 30 | tasks.Add(task); 31 | } 32 | await Task.WhenAll(tasks); 33 | 34 | return Stopwatch.GetTimestamp() - start; 35 | } 36 | 37 | 38 | //[Benchmark] 39 | public async Task SetInitialCapacity_List() 40 | { 41 | long start = Stopwatch.GetTimestamp(); 42 | 43 | List tasks = new List(N); 44 | for (int i = 0; i < N; i++) 45 | { 46 | Task task = Task.Run(async () => 47 | { 48 | await Task.Delay(TimeSpan.FromMilliseconds(delayMillisec)); 49 | }); 50 | tasks.Add(task); 51 | } 52 | await Task.WhenAll(tasks); 53 | 54 | 55 | return Stopwatch.GetTimestamp() - start; 56 | } 57 | 58 | [Benchmark] 59 | public async Task AvoidingTaskRun() 60 | { 61 | long start = Stopwatch.GetTimestamp(); 62 | 63 | List tasks = new List(); 64 | for (int i = 0; i < N; i++) 65 | { 66 | tasks.Add(Task.Delay(TimeSpan.FromMilliseconds(delayMillisec))); 67 | } 68 | await Task.WhenAll(tasks); 69 | 70 | 71 | return Stopwatch.GetTimestamp() - start; 72 | } 73 | 74 | //[Benchmark] 75 | public async Task AvoidingTaskRun_SharingSameDelay_SameTask() 76 | { 77 | long start = Stopwatch.GetTimestamp(); 78 | 79 | Task delayTask = Task.Delay(TimeSpan.FromMilliseconds(delayMillisec)); 80 | 81 | List tasks = new List(); 82 | for (int i = 0; i < N; i++) 83 | { 84 | tasks.Add(delayTask); 85 | } 86 | await Task.WhenAll(tasks); 87 | 88 | 89 | return Stopwatch.GetTimestamp() - start; 90 | } 91 | 92 | //[Benchmark] 93 | public async Task AvoidingTaskRun_SharingSameDelay_DifferentTasks() 94 | { 95 | long start = Stopwatch.GetTimestamp(); 96 | 97 | Task delayTask = Task.Delay(TimeSpan.FromMilliseconds(delayMillisec)); 98 | 99 | List tasks = new List(); 100 | for (int i = 0; i < N; i++) 101 | { 102 | tasks.Add(delayTask.ContinueWith((Task _) => { })); 103 | } 104 | await Task.WhenAll(tasks); 105 | 106 | 107 | return Stopwatch.GetTimestamp() - start; 108 | } 109 | 110 | const string ListEntry = "Hello World"; 111 | 112 | //[Benchmark] 113 | public long SetInitialCapacity_List_String() 114 | { 115 | List items = new List(N); 116 | for (int i = 0; i < N; i++) 117 | { 118 | items.Add(ListEntry); 119 | } 120 | return items.Count; 121 | } 122 | 123 | //[Benchmark(Baseline = true)] 124 | public long Baseline_String() 125 | { 126 | List items = new List(); 127 | for (int i = 0; i < N; i++) 128 | { 129 | items.Add(ListEntry); 130 | } 131 | return items.Count; 132 | } 133 | } 134 | 135 | public class Program 136 | { 137 | public static void Main(string[] _) 138 | { 139 | BenchmarkRunner.Run(); 140 | } 141 | } 142 | } -------------------------------------------------------------------------------- /1MConcurrentTasks/src/ShortRunConfig.cs: -------------------------------------------------------------------------------- 1 | using BenchmarkDotNet.Configs; 2 | using BenchmarkDotNet.Environments; 3 | using BenchmarkDotNet.Jobs; 4 | 5 | namespace Benchmarks.Config 6 | { 7 | public class ShortRunConfig : ManualConfig 8 | { 9 | public ShortRunConfig() 10 | { 11 | AddJob(Job.ShortRun.WithRuntime(CoreRuntime.Core60)); 12 | AddJob(Job.ShortRun.WithRuntime(CoreRuntime.Core70)); 13 | AddJob(Job.ShortRun.WithRuntime(CoreRuntime.Core80)); 14 | //AddJob(Job.ShortRun.WithStrategy(RunStrategy.ColdStart)); 15 | } 16 | } 17 | } 18 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # C# Benchmarks 2 | A collection of random benchmarks in C#. 3 | 4 | ## 1M Concurrent Tasks 5 | Inspired by this hackernews post: https://news.ycombinator.com/item?id=36024209 6 | The blog post: https://pkolaczk.github.io/memory-consumption-of-async/ 7 | 8 | The authors code had some inefficiencies in the C# version. I was curious whether there are big improvements possible without changing the intent of the benchmark. 9 | 10 | The problem statement was thus: 11 | `Let’s launch N concurrent tasks, where each task waits for 10 seconds and then the program exists after all tasks finish. The number of tasks is controlled by the command line argument.` 12 | 13 | Here is the original code: 14 | ```csharp 15 | List tasks = new List(); 16 | for (int i = 0; i < numTasks; i++) 17 | { 18 | Task task = Task.Run(async () => 19 | { 20 | await Task.Delay(TimeSpan.FromSeconds(10)); 21 | }); 22 | tasks.Add(task); 23 | } 24 | await Task.WhenAll(tasks); 25 | 26 | ``` 27 | 28 | 29 | ### Observation 1: Author was not passing an initial size to List. 30 | - This causes the List to be re-sized and copied each time the internal capacity is exhausted. 31 | - Causes a large amount of intermediate allocations and GC pressure. 32 | - Other languages tested have the same inefficiency but this is particularly unfair for managed/garbage collected languages. 33 | - Depending on how frequently GC can run, those intermediate buffers may stick around for awhile leading to higher peak memory utilization. 34 | - In an unmanaged language, those intermediate buffers are likely freed as soon as the new buffer is created and copied to 35 | - The peak memory utilization is likely lower compared to the managed languages. 36 | 37 | Let's see what the impact on memory is by passing an initial capacity: 38 | 39 | ```csharp 40 | List tasks = new List(N); 41 | for (int i = 0; i < N; i++) 42 | { 43 | Task task = Task.Run(async () => 44 | { 45 | await Task.Delay(TimeSpan.FromMilliseconds(delayMillisec)); 46 | }); 47 | tasks.Add(task); 48 | } 49 | await Task.WhenAll(tasks); 50 | ``` 51 | 52 | | Method | Mean | Error | StdDev | Lock Contentions | Gen0 | Gen1 | Gen2 | Allocated | Alloc Ratio | 53 | |------------------------ |--------:|--------:|--------:|-----------------:|-----------:|-----------:|----------:|----------:|------------:| 54 | | SetInitialCapacity_List | 12.15 s | 1.295 s | 0.071 s | 14070.0000 | 55000.0000 | 29000.0000 | 4000.0000 | 419.96 MB | 0.98 | 55 | | Baseline | 12.30 s | 3.038 s | 0.167 s | 14416.0000 | 55000.0000 | 29000.0000 | 4000.0000 | 428.44 MB | 1.00 | 56 | 57 | Surprisingly the result is fairly negligible. The total allocated memory surprised me as well and was a lot higher than I expected. This means that the size of the array itself is negligle compared to the contents of the array. That means that the memory footprint of is pretty high. 58 | - An array of size 1M is going to use about ~8MB of memory (64-bit/8-byte references * 1M) 59 | - If the List re-size logic follows exponential growth (1 -> 2 -> 4 -> ... -> ~500K -> ~1M), then the combined size of all intermediate buffers is approximately 1M. 60 | - This roughly matches the difference we see ~9MB. 61 | 62 | To confirm the theory, let's benchmark resizing a list without all of the task stuff. 63 | - This benchmark adds the same string to a list 1M times, first with an initial capacity on the list of 1M, and second without a capacity. 64 | 65 | | Method | Mean | Error | StdDev | Ratio | Gen0 | Gen1 | Gen2 | Allocated | Alloc Ratio | 66 | |------------------------------- |---------:|----------:|----------:|------:|---------:|---------:|---------:|----------:|------------:| 67 | | SetInitialCapacity_List_String | 5.135 ms | 1.744 ms | 0.0956 ms | 0.61 | 265.6250 | 265.6250 | 265.6250 | 7.63 MB | 0.48 | 68 | | Baseline_String | 8.401 ms | 10.838 ms | 0.5940 ms | 1.00 | 453.1250 | 437.5000 | 437.5000 | 16 MB | 1.00 | 69 | 70 | #### Conclusion: Minimal impact on memory usage 71 | - The lack of initial List capacity doesn't contribute much to the overall peak memory usage. 72 | - The Task objects themselves contribute significantly more memory. 73 | 74 | ### Observation 2: Delay task is unnecessarily wrapped in Task.Run(...) 75 | - By default .NET will eagerly execute a Task when it is created. 76 | - When the first real async operation occurs, usually there is a yield operation that will yield back to the caller. 77 | - To immediately yield a new Task without running it, you can call Task.Run(...) which creates the Task and schedules it to the Threadpool 78 | - Typically this is needed for CPU-bound work to avoid blocking the caller. 79 | - For IO-bound async code, this is typically not necessary. 80 | 81 | Task.Delay(...) will immediately yield to the caller, so the Task.Run(...) isn't giving us any benefit here. 82 | 83 | Let's try the same benchmark without the Task.Run(...) call: 84 | 85 | ```csharp 86 | List tasks = new List(); 87 | for (int i = 0; i < N; i++) 88 | { 89 | tasks.Add(Task.Delay(TimeSpan.FromMilliseconds(delayMillisec))); 90 | } 91 | await Task.WhenAll(tasks); 92 | ``` 93 | 94 | | Method | Mean | Error | StdDev | Ratio | Gen0 | Completed Work Items | Lock Contentions | Gen1 | Gen2 | Allocated | Alloc Ratio | 95 | |---------------- |--------:|--------:|--------:|------:|-----------:|---------------------:|-----------------:|-----------:|----------:|----------:|------------:| 96 | | AvoidingTaskRun | 10.93 s | 1.255 s | 0.069 s | 0.91 | 23000.0000 | 1000001.0000 | 7336.0000 | 13000.0000 | 3000.0000 | 183.86 MB | 0.43 | 97 | | Baseline | 12.05 s | 0.978 s | 0.054 s | 1.00 | 55000.0000 | 2000046.0000 | 13132.0000 | 29000.0000 | 4000.0000 | 428.46 MB | 1.00 | 98 | 99 | Clearly the result is significant. 100 | - Memory is roughly halved if we get rid of the Task. 101 | - Less queueing delay in executing all the tasks with the benchmark completing in ~11s rather than ~12s as before. 102 | - A perfect result would be ~10s. 103 | 104 | #### Conclusion: Significant impact on memory usage 105 | 106 | ### Observation 3 (from kevingadd): What if a single Task.Delay was used instead of N? 107 | 108 | There are two ways I thought of to test this. The first is to create one delay, but continue to use N different tasks. I put N continuations on the delay task to achieve this. 109 | ```csharp 110 | Task delayTask = Task.Delay(TimeSpan.FromMilliseconds(delayMillisec)); 111 | List tasks = new List(); 112 | for (int i = 0; i < N; i++) 113 | { 114 | tasks.Add(delayTask.ContinueWith((Task _) => { })); 115 | } 116 | await Task.WhenAll(tasks); 117 | ``` 118 | 119 | The second is to create one delay, with one task and then await on it N times (IMO this doesn't quite match the original intent of "...N concurrent tasks, where each task waits for 10 seconds...") e.g: 120 | ```csharp 121 | Task delayTask = Task.Delay(TimeSpan.FromMilliseconds(delayMillisec)); 122 | List tasks = new List(); 123 | for (int i = 0; i < N; i++) 124 | { 125 | tasks.Add(delayTask); 126 | } 127 | await Task.WhenAll(tasks); 128 | ``` 129 | 130 | Lets see what the impact is of both approaches: 131 | | Method | Mean | Error | StdDev | Ratio | Completed Work Items | Lock Contentions | Gen0 | Gen1 | Gen2 | Allocated | Alloc Ratio | 132 | |------------------------------------------------ |--------:|--------:|--------:|------:|---------------------:|-----------------:|-----------:|-----------:|----------:|----------:|------------:| 133 | | AvoidingTaskRun_SharingSameDelay_SameTask | 10.03 s | 0.074 s | 0.004 s | 0.83 | 1.0000 | - | 1000.0000 | 1000.0000 | 1000.0000 | 39.63 MB | 0.09 | 134 | | AvoidingTaskRun_SharingSameDelay_DifferentTasks | 10.73 s | 1.583 s | 0.087 s | 0.89 | 1000001.0000 | - | 15000.0000 | 8000.0000 | 2000.0000 | 146.45 MB | 0.34 | 135 | | Baseline | 12.12 s | 0.284 s | 0.016 s | 1.00 | 2000034.0000 | 14365.0000 | 55000.0000 | 29000.0000 | 4000.0000 | 428.38 MB | 1.00 | 136 | 137 | #### Conclusion: Significant impact on memory usage 138 | - If we allow ourselves to create 1 task and wait on in N times, the improvement is ridiculous, bringing us down to ~39MB or allocations. 139 | - If we do only a single delay and still create N tasks to wait on it, the improvement is still pretty good. 140 | 141 | ### Observation 4: How do newer .NET runtimes do? .NET 7/.NET 8? 142 | 143 | This requires adding a new job to the benchmark config. Here are the results: 144 | 145 | | Method | Runtime | Mean | Error | StdDev | Ratio | Gen0 | Completed Work Items | Lock Contentions | Gen1 | Gen2 | Allocated | Alloc Ratio | 146 | |---------------- |--------- |--------:|--------:|--------:|------:|-----------:|---------------------:|-----------------:|-----------:|----------:|----------:|------------:| 147 | | Baseline | .NET 6.0 | 12.12 s | 2.904 s | 0.159 s | 1.00 | 55000.0000 | 2000033.0000 | 14887.0000 | 29000.0000 | 4000.0000 | 428.41 MB | 1.00 | 148 | | AvoidingTaskRun | .NET 6.0 | 10.63 s | 1.343 s | 0.074 s | 0.88 | 23000.0000 | 1000000.0000 | 4770.0000 | 13000.0000 | 3000.0000 | 183.85 MB | 0.43 | 149 | | | | | | | | | | | | | | | 150 | | Baseline | .NET 7.0 | 11.25 s | 0.367 s | 0.020 s | 1.00 | 55000.0000 | 2000002.0000 | 10811.0000 | 54000.0000 | 4000.0000 | 428.21 MB | 1.00 | 151 | | AvoidingTaskRun | .NET 7.0 | 10.65 s | 0.234 s | 0.013 s | 0.95 | 23000.0000 | 1000000.0000 | 4489.0000 | 22000.0000 | 3000.0000 | 183.85 MB | 0.43 | 152 | | | | | | | | | | | | | | | 153 | | Baseline | .NET 8.0 | 11.42 s | 1.991 s | 0.109 s | 1.00 | 53000.0000 | 2000015.0000 | 9165.0000 | 52000.0000 | 4000.0000 | 405.34 MB | 1.00 | 154 | | AvoidingTaskRun | .NET 8.0 | 10.63 s | 0.873 s | 0.048 s | 0.93 | 23000.0000 | 1000000.0000 | 3882.0000 | 22000.0000 | 3000.0000 | 176.22 MB | 0.43 | 155 | 156 | 157 | The improvement is minor. The memory usage has a negligible difference from .NET 6 to .NET 7. But look at those lock contentions. It's reduced by about 30% on the baseline example. That reflects improvements in the runtime itself. 158 | 159 | Moving to .NET 8 improvements things even more. Allocations are improved a bit on both the baseline and optimized version (without Task.Run(...)). And again the number of lock contentions is reduced. 160 | 161 | #### Conclusion: Minor impact on memory usage. Runtime itself greatly improved from .NET 6 -> .NET 7. 162 | --------------------------------------------------------------------------------