├── ReadLinesBenchmarks ├── ReadLinesBenchmarks.csproj └── Program.cs ├── README.md ├── ReadLinesBenchmarks.sln ├── .gitattributes └── .gitignore /ReadLinesBenchmarks/ReadLinesBenchmarks.csproj: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | Exe 5 | net5.0 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Source code referenced from my blog post 2 | 3 | ## Evaluating "ReadLine using System.IO.Pipelines" Performance in C# 4 | https://medium.com/@joni2nja/evaluating-readline-using-system-io-pipelines-performance-in-c-69e9ed658920 5 | 6 | ## Evaluating "ReadLine using System.IO.Pipelines" Performance in C# - Part 2 7 | https://medium.com/@joni2nja/evaluating-readline-using-system-io-pipelines-performance-in-c-part-2-b9d22c95254b 8 | 9 | ## Use System.IO.Pipelines and System.Threading.Channels APIs to Boost Performance 10 | https://medium.com/@joni2nja/use-system-io-pipelines-and-system-threading-channels-apis-to-boost-performance-832d7ab7c719 11 | (branch: pipelines-and-channels) 12 | -------------------------------------------------------------------------------- /ReadLinesBenchmarks.sln: -------------------------------------------------------------------------------- 1 | 2 | Microsoft Visual Studio Solution File, Format Version 12.00 3 | # Visual Studio Version 16 4 | VisualStudioVersion = 16.0.30404.54 5 | MinimumVisualStudioVersion = 10.0.40219.1 6 | Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ReadLinesBenchmarks", "ReadLinesBenchmarks\ReadLinesBenchmarks.csproj", "{0F9C56AF-9A11-485E-8B86-089D9347DE34}" 7 | EndProject 8 | Global 9 | GlobalSection(SolutionConfigurationPlatforms) = preSolution 10 | Debug|Any CPU = Debug|Any CPU 11 | Release|Any CPU = Release|Any CPU 12 | EndGlobalSection 13 | GlobalSection(ProjectConfigurationPlatforms) = postSolution 14 | {0F9C56AF-9A11-485E-8B86-089D9347DE34}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 15 | {0F9C56AF-9A11-485E-8B86-089D9347DE34}.Debug|Any CPU.Build.0 = Debug|Any CPU 16 | {0F9C56AF-9A11-485E-8B86-089D9347DE34}.Release|Any CPU.ActiveCfg = Release|Any CPU 17 | {0F9C56AF-9A11-485E-8B86-089D9347DE34}.Release|Any CPU.Build.0 = Release|Any CPU 18 | EndGlobalSection 19 | GlobalSection(SolutionProperties) = preSolution 20 | HideSolutionNode = FALSE 21 | EndGlobalSection 22 | GlobalSection(ExtensibilityGlobals) = postSolution 23 | SolutionGuid = {A893994E-E1F2-435E-BCCB-5C4DA445316E} 24 | EndGlobalSection 25 | EndGlobal 26 | -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- 1 | ############################################################################### 2 | # Set default behavior to automatically normalize line endings. 3 | ############################################################################### 4 | * text=auto 5 | 6 | ############################################################################### 7 | # Set default behavior for command prompt diff. 8 | # 9 | # This is need for earlier builds of msysgit that does not have it on by 10 | # default for csharp files. 11 | # Note: This is only used by command line 12 | ############################################################################### 13 | #*.cs diff=csharp 14 | 15 | ############################################################################### 16 | # Set the merge driver for project and solution files 17 | # 18 | # Merging from the command prompt will add diff markers to the files if there 19 | # are conflicts (Merging from VS is not affected by the settings below, in VS 20 | # the diff markers are never inserted). Diff markers may cause the following 21 | # file extensions to fail to load in VS. An alternative would be to treat 22 | # these files as binary and thus will always conflict and require user 23 | # intervention with every merge. To do so, just uncomment the entries below 24 | ############################################################################### 25 | #*.sln merge=binary 26 | #*.csproj merge=binary 27 | #*.vbproj merge=binary 28 | #*.vcxproj merge=binary 29 | #*.vcproj merge=binary 30 | #*.dbproj merge=binary 31 | #*.fsproj merge=binary 32 | #*.lsproj merge=binary 33 | #*.wixproj merge=binary 34 | #*.modelproj merge=binary 35 | #*.sqlproj merge=binary 36 | #*.wwaproj merge=binary 37 | 38 | ############################################################################### 39 | # behavior for image files 40 | # 41 | # image files are treated as binary by default. 42 | ############################################################################### 43 | #*.jpg binary 44 | #*.png binary 45 | #*.gif binary 46 | 47 | ############################################################################### 48 | # diff behavior for common document formats 49 | # 50 | # Convert binary document formats to text before diffing them. This feature 51 | # is only available from the command line. Turn it on by uncommenting the 52 | # entries below. 53 | ############################################################################### 54 | #*.doc diff=astextplain 55 | #*.DOC diff=astextplain 56 | #*.docx diff=astextplain 57 | #*.DOCX diff=astextplain 58 | #*.dot diff=astextplain 59 | #*.DOT diff=astextplain 60 | #*.pdf diff=astextplain 61 | #*.PDF diff=astextplain 62 | #*.rtf diff=astextplain 63 | #*.RTF diff=astextplain 64 | -------------------------------------------------------------------------------- /ReadLinesBenchmarks/Program.cs: -------------------------------------------------------------------------------- 1 | using BenchmarkDotNet.Attributes; 2 | using BenchmarkDotNet.Running; 3 | using System; 4 | using System.Buffers; 5 | using System.Collections.Generic; 6 | using System.IO; 7 | using System.IO.Pipelines; 8 | using System.Linq; 9 | using System.Runtime.CompilerServices; 10 | using System.Text; 11 | using System.Threading.Tasks; 12 | 13 | namespace ReadLinesBenchmarks 14 | { 15 | [DisassemblyDiagnoser(printSource: true)] 16 | [MemoryDiagnoser] 17 | public class ReadLinesBenchmarks 18 | { 19 | private static ReadOnlySpan NewLine => new[] { (byte)'\r', (byte)'\n' }; 20 | 21 | private Stream _stream; 22 | 23 | [Params(20/*, 300_000*/)] 24 | public int LineNumber { get; set; } 25 | 26 | [ParamsSource(nameof(LineCharMultiplierValues))] 27 | public int LineCharMultiplier { get; set; } 28 | public IEnumerable LineCharMultiplierValues => new[] { 1, 2, 8, 1000 }; 29 | //public IEnumerable LineCharMultiplierValues => Enumerable.Range(1, 15).Concat(new[] { 20, 30, 50, 80, 100 }); 30 | 31 | [GlobalSetup] 32 | public void GlobalSetup() 33 | { 34 | _stream = PrepareStream(); 35 | } 36 | 37 | [GlobalCleanup] 38 | public void GlobalCleanup() 39 | { 40 | _stream.Dispose(); 41 | } 42 | 43 | public Stream PrepareStream() 44 | { 45 | var stream = new MemoryStream(); 46 | 47 | using var sw = new StreamWriter(stream, Encoding.UTF8, leaveOpen: true); 48 | foreach (var no in Enumerable.Range(1, LineNumber)) 49 | { 50 | foreach (var _ in Enumerable.Range(1, LineCharMultiplier)) 51 | { 52 | sw.Write($"ABC{no:D7}"); 53 | } 54 | sw.WriteLine(); 55 | } 56 | sw.Flush(); 57 | 58 | return stream; 59 | } 60 | 61 | [Benchmark(Baseline = true)] 62 | public async Task ReadLineUsingStringReaderAsync() 63 | { 64 | _stream.Seek(0, SeekOrigin.Begin); 65 | 66 | var sr = new StreamReader(_stream, Encoding.UTF8); 67 | string str; 68 | while ((str = await sr.ReadLineAsync()) is not null) 69 | { 70 | // simulate string processing 71 | str = str.AsSpan().Slice(0, 5).ToString(); 72 | } 73 | 74 | return str; 75 | } 76 | 77 | [Benchmark] 78 | public async Task ReadLineUsingPipelineAsync() 79 | { 80 | _stream.Seek(0, SeekOrigin.Begin); 81 | 82 | var reader = PipeReader.Create(_stream, new StreamPipeReaderOptions(leaveOpen: true)); 83 | string str; 84 | while (true) 85 | { 86 | ReadResult result = await reader.ReadAsync(); 87 | ReadOnlySequence buffer = result.Buffer; 88 | 89 | while ((str = ReadLine(ref buffer)) is not null) 90 | { 91 | // simulate string processing 92 | str = str.AsSpan().Slice(0, 5).ToString(); 93 | } 94 | 95 | reader.AdvanceTo(buffer.Start, buffer.End); 96 | 97 | if (result.IsCompleted) break; 98 | } 99 | 100 | await reader.CompleteAsync(); 101 | return str; 102 | } 103 | 104 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 105 | private static string ReadLine(ref ReadOnlySequence buffer) 106 | { 107 | var reader = new SequenceReader(buffer); 108 | 109 | if (reader.TryReadTo(out var line, NewLine)) 110 | { 111 | buffer = buffer.Slice(reader.Position); 112 | return Encoding.UTF8.GetString(line); 113 | } 114 | 115 | return default; 116 | } 117 | 118 | [Benchmark] 119 | public async Task ReadLineUsingPipelineVer2Async() 120 | { 121 | _stream.Seek(0, SeekOrigin.Begin); 122 | 123 | var reader = PipeReader.Create(_stream, new StreamPipeReaderOptions(leaveOpen: true)); 124 | string str; 125 | 126 | while (true) 127 | { 128 | ReadResult result = await reader.ReadAsync(); 129 | ReadOnlySequence buffer = result.Buffer; 130 | 131 | str = ProcessLine(ref buffer); 132 | 133 | reader.AdvanceTo(buffer.Start, buffer.End); 134 | 135 | if (result.IsCompleted) break; 136 | } 137 | 138 | await reader.CompleteAsync(); 139 | return str; 140 | } 141 | 142 | [MethodImpl(MethodImplOptions.AggressiveInlining)] 143 | private static string ProcessLine(ref ReadOnlySequence buffer) 144 | { 145 | string str = null; 146 | 147 | if (buffer.IsSingleSegment) 148 | { 149 | var span = buffer.FirstSpan; 150 | int consumed; 151 | while (span.Length > 0) 152 | { 153 | var newLine = span.IndexOf(NewLine); 154 | 155 | if (newLine == -1) break; 156 | 157 | var line = span.Slice(0, newLine); 158 | str = Encoding.UTF8.GetString(line); 159 | 160 | // simulate string processing 161 | str = str.AsSpan().Slice(0, 5).ToString(); 162 | 163 | consumed = line.Length + NewLine.Length; 164 | span = span.Slice(consumed); 165 | buffer = buffer.Slice(consumed); 166 | } 167 | } 168 | else 169 | { 170 | var sequenceReader = new SequenceReader(buffer); 171 | 172 | while (!sequenceReader.End) 173 | { 174 | while (sequenceReader.TryReadTo(out var line, NewLine)) 175 | { 176 | str = Encoding.UTF8.GetString(line); 177 | 178 | // simulate string processing 179 | str = str.AsSpan().Slice(0, 5).ToString(); 180 | } 181 | 182 | buffer = buffer.Slice(sequenceReader.Position); 183 | sequenceReader.Advance(buffer.Length); 184 | } 185 | } 186 | 187 | return str; 188 | } 189 | } 190 | 191 | class Program 192 | { 193 | static void Main(string[] args) 194 | { 195 | #if DEBUG 196 | var summary = BenchmarkRunner.Run(new BenchmarkDotNet.Configs.DebugInProcessConfig()); 197 | #else 198 | var summary = BenchmarkRunner.Run(); 199 | #endif 200 | } 201 | } 202 | } 203 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | ## Ignore Visual Studio temporary files, build results, and 2 | ## files generated by popular Visual Studio add-ons. 3 | ## 4 | ## Get latest from https://github.com/github/gitignore/blob/master/VisualStudio.gitignore 5 | 6 | # User-specific files 7 | *.rsuser 8 | *.suo 9 | *.user 10 | *.userosscache 11 | *.sln.docstates 12 | 13 | # User-specific files (MonoDevelop/Xamarin Studio) 14 | *.userprefs 15 | 16 | # Build results 17 | [Dd]ebug/ 18 | [Dd]ebugPublic/ 19 | [Rr]elease/ 20 | [Rr]eleases/ 21 | x64/ 22 | x86/ 23 | [Aa][Rr][Mm]/ 24 | [Aa][Rr][Mm]64/ 25 | bld/ 26 | [Bb]in/ 27 | [Oo]bj/ 28 | [Ll]og/ 29 | 30 | # Visual Studio 2015/2017 cache/options directory 31 | .vs/ 32 | # Uncomment if you have tasks that create the project's static files in wwwroot 33 | #wwwroot/ 34 | 35 | # Visual Studio 2017 auto generated files 36 | Generated\ Files/ 37 | 38 | # MSTest test Results 39 | [Tt]est[Rr]esult*/ 40 | [Bb]uild[Ll]og.* 41 | 42 | # NUNIT 43 | *.VisualState.xml 44 | TestResult.xml 45 | 46 | # Build Results of an ATL Project 47 | [Dd]ebugPS/ 48 | [Rr]eleasePS/ 49 | dlldata.c 50 | 51 | # Benchmark Results 52 | BenchmarkDotNet.Artifacts/ 53 | 54 | # .NET Core 55 | project.lock.json 56 | project.fragment.lock.json 57 | artifacts/ 58 | 59 | # StyleCop 60 | StyleCopReport.xml 61 | 62 | # Files built by Visual Studio 63 | *_i.c 64 | *_p.c 65 | *_h.h 66 | *.ilk 67 | *.meta 68 | *.obj 69 | *.iobj 70 | *.pch 71 | *.pdb 72 | *.ipdb 73 | *.pgc 74 | *.pgd 75 | *.rsp 76 | *.sbr 77 | *.tlb 78 | *.tli 79 | *.tlh 80 | *.tmp 81 | *.tmp_proj 82 | *_wpftmp.csproj 83 | *.log 84 | *.vspscc 85 | *.vssscc 86 | .builds 87 | *.pidb 88 | *.svclog 89 | *.scc 90 | 91 | # Chutzpah Test files 92 | _Chutzpah* 93 | 94 | # Visual C++ cache files 95 | ipch/ 96 | *.aps 97 | *.ncb 98 | *.opendb 99 | *.opensdf 100 | *.sdf 101 | *.cachefile 102 | *.VC.db 103 | *.VC.VC.opendb 104 | 105 | # Visual Studio profiler 106 | *.psess 107 | *.vsp 108 | *.vspx 109 | *.sap 110 | 111 | # Visual Studio Trace Files 112 | *.e2e 113 | 114 | # TFS 2012 Local Workspace 115 | $tf/ 116 | 117 | # Guidance Automation Toolkit 118 | *.gpState 119 | 120 | # ReSharper is a .NET coding add-in 121 | _ReSharper*/ 122 | *.[Rr]e[Ss]harper 123 | *.DotSettings.user 124 | 125 | # JustCode is a .NET coding add-in 126 | .JustCode 127 | 128 | # TeamCity is a build add-in 129 | _TeamCity* 130 | 131 | # DotCover is a Code Coverage Tool 132 | *.dotCover 133 | 134 | # AxoCover is a Code Coverage Tool 135 | .axoCover/* 136 | !.axoCover/settings.json 137 | 138 | # Visual Studio code coverage results 139 | *.coverage 140 | *.coveragexml 141 | 142 | # NCrunch 143 | _NCrunch_* 144 | .*crunch*.local.xml 145 | nCrunchTemp_* 146 | 147 | # MightyMoose 148 | *.mm.* 149 | AutoTest.Net/ 150 | 151 | # Web workbench (sass) 152 | .sass-cache/ 153 | 154 | # Installshield output folder 155 | [Ee]xpress/ 156 | 157 | # DocProject is a documentation generator add-in 158 | DocProject/buildhelp/ 159 | DocProject/Help/*.HxT 160 | DocProject/Help/*.HxC 161 | DocProject/Help/*.hhc 162 | DocProject/Help/*.hhk 163 | DocProject/Help/*.hhp 164 | DocProject/Help/Html2 165 | DocProject/Help/html 166 | 167 | # Click-Once directory 168 | publish/ 169 | 170 | # Publish Web Output 171 | *.[Pp]ublish.xml 172 | *.azurePubxml 173 | # Note: Comment the next line if you want to checkin your web deploy settings, 174 | # but database connection strings (with potential passwords) will be unencrypted 175 | *.pubxml 176 | *.publishproj 177 | 178 | # Microsoft Azure Web App publish settings. Comment the next line if you want to 179 | # checkin your Azure Web App publish settings, but sensitive information contained 180 | # in these scripts will be unencrypted 181 | PublishScripts/ 182 | 183 | # NuGet Packages 184 | *.nupkg 185 | # The packages folder can be ignored because of Package Restore 186 | **/[Pp]ackages/* 187 | # except build/, which is used as an MSBuild target. 188 | !**/[Pp]ackages/build/ 189 | # Uncomment if necessary however generally it will be regenerated when needed 190 | #!**/[Pp]ackages/repositories.config 191 | # NuGet v3's project.json files produces more ignorable files 192 | *.nuget.props 193 | *.nuget.targets 194 | 195 | # Microsoft Azure Build Output 196 | csx/ 197 | *.build.csdef 198 | 199 | # Microsoft Azure Emulator 200 | ecf/ 201 | rcf/ 202 | 203 | # Windows Store app package directories and files 204 | AppPackages/ 205 | BundleArtifacts/ 206 | Package.StoreAssociation.xml 207 | _pkginfo.txt 208 | *.appx 209 | 210 | # Visual Studio cache files 211 | # files ending in .cache can be ignored 212 | *.[Cc]ache 213 | # but keep track of directories ending in .cache 214 | !?*.[Cc]ache/ 215 | 216 | # Others 217 | ClientBin/ 218 | ~$* 219 | *~ 220 | *.dbmdl 221 | *.dbproj.schemaview 222 | *.jfm 223 | *.pfx 224 | *.publishsettings 225 | orleans.codegen.cs 226 | 227 | # Including strong name files can present a security risk 228 | # (https://github.com/github/gitignore/pull/2483#issue-259490424) 229 | #*.snk 230 | 231 | # Since there are multiple workflows, uncomment next line to ignore bower_components 232 | # (https://github.com/github/gitignore/pull/1529#issuecomment-104372622) 233 | #bower_components/ 234 | 235 | # RIA/Silverlight projects 236 | Generated_Code/ 237 | 238 | # Backup & report files from converting an old project file 239 | # to a newer Visual Studio version. Backup files are not needed, 240 | # because we have git ;-) 241 | _UpgradeReport_Files/ 242 | Backup*/ 243 | UpgradeLog*.XML 244 | UpgradeLog*.htm 245 | ServiceFabricBackup/ 246 | *.rptproj.bak 247 | 248 | # SQL Server files 249 | *.mdf 250 | *.ldf 251 | *.ndf 252 | 253 | # Business Intelligence projects 254 | *.rdl.data 255 | *.bim.layout 256 | *.bim_*.settings 257 | *.rptproj.rsuser 258 | *- Backup*.rdl 259 | 260 | # Microsoft Fakes 261 | FakesAssemblies/ 262 | 263 | # GhostDoc plugin setting file 264 | *.GhostDoc.xml 265 | 266 | # Node.js Tools for Visual Studio 267 | .ntvs_analysis.dat 268 | node_modules/ 269 | 270 | # Visual Studio 6 build log 271 | *.plg 272 | 273 | # Visual Studio 6 workspace options file 274 | *.opt 275 | 276 | # Visual Studio 6 auto-generated workspace file (contains which files were open etc.) 277 | *.vbw 278 | 279 | # Visual Studio LightSwitch build output 280 | **/*.HTMLClient/GeneratedArtifacts 281 | **/*.DesktopClient/GeneratedArtifacts 282 | **/*.DesktopClient/ModelManifest.xml 283 | **/*.Server/GeneratedArtifacts 284 | **/*.Server/ModelManifest.xml 285 | _Pvt_Extensions 286 | 287 | # Paket dependency manager 288 | .paket/paket.exe 289 | paket-files/ 290 | 291 | # FAKE - F# Make 292 | .fake/ 293 | 294 | # JetBrains Rider 295 | .idea/ 296 | *.sln.iml 297 | 298 | # CodeRush personal settings 299 | .cr/personal 300 | 301 | # Python Tools for Visual Studio (PTVS) 302 | __pycache__/ 303 | *.pyc 304 | 305 | # Cake - Uncomment if you are using it 306 | # tools/** 307 | # !tools/packages.config 308 | 309 | # Tabs Studio 310 | *.tss 311 | 312 | # Telerik's JustMock configuration file 313 | *.jmconfig 314 | 315 | # BizTalk build output 316 | *.btp.cs 317 | *.btm.cs 318 | *.odx.cs 319 | *.xsd.cs 320 | 321 | # OpenCover UI analysis results 322 | OpenCover/ 323 | 324 | # Azure Stream Analytics local run output 325 | ASALocalRun/ 326 | 327 | # MSBuild Binary and Structured Log 328 | *.binlog 329 | 330 | # NVidia Nsight GPU debugger configuration file 331 | *.nvuser 332 | 333 | # MFractors (Xamarin productivity tool) working folder 334 | .mfractor/ 335 | 336 | # Local History for Visual Studio 337 | .localhistory/ 338 | 339 | # BeatPulse healthcheck temp database 340 | healthchecksdb --------------------------------------------------------------------------------