├── .assets ├── annotations.png └── summary.png ├── .github ├── ISSUE_TEMPLATE │ ├── bug-report.yml │ └── config.yml ├── dependabot.yml └── workflows │ └── main.yml ├── .gitignore ├── Directory.Build.props ├── GitHubActionsTestLogger.Demo ├── GitHubActionsTestLogger.Demo.csproj ├── Readme.md └── SampleTests.cs ├── GitHubActionsTestLogger.Tests ├── AnnotationSpecs.cs ├── Fakes │ └── FakeTestLoggerEvents.cs ├── GitHubActionsTestLogger.Tests.csproj ├── InitializationSpecs.cs ├── SummarySpecs.cs ├── Utils │ ├── Extensions │ │ └── TestLoggerContextExtensions.cs │ └── TestResultBuilder.cs └── xunit.runner.json ├── GitHubActionsTestLogger.sln ├── GitHubActionsTestLogger ├── GitHubActionsTestLogger.csproj ├── GitHubWorkflow.cs ├── TestLogger.cs ├── TestLoggerContext.cs ├── TestLoggerOptions.cs ├── TestRunStatistics.cs ├── TestSummaryTemplate.cshtml └── Utils │ ├── ContentionTolerantWriteFileStream.cs │ ├── Extensions │ ├── GenericExtensions.cs │ ├── StringExtensions.cs │ ├── TestCaseExtensions.cs │ ├── TestResultExtensions.cs │ ├── TestRunCriteriaExtensions.cs │ └── TimeSpanExtensions.cs │ ├── PathEx.cs │ ├── RandomEx.cs │ └── StackFrame.cs ├── License.txt ├── NuGet.config ├── Readme.md └── favicon.png /.assets/annotations.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Tyrrrz/GitHubActionsTestLogger/81305a08e78b05e45ef81e4ae768497b1f8700b7/.assets/annotations.png -------------------------------------------------------------------------------- /.assets/summary.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Tyrrrz/GitHubActionsTestLogger/81305a08e78b05e45ef81e4ae768497b1f8700b7/.assets/summary.png -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/bug-report.yml: -------------------------------------------------------------------------------- 1 | name: 🐛 Bug report 2 | description: Report broken functionality. 3 | labels: [bug] 4 | 5 | body: 6 | - type: markdown 7 | attributes: 8 | value: | 9 | - Avoid generic or vague titles such as "Something's not working" or "A couple of problems" — be as descriptive as possible. 10 | - Keep your issue focused on one single problem. If you have multiple bug reports, please create a separate issue for each of them. 11 | - Issues should represent **complete and actionable** work items. If you are unsure about something or have a question, please start a [discussion](https://github.com/Tyrrrz/GitHubActionsTestLogger/discussions/new) instead. 12 | - Remember that **GitHubActionsTestLogger** is an open-source project funded by the community. If you find it useful, **please consider [donating](https://tyrrrz.me/donate) to support its development**. 13 | 14 | ___ 15 | 16 | - type: input 17 | attributes: 18 | label: Version 19 | description: Which version of the package does this bug affect? Make sure you're not using an outdated version. 20 | placeholder: v1.0.0 21 | validations: 22 | required: true 23 | 24 | - type: input 25 | attributes: 26 | label: Platform 27 | description: Which platform do you experience this bug on? 28 | placeholder: .NET 7.0 / Windows 11 29 | validations: 30 | required: true 31 | 32 | - type: textarea 33 | attributes: 34 | label: Steps to reproduce 35 | description: > 36 | Minimum steps required to reproduce the bug, including prerequisites, code snippets, or other relevant items. 37 | The information provided in this field must be readily actionable, meaning that anyone should be able to reproduce the bug by following these steps. 38 | If the reproduction steps are too complex to fit in this field, please provide a link to a repository instead. 39 | placeholder: | 40 | - Step 1 41 | - Step 2 42 | - Step 3 43 | validations: 44 | required: true 45 | 46 | - type: textarea 47 | attributes: 48 | label: Details 49 | description: Clear and thorough explanation of the bug, including any additional information you may find relevant. 50 | placeholder: | 51 | - Expected behavior: ... 52 | - Actual behavior: ... 53 | validations: 54 | required: true 55 | 56 | - type: checkboxes 57 | attributes: 58 | label: Checklist 59 | description: Quick list of checks to ensure that everything is in order. 60 | options: 61 | - label: I have looked through existing issues to make sure that this bug has not been reported before 62 | required: true 63 | - label: I have provided a descriptive title for this issue 64 | required: true 65 | - label: I have made sure that this bug is reproducible on the latest version of the package 66 | required: true 67 | - label: I have provided all the information needed to reproduce this bug as efficiently as possible 68 | required: true 69 | - label: I have sponsored this project 70 | required: false 71 | 72 | - type: markdown 73 | attributes: 74 | value: | 75 | If you are struggling to provide actionable reproduction steps, or if something else is preventing you from creating a complete bug report, please start a [discussion](https://github.com/Tyrrrz/GitHubActionsTestLogger/discussions/new) instead. 76 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/config.yml: -------------------------------------------------------------------------------- 1 | blank_issues_enabled: false 2 | contact_links: 3 | - name: ⚠ Feature request 4 | url: https://github.com/Tyrrrz/.github/blob/master/docs/project-status.md 5 | about: Sorry, but this project is in maintenance mode and no longer accepts new feature requests. 6 | - name: 🗨 Discussions 7 | url: https://github.com/Tyrrrz/GitHubActionsTestLogger/discussions/new 8 | about: Ask and answer questions. 9 | - name: 💬 Discord server 10 | url: https://discord.gg/2SUWKFnHSm 11 | about: Chat with the project community. 12 | -------------------------------------------------------------------------------- /.github/dependabot.yml: -------------------------------------------------------------------------------- 1 | version: 2 2 | updates: 3 | - package-ecosystem: github-actions 4 | directory: "/" 5 | schedule: 6 | interval: monthly 7 | labels: 8 | - enhancement 9 | groups: 10 | actions: 11 | patterns: 12 | - "*" 13 | - package-ecosystem: nuget 14 | directory: "/" 15 | schedule: 16 | interval: monthly 17 | labels: 18 | - enhancement 19 | groups: 20 | nuget: 21 | patterns: 22 | - "*" 23 | -------------------------------------------------------------------------------- /.github/workflows/main.yml: -------------------------------------------------------------------------------- 1 | name: main 2 | 3 | on: 4 | workflow_dispatch: 5 | inputs: 6 | package-version: 7 | type: string 8 | description: Package version 9 | required: false 10 | deploy: 11 | type: boolean 12 | description: Deploy package 13 | required: false 14 | default: false 15 | push: 16 | branches: 17 | - master 18 | tags: 19 | - "*" 20 | pull_request: 21 | branches: 22 | - master 23 | 24 | jobs: 25 | main: 26 | uses: Tyrrrz/.github/.github/workflows/nuget.yml@master 27 | with: 28 | deploy: ${{ inputs.deploy || github.ref_type == 'tag' }} 29 | package-version: ${{ inputs.package-version || (github.ref_type == 'tag' && github.ref_name) || format('0.0.0-ci-{0}', github.sha) }} 30 | dotnet-version: 9.0.x 31 | # Don't use the default logger (which is GitHubActionsTestLogger) because that 32 | # causes contention issues since we're literally building and testing that logger. 33 | dotnet-test-logger: console 34 | secrets: 35 | CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} 36 | NUGET_TOKEN: ${{ secrets.NUGET_TOKEN }} 37 | DISCORD_WEBHOOK: ${{ secrets.DISCORD_WEBHOOK }} 38 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # User-specific files 2 | .vs/ 3 | .idea/ 4 | *.suo 5 | *.user 6 | 7 | # Build results 8 | bin/ 9 | obj/ 10 | 11 | # Test results 12 | TestResults/ -------------------------------------------------------------------------------- /Directory.Build.props: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 0.0.0-dev 5 | Tyrrrz 6 | Copyright (C) Oleksii Holub 7 | latest 8 | enable 9 | true 10 | false 11 | false 12 | 13 | 14 | 15 | 16 | annotations 17 | 18 | 19 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger.Demo/GitHubActionsTestLogger.Demo.csproj: -------------------------------------------------------------------------------- 1 | 2 | 3 | net9.0 4 | 5 | false 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger.Demo/Readme.md: -------------------------------------------------------------------------------- 1 | # GitHub Actions Test Logger Demo 2 | 3 | To run the demo tests, use the following command: 4 | 5 | ```console 6 | $ dotnet test -p:IsTestProject=true --logger GitHubActions 7 | ``` 8 | 9 | To produce a test summary, provide the output file path by setting the `GITHUB_STEP_SUMMARY` environment variable: 10 | 11 | ```powershell 12 | $env:GITHUB_STEP_SUMMARY="./test-summary.md" 13 | ``` 14 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger.Demo/SampleTests.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using Xunit; 3 | 4 | namespace GitHubActionsTestLogger.Demo; 5 | 6 | public class SampleTests 7 | { 8 | [Fact] 9 | public void Test1() => Assert.True(true); 10 | 11 | [Fact] 12 | public void Test2() => throw new InvalidOperationException(); 13 | 14 | [Fact] 15 | public void Test3() => Assert.True(false); 16 | } 17 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger.Tests/AnnotationSpecs.cs: -------------------------------------------------------------------------------- 1 | using System.IO; 2 | using FluentAssertions; 3 | using GitHubActionsTestLogger.Tests.Utils; 4 | using GitHubActionsTestLogger.Tests.Utils.Extensions; 5 | using Microsoft.VisualStudio.TestPlatform.ObjectModel; 6 | using Xunit; 7 | using Xunit.Abstractions; 8 | 9 | namespace GitHubActionsTestLogger.Tests; 10 | 11 | public class AnnotationSpecs(ITestOutputHelper testOutput) 12 | { 13 | [Fact] 14 | public void I_can_use_the_logger_to_produce_annotations_for_failed_tests() 15 | { 16 | // Arrange 17 | using var commandWriter = new StringWriter(); 18 | 19 | var context = new TestLoggerContext( 20 | new GitHubWorkflow(commandWriter, TextWriter.Null), 21 | TestLoggerOptions.Default 22 | ); 23 | 24 | // Act 25 | context.SimulateTestRun( 26 | new TestResultBuilder() 27 | .SetDisplayName("Test1") 28 | .SetOutcome(TestOutcome.Failed) 29 | .SetErrorMessage("ErrorMessage") 30 | .Build(), 31 | new TestResultBuilder().SetDisplayName("Test2").SetOutcome(TestOutcome.Passed).Build(), 32 | new TestResultBuilder().SetDisplayName("Test3").SetOutcome(TestOutcome.Skipped).Build() 33 | ); 34 | 35 | // Assert 36 | var output = commandWriter.ToString().Trim(); 37 | 38 | output.Should().StartWith("::error "); 39 | output.Should().Contain("Test1"); 40 | output.Should().Contain("ErrorMessage"); 41 | 42 | output.Should().NotContain("Test2"); 43 | output.Should().NotContain("Test3"); 44 | 45 | testOutput.WriteLine(output); 46 | } 47 | 48 | [Fact] 49 | public void I_can_use_the_logger_to_produce_annotations_that_include_source_information() 50 | { 51 | // .NET test platform never sends source information, so we can only 52 | // rely on exception stack traces to get it. 53 | 54 | // Arrange 55 | using var commandWriter = new StringWriter(); 56 | 57 | var context = new TestLoggerContext( 58 | new GitHubWorkflow(commandWriter, TextWriter.Null), 59 | TestLoggerOptions.Default 60 | ); 61 | 62 | // Act 63 | context.SimulateTestRun( 64 | new TestResultBuilder() 65 | .SetDisplayName("I can execute a command with buffering and cancel it immediately") 66 | .SetFullyQualifiedName( 67 | "CliWrap.Tests.CancellationSpecs.I_can_execute_a_command_with_buffering_and_cancel_it_immediately()" 68 | ) 69 | .SetOutcome(TestOutcome.Failed) 70 | .SetErrorMessage("ErrorMessage") 71 | .SetErrorStackTrace( 72 | """ 73 | at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message) 74 | at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message) 75 | at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message) 76 | at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc) 77 | at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc) 78 | at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args) 79 | at FluentAssertions.Primitives.BooleanAssertions.BeFalse(String because, Object[] becauseArgs) 80 | at CliWrap.Tests.CancellationSpecs.I_can_execute_a_command_with_buffering_and_cancel_it_immediately() in /dir/CliWrap.Tests/CancellationSpecs.cs:line 75 81 | """ 82 | ) 83 | .Build() 84 | ); 85 | 86 | // Assert 87 | var output = commandWriter.ToString().Trim(); 88 | 89 | output.Should().StartWith("::error "); 90 | output.Should().Contain("file=/dir/CliWrap.Tests/CancellationSpecs.cs"); 91 | output.Should().Contain("line=75"); 92 | output.Should().Contain("I can execute a command with buffering and cancel it immediately"); 93 | output.Should().Contain("ErrorMessage"); 94 | 95 | testOutput.WriteLine(output); 96 | } 97 | 98 | [Fact] 99 | public void I_can_use_the_logger_to_produce_annotations_that_include_source_information_for_async_tests() 100 | { 101 | // .NET test platform never sends source information, so we can only 102 | // rely on exception stack traces to get it. 103 | 104 | // Arrange 105 | using var commandWriter = new StringWriter(); 106 | 107 | var context = new TestLoggerContext( 108 | new GitHubWorkflow(commandWriter, TextWriter.Null), 109 | TestLoggerOptions.Default 110 | ); 111 | 112 | // Act 113 | context.SimulateTestRun( 114 | new TestResultBuilder() 115 | .SetDisplayName("SendEnvelopeAsync_ItemRateLimit_DropsItem") 116 | .SetFullyQualifiedName( 117 | "Sentry.Tests.Internals.Http.HttpTransportTests.SendEnvelopeAsync_ItemRateLimit_DropsItem()" 118 | ) 119 | .SetOutcome(TestOutcome.Failed) 120 | .SetErrorMessage("ErrorMessage") 121 | .SetErrorStackTrace( 122 | """ 123 | at System.Net.Http.HttpContent.CheckDisposed() 124 | at System.Net.Http.HttpContent.ReadAsStringAsync() 125 | at Sentry.Tests.Internals.Http.HttpTransportTests.d__3.MoveNext() in /dir/Sentry.Tests/Internals/Http/HttpTransportTests.cs:line 187 126 | --- End of stack trace from previous location where exception was thrown --- 127 | at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 128 | at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 129 | --- End of stack trace from previous location where exception was thrown --- 130 | at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 131 | at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 132 | --- End of stack trace from previous location where exception was thrown --- 133 | at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 134 | at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 135 | """ 136 | ) 137 | .Build() 138 | ); 139 | 140 | // Assert 141 | var output = commandWriter.ToString().Trim(); 142 | 143 | output.Should().StartWith("::error "); 144 | output.Should().Contain("file=/dir/Sentry.Tests/Internals/Http/HttpTransportTests.cs"); 145 | output.Should().Contain("line=187"); 146 | output.Should().Contain("SendEnvelopeAsync_ItemRateLimit_DropsItem"); 147 | output.Should().Contain("ErrorMessage"); 148 | 149 | testOutput.WriteLine(output); 150 | } 151 | 152 | [Fact] 153 | public void I_can_use_the_logger_to_produce_annotations_that_include_the_test_name() 154 | { 155 | // Arrange 156 | using var commandWriter = new StringWriter(); 157 | 158 | var context = new TestLoggerContext( 159 | new GitHubWorkflow(commandWriter, TextWriter.Null), 160 | new TestLoggerOptions 161 | { 162 | AnnotationTitleFormat = "<@test>", 163 | AnnotationMessageFormat = "[@test]", 164 | } 165 | ); 166 | 167 | // Act 168 | context.SimulateTestRun( 169 | new TestResultBuilder().SetDisplayName("Test1").SetOutcome(TestOutcome.Failed).Build() 170 | ); 171 | 172 | // Assert 173 | var output = commandWriter.ToString().Trim(); 174 | 175 | output.Should().Contain(""); 176 | output.Should().Contain("[Test1]"); 177 | 178 | testOutput.WriteLine(output); 179 | } 180 | 181 | [Fact] 182 | public void I_can_use_the_logger_to_produce_annotations_that_include_test_traits() 183 | { 184 | // Arrange 185 | using var commandWriter = new StringWriter(); 186 | 187 | var context = new TestLoggerContext( 188 | new GitHubWorkflow(commandWriter, TextWriter.Null), 189 | new TestLoggerOptions 190 | { 191 | AnnotationTitleFormat = "<@traits.Category -> @test>", 192 | AnnotationMessageFormat = "[@traits.Category -> @test]", 193 | } 194 | ); 195 | 196 | // Act 197 | context.SimulateTestRun( 198 | new TestResultBuilder() 199 | .SetDisplayName("Test1") 200 | .SetTrait("Category", "UI Test") 201 | .SetTrait("Document", "SS01") 202 | .SetOutcome(TestOutcome.Failed) 203 | .Build() 204 | ); 205 | 206 | // Assert 207 | var output = commandWriter.ToString().Trim(); 208 | 209 | output.Should().Contain(" Test1>"); 210 | output.Should().Contain("[UI Test -> Test1]"); 211 | 212 | testOutput.WriteLine(output); 213 | } 214 | 215 | [Fact] 216 | public void I_can_use_the_logger_to_produce_annotations_that_include_the_error_message() 217 | { 218 | // Arrange 219 | using var commandWriter = new StringWriter(); 220 | 221 | var context = new TestLoggerContext( 222 | new GitHubWorkflow(commandWriter, TextWriter.Null), 223 | new TestLoggerOptions 224 | { 225 | AnnotationTitleFormat = "<@test: @error>", 226 | AnnotationMessageFormat = "[@test: @error]", 227 | } 228 | ); 229 | 230 | // Act 231 | context.SimulateTestRun( 232 | new TestResultBuilder() 233 | .SetDisplayName("Test1") 234 | .SetOutcome(TestOutcome.Failed) 235 | .SetErrorMessage("ErrorMessage") 236 | .Build() 237 | ); 238 | 239 | // Assert 240 | var output = commandWriter.ToString().Trim(); 241 | 242 | output.Should().Contain(""); 243 | output.Should().Contain("[Test1: ErrorMessage]"); 244 | 245 | testOutput.WriteLine(output); 246 | } 247 | 248 | [Fact] 249 | public void I_can_use_the_logger_to_produce_annotations_that_include_the_error_stacktrace() 250 | { 251 | // Arrange 252 | using var commandWriter = new StringWriter(); 253 | 254 | var context = new TestLoggerContext( 255 | new GitHubWorkflow(commandWriter, TextWriter.Null), 256 | new TestLoggerOptions 257 | { 258 | AnnotationTitleFormat = "<@test: @trace>", 259 | AnnotationMessageFormat = "[@test: @trace]", 260 | } 261 | ); 262 | 263 | // Act 264 | context.SimulateTestRun( 265 | new TestResultBuilder() 266 | .SetDisplayName("Test1") 267 | .SetOutcome(TestOutcome.Failed) 268 | .SetErrorStackTrace("ErrorStackTrace") 269 | .Build() 270 | ); 271 | 272 | // Assert 273 | var output = commandWriter.ToString().Trim(); 274 | 275 | output.Should().Contain(""); 276 | output.Should().Contain("[Test1: ErrorStackTrace]"); 277 | 278 | testOutput.WriteLine(output); 279 | } 280 | 281 | [Fact] 282 | public void I_can_use_the_logger_to_produce_annotations_that_include_the_target_framework_version() 283 | { 284 | // Arrange 285 | using var commandWriter = new StringWriter(); 286 | 287 | var context = new TestLoggerContext( 288 | new GitHubWorkflow(commandWriter, TextWriter.Null), 289 | new TestLoggerOptions 290 | { 291 | AnnotationTitleFormat = "<@test (@framework)>", 292 | AnnotationMessageFormat = "[@test (@framework)]", 293 | } 294 | ); 295 | 296 | // Act 297 | context.SimulateTestRun( 298 | "FakeTests.dll", 299 | "FakeTargetFramework", 300 | new TestResultBuilder() 301 | .SetDisplayName("Test1") 302 | .SetOutcome(TestOutcome.Failed) 303 | .SetErrorStackTrace("ErrorStackTrace") 304 | .Build() 305 | ); 306 | 307 | // Assert 308 | var output = commandWriter.ToString().Trim(); 309 | 310 | output.Should().Contain(""); 311 | output.Should().Contain("[Test1 (FakeTargetFramework)]"); 312 | 313 | testOutput.WriteLine(output); 314 | } 315 | 316 | [Fact] 317 | public void I_can_use_the_logger_to_produce_annotations_that_include_line_breaks() 318 | { 319 | // Arrange 320 | using var commandWriter = new StringWriter(); 321 | 322 | var context = new TestLoggerContext( 323 | new GitHubWorkflow(commandWriter, TextWriter.Null), 324 | new TestLoggerOptions { AnnotationMessageFormat = "foo\\nbar" } 325 | ); 326 | 327 | // Act 328 | context.SimulateTestRun( 329 | new TestResultBuilder().SetDisplayName("Test1").SetOutcome(TestOutcome.Failed).Build() 330 | ); 331 | 332 | // Assert 333 | var output = commandWriter.ToString().Trim(); 334 | 335 | output.Should().Contain("foo%0Abar"); 336 | 337 | testOutput.WriteLine(output); 338 | } 339 | } 340 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger.Tests/Fakes/FakeTestLoggerEvents.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using Microsoft.VisualStudio.TestPlatform.ObjectModel.Client; 3 | using Microsoft.VisualStudio.TestPlatform.ObjectModel.Logging; 4 | 5 | namespace GitHubActionsTestLogger.Tests.Fakes; 6 | 7 | #pragma warning disable CS0067 8 | 9 | internal class FakeTestLoggerEvents : TestLoggerEvents 10 | { 11 | public override event EventHandler? TestRunMessage; 12 | public override event EventHandler? TestRunStart; 13 | public override event EventHandler? TestResult; 14 | public override event EventHandler? TestRunComplete; 15 | public override event EventHandler? DiscoveryStart; 16 | public override event EventHandler? DiscoveryMessage; 17 | public override event EventHandler? DiscoveredTests; 18 | public override event EventHandler? DiscoveryComplete; 19 | } 20 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger.Tests/GitHubActionsTestLogger.Tests.csproj: -------------------------------------------------------------------------------- 1 | 2 | 3 | net9.0 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger.Tests/InitializationSpecs.cs: -------------------------------------------------------------------------------- 1 | using System.Collections.Generic; 2 | using System.IO; 3 | using FluentAssertions; 4 | using GitHubActionsTestLogger.Tests.Fakes; 5 | using Xunit; 6 | 7 | namespace GitHubActionsTestLogger.Tests; 8 | 9 | public class InitializationSpecs 10 | { 11 | [Fact] 12 | public void I_can_use_the_logger_with_the_default_configuration() 13 | { 14 | // Arrange 15 | var logger = new TestLogger(); 16 | var events = new FakeTestLoggerEvents(); 17 | 18 | // Act 19 | logger.Initialize(events, Directory.GetCurrentDirectory()); 20 | 21 | // Assert 22 | logger.Context.Should().NotBeNull(); 23 | logger.Context?.Options.Should().BeEquivalentTo(TestLoggerOptions.Default); 24 | } 25 | 26 | [Fact] 27 | public void I_can_use_the_logger_with_an_empty_configuration() 28 | { 29 | // Arrange 30 | var logger = new TestLogger(); 31 | var events = new FakeTestLoggerEvents(); 32 | 33 | // Act 34 | logger.Initialize(events, new Dictionary()); 35 | 36 | // Assert 37 | logger.Context.Should().NotBeNull(); 38 | logger.Context?.Options.Should().BeEquivalentTo(TestLoggerOptions.Default); 39 | } 40 | 41 | [Fact] 42 | public void I_can_use_the_logger_with_a_custom_configuration() 43 | { 44 | // Arrange 45 | var logger = new TestLogger(); 46 | 47 | var events = new FakeTestLoggerEvents(); 48 | var parameters = new Dictionary 49 | { 50 | ["annotations.titleFormat"] = "TitleFormat", 51 | ["annotations.messageFormat"] = "MessageFormat", 52 | ["summary.includePassedTests"] = "true", 53 | ["summary.includeSkippedTests"] = "true", 54 | ["summary.includeNotFoundTests"] = "true", 55 | }; 56 | 57 | // Act 58 | logger.Initialize(events, parameters); 59 | 60 | // Assert 61 | logger.Context.Should().NotBeNull(); 62 | logger.Context?.Options.AnnotationTitleFormat.Should().Be("TitleFormat"); 63 | logger.Context?.Options.AnnotationMessageFormat.Should().Be("MessageFormat"); 64 | logger.Context?.Options.SummaryIncludePassedTests.Should().BeTrue(); 65 | logger.Context?.Options.SummaryIncludeSkippedTests.Should().BeTrue(); 66 | logger.Context?.Options.SummaryIncludeNotFoundTests.Should().BeTrue(); 67 | } 68 | } 69 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger.Tests/SummarySpecs.cs: -------------------------------------------------------------------------------- 1 | using System.IO; 2 | using FluentAssertions; 3 | using GitHubActionsTestLogger.Tests.Utils; 4 | using GitHubActionsTestLogger.Tests.Utils.Extensions; 5 | using Microsoft.VisualStudio.TestPlatform.ObjectModel; 6 | using Xunit; 7 | using Xunit.Abstractions; 8 | 9 | namespace GitHubActionsTestLogger.Tests; 10 | 11 | public class SummarySpecs(ITestOutputHelper testOutput) 12 | { 13 | [Fact] 14 | public void I_can_use_the_logger_to_produce_a_summary_that_includes_the_test_suite_name() 15 | { 16 | // Arrange 17 | using var summaryWriter = new StringWriter(); 18 | 19 | var context = new TestLoggerContext( 20 | new GitHubWorkflow(TextWriter.Null, summaryWriter), 21 | TestLoggerOptions.Default 22 | ); 23 | 24 | // Act 25 | context.SimulateTestRun("TestProject.dll"); 26 | 27 | // Assert 28 | var output = summaryWriter.ToString().Trim(); 29 | 30 | output.Should().Contain("TestProject"); 31 | 32 | testOutput.WriteLine(output); 33 | } 34 | 35 | [Fact] 36 | public void I_can_use_the_logger_to_produce_a_summary_that_includes_the_list_of_passed_tests() 37 | { 38 | // Arrange 39 | using var summaryWriter = new StringWriter(); 40 | 41 | var context = new TestLoggerContext( 42 | new GitHubWorkflow(TextWriter.Null, summaryWriter), 43 | new TestLoggerOptions { SummaryIncludePassedTests = true } 44 | ); 45 | 46 | // Act 47 | context.SimulateTestRun( 48 | new TestResultBuilder() 49 | .SetDisplayName("Test1") 50 | .SetFullyQualifiedName("TestProject.SomeTests.Test1") 51 | .SetOutcome(TestOutcome.Passed) 52 | .Build(), 53 | new TestResultBuilder() 54 | .SetDisplayName("Test2") 55 | .SetFullyQualifiedName("TestProject.SomeTests.Test2") 56 | .SetOutcome(TestOutcome.Passed) 57 | .Build(), 58 | new TestResultBuilder() 59 | .SetDisplayName("Test3") 60 | .SetFullyQualifiedName("TestProject.SomeTests.Test3") 61 | .SetOutcome(TestOutcome.Passed) 62 | .Build(), 63 | new TestResultBuilder() 64 | .SetDisplayName("Test4") 65 | .SetFullyQualifiedName("TestProject.SomeTests.Test4") 66 | .SetOutcome(TestOutcome.Failed) 67 | .SetErrorMessage("ErrorMessage4") 68 | .Build() 69 | ); 70 | 71 | // Assert 72 | var output = summaryWriter.ToString().Trim(); 73 | 74 | output.Should().Contain("Test1"); 75 | output.Should().Contain("Test2"); 76 | output.Should().Contain("Test3"); 77 | output.Should().Contain("Test4"); 78 | 79 | testOutput.WriteLine(output); 80 | } 81 | 82 | [Fact] 83 | public void I_can_use_the_logger_to_produce_a_summary_that_includes_the_list_of_failed_tests() 84 | { 85 | // Arrange 86 | using var summaryWriter = new StringWriter(); 87 | 88 | var context = new TestLoggerContext( 89 | new GitHubWorkflow(TextWriter.Null, summaryWriter), 90 | TestLoggerOptions.Default 91 | ); 92 | 93 | // Act 94 | context.SimulateTestRun( 95 | new TestResultBuilder() 96 | .SetDisplayName("Test1") 97 | .SetFullyQualifiedName("TestProject.SomeTests.Test1") 98 | .SetOutcome(TestOutcome.Failed) 99 | .SetErrorMessage("ErrorMessage1") 100 | .Build(), 101 | new TestResultBuilder() 102 | .SetDisplayName("Test2") 103 | .SetFullyQualifiedName("TestProject.SomeTests.Test2") 104 | .SetOutcome(TestOutcome.Failed) 105 | .SetErrorMessage("ErrorMessage2") 106 | .Build(), 107 | new TestResultBuilder() 108 | .SetDisplayName("Test3") 109 | .SetFullyQualifiedName("TestProject.SomeTests.Test3") 110 | .SetOutcome(TestOutcome.Failed) 111 | .SetErrorMessage("ErrorMessage3") 112 | .Build(), 113 | new TestResultBuilder() 114 | .SetDisplayName("Test4") 115 | .SetFullyQualifiedName("TestProject.SomeTests.Test4") 116 | .SetOutcome(TestOutcome.Passed) 117 | .Build(), 118 | new TestResultBuilder() 119 | .SetDisplayName("Test5") 120 | .SetFullyQualifiedName("TestProject.SomeTests.Test5") 121 | .SetOutcome(TestOutcome.Skipped) 122 | .Build() 123 | ); 124 | 125 | // Assert 126 | var output = summaryWriter.ToString().Trim(); 127 | 128 | output.Should().Contain("Test1"); 129 | output.Should().Contain("ErrorMessage1"); 130 | output.Should().Contain("Test2"); 131 | output.Should().Contain("ErrorMessage2"); 132 | output.Should().Contain("Test3"); 133 | output.Should().Contain("ErrorMessage3"); 134 | 135 | output.Should().NotContain("Test4"); 136 | output.Should().NotContain("Test5"); 137 | 138 | testOutput.WriteLine(output); 139 | } 140 | 141 | [Fact] 142 | public void I_can_use_the_logger_to_produce_a_summary_that_includes_the_list_of_skipped_tests() 143 | { 144 | // Arrange 145 | using var summaryWriter = new StringWriter(); 146 | 147 | var context = new TestLoggerContext( 148 | new GitHubWorkflow(TextWriter.Null, summaryWriter), 149 | new TestLoggerOptions { SummaryIncludeSkippedTests = true } 150 | ); 151 | 152 | // Act 153 | context.SimulateTestRun( 154 | new TestResultBuilder() 155 | .SetDisplayName("Test1") 156 | .SetFullyQualifiedName("TestProject.SomeTests.Test1") 157 | .SetOutcome(TestOutcome.Skipped) 158 | .Build(), 159 | new TestResultBuilder() 160 | .SetDisplayName("Test2") 161 | .SetFullyQualifiedName("TestProject.SomeTests.Test2") 162 | .SetOutcome(TestOutcome.Skipped) 163 | .Build(), 164 | new TestResultBuilder() 165 | .SetDisplayName("Test3") 166 | .SetFullyQualifiedName("TestProject.SomeTests.Test3") 167 | .SetOutcome(TestOutcome.Skipped) 168 | .Build(), 169 | new TestResultBuilder() 170 | .SetDisplayName("Test4") 171 | .SetFullyQualifiedName("TestProject.SomeTests.Test4") 172 | .SetOutcome(TestOutcome.Failed) 173 | .SetErrorMessage("ErrorMessage4") 174 | .Build() 175 | ); 176 | 177 | // Assert 178 | var output = summaryWriter.ToString().Trim(); 179 | 180 | output.Should().Contain("Test1"); 181 | output.Should().Contain("Test2"); 182 | output.Should().Contain("Test3"); 183 | output.Should().Contain("Test4"); 184 | 185 | testOutput.WriteLine(output); 186 | } 187 | 188 | [Fact] 189 | public void I_can_use_the_logger_to_produce_a_summary_that_includes_empty_test_suites() 190 | { 191 | // Arrange 192 | using var summaryWriter = new StringWriter(); 193 | 194 | var context = new TestLoggerContext( 195 | new GitHubWorkflow(TextWriter.Null, summaryWriter), 196 | TestLoggerOptions.Default 197 | ); 198 | 199 | // Act 200 | context.SimulateTestRun(); 201 | 202 | // Assert 203 | var output = summaryWriter.ToString().Trim(); 204 | output.Should().Contain("⚪️ FakeTests"); 205 | 206 | testOutput.WriteLine(output); 207 | } 208 | 209 | [Fact] 210 | public void I_can_use_the_logger_to_produce_a_summary_that_does_not_include_empty_test_suites() 211 | { 212 | // Arrange 213 | using var summaryWriter = new StringWriter(); 214 | 215 | var context = new TestLoggerContext( 216 | new GitHubWorkflow(TextWriter.Null, summaryWriter), 217 | new TestLoggerOptions { SummaryIncludeNotFoundTests = false } 218 | ); 219 | 220 | // Act 221 | context.SimulateTestRun(); 222 | 223 | // Assert 224 | var output = summaryWriter.ToString().Trim(); 225 | output.Should().BeNullOrEmpty(); 226 | 227 | testOutput.WriteLine(output); 228 | } 229 | } 230 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger.Tests/Utils/Extensions/TestLoggerContextExtensions.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Collections.Generic; 3 | using System.Collections.ObjectModel; 4 | using System.Linq; 5 | using Microsoft.VisualStudio.TestPlatform.ObjectModel; 6 | using Microsoft.VisualStudio.TestPlatform.ObjectModel.Client; 7 | using Microsoft.VisualStudio.TestPlatform.ObjectModel.Logging; 8 | 9 | namespace GitHubActionsTestLogger.Tests.Utils.Extensions; 10 | 11 | internal static class TestLoggerContextExtensions 12 | { 13 | public static void SimulateTestRun( 14 | this TestLoggerContext context, 15 | string testSuiteFilePath, 16 | string targetFrameworkName, 17 | params IReadOnlyList testResults 18 | ) 19 | { 20 | context.HandleTestRunStart( 21 | new TestRunStartEventArgs( 22 | new TestRunCriteria( 23 | [testSuiteFilePath], 24 | 1, 25 | true, 26 | // lang=xml 27 | $""" 28 | 29 | 30 | {targetFrameworkName} 31 | 32 | 33 | """ 34 | ) 35 | ) 36 | ); 37 | 38 | foreach (var testResult in testResults) 39 | context.HandleTestResult(new TestResultEventArgs(testResult)); 40 | 41 | context.HandleTestRunComplete( 42 | new TestRunCompleteEventArgs( 43 | new TestRunStatistics( 44 | new Dictionary 45 | { 46 | [TestOutcome.Passed] = testResults.Count(r => 47 | r.Outcome == TestOutcome.Passed 48 | ), 49 | [TestOutcome.Failed] = testResults.Count(r => 50 | r.Outcome == TestOutcome.Failed 51 | ), 52 | [TestOutcome.Skipped] = testResults.Count(r => 53 | r.Outcome == TestOutcome.Skipped 54 | ), 55 | [TestOutcome.None] = testResults.Count(r => r.Outcome == TestOutcome.None), 56 | } 57 | ), 58 | false, 59 | false, 60 | null, 61 | new Collection(), 62 | TimeSpan.FromSeconds(1.234) 63 | ) 64 | ); 65 | } 66 | 67 | public static void SimulateTestRun( 68 | this TestLoggerContext context, 69 | string testSuiteName, 70 | params IReadOnlyList testResults 71 | ) => context.SimulateTestRun(testSuiteName, "FakeTargetFramework", testResults); 72 | 73 | public static void SimulateTestRun( 74 | this TestLoggerContext context, 75 | params IReadOnlyList testResults 76 | ) => context.SimulateTestRun("FakeTests.dll", testResults); 77 | } 78 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger.Tests/Utils/TestResultBuilder.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using Microsoft.VisualStudio.TestPlatform.ObjectModel; 3 | 4 | namespace GitHubActionsTestLogger.Tests.Utils; 5 | 6 | internal class TestResultBuilder 7 | { 8 | private TestResult _testResult = new( 9 | new TestCase 10 | { 11 | Id = Guid.NewGuid(), 12 | Source = "FakeTests.dll", 13 | FullyQualifiedName = "FakeTests.FakeTest", 14 | DisplayName = "FakeTest", 15 | } 16 | ); 17 | 18 | public TestResultBuilder SetDisplayName(string displayName) 19 | { 20 | _testResult.TestCase.DisplayName = displayName; 21 | return this; 22 | } 23 | 24 | public TestResultBuilder SetFullyQualifiedName(string fullyQualifiedName) 25 | { 26 | _testResult.TestCase.FullyQualifiedName = fullyQualifiedName; 27 | return this; 28 | } 29 | 30 | public TestResultBuilder SetTrait(string name, string value) 31 | { 32 | _testResult.TestCase.Traits.Add(name, value); 33 | return this; 34 | } 35 | 36 | public TestResultBuilder SetOutcome(TestOutcome outcome) 37 | { 38 | _testResult.Outcome = outcome; 39 | return this; 40 | } 41 | 42 | public TestResultBuilder SetErrorMessage(string message) 43 | { 44 | _testResult.ErrorMessage = message; 45 | return this; 46 | } 47 | 48 | public TestResultBuilder SetErrorStackTrace(string stackTrace) 49 | { 50 | _testResult.ErrorStackTrace = stackTrace; 51 | return this; 52 | } 53 | 54 | public TestResult Build() 55 | { 56 | var testResult = _testResult; 57 | _testResult = new TestResult(new TestCase()); 58 | 59 | return testResult; 60 | } 61 | } 62 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger.Tests/xunit.runner.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://xunit.net/schema/current/xunit.runner.schema.json", 3 | "methodDisplayOptions": "all", 4 | "methodDisplay": "method" 5 | } -------------------------------------------------------------------------------- /GitHubActionsTestLogger.sln: -------------------------------------------------------------------------------- 1 |  2 | Microsoft Visual Studio Solution File, Format Version 12.00 3 | Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "GitHubActionsTestLogger", "GitHubActionsTestLogger\GitHubActionsTestLogger.csproj", "{FF25D237-FBD6-4DD9-8275-DDD752647647}" 4 | EndProject 5 | Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Misc", "Misc", "{71B00042-147C-45E0-AD34-9B6841E665E3}" 6 | ProjectSection(SolutionItems) = preProject 7 | License.txt = License.txt 8 | Readme.md = Readme.md 9 | Directory.Build.props = Directory.Build.props 10 | EndProjectSection 11 | EndProject 12 | Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "GitHubActionsTestLogger.Tests", "GitHubActionsTestLogger.Tests\GitHubActionsTestLogger.Tests.csproj", "{AB9190BD-F358-4765-9068-A1EEB668463B}" 13 | EndProject 14 | Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "GitHubActionsTestLogger.Demo", "GitHubActionsTestLogger.Demo\GitHubActionsTestLogger.Demo.csproj", "{2EAAD162-BE96-49E7-9CEC-029921010292}" 15 | EndProject 16 | Global 17 | GlobalSection(SolutionConfigurationPlatforms) = preSolution 18 | Debug|Any CPU = Debug|Any CPU 19 | Release|Any CPU = Release|Any CPU 20 | EndGlobalSection 21 | GlobalSection(ProjectConfigurationPlatforms) = postSolution 22 | {FF25D237-FBD6-4DD9-8275-DDD752647647}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 23 | {FF25D237-FBD6-4DD9-8275-DDD752647647}.Debug|Any CPU.Build.0 = Debug|Any CPU 24 | {FF25D237-FBD6-4DD9-8275-DDD752647647}.Release|Any CPU.ActiveCfg = Release|Any CPU 25 | {FF25D237-FBD6-4DD9-8275-DDD752647647}.Release|Any CPU.Build.0 = Release|Any CPU 26 | {AB9190BD-F358-4765-9068-A1EEB668463B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 27 | {AB9190BD-F358-4765-9068-A1EEB668463B}.Debug|Any CPU.Build.0 = Debug|Any CPU 28 | {AB9190BD-F358-4765-9068-A1EEB668463B}.Release|Any CPU.ActiveCfg = Release|Any CPU 29 | {AB9190BD-F358-4765-9068-A1EEB668463B}.Release|Any CPU.Build.0 = Release|Any CPU 30 | {2EAAD162-BE96-49E7-9CEC-029921010292}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 31 | {2EAAD162-BE96-49E7-9CEC-029921010292}.Debug|Any CPU.Build.0 = Debug|Any CPU 32 | {2EAAD162-BE96-49E7-9CEC-029921010292}.Release|Any CPU.ActiveCfg = Release|Any CPU 33 | {2EAAD162-BE96-49E7-9CEC-029921010292}.Release|Any CPU.Build.0 = Release|Any CPU 34 | EndGlobalSection 35 | EndGlobal 36 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/GitHubActionsTestLogger.csproj: -------------------------------------------------------------------------------- 1 | 2 | 3 | netstandard2.0;net9.0 4 | true 5 | true 9 | true 13 | 14 | 15 | $(Company) 16 | Custom test logger that reports test results in a structured format that GitHub Actions understands 17 | github actions test logger 18 | https://github.com/Tyrrrz/GitHubActionsTestLogger 19 | https://github.com/Tyrrrz/GitHubActionsTestLogger/releases 20 | favicon.png 21 | MIT 22 | true 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 38 | 39 | 40 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/GitHubWorkflow.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Collections.Generic; 3 | using System.IO; 4 | using System.Linq; 5 | using GitHubActionsTestLogger.Utils; 6 | using GitHubActionsTestLogger.Utils.Extensions; 7 | 8 | namespace GitHubActionsTestLogger; 9 | 10 | // https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions 11 | public partial class GitHubWorkflow(TextWriter commandWriter, TextWriter summaryWriter) 12 | { 13 | private void InvokeCommand( 14 | string command, 15 | string message, 16 | IReadOnlyDictionary? options = null 17 | ) 18 | { 19 | // URL-encode certain characters to ensure they don't get parsed as command tokens 20 | // https://pakstech.com/blog/github-actions-workflow-commands 21 | static string Escape(string value) => 22 | value 23 | .Replace("%", "%25", StringComparison.Ordinal) 24 | .Replace("\n", "%0A", StringComparison.Ordinal) 25 | .Replace("\r", "%0D", StringComparison.Ordinal); 26 | 27 | var formattedOptions = options 28 | ?.Select(kvp => Escape(kvp.Key) + '=' + Escape(kvp.Value)) 29 | .Pipe(s => string.Join(",", s)); 30 | 31 | // Command should start at the beginning of the line, so add a newline 32 | // to make sure there is no preceding text. 33 | // Preceding text may sometimes appear if the .NET CLI is running with 34 | // ANSI color codes enabled. 35 | commandWriter.WriteLine(); 36 | 37 | commandWriter.WriteLine($"::{command} {formattedOptions}::{Escape(message)}"); 38 | 39 | // This newline is just for symmetry 40 | commandWriter.WriteLine(); 41 | 42 | commandWriter.Flush(); 43 | } 44 | 45 | public void CreateErrorAnnotation( 46 | string title, 47 | string message, 48 | string? filePath = null, 49 | int? line = null, 50 | int? column = null 51 | ) 52 | { 53 | var options = new Dictionary { ["title"] = title }; 54 | 55 | if (!string.IsNullOrWhiteSpace(filePath)) 56 | options["file"] = filePath; 57 | 58 | if (line is not null) 59 | options["line"] = line.Value.ToString(); 60 | 61 | if (column is not null) 62 | options["col"] = column.Value.ToString(); 63 | 64 | InvokeCommand("error", message, options); 65 | } 66 | 67 | public void CreateSummary(string content) 68 | { 69 | // Other steps may have reported summaries that contain HTML tags, 70 | // which can screw up markdown parsing, so we need to make sure 71 | // there's at least two newlines before our summary to be safe. 72 | // https://github.com/Tyrrrz/GitHubActionsTestLogger/issues/22 73 | summaryWriter.WriteLine(); 74 | summaryWriter.WriteLine(); 75 | 76 | summaryWriter.WriteLine(content); 77 | summaryWriter.Flush(); 78 | } 79 | } 80 | 81 | public partial class GitHubWorkflow 82 | { 83 | public static GitHubWorkflow Default { get; } = 84 | new( 85 | // Commands are written to the standard output 86 | Console.Out, 87 | // Summary is written to the file specified by an environment variable. 88 | // We may need to write to the summary file from multiple test suites in parallel, 89 | // so we should use a stream that delays acquiring the file lock until the very last moment, 90 | // and employs retry logic to handle potential race conditions. 91 | Environment 92 | .GetEnvironmentVariable("GITHUB_STEP_SUMMARY") 93 | ?.Pipe(f => new ContentionTolerantWriteFileStream(f, FileMode.Append)) 94 | .Pipe(s => new StreamWriter(s)) ?? TextWriter.Null 95 | ); 96 | 97 | public static string? TryGenerateFilePermalink(string filePath, int? line = null) 98 | { 99 | var serverUrl = Environment.GetEnvironmentVariable("GITHUB_SERVER_URL"); 100 | var repositorySlug = Environment.GetEnvironmentVariable("GITHUB_REPOSITORY"); 101 | var workspacePath = Environment.GetEnvironmentVariable("GITHUB_WORKSPACE"); 102 | var commitHash = Environment.GetEnvironmentVariable("GITHUB_SHA"); 103 | 104 | if ( 105 | string.IsNullOrWhiteSpace(serverUrl) 106 | || string.IsNullOrWhiteSpace(repositorySlug) 107 | || string.IsNullOrWhiteSpace(workspacePath) 108 | || string.IsNullOrWhiteSpace(commitHash) 109 | ) 110 | { 111 | return null; 112 | } 113 | 114 | var filePathRelative = 115 | // If the file path starts with /_/ but the workspace path doesn't, 116 | // then it's safe to assume that the file path has already been normalized 117 | // by the Deterministic Build feature of MSBuild. 118 | // In this case, we only need to remove the leading /_/ from the file path 119 | // to get the correct relative path. 120 | filePath.StartsWith("/_/", StringComparison.Ordinal) 121 | && !workspacePath.StartsWith("/_/", StringComparison.Ordinal) 122 | ? filePath[3..] 123 | : PathEx.GetRelativePath(workspacePath, filePath); 124 | 125 | var filePathRoute = filePathRelative.Replace('\\', '/').Trim('/'); 126 | var lineMarker = line?.Pipe(l => $"#L{l}"); 127 | 128 | return $"{serverUrl}/{repositorySlug}/blob/{commitHash}/{filePathRoute}{lineMarker}"; 129 | } 130 | } 131 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/TestLogger.cs: -------------------------------------------------------------------------------- 1 | using System.Collections.Generic; 2 | using Microsoft.VisualStudio.TestPlatform.ObjectModel; 3 | using Microsoft.VisualStudio.TestPlatform.ObjectModel.Client; 4 | 5 | namespace GitHubActionsTestLogger; 6 | 7 | [FriendlyName("GitHubActions")] 8 | [ExtensionUri("logger://tyrrrz/ghactions/v2")] 9 | public class TestLogger : ITestLoggerWithParameters 10 | { 11 | public TestLoggerContext? Context { get; private set; } 12 | 13 | private void Initialize(TestLoggerEvents events, TestLoggerOptions options) 14 | { 15 | var context = new TestLoggerContext(GitHubWorkflow.Default, options); 16 | 17 | events.TestRunStart += (_, args) => context.HandleTestRunStart(args); 18 | events.TestResult += (_, args) => context.HandleTestResult(args); 19 | events.TestRunComplete += (_, args) => context.HandleTestRunComplete(args); 20 | 21 | Context = context; 22 | } 23 | 24 | public void Initialize(TestLoggerEvents events, string testRunDirectory) => 25 | Initialize(events, TestLoggerOptions.Default); 26 | 27 | public void Initialize(TestLoggerEvents events, Dictionary parameters) => 28 | Initialize(events, TestLoggerOptions.Resolve(parameters)); 29 | } 30 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/TestLoggerContext.cs: -------------------------------------------------------------------------------- 1 | using System.Collections.Generic; 2 | using System.IO; 3 | using System.Linq; 4 | using System.Text; 5 | using System.Threading; 6 | using GitHubActionsTestLogger.Utils.Extensions; 7 | using Microsoft.VisualStudio.TestPlatform.ObjectModel; 8 | using Microsoft.VisualStudio.TestPlatform.ObjectModel.Client; 9 | using Microsoft.VisualStudio.TestPlatform.ObjectModel.Logging; 10 | 11 | namespace GitHubActionsTestLogger; 12 | 13 | public class TestLoggerContext(GitHubWorkflow github, TestLoggerOptions options) 14 | { 15 | private readonly Lock _lock = new(); 16 | private TestRunCriteria? _testRunCriteria; 17 | private readonly List _testResults = []; 18 | 19 | public TestLoggerOptions Options { get; } = options; 20 | 21 | private string FormatAnnotation(string format, TestResult testResult) 22 | { 23 | var buffer = new StringBuilder(format); 24 | 25 | // Escaped new line token (backwards compat) 26 | buffer.Replace("\\n", "\n"); 27 | 28 | // Name token 29 | buffer 30 | .Replace("@test", testResult.TestCase.DisplayName) 31 | // Backwards compat 32 | .Replace("$test", testResult.TestCase.DisplayName); 33 | 34 | // Trait tokens 35 | foreach (var trait in testResult.Traits.Union(testResult.TestCase.Traits)) 36 | { 37 | buffer 38 | .Replace($"@traits.{trait.Name}", trait.Value) 39 | // Backwards compat 40 | .Replace($"$traits.{trait.Name}", trait.Value); 41 | } 42 | 43 | // Error message 44 | buffer 45 | .Replace("@error", testResult.ErrorMessage ?? "") 46 | // Backwards compat 47 | .Replace("$error", testResult.ErrorMessage ?? ""); 48 | 49 | // Error trace 50 | buffer 51 | .Replace("@trace", testResult.ErrorStackTrace ?? "") 52 | // Backwards compat 53 | .Replace("$trace", testResult.ErrorStackTrace ?? ""); 54 | 55 | // Target framework 56 | buffer 57 | .Replace("@framework", _testRunCriteria?.TryGetTargetFramework() ?? "") 58 | // Backwards compat 59 | .Replace("$framework", _testRunCriteria?.TryGetTargetFramework() ?? ""); 60 | 61 | return buffer.Trim().ToString(); 62 | } 63 | 64 | private string FormatAnnotationTitle(TestResult testResult) => 65 | FormatAnnotation(Options.AnnotationTitleFormat, testResult); 66 | 67 | private string FormatAnnotationMessage(TestResult testResult) => 68 | FormatAnnotation(Options.AnnotationMessageFormat, testResult); 69 | 70 | public void HandleTestRunStart(TestRunStartEventArgs args) 71 | { 72 | using (_lock.EnterScope()) 73 | { 74 | _testRunCriteria = args.TestRunCriteria; 75 | } 76 | } 77 | 78 | public void HandleTestResult(TestResultEventArgs args) 79 | { 80 | using (_lock.EnterScope()) 81 | { 82 | // Report failed test results to job annotations 83 | if (args.Result.Outcome == TestOutcome.Failed) 84 | { 85 | github.CreateErrorAnnotation( 86 | FormatAnnotationTitle(args.Result), 87 | FormatAnnotationMessage(args.Result), 88 | args.Result.TryGetSourceFilePath(), 89 | args.Result.TryGetSourceLine() 90 | ); 91 | } 92 | 93 | // Record all test results to write them to the summary later 94 | _testResults.Add(args.Result); 95 | } 96 | } 97 | 98 | public void HandleTestRunComplete(TestRunCompleteEventArgs args) 99 | { 100 | using (_lock.EnterScope()) 101 | { 102 | var testSuite = 103 | _testRunCriteria?.Sources?.FirstOrDefault()?.Pipe(Path.GetFileNameWithoutExtension) 104 | ?? "Unknown Test Suite"; 105 | 106 | var targetFramework = 107 | _testRunCriteria?.TryGetTargetFramework() ?? "Unknown Target Framework"; 108 | 109 | var testRunStatistics = new TestRunStatistics( 110 | (int?)args.TestRunStatistics?[TestOutcome.Passed] 111 | ?? _testResults.Count(r => r.Outcome == TestOutcome.Passed), 112 | (int?)args.TestRunStatistics?[TestOutcome.Failed] 113 | ?? _testResults.Count(r => r.Outcome == TestOutcome.Failed), 114 | (int?)args.TestRunStatistics?[TestOutcome.Skipped] 115 | ?? _testResults.Count(r => r.Outcome == TestOutcome.Skipped), 116 | (int?)args.TestRunStatistics?.ExecutedTests ?? _testResults.Count, 117 | args.ElapsedTimeInRunningTests 118 | ); 119 | 120 | var testResults = _testResults 121 | .Where(r => 122 | r.Outcome == TestOutcome.Failed 123 | || r.Outcome == TestOutcome.Passed && Options.SummaryIncludePassedTests 124 | || r.Outcome == TestOutcome.Skipped && Options.SummaryIncludeSkippedTests 125 | ) 126 | .ToArray(); 127 | 128 | var template = new TestSummaryTemplate 129 | { 130 | TestSuite = testSuite, 131 | TargetFramework = targetFramework, 132 | TestRunStatistics = testRunStatistics, 133 | TestResults = testResults, 134 | }; 135 | 136 | if ( 137 | !Options.SummaryIncludeNotFoundTests 138 | && testRunStatistics.OverallOutcome == TestOutcome.NotFound 139 | ) 140 | { 141 | return; 142 | } 143 | 144 | github.CreateSummary(template.Render()); 145 | } 146 | } 147 | } 148 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/TestLoggerOptions.cs: -------------------------------------------------------------------------------- 1 | using System.Collections.Generic; 2 | using GitHubActionsTestLogger.Utils.Extensions; 3 | 4 | namespace GitHubActionsTestLogger; 5 | 6 | public partial class TestLoggerOptions 7 | { 8 | public string AnnotationTitleFormat { get; init; } = "@test"; 9 | 10 | public string AnnotationMessageFormat { get; init; } = "@error"; 11 | 12 | public bool SummaryIncludePassedTests { get; init; } 13 | 14 | public bool SummaryIncludeSkippedTests { get; init; } 15 | 16 | public bool SummaryIncludeNotFoundTests { get; init; } = true; 17 | } 18 | 19 | public partial class TestLoggerOptions 20 | { 21 | public static TestLoggerOptions Default { get; } = new(); 22 | 23 | public static TestLoggerOptions Resolve(IReadOnlyDictionary parameters) => 24 | new() 25 | { 26 | AnnotationTitleFormat = 27 | parameters.GetValueOrDefault("annotations.titleFormat") 28 | ?? Default.AnnotationTitleFormat, 29 | AnnotationMessageFormat = 30 | parameters.GetValueOrDefault("annotations.messageFormat") 31 | ?? Default.AnnotationMessageFormat, 32 | SummaryIncludePassedTests = 33 | parameters.GetValueOrDefault("summary.includePassedTests")?.Pipe(bool.Parse) 34 | ?? Default.SummaryIncludePassedTests, 35 | SummaryIncludeSkippedTests = 36 | parameters.GetValueOrDefault("summary.includeSkippedTests")?.Pipe(bool.Parse) 37 | ?? Default.SummaryIncludeSkippedTests, 38 | SummaryIncludeNotFoundTests = 39 | parameters.GetValueOrDefault("summary.includeNotFoundTests")?.Pipe(bool.Parse) 40 | ?? Default.SummaryIncludeNotFoundTests, 41 | }; 42 | } 43 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/TestRunStatistics.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using Microsoft.VisualStudio.TestPlatform.ObjectModel; 3 | 4 | namespace GitHubActionsTestLogger; 5 | 6 | internal record TestRunStatistics( 7 | int PassedTestCount, 8 | int FailedTestCount, 9 | int SkippedTestCount, 10 | int TotalTestCount, 11 | TimeSpan OverallDuration 12 | ) 13 | { 14 | public TestOutcome OverallOutcome { get; } = 15 | true switch 16 | { 17 | _ when FailedTestCount > 0 => TestOutcome.Failed, 18 | _ when PassedTestCount > 0 => TestOutcome.Passed, 19 | _ when SkippedTestCount > 0 => TestOutcome.Skipped, 20 | _ when TotalTestCount == 0 => TestOutcome.NotFound, 21 | _ => TestOutcome.None, 22 | }; 23 | } 24 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/TestSummaryTemplate.cshtml: -------------------------------------------------------------------------------- 1 | @using System 2 | @using System.Collections.Generic 3 | @using System.Linq 4 | @using GitHubActionsTestLogger.Utils.Extensions 5 | @using Microsoft.VisualStudio.TestPlatform.ObjectModel 6 | 7 | @inherits RazorBlade.HtmlTemplate 8 | 9 | @functions 10 | { 11 | public required string TestSuite { get; init; } 12 | 13 | public required string TargetFramework { get; init; } 14 | 15 | public required TestRunStatistics TestRunStatistics { get; init; } 16 | 17 | public required IReadOnlyList TestResults { get; init; } 18 | } 19 | 20 |
21 | @{ 22 | var overallOutcomeEmoji = TestRunStatistics.OverallOutcome switch 23 | { 24 | TestOutcome.Passed => "🟢", 25 | TestOutcome.Failed => "🔴", 26 | TestOutcome.Skipped => "🟡", 27 | TestOutcome.NotFound => "⚪️", 28 | _ => "\u2753" 29 | }; 30 | } 31 | 32 | 33 | @overallOutcomeEmoji @TestSuite (@TargetFramework) 34 | 35 | 36 | @* This adds a margin that is smaller than
*@ 37 |

38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 51 | 56 | 61 | 64 | 67 | 68 |
✓  Passed✘  Failed↷  Skipped∑  Total⧗  Elapsed
47 | @(TestRunStatistics.PassedTestCount > 0 48 | ? TestRunStatistics.PassedTestCount.ToString() 49 | : "—") 50 | 52 | @(TestRunStatistics.FailedTestCount > 0 53 | ? TestRunStatistics.FailedTestCount.ToString() 54 | : "—") 55 | 57 | @(TestRunStatistics.SkippedTestCount > 0 58 | ? TestRunStatistics.SkippedTestCount.ToString() 59 | : "—") 60 | 62 | @TestRunStatistics.TotalTestCount 63 | 65 | @TestRunStatistics.OverallDuration.ToHumanString() 66 |
69 | 70 | @{ 71 | var testResultGroups = TestResults 72 | .GroupBy(r => r.TestCase.GetTypeFullyQualifiedName(), StringComparer.Ordinal) 73 | .Select(g => new 74 | { 75 | TypeFullyQualifiedName = g.Key, 76 | TypeName = g.First().TestCase.GetTypeMinimallyQualifiedName(), 77 | TestResults = g 78 | .OrderByDescending(r => r.Outcome == TestOutcome.Failed) 79 | .ThenByDescending(r => r.Outcome == TestOutcome.Passed) 80 | .ThenBy(r => r.TestCase.DisplayName, StringComparer.Ordinal) 81 | .ToArray() 82 | }) 83 | .OrderByDescending(g => g.TestResults.Any(r => r.Outcome == TestOutcome.Failed)) 84 | .ThenByDescending(g => g.TestResults.Any(r => r.Outcome == TestOutcome.Passed)) 85 | .ThenBy(g => g.TypeName, StringComparer.Ordinal); 86 | } 87 | 88 |
    89 | @foreach (var testResultGroup in testResultGroups) 90 | { 91 | var failedTestCount = testResultGroup.TestResults.Count(r => r.Outcome == TestOutcome.Failed); 92 | 93 |
  • 94 | @testResultGroup.TypeName 95 | 96 | @if (failedTestCount > 0) 97 | { 98 | @(" ")(@failedTestCount failed) 99 | } 100 | 101 | @* This adds a margin that is smaller than
    *@ 102 |

    103 | 104 |
      105 | @foreach (var testResult in testResultGroup.TestResults) 106 | { 107 | var outcomeEmoji = testResult.Outcome switch 108 | { 109 | TestOutcome.Passed => "🟩", 110 | TestOutcome.Failed => "🟥", 111 | TestOutcome.Skipped => "🟨", 112 | TestOutcome.NotFound => "⬜", 113 | _ => "\u2753" 114 | }; 115 | 116 | // Use the display name if it's different from the fully qualified name, 117 | // otherwise use the minimally qualified name. 118 | var testName = !string.Equals( 119 | testResult.TestCase.DisplayName, 120 | testResult.TestCase.FullyQualifiedName, 121 | StringComparison.Ordinal) 122 | ? testResult.TestCase.DisplayName 123 | : testResult.TestCase.GetMinimallyQualifiedName(); 124 | 125 | // Test source permalink 126 | var filePath = testResult.TryGetSourceFilePath(); 127 | var fileLine = testResult.TryGetSourceLine(); 128 | var url = filePath?.Pipe(p => GitHubWorkflow.TryGenerateFilePermalink(p, fileLine)); 129 | 130 |
    • 131 | @outcomeEmoji 132 | 133 | @if (!string.IsNullOrWhiteSpace(url)) 134 | { 135 | @(" ")@testName 136 | } 137 | else 138 | { 139 | @(" ")@testName 140 | } 141 | 142 | @if (!string.IsNullOrWhiteSpace(testResult.ErrorMessage)) 143 | { 144 | WriteMarkdown( 145 | "```yml", 146 | testResult.ErrorMessage, 147 | testResult.ErrorStackTrace, 148 | "```" 149 | ); 150 | } 151 |
      • 152 | } 153 |
    154 | 155 | @* This adds a margin that is smaller than
    *@ 156 |

    157 |
    • 158 | } 159 |
160 |
161 | 162 | @functions 163 | { 164 | // In order to produce HTML that's also valid Markdown, we need to 165 | // remove some whitespace inside literals. 166 | public new void WriteLiteral(string? literal) 167 | { 168 | if (!string.IsNullOrEmpty(literal)) 169 | { 170 | base.WriteLiteral( 171 | literal 172 | // Remove indentation 173 | .Replace(" ", "", StringComparison.Ordinal) 174 | // Remove linebreaks 175 | .Replace("\r", "", StringComparison.Ordinal) 176 | .Replace("\n", "", StringComparison.Ordinal) 177 | ); 178 | } 179 | else 180 | { 181 | base.WriteLiteral(literal); 182 | } 183 | } 184 | 185 | // Using params here to write multiple lines as a workaround for the 186 | // fact that Razor does not support multiline raw string literals. 187 | private void WriteMarkdown(params IEnumerable lines) 188 | { 189 | // Two line breaks are required to separate markdown from HTML 190 | base.WriteLiteral("\n\n"); 191 | 192 | foreach (var line in lines) 193 | { 194 | base.WriteLiteral(line); 195 | base.WriteLiteral("\n"); 196 | } 197 | } 198 | } -------------------------------------------------------------------------------- /GitHubActionsTestLogger/Utils/ContentionTolerantWriteFileStream.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Collections.Generic; 3 | using System.Diagnostics.CodeAnalysis; 4 | using System.IO; 5 | using System.Linq; 6 | using System.Threading; 7 | 8 | namespace GitHubActionsTestLogger.Utils; 9 | 10 | internal class ContentionTolerantWriteFileStream(string filePath, FileMode fileMode) : Stream 11 | { 12 | private readonly List _buffer = new(1024); 13 | 14 | [ExcludeFromCodeCoverage] 15 | public override bool CanRead => false; 16 | 17 | [ExcludeFromCodeCoverage] 18 | public override bool CanSeek => false; 19 | 20 | [ExcludeFromCodeCoverage] 21 | public override bool CanWrite => true; 22 | 23 | [ExcludeFromCodeCoverage] 24 | public override long Length => _buffer.Count; 25 | 26 | [ExcludeFromCodeCoverage] 27 | public override long Position { get; set; } 28 | 29 | // Backoff and retry if the file is locked 30 | private FileStream CreateInnerStream() 31 | { 32 | for (var retriesRemaining = 10; ; retriesRemaining--) 33 | { 34 | try 35 | { 36 | return new FileStream(filePath, fileMode); 37 | } 38 | catch (IOException) when (retriesRemaining > 0) 39 | { 40 | // Variance in delay to avoid overlapping back-offs 41 | Thread.Sleep(RandomEx.Shared.Next(200, 1000)); 42 | } 43 | } 44 | } 45 | 46 | public override void Write(byte[] buffer, int offset, int count) => 47 | _buffer.AddRange(buffer.Skip(offset).Take(count)); 48 | 49 | public override void Flush() 50 | { 51 | using var stream = CreateInnerStream(); 52 | stream.Write(_buffer.ToArray(), 0, _buffer.Count); 53 | } 54 | 55 | [ExcludeFromCodeCoverage] 56 | protected override void Dispose(bool disposing) 57 | { 58 | base.Dispose(disposing); 59 | _buffer.Clear(); 60 | } 61 | 62 | [ExcludeFromCodeCoverage] 63 | public override int Read(byte[] buffer, int offset, int count) => 64 | throw new NotSupportedException(); 65 | 66 | [ExcludeFromCodeCoverage] 67 | public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException(); 68 | 69 | [ExcludeFromCodeCoverage] 70 | public override void SetLength(long value) => throw new NotSupportedException(); 71 | } 72 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/Utils/Extensions/GenericExtensions.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | 3 | namespace GitHubActionsTestLogger.Utils.Extensions; 4 | 5 | internal static class GenericExtensions 6 | { 7 | public static TOut Pipe(this TIn input, Func transform) => 8 | transform(input); 9 | } 10 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/Utils/Extensions/StringExtensions.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Globalization; 3 | using System.Text; 4 | 5 | namespace GitHubActionsTestLogger.Utils.Extensions; 6 | 7 | internal static class StringExtensions 8 | { 9 | public static string SubstringUntil( 10 | this string str, 11 | string sub, 12 | StringComparison comparison = StringComparison.Ordinal 13 | ) 14 | { 15 | var index = str.IndexOf(sub, comparison); 16 | return index < 0 ? str : str[..index]; 17 | } 18 | 19 | public static string SubstringUntilLast( 20 | this string str, 21 | string sub, 22 | StringComparison comparison = StringComparison.Ordinal 23 | ) 24 | { 25 | var index = str.LastIndexOf(sub, comparison); 26 | return index < 0 ? str : str[..index]; 27 | } 28 | 29 | public static string SubstringAfter( 30 | this string str, 31 | string sub, 32 | StringComparison comparison = StringComparison.Ordinal 33 | ) 34 | { 35 | var index = str.IndexOf(sub, comparison); 36 | return index >= 0 ? str.Substring(index + sub.Length, str.Length - index - sub.Length) : ""; 37 | } 38 | 39 | public static string SubstringAfterLast( 40 | this string str, 41 | string sub, 42 | StringComparison comparison = StringComparison.Ordinal 43 | ) 44 | { 45 | var index = str.LastIndexOf(sub, comparison); 46 | return index >= 0 ? str.Substring(index + sub.Length, str.Length - index - sub.Length) : ""; 47 | } 48 | 49 | public static int? TryParseInt(this string? str) => 50 | int.TryParse(str, NumberStyles.Integer, CultureInfo.InvariantCulture, out var result) 51 | ? result 52 | : null; 53 | 54 | public static StringBuilder Trim(this StringBuilder builder) 55 | { 56 | while (builder.Length > 0 && char.IsWhiteSpace(builder[0])) 57 | builder.Remove(0, 1); 58 | 59 | while (builder.Length > 0 && char.IsWhiteSpace(builder[^1])) 60 | builder.Remove(builder.Length - 1, 1); 61 | 62 | return builder; 63 | } 64 | } 65 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/Utils/Extensions/TestCaseExtensions.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.IO; 3 | using Microsoft.VisualStudio.TestPlatform.ObjectModel; 4 | 5 | namespace GitHubActionsTestLogger.Utils.Extensions; 6 | 7 | internal static class TestCaseExtensions 8 | { 9 | public static string GetTypeFullyQualifiedName(this TestCase testCase) => 10 | testCase 11 | .FullyQualifiedName 12 | // Strip the test cases (if this is a parameterized test method) 13 | .SubstringUntil("(", StringComparison.OrdinalIgnoreCase) 14 | // Strip everything after the last dot, to leave the full type name 15 | .SubstringUntilLast(".", StringComparison.OrdinalIgnoreCase); 16 | 17 | public static string GetTypeMinimallyQualifiedName(this TestCase testCase) 18 | { 19 | var fullyQualifiedName = testCase.GetTypeFullyQualifiedName(); 20 | 21 | // We assume that the test assembly name matches the namespace. 22 | // This is not always true, but it's the best we can do. 23 | var nameSpace = Path.GetFileNameWithoutExtension(testCase.Source); 24 | 25 | // Strip the namespace from the type name, if it's there 26 | if (fullyQualifiedName.StartsWith(nameSpace + '.', StringComparison.Ordinal)) 27 | return fullyQualifiedName[(nameSpace.Length + 1)..]; 28 | 29 | return fullyQualifiedName; 30 | } 31 | 32 | public static string GetMinimallyQualifiedName(this TestCase testCase) 33 | { 34 | var fullyQualifiedName = testCase.GetTypeFullyQualifiedName(); 35 | 36 | // Strip the full type name from the test method name, if it's there 37 | return testCase.FullyQualifiedName.StartsWith(fullyQualifiedName, StringComparison.Ordinal) 38 | ? testCase.FullyQualifiedName[(fullyQualifiedName.Length + 1)..] 39 | : testCase.FullyQualifiedName; 40 | } 41 | } 42 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/Utils/Extensions/TestResultExtensions.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Linq; 3 | using Microsoft.VisualStudio.TestPlatform.ObjectModel; 4 | 5 | namespace GitHubActionsTestLogger.Utils.Extensions; 6 | 7 | internal static class TestResultExtensions 8 | { 9 | // This method attempts to get the stack frame that represents the call to the test method. 10 | // Obviously, this only works if the test throws an exception. 11 | private static StackFrame? TryGetTestStackFrame(this TestResult testResult) 12 | { 13 | if (string.IsNullOrWhiteSpace(testResult.ErrorStackTrace)) 14 | return null; 15 | 16 | if (string.IsNullOrWhiteSpace(testResult.TestCase.FullyQualifiedName)) 17 | return null; 18 | 19 | var testMethodFullyQualifiedName = testResult.TestCase.FullyQualifiedName.SubstringUntil( 20 | "(", 21 | StringComparison.OrdinalIgnoreCase 22 | ); 23 | 24 | var testMethodName = testMethodFullyQualifiedName.SubstringAfterLast( 25 | ".", 26 | StringComparison.OrdinalIgnoreCase 27 | ); 28 | 29 | return StackFrame 30 | .ParseMany(testResult.ErrorStackTrace) 31 | .LastOrDefault(f => 32 | // Sync method call 33 | // e.g. MyTests.EnsureOnePlusOneEqualsTwo() 34 | f.MethodCall.StartsWith( 35 | testMethodFullyQualifiedName, 36 | StringComparison.OrdinalIgnoreCase 37 | ) 38 | || 39 | // Async method call 40 | // e.g. MyTests.d__3.MoveNext() 41 | f.MethodCall.Contains( 42 | '<' + testMethodName + '>', 43 | StringComparison.OrdinalIgnoreCase 44 | ) 45 | ); 46 | } 47 | 48 | public static string? TryGetSourceFilePath(this TestResult testResult) 49 | { 50 | // See if it was provided directly (requires source information collection to be enabled) 51 | if (!string.IsNullOrWhiteSpace(testResult.TestCase.CodeFilePath)) 52 | return testResult.TestCase.CodeFilePath; 53 | 54 | // Try to extract it from the stack trace (works only if there was an exception) 55 | var stackFrame = testResult.TryGetTestStackFrame(); 56 | if (!string.IsNullOrWhiteSpace(stackFrame?.FilePath)) 57 | return stackFrame.FilePath; 58 | 59 | return null; 60 | } 61 | 62 | public static int? TryGetSourceLine(this TestResult testResult) 63 | { 64 | // See if it was provided directly (requires source information collection to be enabled) 65 | if (testResult.TestCase.LineNumber > 0) 66 | return testResult.TestCase.LineNumber; 67 | 68 | // Try to extract it from the stack trace (works only if there was an exception) 69 | return testResult.TryGetTestStackFrame()?.Line; 70 | } 71 | } 72 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/Utils/Extensions/TestRunCriteriaExtensions.cs: -------------------------------------------------------------------------------- 1 | using System.Xml.Linq; 2 | using Microsoft.VisualStudio.TestPlatform.ObjectModel.Client; 3 | 4 | namespace GitHubActionsTestLogger.Utils.Extensions; 5 | 6 | internal static class TestRunCriteriaExtensions 7 | { 8 | public static string? TryGetTargetFramework(this TestRunCriteria testRunCriteria) 9 | { 10 | if (string.IsNullOrWhiteSpace(testRunCriteria.TestRunSettings)) 11 | return null; 12 | 13 | return (string?) 14 | XElement 15 | .Parse(testRunCriteria.TestRunSettings) 16 | .Element("RunConfiguration") 17 | ?.Element("TargetFrameworkVersion"); 18 | } 19 | } 20 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/Utils/Extensions/TimeSpanExtensions.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | 3 | namespace GitHubActionsTestLogger.Utils.Extensions; 4 | 5 | internal static class TimeSpanExtensions 6 | { 7 | public static string ToHumanString(this TimeSpan timeSpan) => 8 | timeSpan switch 9 | { 10 | { TotalSeconds: <= 1 } => timeSpan.Milliseconds + "ms", 11 | { TotalMinutes: <= 1 } => timeSpan.Seconds + "s", 12 | { TotalHours: <= 1 } => timeSpan.Minutes + "m" + timeSpan.Seconds + "s", 13 | _ => timeSpan.Hours + "h" + timeSpan.Minutes + "m", 14 | }; 15 | } 16 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/Utils/PathEx.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.IO; 3 | using System.Linq; 4 | using System.Runtime.InteropServices; 5 | 6 | namespace GitHubActionsTestLogger.Utils; 7 | 8 | internal static class PathEx 9 | { 10 | private static readonly StringComparison PathStringComparison = RuntimeInformation.IsOSPlatform( 11 | OSPlatform.Windows 12 | ) 13 | ? StringComparison.OrdinalIgnoreCase 14 | : StringComparison.Ordinal; 15 | 16 | // This method exists on .NET 5+ but it's impossible to polyfill static 17 | // members, so we'll just use this one on all targets. 18 | public static string GetRelativePath(string basePath, string path) 19 | { 20 | var basePathSegments = basePath.Split( 21 | Path.DirectorySeparatorChar, 22 | Path.AltDirectorySeparatorChar 23 | ); 24 | var pathSegments = path.Split(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar); 25 | 26 | var commonSegmentsCount = 0; 27 | for (var i = 0; i < basePathSegments.Length && i < pathSegments.Length; i++) 28 | { 29 | if (!string.Equals(basePathSegments[i], pathSegments[i], PathStringComparison)) 30 | break; 31 | 32 | commonSegmentsCount++; 33 | } 34 | 35 | return string.Join( 36 | Path.DirectorySeparatorChar.ToString(), 37 | pathSegments.Skip(commonSegmentsCount) 38 | ); 39 | } 40 | } 41 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/Utils/RandomEx.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | 3 | namespace GitHubActionsTestLogger.Utils; 4 | 5 | internal static class RandomEx 6 | { 7 | public static Random Shared { get; } = new(); 8 | } 9 | -------------------------------------------------------------------------------- /GitHubActionsTestLogger/Utils/StackFrame.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Collections.Generic; 3 | using System.Linq; 4 | using System.Text.RegularExpressions; 5 | using GitHubActionsTestLogger.Utils.Extensions; 6 | 7 | namespace GitHubActionsTestLogger.Utils; 8 | 9 | // Adapted from https://github.com/atifaziz/StackTraceParser 10 | internal partial class StackFrame(string methodCall, string? filePath, int? line) 11 | { 12 | public string MethodCall { get; } = methodCall; 13 | 14 | public string? FilePath { get; } = filePath; 15 | 16 | public int? Line { get; } = line; 17 | } 18 | 19 | internal partial class StackFrame 20 | { 21 | private const string Space = @"[\x20\t]"; 22 | private const string NotSpace = @"[^\x20\t]"; 23 | 24 | private static readonly Regex Pattern = new( 25 | $$""" 26 | ^ 27 | {{Space}}* 28 | \w+ {{Space}}+ 29 | (? 30 | (? {{NotSpace}}+ ) \. 31 | (? {{NotSpace}}+? ) {{Space}}* 32 | (? \( ( {{Space}}* \) 33 | | (? .+?) {{Space}}+ (? .+?) 34 | (, {{Space}}* (? .+?) {{Space}}+ (? .+?) )* \) ) ) 35 | ( {{Space}}+ 36 | ( # Microsoft .NET stack traces 37 | \w+ {{Space}}+ 38 | (? ( [a-z] \: # Windows rooted path starting with a drive letter 39 | | / ) # Unix rooted path starting with a forward-slash 40 | .+? ) 41 | \: \w+ {{Space}}+ 42 | (? [0-9]+ ) \p{P}? 43 | | # Mono stack traces 44 | \[0x[0-9a-f]+\] {{Space}}+ \w+ {{Space}}+ 45 | <(? [^>]+ )> 46 | :(? [0-9]+ ) 47 | ) 48 | )? 49 | ) 50 | \s* 51 | $ 52 | """, 53 | RegexOptions.IgnoreCase 54 | | RegexOptions.Multiline 55 | | RegexOptions.ExplicitCapture 56 | | RegexOptions.CultureInvariant 57 | | RegexOptions.IgnorePatternWhitespace 58 | | RegexOptions.Compiled, 59 | // Cap the evaluation time to make it obvious should the expression 60 | // fall into the "catastrophic backtracking" trap due to over 61 | // generalization. 62 | // https://github.com/atifaziz/StackTraceParser/issues/4 63 | TimeSpan.FromSeconds(5) 64 | ); 65 | 66 | public static IEnumerable ParseMany(string text) => 67 | from Match m in Pattern.Matches(text) 68 | select m.Groups into groups 69 | select new StackFrame( 70 | groups["type"].Value + '.' + groups["method"].Value, 71 | groups["file"].Value, 72 | groups["line"].Value.TryParseInt() 73 | ); 74 | } 75 | -------------------------------------------------------------------------------- /License.txt: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020-2024 Oleksii Holub 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /NuGet.config: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | -------------------------------------------------------------------------------- /Readme.md: -------------------------------------------------------------------------------- 1 | # GitHub Actions Test Logger 2 | 3 | [![Status](https://img.shields.io/badge/status-maintenance-ffd700.svg)](https://github.com/Tyrrrz/.github/blob/master/docs/project-status.md) 4 | [![Made in Ukraine](https://img.shields.io/badge/made_in-ukraine-ffd700.svg?labelColor=0057b7)](https://tyrrrz.me/ukraine) 5 | [![Build](https://img.shields.io/github/actions/workflow/status/Tyrrrz/GitHubActionsTestLogger/main.yml?branch=master)](https://github.com/Tyrrrz/GitHubActionsTestLogger/actions) 6 | [![Coverage](https://img.shields.io/codecov/c/github/Tyrrrz/GitHubActionsTestLogger/master)](https://codecov.io/gh/Tyrrrz/GitHubActionsTestLogger) 7 | [![Version](https://img.shields.io/nuget/v/GitHubActionsTestLogger.svg)](https://nuget.org/packages/GitHubActionsTestLogger) 8 | [![Downloads](https://img.shields.io/nuget/dt/GitHubActionsTestLogger.svg)](https://nuget.org/packages/GitHubActionsTestLogger) 9 | [![Discord](https://img.shields.io/discord/869237470565392384?label=discord)](https://discord.gg/2SUWKFnHSm) 10 | [![Fuck Russia](https://img.shields.io/badge/fuck-russia-e4181c.svg?labelColor=000000)](https://twitter.com/tyrrrz/status/1495972128977571848) 11 | 12 | 13 | 14 | 15 | 16 |
Development of this project is entirely funded by the community. Consider donating to support!
17 | 18 |

19 | Icon 20 |

21 | 22 | **GitHub Actions Test Logger** is a custom logger for `dotnet test` that integrates with GitHub Actions. 23 | When using this logger, failed tests are listed in job annotations and highlighted in code diffs. 24 | Additionally, this logger also generates a job summary that contains detailed information about the executed test run. 25 | 26 | ## Terms of use[[?]](https://github.com/Tyrrrz/.github/blob/master/docs/why-so-political.md) 27 | 28 | By using this project or its source code, for any purpose and in any shape or form, you grant your **implicit agreement** to all the following statements: 29 | 30 | - You **condemn Russia and its military aggression against Ukraine** 31 | - You **recognize that Russia is an occupant that unlawfully invaded a sovereign state** 32 | - You **support Ukraine's territorial integrity, including its claims over temporarily occupied territories of Crimea and Donbas** 33 | - You **reject false narratives perpetuated by Russian state propaganda** 34 | 35 | To learn more about the war and how you can help, [click here](https://tyrrrz.me/ukraine). Glory to Ukraine! 🇺🇦 36 | 37 | ## Install 38 | 39 | - 📦 [NuGet](https://nuget.org/packages/GitHubActionsTestLogger): `dotnet add package GitHubActionsTestLogger` 40 | 41 | ## Screenshots 42 | 43 | ![annotations](.assets/annotations.png) 44 | ![summary](.assets/summary.png) 45 | 46 | ## Usage 47 | 48 | To use **GitHub Actions Test Logger**, install it in your test project and modify your GitHub Actions workflow by adding `--logger GitHubActions` to `dotnet test`: 49 | 50 | ```yaml 51 | name: main 52 | on: [push, pull_request] 53 | 54 | jobs: 55 | build: 56 | runs-on: ubuntu-latest 57 | 58 | steps: 59 | - name: Checkout 60 | uses: actions/checkout@v4 61 | 62 | - name: Install .NET 63 | uses: actions/setup-dotnet@v4 64 | 65 | - name: Build & test 66 | run: dotnet test --configuration Release --logger GitHubActions 67 | ``` 68 | 69 | By default, the logger will only report failed tests in the job summary and annotations. 70 | If you want the summary to include detailed information about passed and skipped tests as well, update the workflow as follows: 71 | 72 | ```yaml 73 | jobs: 74 | build: 75 | runs-on: ubuntu-latest 76 | 77 | steps: 78 | # ... 79 | 80 | - name: Build & test 81 | run: > 82 | dotnet test 83 | --configuration Release 84 | --logger "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true" 85 | ``` 86 | 87 | > **Warning**: 88 | > The new testing platform (i.e. `Microsoft.Testing.Platform`) [is not yet supported](https://github.com/Tyrrrz/GitHubActionsTestLogger/issues/41). This is because VSTest and MTP are using different extensibility models, and this project existed before MTP existed. 89 | > To use **GitHub Actions Test Logger**, make sure to use the classic testing experience (`vstest`) instead. 90 | 91 | > **Important**: 92 | > Ensure that your test project references the latest version of **Microsoft.NET.Test.Sdk**. 93 | > Older versions of this package may not be compatible with the logger. 94 | 95 | > **Important**: 96 | > If you are using **.NET SDK v2.2 or lower**, you need to [set the `` property to `true` in your test project](https://github.com/Tyrrrz/GitHubActionsTestLogger/issues/5#issuecomment-648431667). 97 | 98 | ### Collecting source information 99 | 100 | **GitHub Actions Test Logger** can leverage source information to link reported test results to the locations in the source code where the corresponding tests are defined. 101 | By default, `dotnet test` does not collect source information, so the logger relies on stack traces to extract it manually. 102 | This approach only works for failed tests, and even then may not always be fully accurate. 103 | 104 | To instruct the runner to collect source information, add the `RunConfiguration.CollectSourceInformation=true` argument to the command as shown below: 105 | 106 | ```yml 107 | jobs: 108 | build: 109 | runs-on: ubuntu-latest 110 | 111 | steps: 112 | # ... 113 | 114 | - name: Build & test 115 | # Note that the space after the last double dash (--) is intentional 116 | run: > 117 | dotnet test 118 | --configuration Release 119 | --logger GitHubActions 120 | -- 121 | RunConfiguration.CollectSourceInformation=true 122 | ``` 123 | 124 | > **Note**: 125 | > This option can also be enabled by setting the corresponding property in a [`.runsettings` file](https://learn.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file) instead. 126 | 127 | > **Warning**: 128 | > Source information collection may not work properly with legacy .NET Framework. 129 | 130 | ### Customizing behavior 131 | 132 | When running `dotnet test`, you can customize the logger's behavior by passing additional options: 133 | 134 | ```yml 135 | jobs: 136 | build: 137 | runs-on: ubuntu-latest 138 | 139 | steps: 140 | # ... 141 | 142 | - name: Build & test 143 | run: > 144 | dotnet test 145 | --configuration Release 146 | --logger "GitHubActions;annotations.titleFormat=@test;annotations.messageFormat=@error" 147 | ``` 148 | 149 | #### Custom annotation title 150 | 151 | Use the `annotations.titleFormat` option to specify the annotation title format used for reporting test failures. 152 | 153 | The following replacement tokens are available: 154 | 155 | - `@test` — replaced with the display name of the test 156 | - `@traits.TRAIT_NAME` — replaced with the value of the trait named `TRAIT_NAME` 157 | - `@error` — replaced with the error message 158 | - `@trace` — replaced with the stack trace 159 | - `@framework` — replaced with the target framework 160 | 161 | **Default**: `@test`. 162 | 163 | **Examples**: 164 | 165 | - `@test` → `MyTests.Test1` 166 | - `[@traits.Category] @test` → `[UI Tests] MyTests.Test1` 167 | - `@test (@framework)` → `MyTests.Test1 (.NETCoreApp,Version=v6.0)` 168 | 169 | #### Custom annotation message 170 | 171 | Use the `annotations.messageFormat` option to specify the annotation message format used for reporting test failures. 172 | Supports the same replacement tokens as [`annotations.titleFormat`](#custom-annotation-title). 173 | 174 | **Default**: `@error`. 175 | 176 | **Examples**: 177 | 178 | - `@error` → `AssertionException: Expected 'true' but found 'false'` 179 | - `@error\n@trace` → `AssertionException: Expected 'true' but found 'false'`, followed by stacktrace on the next line 180 | 181 | #### Include passed tests in summary 182 | 183 | Use the `summary.includePassedTests` option to specify whether passed tests should be included in the summary. 184 | If you want to link passed tests to their corresponding source definitions, make sure to also enable [source information collection](#collecting-source-information). 185 | 186 | **Default**: `false`. 187 | 188 | > **Warning**: 189 | > If your test suite is really large, enabling this option may cause the summary to exceed the [maximum allowed size](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#step-isolation-and-limits). 190 | 191 | #### Include skipped tests in summary 192 | 193 | Use the `summary.includeSkippedTests` option to specify whether skipped tests should be included in the summary. 194 | If you want to link skipped tests to their corresponding source definitions, make sure to also enable [source information collection](#collecting-source-information). 195 | 196 | **Default**: `false`. 197 | 198 | > **Warning**: 199 | > If your test suite is really large, enabling this option may cause the summary to exceed the [maximum allowed size](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#step-isolation-and-limits). 200 | 201 | #### Include not found tests in summary 202 | 203 | Use the `summary.includeNotFoundTests` option to specify whether empty test assemblies should be included in the summary. 204 | 205 | Using [test filters](https://learn.microsoft.com/en-us/dotnet/core/testing/selective-unit-tests) might result in some test assemblies not yielding any matching tests. 206 | This might be done on purpose in which case reporting these may not be helpful. 207 | 208 | **Default**: `true`. 209 | -------------------------------------------------------------------------------- /favicon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Tyrrrz/GitHubActionsTestLogger/81305a08e78b05e45ef81e4ae768497b1f8700b7/favicon.png --------------------------------------------------------------------------------