├── README.md
└── examples
    ├── conventions.xml
    ├── junit-basic.xml
    ├── junit-complete.xml
    ├── testcase-output.xml
    └── testcase-properties.xml


/README.md:
--------------------------------------------------------------------------------
  1 | # Common JUnit XML Format & Examples
  2 | 
  3 | _**Summary:** This project documents the common use of JUnit-style XML files by testing and CI tools, including full examples & common conventions. The goal of this project is to be the reference guide for JUnit-style XML files._
  4 | 
  5 | The JUnit XML file format has been made popular by the JUnit project[^1] and has since become the de facto standard format to exchange test results between tools. JUnit-style XML files are generated by countless testing frameworks and are processed by CI tools, build servers and test management software to read and display test results.
  6 | 
  7 | There is no official specification for the JUnit XML file format and various tools generate and support different flavors of this format. The goal of this project here is to document a common set of elements, attributes and conventions supported by many tools[^2]. If you are building software that produces or consumes JUnit-style XML files, this project might be a good starting point to learn about the format.
  8 | 
  9 | 1. [Basic JUnit XML structure](#structure)
 10 | 2. [Complete JUnit XML example](#example)
 11 | 3. [Properties for suites & cases](#properties)
 12 |     1. [Properties in test output](#properties-output)
 13 | 4. [Attachments, screenshots and artifacts](#attachments)
 14 |     1. [Attachment file properties](#attachments-files)
 15 |     2. [Attachment URL properties](#attachments-urls)
 16 |     3. [Attachment inline properties](#attachments-inline)
 17 |     4. [Attachments in test output](#attachments-output)
 18 | 5. [Common properties and conventions](#conventions)
 19 |     1. [Attachments](#conventions-attachments)
 20 |     2. [Steps](#conventions-steps)
 21 |     3. [Type hints](#conventions-types)
 22 | 6. [About this project](#about)
 23 | 
 24 | <a id="structure"></a>
 25 | ## Basic JUnit XML structure
 26 | 
 27 | The following example shows the basic structure of a JUnit-style XML file. In this example we have two root test suites (think classes/folders), one sub test suite, and a couple of test cases (test results) in each suite.
 28 | 
 29 | A test case is considered successful (passed) unless there is another result element underneath it. Most tools support the result elements `skipped`, `failure` and `error`.
 30 | 
 31 | This basic example is not complete as it leaves out various attributes and additional elements supported by most tools. See the next section for a full example.
 32 | 
 33 | <a href="examples/junit-basic.xml"><strong>Open example XML file</strong> (`junit-basic.xml`) <strong>→</strong></a>
 34 | 
 35 | ```xml
 36 | <?xml version="1.0" encoding="UTF-8"?>
 37 | <testsuites time="15.682687">
 38 |     <testsuite name="Tests.Registration" time="6.605871">
 39 |         <testcase name="testCase1" classname="Tests.Registration" time="2.113871" />
 40 |         <testcase name="testCase2" classname="Tests.Registration" time="1.051" />
 41 |         <testcase name="testCase3" classname="Tests.Registration" time="3.441" />
 42 |     </testsuite>
 43 |     <testsuite name="Tests.Authentication" time="9.076816">
 44 |         <testsuite name="Tests.Authentication.Login" time="4.356">
 45 |             <testcase name="testCase4" classname="Tests.Authentication.Login" time="2.244" />
 46 |             <testcase name="testCase5" classname="Tests.Authentication.Login" time="0.781" />
 47 |             <testcase name="testCase6" classname="Tests.Authentication.Login" time="1.331" />
 48 |         </testsuite>
 49 |         <testcase name="testCase7" classname="Tests.Authentication" time="2.508" />
 50 |         <testcase name="testCase8" classname="Tests.Authentication" time="1.230816" />
 51 |         <testcase name="testCase9" classname="Tests.Authentication" time="0.982">
 52 |             <failure message="Assertion error message" type="AssertionError">
 53 |                 <!-- Call stack printed here -->
 54 |             </failure>            
 55 |         </testcase>
 56 |     </testsuite>
 57 | </testsuites>
 58 | ```
 59 | 
 60 | <a id="example"></a>
 61 | ## Complete JUnit XML example
 62 | 
 63 | The following example lists all commonly supported elements, attributes and typical values used by tools that generate or read JUnit-style XML files. Not all elements are supported by all tools, and not all tools follow and understand the same conventions.
 64 | 
 65 | For example, [test cases properties](#properties) are not supported by all tools yet. Likewise, not all tools have options to [output attachments](#attachments). And different tools also expect or use different conventions for [names and values](#conventions).
 66 | 
 67 | That said, if you are building tools that work with JUnit-style XML files, it can be a good idea to try to support most or all of these common elements and attributes.
 68 | 
 69 | <a href="examples/junit-complete.xml"><strong>Open example XML file</strong> (`junit-complete.xml`) <strong>→</strong></a>
 70 | 
 71 | ```xml
 72 | <?xml version="1.0" encoding="UTF-8"?>
 73 | 
 74 | <!-- <testsuites> Usually the root element of a JUnit XML file. Some tools leave out 
 75 | the <testsuites> element if there is only a single top-level <testsuite> element (which
 76 | is then used as the root element).
 77 | 
 78 | name        Name of the entire test run
 79 | tests       Total number of tests in this file
 80 | failures    Total number of failed tests in this file
 81 | errors      Total number of errored tests in this file
 82 | skipped     Total number of skipped tests in this file
 83 | assertions  Total number of assertions for all tests in this file
 84 | time        Aggregated time of all tests in this file in seconds
 85 | timestamp   Date and time of when the test run was executed (in ISO 8601 format)
 86 | -->
 87 | <testsuites name="Test run" tests="8" failures="1" errors="1" skipped="1" 
 88 |     assertions="20" time="16.082687" timestamp="2021-04-02T15:48:23">
 89 |     
 90 |     <!-- <testsuite> A test suite usually represents a class, folder or group of tests.
 91 |     There can be many test suites in an XML file, and there can be test suites under other
 92 |     test suites.
 93 | 
 94 |     name        Name of the test suite (e.g. class name or folder name)
 95 |     tests       Total number of tests in this suite
 96 |     failures    Total number of failed tests in this suite
 97 |     errors      Total number of errored tests in this suite
 98 |     skipped     Total number of skipped tests in this suite
 99 |     assertions  Total number of assertions for all tests in this suite
100 |     time        Aggregated time of all tests in this file in seconds
101 |     timestamp   Date and time of when the test suite was executed (in ISO 8601 format)
102 |     file        Source code file of this test suite
103 |     -->
104 |     <testsuite name="Tests.Registration" tests="8" failures="1" errors="1" skipped="1" 
105 |         assertions="20" time="16.082687" timestamp="2021-04-02T15:48:23" 
106 |         file="tests/registration.code">
107 | 
108 |         <!-- <properties> Test suites (and test cases, see below) can have additional 
109 |         properties such as environment variables or version numbers. -->
110 |         <properties>
111 |             <!-- <property> Each property has a name and value. Some tools also support
112 |             properties with text values instead of value attributes. -->
113 |             <property name="version" value="1.774" />
114 |             <property name="commit" value="ef7bebf" />
115 |             <property name="browser" value="Google Chrome" />
116 |             <property name="ci" value="https://github.com/actions/runs/1234" />
117 |             <property name="config">
118 |                 Config line #1
119 |                 Config line #2
120 |                 Config line #3
121 |             </property>
122 |         </properties>
123 | 
124 |         <!-- <system-out> Optionally data written to standard out for the suite. 
125 |         Also supported on a test case level, see below. -->
126 |         <system-out>Data written to standard out.</system-out>
127 | 
128 |         <!-- <system-err> Optionally data written to standard error for the suite. 
129 |         Also supported on a test case level, see below. -->
130 |         <system-err>Data written to standard error.</system-err>
131 | 
132 |         <!-- <testcase> There are one or more test cases in a test suite. A test passed
133 |         if there isn't an additional result element (skipped, failure, error).
134 | 
135 |         name        The name of this test case, often the method name
136 |         classname   The name of the parent class/folder, often the same as the suite's name
137 |         assertions  Number of assertions checked during test case execution
138 |         time        Execution time of the test in seconds
139 |         file        Source code file of this test case
140 |         line        Source code line number of the start of this test case
141 |         -->
142 |         <testcase name="testCase1" classname="Tests.Registration" assertions="2"
143 |             time="2.436" file="tests/registration.code" line="24" />
144 |         <testcase name="testCase2" classname="Tests.Registration" assertions="6"
145 |             time="1.534" file="tests/registration.code" line="62" />
146 |         <testcase name="testCase3" classname="Tests.Registration" assertions="3"
147 |             time="0.822" file="tests/registration.code" line="102" />
148 |         
149 |         <!-- Example of a test case that was skipped -->
150 |         <testcase name="testCase4" classname="Tests.Registration" assertions="0"
151 |             time="0" file="tests/registration.code" line="164">
152 |             <!-- <skipped> Indicates that the test was not executed. Can have an optional
153 |             message describing why the test was skipped. -->
154 |             <skipped message="Test was skipped." />
155 |         </testcase>
156 | 
157 |         <!-- Example of a test case that failed. -->
158 |         <testcase name="testCase5" classname="Tests.Registration" assertions="2"
159 |             time="2.902412" file="tests/registration.code" line="202">
160 |             <!-- <failure> The test failed because one of the assertions/checks failed.
161 |             Can have a message and failure type, often the assertion type or class. The text
162 |             content of the element often includes the failure description or stack trace. -->
163 |             <failure message="Expected value did not match." type="AssertionError">
164 |                 <!-- Failure description or stack trace -->
165 |             </failure>
166 |         </testcase>
167 | 
168 |         <!-- Example of a test case that had errors. -->
169 |         <testcase name="testCase6" classname="Tests.Registration" assertions="0"
170 |             time="3.819" file="tests/registration.code" line="235">
171 |             <!-- <error> The test had an unexpected error during execution. Can have a 
172 |             message and error type, often the exception type or class. The text
173 |             content of the element often includes the error description or stack trace. -->
174 |             <error message="Division by zero." type="ArithmeticError">
175 |                 <!-- Error description or stack trace -->
176 |             </error>
177 |         </testcase>
178 | 
179 |         <!-- Example of a test case with outputs. -->
180 |         <testcase name="testCase7" classname="Tests.Registration" assertions="3"
181 |             time="2.944" file="tests/registration.code" line="287">
182 |             <!-- <system-out> Optional data written to standard out for the test case. -->
183 |             <system-out>Data written to standard out.</system-out>
184 | 
185 |             <!-- <system-err> Optional data written to standard error for the test case. -->
186 |             <system-err>Data written to standard error.</system-err>
187 |         </testcase>
188 | 
189 |         <!-- Example of a test case with properties -->
190 |         <testcase name="testCase8" classname="Tests.Registration" assertions="4"
191 |             time="1.625275" file="tests/registration.code" line="302">
192 |             <!-- <properties> Some tools also support properties for test cases. -->
193 |             <properties>
194 |                 <property name="priority" value="high" />
195 |                 <property name="language" value="english" />
196 |                 <property name="author" value="Adrian" />
197 |                 <property name="attachment" value="screenshots/dashboard.png" />
198 |                 <property name="attachment" value="screenshots/users.png" />
199 |                 <property name="description">
200 |                     This text describes the purpose of this test case and provides
201 |                     an overview of what the test does and how it works.
202 |                 </property>
203 |             </properties>
204 |         </testcase>
205 |     </testsuite>
206 | </testsuites>
207 | ```
208 | 
209 | <a id="properties"></a>
210 | ## Properties for suites and cases
211 | 
212 | Properties can be included for suites and cases to record additional information, such as version numbers, environment variables or links to CI pipelines. Properties are name / value pairs. Some tools support multi-line properties by including the property value as the element's text content.
213 | 
214 | <a href="examples/testcase-properties.xml"><strong>Open example XML file</strong> (`testcase-properties.xml`) <strong>→</strong></a><br>
215 | <a href="examples/testcase-output.xml"><strong>Open example XML file</strong> (`testcase-output.xml`) <strong>→</strong></a>
216 | 
217 | ```xml
218 | <testsuite name="Tests.Registration" time="7.005871">
219 |     <properties>
220 |         <property name="version" value="1.774" />
221 |         <property name="commit" value="ef7bebf" />
222 |         <property name="browser" value="Google Chrome" />
223 |         <property name="ci" value="https://github.com/actions/runs/1234" />
224 |         <property name="config"> <!-- Some tools support properties with text content -->
225 |             Config line #1
226 |             Config line #2
227 |             Config line #3
228 |         </property>
229 |     </properties>
230 | 
231 |     <!-- Test cases here -->
232 | </testsuite>
233 | ```
234 | 
235 | Originally, properties were only supported for test suite elements, not cases. However, many tools support properties for test cases now, including test frameworks such as [pytest](https://www.pytest.org/) and [Playwright](https://playwright.dev/), or test management tools such as [Testmo](https://www.testmo.com/) and many others. Often it can be useful to include additional details for test cases such as custom fields, test data or attachments.
236 | 
237 | ```xml
238 | <testcase name="testCase3" classname="Tests.Registration" time="3.441">
239 |     <properties>
240 |         <property name="priority" value="high" />
241 |         <property name="language" value="english" />
242 |         <property name="author" value="Adrian" />
243 |         <property name="attachment" value="screenshots/dashboard.png" />
244 |         <property name="attachment" value="screenshots/users.png" />
245 | 
246 |         <!-- Optional support for properties with text values -->
247 |         <property name="description">
248 |             This text describes the purpose of this test case and provides
249 |             an overview of what the test does and how it works.
250 |         </property>
251 |     </properties>
252 | 
253 |     <!-- Optional output or error/failure/skipped -->
254 | </testcase>
255 | ```
256 | 
257 | There can be multiple properties with the same name (e.g. multiple properties with the name `attachment`). Tools that read XML files should support this. However, many test frameworks have APIs that only support one property per name. In this case you should choose unique names for properties (e.g. `attachment1` and `attachment2`, [see below](#conventions)).
258 | 
259 | <a id="properties-output"></a>
260 | ### Properties in test output
261 | 
262 | Many test frameworks don't support properties for test cases yet. Another option to include properties in XML files is to add them to the test output (similar to referencing [attachments in the output](#attachments-output)). The advantage of this is that developers and testers can output properties even if their testing tool doesn't support test case properties.
263 | 
264 | ```xml
265 | <testcase name="testCase3" classname="Tests.Registration" time="3.441">
266 |     <system-out>
267 | Output line #1
268 | Output line #2
269 | 
270 | [[PROPERTY|author=Adrian]]
271 | [[PROPERTY|language=english]]
272 | 
273 | [[PROPERTY|browser-log]]
274 | Log line #1
275 | Log line #2
276 | Log line #3
277 | [[/PROPERTY]]
278 |     </system-out>
279 | </testcase>
280 | ```
281 | 
282 | Each property is listed on its own line without any additional text. Each property consists of a name / value pair. Optionally, multi-line properties just specify the name without a value in the opening tag and end with a closing tag on its own line.
283 | 
284 | <a id="attachments"></a>
285 | ## Attachments, screenshots and artifacts
286 | 
287 | Tests often generate or use additional files as part of a test run, such as log files, screenshots or build artifacts. Sometimes it can be useful to reference or include such files in the XML file. For example, for browser and UI tests it can be useful to include one or more screenshots to better understand why a test failed. Many testing tools support one or more of the following options for attachments.
288 | 
289 | <a id="attachments-files"></a>
290 | ### Attachment file properties
291 | 
292 | The simplest way to reference files is to list local file paths as properties. This can be included for test suites and test cases. Different tools expect different property _names_ for attachments, but many tools simply expect the properties to be named `attachment` (or alternatively `attachment1` etc.).
293 | 
294 | ```xml
295 | <testcase name="testCase3" classname="Tests.Registration" time="3.441">
296 |     <properties>
297 |         <property name="attachment" value="screenshots/dashboard.png" />
298 |         <property name="attachment" value="screenshots/users.png" />
299 |     </properties>
300 | </testcase>
301 | ```
302 | 
303 | <a id="attachments-urls"></a>
304 | ### Attachment URL properties
305 | 
306 | Files might also be uploaded to and stored with CI services or on build servers. For such scenarios it can be useful to reference files as URLs.
307 | 
308 | ```xml
309 | <testcase name="testCase3" classname="Tests.Registration" time="3.441">
310 |     <properties>
311 |         <property name="attachment1" value="https://build-server/runs/45112/screenshot1.png" />
312 |         <property name="attachment2" value="https://build-server/runs/45112/screenshot2.png" />
313 |     </properties>
314 | </testcase>
315 | ```
316 | 
317 | <a id="attachments-inline"></a>
318 | ### Attachment inline properties
319 | 
320 | For smaller files and to make it easier to record and transfer files together with an XML file, attachments can also be included inline as data URIs. Filenames can optionally be included with the non-standard `filename` attribute. Data should be encoded with base64.
321 | 
322 | ```xml
323 | <testcase name="testCase3" classname="Tests.Registration" time="3.441">
324 |     <properties>
325 |         <property name="attachment1" value="data:image/png;filename=dot.png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAATXL0Y4OHwAAAABJRU5ErkJggg==" />
326 |         <property name="attachment2" value="data:text/html;base64,SGVsbG8sIFdvcmxkIQ==" />
327 |     </properties>
328 | </testcase>
329 | ```
330 | 
331 | <a id="attachments-output"></a>
332 | ### Attachments in test output
333 | 
334 | Some tools support another way to reference files from the test output. This convention was first introduced by the [Jenkins JUnit Attachments](https://plugins.jenkins.io/junit-attachments/) plugin. This format is often used with tools that don't support test case properties. Each attachment is listed on a separate line (without any additional text) as part of the test's output (`system-out` or `system-err`) using the following format.
335 | 
336 | ```xml
337 | <testcase name="testCase3" classname="Tests.Registration" time="3.441">
338 |     <system-out>
339 | Output line #1
340 | Output line #2
341 | [[ATTACHMENT|screenshots/dashboard.png]]
342 | [[ATTACHMENT|screenshots/users.png]]
343 |     </system-out>
344 | </testcase>
345 | ```
346 | 
347 | <a id="conventions"></a>
348 | ## Common properties and conventions
349 | 
350 | Test frameworks and testing tools can generally write and read properties with any name and value. There aren't any standards on how to name properties or how to format your values. However, various testing tools that read JUnit-style XML files expect certain property names and values. This section lists examples for commonly used property names and how tools format property values.
351 | 
352 | <a href="examples/testcase-properties.xml"><strong>Open example XML file</strong> (`conventions.xml`) <strong>→</strong></a>
353 | 
354 | <a id="conventions-attachments"></a>
355 | ### Attachments
356 | 
357 | Name: `attachment*`<br>
358 | Files attached to suites or cases, such as log files, screenshots, build artifacts or configurations.
359 | 
360 | ```xml
361 | <!-- Either use multiple properties named "attachment" -->
362 | <property name="attachment" value="screenshots/failure1.png" />
363 | <property name="attachment" value="screenshots/failure2.png" />
364 | 
365 | <!-- Or append a unique index if your test framework requires this -->
366 | <property name="attachment1" value="browser/console.log" />
367 | <property name="attachment2" value="browser/config.json" />
368 | ```
369 | 
370 | <a id="conventions-steps"></a>
371 | ### Steps
372 | 
373 | Name: `step*`<br>
374 | Steps can be useful to document the exact steps a test case performs. Tools can render
375 | all steps with optional statuses and sub fields.
376 | 
377 | ```xml
378 | <!-- Either use multiple properties named "step" -->
379 | <property name="step" value="This is a test step." />
380 | <property name="step" value="Tools will automatically index all steps." />
381 | 
382 | <!-- Or append a unique index if your test framework requires this -->
383 | <property name="step1" value="You can also use unique step property names if needed." />
384 | <property name="step2" value="And this is another test step." />
385 | 
386 | <!-- Some tools support properties with text content -->
387 | <property name="step">
388 |     This is a more complex test step with multiple lines
389 |     of text using a property with a text value.
390 | </property>
391 | 
392 | <!-- You can optionally also include a step result as part of the property name.
393 | Check your CI or test management tool if it supports statuses. -->
394 | <property name="step[passed]" value="This is a test step that passed." />
395 | <property name="step[failure]" value="And this step had a failure." />
396 | <property name="step[skipped]" value="This step wasn't executed." />
397 | <property name="step[error]" value="And this step had an error." />
398 | 
399 | <!-- Some tools support custom HTML tags to document step sub fields. Check
400 | your CI or test management tool to learn about HTML support. -->
401 | <property name="step[passed]"><![CDATA[
402 |     <test-step-subfield name="Step">
403 |         This line describes this step. You can also use <strong>HTML</strong>.
404 |     </test-step-subfield>
405 |     <test-step-subfield name="Expected">
406 |         You can optionally include more sub fields, e.g. for the expected results.
407 |     </test-step-subfield>
408 | ]]></property>
409 | ```
410 | 
411 | <a id="conventions-types"></a>
412 | ### Type hints
413 | 
414 | Sometimes it can be useful to use more complex types for property values, such as HTML. Or tools support rendering specific property values in a special way. For example, URLs could be rendered as clickable links.
415 | 
416 | To avoid tools having to guess the type of a property value by its content, some tools support _type hints_ for property values. Tools that generate JUnit-style XML files (e.g. test automation frameworks) do not need to provide special support for this, as type hints are just provided as part of property names, using the format `type:name`.
417 | 
418 | There is no standard of types that tools support, so refer to your CI or test management tool's documentation to see if and which types might be supported. Here are some examples for types that tools might support:
419 | 
420 | ```xml
421 | <!-- Some tools also support type hints as part of the property names. Check your
422 | CI or test management tool to learn if and which type hints are supported. Type hints
423 | are removed from the property names. -->
424 | 
425 | <!-- URLs can be rendered as links -->
426 | <property name="url:github" value="https://github.com/org/project/blob/code.js#L200" />
427 | 
428 | <!-- Console values are rendered as monospaced code block -->
429 | <property name="console:log">
430 | Log line #1
431 | Log line #2
432 | Log line #3
433 | </property>
434 | 
435 | <!-- Some tools might support a limited subset of HTML for rich content -->
436 | <property name="html:output">
437 |     <![CDATA[<h1>Title</h1><p>Text</p>]]>
438 | </property>
439 | ```
440 | 
441 | <a id="about"></a>
442 | ## About this project
443 | 
444 | The goal of this project is to document the common use of JUnit-style XML files and describe how many tools implement support for this format. This project can be a starting point for builders of tools that generate or consume JUnit-style XML files. This file format has evolved over time and tools are adding support for test case properties and attachments as outlined in the above examples.
445 | 
446 | The information in this project has been collected by the team at [Testmo](https://www.testmo.com/). Please submit an issue if you have questions or feedback or think some additional details should be added.
447 | 
448 | [^1]: More precisely, the JUnit XML format was first introduced by the Ant build tool.
449 | [^2]: The format has evolved over time and new elements are used by tools now, so any original schema & XSD files are usually outdated.
450 | 


--------------------------------------------------------------------------------
/examples/conventions.xml:
--------------------------------------------------------------------------------
  1 | <?xml version="1.0" encoding="UTF-8"?>
  2 | <!--
  3 | This file provides examples for various conventions on how tools use property
  4 | names and values for special fields, attachments, tags, steps and more.
  5 | 
  6 | Example by Testmo. Copyright 2023 Testmo GmbH. All rights reserved.
  7 | Testmo test management software - https://www.testmo.com/
  8 | -->
  9 | <testsuites>
 10 |     <testsuite name="Tests">
 11 |         <testcase name="testExample" classname="Tests">
 12 |             <properties>
 13 |                 <!-- Regular basic properties and values -->
 14 |                 <property name="priority" value="high" />
 15 |                 <property name="author" value="Adrian" />
 16 | 
 17 |                 <!-- Example for a property with text value -->
 18 |                 <property name="description">
 19 |                     This text describes the purpose of this test case and provides
 20 |                     an overview of what the test does and how it works.
 21 |                 </property>
 22 | 
 23 |                 <!-- Some tools also support type hints as part of the property names. Check your
 24 |                 CI or test management tool to learn if and which type hints are supported. Type hints
 25 |                 are removed from the property names. -->
 26 |                 <property name="url:github" value="https://github.com/org/project/blob/code.js#L200" />
 27 |                 <property name="html:output">
 28 |                     <![CDATA[<h1>Title</h1><p>Text</p>]]>
 29 |                 </property>
 30 | 
 31 |                 <!-- Different ways to reference attachments -->
 32 |                 <property name="attachment" value="screenshots/dashboard.png" />
 33 |                 <property name="attachment" value="screenshots/users.png" />
 34 | 
 35 |                 <property name="attachment1" value="logs/browser/console.txt" />
 36 |                 <property name="attachment2" value="logs/server/access.txt" />
 37 | 
 38 |                 <property name="attachment" value="https://build-server/runs/45112/screenshot1.png" />
 39 |                 <property name="attachment" value="data:text/html;base64,SGVsbG8sIFdvcmxkIQ==" />
 40 | 
 41 |                 <!-- Different ways to document steps -->
 42 |                 <property name="step" value="This is a test step." />
 43 |                 <property name="step" value="Tools will automatically index all steps." />
 44 | 
 45 |                 <property name="step1" value="You can also use unique step property names if needed." />
 46 |                 <property name="step2" value="And this is another test step." />
 47 | 
 48 |                 <property name="step">
 49 |                     This is a more complex test step with multiple lines
 50 |                     of text using a property with a text value.
 51 |                 </property>
 52 | 
 53 |                 <!-- You can optionally also include a step result as part of the property name.
 54 |                 Check your CI or test management tool if it supports statuses. -->
 55 |                 <property name="step[passed]" value="This is a test step that passed." />
 56 |                 <property name="step[failure]" value="And this step had a failure." />
 57 |                 <property name="step[skipped]" value="This step wasn't executed." />
 58 |                 <property name="step[error]" value="And this step had an error." />
 59 | 
 60 |                 <!-- Some tools support custom HTML tags to document step sub fields. Check
 61 |                 your CI or test management tool to learn about HTML support. -->
 62 |                 <property name="step[passed]"><![CDATA[
 63 |                     <test-step-subfield name="Step">
 64 |                         This line describes this step. You can also use <strong>HTML</strong>.
 65 |                     </test-step-subfield>
 66 |                     <test-step-subfield name="Expected">
 67 |                         You can optionally include more sub fields, e.g. for the expected results.
 68 |                     </test-step-subfield>
 69 |                 ]]></property>
 70 |             </properties>
 71 | 
 72 |             <!-- For testing tools that don't support test case properties yet,
 73 |             developers can alternatively output properties as part of the system out.
 74 |             All the above conventions (property names and values) work exactly the same.
 75 |             Additionally, many tools support attachments directly in system out as well. -->
 76 |             <system-out><![CDATA[
 77 | Output line #1
 78 | Output line #2
 79 | 
 80 | [[PROPERTY|author=Adrian]]
 81 | [[PROPERTY|language=english]]
 82 | 
 83 | [[PROPERTY|browser-log]]
 84 | Log line #1
 85 | Log line #2
 86 | Log line #3
 87 | [[/PROPERTY]]
 88 | 
 89 | [[PROPERTY|html:output]]
 90 | <h1>Title</h1>
 91 | <p>This is an <strong>HTML</strong> example.</p>
 92 | [[/PROPERTY]]
 93 | 
 94 | [[ATTACHMENT|screenshots/dashboard.png]]
 95 | [[ATTACHMENT|screenshots/users.png]]]]>
 96 |             </system-out>
 97 |         </testcase>
 98 |     </testsuite>
 99 | </testsuites>
100 | 


--------------------------------------------------------------------------------
/examples/junit-basic.xml:
--------------------------------------------------------------------------------
 1 | <?xml version="1.0" encoding="UTF-8"?>
 2 | <!--
 3 | This is a basic JUnit-style XML example to highlight the basis structure.
 4 | 
 5 | Example by Testmo. Copyright 2023 Testmo GmbH. All rights reserved.
 6 | Testmo test management software - https://www.testmo.com/
 7 | -->
 8 | <testsuites time="15.682687">
 9 |     <testsuite name="Tests.Registration" time="6.605871">
10 |         <testcase name="testCase1" classname="Tests.Registration" time="2.113871" />
11 |         <testcase name="testCase2" classname="Tests.Registration" time="1.051" />
12 |         <testcase name="testCase3" classname="Tests.Registration" time="3.441" />
13 |     </testsuite>
14 |     <testsuite name="Tests.Authentication" time="9.076816">
15 |         <testsuite name="Tests.Authentication.Login" time="4.356">
16 |             <testcase name="testCase4" classname="Tests.Authentication.Login" time="2.244" />
17 |             <testcase name="testCase5" classname="Tests.Authentication.Login" time="0.781" />
18 |             <testcase name="testCase6" classname="Tests.Authentication.Login" time="1.331" />
19 |         </testsuite>
20 |         <testcase name="testCase7" classname="Tests.Authentication" time="2.508" />
21 |         <testcase name="testCase8" classname="Tests.Authentication" time="1.230816" />
22 |         <testcase name="testCase9" classname="Tests.Authentication" time="0.982">
23 |             <failure message="Assertion error message" type="AssertionError">
24 |                 <!-- Call stack printed here -->
25 |             </failure>
26 |         </testcase>
27 |     </testsuite>
28 | </testsuites>
29 | 


--------------------------------------------------------------------------------
/examples/junit-complete.xml:
--------------------------------------------------------------------------------
  1 | <?xml version="1.0" encoding="UTF-8"?>
  2 | <!--
  3 | This is a JUnit-style XML example with commonly used tags and attributes.
  4 | 
  5 | Example by Testmo. Copyright 2023 Testmo GmbH. All rights reserved.
  6 | Testmo test management software - https://www.testmo.com/
  7 | -->
  8 | 
  9 | <!-- <testsuites> Usually the root element of a JUnit XML file. Some tools leave out
 10 | the <testsuites> element if there is only a single top-level <testsuite> element (which
 11 | is then used as the root element).
 12 | 
 13 | name        Name of the entire test run
 14 | tests       Total number of tests in this file
 15 | failures    Total number of failed tests in this file
 16 | errors      Total number of errored tests in this file
 17 | skipped     Total number of skipped tests in this file
 18 | assertions  Total number of assertions for all tests in this file
 19 | time        Aggregated time of all tests in this file in seconds
 20 | timestamp   Date and time of when the test run was executed (in ISO 8601 format)
 21 | -->
 22 | <testsuites name="Test run" tests="8" failures="1" errors="1" skipped="1"
 23 |     assertions="20" time="16.082687" timestamp="2021-04-02T15:48:23">
 24 | 
 25 |     <!-- <testsuite> A test suite usually represents a class, folder or group of tests.
 26 |     There can be many test suites in an XML file, and there can be test suites under other
 27 |     test suites.
 28 | 
 29 |     name        Name of the test suite (e.g. class name or folder name)
 30 |     tests       Total number of tests in this suite
 31 |     failures    Total number of failed tests in this suite
 32 |     errors      Total number of errored tests in this suite
 33 |     skipped     Total number of skipped tests in this suite
 34 |     assertions  Total number of assertions for all tests in this suite
 35 |     time        Aggregated time of all tests in this file in seconds
 36 |     timestamp   Date and time of when the test suite was executed (in ISO 8601 format)
 37 |     file        Source code file of this test suite
 38 |     -->
 39 |     <testsuite name="Tests.Registration" tests="8" failures="1" errors="1" skipped="1"
 40 |         assertions="20" time="16.082687" timestamp="2021-04-02T15:48:23"
 41 |         file="tests/registration.code">
 42 | 
 43 |         <!-- <properties> Test suites (and test cases, see below) can have additional
 44 |         properties such as environment variables or version numbers. -->
 45 |         <properties>
 46 |             <!-- <property> Each property has a name and value. Some tools also support
 47 |             properties with text values instead of value attributes. -->
 48 |             <property name="version" value="1.774" />
 49 |             <property name="commit" value="ef7bebf" />
 50 |             <property name="browser" value="Google Chrome" />
 51 |             <property name="ci" value="https://github.com/actions/runs/1234" />
 52 |             <property name="config">
 53 |                 Config line #1
 54 |                 Config line #2
 55 |                 Config line #3
 56 |             </property>
 57 |         </properties>
 58 | 
 59 |         <!-- <system-out> Optionally data written to standard out for the suite.
 60 |         Also supported on a test case level, see below. -->
 61 |         <system-out>Data written to standard out.</system-out>
 62 | 
 63 |         <!-- <system-err> Optionally data written to standard error for the suite.
 64 |         Also supported on a test case level, see below. -->
 65 |         <system-err>Data written to standard error.</system-err>
 66 | 
 67 |         <!-- <testcase> There are one or more test cases in a test suite. A test passed
 68 |         if there isn't an additional result element (skipped, failure, error).
 69 | 
 70 |         name        The name of this test case, often the method name
 71 |         classname   The name of the parent class/folder, often the same as the suite's name
 72 |         assertions  Number of assertions checked during test case execution
 73 |         time        Execution time of the test in seconds
 74 |         file        Source code file of this test case
 75 |         line        Source code line number of the start of this test case
 76 |         -->
 77 |         <testcase name="testCase1" classname="Tests.Registration" assertions="2"
 78 |             time="2.436" file="tests/registration.code" line="24" />
 79 |         <testcase name="testCase2" classname="Tests.Registration" assertions="6"
 80 |             time="1.534" file="tests/registration.code" line="62" />
 81 |         <testcase name="testCase3" classname="Tests.Registration" assertions="3"
 82 |             time="0.822" file="tests/registration.code" line="102" />
 83 | 
 84 |         <!-- Example of a test case that was skipped -->
 85 |         <testcase name="testCase4" classname="Tests.Registration" assertions="0"
 86 |             time="0" file="tests/registration.code" line="164">
 87 |             <!-- <skipped> Indicates that the test was not executed. Can have an optional
 88 |             message describing why the test was skipped. -->
 89 |             <skipped message="Test was skipped." />
 90 |         </testcase>
 91 | 
 92 |         <!-- Example of a test case that failed. -->
 93 |         <testcase name="testCase5" classname="Tests.Registration" assertions="2"
 94 |             time="2.902412" file="tests/registration.code" line="202">
 95 |             <!-- <failure> The test failed because one of the assertions/checks failed.
 96 |             Can have a message and failure type, often the assertion type or class. The text
 97 |             content of the element often includes the failure description or stack trace. -->
 98 |             <failure message="Expected value did not match." type="AssertionError">
 99 |                 <!-- Failure description or stack trace -->
100 |             </failure>
101 |         </testcase>
102 | 
103 |         <!-- Example of a test case that had errors. -->
104 |         <testcase name="testCase6" classname="Tests.Registration" assertions="0"
105 |             time="3.819" file="tests/registration.code" line="235">
106 |             <!-- <error> The test had an unexpected error during execution. Can have a
107 |             message and error type, often the exception type or class. The text
108 |             content of the element often includes the error description or stack trace. -->
109 |             <error message="Division by zero." type="ArithmeticError">
110 |                 <!-- Error description or stack trace -->
111 |             </error>
112 |         </testcase>
113 | 
114 |         <!-- Example of a test case with outputs. -->
115 |         <testcase name="testCase7" classname="Tests.Registration" assertions="3"
116 |             time="2.944" file="tests/registration.code" line="287">
117 |             <!-- <system-out> Optional data written to standard out for the test case. -->
118 |             <system-out>Data written to standard out.</system-out>
119 | 
120 |             <!-- <system-err> Optional data written to standard error for the test case. -->
121 |             <system-err>Data written to standard error.</system-err>
122 |         </testcase>
123 | 
124 |         <!-- Example of a test case with properties -->
125 |         <testcase name="testCase8" classname="Tests.Registration" assertions="4"
126 |             time="1.625275" file="tests/registration.code" line="302">
127 |             <!-- <properties> Some tools also support properties for test cases. -->
128 |             <properties>
129 |                 <property name="priority" value="high" />
130 |                 <property name="language" value="english" />
131 |                 <property name="author" value="Adrian" />
132 |                 <property name="attachment" value="screenshots/dashboard.png" />
133 |                 <property name="attachment" value="screenshots/users.png" />
134 |                 <property name="description">
135 |                     This text describes the purpose of this test case and provides
136 |                     an overview of what the test does and how it works.
137 |                 </property>
138 |             </properties>
139 |         </testcase>
140 |     </testsuite>
141 | </testsuites>
142 | 


--------------------------------------------------------------------------------
/examples/testcase-output.xml:
--------------------------------------------------------------------------------
 1 | <?xml version="1.0" encoding="UTF-8"?>
 2 | <!--
 3 | This example shows different ways to include properties in the XML file.
 4 | 
 5 | Example by Testmo. Copyright 2023 Testmo GmbH. All rights reserved.
 6 | Testmo test management software - https://www.testmo.com/
 7 | -->
 8 | <testsuites>
 9 |     <testsuite name="Tests">
10 |         <!-- Most tools already support properties for test suites -->
11 |         <properties>
12 |             <property name="version" value="1.774" />
13 |             <property name="commit" value="ef7bebf" />
14 |             <property name="browser" value="Google Chrome" />
15 |             <property name="ci" value="https://github.com/actions/runs/1234" />
16 |             <property name="config"> <!-- Some tools support properties with text content -->
17 |                 Config line #1
18 |                 Config line #2
19 |                 Config line #3
20 |             </property>
21 |         </properties>
22 |         <testcase name="testExample" classname="Tests">
23 |             <!-- More and more tools also support properties for test cases now -->
24 |             <properties>
25 |                 <property name="priority" value="high" />
26 |                 <property name="language" value="english" />
27 |                 <property name="author" value="Adrian" />
28 |                 <property name="attachment" value="screenshots/dashboard.png" />
29 |                 <property name="attachment" value="screenshots/users.png" />
30 | 
31 |                 <!-- Optional support for properties with text values -->
32 |                 <property name="description">
33 |                     This text describes the purpose of this test case and provides
34 |                     an overview of what the test does and how it works.
35 |                 </property>
36 |             </properties>
37 | 
38 |             <!-- For testing tools that don't support test case properties yet,
39 |             developers can alternatively output properties as part of the system out,
40 |             similar to how many tools already support attachments in system out. -->
41 |             <system-out>
42 | Output line #1
43 | Output line #2
44 | 
45 | [[PROPERTY|author=Adrian]]
46 | [[PROPERTY|language=english]]
47 | 
48 | [[PROPERTY|browser-log]]
49 | Log line #1
50 | Log line #2
51 | Log line #3
52 | [[/PROPERTY]]
53 | 
54 | [[ATTACHMENT|screenshots/dashboard.png]]
55 | [[ATTACHMENT|screenshots/users.png]]
56 |             </system-out>
57 |         </testcase>
58 |     </testsuite>
59 | </testsuites>
60 | 


--------------------------------------------------------------------------------
/examples/testcase-properties.xml:
--------------------------------------------------------------------------------
 1 | <?xml version="1.0" encoding="UTF-8"?>
 2 | <!--
 3 | This example shows how test suite and test case properties are supported by various tools.
 4 | 
 5 | Example by Testmo. Copyright 2023 Testmo GmbH. All rights reserved.
 6 | Testmo test management software - https://www.testmo.com/
 7 | -->
 8 | <testsuites>
 9 |     <testsuite name="Tests">
10 |         <!-- Most tools already support properties for test suites -->
11 |         <properties>
12 |             <property name="version" value="1.774" />
13 |             <property name="commit" value="ef7bebf" />
14 |             <property name="browser" value="Google Chrome" />
15 |             <property name="ci" value="https://github.com/actions/runs/1234" />
16 |             <property name="config"> <!-- Some tools support properties with text content -->
17 |                 Config line #1
18 |                 Config line #2
19 |                 Config line #3
20 |             </property>
21 |         </properties>
22 |         <testcase name="testExample" classname="Tests">
23 |             <!-- More and more tools also support properties for test cases now -->
24 |             <properties>
25 |                 <property name="priority" value="high" />
26 |                 <property name="language" value="english" />
27 |                 <property name="author" value="Adrian" />
28 |                 <property name="attachment" value="screenshots/dashboard.png" />
29 |                 <property name="attachment" value="screenshots/users.png" />
30 | 
31 |                 <!-- Optional support for properties with text values -->
32 |                 <property name="description">
33 |                     This text describes the purpose of this test case and provides
34 |                     an overview of what the test does and how it works.
35 |                 </property>
36 |             </properties>
37 |         </testcase>
38 |     </testsuite>
39 | </testsuites>
40 | 


--------------------------------------------------------------------------------