├── .github
└── workflows
│ └── auto-publish.yml
├── .gitignore
├── .pr-preview.json
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── EXPLAINER.md
├── LICENSE.md
├── Makefile
├── README.md
├── index.bs
├── network-reporting.bs
├── security-and-privacy-questionnaire.md
├── tidyconfig.txt
└── w3c.json
/.github/workflows/auto-publish.yml:
--------------------------------------------------------------------------------
1 | # .github/workflows/auto-publish.yml
2 | name: Reporting API publication
3 | on:
4 | pull_request: {}
5 | push:
6 | branches: [main]
7 |
8 | jobs:
9 | reporting:
10 | name: Publish Reporting API WD
11 | runs-on: ubuntu-latest
12 | steps:
13 | - uses: actions/checkout@v2
14 | - uses: w3c/spec-prod@v2
15 | with:
16 | SOURCE: index.bs
17 | TOOLCHAIN: bikeshed
18 | BUILD_FAIL_ON: warning
19 | W3C_ECHIDNA_TOKEN: ${{ secrets.W3C_TR_TOKEN }}
20 | W3C_WG_DECISION_URL: https://lists.w3.org/Archives/Public/public-web-perf/2021Apr/0005.html
21 | W3C_BUILD_OVERRIDE: |
22 | shortName: reporting
23 | status: WD
24 | GH_PAGES_BRANCH: gh-pages
25 | network-reporting:
26 | name: Publish Network Reporting API ED
27 | runs-on: ubuntu-latest
28 | steps:
29 | - uses: actions/checkout@v2
30 | - uses: w3c/spec-prod@v2
31 | with:
32 | SOURCE: network-reporting.bs
33 | TOOLCHAIN: bikeshed
34 | BUILD_FAIL_ON: link-error
35 | GH_PAGES_BRANCH: gh-pages
36 |
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
1 | /index.html
2 |
--------------------------------------------------------------------------------
/.pr-preview.json:
--------------------------------------------------------------------------------
1 | {
2 | "src_file": "index.src.html",
3 | "type": "bikeshed",
4 | "params": {
5 | "force": 1
6 | }
7 | }
8 |
--------------------------------------------------------------------------------
/CODE_OF_CONDUCT.md:
--------------------------------------------------------------------------------
1 | # Code of Conduct
2 |
3 | All documentation, code and communication under this repository are covered by the [W3C Code of Ethics and Professional Conduct](https://www.w3.org/Consortium/cepc/).
4 |
--------------------------------------------------------------------------------
/CONTRIBUTING.md:
--------------------------------------------------------------------------------
1 | # Web Performance Working Group
2 |
3 | Contributions to this repository are intended to become part of Recommendation-track documents governed by the
4 | [W3C Patent Policy](http://www.w3.org/Consortium/Patent-Policy-20040205/) and
5 | [Software and Document License](http://www.w3.org/Consortium/Legal/copyright-software). To make substantive contributions to specifications, you must either participate
6 | in the relevant W3C Working Group or make a non-member patent licensing commitment.
7 |
8 | If you are not the sole contributor to a contribution (pull request), please identify all
9 | contributors in the pull request comment.
10 |
11 | To add a contributor (other than yourself, that's automatic), mark them one per line as follows:
12 |
13 | ```
14 | +@github_username
15 | ```
16 |
17 | If you added a contributor by mistake, you can remove them in a comment with:
18 |
19 | ```
20 | -@github_username
21 | ```
22 |
23 | If you are making a pull request on behalf of someone else but you had no part in designing the
24 | feature, you can remove yourself with the above syntax.
25 |
--------------------------------------------------------------------------------
/EXPLAINER.md:
--------------------------------------------------------------------------------
1 | # Reporting API #
2 | The Reporting API is a mechanism for web servers to tell browsers where to send errors and other information about a browsing session. This explainer summarizes the basic usage. For details see the [Reporting API specification](http://wicg.github.io/reporting/) and the specifications of other features that use the Reporting API such as [Content Security Policy](https://w3c.github.io/webappsec-csp/#reporting), [Deprecation Reports](https://wicg.github.io/deprecation-reporting/), or [Cross-origin Opener Policy](https://github.com/camillelamy/explainers/blob/master/coop_reporting.md).
3 |
4 | ## The problem ##
5 | When a web application encounters some error or potential problem it is important that the application author have some mechanism to be made aware of that error. Common errors such as unhandled JavaScript exceptions can be observed in script. But other errors may occur when it's not possible to rely on running script (such as a browser crash, or a Content Security Policy violation that prevents the page from being loaded). The Reporting API provides a generic mechanism for a browser to report errors back to an HTTP Server in an out-of-band fashion.
6 |
7 | ## Enabling reporting ##
8 | Reporting is enabled by specifying a `Reporting-Endpoints` header in the HTTP response, eg:
9 | ```http
10 | Reporting-Endpoints: default="https://example.com/reports"
11 | ```
12 | The header defines the names for each reporting endpoint. Different features have different ways of specifying which reporting endpoint to use for particular reports. Some, like CSP, use a `report-to` parameter; others, such as deprecation reports, use the `default` endpoint if it is specified.
13 |
14 | When reports are generated they are cached for a short time and then sent to the appropriate endpoint, which will receive an HTTP `POST` request with JSON-formatted body containing an array of reports, eg:
15 | ```http
16 | POST /reports HTTP/1.1
17 | Host: example.com
18 | ...
19 | Content-Type: application/reports+json
20 |
21 | [{
22 | "type": "myreport",
23 | "age": 10,
24 | "url": "https://example.com/originatingpage/",
25 | "user_agent": "FooBrowser/1.0 (KHTML, sorta like Gecko)",
26 | "body": {
27 | ...
28 | }
29 | }]
30 | ```
31 |
32 | Each report has:
33 | - `type`: a string that indicates the category of report.
34 | - `age`: the number of seconds between when the report was triggered and when it was sent.
35 | - `url`: the URL of the page which triggered the report.
36 | - `user_agent`: the reporting browser's user agent.
37 | - `body`: the contents of the report as defined by the `type`.
38 |
39 | ## Example report: Deprecations ##
40 | Some report types are defined which are always enabled (and sent to the `default` reporting [group](http://wicg.github.io/reporting/#id-member)).
41 | [Deprecation Reports](https://wicg.github.io/deprecation-reporting/) are reports indicating that a browser API or feature has been used which is expected to stop working in a future update to the browser. For example:
42 |
43 | ```json
44 | {
45 | "type": "deprecation",
46 | "age": 10,
47 | "url": "https://example.com/",
48 | "user_agent": "BarBrowser/98.0 (Mozilla/5.0 compatiblish)",
49 | "body": {
50 | "id": "websql",
51 | "anticipatedRemoval": "1/1/2020",
52 | "message": "WebSQL is deprecated and will be removed in Chrome 97 around January 2020",
53 | "sourceFile": "https://example.com/index.js",
54 | "lineNumber": 1234,
55 | "columnNumber": 42
56 | }
57 | }
58 | ```
59 |
60 | The report body has the following properties:
61 | - `id` (required): an implementation-defined string identifying the feature or API that will be removed. This string can be used for grouping and counting related reports.
62 | - `anticipatedRemoval`: A date indicating roughly when the browser version without the specified API will be generally available (excluding "beta" or other pre-release channels). This value should be used to sort or prioritize warnings. When omitted the deprecation should be considered low priority (removal may not actually occur).
63 | - `message`: A developer-readable message with details (typically matching what would be displayed on the developer console). The message is not guaranteed to be unique for a given `id` (eg. it may contain additional context on how the API was used).
64 | - `sourceFile`: If known, the file which first used the indicated API
65 | - `lineNumber`: if known, the line number in `sourceFile` where the indicated API was first used.
66 | - `columnNumber`: if known, the column number in `sourceFile` where the indicated API was first used.
67 |
68 |
69 | ## ReportingObserver - Observing reports from JavaScript
70 | In addition to (or even instead of) having reports delivered to an endpoint, it can be convenient to be informed of reports from within the page's JavaScript (eg. for analytics libraries which have no way to influence HTTP headers). This doesn't make sense or isn't possible for all reports (eg. crashes), but is most useful for reports generated as a direct result of something the page's script has done (such as a deprecation warning).
71 |
72 | ```javascript
73 | function onReport(reports, observer) {
74 | for(let report of reports) {
75 | if (report.type == "deprecation") {
76 | sendDeprecationAnalytics(JSON.stringify(report.body));
77 | }
78 | }
79 | }
80 |
81 | let observer = new ReportingObserver(onReport);
82 | observer.observe();
83 | ```
84 |
85 | Shortly after a report corresponding to a given JavaScript context is generated (even if there are no endpoints registered), all `ReportingObserver` callback functions in that context are invoked with a copy of the report as a JavaScript object. Since the exact details of reports can vary from one browser to another, applications generally should not change their behavior based on the presence or contents of a report, but use this API only for analytics purposes.
86 |
--------------------------------------------------------------------------------
/LICENSE.md:
--------------------------------------------------------------------------------
1 | All documents in this Repository are licensed by contributors under the [W3C Software and Document License](http://www.w3.org/Consortium/Legal/copyright-software).
2 |
--------------------------------------------------------------------------------
/Makefile:
--------------------------------------------------------------------------------
1 | all: index.html
2 |
3 | force:
4 | bikeshed -f spec ./index.src.html
5 |
6 | index.html: index.src.html
7 | bikeshed -f spec ./index.src.html
8 |
9 | publish:
10 | git push origin master master:gh-pages
11 |
12 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | Reporting API
2 | ===========
3 |
4 | This document defines a generic reporting framework which allows web developers to associate a set of named reporting endpoints with an origin. Various platform features (like Content Security Policy, Network Error Reporting, and others) will use these endpoints to deliver feature-specific reports in a consistent manner.
5 |
6 |
7 | * Read latest draft: https://w3c.github.io/reporting/
8 | * Discuss on [open issues](https://github.com/w3c/reporting/issues)
9 |
10 | See also the [EXPLAINER](https://github.com/w3c/reporting/blob/master/EXPLAINER.md)
11 |
--------------------------------------------------------------------------------
/index.bs:
--------------------------------------------------------------------------------
1 | Reporting API
2 |
3 | Status: ED
4 | ED: https://w3c.github.io/reporting/
5 | Shortname: reporting
6 | TR: https://www.w3.org/TR/reporting-1/
7 | Group: webperf
8 | Editor: Douglas Creager 103120, GitHub, dcreager@dcreager.net
9 | Editor: Ian Clelland 76841, Google Inc., iclelland@google.com
10 | Editor: Mike West 56384, Google Inc., mkwst@google.com
11 | Former Editor: Ilya Grigorik 56102, Google Inc., igrigorik@google.com
12 | Former Editor: Paul Meyer 99916, Google Inc., paulmeyer@google.com
13 | Abstract:
14 | This document defines a generic reporting framework which allows web
15 | developers to associate a set of named reporting endpoints with an origin.
16 | Various platform features can use these endpoints to deliver
17 | feature-specific reports in a consistent manner.
18 | Level: 1
19 | Indent: 2
20 | Version History: https://github.com/w3c/reporting/commits/main/index.src.html
21 | Boilerplate: omit conformance
22 | Repository: https://github.com/w3c/reporting/
23 | Markup Shorthands: css off, markdown on
24 |
25 |
26 | spec: CSP; urlPrefix: https://w3c.github.io/webappsec-csp/
27 | type: dfn
28 | text: Content-Security-Policy
29 | text: reports directive; url: directives-reporting
30 | spec: FETCH; urlPrefix: https://fetch.spec.whatwg.org/
31 | type: dfn
32 | text: navigation request
33 | text: response; url: concept-response
34 | text: request; url: concept-request
35 | text: header; url: concept-header
36 | text: header list; url: concept-header-list
37 | text: main fetch
38 | text: fetch; url: concept-fetch
39 | text: wait for a response
40 | text: ok status
41 | type: attribute
42 | for: response
43 | text: url; url: concept-response-url
44 | text: HTTPS state; url: concept-response-https-state
45 | text: header list; url: concept-response-header-list
46 | for: request
47 | text: target browsing context; url: concept-request-target-browsing-context
48 | for: header
49 | text: name; url: concept-header-name
50 | text: value; url: concept-header-value
51 | spec: SECURE-CONTEXTS; urlPrefix: https://w3c.github.io/webappsec-secure-contexts/
52 | type: dfn
53 | text: potentially trustworthy; url: is-origin-trustworthy
54 | spec: URL; urlPrefix: https://url.spec.whatwg.org/
55 | type: dfn
56 | text: origin of a url; url: concept-url-origin
57 | text: URL serializer; url: concept-url-serializer
58 | text: URL parser; url: concept-url-parser
59 | type: interface
60 | text: URL; url: concept-url
61 | type: attribute
62 | for: URL
63 | text: username; url: concept-url-username
64 | text: password; url: concept-url-password
65 | spec: HTML; urlPrefix: https://html.spec.whatwg.org/multipage/
66 | urlPrefix: webappapis.html
67 | type: dfn
68 | text: environment settings object
69 | text: creation URL
70 | text: queue a task
71 | urlPrefix: browsers.html
72 | type: dfn
73 | text: origin
74 | urlPrefix: system-state.html
75 | type: dfn
76 | text: navigator.userAgent; url: dom-navigator-useragent
77 | spec: RFC2782; for: SRV; urlPrefix: https://tools.ietf.org/html/rfc2782
78 | type: dfn
79 | text: SRV record; url:
80 | text: target selection algorithm; url: page-4
81 | spec: RFC6797; urlPrefix: https://tools.ietf.org/html/rfc6797
82 | type: dfn
83 | url: section-8.2
84 | text: superdomain match
85 | spec: RFC8259; urlPrefix: https://tools.ietf.org/html/rfc8259
86 | type: dfn
87 | text: JSON text; url: section-2
88 | spec: RFC7230; urlPrefix: https://tools.ietf.org/html/rfc7230
89 | type: grammar
90 | text: OWS; url: section-3.2.3
91 | text: BWS; url: section-3.2.3
92 | text: token; url: section-3.2.6
93 | text: quoted-string; url: section-3.2.6
94 | text: #rule; url: section-7
95 | spec: RFC7469; urlPrefix: https://tools.ietf.org/html/rfc7469
96 | type: dfn
97 | text: Public-Key-Pins; url: section-2.1
98 | spec: HTTP-JFV; urlPrefix: https://tools.ietf.org/html/draft-reschke-http-jfv
99 | type: grammar
100 | text: json-field-value; url: section-2
101 | spec: webdriver; urlPrefix: https://w3c.github.io/webdriver/#
102 | type: dfn
103 | text: current browsing context; url: dfn-current-browsing-context
104 | text: handle any user prompts; url: dfn-handle-any-user-prompts
105 | text: WebDriver error; url: dfn-error
106 | text: WebDriver error code; url: dfn-error-code
107 | text: extension command; url: dfn-extension-command
108 | text: extension command uri template; url: dfn-extension-command-uri-template
109 | text: invalid argument; url: dfn-invalid-argument
110 | text: no such window; url: dfn-no-such-window
111 | text: local end; url: dfn-local-end
112 | text: remote end steps; url: dfn-remote-end-steps
113 | text: session; url: dfn-session
114 | text: success; url: dfn-success
115 | text: trying; url: dfn-try
116 | spec: STRUCTURED-FIELDS; urlPrefix: https://www.rfc-editor.org/rfc/rfc8941.html#
117 | type: grammar
118 | text: sf-dictionary; url: section-3.2
119 |
120 |
121 | {
122 | "SECURE-CONTEXTS": {
123 | "authors": [ "Mike West", "Yan Zhu" ],
124 | "href": "https://w3c.github.io/webappsec-secure-contexts/",
125 | "title": "Secure Contexts",
126 | "publisher": "W3C"
127 | },
128 | "STRUCTURED-FIELDS": {
129 | "authors": [ "Mark Nottingham", "Poul-Henning Kamp" ],
130 | "href": "https://www.rfc-editor.org/rfc/rfc8941.html",
131 | "title": "Structured Field Values for HTTP",
132 | "publisher": "IETF"
133 | }
134 | }
135 |
136 |
137 |
138 | Introduction
139 |
140 | This document provides three pieces of infrastructure for generic reporting,
141 | which may be used or extended by other specifications:
142 |
143 | 1. A generic framework for defining report types and reporting endpoints, and
144 | a document format for sending reports to endpoints over HTTP.
145 |
146 | 2. A specific mechanism for configuring reporting endpoints in a document or
147 | worker, and for delivering reports whose lifetime is tied to that document
148 | or worker.
149 |
150 | 3. A JavaScript interface for observing reports generated within a document or
151 | worker.
152 |
153 | Other specifications may extend or make use of these pieces, for instance by
154 | defining concrete report types, or alternative configuration or delivery
155 | mechanisms for non-document-based reports.
156 |
157 | Guarantees
158 |
159 | This specification aims to provide a best-effort report delivery system that
160 | executes out-of-band with website activity. The user agent will be able to do
161 | a better job prioritizing and scheduling delivery of reports, as it has an
162 | overview of cross-origin activity that individual websites do not, and can
163 | deliver reports based on error conditions that would prevent a website from
164 | loading in the first place.
165 |
166 | The delivery is not, however, guaranteed in any way, and reporting is not
167 | intended to be used as a reliable communications channel. Network conditions
168 | may prevent reports from reaching their destination at all, and user agents
169 | are permitted to reject and not deliver a report for any reason.
170 |
171 | Examples
172 |
173 |
174 | MegaCorp Inc. wants to collect Content Security Policy and Key Pinning
175 | violation reports. It can do so by delivering the following header to
176 | define a set of reporting endpoints named "`endpoint-1`":
177 |
178 |
179 | Reporting-Endpoints: endpoint-1="https://example.com/reports"
180 |
181 |
182 | And the following headers, which direct CSP and HPKP reports to that
183 | endpoint:
184 |
185 |
186 | Content-Security-Policy: ...; report-to endpoint-1
187 | Public-Key-Pins: ...; report-to=endpoint-1
188 |
189 |
192 | After processing reports for a little while, MegaCorp Inc. decides to split
193 | the processing of these two types of reports out into two distinct endpoints
194 | in order to make the processing scripts simpler. It can do so by delivering
195 | the following header to define two reporting endpoints:
196 |
197 |
198 | Reporting-Endpoints: csp-endpoint="https://example.com/csp-reports",
199 | hpkp-endpoint="https://example.com/hpkp-reports"
200 |
201 |
202 | And the following headers, which direct CSP and HPKP reports to those named
203 | endpoints:
204 |
205 |
206 | Content-Security-Policy: ...; report-to csp-endpoint
207 | Public-Key-Pins: ...; report-to=hpkp-endpoint
208 |
209 |
Generic Reporting Framework
214 |
215 |
216 | This section defines the generic concepts of reports and endpoints, and how
217 | reports are serialized into the `application/reports+json` format.
219 |
220 | Concepts
221 |
222 | Endpoints
223 |
224 | An endpoint is location to which reports for a
225 | particular origin may be sent.
226 |
227 | Each endpoint has a
228 | name, which is an ASCII string.
229 |
230 | Each endpoint has a url,
231 | which is a {{URL}}.
232 |
233 | Each endpoint has a
234 | failures, which is a non-negative
235 | integer representing the number of consecutive times this endpoint has failed
236 | to respond to a request.
237 |
238 | Report Type
239 |
240 | A report type is a non-empty string that specifies the set
241 | of data that is contained in the [=report/body=] of a report.
242 |
243 | When a report type is defined (in this spec or others), it can be
244 | specified to be visible to ReportingObservers, meaning
245 | that reports of that type can be observed by a reporting
246 | observer. By default, report types are not visible to
247 | ReportingObservers.
248 |
249 | Reports
250 |
251 | A report is a collection of arbitrary data which the user
252 | agent is expected to deliver to a specified endpoint.
253 |
254 | Each report has a body, which is either
255 | `null` or an object which can be serialized into a JSON text. The
256 | fields contained in a report's [=report/body=] are determined by
257 | the report's [=report/type=].
258 |
259 | Each report has a url, which
260 | is typically the address of the `Document` or `Worker` from which the report
261 | was generated.
262 |
263 | Note: We strip the username, password, and fragment from this serialized URL.
264 | See [[#capability-urls]].
265 |
266 | Each report has a user agent, which is
267 | the value of the User-Agent header of the request
268 | from which the report was generated.
269 |
270 | Note: The user agent of a report represents the
271 | User-Agent sent by the browser for the page which generated the
272 | report. This is potentially distinct from the User-Agent
273 | sent in the HTTP headers when uploading the report to a collector — for
274 | instance, where the browser has chosen to use a non-default
275 | User-Agent string such as the "request desktop site" feature.
276 |
277 | Each report has a destination,
278 | which is a string representing the {{endpoint/name}} of the endpoint
279 | that the report will be sent to.
280 |
281 | Each report has a type, which is a report type.
283 |
284 | Each report has a timestamp,
285 | which records the time at which the report was generated, in milliseconds
286 | since the unix epoch.
287 |
288 | Each report has an attempts
289 | counter, which is a non-negative integer representing the number of times the
290 | user agent attempted to deliver the report.
291 |
292 |
293 |
294 | The media type used when POSTing reports to a specified endpoint is
295 | `application/reports+json`.
296 |
297 |
298 | Queue |data| as |type| for |destination|
299 |
300 |
301 | To generate a report given a
302 | serializable object (|data|), a string (|type|), another string
303 | (|destination|), an optional environment settings object
304 | (|settings|), and an optional {{URL}} (|url|):
305 |
306 | 1. Let |report| be a new report object with its values initialized as
307 | follows:
308 |
309 | : [=report/body=]
310 | :: |data|
311 | : [=report/user agent=]
312 | :: The current value of navigator.userAgent
313 | : [=report/destination=]
314 | :: |destination|
315 | : [=report/type=]
316 | :: |type|
317 | : [=report/timestamp=]
318 | :: The current timestamp.
319 | : [=report/attempts=]
320 | :: 0
321 |
322 | 2. If |url| was not provided by the caller, let |url| be |settings|'s
323 | creation URL.
324 |
325 | 3. Set |url|'s {{URL/username}} to the empty string, and its {{URL/password}}
326 | to `null`.
327 |
328 | 4. Set |report|'s [=report/url=] to the result of executing the URL
329 | serializer on |url| with the exclude fragment flag set.
330 |
331 | 6. Return |report|.
332 |
333 | Note: reporting observers can only observe reports from the
334 | same environment settings object.
335 |
336 | Note: We strip the username, password, and fragment from the serialized URL
337 | in the report. See [[#capability-urls]].
338 |
339 | Note: The user agent MAY reject reports for any reason. This API does not
340 | guarantee delivery of arbitrary amounts of data, for instance.
341 |
342 | Note: Non user agent clients (with no JavaScript engine) should not interact
343 | with reporting observers, and thus should return in step 6.
344 |
345 |
346 | Serialize Reports
347 |
348 | To serialize a list of |reports| to JSON,
349 |
350 | 1. Let |collection| be an empty list.
351 |
352 | 2. For each |report| in |reports|:
353 |
354 | 1. Let |data| be a map with the following key/value pairs:
355 |
356 | : `age`
357 | :: The number of milliseconds between |report|'s [=report/timestamp=]
358 | and the current time.
359 | : `type`
360 | :: |report|'s [=report/type=]
361 | : `url`
362 | :: |report|'s [=report/url=]
363 | : `user_agent`
364 | :: |report|'s [=report/user agent=]
365 | : `body`
366 | :: |report|'s [=report/body=]
367 |
368 | Note: Client clocks are unreliable and subject to skew. We therefore
369 | deliver an `age` attribute rather than an absolute timestamp. See
370 | also [[#fingerprinting-clock-skew]]
371 |
372 | 2. Increment |report|'s [=report/attempts=].
373 |
374 | 3. Append |data| to |collection|.
375 |
376 | 3. Return the [=byte sequence=] resulting from executing [=serialize an Infra
377 | value to JSON bytes=] on |collection|.
378 |
379 |
380 |
381 | Document Centered Reporting
382 |
383 | This section defines the mechanism for configuring reporting endpoints for
384 | reports generated by actions in a document (or in a worker script). Such
385 | reports have a lifetime which is tied to that of the document or worker where
386 | they were generated.
387 |
388 | Document configuration
389 |
390 | Each object implementing {{WindowOrWorkerGlobalScope}} has an endpoints list, which
392 | is a list of endpoints, each of which MUST have a distinct
393 | {{endpoint/name}}. (Uniqueness is guaranteed by the algorithm in
394 | [[#process-header]].)
395 |
396 | Each object implementing {{WindowOrWorkerGlobalScope}} has an reports list, which is
398 | a list of reports.
399 |
400 | To initialize a global's endpoint list, given a
401 | {{WindowOrWorkerGlobalScope}} (|scope|) and a response (|response|),
402 | set |scope|'s endpoints to the result
403 | of executing [[#process-header]] given |response|.
404 |
405 |
406 |
407 | A server MAY define a set of reporting endpoints for a document or a worker
408 | script resource it returns, via the `Reporting-Endpoints` HTTP response
409 | header field. This mechanism is defined in [[#header]], and its processing in
410 | [[#process-header]].
411 |
412 | The value of the `Reporting-Endpoints` HTTP response header
413 | field is used to construct the reporting configuration for a resource.
414 |
415 | `Reporting-Endpoints` is a Dictionary Structured Field
416 | [[STRUCTURED-FIELDS]]. Each entry in the dictionary defines an
417 | endpoint to which reports may be delivered. The entry value MUST be a
418 | string.
419 |
420 | Each endpoint is defined by a String Item, which is interpreted as a
421 | URI-reference. If its value is not a valid URI-reference, that endpoint
422 | member MUST be ignored.
423 |
424 | Moreover, the URL that the member's value represents MUST be potentially
425 | trustworthy [[!SECURE-CONTEXTS]]. Non-secure endpoints will be ignored.
426 |
427 | No parameters are defined for endpoints, and any parameters
428 | which are specified will be silently ignored.
429 |
430 | The header is represented by the following ABNF grammar [[!RFC5234]]:
431 |
432 |
433 | Reporting-Endpoints = sf-dictionary
434 |
435 |
436 |
439 |
440 | Given a response (|response|), this algorithm extracts and returns a
441 | list of endpoints.
442 |
443 | 1. Abort these steps if |response|'s HTTPS
444 | state is not "`modern`", and the origin
445 | of |response|'s url is not potentially
446 | trustworthy.
447 |
448 | 2. Let |parsed header| be the result of executing [=get a structured field
449 | value=] given "Reporting-Endpoints" and "dictionary" from |response|'s header list.
451 |
452 | 3. If |parsed header| is null, abort these steps.
453 |
454 | 4. Let |endpoints| be an empty list.
455 |
456 | 5. For each |name| → |value_and_parameters| of |parsed header|:
457 |
458 | 1. Let |endpoint url string| be the first element of the tuple
459 | |value_and_parameters|. If |endpoint url string| is not a string,
460 | then continue.
461 |
462 | 2. Let |endpoint url| be the result of executing the URL parser
463 | on |endpoint url string|, with base URL set to
464 | |response|'s url. If |endpoint url| is
465 | failure, then continue.
466 |
467 | 3. If |endpoint url|'s origin is not potentially
468 | trustworthy, then continue.
469 |
470 | 4. Let |endpoint| be a new endpoint whose properties are set
471 | as follows:
472 |
473 | : {{endpoint/name}}
474 | :: |name|
475 | : {{endpoint/url}}
476 | :: |endpoint url|
477 | : {{endpoint/failures}}
478 | :: 0
479 |
480 | 5. Add |endpoint| to |endpoints|.
481 |
482 | 6. Return |endpoints|.
483 |
484 | Report Generation
485 |
486 | Generate report of |type| with
487 | |data|
488 |
489 | When the user agent is to generate and queue a report for a {{Document}}
490 | or {{WorkerGlobalScope}} object (|context|),
491 | given a string (|type|),
492 | a string (|destination|), and
493 | a serializable object (|data|),
494 | it must run the following steps:
495 |
496 | 1. Let |settings| be |context|'s [=relevant settings object=].
497 |
498 | 2. Let |report| be the result of running [=generate a report=] with |data|,
499 | |type|, |destination| and |settings|.
500 |
501 | 1. If |settings| is given, then
502 |
503 | 1. Let |scope| be |settings|'s [=environment settings object/global
504 | object=].
505 |
506 | 1. If |scope| is an object implementing {{WindowOrWorkerGlobalScope}},
507 | then execute [[#notify-observers]] with |scope| and |report|.
508 |
509 | 3. Append |report| to |context|'s [=reports=].
510 |
511 | Report Delivery
512 |
513 | Over time, various features will queue up a list of reports in
514 | documents and workers. The user agent will periodically grab the list of
515 | currently queued reports, and deliver them to the associated endpoints. This
516 | document does not define a schedule for the user agent to follow, and assumes
517 | that the user agent will have enough contextual information to deliver reports
518 | in a timely manner, balanced against impacting a user's experience.
519 |
520 | That said, a user agent SHOULD make an effort to deliver reports as soon as
521 | possible after queuing, as a report's data might be significantly more useful
522 | in the period directly after its generation than it would be a day or a week
523 | later.
524 |
525 | Send reports
526 |
527 | A user agent sends a list of reports (|reports|) for
528 | {{WindowOrWorkerGlobalScope}} object (|context|) by executing the following
529 | steps:
530 |
531 | 1. Let |endpoint map| be an empty map of endpoint objects to lists of
532 | report objects.
533 |
534 | 2. For each |report| in |reports|:
535 |
536 | 1. If there exists an endpoint (|endpoint|) in |context|'s
537 | [=WindowOrWorkerGlobalScope/endpoints=] list whose {{endpoint/name}}
538 | is |report|'s [=report/destination=]:
539 |
540 | 1. Append |report| to |endpoint map|'s list of reports for
541 | |endpoint|.
542 |
543 | 2. Otherwise, remove |report| from |reports|.
544 |
545 | 3. For each (|endpoint|, |report list|) pair in |endpoint map|:
546 |
547 | 1. Let |origin map| be an empty map of origins to
548 | lists of report objects.
549 |
550 | 2. For each |report| in |report list|:
551 |
552 | 1. Let |origin| be the origin of |report|'s [=report/url=].
553 |
554 | 2. Append |report| to |origin map|'s list of reports for |origin|.
555 |
556 | 3. For each (|origin|, |per-origin reports|) pair in |origin map|,
557 | execute the following steps asynchronously:
558 |
559 | 1. Let |result| be the result of executing [[#try-delivery]] on
560 | |endpoint|, |origin|, and |per-origin reports|.
561 |
562 | 2. If |result| is "`Failure`":
563 |
564 | 1. Increment |endpoint|'s {{endpoint/failures}}.
565 |
566 | 3. If |result| is "`Remove Endpoint`":
567 |
568 | 1. Remove |endpoint| from |context|'s
569 | [=WindowOrWorkerGlobalScope/endpoints=] list.
570 |
571 | 4. Remove each report from |reports|.
572 |
573 | ISSUE: We don't specify any retry mechanism here for failed reports.
574 | We may want to add one here, or provide some indication that the
575 | delivery failed.
576 |
577 | Note: User agents MAY decide to attempt delivery for only a subset of the
578 | collected reports or endpoints (because, for example, sending all the reports
579 | at once would consume an unreasonable amount of bandwidth, etc). As reports
580 | are only removed from the cache after delivery has been attempted, skipped
581 | reports will simply be delivered later.
582 |
583 |
584 | Attempt to deliver |reports| to |endpoint|
585 |
586 |
587 | Given an endpoint (|endpoint|), an origin
588 | (|origin|), and a list of reports (|reports|), this algorithm will
589 | construct a request, and attempt to deliver it to |endpoint|. It
590 | returns "`Success`" if that delivery succeeds, "`Remove Endpoint`" if the
591 | endpoint explicitly removes itself as a reporting endpoint by sending a 410
592 | response, and "`Failure`" otherwise.
593 |
594 |
595 | 1. Let |body| be the result of executing [=serialize a list of reports to
596 | JSON=] on |reports|.
597 |
598 | 2. Let |request| be a new request with the following properties
599 | [[FETCH]]:
600 |
601 | : `method`
602 | :: "`POST`"
603 | : `url`
604 | :: |endpoint|'s {{endpoint/url}}
605 | : `origin`
606 | :: |origin|
607 | : `header list`
608 | :: A new header list containing a header named
609 | \``Content-Type`\` whose value is \``application/reports+json`\`
610 | : `client`
611 | :: `null`
612 | : `window`
613 | :: "`no-window`"
614 | : `service-workers mode`
615 | :: "`none`"
616 | : `initiator`
617 | :: ""
618 | : `destination`
619 | :: "`report`"
620 | : `mode`
621 | :: "`cors`"
622 | : `unsafe-request` flag
623 | :: set
624 | : `credentials`
625 | :: "`same-origin`"
626 | : `body`
627 | :: A [=/body=] whose [=body/source=] is |body|.
628 |
629 | Note: Reports are sent with credentials set to `same-origin`. This allows
630 | reporting endpoints which are same-origin with the reporting page to get
631 | extra context about the nature of the report: for example, to understand
632 | whether a given user's account is triggering errors consistently, or if a
633 | certain sequence of actions taken on other pages is triggering a report on
634 | this page. This does not leak any new information to the reporting
635 | endpoint that it could not obtain in other ways. That is not the case for
636 | cross-origin reporting endpoints, so they do not receive credentials.
637 |
638 | 4. Queue a task to fetch |request|.
639 |
640 | 5. Wait for a response (|response|).
641 |
642 | 6. If |response|'s `status` is an OK status (200-299), return
643 | "`Success`".
644 |
645 | 7. If |response|'s `status` is `410 Gone` [[!RFC9110]], return "`Remove
646 | Endpoint`".
647 |
648 | 8. Return "`Failure`".
649 |
650 | Strip URL for use in reports
651 | To strip URL for use in reports given a [=/URL=] |url|,
652 | perform the following steps. They return a string representing the URL for use in reports.
653 |
654 | 1. If |url|'s scheme is not an HTTP(S) scheme,
655 | then return |url|'s scheme.
656 |
657 | 2. Set |url|’s fragment to the empty string.
658 |
659 | 3. Set |url|’s username to the empty string.
660 |
661 | 4. Set |url|’s password to the empty string.
662 |
663 | 5. Return the result of executing the URL serializer on |url|.
664 |
665 |
666 |
667 |
668 | Reporting Observers
669 |
670 | A reporting observer observes some types of reports from
671 | JavaScript, and is represented in JavaScript by the {{ReportingObserver}}
672 | object.
673 |
674 | Each object implementing {{WindowOrWorkerGlobalScope}} has a registered reporting observer list,
676 | which is an ordered set of reporting observers.
677 |
678 | Any reporting observer that is in a registered reporting observer
679 | list is considered registered.
680 |
681 | Each object implementing {{WindowOrWorkerGlobalScope}} has a report buffer, which is a list of reports that have been generated in that
684 | {{WindowOrWorkerGlobalScope}}. This list is initially empty, and the reports
685 | are stored in the same order in which they are generated.
686 |
687 | Note: The purpose of the report buffer is to allow reporting
688 | observers to observe reports that were generated earlier than that
689 | observer could be created (via the {{buffered}} option). For example, some
690 | reports might be generated during an earlier stage of page loading than when
691 | an observer could first be created, or before a JavaScript library is loaded
692 | that wishes to observe these reports.
693 |
694 | Note: Reporting observers are only relevant for user agents with
695 | JavaScript engines.
696 |
697 | Interface {{ReportingObserver}}
698 |
699 |
700 | [Exposed=(Window,Worker)]
701 | interface ReportBody {
702 | [Default] object toJSON();
703 | };
704 |
705 | [Exposed=(Window,Worker)]
706 | interface Report {
707 | [Default] object toJSON();
708 | readonly attribute DOMString type;
709 | readonly attribute DOMString url;
710 | readonly attribute ReportBody? body;
711 | };
712 |
713 | [Exposed=(Window,Worker)]
714 | interface ReportingObserver {
715 | constructor(ReportingObserverCallback callback, optional ReportingObserverOptions options = {});
716 | undefined observe();
717 | undefined disconnect();
718 | ReportList takeRecords();
719 | };
720 |
721 | callback ReportingObserverCallback = undefined (sequence<Report> reports, ReportingObserver observer);
722 |
723 | dictionary ReportingObserverOptions {
724 | sequence<DOMString> types;
725 | boolean buffered = false;
726 | };
727 |
728 | typedef sequence<Report> ReportList;
729 |
730 |
731 | A Report is the application exposed
732 | representation of a report. type
733 | returns [=report/type=], url returns
734 | [=report/url=], and body returns
735 | [=report/body=].
736 |
737 | Each {{ReportingObserver}} object has these associated concepts:
738 | - A callback function set on creation.
739 | - A {{ReportingObserverOptions}} dictionary called
740 | options.
741 | - A list of {{Report}} objects called the report
742 | queue, which is initially empty.
743 |
744 | A {{ReportList}} represents a sequence of {{Report}}s, providing developers
745 | with all the convenience methods found on JavaScript arrays.
746 |
747 | The
748 | ReportingObserver(|callback|, |options|) constructor, when
749 | invoked, must run these steps:
750 |
751 | 1. Create a new {{ReportingObserver}} object |observer|.
752 |
753 | 2. Set |observer|'s callback to |callback|.
754 |
755 | 3. Set |observer|'s options to |options|.
756 |
757 | 4. Return |observer|.
758 |
759 | The observe()
760 | method, when invoked, must run these steps:
761 |
762 | 1. Let |global| be the be the [=relevant global object=] of this.
763 |
764 | 2. Append this to the |global|'s registered reporting
765 | observer list.
766 |
767 | 3. If this's {{buffered}} option is false, return.
768 |
769 | 4. Set this's {{buffered}} option to false.
770 |
771 | 5. For each |report| in |global|'s report buffer, queue a task
772 | to execute [[#add-report]] with |report| and this.
773 |
774 | The disconnect() method,
775 | when invoked, must run these steps:
776 |
777 | 1. If this is not registered, return.
778 |
779 | 2. Let |global| be the [=relevant global object=] of this.
780 |
781 | 3. Remove this from |global|'s registered reporting
782 | observer list.
783 |
784 | The takeRecords() method,
785 | when invoked, must run these steps:
786 |
787 | 1. Let |reports| be a copy of this's report queue.
788 |
789 | 2. Empty this's report queue.
790 |
791 | 3. Return |reports|.
792 |
793 |
794 | Notify reporting observers on |scope| with |report|
795 |
796 |
797 | This algorithm makes |report|'s contents available to any registered
798 | reporting observers on the provided {{WindowOrWorkerGlobalScope}}.
799 |
800 | 1. For each {{ReportingObserver}} |observer| registered with |scope|,
801 | execute [[#add-report]] on |report| and |observer|.
802 |
803 | 2. Append |report| to |scope|'s report buffer.
804 |
805 | 3. Let |type| be |report|'s [=report/type=].
806 |
807 | 4. If |scope|'s report buffer now contains more than 100 reports with
808 | [=report/type=] equal to |type|, remove the earliest item with
809 | [=report/type=] equal to |type| in the report buffer.
810 |
811 |
812 | Add |report| to |observer|
813 |
814 |
815 | Given a report |report| and a {{ReportingObserver}} |observer|, this
816 | algorithm adds |report| to |observer|'s report queue, so long as
817 | |report|'s [=report/type=] is observable by |observer|.
818 |
819 | 1. If |report|'s [=report/type=] is not visible to
820 | ReportingObservers, return.
821 |
822 | 2. If |observer|'s options has a non-empty
823 | {{ReportingObserverOptions/types}} member which does not contain |report|'s
824 | [=report/type=], return.
825 |
826 | 3. Create a new {{Report}} |r| with {{Report/type}} initialized to |report|'s
827 | [=report/type=], {{Report/url}} initialized to |report|'s [=report/url=],
828 | and {{Report/body}} initialized to |report|'s [=report/body=].
829 |
830 | Issue: how to polymorphically initialize body?
831 |
832 | 3. Append |r| to |observer|'s report queue.
833 |
834 | 4. If the size of |observer|'s report queue is 1:
835 | 1. Let |global| be |observer|'s [=relevant global object=].
836 | 2. Queue a task to [[#invoke-observers]] with a copy of |global|'s
837 | registered reporting observer list.
838 |
839 |
840 | Invoke reporting observers with |notify list|
841 |
842 |
843 | This algorithm invokes observer callback functions for reports of previously
844 | observed behavior.
845 |
846 | 1. For each {{ReportingObserver}} |observer| in |notify list|:
847 |
848 | 1. If |observer|'s report queue is empty, then continue.
849 |
850 | 2. Let |reports| be a copy of |observer|'s report queue
851 |
852 | 3. Empty |observer|'s report queue
853 |
854 | 4. Invoke |observer|'s callback with
855 | « |reports|, |observer| » and "`report`", and with |observer| as the
856 | callback this value.
857 |
858 |
859 |
860 | Implementation Considerations
861 |
862 | Delivery
863 |
864 | The user agent SHOULD attempt to deliver reports as soon as possible to
865 | provide feedback to developers as quickly as possible. However, when this
866 | desire is balanced against the impact on the user, the user wins. With that
867 | in mind, the user agent MAY delay delivery of reports based on its knowledge
868 | of the user's activities and context.
869 |
870 | For instance, the user agent SHOULD prioritize the transmission of reporting
871 | data lower than other network traffic. The user's explicit activities on a
872 | website should preempt reporting traffic.
873 |
874 | The user agent MAY choose to withhold report delivery entirely until the user
875 | is on a fast, cheap network in order to prevent unnecessary data cost.
876 |
877 | The user agent MAY choose to prioritize reports from particular origins over
878 | others (perhaps those that the user visits most often?)
879 |
880 | Garbage Collection
881 |
882 | Periodically, the user agent SHOULD walk through the cached reports and endpoints, and discard those that are
884 | no longer relevant. These include:
885 |
886 | * endpoints whose {{endpoint/failures}} exceed
887 | some user-agent-defined threshold (~5 seems reasonable)
888 | * reports which have not been delivered in some arbitrary
889 | period of time (perhaps ~2 days?)
890 |
891 | For any reports that are discarded, these reports should
892 | also be removed from the report buffer of any reporting
893 | observer.
894 |
895 |
896 |
897 | Sample Reports
898 |
899 | *This section is non-normative.*
900 |
901 | This example shows the format in which reports are sent by the user agent to
902 | the reporting endpoint. The sample submission contains three reports which
903 | have been bundled together and sent in a single HTTP request. (The report
904 | types and bodies themselves are not intended to be representative of any
905 | actual feature, as those are outside of the scope of this specification).
906 |
907 |
908 |
909 | POST / HTTP/1.1
910 | Host: example.com
911 | ...
912 | Content-Type: application/reports+json
913 |
914 | [{
915 | "type": "security-violation",
916 | "age": 10,
917 | "url": "https://example.com/vulnerable-page/",
918 | "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
919 | "body": {
920 | "blocked": "https://evil.com/evil.js",
921 | "policy": "bad-behavior 'none'",
922 | "status": 200,
923 | "referrer": "https://evil.com/"
924 | }
925 | }, {
926 | "type": "certificate-issue",
927 | "age": 32,
928 | "url": "https://www.example.com/",
929 | "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
930 | "body": {
931 | "date-time": "2014-04-06T13:00:50Z",
932 | "hostname": "www.example.com",
933 | "port": 443,
934 | "effective-expiration-date": "2014-05-01T12:40:50Z",
935 | "served-certificate-chain": [
936 | "-----BEGIN CERTIFICATE-----\n
937 | MIIEBDCCAuygAwIBAgIDAjppMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT\n
938 | ...
939 | HFa9llF7b1cq26KqltyMdMKVvvBulRP/F/A8rLIQjcxz++iPAsbw+zOzlTvjwsto\n
940 | WHPbqCRiOwY1nQ2pM714A5AuTHhdUDqB1O6gyHA43LL5Z/qHQF1hwFGPa4NrzQU6\n
941 | yuGnBXj8ytqU0CwIPX4WecigUCAkVDNx\n
942 | -----END CERTIFICATE-----",
943 | ...
944 | ]
945 | }
946 | }, {
947 | "type": "cpu-on-fire",
948 | "age": 29,
949 | "url": "https://example.com/thing.js",
950 | "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
951 | "body": {
952 | "temperature": 614.0
953 | }
954 | }]
955 |
956 |
Security Considerations
1030 |
1031 | Capability URLs
1032 |
1033 | Some URLs are valuable in and of themselves. They may contain explicit
1034 | credentials in the username and password portion of the URL, or may grant
1035 | access to some resource to anyone with knowledge of the URL path.
1036 | Additionally, they may contain information which was never intended leave the
1037 | user's browser in the URL fragment. See [[CAPABILITY-URLS]] for more
1038 | information.
1039 |
1040 | To mitigate the possibility that such URLs will be leaked via this reporting
1041 | mechanism, the algorithms here strip out credential information and fragment
1042 | data from the URL sent as a report's originator. It is still possible,
1043 | however, for sensitive information in the URL's path to be leaked this way.
1044 | Sites which use such URLs may need to operate their own reporting endpoints.
1045 |
1046 | Additionally, such URLs may be present in a report's [=report/body=].
1047 | Specifications which extend this API and which include any URLs in a report's
1048 | [=report/body=] SHOULD require that they be similarly stripped.
1049 |
1050 |
1051 |
1052 | Privacy Considerations
1053 |
1054 | Network Leakage
1055 |
1056 | Because there is a delay between a page being loaded and a report being
1057 | generated and sent, it's entirely possible for a report generated while a user
1058 | is on one network to be sent while the user is on another network.
1059 |
1060 | This behaviour is limited to the lifetime of the document which generated the
1061 | reports, though, and such a document could be generating traffic on the new
1062 | network through other means in any case, even after the document is closed,
1063 | through mechanisms such as `navigator.sendBeacon`.
1064 |
1065 | ISSUE(WICG/background-sync#107): Consider mitigations. For example, we could
1066 | drop reports if we change from one network to another.
1067 |
1068 | Clock Skew
1069 |
1070 | Each report is delivered along with an `age` property, rather than the
1071 | timestamp at which it was generated. We do this because each user's local
1072 | clock will be skewed from the clock on the server by an arbitrary amount.
1073 | The difference between the time the report was generated and the time it
1074 | was sent will be stable, regardless of clock skew, and we can avoid the
1075 | fingerprinting risk of exposing the clock skew via this API.
1076 |
1077 | Cross-origin correlation
1078 |
1079 | If multiple origins all use the same reporting endpoint, that endpoint may
1080 | learn that a particular user has interacted with a certain set of websites,
1081 | as it will receive origin-tagged reports from each. This doesn't seem worse
1082 | than the status quo ability to track the same information from cooperative
1083 | origins, and doesn't grant any new tracking ability above and beyond what's
1084 | possible with `![]()
` today.
1085 |
1086 | Disabling Reporting
1087 |
1088 | Reporting is, to some extent, a question of commons. In the aggregate, it
1089 | seems useful for everyone for reports to be delivered. There is direct benefit
1090 | to developers, as they can fix bugs, which means there's indirect benefit to
1091 | users, as the sites they enjoy will be more stable and enjoyable. As a
1092 | concrete example, Content Security Policy grants something like herd immunity
1093 | to cross-site scripting attacks by alerting developers about potential holes
1094 | in their sites' defenses. Fixing those bugs helps every user, even those whose
1095 | user agents don't support Content Security Policy.
1096 |
1097 | The calculus, of course, depends on the nature of data that's being delivered,
1098 | and the relative maliciousness of the reporting endpoints, but that's the
1099 | value proposition in broad strokes.
1100 |
1101 | That said, it can't be the case that this general benefit be allowed to take
1102 | priority over the ability of a user to individually opt-out of such a system.
1103 | Sending reports costs bandwidth, and potentially could reveal some small
1104 | amount of additional information above and beyond what a website can obtain
1105 | in-band ([[NETWORK-ERROR-LOGGING]], for instance). User agents MUST allow
1106 | users to disable reporting with some reasonable amount of granularity in order
1107 | to maintain the priority of constituencies espoused in
1108 | [[HTML-DESIGN-PRINCIPLES]].
1109 |
1110 |
1111 |
1112 |
1113 | IANA Considerations
1114 |
1115 |
1116 |
1117 | The permanent message header field registry should be updated
1118 | with the following registration: [[!RFC3864]]
1119 |
1120 | : Header field name
1121 | :: `Reporting-Endpoints`
1122 | : Applicable protocol
1123 | :: http
1124 | : Status
1125 | :: standard
1126 | : Author/Change controller
1127 | :: W3C
1128 | : Specification document
1129 | :: This specification (see [[#header]])
1130 |
1131 |
1132 |
1133 | : Type name
1134 | :: application
1135 | : Subtype name
1136 | :: reports+json
1137 | : Required parameters
1138 | :: N/A
1139 | : Optional parameters
1140 | :: N/A
1141 | : Encoding considerations
1142 | :: Encoding considerations are identical to those specified for the "application/json" media type. See [[RFC8259]].
1143 | : Security considerations
1144 | :: See [[#security]].
1145 | : Interoperability considerations
1146 | :: This document specifies the format of conforming messages and the interpretation thereof.
1147 | : Published specification
1148 | :: [[#media-type]]
1149 | : Applications that use this media type
1150 | : Fragment identifier considerations
1151 | : Additional information
1152 | :: N/A
1153 | : Person and email address to contact for further information
1154 | :: This document's editors.
1155 | : Intended usage:
1156 | :: COMMON
1157 | : Restrictions on usage:
1158 | :: N/A
1159 | : Author
1160 | :: This document's editors.
1161 | : Change controller
1162 | :: W3C
1163 | : Provisional registration?
1164 | :: Yes.
1165 |
1166 |
--------------------------------------------------------------------------------
/network-reporting.bs:
--------------------------------------------------------------------------------
1 | Network Reporting API
2 |
3 | Status: ED
4 | ED: https://w3c.github.io/reporting/network-reporting
5 | Shortname: network-reporting
6 | Group: webperf
7 | Editor: Douglas Creager 103120, GitHub, dcreager@dcreager.net
8 | Editor: Ian Clelland 76841, Google Inc., iclelland@google.com
9 | Editor: Mike West 56384, Google Inc., mkwst@google.com
10 | Former Editor: Ilya Grigorik 56102, Google Inc., igrigorik@google.com
11 | Former Editor: Paul Meyer 99916, Google Inc., paulmeyer@google.com
12 | Abstract:
13 | This document extends the generic reporting framework from the Reporting API
14 | with a mechanisms for web developers to associate groups of reporting
15 | endpoints with origins they control, and defines how reports can reliably be
16 | sent to those endpoints, in order to be able to report on network conditions,
17 | or other concerns which exceed the scope of a single document, in a consistent
18 | manner.
19 | Level: 1
20 | Indent: 2
21 | Version History: https://github.com/w3c/reporting/commits/master/index.src.html
22 | Boilerplate: omit conformance
23 | Repository: https://github.com/w3c/reporting/
24 | Markup Shorthands: css off, markdown on
25 |
26 |
27 | spec: CSP; urlPrefix: https://w3c.github.io/webappsec-csp/
28 | type: dfn
29 | text: Content-Security-Policy
30 | text: reports directive; url: directives-reporting
31 | spec: FETCH; urlPrefix: https://fetch.spec.whatwg.org/
32 | type: dfn
33 | text: navigation request
34 | text: response; url: concept-response
35 | text: request; url: concept-request
36 | text: header; url: concept-header
37 | text: header list; url: concept-header-list
38 | text: main fetch
39 | text: fetch; url: concept-fetch
40 | text: wait for a response
41 | text: ok status
42 | type: attribute
43 | for: response
44 | text: url; url: concept-response-url
45 | text: HTTPS state; url: concept-response-https-state
46 | text: header list; url: concept-response-header-list
47 | for: request
48 | text: target browsing context; url: concept-request-target-browsing-context
49 | for: header
50 | text: name; url: concept-header-name
51 | text: value; url: concept-header-value
52 | spec: SECURE-CONTEXTS; urlPrefix: https://w3c.github.io/webappsec-secure-contexts/
53 | type: dfn
54 | text: potentially trustworthy; url: is-origin-trustworthy
55 | spec: URL; urlPrefix: https://url.spec.whatwg.org/
56 | type: dfn
57 | text: domain; url: concept-domain
58 | text: origin of a url; url: concept-url-origin
59 | text: URL serializer; url: concept-url-serializer
60 | text: URL parser; url: concept-url-parser
61 | type: interface
62 | text: URL; url: concept-url
63 | type: attribute
64 | for: URL
65 | text: username; url: concept-url-username
66 | text: password; url: concept-url-password
67 | spec: HTML; urlPrefix: https://html.spec.whatwg.org/multipage/
68 | urlPrefix: infrastructure.html
69 | type: dfn
70 | text: ASCII case-insensitive
71 | urlPrefix: webappapis.html
72 | type: dfn
73 | text: global object
74 | text: environment settings object
75 | text: creation URL
76 | text: queue a task
77 | urlPrefix: browsers.html
78 | type: dfn
79 | text: origin
80 | text: top-level browsing context
81 | urlPrefix: system-state.html
82 | type: dfn
83 | text: navigator.userAgent; url: dom-navigator-useragent
84 | spec: RFC2782; for: SRV; urlPrefix: https://tools.ietf.org/html/rfc2782
85 | type: dfn
86 | text: SRV record; url:
87 | text: target selection algorithm; url: page-4
88 | spec: RFC3986; urlPrefix: https://tools.ietf.org/html/rfc3986
89 | type: grammar
90 | text: absolute-uri; url: section-4.3
91 | spec: RFC6797; urlPrefix: https://tools.ietf.org/html/rfc6797
92 | type: dfn
93 | url: section-8.2
94 | text: superdomain match
95 | text: congruent match
96 | spec: RFC8259; urlPrefix: https://tools.ietf.org/html/rfc8259
97 | type: dfn
98 | text: JSON text; url: section-2
99 | spec: RFC7230; urlPrefix: https://tools.ietf.org/html/rfc7230
100 | type: grammar
101 | text: OWS; url: section-3.2.3
102 | text: BWS; url: section-3.2.3
103 | text: token; url: section-3.2.6
104 | text: quoted-string; url: section-3.2.6
105 | text: #rule; url: section-7
106 | spec: RFC7234; urlPrefix: https://tools.ietf.org/html/rfc7234
107 | type: grammar
108 | text: delta-seconds; url: section-1.2.1
109 | spec: RFC7469; urlPrefix: https://tools.ietf.org/html/rfc7469
110 | type: dfn
111 | text: Public-Key-Pins; url: section-2.1
112 | spec: HTTP-JFV; urlPrefix: https://tools.ietf.org/html/draft-reschke-http-jfv
113 | type: grammar
114 | text: json-field-value; url: section-2
115 | spec: ECMASCRIPT; urlPrefix: https://tc39.github.io/ecma262/
116 | type: dfn
117 | text: Realm
118 | text: Date object; url: sec-date-objects
119 | type: interface
120 | text: Date; url: sec-date-objects
121 | spec: webdriver; urlPrefix: https://w3c.github.io/webdriver/webdriver-spec.html#
122 | type: dfn
123 | text: current browsing context; url: dfn-current-browsing-context
124 | text: handle any user prompts; url: dfn-handle-any-user-prompts
125 | text: WebDriver error; url: dfn-error
126 | text: WebDriver error code; url: dfn-error-code
127 | text: extension command; url: dfn-extension-command
128 | text: extension command uri template; url: dfn-extension-command-uri-template
129 | text: invalid argument; url: dfn-invalid-argument
130 | text: no such window; url: dfn-no-such-window
131 | text: local end; url: dfn-local-end
132 | text: remote end steps; url: dfn-remote-end-steps
133 | text: session; url: dfn-session
134 | text: success; url: dfn-success
135 | text: trying; url: dfn-try
136 | spec: origin-policy; urlPrefix: https://wicg.github.io/origin-policy/
137 | type: dfn
138 | text: origin policy manifest; url: origin-policy-manifest
139 |
140 |
141 | {
142 | "SECURE-CONTEXTS": {
143 | "authors": [ "Mike West", "Yan Zhu" ],
144 | "href": "https://w3c.github.io/webappsec-secure-contexts/",
145 | "title": "Secure Contexts",
146 | "publisher": "W3C"
147 | }
148 | }
149 |
150 |
151 |
152 | Introduction
153 |
154 | This document extends the concepts defined in [[REPORTING]] to enable a class
155 | of reports which are not tied to the lifetime of any particular document. This
156 | enables network errors to be reported on, even (or especially) in cases where
157 | a document could not be loaded.
158 |
159 | Decoupling reports from documents implies two major differences from the
160 | document-centred reporting defined in [[REPORTING]]: First, configuration of
161 | reporting must be done at the origin level, rather than through document
162 | response headers. Second, the reports are queued and delivered by the user
163 | agent separately from document reports.
164 |
165 | Guarantees
166 |
167 | This specification aims to provide a best-effort report delivery system that
168 | executes out-of-band with website activity. The user agent will be able to do
169 | a better job prioritizing and scheduling delivery of reports, as it has an
170 | overview of cross-origin activity that individual websites do not, and can
171 | deliver reports based on error conditions that would prevent a website from
172 | loading in the first place.
173 |
174 | The delivery is not, however, guaranteed in a strict sense. We spell out a
175 | reasonable set of retry rules in the algorithms below, but it's quite possible
176 | for a report to be dropped on the floor if things go badly.
177 |
178 | Reporting can generate a good deal of traffic, so we allow developers to set
179 | up groups of endpoints, using a failover and
180 | load-balancing mechanism inspired by the DNS SRV record. The user
181 | agent will do its best to deliver a particular report to at most
182 | one endpoint in a group. Endpoints can be assigned weights to
183 | distribute load, with each endpoint receiving a specified fraction of
184 | reporting traffic. Endpoints can be assigned priorities, allowing developers
185 | to set up fallback collectors that are only tried when uploads to primary
186 | collectors fail.
187 |
188 | Examples
189 |
190 |
191 | MegaCorp Inc. wants to collect Network Error Log reports for its site.
192 | It can do so by serving an origin policy manifest with the following key,
193 | to define a set of reporting endpoints named "`endpoint-1`":
194 |
195 |
196 | {
197 | "network_reporting_endpoints": {
198 | "group": "endpoint-1",
199 | "max_age": 10886400,
200 | "endpoints": [
201 | { "url": "https://example.com/reports", "priority": 1 },
202 | { "url": "https://backup.com/reports", "priority": 2 }
203 | ] }
204 | }
205 |
206 |
207 | And the following headers, which direct NEL reports to that group:
208 |
209 |
210 | NEL: { ..., "report-to": "endpoint-1" }
211 |
212 |
Concepts
217 |
218 | Endpoint groups
219 |
220 | An endpoint group is a set of network reporting endpoints that will
222 | be used together for backup and failover purposes.
223 |
224 | Each endpoint group has a
225 | name, which is an ASCII
226 | string.
227 |
228 | Each endpoint group has an endpoints list, which is a list of
230 | network reporting endpoints.
231 |
232 | Each endpoint group has a subdomains flag, which is either "`include`" or
234 | "`exclude`".
235 |
236 | Each endpoint group has a ttl representing the number of seconds the group
238 | remains valid for an origin.
239 |
240 | Each endpoint group has a creation which is the timestamp at which the group was
242 | added to an origin.
243 |
244 | A endpoint group is expired if its {{endpoint group/creation}}
246 | plus its {{endpoint group/ttl}} represents a time in the past.
247 |
248 | Network reporting endpoints
249 |
250 | A network reporting endpoint is an endpoint, which
251 | is extended with these additional attributes:
252 |
253 | Each network reporting endpoint has a priority, which is a
255 | non-negative integer.
256 |
257 | Each network reporting endpoint has a weight, which is a
259 | non-negative integer.
260 |
261 | Each network reporting endpoint has a retry_after, which is
263 | either `null`, or a timestamp after which delivery should be retried.
264 |
265 | An network reporting endpoint is pending if its
267 | {{network reporting endpoint/retry_after}} is not `null`, and represents a
268 | time in the future.
269 |
270 | Clients
271 |
272 | A client represents a particular origin's relationship to
273 | a set of endpoints.
274 |
275 | Each client has an origin,
276 | which is an origin.
277 |
278 | Each client has an endpoint-groups list, which is a list of endpoint
280 | groups, each of which MUST have a distinct {{endpoint group/name}}.
281 | (The algorithm in [[#process-configuration]] guarantees this by keeping only
282 | the first entry in the configuration member with a particular name.)
283 |
284 | Failover and load balancing
285 |
286 | The network reporting endpoints in an endpoint group that all
287 | have the same {{network reporting endpoint/priority}} form a failover class. Failover classes allow the developer to
289 | provide backup collectors (those with higher
290 | {{network reporting endpoint/priority}} values) that will only receive reports
291 | if **all** of the primary collectors (those with lower
292 | {{network reporting endpoint/priority}} values) fail.
293 |
294 | Developers can assign each network reporting endpoint in a failover
295 | class a {{network reporting endpoint/weight}}, which determines how report
296 | traffic is balanced across the failover class.
297 |
298 | The algorithm that implements these rules is described in
299 | [[#choose-endpoint]].
300 |
301 | Note: The {{network reporting endpoint/priority}} and
302 | {{network reporting endpoint/weight}} fields have the same semantics as the
303 | corresponding fields in a DNS SRV record.
304 |
305 | Note: Failover and load balancing is a feature that would be generally useful
306 | outside of Reporting. Reporting delegates to the [[FETCH]] API to actually
307 | upload reports once an endpoint has been selected. If, in the future, the
308 | Fetch API adds native support for failover and load balancing of requests, a
309 | future version of this specification will be updated to use it instead of this
310 | bespoke mechanism.
311 |
312 | Storage
313 |
314 | A conformant user agent MUST provide a reporting cache, which
315 | is a storage mechanism that maintains a set of endpoint groups
316 | that websites have instructed the user agent to associate with their
317 | origins, and a set of reports which are queued for
318 | delivery.
319 |
320 | This storage mechanism is opaque, vendor-specific, and not exposed to the
321 | web, but it MUST provide the following methods which will be used in the
322 | algorithms this document defines:
323 |
324 | 1. Insert, update, and remove clients.
325 | 2. Enqueue and dequeue reports for delivery.
326 | 3. Retrieve a list of client objects for an origin.
327 | 4. Retrieve a list of queued report objects.
328 | 5. Clear the cache.
329 |
330 |
331 |
332 |
333 | Endpoint Delivery
334 |
335 | A server MAY define a set of endpoint groups for an origin it controls
336 | through an origin policy manifest [[!ORIGIN-POLICY]].
337 |
338 | Endpoint groups are specified with the `"network_reporting_endpoints"`
339 | member, which defines the endpoint groups to be associated with that
340 | origin.
341 |
342 | This member is defined in [[#network_reporting_endpoints-policy-item]], and
343 | its processing in [[#process-configuration]].
344 |
345 | The
346 | "network_reporting_endpoints" policy item
347 |
348 | The `network_reporting_endpoints` member defines the endpoint
349 | groups to be associated with the origin.
350 |
351 | If present, the member must be an array of objects.
352 |
353 | Each object in the array defines a endpoint group to which
354 | reports may be delivered, and will be parsed as defined in
355 | [[#process-configuration]].
356 |
357 | The following subsections define the set of known members which may be
358 | specified for each object in the array. Future versions of this document may
359 | define additional such members, and user agents MUST ignore unknown members
360 | when parsing the configuration.
361 |
362 | The `group` member
363 |
364 | The OPTIONAL `group` member is a
365 | string that associates a {{endpoint group/name}} with the endpoint
366 | group.
367 |
368 | If present, the member's value MUST be a string. If not present, the
369 | endpoint group will be given the {{endpoint group/name}}
370 | "`default`".
371 |
372 | The `include_subdomains` member
373 |
374 | The OPTIONAL `include_subdomains`
375 | member is a boolean that enables this endpoint group for all
376 | subdomains of the current origin's [=origin/host=].
377 |
378 | The `max_age` member
379 |
380 | The REQUIRED `max_age`
381 | member defines the endpoint group's lifetime, as a non-negative
382 | integer number of seconds.
383 |
384 | The member's value MUST be a non-negative number.
385 |
386 | A value of "`0`" will cause the endpoint group to be removed
387 | from the user agent's reporting cache.
388 |
389 | The `endpoints` member
390 |
391 | The REQUIRED `endpoints`
392 | member defines the list of endpoints that
393 | belong to this endpoint group.
394 |
395 | The member's value MUST be an array of JSON objects.
396 |
397 | The following subsections define the initial set of known members in each
398 | JSON object in the array. Future versions of this document may define
399 | additional such members, and user agents MUST ignore unknown members when
400 | parsing the elements of the array.
401 |
402 |
407 |
408 | The `endpoints.url` member
409 |
410 | The REQUIRED `url` member is a
411 | string that defines the location of the endpoint.
412 |
413 | The member's value MUST be a string. Moreover, the URL that the member's value
414 | represents MUST be potentially trustworthy [[!SECURE-CONTEXTS]].
415 | Non-secure endpoints will be ignored.
416 |
417 | The `endpoints.priority` member
418 |
419 | The OPTIONAL `priority` member is
420 | a number that defines which failover class the endpoint belongs
421 | to.
422 |
423 | The member's value, if present, MUST be a non-negative integer.
424 |
425 | The `endpoints.weight` member
426 |
427 | The OPTIONAL `weight` member is a
428 | number that defines load balancing for the failover class that the endpoint belongs to.
430 |
431 | The member's value, if present, MUST be a non-negative integer.
432 |
433 |
434 | Process origin policy configuration
435 |
436 |
437 | Given a map (|parsed|), and an origin (|origin|),
438 | this algorithm extracts a list of network reporting endpoints
439 | and endpoint groups for |origin|, and updates the reporting
440 | cache accordingly.
441 |
442 | Note: This algorithm is called from around step 9 of
443 | [[ORIGIN-POLICY#parse-a-string-into-an-origin-policy]], and only updates the
444 | reporting cache if the |response| has been delivered securely.
445 |
446 | ISSUE: Origin Policy monkey patching. Talk to Domenic.
447 |
448 | 1. Let |groups| be an empty list.
449 |
450 | 2. If |parsed|["network_reporting_endpoints"] exists and is a list, then for
451 | each |item| in |parsed|["network_reporting_endpoints"]:
452 |
453 | 1. If |item| has no member named "`max_age`", or that member's
455 | value is not a number, skip to the next |item|.
456 |
457 | 2. If |item| has no member named "`endpoints`", or that member's
459 | value is not an array, skip to the next |item|.
460 |
461 | 3. Let |name| be |item|'s "`group`" member's value if
463 | present, and "`default`" otherwise.
464 |
465 | 4. If there is already a endpoint group in |groups| whose
466 | {{endpoint group/name}} is |name|, skip to the next |item|.
467 |
468 | 5. Let |endpoints| be an empty list.
469 |
470 | 6. For each |endpoint item| in the value of |item|'s "`endpoints`" member:
472 |
473 | 1. If |endpoint item| has no member named "`url`", or that member's
475 | value is not a string, or if that value is not an
476 | absolute-URL string or a path-absolute-URL string,
477 | skip to the next |endpoint item|.
478 |
479 | 2. Let |endpoint url| be the result of executing the URL
480 | parser on |endpoint item|'s "`url`" member's value, with
482 | base URL set to |response|'s url. If |endpoint url| is failure, skip to the next
484 | |endpoint item|.
485 |
486 | 3. If |endpoint item| has a member named "`priority`", whose value is
488 | not a non-negative integer, skip to the next |endpoint item|.
489 |
490 | 4. If |endpoint item| has a member named "`weight`", whose value is
492 | not a non-negative integer, skip to the next |endpoint item|.
493 |
494 | 5. Let |endpoint| be a new network reporting endpoint whose
495 | properties are set as follows:
496 |
497 | : {{endpoint/name}}
498 | :: `null`
499 | : {{endpoint/url}}
500 | :: |endpoint url|
501 | : {{network reporting endpoint/priority}}
502 | :: The value of the |endpoint item|'s "`priority`" member, if
504 | present; `1` otherwise.
505 | : {{network reporting endpoint/weight}}
506 | :: The value of the |endpoint item|'s "`weight`" member, if
508 | present; `1` otherwise.
509 | : {{endpoint/failures}}
510 | :: 0
511 | : {{network reporting endpoint/retry_after}}
512 | :: `null`
513 |
514 | 5. Add |endpoint| to |endpoints|.
515 |
516 | 7. Let |group| be a new endpoint group whose properties
517 | are set as follows:
518 |
519 | : {{endpoint group/name}}
520 | :: |name|
521 | : {{endpoint group/subdomains}}
522 | :: "`include`" if |item| has a member named "`include_subdomains`" whose
524 | value is `true`, "`exclude`" otherwise.
525 | : {{endpoint group/ttl}}
526 | :: |item|'s "`max_age`"
527 | member's value.
528 | : {{endpoint group/creation}}
529 | :: The current timestamp
530 | : {{endpoint group/endpoints}}
531 | :: |endpoints|
532 |
533 | 8. Add |group| to |groups|.
534 |
535 | 3. Let |client| be a new client whose properties are set as follows:
536 |
537 | : {{client/origin}}
538 | :: |origin|
539 | : {{client/endpoint-groups}}
540 | :: |groups|
541 |
542 | 4. If there is already an entry in the reporting cache for |origin|,
543 | remove it.
544 |
545 | 5. Insert |client| into the reporting cache for |origin|.
546 |
547 |
548 |
549 |
550 | Report Generation
551 |
552 | Network reports can be generated with or without an active document. If a
553 | document is present, and can be considered the source of the report, then the
554 | report generated may be visible to reporting observers in that document.
555 |
556 | When a user agent is to generate a network report, given a
557 | string (|type|), another string (|endpoint group|), a serializable object
558 | (|data|), and an optional {{Document}} (|document|), it must run the
559 | following steps:
560 |
561 | 1. If |document| is given, then
562 |
563 | 1. Let |settings| be |document|'s [=environment settings object=].
564 |
565 | 2. Let |report| be the result of running [[REPORTING#queue-report]] with
566 | |data|, |type|, |endpoint group| and |settings|.
567 |
568 | 2. Otherwise, let |report| be the result of running
569 | [[REPORTING#queue-report]] with |data|, |type|, and |endpoint group|.
570 |
571 | 3. Append |report| to the reporting cache.
572 |
573 | Report Delivery
574 |
575 | Over time, various features will queue up a list of reports in the
576 | user agent's reporting cache. The user agent will periodically grab
577 | the list of currently pending reports, and deliver them to the associated
578 | endpoints. This document does not define a schedule for the user agent to
579 | follow, and assumes that the user agent will have enough contextual
580 | information to deliver reports in a timely manner, balanced against impacting
581 | a user's experience.
582 |
583 | That said, a user agent SHOULD make an effort to deliver reports as soon as
584 | possible after queuing, as a report's data might be significantly more useful
585 | in the period directly after its generation than it would be a day or a week
586 | later.
587 |
588 |
589 | Choose an |endpoint| from a |group|
590 |
591 |
592 | Note: This algorithm is the same as the target selection
593 | algorithm used for DNS SRV records.
594 |
595 | Given an endpoint group (|group|), this algorithm chooses an arbitrary
596 | eligible endpoint from the group, if there is one, taking into
597 | account the {{network reporting endpoint/priority}} and
598 | {{network reporting endpoint/weight}} of the endpoints.
600 |
601 | 1. Let |endpoints| be a copy of |group|'s {{endpoint group/endpoints}} list.
602 |
603 | 2. Remove every |endpoint| from |endpoints| that is
604 | pending.
605 |
606 | 3. If |endpoints| is empty, return `null`.
607 |
608 | 4. Let |priority| be the minimum {{network reporting endpoint/priority}}
609 | value of each |endpoint| in |endpoints|.
610 |
611 | 5. Remove every |endpoint| from |endpoints| whose
612 | {{network reporting endpoint/priority}} value is not equal to |priority|.
613 |
614 | 6. If |endpoints| is empty, return `null`.
615 |
616 | 7. Let |total weight| be the sum of the {{network reporting endpoint/weight}}
617 | value of each |endpoint| in |endpoints|.
618 |
619 | 8. Let |weight| be a random number ≥ 0 and ≤ |total weight|.
620 |
621 | 9. For each |endpoint| in |endpoints|:
622 |
623 | 1. If |weight| is less than or equal to |endpoint|'s
624 | {{network reporting endpoint/weight}}, return |endpoint|.
625 |
626 | 2. Subtract |endpoint|'s {{network reporting endpoint/weight}} from
627 | |weight|.
628 |
629 | 10. It should not be possible to fall through to here, since the random number
630 | chosen earlier will be less than or equal to |total weight|.
631 |
632 |
633 | Send reports
634 |
635 |
636 | A user agent sends reports by executing the following steps:
637 |
638 | 1. Let |reports| be a copy of the list of queued report objects in
639 | reporting cache.
640 |
641 | 2. Let |endpoint map| be an empty map of network reporting endpoint
642 | objects to lists of report objects.
643 |
644 | 3. For each |report| in |reports|:
645 |
646 | 1. Let |origin| be the origin of |report|'s [=report/url=].
647 |
648 | 2. Let |client| be the entry in the reporting cache for
649 | |origin|.
650 |
651 | 3. If there exists an endpoint group (|group|) in
652 | |client|'s {{client/endpoint-groups}} list whose
653 | {{endpoint group/name}} is |report|'s [=report/destination=]:
654 |
655 | 1. Let |endpoint| be the result of executing [[#choose-endpoint]] on
656 | |group|.
657 |
658 | 2. If |endpoint| is not `null`:
659 |
660 | 1. Append |report| to |endpoint map|'s list of reports for
661 | |endpoint|.
662 |
663 | 2. Skip to the next |report|.
664 |
665 | 4. If |origin| is a [=tuple origin=] whose [=origin/host=] is a
666 | [=domain=]:
667 |
668 | 1. For each |parent domain| that is a superdomain match
669 | for |origin|'s [=origin/host=] [[!RFC6797]], considering longer
670 | domains first:
671 |
672 | 1. Let |parent origin| be a copy of |origin|, with its
673 | [=origin/host=] replaced with |parent domain|.
674 |
675 | 2. Let |client| be the entry in the reporting cache for
676 | |parent origin|.
677 |
678 | 3. If there exists an endpoint group (|group|) in
679 | |client|'s {{client/endpoint-groups}} list whose {{endpoint
680 | group/name}} is |report|'s [=report/destination=] and
681 | whose {{endpoint group/subdomains}} flag is "`include`":
682 |
683 | 1. Let |endpoint| be the result of executing
684 | [[#choose-endpoint]] on |group|.
685 |
686 | 2. If |endpoint| is not `null`:
687 |
688 | 1. Append |report| to |endpoint map|'s list of reports
689 | for |endpoint|.
690 |
691 | 2. Skip to the next |report|.
692 |
693 | Note: This algorithm ensures that more specific {{endpoint
694 | group/subdomains}} policies take precendence over less specific ones,
695 | and that {{endpoint group/subdomains}} policies are ignored for any
696 | non-[=domain=] origins (e.g., for a request to a raw IP address).
697 |
698 | 5. If we reach this step, the |report| did not match any network
699 | reporting endpoint and the user agent MAY remove |report| from the
700 | reporting cache directly. Depending on load, the user agent MAY
701 | instead wait for [[#gc]] at some point in the future.
702 |
703 | 4. For each (|endpoint|, |reports|) pair in |endpoint map|:
704 |
705 | 1. Let |origin map| be an empty map of origins to
706 | lists of report objects.
707 |
708 | 2. For each |report| in |reports|:
709 |
710 | 1. Let |origin| be the origin of |report|'s [=report/url=].
711 |
712 | 2. Append |report| to |origin map|'s list of reports for |origin|.
713 |
714 | 3. For each (|origin|, |per-origin reports|) pair in |origin map|,
715 | execute the following steps asynchronously:
716 |
717 | 1. Let |result| be the result of executing [[REPORTING#try-delivery]]
718 | on |endpoint|, |origin|, and |per-origin reports|.
719 |
720 | 2. If |result| is "`Success`":
721 |
722 | 1. Set |endpoint|'s {{endpoint/failures}} to 0, and its
723 | {{network reporting endpoint/retry_after}} to `null`.
724 |
725 | 2. Remove each report in |reports| from the reporting
726 | cache.
727 |
728 | Otherwise, if |result| is "`Remove Endpoint`":
729 |
730 | 1. Remove |endpoint| from the reporting cache.
731 |
732 | Note: |reports| remain in the reporting cache for potential
733 | delivery to other endpoints.
734 |
735 | Otherwise (if |result| is "`Failure`"):
736 |
737 | 1. Increment |endpoint|'s {{endpoint/failures}}.
738 |
739 | 2. Set |endpoint|'s {{network reporting endpoint/retry_after}} to
740 | a point in the future which the user agent chooses.
741 |
742 | Note: We don't specify a particular algorithm here, but user
743 | agents are encouraged to employ some sort of exponential
744 | backoff algorithm which increases the retry period with the
745 | number of failures, with the addition of some random jitter to
746 | ensure that temporary failures don't lead to a crush of
747 | reports all being retried on the same schedule.
748 |
749 | ISSUE: Add in a reasonable reference describing a good
750 | algorithm. Wikipedia, if nothing else.
751 |
752 | Note: User agents MAY decide to attempt delivery for only a subset of the
753 | collected reports or endpoints (because, for example, sending all the reports
754 | at once would consume an unreasonable amount of bandwidth, etc). As reports
755 | are only removed from the cache when they're successfully delivered, skipped
756 | reports will simply be delivered later.
757 |
758 |
759 |
760 |
761 | Implementation Considerations
762 |
763 | Delivery
764 |
765 | The user agent SHOULD attempt to deliver reports as soon as possible to
766 | provide feedback to developers as quickly as possible. However, when this
767 | desire is balanced against the impact on the user, the user wins. With that
768 | in mind, the user agent MAY delay delivery of reports based on its knowledge
769 | of the user's activities and context.
770 |
771 | For instance, the user agent SHOULD prioritize the transmission of reporting
772 | data lower than other network traffic. The user's explicit activities on a
773 | website should preempt reporting traffic.
774 |
775 | The user agent MAY choose to withhold report delivery entirely until the user
776 | is on a fast, cheap network in order to prevent unnecessary data cost.
777 |
778 | The user agent MAY choose to prioritize reports from particular origins over
779 | others (perhaps those that the user visits most often?)
780 |
781 | Garbage Collection
782 |
783 | Periodically, the user agent SHOULD walk through the cached reports
784 | and endpoints, and discard those that are no
785 | longer relevant. These include:
786 |
787 | * endpoint groups which are expired.
788 | * endpoint groups which have not been used in some arbitrary
789 | period of time (perhaps a ~week?)
790 | * reports whose [=report/attempts=] exceed
791 | some user-agent-defined threshold (~5 seems reasonable.)
792 | * reports which have not been delivered in some arbitrary period of
793 | time (perhaps ~2 days?)
794 |
795 |
796 |
797 | Sample Reports
798 |
799 |
800 |
801 | POST / HTTP/1.1
802 | Host: example.com
803 | ...
804 | Content-Type: application/reports+json
805 |
806 | [{
807 | "type": "csp",
808 | "age": 10,
809 | "url": "https://example.com/vulnerable-page/",
810 | "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
811 | "body": {
812 | "blocked": "https://evil.com/evil.js",
813 | "directive": "script-src",
814 | "policy": "script-src 'self'; object-src 'none'",
815 | "status": 200,
816 | "referrer": "https://evil.com/"
817 | }
818 | }, {
819 | "type": "hpkp",
820 | "age": 32,
821 | "url": "https://www.example.com/",
822 | "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
823 | "body": {
824 | "date-time": "2014-04-06T13:00:50Z",
825 | "hostname": "www.example.com",
826 | "port": 443,
827 | "effective-expiration-date": "2014-05-01T12:40:50Z"
828 | "include-subdomains": false,
829 | "served-certificate-chain": [
830 | "-----BEGIN CERTIFICATE-----\n
831 | MIIEBDCCAuygAwIBAgIDAjppMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT\n
832 | ...
833 | HFa9llF7b1cq26KqltyMdMKVvvBulRP/F/A8rLIQjcxz++iPAsbw+zOzlTvjwsto\n
834 | WHPbqCRiOwY1nQ2pM714A5AuTHhdUDqB1O6gyHA43LL5Z/qHQF1hwFGPa4NrzQU6\n
835 | yuGnBXj8ytqU0CwIPX4WecigUCAkVDNx\n
836 | -----END CERTIFICATE-----",
837 | ...
838 | ]
839 | }
840 | }, {
841 | "type": "nel",
842 | "age": 29,
843 | "url": "https://example.com/thing.js",
844 | "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
845 | "body": {
846 | "referrer": "https://www.example.com/",
847 | "server-ip": "234.233.232.231",
848 | "protocol": "",
849 | "status-code": 0,
850 | "elapsed-time": 143,
851 | "age": 0,
852 | "type": "http.dns.name_not_resolved"
853 | }
854 | }]
855 |
856 |
Security Considerations
861 |
862 | Capability URLs
863 |
864 | Some URLs are valuable in and of themselves. To mitigate the possibility
865 | that such URLs will be leaked via this reporting mechanism, we strip out
866 | credential information and fragment data from the URL we store as a
867 | report's originator. It is still possible, however, for a feature
868 | to unintentionally leak such data via a report's [=report/body=]. Implementers
869 | SHOULD ensure that URLs contained in a report's body are similarly stripped.
870 |
871 |
872 |
873 | Privacy Considerations
874 |
875 | Network Leakage
876 |
877 | Because this reporting mechanism is out-of-band, and doesn't rely on a page
878 | being open, it's entirely possible for a report generated while a user is on
879 | one network to be sent while the user is on another network, even if they
880 | don't explicitly open the page from which the report was sent.
881 |
882 | ISSUE(w3c/BackgroundSync#107): Consider mitigations. For example, we could
883 | drop reports if we change from one network to another.
884 |
885 | Clock Skew
886 |
887 | Each report is delivered along with an `age` property, rather than the
888 | timestamp at which it was generated. We do this because each user's local
889 | clock will be skewed from the clock on the server by an arbitrary amount.
890 | The difference between the time the report was generated and the time it
891 | was sent will be stable, regardless of clock skew, and we can avoid the
892 | fingerprinting risk of exposing the clock skew via this API.
893 |
894 | Cross-origin correlation
895 |
896 | If multiple origins all use the same reporting endpoint, that endpoint may
897 | learn that a particular user has interacted with a certain set of websites,
898 | as it will receive origin-tagged reports from each. This doesn't seem worse
899 | than the status quo ability to track the same information from cooperative
900 | origins, and doesn't grant any new tracking ability above and beyond what's
901 | possible with `![]()
` today.
902 |
903 | Subdomains
904 |
905 | This specification allows any resource on a host to declare a set of reporting
906 | endpoints for that host and each of its subdomains. This doesn't have privacy
907 | implications in and of itself (beyond those noted in [[#clear-cache]]), as the
908 | reporting endpoints themselves don't take any real action, as features will
909 | need to opt-into using these reporting endpoints explicitly. Those features
910 | certainly will have privacy implications, and should carefully consider
911 | whether they should be enabled across origin boundaries.
912 |
913 | Clearing the reporting cache
914 |
915 | A user agent's reporting cache contains data about a user's activity
916 | on the web, and user agents ought to handle this data carefully. In
917 | particular, if a user agent gives users the ability to clear their site data,
918 | browsing history, browsing cache, or similar, the user agent MUST also clear
919 | the reporting cache. Note that this includes both the pending reports
920 | themselves, as well as the endpoints to which they would be sent. Both MUST
921 | be cleared.
922 |
923 | Disabling Reporting
924 |
925 | Reporting is, to some extent, a question of commons. In the aggregate, it
926 | seems useful for everyone for reports to be delivered. There is direct benefit
927 | to developers, as they can fix bugs, which means there's indirect benefit to
928 | users, as the sites they enjoy will be more stable and enjoyable. As a
929 | concrete example, Content Security Policy grants something like herd immunity
930 | to cross-site scripting attacks by alerting developers about potential holes
931 | in their sites' defenses. Fixing those bugs helps every user, even those whose
932 | user agents don't support Content Security Policy.
933 |
934 | The calculus, of course, depends on the nature of data that's being delivered,
935 | and the relative maliciousness of the reporting endpoints, but that's the
936 | value proposition in broad strokes.
937 |
938 | That said, it can't be the case that this general benefit be allowed to take
939 | priority over the ability of a user to individually opt-out of such a system.
940 | Sending reports costs bandwidth, and potentially could reveal some small
941 | amount of additional information above and beyond what a website can obtain
942 | in-band ([[NETWORK-ERROR-LOGGING]], for instance). User agents MUST allow
943 | users to disable reporting with some reasonable amount of granularity in order
944 | to maintain the priority of constituencies espoused in
945 | [[HTML-DESIGN-PRINCIPLES]].
946 |
947 |
--------------------------------------------------------------------------------
/security-and-privacy-questionnaire.md:
--------------------------------------------------------------------------------
1 | # [Self-Review Questionnaire: Security and Privacy](https://w3ctag.github.io/security-questionnaire/)
2 |
3 | For further explanation see [the full questionnaire](https://w3ctag.github.io/security-questionnaire/).
4 |
5 | 01. What information might this feature expose to Web sites or other parties,
6 | and for what purposes is that exposure necessary?
7 |
8 | The purpose of the Reporting API is to collect information about events occurring within a page,
9 | about failures related to performance, the site's development, security feature deployments, etc.,
10 | and delivering those to an endpoint. These are collected from users "in the wild", rather than
11 | from testers "in the lab", as users typically exercising more parts of a site than testing
12 | infrastructure does, with a variety of interactions and devices which the lab can't match.
13 |
14 | 02. Is this specification exposing the minimum amount of information necessary
15 | to power its features?
16 |
17 | Besides the user agent, and the url the user is interacting with, both of which are available to
18 | any running scripts on the page, this specification does not prescribe any particular other pieces
19 | of information to be included in a report. That is left to other specifications which integrate
20 | with this one, which define the structure of their reports and the events which trigger their
21 | sending.
22 |
23 | 03. How does this specification deal with personal information,
24 | personally-identifiable information (PII), or information derived from
25 | them?
26 |
27 | The Reporting API can potentially expose a user's IP address to a third-party server, though this
28 | is exactly the same as existing capabilities with subresource fetching. Since the IP or other
29 | network addressing data which was valid when a report was generated may be sensitive, or may
30 | identify the user or expose their behavior after they have chosen to switch to a different
31 | network connection, the specification suggests that those reports may be dropped if the network
32 | changes. It does yet not go as far as requiring this, as the current state is no different than
33 | other beacons.
34 |
35 | 04. How does this specification deal with sensitive information?
36 |
37 | See above, generally.
38 |
39 | 05. Does this specification introduce new state for an origin that persists
40 | across browsing sessions?
41 |
42 | The Reporting API does not. The portions of this specification which did that at one point have
43 | been split into a separate specification (Network Reporting), and have been removed from this one.
44 |
45 | 06. Does this specification expose information about the underlying
46 | platform to origins?
47 |
48 | No, the Reporting API does not. It would be possible for a feature integrating with Reporting to
49 | expose such information, but that would be an issue for that feature.
50 |
51 | 07. Does this specification allow an origin access to sensors on a user’s
52 | device?
53 |
54 | No.
55 |
56 | 08. What data does this specification expose to an origin? Please also
57 | document what data is identical to data exposed by other features, in the
58 | same or different contexts.
59 |
60 | The Reporting API on its own exposes a users IP address and User Agent string to reporting
61 | endpoints, within the report body. This is identical to data exposed through mechanisms such as
62 | subresource fetching, XHR, or `navigator.sendBeacon`. Other specifications which integrate with
63 | Reporting will add more information to specific reports, but they should be subject to their own
64 | privacy and security review.
65 |
66 | 09. Does this specification enable new script execution/loading mechanisms?
67 |
68 | No.
69 |
70 | 10. Does this specification allow an origin to access other devices?
71 |
72 | No.
73 |
74 | 11. Does this specification allow an origin some measure of control over a user
75 | agent's native UI?
76 |
77 | No.
78 |
79 | 12. What temporary identifiers might this specification create or expose
80 | to the web?
81 |
82 | None.
83 |
84 | 13. How does this specification distinguish between behavior in first-party and
85 | third-party contexts?
86 |
87 | The specification allows reporting endpoints to be declared which are third-party to the
88 | page which declares them. This allows an ecosystem of third-party report analysis services,
89 | and opens the possibility of report aggregators and anonymizers, which are separate from
90 | the sites where the reports are generated. In this scenario, the Reporting API insists
91 | that the remote endpoint follow the CORS protocol, and additionally, any credentials are
92 | stripped from the outgoing requests, so that users cannot be tracked across sites through
93 | a reporting endpoint.
94 |
95 | 14. How does this specification work in the context of a user agent’s Private
96 | Browsing or "incognito" mode?
97 |
98 | This is not covered by the specification at all. In practice, the mechanism will continue
99 | to operate in incognito mode, but there will be no credentials available to send, unless
100 | the user has chosen to sign in while in incognito. Additionally, reports generated from an
101 | incognito mode window will not be sent in the same request as those generated in other
102 | windows, even when they are destined for the same endpoint.
103 |
104 | 15. Does this specification have both "Security Considerations" and "Privacy
105 | Considerations" sections?
106 |
107 | Yes. See [Security Considerations](https://w3c.github.io/reporting/#security) and
108 | [Privacy Considerations](https://w3c.github.io/reporting/#privacy).
109 |
110 | 16. Does this specification allow downgrading default security characteristics?
111 |
112 | No.
113 |
114 | 17. What should this questionnaire have asked?
115 |
116 | ¯\\\_(ツ)\_/¯
117 |
--------------------------------------------------------------------------------
/tidyconfig.txt:
--------------------------------------------------------------------------------
1 | char-encoding: utf8
2 | indent: yes
3 | wrap: 80
4 | tidy-mark: no
5 | newline: LF
6 |
--------------------------------------------------------------------------------
/w3c.json:
--------------------------------------------------------------------------------
1 | {
2 | "group": 45211,
3 | "contacts": [
4 | "plehegar"
5 | ],
6 | "shortName": "reporting",
7 | "repo-type": "rec-track"
8 | }
--------------------------------------------------------------------------------