├── _includes
├── examples
│ └── placeholder.isl
├── callout.html
├── footer.html
├── header.html
├── note.html
├── head.html
├── grammar-element.md
├── example.md
└── grammar-2-0.txt
├── assets
├── favicon.ico
├── fonts
│ ├── FontAwesome.otf
│ ├── fontawesome-webfont.eot
│ ├── fontawesome-webfont.ttf
│ ├── fontawesome-webfont.woff
│ └── fontawesome-webfont.woff2
├── wasm_ion_schema_bg.wasm
├── wasm_ion_schema_bg.wasm.d.ts
├── main.scss
├── wasm_ion_schema.d.ts
├── ion-schema-widget.js
└── wasm_ion_schema.js
├── NOTICE
├── .gitmodules
├── .gitignore
├── .github
└── PULL_REQUEST_TEMPLATE.md
├── _sass
├── ionstyle.scss
├── _github-highlight.scss
├── _layout.scss
├── _callout.scss
└── _base.scss
├── _data
└── navigation.yml
├── CODE_OF_CONDUCT.md
├── news.md
├── _posts
├── 2018-10-29-ion-schema-spec-released.md
├── 2022-08-25-rfc-ion-schema-2_0-approved.md
├── 2022-05-28-ion-schema-rust-0_3_0-released.md
├── 2022-07-14-ion-schema-rust-0_4_0-released.md
├── 2021-11-29-ion-schema-kotlin-1_2_1-released.md
├── 2022-07-01-ion-schema-kotlin-1_3_0-released.md
├── 2018-11-05-ion-schema-kotlin-alpha-released.md
├── 2019-07-15-ion-schema-kotlin-1_0_0-released.md
├── 2022-09-12-web-schema-sandbox.md
├── 2020-07-14-ion-schema-kotlin-1_1-released.md
├── 2022-07-26-rfc-ion-schema-2_0-public-comment.md
├── 2022-09-13-ion-schema-rust-0_5_0-released.md
├── 2021-12-17-ion-schema-rust-0_1_0-released.md
└── 2023-06-07-launch-invalid-transitive-import-tool.md
├── _layouts
├── default.html
└── news_item.html
├── _config.yml
├── Gemfile
├── libs.md
├── docs
├── isl-2-0
│ └── bnf-grammar.md
├── index.md
├── cookbook
│ ├── ion-schema-schemas.md
│ ├── ignore-occurs-requirements.md
│ ├── logical-relationships.md
│ ├── sql-decimals.md
│ └── ion-schema-rust-getting-started.md
├── isl-versioning.md
├── implementing.md
└── process-for-changing-ion-schema.md
├── README.md
├── sandbox.md
├── index.md
├── CONTRIBUTING.md
├── Gemfile.lock
├── LICENSE
└── rfcs
└── ion_schema_2_0
├── ion_schema_2_0.md
└── language_versions.md
/_includes/examples/placeholder.isl:
--------------------------------------------------------------------------------
1 | $ion_schema_2_0
2 | type::{
3 | name: placeholder
4 | }
--------------------------------------------------------------------------------
/assets/favicon.ico:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/amazon-ion/ion-schema/HEAD/assets/favicon.ico
--------------------------------------------------------------------------------
/NOTICE:
--------------------------------------------------------------------------------
1 | Amazon Ion Schema
2 | Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
3 |
--------------------------------------------------------------------------------
/assets/fonts/FontAwesome.otf:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/amazon-ion/ion-schema/HEAD/assets/fonts/FontAwesome.otf
--------------------------------------------------------------------------------
/assets/wasm_ion_schema_bg.wasm:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/amazon-ion/ion-schema/HEAD/assets/wasm_ion_schema_bg.wasm
--------------------------------------------------------------------------------
/.gitmodules:
--------------------------------------------------------------------------------
1 | [submodule "assets/ace-builds"]
2 | path = assets/ace-builds
3 | url = https://github.com/ajaxorg/ace-builds.git
4 |
--------------------------------------------------------------------------------
/assets/fonts/fontawesome-webfont.eot:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/amazon-ion/ion-schema/HEAD/assets/fonts/fontawesome-webfont.eot
--------------------------------------------------------------------------------
/assets/fonts/fontawesome-webfont.ttf:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/amazon-ion/ion-schema/HEAD/assets/fonts/fontawesome-webfont.ttf
--------------------------------------------------------------------------------
/assets/fonts/fontawesome-webfont.woff:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/amazon-ion/ion-schema/HEAD/assets/fonts/fontawesome-webfont.woff
--------------------------------------------------------------------------------
/assets/fonts/fontawesome-webfont.woff2:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/amazon-ion/ion-schema/HEAD/assets/fonts/fontawesome-webfont.woff2
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
1 | ### Jekyll
2 | _site/
3 | .sass-cache/
4 | .jekyll-cache/
5 | .jekyll-metadata
6 | ### Intellij
7 | **/.idea/
8 | ### MacOS
9 | **/.DS_Store
10 |
--------------------------------------------------------------------------------
/.github/PULL_REQUEST_TEMPLATE.md:
--------------------------------------------------------------------------------
1 | *Issue #, if available:*
2 |
3 | *Description of changes:*
4 |
5 |
6 | By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
7 |
--------------------------------------------------------------------------------
/_sass/ionstyle.scss:
--------------------------------------------------------------------------------
1 | // Import partials.
2 | // Based off of 'whiteglass' theme, but modified substantially.
3 | // https://github.com/yous/whiteglass/
4 | @import
5 | "base",
6 | "layout",
7 | "github-highlight",
8 | "callout"
9 | ;
10 |
--------------------------------------------------------------------------------
/_data/navigation.yml:
--------------------------------------------------------------------------------
1 | main:
2 | - title: "Home"
3 | url: /
4 | - title: "Sandbox"
5 | url: /sandbox
6 | - title: "News"
7 | url: /news
8 | - title: "Docs"
9 | url: /docs
10 | - title: "Libs"
11 | url: /libs
12 |
--------------------------------------------------------------------------------
/_includes/callout.html:
--------------------------------------------------------------------------------
1 |
5 |
6 |
7 | {{include.content}}
8 |
9 |
--------------------------------------------------------------------------------
/CODE_OF_CONDUCT.md:
--------------------------------------------------------------------------------
1 | ## Code of Conduct
2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
4 | opensource-codeofconduct@amazon.com with any additional questions or comments.
5 |
--------------------------------------------------------------------------------
/news.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: News
3 | ---
4 |
5 | # {{ page.title }}
6 |
7 | {% for post in site.posts %}
8 |
9 | **{{ post.title }} **
10 | _{{post.date | date_to_long_string}}_
11 | {{post.excerpt}}
12 | Read more
13 | {% endfor %}
14 |
15 |
--------------------------------------------------------------------------------
/_posts/2018-10-29-ion-schema-spec-released.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "Ion Schema Specification 1.0 Released"
4 | date: 2018-10-29
5 | categories: news
6 | ---
7 | This new specification describes a language and set of constraints used to declaratively constrain Ion values.
8 |
9 | | [Ion Schema Specification]({{"/docs/spec.html" | relative_url}}) |
10 |
11 |
--------------------------------------------------------------------------------
/_posts/2022-08-25-rfc-ion-schema-2_0-approved.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "RFC: Ion Schema 2.0 - Approved"
4 | date: 2022-08-25
5 | categories: news ion-schema
6 | ---
7 |
8 | [RFC: Ion Schema 2.0](../rfcs/ion_schema_2_0/ion_schema_2_0.md) as been approved.
9 | A new version of the Ion Schema Specification that incorporates these changes will be published in the coming weeks.
10 |
--------------------------------------------------------------------------------
/_posts/2022-05-28-ion-schema-rust-0_3_0-released.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "Ion Schema Rust 0.3.0 Released"
4 | date: 2022-05-28
5 | categories: news ion-schema-rust
6 | ---
7 |
8 | Ion Schema Rust 0.3.0 is now available.
9 |
10 | | [Release Notes v0.3.0](https://github.com/amzn/ion-schema-rust/releases/tag/v0.3.0) | [Ion Schema Rust](https://github.com/amzn/ion-schema-rust) |
11 |
12 |
--------------------------------------------------------------------------------
/_posts/2022-07-14-ion-schema-rust-0_4_0-released.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "Ion Schema Rust 0.4.0 Released"
4 | date: 2022-07-14
5 | categories: news ion-schema-rust
6 | ---
7 |
8 | Ion Schema Rust 0.4.0 is now available.
9 |
10 | | [Release Notes v0.4.0](https://github.com/amzn/ion-schema-rust/releases/tag/v0.4.0) | [Ion Schema Rust](https://github.com/amzn/ion-schema-rust) |
11 |
12 |
--------------------------------------------------------------------------------
/_posts/2021-11-29-ion-schema-kotlin-1_2_1-released.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "Ion Schema Kotlin 1.2.1 Released"
4 | date: 2021-11-29
5 | categories: news ion-schema-kotlin
6 | ---
7 |
8 | Ion Schema Kotlin 1.2.1 is now available.
9 |
10 | | [Release Notes v1.2.1](https://github.com/amzn/ion-schema-kotlin/releases/tag/v1.2.1) | [Ion Schema Kotlin](https://github.com/amzn/ion-schema-kotlin) |
11 |
12 |
--------------------------------------------------------------------------------
/_posts/2022-07-01-ion-schema-kotlin-1_3_0-released.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "Ion Schema Kotlin 1.3.0 Released"
4 | date: 2022-07-01
5 | categories: news ion-schema-kotlin
6 | ---
7 |
8 | Ion Schema Kotlin 1.3.0 is now available.
9 |
10 | | [Release Notes v1.3.0](https://github.com/amzn/ion-schema-kotlin/releases/tag/v1.3.0) | [Ion Schema Kotlin](https://github.com/amzn/ion-schema-kotlin) |
11 |
12 |
--------------------------------------------------------------------------------
/_layouts/default.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 | {% include head.html %}
5 |
6 |
7 |
8 | {% include header.html %}
9 |
10 |
11 |
12 | {{ content }}
13 |
14 |
15 |
16 | {% include footer.html %}
17 |
18 |
19 |
20 |
21 |
--------------------------------------------------------------------------------
/_posts/2018-11-05-ion-schema-kotlin-alpha-released.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "Ion Schema Kotlin (alpha) Open Sourced"
4 | date: 2018-11-05
5 | categories: news
6 | ---
7 | A reference implementation of Ion Schema is now available as open source software. It is written in Kotlin, and should be considered alpha software.
8 |
9 | | [Ion Schema Kotlin](https://github.com/amzn/ion-schema-kotlin) |
10 |
11 |
--------------------------------------------------------------------------------
/_config.yml:
--------------------------------------------------------------------------------
1 | name: Ion Schema
2 | title: Amazon Ion Schema
3 | description: Amazon Ion Schema defines a grammar and constraints for narrowing the universe of Ion values.
4 | lang: en
5 | baseurl: /ion-schema
6 | markdown: kramdown
7 | kramdown:
8 | input: GFM
9 | toc_levels: "1,2"
10 | highlighter: rouge
11 | defaults:
12 | -
13 | scope:
14 | path: ""
15 | values:
16 | layout: default
17 | plugins:
18 | - jekyll-redirect-from
19 |
--------------------------------------------------------------------------------
/_includes/footer.html:
--------------------------------------------------------------------------------
1 |
13 |
--------------------------------------------------------------------------------
/_posts/2019-07-15-ion-schema-kotlin-1_0_0-released.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "Ion Schema Kotlin 1.0 Released"
4 | date: 2019-07-15
5 | categories: news
6 | ---
7 | This release is a complete implementation of the [Ion Schema Specification](https://amzn.github.io/ion-schema/docs/spec.html).
8 |
9 | | [Release Notes](https://github.com/amzn/ion-schema-kotlin/releases/tag/v1.0.0) | [Ion Schema Kotlin](https://github.com/amzn/ion-schema-kotlin) |
10 |
11 |
--------------------------------------------------------------------------------
/_posts/2022-09-12-web-schema-sandbox.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "New tool: Ion Schema Sandbox"
4 | date: 2022-09-12
5 | categories: news ion-schema
6 | ---
7 |
8 | A browser-based sandbox environment for Ion Schema has been added to the Ion Schema website.
9 |
10 | This sandbox can be used to validate Ion values for a particular type defined in Ion Schema.
11 | The sandbox is created using `ion-schema-rust` latest pre-release version.
12 |
13 | | [Ion Schema Sandbox]({{site.baseurl}}/sandbox) |
14 |
--------------------------------------------------------------------------------
/Gemfile:
--------------------------------------------------------------------------------
1 | # frozen_string_literal: true
2 |
3 | source "https://rubygems.org"
4 | ruby RUBY_VERSION
5 |
6 | git_source(:github) {|repo_name| "https://github.com/#{repo_name}" }
7 |
8 | # Hello! This is where you manage which Jekyll version is used to run.
9 | # When you want to use a different version, change it below, save the
10 | # file and run `bundle install`. Run Jekyll with `bundle exec`, like so:
11 | #
12 | # bundle exec jekyll serve
13 | gem "github-pages", group: :jekyll_plugins
14 | gem "webrick", "~> 1.8"
15 |
--------------------------------------------------------------------------------
/_posts/2020-07-14-ion-schema-kotlin-1_1-released.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "Ion Schema Kotlin 1.1 Released"
4 | date: 2020-07-14
5 | categories: news schema ion-schema-kotlin
6 | ---
7 | This release provides access to the ISL underlying Schema and Type objects, enables custom schema caching logic, provides graceful handling of redundant imports, and more.
8 |
9 | | [Release Notes](https://github.com/amzn/ion-schema-kotlin/releases/tag/v1.1.0) | [Ion Schema Kotlin](https://github.com/amzn/ion-schema-kotlin) |
10 |
11 |
--------------------------------------------------------------------------------
/_includes/header.html:
--------------------------------------------------------------------------------
1 |
--------------------------------------------------------------------------------
/_posts/2022-07-26-rfc-ion-schema-2_0-public-comment.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "RFC: Ion Schema 2.0 - Open for Comment"
4 | date: 2022-07-26
5 | categories: news ion-schema
6 | ---
7 |
8 | An RFC was published to the [Ion Schema](https://github.com/amzn/ion-schema) GitHub repository proposing a new major version of Ion Schema. Questions, comments, and suggestions are welcome and can be added to the ‘Conversation’ tab of the pull request or linked issues.
9 |
10 | The public comment period for this RFC is open until 21 August 2022.
11 |
12 | | [Ion Schema 2.0 RFC](https://github.com/amzn/ion-schema/pull/69) |
13 |
--------------------------------------------------------------------------------
/_posts/2022-09-13-ion-schema-rust-0_5_0-released.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "Ion Schema Rust 0.5.0 Released"
4 | date: 2022-09-13
5 | categories: news ion-schema-rust
6 | ---
7 |
8 | Ion Schema Rust 0.5.0 is now available.
9 | This beta release includes all the functionalities of Ion Schema as per the [Ion Schema 1.0 specification]({{site.baseurl}}/docs/isl-1-0/spec).
10 |
11 | | [Release Notes v0.5.0](https://github.com/amzn/ion-schema-rust/releases/tag/v0.5.0) | [Ion Schema Rust](https://github.com/amzn/ion-schema-rust) | [Getting started cookbook]({{site.baseurl}}/docs/cookbook/ion-schema-rust-getting-started)
12 |
13 |
--------------------------------------------------------------------------------
/libs.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: Libraries
3 | ---
4 |
5 | # {{ page.title }}
6 |
7 | | Name | Latest Version | Repository | Documentation |
8 | |------|----------------|------------|---------------|
9 | | ion-schema-kotlin | [1.8.0](https://github.com/amzn/ion-schema-kotlin/releases/latest) (Dec 15, 2023) | [Link](https://github.com/amzn/ion-schema-kotlin) | [Link](https://www.javadoc.io/doc/com.amazon.ion/ion-schema-kotlin/latest/index.html) |
10 | | ion-schema-rust | [0.15.0](https://github.com/amzn/ion-schema-rust/releases/latest) (Dec 13, 2024) | [Link](https://github.com/amzn/ion-schema-rust) | [Link](https://docs.rs/ion-schema/latest/ion_schema/)
11 |
--------------------------------------------------------------------------------
/docs/isl-2-0/bnf-grammar.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: Ion Schema 2.0 BNF-Style Grammar
3 | ---
4 | # {{page.title}}
5 |
6 | This grammar is intended as a learning aid and is _not_ authoritative.
7 |
8 | Some limitations of this grammar are that it cannot accurately represent open content, that it excludes equivalent encodings of an Ion value (e.g. the symbol `year` could also be `'year'`), that it does not describe a valid ISL regex string, and that it does not describe reserved words that cannot be used as a type name.
9 |
10 | {% capture grammar %}{% include grammar-2-0.txt %}{% endcapture %}
11 |
12 | {{ grammar | escape_once }}
13 |
--------------------------------------------------------------------------------
/_layouts/news_item.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 | {% include head.html %}
5 |
6 |
7 |
8 | {% include header.html %}
9 |
10 |
11 |
12 |
13 |
14 |
{{page.title}}
15 |
{{page.date | date_to_long_string}}
16 |
17 | {{ content }}
18 |
19 |
20 |
21 |
22 | {% include footer.html %}
23 |
24 |
25 |
26 |
27 |
--------------------------------------------------------------------------------
/_posts/2021-12-17-ion-schema-rust-0_1_0-released.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "Ion Schema Rust 0.1 Released"
4 | date: 2021-12-17
5 | categories: news
6 | ---
7 | This release is a partial implementation of the [Ion Schema Specification](https://amzn.github.io/ion-schema/docs/spec
8 | .html) which has the [logic constraints](https://amzn.github.io/ion-schema/docs/spec.html#logic-constraints) and
9 | [type constraint](https://amzn.github.io/ion-schema/docs/spec.html#type) implemented.
10 |
11 | | [Release Notes](https://github.com/amzn/ion-schema-rust/releases/tag/v0.1.0) | [Ion Schema Rust](https://github.com/amzn/ion-schema-rust) |
12 |
13 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # ion-schema
2 | Ion Schema specification and docs
3 |
4 | ## Development
5 |
6 | To test locally, you must have [Ruby](https://www.ruby-lang.org/en/documentation/installation/) and [Bundler](https://bundler.io/) installed.
7 |
8 | In the project root directory, run:
9 | ```shell
10 | bundle exec jekyll serve
11 | ```
12 |
13 | You can now view the GitHub pages site in your favorite browser, served from your computer.
14 |
15 | For full information about testing GitHub pages sites locally, see the [official documentation](https://docs.github.com/en/pages/setting-up-a-github-pages-site-with-jekyll/testing-your-github-pages-site-locally-with-jekyll) on GitHub.
16 |
--------------------------------------------------------------------------------
/_includes/note.html:
--------------------------------------------------------------------------------
1 |
4 | {% if include.type=="note" %}
5 | Note: {{include.content}}
6 | {% elsif include.type=="tip" %}
7 | Tip: {{include.content}}
8 | {% elsif include.type=="important" %}
9 | Important: {{include.content}}
10 | {% elsif include.type=="warning" %}
11 | Warning: {{include.content}}
12 | {% endif %}
--------------------------------------------------------------------------------
/docs/index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: Docs
3 | permalink: /docs/
4 | redirect_from:
5 | - /docs/spec
6 | - /docs/cookbook
7 | ---
8 |
9 | # {{ page.title }}
10 |
11 | ### Specifications
12 |
13 | * [Ion Schema Language Versioning](isl-versioning)
14 | * [Ion Schema 1.0 Specification](isl-1-0/spec)
15 | * [Ion Schema 2.0 Specification](isl-2-0/spec)
16 | * [Process for Changing Ion Schema](process-for-changing-ion-schema.md)
17 |
18 | ### Cookbooks
19 |
20 | * [Modeling Logical Relationships](cookbook/logical-relationships)
21 | * [Modeling SQL Decimals](cookbook/sql-decimals)
22 | * [Optionally ignoring the occurs requirement for fields](cookbook/ignore-occurs-requirements)
23 | * [Using Ion Schema Schemas](cookbook/ion-schema-schemas)
24 | * [Getting started with `ion-schema-rust`](cookbook/ion-schema-rust-getting-started)
25 |
26 | ### Other Information
27 |
28 | * [Considerations for Implementing Ion Schema](implementing)
29 |
--------------------------------------------------------------------------------
/_includes/head.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 | {% assign title = page.title | default: site.title | escape %}
6 | {% assign canonical = page.url | replace:'index.html','' | absolute_url %}
7 | {% assign description = page.description | default: page.excerpt | default: site.description | strip_html | normalize_whitespace | escape %}
8 |
9 | {{ page.title }}
10 |
11 | {% if page.keywords %}
12 | {% if page.keywords.first %}
13 | {% assign keywords = page.keywords | join: ',' %}
14 | {% else %}
15 | {% assign keywords = page.keywords %}
16 | {% endif %}
17 |
18 | {% endif %}
19 |
20 |
21 |
22 |
23 |
24 |
--------------------------------------------------------------------------------
/_includes/grammar-element.md:
--------------------------------------------------------------------------------
1 |
7 | {% assign productions = include.productions | upcase | split: "," %}
8 | {% capture grammar %}
9 | {% include grammar-2-0.txt %}
10 | {% endcapture %}
11 | {% assign grammar_lines = grammar | escape_once | newline_to_br | split: ' ' %}
12 |
13 |
14 |
15 | {%- for production in productions -%}
16 | {%- assign is_in_production = false -%}
17 | {%- assign production_start = "<PRODUCTION> ::=" | escape_once | replace: "PRODUCTION", production -%}
18 | {%- for grammar_line in grammar_lines -%}
19 | {%- assign grammar_line_stripped = grammar_line | strip -%}
20 | {%- if grammar_line contains production_start -%}{%- assign is_in_production = true -%}{%- endif -%}
21 | {%- if is_in_production -%}{{- grammar_line | escape_once -}}{%- endif -%}
22 | {%- if is_in_production and grammar_line_stripped == "" -%}{%- break -%}{%- endif -%}
23 | {%- endfor -%}
24 | {%- endfor -%}
25 |
26 |
27 |
--------------------------------------------------------------------------------
/_includes/example.md:
--------------------------------------------------------------------------------
1 |
13 | {% capture content %}
14 | {%- if include.markdown -%}{{include.markdown}}
15 | {% elsif include.code_file %}
16 | ```{{ include.lang | default: "" }}
17 | {% include {{ include.code_file }} %}
18 | ```
19 | {% else %}
20 | ```{{ include.lang | default: "" }}
21 | {{ include.code | strip }}
22 | ```
23 | {% endif %}
24 | {%- endcapture -%}
25 | {::options parse_block_html="true" /}
26 |
27 |
28 |
29 | **Example: {{ include.title }}**
30 |
31 | {{ content }}
32 |
33 |
34 |
35 | {::options parse_block_html="false" /}
--------------------------------------------------------------------------------
/assets/wasm_ion_schema_bg.wasm.d.ts:
--------------------------------------------------------------------------------
1 | /* tslint:disable */
2 | /* eslint-disable */
3 | export const memory: WebAssembly.Memory;
4 | export function __wbg_schemavalidationresult_free(a: number): void;
5 | export function schemavalidationresult_new(a: number, b: number, c: number, d: number, e: number, f: number, g: number): number;
6 | export function schemavalidationresult_result(a: number): number;
7 | export function schemavalidationresult_set_result(a: number, b: number): void;
8 | export function schemavalidationresult_value(a: number, b: number): void;
9 | export function schemavalidationresult_set_value(a: number, b: number, c: number): void;
10 | export function schemavalidationresult_error(a: number, b: number): void;
11 | export function schemavalidationresult_set_error(a: number, b: number, c: number): void;
12 | export function schemavalidationresult_has_error(a: number): number;
13 | export function schemavalidationresult_set_has_error(a: number, b: number): void;
14 | export function schemavalidationresult_violations(a: number): number;
15 | export function validate(a: number, b: number, c: number, d: number, e: number, f: number, g: number): number;
16 | export function __wbindgen_malloc(a: number): number;
17 | export function __wbindgen_realloc(a: number, b: number, c: number): number;
18 | export function __wbindgen_add_to_stack_pointer(a: number): number;
19 | export function __wbindgen_free(a: number, b: number): void;
20 |
--------------------------------------------------------------------------------
/sandbox.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: Give Ion Schema a Try!
3 | ---
4 |
5 | # {{ page.title }}
6 |
7 |
8 |
9 | {% include note.html type="note" content="This sandbox uses `ion-schema-rust` (pre-release version) to validate Ion value using given schema" %}
10 |
11 |
12 |
13 | Predefined examples:
14 |
15 |
84 | [1]: docs/spec.html
85 | [2]: news.html
86 | [3]: https://amzn.github.io/ion-docs/
87 |
88 |
--------------------------------------------------------------------------------
/_sass/_github-highlight.scss:
--------------------------------------------------------------------------------
1 | /***
2 | * Generated: rougify style github
3 | */
4 | .highlight table td { padding: 5px; }
5 | .highlight table pre { margin: 0; }
6 | .highlight {
7 | background-color: $background-pre;
8 |
9 | .cm { color: #999988; font-style: italic; }
10 | .cp { color: #999999; font-weight: bold; }
11 | .c1 { color: #999988; font-style: italic; }
12 | .cs { color: #999999; font-weight: bold; font-style: italic; }
13 | .c .cd { color: #999988; font-style: italic; }
14 | .err { color: #a61717; background-color: #e3d2d2; }
15 | .gd { color: #000000; background-color: #ffdddd; }
16 | .ge { color: #000000; font-style: italic; }
17 | .gr { color: #aa0000; }
18 | .gh { color: #999999; }
19 | .gi { color: #000000; background-color: #ddffdd; }
20 | .go { color: #888888; }
21 | .gp { color: #555555; }
22 | .gs { font-weight: bold; }
23 | .gu { color: #aaaaaa; }
24 | .gt { color: #aa0000; }
25 | .kc { color: #000000; font-weight: bold; }
26 | .kd { color: #000000; font-weight: bold; }
27 | .kn { color: #000000; font-weight: bold; }
28 | .kp { color: #000000; font-weight: bold; }
29 | .kr { color: #000000; font-weight: bold; }
30 | .kt { color: #445588; font-weight: bold; }
31 | .k .kv { color: #000000; font-weight: bold; }
32 | .mf { color: #009999; }
33 | .mh { color: #009999; }
34 | .il { color: #009999; }
35 | .mi { color: #009999; }
36 | .mo { color: #009999; }
37 | .m .mb .mx { color: #009999; }
38 | .sb { color: #d14; }
39 | .sc { color: #d14; }
40 | .sd { color: #d14; }
41 | .s2 { color: #d14; }
42 | .se { color: #d14; }
43 | .sh { color: #d14; }
44 | .si { color: #d14; }
45 | .sx { color: #d14; }
46 | .sr { color: #009926; }
47 | .s1 { color: #d14; }
48 | .ss { color: #990073; }
49 | .s { color: #d14; }
50 | .na { color: #008080; }
51 | .bp { color: #999999; }
52 | .nb { color: #0086B3; }
53 | .nc { color: #445588; font-weight: bold; }
54 | .no { color: #008080; }
55 | .nd { color: #3c5d5d; font-weight: bold; }
56 | .ni { color: #800080; }
57 | .ne { color: #990000; font-weight: bold; }
58 | .nf { color: #990000; font-weight: bold; }
59 | .nl { color: #990000; font-weight: bold; }
60 | .nn { color: #555555; }
61 | .nt { color: #000080; }
62 | .vc { color: #008080; }
63 | .vg { color: #008080; }
64 | .vi { color: #008080; }
65 | .nv { color: #008080; }
66 | .ow { color: #000000; font-weight: bold; }
67 | .o { color: #000000; font-weight: bold; }
68 | .w { color: #bbbbbb; }
69 | }
70 |
71 |
--------------------------------------------------------------------------------
/assets/main.scss:
--------------------------------------------------------------------------------
1 | ---
2 | # Only the main Sass file needs front matter (the dashes are enough)
3 | ---
4 | @charset "utf-8";
5 |
6 | // Our variables
7 | $base-font-family: "Apple SD Gothic Neo", AppleGothic, NanumBarunGothic, "Malgun Gothic", Dotum, sans-serif;
8 | $monospace-font-family: Monaco, Menlo, Consolas, "Courier New", DotumChe, monospace;
9 | $base-font-size: 16px;
10 | $base-font-weight: 400;
11 | $small-font-size: $base-font-size * 0.875;
12 | $base-line-height: 1.5;
13 |
14 | $spacing-unit: 30px;
15 |
16 | $text-color: #111;
17 | $background-color: #fdfdfd;
18 | $background-pre: #f0f0f0;
19 | $brand-color: #2568ba;
20 |
21 | $grey-color: #828282;
22 | $grey-color-light: lighten($grey-color, 40%);
23 | $grey-color-dark: darken($grey-color, 25%);
24 |
25 | // Width of the content area
26 | $content-width: 800px;
27 |
28 | $on-palm: 600px;
29 | $on-laptop: 800px;
30 |
31 |
32 | // ion schema sandbox styling
33 | #schema {
34 | background-color: $background-color;
35 | height: 200px;
36 | position: relative;
37 | }
38 | #value {
39 | background-color: $background-color;
40 | height: 100px;
41 | position: relative;
42 | }
43 | button {
44 | background-color: $background-color;
45 | border: 1px solid #d5d9d9;
46 | border-radius: 3px;
47 | box-sizing: border-box;
48 | color: #0f1111;
49 | cursor: pointer;
50 | display: inline-block;
51 | font-family: "Amazon Ember",sans-serif;
52 | font-size: 13px;
53 | line-height: 18px;
54 | position: relative;
55 | text-align: center;
56 | touch-action: manipulation;
57 | vertical-align: middle;
58 | width: 60px;
59 | }
60 | button:hover {
61 | background-color: #f7fafa;
62 | }
63 | button:focus {
64 | border-color: #008296;
65 | outline: 0;
66 | }
67 | #violation {
68 | border-left: 0px;
69 | background-color: $background-color;
70 | text-align: left;
71 | }
72 | #result {
73 | border-left: 0px;
74 | background-color: $background-color;
75 | }
76 | select {
77 | position: absolute;
78 | min-width: 160px;
79 | box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2);
80 | z-index: 1;
81 | }
82 | option {
83 | display: block;
84 | position: absolute;
85 | background-color: #f9f9f9;
86 | min-width: 160px;
87 | box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2);
88 | z-index: 1;
89 | color: black;
90 | padding: 12px 16px;
91 | text-decoration: none;
92 | }
93 |
94 | // whiteglass also includes a mixin for defining media queries.
95 | // Use media queries like this:
96 | // @include media-query($on-palm) {
97 | // .wrapper {
98 | // padding-right: $spacing-unit / 2;
99 | // padding-left: $spacing-unit / 2;
100 | // }
101 | // }
102 | @mixin media-query($device) {
103 | @media screen and (max-width: $device) {
104 | @content;
105 | }
106 | }
107 |
108 | // Import partials from the `ionstyle` theme.
109 | @import "ionstyle";
110 |
--------------------------------------------------------------------------------
/_sass/_layout.scss:
--------------------------------------------------------------------------------
1 | /**
2 | * Site header
3 | */
4 | .site-header {
5 | min-height: 56px;
6 |
7 | // Positioning context for the mobile navigation icon
8 | position: relative;
9 | }
10 |
11 | .site-title {
12 | font-size: 26px;
13 | font-weight: 300;
14 | line-height: 56px;
15 | letter-spacing: -1px;
16 | margin-bottom: 0;
17 | float: left;
18 |
19 | &,
20 | &:visited {
21 | color: $grey-color-dark;
22 | }
23 | }
24 |
25 | .site-nav {
26 | float: right;
27 | line-height: 56px;
28 |
29 | .page-link {
30 | color: $text-color;
31 | line-height: $base-line-height;
32 |
33 | // Gaps between nav items, but not on the last one
34 | &:not(:last-child) {
35 | margin-right: 20px;
36 | }
37 | }
38 |
39 | @include media-query($on-palm) {
40 | .page-link {
41 | padding: 20px 0;
42 |
43 | &:not(:last-child) {
44 | margin-right: 0;
45 | }
46 | margin-left: 20px;
47 | }
48 | }
49 | }
50 |
51 |
52 | /**
53 | * Site footer
54 | */
55 | .site-footer {
56 | padding: $spacing-unit 0;
57 | font-size: 15px;
58 | color: $grey-color;
59 | text-align: center;
60 | }
61 |
62 |
63 | /**
64 | * Page content
65 | */
66 | .page-content {
67 | padding: $spacing-unit 0;
68 |
69 | table {
70 | border-collapse: collapse;
71 | border: 1px solid rgb(234, 234, 234);
72 | }
73 |
74 | th {
75 | -webkit-background-clip: border-box;
76 | -webkit-background-origin: padding-box;
77 | -webkit-background-size: auto;
78 | -webkit-border-horizontal-spacing: 0px;
79 | -webkit-border-vertical-spacing: 0px;
80 | -webkit-box-shadow: rgba(255, 255, 255, 0.498039) 0 1px 0 0 inset;
81 | background-attachment: scroll;
82 | background-clip: border-box;
83 | background-color: rgba(0, 0, 0, 0);
84 | background-image: linear-gradient(rgb(248, 248, 248), rgb(238, 238, 238));
85 | background-origin: padding-box;
86 | background-size: auto;
87 | box-shadow: rgba(255, 255, 255, 0.498039) 0 1px 0 0 inset;
88 | border: 1px solid rgb(238, 238, 238);
89 | box-sizing: border-box;
90 | color: rgb(17, 17, 17);
91 | font-weight: bold;
92 | font-size: 90%;
93 | height: 33px;
94 | line-height: 19px;
95 | padding: .5em 1em;
96 | text-align: left;
97 | vertical-align: top;
98 | }
99 |
100 | td {
101 | -webkit-border-horizontal-spacing: 0px;
102 | -webkit-border-vertical-spacing: 0px;
103 | border-color: rgb(234, 234, 234);
104 | border-style: solid;
105 | border-width: 1px;
106 | border-collapse: collapse;
107 | box-sizing: border-box;
108 | color: rgb(17, 17, 17);
109 | height: 33px;
110 | line-height: 1.3em;
111 | padding: .5em 1em;
112 | vertical-align: top;
113 | }
114 |
115 | tr:nth-child(even) {
116 | background: rgb(234, 234, 234);
117 | }
118 |
119 | tr:nth-child(odd) {
120 | background: white;
121 | }
122 |
123 | pre {
124 | border-radius: .25em;
125 | position: relative;
126 | padding: 0 .5em 0 .5em;
127 | border-left: 0.4em #ddd solid;
128 | }
129 |
130 | blockquote {
131 | border-radius: .25em;
132 | border-left: 0.4em;
133 | line-height: 1.4em;
134 | font-size: 90%;
135 | }
136 | }
137 |
138 | .page-heading {
139 | font-size: 20px;
140 | }
141 |
--------------------------------------------------------------------------------
/assets/wasm_ion_schema.d.ts:
--------------------------------------------------------------------------------
1 | /* tslint:disable */
2 | /* eslint-disable */
3 | /**
4 | * @param {string} ion
5 | * @param {string} schema
6 | * @param {string} schema_type
7 | * @param {boolean} is_document
8 | * @returns {SchemaValidationResult}
9 | */
10 | export function validate(ion: string, schema: string, schema_type: string, is_document: boolean): SchemaValidationResult;
11 | /**
12 | */
13 | export class SchemaValidationResult {
14 | free(): void;
15 | /**
16 | * @param {boolean} r
17 | * @param {Array} v
18 | * @param {string} val
19 | * @param {boolean} has_error
20 | * @param {string} error
21 | */
22 | constructor(r: boolean, v: Array, val: string, has_error: boolean, error: string);
23 | /**
24 | * @returns {boolean}
25 | */
26 | result(): boolean;
27 | /**
28 | * @param {boolean} val
29 | */
30 | set_result(val: boolean): void;
31 | /**
32 | * @returns {string}
33 | */
34 | value(): string;
35 | /**
36 | * @param {string} val
37 | */
38 | set_value(val: string): void;
39 | /**
40 | * @returns {string}
41 | */
42 | error(): string;
43 | /**
44 | * @param {string} val
45 | */
46 | set_error(val: string): void;
47 | /**
48 | * @returns {boolean}
49 | */
50 | has_error(): boolean;
51 | /**
52 | * @param {boolean} val
53 | */
54 | set_has_error(val: boolean): void;
55 | /**
56 | * @returns {Array}
57 | */
58 | violations(): Array;
59 | }
60 |
61 | export type InitInput = RequestInfo | URL | Response | BufferSource | WebAssembly.Module;
62 |
63 | export interface InitOutput {
64 | readonly memory: WebAssembly.Memory;
65 | readonly __wbg_schemavalidationresult_free: (a: number) => void;
66 | readonly schemavalidationresult_new: (a: number, b: number, c: number, d: number, e: number, f: number, g: number) => number;
67 | readonly schemavalidationresult_result: (a: number) => number;
68 | readonly schemavalidationresult_set_result: (a: number, b: number) => void;
69 | readonly schemavalidationresult_value: (a: number, b: number) => void;
70 | readonly schemavalidationresult_set_value: (a: number, b: number, c: number) => void;
71 | readonly schemavalidationresult_error: (a: number, b: number) => void;
72 | readonly schemavalidationresult_set_error: (a: number, b: number, c: number) => void;
73 | readonly schemavalidationresult_has_error: (a: number) => number;
74 | readonly schemavalidationresult_set_has_error: (a: number, b: number) => void;
75 | readonly schemavalidationresult_violations: (a: number) => number;
76 | readonly validate: (a: number, b: number, c: number, d: number, e: number, f: number, g: number) => number;
77 | readonly __wbindgen_malloc: (a: number) => number;
78 | readonly __wbindgen_realloc: (a: number, b: number, c: number) => number;
79 | readonly __wbindgen_add_to_stack_pointer: (a: number) => number;
80 | readonly __wbindgen_free: (a: number, b: number) => void;
81 | }
82 |
83 | export type SyncInitInput = BufferSource | WebAssembly.Module;
84 | /**
85 | * Instantiates the given `module`, which can either be bytes or
86 | * a precompiled `WebAssembly.Module`.
87 | *
88 | * @param {SyncInitInput} module
89 | *
90 | * @returns {InitOutput}
91 | */
92 | export function initSync(module: SyncInitInput): InitOutput;
93 |
94 | /**
95 | * If `module_or_path` is {RequestInfo} or {URL}, makes a request and
96 | * for everything else, calls `WebAssembly.instantiate` directly.
97 | *
98 | * @param {InitInput | Promise} module_or_path
99 | *
100 | * @returns {Promise}
101 | */
102 | export default function init (module_or_path?: InitInput | Promise): Promise;
103 |
--------------------------------------------------------------------------------
/CONTRIBUTING.md:
--------------------------------------------------------------------------------
1 | # Contributing Guidelines
2 |
3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
4 | documentation, we greatly value feedback and contributions from our community.
5 |
6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
7 | information to effectively respond to your bug report or contribution.
8 |
9 |
10 | ## Reporting Bugs/Feature Requests
11 |
12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features.
13 |
14 | When filing an issue, please check [existing open](https://github.com/amzn/amazon-ion-schema/issues), or [recently closed](https://github.com/amzn/amazon-ion-schema/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
16 |
17 | * A reproducible test case or series of steps
18 | * The version of our code being used
19 | * Any modifications you've made relevant to the bug
20 | * Anything unusual about your environment or deployment
21 |
22 |
23 | ## Contributing via Pull Requests
24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
25 |
26 | 1. You are working against the latest source on the *master* branch.
27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
29 |
30 | To send us a pull request, please:
31 |
32 | 1. Fork the repository.
33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
34 | 3. Ensure local tests pass.
35 | 4. Commit to your fork using clear commit messages.
36 | 5. Send us a pull request, answering any default questions in the pull request interface.
37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
38 |
39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
41 |
42 |
43 | ## Finding contributions to work on
44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/amzn/amazon-ion-schema/labels/help%20wanted) issues is a great place to start.
45 |
46 |
47 | ## Code of Conduct
48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
50 | opensource-codeofconduct@amazon.com with any additional questions or comments.
51 |
52 |
53 | ## Security issue notifications
54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
55 |
56 |
57 | ## Licensing
58 |
59 | See the [LICENSE](https://github.com/amzn/amazon-ion-schema/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
60 |
61 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.
62 |
--------------------------------------------------------------------------------
/_sass/_callout.scss:
--------------------------------------------------------------------------------
1 | .bs-callout {
2 | padding: 20px;
3 | margin: 20px 0;
4 | border: 1px solid #eee;
5 | border-left-width: 5px;
6 | border-radius: 3px;
7 | }
8 | .bs-callout h4 {
9 | margin-top: 0;
10 | margin-bottom: 5px;
11 | }
12 | .bs-callout p:last-child {
13 | margin-bottom: 0;
14 | }
15 | .bs-callout code {
16 | border-radius: 3px;
17 | }
18 | .bs-callout+.bs-callout {
19 | margin-top: -5px;
20 | }
21 | .bs-callout-default {
22 | border-left-color: #777;
23 | }
24 | .bs-callout-default h4 {
25 | color: #777;
26 | }
27 | .bs-callout-primary {
28 | border-left-color: #428bca;
29 | }
30 | .bs-callout-primary h4 {
31 | color: #428bca;
32 | }
33 | .bs-callout-success {
34 | border-left-color: #5cb85c;
35 | }
36 | .bs-callout-success h4 {
37 | color: #5cb85c;
38 | }
39 | .bs-callout-danger {
40 | border-left-color: #d9534f;
41 | }
42 | .bs-callout-danger h4 {
43 | color: #d9534f;
44 | }
45 | .bs-callout-warning {
46 | border-left-color: #f0ad4e;
47 | }
48 | .bs-callout-warning h4 {
49 | color: #f0ad4e;
50 | }
51 | .bs-callout-info {
52 | border-left-color: #5bc0de;
53 | }
54 | .bs-callout-info h4 {
55 | color: #5bc0de;
56 | }
57 |
58 | .bs-callout-example {
59 | border-left-color: #ffcc33;
60 | }
61 | .bs-callout-example h4 {
62 | color: #ffcc33;
63 | }
64 | div.bs-callout > details > pre {
65 | border: 0;
66 | padding-top: 2px;
67 | padding-bottom: 2px;
68 | }
69 |
70 | .bs-callout-grammar {
71 | border-left-color: #777;
72 | }
73 | .bs-callout-grammar h4 {
74 | color: #777;
75 | }
76 | .bs-callout-grammar pre {
77 | background-color: white;
78 | border: 0;
79 | margin: 0;
80 | padding: 0;
81 | }
82 |
83 | // Alerts
84 | .alert, .callout {
85 | overflow: hidden;
86 | }
87 | .alert-warning {
88 | color: #444;
89 | }
90 | div.alert code, h2 code {
91 | background-color: transparent !important;
92 | }
93 | /* without this, the links in these notes aren't visible.*/
94 | .alert a {
95 | text-decoration: underline;
96 | }
97 | .note code, .alert code, .warning code, div#toc code, h2 code, h3 code, h4 code {
98 | color: inherit;
99 | padding: 0px;
100 | }
101 |
102 | .alert {
103 | margin-bottom:10px;
104 | margin-top:10px;
105 | }
106 |
107 |
108 | .alert {
109 | padding: 15px;
110 | margin-bottom: 20px;
111 | border: 1px solid transparent;
112 | border-radius: 4px
113 | }
114 |
115 | .alert h4 {
116 | margin-top: 0;
117 | color: inherit
118 | }
119 |
120 | .alert .alert-link {
121 | font-weight: 700
122 | }
123 |
124 | .alert>p,
125 | .alert>ul {
126 | margin-bottom: 0
127 | }
128 |
129 | .alert>p+p {
130 | margin-top: 5px
131 | }
132 |
133 | .alert-dismissable,
134 | .alert-dismissible {
135 | padding-right: 35px
136 | }
137 |
138 | .alert-dismissable .close,
139 | .alert-dismissible .close {
140 | position: relative;
141 | top: -2px;
142 | right: -21px;
143 | color: inherit
144 | }
145 |
146 | .alert-success {
147 | color: #3c763d;
148 | background-color: #dff0d8;
149 | border-color: #d6e9c6
150 | }
151 |
152 | .alert-success hr {
153 | border-top-color: #c9e2b3
154 | }
155 |
156 | .alert-success .alert-link {
157 | color: #2b542c
158 | }
159 |
160 | .alert-info {
161 | color: #31708f;
162 | background-color: #d9edf7;
163 | border-color: #bce8f1
164 | }
165 |
166 | .alert-info hr {
167 | border-top-color: #a6e1ec
168 | }
169 |
170 | .alert-info .alert-link {
171 | color: #245269
172 | }
173 |
174 | .alert-warning {
175 | color: #8a6d3b;
176 | background-color: #fcf8e3;
177 | border-color: #faebcc
178 | }
179 |
180 | .alert-warning hr {
181 | border-top-color: #f7e1b5
182 | }
183 |
184 | .alert-warning .alert-link {
185 | color: #66512c
186 | }
187 |
188 | .alert-danger {
189 | color: #a94442;
190 | background-color: #f2dede;
191 | border-color: #ebccd1
192 | }
193 |
194 | .alert-danger hr {
195 | border-top-color: #e4b9c0
196 | }
197 |
198 | .alert-danger .alert-link {
199 | color: #843534
200 | }
201 |
--------------------------------------------------------------------------------
/docs/cookbook/logical-relationships.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: Expressing logical relationships between fields
3 | ---
4 | # {{ page.title }}
5 | _(Applies to Ion Schema 1.0 and Ion Schema 2.0.)_
6 |
7 |
8 | Sometimes you may need to model logical relationships between fields. Consider the following:
9 | ```ion
10 | type::{
11 | name: Address,
12 | fields: {
13 | country: string,
14 | street: string,
15 | postal_code: string,
16 | zip: int,
17 | state: string,
18 | province: string,
19 | }
20 | }
21 | ```
22 | The fields `postal_code` and `province` would be used for Canadian addresses, whereas `state` and `zip` would be used for US addresses.
23 | An address having a `province` and a `state` or a `province` and a `zip` would not be valid.
24 |
25 | Sometimes these situations can be resolved by a careful rethinking of the data model.
26 | (In this case, one could create a field that represents a province _or_ state and another field that can represent a postal code _or_ zip code.)
27 | However, it is not always possible to modify the structure of the data if you already have data or if the data model is shared with other systems.
28 |
29 | In many cases, it is possible (even preferable) to treat the type as a union of other types.
30 | For example, `Address` is union of `USAddress` and `CanadianAddress`.
31 | ```ion
32 | type::{
33 | name: Address,
34 | one_of: [USAddress, CanadianAddress]
35 | }
36 | type::{
37 | name: USAddress,
38 | fields: {
39 | street: string,
40 | state: string,
41 | country: string,
42 | zip: int,
43 | }
44 | }
45 | type::{
46 | name: CanadianAddress,
47 | fields: {
48 | street: string,
49 | province: string,
50 | country: string,
51 | postal_code: string,
52 | }
53 | }
54 | ```
55 |
56 | ### Advanced cases
57 | If the logical relationships in your data model are not easily described as a union of other types, or you are having difficulty determining how to break down
58 | the sub-types of the union, you can follow these steps to express the relationships in ISL.
59 |
60 | 1. Write down the relationships as a propositional logic statement: `(zip ↔ state) AND (province ↔ postal_code) AND (state ↔ ~province)`
61 | 2. Convert the statement to [Disjunctive Normal Form](https://en.wikipedia.org/wiki/Disjunctive_normal_form): `(zip AND state AND ~province AND ~postal_code) OR (~zip AND ~state AND province AND postal_code)`
62 | 3. Convert each conjunction to an ISL inline type and combine in an `any_of` constraint.
63 |
64 | ```ion
65 | type::{
66 | name: Address,
67 | fields: {
68 | country: string,
69 | street: string,
70 | postal_code: string,
71 | zip: int,
72 | state: string,
73 | province: string,
74 | },
75 | any_of: [
76 | { fields: { zip: {occurs:required}, state: {occurs:required}, postal_code: nothing, province: nothing } },
77 | { fields: { zip: nothing, state: nothing, postal_code: {occurs:required}, province: {occurs:required} } }
78 | ]
79 | }
80 | ```
81 |
82 |
83 | Here are some more examples demonstrating how propositional logic can be represented as ISL.
84 |
85 | ```ion
86 | type::{
87 | name: a_or_b, // A V B
88 | type: struct,
89 | any_of: [
90 | { fields: { a: { occurs: required } } },
91 | { fields: { b: { occurs: required } } }
92 | ]
93 | }
94 |
95 | type::{
96 | name: a_implies_b, // A → B ≡ A' V B
97 | type: struct,
98 | any_of: [
99 | { fields: { a: nothing } },
100 | { fields: { b: { occurs: required } } }
101 | ]
102 | }
103 |
104 | type::{
105 | name: a_implies_not_b, // A → B' ≡ A' V B'
106 | type: struct,
107 | any_of: [
108 | { fields: { a: nothing } },
109 | { fields: { b: nothing } }
110 | ]
111 | }
112 |
113 | type::{
114 | name: a_iff_b, // A ↔ B ≡ (A Λ B) V (A' Λ B')
115 | type: struct,
116 | any_of: [
117 | { fields: { a: { occurs: required }, b: { occurs: required } } },
118 | { fields: { a: nothing, b: nothing } }
119 | ]
120 | }
121 |
122 | type::{
123 | name: a_xor_b, // A ↔ B' ≡ (A Λ B') V (A' Λ B)
124 | type: struct,
125 | any_of: [
126 | { fields: { a: { occurs: required } , b: nothing } },
127 | { fields: { a: nothing, b: { occurs: required } } },
128 | ]
129 | }
130 | ```
131 |
132 | This strategy can be applied just as easily to elements in a list.
133 |
134 | ```ion
135 | type::{
136 | name: contains_a_implies_contains_b,
137 | type: list,
138 | any_of: [
139 | { not: { contains: [A] } },
140 | { contains: [B] }
141 | ]
142 | }
143 | ```
144 |
--------------------------------------------------------------------------------
/_sass/_base.scss:
--------------------------------------------------------------------------------
1 | /**
2 | * Reset some basic elements
3 | */
4 | body, h1, h2, h3, h4, h5, h6,
5 | p, blockquote, pre, hr,
6 | dl, dd, ol, ul, figure {
7 | margin: 0;
8 | padding: 0;
9 | }
10 |
11 |
12 |
13 | /**
14 | * Basic styling
15 | */
16 | body {
17 | font: $base-font-weight #{$base-font-size}/#{$base-line-height} $base-font-family;
18 | color: $text-color;
19 | background-color: $background-color;
20 | -webkit-text-size-adjust: 100%;
21 | -webkit-font-feature-settings: "kern" 1;
22 | -moz-font-feature-settings: "kern" 1;
23 | -o-font-feature-settings: "kern" 1;
24 | font-feature-settings: "kern" 1;
25 | font-kerning: normal;
26 | }
27 |
28 | table {
29 | margin: 0 0 20px 0;
30 | text-align: left;
31 | }
32 |
33 | /**
34 | * Set `margin-bottom` to maintain vertical rhythm
35 | */
36 | h1, h2, h3, h4, h5, h6,
37 | p, blockquote, pre,
38 | ul, ol, dl, figure,
39 | %vertical-rhythm {
40 | margin-bottom: $spacing-unit / 2;
41 | }
42 |
43 |
44 |
45 | /**
46 | * Images
47 | */
48 | img {
49 | max-width: 100%;
50 | vertical-align: middle;
51 | }
52 |
53 |
54 |
55 | /**
56 | * Figures
57 | */
58 | figure > img {
59 | display: block;
60 | }
61 |
62 | figcaption {
63 | font-size: $small-font-size;
64 | }
65 |
66 |
67 |
68 | /**
69 | * Lists
70 | */
71 | ul, ol {
72 | margin-left: $spacing-unit;
73 | }
74 |
75 | li {
76 | > ul,
77 | > ol {
78 | margin-bottom: 0;
79 | }
80 | }
81 |
82 |
83 |
84 | /**
85 | * Headings
86 | */
87 | h1, h2, h3, h4, h5, h6 {
88 | font-weight: $base-font-weight+200;
89 |
90 | code {
91 | font-size: inherit;
92 | }
93 | }
94 |
95 |
96 |
97 | /**
98 | * Links
99 | */
100 | a {
101 | color: $brand-color;
102 | text-decoration: none;
103 |
104 | &:visited {
105 | color: darken($brand-color, 10%);
106 | }
107 |
108 | &:hover {
109 | color: $text-color;
110 | text-decoration: underline;
111 | }
112 | }
113 |
114 |
115 |
116 | /**
117 | * Blockquotes
118 | */
119 | blockquote {
120 | color: $grey-color;
121 | border-left: 4px solid $grey-color-light;
122 | padding: $spacing-unit / 2;
123 | font-size: 18px;
124 |
125 | > :last-child {
126 | margin-bottom: 0;
127 | }
128 | }
129 |
130 |
131 |
132 | /**
133 | * Rules
134 | */
135 | hr {
136 | height: 4px;
137 | margin: $spacing-unit / 2 0;
138 | border: 0;
139 | background-color: $grey-color-light;
140 | }
141 |
142 |
143 |
144 | /**
145 | * Code formatting
146 | */
147 | pre,
148 | code {
149 | font-family: $monospace-font-family;
150 | font-size: 13px;
151 | background-color: $background-pre;
152 | }
153 |
154 | code {
155 | padding: 1px 5px;
156 | }
157 |
158 | pre {
159 | padding: 8px 12px;
160 | overflow-x: auto;
161 |
162 | > code {
163 | border: 0;
164 | padding-right: 0;
165 | padding-left: 0;
166 | }
167 | }
168 |
169 | @media print {
170 | pre, code {
171 | white-space: pre-wrap;
172 | }
173 | }
174 |
175 | /*
176 | Snackbar
177 | */
178 | #snackbar {
179 | visibility: hidden;
180 | min-width: 250px;
181 | margin-left: -125px;
182 | background-color: #333;
183 | color: #fff;
184 | text-align: center;
185 | border-radius: 2px;
186 | padding: 16px;
187 | position: fixed;
188 | z-index: 1;
189 | left: 50%;
190 | bottom: 30px;
191 | font-size: 17px;
192 | }
193 | #snackbar.show {
194 | visibility: visible;
195 | -webkit-animation: fadein 0.5s, fadeout 0.5s 2.5s;
196 | animation: fadein 0.5s, fadeout 0.5s 2.5s;
197 | }
198 | @-webkit-keyframes fadein {
199 | from {bottom: 0; opacity: 0;}
200 | to {bottom: 30px; opacity: 1;}
201 | }
202 | @keyframes fadein {
203 | from {bottom: 0; opacity: 0;}
204 | to {bottom: 30px; opacity: 1;}
205 | }
206 | @-webkit-keyframes fadeout {
207 | from {bottom: 30px; opacity: 1;}
208 | to {bottom: 0; opacity: 0;}
209 | }
210 | @keyframes fadeout {
211 | from {bottom: 30px; opacity: 1;}
212 | to {bottom: 0; opacity: 0;}
213 | }
214 |
215 | /**
216 | * Wrapper
217 | */
218 | .wrapper {
219 | max-width: -webkit-calc(#{$content-width} - (#{$spacing-unit} * 2));
220 | max-width: calc(#{$content-width} - (#{$spacing-unit} * 2));
221 | margin-right: auto;
222 | margin-left: auto;
223 | padding-right: $spacing-unit;
224 | padding-left: $spacing-unit;
225 | @extend %clearfix;
226 |
227 | @include media-query($on-laptop) {
228 | max-width: -webkit-calc(#{$content-width} - (#{$spacing-unit}));
229 | max-width: calc(#{$content-width} - (#{$spacing-unit}));
230 | padding-right: $spacing-unit / 2;
231 | padding-left: $spacing-unit / 2;
232 | }
233 | }
234 |
235 |
236 |
237 | /**
238 | * Clearfix
239 | */
240 | %clearfix:after {
241 | content: "";
242 | display: table;
243 | clear: both;
244 | }
245 |
--------------------------------------------------------------------------------
/docs/isl-versioning.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: Ion Schema Language Versioning
3 | ---
4 | # {{ page.title }}
5 |
6 | ## Introduction
7 |
8 | The purpose of this document is to describe how the Ion Schema Language will be versioned. This is a summary of the versioning rules introduced in [RFC: Ion Schema Language Versioning](../rfcs/ion_schema_2_0/language_versions).
9 |
10 | *The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119).*
11 |
12 | ## Glossary
13 |
14 | * **Ion Schema Language (ISL):** the syntax, grammar, and set of constraints for validating Ion data, as well as the rules that govern how an Ion Schema implementation should interpret and apply schemas to Ion data.
15 | * **Ion Schema Language (ISL) version:** a specific version of the Ion Schema Language; synonymous with "Ion Schema version" and "language version"
16 | * **Ion Schema Specification:** the text that describes the Ion Schema Language.
17 | * **Schema Document**: A single stream of ion values that conforms to the Ion Schema Specification. (In IonJava terms, an IonDatagram that conforms to the spec.)
18 |
19 | ## What is versioned?
20 |
21 | * The *Ion Schema Language* SHALL be versioned.
22 | * The documents that describe the Ion Schema Language (the _Ion Schema Specification_) SHALL NOT have versioned releases.
23 |
24 | ## How is it versioned?
25 |
26 | * The Ion Schema Language version SHALL consist of a major and minor version components.
27 | * The major and minor version components SHALL be non-negative integers.
28 | * Within a schema document, the format SHALL be `$ion_schema__`.
29 | * In the specification text and other documentation, the format SHALL be `.`.
30 |
31 | _While this is similar to [Semantic Versioning](https://semver.org/), there are two significant differences—(1) Semantic Versioning includes patch versions, whereas Ion Schema Language versioning does not; and (2) Semantic Versioning releases are immutable, but the Ion Schema Language may be modified without changing the version number when the modification is an unsubstantive clarification or bugfix._
32 |
33 | ## Major Versions
34 |
35 | * A new major version MAY contain any sort of changes, including backwards incompatible changes.
36 | * Upon creating a new major version, the minor version component SHALL be reset to `0`.
37 |
38 | ## Minor Versions
39 |
40 | * Each new minor version of the Ion Schema Language SHALL allow any Ion Schema document that is valid against any previous minor version of the Ion Schema Language, within the same major version, to be updated to the new Specification version with equivalent semantics.
41 | * Such an update MUST only require changing the ISL version marker to the new minor version. For example, a valid Ion Schema 2.2 document, upon changing its ISL version marker to `$ion_schema_2_3`, SHALL be a valid Ion Schema 2.3 document, semantically equivalent to the original Ion Schema 2.2 document.
42 | * New minor versions of the Ion Schema Specification MUST be written to ensure this form of backward compatibility.
43 | * A change that qualifies as a minor version update MAY be released as a major version update as a way to signal to users that it is a particularly large or significant change.
44 |
45 | ## Cross-version Compatibility
46 |
47 | * A schema SHALL be allowed to import another schema that uses *any* other minor version in the same major version.
48 | * Any new major version of Ion Schema SHALL define its own rules regarding the ability to import schemas from prior major versions of Ion Schema.
49 | * A new major version SHOULD allow importing from the most recent prior major version unless there is a technical reason why it is not possible.
50 |
51 | ## Ion Schema Version Markers
52 |
53 | * The keyspace reserved for ISL version markers SHALL be the set of symbols that matches the regular expression `^\$ion_schema_\d.*$`.
54 | * A *valid* ISL version marker SHALL match the regular expression `^\$ion_schema_[1-9]\d*_(0|[1-9]\d*)$`.
55 | * An ISL Version Marker in the form `$ion_schema_X_Y` SHALL indicate that X and Y are the major and minor version respectively.
56 | * The first value in the schema SHOULD be an ISL Version Marker; if a top-level ISL value is encountered before encountering an ISL version marker, that schema SHALL use ISL 1.0.
57 |
58 | ## Requirements for Ion Schema Implementations
59 |
60 | * If symbol in the ISL Version Marker reserved keyspace is encountered and that symbol is not a valid ISL Version Marker, implementations MUST raise an error.
61 | * If an ISL Version Marker is encountered and that version marker refers to a version that is unknown to or unsupported by the implementation, that implementation MUST raise an error.
62 | * If more than one ISL Version Marker is found in the value stream of a single Ion Schema, the implementation MUST raise an error.
63 | * If an ISL Version Marker is encountered at any point after the header or a type definition, the implementation MUST raise an error.
64 | * If an implementation supports ISL version `X.Y`, then it MUST support all minor versions from `X.0` to `X.Y`.
65 |
--------------------------------------------------------------------------------
/docs/cookbook/sql-decimals.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: Modeling SQL Decimals in Ion Schema
3 | ---
4 | # {{ page.title }}
5 | _(Applies to Ion Schema 1.0 and Ion Schema 2.0.
6 | Example code uses Ion Schema 2.0; for Ion Schema 1.0, replace `exponent` with `scale` and negate the argument.)_
7 |
8 | Ion `decimal` and SQL `DECIMAL` have some fundamental differences that make it complex to compare values between the two types.
9 |
10 | A SQL `DECIMAL` is an exact numeric type, having a precision `P` and a scale `S`. `P` is a positive integer that
11 | determines the number of significant digits in a particular radix (base-10 for `DECIMAL`), and `S` is a non-negative
12 | integer. Every value of a `DECIMAL` type of scale `S` is of the form n × 10–S , where n is an integer such
13 | that –10P < n < 10P . (For more details about SQL numeric types, including `DECIMAL`, see SQL-92 §4.4.)
14 | Therefore, the precision of the number `12.34` is not determined by the actual number of digits, but (speaking informally)
15 | by the data type it is assigned. `12.34` can be assigned to `DECIMAL(38,2)` just as easily as `DECIMAL(4,2)`. That is to
16 | say, precision and scale are a property of the `DECIMAL` data _type_ rather than being inherent in the value.
17 |
18 | On the other hand, the [Ion decimal](https://amzn.github.io/ion-docs/docs/decimal.html) data type has an _exponent_ property instead of a scale property.
19 | Because an Ion decimal can have more than one encoding, the value of the exponent property is actually the _adjusted exponent_ as defined in the [IBM Hursley Lab General Decimal Arithmetic Specification](https://speleotrove.com/decimal/damodel.html):
20 |
21 | > The _adjusted exponent_ is the value of the exponent of a number when that number is expressed as though in scientific notation with one digit (non-zero unless the coefficient is 0) before any decimal point.
22 | > This is given by the value of the `exponent+(clength–1)`, where `clength` is the length of the coefficient in decimal digits.
23 |
24 | For example, the adjusted exponent of the values `0.00123d2` and `0.123d0` is the exponent of the adjusted form, `1.23d-1`.
25 | Informally, the exponent property is like the negation of scale, except that positive and negative numbers are valid exponents, but scale may only be non-negative.
26 | Finally, the Ion decimal _data type_ has no particular precision or exponent as it is an _arbitrary precision_ data type.
27 | Instead, the precision and exponent of an Ion decimal value are inherent in the value itself.
28 |
29 | As a result of these differences, when converting a value from a SQL `DECIMAL` to an Ion `decimal`, it is _possible_ to
30 | preserve the scale of the `DECIMAL` (as it becomes the exponent of the Ion decimal, multiplied by -1), but the precision
31 | is lost. When converting from an Ion `decimal` to a `DECIMAL(p,s)`, the precision and exponent of the Ion value are always
32 | lost because the value is converted into having the precision and scale of `DECIMAL(p,s)`.
33 |
34 | That being said, here are some questions that one might be trying to answer by modeling SQL `DECIMAL` and how to approach
35 | modeling them in Ion Schema.
36 |
37 | **Does `x` have the exact precision and the exact scale of `DECIMAL(p,s)`?**
38 |
39 | This is probably the least useful way to model a SQL `DECIMAL`, and is included here only to help explain why exact
40 | precision is not useful when modeling `DECIMAL` in Ion Schema.
41 |
42 | Using `DECIMAL(5,2)` as our example, we can model an _exact_ precision and scale like this:
43 | ```
44 | type::{
45 | precision: 5,
46 | exponent: -2,
47 | }
48 | ```
49 | This will only allow `decimal`s with 5 digits, and two after the decimal point. However, this will not accept valid
50 | `DECIMAL(5,2)` values such as `1.00`.
51 |
52 | **Does `x` have a compatible precision and the exact scale of `DECIMAL(p,s)`?**
53 |
54 | This way of modeling a SQL `DECIMAL` is useful for scenarios where the value was a SQL `DECIMAL(p,s)` before being
55 | converted to Ion.
56 |
57 | This can be separated into two parts:
58 | 1. Does `x` have exactly `s` digits after the decimal point?
59 | 2. Does `x` have less than or equal to `p` digits? (Or is `-10^(p-s) < x < 10^(p-s)`?)
60 |
61 |
62 | Using `DECIMAL(5,2)` as our example, we can model a _compatible_ precision and _exact_ scale like this:
63 | ```
64 | type::{
65 | precision: range::[min, 5],
66 | exponent: -2,
67 | }
68 | ```
69 | This will accept values such as `100.00`, `1.50`, and `0.01`. It will reject values such as `100`, `1.5`, `0.010`, and `1000.0`.
70 |
71 | Instead of using the `precision` constraint, we could also use `valid_values: range::[-999.99, 999.99]` to achieve the same result.
72 |
73 | **Does `x` fit in a `DECIMAL(p,s)` without having to round or truncate any digits after the decimal point?**
74 |
75 | This is probably the most useful way to model a SQL `DECIMAL` in Ion Schema. It can be used for validating that data
76 | _could_ have been a `DECIMAL(p,s)` before it was converted to Ion, and that the value could be converted to a
77 | `DECIMAL(p,s)` without any rounding or truncating.
78 |
79 | This can be separated into two parts:
80 | 1. Does `x` have less than or equal to `s` digits after the decimal point?
81 | 2. Is `-10^(p-s) < x < 10^(p-s)`?
82 |
83 | Using `DECIMAL(5,2)` as our example again, we can model a _compatible_ precision and scale like this:
84 | ```
85 | type::{
86 | exponent: [-2, max],
87 | // Exponent is not fixed, so we cannot use the `precision` constraint. Must use `valid_values` instead.
88 | valid_values: range::[ -999.99, 999.99 ],
89 | }
90 | ```
91 | This will accept values such as `1.5`, `1.50`, `5d2`, and `500`. It will reject values such as `1200`, `1.2d5`, `1.501`,
92 | and `1.500`.
93 |
--------------------------------------------------------------------------------
/_includes/grammar-2-0.txt:
--------------------------------------------------------------------------------
1 | ::= $ion_schema_2_0
2 |
3 | ::= [] ... []
4 |
5 | ::= schema_header::{ ... }
6 |
7 | ::=
8 | |
9 |
10 | ::= imports: [ ... ],
11 |
12 | ::=
13 | |
14 | |
15 |
16 | ::= { id: }
17 |
18 | ::= { id: , type: }
19 |
20 | ::= { id: , type: , as: }
21 |
22 | ::= user_reserved_fields: { ... }
23 |
24 | ::= schema_header: [ ... ],
25 | | type: [ ... ],
26 | | schema_footer: [ ... ],
27 |
28 | ::= schema_footer::{ }
29 |
30 | ::= type::{ name: , ... }
31 |
32 | ::= { ... }
33 |
34 | ::=
35 | |
36 |
37 | ::=
38 |
39 | ::=
40 | | $null_or::
41 | |
42 | | $null_or::
43 | |
44 | | $null_or::
45 |
46 | ::= occurs:
47 | | occurs:
48 | | occurs: optional
49 | | occurs: required
50 |
51 | ::= { , ... }
52 | |
53 |
54 | ::=
55 | |
56 | |
57 |
58 | ::= exclusive::
59 | | ""
60 |
61 | ::= range::[ , ]
62 | | range::[ min, ]
63 | | range::[ , max ]
64 |
65 | ::= range::[ , ]
66 | | range::[ min, ]
67 | | range::[ , max ]
68 |
69 | ::= range::[ , ]
70 | | range::[ min, ]
71 | | range::[ , max ]
72 |
73 | ::= range::[ , ]
74 | | range::[ min, ]
75 | | range::[ , max ]
76 |
77 | ::=
78 | |
79 | |
80 | |
81 | |
82 | |
83 | |
84 | |
85 | |
86 | |
87 | |
88 | |
89 | |
90 | |
91 | |
92 | |
93 | |
94 | |
95 | |
96 | |
97 | |
98 | |
99 |
100 | ::= all_of: [ ... ]
101 |
102 | ::= required::
103 | | closed::
104 |
105 | ::= annotations: ... [ ... ]
106 | | annotations:
107 |
108 | ::= any_of: [ ... ]
109 |
110 | ::= byte_length:
111 | | byte_length:
112 |
113 | ::= codepoint_length:
114 | | codepoint_length:
115 |
116 | ::= container_length:
117 | | container_length:
118 |
119 | ::= contains: [ ... ]
120 |
121 | ::= element:
122 | | element: distinct::
123 |
124 | ::= exponent:
125 | | exponent:
126 |
127 | ::= :
128 |
129 | ::= fields: { ... }
130 | | fields: closed::{ ... }
131 |
132 | ::= field_names:
133 | | field_names: distinct::
134 |
135 | ::= ieee754_float: binary16
136 | | ieee754_float: binary32
137 | | ieee754_float: binary64
138 |
139 | ::= not:
140 |
141 | ::= one_of: [ ... ]
142 |
143 | ::= ordered_elements: [ ... ]
144 |
145 | ::= precision:
146 | | precision:
147 |
148 | ::= regex:
149 | | regex: i::
150 | | regex: m::
151 | | regex: i::m::
152 |
153 | ::= timestamp_offset: [ "[+|-]hh:mm"... ]
154 |
155 | ::= year
156 | | month
157 | | day
158 | | minute
159 | | second
160 | | millisecond
161 | | microsecond
162 | | nanosecond
163 |
164 | ::= timestamp_precision:
165 | | timestamp_precision:
166 |
167 | ::= type:
168 |
169 | ::= utf8_byte_length:
170 | | utf8_byte_length:
171 |
172 | ::= valid_values: [ ... ]
173 | | valid_values:
174 | | valid_values:
175 |
176 | ::=
177 | |
178 | |
--------------------------------------------------------------------------------
/docs/implementing.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: Implementation Considerations
3 | ---
4 | # {{ page.title }}
5 |
6 | This is a collection of thoughts and learnings that were gathered over the course of creating and maintaining `ion-schema-kotlin` and `ion-schema-rust`.
7 | They have been collected here in the hope that they will be useful for anyone creating a new implementation of Ion Schema or considering writing a related tool or library.
8 | This document is not an authoritative document, and implementers are not required to follow any of the suggestions presented here.
9 |
10 | ## General Notes
11 |
12 | - When a value is not valid for a particular type, it is generally helpful to surface all the reasons the value isn't valid, rather than just the first reason.
13 | - Runtime resolution of a schema over a network presents availability and security risks, and should therefore be avoided.
14 | - Consider when warnings should be issued instead of errors (e.g., if a schema authority is configured to reference a URL, or a type is defined such that there are no valid values for it).
15 | - As ISL itself allows for open content, consider providing an API to make that content available to callers.
16 |
17 | ## Modeling the Ion Schema Language
18 |
19 | The only model that is required by the specification is the Ion Schema Language itself.
20 | The original Kotlin implementation of Ion Schema only operated on the ISL Ion.
21 | Consequently, the parsing and evaluation of a schema were tightly coupled, making it more difficult to support multiple versions of ISL.
22 | Furthermore, programmatic manipulation of schemas was difficult because clients of the library had to manipulate the Ion DOM (or even Ion text) in order to construct or modify a schema.
23 | Therefore, it is highly recommended to implement at least one of ISL Data Objects or an internal AST.
24 |
25 | ### ISL Data Objects
26 |
27 | * Should be idiomatic for the language in which it is written.
28 | * Should not be modeled using an Ion DOM or require Ion DOM values for construction, except for constraints such as `valid_values` and `contains`, where the constraint is a check for a match between Ion values.
29 | * Support for open content is optional, but recommended. Open content can be modeled using the Ion DOM.
30 | * Ideally, anything you can build using ISL data objects should be valid ISL.
31 | * Does not need to have total fidelity with ISL Ion, but must have semantic fidelity. (For example, it’s okay if it loses the difference between `occurs: required`, `occurs: 1` , and `occurs: [exclusive::0, 1]`.)
32 | * May function as a type-safe DSL for programmatic construction of ISL types/schemas.
33 | * May have syntactic sugar for areas where total fidelity is not preserved so that it appears to be closer to ISL Ion. (E.g. a static function `Occurs.required()` that builds a data object representing `occurs: [1, 1]`.)
34 | * Does not necessarily need to support every version of ISL *syntax*.
35 | * Consider the difference between a concrete syntax tree and an abstract syntax tree. You probably want this to be more like an abstract syntax tree so that you can enforce a canonical representation of certain things that can be represented more than one way. Also, an abstract syntax tree makes it easier to support multiple versions of ISL in a single model.
36 |
37 | ### Internal Representation
38 |
39 | * Can be whatever you want.
40 | * Must have semantic fidelity with ISL Ion.
41 | * Should not be exported or publicly exposed.
42 | * Useful for storing optimized forms of schemas (e.g. compiled regex instead of a regex string or resolved, numbered references instead of named type references).
43 |
44 | ## Built-in types
45 |
46 | It may be advantageous to hard-code only the most minimal set of types possible, and allow the remaining built-in types to be defined using ISL. (One potentially useful strategy is to have an `IonSchemaCoreTypesAuthority` where the schema id is the Ion Schema Version Marker. That way, if the built-in types ever change, it’s easy for an implementation to correctly load the right core types.)
47 |
48 | As of Ion Schema 1.0 and 2.0, the hard coded types must be `document` and all of the Ion types (`$null`, `$bool`, `$int`, `$float`, `$decimal`, `$string`, `$symbol`, `$blob`, `$clob`, `$timestamp`, `$list`, `$sexp`, and `$struct`.)
49 |
50 | The remaining built-in types can be defined as follows:
51 | ```
52 | // The top type
53 | type::{
54 | name: $any,
55 | }
56 |
57 | // The bottom type
58 | type::{
59 | name: nothing,
60 | valid_values: [],
61 | }
62 |
63 | // Non-null variants of the Ion types; repeat similarly for each type
64 | type::{
65 | name: bool,
66 | type: $bool,
67 | not: { valid_values: [null.bool] }
68 | }
69 |
70 | // Union types; repeat similarly for lob, $text, text, $number, number, any
71 | type::{
72 | name: $lob,
73 | one_of: [ $blob, $clob ],
74 | }
75 | ```
76 |
77 | ## Caching Imports
78 |
79 | It is possible to have an import graph like this (imports are *down*, so A imports B, etc.)
80 |
81 | ```
82 | A
83 | / \
84 | B C
85 | / \ /
86 | D E
87 | |
88 | F
89 | /|\
90 | G H I
91 | ```
92 |
93 | Here, we have two paths from A to E and everything that E imports. Therefore, we need to cache a schema by its schemaId when we load it, even it is not loaded directly by the user. If we don’t, then we risk performing double (or more) I/O when there is more than one path to import a given dependency.
94 |
95 | ## Ranges
96 |
97 | Ranges in Ion Schema can be confusing to implement.
98 | While there are many types of ranges that might appear similar, there are fundamentally two distinct types of ranges.
99 |
100 | **Discrete Range**
101 |
102 | Discrete (or integral) ranges are ranges over integers and integer-like things (such as timestamp precision, which is essentially an integer wrapped by an enum).
103 | These ranges are compared against some _property_ of a value and are used for `scale` (ISL 1.0), `exponent` (ISL 2.0), `precision`, `timestamp_precision`, `timestamp_offset`, `byte_length`, `container_length`, `codepoint_length`, `utf8_byte_length`, and `occurs`.
104 |
105 | **Value Range**
106 |
107 | Value ranges are ranges over Ion values. These ranges are compared against other Ion values, and are only used with the `valid_values` constraint. They are further subdivided into NumberRange and TimestampRange.
108 |
109 | ## The top type `$any`
110 |
111 | In Ion Schema 2.0, every type implicitly starts out as the top type (i.e. `type: $any`). You should not actually add that in your implementation, since it is identical to a type definition with no constraints.
112 |
113 | In other words, do not add a `type: $any` to the internal model of a type definition because that would be redundant.
114 |
--------------------------------------------------------------------------------
/Gemfile.lock:
--------------------------------------------------------------------------------
1 | GEM
2 | remote: https://rubygems.org/
3 | specs:
4 | activesupport (8.0.2)
5 | base64
6 | benchmark (>= 0.3)
7 | bigdecimal
8 | concurrent-ruby (~> 1.0, >= 1.3.1)
9 | connection_pool (>= 2.2.5)
10 | drb
11 | i18n (>= 1.6, < 2)
12 | logger (>= 1.4.2)
13 | minitest (>= 5.1)
14 | securerandom (>= 0.3)
15 | tzinfo (~> 2.0, >= 2.0.5)
16 | uri (>= 0.13.1)
17 | addressable (2.8.7)
18 | public_suffix (>= 2.0.2, < 7.0)
19 | base64 (0.2.0)
20 | benchmark (0.4.1)
21 | bigdecimal (3.2.2)
22 | coffee-script (2.4.1)
23 | coffee-script-source
24 | execjs
25 | coffee-script-source (1.12.2)
26 | colorator (1.1.0)
27 | commonmarker (0.23.11)
28 | concurrent-ruby (1.3.5)
29 | connection_pool (2.5.3)
30 | csv (3.3.5)
31 | dnsruby (1.72.4)
32 | base64 (~> 0.2.0)
33 | logger (~> 1.6.5)
34 | simpleidn (~> 0.2.1)
35 | drb (2.2.3)
36 | em-websocket (0.5.3)
37 | eventmachine (>= 0.12.9)
38 | http_parser.rb (~> 0)
39 | ethon (0.16.0)
40 | ffi (>= 1.15.0)
41 | eventmachine (1.2.7)
42 | execjs (2.10.0)
43 | faraday (2.13.4)
44 | faraday-net_http (>= 2.0, < 3.5)
45 | json
46 | logger
47 | faraday-net_http (3.4.1)
48 | net-http (>= 0.5.0)
49 | ffi (1.17.2)
50 | forwardable-extended (2.6.0)
51 | gemoji (4.1.0)
52 | github-pages (232)
53 | github-pages-health-check (= 1.18.2)
54 | jekyll (= 3.10.0)
55 | jekyll-avatar (= 0.8.0)
56 | jekyll-coffeescript (= 1.2.2)
57 | jekyll-commonmark-ghpages (= 0.5.1)
58 | jekyll-default-layout (= 0.1.5)
59 | jekyll-feed (= 0.17.0)
60 | jekyll-gist (= 1.5.0)
61 | jekyll-github-metadata (= 2.16.1)
62 | jekyll-include-cache (= 0.2.1)
63 | jekyll-mentions (= 1.6.0)
64 | jekyll-optional-front-matter (= 0.3.2)
65 | jekyll-paginate (= 1.1.0)
66 | jekyll-readme-index (= 0.3.0)
67 | jekyll-redirect-from (= 0.16.0)
68 | jekyll-relative-links (= 0.6.1)
69 | jekyll-remote-theme (= 0.4.3)
70 | jekyll-sass-converter (= 1.5.2)
71 | jekyll-seo-tag (= 2.8.0)
72 | jekyll-sitemap (= 1.4.0)
73 | jekyll-swiss (= 1.0.0)
74 | jekyll-theme-architect (= 0.2.0)
75 | jekyll-theme-cayman (= 0.2.0)
76 | jekyll-theme-dinky (= 0.2.0)
77 | jekyll-theme-hacker (= 0.2.0)
78 | jekyll-theme-leap-day (= 0.2.0)
79 | jekyll-theme-merlot (= 0.2.0)
80 | jekyll-theme-midnight (= 0.2.0)
81 | jekyll-theme-minimal (= 0.2.0)
82 | jekyll-theme-modernist (= 0.2.0)
83 | jekyll-theme-primer (= 0.6.0)
84 | jekyll-theme-slate (= 0.2.0)
85 | jekyll-theme-tactile (= 0.2.0)
86 | jekyll-theme-time-machine (= 0.2.0)
87 | jekyll-titles-from-headings (= 0.5.3)
88 | jemoji (= 0.13.0)
89 | kramdown (= 2.4.0)
90 | kramdown-parser-gfm (= 1.1.0)
91 | liquid (= 4.0.4)
92 | mercenary (~> 0.3)
93 | minima (= 2.5.1)
94 | nokogiri (>= 1.16.2, < 2.0)
95 | rouge (= 3.30.0)
96 | terminal-table (~> 1.4)
97 | webrick (~> 1.8)
98 | github-pages-health-check (1.18.2)
99 | addressable (~> 2.3)
100 | dnsruby (~> 1.60)
101 | octokit (>= 4, < 8)
102 | public_suffix (>= 3.0, < 6.0)
103 | typhoeus (~> 1.3)
104 | html-pipeline (2.14.3)
105 | activesupport (>= 2)
106 | nokogiri (>= 1.4)
107 | http_parser.rb (0.8.0)
108 | i18n (1.14.7)
109 | concurrent-ruby (~> 1.0)
110 | jekyll (3.10.0)
111 | addressable (~> 2.4)
112 | colorator (~> 1.0)
113 | csv (~> 3.0)
114 | em-websocket (~> 0.5)
115 | i18n (>= 0.7, < 2)
116 | jekyll-sass-converter (~> 1.0)
117 | jekyll-watch (~> 2.0)
118 | kramdown (>= 1.17, < 3)
119 | liquid (~> 4.0)
120 | mercenary (~> 0.3.3)
121 | pathutil (~> 0.9)
122 | rouge (>= 1.7, < 4)
123 | safe_yaml (~> 1.0)
124 | webrick (>= 1.0)
125 | jekyll-avatar (0.8.0)
126 | jekyll (>= 3.0, < 5.0)
127 | jekyll-coffeescript (1.2.2)
128 | coffee-script (~> 2.2)
129 | coffee-script-source (~> 1.12)
130 | jekyll-commonmark (1.4.0)
131 | commonmarker (~> 0.22)
132 | jekyll-commonmark-ghpages (0.5.1)
133 | commonmarker (>= 0.23.7, < 1.1.0)
134 | jekyll (>= 3.9, < 4.0)
135 | jekyll-commonmark (~> 1.4.0)
136 | rouge (>= 2.0, < 5.0)
137 | jekyll-default-layout (0.1.5)
138 | jekyll (>= 3.0, < 5.0)
139 | jekyll-feed (0.17.0)
140 | jekyll (>= 3.7, < 5.0)
141 | jekyll-gist (1.5.0)
142 | octokit (~> 4.2)
143 | jekyll-github-metadata (2.16.1)
144 | jekyll (>= 3.4, < 5.0)
145 | octokit (>= 4, < 7, != 4.4.0)
146 | jekyll-include-cache (0.2.1)
147 | jekyll (>= 3.7, < 5.0)
148 | jekyll-mentions (1.6.0)
149 | html-pipeline (~> 2.3)
150 | jekyll (>= 3.7, < 5.0)
151 | jekyll-optional-front-matter (0.3.2)
152 | jekyll (>= 3.0, < 5.0)
153 | jekyll-paginate (1.1.0)
154 | jekyll-readme-index (0.3.0)
155 | jekyll (>= 3.0, < 5.0)
156 | jekyll-redirect-from (0.16.0)
157 | jekyll (>= 3.3, < 5.0)
158 | jekyll-relative-links (0.6.1)
159 | jekyll (>= 3.3, < 5.0)
160 | jekyll-remote-theme (0.4.3)
161 | addressable (~> 2.0)
162 | jekyll (>= 3.5, < 5.0)
163 | jekyll-sass-converter (>= 1.0, <= 3.0.0, != 2.0.0)
164 | rubyzip (>= 1.3.0, < 3.0)
165 | jekyll-sass-converter (1.5.2)
166 | sass (~> 3.4)
167 | jekyll-seo-tag (2.8.0)
168 | jekyll (>= 3.8, < 5.0)
169 | jekyll-sitemap (1.4.0)
170 | jekyll (>= 3.7, < 5.0)
171 | jekyll-swiss (1.0.0)
172 | jekyll-theme-architect (0.2.0)
173 | jekyll (> 3.5, < 5.0)
174 | jekyll-seo-tag (~> 2.0)
175 | jekyll-theme-cayman (0.2.0)
176 | jekyll (> 3.5, < 5.0)
177 | jekyll-seo-tag (~> 2.0)
178 | jekyll-theme-dinky (0.2.0)
179 | jekyll (> 3.5, < 5.0)
180 | jekyll-seo-tag (~> 2.0)
181 | jekyll-theme-hacker (0.2.0)
182 | jekyll (> 3.5, < 5.0)
183 | jekyll-seo-tag (~> 2.0)
184 | jekyll-theme-leap-day (0.2.0)
185 | jekyll (> 3.5, < 5.0)
186 | jekyll-seo-tag (~> 2.0)
187 | jekyll-theme-merlot (0.2.0)
188 | jekyll (> 3.5, < 5.0)
189 | jekyll-seo-tag (~> 2.0)
190 | jekyll-theme-midnight (0.2.0)
191 | jekyll (> 3.5, < 5.0)
192 | jekyll-seo-tag (~> 2.0)
193 | jekyll-theme-minimal (0.2.0)
194 | jekyll (> 3.5, < 5.0)
195 | jekyll-seo-tag (~> 2.0)
196 | jekyll-theme-modernist (0.2.0)
197 | jekyll (> 3.5, < 5.0)
198 | jekyll-seo-tag (~> 2.0)
199 | jekyll-theme-primer (0.6.0)
200 | jekyll (> 3.5, < 5.0)
201 | jekyll-github-metadata (~> 2.9)
202 | jekyll-seo-tag (~> 2.0)
203 | jekyll-theme-slate (0.2.0)
204 | jekyll (> 3.5, < 5.0)
205 | jekyll-seo-tag (~> 2.0)
206 | jekyll-theme-tactile (0.2.0)
207 | jekyll (> 3.5, < 5.0)
208 | jekyll-seo-tag (~> 2.0)
209 | jekyll-theme-time-machine (0.2.0)
210 | jekyll (> 3.5, < 5.0)
211 | jekyll-seo-tag (~> 2.0)
212 | jekyll-titles-from-headings (0.5.3)
213 | jekyll (>= 3.3, < 5.0)
214 | jekyll-watch (2.2.1)
215 | listen (~> 3.0)
216 | jemoji (0.13.0)
217 | gemoji (>= 3, < 5)
218 | html-pipeline (~> 2.2)
219 | jekyll (>= 3.0, < 5.0)
220 | json (2.13.1)
221 | kramdown (2.4.0)
222 | rexml
223 | kramdown-parser-gfm (1.1.0)
224 | kramdown (~> 2.0)
225 | liquid (4.0.4)
226 | listen (3.9.0)
227 | rb-fsevent (~> 0.10, >= 0.10.3)
228 | rb-inotify (~> 0.9, >= 0.9.10)
229 | logger (1.6.6)
230 | mercenary (0.3.6)
231 | mini_portile2 (2.8.9)
232 | minima (2.5.1)
233 | jekyll (>= 3.5, < 5.0)
234 | jekyll-feed (~> 0.9)
235 | jekyll-seo-tag (~> 2.1)
236 | minitest (5.25.5)
237 | net-http (0.6.0)
238 | uri
239 | nokogiri (1.18.9)
240 | mini_portile2 (~> 2.8.2)
241 | racc (~> 1.4)
242 | octokit (4.25.1)
243 | faraday (>= 1, < 3)
244 | sawyer (~> 0.9)
245 | pathutil (0.16.2)
246 | forwardable-extended (~> 2.6)
247 | public_suffix (5.1.1)
248 | racc (1.8.1)
249 | rb-fsevent (0.11.2)
250 | rb-inotify (0.11.1)
251 | ffi (~> 1.0)
252 | rexml (3.4.2)
253 | rouge (3.30.0)
254 | rubyzip (2.4.1)
255 | safe_yaml (1.0.5)
256 | sass (3.7.4)
257 | sass-listen (~> 4.0.0)
258 | sass-listen (4.0.0)
259 | rb-fsevent (~> 0.9, >= 0.9.4)
260 | rb-inotify (~> 0.9, >= 0.9.7)
261 | sawyer (0.9.2)
262 | addressable (>= 2.3.5)
263 | faraday (>= 0.17.3, < 3)
264 | securerandom (0.4.1)
265 | simpleidn (0.2.3)
266 | terminal-table (1.8.0)
267 | unicode-display_width (~> 1.1, >= 1.1.1)
268 | typhoeus (1.4.1)
269 | ethon (>= 0.9.0)
270 | tzinfo (2.0.6)
271 | concurrent-ruby (~> 1.0)
272 | unicode-display_width (1.8.0)
273 | uri (1.0.3)
274 | webrick (1.9.1)
275 |
276 | PLATFORMS
277 | ruby
278 |
279 | DEPENDENCIES
280 | github-pages
281 | webrick (~> 1.8)
282 |
283 | RUBY VERSION
284 | ruby 3.4.5p51
285 |
286 | BUNDLED WITH
287 | 2.6.9
288 |
--------------------------------------------------------------------------------
/_posts/2023-06-07-launch-invalid-transitive-import-tool.md:
--------------------------------------------------------------------------------
1 | ---
2 | layout: news_item
3 | title: "🚀 Launch Announcement 🚀 Transitive Import Repair Tool"
4 | date: 2023-06-07
5 | categories: news ion-schema-kotlin
6 | ---
7 |
8 | The CLI in `ion-schema-kotlin` now includes a command for repairing schemas that rely on the bug that allowed transitive imports.
9 |
10 | # What is the problem?
11 |
12 | The [Imports section](https://amazon-ion.github.io/ion-schema/docs/isl-1-0/spec#imports) of the Ion Schema 1.0 spec was previously underspecified with respect to how imports should be resolved (see amazon-ion/ion-schema#39).
13 |
14 | For example, if Schema A imports all types from Schema B, and Schema B imports something from Schema C, should Schema A also be able to reference the type from Schema C without directly importing it (transitive import resolution), or must Schema A explicitly import Schema C to reference any types declared in Schema C (direct import resolution)?
15 |
16 | The original implementation resolved imports transitively, which led to issues when schemas have so-called "diamond" imports (amazon-ion/ion-schema-kotlin#117) or circular imports (amazon-ion/ion-schema-kotlin#58).
17 |
18 | Transitively resolved imports can also lead to unexpected type name conflicts when a user is authoring a schema that depends on another schema owned by someone else. If ALL schemas are known at compile time, name conflicts can be detected and fixed early, before the schemas are deployed to production. When some schemas are authored at runtime or by users of the application, name conflicts can present an availability risk.
19 |
20 | Suppose we have Schemas A through Z, each of which imports the next schema in alphabetical order. If imports are resolved transitively, then even if Schema A does not rely on any types from Schema Z, a change to Schema Z could result in a name conflict with a type that is declared in Schema A, thus rendering Schema A unusable. If imports are not transitive, then Schema A is only affected by changes in Schema B. Direct-import-only resolution does not _entirely_ remove the potential for name conflicts, but it significantly reduces the number of possible schemas that can contribute to a naming conflict within a graph of interdependent schemas.
21 |
22 | If you want to entirely eliminate the possibility of a name conflict, you can do so by always importing specific types by name (or by importing nothing at all).
23 |
24 | # What are we doing about it?
25 |
26 | The Ion Schema 1.0 specification has been clarified to say that only top-level, declared types may be imported from another schema.
27 |
28 | Ion Schema Kotlin is being fixed so that the resolution of type references will only take into account the direct imports of a schema. This will be a breaking change for some users of Ion Schema, so the fix is being released in two phases.
29 |
30 | The bug fix was released in `ion-schema-kotlin-1.2.0` as an optional behavior. Customers will have to explicitly opt in by setting the `allowTransitiveImports(false)` builder option.
31 |
32 | In `ion-schema-kotlin-2.0.0`, the bug fix will become the default behavior.
33 |
34 | In addition, the `ion-schema-kotlin` repo has a CLI tool to help you fix any affected schemas.
35 |
36 | # How can I tell if I am affected by this change?
37 |
38 | It _does_ affect you if you have created any schemas that (intentionally or unintentionally) use a “transitive” import, or if you vend schemas to other teams who have created their own schemas that have used a “transitive” import.
39 |
40 | ...but we don’t necessarily expect teams to know whether they have done this, so we have a few ways for you to find out.
41 |
42 | #### Method 1
43 |
44 | Here is a rubric that can help.
45 |
46 | * If you don’t use any imports to create dependencies between schemas, then this bug **_does not_** affect you. E.g.
47 |
48 | ```
49 |
50 | schema_header::{
51 | imports: [
52 | { id: "birds.isl" }, // <-- you have none of these
53 | { id: "cats.isl", type: lion }, // <-- or these
54 | { id: "cats.isl", type: cougar, as: puma }, // <-- or these
55 | ]
56 | }
57 |
58 | type::{
59 | name: poodle_list,
60 | type: list,
61 | element: { id: "dogs.isl", type: poodle } // <-- and you have no inline
62 | // imports, like this one
63 | }
64 |
65 | ```
66 |
67 | * If the dependency tree of your schemas is never more than 2 levels deep (i.e. root and one layer of imports) then this bug **_does not_** affect you.
68 | * If you have any schema that contains _only_ import statements (i.e. because it is a schema to aggregate other schemas), then this **_does_** affect you.
69 |
70 | ```
71 | // A schema such as this one
72 | schema_header::{
73 | imports:[
74 | { id: "cats.isl" },
75 | { id: "dogs.isl" },
76 | { id: "birds.isl" },
77 | ]
78 | }
79 | // Note—there are intentionally no types between the header and the footer
80 | schema_footer::{}
81 | ```
82 |
83 | * Otherwise, this bug **_might_** affect you.
84 |
85 | #### Method 2
86 |
87 | Use the CLI command (mentioned below) to attempt to repair your schemas.
88 | If you run the rewriter tool and the diff of non-whitespace changes between any of your existing schemas and the corresponding rewritten schemas is non-empty, then it _probably does_ affect you.
89 | (There are some cases where the tool might reorganize some imports resulting in a diff, but the semantics of the schema are unchanged.)
90 | If you run the rewriter tool on every schema that your application could use (i.e. the universe of all possible schemas that can be loaded by the Authorities you are using, which is only finite if you are the author of all of your schemas), and the diff between the old and new schemas is empty (or only whitespace), then the bug/bugfix **_does not_** affect you.
91 |
92 | #### Method 3
93 |
94 | Provide a logging callback for the Ion Schema System. Wait a while, and check your application’s logs. If you have any warning messages about transitive imports, then it **_does_** affect you. If there are no warning messages, it **_might_** affect you.
95 |
96 | Here are examples of how you can configure a callback function to receive the warning messages using [`withWarningMessageCallback`](https://github.com/amzn/ion-schema-kotlin/blob/master/src/com/amazon/ionschema/IonSchemaSystemBuilder.kt#L124,L162).
97 |
98 | **Kotlin**
99 |
100 | ```kotlin
101 | val iss = IonSchemaSystemBuilder.standard()
102 | .withWarningMessageCallback { println(it) }
103 | // Configure Authorities, etc. as usual
104 | .build()
105 | ```
106 |
107 | **Java**
108 |
109 | ```java
110 | IonSchemaSystem iss = IonSchemaSystemBuilder.standard()
111 | // Use a lambda function...
112 | .withWarningMessageCallback(it -> { System.out.println(it); })
113 | // ... or a method reference
114 | .withWarningMessageCallback((Consumer) System.out::println)
115 | // Configure Authorities, etc. as usual
116 | .build()
117 | ```
118 |
119 | In most applications, you should substitute your own logging framework or metrics recorder instead of printing to `stdout`.
120 |
121 |
122 | # How do I fix my schemas?
123 |
124 | Check out the `ion-schema-kotlin` GitHub repository, and use the bundled CLI to update your schemas.
125 |
126 | ```shell
127 | git clone --recursive https://github.com/amazon-ion/ion-schema-kotlin.git
128 | cd ion-schema-kotlin
129 | ./ion-schema-cli repair fix-transitive-imports --help
130 | ```
131 |
132 | Follow the instructions in the command help to fix your schemas. In most cases, the default options will suffice, and you can simply run:
133 |
134 | ```shell
135 | ./ion-schema-cli repair fix-transitive-imports
136 | ```
137 |
138 |
139 | # FAQ
140 |
141 | ### What about schemas that only aggregate types from other schemas?
142 |
143 | E.g.
144 | ```
145 | schema_header::{
146 | imports: [
147 | { id: "common.isl" },
148 | { id: "api_1.isl" },
149 | { id: "api_2.isl" },
150 | { id: "api_3.isl" },
151 | ]
152 | }
153 | schema_footer::{}
154 | ```
155 |
156 | The use case is valid, but this implementation is no longer valid because it relied on the buggy behavior.
157 |
158 | Instead, you must explicitly declare types in your schema to “re-export” them.
159 |
160 | E.g.
161 | ```
162 | type::{ name: foo, type: { id: "common.isl", type: foo } }
163 | type::{ name: api_1_type, type: { id: "api_1.isl", type: api_1_type } }
164 | type::{ name: an_alias_for_api_1_type, type: { id: "api_1.isl", type: api_1_type } }
165 | // etc.
166 | ```
167 |
168 | The schema fixer/rewriter tool will know how to handle this sort of schema, and can expand the imports into type definitions for you.
169 |
170 | ### How do customers upgrade when mixing schemas authored by multiple teams?
171 |
172 | Who needs to upgrade their schemas first?
173 |
174 | * If the consumer only consumes types by importing them from an aggregating schema, the vendor and consumer can safely update their schemas in any order.
175 | * If the only necessary changes by the vending team are rewrites of aggregating schemas (i.e. schema has imports but not types), the vending team can do this either before or after the consumer schemas are updated.
176 | * If vending schemas as an artifact in some package manager, the vending team can run the rewriter tool and save the results in a new version of the package distribution. Then consuming teams can pick up that new version at their convenience.
177 | * If none of the above options are possible, consumers must update schemas first, then vendors can update their schemas, then everyone can update their schema systems.
178 | * Consumers cannot update their schema systems to the new behavior until the vended schemas have been fixed (or are known to be good).
179 | * Cyclical cross-team dependencies could cause a problem where all the schemas need to be updated at the same time. As long as schemas are version-controlled this should still be reasonably straightforward. Feel free to open an issue if you need help in this situation.
180 |
181 | ### I enabled the bug fix and now my application is broken. What do I do?
182 |
183 | Don’t panic! This is mostly harmless. As soon as you roll back your changes, you should be fine.
184 |
185 | If you cannot roll back your entire deployment because of some other change that cannot be reverted, and you have confirmed that the problem is because you have picked up the bugfix before you were ready for it, you can use the `allowTransitiveImports(true)` builder option to disable the bugfix.
186 |
187 | Once you have mitigated the immediate impact to your application, refer back to this page to determine what went wrong. If you are still unable to resolve your issue, create an issue to let us know.
188 |
189 | ### If I run the schema rewriter tool, can I still load those schemas with an old version of Ion Schema Kotlin?
190 |
191 | Yes. This bugfix does not introduce any new Ion Schema language features—it only removes the use of an edge case that was previously underspecified in the spec.
192 |
193 | ### Will `ion-schema-rust` have an option to allow transitive imports?
194 |
195 | There will be no support allow transitive imports in `ion-schema-rust`.
196 |
197 | ### Does Ion Schema 2.0 allow transitive imports?
198 |
199 | No. Ion Schema 2.0 does not allow transitive imports and has never been affected by this bug.
200 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 |
2 | Apache License
3 | Version 2.0, January 2004
4 | http://www.apache.org/licenses/
5 |
6 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7 |
8 | 1. Definitions.
9 |
10 | "License" shall mean the terms and conditions for use, reproduction,
11 | and distribution as defined by Sections 1 through 9 of this document.
12 |
13 | "Licensor" shall mean the copyright owner or entity authorized by
14 | the copyright owner that is granting the License.
15 |
16 | "Legal Entity" shall mean the union of the acting entity and all
17 | other entities that control, are controlled by, or are under common
18 | control with that entity. For the purposes of this definition,
19 | "control" means (i) the power, direct or indirect, to cause the
20 | direction or management of such entity, whether by contract or
21 | otherwise, or (ii) ownership of fifty percent (50%) or more of the
22 | outstanding shares, or (iii) beneficial ownership of such entity.
23 |
24 | "You" (or "Your") shall mean an individual or Legal Entity
25 | exercising permissions granted by this License.
26 |
27 | "Source" form shall mean the preferred form for making modifications,
28 | including but not limited to software source code, documentation
29 | source, and configuration files.
30 |
31 | "Object" form shall mean any form resulting from mechanical
32 | transformation or translation of a Source form, including but
33 | not limited to compiled object code, generated documentation,
34 | and conversions to other media types.
35 |
36 | "Work" shall mean the work of authorship, whether in Source or
37 | Object form, made available under the License, as indicated by a
38 | copyright notice that is included in or attached to the work
39 | (an example is provided in the Appendix below).
40 |
41 | "Derivative Works" shall mean any work, whether in Source or Object
42 | form, that is based on (or derived from) the Work and for which the
43 | editorial revisions, annotations, elaborations, or other modifications
44 | represent, as a whole, an original work of authorship. For the purposes
45 | of this License, Derivative Works shall not include works that remain
46 | separable from, or merely link (or bind by name) to the interfaces of,
47 | the Work and Derivative Works thereof.
48 |
49 | "Contribution" shall mean any work of authorship, including
50 | the original version of the Work and any modifications or additions
51 | to that Work or Derivative Works thereof, that is intentionally
52 | submitted to Licensor for inclusion in the Work by the copyright owner
53 | or by an individual or Legal Entity authorized to submit on behalf of
54 | the copyright owner. For the purposes of this definition, "submitted"
55 | means any form of electronic, verbal, or written communication sent
56 | to the Licensor or its representatives, including but not limited to
57 | communication on electronic mailing lists, source code control systems,
58 | and issue tracking systems that are managed by, or on behalf of, the
59 | Licensor for the purpose of discussing and improving the Work, but
60 | excluding communication that is conspicuously marked or otherwise
61 | designated in writing by the copyright owner as "Not a Contribution."
62 |
63 | "Contributor" shall mean Licensor and any individual or Legal Entity
64 | on behalf of whom a Contribution has been received by Licensor and
65 | subsequently incorporated within the Work.
66 |
67 | 2. Grant of Copyright License. Subject to the terms and conditions of
68 | this License, each Contributor hereby grants to You a perpetual,
69 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70 | copyright license to reproduce, prepare Derivative Works of,
71 | publicly display, publicly perform, sublicense, and distribute the
72 | Work and such Derivative Works in Source or Object form.
73 |
74 | 3. Grant of Patent License. Subject to the terms and conditions of
75 | this License, each Contributor hereby grants to You a perpetual,
76 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77 | (except as stated in this section) patent license to make, have made,
78 | use, offer to sell, sell, import, and otherwise transfer the Work,
79 | where such license applies only to those patent claims licensable
80 | by such Contributor that are necessarily infringed by their
81 | Contribution(s) alone or by combination of their Contribution(s)
82 | with the Work to which such Contribution(s) was submitted. If You
83 | institute patent litigation against any entity (including a
84 | cross-claim or counterclaim in a lawsuit) alleging that the Work
85 | or a Contribution incorporated within the Work constitutes direct
86 | or contributory patent infringement, then any patent licenses
87 | granted to You under this License for that Work shall terminate
88 | as of the date such litigation is filed.
89 |
90 | 4. Redistribution. You may reproduce and distribute copies of the
91 | Work or Derivative Works thereof in any medium, with or without
92 | modifications, and in Source or Object form, provided that You
93 | meet the following conditions:
94 |
95 | (a) You must give any other recipients of the Work or
96 | Derivative Works a copy of this License; and
97 |
98 | (b) You must cause any modified files to carry prominent notices
99 | stating that You changed the files; and
100 |
101 | (c) You must retain, in the Source form of any Derivative Works
102 | that You distribute, all copyright, patent, trademark, and
103 | attribution notices from the Source form of the Work,
104 | excluding those notices that do not pertain to any part of
105 | the Derivative Works; and
106 |
107 | (d) If the Work includes a "NOTICE" text file as part of its
108 | distribution, then any Derivative Works that You distribute must
109 | include a readable copy of the attribution notices contained
110 | within such NOTICE file, excluding those notices that do not
111 | pertain to any part of the Derivative Works, in at least one
112 | of the following places: within a NOTICE text file distributed
113 | as part of the Derivative Works; within the Source form or
114 | documentation, if provided along with the Derivative Works; or,
115 | within a display generated by the Derivative Works, if and
116 | wherever such third-party notices normally appear. The contents
117 | of the NOTICE file are for informational purposes only and
118 | do not modify the License. You may add Your own attribution
119 | notices within Derivative Works that You distribute, alongside
120 | or as an addendum to the NOTICE text from the Work, provided
121 | that such additional attribution notices cannot be construed
122 | as modifying the License.
123 |
124 | You may add Your own copyright statement to Your modifications and
125 | may provide additional or different license terms and conditions
126 | for use, reproduction, or distribution of Your modifications, or
127 | for any such Derivative Works as a whole, provided Your use,
128 | reproduction, and distribution of the Work otherwise complies with
129 | the conditions stated in this License.
130 |
131 | 5. Submission of Contributions. Unless You explicitly state otherwise,
132 | any Contribution intentionally submitted for inclusion in the Work
133 | by You to the Licensor shall be under the terms and conditions of
134 | this License, without any additional terms or conditions.
135 | Notwithstanding the above, nothing herein shall supersede or modify
136 | the terms of any separate license agreement you may have executed
137 | with Licensor regarding such Contributions.
138 |
139 | 6. Trademarks. This License does not grant permission to use the trade
140 | names, trademarks, service marks, or product names of the Licensor,
141 | except as required for reasonable and customary use in describing the
142 | origin of the Work and reproducing the content of the NOTICE file.
143 |
144 | 7. Disclaimer of Warranty. Unless required by applicable law or
145 | agreed to in writing, Licensor provides the Work (and each
146 | Contributor provides its Contributions) on an "AS IS" BASIS,
147 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148 | implied, including, without limitation, any warranties or conditions
149 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150 | PARTICULAR PURPOSE. You are solely responsible for determining the
151 | appropriateness of using or redistributing the Work and assume any
152 | risks associated with Your exercise of permissions under this License.
153 |
154 | 8. Limitation of Liability. In no event and under no legal theory,
155 | whether in tort (including negligence), contract, or otherwise,
156 | unless required by applicable law (such as deliberate and grossly
157 | negligent acts) or agreed to in writing, shall any Contributor be
158 | liable to You for damages, including any direct, indirect, special,
159 | incidental, or consequential damages of any character arising as a
160 | result of this License or out of the use or inability to use the
161 | Work (including but not limited to damages for loss of goodwill,
162 | work stoppage, computer failure or malfunction, or any and all
163 | other commercial damages or losses), even if such Contributor
164 | has been advised of the possibility of such damages.
165 |
166 | 9. Accepting Warranty or Additional Liability. While redistributing
167 | the Work or Derivative Works thereof, You may choose to offer,
168 | and charge a fee for, acceptance of support, warranty, indemnity,
169 | or other liability obligations and/or rights consistent with this
170 | License. However, in accepting such obligations, You may act only
171 | on Your own behalf and on Your sole responsibility, not on behalf
172 | of any other Contributor, and only if You agree to indemnify,
173 | defend, and hold each Contributor harmless for any liability
174 | incurred by, or claims asserted against, such Contributor by reason
175 | of your accepting any such warranty or additional liability.
176 |
177 | END OF TERMS AND CONDITIONS
178 |
179 | APPENDIX: How to apply the Apache License to your work.
180 |
181 | To apply the Apache License to your work, attach the following
182 | boilerplate notice, with the fields enclosed by brackets "[]"
183 | replaced with your own identifying information. (Don't include
184 | the brackets!) The text should be enclosed in the appropriate
185 | comment syntax for the file format. We also recommend that a
186 | file or class name and description of purpose be included on the
187 | same "printed page" as the copyright notice for easier
188 | identification within third-party archives.
189 |
190 | Copyright [yyyy] [name of copyright owner]
191 |
192 | Licensed under the Apache License, Version 2.0 (the "License");
193 | you may not use this file except in compliance with the License.
194 | You may obtain a copy of the License at
195 |
196 | http://www.apache.org/licenses/LICENSE-2.0
197 |
198 | Unless required by applicable law or agreed to in writing, software
199 | distributed under the License is distributed on an "AS IS" BASIS,
200 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201 | See the License for the specific language governing permissions and
202 | limitations under the License.
203 |
--------------------------------------------------------------------------------
/assets/ion-schema-widget.js:
--------------------------------------------------------------------------------
1 | import init, {validate} from "./wasm_ion_schema.js";
2 | const validateButton = document.getElementById("validate");
3 | const shareButton = document.getElementById("share");
4 | const dropDownSelection = document.getElementById("examples");
5 |
6 | function initPage() {
7 | let sampleIds = Object.keys(SAMPLES);
8 |
9 | // Check for data in the URL and add it to "SAMPLES"
10 | const params = new Proxy(new URLSearchParams(window.location.search), {
11 | get: (searchParams, prop) => searchParams.get(prop),
12 | });
13 | const SHARED_ID = "shared_in_url"
14 | if (params.schema || params.value || params.type) {
15 | sampleIds.unshift(SHARED_ID) // Add the "Shared in URL" to the top of the list
16 | SAMPLES[SHARED_ID] = {
17 | displayName: "(Shared in URL)",
18 | schema: params.schema ?? "",
19 | value: params.value ?? "",
20 | type: params.type ?? "",
21 | };
22 | }
23 |
24 | // Set up the "samples" dropdown
25 | for (let i = 0; i < sampleIds.length; ++i) {
26 | let id = sampleIds[i]
27 | dropDownSelection.add(new Option(SAMPLES[id].displayName, id));
28 | }
29 | if (SAMPLES[SHARED_ID]) {
30 | dropDownSelection.selectedIndex = sampleIds.indexOf(SHARED_ID)
31 | }
32 | dropDownSelection.onchange()
33 | }
34 |
35 | function populateInputFields(sample) {
36 | let { schema, value, type } = sample;
37 |
38 | ace.edit("schema").setOptions({
39 | mode: 'ace/mode/ion',
40 | theme: 'ace/theme/cloud9_day',
41 | showPrintMargin: false,
42 | tabSize: 2,
43 | value: trimIndent(schema),
44 | });
45 | ace.edit("value").setOptions({
46 | mode: 'ace/mode/ion',
47 | theme: 'ace/theme/cloud9_day',
48 | showPrintMargin: false,
49 | tabSize: 2,
50 | value: trimIndent(value),
51 | });
52 | document.getElementById("schema_type").value = type;
53 |
54 | // clear any previous validation results
55 | const pre = document.getElementById('result');
56 | const resultDiv = document.getElementById('resultdiv');
57 | const violation = document.getElementById('violation');
58 | pre.textContent = "";
59 | violation.textContent = "";
60 | _set_output_style(resultDiv, "primary")
61 | }
62 |
63 | function _set_output_style(resultDiv, styleName) {
64 | var toRemove = [];
65 | resultDiv.classList.forEach(value => {
66 | if (value.startsWith("bs-callout-")) toRemove += value;
67 | })
68 | resultDiv.classList.remove(toRemove)
69 | resultDiv.classList.add(`bs-callout-${styleName}`)
70 | }
71 |
72 | function _generate_violations_table(data, container) {
73 | // Create the table element
74 | let table = document.createElement("table");
75 |
76 | // Get the keys (column names) of the first object in the given data
77 | let cols = Object.keys(data[0]);
78 |
79 | // Create the header element
80 | let thead = document.createElement("thead");
81 | let tr = document.createElement("tr");
82 |
83 | // Loop through the column names and create header cells
84 | cols.forEach((item) => {
85 | let th = document.createElement("th");
86 | th.innerText = item; // Set the column name as the text of the header cell
87 | tr.appendChild(th); // Append the header cell to the header row
88 | });
89 | thead.appendChild(tr); // Append the header row to the header
90 | table.append(tr) // Append the header to the table
91 |
92 | // Loop through the given data and create table rows
93 | data.forEach((item) => {
94 | let tr = document.createElement("tr");
95 |
96 | // Get the values of the current object in the given data
97 | let vals = Object.values(item);
98 |
99 | // Loop through the values and create table cells
100 | vals.forEach((elem) => {
101 | let td = document.createElement("td");
102 | td.innerText = elem; // Set the value as the text of the table cell
103 | tr.appendChild(td); // Append the table cell to the table row
104 | });
105 | table.appendChild(tr); // Append the table row to the table
106 | });
107 | container.appendChild(table) // Append the table to the container element
108 | }
109 |
110 | const show = () => {
111 | const schemaContent = ace.edit("schema").getValue();
112 | const valueContent = ace.edit("value").getValue();
113 |
114 | init()
115 | .then(() => {
116 | const is_document = document.getElementById("document").checked;
117 | const result = validate(valueContent, schemaContent, document.getElementById('schema_type').value, is_document);
118 | const pre = document.getElementById('result');
119 | const resultDiv = document.getElementById('resultdiv');
120 | const container = document.getElementById('violation');
121 |
122 | container.textContent = "";
123 | // check if there is any error while validation and alert with the error
124 | if (result.has_error()) {
125 | _set_output_style(resultDiv, "danger")
126 | pre.textContent = result.error();
127 | } else {
128 | if (result.result()) {
129 | _set_output_style(resultDiv, "success")
130 | pre.textContent = `${result.value()} is valid!`;
131 | } else {
132 | _set_output_style(resultDiv, "warning")
133 | pre.textContent = `${result.value()} is invalid!`;
134 | _generate_violations_table(result.violations(), container)
135 | }
136 | }
137 |
138 | console.log(result);
139 | });
140 | };
141 |
142 | validateButton.addEventListener("click", show);
143 |
144 | const copyUrl = () => {
145 | let url = window.location.href.split('?')[0]
146 | + `?schema=${encodeURIComponent(ace.edit("schema").getValue())}`
147 | + `&value=${encodeURIComponent(ace.edit("value").getValue())}`
148 | + `&type=${encodeURIComponent(document.getElementById('schema_type').value)}`;
149 | let x = document.getElementById("snackbar");
150 | x.innerText = "Copied to clipboard"
151 | x.classList.add("show");
152 | // CSS fadeout is set with a delay of 2.5 seconds and an animation time of 0.5 seconds. Timeout here
153 | // is slightly shorter than 3s to avoid a race condition and have the box flash before vanishing again.
154 | setTimeout(function(){ x.classList.remove("show"); }, 2995);
155 | return navigator.clipboard.writeText(url);
156 | };
157 | shareButton.addEventListener("click", copyUrl);
158 |
159 | dropDownSelection.onchange = function() {
160 | let sample = SAMPLES[dropDownSelection.value];
161 | populateInputFields(sample)
162 | };
163 |
164 | /**
165 | * Detects a common minimal indent of all the input lines, removes it from every line.
166 | * Note that blank lines do not affect the detected indent level.
167 | */
168 | function trimIndent(str) {
169 | let lines = str.split("\n").filter((l) => l.length > 0)
170 | let indentLength = Math.min(...lines.filter((l) => l.trim().length > 0).map((l) => l.length - l.trimStart().length))
171 | return lines.map((l) => l.substring(indentLength)).join("\n")
172 | }
173 |
174 | const SAMPLES = {
175 | simpleTypeDefinition: {
176 | displayName: "Simple Type Definition",
177 | schema: `
178 | $ion_schema_2_0
179 | type::{
180 | name: short_string,
181 | type: string,
182 | codepoint_length: range::[1, 10],
183 | }
184 | `,
185 | type: "short_string",
186 | value: `"Hello World!"`
187 | },
188 | typeDefinitionWithFields: {
189 | displayName: "Type Definition with fields",
190 | schema: `
191 | $ion_schema_2_0
192 | type::{
193 | name: customer,
194 | type: struct,
195 | fields: closed::{
196 | firstName: { type: string, occurs: required },
197 | middleName: string,
198 | lastName: { type: string, occurs: required },
199 | age: { type: int, valid_values: range::[1, max], }
200 | }
201 | }`,
202 | type: "customer",
203 | value: `{ firstName: "John", lastName: "Doe", age: -5 }`,
204 | },
205 | typeDefinitionWithLogicConstraints: {
206 | displayName: "Type Definition with logic constraints",
207 | schema: `
208 | $ion_schema_2_0
209 | type::{
210 | name: string_or_bool,
211 | any_of: [string, bool],
212 | }`,
213 | type: "string_or_bool",
214 | value: `hi`
215 | },
216 | versionedType: {
217 | displayName: "Versioned Type",
218 | schema: `
219 | $ion_schema_2_0
220 | type::{
221 | // The 'widget' type includes all versions of widgets
222 | name: widget,
223 | any_of: [widget_v1, widget_v2],
224 | }
225 | type::{
226 | // The 'widget_latest' type is an alias that always points to the latest version of widget
227 | name: widget_latest,
228 | type: widget_v2,
229 | }
230 | type::{
231 | name: widget_v1,
232 | fields: closed::{
233 | name: string,
234 | part_id: int,
235 | component_ids: { type: list, element: int }
236 | }
237 | }
238 | type::{
239 | name: widget_v2,
240 | fields: closed::{
241 | name: string,
242 | // widget_v2 has a string for the part_id
243 | part_id: string,
244 | component_ids: {
245 | type: list,
246 | // widget_v2s can still be constructed using v1 components,
247 | // so this can be either a string or an int
248 | element: { one_of: [string, int] }
249 | }
250 | }
251 | }
252 | `,
253 | type: "widget",
254 | value: `
255 | // Try validating this as widget, widget_latest, widget_v1, and widget_v2
256 | {
257 | name: "WidgetFoo",
258 | part_id: "177bfe43-e702-44a6-9625-f5eec025ec94",
259 | component_ids: [
260 | 1843,
261 | 623,
262 | "a890c9ca-1ed4-4f82-b1c7-272a50e256d1"
263 | ],
264 | }
265 | `,
266 | },
267 | nestedStructs: {
268 | displayName: "Nested structs",
269 | schema: `
270 | $ion_schema_2_0
271 | type::{
272 | name: non_negative_int,
273 | valid_values: range::[0, max],
274 | }
275 |
276 | type::{
277 | name: package_metadata,
278 | fields: closed::{
279 | component_namespace: {
280 | occurs: required,
281 | type: string,
282 | },
283 | component_name: {
284 | occurs: required,
285 | type: string,
286 | },
287 | version: {
288 | fields: closed::{
289 | major: non_negative_int,
290 | minor: non_negative_int,
291 | patch: non_negative_int,
292 | }
293 | },
294 | licenses: {
295 | // Expected to be a list of SPDX license identifiers
296 | occurs: required,
297 | type: list,
298 | container_length: range::[1, max],
299 | element: string,
300 | }
301 | }
302 | }`,
303 | type: "package_metadata",
304 | value: `
305 | {
306 | component_namespace: "com.amazon.ion",
307 | component_name: "ion-schema-kotlin",
308 | version: { major: 1, minor: 6, patch: 1 },
309 | licenses: ["Apache-2.0"],
310 | }
311 | `
312 | },
313 | }
314 |
315 | initPage();
316 |
--------------------------------------------------------------------------------
/rfcs/ion_schema_2_0/ion_schema_2_0.md:
--------------------------------------------------------------------------------
1 | # RFC: Ion Schema 2.0
2 |
3 |
4 | - [RFC: Ion Schema 2.0](#rfc-ion-schema-20)
5 | - [Summary](#summary)
6 | - [Motivation](#motivation)
7 | - [Description of Changes](#description-of-changes)
8 | - [Compatibility with Ion Schema 1.0](#compatibility-with-ion-schema-10)
9 | - [Appendix — Below the Line/Out of Scope for Ion Schema 2.0](#appendix--below-the-lineout-of-scope-for-ion-schema-20)
10 | - [Schema/Type versioning](#schematype-versioning)
11 | - [User-Defined Constraints](#user-defined-constraints)
12 | - [Other Language Features](#other-language-features)
13 | - [Code Generation](#code-generation)
14 |
15 |
16 | ## Summary
17 |
18 | This RFC proposes a new major version of the Ion Schema Language: **Ion Schema 2.0**.
19 |
20 | Ion Schema 2.0 will be fully interoperable with Ion Schema 1.0.
21 |
22 | Ion Schema 2.0 introduces the following changes:
23 |
24 | * [Ion Schema Language Versions](language_versions.md)
25 | * [Ion Schema 2.0 Open Content](open_content.md)
26 | * [ion-schema #18](https://github.com/amzn/ion-schema/issues/18#issuecomment-1092130146) – Add `ieee754_float` constraint
27 | * [ion-schema #27](https://github.com/amzn/ion-schema/issues/27) – Replace `scale` constraint with `exponent`
28 | * [ion-schema #38](https://github.com/amzn/ion-schema/issues/38#issuecomment-1119833602) – Replace `nullable::` modifier with `$null_or::`
29 | * [ion-schema #43](https://github.com/amzn/ion-schema/issues/43) – Add a way to require that containers have no duplicate elements
30 | * [ion-schema #44](https://github.com/amzn/ion-schema/issues/44#issuecomment-1097296761) – Allow field names to be constrained without exact field names being specified
31 | * [ion-schema #47](https://github.com/amzn/ion-schema/issues/47) – Replace `content: closed` with `closed::` modifier for `fields` constraint
32 | * [ion-schema #51](https://github.com/amzn/ion-schema/issues/51#issuecomment-1105688979) – Improve modeling of annotations in Ion Schema
33 | * [ion-schema #54](https://github.com/amzn/ion-schema/issues/54) – Add support for more ECMA 262 regex features
34 | * [ion-schema #58](https://github.com/amzn/ion-schema/issues/58) – Change the default type from `any` to `$any`
35 | * [ion-schema #72](https://github.com/amzn/ion-schema/issues/72) – Allow timestamp ranges to have endpoints with unknown offsets
36 |
37 | ## Motivation
38 |
39 | [Ion Schema Language](https://amzn.github.io/ion-schema/docs/spec.html) (ISL) is a grammar and constraints for narrowing the universe of Ion values. A schema consists of zero or more types, and a type is a collection of zero or more constraints over the Ion data model.
40 |
41 | Ion Schema Language is declarative and is intended to be portable across platforms and programming environments. Therefore, a schema document must have the same behavior on all equivalently configured Ion Schema implementations. If an implementation does not support all features used in a particular schema document, the implementation must error when it attempts to load that schema.
42 |
43 | As Ion Schema has been used by customers, customers have asked for some new functionality to address feature gaps, and we identified some misfeatures and points of ambiguity that needed to be addressed. However, we realized that it is impossible to evolve the Ion Schema Language in a way that preserves the portability/compatibility goal. This is because:
44 |
45 | * No specification about what requires a major or minor version change.
46 | * There is no way to tell the difference between open/user content and an unknown feature, so it is impossible to raise an error for an unsupported feature.
47 | * Every new constraint is a potentially breaking change because it could interfere with someone’s open content.
48 | * No specification about whether schemas with different ISL versions can import each other
49 |
50 | ## Description of Changes
51 |
52 | * **Versioning Rules for the Ion Schema Language**
53 | Ion Schema will use a major and minor version numbers. (This was implied in Ion Schema 1.0, but not well-defined.) Any backwards-incompatible change requires a new major version. Any other change to the Ion Schema Language requires a new minor version. Edits to the specification for spelling, grammar, or clarity (i.e. any changes that do not affect the ISL syntax or semantics) do not require a new version of Ion Schema. See [Ion Schema Language Versions](language_versions.md) for full details.
54 | * **Open Content in Ion Schema 2.0**
55 | Ion Schema 2.0 defines a set of symbols that are reserved for use by future versions of Ion Schema. A user may use a reserved symbol for open content if and only if that field is explicitly declared as user content in the schema header. See [Ion Schema 2.0 Open Content](open_content.md).
56 | * **Add `ieee754_float` constraint**
57 | If a user wants to model [binary32](https://en.wikipedia.org/wiki/Single-precision_floating-point_format) or [binary16](https://en.wikipedia.org/wiki/Half-precision_floating-point_format) ranges in Ion Schema 1.0, it cannot be done due to precision/scale aspects of binary floating-point. The `ieee754_float` constraint allows us to constraint float values to be only those values that are precisely representable by a specific IEEE 754 binary encoding. See [ion-schema #18](https://github.com/amzn/ion-schema/issues/18#issuecomment-1092130146).
58 | * **Replace `scale` constraint with `exponent` constraint**
59 | The `scale` constraint has some unexpected behavior and is at odds with the Ion Decimal data model. Ion Schema 2.0 replaces `scale` with `exponent`, which closely aligns with the Ion `decimal` data model. See [ion-schema #27](https://github.com/amzn/ion-schema/issues/27).
60 | * **Replace `nullable::` with `$null_or::`**
61 | The `nullable::` annotation is underspecified and behaves in unintuitive ways. A complete solution to correctly identify the allowed types of null is not practical to solve, so `nullable::` will be replaced with `$null_or::` which will be syntactical sugar for a union of `null` (equivalently `null.null`) and the annotated type. See [ion-schema #38](https://github.com/amzn/ion-schema/issues/38#issuecomment-1119833602).
62 | * **Add support for modeling a set**
63 | Ion Schema 1.0 provides no way to model a set. Ion Schema 2.0 introduces the modifier `distinct::` for the `element` constraint which validates that no two elements in a container may be equivalent Ion values. See [ion-schema #43](https://github.com/amzn/ion-schema/issues/43).
64 | * **Add `field_names` constraint**
65 | Ion Schema 1.0 provides no way to validate field names except by listing out explicitly which names are allowed in the `fields` constraint. The `field_names` constraint will allow the text of all field name symbols of a struct to be validated according to a specified type. See [ion-schema #44](https://github.com/amzn/ion-schema/issues/44#issuecomment-1097296761).
66 | * **Replace `content: closed` with `closed::` modifier for `fields` constraint.**
67 | The `content: closed` (pseudo) constraint only *modifies* the `fields` constraint and is confusing when composing types. It will be removed in Ion Schema 2.0 and replaced with a `closed::` modifier for the `fields` constraint. See [ion-schema #47](https://github.com/amzn/ion-schema/issues/47).
68 | * **Improve `annotations` constraint**
69 | In Ion Schema 1.0, the `annotations` constraint has some unexpected and useless behaviors, and it is not possible to model rules such as "1 of N annotations" or "any lowercase annotation". Ion Schema 2.0 eliminates the specific configurations that have confusing behavior, and adds syntax to allow annotations to be validated as if they are a `list` of `symbol`. See [ion-schema #51](https://github.com/amzn/ion-schema/issues/51#issuecomment-1105688979).
70 | * **Add additional regex support**
71 | Ion Schema 2.0 adds support for backslash-escaped character sets inside character classes to the subset of supported ECMA regex features. See [ion-schema #54](https://github.com/amzn/ion-schema/issues/54).
72 | * **Change implicit `type: any` to `type: $any`**
73 | In Ion Schema 1.0, the default `type` is `any` rather than the top type, `$any`. This leads to some unintuitive behavior when handling null values. Ion Schema 2.0 changes the default `type` of a type definition from `any` to `$any`. See [ion-schema #58](https://github.com/amzn/ion-schema/issues/58).
74 | * **Allow timestamp ranges to have endpoints with unknown offsets**
75 | Ion Schema 1.0 prohibits using timestamps with unknown offsets as the upper or lower bound of a timestamp range, which leads to an awkward user experience in some cases. Ion Schema 2.0 allows timestamp ranges to have boundaries with unknown offset. See [ion-schema #72](https://github.com/amzn/ion-schema/issues/72).
76 |
77 | ## Compatibility with Ion Schema 1.0
78 |
79 | *The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this section are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119).*
80 |
81 | Ion Schema 2.0 will be fully interoperable with Ion Schema 1.0.
82 |
83 | * Any Ion Schema implementation with support for Ion Schema 2.0 MUST also support reading Ion Schema 1.0.
84 | * Ion Schema 2.0 SHALL allow importing schemas written in Ion Schema 1.0.
85 | * Any implementation of ISL 1.0 that also implements ISL 2.0 MUST allow types from an ISL 2.0 schema to be imported by an ISL 1.0 schema.
86 | * The interoperability requirements stated here SHALL NOT apply to Ion Schema 3.0 or any future major version unless that major version explicitly restates them. (i.e., Ion Schema 3.0 is allowed to say that implementations must also support Ion Schema 2.0 but are not required to support Ion Schema 1.0.)
87 |
88 | ## Appendix — Below the Line/Out of Scope for Ion Schema 2.0
89 |
90 | ### Schema/Type versioning
91 |
92 | Some users have asked about schema versioning, but they have described multiple different versioning strategies. Based on the use cases that we have learned about, there does not appear to be a one-size-fits-all solution for schema versioning. Ion Schema will need to have its own versioning strategy for `ion-schema-schemas`, and we believe that it is possible via name mangling or a custom authority implementation. (One user had already indicated that name mangling is sufficient for their use case.)
93 |
94 | We plan to add documentation and examples to suggest some methods for managing multiple versions of schemas, but we do not plan to add any features to Ion Schema 2.0 specifically to support versioned schemas and/or types.
95 |
96 | See [ion-schema #66](https://github.com/amzn/ion-schema/issues/66).
97 |
98 | ### User-Defined Constraints
99 |
100 | We considered adding support for user-defined constraints. In Ion Schema 1.0, it is only possible to add a user-defined constraint by creating a custom Ion Schema implementation. In Ion Schema 2.0, this remains possible. Any user-defined constraint would be treated as open content by an implementation that does not support that user-defined constraint.
101 |
102 | However, it would be possible to introduce a feature in Ion Schema Language that would allow users to indicate that certain open content fields are actually a user-defined constraint.
103 |
104 | Ultimately, we chose to defer this because we are not sure if it is the right thing to do, and we believe we can safely introduce it in a new _minor_ version of the Ion Schema Language by extending the open-content mechanism proposed in [Open Content](open_content.md).
105 |
106 | See [ion-schema #68](https://github.com/amzn/ion-schema/issues/68) for spec support for user-defined constraints and [ion-schema-kotlin #33](https://github.com/amzn/ion-schema-kotlin/issues/33) for API support of user-defined constraints.
107 |
108 | ### Other Language Features
109 |
110 | There are several features that we thought of, but they were ultimately excluded because no one has actually asked for the feature (yet). All of these features could safely be introduced in a later *minor* version of Ion Schema, so excluding them is not a one-way door.
111 |
112 | * Add a way to indicate that a particular type cannot be imported by any other schema. See [ion-schema #67](https://github.com/amzn/ion-schema/issues/67).
113 | * Add a way to configure whether number-related constraints should be strict about the Ion type of the number or whether they should use a numerical/mathematical equivalence. See [ion-schema #65](https://github.com/amzn/ion-schema/issues/65).
114 | * Add a default type for schema document so that users don’t need to make seemingly redundant calls to `getType()` in (for example) `loadSchema("foo.isl").getType("foo").validate(ionValue)`. See [ion-schema #15](https://github.com/amzn/ion-schema/issues/15).
115 | * Add a constraint for timestamps that allows for constraining individual fields of the timestamp (e.g. minutes must be a multiple of 5). See [ion-schema #46](https://github.com/amzn/ion-schema/issues/46).
116 | * Add syntax and support for repeated subsequences in the `ordered_elements` constraint. See [ion-schema #41](https://github.com/amzn/ion-schema/issues/41).
117 |
118 | ### Code Generation
119 |
120 | We would like to create tools for generating code from an Ion Schema, but we concluded it is an orthogonal concern that does not belong in the Ion Schema Language specification. You can comment on code generation for JVM in [ion-schema-kotlin #146](https://github.com/amzn/ion-schema-kotlin/issues/146) or C/C++ code generation in [ion-schema #49](https://github.com/amzn/ion-schema/issues/49), or [open a new issue](https://github.com/amzn/ion-schema/issues/new) for a new code generation target.
121 |
--------------------------------------------------------------------------------
/assets/wasm_ion_schema.js:
--------------------------------------------------------------------------------
1 | let wasm;
2 |
3 | const heap = new Array(128).fill(undefined);
4 |
5 | heap.push(undefined, null, true, false);
6 |
7 | function getObject(idx) { return heap[idx]; }
8 |
9 | let heap_next = heap.length;
10 |
11 | function dropObject(idx) {
12 | if (idx < 132) return;
13 | heap[idx] = heap_next;
14 | heap_next = idx;
15 | }
16 |
17 | function takeObject(idx) {
18 | const ret = getObject(idx);
19 | dropObject(idx);
20 | return ret;
21 | }
22 |
23 | function addHeapObject(obj) {
24 | if (heap_next === heap.length) heap.push(heap.length + 1);
25 | const idx = heap_next;
26 | heap_next = heap[idx];
27 |
28 | heap[idx] = obj;
29 | return idx;
30 | }
31 |
32 | const cachedTextDecoder = new TextDecoder('utf-8', { ignoreBOM: true, fatal: true });
33 |
34 | cachedTextDecoder.decode();
35 |
36 | let cachedUint8Memory0 = null;
37 |
38 | function getUint8Memory0() {
39 | if (cachedUint8Memory0 === null || cachedUint8Memory0.byteLength === 0) {
40 | cachedUint8Memory0 = new Uint8Array(wasm.memory.buffer);
41 | }
42 | return cachedUint8Memory0;
43 | }
44 |
45 | function getStringFromWasm0(ptr, len) {
46 | return cachedTextDecoder.decode(getUint8Memory0().subarray(ptr, ptr + len));
47 | }
48 |
49 | function debugString(val) {
50 | // primitive types
51 | const type = typeof val;
52 | if (type == 'number' || type == 'boolean' || val == null) {
53 | return `${val}`;
54 | }
55 | if (type == 'string') {
56 | return `"${val}"`;
57 | }
58 | if (type == 'symbol') {
59 | const description = val.description;
60 | if (description == null) {
61 | return 'Symbol';
62 | } else {
63 | return `Symbol(${description})`;
64 | }
65 | }
66 | if (type == 'function') {
67 | const name = val.name;
68 | if (typeof name == 'string' && name.length > 0) {
69 | return `Function(${name})`;
70 | } else {
71 | return 'Function';
72 | }
73 | }
74 | // objects
75 | if (Array.isArray(val)) {
76 | const length = val.length;
77 | let debug = '[';
78 | if (length > 0) {
79 | debug += debugString(val[0]);
80 | }
81 | for(let i = 1; i < length; i++) {
82 | debug += ', ' + debugString(val[i]);
83 | }
84 | debug += ']';
85 | return debug;
86 | }
87 | // Test for built-in
88 | const builtInMatches = /\[object ([^\]]+)\]/.exec(toString.call(val));
89 | let className;
90 | if (builtInMatches.length > 1) {
91 | className = builtInMatches[1];
92 | } else {
93 | // Failed to match the standard '[object ClassName]'
94 | return toString.call(val);
95 | }
96 | if (className == 'Object') {
97 | // we're a user defined class or Object
98 | // JSON.stringify avoids problems with cycles, and is generally much
99 | // easier than looping through ownProperties of `val`.
100 | try {
101 | return 'Object(' + JSON.stringify(val) + ')';
102 | } catch (_) {
103 | return 'Object';
104 | }
105 | }
106 | // errors
107 | if (val instanceof Error) {
108 | return `${val.name}: ${val.message}\n${val.stack}`;
109 | }
110 | // TODO we could test for more things here, like `Set`s and `Map`s.
111 | return className;
112 | }
113 |
114 | let WASM_VECTOR_LEN = 0;
115 |
116 | const cachedTextEncoder = new TextEncoder('utf-8');
117 |
118 | const encodeString = (typeof cachedTextEncoder.encodeInto === 'function'
119 | ? function (arg, view) {
120 | return cachedTextEncoder.encodeInto(arg, view);
121 | }
122 | : function (arg, view) {
123 | const buf = cachedTextEncoder.encode(arg);
124 | view.set(buf);
125 | return {
126 | read: arg.length,
127 | written: buf.length
128 | };
129 | });
130 |
131 | function passStringToWasm0(arg, malloc, realloc) {
132 |
133 | if (realloc === undefined) {
134 | const buf = cachedTextEncoder.encode(arg);
135 | const ptr = malloc(buf.length);
136 | getUint8Memory0().subarray(ptr, ptr + buf.length).set(buf);
137 | WASM_VECTOR_LEN = buf.length;
138 | return ptr;
139 | }
140 |
141 | let len = arg.length;
142 | let ptr = malloc(len);
143 |
144 | const mem = getUint8Memory0();
145 |
146 | let offset = 0;
147 |
148 | for (; offset < len; offset++) {
149 | const code = arg.charCodeAt(offset);
150 | if (code > 0x7F) break;
151 | mem[ptr + offset] = code;
152 | }
153 |
154 | if (offset !== len) {
155 | if (offset !== 0) {
156 | arg = arg.slice(offset);
157 | }
158 | ptr = realloc(ptr, len, len = offset + arg.length * 3);
159 | const view = getUint8Memory0().subarray(ptr + offset, ptr + len);
160 | const ret = encodeString(arg, view);
161 |
162 | offset += ret.written;
163 | }
164 |
165 | WASM_VECTOR_LEN = offset;
166 | return ptr;
167 | }
168 |
169 | let cachedInt32Memory0 = null;
170 |
171 | function getInt32Memory0() {
172 | if (cachedInt32Memory0 === null || cachedInt32Memory0.byteLength === 0) {
173 | cachedInt32Memory0 = new Int32Array(wasm.memory.buffer);
174 | }
175 | return cachedInt32Memory0;
176 | }
177 | /**
178 | * @param {string} ion
179 | * @param {string} schema
180 | * @param {string} schema_type
181 | * @param {boolean} is_document
182 | * @returns {SchemaValidationResult}
183 | */
184 | export function validate(ion, schema, schema_type, is_document) {
185 | const ptr0 = passStringToWasm0(ion, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
186 | const len0 = WASM_VECTOR_LEN;
187 | const ptr1 = passStringToWasm0(schema, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
188 | const len1 = WASM_VECTOR_LEN;
189 | const ptr2 = passStringToWasm0(schema_type, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
190 | const len2 = WASM_VECTOR_LEN;
191 | const ret = wasm.validate(ptr0, len0, ptr1, len1, ptr2, len2, is_document);
192 | return SchemaValidationResult.__wrap(ret);
193 | }
194 |
195 | /**
196 | */
197 | export class SchemaValidationResult {
198 |
199 | static __wrap(ptr) {
200 | const obj = Object.create(SchemaValidationResult.prototype);
201 | obj.ptr = ptr;
202 |
203 | return obj;
204 | }
205 |
206 | __destroy_into_raw() {
207 | const ptr = this.ptr;
208 | this.ptr = 0;
209 |
210 | return ptr;
211 | }
212 |
213 | free() {
214 | const ptr = this.__destroy_into_raw();
215 | wasm.__wbg_schemavalidationresult_free(ptr);
216 | }
217 | /**
218 | * @param {boolean} r
219 | * @param {Array} v
220 | * @param {string} val
221 | * @param {boolean} has_error
222 | * @param {string} error
223 | */
224 | constructor(r, v, val, has_error, error) {
225 | const ptr0 = passStringToWasm0(val, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
226 | const len0 = WASM_VECTOR_LEN;
227 | const ptr1 = passStringToWasm0(error, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
228 | const len1 = WASM_VECTOR_LEN;
229 | const ret = wasm.schemavalidationresult_new(r, addHeapObject(v), ptr0, len0, has_error, ptr1, len1);
230 | return SchemaValidationResult.__wrap(ret);
231 | }
232 | /**
233 | * @returns {boolean}
234 | */
235 | result() {
236 | const ret = wasm.schemavalidationresult_result(this.ptr);
237 | return ret !== 0;
238 | }
239 | /**
240 | * @param {boolean} val
241 | */
242 | set_result(val) {
243 | wasm.schemavalidationresult_set_result(this.ptr, val);
244 | }
245 | /**
246 | * @returns {string}
247 | */
248 | value() {
249 | try {
250 | const retptr = wasm.__wbindgen_add_to_stack_pointer(-16);
251 | wasm.schemavalidationresult_value(retptr, this.ptr);
252 | var r0 = getInt32Memory0()[retptr / 4 + 0];
253 | var r1 = getInt32Memory0()[retptr / 4 + 1];
254 | return getStringFromWasm0(r0, r1);
255 | } finally {
256 | wasm.__wbindgen_add_to_stack_pointer(16);
257 | wasm.__wbindgen_free(r0, r1);
258 | }
259 | }
260 | /**
261 | * @param {string} val
262 | */
263 | set_value(val) {
264 | const ptr0 = passStringToWasm0(val, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
265 | const len0 = WASM_VECTOR_LEN;
266 | wasm.schemavalidationresult_set_value(this.ptr, ptr0, len0);
267 | }
268 | /**
269 | * @returns {string}
270 | */
271 | error() {
272 | try {
273 | const retptr = wasm.__wbindgen_add_to_stack_pointer(-16);
274 | wasm.schemavalidationresult_error(retptr, this.ptr);
275 | var r0 = getInt32Memory0()[retptr / 4 + 0];
276 | var r1 = getInt32Memory0()[retptr / 4 + 1];
277 | return getStringFromWasm0(r0, r1);
278 | } finally {
279 | wasm.__wbindgen_add_to_stack_pointer(16);
280 | wasm.__wbindgen_free(r0, r1);
281 | }
282 | }
283 | /**
284 | * @param {string} val
285 | */
286 | set_error(val) {
287 | const ptr0 = passStringToWasm0(val, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
288 | const len0 = WASM_VECTOR_LEN;
289 | wasm.schemavalidationresult_set_error(this.ptr, ptr0, len0);
290 | }
291 | /**
292 | * @returns {boolean}
293 | */
294 | has_error() {
295 | const ret = wasm.schemavalidationresult_has_error(this.ptr);
296 | return ret !== 0;
297 | }
298 | /**
299 | * @param {boolean} val
300 | */
301 | set_has_error(val) {
302 | wasm.schemavalidationresult_set_has_error(this.ptr, val);
303 | }
304 | /**
305 | * @returns {Array}
306 | */
307 | violations() {
308 | const ret = wasm.schemavalidationresult_violations(this.ptr);
309 | return takeObject(ret);
310 | }
311 | }
312 |
313 | async function load(module, imports) {
314 | if (typeof Response === 'function' && module instanceof Response) {
315 | if (typeof WebAssembly.instantiateStreaming === 'function') {
316 | try {
317 | return await WebAssembly.instantiateStreaming(module, imports);
318 |
319 | } catch (e) {
320 | if (module.headers.get('Content-Type') != 'application/wasm') {
321 | console.warn("`WebAssembly.instantiateStreaming` failed because your server does not serve wasm with `application/wasm` MIME type. Falling back to `WebAssembly.instantiate` which is slower. Original error:\n", e);
322 |
323 | } else {
324 | throw e;
325 | }
326 | }
327 | }
328 |
329 | const bytes = await module.arrayBuffer();
330 | return await WebAssembly.instantiate(bytes, imports);
331 |
332 | } else {
333 | const instance = await WebAssembly.instantiate(module, imports);
334 |
335 | if (instance instanceof WebAssembly.Instance) {
336 | return { instance, module };
337 |
338 | } else {
339 | return instance;
340 | }
341 | }
342 | }
343 |
344 | function getImports() {
345 | const imports = {};
346 | imports.wbg = {};
347 | imports.wbg.__wbindgen_object_drop_ref = function(arg0) {
348 | takeObject(arg0);
349 | };
350 | imports.wbg.__wbindgen_object_clone_ref = function(arg0) {
351 | const ret = getObject(arg0);
352 | return addHeapObject(ret);
353 | };
354 | imports.wbg.__wbindgen_string_new = function(arg0, arg1) {
355 | const ret = getStringFromWasm0(arg0, arg1);
356 | return addHeapObject(ret);
357 | };
358 | imports.wbg.__wbg_set_20cbc34131e76824 = function(arg0, arg1, arg2) {
359 | getObject(arg0)[takeObject(arg1)] = takeObject(arg2);
360 | };
361 | imports.wbg.__wbg_log_7bb108d119bafbc1 = function(arg0) {
362 | console.log(getObject(arg0));
363 | };
364 | imports.wbg.__wbg_new_b525de17f44a8943 = function() {
365 | const ret = new Array();
366 | return addHeapObject(ret);
367 | };
368 | imports.wbg.__wbg_new_f9876326328f45ed = function() {
369 | const ret = new Object();
370 | return addHeapObject(ret);
371 | };
372 | imports.wbg.__wbg_push_49c286f04dd3bf59 = function(arg0, arg1) {
373 | const ret = getObject(arg0).push(getObject(arg1));
374 | return ret;
375 | };
376 | imports.wbg.__wbindgen_debug_string = function(arg0, arg1) {
377 | const ret = debugString(getObject(arg1));
378 | const ptr0 = passStringToWasm0(ret, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
379 | const len0 = WASM_VECTOR_LEN;
380 | getInt32Memory0()[arg0 / 4 + 1] = len0;
381 | getInt32Memory0()[arg0 / 4 + 0] = ptr0;
382 | };
383 | imports.wbg.__wbindgen_throw = function(arg0, arg1) {
384 | throw new Error(getStringFromWasm0(arg0, arg1));
385 | };
386 |
387 | return imports;
388 | }
389 |
390 | function initMemory(imports, maybe_memory) {
391 |
392 | }
393 |
394 | function finalizeInit(instance, module) {
395 | wasm = instance.exports;
396 | init.__wbindgen_wasm_module = module;
397 | cachedInt32Memory0 = null;
398 | cachedUint8Memory0 = null;
399 |
400 |
401 | return wasm;
402 | }
403 |
404 | function initSync(module) {
405 | const imports = getImports();
406 |
407 | initMemory(imports);
408 |
409 | if (!(module instanceof WebAssembly.Module)) {
410 | module = new WebAssembly.Module(module);
411 | }
412 |
413 | const instance = new WebAssembly.Instance(module, imports);
414 |
415 | return finalizeInit(instance, module);
416 | }
417 |
418 | async function init(input) {
419 | if (typeof input === 'undefined') {
420 | input = new URL('wasm_ion_schema_bg.wasm', import.meta.url);
421 | }
422 | const imports = getImports();
423 |
424 | if (typeof input === 'string' || (typeof Request === 'function' && input instanceof Request) || (typeof URL === 'function' && input instanceof URL)) {
425 | input = fetch(input);
426 | }
427 |
428 | initMemory(imports);
429 |
430 | const { instance, module } = await load(await input, imports);
431 |
432 | return finalizeInit(instance, module);
433 | }
434 |
435 | export { initSync }
436 | export default init;
437 |
--------------------------------------------------------------------------------
/docs/cookbook/ion-schema-rust-getting-started.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: Getting started with `ion-schema-rust`
3 | ---
4 | # {{ page.title }}
5 | _(Applies to Ion Schema 1.0.)_
6 |
7 | This is a getting started cookbook for `ion-schema-rust`. It includes all the examples for using `ion-schema-rust` like loading a schema, using Ion schema to validate Ion values, etc.
8 |
9 | * [How to use this cookbook?](#how-to-use-this-cookbook)
10 | * [Terms](#terms)
11 | * [How to create a `SchemaSystem`?](#how-to-create-a-schemasystem)
12 | * [Creating `DocumentAuthority`](#creating-documentauthority)
13 | * [How to create `FileSystemDocumentAuthority`](#how-to-create-filesystemdocumentauthority)
14 | * [How to create `MapDocumentAuthority`](#how-to-create-mapdocumentauthority)
15 | * [`DocumentAuthority` operations on schema](#documentauthority-operations-on-a-schemasystem)
16 | * [How to get all the authorities that are there for a given `SchemaSystem`?](#how-to-get-all-the-authorities-that-are-there-for-a-given-schemasystem)
17 | * [How to add new authority into a `SchemaSystem`?](#how-to-add-new-authority-into-a-schemasystem)
18 | * [How to create a `SchemsSystem` with given authority or list of authorities?](#how-to-create-a-schemasystem-with-given-authority-or-list-of-authorities)
19 | * [How to load a `Schema`?](#how-to-load-a-schema)
20 | * [Loading a schema](#loading-a-schema)
21 | * [How to create a schema programmatically(`IslSchema`)?](#how-to-create-a-schema-programmaticallyislschema)
22 | * [How to create an `IslType`?](#how-to-definecreate-an-isltype)
23 | * [How to create `IslSchema`?](#how-to-create-islschema)
24 | * [How to validate an Ion value using a `Schema`?](#how-to-validate-an-ion-value-using-a-schema)
25 |
26 | ## How to use this cookbook?
27 | This cookbook explains how to create all the necessary structures required to perform Ion schema validation. It gives sample code with each section explaining the usage of that operation or structure.
28 | In general, the process of Ion schema validation contains two steps.
29 | 1. The first step is to load a correct Ion schema. In this step, we use a `SchemaSystem` which verifies the syntax of given schema file for its correctness.
30 | 2. The second step is to use this generated schema from #1 and validate Ion values using the type definitions defined within the schema.
31 |
32 | This cookbook starts from examples for #1 (including examples for all the required structures to be created to complete #1) and at the end adds examples for #2 (How to validate Ion value using `ion-schema-rust`)
33 |
34 | ## Terms
35 |
36 | * **Schema Id**: A unique identifier provided to each schema.
37 | * **Authority:** Authority is responsible for resolving particular class of schema identifiers. One example of authority is file system authority which resolves schema ids to files relative to a base path.
38 | * **Schema System:** Provides methods for instantiating instances of Schema. Requests each of the provided Authorities, in order to resolve the requested schema id until one successfully resolves it.
39 | * **Schema:** Schema is a collection of 0 or more Types. It contains an optional header, type definitions/ types and an optional footer. The schema header can optionally contain imports for that schema.
40 | * **Import:** An Import allows types from other schemas to be used within a schema definition. These imports are usually defined in the schema header.
41 |
42 | ## How to create a `SchemaSystem`?
43 |
44 | `ion-schema-rust` requires you to create an `SchemaSystem` in order to load a schema from given file location.
45 | Hence, Creating a `SchemaSystem` is the first step to loading a schema file for validation.
46 |
47 | ### Creating `DocumentAuthority`:
48 |
49 | In general, users will create a `DocumentAuthority` and then use it to build the `SchemaSystem`. Then this `SchemaSystem` is used instantiating instances of `Schema`.
50 | There are two types of `DocumentAuthority` available:
51 |
52 | * `FileSystemDocumentAuthority`
53 | * This authority allows to specify the base path to where all the schema files are saved (e.g. `/home/USER/schemas/`)
54 | * Adding this authority your `SchemaSystem` would allow to resolve all schema ids (relative path to schema file, e.g. `my_schema.isl`) that are within this authority’s base path (e.g. `/home/USER/schemas/`)
55 | * In order to load a schema id (e.g. `my_schema.isl`) with this `FileSystemDocumentAuthority` that is added to your `SchemaSystem` would be as simple as: `schema_system.load_schema("my_schema.isl")`
56 | * `MapDocumentAuthority`
57 | * This authority allows to specify a `HashMap` of schema ids as a key and the schema as value.
58 | * Adding this authority your `SchemaSystem` would allow to resolve all schema ids that are within this authority’s map keys.
59 | * In order to load a schema id (e.g. `my_schema`) with this `FileSystemDocumentAuthority` that is added to your `SchemaSystem` would be as simple as: `schema_system.load_schema("my_schema")`
60 |
61 | _Note: A single `SchemaSystem` can contain multiple `DocumentAuthority`s_
62 |
63 | ### How to create `FileSystemDocumentAuthority`?
64 | Creating a `FileSystemAuthority` requires to pass a base path where all the schema files resides.
65 | ```rust
66 | // In this example, it is assumed that all the schema files that will
67 | // later be used to load a schema are inside `sample_schemas` folder
68 |
69 | let file_system_document_authority = FileSystemDocumentAuthority::new(Path::new(
70 | "sample_schemas",
71 | ));
72 | ```
73 |
74 | ### How to create `MapDocumentAuthority`?
75 | Creating a `MapDocumentAuthority` requires to pass a `HashMap` with key-value pair of `(schema_id, schema content)`.
76 | ```rust
77 | // map with (schema id, schema content) to represent `sample_number` schema
78 | let map_authority = [
79 | (
80 | "sample_number.isl", // <--- schema id
81 | // Ion schema as raw String
82 | // |
83 | // v
84 | r#"
85 | schema_header::{
86 | imports: [{ id: "sample_decimal.isl", type: my_decimal, as: other_decimal }],
87 | }
88 | type::{
89 | name: my_int,
90 | type: int,
91 | }
92 | type::{
93 | name: my_number,
94 | all_of: [
95 | my_int,
96 | other_decimal,
97 | ],
98 | }
99 | schema_footer::{
100 | }
101 | "#,
102 | ),
103 | (
104 | "sample_decimal.isl",
105 | r#"
106 | schema_header::{
107 | imports: [],
108 | }
109 | type::{
110 | name: my_decimal,
111 | type: decimal,
112 | }
113 | schema_footer::{
114 | }
115 | "#,
116 | ),
117 | ];
118 |
119 | // Create a MapDocumentAuthority using a map like above with
120 | // schema id as key and schema as value
121 | let map_document_authority = MapDocumentAuthority::new(map_authority);
122 | ```
123 |
124 | Finally, next code block shows how to create a `SchemSystem` using `FileSystemDocumentAuthority`
125 |
126 | ```rust
127 | // Create authorities vector containing all the authorities that will be used to load a schema based on schema id
128 | let document_authorities: Vec> = vec![Box::new(
129 | FileSystemDocumentAuthority::new(Path::new("sample_schemas")), // <--- provide a path to the authority base folder containing schemas
130 | )];
131 |
132 | // Create a new schema system using given document authorities
133 | let mut schema_system = SchemaSystem::new(document_authorities);
134 | ```
135 |
136 | ## `DocumentAuthority` operations on a `SchemaSystem`
137 |
138 | ### How to get all the authorities that are there for a given `SchemaSystem`?
139 | If you want to check which authorities do your `SchemaSystem` contain you can perform following operation:
140 |
141 | ```rust
142 | // assuming the SchemSystem in built into variable: `schema_system`
143 | let result = schema_system.authorities()
144 | ```
145 |
146 | Output of the above operation will have following vector saved inside `result`:
147 | ```rust
148 | vec![Box::new(
149 | FileSystemDocumentAuthority::new(Path::new("sample_schemas")),
150 | )]
151 | ```
152 |
153 | ### How to add new authority into a `SchemaSystem`?
154 | As your `SchemaSystem` starts growing, it might be possible that you would want to add a new authority to the `SchemaSystem`.
155 | For example, you now have a new place where all your schemas are saved. This means you have new `FileSystemAuthority` to add to your `SchemaSystem`.
156 | Adding a new authority to the `SchemaSystem` expands the search area for loading a schema from `SchemaSystem`.
157 | When attempting to resolve a schemaId, `Authority`s are checked in the order in which they were added to the `SchemaSystem`,
158 | so any new `Authority` is added with a lower priority than the `Authority`s that are already present.
159 |
160 | ```rust
161 | // assuming the SchemSystem is built into variable: `schema_system`
162 | // following operation adds new authority with base path `tests` into `schema_system`
163 | schema_system.add_authority(Box::new(FileSystemDocumentAuthority::new(Path::new("test"))));
164 | ```
165 |
166 | ### How to create a `SchemaSystem` with given authority or list of authorities?
167 |
168 | ```rust
169 | // assuming the SchemSystem in built into variable: `schema_system`
170 | // Creating a SchemaSystem with given authority would return a new SchemsSystem
171 | // following operation creates a `new_schema_system` with an authority
172 | // that has base path `tests`
173 | let new_schema_system = schema_system.with_authority(
174 | Box::new(FileSystemDocumentAuthority::new(Path::new("test")))
175 | );
176 |
177 | // For creating an SchemaSystem with given list of authorities
178 | let new_schema_system = schema_system.with_authorities(vec![
179 | Box::new(FileSystemDocumentAuthority::new(Path::new("test"))),
180 | Box::new(FileSystemDocumentAuthority::new(Path::new("ion"))),
181 | ]);
182 | ```
183 |
184 | ## How to load a `Schema`?
185 |
186 | ### Example schema `my_schema.isl`
187 |
188 | This file (`my_schema.isl`) defines a new type (`my_int_type`) based on Ion's int type.
189 |
190 | ```
191 | schema_header::{
192 | imports: [],
193 | }
194 |
195 | type::{
196 | name: my_int_type,
197 | type: int,
198 | }
199 |
200 | schema_footer::{
201 | }
202 | ```
203 |
204 | ### Loading a schema
205 |
206 | ```rust
207 | use ion_schema::authority::{DocumentAuthority, FileSystemDocumentAuthority};
208 | use ion_schema::external::ion_rs::value::owned::OwnedElement;
209 | use ion_schema::result::{ValidationResult, IonSchemaResult};
210 | use ion_schema::types::TypeRef;
211 | use ion_schema::schema::Schema;
212 | use ion_schema::system::SchemaSystem;
213 | use std::path::Path;
214 | use std::rc::Rc;
215 |
216 | fn main() -> IonSchemaResult<()> {
217 | // Create authorities vector containing all the authorities that will be used to load a schema based on schema id
218 | let document_authorities: Vec> = vec![Box::new(
219 | FileSystemDocumentAuthority::new(Path::new("schema")), // provide a path to the authority base folder containing schemas
220 | )];
221 |
222 | // Create a new schema system from given document authorities
223 | let mut schema_system = SchemaSystem::new(document_authorities);
224 |
225 | // Provide schema id for the schema you want to load (schema_id is the schema file name here)
226 | let schema_id = "my_schema.isl";
227 |
228 | // Load schema
229 | let schema: Rc = schema_system.load_schema(schema_id)?;
230 | }
231 | ```
232 |
233 | ## How to create a schema programmatically(`IslSchema`)?
234 |
235 | Programmatic construction of Ion schema refers to an internal model representation of Ion schema and it mostly resembles to the [grammar](https://amzn.github.io/ion-schema/docs/spec.html#grammar) specified in Ion schema spec.
236 |
237 | ### How to define/create an `IslType`?
238 | For an Ion schema type definition like:
239 | ```
240 | type:: {
241 | name:my_type_name,
242 | type: int,
243 | all_of: [
244 | { type: bool }
245 | ]
246 | }
247 | ```
248 | The `IslType` can be created as shown below:
249 | ```rust
250 | let isl_type = IslType::named(
251 | // represents the `name` of the defined type
252 | "my_type_name".to_owned(),
253 | vec![
254 | // represents the `type: int` constraint
255 | IslConstraint::type_constraint(
256 | IslTypeRef::named("int")
257 | ),
258 | // represents `all_of` with anonymous type `{ type: bool }` constraint
259 | IslConstraint::all_of(
260 | vec![
261 | IslTypeRef::anonymous(
262 | vec![
263 | IslConstraint::type_constraint(
264 | IslTypeRef::named("bool")
265 | )
266 | ]
267 | )
268 | ]
269 | )
270 | ]
271 | );
272 | ```
273 |
274 | ### How to create `IslSchema`?
275 |
276 | ```rust
277 | // The `isl-type` defined in the previous section can used here
278 | let isl_schema = IslSchema::new(vec![], vec![isl_type], vec![]);
279 | ```
280 |
281 | ## How to validate an Ion value using a `Schema`?
282 |
283 | _Note: The schema file used in this section is previously defined in `How to load a schema?` section_
284 |
285 | ```rust
286 | // This example uses a schema that was created using how to load a schema section (`my_schema.isl`)?
287 | // Retrieve a particular type from this schema
288 | let type_ref: TypeRef = schema.get_type("my_int_type").unwrap();
289 |
290 | let valid_element: OwnedElement = 5.into();
291 | let invalid_element: OwnedElement = 5e3.into();
292 | let invalid_document_element: Vec = vec![5.into(), true.into(), 6e3.into()];
293 |
294 | // Validate data based on the type: 'my_int_type'
295 | check_value(&valid_element, &type_ref); // this validation passes as the value satisfies integer type constraint
296 | check_value(&invalid_element, &type_ref); // this returns violation as 'my_int_type' expects an integer value
297 | check_value(&invalid_document_element, &type_ref); // this returns violation as 'my_int_type' expects an integer value
298 |
299 |
300 | // Verify if the given value is valid and print violation for invalid value
301 | fn check_value + Debug + Clone>(value: I, type_ref: &TypeRef) {
302 | let validation_result: ValidationResult = type_ref.validate(value.to_owned());
303 | if let Err(violation) = validation_result {
304 | println!("{}", value.into());
305 | println!("{:#?}", violation);
306 | }
307 | }
308 | ```
309 |
310 | ### Output
311 | When run, the code above produces the following output:
312 | ```
313 | 5e3
314 | Violation {
315 | constraint: "my_int_type",
316 | code: TypeConstraintsUnsatisfied,
317 | message: "value didn't satisfy type constraint(s)",
318 | violations: [
319 | Violation {
320 | constraint: "type_constraint",
321 | code: TypeMismatched,
322 | message: "expected type Integer, found Float",
323 | violations: [],
324 | },
325 | ],
326 | }
327 | /* Ion document */ 5 true 6e3 /* end */
328 | Violation {
329 | constraint: "my_int_type",
330 | code: TypeConstraintsUnsatisfied,
331 | message: "value didn't satisfy type constraint(s)",
332 | violations: [
333 | Violation {
334 | constraint: "type_constraint",
335 | code: TypeMismatched,
336 | message: "expected type Integer, found document",
337 | violations: [],
338 | },
339 | ],
340 | }
341 | ```
342 |
343 |
--------------------------------------------------------------------------------
/docs/process-for-changing-ion-schema.md:
--------------------------------------------------------------------------------
1 | # Process for changing the Ion Schema Specification
2 |
3 | This document describes the process to evolve the Ion Schema Language. It only applies to the Ion Schema Language itself; feature proposals for each of the Ion Schema libraries and related projects are out of scope.
4 |
5 | Without Ion Schema specification versions, almost any functional change to the Ion Schema specification is a potentially breaking change for Ion Schema users. When new features are added to the Ion Schema specification, prior releases of Ion Schema implementations will not have support for those features, causing a schema document to be interpreted differently depending on what version of an Ion Schema library is being used. To ensure that a schema document has consistent behavior across all releases of all implementations, new features are grouped into new versions of the Ion Schema specification. (For more details about Ion Schema versions, see [ISL Versioning](https://amazon-ion.github.io/ion-schema/docs/isl-versioning).)
6 |
7 | ## Ion Schema Language Evolution Goals
8 |
9 | The following are the goals that inform the rest of this document.
10 |
11 | * The design work for Ion Schema should be open source.
12 | * The process should be lightweight and should not be an encumbrance to evolving Ion Schema.
13 | * Don’t let perfect be the enemy of good—the most important part of vetting a proposed change is ensuring that it is *not bad*.
14 | * The design of Ion Schema should be *community-driven*. That means that if community members would like to contribute a feature that fits with the stated design guidelines of Ion Schema, the default response should be to accept it unless there is a critical and well documented reason not to.
15 |
16 | ## Ion Schema Design Guidelines
17 |
18 | Loosely based on the [original ISL 1.0 rationale](https://amazon-ion.github.io/ion-schema/docs/isl-1-0/spec#rationale), they are:
19 |
20 | * Follow the principle of least surprise
21 | * Have a minimal set of orthogonal constraints
22 | * Constraints must be subtractive—if you add a constraint to a type, the valid values for the new type must be equal to or subset of the valid values for the original type. (In other words, for every type `t` and every constraint `c`, it MUST hold that for every Ion value `x`, if `x` is valid for `t ∪ c`, then it is also valid for `t`.)
23 | * ISL only constrains the Ion *data model*. (Be careful not to confuse the encoding of a value with the value itself, but it is acceptable to test whether a value *could* be encoded a particular way. E.g. `uft8_byte_length` and `ieee754_float`.)
24 | * ISL should be safe—it should be safe to use Ion Schema libraries with schemas and/or data from untrusted sources. (One implication, for example, is Ion Schema features should not permit arbitrary code execution or create any unmitigable attack vectors.)
25 |
26 | ## Roles
27 |
28 | The following are key actors in the process for evolving the Ion Schema Language. A person may have more than one role.
29 |
30 | * **Contributor:** a person who contributes code, ideas, documentation, or anything else to Ion Schema.
31 | * **Maintainer**: a contributor who is a member of the group that has final decision-making authority for matters of design and implementation of Ion Schema and is responsible for the ongoing development of Ion Schema. For now, this is the Ion Team.
32 | * **Feature Requestor**: a contributor who requests a feature. Usually “the feature requestor” used in reference to a specific feature.
33 |
34 | # The Process
35 |
36 | ## Summary
37 |
38 | * A **feature requestor** submits an issue suggesting an idea or describing a problem.
39 | * Ion Schema **maintainers** informally decide whether it is a good idea (and explain, with adequate justification, why or why not the idea should be included).
40 | * The **feature requestor** or another **contributor** formalizes the details of the feature in an RFC.
41 | * The **maintainers** announce a public comment period. (The public has been able to see and comment on the PR this whole time, but this step is an official, time-bounded, public comment period intended to bring closure to the proposal.)
42 | * If accepted, the **maintainers** will merge the RFC PR.
43 | * Finally, the feature will be adopted by an Ion Schema Version RFC.
44 |
45 | The process is intended to be iterative, so it is always possible to go to a prior step instead of rejecting a proposal outright.
46 |
47 | ## 1. Open an issue on the `ion-schema` GitHub repository
48 |
49 | *The purpose of this step is to decide whether a proposal is, in principle, a good idea for Ion Schema.*
50 |
51 | In the issue description, provide a high-level description of the feature you would like to add to the Ion Schema specification. Describe your use cases, and explain why the current version of Ion Schema fails to address them. List any alternative solutions you may have considered and the reasons that they were dismissed.
52 |
53 | *DO NOT INCLUDE ANY PROPRIETARY INFORMATION IN YOUR REQUEST.*
54 |
55 | This GitHub issue will allow the **maintainers** to discuss the proposal and decide whether they would like to see a more detailed RFC. Keep in mind that a feature’s soundness is not the only facet of the proposal that will be weighed. Any new feature could require:
56 |
57 | * Adding support for the new feature in multiple [Ion Schema libraries](https://amzn.github.io/ion-schema/libs.html).
58 | * Adding comprehensive example data to the centralized test repository, [`ion-schema-tests`.](https://github.com/amzn/ion-schema-tests)
59 | * Planning an upgrade path for customers’ existing data.
60 |
61 | If a change is backwards compatible, it may be included in a new *minor* version of the specification. Any changes that are not backwards compatible must be released in a new *major* version of the specification. To learn more about what kinds of changes can be released in a minor version bump, please review the document [*Ion Schema Language Versioning*](https://amazon-ion.github.io/ion-schema/docs/isl-versioning).
62 |
63 | Following discussion, the Ion Schema **maintainers** will either close the issue or leave it open pending a full specification.
64 | The **requestor** and/or the **maintainers** may write the specification for the feature, but neither are required to do so; the specification may be written by any **contributor**.
65 |
66 | ## 2. Write a specification for the feature
67 |
68 | *The purpose of this step is to formalize a proposal and specify the exact details.*
69 |
70 | *Ion Schema Feature RFCs* are documents which provide a comprehensive description of a proposed enhancement to the Ion Schema Language syntax or semantics. There is no minimum or maximum length—the length will probably be proportional to the complexity of the proposed change. The RFC should contain both prose and technical descriptions (sample implementation, grammar, and/or test cases) of the feature in sufficient detail that someone other than the author could implement the feature.
71 |
72 | Create a Markdown file (`.md`) that describes the feature you would like Ion Schema to support. Consider including the following sections:
73 |
74 | * **Summary:** Write a few sentences explaining the proposed feature at a high level. This should focus on the *what* and *why,* not the *how* of the change.
75 | * **Motivation:** Describe the limitations of the incumbent Ion Schema version you hope to modify. Give examples of data and/or use cases that can lead to problems with processor performance, fidelity, or expressiveness.
76 | * **Proposal:** Provide a thorough walk-through of the feature you propose. Using examples, demonstrate the benefits it offers in applicable situations.
77 | * **Alternatives considered:** This section is an opportunity to head off discussions about ideas that you have already considered and dismissed for one reason or another. Questions to consider addressing include:
78 | * What are some ways to achieve the same thing using features that are already available in the current version of Ion Schema?
79 | * Could the functionality you need be provided using application-defined open content in Ion Schema? (For example, a ISL based code generator does not need a `javadoc` field added to the spec when an open content field called `_javadoc` would suffice.)
80 | * Did you tinker with different syntax and semantics for your feature before ultimately going with your current proposal? List them here and explain why they were less optimal.
81 |
82 | For examples, please see the feature RFCs for Ion Schema [*Open Content*](https://github.com/amazon-ion/ion-schema/blob/208165adb10c889949252e7ccd926862bfe60019/rfcs/ion_schema_2_0/open_content.md) or Ion 1.1 *[Templates](https://github.com/amzn/ion-docs/blob/bf33a708d806e46bc24e6bad4a95c12fa359bac8/rfcs/ion_1_1/feature-templates.md#rfc-ion-templates)* and *[Inlineable Symbols](https://github.com/amzn/ion-docs/blob/bf33a708d806e46bc24e6bad4a95c12fa359bac8/rfcs/ion_1_1/ion_1_1.md#inline-symbols).*
83 | For examples of shorter RFCs [Add Constraints for Smaller IEEE-754 Binary Types · Issue #18 · amazon...](https://github.com/amazon-ion/ion-schema/issues/18#issuecomment-1092130146) and [*Allow field names to be constrained without exact field names being sp...*](https://github.com/amazon-ion/ion-schema/issues/44#issuecomment-1097296761). (Note that these were created prior to the requirement for RFCs to be a markdown document, but in terms of content, they are examples of an RFC.)
84 |
85 | #### Open a feature PR
86 |
87 | When the document is ready, open a PR that adds your `.md` file to the folder `rfcs/`. In the PR description, include:
88 |
89 | 1. A link to the original issue from step 1.
90 | 2. A link directing users to a rendered version of the `.md` content.
91 |
92 | The Ion **maintainers** will add a link to your proposal to [the Ion home page](https://amzn.github.io/ion-docs/) and to [the Ion news page](https://amzn.github.io/ion-docs/news.html). Members of the Ion community at large will be able to comment on the proposal.
93 |
94 | Discussion of the RFC involves two phases. The first phase is not time-bound, and is intended to allow thorough evaluation of the technical details of the proposal, including its impact on the broader Ion ecosystem. Changes may be requested and can be added to the proposal via new commits on the PR. When the Ion Schema **maintainers** deem the proposal is mature, a final comment period of at least two weeks will be announced, marking the beginning of phase two.
95 |
96 | The comment period may be extended at the discretion of the **maintainers**—e.g. due to the complexity of the proposal, because the comment period would fall during a holiday, or because of ongoing, productive discussion about the proposal.
97 |
98 | During the comment period, if the RFC is modified in response to feedback, any modifications should be appended to the PR as new commits, and the RFC document itself should include a change-log with summaries of the changes.
99 | After this time has passed, the **maintainers** will either close the PR or merge it. Merging the PR signals that the **maintainers** intend to add it to the Ion Schema spec in an unscheduled, unspecified future version. These updates will also appear on the Ion home and news pages.
100 |
101 | Note that in some circumstances, the RFC may need to be modified between the PR being merged and it being added to the spec in a new version. Such changes would typically be needed to address technical conflicts that were not identified during the initial RFC discussion. If necessary, the changes will be done in a later PR that does not follow the complete RFC process. The **maintainers** will attempt to include interested parties from the original RFC in the new PR and aim to minimize the changes being introduced.
102 |
103 | ## 3. An Ion Schema Version RFC will adopt the feature
104 |
105 | *Ion Schema Version RFCs* are meta-RFCs that describe a new version of Ion Schema. They are PRs created by the **maintainers**, not by community members proposing a new feature. Version RFCs include a `.md` file that contains:
106 |
107 | * A list of the feature RFCs to be included in the new version, including for each a brief summary and a link to the accepted RFC for that feature. Links to other documentation may also be included.
108 | * The list of all accepted feature RFCs that are *not* included in the new version, with reasons why they are not included.
109 | * If it is a new major version, a definition of the compatibility and import requirements between this version and the prior major version, as required by [Ion Schema Language Versions](https://quip-amazon.com/WxDyAMCsxccV).
110 |
111 | For an example, please see [the Ion Schema 2.0 RFC](https://github.com/amazon-ion/ion-schema/blob/208165adb10c889949252e7ccd926862bfe60019/rfcs/ion_schema_2_0/ion_schema_2_0.md) ([PR](https://github.com/amazon-ion/ion-schema/pull/69)).
112 |
113 | Ion Schema versions are not released on a fixed schedule. The creation of a version RFC is a required step on the path to release but is not an indication of any particular timeline.
114 |
115 | Ion Schema version RFCs are not finalized until they are merged; feature RFCs may be moved between them or removed altogether. Removal does not revoke a previously accepted feature RFC; it only removes it from that version.
116 |
117 | ## 4. A new Ion Schema spec version will be released
118 |
119 | When developer bandwidth is available, the next Ion Schema version RFC will be merged. At this point, **contributors** (in all likelihood, mostly just the **maintainers**) will write a new version of the Ion Schema specification that incorporates the changes in the Ion Schema Version RFC. Designing the exact format of the new specification document, including how individual features’ supported versions will be communicated, is outside of the scope of this document.
120 |
121 | Once the new version of the spec has been merged (via one or more pull requests), Ion Schema libraries can be updated to add support for the features that were included in the new version.
122 |
123 | As before, announcements will be added to the Ion home and news pages.
124 |
125 | # Internal steps for considering a proposal
126 |
127 | *This section is addressed to the Ion Schema ** **maintainers**.* *It is included here in order to be transparent about the approval process.*
128 |
129 | ## Requesting an RFC
130 |
131 | When a community member creates a feature request in `ion-schema`, we (the Ion Schema **maintainers**) will engage with the requester on the GitHub issue. The **maintainers** will come to a consensus on the proposal; if its value proposition is compelling, we will request an RFC to explore the technical details of its implementation in depth. If the proposal does not add value or goes against the design principles of Ion Schema, we will politely decline. However, this does not preclude the **feature requestor** or any other **contributor** from revising and resubmitting their proposal with new ideas or information.
132 |
133 | Either way, the Ion schema **maintainers** must respond with a justification for our decision. For a feature that is approved, a simple message about how the feature will be useful, aligns with the design goals, etc. will suffice. For a rejection, the justification should specifically explain why the idea does not add value to Ion Schema or how it goes against the design principles of Ion Schema.
134 |
135 | If we are interested in an idea but **feature requestor** does not want to create an RFC, we may decide to take it up ourselves, but we are not required to do so.
136 |
137 | ## Approving an RFC
138 |
139 | The Ion Schema **maintainers** are responsible for implementing and maintaining both the spec and most of its implementations, so we are the biggest stakeholder. The RFC cannot move ahead without the **maintainers**’ agreement.
140 | We believe that it should be easy to iterate on Ion Schema, and so the most critical feedback that we can receive is whether something we are introducing is a misfeature or could otherwise cause problems later on. Feedback about the omission of desired features (or part of a feature) will also be considered, but we may decide to defer solving it if there are no backwards compatibility issues that the proposed feature would introduce.
141 |
142 | After feedback has been gathered from Ion Schema users and subject-matter experts, the Ion Schema **maintainers** will incorporate the feedback and make the final decision on the RFC.
143 |
144 | The outcome of this process will be shared on the `ion-schema` GitHub issue, resulting in the PR being merged or closed as appropriate.
145 |
146 | ## Creating an RFC for a new Ion Schema version
147 |
148 | When we should create a new version depends on our subjective judgment of whether it is a good time for a new version. However, we should consider:
149 |
150 | * Are we still in the midst of implementing the last Ion Schema version?
151 | * How many accepted features are waiting to be released in a new version of the ISL spec?
152 | * Has anyone specifically asked for a new version to be released?
153 | * Do we have the bandwidth to implement the new version in Ion Schema libraries?
154 | * Has a **contributor** outside the team of **maintainers** offered to help implement the new version?
155 |
156 | We should be transparent if we do not have the bandwidth to implement a new version, but we should not stand in the way of someone who is willing to invest their own time in improving Ion Schema.
157 |
158 | # FAQ
159 |
160 | #### **Why is there no private comment period?**
161 |
162 | Ion Schema is an open source project, so we are going to try to do our design work out in the open too. We want interested parties to be able to share their thoughts and have a say in the direction of Ion Schema. (See [opensource.guide – Keep Communication Public](https://opensource.guide/best-practices/#keep-communication-public).)
163 |
164 | #### What if I have feedback that contains confidential information?
165 |
166 | If you are employed by Amazon, refer to internal policies about sharing confidential information.
167 |
168 | If you are not employed by Amazon, please do not share any confidential information on GitHub or directly with the **maintainers**. If you need additional time to rework your feedback so that it does not include confidential information, you may ask for the comment period to be extended.
169 |
170 | Regardless of the source, the **maintainers** will attempt to publicly summarize all private feedback.
171 |
172 | #### Can a feature RFC become “un-accepted”?
173 |
174 | Yes. An RFC can be reverted by another RFC that explicitly revokes or supersedes a prior RFC. An RFC that reverts or supersedes a previous RFC is expected to go through the same process as any other RFC.
175 |
--------------------------------------------------------------------------------
/rfcs/ion_schema_2_0/language_versions.md:
--------------------------------------------------------------------------------
1 | # RFC: Ion Schema Language Versions
2 |
3 |
4 | - [Ion Schema Language Versions](#rfc-ion-schema-language-versions)
5 | - [Introduction](#introduction)
6 | - [Definitions/Glossary](#definitionsglossary)
7 | - [Ion Schema Compatibility/Portability Goal](#ion-schema-compatibilityportability-goal)
8 | - [Motivation](#motivation)
9 | - [Incompatibility across implementation versions](#incompatibility-across-implementation-versions)
10 | - [Lack of portability](#lack-of-portability)
11 | - [Underspecification of importing across schema versions](#underspecification-of-importing-across-schema-versions)
12 | - [Backwards Incompatibility caused by Open Content](#backwards-incompatibility-caused-by-open-content)
13 | - [Solution – Ion Schema Language Versioning](#solution--ion-schema-language-versioning)
14 | - [What is versioned?](#what-is-versioned)
15 | - [How is it versioned?](#how-is-it-versioned)
16 | - [Major Versions](#major-versions)
17 | - [Minor Versions](#minor-versions)
18 | - [Examples of different types of changes](#examples-of-different-types-of-changes)
19 | - [ISL Version Marker Syntax and Implementation](#isl-version-marker-syntax-and-implementation)
20 | - [Implications for Schema Imports](#implications-for-schema-imports)
21 | - [Examples – Ion Schema 1.0](#examples--ion-schema-10)
22 | - [Examples – Ion Schema >=2.0](#examples--ion-schema-20)
23 | - [Alternatives Considered](#alternatives-considered)
24 | - [Semantic Versioning](#semantic-versioning)
25 | - [Only track major versions of ISL](#only-track-major-versions-of-isl)
26 | - [Calendar Versioning (CalVer)](#calendar-versioning-calver)
27 | - [Include a promise about semantic equivalence for major version updates](#include-a-promise-about-semantic-equivalence-for-major-version-updates)
28 | - [Frequently Asked Questions](#frequently-asked-questions)
29 | - [Does Ion Schema provide any features for versioning of users’ schemas or types?](#does-ion-schema-provide-any-features-for-versioning-of-users-schemas-or-types)
30 | - [When an implementation adds support for a new ISL major version, must an implementation that follows SemVer also bump its major version?](#when-an-implementation-adds-support-for-a-new-isl-major-version-must-an-implementation-that-follows-semver-also-bump-its-major-version)
31 |
32 |
33 | ## Introduction
34 |
35 | The purpose of this document is to specify how the Ion Schema Language will be versioned.
36 |
37 | *The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119).*
38 |
39 | ## Definitions/Glossary
40 |
41 | * **Ion Schema Language (ISL):** the syntax, grammar, and set of constraints for validating Ion data, as well as the rules that govern how an Ion Schema implementation should interpret and apply schemas to Ion data.
42 | * **Ion Schema Language (ISL) version:** a specific version of the Ion Schema Language; synonymous with "Ion Schema version" and "language version"
43 | * **Ion Schema Specification:** the text that describes the Ion Schema Language.
44 | * **Implementation version**: the (release) version of a library that implements the Ion Schema Specification, such as `ion-schema-kotlin` or `ion-schema-rust`
45 | * **Schema Document**: A single stream of ion values that conforms to the Ion Schema Specification. (In IonJava terms, an IonDatagram that conforms to the spec.)
46 | * **Schema Version**: refers to versions of user-defined schemas/types. Schema versions are not in scope for this document, but it is defined here to clearly differentiate it from the other types of "version" that are mentioned.
47 | * **User defined content (UDC)**: optional content that has no meaning to the schema system but may have meaning to the end user
48 |
49 | ## Ion Schema Compatibility/Portability Goal
50 |
51 | A schema document MUST have the same behavior on all equivalently configured Ion Schema implementations. If an implementation does not support all features used in a particular schema document, the implementation MUST error when it attempts to load that schema.
52 |
53 | ## Motivation
54 |
55 | The Ion Schema 1.0 Specification [requires version markers](https://amzn.github.io/ion-schema/docs/spec.html#schema-definitions) at the start of a schema document, but the specification says nothing else about how Ion Schema is to be versioned. Without clear versioning rules, almost any functional change to the Ion Schema specification is a potentially breaking change for our customers.
56 |
57 | ### Incompatibility across implementation versions
58 |
59 | We cannot change the meaning or behavior of any existing language syntax in a backwards compatible way. If the meaning of any syntax is modified, then schemas would no longer portable because they could behave differently in different applications that take a dependency on different releases of a library that implements Ion Schema.
60 |
61 | ### Lack of portability
62 |
63 | Any behavioral change to the Ion Schema specification is not guaranteed to be portable across different libraries that implement Ion Schema. These libraries will not be updated at exactly the same time, and users of Ion Schema will not update their dependencies at exactly the same time (or possibly never update their dependencies). Ion Schema needs a clearly defined versioning strategy that allows users to specify which behavior to use for a given schema document.
64 |
65 | ### Underspecification of importing across schema versions
66 |
67 | For Ion Schema use cases that have a large number of schemas, perhaps owned by multiple teams, we currently provide no guidance regarding whether schemas with different versions can be used together (i.e. can a schema import another schema with a different version?). Users will not be able to safely use multiple ISL versions or gradually upgrade their ISL version if this remains undefined.
68 |
69 | ### Backwards Incompatibility caused by Open Content
70 |
71 | There can be no Ion Schema 1.1—as long as open content is allowed, almost any change is backwards incompatible. ISL 1.0 broadly allows user-defined ("open") content and so any new syntax has the possibility of name-shadowing user-defined content, causing unexpected behavior. However, restricting user defined content would itself be a backwards incompatible change. This backwards incompatible change will require a new major version, but before that can happen, Ion Schema needs a versioning strategy.
72 |
73 | ## Solution – Ion Schema Language Versioning
74 |
75 | *This solution assumes that Ion Schema 2.0 also introduces some rules to prevent open content from colliding with Ion Schema features. See [Ion Schema 2.0 Open Content](open_content.md)*
76 |
77 | ### What is versioned?
78 |
79 | The *Ion Schema Language* SHALL be versioned. The documents that describe the Ion Schema Language (the specification) SHALL NOT have versioned releases.
80 |
81 | ### How is it versioned?
82 |
83 | The Ion Schema Language version SHALL consist of a major and minor version components. The major and minor version components SHALL be non-negative integers.
84 |
85 | Within a schema document, the format SHALL be `$ion_schema__`. In the specification text and other documentation, it SHALL be `.`.
86 |
87 | Readers may notice that this is similar to [Semantic Versioning](https://semver.org/). However, it differs in two key ways. Semantic Versioning includes patch versions, but Ion Schema Language versioning does not use patch versions. Semantic Versioning is specific that [releases must be immutable](https://semver.org/#spec-item-3), but Ion Schema Language versioning does not provide that guarantee. The Ion Schema Language may be modified without changing the version number when the modification is an unsubstantive clarification or bugfix.
88 |
89 | #### Major Versions
90 |
91 | A new major version MAY contain any sort of changes, including backwards incompatible changes. Upon creating a new major version, the minor version component SHALL be reset to `0`.
92 |
93 | #### Minor Versions
94 |
95 | Each new minor version of the Ion Schema Language SHALL allow any Ion Schema document that is valid against any previous minor version of the Ion Schema Language, within the same major version, to be updated to the new Specification version with equivalent semantics. Such an update MUST only require changing the ISL version marker to the new minor version. For example, a valid Ion Schema 2.2 document, upon changing its ISL version marker to `$ion_schema_2_3`, SHALL be a valid Ion Schema 2.3 document, semantically equivalent to the original Ion Schema 2.2 document. New minor versions of the Ion Schema Specification MUST be written to ensure this form of backward compatibility.
96 |
97 | The guarantee of backwards compatibility is the objective standard by which we differentiate a major and minor version. Without some sort of objective standard, all we have are good intentions in order to decide whether a change requires a new major or minor version.
98 |
99 | A change that qualifies as a minor version update MAY be released as a major version update as a way to signal to users that it is a particularly large or significant change.
100 |
101 | ### Examples of different types of changes
102 |
103 | Any change that is not backwards compatible must be released as a new major version. In other words, given the latest released version N, if we create version N+1, and there are any possible schemas for version N that are not valid and semantically equivalent by simply changing the ISL version marker to N+1, then a new major version is required. Examples of this include placing new restrictions on user-defined content, removing functionality, or changing the ISL versioning rules.
104 |
105 | Any backwards compatible behavior or syntax change can be released as a new minor version. Examples of this include adding new behavior with new syntax to an existing constraint (such as a new modifier), adding a new constraint (assuming open content is appropriately limited), or adding syntactical sugar.
106 |
107 | Some behavior clarifications may be released as an update to an existing minor version. Examples include reclassifying `occurs` to not be a constraint ([ion-schema#48](https://github.com/amzn/ion-schema/pull/48)) and clarifying the interactions between number ranges and special float values ([ion-schema#56](https://github.com/amzn/ion-schema/pull/56)).
108 |
109 | Changes to the Ion Schema Specification that do not affect the Ion Schema Language are out of scope for Ion Schema Language versioning—for example, spelling and grammar changes, adding examples, or renumbering sections of the specification.
110 |
111 | ### ISL Version Marker Syntax and Implementation
112 |
113 | The ISL version marker in a schema document will only specify the major and minor version. The patch version is not included in the ISL version marker because patch versions do not change the behavior of the schema, so they have no relevance in a schema document.
114 |
115 | The Ion Schema specification [requires ISL version markers](https://amzn.github.io/ion-schema/docs/spec.html#schema-definitions), but the `ion-schema-kotlin` implementation [does not enforce that requirement](https://github.com/amzn/ion-schema-kotlin/blob/master/src/com/amazon/ionschema/internal/SchemaImpl.kt#L78). Starting with Ion Schema 2.0, a version marker must be required, but to avoid breaking existing schemas, we must continue to interpret the absence of an ISL version marker as an implied `$ion_schema_1_0`. (*Refer to [Backwards Incompatibility caused by Open Content](#backwards-incompatibility-caused-by-open-content)* *for why we must go directly to Ion Schema 2.0.*)
116 |
117 | * **The keyspace reserved for ISL version markers SHALL be the set of symbols that matches the regular expression `^\$ion_schema_\d.*$`.** Any top-level symbol matching this regular expression must be interpreted as an ISL version marker (regardless of whether it refers to a valid ISL version).
118 | *Why?* We want to avoid any ambiguity or potential confusion for human readers when they see a value that looks like an ISL version marker.(Note that the Ion spec already reserves the `$ion` prefix for Ion and related applications.)
119 | * **A *valid* ISL version marker SHALL match the regular expression `^\$ion_schema_[1-9]\d*_(0|[1-9]\d*)$`.**
120 | *Why?* A valid ISL version marker must contain a major and minor version indicator. No leading zeros are allowed.
121 | * **The first value in the schema SHOULD be an ISL Version Marker.**
122 | *Why?* This is the version-independent convention for signaling the ISL version.
123 | * **If a top-level ISL value is encountered before encountering an ISL version marker, implementations MUST assume `$ion_schema_1_0`.**
124 | *Why?* This behavior is required for backwards compatibility, since the ISL version marker is not enforced for ISL 1.0
125 | * **An ISL Version Marker in the form `$ion_schema_X_Y` SHALL indicate that X and Y are the major and minor version respectively.**
126 | *Why?* This is the version-independent convention for signaling the ISL version.
127 | * **If any ISL Version Marker is not a valid/supported version marker, implementations MUST raise an error.**
128 | *Why?* Implementations must raise an error to meet [the portability/compatibility goal](#ion-schema-compatibilityportability-goal).
129 | * **If more than one Ion Schema version marker is found in the value stream of a single Ion Schema, the implementation MUST raise an error.**
130 | *Why?* Because when there are multiple version markers, we cannot determine which version to use if those versions are different. *Caveat—the Ion Schema Specification does not define how to find the boundaries of a schema document in an Ion stream. Ion Schema Authorities MAY choose to support a means of having more than one schema document contained in a stream of Ion Values, but an Authority MUST return at most 1 schema for a given schema id.*
131 | * **If an implementation supports ISL version `X.Y`, then it MUST also support ISL version `X.Y-1`** **(recursively down to `X.0`)**.
132 | *Why?* Easier for customers to move between minor versions of their schemas. Helps us to guarantee that minor versions are import compatible with each other. It also simplifies the `ion-tests` repository because we can organize and run tests by major version instead of having to re-run them for every minor version—which also helps validate that minor version updates are indeed backwards compatible.
133 |
134 | #### Implications for Schema Imports
135 |
136 | From [Minor Versions](#minor-versions):
137 |
138 | > Each new minor version of the Ion Schema Specification SHALL allow any Ion Schema document that is valid against any previous minor version of the Specification, within the same major version, to be updated to the new Specification version with equivalent semantics. Such an update MUST only require changing the ISL version marker to the new minor version.
139 |
140 | Under this condition, any schema that uses imports must remain semantically equivalent, without requiring anything of the imported schema, so by implication, Ion Schema must allow importing schemas that are still on prior minor versions.
141 |
142 | Because ISL supports cycles in schema dependency graphs, we can infer that **a schema SHALL be allowed to import another schema that uses *any* other minor version in the same major version**. For example, given a dependency of A → B → A, where both A and B are using Ion Schema 2.0, when A is updated to 2.1, A must be allowed to import B and B must be allowed to import A.
143 |
144 | (In other words, any schema using Ion Schema `X.a` MUST be able to import any other schema using Ion Schema `X.b`, both minor versions are supported by the Ion Schema implementation. Since an Ion Schema implementation must always provide support for all prior minor versions for a given major version, it is always possible to import a schema with any different minor version, *up to the highest minor version supported by that implementation*.)
145 |
146 | The Ion Schema Specification does not restrict the possible changes that are allowed in a new major version, so **any new major version of Ion Schema SHALL define its own rules regarding the ability to import schemas from prior major versions of Ion Schema**. A new major version SHOULD allow importing from the most recent prior major version unless there is a technical reason why it is not possible.
147 |
148 | #### Examples – Ion Schema 1.0
149 |
150 | ```
151 | // The first value is not an ISL version marker, so it is implicitly ISL 1.0
152 |
153 | $foo_service_interface_version_1 // Open content
154 |
155 | $ion_schema_1_0 // Ion Schema 1.0 allows out-of-place version markers
156 | // because in Ion Schema 1.0 this is open content.
157 |
158 | $ion_schema_2_0 // Ion Schema 1.0 does not allow version
159 | // markers that are for a different version
160 |
161 | schema_header::{}
162 |
163 | // ... other content ...
164 |
165 | schema_footer::{}
166 | ```
167 |
168 | ```
169 | $ion_schema_1_0 // Explicit version marker
170 |
171 | $ion_schema_1_0 // Ion Schema 1.0 allows out-of-place version markers
172 | // because in Ion Schema 1.0 this is open content.
173 |
174 | schema_header::{}
175 |
176 | // ... other content ...
177 |
178 | schema_footer::{}
179 | ```
180 |
181 | #### Examples – Ion Schema >=2.0
182 |
183 | ```
184 | $ion_schema_2_4 // <-- isl version marker
185 |
186 | $foo_service_interface_version_1 // <-- allowed open content
187 |
188 | $ion_schema_2_4 // <-- ion schema version marker in wrong position
189 | $ion_schema_1_foo // <-- not allowed as open content
190 |
191 | schema_header::{
192 | imports: [
193 | { id: fruit, type: apple },
194 | ]
195 | }
196 |
197 | type::{
198 | name: ion_schema_version_marker,
199 | type symbol,
200 | valid_values: [
201 | $ion_schema_1_0, // <-- all valid;
202 | $ion_schema_2_0, // < not isl version markers because
203 | $ion_schema_2_1, // < they are not a top level values.
204 | $ion_schema_cat_dog, // <
205 | ]
206 | }
207 |
208 | schema_footer::{}
209 | ```
210 |
211 | ## Alternatives Considered
212 |
213 | #### Semantic Versioning
214 |
215 | Using [Semantic Versioning](https://semver.org/) (SemVer) would require patch versions. The chosen solution has no use for patch versions, so we would need to find something to do with patch versions. We could either leave the patch version to always be `0` or we could use patch versions to version non-behavioural changes in the documentation. If we always keep the patch version at `0`, then we have no benefit of using SemVer. If we use patch versions for documentation changes, then we are imposing an unnecessary process on ourselves and future maintainers of the project, and adding complexity that will probably confuse Ion Schema users with no obvious benefit to those users.
216 |
217 | #### Only track major versions of ISL
218 |
219 | While this is possible, it does not benefit our customers as much as tracking minor versions. By tracking minor versions, we get the following benefits over tracking only major versions:
220 |
221 | * Ability to define easy/intuitive rules about backwards compatibility for implementations
222 | * Ability to define straightforward rules about importing schemas of mixed versions
223 | * Minor versions are a signal for customers to know how much effort it is for them to upgrade the ISL version of their schemas
224 |
225 | #### Calendar Versioning (CalVer)
226 |
227 | "[CalVer](https://calver.org/) is a versioning convention based on the project's release calendar, instead of arbitrary numbers." It can be used in conjunction with (modified) SemVer or on its own. For example, one could choose to use the year as the major version while following SemVer for the minor and patch versions.
228 |
229 | CalVer has the benefit (for developers/maintainers) that you do not need to determine backwards compatibility to decide whether a release should be a new major version—instead you release a new major version with all the feature changes on a pre-determined schedule. However, if we use a calendar-based major version, then we cannot provide consistent guarantees about the compatibility between different versions of the Ion Schema Language, which is not as good for Ion Schema users.
230 |
231 | CalVer also has the (general) benefit of making it easy to tie releases to a particular support schedule. This may be useful for a specific *software application*, but it seems to be irrelevant for a *specification*.
232 |
233 | If we use CalVer, it seems like there would be an implied expectation of a specific release schedule. We do not want the spec to be updated on a specific schedule because we want to be able to react to business needs (for example, release a new major version without having to wait for the next calendar year). On the other hand, we do not want to be making releases solely because it is the predetermined time for a release—we expect the frequency of Ion Schema Specification releases to decrease over time as the specification matures.
234 |
235 | Finally, this is a minor consideration, but the Ion Schema 1.0 Specification is already implied to use a versioning scheme (though none is defined) that is not calendar based, so precedent would suggest that we stick to non-calendar-based versioning.
236 |
237 | #### Include a promise about semantic equivalence for major version updates
238 |
239 | We could include the following:
240 |
241 | >Given a valid schema document for a particular major version, when the ISL version marker is changed to the next major version, the schema document SHALL either be (1) a semantically equivalent schema document or (2) not a valid schema document.
242 |
243 | In other words:
244 |
245 | >Changing only the ISL version marker MUST NOT result in a valid but semantically different schema document, even when changing major versions.
246 |
247 | This would make it safer for customers to upgrade their schemas because when they update the ISL version marker, it eliminates the possibility of having a valid but semantically different schema document. In addition, it simplifies the implementation of Ion Schema because there would be no need for implementations to push any versioning concerns to the constraint implementations.
248 |
249 | However, the implication is that a major version update *cannot change the meaning of any existing syntax*—it can only remove features from ISL. (Though in the case of a constraint, for example, the constraint may be re-introduced with altered functionality under a different name.) This could add a disproportionately large amount of friction for seemingly small changes that result in a change of meaning, such as changing whether `+/-inf` are inside the `max/min` range bounds.
250 |
251 | We decided not to do this because upgrading major versions should be an infrequent occurrence. If a change is significant enough to require a new major version, then it will probably be non-trivial for users to update their schemas to a new version. Since it would be non-trivial, we would provide a tool to help Ion Schema users upgrade to the latest version. Furthermore, this promise would make it more difficult for us to introduce changes to ISL.
252 |
253 | ## Frequently Asked Questions
254 |
255 | #### Does Ion Schema provide any features for versioning of users’ schemas or types?
256 |
257 | ISL 1.0 does not provide any special functionality for schema versions. That is up to the discretion of the end user. However, the Ion Schema cookbook should provide suggestions.
258 |
259 | Why does ISL not have schema versioning? Customers have asked about schema versioning, but they have described multiple different versioning strategies, and Ion Schema is not at a point right now where we can choose one to be the blessed way of versioning schemas/types.
260 |
261 | However, Ion Schema will need to choose a versioning strategy for Ion Schema Schemas, because there will need to be multiple versions of the schema schemas. This will be a forcing function for Ion Schema to document schema versioning approaches for the Ion Schema cookbook.
262 |
263 | #### When an implementation adds support for a new ISL major version, must an implementation that follows SemVer also bump its major version?
264 |
265 | No. Adding support for a new ISL version never *requires* the implementation to increase its major version because it is strictly adding functionality. However, if an implementation (that follows SemVer) ever *drops* support for an older version of ISL, that would require a new major version under the SemVer rules.
266 |
--------------------------------------------------------------------------------