├── .github
└── workflows
│ └── markdown_link_checker.yml
├── .gitignore
├── 0000-template.md
├── LICENSE
├── README.md
├── files
├── 0005-after.svg
├── 0005-before.svg
├── 0010-references.sw
├── 0011-expressive-diagnostics-cli.png
├── 0011-expressive-diagnostics-vscode-full-diagnostic.png
├── 0011-expressive-diagnostics-vscode-hints.png
└── 0011-expressive-diagnostics-vscode-problems.png
└── rfcs
├── 0001-rfc-process.md
├── 0002-docstrings.md
├── 0003-intrinsics.md
├── 0004-declaration-engine.md
├── 0005-collection-context.md
├── 0006-configuration-constants.md
├── 0007-evm-support.md
├── 0008-storage-handler.md
├── 0009-structured-modules.md
├── 0010-private-modules.md
├── 0011-references.md
├── 0012-expressive-diagnostics.md
├── 0013-changes-lifecycle.md
├── 0014-abi-errors.md
└── 0015-const-generics.md
/.github/workflows/markdown_link_checker.yml:
--------------------------------------------------------------------------------
1 | name: Check Markdown links
2 |
3 | on:
4 | push:
5 | branches:
6 | - master
7 | tags:
8 | - v*
9 | pull_request:
10 |
11 | jobs:
12 | markdown-link-check:
13 | runs-on: ubuntu-latest
14 | steps:
15 | - uses: actions/checkout@v2
16 | - uses: gaurav-nelson/github-action-markdown-link-check@1.0.14
17 |
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
1 | # Generated by Cargo
2 | # will have compiled files and executables
3 | /target/
4 |
5 | # Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
6 | # More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
7 | Cargo.lock
8 |
9 | # These are backup files generated by rustfmt
10 | **/*.rs.bk
11 |
--------------------------------------------------------------------------------
/0000-template.md:
--------------------------------------------------------------------------------
1 | - Feature Name: (fill me in with a unique ident, `my_awesome_feature`)
2 | - Start Date: (fill me in with today's date, YYYY-MM-DD)
3 | - RFC PR: [FuelLabs/sway-rfcs#0000](https://github.com/FuelLabs/sway-rfcs/pull/001)
4 | - Sway Issue: [FueLabs/sway#0000](https://github.com/FuelLabs/sway/issues/001)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | One paragraph explanation of the feature.
11 |
12 | # Motivation
13 |
14 | [motivation]: #motivation
15 |
16 | Why are we doing this? What use cases does it support? What is the expected outcome?
17 |
18 | # Guide-level explanation
19 |
20 | [guide-level-explanation]: #guide-level-explanation
21 |
22 | Explain the proposal as if it was already included in the language and you were teaching it to another Sway programmer. That generally means:
23 |
24 | - Introducing new named concepts.
25 | - Explaining the feature largely in terms of examples.
26 | - Explaining how Sway programmers should *think* about the feature, and how it should impact the way they use Sway. It should explain the impact as concretely as possible.
27 | - If applicable, provide sample error messages, deprecation warnings, or migration guidance.
28 | - If applicable, describe the differences between teaching this to existing Sway programmers and new Sway programmers.
29 | - If this change is breaking, discuss how existing codebases can adapt to this change.
30 |
31 | For implementation-oriented RFCs (e.g. for compiler internals), this section should focus on how compiler contributors should think about the change, and give examples of its concrete impact. For policy RFCs, this section should provide an example-driven introduction to the policy, and explain its impact in concrete terms.
32 |
33 | # Reference-level explanation
34 |
35 | [reference-level-explanation]: #reference-level-explanation
36 |
37 | This is the technical portion of the RFC. Explain the design in sufficient detail that:
38 |
39 | - Its interaction with other features is clear.
40 | - It is reasonably clear how the feature would be implemented.
41 | - Corner cases are dissected by example.
42 | - If this change is breaking, mention the impact of it here and how the breaking change should be managed.
43 |
44 | The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work.
45 |
46 | # Drawbacks
47 |
48 | [drawbacks]: #drawbacks
49 |
50 | Why should we *not* do this?
51 |
52 | # Rationale and alternatives
53 |
54 | [rationale-and-alternatives]: #rationale-and-alternatives
55 |
56 | - Why is this design the best in the space of possible designs?
57 | - What other designs have been considered and what is the rationale for not choosing them?
58 | - What is the impact of not doing this?
59 |
60 | # Prior art
61 |
62 | [prior-art]: #prior-art
63 |
64 | Discuss prior art, both the good and the bad, in relation to this proposal.
65 | A few examples of what this can include are:
66 |
67 | - For language, library, cargo, tools, and compiler proposals: Does this feature exist in other programming languages and what experience have their community had?
68 | - For community proposals: Is this done by some other community and what were their experiences with it?
69 | - For other teams: What lessons can we learn from what other communities have done here?
70 | - Papers: Are there any published papers or great posts that discuss this? If you have some relevant papers to refer to, this can serve as a more detailed theoretical background.
71 |
72 | This section is intended to encourage you as an author to think about the lessons from other languages, provide readers of your RFC with a fuller picture.
73 | If there is no prior art, that is fine - your ideas are interesting to us whether they are brand new or if it is an adaptation from other languages.
74 |
75 | Note that while precedent set by other languages is some motivation, it does not on its own motivate an RFC.
76 | Please also take into consideration that sway sometimes intentionally diverges from common language features.
77 |
78 | # Unresolved questions
79 |
80 | [unresolved-questions]: #unresolved-questions
81 |
82 | - What parts of the design do you expect to resolve through the RFC process before this gets merged?
83 | - What parts of the design do you expect to resolve through the implementation of this feature before stabilization?
84 | - What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC?
85 |
86 | # Future possibilities
87 |
88 | [future-possibilities]: #future-possibilities
89 |
90 | Think about what the natural extension and evolution of your proposal would
91 | be and how it would affect the language and project as a whole in a holistic
92 | way. Try to use this section as a tool to more fully consider all possible
93 | interactions with the project and language in your proposal.
94 | Also consider how this all fits into the roadmap for the project
95 | and of the relevant sub-team.
96 |
97 | This is also a good place to "dump ideas", if they are out of scope for the
98 | RFC you are writing but otherwise related.
99 |
100 | If you have tried and cannot think of any future possibilities,
101 | you may simply state that you cannot think of anything.
102 |
103 | Note that having something written down in the future-possibilities section
104 | is not a reason to accept the current or a future RFC; such notes should be
105 | in the section on motivation or rationale in this or subsequent RFCs.
106 | The section merely provides additional information.
107 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | Apache License
2 | Version 2.0, January 2004
3 | http://www.apache.org/licenses/
4 |
5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6 |
7 | 1. Definitions.
8 |
9 | "License" shall mean the terms and conditions for use, reproduction,
10 | and distribution as defined by Sections 1 through 9 of this document.
11 |
12 | "Licensor" shall mean the copyright owner or entity authorized by
13 | the copyright owner that is granting the License.
14 |
15 | "Legal Entity" shall mean the union of the acting entity and all
16 | other entities that control, are controlled by, or are under common
17 | control with that entity. For the purposes of this definition,
18 | "control" means (i) the power, direct or indirect, to cause the
19 | direction or management of such entity, whether by contract or
20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the
21 | outstanding shares, or (iii) beneficial ownership of such entity.
22 |
23 | "You" (or "Your") shall mean an individual or Legal Entity
24 | exercising permissions granted by this License.
25 |
26 | "Source" form shall mean the preferred form for making modifications,
27 | including but not limited to software source code, documentation
28 | source, and configuration files.
29 |
30 | "Object" form shall mean any form resulting from mechanical
31 | transformation or translation of a Source form, including but
32 | not limited to compiled object code, generated documentation,
33 | and conversions to other media types.
34 |
35 | "Work" shall mean the work of authorship, whether in Source or
36 | Object form, made available under the License, as indicated by a
37 | copyright notice that is included in or attached to the work
38 | (an example is provided in the Appendix below).
39 |
40 | "Derivative Works" shall mean any work, whether in Source or Object
41 | form, that is based on (or derived from) the Work and for which the
42 | editorial revisions, annotations, elaborations, or other modifications
43 | represent, as a whole, an original work of authorship. For the purposes
44 | of this License, Derivative Works shall not include works that remain
45 | separable from, or merely link (or bind by name) to the interfaces of,
46 | the Work and Derivative Works thereof.
47 |
48 | "Contribution" shall mean any work of authorship, including
49 | the original version of the Work and any modifications or additions
50 | to that Work or Derivative Works thereof, that is intentionally
51 | submitted to Licensor for inclusion in the Work by the copyright owner
52 | or by an individual or Legal Entity authorized to submit on behalf of
53 | the copyright owner. For the purposes of this definition, "submitted"
54 | means any form of electronic, verbal, or written communication sent
55 | to the Licensor or its representatives, including but not limited to
56 | communication on electronic mailing lists, source code control systems,
57 | and issue tracking systems that are managed by, or on behalf of, the
58 | Licensor for the purpose of discussing and improving the Work, but
59 | excluding communication that is conspicuously marked or otherwise
60 | designated in writing by the copyright owner as "Not a Contribution."
61 |
62 | "Contributor" shall mean Licensor and any individual or Legal Entity
63 | on behalf of whom a Contribution has been received by Licensor and
64 | subsequently incorporated within the Work.
65 |
66 | 2. Grant of Copyright License. Subject to the terms and conditions of
67 | this License, each Contributor hereby grants to You a perpetual,
68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69 | copyright license to reproduce, prepare Derivative Works of,
70 | publicly display, publicly perform, sublicense, and distribute the
71 | Work and such Derivative Works in Source or Object form.
72 |
73 | 3. Grant of Patent License. Subject to the terms and conditions of
74 | this License, each Contributor hereby grants to You a perpetual,
75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76 | (except as stated in this section) patent license to make, have made,
77 | use, offer to sell, sell, import, and otherwise transfer the Work,
78 | where such license applies only to those patent claims licensable
79 | by such Contributor that are necessarily infringed by their
80 | Contribution(s) alone or by combination of their Contribution(s)
81 | with the Work to which such Contribution(s) was submitted. If You
82 | institute patent litigation against any entity (including a
83 | cross-claim or counterclaim in a lawsuit) alleging that the Work
84 | or a Contribution incorporated within the Work constitutes direct
85 | or contributory patent infringement, then any patent licenses
86 | granted to You under this License for that Work shall terminate
87 | as of the date such litigation is filed.
88 |
89 | 4. Redistribution. You may reproduce and distribute copies of the
90 | Work or Derivative Works thereof in any medium, with or without
91 | modifications, and in Source or Object form, provided that You
92 | meet the following conditions:
93 |
94 | (a) You must give any other recipients of the Work or
95 | Derivative Works a copy of this License; and
96 |
97 | (b) You must cause any modified files to carry prominent notices
98 | stating that You changed the files; and
99 |
100 | (c) You must retain, in the Source form of any Derivative Works
101 | that You distribute, all copyright, patent, trademark, and
102 | attribution notices from the Source form of the Work,
103 | excluding those notices that do not pertain to any part of
104 | the Derivative Works; and
105 |
106 | (d) If the Work includes a "NOTICE" text file as part of its
107 | distribution, then any Derivative Works that You distribute must
108 | include a readable copy of the attribution notices contained
109 | within such NOTICE file, excluding those notices that do not
110 | pertain to any part of the Derivative Works, in at least one
111 | of the following places: within a NOTICE text file distributed
112 | as part of the Derivative Works; within the Source form or
113 | documentation, if provided along with the Derivative Works; or,
114 | within a display generated by the Derivative Works, if and
115 | wherever such third-party notices normally appear. The contents
116 | of the NOTICE file are for informational purposes only and
117 | do not modify the License. You may add Your own attribution
118 | notices within Derivative Works that You distribute, alongside
119 | or as an addendum to the NOTICE text from the Work, provided
120 | that such additional attribution notices cannot be construed
121 | as modifying the License.
122 |
123 | You may add Your own copyright statement to Your modifications and
124 | may provide additional or different license terms and conditions
125 | for use, reproduction, or distribution of Your modifications, or
126 | for any such Derivative Works as a whole, provided Your use,
127 | reproduction, and distribution of the Work otherwise complies with
128 | the conditions stated in this License.
129 |
130 | 5. Submission of Contributions. Unless You explicitly state otherwise,
131 | any Contribution intentionally submitted for inclusion in the Work
132 | by You to the Licensor shall be under the terms and conditions of
133 | this License, without any additional terms or conditions.
134 | Notwithstanding the above, nothing herein shall supersede or modify
135 | the terms of any separate license agreement you may have executed
136 | with Licensor regarding such Contributions.
137 |
138 | 6. Trademarks. This License does not grant permission to use the trade
139 | names, trademarks, service marks, or product names of the Licensor,
140 | except as required for reasonable and customary use in describing the
141 | origin of the Work and reproducing the content of the NOTICE file.
142 |
143 | 7. Disclaimer of Warranty. Unless required by applicable law or
144 | agreed to in writing, Licensor provides the Work (and each
145 | Contributor provides its Contributions) on an "AS IS" BASIS,
146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147 | implied, including, without limitation, any warranties or conditions
148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149 | PARTICULAR PURPOSE. You are solely responsible for determining the
150 | appropriateness of using or redistributing the Work and assume any
151 | risks associated with Your exercise of permissions under this License.
152 |
153 | 8. Limitation of Liability. In no event and under no legal theory,
154 | whether in tort (including negligence), contract, or otherwise,
155 | unless required by applicable law (such as deliberate and grossly
156 | negligent acts) or agreed to in writing, shall any Contributor be
157 | liable to You for damages, including any direct, indirect, special,
158 | incidental, or consequential damages of any character arising as a
159 | result of this License or out of the use or inability to use the
160 | Work (including but not limited to damages for loss of goodwill,
161 | work stoppage, computer failure or malfunction, or any and all
162 | other commercial damages or losses), even if such Contributor
163 | has been advised of the possibility of such damages.
164 |
165 | 9. Accepting Warranty or Additional Liability. While redistributing
166 | the Work or Derivative Works thereof, You may choose to offer,
167 | and charge a fee for, acceptance of support, warranty, indemnity,
168 | or other liability obligations and/or rights consistent with this
169 | License. However, in accepting such obligations, You may act only
170 | on Your own behalf and on Your sole responsibility, not on behalf
171 | of any other Contributor, and only if You agree to indemnify,
172 | defend, and hold each Contributor harmless for any liability
173 | incurred by, or claims asserted against, such Contributor by reason
174 | of your accepting any such warranty or additional liability.
175 |
176 | END OF TERMS AND CONDITIONS
177 |
178 | APPENDIX: How to apply the Apache License to your work.
179 |
180 | To apply the Apache License to your work, attach the following
181 | boilerplate notice, with the fields enclosed by brackets "[]"
182 | replaced with your own identifying information. (Don't include
183 | the brackets!) The text should be enclosed in the appropriate
184 | comment syntax for the file format. We also recommend that a
185 | file or class name and description of purpose be included on the
186 | same "printed page" as the copyright notice for easier
187 | identification within third-party archives.
188 |
189 | Copyright [yyyy] [name of copyright owner]
190 |
191 | Licensed under the Apache License, Version 2.0 (the "License");
192 | you may not use this file except in compliance with the License.
193 | You may obtain a copy of the License at
194 |
195 | http://www.apache.org/licenses/LICENSE-2.0
196 |
197 | Unless required by applicable law or agreed to in writing, software
198 | distributed under the License is distributed on an "AS IS" BASIS,
199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200 | See the License for the specific language governing permissions and
201 | limitations under the License.
202 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Sway RFCs
2 |
3 | | RFC | Title |
4 | | -------------------------------------------- | ----------------------- |
5 | | [0001](rfcs/0001-rfc-process.md) | RFC Process |
6 | | [0002](rfcs/0002-docstrings.md) | Docstring Comments |
7 | | [0003](rfcs/0003-intrinsics.md) | Intrinsics |
8 | | [0004](rfcs/0004-declaration-engine.md) | Declaration Engine |
9 | | [0005](rfcs/0005-collection-context.md) | Collection Context |
10 | | [0006](rfcs/0006-configuration-constants.md) | Configuration Constants |
11 | | [0007](rfcs/0007-evm-support.md) | EVM Support |
12 | | [0008](rfcs/0008-storage-handler.md) | Storage Handler |
13 | | [0009](rfcs/0009-structured-modules.md) | Structured Modules |
14 | | [0010](rfcs/0010-private-modules.md) | Private Modules |
15 | | [0011](rfcs/0011-references.md) | References |
16 | | [0012](rfcs/0012-expressive-diagnostics.md) | Expressive Diagnostics |
17 | | [0013](rfcs/0013-changes-lifecycle.md) | Changes Lifecycle |
18 | | [0014](rfcs/0014-abi-errors.md) | Abi Errors |
19 | | [0015](rfcs/0015-const-generics.md) | Const Generics |
20 |
--------------------------------------------------------------------------------
/files/0005-after.svg:
--------------------------------------------------------------------------------
1 |
2 |
3 |
5 |
6 |
--------------------------------------------------------------------------------
/files/0005-before.svg:
--------------------------------------------------------------------------------
1 |
2 |
3 |
5 |
6 |
--------------------------------------------------------------------------------
/files/0011-expressive-diagnostics-cli.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/FuelLabs/sway-rfcs/20813dac4377ca4cc558e4feef7b6d39fbb9be36/files/0011-expressive-diagnostics-cli.png
--------------------------------------------------------------------------------
/files/0011-expressive-diagnostics-vscode-full-diagnostic.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/FuelLabs/sway-rfcs/20813dac4377ca4cc558e4feef7b6d39fbb9be36/files/0011-expressive-diagnostics-vscode-full-diagnostic.png
--------------------------------------------------------------------------------
/files/0011-expressive-diagnostics-vscode-hints.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/FuelLabs/sway-rfcs/20813dac4377ca4cc558e4feef7b6d39fbb9be36/files/0011-expressive-diagnostics-vscode-hints.png
--------------------------------------------------------------------------------
/files/0011-expressive-diagnostics-vscode-problems.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/FuelLabs/sway-rfcs/20813dac4377ca4cc558e4feef7b6d39fbb9be36/files/0011-expressive-diagnostics-vscode-problems.png
--------------------------------------------------------------------------------
/rfcs/0001-rfc-process.md:
--------------------------------------------------------------------------------
1 | - Start Date: 2022-06-15
2 | - RFC PR: [FuelLabs/sway#1](https://github.com/FuelLabs/sway-rfcs/pull/1)
3 |
4 | This RFC process is inspired by, and mostly copied from, Rust's RFC process.
5 | See [here](https://github.com/rust-lang/rfcs/blob/master/text/0002-rfc-process.md)
6 | for Rust's version of this.
7 |
8 | Rust's RFC process has proven efficiency that we'd like to emulate.
9 |
10 | # Summary
11 |
12 | The "RFC" (request for comments) process is intended to provide a
13 | consistent and controlled path for new features to enter the language
14 | and standard libraries, so that all stakeholders can be confident about
15 | the direction the language is evolving in.
16 |
17 | # Motivation
18 |
19 | The freewheeling way that we add new features to Sway has been good for
20 | early development, but for Sway to become a mature platform we need to
21 | develop some more self-discipline when it comes to changing the system.
22 | This is a proposal for a more principled RFC process to make it
23 | a more integral part of the overall development process, and one that is
24 | followed consistently to introduce features to Sway.
25 |
26 | # Detailed design
27 |
28 | Many changes, including bug fixes and documentation improvements can be
29 | implemented and reviewed via the normal GitHub pull request workflow.
30 |
31 | Some changes though are "substantial", and we ask that these be put
32 | through a bit of a design process and produce a consensus among the Sway
33 | community and the [core team].
34 |
35 | ## When you need to follow this process
36 |
37 | You need to follow this process if you intend to make "substantial"
38 | changes to the Sway distribution. What constitutes a "substantial"
39 | change is evolving based on community norms, but may include the following.
40 |
41 | - Any semantic or syntactic change to the language that is not a bugfix.
42 | - Removing language features, including those that are feature-gated.
43 | - Changes to the interface between the compiler and libraries,
44 | including lang items and intrinsics.
45 | - Eventually, when we reach stdlib maturity, we will want to use RFCs for
46 | stdlib design as well.
47 |
48 | Some changes do not require an RFC:
49 |
50 | - Rephrasing, reorganizing, refactoring, or otherwise "changing shape
51 | does not change meaning".
52 | - Additions that strictly improve objective, numerical quality
53 | criteria (warning removal, speedup, better platform coverage, more
54 | parallelism, trap more errors, etc.)
55 | - Additions only likely to be _noticed by_ other developers-of-Sway,
56 | invisible to users-of-sway.
57 |
58 | If you submit a pull request to implement a new feature without going
59 | through the RFC process, it may be closed with a polite request to
60 | submit an RFC first.
61 |
62 | ## What the process is
63 |
64 | In short, to get a major feature added to Sway, one must first get the
65 | RFC merged into the RFC repo as a markdown file. At that point the RFC
66 | is 'active' and may be implemented with the goal of eventual inclusion
67 | into Sway.
68 |
69 | * Fork the RFC repo https://github.com/FuelLabs/sway-rfcs (or make a branch if you are core-team).
70 | * Copy `0000-template.md` to `rfcs/0000-my-feature.md` (where
71 | 'my-feature' is descriptive. don't assign an RFC number yet).
72 | * Fill in the RFC.
73 | * Submit a pull request. The pull request is the time to get review of
74 | the design from the larger community.
75 | * Build consensus and integrate feedback. RFCs that have broad support
76 | are much more likely to make progress than those that don't receive any
77 | comments.
78 |
79 | Eventually, somebody on the [core team] will either accept the RFC by
80 | merging the pull request, at which point the RFC is 'active', or
81 | reject it by closing the pull request.
82 |
83 | Whomever merges the RFC should do the following:
84 |
85 | * Assign an id, using the PR number of the RFC pull request. (If the RFC
86 | has multiple pull requests associated with it, choose one PR number,
87 | preferably the minimal one.)
88 | * Add the file in the `rfcs/` directory.
89 | * Create a corresponding issue on [Sway repo](https://github.com/FuelLabs/sway).
90 | * Fill in the remaining metadata in the RFC header, including links for
91 | the original pull request(s) and the newly created Sway issue.
92 | * Add an entry in the [Active RFC List] of the root `README.md`.
93 | * Commit everything.
94 |
95 | Once an RFC becomes active then authors may implement it and submit the
96 | feature as a pull request to the Sway repo. An 'active' is not a rubber
97 | stamp, and in particular still does not mean the feature will ultimately
98 | be merged; it does mean that in principle all the major stakeholders
99 | have agreed to the feature and are amenable to merging it.
100 |
101 | Modifications to active RFC's can be done in followup PR's. An RFC that
102 | makes it through the entire process to implementation is considered
103 | 'complete' and is removed from the [Active RFC List]; an RFC that fails
104 | after becoming active is 'inactive' and moves to the 'inactive' folder.
105 |
106 | [core team]: https://github.com/orgs/FuelLabs/teams/sway-compiler
107 |
--------------------------------------------------------------------------------
/rfcs/0002-docstrings.md:
--------------------------------------------------------------------------------
1 | - Feature Name: `docstrings`
2 | - Start Date: 2022-06-27
3 | - RFC PR: [FuelLabs/sway-rfcs#0002](https://github.com/FuelLabs/sway-rfcs/pull/2)
4 | - Sway Issue: [FueLabs/sway#149](https://github.com/FuelLabs/Sway/issues/149)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | Sway will convert Rust-style docstrings (`///`-style comments) into a `#[doc("documentation string")]` annotation. This annotation can then be consumed by tooling and plugins to produce
11 | rendered documentation, a la `cargo doc` or `docs.rs`.
12 |
13 | # Motivation
14 |
15 | [motivation]: #motivation
16 |
17 | As Sway's designs are largely motivated by Rust's design, we have always wanted to support in-code documentation in this manner. In-code documentation support has been part of the Sway vision
18 | since the beginning.
19 |
20 | This particular approach is both low-friction, in that it utilizes our existing features (annotations, comments, forc plugins); and consistent with Rust's implementation of docstrings. Upon
21 | the implementation of this RFC in Sway, forc plugins should be able to produce documentation and documentation-related tooling based on in-code docstrings.
22 |
23 | # Guide-level explanation
24 |
25 | [guide-level-explanation]: #guide-level-explanation
26 |
27 | A _docstring_ is a line of prose within a Sway source code file. This plain-language string serves to provide exposition and contextual explanation to the subsequent line of Sway code.
28 |
29 | Docstrings, for convenience, may be written as a comment with three slashes. E.g.:
30 |
31 | ```rust
32 | /// The entry point to the script.
33 | fn main() {}
34 | ```
35 |
36 | The above code example shows a _docstring_ documenting a function item, `fn main()`.
37 |
38 | Generally, Sway programmers should find this to be the most expedient and convenient way to provide API documentation for libraries and contract ABIs, and documentation of internal concepts within scripts and predicates.
39 |
40 | This style of docstring should be familiar to Rust programmers, and the concept of docstrings is generally prevalent in modern programming languages and should not require significant explanation or new educational material.
41 |
42 | # Reference-level explanation
43 |
44 | [reference-level-explanation]: #reference-level-explanation
45 |
46 | Docstrings should be implemented as attributes (a.k.a. annotations) ([reference 1](https://github.com/FuelLabs/sway/issues/470), [reference 2](https://github.com/FuelLabs/sway/pull/1518), [reference 3 (rust)](https://doc.rust-lang.org/reference/attributes.html)). Because annotations/attributes can be applied to any `Item` in Sway ([reference 1](https://github.com/FuelLabs/sway/blob/master/sway-parse/src/attribute.rs#L4), [reference 2](https://github.com/FuelLabs/sway/blob/ba30e8e5ccbb0512aacbaee594473da9e0839c3d/sway-parse/src/item/mod.rs#L13)), this means that any `Item` can be documented with this feature.
47 |
48 | A docstring of the format `/// this is a docstring` should be converted to an attribute of the format `#[doc("this is a docstring")]`. This may require work in the attribute parser to support strings as attribute contents, although that is unclear at this moment. It is also possible the attribute system can currently handle this.
49 |
50 |
51 | # Drawbacks
52 |
53 | [drawbacks]: #drawbacks
54 |
55 | There are no foreseeable drawbacks.
56 |
57 | # Rationale and alternatives
58 |
59 | [rationale-and-alternatives]: #rationale-and-alternatives
60 |
61 | In the space of possible docstring designs, this is the most consistent with Rust and also very ergonomic. Additionally, a good chunk of Sway code has already been written with the assumption that this docstring format will be accepted. Therefore, there are already many docstrings written in this way.
62 |
63 | Other designs could include just the attribute without the three-slash-comment-based syntactic sugar, ML-family-style docstrings `{- docstring -}`, or multiline-style:
64 |
65 | ```
66 | /**
67 | * Multiline style docstring.
68 | *
69 | */
70 | ```
71 |
72 | The proposed style is, however, most consistent with our design principles. If we were to leave docstrings unimplemented, we would be missing a core part of the Sway product and would
73 | have no canonical method of in-code documentation.
74 |
75 |
76 | # Prior art
77 |
78 | [prior-art]: #prior-art
79 |
80 | The obvious instance is Rust's docstrings. [Read more about that here](https://doc.rust-lang.org/rust-by-example/meta/doc.html).
81 |
82 | Additionally, many other languages have docstrings either via third party tooling or via native support:
83 |
84 | 1. Javascript (`/**`-style)
85 | 2. Haskell (`{-`-style)
86 | 3. C# (`///` or `/**`)
87 | 4. Ocaml (`(**`-style)
88 |
89 | And many, many more. Native docstring support is generally loved by language communities and is critical to having a consistent documentation experience across the language ecosystem.
90 |
91 | # Unresolved questions
92 |
93 | [unresolved-questions]: #unresolved-questions
94 |
95 | 1. Is the current annotations system robust enough to support this?
96 |
97 | # Future possibilities
98 |
99 | [future-possibilities]: #future-possibilities
100 |
101 | Eventually, we'd like to support things like upwards-associating docstrings (`//!` in Rust) and docstring code tests, where code snippets within docstrings are included in a test suite. These are not necessary for an initial docstring implementation, though.
102 |
--------------------------------------------------------------------------------
/rfcs/0003-intrinsics.md:
--------------------------------------------------------------------------------
1 | - Feature Name: `sway_intrinsics_support`
2 | - Start Date: 2022-07-26
3 | - RFC PR: [FuelLabs/sway-rfcs#12](https://github.com/FuelLabs/sway-rfcs/pull/12)
4 | - Sway Issue: [FueLabs/sway#855](https://github.com/FuelLabs/sway/issues/855)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | This is a discussion document on how Intrinsics must be modelled in the Sway compiler. At the time of writing, we have six intrinsics supported in the compiler, of which three (`SizeOfType`, `SizeOfValue` and `IsReferenceType`) get resolved at compile time, and the other three (`GetStorageKey`, `Eq` and `Gtf`), have sway-ir `Instruction` representations. The document dives in deeper into the following aspects of supporting intrinsics in the Sway compiler.
11 |
12 | - What are we going to gain from this?
13 | - What intrinsics do we want to support?
14 | - How should these intrinsics be represented?
15 | - Testing plan.
16 | - Documentation plan for all supported intrinsics.
17 |
18 | # Motivation
19 |
20 | [motivation]: #motivation
21 | - Type checked core and std library implementation. Today, the implementations are in assembly, which aren’t type checked (just the wrapper functions are typed).
22 | - These libraries, when rewritten using intrinsics instead of assembly, can be more easily ported if we decide to support other targets.
23 | - More optimizations. When operations such as `add` are hidden behind assembly, it’s hard for the compiler to reliably perform optimizations. With explicit representation (either via IR opcodes, or as intrinsic+name - see next section), compiler analyses and transforms become simpler.
24 | - A side effect of easier analysis is that, as an example, we can improve usability by allowing declarations such as `const X = Y + Z`, which we don’t today, because we cannot, yet, statically evaluate `+`.
25 |
26 | # Guide-level explanation
27 |
28 | [guide-level-explanation]: #guide-level-explanation
29 |
30 | While the primary purpose of intrinsics is to expose VM op-codes
31 | in Sway so that their usage in the core and standard libraries are
32 | safer, there is nothing stopping from Sway programmers in using
33 | these intrinsics directly in their code.
34 |
35 | At the Sway programmer level, intrinsics work similar to the core and
36 | standard library functions (i.e., they can be called as mere functions).
37 | A comprehensive list of intrinsincs made available by the Sway compiler
38 | can be referenced here (TBD).
39 |
40 | # Reference-level explanation
41 |
42 | [reference-level-explanation]: #reference-level-explanation
43 |
44 | ## Representation of Intrinsics in the Compiler
45 | - Treat intrinsics the same as function calls. They can always be
46 | distinguished because their name begins with `__`.
47 |
48 | In our current framework, this approach isn’t practical since we inline all function calls before even generating sway-ir.
49 |
50 | For some operations, such as the arithmetic ones, it may make more sense to give them a first-hand representation as IR opcodes (as is done in most languages and compilers), than as mere function calls. While this might make IR analysis and transformation a little simpler, I can’t say that that difference is significant.
51 |
52 | - Treat intrinsics with `IntrinsicCall` AST nodes and lower them to
53 | unique op-codes in the IR. I can’t see a clear advantage to doing
54 | this, except for some intrinsics (such as the arithmetic ones) to
55 | be represented via IR op-codes. Overall, using IR op-codes for
56 | intrinsics (except in the special cases such as for `add` etc)
57 | doesn't seem like a good idea. LLVM, for example
58 | [discourages it](https://llvm.org/docs/ExtendingLLVM.html#introduction-and-warning).
59 |
60 | Whatever be the representation we choose, a goal at this stage is for us to be able to type-check intrinsic calls with a table based approach. That is, the type constraints are expressed in a table and the type checker just references this table to type-check. As is visible today, even with just six intrinsics, without a table based approach, we already have a lot of code to typecheck each intrinsic, all of them doing the same thing, with just minor differences.
61 |
62 | It is possible that we may come across some type constraints on intrinsics that cannot be encoded into our table. In such a case, it may be required to write code for it manually (or perhaps provide a lambda / closure in the table which will be called by the type checker).
63 |
64 | A type-information table to type-check intrinsics could look like this:
65 |
66 | | Intrinsic | Type parameters and types allowed on each type parameter |Type (parameter) of each argument and result type |
67 | | ----------- | ----------- | ----------- |
68 | | `add` | `[ (T, [u64, u32, … ]) ]` | `([T, T], T)` |
69 | | ... | ... | ... |
70 |
71 |
72 | ## What intrinsics do we want to support?
73 | To begin with, VM opcodes that are used through assembly wrappers in the core and std library implementations must be exposed as intrinsics. In the future, we could also provide intrinsics for other VM opcodes that need type checking; and (gas / performance) optimized hand-written assembly code sequences that the compiler’s code generation cannot match.
74 |
75 | A (non-exhaustive) list of intrinsics to support: `std::ops` (arithmetic, logical, comparison), `alloc`, `block::height`, `logging::log` etc.
76 |
77 | Because of the possibility of Sway supporting other targets in the future
78 | (such as EVM), VM specific intrinsics may be declared modularly, so that
79 | the toolchain can choose the ones to load based on the target.
80 |
81 | ## Testing
82 | For each core/std library function that we provide an intrinsic for, move the library function into the testsuite, and for every (prio) use of the library function in the testsuite, replace it with a call to the intrinsic and an assert that calls the (now in the testsuite) library function and asserts equivalence. For newly supported VM opcodes, add new tests
83 |
84 | ## Documentation
85 | Given that Sway programmers are free to use intrinsics, it is essential
86 | that we document all intrinsics categorically in a book, perhaps
87 | alongside the standard library.
88 |
89 | # Drawbacks
90 |
91 | [drawbacks]: #drawbacks
92 |
93 | A lot of operations that we want to support via intrinsics already have working assembly based implementations. The assembly we generate in the compiler for these intrinsics will probably be similar to the handwritten assembly, and from what I can tell, similarly done (i.e., manually) in the compiler. This makes me wonder if it’s all worth it, and instead can we just detect calls (by their path/name) to functions in the core/std libraries that we’re interested in involving in an analysis/optimization and act based on that (as if it were an intrinsic). This assumes that we don’t want to type-check / validate the core/std libraries and that they are going to be error-free.
94 |
95 | # Rationale and alternatives
96 |
97 | [rationale-and-alternatives]: #rationale-and-alternatives
98 |
99 | The main design choice to be made here is in representing Intrinsics
100 | internally in the compiler. The two obvious choices were already
101 | outlined in an earlier section.
102 |
103 | # Prior art
104 |
105 | [prior-art]: #prior-art
106 |
107 | [Intrinsics](https://en.wikipedia.org/wiki/Intrinsic_function) / compiler builtins are a common feature provided by compilers across
108 | many programming languages. While [GCC](https://gcc.gnu.org/onlinedocs/gcc-4.9.2/gcc/Other-Builtins.html#Other-Builtins) and [LLVM ](https://llvm.org/docs/ExtendingLLVM.html) provide a long list of
109 | intrinsics, hardware vendors also provide their own set of intrinsics
110 | to enable programmers to take advantage of special instructions
111 | that the compiler does not directly / efficiently support. See
112 | Intel's [AVX intrinsics](http://www.cis.uoguelph.ca/~wgardner/intel_docs/compilers/compiler_c/main_cls/intref_cls/common/intref_avx_details.htm), for example.
113 |
114 | To summarize, the idea is old and useful, and something that we can implement, without much surprises `¯\_(ツ)_/¯`.
115 |
116 | # Unresolved questions
117 |
118 | [unresolved-questions]: #unresolved-questions
119 |
120 | ...
121 |
122 | # Future possibilities
123 |
124 | [future-possibilities]: #future-possibilities
125 |
126 | ...
--------------------------------------------------------------------------------
/rfcs/0004-declaration-engine.md:
--------------------------------------------------------------------------------
1 | - Feature Name: `declaration_engine`
2 | - Start Date: 2022-07-13
3 | - RFC PR: [FuelLabs/sway-rfcs#0011](https://github.com/FuelLabs/sway-rfcs/pull/11)
4 | - Sway Issue: [FueLabs/sway#1821](https://github.com/FuelLabs/sway/issues/1821), [FueLabs/sway#1692](https://github.com/FuelLabs/sway/issues/1692)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | With its current design, the Sway compiler faces challenges regarding how declarations interact with the type system and code generation. These include:
11 | 1. it is currently impossible to implement trait constraints without extensive special casing/dummy definitions ([FueLabs/sway#970](https://github.com/FuelLabs/sway/issues/970))
12 | 2. function bodies must be inlined during type checking ([FueLabs/sway#1557](https://github.com/FuelLabs/sway/issues/1557))
13 | 3. monomorphization of declarations is unnecessarily duplicated ([FueLabs/sway#862](https://github.com/FuelLabs/sway/issues/862))
14 |
15 | This RFC proposes a solution to this---the "declaration engine". The declaration engine is a redesign of how the compiler thinks about type checking of declarations, and seeks to solve the problems described above. At a high level, the declaration engine stores typed declarations and allows the compiler to reference those typed declarations during type checking, in a more abstract way.
16 |
17 | [*prototype system*](https://github.com/emilyaherbert/declaration-engine-and-collection-context-demo/tree/master/de)
18 |
19 | **Note:**
20 |
21 | This RFC does not include plans to target out of order declarations and dependencies or recursive data structures and functions. This is intentional, as it requires the introduction of the collection context 'on top' of the declaration engine, and I feel that adding the declaration engine first would allow other devs to start working on trait constraints, etc. See the [future possibilities](#future-possibilities) for more details on this (and a preview of the design that I'm thinking about).
22 |
23 | # Motivation
24 |
25 | [motivation]: #motivation
26 |
27 | With the changes introduced by this RFC, the compiler will be able to think about declarations more abstractly and will not be required to inline AST nodes during type checking. In addition to solving the issues above, I believe that this change will create a mechanism in the compiler which will allow us to implement additional optimizations in the future.
28 |
29 | # Guide-level explanation
30 |
31 | [guide-level-explanation]: #guide-level-explanation
32 |
33 | This is probably best explained through some examples.
34 |
35 | ## Trait Constraints ([FueLabs/sway#970](https://github.com/FuelLabs/sway/issues/970))
36 |
37 | Adding trait constraints to the compiler is the primary motivation for adding the declaration engine.
38 |
39 | Take this (currently does not compile) Sway code that uses trait constraints as an example:
40 |
41 | ```rust
42 | script;
43 |
44 | trait Eq {
45 | fn eq(self, other: Self) -> bool;
46 | }
47 |
48 | enum Either {
49 | Left(L),
50 | Right(R)
51 | }
52 |
53 | impl Eq for Either where L: Eq, R: Eq {
54 | fn eq(self, other: Self) -> bool {
55 | match (self, other) {
56 | (Either::Left(a), Either::Left(b)) => a.eq(b),
57 | (Either::Right(a), Either::Right(b)) => a.eq(b),
58 | _ => false
59 | }
60 | }
61 | }
62 |
63 | fn main() -> bool {
64 | let foo = Either::Left::(0u64);
65 | let bar = Either::Right::(false);
66 | foo == bar
67 | }
68 | ```
69 |
70 | The need for a declaration engine comes from the fact that currently, it is intractable to add trait constraints to the compiler. Currently, when type checking a method application, the method declaration for that method is inlined into the expression node of the method application itself. In the case where the 'parent' for that method (in the `a.eq(b)` example, `a` is the parent) is a generic type, it becomes intractable to inline the method declaration, because the compiler does not know what the method declaration is yet, because the type is generic! The method declaration could be from any `impl` block that implements the trait associated with the method (in this example `Eq`).
71 |
72 | ## Recursive Functions ([FueLabs/sway#1557](https://github.com/FuelLabs/sway/issues/1557))
73 |
74 | The intractability of inlining declarations can also be seen in the case of recursive functions. There are a handful of different reasons as to why the Sway compiler is currently unable to support recursive functions, and one of them is infinite computation and infinite code size, resulting from this problem.
75 |
76 | Take this simple example:
77 |
78 | ```rust
79 | script;
80 |
81 | fn zero_to_n(n: u8) -> u8 {
82 | if n == 0 {
83 | 0
84 | } else {
85 | n + zero_to_n(n-1)
86 | }
87 | }
88 |
89 | fn main() -> u8 {
90 | zero_to_n(100)
91 | }
92 | ```
93 |
94 | This code size and computation would explode infinitely without use of the declaration engine.
95 |
96 | ## Monomorphization Optimizations ([FueLabs/sway#862](https://github.com/FuelLabs/sway/issues/862))
97 |
98 | Currently, the compiler performs one iteration of monomorphization per "use" of a declaration. A "use" of a declaration includes function applications, method applications, struct expressions, and enum expressions. But, as you can imagine, this leads to quite a bit of repeated computation. By adding the declaration engine, this gives us the opportunity to introduce a cache system to reduce the number of monomorphizations.
99 |
100 | # Reference-level explanation
101 |
102 | [reference-level-explanation]: #reference-level-explanation
103 |
104 | This design will take after the design of the [type engine](https://github.com/FuelLabs/sway/blob/2816c13698a35752136d4843dbfe5e1b95c26e10/sway-core/src/type_engine/engine.rs#L14). During type checking, the compiler will add the typed declarations to the declaration engine. Then, in places where that declaration is used, a value representing that declaration can be generated in place of inlining the declaration.
105 |
106 | ### Declarations
107 |
108 | During type checking of a declaration, the compiler inserts the typed declaration to the declaration engine:
109 |
110 | ```rust
111 | struct DeclarationId(usize);
112 |
113 | enum DeclarationWrapper {
114 | Function(TypedFunctionDeclaration),
115 | Trait(TypedTraitDeclaration),
116 | TraitFn(TypedTraitFn),
117 | TraitImpl(TypedTraitImpl),
118 | Struct(TypedStructDeclaration),
119 | }
120 |
121 | struct DeclarationEngine {
122 | slab: ConcurrentSlab,
123 | // *declaration_id -> vec of monomorphized copies
124 | // where the declaration_id is the original declaration
125 | monomorphized_copies: LinkedHashMap>,
126 | }
127 |
128 | impl DeclarationEngine {
129 | fn look_up_decl_id(&self, index: DeclarationId) -> DeclarationWrapper {
130 | self.slab.get(index)
131 | }
132 |
133 | fn add_monomorphized_copy(
134 | &mut self,
135 | original_id: DeclarationId,
136 | new_id: DeclarationId,
137 | ) {
138 | match self.monomorphized_copies.get_mut(&*original_id) {
139 | Some(prev) => {
140 | prev.push(new_id);
141 | }
142 | None => {
143 | self.monomorphized_copies.insert(*original_id, vec![new_id]);
144 | }
145 | }
146 | }
147 |
148 | fn insert_function(&self, function: TypedFunctionDeclaration) -> DeclarationId {
149 | self.slab.insert(DeclarationWrapper::Function(function))
150 | }
151 |
152 | fn get_function(
153 | &self,
154 | index: DeclarationId,
155 | ) -> Result {
156 | self.slab.get(index).expect_function()
157 | }
158 |
159 | fn add_monomorphized_function_copy(
160 | &mut self,
161 | original_id: DeclarationId,
162 | new_copy: TypedFunctionDeclaration,
163 | ) {
164 | let new_id = self.slab.insert(DeclarationWrapper::Function(new_copy));
165 | self.add_monomorphized_copy(original_id, new_id)
166 | }
167 |
168 | fn get_monomorphized_function_copies(
169 | &self,
170 | original_id: DeclarationId,
171 | ) -> Result, String> {
172 | self.get_monomorphized_copies(original_id)
173 | .into_iter()
174 | .map(|x| x.expect_function())
175 | .collect::>()
176 | }
177 |
178 | // omit equivalent methods for structs, enums, traits, etc
179 | }
180 | ```
181 |
182 | Then in the typed AST, those declaration nodes look like:
183 |
184 | ```rust
185 | contract;
186 |
187 | // imports
188 |
189 | TypedDeclaration::Function()
190 | // omit equivalents for structs, enums, traits, etc
191 |
192 | // storage declaration
193 |
194 | TypedDeclaration::Function() // "main" function
195 | ```
196 |
197 | ### Usages
198 |
199 | When type checking an expression that references a declaration the compiler uses the unique `DeclarationId` that corresponds with the relevant declaration. For example, in a function application, the compiler uses the unique `DeclarationId` for the corresponding function declaration to retrieve the function declaration, apply monomorphization, perform any necessary type unification, and then access the function's return type to apply to the expression.
200 |
201 | Function application becomes:
202 |
203 | ```rust
204 | enum TypedExpressionVariant {
205 | FunctionApplication {
206 | call_path: CallPath,
207 | declaration_id: DeclarationId,
208 | arguments: Vec<(Ident, TypedExpression)>,
209 | // omitting other fields
210 | },
211 | // omitting other variants
212 | }
213 | ```
214 |
215 | # Drawbacks
216 |
217 | [drawbacks]: #drawbacks
218 |
219 | I believe that the only drawbacks are developer time and implementation complexity.
220 |
221 | # Rationale and alternatives
222 |
223 | [rationale-and-alternatives]: #rationale-and-alternatives
224 |
225 | I feel that this design offers the best compromise between current state and future growth. The concept of a "declaration engine" in general is very broad, which is a good thing! The design in this RFC is a good one to get us started, but I imagine that as the language progresses, it's needs and use cases for the declaration engine will expand and change. Introducing the declaration engine now gives us the opportunity to create a mechanism through which some of those changes could be introduced later down the road.
226 |
227 | That being said, from a high-level perspective, the alternative design would be something similar to what we are doing now, which is more focused on inlining things.
228 |
229 | # Prior art
230 |
231 | [prior-art]: #prior-art
232 |
233 | 1. [rustc item collection for monomorphization](https://github.com/rust-lang/rust/blob/8fe936099a3a2ea236d40212a340fc4a326eb506/compiler/rustc_monomorphize/src/collector.rs)
234 | 2. [rustc collection item context](https://github.com/rust-lang/rust/blob/a8f7e244b785feb1b1d696abf0a7efb5cb7aed30/compiler/rustc_hir_analysis/src/collect.rs)
235 | 3. [idris has a context it uses to resolve types](https://github.com/idris-lang/Idris2/blob/86c060ef13fd8194f849e2a4a4295cd37c0d061c/src/Core/Context/Context.idr)
236 | 4. @tritao wrote [something similar](https://github.com/tritao/CppSharp/blob/22c15789c551ed5d64b05ce48d0353b117865368/src/AST/ASTContext.cs) in his older project.
237 |
238 | # Unresolved questions
239 |
240 | [unresolved-questions]: #unresolved-questions
241 |
242 | I'd really like to get feedback on the fine-level details of the initial implementation that I described above. I can provide additional information or context if that is helpful too.
243 |
244 | # Future possibilities
245 |
246 | [future-possibilities]: #future-possibilities
247 |
248 | This RFC precedes an RFC introducing a collection context.
249 |
250 | ### Collection Context
251 |
252 | Sway wants to support 'out of order' declarations and dependencies and recursive declarations and dependencies. The former has pretty good support already, with some bugs, but the latter is not currently possible because the compiler requires all declarations to be in the 'proper' ordering before type checking can begin.
253 |
254 | I propose two new passes, a "node collection" pass, which constructs a graph of the untyped program relative to any one node. The information collected from this stage is used provide a "look ahead" ability for the "type collection" pass.
255 |
256 | And a "type collection" pass, which performs type checking on the "high-level" information for declarations. "High-level" information would include type checking a function signature (but not a function body), type checking a struct definition and the function signatures of its methods (but not the method bodies). Rust does this same step actually: https://rustc-dev-guide.rust-lang.org/type-checking.html#type-collection
257 |
258 | Once these are added, the compile would roughly look like:
259 |
260 | 1. parsing
261 | 2. node collection
262 | 3. type collection
263 | 4. type inference (the stuff remaining after type collection)
264 | 5. code generation
265 |
266 | **Note:**
267 |
268 | We probably won't be able to support recursive structs and certain recursive enums until we introduce pointer types.
269 |
--------------------------------------------------------------------------------
/rfcs/0005-collection-context.md:
--------------------------------------------------------------------------------
1 | - Feature Name: `collection_context`
2 | - Start Date: 2022-09-19
3 | - RFC PR: [FuelLabs/sway-rfcs#0015](https://github.com/FuelLabs/sway-rfcs/pull/15)
4 | - Sway Issue: [FueLabs/sway#1819](https://github.com/FuelLabs/sway/issues/1819)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | With its current design, the Sway compiler faces challenges regarding how nodes interact
11 | with one another during type checking. These include:
12 |
13 | 1. Mutually recursive dependencies are not supported, and could not be supported without
14 | extensive special casing.
15 | 2. Recursive AST nodes are not currently supported, and could not be supported without
16 | extensive special casing.
17 | 3. This inhibits the introduction of recursive functions, complex application development
18 | requiring circular dependencies, and more.
19 | 4. If a user forgets to import a dependency, it is not tractable to suggest an import to
20 | them at the compiler level without extensive special-casing.
21 |
22 | This RFC proposes a two-themed solution to this problem. At a high level, theme one of this
23 | RFC is a change to the internal design of the compiler to disentangle AST transformation
24 | from type inference and type checking. The second theme of this RFC is a new "collection
25 | context" that allows the compiler to introduce steps for graph collection and type
26 | collection, in addition to type inference and type checking. All together, this allows the
27 | compiler to solve the issues listed above.
28 |
29 | [_prototype system_](https://github.com/emilyaherbert/declaration-engine-and-collection-context-demo/tree/master/de)
30 |
31 | # Motivation
32 |
33 | [motivation]: #motivation
34 |
35 | Why are we doing this? What use cases does it support? What is the expected outcome?
36 |
37 | With the changes introduced by this RFC, the compiler will be able to solve the issues listed
38 | above. Sway could support recursive functions and complex applications that require
39 | circular dependencies. The push to disentangle AST transformation from the other inner
40 | AST operations will also introduce a new mechanism in the compiler that will allow for more
41 | robust growth in the future.
42 |
43 | # Guide-level explanation
44 |
45 | [guide-level-explanation]: #guide-level-explanation
46 |
47 | ## AST transformation disentanglement
48 |
49 | Currently, the compiler does all of the type inference and type checking in a single pass---the
50 | same pass as the AST transformation from the untyped AST to the 'fully typed AST'. This RFC
51 | disentangles these concepts, and replaces the 'fully typed AST' with a 'typeable AST'. All-in-all
52 | the 'typeable AST' is very similar to the 'fully typed AST'---a big part of this change is simply
53 | a mindset shift from 'fully typed' to 'typeable'. However, the 'typeable AST' includes additional
54 | fields that allow the compiler to index into the collection context.
55 |
56 | ## Collection Context
57 |
58 | At a high-level, the collection context represents an AST as a graph by adding AST nodes into
59 | a graph and by drawing edges between relevant AST nodes. An edge is created between two AST
60 | nodes when those two AST nodes share some scope relationship---sharing the same scope, one
61 | being a scoped child of another, etc.
62 |
63 | The collection context is a cyclical directed graph that allows for both parallel edges
64 | and for self-referential edges. This is useful for the AST because it allows us to model
65 | scoping relationships. The 'leaves' of the collection context are nodes from which no other
66 | nodes derive any nested scoping (think literal expressions, variable expression, etc). Edges
67 | in the graph are directed towards outward scope. For example, an AST node in a function body
68 | will have an edge pointing outward in scope towards the AST node of the function declaration
69 | itself. And that function declaration will have and edge pointing outward in scope to the
70 | file its in. In this way, the root of the graph is the entire application itself, as it
71 | encompasses the most outward scope.
72 |
73 | Modeling scope in this way is useful because it allows the compiler to 'look ahead' at
74 | 'future AST nodes' during type collection, type inference, etc. We can think of it in this way.
75 | Given this example (it's a silly example, don't think about the fact that this will never
76 | terminate, that doesn't matter for what we are looking at):
77 |
78 | ```rust
79 | fn ping(n: u64) -> u64 {
80 | return pong(n);
81 | }
82 |
83 | fn pong(n: u64) -> u64 {
84 | return ping(n);
85 | }
86 | ```
87 |
88 | During type inference, when evaluating the AST node for `return pong(n);`, we will need to
89 | know 1) the type signature of `pong` and 2) if `pong` exists at all and is in scope, but we
90 | don't know this information because we haven't done type inference on `pong` yet. Because these
91 | functions are mutually recursive, it is not possible to determine an ordering for which to do
92 | type inference. The collection context gives the ability for the compiler to either 'look
93 | forward' or 'look backward' in the AST given any location where type inference is currently 'standing'.
94 |
95 | ## Mutually Recursive Dependencies
96 |
97 | Take these two Sway files (currently does not compile):
98 |
99 | `alice.sw`:
100 |
101 | ```rust
102 | library alice;
103 |
104 | use ::bob::*;
105 |
106 | fn alice_fn(n: u64) -> u64 {
107 | if n == 0 {
108 | return 99;
109 | } else {
110 | return bob_fn(n-1);
111 | }
112 | }
113 | ```
114 |
115 | `bob.sw`:
116 |
117 | ```rust
118 | library bob;
119 |
120 | use ::alice::*;
121 |
122 | fn bob_fn(n: u64) -> u64 {
123 | if n == 0 {
124 | return 1;
125 | } else {
126 | return alice_fn(n-1);
127 | }
128 | }
129 | ```
130 |
131 | In this case, the collection context would allow the compiler to immutably search the contents of both
132 | `alice.sw` and `bob.sw`, in any order, regardless of whether `alice_fn` is being type checked first or
133 | `bob_fn` is.
134 |
135 | ## "Did you forget to import X in file Y?" Compiler Error
136 |
137 | Take the same example as above, but this time the user has forgotten an import statement:
138 |
139 | `alice.sw`:
140 |
141 | ```rust
142 | library alice;
143 |
144 | // use ::bob::*;
145 | //
146 | // ^^^ they forgot this
147 |
148 | fn alice_fn(n: u64) -> u64 {
149 | if n == 0 {
150 | return 99;
151 | } else {
152 | return bob_fn(n-1);
153 | }
154 | }
155 | ```
156 |
157 | `bob.sw`:
158 |
159 | ```rust
160 | library bob;
161 |
162 | use ::alice::*;
163 |
164 | fn bob_fn(n: u64) -> u64 {
165 | if n == 0 {
166 | return 1;
167 | } else {
168 | return alice_fn(n-1);
169 | }
170 | }
171 | ```
172 |
173 | When type checking `alice_fn`, the compiler is able to use the collection context to see that there does exist
174 | a `bob_fn` function, but that it is out of scope. This information is able to be used to gives better
175 | error messages and better error recovery.
176 |
177 | # Reference-level explanation
178 |
179 | [reference-level-explanation]: #reference-level-explanation
180 |
181 | The [proposed stages of the compiler](https://github.com/emilyaherbert/declaration-engine-and-collection-context-demo/blob/master/de_cc/src/lib.rs#L24) are:
182 |
183 | 1. parsing
184 | 2. [AST transformation from the untyped AST to the typeable AST and graph collection](https://github.com/emilyaherbert/declaration-engine-and-collection-context-demo/blob/master/de_cc/src/semantic_analysis/graph_collection/mod.rs)
185 | - insert instances of `TypeInfo` into the `TypeEngine`
186 | - insert declarations into the `DeclarationEngine`
187 | - create the collection context
188 | - NOT evaluate types in any way
189 | - NOT create constraints on types or perform type unification
190 | - NOT resolve instances of custom types
191 | 3. [type collection](https://github.com/emilyaherbert/declaration-engine-and-collection-context-demo/blob/master/de_cc/src/semantic_analysis/type_collection/mod.rs)
192 | - visiting all intraprocedural definitions (struct/enum/function/trait/etc declarations)
193 | - resolving custom types
194 | - associating type parameters with generics
195 | - NOT visiting non-intraprocedural definitions (function bodies are not visited)
196 | 4. [type inference (+ type checking)](https://github.com/emilyaherbert/declaration-engine-and-collection-context-demo/blob/master/de_cc/src/semantic_analysis/type_inference/mod.rs)
197 | - visiting all function bodies and expressions
198 | - resolving custom types
199 | - monomorphizing as needed
200 |
201 | ## Typeable AST
202 |
203 | The [typeable AST](https://github.com/emilyaherbert/declaration-engine-and-collection-context-demo/blob/master/de_cc/src/language/ty/mod.rs) is defined as (this is a super simplified subset):
204 |
205 | ```rust
206 | struct TyApplication {
207 | files: Vec>,
208 | }
209 |
210 | struct TyFile {
211 | name: String,
212 | nodes: Vec>,
213 | }
214 |
215 | enum TyNode {
216 | StarImport(String),
217 | Declaration(CCIdx),
218 | Expression(TyExpression),
219 | ReturnStatement(TyExpression),
220 | }
221 |
222 | enum TyDeclaration {
223 | Variable(TyVariableDeclaration),
224 | Function(CCIdx),
225 | Trait(CCIdx), // definition omitted for typeable trait declaration
226 | TraitImpl(CCIdx), // definition omitted for typeable trait impl
227 | Struct(CCIdx), // definition omitted for typeable struct declaration
228 | }
229 |
230 | struct TyFunctionDeclaration {
231 | name: String,
232 | type_parameters: Vec,
233 | parameters: Vec,
234 | body: CCIdx,
235 | return_type: TypeId,
236 | }
237 |
238 | struct TyFunctionParameter {
239 | name: String,
240 | type_id: TypeId,
241 | }
242 |
243 | struct TyCodeBlock {
244 | contents: Vec>,
245 | }
246 |
247 | struct TyExpression {
248 | variant: TyExpressionVariant,
249 | type_id: TypeId,
250 | }
251 |
252 | enum TyExpressionVariant {
253 | Literal {
254 | value: Literal,
255 | },
256 | Variable {
257 | name: String,
258 | },
259 | FunctionApplication {
260 | name: String,
261 | type_arguments: Vec,
262 | arguments: Vec,
263 | },
264 | FunctionParameter,
265 | Struct {
266 | struct_name: String,
267 | type_arguments: Vec,
268 | fields: Vec,
269 | },
270 | MethodCall {
271 | parent_name: String,
272 | func_name: String,
273 | type_arguments: Vec,
274 | arguments: Vec,
275 | },
276 | }
277 | ```
278 |
279 | `CCIdx` is an index into the [collection context](https://github.com/emilyaherbert/declaration-engine-and-collection-context-demo/blob/master/de_cc/src/collection_context/mod.rs):
280 |
281 | ```rust
282 | type CollectionGraph = petgraph::Graph;
283 |
284 | enum CollectionNode {
285 | StarImport(String),
286 | Application(TyApplication),
287 | File(TyFile),
288 | Expression(TyExpression),
289 | Return(TyExpression),
290 | Variable(String, TyVariableDeclaration),
291 | Function(String, DeclarationId),
292 | CodeBlock(TyCodeBlock),
293 | Trait(String, DeclarationId),
294 | TraitFn(String, DeclarationId),
295 | TraitImpl(String, DeclarationId),
296 | Struct(String, DeclarationId),
297 | }
298 |
299 | enum CollectionEdge {
300 | ApplicationContents,
301 | FileContents,
302 | SharedScope,
303 | ScopedChild,
304 | }
305 |
306 | struct CCIdx {
307 | inner: T,
308 | idx: CollectionIndex,
309 | }
310 |
311 | struct CollectionIndex(petgraph::NodeIndex);
312 |
313 | struct CollectionContext {
314 | pub(super) graph: CollectionGraph,
315 | pub(super) files: HashMap,
316 | }
317 | ```
318 |
319 | `CollectionEdge`'s are added between nodes that share a scoping relationship. For example,
320 | here is the pseudo-code for
321 | [doing AST transformation and graph collection for a function declaration](https://github.com/emilyaherbert/declaration-engine-and-collection-context-demo/blob/master/de_cc/src/semantic_analysis/graph_collection/declaration.rs):
322 |
323 | ```rust
324 | fn graph_collection_function(
325 | cc: &mut CollectionContext,
326 | func_decl: FunctionDeclaration,
327 | ) -> CCIdx {
328 | // 1. do graph collection on the function parameters
329 |
330 | // 2. do graph collection on the function body
331 | let .. = graph_collection_code_block(cc, func_decl.body);
332 |
333 | // 3. create the `TyFunctionDeclaration` object
334 | let func_decl = ..;
335 |
336 | // 4. insert the function into the declaration engine to get the declaration id
337 |
338 | // 5. add the declaration id to the collection context as a `CollectionNode::Function(..)`
339 |
340 | // 6. create a CCIdx for the declaration id
341 | let func_decl_cc_idx = ..;
342 |
343 | // 7. add an edge from the function body to the function declaration
344 | CCIdx::add_edge(
345 | &func_decl.body,
346 | &func_decl_cc_idx,
347 | CollectionEdge::ScopedChild,
348 | cc,
349 | );
350 |
351 | func_decl_cc_idx
352 | }
353 |
354 | fn graph_collection_code_block(cc: &mut CollectionContext, nodes: Vec) -> CCIdx {
355 | // 1. do graph collection on the ast nodes
356 | let nodes = ..;
357 |
358 | // 2. for every ast node in the code block, connect them under the same shared scope
359 | CCIdx::add_edges_many(&nodes, CollectionEdge::SharedScope, cc);
360 |
361 | // 3. create the `TyCodeBlock` object
362 | let code_block = ..;
363 |
364 | // 4. add the code block to the collection context
365 |
366 | // 5. create a CCIdx for the code block
367 | let code_block_cc_idx = ..;
368 |
369 | // 6. add an edge from every ast node to the code block
370 | CCIdx::add_edges_many_to_one(
371 | &code_block.contents,
372 | &code_block_cc_idx,
373 | CollectionEdge::ScopedChild,
374 | cc,
375 | );
376 |
377 | code_block_cc_idx
378 | }
379 | ```
380 |
381 | When type checking a function call, say in the ping-pong example above, the compiler searches for the
382 | `pong` function using a breadth-first search of the `CollectionContext`, meaning anything in the closest
383 | local scope is discovered first. Because the `TyDeclaration::Function` variant and the
384 | `CollectionNode::Function` variant both hold a `DeclarationId`, this `DeclarationId` can refer to the
385 | same typeable `TyFunctionDeclaration` across passes of type collection, type inference, and type checking.
386 | At certain moments, the types within this `TyFunctionDeclaration` may be `TypeInfo::Unknown` (after graph
387 | collection but before type collection), but as the typeable AST undergoes type inference and type checking,
388 | these `TypeId`'s resolve in the `TypeEngine` to concrete types.
389 |
390 | ## Recursive Data Structures
391 |
392 |
393 |
394 | From my understanding, it's impossible to do type inference on truly recursive data structures (i.e. those
395 | that would be of infinite size) as you need to
396 | [fulfill these requirements]().
397 | For example:
398 |
399 |
400 |
401 | ```rust
402 | struct Data1 {
403 | other: Data2
404 | }
405 |
406 | struct Data2 {
407 | other: Data1
408 | }
409 | ```
410 |
411 |
412 |
413 | In these instances, the compiler performs an [occurs check](https://papl.cs.brown.edu/2019/Type_Inference.html)
414 | to check whether the same type variable occurs on both sides and, if it does, it produces a compile error.
415 |
416 |
417 |
418 | I propose that we use an occurs check detect to recursive data structures until we fully support an equivalent
419 | to something like the Rust `Box` type, which would eliminate this problem.
420 |
421 | ## Program Visualization
422 |
423 | The `CollectionContext` uses [petgraph](https://github.com/petgraph/petgraph) under the hood, meaning we get
424 | to benefit from how awesome this library is. So, one of the cool things we can do is we can actually print
425 | the `CollectionContext` to a `.dot` file and visualize the graph.
426 |
427 | Given this simple generic function example (`UNK` is the unknown type in the prototype):
428 |
429 | ```rust
430 | fn F(n: T) -> T {
431 | let x: T = n;
432 | let y: UNK = 5u8;
433 | return x;
434 | };
435 |
436 | fn main() -> () {
437 | let foo: UNK = F(1u32);
438 | let bar: UNK = F(1u64);
439 | };
440 | ```
441 |
442 | We can visualize the program before type inference:
443 |
444 | 
445 |
446 | and after type inference, notice that the `UNK`'s have disappeared:
447 |
448 | 
449 |
450 | # Drawbacks
451 |
452 | [drawbacks]: #drawbacks
453 |
454 | I would say developer time and lots of merge conflicts. At the very least, disentangling AST transformation
455 | from type inference and type checking is a must-do, in my opinion, even if the collection context
456 | does not get accepted in this form.
457 |
458 | # Rationale and alternatives
459 |
460 | [rationale-and-alternatives]: #rationale-and-alternatives
461 |
462 | - Why is this design the best in the space of possible designs?
463 | - What other designs have been considered and what is the rationale for not choosing them?
464 | - What is the impact of not doing this?
465 |
466 | # Prior art
467 |
468 | [prior-art]: #prior-art
469 |
470 | Rust separates the ideas of
471 | [type collection](https://rustc-dev-guide.rust-lang.org/type-checking.html#type-collection),
472 | [type checking](https://rustc-dev-guide.rust-lang.org/type-checking.html#type-checking),
473 | and [type inference](https://rustc-dev-guide.rust-lang.org/type-inference.html). And these are
474 | all [disentangled from 'AST lowering'](https://rustc-dev-guide.rust-lang.org/overview.html#hir-lowering).
475 |
476 | Relating to the `CollectionContext`, the Rust compiler uses
477 | [queries for demand-driven development](https://rustc-dev-guide.rust-lang.org/query.html). Admittedly,
478 | I'm not as familiar with this on a technical level as I would like to be, but my guess is that they are
479 | using a graph-based approach (combined with the Rust `DefId`) and to achieve the memoization and caching.
480 |
481 | # Unresolved questions
482 |
483 | [unresolved-questions]: #unresolved-questions
484 |
485 | I'd really like to get feedback on how the `CollectionContext` could be improved for use with the LSP.
486 |
487 | In terms of what is out of scope for this RFC, I feel that supporting recursive data structures is
488 | definitely out of scope, as it will require us to implement a pointer type.
489 |
490 | # Future possibilities
491 |
492 | [future-possibilities]: #future-possibilities
493 |
494 | Eventually we should plan to move monomorphization to it's own
495 | [monomorphization collection step](https://rustc-dev-guide.rust-lang.org/backend/monomorph.html#collection)
496 | (likely after type inference).
497 |
498 | I'd really like to see us use the program visualization provided by petgraph. Maybe users could setup
499 | a debug profile for `forc` that allowed them to visualize their contract calls, visualize which library
500 | functions interact with which ABI functions, etc.
501 |
502 | And of course releasing recursive functions is going to be a huge with for Sway I think.
503 |
--------------------------------------------------------------------------------
/rfcs/0006-configuration-constants.md:
--------------------------------------------------------------------------------
1 | - Feature Name: `config_time_constants`
2 | - Start Date: 2022-10-01
3 | - RFC PR: [FuelLabs/sway-rfcs#0006](https://github.com/FuelLabs/sway-rfcs/pull/19)
4 | - Sway Issue: [FueLabs/sway#1498](https://github.com/FuelLabs/sway/issues/1498)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | Configuration time constants can be conceptualized as traditional environment variables. Some bytecode has been compiled by the Sway compiler, and the SDK would like to configure some behavior of that bytecode with additional inputs that won't trigger a recompile.
11 |
12 | There is a similar feature that was implemented [here][pr_2549], but that was a mistaken interpretation of the requirements. In [#2549][pr_2549], a recompile still occurs and the new values are injected via `Forc.toml`, when we actually want these values to be injectable by the SDK.
13 |
14 | [pr_2549]: https://github.com/FuelLabs/sway/pull/2549
15 |
16 | # Motivation
17 |
18 | [motivation]: #motivation
19 |
20 | The primary and only motivation that requires this feature is contract factories. To construct a set of trusted contracts from some template, we need to have a mapping of which parts of the bytecode are just configuration variables and which parts are application logic. By comparing the version with zeroed-out constants we can assert that two contracts are the same, modulo their configuration time constants.
21 |
22 | # Guide-level explanation
23 |
24 | [guide-level-explanation]: #guide-level-explanation
25 |
26 | When a Sway program is compiled, a set of artifacts are produced. Included in these artifacts are a [JSON ABI descriptor](https://fuellabs.github.io/fuel-specs/master/protocol/abi/json_abi_format.html#json-abi-spec) containing a configuration-time-constant descriptor, and the actual bytecode. The object we are discussing here is the configuration-time-constant descriptor, which describes the specific offset within the bytecode, in terms of bytes, to the data section entry for a particular configuration variable.
27 |
28 | The configuration variables for a program are declared in a `configurable` block similarly to `storage` variables. Each configurable variable requires a type annotation as well as an initializer. The value of each configurable variable can be overriden at configuration time by the SDK, `forc`, or some other consumer of the configuration-time-constant descriptor. As an example:
29 |
30 | ```rust
31 | // main.sw
32 | script;
33 |
34 | configurable {
35 | CONTRACT_ADDRESS: b256 = 0x0000000000000000000000000000000000000000000000000000000000000000,
36 | PRICE_RATIO: u64 = 0,
37 | }
38 |
39 | fn main() -> (u64, b256) {
40 | (PRICE_RATIO, CONTRACT_ADDRESS)
41 | }
42 | ```
43 | ```json
44 | // project-abi.json
45 | {
46 | ...
47 | "configurables": [
48 | {
49 | "configurableType": {
50 | "name": "",
51 | "type": 3,
52 | "typeArguments": null
53 | },
54 | "name": "CONTRACT_ADDRESS",
55 | "offset": 3260
56 | },
57 | ...
58 | ],
59 | ...
60 | ```
61 |
62 | As `configurable` values will affect a contract's ID, `forc` users must be able to specify these values for each of their contract dependencies. To enable this, `forc` will allow users to specify a `config/.sw` file for each contract dependency that contains configurable values. This Sway file must export the required constants as specified by the contract dependency's `configurable` block. These files will be compiled independently for each contract dependency, and the resulting `configurable` bytecode section for each dependency will be used to replace its default `configurable` bytecode section.
63 |
64 | # Reference-level explanation
65 |
66 | [reference-level-explanation]: #reference-level-explanation
67 |
68 | The `type` field of the JSON file should describe the type of the configurable in the same manner as types are described in the JSON ABI file.
69 |
70 | A single `configurable` block is allowed in a program and no two configurable variables in a program should have the same name to prevent collisions.
71 |
72 | The range from the offset to the offset plus the size of the type will represent bytes within the bytecode that will be overwritten by the SDK (or `forc`), which will write values according to the ABI encoder's memory layout, which is consistent with Sway's memory layout.
73 |
74 | The change is not breaking as it is entirely new functionality.
75 |
76 | Configuration time constants may not be defined in any library code and must be defined at the top level. Should a library need to reference a configurable value, it should utilize design patterns such as the [builder pattern](https://en.wikipedia.org/wiki/Builder_pattern) to ingest the constants from the top level.
77 |
78 | ## Failure to provide constants
79 |
80 | If the constants are not provided, the SDK and/or `forc` should throw an error. Both tools have knowledge of the required constants due to the descriptor file, so they may check for the constants' presence and report a descriptive error if any are missing.
81 |
82 | ## Interactions with optimization passes
83 |
84 | Configuration time constants are not allowed to be optimized away or constant folded or really touched in any way (for example, a config struct cannot be broken into into its individual elements). They always need an actual spot in the bytecode. So, the optimizor has take that into account. This can be accomplished with a "configurable" section that is entirely different from the data section and that can be left untouched by the rest of the compilation process. That being said, configurable variables that have no uses in a Sway program should be omitted from both the bytecode and the JSON ABI descriptor.
85 |
86 | ## Allowed Types
87 |
88 | Any type that can be handled by the ABI encoder and decoder can be passed along as a config-time constant.
89 |
90 | ## Memory layout
91 |
92 | The memory layout of configurable variables is identical to the memory layout of `const` variables which also matches the layout described in the [Argument Encoding](https://fuellabs.github.io/fuel-specs/master/protocol/abi/argument_encoding.html) section of the ABI spec.
93 |
94 | ## Specifying configuration-time constants for contract-dependencies in `Forc.toml`
95 |
96 | With the recent introduction of [`[contract-dependencies]`](https://github.com/FuelLabs/sway/pull/3016), it would be necessary to allow `forc` projects to specify the configuration constants declared within each of their contract-dependencies. This is important in allowing the user to refer to a specific contract, and to make sure we are constructing the correct contract ID constants during build.
97 |
98 | With configuration time constants, declaring `[contract-dependencies]` in `Forc.toml` might look something like:
99 |
100 | ```toml
101 | [contract-dependencies]
102 | foo = { git = "https://domain.com/owner/repo", config = "config/foo.sw" }
103 | ```
104 |
105 | where `config/foo.sw` has a single `configurable` block with each constant value specified.
106 |
107 | Alternatively, we could remove the need to specify the `config` file location manually by making the `config/` directory a standard location. We could require that, for each contract-dependency that has a `configurable` block, a file exists in the `config/` directory that matches the name of the dependency with `.sw` appended (e.g. `config/foo.sw`). In the case that no `config/foo.sw` file is present, we could either:
108 |
109 | 1. produce an error, or
110 | 2. fallback to using the contract dependency's default values, indicating that we are falling back with a `info!` log or similar.
111 |
112 | # Drawbacks
113 |
114 | [drawbacks]: #drawbacks
115 |
116 | One major drawback is the additional cognitive complexity this adds to the compilation process. If any bug is introduced or the overwriting of the bytecode goes awry in any way, it will result in confusing and inconsistent undefined behavior. We will need to introduce sufficient checks to ensure that an end user would never encounter this situation. An example of testing this behavior would be integration tests that pass in the majority of the ABI-encodable types, fuzzed, as config time constants, and assessing behavior.
117 |
118 | This increases our reliance on the correctness of the ABI encoder in the SDK, and any version mismatches or bugs in the encoder will result in similarly undefined and unpredictable behavior.
119 |
120 | This change also further couples the language to the SDK, further minimizing the use case of writing Sway without the SDK. This can be alleviated by introducing mechanisms to `forc` for defining these values manually, as mentioned in the guide-level explanation.
121 |
122 | # Rationale and alternatives
123 |
124 | [rationale-and-alternatives]: #rationale-and-alternatives
125 |
126 | One alternative is to introduce an environment variables/configuration-time constants section to the transaction format. This could be cleaner, but the impact on Sway would be minimal. Sway would still output a descriptor file, introduce the `configurable` keyword syntax, etc., and the majority of this RFC would still apply. The benefit of adding this to the transaction format would be potential simplification of the process. A couple negatives would be additional churn on the client, VM, and SDK; and further coupling of specifically Sway to the FuelVM.
127 |
128 | If we do not implement configuration-time constants, the workflow of deploying contracts and then calling them from scripts (i.e. the main use case of Sway) would still require manually updating contract addresses in the source code. Without an enshrined tool for configuring values like this, it is likely that some third party `sed`-like workaround would develop in the community to the detriment of the Fuel stack developer experience.
129 |
130 |
131 | # Prior art
132 |
133 | [prior-art]: #prior-art
134 |
135 | [Immutables in Solidity](https://docs.soliditylang.org/en/v0.6.5/contracts.html) are the closest prior art, effectively performing the exact same functionality. Environment variables in traditional software development are also similar in paradigm and utility.
136 |
137 | # Unresolved questions
138 |
139 | [unresolved-questions]: #unresolved-questions
140 |
141 | This change has been discussed in depth among SDK, Sway, and client developers and there is little ambiguity. There could be further iterations on minor details like the specific syntax in the language and the format of the descriptor file.
142 |
143 | # Future possibilities
144 |
145 | [future-possibilities]: #future-possibilities
146 |
147 | Someday, if possible without the SDK importing the compiler, perhaps some constant evaluation of more sophisticated expressions could be done in the SDK, although that's a minor use case.
148 |
149 | It is also possible that configuration time constants are added to the transaction format someday. If that happens, the compiler will not need to change dramatically but the client and SDK will have work to do.
150 |
--------------------------------------------------------------------------------
/rfcs/0008-storage-handler.md:
--------------------------------------------------------------------------------
1 | - Feature Name: `storage_keys`
2 | - Start Date: 2023-02-10)
3 | - RFC PR: [FuelLabs/sway-rfcs#0023](https://github.com/FuelLabs/sway-rfcs/pull/023)
4 | - Sway Issue: [FueLabs/sway#3043](https://github.com/FuelLabs/sway/issues/3043)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | This RFC introduces the concept of a `StorageKey` which describes a particular storage slot via its key. The RFC also reworks how storage accesses are represented in the language and how `StorageKey` can be used to build dynamic storage types, such as `StorageMap` and `StorageVec`, more robustly.
11 |
12 | # Motivation
13 |
14 | [motivation]: #motivation
15 |
16 | The current approach for reading and writing storage slots/variables has multiple flaws, despite being quite ergonomic:
17 |
18 | 1. There is currently no way to reason about the location of a particular storage variable, which makes passing "references" to storage variables impossible. One workaround is manipulating and reading storage slots manually via `std::storage::store` and `std::storage::get`, but this approach is quite unsafe. Alternatively, one could declare a `trait` in a library and implement it in a contract and have its methods access storage variables. Those methods are then used in the library to access storage indirectly. This does not solve the full problem and feels more like a workaround than a real solution.
19 | 1. Dynamic storage types, such as `StorageMap` and `StorageVec`, currently require a "hacky" compiler intrinsic called `__get_storage_key` which is not very well defined and is hard to use. Its implementation also requires that all methods using it are inlined, which is not something that can be guaranteed, even if `#[inline(always)]` is used. Introducing the concept of a `StorageKey` will help us completely remove this intrinsic.
20 | 1. The current implementation of dynamic storage types prevents them from being used as struct fields or as type parameters of other storage types (e.g. `StorageVec>`).
21 |
22 | # Guide-level explanation
23 |
24 | [guide-level-explanation]: #guide-level-explanation
25 |
26 | A `StrorageKey` is a thin wrapper type, in the standard library, around a `b256` key that describes a storage slot and a `u64` offset, in 64 bit words, from the beginning of that slot.
27 |
28 | ```rust
29 | pub struct StorageKey {
30 | key: b256,
31 | offset: u64,
32 | }
33 | ```
34 |
35 | The type `T` describes the type of the data that the `StorageKey` points to. Since each storage slot is 64 bytes long, some types cannot fit in a single storage slot, particuarly if `offset` is non-zero. In that case, the data is stored across multiple consecutive storage slots starting at `key + offset`. More on this in [Storage Slot Assignment](#storage-slot-assignment).
36 |
37 | > **Note**
38 | > We use the `+` sign in `key + offset` throughout this document to refer to the location in storage that `StorageKey { key, offset}` points to. For example, if `key = 0x0...00` and `offset = 10`, then `key + offset` points to the second word in storage slot `0x0...02` because `offset` is in words and each storage slot contains 4 words.
39 |
40 | Reading and writing the data stored at `key + offset` can be accomplished using the `read`, `try_read`, and `write` methods below:
41 |
42 | ```rust
43 | impl StorageKey {
44 | #[storage(read)]
45 | pub fn read(self) -> T {
46 | std::storage::read::(self.key, self.offset).unwrap()
47 | }
48 |
49 | #[storage(read)]
50 | pub fn try_read(self) -> Option {
51 | std::storage::read::(self.key, self.offset)
52 | }
53 |
54 | #[storage(write)]
55 | pub fn write(self, other: T) {
56 | std::storage::write(self.key, self.offset, value);
57 | }
58 | }
59 | ```
60 |
61 | This assumes the existence of functions `std::storage::read` and `std::storage::write` that read and write the appropriate storage slots given a key and an offset and handle all the appropriate storage slot assignment and data conversion.
62 |
63 | Notice that `StorageKey::try_read` returns an `Option` because it is possible that a particular storage slot is not valid, i.e. has not been written before, in which case `None` is returned. If the type `T` spans multiple storage slots, then the method `StorageKey::try_read` returns `None` if at least one the storage slots that `T` spans is invalid. The method `StorageKey::read` is similar to `StorageKey::read` except that it unwraps the returned `Option` internally.
64 |
65 | With the `StorageKey` type available, this RFC proposes a redefinition of the meaning of the expression `storage.....` to return a `StorageKey` "pointing" to the actual data instead of returning the data itself. The RFC also proposes removing the "reassignment" statement `storage.... = ..` from the language, at least temporarily.
66 |
67 | Below is an example showing the behavior of `StorageKey` and the new behavior of storage access expressions:
68 |
69 | ```rust
70 | struct MyStruct {
71 | x: u64,
72 | y: b256,
73 | }
74 |
75 | storage {
76 | s: MyStruct = MyStruct { x: 0, y: ZERO_B256 },
77 | }
78 |
79 | fn foo() {
80 | // Get the storage key to the struct field `x` directly and then use it to store `42` in `x`.
81 | let x_key: StorageKey = storage.s.x;
82 | x_key.write(42);
83 |
84 | // Call `write` and `try_read` directly on `storage.s.y`
85 | storage.s.y.write(ZERO_B256);
86 | let y: Option = storage.s.y.try_read();
87 |
88 | // Get the storage key to the struct `s` and then use it to store `MyStruct { x: 42, y: ZERO_B256 }` in `s`.
89 | let s_key: StorageKey = storage.s;
90 | s_key.write(MyStruct { x: 42, y: ZERO_B256 } );
91 |
92 | // Read the struct `s`
93 | let s: MyStruct = s_key.read();
94 | }
95 | ```
96 |
97 | Below is how the "counter" example now looks like:
98 |
99 | ```rust
100 | contract;
101 |
102 | abi TestContract {
103 | #[storage(write)]
104 | fn initialize_counter(value: u64) -> u64;
105 |
106 | #[storage(read, write)]
107 | fn increment_counter(amount: u64) -> u64;
108 | }
109 |
110 | storage {
111 | counter: u64 = 0,
112 | }
113 |
114 | impl TestContract for Contract {
115 | #[storage(write)]
116 | fn initialize_counter(value: u64) -> u64 {
117 | // Previoulsy: `storage.counter = value`
118 | storage.counter.write(value);
119 |
120 | value
121 | }
122 |
123 | #[storage(read, write)]
124 | fn increment_counter(amount: u64) -> u64 {
125 | // Previously: `let incremented = storage.counter + amount`
126 | let incremented = storage.counter.read() + amount;
127 |
128 | // Previously: `storage.counter = incremented`
129 | storage.counter.write(incremented);
130 |
131 | incremented
132 | }
133 | }
134 | ```
135 |
136 | > **Note**
137 | > Because storage variables defined in a `storage` block have to be initialized, it is generally often to call `StorageKey::read` to get back the data directly instead of having to handle the `Option` returned from `try_read`. This, of course, becomes unsafe if `asm` blocks that clear storage slots are used. Note that the standard library currently has a public method `clear` that we also propose that it should be made private to avoid potential foot guns.
138 |
139 | # Reference-level explanation
140 |
141 | [reference-level-explanation]: #reference-level-explanation
142 |
143 | The compiler will handle storage slots assignment for each storage variable. Each storage access expression will simply return a `StorageKey` containing the key and the offset chosen by the compiler.
144 |
145 | ## Storage Slot Assignment
146 |
147 | Each storage variable in a `storage` block is assigned an index corresponding to its location in the list of variables:
148 |
149 | ```rust
150 | struct MyInnerStruct {
151 | c: u64,
152 | d: b256,
153 | }
154 |
155 | struct MyStruct {
156 | a: u64,
157 | b: MyInnerStruct,
158 | }
159 |
160 | storage {
161 | x: u64 = 0, // Assigned index `0`
162 | y: b256 = ZERO_B256, // Assigned index `1`
163 | str: str[99], // Assigned index '2'
164 | s: MyStruct = MyStruct { a: 0, b: MyInnerStruct { c: 0, d: ZERO_B256 } }, // Assigned index `3`
165 | }
166 | ```
167 |
168 | The "index" assigned to each variable is used to generated a key as follows.
169 |
170 | ### "Small" Types
171 |
172 | Small types are types that fit in a single storage slot. Storage variables that have small types are packed and assigned a single key that is equal to `sha256("storage_")` where `` is the index assigned to the storage variable. In the example above, the key chosen for `x` is:
173 |
174 | ```rust
175 | sha256("storage_0") = 0xf383b0ce51358be57daa3b725fe44acdb2d880604e367199080b4379c41bb6ed
176 | ```
177 |
178 | and the key chosen for `y` is:
179 |
180 | ```rust
181 | sha256("storage_1") = 0xde9090cb50e71c2588c773487d1da7066d0c719849a7e58dc8b6397a25c567c0
182 | ```
183 |
184 | ### "Large" Types
185 |
186 | Large types span multiple storage slots and are packed and laid out sequentially in storage starting at key `sha256("storage_")` where `` is the index assigned to each storage variable. The number of storage slots required can be computed as `(__size_of::() + 31) >> 5` (ceiling of the size of the data type divided by `32`).
187 |
188 | For example, the storage variable `str` above is a string containing `99` characters and its size is `99` bytes which span 2 storage slots. It is also assigned index `2`. Therefore, `str` spans slots with keys `sha256("storage_2")` and `sha256("storage_2") + 1`.
189 |
190 | Variable `s` is a nested struct and is assigned index `3`. Therefore, its fields and subfields are stored as follows:
191 |
192 | - `s.a` is stored as the first word in storage slot with key `sha256("storage_3")`.
193 | - `s.b.c` is stored as the second word in storage slot with key `sha256("storage_3")` (same slot as `s.a`).
194 | - `s.b.d` is the third and fourth words in storage slot with key `sha256("storage_3")` as well as the first and second words in storage slot with key `sha256("storage_3") + 1`.
195 |
196 | ### Empty Types
197 |
198 | There are two possible empty types: structs with no fields and enums with no variants. Empty structs are useful for implementing dynamic storage types such as `StorageMap` and `StorageVec`. Storage variables that have empty types do not consume any storage slots.
199 |
200 | ### Behavior of `StorageKey`
201 |
202 | When a storage variable is accessed, a `StorageKey` is returned of the appropriate type. The `StorageKey` object contains the `b256` key and the `u64` offset required to access the underlying data:
203 |
204 | - `storage.x` returns `StorageKey { key: sha256("storage_0"), offset: 0 }`.
205 | - `storage.y` returns `StorageKey { key: sha256("storage_1"), offset: 0 }`.
206 | - `storage.str` returns `StorageKey { key: sha256("storage_2"), offset: 0 }`.
207 | - `storage.s` returns `StorageKey { key: sha256("storage_3"), offset: 0 }`.
208 | - `storage.s.a` returns `StorageKey { key: sha256("storage_3"), offset: 0 }`.
209 | - `storage.s.b` returns `StorageKey { key: sha256("storage_3"), offset: 1 }`.
210 | - `storage.s.b.c` returns `StorageKey { key: sha256("storage_3"), offset: 1 }`.
211 | - `storage.s.b.d` returns `StorageKey { key: sha256("storage_3"), offset: 2 }`.
212 |
213 | ## Implementation Detail
214 |
215 | The typed expression `TyStorageAccess` in the compiler should now return `std::storage::StorageKey` which should store the key and offset chosen by the compiler according to the rules described in [Storage Slots Assignment](#storage-slot-assignment). The rest of the flow will be handled automatically after removing all the unnecessary logic in the compiler for reading and writing storage slots.
216 |
217 | ## Implementing Dynamic Storage Types using `StorageKey`.
218 |
219 | Dynamic storage types such `StorageMap` and `StorageVec` currently use the intrinsic `__get_storage_key` which is not very well defined and has a fragile implementation. With `StorageKey` available and returned directly from a storage access expression (as in `storage.my_map`), we can re-implement methods like `StorageMap::insert`, `StorageMap::get`, and `StorageMap::remove` in a safer and cleaner way as follows:
220 |
221 | ```rust
222 | /// A persistent key-value pair mapping struct.
223 | pub struct StorageMap {}
224 |
225 | impl StorageKey> {
226 | #[storage(read, write)]
227 | pub fn insert(self, key: K, value: V) {
228 | let key = sha256((key, self.key));
229 | write::(key, 0, value);
230 | }
231 |
232 | #[storage(read)]
233 | pub fn get(self, key: K) -> StorageKey {
234 | StorageKey {
235 | key: sha256((key, self.key)),
236 | offset: 0,
237 | }
238 | }
239 |
240 | #[storage(write)]
241 | pub fn remove(self, key: K) -> bool {
242 | let key = sha256((key, self.key));
243 | clear::(key)
244 | }
245 | }
246 | ```
247 |
248 | With the above new implementation, we should be able to continue to use `StorageMap` as before:
249 |
250 | ```rust
251 | storage {
252 | my_map: StorageMap = StorageMap {}
253 | }
254 |
255 | storage.my_map.insert(0, 0);
256 | let x = storage.my_map.get(0).read();
257 | storage.my_map.remove(0);
258 | ```
259 |
260 | > **Note**
261 | > The method `get` now returns a `StorageKey` instead of the actual data because this simplifies nesting storage maps and other dynamic storage types.
262 |
263 | A similar approach can be followed to re-implement `StorageVec` and other dynamic storage types.
264 |
265 | ## Dynamic Storage Types in Structs
266 |
267 | The code below will now be valid after implementing all of the above:
268 |
269 | ```rust
270 | MyStruct {
271 | map1: StorageMap,
272 | map2: StorageMap,
273 | }
274 |
275 | storage {
276 | s: Mystruct = MyStruct {
277 | map1: StorageMap { },
278 | map2: StorageMap { },
279 | }
280 | }
281 |
282 | storage.s.map1.insert(0, 0);
283 | let x = storage.s.map2.get(0).read();
284 | ```
285 |
286 | It will also be possible to pass a `StorageMap` to a function by passing a `StorageKey` to it as follows:
287 |
288 | ```rust
289 | storage {
290 | my_map: StorageMap = StorageMap {}
291 | }
292 |
293 | fn foo(map: StorageKey>) {
294 | map.insert(0, 0);
295 | }
296 |
297 | fn bar() {
298 | foo(storage.my_map);
299 | }
300 | ```
301 |
302 | ## Nested Dynamic Storage Types
303 |
304 | Nesting dynamic storage type is now trivial given what we have so far:
305 |
306 | ```rust
307 | storage {
308 | nested_map_1: StorageMap>> = StorageMap {},
309 | }
310 |
311 | storage.nested_map_1.get(0).get(0).insert(0, 1);
312 | storage.nested_map_1.get(1).get(1).insert(1, 8);
313 | assert(storage.nested_map_1.get(0).get(0).get(0).read() == 1);
314 | assert(storage.nested_map_1.get(1).get(1).get(1).read() == 8);
315 | ```
316 |
317 | # Drawbacks
318 |
319 | [drawbacks]: #drawbacks
320 |
321 | There are two main drawbacks for the approach proposed above:
322 |
323 | 1. Reading and writing a storage variable is now a bit more complicated and less ergonomic than before because we now have to call `read` and `write` manually. We should consider making this more ergonomic by introducing a simpler syntax that de-sugars to the API calls above. For example, we could de-sugar something like `storage.x = 42` to `storage.x.write(42)`.
324 |
325 | 2. The second drawback is that, because sturcts are packed in storage, then we will often need to read a storage slot before writing to it because we may need to preserve parts of it if the value we're writing is smaller than the slot itself. This means that almost each storage write now also requires a storage read. This is the case in Solidity as well. The upside here is that a smaller number of storage slots will be needed overall.
326 |
327 | # Rationale and alternatives
328 |
329 | [rationale-and-alternatives]: #rationale-and-alternatives
330 |
331 | One alternative is to continue with the current use model which has many flaws so that's not ideal. Another alternative is to introduce an `AsStorageKey` trait that allows calling `storage.x.get_storage_key()` to get the storage key manually, but that approach has its limitations when implementing `StorageMap` and `StorageVec` and requires defining the behavior of `get_storage_key` when called on a regular stack variable.
332 |
333 | One more alternative is to completely remove the concept of a "storage variable" and switch over to thinking about storage like file I/O or databases. However, I think the approach described in this RFC provides a good middle ground where methods like `write` and `read` somewhat behave like I/O methods but also keeps the abstraction of "storage variables" to help user write readable code.
334 |
335 | # Prior art
336 |
337 | [prior-art]: #prior-art
338 |
339 | Solidity's approach to storage is completely different from the above and has its own problems, as storage variable are hard to identify and storage accesses are not explicit enough. Sway's current approach is also flawed as explained in the [Motivation](#motivation) section.
340 |
341 | In Rust, the closest thing to storage accesses is file I/O where a handler is used to read and write to a file similarly to `write` and `read` above.
342 |
343 | # Unresolved questions
344 |
345 | [unresolved-questions]: #unresolved-questions
346 |
347 | - How do we make storage accesses more ergonomic? Or at least as ergonomic as they currently are?
348 |
349 | # Future possibilities
350 |
351 | [future-possibilities]: #future-possibilities
352 |
353 | ## Unpacked storage structs.
354 |
355 | Unpacking structs in storage could be useful in certain situations. The advantage of unpacking sturcts is that each field and subfield gets its own storage slot which often makes reading and writing those fields and subfields cheaper. We may want to consider an annotation in the future that requests that a given struct is unpacked in storage.
356 |
357 | ## `Index` and `IndexAssign` traits
358 |
359 | We can introduce the traits `Index` and `IndexAssign` which would allow implementing `index()` and `index_assign()` for dynamic storage types such as `StorageMap` and `StorageVec` as follows:
360 |
361 | ```rust
362 | impl Index for StorageHandle>
363 | fn index(self, key: K) -> StorageKey {
364 | StorageKey {
365 | key: sha256((key, self.key)),
366 | offset: 0,
367 | }
368 | }
369 | }
370 |
371 | impl IndexAssign for StorageHandle>
372 | #[storage(read, write)]
373 | pub fn index_assign(self, key: K, value: V) {
374 | let key = sha256((key, self.key));
375 | write::(key, 0, value);
376 | }
377 | }
378 | ```
379 |
380 | Note that this deviates from Rust which has `Index` and `IndexMut`, but `IndexMut` likely requires mutable references in the language which we don't have. However, there has been [attempts](https://github.com/rust-lang/rfcs/pull/1129) to introduce `IndexAssign` in Rust but without any success due to reasons that I don't think apply to Sway.
381 |
382 | With the above, the compiler can then de-sugar expressions like `a[i]` to `a.index(i)` and `a[i] = b` to `a.index_assign(i, b)`. For a nested `StorageMap`, the resulting user code would look like:
383 |
384 | ```rust
385 | struct M {
386 | u: b256,
387 | v: u64,
388 | }
389 |
390 | storage {
391 | nested_map_2: StorageMap<(u64, u64), StorageMap>> = StorageMap {},
392 | }
393 |
394 | let m1 = M {
395 | u: 0x1111111111111111111111111111111111111111111111111111111111111111,
396 | v: 1,
397 | };
398 |
399 | storage.nested_map_2[(1, 0)]["0000"][0] = m1;
400 | assert(storage.nested_map_2[(1, 0)]["0000"][0].read() == m1);
401 | ```
402 |
--------------------------------------------------------------------------------
/rfcs/0009-structured-modules.md:
--------------------------------------------------------------------------------
1 | - Feature Name: `standard_module_structure`
2 | - Start Date: 2023-02-14
3 | - RFC PR: [FuelLabs/sway-rfcs#24](https://github.com/FuelLabs/sway-rfcs/pull/24)
4 | - Sway Issue: [FueLabs/sway#4191](https://github.com/FuelLabs/sway/issues/4191)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | This RFC proposes a change to the way defining modules works in Sway.
11 |
12 | Instead of requiring programmers to specify a path to their module definitions
13 | (as in: `dep dir1/dir2/dep_name;`) and a possibly different library name (as in
14 | `library library_name;`), this proposes to have the programmers specify a
15 | single name for a module through the Rust-like syntax of `mod module_name;` and
16 | have the path to the file containing the associated source code and the library
17 | name used for imports be implicitly defined.
18 |
19 | # Motivation
20 |
21 | [motivation]: #motivation
22 |
23 | The current `dep` based import system is confusing and possibly error prone in
24 | that it allows the definition of surprising and unpredictable module structures
25 | that can be completely detached in name from how those modules are imported.
26 |
27 | This permissive scheme also allows namespace collisions wherein two separate
28 | packages or modules can be imported despite having the same name.
29 |
30 | The proposed change should formalize a predictable module structure, reduce the
31 | number of editions required when refactoring a module and eliminate the
32 | possibility of namespace collisions between modules of the same package.
33 |
34 |
35 | # Guide-level explanation
36 |
37 | [guide-level-explanation]: #guide-level-explanation
38 |
39 | `mod` declarations replace the now deprecated `dep` declarations. `mod`
40 | declarations include a single ident specifying the name of the submodule. Note
41 | that this is different to `dep` declarations which would supply a `/`-separated
42 | file system path to the submodule. Since the name of the submodule is specified
43 | in the declaration, submodules no longer need to supply their name via a
44 | `library name;` header at the start of the file and can instead use the plain `library;`
45 |
46 | The name of a submodule is used to find the source file for that submodule. For
47 | a submodule named `bar` of a module `foo`, either the file `foo/bar.sw` or
48 | `bar.sw` must exist. These paths are relative to the file defining the `foo`
49 | module containing the `mod bar;` declaration. It is an error for both or none
50 | of these paths to exist.
51 |
52 | Module paths behave the same as they previously did except that they use module
53 | names instead of library names.
54 |
55 | ## Example
56 |
57 | These example sources show the rules in action.
58 |
59 | * `src/lib.sw`
60 | ```sway
61 | library;
62 |
63 | mod foo;
64 | mod bar;
65 |
66 | use foo::fun1;
67 | use bar::fun2;
68 | use bar::bar::fun4;
69 |
70 | pub fn function() -> u32 {
71 | fun1() + fun2() + fun4()
72 | }
73 | ```
74 |
75 | * `src/foo.sw`
76 | ```sway
77 | library;
78 |
79 | pub fn fun1() -> u32 { 1 }
80 | ```
81 |
82 | * `src/bar.sw`
83 | ```sway
84 | library;
85 |
86 | mod foo;
87 | mod bar;
88 |
89 | use foo::fun3;
90 | use bar::fun4;
91 |
92 | pub fn fun2() -> u32 {
93 | fun3() + fun4()
94 | }
95 | ```
96 |
97 | * `src/bar/foo.sw`
98 | ```sway
99 | library;
100 |
101 | pub fn fun3() -> u32 {
102 | 3
103 | }
104 | ```
105 |
106 | * `src/bar/bar.sw`
107 | ```sway
108 | library;
109 |
110 | pub fn fun4() -> u32 {
111 | 4
112 | }
113 | ```
114 |
115 |
116 |
117 | # Reference-level explanation
118 |
119 | [reference-level-explanation]: #reference-level-explanation
120 |
121 | The implementation requirements of this change are deliberately small and
122 | straightforward: introduce `mod` statements to replace `dep` statements, remove
123 | the need for an ident argument to `library`, deduce the library name from
124 | the source folder structure, add checks to ensure the structure isn't ambiguous.
125 |
126 | The interactions with other features are minimal.
127 | However we might be able to refactor some compiler behavior after establishing
128 | that dependency and library names are no longer different.
129 |
130 | # Drawbacks
131 |
132 | [drawbacks]: #drawbacks
133 |
134 | This is a breaking change and will require changing all sway module imports and
135 | restructuring projects that use the existing permissive scheme in creative
136 | ways, but this is both rare (it seems that most follow Rust like conventions
137 | already in practice) and will result in better code structure and less
138 | boilerplate.
139 |
140 | # Rationale and alternatives
141 |
142 | [rationale-and-alternatives]: #rationale-and-alternatives
143 |
144 | We could also follow the existing Rust convention and allow both `submodule.sw`
145 | and `submodule/mod.sw` or only allow the latter instead of the former as this
146 | proposal does. But Rust's allowance of both is an artifact of backwards
147 | compatibility rather than a deliberate design choice. It is likely best to
148 | settle on and enforce a single convention.
149 |
150 | If we do not enforce any module structure and hold to the current behavior,
151 | the issues and confusion will remain and it will become harder to change in the
152 | future.
153 |
154 | # Prior art
155 |
156 | [prior-art]: #prior-art
157 |
158 | Rust's module structure is the prior art we are taking cues from, but there is
159 | also a [previous, now abandoned,
160 | RFC](https://github.com/FuelLabs/sway-rfcs/pull/8) that sought similar but more
161 | wide changes to the way paths and module imports work in Sway.
162 |
163 | The present RFC tries to make the change as small and painless as possible so
164 | that we may build upon it later.
165 |
166 | # Unresolved questions
167 |
168 | [unresolved-questions]: #unresolved-questions
169 |
170 | The elimination of file type headers such as `library;` or `contract;` when
171 | they aren't needed is not considered here. Neither are wider issues about the
172 | structure and type of package exports. But clarifying the module structure will
173 | definitely give us a more stable base to consider these.
174 |
175 | We also do not consider the visibility of modules here and reproduce the
176 | public by default scheme of `dep`.
177 |
178 | # Future possibilities
179 |
180 | [future-possibilities]: #future-possibilities
181 |
182 | We could consider eliminating module type headers like `library;`, `contract;`
183 | and `predicate;` from the source altogether and defining the exports in the
184 | `Forc.toml` or implicitly through usage.
185 |
186 | We should also consider making the visibility of modules explicit through a
187 | `pub mod` syntax and make them private by default.
188 |
189 | Another extension could be to support `mod foo { /* ... */ }` syntax as Rust
190 | does to specify, possibly nested, submodules inside of the same source file.
191 |
--------------------------------------------------------------------------------
/rfcs/0010-private-modules.md:
--------------------------------------------------------------------------------
1 | - Feature Name: `private_modules`
2 | - Start Date: 2023-04-05
3 | - RFC PR: [FuelLabs/sway-rfcs#0007](https://github.com/FuelLabs/sway-rfcs/pull/25)
4 | - Sway Issue: [FueLabs/sway#4446](https://github.com/FuelLabs/sway/issues/4446)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | This RFC proposes introducing two changes.
11 |
12 | First, instead of exposing the entire module structure by default and
13 | modulating the privacy of elements globally and one by one, this RFC proposes
14 | to make all modules private by default and only allow importing symbols
15 | externally if they have explicitly been made public, including submodules.
16 |
17 | Second, and to make the former change easier to deal with, we seek to introduce
18 | public reexports with the `pub use` syntax.
19 |
20 | # Motivation
21 |
22 | [motivation]: #motivation
23 |
24 | This is a follow up to our previous change in the definition of the module
25 | structure, and the aim is to further clarify and make deliberate the module
26 | structure of Sway packages.
27 |
28 | This pair of features will help prevent leaky
29 | abstractions by hiding the implementation details of a module by default. It will help
30 | Sway programmers to design clear and deliberate library APIs that are not bound
31 | by the specifics of an implementation.
32 |
33 | # Guide-level explanation
34 |
35 | [guide-level-explanation]: #guide-level-explanation
36 |
37 | ## Visibility and Privacy
38 |
39 | Sway's symbol name resolution operates on a global hierarchy of namespaces.
40 | To control whether a symbol can be used in the context of a module we check if
41 | each use can be allowed or not, and if not produce an error.
42 |
43 | **By default everything is private.** There are two exceptions to this:
44 |
45 | * Associated items of a public trait are public by default
46 | * Enum variants of a public enum are public by default
47 |
48 | When an item is marked with `pub`, it is made public. It is public in the sense
49 | that it is accessible to the outside world.
50 |
51 |
52 | We allow item access in two cases:
53 |
54 | * if an item is public, then it can be accessed outside of a module `m` if you
55 | can access all the item's ancestor modules from `m`. You may also be able to
56 | name the item through reexports.
57 | * if an item is private, then it can be accessed by the current module and
58 | its descendants.
59 |
60 | ## Reexports
61 |
62 | We allow publicly reexporting items using the `pub use` syntax. This allo the
63 | item to be imported according to the visibility rules above as if it were
64 | declared where the reexport is stated. It also brings the item to the scoped
65 | context the same way a regular `use` would.
66 |
67 | ## Example
68 |
69 | Here's a small Sway library that combines all these concepts.
70 |
71 | `lib.sw`
72 | ```sway
73 | library;
74 |
75 | // this module will be accessible outside the library
76 | pub mod alpha;
77 | // this module will not be accessible outside the library
78 | mod beta;
79 |
80 | fn baz() {
81 | ::alpha::foo();
82 |
83 | // Error: ::alpha::bar is private
84 | // ::alpha::bar();
85 |
86 | ::beta::foo();
87 |
88 | // Error: ::beta::bar is private
89 | // ::beta::bar();
90 |
91 | // Error: ::beta::gamma is private
92 | // ::beta::gamma::foo();
93 |
94 | // Error: ::beta::gamma is private
95 | // ::beta::gamma::bar();
96 |
97 | ::beta::gamma_foo();
98 |
99 | }
100 |
101 | fn main() { baz() }
102 | ```
103 |
104 | `alpha.sw`
105 | ```sway
106 | library;
107 |
108 | pub fn foo() {}
109 | fn bar() {}
110 | ```
111 |
112 | `beta.sw`
113 | ```sway
114 | library;
115 |
116 | mod gamma;
117 |
118 | pub use gamma::foo as gamma_foo;
119 | // Error: gamma::bar is private
120 | // pub use gamma::bar as gamma_bar;
121 |
122 | pub fn foo() {}
123 | fn bar() {}
124 | ```
125 |
126 | `beta/gamma.sw`
127 | ```sway
128 | library;
129 |
130 | pub fn foo() {}
131 | fn bar() {}
132 | ````
133 |
134 | ## Migration
135 |
136 | This is of course a breaking change. Existing codebases can replicate the
137 | current behavior by replacing all their instances of `mod` by `pub mod`.
138 | However it is also a good opportunity to think about the API design of
139 | a codebase and only publicize what is required.
140 |
141 | # Reference-level explanation
142 |
143 | [reference-level-explanation]: #reference-level-explanation
144 |
145 | We'll need to add the optional `pub` qualifier to both import statements and
146 | module declarations.
147 |
148 | Internally, this change will require a more complex handling of `Visibility` in
149 | import handling and when we resolve call paths. We will need to validate the
150 | visibility of all elements of a path.
151 |
152 | This may require adding module declarations to the declaration engine because
153 | we will need to hold the visibility of modules. This would also be helpful if
154 | we want to later allow importing and reexporting modules themselves.
155 |
156 | We should also give specific consideration to imports of enum variants and
157 | trait associated items.
158 |
159 | As for reexports, they should be straightforward to implement as a specific
160 | behavior of type checking `AstNodeContent::UseStatement` with or without a
161 | special `TyDecl` variant.
162 |
163 | # Drawbacks
164 |
165 | [drawbacks]: #drawbacks
166 |
167 | This is yet another breaking change that will require a lot of edits from Sway
168 | programmers (if trivial ones). We may want to reconsider if we don't
169 | care to hold to Rust's private by default idiom, though it is probably in our
170 | best interest to do such a drastic change as early as possible if we want to do
171 | it all.
172 |
173 | # Rationale and alternatives
174 |
175 | [rationale-and-alternatives]: #rationale-and-alternatives
176 |
177 | These simple rules are a powerful and battle tested way of creating module
178 | hierarchies that hide implementation details.
179 | They are also what programmers coming from Rust would expect.
180 |
181 | Not changing this default would make it difficult to hide the implementation
182 | details of a Sway library or provide a specific interface.
183 | And the longer we wait, the harder it will be to make this change if we want to.
184 |
185 | # Prior art
186 |
187 | [prior-art]: #prior-art
188 |
189 | Once again, Rust's privacy rules and the long history of modular programming
190 | languages are the prior art we take cues from.
191 |
192 | # Unresolved questions
193 |
194 | [unresolved-questions]: #unresolved-questions
195 |
196 | This RFC does not propose to allow importing modules by themselves (i.e.: `use foo;`
197 | for a `mod foo;`), although it should be taken in consideration as a
198 | possible extension by the implementation.
199 |
200 | One thing the implementation may want to consider is an optimized or cached way
201 | of doing visibility checks for paths as enforcing the new rules will be of a
202 | higher order of complexity than a single declaration check.
203 |
204 | # Future possibilities
205 |
206 | [future-possibilities]: #future-possibilities
207 |
208 | As mentioned, allowing module imports is a possible extension.
209 |
210 | Another natural extension is to allow qualifiers to `pub` the same way
211 | that Rust does to allow for more fine grained control of privacy similar to
212 | `pub(crate)`, `pub(self)`, etc.
213 |
--------------------------------------------------------------------------------
/rfcs/0011-references.md:
--------------------------------------------------------------------------------
1 | - Feature Name: `references`
2 | - Start Date: 2023-08-17
3 | - RFC PR: [FuelLabs/sway-rfcs#28](https://github.com/FuelLabs/sway-rfcs/issues/28)
4 | - Sway Issue: [FueLabs/sway#5063](https://github.com/FuelLabs/sway/issues/5063)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | This RFC aims to make the paradigm for handling reference types clear, fully typed and to
11 | eliminate redundant concepts.
12 |
13 | # Motivation
14 |
15 | [motivation]: #motivation
16 |
17 | There currently exists a kludge of features on top of pointers that have
18 | various levels of type correctness, offer various paradigms and have less than
19 | coherent syntax.
20 |
21 | We want to clarify the rules for passing dynamic data to and from functions, how
22 | they are expressed in the type system and through syntax, and eliminate untyped
23 | values such as `raw_ptr` and `raw_slice`.
24 |
25 | # Guide-level explanation
26 |
27 | [guide-level-explanation]: #guide-level-explanation
28 |
29 | Unlike Rust, Sway does not manage lifetimes. Due to the nature of smart contract
30 | execution, dynamic memory usage only grows and allocated memory is allocated for
31 | the entire lifetime of the execution. In Rust terms, every dynamic value has a
32 | lifetime of `'static`.
33 |
34 | This allows us to do away with some complexity and handle values more like they
35 | would be in Java or ML.
36 | In most cases, you shouldn't need to use explicit pointer types.
37 |
38 | ## Data types
39 |
40 | In Sway there are two types of data types. Built-in types and compound types.
41 |
42 | Built-in types include `u8`, `u16`, `u32`, `u64`, and pointers such as `*mut u8`.
43 | Compound types are arrays, structs, tuples, and enums.
44 |
45 | Both built-in and compound types live for the lifetime of the function they are declared in.
46 | They are passed and returned by value, which means their value is directly copied.
47 |
48 | Internally, access to compound types (also called aggregates within the compiler) is
49 | realized via pointer to a memory region. That's why those types are internally called
50 | reference types. Note that from the Sway programmer's perspective, they still have value semantics.
51 |
52 | There are two kinds of such internal references. Slim and fat.
53 |
54 | Slim references are implemented using a single address. Slim references
55 | include structs, enums, and arrays, such as `String`, `Option`, `[u8; 42]`.
56 |
57 | Fat references are implemented using an address and a small amount of additional
58 | data, either a length or a pointer to a more complex table. Fat references
59 | include slices, such as `[u8]`, `str`.
60 |
61 | ## References
62 |
63 | References mentioned above are an internal compiler construct, but from the programmer's
64 | perspective still regular value types, copied in assignments, and passed and returned by value.
65 |
66 | From the programmer's perspective, it is sometimes desirable to have a reference to a value type,
67 | such as for passing a mutable reference to value to a function or declaring a recursive data type.
68 |
69 | This can be achieved with the new language construct, _references_.
70 |
71 | References to values can be obtained by the reference operator &.
72 | References have their own mutability and can point to mutable or immutable values.
73 | Thus, we support mutable references to mutable values and any combination of the two.
74 |
75 | The below example shows how a reference can be declared.
76 |
77 | ```sway
78 | let mut m_i = 0u64;
79 | let r_m_i = &mut m_i; // Immutable reference to a mutable `m_i`.
80 | let mut m_r_m_i = &mut m_i; // Mutable reference to a mutable `m_i`.
81 | let r_m_i = &m_i; // Immutable reference to an immutable (via reference) `m_i`.
82 | let mut r_m_i = &m_i; // Mutable reference to an immutable (via reference) `m_i`.
83 |
84 | let i = 0u64;
85 | let r_m_i = &mut i; // ERROR: `i` is not mutable.
86 | ```
87 |
88 | The dereference operator * is used to access the referenced value.
89 | In addition, if the reference refers to a compound type, the operators . and [] automatically dereference.
90 |
91 | ```sway
92 | let mut r_i = &i; // Mutable reference to immutable a `u64`: r_i: &u64.
93 | let mut r_m_i = &mut m_i; // Mutable reference to a mutable `u64`: r_i: &mut u64.
94 |
95 | *r_i = 1; // ERROR: Referenced value is not mutable.
96 | r_1 = &x; // OK: `r_1` is mutable.
97 |
98 | *r_m_i = 1; // OK: Changes `m_i`.
99 | r_m_i = &x; // OK: `r_m_i` is mutable.
100 |
101 | // Accessing built-in types and enums over reference via dereferencing operator (*).
102 | let a = 2 * *r_i;
103 |
104 | let mut s = Struct { x: 0u64 };
105 | let r_s = &mut s; // `r_s` is immutable reference to a mutable struct.
106 |
107 | r_s.x = 1; // Same as `(*r_s).x = 1`.
108 | ```
109 |
110 | Here, we are listing the major properties of references:
111 | - References have their own mutability and can point to mutable or immutable values.
112 | - & operator defines the reference. * operator is dereferencing.
113 | - . and [] operators also dereference if the reference is a reference to a struct/tuple or array, respectively.
114 | - References can be parts of aggregates.
115 | - References can reference other references.
116 | - References can be used in pattern matching and deconstructing.
117 | - References can reference parts of aggregates. E.g., having a reference to an array element.
118 | - References can be passed to and returned from functions.
119 | - References will play well with iterators and the `for` loop, once implemented.
120 | - References can be used together with generics.
121 | - References can be taken from arbitrary expressions, including constants and literals.
122 | - References can be used in type aliases.
123 | - `self` keyword will become a reference, and us such comply to the reference passing syntax.
124 | - References cannot be used in storage.
125 | - References cannot be used in ABIs and `main` functions.
126 | - Equality of references is the equality of the values they refer to if the underlying type implements `std::ops::Eq`.
127 | - `__addr_of` called on a reference returns the address the reference points to.
128 |
129 | For detailed examples of syntax and semantics see the accompanied file [0010-references.sw](../files/0010-references.sw).
130 |
131 | ## Pointers
132 |
133 | There are cases where one may want to directly manipulate memory addresses with pointer arithmetic.
134 | To allow for this we have a pointer type that represents a single address: `*mut T` where `T` is the type being pointed to.
135 |
136 | Pointers can be obtained by using the `__addr_of` intrinsic on a reference and dereferenced using the `__deref` intrinsic, like so:
137 |
138 | ```sway
139 | let val: u64 = 1;
140 | let ptr: *mut u64 = __addr_of(&val);
141 | let ptr_val: u64 = __deref(ptr);
142 | assert_eq(val, ptr_val);
143 |
144 | let ref: &u64 = &1;
145 | let ptr: *mut &u64 = __addr_of(&ref);
146 | let ptr_val: &u64 = __deref(ptr);
147 | assert_eq(ref, ptr_val); // Equality of references is the equality of referenced values.
148 |
149 | let ref: &u64 = &1;
150 | let ptr: *mut u64 = __addr_of(ref); // Returns the address od the referenced value.
151 | let ptr_val: u64 = __deref(ptr);
152 | assert_eq(ref, &ptr_val); // Equality of references is the equality of referenced values.
153 | ```
154 |
155 | A pointer that does not point to memory currently allocated by the program is considered to be an _invalid pointer_.
156 | Dereferencing an invalid pointer is an undefined behavior that depends on the underlying VM implementation.
157 |
158 | E.g., FuelVM has the concept of stack and heap, and instructions that access memory must pass the [ownership check](https://specs.fuel.network/master/fuel-vm/index.html#access-rights). Thus, dereferencing and accessing an invalid pointer in case of the FuelVM will result in VM panic.
159 |
160 | Other VMs could have different behavior. Thus, from the language perspective, dereferencing an invalid pointer is an undefined behavior.
161 |
162 | ## Slices
163 |
164 | Slices represent contiguous areas of dynamic memory.
165 | They can be used to represent dynamically sized data.
166 | Slices are represented by a pair containing a pointer to the data and a length.
167 |
168 | String slices, of type `str` represent a series of bytes, encoding a valid UTF-8 string.
169 | This is the type returned by string literals.
170 |
171 | ```sway
172 | let _: str = "Lorem Ipsum";
173 | ````
174 |
175 | Typed slices, of type `[T]`, represent contiguous series of elements of a type `T`.
176 |
177 | ```sway
178 | let _: [u64] = [1, 2, 3].as_slice();
179 | ````
180 |
181 | Slices can be obtained from arrays and other slices by using the `__slice`
182 | intrinsic. This will produce a smaller slice of the argument type at the
183 | specified indices. This slicing is bounds-checked and will produce a revert if
184 | slicing out of bounds.
185 |
186 |
187 | ```sway
188 | let array: [u64; 4] = [1, 2, 3, 4];
189 |
190 | // will produce 1, 2, 3, 4
191 | let slice: [u64] = __slice(array, 0, 4);
192 |
193 | // will produce 2, 3
194 | let slice: [u64] = __slice(array, 1, 3);
195 |
196 | // will revert
197 | // let slice: [u64] = __slice(array, 0, 5);
198 | ````
199 |
200 | Elements of slices can be obtained using the `__slice_elem` intrinsic.
201 | This will return the element at the given index. Invalid indices will produce a revert.
202 |
203 | ```sway
204 | let slice: [u64] = [1, 2, 3, 4].as_slice();
205 |
206 | let elem = __slice_elem(slice, 2);
207 | assert_eq(elem, 3);
208 |
209 | // will revert
210 | // let elem = __slice_elem(slice, 4);
211 | ```
212 |
213 | ## Passing values
214 |
215 | When values are passed as arguments to a function they are either mutable or
216 | immutable which is denoted by the `mut` prefix before the argument name.
217 |
218 | Mutable value type arguments can be reassigned.
219 |
220 | Mutable reference arguments can both be reassigned and have their fields
221 | assigned to, depending on the declaration of the reference.
222 |
223 | ```sway
224 | fn foo(
225 | ref: &u32, // Immutable reference to immutable value.
226 | mut m_ref: &u32, // Mutable reference to immutable value.
227 | mut m_ref_m: &mut u32, // Mutable reference to a mutable value.
228 | val: u32 // Immutable value passed by-value.
229 | mut m_val: u32, // Mutable value passed by-value.
230 | ) {
231 | *ref = 0; // ERROR: The referenced value is immutable.
232 | ref = &0; // ERROR: The reference `ref` is immutable.
233 |
234 | *m_ref = 0; // ERROR: The referenced value is immutable.
235 | m_ref = &0; // OK: The reference `m_ref` is mutable.
236 |
237 | *m_ref_m = 0; // OK: The referenced value is mutable.
238 | m_ref_m = &0; // OK: The reference `m_ref_m` is mutable.
239 |
240 | val = 0; // ERROR: `val` is immutable.
241 |
242 | m_val = 0; // OK: `m_val` is mutable.
243 | }
244 | ```
245 |
246 | For more detailed examples see the section _Passing and returning references from functions_ in the accompanied file [0010-references.sw](../files/0010-references.sw).
247 |
248 | ## Returning values
249 |
250 | When returning values from functions, copies are returned, as per the by-value semantics of all Sway types.
251 |
252 | When returning references, the mutability of the referenced value can be explicitly specified.
253 |
254 | ```sway
255 | fn foo() -> &u32 { // Returns a reference to an immutable `u32`.
256 | &0
257 | }
258 |
259 | fn bar() -> &mut u32 { // Returns a reference to a mutable `u32`.
260 | &0
261 | }
262 |
263 |
264 | pub fn main() {
265 | let r_foo = foo();
266 | *r_foo = 1; // ERROR: `r_foo` is a reference to immutable value.
267 |
268 | let r_bar = bar();
269 | *r_bar = 1; // OK: `r_bar` is a reference to mutable value.
270 |
271 | let mut m_r_foo = foo();
272 | m_r_foo = &1; // OK: `m_r_foo` is a mutable reference to immutable value.
273 |
274 | let mut r_bar = bar();
275 | m_r_bar = &1; // OK: `m_r_bar` is a mutable reference to mutable value.
276 | }
277 |
278 | ```
279 |
280 | For more detailed examples see the section _Passing and returning references from functions_ in the accompanied file [0010-references.sw](../files/0010-references.sw).
281 |
282 |
283 | ## Methods
284 |
285 | Methods definitions use `self` to refer to the value the method is being called
286 | on, this follows the same rules defined for functions, except of course `self`
287 | can't be reassigned.
288 |
289 | ```sway
290 | struct S {
291 | a: u32,
292 | }
293 |
294 | impl S {
295 | pub fn foo(&self) {
296 | // illegal since self is immutable
297 | // self.a = 0;
298 | }
299 | pub fn foo(&mut self) {
300 | // legal since self is mutable
301 | self.a = 0;
302 | }
303 | }
304 |
305 | ```
306 |
307 | For more detailed examples see the section _`self` keyword_ in the accompanied file [0010-references.sw](../files/0010-references.sw).
308 |
309 |
310 | ## Migration
311 |
312 | Existing codebases can adapt to this change by changing every occurrence of `ref
313 | mut` to `&mut` and every use of `raw_slice` and `raw_ptr` to uses of references, typed
314 | pointer and slice types.
315 |
316 | In general, switching untyped pointers to a properly typed reference, even a generic
317 | `&T`, is preferred. Pointers for whom the pointee isn't a value that can be
318 | named may need to use `&()` or `*mut ()`, the former is preferred.
319 |
320 | # Reference-level explanation
321 |
322 | [reference-level-explanation]: #reference-level-explanation
323 |
324 | ## Slices
325 |
326 | Implementing the new slice types should require:
327 |
328 | * parser changes to introduce the new syntax
329 | * adding the new types to the type system (this is already partially done)
330 | * implementing the `__slice` and `__slice_elem` intrinsics
331 |
332 | The slicing intrinsics should be thoroughly tested (specifically on the bounds checking).
333 |
334 | There are considerations to be given to the element sizes. We'll want to finish
335 | current changes to the memory representation of small values so that `[u8]`
336 | represents a continuous series of bytes without padding or provide this capability in other ways.
337 |
338 | One thing to keep in mind is that slices' dynamic size makes them difficult to
339 | deal with for the current ABI and they can't be returned as element of other
340 | dynamic types. This is to be addressed by [a later RFC](https://github.com/FuelLabs/sway-rfcs/issues/29).
341 |
342 | String literals are meant to return `str` and string array types are meant to
343 | be totally replaced by `str`, however due to this ABI issue we'll need to keep
344 | them around until further changes so string data can still be passed through
345 | the ABI. We may want to consider temporarily introducing an intrinsic to convert
346 | a string slice to a string array of a given size (with or without some bounds
347 | checking).
348 |
349 | `raw_slice` backs a lot of our dynamic types but we should be able to replace
350 | most of its uses with a properly represented `[u8]`.
351 |
352 | ## References
353 |
354 | Implementing the new reference construct should require:
355 |
356 | * parser changes to introduce the new syntax for referencing (`&`) and dereferencing (`*`).
357 | * adding the new types to the type system, `&T` and `&mut T`.
358 | * implementing heap allocation for values.
359 | * implementing escape analysis to avoid heap allocation.
360 |
361 | Escape analysis will be improved over time. We want to stepwise come to the solution that
362 | heap-allocates only values whose actual lifecycle is `static` and that cannot be promoted
363 | to live on a stack of any of the functions in the call stack.
364 |
365 | ## Pointers
366 |
367 | `*mut` should provide a fully typed alternative to `raw_ptr`.
368 |
369 | `__addr_of` will have to be altered to return a `*mut` and we'll need to
370 | introduce `__deref`.
371 |
372 | We will not check pointer mutability at this time, hence the `mut` in `*mut`,
373 | however we'll probably want to introduce `*const` once typed pointers are
374 | stabilized.
375 |
376 | Using this instead of `raw_ptr` will require heavy edits of `std` and `core`.
377 |
378 | We may need to support pointer casts without asm block hacks. Introducing a
379 | `__ptr_cast` intrinsic is on the table if this becomes a requirement.
380 |
381 |
382 | # Drawbacks
383 |
384 | [drawbacks]: #drawbacks
385 |
386 | Changing the base assumptions about what a type represents is a major change and
387 | programmers coming from Rust may be surprised at the differences. However the
388 | differences are easy enough to explain and being able to shed the complexity of
389 | borrow semantics is worth the cost. We may want to make the particulars of how
390 | references work in Sway explicit in documentation aimed at programmers coming
391 | from Rust.
392 |
393 | Abstracting away memory management too much might be a drawback for Sway and its
394 | positioning as a smart-contract language. However maintaining access to pointer
395 | arithmetic should still allow for atypical usage patterns without compromising
396 | the use of safe and zero cost abstractions for most cases.
397 |
398 | # Rationale and alternatives
399 |
400 | [rationale-and-alternatives]: #rationale-and-alternatives
401 |
402 | To make the use of references more explicit and maintain syntax legibility for
403 | people coming from Rust we have chosen to use `&` everywhere a reference
404 | is used. However the reason for Rust's use of this syntax rather than ML's
405 | terseness is to support a borrow checker and guarantees that we do not make.
406 | This could be misleading at first for programers coming from Rust. We will properly
407 | document the difference between Sway references and Rust borrowing.
408 | We believe that the difference, properly presented, will be straightforward and
409 | easy to understand. Apart from that, the chosen syntax should feel natural for
410 | programers coming from Rust.
411 |
412 | Since we're leaning more towards ML, we may also have chosen to use `ref`
413 | to denote reference types. However we already majorly borrow
414 | from Rust for our library abstractions. Moreover, the `ref` syntax felt much verbose.
415 |
416 | # Prior art
417 |
418 | [prior-art]: #prior-art
419 |
420 | A lot of the design choices of this RFC have to do with striking the balance
421 | between Rust and ML's levels of abstraction. Sway is not meant to be a general
422 | purpose systems programming language, but it is not a functional memory managed
423 | language either. We want to maintain access to memory primitives whilst making
424 | most Sway code terse, legible and easily understandable.
425 |
426 | As prior art we considered Rust, ML and other managed GC languages such as Java
427 | or C# that have similar simplified memory models. For the parts of the references
428 | semantics we borrow from C++.
429 |
430 |
431 | # Unresolved questions
432 |
433 | [unresolved-questions]: #unresolved-questions
434 |
435 | The exact memory representation of values, by themselves or as part of a slice
436 | is not fully addressed here. The ability to represent a series of bytes is a
437 | hard requirement but how we do this and what specific types look like in memory
438 | will have to be decided by the implementation.
439 |
440 | How dynamic types interact with the ABI will have to be resolved in a later RFC.
441 |
442 | # Future possibilities
443 |
444 | [future-possibilities]: #future-possibilities
445 |
446 | We should consider introducing slicing and indexing operator such as `[0..n]`
447 | and `[0]`. We needn't introduce the notion of a range yet to make those work.
448 |
449 | As discussed, a natural extension of `*mut` is `*const`.
450 |
--------------------------------------------------------------------------------
/rfcs/0012-expressive-diagnostics.md:
--------------------------------------------------------------------------------
1 | - Feature Name: `expressive_diagnostics`
2 | - Start Date: 2023-08-07
3 | - RFC PR: [FuelLabs/sway-rfcs#30](https://github.com/FuelLabs/sway-rfcs/pull/30)
4 | - Sway Issue: [FueLabs/sway#5079](https://github.com/FuelLabs/sway/issues/5079)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | Expressive diagnostics will provide detailed, helpful, and human friendly diagnostics (warnings and errors) to Sway programmers. The diagnostic messages will be focused on the code that the programmer wrote and will lead the programmer to a possible issue resolution. At the same time, the internal implementation of diagnostics will empower Sway language developers to easily define detailed and helpful diagnostics. The implementation, the Diagnostics API, will also enforce consistent warning and error reporting.
11 |
12 | # Motivation
13 |
14 | [motivation]: #motivation
15 |
16 | There is an overwhelming evidence, based on experience[1],[2],[3],[4] and research[5],[6], that shows that the quality of compiler diagnostics has a significant influence on:
17 |
18 | - the programmer's perception of the language
19 | - steepness of the language learning curve
20 | - and programmer's productivity.
21 |
22 | Experience from several communities, like Rust[2], Elm[3], and C++[1],[5],[7], to name a few, has shown that improving diagnostics had a great impact on the three points listed above.
23 |
24 | At the moment, Sway diagnostics vary significantly in level of details, wording, and helpfulness. We have messages that strive to be helpful and give hints to the programmer. E.g.:
25 |
26 | ```
27 | Generic type \"{name}\" is not in scope. Perhaps you meant to specify type parameters in \
28 | the function signature? For example: \n`fn \
29 | {fn_name}<{comma_separated_generic_params}>({args}) -> ... `
30 | ```
31 |
32 | The way how we structure those messages is, however, not standardized. E.g., here is an another attempt to provide a detailed error explanation, formatted and conveyed in a completely different way:
33 |
34 | ```
35 | expected: {expected} \n\
36 | found: {given} \n\
37 | help: The definition of this {decl_type} must \
38 | match the one in the {interface_name} declaration.
39 | ```
40 |
41 | At the moment, although detailed and helpful, these messages are "packed" within a single piece of text pointing to a single place in code. This significantly limits amount of helpful information that can be provided to the programmer. It also limits the presentation of the context in which the diagnostic occurs. The overall context usually spans across several points in code. We want to be able to place the useful information on the exact places in code that are relevant to the diagnostic.
42 |
43 | Sway programmers are also confronted with short, sometimes cryptic error messages, containing jargon used by compiler developers. E.g.:
44 |
45 | ```
46 | Invalid value \"{value}\"
47 | ```
48 |
49 | ```
50 | Constant requires expression.
51 | ```
52 |
53 | Internally, expressive diagnostics will empower Sway language developers to easily define detailed and helpful diagnostics that will be focused on the exact code that the programmer wrote. Diagnostics API will also provide a unified way to define diagnostics. (At the moment we have three different approaches for defining diagnostics. `CompileError`, `CompileWarning`, and `ParseError` are representatives of those different approaches.)
54 |
55 | Externally, we expect to see the effects experienced in other communities:
56 |
57 | - better perception of Sway as a language
58 | - easier language learning curve
59 | - increased efficiency in resolving compiler errors and warnings.
60 |
61 | # Guide-level explanation
62 |
63 | [guide-level-explanation]: #guide-level-explanation
64 |
65 | Diagnostics will be displayed in different clients. E.g.:
66 |
67 | - `forc` CLI output.
68 | - VS Code hints.
69 | - VS Code compiler output.
70 | - VS Code Problems.
71 |
72 | Each client can choose which part of diagnostic to show and how.
73 |
74 | When using `forc` CLI Sway programmers will get detailed diagnostics consisting of the following elements:
75 |
76 | - _Level_: Error or Warning.
77 | - _Code_: Unique code of the diagnostic.
78 | - _Reason_: Short description of the diagnostic, not related to a specific issue that the programmer did. The _reason_ answers the question "Why is this an error or warning?" E.g., Because - "Match pattern variable is already defined". It points out the general language rule that was violated.
79 | - _Issue_: Description of the concrete issue caused by the programmer, placed in the source code. E.g., "Variable "x" is already defined in this match arm."
80 | - _Hints_: Detailed descriptions of the diagnostic, placed in the source code. They point to other places in code that give additional contextual information about the issue.
81 | - _Help_: Additional friendly information, that helps better understanding and solving the issue. Same as hints, help entries can be related to a place in code, or they can be placed in the footnotes.
82 |
83 | 
84 |
85 | When using IDEs like VS Code Sway programmers will have the experience similar to one offered by the [Rust analyzer](https://rust-analyzer.github.io/).
86 |
87 | Popup in the editor could provide the _reason_, _issue_, _hints_, as well as the _help_, similar to this Rust example:
88 |
89 | 
90 |
91 | Programmer will have the option to open the full compiler diagnostic, getting the same output like when using `forc` CLI, similar to this Rust example:
92 |
93 | 
94 |
95 | Diagnostics will also be displayed in VS Code problems:
96 |
97 | 
98 |
99 | ## Wording guidelines
100 |
101 | When defining diagnostics, it is important to have in mind all the clients listed above. A diagnostic must not be optimized for one output, e.g., CLI but be hardly understandable in e.g., VS Code hints.
102 |
103 | To ensure consistency and apply best practices[5] the diagnostics will follow these guidelines:
104 |
105 | - _Reason_ will be short and point out the language constraint which was violated. It will _not_ finish in full stop, to emphasize brevity.
106 | - _Issue_ will be short and point out the specific situation encountered in code. It is written in plain english, using proper punctuation and grammar rules.
107 | - _Reason_ and _issue_ are given in plain english language, free of, e.g., compiler jargon.
108 | - _Hints_ and _help_ are as well written in plain english, using proper punctuation and grammar rules.
109 | - _Hints_ and _help_ try to give as much of useful context as possible and to be as specific as possible.
110 | - _Help_ in footnotes should be used rarely, only for general explanations and suggestions. Preferably, _help_ should be related to a place in code.
111 | - Identifier and type names in messages are enclosed in "double quotes". E.g., "x" or "(u32, bool)".
112 | - Code samples in messages are enclosed in \`grave accents\`. E.g., \`let x = 0\`.
113 | - Articles "the" and "a/an" are not used at the beginning of a sentence. E.g., "Variable "X" is already..." instead of "The variable "X" is already...". They can be used in formulations like "This is _the_ original declaration...".
114 |
115 | To avoid unnecessary complexity that comes through high number of diagnostic reasons (both for Sway language developers and Sway programmers), we will introduce new error codes restrictively. Reusing existing error codes and reasons will be the preferable option. To communicate specific cases we will use _hints_ and _help_.
116 |
117 | Here is an example. The existing error message:
118 |
119 | ```
120 | Name "MyName" is defined multiple times.
121 | ```
122 |
123 | could, depending on the context, have the following form:
124 |
125 | ```
126 | Reason: Item names must be unique
127 | Issue: There is already an imported struct with the name "MyName"
128 | Hints: [error] This is the enum "MyName" that has the same name as the imported struct.
129 | [info] The struct "MyName" gets imported here.
130 | [info] This is the original declaration of the imported struct "MyName".
131 | Help: Items like structs, enums, traits, and ABIs must have a unique name in scope.
132 | Consider renaming the enum "MyName" or using an alias for the imported struct.
133 | ```
134 |
135 | ## Diagnostic codes
136 |
137 | A diagnostic _code_ uniquely identifies a particular _reason_. Considering that we already have these diagnostic areas:
138 |
139 | - Lexical analysis
140 | - Parsing
141 | - Parse tree conversion
142 | - Type checking
143 | - Semantic analysis
144 | - Warnings
145 |
146 | the pragmatic way to ensure uniqueness and assignment of a new code number is to have a code made of three parts:
147 |
148 | <letter E or W for error or warning respectively><one digit area code><three digit code unique for area>
149 |
150 | E.g, _E4001_ would be the first error in the semantic analysis area.
151 |
152 | Diagnostic codes will be used to:
153 |
154 | - point to detailed diagnostic explanation similar to [Rust online help for error messages](https://doc.rust-lang.org/error_codes/error-index.html).
155 | - suppress warnings by using a Sway equivalent of Rust's [#[allow]](https://docs.rs/allow/latest/allow/).
156 |
157 | In case of warnings, in addition to the _code_ explained above, diagnostic will have a human-readable identifier which will allow us to have human-readable, self-documenting `#[allow]`s, similar to the [Clippy lint identifiers](https://rust-lang.github.io/rust-clippy/master/index.html#/Configuration).
158 |
159 | When [defining warnings via proc-macros](#reference-level-explanation), the human-readable identifier will be defined, next to the _code_:
160 |
161 | ```Rust
162 | ...
163 | #[warning(
164 | reason(1, name_is_not_idiomatic, "Name is not idiomatic"),
165 | ...
166 | ```
167 |
168 | This identifier can then be used in `#[allow]`s:
169 |
170 | ```Rust
171 | #[allow(name_is_not_idiomatic)]
172 | type int8_t = i8;
173 | ```
174 |
175 | In addition, we will add the option to Sway compiler to treat warnings as errors.
176 |
177 | # Reference-level explanation
178 |
179 | [reference-level-explanation]: #reference-level-explanation
180 |
181 | We already have the `Diagnostic` struct modeled after the definition given above. This allows us to enforce the usage of expressive diagnostics via API that models the explained approach.
182 |
183 | `forc` uses the `ToDiagnostic` trait, implemented at the moment only by `CompileError` and `CompileWarning`, to rended expressive diagnostics for error and warning messages that support them.
184 |
185 | Both implementations of `ToDiagnostic` provide a fallback that renders a diagnostic indistinguishable from the existing error and warning messages. This ensures backward compatibility and gradual switch to expressive diagnostics.
186 |
187 | At the moment, `Diagnostic` instances can be created only imperatively, by using the Diagnostics API and manually creating the whole `Diagnostic` structure, `Reason`, `Code`, `Issue`, `Hint`s, etc. This is cumbersome and requires writing boilerplate code. It can be compared to creating error messages without [thiserror](https://docs.rs/thiserror/latest/thiserror/) as we do in `CompileWarning`.
188 |
189 | The proposal is to have a proc-macro for declarative definition of an expressive diagnostic.
190 |
191 | E.g., the const shadowing diagnostic given above would be defined as follows:
192 |
193 | ```Rust
194 | #[derive(Diagnostic, Debug, Clone, PartialEq, Eq, Hash)]
195 | #[diagnostic(area = DiagnosticArea::SemanticAnalysis)]
196 | pub enum CompileError {
197 | ...
198 | #[error(
199 | reason(1, "Constants cannot be shadowed"),
200 | issue(
201 | name.span(),
202 | "{variable_or_constant} \"{name}\" shadows {}constant with the same name",
203 | if constant_decl.is_some() { "imported " } else { "" }
204 | ),
205 | info(
206 | constant_span.clone(),
207 | "Constant \"{name}\" {} here{}.",
208 | if constant_decl.is_some() { "gets imported" } else { "is declared" },
209 | if *is_alias { " as alias" } else { "" }
210 | ),
211 | info(
212 | constant_decl.map(|x| x.clone()),
213 | "This is the original declaration of the imported constant \"{name}\"."
214 | ),
215 | error(
216 | name.span(),
217 | "Shadowing via {} \"{name}\" happens here.",
218 | if variable_or_constant == "Variable" { "variable" } else { "new constant" }
219 | ),
220 | help("Unlike variables, constants cannot be shadowed by other constants or variables."),
221 | help(
222 | match (variable_or_constant.as_str(), constant_decl.is_some()) {
223 | ("Variable", false) => format!("Consider renaming either the variable \"{name}\" or the constant \"{name}\"."),
224 | ("Constant", false) => "Consider renaming one of the constants.".to_string(),
225 | (variable_or_constant, true) => format!(
226 | "Consider renaming the {} \"{name}\" or using {} for the imported constant.",
227 | variable_or_constant.to_lowercase(),
228 | if *is_alias { "a different alias" } else { "an alias" }
229 | ),
230 | _ => unreachable!("We can have only the listed combinations: variable/constant shadows a non imported/imported constant.")
231 | }
232 | )
233 | )]
234 | ConstantsCannotBeShadowed {
235 | variable_or_constant: String,
236 | name: Ident,
237 | constant_span: Span,
238 | constant_decl: Option,
239 | is_alias: bool,
240 | },
241 | ...
242 | }
243 | ```
244 |
245 | _Issue_ and _infos_ point to places in code, and are commonly named _labels_. The places in code they refer to are given by `Span`s.
246 | `issue` and `info` elements of `#[error]` take `Span` or `Option` as their first argument, as seen above. If the passed `Span` parameter is `None` the _label_ is considered _not to be in source code_.
247 |
248 | For _hints_, not being in source code means that they are not rendered at all. Diagnostics API will ignore them and clients (CLI, LSP) will not be aware of them.
249 |
250 | For the _issue_, clients will always display it as a part of diagnostic description. If it is in code, clients can choose to additionally display it in code.
251 | E.g., `forc` always shows the _issue_ text as a part of the diagnostic description, but if the _issue_ is in code and there is a _hint_ pointing to the same place in code, the _issue_ will not be rendered as a _label_ in code. This overloading of the _issue_ text by a _hint_ is visible on the above example and is enforced via Diagnostics API. It gives us the flexibility to phrase the diagnostic description and the hints in an independent way, while allowing backward compatibility with the current warnings and errors that have only a single message and span.
252 |
253 | Using `None` to denote non-existence of an element allows declarative definition of all the _hints_, knowing that some of them might not be shown, depending on the concrete `Span` value passed to the enum variant.
254 |
255 | Our `Diagnostic` derive and `thiserror`'s `Error` derive would coexist as long as the "old-style" diagnostic do not get fully replaced with expressive diagnostic. The existing fallback mechanism would still be generated by the `Diagnostic` derive.
256 |
257 | Treating warnings as errors should be straightforward. It requires an additional compilation flag whose check will be added at final receivers of the diagnostics, before the next step in the compilation pipeline. E.g., before we generate IR we treat warnings as errors and stop the IR generation.
258 |
259 | # Drawbacks
260 |
261 | [drawbacks]: #drawbacks
262 |
263 | The only drawback I can think of is the time and effort needed to fully roll out the expressive diagnostic. This means to replace _all_ of the existing diagnostics with their expressive counterparts.
264 |
265 | Out of the experience gained while working on [supporting multi-span errors and warnings](https://github.com/FuelLabs/sway/issues/21) I can say that crafting a good expressive diagnostic takes time, as well as adjusting existing e2e tests. Coding the diagnostics itself is a negligible effort.
266 |
267 | But this replacement can and should be done gradually, as a side task. It should be distributed among Sway language developers and other teams that want to contribute. The same approach was taken, e.g., by [Rust](https://github.com/rust-lang/rust/issues/35233) and [Elm](https://elm-lang.org/news/compiler-errors-for-humans) communities.
268 |
269 | # Rationale and alternatives
270 |
271 | [rationale-and-alternatives]: #rationale-and-alternatives
272 |
273 | Not implementing expressive diagnostics would mean staying behind the modern approach to compiler diagnostics. Expressive diagnostics are becoming de facto standard in compiler development (see the [Prior art](#prior-art) section below). To position Sway as a modern and developer friendly language, we have to follow this standard. I don't see how we can choose not to implement expressive diagnostics.
274 |
275 | To the proposed design, the approach and its alternatives were shortly discussed when implementing [support for multi-span errors and warnings](https://github.com/FuelLabs/sway/issues/21).
276 |
277 | ## Alternative 1: Using generic diagnostic structure
278 |
279 | This approach would mean having a possibility do define an arbitrary diagnostic, similar to an arbitrary output that can be achieved by using `Snippet`, `Slice`, `Annotation`, and other abstractions provided by [annotate-snippets](https://github.com/rust-lang/annotate-snippets-rs). We have concluded that this additional freedom would not give us any more expressive power, but just create additional confusion when defining diagnostics. It would also very likely lead toward inconsistencies similar to those we have now (different wording, different ways to explain an issue, etc.).
280 |
281 | ## Alternative 2: Evaluating and using `miette` crate
282 |
283 | [miette](https://github.com/zkat/miette) crate provides a holistic library for defining, reporting, and rendering diagnostics. We have concluded that using `miette` would mean changing our existing infrastructure that already works well plus losing the flexibility if we decide to do thing differently then envisioned by `miette`. The conclusion was to approach `miette` as an inspiration for our own API.
284 |
285 | # Prior art
286 |
287 | [prior-art]: #prior-art
288 |
289 | ## Expressive diagnostics
290 | Expressive diagnostics are gaining relevance, or are already significant part of compilers like:
291 |
292 | - Rust[2]
293 | - GCC[5], [7]
294 | - Clang[1]
295 | - Elm[3]
296 |
297 | Microsoft puts an effort in improving diagnostics in their VC++ compiler[5].
298 |
299 | Java error messages, traditionally knowing to be terse and difficult for novices and students, are enhanced with tools like Decaf, or Expresso.
300 |
301 | ## Using macros for diagnostic definition
302 |
303 | `thiserror` and its macros are de facto standard for defining error messages in Rust.
304 |
305 | `miette` follows the same approach for defining rich diagnostics, with its `diagnostic` macro:
306 |
307 | ```Rust
308 | #[diagnostic(
309 | code(oops::my::bad),
310 | url(docsrs),
311 | help("try doing it better next time?")
312 | )]
313 | ```
314 |
315 | ## Using diagnostic codes
316 |
317 | Unique diagnostic codes can be find in many compilers including [Rust](https://doc.rust-lang.org/error_codes/error-index.html), [Clang](https://clang.llvm.org/docs/InternalsManual.html#the-diagnostics-subsystem), and [C#](https://github.com/thomaslevesque/GenerateCSharpErrors/blob/master/CSharpErrorsAndWarnings.md). They can be used to:
318 |
319 | - point to detailed diagnostic explanation
320 | - suppress warnings
321 |
322 | Rust takes advantage of both possibilities by offering [online help for error messages](https://doc.rust-lang.org/error_codes/error-index.html) and a possibility to suppress warnings by using [#[allow]](https://docs.rs/allow/latest/allow/).
323 |
324 | # Unresolved questions
325 |
326 | [unresolved-questions]: #unresolved-questions
327 |
328 | The RFC has no unresolved questions. During the discussion, the following questions were resolved:
329 |
330 | - the structure of the Diagnostic API.
331 | - the usage of symbolic warning identifiers like e.g., `name_is_not_idiomatic` in addition to warning codes like e.g., `W0001`.
332 |
333 | # Future possibilities
334 |
335 | [future-possibilities]: #future-possibilities
336 |
337 | Once we get online documentation for diagnostics (as proposed in [Sway 3512](https://github.com/FuelLabs/sway/issues/3512)) we can extend diagnostic messages with links to detailed explanation by adding an element like:
338 |
339 | ```
340 | info: For more information see: https://.../E4001
341 | ```
342 |
343 | Similar to Rust, we can also add the _explain_ command to `forc`. E.g.:
344 |
345 | ```
346 | forc explain E4001
347 | ```
348 |
349 | Once _The Sway Book_ and _The Sway Reference_ become stable we can also enhance `Diagnostic` by adding references to the documentation. E.g.:
350 |
351 | ```Rust
352 | #[error(
353 | reason(1, "Constants cannot be shadowed"),
354 | ...
355 | book("Shadowing", "relative/path/to/chapter/shadowing"),
356 | ref("Constants", "relative/path/to/chapter/constants"),
357 | ...
358 | )]
359 | ```
360 |
361 | This would render additional _info_ lines. E.g.:
362 |
363 | ```
364 | info: For more information, see:
365 | - the chapter "Shadowing" in The Sway Book: https://.../shadowing
366 | - the chapter "Constants" in The Sway Reference: https://.../constants
367 | ```
368 |
369 | # References
370 |
371 | [references]: #references
372 |
373 | - \[1\] [Expressive Diagnostics](https://clang.llvm.org/diagnostics.html)
374 | - \[2\] [Shape of errors to come](https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-come.html)
375 | - \[3\] [Compiler Errors for Humans](https://elm-lang.org/news/compiler-errors-for-humans)
376 | - \[4\] [Compilers as Assistants](https://elm-lang.org/news/compilers-as-assistants)
377 | - \[5\] [Concepts Error Messages for Humans](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2429r0.pdf)
378 | - \[6\] [Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research](https://www.brettbecker.com/wp-content/uploads/2019/12/becker2019compiler.pdf)
379 | - \[7\] [Usability improvements in GCC 8](https://developers.redhat.com/blog/2018/03/15/gcc-8-usability-improvements#)
380 |
381 | [1]: https://clang.llvm.org/diagnostics.html "Expressive Diagnostics"
382 | [2]: https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-come.html "Shape of errors to come"
383 | [3]: https://elm-lang.org/news/compiler-errors-for-humans "Compiler Errors for Humans"
384 | [4]: https://elm-lang.org/news/compilers-as-assistants "Compilers as Assistants"
385 | [5]: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2429r0.pdf "Concepts Error Messages for Humans"
386 | [6]: https://www.brettbecker.com/wp-content/uploads/2019/12/becker2019compiler.pdf "Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research"
387 | [7]: https://developers.redhat.com/blog/2018/03/15/gcc-8-usability-improvements "Usability improvements in GCC 8"
388 |
--------------------------------------------------------------------------------
/rfcs/0013-changes-lifecycle.md:
--------------------------------------------------------------------------------
1 | - Feature Name: changes-lifecycle
2 | - Start Date: 2024-08-05
3 | - RFC PR: [FuelLabs/sway-rfcs#41](https://github.com/FuelLabs/sway-rfcs/pull/41)
4 |
5 | # Summary
6 |
7 | [summary]: #summary
8 |
9 | This RFC is a guide on how to introduce changes following the compiler release lifecycle.
10 |
11 | # Motivation
12 |
13 | [motivation]: #motivation
14 |
15 | To maximize user experience when updating the compiler, we need a guide on how to introduce changes.
16 |
17 | # Guide-level explanation
18 |
19 | [guide-level-explanation]: #guide-level-explanation
20 |
21 | The compiler will follow a "rolling release" scheme, which means that periodically (to be specified) a
22 | new version will be released following "semver versioning".
23 |
24 | This means that as soon as the compiler reaches version "1.0.0", the next **version** will be "1.1.0"; and
25 | after that and if needed, the next **patch** would be "1.1.1".
26 |
27 | The only changes that will trigger "patches" are:
28 |
29 | 1. Urgent security fixes;
30 | 2. Fixing bugs that render "stable" functionality unusable;
31 |
32 | Major version bumps, from "1.X.X" to "2.0.0", will be reserved to major breaking changes. All other changes, those specified in this document, will follow this guide and bump only the minor version as "1.1.X" to "1.2.0".
33 |
34 | ## Release Notes
35 |
36 | To track all release notes, a special file named `ReleaseNotes.md` will be created in the repository's root directory.
37 |
38 | When a version is being launched, for example, `1.10` a new section `1.11-nightly` will be created empty. From this time on, all new PRs will create a new item under the "next version", `1.11-nightly` in this example.
39 |
40 | Each item will follow the template:
41 |
42 | ```
43 | # Version 1.11-nightly
44 |
45 | ## Forc
46 |
47 | ### Added
48 | ### Changed
49 | ### Deprecated
50 | ### Removed
51 | ### Fixed
52 | ### Security
53 |
54 | ## Tools
55 |
56 | ...
57 |
58 | ## Sway
59 |
60 | ...
61 |
62 | ## Standard Libraries
63 |
64 | ...
65 |
66 | # Version 1.10
67 |
68 | ...
69 | ```
70 |
71 | Inside each section, items will follow the template of a user friendly one line description and a link to the PR.
72 |
73 | ```
74 | - Index operator using Index trait [#6356](https://github.com/FuelLabs/sway/pull/6356)
75 | ```
76 |
77 | Given that all PRs will touch this file, to avoid conflicts and decrease the experience when merging them, we will use "git custom merge driver" ([https://git-scm.com/docs/gitattributes#\_defining_a_custom_merge_driver](https://git-scm.com/docs/gitattributes#_defining_a_custom_merge_driver)).
78 |
79 | This allows a custom merge strategy to a specific file using ".gitattributes".
80 |
81 | ```
82 | ReleaseNotes.md merge=union
83 | ```
84 |
85 | ## Breaking changes
86 |
87 | A breaking change is a difference in functionality from the previous version of the compiler that may require an update to Sway code in order for it to compile.
88 |
89 | The following changes are defined to be breaking changes and will need to follow the process of being gated by feature flags.
90 |
91 | 1. Code without deprecated warning to be flagged as an error;
92 | 2. ABI JSON properties to be removed or have their type changed;
93 | 3. Binary encoding that will break how SDK (and others) communicate with the compiler, including:
94 | - How scripts/predicates accept arguments on `main`;
95 | - How scripts/predicates return data from `main`;
96 | - How the `contract method selector` is encoded;
97 | - How `contract method` arguments are encoded;
98 | - How `log` data is encoded;
99 | - How `message` data is encoded;
100 | 4. IR changes;
101 | 5. Receipt parsers to break;
102 | 6. When a compiler feature or a standard library produces different behavior for the same code (semantic changes);
103 | 7. When `storage` is impacted. Particularly addresses.
104 |
105 | ## Feature flags
106 |
107 | Any complex change that needs to be gated will follow these steps.
108 |
109 | 1. A specific GitHub issue labeled with `feature-*` and `track-feature`. This issue should be the umbrella for everything related to this feature;
110 | 2. A preliminary PR enabling the feature flag should be created and linked to the umbrella issue; This PR will enable the feature flag `--experimental ` on all tools.
111 | 3. As many PRs will be created and merged as normal;
112 | 4. A chapter inside `Sway Unstable Book` will be created and updated as needed;
113 | 5. When the feature is ready, a closing PR will be created and wait until the feature flag is enabled by default.
114 | 6. On a later date, the feature flag can be removed making the feature the default behavior of the compiler and the docs should be migrated to the stable portion of the book.
115 |
116 | Once the feature is merged into `master`, it will not be possible to "turn off" the feature. In the same sense
117 | that it is not possible to "turn off" a "match expression" or any other language feature.
118 |
119 | If the feature contains some configuration or choice, the "default" value will be the default, and other options
120 | will be available, but it cannot be turned off.
121 |
122 | ## Conditional compilation
123 |
124 | There are cases where conditional compilation will be needed. For these cases, each experimental feature will also have a corresponding `#[cfg(...)]`, like the example below:
125 |
126 | ```sway
127 | #[cfg(experimental_new_encoding = false)]
128 | const CONTRACT_ID = 0x14ed3cd06c2947248f69d54bfa681fe40d26267be84df7e19e253622b7921bbe;
129 | #[cfg(experimental_new_encoding = true)]
130 | const CONTRACT_ID = 0x316c03d37b53eaeffe22c2d2df50d675e2b2ee07bd8b73f852e686129aeba462;
131 | ```
132 |
133 | # Enabling features on Sway
134 |
135 | Features can be enabled inside the `Forc.toml` file:
136 |
137 | ```toml
138 | experimental = ["...", "..."]
139 | ```
140 |
141 | or using the CLI
142 |
143 | ```
144 | > forc ... --experimental some_feature,another_feature
145 | > forc ... --no-experimental some_feature,another_feature
146 | ```
147 |
148 | or even using environment variables
149 |
150 | ```
151 | > FORC_EXPERIMENTAL=some_feature,another_feature forc ...
152 | > FORC_NO_EXPERIMENTAL=some_feature,another_feature forc ...
153 | ```
154 |
155 | The order matters so for example if `feature_a` is turned on on `test.toml`, it can be turned off by the CLI or by environment variables.
156 |
157 | If a feature is not turned on by `forc.toml`, it can still be turned on by the CLI and environment variables.
158 |
159 | A special token `*` will mean "all features" in the sense that all features can be turned on or turned off.
160 |
161 | ```
162 | > forc ... --experimental *
163 | ```
164 |
165 | This is specially useful to control which features are enabled
166 |
167 | ```
168 | > forc .. --no-experimental * --experimental some_feature
169 | ```
170 |
171 | These flags also need to be enabled programmatically by any compiler driver, like tests.
172 |
173 | Unlike `Rust` we will not support features inside Sway code like the example below, because some features will span across multiple tools. That would demand `forc` to parse, or ask the Sway compiler if a feature is enabled or not.
174 |
175 | ```sway
176 | #![enable(some_experimental_feature)]
177 | ```
178 |
179 | ## Unstable book and forc-doc
180 |
181 | The idea of an unstable book is to be a repository of documentation, decisions, or even a devlog of the feature.
182 | Its unstructured nature is intentional and serves the purpose of unburdening developers to keep an update and formal
183 | documentation of a feature that will likely change.
184 |
185 | Ideally, each chapter will have a link to the GitHub issue, discussions, references and whatever else is necessary
186 | to allow stakeholders to give feedback on the new feature.
187 |
188 | `forc-doc` will also understand about experimental types and functions/methods. They will have special decoration in the documentation, so users know if and which features need to be enabled.
189 |
190 | # Reference-level explanation
191 |
192 | [reference-level-explanation]: #reference-level-explanation
193 |
194 | This section contains the suggested guide on how to introduce changes to different parts of the compiler and associated tools.
195 |
196 | ## `sway-features` crate
197 |
198 | A new crate named `sway-features` will be created and will contain **ALL** features and their metadata. The best suggestion is a macro to define features where enums, documentation, etc., will be generated.
199 |
200 | This macro also needs to generate code for the errors and warnings related to each error.
201 |
202 | Features will be parsed in this order:
203 |
204 | ```rust
205 | let mut features = ExperimentalFeatures::default();
206 | features.parse_from_toml();
207 | features.parse_from_cli();
208 | features.parse_from_environment_variables();
209 | ```
210 |
211 | This means that environment variables overwrite CLI arguments, which overwrite TOML configuration.
212 |
213 | If a feature requires or allows new configurations, these configurations should be optional, and as soon as confirmed the feature is enabled, it should verify if the required configuration is available.
214 |
215 | ```
216 | > forc ... --experimental a_feature --a-feature-option 1
217 | ```
218 |
219 | ## Lexer and Parser
220 |
221 | To allow as user-friendly error messages as possible, in all possible cases, we want both the lexer and the parser to parse the new syntax even with the feature off.
222 |
223 | After that, the lexer and the parser will mark that an experimental feature was lexed/parsed; and a check will guarantee that no disabled experimental was parsed.
224 |
225 | The error message will have a message explaining that the feature is experimental and a GitHub link for more details on the stabilization lifecycle.
226 |
227 | ```rust
228 | // always parse new syntax
229 | let new_syntax = parse_new_syntax();
230 | if ctx.experimental.is_disabled(Features::NEW_FEATURE) {
231 | handler.emit_error(...);
232 | }
233 | ```
234 |
235 | ## Formatting
236 |
237 | Formatting should always format new syntax, even when the flag is off. To avoid breaking other parts of the code, or even worse, removing code.
238 |
239 | ## LSP
240 |
241 | LSP can take advantage of specific error messages, and suggest users to enable the corresponding feature.
242 |
243 | ## CST, AST, and Typed Tree
244 |
245 | All trees must always support new features, which means that new nodes will always exist. Their specific behavior, desugaring, etc., will be gated by the experimental feature.
246 |
247 | More specifically, this means that variants should not be behind compiler flags.
248 |
249 | ## Tests
250 |
251 | To allow these experimental features to be tested, it is necessary to specify which configurations each test must run.
252 | Today, by default, a test runs with the "default compiler configuration", but it is necessary to allow tests to
253 | run with any combination of compiler configuration.
254 |
255 | To allow complete tests of any compiler configuration, any configuration that today resides inside `test.toml`
256 | needs to be configured for each configuration.
257 |
258 | An example is the `encoding v1` which has different inputs and outputs from `encoding v0` when testing a script
259 | that takes arguments.
260 |
261 | To support this tests, when needed, can have multiple `test.toml` with suffixes to differentiate them.
262 |
263 | ```
264 | test.toml
265 | test.feature_a.toml
266 | ```
267 |
268 | To avoid duplications `test.feature_a.toml` will inherit properties from `test.toml`.
269 | And new properties will be created to allow the configuration of the compiler:
270 |
271 | ```
272 | [environment_variables]
273 | SOME_VAR = "1"
274 |
275 | [forc]
276 | cli = "--experimental feature_a,feature_b"
277 | ```
278 |
279 | The same strategy will be used for snapshot tests: multiple `snapshot.toml`. So, for example, `snapshot.feature_a.toml` will generate `snapshot@feature_a.snap`.
280 |
281 | # Drawbacks
282 |
283 | [drawbacks]: #drawbacks
284 |
285 | This will increase the complexity of the compiler. Not all flags used to compile end up in the JSON ABI, or other outputs. Which can make reproducibility harder.
286 |
287 | # Rationale and alternatives
288 |
289 | [rationale-and-alternatives]: #rationale-and-alternatives
290 |
291 | There are only two other alternatives, which we consider the worst options:
292 |
293 | 1. Keep experimental features in branches;
294 | 2. Keep experimental features behind compiler flags (conditional compilation);
295 |
296 | The first one normally creates more problems than it solves. "Integration hell" becomes a reality sooner than later.
297 | The second does not decrease the complexity of the code, and it decreases the testability of features, given that they will not be in the "release binary".
298 |
299 | # Prior art
300 |
301 | [prior-art]: #prior-art
302 |
303 | These are the sources and stacks used as motivation for this RFC.
304 |
305 | ## How `rustc` does it
306 |
307 | https://rustc-dev-guide.rust-lang.org/implementing_new_features.html#stability-in-code
308 |
309 | 1. Open a tracking issue;
310 | 2. Pick a name for the feature gate;
311 | 3. Add the feature gate to the compiler code;
312 | 4. If the feature gate is not set, you should either maintain the pre-feature behavior or raise an error, depending on what makes sense;
313 | 5. Add a test to ensure the feature cannot be used without a feature gate;
314 | 6. Add a section to the unstable book;
315 | 7. Write a lot of tests for the new feature;
316 | 8. Get your PR reviewed and land it.
317 | 9. Add a section to the unstable book;
318 |
319 | ## changeset
320 |
321 | https://github.com/changesets/changesets
322 |
323 | ## keepachangelog
324 |
325 | Changes will categorized following https://keepachangelog.com/en/1.1.0/
326 |
327 | # Unresolved questions
328 |
329 | [unresolved-questions]: #unresolved-questions
330 |
331 | ## Where should we save "release notes"?
332 |
333 | 1. In files inside the repo? - Saving in files sounds like the best approach, but if we use the same file, all PRs will conflict. To avoid this we can do like `changeset` and use random names.
334 | 2. In GitHub PR descriptions? - This is the easiest approach, but it can be cumbersome to recover these messages.
335 | 3. In git commit messages? - Given that the commit message is "created" when the PR is merged, there is no way to guarantee that "release notes" will exist or be "parseable" when we need them.
336 | - https://git-cliff.org/
337 |
338 | ## Should new warnings be considered breaking changes?
339 |
340 | 1. Normally warnings should not be considered breaking changes, because they do not break anything.
341 | 2. On the other hand, if the team treats warnings as errors, new warnings will break CIs, and demand
342 | developers' attention. Not all fixes are trivial and can demand bigger code changes than the user would expect
343 | from a minor update from the compiler.
344 |
345 | ## How to deal with "contract id"?
346 |
347 | Should we consider contract ID changes to be breaking changes?
348 |
349 | # Future possibilities
350 |
351 | [future-possibilities]: #future-possibilities
352 |
--------------------------------------------------------------------------------
/rfcs/0014-abi-errors.md:
--------------------------------------------------------------------------------
1 | - Feature Name: `error_type`
2 | - Start Date: 2024-11-18
3 | - RFC PR: [FuelLabs/sway-rfcs#43](https://github.com/FuelLabs/sway-rfcs/pull/43)
4 | - Sway Issue: [FueLabs/sway#6765](https://github.com/FuelLabs/sway/issues/6765)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | This RFC aims to standardize error handling in Sway so that there is a
11 | deliberate and clear way to produce error messages that are easily consumable by
12 | language tools while keeping error strings out of on-chain artifacts.
13 |
14 | # Motivation
15 |
16 | [motivation]: #motivation
17 |
18 | Currently, reverting a Sway program only produces an opaque error code.
19 | Sometimes this code is unusual enough that with the source code of the contract
20 | it is possible to figure out what is going on, but a lot of the time it's a
21 | meaningless number that only means that some failure happened somewhere.
22 |
23 | It's not reasonable to expect people to be able to debug Sway programs in these
24 | conditions, so we need to improve this.
25 |
26 | We want irrecoverable errors to carry user defined error messages and source
27 | information about where the error was invoked. We want those error types to be
28 | something that can also be used for recoverable errors.
29 |
30 | To this end we want to introduce a mechanism that makes error codes an
31 | understandable part of the ABI, and language features that allow the user to
32 | define error messages, error types and invoke those errors with the appropriate
33 | information being generated.
34 |
35 | We also want recoverable errors to connect to this mechanism so that all error
36 | handling in a given Sway program can go through one or a small set of user
37 | defined types and error strings don't affect the final bytecode size.
38 |
39 | # Guide-level explanation
40 |
41 | [guide-level-explanation]: #guide-level-explanation
42 |
43 | ## Error types
44 |
45 | Error types are special enums that allow developers to signal that each variant
46 | is associated with an error message.
47 |
48 | An error type implements the `Error` trait, but it is not recommended that you
49 | do so manually.
50 |
51 | Instead, to create a new error type, use the following syntax and annotations on
52 | an enum:
53 |
54 | ```sway
55 | #[error_type]
56 | enum MyError {
57 | #[error("error A")]
58 | A: (),
59 | #[error("error B")]
60 | B: (u64, u8),
61 | }
62 | ```
63 |
64 | This will automatically implement the `Error` marker trait for this enum
65 | and let the compiler know what error messages correspond to what variants of
66 | the enum.
67 |
68 | It will also populate the ABI specification file with error messages in the
69 | metadata of the enum type. Allowing users to check the meaning of an error
70 | without the need to store and decode error messages on chain.
71 |
72 | For instance, the following contract method may return a `MyError` variant which
73 | will be compact but can still be associated with an error message:
74 |
75 | ```sway
76 | fn do_something(self) -> Result<(), MyError> {
77 | // ...
78 | return Err(MyError::A); // user will know that this means "error A"
79 | // ...
80 | }
81 | ```
82 |
83 | ## `panic`
84 |
85 | When encountering an irrecoverable error in a Sway program, it is customary to
86 | revert and produce an informative error code.
87 |
88 | The recommended way of doing this is to use a `panic` expression.
89 |
90 | `panic` a syntax element similar to `return` except it aborts the execution of the entire program.
91 |
92 | Moreover, it will produce a unique revert code for each of its invocations and
93 | populate the ABI specification file with a list of such codes and corresponding
94 | information about where it was invoked and with what kind of arguments.
95 |
96 | If all you want is an error message, you can panic with a string literal:
97 |
98 | ```sway
99 | // ...
100 | // some error happened
101 | panic "some error happened";
102 | ```
103 |
104 | But if you want more functionality, you should use an error type:
105 |
106 | ```sway
107 | panic MyError::A;
108 | // ...
109 | panic MyError::B((1024, 42));
110 | ```
111 |
112 | Panicking with an error type will do two things: log the error so that a
113 | corresponding log receipt is produced, and revert with an error code that
114 | corresponds to the invocation location.
115 |
116 | ## Revert codes
117 |
118 | All revert codes after `0xffff_ffff_ffff_0000` are reserved for use by the
119 | compiler and the standard library.
120 |
121 | Codes produced by `panic` start at `0xffff_ffff_0000_0000` but may be allocated
122 | at will by the compiler, they are not guaranteed to be sequential.
123 |
124 | Let's go through how to decode a set of error codes for the following example
125 | program:
126 |
127 | ```sway
128 | script;
129 |
130 | #[error_type]
131 | enum MyError {
132 | #[error("error A")]
133 | A: (),
134 | #[error("error B")]
135 | B: (u64, u8),
136 | }
137 |
138 | pub fn main(a: u8) {
139 | match a {
140 | 0 => panic "a str panic",
141 | 1 => panic "another str panic",
142 | 2 => panic MyError::A,
143 | 3 => panic MyError::A,
144 | 4 => panic MyError::B((1024, 42)),
145 | 5 => panic MyError::B((0, 16)),
146 | _ => {}
147 | }
148 | }
149 | ```
150 |
151 | this would produce, when compiled, the following ABI specification file:
152 |
153 | ```json
154 |
155 | {
156 | "programType": "script",
157 | "specVersion": "1",
158 | "encodingVersion": "1",
159 | "concreteTypes": [
160 | {
161 | "type": "()",
162 | "concreteTypeId": "2e38e77b22c314a449e91fafed92a43826ac6aa403ae6a8acb6cf58239fbaf5d"
163 | },
164 | {
165 | "type": "enum MyError",
166 | "concreteTypeId": "44781f4b1eb667f225275b0a1c877dd4b9a8ab01f3cd01f8ed84f95c6cd2f363",
167 | "metadataTypeId": 1
168 | },
169 | {
170 | "type": "u8",
171 | "concreteTypeId": "c89951a24c6ca28c13fd1cfdc646b2b656d69e61a92b91023be7eb58eb914b6b"
172 | }
173 | ],
174 | "metadataTypes": [
175 | {
176 | "type": "(_, _)",
177 | "metadataTypeId": 0,
178 | "components": [
179 | {
180 | "name": "__tuple_element",
181 | "typeId": 2
182 | },
183 | {
184 | "name": "__tuple_element",
185 | "typeId": "c89951a24c6ca28c13fd1cfdc646b2b656d69e61a92b91023be7eb58eb914b6b"
186 | }
187 | ]
188 | },
189 | {
190 | "type": "enum MyError",
191 | "metadataTypeId": 1,
192 | "components": [
193 | {
194 | "name": "A",
195 | "errorMessage": "error A",
196 | "typeId": "2e38e77b22c314a449e91fafed92a43826ac6aa403ae6a8acb6cf58239fbaf5d"
197 | },
198 | {
199 | "name": "B",
200 | "errorMessage": "error B",
201 | "typeId": 0
202 | }
203 | ]
204 | },
205 | {
206 | "type": "u64",
207 | "metadataTypeId": 2
208 | }
209 | ],
210 | "functions": [
211 | {
212 | "inputs": [
213 | {
214 | "name": "a",
215 | "concreteTypeId": "c89951a24c6ca28c13fd1cfdc646b2b656d69e61a92b91023be7eb58eb914b6b"
216 | }
217 | ],
218 | "name": "main",
219 | "output": "2e38e77b22c314a449e91fafed92a43826ac6aa403ae6a8acb6cf58239fbaf5d",
220 | "attributes": null
221 | }
222 | ],
223 | "loggedTypes": [
224 | {
225 | "logId": "4933727799282657266",
226 | "concreteTypeId": "44781f4b1eb667f225275b0a1c877dd4b9a8ab01f3cd01f8ed84f95c6cd2f363"
227 | }
228 | ],
229 | "messagesTypes": [],
230 | "configurables": [],
231 | "errorCodes": {
232 | "18446744069414584320": {
233 | "pos": {
234 | "file": "main.sw",
235 | "line": 13,
236 | "col": 10
237 | },
238 | "logId": null,
239 | "msg": "a str panic"
240 | },
241 | "18446744069414584321": {
242 | "pos": {
243 | "file": "main.sw",
244 | "line": 14,
245 | "col": 10
246 | },
247 | "logId": null,
248 | "msg": "another str panic"
249 | },
250 | "18446744069414584322": {
251 | "pos": {
252 | "file": "main.sw",
253 | "line": 15,
254 | "col": 10
255 | },
256 | "logId": "4933727799282657266",
257 | "msg": null,
258 | },
259 | "18446744069414584323": {
260 | "pos": {
261 | "file": "main.sw",
262 | "line": 16,
263 | "col": 10
264 | },
265 | "logId": "4933727799282657266",
266 | "msg": null,
267 | }
268 | "18446744069414584324": {
269 | "pos": {
270 | "file": "main.sw",
271 | "line": 17,
272 | "col": 10
273 | },
274 | "logId": "4933727799282657266",
275 | "msg": null,
276 | },
277 | "18446744069414584325": {
278 | "pos": {
279 | "file": "main.sw",
280 | "line": 18,
281 | "col": 10
282 | },
283 | "logId": "4933727799282657266",
284 | "msg": null,
285 | }
286 | }
287 | }
288 | ```
289 |
290 | If you run the script with argument `0`, you will get a revert code
291 | `18446744069414584320` and no log receipts. Looking in the `"errorCodes"`
292 | section of the ABI spec, you can see that this code corresponds to no log and a
293 | message `"a str panic"`.
294 |
295 | If you run the script with argument `4`, you will get a revert code
296 | `18446744069414584324` and a log receipt. Looking in the `"errorCodes"`
297 | section of the ABI spec, you can see that this code corresponds
298 | to no immediate message and a `"logId": "4933727799282657266"`. In
299 | `"loggedTypes"` you can see that this id corresponds to a `"concreteTypeId":
300 | "44781f4b1eb667f225275b0a1c877dd4b9a8ab01f3cd01f8ed84f95c6cd2f363"`, which in
301 | turn corresponds to type `"enum MyError"` with a `"metadataTypeId": 1`, which in
302 | turn contains variants as components with an `errorMessage` field. When decoding
303 | the corresponding log, you'll get a `MyError::B((1024, 42))` value which
304 | corresponds to error message `"error B"`.
305 |
306 | # Reference-level explanation
307 |
308 | [reference-level-explanation]: #reference-level-explanation
309 |
310 | ## Error types
311 |
312 | Error types are recognized using the `Error` trait, but it is a simple marker
313 | that does not carry behavior (for now).
314 |
315 | ```sway
316 | trait Error: AbiEncode {}
317 | ```
318 |
319 | We need to introduce two new annotations.
320 |
321 | `#[error_type]` generates an implementation of `Error` for a given type
322 | and should let the compiler know that it should carry around error message
323 | information about this type.
324 |
325 | `#[error("")]` should only be usable in a `#[error_type]` marked enum
326 | definition, and to not exhaustively mark variants of such an enum with an error
327 | message annotation is considered an error. Using more than one `#[error("")]`
328 | annotation on a variant is considered an error.
329 |
330 | When generating ABI files, the compiler should populate the `"metadataTypes"`
331 | section accordingly and include a `"errorMessage"` field in the components of
332 | marked enums.
333 |
334 | ## `panic` expression
335 |
336 | The `panic` expression has two modes which determine what kind of fields are
337 | produced in the (new) `"errorCodes"` section of the ABI file.
338 |
339 | Either it is used with a `str` literal (or a const-evaluable such value) and
340 | produces `"msg"` fields. Or it is used with an `Error` implementing type, and
341 | produces `"logId"` fields. Any other use is an error.
342 |
343 | In any valid case, it also records in the `"errorCodes"` section location
344 | information about every `panic` invocation's position in the codebase.
345 |
346 | As for runtime behavior, if the argument is an error type, it logs it before
347 | reverting, and in every valid case, it eventually reverts with a unique error
348 | code for every `panic` invocation, which is recorded in the `"errorCodes"`
349 | section.
350 |
351 | `panic` can be used in a expression and evalutates to `!`.
352 |
353 | ## Integration considerations
354 |
355 | Manual reverts with custom codes should still be available to developers, but
356 | discouraged.
357 |
358 | The standard library should use error types for both
359 | recoverable and irrecoverable errors. The standard library should
360 | never produce a revert code that doesn't have a corresponding documented error
361 | message.
362 |
363 | The existing special error codes produced by the standard library (such as
364 | `FAILED_ASSERT_SIGNAL`) should be migrated to use this mechanism.
365 |
366 | We should also introduce compiler warnings for manually producing revert codes
367 | that may conflict with either the reserved code range or the auto-generated panic
368 | code range.
369 |
370 | SDK integration of this feature is open ended, but we should at least aim to
371 | be able to use error message information to decode revert codes and error types
372 | that are directly returned.
373 |
374 |
375 |
376 | # Drawbacks
377 |
378 | [drawbacks]: #drawbacks
379 |
380 | Introducing a new syntax element is the main drawback of this solution.
381 |
382 | It pollutes the namespace with another reserved word.
383 |
384 | It also moves more complexity to the compiler rather than let the user define it.
385 | This is a shared problem with the error type annotations.
386 |
387 | # Rationale and alternatives
388 |
389 | [rationale-and-alternatives]: #rationale-and-alternatives
390 |
391 | There are drawbacks to moving more special behavior the compiler. However, we
392 | do not have sufficiently powerful meta-programming features that would allow
393 | developers to define features of this caliber. Therefore, the language must
394 | shoulder the burden of this complexity.
395 |
396 | Given the benefits of having standardized errors, these trade-offs seem
397 | acceptable, so long as we allow the solution to be extended in the future to
398 | be more user defined. The `Error` trait is the main vector of such future
399 | changes in this case.
400 |
401 | Letting the errors be populated directly in the bytecode would be a lot simpler,
402 | but it is not desirable because it extends the size of the binary for no gain.
403 |
404 | Moreover, the question of the location of error strings can be asked. Why
405 | put them in the ABI specification file? ABI files are soon to benefit from
406 | a registry that will allow users to easily fetch them, they already contain
407 | type metadata that can be extended, and they ultimately are our equivalent to
408 | headers, encompassing all the required information to call a contract and get
409 | a meaningful answer. Errors are part of making this answer meaningful and it is
410 | typical for error messages to live in headers.
411 |
412 | If we do not specify some kind of error standard, debugging any sort of complex
413 | Sway error will remain extremely difficult and supporting Sway contracts would
414 | be an almost impossible task.
415 |
416 | # Prior art
417 |
418 | [prior-art]: #prior-art
419 |
420 | This RFC is somewhat inspired by Rust's error library ecosystem. We do not
421 | have enough metaprogramming facilities to allow users to define their own way
422 | of producing errors, but one can identify two kinds of Rust error handling
423 | libraries. Those that like `anyhow` care most about letting the user add some
424 | string based error context. And those like `thiserror` that allow one to build
425 | complex error types.
426 |
427 | We attempt here to cater to both crowds through `panic`'s dual modes.
428 |
429 | Moreover, Ethereum style embedding of errors in a `b256` isn't practical for us
430 | or generally desirable since the FuelVM uses 64 bit error codes and we aim to
431 | minimize the footprint of errors on deployed contracts.
432 |
433 | # Unresolved questions
434 |
435 | [unresolved-questions]: #unresolved-questions
436 |
437 | The extent of the SDK integration of this feature is quite open ended. But
438 | implementation of this should give a good enough base to decide how much support
439 | we want and to extend it in the future.
440 |
441 | # Future possibilities
442 |
443 | [future-possibilities]: #future-possibilities
444 |
445 | In the future we may want to include more metadata for panic invocations or
446 | error types.
447 |
448 | We may also allow error strings to be format strings so that you may use your
449 | enum's values in the error message.
450 |
451 | Improvements to our const evaluation facilities may also prompt a rework of the
452 | `Error` trait so that the error messages are generated arbitrarily through
453 | a `const fn` instead of statically defined through annotations. This would also
454 | allow us to do away with custom annotations.
455 |
456 | We should also consider better syntax and handling for recoverable errors
457 | implementing `Error`, such as a `?` operator similar to Rust's.
458 |
459 |
--------------------------------------------------------------------------------
/rfcs/0015-const-generics.md:
--------------------------------------------------------------------------------
1 | - Feature Name: const_generics
2 | - Start Date: 2024-10-27
3 | - RFC PR: [FuelLabs/sway-rfcs#42](https://github.com/FuelLabs/sway-rfcs/pull/42)
4 | - Sway Issue: [FueLabs/sway#0000](https://github.com/FuelLabs/sway/issues/001)
5 |
6 | # Summary
7 |
8 | [summary]: #summary
9 |
10 | Allows constant values as generic arguments.
11 |
12 | # Motivation
13 |
14 | [motivation]: #motivation
15 |
16 | Some types have constants, specifically unsigned integers as part of their definition (e.g. arrays and string arrays). Without `const generics` it is impossible to have a single `impl` item for all instances of these types.
17 |
18 | # Guide-level explanation
19 |
20 | [guide-level-explanation]: #guide-level-explanation
21 |
22 | `const generics` refer to the language syntax where generic parameters are defined as constant values, instead of types.
23 |
24 | A simple example would be:
25 |
26 | ```rust
27 | fn id(array: [u64; SIZE]) -> [u64; SIZE] {
28 | array
29 | }
30 | ```
31 |
32 | This also allows `impl` items such as
33 |
34 | ```rust
35 | impl AbiEncode for str[N] {
36 | ...
37 | }
38 | ```
39 |
40 | This constant can be inferred or explicitly specified. When inferred, the syntax is no different than just using the item:
41 |
42 | ```rust
43 | id([1u8])
44 | ```
45 |
46 | In the example above, the Sway compiler will infer `SIZE` to be one, because `id` parameter will be infered to be `[u8; 1]`.
47 |
48 | For the cases where the compiler cannot infer this value, or this value comes from an expression, it is possible to do:
49 |
50 | ```rust
51 | id::<1>([1u8]);
52 | id::<{1 + 1}>([1u8, 2u8]);
53 | ```
54 |
55 | When the value is not a literal, but an expression, it is named "const generic expression" and it needs to be enclosed by curly braces. This will fail if the expression cannot be evaluated as `const`.
56 |
57 | # Reference-level explanation
58 |
59 | [reference-level-explanation]: #reference-level-explanation
60 |
61 | This new syntax has three forms: declarations, instantiations, and references.
62 |
63 | "Const generics declarations" can appear anywhere all other generic arguments declarations are valid:
64 |
65 | 1. Function/method declaration;
66 | 1. Struct/Enum declaration;
67 | 1. `impl` declarations.
68 |
69 | ```rust
70 | // 1
71 | fn id(...) { ... }
72 |
73 | // 2
74 | struct SpecialArray {
75 | inner: [u8; N],
76 | }
77 |
78 | //3
79 | impl AbiEncode for str[N] {
80 | ...
81 | }
82 | ```
83 |
84 | "Const generics instantiations" can appear anywhere all other generic argument instantiations are valid:
85 |
86 | 1. Function/method reference;
87 | 1. Struct/Enum reference;
88 | 1. Fully qualified call paths.
89 |
90 | ```rust
91 | // 1
92 | id::<1>([1u8]);
93 | special_array.do_something::<1>();
94 |
95 | // 2
96 | SpecialArray::<1>::new();
97 |
98 | // 3
99 | as SpecialArrayTrait::<1>>::f();
100 | ```
101 |
102 | "Const generics references" can appear anywhere any other identifier appears. For semantic purposes, there is no difference between the reference of a "const generics" and a "normal" const.
103 |
104 | ```rust
105 | fn f() {
106 | __log(I);
107 | }
108 | ```
109 |
110 | Different from type generics, const generics cannot appear at:
111 |
112 | 1. constraints;
113 | 1. where types are expected.
114 |
115 | ```rust
116 | // 1 - INVALID
117 | fn f() where I > 10 {
118 | ...
119 | }
120 |
121 | // 2 - INVALID
122 | fn f() -> Vec {
123 | Vec::::new()
124 | }
125 | ```
126 |
127 | ## Const Value Specialization
128 |
129 | By the nature of `impl` declarations, it is possible to specialize `impl` items for some specific types. For example:
130 |
131 | ```rust
132 | impl SomeStruct {
133 | fn f() {
134 | }
135 | }
136 | ```
137 |
138 | In the example above, `f` is only available when the generic argument of `SomeStruct` is known to be `bool`. This kind of specialization will not be supported for "const generics", which means that the example below will not be supported:
139 |
140 | ```rust
141 | impl SomeIntegerStruct<1> {
142 | ...
143 | }
144 |
145 | impl SomeBoolStruct {
146 | ...
147 | }
148 | ```
149 |
150 | The main reason for forbidding this is that apart from `bool`, which only needs two values, all other constants would demand complex syntax to guarantee the completeness and uniqueness of all implementations, the same way that `match` expressions do.
151 |
152 | ## Monomorphization
153 |
154 | As with other generic arguments, `const generics` monormorphize functions, which means that a new "TyFunctionDecl", for example, will be created for which value that is instantiated.
155 |
156 | Prevention of code bloat will be the responsibility of the optimizer.
157 |
158 | Monomorphization for const generics has one extra complexity. To support arbitrary expressions it is needed to "solve" an equation. For example, if a variable is typed as `[u64; 1]` and a method with its `Self` type as `[u64; N + 1]` is called, the monomorphization process needs to know that `N` needs to be valued as `0` and if the variable is `[u64; 2]`, `N` will be `1`.
159 |
160 | ## Type Engine changes
161 |
162 | By the nature of `const generics`, it will be possible to write expressions inside types. Initially, only simple references will be
163 | supported, but at some point, more complex expressions will be needed, for example:
164 |
165 | ```sway
166 | fn len(a: [T; N]) { ... }
167 |
168 | fn bigger(a: [u64; N]) -> [u64; N + 1] {
169 | [0; N + 1]
170 | }
171 | ```
172 |
173 | This poses the challenge of having an expression tree inside of the type system. Currently Sway already
174 | has three expression trees: `Expr`, `ExpressionKind` and `TyExpressionVariant`. This demands a new one,
175 | given that the first two allow much more complex expressions than what `const generics` wants to support;
176 | and the last one is only created after some `TypeInfo` already exists, and thus cannot be used.
177 |
178 | At the same time, the parser should be able to parse any expression and return a friendly error that such expression
179 | is not supported.
180 |
181 | So, in case of an unsupported expression the parser will parser the `const generic` expression as it does for normal `Expr`and will lower it to the type system expression enum, but in the place of the unsupported expression will return a `TypeInfo::ErrorRecovery`.
182 |
183 | These expressions will also increase the complexity of all type related algorithms such as:
184 |
185 | 1. Unification
186 | 2. PartialEq
187 | 3. Hash
188 |
189 | In the simplest case, it is very clear how to unify `TypeInfo::Array(..., Length::Literal(1))` and `TypeInfo::Array(..., Length::Expression("N"))`.
190 | But more complex cases such as `TypeInfo::Array(..., Length::Expression("N"))` and `TypeInfo::Array(..., Length::Expression("N + 1"))`, is not clear
191 | if these types are unifiable, equal or simply different.
192 |
193 | ## Method call search algorithm
194 |
195 | When a method is called, the algorithm that searches which method is called uses the method `TraitMap::get_impls`.
196 | Currently, this method does an `O(1)` search to find all methods applicable to a type. For example:
197 |
198 | ```sway
199 | impl [u64; 1] {
200 | fn len_for_size_one(&self) { ... }
201 | }
202 | ```
203 |
204 | would create a map with something like
205 |
206 | ```
207 | Placeholder -> [...]
208 | ...
209 | [u64; 1] -> [..., len_for_size_one,...]
210 | ...
211 | ```
212 |
213 | The algorithms first create a `TypeRootFilter`, which is very similar to `TypeInfo`. And uses this `enum` to search the hash table.
214 | After that, it "generalizes" the filter and searches for `TypeRootFilter::Placeholder`.
215 |
216 | To fully support `const generics` and `const value specialization`, the compiler will now keep generalizing the
217 | searched type until it hits `Placeholder`. For example, searching for `[u64; 1]` will actually search for:
218 |
219 | 1. [u64; 1];
220 | 1. [u64; Placeholder];
221 | 1. Placeholder;
222 |
223 | The initial implementation will do this generalization only for `const generics`, but it also makes sense to
224 | generalize this with other types such as `Vec`.
225 |
226 | 1. Vec\;
227 | 1. Vec\;
228 | 1. Placeholder;
229 |
230 | This gets more complex as the number of `generics` and `const generics` increases. For example:
231 |
232 | ```sway
233 | struct VecWithSmallVecOptimization { ... }
234 | ```
235 |
236 | Searching for this type would search:
237 |
238 | 1. VecWithSmallVecOptimization\
239 | 1. VecWithSmallVecOptimization\
240 | 1. VecWithSmallVecOptimization\
241 | 1. VecWithSmallVecOptimization\
242 | 1. Placeholder
243 |
244 | More research is needed to understand if this change can potentially change the semantics of any program written in Sway.
245 |
246 | # Implementation Roadmap
247 |
248 | 1. Creation of the feature flag `const_generics`;
249 | 1. Implementation of "const generics references";
250 | ```rust
251 | fn f() { __log(I); }
252 | ```
253 | 3. The compiler will be able to encode arrays of any size; Which means being able to implement the following in the "core" lib and using arrays of any size as "configurables";
254 | ```rust
255 | impl AbiEncode for [T; N] { ... }
256 | ```
257 | 4. Being able to `abi_encode` arrays of any size;
258 | ```rust
259 | fn f(s: [T; N]) -> raw_slice {
260 | <[T; N] as AbiEncode>::abi_encode(...);
261 | }
262 | f::<1>([1])
263 | ```
264 | 5. Inference of the example above;
265 | ```rust
266 | f([1]);
267 | core::encode([1]);
268 | ```
269 | 6. Struct/enum support for const generics;
270 | 7. Function/method declaration;
271 | 8. `impl` declarations;
272 | 9. Function/method reference.
273 |
274 | # Drawbacks
275 |
276 | [drawbacks]: #drawbacks
277 |
278 | None
279 |
280 | # Rationale and alternatives
281 |
282 | [rationale-and-alternatives]: #rationale-and-alternatives
283 |
284 | # Prior art
285 |
286 | [prior-art]: #prior-art
287 |
288 | This RFC is partially based on Rust's own const generic system:
289 | - https://doc.rust-lang.org/reference/items/generics.html#const-generics
290 | - https://blog.rust-lang.org/inside-rust/2021/09/06/Splitting-const-generics.html
291 | - https://rust-lang.github.io/rfcs/2000-const-generics.html
292 | - https://doc.rust-lang.org/beta/unstable-book/language-features/generic-const-exprs.html
293 |
294 | # Unresolved questions
295 |
296 | [unresolved-questions]: #unresolved-questions
297 |
298 | 1. What is the impact of changing the "method call search algorithm"?
299 |
300 | # Future possibilities
301 |
302 | [future-possibilities]: #future-possibilities
303 |
304 | As mentioned above, implementing constraints like where N > 0.
305 |
--------------------------------------------------------------------------------