├── .github └── settings.yml ├── 0000-template.md ├── LICENSE ├── README.md ├── _config.yml ├── images ├── fpc │ ├── full-detail │ │ ├── fpc-cc-execution.png │ │ ├── fpc-cc-invocation.png │ │ ├── fpc-components.png │ │ ├── fpc-integration-touchpoints.png │ │ ├── fpc-key-dist.png │ │ ├── fpc-lifecycle-v2.png │ │ ├── fpc-registration.png │ │ └── fpc-validation.png │ └── high-level │ │ ├── Slide1.png │ │ ├── Slide2.png │ │ ├── Slide3.png │ │ ├── Slide4.png │ │ └── hl-diagrams.pptx └── lts_release_diagram.png ├── index.md ├── rfcs.md └── text ├── 0000-admin_sdk.md ├── 0000-fabric-extend-chaincode-lifecycle-api-to-query-all-aproved-chaincode.md ├── 0000-fabric-gateway.md ├── 0000-fabric-private-chaincode-1.0.md ├── 0000-fabric-website.md ├── 0000-ledger-checkpointing.md ├── 0000-opentelemetry-tracing.md ├── 0000-orderer-snapshot.md ├── 0000-private_data_purge.md ├── 0001-chaincode-go-new-programming-model.md ├── 0002-go-sdk-programming-model.md ├── 0003-config-transaction-library.md ├── 0004-channel-participation-api-without-system-channel.md ├── 0005-lts-release-strategy.md ├── 0010-bft-signatures.md ├── 0011-peer-chaincode-optimization.md ├── 006-bft-based-ordering-service.md └── orderer-v3.md /.github/settings.yml: -------------------------------------------------------------------------------- 1 | repository: 2 | name: fabric-rfcs 3 | description: RFC process for Hyperledger Fabric. The RFC (request for comments) 4 | process is intended to provide a consistent and controlled path for major 5 | changes to Fabric and other official project components. https://wiki.hyperledger.org/display/fabric 6 | homepage: https://hyperledger.github.io/fabric-rfcs/ 7 | default_branch: main 8 | has_downloads: true 9 | has_issues: false 10 | has_projects: true 11 | has_wiki: false 12 | archived: false 13 | private: false 14 | allow_squash_merge: true 15 | allow_merge_commit: false 16 | allow_rebase_merge: true 17 | -------------------------------------------------------------------------------- /0000-template.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: RFC Template 4 | nav_order: 3 5 | --- 6 | 7 | - Feature Name: (fill me in with a unique identifier, my_awesome_feature) 8 | - Start Date: (fill me in with today's date, YYYY-MM-DD) 9 | - RFC PR: (leave this empty) 10 | - Fabric Component: (fill me in with underlined fabric component, core, orderer/consensus and etc.) 11 | - Fabric Issue: (leave this empty) 12 | 13 | # Summary 14 | [summary]: #summary 15 | 16 | One paragraph explanation of the feature. 17 | 18 | # Motivation 19 | [motivation]: #motivation 20 | 21 | Why are we doing this? What use cases does it support? What is the expected 22 | outcome? 23 | 24 | # Guide-level explanation 25 | [guide-level-explanation]: #guide-level-explanation 26 | 27 | Explain the proposal as if it was already included in Fabric and you were 28 | teaching it to another Fabric developer. That generally means: 29 | 30 | - Introducing new named concepts. 31 | - Explaining the feature largely in terms of examples. 32 | - Explaining how Fabric programmers should *think* about the feature, and how 33 | it should impact the way they use fabric. It should explain the impact as 34 | concretely as possible. 35 | - If applicable, provide sample error messages, deprecation warnings, or 36 | migration guidance. 37 | - If applicable, describe the differences between teaching this to established 38 | and new Fabric developers. 39 | - If applicable, describe any changes that may affect the security of 40 | communications or administration. 41 | 42 | For implementation-oriented RFCs (e.g. for validator internals), this section 43 | should focus on how contributors should think about the change, and give 44 | examples of its concrete impact. For policy RFCs, this section should provide 45 | an example-driven introduction to the policy, and explain its impact in 46 | concrete terms. 47 | 48 | # Reference-level explanation 49 | [reference-level-explanation]: #reference-level-explanation 50 | 51 | This is the technical portion of the RFC. Explain the design in sufficient 52 | detail that: 53 | 54 | - Its interaction with other features is clear. 55 | - It is reasonably clear how the feature would be implemented. 56 | - Corner cases are dissected by example. 57 | 58 | The section should return to the examples given in the previous section, and 59 | explain more fully how the detailed proposal makes those examples work. 60 | 61 | # Drawbacks 62 | [drawbacks]: #drawbacks 63 | 64 | Why should we *not* do this? Are there any risks that must be considered along with 65 | this rfc. 66 | 67 | # Rationale and alternatives 68 | [alternatives]: #alternatives 69 | 70 | - Why is this design the best in the space of possible designs? 71 | - What other designs have been considered and what is the rationale for not 72 | choosing them? 73 | - What is the impact of not doing this? 74 | 75 | # Prior art 76 | [prior-art]: #prior-art 77 | 78 | Discuss prior art, both the good and the bad, in relation to this proposal. 79 | A few examples of what this can include are: 80 | 81 | - For consensus, global state, transaction processors, and smart contracts 82 | implementation proposals: Does this feature exists in other distributed 83 | ledgers and what experience have their communities had? 84 | - For community proposals: Is this done by some other community and what were 85 | their experiences with it? 86 | - For other teams: What lessons can we learn from what other communities have 87 | done here? 88 | - Papers: Are there any published papers or great posts that discuss this? If 89 | you have some relevant papers to refer to, this can serve as a more detailed 90 | theoretical background. 91 | 92 | This section is intended to encourage you as an author to think about the 93 | lessons from other distributed ledgers, provide readers of your RFC with 94 | a fuller picture. If there is no prior art, that is fine - your ideas are 95 | interesting to us whether they are brand new or if it is an adaptation. 96 | 97 | Note that while precedent set by other distributed ledgers is some motivation, 98 | it does not on its own motivate an RFC. Please also take into consideration 99 | that Fabric sometimes intentionally diverges from common distributed 100 | ledger/blockchain features. 101 | 102 | # Testing 103 | [testing]: #testing 104 | 105 | - What kinds of test development and execution will be required in order 106 | to validate this proposal, beyond the usual mandatory unit tests? 107 | - List integration test scenarios which will outline correctness of proposed functionality. 108 | 109 | # Dependencies 110 | [dependencies]: #dependencies 111 | 112 | - Describe all dependencies that this proposal might have on other RFCs, known JIRA issues, 113 | Hyperledger Fabric components. Dependencies upon RFCs or issues should be recorded as 114 | links in the proposals issue itself. 115 | 116 | - List down related RFCs proposals that depend upon current RFC, and likewise make sure 117 | they are also linked to this RFC. 118 | 119 | # Unresolved questions 120 | [unresolved]: #unresolved-questions 121 | 122 | - What parts of the design do you expect to resolve through the RFC process 123 | before this gets merged? 124 | - What parts of the design do you expect to resolve through the implementation 125 | of this feature before stabilization? 126 | - What related issues do you consider out of scope for this RFC that could be 127 | addressed in the future independently of the solution that comes out of this 128 | RFC? 129 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright {yyyy} {name of copyright owner} 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | 203 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: RFCs Process 4 | nav_order: 2 5 | --- 6 | # Hyperledger Fabric RFCs Process 7 | 8 | Many changes, including bug fixes and documentation improvements can be 9 | implemented and reviewed via the normal [GitHub pull request workflow](https://guides.github.com/introduction/flow/). 10 | 11 | Some changes though are substantial, and we ask that these be put through a bit 12 | of a design process and produce a consensus among the Fabric maintainers and 13 | broader community. 14 | 15 | The RFC (request for comments) process is intended to provide a consistent and 16 | controlled path for major changes to Fabric and other official project 17 | components, so that all stakeholders can be confident about the direction in 18 | which Fabric is evolving. 19 | 20 | This process is intended to be substantially similar to the RFCs process other 21 | Hyperledger teams have adopted, customized as necessary for use with Fabric. 22 | The `README.md` and `0000-template.md` files were forked from the 23 | [Sawtooth RFCs repo](https://github.com/hyperledger/sawtooth-rfcs), which was 24 | derived from the Rust project. 25 | 26 | ## Table of Contents 27 | 28 | - [When you need to follow this process] 29 | - [Before creating an RFC] 30 | - [What the process is] 31 | - [The RFC life-cycle] 32 | - [Reviewing RFCs] 33 | - [Implementing an RFC] 34 | - [License] 35 | - [Contributions] 36 | 37 | ## When you need to follow this process 38 | 39 | [When you need to follow this process]: #when-you-need-to-follow-this-process 40 | 41 | You need to follow this process if you intend to make substantial changes to 42 | Fabric or any of its sub-components including but not limited to 43 | fabric-baseimage, fabric-sdk-node, fabric-sdk-java, fabric-ca, 44 | fabric-chaincode-go, fabric-chaincode-java, fabric-chaincode-node, 45 | fabric-chaincode-evm, fabric-protos, fabric-protos-go, or the RFC process 46 | itself. What constitutes a substantial change is evolving based on community 47 | norms and varies depending on what part of the ecosystem you are proposing to 48 | change, but may include the following: 49 | 50 | - Architectural changes 51 | - Substantial changes to component interfaces 52 | - New core features 53 | - Backward incompatible changes 54 | - Changes that affect the security of communications or administration 55 | 56 | Some changes do not require an RFC: 57 | 58 | - Rephrasing, reorganizing, refactoring, or otherwise "changing shape does not 59 | change meaning". 60 | - Additions that strictly improve objective, numerical quality criteria 61 | (warning removal, speedup, better platform coverage, more parallelism, trap 62 | more errors, etc.). 63 | 64 | If you submit a pull request to implement a new feature without going through 65 | the RFC process, it may be closed with a polite request to submit an RFC first. 66 | 67 | ## Before creating an RFC 68 | 69 | [Before creating an RFC]: #before-creating-an-rfc 70 | 71 | A hastily-proposed RFC can hurt its chances of acceptance. Low quality 72 | proposals, proposals for previously-rejected changes, or those that don't fit 73 | into the near-term roadmap, may be quickly rejected, which can be demotivating 74 | for the unprepared contributor. Laying some groundwork ahead of the RFC can 75 | make the process smoother. 76 | 77 | Although there is no single way to prepare for submitting an RFC, it is 78 | generally a good idea to pursue feedback from other project developers 79 | beforehand, to ascertain that the RFC may be desirable; having a consistent 80 | impact on the project requires concerted effort toward consensus-building. 81 | 82 | The most common preparations for writing and submitting an RFC include 83 | talking the idea over to the [Fabric mailing list](https://lists.hyperledger.org/g/fabric/topics). 84 | 85 | As a rule of thumb, receiving encouraging feedback from long-standing 86 | project developers, and particularly the project's maintainers is a good 87 | indication that the RFC is worth pursuing. 88 | 89 | ## What the process is 90 | 91 | [What the process is]: #what-the-process-is 92 | 93 | In short, to get a major feature added to Fabric, one must first get the RFC 94 | merged into the RFC repository as a markdown file. At that point the RFC is 95 | "active" and may be implemented with the goal of eventual inclusion into 96 | Fabric. 97 | 98 | - Fork [the RFC repository](https://github.com/hyperledger/fabric-rfcs). 99 | - Copy `0000-template.md` to `text/0000-my-feature.md`, where "my-feature" is 100 | descriptive. Don't assign an RFC number yet. 101 | - Fill in the RFC. Put care into the details — RFCs that do not present 102 | convincing motivation, demonstrate understanding of the impact of the design, 103 | or are disingenuous about the drawbacks or alternatives tend to be 104 | poorly received. 105 | - Submit a pull request. The pull request will be assigned to a maintainer, and 106 | will receive design feedback from the larger community; the RFC author should 107 | be prepared to revise it in response. 108 | - Build consensus and integrate feedback. RFCs that have broad support are much 109 | more likely to make progress than those that don't receive any comments. Feel 110 | free to reach out to the pull request assignee in particular to get help 111 | identifying stakeholders and obstacles. 112 | - The maintainers will discuss the RFC pull request, as much as possible in the 113 | comment thread of the pull request itself. Offline discussion will be 114 | summarized on the pull request comment thread. 115 | - A good way to build consensus on a RFC pull request is to summarize the RFC on a 116 | community contributor meeting. Coordinate with a maintainer to get on a contributor 117 | meeting agenda. While this is not necessary, it may help to foster sufficient 118 | consensus such that the RFC can proceed to final comment period. 119 | - RFCs rarely go through this process unchanged, especially as alternatives and 120 | drawbacks are shown. You can make edits, big and small, to the RFC to clarify 121 | or change the design, but make changes as new commits to the pull request, and 122 | leave a comment on the pull request explaining your changes. Specifically, do 123 | not squash or rebase commits after they are visible on the pull request. 124 | - At some point, a Fabric maintainer will propose a "motion for final comment 125 | period" (FCP), along with a *disposition* for the RFC (merge, close, or 126 | postpone). 127 | - This step is taken when enough of the tradeoffs have been discussed that 128 | the maintainers are in a position to make a decision. That does not require 129 | consensus amongst all participants in the RFC thread (which is usually 130 | impossible). However, the argument supporting the disposition on the RFC 131 | needs to have already been clearly articulated, and there should not be a 132 | strong consensus *against* that position. Fabric maintainers will use their 133 | best judgment in taking this step, and the FCP itself ensures there is ample 134 | time and notification for stakeholders to push back if it is made 135 | prematurely. 136 | - For RFCs with lengthy discussion, the motion to FCP is usually preceded by 137 | a *summary comment* trying to lay out the current state of the discussion and 138 | major trade-offs/points of disagreement. 139 | - Before actually entering FCP, the Fabric maintainer who proposes that the 140 | RFC enter FCP ensures that other interested maintainers have reviewed the RFC 141 | and at least two other maintainers (three total) have indicated agreement; 142 | this is often the point at which many maintainers first review the RFC in 143 | full depth. Note that maintainers from any Fabric repository may review and 144 | indicate agreement, especially for RFCs that impact multiple repositories. 145 | - The FCP lasts one week, or seven calendar days. It is also advertised widely, 146 | e.g. in the [Fabric Mailing List](https://lists.hyperledger.org/g/fabric/topics). 147 | This way all stakeholders have a chance to lodge any final objections before a 148 | decision is reached. 149 | - In most cases, the FCP period is quiet since the most interested maintainers 150 | have already indicated agreement, and the RFC is either merged or 151 | closed. However, sometimes substantial new arguments or ideas are raised, the 152 | FCP is canceled, and the RFC goes back into development mode. 153 | 154 | ## The RFC life-cycle 155 | 156 | [The RFC life-cycle]: #the-rfc-life-cycle 157 | 158 | Once an RFC is merged, it becomes "active" and developers may implement it and submit the code 159 | change as a pull request to the corresponding Fabric repo. Being "active" is 160 | not a rubber stamp, and it does not mean the change will ultimately be merged; 161 | it does mean that in principle all the major stakeholders have agreed to the 162 | change, and are amenable to merging it. 163 | 164 | Furthermore, the fact that a given RFC has been accepted and is "active" 165 | implies nothing about what priority is assigned to its implementation, nor does 166 | it imply anything about whether a Fabric developer has been assigned the task 167 | of implementing the feature. While it is not *necessary* that the author of the 168 | RFC also write the implementation, it is by far the most effective way to see 169 | an RFC through to completion: authors should not expect that other project 170 | developers will take on responsibility for implementing their accepted feature. 171 | 172 | Modifications to active RFCs can be done in follow-up pull requests. We strive 173 | to write each RFC in a manner that it will reflect the final design of the 174 | feature; but the nature of the process means that we cannot expect every merged 175 | RFC to actually reflect what the end result will be at the time of the next 176 | major release. 177 | 178 | In general, once accepted, RFCs should not be substantially changed. Only very 179 | minor changes should be submitted as amendments. More substantial changes 180 | should be new RFCs, with a note added to the original RFC. Exactly what counts 181 | as a "very minor change" is up to the maintainers to decide. 182 | 183 | ## Reviewing RFCs 184 | 185 | [Reviewing RFCs]: #reviewing-rfcs 186 | 187 | While the RFC pull request is up, the maintainers may schedule meetings with 188 | the author and/or relevant stakeholders to discuss the issues in greater 189 | detail, and in some cases the topic may be discussed at a contributors meeting. 190 | In either case a summary from the meeting will be posted back to the RFC pull 191 | request. 192 | 193 | The Fabric maintainers make the final decisions about RFCs after the benefits 194 | and drawbacks are well understood. These decisions can be made at any time, but 195 | the maintainers will regularly issue decisions. When a decision is made, the 196 | RFC pull request will either be merged or closed. In either case, if the 197 | reasoning is not clear from the discussion in thread, the maintainers will add 198 | a comment describing the rationale for the decision. 199 | 200 | ## Implementing an RFC 201 | 202 | [Implementing an RFC]: #implementing-an-rfc 203 | 204 | Some accepted RFCs represent vital changes that need to be implemented right 205 | away. Other accepted RFCs can represent changes that can wait until a 206 | developer feels like doing the work. Every accepted RFC has an associated 207 | issue tracking its implementation in the [Fabric issues](https://github.com/hyperledger/fabric/issues). 208 | 209 | The author of an RFC is not obligated to implement it. Of course, the RFC 210 | author, as any other developer, is welcome to post an implementation for review 211 | after the RFC has been accepted. Use GitHub issues for this. 212 | 213 | ## License 214 | 215 | [License]: #license 216 | 217 | This repository is licensed under [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0) 218 | ([LICENSE](LICENSE)). 219 | 220 | ## Contributions 221 | 222 | [Contributions]: #contributions 223 | 224 | Unless you explicitly state otherwise, any contribution intentionally submitted 225 | for inclusion in the work by you, as defined in the Apache-2.0 license, shall 226 | be licensed as above, without any additional terms or conditions. 227 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | remote_theme: pmarsceill/just-the-docs 2 | theme: jekyll-theme-cayman -------------------------------------------------------------------------------- /images/fpc/full-detail/fpc-cc-execution.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/full-detail/fpc-cc-execution.png -------------------------------------------------------------------------------- /images/fpc/full-detail/fpc-cc-invocation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/full-detail/fpc-cc-invocation.png -------------------------------------------------------------------------------- /images/fpc/full-detail/fpc-components.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/full-detail/fpc-components.png -------------------------------------------------------------------------------- /images/fpc/full-detail/fpc-integration-touchpoints.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/full-detail/fpc-integration-touchpoints.png -------------------------------------------------------------------------------- /images/fpc/full-detail/fpc-key-dist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/full-detail/fpc-key-dist.png -------------------------------------------------------------------------------- /images/fpc/full-detail/fpc-lifecycle-v2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/full-detail/fpc-lifecycle-v2.png -------------------------------------------------------------------------------- /images/fpc/full-detail/fpc-registration.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/full-detail/fpc-registration.png -------------------------------------------------------------------------------- /images/fpc/full-detail/fpc-validation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/full-detail/fpc-validation.png -------------------------------------------------------------------------------- /images/fpc/high-level/Slide1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/high-level/Slide1.png -------------------------------------------------------------------------------- /images/fpc/high-level/Slide2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/high-level/Slide2.png -------------------------------------------------------------------------------- /images/fpc/high-level/Slide3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/high-level/Slide3.png -------------------------------------------------------------------------------- /images/fpc/high-level/Slide4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/high-level/Slide4.png -------------------------------------------------------------------------------- /images/fpc/high-level/hl-diagrams.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/fpc/high-level/hl-diagrams.pptx -------------------------------------------------------------------------------- /images/lts_release_diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hyperledger/fabric-rfcs/717d60a6782abfacebc66baeff0b2d45e7fc5217/images/lts_release_diagram.png -------------------------------------------------------------------------------- /index.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Hyperledger Fabric RFCs 4 | nav_order: 1 5 | --- 6 | 7 | # RFCs 8 | 9 | The RFC (request for comments) process is intended to provide a consistent and 10 | controlled path for major changes to Fabric and other official project 11 | components, so that all stakeholders can be confident about the direction in 12 | which Fabric is evolving. 13 | 14 | - [Template](0000-template.md) 15 | - [Process](README.md) 16 | -------------------------------------------------------------------------------- /rfcs.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: RFCs 4 | has_children: true 5 | nav_order: 3 6 | --- 7 | 8 | # List of RFCs -------------------------------------------------------------------------------- /text/0000-admin_sdk.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Fabric admin SDK 4 | nav_order: 3 5 | --- 6 | 7 | - Feature Name: Fabric admin SDK 8 | - Start Date: 2022-11-11 9 | - RFC PR: (leave this empty) 10 | - Fabric Component: fabric-admin 11 | - Fabric Issue: (leave this empty) 12 | 13 | # Summary 14 | [summary]: #summary 15 | 16 | This RFC proposes a new administrative SDK for Fabric to support implementers of Kubernetes operators, CLI commands, and automated deployment pipelines. The primary target implementation language for this API is Go, with the aim to deliver implementations in the same set of languages as the client application APIs, namely: Go, Node (TypeScript / JavaScript) and Java. Key objectives are to provide consistent capability across all implementation languages, and to avoid any dependency on core Fabric since this is not intended to be consumed as a library. 17 | 18 | 19 | # Motivation 20 | [motivation]: #motivation 21 | 22 | There are two primary motivators for this work: 23 | 24 | 1. Legacy client application SDKs developed alongside Fabric v1 provide varying levels of administrative capability, and this is relied upon by a subset of the Fabric user base to support automated deployment. Client application APIs developed alongside Fabric v2 focus only on the Fabric programming model, supporting business applications to submit and evaluate transactions, and to receive events. Fabric v2.5 plans to deprecate the legacy SDKs, leaving a gap in support for programmatic deployment of Fabric. 25 | 26 | 2. The `peer` command provides both the server-side peer and the Fabric CLI implementations. Therefore the CLI commands are tightly coupled to the core Fabric codebase. A new Fabric admin API would provide a cleaner base on which to build Fabric CLI commands, decoupled from the core Fabric codebase. 27 | 28 | # Guide-level explanation 29 | [guide-level-explanation]: #guide-level-explanation 30 | 31 | The Fabric admin API aims to deliver the following minimum set of capability: 32 | 33 | - Chaincode deployment (using v2 chaincode lifecycle): 34 | - Install chaincode 35 | - Query installed chaincode 36 | - Approve installed chaincode 37 | - Query approved chaincode 38 | - Commit approved chaincode 39 | - Query committed chaincode 40 | 41 | - Channel configuration: 42 | - Create channel 43 | - Join peers 44 | - Set anchor peers 45 | 46 | Additional capability may be included to aid deployment or common configuration tasks, where this does not represent duplication of capability already easily achievable using existing programmatic APIs. 47 | 48 | # Reference-level explanation 49 | [reference-level-explanation]: #reference-level-explanation 50 | 51 | The implementation takes the following design approach: 52 | 53 | - Expose a relatively simple API that provides a layer of abstraction around the mechanics of invoking gRPC and HTTP/RESTful services provided by Fabric to achieve administrative tasks. 54 | - Loose coupling between API implementation and network connection management, allowing the caller to retain control of creation and configuration of network connections. 55 | - Loose coupling between API and cryptographic implementations, allowing the caller to inject the cryptographic credentials and signing implementation. 56 | - Consistency of capability across different language implementations while following language idioms. 57 | 58 | The admin SDK interacts with Fabric using only well-defined gRPC services and HTTP/RESTful APIs. To simplify the implementation and provide consistency with current client application APIs, this includes the use of Gateway gRPC services, either directly using gRPC client APIs or using the Fabric Gateway client API. 59 | 60 | ## Example API methods 61 | 62 | There follow some examples of proposed API calls provided by the admin SDK. 63 | 64 | ### Install chaincode 65 | 66 | ```go 67 | func Install( 68 | ctx context.Context, 69 | peerConnection grpc.ClientConnInterface, 70 | signer identity.SignerSerializer, 71 | packageReader io.Reader, 72 | callOptions ...grpc.CallOption, 73 | ) error 74 | ``` 75 | 76 | - `ctx` allows the caller to cancel the operation, either directly or after a timeout period. 77 | - `peerConnection` caller provided gRPC connection used to make the call, which has been configured appropriately and may be shared between multiple calls. 78 | - `signer` encapsulates the signing implementation and client credentials. 79 | - `packageReader` supplies the chaincode package content. 80 | - `callOptions` are call-specific gRPC options. 81 | 82 | ### Query installed chaincode 83 | 84 | ```go 85 | func QueryInstalled( 86 | ctx context.Context, 87 | peerConnection grpc.ClientConnInterface, 88 | signer identity.SignerSerializer, 89 | callOptions ...grpc.CallOption, 90 | ) (*lifecycle.QueryInstalledChaincodesResult, error) 91 | ``` 92 | 93 | The parameters are common with the proposed install chaincode API. The return value is a protocol buffer message, defined by Fabric and included in [fabric-protos](https://hyperledger.github.io/fabric-protos/). 94 | 95 | # Drawbacks 96 | [drawbacks]: #drawbacks 97 | 98 | An additional SDK requires additional development effort, support and ongoing maintenance. 99 | 100 | While significantly simplifying the implementation and maintenance burden, making use of Gateway gRPC services and/or the Fabric Gateway client API limit use of the admin SDK to Fabric v2.4 and later. The admin capability in legacy SDKs is available for earlier Fabric versions. 101 | 102 | # Rationale and alternatives 103 | [alternatives]: #alternatives 104 | 105 | Administrative tasks are currently possible using the Fabric CLI and an alternative is to continue with them as the only supported administrative tool. However, there is real community interest in programmatic configuration, and community members already actively contributing to an admin SDK implementation. A Go admin SDK dramatically simplifies the development and maintenance effort required to implement both Kubernetes operators and new Fabric CLI commands. 106 | 107 | # Prior art 108 | [prior-art]: #prior-art 109 | 110 | Legacy SDKs provide varying levels of admin capability. Their development in entirely separate codebases has naturally led to inconsistency in capability and API design. As part of larger packages, not focused purely on admin tasks, the ongoing maintenance and evolution of their admin capability has become impractical. 111 | 112 | Active development on a new admin SDK is taking place at [Hyperledger-TWGC/fabric-admin-sdk](https://github.com/Hyperledger-TWGC/fabric-admin-sdk). This RFC proposes adopting that as the basis of a Hyperledger Fabric admin SDK. 113 | 114 | # Testing 115 | [testing]: #testing 116 | 117 | In additional to typical test-driven development using unit tests for specific language implementations, integration tests will be required to confirm that admin capabilities work correctly with a real Fabric deployment. These integration tests should be fully automated and run as part of the continuous integration pipeline. 118 | 119 | Since one of the goals is to provide consistent capability across different language implementations of the admin SDK, it is desirable to run a consistent set of integration tests against each language implementation. One possible approach for achieving this is to produce language-agnostic test definitions that are run against all language implementations. The approach has been used successfully in the [Fabric Gateway client API](https://github.com/hyperledger/fabric-gateway/tree/main/scenario/features) using [Cucumber](https://cucumber.io/) as the test framework. 120 | 121 | # Dependencies 122 | [dependencies]: #dependencies 123 | 124 | The admin SDK is expected to include [fabric-protos-go-apiv2](https://github.com/hyperledger/fabric-protos-go-apiv2) and possibly also [fabric-gateway](https://github.com/hyperledger/fabric-gateway) as direct dependencies. There will be dependencies on [gRPC](https://grpc.io/) APIs to interact with gRPC services provided by Fabric, both directly and indirectly through the protocol buffer bindings. 125 | 126 | Dependencies **must not include** core Fabric, which is not intended to be consumed as a library. Where utilities contained within core Fabric are found to be broadly applicable both in Fabric and the admin SDK (and ideally also in other projects), those utilities could be extracted to a separate project, such as [fabric-lib-go](https://github.com/hyperledger/fabric-lib-go) and [fabric-config](https://github.com/hyperledger/fabric-config). 127 | 128 | # Unresolved questions 129 | [unresolved]: #unresolved-questions 130 | 131 | The fine details of the admin API are expected to evolve as capability is implemented as user stories. The end-user usage experience provided by the API in specific scenarios, such as implementation of a Kubernetes operator and automated Fabric deployment, will guide the design and implementation. The intention is to socialize the API with the community throughout the development process to solicit feedback. 132 | 133 | While one of the motivators for an admin API is to provide a foundation on which new Fabric CLI commands are built, those CLI commands are out of scope for this RFC. 134 | -------------------------------------------------------------------------------- /text/0000-fabric-extend-chaincode-lifecycle-api-to-query-all-aproved-chaincode.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Extend chaincode lifecycle API to query all approved chaincodes 4 | nav_order: 3 5 | --- 6 | 7 | - Feature Name: extend_chaincode_lifecycle_function_to_query_all_approved_chaincode 8 | - Start Date: 2021-08-02 9 | - RFC PR: (leave this empty) 10 | - Fabric Component: fabric, fabric-protos 11 | - Fabric Issue: (leave this empty) 12 | 13 | # Summary 14 | [summary]: #summary 15 | 16 | ## Extend chaincode lifecycle API to query all approved chaincodes 17 | 18 | Currently, Fabric peer CLI 'queryapproved' could query the details of an approved chaincode definitions, and this CLI requires a mandatory chaincode name parameter to query a specific chaincode definitions; however, in some cases we expect to query out all approved chaincode definitions. 19 | 20 | So, here we propose to extend the 'queryapproved' function for querying the details of all the approved chaincode definition to Fabric peer CLI. 21 | 22 | The following shows a basic design of this extension. 23 | 24 | - There is no new command is involved. 25 | - Command 'queryapproved' will be extended to support this function: 26 | - make parameter (-n|--name [chaincode_name]) optional 27 | - if parameter (-n|--name [chaincode_name]) is provided, keep current command behavior 28 | - if parameter (-n|--name [chaincode_name]) is not provided, query the details of all approved chaincode, plus additional chaincode name in details output. 29 | 30 | *The example of query all approved chaincodes with JSON output format:* 31 | ``` 32 | $ peer lifecycle chaincode queryapproved --channelID --peerAddresses peer0.org1.example.com:7051 --output json 33 | { 34 | "chaincode_definitions": [ 35 | { 36 | "name": "", 37 | "sequence": 1, 38 | "version": "1.0", 39 | "endorsement_plugin": "escc", 40 | "validation_plugin": "vscc", 41 | "validation_parameter": "EiAvQ2hhbm5lbC9BcHBsaWNhdGlvbi9FbmRvcnNlbWVudA==", 42 | "collections": {}, 43 | "init_required": true, 44 | "source": { 45 | "Type": { 46 | "LocalPackage": { 47 | "package_id": "-1.0:e60b4fc692998844183e70ca6ae15bcd4632ef1b0e93193567a6669fb945d86d" 48 | } 49 | } 50 | } 51 | }, 52 | { 53 | "name": "", 54 | "sequence": 1, 55 | "version": "1.0", 56 | "endorsement_plugin": "escc", 57 | "validation_plugin": "vscc", 58 | "validation_parameter": "EiAvQ2hhbm5lbC9BcHBsaWNhdGlvbi9FbmRvcnNlbWVudA==", 59 | "collections": {}, 60 | "init_required": true, 61 | "source": { 62 | "Type": { 63 | "LocalPackage": { 64 | "package_id": "-1.0:23d718fa220eb599a412dfea13c18958b58fd0ffe4a42e2335b17c2c5fa102e9" 65 | } 66 | } 67 | } 68 | }, 69 | { 70 | "name": "", 71 | "sequence": 2, 72 | "version": "2.0", 73 | "endorsement_plugin": "escc", 74 | "validation_plugin": "vscc", 75 | "validation_parameter": "EiAvQ2hhbm5lbC9BcHBsaWNhdGlvbi9FbmRvcnNlbWVudA==", 76 | "collections": {}, 77 | "init_required": true, 78 | "source": { 79 | "Type": { 80 | "LocalPackage": { 81 | "package_id": "-2.0:23d718fa220eb599a412dfea13c18958b58fd0ffe4a42e2335b17c2c5fa102e9" 82 | } 83 | } 84 | } 85 | } 86 | ] 87 | } 88 | ``` 89 | 90 | # Motivation 91 | [motivation]: #motivation 92 | 93 | In some cases we expect to query out all approved chaincode definitions. 94 | 95 | # Guide-level explanation 96 | [guide-level-explanation]: #guide-level-explanation 97 | 98 | The original 'queryapproved' implementation can be found by referring to a series of PRs, including this [PR](https://github.com/hyperledger/fabric-protos/pull/25). 99 | 100 | # Reference-level explanation 101 | [reference-level-explanation]: #reference-level-explanation 102 | 103 | The command reference of the original 'queryapproved' is [here](https://hyperledger-fabric.readthedocs.io/en/latest/commands/peerlifecycle.html#peer-lifecycle-chaincode-queryapproved). 104 | 105 | # Testing 106 | [testing]: #testing 107 | 108 | - Enhance existing unit and integration tests on 'queryapproved' command. 109 | 110 | # Dependencies 111 | [dependencies]: #dependencies 112 | 113 | - fabric-protos 114 | - This feature needs to extend lifecycle protos (e.g., Adding 'QueryApprovedChaincodeResults'). 115 | -------------------------------------------------------------------------------- /text/0000-fabric-gateway.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Fabric Gateway 4 | nav_order: 3 5 | --- 6 | 7 | - Feature Name: Fabric Gateway 8 | - Start Date: 2020-09-08 9 | - RFC PR: (leave this empty) 10 | - Fabric Component: fabric-gateway 11 | - Fabric Issue: (leave this empty) 12 | 13 | # Summary 14 | [summary]: #summary 15 | 16 | The Fabric Gateway is a new component that will run either as a standalone process, or embedded with the peer, and will implement as much of the high-level 'gateway' programming model as possible. This will remove much of the transaction submission logic from the client application which will simplify the maintenance of the SDKs. It will also simplify the administrative overhead of running a Fabric network because client applications will be able to connect and submit transactions via a single network port rather than the current situation where ports have to be opened to multiple peers across potentially multiple organisations. 17 | 18 | # Motivation 19 | [motivation]: #motivation 20 | 21 | The high-level "gateway" programming model has proved to be very popular with users since its introduction in v1.4 in Node SDK. Since then, it has been implemented in the Java and Go SDKs. This has led to a large amount of code that has to be maintained in the SDKs which increases the cost of implementing new feature which have to be replicated to keep the SDKs in step. It also increases the cost of creating an SDK for a new language. Moving all of the high-level logic into its own process (or peer) and exposing a simplified gRPC interface will drastically reduce the complexity of the SDKs. 22 | 23 | By providing a single entry point to a Fabric network, client applications can interact with complex network topologies without multiple ports having to be opened and secured to each peer and ordering node. The client will connect only to the gateway which will act on its behalf with the rest of the network. 24 | 25 | # Guide-level explanation 26 | [guide-level-explanation]: #guide-level-explanation 27 | 28 | The Fabric Gateway is an embodiment of the high-level 'gateway' Fabric programming model in a server component that will form part of a Fabric network alongside Peers, Orderers and CAs. It will either be stood up inside its own process (optionally in a docker container), or it will be hosted inside a peer. Either way, it exposes its functionality to clients via a gRPC interface. The Gateway server component will be implemented in Go. 29 | 30 | Lightweight client SDKs are used to connect to the Gateway for the purpose of invoking transactions. The gateway will interact with the rest of the network on behalf of the client eliminating the need for the client application to connect directly to peers and orderers. 31 | 32 | The concepts and APIs will be familiar to Fabric programmers who are already using the new programming model. 33 | 34 | The scope of functionality exposed to the client by this component will include transaction evaluate/submit and event listening. 35 | 36 | # Reference-level explanation 37 | [reference-level-explanation]: #reference-level-explanation 38 | 39 | The Gateway runs as a client process associated with an organisation and requires an identity (issued by the org's CA) in order to interact with the discovery service and to register for events (Deliver Client). 40 | 41 | The Gateway also runs as a gRPC server and exposes the following services to client applications (SDKs): 42 | 43 | ``` 44 | service Gateway { 45 | rpc Endorse(ProposedTransaction) returns (PreparedTransaction) {} 46 | rpc Submit(PreparedTransaction) returns (stream Event) {} 47 | rpc Evaluate(ProposedTransaction) returns (Result) {} 48 | } 49 | ``` 50 | 51 | This exact detail of this protobuf definition may evolve slightly depending on implementation experience, but this shows the general design. 52 | 53 | The client credentials (private key) are never passed to the Gateway. Client applications are responsible for managing their user keys (optionally using SDK Wallets) and signing the protobuf payloads. 54 | 55 | The SDKs will use these services to implement the `SubmitTransaction` and `EvaluateTransaction` functions as follows: 56 | 57 | __Submitting a transaction__ is a two step process (performed by the client SDK): 58 | - __Endorse__ 59 | - The client creates a ProposedTransaction message, which contains a SignedProposal message as defined in `fabric-protos/peer/proposal.proto` and signed with the user's identity. The ProposedTransaction message also contains other optional properties, such as the endorsing orgs if the client wants to specify that. 60 | - The `Endorse` service is invoked on the Gateway, passing the ProposedTransaction message 61 | - The Gateway will determine the endorsement plan for the requested chaincode and forward to the appropriate peers for endorsement. It will return to the client a `PreparedTransaction` message which contains a `Envelope` message as defined in `fabric-protos/common/common.proto`. It will also contain other information, such as the return value of the chaincode function and the transaction ID so that the client doesn't necessarily need to unpack the `Envelope` payload. 62 | - __Submit__ 63 | - The client signs the hash of the Envelope payload using the user's private key and sets the signature field in the `Envelope` in the `PreparedTransaction`. 64 | - The `Submit` service is invoked on the Gateway, passing the signed `PreparedTransaction` message. A stream is opened to return multiple return values. 65 | - The Gateway will register transaction event listeners for the given channel/txId. 66 | - It will then broadcast the `Envelope` to the ordering service. 67 | - The success/error response is passed back to the client in the stream 68 | - The Gateway awaits sufficient tx commit events before returning and closing the stream, indicating to the client that transaction has been committed. 69 | 70 | __Evaluating a transaction__ is a simple process of invoking the `Evaluate` service passing a SignedProposal message. The Gateway passes the request to a peer of it's choosing according to a defined policy (probably same org, highest block count) and returns the chaincode function return value to the client. 71 | 72 | ## Launching the Gateway 73 | 74 | #### Standalone process 75 | When running standalone, the Gateway is effectively a 'client' application to a set of peers. A gateway instance is associated with an organization and will be configured to connect to a peer in the same organization in order to invoke discovery. To do this it will need a signing identity in the channel writers policy. 76 | 77 | To run the gateway server, the following parameters must to be supplied: 78 | - The url of at least one peer in the org. Once connected, the discovery service will be invoked to find other peers. 79 | - The MSPID associated with the gateway. 80 | - The signing identity of the gateway (cert and key), e.g. location of PEM files on disk. 81 | - The CA certificate so that the gateway can make TLS connections to other components (peers/orderers) in the organization 82 | 83 | Note that the signing identity is required so the gateway server can make discovery requests and to register event listeners. It is not used for submitting any transactions on behalf of end users. 84 | 85 | #### Embedded in Peer 86 | When embedded in a peer, the gateway will register its gRPC service with the host peer's gRPC server. 87 | The gateway does not need its own (client) identity since it can get all the discovery and event information directly from the host peer rather than via a gRPC signed request. 88 | 89 | ### Discovery 90 | 91 | On startup, the gateway will connect to its primary peer(s), as specified in the command line option (or its host, if embedded in a peer). The discovery service will be invoked to build and maintain a cache of the network topology per channel. This will be used to identify endorsers based on the chaincode endorsement policy. 92 | 93 | #### Collection endorsement policy 94 | 95 | Discovery can be used to get endorsing peers for a given chaincode where the transaction function will write to a given collection, and this will take into account the collection level endorsement policy. 96 | Ideally we could take care of these endorsement requirements in the Gateway without the client needing to have any implicit knowledge of them. Suggest extending contract metadata to include information on collections, which can be queried by Fabric Gateway. If that isn't possible then we might need some way for the client to specify the collections that will be used (not desirable). 97 | 98 | ### State based endorsement 99 | 100 | This is driven by business logic and rules. Requires the client to supply constraints on endorsing organisations as part of the proposal endorsement. 101 | 102 | ### Chaincode-to-chaincode endorsement 103 | 104 | Augment contract metadata for transactions with additional chaincodes / transactions that will be invoked, then we use discovery to narrow the set of endorsers. 105 | 106 | ### Proposal response consistency checking 107 | 108 | Current client SDK implementations do minimal checking of proposal responses before sending to the orderer. There have been some requests for more rigorous checking of proposal responses prior to submit to the orderer. 109 | 110 | Prior to sending to the orderer, Fabric Gateway should check that proposal responses are: 111 | - Consistent, including having matching response payloads. 112 | - Meet all applicable endorsement requirements. 113 | 114 | ### Ensure query results reflect latest ledger state 115 | 116 | Query results should be obtained from peers whose ledger height is at least that of the most recent events sent to clients. 117 | 118 | Scenario: 119 | - Client receipt of chaincode event triggers business process. 120 | - Business process queries ledger to obtain private data, which cannot be included in event payload since it is private. 121 | The private data must be consistent with the state of the ledger after commit of the transaction that emitted the chaincode event, so the query must be processed by a peer with ledger height of at least the block number that contained the chaincode event. 122 | 123 | Implementation options: 124 | - Only direct queries (and endorsement requests) to the peer(s) with the highest ledger height. 125 | - Client proposals include last observed ledger height so Gateway can target any peers with at least that ledger height. 126 |  A view of ledger height can be tracked by the Gateway from block events it has received. 127 | 128 | ## Scaling and load balancing 129 | 130 | Currently a client application is responsible for load balancing its requests between multiple peers. The Gateway will handle this on behalf of the set of currently connected client applications. 131 | 132 | Ideally, the Gateway itself will be stateless (i.e. not maintain client session state) allowing clients to be routed though an appropriate load balancer. 133 | 134 | ## Trust assumptions 135 | 136 | The Gateway is acting as a proxy between the client application and the peers/orderers. The signed transaction proposal is created in the client using one of the SDKs, and gateway passes that directly to the endorsing peers without any modification. If the client wishes, it can inspect the contents of the 'PreparedTransaction' message returned by the Endorse function prior to signing and submitting. The Gateway will partially unpack the proposal responses in order to add the chaincode function return value and the transaction id to the PreparedTransaction message. This allows the clients to access these commonly used values without the overhead of unpacking the deeply nested transaction 'Envelope' payload. 137 | 138 | Trusting clients can call the SDK's submitTransaction function to endorse/sign/submit in a single line of code for convenience. Users of the existing high level programming model are already comfortable with this procedure. 139 | 140 | ## Authentication and Authorization 141 | 142 | The Gateway itself will not add any message level authentication mechanism other than that already provided by the peers. As a proxy, authentication and authorization is handled by peer/orderer nodes exactly as if the client was connecting to them directly. 143 | 144 | Connection of the client applications to the Gateway can be secured by mutual TLS if required. 145 | 146 | The standalone Gateway does need to have an identity that allows it to connect to peers as a client in its own right to: 147 | - Do network discovery to obtain network topology and endorsement plans. 148 | - Listen to events to allow it to check for commit disposition of submitted transactions that it has routed to the ordering service on behalf of a client. 149 | 150 | ## Gateway SDKs 151 | 152 | A set of SDKs will be created to allow client applications to interact with the Fabric Gateway. The APIs exposed by these SDKs will be, where possible, the same as the current high-level 'gateway' SDKs. 153 | 154 | The following classes/structures will be available as part of the gateway programming model: 155 | - Gateway 156 | - Network 157 | - Contract 158 | - Transaction 159 | 160 | The existing `submitTransaction()` and `evaluateTransaction()` API methods will be implemented using the Gateway gRPC services described above. 161 | 162 | Based on user feedback of the existing gateway SDKs, the following additions are planned to be supported by this implementation: 163 | - Support for 'offline signing', allowing clients to write REST servers that don't directly have access to users' private keys. 164 | - Support for 'asynchronous submit', allowing client applications to continue processing while the transaction commit proceeds in the background. For Go this will be implemented by returning a channel; for Node by returning a Promise; and for Java by returning a Future. 165 | 166 | SDKs will be created for the following languages: 167 | - Node (Typescript/Javascript) 168 | - Go 169 | - Java 170 | 171 | It is proposed that these SDKs will be maintained in the same GitHub repository as the gateway itself (`hyperledger/fabric-gateway`) to ensure that they all stay up to date with each other and with the core gateway. This will be enforced in the CI pipeline by running the end-to-end scenario tests across all SDKs. 172 | 173 | Publication of SDK releases will be as follows: 174 | - Node SDK published to NPM. 175 | - Java SDK published to Maven Central. 176 | - Go SDK published to separate GitHub repo (e.g. fabric-gateway-go) and tagged. 177 | 178 | Contributors are invited to create SDKs for other languages, e.g. Python, Rust. 179 | 180 | 181 | # Drawbacks 182 | [drawbacks]: #drawbacks 183 | 184 | When run standalone, the gateway would add another process to an (arguably) already complex Fabric architecture. Also it will require its own signing identity in the channel writers policy in order to invoke the discovery service, which might be a security concern in some environments. These concerns are mitigated by embedding the gateway server within the organisation's existing peers. 185 | 186 | However, if embedded in an existing peer, then the gateway will not be able to be scaled independently of the peers. Also, there are scenarios where organisations don't run their own peer(s). In both of these cases, running a standalone gateway will be advantageous. 187 | 188 | # Rationale and alternatives 189 | [alternatives]: #alternatives 190 | 191 | - Why is this design the best in the space of possible designs? 192 | - What other designs have been considered and what is the rationale for not 193 | choosing them? 194 | - What is the impact of not doing this? 195 | 196 | # Prior art 197 | [prior-art]: #prior-art 198 | 199 | The Gateway SDK has been available as the high level programming model in the Fabric SDKs since v1.4. It has been thoroughly tested in many scenarios. This Fabric Gateway component is an embodiment of that programming model in a server component. 200 | 201 | # Testing 202 | [testing]: #testing 203 | 204 | In addition to the usual unit tests for all functions, a comprehensive end to end 205 | scenario test suite will be created from the outset. A set of 'cucumber' BDD style 206 | tests will be taken from the existing Java and Node SDKs to test all the major functions 207 | of the gateway against a multi-org multi-peer network. These tests will include: 208 | - evaluation and submission of transactions 209 | - event handling scenarios 210 | - transient / private data 211 | - error handling 212 | 213 | The CI pipeline will run the unit tests for the Gateway and its SDKs as well as the end-to-end scenario (cucumber) tests against all SDKs. 214 | 215 | # Dependencies 216 | [dependencies]: #dependencies 217 | 218 | - This will depend on the peer implementation in `hyperledger/fabric` and the protobuf definitions in `hyperledger/fabric-protos` repo. 219 | 220 | # Unresolved questions 221 | [unresolved]: #unresolved-questions 222 | 223 | ### Client inspection of proposal responses 224 | 225 | Fabric is designed such that the client should have the opportunity to inspect the transaction response payload and then make a decision on whether or not that transaction should be sent to the orderer to be committed. 226 | 227 | There is the possibility of a client being able to send proposals that are subsequently not submitted to the orderer and recorded on the ledger, either because: 228 | - The proposal is not successfully endorsed; or 229 | - The client actively chooses not to submit a successfully endorsed transaction. 230 | 231 | This is viewed by some as a potential security vulnerability as it allows an attacker to probe the system. 232 | 233 | Fabric Gateway should provide the capability for some kind of audit of submitted proposals, including their transaction ID so they can be reconciled against committed transactions. 234 | -------------------------------------------------------------------------------- /text/0000-fabric-website.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Fabric_Website 4 | nav_order: 3 5 | --- 6 | 7 | - Feature Name: Fabric Website 8 | - Start Date: (fill me in with today's date, 2020-08-14) 9 | - RFC PR: (leave this empty) 10 | - Fabric Issue: (leave this empty) 11 | 12 | # Summary 13 | [summary]: #summary 14 | 15 | This RFC details the proposal for a new website for using Hyperledger Fabric. This website would serve as a "front door" for the project, and collate the various resources available to help users get started with Fabric. 16 | 17 | The website has been deployed to Heroku for development purposes and is available to browse [here](https://warm-reaches-04614.herokuapp.com/). 18 | 19 | # Motivation 20 | [motivation]: #motivation 21 | 22 | Presently, there are multiple different places to get started with Hyperledger Fabric. It can be difficult for a new user to know which resource is the best one for them to start with based on what they would like to achieve. The available documentation has excellent content for getting started and explains everything well, but there is also a large amount of additional content on other relevant topics (for example, performance testing) that is also useful to new users. 23 | 24 | This website aims to surface this content and bring it with the existing documentation into one easy to consume place for users. 25 | 26 | There is an existing website at https://www.hyperledger.org/use/fabric that serves as a landing page for people interested in Fabric. However this has several limitations - namely that it is based on a template that is used for all Hyperledger projects, and is unable to be extended by the fabric maintainers. 27 | 28 | Other Hyperledger projects, such as [Sawtooth](https://sawtooth.hyperledger.org/), already have websites similar to what this RFC proposes, that collate all relevant resources into one place. Adding a website with a similar purpose for Fabric will serve as a "front door" to the project, and give it a more approachable and inviting feel. 29 | 30 | This website is for any user of Hyperledger Fabric, whether they are a developer of smart contracts or client applications, a network operator, or someone wishing to get involved in the Fabric community. It is for both new users wishing to get started with Fabric and existing users wanting to have a single place to access the resources they use most. 31 | 32 | The expected outcome for this project is to have a fully functioning website for our users, which is continually being updated with the latest and best resources. 33 | 34 | # Guide-level explanation 35 | [guide-level-explanation]: #guide-level-explanation 36 | 37 | The current proposed structure for the website is that it will be made up of four primary sections, separated by tabs. These sections are based on the different types of Fabric user that could use this website as a resource. The sections are: 38 | 39 | - An **Overview** tab, that will include general information about Hyperledger Fabric and what it is; 40 | - An **Operators** tab, that will include an explanation of the role of a network operator and links to resources this type of user might find most useful, such as the operations guide and deployment guide; 41 | - A **Developers** tab, that will include an explanation of the role of a developer and links to resources this type of user might find most useful, such as the VS Code extension and developer tooling, the commercial paper sample, and the latest chaincode and gateway SDK versions; 42 | - A **Community** tab, that will include links to the most relevant resources for contributing to Fabric, such as the main Fabric gitHub repository, the Jira dashboard, and the contributing guide. 43 | 44 | References and links common to all user types (StackOverflow, the Hyperledger Fabric wiki, social media profiles etc.) are included at the bottom of all tabs. 45 | 46 | By laying the content out in this way, it is hoped that users will be able to easily locate the documentation they need to successfully achieve their Fabric use case. 47 | 48 | 49 | # Reference-level explanation 50 | [reference-level-explanation]: #reference-level-explanation 51 | 52 | The proposed website will be implemented using React. It will be deployed to GitHub pages, after which it will be accessible at the link hyperledger.github.io/fabric-website. After deployment, the URL https://fabric.hyperledger.org/ will also redirect to hyperledger.github.io/fabric-website, similar to the existing website for Sawtooth. 53 | 54 | The website will not interact directly with other Fabric features or documentation. Instead, it is intended to serve as a "front door"; it should include links to all relevant resources, but not attempt to pull in or display their content directly. 55 | 56 | The content that is included in the website will consist primarily of descriptions of use cases and links to appropriate tutorials or documentation for those use cases. It will include links to multiple versions of the documentation so that users have the correct information for the version of Fabric that they are using. These links will need to be updated/added to when new versions of these resources are made available. 57 | 58 | -------------------------------------------------------------------------------- /text/0000-opentelemetry-tracing.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: OpenTelemetry Integration 4 | nav_order: 3 5 | --- 6 | 7 | - Feature Name: OpenTelemetry Tracing Integration 8 | - Start Date: 2021-01-12 9 | - RFC PR: (leave this empty) 10 | - Fabric Component: core, sdks 11 | - Fabric Issue: (leave this empty) 12 | 13 | # Summary 14 | [summary]: #summary 15 | 16 | This request for comments proposes integrating Hyperledger Fabric, its SDKs, core and chaincode components with the OpenTelemetry project. 17 | It introduces the concept of tracing, reporting the execution of chaincode, core components and SDKs to help correlate activities with the chain. 18 | 19 | 20 | # Motivation 21 | [motivation]: #motivation 22 | 23 | The OpenTelemetry project is a CNCF project that aims to standardize observability. 24 | OpenTelemetry is working to create a standard around representing metrics, traces and logs. 25 | This proposal aims to bring traceability of code execution in Hyperledger Fabric across peers, orderers and chaincode. 26 | 27 | 28 | # Guide-level explanation 29 | [guide-level-explanation]: #guide-level-explanation 30 | 31 | Observability comes from the DevOps world, and is defined in [Wikipedia](https://en.wikipedia.org/wiki/Observability) as "a measure of how well internal states of a system 32 | can be inferred from knowledge of its external outputs". 33 | Software that is observable exposes its behaviors by sending traces, which represent the path of execution of a particular request. 34 | Each trace can be represented as a set of spans that are correlated as child/parent or sequential relationships. 35 | Observable software is exposing metrics, representing the internal state of the components, as well as logs, emitted from the execution of the software. 36 | 37 | In Hyperledger Fabric, we rely on the OpenTelemetry framework to report traces and correlate them across services. 38 | 39 | In practice, this means a request made by a client connected via the Fabric SDK to a node can pass on the context of a trace. 40 | Peers and orderers propagate the trace context and create spans indicating their own interaction. 41 | 42 | Blockchain operators can reconstitute a graph of the interaction of all the components at play to create a service map. 43 | This helps uncover trends and issues of performance, as well as shortening the time it takes to investigate problems. 44 | It offers some security capabilities, such as detecting unexpected executions, or react quickly to performance changes. 45 | Traces also report as errors when components are unavailable. This allows for monitoring and alerting systems. 46 | 47 | Hyperledger Fabric developers can take advantage of those techniques with no code changes. 48 | Each SDK execution generates a top-level trace and will report to an endpoint provided by the environment. 49 | 50 | Developers may also create a trace before calling out to Fabric in their client code. 51 | The current trace information will be sent along with the message to the peer. 52 | 53 | # Reference-level explanation 54 | [reference-level-explanation]: #reference-level-explanation 55 | 56 | ## Message headers 57 | 58 | The trace information is passed in as an optional gRPC metadata header. 59 | 60 | ## Changes to client SDKs 61 | 62 | Client SDKs automatically create a trace to capture the call to the chain. 63 | 64 | The trace is created with the kind Span.Client. 65 | 66 | If a current trace exists, the new client trace adds it as its parent. 67 | 68 | The trace and span ID must be sent to the chain as a message header. 69 | 70 | The SDK should use the standard environment variable environment to let users define how and if they want to report 71 | trace data to an endpoint of their choosing. 72 | 73 | ## Changes to peers and orderers 74 | 75 | Peers and orderers capture and propagate trace information using an optional gRPC metadata header, if enabled. 76 | 77 | # Drawbacks 78 | [drawbacks]: #drawbacks 79 | 80 | OpenTelemetry is still relatively young, yet has reached maturity for traces support. 81 | 82 | The OpenTelemetry reporting system happens securely over Protobuf, with containers and client applications sending data. 83 | This requires that an OpenTelemetry-compatible endpoint is present to receive the data. 84 | 85 | # Rationale and alternatives 86 | [alternatives]: #alternatives 87 | 88 | This design allows full observability of Hyperledger Fabric, to a degree of detail that will help developers understand 89 | the impact of their deployment topology and organize operations using the latest framework. This allows Hyperledger Fabric 90 | to report meaningful data just like cloud-native applications. 91 | 92 | Alternatively, developers can develop their own homegrown designs to report data along the way, or use logs only 93 | to understand how the system performed and investigate issues. 94 | 95 | # Prior art 96 | [prior-art]: #prior-art 97 | 98 | We have [brought a similar design to Hyperledger Besu](https://github.com/hyperledger/besu/pull/1557), 99 | where we started adding tracing to the HTTP JSON-RPC service 100 | used by clients to communicate with the node. We also created traces in critical processes throughout the client 101 | to understand the performance of the code running there. We created a complete application that can report the state 102 | of the client and the health of its internal mechanisms. 103 | 104 | We also have published a [webinar with the OpenTelemetry project](https://www.cncf.io/webinars/observability-of-multi-party-computation-with-opentelemetry/) 105 | showcasing how Hyperledger Fabric can rely on OpenTelemetry to report the state of execution and 106 | eliminate the black box feeling operators contend with. 107 | 108 | 109 | # Testing 110 | [testing]: #testing 111 | 112 | In addition to the current integration testing scenarios, it will be required to test the system alongside an instance 113 | of the OpenTelemetry collector collecting traces from the client applications. 114 | The OpenTelemetry instance can report to a zipkin application all this information so testers can verify the correctness of the traces published. 115 | 116 | # Dependencies 117 | [dependencies]: #dependencies 118 | 119 | None. 120 | 121 | # Unresolved questions 122 | [unresolved]: #unresolved-questions 123 | 124 | -------------------------------------------------------------------------------- /text/0000-orderer-snapshot.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Orderer ledger snapshots 4 | nav_order: 3 5 | --- 6 | 7 | - Feature Name: OSN_snapshots 8 | - Start Date: 2021-03-23 9 | - RFC PR: 10 | - Fabric Component: orderer, ledger 11 | - Fabric Issue: 12 | 13 | # Summary 14 | [summary]: #summary 15 | 16 | The RFC introduces ledger snapshot feature at orderers, similar to the same feature at peers. 17 | 18 | # Motivation 19 | [motivation]: #motivation 20 | 21 | Real world use cases show the necessity of controlling the disk space consumed by block ledger, and this RFC is mainly motivated by the need to reduce space consumption at orderer nodes. 22 | Additionally this RFC presents a better way to join a channel without pulling the huge number of all historical blocks, which also may take significant time and may pose a problem for scaling of an existing Fabric network. 23 | 24 | # Guide-level explanation 25 | [guide-level-explanation]: #guide-level-explanation 26 | 27 | This RFC introduces a notion of a **ledger snapshot for OSNs** (ordering service nodes). With this notion two new logical operations appear at the orderer REST API and in the CLI tool *osnadmin*: 28 | - Create a snapshot, 29 | - Join channel from a snapshot. 30 | 31 | Ledger snapshot is a set of files containing all the necessary information for OSN to join a channel from an arbitrary block in the chain without loss of any functionality except the ability to deliver an earlier block. 32 | 33 | A **local orderer administrator** is a role that was introduced in RFC-0004 "Channel participation API without system channel" and is currently implemented with the help of "Admin Configuration" section in the OSN YAML config file (). Only the identity associated with this role is able to invoke the API defined in this RFC, that is existing **channel participation API** and newly defined **ledger management API**. 34 | 35 | ## Guide: Create snapshot 36 | 37 | To create a ledger snapshot use the following REST API (optional block_number should be supplied as a parameter in the request body): 38 | 39 | POST /ledger/v1/{channelID}/snapshot 40 | 41 | or the following CLI command: 42 | 43 | $ osnadmin ledger snapshot -c [-b ] 44 | 45 | If the optional parameter is omitted, snapshot will be created from the latest block, i.e. from the block number equal to ledgerHeight-1. 46 | 47 | A process of snapshot creation will start immediately if the block number is omitted, or if the block number is specified and is less than current ledger height. If the block number is specified and is in future (>= ledger height) then the operation will immediately fail with a reasonable error message. 48 | 49 | The process of snapshot creation is a synchronous one; upon the successful completion of the "ledger snapshot" API call the snapshot itself is ready and can be accessed in the following directory in the file system: 50 | ``` 51 | /completed// 52 | ``` 53 | 54 | ## Guide: Join channel from a snapshot 55 | 56 | To join a channel from a snapshot use the following REST API (files from the snapshot must be placed in the body, using multipart/form-data content-type): 57 | 58 | POST /participation/v1/joinbysnapshot 59 | 60 | or the following CLI command: 61 | 62 | $ osnadmin channel joinbysnapshot -s 63 | 64 | There is no need to specify {channelID} when using the above API, as it will be extracted from the snapshot content. 65 | 66 | To check the status of the "join by snapshot" operation use the following REST API: 67 | 68 | GET /participation/v1/channels/{channelID} 69 | 70 | or the following CLI command: 71 | 72 | $ osnadmin channel list -c 73 | 74 | ## Guide: Prune block ledger of an existing channel 75 | [guide-prune-block-ledger-of-an-existing-channel]: #guide-prune-block-ledger-of-an-existing-channel 76 | 77 | To prune an existing channel ledger do the following: 78 | - create a new snapshot at the desired block number (using "ledger snapshot" API), 79 | - remove the channel from the OSN (using "channel remove" API), 80 | - join the channel from the snapshot (using "channel joinbysnapshot" API). 81 | 82 | This approach has some drawbacks, see [Drawbacks section](#drawbacks). 83 | 84 | # Reference-level explanation 85 | [reference-level-explanation]: #reference-level-explanation 86 | 87 | ## Changes to the existing API 88 | 89 | This RFC adds a new endpoint to the existing **channel participation API** to allow joining a channel by snapshot. Additionally this RFC defines new **ledger management API** and introduces a new endpoint of this API to allow snapshot creation. In future we plan to add at least one more endpoint to this API which will enable ledger pruning "on the fly" without interruption in OSN service to resolve the known [drawbacks](#drawbacks). 90 | 91 | Technically we achieve the above by following these steps: 92 | 1. Introduce new REST API endpoint **/participation/v1/joinbysnapshot** while keeping all existing functionality of the channel participation API, that is keeping "join by join-block", "remove channel", and "list channel(s)" API endpoints. 93 | 2. Add new REST API handler for the ledger management API with prefix **/ledger/v1/**. Define a single endpoint of this API **/ledger/v1/{channelID}/snapshot** to allow snapshot creation. 94 | 3. Add "Ledger Management API Configuration" section in the OSN YAML config file. 95 | 4. Add support of two new snapshot operations to the CLI tool *osnadmin*, specifically the commands *osnadmin channel joinbysnapshot* and *osnadmin ledger snapshot*. 96 | 97 | ## Snapshot content 98 | 99 | We propose the OSN snapshot to follow the same logical structure as peer snapshot, that is to comprise a directory with a number of files in it, one of the files being a metadata JSON file. But here the similarity ends, as the content of the files will be drastically different. 100 | Here is the content of the OSN snapshot: 101 | 1. Snapshot version - to allow for the future extensions. Current version is 1. 102 | 2. Channel ID. 103 | 3. lastBlockNum - the number of the last pruned block. 104 | 4. last config block before lastBlockNum - this is required to verify the newly pulled blocks. The sequence number of the last config block must be lower or equal to lastBlockNum. 105 | 5. lastBlockHash - the hash of the last pruned block. 106 | 6. prevBlockHash - technically not required, but this will make internal structure BlockchainInfo complete. 107 | 7. ConsenterMetadata of the last pruned block (lastBlockNum) 108 | 109 | Version 1 of the OSN snapshot described in this RFC defines three files as the content of the snapshot directory: 110 | 1. Last config block for the channel before lastBlockNum. Content: The block data with block metadata removed completely, in protobuf binary serialization format. Name: "lastConfig.block" 111 | 2. Value of OrdererBlockMetadata.ConsenterMetadata of the last pruned block (note: this is coming not from the last config block, but rather from the last pruned block, that is from the block with sequence number lastBlockNum). Content: ConsenterMetadata value in protobuf binary serialization format. Name: "ConsenterMetadata.pb" 112 | 3. Metadata file containing all the rest of the snapshot content. Name: "metadata.json" 113 | 114 | Version 1 of the OSN snapshot defines the following JSON fields in the "metadata.json" file: 115 | ```json 116 | { 117 | "version": 1, 118 | "channelID": string, 119 | "lastBlockNum": number, 120 | "lastBlockHash": string, 121 | "prevBlockHash": string | null, // null is valid only when lastBlockNum == 0 122 | "parts": [ // array of all the files in the snapshot 123 | "lastConfig.block", 124 | "ConsenterMetadata.pb" 125 | ] 126 | } 127 | ``` 128 | 129 | Unlike peer snapshot the OSN snapshot does not contain state DB, private data configs, the list of TxIDs and commit hash - as the OSN is purposefully designed to be unaware of these. So these two kinds of snapshots (peer snapshot and OSN snapshot) bear resemblance only on the conceptual level, being completely different in the content. 130 | 131 | ## Future-compatibility of the "join by snapshot" API 132 | 133 | 1. REST API future-compatibility. 134 | To maintain compatibility with the future modifications of the snapshot format, this RFC mandates the following: 135 | 136 | 1.1. The POST request to **/participation/v1/joinbysnapshot** endpoint must use multipart/form-data content-type. 137 | 138 | 1.2. First part of the multipart body must have the name "metadata.json" and must always contain "metadata.json" file. 139 | 140 | 1.3. API handler must read the "metadata.json" part first, confirm the validity of the JSON format and extract the fields defined for the version 1. Additional fields may be present in future snapshot versions; the presence of the additional fields not defined in version 1 format should not prevent the process from successful completion. 141 | 142 | 1.4. API handler checks the validity of all the metadata fields, ignoring the fields not defined in version 1. In case the validity check fails, the API handler must fail immediately with an appropriate error. 143 | 144 | 1.5. After the successful check of the metadata, the handler searches for two more parts with the names "lastConfig.block" and "ConsenterMetadata.pb" containing the respective files. Additional body parts may be present in future snapshot versions; the presence of the additional body parts not defined in version 1 format should not prevent the process from successful completion. 145 | 146 | 1.6. API handler checks the validity of the found "lastConfig.block" and "ConsenterMetadata.pb" files. In case the validity check fails, the API handler must fail immediately with an appropriate error. Otherwise it proceeds with the "join by snapshot" process. 147 | 148 | 2. CLI tool *osnadmin* future-compatibility. 149 | When processing *osnadmin channel joinbysnapshot* command, the following checks must be performed: 150 | 151 | 2.1. Firstly, the CLI tool should find the file "metadata.json" in the specified directory . 152 | 153 | 2.2. Next it must confirm the validity of the JSON format and the presence of all metadata fields defined in version 1. Additional fields may be present in future snapshot versions; the presence of the additional fields not defined in version 1 format should not prevent the process from successful completion. 154 | 155 | 2.3. Next, the CLI tool checks the validity of all the metadata fields, ignoring the fields not defined in version 1. In particular all the files defined in "parts" object must be present. In case the validity check fails, the process must fail immediately with an appropriate error. 156 | 157 | 2.4. Lastly, the CLI tool creates POST request to **/participation/v1/joinbysnapshot** endpoint and places all the files defined in the "parts" object of the metadata JSON into the body of the request according to the REST API specification. Name of each part must be the name of the file as defined in "parts" object of the metadata JSON. First part must be the "metadata.json" itself. 158 | 159 | ## Implementation considerations 160 | 161 | When a channel is joined from a snapshot by "channel joinbysnapshot" command, we have to save the following info on the block storage: 162 | 1. Last config block from snapshot 163 | 2. BootstrappingSnapshotInfo 164 | 165 | For BootstrappingSnapshotInfo persistence the implementation is already there, but for the last config block a new persistent unit of storage (a file in case of file-based ledger) should be created. 166 | A special attention should be paid to the use case of **Deliver** service being asked to deliver a single block with the number equal to the block number of the last config block from a snapshot - in this case the delivery request must succeed by returning the last config block from the new persistent unit of storage (from a separate file) and not from the continuous block ledger. In general, the ledger storage interface should be aware of a possibility that aside from a continuous block ledger there is a special case of a single block representing a last config block from a snapshot which should also be successfully returned when requested by block number. 167 | Additionally all the procedures inside the OSN codebase looking up the last config block in the OSN ledger storage should be modified to return the last config block from the snapshot in case the last config block and/or a last block is not found in the block ledger itself. 168 | 169 | The channel bootstrapping from snapshot must be crash-consistent, that is if a crash takes place while bootstrapping a channel, upon restart the orderer, either the bootstrapped channel exists in the usable state or not at all. In particular this means that the creation of the above two pieces of information (last config block and BootstrappingSnapshotInfo) must be an atomic transaction, and there should be no possibility after a crash to have one of them but not another. 170 | Possible implementations of such an atomic transaction may be keeping both pieces of information in the same file and creating the file atomically, or keeping them in separate files in the same directory and creating the directory atomically. 171 | 172 | ## System channel co-existence 173 | 174 | The following API can work both with system channel and without it: 175 | - "channel list" - existing API, keep existing behaviour 176 | 177 | The following APIs can work only without system channel, will fail in presence of system channel: 178 | - "channel joinbysnapshot" 179 | - "channel remove" - existing API, keep existing behaviour 180 | - "channel join" - existing API, keep existing behaviour 181 | - "ledger snapshot" 182 | 183 | ## New channel creation ("genesis" snapshot) 184 | 185 | A new channel can be created using the existing API "channel join" with a genesis block as a join-block. 186 | 187 | Theoretically a "genesis" snapshot also can be constructed. For this a genesis block should be used as a last config block and lastBlockNum should be set to 0. This RFC does not define any specific tool capable of creation of a "genesis" snapshot, instead this RFC mandates that such a snapshot should be accepted as a valid one. In particular a snapshot should be considered as valid if prevBlockHash is omitted from the metadata JSON file or is set to "null", but only when lastBlockNum is equal to 0. 188 | 189 | ## A **follower** mode 190 | 191 | Exactly as when using "channel join" API the orderer bootstrapping from a snapshot should start in the **follower** mode pulling the blocks without processing them until it finds a config block confirming its membership in the channel, unless it can be established that it is a member of the channel already (a member of the consenter set in case of RAFT). 192 | 193 | # Drawbacks 194 | [drawbacks]: #drawbacks 195 | 196 | There are three main drawbacks in the proposed ledger pruning method (see [Guide: Prune block ledger of an existing channel](#guide-prune-block-ledger-of-an-existing-channel)): 197 | - an interruption in the service of the orderer is required, 198 | - this works only in the absence of system channel, because "channel joinbysnapshot" works only in the absence of system channel. 199 | 200 | To resolve the above problems a new RFC will be issued introducing a separate **Prune** endpoint to the **ledger management API** which will allow to prune the ledger without stopping the service and will work both in presence and in absence of system channel. 201 | 202 | ## Admin risks 203 | 204 | Allowing ledger pruning in OSNs introduces a new risk at the network level. It is possible to end up in a situation when all orderers are pruned at some point and the peer snapshot above that point is not accessible. In this situation it is impossible to bootstrap a new peer. 205 | We believe that the benefits of the ability to manage disk space outweigh the possible risk; nevertheless the admins should apply extreme caution when pruning OSNs. 206 | Possible solutions and the network admin level may be: 207 | - always keep at least one orderer un-pruned; 208 | - ensure the existence of a peer snapshot above the lowest pruning point of all orderers. 209 | 210 | # Rationale and alternatives 211 | [alternatives]: #alternatives 212 | 213 | ## Alternatives considered 214 | 215 | - Implement only pruning without the snapshot functionality. 216 | This still requires storing some information about the history of the ledger, as a bare minimum we need the last config block and the hash of the last pruned block. So why should we invent a new mechanism to support pruning, we can as well use the snapshot format to store the required info. 217 | - Store the information about the first known block in a new field (firstBlockNum) somewhere in the blockStore or fileLedger structure. 218 | Again, we already have a code implemented that stores this info in BootstrappingSnapshotInfo structure. It is better to rely on the already tested code to achieve the same functionality. 219 | 220 | # Rationale 221 | 222 | Strictly speaking to achieve the main goal - orderer ledger pruning - we don't need the snapshot functionality. But due to the implementation reasons the notion of snapshot arises in some or another way. For example we need to store somewhere the last config block and the number of the last pruned block (or the number of the first block existing in the pruned ledger). So we decided that it is better to rely on the already existing code supporting snapshots and slightly modify it to suit the needs of the OSN snapshot. 223 | 224 | In addition to the above, joining a channel from a snapshot solves many problems with the existing join-block implementation from RFC-0004 "Channel participation API without system channel", namely: 225 | - the need to pull potentially huge number of the historical blocks 226 | - the fact that the config in the join-block may be outdated and not representing the latest config 227 | 228 | # Prior art 229 | [prior-art]: #prior-art 230 | 231 | RFC-0000 "Ledger Checkpointing" 232 | 233 | RFC-0004 "Channel participation API without system channel" 234 | 235 | # Testing 236 | [testing]: #testing 237 | 238 | Integration tests will cover the following scenarios: 239 | 240 | 1. Snapshot generation 241 | - start with a set of OSNs, with a channel, with blocks 242 | - create a snapshot from the same non-genesis block on all the OSNs 243 | - verify that the snapshot generated across OSNs is exactly same 244 | 2. Bootstrap: 245 | - start with a set of empty OSNs 246 | - construct a "genesis" snapshot 247 | - join all to the application channel from the "genesis" snapshot 248 | 3. Add new channel: 249 | - start with a set of OSNs with a channel 250 | - construct a "genesis" snapshot for the second app channel 251 | - join a subset of OSNs to the second channel from the "genesis" snapshot 252 | 4. Add new OSN to the channel: 253 | - start with a set of OSNs, with a channel, with blocks 254 | - create a snapshot from some non-genesis block 255 | - add the new OSN to the channel from the snapshot: 256 | - as a follower (not in consensus set), adding it after catch-up 257 | - as a member (in consensus set) 258 | 5. Add new channel without genesis block: 259 | - start with a set of OSNs, with a channel, with blocks, with a non-genesis config block among them 260 | - create a snapshot from some block after the last config block 261 | - remove the OSN from channel 262 | - start a set of additional OSNs 263 | - add all OSNs to the channel from the created snapshot 264 | 6. Snapshot creation after bootstrapping: 265 | - start with a set of OSNs, with a channel, with blocks, with a non-genesis config block among them 266 | - create a snapshot from some block after the last config block 267 | - start a new OSN, no channels 268 | - new OSN joins the channel from the snapshot 269 | - add some blocks to the channel without a config block 270 | - trigger snapshot creation 271 | - check the correctness of the created snapshot 272 | - add a config block and some more blocks to the channel 273 | - trigger snapshot creation 274 | - check the correctness of the created snapshot 275 | 7. System channel restrictions: 276 | - start with a set of OSNs with the system channel 277 | - make sure that the following operations fail: 278 | - "channel joinbysnapshot" 279 | - "ledger snapshot" 280 | - create an app channel with blocks 281 | - make sure that the following operation succeeds: 282 | - "channel list" 283 | - remove system channel 284 | - make sure that two operations that previously failed now succeed 285 | 8. Failure during bootstrapping the OSN from a snapshot 286 | - start "channel joinbysnapshot" operation 287 | - kill the OSN during the bootstrap 288 | - restart the OSN 289 | - verify that the OSN can again join the channel with the same snapshot 290 | - verify that the OSN can again join the channel with a different snapshot 291 | 292 | # Dependencies 293 | [dependencies]: #dependencies 294 | 295 | **RFC-0004 "Channel participation API without system channel"** 296 | 297 | This RFC relies on the **channel participation API** introduced in RFC-0004 and extends it for the use case of joining a channel from a snapshot without breaking compatibility with the existing functionality of the channel participation API. 298 | 299 | # Unresolved questions 300 | [unresolved]: #unresolved-questions 301 | 302 | ## Questions to be resolved during the implementation 303 | 304 | - Should we introduce a new return code to **Deliver** call when a block before the snapshot start is requested? 305 | 306 | It appears that the current codebase already returns a reasonable error code “NOT_FOUND“ with the log record “Error reading from channel, cause was: NOT_FOUND“ which indicates that the channel is there, but the block was not found. Because of this we do not include a new error code for this particular use case in the RFC. 307 | 308 | ## Questions to be resolved in future RFCs 309 | 310 | - The problem of channel divergence that was pointed out in RFC-0004 "Channel participation API without system channel" is still unresolved. 311 | 312 | It is possible to join the channel with the same channelID by using the snapshots created from different config blocks with the same block number. 313 | 314 | - How to resolve a situation when all the orderers are pruned and a peer tries to fetch a block below the pruning point? 315 | 316 | The peer will receive an error code “NOT_FOUND“, but the problem is that it will not be able to proceed further and will be basically stuck at this point. This is something that has to be resolved on the network admin level and the exact resolution is unclear at this point. 317 | -------------------------------------------------------------------------------- /text/0000-private_data_purge.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Private Data Purge 4 | nav_order: 3 5 | --- 6 | 7 | - Feature Name: private_data_purge) 8 | - Start Date: 2021-10-01) 9 | - RFC PR: (leave this empty) 10 | - Fabric Component: chaincode, ledger 11 | - Fabric Issue: (leave this empty) 12 | 13 | # Summary 14 | [summary]: #summary 15 | 16 | The private data collections feature in Fabric allows the dissemination of the sensitive data to a subset of organizations in a channel. A peer that is eligible to receive the private data of a collection within a namespace, maintains all the versions of the private data in its private data store forever. This RFC proposes a new operation *purge* on private data via chaincode shim that allows the deletion of one or more private data keys from the current state (statedb), and in addition, purging of all the historical versions of the supplied keys from the private data store permanently. In addition, it lays down a basic groundwork for enabling history-related queries on private data. 17 | 18 | # Motivation 19 | [motivation]: #motivation 20 | 21 | The primary motivation of this feature is to enable users to implement right to forget in the context of private data in a Fabric application. This may help users to meet certain business compliance about data privacy. At present, there exists two ways to retrieve historical versions of private data. An end user can retrieve historical versions of private data from a peer using the block Deliver service (`DeliverWithPrivateData` API). Additionally, other peers, potentially from other organizations, can access to the historical versions via the private data reconciliation process. Further, in future, we may expose an API for users to access the history of a specific private key in a collection, similar to the existing API for accessing the history of a key in the chaincode namespace. By enabling the purging of desired private data, the proposed feature makes peer prevent sending private data out from these APIs and eventually deletes it permanently from its persistent stores. An added advantage of this feature is that it enables users to free up some storage space by purging the historical versions of private data. 22 | 23 | # Guide-level explanation 24 | [guide-level-explanation]: #guide-level-explanation 25 | To enable private data purge feature, we introduce a new API in chaincode shim. Below is the function signature of this new API. 26 | 27 | **`PurgePrivateData(collection, key string) error`** 28 | 29 | As far as the latest state of a channel, maintained in statedb, is concerned, this new operation, `PurgePrivateData`, is no different from existing operation, `DeletePrivateData`. Hence, an application or chaincode would not notice any difference in behavior. The additional semantics associated with `PurgePrivateData` primarily impact private data store and reconciliation of private data. Both of these are not exposed to end users as of now (beyond the APIs mentioned above). 30 | 31 | 32 | For understanding the usage of this API, lets assume a scenario where a chaincode defines a private data collection and the collection contains personally identifiable information for customers such that the chaincode maintains PII info of a customer in the form of three keys, namely {CustomerID}_PII_Name, {CustomerID}_PII_DOB, and {CustomerID}_PII_Address. While name and date-of-birth may not change, a customer may have updated his address a few times and hence peers of eligible orgs would have persisted all the historical values for the address. Now, the customer may want the Fabric network to purge any of its PII data. 33 | 34 | In order to achieve this, the application may invoke a chaincode transaction in which the chaincode in turn invokes the *PurgePrivateData* API three times - once for each of the above mentioned PII keys. Finally, the application would get enough endorsements to satisfy the endorsement policies for each of the above keys and submits the transaction to the orderers. Once the transaction is committed on a peer, the peer stops sending out any version of any of these private data keys and eventually deletes from the persistent stores. 35 | 36 | Note that we will gate this new API behind a capability tag so that this new operation is interpreted on all the peers in a channel and assures the user that the data will be purged if the transaction is committed as a valid transaction. In the absence of the capability, the endorser will return error during the invocation of `PurgePrivateData`. As this is a newly added function, it is sufficient to gate it at the endorsing time that ensures that no transaction with purge operation gets committed before enabling the capability. 37 | 38 | # Reference-level explanation 39 | [reference-level-explanation]: #reference-level-explanation 40 | 41 | Before diving into the proposed design, we describe some background, challenges, and broad requirements for the desired behavior for implementing private data purge. 42 | 43 | ## Background, challenges, and Requirements 44 | We divide this section into two subsections. 45 | 46 | ### Deletion of private data from statedb and private data store 47 | In regard of actual deletion of private data keys specified in a purge transaction, following is the desired behavior of a peer. 48 | - Deletion of the private data key from the state should be done atomically with the other state updates caused by the transaction. In other words, as far as state updates are concerned from the perspective of future transaction simulation, the *purge* operation is no different from *delete* and merely deleting from hashed state is sufficient at commit time. 49 | - While the hashed key must be deleted from state atomically, the private key itself could be deleted in the background (not atomically with commit). To keep the implementation simple however, as a first implementation the private key will also be deleted atomically at commit time. 50 | - The deletion of all the historical versions from the private data store is expected to be time consuming. Moreover, it does not have any bearing on the transaction execution. So, in the interest of not directly impacting the transaction throughput, it is desired to perform this operation in the background 51 | - Despite performing the deletion from private data store in the background, the peer should behave for all external communication as if the data is actually purged, i.e., not send out the data in API responses after the purge transaction has been committed. 52 | - In some scenarios, when an org is removed from a collection, the peer of the org stops receiving future private data operations for the collection. This very well includes the private write-set that includes the purge operation itself. However, such peers may already have the historical versions of private data and latest of those versions in its private state. So, it poses a requirement for performing the deletion of private key versions both from the private data store and from the private key, while working with the hashed write-set, which is delivered to all the peers, as part of transaction inside the block. 53 | 54 | ### Impact and desired changes on private data reconciliation 55 | Most impacted function from this feature would be [private data reconciliation](https://hyperledger-fabric.readthedocs.io/en/latest/private-data-arch.html#private-data-reconciliation). As a recap, the private data reconciliation enables an eligible peer to retrieve latest and historical versions of private data from other eligible peers. The reconciliation is triggered by the requesting peer when the peer has some versions of private data missing in its private data store. This missing data situation is caused by one of the following situations. 56 | 57 | 1) Private data could not be delivered to the peer along with the block at the block commit time 58 | 2) A peer (org) is added to an existing collection, so peer catches up by pulling all historical versions of private data 59 | 60 | In the current implementation of reconciliation, the basic unit of data transfer in the reconciliation is a tuple i.e., the entire private write-set of a collection within a transaction is what the requesting peer asks for from another eligible peer. The requesting peer, before accepting the private write-set from the replying peer, verifies the contents by matching the hash of the entire private write-set with the expected hash for this collection that is included in the block. However, when we implement the private data purge feature, the replying peer may send a subset of the private write-set as some of the keys may have been purged. This change in behavior motivates the following changes in the reconciliation process. 61 | 62 | - The replying peer filters the already purged keys before sending out the response to reconciliation request 63 | - For example, if a private key purge transaction is committed at block number 100, the replying peer will remove the private key from the response if the version of the key being sent was committed in a block prior to block number 100. 64 | 65 | - The requesting peer filters the already purged keys 66 | - In some scenarios, the replying peer of private data historical version may be behind the requesting peer in the terms of block chain height. For example, assume that a private key purge transaction is committed at block number 100. The requesting peer is at height 101 and seek to retrieve private data corresponding to block 50. The replying peer is at height 60, i.e., it can deliver the requested private data but has not yet seen the purge transaction. In this case, the requesting peer needs to trim the received write-set by removing the purged keys that it is aware of before persisting the write-set. 67 | 68 | - The requesting peer accepts and verifies subset of the collection 69 | - Contrary to the above situation, the requesting peer may be behind and may not be aware of the purge transaction yet. While, the replying peer sends the subset of the collection write-set. In this scenario, the requesting peer needs to verify the subset by comparing the hashes of the individual keys (as opposed to the hash of the entire collection write-set). Moreover, the requesting peer cannot remove the collection from its list of still missing data, until the requesting peer itself sees the purge transaction, as the replying peer may be playing malicious by deliberately sending a selective subset. 70 | 71 | 72 | Keeping the above requirements and desired behavior in consideration, below is the the proposed design. First, we describe the new key types that we would maintain in the private data store and then, we discuss various operations in Fabric that would create, update, read, or delete these keys to implement this feature. 73 | 74 | ## Proposed Design 75 | We divide the proposed design in four subsections. First, we describe the changes in the Write-set Proto message. Second, we introduce two new key types that we would maintain in the private data store. Third, we describe the proposed changes in the existing functions and finally, we describe the new functions that we would add. 76 | 77 | ### Changes in Write-set Proto 78 | We capture the Hashed write-set for a particular key in the proto message [`KVWriteHash`](https://github.com/hyperledger/fabric-protos/blob/9c69228417592158899cb0dd4a77a3eedf25225f/ledger/rwset/kvrwset/kv_rwset.proto#L57). In the proposed design, We would extend this proto to capture the purge operation. We do this by adding an additional field `is_purge`. 79 | - Note that unlike existing fields in the hashed-write-set, this field will not have any corresponding field in the raw private-write-set. 80 | - Also, when we set `is_purge` to true, as a result of invocation of the newly introduced chaincode shim API, we also set the existing field `is_delete` set to true both in the hashed-write-set as well as in the private-write-set. In the terms of semantics, we interpret `is_delete` as the instruction for deleting the key from the latest state while `is_purge` indicates purging from the private data store. 81 | 82 | ### New Keys in Private data store 83 | In this design, we maintain two types of additional keys in the private data store. We refer to these new types of keys as *PurgeMarker* and *HashedIndex*. Below, is the format and intended use of these keys. 84 | 1) PurgeMarker 85 | A PurgeMarker key is intended to record a purge operation for a key. The design of this marker key would be `Ns_Coll_Hash(PrivateKey)`, while the marker value would contain `CommittingBlkNum_CommittingTxNum`. For illustration, assume that a purge transaction appears in block number 100, at position 20 that specifies for purging the keys `key1` and `key2` under collection `myColl` for chaincode `myCC`. When processing this transaction, we insert the two keys into private data store as `myCC_myColl_hash(key1)` with value `100_20` and `myCC_myColl_hash(Key2)` with value `100_20`. Note that the separation of different components of a key and value shown by an underscore is just for the illustration. For the reasons discussed later, we intend to keep these marker keys forever in the initial implementation. The assumption is that the purge operation is expected to be relatively less frequent, and hence, it should not be a concern from storage space point of view. We may revisit this assumption however after an initial implementation and assessment. 86 | 87 | - Although not required, we can also create a PurgeMarker record at the collection level to serve as an optimization in the implementation to efficiently identify whether any key has ever been purged for a given collection. 88 | 89 | 2) HashedIndex 90 | A HashedIndex key is intended to act as an index for the data keys in private data store, so that when a key is eventually purged we know from where to purge it in the private data store. The format of this key would be `Ns_Coll_Hash(PrivateKey)_blkNum_txNum`. For each version of a private key in the private data store, we would maintain such an index key. For illustration, assume that a key `key1` under collection `myColl` in chaincode `myCC` is inserted at block 100, transaction 10. Later this key is updated in block 200, transaction 20 and finally, the key is deleted in block 300, transaction 30. In this case, today, we maintain in the private data store all three private write-sets that contain, potentially along with other, the above three mentioned operations. In this example, for `key1`, we would maintain three HashedIndex keys as `myCC_myColl_hash(key1)_100_10`, `myCC_myColl_hash(key1)_200_20`, and `myCC_myColl_hash(key1)_300_30`. The value for the HashedIndex key will be the private key pre-image itself, to serve as a reference for which private keys can be purged from the peer for a given hashed key in a transaction write-set. For instance, in the above two examples, the values would be `key1` and `key2` respectively. 91 | 92 | - Note that the format of this key look same as the format of the PurgeMarker key - however, we always prepend a unique byte for the key type so, no two keys of different type ever clash. 93 | 94 | - Unlike the PurgeMarker key, we delete an HashedIndex key when we delete the corresponding data, either because of newly introduced explicit purge operation in this RFC or because of an implicit purge operation caused by [expiry of private data](https://hyperledger-fabric.readthedocs.io/en/release-2.2/private_data_tutorial.html#purge-private-data). 95 | 96 | - An important point to note here is that we would need to create these index keys retroactively - i.e., when a peer is upgraded to a version that would introduce private data purge feature, as a part of the upgrade, we would create these index keys. 97 | 98 | ### Changes in existing operations 99 | 100 | #### Block commit 101 | In the current code, when computing the state updates for a block, for each updated key, we take the latest value (i.e., the value set by the highest valid transaction in the block for the key). With the purge private data feature, while we still compute the state updates as is but, in addition, we perform following two data-set for committing to the private data store. 102 | - Compute a set of PurgeMarker keys such that the set contains one entry for each hashed-key that is marked for purge by at least one transaction in the block. Further, if there exist more than one transactions that purge the same key, we ignore the lower transaction for the key. For example, assume block 100 contains transaction 10 that purges the key `key1` and `key2` and transaction 15 that purges the key `key1` and `key3` for collection `myColl` in chaincode `myCC`. In this case the set of PurgeMarker keys would look like `myCC_myColl_hash(Key1)` (with value `100_15`), `myCC_myColl_hash(Key2)` (with value `100_10`), and `myCC_myColl_hash(Key3)` (with value `100_15`). 103 | - Compute HashedIndex keys for each private key-value present in the private write-set. 104 | 105 | 106 | #### Private data commit via reconciliation 107 | As mentioned previously, private data reconciliation is going to be most affected of the existing functions. Below is a list of changes that we would make in the reconciliation. 108 | 109 | 1) When we commit private data via reconciliation, the requesting peer may receive the keys that have already been purged. This would be the case when the replying peer has not yet seen the purge transaction. However, the requesting peer may have seen the purge transaction. In order to not save the purged keys, we would first compute the expected private write-set by removing the already purged keys on the requesting peer from the hashed write-set from block, by intersecting with PurgeMarker keys. This implicitly means that we need to maintain PurgeMarker keys forever (theoretically, at least, until no more missing data, including the missing data for ineligible collections, exists below the committing block of PurgeMarker key). 110 | 111 | 2) In the current code, during reconciliation, a requesting peer verifies the data by computing the hash of the private write-set received against the hash of the collection present in the corresponding transaction in the block. On a side note, if a peer has been bootstrapped via a snapshot, this is true only for the private data that has been added after the height of the bootstrapping snapshot. Now, in the presence of the private data purge feature at a key level, a peer may receive a partial private write-set, because one or more keys in the private write-set may have been purged by the replying peer. Hence, it is required for the requesting peer to perform the verification of hashes at the level of individual key-value pairs. 112 | 113 | 3) During reconciliation, a requesting peer may receive a subset of the expected write-set, as computed in (1) above, including an empty write-set, because the requesting peer may yet not have seen the purge transaction while the replying peer has seen it. To handle this, the requesting peer accepts the subset of the expected write-set and puts the missing data request into the deprioritized list and is attempted again at a low frequency interval. Eventually, the requesting peer catches up to block height and executes the purge transaction and gets aware of the purge transaction. Finally, when the expected keys, as computed in (1) above, becomes the empty-set, the peer updates the missing keys info as fully reconciled. 114 | 115 | #### Queries on Private Data Store 116 | The queries need to filter the private write-sets that it sends out as a response to a query. This is intended to be achieved with the help of PurgeMarker key, if the version of the key is lower than the version present in one of the corresponding PurgeMarkerKey keys. This is required because the actual data purge may take some time after having created the marker keys. 117 | 118 | 119 | ### New Operations 120 | 121 | #### Purging from Private Data Store and private state 122 | As mentioned in section [Background, challenges, and Requirements](#background-challenges-and-requirements), we intend to perform the private data store deletions of the historic keys in the background. Therefore, at the time of processing a private data purge transaction, we insert the marker key at the time of block commit, as described in the section [Block commit](#block-commit). In order to perform the actual purge of the data from private data store, we run a background goroutine that reads the PurgeMarker keys and scans all the relevant HashedIndex keys such that the index keys contain lower blkNum_TxNum than what is present in the PurgeMarker key. Finally, we would use the index key to locate the actual private write-set and we would trim the write-set by purging the intended key (recall that the HashedIndex value contains the actual private key as a reference for deletions). In the implementation, we intend to add an info field in the Marker key that would indicate whether the particular marker key has been processed. Also, if another purge operation is specified on the same key in the future, we delete the existing PurgeMarker for the key 123 | 124 | #### Creating the HashedIndex keys retroactively 125 | As mentioned in the section [New Keys in Private data store](#new-keys-in-private-data-store), when starting the peer version that contains purge support for the first time, we would need to retroactively create the HashedIndex keys for already committed private data in the private data store. We intend to do this automatically at the start of peer using the framework we introduced in data format upgrade in version 2.0. 126 | 127 | [drawbacks]: #drawbacks 128 | # Drawbacks 129 | If historical versions of private data are purged, it will not be possible to rollback the peers to a previous height without losing the purged private keys forever. However, this is not a big drawback as reset/rollback is not a strategic direction for Fabric. 130 | 131 | # Rationale and alternatives 132 | [alternatives]: #alternatives 133 | - Maintaining the PurgeMarker keys forever may monotonically increase the storage demand. An alternate design choice is to reformat the value for the missing keys in the private data store. In the current code, we maintain the missing data information just at the collection level, not at the level of individual key level - because, so far purging the individual keys were not possible. In the changed format, we can maintain the hashes of individual keys (or key-values both) as they appear in the block. We already do this in a specific scenario - when a peer is bootstrapped from a snapshot, the private data hashes cannot be fetched from the corresponding block and hence we load them in the private data store during bootstrapping a channel from a snapshot and maintain until the peer receives the private data via reconciliation. 134 | 135 | In this scheme, we would simply keep deleting the key hashes from the missing data entry and hence the peer knows exactly what keys are purged. Though, this scheme is expected to increase the performance for the reconciliation, but this would come at its own storage cost of maintaining the hashes for individual keys. However, this is expected to induce much bigger storage demand than maintaining the PurgeMarker keys because, in the case of Purge Marker we maintain at most one marker for a given key unlike, storing hashes repeatedly for each version of the key. As reconciliation is not a very frequent scenario, we prefer low storage cost over reconciliation efficiency and hence chose the PurgeMarker based approach as compared to the alternate design mentioned here. 136 | 137 | - Maintaining HashedIndex keys adds additional storage demand. The alternate choice is not to maintain these indexes, however, in absence of these, we would be required to scan and unmarshal entire data in the private data store each time we process a PurgeMarker. In the proposed design we maintain HashedIndex such that for a PurgeMarker all the relevant HashedIndex are scanned sequentially followed by random access for relevant items only. Moreover, as the purge operation is supposed to free up space, the effect of additional index keys may not be high. Also, these HashedIndex keys maintain the raw private key as well (as the value of the index key) otherwise, in addition to scan and unmarshal, we would need to compute the hashes of all the keys as well. 138 | 139 | - We always desire to avoid the need of introducing a new capability as it requires coordination between admins of different orgs. In the context of private data purge feature, as far as the latest state and transaction processing is concerned, we can introduce this feature without gating it behind a new capability tag -- Because, as mentioned above, we would always set `isDelete` to true when we set `is_purge` to true and `isDelete` would be interpreted the same way by the existing and the new version. Still, without introducing a new capability, in a channel, peers different versions would behave differently for purging of historical versions. Which raises some concerns as the purpose of introducing this feature via a transaction is to ensure that the requested private data is purged fully. Otherwise, this feature could have very well been exposed as an admin API on peer that a user can invoke on selective peers. Moreover, this in turns makes reconciliation process more challenging as a peer that missed processing a purge transaction, would not learn this fact and would keep seeking for the missed private data. As an alternate of this, we considered to provide an offline utility that would scan the block store for the purge-transactions and create the PurgeMarker keys in the private data store. The admin can use this utility on their peer either before or after upgrading their peers without coordinating with others. Though this avoids the need to set the capability but it comes with its own challenges which would be newer as compared to the known procedure of setting the capability. 140 | 141 | # Testing 142 | [testing]: #testing 143 | In addition to regular unit tests and component level integration tests, following behavior should be verified by the end-to-end integration tests. For testing the purged data, we intend to use deliver service APIs to fetch a particular version of private data. 144 | 145 | 1) User is able to submit a purge transaction that involves more than one keys 146 | 147 | 2) The endorsement policy is evaluated correctly for a purge transaction under different endorsement policy settings (e.g., collection level/ key-hash based) 148 | 149 | 3) Data is purged on an eligible peer 150 | - Add a few keys into a collection 151 | - Issue a purge transaction for some of the keys 152 | - Verify that all the versions of the intended keys are purged while the remaining keys still exist 153 | - Repeat above to purge all keys to test the corner case 154 | 155 | 4) Data is purged on previously eligible but now ineligible peer 156 | - Add a few keys into a collection 157 | - Submit a collection config update to remove an org 158 | - Issue a purge transaction to delete few keys 159 | - The removed orgs peer should have purged the historical versions of intended key 160 | 161 | 5) A new peer able to reconcile from a purged peer 162 | - Stop one of the peers of an eligible org 163 | - Add a few keys into a collection 164 | - Issue a purge transaction for some of the keys 165 | - Start the stopped peer and the peer should reconcile the partial available data 166 | 167 | 6) The purge transaction takes effect only if the corresponding capability is set 168 | - Add a few keys into a collection 169 | - Issue a purge transaction without setting the new capability 170 | - Verify that the purge behaves exactly as `delete` - i.e., deletes from state but does not purge the historical versions 171 | - Issue a config transaction to set the new capability 172 | - Now the purge transact should delete the historical versions as well 173 | 174 | 175 | # Dependencies 176 | [dependencies]: #dependencies 177 | 178 | In the future, we if decide to provide another API that allows access to history of private data keys, the HashedIndex keys introduced in this RFC can be leveraged as is. 179 | 180 | # Unresolved questions 181 | [unresolved]: #unresolved-questions 182 | -------------------------------------------------------------------------------- /text/0001-chaincode-go-new-programming-model.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Chaincode Go new programming model 4 | parent: RFCs 5 | --- 6 | 7 | - Feature Name: chaincode_go_new_programming_model 8 | - Start Date: 2019-11-15 9 | - RFC PR: 10 | - Fabric Component: fabric-contract-api-go 11 | - Fabric Issue: FAB-16640 12 | 13 | # Summary 14 | [summary]: #summary 15 | 16 | The updated programming model has been implemented in Java and Node [FAB-11246](https://jira.hyperledger.org/browse/FAB-11246). This RFC is therefore suggesting that Go chaincode be updated to follow this new model. 17 | 18 | This new implementation will be situated in a new repo `fabric-contract-api-go` with myself (@awjh-ibm) and @heatherlp as initial maintainers. 19 | 20 | # Motivation 21 | [motivation]: #motivation 22 | 23 | The existing programming model in Go requires a certain amount of boiler plate code to be written by the developer, things such as routing in the Invoke function, converting data from its arg type to the usable type in the transaction and from its transaction type to the return type. The new programming model as implemented in Java and Node reduces this boiler plate for developers and minimizes the amount of code they have to write to implement their smart contract. 24 | 25 | The existing programming model in Go provides zero way for an outside application or developer (with access to the network) to know the interface that the chaincode provides. By moving to the new programming model metadata will be made available for Go chaincode allowing developers to build external applications for interacting with the smart contract without needing to see the code. 26 | 27 | There is further motivation in including this in Go for synchronisation across supported fabric languages. 28 | 29 | Motivation for the new programming model as used in Java and Node can be found [here](https://docs.google.com/document/d/1_np3fnT_OludRGcF3PbubDooNsH8J-_G7UaWhk8a_cU/edit?usp=sharing) 30 | 31 | # Guide-level explanation 32 | [guide-level-explanation]: #guide-level-explanation 33 | 34 | Explanation was discussed on the community call and shared in the mailing list. The PDF of this can be found in the mailing list here: https://lists.hyperledger.org/g/fabric/topic/55666355 35 | 36 | # Reference-level explanation 37 | [reference-level-explanation]: #reference-level-explanation 38 | 39 | Explanation was discussed on the community call and shared in the mailing list. The PDF of this can be found in the mailing list here: https://lists.hyperledger.org/g/fabric/topic/55666355 40 | 41 | # Drawbacks 42 | [drawbacks]: #drawbacks 43 | N/A 44 | 45 | # Rationale and alternatives 46 | [alternatives]: #alternatives 47 | 48 | The rationale behind this is explained in the motivation. 49 | 50 | An alternative would be to remain as is and leave Go chaincode without the benefits of the new programming model and out of step with Node and Java. 51 | 52 | # Prior art 53 | [prior-art]: #prior-art 54 | 55 | Other chaincode languages already include the feature: 56 | - Node https://github.com/hyperledger/fabric-chaincode-node 57 | - Java https://github.com/hyperledger/fabric-chaincode-java 58 | 59 | Proposed implementation for Go can be found here: 60 | https://github.com/awjh-ibm/fabric-contract-api-go 61 | 62 | # Testing 63 | [testing]: #testing 64 | 65 | - Unit tests will be written for new code 66 | - Functional tests in cucumber to cover new package 67 | - Integration tests added to the upcoming chaincode test repository [FAB-15254](https://jira.hyperledger.org/browse/FAB-15254) 68 | 69 | # Dependencies 70 | [dependencies]: #dependencies 71 | 72 | - New samples will need to be written to cover the new programming model like has been added for Java and Node: 73 | - fabcar ([implementation](https://github.com/awjh-ibm/fabric-samples/tree/fabcar-go/chaincode/fabcar/go)) 74 | - commercial paper ([implementation](https://github.com/awjh-ibm/fabric-samples/tree/commercial-paper/commercial-paper)) 75 | 76 | # Unresolved questions 77 | [unresolved]: #unresolved-questions 78 | N/A 79 | -------------------------------------------------------------------------------- /text/0002-go-sdk-programming-model.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Implement Fabric programming model in the Go SDK 4 | parent: RFCs 5 | --- 6 | 7 | - Feature Name: Implement Fabric programming model in the Go SDK 8 | - Start Date: 2019-11-21 9 | - RFC PR: (leave this empty) 10 | - Fabric Component: fabric-sdk-go 11 | - Fabric Issue: (leave this empty) 12 | 13 | # Summary 14 | [summary]: #summary 15 | 16 | This feature will implement the new Fabric programming model for the Go SDK. The new Fabric programming model provides a set of higher level abstractions designed to enable developers to easily write smart contracts and client applications with minimal boilerplate code. 17 | 18 | # Motivation 19 | [motivation]: #motivation 20 | 21 | The motivation of this work is to provide a consistent set of concepts and APIs across all of the SDKs supported by Fabric. The ideal outcome is that the Go SDK become an 'officially supported' SDK alongside the Node and Java SDKs, and included in the official Fabric docs and samples. 22 | 23 | # Guide-level explanation 24 | [guide-level-explanation]: #guide-level-explanation 25 | 26 | The Fabric documentation (v1.4 onwards) contains a chapter 'Developing Applications' which describes how to develop smart contracts and client applications using the new programming model. The chapter illustrates the concepts using the example of a commercial paper, and gives code examples as it guides the user through the development steps. This documentation, and related samples, currently applies to the Node and Java SDKs and chaincode contract APIs. This RFC describes the implementation of this in the Go SDK and is related to [RFC 0001](https://github.com/hyperledger/fabric-rfcs/blob/main/text/0001-chaincode-go-new-programming-model.md) which describes the Go implementation of the chaincode contract API. 27 | 28 | Note that this new programming model does not break any existing APIs. Client applications written using these APIs will continue to work unchanged. 29 | 30 | As described in the documention for [client applications](https://hyperledger-fabric.readthedocs.io/en/release-1.4/developapps/application.html), the following structures are available: 31 | 32 | ### Wallet 33 | The wallet is used to manage the storage of identities for use within the client application. The SDK provides two wallet implementations out of the box: 34 | - File system wallet. Stores the identity on the local file system. The format used is compatible with the Node and Java SDKs, and can be used interchangably. 35 | - In-memory wallet. Provided primarily for testing purposes. 36 | 37 | Developers can create their own wallet implementations using the interface (and samples) provided if they wish to use custom storage mechanisms 38 | 39 | ### Gateway 40 | Logical concept representing the connection point a client makes to the Fabric network. The user configures the gateway using a network config (json/yaml) and an identity from the wallet. Other options can also be configured on the gateway in a declarative manner. 41 | 42 | ### Network channel 43 | 44 | Represents a channel available to the gateway peer. 45 | 46 | ### Contract 47 | 48 | Represents an instance of a smart contract available on a network channel. It has two primary methods: 49 | 50 | - `EvaluateTransaction()` - Invokes a transaction function in the smart contract (chaincode). 51 | - `SubmitTransaction()` - Invokes a transaction function in the smart contract and commits its RW-set to the ledger, subject to consensus. 52 | 53 | ### Transaction 54 | 55 | Enables finer control of transaction parameters (e.g. transient data). 56 | 57 | 58 | # Reference-level explanation 59 | [reference-level-explanation]: #reference-level-explanation 60 | 61 | Given the concepts and structures introduced in the previous section, a client application written to this programming model would have the following general structure (note this has been derived from an early prototype): 62 | 63 | ``` 64 | package main 65 | 66 | import ( 67 | "fmt" 68 | "github.com/hyperledger/fabric-sdk-go/pkg/gateway" 69 | ) 70 | 71 | func main() { 72 | wallet := gateway.NewInMemoryWallet() 73 | wallet.Put("user", gateway.NewX509Identity(testCert, testPrivKey)) 74 | 75 | gw, err := gateway.Connect("connection-tls.json", 76 | gateway.WithIdentity(wallet, "user"), 77 | gateway.WithDiscovery(false), 78 | gateway.WithCommitHandler(gateway.DefaultCommitHandlers.OrgAny)) 79 | 80 | network, err := gw.GetNetwork("mychannel") 81 | 82 | if err != nil { 83 | fmt.Printf("Failed to get network: %s", err) 84 | return 85 | } 86 | 87 | contract := network.GetContract("fabcar") 88 | 89 | result, err := contract.EvaluateTransaction("queryAllCars") 90 | 91 | if err != nil { 92 | fmt.Printf("Failed to evaluate transaction: %s", err) 93 | return 94 | } 95 | fmt.Println(result) 96 | 97 | result, err = contract.SubmitTransaction("createCar", "CAR10", "VW", "Polo", "Grey", "Mary") 98 | 99 | if err != nil { 100 | fmt.Printf("Failed to submit transaction: %s", err) 101 | return 102 | } 103 | fmt.Println(result) 104 | 105 | // the user might prefer a 'helper' method to reduce boilerplate error handling 106 | handle(contract.EvaluateTransaction("queryCar", "CAR10")) 107 | 108 | // create a Transaction object to pass transient data or override endorsers 109 | transient := make(map[string][]byte) 110 | transient["price"] = []byte("8500") 111 | txn, err := contract.CreateTransaction("changeCarOwner", gateway.WithTransient(transient)) 112 | result, _ = txn.Submit("CAR10", "Archie") 113 | 114 | handle(contract.EvaluateTransaction("queryCar", "CAR10")) 115 | } 116 | 117 | func handle(result string, err error) { 118 | if err != nil { 119 | panic(err) 120 | } 121 | fmt.Println(result) 122 | } 123 | ``` 124 | 125 | ### Wallet 126 | The wallet interface provides the following methods: 127 | - `Put(label, identity)` 128 | - `Get(label)` 129 | - `Exists(label)` 130 | - `List()` 131 | - `Remove(label)` 132 | 133 | ### Gateway 134 | The gateway is configured and constructed using a set of options, in common with the Java and Node SDKs. The `Connect()` function accepts functional option arguments which is idiomatic in Go and consistent with the existing Go SDK. The GatewayOption functions are available: 135 | - `WithIdentity(wallet, label)` - select an identity from a wallet to associate with the gateway 136 | - `WithCommitHandler(handler)` - override the default commit handler strategy 137 | - `WithQueryHandler(handler)` - override the default query handler strategy 138 | - `WithDiscovery(enabled)` - Enables (default) or disables service discovery 139 | - `Connect(networkConfig, ...GatewayOption)` - instantiate the Gateway with the the path to network config (json/yaml) plus any of the above options 140 | 141 | The Gateway interface has the following method: 142 | - `GetNetwork(name)` - returns an initialised network channel 143 | 144 | ### Network 145 | The Network interface has the following method: 146 | - `GetContract(name)` - returns a reference to the named contract in the chaincode 147 | 148 | ### Contract 149 | The Contract interface has the following methods: 150 | - `EvaluateTransaction(name, ...args)` 151 | - `SubmitTransaction(name, ...args)` 152 | - `CreateTransaction(name, ...TransactionOption)` 153 | 154 | The `EvaluateTransaction` method runs the transaction (proposal) function in the smart contract and returns whatever value that function returns. The peer(s) that gets targetted is controlled by the QueryHandler. 155 | The `SubmitTransaction` method sends the transaction proposal to endorsing peers (determined by the discovery service), collates the resonses and sends to the orderer. This method awaits commit events from peers, as controlled by the CommitHandler, before returning to the client. 156 | The `CreateTransaction` method creates a transaction object representing the named transaction function. Additional information can be associated with the transaction request using the `TransactionOption` functional arguments before it is submitted. The following options are supported: 157 | - `WithTransient(transientMap)` - associates transient data with the transaction proposal 158 | - `WithEndorsingPeers(...peer)` - targets specific peers for this transaction request, overriding the discovery service. Useful for targetting private data. 159 | 160 | 161 | #### CommitHandler 162 | The commit handler is a pluggable mechanism that controls how many commit events should be received before the client can continue processing. It is selected as an option on the Gateway. The following policies are provided out of the box: 163 | - Wait for __all__ available peers in __your organisation__ to emit commit events – DEFAULT 164 | - Wait for __first__ peer in __your organisation__ to emit commit event 165 | - Wait for __all__ available peers in the __network channel__ to emit commit events 166 | - Wait for __first__ peer in the __network channel__ to emit commit event 167 | Users can also implement their own commit handlers if they wish to have other behaviours. 168 | 169 | #### QueryHandler 170 | The query handler is a pluggable mechanism that controls which peers to target when evaluating a transaction function using `EvaluateTransaction()`. It is selected as an option on the Gateway. The following policies are provided out of the box: 171 | - Stick to a __single peer__. Move to another one if it stops responding – DEFAULT 172 | - __Round-robin__ between available peers within your organisation. 173 | Users can also implement their own query handlers if they wish to have other behaviours. 174 | 175 | ### Transaction 176 | - `Evaluate(...args)` - As with EvaluateTransaction() above, but with TransactionOptions (e.g. transient data). 177 | - `Submit(...args)` - As with SubmitTransaction() above, but with TransactionOptions (e.g. transient data). 178 | 179 | # Drawbacks 180 | [drawbacks]: #drawbacks 181 | 182 | It could be argued that this is not needed because all of its capabilities can already be done in the existing Go SDK. 183 | 184 | This is being proposed not because of any underlying deficiencies in the existing APIs, but rather to provide a __consistent__ set of concepts and structures across __all__ of the __supported__ developer APIs in Fabric. 185 | 186 | # Rationale and alternatives 187 | [alternatives]: #alternatives 188 | 189 | The primary rationale is around consistency with the other SDKs and to align with the programming model as described in the official Fabric documentation. Alternatives could always be proposed, but they would not necessarily be consistent with the other SDKs or the programming model. 190 | 191 | # Prior art 192 | [prior-art]: #prior-art 193 | 194 | Significant experience and feedback has now been gained from user adoption of this programming model in Node and Java which have led to incremental improvements. 195 | 196 | # Testing 197 | [testing]: #testing 198 | 199 | In common with the Node and Java SDKs, there is a strong emphasis on very high coverage of unit tests and scenario (integration) tests. Appropriate, popular frameworks are adopted for each technology. 200 | 201 | Common cucumber scripts (BDD tests) have been implemented across Node and Java SDKs. It is intended to implement these for Go to enforce common behaviour across all SDKs. 202 | 203 | # Dependencies 204 | [dependencies]: #dependencies 205 | 206 | This proposal builds (and depends) on the existing Go SDK as implemented in the `hyperledger/fabric-sdk-go` repo. 207 | 208 | New client samples for Fabcar and Commercial Paper, implemented in Go, will be added to the `hyperledger/fabric-samples` repo. 209 | 210 | This is also related to, but does not depend on, [RFC 0001](https://github.com/hyperledger/fabric-rfcs/blob/main/text/0001-chaincode-go-new-programming-model.md) which brings the new programming model to the Go chaincode API. 211 | 212 | # Unresolved questions 213 | [unresolved]: #unresolved-questions 214 | -------------------------------------------------------------------------------- /text/0003-config-transaction-library.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Create a standalone config transaction library 4 | parent: RFCs 5 | --- 6 | 7 | - Feature Name: Create a standalone config transaction library that generates channel config creation and update envelopes 8 | - Start Date: 2020-03-20 9 | - RFC PR: (leave this empty) 10 | - Fabric Component: fabric, fabric-config 11 | - Fabric Issue: (leave this empty) 12 | 13 | # Summary 14 | [summary]: #summary 15 | 16 | This feature will implement a consumable standalone library that supports generating configuration envelopes for 17 | operations including application and system channel creation, channel configuration updates, and attaching 18 | endorsing signatures to generated envelopes. 19 | 20 | # Motivation 21 | [motivation]: #motivation 22 | 23 | The motivation for this work is to provide a consistent API that is actually intended for consumption by external 24 | clients for any operations involving channel config transactions. One example usage case could be a 25 | CLI tool for creating config updates that takes as flags properties to be updated alongside an existing config transaction. 26 | This CLI tool could then pass these parameters down to the configtx package for generating a config update envelope with the computed 27 | read and write sets. Currently `configtxgen` is available and frequently misused as a tool for these operations, but it was never intended for 28 | production usage. Much of the existing process for generating configuration updates is both tedious and error prone involving 29 | decoding a config transaction to JSON with the protolator tool and manually modifying fields prior to encoding back to a protobuf 30 | and submitting for update. 31 | 32 | # Guide-level explanation 33 | [guide-level-explanation]: #guide-level-explanation 34 | 35 | Work on this consumable library has already begun in [`fabric/pkg/configtx`](https://github.com/hyperledger/fabric/tree/master/pkg/configtx) 36 | and tracked in [FAB-17628](https://jira.hyperledger.org/browse/FAB-17628). Example usage has been included in the 37 | [godocs](https://godoc.org/github.com/hyperledger/fabric/pkg/configtx#pkg-examples) for this package and can be referenced for external tooling. 38 | 39 | # Reference-level explanation 40 | [reference-level-explanation]: #reference-level-explanation 41 | 42 | API documentation already exists in the godocs for this package and can be referenced [here](https://godoc.org/github.com/hyperledger/fabric/pkg/configtx). 43 | 44 | # Drawbacks 45 | [drawbacks]: #drawbacks 46 | 47 | Many potential consumers may already have a stable method for updating configuration transactions and may be 48 | unwilling to make amendments to their processes by consuming this library. It could be argued that even if this library 49 | exists, no external clients already using `configtxgen` would want to switch to consuming it. Another consideration is this library 50 | will only be provided in Golang, but may eventually support TinyGo compilation to WebAssembly for usage as a web module. 51 | 52 | # Rationale and alternatives 53 | [alternatives]: #alternatives 54 | 55 | Eventually it will become too onerous to maintain current workarounds for channel configuration management 56 | as new features are added around channel config. Rather than having to manually update bits and pieces of workaround 57 | code, consumption of this library will become a consistent method for adopting new features. 58 | 59 | # Prior art 60 | [prior-art]: #prior-art 61 | 62 | Consumable Golang packages have consistent and effective APIs, are well-documented, leverage universal coding standards, 63 | and provide useful examples of common usage. 64 | 65 | # Testing 66 | [testing]: #testing 67 | 68 | This package has high coverage of core features on both a unit level and integration test level. Example usage tests are also 69 | included in the godocs. 70 | 71 | # Dependencies 72 | [dependencies]: #dependencies 73 | 74 | In an effort to make this library as lightweight as possible, we've reduced the number of required dependencies to just a handful including 75 | the Golang standard library and the following imported `hyperledger` packages and components: 76 | - `fabric-protos-go` 77 | - `fabric/common/policydsl` 78 | - `fabric/common/tools/protolator` 79 | 80 | # Unresolved questions 81 | [unresolved]: #unresolved-questions 82 | 83 | As part of this RFC process we would like to also move the configtx package code we've already started on in `hyperledger/fabric/pkg/configtx` to 84 | a separate `hyperledger/fabric-config` repo to remove it from fabric's release cycle. As a standalone consumable library, 85 | this package does not need to exist in `hyperledger/fabric` or be tied to a specific release. 86 | -------------------------------------------------------------------------------- /text/0005-lts-release-strategy.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Fabric LTS release strategy 4 | parent: RFCs 5 | --- 6 | 7 | - Feature Name: Fabric LTS release strategy 8 | - Start Date: 2020-03-18 9 | - RFC PR: 10 | - Fabric Component: 11 | - Fabric Issue: 12 | 13 | # Summary 14 | [summary]: #summary 15 | 16 | Define Long-term support (LTS) release strategy for Hyperledger Fabric 17 | 18 | Long-term support definition from wikipedia.org: 19 | 20 | > **Long-term support (LTS)** is a product lifecycle management policy in which a stable release of computer software is maintained for a longer period of time than the standard edition. 21 | 22 | > **LTS** applies the tenets of reliability engineering to the software development process and software release life cycle. Long-term support extends the period of software maintenance; it also alters the type and frequency of software updates (patches) to reduce the risk, expense, and disruption of software deployment, while promoting the dependability of the software. 23 | 24 | # Motivation 25 | [motivation]: #motivation 26 | 27 | Administrators of production deployments find LTS releases desirable as they provide confidence that a certain software release will be maintained for an extended period of time. 28 | 29 | Specifically, for Hyperledger Fabric, administrators can expect the following of a LTS release: 30 | 31 | - Stable and well-tested code 32 | - API contract stability 33 | - 3rd digit patch releases for bug and security fixes, delivered for an extended period of time on a regular cadence (or more frequently as needed) 34 | - Minimal set of feature additions and other changes that can easily be applied, reducing the risk of functional regressions and bugs 35 | 36 | Similarly, there are benefits to Hyperledger Fabric maintainers, code contributors, and the wider community: 37 | 38 | - New features and other changes can quickly be applied to the main branch, and distributed to the user community for trial, without impacting production deployments. 39 | - Community feedback on new features can be solicited and acted upon. 40 | - Bug fixes only need to be backported to a small number of designated LTS releases. 41 | - Extra tests (e.g. upgrade tests for non-subsequent versions) only need to be executed against a small number of designated LTS releases. 42 | 43 | # Guide-level explanation 44 | [guide-level-explanation]: #guide-level-explanation 45 | 46 | ## Release cadence 47 | 48 | Hyperledger Fabric will continue the pattern of quarterly minor releases, e.g. v2.0, v2.1, v2.2, etc. 49 | 50 | ## LTS release cadence 51 | 52 | Because a new major release typically has large new features that may not yet be tried by the user community, and because SDKs may lag in support of the new release, it is not expected that a new major release will immediately be designated as a LTS release, for example, v2.0 is not a LTS release. 53 | 54 | Each major release will eventually have at least one minor release designated by the Fabric maintainers as the LTS release, e.g. v1.4 for v1.x, v2.2 for v2.x (all references to future versions are hypothetical). 55 | 56 | After a LTS release is designated, quarterly releases will continue as normal. The Fabric maintainers will decide whether a next release is another minor release, or a new major release. There is no predefined timing for next major version. The decision is typically based on semantic versioning considerations, such as whether API changes are needed or older capabilities need to be removed. Other considerations may also apply, for example significant upgrade steps may motivate a shift to a new major version. 57 | 58 | If a major release is not delivered for an extended period of time, the maintainers may designate a subsequent minor release as the next LTS release, for example if v3.1 is a LTS release and there is no need to increment to 4.0 for several quarters, the maintainers may decide to designate v3.4 as the next LTS release. 59 | 60 | ## LTS 3rd digit patch releases 61 | 62 | For LTS releases, 3rd digit patch releases will be provided for bug and security fixes approximately every three months (may be more frequent if important fixes become available sooner, and may be deferred if there are no fixes to publish). Significant new features and other changes will not be included in 3rd digit patch releases, in order to ensure the stability of the LTS release and reduce the risk of functional regressions and bugs. 63 | 64 | It is worth noting that the v1.4 LTS release did not follow the above principles. Had the v1.4 LTS release followed the above principles, the Raft ordering service feature would likely have been introduced in a v1.5 release, and Kafka to Raft migration in a v1.6 release. We mention only to illustrate the above principles. 65 | 66 | ## LTS release duration 67 | 68 | Once a *subsequent* LTS release is designated, users can expect 3rd digit patch releases to address critical bugs on the *prior* LTS release for approximately 9 more months. The maintainers will determine which fixes should to be backported to the latest LTS release, versus which fixes need to be further backported to the prior LTS release. 3rd digit patch releases should be expected less frequently for prior LTS releases, since only critical fixes will be published. The overlap period is intended to provide users a time window to upgrade their nodes, and any channel capabilities (if required). 69 | 70 | ## LTS release diagram 71 | The following diagram represents the LTS release cadence and duration using hypothetical Fabric releases: 72 | ![LTS release diagram](../images/lts_release_diagram.png) 73 | 74 | ## LTS to LTS compatibility 75 | 76 | Features that are related to capabilities will remain available for two LTS releases, but may be removed in subsequent LTS releases. For example, v1.x legacy chaincode management will be available through v2.2 (assuming v2.2 is a LTS release), but is expected to not be usable in the subsequent LTS release. This provides one full LTS cycle for channels to transition to new capabilities. For example, a production Fabric network may perform a rolling upgrade from v1.4.x nodes to v2.2.x nodes, then enable v2.x channel capabilities to take advantage of the new chaincode lifecycle, and then eventually upgrade to v3.x LTS nodes, where the legacy chaincode lifecycle may not be available. 77 | 78 | ## LTS and upgrade testing 79 | 80 | The Fabric project expects to test single step upgrades (e.g. v2.0 to v2.1), and LTS to next-LTS upgrades (e.g. v1.4.x to v2.2). Other upgrade paths will not be tested and are not guaranteed to work. 81 | 82 | 83 | # Prior art and alternatives 84 | [prior-art]: #prior-art 85 | 86 | While many open source projects provide LTS releases, there is no industry standard for LTS release approach. Projects use many different variants of LTS approaches to best suit their project's particular needs. 87 | 88 | Given Fabric's time-based releases, major and minor release cadence, and desire to designate a minor release a LTS release after the associated major release has been available for some time, the [Node.js project's LTS model](https://nodejs.org/en/about/releases/) seems like a good fit generally. 89 | 90 | However, the Node.js project only designates every other major release as a LTS release, which is not desirable for the Fabric project given the rapid innovation in the distributed ledger market. Additionally, the Node.js project provides maintenance for a fixed amount of time for each LTS release, while the Fabric maintainers prefer to set the duration based on an overlap period with the subsequent LTS release, again due to the rapid innovation in the distributed ledger market and need to be flexible with respect to major release, minor release, and LTS release timing. 91 | -------------------------------------------------------------------------------- /text/0010-bft-signatures.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Fabric core block retrieval and signature verification 4 | nav_order: 3 5 | --- 6 | 7 | - Feature Name: Fabric core block retrieval and signature verification 8 | - Start Date: (fill me in with today's date, 2022-02-01) 9 | - RFC PR: (leave this empty) 10 | - Fabric Component: peer, orderer 11 | - Fabric Issue: (leave this empty) 12 | 13 | 14 | # Table of Contents 15 | 1. [Summary](#summary) 16 | 2. [Motivation](#motivation) 17 | 3. [Existing Approaches](#existing-approaches-for-block-replication-and-block-signature-verification) 18 | 4. [Proposed Techniques](#proposed-techniques) 19 | 5. [Implementation Plan](#implementation-plan) 20 | 6. [Testing](#testing) 21 | 22 | # Summary 23 | [summary]: #summary 24 | 25 | The Hyperledger Fabric core (orderers and peers) are only crash fault tolerant, and cannot handle arbitrary behavior of its ordering service nodes. 26 | 27 | 28 | The current Fabric peers and orderers are implemented to retrieve blocks from a single orderer 29 | and there is currently no design of how blocks are to be retrieved in the Byzantine setting or how signatures on blocks should be verified. 30 | 31 | 32 | 33 | Block retrieval from Byzantine orderers needs to tackle denial of service from malicious orderers. 34 | 35 | The challenge with block signature validation in the Byzantine setting is avoiding the linear space overhead in the number of certificates that need to be stored as part of the signature information. 36 | 37 | While forks of Fabric with a Byzantine Fault Tolerant ordering service [exist](https://github.com/SmartBFT-Go/fabric/), they have not been incorporated into the official Hyperledger Fabric project. 38 | 39 | This RFC aims to: 40 | 41 | - Give an overview of how denial of service resistant block retrieval and block signature validation is done in SmartBFT. 42 | - Define how denial of service resistant block retrieval should be conducted in Hyperledger Fabric. 43 | - Define how Byzantine block signature validation should be constructed and encoded in Hyperledger Fabric. 44 | 45 | # Motivation 46 | [motivation]: #motivation 47 | 48 | Currently, Hyperledger Fabric only supports ordering service types that are Crash Fault Tolerant (CFT), not Byzantine Fault Tolerant (BFT). 49 | 50 | There are two main attacks that are considered outside of the crash fault tolerant threat model, but must be protected against in the Byzantine fault tolerant setting: 51 | 52 | **Block withholding**: A Byzantine orderer might be reluctant to send newly formed blocks to peers that connect to it, either because of malicious intent or to save egress network bandwidth charges from its hosting provider. 53 | Fabric architecture relies on peers receiving newly formed blocks as fast as possible, because otherwise client transaction submission times out and smart contract simulation results in stale reads. 54 | 55 | **Block forgery**: A Byzantine orderer might try to forge a block and send it to peers or other ordering nodes that replicate from it, in order to alter the smart contract execution on peers by subterfuge. 56 | One way of doing this, is to omit a transaction from a block, or re-order transactions within a block. 57 | 58 | To circumvent block forgery attacks, the following invariant should be preserved: *A set of signatures on a block should only be considered valid if it is impossible for the parties running the protocol to construct a set of signatures on a different block that is also considered valid*. 59 | 60 | The aforementioned invariant can be implemented in different ways, such as requiring a **threshold of signatures** from different orderers on a block, or requiring a single **threshold signature** on a block. 61 | The threshold itself can vary depending on the consensus algorithm and on the stage in it that the signatures is produced: Signing a block post consensus requires signatures from only `f+1` orderers (at the expense of latency), while signing during consensus requires signatures from `2f+1` orderers (at the expense of size and computation). 62 | 63 | 64 | # Naive approaches for block replication and block signature verification 65 | 66 | ### Block replication 67 | 68 | In a Byzantine setting, there is a fundamental assumption that there is an upper bound on the number of malicious or failed nodes 69 | among all the nodes that run the protocol, and that upper bound is denoted as `f`. 70 | In BFT, the ordering service is comprised of `n>3f` for some `f>0`, so we have up to `f` nodes which are either offline, unreachable or malicious, 71 | and at least `n-f` nodes that are online, reachable and follow the protocol. 72 | 73 | An easy way of ensuring malicious orderers do not refrain from sending newly formed blocks is to simply pull blocks from `f+1` distinct orderers. 74 | This ensures that even if we happen to connect to all `f` malicious orderers that will not send us blocks, the remaining orderer will, since it is honest. 75 | 76 | Obviously, such an approach increases the network bandwidth utilization of the orderers by a factor of `f` which is linear in the number of total ordering service nodes that run the consensus protocol. 77 | 78 | ### Block signature verification 79 | 80 | A Hyperledger Fabric block is designed to carry an arbitrary number of signatures in its metadata section: 81 | 82 | ``` 83 | message MetadataSignature { 84 | bytes signature_header = 1; // An encoded SignatureHeader 85 | bytes signature = 2; // The signature over the concatenation of the Metadata value bytes, signatureHeader, and block header 86 | } 87 | ``` 88 | 89 | The signature header contains the identity of the signer which carries an x509 certificate and the MSP identity that one of its certificate authorities issued that certificate: 90 | 91 | ``` 92 | message SignatureHeader { 93 | // Creator of the message, a marshaled msp.SerializedIdentity 94 | bytes creator = 1; 95 | 96 | // Arbitrary number that may only be used once. Can be used to detect replay attacks. 97 | bytes nonce = 2; 98 | } 99 | ``` 100 | 101 | This means that when signed by `2f+1` (or `f+1`) orderers, each block contains a number of signatures that is linear in the number of ordering service nodes that participate in the consensus protocol. 102 | When considering an ordering service that can tolerat two failures (`f=2`, hence `n=7`) this means that each block carries 3.5KB of x509 certificates, 103 | even if the block itself contains only a single transaction (that is often 3.5KB in size itself). 104 | 105 | Clearly, encoding identities each time in the signature header of each orderer signature is wasteful and therefore we should strive to eliminate this overhead. 106 | 107 | ## Existing approaches for block replication and block signature verification 108 | [Existing approaches]: #existing-approaches-for-block-replication-and-block-signature-verification 109 | 110 | ### Attestation for block existence 111 | 112 | An approach for block denial of service resistance taken by [SmartBFT](https://arxiv.org/abs/2107.06922) is to connect to (at least)`f+1` ordering service nodes, fetch blocks only from one of them and fetch attestations that blocks exist from the remaining `f` (or more). 113 | If a block existence attestation for a sequence **i** is found valid but the orderer designated to send blocks withholds block **i** for too long, the role assignment of block source and attestation for block existence source is re-shuffled. 114 | 115 | An attestation for block existence is simply a regular block (along with its `2f+1` signatures) pruned from its transactions. The signatures can still be verified despite the transactions being pruned from the block thanks 116 | to the fact that in Fabric the signing algorithm is applied on the ASN1 encoding of the header block which carries the hash of the concatenation of all transactions: 117 | 118 | ``` 119 | message BlockHeader { 120 | uint64 number = 1; // The position in the blockchain 121 | bytes previous_hash = 2; // The hash of the previous block header 122 | bytes data_hash = 3; // The hash of the BlockData 123 | } 124 | ``` 125 | 126 | Let's consider the two possible outcomes based on the orderer that is selected for block retrieval: 127 | 128 | 1. If orderer that blocks are pulled from is withholding newly formed blocks, then there is at least one honest orderer that will not withhold attestations of block existence. Hence, a re-shuffling of role assignment will occur until an honest orderer will be the block source. 129 | 2. Else, the orderer that blocks are pulled from is not withholding newly formed blocks. However, it is possible that new blocks are not formed due to lack of client activity. It is imperative to ensure that malicious ordering nodes cannot make honest ordering nodes be falsely suspected of block withholding. 130 | As each "block existence attestation" contains `2f+1` signatures it is impossible to forge one. Hence, if a node sends a attestation for block existence, it means this a block with such sequence number was indeed appended to the blockchain. 131 | 132 | In SmartBFT, in order for an orderer to convey the intent of receiving only blocks without transactions, 133 | the [SeekInfo](https://github.com/hyperledger/fabric-protos/blob/main/orderer/ab.proto#L45) message is expanded to contain also a `SeekContentType` field: 134 | 135 | ``` 136 | // SeekContentType indicates what type of content to deliver in response to a request. If BLOCK is specified, 137 | // the orderer will stream blocks back to the peer. This is the default behavior. If HEADER_WITH_SIG is specified, the 138 | // orderer will stream only a the header and the signature, and the payload field will be set to nil. This allows 139 | // the requester to ascertain that the respective signed block exists in the orderer (or cluster of orderers). 140 | enum SeekContentType { 141 | BLOCK = 0; 142 | HEADER_WITH_SIG =1; 143 | } 144 | ``` 145 | 146 | By making the default value be `BLOCK` the API is backward compatible to old consumers. 147 | 148 | Indeed, the SmartBFT's approach for Byzantine fault tolerant block replication overcomes the linear overhead of the naive approach. However, a malicious ordering node can still withhold a block up until right before the suspicion timeout expires at the party that pulls the block. 149 | Hence, while a malicious orderer may not withhold blocks indefinitely, it may slow down the replication of its consumers. 150 | 151 | Another minor inconvenience is that the peer admin needs to configure the peer correctly to use the Byzantine fault tolerant implementation instead of the regular one. 152 | 153 | ### Signature identity referencing 154 | 155 | In SmartBFT, like Raft, each configuration block contains a set of consenter (ordering service consensus protocol participant) definitions. 156 | Each such consenter definition contains various information that is used either to connect or to authenticate the consenter: 157 | 158 | ``` 159 | message Consenter { 160 | uint64 consenter_id = 1; 161 | string host = 2; 162 | uint32 port = 3; 163 | string msp_id = 4; 164 | bytes identity = 5; 165 | bytes client_tls_cert = 6; 166 | bytes server_tls_cert = 7; 167 | } 168 | ``` 169 | 170 | When validating a signature signed by orderer with numerical identifier **j**, the mapping between **j** and the matching identity (which carries the public key) is taken from the latest configuration block. 171 | 172 | Recall, the naive approach to encode several signatures on a block in its metadata involves encoding an amount of orderer identities that is linear in the number of ordering nodes. 173 | 174 | To that end, SmartBFT substitutes the identity (over 700B in size) with its consensus node identifier (a 64 bit number). 175 | As a result, the `MetadataSignature` now contains three additional fields: 176 | 177 | ``` 178 | message MetadataSignature { 179 | bytes signature_header = 1; // An encoded SignatureHeader 180 | bytes signature = 2; // The signature over the concatenation of the Metadata value bytes, signatureHeader, and block header 181 | uint64 signer_id = 3; 182 | bytes nonce = 4; 183 | } 184 | ``` 185 | 186 | When a Fabric node validates a signature on a block, an ephemeral `SignatureHeader` is constructed where the `nonce` is taken from the 187 | `MetadataSignature` and the `creator` is the `identity` matching the `signer_id`. 188 | 189 | This technique reduces the aforementioned overhead of 3.5KB to a meager 40 bytes. 190 | 191 | 192 | While the approach of the SmartBFT library integration works, it creates a coupling between the consensus implementation and the rest of Fabric, and this limits incorporation of other consensus implementations into Fabric. 193 | 194 | More specifically, it requires peers to understand how to decode the consensus metadata of ordering service nodes. That means that the peer needs to be aware of the type of ordering service it is working with, which breaks the abstraction of the ordering service as a service that receives as input transactions, and outputs blocks. 195 | 196 | 197 | # Proposed techniques 198 | [technique]: #proposed-technique 199 | 200 | The proposed approach is heavily inspired by SmartBFT but is designed to be consensus agnostic, minimize implementation and operational complexity. 201 | 202 | ### Block replication 203 | 204 | The proposed technique for block replication is similar to the approach of SmartBFT where a re-shuffling takes place not only upon suspicion of the block source but also in a timely fashion in order to be resilient to overloaded or deliberately slow orderers. 205 | In order to save network bandwidth and processing power when the network is idle, the automatic time-based re-shuffling will only take place if new blocks are received. 206 | In other words, a re-shuffling takes place after every `T` seconds in which some blocks are received. 207 | If no blocks are received during a timeframe `T'`, the node will probe for block attestations to see if a block withholding takes place. 208 | The re-shuffling strategy would be configurable and the node operator would be able to disable it if needed. 209 | 210 | There will not be a **need** to specify in the peer's local configuration (or an orderer's local configuration) which of the implementations (CFT or BFT) to use. 211 | Notice that a block validation policy that mandates signatures from multiple ordering nodes, is **never** satisfied by **only** principal sets of size one. 212 | Or, equivalently, a block validation policy that is satisfied by **only** principal sets of size one never requires signatures from multiple ordering nodes. 213 | Evidently, policies of the former type are **always** BFT policies, while policies of the latter type might be either a crash fault policy or a BFT policy that utilizes a threshold signature scheme. 214 | 215 | Therefore, Fabric nodes will pick the BFT implementation if either the block validation policy implies the need, or if an override is activated in the node's local configuration. 216 | Note that overriding the local configuration will only affect protection against block withholding attacks, and cannot impact policy validation. 217 | Therefore, the in the worst scenario of a misconfiguration, malicious ordereres still cannot forge blocks. 218 | 219 | ### Block attestation 220 | 221 | We follow the approach of the SmartBFT implementation. The `SeekInfo` message is augmented with type `enum SeekContentType` and field `SeekContentType content_type = 5;`, as shown below: 222 | 223 | ```protobuf 224 | message SeekInfo { 225 | enum SeekBehavior { 226 | BLOCK_UNTIL_READY = 0; 227 | FAIL_IF_NOT_READY = 1; 228 | } 229 | 230 | enum SeekErrorResponse { 231 | STRICT = 0; 232 | BEST_EFFORT = 1; 233 | } 234 | 235 | // SeekContentType indicates what type of content to deliver in response to a request. If BLOCK is specified, 236 | // the orderer will stream blocks back to the peer. This is the default behavior. If HEADER_WITH_SIG is specified, the 237 | // orderer will stream only a the header and the signature(s), and the payload field will be set to nil. This allows 238 | // the requester to ascertain that the respective signed block exists in the orderer (or cluster of orderers). 239 | enum SeekContentType { 240 | BLOCK = 0; 241 | HEADER_WITH_SIG =1; 242 | } 243 | 244 | SeekPosition start = 1; // The position to start the deliver from 245 | SeekPosition stop = 2; // The position to stop the deliver 246 | SeekBehavior behavior = 3; // The behavior when a missing block is encountered 247 | SeekErrorResponse error_response = 4; // How to respond to errors reported to the deliver service 248 | SeekContentType content_type = 5; // Defines what type of content to deliver in response to a request 249 | } 250 | ``` 251 | 252 | When a stream of Header+Signatures is needed from the orderer, the peer (or any other client) will simply set the `ContentType` in the `SeekInfo` message to `HEADER_WITH_SIG`. 253 | In this case, the orderer will simply set the block data to `nil`. 254 | ```go 255 | if seekInfo.ContentType == ab.SeekInfo_HEADER_WITH_SIG { 256 | block.Data = nil 257 | } 258 | ``` 259 | Since a content type of `BLOCK` is the default value, no modification is needed to clients that currently request a stream of blocks. 260 | 261 | With this capability in place, a peer can detect block censorship by either (i) periodically and randomly sampling orderers for block attestations, or (ii) continuously asking for block attestations from some subset (or all) of orderers. 262 | 263 | ### Block signature verification 264 | 265 | The proposed technique for block signature verification should be generic and consensus agnostic, in order to facilitate incorporation of multiple consensus types without the need for Fabric nodes to understand which consensus implementation is employed. 266 | 267 | Apart from the already existing block validation policy: 268 | 269 | ``` 270 | # BlockValidation specifies what signatures must be included in the block 271 | # from the orderer for the peer to validate it. 272 | BlockValidation: 273 | Type: ImplicitMeta 274 | Rule: "ANY Writers" 275 | ``` 276 | 277 | The `Orderer` section in the channel config will contain a mapping from numerical identifiers to tuples of serialized identities: 278 | 279 | ``` 280 | Orderer: &OrdererDefaults 281 | ... 282 | ConsenterMapping: 283 | - Id: 1 284 | MSPID: OrdererOrg1 285 | Identity: 286 | - Id: 2 287 | MSPID: OrdererOrg2 288 | Identity: 289 | ... 290 | ``` 291 | 292 | A typical block validation policy for BFT can be a [signature policy](https://github.com/hyperledger/fabric-protos/blob/release-2.4/common/policies.proto#L28-L33) 293 | with a rule of a suitable threshold (e.g `2f+1` or `f+1` depending on `n`, `f` and the consensus protocol) and with identity principals that correspond 294 | to the `ConsenterMapping` in the configuration block. 295 | 296 | 297 | Upon a configuration update, it will be up to the (Fabric side of the) consensus implementation to validate that: 298 | 299 | - The block validation policy matches the required threshold (e.g `2f+1`) of the consensus protocol. 300 | - The `ConsenterMapping` is not in conflict with the opaque consensus metadata. 301 | 302 | The problem of correctly composing the consenter mapping and block validation policy at bootstrapping is considered out of scope of this RFC, 303 | as this is consensus specific and can be addressed with custom tooling, such as `configtxgen` enhancements or any other tooling. 304 | 305 | 306 | The `MetadataSignature` shall be expanded to contain a `uint32` field of `signer_id` and the `bytes` field of the `nonce` as in SmartBFT. 307 | Accordingly, the procedure to build a signed data element from a `MetadataSignature` will first look if the `signature_header` is not empty, and if so it will proceed to regular processing. 308 | Otherwise, if the `signature_header` is empty, an ephemeral one will be reconstructed from the `nonce`, `signer_id` and the latest channel config and then procesing can proceed as formerly mentioned. 309 | 310 | # Implementation plan 311 | [implementation]: #implementation-plan 312 | 313 | The following components are to be changed: 314 | 315 | - **Channel configuration**: Addition of `ConsenterMapping` to the channel configuration (gated behind a V3.0 [capability](https://hyperledger-fabric.readthedocs.io/en/latest/capabilities_concept.html)). 316 | - **Peer and orderer local configuration**: Addition of BFT or CFT block replication toggle (automatic, BFT, CFT). 317 | - **Orderer block replication primitive**: The `BlockPuller` is an [abstraction](https://github.com/hyperledger/fabric/blob/main/orderer/consensus/etcdraft/chain.go#L83-L87) that is given to leader based consensus plugins 318 | and is backed by underlying code in the `orderer/common/cluster` package. The abstraction can remain as-is, but its underlying implementation 319 | would need to change in order to be able to pull attestations of block existence (blocks without transactions). 320 | Similarly,the same underlying implementation is used when onboarding orderers. 321 | - **Peer block replication primitive**: The peer currently contains an [implementation](https://github.com/hyperledger/fabric/blob/main/internal/pkg/peer/blocksprovider/blocksprovider.go) that fetches blocks from a single orderer. 322 | The implementation would need to be changed accordingly to protect against block denial of service attacks. 323 | - **Common deliver service**: The [common deliver service](https://github.com/hyperledger/fabric/blob/main/common/deliver/deliver.go), used by both peers and orderers when sending blocks to subscribers, will be enhanced by an ability to 324 | redact the transactions from a block prior to its dispatch based on the intent derived from the `SeekInfo` message as done in SmartBFT. 325 | 326 | This RFC suggests to build a re-usable component for block pulling that will be used by both the peer and the orderer, and it will have a simple API as the [BlockPuller](https://github.com/hyperledger/fabric/blob/main/orderer/common/cluster/deliver.go#L30-L55): 327 | 328 | ``` 329 | // BlockFetcher is used to pull blocks from ordering service nodes 330 | type BlockFetcher interface { 331 | PullBlock(seq uint64) *common.Block 332 | Close() 333 | } 334 | ``` 335 | 336 | The `BlockFetcher` component will perform all bookkeeping required for both CFT and BFT use cases and will then be incorporated in the orderer and peer block replication code. 337 | This will let the project get rid of multiple implementations that perform the same thing, as currently there is a duplicity in the form of the peer deliver client and the orderer's cluster infrastructure. 338 | 339 | # Testing 340 | [testing]: #testing 341 | 342 | All new components build will be designed to be testable and stable, and will have unit tests with good code coverage. 343 | Additionally, until we have a Byzantine Fault Tolerant orderer up and running, integration tests will be made that 344 | artificially sign Fabric blocks and existing peers will pull them from an artificial endpoint, in order to test the block signature validation code works properly. 345 | This will also demonstrate the consensus agnosticism of the approach (since no real orderer nor a consensus protocol will be deployed in the test). 346 | 347 | # Edit History 348 | 349 | - 16 May 2023: Update block attestation implementation to follow the SmartBFT approach. -------------------------------------------------------------------------------- /text/0011-peer-chaincode-optimization.md: -------------------------------------------------------------------------------- 1 | 2 | --- 3 | layout: default 4 | title: Optimization of Communication Protocol between Hyperledger Fabric Endorsing Nodes and Chaincodes 5 | nav_order: 3 6 | --- 7 | 8 | - Feature Name: Peer to Chaincode Communication Optimization 9 | - Start Date: (fill me in with today's date, 2022-03-01) 10 | - RFC PR: (leave this empty) 11 | - Fabric Component: peer, chaincode 12 | - Fabric Issue: (leave this empty) 13 | 14 | 15 | ## Abstract 16 | This RFC proposes an enhancement to the communication protocol between Hyperledger Fabric endorsing nodes and chaincodes by introducing a batching mechanism. The current protocol requires each state modification command, such as `PutState` or `DelState`, to be sent individually, creating unnecessary overhead. This optimization minimizes the frequency and volume of communication between the chaincode and peer nodes, improving performance without changing the transaction semantics or requiring modification. 17 | 18 | --- 19 | 20 | ## Motivation 21 | In the current Hyperledger Fabric architecture, every state-changing operation triggers a distinct message from the chaincode to the peer node. This design causes several inefficiencies: 22 | 23 | 1. **Communication Overhead:** The need to send individual messages for each state change operation increases the total number of messages transmitted, especially for workloads involving many updates. Each message exchange introduces latency and network overhead. 24 | 25 | 2. **Performance Degradation:** Applications that perform multiple state modifications in loops experience high latency due to the large number of message exchanges. As a result, transaction throughput decreases, and system resources are inefficiently utilized. 26 | 27 | ### Objectives of the Optimization 28 | The proposed batching mechanism aims to address these challenges by: 29 | - **Reducing Message Volume:** Grouping multiple state changes into a single `ChangeStateBatch` message reduces the number of messages exchanged between chaincode and peers. 30 | - **Improving Resource Utilization:** By transmitting fewer messages, the network load is reduced, and processing overhead on both the peer and chaincode is minimized. 31 | - **Ensuring Backward Compatibility:** Existing applications can continue to function as before without modification. If the batching feature is not supported by a component, the system will gracefully fall back to the original communication model. 32 | 33 | --- 34 | 35 | ## Problem Illustration with Sample Chaincode 36 | The following chaincode example demonstrates the inefficiencies caused by the current communication protocol. Each iteration of the loop sends an individual `PutState` message to the peer node, resulting in unnecessary overhead: 37 | 38 | ```go 39 | type OnlyPut struct{} 40 | 41 | // Init initializes the chaincode. 42 | func (t *OnlyPut) Init(_ shim.ChaincodeStubInterface) *pb.Response { 43 | return shim.Success(nil) 44 | } 45 | 46 | // Invoke - Entry point for invocations. 47 | func (t *OnlyPut) Invoke(stub shim.ChaincodeStubInterface) *pb.Response { 48 | function, args := stub.GetFunctionAndParameters() 49 | switch function { 50 | case "invoke": 51 | return t.put(stub, args) 52 | default: 53 | return shim.Error("Received unknown function invocation") 54 | } 55 | } 56 | 57 | // Both params should be marshaled JSON data and base64 encoded. 58 | func (t *OnlyPut) put(stub shim.ChaincodeStubInterface, args []string) *pb.Response { 59 | if len(args) != 1 { 60 | return shim.Error("Incorrect number of arguments. Expecting 1") 61 | } 62 | num, _ := strconv.Atoi(args[0]) 63 | for i := 0; i < num; i++ { 64 | key := "key" + strconv.Itoa(i) 65 | if err := stub.PutState(key, []byte(key)); err != nil { 66 | return shim.Error(err.Error()) 67 | } 68 | } 69 | return shim.Success(nil) 70 | } 71 | ``` 72 | 73 | --- 74 | 75 | ## Proposed Solution 76 | 77 | The proposed solution introduces a **batching mechanism** where multiple state changes are grouped into a single `ChangeStateBatch` message. This new message type will encapsulate all state changes and transmit them as a batch to the peer node. During the `Ready` message exchange, the peer and chaincode will negotiate the capability to use batching, ensuring backward compatibility. 78 | 79 | ### New Message Format 80 | ```proto 81 | message ChangeStateBatch { 82 | repeated StateKV kvs = 1; 83 | } 84 | 85 | message StateKV { 86 | enum Type { 87 | UNDEFINED = 0; 88 | PUT_STATE = 9; 89 | DEL_STATE = 10; 90 | PUT_STATE_METADATA = 21; 91 | PURGE_PRIVATE_DATA = 23; 92 | } 93 | 94 | string key = 1; 95 | bytes value = 2; 96 | string collection = 3; 97 | StateMetadata metadata = 4; 98 | Type type = 5; 99 | } 100 | ``` 101 | 102 | --- 103 | 104 | ### Process Flow with ASCII Diagram 105 | The diagram below illustrates the message exchange between the chaincode and peer, showcasing the integration of the batching mechanism: 106 | 107 | ``` 108 | +------------+ +-----------+ 109 | | Chaincode | | Peer | 110 | +------------+ +-----------+ 111 | | | 112 | |--- Ready (UsePutStateBatch, MaxSizePutStateBatch) --> | 113 | | | 114 | |<---- Acknowledgement Ready -------------------------- | 115 | | | 116 | | Batch multiple commands | 117 | | | 118 | |--- ChangeStateBatch (Batch of Commands) -------------> | 119 | | | 120 | |<--- Acknowledgement (Success/Failure) ---------------- | 121 | | | 122 | ``` 123 | 124 | --- 125 | 126 | ## Benchmark Results 127 | 128 | ### Before Optimization 129 | ``` 130 | Only PUT State 131 | Name | N | Min | Median | Mean | StdDev | Max 132 | =========================================================================================== 133 | invoke N-5 cycle-1 [duration] | 5 | 62.9ms | 64.2ms | 437.8ms | 458.4ms | 999.9ms 134 | invoke N-200 cycle-1000 [duration] | 200 | 297ms | 649.5ms | 651ms | 335.6ms | 1.0139s 135 | invoke N-200 cycle-10000 [duration] | 200 | 2.3774s | 2.4011s | 2.4037s | 16.8ms | 2.5557s 136 | ``` 137 | 138 | ### After Optimization 139 | ``` 140 | Only PUT State 141 | Name | N | Min | Median | Mean | StdDev | Max 142 | =========================================================================================== 143 | invoke N-5 cycle-1 [duration] | 5 | 62.2ms | 62.7ms | 437.9ms | 459.7ms | 1.0013s 144 | invoke N-200 cycle-1000 [duration] | 200 | 71.8ms | 541.3ms | 539.4ms | 427.6ms | 1.0085s 145 | invoke N-200 cycle-10000 [duration] | 200 | 176.2ms | 586.4ms | 588.9ms | 336ms | 1.002s 146 | ``` 147 | The cycle of 10000 updates rounds clearly demonstrates the performance improvement of the batching mechanism. 148 | 149 | 150 | ## Order of Repository Changes 151 | 152 | To implement the proposed batching mechanism, modifications need to be made across multiple repositories. Below is the order in which the repositories should be updated to ensure smooth integration: 153 | 154 | 1. github.com/hyperledger/fabric-protos 155 | • Define the new ChangeStateBatch message format. 156 | • Ensure the protobuf schema includes all relevant message fields such as StateKV and Type. 157 | 2. github.com/hyperledger/fabric-chaincode-go 158 | • Update the chaincode framework to support batching logic. 159 | • Add negotiation mechanisms for the Ready message exchange to determine if the batching feature is enabled. 160 | • Modify internal functions to collect state modifications and prepare ChangeStateBatch messages for transmission. 161 | 3. github.com/hyperledger/fabric 162 | • Implement the handler logic to process the new ChangeStateBatch messages. 163 | • Update the peer’s transaction context to support batched state changes. 164 | • Ensure that any failure within a batch rolls back all changes, preserving consistency with existing transactional behavior. 165 | 166 | This sequence ensures that all dependencies are resolved correctly, avoiding integration issues when introducing the batching mechanism. 167 | 168 | --- 169 | 170 | ## Conclusion 171 | The proposed batching mechanism offers a simple yet effective way to optimize communication between Hyperledger Fabric chaincodes and peers. By reducing the number of messages exchanged, the system achieves better performance and resource utilization while maintaining transactional integrity and backward compatibility. 172 | 173 | ## Amendment 174 | 175 | ### Concerns Raised 176 | 177 | During the RFC discussion, the following concerns were raised about the proposed batching mechanism: maintaining the consistency of read-write order, managing existing read-write relationships, ensuring compatibility with current chaincodes, and addressing the impact on interleaved operations[1][2][3]. Altering the sequence of reads and writes during batching could lead to unexpected behavior or different error messages, particularly for applications that rely on the immediate execution of commands. Additionally, operations like SetState involve implicit reads or validations, such as metadata checks or collection name verification, which are currently handled interactively. Transitioning these operations to a batched model introduces complexity and risks compromising correctness. To mitigate these risks, a thorough review of the transaction simulator code is essential, as certain write operations, such as SetState, depend on reading the current state or metadata to function as expected. 178 | 179 | [1]: https://github.com/hyperledger/fabric-rfcs/pull/58#issuecomment-2458605718 180 | [2]: https://github.com/hyperledger/fabric-rfcs/pull/58#issuecomment-2460109295 181 | [3]: https://github.com/hyperledger/fabric-rfcs/pull/58#issuecomment-2473626315 182 | 183 | 184 | ### Proposed Solutions 185 | 186 | 1. Maintaining Read-Write Order 187 | 188 | • Solution: Ensure that the batching mechanism processes operations in the same order as they are invoked in the chaincode. This approach can be achieved by batching only consecutive write operations and sending read operations interactively. 189 | • Implementation: The chaincode shim can collect PutState or DelState operations into a batch, but any GetState operation will immediately flush the current batch and process the read interactively. This guarantees consistency with the current behavior. 190 | 191 | 2. Handling Dependencies and Validation 192 | 193 | • Solution: Retain validation logic (e.g., metadata reads and collection name checks) on the peer side to ensure correctness. Avoid shifting these responsibilities to the shim, as it increases complexity and risk. 194 | • Implementation: The peer should process batched operations in a way that preserves existing checks and validations, ensuring no functional changes to current read-write dependencies. 195 | 196 | 3. Explicit Developer Control 197 | 198 | • Solution: Introduce explicit APIs for developers to enable and manage batching in their chaincode. This gives developers control over when and how batching is applied, allowing for easier debugging and incremental adoption. 199 | • Implementation: APIs like StartBatch and FinishBatch can provide clear demarcation points for batching operations, ensuring developers understand and control the batching process. 200 | 201 | ## Final Solution 202 | 203 | After careful consideration of the various approaches, option #3 - Explicit Developer Control - was selected as the final solution. This choice was made for several key reasons: 204 | 205 | 1. Safety and Predictability: By giving developers explicit control over batching through dedicated APIs, we minimize the risk of unexpected behavior that could arise from implicit batching mechanisms. Developers can clearly understand and control when batching occurs. 206 | 207 | 2. Debugging and Observability: The explicit nature of the batching APIs makes it easier to debug issues by providing clear entry and exit points for batched operations. This visibility is crucial for troubleshooting in production environments. 208 | 209 | 3. Gradual Adoption Path: This approach allows for incremental adoption of the batching feature. Organizations can update their chaincodes at their own pace, testing and validating the performance benefits without forcing a system-wide change. 210 | 211 | 4. Preservation of Existing Semantics: The explicit APIs ensure that the current chaincode behavior remains unchanged unless specifically opted into batching. This maintains compatibility while still offering performance improvements where desired. 212 | 213 | 5. Simplified Implementation: Having clear batching boundaries reduces the complexity of managing state consistency and validation checks, as the system knows exactly when batching begins and ends. 214 | 215 | 216 | ### Extend Chaincode Stub API with Explicit Batching APIs 217 | 218 | Introduce new APIs in the chaincode stub to explicitly control batching operations: 219 | 220 | ```go 221 | // StartWriteBatch initializes a new batch for write operations. 222 | func (stub *ChaincodeStub) StartWriteBatch() { 223 | // Implementation to begin batching write operations. 224 | } 225 | 226 | // FinishWriteBatch flushes the current batch and sends all collected write operations to the peer. 227 | // Returns a serialized set of results for each operation with the corresponding status. 228 | func (stub *ChaincodeStub) FinishWriteBatch() error { 229 | // Implementation to process the batched operations. 230 | } 231 | ``` 232 | 233 | Advantages of Explicit Batching 234 | 235 | 1. Developer Awareness: Developers can explicitly enable batching, ensuring they understand the impact on transaction behavior. 236 | 2. Incremental Adoption: Existing chaincodes can operate without modification, while developers can selectively enable batching for specific chaincodes. 237 | 3. Ease of Debugging: The explicit demarcation of batched operations simplifies debugging, as developers can correlate logs and behavior directly with batching commands. 238 | 4. Backward Compatibility: Systems without batching support will continue to operate as before, ensuring seamless integration. 239 | 240 | ### Example Usage 241 | 242 | ```go 243 | func (t *OnlyPut) put(stub shim.ChaincodeStubInterface, args []string) *pb.Response { 244 | if len(args) != 1 { 245 | return shim.Error("Incorrect number of arguments. Expecting 1") 246 | } 247 | num, _ := strconv.Atoi(args[0]) 248 | 249 | // Start batching write operations 250 | stub.StartWriteBatch() 251 | for i := 0; i < num; i++ { 252 | key := "key" + strconv.Itoa(i) 253 | if err := stub.PutState(key, []byte(key)); err != nil { 254 | return shim.Error(err.Error()) 255 | } 256 | } 257 | // Finish batching and send all operations to the peer 258 | err := stub.FinishWriteBatch() 259 | if err != nil { 260 | return shim.Error(err.Error()) 261 | } 262 | 263 | return shim.Success(nil) 264 | } 265 | ``` 266 | 267 | This approach balances performance optimization with consistency, developer control, and backward compatibility. -------------------------------------------------------------------------------- /text/orderer-v3.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Fabric ordering service consensus agnostic framework 4 | nav_order: 3 5 | --- 6 | 7 | - Feature Name: A consensus agnostic framework for building a Fabric ordering service 8 | - Start Date: (fill me in with today's date, 2022-03-01) 9 | - RFC PR: (leave this empty) 10 | - Fabric Component: orderer 11 | - Fabric Issue: (leave this empty) 12 | 13 | 14 | # Table of Contents 15 | 1. [Summary](#summary) 16 | 2. [Motivation](#motivation) 17 | 3. [Ordering service framework building blocks](#identifying-building-blocks) 18 | 4. [Ordering service node construction](#connecting-the-pieces-together) 19 | 5. [Orderer to orderer communication and authentication](#orderer-to-orderer-authentication-and-communication) 20 | 6. [Backward compatibility and upgrade](#backward-compatibility-and-upgrade-from-a-v2-network-to-a-v3-network) 21 | 7. [Implementation Plan and Testing](#implementation-plan-and-testing) 22 | 23 | # Summary 24 | [summary]: #summary 25 | 26 | The Fabric ordering service node is designed to support multiple consensus protocols through an abstraction of consensus plugins. 27 | Consensus plugins order transactions into blocks on behalf of the Fabric ordering node, and the latter provides consensus plugins with 28 | networking, storage and cryptography software infrastructure. 29 | Unfortunately, the consensus plugin approach not only makes it hard to integrate new types of consensus protocols into the Fabric ordering 30 | node, but also induces un-needed semantic dependency of the rest of the orderer codebase on the consensus plugins. 31 | 32 | This RFC proposes a paradigm for structuring the ordering service node that can be seen as the opposite one to the consensus plugin one. 33 | Simply put, instead of having the ordering service node contain multiple consensus protocol implementations, it proposes to decouple 34 | the ordering service node into discrete software packages and then to have the consensus plugins tie them together, forming ordering 35 | service node binaries per consensus protocol. 36 | 37 | Additionally, the RFC discusses the current approach for orderer to orderer authentication and proposes an alternative design 38 | which is more friendly to consume and deploy. 39 | 40 | This RFC aims to: 41 | 42 | - Define building blocks that consensus protocols may require to be integrated to Fabric. 43 | - Outline how the aforementioned building blocks can be used to construct Fabric ordering service nodes. 44 | - Design a more consumer friendly orderer to orderer authentication scheme. 45 | 46 | # Motivation 47 | [motivation]: #motivation 48 | 49 | 50 | The Hyperledger Fabric ordering service is designed to support various types of consensus protocols. 51 | The support for each consensus protocol is manifested in the form of a "plugin", a package that wraps around a consensus protocol implementation 52 | and connects the rest of the ordering service code base to the consensus protocol. 53 | 54 | The Fabric orderer codebase currently embeds the following consensus plugins: 55 | 56 | - **Solo**: A single node orderer, useful for development, testing and proof of concepts. 57 | - **Kafka**: Several ordering nodes relay transactions into an [Apache Kafka](https://kafka.apache.org/) system, 58 | pull transactions back in the same order and then cut them into identical blocks via synchronization messages also sent through Kafka. 59 | - **Raft**: Several ordering nodes run the [Raft](https://raft.github.io/) protocol by embedding the [Raft library of etcd](https://github.com/etcd-io/etcd/tree/main/raft). 60 | 61 | Apart from the official Hyperledger Fabric consensus implementation, there is also a [Fabric fork](https://github.com/SmartBFT-Go/fabric) of a [BFT library](https://github.com/SmartBFT-Go/consensus) called SmartBFT. 62 | 63 | A BFT consensus protocol might have a different approach for onboarding and state snapshot replication compared to a CFT consensus protocol. 64 | In particular, a state snapshot sent between nodes in CFT is always assumed to be trusted, while in BFT it is not the case. 65 | While it is tempting to have common code that performs state snapshot transfer that caters to both adversarial models, 66 | such code is most likely to be complex and hard to follow and test. 67 | 68 | Each orderer type requires Fabric to understand how to initialize instances of it from the consensus metadata in the configuration blocks. 69 | However, this makes it impossible for external contributors to use their own consensus implementation without either forking Fabric entirely, 70 | or making intrusive changes in the official Fabric orderer and peer code base. 71 | 72 | Undoubtedly, an alternative architecture which doesn't require Fabric codebase changes to add support for new 73 | consensus protocols would be a sought-after property. 74 | 75 | # Removal of deprecated features 76 | [deprecations]: #removal-of-deprecated-features 77 | 78 | Naturally, the Fabric ordering service framework will not contain any consensus specific components. 79 | However, it should be discussed which consensus implementations that exist today would be re-built with it. 80 | 81 | - **Solo**: For the sake of reducing the size of the code to be maintained, the Solo orderer will be discontinued and replaced with deployment of a single Raft orderer. 82 | - **Kafka**: The Kafka ordering service offers no advantages over the Raft ordering service, and as such, it will not be implemented with the new ordering service framework. 83 | 84 | ### Removal of system channel 85 | 86 | Fabric nodes are multi-tenant and each node can simultaneously be part of multiple Fabric blockchains. 87 | Each such a blockchain is termed a "channel" and has its own authentication policies, its own consensus instance and its own ledger. 88 | 89 | Peers join to channels by being given channel configuration blocks to bootstrap from. 90 | Orderers, however, can join channels in two different mechanisms: 91 | 92 | - They can be given channel configuration blocks, and join channels in a similar fashion to peers. 93 | - They can participate in an orderer-only channel, termed the "system channel" which contains transactions that create channels. 94 | 95 | The latter mechanism has a significant drawback compared to the former: All ordering nodes must be part of the system channel. 96 | 97 | Even though the system channel is usually dormant, its very existence is a severe privacy problem. 98 | Consider ordering service nodes each run by a different organization: **O1, O2, O3** and participating in three channels: 99 | A: **O1, O2** 100 | B: **O2, O3** 101 | C: **O3, O1*** 102 | 103 | The very presence of a system channel tells the organization **O2** that **O1** has a business relation with **O3**. 104 | Since by definition there can only be a single system channel for an orderer, then it is impossible for a node in **O1** to hide the existence 105 | of possible interaction with **O3** because both **O1** and **O3** need to be part of the same system channel that **O2** is part of. 106 | 107 | In addition to the aforementioned privacy problem, having two different mechanisms to achieve the same thing only needlessly increases the codebase and complexity. 108 | 109 | Therefore, the new ordering service framework will not support a system channel, and as such, Fabric v3 orderers will not have system channels at all. 110 | As a consequence, before a system channel orderer is to be upgraded to a v3 orderer, the channel participation API needs to be activated beforehand, 111 | and after the upgrade takes place, joining back to a system channel will no longer be possible. 112 | 113 | # Identifying building blocks 114 | [building-blocks]: #identifying-building-blocks 115 | 116 | Apart from consensus, which agrees on orders of transaction, the orderer contains infrastructure that manages various aspects of its lifecycle and operations: 117 | 118 | - **Chain management**: Spawns instances of the consensus protocols per channel and dispatches messages and operations to the right 119 | consensus instance. 120 | - **Block replication service**: Server-side API that sends blocks upon request. 121 | - **Block synchronization utility**: Pulls blocks from other ordering service nodes and appends them to the local ledger. 122 | - **Communication service**: Authenticates client connections, receives messages from remote nodes and sends message to remote nodes. 123 | - **Ledger subsystem**: Appends blocks to persistent storage and retrieves blocks by their index. 124 | - **Channel participation service**: Manages local addition and removal of ledgers per channel. 125 | - **Transaction filtering**: Enforces access control and validity checks on transactions both received from clients or from nodes. 126 | - **Membership Service Provider subsystem**: Classifies identities into principals and drives policy authorization. 127 | 128 | In this RFC, it is proposed that these building blocks be made consensus independent, and the building blocks that cannot be consensus independent will be 129 | part of the consensus plugin that wraps around the consensus independent building blocks. 130 | 131 | Therefore, the new organization of the components in the new ordering service node of a given type will look as follows: 132 | 133 | ``` 134 | ____________________________________________ 135 | | | 136 | | chain management--------consensus instance | 137 | | | | | 138 | | | -----------consensus instance | 139 | | | | | 140 | | |_______ | | 141 | | ________|______________|____ | 142 | | | | | 143 | | |Ordering service framework| | 144 | | | (B) | | 145 | | |__________________________| | 146 | | | 147 | | ordering service node (A) | 148 | |____________________________________________| 149 | ``` 150 | 151 | The ordering service framework (denoted **B**) refers to all building blocks above which are consensus independent. 152 | Wrapping around the components of the ordering service framework is a layer (denoted **A**) which is consensus specific 153 | and includes the chain management, and the consensus instances (one per channel). 154 | 155 | 156 | 157 | A lot of the existing components of the ordering service are rather generic and can be re-used. 158 | 159 | - **Block replication service**: The blocks are read from the ledger, so the code is orthogonal to how blocks are formed. 160 | - **Block synchronization utility**: The current code needs to be copied aside and then restructured to support pulling block attestations. 161 | - **Communication service**: The current communication infrastructure is generic and can be re-used, however an alternative one will be built (see below). 162 | - **Ledger subsystem**: The ledger can be completely re-used in all different consensus implementations. 163 | - **Channel participation service**: The channel participation is administered via a local administrative API, so it's consensus independent. 164 | - **Transaction filtering**: The transaction filtering does not contain any consensus specific logic. Consensus specific orderer logic is to be implemented as a separate layer. 165 | - **Membership Service Provider subsystem**: The MSP is a self-contained object which is re-used throughout the Fabric code base and is not dependent on anything else. 166 | 167 | However, some components of the ordering service would be gone and would need to be implemented by each consensus specific implementation: 168 | 169 | **Chain management**: At a first step, the chain management is not going to be part of the ordering service framework. 170 | However, when the first implementation of an ordering service that utilizes the framework will be implemented, it will be discussed 171 | how to extract even that part to be consensus agnostic. The reason is that the current chain management attempts to support multiple consensus implementation, 172 | as well as support running both with and without the system channel. 173 | 174 | 175 | 176 | # Connecting the pieces together 177 | [connecting]: #connecting-the-pieces-together 178 | 179 | Now, having common and consensus independent building blocks that perform various functions, 180 | an ordering service for a given consensus protocol can be implemented without changing Fabric codebase. 181 | 182 | The developers who implement the ordering service node will just import the needed packages from the Fabric ordering service framework, 183 | and will implement a binary that services the [Atomic Broadcast gRPC API](https://github.com/hyperledger/fabric-protos/blob/main/orderer/ab.proto#L80-L86). 184 | 185 | However, how can we ensure that peers can successfully consume the blocks produced by the ordering service node 186 | which they have no knowledge about its underlying consensus protocol? 187 | 188 | To that end, a [separate RFC](https://github.com/hyperledger/fabric-rfcs/pull/48) which deals with block consumption by peers (and other orderers) was created. 189 | That RFC defines how peers: 190 | - Identify from which ordering service node to pull blocks from and how to detect block withholding. 191 | - Know how many signatures to verify for each block. 192 | 193 | 194 | ### The benefits of the proposed approach 195 | **Ease of integration**: Once peers have a consensus independent mechanism to pulls blocks and verify them, an addition of an ordering service node type does not affect the Fabric core codebase. 196 | In fact, by making the Fabric ordering service framework "import friendly" from external repositories, 197 | the various development teams of the open source community could implement their own consensus protocols that 198 | fits their setting and integration with Fabric peers would not require changes in the Fabric codebase. 199 | 200 | **Proper responsibility for support**: One of the inhibitors for integrating new consensus protocols and libraries into Fabric 201 | is that troubleshooting an issue requires knowledge of the consensus protocol and its implementation. 202 | Since with the proposed approach a new consensus protocol can be introduced to Hyperedger Fabric without influencing the codebase of the Fabric core, 203 | it will not be expected nor required from the Fabric core maintainers to troubleshoot or maintain ordering service implementations 204 | that are not part of the official Fabric core repositories. This fact could expedite the emergence of new scientific and industrial consensus implementations 205 | for Fabric. 206 | 207 | # Orderer-to-orderer authentication and communication 208 | [communication]: #orderer-to-orderer-authentication-and-communication 209 | 210 | ### gRPC API changes 211 | 212 | The current [gRPC API](https://github.com/hyperledger/fabric-protos/blob/release-2.4/orderer/cluster.proto) that orderers use to communicate, declares the channel each message is routed to in the message itself: 213 | ``` 214 | // ConsensusRequest is a consensus specific message sent to a cluster member. 215 | message ConsensusRequest { 216 | string channel = 1; 217 | bytes payload = 2; 218 | bytes metadata = 3; 219 | } 220 | ``` 221 | 222 | In the implementation of the communication API, gRPC streams never share channels and a message for channel *foo* will never be sent through 223 | the same gRPC stream of channel *bar* and vice versa. 224 | 225 | The implemented approach is inefficient as it makes the channel name to be sent over and over redundantly. 226 | We shall define a gRPC API that is similar to the existing one, but with the channel name sent only during authentication (the `Auth` message will contain the channel name): 227 | 228 | ``` 229 | // OrdererMessaging defines communication between orderers. 230 | service OrdererMessaging { 231 | // Connect estabishes a one way stream between orderers 232 | rpc Connect(stream Request) returns (Response); 233 | } 234 | 235 | 236 | // Request wraps a message that is sent to an orderer. 237 | message Request { 238 | oneof payload { 239 | // consensus_request is a consensus specific message. 240 | ConsensusRequest consensus_request = 2; 241 | // submit_request is a relay of a transaction. 242 | SubmitRequest submit_request = 3; 243 | // Auth authenticates the orderer that initiated the stream. 244 | Auth auth = 4; 245 | } 246 | } 247 | 248 | message Auth { 249 | ... 250 | string channel = 7; 251 | ... 252 | } 253 | 254 | ``` 255 | 256 | The `ConsensusRequest` and `SubmitRequest` will be defined similarly as the [v2.x gRPC API](https://github.com/hyperledger/fabric-protos/blob/release-2.4/orderer/cluster.proto), 257 | but without channels in them. The channel will be only encoded in the `Auth` message. The first message that is expected to be sent via 258 | the gRPC stream is an `Auth` message, and the rest are either `ConsensusRequest` or `SubmitRequest` messages. 259 | 260 | 261 | 262 | ### The current authentication mechanism: Mutual TLS with TLS pinning 263 | 264 | The Raft consensus library uses numerical identifiers to represent nodes taking part in the consensus protocol. 265 | When the Fabric runtime receives a request from the Raft library to send a message to a remote node, 266 | it establishes a gRPC stream over mutual TLS with the remote node, and sends the message over the stream. 267 | Similarly, when a message is received from a remote node to a Raft orderer, the Fabric runtime locates the numerical identifier 268 | of the remote node that corresponds to the TLS certificate extracted from the TLS connection and then passes the message tagged with the numerical identifier. 269 | The conversion between the TLS certificate to the numerical identifier is done via a mapping that is found in the configuration blocks. 270 | 271 | Clearly, it is imperative that unauthorized parties will not be able to impersonate ordering service nodes. 272 | When performing in a Byzantine setting where malicious orderers might attempt impersonation, it is vital to authenticate which node 273 | sent the message. While it is possible to have each orderer sign every message it sends, it is inefficient. 274 | 275 | The current authentication mechanism of Fabric orderer-to-orderer communication relies on mutual TLS. 276 | More specifically, whenever an orderer wants to send a message to another orderer, it establishes a TLS connection to the remote orderer and the remote orderer identifies 277 | the connection initiator by its TLS certificate. This is a rather simple approach and its security entirely draws upon the native Go TLS runtime. 278 | 279 | ``` 280 | __________ ___________ 281 | | | ------------------> | | 282 | | orderer1 | <------------------ | orderer2 | 283 | | | | | 284 | |__________| |__________| 285 | ``` 286 | 287 | However, this approach has several drawbacks: 288 | 1. **TLS keys might not be stored in HSM**: The Go TLS stack doesn't provide HSM integration capabilities, and therefore TLS private keys of orderer nodes are more susceptible for theft. 289 | A theft of a TLS private key means that an adversary may send messages on behalf of the victim node, which is catastrophic in a crash fault tolerant (CFT) setting. 290 | Although it is possible to override the `PrivateKey` reference of the `tls.Certifciate` with a signer implementation that communicates with an HSM, existing users which already have key management systems 291 | built will be reluctant to make the required changes. 292 | 293 | 294 | 295 | 2. **Mutual TLS authentication is challenging when combined with reverse proxies**: In case the organization hosting the orderer has a TLS terminating reverse proxy that routes ingress traffic into the organization, 296 | it is challenging for that organization to host an ordering service node that relies on mutual TLS authentication. 297 | 298 | 3. **TLS certificates are often issued by commercial CAs, not internal CAs**: When a certificate is issued by a commercial CA, its lifespan might be significantly shorter. 299 | One of the benefits of having certificates issued by a commercial CA, is that it doesn't require clients to be configured with root TLS CA certificates, since the operating system's TLS CA certificate pool is used instead. 300 | However, commercial CAs such as "Let's Encrypt" issue short term TLS certificates. Although a channel config update is not required when preserving the old public key, this option is left to the mercy of the commercial TLS CA API and implementation 301 | and is not always available. 302 | 303 | 304 | ### The proposed alternative authentication mechanism: Enrollment certificate handshake based authentication 305 | 306 | Fabric has two separate certificate infrastructures: The enrollment certificate infrastructure, which governs signatures and access control, and the TLS infrastructure which governs authentication of TLS sessions. 307 | 308 | Instead of using TLS certificates, we will use enrollment certificates for authentication. Namely, we will have a mapping of enrollment certificates to numerical identifiers in the configuration blocks. 309 | The authentication protocol will consist of a single message that authenticates the initiator node to the responder node. 310 | We do not need to authenticate the responder node to the initiator node, because the gRPC stream is one way. 311 | ```` 312 | Iniatiator Responder 313 | _____________ ____________ 314 | 315 | --------[Auth]------> 316 | 317 | ```` 318 | 319 | **Auth**: The content of the `Auth` message is as follows: 320 | 321 | ``` 322 | message Auth { 323 | uint32 version = 1; 324 | bytes signature = 2; 325 | Timestamp timestamp = 3; 326 | uint64 from_id = 4; 327 | uint64 to_id = 5; 328 | bytes session_binding = 6; 329 | string channel = 7; 330 | } 331 | ``` 332 | 333 | The `from_id` is the numerical identifier of the initiator of the connection. It is needed for the responder to identify from which node 334 | the subsequent messages received after the `Auth` message are sent from. 335 | 336 | The `to_id` is the numerical identifier of the node that is being connected to. It is needed in order for the request to not be re-used for other nodes. 337 | Without `to_id`, the responder node can authenticate to other nodes on behalf of the initiator node. 338 | 339 | The `timestamp` is a timestamp that indicates the freshness of the request, and it needs to be within a margin of the responder's local time. 340 | 341 | The `signature` is the signature that is verifiable under the initiator's public key over the ASN1 encoding of ``. 342 | 343 | The `version` field determines what fields the signature was computed over, and the current version is zero. 344 | 345 | The `session_binding` prevents MITM (Man In The Middle) attacks, and it is explained next. 346 | 347 | 348 | ### Protecting against Man-In-The-Middle attacks 349 | 350 | The safety of the protocol described in the previous paragraph cannot be guaranteed in case the transport layer is hijacked by an adversary. 351 | Consider a scenario where an adversary obtains the private key that corresponds to the TLS certificate of the responder node. 352 | The adversary can hijack the network traffic between the initiator and the responder, wait for the `Auth` message, and then impersonate the 353 | initiator before the responder and impersonate the responder before the initiator by relaying the `Auth` through itself. 354 | 355 | As noted earlier, TLS certificate private keys are more susceptible for theft as they might not be stored in HSM. 356 | In such a case, it is still possible to detect a MITM attack by taking advantage of [Keying Material Exporters](https://datatracker.ietf.org/doc/html/rfc5705). 357 | The [ExportKeyingMaterial](https://pkg.go.dev/crypto/tls#ConnectionState.ExportKeyingMaterial) API allows both ends of a TLS connection to derive a shared key from the 358 | `client_random`, `server_random`, `master_secret` used to establish the TLS session. Since this key is identical to the initiator and responder, 359 | it can be used by the responder to detect whether the message is being relayed by an attacker. 360 | 361 | The detection is quite simple: The initiator node and the receiver node both compute: `r = Hash(version || timestamp || from_id || to_id, channel)` 362 | and then `session_binding = ExportKeyingMaterial(r)`. The initiator node encodes `session_binding` in the `Auth` message, and the responder node 363 | expects the `session_binding` from the `Auth` message to match what it computed on its own. 364 | 365 | In case an attacker manages to obtain the TLS private key of the responder node and performs a MITM attack, the function `ExportKeyingMaterial` 366 | on the initiator node and the real responder node would yield different values, because the `master_secret` will differ. 367 | The `master_secret` will differ because Fabric orderers will be restricted to use ECDHE key exchange. With ECHDE key exchange, a MITM with the same `master_secret` 368 | across both the initiator and responder means that the initiator and responder each picked the public key share, but the attacker managed to compute the shared key, 369 | thus breaking the [CDH assumption](https://en.wikipedia.org/wiki/Computational_Diffie%E2%80%93Hellman_assumption). 370 | 371 | In case the responder node is located behind a TLS terminating reverse proxy, it can be configured to ignore the `session_binding` check, 372 | but the check will be enabled by default. 373 | 374 | # Backward compatibility and upgrade from a v2 network to a v3 network 375 | [upgrade]: #backward-compatibility-and-upgrade-from-a-v2-network-to-a-v3-network 376 | 377 | A Raft orderer implemented using the new ordering service framework will support the v2 channel configuration. 378 | This also means that TLS pinning based authentication will still be an optional communication medium, and it will not be mandatory 379 | to move to enrollment certificate based authentication. 380 | 381 | An upgrade from v2 to v3 will take place by first upgrading the binary, and then activating the v3 capability. 382 | The latter is needed for the channel configuration schema fields introduced in [the block verification RFC](https://github.com/hyperledger/fabric-rfcs/pull/48) to be used. 383 | 384 | # Implementation Plan and Testing 385 | [impl-testing]: #implementation-plan-and-testing 386 | 387 | All new components build will be designed to be testable and stable, and will have unit tests with good code coverage. 388 | As this RFC deals with building a framework, there are no integration tests to be built, but only additional unit tests. 389 | 390 | The Raft ordering service will be re-implemented using the framework, and as such, the existing Raft integration tests will be used 391 | as a sanity test. 392 | 393 | After the work has been completed, any ordering service code that is unrelated to the ordering service framework will be removed from the 394 | Fabric main branch. 395 | --------------------------------------------------------------------------------