├── .github ├── semantic.yml └── workflows │ └── main.yml ├── .gitignore ├── LICENSE ├── Makefile ├── README.md ├── command ├── command.pb.go └── command.proto ├── config.go ├── dispatcher.go ├── dispatcher_test.go ├── docs └── images │ └── dispatcher-architecture.svg ├── example ├── Makefile ├── README.md ├── go.mod ├── go.sum ├── main.go ├── node-follower-conf.yml ├── node-leader-conf.yml └── test.http ├── go.mod ├── go.sum ├── http ├── mocks │ └── mock_store.go ├── service.go └── service_test.go ├── store ├── engine.go ├── engine_test.go ├── fsm.go ├── logstore │ ├── bolt_store.go │ ├── bolt_store_test.go │ └── util.go ├── mocks │ └── mock_distributed_enforcer.go ├── store.go ├── store_test.go ├── stream_layer.go └── stream_layer_test.go └── testdata └── ca ├── ca-config.json ├── ca-csr.json ├── ca-key.pem ├── ca.csr ├── ca.pem ├── generate.sh ├── peer-csr.json ├── peer-key.pem ├── peer.csr └── peer.pem /.github/semantic.yml: -------------------------------------------------------------------------------- 1 | # Always validate the PR title AND all the commits 2 | titleAndCommits: true 3 | -------------------------------------------------------------------------------- /.github/workflows/main.yml: -------------------------------------------------------------------------------- 1 | name: Main 2 | 3 | on: [ push, pull_request ] 4 | 5 | jobs: 6 | test: 7 | runs-on: ubuntu-latest 8 | strategy: 9 | fail-fast: false 10 | matrix: 11 | go: [ 1.14, 1.15, 1.16 ] 12 | steps: 13 | - uses: actions/checkout@v2 14 | 15 | - name: Set up Go 16 | uses: actions/setup-go@v2 17 | with: 18 | go-version: ${{ matrix.go }} 19 | 20 | - name: Run go test 21 | run: make test 22 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Binaries for programs and plugins 2 | *.exe 3 | *.exe~ 4 | *.dll 5 | *.so 6 | *.dylib 7 | 8 | # Test binary, built with `go test -c` 9 | *.test 10 | 11 | # Output of the go coverage tool, specifically when used with LiteIDE 12 | *.out 13 | 14 | # Dependency directories (remove the comment below to include it) 15 | # vendor/ 16 | 17 | .idea 18 | 19 | example/tmp/* 20 | example/example 21 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [2021] [Casbin] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | fmt: 2 | go fmt ./... 3 | 4 | proto: 5 | protoc --go_out=. --go_opt=paths=source_relative ./command/command.proto 6 | 7 | test: 8 | go test -v ./... -p 1 -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # hraft-dispatcher 2 | 3 | [![Go Report Card](https://goreportcard.com/badge/github.com/casbin/hraft-dispatcher)](https://goreportcard.com/report/github.com/casbin/hraft-dispatcher) 4 | [![Build Status](https://github.com/casbin/hraft-dispatcher/actions/workflows/main.yml/badge.svg)](https://github.com/casbin/hraft-dispatcher/actions?workflow=main) 5 | [![Godoc](https://godoc.org/github.com/casbin/hraft-dispatcher?status.svg)](https://pkg.go.dev/github.com/casbin/hraft-dispatcher) 6 | [![Release](https://img.shields.io/github/v/release/casbin/hraft-dispatcher)](https://github.com/casbin/hraft-dispatcher/releases) 7 | [![Sourcegraph](https://sourcegraph.com/github.com/casbin/hraft-dispatcher/-/badge.svg)](https://sourcegraph.com/github.com/casbin/hraft-dispatcher?badge) 8 | [![License](https://img.shields.io/github/license/casbin/hraft-dispatcher?color=blue)](https://github.com/casbin/hraft-dispatcher/blob/main/LICENSE) 9 | 10 | A dispatcher based on Hashicorp's Raft for Casbin. 11 | 12 | ## Project Status 13 | 14 | hraft-dispatcher is beta version. 15 | 16 | ## Getting started 17 | 18 | ### Installation 19 | 20 | Go version 1.14+ and Casbin vervsion 2.24+ is required. 21 | 22 | ```shell 23 | go get github.com/casbin/hraft-dispatcher 24 | ``` 25 | 26 | ### Prerequisite 27 | 28 | You have to provide a completely new Casbin environment without Adapter, all the policies are handled by hraft-dispatcher. 29 | When the leader node starts for the first time, you can add the default policy to hraft-dispatcher. 30 | 31 | ### Example 32 | 33 | An example is provided [here](./example). 34 | 35 | ### Security 36 | 37 | We support enable TLS on HTTP service and Raft service. 38 | If you provide the TLS config is not nil, we will configure this to HTTP service and Raft service, and the HTTP upgrade HTTPS. 39 | 40 | when TLS is enabled, a peer certificate must be provided. It is recommended to use [cfssl](https://github.com/cloudflare/cfssl) to generate this certificate, our generate script is [here](./testdata/ca/generate.sh). 41 | 42 | Here is out configuration, you can find it in [example](./example/main.go): 43 | ```go 44 | tls.Config{ 45 | RootCAs: rootCAPool, 46 | ClientCAs: rootCAPool, 47 | ClientAuth: tls.RequireAndVerifyClientCert, 48 | Certificates: []tls.Certificate{cert}, 49 | } 50 | ``` 51 | 52 | ## Architecture 53 | 54 | hraft-dispatcher is a [dispatcher](https://casbin.org/docs/dispatchers/) plug-in based on [hashicorp/raft](https://github.com/hashicorp/raft) implementation. 55 | 56 | hraft-dispatcher includes an HTTP service, and a Raft service: 57 | 58 | - HTTP service is used to forward data from follower node to follower node 59 | - Raft service is used to maintain the policy consistency of each node 60 | 61 | If you set up a dispatcher in Casbin, it forwards the following request to dispatcher: 62 | 63 | - AddPolicy 64 | - RemovePolicy 65 | - AddPolicies 66 | - RemovePolicies 67 | - RemoveFilteredPolicy 68 | - UpdatePolicy 69 | - UpdatePolicies 70 | - ClearPolicy 71 | 72 | In dispatcher, we are use Raft consensus protocol to maintain the policy, and use the [bbolt](https://github.com/etcd-io/bbolt) to storage the policy of each node. 73 | 74 | hraft-dispatcher overall architecture looks like this: 75 | 76 | ![overall architecture](./docs/images/dispatcher-architecture.svg) 77 | 78 | ## Limitations 79 | 80 | - Adapter: You cannot use Adapter in Casbin, hraft-dispatcher has its own Adapter, which uses the [bbolt](https://github.com/etcd-io/bbolt) to storage the policy. 81 | - You cannot call the following methods, which will affect data consistency: 82 | - LoadPolicy - All policies are maintained by hraft-dispatcher 83 | - SavePolicy - All policies are maintained by hraft-dispatcher 84 | 85 | 86 | ## Project reference 87 | 88 | Much of the inspiration comes from the following projects: 89 | 90 | - [rqlite](https://github.com/rqlite/rqlite) 91 | - [vault](https://github.com/hashicorp/vault) 92 | 93 | Thanks for everyone's contribution. 94 | 95 | ## Contribution 96 | 97 | Thank you for your interest in contributing! 98 | 99 | ## License 100 | 101 | This project is under Apache 2.0 License. See the [LICENSE](LICENSE) file for the full license text. 102 | -------------------------------------------------------------------------------- /command/command.proto: -------------------------------------------------------------------------------- 1 | syntax = "proto3"; 2 | 3 | package command; 4 | 5 | option go_package = "github.com/casbin/hraft-dispatcher/command"; 6 | 7 | message StringArray { 8 | repeated string items = 1; 9 | } 10 | 11 | message AddPoliciesRequest { 12 | string sec = 1; 13 | string pType = 2; 14 | repeated StringArray rules = 3; 15 | } 16 | 17 | message RemovePoliciesRequest { 18 | string sec = 1; 19 | string pType = 2; 20 | repeated StringArray rules = 3; 21 | } 22 | 23 | message RemoveFilteredPolicyRequest { 24 | string sec = 1; 25 | string pType = 2; 26 | int32 fieldIndex = 3; 27 | repeated string fieldValues = 4; 28 | } 29 | 30 | message UpdatePolicyRequest { 31 | string sec = 1; 32 | string pType = 2; 33 | repeated string newRule = 3; 34 | repeated string oldRule = 4; 35 | } 36 | 37 | message UpdatePoliciesRequest { 38 | string sec = 1; 39 | string pType = 2; 40 | repeated StringArray newRules = 3; 41 | repeated StringArray oldRules = 4; 42 | } 43 | 44 | message UpdateFilteredPoliciesRequest { 45 | string sec = 1; 46 | string pType = 2; 47 | repeated StringArray newRules = 3; 48 | repeated StringArray oldRules = 4; 49 | } 50 | 51 | message Command { 52 | enum Type { 53 | COMMAND_TYPE_ADD_POLICIES = 0; 54 | COMMAND_TYPE_REMOVE_POLICIES = 1; 55 | COMMAND_TYPE_REMOVE_FILTERED_POLICY = 2; 56 | COMMAND_TYPE_UPDATE_POLICY = 3; 57 | COMMAND_TYPE_UPDATE_POLICIES = 4; 58 | COMMAND_TYPE_CLEAR_POLICY = 5; 59 | COMMAND_TYPE_UPDATE_FILTERED_POLICIES = 6; 60 | } 61 | 62 | Type type = 1; 63 | bytes data = 2; 64 | } 65 | 66 | message AddNodeRequest { 67 | string id = 1; 68 | string address = 2; 69 | } 70 | 71 | message RemoveNodeRequest { 72 | string id = 1; 73 | } -------------------------------------------------------------------------------- /config.go: -------------------------------------------------------------------------------- 1 | package hraftdispatcher 2 | 3 | import ( 4 | "crypto/tls" 5 | "github.com/casbin/casbin/v2" 6 | "github.com/hashicorp/raft" 7 | ) 8 | 9 | // Config holds dispatcher config. 10 | type Config struct { 11 | // Enforcer is a enforcer of casbin. 12 | Enforcer casbin.IDistributedEnforcer 13 | // ServerID is a unique string identifying this server for all time. 14 | ServerID string 15 | // JoinAddress is used to tells the current node to join an existing cluster. 16 | JoinAddress string 17 | // DataDir holds raft data. 18 | DataDir string 19 | // ListenAddress is a network address for raft server and HTTP(S) server, 20 | // the address is a specified address, such as 10.1.1.19:6780. 21 | ListenAddress string 22 | // TLSConfig is used to configure a TLS server and client. 23 | // If TLSConfig is not nil, we will set TLSConfig to the raft server and the HTTPS server, 24 | // otherwise we will start a server without any security. 25 | // 26 | // Note: 27 | // You have to provide a peer certificate when TLSConfig is not nil, 28 | // we recommend using cfssl tool to create this certificates. 29 | TLSConfig *tls.Config 30 | // RaftConfig provides any necessary configuration for the Raft server. 31 | RaftConfig *raft.Config 32 | } 33 | -------------------------------------------------------------------------------- /dispatcher.go: -------------------------------------------------------------------------------- 1 | package hraftdispatcher 2 | 3 | import ( 4 | "context" 5 | "crypto/tls" 6 | "fmt" 7 | "net" 8 | 9 | "github.com/soheilhy/cmux" 10 | 11 | "github.com/hashicorp/go-multierror" 12 | 13 | "github.com/casbin/casbin/v2/persist" 14 | "github.com/casbin/hraft-dispatcher/command" 15 | "github.com/casbin/hraft-dispatcher/http" 16 | "github.com/casbin/hraft-dispatcher/store" 17 | "github.com/hashicorp/raft" 18 | "github.com/pkg/errors" 19 | "go.uber.org/zap" 20 | ) 21 | 22 | var _ persist.Dispatcher = &HRaftDispatcher{} 23 | 24 | // HRaftDispatcher implements the persist.Dispatcher interface. 25 | type HRaftDispatcher struct { 26 | store http.Store 27 | tlsConfig *tls.Config 28 | httpService *http.Service 29 | shutdownFn func() error 30 | 31 | logger *zap.Logger 32 | } 33 | 34 | // NewHRaftDispatcher returns a HRaftDispatcher. 35 | func NewHRaftDispatcher(config *Config) (*HRaftDispatcher, error) { 36 | return NewHRaftDispatcherWithLogger(config, zap.NewExample()) 37 | } 38 | 39 | // NewHRaftDispatcher returns a HRaftDispatcher. 40 | func NewHRaftDispatcherWithLogger(config *Config, logger *zap.Logger) (*HRaftDispatcher, error) { 41 | if config == nil { 42 | return nil, errors.New("config is not provided") 43 | } 44 | 45 | if config.Enforcer == nil { 46 | return nil, errors.New("Enforcer is not provided in config") 47 | } 48 | 49 | if len(config.DataDir) == 0 { 50 | return nil, errors.New("DataDir is not provided in config") 51 | } 52 | 53 | if len(config.ListenAddress) == 0 { 54 | return nil, errors.New("ListenAddress is not provided in config") 55 | } 56 | 57 | if len(config.ServerID) == 0 { 58 | config.ServerID = config.ListenAddress 59 | } 60 | 61 | if logger == nil { 62 | return nil, errors.New("no logger provided") 63 | } 64 | 65 | // check ListenAddress is network address 66 | listenAddress, err := net.ResolveTCPAddr("tcp", config.ListenAddress) 67 | if err != nil { 68 | return nil, err 69 | } 70 | if listenAddress.IP == nil { 71 | return nil, errors.New("host is omitted in ListenAddress") 72 | } 73 | ip := net.ParseIP(listenAddress.IP.String()) 74 | if ip != nil && ip.IsUnspecified() { 75 | return nil, fmt.Errorf("cannot use unspecified IP %s", ip) 76 | } 77 | 78 | var ln net.Listener 79 | if config.TLSConfig == nil { 80 | ln, err = net.Listen("tcp", config.ListenAddress) 81 | } else { 82 | ln, err = tls.Listen("tcp", config.ListenAddress, config.TLSConfig) 83 | } 84 | if err != nil { 85 | return nil, err 86 | } 87 | 88 | mux := cmux.New(ln) 89 | httpLn := mux.Match(cmux.HTTP1Fast()) 90 | raftLn := mux.Match(cmux.Any()) 91 | go mux.Serve() 92 | 93 | streamLayer, err := store.NewTCPStreamLayer(raftLn, config.TLSConfig) 94 | if err != nil { 95 | return nil, err 96 | } 97 | 98 | storeConfig := &store.Config{ 99 | ID: config.ServerID, 100 | Dir: config.DataDir, 101 | NetworkTransportConfig: &raft.NetworkTransportConfig{ 102 | Stream: streamLayer, 103 | MaxPool: 5, 104 | Logger: nil, 105 | }, 106 | Enforcer: config.Enforcer, 107 | RaftConfig: config.RaftConfig, 108 | } 109 | s, err := store.NewStore(logger, storeConfig) 110 | if err != nil { 111 | logger.Error(err.Error()) 112 | return nil, err 113 | } 114 | 115 | isNewCluster := !s.IsInitializedCluster() 116 | enableBootstrap := false 117 | 118 | if isNewCluster == true { 119 | enableBootstrap = true 120 | } 121 | 122 | if len(config.JoinAddress) != 0 { 123 | enableBootstrap = false 124 | } 125 | 126 | if enableBootstrap { 127 | logger.Info("bootstrapping a new cluster") 128 | } else { 129 | logger.Info("skip bootstrapping a new cluster") 130 | } 131 | 132 | err = s.Start(enableBootstrap) 133 | if err != nil { 134 | logger.Error("failed to start raft service", zap.Error(err)) 135 | return nil, err 136 | } 137 | 138 | if enableBootstrap { 139 | err = s.WaitLeader() 140 | if err != nil { 141 | logger.Error(err.Error()) 142 | } 143 | } 144 | 145 | if isNewCluster && config.JoinAddress != config.ListenAddress && len(config.JoinAddress) != 0 { 146 | logger.Info("start joining the current node to existing cluster") 147 | err = http.DoJoinNodeRequest(config.JoinAddress, config.ServerID, config.ListenAddress, config.TLSConfig) 148 | if err != nil { 149 | logger.Error("failed to join the current node to existing cluster", zap.String("nodeID", config.ServerID), zap.String("nodeAddress", config.ListenAddress), zap.String("clusterAddress", config.JoinAddress), zap.Error(err)) 150 | return nil, err 151 | } 152 | logger.Info("the current node has joined to existing cluster") 153 | } 154 | 155 | httpService, err := http.NewService(logger, httpLn, config.TLSConfig, s) 156 | if err != nil { 157 | return nil, err 158 | } 159 | 160 | err = httpService.Start() 161 | if err != nil { 162 | return nil, err 163 | } 164 | 165 | h := &HRaftDispatcher{ 166 | store: s, 167 | tlsConfig: config.TLSConfig, 168 | httpService: httpService, 169 | logger: logger, 170 | } 171 | 172 | h.shutdownFn = func() error { 173 | var ret error 174 | 175 | err := s.Stop() 176 | if err != nil { 177 | ret = multierror.Append(ret, err) 178 | } 179 | 180 | err = httpService.Stop(context.Background()) 181 | if err != nil { 182 | ret = multierror.Append(ret, err) 183 | } 184 | 185 | err = ln.Close() 186 | if err != nil { 187 | ret = multierror.Append(ret, err) 188 | } 189 | 190 | return ret 191 | } 192 | 193 | return h, nil 194 | } 195 | 196 | // 197 | 198 | //AddPolicies implements the persist.Dispatcher interface. 199 | func (h *HRaftDispatcher) AddPolicies(sec string, pType string, rules [][]string) error { 200 | var items []*command.StringArray 201 | for _, rule := range rules { 202 | var item = &command.StringArray{Items: rule} 203 | items = append(items, item) 204 | } 205 | 206 | addPolicyRequest := &command.AddPoliciesRequest{ 207 | Sec: sec, 208 | PType: pType, 209 | Rules: items, 210 | } 211 | return h.httpService.DoAddPolicyRequest(addPolicyRequest) 212 | } 213 | 214 | // RemovePolicies implements the persist.Dispatcher interface. 215 | func (h *HRaftDispatcher) RemovePolicies(sec string, pType string, rules [][]string) error { 216 | var items []*command.StringArray 217 | for _, rule := range rules { 218 | var item = &command.StringArray{Items: rule} 219 | items = append(items, item) 220 | } 221 | 222 | request := &command.RemovePoliciesRequest{ 223 | Sec: sec, 224 | PType: pType, 225 | Rules: items, 226 | } 227 | return h.httpService.DoRemovePolicyRequest(request) 228 | } 229 | 230 | // RemoveFilteredPolicy implements the persist.Dispatcher interface. 231 | func (h *HRaftDispatcher) RemoveFilteredPolicy(sec string, pType string, fieldIndex int, fieldValues ...string) error { 232 | request := &command.RemoveFilteredPolicyRequest{ 233 | Sec: sec, 234 | PType: pType, 235 | FieldIndex: int32(fieldIndex), 236 | FieldValues: fieldValues, 237 | } 238 | return h.httpService.DoRemoveFilteredPolicyRequest(request) 239 | } 240 | 241 | // ClearPolicy implements the persist.Dispatcher interface. 242 | func (h *HRaftDispatcher) ClearPolicy() error { 243 | return h.httpService.DoClearPolicyRequest() 244 | } 245 | 246 | // UpdatePolicy implements the persist.Dispatcher interface. 247 | func (h *HRaftDispatcher) UpdatePolicy(sec string, pType string, oldRule, newRule []string) error { 248 | request := &command.UpdatePolicyRequest{ 249 | Sec: sec, 250 | PType: pType, 251 | OldRule: oldRule, 252 | NewRule: newRule, 253 | } 254 | return h.httpService.DoUpdatePolicyRequest(request) 255 | } 256 | 257 | // UpdateFilteredPolicies implements the persist.Dispatcher interface. 258 | func (h *HRaftDispatcher) UpdateFilteredPolicies(sec string, pType string, oldRules, newRules [][]string) error { 259 | var olds []*command.StringArray 260 | for _, rule := range oldRules { 261 | var item = &command.StringArray{Items: rule} 262 | olds = append(olds, item) 263 | } 264 | 265 | var news []*command.StringArray 266 | for _, rule := range newRules { 267 | var item = &command.StringArray{Items: rule} 268 | news = append(news, item) 269 | } 270 | request := &command.UpdateFilteredPoliciesRequest{ 271 | Sec: sec, 272 | PType: pType, 273 | OldRules: olds, 274 | NewRules: news, 275 | } 276 | return h.httpService.DoUpdateFilteredPoliciesRequest(request) 277 | } 278 | 279 | // UpdatePolicies implements the persist.Dispatcher interface. 280 | func (h *HRaftDispatcher) UpdatePolicies(sec string, pType string, oldRules, newRules [][]string) error { 281 | var olds []*command.StringArray 282 | for _, rule := range oldRules { 283 | var item = &command.StringArray{Items: rule} 284 | olds = append(olds, item) 285 | } 286 | 287 | var news []*command.StringArray 288 | for _, rule := range newRules { 289 | var item = &command.StringArray{Items: rule} 290 | news = append(news, item) 291 | } 292 | 293 | request := &command.UpdatePoliciesRequest{ 294 | Sec: sec, 295 | PType: pType, 296 | OldRules: olds, 297 | NewRules: news, 298 | } 299 | return h.httpService.DoUpdatePoliciesRequest(request) 300 | } 301 | 302 | // JoinNode joins a node to the current cluster. 303 | func (h *HRaftDispatcher) JoinNode(serverID, serverAddress string) error { 304 | request := &command.AddNodeRequest{ 305 | Id: serverID, 306 | Address: serverAddress, 307 | } 308 | return h.httpService.DoJoinNodeRequest(request) 309 | } 310 | 311 | // JoinNode joins a node from the current cluster. 312 | func (h *HRaftDispatcher) RemoveNode(serverID string) error { 313 | request := &command.RemoveNodeRequest{ 314 | Id: serverID, 315 | } 316 | return h.httpService.DoRemoveNodeRequest(request) 317 | } 318 | 319 | // Shutdown is used to close the http and raft service. 320 | func (h *HRaftDispatcher) Shutdown() error { 321 | return h.shutdownFn() 322 | } 323 | 324 | // Stats is used to get stats of currently service. 325 | func (h *HRaftDispatcher) Stats() (map[string]interface{}, error) { 326 | return h.store.Stats() 327 | } 328 | -------------------------------------------------------------------------------- /dispatcher_test.go: -------------------------------------------------------------------------------- 1 | package hraftdispatcher 2 | 3 | import ( 4 | "crypto/tls" 5 | "crypto/x509" 6 | "github.com/casbin/hraft-dispatcher/store/mocks" 7 | "io/ioutil" 8 | "os" 9 | "testing" 10 | "time" 11 | 12 | "github.com/casbin/casbin/v2" 13 | "github.com/casbin/casbin/v2/model" 14 | . "github.com/smartystreets/goconvey/convey" 15 | "github.com/stretchr/testify/assert" 16 | ) 17 | 18 | func ToGenericArray(arr []string) []interface{} { 19 | s := make([]interface{}, len(arr)) 20 | for i, v := range arr { 21 | s[i] = v 22 | } 23 | return s 24 | } 25 | 26 | func TestDispatcher(t *testing.T) { 27 | var leaderEnforcer casbin.IDistributedEnforcer 28 | var leaderDispatcher *HRaftDispatcher 29 | var followerEnforcer casbin.IDistributedEnforcer 30 | var followerDispatcher *HRaftDispatcher 31 | 32 | dataDir, err := ioutil.TempDir("", "casbin-hraft-dispatcher-") 33 | assert.NoError(t, err) 34 | defer os.RemoveAll(dataDir) 35 | 36 | leaderRaftAddress := "127.0.0.1:6780" 37 | leaderEnforcer, leaderDispatcher, err = newNode(dataDir, leaderRaftAddress, "") 38 | assert.NoError(t, err) 39 | defer leaderDispatcher.Shutdown() 40 | 41 | followerRaftAddress := "127.0.0.1:6790" 42 | followerEnforcer, followerDispatcher, err = newNode(dataDir, followerRaftAddress, leaderRaftAddress) 43 | assert.NoError(t, err) 44 | defer followerDispatcher.Shutdown() 45 | 46 | Convey("test dispatcher", t, func() { 47 | Convey("test in leader node", func() { 48 | Convey("test AddPolicy()", func() { 49 | rules := [][]string{ 50 | {"role:admin", "/", "GET"}, 51 | {"role:admin", "/", "POST"}, 52 | {"role:admin", "/", "PUT"}, 53 | } 54 | for _, rule := range rules { 55 | _, err := leaderEnforcer.AddPolicy(rule) 56 | So(err, ShouldBeNil) 57 | } 58 | 59 | <-time.After(3 * time.Second) 60 | 61 | for _, rule := range rules { 62 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 63 | So(err, ShouldBeNil) 64 | So(ok, ShouldBeTrue) 65 | 66 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 67 | So(err, ShouldBeNil) 68 | So(ok, ShouldBeTrue) 69 | } 70 | }) 71 | 72 | Convey("test UpdatePolicy()", func() { 73 | oldRule := []string{"role:admin", "/", "GET"} 74 | newRule := []string{"role:admin", "/", "DELETE"} 75 | _, err := leaderEnforcer.UpdatePolicy(oldRule, newRule) 76 | So(err, ShouldBeNil) 77 | 78 | <-time.After(3 * time.Second) 79 | 80 | expectedTrue := [][]string{ 81 | {"role:admin", "/", "DELETE"}, 82 | {"role:admin", "/", "POST"}, 83 | {"role:admin", "/", "PUT"}, 84 | } 85 | for _, rule := range expectedTrue { 86 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 87 | So(err, ShouldBeNil) 88 | So(ok, ShouldBeTrue) 89 | 90 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 91 | So(err, ShouldBeNil) 92 | So(ok, ShouldBeTrue) 93 | } 94 | }) 95 | 96 | Convey("test RemovePolicy()", func() { 97 | _, err := leaderEnforcer.RemovePolicy("role:admin", "/", "POST") 98 | So(err, ShouldBeNil) 99 | 100 | <-time.After(3 * time.Second) 101 | 102 | expectedTrueRules := [][]string{ 103 | {"role:admin", "/", "DELETE"}, 104 | {"role:admin", "/", "PUT"}, 105 | } 106 | for _, rule := range expectedTrueRules { 107 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 108 | So(err, ShouldBeNil) 109 | So(ok, ShouldBeTrue) 110 | 111 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 112 | So(err, ShouldBeNil) 113 | So(ok, ShouldBeTrue) 114 | } 115 | 116 | expectedFalseRules := [][]string{ 117 | {"role:admin", "/", "GET"}, 118 | {"role:admin", "/", "POST"}, 119 | } 120 | for _, rule := range expectedFalseRules { 121 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 122 | So(err, ShouldBeNil) 123 | So(ok, ShouldBeFalse) 124 | 125 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 126 | So(err, ShouldBeNil) 127 | So(ok, ShouldBeFalse) 128 | } 129 | }) 130 | 131 | Convey("test ClearPolicy()", func() { 132 | leaderEnforcer.ClearPolicy() 133 | 134 | <-time.After(time.Second * 3) 135 | 136 | rules := [][]string{ 137 | {"role:admin", "/", "GET"}, 138 | {"role:admin", "/", "POST"}, 139 | {"role:admin", "/", "PUT"}, 140 | {"role:admin", "/", "DELETE"}, 141 | } 142 | 143 | for _, rule := range rules { 144 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 145 | So(err, ShouldBeNil) 146 | So(ok, ShouldBeFalse) 147 | 148 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 149 | So(err, ShouldBeNil) 150 | So(ok, ShouldBeFalse) 151 | } 152 | }) 153 | 154 | Convey("test AddPolicies()", func() { 155 | rules := [][]string{ 156 | {"role:admin", "/", "GET"}, 157 | {"role:admin", "/", "POST"}, 158 | {"role:admin", "/", "PUT"}, 159 | {"role:admin", "/", "DELETE"}, 160 | } 161 | 162 | _, err := leaderEnforcer.AddPolicies(rules) 163 | So(err, ShouldBeNil) 164 | 165 | <-time.After(3 * time.Second) 166 | 167 | for _, rule := range rules { 168 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 169 | So(err, ShouldBeNil) 170 | So(ok, ShouldBeTrue) 171 | 172 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 173 | So(err, ShouldBeNil) 174 | So(ok, ShouldBeTrue) 175 | } 176 | }) 177 | 178 | Convey("test UpdatePolicies()", func() { 179 | oldRules := [][]string{{"role:admin", "/", "GET"}, {"role:admin", "/", "POST"}} 180 | newRules := [][]string{{"role:admin", "/admin", "GET"}, {"role:admin", "/admin", "POST"}} 181 | _, err := leaderEnforcer.UpdatePolicies(oldRules, newRules) 182 | So(err, ShouldBeNil) 183 | 184 | <-time.After(3 * time.Second) 185 | 186 | expectedTrue := [][]string{ 187 | {"role:admin", "/admin", "GET"}, 188 | {"role:admin", "/admin", "POST"}, 189 | {"role:admin", "/", "PUT"}, 190 | {"role:admin", "/", "DELETE"}, 191 | } 192 | 193 | for _, rule := range expectedTrue { 194 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 195 | So(err, ShouldBeNil) 196 | So(ok, ShouldBeTrue) 197 | 198 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 199 | So(err, ShouldBeNil) 200 | So(ok, ShouldBeTrue) 201 | } 202 | 203 | expectedFalse := [][]string{ 204 | {"role:admin", "/", "GET"}, 205 | {"role:admin", "/", "POST"}, 206 | } 207 | 208 | for _, rule := range expectedFalse { 209 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 210 | So(err, ShouldBeNil) 211 | So(ok, ShouldBeFalse) 212 | 213 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 214 | So(err, ShouldBeNil) 215 | So(ok, ShouldBeFalse) 216 | } 217 | }) 218 | 219 | Convey("test RemovePolicies()", func() { 220 | rules := [][]string{ 221 | {"role:admin", "/", "GET"}, 222 | {"role:admin", "/", "GET"}, 223 | {"role:admin", "/admin", "GET"}, 224 | {"role:admin", "/admin", "POST"}, 225 | {"role:admin", "/", "PUT"}, 226 | {"role:admin", "/", "DELETE"}, 227 | } 228 | 229 | _, err := leaderEnforcer.RemovePolicies(rules) 230 | So(err, ShouldBeNil) 231 | 232 | <-time.After(3 * time.Second) 233 | 234 | for _, rule := range rules { 235 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 236 | So(err, ShouldBeNil) 237 | So(ok, ShouldBeFalse) 238 | 239 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 240 | So(err, ShouldBeNil) 241 | So(ok, ShouldBeFalse) 242 | } 243 | }) 244 | 245 | Convey("cleanup test", func() { 246 | leaderEnforcer.ClearPolicy() 247 | 248 | <-time.After(time.Second * 3) 249 | }) 250 | }) 251 | 252 | Convey("test in follower node", func() { 253 | Convey("test AddPolicy()", func() { 254 | rules := [][]string{ 255 | {"role:admin", "/", "GET"}, 256 | {"role:admin", "/", "POST"}, 257 | {"role:admin", "/", "PUT"}, 258 | } 259 | for _, rule := range rules { 260 | _, err := followerEnforcer.AddPolicy(rule) 261 | So(err, ShouldBeNil) 262 | } 263 | 264 | <-time.After(3 * time.Second) 265 | 266 | for _, rule := range rules { 267 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 268 | So(err, ShouldBeNil) 269 | So(ok, ShouldBeTrue) 270 | 271 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 272 | So(err, ShouldBeNil) 273 | So(ok, ShouldBeTrue) 274 | } 275 | }) 276 | 277 | Convey("test UpdatePolicy()", func() { 278 | oldRule := []string{"role:admin", "/", "GET"} 279 | newRule := []string{"role:admin", "/", "DELETE"} 280 | _, err := followerEnforcer.UpdatePolicy(oldRule, newRule) 281 | So(err, ShouldBeNil) 282 | 283 | <-time.After(3 * time.Second) 284 | 285 | expectedTrue := [][]string{ 286 | {"role:admin", "/", "DELETE"}, 287 | {"role:admin", "/", "POST"}, 288 | {"role:admin", "/", "PUT"}, 289 | } 290 | for _, rule := range expectedTrue { 291 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 292 | So(err, ShouldBeNil) 293 | So(ok, ShouldBeTrue) 294 | 295 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 296 | So(err, ShouldBeNil) 297 | So(ok, ShouldBeTrue) 298 | } 299 | }) 300 | 301 | Convey("test RemovePolicy()", func() { 302 | _, err := followerEnforcer.RemovePolicy("role:admin", "/", "POST") 303 | So(err, ShouldBeNil) 304 | 305 | <-time.After(3 * time.Second) 306 | 307 | expectedTrueRules := [][]string{ 308 | {"role:admin", "/", "DELETE"}, 309 | {"role:admin", "/", "PUT"}, 310 | } 311 | for _, rule := range expectedTrueRules { 312 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 313 | So(err, ShouldBeNil) 314 | So(ok, ShouldBeTrue) 315 | 316 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 317 | So(err, ShouldBeNil) 318 | So(ok, ShouldBeTrue) 319 | } 320 | 321 | expectedFalseRules := [][]string{ 322 | {"role:admin", "/", "GET"}, 323 | {"role:admin", "/", "POST"}, 324 | } 325 | for _, rule := range expectedFalseRules { 326 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 327 | So(err, ShouldBeNil) 328 | So(ok, ShouldBeFalse) 329 | 330 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 331 | So(err, ShouldBeNil) 332 | So(ok, ShouldBeFalse) 333 | } 334 | }) 335 | 336 | Convey("test ClearPolicy()", func() { 337 | followerEnforcer.ClearPolicy() 338 | 339 | <-time.After(3 * time.Second) 340 | 341 | rules := [][]string{ 342 | {"role:admin", "/", "GET"}, 343 | {"role:admin", "/", "POST"}, 344 | {"role:admin", "/", "PUT"}, 345 | {"role:admin", "/", "DELETE"}, 346 | } 347 | 348 | for _, rule := range rules { 349 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 350 | So(err, ShouldBeNil) 351 | So(ok, ShouldBeFalse) 352 | 353 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 354 | So(err, ShouldBeNil) 355 | So(ok, ShouldBeFalse) 356 | } 357 | }) 358 | 359 | Convey("test AddPolicies()", func() { 360 | rules := [][]string{ 361 | {"role:admin", "/", "GET"}, 362 | {"role:admin", "/", "POST"}, 363 | {"role:admin", "/", "PUT"}, 364 | {"role:admin", "/", "DELETE"}, 365 | } 366 | 367 | _, err := followerEnforcer.AddPolicies(rules) 368 | So(err, ShouldBeNil) 369 | 370 | <-time.After(3 * time.Second) 371 | 372 | for _, rule := range rules { 373 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 374 | So(err, ShouldBeNil) 375 | So(ok, ShouldBeTrue) 376 | 377 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 378 | So(err, ShouldBeNil) 379 | So(ok, ShouldBeTrue) 380 | } 381 | }) 382 | 383 | Convey("test UpdatePolicies()", func() { 384 | oldRules := [][]string{{"role:admin", "/", "GET"}, {"role:admin", "/", "POST"}} 385 | newRules := [][]string{{"role:admin", "/admin", "GET"}, {"role:admin", "/admin", "POST"}} 386 | _, err := followerEnforcer.UpdatePolicies(oldRules, newRules) 387 | So(err, ShouldBeNil) 388 | 389 | <-time.After(3 * time.Second) 390 | 391 | expectedTrue := [][]string{ 392 | {"role:admin", "/admin", "GET"}, 393 | {"role:admin", "/admin", "POST"}, 394 | {"role:admin", "/", "PUT"}, 395 | {"role:admin", "/", "DELETE"}, 396 | } 397 | 398 | for _, rule := range expectedTrue { 399 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 400 | So(err, ShouldBeNil) 401 | So(ok, ShouldBeTrue) 402 | 403 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 404 | So(err, ShouldBeNil) 405 | So(ok, ShouldBeTrue) 406 | } 407 | 408 | expectedFalse := [][]string{ 409 | {"role:admin", "/", "GET"}, 410 | {"role:admin", "/", "POST"}, 411 | } 412 | 413 | for _, rule := range expectedFalse { 414 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 415 | So(err, ShouldBeNil) 416 | So(ok, ShouldBeFalse) 417 | 418 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 419 | So(err, ShouldBeNil) 420 | So(ok, ShouldBeFalse) 421 | } 422 | }) 423 | 424 | Convey("test RemovePolicies()", func() { 425 | rules := [][]string{ 426 | {"role:admin", "/", "GET"}, 427 | {"role:admin", "/", "GET"}, 428 | {"role:admin", "/admin", "GET"}, 429 | {"role:admin", "/admin", "POST"}, 430 | {"role:admin", "/", "PUT"}, 431 | {"role:admin", "/", "DELETE"}, 432 | } 433 | 434 | _, err := followerEnforcer.RemovePolicies(rules) 435 | So(err, ShouldBeNil) 436 | 437 | <-time.After(3 * time.Second) 438 | 439 | for _, rule := range rules { 440 | ok, err := leaderEnforcer.Enforce(ToGenericArray(rule)...) 441 | So(err, ShouldBeNil) 442 | So(ok, ShouldBeFalse) 443 | 444 | ok, err = followerEnforcer.Enforce(ToGenericArray(rule)...) 445 | So(err, ShouldBeNil) 446 | So(ok, ShouldBeFalse) 447 | } 448 | }) 449 | 450 | Convey("cleanup test", func() { 451 | leaderEnforcer.ClearPolicy() 452 | <-time.After(time.Second * 3) 453 | }) 454 | }) 455 | }) 456 | } 457 | 458 | func getTLSConfig() (*tls.Config, error) { 459 | rootCAPool := x509.NewCertPool() 460 | rootCA, err := ioutil.ReadFile("./testdata/ca/ca.pem") 461 | if err != nil { 462 | return nil, err 463 | } 464 | rootCAPool.AppendCertsFromPEM(rootCA) 465 | 466 | cert, err := tls.LoadX509KeyPair("./testdata/ca/peer.pem", "./testdata/ca/peer-key.pem") 467 | if err != nil { 468 | return nil, err 469 | } 470 | 471 | config := &tls.Config{ 472 | RootCAs: rootCAPool, 473 | ClientCAs: rootCAPool, 474 | ClientAuth: tls.RequireAndVerifyClientCert, 475 | Certificates: []tls.Certificate{cert}, 476 | } 477 | 478 | return config, nil 479 | } 480 | 481 | func newNode(dataDir, raftListenAddress, joinAddress string) (casbin.IDistributedEnforcer, *HRaftDispatcher, error) { 482 | var modelText = ` 483 | [request_definition] 484 | r = sub, obj, act 485 | 486 | [policy_definition] 487 | p = sub, obj, act 488 | 489 | [role_definition] 490 | g = _, _ 491 | 492 | [policy_effect] 493 | e = some(where (p.eft == allow)) 494 | 495 | [matchers] 496 | m = g(r.sub, p.sub) && r.obj == p.obj && r.act == p.act 497 | ` 498 | m, err := model.NewModelFromString(modelText) 499 | if err != nil { 500 | return nil, nil, err 501 | } 502 | 503 | e, err := casbin.NewDistributedEnforcer(m) 504 | if err != nil { 505 | return nil, nil, err 506 | } 507 | 508 | tlsConfig, err := getTLSConfig() 509 | if err != nil { 510 | return nil, nil, err 511 | } 512 | 513 | dir, err := ioutil.TempDir(dataDir, "data-") 514 | if err != nil { 515 | return nil, nil, err 516 | } 517 | 518 | dispatcher, err := NewHRaftDispatcher(&Config{ 519 | Enforcer: e, 520 | JoinAddress: joinAddress, 521 | ListenAddress: raftListenAddress, 522 | TLSConfig: tlsConfig, 523 | DataDir: dir, 524 | }) 525 | if err != nil { 526 | return nil, nil, err 527 | } 528 | 529 | e.SetDispatcher(dispatcher) 530 | 531 | return e, dispatcher, nil 532 | } 533 | 534 | func TestNewHRaftDispatcher(t *testing.T) { 535 | _, err := NewHRaftDispatcher(&Config{ 536 | Enforcer: &mocks.MockIDistributedEnforcer{}, 537 | ServerID: "test", 538 | JoinAddress: "", 539 | DataDir: "/tmp/hraft-dispatcher", 540 | ListenAddress: ":6780", 541 | TLSConfig: nil, 542 | RaftConfig: nil, 543 | }) 544 | assert.EqualError(t, err, "host is omitted in ListenAddress") 545 | 546 | _, err = NewHRaftDispatcher(&Config{ 547 | Enforcer: &mocks.MockIDistributedEnforcer{}, 548 | ServerID: "test", 549 | JoinAddress: "", 550 | DataDir: "/tmp/hraft-dispatcher", 551 | ListenAddress: "0.0.0.0:6780", 552 | TLSConfig: nil, 553 | RaftConfig: nil, 554 | }) 555 | assert.EqualError(t, err, "cannot use unspecified IP 0.0.0.0") 556 | } 557 | -------------------------------------------------------------------------------- /docs/images/dispatcher-architecture.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | Produced by OmniGraffle 7.12.1 22 | 2021-03-01 15:16:42 +0000 23 | 24 | 25 | Canvas 1 26 | 27 | Layer 1 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | Casbin 36 | 37 | 38 | 39 | 40 | 41 | 42 | Dispatcher 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | Casbin 53 | 54 | 55 | 56 | 57 | 58 | 59 | Dispatcher 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | Storage 72 | 73 | 74 | 75 | 76 | 77 | Storage 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | Raft 92 | 93 | 94 | 95 | 96 | Node 1 97 | 98 | 99 | 100 | 101 | Node 2 102 | 103 | 104 | 105 | 106 | 107 | -------------------------------------------------------------------------------- /example/Makefile: -------------------------------------------------------------------------------- 1 | APP=example 2 | 3 | clean: 4 | rm -f ./${APP} 5 | 6 | build: 7 | go build -o ${APP} ./main.go 8 | 9 | run-leader: build 10 | ./${APP} --config-file ./node-leader-conf.yml 11 | 12 | run-follower: build 13 | ./${APP} --config-file ./node-follower-conf.yml 14 | -------------------------------------------------------------------------------- /example/README.md: -------------------------------------------------------------------------------- 1 | ## Introduction 2 | 3 | The example provides how to use hraft-dispatcher. We started an HTTP service to forward the received user request to casbin. 4 | 5 | ### Service details 6 | 7 | Leader node: 8 | - HTTP service - 127.0.0.1:6780 9 | - Dispatcher 10 | - Raft service - 127.0.0.1:6781 11 | - HTTP(S) service - 127.0.0.1:6781 12 | 13 | Follower node: 14 | - HTTP service - 127.0.0.1:6790 15 | - Dispatcher 16 | - Raft service - 127.0.0.1:6781 17 | - HTTP(S) service - 127.0.0.1:6781 18 | 19 | ### HTTP API 20 | 21 | #### Add a policy 22 | 23 | --- 24 | 25 | **Method**: `PUT` 26 | **URL**: `/policies` 27 | **Data Type**: `String[]` 28 | **Data Example**: `["role:admin", "/", "GET"]` 29 | 30 | #### Remove a policy 31 | 32 | --- 33 | 34 | **Method**: `DELETE` 35 | **URL**: `/policies` 36 | **Data Type**: `String[]` 37 | **Data Example**: `["role:admin", "/", "GET"]` 38 | 39 | #### Enforcer 40 | 41 | --- 42 | 43 | **Method**: `PUT` 44 | **URL**: `/enforcer` 45 | **Data Type**: `String[]` 46 | **Data Example**: `["role:admin", "/", "GET"]` 47 | 48 | ## Getting started 49 | 50 | To run the leader node: 51 | ```shell 52 | make run-leader 53 | ``` 54 | 55 | To run the follower node: 56 | ```shell 57 | make run-follower 58 | ``` 59 | 60 | The test case is [here](./test.http). 61 | -------------------------------------------------------------------------------- /example/go.mod: -------------------------------------------------------------------------------- 1 | module example 2 | 3 | go 1.16 4 | 5 | require ( 6 | github.com/casbin/casbin/v2 v2.25.6 7 | github.com/casbin/hraft-dispatcher v0.0.7 8 | github.com/go-chi/chi v1.5.4 9 | github.com/json-iterator/go v1.1.10 10 | github.com/pkg/errors v0.9.1 11 | gopkg.in/yaml.v2 v2.4.0 12 | ) 13 | -------------------------------------------------------------------------------- /example/go.sum: -------------------------------------------------------------------------------- 1 | cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= 2 | github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ= 3 | github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= 4 | github.com/DataDog/datadog-go v2.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ= 5 | github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible h1:1G1pk05UrOh0NlF1oeaaix1x8XzrfjIDK47TY0Zehcw= 6 | github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible/go.mod h1:r7JcOSlj0wfOMncg0iLm8Leh48TZaKVeNIfJntJ2wa0= 7 | github.com/armon/go-metrics v0.0.0-20190430140413-ec5e00d3c878 h1:EFSB7Zo9Eg91v7MJPVsifUysc/wPdN+NOnVe6bWbdBM= 8 | github.com/armon/go-metrics v0.0.0-20190430140413-ec5e00d3c878/go.mod h1:3AMJUQhVx52RsWOnlkpikZr01T/yAVN2gn0861vByNg= 9 | github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= 10 | github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps= 11 | github.com/casbin/casbin/v2 v2.24.0/go.mod h1:wUgota0cQbTXE6Vd+KWpg41726jFRi7upxio0sR+Xd0= 12 | github.com/casbin/casbin/v2 v2.25.6 h1:UAPudZTpXWjiOJFkPeZSgoxhxH94L+Hl1ZaMvuyfNxE= 13 | github.com/casbin/casbin/v2 v2.25.6/go.mod h1:wUgota0cQbTXE6Vd+KWpg41726jFRi7upxio0sR+Xd0= 14 | github.com/casbin/hraft-dispatcher v0.0.6 h1:gPDtEV0zhavLyIsZU9NS0Z8/eokDDSZ7SY0mo3/kzBI= 15 | github.com/casbin/hraft-dispatcher v0.0.6/go.mod h1:oFnGwEKMfsOg1T19BWPpYXlZM76E2puOrZzc41n6/RQ= 16 | github.com/casbin/hraft-dispatcher v0.0.7 h1:RH8zjEMLzsU/kn7zIOI4qojEOXtov7+SnK/p/PrycyM= 17 | github.com/casbin/hraft-dispatcher v0.0.7/go.mod h1:oFnGwEKMfsOg1T19BWPpYXlZM76E2puOrZzc41n6/RQ= 18 | github.com/cenkalti/backoff/v4 v4.1.0 h1:c8LkOFQTzuO0WBM/ae5HdGQuZPfPxp7lqBRwQRm4fSc= 19 | github.com/cenkalti/backoff/v4 v4.1.0/go.mod h1:scbssz8iZGpm3xbr14ovlUdkxfGXNInqkPWOWmG2CLw= 20 | github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= 21 | github.com/circonus-labs/circonus-gometrics v2.3.1+incompatible/go.mod h1:nmEj6Dob7S7YxXgwXpfOuvO54S+tGdZdw9fuRZt25Ag= 22 | github.com/circonus-labs/circonusllhist v0.1.3/go.mod h1:kMXHVDlOchFAehlya5ePtbp5jckzBHf4XRpQvBOLI+I= 23 | github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= 24 | github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 25 | github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= 26 | github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 27 | github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= 28 | github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= 29 | github.com/go-chi/chi v1.5.1/go.mod h1:REp24E+25iKvxgeTfHmdUoL5x15kBiDBlnIl5bCwe2k= 30 | github.com/go-chi/chi v1.5.4 h1:QHdzF2szwjqVV4wmByUnTcsbIg7UGaQ0tPF2t5GcAIs= 31 | github.com/go-chi/chi v1.5.4/go.mod h1:uaf8YgoFazUOkPBG7fxPftUylNumIev9awIWOENIuEg= 32 | github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= 33 | github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= 34 | github.com/golang/mock v1.4.4 h1:l75CXGRSwbaYNpl/Z2X1XIIAMSCquvXgpVZDhwEIJsc= 35 | github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4= 36 | github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= 37 | github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= 38 | github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= 39 | github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= 40 | github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= 41 | github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= 42 | github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= 43 | github.com/golang/protobuf v1.4.1 h1:ZFgWrT+bLgsYPirOnRfKLYJLvssAegOj/hgyMFdJZe0= 44 | github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= 45 | github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= 46 | github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= 47 | github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= 48 | github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= 49 | github.com/google/go-cmp v0.5.0 h1:/QaMHBdZ26BB3SSst0Iwl10Epc+xhTquomWX0oZEB6w= 50 | github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= 51 | github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= 52 | github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= 53 | github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8= 54 | github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= 55 | github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA= 56 | github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= 57 | github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= 58 | github.com/hashicorp/go-hclog v0.9.1 h1:9PZfAcVEvez4yhLH2TBU64/h/z4xlFI80cWXRrxuKuM= 59 | github.com/hashicorp/go-hclog v0.9.1/go.mod h1:5CU+agLiy3J7N7QjHK5d05KxGsuXiQLrjA0H7acj2lQ= 60 | github.com/hashicorp/go-immutable-radix v1.0.0 h1:AKDB1HM5PWEA7i4nhcpwOrO2byshxBjXVn/J/3+z5/0= 61 | github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= 62 | github.com/hashicorp/go-msgpack v0.5.5 h1:i9R9JSrqIz0QVLz3sz+i3YJdT7TTSLcfLLzJi9aZTuI= 63 | github.com/hashicorp/go-msgpack v0.5.5/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM= 64 | github.com/hashicorp/go-multierror v1.1.0 h1:B9UzwGQJehnUY1yNrnwREHc3fGbC2xefo8g4TbElacI= 65 | github.com/hashicorp/go-multierror v1.1.0/go.mod h1:spPvp8C1qA32ftKqdAHm4hHTbPw+vmowP0z+KUhOZdA= 66 | github.com/hashicorp/go-retryablehttp v0.5.3/go.mod h1:9B5zBasrRhHXnJnui7y6sL7es7NDiJgTc6Er0maI1Xs= 67 | github.com/hashicorp/go-uuid v1.0.0 h1:RS8zrF7PhGwyNPOtxSClXXj9HA8feRnJzgnI1RJCSnM= 68 | github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= 69 | github.com/hashicorp/golang-lru v0.5.0 h1:CL2msUPvZTLb5O648aiLNJw3hnBxN2+1Jq8rCOH9wdo= 70 | github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= 71 | github.com/hashicorp/raft v1.2.0 h1:mHzHIrF0S91d3A7RPBvuqkgB4d/7oFJZyvf1Q4m7GA0= 72 | github.com/hashicorp/raft v1.2.0/go.mod h1:vPAJM8Asw6u8LxC3eJCUZmRP/E4QmUGE1R7g7k8sG/8= 73 | github.com/hashicorp/raft-boltdb v0.0.0-20171010151810-6e5ba93211ea/go.mod h1:pNv7Wc3ycL6F5oOWn+tPGo2gWD4a5X+yp/ntwdKLjRk= 74 | github.com/json-iterator/go v1.1.10 h1:Kz6Cvnvv2wGdaG/V8yMvfkmNiXq9Ya2KUv4rouJJr68= 75 | github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= 76 | github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= 77 | github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= 78 | github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= 79 | github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI= 80 | github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= 81 | github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= 82 | github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= 83 | github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= 84 | github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= 85 | github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421 h1:ZqeYNhU3OHLH3mGKHDcjJRFFRrJa6eAM5H+CtDdOsPc= 86 | github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= 87 | github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742 h1:Esafd1046DLDQ0W1YjYsBW+p8U2u7vzgW2SQVmlNazg= 88 | github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= 89 | github.com/pascaldekloe/goe v0.1.0 h1:cBOtyMzM9HTpWjXfbbunk26uA6nG3a8n06Wieeh0MwY= 90 | github.com/pascaldekloe/goe v0.1.0/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= 91 | github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 92 | github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= 93 | github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 94 | github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= 95 | github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= 96 | github.com/prometheus/client_golang v0.9.2/go.mod h1:OsXs2jCmiKlQ1lTBmv21f2mNfw4xf/QclQDMrYNZzcM= 97 | github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= 98 | github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= 99 | github.com/prometheus/common v0.0.0-20181126121408-4724e9255275/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= 100 | github.com/prometheus/procfs v0.0.0-20181204211112-1dc9a6cbc91a/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= 101 | github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= 102 | github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d h1:zE9ykElWQ6/NYmHa3jpm/yHnI4xSofP+UP6SpjHcSeM= 103 | github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= 104 | github.com/smartystreets/goconvey v1.6.4 h1:fv0U8FUIMPNf1L9lnHLvLhgicrIVChEkdzIKYqbNC9s= 105 | github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= 106 | github.com/soheilhy/cmux v0.1.4 h1:0HKaf1o97UwFjHH9o5XsHUOF+tqmdA7KEzXLpiyaw0E= 107 | github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= 108 | github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= 109 | github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= 110 | github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= 111 | github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= 112 | github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0= 113 | github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= 114 | github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926/go.mod h1:9ESjWnEqriFuLhtthL60Sar/7RFoluCcXsuvEwTV5KM= 115 | go.etcd.io/bbolt v1.3.5 h1:XAzx9gjCb0Rxj7EoqcClPD1d5ZBxZJk0jbuoPHenBt0= 116 | go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ= 117 | go.uber.org/atomic v1.6.0 h1:Ezj3JGmsOnG1MoRWQkPBsKLe9DwWD9QeXzTRzzldNVk= 118 | go.uber.org/atomic v1.6.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ= 119 | go.uber.org/multierr v1.5.0 h1:KCa4XfM8CWFCpxXRGok+Q0SS/0XBhMDbHHGABQLvD2A= 120 | go.uber.org/multierr v1.5.0/go.mod h1:FeouvMocqHpRaaGuG9EjoKcStLC43Zu/fmqdUMPcKYU= 121 | go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee h1:0mgffUl7nfd+FpvXMVz4IDEaUSmT1ysygQC7qYo7sG4= 122 | go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee/go.mod h1:vJERXedbb3MVM5f9Ejo0C68/HhF8uaILCdgjnY+goOA= 123 | go.uber.org/zap v1.16.0 h1:uFRZXykJGK9lLY4HtgSw44DnIcAM+kRBP7x5m+NpAOM= 124 | go.uber.org/zap v1.16.0/go.mod h1:MA8QOfq0BHJwdXa996Y4dYkAqRKB8/1K1QMMZVaNZjQ= 125 | golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= 126 | golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= 127 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= 128 | golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= 129 | golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= 130 | golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= 131 | golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= 132 | golang.org/x/lint v0.0.0-20190930215403-16217165b5de h1:5hukYrvBGR8/eNkX5mdUezrA6JiaEZDtJb9Ei+1LlBs= 133 | golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= 134 | golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= 135 | golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 136 | golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 137 | golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 138 | golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 139 | golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 140 | golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 141 | golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 142 | golang.org/x/net v0.0.0-20210226172049-e18ecbb05110 h1:qWPm9rbaAMKs8Bq/9LRpbMqxWRVUAQwMI9fVrssnTfw= 143 | golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= 144 | golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= 145 | golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 146 | golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 147 | golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 148 | golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 149 | golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 150 | golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 151 | golang.org/x/sys v0.0.0-20190523142557-0e01d883c5c5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 152 | golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 153 | golang.org/x/sys v0.0.0-20201119102817-f84b799fce68 h1:nxC68pudNYkKU6jWhgrqdreuFiOQWj1Fs7T3VrH4Pjw= 154 | golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 155 | golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= 156 | golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= 157 | golang.org/x/text v0.3.3 h1:cokOdA+Jmi5PJGXLlLllQSgYigAEfHXJAERHVMaCc2k= 158 | golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= 159 | golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 160 | golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 161 | golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= 162 | golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= 163 | golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= 164 | golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= 165 | golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= 166 | golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= 167 | golang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 168 | golang.org/x/tools v0.0.0-20191029190741-b9c20aec41a5 h1:hKsoRgsbwY1NafxrwTs+k64bikrLBkAgPir1TNCj3Zs= 169 | golang.org/x/tools v0.0.0-20191029190741-b9c20aec41a5/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 170 | golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 171 | golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4= 172 | golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 173 | google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= 174 | google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= 175 | google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= 176 | google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= 177 | google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013 h1:+kGHl1aib/qcwaRi1CbqBZ1rk19r85MNUf8HaBghugY= 178 | google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= 179 | google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= 180 | google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= 181 | google.golang.org/grpc v1.27.0 h1:rRYRFMVgRv6E0D70Skyfsr28tDXIuuPZyWGMPdMcnXg= 182 | google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= 183 | google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= 184 | google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= 185 | google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= 186 | google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= 187 | google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= 188 | google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= 189 | google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= 190 | google.golang.org/protobuf v1.25.0 h1:Ejskq+SyPohKW+1uil0JJMtmHCgJPJ/qWTxr8qp+R4c= 191 | google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= 192 | gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 193 | gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY= 194 | gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 195 | gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= 196 | gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 197 | gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= 198 | gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= 199 | gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo= 200 | gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= 201 | honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= 202 | honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= 203 | honnef.co/go/tools v0.0.1-2019.2.3 h1:3JgtbtFHMiCmsznwGVTUWbgGov+pVqnlf1dEJTNAXeM= 204 | honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= 205 | -------------------------------------------------------------------------------- /example/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "crypto/tls" 5 | "crypto/x509" 6 | "flag" 7 | "io/ioutil" 8 | "log" 9 | "net/http" 10 | "os" 11 | "os/signal" 12 | 13 | "github.com/go-chi/chi" 14 | jsoniter "github.com/json-iterator/go" 15 | 16 | "github.com/casbin/casbin/v2" 17 | "github.com/casbin/casbin/v2/model" 18 | hraftdispatcher "github.com/casbin/hraft-dispatcher" 19 | "github.com/pkg/errors" 20 | "gopkg.in/yaml.v2" 21 | ) 22 | 23 | type TLS struct { 24 | RootCert string `yaml:"rootCert"` 25 | Cert string `yaml:"cert"` 26 | Key string `yaml:"key"` 27 | } 28 | 29 | type Config struct { 30 | TLS TLS `yaml:"tls"` 31 | ServerID string `yaml:"serverID"` 32 | DataDir string `yaml:"dataDir"` 33 | JoinAddress string `yaml:"joinAddress"` 34 | ListenAddress string `yaml:"listenAddress"` 35 | 36 | HTTPListenAddress string `yaml:"httpListenAddress"` 37 | } 38 | 39 | func main() { 40 | var configFile string 41 | flag.StringVar(&configFile, "config-file", "", "The path to the configuration file.") 42 | if len(os.Args) < 1 { 43 | log.Fatalf(`Usage: %s --config-file=""`, os.Args[0]) 44 | } 45 | flag.Parse() 46 | 47 | b, err := ioutil.ReadFile(configFile) 48 | if err != nil { 49 | log.Fatal(errors.Wrap(err, "failed to read the configuration filed")) 50 | } 51 | 52 | var config Config 53 | err = yaml.Unmarshal(b, &config) 54 | if err != nil { 55 | log.Fatal(errors.Wrap(err, "failed to unmarshal the configuration")) 56 | } 57 | 58 | // Load cert 59 | rootCAPool := x509.NewCertPool() 60 | rootCA, err := ioutil.ReadFile(config.TLS.RootCert) 61 | if err != nil { 62 | log.Fatal(err) 63 | } 64 | rootCAPool.AppendCertsFromPEM(rootCA) 65 | 66 | cert, err := tls.LoadX509KeyPair(config.TLS.Cert, config.TLS.Key) 67 | if err != nil { 68 | log.Fatal(err) 69 | } 70 | 71 | tlsConfig := &tls.Config{ 72 | RootCAs: rootCAPool, 73 | ClientCAs: rootCAPool, 74 | ClientAuth: tls.RequireAndVerifyClientCert, 75 | Certificates: []tls.Certificate{cert}, 76 | } 77 | 78 | // New a Enforcer 79 | var modelText = ` 80 | [request_definition] 81 | r = sub, obj, act 82 | 83 | [policy_definition] 84 | p = sub, obj, act 85 | 86 | [role_definition] 87 | g = _, _ 88 | 89 | [policy_effect] 90 | e = some(where (p.eft == allow)) 91 | 92 | [matchers] 93 | m = g(r.sub, p.sub) && r.obj == p.obj && r.act == p.act 94 | ` 95 | m, err := model.NewModelFromString(modelText) 96 | if err != nil { 97 | log.Fatal(err) 98 | } 99 | 100 | // Adapter is not required here. 101 | // Dispatcher = Adapter + Watcher 102 | e, err := casbin.NewDistributedEnforcer(m) 103 | if err != nil { 104 | log.Fatal(err) 105 | } 106 | 107 | // New a Dispatcher 108 | dispatcher, err := hraftdispatcher.NewHRaftDispatcher(&hraftdispatcher.Config{ 109 | Enforcer: e, 110 | JoinAddress: config.JoinAddress, 111 | ListenAddress: config.ListenAddress, 112 | TLSConfig: tlsConfig, 113 | DataDir: config.DataDir, 114 | }) 115 | if err != nil { 116 | log.Fatal(err) 117 | } 118 | 119 | e.SetDispatcher(dispatcher) 120 | 121 | mountHTTP(config.HTTPListenAddress, e) 122 | 123 | done := make(chan bool, 1) 124 | quit := make(chan os.Signal, 1) 125 | 126 | signal.Notify(quit, os.Interrupt) 127 | 128 | go func() { 129 | <-quit 130 | log.Println("server is shutting down...") 131 | 132 | if err := dispatcher.Shutdown(); err != nil { 133 | log.Fatalf("failed to shutdown the server: %v\n", err) 134 | } 135 | close(done) 136 | }() 137 | 138 | <-done 139 | log.Println("server stopped") 140 | } 141 | 142 | func mountHTTP(address string, enforcer casbin.IDistributedEnforcer) { 143 | r := chi.NewRouter() 144 | r.Put("/policies", func(writer http.ResponseWriter, request *http.Request) { 145 | json, err := ioutil.ReadAll(request.Body) 146 | if err != nil { 147 | http.Error(writer, err.Error(), http.StatusBadRequest) 148 | return 149 | } 150 | var data []interface{} 151 | err = jsoniter.Unmarshal(json, &data) 152 | if err != nil { 153 | http.Error(writer, err.Error(), http.StatusBadRequest) 154 | return 155 | } 156 | _, err = enforcer.AddPolicy(data...) 157 | if err != nil { 158 | http.Error(writer, err.Error(), http.StatusServiceUnavailable) 159 | return 160 | } 161 | }) 162 | 163 | r.Delete("/policies", func(writer http.ResponseWriter, request *http.Request) { 164 | json, err := ioutil.ReadAll(request.Body) 165 | if err != nil { 166 | http.Error(writer, err.Error(), http.StatusBadRequest) 167 | return 168 | } 169 | var data []interface{} 170 | err = jsoniter.Unmarshal(json, &data) 171 | if err != nil { 172 | http.Error(writer, err.Error(), http.StatusBadRequest) 173 | return 174 | } 175 | _, err = enforcer.RemovePolicy(data...) 176 | if err != nil { 177 | http.Error(writer, err.Error(), http.StatusServiceUnavailable) 178 | return 179 | } 180 | }) 181 | 182 | r.Put("/enforcer", func(writer http.ResponseWriter, request *http.Request) { 183 | json, err := ioutil.ReadAll(request.Body) 184 | if err != nil { 185 | http.Error(writer, err.Error(), http.StatusBadRequest) 186 | return 187 | } 188 | var data []interface{} 189 | err = jsoniter.Unmarshal(json, &data) 190 | if err != nil { 191 | http.Error(writer, err.Error(), http.StatusBadRequest) 192 | return 193 | } 194 | ok, err := enforcer.Enforce(data...) 195 | if err != nil { 196 | http.Error(writer, err.Error(), http.StatusServiceUnavailable) 197 | return 198 | } 199 | if ok { 200 | writer.Write([]byte("Authorized")) 201 | } else { 202 | writer.Write([]byte("Unauthorized")) 203 | } 204 | }) 205 | 206 | go func() { 207 | err := http.ListenAndServe(address, r) 208 | if err != http.ErrServerClosed { 209 | log.Fatal(err) 210 | } 211 | }() 212 | } 213 | -------------------------------------------------------------------------------- /example/node-follower-conf.yml: -------------------------------------------------------------------------------- 1 | httpListenAddress: 127.0.0.1:6790 2 | 3 | # Dispatcher config 4 | tls: 5 | rootCert: ../testdata/ca/ca.pem 6 | key: ../testdata/ca/peer-key.pem 7 | cert: ../testdata/ca/peer.pem 8 | 9 | serverID: node-leader 10 | dataDir: ./tmp/follow-data 11 | listenAddress: 127.0.0.1:6791 12 | joinAddress: 127.0.0.1:6781 13 | -------------------------------------------------------------------------------- /example/node-leader-conf.yml: -------------------------------------------------------------------------------- 1 | httpListenAddress: 127.0.0.1:6780 2 | 3 | # Dispatcher config 4 | tls: 5 | rootCert: ../testdata/ca/ca.pem 6 | key: ../testdata/ca/peer-key.pem 7 | cert: ../testdata/ca/peer.pem 8 | 9 | serverID: node-leader 10 | dataDir: ./tmp/leader-data 11 | listenAddress: 127.0.0.1:6781 12 | joinAddress: "" 13 | 14 | -------------------------------------------------------------------------------- /example/test.http: -------------------------------------------------------------------------------- 1 | ### add a rule in leader node 2 | PUT http://127.0.0.1:6780/policies 3 | Content-Type: application/json 4 | 5 | [ 6 | "role:admin", 7 | "/", 8 | "GET" 9 | ] 10 | 11 | > {% 12 | client.test("Request executed successfully", function() { 13 | client.assert(response.status === 200); 14 | }); 15 | %} 16 | 17 | ### add a rule in follower node 18 | PUT http://127.0.0.1:6790/policies 19 | Content-Type: application/json 20 | 21 | [ 22 | "role:admin", 23 | "/", 24 | "POST" 25 | ] 26 | 27 | > {% 28 | client.test("Request executed successfully", function() { 29 | client.assert(response.status === 200); 30 | }); 31 | %} 32 | 33 | ### enforcer a rule in leader node 34 | PUT http://127.0.0.1:6780/enforcer 35 | Content-Type: application/json 36 | 37 | [ 38 | "role:admin", 39 | "/", 40 | "GET" 41 | ] 42 | 43 | > {% 44 | client.test("Request executed successfully", function() { 45 | client.assert(response.body === "Authorized"); 46 | }); 47 | %} 48 | 49 | ### enforcer a rule in leader node 50 | PUT http://127.0.0.1:6780/enforcer 51 | Content-Type: application/json 52 | 53 | [ 54 | "role:admin", 55 | "/", 56 | "POST" 57 | ] 58 | 59 | > {% 60 | client.test("Request executed successfully", function() { 61 | client.assert(response.body === "Authorized"); 62 | }); 63 | %} 64 | 65 | ### remove a rule in leader node 66 | DELETE http://127.0.0.1:6790/policies 67 | Content-Type: application/json 68 | 69 | [ 70 | "role:admin", 71 | "/", 72 | "POST" 73 | ] 74 | 75 | > {% 76 | client.test("Request executed successfully", function() { 77 | client.assert(response.status === 200); 78 | }); 79 | %} 80 | 81 | ### remove a rule in follower node 82 | DELETE http://127.0.0.1:6790/policies 83 | Content-Type: application/json 84 | 85 | [ 86 | "role:admin", 87 | "/", 88 | "GET" 89 | ] 90 | 91 | > {% 92 | client.test("Request executed successfully", function() { 93 | client.assert(response.status === 200); 94 | }); 95 | %} 96 | 97 | 98 | ### enforcer a rule in leader node 99 | PUT http://127.0.0.1:6780/enforcer 100 | Content-Type: application/json 101 | 102 | [ 103 | "role:admin", 104 | "/", 105 | "POST" 106 | ] 107 | 108 | > {% 109 | client.test("Request executed successfully", function() { 110 | client.assert(response.body === "Unauthorized"); 111 | }); 112 | %} 113 | 114 | ### enforcer a rule in follower node 115 | PUT http://127.0.0.1:6790/enforcer 116 | Content-Type: application/json 117 | 118 | [ 119 | "role:admin", 120 | "/", 121 | "GET" 122 | ] 123 | 124 | > {% 125 | client.test("Request executed successfully", function() { 126 | client.assert(response.body === "Unauthorized"); 127 | }); 128 | %} 129 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/casbin/hraft-dispatcher 2 | 3 | go 1.15 4 | 5 | require ( 6 | github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible 7 | github.com/casbin/casbin/v2 v2.35.0 8 | github.com/cenkalti/backoff/v4 v4.1.0 9 | github.com/go-chi/chi v1.5.1 10 | github.com/golang/mock v1.4.4 11 | github.com/hashicorp/go-msgpack v0.5.5 12 | github.com/hashicorp/go-multierror v1.1.0 13 | github.com/hashicorp/raft v1.2.0 14 | github.com/json-iterator/go v1.1.10 15 | github.com/pkg/errors v0.8.1 16 | github.com/smartystreets/goconvey v1.6.4 17 | github.com/soheilhy/cmux v0.1.4 18 | github.com/stretchr/testify v1.6.1 19 | go.etcd.io/bbolt v1.3.5 20 | go.uber.org/zap v1.16.0 21 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 // indirect 22 | golang.org/x/net v0.0.0-20210226172049-e18ecbb05110 23 | google.golang.org/protobuf v1.25.0 24 | gopkg.in/yaml.v2 v2.4.0 25 | ) 26 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= 2 | github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ= 3 | github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= 4 | github.com/DataDog/datadog-go v2.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ= 5 | github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible h1:1G1pk05UrOh0NlF1oeaaix1x8XzrfjIDK47TY0Zehcw= 6 | github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible/go.mod h1:r7JcOSlj0wfOMncg0iLm8Leh48TZaKVeNIfJntJ2wa0= 7 | github.com/armon/go-metrics v0.0.0-20190430140413-ec5e00d3c878 h1:EFSB7Zo9Eg91v7MJPVsifUysc/wPdN+NOnVe6bWbdBM= 8 | github.com/armon/go-metrics v0.0.0-20190430140413-ec5e00d3c878/go.mod h1:3AMJUQhVx52RsWOnlkpikZr01T/yAVN2gn0861vByNg= 9 | github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= 10 | github.com/boltdb/bolt v1.3.1 h1:JQmyP4ZBrce+ZQu0dY660FMfatumYDLun9hBCUVIkF4= 11 | github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps= 12 | github.com/casbin/casbin/v2 v2.23.4 h1:izvAG3KA49C3/m1zpYfkLcZlkYQO5VeHj7dhwurwZR4= 13 | github.com/casbin/casbin/v2 v2.23.4/go.mod h1:wUgota0cQbTXE6Vd+KWpg41726jFRi7upxio0sR+Xd0= 14 | github.com/casbin/casbin/v2 v2.24.0 h1:peiFTw+PNpAMSz7hDDz768nEwzUsBMMZNV4yFGCayI8= 15 | github.com/casbin/casbin/v2 v2.24.0/go.mod h1:wUgota0cQbTXE6Vd+KWpg41726jFRi7upxio0sR+Xd0= 16 | github.com/casbin/casbin/v2 v2.35.0 h1:f0prVg9LgTJTihjAxWEZhfJptXvah1GpZh12sb5KXNA= 17 | github.com/casbin/casbin/v2 v2.35.0/go.mod h1:vByNa/Fchek0KZUgG5wEsl7iFsiviAYKRtgrQfcJqHg= 18 | github.com/cenkalti/backoff/v4 v4.1.0 h1:c8LkOFQTzuO0WBM/ae5HdGQuZPfPxp7lqBRwQRm4fSc= 19 | github.com/cenkalti/backoff/v4 v4.1.0/go.mod h1:scbssz8iZGpm3xbr14ovlUdkxfGXNInqkPWOWmG2CLw= 20 | github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= 21 | github.com/circonus-labs/circonus-gometrics v2.3.1+incompatible/go.mod h1:nmEj6Dob7S7YxXgwXpfOuvO54S+tGdZdw9fuRZt25Ag= 22 | github.com/circonus-labs/circonusllhist v0.1.3/go.mod h1:kMXHVDlOchFAehlya5ePtbp5jckzBHf4XRpQvBOLI+I= 23 | github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= 24 | github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 25 | github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= 26 | github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 27 | github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= 28 | github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= 29 | github.com/go-chi/chi v1.5.1 h1:kfTK3Cxd/dkMu/rKs5ZceWYp+t5CtiE7vmaTv3LjC6w= 30 | github.com/go-chi/chi v1.5.1/go.mod h1:REp24E+25iKvxgeTfHmdUoL5x15kBiDBlnIl5bCwe2k= 31 | github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= 32 | github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= 33 | github.com/golang/mock v1.4.4 h1:l75CXGRSwbaYNpl/Z2X1XIIAMSCquvXgpVZDhwEIJsc= 34 | github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4= 35 | github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM= 36 | github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= 37 | github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= 38 | github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= 39 | github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= 40 | github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= 41 | github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= 42 | github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= 43 | github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= 44 | github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= 45 | github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= 46 | github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= 47 | github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= 48 | github.com/google/go-cmp v0.5.0 h1:/QaMHBdZ26BB3SSst0Iwl10Epc+xhTquomWX0oZEB6w= 49 | github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= 50 | github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= 51 | github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= 52 | github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8= 53 | github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= 54 | github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA= 55 | github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= 56 | github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= 57 | github.com/hashicorp/go-hclog v0.9.1 h1:9PZfAcVEvez4yhLH2TBU64/h/z4xlFI80cWXRrxuKuM= 58 | github.com/hashicorp/go-hclog v0.9.1/go.mod h1:5CU+agLiy3J7N7QjHK5d05KxGsuXiQLrjA0H7acj2lQ= 59 | github.com/hashicorp/go-immutable-radix v1.0.0 h1:AKDB1HM5PWEA7i4nhcpwOrO2byshxBjXVn/J/3+z5/0= 60 | github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= 61 | github.com/hashicorp/go-msgpack v0.5.5 h1:i9R9JSrqIz0QVLz3sz+i3YJdT7TTSLcfLLzJi9aZTuI= 62 | github.com/hashicorp/go-msgpack v0.5.5/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM= 63 | github.com/hashicorp/go-multierror v1.1.0 h1:B9UzwGQJehnUY1yNrnwREHc3fGbC2xefo8g4TbElacI= 64 | github.com/hashicorp/go-multierror v1.1.0/go.mod h1:spPvp8C1qA32ftKqdAHm4hHTbPw+vmowP0z+KUhOZdA= 65 | github.com/hashicorp/go-retryablehttp v0.5.3/go.mod h1:9B5zBasrRhHXnJnui7y6sL7es7NDiJgTc6Er0maI1Xs= 66 | github.com/hashicorp/go-uuid v1.0.0 h1:RS8zrF7PhGwyNPOtxSClXXj9HA8feRnJzgnI1RJCSnM= 67 | github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= 68 | github.com/hashicorp/golang-lru v0.5.0 h1:CL2msUPvZTLb5O648aiLNJw3hnBxN2+1Jq8rCOH9wdo= 69 | github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= 70 | github.com/hashicorp/raft v1.2.0 h1:mHzHIrF0S91d3A7RPBvuqkgB4d/7oFJZyvf1Q4m7GA0= 71 | github.com/hashicorp/raft v1.2.0/go.mod h1:vPAJM8Asw6u8LxC3eJCUZmRP/E4QmUGE1R7g7k8sG/8= 72 | github.com/hashicorp/raft-boltdb v0.0.0-20171010151810-6e5ba93211ea h1:xykPFhrBAS2J0VBzVa5e80b5ZtYuNQtgXjN40qBZlD4= 73 | github.com/hashicorp/raft-boltdb v0.0.0-20171010151810-6e5ba93211ea/go.mod h1:pNv7Wc3ycL6F5oOWn+tPGo2gWD4a5X+yp/ntwdKLjRk= 74 | github.com/json-iterator/go v1.1.10 h1:Kz6Cvnvv2wGdaG/V8yMvfkmNiXq9Ya2KUv4rouJJr68= 75 | github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= 76 | github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= 77 | github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= 78 | github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= 79 | github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI= 80 | github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= 81 | github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= 82 | github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= 83 | github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= 84 | github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= 85 | github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421 h1:ZqeYNhU3OHLH3mGKHDcjJRFFRrJa6eAM5H+CtDdOsPc= 86 | github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= 87 | github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742 h1:Esafd1046DLDQ0W1YjYsBW+p8U2u7vzgW2SQVmlNazg= 88 | github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= 89 | github.com/pascaldekloe/goe v0.1.0 h1:cBOtyMzM9HTpWjXfbbunk26uA6nG3a8n06Wieeh0MwY= 90 | github.com/pascaldekloe/goe v0.1.0/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= 91 | github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I= 92 | github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 93 | github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= 94 | github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= 95 | github.com/prometheus/client_golang v0.9.2/go.mod h1:OsXs2jCmiKlQ1lTBmv21f2mNfw4xf/QclQDMrYNZzcM= 96 | github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= 97 | github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= 98 | github.com/prometheus/common v0.0.0-20181126121408-4724e9255275/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= 99 | github.com/prometheus/procfs v0.0.0-20181204211112-1dc9a6cbc91a/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= 100 | github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= 101 | github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d h1:zE9ykElWQ6/NYmHa3jpm/yHnI4xSofP+UP6SpjHcSeM= 102 | github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= 103 | github.com/smartystreets/goconvey v1.6.4 h1:fv0U8FUIMPNf1L9lnHLvLhgicrIVChEkdzIKYqbNC9s= 104 | github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= 105 | github.com/soheilhy/cmux v0.1.4 h1:0HKaf1o97UwFjHH9o5XsHUOF+tqmdA7KEzXLpiyaw0E= 106 | github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= 107 | github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= 108 | github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= 109 | github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= 110 | github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= 111 | github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0= 112 | github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= 113 | github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926/go.mod h1:9ESjWnEqriFuLhtthL60Sar/7RFoluCcXsuvEwTV5KM= 114 | go.etcd.io/bbolt v1.3.5 h1:XAzx9gjCb0Rxj7EoqcClPD1d5ZBxZJk0jbuoPHenBt0= 115 | go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ= 116 | go.uber.org/atomic v1.6.0 h1:Ezj3JGmsOnG1MoRWQkPBsKLe9DwWD9QeXzTRzzldNVk= 117 | go.uber.org/atomic v1.6.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ= 118 | go.uber.org/multierr v1.5.0 h1:KCa4XfM8CWFCpxXRGok+Q0SS/0XBhMDbHHGABQLvD2A= 119 | go.uber.org/multierr v1.5.0/go.mod h1:FeouvMocqHpRaaGuG9EjoKcStLC43Zu/fmqdUMPcKYU= 120 | go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee h1:0mgffUl7nfd+FpvXMVz4IDEaUSmT1ysygQC7qYo7sG4= 121 | go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee/go.mod h1:vJERXedbb3MVM5f9Ejo0C68/HhF8uaILCdgjnY+goOA= 122 | go.uber.org/zap v1.16.0 h1:uFRZXykJGK9lLY4HtgSw44DnIcAM+kRBP7x5m+NpAOM= 123 | go.uber.org/zap v1.16.0/go.mod h1:MA8QOfq0BHJwdXa996Y4dYkAqRKB8/1K1QMMZVaNZjQ= 124 | golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= 125 | golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= 126 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= 127 | golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= 128 | golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= 129 | golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= 130 | golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= 131 | golang.org/x/lint v0.0.0-20190930215403-16217165b5de h1:5hukYrvBGR8/eNkX5mdUezrA6JiaEZDtJb9Ei+1LlBs= 132 | golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= 133 | golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= 134 | golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 135 | golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 136 | golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 137 | golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 138 | golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 139 | golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 140 | golang.org/x/net v0.0.0-20190620200207-3b0461eec859 h1:R/3boaszxrf1GEUWTVDzSKVwLmSJpwZ1yqXm8j0v2QI= 141 | golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 142 | golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb h1:eBmm0M9fYhWpKZLjQUUKka/LtIxf46G4fxeEz5KJr9U= 143 | golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= 144 | golang.org/x/net v0.0.0-20210226172049-e18ecbb05110 h1:qWPm9rbaAMKs8Bq/9LRpbMqxWRVUAQwMI9fVrssnTfw= 145 | golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= 146 | golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= 147 | golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 148 | golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 149 | golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 150 | golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 151 | golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 152 | golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 153 | golang.org/x/sys v0.0.0-20190523142557-0e01d883c5c5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 154 | golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 155 | golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f h1:+Nyd8tzPX9R7BWHguqsrbFdRx3WQ/1ib8I44HXV5yTA= 156 | golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 157 | golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 158 | golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= 159 | golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg= 160 | golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= 161 | golang.org/x/text v0.3.3 h1:cokOdA+Jmi5PJGXLlLllQSgYigAEfHXJAERHVMaCc2k= 162 | golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= 163 | golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 164 | golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 165 | golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= 166 | golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= 167 | golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= 168 | golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= 169 | golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= 170 | golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= 171 | golang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 172 | golang.org/x/tools v0.0.0-20191029190741-b9c20aec41a5 h1:hKsoRgsbwY1NafxrwTs+k64bikrLBkAgPir1TNCj3Zs= 173 | golang.org/x/tools v0.0.0-20191029190741-b9c20aec41a5/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 174 | golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 175 | golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4= 176 | golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 177 | google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= 178 | google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= 179 | google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= 180 | google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= 181 | google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= 182 | google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= 183 | google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= 184 | google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= 185 | google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= 186 | google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= 187 | google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= 188 | google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= 189 | google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= 190 | google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= 191 | google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= 192 | google.golang.org/protobuf v1.25.0 h1:Ejskq+SyPohKW+1uil0JJMtmHCgJPJ/qWTxr8qp+R4c= 193 | google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= 194 | gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 195 | gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY= 196 | gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 197 | gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= 198 | gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw= 199 | gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 200 | gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= 201 | gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= 202 | gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo= 203 | gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= 204 | honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= 205 | honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= 206 | honnef.co/go/tools v0.0.1-2019.2.3 h1:3JgtbtFHMiCmsznwGVTUWbgGov+pVqnlf1dEJTNAXeM= 207 | honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= 208 | -------------------------------------------------------------------------------- /http/mocks/mock_store.go: -------------------------------------------------------------------------------- 1 | // Code generated by MockGen. DO NOT EDIT. 2 | // Source: service.go 3 | 4 | // Package mocks is a generated GoMock package. 5 | package mocks 6 | 7 | import ( 8 | reflect "reflect" 9 | 10 | command "github.com/casbin/hraft-dispatcher/command" 11 | gomock "github.com/golang/mock/gomock" 12 | ) 13 | 14 | // MockStore is a mock of Store interface. 15 | type MockStore struct { 16 | ctrl *gomock.Controller 17 | recorder *MockStoreMockRecorder 18 | } 19 | 20 | // MockStoreMockRecorder is the mock recorder for MockStore. 21 | type MockStoreMockRecorder struct { 22 | mock *MockStore 23 | } 24 | 25 | // NewMockStore creates a new mock instance. 26 | func NewMockStore(ctrl *gomock.Controller) *MockStore { 27 | mock := &MockStore{ctrl: ctrl} 28 | mock.recorder = &MockStoreMockRecorder{mock} 29 | return mock 30 | } 31 | 32 | // EXPECT returns an object that allows the caller to indicate expected use. 33 | func (m *MockStore) EXPECT() *MockStoreMockRecorder { 34 | return m.recorder 35 | } 36 | 37 | // AddPolicies mocks base method. 38 | func (m *MockStore) AddPolicies(request *command.AddPoliciesRequest) error { 39 | m.ctrl.T.Helper() 40 | ret := m.ctrl.Call(m, "AddPolicies", request) 41 | ret0, _ := ret[0].(error) 42 | return ret0 43 | } 44 | 45 | // AddPolicies indicates an expected call of AddPolicies. 46 | func (mr *MockStoreMockRecorder) AddPolicies(request interface{}) *gomock.Call { 47 | mr.mock.ctrl.T.Helper() 48 | return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "AddPolicies", reflect.TypeOf((*MockStore)(nil).AddPolicies), request) 49 | } 50 | 51 | // ClearPolicy mocks base method. 52 | func (m *MockStore) ClearPolicy() error { 53 | m.ctrl.T.Helper() 54 | ret := m.ctrl.Call(m, "ClearPolicy") 55 | ret0, _ := ret[0].(error) 56 | return ret0 57 | } 58 | 59 | // ClearPolicy indicates an expected call of ClearPolicy. 60 | func (mr *MockStoreMockRecorder) ClearPolicy() *gomock.Call { 61 | mr.mock.ctrl.T.Helper() 62 | return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ClearPolicy", reflect.TypeOf((*MockStore)(nil).ClearPolicy)) 63 | } 64 | 65 | // JoinNode mocks base method. 66 | func (m *MockStore) JoinNode(serverID, address string) error { 67 | m.ctrl.T.Helper() 68 | ret := m.ctrl.Call(m, "JoinNode", serverID, address) 69 | ret0, _ := ret[0].(error) 70 | return ret0 71 | } 72 | 73 | // JoinNode indicates an expected call of JoinNode. 74 | func (mr *MockStoreMockRecorder) JoinNode(serverID, address interface{}) *gomock.Call { 75 | mr.mock.ctrl.T.Helper() 76 | return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "JoinNode", reflect.TypeOf((*MockStore)(nil).JoinNode), serverID, address) 77 | } 78 | 79 | // Leader mocks base method. 80 | func (m *MockStore) Leader() (bool, string) { 81 | m.ctrl.T.Helper() 82 | ret := m.ctrl.Call(m, "Leader") 83 | ret0, _ := ret[0].(bool) 84 | ret1, _ := ret[1].(string) 85 | return ret0, ret1 86 | } 87 | 88 | // Leader indicates an expected call of Leader. 89 | func (mr *MockStoreMockRecorder) Leader() *gomock.Call { 90 | mr.mock.ctrl.T.Helper() 91 | return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Leader", reflect.TypeOf((*MockStore)(nil).Leader)) 92 | } 93 | 94 | // RemoveFilteredPolicy mocks base method. 95 | func (m *MockStore) RemoveFilteredPolicy(request *command.RemoveFilteredPolicyRequest) error { 96 | m.ctrl.T.Helper() 97 | ret := m.ctrl.Call(m, "RemoveFilteredPolicy", request) 98 | ret0, _ := ret[0].(error) 99 | return ret0 100 | } 101 | 102 | // RemoveFilteredPolicy indicates an expected call of RemoveFilteredPolicy. 103 | func (mr *MockStoreMockRecorder) RemoveFilteredPolicy(request interface{}) *gomock.Call { 104 | mr.mock.ctrl.T.Helper() 105 | return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "RemoveFilteredPolicy", reflect.TypeOf((*MockStore)(nil).RemoveFilteredPolicy), request) 106 | } 107 | 108 | // RemoveNode mocks base method. 109 | func (m *MockStore) RemoveNode(serverID string) error { 110 | m.ctrl.T.Helper() 111 | ret := m.ctrl.Call(m, "RemoveNode", serverID) 112 | ret0, _ := ret[0].(error) 113 | return ret0 114 | } 115 | 116 | // RemoveNode indicates an expected call of RemoveNode. 117 | func (mr *MockStoreMockRecorder) RemoveNode(serverID interface{}) *gomock.Call { 118 | mr.mock.ctrl.T.Helper() 119 | return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "RemoveNode", reflect.TypeOf((*MockStore)(nil).RemoveNode), serverID) 120 | } 121 | 122 | // RemovePolicies mocks base method. 123 | func (m *MockStore) RemovePolicies(request *command.RemovePoliciesRequest) error { 124 | m.ctrl.T.Helper() 125 | ret := m.ctrl.Call(m, "RemovePolicies", request) 126 | ret0, _ := ret[0].(error) 127 | return ret0 128 | } 129 | 130 | // RemovePolicies indicates an expected call of RemovePolicies. 131 | func (mr *MockStoreMockRecorder) RemovePolicies(request interface{}) *gomock.Call { 132 | mr.mock.ctrl.T.Helper() 133 | return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "RemovePolicies", reflect.TypeOf((*MockStore)(nil).RemovePolicies), request) 134 | } 135 | 136 | // Stats mocks base method. 137 | func (m *MockStore) Stats() (map[string]interface{}, error) { 138 | m.ctrl.T.Helper() 139 | ret := m.ctrl.Call(m, "Stats") 140 | ret0, _ := ret[0].(map[string]interface{}) 141 | ret1, _ := ret[1].(error) 142 | return ret0, ret1 143 | } 144 | 145 | // Stats indicates an expected call of Stats. 146 | func (mr *MockStoreMockRecorder) Stats() *gomock.Call { 147 | mr.mock.ctrl.T.Helper() 148 | return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Stats", reflect.TypeOf((*MockStore)(nil).Stats)) 149 | } 150 | 151 | // UpdateFilteredPolicies mocks base method. 152 | func (m *MockStore) UpdateFilteredPolicies(request *command.UpdateFilteredPoliciesRequest) error { 153 | m.ctrl.T.Helper() 154 | ret := m.ctrl.Call(m, "UpdateFilteredPolicies", request) 155 | ret0, _ := ret[0].(error) 156 | return ret0 157 | } 158 | 159 | // UpdateFilteredPolicies indicates an expected call of UpdateFilteredPolicies. 160 | func (mr *MockStoreMockRecorder) UpdateFilteredPolicies(request interface{}) *gomock.Call { 161 | mr.mock.ctrl.T.Helper() 162 | return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "UpdateFilteredPolicies", reflect.TypeOf((*MockStore)(nil).UpdateFilteredPolicies), request) 163 | } 164 | 165 | // UpdatePolicies mocks base method. 166 | func (m *MockStore) UpdatePolicies(request *command.UpdatePoliciesRequest) error { 167 | m.ctrl.T.Helper() 168 | ret := m.ctrl.Call(m, "UpdatePolicies", request) 169 | ret0, _ := ret[0].(error) 170 | return ret0 171 | } 172 | 173 | // UpdatePolicies indicates an expected call of UpdatePolicies. 174 | func (mr *MockStoreMockRecorder) UpdatePolicies(request interface{}) *gomock.Call { 175 | mr.mock.ctrl.T.Helper() 176 | return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "UpdatePolicies", reflect.TypeOf((*MockStore)(nil).UpdatePolicies), request) 177 | } 178 | 179 | // UpdatePolicy mocks base method. 180 | func (m *MockStore) UpdatePolicy(request *command.UpdatePolicyRequest) error { 181 | m.ctrl.T.Helper() 182 | ret := m.ctrl.Call(m, "UpdatePolicy", request) 183 | ret0, _ := ret[0].(error) 184 | return ret0 185 | } 186 | 187 | // UpdatePolicy indicates an expected call of UpdatePolicy. 188 | func (mr *MockStoreMockRecorder) UpdatePolicy(request interface{}) *gomock.Call { 189 | mr.mock.ctrl.T.Helper() 190 | return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "UpdatePolicy", reflect.TypeOf((*MockStore)(nil).UpdatePolicy), request) 191 | } 192 | -------------------------------------------------------------------------------- /http/service.go: -------------------------------------------------------------------------------- 1 | package http 2 | 3 | import ( 4 | "bytes" 5 | "context" 6 | "crypto/tls" 7 | "fmt" 8 | "io/ioutil" 9 | "net" 10 | "net/http" 11 | "net/http/pprof" 12 | "strings" 13 | "time" 14 | 15 | "golang.org/x/net/http2" 16 | 17 | "github.com/hashicorp/raft" 18 | 19 | "github.com/hashicorp/go-multierror" 20 | 21 | "github.com/go-chi/chi" 22 | jsoniter "github.com/json-iterator/go" 23 | "github.com/pkg/errors" 24 | 25 | "github.com/casbin/hraft-dispatcher/command" 26 | 27 | "go.uber.org/zap" 28 | ) 29 | 30 | //go:generate mockgen -destination ./mocks/mock_store.go -package mocks -source service.go 31 | 32 | // Store provides an interface that can be implemented by raft. 33 | type Store interface { 34 | // AddPolicies adds a set of rules to the current policy. 35 | AddPolicies(request *command.AddPoliciesRequest) error 36 | // RemovePolicies removes a set of rules from the current policy. 37 | RemovePolicies(request *command.RemovePoliciesRequest) error 38 | // RemoveFilteredPolicy removes a set of rules that match a pattern from the current policy. 39 | RemoveFilteredPolicy(request *command.RemoveFilteredPolicyRequest) error 40 | // UpdatePolicy updates a rule of policy. 41 | UpdatePolicy(request *command.UpdatePolicyRequest) error 42 | // UpdatePolicies updates a set of rules of policy. 43 | UpdatePolicies(request *command.UpdatePoliciesRequest) error 44 | // UpdateFilteredPolicies updates a set of rules of policy. 45 | UpdateFilteredPolicies(request *command.UpdateFilteredPoliciesRequest) error 46 | // ClearPolicy clears all policies. 47 | ClearPolicy() error 48 | 49 | // JoinNode joins a node with a given serverID and network address to cluster. 50 | JoinNode(serverID string, address string) error 51 | // RemoveNode removes a node with a given serverID from cluster. 52 | RemoveNode(serverID string) error 53 | // Leader checks if it is a leader and returns network address. 54 | Leader() (bool, string) 55 | 56 | // Stats returns stats. 57 | Stats() (map[string]interface{}, error) 58 | } 59 | 60 | // Service setups a HTTP service for forward data of raft node. 61 | type Service struct { 62 | srv *http.Server 63 | ln net.Listener 64 | store Store 65 | httpClient *http.Client 66 | tlsConfig *tls.Config 67 | logger *zap.Logger 68 | } 69 | 70 | // NewService creates a Service. 71 | func NewService(logger *zap.Logger, ln net.Listener, tlsConfig *tls.Config, store Store) (*Service, error) { 72 | if ln == nil { 73 | return nil, errors.New("net.Listener is provided") 74 | } 75 | 76 | if store == nil { 77 | return nil, errors.New("store is not provided") 78 | } 79 | 80 | httpClient := &http.Client{ 81 | Timeout: 10 * time.Second, 82 | } 83 | 84 | if tlsConfig != nil { 85 | // TODO: using http2.Transport always return unexpected eof on cmux. 86 | httpClient.Transport = &http.Transport{ 87 | TLSClientConfig: tlsConfig, 88 | } 89 | } 90 | 91 | s := &Service{ 92 | logger: logger, 93 | store: store, 94 | httpClient: httpClient, 95 | ln: ln, 96 | tlsConfig: tlsConfig, 97 | } 98 | 99 | r := chi.NewRouter() 100 | r.Route("/policies", func(r chi.Router) { 101 | r.Put("/add", s.handleAddPolicy) 102 | r.Put("/update", s.handleUpdatePolicy) 103 | r.Put("/remove", s.handleRemovePolicy) 104 | }) 105 | r.Route("/nodes", func(r chi.Router) { 106 | r.Put("/join", s.handleJoinNode) 107 | r.Put("/remove", s.handleRemoveNode) 108 | }) 109 | 110 | // add pprof 111 | r.HandleFunc("/debug/pprof/", pprof.Index) 112 | r.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline) 113 | r.HandleFunc("/debug/pprof/profile", pprof.Profile) 114 | r.HandleFunc("/debug/pprof/symbol", pprof.Symbol) 115 | r.HandleFunc("/debug/pprof/trace", pprof.Trace) 116 | 117 | s.srv = &http.Server{ 118 | Handler: r, 119 | ReadHeaderTimeout: 10 * time.Second, 120 | ReadTimeout: 30 * time.Second, 121 | IdleTimeout: 5 * time.Minute, 122 | TLSConfig: tlsConfig, 123 | } 124 | 125 | if tlsConfig != nil { 126 | _ = http2.ConfigureServer(s.srv, nil) 127 | } 128 | 129 | return s, nil 130 | } 131 | 132 | // Start starts this service. 133 | // It always returns a non-nil error. After Shutdown or Close, the returned error is http.ErrServerClosed. 134 | func (s *Service) Start() error { 135 | go func() { 136 | var err error 137 | if s.tlsConfig == nil { 138 | s.logger.Info(fmt.Sprintf("listening and serving HTTP on %s", s.ln.Addr())) 139 | } else { 140 | s.logger.Info(fmt.Sprintf("listening and serving HTTPS on %s", s.ln.Addr())) 141 | } 142 | err = s.srv.Serve(s.ln) 143 | if err != nil && err != http.ErrServerClosed { 144 | s.logger.Error("unable to serve http", zap.Error(err)) 145 | } 146 | }() 147 | 148 | return nil 149 | } 150 | 151 | // Stop stops this service. 152 | func (s *Service) Stop(ctx context.Context) error { 153 | var ret error 154 | err := s.srv.Shutdown(ctx) 155 | if err != nil { 156 | ret = multierror.Append(ret, err) 157 | } 158 | err = s.ln.Close() 159 | if err != nil { 160 | ret = multierror.Append(ret, err) 161 | } 162 | return ret 163 | } 164 | 165 | // getRedirectURL returns a URL by the given host. 166 | func (s *Service) getRedirectURL(r *http.Request, host string) string { 167 | rq := r.URL.RawQuery 168 | if rq != "" { 169 | rq = fmt.Sprintf("?%s", rq) 170 | } 171 | 172 | var scheme string 173 | if strings.HasPrefix(r.RequestURI, "https") { 174 | scheme = "https" 175 | } else if strings.HasPrefix(r.RequestURI, "http") { 176 | scheme = "http" 177 | } 178 | 179 | if len(scheme) == 0 { 180 | scheme = s.GetScheme() 181 | } 182 | 183 | return fmt.Sprintf("%s://%s%s%s", scheme, host, r.URL.Path, rq) 184 | } 185 | 186 | func (s *Service) GetScheme() string { 187 | scheme := "https" 188 | if s.tlsConfig == nil { 189 | scheme = "http" 190 | } 191 | return scheme 192 | } 193 | 194 | // handleStoreResponse checks the error returned by store. 195 | // If the error is nil, the server returns http.StatusOK. 196 | // If the error is raft.ErrNotLeader, the server forward the request to the leader node, 197 | // otherwise the server returns http.StatusServiceUnavailable. 198 | func (s *Service) handleStoreResponse(err error, w http.ResponseWriter, r *http.Request) { 199 | if err == nil { 200 | w.WriteHeader(http.StatusOK) 201 | return 202 | } 203 | if err == raft.ErrNotLeader { 204 | isLeader, leaderAddr := s.store.Leader() 205 | if !isLeader { 206 | if len(leaderAddr) == 0 { 207 | s.logger.Error("failed to get the leader address") 208 | w.WriteHeader(http.StatusServiceUnavailable) 209 | return 210 | } 211 | redirectURL := s.getRedirectURL(r, leaderAddr) 212 | http.Redirect(w, r, redirectURL, http.StatusTemporaryRedirect) 213 | return 214 | } 215 | } else { 216 | http.Error(w, err.Error(), http.StatusServiceUnavailable) 217 | } 218 | } 219 | 220 | // handleAddPolicy handles the request to add a set of rules. 221 | func (s *Service) handleAddPolicy(w http.ResponseWriter, r *http.Request) { 222 | data, err := ioutil.ReadAll(r.Body) 223 | if err != nil { 224 | http.Error(w, err.Error(), http.StatusBadRequest) 225 | return 226 | } 227 | var cmd command.AddPoliciesRequest 228 | err = jsoniter.Unmarshal(data, &cmd) 229 | if err != nil { 230 | http.Error(w, err.Error(), http.StatusBadRequest) 231 | return 232 | } 233 | err = s.store.AddPolicies(&cmd) 234 | s.handleStoreResponse(err, w, r) 235 | } 236 | 237 | // handleRemovePolicy handles the request to remove a set of rules. 238 | func (s *Service) handleRemovePolicy(w http.ResponseWriter, r *http.Request) { 239 | removeType := r.URL.Query().Get("type") 240 | switch removeType { 241 | case "all": 242 | err := s.store.ClearPolicy() 243 | s.handleStoreResponse(err, w, r) 244 | case "filtered": 245 | data, err := ioutil.ReadAll(r.Body) 246 | if err != nil { 247 | http.Error(w, err.Error(), http.StatusBadRequest) 248 | return 249 | } 250 | var cmd command.RemoveFilteredPolicyRequest 251 | err = jsoniter.Unmarshal(data, &cmd) 252 | if err != nil { 253 | http.Error(w, err.Error(), http.StatusBadRequest) 254 | return 255 | } 256 | err = s.store.RemoveFilteredPolicy(&cmd) 257 | s.handleStoreResponse(err, w, r) 258 | case "": 259 | data, err := ioutil.ReadAll(r.Body) 260 | if err != nil { 261 | http.Error(w, err.Error(), http.StatusBadRequest) 262 | return 263 | } 264 | var cmd command.RemovePoliciesRequest 265 | err = jsoniter.Unmarshal(data, &cmd) 266 | if err != nil { 267 | http.Error(w, err.Error(), http.StatusBadRequest) 268 | return 269 | } 270 | err = s.store.RemovePolicies(&cmd) 271 | s.handleStoreResponse(err, w, r) 272 | default: 273 | w.WriteHeader(http.StatusBadRequest) 274 | } 275 | } 276 | 277 | // handleUpdatePolicy handles the request to update a rule. 278 | func (s *Service) handleUpdatePolicy(w http.ResponseWriter, r *http.Request) { 279 | updateType := r.URL.Query().Get("type") 280 | switch updateType { 281 | case "batch": 282 | data, err := ioutil.ReadAll(r.Body) 283 | if err != nil { 284 | http.Error(w, err.Error(), http.StatusBadRequest) 285 | return 286 | } 287 | var cmd command.UpdatePoliciesRequest 288 | err = jsoniter.Unmarshal(data, &cmd) 289 | if err != nil { 290 | http.Error(w, err.Error(), http.StatusBadRequest) 291 | return 292 | } 293 | err = s.store.UpdatePolicies(&cmd) 294 | s.handleStoreResponse(err, w, r) 295 | case "filtered": 296 | data, err := ioutil.ReadAll(r.Body) 297 | if err != nil { 298 | http.Error(w, err.Error(), http.StatusBadRequest) 299 | return 300 | } 301 | var cmd command.UpdateFilteredPoliciesRequest 302 | err = jsoniter.Unmarshal(data, &cmd) 303 | if err != nil { 304 | http.Error(w, err.Error(), http.StatusBadRequest) 305 | return 306 | } 307 | err = s.store.UpdateFilteredPolicies(&cmd) 308 | s.handleStoreResponse(err, w, r) 309 | case "": 310 | data, err := ioutil.ReadAll(r.Body) 311 | if err != nil { 312 | http.Error(w, err.Error(), http.StatusBadRequest) 313 | return 314 | } 315 | var cmd command.UpdatePolicyRequest 316 | err = jsoniter.Unmarshal(data, &cmd) 317 | if err != nil { 318 | http.Error(w, err.Error(), http.StatusBadRequest) 319 | return 320 | } 321 | err = s.store.UpdatePolicy(&cmd) 322 | s.handleStoreResponse(err, w, r) 323 | default: 324 | w.WriteHeader(http.StatusBadRequest) 325 | } 326 | } 327 | 328 | func (s *Service) handleJoinNode(w http.ResponseWriter, r *http.Request) { 329 | data, err := ioutil.ReadAll(r.Body) 330 | if err != nil { 331 | http.Error(w, err.Error(), http.StatusBadRequest) 332 | return 333 | } 334 | var cmd command.AddNodeRequest 335 | err = jsoniter.Unmarshal(data, &cmd) 336 | if err != nil { 337 | http.Error(w, err.Error(), http.StatusBadRequest) 338 | return 339 | } 340 | err = s.store.JoinNode(cmd.Id, cmd.Address) 341 | s.handleStoreResponse(err, w, r) 342 | } 343 | 344 | func (s *Service) handleRemoveNode(w http.ResponseWriter, r *http.Request) { 345 | data, err := ioutil.ReadAll(r.Body) 346 | if err != nil { 347 | http.Error(w, err.Error(), http.StatusBadRequest) 348 | return 349 | } 350 | var cmd command.RemoveNodeRequest 351 | err = jsoniter.Unmarshal(data, &cmd) 352 | if err != nil { 353 | http.Error(w, err.Error(), http.StatusBadRequest) 354 | return 355 | } 356 | err = s.store.RemoveNode(cmd.Id) 357 | s.handleStoreResponse(err, w, r) 358 | } 359 | 360 | func (s *Service) Addr() string { 361 | return s.ln.Addr().String() 362 | } 363 | 364 | func (s *Service) DoAddPolicyRequest(request *command.AddPoliciesRequest) error { 365 | b, err := jsoniter.Marshal(request) 366 | if err != nil { 367 | return err 368 | } 369 | 370 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("%s://%s/policies/add", s.GetScheme(), s.Addr()), bytes.NewBuffer(b)) 371 | if err != nil { 372 | return err 373 | } 374 | 375 | resp, err := s.httpClient.Do(r) 376 | if err != nil { 377 | return err 378 | } 379 | if resp.StatusCode != http.StatusOK { 380 | return errors.New(http.StatusText(http.StatusServiceUnavailable)) 381 | } 382 | 383 | return nil 384 | } 385 | 386 | func (s *Service) DoRemovePolicyRequest(request *command.RemovePoliciesRequest) error { 387 | b, err := jsoniter.Marshal(request) 388 | if err != nil { 389 | return err 390 | } 391 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("%s://%s/policies/remove", s.GetScheme(), s.Addr()), bytes.NewBuffer(b)) 392 | if err != nil { 393 | return err 394 | } 395 | 396 | resp, err := s.httpClient.Do(r) 397 | if err != nil { 398 | return err 399 | } 400 | if resp.StatusCode != http.StatusOK { 401 | return errors.New(http.StatusText(http.StatusServiceUnavailable)) 402 | } 403 | 404 | return nil 405 | } 406 | 407 | func (s *Service) DoRemoveFilteredPolicyRequest(request *command.RemoveFilteredPolicyRequest) error { 408 | b, err := jsoniter.Marshal(request) 409 | if err != nil { 410 | return err 411 | } 412 | 413 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("%s://%s/policies/remove?type=filtered", s.GetScheme(), s.Addr()), bytes.NewBuffer(b)) 414 | if err != nil { 415 | return err 416 | } 417 | 418 | resp, err := s.httpClient.Do(r) 419 | if err != nil { 420 | return err 421 | } 422 | if resp.StatusCode != http.StatusOK { 423 | return errors.New(http.StatusText(http.StatusServiceUnavailable)) 424 | } 425 | 426 | return nil 427 | } 428 | 429 | func (s *Service) DoClearPolicyRequest() error { 430 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("%s://%s/policies/remove?type=all", s.GetScheme(), s.Addr()), nil) 431 | if err != nil { 432 | return err 433 | } 434 | 435 | resp, err := s.httpClient.Do(r) 436 | if err != nil { 437 | return err 438 | } 439 | if resp.StatusCode != http.StatusOK { 440 | return errors.New(http.StatusText(http.StatusServiceUnavailable)) 441 | } 442 | 443 | return nil 444 | } 445 | 446 | func (s *Service) DoUpdatePolicyRequest(request *command.UpdatePolicyRequest) error { 447 | b, err := jsoniter.Marshal(request) 448 | if err != nil { 449 | return err 450 | } 451 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("%s://%s/policies/update", s.GetScheme(), s.Addr()), bytes.NewBuffer(b)) 452 | if err != nil { 453 | return err 454 | } 455 | 456 | resp, err := s.httpClient.Do(r) 457 | if err != nil { 458 | return err 459 | } 460 | if resp.StatusCode != http.StatusOK { 461 | return errors.New(http.StatusText(http.StatusServiceUnavailable)) 462 | } 463 | 464 | return nil 465 | } 466 | 467 | func (s *Service) DoUpdateFilteredPoliciesRequest(request *command.UpdateFilteredPoliciesRequest) error { 468 | b, err := jsoniter.Marshal(request) 469 | if err != nil { 470 | return err 471 | } 472 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("%s://%s/policies/update?type=filtered", s.GetScheme(), s.Addr()), bytes.NewBuffer(b)) 473 | if err != nil { 474 | return err 475 | } 476 | 477 | resp, err := s.httpClient.Do(r) 478 | if err != nil { 479 | return err 480 | } 481 | if resp.StatusCode != http.StatusOK { 482 | return errors.New(http.StatusText(http.StatusServiceUnavailable)) 483 | } 484 | return nil 485 | } 486 | 487 | func (s *Service) DoUpdatePoliciesRequest(request *command.UpdatePoliciesRequest) error { 488 | b, err := jsoniter.Marshal(request) 489 | if err != nil { 490 | return err 491 | } 492 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("%s://%s/policies/update?type=batch", s.GetScheme(), s.Addr()), bytes.NewBuffer(b)) 493 | if err != nil { 494 | return err 495 | } 496 | 497 | resp, err := s.httpClient.Do(r) 498 | if err != nil { 499 | return err 500 | } 501 | if resp.StatusCode != http.StatusOK { 502 | return errors.New(http.StatusText(http.StatusServiceUnavailable)) 503 | } 504 | 505 | return nil 506 | } 507 | 508 | func (s *Service) DoJoinNodeRequest(request *command.AddNodeRequest) error { 509 | b, err := jsoniter.Marshal(request) 510 | if err != nil { 511 | return err 512 | } 513 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("%s://%s/nodes/join", s.GetScheme(), s.Addr()), bytes.NewBuffer(b)) 514 | if err != nil { 515 | return err 516 | } 517 | 518 | resp, err := s.httpClient.Do(r) 519 | if err != nil { 520 | return err 521 | } 522 | if resp.StatusCode != http.StatusOK { 523 | return errors.New(http.StatusText(http.StatusServiceUnavailable)) 524 | } 525 | 526 | return nil 527 | } 528 | 529 | func (s *Service) DoRemoveNodeRequest(request *command.RemoveNodeRequest) error { 530 | b, err := jsoniter.Marshal(request) 531 | if err != nil { 532 | return err 533 | } 534 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("%s://%s/nodes/remove", s.GetScheme(), s.Addr()), bytes.NewBuffer(b)) 535 | if err != nil { 536 | return err 537 | } 538 | 539 | resp, err := s.httpClient.Do(r) 540 | if err != nil { 541 | return err 542 | } 543 | if resp.StatusCode != http.StatusOK { 544 | return errors.New(http.StatusText(http.StatusServiceUnavailable)) 545 | } 546 | 547 | return nil 548 | } 549 | 550 | func DoJoinNodeRequest(clusterAddress string, nodeID string, nodeAddress string, tlsConfig *tls.Config) error { 551 | tr := &http.Transport{ 552 | TLSClientConfig: tlsConfig, 553 | } 554 | client := http.Client{Transport: tr} 555 | 556 | data := &command.AddNodeRequest{ 557 | Address: nodeAddress, 558 | Id: nodeID, 559 | } 560 | 561 | b, err := jsoniter.Marshal(data) 562 | if err != nil { 563 | return err 564 | } 565 | 566 | scheme := "https" 567 | if tlsConfig == nil { 568 | scheme = "http" 569 | } 570 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("%s://%s/nodes/join", scheme, clusterAddress), bytes.NewBuffer(b)) 571 | if err != nil { 572 | return err 573 | } 574 | 575 | resp, err := client.Do(r) 576 | if err != nil { 577 | return err 578 | } 579 | if resp.StatusCode != http.StatusOK { 580 | return errors.New(http.StatusText(http.StatusServiceUnavailable)) 581 | } 582 | 583 | return nil 584 | } 585 | -------------------------------------------------------------------------------- /http/service_test.go: -------------------------------------------------------------------------------- 1 | package http 2 | 3 | import ( 4 | "bytes" 5 | "context" 6 | "crypto/tls" 7 | "crypto/x509" 8 | "fmt" 9 | "io/ioutil" 10 | "net" 11 | "net/http" 12 | "net/http/httptest" 13 | "testing" 14 | "time" 15 | 16 | "go.uber.org/zap" 17 | 18 | "github.com/hashicorp/raft" 19 | "github.com/pkg/errors" 20 | 21 | "github.com/casbin/hraft-dispatcher/command" 22 | "github.com/casbin/hraft-dispatcher/http/mocks" 23 | "github.com/golang/mock/gomock" 24 | jsoniter "github.com/json-iterator/go" 25 | "github.com/stretchr/testify/assert" 26 | ) 27 | 28 | func TestNewService(t *testing.T) { 29 | ctl := gomock.NewController(t) 30 | defer ctl.Finish() 31 | 32 | store := mocks.NewMockStore(ctl) 33 | ln, err := net.Listen("tcp", "127.0.0.1:0") 34 | assert.NoError(t, err) 35 | s, err := NewService(zap.NewExample(), ln, nil, store) 36 | assert.NoError(t, err) 37 | assert.NotNil(t, s) 38 | } 39 | 40 | func TestRedirect(t *testing.T) { 41 | ctl := gomock.NewController(t) 42 | defer ctl.Finish() 43 | 44 | store := mocks.NewMockStore(ctl) 45 | ln, err := net.Listen("tcp", "127.0.0.1:0") 46 | assert.NoError(t, err) 47 | s, err := NewService(zap.NewExample(), ln, nil, store) 48 | assert.NoError(t, err) 49 | 50 | r := httptest.NewRequest(http.MethodPut, "https://127.0.0.1:6971/policies/add", nil) 51 | actualURL := s.getRedirectURL(r, "127.0.0.1:6970") 52 | expectedURL := "https://127.0.0.1:6970/policies/add" 53 | assert.Equal(t, expectedURL, actualURL) 54 | } 55 | 56 | func TestNotLeaderError(t *testing.T) { 57 | ctl := gomock.NewController(t) 58 | defer ctl.Finish() 59 | 60 | store := mocks.NewMockStore(ctl) 61 | ln, err := net.Listen("tcp", "127.0.0.1:0") 62 | assert.NoError(t, err) 63 | s, err := NewService(zap.NewExample(), ln, nil, store) 64 | assert.NoError(t, err) 65 | 66 | w := httptest.NewRecorder() 67 | s.handleStoreResponse(nil, w, httptest.NewRequest(http.MethodPut, "https://testing", nil)) 68 | assert.Equal(t, w.Code, http.StatusOK) 69 | 70 | w = httptest.NewRecorder() 71 | s.handleStoreResponse(errors.New("test error"), w, httptest.NewRequest(http.MethodPut, "https://testing", nil)) 72 | assert.Equal(t, w.Code, http.StatusServiceUnavailable) 73 | 74 | store.EXPECT().Leader().Return(false, "127.0.0.1:6790") 75 | w = httptest.NewRecorder() 76 | s.handleStoreResponse(raft.ErrNotLeader, w, httptest.NewRequest(http.MethodPut, "https://testing/add", nil)) 77 | assert.Equal(t, w.Header().Get("Location"), "https://127.0.0.1:6790/add") 78 | assert.Equal(t, w.Code, http.StatusTemporaryRedirect) 79 | } 80 | 81 | func TestAddPolicy(t *testing.T) { 82 | ctl := gomock.NewController(t) 83 | defer ctl.Finish() 84 | 85 | store := mocks.NewMockStore(ctl) 86 | 87 | ts := httptest.NewUnstartedServer(nil) 88 | ts.EnableHTTP2 = true 89 | ts.StartTLS() 90 | defer ts.Close() 91 | 92 | ln, err := tls.Listen("tcp", "127.0.0.1:0", ts.TLS) 93 | assert.NoError(t, err) 94 | s, err := NewService(zap.NewExample(), ln, ts.TLS, store) 95 | assert.NoError(t, err) 96 | 97 | err = s.Start() 98 | assert.NoError(t, err) 99 | defer s.Stop(context.Background()) 100 | 101 | addPolicyRequest := &command.AddPoliciesRequest{ 102 | Sec: "p", 103 | PType: "p", 104 | Rules: []*command.StringArray{{Items: []string{"role:admin", "/", "*"}}}, 105 | } 106 | store.EXPECT().AddPolicies(addPolicyRequest).Return(nil) 107 | 108 | b, err := jsoniter.Marshal(addPolicyRequest) 109 | assert.NoError(t, err) 110 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("https://%s/policies/add", s.Addr()), bytes.NewBuffer(b)) 111 | assert.NoError(t, err) 112 | 113 | resp, err := ts.Client().Do(r) 114 | assert.NoError(t, err) 115 | assert.Equal(t, http.StatusOK, resp.StatusCode) 116 | } 117 | 118 | func TestRemovePolicy(t *testing.T) { 119 | ctl := gomock.NewController(t) 120 | defer ctl.Finish() 121 | 122 | store := mocks.NewMockStore(ctl) 123 | 124 | ts := httptest.NewUnstartedServer(nil) 125 | ts.EnableHTTP2 = true 126 | ts.StartTLS() 127 | defer ts.Close() 128 | 129 | ln, err := tls.Listen("tcp", "127.0.0.1:0", ts.TLS) 130 | assert.NoError(t, err) 131 | s, err := NewService(zap.NewExample(), ln, ts.TLS, store) 132 | assert.NoError(t, err) 133 | 134 | err = s.Start() 135 | assert.NoError(t, err) 136 | defer s.Stop(context.Background()) 137 | 138 | removePolicyRequest := &command.RemovePoliciesRequest{ 139 | Sec: "p", 140 | PType: "p", 141 | Rules: []*command.StringArray{{Items: []string{"role:admin", "/", "*"}}}, 142 | } 143 | store.EXPECT().RemovePolicies(removePolicyRequest).Return(nil) 144 | 145 | b, err := jsoniter.Marshal(removePolicyRequest) 146 | assert.NoError(t, err) 147 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("https://%s/policies/remove", s.Addr()), bytes.NewBuffer(b)) 148 | assert.NoError(t, err) 149 | 150 | resp, err := ts.Client().Do(r) 151 | assert.NoError(t, err) 152 | assert.Equal(t, http.StatusOK, resp.StatusCode) 153 | } 154 | 155 | func TestRemoveFilteredPolicy(t *testing.T) { 156 | ctl := gomock.NewController(t) 157 | defer ctl.Finish() 158 | 159 | store := mocks.NewMockStore(ctl) 160 | 161 | ts := httptest.NewUnstartedServer(nil) 162 | ts.EnableHTTP2 = true 163 | ts.StartTLS() 164 | defer ts.Close() 165 | 166 | ln, err := tls.Listen("tcp", "127.0.0.1:0", ts.TLS) 167 | assert.NoError(t, err) 168 | s, err := NewService(zap.NewExample(), ln, ts.TLS, store) 169 | assert.NoError(t, err) 170 | 171 | err = s.Start() 172 | assert.NoError(t, err) 173 | defer s.Stop(context.Background()) 174 | 175 | removeFilteredPolicyRequest := &command.RemoveFilteredPolicyRequest{ 176 | Sec: "p", 177 | PType: "p", 178 | FieldIndex: 0, 179 | FieldValues: []string{"role:admin"}, 180 | } 181 | store.EXPECT().RemoveFilteredPolicy(removeFilteredPolicyRequest).Return(nil) 182 | 183 | b, err := jsoniter.Marshal(removeFilteredPolicyRequest) 184 | assert.NoError(t, err) 185 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("https://%s/policies/remove?type=filtered", s.Addr()), bytes.NewBuffer(b)) 186 | assert.NoError(t, err) 187 | 188 | resp, err := ts.Client().Do(r) 189 | assert.NoError(t, err) 190 | assert.Equal(t, http.StatusOK, resp.StatusCode) 191 | } 192 | 193 | func TestUpdatePolicy(t *testing.T) { 194 | ctl := gomock.NewController(t) 195 | defer ctl.Finish() 196 | 197 | store := mocks.NewMockStore(ctl) 198 | 199 | ts := httptest.NewUnstartedServer(nil) 200 | ts.EnableHTTP2 = true 201 | ts.StartTLS() 202 | defer ts.Close() 203 | 204 | ln, err := tls.Listen("tcp", "127.0.0.1:0", ts.TLS) 205 | assert.NoError(t, err) 206 | s, err := NewService(zap.NewExample(), ln, ts.TLS, store) 207 | assert.NoError(t, err) 208 | 209 | err = s.Start() 210 | assert.NoError(t, err) 211 | defer s.Stop(context.Background()) 212 | 213 | updatePolicyRequest := &command.UpdatePolicyRequest{ 214 | Sec: "p", 215 | PType: "p", 216 | OldRule: []string{"role:admin", "/", "*"}, 217 | NewRule: []string{"role:admin", "/admin", "*"}, 218 | } 219 | store.EXPECT().UpdatePolicy(updatePolicyRequest).Return(nil) 220 | 221 | b, err := jsoniter.Marshal(updatePolicyRequest) 222 | assert.NoError(t, err) 223 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("https://%s/policies/update", s.Addr()), bytes.NewBuffer(b)) 224 | assert.NoError(t, err) 225 | 226 | resp, err := ts.Client().Do(r) 227 | assert.NoError(t, err) 228 | assert.Equal(t, http.StatusOK, resp.StatusCode) 229 | } 230 | 231 | func TestClearPolicy(t *testing.T) { 232 | ctl := gomock.NewController(t) 233 | defer ctl.Finish() 234 | 235 | store := mocks.NewMockStore(ctl) 236 | 237 | ts := httptest.NewUnstartedServer(nil) 238 | ts.EnableHTTP2 = true 239 | ts.StartTLS() 240 | defer ts.Close() 241 | 242 | ln, err := tls.Listen("tcp", "127.0.0.1:0", ts.TLS) 243 | assert.NoError(t, err) 244 | s, err := NewService(zap.NewExample(), ln, ts.TLS, store) 245 | assert.NoError(t, err) 246 | 247 | err = s.Start() 248 | assert.NoError(t, err) 249 | defer s.Stop(context.Background()) 250 | 251 | store.EXPECT().ClearPolicy().Return(nil) 252 | 253 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("https://%s/policies/remove?type=all", s.Addr()), nil) 254 | assert.NoError(t, err) 255 | 256 | resp, err := ts.Client().Do(r) 257 | assert.NoError(t, err) 258 | assert.Equal(t, http.StatusOK, resp.StatusCode) 259 | } 260 | 261 | func TestJoinNode(t *testing.T) { 262 | ctl := gomock.NewController(t) 263 | defer ctl.Finish() 264 | 265 | store := mocks.NewMockStore(ctl) 266 | 267 | ts := httptest.NewUnstartedServer(nil) 268 | ts.EnableHTTP2 = true 269 | ts.StartTLS() 270 | defer ts.Close() 271 | 272 | ln, err := tls.Listen("tcp", "127.0.0.1:0", ts.TLS) 273 | assert.NoError(t, err) 274 | s, err := NewService(zap.NewExample(), ln, ts.TLS, store) 275 | assert.NoError(t, err) 276 | 277 | err = s.Start() 278 | assert.NoError(t, err) 279 | defer s.Stop(context.Background()) 280 | 281 | addNodeRequest := &command.AddNodeRequest{ 282 | Id: "test-main", 283 | Address: "10.0.7.10", 284 | } 285 | store.EXPECT().JoinNode(addNodeRequest.Id, addNodeRequest.Address).Return(nil) 286 | 287 | b, err := jsoniter.Marshal(addNodeRequest) 288 | assert.NoError(t, err) 289 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("https://%s/nodes/join", s.Addr()), bytes.NewReader(b)) 290 | assert.NoError(t, err) 291 | 292 | resp, err := ts.Client().Do(r) 293 | assert.NoError(t, err) 294 | assert.Equal(t, http.StatusOK, resp.StatusCode) 295 | } 296 | 297 | func GetTLSConfig() (*tls.Config, error) { 298 | rootCAPool := x509.NewCertPool() 299 | rootCA, err := ioutil.ReadFile("../testdata/ca/ca.pem") 300 | if err != nil { 301 | return nil, err 302 | } 303 | rootCAPool.AppendCertsFromPEM(rootCA) 304 | 305 | cert, err := tls.LoadX509KeyPair("../testdata/ca/peer.pem", "../testdata/ca/peer-key.pem") 306 | if err != nil { 307 | return nil, err 308 | } 309 | 310 | config := &tls.Config{ 311 | RootCAs: rootCAPool, 312 | ClientCAs: rootCAPool, 313 | ClientAuth: tls.RequireAndVerifyClientCert, 314 | Certificates: []tls.Certificate{cert}, 315 | } 316 | 317 | return config, nil 318 | } 319 | 320 | func TestRemoveNode(t *testing.T) { 321 | ctl := gomock.NewController(t) 322 | defer ctl.Finish() 323 | 324 | store := mocks.NewMockStore(ctl) 325 | 326 | ts := httptest.NewUnstartedServer(nil) 327 | ts.EnableHTTP2 = true 328 | ts.StartTLS() 329 | defer ts.Close() 330 | 331 | ln, err := tls.Listen("tcp", "127.0.0.1:0", ts.TLS) 332 | assert.NoError(t, err) 333 | 334 | <-time.After(3 * time.Second) 335 | s, err := NewService(zap.NewExample(), ln, ts.TLS, store) 336 | assert.NoError(t, err) 337 | 338 | err = s.Start() 339 | assert.NoError(t, err) 340 | defer s.Stop(context.Background()) 341 | 342 | removeNodeRequest := &command.RemoveNodeRequest{ 343 | Id: "test-main", 344 | } 345 | store.EXPECT().RemoveNode(removeNodeRequest.Id).Return(nil) 346 | 347 | b, err := jsoniter.Marshal(removeNodeRequest) 348 | assert.NoError(t, err) 349 | r, err := http.NewRequest(http.MethodPut, fmt.Sprintf("https://%s/nodes/remove", s.Addr()), bytes.NewReader(b)) 350 | assert.NoError(t, err) 351 | 352 | resp, err := ts.Client().Do(r) 353 | assert.NoError(t, err) 354 | assert.Equal(t, http.StatusOK, resp.StatusCode) 355 | } 356 | -------------------------------------------------------------------------------- /store/engine.go: -------------------------------------------------------------------------------- 1 | package store 2 | 3 | import ( 4 | "bytes" 5 | "compress/gzip" 6 | "fmt" 7 | "io" 8 | "io/ioutil" 9 | "path/filepath" 10 | "strconv" 11 | "sync" 12 | "time" 13 | 14 | "github.com/casbin/casbin/v2" 15 | jsoniter "github.com/json-iterator/go" 16 | "github.com/pkg/errors" 17 | bolt "go.etcd.io/bbolt" 18 | "go.uber.org/zap" 19 | ) 20 | 21 | const ( 22 | databaseFilename = "casbin.db" 23 | ) 24 | 25 | var ( 26 | policyBucketName = []byte("policy_rules") 27 | ) 28 | 29 | // PolicyOperator is used to update policies and provide persistence. 30 | type PolicyOperator struct { 31 | enforcer casbin.IDistributedEnforcer 32 | db *bolt.DB 33 | l *sync.Mutex 34 | logger *zap.Logger 35 | } 36 | 37 | // NewPolicyOperator returns a PolicyOperator. 38 | func NewPolicyOperator(logger *zap.Logger, path string, e casbin.IDistributedEnforcer) (*PolicyOperator, error) { 39 | p := &PolicyOperator{ 40 | enforcer: e, 41 | l: &sync.Mutex{}, 42 | logger: logger, 43 | } 44 | dbPath := filepath.Join(path, databaseFilename) 45 | if err := p.openDBFile(dbPath); err != nil { 46 | return nil, errors.Wrapf(err, "failed to open bolt file") 47 | } 48 | 49 | return p, nil 50 | } 51 | 52 | // openDBFile opens the bolt file. 53 | func (p *PolicyOperator) openDBFile(dbPath string) error { 54 | if len(dbPath) == 0 { 55 | return errors.New("dbPath cannot be an empty") 56 | } 57 | 58 | boltDB, err := bolt.Open(dbPath, 0666, &bolt.Options{Timeout: 1 * time.Second}) 59 | if err != nil { 60 | return err 61 | } 62 | 63 | p.db = boltDB 64 | 65 | return p.createBucket(policyBucketName) 66 | } 67 | 68 | // Restore is used to restore a database from io.ReadCloser. 69 | func (p *PolicyOperator) Restore(rc io.ReadCloser) error { 70 | p.l.Lock() 71 | defer p.l.Unlock() 72 | 73 | dbPath := p.db.Path() 74 | err := p.db.Close() 75 | if err != nil { 76 | p.logger.Error("failed to close database file", zap.Error(err)) 77 | return err 78 | } 79 | 80 | gz, err := gzip.NewReader(rc) 81 | if err != nil { 82 | p.logger.Error("failed to new gzip", zap.Error(err)) 83 | return err 84 | } 85 | 86 | buf := new(bytes.Buffer) 87 | if _, err := io.Copy(buf, gz); err != nil { 88 | p.logger.Error("failed to copy data", zap.Error(err)) 89 | return err 90 | } 91 | 92 | err = gz.Close() 93 | if err != nil { 94 | p.logger.Error("failed to close the gzip", zap.Error(err)) 95 | return err 96 | } 97 | 98 | err = ioutil.WriteFile(dbPath, buf.Bytes(), 0600) 99 | if err != nil { 100 | p.logger.Error("failed to restore the database file", zap.Error(err)) 101 | return err 102 | } 103 | 104 | err = p.openDBFile(dbPath) 105 | if err != nil { 106 | p.logger.Error("failed to open the database file", zap.Error(err)) 107 | return err 108 | } 109 | 110 | err = p.loadPolicy() 111 | if err != nil { 112 | p.logger.Error("failed to load policy from bolt", zap.Error(err)) 113 | return errors.Wrapf(err, "failed to load policy from bolt") 114 | } 115 | return nil 116 | } 117 | 118 | // Backup writes the database to bytes with gzip. 119 | func (p *PolicyOperator) Backup() ([]byte, error) { 120 | p.l.Lock() 121 | defer p.l.Unlock() 122 | 123 | writer := new(bytes.Buffer) 124 | gz, err := gzip.NewWriterLevel(writer, gzip.BestCompression) 125 | 126 | err = p.db.View(func(tx *bolt.Tx) error { 127 | _, err := tx.WriteTo(gz) 128 | return err 129 | }) 130 | if err != nil { 131 | p.logger.Error("failed to backup database file", zap.Error(err)) 132 | return nil, err 133 | } 134 | 135 | err = gz.Close() 136 | if err != nil { 137 | p.logger.Error("failed to close the gzip", zap.Error(err)) 138 | return nil, err 139 | } 140 | 141 | return writer.Bytes(), nil 142 | } 143 | 144 | // createBucket creates a bucket with the given name. 145 | func (p *PolicyOperator) createBucket(name []byte) error { 146 | return p.db.Update(func(tx *bolt.Tx) error { 147 | _, err := tx.CreateBucketIfNotExists(name) 148 | if err != nil { 149 | return errors.Wrap(err, fmt.Sprintf("failed to create %s bucket", name)) 150 | } 151 | return nil 152 | }) 153 | } 154 | 155 | // loadPolicy clears the policies held by enforcer, and loads policy from database. 156 | func (p *PolicyOperator) loadPolicy() error { 157 | err := p.enforcer.ClearPolicySelf(nil) 158 | if err != nil { 159 | p.logger.Error("failed to call loadPolicy", zap.Error(err)) 160 | return err 161 | } 162 | 163 | err = p.db.View(func(tx *bolt.Tx) error { 164 | bkt := tx.Bucket(policyBucketName) 165 | err := bkt.ForEach(func(k, v []byte) error { 166 | var rule Rule 167 | err := jsoniter.Unmarshal(k, &rule) 168 | if err != nil { 169 | return err 170 | } 171 | 172 | _, err = p.enforcer.AddPoliciesSelf(nil, rule.Sec, rule.PType, [][]string{rule.Rule}) 173 | return err 174 | }) 175 | return err 176 | }) 177 | if err != nil { 178 | p.logger.Error("failed to load policy from database", zap.Error(err)) 179 | } 180 | p.logger.Info("load policy completed") 181 | return err 182 | } 183 | 184 | // AddPolicies adds a set of rules. 185 | func (p *PolicyOperator) AddPolicies(sec, pType string, rules [][]string) error { 186 | p.l.Lock() 187 | defer p.l.Unlock() 188 | 189 | effected, err := p.enforcer.AddPoliciesSelf(nil, sec, pType, rules) 190 | if err != nil { 191 | return err 192 | } 193 | if len(effected) == 0 { 194 | return nil 195 | } 196 | 197 | err = p.db.Update(func(tx *bolt.Tx) error { 198 | bkt := tx.Bucket(policyBucketName) 199 | for _, item := range rules { 200 | key, err := newRuleBytes(sec, pType, item) 201 | if err != nil { 202 | return err 203 | } 204 | 205 | value, err := bkt.NextSequence() 206 | if err != nil { 207 | return err 208 | } 209 | 210 | err = bkt.Put(key, []byte(strconv.FormatUint(value, 10))) 211 | if err != nil { 212 | return err 213 | } 214 | } 215 | return nil 216 | }) 217 | if err != nil { 218 | p.logger.Error("failed to persist to database", zap.Error(err)) 219 | } 220 | 221 | return err 222 | } 223 | 224 | // RemovePolicies removes a set of rules. 225 | func (p *PolicyOperator) RemovePolicies(sec, pType string, rules [][]string) error { 226 | p.l.Lock() 227 | defer p.l.Unlock() 228 | 229 | effected, err := p.enforcer.RemovePoliciesSelf(nil, sec, pType, rules) 230 | if err != nil { 231 | p.logger.Error("failed to call RemovePolicySelf", zap.Error(err)) 232 | return err 233 | } 234 | if len(effected) == 0 { 235 | return nil 236 | } 237 | 238 | err = p.db.Update(func(tx *bolt.Tx) error { 239 | bkt := tx.Bucket(policyBucketName) 240 | for _, item := range rules { 241 | key, err := newRuleBytes(sec, pType, item) 242 | if err != nil { 243 | return err 244 | } 245 | 246 | err = bkt.Delete(key) 247 | if err != nil { 248 | return err 249 | } 250 | } 251 | return nil 252 | }) 253 | 254 | return err 255 | } 256 | 257 | // RemoveFilteredPolicy removes a set of rules that match a pattern. 258 | func (p *PolicyOperator) RemoveFilteredPolicy(sec string, pType string, fieldIndex int, fieldValues ...string) error { 259 | p.l.Lock() 260 | defer p.l.Unlock() 261 | 262 | effected, err := p.enforcer.RemoveFilteredPolicySelf(nil, sec, pType, fieldIndex, fieldValues...) 263 | if err != nil { 264 | p.logger.Error("failed to call RemoveFilteredPolicySelf", zap.Error(err)) 265 | return err 266 | } 267 | if len(effected) == 0 { 268 | return nil 269 | } 270 | 271 | err = p.db.Update(func(tx *bolt.Tx) error { 272 | bkt := tx.Bucket(policyBucketName) 273 | for _, item := range effected { 274 | key, err := newRuleBytes(sec, pType, item) 275 | if err != nil { 276 | return err 277 | } 278 | 279 | err = bkt.Delete(key) 280 | if err != nil { 281 | return err 282 | } 283 | } 284 | return nil 285 | }) 286 | if err != nil { 287 | p.logger.Error("failed to persist to database", zap.Error(err)) 288 | } 289 | 290 | return err 291 | } 292 | 293 | //UpdatePolicy replaces an existing rule. 294 | func (p *PolicyOperator) UpdatePolicy(sec, pType string, oldRule, newRule []string) error { 295 | p.l.Lock() 296 | defer p.l.Unlock() 297 | 298 | effected, err := p.enforcer.UpdatePolicySelf(nil, sec, pType, oldRule, newRule) 299 | if err != nil { 300 | p.logger.Error("failed to call UpdatePolicySelf", zap.Error(err)) 301 | return err 302 | } 303 | if effected == false { 304 | return nil 305 | } 306 | 307 | err = p.db.Update(func(tx *bolt.Tx) error { 308 | bkt := tx.Bucket(policyBucketName) 309 | 310 | newKey, err := newRuleBytes(sec, pType, newRule) 311 | if err != nil { 312 | return err 313 | } 314 | value, err := bkt.NextSequence() 315 | if err != nil { 316 | return err 317 | } 318 | 319 | err = bkt.Put(newKey, []byte(strconv.FormatUint(value, 10))) 320 | if err != nil { 321 | return err 322 | } 323 | 324 | oldKey, err := newRuleBytes(sec, pType, oldRule) 325 | if err != nil { 326 | return err 327 | } 328 | return bkt.Delete(oldKey) 329 | }) 330 | if err != nil { 331 | p.logger.Error("failed to persist to database", zap.Error(err)) 332 | } 333 | 334 | return err 335 | } 336 | 337 | //UpdatePolicies replaces a set of existing rule. 338 | func (p *PolicyOperator) UpdatePolicies(sec, pType string, oldRules, newRules [][]string) error { 339 | p.l.Lock() 340 | defer p.l.Unlock() 341 | 342 | effected, err := p.enforcer.UpdatePoliciesSelf(nil, sec, pType, oldRules, newRules) 343 | if err != nil { 344 | p.logger.Error("failed to call UpdatePoliciesSelf", zap.Error(err)) 345 | return err 346 | } 347 | if effected == false { 348 | return nil 349 | } 350 | 351 | err = p.db.Update(func(tx *bolt.Tx) error { 352 | bkt := tx.Bucket(policyBucketName) 353 | 354 | for _, newRule := range newRules { 355 | newKey, err := newRuleBytes(sec, pType, newRule) 356 | if err != nil { 357 | return err 358 | } 359 | value, err := bkt.NextSequence() 360 | if err != nil { 361 | return err 362 | } 363 | 364 | err = bkt.Put(newKey, []byte(strconv.FormatUint(value, 10))) 365 | if err != nil { 366 | return err 367 | } 368 | } 369 | 370 | for _, oldRule := range oldRules { 371 | oldKey, err := newRuleBytes(sec, pType, oldRule) 372 | if err != nil { 373 | return err 374 | } 375 | return bkt.Delete(oldKey) 376 | } 377 | return nil 378 | }) 379 | if err != nil { 380 | p.logger.Error("failed to persist to database", zap.Error(err)) 381 | } 382 | 383 | return nil 384 | } 385 | 386 | //UpdateFilteredPolicies replaces a set of existing rule. 387 | func (p *PolicyOperator) UpdateFilteredPolicies(sec, pType string, oldRules, newRules [][]string) error { 388 | p.l.Lock() 389 | defer p.l.Unlock() 390 | 391 | effected, err := p.enforcer.UpdatePoliciesSelf(nil, sec, pType, oldRules, newRules) 392 | if err != nil { 393 | p.logger.Error("failed to call UpdatePoliciesSelf", zap.Error(err)) 394 | return err 395 | } 396 | if effected == false { 397 | return nil 398 | } 399 | 400 | err = p.db.Update(func(tx *bolt.Tx) error { 401 | bkt := tx.Bucket(policyBucketName) 402 | 403 | for _, newRule := range newRules { 404 | newKey, err := newRuleBytes(sec, pType, newRule) 405 | if err != nil { 406 | return err 407 | } 408 | value, err := bkt.NextSequence() 409 | if err != nil { 410 | return err 411 | } 412 | 413 | err = bkt.Put(newKey, []byte(strconv.FormatUint(value, 10))) 414 | if err != nil { 415 | return err 416 | } 417 | } 418 | 419 | for _, oldRule := range oldRules { 420 | oldKey, err := newRuleBytes(sec, pType, oldRule) 421 | if err != nil { 422 | return err 423 | } 424 | return bkt.Delete(oldKey) 425 | } 426 | return nil 427 | }) 428 | if err != nil { 429 | p.logger.Error("failed to persist to database", zap.Error(err)) 430 | } 431 | 432 | return nil 433 | } 434 | 435 | // ClearPolicy clears all rules. 436 | func (p *PolicyOperator) ClearPolicy() error { 437 | p.l.Lock() 438 | defer p.l.Unlock() 439 | 440 | err := p.enforcer.ClearPolicySelf(nil) 441 | if err != nil { 442 | p.logger.Error("failed to call ClearPolicySelf", zap.Error(err)) 443 | return err 444 | } 445 | 446 | err = p.db.Update(func(tx *bolt.Tx) error { 447 | err := tx.DeleteBucket(policyBucketName) 448 | if err != nil { 449 | return err 450 | } 451 | _, err = tx.CreateBucket(policyBucketName) 452 | if err != nil { 453 | return err 454 | } 455 | return nil 456 | }) 457 | if err != nil { 458 | p.logger.Error("failed to persist to database", zap.Error(err)) 459 | } 460 | 461 | return err 462 | } 463 | 464 | type Rule struct { 465 | Sec string `json:"sec"` 466 | PType string `json:"p_type"` 467 | Rule []string `json:"rule"` 468 | } 469 | 470 | func newRuleBytes(sec, pType string, rule []string) ([]byte, error) { 471 | r := Rule{ 472 | Sec: sec, 473 | PType: pType, 474 | Rule: rule, 475 | } 476 | 477 | key, err := jsoniter.Marshal(r) 478 | if err != nil { 479 | return nil, err 480 | } 481 | return key, nil 482 | } 483 | -------------------------------------------------------------------------------- /store/engine_test.go: -------------------------------------------------------------------------------- 1 | package store 2 | 3 | import ( 4 | "bytes" 5 | "io/ioutil" 6 | "os" 7 | "path" 8 | "testing" 9 | 10 | "go.uber.org/zap" 11 | 12 | "github.com/casbin/hraft-dispatcher/store/mocks" 13 | "github.com/golang/mock/gomock" 14 | "github.com/stretchr/testify/assert" 15 | ) 16 | 17 | //go:generate mockgen -destination ./mocks/mock_distributed_enforcer.go -package mocks github.com/casbin/casbin/v2 IDistributedEnforcer 18 | 19 | func TestPolicyOperator_AddPolicies(t *testing.T) { 20 | ctl := gomock.NewController(t) 21 | defer ctl.Finish() 22 | 23 | e := mocks.NewMockIDistributedEnforcer(ctl) 24 | 25 | dir, err := ioutil.TempDir("", "casbin-hraft-") 26 | assert.NoError(t, err) 27 | defer os.RemoveAll(dir) 28 | 29 | p, err := NewPolicyOperator(zap.NewExample(), dir, e) 30 | assert.NoError(t, err) 31 | 32 | e.EXPECT().AddPoliciesSelf(nil, "p", "p", [][]string{{"role:admin", "/", "*"}, {"role:user", "/", "GET"}}).Return([][]string{{"role:admin", "/", "*"}, {"role:user", "/", "GET"}}, nil) 33 | err = p.AddPolicies("p", "p", [][]string{{"role:admin", "/", "*"}, {"role:user", "/", "GET"}}) 34 | assert.NoError(t, err) 35 | } 36 | 37 | func TestPolicyOperator_RemovePolicies(t *testing.T) { 38 | ctl := gomock.NewController(t) 39 | defer ctl.Finish() 40 | 41 | e := mocks.NewMockIDistributedEnforcer(ctl) 42 | 43 | dir, err := ioutil.TempDir("", "casbin-hraft-") 44 | assert.NoError(t, err) 45 | defer os.RemoveAll(dir) 46 | 47 | p, err := NewPolicyOperator(zap.NewExample(), dir, e) 48 | assert.NoError(t, err) 49 | 50 | e.EXPECT().RemovePoliciesSelf(nil, "p", "p", [][]string{{"role:admin", "/", "*"}, {"role:user", "/", "GET"}}).Return([][]string{{"role:admin", "/", "*"}, {"role:user", "/", "GET"}}, nil) 51 | err = p.RemovePolicies("p", "p", [][]string{{"role:admin", "/", "*"}, {"role:user", "/", "GET"}}) 52 | assert.NoError(t, err) 53 | } 54 | 55 | func TestPolicyOperator_RemoveFilteredPolicy(t *testing.T) { 56 | ctl := gomock.NewController(t) 57 | defer ctl.Finish() 58 | 59 | e := mocks.NewMockIDistributedEnforcer(ctl) 60 | 61 | dir, err := ioutil.TempDir("", "casbin-hraft-") 62 | assert.NoError(t, err) 63 | defer os.RemoveAll(dir) 64 | 65 | p, err := NewPolicyOperator(zap.NewExample(), dir, e) 66 | assert.NoError(t, err) 67 | 68 | e.EXPECT().RemoveFilteredPolicySelf(nil, "p", "p", 0, "role:user").Return([][]string{{"role:user", "/", "GET"}}, nil) 69 | err = p.RemoveFilteredPolicy("p", "p", 0, "role:user") 70 | assert.NoError(t, err) 71 | } 72 | 73 | func TestPolicyOperator_UpdatePolicy(t *testing.T) { 74 | ctl := gomock.NewController(t) 75 | defer ctl.Finish() 76 | 77 | e := mocks.NewMockIDistributedEnforcer(ctl) 78 | 79 | dir, err := ioutil.TempDir("", "casbin-hraft-") 80 | assert.NoError(t, err) 81 | defer os.RemoveAll(dir) 82 | 83 | p, err := NewPolicyOperator(zap.NewExample(), dir, e) 84 | assert.NoError(t, err) 85 | 86 | e.EXPECT().UpdatePolicySelf(nil, "p", "p", []string{"role:admin", "/", "*"}, []string{"role:admin", "/admin", "*"}).Return(true, nil) 87 | err = p.UpdatePolicy("p", "p", []string{"role:admin", "/", "*"}, []string{"role:admin", "/admin", "*"}) 88 | assert.NoError(t, err) 89 | } 90 | 91 | func TestPolicyOperator_LoadPolicy(t *testing.T) { 92 | ctl := gomock.NewController(t) 93 | defer ctl.Finish() 94 | 95 | e := mocks.NewMockIDistributedEnforcer(ctl) 96 | 97 | dir, err := ioutil.TempDir("", "casbin-hraft-") 98 | assert.NoError(t, err) 99 | defer os.RemoveAll(dir) 100 | 101 | p, err := NewPolicyOperator(zap.NewExample(), dir, e) 102 | assert.NoError(t, err) 103 | 104 | e.EXPECT().AddPoliciesSelf(nil, "p", "p", [][]string{{"role:admin", "/", "*"}, {"role:user", "/", "GET"}}).Return([][]string{{"role:admin", "/", "*"}, {"role:user", "/", "GET"}}, nil) 105 | err = p.AddPolicies("p", "p", [][]string{{"role:admin", "/", "*"}, {"role:user", "/", "GET"}}) 106 | assert.NoError(t, err) 107 | 108 | e.EXPECT().ClearPolicySelf(nil) 109 | e.EXPECT().AddPoliciesSelf(nil, "p", "p", [][]string{{"role:admin", "/", "*"}}) 110 | e.EXPECT().AddPoliciesSelf(nil, "p", "p", [][]string{{"role:user", "/", "GET"}}) 111 | err = p.loadPolicy() 112 | assert.NoError(t, err) 113 | } 114 | 115 | func TestPolicyOperator_Backup_Restore(t *testing.T) { 116 | ctl := gomock.NewController(t) 117 | defer ctl.Finish() 118 | 119 | e := mocks.NewMockIDistributedEnforcer(ctl) 120 | 121 | dir, err := ioutil.TempDir("", "casbin-hraft-") 122 | assert.NoError(t, err) 123 | defer os.RemoveAll(dir) 124 | 125 | p, err := NewPolicyOperator(zap.NewExample(), dir, e) 126 | assert.NoError(t, err) 127 | 128 | e.EXPECT().AddPoliciesSelf(nil, "p", "p", [][]string{{"role:admin", "/", "*"}, {"role:user", "/", "GET"}}).Return([][]string{{"role:admin", "/", "*"}, {"role:user", "/", "GET"}}, nil) 129 | err = p.AddPolicies("p", "p", [][]string{{"role:admin", "/", "*"}, {"role:user", "/", "GET"}}) 130 | assert.NoError(t, err) 131 | 132 | b, err := p.Backup() 133 | err = ioutil.WriteFile(path.Join(dir, "backup.db"), b, 0666) 134 | assert.NoError(t, err) 135 | 136 | e.EXPECT().ClearPolicySelf(nil) 137 | e.EXPECT().AddPoliciesSelf(nil, "p", "p", [][]string{{"role:admin", "/", "*"}}) 138 | e.EXPECT().AddPoliciesSelf(nil, "p", "p", [][]string{{"role:user", "/", "GET"}}) 139 | err = p.Restore(ioutil.NopCloser(bytes.NewBuffer(b))) 140 | assert.NoError(t, err) 141 | } 142 | -------------------------------------------------------------------------------- /store/fsm.go: -------------------------------------------------------------------------------- 1 | package store 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/casbin/hraft-dispatcher/command" 7 | "google.golang.org/protobuf/proto" 8 | 9 | "io" 10 | 11 | "github.com/casbin/casbin/v2" 12 | "github.com/hashicorp/raft" 13 | "go.uber.org/zap" 14 | ) 15 | 16 | // FSM is state storage. 17 | type FSM struct { 18 | logger *zap.Logger 19 | policyOperator *PolicyOperator 20 | } 21 | 22 | // NewFSM returns a FSM. 23 | func NewFSM(logger *zap.Logger, path string, enforcer casbin.IDistributedEnforcer) (*FSM, error) { 24 | p, err := NewPolicyOperator(logger, path, enforcer) 25 | if err != nil { 26 | return nil, err 27 | } 28 | 29 | f := &FSM{ 30 | logger: logger, 31 | policyOperator: p, 32 | } 33 | return f, err 34 | } 35 | 36 | // Apply applies log from raft. 37 | // It will parse the command of casbin from log, and pass the command to casbin. 38 | func (f *FSM) Apply(log *raft.Log) interface{} { 39 | var cmd command.Command 40 | err := proto.Unmarshal(log.Data, &cmd) 41 | if err != nil { 42 | f.logger.Error("cannot to unmarshal the command", zap.Error(err), zap.ByteString("command", log.Data)) 43 | return err 44 | } 45 | switch cmd.Type { 46 | case command.Command_COMMAND_TYPE_ADD_POLICIES: 47 | var request command.AddPoliciesRequest 48 | err := proto.Unmarshal(cmd.Data, &request) 49 | if err != nil { 50 | f.logger.Error("cannot to unmarshal the request", zap.Error(err), zap.ByteString("request", cmd.Data)) 51 | return err 52 | } 53 | var rules [][]string 54 | for _, rule := range request.Rules { 55 | rules = append(rules, rule.GetItems()) 56 | } 57 | err = f.policyOperator.AddPolicies(request.Sec, request.PType, rules) 58 | if err != nil { 59 | f.logger.Error("apply the add policies request failed", zap.Error(err), zap.String("request", request.String())) 60 | } else { 61 | f.logger.Info("add policies request applied", 62 | zap.String("sec", request.Sec), 63 | zap.String("pType", request.PType), 64 | zap.Any("rules", rules), 65 | ) 66 | } 67 | return err 68 | case command.Command_COMMAND_TYPE_REMOVE_POLICIES: 69 | var request command.RemovePoliciesRequest 70 | err := proto.Unmarshal(cmd.Data, &request) 71 | if err != nil { 72 | f.logger.Error("cannot to unmarshal the request", zap.Error(err), zap.ByteString("request", cmd.Data)) 73 | return err 74 | } 75 | var rules [][]string 76 | for _, rule := range request.Rules { 77 | rules = append(rules, rule.GetItems()) 78 | } 79 | err = f.policyOperator.RemovePolicies(request.Sec, request.PType, rules) 80 | if err != nil { 81 | f.logger.Error("apply the remove policies request failed", zap.Error(err), zap.String("request", request.String())) 82 | } else { 83 | f.logger.Info("remove policies request applied", 84 | zap.String("sec", request.Sec), 85 | zap.String("pType", request.PType), 86 | zap.Any("rules", rules), 87 | ) 88 | } 89 | return err 90 | case command.Command_COMMAND_TYPE_REMOVE_FILTERED_POLICY: 91 | var request command.RemoveFilteredPolicyRequest 92 | err := proto.Unmarshal(cmd.Data, &request) 93 | if err != nil { 94 | f.logger.Error("cannot to unmarshal the request", zap.Error(err), zap.ByteString("request", cmd.Data)) 95 | return err 96 | } 97 | err = f.policyOperator.RemoveFilteredPolicy(request.Sec, request.PType, int(request.FieldIndex), request.FieldValues...) 98 | if err != nil { 99 | f.logger.Error("apply the remove filtered policy request failed", zap.Error(err), zap.String("request", request.String())) 100 | } else { 101 | f.logger.Info("remove filtered policy request applied", 102 | zap.String("sec", request.Sec), 103 | zap.String("pType", request.PType), 104 | zap.Int32("fieldIndex", request.FieldIndex), 105 | zap.Any("fieldValues", request.FieldValues), 106 | ) 107 | } 108 | return err 109 | case command.Command_COMMAND_TYPE_UPDATE_POLICY: 110 | var request command.UpdatePolicyRequest 111 | err := proto.Unmarshal(cmd.Data, &request) 112 | if err != nil { 113 | f.logger.Error("cannot to unmarshal the request", zap.Error(err), zap.ByteString("request", cmd.Data)) 114 | return err 115 | } 116 | err = f.policyOperator.UpdatePolicy(request.Sec, request.PType, request.OldRule, request.NewRule) 117 | if err != nil { 118 | f.logger.Error("apply the update policy request failed", zap.Error(err), zap.String("request", request.String())) 119 | } else { 120 | f.logger.Info("update policy request applied", 121 | zap.String("sec", request.Sec), 122 | zap.String("pType", request.PType), 123 | zap.Any("oldRule", request.OldRule), 124 | zap.Any("newRule", request.NewRule), 125 | ) 126 | } 127 | return err 128 | case command.Command_COMMAND_TYPE_UPDATE_POLICIES: 129 | var request command.UpdatePoliciesRequest 130 | err := proto.Unmarshal(cmd.Data, &request) 131 | if err != nil { 132 | f.logger.Error("cannot to unmarshal the request", zap.Error(err), zap.ByteString("request", cmd.Data)) 133 | return err 134 | } 135 | var oldRules [][]string 136 | for _, rule := range request.OldRules { 137 | oldRules = append(oldRules, rule.GetItems()) 138 | } 139 | var newRules [][]string 140 | for _, rule := range request.NewRules { 141 | newRules = append(newRules, rule.GetItems()) 142 | } 143 | 144 | err = f.policyOperator.UpdatePolicies(request.Sec, request.PType, oldRules, newRules) 145 | if err != nil { 146 | f.logger.Error("apply the update policies request failed", zap.Error(err), zap.String("request", request.String())) 147 | } else { 148 | f.logger.Info("update policies request applied", 149 | zap.String("sec", request.Sec), 150 | zap.String("pType", request.PType), 151 | zap.Any("oldRules", request.OldRules), 152 | zap.Any("newRules", request.NewRules), 153 | ) 154 | } 155 | return err 156 | case command.Command_COMMAND_TYPE_UPDATE_FILTERED_POLICIES: 157 | var request command.UpdateFilteredPoliciesRequest 158 | err := proto.Unmarshal(cmd.Data, &request) 159 | if err != nil { 160 | f.logger.Error("cannot to unmarshal the request", zap.Error(err), zap.ByteString("request", cmd.Data)) 161 | return err 162 | } 163 | var oldRules [][]string 164 | for _, rule := range request.OldRules { 165 | oldRules = append(oldRules, rule.GetItems()) 166 | } 167 | var newRules [][]string 168 | for _, rule := range request.NewRules { 169 | newRules = append(newRules, rule.GetItems()) 170 | } 171 | 172 | err = f.policyOperator.UpdateFilteredPolicies(request.Sec, request.PType, oldRules, newRules) 173 | if err != nil { 174 | f.logger.Error("apply the update policies request failed", zap.Error(err), zap.String("request", request.String())) 175 | } else { 176 | f.logger.Info("update policies request applied", 177 | zap.String("sec", request.Sec), 178 | zap.String("pType", request.PType), 179 | zap.Any("oldRules", request.OldRules), 180 | zap.Any("newRules", request.NewRules), 181 | ) 182 | } 183 | return err 184 | case command.Command_COMMAND_TYPE_CLEAR_POLICY: 185 | err := f.policyOperator.ClearPolicy() 186 | if err != nil { 187 | f.logger.Error("apply the clear policy request failed", zap.Error(err)) 188 | } else { 189 | f.logger.Info("clear policy request applied") 190 | } 191 | return err 192 | default: 193 | err := fmt.Errorf("unknown command: %v", log) 194 | f.logger.Error(err.Error()) 195 | return err 196 | } 197 | } 198 | 199 | // Restore is used to restore an FSM from a snapshot. It is not called 200 | // concurrently with any other command. The FSM must discard all previous 201 | // state. 202 | // When Raft starts, if the local snapshot exists, call Restore, otherwise 203 | // read log from logstore and pass it to Apply. 204 | func (f *FSM) Restore(rc io.ReadCloser) error { 205 | f.logger.Info("restore an FSM from the snapshot") 206 | err := f.policyOperator.Restore(rc) 207 | if err != nil { 208 | f.logger.Error("failed to restore an FSM from the snapshot", zap.Error(err)) 209 | } 210 | return err 211 | } 212 | 213 | // Snapshot is used to support log compaction. This call should 214 | // return an FSMSnapshot which can be used to save a point-in-time 215 | // snapshot of the FSM. Apply and Snapshot are not called in multiple 216 | // threads, but Apply will be called concurrently with Persist. This means 217 | // the FSM should be implemented in a fashion that allows for concurrent 218 | // updates while a snapshot is happening. 219 | func (f *FSM) Snapshot() (raft.FSMSnapshot, error) { 220 | f.logger.Info("save the snapshot to local") 221 | b, err := f.policyOperator.Backup() 222 | if err != nil { 223 | f.logger.Error("failed to save the snapshot", zap.Error(err)) 224 | } 225 | return &fsmSnapshot{data: b, logger: f.logger}, nil 226 | } 227 | 228 | type fsmSnapshot struct { 229 | data []byte 230 | logger *zap.Logger 231 | } 232 | 233 | func (f *fsmSnapshot) Persist(sink raft.SnapshotSink) error { 234 | err := func() error { 235 | if _, err := sink.Write(f.data); err != nil { 236 | f.logger.Error("cannot to write to sink", zap.Error(err)) 237 | return err 238 | } 239 | return sink.Close() 240 | }() 241 | 242 | if err != nil { 243 | f.logger.Error("cannot to persist the fsm snapshot", zap.Error(err)) 244 | return sink.Cancel() 245 | } 246 | 247 | return nil 248 | } 249 | 250 | func (f *fsmSnapshot) Release() { 251 | // noop 252 | } 253 | -------------------------------------------------------------------------------- /store/logstore/bolt_store.go: -------------------------------------------------------------------------------- 1 | package logstore 2 | 3 | import ( 4 | "errors" 5 | 6 | "github.com/hashicorp/raft" 7 | bolt "go.etcd.io/bbolt" 8 | ) 9 | 10 | const ( 11 | // Permissions to use on the db file. This is only used if the 12 | // database file does not exist and needs to be created. 13 | dbFileMode = 0600 14 | ) 15 | 16 | var ( 17 | // Bucket names we perform transactions in 18 | dbLogs = []byte("logs") 19 | dbConf = []byte("conf") 20 | 21 | // An error indicating a given key does not exist 22 | ErrKeyNotFound = errors.New("not found") 23 | ) 24 | 25 | // BoltStore provides access to BoltDB for Raft to store and retrieve 26 | // log entries. It also provides key/value storage, and can be used as 27 | // a LogStore and StableStore. 28 | type BoltStore struct { 29 | // conn is the underlying handle to the db. 30 | conn *bolt.DB 31 | 32 | // The path to the Bolt database file 33 | path string 34 | } 35 | 36 | // Options contains all the configuraiton used to open the BoltDB 37 | type Options struct { 38 | // Path is the file path to the BoltDB to use 39 | Path string 40 | 41 | // BoltOptions contains any specific BoltDB options you might 42 | // want to specify [e.g. open timeout] 43 | BoltOptions *bolt.Options 44 | 45 | // NoSync causes the database to skip fsync calls after each 46 | // write to the log. This is unsafe, so it should be used 47 | // with caution. 48 | NoSync bool 49 | } 50 | 51 | // readOnly returns true if the contained bolt options say to open 52 | // the DB in readOnly mode [this can be useful to tools that want 53 | // to examine the log] 54 | func (o *Options) readOnly() bool { 55 | return o != nil && o.BoltOptions != nil && o.BoltOptions.ReadOnly 56 | } 57 | 58 | // NewBoltStore takes a file path and returns a connected Raft backend. 59 | func NewBoltStore(path string) (*BoltStore, error) { 60 | return New(Options{Path: path}) 61 | } 62 | 63 | // New uses the supplied options to open the BoltDB and prepare it for use as a raft backend. 64 | func New(options Options) (*BoltStore, error) { 65 | // Try to connect 66 | handle, err := bolt.Open(options.Path, dbFileMode, options.BoltOptions) 67 | if err != nil { 68 | return nil, err 69 | } 70 | handle.NoSync = options.NoSync 71 | 72 | // Create the new store 73 | store := &BoltStore{ 74 | conn: handle, 75 | path: options.Path, 76 | } 77 | 78 | // If the store was opened read-only, don't try and create buckets 79 | if !options.readOnly() { 80 | // Set up our buckets 81 | if err := store.initialize(); err != nil { 82 | store.Close() 83 | return nil, err 84 | } 85 | } 86 | return store, nil 87 | } 88 | 89 | // initialize is used to set up all of the buckets. 90 | func (b *BoltStore) initialize() error { 91 | tx, err := b.conn.Begin(true) 92 | if err != nil { 93 | return err 94 | } 95 | defer tx.Rollback() 96 | 97 | // Create all the buckets 98 | if _, err := tx.CreateBucketIfNotExists(dbLogs); err != nil { 99 | return err 100 | } 101 | if _, err := tx.CreateBucketIfNotExists(dbConf); err != nil { 102 | return err 103 | } 104 | 105 | return tx.Commit() 106 | } 107 | 108 | // Close is used to gracefully close the DB connection. 109 | func (b *BoltStore) Close() error { 110 | return b.conn.Close() 111 | } 112 | 113 | // FirstIndex returns the first known index from the Raft log. 114 | func (b *BoltStore) FirstIndex() (uint64, error) { 115 | tx, err := b.conn.Begin(false) 116 | if err != nil { 117 | return 0, err 118 | } 119 | defer tx.Rollback() 120 | 121 | curs := tx.Bucket(dbLogs).Cursor() 122 | if first, _ := curs.First(); first == nil { 123 | return 0, nil 124 | } else { 125 | return bytesToUint64(first), nil 126 | } 127 | } 128 | 129 | // LastIndex returns the last known index from the Raft log. 130 | func (b *BoltStore) LastIndex() (uint64, error) { 131 | tx, err := b.conn.Begin(false) 132 | if err != nil { 133 | return 0, err 134 | } 135 | defer tx.Rollback() 136 | 137 | curs := tx.Bucket(dbLogs).Cursor() 138 | if last, _ := curs.Last(); last == nil { 139 | return 0, nil 140 | } else { 141 | return bytesToUint64(last), nil 142 | } 143 | } 144 | 145 | // GetLog is used to retrieve a log from BoltDB at a given index. 146 | func (b *BoltStore) GetLog(idx uint64, log *raft.Log) error { 147 | tx, err := b.conn.Begin(false) 148 | if err != nil { 149 | return err 150 | } 151 | defer tx.Rollback() 152 | 153 | bucket := tx.Bucket(dbLogs) 154 | val := bucket.Get(uint64ToBytes(idx)) 155 | 156 | if val == nil { 157 | return raft.ErrLogNotFound 158 | } 159 | return decodeMsgPack(val, log) 160 | } 161 | 162 | // StoreLog is used to store a single raft log 163 | func (b *BoltStore) StoreLog(log *raft.Log) error { 164 | return b.StoreLogs([]*raft.Log{log}) 165 | } 166 | 167 | // StoreLogs is used to store a set of raft logs 168 | func (b *BoltStore) StoreLogs(logs []*raft.Log) error { 169 | tx, err := b.conn.Begin(true) 170 | if err != nil { 171 | return err 172 | } 173 | defer tx.Rollback() 174 | 175 | for _, log := range logs { 176 | key := uint64ToBytes(log.Index) 177 | val, err := encodeMsgPack(log) 178 | if err != nil { 179 | return err 180 | } 181 | bucket := tx.Bucket(dbLogs) 182 | if err := bucket.Put(key, val.Bytes()); err != nil { 183 | return err 184 | } 185 | } 186 | 187 | return tx.Commit() 188 | } 189 | 190 | // DeleteRange is used to delete logs within a given range inclusively. 191 | func (b *BoltStore) DeleteRange(min, max uint64) error { 192 | minKey := uint64ToBytes(min) 193 | 194 | tx, err := b.conn.Begin(true) 195 | if err != nil { 196 | return err 197 | } 198 | defer tx.Rollback() 199 | 200 | curs := tx.Bucket(dbLogs).Cursor() 201 | for k, _ := curs.Seek(minKey); k != nil; k, _ = curs.Next() { 202 | // Handle out-of-range log index 203 | if bytesToUint64(k) > max { 204 | break 205 | } 206 | 207 | // Delete in-range log index 208 | if err := curs.Delete(); err != nil { 209 | return err 210 | } 211 | } 212 | 213 | return tx.Commit() 214 | } 215 | 216 | // Set is used to set a key/value set outside of the raft log 217 | func (b *BoltStore) Set(k, v []byte) error { 218 | tx, err := b.conn.Begin(true) 219 | if err != nil { 220 | return err 221 | } 222 | defer tx.Rollback() 223 | 224 | bucket := tx.Bucket(dbConf) 225 | if err := bucket.Put(k, v); err != nil { 226 | return err 227 | } 228 | 229 | return tx.Commit() 230 | } 231 | 232 | // Get is used to retrieve a value from the k/v store by key 233 | func (b *BoltStore) Get(k []byte) ([]byte, error) { 234 | tx, err := b.conn.Begin(false) 235 | if err != nil { 236 | return nil, err 237 | } 238 | defer tx.Rollback() 239 | 240 | bucket := tx.Bucket(dbConf) 241 | val := bucket.Get(k) 242 | 243 | if val == nil { 244 | return nil, ErrKeyNotFound 245 | } 246 | return append([]byte(nil), val...), nil 247 | } 248 | 249 | // SetUint64 is like Set, but handles uint64 values 250 | func (b *BoltStore) SetUint64(key []byte, val uint64) error { 251 | return b.Set(key, uint64ToBytes(val)) 252 | } 253 | 254 | // GetUint64 is like Get, but handles uint64 values 255 | func (b *BoltStore) GetUint64(key []byte) (uint64, error) { 256 | val, err := b.Get(key) 257 | if err != nil { 258 | return 0, err 259 | } 260 | return bytesToUint64(val), nil 261 | } 262 | 263 | // Sync performs an fsync on the database file handle. This is not necessary 264 | // under normal operation unless NoSync is enabled, in which this forces the 265 | // database file to sync against the disk. 266 | func (b *BoltStore) Sync() error { 267 | return b.conn.Sync() 268 | } 269 | -------------------------------------------------------------------------------- /store/logstore/bolt_store_test.go: -------------------------------------------------------------------------------- 1 | package logstore 2 | 3 | import ( 4 | "bytes" 5 | "io/ioutil" 6 | "os" 7 | "reflect" 8 | "testing" 9 | "time" 10 | 11 | "github.com/hashicorp/raft" 12 | bolt "go.etcd.io/bbolt" 13 | ) 14 | 15 | func testBoltStore(t testing.TB) *BoltStore { 16 | fh, err := ioutil.TempFile("", "bolt") 17 | if err != nil { 18 | t.Fatalf("err: %s", err) 19 | } 20 | defer os.RemoveAll(fh.Name()) 21 | 22 | // Successfully creates and returns a store 23 | store, err := NewBoltStore(fh.Name()) 24 | if err != nil { 25 | t.Fatalf("err: %s", err) 26 | } 27 | 28 | return store 29 | } 30 | 31 | func testRaftLog(idx uint64, data string) *raft.Log { 32 | return &raft.Log{ 33 | Data: []byte(data), 34 | Index: idx, 35 | } 36 | } 37 | 38 | func TestBoltStore_Implements(t *testing.T) { 39 | var store interface{} = &BoltStore{} 40 | if _, ok := store.(raft.StableStore); !ok { 41 | t.Fatalf("BoltStore does not implement raft.StableStore") 42 | } 43 | if _, ok := store.(raft.LogStore); !ok { 44 | t.Fatalf("BoltStore does not implement raft.LogStore") 45 | } 46 | } 47 | 48 | func TestBoltOptionsTimeout(t *testing.T) { 49 | fh, err := ioutil.TempFile("", "bolt") 50 | if err != nil { 51 | t.Fatalf("err: %s", err) 52 | } 53 | defer os.RemoveAll(fh.Name()) 54 | 55 | options := Options{ 56 | Path: fh.Name(), 57 | BoltOptions: &bolt.Options{ 58 | Timeout: time.Second / 10, 59 | }, 60 | } 61 | store, err := New(options) 62 | if err != nil { 63 | t.Fatalf("err: %v", err) 64 | } 65 | defer store.Close() 66 | 67 | // trying to open it again should timeout 68 | doneCh := make(chan error, 1) 69 | go func() { 70 | _, err := New(options) 71 | doneCh <- err 72 | }() 73 | select { 74 | case err := <-doneCh: 75 | if err == nil || err.Error() != "timeout" { 76 | t.Errorf("Expected timeout error but got %v", err) 77 | } 78 | case <-time.After(5 * time.Second): 79 | t.Errorf("Gave up waiting for timeout response") 80 | } 81 | } 82 | 83 | func TestBoltOptionsReadOnly(t *testing.T) { 84 | fh, err := ioutil.TempFile("", "bolt") 85 | if err != nil { 86 | t.Fatalf("err: %s", err) 87 | } 88 | defer os.RemoveAll(fh.Name()) 89 | store, err := NewBoltStore(fh.Name()) 90 | if err != nil { 91 | t.Fatalf("err: %s", err) 92 | } 93 | // Create the log 94 | log := &raft.Log{ 95 | Data: []byte("log1"), 96 | Index: 1, 97 | } 98 | // Attempt to store the log 99 | if err := store.StoreLog(log); err != nil { 100 | t.Fatalf("err: %s", err) 101 | } 102 | 103 | store.Close() 104 | options := Options{ 105 | Path: fh.Name(), 106 | BoltOptions: &bolt.Options{ 107 | Timeout: time.Second / 10, 108 | ReadOnly: true, 109 | }, 110 | } 111 | roStore, err := New(options) 112 | if err != nil { 113 | t.Fatalf("err: %s", err) 114 | } 115 | defer roStore.Close() 116 | result := new(raft.Log) 117 | if err := roStore.GetLog(1, result); err != nil { 118 | t.Fatalf("err: %s", err) 119 | } 120 | 121 | // Ensure the log comes back the same 122 | if !reflect.DeepEqual(log, result) { 123 | t.Errorf("bad: %v", result) 124 | } 125 | // Attempt to store the log, should fail on a read-only store 126 | err = roStore.StoreLog(log) 127 | if err != bolt.ErrDatabaseReadOnly { 128 | t.Errorf("expecting error %v, but got %v", bolt.ErrDatabaseReadOnly, err) 129 | } 130 | } 131 | 132 | func TestNewBoltStore(t *testing.T) { 133 | fh, err := ioutil.TempFile("", "bolt") 134 | if err != nil { 135 | t.Fatalf("err: %s", err) 136 | } 137 | defer os.RemoveAll(fh.Name()) 138 | 139 | // Successfully creates and returns a store 140 | store, err := NewBoltStore(fh.Name()) 141 | if err != nil { 142 | t.Fatalf("err: %s", err) 143 | } 144 | 145 | // Ensure the file was created 146 | if store.path != fh.Name() { 147 | t.Fatalf("unexpected file path %q", store.path) 148 | } 149 | if _, err := os.Stat(fh.Name()); err != nil { 150 | t.Fatalf("err: %s", err) 151 | } 152 | 153 | // Close the store so we can open again 154 | if err := store.Close(); err != nil { 155 | t.Fatalf("err: %s", err) 156 | } 157 | 158 | // Ensure our tables were created 159 | db, err := bolt.Open(fh.Name(), dbFileMode, nil) 160 | if err != nil { 161 | t.Fatalf("err: %s", err) 162 | } 163 | tx, err := db.Begin(true) 164 | if err != nil { 165 | t.Fatalf("err: %s", err) 166 | } 167 | if _, err := tx.CreateBucket([]byte(dbLogs)); err != bolt.ErrBucketExists { 168 | t.Fatalf("bad: %v", err) 169 | } 170 | if _, err := tx.CreateBucket([]byte(dbConf)); err != bolt.ErrBucketExists { 171 | t.Fatalf("bad: %v", err) 172 | } 173 | } 174 | 175 | func TestBoltStore_FirstIndex(t *testing.T) { 176 | store := testBoltStore(t) 177 | defer store.Close() 178 | defer os.Remove(store.path) 179 | 180 | // Should get 0 index on empty log 181 | idx, err := store.FirstIndex() 182 | if err != nil { 183 | t.Fatalf("err: %s", err) 184 | } 185 | if idx != 0 { 186 | t.Fatalf("bad: %v", idx) 187 | } 188 | 189 | // Set a mock raft log 190 | logs := []*raft.Log{ 191 | testRaftLog(1, "log1"), 192 | testRaftLog(2, "log2"), 193 | testRaftLog(3, "log3"), 194 | } 195 | if err := store.StoreLogs(logs); err != nil { 196 | t.Fatalf("bad: %s", err) 197 | } 198 | 199 | // Fetch the first Raft index 200 | idx, err = store.FirstIndex() 201 | if err != nil { 202 | t.Fatalf("err: %s", err) 203 | } 204 | if idx != 1 { 205 | t.Fatalf("bad: %d", idx) 206 | } 207 | } 208 | 209 | func TestBoltStore_LastIndex(t *testing.T) { 210 | store := testBoltStore(t) 211 | defer store.Close() 212 | defer os.Remove(store.path) 213 | 214 | // Should get 0 index on empty log 215 | idx, err := store.LastIndex() 216 | if err != nil { 217 | t.Fatalf("err: %s", err) 218 | } 219 | if idx != 0 { 220 | t.Fatalf("bad: %v", idx) 221 | } 222 | 223 | // Set a mock raft log 224 | logs := []*raft.Log{ 225 | testRaftLog(1, "log1"), 226 | testRaftLog(2, "log2"), 227 | testRaftLog(3, "log3"), 228 | } 229 | if err := store.StoreLogs(logs); err != nil { 230 | t.Fatalf("bad: %s", err) 231 | } 232 | 233 | // Fetch the last Raft index 234 | idx, err = store.LastIndex() 235 | if err != nil { 236 | t.Fatalf("err: %s", err) 237 | } 238 | if idx != 3 { 239 | t.Fatalf("bad: %d", idx) 240 | } 241 | } 242 | 243 | func TestBoltStore_GetLog(t *testing.T) { 244 | store := testBoltStore(t) 245 | defer store.Close() 246 | defer os.RemoveAll(store.path) 247 | 248 | log := new(raft.Log) 249 | 250 | // Should return an error on non-existent log 251 | if err := store.GetLog(1, log); err != raft.ErrLogNotFound { 252 | t.Fatalf("expected raft log not found error, got: %v", err) 253 | } 254 | 255 | // Set a mock raft log 256 | logs := []*raft.Log{ 257 | testRaftLog(1, "log1"), 258 | testRaftLog(2, "log2"), 259 | testRaftLog(3, "log3"), 260 | } 261 | if err := store.StoreLogs(logs); err != nil { 262 | t.Fatalf("bad: %s", err) 263 | } 264 | 265 | // Should return the proper log 266 | if err := store.GetLog(2, log); err != nil { 267 | t.Fatalf("err: %s", err) 268 | } 269 | if !reflect.DeepEqual(log, logs[1]) { 270 | t.Fatalf("bad: %#v", log) 271 | } 272 | } 273 | 274 | func TestBoltStore_SetLog(t *testing.T) { 275 | store := testBoltStore(t) 276 | defer store.Close() 277 | defer os.RemoveAll(store.path) 278 | 279 | // Create the log 280 | log := &raft.Log{ 281 | Data: []byte("log1"), 282 | Index: 1, 283 | } 284 | 285 | // Attempt to store the log 286 | if err := store.StoreLog(log); err != nil { 287 | t.Fatalf("err: %s", err) 288 | } 289 | 290 | // Retrieve the log again 291 | result := new(raft.Log) 292 | if err := store.GetLog(1, result); err != nil { 293 | t.Fatalf("err: %s", err) 294 | } 295 | 296 | // Ensure the log comes back the same 297 | if !reflect.DeepEqual(log, result) { 298 | t.Fatalf("bad: %v", result) 299 | } 300 | } 301 | 302 | func TestBoltStore_SetLogs(t *testing.T) { 303 | store := testBoltStore(t) 304 | defer store.Close() 305 | defer os.RemoveAll(store.path) 306 | 307 | // Create a set of logs 308 | logs := []*raft.Log{ 309 | testRaftLog(1, "log1"), 310 | testRaftLog(2, "log2"), 311 | } 312 | 313 | // Attempt to store the logs 314 | if err := store.StoreLogs(logs); err != nil { 315 | t.Fatalf("err: %s", err) 316 | } 317 | 318 | // Ensure we stored them all 319 | result1, result2 := new(raft.Log), new(raft.Log) 320 | if err := store.GetLog(1, result1); err != nil { 321 | t.Fatalf("err: %s", err) 322 | } 323 | if !reflect.DeepEqual(logs[0], result1) { 324 | t.Fatalf("bad: %#v", result1) 325 | } 326 | if err := store.GetLog(2, result2); err != nil { 327 | t.Fatalf("err: %s", err) 328 | } 329 | if !reflect.DeepEqual(logs[1], result2) { 330 | t.Fatalf("bad: %#v", result2) 331 | } 332 | } 333 | 334 | func TestBoltStore_DeleteRange(t *testing.T) { 335 | store := testBoltStore(t) 336 | defer store.Close() 337 | defer os.RemoveAll(store.path) 338 | 339 | // Create a set of logs 340 | log1 := testRaftLog(1, "log1") 341 | log2 := testRaftLog(2, "log2") 342 | log3 := testRaftLog(3, "log3") 343 | logs := []*raft.Log{log1, log2, log3} 344 | 345 | // Attempt to store the logs 346 | if err := store.StoreLogs(logs); err != nil { 347 | t.Fatalf("err: %s", err) 348 | } 349 | 350 | // Attempt to delete a range of logs 351 | if err := store.DeleteRange(1, 2); err != nil { 352 | t.Fatalf("err: %s", err) 353 | } 354 | 355 | // Ensure the logs were deleted 356 | if err := store.GetLog(1, new(raft.Log)); err != raft.ErrLogNotFound { 357 | t.Fatalf("should have deleted log1") 358 | } 359 | if err := store.GetLog(2, new(raft.Log)); err != raft.ErrLogNotFound { 360 | t.Fatalf("should have deleted log2") 361 | } 362 | } 363 | 364 | func TestBoltStore_Set_Get(t *testing.T) { 365 | store := testBoltStore(t) 366 | defer store.Close() 367 | defer os.RemoveAll(store.path) 368 | 369 | // Returns error on non-existent key 370 | if _, err := store.Get([]byte("bad")); err != ErrKeyNotFound { 371 | t.Fatalf("expected not found error, got: %q", err) 372 | } 373 | 374 | k, v := []byte("hello"), []byte("world") 375 | 376 | // Try to set a k/v pair 377 | if err := store.Set(k, v); err != nil { 378 | t.Fatalf("err: %s", err) 379 | } 380 | 381 | // Try to read it back 382 | val, err := store.Get(k) 383 | if err != nil { 384 | t.Fatalf("err: %s", err) 385 | } 386 | if !bytes.Equal(val, v) { 387 | t.Fatalf("bad: %v", val) 388 | } 389 | } 390 | 391 | func TestBoltStore_SetUint64_GetUint64(t *testing.T) { 392 | store := testBoltStore(t) 393 | defer store.Close() 394 | defer os.RemoveAll(store.path) 395 | 396 | // Returns error on non-existent key 397 | if _, err := store.GetUint64([]byte("bad")); err != ErrKeyNotFound { 398 | t.Fatalf("expected not found error, got: %q", err) 399 | } 400 | 401 | k, v := []byte("abc"), uint64(123) 402 | 403 | // Attempt to set the k/v pair 404 | if err := store.SetUint64(k, v); err != nil { 405 | t.Fatalf("err: %s", err) 406 | } 407 | 408 | // Read back the value 409 | val, err := store.GetUint64(k) 410 | if err != nil { 411 | t.Fatalf("err: %s", err) 412 | } 413 | if val != v { 414 | t.Fatalf("bad: %v", val) 415 | } 416 | } 417 | -------------------------------------------------------------------------------- /store/logstore/util.go: -------------------------------------------------------------------------------- 1 | package logstore 2 | 3 | import ( 4 | "bytes" 5 | "encoding/binary" 6 | 7 | "github.com/hashicorp/go-msgpack/codec" 8 | ) 9 | 10 | // Decode reverses the encode operation on a byte slice input 11 | func decodeMsgPack(buf []byte, out interface{}) error { 12 | r := bytes.NewBuffer(buf) 13 | hd := codec.MsgpackHandle{} 14 | dec := codec.NewDecoder(r, &hd) 15 | return dec.Decode(out) 16 | } 17 | 18 | // Encode writes an encoded object to a new bytes buffer 19 | func encodeMsgPack(in interface{}) (*bytes.Buffer, error) { 20 | buf := bytes.NewBuffer(nil) 21 | hd := codec.MsgpackHandle{} 22 | enc := codec.NewEncoder(buf, &hd) 23 | err := enc.Encode(in) 24 | return buf, err 25 | } 26 | 27 | // Converts bytes to an integer 28 | func bytesToUint64(b []byte) uint64 { 29 | return binary.BigEndian.Uint64(b) 30 | } 31 | 32 | // Converts a uint to a byte slice 33 | func uint64ToBytes(u uint64) []byte { 34 | buf := make([]byte, 8) 35 | binary.BigEndian.PutUint64(buf, u) 36 | return buf 37 | } 38 | -------------------------------------------------------------------------------- /store/store.go: -------------------------------------------------------------------------------- 1 | package store 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "os" 7 | "path" 8 | "path/filepath" 9 | "time" 10 | 11 | "github.com/casbin/hraft-dispatcher/store/logstore" 12 | 13 | "github.com/cenkalti/backoff/v4" 14 | "github.com/pkg/errors" 15 | 16 | "github.com/casbin/casbin/v2" 17 | "github.com/casbin/hraft-dispatcher/command" 18 | "github.com/casbin/hraft-dispatcher/http" 19 | "google.golang.org/protobuf/proto" 20 | 21 | "github.com/hashicorp/go-multierror" 22 | 23 | "github.com/hashicorp/raft" 24 | "go.uber.org/zap" 25 | ) 26 | 27 | const ( 28 | raftDBName = "raft.db" 29 | retainSnapshotCount = 2 30 | raftTimeout = 10 * time.Second 31 | ) 32 | 33 | var _ http.Store = &Store{} 34 | 35 | // Store is responsible for synchronization policy and storage policy by Raft protocol. 36 | type Store struct { 37 | dataDir string 38 | serverAddress string 39 | serverID string 40 | 41 | raft *raft.Raft 42 | raftConfig *raft.Config 43 | networkTransportConfig *raft.NetworkTransportConfig 44 | transport raft.Transport 45 | snapshotStore raft.SnapshotStore 46 | logStore raft.LogStore 47 | stableStore raft.StableStore 48 | fms raft.FSM 49 | boltStore *logstore.BoltStore 50 | 51 | enforcer casbin.IDistributedEnforcer 52 | 53 | // inMemory is used for testing. 54 | inMemory bool 55 | 56 | logger *zap.Logger 57 | } 58 | 59 | type Config struct { 60 | ID string 61 | Dir string 62 | NetworkTransportConfig *raft.NetworkTransportConfig 63 | Enforcer casbin.IDistributedEnforcer 64 | RaftConfig *raft.Config 65 | } 66 | 67 | // NewStore return a instance of Store. 68 | func NewStore(logger *zap.Logger, config *Config) (*Store, error) { 69 | s := &Store{ 70 | dataDir: config.Dir, 71 | serverID: config.ID, 72 | logger: logger, 73 | networkTransportConfig: config.NetworkTransportConfig, 74 | enforcer: config.Enforcer, 75 | raftConfig: config.RaftConfig, 76 | } 77 | 78 | return s, nil 79 | } 80 | 81 | // Start performs initialization and runs server 82 | func (s *Store) Start(enableBootstrap bool) error { 83 | var config *raft.Config 84 | if s.raftConfig == nil { 85 | config = raft.DefaultConfig() 86 | s.raftConfig = config 87 | } else { 88 | config = s.raftConfig 89 | } 90 | 91 | if len(config.LocalID) == 0 { 92 | config.LocalID = raft.ServerID(s.serverID) 93 | } 94 | 95 | var transport raft.Transport 96 | if s.inMemory { 97 | _, transport = raft.NewInmemTransport(raft.ServerAddress(s.serverAddress)) 98 | } else { 99 | transport = raft.NewNetworkTransportWithConfig(s.networkTransportConfig) 100 | } 101 | s.transport = transport 102 | 103 | var snapshots raft.SnapshotStore 104 | if s.inMemory { 105 | snapshots = raft.NewInmemSnapshotStore() 106 | } else { 107 | fileSnapshots, err := raft.NewFileSnapshotStore(s.dataDir, retainSnapshotCount, os.Stderr) 108 | if err != nil { 109 | s.logger.Error("failed to new file snapshot store", zap.Error(err), zap.String("raftData", s.dataDir)) 110 | return err 111 | } 112 | snapshots = fileSnapshots 113 | } 114 | s.snapshotStore = snapshots 115 | 116 | if s.inMemory { 117 | inMemStore := raft.NewInmemStore() 118 | s.logStore = inMemStore 119 | s.stableStore = inMemStore 120 | } else { 121 | dbPath := filepath.Join(s.dataDir, raftDBName) 122 | boltDB, err := logstore.NewBoltStore(dbPath) 123 | if err != nil { 124 | s.logger.Error("failed to new bolt store", zap.Error(err), zap.String("path", dbPath)) 125 | return err 126 | } 127 | 128 | s.boltStore = boltDB 129 | s.logStore = boltDB 130 | s.stableStore = boltDB 131 | } 132 | 133 | fsm, err := NewFSM(s.logger, s.dataDir, s.enforcer) 134 | if err != nil { 135 | s.logger.Error("failed to new fsm", zap.Error(err)) 136 | return err 137 | } 138 | 139 | ra, err := raft.NewRaft(config, fsm, s.logStore, s.stableStore, s.snapshotStore, s.transport) 140 | if err != nil { 141 | s.logger.Error("failed to new raft", zap.Error(err)) 142 | return err 143 | } 144 | s.raft = ra 145 | 146 | if enableBootstrap { 147 | configuration := raft.Configuration{ 148 | Servers: []raft.Server{ 149 | { 150 | ID: raft.ServerID(s.serverID), 151 | Address: s.transport.LocalAddr(), 152 | }, 153 | }, 154 | } 155 | 156 | f := ra.BootstrapCluster(configuration) 157 | if f.Error() != nil && f.Error() != raft.ErrCantBootstrap { 158 | s.logger.Error("failed to boostrap cluster", zap.Error(f.Error())) 159 | return f.Error() 160 | } 161 | } 162 | s.logger.Info(fmt.Sprintf("listening and serving Raft on %s", transport.LocalAddr())) 163 | return nil 164 | } 165 | 166 | // Stop is used to close the raft node, which always returns nil. 167 | func (s *Store) Stop() error { 168 | var result error 169 | shutdown := s.raft.Shutdown() 170 | if shutdown.Error() != nil { 171 | s.logger.Error("failed to stop the raft server", zap.Error(shutdown.Error())) 172 | result = multierror.Append(result, shutdown.Error()) 173 | } 174 | 175 | if !s.inMemory { 176 | err := s.boltStore.Close() 177 | if err != nil { 178 | s.logger.Error("failed to close bolt database", zap.Error(err)) 179 | result = multierror.Append(result, err) 180 | } 181 | } 182 | 183 | err := s.networkTransportConfig.Stream.Close() 184 | if err != nil { 185 | result = multierror.Append(result, err) 186 | } 187 | 188 | return result 189 | } 190 | 191 | // IsInitializedCluster checks whether the cluster has been initialized. 192 | func (s *Store) IsInitializedCluster() bool { 193 | if _, err := os.Stat(path.Join(s.dataDir, raftDBName)); err != nil { 194 | if os.IsNotExist(err) { 195 | return false 196 | } 197 | } 198 | return true 199 | } 200 | 201 | // WaitLeader detects the leader address in the current cluster. 202 | func (s *Store) WaitLeader() error { 203 | if s.raft.Leader() != "" { 204 | return nil 205 | } 206 | 207 | ticker := backoff.NewTicker(backoff.NewExponentialBackOff()) 208 | ctx, cancel := context.WithTimeout(context.Background(), time.Minute) 209 | defer cancel() 210 | 211 | for { 212 | select { 213 | case <-ctx.Done(): 214 | ticker.Stop() 215 | return errors.New("failed to detect the current leader") 216 | case <-ticker.C: 217 | if s.raft.Leader() != "" { 218 | ticker.Stop() 219 | return nil 220 | } 221 | } 222 | } 223 | } 224 | 225 | // Address returns the address of the current node. 226 | func (s *Store) Address() string { 227 | return s.networkTransportConfig.Stream.Addr().String() 228 | } 229 | 230 | // ID returns the id of the current node. 231 | func (s *Store) ID() string { 232 | return s.serverID 233 | } 234 | 235 | // DataDir returns the data directory of the current node. 236 | func (s *Store) DataDir() string { 237 | return s.dataDir 238 | } 239 | 240 | // applyProtoMessage applies a proto message. 241 | func (s *Store) applyProtoMessage(m proto.Message) error { 242 | cmd, err := proto.Marshal(m) 243 | if err != nil { 244 | return err 245 | } 246 | return s.raft.Apply(cmd, raftTimeout).Error() 247 | } 248 | 249 | // AddPolicy implements the http.Store interface. 250 | func (s *Store) AddPolicies(request *command.AddPoliciesRequest) error { 251 | data, err := proto.Marshal(request) 252 | if err != nil { 253 | return err 254 | } 255 | cmd := &command.Command{ 256 | Type: command.Command_COMMAND_TYPE_ADD_POLICIES, 257 | Data: data, 258 | } 259 | return s.applyProtoMessage(cmd) 260 | } 261 | 262 | // RemovePolicies implements the http.Store interface. 263 | func (s *Store) RemovePolicies(request *command.RemovePoliciesRequest) error { 264 | data, err := proto.Marshal(request) 265 | if err != nil { 266 | return err 267 | } 268 | cmd := &command.Command{ 269 | Type: command.Command_COMMAND_TYPE_REMOVE_POLICIES, 270 | Data: data, 271 | } 272 | return s.applyProtoMessage(cmd) 273 | } 274 | 275 | // RemoveFilteredPolicy implements the http.Store interface. 276 | func (s *Store) RemoveFilteredPolicy(request *command.RemoveFilteredPolicyRequest) error { 277 | data, err := proto.Marshal(request) 278 | if err != nil { 279 | return err 280 | } 281 | cmd := &command.Command{ 282 | Type: command.Command_COMMAND_TYPE_REMOVE_FILTERED_POLICY, 283 | Data: data, 284 | } 285 | return s.applyProtoMessage(cmd) 286 | } 287 | 288 | // UpdatePolicy implements the http.Store interface. 289 | func (s *Store) UpdatePolicy(request *command.UpdatePolicyRequest) error { 290 | data, err := proto.Marshal(request) 291 | if err != nil { 292 | return err 293 | } 294 | cmd := &command.Command{ 295 | Type: command.Command_COMMAND_TYPE_UPDATE_POLICY, 296 | Data: data, 297 | } 298 | return s.applyProtoMessage(cmd) 299 | } 300 | 301 | // UpdatePolicies implements the http.Store interface. 302 | func (s *Store) UpdatePolicies(request *command.UpdatePoliciesRequest) error { 303 | data, err := proto.Marshal(request) 304 | if err != nil { 305 | return err 306 | } 307 | cmd := &command.Command{ 308 | Type: command.Command_COMMAND_TYPE_UPDATE_POLICIES, 309 | Data: data, 310 | } 311 | return s.applyProtoMessage(cmd) 312 | } 313 | 314 | // UpdateFilteredPolicies implements the http.Store interface. 315 | func (s *Store) UpdateFilteredPolicies(request *command.UpdateFilteredPoliciesRequest) error { 316 | data, err := proto.Marshal(request) 317 | if err != nil { 318 | return err 319 | } 320 | cmd := &command.Command{ 321 | Type: command.Command_COMMAND_TYPE_UPDATE_FILTERED_POLICIES, 322 | Data: data, 323 | } 324 | return s.applyProtoMessage(cmd) 325 | } 326 | 327 | // ClearPolicy implements the http.Store interface. 328 | func (s *Store) ClearPolicy() error { 329 | cmd := &command.Command{ 330 | Type: command.Command_COMMAND_TYPE_CLEAR_POLICY, 331 | Data: nil, 332 | } 333 | return s.applyProtoMessage(cmd) 334 | } 335 | 336 | // JoinNode implements the http.Store interface. 337 | func (s *Store) JoinNode(serverID string, address string) error { 338 | i := s.raft.AddVoter(raft.ServerID(serverID), raft.ServerAddress(address), 0, 0) 339 | return i.Error() 340 | } 341 | 342 | // RemoveNode implements the http.Store interface. 343 | func (s *Store) RemoveNode(serverID string) error { 344 | i := s.raft.RemoveServer(raft.ServerID(serverID), 0, 0) 345 | return i.Error() 346 | } 347 | 348 | // Leader implements the http.Store interface. 349 | func (s *Store) Leader() (bool, string) { 350 | _ = s.WaitLeader() 351 | return s.raft.State() == raft.Leader, string(s.raft.Leader()) 352 | } 353 | 354 | // Leader implements the http.Store interface. 355 | func (s *Store) Stats() (map[string]interface{}, error) { 356 | result := map[string]interface{}{ 357 | "node_id": s.serverID, 358 | "node_address": s.networkTransportConfig.Stream.Addr(), 359 | "leader": map[string]string{ 360 | "address": string(s.raft.Leader()), 361 | }, 362 | "data_dir": s.dataDir, 363 | } 364 | 365 | return result, nil 366 | } 367 | -------------------------------------------------------------------------------- /store/store_test.go: -------------------------------------------------------------------------------- 1 | package store 2 | 3 | import ( 4 | "crypto/tls" 5 | "crypto/x509" 6 | "io/ioutil" 7 | "os" 8 | "testing" 9 | "time" 10 | 11 | "go.uber.org/zap" 12 | 13 | "github.com/casbin/casbin/v2" 14 | 15 | "github.com/casbin/hraft-dispatcher/command" 16 | "github.com/casbin/hraft-dispatcher/store/mocks" 17 | "github.com/golang/mock/gomock" 18 | "github.com/hashicorp/raft" 19 | . "github.com/smartystreets/goconvey/convey" 20 | "github.com/stretchr/testify/assert" 21 | ) 22 | 23 | func GetLocalIP() string { 24 | return "127.0.0.1" 25 | } 26 | 27 | var rootCA = []byte(` 28 | -----BEGIN CERTIFICATE----- 29 | MIIDaDCCAlCgAwIBAgIUcpVt/SzT9IPq/HWZOTP8WXB//5owDQYJKoZIhvcNAQEL 30 | BQAwTDELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAkNBMRYwFAYDVQQHEw1TYW4gRnJh 31 | bmNpc2NvMRgwFgYDVQQDEw9ocmFmdGRpc3BhdGNoZXIwHhcNMjEwMjE1MTUwNDAw 32 | WhcNMjYwMjE0MTUwNDAwWjBMMQswCQYDVQQGEwJVUzELMAkGA1UECBMCQ0ExFjAU 33 | BgNVBAcTDVNhbiBGcmFuY2lzY28xGDAWBgNVBAMTD2hyYWZ0ZGlzcGF0Y2hlcjCC 34 | ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL0pK4CAtBjs1IM3savAJs5p 35 | IJdWQRi19SE1U6ypy/UMQJRrhdSm9J1RuYNQfELl0RkWaNDsi0L6G0kbLMs5kkFj 36 | 7ySpPeCTE6UhGOCWXwWQ/eqJNgeJWMwLiErX+fIT/SE3xuu4e5xJkF1gVAGpGp6Z 37 | Uhj2Ilc+xRexIhbv6kgbkh+cG1M0qxGWsb8QQ7ohz2yHMsEkG/lsHxehME0K7Y1j 38 | EotLuAEC/UHSzYqA643V2a7rFSmM+sQMzBJ6H2ENsglpVCfLyoP5iTbhnOnHwqUS 39 | sMk9kfedo0mjLjFmCW81SUjY3Y/h8H2kPPmmy8aUT9H120EGoIre7FZyiZBu5hEC 40 | AwEAAaNCMEAwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0O 41 | BBYEFMvdedWPdZrkG0ZNAV8fIX+F6Dg2MA0GCSqGSIb3DQEBCwUAA4IBAQCmkvnH 42 | XKURz70aU5DXeuR4/8kvEk/Wv25154rZfaa7g7uEfSITvgj+7e65HJW53BT7Xrpj 43 | xfALJriaONcf4xu+ov1rdMdojpYnRwNC4B5tiNUK+IEkr/9iyOk7Tu3fY1hwbzLz 44 | DjfUP+9aOdzBW4xapVpM+YlvIR525rXl+P062SEm9pUyWvDFAb3tKHX9W6Hjzm5w 45 | 50qShl7ncwwtTYUylVUZcS1xOh0vF4WPtGWbuN0jfQjLFDeyG2l78jcl/ngsIiyT 46 | KTExqCSHJ/6FT9zQnYhXEKU4BLIzqeS8Kfnd7h+3URYZNs5GlfujhLflaR6/Xxp1 47 | /P7UwWnW1xZf+iAP 48 | -----END CERTIFICATE----- 49 | `) 50 | 51 | var peerCA = []byte(` 52 | -----BEGIN CERTIFICATE----- 53 | MIIDlzCCAn+gAwIBAgIUUVw2A/52WC/efPDTngQd47EJWqgwDQYJKoZIhvcNAQEL 54 | BQAwTDELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAkNBMRYwFAYDVQQHEw1TYW4gRnJh 55 | bmNpc2NvMRgwFgYDVQQDEw9ocmFmdGRpc3BhdGNoZXIwIBcNMjEwMjE1MTUwNDAw 56 | WhgPMjEyMTAxMjIxNTA0MDBaMEwxCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJDQTEW 57 | MBQGA1UEBxMNU2FuIEZyYW5jaXNjbzEYMBYGA1UEAxMPaHJhZnRkaXNwYXRjaGVy 58 | MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxHaJUQTh3eM8ztbC9Vnq 59 | 9NtqaPJ7+GcymOzPjDBVLGuBzbTY2Hd7D6gV0Hih9QW9yneN/jI9u4e3CHfln3aQ 60 | 0t7rVvu3CAEx+34M49nWIZNebKU+j3YSmCkxl/I2fD3x/vSvTy41Ey4DHVjbw35c 61 | DE/vg6eU+nnkX96tbrVZgUnZfrSaI8xmdO64DLqOCNUcl0LM8zXAgH+bIkKM6y4U 62 | 6AAEsKsZ5CNpMw0nopCnuQZu0NFFNM9qgUdbJLSHOqGbFLiIGX+PNis9dVggZX3U 63 | dzbyFHtAywM53S5Hq0Q2S2dYQHLhSFkWgwCZEDP+Oi3j46vXVHzNZgJng+wHNiCB 64 | zwIDAQABo28wbTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEG 65 | CCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFHPb6d7J8ai2eYyX+0aF 66 | MzKPCHktMA8GA1UdEQQIMAaHBH8AAAEwDQYJKoZIhvcNAQELBQADggEBAHtgS5pR 67 | WL72X6p8+levu0lumia5fOAmbDylarcLn77qFbl0xK6ebDBQx6HXk1ZOzVB+DcJj 68 | z/BJ6gdoT9211C0VeT4VGE8rh28IRL9mhOXv1/sXuJo8qPH+VwR95phFD1+wcwBG 69 | f6GJde1qncnc79Sh+KfcYKpmbNxBLZA96LKRrN+kaytQDeB0EFTZBMc9XhdQsm/8 70 | 3QMkGh7Q1wJNzLmw8nUDmz8gtucQ00pwDe4PKSmrsFSJq62aMYhNeqItEGT3WV0e 71 | EzHG/uXVRwqgOepAA3p4neKaAH1G+oGWEzcI0EPPB3uCJzGLJTEU6QX0ISpyN8sT 72 | ihn/9JzaGivK+fk= 73 | -----END CERTIFICATE----- 74 | `) 75 | 76 | var peerKey = []byte(` 77 | -----BEGIN RSA PRIVATE KEY----- 78 | MIIEowIBAAKCAQEAxHaJUQTh3eM8ztbC9Vnq9NtqaPJ7+GcymOzPjDBVLGuBzbTY 79 | 2Hd7D6gV0Hih9QW9yneN/jI9u4e3CHfln3aQ0t7rVvu3CAEx+34M49nWIZNebKU+ 80 | j3YSmCkxl/I2fD3x/vSvTy41Ey4DHVjbw35cDE/vg6eU+nnkX96tbrVZgUnZfrSa 81 | I8xmdO64DLqOCNUcl0LM8zXAgH+bIkKM6y4U6AAEsKsZ5CNpMw0nopCnuQZu0NFF 82 | NM9qgUdbJLSHOqGbFLiIGX+PNis9dVggZX3UdzbyFHtAywM53S5Hq0Q2S2dYQHLh 83 | SFkWgwCZEDP+Oi3j46vXVHzNZgJng+wHNiCBzwIDAQABAoIBAEOn8oflH4dTHvi3 84 | +rGVgpVKDm4Pu2OC3mjNfHfxmRNP/oaBlf+NveJZZxHAyT1g+cgEvfBhCuNOzFht 85 | ObVdlmgX/oGY86IdD0JlWTkKJnSvlF/j1BSBe8vMu9hwwBSvHGxJhSnGZt6xBL+R 86 | fzTmifpveLMk/ef4HA5r19v9NdKQqhxF3SfY2SPCcdwrtsbTxwKvKXaIKIzQ0j5o 87 | HGQPwiCLWYXWIBfwqpNB0FWpnBhSIUi3U/fYrfdRWsBCaCKaJFLrknTYk2jPTYHd 88 | kXUNBwUyiRpD+c8MCVXhVrd6lg8MYK4qKs2k5VtfDtNM5uxegD2elEvbB9XDbdL9 89 | pXBP0okCgYEA8h51ocpteiyv/kZfHnRgHgIB9zGOBHXK2oIir843jezdTnLjLKqa 90 | GY8/6Rp6Djmbnw8q5wPtP1OgT2Zlvt93fjd1o2zZsA5LfBo07T4BQ0hIPQLSBkl0 91 | GfKKnPJBgKlhDSiopr3xDlPkx8BX9PP1Sf/G5ewYC/QpWW55NP4s02UCgYEAz7n9 92 | 9WRRFa1fXSEvcsXCOxHpKc+/XG6ZfptPnrmHVGv+MrHNE8GTWupoTr8l1gNsp1IH 93 | I3pG0THEMlNGApF4u1v2tZ5yNnsoVeP9Zrwy9i3E56weIOqBlpCxbjW8FPmTgVzt 94 | Xp/2IxBaZQn9YdCLSN2oX1Pqb7K3J84PzaXW/yMCgYBnVuDWQVQgxVoIqXiHwxwT 95 | MsAsBZacCLqgMNMlPlsv1F1Q0nBr7BUBu8aHc6mM0MG/TfX9zAtC8CqIOShMI40Y 96 | 7grjyd3P6woE2hqk98YKNZu/jqidzlQjjwXinvOeOq0VtLjnEkME3oHTUCE6h7W1 97 | 89ms4OwSjg/n/+Lz31i6kQKBgQCXM/M/o/3BoalAyN8Y1ApFpQvre2T3iyn/ll2m 98 | U7XGJbWqgPGd59Gy4915NHn+BhAY2wSHNoJF08vUNflH9UvEVXSHTwYj0hHSM1pI 99 | ZcVSnI4vdIGZxBj/1+LPLh3xxpkwGMxPjHBFpamm0la11G8OYwokGZkUJSpctwmZ 100 | z5VnsQKBgHdLPABKFGfxwsLFEr6vnwS3CmUcwbijDy7YGW//HOSk+sqsPuJclaSt 101 | BIBwORizBRWQd+C5aDa3lu9pkBjlZ+b57RnYq+bzQbNDHUt3c7BtcCujceD0uVXz 102 | /hug3iKji4mRJKDqV+5lYNG+Q6T3vZ5c7NbMo2LmeNXu/1aOxk4z 103 | -----END RSA PRIVATE KEY----- 104 | `) 105 | 106 | func GetTLSConfig() (*tls.Config, error) { 107 | rootCAPool := x509.NewCertPool() 108 | rootCA, err := ioutil.ReadFile("../testdata/ca/ca.pem") 109 | if err != nil { 110 | return nil, err 111 | } 112 | rootCAPool.AppendCertsFromPEM(rootCA) 113 | 114 | cert, err := tls.LoadX509KeyPair("../testdata/ca/peer.pem", "../testdata/ca/peer-key.pem") 115 | if err != nil { 116 | return nil, err 117 | } 118 | 119 | config := &tls.Config{ 120 | RootCAs: rootCAPool, 121 | ClientCAs: rootCAPool, 122 | ClientAuth: tls.RequireAndVerifyClientCert, 123 | Certificates: []tls.Certificate{cert}, 124 | } 125 | 126 | return config, nil 127 | } 128 | 129 | func TestStore_SingleNode(t *testing.T) { 130 | ctl := gomock.NewController(t) 131 | defer ctl.Finish() 132 | 133 | enforcer := mocks.NewMockIDistributedEnforcer(ctl) 134 | raftID := "node-leader" 135 | raftAddress := GetLocalIP() + ":6790" 136 | 137 | store, err := newStore(enforcer, raftID, raftAddress, true) 138 | assert.NoError(t, err) 139 | defer store.Stop() 140 | defer os.RemoveAll(store.DataDir()) 141 | 142 | err = store.WaitLeader() 143 | assert.NoError(t, err) 144 | 145 | Convey("TestStore_SingleNode", t, func() { 146 | Convey("AddPolicy()", func() { 147 | sec := "p" 148 | pType := "p" 149 | originalRules := [][]string{{"role:admin", "/", "*"}} 150 | var rules []*command.StringArray 151 | for _, rule := range originalRules { 152 | rules = append(rules, &command.StringArray{Items: rule}) 153 | } 154 | 155 | request := &command.AddPoliciesRequest{ 156 | Sec: sec, 157 | PType: pType, 158 | Rules: rules, 159 | } 160 | 161 | enforcer.EXPECT().AddPoliciesSelf(nil, sec, pType, originalRules).Return(originalRules, nil) 162 | err := store.AddPolicies(request) 163 | So(err, ShouldBeNil) 164 | }) 165 | 166 | Convey("RemovePolicy()", func() { 167 | sec := "p" 168 | pType := "p" 169 | originalRules := [][]string{{"role:admin", "/", "*"}} 170 | var rules []*command.StringArray 171 | for _, rule := range originalRules { 172 | rules = append(rules, &command.StringArray{Items: rule}) 173 | } 174 | 175 | request := &command.RemovePoliciesRequest{ 176 | Sec: sec, 177 | PType: pType, 178 | Rules: rules, 179 | } 180 | 181 | enforcer.EXPECT().RemovePoliciesSelf(nil, sec, pType, originalRules).Return(originalRules, nil) 182 | err := store.RemovePolicies(request) 183 | So(err, ShouldBeNil) 184 | }) 185 | 186 | Convey("RemoveFilteredPolicy()", func() { 187 | sec := "p" 188 | pType := "p" 189 | fieldIndex := 0 190 | fieldValues := []string{"role:admin"} 191 | effected := [][]string{{"role:admin", "/", "*"}} 192 | 193 | request := &command.RemoveFilteredPolicyRequest{ 194 | Sec: sec, 195 | PType: pType, 196 | FieldIndex: int32(fieldIndex), 197 | FieldValues: fieldValues, 198 | } 199 | 200 | enforcer.EXPECT().RemoveFilteredPolicySelf(nil, sec, pType, fieldIndex, fieldValues).Return(effected, nil) 201 | err := store.RemoveFilteredPolicy(request) 202 | So(err, ShouldBeNil) 203 | }) 204 | 205 | Convey("UpdatePolicy()", func() { 206 | sec := "p" 207 | pType := "p" 208 | oldRule := []string{"role:admin", "/", "*"} 209 | newRule := []string{"role:admin", "/admin", "*"} 210 | 211 | request := &command.UpdatePolicyRequest{ 212 | Sec: sec, 213 | PType: pType, 214 | OldRule: oldRule, 215 | NewRule: newRule, 216 | } 217 | 218 | enforcer.EXPECT().UpdatePolicySelf(nil, sec, pType, oldRule, newRule).Return(true, nil) 219 | err := store.UpdatePolicy(request) 220 | So(err, ShouldBeNil) 221 | }) 222 | 223 | Convey("ClearPolicy()", func() { 224 | enforcer.EXPECT().ClearPolicySelf(nil).Return(nil) 225 | err := store.ClearPolicy() 226 | So(err, ShouldBeNil) 227 | }) 228 | 229 | Convey("ID()", func() { 230 | assert.Equal(t, raftID, store.ID()) 231 | So(store.ID(), ShouldEqual, raftID) 232 | }) 233 | 234 | Convey("Address()", func() { 235 | So(store.Address(), ShouldEqual, raftAddress) 236 | }) 237 | 238 | Convey("Leader()", func() { 239 | isLeader, leaderAddress := store.Leader() 240 | 241 | So(isLeader, ShouldBeTrue) 242 | So(leaderAddress, ShouldEqual, raftAddress) 243 | }) 244 | 245 | Convey("IsInitializedCluster()", func() { 246 | ok := store.IsInitializedCluster() 247 | So(ok, ShouldBeTrue) 248 | }) 249 | }) 250 | } 251 | 252 | func TestStore_MultipleNode(t *testing.T) { 253 | // mock leader enforcer 254 | leaderCtl := gomock.NewController(t) 255 | defer leaderCtl.Finish() 256 | leaderEnforcer := mocks.NewMockIDistributedEnforcer(leaderCtl) 257 | 258 | // mock follower 259 | followerCtl := gomock.NewController(t) 260 | defer followerCtl.Finish() 261 | followerEnforcer := mocks.NewMockIDistributedEnforcer(followerCtl) 262 | 263 | localIP := GetLocalIP() 264 | leaderAddress := localIP + ":6790" 265 | followerAddress := localIP + ":6780" 266 | 267 | leaderID := "node-leader" 268 | followerID := "node-follower" 269 | 270 | leaderStore, err := newStore(leaderEnforcer, leaderID, leaderAddress, true) 271 | assert.NoError(t, err) 272 | defer leaderStore.Stop() 273 | 274 | err = leaderStore.WaitLeader() 275 | assert.NoError(t, err) 276 | 277 | followerStore, err := newStore(followerEnforcer, followerID, followerAddress, false) 278 | assert.NoError(t, err) 279 | defer followerStore.Stop() 280 | 281 | err = leaderStore.JoinNode(followerStore.ID(), followerStore.Address()) 282 | assert.NoError(t, err) 283 | 284 | err = followerStore.WaitLeader() 285 | assert.NoError(t, err) 286 | 287 | Convey("TestStore_MultipleNode", t, func() { 288 | Convey("AddPolicy()", func() { 289 | sec := "p" 290 | pType := "p" 291 | originalRules := [][]string{{"role:admin", "/", "*"}} 292 | var rules []*command.StringArray 293 | for _, rule := range originalRules { 294 | rules = append(rules, &command.StringArray{Items: rule}) 295 | } 296 | 297 | request := &command.AddPoliciesRequest{ 298 | Sec: sec, 299 | PType: pType, 300 | Rules: rules, 301 | } 302 | 303 | leaderEnforcer.EXPECT().AddPoliciesSelf(nil, sec, pType, originalRules).Return(originalRules, nil) 304 | followerEnforcer.EXPECT().AddPoliciesSelf(nil, sec, pType, originalRules).Return(originalRules, nil) 305 | err := leaderStore.AddPolicies(request) 306 | So(err, ShouldBeNil) 307 | 308 | // Waiting for synchronization data to follow node. 309 | <-time.After(2 * time.Second) 310 | }) 311 | 312 | Convey("RemovePolicy()", func() { 313 | sec := "p" 314 | pType := "p" 315 | originalRules := [][]string{{"role:admin", "/", "*"}} 316 | var rules []*command.StringArray 317 | for _, rule := range originalRules { 318 | rules = append(rules, &command.StringArray{Items: rule}) 319 | } 320 | 321 | request := &command.RemovePoliciesRequest{ 322 | Sec: sec, 323 | PType: pType, 324 | Rules: rules, 325 | } 326 | 327 | leaderEnforcer.EXPECT().RemovePoliciesSelf(nil, sec, pType, originalRules).Return(originalRules, nil) 328 | followerEnforcer.EXPECT().RemovePoliciesSelf(nil, sec, pType, originalRules).Return(originalRules, nil) 329 | err := leaderStore.RemovePolicies(request) 330 | So(err, ShouldBeNil) 331 | 332 | // Waiting for synchronization data to follow node. 333 | <-time.After(2 * time.Second) 334 | }) 335 | 336 | Convey("RemoveFilteredPolicy()", func() { 337 | sec := "p" 338 | pType := "p" 339 | fieldIndex := 0 340 | fieldValues := []string{"role:admin"} 341 | effected := [][]string{{"role:admin", "/", "*"}} 342 | 343 | request := &command.RemoveFilteredPolicyRequest{ 344 | Sec: sec, 345 | PType: pType, 346 | FieldIndex: int32(fieldIndex), 347 | FieldValues: fieldValues, 348 | } 349 | 350 | leaderEnforcer.EXPECT().RemoveFilteredPolicySelf(nil, sec, pType, fieldIndex, fieldValues).Return(effected, nil) 351 | followerEnforcer.EXPECT().RemoveFilteredPolicySelf(nil, sec, pType, fieldIndex, fieldValues).Return(effected, nil) 352 | err := leaderStore.RemoveFilteredPolicy(request) 353 | So(err, ShouldBeNil) 354 | 355 | // Waiting for synchronization data to follow node. 356 | <-time.After(2 * time.Second) 357 | }) 358 | 359 | Convey("UpdatePolicy()", func() { 360 | sec := "p" 361 | pType := "p" 362 | oldRule := []string{"role:admin", "/", "*"} 363 | newRule := []string{"role:admin", "/admin", "*"} 364 | 365 | request := &command.UpdatePolicyRequest{ 366 | Sec: sec, 367 | PType: pType, 368 | OldRule: oldRule, 369 | NewRule: newRule, 370 | } 371 | 372 | leaderEnforcer.EXPECT().UpdatePolicySelf(nil, sec, pType, oldRule, newRule).Return(true, nil) 373 | followerEnforcer.EXPECT().UpdatePolicySelf(nil, sec, pType, oldRule, newRule).Return(true, nil) 374 | err := leaderStore.UpdatePolicy(request) 375 | So(err, ShouldBeNil) 376 | 377 | // Waiting for synchronization data to follow node. 378 | <-time.After(2 * time.Second) 379 | }) 380 | 381 | Convey("ClearPolicy()", func() { 382 | leaderEnforcer.EXPECT().ClearPolicySelf(nil).Return(nil) 383 | followerEnforcer.EXPECT().ClearPolicySelf(nil).Return(nil) 384 | err := leaderStore.ClearPolicy() 385 | So(err, ShouldBeNil) 386 | 387 | // Waiting for synchronization data to follow node. 388 | <-time.After(2 * time.Second) 389 | }) 390 | 391 | Convey("ID()", func() { 392 | So(leaderStore.ID(), ShouldEqual, leaderID) 393 | So(followerStore.ID(), ShouldEqual, followerID) 394 | }) 395 | 396 | Convey("Address()", func() { 397 | So(leaderStore.Address(), ShouldEqual, leaderAddress) 398 | So(followerStore.Address(), ShouldEqual, followerAddress) 399 | }) 400 | 401 | Convey("Leader()", func() { 402 | isLeader, address := leaderStore.Leader() 403 | So(isLeader, ShouldBeTrue) 404 | So(address, ShouldEqual, leaderAddress) 405 | 406 | isLeader, address = followerStore.Leader() 407 | So(isLeader, ShouldBeFalse) 408 | So(address, ShouldEqual, leaderAddress) 409 | }) 410 | 411 | Convey("RemoveNode()", func() { 412 | err := leaderStore.RemoveNode(followerAddress) 413 | So(err, ShouldBeNil) 414 | }) 415 | 416 | Convey("IsInitializedCluster()", func() { 417 | ok := leaderStore.IsInitializedCluster() 418 | So(ok, ShouldBeTrue) 419 | 420 | ok = followerStore.IsInitializedCluster() 421 | So(ok, ShouldBeTrue) 422 | }) 423 | }) 424 | } 425 | 426 | func newStore(enforcer casbin.IDistributedEnforcer, id string, address string, enableBootstrap bool) (*Store, error) { 427 | dir, err := ioutil.TempDir("", "casbin-hraft-") 428 | if err != nil { 429 | return nil, err 430 | } 431 | 432 | tlsConfig, err := GetTLSConfig() 433 | if err != nil { 434 | return nil, err 435 | } 436 | 437 | ln, err := tls.Listen("tcp", address, tlsConfig) 438 | if err != nil { 439 | return nil, err 440 | } 441 | 442 | streamLayer, err := NewTCPStreamLayer(ln, tlsConfig) 443 | if err != nil { 444 | return nil, err 445 | } 446 | 447 | store, err := NewStore(zap.NewExample(), &Config{ 448 | ID: id, 449 | Dir: dir, 450 | NetworkTransportConfig: &raft.NetworkTransportConfig{ 451 | Logger: nil, 452 | Stream: streamLayer, 453 | MaxPool: 5, 454 | Timeout: 10 * time.Second, 455 | }, 456 | Enforcer: enforcer, 457 | }) 458 | if err != nil { 459 | return nil, err 460 | } 461 | 462 | err = store.Start(enableBootstrap) 463 | if err != nil { 464 | return nil, err 465 | } 466 | 467 | return store, nil 468 | } 469 | -------------------------------------------------------------------------------- /store/stream_layer.go: -------------------------------------------------------------------------------- 1 | package store 2 | 3 | import ( 4 | "crypto/tls" 5 | "net" 6 | "time" 7 | 8 | "github.com/hashicorp/raft" 9 | ) 10 | 11 | // StreamLayer implements the raft.StreamLayer interface base on TCP. 12 | type TCPStreamLayer struct { 13 | ln net.Listener 14 | tlsConfig *tls.Config 15 | } 16 | 17 | // NewStreamLayer returns a StreamLayer. 18 | func NewTCPStreamLayer(ln net.Listener, tlsConfig *tls.Config) (*TCPStreamLayer, error) { 19 | layer := &TCPStreamLayer{ 20 | ln: ln, 21 | tlsConfig: tlsConfig, 22 | } 23 | return layer, nil 24 | } 25 | 26 | // Dial implements the StreamLayer interface. 27 | func (t *TCPStreamLayer) Dial(address raft.ServerAddress, timeout time.Duration) (net.Conn, error) { 28 | if t.tlsConfig == nil { 29 | return net.DialTimeout("tcp", string(address), timeout) 30 | } 31 | 32 | dialer := &net.Dialer{ 33 | Timeout: timeout, 34 | } 35 | return tls.DialWithDialer(dialer, "tcp", string(address), t.tlsConfig) 36 | } 37 | 38 | // Accept implements the net.Listener interface. 39 | func (t *TCPStreamLayer) Accept() (c net.Conn, err error) { 40 | return t.ln.Accept() 41 | } 42 | 43 | // Close implements the net.Listener interface. 44 | func (t *TCPStreamLayer) Close() (err error) { 45 | return t.ln.Close() 46 | } 47 | 48 | // Addr implements the net.Listener interface. 49 | func (t *TCPStreamLayer) Addr() net.Addr { 50 | return t.ln.Addr() 51 | } 52 | -------------------------------------------------------------------------------- /store/stream_layer_test.go: -------------------------------------------------------------------------------- 1 | package store 2 | 3 | import ( 4 | "crypto/tls" 5 | "net/http/httptest" 6 | "testing" 7 | 8 | "github.com/stretchr/testify/assert" 9 | ) 10 | 11 | func TestNewTCPStreamLayer(t *testing.T) { 12 | ts := httptest.NewUnstartedServer(nil) 13 | ts.EnableHTTP2 = true 14 | ts.StartTLS() 15 | defer ts.Close() 16 | 17 | ln, err := tls.Listen("tcp", "0.0.0.0:0", ts.TLS) 18 | assert.NoError(t, err) 19 | 20 | layer, err := NewTCPStreamLayer(ln, ts.TLS) 21 | assert.NoError(t, err) 22 | defer layer.Close() 23 | } 24 | -------------------------------------------------------------------------------- /testdata/ca/ca-config.json: -------------------------------------------------------------------------------- 1 | { 2 | "signing": { 3 | "default": { 4 | "expiry": "876000h" 5 | }, 6 | "profiles": { 7 | "peer": { 8 | "expiry": "876000h", 9 | "usages": [ 10 | "signing", 11 | "key encipherment", 12 | "server auth", 13 | "client auth" 14 | ] 15 | } 16 | } 17 | } 18 | } 19 | 20 | -------------------------------------------------------------------------------- /testdata/ca/ca-csr.json: -------------------------------------------------------------------------------- 1 | { 2 | "CN": "hraftdispatcher", 3 | "hosts": [""], 4 | "key": { 5 | "algo": "rsa", 6 | "size": 2048 7 | }, 8 | "names": [ 9 | { 10 | "C": "US", 11 | "ST": "CA", 12 | "L": "San Francisco" 13 | } 14 | ] 15 | } 16 | -------------------------------------------------------------------------------- /testdata/ca/ca-key.pem: -------------------------------------------------------------------------------- 1 | -----BEGIN RSA PRIVATE KEY----- 2 | MIIEowIBAAKCAQEAvSkrgIC0GOzUgzexq8Amzmkgl1ZBGLX1ITVTrKnL9QxAlGuF 3 | 1Kb0nVG5g1B8QuXRGRZo0OyLQvobSRssyzmSQWPvJKk94JMTpSEY4JZfBZD96ok2 4 | B4lYzAuIStf58hP9ITfG67h7nEmQXWBUAakanplSGPYiVz7FF7EiFu/qSBuSH5wb 5 | UzSrEZaxvxBDuiHPbIcywSQb+WwfF6EwTQrtjWMSi0u4AQL9QdLNioDrjdXZrusV 6 | KYz6xAzMEnofYQ2yCWlUJ8vKg/mJNuGc6cfCpRKwyT2R952jSaMuMWYJbzVJSNjd 7 | j+HwfaQ8+abLxpRP0fXbQQagit7sVnKJkG7mEQIDAQABAoIBAEqchJYpLeBu+dvQ 8 | CfqOzj+4Y6yNzxtqGghBOoHub452iLqCNsWMHQw4gg3zny/b6kAVAL73rm56rGyH 9 | lYGLcc7tFhTF1KxSFvuZNHDRnyVY7W6nNHKAkE/bdKTgz6vWio+hqoqrt8YLli/e 10 | oQQOqXtJSoKiAyWakq8npSSa+TP7Tep0bd+DOsikunPdLSVNsiIytMA8FoUAXIO1 11 | 2Bys/cpr7Y9xPq79AKxT9LdhJc3adOCOhxXIF7WIlInDyRb7c55PpKjJGrxdipM+ 12 | c6GOPYA4czKhkCJT0wWHV+SCIS70BdTeibWfIkKGu9MeJYrajPoBjvoaii/lwZFx 13 | MYSFIqkCgYEA7Yxgnmy65+c8aJLT0uYUZ6VEj+9fKDGryypHF+UyeCp7MKMIpTww 14 | y2cUbEyMqXkG6c8+agO4fWQdEq71xOrq5AFpSvBTm1c4ZWZDRUsBc9vWfxW1qAvW 15 | 9R1wbII0ELBP6a+Xyy+1qTu5Z2we83wi3EnZ/fMykzIbfZKkmgSCtocCgYEAy9qc 16 | osp1TY4knnQ8ltUYAeoIRsTY3sZG4PMDWCNX7dYBLPXrqmfaI/oIJiVQ69+GrOXM 17 | 9PGq67iMVYVDowZw+bUElPe25YoKajgyXjIoGifKzqhfTX1f6RV06pEMo6psgYS7 18 | HTpkum+kD9NFFxZpDhOuNJ1gVtg4I2ilo7/ojKcCgYA0Z7Wg/ae2JWNERrAIZM23 19 | l9QqVMqPWXXm2irY8K1TqTuIHU949HBnTxRIWBLoCFUoG5pPVQbMUtu/FW4OxSyQ 20 | pGcQKzimmnVMmWTZ0pQ9dRA69RUgP4xqgtYcehT2YY8fpKc+IgZXkV9zm3ywhoBC 21 | TOCUj6RmqLU73LAHcltL3QKBgQCwxuMOSINwvdMDlcj1A9XN0yVaOVnX1hmg+Rvt 22 | Urjthqd+VcQIAgcpK6nlanbCFRS02pLh1rBhuaf4gZglx3AxGsjeZfI1G3/snM8Q 23 | 4G3mQJuemmQKzq5dcj3xK9fKFgACURfyQLEY+7dKAevoGYcArhBjkBX7OYV1L0p0 24 | tfb0AQKBgGjoFT0fasvz7seITvDIrlqa99KDMsBClbYnAZCgKgO28BLokx0AorbJ 25 | byUrPEEPa4OJKaIbfOwRuSx5OBQCqOKB1E2SHkRA32N30r6AvQxGFwS2c35F+wrs 26 | xAsmk7id0TpShNSF8ff6PIUPoSq9UqOcSh+hjIThKuZ60cGVVf17 27 | -----END RSA PRIVATE KEY----- 28 | -------------------------------------------------------------------------------- /testdata/ca/ca.csr: -------------------------------------------------------------------------------- 1 | -----BEGIN CERTIFICATE REQUEST----- 2 | MIICrzCCAZcCAQAwTDELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAkNBMRYwFAYDVQQH 3 | Ew1TYW4gRnJhbmNpc2NvMRgwFgYDVQQDEw9ocmFmdGRpc3BhdGNoZXIwggEiMA0G 4 | CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC9KSuAgLQY7NSDN7GrwCbOaSCXVkEY 5 | tfUhNVOsqcv1DECUa4XUpvSdUbmDUHxC5dEZFmjQ7ItC+htJGyzLOZJBY+8kqT3g 6 | kxOlIRjgll8FkP3qiTYHiVjMC4hK1/nyE/0hN8bruHucSZBdYFQBqRqemVIY9iJX 7 | PsUXsSIW7+pIG5IfnBtTNKsRlrG/EEO6Ic9shzLBJBv5bB8XoTBNCu2NYxKLS7gB 8 | Av1B0s2KgOuN1dmu6xUpjPrEDMwSeh9hDbIJaVQny8qD+Yk24Zzpx8KlErDJPZH3 9 | naNJoy4xZglvNUlI2N2P4fB9pDz5psvGlE/R9dtBBqCK3uxWcomQbuYRAgMBAAGg 10 | HjAcBgkqhkiG9w0BCQ4xDzANMAsGA1UdEQQEMAKCADANBgkqhkiG9w0BAQsFAAOC 11 | AQEAEMuO0bGksokp3H9yNs1f/LEm9HYrnIBaxBhelwDJd50gIiLOC7uXFPwI/A29 12 | Bz2HwkuaekGChAlwo4Xt/eaHwdJI3BgsCdrUyZ/TgWHh62phuz9crVqeeEP1FK7n 13 | X+0nofSPRwGNv40YgvXG51LJ2jZKDjUZf2MvexLViLXTxXxlK31hyndaoU3bZsg9 14 | LcDXHRSd9kdmblRiX1xeJWBKtKDlDleAeibN1Q6Mke4Fzvyx0gkzdaVxDX98mo1z 15 | TtlTHXPGMmKLxPo31Qd487R4nn9joOGeqUhSZKuFX4cTXhu9LfgI5/CoaO7Du69+ 16 | Qc2EK5cLj7d5JzZ0slinntQwKQ== 17 | -----END CERTIFICATE REQUEST----- 18 | -------------------------------------------------------------------------------- /testdata/ca/ca.pem: -------------------------------------------------------------------------------- 1 | -----BEGIN CERTIFICATE----- 2 | MIIDaDCCAlCgAwIBAgIUcpVt/SzT9IPq/HWZOTP8WXB//5owDQYJKoZIhvcNAQEL 3 | BQAwTDELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAkNBMRYwFAYDVQQHEw1TYW4gRnJh 4 | bmNpc2NvMRgwFgYDVQQDEw9ocmFmdGRpc3BhdGNoZXIwHhcNMjEwMjE1MTUwNDAw 5 | WhcNMjYwMjE0MTUwNDAwWjBMMQswCQYDVQQGEwJVUzELMAkGA1UECBMCQ0ExFjAU 6 | BgNVBAcTDVNhbiBGcmFuY2lzY28xGDAWBgNVBAMTD2hyYWZ0ZGlzcGF0Y2hlcjCC 7 | ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL0pK4CAtBjs1IM3savAJs5p 8 | IJdWQRi19SE1U6ypy/UMQJRrhdSm9J1RuYNQfELl0RkWaNDsi0L6G0kbLMs5kkFj 9 | 7ySpPeCTE6UhGOCWXwWQ/eqJNgeJWMwLiErX+fIT/SE3xuu4e5xJkF1gVAGpGp6Z 10 | Uhj2Ilc+xRexIhbv6kgbkh+cG1M0qxGWsb8QQ7ohz2yHMsEkG/lsHxehME0K7Y1j 11 | EotLuAEC/UHSzYqA643V2a7rFSmM+sQMzBJ6H2ENsglpVCfLyoP5iTbhnOnHwqUS 12 | sMk9kfedo0mjLjFmCW81SUjY3Y/h8H2kPPmmy8aUT9H120EGoIre7FZyiZBu5hEC 13 | AwEAAaNCMEAwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0O 14 | BBYEFMvdedWPdZrkG0ZNAV8fIX+F6Dg2MA0GCSqGSIb3DQEBCwUAA4IBAQCmkvnH 15 | XKURz70aU5DXeuR4/8kvEk/Wv25154rZfaa7g7uEfSITvgj+7e65HJW53BT7Xrpj 16 | xfALJriaONcf4xu+ov1rdMdojpYnRwNC4B5tiNUK+IEkr/9iyOk7Tu3fY1hwbzLz 17 | DjfUP+9aOdzBW4xapVpM+YlvIR525rXl+P062SEm9pUyWvDFAb3tKHX9W6Hjzm5w 18 | 50qShl7ncwwtTYUylVUZcS1xOh0vF4WPtGWbuN0jfQjLFDeyG2l78jcl/ngsIiyT 19 | KTExqCSHJ/6FT9zQnYhXEKU4BLIzqeS8Kfnd7h+3URYZNs5GlfujhLflaR6/Xxp1 20 | /P7UwWnW1xZf+iAP 21 | -----END CERTIFICATE----- 22 | -------------------------------------------------------------------------------- /testdata/ca/generate.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | cfssl gencert -initca ca-csr.json | cfssljson -bare ca 4 | cfssl gencert -ca=./ca.pem -ca-key=./ca-key.pem -config=./ca-config.json -profile=peer peer-csr.json | cfssljson -bare peer 5 | -------------------------------------------------------------------------------- /testdata/ca/peer-csr.json: -------------------------------------------------------------------------------- 1 | { 2 | "CN": "hraftdispatcher", 3 | "hosts": ["127.0.0.1"], 4 | "key": { 5 | "algo": "rsa", 6 | "size": 2048 7 | }, 8 | "names": [ 9 | { 10 | "C": "US", 11 | "ST": "CA", 12 | "L": "San Francisco" 13 | } 14 | ] 15 | } 16 | -------------------------------------------------------------------------------- /testdata/ca/peer-key.pem: -------------------------------------------------------------------------------- 1 | -----BEGIN RSA PRIVATE KEY----- 2 | MIIEowIBAAKCAQEAxHaJUQTh3eM8ztbC9Vnq9NtqaPJ7+GcymOzPjDBVLGuBzbTY 3 | 2Hd7D6gV0Hih9QW9yneN/jI9u4e3CHfln3aQ0t7rVvu3CAEx+34M49nWIZNebKU+ 4 | j3YSmCkxl/I2fD3x/vSvTy41Ey4DHVjbw35cDE/vg6eU+nnkX96tbrVZgUnZfrSa 5 | I8xmdO64DLqOCNUcl0LM8zXAgH+bIkKM6y4U6AAEsKsZ5CNpMw0nopCnuQZu0NFF 6 | NM9qgUdbJLSHOqGbFLiIGX+PNis9dVggZX3UdzbyFHtAywM53S5Hq0Q2S2dYQHLh 7 | SFkWgwCZEDP+Oi3j46vXVHzNZgJng+wHNiCBzwIDAQABAoIBAEOn8oflH4dTHvi3 8 | +rGVgpVKDm4Pu2OC3mjNfHfxmRNP/oaBlf+NveJZZxHAyT1g+cgEvfBhCuNOzFht 9 | ObVdlmgX/oGY86IdD0JlWTkKJnSvlF/j1BSBe8vMu9hwwBSvHGxJhSnGZt6xBL+R 10 | fzTmifpveLMk/ef4HA5r19v9NdKQqhxF3SfY2SPCcdwrtsbTxwKvKXaIKIzQ0j5o 11 | HGQPwiCLWYXWIBfwqpNB0FWpnBhSIUi3U/fYrfdRWsBCaCKaJFLrknTYk2jPTYHd 12 | kXUNBwUyiRpD+c8MCVXhVrd6lg8MYK4qKs2k5VtfDtNM5uxegD2elEvbB9XDbdL9 13 | pXBP0okCgYEA8h51ocpteiyv/kZfHnRgHgIB9zGOBHXK2oIir843jezdTnLjLKqa 14 | GY8/6Rp6Djmbnw8q5wPtP1OgT2Zlvt93fjd1o2zZsA5LfBo07T4BQ0hIPQLSBkl0 15 | GfKKnPJBgKlhDSiopr3xDlPkx8BX9PP1Sf/G5ewYC/QpWW55NP4s02UCgYEAz7n9 16 | 9WRRFa1fXSEvcsXCOxHpKc+/XG6ZfptPnrmHVGv+MrHNE8GTWupoTr8l1gNsp1IH 17 | I3pG0THEMlNGApF4u1v2tZ5yNnsoVeP9Zrwy9i3E56weIOqBlpCxbjW8FPmTgVzt 18 | Xp/2IxBaZQn9YdCLSN2oX1Pqb7K3J84PzaXW/yMCgYBnVuDWQVQgxVoIqXiHwxwT 19 | MsAsBZacCLqgMNMlPlsv1F1Q0nBr7BUBu8aHc6mM0MG/TfX9zAtC8CqIOShMI40Y 20 | 7grjyd3P6woE2hqk98YKNZu/jqidzlQjjwXinvOeOq0VtLjnEkME3oHTUCE6h7W1 21 | 89ms4OwSjg/n/+Lz31i6kQKBgQCXM/M/o/3BoalAyN8Y1ApFpQvre2T3iyn/ll2m 22 | U7XGJbWqgPGd59Gy4915NHn+BhAY2wSHNoJF08vUNflH9UvEVXSHTwYj0hHSM1pI 23 | ZcVSnI4vdIGZxBj/1+LPLh3xxpkwGMxPjHBFpamm0la11G8OYwokGZkUJSpctwmZ 24 | z5VnsQKBgHdLPABKFGfxwsLFEr6vnwS3CmUcwbijDy7YGW//HOSk+sqsPuJclaSt 25 | BIBwORizBRWQd+C5aDa3lu9pkBjlZ+b57RnYq+bzQbNDHUt3c7BtcCujceD0uVXz 26 | /hug3iKji4mRJKDqV+5lYNG+Q6T3vZ5c7NbMo2LmeNXu/1aOxk4z 27 | -----END RSA PRIVATE KEY----- 28 | -------------------------------------------------------------------------------- /testdata/ca/peer.csr: -------------------------------------------------------------------------------- 1 | -----BEGIN CERTIFICATE REQUEST----- 2 | MIICszCCAZsCAQAwTDELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAkNBMRYwFAYDVQQH 3 | Ew1TYW4gRnJhbmNpc2NvMRgwFgYDVQQDEw9ocmFmdGRpc3BhdGNoZXIwggEiMA0G 4 | CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDEdolRBOHd4zzO1sL1Wer022po8nv4 5 | ZzKY7M+MMFUsa4HNtNjYd3sPqBXQeKH1Bb3Kd43+Mj27h7cId+WfdpDS3utW+7cI 6 | ATH7fgzj2dYhk15spT6PdhKYKTGX8jZ8PfH+9K9PLjUTLgMdWNvDflwMT++Dp5T6 7 | eeRf3q1utVmBSdl+tJojzGZ07rgMuo4I1RyXQszzNcCAf5siQozrLhToAASwqxnk 8 | I2kzDSeikKe5Bm7Q0UU0z2qBR1sktIc6oZsUuIgZf482Kz11WCBlfdR3NvIUe0DL 9 | AzndLkerRDZLZ1hAcuFIWRaDAJkQM/46LePjq9dUfM1mAmeD7Ac2IIHPAgMBAAGg 10 | IjAgBgkqhkiG9w0BCQ4xEzARMA8GA1UdEQQIMAaHBH8AAAEwDQYJKoZIhvcNAQEL 11 | BQADggEBAJcmgVK/3d9x++7TyPQQJtaWDNvKPgAnZCf6lRqNp0YiacyaCUxOugs8 12 | MB3uJcz0B3tMdaByEJ6mdiQFePhgoIGqN5LL/Ns+irLIOngfwkcAD5iSJFWBA5IE 13 | 4Yd/zhSNf52LP8m66xrty1JPDlPuDWmTv5RAJMEHx+Wo8pqcVdx7rTC4DHuDYwkF 14 | 4MxlA8n8ZaA9j0X89LtuGgo6cUeKQaTXggdvjfWzylOJvhLarzKRfxSqeQmFZgQM 15 | 0jAGcW6xA3qJjt4LRZdGPv79AKnKNIoBoZ9DiIrM+unezcE+CgGVdmYG1hEPaGpy 16 | w79TXmOUhCfaehwci9wIsR1cj4LDcWY= 17 | -----END CERTIFICATE REQUEST----- 18 | -------------------------------------------------------------------------------- /testdata/ca/peer.pem: -------------------------------------------------------------------------------- 1 | -----BEGIN CERTIFICATE----- 2 | MIIDlzCCAn+gAwIBAgIUUVw2A/52WC/efPDTngQd47EJWqgwDQYJKoZIhvcNAQEL 3 | BQAwTDELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAkNBMRYwFAYDVQQHEw1TYW4gRnJh 4 | bmNpc2NvMRgwFgYDVQQDEw9ocmFmdGRpc3BhdGNoZXIwIBcNMjEwMjE1MTUwNDAw 5 | WhgPMjEyMTAxMjIxNTA0MDBaMEwxCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJDQTEW 6 | MBQGA1UEBxMNU2FuIEZyYW5jaXNjbzEYMBYGA1UEAxMPaHJhZnRkaXNwYXRjaGVy 7 | MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxHaJUQTh3eM8ztbC9Vnq 8 | 9NtqaPJ7+GcymOzPjDBVLGuBzbTY2Hd7D6gV0Hih9QW9yneN/jI9u4e3CHfln3aQ 9 | 0t7rVvu3CAEx+34M49nWIZNebKU+j3YSmCkxl/I2fD3x/vSvTy41Ey4DHVjbw35c 10 | DE/vg6eU+nnkX96tbrVZgUnZfrSaI8xmdO64DLqOCNUcl0LM8zXAgH+bIkKM6y4U 11 | 6AAEsKsZ5CNpMw0nopCnuQZu0NFFNM9qgUdbJLSHOqGbFLiIGX+PNis9dVggZX3U 12 | dzbyFHtAywM53S5Hq0Q2S2dYQHLhSFkWgwCZEDP+Oi3j46vXVHzNZgJng+wHNiCB 13 | zwIDAQABo28wbTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEG 14 | CCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFHPb6d7J8ai2eYyX+0aF 15 | MzKPCHktMA8GA1UdEQQIMAaHBH8AAAEwDQYJKoZIhvcNAQELBQADggEBAHtgS5pR 16 | WL72X6p8+levu0lumia5fOAmbDylarcLn77qFbl0xK6ebDBQx6HXk1ZOzVB+DcJj 17 | z/BJ6gdoT9211C0VeT4VGE8rh28IRL9mhOXv1/sXuJo8qPH+VwR95phFD1+wcwBG 18 | f6GJde1qncnc79Sh+KfcYKpmbNxBLZA96LKRrN+kaytQDeB0EFTZBMc9XhdQsm/8 19 | 3QMkGh7Q1wJNzLmw8nUDmz8gtucQ00pwDe4PKSmrsFSJq62aMYhNeqItEGT3WV0e 20 | EzHG/uXVRwqgOepAA3p4neKaAH1G+oGWEzcI0EPPB3uCJzGLJTEU6QX0ISpyN8sT 21 | ihn/9JzaGivK+fk= 22 | -----END CERTIFICATE----- 23 | --------------------------------------------------------------------------------