├── .github ├── ISSUE_TEMPLATE │ ├── bug_report.md │ └── enhancement.md └── SECURITY.md ├── .gitignore ├── .prow.sh ├── CONTRIBUTING.md ├── Dockerfile ├── LICENSE ├── Makefile ├── OWNERS ├── README.md ├── RELEASE.md ├── SECURITY_CONTACTS ├── cloudbuild.yaml ├── cmd └── csi-adapter │ ├── cmd.go │ ├── driver.go │ └── main.go ├── code-of-conduct.md ├── docs └── deployment-guide.md ├── go.mod ├── go.sum ├── kustomization.yaml ├── pkg ├── client │ ├── fake │ │ ├── node_client.go │ │ └── provisioner_client.go │ ├── node_client.go │ ├── node_client_test.go │ └── provisioner_client.go ├── controller │ └── controller.go ├── identity │ └── identity.go ├── node │ ├── node.go │ ├── node_test.go │ ├── provisioner.go │ └── provisioner_test.go └── util │ ├── errors.go │ ├── events.go │ ├── test.go │ ├── test │ ├── file.go │ └── utils.go │ └── util.go ├── resources ├── daemonset.yaml ├── rbac.yaml └── sa.yaml └── sample ├── bucketaccessrequest.yaml ├── bucketrequest.yaml ├── classes ├── bucketaccessclass.yaml ├── bucketclass.yaml └── configmap.yaml └── pod ├── pod.yaml └── svc.yaml /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug Report 3 | about: Use this template for reporting bugs or issues. 4 | title: "[DATE] - Title" 5 | labels: bug 6 | --- 7 | # Bug Report 8 | 9 | 13 | 14 | **What happened**: 15 | 16 | **What you expected to happen**: 17 | 18 | **How to reproduce this bug (as minimally and precisely as possible)**: 19 | 20 | **Anything else relevant for this bug report?**: 21 | 22 | **Environment**: 23 | 24 | - Kubernetes version (use `kubectl version`), please list client and server: 25 | - CSI Adapter version (provide the release tag or commit hash): 26 | - Provisoner name and version (provide the release tag or commit hash): 27 | - Cloud provider or hardware configuration: 28 | - OS (e.g: `cat /etc/os-release`): 29 | - Kernel (e.g. `uname -a`): 30 | - Install tools: 31 | - Network plugin and version (if this is a network-related bug): 32 | - Others: 33 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/enhancement.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Enhancement/Feature Request 3 | about: Use this template to request a new feature or enhancement for the COSI CSI Adapter 4 | title: "[DATE] - Title" 5 | --- 6 | # Enhancement 7 | 8 | **Is your feature request related to a problem?/Why is this needed** 9 | 10 | 11 | **Describe the solution you'd like in detail** 12 | 13 | 14 | **Describe alternatives you've considered** 15 | 16 | 17 | **Additional context** 18 | 19 | -------------------------------------------------------------------------------- /.github/SECURITY.md: -------------------------------------------------------------------------------- 1 | # Security Policy 2 | 3 | ## Supported Versions 4 | 5 | Information about supported Kubernetes versions can be found on the 6 | [Kubernetes version and version skew support policy] page on the Kubernetes website. 7 | 8 | ## Reporting a Vulnerability 9 | 10 | Instructions for reporting a vulnerability can be found on the 11 | [Kubernetes Security and Disclosure Information] page. 12 | 13 | [Kubernetes version and version skew support policy]: https://kubernetes.io/docs/setup/release/version-skew-policy/#supported-versions 14 | [Kubernetes Security and Disclosure Information]: https://kubernetes.io/docs/reference/issues-security/security/#report-a-vulnerability 15 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | vendor 3 | bin/csi-adapter 4 | release-tools 5 | .DS_Store 6 | *.tmp 7 | .build 8 | *.swp 9 | travis.yml -------------------------------------------------------------------------------- /.prow.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | . release-tools/prow.sh 4 | 5 | main -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Welcome to Kubernetes. We are excited about the prospect of you joining our [community](https://git.k8s.io/community)! The Kubernetes community abides by the CNCF [code of conduct](code-of-conduct.md). Here is an excerpt: 4 | 5 | _As contributors and maintainers of this project, and in the interest of fostering an open and welcoming community, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities._ 6 | 7 | ## Getting Started 8 | 9 | We have full documentation on how to get started contributing here: 10 | 11 | 14 | 15 | - [Contributor License Agreement](https://git.k8s.io/community/CLA.md) Kubernetes projects require that you sign a Contributor License Agreement (CLA) before we can accept your pull requests 16 | - [Kubernetes Contributor Guide](https://git.k8s.io/community/contributors/guide) - Main contributor documentation, or you can just jump directly to the [contributing section](https://git.k8s.io/community/contributors/guide#contributing) 17 | - [Contributor Cheat Sheet](https://git.k8s.io/community/contributors/guide/contributor-cheatsheet) - Common resources for existing developers 18 | 19 | ## Mentorship 20 | 21 | - [Mentoring Initiatives](https://git.k8s.io/community/mentoring) - We have a diverse set of mentorship programs available that are always looking for volunteers! 22 | 23 | ## Contact Information 24 | 25 | - [Slack](https://kubernetes.slack.com/messages/sig-storage) 26 | - [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-sig-storage) 27 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | # Copyright 2021 The Kubernetes Authors. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | FROM alpine 16 | LABEL maintainers="Kubernetes Authors" 17 | LABEL description="COSI CSI Adapter" 18 | 19 | COPY ./bin/csi-adapter csi-adapter 20 | ENTRYPOINT ["/csi-adapter"] 21 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright {yyyy} {name of copyright owner} 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # Copyright 2020 The Kubernetes Authors. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | all: reltools build 16 | .PHONY: reltools 17 | reltools: release-tools/build.make 18 | release-tools/build.make: 19 | $(eval CURDIR := $(shell pwd)) 20 | $(eval TMP := $(shell mktemp -d)) 21 | $(shell cd ${TMP} && git clone https://github.com/kubernetes-sigs/container-object-storage-interface-spec) 22 | $(shell cp -r ${TMP}/container-object-storage-interface-spec/release-tools ${CURDIR}/) 23 | $(shell rm -rf ${TMP}) 24 | ln -s release-tools/travis.yml travis.yml 25 | 26 | CMDS=csi-adapter 27 | 28 | include release-tools/build.make 29 | 30 | IMAGE_NAME=quay.io/containerobjectstorage/objectstorage-csi-adapter 31 | IMAGE_TAGS=canary -------------------------------------------------------------------------------- /OWNERS: -------------------------------------------------------------------------------- 1 | # See the OWNERS docs at https://go.k8s.io/owners 2 | 3 | approvers: 4 | - jsafrane 5 | - msau42 6 | - saad-ali 7 | - xing-yang 8 | - wlan0 9 | - brahmaroutu 10 | - rrati 11 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Container Object Storage Interface CSI Adapter 2 | 3 | CSI driver to bootstrap COSI workloads 4 | 5 | ## Documentation 6 | 7 | COSI documentation can be found on project [site](https://container-object-storage-interface.github.io/). 8 | ## Community, discussion, contribution, and support 9 | 10 | You can reach the maintainers of this project at: 11 | 12 | - [#sig-storage-cosi](https://kubernetes.slack.com/messages/sig-storage-cosi) slack channel 13 | - [container-object-storage-interface](https://groups.google.com/g/container-object-storage-interface-wg?pli=1) mailing list 14 | ### Code of conduct 15 | 16 | Participation in the Kubernetes community is governed by the [Kubernetes Code of Conduct](code-of-conduct.md). 17 | -------------------------------------------------------------------------------- /RELEASE.md: -------------------------------------------------------------------------------- 1 | # Release Process 2 | 3 | The Kubernetes Template Project is released on an as-needed basis. The process is as follows: 4 | 5 | 1. An issue is proposing a new release with a changelog since the last release 6 | 1. All [OWNERS](OWNERS) must LGTM this release 7 | 1. An OWNER runs `git tag -s $VERSION` and inserts the changelog and pushes the tag with `git push $VERSION` 8 | 1. The release issue is closed 9 | 1. An announcement email is sent to `kubernetes-dev@googlegroups.com` with the subject `[ANNOUNCE] kubernetes-template-project $VERSION is released` 10 | -------------------------------------------------------------------------------- /SECURITY_CONTACTS: -------------------------------------------------------------------------------- 1 | # Defined below are the security contacts for this repo. 2 | # 3 | # They are the contact point for the Product Security Committee to reach out 4 | # to for triaging and handling of incoming issues. 5 | # 6 | # The below names agree to abide by the 7 | # [Embargo Policy](https://git.k8s.io/security/private-distributors-list.md#embargo-policy) 8 | # and will be removed and replaced if they violate that agreement. 9 | # 10 | # DO NOT REPORT SECURITY VULNERABILITIES DIRECTLY TO THESE NAMES, FOLLOW THE 11 | # INSTRUCTIONS AT https://kubernetes.io/security/ 12 | 13 | jsafrane 14 | msau42 15 | saad-ali 16 | xing-yang 17 | -------------------------------------------------------------------------------- /cloudbuild.yaml: -------------------------------------------------------------------------------- 1 | # See https://cloud.google.com/cloud-build/docs/build-config 2 | timeout: 3000s 3 | options: 4 | substitution_option: ALLOW_LOOSE 5 | steps: 6 | - name: "gcr.io/k8s-staging-test-infra/gcb-docker-gcloud:v20221214-1b4dd4d69a" 7 | entrypoint: make 8 | args: 9 | - build 10 | -------------------------------------------------------------------------------- /cmd/csi-adapter/cmd.go: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright 2020 The Kubernetes Authors. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"); 5 | you may not use this file except in compliance with the License. 6 | You may obtain a copy of the License at 7 | 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | 10 | Unless required by applicable law or agreed to in writing, software 11 | distributed under the License is distributed on an "AS IS" BASIS, 12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | See the License for the specific language governing permissions and 14 | limitations under the License. 15 | */ 16 | 17 | package main 18 | 19 | import ( 20 | "flag" 21 | "os" 22 | 23 | "github.com/spf13/cobra" 24 | "github.com/spf13/viper" 25 | _ "k8s.io/klog/v2" 26 | ) 27 | 28 | var Version string 29 | 30 | // flags 31 | var ( 32 | identity string 33 | nodeID string 34 | protocol string 35 | listen string 36 | dataRoot string 37 | volumeLimit int64 38 | ) 39 | 40 | var driverCmd = &cobra.Command{ 41 | Use: os.Args[0], 42 | Short: "Ephemeral CSI driver for use in the COSI", 43 | Long: "This Container Storage Interface (CSI) driver provides the ability to reference Bucket and BucketAccess objects, extracting connection/credential information and writing it to the Pod's filesystem. This driver does not manage the lifecycle of the bucket or the backing of the objects themselves, it only acts as the middle-man.", 44 | SilenceUsage: true, 45 | RunE: func(c *cobra.Command, args []string) error { 46 | return driver(args) 47 | }, 48 | } 49 | 50 | func init() { 51 | Version = "v0.0.1" 52 | 53 | viper.AutomaticEnv() 54 | // parse the go default flagset to get flags for klog and other packages in future 55 | driverCmd.PersistentFlags().AddGoFlagSet(flag.CommandLine) 56 | // defaulting this to true so that logs are printed to console 57 | _ = flag.Set("logtostderr", "true") 58 | 59 | driverCmd.PersistentFlags().StringVarP(&identity, "identity", "i", identity, "identity of this COSI CSI driver") 60 | driverCmd.PersistentFlags().StringVarP(&nodeID, "node-id", "n", nodeID, "identity of the node in which COSI CSI driver is running") 61 | driverCmd.PersistentFlags().StringVarP(&listen, "listen", "l", listen, "address of the listening socket for the node server") 62 | driverCmd.PersistentFlags().StringVarP(&protocol, "protocol", "p", protocol, "must be one of tcp, tcp4, tcp6, unix, unixpacket") 63 | driverCmd.PersistentFlags().StringVarP(&dataRoot, "data-path", "d", protocol, "the path to the directory for storing secrets") 64 | driverCmd.PersistentFlags().Int64VarP(&volumeLimit, "max-volumes", "m", volumeLimit, "the maximum amount of volumes which can be assigned to a node") 65 | 66 | _ = driverCmd.PersistentFlags().MarkHidden("alsologtostderr") 67 | _ = driverCmd.PersistentFlags().MarkHidden("log_backtrace_at") 68 | _ = driverCmd.PersistentFlags().MarkHidden("log_dir") 69 | _ = driverCmd.PersistentFlags().MarkHidden("logtostderr") 70 | _ = driverCmd.PersistentFlags().MarkHidden("master") 71 | _ = driverCmd.PersistentFlags().MarkHidden("stderrthreshold") 72 | _ = driverCmd.PersistentFlags().MarkHidden("vmodule") 73 | 74 | // suppress the incorrect prefix in klog output 75 | _ = flag.CommandLine.Parse([]string{}) 76 | _ = viper.BindPFlags(driverCmd.PersistentFlags()) 77 | } 78 | 79 | func Execute() error { 80 | return driverCmd.Execute() 81 | } 82 | -------------------------------------------------------------------------------- /cmd/csi-adapter/driver.go: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright 2020 The Kubernetes Authors. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"); 5 | you may not use this file except in compliance with the License. 6 | You may obtain a copy of the License at 7 | 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | 10 | Unless required by applicable law or agreed to in writing, software 11 | distributed under the License is distributed on an "AS IS" BASIS, 12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | See the License for the specific language governing permissions and 14 | limitations under the License. 15 | */ 16 | 17 | package main 18 | 19 | import ( 20 | "os" 21 | 22 | csicommon "github.com/kubernetes-csi/drivers/pkg/csi-common" 23 | "k8s.io/klog/v2" 24 | 25 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/controller" 26 | id "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/identity" 27 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/node" 28 | ) 29 | 30 | func driver(args []string) error { 31 | if protocol == "unix" { 32 | if err := os.RemoveAll(listen); err != nil { 33 | klog.Fatalf("could not prepare socket: %v", err) 34 | } 35 | } 36 | 37 | idServer, err := id.NewIdentityServer(identity, Version, map[string]string{}) 38 | if err != nil { 39 | return err 40 | } 41 | klog.InfoS("identity server prepared") 42 | 43 | nodeServer := node.NewNodeServerOrDie(identity, nodeID, dataRoot, volumeLimit) 44 | controllerServer, err := controller.NewControllerServer() 45 | 46 | s := csicommon.NewNonBlockingGRPCServer() 47 | s.Start(listen, idServer, controllerServer, nodeServer) 48 | s.Wait() 49 | 50 | return nil 51 | } 52 | -------------------------------------------------------------------------------- /cmd/csi-adapter/main.go: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright 2020 The Kubernetes Authors. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"); 5 | you may not use this file except in compliance with the License. 6 | You may obtain a copy of the License at 7 | 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | 10 | Unless required by applicable law or agreed to in writing, software 11 | distributed under the License is distributed on an "AS IS" BASIS, 12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | See the License for the specific language governing permissions and 14 | limitations under the License. 15 | */ 16 | 17 | package main 18 | 19 | import ( 20 | "os" 21 | "os/signal" 22 | "syscall" 23 | 24 | "k8s.io/klog/v2" 25 | ) 26 | 27 | func main() { 28 | sigs := make(chan os.Signal, 1) 29 | signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM) 30 | 31 | go func() { 32 | s := <-sigs 33 | klog.InfoS("Exiting on signal", "signal", s.String(), "value", s) 34 | }() 35 | 36 | if err := Execute(); err != nil { 37 | os.Exit(1) 38 | } 39 | } 40 | -------------------------------------------------------------------------------- /code-of-conduct.md: -------------------------------------------------------------------------------- 1 | # Kubernetes Community Code of Conduct 2 | 3 | Please refer to our [Kubernetes Community Code of Conduct](https://git.k8s.io/community/code-of-conduct.md) 4 | -------------------------------------------------------------------------------- /docs/deployment-guide.md: -------------------------------------------------------------------------------- 1 | # Deploying Container Object Storage Interface CSI Adapter On Kubernetes 2 | 3 | This document describes steps for Kubernetes administrators to setup Container Object Storage Interface (COSI) CSI Adapter onto a Kubernetes cluster. 4 | 5 | COSI CSI Adapter can be setup using the [kustomization file](https://github.com/kubernetes-sigs/container-object-storage-interface-csi-adapter/blob/master/kustomization.yaml) from the [container-object-storage-interface-csi-adapter](https://github.com/kubernetes-sigs/container-object-storage-interface-csi-adapter) repository with following command: 6 | 7 | ```sh 8 | kubectl create -k github.com/kubernetes-sigs/container-object-storage-interface-csi-adapter 9 | ``` 10 | The output should look like the following: 11 | ```sh 12 | storageclass.storage.k8s.io/objectstorage.k8s.io created 13 | serviceaccount/objectstorage-csi-adapter-sa created 14 | role.rbac.authorization.k8s.io/objectstorage-csi-adapter created 15 | clusterrole.rbac.authorization.k8s.io/objectstorage-csi-adapter-role created 16 | rolebinding.rbac.authorization.k8s.io/objectstorage-csi-adapter created 17 | clusterrolebinding.rbac.authorization.k8s.io/objectstorage-csi-adapter created 18 | secret/objectstorage.k8s.io created 19 | daemonset.apps/objectstorage-csi-adapter created 20 | csidriver.storage.k8s.io/objectstorage.k8s.io created 21 | ``` 22 | 23 | The CSI Adapter will be deployed in the `default` namespace. 24 | 25 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module sigs.k8s.io/container-object-storage-interface-csi-adapter 2 | 3 | go 1.15 4 | 5 | require ( 6 | github.com/container-storage-interface/spec v1.3.0 7 | github.com/google/go-cmp v0.5.2 8 | github.com/kubernetes-csi/csi-lib-utils v0.9.1 // indirect 9 | github.com/kubernetes-csi/drivers v1.0.2 10 | github.com/pkg/errors v0.9.1 11 | github.com/spf13/cobra v1.1.3 12 | github.com/spf13/viper v1.7.1 13 | google.golang.org/grpc v1.36.0 14 | k8s.io/api v0.20.4 15 | k8s.io/apimachinery v0.20.4 16 | k8s.io/client-go v0.20.0 17 | k8s.io/klog/v2 v2.8.0 18 | k8s.io/mount-utils v0.21.0 19 | k8s.io/utils v0.0.0-20210111153108-fddb29f9d009 // indirect 20 | sigs.k8s.io/container-object-storage-interface-api v0.0.0-20210417043410-0af83d5058ab 21 | sigs.k8s.io/controller-runtime v0.6.3 22 | ) 23 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= 2 | cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= 3 | cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU= 4 | cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU= 5 | cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY= 6 | cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc= 7 | cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0= 8 | cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To= 9 | cloud.google.com/go v0.51.0/go.mod h1:hWtGJ6gnXH+KgDv+V0zFGDvpi07n3z8ZNj3T1RW0Gcw= 10 | cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4= 11 | cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M= 12 | cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc= 13 | cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= 14 | cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= 15 | cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= 16 | cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= 17 | cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= 18 | cloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk= 19 | cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= 20 | cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= 21 | cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= 22 | cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw= 23 | cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos= 24 | cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk= 25 | dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= 26 | github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8= 27 | github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24= 28 | github.com/Azure/go-autorest/autorest v0.9.0/go.mod h1:xyHB1BMZT0cuDHU7I0+g046+BFDTQ8rEZB0s4Yfa6bI= 29 | github.com/Azure/go-autorest/autorest v0.9.6/go.mod h1:/FALq9T/kS7b5J5qsQ+RSTUdAmGFqi0vUdVNNx8q630= 30 | github.com/Azure/go-autorest/autorest v0.11.1/go.mod h1:JFgpikqFJ/MleTTxwepExTKnFUKKszPS8UavbQYUMuw= 31 | github.com/Azure/go-autorest/autorest/adal v0.5.0/go.mod h1:8Z9fGy2MpX0PvDjB1pEgQTmVqjGhiHBW7RJJEciWzS0= 32 | github.com/Azure/go-autorest/autorest/adal v0.8.2/go.mod h1:ZjhuQClTqx435SRJ2iMlOxPYt3d2C/T/7TiQCVZSn3Q= 33 | github.com/Azure/go-autorest/autorest/adal v0.9.0/go.mod h1:/c022QCutn2P7uY+/oQWWNcK9YU+MH96NgK+jErpbcg= 34 | github.com/Azure/go-autorest/autorest/adal v0.9.5/go.mod h1:B7KF7jKIeC9Mct5spmyCB/A8CG/sEz1vwIRGv/bbw7A= 35 | github.com/Azure/go-autorest/autorest/date v0.1.0/go.mod h1:plvfp3oPSKwf2DNjlBjWF/7vwR+cUD/ELuzDCXwHUVA= 36 | github.com/Azure/go-autorest/autorest/date v0.2.0/go.mod h1:vcORJHLJEh643/Ioh9+vPmf1Ij9AEBM5FuBIXLmIy0g= 37 | github.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74= 38 | github.com/Azure/go-autorest/autorest/mocks v0.1.0/go.mod h1:OTyCOPRA2IgIlWxVYxBee2F5Gr4kF2zd2J5cFRaIDN0= 39 | github.com/Azure/go-autorest/autorest/mocks v0.2.0/go.mod h1:OTyCOPRA2IgIlWxVYxBee2F5Gr4kF2zd2J5cFRaIDN0= 40 | github.com/Azure/go-autorest/autorest/mocks v0.3.0/go.mod h1:a8FDP3DYzQ4RYfVAxAN3SVSiiO77gL2j2ronKKP0syM= 41 | github.com/Azure/go-autorest/autorest/mocks v0.4.0/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k= 42 | github.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k= 43 | github.com/Azure/go-autorest/logger v0.1.0/go.mod h1:oExouG+K6PryycPJfVSxi/koC6LSNgds39diKLz7Vrc= 44 | github.com/Azure/go-autorest/logger v0.2.0/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8= 45 | github.com/Azure/go-autorest/tracing v0.5.0/go.mod h1:r/s2XiOKccPW3HrqB+W0TQzfbtp2fGCgRFtBroKn4Dk= 46 | github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU= 47 | github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ= 48 | github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= 49 | github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= 50 | github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ= 51 | github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU= 52 | github.com/PuerkitoBio/purell v1.0.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0= 53 | github.com/PuerkitoBio/purell v1.1.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0= 54 | github.com/PuerkitoBio/purell v1.1.1 h1:WEQqlqaGbrPkxLJWfBwQmfEAE1Z7ONdDLqrN38tNFfI= 55 | github.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0= 56 | github.com/PuerkitoBio/urlesc v0.0.0-20160726150825-5bd2802263f2/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE= 57 | github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 h1:d+Bc7a5rLufV/sSk/8dngufqelfh6jnri85riMAaF/M= 58 | github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE= 59 | github.com/agnivade/levenshtein v1.0.1/go.mod h1:CURSv5d9Uaml+FovSIICkLbAUZ9S4RqaHDIsdSBg7lM= 60 | github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= 61 | github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= 62 | github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= 63 | github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= 64 | github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8= 65 | github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o= 66 | github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8= 67 | github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY= 68 | github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= 69 | github.com/asaskevich/govalidator v0.0.0-20180720115003-f9ffefc3facf/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY= 70 | github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY= 71 | github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= 72 | github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= 73 | github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= 74 | github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= 75 | github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= 76 | github.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJmJgSg28kpZDP6UIiPt0e0Oz0kqKNGyRaWEPv84= 77 | github.com/blang/semver v3.5.0+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk= 78 | github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= 79 | github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko= 80 | github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc= 81 | github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+qY= 82 | github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= 83 | github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= 84 | github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= 85 | github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= 86 | github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= 87 | github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= 88 | github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= 89 | github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8= 90 | github.com/container-storage-interface/spec v1.2.0/go.mod h1:6URME8mwIBbpVyZV93Ce5St17xBiQJQY67NDsuohiy4= 91 | github.com/container-storage-interface/spec v1.3.0 h1:wMH4UIoWnK/TXYw8mbcIHgZmB6kHOeIsYsiaTJwa6bc= 92 | github.com/container-storage-interface/spec v1.3.0/go.mod h1:6URME8mwIBbpVyZV93Ce5St17xBiQJQY67NDsuohiy4= 93 | github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk= 94 | github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE= 95 | github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE= 96 | github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk= 97 | github.com/coreos/go-oidc v2.1.0+incompatible/go.mod h1:CgnwVTmzoESiwO9qyAFEMiHoZ1nMCKZlZ9V6mm3/LKc= 98 | github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= 99 | github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= 100 | github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= 101 | github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= 102 | github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= 103 | github.com/coreos/pkg v0.0.0-20180108230652-97fdf19511ea/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= 104 | github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= 105 | github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE= 106 | github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= 107 | github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY= 108 | github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= 109 | github.com/dave/jennifer v1.4.1/go.mod h1:7jEdnm+qBcxl8PC0zyp7vxcpSRnzXSt9r39tpTVGlwA= 110 | github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 111 | github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= 112 | github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 113 | github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= 114 | github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no= 115 | github.com/docker/docker v0.7.3-0.20190327010347-be7ac8be2ae0/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= 116 | github.com/docker/go-units v0.3.3/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= 117 | github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= 118 | github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96/go.mod h1:Qh8CwZgvJUkLughtfhJv5dyTYa91l1fOUCrgjqmcifM= 119 | github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= 120 | github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= 121 | github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= 122 | github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc= 123 | github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs= 124 | github.com/emicklei/go-restful v2.9.5+incompatible h1:spTtZBk5DYEvbxMVutUuTyh1Ao2r4iyvLdACqsl/Ljk= 125 | github.com/emicklei/go-restful v2.9.5+incompatible/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs= 126 | github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= 127 | github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= 128 | github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= 129 | github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= 130 | github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= 131 | github.com/evanphx/json-patch v4.2.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= 132 | github.com/evanphx/json-patch v4.5.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= 133 | github.com/evanphx/json-patch v4.9.0+incompatible h1:kLcOMZeuLAJvL2BPWLMIj5oaZQobrkAqrL+WFZwQses= 134 | github.com/evanphx/json-patch v4.9.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= 135 | github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= 136 | github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k= 137 | github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= 138 | github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4= 139 | github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= 140 | github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= 141 | github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= 142 | github.com/globalsign/mgo v0.0.0-20180905125535-1ca0a4f7cbcb/go.mod h1:xkRDCp4j0OGD1HRkm4kmhM+pmpv3AKq5SU7GMg4oO/Q= 143 | github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8/go.mod h1:xkRDCp4j0OGD1HRkm4kmhM+pmpv3AKq5SU7GMg4oO/Q= 144 | github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= 145 | github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= 146 | github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= 147 | github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= 148 | github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= 149 | github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= 150 | github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= 151 | github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas= 152 | github.com/go-logr/logr v0.2.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU= 153 | github.com/go-logr/logr v0.4.0 h1:K7/B1jt6fIBQVd4Owv2MqGQClcgf0R266+7C/QjRcLc= 154 | github.com/go-logr/logr v0.4.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU= 155 | github.com/go-logr/zapr v0.1.0 h1:h+WVe9j6HAA01niTJPA/kKH0i7e0rLZBCwauQFcRE54= 156 | github.com/go-logr/zapr v0.1.0/go.mod h1:tabnROwaDl0UNxkVeFRbY8bwB37GwRv0P8lg6aAiEnk= 157 | github.com/go-openapi/analysis v0.0.0-20180825180245-b006789cd277/go.mod h1:k70tL6pCuVxPJOHXQ+wIac1FUrvNkHolPie/cLEU6hI= 158 | github.com/go-openapi/analysis v0.17.0/go.mod h1:IowGgpVeD0vNm45So8nr+IcQ3pxVtpRoBWb8PVZO0ik= 159 | github.com/go-openapi/analysis v0.18.0/go.mod h1:IowGgpVeD0vNm45So8nr+IcQ3pxVtpRoBWb8PVZO0ik= 160 | github.com/go-openapi/analysis v0.19.2/go.mod h1:3P1osvZa9jKjb8ed2TPng3f0i/UY9snX6gxi44djMjk= 161 | github.com/go-openapi/analysis v0.19.5/go.mod h1:hkEAkxagaIvIP7VTn8ygJNkd4kAYON2rCu0v0ObL0AU= 162 | github.com/go-openapi/errors v0.17.0/go.mod h1:LcZQpmvG4wyF5j4IhA73wkLFQg+QJXOQHVjmcZxhka0= 163 | github.com/go-openapi/errors v0.18.0/go.mod h1:LcZQpmvG4wyF5j4IhA73wkLFQg+QJXOQHVjmcZxhka0= 164 | github.com/go-openapi/errors v0.19.2/go.mod h1:qX0BLWsyaKfvhluLejVpVNwNRdXZhEbTA4kxxpKBC94= 165 | github.com/go-openapi/jsonpointer v0.0.0-20160704185906-46af16f9f7b1/go.mod h1:+35s3my2LFTysnkMfxsJBAMHj/DoqoB9knIWoYG/Vk0= 166 | github.com/go-openapi/jsonpointer v0.17.0/go.mod h1:cOnomiV+CVVwFLk0A/MExoFMjwdsUdVpsRhURCKh+3M= 167 | github.com/go-openapi/jsonpointer v0.18.0/go.mod h1:cOnomiV+CVVwFLk0A/MExoFMjwdsUdVpsRhURCKh+3M= 168 | github.com/go-openapi/jsonpointer v0.19.2/go.mod h1:3akKfEdA7DF1sugOqz1dVQHBcuDBPKZGEoHC/NkiQRg= 169 | github.com/go-openapi/jsonpointer v0.19.3 h1:gihV7YNZK1iK6Tgwwsxo2rJbD1GTbdm72325Bq8FI3w= 170 | github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= 171 | github.com/go-openapi/jsonreference v0.0.0-20160704190145-13c6e3589ad9/go.mod h1:W3Z9FmVs9qj+KR4zFKmDPGiLdk1D9Rlm7cyMvf57TTg= 172 | github.com/go-openapi/jsonreference v0.17.0/go.mod h1:g4xxGn04lDIRh0GJb5QlpE3HfopLOL6uZrK/VgnsK9I= 173 | github.com/go-openapi/jsonreference v0.18.0/go.mod h1:g4xxGn04lDIRh0GJb5QlpE3HfopLOL6uZrK/VgnsK9I= 174 | github.com/go-openapi/jsonreference v0.19.2/go.mod h1:jMjeRr2HHw6nAVajTXJ4eiUwohSTlpa0o73RUL1owJc= 175 | github.com/go-openapi/jsonreference v0.19.3/go.mod h1:rjx6GuL8TTa9VaixXglHmQmIL98+wF9xc8zWvFonSJ8= 176 | github.com/go-openapi/jsonreference v0.19.4 h1:3Vw+rh13uq2JFNxgnMTGE1rnoieU9FmyE1gvnyylsYg= 177 | github.com/go-openapi/jsonreference v0.19.4/go.mod h1:RdybgQwPxbL4UEjuAruzK1x3nE69AqPYEJeo/TWfEeg= 178 | github.com/go-openapi/loads v0.17.0/go.mod h1:72tmFy5wsWx89uEVddd0RjRWPZm92WRLhf7AC+0+OOU= 179 | github.com/go-openapi/loads v0.18.0/go.mod h1:72tmFy5wsWx89uEVddd0RjRWPZm92WRLhf7AC+0+OOU= 180 | github.com/go-openapi/loads v0.19.0/go.mod h1:72tmFy5wsWx89uEVddd0RjRWPZm92WRLhf7AC+0+OOU= 181 | github.com/go-openapi/loads v0.19.2/go.mod h1:QAskZPMX5V0C2gvfkGZzJlINuP7Hx/4+ix5jWFxsNPs= 182 | github.com/go-openapi/loads v0.19.4/go.mod h1:zZVHonKd8DXyxyw4yfnVjPzBjIQcLt0CCsn0N0ZrQsk= 183 | github.com/go-openapi/runtime v0.0.0-20180920151709-4f900dc2ade9/go.mod h1:6v9a6LTXWQCdL8k1AO3cvqx5OtZY/Y9wKTgaoP6YRfA= 184 | github.com/go-openapi/runtime v0.19.0/go.mod h1:OwNfisksmmaZse4+gpV3Ne9AyMOlP1lt4sK4FXt0O64= 185 | github.com/go-openapi/runtime v0.19.4/go.mod h1:X277bwSUBxVlCYR3r7xgZZGKVvBd/29gLDlFGtJ8NL4= 186 | github.com/go-openapi/spec v0.0.0-20160808142527-6aced65f8501/go.mod h1:J8+jY1nAiCcj+friV/PDoE1/3eeccG9LYBs0tYvLOWc= 187 | github.com/go-openapi/spec v0.17.0/go.mod h1:XkF/MOi14NmjsfZ8VtAKf8pIlbZzyoTvZsdfssdxcBI= 188 | github.com/go-openapi/spec v0.18.0/go.mod h1:XkF/MOi14NmjsfZ8VtAKf8pIlbZzyoTvZsdfssdxcBI= 189 | github.com/go-openapi/spec v0.19.2/go.mod h1:sCxk3jxKgioEJikev4fgkNmwS+3kuYdJtcsZsD5zxMY= 190 | github.com/go-openapi/spec v0.19.3/go.mod h1:FpwSN1ksY1eteniUU7X0N/BgJ7a4WvBFVA8Lj9mJglo= 191 | github.com/go-openapi/spec v0.19.12 h1:OO9WrvhDwtiMY/Opr1j1iFZzirI3JW4/bxNFRcntAr4= 192 | github.com/go-openapi/spec v0.19.12/go.mod h1:gwrgJS15eCUgjLpMjBJmbZezCsw88LmgeEip0M63doA= 193 | github.com/go-openapi/strfmt v0.17.0/go.mod h1:P82hnJI0CXkErkXi8IKjPbNBM6lV6+5pLP5l494TcyU= 194 | github.com/go-openapi/strfmt v0.18.0/go.mod h1:P82hnJI0CXkErkXi8IKjPbNBM6lV6+5pLP5l494TcyU= 195 | github.com/go-openapi/strfmt v0.19.0/go.mod h1:+uW+93UVvGGq2qGaZxdDeJqSAqBqBdl+ZPMF/cC8nDY= 196 | github.com/go-openapi/strfmt v0.19.3/go.mod h1:0yX7dbo8mKIvc3XSKp7MNfxw4JytCfCD6+bY1AVL9LU= 197 | github.com/go-openapi/swag v0.0.0-20160704191624-1d0bd113de87/go.mod h1:DXUve3Dpr1UfpPtxFw+EFuQ41HhCWZfha5jSVRG7C7I= 198 | github.com/go-openapi/swag v0.17.0/go.mod h1:AByQ+nYG6gQg71GINrmuDXCPWdL640yX49/kXLo40Tg= 199 | github.com/go-openapi/swag v0.18.0/go.mod h1:AByQ+nYG6gQg71GINrmuDXCPWdL640yX49/kXLo40Tg= 200 | github.com/go-openapi/swag v0.19.2/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk= 201 | github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk= 202 | github.com/go-openapi/swag v0.19.11 h1:RFTu/dlFySpyVvJDfp/7674JY4SDglYWKztbiIGFpmc= 203 | github.com/go-openapi/swag v0.19.11/go.mod h1:Uc0gKkdR+ojzsEpjh39QChyu92vPgIr72POcgHMAgSY= 204 | github.com/go-openapi/validate v0.18.0/go.mod h1:Uh4HdOzKt19xGIGm1qHf/ofbX1YQ4Y+MYsct2VUrAJ4= 205 | github.com/go-openapi/validate v0.19.2/go.mod h1:1tRCw7m3jtI8eNWEEliiAqUIcBztB2KDnRCRMUi7GTA= 206 | github.com/go-openapi/validate v0.19.5/go.mod h1:8DJv2CVJQ6kGNpFW6eV9N3JviE1C85nY1c2z52x1Gk4= 207 | github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= 208 | github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= 209 | github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4= 210 | github.com/gogo/protobuf v1.3.1 h1:DqDEcV5aeaTmdFBePNpYsp3FlcVH/2ISVVM9Qf8PSls= 211 | github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o= 212 | github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58= 213 | github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= 214 | github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= 215 | github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= 216 | github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= 217 | github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= 218 | github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e h1:1r7pUrabqp18hOBcwBwiTsbnFeTZHV9eER/QT5JVZxY= 219 | github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= 220 | github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= 221 | github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= 222 | github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y= 223 | github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= 224 | github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= 225 | github.com/golang/protobuf v0.0.0-20161109072736-4bd1920723d7/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= 226 | github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= 227 | github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= 228 | github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= 229 | github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= 230 | github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= 231 | github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= 232 | github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= 233 | github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= 234 | github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= 235 | github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= 236 | github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= 237 | github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= 238 | github.com/golang/protobuf v1.4.3 h1:JjCZWpVbqXDqFVmTfYWEVTMIYrL/NPdPSCHPJ0T/raM= 239 | github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= 240 | github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= 241 | github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= 242 | github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= 243 | github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= 244 | github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= 245 | github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= 246 | github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= 247 | github.com/google/go-cmp v0.5.2 h1:X2ev0eStA3AbceY54o37/0PQ/UWqKEiiO2dKL5OPaFM= 248 | github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= 249 | github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= 250 | github.com/google/gofuzz v1.1.0 h1:Hsa8mG0dQ46ij8Sl2AYJDUv1oA9/d6Vk+3LG99Oe02g= 251 | github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= 252 | github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= 253 | github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= 254 | github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= 255 | github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= 256 | github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= 257 | github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= 258 | github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= 259 | github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= 260 | github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= 261 | github.com/google/uuid v1.1.2 h1:EVhdT+1Kseyi1/pUmXKaFxYsDNy9RQYkMWRH68J/W7Y= 262 | github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= 263 | github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= 264 | github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= 265 | github.com/googleapis/gnostic v0.0.0-20170729233727-0c5108395e2d/go.mod h1:sJBsCZ4ayReDTBIg8b9dl28c5xFWyhBTVRp3pOg5EKY= 266 | github.com/googleapis/gnostic v0.1.0/go.mod h1:sJBsCZ4ayReDTBIg8b9dl28c5xFWyhBTVRp3pOg5EKY= 267 | github.com/googleapis/gnostic v0.3.1/go.mod h1:on+2t9HRStVgn95RSsFWFz+6Q0Snyqv1awfrALZdbtU= 268 | github.com/googleapis/gnostic v0.4.1 h1:DLJCy1n/vrD4HPjOvYcT8aYQXpPIzoRZONaYwyycI+I= 269 | github.com/googleapis/gnostic v0.4.1/go.mod h1:LRhVm6pbyptWbWbuZ38d1eyptfvIytN3ir6b65WBswg= 270 | github.com/gophercloud/gophercloud v0.1.0/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8= 271 | github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8= 272 | github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= 273 | github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ= 274 | github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ= 275 | github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= 276 | github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA= 277 | github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= 278 | github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= 279 | github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk= 280 | github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= 281 | github.com/grpc-ecosystem/grpc-gateway v1.9.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= 282 | github.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q= 283 | github.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8= 284 | github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= 285 | github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= 286 | github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= 287 | github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM= 288 | github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk= 289 | github.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU= 290 | github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU= 291 | github.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4= 292 | github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= 293 | github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= 294 | github.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90= 295 | github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= 296 | github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= 297 | github.com/hashicorp/golang-lru v0.5.4 h1:YDjusn29QI/Das2iO9M0BHnIbxPeyuCHsjMW+lJfyTc= 298 | github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= 299 | github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4= 300 | github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ= 301 | github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= 302 | github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ= 303 | github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I= 304 | github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc= 305 | github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= 306 | github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= 307 | github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= 308 | github.com/imdario/mergo v0.3.9 h1:UauaLniWCFHWd+Jp9oCEkTBj8VO/9DKg3PV3VCNMDIg= 309 | github.com/imdario/mergo v0.3.9/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= 310 | github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM= 311 | github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= 312 | github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo= 313 | github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= 314 | github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= 315 | github.com/json-iterator/go v1.1.8/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= 316 | github.com/json-iterator/go v1.1.10 h1:Kz6Cvnvv2wGdaG/V8yMvfkmNiXq9Ya2KUv4rouJJr68= 317 | github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= 318 | github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= 319 | github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= 320 | github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= 321 | github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= 322 | github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= 323 | github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q= 324 | github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00= 325 | github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= 326 | github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= 327 | github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= 328 | github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= 329 | github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= 330 | github.com/kr/pretty v0.2.0 h1:s5hAObm+yFO5uHYt5dYjxi2rXrsnmRpJx4OYvIWUaQs= 331 | github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= 332 | github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= 333 | github.com/kr/pty v1.1.5/go.mod h1:9r2w37qlBe7rQ6e1fg1S/9xpWHSnaqNdHD3WcMdbPDA= 334 | github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= 335 | github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= 336 | github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= 337 | github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= 338 | github.com/kubernetes-csi/csi-lib-utils v0.9.1 h1:sGq6ifVujfMSkfTsMZip44Ttv8SDXvsBlFk9GdYl/b8= 339 | github.com/kubernetes-csi/csi-lib-utils v0.9.1/go.mod h1:8E2jVUX9j3QgspwHXa6LwyN7IHQDjW9jX3kwoWnSC+M= 340 | github.com/kubernetes-csi/drivers v1.0.2 h1:kaEAMfo+W5YFr23yedBIY+NGnNjr6/PbPzx7N4GYgiQ= 341 | github.com/kubernetes-csi/drivers v1.0.2/go.mod h1:V6rHbbSLCZGaQoIZ8MkyDtoXtcKXZM0F7N3bkloDCOY= 342 | github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= 343 | github.com/magiconair/properties v1.8.1 h1:ZC2Vc7/ZFkGmsVC9KvOjumD+G5lXy2RtTKyzRKO2BQ4= 344 | github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= 345 | github.com/mailru/easyjson v0.0.0-20160728113105-d5b7844b561a/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= 346 | github.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= 347 | github.com/mailru/easyjson v0.0.0-20190312143242-1de009706dbe/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= 348 | github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= 349 | github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= 350 | github.com/mailru/easyjson v0.7.0 h1:aizVhC/NAAcKWb+5QsU1iNOZb4Yws5UO2I+aIprQITM= 351 | github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs= 352 | github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= 353 | github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= 354 | github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= 355 | github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU= 356 | github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= 357 | github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 h1:I0XW9+e1XWDxdcEniV4rQAIOPUGDq67JSCiRCgGCZLI= 358 | github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= 359 | github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg= 360 | github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc= 361 | github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= 362 | github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= 363 | github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI= 364 | github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg= 365 | github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY= 366 | github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= 367 | github.com/mitchellh/mapstructure v1.1.2 h1:fmNYVwqnSfB9mZU6OS2O6GsXM+wcskZDuKQzvN1EDeE= 368 | github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= 369 | github.com/moby/term v0.0.0-20200312100748-672ec06f55cd/go.mod h1:DdlQx2hp0Ss5/fLikoLlEeIYiATotOjgB//nb973jeo= 370 | github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= 371 | github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= 372 | github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= 373 | github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= 374 | github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9AWI= 375 | github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= 376 | github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= 377 | github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= 378 | github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= 379 | github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw= 380 | github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs= 381 | github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= 382 | github.com/nxadm/tail v1.4.4 h1:DQuhQpB1tVlglWS2hLQ5OV6B5r8aGxSrPc5Qo6uTN78= 383 | github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A= 384 | github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U= 385 | github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo= 386 | github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= 387 | github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= 388 | github.com/onsi/ginkgo v1.11.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= 389 | github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= 390 | github.com/onsi/ginkgo v1.14.2 h1:8mVmC9kjFFmA8H4pKMUhcblgifdkOIXPvbhN1T36q1M= 391 | github.com/onsi/ginkgo v1.14.2/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY= 392 | github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA= 393 | github.com/onsi/gomega v1.7.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= 394 | github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= 395 | github.com/onsi/gomega v1.10.1 h1:o0+MgICZLuZ7xjH7Vx6zS/zcu93/BEp1VwkIW1mEXCE= 396 | github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= 397 | github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= 398 | github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k= 399 | github.com/pelletier/go-toml v1.2.0 h1:T5zMGML61Wp+FlcbWjRDT7yAxhJNAiPPLOFECq181zc= 400 | github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= 401 | github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= 402 | github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 403 | github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 404 | github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= 405 | github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 406 | github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= 407 | github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= 408 | github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI= 409 | github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA= 410 | github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= 411 | github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso= 412 | github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= 413 | github.com/prometheus/client_golang v1.7.1 h1:NTGy1Ja9pByO+xAeH/qiWnLrKtr3hJPNjaVUwnjpdpA= 414 | github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= 415 | github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= 416 | github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= 417 | github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= 418 | github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M= 419 | github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= 420 | github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= 421 | github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= 422 | github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= 423 | github.com/prometheus/common v0.10.0 h1:RyRA7RzGXQZiW+tGMr7sxa85G1z0yOpM1qq5c8lNawc= 424 | github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= 425 | github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= 426 | github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= 427 | github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= 428 | github.com/prometheus/procfs v0.0.11/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= 429 | github.com/prometheus/procfs v0.1.3 h1:F0+tqvhOksq22sc6iCHF5WGlWjdwj92p0udFh1VFBS8= 430 | github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= 431 | github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU= 432 | github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg= 433 | github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= 434 | github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g= 435 | github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= 436 | github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= 437 | github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc= 438 | github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo= 439 | github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= 440 | github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= 441 | github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= 442 | github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= 443 | github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d h1:zE9ykElWQ6/NYmHa3jpm/yHnI4xSofP+UP6SpjHcSeM= 444 | github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= 445 | github.com/smartystreets/goconvey v1.6.4 h1:fv0U8FUIMPNf1L9lnHLvLhgicrIVChEkdzIKYqbNC9s= 446 | github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= 447 | github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= 448 | github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= 449 | github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= 450 | github.com/spf13/afero v1.2.2 h1:5jhuqJyZCZf2JRofRvN/nIFgIWNzPa3/Vz8mYylgbWc= 451 | github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= 452 | github.com/spf13/cast v1.3.0 h1:oget//CVOEoFewqQxwr0Ej5yjygnqGkvggSE/gB35Q8= 453 | github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= 454 | github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= 455 | github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU= 456 | github.com/spf13/cobra v1.1.3 h1:xghbfqPkxzxP3C/f3n5DdpAbdKLj4ZE4BWQI362l53M= 457 | github.com/spf13/cobra v1.1.3/go.mod h1:pGADOWyqRD/YMrPZigI/zbliZ2wVD/23d+is3pSWzOo= 458 | github.com/spf13/jwalterweatherman v1.0.0 h1:XHEdyB+EcvlqZamSM4ZOMGlc93t6AcsBEu9Gc1vn7yk= 459 | github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= 460 | github.com/spf13/pflag v0.0.0-20170130214245-9ff6c6923cff/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= 461 | github.com/spf13/pflag v1.0.1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= 462 | github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= 463 | github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= 464 | github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= 465 | github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s= 466 | github.com/spf13/viper v1.7.0/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5qpdg= 467 | github.com/spf13/viper v1.7.1 h1:pM5oEahlgWv/WnHXpgbKz7iLIxRf65tye2Ci+XFK5sk= 468 | github.com/spf13/viper v1.7.1/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5qpdg= 469 | github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= 470 | github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= 471 | github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE= 472 | github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= 473 | github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= 474 | github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= 475 | github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= 476 | github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0= 477 | github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= 478 | github.com/subosito/gotenv v1.2.0 h1:Slr1R9HxAlEKefgq5jn9U+DnETlIUa6HfgEzj0g5d7s= 479 | github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw= 480 | github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk= 481 | github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= 482 | github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= 483 | github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0= 484 | github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA= 485 | github.com/vektah/gqlparser v1.1.2/go.mod h1:1ycwN7Ij5njmMkPPAOaRFY4rET2Enx7IkVv3vaXspKw= 486 | github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU= 487 | github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q= 488 | go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= 489 | go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= 490 | go.etcd.io/etcd v0.0.0-20191023171146-3cf2f69b5738/go.mod h1:dnLIgRNXwCJa5e+c6mIZCrds/GIG4ncV9HhK5PX7jPg= 491 | go.mongodb.org/mongo-driver v1.0.3/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM= 492 | go.mongodb.org/mongo-driver v1.1.1/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM= 493 | go.mongodb.org/mongo-driver v1.1.2/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM= 494 | go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= 495 | go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= 496 | go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= 497 | go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= 498 | go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= 499 | go.uber.org/atomic v1.4.0 h1:cxzIVoETapQEqDhQu3QfnvXAV4AlzcvUCxkVUFw3+EU= 500 | go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= 501 | go.uber.org/multierr v1.1.0 h1:HoEmRHQPVSqub6w2z2d2EOVs2fjyFRGyofhKuyDq0QI= 502 | go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0= 503 | go.uber.org/zap v1.10.0 h1:ORx85nbTijNz8ljznvCMR1ZBIPKFn3jQrag10X2AsuM= 504 | go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= 505 | golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= 506 | golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= 507 | golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= 508 | golang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= 509 | golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= 510 | golang.org/x/crypto v0.0.0-20190320223903-b7391e95e576/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= 511 | golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= 512 | golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= 513 | golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= 514 | golang.org/x/crypto v0.0.0-20190617133340-57b3e21c3d56/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= 515 | golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= 516 | golang.org/x/crypto v0.0.0-20191206172530-e9b2fee46413/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= 517 | golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= 518 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= 519 | golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0 h1:hb9wdF1z5waM+dSIICn1l0DkLVDT3hqhhQsDNUmHPRE= 520 | golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= 521 | golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= 522 | golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= 523 | golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= 524 | golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek= 525 | golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY= 526 | golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= 527 | golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= 528 | golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= 529 | golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM= 530 | golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU= 531 | golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= 532 | golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= 533 | golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= 534 | golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= 535 | golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= 536 | golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= 537 | golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= 538 | golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= 539 | golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= 540 | golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs= 541 | golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= 542 | golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= 543 | golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE= 544 | golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o= 545 | golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= 546 | golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY= 547 | golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= 548 | golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= 549 | golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= 550 | golang.org/x/net v0.0.0-20170114055629-f2499483f923/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 551 | golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 552 | golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 553 | golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 554 | golang.org/x/net v0.0.0-20181005035420-146acd28ed58/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 555 | golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 556 | golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 557 | golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 558 | golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 559 | golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 560 | golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 561 | golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 562 | golang.org/x/net v0.0.0-20190320064053-1272bf9dcd53/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 563 | golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 564 | golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 565 | golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 566 | golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= 567 | golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 568 | golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 569 | golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 570 | golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 571 | golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 572 | golang.org/x/net v0.0.0-20191002035440-2ec189313ef0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 573 | golang.org/x/net v0.0.0-20191004110552-13f9640d40b9/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 574 | golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 575 | golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 576 | golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 577 | golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 578 | golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 579 | golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 580 | golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= 581 | golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= 582 | golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= 583 | golang.org/x/net v0.0.0-20201110031124-69a78807bb2b h1:uwuIcX0g4Yl1NC5XAz37xsr2lTtcqevgzYNVt49waME= 584 | golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= 585 | golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= 586 | golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= 587 | golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= 588 | golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= 589 | golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d h1:TzXSXBo42m9gQenoE3b9BGiEpg5IG2JkU5FkPIawgtw= 590 | golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= 591 | golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 592 | golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 593 | golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 594 | golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 595 | golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 596 | golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 597 | golang.org/x/sys v0.0.0-20170830134202-bb24a47a89ea/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 598 | golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 599 | golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 600 | golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 601 | golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 602 | golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 603 | golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 604 | golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 605 | golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 606 | golang.org/x/sys v0.0.0-20190209173611-3b5209105503/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 607 | golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 608 | golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 609 | golang.org/x/sys v0.0.0-20190321052220-f7bb7a8bee54/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 610 | golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 611 | golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 612 | golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 613 | golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 614 | golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 615 | golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 616 | golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 617 | golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 618 | golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 619 | golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 620 | golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 621 | golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 622 | golang.org/x/sys v0.0.0-20191022100944-742c48ecaeb7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 623 | golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 624 | golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 625 | golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 626 | golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 627 | golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 628 | golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 629 | golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 630 | golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 631 | golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 632 | golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 633 | golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 634 | golang.org/x/sys v0.0.0-20200519105757-fe76b779f299/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 635 | golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 636 | golang.org/x/sys v0.0.0-20200622214017-ed371f2e16b4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 637 | golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 638 | golang.org/x/sys v0.0.0-20201112073958-5cba982894dd h1:5CtCZbICpIOFdgO940moixOPjc0178IU44m4EjOO5IY= 639 | golang.org/x/sys v0.0.0-20201112073958-5cba982894dd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 640 | golang.org/x/text v0.0.0-20160726164857-2910a502d2bf/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= 641 | golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= 642 | golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= 643 | golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= 644 | golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= 645 | golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= 646 | golang.org/x/text v0.3.4 h1:0YWbFKbhXG/wIiuHDSKpS0Iy7FSA+u45VtBMfQcFTTc= 647 | golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= 648 | golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= 649 | golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= 650 | golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= 651 | golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= 652 | golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e h1:EHBhcS0mlXEAVwNyO2dLfjToGsyY4j24pTs2ScHnX7s= 653 | golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= 654 | golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 655 | golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 656 | golang.org/x/tools v0.0.0-20181011042414-1f849cf54d09/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 657 | golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 658 | golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 659 | golang.org/x/tools v0.0.0-20190125232054-d66bd3c5d5a6/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 660 | golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= 661 | golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= 662 | golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= 663 | golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= 664 | golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= 665 | golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= 666 | golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= 667 | golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= 668 | golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= 669 | golang.org/x/tools v0.0.0-20190614205625-5aca471b1d59/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= 670 | golang.org/x/tools v0.0.0-20190617190820-da514acc4774/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= 671 | golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= 672 | golang.org/x/tools v0.0.0-20190624222133-a101b041ded4/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= 673 | golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= 674 | golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 675 | golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 676 | golang.org/x/tools v0.0.0-20190920225731-5eefd052ad72/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 677 | golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 678 | golang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 679 | golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 680 | golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 681 | golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 682 | golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 683 | golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 684 | golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= 685 | golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= 686 | golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= 687 | golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= 688 | golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= 689 | golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= 690 | golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= 691 | golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= 692 | golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= 693 | golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= 694 | golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 695 | golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 696 | golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 697 | golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE= 698 | golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 699 | gomodules.xyz/jsonpatch/v2 v2.0.1 h1:xyiBuvkD2g5n7cYzx6u2sxQvsAy4QJsZFCzGVdzOXZ0= 700 | gomodules.xyz/jsonpatch/v2 v2.0.1/go.mod h1:IhYNNY4jnS53ZnfE4PAmpKtDpTCj1JFXc+3mwe7XcUU= 701 | google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE= 702 | google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M= 703 | google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= 704 | google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= 705 | google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= 706 | google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= 707 | google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= 708 | google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= 709 | google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= 710 | google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= 711 | google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= 712 | google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= 713 | google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= 714 | google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0= 715 | google.golang.org/appengine v1.6.5 h1:tycE03LOZYQNhDpS27tcQdAzLCVMaj7QT2SXxebnpCM= 716 | google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= 717 | google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= 718 | google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= 719 | google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= 720 | google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= 721 | google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= 722 | google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= 723 | google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= 724 | google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8= 725 | google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= 726 | google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= 727 | google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= 728 | google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= 729 | google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= 730 | google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= 731 | google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA= 732 | google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= 733 | google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= 734 | google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= 735 | google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013 h1:+kGHl1aib/qcwaRi1CbqBZ1rk19r85MNUf8HaBghugY= 736 | google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= 737 | google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= 738 | google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38= 739 | google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= 740 | google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= 741 | google.golang.org/grpc v1.23.1/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= 742 | google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY= 743 | google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= 744 | google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= 745 | google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= 746 | google.golang.org/grpc v1.29.0/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk= 747 | google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= 748 | google.golang.org/grpc v1.36.0 h1:o1bcQ6imQMIOpdrO3SWf2z5RV72WbDwdXuK0MDlc8As= 749 | google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= 750 | google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= 751 | google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= 752 | google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= 753 | google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= 754 | google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= 755 | google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= 756 | google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= 757 | google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= 758 | google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4= 759 | google.golang.org/protobuf v1.25.0 h1:Ejskq+SyPohKW+1uil0JJMtmHCgJPJ/qWTxr8qp+R4c= 760 | google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= 761 | gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= 762 | gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 763 | gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 764 | gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo= 765 | gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 766 | gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU= 767 | gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 768 | gopkg.in/cheggaaa/pb.v1 v1.0.25/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw= 769 | gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= 770 | gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= 771 | gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= 772 | gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= 773 | gopkg.in/ini.v1 v1.51.0 h1:AQvPpx3LzTDM0AjnIRlVFwFFGC+npRopjZxLJj6gdno= 774 | gopkg.in/ini.v1 v1.51.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= 775 | gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k= 776 | gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo= 777 | gopkg.in/square/go-jose.v2 v2.2.2/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI= 778 | gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ= 779 | gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= 780 | gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74= 781 | gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 782 | gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 783 | gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 784 | gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 785 | gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 786 | gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 787 | gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= 788 | gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= 789 | gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo= 790 | gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= 791 | gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw= 792 | gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk= 793 | honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= 794 | honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= 795 | honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= 796 | honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= 797 | honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= 798 | honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= 799 | k8s.io/api v0.18.6/go.mod h1:eeyxr+cwCjMdLAmr2W3RyDI0VvTawSg/3RFFBEnmZGI= 800 | k8s.io/api v0.19.0/go.mod h1:I1K45XlvTrDjmj5LoM5LuP/KYrhWbjUKT/SoPG0qTjw= 801 | k8s.io/api v0.19.4/go.mod h1:SbtJ2aHCItirzdJ36YslycFNzWADYH3tgOhvBEFtZAk= 802 | k8s.io/api v0.20.0/go.mod h1:HyLC5l5eoS/ygQYl1BXBgFzWNlkHiAuyNAbevIn+FKg= 803 | k8s.io/api v0.20.4 h1:xZjKidCirayzX6tHONRQyTNDVIR55TYVqgATqo6ZULY= 804 | k8s.io/api v0.20.4/go.mod h1:++lNL1AJMkDymriNniQsWRkMDzRaX2Y/POTUi8yvqYQ= 805 | k8s.io/apiextensions-apiserver v0.18.6 h1:vDlk7cyFsDyfwn2rNAO2DbmUbvXy5yT5GE3rrqOzaMo= 806 | k8s.io/apiextensions-apiserver v0.18.6/go.mod h1:lv89S7fUysXjLZO7ke783xOwVTm6lKizADfvUM/SS/M= 807 | k8s.io/apimachinery v0.18.6/go.mod h1:OaXp26zu/5J7p0f92ASynJa1pZo06YlV9fG7BoWbCko= 808 | k8s.io/apimachinery v0.19.0/go.mod h1:DnPGDnARWFvYa3pMHgSxtbZb7gpzzAZ1pTfaUNDVlmA= 809 | k8s.io/apimachinery v0.19.4/go.mod h1:DnPGDnARWFvYa3pMHgSxtbZb7gpzzAZ1pTfaUNDVlmA= 810 | k8s.io/apimachinery v0.20.0/go.mod h1:WlLqWAHZGg07AeltaI0MV5uk1Omp8xaN0JGLY6gkRpU= 811 | k8s.io/apimachinery v0.20.4 h1:vhxQ0PPUUU2Ns1b9r4/UFp13UPs8cw2iOoTjnY9faa0= 812 | k8s.io/apimachinery v0.20.4/go.mod h1:WlLqWAHZGg07AeltaI0MV5uk1Omp8xaN0JGLY6gkRpU= 813 | k8s.io/apiserver v0.18.6/go.mod h1:Zt2XvTHuaZjBz6EFYzpp+X4hTmgWGy8AthNVnTdm3Wg= 814 | k8s.io/client-go v0.18.6/go.mod h1:/fwtGLjYMS1MaM5oi+eXhKwG+1UHidUEXRh6cNsdO0Q= 815 | k8s.io/client-go v0.19.0/go.mod h1:H9E/VT95blcFQnlyShFgnFT9ZnJOAceiUHM3MlRC+mU= 816 | k8s.io/client-go v0.19.4/go.mod h1:ZrEy7+wj9PjH5VMBCuu/BDlvtUAku0oVFk4MmnW9mWA= 817 | k8s.io/client-go v0.20.0 h1:Xlax8PKbZsjX4gFvNtt4F5MoJ1V5prDvCuoq9B7iax0= 818 | k8s.io/client-go v0.20.0/go.mod h1:4KWh/g+Ocd8KkCwKF8vUNnmqgv+EVnQDK4MBF4oB5tY= 819 | k8s.io/code-generator v0.18.6/go.mod h1:TgNEVx9hCyPGpdtCWA34olQYLkh3ok9ar7XfSsr8b6c= 820 | k8s.io/component-base v0.18.6/go.mod h1:knSVsibPR5K6EW2XOjEHik6sdU5nCvKMrzMt2D4In14= 821 | k8s.io/component-base v0.19.0/go.mod h1:dKsY8BxkA+9dZIAh2aWJLL/UdASFDNtGYTCItL4LM7Y= 822 | k8s.io/gengo v0.0.0-20190128074634-0689ccc1d7d6/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0= 823 | k8s.io/gengo v0.0.0-20200114144118-36b2048a9120/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0= 824 | k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0= 825 | k8s.io/klog v0.0.0-20181102134211-b9b56d5dfc92/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk= 826 | k8s.io/klog v0.3.0/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk= 827 | k8s.io/klog v1.0.0 h1:Pt+yjF5aB1xDSVbau4VsWe+dQNzA0qv1LlXdC2dF6Q8= 828 | k8s.io/klog v1.0.0/go.mod h1:4Bi6QPql/J/LkTDqv7R/cd3hPo4k2DG6Ptcz060Ez5I= 829 | k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE= 830 | k8s.io/klog/v2 v2.2.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y= 831 | k8s.io/klog/v2 v2.4.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y= 832 | k8s.io/klog/v2 v2.8.0 h1:Q3gmuM9hKEjefWFFYF0Mat+YyFJvsUyYuwyNNJ5C9Ts= 833 | k8s.io/klog/v2 v2.8.0/go.mod h1:hy9LJ/NvuK+iVyP4Ehqva4HxZG/oXyIS3n3Jmire4Ec= 834 | k8s.io/kube-openapi v0.0.0-20200410145947-61e04a5be9a6/go.mod h1:GRQhZsXIAJ1xR0C9bd8UpWHZ5plfAS9fzPjJuQ6JL3E= 835 | k8s.io/kube-openapi v0.0.0-20200805222855-6aeccd4b50c6/go.mod h1:UuqjUnNftUyPE5H64/qeyjQoUZhGpeFDVdxjTeEVN2o= 836 | k8s.io/kube-openapi v0.0.0-20200923155610-8b5066479488/go.mod h1:UuqjUnNftUyPE5H64/qeyjQoUZhGpeFDVdxjTeEVN2o= 837 | k8s.io/kube-openapi v0.0.0-20201113171705-d219536bb9fd h1:sOHNzJIkytDF6qadMNKhhDRpc6ODik8lVC6nOur7B2c= 838 | k8s.io/kube-openapi v0.0.0-20201113171705-d219536bb9fd/go.mod h1:WOJ3KddDSol4tAGcJo0Tvi+dK12EcqSLqcWsryKMpfM= 839 | k8s.io/mount-utils v0.21.0 h1:Z8mCBpIBG26Q9TFg6d0Wvai6AL1mMPqSYBbNVxo6J2A= 840 | k8s.io/mount-utils v0.21.0/go.mod h1:dwXbIPxKtTjrBEaX1aK/CMEf1KZ8GzMHpe3NEBfdFXI= 841 | k8s.io/utils v0.0.0-20200324210504-a9aa75ae1b89/go.mod h1:sZAwmy6armz5eXlNoLmJcl4F1QuKu7sr+mFQ0byX7Ew= 842 | k8s.io/utils v0.0.0-20200603063816-c1c6865ac451/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= 843 | k8s.io/utils v0.0.0-20200729134348-d5654de09c73/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= 844 | k8s.io/utils v0.0.0-20201110183641-67b214c5f920/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= 845 | k8s.io/utils v0.0.0-20210111153108-fddb29f9d009 h1:0T5IaWHO3sJTEmCP6mUlBvMukxPKUQWqiI/YuiBNMiQ= 846 | k8s.io/utils v0.0.0-20210111153108-fddb29f9d009/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= 847 | rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= 848 | rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= 849 | rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= 850 | sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.7/go.mod h1:PHgbrJT7lCHcxMU+mDHEm+nx46H4zuuHZkDP6icnhu0= 851 | sigs.k8s.io/container-object-storage-interface-api v0.0.0-20210417043410-0af83d5058ab h1:mKZ+ua1nJHPYNb5NkyxFsObb0bElpCiwt8k6xeNsH1Y= 852 | sigs.k8s.io/container-object-storage-interface-api v0.0.0-20210417043410-0af83d5058ab/go.mod h1:WTzZGS4Q6MdQqDihJdMh2kCvqMx9Amhx0KIainA4lXQ= 853 | sigs.k8s.io/container-object-storage-interface-spec v0.0.0-20210329232956-3bbacbbc9c19 h1:LrLrBCBqO7O/VjJtTrDSj3/f7hLSQaCIouLZFnHGxFg= 854 | sigs.k8s.io/container-object-storage-interface-spec v0.0.0-20210329232956-3bbacbbc9c19/go.mod h1:kafkL5l/lTUrZXhVi/9p1GzpEE/ts29BkWkL3Ao33WU= 855 | sigs.k8s.io/controller-runtime v0.6.3 h1:SBbr+inLPEKhvlJtrvDcwIpm+uhDvp63Bl72xYJtoOE= 856 | sigs.k8s.io/controller-runtime v0.6.3/go.mod h1:WlZNXcM0++oyaQt4B7C2lEE5JYRs8vJUzRP4N4JpdAY= 857 | sigs.k8s.io/structured-merge-diff/v3 v3.0.0-20200116222232-67a7b8c61874/go.mod h1:PlARxl6Hbt/+BC80dRLi1qAmnMqwqDg62YvvVkZjemw= 858 | sigs.k8s.io/structured-merge-diff/v3 v3.0.0/go.mod h1:PlARxl6Hbt/+BC80dRLi1qAmnMqwqDg62YvvVkZjemw= 859 | sigs.k8s.io/structured-merge-diff/v4 v4.0.1/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw= 860 | sigs.k8s.io/structured-merge-diff/v4 v4.0.2 h1:YHQV7Dajm86OuqnIR6zAelnDWBRjo+YhYV9PmGrh1s8= 861 | sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw= 862 | sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o= 863 | sigs.k8s.io/yaml v1.2.0 h1:kr/MCeFWJWTwyaHoR9c8EjH9OumOmoF9YGiZd7lFm/Q= 864 | sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc= 865 | -------------------------------------------------------------------------------- /kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | namespace: default 4 | commonAnnotations: 5 | cosi.storage.k8s.io/authors: "Kubernetes Authors" 6 | cosi.storage.k8s.io/license: "Apache V2" 7 | cosi.storage.k8s.io/support: "https://github.com/kubernetes-sigs/container-object-storage-api" 8 | commonLabels: 9 | cosi.storage.k8s.io/version: "dev" 10 | 11 | resources: 12 | - resources/daemonset.yaml 13 | - resources/sa.yaml 14 | - resources/rbac.yaml 15 | -------------------------------------------------------------------------------- /pkg/client/fake/node_client.go: -------------------------------------------------------------------------------- 1 | package fake 2 | 3 | import ( 4 | "context" 5 | "k8s.io/client-go/tools/record" 6 | 7 | v1 "k8s.io/api/core/v1" 8 | 9 | "sigs.k8s.io/container-object-storage-interface-api/apis/objectstorage.k8s.io/v1alpha1" 10 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/client" 11 | ) 12 | 13 | var _ client.NodeClient = &FakeNodeClient{} 14 | 15 | type FakeNodeClient struct { 16 | MockGetBAR func(ctx context.Context, pod *v1.Pod, barName, barNs string) (*v1alpha1.BucketAccessRequest, error) 17 | MockGetBA func(ctx context.Context, pod *v1.Pod, baName string) (*v1alpha1.BucketAccess, error) 18 | MockGetBR func(ctx context.Context, pod *v1.Pod, brName, brNs string) (*v1alpha1.BucketRequest, error) 19 | MockGetB func(ctx context.Context, pod *v1.Pod, bName string) (*v1alpha1.Bucket, error) 20 | MockGetPod func(ctx context.Context, podName, podNs string) (*v1.Pod, error) 21 | 22 | MockGetResources func(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) 23 | 24 | MockAddBAFinalizer func(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error 25 | MockRemoveBAFinalizer func(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error 26 | } 27 | 28 | func (f FakeNodeClient) GetPod(ctx context.Context, podName, podNs string) (*v1.Pod, error) { 29 | return f.MockGetPod(ctx, podName, podNs) 30 | } 31 | 32 | var fRecorder = record.NewFakeRecorder(10) 33 | 34 | func (f FakeNodeClient) Recorder() record.EventRecorder { 35 | return fRecorder 36 | } 37 | 38 | func (f FakeNodeClient) GetBAR(ctx context.Context, pod *v1.Pod, barName, barNs string) (*v1alpha1.BucketAccessRequest, error) { 39 | return f.MockGetBAR(ctx, pod, barName, barNs) 40 | } 41 | 42 | func (f FakeNodeClient) GetBA(ctx context.Context, pod *v1.Pod, baName string) (*v1alpha1.BucketAccess, error) { 43 | return f.MockGetBA(ctx, pod, baName) 44 | } 45 | 46 | func (f FakeNodeClient) GetBR(ctx context.Context, pod *v1.Pod, brName, brNs string) (*v1alpha1.BucketRequest, error) { 47 | return f.MockGetBR(ctx, pod, brName, brNs) 48 | } 49 | 50 | func (f FakeNodeClient) GetB(ctx context.Context, pod *v1.Pod, bName string) (*v1alpha1.Bucket, error) { 51 | return f.MockGetB(ctx, pod, bName) 52 | } 53 | 54 | func (f FakeNodeClient) GetResources(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) { 55 | return f.MockGetResources(ctx, barName, podName, podNs) 56 | } 57 | 58 | func (f FakeNodeClient) AddBAFinalizer(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error { 59 | return f.MockAddBAFinalizer(ctx, ba, BAFinalizer) 60 | } 61 | 62 | func (f FakeNodeClient) RemoveBAFinalizer(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error { 63 | return f.MockRemoveBAFinalizer(ctx, ba, BAFinalizer) 64 | } 65 | -------------------------------------------------------------------------------- /pkg/client/fake/provisioner_client.go: -------------------------------------------------------------------------------- 1 | package fake 2 | 3 | import ( 4 | "os" 5 | 6 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/client" 7 | ) 8 | 9 | var _ client.ProvisionerClient = &MockProvisionerClient{} 10 | 11 | type MockProvisionerClient struct { 12 | MockMkdirAll func(path string, perm os.FileMode) error 13 | MockRemoveAll func(path string) error 14 | MockWriteFile func(data []byte, filepath string) error 15 | MockReadFile func(filename string) ([]byte, error) 16 | } 17 | 18 | func (p MockProvisionerClient) ReadFile(filename string) ([]byte, error) { 19 | return p.MockReadFile(filename) 20 | } 21 | 22 | func (p MockProvisionerClient) MkdirAll(path string, perm os.FileMode) error { 23 | return p.MockMkdirAll(path, perm) 24 | } 25 | 26 | func (p MockProvisionerClient) RemoveAll(path string) error { 27 | return p.MockRemoveAll(path) 28 | } 29 | 30 | func (p MockProvisionerClient) WriteFile(data []byte, filepath string) error { 31 | return p.MockWriteFile(data, filepath) 32 | } 33 | -------------------------------------------------------------------------------- /pkg/client/node_client.go: -------------------------------------------------------------------------------- 1 | package client 2 | 3 | import ( 4 | "context" 5 | "encoding/json" 6 | "fmt" 7 | 8 | "github.com/pkg/errors" 9 | v1 "k8s.io/api/core/v1" 10 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 11 | "k8s.io/client-go/kubernetes" 12 | "k8s.io/client-go/kubernetes/scheme" 13 | typedcorev1 "k8s.io/client-go/kubernetes/typed/core/v1" 14 | "k8s.io/client-go/rest" 15 | "k8s.io/client-go/tools/record" 16 | "k8s.io/klog/v2" 17 | "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" 18 | 19 | "sigs.k8s.io/container-object-storage-interface-api/apis/objectstorage.k8s.io/v1alpha1" 20 | cs "sigs.k8s.io/container-object-storage-interface-api/clientset/typed/objectstorage.k8s.io/v1alpha1" 21 | 22 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/util" 23 | ) 24 | 25 | const ( 26 | PodNameKey = "csi.storage.k8s.io/pod.name" 27 | PodNamespaceKey = "csi.storage.k8s.io/pod.namespace" 28 | 29 | BarNameKey = "bar-name" 30 | ) 31 | 32 | var _ NodeClient = &nodeClient{} 33 | 34 | func newRecorder(kubeClient *kubernetes.Clientset, driverName, nodeID string) record.EventRecorder { 35 | eventBroadcaster := record.NewBroadcaster() 36 | eventBroadcaster.StartLogging(klog.Infof) 37 | eventBroadcaster.StartRecordingToSink( 38 | &typedcorev1.EventSinkImpl{Interface: kubeClient.CoreV1().Events("")}) 39 | return eventBroadcaster.NewRecorder(scheme.Scheme, v1.EventSource{Component: driverName, Host: nodeID}) 40 | } 41 | 42 | type nodeClient struct { 43 | cosiClient cs.ObjectstorageV1alpha1Interface 44 | kubeClient kubernetes.Interface 45 | recorder record.EventRecorder 46 | } 47 | 48 | type NodeClient interface { 49 | GetBAR(ctx context.Context, pod *v1.Pod, barName, barNs string) (*v1alpha1.BucketAccessRequest, error) 50 | GetBA(ctx context.Context, pod *v1.Pod, baName string) (*v1alpha1.BucketAccess, error) 51 | GetBR(ctx context.Context, pod *v1.Pod, brName, brNs string) (*v1alpha1.BucketRequest, error) 52 | GetB(ctx context.Context, pod *v1.Pod, bName string) (*v1alpha1.Bucket, error) 53 | GetPod(ctx context.Context, podName, podNs string) (*v1.Pod, error) 54 | 55 | GetResources(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) 56 | 57 | AddBAFinalizer(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error 58 | RemoveBAFinalizer(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error 59 | 60 | Recorder() record.EventRecorder 61 | } 62 | 63 | func NewClientOrDie(driverName, nodeId string) NodeClient { 64 | config, err := rest.InClusterConfig() 65 | if err != nil { 66 | panic(err.Error()) 67 | } 68 | // The following function calls may panic based on the config 69 | client := cs.NewForConfigOrDie(config) 70 | kube := kubernetes.NewForConfigOrDie(config) 71 | return &nodeClient{ 72 | cosiClient: client, 73 | kubeClient: kube, 74 | recorder: newRecorder(kube, driverName, nodeId), 75 | } 76 | } 77 | 78 | func ParseVolumeContext(volCtx map[string]string) (barname, podname, podns string, err error) { 79 | klog.Info("parsing bucketAccessRequest namespace/name from volume context") 80 | 81 | if barname, err = util.ParseValue(BarNameKey, volCtx); err != nil { 82 | return 83 | } 84 | if podname, err = util.ParseValue(PodNameKey, volCtx); err != nil { 85 | return 86 | } 87 | if podns, err = util.ParseValue(PodNamespaceKey, volCtx); err != nil { 88 | return 89 | } 90 | return barname, podname, podns, nil 91 | } 92 | 93 | func (n *nodeClient) GetBAR(ctx context.Context, pod *v1.Pod, barName, barNs string) (*v1alpha1.BucketAccessRequest, error) { 94 | klog.Infof("getting bucketAccessRequest %q", fmt.Sprintf("%s/%s", barNs, barName)) 95 | bar, err := n.cosiClient.BucketAccessRequests(barNs).Get(ctx, barName, metav1.GetOptions{}) 96 | if err != nil { 97 | return nil, util.LogErr(errors.Wrap(err, util.WrapErrorGetBARFailed)) 98 | } 99 | // TODO: BAR.Spec.BucketRequestName can be unset if the BucketName is set 100 | if len(bar.Spec.BucketRequestName) == 0 { 101 | util.EmitWarningEvent(n.recorder, pod, util.BARBucketRequestNotSet) 102 | return nil, util.LogErr(util.ErrorBARUnsetBR) 103 | } 104 | if !bar.Status.AccessGranted { 105 | util.EmitWarningEvent(n.recorder, pod, util.BARAccessNotGranted) 106 | return nil, util.LogErr(util.ErrorBARNoAccess) 107 | } 108 | if len(bar.Status.BucketAccessName) == 0 { 109 | util.EmitWarningEvent(n.recorder, pod, util.BARBucketAccessNotSet) 110 | return nil, util.LogErr(util.ErrorBARUnsetBA) 111 | } 112 | return bar, nil 113 | } 114 | 115 | func (n *nodeClient) GetBA(ctx context.Context, pod *v1.Pod, baName string) (*v1alpha1.BucketAccess, error) { 116 | klog.Infof("getting bucketAccess %q", fmt.Sprintf("%s", baName)) 117 | ba, err := n.cosiClient.BucketAccesses().Get(ctx, baName, metav1.GetOptions{}) 118 | if err != nil { 119 | return nil, util.LogErr(errors.Wrap(err, util.WrapErrorGetBAFailed)) 120 | } 121 | if !ba.Status.AccessGranted { 122 | util.EmitWarningEvent(n.recorder, pod, util.BAAccessNotGranted) 123 | return nil, util.LogErr(util.ErrorBANoAccess) 124 | } 125 | if ba.Status.MintedSecret == nil { 126 | util.EmitWarningEvent(n.recorder, pod, util.BAMintedSecretNotSet) 127 | return nil, util.LogErr(util.ErrorBANoMintedSecret) 128 | } 129 | return ba, nil 130 | } 131 | 132 | func (n *nodeClient) GetBR(ctx context.Context, pod *v1.Pod, brName, brNs string) (*v1alpha1.BucketRequest, error) { 133 | klog.Infof("getting bucketRequest %q", brName) 134 | br, err := n.cosiClient.BucketRequests(brNs).Get(ctx, brName, metav1.GetOptions{}) 135 | if err != nil { 136 | return nil, util.LogErr(errors.Wrap(err, util.WrapErrorGetBRFailed)) 137 | } 138 | if !br.Status.BucketAvailable { 139 | util.EmitWarningEvent(n.recorder, pod, util.BRNotAvailable) 140 | return nil, util.LogErr(util.ErrorBRNotAvailable) 141 | } 142 | if len(br.Status.BucketName) == 0 { 143 | util.EmitWarningEvent(n.recorder, pod, util.BRBucketNameNotSet) 144 | return nil, util.LogErr(util.ErrorBRUnsetBucketName) 145 | } 146 | return br, nil 147 | } 148 | 149 | func (n *nodeClient) GetB(ctx context.Context, pod *v1.Pod, bName string) (*v1alpha1.Bucket, error) { 150 | klog.Infof("getting bucket %q", bName) 151 | // is BucketInstanceName the correct field, or should it be BucketClass 152 | bkt, err := n.cosiClient.Buckets().Get(ctx, bName, metav1.GetOptions{}) 153 | if err != nil { 154 | return nil, util.LogErr(errors.Wrap(err, util.WrapErrorGetBFailed)) 155 | } 156 | if !bkt.Status.BucketAvailable { 157 | util.EmitWarningEvent(n.recorder, pod, util.BNotAvailable) 158 | return nil, util.LogErr(util.ErrorBNotAvailable) 159 | } 160 | return bkt, nil 161 | } 162 | 163 | func (n *nodeClient) GetPod(ctx context.Context, podName, podNs string) (*v1.Pod, error) { 164 | return n.kubeClient.CoreV1().Pods(podNs).Get(ctx, podName, metav1.GetOptions{}) 165 | } 166 | 167 | func (n *nodeClient) GetResources(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) { 168 | var bar *v1alpha1.BucketAccessRequest 169 | 170 | if pod, err = n.GetPod(ctx, podName, podNs); err != nil { 171 | return 172 | } 173 | 174 | if bar, err = n.GetBAR(ctx, pod, barName, podNs); err != nil { 175 | return 176 | } 177 | 178 | if ba, err = n.GetBA(ctx, pod, bar.Status.BucketAccessName); err != nil { 179 | return 180 | } 181 | 182 | if bkt, err = n.GetB(ctx, pod, ba.Spec.BucketName); err != nil { 183 | return 184 | } 185 | 186 | if secret, err = n.kubeClient.CoreV1().Secrets(ba.Status.MintedSecret.Namespace).Get(ctx, ba.Status.MintedSecret.Name, metav1.GetOptions{}); err != nil { 187 | util.EmitWarningEvent(n.recorder, pod, util.MintedSecretNotFound) 188 | err = errors.Wrap(err, util.WrapErrorGetSecretFailed) 189 | return 190 | } 191 | util.EmitNormalEvent(n.recorder, pod, util.AllResourcesReady) 192 | return 193 | } 194 | 195 | func GetProtocol(bkt *v1alpha1.Bucket) ([]byte, error) { 196 | klog.Infof("bucket protocol %+v", bkt.Spec.Protocol) 197 | var ( 198 | data []byte 199 | err error 200 | protocolConnection interface{} 201 | ) 202 | 203 | switch { 204 | case bkt.Spec.Protocol.S3 != nil: 205 | protocolConnection = bkt.Spec.Protocol.S3 206 | case bkt.Spec.Protocol.AzureBlob != nil: 207 | protocolConnection = bkt.Spec.Protocol.AzureBlob 208 | case bkt.Spec.Protocol.GCS != nil: 209 | protocolConnection = bkt.Spec.Protocol.GCS 210 | default: 211 | err = util.ErrorInvalidProtocol 212 | } 213 | 214 | if err != nil { 215 | return nil, util.LogErr(err) 216 | } 217 | 218 | if data, err = json.Marshal(protocolConnection); err != nil { 219 | return nil, util.LogErr(errors.Wrap(err, util.WrapErrorMarshalProtocolFailed)) 220 | } 221 | return data, nil 222 | } 223 | 224 | func (n *nodeClient) AddBAFinalizer(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error { 225 | controllerutil.AddFinalizer(ba, BAFinalizer) 226 | if _, err := n.cosiClient.BucketAccesses().Update(ctx, ba, metav1.UpdateOptions{}); err != nil { 227 | return err 228 | } 229 | return nil 230 | } 231 | 232 | func (n *nodeClient) RemoveBAFinalizer(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error { 233 | controllerutil.RemoveFinalizer(ba, BAFinalizer) 234 | if _, err := n.cosiClient.BucketAccesses().Update(ctx, ba, metav1.UpdateOptions{}); err != nil { 235 | return err 236 | } 237 | return nil 238 | } 239 | 240 | func (n *nodeClient) Recorder() record.EventRecorder { 241 | return n.recorder 242 | } 243 | -------------------------------------------------------------------------------- /pkg/client/node_client_test.go: -------------------------------------------------------------------------------- 1 | package client 2 | 3 | import ( 4 | "context" 5 | "encoding/json" 6 | "fmt" 7 | "k8s.io/client-go/tools/record" 8 | "testing" 9 | 10 | "github.com/google/go-cmp/cmp" 11 | "github.com/pkg/errors" 12 | corev1 "k8s.io/api/core/v1" 13 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 14 | "k8s.io/client-go/kubernetes" 15 | k8sfake "k8s.io/client-go/kubernetes/fake" 16 | 17 | "sigs.k8s.io/container-object-storage-interface-api/apis/objectstorage.k8s.io/v1alpha1" 18 | cosifake "sigs.k8s.io/container-object-storage-interface-api/clientset/fake" 19 | cs "sigs.k8s.io/container-object-storage-interface-api/clientset/typed/objectstorage.k8s.io/v1alpha1" 20 | 21 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/util" 22 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/util/test" 23 | ) 24 | 25 | var ( 26 | ctx = context.Background() 27 | ) 28 | 29 | func TestGetBAR(t *testing.T) { 30 | type args struct { 31 | prepare func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) 32 | barName string 33 | barNs string 34 | } 35 | 36 | type want struct { 37 | bar *v1alpha1.BucketAccessRequest 38 | err error 39 | } 40 | 41 | cases := map[string]struct { 42 | args 43 | want 44 | }{ 45 | "Successful": { 46 | args: args{ 47 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 48 | _, _ = cosi.BucketAccessRequests(testutils.Namespace).Create(ctx, testutils.GetBAR(), metav1.CreateOptions{}) 49 | }, 50 | barName: "bucketAccessRequestName", 51 | barNs: testutils.Namespace, 52 | }, 53 | want: want{ 54 | bar: testutils.GetBAR(), 55 | err: nil, 56 | }, 57 | }, 58 | "NotFound": { 59 | args: args{ 60 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 61 | _, _ = cosi.BucketAccessRequests(testutils.Namespace).Create(ctx, testutils.GetBAR(), metav1.CreateOptions{}) 62 | }, 63 | barName: "wrongName", 64 | barNs: testutils.Namespace, 65 | }, 66 | want: want{ 67 | bar: nil, 68 | err: errors.Wrap(fmt.Errorf("%s \"%s\" not found", "bucketaccessrequests.objectstorage.k8s.io", "wrongName"), util.WrapErrorGetBARFailed), 69 | }, 70 | }, 71 | "FailNoAccess": { 72 | args: args{ 73 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 74 | bar := testutils.GetBAR() 75 | bar.Status.AccessGranted = false 76 | _, _ = cosi.BucketAccessRequests(testutils.Namespace).Create(ctx, bar, metav1.CreateOptions{}) 77 | }, 78 | barName: "bucketAccessRequestName", 79 | barNs: testutils.Namespace, 80 | }, 81 | want: want{ 82 | bar: nil, 83 | err: util.ErrorBARNoAccess, 84 | }, 85 | }, 86 | "FailNoBAName": { 87 | args: args{ 88 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 89 | bar := testutils.GetBAR() 90 | bar.Status.BucketAccessName = "" 91 | _, _ = cosi.BucketAccessRequests(testutils.Namespace).Create(ctx, bar, metav1.CreateOptions{}) 92 | }, 93 | barName: "bucketAccessRequestName", 94 | barNs: testutils.Namespace, 95 | }, 96 | want: want{ 97 | bar: nil, 98 | err: util.ErrorBARUnsetBA, 99 | }, 100 | }, 101 | "FailNoBRName": { 102 | args: args{ 103 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 104 | bar := testutils.GetBAR() 105 | bar.Spec.BucketRequestName = "" 106 | _, _ = cosi.BucketAccessRequests(testutils.Namespace).Create(ctx, bar, metav1.CreateOptions{}) 107 | }, 108 | barName: "bucketAccessRequestName", 109 | barNs: testutils.Namespace, 110 | }, 111 | want: want{ 112 | bar: nil, 113 | err: util.ErrorBARUnsetBR, 114 | }, 115 | }, 116 | } 117 | 118 | for name, tc := range cases { 119 | t.Run(name, func(t *testing.T) { 120 | nc := &nodeClient{ 121 | kubeClient: k8sfake.NewSimpleClientset(), 122 | cosiClient: cosifake.NewSimpleClientset().ObjectstorageV1alpha1(), 123 | recorder: record.NewFakeRecorder(10), 124 | } 125 | 126 | tc.prepare(nc.kubeClient, nc.cosiClient) 127 | 128 | bar, err := nc.GetBAR(ctx, testutils.GetPod(), tc.barName, tc.barNs) 129 | 130 | if diff := cmp.Diff(tc.want.bar, bar); diff != "" { 131 | t.Errorf("r: -want, +got:\n%s", diff) 132 | } 133 | 134 | if diff := cmp.Diff(tc.want.err, err, util.EquateErrors()); diff != "" { 135 | t.Errorf("r: -want, +got:\n%s", diff) 136 | } 137 | }) 138 | } 139 | } 140 | 141 | func TestGetBA(t *testing.T) { 142 | type args struct { 143 | prepare func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) 144 | baName string 145 | } 146 | 147 | type want struct { 148 | ba *v1alpha1.BucketAccess 149 | err error 150 | } 151 | 152 | cases := map[string]struct { 153 | args 154 | want 155 | }{ 156 | "Successful": { 157 | args: args{ 158 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 159 | _, _ = cosi.BucketAccesses().Create(ctx, testutils.GetBA(), metav1.CreateOptions{}) 160 | }, 161 | baName: "bucketAccessName", 162 | }, 163 | want: want{ 164 | ba: testutils.GetBA(), 165 | err: nil, 166 | }, 167 | }, 168 | "NotFound": { 169 | args: args{ 170 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 171 | _, _ = cosi.BucketAccesses().Create(ctx, testutils.GetBA(), metav1.CreateOptions{}) 172 | }, 173 | baName: "wrongName", 174 | }, 175 | want: want{ 176 | ba: nil, 177 | err: errors.Wrap(fmt.Errorf("%s \"%s\" not found", "bucketaccesses.objectstorage.k8s.io", "wrongName"), util.WrapErrorGetBAFailed), 178 | }, 179 | }, 180 | "FailNoAccess": { 181 | args: args{ 182 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 183 | ba := testutils.GetBA() 184 | ba.Status.AccessGranted = false 185 | _, _ = cosi.BucketAccesses().Create(ctx, ba, metav1.CreateOptions{}) 186 | }, 187 | baName: "bucketAccessName", 188 | }, 189 | want: want{ 190 | ba: nil, 191 | err: util.ErrorBANoAccess, 192 | }, 193 | }, 194 | "FailNoMintedSecretRef": { 195 | args: args{ 196 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 197 | ba := testutils.GetBA() 198 | ba.Status.MintedSecret = nil 199 | _, _ = cosi.BucketAccesses().Create(ctx, ba, metav1.CreateOptions{}) 200 | }, 201 | baName: "bucketAccessName", 202 | }, 203 | want: want{ 204 | ba: nil, 205 | err: util.ErrorBANoMintedSecret, 206 | }, 207 | }, 208 | } 209 | 210 | for name, tc := range cases { 211 | t.Run(name, func(t *testing.T) { 212 | nc := &nodeClient{ 213 | kubeClient: k8sfake.NewSimpleClientset(), 214 | cosiClient: cosifake.NewSimpleClientset().ObjectstorageV1alpha1(), 215 | recorder: record.NewFakeRecorder(10), 216 | } 217 | 218 | tc.prepare(nc.kubeClient, nc.cosiClient) 219 | 220 | ba, err := nc.GetBA(ctx, testutils.GetPod(), tc.baName) 221 | 222 | if diff := cmp.Diff(tc.want.ba, ba); diff != "" { 223 | t.Errorf("r: -want, +got:\n%s", diff) 224 | } 225 | 226 | if diff := cmp.Diff(tc.want.err, err, util.EquateErrors()); diff != "" { 227 | t.Errorf("r: -want, +got:\n%s", diff) 228 | } 229 | }) 230 | } 231 | } 232 | 233 | func TestGetBR(t *testing.T) { 234 | type args struct { 235 | prepare func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) 236 | brName string 237 | brNs string 238 | } 239 | 240 | type want struct { 241 | br *v1alpha1.BucketRequest 242 | err error 243 | } 244 | 245 | cases := map[string]struct { 246 | args 247 | want 248 | }{ 249 | "Successful": { 250 | args: args{ 251 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 252 | _, _ = cosi.BucketRequests(testutils.Namespace).Create(ctx, testutils.GetBR(), metav1.CreateOptions{}) 253 | }, 254 | brName: "bucketRequestName", 255 | brNs: testutils.Namespace, 256 | }, 257 | want: want{ 258 | br: testutils.GetBR(), 259 | err: nil, 260 | }, 261 | }, 262 | "NotFound": { 263 | args: args{ 264 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 265 | _, _ = cosi.BucketRequests(testutils.Namespace).Create(ctx, testutils.GetBR(), metav1.CreateOptions{}) 266 | }, 267 | brName: "wrongName", 268 | brNs: testutils.Namespace, 269 | }, 270 | want: want{ 271 | br: nil, 272 | err: errors.Wrap(fmt.Errorf("%s \"%s\" not found", "bucketrequests.objectstorage.k8s.io", "wrongName"), util.WrapErrorGetBRFailed), 273 | }, 274 | }, 275 | "FailNotAvailable": { 276 | args: args{ 277 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 278 | br := testutils.GetBR() 279 | br.Status.BucketAvailable = false 280 | _, _ = cosi.BucketRequests(testutils.Namespace).Create(ctx, br, metav1.CreateOptions{}) 281 | }, 282 | brName: "bucketRequestName", 283 | brNs: testutils.Namespace, 284 | }, 285 | want: want{ 286 | br: nil, 287 | err: util.ErrorBRNotAvailable, 288 | }, 289 | }, 290 | "FailNoBName": { 291 | args: args{ 292 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 293 | br := testutils.GetBR() 294 | br.Status.BucketName = "" 295 | _, _ = cosi.BucketRequests(testutils.Namespace).Create(ctx, br, metav1.CreateOptions{}) 296 | }, 297 | brName: "bucketRequestName", 298 | brNs: testutils.Namespace, 299 | }, 300 | want: want{ 301 | br: nil, 302 | err: util.ErrorBRUnsetBucketName, 303 | }, 304 | }, 305 | } 306 | 307 | for name, tc := range cases { 308 | t.Run(name, func(t *testing.T) { 309 | nc := &nodeClient{ 310 | kubeClient: k8sfake.NewSimpleClientset(), 311 | cosiClient: cosifake.NewSimpleClientset().ObjectstorageV1alpha1(), 312 | recorder: record.NewFakeRecorder(10), 313 | } 314 | 315 | tc.prepare(nc.kubeClient, nc.cosiClient) 316 | 317 | br, err := nc.GetBR(ctx, testutils.GetPod(), tc.brName, tc.brNs) 318 | 319 | if diff := cmp.Diff(tc.want.br, br); diff != "" { 320 | t.Errorf("r: -want, +got:\n%s", diff) 321 | } 322 | 323 | if diff := cmp.Diff(tc.want.err, err, util.EquateErrors()); diff != "" { 324 | t.Errorf("r: -want, +got:\n%s", diff) 325 | } 326 | }) 327 | } 328 | } 329 | 330 | func TestGetB(t *testing.T) { 331 | type args struct { 332 | prepare func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) 333 | bName string 334 | } 335 | 336 | type want struct { 337 | b *v1alpha1.Bucket 338 | err error 339 | } 340 | 341 | cases := map[string]struct { 342 | args 343 | want 344 | }{ 345 | "Successful": { 346 | args: args{ 347 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 348 | _, _ = cosi.Buckets().Create(ctx, testutils.GetB(), metav1.CreateOptions{}) 349 | }, 350 | bName: "bucketName", 351 | }, 352 | want: want{ 353 | b: testutils.GetB(), 354 | err: nil, 355 | }, 356 | }, 357 | "NotFound": { 358 | args: args{ 359 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 360 | _, _ = cosi.Buckets().Create(ctx, testutils.GetB(), metav1.CreateOptions{}) 361 | }, 362 | bName: "wrongName", 363 | }, 364 | want: want{ 365 | b: nil, 366 | err: errors.Wrap(fmt.Errorf("%s \"%s\" not found", "buckets.objectstorage.k8s.io", "wrongName"), util.WrapErrorGetBFailed), 367 | }, 368 | }, 369 | "FailNotAvailable": { 370 | args: args{ 371 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 372 | b := testutils.GetB() 373 | b.Status.BucketAvailable = false 374 | _, _ = cosi.Buckets().Create(ctx, b, metav1.CreateOptions{}) 375 | }, 376 | bName: "bucketName", 377 | }, 378 | want: want{ 379 | b: nil, 380 | err: util.ErrorBNotAvailable, 381 | }, 382 | }, 383 | } 384 | 385 | for name, tc := range cases { 386 | t.Run(name, func(t *testing.T) { 387 | nc := &nodeClient{ 388 | kubeClient: k8sfake.NewSimpleClientset(), 389 | cosiClient: cosifake.NewSimpleClientset().ObjectstorageV1alpha1(), 390 | recorder: record.NewFakeRecorder(10), 391 | } 392 | 393 | tc.prepare(nc.kubeClient, nc.cosiClient) 394 | 395 | b, err := nc.GetB(ctx, testutils.GetPod(), tc.bName) 396 | 397 | if diff := cmp.Diff(tc.want.b, b); diff != "" { 398 | t.Errorf("r: -want, +got:\n%s", diff) 399 | } 400 | 401 | if diff := cmp.Diff(tc.want.err, err, util.EquateErrors()); diff != "" { 402 | t.Errorf("r: -want, +got:\n%s", diff) 403 | } 404 | }) 405 | } 406 | } 407 | 408 | func TestGetResources(t *testing.T) { 409 | type args struct { 410 | prepare func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) 411 | barName string 412 | barNs string 413 | } 414 | 415 | type want struct { 416 | b *v1alpha1.Bucket 417 | ba *v1alpha1.BucketAccess 418 | secret *corev1.Secret 419 | err error 420 | } 421 | 422 | cases := map[string]struct { 423 | args 424 | want 425 | }{ 426 | "Successful": { 427 | args: args{ 428 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 429 | _, _ = cosi.Buckets().Create(ctx, testutils.GetB(), metav1.CreateOptions{}) 430 | _, _ = cosi.BucketAccessRequests(testutils.Namespace).Create(ctx, testutils.GetBAR(), metav1.CreateOptions{}) 431 | _, _ = cosi.BucketAccesses().Create(ctx, testutils.GetBA(), metav1.CreateOptions{}) 432 | 433 | _, _ = cs.CoreV1().Secrets(testutils.Namespace).Create(ctx, testutils.GetSecret(), metav1.CreateOptions{}) 434 | 435 | _, _ = cs.CoreV1().Pods(testutils.Namespace).Create(ctx, testutils.GetPod(), metav1.CreateOptions{}) 436 | }, 437 | barName: "bucketAccessRequestName", 438 | barNs: testutils.Namespace, 439 | }, 440 | want: want{ 441 | b: testutils.GetB(), 442 | ba: testutils.GetBA(), 443 | secret: testutils.GetSecret(), 444 | err: nil, 445 | }, 446 | }, 447 | "failedMissingBAR": { 448 | args: args{ 449 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 450 | _, _ = cosi.Buckets().Create(ctx, testutils.GetB(), metav1.CreateOptions{}) 451 | _, _ = cosi.BucketAccesses().Create(ctx, testutils.GetBA(), metav1.CreateOptions{}) 452 | 453 | _, _ = cs.CoreV1().Secrets(testutils.Namespace).Create(ctx, testutils.GetSecret(), metav1.CreateOptions{}) 454 | _, _ = cs.CoreV1().Pods(testutils.Namespace).Create(ctx, testutils.GetPod(), metav1.CreateOptions{}) 455 | }, 456 | barName: "bucketAccessRequestName", 457 | barNs: testutils.Namespace, 458 | }, 459 | want: want{ 460 | err: errors.Wrap(fmt.Errorf("%s \"%s\" not found", "bucketaccessrequests.objectstorage.k8s.io", "bucketAccessRequestName"), util.WrapErrorGetBARFailed), 461 | }, 462 | }, 463 | "failedMissingBA": { 464 | args: args{ 465 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 466 | _, _ = cosi.Buckets().Create(ctx, testutils.GetB(), metav1.CreateOptions{}) 467 | _, _ = cosi.BucketAccessRequests(testutils.Namespace).Create(ctx, testutils.GetBAR(), metav1.CreateOptions{}) 468 | 469 | _, _ = cs.CoreV1().Secrets(testutils.Namespace).Create(ctx, testutils.GetSecret(), metav1.CreateOptions{}) 470 | _, _ = cs.CoreV1().Pods(testutils.Namespace).Create(ctx, testutils.GetPod(), metav1.CreateOptions{}) 471 | }, 472 | barName: "bucketAccessRequestName", 473 | barNs: testutils.Namespace, 474 | }, 475 | want: want{ 476 | err: errors.Wrap(fmt.Errorf("%s \"%s\" not found", "bucketaccesses.objectstorage.k8s.io", "bucketAccessName"), util.WrapErrorGetBAFailed), 477 | }, 478 | }, 479 | "failedMissingB": { 480 | args: args{ 481 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 482 | _, _ = cosi.BucketAccessRequests(testutils.Namespace).Create(ctx, testutils.GetBAR(), metav1.CreateOptions{}) 483 | _, _ = cosi.BucketAccesses().Create(ctx, testutils.GetBA(), metav1.CreateOptions{}) 484 | 485 | _, _ = cs.CoreV1().Secrets(testutils.Namespace).Create(ctx, testutils.GetSecret(), metav1.CreateOptions{}) 486 | _, _ = cs.CoreV1().Pods(testutils.Namespace).Create(ctx, testutils.GetPod(), metav1.CreateOptions{}) 487 | }, 488 | barName: "bucketAccessRequestName", 489 | barNs: testutils.Namespace, 490 | }, 491 | want: want{ 492 | ba: testutils.GetBA(), 493 | err: errors.Wrap(fmt.Errorf("%s \"%s\" not found", "buckets.objectstorage.k8s.io", "bucketName"), util.WrapErrorGetBFailed), 494 | }, 495 | }, 496 | "failedMissingSecret": { 497 | args: args{ 498 | prepare: func(cs kubernetes.Interface, cosi cs.ObjectstorageV1alpha1Interface) { 499 | _, _ = cosi.Buckets().Create(ctx, testutils.GetB(), metav1.CreateOptions{}) 500 | _, _ = cosi.BucketAccessRequests(testutils.Namespace).Create(ctx, testutils.GetBAR(), metav1.CreateOptions{}) 501 | _, _ = cosi.BucketAccesses().Create(ctx, testutils.GetBA(), metav1.CreateOptions{}) 502 | _, _ = cs.CoreV1().Pods(testutils.Namespace).Create(ctx, testutils.GetPod(), metav1.CreateOptions{}) 503 | }, 504 | barName: "bucketAccessRequestName", 505 | barNs: testutils.Namespace, 506 | }, 507 | want: want{ 508 | b: testutils.GetB(), 509 | ba: testutils.GetBA(), 510 | err: errors.Wrap(fmt.Errorf("%s \"%s\" not found", "secrets", "mintedSecretName"), util.WrapErrorGetSecretFailed), 511 | }, 512 | }, 513 | } 514 | 515 | for name, tc := range cases { 516 | t.Run(name, func(t *testing.T) { 517 | nc := &nodeClient{ 518 | kubeClient: k8sfake.NewSimpleClientset(), 519 | cosiClient: cosifake.NewSimpleClientset().ObjectstorageV1alpha1(), 520 | recorder: record.NewFakeRecorder(10), 521 | } 522 | 523 | tc.prepare(nc.kubeClient, nc.cosiClient) 524 | 525 | b, ba, secret, _, err := nc.GetResources(ctx, tc.barName, "podName", testutils.Namespace) 526 | 527 | if diff := cmp.Diff(tc.want.b, b); diff != "" { 528 | t.Errorf("r: -want, +got:\n%s", diff) 529 | } 530 | 531 | if diff := cmp.Diff(tc.want.ba, ba); diff != "" { 532 | t.Errorf("r: -want, +got:\n%s", diff) 533 | } 534 | 535 | if diff := cmp.Diff(tc.want.secret, secret); diff != "" { 536 | t.Errorf("r: -want, +got:\n%s", diff) 537 | } 538 | 539 | if diff := cmp.Diff(tc.want.err, err, util.EquateErrors()); diff != "" { 540 | t.Errorf("r: -want, +got:\n%s", diff) 541 | } 542 | }) 543 | } 544 | } 545 | 546 | func TestGetProtocol(t *testing.T) { 547 | type args struct { 548 | prepare func(bkt *v1alpha1.Bucket) *v1alpha1.Bucket 549 | } 550 | 551 | type want struct { 552 | data string 553 | err error 554 | } 555 | 556 | cases := map[string]struct { 557 | args 558 | want 559 | }{ 560 | "SuccessfulS3": { 561 | args: args{ 562 | prepare: func(bkt *v1alpha1.Bucket) *v1alpha1.Bucket { 563 | bkt.Spec.Protocol = v1alpha1.Protocol{ 564 | S3: &v1alpha1.S3Protocol{ 565 | Endpoint: "endpoint", 566 | BucketName: "bucketName", 567 | Region: "region", 568 | SignatureVersion: "signatureVersion", 569 | }, 570 | } 571 | return bkt 572 | }, 573 | }, 574 | want: want{ 575 | data: `{"endpoint":"endpoint", "bucketName":"bucketName", "region":"region", "signatureVersion":"signatureVersion"}`, 576 | err: nil, 577 | }, 578 | }, 579 | "SuccessfulGCP": { 580 | args: args{ 581 | prepare: func(bkt *v1alpha1.Bucket) *v1alpha1.Bucket { 582 | bkt.Spec.Protocol = v1alpha1.Protocol{ 583 | GCS: &v1alpha1.GCSProtocol{ 584 | BucketName: "bucketName", 585 | PrivateKeyName: "privateKeyName", 586 | ProjectID: "projectID", 587 | ServiceAccount: "serviceAccount", 588 | }, 589 | } 590 | return bkt 591 | }, 592 | }, 593 | want: want{ 594 | data: `{"bucketName":"bucketName", "privateKeyName":"privateKeyName", "projectID":"projectID", "serviceAccount":"serviceAccount"}`, 595 | err: nil, 596 | }, 597 | }, 598 | "SuccessfulAzure": { 599 | args: args{ 600 | prepare: func(bkt *v1alpha1.Bucket) *v1alpha1.Bucket { 601 | bkt.Spec.Protocol = v1alpha1.Protocol{ 602 | AzureBlob: &v1alpha1.AzureProtocol{ 603 | ContainerName: "containerName", 604 | StorageAccount: "storageAccount", 605 | }, 606 | } 607 | return bkt 608 | }, 609 | }, 610 | want: want{ 611 | data: `{"containerName":"containerName", "storageAccount":"storageAccount"}`, 612 | err: nil, 613 | }, 614 | }, 615 | "FailMissingProtocol": { 616 | args: args{ 617 | prepare: func(bkt *v1alpha1.Bucket) *v1alpha1.Bucket { 618 | bkt.Spec.Protocol = v1alpha1.Protocol{} 619 | return bkt 620 | }, 621 | }, 622 | want: want{ 623 | err: util.ErrorInvalidProtocol, 624 | }, 625 | }, 626 | } 627 | 628 | for name, tc := range cases { 629 | t.Run(name, func(t *testing.T) { 630 | 631 | bkt := testutils.GetB() 632 | 633 | data, err := GetProtocol(tc.prepare(bkt)) 634 | 635 | var wantData interface{} 636 | var haveData interface{} 637 | 638 | _ = json.Unmarshal(data, &haveData) 639 | _ = json.Unmarshal([]byte(tc.data), &wantData) 640 | 641 | if diff := cmp.Diff(wantData, haveData); diff != "" { 642 | t.Errorf("r: -want, +got:\n%s", diff) 643 | } 644 | 645 | if diff := cmp.Diff(tc.want.err, err, util.EquateErrors()); diff != "" { 646 | t.Errorf("r: -want, +got:\n%s", diff) 647 | } 648 | }) 649 | } 650 | } 651 | -------------------------------------------------------------------------------- /pkg/client/provisioner_client.go: -------------------------------------------------------------------------------- 1 | package client 2 | 3 | import ( 4 | "github.com/pkg/errors" 5 | "io/ioutil" 6 | "os" 7 | 8 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/util" 9 | ) 10 | 11 | type ProvisionerClient interface { 12 | MkdirAll(path string, perm os.FileMode) error 13 | RemoveAll(path string) error 14 | WriteFile(data []byte, filepath string) error 15 | ReadFile(filename string) ([]byte, error) 16 | } 17 | 18 | func NewProvisionerClient() ProvisionerClient { 19 | return &provisionerClient{} 20 | } 21 | 22 | var _ ProvisionerClient = &provisionerClient{} 23 | 24 | type provisionerClient struct{} 25 | 26 | func (p provisionerClient) ReadFile(filename string) ([]byte, error) { 27 | return ioutil.ReadFile(filename) 28 | } 29 | 30 | func (p provisionerClient) MkdirAll(path string, perm os.FileMode) error { 31 | return os.MkdirAll(path, perm) 32 | } 33 | 34 | func (p provisionerClient) RemoveAll(path string) error { 35 | return os.RemoveAll(path) 36 | } 37 | 38 | func (p provisionerClient) WriteFile(data []byte, filepath string) error { 39 | file, err := os.OpenFile(filepath, os.O_CREATE|os.O_WRONLY|os.O_EXCL, os.FileMode(0440)) 40 | if err != nil { 41 | return util.LogErr(errors.Wrap(err, util.WrapErrorCreatingFile)) 42 | } 43 | 44 | defer file.Close() 45 | _, err = file.Write(data) 46 | if err != nil { 47 | return util.LogErr(errors.Wrap(err, util.WrapErrorWritingToFile)) 48 | } 49 | return nil 50 | } 51 | -------------------------------------------------------------------------------- /pkg/controller/controller.go: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright 2020 The Kubernetes Authors. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"); 5 | you may not use this file except in compliance with the License. 6 | You may obtain a copy of the License at 7 | 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | 10 | Unless required by applicable law or agreed to in writing, software 11 | distributed under the License is distributed on an "AS IS" BASIS, 12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | See the License for the specific language governing permissions and 14 | limitations under the License. 15 | */ 16 | 17 | package controller 18 | 19 | import ( 20 | "github.com/container-storage-interface/spec/lib/go/csi" 21 | ) 22 | 23 | func NewControllerServer() (csi.ControllerServer, error) { 24 | return &ControllerServer{}, nil 25 | } 26 | 27 | type ControllerServer struct { 28 | csi.UnimplementedControllerServer 29 | } 30 | -------------------------------------------------------------------------------- /pkg/identity/identity.go: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright 2020 The Kubernetes Authors. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"); 5 | you may not use this file except in compliance with the License. 6 | You may obtain a copy of the License at 7 | 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | 10 | Unless required by applicable law or agreed to in writing, software 11 | distributed under the License is distributed on an "AS IS" BASIS, 12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | See the License for the specific language governing permissions and 14 | limitations under the License. 15 | */ 16 | 17 | package identity 18 | 19 | import ( 20 | "context" 21 | "google.golang.org/grpc/codes" 22 | "google.golang.org/grpc/status" 23 | 24 | "github.com/container-storage-interface/spec/lib/go/csi" 25 | ) 26 | 27 | func NewIdentityServer(ident, version string, manifest map[string]string) (csi.IdentityServer, error) { 28 | return &IdentityServer{ 29 | Identity: ident, 30 | Version: version, 31 | Manifest: manifest, 32 | }, nil 33 | } 34 | 35 | type IdentityServer struct { 36 | Identity string 37 | Version string 38 | Manifest map[string]string 39 | } 40 | 41 | func (i *IdentityServer) GetPluginInfo(ctx context.Context, req *csi.GetPluginInfoRequest) (*csi.GetPluginInfoResponse, error) { 42 | if i.Identity == "" { 43 | return nil, status.Error(codes.Unavailable, "Driver name not configured") 44 | } 45 | 46 | if i.Version == "" { 47 | return nil, status.Error(codes.Unavailable, "Driver is missing version") 48 | } 49 | 50 | return &csi.GetPluginInfoResponse{ 51 | Name: i.Identity, 52 | VendorVersion: i.Version, 53 | }, nil 54 | } 55 | 56 | func (i *IdentityServer) Probe(ctx context.Context, req *csi.ProbeRequest) (*csi.ProbeResponse, error) { 57 | return &csi.ProbeResponse{}, nil 58 | } 59 | 60 | func (i *IdentityServer) GetPluginCapabilities(ctx context.Context, req *csi.GetPluginCapabilitiesRequest) (*csi.GetPluginCapabilitiesResponse, error) { 61 | return &csi.GetPluginCapabilitiesResponse{}, nil 62 | } 63 | -------------------------------------------------------------------------------- /pkg/node/node.go: -------------------------------------------------------------------------------- 1 | package node 2 | 3 | import ( 4 | "context" 5 | "encoding/json" 6 | "github.com/container-storage-interface/spec/lib/go/csi" 7 | "github.com/pkg/errors" 8 | "google.golang.org/grpc/codes" 9 | "google.golang.org/grpc/status" 10 | "k8s.io/klog/v2" 11 | "k8s.io/mount-utils" 12 | 13 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/client" 14 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/util" 15 | ) 16 | 17 | var _ csi.NodeServer = &NodeServer{} 18 | 19 | const ( 20 | credsFileName = "credentials" 21 | protocolFileName = "protocolConn.json" 22 | metadataFilename = "metadata.json" 23 | ) 24 | 25 | func NewNodeServerOrDie(driverName, nodeID, dataRoot string, volumeLimit int64) csi.NodeServer { 26 | cosiClient := client.NewClientOrDie(driverName, nodeID) 27 | return &NodeServer{ 28 | name: driverName, 29 | nodeID: nodeID, 30 | volumeLimit: volumeLimit, 31 | cosiClient: cosiClient, 32 | provisioner: NewProvisioner(dataRoot, mount.New(""), client.NewProvisionerClient()), 33 | } 34 | } 35 | 36 | // NodeServer implements the NodePublishVolume and NodeUnpublishVolume methods 37 | // of the csi.NodeServer 38 | type NodeServer struct { 39 | csi.UnimplementedNodeServer 40 | name string 41 | nodeID string 42 | volumeLimit int64 43 | cosiClient client.NodeClient 44 | provisioner Provisioner 45 | } 46 | 47 | func (n *NodeServer) NodePublishVolume(ctx context.Context, request *csi.NodePublishVolumeRequest) (*csi.NodePublishVolumeResponse, error) { 48 | klog.Infof("NodePublishVolume: volId: %v, targetPath: %v\n", request.GetVolumeId(), request.GetTargetPath()) 49 | 50 | barName, podName, podNs, err := client.ParseVolumeContext(request.GetVolumeContext()) 51 | if err != nil { 52 | return nil, status.Error(codes.InvalidArgument, err.Error()) 53 | } 54 | 55 | bkt, ba, secret, pod, err := n.cosiClient.GetResources(ctx, barName, podName, podNs) 56 | if err != nil { 57 | return nil, status.Error(codes.FailedPrecondition, err.Error()) 58 | } 59 | 60 | protocolConnection, err := client.GetProtocol(bkt) 61 | if err != nil { 62 | return nil, status.Error(codes.FailedPrecondition, err.Error()) 63 | } 64 | 65 | klog.Infof("bucket %q has protocol %q", bkt.Name, bkt.Spec.Protocol) 66 | 67 | if err := n.provisioner.createDir(request.GetVolumeId()); err != nil { 68 | return nil, status.Error(codes.Internal, err.Error()) 69 | } 70 | 71 | cleanup := func(err error, errWrap string) (*csi.NodePublishVolumeResponse, error) { 72 | rmErr := errors.Wrap(n.provisioner.removeDir(request.GetVolumeId()), util.WrapErrorFailedRemoveDirectory) 73 | if rmErr != nil { 74 | return nil, status.Error(codes.Internal, errors.Wrap(rmErr, errWrap).Error()) 75 | } 76 | return nil, status.Error(codes.Internal, errors.Wrap(err, errWrap).Error()) 77 | } 78 | 79 | creds, err := util.ParseData(secret) 80 | if err != nil { 81 | return cleanup(err, util.WrapErrorFailedToParseSecret) 82 | } 83 | 84 | if err := n.provisioner.writeFileToVolumeMount(protocolConnection, request.GetVolumeId(), protocolFileName); err != nil { 85 | return cleanup(err, util.WrapErrorFailedToWriteProtocol) 86 | } 87 | 88 | if err := n.provisioner.writeFileToVolumeMount(creds, request.GetVolumeId(), credsFileName); err != nil { 89 | return cleanup(err, util.WrapErrorFailedToWriteCredentials) 90 | } 91 | 92 | util.EmitNormalEvent(n.cosiClient.Recorder(), pod, util.CredentialsWritten) 93 | 94 | err = n.provisioner.mountDir(request.GetVolumeId(), request.GetTargetPath()) 95 | if err != nil { 96 | return cleanup(err, util.WrapErrorFailedToMountVolume) 97 | } 98 | 99 | meta := Metadata{ 100 | BaName: ba.Name, 101 | PodName: podName, 102 | PodNamespace: podNs, 103 | } 104 | 105 | err = n.cosiClient.AddBAFinalizer(ctx, ba, meta.finalizer()) 106 | if err != nil { 107 | return cleanup(err, util.WrapErrorFailedToAddFinalizer) 108 | } 109 | 110 | data, err := json.Marshal(meta) 111 | if err != nil { 112 | return cleanup(err, util.WrapErrorFailedToMarshalMetadata) 113 | } 114 | 115 | // Write the BA.name to a metadata file in our volume, this is not mounted to the app pod 116 | if err := n.provisioner.writeFileToVolume(data, request.GetVolumeId(), metadataFilename); err != nil { 117 | return cleanup(err, util.WrapErrorFailedToWriteMetadata) 118 | } 119 | 120 | util.EmitNormalEvent(n.cosiClient.Recorder(), pod, util.SuccessfullyPublishedVolume) 121 | 122 | return &csi.NodePublishVolumeResponse{}, nil 123 | } 124 | 125 | func (n *NodeServer) NodeUnpublishVolume(ctx context.Context, request *csi.NodeUnpublishVolumeRequest) (*csi.NodeUnpublishVolumeResponse, error) { 126 | klog.Infof("NodeUnpublishVolume: volId: %v, targetPath: %v\n", request.GetVolumeId(), request.GetTargetPath()) 127 | 128 | data, err := n.provisioner.readFileFromVolume(request.GetVolumeId(), metadataFilename) 129 | if err != nil { 130 | return nil, status.Error(codes.Internal, errors.Wrap(err, util.WrapErrorFailedToReadMetadataFile).Error()) 131 | } 132 | 133 | meta := Metadata{} 134 | err = json.Unmarshal(data, &meta) 135 | if err != nil { 136 | return nil, status.Error(codes.Internal, errors.Wrap(err, util.WrapErrorFailedToUnmarshalMetadata).Error()) 137 | } 138 | 139 | klog.InfoS("read metadata file", "metadata", meta) 140 | 141 | pod, err := n.cosiClient.GetPod(ctx, meta.PodName, meta.PodNamespace) 142 | if err != nil { 143 | return nil, status.Error(codes.Internal, err.Error()) 144 | } 145 | 146 | ba, err := n.cosiClient.GetBA(ctx, pod, meta.BaName) 147 | if err != nil { 148 | return nil, status.Error(codes.Internal, err.Error()) 149 | } 150 | 151 | err = n.provisioner.removeMount(request.GetTargetPath()) 152 | if err != nil { 153 | return nil, status.Error(codes.Internal, err.Error()) 154 | } 155 | 156 | err = n.provisioner.removeDir(request.GetVolumeId()) 157 | if err != nil { 158 | return nil, status.Error(codes.Internal, errors.Wrap(err, util.WrapErrorFailedToRemoveDir).Error()) 159 | } 160 | 161 | err = n.cosiClient.RemoveBAFinalizer(ctx, ba, meta.finalizer()) 162 | if err != nil { 163 | return nil, status.Error(codes.Internal, errors.Wrap(err, util.WrapErrorFailedToRemoveFinalizer).Error()) 164 | } 165 | 166 | util.EmitNormalEvent(n.cosiClient.Recorder(), pod, util.SuccessfullyUnpublishedVolume) 167 | 168 | return &csi.NodeUnpublishVolumeResponse{}, nil 169 | } 170 | 171 | func (n *NodeServer) NodeGetInfo(ctx context.Context, request *csi.NodeGetInfoRequest) (*csi.NodeGetInfoResponse, error) { 172 | resp := &csi.NodeGetInfoResponse{ 173 | NodeId: n.nodeID, 174 | MaxVolumesPerNode: n.volumeLimit, 175 | } 176 | return resp, nil 177 | } 178 | 179 | func (n *NodeServer) NodeGetCapabilities(ctx context.Context, req *csi.NodeGetCapabilitiesRequest) (*csi.NodeGetCapabilitiesResponse, error) { 180 | return &csi.NodeGetCapabilitiesResponse{}, nil 181 | } 182 | -------------------------------------------------------------------------------- /pkg/node/node_test.go: -------------------------------------------------------------------------------- 1 | package node 2 | 3 | import ( 4 | "context" 5 | "encoding/json" 6 | "fmt" 7 | "os" 8 | "path/filepath" 9 | "testing" 10 | 11 | "github.com/pkg/errors" 12 | "google.golang.org/grpc/codes" 13 | "google.golang.org/grpc/status" 14 | v1 "k8s.io/api/core/v1" 15 | "sigs.k8s.io/container-object-storage-interface-api/apis/objectstorage.k8s.io/v1alpha1" 16 | testutils "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/util/test" 17 | 18 | "github.com/container-storage-interface/spec/lib/go/csi" 19 | "github.com/google/go-cmp/cmp" 20 | "k8s.io/mount-utils" 21 | 22 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/client" 23 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/client/fake" 24 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/util" 25 | ) 26 | 27 | const ( 28 | name = "testName" 29 | nodeId = "testNodeID" 30 | volLimit = 100 31 | 32 | podName = "testPodName" 33 | 34 | provVolumeId = "volId-123456789" 35 | provTargetPath = "/var/lib/pod/secret" 36 | ) 37 | 38 | var ( 39 | ctx = context.Background() 40 | ) 41 | 42 | func genRPCError(code codes.Code, err error) error { 43 | return status.Error(code, err.Error()) 44 | } 45 | 46 | type ProvisionerModifier func(provisioner *Provisioner) 47 | 48 | func getTestProvisioner(provisionerClient *fake.MockProvisionerClient, mod ...ProvisionerModifier) Provisioner { 49 | mounter := mount.NewFakeMounter([]mount.MountPoint{}) 50 | prov := &Provisioner{ 51 | dataPath: "", 52 | mounter: mounter, 53 | pclient: provisionerClient, 54 | } 55 | for _, v := range mod { 56 | v(prov) 57 | } 58 | return *prov 59 | } 60 | 61 | func withErrorMap(errMap map[string]error) ProvisionerModifier { 62 | return func(provisioner *Provisioner) { 63 | fm := provisioner.mounter.(*mount.FakeMounter) 64 | fm.MountCheckErrors = errMap 65 | } 66 | } 67 | 68 | func withMountPoints(mps []mount.MountPoint) ProvisionerModifier { 69 | return func(provisioner *Provisioner) { 70 | fm := provisioner.mounter.(*mount.FakeMounter) 71 | fm.MountPoints = mps 72 | } 73 | } 74 | 75 | func TestNodePublishVolume(t *testing.T) { 76 | type args struct { 77 | nclient *fake.FakeNodeClient 78 | provisioner Provisioner 79 | request *csi.NodePublishVolumeRequest 80 | } 81 | 82 | type want struct { 83 | response *csi.NodePublishVolumeResponse 84 | err error 85 | } 86 | 87 | cases := map[string]struct { 88 | args 89 | want 90 | }{ 91 | "Successful": { 92 | args: args{ 93 | provisioner: getTestProvisioner( 94 | &fake.MockProvisionerClient{ 95 | MockMkdirAll: func(path string, perm os.FileMode) error { 96 | return nil 97 | }, 98 | MockWriteFile: func(data []byte, filepath string) error { 99 | return nil 100 | }, 101 | MockRemoveAll: func(path string) error { 102 | return nil 103 | }, 104 | }, 105 | ), 106 | nclient: &fake.FakeNodeClient{ 107 | MockGetResources: func(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) { 108 | tempBar := testutils.GetBAR() 109 | if tempBar.Namespace == podNs && tempBar.Name == barName { 110 | ba = testutils.GetBA() 111 | bkt = testutils.GetB() 112 | secret = testutils.GetSecret() 113 | } 114 | return bkt, ba, secret, testutils.GetPod(), nil 115 | }, 116 | MockAddBAFinalizer: func(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error { 117 | return nil 118 | }, 119 | }, 120 | request: &csi.NodePublishVolumeRequest{ 121 | VolumeContext: map[string]string{ 122 | client.BarNameKey: testutils.GetBAR().Name, 123 | client.PodNameKey: podName, 124 | client.PodNamespaceKey: testutils.Namespace, 125 | }, 126 | VolumeId: provVolumeId, 127 | TargetPath: provTargetPath, 128 | }, 129 | }, 130 | want: want{ 131 | response: &csi.NodePublishVolumeResponse{}, 132 | err: nil, 133 | }, 134 | }, 135 | "ErrorFailedToParseVolume": { 136 | args: args{ 137 | provisioner: getTestProvisioner( 138 | &fake.MockProvisionerClient{}, 139 | ), 140 | nclient: &fake.FakeNodeClient{}, 141 | request: &csi.NodePublishVolumeRequest{ 142 | VolumeContext: map[string]string{ 143 | client.PodNameKey: podName, 144 | client.PodNamespaceKey: testutils.Namespace, 145 | }, 146 | VolumeId: provVolumeId, 147 | TargetPath: provTargetPath, 148 | }, 149 | }, 150 | want: want{ 151 | response: nil, 152 | err: genRPCError(codes.InvalidArgument, fmt.Errorf(util.ErrorTemplateVolCtxUnset, client.BarNameKey)), 153 | }, 154 | }, 155 | "ErrorInvalidBucketProtocol": { 156 | args: args{ 157 | provisioner: getTestProvisioner( 158 | &fake.MockProvisionerClient{}, 159 | ), 160 | nclient: &fake.FakeNodeClient{ 161 | MockGetResources: func(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) { 162 | bkt = testutils.GetB( 163 | testutils.WithProtocol(v1alpha1.Protocol{}), 164 | ) 165 | ba = testutils.GetBA() 166 | secret = testutils.GetSecret() 167 | return 168 | }, 169 | }, 170 | request: &csi.NodePublishVolumeRequest{ 171 | VolumeContext: map[string]string{ 172 | client.BarNameKey: testutils.GetBAR().Name, 173 | 174 | client.PodNameKey: podName, 175 | client.PodNamespaceKey: testutils.Namespace, 176 | }, 177 | VolumeId: provVolumeId, 178 | TargetPath: provTargetPath, 179 | }, 180 | }, 181 | want: want{ 182 | response: nil, 183 | err: genRPCError(codes.FailedPrecondition, util.ErrorInvalidProtocol), 184 | }, 185 | }, 186 | "ErrorMkdirFailed": { 187 | args: args{ 188 | provisioner: getTestProvisioner( 189 | &fake.MockProvisionerClient{ 190 | MockMkdirAll: func(path string, perm os.FileMode) error { 191 | return errBoom 192 | }, 193 | }, 194 | ), 195 | nclient: &fake.FakeNodeClient{ 196 | MockGetResources: func(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) { 197 | bkt = testutils.GetB() 198 | ba = testutils.GetBA() 199 | secret = testutils.GetSecret() 200 | return 201 | }, 202 | }, 203 | request: &csi.NodePublishVolumeRequest{ 204 | VolumeContext: map[string]string{ 205 | client.BarNameKey: testutils.GetBAR().Name, 206 | client.PodNameKey: podName, 207 | client.PodNamespaceKey: testutils.Namespace, 208 | }, 209 | VolumeId: provVolumeId, 210 | TargetPath: provTargetPath, 211 | }, 212 | }, 213 | want: want{ 214 | response: nil, 215 | err: genRPCError(codes.Internal, errors.Wrap(errBoom, util.WrapErrorMkdirFailed)), 216 | }, 217 | }, 218 | "ErrorFailedToCreateFile": { 219 | args: args{ 220 | provisioner: getTestProvisioner( 221 | &fake.MockProvisionerClient{ 222 | MockMkdirAll: func(path string, perm os.FileMode) error { 223 | return nil 224 | }, 225 | MockWriteFile: func(data []byte, filepath string) error { 226 | return errBoom 227 | }, 228 | MockRemoveAll: func(path string) error { 229 | return nil 230 | }, 231 | }, 232 | ), 233 | nclient: &fake.FakeNodeClient{ 234 | MockGetResources: func(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) { 235 | bkt = testutils.GetB() 236 | ba = testutils.GetBA() 237 | secret = testutils.GetSecret() 238 | return 239 | }, 240 | }, 241 | request: &csi.NodePublishVolumeRequest{ 242 | VolumeContext: map[string]string{ 243 | client.BarNameKey: testutils.GetBAR().Name, 244 | client.PodNameKey: podName, 245 | client.PodNamespaceKey: testutils.Namespace, 246 | }, 247 | VolumeId: provVolumeId, 248 | TargetPath: provTargetPath, 249 | }, 250 | }, 251 | want: want{ 252 | response: nil, 253 | err: genRPCError(codes.Internal, testutils.MultipleWrap(errBoom, util.WrapErrorFailedToCreateBucketFile, util.WrapErrorFailedToWriteProtocol)), 254 | }, 255 | }, 256 | "ErrorFailedToCreateFileRmFailed": { 257 | args: args{ 258 | provisioner: getTestProvisioner( 259 | &fake.MockProvisionerClient{ 260 | MockMkdirAll: func(path string, perm os.FileMode) error { 261 | return nil 262 | }, 263 | MockWriteFile: func(data []byte, filepath string) error { 264 | return errBoom 265 | }, 266 | MockRemoveAll: func(path string) error { 267 | return errBoom 268 | }, 269 | }, 270 | ), 271 | nclient: &fake.FakeNodeClient{ 272 | MockGetResources: func(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) { 273 | bkt = testutils.GetB() 274 | ba = testutils.GetBA() 275 | secret = testutils.GetSecret() 276 | return 277 | }, 278 | }, 279 | request: &csi.NodePublishVolumeRequest{ 280 | VolumeContext: map[string]string{ 281 | client.BarNameKey: testutils.GetBAR().Name, 282 | client.PodNameKey: podName, 283 | client.PodNamespaceKey: testutils.Namespace, 284 | }, 285 | VolumeId: provVolumeId, 286 | TargetPath: provTargetPath, 287 | }, 288 | }, 289 | want: want{ 290 | response: nil, 291 | err: genRPCError(codes.Internal, testutils.MultipleWrap(errBoom, util.WrapErrorFailedRemoveDirectory, util.WrapErrorFailedToWriteProtocol)), 292 | }, 293 | }, 294 | "ErrorFailedToMountDirMkdir": { 295 | args: args{ 296 | provisioner: getTestProvisioner( 297 | &fake.MockProvisionerClient{ 298 | MockMkdirAll: func(path string, perm os.FileMode) error { 299 | if path == provTargetPath { 300 | return errBoom 301 | } 302 | return nil 303 | }, 304 | MockWriteFile: func(data []byte, filepath string) error { 305 | return nil 306 | }, 307 | MockRemoveAll: func(path string) error { 308 | return nil 309 | }, 310 | }, 311 | ), 312 | nclient: &fake.FakeNodeClient{ 313 | MockGetResources: func(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) { 314 | bkt = testutils.GetB() 315 | ba = testutils.GetBA() 316 | secret = testutils.GetSecret() 317 | return 318 | }, 319 | }, 320 | request: &csi.NodePublishVolumeRequest{ 321 | VolumeContext: map[string]string{ 322 | client.BarNameKey: testutils.GetBAR().Name, 323 | client.PodNameKey: podName, 324 | client.PodNamespaceKey: testutils.Namespace, 325 | }, 326 | VolumeId: provVolumeId, 327 | TargetPath: provTargetPath, 328 | }, 329 | }, 330 | want: want{ 331 | response: nil, 332 | err: genRPCError(codes.Internal, testutils.MultipleWrap(errBoom, util.WrapErrorFailedToMkdirForMount, util.WrapErrorFailedToMountVolume)), 333 | }, 334 | }, 335 | "ErrorFailedToMountDir": { 336 | args: args{ 337 | provisioner: getTestProvisioner( 338 | &fake.MockProvisionerClient{ 339 | MockMkdirAll: func(path string, perm os.FileMode) error { 340 | return nil 341 | }, 342 | MockWriteFile: func(data []byte, filepath string) error { 343 | return nil 344 | }, 345 | MockRemoveAll: func(path string) error { 346 | return nil 347 | }, 348 | }, withErrorMap(map[string]error{ 349 | provTargetPath: errBoom, 350 | }), 351 | ), 352 | nclient: &fake.FakeNodeClient{ 353 | MockGetResources: func(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) { 354 | bkt = testutils.GetB() 355 | ba = testutils.GetBA() 356 | secret = testutils.GetSecret() 357 | return 358 | }, 359 | }, 360 | request: &csi.NodePublishVolumeRequest{ 361 | VolumeContext: map[string]string{ 362 | client.BarNameKey: testutils.GetBAR().Name, 363 | client.PodNameKey: podName, 364 | client.PodNamespaceKey: testutils.Namespace, 365 | }, 366 | VolumeId: provVolumeId, 367 | TargetPath: provTargetPath, 368 | }, 369 | }, 370 | want: want{ 371 | response: nil, 372 | err: genRPCError(codes.Internal, errors.Wrap(errBoom, util.WrapErrorFailedToMountVolume)), 373 | }, 374 | }, 375 | "ErrorFailedToAddBAFinalizer": { 376 | args: args{ 377 | provisioner: getTestProvisioner( 378 | &fake.MockProvisionerClient{ 379 | MockMkdirAll: func(path string, perm os.FileMode) error { 380 | return nil 381 | }, 382 | MockWriteFile: func(data []byte, filepath string) error { 383 | return nil 384 | }, 385 | MockRemoveAll: func(path string) error { 386 | return nil 387 | }, 388 | }, 389 | ), 390 | nclient: &fake.FakeNodeClient{ 391 | MockGetResources: func(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) { 392 | bkt = testutils.GetB() 393 | ba = testutils.GetBA() 394 | secret = testutils.GetSecret() 395 | return 396 | }, 397 | MockAddBAFinalizer: func(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error { 398 | return errBoom 399 | }, 400 | }, 401 | request: &csi.NodePublishVolumeRequest{ 402 | VolumeContext: map[string]string{ 403 | client.BarNameKey: testutils.GetBAR().Name, 404 | client.PodNameKey: podName, 405 | client.PodNamespaceKey: testutils.Namespace, 406 | }, 407 | VolumeId: provVolumeId, 408 | TargetPath: provTargetPath, 409 | }, 410 | }, 411 | want: want{ 412 | response: nil, 413 | err: genRPCError(codes.Internal, errors.Wrap(errBoom, util.WrapErrorFailedToAddFinalizer)), 414 | }, 415 | }, 416 | "ErrorFailedToWriteMetadata": { 417 | args: args{ 418 | provisioner: getTestProvisioner( 419 | &fake.MockProvisionerClient{ 420 | MockMkdirAll: func(path string, perm os.FileMode) error { 421 | return nil 422 | }, 423 | MockWriteFile: func(data []byte, fp string) error { 424 | if filepath.Base(fp) == metadataFilename { 425 | return errBoom 426 | } 427 | return nil 428 | }, 429 | MockRemoveAll: func(path string) error { 430 | return nil 431 | }, 432 | }, 433 | ), 434 | nclient: &fake.FakeNodeClient{ 435 | MockGetResources: func(ctx context.Context, barName, podName, podNs string) (bkt *v1alpha1.Bucket, ba *v1alpha1.BucketAccess, secret *v1.Secret, pod *v1.Pod, err error) { 436 | bkt = testutils.GetB() 437 | ba = testutils.GetBA() 438 | secret = testutils.GetSecret() 439 | return 440 | }, 441 | MockAddBAFinalizer: func(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error { 442 | return nil 443 | }, 444 | }, 445 | request: &csi.NodePublishVolumeRequest{ 446 | VolumeContext: map[string]string{ 447 | client.BarNameKey: testutils.GetBAR().Name, 448 | client.PodNameKey: podName, 449 | client.PodNamespaceKey: testutils.Namespace, 450 | }, 451 | VolumeId: provVolumeId, 452 | TargetPath: provTargetPath, 453 | }, 454 | }, 455 | want: want{ 456 | response: nil, 457 | err: genRPCError(codes.Internal, testutils.MultipleWrap(errBoom, util.WrapErrorFailedToCreateVolumeFile, util.WrapErrorFailedToWriteMetadata)), 458 | }, 459 | }, 460 | } 461 | 462 | for name, tc := range cases { 463 | t.Run(name, func(t *testing.T) { 464 | ns := &NodeServer{ 465 | name: name, 466 | nodeID: nodeId, 467 | cosiClient: tc.nclient, 468 | provisioner: tc.provisioner, 469 | volumeLimit: volLimit, 470 | } 471 | 472 | response, err := ns.NodePublishVolume(ctx, tc.request) 473 | 474 | if diff := cmp.Diff(tc.want.response, response); diff != "" { 475 | t.Errorf("r: -want, +got:\n%s", diff) 476 | } 477 | 478 | if diff := cmp.Diff(tc.want.err, err, util.EquateErrors()); diff != "" { 479 | t.Errorf("r: -want, +got:\n%s", diff) 480 | } 481 | }) 482 | } 483 | } 484 | 485 | func TestNodeUnpublishVolume(t *testing.T) { 486 | type args struct { 487 | nclient *fake.FakeNodeClient 488 | provisioner Provisioner 489 | request *csi.NodeUnpublishVolumeRequest 490 | } 491 | 492 | type want struct { 493 | response *csi.NodeUnpublishVolumeResponse 494 | err error 495 | } 496 | 497 | cases := map[string]struct { 498 | args 499 | want 500 | }{ 501 | "Successful": { 502 | args: args{ 503 | provisioner: getTestProvisioner( 504 | &fake.MockProvisionerClient{ 505 | MockRemoveAll: func(path string) error { 506 | return nil 507 | }, 508 | MockReadFile: func(filename string) ([]byte, error) { 509 | meta := Metadata{ 510 | BaName: "bucketAccessName", 511 | PodName: podName, 512 | PodNamespace: testutils.Namespace, 513 | } 514 | return json.Marshal(meta) 515 | }, 516 | }, withMountPoints([]mount.MountPoint{ 517 | { 518 | Path: provTargetPath, 519 | }, 520 | }), 521 | ), 522 | nclient: &fake.FakeNodeClient{ 523 | MockGetBA: func(ctx context.Context, pod *v1.Pod, baName string) (*v1alpha1.BucketAccess, error) { 524 | tempBa := testutils.GetBA() 525 | if tempBa.Name == baName { 526 | return tempBa, nil 527 | } 528 | return nil, errBoom 529 | }, 530 | MockRemoveBAFinalizer: func(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error { 531 | return nil 532 | }, 533 | MockGetPod: func(ctx context.Context, podName, podNs string) (*v1.Pod, error) { 534 | return testutils.GetPod(), nil 535 | }, 536 | }, 537 | request: &csi.NodeUnpublishVolumeRequest{ 538 | VolumeId: provVolumeId, 539 | TargetPath: provTargetPath, 540 | }, 541 | }, 542 | want: want{ 543 | response: &csi.NodeUnpublishVolumeResponse{}, 544 | err: nil, 545 | }, 546 | }, 547 | "FailedToReadFile": { 548 | args: args{ 549 | provisioner: getTestProvisioner( 550 | &fake.MockProvisionerClient{ 551 | MockReadFile: func(filename string) ([]byte, error) { 552 | return nil, errBoom 553 | }, 554 | }, 555 | ), 556 | nclient: &fake.FakeNodeClient{}, 557 | request: &csi.NodeUnpublishVolumeRequest{ 558 | VolumeId: provVolumeId, 559 | TargetPath: provTargetPath, 560 | }, 561 | }, 562 | want: want{ 563 | response: nil, 564 | err: genRPCError(codes.Internal, errors.Wrap(errBoom, util.WrapErrorFailedToReadMetadataFile)), 565 | }, 566 | }, 567 | "FailedToUnmarshalMetadataFile": { 568 | args: args{ 569 | provisioner: getTestProvisioner( 570 | &fake.MockProvisionerClient{ 571 | MockReadFile: func(filename string) ([]byte, error) { 572 | s := "{" 573 | return []byte(s), nil 574 | }, 575 | }, 576 | ), 577 | nclient: &fake.FakeNodeClient{}, 578 | request: &csi.NodeUnpublishVolumeRequest{ 579 | VolumeId: provVolumeId, 580 | TargetPath: provTargetPath, 581 | }, 582 | }, 583 | want: want{ 584 | response: nil, 585 | err: genRPCError(codes.Internal, errors.Wrap(errors.New("unexpected end of JSON input"), util.WrapErrorFailedToUnmarshalMetadata)), 586 | }, 587 | }, 588 | "FailedToGetBA": { 589 | args: args{ 590 | provisioner: getTestProvisioner( 591 | &fake.MockProvisionerClient{ 592 | MockReadFile: func(filename string) ([]byte, error) { 593 | meta := Metadata{ 594 | BaName: "wrongName", 595 | PodName: podName, 596 | PodNamespace: testutils.Namespace, 597 | } 598 | return json.Marshal(meta) 599 | }, 600 | }, 601 | ), 602 | nclient: &fake.FakeNodeClient{ 603 | MockGetBA: func(ctx context.Context, pod *v1.Pod, baName string) (*v1alpha1.BucketAccess, error) { 604 | tempBa := testutils.GetBA() 605 | if tempBa.Name == baName { 606 | return tempBa, nil 607 | } 608 | return nil, errBoom 609 | }, 610 | MockGetPod: func(ctx context.Context, podName, podNs string) (*v1.Pod, error) { 611 | return testutils.GetPod(), nil 612 | }, 613 | }, 614 | request: &csi.NodeUnpublishVolumeRequest{ 615 | VolumeId: provVolumeId, 616 | TargetPath: provTargetPath, 617 | }, 618 | }, 619 | want: want{ 620 | response: nil, 621 | err: genRPCError(codes.Internal, errBoom), 622 | }, 623 | }, 624 | "FailedToRemoveMount": { 625 | args: args{ 626 | provisioner: getTestProvisioner( 627 | &fake.MockProvisionerClient{ 628 | MockRemoveAll: func(path string) error { 629 | return nil 630 | }, 631 | MockReadFile: func(filename string) ([]byte, error) { 632 | meta := Metadata{ 633 | BaName: "bucketAccessName", 634 | PodName: podName, 635 | PodNamespace: testutils.Namespace, 636 | } 637 | return json.Marshal(meta) 638 | }, 639 | }, withErrorMap(map[string]error{ 640 | "/var/lib": errBoom, 641 | }), 642 | ), 643 | nclient: &fake.FakeNodeClient{ 644 | MockGetBA: func(ctx context.Context, pod *v1.Pod, baName string) (*v1alpha1.BucketAccess, error) { 645 | return testutils.GetBA(), nil 646 | }, 647 | MockGetPod: func(ctx context.Context, podName, podNs string) (*v1.Pod, error) { 648 | return testutils.GetPod(), nil 649 | }, 650 | }, 651 | request: &csi.NodeUnpublishVolumeRequest{ 652 | VolumeId: provVolumeId, 653 | TargetPath: "/var/lib", 654 | }, 655 | }, 656 | want: want{ 657 | response: nil, 658 | err: genRPCError(codes.Internal, errors.Wrap(errBoom, util.WrapErrorFailedToUnmountVolume)), 659 | }, 660 | }, 661 | "FailedToRemoveDir": { 662 | args: args{ 663 | provisioner: getTestProvisioner( 664 | &fake.MockProvisionerClient{ 665 | MockRemoveAll: func(path string) error { 666 | return errBoom 667 | }, 668 | MockReadFile: func(filename string) ([]byte, error) { 669 | meta := Metadata{ 670 | BaName: "bucketAccessName", 671 | PodName: podName, 672 | PodNamespace: testutils.Namespace, 673 | } 674 | return json.Marshal(meta) 675 | }, 676 | }, 677 | ), 678 | nclient: &fake.FakeNodeClient{ 679 | MockGetBA: func(ctx context.Context, pod *v1.Pod, baName string) (*v1alpha1.BucketAccess, error) { 680 | return testutils.GetBA(), nil 681 | }, 682 | MockGetPod: func(ctx context.Context, podName, podNs string) (*v1.Pod, error) { 683 | return testutils.GetPod(), nil 684 | }, 685 | }, 686 | request: &csi.NodeUnpublishVolumeRequest{ 687 | VolumeId: provVolumeId, 688 | TargetPath: provTargetPath, 689 | }, 690 | }, 691 | want: want{ 692 | response: nil, 693 | err: genRPCError(codes.Internal, errors.Wrap(errBoom, util.WrapErrorFailedToRemoveDir)), 694 | }, 695 | }, 696 | "FailedToRemoveFinalizer": { 697 | args: args{ 698 | provisioner: getTestProvisioner( 699 | &fake.MockProvisionerClient{ 700 | MockRemoveAll: func(path string) error { 701 | return nil 702 | }, 703 | MockReadFile: func(filename string) ([]byte, error) { 704 | meta := Metadata{ 705 | BaName: "bucketAccessName", 706 | PodName: podName, 707 | PodNamespace: testutils.Namespace, 708 | } 709 | return json.Marshal(meta) 710 | }, 711 | }, 712 | ), 713 | nclient: &fake.FakeNodeClient{ 714 | MockGetBA: func(ctx context.Context, pod *v1.Pod, baName string) (*v1alpha1.BucketAccess, error) { 715 | return testutils.GetBA(), nil 716 | }, 717 | MockRemoveBAFinalizer: func(ctx context.Context, ba *v1alpha1.BucketAccess, BAFinalizer string) error { 718 | return errBoom 719 | }, 720 | MockGetPod: func(ctx context.Context, podName, podNs string) (*v1.Pod, error) { 721 | return testutils.GetPod(), nil 722 | }, 723 | }, 724 | request: &csi.NodeUnpublishVolumeRequest{ 725 | VolumeId: provVolumeId, 726 | TargetPath: provTargetPath, 727 | }, 728 | }, 729 | want: want{ 730 | response: nil, 731 | err: genRPCError(codes.Internal, errors.Wrap(errBoom, util.WrapErrorFailedToRemoveFinalizer)), 732 | }, 733 | }, 734 | } 735 | 736 | for name, tc := range cases { 737 | t.Run(name, func(t *testing.T) { 738 | ns := &NodeServer{ 739 | name: name, 740 | nodeID: nodeId, 741 | cosiClient: tc.nclient, 742 | provisioner: tc.provisioner, 743 | volumeLimit: volLimit, 744 | } 745 | 746 | response, err := ns.NodeUnpublishVolume(ctx, tc.request) 747 | 748 | if diff := cmp.Diff(tc.want.response, response); diff != "" { 749 | t.Errorf("r: -want, +got:\n%s", diff) 750 | } 751 | 752 | if diff := cmp.Diff(tc.want.err, err, util.EquateErrors()); diff != "" { 753 | t.Errorf("r: -want, +got:\n%s", diff) 754 | } 755 | }) 756 | } 757 | } 758 | -------------------------------------------------------------------------------- /pkg/node/provisioner.go: -------------------------------------------------------------------------------- 1 | package node 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | "path/filepath" 7 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/util" 8 | 9 | "github.com/pkg/errors" 10 | "k8s.io/klog/v2" 11 | "k8s.io/mount-utils" 12 | 13 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/client" 14 | ) 15 | 16 | const ( 17 | finalizer = "cosi.objectstorage.k8s.io/bucketaccess-protection" 18 | ) 19 | 20 | type Provisioner struct { 21 | dataPath string 22 | mounter mount.Interface 23 | pclient client.ProvisionerClient 24 | } 25 | 26 | func NewProvisioner(dataPath string, p mount.Interface, pc client.ProvisionerClient) Provisioner { 27 | return Provisioner{ 28 | dataPath: dataPath, 29 | mounter: p, 30 | pclient: pc, 31 | } 32 | } 33 | 34 | func (p Provisioner) volPath(volID string) string { 35 | return filepath.Join(p.dataPath, volID) 36 | } 37 | 38 | func (p Provisioner) bucketPath(volID string) string { 39 | return filepath.Join(p.dataPath, volID, "bucket") 40 | } 41 | 42 | func (p Provisioner) createDir(volID string) error { 43 | if err := p.pclient.MkdirAll(p.bucketPath(volID), 0750); err != nil { 44 | return errors.Wrap(err, util.WrapErrorMkdirFailed) 45 | } 46 | return nil 47 | } 48 | 49 | func (p Provisioner) removeDir(volID string) error { 50 | if err := p.pclient.RemoveAll(p.volPath(volID)); err != nil && !os.IsNotExist(err) { 51 | return err 52 | } 53 | return nil 54 | } 55 | 56 | func (p Provisioner) mountDir(volID, targetPath string) error { 57 | // Check if the target path is already mounted. Prevent remounting. 58 | notMnt, err := mount.IsNotMountPoint(p.mounter, targetPath) 59 | if err != nil { 60 | klog.Error(err) 61 | if os.IsNotExist(err) { 62 | if err = p.pclient.MkdirAll(targetPath, 0750); err != nil { 63 | return errors.Wrap(err, util.WrapErrorFailedToMkdirForMount) 64 | } 65 | notMnt = true 66 | } else { 67 | return err 68 | } 69 | } 70 | 71 | if !notMnt { 72 | return fmt.Errorf(util.ErrorTemplateVolumeAlreadyMounted, targetPath) 73 | } 74 | 75 | if err := p.mounter.Mount(p.bucketPath(volID), targetPath, "", []string{"bind"}); err != nil { 76 | return errors.Wrap(err, fmt.Sprintf(util.ErrorTemplateMountFailed, p.bucketPath(volID), targetPath)) 77 | } 78 | return nil 79 | } 80 | 81 | func (p Provisioner) writeFileToVolumeMount(data []byte, volID, fileName string) error { 82 | err := p.pclient.WriteFile(data, filepath.Join(p.bucketPath(volID), fileName)) 83 | if err != nil { 84 | return errors.Wrap(err, util.WrapErrorFailedToCreateBucketFile) 85 | } 86 | return nil 87 | } 88 | 89 | func (p Provisioner) writeFileToVolume(data []byte, volID, fileName string) error { 90 | err := p.pclient.WriteFile(data, filepath.Join(p.volPath(volID), fileName)) 91 | if err != nil { 92 | return errors.Wrap(err, util.WrapErrorFailedToCreateVolumeFile) 93 | } 94 | return nil 95 | } 96 | 97 | func (p Provisioner) readFileFromVolume(volID, fileName string) ([]byte, error) { 98 | return p.pclient.ReadFile(filepath.Join(p.volPath(volID), fileName)) 99 | } 100 | 101 | func (p Provisioner) removeMount(path string) error { 102 | err := mount.CleanupMountPoint(path, p.mounter, true) 103 | if err != nil && !os.IsNotExist(err) { 104 | klog.ErrorS(err, "failed to clean and unmount target path", "targetPath", path) 105 | return errors.Wrap(err, util.WrapErrorFailedToUnmountVolume) 106 | } 107 | return nil 108 | } 109 | 110 | type Metadata struct { 111 | BaName string `json:"baName"` 112 | PodName string `json:"podName"` 113 | PodNamespace string `json:"podNamespace"` 114 | } 115 | 116 | func (m Metadata) finalizer() string { 117 | return fmt.Sprintf("%s-%s-%s", finalizer, m.PodNamespace, m.PodName) 118 | } 119 | -------------------------------------------------------------------------------- /pkg/node/provisioner_test.go: -------------------------------------------------------------------------------- 1 | package node 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | "testing" 7 | 8 | "github.com/google/go-cmp/cmp" 9 | "github.com/pkg/errors" 10 | "k8s.io/mount-utils" 11 | 12 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/client" 13 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/client/fake" 14 | "sigs.k8s.io/container-object-storage-interface-csi-adapter/pkg/util" 15 | ) 16 | 17 | const ( 18 | volumeId = "vol-id1234567890" 19 | targetPath = "/data/cosi" 20 | ) 21 | 22 | var errBoom = errors.New("boom") 23 | 24 | func TestMountDir(t *testing.T) { 25 | type args struct { 26 | rclient client.ProvisionerClient 27 | volId string 28 | targetPath string 29 | mp mount.Interface 30 | } 31 | 32 | type want struct { 33 | err error 34 | } 35 | 36 | cases := map[string]struct { 37 | args 38 | want 39 | }{ 40 | "SuccessfulCreateDir": { 41 | args: args{ 42 | mp: &mount.FakeMounter{ 43 | MountPoints: []mount.MountPoint{}, 44 | }, 45 | volId: volumeId, 46 | targetPath: targetPath, 47 | rclient: &fake.MockProvisionerClient{ 48 | MockMkdirAll: func(path string, perm os.FileMode) error { 49 | return nil 50 | }, 51 | }, 52 | }, 53 | want: want{ 54 | err: nil, 55 | }, 56 | }, 57 | "SuccessfulNotMount": { 58 | args: args{ 59 | mp: &mount.FakeMounter{ 60 | MountPoints: []mount.MountPoint{}, 61 | }, 62 | volId: volumeId, 63 | targetPath: "/var/lib", 64 | rclient: &fake.MockProvisionerClient{}, 65 | }, 66 | want: want{ 67 | err: nil, 68 | }, 69 | }, 70 | "FailMountPathIssue": { 71 | args: args{ 72 | mp: &mount.FakeMounter{ 73 | MountPoints: []mount.MountPoint{}, 74 | MountCheckErrors: map[string]error{ 75 | targetPath: errBoom, 76 | }, 77 | }, 78 | volId: volumeId, 79 | targetPath: targetPath, 80 | rclient: &fake.MockProvisionerClient{}, 81 | }, 82 | want: want{ 83 | err: errBoom, 84 | }, 85 | }, 86 | "FailMountMkdirFailed": { 87 | args: args{ 88 | mp: &mount.FakeMounter{ 89 | MountPoints: []mount.MountPoint{}, 90 | }, 91 | volId: volumeId, 92 | targetPath: targetPath, 93 | rclient: &fake.MockProvisionerClient{ 94 | MockMkdirAll: func(path string, perm os.FileMode) error { 95 | return errBoom 96 | }, 97 | }, 98 | }, 99 | want: want{ 100 | err: errors.Wrap(errBoom, util.WrapErrorFailedToMkdirForMount), 101 | }, 102 | }, 103 | "FailIsAlreadyMountPath": { 104 | args: args{ 105 | mp: &mount.FakeMounter{ 106 | MountPoints: []mount.MountPoint{ 107 | { 108 | Path: "/var/lib", 109 | }, 110 | }, 111 | }, 112 | volId: volumeId, 113 | targetPath: "/var/lib", 114 | rclient: &fake.MockProvisionerClient{}, 115 | }, 116 | want: want{ 117 | err: fmt.Errorf("%s is already mounted", "/var/lib"), 118 | }, 119 | }, 120 | } 121 | 122 | for name, tc := range cases { 123 | t.Run(name, func(t *testing.T) { 124 | p := &Provisioner{ 125 | dataPath: "", 126 | mounter: tc.mp, 127 | pclient: tc.rclient, 128 | } 129 | 130 | err := p.mountDir(tc.volId, tc.targetPath) 131 | 132 | if diff := cmp.Diff(tc.want.err, err, util.EquateErrors()); diff != "" { 133 | t.Errorf("r: -want, +got:\n%s", diff) 134 | } 135 | }) 136 | } 137 | } 138 | -------------------------------------------------------------------------------- /pkg/util/errors.go: -------------------------------------------------------------------------------- 1 | package util 2 | 3 | import "errors" 4 | 5 | const ( 6 | WrapErrorGetBARFailed = "get bucketAccessRequest failed" 7 | WrapErrorGetBAFailed = "get bucketAccess failed" 8 | WrapErrorGetBRFailed = "get bucketRequest failed" 9 | WrapErrorGetBFailed = "get bucket failed" 10 | 11 | WrapErrorGetSecretFailed = "failed to get minted secret from bucketAccess" 12 | 13 | WrapErrorMarshalProtocolFailed = "failed to marshal bucket protocol" 14 | 15 | WrapErrorMkdirFailed = "failed to mkdir for bucketPath on publish" 16 | WrapErrorFailedToCreateVolumeFile = "failed to create file in ephemeral volume" 17 | WrapErrorFailedToCreateBucketFile = "failed to create file in bucket mount folder" 18 | 19 | WrapErrorFailedRemoveDirectory = "failed to remove directory after error" 20 | WrapErrorFailedToParseSecret = "failed to parse secret" 21 | WrapErrorFailedToWriteProtocol = "failed to write protocolConnection to mount volume" 22 | WrapErrorFailedToWriteCredentials = "failed to write credentials to mount volume" 23 | WrapErrorFailedToMountVolume = "failed to mount ephemeral volume to pod" 24 | 25 | WrapErrorFailedToAddFinalizer = "failed to add finalizer to bucketAccess" 26 | WrapErrorFailedToMarshalMetadata = "failed to marshal Metadata struct" 27 | WrapErrorFailedToWriteMetadata = "failed to write metadata to disk" 28 | WrapErrorFailedToMkdirForMount = "failed to mkdir when mounting bucket" 29 | 30 | WrapErrorFailedToReadMetadataFile = "failed to read metadata file from volume" 31 | WrapErrorFailedToUnmarshalMetadata = "failed unable to unmarshal metadata from volume" 32 | WrapErrorFailedToRemoveFinalizer = "failed to remove finalizer from bucketAccess" 33 | WrapErrorFailedToUnmountVolume = "failed to unmount and clean volume" 34 | WrapErrorFailedToRemoveDir = "failed to remove directory" 35 | 36 | WrapErrorCreatingFile = "error when creating file" 37 | WrapErrorWritingToFile = "error when writing file" 38 | ) 39 | 40 | var ( 41 | ErrorBARNoAccess = errors.New("bucketAccessRequest does not grant access") 42 | ErrorBARUnsetBR = errors.New("bucketAccessRequest.Spec.BucketRequestName unset") 43 | ErrorBARUnsetBA = errors.New("bucketAccessRequest.Status.BucketAccessName unset") 44 | 45 | ErrorBANoAccess = errors.New("bucketAccess does not grant access") 46 | ErrorBANoMintedSecret = errors.New("bucketAccess.Status.MintedSecretName unset") 47 | 48 | ErrorBRNotAvailable = errors.New("bucketRequest is not available yet") 49 | ErrorBRUnsetBucketName = errors.New("bucketRequest.Status.BucketInstanceName unset") 50 | 51 | ErrorBNotAvailable = errors.New("bucket is not available yet") 52 | 53 | ErrorInvalidProtocol = errors.New("unrecognized protocol, unable to extract connection data") 54 | ) 55 | 56 | var ( 57 | ErrorTemplateVolCtxUnset = "required volume context key unset: %v" 58 | ErrorTemplateVolumeAlreadyMounted = "%s is already mounted" 59 | ErrorTemplateMountFailed = "failed to mount device: %s at %s" 60 | ) 61 | -------------------------------------------------------------------------------- /pkg/util/events.go: -------------------------------------------------------------------------------- 1 | package util 2 | 3 | import ( 4 | corev1 "k8s.io/api/core/v1" 5 | "k8s.io/apimachinery/pkg/runtime" 6 | "k8s.io/client-go/tools/record" 7 | ) 8 | 9 | const ( 10 | BARNotReady = "BARNotReady" 11 | BANotReady = "BANotReady" 12 | BRNotReady = "BRNotReady" 13 | BNotReady = "BNotReady" 14 | 15 | ResourcesReady = "ResourceReady" 16 | WritingCredentials = "WritingCredentials" 17 | SuccessfulPublish = "Success" 18 | ) 19 | 20 | var ( 21 | BARBucketRequestNotSet = EventResource{ 22 | reason: BARNotReady, 23 | message: "Bucket Access Request spec field bucketRequestName is not set", 24 | } 25 | BARAccessNotGranted = EventResource{ 26 | reason: BARNotReady, 27 | message: "Bucket Access Request has not been granted access yet", 28 | } 29 | BARBucketAccessNotSet = EventResource{ 30 | reason: BARNotReady, 31 | message: "Bucket Access Request status field bucketAccessName is not set", 32 | } 33 | 34 | BAAccessNotGranted = EventResource{ 35 | reason: BANotReady, 36 | message: "Bucket Access has not been granted access yet", 37 | } 38 | BAMintedSecretNotSet = EventResource{ 39 | reason: BANotReady, 40 | message: "Bucket Access does not have reference to minted secret", 41 | } 42 | 43 | BRNotAvailable = EventResource{ 44 | reason: BRNotReady, 45 | message: "Bucket Request is not available yet", 46 | } 47 | BRBucketNameNotSet = EventResource{ 48 | reason: BRNotReady, 49 | message: "Bucket Request status field bucketName is not set", 50 | } 51 | 52 | BNotAvailable = EventResource{ 53 | reason: BNotReady, 54 | message: "Bucket is not available yet", 55 | } 56 | 57 | MintedSecretNotFound = EventResource{ 58 | reason: BANotReady, 59 | message: "Minted credentials secret not found", 60 | } 61 | ) 62 | 63 | var ( 64 | AllResourcesReady = EventResource{ 65 | reason: ResourcesReady, 66 | message: "Bucket resources already found and ready", 67 | } 68 | 69 | CredentialsWritten = EventResource{ 70 | reason: WritingCredentials, 71 | message: "All connection information written to volume mount", 72 | } 73 | 74 | SuccessfullyPublishedVolume = EventResource{ 75 | reason: SuccessfulPublish, 76 | message: "Publish credentials completed successfully", 77 | } 78 | 79 | SuccessfullyUnpublishedVolume = EventResource{ 80 | reason: SuccessfulPublish, 81 | message: "Volume successfully unpublished from pod", 82 | } 83 | ) 84 | 85 | type EventResource struct { 86 | reason string 87 | message string 88 | } 89 | 90 | func EmitWarningEvent(recorder record.EventRecorder, object runtime.Object, resource EventResource) { 91 | recorder.Event(object, corev1.EventTypeWarning, resource.reason, resource.message) 92 | } 93 | 94 | func EmitNormalEvent(recorder record.EventRecorder, object runtime.Object, resource EventResource) { 95 | recorder.Event(object, corev1.EventTypeNormal, resource.reason, resource.message) 96 | } 97 | -------------------------------------------------------------------------------- /pkg/util/test.go: -------------------------------------------------------------------------------- 1 | package util 2 | 3 | import ( 4 | "github.com/google/go-cmp/cmp" 5 | "reflect" 6 | ) 7 | 8 | func EquateErrors() cmp.Option { 9 | return cmp.Comparer(func(a, b error) bool { 10 | if a == nil || b == nil { 11 | return a == nil && b == nil 12 | } 13 | 14 | av := reflect.ValueOf(a) 15 | bv := reflect.ValueOf(b) 16 | if av.Type() != bv.Type() { 17 | return false 18 | } 19 | 20 | return a.Error() == b.Error() 21 | }) 22 | } 23 | -------------------------------------------------------------------------------- /pkg/util/test/file.go: -------------------------------------------------------------------------------- 1 | package testutils 2 | -------------------------------------------------------------------------------- /pkg/util/test/utils.go: -------------------------------------------------------------------------------- 1 | package testutils 2 | 3 | import ( 4 | "github.com/pkg/errors" 5 | corev1 "k8s.io/api/core/v1" 6 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 7 | "sigs.k8s.io/container-object-storage-interface-api/apis/objectstorage.k8s.io/v1alpha1" 8 | ) 9 | 10 | const ( 11 | Namespace = "test" 12 | ) 13 | 14 | func GetPod() *corev1.Pod { 15 | return &corev1.Pod{ 16 | ObjectMeta: metav1.ObjectMeta{ 17 | Name: "podName", 18 | Namespace: Namespace, 19 | }, 20 | } 21 | } 22 | 23 | func GetBAR() *v1alpha1.BucketAccessRequest { 24 | return &v1alpha1.BucketAccessRequest{ 25 | ObjectMeta: metav1.ObjectMeta{ 26 | Namespace: Namespace, 27 | Name: "bucketAccessRequestName", 28 | }, 29 | Spec: v1alpha1.BucketAccessRequestSpec{ 30 | BucketName: "bucketName", 31 | BucketRequestName: "bucketRequestName", 32 | BucketAccessClassName: "bucketAccessClassName", 33 | ServiceAccountName: "serviceAccountName", 34 | }, 35 | Status: v1alpha1.BucketAccessRequestStatus{ 36 | AccessGranted: true, 37 | BucketAccessName: "bucketAccessName", 38 | }, 39 | } 40 | } 41 | 42 | func GetBA() *v1alpha1.BucketAccess { 43 | return &v1alpha1.BucketAccess{ 44 | ObjectMeta: metav1.ObjectMeta{ 45 | Name: "bucketAccessName", 46 | }, 47 | Spec: v1alpha1.BucketAccessSpec{ 48 | BucketName: "bucketName", 49 | BucketAccessRequest: &corev1.ObjectReference{ 50 | Name: "bucketAccessRequest", 51 | Namespace: Namespace, 52 | }, 53 | ServiceAccount: &corev1.ObjectReference{ 54 | Name: "serviceAccount", 55 | Namespace: Namespace, 56 | }, 57 | PolicyActionsConfigMapData: "policyActionData", 58 | }, 59 | Status: v1alpha1.BucketAccessStatus{ 60 | AccessGranted: true, 61 | MintedSecret: &corev1.SecretReference{ 62 | Name: "mintedSecretName", 63 | Namespace: Namespace, 64 | }, 65 | AccountID: "accountId", 66 | }, 67 | } 68 | } 69 | 70 | func GetSecret() *corev1.Secret { 71 | return &corev1.Secret{ 72 | ObjectMeta: metav1.ObjectMeta{ 73 | Name: "mintedSecretName", 74 | Namespace: Namespace, 75 | }, 76 | Immutable: nil, 77 | Data: map[string][]byte{ 78 | "credentials": []byte("test"), 79 | }, 80 | Type: corev1.SecretTypeOpaque, 81 | } 82 | } 83 | 84 | func GetBR() *v1alpha1.BucketRequest { 85 | return &v1alpha1.BucketRequest{ 86 | ObjectMeta: metav1.ObjectMeta{ 87 | Namespace: Namespace, 88 | Name: "bucketRequestName", 89 | }, 90 | Spec: v1alpha1.BucketRequestSpec{ 91 | BucketClassName: "bucketClassName", 92 | }, 93 | Status: v1alpha1.BucketRequestStatus{ 94 | BucketAvailable: true, 95 | BucketName: "bucketName", 96 | }, 97 | } 98 | } 99 | 100 | func GetB(mod ...BktModifier) *v1alpha1.Bucket { 101 | bkt := &v1alpha1.Bucket{ 102 | ObjectMeta: metav1.ObjectMeta{ 103 | Name: "bucketName", 104 | }, 105 | Spec: v1alpha1.BucketSpec{ 106 | Provisioner: "test-provisioner", 107 | BucketClassName: "bucketClassName", 108 | BucketRequest: &corev1.ObjectReference{ 109 | Name: "bucketRequestName", 110 | Namespace: Namespace, 111 | }, 112 | Protocol: v1alpha1.Protocol{ 113 | S3: &v1alpha1.S3Protocol{ 114 | Endpoint: "endpoint", 115 | BucketName: "bucketName", 116 | Region: "region", 117 | SignatureVersion: "signatureVersion", 118 | }, 119 | }, 120 | }, 121 | Status: v1alpha1.BucketStatus{ 122 | BucketAvailable: true, 123 | BucketID: "bucketId", 124 | }, 125 | } 126 | 127 | for _, m := range mod { 128 | m(bkt) 129 | } 130 | 131 | return bkt 132 | } 133 | 134 | type BktModifier func(bkt *v1alpha1.Bucket) 135 | 136 | func WithProtocol(proto v1alpha1.Protocol) BktModifier { 137 | return func(bkt *v1alpha1.Bucket) { 138 | bkt.Spec.Protocol = proto 139 | } 140 | } 141 | 142 | func MultipleWrap(err error, wrappers ...string) error { 143 | var te error 144 | for _, v := range wrappers { 145 | te = errors.Wrap(err, v) 146 | err = te 147 | } 148 | return te 149 | } 150 | -------------------------------------------------------------------------------- /pkg/util/util.go: -------------------------------------------------------------------------------- 1 | package util 2 | 3 | import ( 4 | "encoding/json" 5 | "fmt" 6 | 7 | v1 "k8s.io/api/core/v1" 8 | "k8s.io/klog/v2" 9 | ) 10 | 11 | func ParseData(s *v1.Secret) ([]byte, error) { 12 | output := make(map[string]string) 13 | for key, value := range s.Data { 14 | output[key] = string(value) 15 | } 16 | data, err := json.Marshal(output) 17 | if err != nil { 18 | return nil, err 19 | } 20 | return data, nil 21 | } 22 | 23 | func ParseValue(key string, volCtx map[string]string) (string, error) { 24 | value, ok := volCtx[key] 25 | if !ok { 26 | return "", fmt.Errorf(ErrorTemplateVolCtxUnset, key) 27 | } 28 | return value, nil 29 | } 30 | 31 | // logErr should be called at the interface method scope, prior to returning errors to the gRPC client. 32 | func LogErr(e error) error { 33 | if e == nil { 34 | return nil 35 | } 36 | klog.Error(e) 37 | return e 38 | } 39 | -------------------------------------------------------------------------------- /resources/daemonset.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: CSIDriver 3 | metadata: 4 | name: objectstorage.k8s.io 5 | spec: 6 | volumeLifecycleModes: 7 | - Ephemeral 8 | podInfoOnMount: true 9 | attachRequired: false 10 | --- 11 | apiVersion: v1 12 | kind: Secret 13 | metadata: 14 | name: objectstorage.k8s.io 15 | data: 16 | key: none 17 | --- 18 | apiVersion: storage.k8s.io/v1 19 | kind: StorageClass 20 | metadata: 21 | name: objectstorage.k8s.io 22 | provisioner: objectstorage.k8s.io 23 | reclaimPolicy: Delete 24 | volumeBindingMode: Immediate 25 | parameters: 26 | disable.csi.storage.k8s.io/provisioner-secret-name: objectstorage.k8s.io 27 | disable.csi.storage.k8s.io/provisioner-secret-namespace: default 28 | disable.csi.storage.k8s.io/fstype: xfs 29 | fstype: xfs 30 | --- 31 | kind: DaemonSet 32 | apiVersion: apps/v1 33 | metadata: 34 | name: objectstorage-csi-adapter 35 | labels: 36 | app.kubernetes.io/part-of: cosi 37 | app.kubernetes.io/version: main 38 | app.kubernetes.io/component: csi-adapter 39 | app.kubernetes.io/name: objectstorage-csi-adapter 40 | spec: 41 | selector: 42 | matchLabels: 43 | app.kubernetes.io/part-of: cosi 44 | app.kubernetes.io/component: csi-adapter 45 | app.kubernetes.io/name: objectstorage-csi-adapter 46 | template: 47 | metadata: 48 | labels: 49 | app.kubernetes.io/part-of: cosi 50 | app.kubernetes.io/version: main 51 | app.kubernetes.io/component: csi-adapter 52 | app.kubernetes.io/name: objectstorage-csi-adapter 53 | spec: 54 | serviceAccountName: objectstorage-csi-adapter-sa 55 | volumes: 56 | - hostPath: 57 | path: /var/lib/kubelet/plugins/objectstorage.k8s.io 58 | type: DirectoryOrCreate 59 | name: socket-dir 60 | - hostPath: 61 | path: /var/lib/kubelet/plugins_registry 62 | type: Directory 63 | name: registration-dir 64 | - hostPath: 65 | path: /var/lib/cosi-data/ 66 | type: DirectoryOrCreate 67 | name: cosi-data-dir 68 | - name: mountpoint-dir 69 | hostPath: 70 | path: /var/lib/kubelet/pods 71 | type: DirectoryOrCreate 72 | containers: 73 | - name: node-driver-registrar 74 | image: registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.0.1 75 | args: 76 | - "--v=5" 77 | - "--csi-address=/csi/csi.sock" 78 | - "--kubelet-registration-path=/var/lib/kubelet/plugins/objectstorage.k8s.io/csi.sock" 79 | securityContext: 80 | # This is necessary only for systems with SELinux, where 81 | # non-privileged sidecar containers cannot access unix domain socket 82 | # created by privileged CSI driver container. 83 | privileged: true 84 | env: 85 | - name: KUBE_NODE_NAME 86 | valueFrom: 87 | fieldRef: 88 | fieldPath: spec.nodeName 89 | volumeMounts: 90 | - mountPath: /csi 91 | name: socket-dir 92 | - mountPath: /registration 93 | name: registration-dir 94 | terminationMessagePolicy: FallbackToLogsOnError 95 | terminationMessagePath: /var/logs/driver-registrar-termination-log 96 | - name: objectstorage-csi-adapter 97 | image: quay.io/containerobjectstorage/objectstorage-csi-adapter:canary 98 | args: 99 | - "--v=5" 100 | - "--identity=objectstorage.k8s.io" 101 | - "--listen=$(CSI_ENDPOINT)" 102 | - "--protocol=$(CSI_PROTO)" 103 | - "--node-id=$(KUBE_NODE_NAME)" 104 | - "--data-path=$(DATA_PATH)" 105 | - "--max-volumes=$(MAX_VOLUMES)" 106 | env: 107 | - name: CSI_ENDPOINT 108 | value: unix:///csi/csi.sock 109 | - name: CSI_PROTO 110 | value: unix 111 | - name: KUBE_NODE_NAME 112 | valueFrom: 113 | fieldRef: 114 | fieldPath: spec.nodeName 115 | - name: DATA_PATH 116 | value: /cosi-secret-dir 117 | - name: MAX_VOLUMES 118 | value: "100" 119 | imagePullPolicy: Always 120 | securityContext: 121 | # This is necessary only for systems with SELinux, where 122 | # non-privileged sidecar containers cannot access unix domain socket 123 | # created by privileged CSI driver container. 124 | privileged: true 125 | terminationMessagePolicy: FallbackToLogsOnError 126 | terminationMessagePath: /var/log/driver-termination-log 127 | ports: 128 | - containerPort: 9898 129 | name: healthz 130 | protocol: TCP 131 | livenessProbe: 132 | failureThreshold: 5 133 | httpGet: 134 | path: /healthz 135 | port: healthz 136 | initialDelaySeconds: 10 137 | timeoutSeconds: 3 138 | periodSeconds: 2 139 | volumeMounts: 140 | - mountPath: /csi 141 | name: socket-dir 142 | - mountPath: /cosi-secret-dir 143 | name: cosi-data-dir 144 | mountPropagation: Bidirectional 145 | - name: mountpoint-dir 146 | mountPath: /var/lib/kubelet/pods 147 | mountPropagation: Bidirectional 148 | - name: liveness-probe 149 | volumeMounts: 150 | - mountPath: /csi 151 | name: socket-dir 152 | image: registry.k8s.io/sig-storage/livenessprobe:v2.2.0 153 | args: 154 | - --csi-address=/csi/csi.sock 155 | - --health-port=9898 156 | terminationMessagePolicy: FallbackToLogsOnError 157 | terminationMessagePath: /var/log/driver-liveness-termination-log 158 | -------------------------------------------------------------------------------- /resources/rbac.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | kind: ClusterRole 3 | apiVersion: rbac.authorization.k8s.io/v1 4 | metadata: 5 | name: objectstorage-csi-adapter-role 6 | namespace: default 7 | labels: 8 | app.kubernetes.io/part-of: cosi 9 | app.kubernetes.io/version: main 10 | app.kubernetes.io/component: csi-adapter 11 | app.kubernetes.io/name: objectstorage-csi-adapter 12 | rules: 13 | - apiGroups: ["objectstorage.k8s.io"] 14 | resources: ["bucketrequests", "bucketaccessrequests", "buckets"] 15 | verbs: ["get", "list", "watch"] 16 | - apiGroups: [""] 17 | resources: ["events"] 18 | verbs: ["list", "watch", "create", "update", "patch"] 19 | - apiGroups: [""] 20 | resources: ["pods", "secrets"] 21 | verbs: ["get", "watch", "list"] 22 | - apiGroups: ["objectstorage.k8s.io"] 23 | resources: ["bucketaccesses"] 24 | verbs: ["get", "list", "watch", "update"] 25 | --- 26 | kind: ClusterRoleBinding 27 | apiVersion: rbac.authorization.k8s.io/v1 28 | metadata: 29 | name: objectstorage-csi-adapter 30 | namespace: default 31 | labels: 32 | app.kubernetes.io/part-of: cosi 33 | app.kubernetes.io/version: main 34 | app.kubernetes.io/component: csi-adapter 35 | app.kubernetes.io/name: objectstorage-csi-adapter 36 | subjects: 37 | - kind: ServiceAccount 38 | name: objectstorage-csi-adapter-sa 39 | namespace: default 40 | roleRef: 41 | kind: ClusterRole 42 | name: objectstorage-csi-adapter-role 43 | apiGroup: rbac.authorization.k8s.io 44 | --- 45 | kind: Role 46 | apiVersion: rbac.authorization.k8s.io/v1 47 | metadata: 48 | name: objectstorage-csi-adapter 49 | namespace: default 50 | labels: 51 | app.kubernetes.io/part-of: cosi 52 | app.kubernetes.io/version: main 53 | app.kubernetes.io/component: csi-adapter 54 | app.kubernetes.io/name: objectstorage-csi-adapter 55 | rules: 56 | - apiGroups: ["coordination.k8s.io"] 57 | resources: ["leases"] 58 | verbs: ["get", "watch", "list", "delete", "update", "create"] 59 | --- 60 | kind: RoleBinding 61 | apiVersion: rbac.authorization.k8s.io/v1 62 | metadata: 63 | name: objectstorage-csi-adapter 64 | namespace: default 65 | labels: 66 | app.kubernetes.io/part-of: cosi 67 | app.kubernetes.io/version: main 68 | app.kubernetes.io/component: csi-adapter 69 | app.kubernetes.io/name: objectstorage-csi-adapter 70 | subjects: 71 | - kind: ServiceAccount 72 | name: objectstorage-csi-adapter-sa 73 | namespace: default 74 | roleRef: 75 | kind: Role 76 | name: objectstorage-csi-adapter 77 | apiGroup: rbac.authorization.k8s.io 78 | -------------------------------------------------------------------------------- /resources/sa.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: ServiceAccount 4 | metadata: 5 | name: objectstorage-csi-adapter-sa 6 | labels: 7 | app.kubernetes.io/part-of: cosi 8 | app.kubernetes.io/version: main 9 | app.kubernetes.io/component: csi-adapter 10 | app.kubernetes.io/name: objectstorage-csi-adapter 11 | -------------------------------------------------------------------------------- /sample/bucketaccessrequest.yaml: -------------------------------------------------------------------------------- 1 | # 2 | # Note, this resource would typically be created by the end user to 3 | # create or gain access to an existing bucket with a specific user and 4 | # access policy 5 | # 6 | kind: BucketAccessRequest 7 | apiVersion: objectstorage.k8s.io/v1alpha1 8 | metadata: 9 | name: sample-bar 10 | spec: 11 | bucketAccessClassName: sample-bac 12 | bucketRequestName: sample-br 13 | status: 14 | accessGranted: true 15 | -------------------------------------------------------------------------------- /sample/bucketrequest.yaml: -------------------------------------------------------------------------------- 1 | # 2 | # Note, this resource would typically be created by the end user to 3 | # create or gain access to an existing bucket 4 | # 5 | kind: BucketRequest 6 | apiVersion: objectstorage.k8s.io/v1alpha1 7 | metadata: 8 | name: sample-br 9 | spec: 10 | bucketClassName: sample-bc 11 | bucketPrefix: test 12 | status: 13 | bucketAvailable: true 14 | -------------------------------------------------------------------------------- /sample/classes/bucketaccessclass.yaml: -------------------------------------------------------------------------------- 1 | # 2 | # Note, this resource would typically be created by the kubeadmin and 3 | # not the end user 4 | # 5 | kind: BucketAccessClass 6 | apiVersion: objectstorage.k8s.io/v1alpha1 7 | metadata: 8 | name: sample-bac 9 | policyActionsConfigMap: 10 | name: sample-cm 11 | namespace: default -------------------------------------------------------------------------------- /sample/classes/bucketclass.yaml: -------------------------------------------------------------------------------- 1 | # 2 | # Note, this resource would typically be created by the kubeadmin and 3 | # not the end user 4 | # 5 | kind: BucketClass 6 | apiVersion: objectstorage.k8s.io/v1alpha1 7 | metadata: 8 | name: sample-bc 9 | provisioner: minio.objectstorage.k8s.io 10 | isDefaultBucketClass: true 11 | allowedNamespaces: 12 | - default 13 | protocol: 14 | s3: 15 | bucketName: samplebucket 16 | endpoint: minio 17 | region: us-east-1 18 | signatureVersion: S3V4 19 | deletionPolicy: Delete 20 | -------------------------------------------------------------------------------- /sample/classes/configmap.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ConfigMap 3 | metadata: 4 | name: sample-cm 5 | data: 6 | provisioner_data: "test_data" 7 | -------------------------------------------------------------------------------- /sample/pod/pod.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Pod 3 | metadata: 4 | name: sample-pod 5 | spec: 6 | containers: 7 | - name: web 8 | image: quay.io/krishchow/cosi-app:latest 9 | imagePullPolicy: Always 10 | ports: 11 | - name: web 12 | containerPort: 80 13 | protocol: TCP 14 | volumeMounts: 15 | - name: cosi-secrets 16 | mountPath: /data/cosi 17 | env: 18 | - name: PORT 19 | value: "3001" 20 | volumes: 21 | - name: cosi-secrets 22 | csi: 23 | driver: objectstorage.k8s.io 24 | volumeAttributes: 25 | bar-name: sample-bar 26 | -------------------------------------------------------------------------------- /sample/pod/svc.yaml: -------------------------------------------------------------------------------- 1 | kind: Service 2 | apiVersion: v1 3 | metadata: 4 | name: minio 5 | spec: 6 | type: ExternalName 7 | externalName: minio.objectstorage-provisioner-ns.svc.cluster.local 8 | ports: 9 | - port: 9000 10 | --------------------------------------------------------------------------------