├── .gitignore ├── LICENSE ├── Makefile ├── README.md ├── cmd └── main.go ├── config ├── .golangci.yml ├── .hadolint.yml └── .yamllint.yml ├── deployments ├── Dockerfile.install-minicni ├── manifests │ └── minicni.yaml └── scripts │ └── install-minicni.sh ├── go.mod ├── go.sum ├── pkg ├── args │ └── args.go ├── handler │ ├── filehandler.go │ └── handler.go ├── nettool │ ├── ip.go │ ├── ip_test.go │ ├── link.go │ └── route.go └── version │ └── version.go ├── scripts ├── gobuild.sh └── lint_go.sh └── tests └── test-pods.yaml /.gitignore: -------------------------------------------------------------------------------- 1 | # Binaries for programs and plugins 2 | *.exe 3 | *.exe~ 4 | *.dll 5 | *.so 6 | *.dylib 7 | 8 | # Test binary, built with `go test -c` 9 | *.test 10 | 11 | # Output of the go coverage tool, specifically when used with LiteIDE 12 | *.out 13 | 14 | # Dependency directories (remove the comment below to include it) 15 | # vendor/ 16 | 17 | build/_output/ 18 | 19 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # Image URL to use all building/pushing image targets; 2 | # Use your own docker registry and image name for dev/test by overridding the 3 | # IMAGE_REPO, IMAGE_NAME and RELEASE_TAG environment variable. 4 | IMAGE_REPO ?= quay.io/morvencao 5 | IMAGE_NAME ?= install-minicni 6 | CNI_NAME ?= minicni 7 | VERSION ?= $(shell date +v%Y%m%d)-$(shell git describe --match=$(git rev-parse --short=8 HEAD) --tags --always --dirty) 8 | RELEASE_VERSION ?= $(shell cat ./pkg/version/version.go | grep "Version =" | awk '{ print $$3}' | tr -d '"') 9 | 10 | PWD := $(shell pwd) 11 | BASE_DIR := $(shell basename $(PWD)) 12 | 13 | # Github host to use for checking the source tree; 14 | # Override this variable ue with your own value if you're working on forked repo. 15 | GIT_HOST ?= github.com/morvencao 16 | 17 | LOCAL_OS := $(shell uname) 18 | ifeq ($(LOCAL_OS),Linux) 19 | TARGET_OS ?= linux 20 | XARGS_FLAGS="-r" 21 | else ifeq ($(LOCAL_OS),Darwin) 22 | TARGET_OS ?= darwin 23 | XARGS_FLAGS= 24 | else 25 | $(error "This system's OS $(LOCAL_OS) isn't recognized/supported") 26 | endif 27 | 28 | ARCH := $(shell uname -m) 29 | LOCAL_ARCH := "amd64" 30 | ifeq ($(ARCH),x86_64) 31 | LOCAL_ARCH="amd64" 32 | else ifeq ($(ARCH),ppc64le) 33 | LOCAL_ARCH="ppc64le" 34 | else ifeq ($(ARCH),s390x) 35 | LOCAL_ARCH="s390x" 36 | else 37 | $(error "This system's ARCH $(ARCH) isn't recognized/supported") 38 | endif 39 | 40 | TESTARGS_DEFAULT := "-v" 41 | export TESTARGS ?= $(TESTARGS_DEFAULT) 42 | 43 | FINDFILES=find . \( -path ./.git -o -path ./.github \) -prune -o -type f 44 | 45 | XARGS = xargs -0 $(XARGS_FLAGS) 46 | CLEANXARGS = xargs $(XARGS_FLAGS) 47 | 48 | all: fmt lint test build image 49 | 50 | ############################################################ 51 | # format section 52 | ############################################################ 53 | 54 | fmt: format-go 55 | 56 | format-go: 57 | @$(FINDFILES) -name '*.go' \( ! \( -name '*.gen.go' -o -name '*.pb.go' \) \) -print0 | $(XARGS) goimports -w -local $(GIT_HOST) 58 | 59 | ############################################################ 60 | # lint section 61 | ############################################################ 62 | 63 | lint: lint-go lint-scripts lint-yaml lint-dockerfiles 64 | 65 | lint-go: 66 | @$(FINDFILES) -name '*.go' \( ! \( -name '*.gen.go' -o -name '*.pb.go' \) \) -print0 | $(XARGS) ./scripts/lint_go.sh 67 | 68 | lint-scripts: 69 | @$(FINDFILES) -name '*.sh' -print0 | $(XARGS) shellcheck 70 | 71 | lint-yaml: 72 | @$(FINDFILES) \( -name '*.yml' -o -name '*.yaml' \) -print0 | $(XARGS) grep -L -e "{{" | $(CLEANXARGS) yamllint -c ./config/.yamllint.yml 73 | 74 | lint-dockerfiles: 75 | @$(FINDFILES) -name 'Dockerfile*' -print0 | $(XARGS) hadolint -c ./config/.hadolint.yml 76 | 77 | ############################################################ 78 | # test section 79 | ############################################################ 80 | 81 | test: 82 | @echo "Running the tests for the $(CNI_NAME)..." 83 | @go test $(TESTARGS) ./... 84 | 85 | ############################################################ 86 | # build section 87 | ############################################################ 88 | 89 | build: 90 | @echo "Building the $(CNI_NAME) on $(LOCAL_ARCH)..." 91 | @GOARCH=$(LOCAL_ARCH) ./scripts/gobuild.sh build/_output/bin/$(CNI_NAME) ./cmd 92 | 93 | ############################################################ 94 | # image section 95 | ############################################################ 96 | 97 | image: build-image push-image 98 | 99 | build-image: build 100 | @echo "Building the $(IMAGE_NAME) docker image..." 101 | @docker build -t $(IMAGE_REPO)/$(IMAGE_NAME):$(RELEASE_VERSION) -f deployments/Dockerfile.install-minicni . 102 | 103 | push-image: build-image 104 | @echo "Pushing the $(IMAGE_NAME) docker image..." 105 | @docker push $(IMAGE_REPO)/$(IMAGE_NAME):$(RELEASE_VERSION) 106 | 107 | ############################################################ 108 | # clean section 109 | ############################################################ 110 | clean: 111 | @rm -rf build/_output 112 | 113 | .PHONY: all fmt lint test build image clean 114 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # minicni 2 | 3 | [![Go Report Card](https://goreportcard.com/badge/github.com/morvencao/minicni)](https://goreportcard.com/report/github.com/morvencao/minicni) 4 | [![Go Reference](https://pkg.go.dev/badge/github.com/morvencao/minicni.svg)](https://pkg.go.dev/github.com/morvencao/minicni) 5 | 6 | A simple CNI plugin implementation for kubernetes written in golang. Read [使用 Go 从零开始实现 CNI](https://morven.life/posts/write_your_own_cni_with_golang/) for more information. 7 | 8 | ## TL;DR 9 | 10 | Read the following articles about container and kubernetes network: 11 | 12 | - [容器网络(一)](https://morven.life/posts/networking-4-docker-sigle-host/) 13 | - [容器网络(二)](https://morven.life/posts/networking-5-docker-multi-hosts/) 14 | - [浅聊 Kubernetes 网络模型](https://morven.life/posts/networking-6-k8s-summary/) 15 | - [Container Network Interface Specification](https://github.com/containernetworking/cni/blob/master/SPEC.md) 16 | 17 | This repo is responsible for implementing a Kubernetes overlay network, as well as for allocating and configuring network interfaces in pods. With minicni plugin installed into a kubernetes cluster, it should be able to achieve the following targets: 18 | 19 | - All the podd can communicate with each other directly without NAT. 20 | - All the nodes can communicate with all pods (and vice versa) without NAT. 21 | - The IP that a pod sees itself as is the same IP that others see it as. 22 | 23 | ## Prerequisites 24 | 25 | A running kubernetes cluster without any CNI plugins installed. There are many kubernetes installer tools, but [kubeadm](https://kubernetes.io/docs/reference/setup-tools/kubeadm/) is the most flexible one as it allows the use of your own network plug-in. Read the official doc for how to [Creating a cluster with kubeadm](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/) 26 | 27 | ## Build and Test 28 | 29 | 1. build the minicni binary: 30 | 31 | ``` 32 | make build 33 | ``` 34 | 35 | 2. build and push the minicni installer image: 36 | 37 | ``` 38 | IMAGE_REPO= IMAGE_NAME = install-minicni make image 39 | ``` 40 | 41 | > Note: Login your docker registry before pushing the minicni installer image. 42 | 43 | 3. Deploy the minicni installer into your kubernetes cluster: 44 | 45 | ``` 46 | kubectl apply -f deployments/manifests/minicni.yaml 47 | ``` 48 | 49 | 4. Verify the minicni is installed successfully: 50 | 51 | ``` 52 | # kubectl -n kube-system get pod -l app=minicni 53 | NAME READY STATUS RESTARTS AGE 54 | minicni-node-7dmsw 1/1 Running 0 38m 55 | minicni-node-87c45 1/1 Running 0 38m 56 | ``` 57 | 58 | 5. Deploy the test pods with networking debug tools into your kubernetes cluster: 59 | 60 | ``` 61 | kubectl apply -f tests/test-pods.yaml 62 | ``` 63 | 64 | > Note: Make sure to label the master and worker node so that the testing pods can be scheduled to correct node. 65 | 66 | 6. Verify the networking connections: 67 | 68 | - pod to host node 69 | - pod to other nodes 70 | - pod to pod in the same node 71 | - pod to pod across nodes 72 | 73 | ## Known issues: 74 | 75 | 1. By default pod-to-pod traffic is drop by the linux kernel because linux treats interfaces in non-root network namespaces as if they were external, see discussion [here](https://serverfault.com/questions/162366/iptables-bridge-and-forward-chain) To workaround this, we need to manually to add the following iptables rules in each cluster node: 76 | 77 | ``` 78 | iptables -t filter -A FORWARD -s -j ACCEPT 79 | iptables -t filter -A FORWARD -d -j ACCEPT 80 | ``` 81 | 82 | 2. For pod-to-pod communications across nodes, we need to add host gateway routes just like what [Calico](https://docs.projectcalico.org/networking/openstack/host-routes) does, the feature will be added in the future. For now, we have to manually to add the following route rules in each node: 83 | 84 | ``` 85 | ip route add 172.18.1.0/24 via 10.11.97.173 dev ens4 # run on master 86 | ip route add 172.18.0.0/24 via 10.11.97.64 dev ens4 # run on worker 87 | ``` 88 | 89 | > Note: In the command above, we have one master and one worker node, `172.18.1.0/24` is subnet for the worker node, `10.11.97.173` is the IPv4 address of the worker node; `172.18.0.0/24` is the subnet for the master node, `10.11.97.64` is the IPv4 address of the master node. 90 | 91 | -------------------------------------------------------------------------------- /cmd/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | "runtime" 7 | 8 | "github.com/morvencao/minicni/pkg/args" 9 | "github.com/morvencao/minicni/pkg/handler" 10 | ) 11 | 12 | const ( 13 | IPStore = "/tmp/reserved_ips" 14 | ) 15 | 16 | func init() { 17 | // this ensures that main runs only on main thread (thread group leader). 18 | // since namespace ops (unshare, setns) are done for a single thread, we 19 | // must ensure that the goroutine does not jump from OS thread to thread 20 | runtime.LockOSThread() 21 | } 22 | 23 | func main() { 24 | cmd, cmdArgs, err := args.GetArgsFromEnv() 25 | if err != nil { 26 | fmt.Fprintf(os.Stderr, "getting cmd arguments with error: %v", err) 27 | } 28 | 29 | fh := handler.NewFileHandler(IPStore) 30 | 31 | switch cmd { 32 | case "ADD": 33 | err = fh.HandleAdd(cmdArgs) 34 | case "DEL": 35 | err = fh.HandleDel(cmdArgs) 36 | case "CHECK": 37 | err = fh.HandleCheck(cmdArgs) 38 | case "VERSION": 39 | err = fh.HandleVersion(cmdArgs) 40 | default: 41 | err = fmt.Errorf("unknown CNI_COMMAND: %s", cmd) 42 | } 43 | if err != nil { 44 | fmt.Fprintf(os.Stderr, "Failed to handle CNI_COMMAND %q: %v", cmd, err) 45 | os.Exit(1) 46 | } 47 | } 48 | -------------------------------------------------------------------------------- /config/.golangci.yml: -------------------------------------------------------------------------------- 1 | service: 2 | # When updating this, also update the version stored in docker/build-tools/Dockerfile in the multicloudlab/tools repo. 3 | golangci-lint-version: 1.18.x # use the fixed version to not introduce new linters unexpectedly 4 | run: 5 | # timeout for analysis, e.g. 30s, 5m, default is 1m 6 | deadline: 20m 7 | 8 | # which dirs to skip: they won't be analyzed; 9 | # can use regexp here: generated.*, regexp is applied on full path; 10 | # default value is empty list, but next dirs are always skipped independently 11 | # from this option's value: 12 | # vendor$, third_party$, testdata$, examples$, Godeps$, builtin$ 13 | skip-dirs: 14 | - genfiles$ 15 | - vendor$ 16 | 17 | # which files to skip: they will be analyzed, but issues from them 18 | # won't be reported. Default value is empty list, but there is 19 | # no need to include all autogenerated files, we confidently recognize 20 | # autogenerated files. If it's not please let us know. 21 | skip-files: 22 | - ".*\\.pb\\.go" 23 | - ".*\\.gen\\.go" 24 | 25 | linters: 26 | # please, do not use `enable-all`: it's deprecated and will be removed soon. 27 | # inverted configuration with `enable-all` and `disable` is not scalable during updates of golangci-lint 28 | disable-all: true 29 | enable: 30 | - deadcode 31 | - errcheck 32 | - goconst 33 | - gocritic 34 | - gocyclo 35 | - gofmt 36 | - goimports 37 | - golint 38 | - gosec 39 | - gosimple 40 | - govet 41 | - ineffassign 42 | - interfacer 43 | - lll 44 | - misspell 45 | - staticcheck 46 | - structcheck 47 | - stylecheck 48 | - typecheck 49 | - unconvert 50 | - unparam 51 | - unused 52 | - varcheck 53 | # don't enable: 54 | # - bodyclose 55 | # - depguard 56 | # - dogsled 57 | # - dupl 58 | # - funlen 59 | # - gochecknoglobals 60 | # - gochecknoinits 61 | # - gocognit 62 | # - godox 63 | # - maligned 64 | # - nakedret 65 | # - prealloc 66 | # - scopelint 67 | # - whitespace 68 | 69 | linters-settings: 70 | errcheck: 71 | # report about not checking of errors in type assetions: `a := b.(MyStruct)`; 72 | # default is false: such cases aren't reported by default. 73 | check-type-assertions: false 74 | 75 | # report about assignment of errors to blank identifier: `num, _ := strconv.Atoi(numStr)`; 76 | # default is false: such cases aren't reported by default. 77 | check-blank: false 78 | govet: 79 | # report about shadowed variables 80 | check-shadowing: false 81 | golint: 82 | # minimal confidence for issues, default is 0.8 83 | min-confidence: 0.0 84 | gofmt: 85 | # simplify code: gofmt with `-s` option, true by default 86 | simplify: true 87 | goimports: 88 | # put imports beginning with prefix after 3rd-party packages; 89 | # it's a comma-separated list of prefixes 90 | local-prefixes: github.com/IBM/ 91 | maligned: 92 | # print struct with more effective memory layout or not, false by default 93 | suggest-new: true 94 | misspell: 95 | # Correct spellings using locale preferences for US or UK. 96 | # Default is to use a neutral variety of English. 97 | # Setting locale to US will correct the British spelling of 'colour' to 'color'. 98 | locale: US 99 | ignore-words: 100 | - cancelled 101 | lll: 102 | # max line length, lines longer will be reported. Default is 120. 103 | # '\t' is counted as 1 character by default, and can be changed with the tab-width option 104 | line-length: 160 105 | # tab width in spaces. Default to 1. 106 | tab-width: 1 107 | unused: 108 | # treat code as a program (not a library) and report unused exported identifiers; default is false. 109 | # XXX: if you enable this setting, unused will report a lot of false-positives in text editors: 110 | # if it's called for subdir of a project it can't find funcs usages. All text editor integrations 111 | # with golangci-lint call it on a directory with the changed file. 112 | check-exported: false 113 | unparam: 114 | # call graph construction algorithm (cha, rta). In general, use cha for libraries, 115 | # and rta for programs with main packages. Default is cha. 116 | algo: cha 117 | 118 | # Inspect exported functions, default is false. Set to true if no external program/library imports your code. 119 | # XXX: if you enable this setting, unparam will report a lot of false-positives in text editors: 120 | # if it's called for subdir of a project it can't find external interfaces. All text editor integrations 121 | # with golangci-lint call it on a directory with the changed file. 122 | check-exported: false 123 | gocritic: 124 | enabled-checks: 125 | - appendCombine 126 | - argOrder 127 | - assignOp 128 | - badCond 129 | - boolExprSimplify 130 | - builtinShadow 131 | - captLocal 132 | - caseOrder 133 | - codegenComment 134 | - commentedOutCode 135 | - commentedOutImport 136 | - defaultCaseOrder 137 | - deprecatedComment 138 | - docStub 139 | - dupArg 140 | - dupBranchBody 141 | - dupCase 142 | - dupSubExpr 143 | - elseif 144 | - emptyFallthrough 145 | - equalFold 146 | - flagDeref 147 | - flagName 148 | - hexLiteral 149 | - indexAlloc 150 | - initClause 151 | - methodExprCall 152 | - nilValReturn 153 | - octalLiteral 154 | - offBy1 155 | - rangeExprCopy 156 | - regexpMust 157 | - sloppyLen 158 | - stringXbytes 159 | - switchTrue 160 | - typeAssertChain 161 | - typeSwitchVar 162 | - typeUnparen 163 | - underef 164 | - unlambda 165 | - unnecessaryBlock 166 | - unslice 167 | - valSwap 168 | - weakCond 169 | 170 | # Unused 171 | # - yodaStyleExpr 172 | # - appendAssign 173 | # - commentFormatting 174 | # - emptyStringTest 175 | # - exitAfterDefer 176 | # - ifElseChain 177 | # - hugeParam 178 | # - importShadow 179 | # - nestingReduce 180 | # - paramTypeCombine 181 | # - ptrToRefParam 182 | # - rangeValCopy 183 | # - singleCaseSwitch 184 | # - sloppyReassign 185 | # - unlabelStmt 186 | # - unnamedResult 187 | # - wrapperFunc 188 | 189 | issues: 190 | # List of regexps of issue texts to exclude, empty list by default. 191 | # But independently from this option we use default exclude patterns, 192 | # it can be disabled by `exclude-use-default: false`. To list all 193 | # excluded by default patterns execute `golangci-lint run --help` 194 | exclude: 195 | - composite literal uses unkeyed fields 196 | 197 | exclude-rules: 198 | # Exclude some linters from running on test files. 199 | - path: _test\.go$|^tests/|^samples/ 200 | linters: 201 | - errcheck 202 | - maligned 203 | 204 | # Independently from option `exclude` we use default exclude patterns, 205 | # it can be disabled by this option. To list all 206 | # excluded by default patterns execute `golangci-lint run --help`. 207 | # Default value for this option is true. 208 | exclude-use-default: true 209 | 210 | # Maximum issues count per one linter. Set to 0 to disable. Default is 50. 211 | max-per-linter: 0 212 | 213 | # Maximum count of issues with the same text. Set to 0 to disable. Default is 3. 214 | max-same-issues: 0 215 | -------------------------------------------------------------------------------- /config/.hadolint.yml: -------------------------------------------------------------------------------- 1 | ignored: 2 | - FAKE_DL3003 3 | - DL3006 4 | - DL3007 5 | - DL3018 6 | 7 | trustedRegistries: 8 | - docker.io 9 | - gcr.io 10 | - quay.io 11 | -------------------------------------------------------------------------------- /config/.yamllint.yml: -------------------------------------------------------------------------------- 1 | rules: 2 | braces: disable 3 | brackets: disable 4 | colons: enable 5 | commas: disable 6 | comments: disable 7 | comments-indentation: disable 8 | document-end: disable 9 | document-start: disable 10 | empty-lines: disable 11 | empty-values: enable 12 | hyphens: enable 13 | indentation: disable 14 | key-duplicates: enable 15 | key-ordering: disable 16 | line-length: disable 17 | new-line-at-end-of-file: disable 18 | new-lines: enable 19 | octal-values: enable 20 | quoted-strings: disable 21 | trailing-spaces: disable 22 | truthy: disable 23 | -------------------------------------------------------------------------------- /deployments/Dockerfile.install-minicni: -------------------------------------------------------------------------------- 1 | FROM alpine:3.13 2 | 3 | LABEL description="Installer for the minicni." 4 | 5 | RUN apk --update --no-cache add curl jq bash && \ 6 | rm -rf /var/lib/apt/lists/* 7 | 8 | COPY build/_output/bin/minicni /cni-plugins/minicni 9 | COPY deployments/scripts/install-minicni.sh /install-minicni.sh 10 | 11 | ENTRYPOINT ["/install-minicni.sh"] 12 | -------------------------------------------------------------------------------- /deployments/manifests/minicni.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: rbac.authorization.k8s.io/v1 3 | kind: ClusterRole 4 | metadata: 5 | name: minicni 6 | labels: 7 | app: minicni 8 | rules: 9 | - apiGroups: [""] 10 | resources: ["pods", "nodes"] 11 | verbs: ["get"] 12 | --- 13 | apiVersion: v1 14 | kind: ServiceAccount 15 | metadata: 16 | name: minicni 17 | namespace: kube-system 18 | labels: 19 | app: minicni 20 | --- 21 | apiVersion: rbac.authorization.k8s.io/v1 22 | kind: ClusterRoleBinding 23 | metadata: 24 | name: minicni 25 | labels: 26 | app: minicni 27 | roleRef: 28 | apiGroup: rbac.authorization.k8s.io 29 | kind: ClusterRole 30 | name: minicni 31 | subjects: 32 | - kind: ServiceAccount 33 | name: minicni 34 | namespace: kube-system 35 | --- 36 | # The configmap will be used to configure CNI installation. 37 | apiVersion: v1 38 | kind: ConfigMap 39 | metadata: 40 | name: minicni-config 41 | namespace: kube-system 42 | labels: 43 | app: minicni 44 | data: 45 | # The CNI network configuration to add to the plugin chain on each node. The special 46 | # values in this config will be automatically populated. 47 | cni_network_config: |- 48 | { 49 | "cniVersion": "0.1.0", 50 | "name": "minicni", 51 | "type": "minicni", 52 | "bridge": "minicni0", 53 | "mtu": 1500, 54 | "subnet": __NODE_SUBNET__ 55 | } 56 | 57 | --- 58 | # This manifest used to installs the minicni plugin and config on each master and worker node 59 | # in a Kubernetes cluster with install-minicni.sh script in the container. 60 | apiVersion: apps/v1 61 | kind: DaemonSet 62 | metadata: 63 | name: minicni-node 64 | namespace: kube-system 65 | labels: 66 | app: minicni 67 | spec: 68 | selector: 69 | matchLabels: 70 | app: minicni 71 | updateStrategy: 72 | type: RollingUpdate 73 | rollingUpdate: 74 | maxUnavailable: 1 75 | template: 76 | metadata: 77 | labels: 78 | app: minicni 79 | annotations: 80 | # Mark this pod as a critical add-on to ensure it gets 81 | # priority scheduling and that its resources are reserved 82 | # if it ever gets evicted. 83 | scheduler.alpha.kubernetes.io/critical-pod: '' 84 | spec: 85 | nodeSelector: 86 | # The minicni currently only works on linux node. 87 | beta.kubernetes.io/os: linux 88 | hostNetwork: true 89 | tolerations: 90 | # Make sure minicni-node gets scheduled on all nodes. 91 | - effect: NoSchedule 92 | operator: Exists 93 | # Mark the pod as a critical add-on for rescheduling. 94 | - key: CriticalAddonsOnly 95 | operator: Exists 96 | - effect: NoExecute 97 | operator: Exists 98 | serviceAccountName: minicni 99 | containers: 100 | # This container installs the minicni binary 101 | # and CNI network config file on each node. 102 | - name: install-minicni 103 | image: quay.io/morvencao/install-minicni:0.1.0 104 | imagePullPolicy: Always 105 | env: 106 | # Pod name 107 | - name: POD_NAME 108 | valueFrom: 109 | fieldRef: 110 | fieldPath: metadata.name 111 | # Node name 112 | - name: NODE_NAME 113 | valueFrom: 114 | fieldRef: 115 | fieldPath: spec.nodeName 116 | # Name of the CNI config file to create. 117 | - name: CNI_CONF_NAME 118 | value: "10-minicni.conf" 119 | # The CNI network config to install on each node. 120 | - name: CNI_NETWORK_CONFIG 121 | valueFrom: 122 | configMapKeyRef: 123 | name: minicni-config 124 | key: cni_network_config 125 | volumeMounts: 126 | - mountPath: /host/opt/cni/bin 127 | name: cni-bin-dir 128 | - mountPath: /host/etc/cni/net.d 129 | name: cni-net-dir 130 | volumes: 131 | # CNI bininary and configuration directories 132 | - name: cni-bin-dir 133 | hostPath: 134 | path: /opt/cni/bin 135 | - name: cni-net-dir 136 | hostPath: 137 | path: /etc/cni/net.d 138 | --- 139 | 140 | -------------------------------------------------------------------------------- /deployments/scripts/install-minicni.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Script to install minicni on a Kubernetes host. 4 | # - Expects the host CNI binary path to be mounted at /host/opt/cni/bin. 5 | # - Expects the host CNI network config path to be mounted at /host/etc/cni/net.d. 6 | # - Expects the desired CNI config in the CNI_NETWORK_CONFIG env variable. 7 | # - Expects the desired node name in the NODE_NAME env variable. 8 | 9 | # Ensure all variables are defined, and that the script fails when an error is hit. 10 | set -eu 11 | 12 | CNI_NET_DIR=${CNI_NET_DIR:-/host/etc/cni/net.d} 13 | CNI_CONF_NAME=${CNI_CONF_NAME:-10-minicni.conf} 14 | CNI_PLUGINS_DIR=${CNI_PLUGINS_DIR:-/host/opt/cni/bin} 15 | CURRENT_CNI_PLUGINS_DIR=${CURRENT_CNI_PLUGINS_DIR:-/cni-plugins} 16 | 17 | function exit_with_message() { 18 | echo "$1" 19 | exit 1 20 | } 21 | 22 | # Clean up any existing binaries/config/assets. 23 | function cleanup() { 24 | echo "Cleaning up and exiting." 25 | if [ -e "${CNI_NET_DIR}/${CNI_CONF_NAME}" ]; then 26 | echo "Removing minicni config ${CNI_NET_DIR}/${CNI_CONF_NAME} from CNI configuration directory." 27 | rm "${CNI_NET_DIR}/${CNI_CONF_NAME}" 28 | fi 29 | if [ -e "${CNI_PLUGINS_DIR}/minicni" ]; then 30 | echo "Removing minicni binary ${CNI_PLUGINS_DIR}/minicni from CNI plugins directory." 31 | rm "${CNI_PLUGINS_DIR}/minicni" 32 | fi 33 | echo "Exiting." 34 | } 35 | 36 | # error handle 37 | trap cleanup EXIT 38 | 39 | ########################################################################################## 40 | # Copy the CNI plugins to the plugin directory 41 | ########################################################################################## 42 | if [ ! -w "${CNI_PLUGINS_DIR}" ]; 43 | then 44 | exit_with_message "$CNI_PLUGINS_DIR is not writeable" 45 | fi 46 | 47 | for plugin in "${CURRENT_CNI_PLUGINS_DIR}"/*; 48 | do 49 | cp "${plugin}" "${CNI_PLUGINS_DIR}/" || exit_with_message "Failed to copy ${plugin} to ${CNI_PLUGINS_DIR}." 50 | done 51 | 52 | echo "COPY all CNI plugins from ${CURRENT_CNI_PLUGINS_DIR} to ${CNI_PLUGINS_DIR}." 53 | 54 | ########################################################################################## 55 | # Generate the CNI configuration and move to CNI configuration directory 56 | ########################################################################################## 57 | 58 | # The environment variables used to connect to the kube-apiserver 59 | SERVICE_ACCOUNT_PATH=/var/run/secrets/kubernetes.io/serviceaccount 60 | SERVICEACCOUNT_TOKEN=$(cat $SERVICE_ACCOUNT_PATH/token) 61 | KUBE_CACERT=${KUBE_CACERT:-$SERVICE_ACCOUNT_PATH/ca.crt} 62 | KUBERNETES_SERVICE_PROTOCOL=${KUBERNETES_SERVICE_PROTOCOL-https} 63 | 64 | # Check if we're running as a k8s pod. 65 | if [ -f "$SERVICE_ACCOUNT_PATH/token" ]; 66 | then 67 | # some variables should be automatically set inside a pod 68 | if [ -z "${KUBERNETES_SERVICE_HOST}" ]; then 69 | exit_with_message "KUBERNETES_SERVICE_HOST not set" 70 | fi 71 | if [ -z "${KUBERNETES_SERVICE_PORT}" ]; then 72 | exit_with_message "KUBERNETES_SERVICE_PORT not set" 73 | fi 74 | fi 75 | 76 | # exit if the NODE_NAME environment variable is not set. 77 | if [[ -z "${NODE_NAME}" ]]; 78 | then 79 | exit_with_message "NODE_NAME not set." 80 | fi 81 | 82 | 83 | NODE_RESOURCE_PATH="${KUBERNETES_SERVICE_PROTOCOL}://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}/api/v1/nodes/${NODE_NAME}" 84 | NODE_SUBNET=$(curl --cacert "${KUBE_CACERT}" --header "Authorization: Bearer ${SERVICEACCOUNT_TOKEN}" -X GET "${NODE_RESOURCE_PATH}" | jq ".spec.podCIDR") 85 | 86 | # Check if the node subnet is valid IPv4 CIDR address 87 | IPV4_CIDR_REGEX="(((25[0-5]|2[0-4][0-9]|1?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|1?[0-9][0-9]?))(\/([8-9]|[1-2][0-9]|3[0-2]))([^0-9.]|$)" 88 | if [[ ${NODE_SUBNET} =~ ${IPV4_CIDR_REGEX} ]] 89 | then 90 | echo "${NODE_SUBNET} is a valid IPv4 CIDR address." 91 | else 92 | exit_with_message "${NODE_SUBNET} is not a valid IPv4 CIDR address!" 93 | fi 94 | 95 | # exit if the NODE_NAME environment variable is not set. 96 | if [[ -z "${CNI_NETWORK_CONFIG}" ]]; 97 | then 98 | exit_with_message "CNI_NETWORK_CONFIG not set." 99 | fi 100 | 101 | TMP_CONF='/minicni.conf.tmp' 102 | cat >"${TMP_CONF}" <= 0; i-- { 30 | ip[i]++ 31 | if ip[i] > 0 { 32 | break 33 | } 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /pkg/nettool/ip_test.go: -------------------------------------------------------------------------------- 1 | package nettool 2 | 3 | import ( 4 | "reflect" 5 | "testing" 6 | ) 7 | 8 | func TestGetAllIPs(t *testing.T) { 9 | tests := []struct { 10 | name string 11 | input string 12 | want []string 13 | wantErr bool 14 | }{ 15 | { 16 | name: "valid cidr input", 17 | input: "192.168.0.0/30", 18 | want: []string{"192.168.0.1/30", "192.168.0.2/30"}, 19 | wantErr: false, 20 | }, 21 | { 22 | name: "invalid cidr input", 23 | input: "192.168.0/30", 24 | want: nil, 25 | wantErr: true, 26 | }, 27 | { 28 | name: "valid cidr input with more ips", 29 | input: "62.76.47.12/28", 30 | want: []string{"62.76.47.1/28", "62.76.47.2/28", "62.76.47.3/28", "62.76.47.4/28", 31 | "62.76.47.5/28", "62.76.47.6/28", "62.76.47.7/28", "62.76.47.8/28", "62.76.47.9/28", 32 | "62.76.47.10/28", "62.76.47.11/28", "62.76.47.12/28", "62.76.47.13/28", "62.76.47.14/28"}, 33 | wantErr: false, 34 | }, 35 | { 36 | name: "valid cidr input with more and more ips", 37 | input: "172.18.1.0/24", 38 | want: []string{"172.18.1.1/24", "172.18.1.2/24", "172.18.1.3/24", "172.18.1.4/24", "172.18.1.5/24", "172.18.1.6/24", "172.18.1.7/24", "172.18.1.8/24", 39 | "172.18.1.9/24", "172.18.1.10/24", "172.18.1.11/24", "172.18.1.12/24", "172.18.1.13/24", "172.18.1.14/24", "172.18.1.15/24", "172.18.1.16/24", 40 | "172.18.1.17/24", "172.18.1.18/24", "172.18.1.19/24", "172.18.1.20/24", "172.18.1.21/24", "172.18.1.22/24", "172.18.1.23/24", "172.18.1.24/24", 41 | "172.18.1.25/24", "172.18.1.26/24", "172.18.1.27/24", "172.18.1.28/24", "172.18.1.29/24", "172.18.1.30/24", "172.18.1.31/24", "172.18.1.32/24", 42 | "172.18.1.33/24", "172.18.1.34/24", "172.18.1.35/24", "172.18.1.36/24", "172.18.1.37/24", "172.18.1.38/24", "172.18.1.39/24", "172.18.1.40/24", 43 | "172.18.1.41/24", "172.18.1.42/24", "172.18.1.43/24", "172.18.1.44/24", "172.18.1.45/24", "172.18.1.46/24", "172.18.1.47/24", "172.18.1.48/24", 44 | "172.18.1.49/24", "172.18.1.50/24", "172.18.1.51/24", "172.18.1.52/24", "172.18.1.53/24", "172.18.1.54/24", "172.18.1.55/24", "172.18.1.56/24", 45 | "172.18.1.57/24", "172.18.1.58/24", "172.18.1.59/24", "172.18.1.60/24", "172.18.1.61/24", "172.18.1.62/24", "172.18.1.63/24", "172.18.1.64/24", 46 | "172.18.1.65/24", "172.18.1.66/24", "172.18.1.67/24", "172.18.1.68/24", "172.18.1.69/24", "172.18.1.70/24", "172.18.1.71/24", "172.18.1.72/24", 47 | "172.18.1.73/24", "172.18.1.74/24", "172.18.1.75/24", "172.18.1.76/24", "172.18.1.77/24", "172.18.1.78/24", "172.18.1.79/24", "172.18.1.80/24", 48 | "172.18.1.81/24", "172.18.1.82/24", "172.18.1.83/24", "172.18.1.84/24", "172.18.1.85/24", "172.18.1.86/24", "172.18.1.87/24", "172.18.1.88/24", 49 | "172.18.1.89/24", "172.18.1.90/24", "172.18.1.91/24", "172.18.1.92/24", "172.18.1.93/24", "172.18.1.94/24", "172.18.1.95/24", "172.18.1.96/24", 50 | "172.18.1.97/24", "172.18.1.98/24", "172.18.1.99/24", "172.18.1.100/24", "172.18.1.101/24", "172.18.1.102/24", "172.18.1.103/24", "172.18.1.104/24", 51 | "172.18.1.105/24", "172.18.1.106/24", "172.18.1.107/24", "172.18.1.108/24", "172.18.1.109/24", "172.18.1.110/24", "172.18.1.111/24", "172.18.1.112/24", 52 | "172.18.1.113/24", "172.18.1.114/24", "172.18.1.115/24", "172.18.1.116/24", "172.18.1.117/24", "172.18.1.118/24", "172.18.1.119/24", "172.18.1.120/24", 53 | "172.18.1.121/24", "172.18.1.122/24", "172.18.1.123/24", "172.18.1.124/24", "172.18.1.125/24", "172.18.1.126/24", "172.18.1.127/24", "172.18.1.128/24", 54 | "172.18.1.129/24", "172.18.1.130/24", "172.18.1.131/24", "172.18.1.132/24", "172.18.1.133/24", "172.18.1.134/24", "172.18.1.135/24", "172.18.1.136/24", 55 | "172.18.1.137/24", "172.18.1.138/24", "172.18.1.139/24", "172.18.1.140/24", "172.18.1.141/24", "172.18.1.142/24", "172.18.1.143/24", "172.18.1.144/24", 56 | "172.18.1.145/24", "172.18.1.146/24", "172.18.1.147/24", "172.18.1.148/24", "172.18.1.149/24", "172.18.1.150/24", "172.18.1.151/24", "172.18.1.152/24", 57 | "172.18.1.153/24", "172.18.1.154/24", "172.18.1.155/24", "172.18.1.156/24", "172.18.1.157/24", "172.18.1.158/24", "172.18.1.159/24", "172.18.1.160/24", 58 | "172.18.1.161/24", "172.18.1.162/24", "172.18.1.163/24", "172.18.1.164/24", "172.18.1.165/24", "172.18.1.166/24", "172.18.1.167/24", "172.18.1.168/24", 59 | "172.18.1.169/24", "172.18.1.170/24", "172.18.1.171/24", "172.18.1.172/24", "172.18.1.173/24", "172.18.1.174/24", "172.18.1.175/24", "172.18.1.176/24", 60 | "172.18.1.177/24", "172.18.1.178/24", "172.18.1.179/24", "172.18.1.180/24", "172.18.1.181/24", "172.18.1.182/24", "172.18.1.183/24", "172.18.1.184/24", 61 | "172.18.1.185/24", "172.18.1.186/24", "172.18.1.187/24", "172.18.1.188/24", "172.18.1.189/24", "172.18.1.190/24", "172.18.1.191/24", "172.18.1.192/24", 62 | "172.18.1.193/24", "172.18.1.194/24", "172.18.1.195/24", "172.18.1.196/24", "172.18.1.197/24", "172.18.1.198/24", "172.18.1.199/24", "172.18.1.200/24", 63 | "172.18.1.201/24", "172.18.1.202/24", "172.18.1.203/24", "172.18.1.204/24", "172.18.1.205/24", "172.18.1.206/24", "172.18.1.207/24", "172.18.1.208/24", 64 | "172.18.1.209/24", "172.18.1.210/24", "172.18.1.211/24", "172.18.1.212/24", "172.18.1.213/24", "172.18.1.214/24", "172.18.1.215/24", "172.18.1.216/24", 65 | "172.18.1.217/24", "172.18.1.218/24", "172.18.1.219/24", "172.18.1.220/24", "172.18.1.221/24", "172.18.1.222/24", "172.18.1.223/24", "172.18.1.224/24", 66 | "172.18.1.225/24", "172.18.1.226/24", "172.18.1.227/24", "172.18.1.228/24", "172.18.1.229/24", "172.18.1.230/24", "172.18.1.231/24", "172.18.1.232/24", 67 | "172.18.1.233/24", "172.18.1.234/24", "172.18.1.235/24", "172.18.1.236/24", "172.18.1.237/24", "172.18.1.238/24", "172.18.1.239/24", "172.18.1.240/24", 68 | "172.18.1.241/24", "172.18.1.242/24", "172.18.1.243/24", "172.18.1.244/24", "172.18.1.245/24", "172.18.1.246/24", "172.18.1.247/24", "172.18.1.248/24", 69 | "172.18.1.249/24", "172.18.1.250/24", "172.18.1.251/24", "172.18.1.252/24", "172.18.1.253/24", "172.18.1.254/24"}, 70 | wantErr: false, 71 | }, 72 | } 73 | 74 | for _, tt := range tests { 75 | t.Run(tt.name, func(t *testing.T) { 76 | allips, err := GetAllIPs(tt.input) 77 | if (err != nil) != tt.wantErr { 78 | t.Errorf("GetAllIPs error = %v, wantErr %v", err, tt.wantErr) 79 | } 80 | if !reflect.DeepEqual(allips, tt.want) { 81 | t.Errorf("wanted:\n%v\ngot:\n%v", tt.want, allips) 82 | } 83 | }) 84 | } 85 | } 86 | -------------------------------------------------------------------------------- /pkg/nettool/link.go: -------------------------------------------------------------------------------- 1 | package nettool 2 | 3 | import ( 4 | "crypto/rand" 5 | "fmt" 6 | "net" 7 | "os" 8 | 9 | "github.com/containernetworking/plugins/pkg/ns" 10 | "github.com/vishvananda/netlink" 11 | ) 12 | 13 | // CreateOrUpdateBridge creates or updates bridge and sets its as the gateway of container network 14 | func CreateOrUpdateBridge(name, ip string, mtu int) (*netlink.Bridge, error) { 15 | br := &netlink.Bridge{ 16 | LinkAttrs: netlink.LinkAttrs{ 17 | Name: name, 18 | MTU: mtu, 19 | TxQLen: -1, 20 | }, 21 | } 22 | 23 | // ip address for bridge 24 | ipaddr, ipnet, err := net.ParseCIDR(ip) 25 | if err != nil { 26 | return nil, fmt.Errorf("failed to parse ip address %q: %v", ip, err) 27 | } 28 | ipnet.IP = ipaddr 29 | addr := &netlink.Addr{IPNet: ipnet} 30 | 31 | l, err := netlink.LinkByName(name) 32 | if err != nil { 33 | if _, ok := err.(netlink.LinkNotFoundError); ok { 34 | if err := netlink.LinkAdd(br); err != nil { 35 | return nil, fmt.Errorf("failed to create bridge %q with error: %v", name, err) 36 | } 37 | if err = netlink.AddrAdd(br, addr); err != nil { 38 | return nil, fmt.Errorf("failed to set address: %q for bridge %q: %v", addr, name, err) 39 | } 40 | if err = netlink.LinkSetUp(br); err != nil { 41 | return nil, fmt.Errorf("failed to set bridge %q up: %v", name, err) 42 | } 43 | return br, nil 44 | } 45 | return nil, fmt.Errorf("could not find link %s: %v", name, err) 46 | } 47 | currentBr, ok := l.(*netlink.Bridge) 48 | if !ok { 49 | return nil, fmt.Errorf("link %s already exists but is not a bridge type", name) 50 | } 51 | addrs, err := netlink.AddrList(currentBr, netlink.FAMILY_ALL) 52 | if err != nil { 53 | return nil, fmt.Errorf("failed to list address for bridge %q: %v", name, err) 54 | } 55 | switch { 56 | case len(addrs) > 1: 57 | return nil, fmt.Errorf("unexpected addresses for bridge %q: %v", name, addrs) 58 | case len(addrs) == 1: 59 | if !addr.Equal(addrs[0]) { 60 | // try to replace the address for the bridge 61 | if err = netlink.AddrReplace(currentBr, addr); err != nil { 62 | return nil, fmt.Errorf("failed to replace address: %q for bridge %q: %v", addr, name, err) 63 | } 64 | } 65 | default: 66 | if err = netlink.AddrAdd(currentBr, addr); err != nil { 67 | return nil, fmt.Errorf("failed to set address: %q for bridge %q: %v", addr, name, err) 68 | } 69 | } 70 | return currentBr, nil 71 | } 72 | 73 | // SetupVeth sets up a pair of virtual ethernet devices in container netns 74 | // and then move the host-side veth into the hostNS namespace. 75 | func SetupVeth(netns ns.NetNS, br *netlink.Bridge, ifName, ip, gwip string, mtu int) error { 76 | err := netns.Do(func(hostNS ns.NetNS) error { 77 | hostVethName, veth, err := makeVethPair(ifName, mtu) 78 | if err != nil { 79 | return err 80 | } 81 | ipaddr, ipnet, err := net.ParseCIDR(ip) 82 | if err != nil { 83 | return fmt.Errorf("failed to parse ip address %q: %v", ip, err) 84 | } 85 | ipnet.IP = ipaddr 86 | if err = netlink.AddrAdd(veth, &netlink.Addr{IPNet: ipnet}); err != nil { 87 | return fmt.Errorf("failed to set address: %q for veth %q: %v", ipnet, ifName, err) 88 | } 89 | if err = netlink.LinkSetUp(veth); err != nil { 90 | return fmt.Errorf("failed to set veth %q up: %v", ifName, err) 91 | } 92 | 93 | // add bridge IP as the default route for container 94 | gwNetIP, _, err := net.ParseCIDR(gwip) 95 | if err != nil { 96 | return fmt.Errorf("failed to parse gateway IP %q: %v", gwip, err) 97 | } 98 | if err = AddDefaultRoute(gwNetIP, veth); err != nil { 99 | return fmt.Errorf("failed to add default route for %q: %v", ifName, err) 100 | } 101 | 102 | hostVeth, err := netlink.LinkByName(hostVethName) 103 | if err != nil { 104 | return fmt.Errorf("failed to lookup hostveth %q: %v", hostVethName, err) 105 | } 106 | if err = netlink.LinkSetNsFd(hostVeth, int(hostNS.Fd())); err != nil { 107 | return fmt.Errorf("failed to set hostveth %q to host netns: %v", hostVethName, err) 108 | } 109 | err = hostNS.Do(func(_ ns.NetNS) error { 110 | hostVeth, err = netlink.LinkByName(hostVethName) 111 | if err != nil { 112 | return fmt.Errorf("failed to lookup hostveth %q in %q: %v", hostVethName, hostNS.Path(), err) 113 | } 114 | if err = netlink.LinkSetUp(hostVeth); err != nil { 115 | return fmt.Errorf("failed to set veth %q up: %v", hostVethName, err) 116 | } 117 | 118 | // connect the host veth to the bridge 119 | if err = netlink.LinkSetMaster(hostVeth, br); err != nil { 120 | return fmt.Errorf("failed to connect %q to bridge %v: %v", hostVethName, br.Name, err) 121 | } 122 | return nil 123 | }) 124 | if err != nil { 125 | return fmt.Errorf("failed to set hostveth %q: %v", hostVethName, err) 126 | } 127 | return nil 128 | }) 129 | if err != nil { 130 | return fmt.Errorf("failed to set veth %q: %v", ifName, err) 131 | } 132 | 133 | return nil 134 | } 135 | 136 | // makeVethPair create veth pair and peer name with random string with "veth" prefix 137 | func makeVethPair(name string, mtu int) (string, *netlink.Veth, error) { 138 | peerName, err := generateRandomVethName() 139 | if err != nil { 140 | return "", nil, err 141 | } 142 | veth := &netlink.Veth{ 143 | LinkAttrs: netlink.LinkAttrs{ 144 | Name: name, 145 | //Flags: net.FlagUp, 146 | MTU: mtu, 147 | }, 148 | PeerName: peerName, 149 | } 150 | 151 | err = netlink.LinkAdd(veth) 152 | switch { 153 | case err == nil: 154 | return peerName, veth, nil 155 | case os.IsExist(err): 156 | return "", nil, fmt.Errorf("failed to create veth because veth name %q already exists", name) 157 | default: 158 | return "", nil, fmt.Errorf("failed to create veth %q with error: %v", name, err) 159 | } 160 | } 161 | 162 | // generateRandomVethName generate random string start with "veth" 163 | func generateRandomVethName() (string, error) { 164 | rd := make([]byte, 4) 165 | if _, err := rand.Read(rd); err != nil { 166 | return "", fmt.Errorf("failed to gererate random veth name: %v", err) 167 | } 168 | 169 | return fmt.Sprintf("veth%x", rd), nil 170 | } 171 | 172 | // GetVethIPInNS return the IP address for the ifName in container Namespace 173 | func GetVethIPInNS(netns ns.NetNS, ifName string) (string, error) { 174 | ip := "" 175 | err := netns.Do(func(_ ns.NetNS) error { 176 | l, err := netlink.LinkByName(ifName) 177 | if err != nil { 178 | return fmt.Errorf("failed to lookup veth %q in %q: %v", ifName, netns.Path(), err) 179 | } 180 | veth, ok := l.(*netlink.Veth) 181 | if !ok { 182 | return fmt.Errorf("link %s already exists but is not a veth type", ifName) 183 | } 184 | addrs, err := netlink.AddrList(veth, netlink.FAMILY_ALL) 185 | if err != nil { 186 | return fmt.Errorf("failed to list address for veth %q: %v", ifName, err) 187 | } 188 | switch { 189 | case len(addrs) > 1: 190 | return fmt.Errorf("unexpected addresses for veth %q: %v", ifName, addrs) 191 | case len(addrs) == 1: 192 | ip = addrs[0].IPNet.String() 193 | default: 194 | return fmt.Errorf("no address set for veth %q", ifName) 195 | } 196 | return nil 197 | }) 198 | if err != nil { 199 | return "", err 200 | } 201 | return ip, nil 202 | } 203 | -------------------------------------------------------------------------------- /pkg/nettool/route.go: -------------------------------------------------------------------------------- 1 | // Copyright 2015-2017 CNI authors 2 | package nettool 3 | 4 | import ( 5 | "net" 6 | 7 | "github.com/vishvananda/netlink" 8 | ) 9 | 10 | // AddRoute adds a universally-scoped route to a device. 11 | func AddRoute(ipn *net.IPNet, gw net.IP, dev netlink.Link) error { 12 | return netlink.RouteAdd(&netlink.Route{ 13 | LinkIndex: dev.Attrs().Index, 14 | Scope: netlink.SCOPE_UNIVERSE, 15 | Dst: ipn, 16 | Gw: gw, 17 | }) 18 | } 19 | 20 | // AddHostRoute adds a host-scoped route to a device. 21 | func AddHostRoute(ipn *net.IPNet, gw net.IP, dev netlink.Link) error { 22 | return netlink.RouteAdd(&netlink.Route{ 23 | LinkIndex: dev.Attrs().Index, 24 | Scope: netlink.SCOPE_HOST, 25 | Dst: ipn, 26 | Gw: gw, 27 | }) 28 | } 29 | 30 | // AddDefaultRoute sets the default route on the given gateway. 31 | func AddDefaultRoute(gw net.IP, dev netlink.Link) error { 32 | _, defNet, _ := net.ParseCIDR("0.0.0.0/0") 33 | return AddRoute(defNet, gw, dev) 34 | } 35 | -------------------------------------------------------------------------------- /pkg/version/version.go: -------------------------------------------------------------------------------- 1 | package version 2 | 3 | var ( 4 | Version = "0.1.0" 5 | ) 6 | 7 | // nolint 8 | type VersionInfo struct { 9 | CniVersion string `json:"cniVersion"` 10 | SupportedVersions []string `json:"supportedVersions"` 11 | } 12 | -------------------------------------------------------------------------------- /scripts/gobuild.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # This script builds and version stamps the output 4 | 5 | VERBOSE=${VERBOSE:-"0"} 6 | V="" 7 | if [[ "${VERBOSE}" == "1" ]];then 8 | V="-x" 9 | set -x 10 | fi 11 | 12 | OUT=${1:?"output path"} 13 | shift 14 | 15 | set -e 16 | 17 | BUILD_GOOS=${GOOS:-linux} 18 | BUILD_GOARCH=${GOARCH:-amd64} 19 | GOBINARY=${GOBINARY:-go} 20 | BUILDINFO=${BUILDINFO:-""} 21 | STATIC=${STATIC:-1} 22 | GOBUILDFLAGS=${GOBUILDFLAGS:-} 23 | GCFLAGS=${GCFLAGS:-} 24 | LDFLAGS=${LDFLAGS:-"-extldflags -static"} 25 | # Split GOBUILDFLAGS by spaces into an array called GOBUILDFLAGS_ARRAY. 26 | IFS=' ' read -r -a GOBUILDFLAGS_ARRAY <<< "$GOBUILDFLAGS" 27 | 28 | export CGO_ENABLED=0 29 | 30 | if [[ "${STATIC}" != "1" ]];then 31 | LDFLAGS="" 32 | fi 33 | 34 | time GOOS=${BUILD_GOOS} GOARCH=${BUILD_GOARCH} ${GOBINARY} build \ 35 | ${V} "${GOBUILDFLAGS_ARRAY[@]}" ${GCFLAGS:+-gcflags "${GCFLAGS}"} \ 36 | -o "${OUT}" \ 37 | -ldflags "${LDFLAGS}" "${@}" 38 | -------------------------------------------------------------------------------- /scripts/lint_go.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | GOGC=25 golangci-lint run -c ./config/.golangci.yml 4 | -------------------------------------------------------------------------------- /tests/test-pods.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Pod 3 | metadata: 4 | name: httpbin-master 5 | spec: 6 | containers: 7 | - name: httpbin 8 | image: kennethreitz/httpbin:latest 9 | imagePullPolicy: IfNotPresent 10 | ports: 11 | - containerPort: 80 12 | nodeSelector: 13 | beta.kubernetes.io/os: linux 14 | role: master 15 | tolerations: 16 | # Make sure this pod gets scheduled on all nodes. 17 | - effect: NoSchedule 18 | operator: Exists 19 | # Mark the pod as a critical add-on for rescheduling. 20 | - key: CriticalAddonsOnly 21 | operator: Exists 22 | - effect: NoExecute 23 | operator: Exists 24 | --- 25 | apiVersion: v1 26 | kind: Pod 27 | metadata: 28 | name: netshoot-master 29 | spec: 30 | containers: 31 | - name: ubuntu 32 | image: nicolaka/netshoot:latest 33 | imagePullPolicy: IfNotPresent 34 | command: 35 | - "bin/bash" 36 | - "-c" 37 | - "sleep 10000" 38 | nodeSelector: 39 | beta.kubernetes.io/os: linux 40 | role: master 41 | tolerations: 42 | # Make sure this pod gets scheduled on all nodes. 43 | - effect: NoSchedule 44 | operator: Exists 45 | # Mark the pod as a critical add-on for rescheduling. 46 | - key: CriticalAddonsOnly 47 | operator: Exists 48 | - effect: NoExecute 49 | operator: Exists 50 | --- 51 | apiVersion: v1 52 | kind: Pod 53 | metadata: 54 | name: httpbin-worker 55 | spec: 56 | containers: 57 | - name: httpbin 58 | image: kennethreitz/httpbin:latest 59 | imagePullPolicy: IfNotPresent 60 | ports: 61 | - containerPort: 80 62 | nodeSelector: 63 | beta.kubernetes.io/os: linux 64 | role: worker 65 | --- 66 | apiVersion: v1 67 | kind: Pod 68 | metadata: 69 | name: netshoot-worker 70 | spec: 71 | containers: 72 | - name: ubuntu 73 | image: nicolaka/netshoot:latest 74 | imagePullPolicy: IfNotPresent 75 | command: 76 | - "bin/bash" 77 | - "-c" 78 | - "sleep 10000" 79 | nodeSelector: 80 | beta.kubernetes.io/os: linux 81 | role: worker 82 | 83 | --------------------------------------------------------------------------------