├── .github ├── ISSUE_TEMPLATE │ ├── bug_report.md │ ├── feature_request.md │ └── support_request.md ├── PULL_REQUEST_TEMPLATE.md ├── mergeable.yml └── workflows │ ├── ci.yaml │ ├── docker.yaml │ ├── helm.yaml │ └── scorecard.yml ├── .gitignore ├── .golangci.yml ├── .licensei.toml ├── ADOPTERS.md ├── CODEOWNERS ├── Dockerfile ├── LICENSE ├── Makefile ├── PROJECT ├── README.md ├── api ├── go.mod ├── go.sum └── v1alpha1 │ ├── cluster_feature_types.go │ ├── cluster_metadata.go │ ├── cluster_types.go │ ├── groupversion_info.go │ ├── resourcesyncrule_match.go │ ├── resourcesyncrule_types.go │ └── zz_generated.deepcopy.go ├── cmd └── manager │ ├── build.go │ ├── config.go │ └── main.go ├── controllers ├── cluster_conditions.go ├── cluster_reconciler.go ├── cluster_reconciler_test.go ├── cluster_reconciler_watches.go ├── common.go ├── core_syncers.go ├── errors.go ├── remote_cluster_feature_reconciler.go ├── remote_cluster_reconciler.go ├── resource_sync_rule_reconciler.go ├── suite_test.go └── sync_reconciler.go ├── deploy └── charts │ ├── cluster-registry │ ├── .helmignore │ ├── Chart.yaml │ ├── LICENSE │ ├── README.md │ ├── crds │ │ ├── clusterregistry.k8s.cisco.com_clusterfeatures.yaml │ │ ├── clusterregistry.k8s.cisco.com_clusters.yaml │ │ └── clusterregistry.k8s.cisco.com_resourcesyncrules.yaml │ ├── examples │ │ ├── clusterregistry_v1alpha1_cluster.yaml │ │ ├── clusterregistry_v1alpha1_clusterfeature.yaml │ │ ├── clusterregistry_v1alpha1_resourcesyncrule.yaml │ │ └── test_secret.yaml │ ├── templates │ │ ├── _helpers.tpl │ │ ├── cluster-validator-vwhc.yaml │ │ ├── deployment.yaml │ │ ├── poddistruptionbudget.yaml │ │ ├── rbac-leader-election.yaml │ │ ├── rbac.yaml │ │ ├── reader-rbac.yaml │ │ └── service.yaml │ └── values.yaml │ ├── embed.go │ ├── embed_test.go │ └── go.mod ├── docs ├── development.md ├── gitops.md └── img │ ├── clean-k8s-arch.jpg │ ├── final-arch.jpg │ ├── installed-cr.jpg │ └── ui-cr-1.png ├── go.mod ├── go.sum ├── hack └── boilerplate.go.txt ├── internal └── config │ └── config.go ├── pkg ├── cert │ ├── certificate.go │ ├── renewer.go │ └── webhook_certifier.go ├── clustermeta │ ├── distribution.go │ ├── distribution_test.go │ ├── distributions.go │ ├── helper_test.go │ ├── metadata.go │ ├── provider.go │ ├── provider_test.go │ ├── providers.go │ └── testdata │ │ ├── amazon-eks.yaml │ │ ├── amazon-openshift.yaml │ │ ├── amazon-pke.yaml │ │ ├── azure-aks.yaml │ │ ├── azure-pke.yaml │ │ ├── cisco-iks.yaml │ │ ├── gcp-gke.yaml │ │ ├── kind-kind.yaml │ │ ├── unknown-distribution.yaml │ │ ├── unknown-k3s.yaml │ │ ├── unknown-provider.yaml │ │ ├── unknown-rke.yaml │ │ └── vsphere-pke.yaml ├── clusters │ ├── cluster.go │ ├── cluster_features.go │ ├── cluster_test.go │ ├── managed_controller.go │ ├── managed_reconciler.go │ └── manager.go ├── ratelimit │ └── ratelimit.go ├── signals │ └── signals.go ├── util │ ├── cluster_secret.go │ ├── kubeconfig.go │ ├── overlaypatchtemplate.go │ ├── provision_local_cluster.go │ ├── resources.go │ └── util_test.go └── webhooks │ └── cluster_validator.go └── scripts ├── download-deps.sh └── install_kubebuilder_tools.sh /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve the Cluster registry controller 4 | 5 | --- 6 | 7 | **Describe the bug** 8 | A clear and concise description of what the bug is. 9 | 10 | **Steps to reproduce the issue:** 11 | Please describe the steps to reproduce the issue. 12 | 13 | **Expected behavior** 14 | A clear and concise description of what you expected to happen. 15 | 16 | **Screenshots** 17 | If applicable, add screenshots to help explain your problem. 18 | 19 | **Additional context** 20 | Add any other context about the problem like release number version, branch, etc. 21 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for this project 4 | 5 | --- 6 | 7 | **Is your feature request related to a problem? Please describe.** 8 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 9 | 10 | **Describe the solution you'd like to see** 11 | A clear and concise description of what would you like to happen. 12 | 13 | **Describe alternatives you've considered** 14 | A clear and concise description of any alternative solutions or features you've considered. 15 | 16 | **Additional context** 17 | Add any other context or screenshots about the feature request here. 18 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/support_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: ⛔ Support request 3 | --- 4 | 5 | We use GitHub issues to discuss Cluster registry controller bugs and new features. 6 | 7 | Thanks! 8 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | | Q | A 2 | | --------------- | --- 3 | | Bug fix? | no|yes 4 | | New feature? | no|yes 5 | | API breaks? | no|yes 6 | | Deprecations? | no|yes 7 | | Related tickets | fixes #X, partially #Y, mentioned in #Z 8 | | License | Apache 2.0 9 | 10 | 11 | ### What's in this PR? 12 | 13 | 14 | 15 | ### Why? 16 | 17 | 18 | 19 | ### Additional context 20 | 21 | 22 | 23 | ### Checklist 24 | 25 | 26 | - [ ] Implementation tested 27 | - [ ] User guide and development docs updated (if needed) 28 | 29 | ### To Do 30 | 31 | - [ ] If the PR is not complete but you want to discuss the approach, list what remains to be done here 32 | -------------------------------------------------------------------------------- /.github/mergeable.yml: -------------------------------------------------------------------------------- 1 | version: 2 2 | mergeable: 3 | - when: pull_request.* 4 | validate: 5 | - do: title 6 | must_exclude: 7 | regex: '^(\[wip\]|wip:)' 8 | message: 'WIP tag in PR title' 9 | - do: label 10 | must_exclude: 11 | regex: 'wip' 12 | message: 'WIP label on PR' 13 | - do: description 14 | and: 15 | - must_exclude: 16 | regex: '\[ \]' 17 | message: 'Remaining tasks in the description.' 18 | - must_exclude: 19 | regex: 'no\|yes|fixes #X, partially #Y, mentioned in #Z' 20 | message: 'Please fill out the PR template.' 21 | -------------------------------------------------------------------------------- /.github/workflows/ci.yaml: -------------------------------------------------------------------------------- 1 | name: CI 2 | 3 | on: 4 | pull_request: 5 | 6 | env: 7 | GO_VERSION: 1.18 8 | GOFLAGS: -mod=readonly 9 | 10 | jobs: 11 | build: 12 | name: Build project 13 | runs-on: ubuntu-latest 14 | 15 | steps: 16 | - name: Check out code 17 | uses: actions/checkout@v2 18 | 19 | - name: Setup Go 20 | uses: actions/setup-go@v2 21 | with: 22 | go-version: ${{ env.GO_VERSION }} 23 | 24 | - name: Cache Go module dependencies 25 | id: cache-go-module-dependencies 26 | uses: actions/cache@v2 27 | with: 28 | path: ~/go/pkg/mod 29 | key: go-mod-cache-${{ runner.os }}-${{ env.GO_VERSION }}-${{ hashFiles('go.sum') }} 30 | restore-keys: | 31 | go-mod-cache-${{ runner.os }}-${{ env.GO_VERSION }} 32 | go-mod-cache-${{ runner.os }} 33 | go-mod-cache 34 | 35 | - name: Set Git refname 36 | id: set-git-refname 37 | run: echo ::set-output name=git_refname::$(echo "${{ github.ref }}" | sed -r 's@refs/(heads|pull|tags)/@@g' ) 38 | 39 | - name: Cache build dependencies 40 | id: cache-build-dependencies 41 | uses: actions/cache@v2 42 | with: 43 | path: bin/ 44 | key: build-deps-v2-${{ steps.set-git-refname.outputs.git_refname }}-{{ hashFiles('scripts/download-deps.sh') }} 45 | restore-keys: | 46 | build-deps-v2-${{ steps.set-git-refname.outputs.git_refname }} 47 | build-deps-v2 48 | 49 | - name: Cache licenses 50 | id: cache-licenses 51 | uses: actions/cache@v2 52 | with: 53 | path: .licensei.cache 54 | key: licensei-v1-${{ steps.set-git-refname.outputs.git_refname }}-${{ hashFiles('go.sum') }} 55 | restore-keys: | 56 | licensei-v1-${{ steps.set-git-refname.outputs.git_refname }} 57 | licensei-v1 58 | 59 | - name: Download license information for dependencies 60 | env: 61 | GITHUB_TOKEN: ${{ github.token }} # Note: this is required for licensei auth in steps to avoid rate-limiting. 62 | run: make license-cache 63 | 64 | - name: List license information for dependencies 65 | env: 66 | GITHUB_TOKEN: ${{ github.token }} # Note: this is required for licensei auth in steps to avoid rate-limiting. 67 | run: ./bin/licensei list 68 | 69 | - name: Check dependency licenses 70 | env: 71 | GITHUB_TOKEN: ${{ github.token }} # Note: this is required for licensei auth in steps to avoid rate-limiting. 72 | run: make license-check 73 | 74 | - name: Generate manifests 75 | run: make manifests 76 | 77 | - name: Run unit tests 78 | run: make test 79 | 80 | - name: Run lint 81 | run: make lint 82 | 83 | - name: Run build 84 | run: make binary 85 | -------------------------------------------------------------------------------- /.github/workflows/docker.yaml: -------------------------------------------------------------------------------- 1 | name: Docker 2 | 3 | on: 4 | push: 5 | tags: 6 | - "v[0-9]+.[0-9]+.[0-9]+" 7 | - "v[0-9]+.[0-9]+.[0-9]+-dev.[0-9]+" 8 | - "v[0-9]+.[0-9]+.[0-9]+-rc.[0-9]+" 9 | pull_request: 10 | 11 | env: 12 | PLATFORMS: linux/amd64,linux/arm64 13 | REGISTRY: ghcr.io 14 | IMAGE_NAME: ${{ github.repository }} 15 | 16 | jobs: 17 | docker: 18 | runs-on: ubuntu-latest 19 | permissions: 20 | contents: read 21 | packages: write 22 | steps: 23 | - 24 | name: Checkout 25 | uses: actions/checkout@v3 26 | - 27 | name: Extract metadata (tags, labels) for Docker 28 | id: meta 29 | uses: docker/metadata-action@v4 30 | with: 31 | images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} 32 | tags: type=semver,pattern={{raw}} 33 | - 34 | name: Set up Docker Buildx 35 | uses: docker/setup-buildx-action@v2 36 | with: 37 | platforms: ${{ env.PLATFORMS }} 38 | - 39 | name: Log in to the Container registry 40 | if: github.event_name != 'pull_request' 41 | uses: docker/login-action@v2 42 | with: 43 | registry: ${{ env.REGISTRY }} 44 | username: ${{ github.actor }} 45 | password: ${{ secrets.GITHUB_TOKEN }} 46 | - 47 | name: Build and push 48 | uses: docker/build-push-action@v4 49 | with: 50 | context: . 51 | platforms: ${{ env.PLATFORMS }} 52 | push: ${{ github.event_name != 'pull_request' }} 53 | tags: ${{ steps.meta.outputs.tags }} 54 | labels: ${{ steps.meta.outputs.labels }} 55 | cache-from: type=gha 56 | cache-to: type=gha,mode=max 57 | -------------------------------------------------------------------------------- /.github/workflows/helm.yaml: -------------------------------------------------------------------------------- 1 | name: Helm 2 | 3 | on: 4 | push: 5 | tags: 6 | - "deploy/charts/v[0-9]+.[0-9]+.[0-9]+" 7 | 8 | jobs: 9 | release: 10 | runs-on: ubuntu-latest 11 | steps: 12 | - uses: actions/checkout@v2 13 | - name: Publish Helm chart 14 | uses: stefanprodan/helm-gh-pages@v1.5.0 15 | with: 16 | token: "${{ secrets.GITHUB_TOKEN }}" 17 | charts_dir: deploy/charts 18 | -------------------------------------------------------------------------------- /.github/workflows/scorecard.yml: -------------------------------------------------------------------------------- 1 | name: scorecard 2 | 3 | on: 4 | push: 5 | branches: 6 | # Run on pushes to default branch 7 | - main 8 | schedule: 9 | # Run weekly on Saturdays 10 | - cron: "30 1 * * 6" 11 | # Run when branch protection rules change 12 | branch_protection_rule: 13 | # Run the workflow manually 14 | workflow_dispatch: 15 | 16 | # Declare default permissions as read-only 17 | permissions: read-all 18 | 19 | jobs: 20 | run-scorecard: 21 | # Call reusable workflow file 22 | uses: cisco-ospo/.github/.github/workflows/_scorecard.yml@main 23 | permissions: 24 | id-token: write 25 | security-events: write 26 | secrets: inherit 27 | with: 28 | # Publish results of Scorecard analysis 29 | publish-results: true 30 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | # Binaries for programs and plugins 3 | *.exe 4 | *.exe~ 5 | *.dll 6 | *.so 7 | *.dylib 8 | bin 9 | 10 | # Test binary, build with `go test -c` 11 | *.test 12 | 13 | # Output of the go coverage tool, specifically when used with LiteIDE 14 | *.out 15 | 16 | # Kubernetes Generated files - skip generated files, except for vendored files 17 | 18 | !vendor/**/zz_generated.* 19 | 20 | # editor and IDE paraphernalia 21 | .idea 22 | *.swp 23 | *.swo 24 | *~ 25 | 26 | /.licensei.cache 27 | 28 | .vscode 29 | -------------------------------------------------------------------------------- /.golangci.yml: -------------------------------------------------------------------------------- 1 | run: 2 | skip-dirs: 3 | - .gen 4 | skip-files: 5 | - ".*zz_.*\\.go$" 6 | 7 | linters: 8 | enable-all: true 9 | disable: 10 | - lll 11 | - gochecknoinits 12 | - gochecknoglobals 13 | - funlen 14 | - godot 15 | - godox 16 | - wsl 17 | - dupl 18 | - wrapcheck 19 | - forbidigo 20 | - varnamelen 21 | - ireturn 22 | - gci 23 | - containedctx 24 | - contextcheck 25 | 26 | - gomoddirectives 27 | 28 | # - goconst 29 | # - gocritic 30 | # - gocognit 31 | # - gomnd 32 | # - nestif 33 | # - testpackage 34 | 35 | # special cases only 36 | - exhaustivestruct 37 | 38 | # deprecated 39 | - maligned 40 | - interfacer 41 | - golint 42 | - scopelint 43 | 44 | linters-settings: 45 | gomnd: 46 | settings: 47 | mnd: 48 | checks: [case, operation, return, assign] 49 | gocognit: 50 | min-complexity: 50 51 | cyclop: 52 | max-complexity: 40 53 | golint: 54 | min-confidence: 0.1 55 | gocyclo: 56 | min-complexity: 40 57 | goimports: 58 | local-prefixes: github.com/banzaicloud,github.com/cisco-open 59 | gocritic: 60 | disabled-checks: 61 | - ifElseChain 62 | 63 | issues: 64 | # mainly because of the operator, but we are using helm chart names 65 | # as package names 66 | exclude: 67 | - underscore in package name 68 | - should not use underscores in package names 69 | 70 | exclude-rules: 71 | # zz_ files are messing up the receiver name 72 | - linters: 73 | - stylecheck 74 | text: "ST1016:" 75 | # fake client is still alive 76 | - linters: 77 | - staticcheck 78 | text: "SA1019:" 79 | -------------------------------------------------------------------------------- /.licensei.toml: -------------------------------------------------------------------------------- 1 | approved = [ 2 | "mit", 3 | "apache-2.0", 4 | "bsd-3-clause", 5 | "bsd-2-clause", 6 | "mpl-2.0", 7 | ] 8 | 9 | ignored = [ 10 | "github.com/davecgh/go-spew", # ISC license 11 | "github.com/gogo/protobuf", 12 | "google.golang.org/protobuf", 13 | "github.com/ghodss/yaml", 14 | "sigs.k8s.io/yaml", # Forked from above 15 | "gopkg.in/fsnotify.v1", 16 | 17 | "github.com/cisco-open/cluster-registry-controller/api", 18 | "github.com/cisco-open/cluster-registry-controller/deploy/charts", 19 | ] 20 | 21 | [header] 22 | ignoreFiles = [ 23 | "*.pb.go", 24 | "*.gogen.go", 25 | "zz_generated.deepcopy.go", 26 | ] 27 | 28 | template = """Copyright (c) :YEAR: Cisco and/or its affiliates. All rights reserved. 29 | """ 30 | -------------------------------------------------------------------------------- /ADOPTERS.md: -------------------------------------------------------------------------------- 1 | # Adopters 2 | 3 | This is a list of adopters of cluster registry controller: 4 | 5 | - The open-source Cisco Istio operator relies on the cluster registry controller to form multi cluster Istio meshes 6 | automatically: https://github.com/banzaicloud/istio-operator/tree/release-1.12/internal/assets/manifests/resource-sync-rule/templates 7 | - The open-source Cisco Koperator: https://github.com/banzaicloud/koperator 8 | - Cisco Service Mesh Manager (SMM) https://eti.cisco.com/appnet 9 | - Cisco Streaming Data Manager (SDM) https://eti.cisco.com/appnet 10 | -------------------------------------------------------------------------------- /CODEOWNERS: -------------------------------------------------------------------------------- 1 | 2 | * @cisco-open/cluster-registry-controller-maintainers 3 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | ARG GID=1000 3 | ARG UID=1000 4 | 5 | # Build the manager binary 6 | FROM golang:1.18.0 as builder 7 | ARG GITHUB_ACCESS_TOKEN 8 | ARG GID 9 | ARG UID 10 | 11 | # Create user and group 12 | RUN groupadd -g ${GID} appgroup && \ 13 | useradd -l -u ${UID} --gid appgroup appuser 14 | 15 | ARG GOPROXY="https://proxy.golang.org,direct" 16 | ENV GOPROXY="${GOPROXY}" 17 | ENV GOPRIVATE='github.com/cisco-open,github.com/banzaicloud' 18 | ENV GONOPROXY='gopkg.in,go.uber.org' 19 | ENV GOFLAGS="-mod=readonly" 20 | 21 | WORKDIR /workspace/ 22 | 23 | # Copy the Go Modules manifests 24 | COPY ./go.mod /workspace/ 25 | COPY ./go.sum /workspace/ 26 | # Copy the API Go Modules manifests 27 | COPY api/go.mod api/go.mod 28 | COPY api/go.sum api/go.sum 29 | 30 | RUN --mount=type=cache,target=/root/.cache/go-build --mount=type=cache,target=/go/pkg/mod go mod download 31 | 32 | COPY ./ /workspace/ 33 | 34 | # Build 35 | RUN --mount=type=cache,target=/root/.cache/go-build --mount=type=cache,target=/go/pkg/mod CGO_ENABLED=0 GOOS=linux GOARCH=${TARGETARCH} make binary 36 | 37 | # Use distroless as minimal base image to package the manager binary 38 | # Refer to https://github.com/GoogleContainerTools/distroless for more details 39 | FROM gcr.io/distroless/static:nonroot 40 | ARG GID 41 | ARG UID 42 | WORKDIR / 43 | COPY --from=builder /workspace/bin/manager . 44 | COPY --from=builder /etc/passwd /etc/passwd 45 | COPY --from=builder /etc/group /etc/group 46 | USER ${UID}:${GID} 47 | 48 | ENTRYPOINT ["/manager"] 49 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # Build variables 2 | VERSION ?= $(shell git symbolic-ref -q --short HEAD || git describe --tags --exact-match) 3 | COMMIT_HASH ?= $(shell git rev-parse --short HEAD 2>/dev/null) 4 | BUILD_DATE ?= $(shell date +%FT%T%z) 5 | UNAME := $(shell uname) 6 | LDFLAGS += -X main.version=${VERSION} -X main.commitHash=${COMMIT_HASH} -X main.buildDate=${BUILD_DATE} 7 | 8 | # Image URL to use all building/pushing image targets 9 | IMG ?= controller:latest 10 | 11 | # Produce CRDs that work back to Kubernetes 1.11 (no version conversion) 12 | CRD_OPTIONS ?= "crd:trivialVersions=true,preserveUnknownFields=false" 13 | REPO_ROOT=$(shell git rev-parse --show-toplevel) 14 | GOTESTSUM_VERSION = 0.6.0 15 | KUBEBUILDER_TOOLS_VERSION=1.19.2 16 | KUBEBUILDER_ASSETS_BINARY_DIR = kubebuilder-tools/${KUBEBUILDER_TOOLS_VERSION}/bin 17 | LICENSEI_VERSION = 0.5.0 18 | GOLANGCI_VERSION = 1.45.2 19 | CHART_NAME = cluster-registry 20 | 21 | # Get the currently used golang install path (in GOPATH/bin, unless GOBIN is set) 22 | ifeq (,$(shell go env GOBIN)) 23 | GOBIN=$(shell go env GOPATH)/bin 24 | else 25 | GOBIN=$(shell go env GOBIN) 26 | endif 27 | 28 | bin/gotestsum: bin/gotestsum-${GOTESTSUM_VERSION} 29 | @ln -sf gotestsum-${GOTESTSUM_VERSION} bin/gotestsum 30 | bin/gotestsum-${GOTESTSUM_VERSION}: 31 | @mkdir -p bin 32 | ifeq ($(UNAME), Darwin) 33 | curl -L https://github.com/gotestyourself/gotestsum/releases/download/v${GOTESTSUM_VERSION}/gotestsum_${GOTESTSUM_VERSION}_darwin_amd64.tar.gz | tar -zOxf - gotestsum > ./bin/gotestsum-${GOTESTSUM_VERSION} && chmod +x ./bin/gotestsum-${GOTESTSUM_VERSION} 34 | endif 35 | ifeq ($(UNAME), Linux) 36 | curl -L https://github.com/gotestyourself/gotestsum/releases/download/v${GOTESTSUM_VERSION}/gotestsum_${GOTESTSUM_VERSION}_linux_amd64.tar.gz | tar -zOxf - gotestsum > ./bin/gotestsum-${GOTESTSUM_VERSION} && chmod +x ./bin/gotestsum-${GOTESTSUM_VERSION} 37 | endif 38 | 39 | .PHONY: manifests 40 | manifests: ensure-tools ## Generate manifests 41 | cd api/v1alpha1 && ${REPO_ROOT}/bin/controller-gen $(CRD_OPTIONS) object:headerFile="${REPO_ROOT}/hack/boilerplate.go.txt" paths="./..." output:crd:artifacts:config=${REPO_ROOT}/deploy/charts/cluster-registry/crds 42 | 43 | .PHONY: test 44 | test: bin/gotestsum ensure-tools fmt vet # Run tests 45 | KUBEBUILDER_ATTACH_CONTROL_PLANE_OUTPUT=true KUBEBUILDER_ASSETS="$${PWD}/bin/${KUBEBUILDER_ASSETS_BINARY_DIR}" bin/gotestsum ./... -coverprofile cover.out 46 | cd deploy/charts && go test 47 | 48 | .PHONY: manager 49 | manager: fmt vet binary ## Build manager binary 50 | 51 | .PHONY: binary 52 | binary: 53 | go build -ldflags "${LDFLAGS}" -o bin/manager ./cmd/manager 54 | 55 | .PHONY: run 56 | run: fmt vet ## Run against the configured Kubernetes cluster in ~/.kube/config 57 | go run ./cmd/manager/ 58 | 59 | fmt: ## Run go fmt against code 60 | go fmt ./... 61 | 62 | vet: ## Run go vet against code 63 | go vet ./... 64 | 65 | .PHONY: ensure-tools 66 | ensure-tools: 67 | @echo "ensure tools" 68 | @scripts/download-deps.sh 69 | @scripts/install_kubebuilder_tools.sh --kubebuilder-tools-version=${KUBEBUILDER_TOOLS_VERSION} 70 | 71 | # This will clean up the local kubebuilder-tools directory and re-download the assets again. 72 | .PHONY: download-kubebuilder-tools 73 | download-kubebuilder-tools: 74 | @rm -r bin/kubebuilder-tools/${KUBEBUILDER_TOOLS_VERSION} 75 | @scripts/install_kubebuilder_tools.sh --kubebuilder-tools-version=${KUBEBUILDER_TOOLS_VERSION} 76 | 77 | # Build the docker image 78 | docker-build: 79 | docker build . -t ${IMG} --build-arg GITHUB_ACCESS_TOKEN="${GITHUB_ACCESS_TOKEN}" 80 | 81 | # Push the docker image 82 | docker-push: 83 | docker push ${IMG} 84 | 85 | bin/licensei: bin/licensei-${LICENSEI_VERSION} 86 | @ln -sf licensei-${LICENSEI_VERSION} bin/licensei 87 | bin/licensei-${LICENSEI_VERSION}: 88 | @mkdir -p bin 89 | curl -sfL https://raw.githubusercontent.com/goph/licensei/master/install.sh | bash -s v${LICENSEI_VERSION} 90 | @mv bin/licensei $@ 91 | 92 | .PHONY: license-check 93 | license-check: bin/licensei ## Run license check 94 | bin/licensei check 95 | bin/licensei header 96 | 97 | .PHONY: license-cache 98 | license-cache: bin/licensei ## Generate license cache 99 | bin/licensei cache 100 | 101 | bin/golangci-lint: bin/golangci-lint-${GOLANGCI_VERSION} 102 | @ln -sf golangci-lint-${GOLANGCI_VERSION} bin/golangci-lint 103 | bin/golangci-lint-${GOLANGCI_VERSION}: 104 | @mkdir -p bin 105 | curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | bash -s -- -b ./bin/ v${GOLANGCI_VERSION} 106 | @mv bin/golangci-lint $@ 107 | 108 | DISABLED_LINTERS ?= --disable=gci --disable=goimports --disable=gofumpt 109 | .PHONY: lint 110 | lint: bin/golangci-lint ## Run linter 111 | # "unused" linter is a memory hog, but running it separately keeps it contained (probably because of caching) 112 | bin/golangci-lint run --disable=unused -c .golangci.yml --timeout 2m 113 | bin/golangci-lint run -c .golangci.yml --timeout 2m 114 | 115 | .PHONY: lint-fix 116 | lint-fix: bin/golangci-lint ## Run linter & fix 117 | # "unused" linter is a memory hog, but running it separately keeps it contained (probably because of caching) 118 | bin/golangci-lint run --disable=unused -c .golangci.yml --fix 119 | bin/golangci-lint run -c .golangci.yml --fix 120 | 121 | .PHONY: help 122 | .DEFAULT_GOAL := help 123 | help: 124 | @grep -h -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' 125 | 126 | .PHONY: tidy 127 | tidy: ## Execute go mod tidy 128 | go mod tidy 129 | go mod download all 130 | -------------------------------------------------------------------------------- /PROJECT: -------------------------------------------------------------------------------- 1 | domain: k8s.cisco.com 2 | repo: github.com/cisco-open/cluster-registry-controller 3 | resources: 4 | - group: clusterregistry 5 | kind: Cluster 6 | version: v1alpha1 7 | - group: clusterregistry 8 | kind: ResourceSyncRule 9 | version: v1alpha1 10 | - group: clusterregistry 11 | kind: ClusterFeature 12 | version: v1alpha1 13 | version: "2" 14 | -------------------------------------------------------------------------------- /api/go.mod: -------------------------------------------------------------------------------- 1 | module github.com/cisco-open/cluster-registry-controller/api 2 | 3 | go 1.18 4 | 5 | require ( 6 | github.com/banzaicloud/operator-tools v0.21.0 7 | github.com/tidwall/gjson v1.9.3 8 | k8s.io/api v0.19.7 9 | k8s.io/apimachinery v0.19.7 10 | sigs.k8s.io/controller-runtime v0.6.2 11 | ) 12 | 13 | require ( 14 | emperror.dev/errors v0.8.0 // indirect 15 | github.com/briandowns/spinner v1.11.1 // indirect 16 | github.com/cppforlife/go-patch v0.2.0 // indirect 17 | github.com/fatih/color v1.7.0 // indirect 18 | github.com/go-logr/logr v0.2.1 // indirect 19 | github.com/gogo/protobuf v1.3.2 // indirect 20 | github.com/google/gofuzz v1.1.0 // indirect 21 | github.com/iancoleman/orderedmap v0.0.0-20190318233801-ac98e3ecb4b0 // indirect 22 | github.com/json-iterator/go v1.1.10 // indirect 23 | github.com/mattn/go-colorable v0.1.2 // indirect 24 | github.com/mattn/go-isatty v0.0.8 // indirect 25 | github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect 26 | github.com/modern-go/reflect2 v1.0.1 // indirect 27 | github.com/pkg/errors v0.9.1 // indirect 28 | github.com/spf13/cast v1.3.1 // indirect 29 | github.com/tidwall/match v1.1.1 // indirect 30 | github.com/tidwall/pretty v1.2.0 // indirect 31 | github.com/wayneashleyberry/terminal-dimensions v1.0.0 // indirect 32 | go.uber.org/atomic v1.7.0 // indirect 33 | go.uber.org/multierr v1.6.0 // indirect 34 | golang.org/x/net v0.0.0-20201110031124-69a78807bb2b // indirect 35 | golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f // indirect 36 | golang.org/x/text v0.3.8 // indirect 37 | gopkg.in/inf.v0 v0.9.1 // indirect 38 | gopkg.in/yaml.v2 v2.3.0 // indirect 39 | k8s.io/apiextensions-apiserver v0.19.2 // indirect 40 | k8s.io/klog/v2 v2.2.0 // indirect 41 | k8s.io/utils v0.0.0-20200729134348-d5654de09c73 // indirect 42 | sigs.k8s.io/structured-merge-diff/v4 v4.0.1 // indirect 43 | sigs.k8s.io/yaml v1.2.0 // indirect 44 | ) 45 | -------------------------------------------------------------------------------- /api/v1alpha1/cluster_feature_types.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package v1alpha1 16 | 17 | import ( 18 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 19 | ) 20 | 21 | // ClusterFeatureSpec defines the desired state of ClusterFeature 22 | type ClusterFeatureSpec struct { 23 | FeatureName string `json:"featureName"` 24 | } 25 | 26 | // ClusterFeatureStatus defines the observed state of ClusterFeature 27 | type ClusterFeatureStatus struct{} 28 | 29 | // +kubebuilder:object:root=true 30 | 31 | // ClusterFeature is the Schema for the clusterfeatures API 32 | // +kubebuilder:subresource:status 33 | // +kubebuilder:resource:path=clusterfeatures,scope=Cluster,shortName=cf 34 | // +kubebuilder:printcolumn:name="Feature",type="string",JSONPath=".spec.featureName" 35 | type ClusterFeature struct { 36 | metav1.TypeMeta `json:",inline"` 37 | metav1.ObjectMeta `json:"metadata,omitempty"` 38 | 39 | Spec ClusterFeatureSpec `json:"spec"` 40 | Status ClusterFeatureStatus `json:"status,omitempty"` 41 | } 42 | 43 | // +kubebuilder:object:root=true 44 | 45 | // ClusterFeatureList contains a list of ClusterFeature 46 | type ClusterFeatureList struct { 47 | metav1.TypeMeta `json:",inline"` 48 | metav1.ListMeta `json:"metadata,omitempty"` 49 | Items []ClusterFeature `json:"items"` 50 | } 51 | 52 | func init() { 53 | SchemeBuilder.Register(&ClusterFeature{}, &ClusterFeatureList{}) 54 | } 55 | -------------------------------------------------------------------------------- /api/v1alpha1/cluster_metadata.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package v1alpha1 16 | 17 | // +k8s:deepcopy-gen=true 18 | type ClusterMetadata struct { 19 | Provider string `json:"provider,omitempty"` 20 | Distribution string `json:"distribution,omitempty"` 21 | KubeProxyVersions []string `json:"kubeProxyVersions,omitempty"` 22 | KubeletVersions []string `json:"kubeletVersions,omitempty"` 23 | Version string `json:"version,omitempty"` 24 | Locality *Locality `json:"locality,omitempty"` 25 | } 26 | 27 | // +k8s:deepcopy-gen=true 28 | type Locality struct { 29 | Region string `json:"region,omitempty"` 30 | Regions []string `json:"regions,omitempty"` 31 | Zones []string `json:"zones,omitempty"` 32 | } 33 | -------------------------------------------------------------------------------- /api/v1alpha1/groupversion_info.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | // Package v1alpha1 contains API Schema definitions for the clusterregistry v1alpha1 API group 16 | // +kubebuilder:object:generate=true 17 | // +groupName=clusterregistry.k8s.cisco.com 18 | package v1alpha1 19 | 20 | import ( 21 | "k8s.io/apimachinery/pkg/runtime/schema" 22 | "sigs.k8s.io/controller-runtime/pkg/scheme" 23 | ) 24 | 25 | var ( 26 | // GroupVersion is group version used to register these objects 27 | GroupVersion = schema.GroupVersion{Group: "clusterregistry.k8s.cisco.com", Version: "v1alpha1"} 28 | 29 | // SchemeBuilder is used to add go types to the GroupVersionKind scheme 30 | SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} 31 | 32 | // AddToScheme adds the types in this group-version to the given scheme. 33 | AddToScheme = SchemeBuilder.AddToScheme 34 | ) 35 | -------------------------------------------------------------------------------- /api/v1alpha1/resourcesyncrule_types.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package v1alpha1 16 | 17 | import ( 18 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 19 | "k8s.io/apimachinery/pkg/util/intstr" 20 | 21 | "github.com/banzaicloud/operator-tools/pkg/resources" 22 | "github.com/banzaicloud/operator-tools/pkg/types" 23 | ) 24 | 25 | const ( 26 | OwnershipAnnotation = "cluster-registry.k8s.cisco.com/resource-owner-cluster-id" 27 | OriginalGVKAnnotation = "cluster-registry.k8s.cisco.com/original-group-version-kind" 28 | ClusterDisabledAnnotation = "cluster-registry.k8s.cisco.com/cluster-disabled" 29 | SyncDisabledAnnotation = "cluster-registry.k8s.cisco.com/resource-sync-disabled" 30 | ) 31 | 32 | type ResourceSyncRuleSpec struct { 33 | ClusterFeatureMatches []ClusterFeatureMatch `json:"clusterFeatureMatch,omitempty"` 34 | GVK resources.GroupVersionKind `json:"groupVersionKind"` 35 | Rules []SyncRule `json:"rules"` 36 | } 37 | 38 | type ClusterFeatureMatch struct { 39 | FeatureName string `json:"featureName,omitempty"` 40 | MatchLabels map[string]string `json:"matchLabels,omitempty"` 41 | MatchExpressions []metav1.LabelSelectorRequirement `json:"matchExpressions,omitempty"` 42 | } 43 | 44 | type SyncRule struct { 45 | Matches []SyncRuleMatch `json:"match,omitempty"` 46 | Mutations Mutations `json:"mutations,omitempty"` 47 | } 48 | 49 | type Mutations struct { 50 | Annotations *AnnotationMutations `json:"annotations,omitempty"` 51 | GVK *resources.GroupVersionKind `json:"groupVersionKind,omitempty"` 52 | Labels *LabelMutations `json:"labels,omitempty"` 53 | Overrides []resources.K8SResourceOverlayPatch `json:"overrides,omitempty"` 54 | SyncStatus bool `json:"syncStatus,omitempty"` 55 | } 56 | 57 | func (m Mutations) GetGVK() resources.GroupVersionKind { 58 | if m.GVK != nil { 59 | return *m.GVK 60 | } 61 | 62 | return resources.GroupVersionKind{} 63 | } 64 | 65 | func (m Mutations) GetAnnotations() AnnotationMutations { 66 | if m.Annotations != nil { 67 | return *m.Annotations 68 | } 69 | 70 | return AnnotationMutations{} 71 | } 72 | 73 | func (m Mutations) GetLabels() LabelMutations { 74 | if m.Labels != nil { 75 | return *m.Labels 76 | } 77 | 78 | return LabelMutations{} 79 | } 80 | 81 | type AnnotationMutations struct { 82 | Add map[string]string `json:"add,omitempty"` 83 | Remove []string `json:"remove,omitempty"` 84 | } 85 | 86 | type LabelMutations struct { 87 | Add map[string]string `json:"add,omitempty"` 88 | Remove []string `json:"remove,omitempty"` 89 | } 90 | 91 | type SyncRuleMatch struct { 92 | Annotations []AnnotationSelector `json:"annotations,omitempty"` 93 | Content []ContentSelector `json:"content,omitempty"` 94 | Labels []metav1.LabelSelector `json:"labels,omitempty"` 95 | Namespaces []string `json:"namespaces,omitempty"` 96 | ObjectKey types.ObjectKey `json:"objectKey,omitempty"` 97 | } 98 | 99 | type AnnotationSelector struct { 100 | MatchAnnotations map[string]string `json:"matchAnnotations,omitempty"` 101 | MatchExpressions []AnnotationSelectorRequirement `json:"matchExpressions,omitempty"` 102 | } 103 | 104 | type ContentSelector struct { 105 | Key string `json:"key"` 106 | Value intstr.IntOrString `json:"value"` 107 | } 108 | 109 | // +kubebuilder:validation:Pattern=`^(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?$` 110 | type AnnotationValue string 111 | 112 | // A annotation selector requirement is a selector that contains values, a key, and an operator that 113 | // relates the key and values. 114 | type AnnotationSelectorRequirement struct { 115 | // key is the label key that the selector applies to. 116 | // +patchMergeKey=key 117 | // +patchStrategy=merge 118 | Key string `json:"key" patchStrategy:"merge" patchMergeKey:"key" protobuf:"bytes,1,opt,name=key"` 119 | // operator represents a key's relationship to a set of values. 120 | // Valid operators are In, NotIn, Exists and DoesNotExist. 121 | Operator metav1.LabelSelectorOperator `json:"operator" protobuf:"bytes,2,opt,name=operator,casttype=LabelSelectorOperator"` 122 | // values is an array of string values. If the operator is In or NotIn, 123 | // the values array must be non-empty. If the operator is Exists or DoesNotExist, 124 | // the values array must be empty. This array is replaced during a strategic 125 | // merge patch. 126 | // +optional 127 | Values []AnnotationValue `json:"values,omitempty" protobuf:"bytes,3,rep,name=values"` 128 | } 129 | 130 | // An annotation selector operator is the set of operators that can be used in a selector requirement. 131 | type AnnotationSelectorOperator string 132 | 133 | const ( 134 | AnnotationSelectorOpExists AnnotationSelectorOperator = "Exists" 135 | AnnotationSelectorOpDoesNotExist AnnotationSelectorOperator = "DoesNotExist" 136 | ) 137 | 138 | type ResourceSyncRuleStatus struct{} 139 | 140 | // +kubebuilder:object:root=true 141 | 142 | // ResourceSyncRule is the Schema for the resource sync rule API 143 | // +kubebuilder:subresource:status 144 | // +kubebuilder:resource:path=resourcesyncrules,scope=Cluster,shortName=rsr 145 | type ResourceSyncRule struct { 146 | metav1.TypeMeta `json:",inline"` 147 | metav1.ObjectMeta `json:"metadata,omitempty"` 148 | 149 | Spec ResourceSyncRuleSpec `json:"spec,omitempty"` 150 | Status ResourceSyncRuleStatus `json:"status,omitempty"` 151 | } 152 | 153 | // +kubebuilder:object:root=true 154 | 155 | // ClusterList contains a list of Cluster 156 | type ResourceSyncRuleList struct { 157 | metav1.TypeMeta `json:",inline"` 158 | metav1.ListMeta `json:"metadata,omitempty"` 159 | Items []ResourceSyncRule `json:"items"` 160 | } 161 | 162 | func init() { 163 | SchemeBuilder.Register(&ResourceSyncRule{}, &ResourceSyncRuleList{}) 164 | } 165 | -------------------------------------------------------------------------------- /cmd/manager/build.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package main 16 | 17 | // Provisioned by ldflags 18 | // nolint: gochecknoglobals 19 | var ( 20 | version string 21 | commitHash string 22 | buildDate string 23 | ) 24 | -------------------------------------------------------------------------------- /cmd/manager/main.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package main 16 | 17 | import ( 18 | "context" 19 | "os" 20 | "time" 21 | 22 | "sigs.k8s.io/controller-runtime/pkg/healthz" 23 | 24 | "go.uber.org/zap/zapcore" 25 | "k8s.io/apimachinery/pkg/runtime" 26 | clientgoscheme "k8s.io/client-go/kubernetes/scheme" 27 | _ "k8s.io/client-go/plugin/pkg/client/auth/gcp" 28 | ctrl "sigs.k8s.io/controller-runtime" 29 | "sigs.k8s.io/controller-runtime/pkg/client" 30 | "sigs.k8s.io/controller-runtime/pkg/log/zap" 31 | "sigs.k8s.io/controller-runtime/pkg/webhook" 32 | 33 | "github.com/banzaicloud/operator-tools/pkg/logger" 34 | 35 | // +kubebuilder:scaffold:imports 36 | clusterregistryv1alpha1 "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 37 | "github.com/cisco-open/cluster-registry-controller/controllers" 38 | "github.com/cisco-open/cluster-registry-controller/internal/config" 39 | "github.com/cisco-open/cluster-registry-controller/pkg/cert" 40 | "github.com/cisco-open/cluster-registry-controller/pkg/clusters" 41 | "github.com/cisco-open/cluster-registry-controller/pkg/signals" 42 | "github.com/cisco-open/cluster-registry-controller/pkg/util" 43 | "github.com/cisco-open/cluster-registry-controller/pkg/webhooks" 44 | ) 45 | 46 | var ( 47 | scheme = runtime.NewScheme() 48 | setupLog = ctrl.Log.WithName("setup") 49 | ) 50 | 51 | const FriendlyServiceName = "cluster-registry" 52 | 53 | func init() { 54 | _ = clientgoscheme.AddToScheme(scheme) 55 | 56 | _ = clusterregistryv1alpha1.AddToScheme(scheme) 57 | // +kubebuilder:scaffold:scheme 58 | } 59 | 60 | func main() { 61 | configuration := configure() 62 | 63 | if configuration.Logging.Format == config.LogFormatConsole { 64 | logger.GlobalLogLevel = int(configuration.Logging.Verbosity) 65 | ctrl.SetLogger(logger.New(logger.WithTime(time.RFC3339))) // , logger.Out(ioutil.Discard))) 66 | } else { 67 | ctrl.SetLogger(zap.New( 68 | zap.UseDevMode(false), 69 | zap.Level(zapcore.Level(0-configuration.Logging.Verbosity)), 70 | )) 71 | } 72 | 73 | if configuration.ProvisionLocalCluster != "" { 74 | client, err := client.New(ctrl.GetConfigOrDie(), client.Options{ 75 | Scheme: scheme, 76 | }) 77 | if err != nil { 78 | setupLog.Error(err, "cannot connect to kubernetes cluster") 79 | os.Exit(1) 80 | } 81 | 82 | err = util.ProvisionLocalClusterObject(client, 83 | ctrl.Log.WithName("provision-local-cluster"), 84 | config.Configuration(configuration)) 85 | if err != nil { 86 | setupLog.Error(err, "cannot provision local cluster object") 87 | os.Exit(1) 88 | } 89 | } 90 | 91 | leaseDuration := configuration.LeaderElection.LeaseDuration 92 | renewDeadline := leaseDuration / 2 //nolint:gomnd 93 | 94 | options := ctrl.Options{ 95 | Scheme: scheme, 96 | MetricsBindAddress: configuration.MetricsAddr, 97 | LeaderElection: configuration.LeaderElection.Enabled, 98 | LeaderElectionID: configuration.LeaderElection.Name, 99 | LeaderElectionNamespace: configuration.LeaderElection.Namespace, 100 | LeaseDuration: &leaseDuration, 101 | RenewDeadline: &renewDeadline, 102 | LeaderElectionReleaseOnCancel: configuration.LeaderElection.ReleaseOnExit, 103 | HealthProbeBindAddress: configuration.HealthAddr, 104 | } 105 | 106 | if configuration.ClusterValidatorWebhook.Enabled { 107 | options.CertDir = configuration.ClusterValidatorWebhook.CertificateDirectory 108 | options.Port = int(configuration.ClusterValidatorWebhook.Port) 109 | } 110 | 111 | mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), options) 112 | if err != nil { 113 | setupLog.Error(err, "unable to start manager") 114 | os.Exit(1) 115 | } 116 | 117 | // Use ping readiness if webhook is not enabled 118 | readyzCheckSelector := healthz.Ping 119 | 120 | if configuration.ClusterValidatorWebhook.Enabled { 121 | clusterValidatorLogger := ctrl.Log.WithName("cluster-validator") 122 | 123 | mgr.GetWebhookServer().Register( 124 | "/validate-cluster", 125 | &webhook.Admission{ 126 | Handler: webhooks.NewClusterValidator(clusterValidatorLogger, mgr), 127 | }, 128 | ) 129 | 130 | clusterValidatorCertRenewer, err := cert.NewRenewer( 131 | clusterValidatorLogger, 132 | nil, 133 | configuration.ClusterValidatorWebhook.CertificateDirectory, 134 | true, 135 | ) 136 | if err != nil { 137 | setupLog.Error(err, "initializing certificate renewer failed") 138 | 139 | os.Exit(1) 140 | } 141 | 142 | clusterWebhookCertifier := cert.NewWebhookCertifier( 143 | clusterValidatorLogger, 144 | configuration.ClusterValidatorWebhook.Name, 145 | configuration.Namespace, 146 | mgr, 147 | clusterValidatorCertRenewer, 148 | ) 149 | err = mgr.Add(clusterWebhookCertifier) 150 | if err != nil { 151 | setupLog.Error(err, "adding certificate provisioner to manager failed") 152 | 153 | os.Exit(1) 154 | } 155 | } 156 | 157 | ctx := signals.NotifyContext(context.Background()) 158 | 159 | clustersManager := clusters.NewManager(ctx) 160 | 161 | if err = controllers.NewResourceSyncRuleReconciler("resource-sync-rules", ctrl.Log.WithName("controllers").WithName("resource-sync-rule"), clustersManager, config.Configuration(configuration)).SetupWithManager(ctx, mgr); err != nil { 162 | setupLog.Error(err, "unable to create controller", "controller", "resource-sync-rule") 163 | os.Exit(1) 164 | } 165 | 166 | if err = controllers.NewClusterReconciler("clusters", ctrl.Log.WithName("controllers").WithName("cluster"), clustersManager, config.Configuration(configuration)).SetupWithManager(ctx, mgr); err != nil { 167 | setupLog.Error(err, "unable to create controller", "controller", "cluster") 168 | os.Exit(1) 169 | } 170 | // +kubebuilder:scaffold:builder 171 | 172 | if err = mgr.AddReadyzCheck("readyz", readyzCheckSelector); err != nil { 173 | setupLog.Error(err, "unable to set up ready check") 174 | os.Exit(1) 175 | } 176 | 177 | setupLog.Info("starting manager") 178 | if err := mgr.Start(ctx); err != nil { 179 | setupLog.Error(err, "problem running manager") 180 | os.Exit(1) 181 | } 182 | } 183 | -------------------------------------------------------------------------------- /controllers/cluster_conditions.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package controllers 16 | 17 | import ( 18 | "time" 19 | 20 | corev1 "k8s.io/api/core/v1" 21 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 22 | "k8s.io/client-go/tools/record" 23 | 24 | clusterregistryv1alpha1 "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 25 | ) 26 | 27 | type ClusterConditionsMap map[clusterregistryv1alpha1.ClusterConditionType]clusterregistryv1alpha1.ClusterCondition 28 | 29 | func SetCondition(cluster *clusterregistryv1alpha1.Cluster, currentConditions ClusterConditionsMap, condition clusterregistryv1alpha1.ClusterCondition, recorder record.EventRecorder) { 30 | var ok bool 31 | var currentCondition clusterregistryv1alpha1.ClusterCondition 32 | 33 | if currentCondition, ok = currentConditions[condition.Type]; !ok { 34 | currentCondition = condition 35 | } else { 36 | if currentCondition.Status != condition.Status { 37 | currentCondition.LastTransitionTime = metav1.NewTime(time.Now()) 38 | eventType := corev1.EventTypeNormal 39 | if (condition.TrueIsFailure && condition.Status == corev1.ConditionTrue) || (!condition.TrueIsFailure && condition.Status == corev1.ConditionFalse) { 40 | eventType = corev1.EventTypeWarning 41 | } 42 | recorder.Event(cluster, eventType, condition.Reason, condition.Message) 43 | } 44 | currentCondition.Message = condition.Message 45 | currentCondition.Reason = condition.Reason 46 | currentCondition.Status = condition.Status 47 | } 48 | 49 | if currentCondition.LastTransitionTime.IsZero() { 50 | currentCondition.LastTransitionTime = metav1.NewTime(time.Now()) 51 | } 52 | currentCondition.LastHeartbeatTime = metav1.NewTime(time.Now()) 53 | 54 | currentConditions[condition.Type] = currentCondition 55 | } 56 | 57 | func GetCurrentConditions(cluster *clusterregistryv1alpha1.Cluster) ClusterConditionsMap { 58 | currentConditions := make(ClusterConditionsMap) 59 | for _, condition := range cluster.Status.Conditions { 60 | currentConditions[condition.Type] = condition 61 | } 62 | 63 | return currentConditions 64 | } 65 | 66 | func GetCurrentCondition(cluster *clusterregistryv1alpha1.Cluster, t clusterregistryv1alpha1.ClusterConditionType) clusterregistryv1alpha1.ClusterCondition { 67 | for _, condition := range cluster.Status.Conditions { 68 | if condition.Type == t { 69 | return condition 70 | } 71 | } 72 | 73 | return clusterregistryv1alpha1.ClusterCondition{} 74 | } 75 | 76 | func LocalClusterCondition(isClusterLocal bool) clusterregistryv1alpha1.ClusterCondition { 77 | condition := clusterregistryv1alpha1.ClusterCondition{ 78 | Type: clusterregistryv1alpha1.ClusterConditionTypeLocalCluster, 79 | Status: corev1.ConditionUnknown, 80 | } 81 | 82 | if isClusterLocal { 83 | condition.Reason = "ClusterIsLocal" 84 | condition.Message = "cluster is local" 85 | condition.Status = corev1.ConditionTrue 86 | 87 | return condition 88 | } 89 | 90 | condition.Reason = "ClusterIsNotLocal" 91 | condition.Message = "cluster is not local" 92 | condition.Status = corev1.ConditionFalse 93 | 94 | return condition 95 | } 96 | 97 | func LocalClusterConflictCondition(conflict bool) clusterregistryv1alpha1.ClusterCondition { 98 | condition := clusterregistryv1alpha1.ClusterCondition{ 99 | Type: clusterregistryv1alpha1.ClusterConditionTypeLocalConflict, 100 | Status: corev1.ConditionUnknown, 101 | 102 | TrueIsFailure: true, 103 | } 104 | 105 | if conflict { 106 | condition.Reason = "LocalClusterIsConflicting" 107 | condition.Message = ErrLocalClusterConflict.Error() 108 | condition.Status = corev1.ConditionTrue 109 | 110 | return condition 111 | } 112 | 113 | condition.Reason = "LocalClusterIsNotConflicting" 114 | condition.Message = "only one local cluster is defined" 115 | condition.Status = corev1.ConditionFalse 116 | 117 | return condition 118 | } 119 | 120 | func ClusterMetadataCondition(err error) clusterregistryv1alpha1.ClusterCondition { 121 | condition := clusterregistryv1alpha1.ClusterCondition{ 122 | Type: clusterregistryv1alpha1.ClusterConditionTypeClusterMetadata, 123 | Status: corev1.ConditionUnknown, 124 | } 125 | 126 | if err == nil { 127 | condition.Reason = "ClusterMetadataSet" 128 | condition.Message = "cluster metadata is set" 129 | condition.Status = corev1.ConditionTrue 130 | 131 | return condition 132 | } 133 | 134 | condition.Reason = "ClusterMetadataIsNotSet" 135 | condition.Message = err.Error() 136 | condition.Status = corev1.ConditionFalse 137 | 138 | return condition 139 | } 140 | 141 | func ClusterReadyCondition(err error) clusterregistryv1alpha1.ClusterCondition { 142 | condition := clusterregistryv1alpha1.ClusterCondition{ 143 | Type: clusterregistryv1alpha1.ClusterConditionTypeReady, 144 | Status: corev1.ConditionUnknown, 145 | } 146 | 147 | if err == nil { 148 | condition.Reason = "ClusterIsReady" 149 | condition.Message = "cluster is ready" 150 | condition.Status = corev1.ConditionTrue 151 | 152 | return condition 153 | } 154 | 155 | condition.Reason = "ClusterIsNotReady" 156 | condition.Message = err.Error() 157 | condition.Status = corev1.ConditionFalse 158 | 159 | return condition 160 | } 161 | 162 | func ClustersSyncedCondition(err error) clusterregistryv1alpha1.ClusterCondition { 163 | condition := clusterregistryv1alpha1.ClusterCondition{ 164 | Type: clusterregistryv1alpha1.ClusterConditionTypeClustersSynced, 165 | Status: corev1.ConditionUnknown, 166 | } 167 | 168 | if err == nil { 169 | condition.Reason = "ClustersAreSynced" 170 | condition.Message = "all participating clusters are in sync" 171 | condition.Status = corev1.ConditionTrue 172 | 173 | return condition 174 | } 175 | 176 | condition.Reason = "ClustersAreNotSynced" 177 | condition.Message = err.Error() 178 | condition.Status = corev1.ConditionFalse 179 | 180 | return condition 181 | } 182 | -------------------------------------------------------------------------------- /controllers/cluster_reconciler_test.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package controllers_test 16 | 17 | import ( 18 | "context" 19 | "time" 20 | 21 | . "github.com/onsi/ginkgo" 22 | . "github.com/onsi/gomega" 23 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 24 | "k8s.io/apimachinery/pkg/types" 25 | 26 | clusterregistryv1alpha1 "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 27 | ) 28 | 29 | var _ = Describe("Cluster controller", func() { 30 | const ( 31 | ClusterName = "demo" 32 | 33 | timeout = time.Second * 10 34 | // duration = time.Second * 10 35 | interval = time.Millisecond * 250 36 | ) 37 | 38 | It("reconciles objects properly", func() { 39 | By("By creating a new Cluster") 40 | ctx := context.Background() 41 | cluster := &clusterregistryv1alpha1.Cluster{ 42 | TypeMeta: metav1.TypeMeta{ 43 | APIVersion: clusterregistryv1alpha1.GroupVersion.String(), 44 | Kind: "Cluster", 45 | }, 46 | ObjectMeta: metav1.ObjectMeta{ 47 | Name: ClusterName, 48 | }, 49 | Spec: clusterregistryv1alpha1.ClusterSpec{ 50 | ClusterID: "abc", 51 | }, 52 | } 53 | Expect(k8sClient.Create(ctx, cluster)).Should(Succeed()) 54 | 55 | createdCluster := &clusterregistryv1alpha1.Cluster{} 56 | Eventually(func() bool { 57 | err := k8sClient.Get(ctx, types.NamespacedName{ 58 | Name: cluster.Name, 59 | }, createdCluster) 60 | 61 | return err == nil 62 | }, timeout, interval).Should(BeTrue()) 63 | Expect(createdCluster.Spec.ClusterID).Should(Equal(cluster.Spec.ClusterID)) 64 | }) 65 | }) 66 | -------------------------------------------------------------------------------- /controllers/cluster_reconciler_watches.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package controllers 16 | 17 | import ( 18 | "context" 19 | 20 | corev1 "k8s.io/api/core/v1" 21 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 22 | "k8s.io/apimachinery/pkg/types" 23 | ctrl "sigs.k8s.io/controller-runtime" 24 | "sigs.k8s.io/controller-runtime/pkg/builder" 25 | "sigs.k8s.io/controller-runtime/pkg/client" 26 | "sigs.k8s.io/controller-runtime/pkg/event" 27 | "sigs.k8s.io/controller-runtime/pkg/handler" 28 | "sigs.k8s.io/controller-runtime/pkg/predicate" 29 | "sigs.k8s.io/controller-runtime/pkg/reconcile" 30 | "sigs.k8s.io/controller-runtime/pkg/source" 31 | 32 | clusterregistryv1alpha1 "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 33 | ) 34 | 35 | func (r *ClusterReconciler) watchLocalClustersForConflict(ctx context.Context, b *builder.Builder) { 36 | b.Watches( 37 | &source.Kind{Type: &clusterregistryv1alpha1.Cluster{ 38 | TypeMeta: metav1.TypeMeta{ 39 | Kind: "Cluster", 40 | APIVersion: clusterregistryv1alpha1.SchemeBuilder.GroupVersion.String(), 41 | }, 42 | }}, 43 | handler.EnqueueRequestsFromMapFunc(func(object client.Object) []ctrl.Request { 44 | reqs := make([]reconcile.Request, 0) 45 | clusters, err := GetClusters(ctx, r.GetClient()) 46 | if err != nil { 47 | r.GetLogger().Error(err, "") 48 | 49 | return nil 50 | } 51 | 52 | for _, c := range clusters { 53 | nsn := types.NamespacedName{ 54 | Name: c.Name, 55 | Namespace: c.Namespace, 56 | } 57 | if c.Status.Type == clusterregistryv1alpha1.ClusterTypeLocal { 58 | reqs = append(reqs, ctrl.Request{ 59 | NamespacedName: nsn, 60 | }) 61 | } 62 | } 63 | 64 | return reqs 65 | }), 66 | builder.WithPredicates(&predicate.Funcs{ 67 | GenericFunc: func(event.GenericEvent) bool { return false }, 68 | UpdateFunc: func(event.UpdateEvent) bool { return false }, 69 | })) 70 | } 71 | 72 | func (r *ClusterReconciler) watchClusterRegistrySecrets(ctx context.Context, b *builder.Builder) { 73 | b.Watches( 74 | &source.Kind{Type: &corev1.Secret{ 75 | TypeMeta: metav1.TypeMeta{ 76 | Kind: "Secret", 77 | APIVersion: corev1.SchemeGroupVersion.String(), 78 | }, 79 | }}, 80 | handler.EnqueueRequestsFromMapFunc(func(object client.Object) []ctrl.Request { 81 | reqs := make([]reconcile.Request, 0) 82 | if secret, ok := object.(*corev1.Secret); ok { 83 | if secret.Type != clusterregistryv1alpha1.SecretTypeClusterRegistry { 84 | return nil 85 | } 86 | clusters, err := GetClusters(ctx, r.GetClient()) 87 | if err != nil { 88 | r.GetLogger().Error(err, "") 89 | 90 | return nil 91 | } 92 | 93 | for _, c := range clusters { 94 | nsn := types.NamespacedName{ 95 | Name: c.Name, 96 | Namespace: c.Namespace, 97 | } 98 | if c.Spec.AuthInfo.SecretRef.Name == secret.Name && c.Spec.AuthInfo.SecretRef.Namespace == secret.Namespace { 99 | reqs = append(reqs, ctrl.Request{ 100 | NamespacedName: nsn, 101 | }) 102 | } 103 | } 104 | 105 | return reqs 106 | } 107 | 108 | return nil 109 | }), 110 | ) 111 | } 112 | -------------------------------------------------------------------------------- /controllers/common.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package controllers 16 | 17 | import ( 18 | "context" 19 | 20 | "emperror.dev/errors" 21 | "github.com/go-logr/logr" 22 | corev1 "k8s.io/api/core/v1" 23 | apierrors "k8s.io/apimachinery/pkg/api/errors" 24 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 25 | "k8s.io/apimachinery/pkg/types" 26 | "k8s.io/client-go/util/workqueue" 27 | "sigs.k8s.io/controller-runtime/pkg/client" 28 | "sigs.k8s.io/controller-runtime/pkg/handler" 29 | "sigs.k8s.io/controller-runtime/pkg/predicate" 30 | 31 | clusterregistryv1alpha1 "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 32 | ) 33 | 34 | type QueueAwareReconciler interface { 35 | setQueue(q workqueue.RateLimitingInterface) 36 | } 37 | 38 | type InMemorySource struct { 39 | reconciler QueueAwareReconciler 40 | } 41 | 42 | func (s *InMemorySource) String() string { 43 | return "in-memory" 44 | } 45 | 46 | func (s *InMemorySource) Start(ctx context.Context, h handler.EventHandler, q workqueue.RateLimitingInterface, p ...predicate.Predicate) error { 47 | s.reconciler.setQueue(q) 48 | 49 | return nil 50 | } 51 | 52 | func GetClusters(ctx context.Context, c client.Client) (map[types.UID]clusterregistryv1alpha1.Cluster, error) { 53 | clusterList := &clusterregistryv1alpha1.ClusterList{} 54 | err := c.List(ctx, clusterList) 55 | if err != nil { 56 | return nil, errors.WrapIf(err, "could not list clusters") 57 | } 58 | 59 | clusters := make(map[types.UID]clusterregistryv1alpha1.Cluster) 60 | for _, c := range clusterList.Items { 61 | clusters[c.Spec.ClusterID] = c 62 | } 63 | 64 | return clusters, nil 65 | } 66 | 67 | func GetClusterID(ctx context.Context, c client.Client) (types.UID, error) { 68 | ns := &corev1.Namespace{} 69 | err := c.Get(ctx, client.ObjectKey{ 70 | Name: metav1.NamespaceSystem, 71 | }, ns) 72 | if err != nil { 73 | return "", err 74 | } 75 | 76 | return ns.UID, nil 77 | } 78 | 79 | func UpdateCluster(ctx context.Context, reconcileError error, c client.Client, cluster *clusterregistryv1alpha1.Cluster, currentConditions ClusterConditionsMap, log logr.Logger) error { 80 | conditions := make([]clusterregistryv1alpha1.ClusterCondition, 0) 81 | for _, condition := range currentConditions { 82 | // remove sync condition from local clusters 83 | if cluster.Status.Type == clusterregistryv1alpha1.ClusterTypeLocal && condition.Type == clusterregistryv1alpha1.ClusterConditionTypeClustersSynced { 84 | continue 85 | } 86 | conditions = append(conditions, condition) 87 | } 88 | cluster.Status.Conditions = conditions 89 | 90 | if reconcileError != nil { 91 | // status could be overwritten earlier, only override the ready state here 92 | if cluster.Status.State == clusterregistryv1alpha1.ClusterStateReady { 93 | cluster.Status.State = clusterregistryv1alpha1.ClusterStateFailed 94 | } 95 | cluster.Status.Message = reconcileError.Error() 96 | updateErr := UpdateClusterStatus(ctx, c, cluster, log) 97 | if updateErr != nil { 98 | log.Error(updateErr, "could not update resource status") 99 | } 100 | //nolint:errorlint 101 | if e, ok := reconcileError.(interface{ IsPermanent() bool }); ok && e.IsPermanent() { 102 | log.Error(reconcileError, "", errors.GetDetails(reconcileError)...) 103 | reconcileError = nil 104 | } 105 | 106 | return reconcileError 107 | } 108 | 109 | err := UpdateClusterStatus(ctx, c, cluster, log) 110 | if err != nil { 111 | return errors.WithStackIf(err) 112 | } 113 | 114 | return nil 115 | } 116 | 117 | func UpdateClusterStatus(ctx context.Context, c client.Client, cluster *clusterregistryv1alpha1.Cluster, log logr.Logger) error { 118 | desired := cluster.DeepCopy() 119 | 120 | log.Info("update cluster status") 121 | 122 | err := c.Status().Update(ctx, cluster) 123 | if apierrors.IsConflict(err) { 124 | current := cluster.DeepCopy() 125 | err = c.Get(ctx, client.ObjectKey{ 126 | Name: current.Name, 127 | }, current) 128 | if err != nil { 129 | return errors.WrapIf(err, "could not get cluster") 130 | } 131 | desired.SetResourceVersion(current.GetResourceVersion()) 132 | err = c.Status().Update(ctx, desired) 133 | if err != nil { 134 | return errors.WrapIf(err, "could not update cluster status") 135 | } 136 | } 137 | if err != nil { 138 | return errors.WrapIf(err, "could not update status") 139 | } 140 | 141 | return nil 142 | } 143 | -------------------------------------------------------------------------------- /controllers/core_syncers.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package controllers 16 | 17 | import ( 18 | "emperror.dev/errors" 19 | corev1 "k8s.io/api/core/v1" 20 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 21 | "k8s.io/apimachinery/pkg/util/intstr" 22 | "sigs.k8s.io/controller-runtime/pkg/client" 23 | "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" 24 | 25 | "github.com/banzaicloud/operator-tools/pkg/reconciler" 26 | "github.com/banzaicloud/operator-tools/pkg/resources" 27 | clusterregistryv1alpha1 "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 28 | ) 29 | 30 | const ( 31 | CoreResourcesSourceFeatureName = "cluster-registry-core-resources-source" 32 | CoreResourceLabelName = "cluster-registry-controller.k8s.cisco.com/core-sync-resource" 33 | ) 34 | 35 | func (r *ClusterReconciler) reconcileCoreSyncers(cluster *clusterregistryv1alpha1.Cluster) error { 36 | rec := reconciler.NewGenericReconciler( 37 | r.GetManager().GetClient(), 38 | r.GetLogger(), 39 | reconciler.ReconcilerOpts{ 40 | EnableRecreateWorkloadOnImmutableFieldChange: true, 41 | Scheme: r.GetManager().GetScheme(), 42 | }, 43 | ) 44 | 45 | clusterFeatureDesiredState := reconciler.StateAbsent 46 | if r.config.CoreResourcesSourceEnabled { 47 | clusterFeatureDesiredState = reconciler.StatePresent 48 | } 49 | 50 | for o, ds := range map[client.Object]reconciler.DesiredState{ 51 | r.coreSyncersClusterFeature(): clusterFeatureDesiredState, 52 | r.clustersSyncRule(): reconciler.StatePresent, 53 | r.clusterSecretsSyncRule(): reconciler.StatePresent, 54 | r.resourceSyncRuleSync(): reconciler.StatePresent, 55 | } { 56 | // set resource owner 57 | if err := controllerutil.SetControllerReference(cluster, o, r.GetManager().GetScheme()); err != nil { 58 | return errors.WrapIfWithDetails(err, "couldn't set local Cluster as resource owner for resource", 59 | "namespace", o.GetNamespace(), "name", o.GetName()) 60 | } 61 | _, err := rec.ReconcileResource(o, ds) 62 | if err != nil { 63 | return err 64 | } 65 | } 66 | 67 | return nil 68 | } 69 | 70 | func (r *ClusterReconciler) coreSyncersClusterFeature() *clusterregistryv1alpha1.ClusterFeature { 71 | return &clusterregistryv1alpha1.ClusterFeature{ 72 | ObjectMeta: metav1.ObjectMeta{ 73 | Name: "cluster-registry-core-resources", 74 | Labels: map[string]string{ 75 | CoreResourceLabelName: "true", 76 | }, 77 | }, 78 | Spec: clusterregistryv1alpha1.ClusterFeatureSpec{ 79 | FeatureName: CoreResourcesSourceFeatureName, 80 | }, 81 | } 82 | } 83 | 84 | func (r *ClusterReconciler) clustersSyncRule() *clusterregistryv1alpha1.ResourceSyncRule { 85 | return &clusterregistryv1alpha1.ResourceSyncRule{ 86 | ObjectMeta: metav1.ObjectMeta{ 87 | Name: "cluster-registry-core-resource-clusters-sink", 88 | Labels: map[string]string{ 89 | CoreResourceLabelName: "true", 90 | }, 91 | }, 92 | Spec: clusterregistryv1alpha1.ResourceSyncRuleSpec{ 93 | ClusterFeatureMatches: []clusterregistryv1alpha1.ClusterFeatureMatch{ 94 | { 95 | FeatureName: CoreResourcesSourceFeatureName, 96 | }, 97 | }, 98 | GVK: resources.GroupVersionKind(clusterregistryv1alpha1.SchemeBuilder.GroupVersion.WithKind("Cluster")), 99 | Rules: []clusterregistryv1alpha1.SyncRule{ 100 | { 101 | Mutations: clusterregistryv1alpha1.Mutations{ 102 | SyncStatus: false, 103 | }, 104 | }, 105 | }, 106 | }, 107 | } 108 | } 109 | 110 | func (r *ClusterReconciler) clusterSecretsSyncRule() *clusterregistryv1alpha1.ResourceSyncRule { 111 | return &clusterregistryv1alpha1.ResourceSyncRule{ 112 | ObjectMeta: metav1.ObjectMeta{ 113 | Name: "cluster-registry-core-resource-cluster-secrets-sink", 114 | Labels: map[string]string{ 115 | CoreResourceLabelName: "true", 116 | }, 117 | }, 118 | Spec: clusterregistryv1alpha1.ResourceSyncRuleSpec{ 119 | ClusterFeatureMatches: []clusterregistryv1alpha1.ClusterFeatureMatch{ 120 | { 121 | FeatureName: CoreResourcesSourceFeatureName, 122 | }, 123 | }, 124 | GVK: resources.GroupVersionKind(corev1.SchemeGroupVersion.WithKind("Secret")), 125 | Rules: []clusterregistryv1alpha1.SyncRule{ 126 | { 127 | Matches: []clusterregistryv1alpha1.SyncRuleMatch{ 128 | { 129 | Content: []clusterregistryv1alpha1.ContentSelector{ 130 | { 131 | Key: "type", 132 | Value: intstr.FromString(string(clusterregistryv1alpha1.SecretTypeClusterRegistry)), 133 | }, 134 | }, 135 | Namespaces: []string{ 136 | r.config.Namespace, 137 | }, 138 | }, 139 | }, 140 | Mutations: clusterregistryv1alpha1.Mutations{ 141 | SyncStatus: false, 142 | }, 143 | }, 144 | }, 145 | }, 146 | } 147 | } 148 | 149 | func (r *ClusterReconciler) resourceSyncRuleSync() *clusterregistryv1alpha1.ResourceSyncRule { 150 | return &clusterregistryv1alpha1.ResourceSyncRule{ 151 | ObjectMeta: metav1.ObjectMeta{ 152 | Name: "cluster-registry-core-resource-syncrules-sink", 153 | Labels: map[string]string{ 154 | CoreResourceLabelName: "true", 155 | }, 156 | }, 157 | Spec: clusterregistryv1alpha1.ResourceSyncRuleSpec{ 158 | ClusterFeatureMatches: []clusterregistryv1alpha1.ClusterFeatureMatch{ 159 | { 160 | FeatureName: CoreResourcesSourceFeatureName, 161 | }, 162 | }, 163 | GVK: resources.GroupVersionKind(clusterregistryv1alpha1.SchemeBuilder.GroupVersion.WithKind("ResourceSyncRule")), 164 | Rules: []clusterregistryv1alpha1.SyncRule{ 165 | { 166 | Mutations: clusterregistryv1alpha1.Mutations{ 167 | SyncStatus: false, 168 | }, 169 | }, 170 | }, 171 | }, 172 | } 173 | } 174 | -------------------------------------------------------------------------------- /controllers/errors.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package controllers 16 | 17 | import ( 18 | "runtime" 19 | 20 | "emperror.dev/errors" 21 | ) 22 | 23 | var ( 24 | ErrInvalidClusterID = errors.New("invalid cluster ID") 25 | ErrInvalidSecretContent = errors.New("could not found k8s config in secret") 26 | ErrInvalidSecret = errors.New("invalid secret type") 27 | ErrLocalClusterConflict = errors.New("multiple local clusters are defined") 28 | ) 29 | 30 | func WrapAsPermanentError(err error) error { 31 | return &permanentError{err, callers(2)} 32 | } 33 | 34 | type permanentError struct { 35 | error 36 | *stack 37 | } 38 | 39 | func (w *permanentError) Cause() error { return w.error } 40 | 41 | // Unwrap provides compatibility for Go 1.13 error chains. 42 | func (w *permanentError) Unwrap() error { return w.error } 43 | 44 | func (w *permanentError) IsPermanent() bool { 45 | return true 46 | } 47 | 48 | type ( 49 | StackTrace = errors.StackTrace 50 | Frame = errors.Frame 51 | stack []uintptr 52 | ) 53 | 54 | // callers is based on the function with the same name in github.com/pkg/errors, 55 | // but accepts a custom depth (useful to customize the error constructor caller depth). 56 | func callers(depth int) *stack { 57 | const maxDepth = 32 58 | 59 | var pcs [maxDepth]uintptr 60 | 61 | n := runtime.Callers(2+depth, pcs[:]) 62 | 63 | var st stack = pcs[0:n] 64 | 65 | return &st 66 | } 67 | 68 | func (s *stack) StackTrace() StackTrace { 69 | f := make([]Frame, len(*s)) 70 | for i := 0; i < len(f); i++ { 71 | f[i] = Frame((*s)[i]) 72 | } 73 | 74 | return f 75 | } 76 | -------------------------------------------------------------------------------- /controllers/remote_cluster_feature_reconciler.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package controllers 16 | 17 | import ( 18 | "context" 19 | 20 | "emperror.dev/errors" 21 | "github.com/go-logr/logr" 22 | apierrors "k8s.io/apimachinery/pkg/api/errors" 23 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 24 | ctrl "sigs.k8s.io/controller-runtime" 25 | "sigs.k8s.io/controller-runtime/pkg/controller" 26 | "sigs.k8s.io/controller-runtime/pkg/handler" 27 | "sigs.k8s.io/controller-runtime/pkg/predicate" 28 | "sigs.k8s.io/controller-runtime/pkg/source" 29 | 30 | clusterregistryv1alpha1 "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 31 | "github.com/cisco-open/cluster-registry-controller/pkg/clusters" 32 | ) 33 | 34 | type ClusterFeatureReconciler struct { 35 | clusters.ManagedReconciler 36 | 37 | cluster *clusters.Cluster 38 | } 39 | 40 | func NewClusterFeatureReconciler(name string, cluster *clusters.Cluster, log logr.Logger) *ClusterFeatureReconciler { 41 | return &ClusterFeatureReconciler{ 42 | ManagedReconciler: clusters.NewManagedReconciler(name, log), 43 | 44 | cluster: cluster, 45 | } 46 | } 47 | 48 | func (r *ClusterFeatureReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { 49 | log := r.GetLogger().WithValues("clusterFeature", req.NamespacedName, "cluster", r.cluster.GetName()) 50 | 51 | feature := &clusterregistryv1alpha1.ClusterFeature{} 52 | err := r.GetClient().Get(ctx, req.NamespacedName, feature) 53 | if apierrors.IsNotFound(err) { 54 | log.Info("delete feature") 55 | r.cluster.RemoveFeature(req.NamespacedName.String()) 56 | 57 | return ctrl.Result{}, nil 58 | } 59 | if err != nil { 60 | return ctrl.Result{}, errors.WrapIf(err, "could not get object") 61 | } 62 | 63 | log.Info("add feature") 64 | r.cluster.AddFeature(clusters.NewClusterFeature(req.NamespacedName.String(), feature.Spec.FeatureName, feature.GetLabels())) 65 | 66 | return ctrl.Result{}, nil 67 | } 68 | 69 | func (r *ClusterFeatureReconciler) SetupWithController(ctx context.Context, ctrl controller.Controller) error { 70 | err := r.ManagedReconciler.SetupWithController(ctx, ctrl) 71 | if err != nil { 72 | return err 73 | } 74 | 75 | err = ctrl.Watch( 76 | &source.Kind{ 77 | Type: &clusterregistryv1alpha1.ClusterFeature{ 78 | TypeMeta: metav1.TypeMeta{ 79 | Kind: "ClusterFeature", 80 | APIVersion: clusterregistryv1alpha1.SchemeBuilder.GroupVersion.String(), 81 | }, 82 | }, 83 | }, 84 | &handler.EnqueueRequestForObject{}, 85 | predicate.Funcs{}, 86 | ) 87 | if err != nil { 88 | return errors.WrapIf(err, "could not create watch for cluster features") 89 | } 90 | 91 | return nil 92 | } 93 | -------------------------------------------------------------------------------- /controllers/suite_test.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package controllers_test 16 | 17 | import ( 18 | "context" 19 | "path/filepath" 20 | "testing" 21 | 22 | . "github.com/onsi/ginkgo" 23 | . "github.com/onsi/gomega" 24 | "k8s.io/client-go/kubernetes/scheme" 25 | "k8s.io/client-go/rest" 26 | ctrl "sigs.k8s.io/controller-runtime" 27 | "sigs.k8s.io/controller-runtime/pkg/client" 28 | "sigs.k8s.io/controller-runtime/pkg/envtest" 29 | "sigs.k8s.io/controller-runtime/pkg/envtest/printer" 30 | logf "sigs.k8s.io/controller-runtime/pkg/log" 31 | "sigs.k8s.io/controller-runtime/pkg/log/zap" 32 | 33 | // +kubebuilder:scaffold:imports 34 | clusterregistryv1alpha1 "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 35 | "github.com/cisco-open/cluster-registry-controller/controllers" 36 | "github.com/cisco-open/cluster-registry-controller/internal/config" 37 | "github.com/cisco-open/cluster-registry-controller/pkg/clusters" 38 | ) 39 | 40 | // These tests use Ginkgo (BDD-style Go testing framework). Refer to 41 | // http://onsi.github.io/ginkgo/ to learn more about Ginkgo. 42 | 43 | var ( 44 | cfg *rest.Config 45 | k8sClient client.Client 46 | testEnv *envtest.Environment 47 | ) 48 | 49 | func TestAPIs(t *testing.T) { 50 | t.Parallel() 51 | RegisterFailHandler(Fail) 52 | 53 | RunSpecsWithDefaultAndCustomReporters(t, 54 | "Controller Suite", 55 | []Reporter{printer.NewlineReporter{}}) 56 | } 57 | 58 | var _ = BeforeSuite(func(done Done) { 59 | logf.SetLogger(zap.New(zap.UseDevMode(true), zap.WriteTo(GinkgoWriter))) 60 | 61 | By("bootstrapping test environment") 62 | testEnv = &envtest.Environment{ 63 | CRDDirectoryPaths: []string{filepath.Join("..", "deploy", "charts", "cluster-registry", "crds")}, 64 | } 65 | 66 | var err error 67 | cfg, err = testEnv.Start() 68 | Expect(err).ToNot(HaveOccurred()) 69 | Expect(cfg).ToNot(BeNil()) 70 | 71 | err = clusterregistryv1alpha1.AddToScheme(scheme.Scheme) 72 | Expect(err).NotTo(HaveOccurred()) 73 | 74 | // +kubebuilder:scaffold:scheme 75 | 76 | stop := ctrl.SetupSignalHandler() 77 | 78 | k8sManager, err := ctrl.NewManager(cfg, ctrl.Options{ 79 | Scheme: scheme.Scheme, 80 | MetricsBindAddress: "0", 81 | LeaderElection: false, 82 | Port: 0, 83 | }) 84 | Expect(err).ToNot(HaveOccurred()) 85 | Expect(k8sManager).ToNot(BeNil()) 86 | 87 | ctx := context.Background() 88 | 89 | err = controllers.NewClusterReconciler("clusters", ctrl.Log.WithName("controllers").WithName("cluster"), clusters.NewManager(ctx), config.Configuration{}).SetupWithManager(ctx, k8sManager) 90 | Expect(err).ToNot(HaveOccurred()) 91 | 92 | go func() { 93 | err = k8sManager.Start(stop) 94 | Expect(err).ToNot(HaveOccurred()) 95 | }() 96 | 97 | k8sClient = k8sManager.GetClient() 98 | Expect(k8sClient).ToNot(BeNil()) 99 | 100 | close(done) 101 | }, 60) 102 | 103 | var _ = AfterSuite(func() { 104 | By("tearing down the test environment") 105 | err := testEnv.Stop() 106 | Expect(err).ToNot(HaveOccurred()) 107 | }) 108 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/.helmignore: -------------------------------------------------------------------------------- 1 | # Patterns to ignore when building packages. 2 | # This supports shell glob matching, relative path matching, and 3 | # negation (prefixed with !). Only one pattern per line. 4 | .DS_Store 5 | # Common VCS dirs 6 | .git/ 7 | .gitignore 8 | .bzr/ 9 | .bzrignore 10 | .hg/ 11 | .hgignore 12 | .svn/ 13 | # Common backup files 14 | *.swp 15 | *.bak 16 | *.tmp 17 | *.orig 18 | *~ 19 | # Various IDEs 20 | .project 21 | .idea/ 22 | *.tmproj 23 | .vscode/ 24 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | name: cluster-registry 3 | description: Cluster registry controller for K8s 4 | home: https://eti.cisco.com/ 5 | 6 | maintainers: 7 | - email: ciscoeti@cisco.com 8 | name: Cisco 9 | sources: 10 | - https://github.com/cisco-open/cluster-registry-controller 11 | 12 | version: 0.2.12 13 | appVersion: v0.2.12 14 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/README.md: -------------------------------------------------------------------------------- 1 | # Cluster-registry chart 2 | 3 | The [cluster registry controller](https://github.com/cisco-open/cluster-registry-controller) helps to form a group of Kubernetes clusters and synchronize 4 | any K8s resources across those clusters arbitrarily. 5 | 6 | ## Prerequisites 7 | 8 | - Helm3 9 | 10 | ## Installing the chart 11 | 12 | To install the chart: 13 | 14 | ```bash 15 | ❯ helm repo add cluster-registry https://cisco-open.github.io/cluster-registry-controller 16 | ❯ helm install --namespace=cluster-registry --create-namespace cluster-registry cluster-registry/cluster-registry --set localCluster.name=primary 17 | ``` 18 | 19 | ## Uninstalling the Chart 20 | 21 | To uninstall/delete the `cluster-registry` release: 22 | 23 | ```bash 24 | ❯ helm uninstall cluster-registry 25 | ``` 26 | 27 | The command removes all the Kubernetes components associated with the chart and deletes the release. 28 | 29 | ## Configuration 30 | 31 | The following table lists the configurable parameters of the Cluster Registry chart and their default values. 32 | 33 | Parameter | Description | Default 34 | --------- |--| ------- 35 | `replicas` | Operator deployment replica count | `1` 36 | `localCluster.name` | Specify to automatically provision the cluster object with this name upon first start | `""` 37 | `localCluster.manageSecret` | If true, automatically provisions the secret for the K8s API server access | `"true"` 38 | `istio.revision` | Sets the `istio.io/rev` label on the deployment | `""` 39 | `podAnnotations` | Operator deployment pod annotations (YAML) | `{}` 40 | `imagePullSecrets` | Operator deployment image pull secrets | `[]` 41 | `podSecurityContext` | Operator deployment pod security context (YAML) | runAsUser: `65534`, runAsGroup: `65534` 42 | `securityContext` | Operator deployment security context (YAML) | allowPrivilegeEscalation: `false` 43 | `image.repository` | Operator container image repository | `ghcr.io/cisco-open/cluster-registry-controller` 44 | `image.tag` | Operator container image tag | `v0.2.12` 45 | `image.pullPolicy` | Operator container image pull policy | `IfNotPresent` 46 | `nodeselector` | Operator deployment node selector (YAML) | `{}` 47 | `affinity` | Operator deployment affinity (YAML) | `{}` 48 | `tolerations` | Operator deployment tolerations | `[]` 49 | `resources` | CPU/Memory resource requests/limits (YAML) | Requests: Memory: `100Mi`, CPU: `100m`, Limits: Memory: `200Mi`, CPU: `300m` 50 | `service.type` | Operator service type | `"ClusterIP"` 51 | `service.port` | Operator service port | `8080` 52 | `serviceAccount.annotations` | Operator service account annotations (YAML) | `{}` 53 | `podDisruptionBudget.enabled` | If true, PodDisruptionBudget is deployed for the operator | `false` 54 | `controller.leaderElection.enabled` | If true, leader election is enabled for the operator deployment | `true` 55 | `controller.leaderElection.name` | Name override for the leader election configmap | `cluster-registry-leader-election` 56 | `controller.log.format` | Format for controller logs | `"json"` 57 | `controller.log.verbosity` | Verbosity for controller logs | `0` 58 | `controller.workers` | Number of workers | `2` 59 | `controller.apiServerEndpointAddress` | Address of the cluster's K8s API server, which is publicly or from the specified network | `""` 60 | `controller.network.name` | Name of the network where the cluster is reachable | `"default"` 61 | `controller.coreResourceSource.enabled` | If true, the core resources (Cluster, ResourceSyncRule, secrets) could be synced from this cluster | `true` 62 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/crds/clusterregistry.k8s.cisco.com_clusterfeatures.yaml: -------------------------------------------------------------------------------- 1 | 2 | --- 3 | apiVersion: apiextensions.k8s.io/v1 4 | kind: CustomResourceDefinition 5 | metadata: 6 | annotations: 7 | controller-gen.kubebuilder.io/version: v0.6.2 8 | creationTimestamp: null 9 | name: clusterfeatures.clusterregistry.k8s.cisco.com 10 | spec: 11 | group: clusterregistry.k8s.cisco.com 12 | names: 13 | kind: ClusterFeature 14 | listKind: ClusterFeatureList 15 | plural: clusterfeatures 16 | shortNames: 17 | - cf 18 | singular: clusterfeature 19 | scope: Cluster 20 | versions: 21 | - additionalPrinterColumns: 22 | - jsonPath: .spec.featureName 23 | name: Feature 24 | type: string 25 | name: v1alpha1 26 | schema: 27 | openAPIV3Schema: 28 | description: ClusterFeature is the Schema for the clusterfeatures API 29 | properties: 30 | apiVersion: 31 | description: 'APIVersion defines the versioned schema of this representation 32 | of an object. Servers should convert recognized schemas to the latest 33 | internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' 34 | type: string 35 | kind: 36 | description: 'Kind is a string value representing the REST resource this 37 | object represents. Servers may infer this from the endpoint the client 38 | submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' 39 | type: string 40 | metadata: 41 | type: object 42 | spec: 43 | description: ClusterFeatureSpec defines the desired state of ClusterFeature 44 | properties: 45 | featureName: 46 | type: string 47 | required: 48 | - featureName 49 | type: object 50 | status: 51 | description: ClusterFeatureStatus defines the observed state of ClusterFeature 52 | type: object 53 | required: 54 | - spec 55 | type: object 56 | served: true 57 | storage: true 58 | subresources: 59 | status: {} 60 | status: 61 | acceptedNames: 62 | kind: "" 63 | plural: "" 64 | conditions: [] 65 | storedVersions: [] 66 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/examples/clusterregistry_v1alpha1_cluster.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: clusterregistry.k8s.cisco.com/v1alpha1 2 | kind: Cluster 3 | metadata: 4 | name: local-cluster 5 | spec: 6 | # clusterID of kube-system namespace 7 | clusterID: 995ba61b-0073-4e42-89e2-18282b1801ac 8 | # secret reference for access credentails (kubeconfig) 9 | authInfo: 10 | secretRef: 11 | name: "local-cluster" 12 | namespace: "cluster-registry" 13 | # specific apiserver endpoint overrides 14 | kubernetesApiEndpoints: 15 | - serverAddress: 192.168.100.1:443 16 | clientNetwork: privatenet 17 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/examples/clusterregistry_v1alpha1_clusterfeature.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: clusterregistry.k8s.cisco.com/v1alpha1 2 | kind: ClusterFeature 3 | metadata: 4 | name: test-secret-source 5 | spec: 6 | featureName: test-secret-feature 7 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/examples/clusterregistry_v1alpha1_resourcesyncrule.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: clusterregistry.k8s.cisco.com/v1alpha1 2 | kind: ResourceSyncRule 3 | metadata: 4 | annotations: 5 | # disable syncing this ResourceSyncRule resource itself to all clusters 6 | cluster-registry.k8s.cisco.com/resource-sync-disabled: "true" 7 | name: test-secret-sink 8 | spec: 9 | clusterFeatureMatch: 10 | # only sync from clusters where a ClusterFeature is present with this featurename 11 | - featureName: test-secret-feature 12 | groupVersionKind: 13 | kind: Secret 14 | version: v1 15 | rules: 16 | - match: 17 | - objectKey: 18 | name: test-secret 19 | namespace: cluster-registry 20 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/examples/test_secret.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Secret 3 | metadata: 4 | name: test-secret 5 | data: {} -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/templates/_helpers.tpl: -------------------------------------------------------------------------------- 1 | {{/* vim: set filetype=mustache: */}} 2 | {{/* 3 | Expand the name of the chart. 4 | */}} 5 | {{- define "cluster-registry.name" -}} 6 | {{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} 7 | {{- end }} 8 | 9 | {{/* 10 | Create a default fully qualified app name. 11 | We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). 12 | If release name contains chart name it will be used as a full name. 13 | */}} 14 | {{- define "cluster-registry.fullname" -}} 15 | {{- if .Values.fullnameOverride }} 16 | {{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} 17 | {{- else }} 18 | {{- $name := default .Chart.Name .Values.nameOverride }} 19 | {{- if contains $name .Release.Name }} 20 | {{- .Release.Name | trunc 63 | trimSuffix "-" }} 21 | {{- else }} 22 | {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} 23 | {{- end }} 24 | {{- end }} 25 | {{- end }} 26 | 27 | {{/* 28 | Create chart name and version as used by the chart label. 29 | */}} 30 | {{- define "cluster-registry.chart" -}} 31 | {{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} 32 | {{- end }} 33 | 34 | 35 | {{- define "cluster-registry-controller.fullname" -}} 36 | {{ include "cluster-registry.fullname" . }}-controller 37 | {{- end }} 38 | 39 | {{- define "cluster-registry-controller.name" -}} 40 | {{ include "cluster-registry.name" . }}-controller 41 | {{- end }} 42 | 43 | {{- define "cluster-registry-controller.labels" }} 44 | app: {{ include "cluster-registry-controller.fullname" . }} 45 | app.kubernetes.io/name: {{ include "cluster-registry-controller.name" . }} 46 | helm.sh/chart: {{ include "cluster-registry.chart" . }} 47 | app.kubernetes.io/managed-by: {{ .Release.Service }} 48 | app.kubernetes.io/instance: {{ .Release.Name }} 49 | app.kubernetes.io/version: {{ .Chart.AppVersion | replace "+" "_" }} 50 | app.kubernetes.io/component: cluster-registry-controller 51 | app.kubernetes.io/part-of: {{ include "cluster-registry.name" . }} 52 | {{- end }} 53 | 54 | {{- define "cluster-registry-controller.selectorLabels" -}} 55 | app.kubernetes.io/name: {{ include "cluster-registry-controller.name" . }} 56 | app.kubernetes.io/instance: {{ .Release.Name }} 57 | {{- end }} 58 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/templates/cluster-validator-vwhc.yaml: -------------------------------------------------------------------------------- 1 | {{- if .Values.webhooks.clusterValidator.enabled -}} 2 | apiVersion: admissionregistration.k8s.io/v1 3 | kind: ValidatingWebhookConfiguration 4 | metadata: 5 | name: "{{ include "cluster-registry-controller.fullname" . }}-{{ .Values.webhooks.clusterValidator.nameSuffix }}" 6 | namespace: {{ .Release.Namespace }} 7 | webhooks: 8 | - name: cluster-validator.clusterregistry.k8s.cisco.com 9 | clientConfig: 10 | service: 11 | name: "{{ include "cluster-registry-controller.fullname" . }}" 12 | namespace: {{ .Release.Namespace }} 13 | path: /validate-cluster 14 | port: 443 15 | failurePolicy: Ignore 16 | matchPolicy: Equivalent 17 | rules: 18 | - apiGroups: 19 | - clusterregistry.k8s.cisco.com 20 | apiVersions: 21 | - v1alpha1 22 | operations: 23 | - CREATE 24 | - UPDATE 25 | resources: 26 | - clusters 27 | scope: '*' 28 | sideEffects: None 29 | timeoutSeconds: 30 30 | admissionReviewVersions: 31 | - v1 32 | {{- end -}} 33 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/templates/deployment.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: {{ include "cluster-registry-controller.fullname" . }} 5 | namespace: {{ .Release.Namespace }} 6 | labels: 7 | {{- include "cluster-registry-controller.labels" . | nindent 4 }} 8 | spec: 9 | replicas: {{ .Values.replicas }} 10 | selector: 11 | matchLabels: 12 | {{- include "cluster-registry-controller.selectorLabels" . | nindent 6 }} 13 | template: 14 | metadata: 15 | annotations: 16 | {{- if .Values.istio.revision }} 17 | istio.io/rev: {{ .Values.istio.revision }} 18 | {{- end }} 19 | {{- with .Values.podAnnotations }} 20 | {{- toYaml . | nindent 8 }} 21 | {{- end }} 22 | labels: 23 | {{- include "cluster-registry-controller.labels" . | nindent 8 }} 24 | spec: 25 | securityContext: 26 | {{- toYaml .Values.podSecurityContext | nindent 8 }} 27 | serviceAccountName: {{ include "cluster-registry-controller.fullname" . }} 28 | containers: 29 | - name: manager 30 | securityContext: 31 | {{- toYaml .Values.securityContext | nindent 12 }} 32 | image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}" 33 | imagePullPolicy: {{ .Values.image.pullPolicy }} 34 | command: 35 | - /manager 36 | args: 37 | - "--cluster-validator-webhook-enabled={{ .Values.webhooks.clusterValidator.enabled }}" 38 | {{- if and (.Values.webhooks.clusterValidator.enabled) (.Values.webhooks.clusterValidator.nameSuffix) }} 39 | - "--cluster-validator-webhook-name={{ include "cluster-registry-controller.fullname" . }}-{{ .Values.webhooks.clusterValidator.nameSuffix }}" 40 | {{- end }} 41 | {{- if and (.Values.webhooks.clusterValidator.enabled) (.Values.webhooks.clusterValidator.port) }} 42 | - "--cluster-validator-webhook-port={{ .Values.webhooks.clusterValidator.port }}" 43 | {{- end }} 44 | {{- if and (.Values.webhooks.clusterValidator.enabled) (.Values.webhooks.clusterValidator.certificateDirectory) }} 45 | - "--cluster-validator-webhook-certificate-directory={{ .Values.webhooks.clusterValidator.certificateDirectory }}" 46 | {{- end }} 47 | ports: 48 | - name: metrics 49 | containerPort: {{ .Values.service.port }} 50 | protocol: TCP 51 | - name: health 52 | containerPort: {{ .Values.health.port }} 53 | protocol: TCP 54 | {{- if and (.Values.webhooks.clusterValidator.enabled) (.Values.webhooks.clusterValidator.port) }} 55 | - containerPort: {{ .Values.webhooks.clusterValidator.port }} 56 | name: http-cl-val-wh 57 | protocol: TCP 58 | {{- end }} 59 | livenessProbe: 60 | httpGet: 61 | path: /metrics 62 | port: metrics 63 | readinessProbe: 64 | httpGet: 65 | path: /readyz 66 | port: health 67 | resources: 68 | {{- toYaml .Values.resources | nindent 12 }} 69 | env: 70 | - name: METRICS_ADDR 71 | value: ":{{ .Values.service.port }}" 72 | - name: LEADER_ELECTION_ENABLED 73 | value: "{{ .Values.controller.leaderElection.enabled }}" 74 | - name: LEADER_ELECTION_NAME 75 | value: "{{ .Values.controller.leaderElection.name }}" 76 | - name: LEADER_ELECTION_NAMESPACE 77 | value: "{{ .Release.Namespace }}" 78 | - name: LOG_FORMAT 79 | value: "{{ .Values.controller.log.format }}" 80 | - name: LOG_VERBOSITY 81 | value: "{{ .Values.controller.log.verbosity }}" 82 | {{ if .Values.localCluster.name }} 83 | - name: PROVISION_LOCAL_CLUSTER 84 | value: "{{ .Values.localCluster.name }}" 85 | {{ end }} 86 | - name: NAMESPACE 87 | valueFrom: 88 | fieldRef: 89 | fieldPath: metadata.namespace 90 | - name: READER_SERVICE_ACCOUNT_NAME 91 | value: "{{ include "cluster-registry-controller.fullname" . }}-reader" 92 | - name: NETWORK_NAME 93 | value: "{{ .Values.controller.network.name }}" 94 | - name: MANAGE_LOCAL_CLUSTER_SECRET 95 | value: "{{ .Values.localCluster.manageSecret }}" 96 | - name: APISERVER_ENDPOINT_ADDRESS 97 | value: "{{ .Values.controller.apiServerEndpointAddress }}" 98 | - name: CORE_RESOURCES_SOURCE_ENABLED 99 | value: "{{ .Values.controller.coreResourceSource.enabled }}" 100 | {{- with .Values.nodeSelector }} 101 | nodeSelector: 102 | {{- toYaml . | nindent 8 }} 103 | {{- end }} 104 | {{- with .Values.affinity }} 105 | affinity: 106 | {{- toYaml . | nindent 8 }} 107 | {{- end }} 108 | {{- with .Values.tolerations }} 109 | tolerations: 110 | {{- toYaml . | nindent 8 }} 111 | {{- end }} 112 | {{- with .Values.imagePullSecrets }} 113 | imagePullSecrets: 114 | {{- toYaml . | nindent 8 }} 115 | {{- end }} 116 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/templates/poddistruptionbudget.yaml: -------------------------------------------------------------------------------- 1 | {{- if .Values.podDisruptionBudget.enabled }} 2 | apiVersion: policy/v1 3 | kind: PodDisruptionBudget 4 | metadata: 5 | name: {{ include "cluster-registry-controller.fullname" . }} 6 | namespace: {{ .Release.Namespace }} 7 | labels: 8 | {{- include "cluster-registry-controller.labels" . | nindent 4 }} 9 | spec: 10 | minAvailable: 1 11 | selector: 12 | matchLabels: 13 | {{- include "cluster-registry-controller.selectorLabels" . | nindent 6 }} 14 | {{- end }} 15 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/templates/rbac-leader-election.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: rbac.authorization.k8s.io/v1 2 | kind: Role 3 | metadata: 4 | name: {{ include "cluster-registry-controller.fullname" . }}-leader-election 5 | namespace: {{ .Release.Namespace }} 6 | labels: 7 | {{- include "cluster-registry-controller.labels" . | nindent 4 }} 8 | rules: 9 | - apiGroups: 10 | - "" 11 | resources: 12 | - configmaps 13 | verbs: 14 | - get 15 | - list 16 | - watch 17 | - create 18 | - update 19 | - patch 20 | - delete 21 | - apiGroups: 22 | - "" 23 | resources: 24 | - configmaps/status 25 | verbs: 26 | - get 27 | - update 28 | - patch 29 | - apiGroups: 30 | - "" 31 | resources: 32 | - events 33 | verbs: 34 | - create 35 | --- 36 | apiVersion: rbac.authorization.k8s.io/v1 37 | kind: RoleBinding 38 | metadata: 39 | name: {{ include "cluster-registry-controller.fullname" . }}-leader-election 40 | namespace: {{ .Release.Namespace }} 41 | labels: 42 | {{- include "cluster-registry-controller.labels" . | nindent 4 }} 43 | roleRef: 44 | apiGroup: rbac.authorization.k8s.io 45 | kind: Role 46 | name: {{ include "cluster-registry-controller.fullname" . }}-leader-election 47 | subjects: 48 | - kind: ServiceAccount 49 | name: {{ include "cluster-registry-controller.fullname" . }} 50 | namespace: {{ .Release.Namespace }} 51 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/templates/rbac.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ServiceAccount 3 | metadata: 4 | name: {{ include "cluster-registry-controller.fullname" . }} 5 | namespace: {{ .Release.Namespace }} 6 | labels: 7 | {{- include "cluster-registry-controller.labels" . | nindent 4 }} 8 | {{- with .Values.serviceAccount.annotations }} 9 | annotations: 10 | {{- toYaml . | nindent 4 }} 11 | {{- end }} 12 | --- 13 | apiVersion: rbac.authorization.k8s.io/v1 14 | kind: ClusterRole 15 | metadata: 16 | name: {{ include "cluster-registry-controller.fullname" . }}-aggregated 17 | aggregationRule: 18 | clusterRoleSelectors: 19 | - matchLabels: 20 | cluster-registry.k8s.cisco.com/controller-aggregated: "true" 21 | rules: [] 22 | --- 23 | apiVersion: rbac.authorization.k8s.io/v1 24 | kind: ClusterRole 25 | metadata: 26 | name: {{ include "cluster-registry-controller.fullname" . }} 27 | labels: 28 | {{- include "cluster-registry-controller.labels" . | nindent 4 }} 29 | cluster-registry.k8s.cisco.com/controller-aggregated: "true" 30 | rules: 31 | - apiGroups: 32 | - coordination.k8s.io 33 | resources: 34 | - leases 35 | verbs: 36 | - '*' 37 | - apiGroups: ["clusterregistry.k8s.cisco.com"] 38 | resources: ["*"] 39 | verbs: 40 | - get 41 | - list 42 | - watch 43 | - create 44 | - update 45 | - delete 46 | - patch 47 | - apiGroups: [""] 48 | resources: 49 | - events 50 | verbs: 51 | - get 52 | - list 53 | - watch 54 | - create 55 | - update 56 | - delete 57 | - patch 58 | - apiGroups: [""] 59 | resources: 60 | - namespaces 61 | - nodes 62 | - secrets 63 | - serviceaccounts 64 | verbs: 65 | - get 66 | - list 67 | - watch 68 | - apiGroups: [""] 69 | resources: 70 | - secrets 71 | verbs: 72 | - create 73 | - update 74 | - delete 75 | - patch 76 | - apiGroups: 77 | - admissionregistration.k8s.io 78 | resources: 79 | - mutatingwebhookconfigurations 80 | - validatingwebhookconfigurations 81 | verbs: 82 | - get 83 | - list 84 | - watch 85 | - create 86 | - update 87 | - delete 88 | - patch 89 | --- 90 | apiVersion: rbac.authorization.k8s.io/v1 91 | kind: ClusterRoleBinding 92 | metadata: 93 | name: {{ include "cluster-registry-controller.fullname" . }} 94 | labels: 95 | {{- include "cluster-registry-controller.labels" . | nindent 4 }} 96 | roleRef: 97 | kind: ClusterRole 98 | name: {{ include "cluster-registry-controller.fullname" . }}-aggregated 99 | apiGroup: rbac.authorization.k8s.io 100 | subjects: 101 | - kind: ServiceAccount 102 | name: {{ include "cluster-registry-controller.fullname" . }} 103 | namespace: {{ .Release.Namespace }} 104 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/templates/reader-rbac.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ServiceAccount 3 | metadata: 4 | name: {{ include "cluster-registry-controller.fullname" . }}-reader 5 | namespace: {{ .Release.Namespace }} 6 | --- 7 | apiVersion: rbac.authorization.k8s.io/v1 8 | kind: ClusterRole 9 | metadata: 10 | name: {{ include "cluster-registry-controller.fullname" . }}-reader-aggregated 11 | aggregationRule: 12 | clusterRoleSelectors: 13 | - matchLabels: 14 | cluster-registry.k8s.cisco.com/reader-aggregated: "true" 15 | rules: [] 16 | --- 17 | apiVersion: rbac.authorization.k8s.io/v1 18 | kind: ClusterRole 19 | metadata: 20 | name: {{ include "cluster-registry-controller.fullname" . }}-reader 21 | labels: 22 | {{- include "cluster-registry-controller.labels" . | nindent 4 }} 23 | cluster-registry.k8s.cisco.com/reader-aggregated: "true" 24 | rules: 25 | - apiGroups: ["clusterregistry.k8s.cisco.com"] 26 | resources: ["*"] 27 | verbs: 28 | - get 29 | - list 30 | - watch 31 | - apiGroups: [""] 32 | resources: 33 | - namespaces 34 | - nodes 35 | - secrets 36 | verbs: 37 | - get 38 | - list 39 | - watch 40 | --- 41 | apiVersion: rbac.authorization.k8s.io/v1 42 | kind: ClusterRoleBinding 43 | metadata: 44 | name: {{ include "cluster-registry-controller.fullname" . }}-reader 45 | roleRef: 46 | apiGroup: rbac.authorization.k8s.io 47 | kind: ClusterRole 48 | name: {{ include "cluster-registry-controller.fullname" . }}-reader-aggregated 49 | subjects: 50 | - kind: ServiceAccount 51 | name: {{ include "cluster-registry-controller.fullname" . }}-reader 52 | namespace: {{ .Release.Namespace }} 53 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/templates/service.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: {{ include "cluster-registry-controller.fullname" . }} 5 | namespace: {{ .Release.Namespace }} 6 | labels: 7 | {{- include "cluster-registry-controller.labels" . | nindent 4 }} 8 | spec: 9 | type: {{ .Values.service.type }} 10 | ports: 11 | - port: {{ .Values.service.port }} 12 | targetPort: metrics 13 | protocol: TCP 14 | name: http-metrics 15 | {{- if .Values.webhooks.clusterValidator.enabled }} 16 | - port: 443 17 | targetPort: http-cl-val-wh 18 | protocol: TCP 19 | name: http-cl-val-wh 20 | {{- end }} 21 | selector: 22 | {{- include "cluster-registry-controller.selectorLabels" . | nindent 4 }} 23 | -------------------------------------------------------------------------------- /deploy/charts/cluster-registry/values.yaml: -------------------------------------------------------------------------------- 1 | # Default values for cluster-registry-controller 2 | # This is a YAML-formatted file. 3 | # Declare variables to be passed into your templates. 4 | 5 | replicas: 1 6 | 7 | localCluster: 8 | # specify to automatically provision the cluster object upon first start 9 | name: "" 10 | manageSecret: true 11 | 12 | istio: 13 | revision: "" 14 | 15 | podAnnotations: {} 16 | imagePullSecrets: [] 17 | 18 | podSecurityContext: 19 | runAsNonRoot: true 20 | seccompProfile: 21 | type: RuntimeDefault 22 | securityContext: 23 | allowPrivilegeEscalation: false 24 | capabilities: 25 | drop: 26 | - ALL 27 | image: 28 | repository: ghcr.io/cisco-open/cluster-registry-controller 29 | tag: v0.2.12 30 | pullPolicy: IfNotPresent 31 | 32 | nodeSelector: {} 33 | affinity: {} 34 | tolerations: [] 35 | resources: 36 | requests: 37 | memory: "150Mi" 38 | cpu: "100m" 39 | limits: 40 | memory: "300Mi" 41 | cpu: "300m" 42 | 43 | service: 44 | type: ClusterIP 45 | port: 8080 46 | 47 | health: 48 | port: 8090 49 | 50 | serviceAccount: 51 | annotations: {} 52 | 53 | podDisruptionBudget: 54 | enabled: false 55 | 56 | controller: 57 | leaderElection: 58 | enabled: true 59 | name: "cluster-registry-leader-election" 60 | log: 61 | format: json 62 | verbosity: 0 63 | workers: 2 64 | apiServerEndpointAddress: "" 65 | network: 66 | name: "default" 67 | coreResourceSource: 68 | enabled: true 69 | 70 | webhooks: 71 | # clusterValidator is a validation admission webhook for cluster custom 72 | # resources. 73 | clusterValidator: 74 | # Enabled is the switch for turning the webhook on or off. 75 | enabled: true 76 | 77 | # CertificateDirectory is the path storing the certificate files. 78 | certificateDirectory: /tmp/webhooks/clusterValidator/certificates 79 | 80 | # Name is the suffix for the webhook's identifier. The controller's name 81 | # will be prepended to this value with a dash. 82 | nameSuffix: cluster-validator-webhook 83 | 84 | # Port is the port number on which the webhook is served in the container. 85 | port: 9443 86 | -------------------------------------------------------------------------------- /deploy/charts/embed.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package charts 16 | 17 | import ( 18 | "embed" 19 | "io/fs" 20 | ) 21 | 22 | var ( 23 | //go:embed cluster-registry cluster-registry/templates/_helpers.tpl 24 | clusterRegistryEmbed embed.FS 25 | 26 | // ClusterRegistry exposes the cluster-registry chart using relative file paths from the chart root 27 | ClusterRegistry fs.FS 28 | ) 29 | 30 | func init() { 31 | var err error 32 | ClusterRegistry, err = fs.Sub(clusterRegistryEmbed, "cluster-registry") 33 | if err != nil { 34 | panic(err) 35 | } 36 | } 37 | -------------------------------------------------------------------------------- /deploy/charts/embed_test.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package charts_test 16 | 17 | import ( 18 | "io" 19 | "os" 20 | "testing" 21 | 22 | "github.com/cisco-open/cluster-registry-controller/deploy/charts" 23 | ) 24 | 25 | func TestEmbed(t *testing.T) { 26 | file, err := charts.ClusterRegistry.Open("Chart.yaml") 27 | if err != nil { 28 | t.Fatalf("%+v", err) 29 | } 30 | 31 | embeddedContent, err := io.ReadAll(file) 32 | if err != nil { 33 | t.Fatalf("%+v", err) 34 | } 35 | 36 | localContent, err := os.ReadFile("cluster-registry/Chart.yaml") 37 | if err != nil { 38 | t.Fatalf("%+v", err) 39 | } 40 | 41 | if string(embeddedContent) != string(localContent) { 42 | t.Fatalf("embedded content %s does not equal local content %s", string(embeddedContent), string(localContent)) 43 | } 44 | } 45 | -------------------------------------------------------------------------------- /deploy/charts/go.mod: -------------------------------------------------------------------------------- 1 | module github.com/cisco-open/cluster-registry-controller/deploy/charts 2 | 3 | go 1.18 4 | -------------------------------------------------------------------------------- /docs/development.md: -------------------------------------------------------------------------------- 1 | # Developer Guide 2 | 3 | ## How to run cluster registry controller in your cluster with your changes 4 | 5 | Cluster registry controller is built on the [kubebuilder](https://github.com/kubernetes-sigs/kubebuilder) project. 6 | 7 | If you make changes and would like to try your own version, create your own image: 8 | 9 | make docker-build docker-push deploy IMG=/cluster-registry-controller: 10 | 11 | Watch the operator's logs with: 12 | 13 | kubectl logs -f -n cluster-registry cluster-registry-controller-manager -c manager 14 | -------------------------------------------------------------------------------- /docs/img/clean-k8s-arch.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisco-open/cluster-registry-controller/ab02d54e1b0591bcf73ed87e9ad7b5eb5fb8403b/docs/img/clean-k8s-arch.jpg -------------------------------------------------------------------------------- /docs/img/final-arch.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisco-open/cluster-registry-controller/ab02d54e1b0591bcf73ed87e9ad7b5eb5fb8403b/docs/img/final-arch.jpg -------------------------------------------------------------------------------- /docs/img/installed-cr.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisco-open/cluster-registry-controller/ab02d54e1b0591bcf73ed87e9ad7b5eb5fb8403b/docs/img/installed-cr.jpg -------------------------------------------------------------------------------- /docs/img/ui-cr-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisco-open/cluster-registry-controller/ab02d54e1b0591bcf73ed87e9ad7b5eb5fb8403b/docs/img/ui-cr-1.png -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/cisco-open/cluster-registry-controller 2 | 3 | go 1.18 4 | 5 | require ( 6 | emperror.dev/errors v0.8.0 7 | github.com/Masterminds/sprig v2.22.0+incompatible 8 | github.com/banzaicloud/k8s-objectmatcher v1.8.0 9 | github.com/banzaicloud/operator-tools v0.24.1-0.20210917222015-90c6c0b3cffe 10 | github.com/cenkalti/backoff v2.2.1+incompatible 11 | github.com/cisco-open/cluster-registry-controller/api v0.0.1 12 | github.com/gertd/go-pluralize v0.1.7 13 | github.com/go-logr/logr v0.4.0 14 | github.com/go-logr/zapr v0.4.0 // indirect 15 | github.com/gomodule/redigo v1.8.4 // indirect 16 | github.com/onsi/ginkgo v1.16.4 17 | github.com/onsi/gomega v1.14.0 18 | github.com/spf13/pflag v1.0.5 19 | github.com/spf13/viper v1.7.0 20 | github.com/throttled/throttled v2.2.5+incompatible 21 | go.uber.org/zap v1.18.1 22 | gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect 23 | k8s.io/api v0.21.3 24 | k8s.io/apimachinery v0.21.3 25 | k8s.io/client-go v0.21.3 26 | sigs.k8s.io/controller-runtime v0.9.5 27 | sigs.k8s.io/yaml v1.2.0 28 | ) 29 | 30 | require ( 31 | cloud.google.com/go v0.54.0 // indirect 32 | github.com/Masterminds/goutils v1.1.1 // indirect 33 | github.com/Masterminds/semver v1.5.0 // indirect 34 | github.com/beorn7/perks v1.0.1 // indirect 35 | github.com/briandowns/spinner v1.12.0 // indirect 36 | github.com/cespare/xxhash/v2 v2.1.1 // indirect 37 | github.com/cppforlife/go-patch v0.2.0 // indirect 38 | github.com/davecgh/go-spew v1.1.1 // indirect 39 | github.com/evanphx/json-patch v4.11.0+incompatible // indirect 40 | github.com/fatih/color v1.10.0 // indirect 41 | github.com/fsnotify/fsnotify v1.4.9 // indirect 42 | github.com/gogo/protobuf v1.3.2 // indirect 43 | github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e // indirect 44 | github.com/golang/protobuf v1.5.2 // indirect 45 | github.com/google/go-cmp v0.5.5 // indirect 46 | github.com/google/gofuzz v1.1.0 // indirect 47 | github.com/google/uuid v1.1.2 // indirect 48 | github.com/googleapis/gnostic v0.5.5 // indirect 49 | github.com/hashicorp/golang-lru v0.5.4 // indirect 50 | github.com/hashicorp/hcl v1.0.0 // indirect 51 | github.com/huandu/xstrings v1.3.2 // indirect 52 | github.com/iancoleman/orderedmap v0.2.0 // indirect 53 | github.com/imdario/mergo v0.3.12 // indirect 54 | github.com/json-iterator/go v1.1.12 // indirect 55 | github.com/magiconair/properties v1.8.1 // indirect 56 | github.com/mattn/go-colorable v0.1.8 // indirect 57 | github.com/mattn/go-isatty v0.0.12 // indirect 58 | github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect 59 | github.com/mitchellh/copystructure v1.1.1 // indirect 60 | github.com/mitchellh/mapstructure v1.1.2 // indirect 61 | github.com/mitchellh/reflectwalk v1.0.1 // indirect 62 | github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect 63 | github.com/modern-go/reflect2 v1.0.2 // indirect 64 | github.com/nxadm/tail v1.4.8 // indirect 65 | github.com/pelletier/go-toml v1.2.0 // indirect 66 | github.com/pkg/errors v0.9.1 // indirect 67 | github.com/prometheus/client_golang v1.11.1 // indirect 68 | github.com/prometheus/client_model v0.2.0 // indirect 69 | github.com/prometheus/common v0.26.0 // indirect 70 | github.com/prometheus/procfs v0.6.0 // indirect 71 | github.com/spf13/afero v1.2.2 // indirect 72 | github.com/spf13/cast v1.3.1 // indirect 73 | github.com/spf13/jwalterweatherman v1.0.0 // indirect 74 | github.com/subosito/gotenv v1.2.0 // indirect 75 | github.com/tidwall/gjson v1.9.3 // indirect 76 | github.com/tidwall/match v1.1.1 // indirect 77 | github.com/tidwall/pretty v1.2.0 // indirect 78 | github.com/wayneashleyberry/terminal-dimensions v1.0.0 // indirect 79 | go.uber.org/atomic v1.7.0 // indirect 80 | go.uber.org/multierr v1.6.0 // indirect 81 | golang.org/x/crypto v0.1.0 // indirect 82 | golang.org/x/net v0.7.0 // indirect 83 | golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d // indirect 84 | golang.org/x/sys v0.5.0 // indirect 85 | golang.org/x/term v0.5.0 // indirect 86 | golang.org/x/text v0.7.0 // indirect 87 | golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac // indirect 88 | gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect 89 | google.golang.org/appengine v1.6.7 // indirect 90 | google.golang.org/protobuf v1.26.0 // indirect 91 | gopkg.in/inf.v0 v0.9.1 // indirect 92 | gopkg.in/ini.v1 v1.51.0 // indirect 93 | gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect 94 | gopkg.in/yaml.v2 v2.4.0 // indirect 95 | k8s.io/apiextensions-apiserver v0.21.3 // indirect 96 | k8s.io/component-base v0.21.3 // indirect 97 | k8s.io/klog/v2 v2.8.0 // indirect 98 | k8s.io/kube-openapi v0.0.0-20210305001622-591a79e4bda7 // indirect 99 | k8s.io/utils v0.0.0-20210722164352-7f3ee0f31471 // indirect 100 | sigs.k8s.io/structured-merge-diff/v4 v4.1.2 // indirect 101 | ) 102 | 103 | replace github.com/cisco-open/cluster-registry-controller/api => ./api 104 | -------------------------------------------------------------------------------- /hack/boilerplate.go.txt: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright 2022 Cisco Systems, Inc. and/or its affiliates. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"); 5 | you may not use this file except in compliance with the License. 6 | You may obtain a copy of the License at 7 | 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | 10 | Unless required by applicable law or agreed to in writing, software 11 | distributed under the License is distributed on an "AS IS" BASIS, 12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | See the License for the specific language governing permissions and 14 | limitations under the License. 15 | */ 16 | -------------------------------------------------------------------------------- /internal/config/config.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package config 16 | 17 | import "time" 18 | 19 | type Configuration struct { 20 | MetricsAddr string `mapstructure:"metrics-addr" json:"metricsAddr,omitempty"` 21 | HealthAddr string `mapstructure:"health-addr" json:"healthAddr,omitempty"` 22 | LeaderElection LeaderElection `mapstructure:"leader-election" json:"leaderElection,omitempty"` 23 | Logging Logging `mapstructure:"log" json:"logging,omitempty"` 24 | ClusterController ClusterController `mapstructure:"clusterController" json:"clusterController,omitempty"` 25 | SyncController SyncController `mapstructure:"syncController" json:"syncController,omitempty"` 26 | Namespace string `mapstructure:"namespace" json:"namespace,omitempty"` 27 | ProvisionLocalCluster string `mapstructure:"provision-local-cluster" json:"provisionLocalCluster,omitempty"` 28 | ManageLocalClusterSecret bool `mapstructure:"manage-local-cluster-secret" json:"manageLocalClusterSecret,omitempty"` 29 | ReaderServiceAccountName string `mapstructure:"reader-service-account-name" json:"readerServiceAccountName,omitempty"` 30 | NetworkName string `mapstructure:"network-name" json:"networkName,omitempty"` 31 | APIServerEndpointAddress string `mapstructure:"apiserver-endpoint-address" json:"apiServerEndpointAddress,omitempty"` 32 | CoreResourcesSourceEnabled bool `mapstructure:"core-resources-source-enabled" json:"coreResourcesSourceEnabled,omitempty"` 33 | 34 | // ClusterValidatorWebhook configures the cluster CR validator webhook for 35 | // the operator. 36 | ClusterValidatorWebhook ClusterValidatorWebhook `mapstructure:"cluster-validator-webhook" json:"clusterValidatorWebhook"` 37 | } 38 | 39 | // ClusterValidatorWebhook describes the configuration options for the cluster 40 | // CR validator webhook. 41 | type ClusterValidatorWebhook struct { 42 | // Enabled is the indicator to determine whether the webhook is enabled. 43 | Enabled bool `mapstructure:"enabled" json:"enabled,omitempty"` 44 | 45 | // Certificate directory is the directory where the cluster CR validator webhook stores its certificates locally. 46 | CertificateDirectory string `mapstructure:"certificate-directory" json:"certificateDirectory,omitempty"` 47 | 48 | // Name is the name of the cluster CR validator webhook resource. 49 | Name string `mapstructure:"name" json:"name,omitempty"` 50 | 51 | // Port is the port the cluster CR validator webhook serves on. 52 | Port uint `mapstructure:"port" json:"port,omitempty"` 53 | } 54 | 55 | type ClusterController struct { 56 | WorkerCount int `mapstructure:"workerCount" json:"workerCount,omitempty"` 57 | RefreshIntervalSeconds int `mapstructure:"refreshIntervalSeconds" json:"refreshIntervalSeconds,omitempty"` 58 | } 59 | 60 | type SyncController struct { 61 | WorkerCount int `mapstructure:"workerCount" json:"workerCount,omitempty"` 62 | RateLimit SyncControllerRateLimit `mapstructure:"rateLimit" json:"rateLimit,omitempty"` 63 | } 64 | 65 | type SyncControllerRateLimit struct { 66 | MaxKeys int `mapstructure:"maxKeys" json:"maxKeys,omitempty"` 67 | MaxRatePerSecond int `mapstructure:"maxRatePerSecond" json:"maxRatePerSecond,omitempty"` 68 | MaxBurst int `mapstructure:"maxBurst" json:"maxBurst,omitempty"` 69 | } 70 | 71 | type LeaderElection struct { 72 | Enabled bool `mapstructure:"enabled" json:"enabled,omitempty"` 73 | Name string `mapstructure:"name" json:"name,omitempty"` 74 | Namespace string `mapstructure:"namespace" json:"namespace,omitempty"` 75 | LeaseDuration time.Duration `mapsturecture:"leaseDuration" json:"leaseDuration,omitempty"` 76 | ReleaseOnExit bool `mapstructure:"releaseOnExit" json:"releaseOnExit,omitempty"` 77 | } 78 | 79 | type ( 80 | LogFormat string 81 | ) 82 | 83 | const ( 84 | LogFormatConsole LogFormat = "console" 85 | LogFormatJSON LogFormat = "json" 86 | ) 87 | 88 | type Logging struct { 89 | Verbosity int8 `mapstructure:"verbosity" json:"level,omitempty"` 90 | Format LogFormat `mapstructure:"format" json:"format,omitempty"` 91 | } 92 | -------------------------------------------------------------------------------- /pkg/cert/renewer.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package cert 16 | 17 | import ( 18 | "context" 19 | "time" 20 | 21 | "emperror.dev/errors" 22 | "github.com/go-logr/logr" 23 | ) 24 | 25 | // AfterCheckFunctionType is the function signature for functions run if the 26 | // check was triggered from outside or after the certificate was renewed. 27 | type AfterCheckFunctionType func(certificate *Certificate, needsUpdate bool) error 28 | 29 | // Renewer handles the automatic renewal of certificates. 30 | type Renewer struct { 31 | // logger is the interface containing log helper functions. 32 | logger logr.Logger 33 | 34 | // dnsNames are holding the DNS names for the server certificate generation. 35 | dnsNames []string 36 | 37 | // certificateDirectoryPath is the path of the directory where the generated 38 | // certificates should be written. 39 | certificateDirectoryPath string 40 | 41 | // afterCheckFunctions are optional callbacks for triggered checks and 42 | // renewals. 43 | afterCheckFunctions []AfterCheckFunctionType 44 | } 45 | 46 | // NewRenewer returns a certificate renewer configured to the specified values. 47 | func NewRenewer( 48 | logger logr.Logger, 49 | dnsNames []string, 50 | certificateDirectoryPath string, 51 | shouldCheckCertificate bool, 52 | afterCheckFunctions ...AfterCheckFunctionType, 53 | ) (*Renewer, error) { 54 | if dnsNames == nil { 55 | dnsNames = []string{} 56 | } 57 | 58 | renewer := &Renewer{ 59 | logger: logger, 60 | dnsNames: dnsNames, 61 | certificateDirectoryPath: certificateDirectoryPath, 62 | afterCheckFunctions: afterCheckFunctions, 63 | } 64 | 65 | if shouldCheckCertificate { 66 | err := renewer.checkCertificate(true) 67 | if err != nil { 68 | return nil, errors.Wrap(err, "checking certificate failed") 69 | } 70 | } 71 | 72 | return renewer, nil 73 | } 74 | 75 | func (renewer *Renewer) Start(ctx context.Context, triggers <-chan struct{}) error { 76 | if renewer == nil { 77 | return errors.Errorf("invalid nil renewer") 78 | } 79 | 80 | err := renewer.checkCertificate(true) 81 | if err != nil { 82 | return errors.Wrap(err, "checking certificate failed") 83 | } 84 | 85 | ticker := time.NewTicker(30 * time.Second) 86 | defer ticker.Stop() 87 | 88 | for { 89 | select { 90 | case <-triggers: 91 | _ = renewer.checkCertificate(true) 92 | case <-ticker.C: 93 | _ = renewer.checkCertificate(false) 94 | case <-ctx.Done(): 95 | return nil 96 | } 97 | } 98 | } 99 | 100 | // CheckCertificate checks the certificate to be valid, renews it if it is 101 | // not and runs the after check functions. 102 | func (renewer *Renewer) checkCertificate(wasTriggered bool) error { 103 | if renewer == nil { 104 | return errors.Errorf("invalid nil renewer") 105 | } 106 | 107 | renewer.logger.Info("checking certificate") 108 | 109 | isValid, certificate, err := renewer.verifyCertificate() 110 | if err != nil { 111 | renewer.logger.Error(err, "verifying certificate failed") 112 | } 113 | 114 | var wasRenewed bool 115 | 116 | if isValid { 117 | renewer.logger.Info("certificate is valid") 118 | } else { 119 | renewer.logger.Info("certificate is invalid") 120 | 121 | certificate, err = renewer.renewCertificate() 122 | if err != nil { 123 | renewer.logger.Error(err, "renewing certificate failed") 124 | } 125 | 126 | wasRenewed = true 127 | } 128 | 129 | for _, afterCheckFunction := range renewer.afterCheckFunctions { 130 | err = afterCheckFunction(certificate, wasRenewed || wasTriggered) 131 | if err != nil { 132 | renewer.logger.Error(err, "running after check function failed") 133 | } 134 | } 135 | 136 | return nil 137 | } 138 | 139 | // renewCertificate generates and returns a new certificate for the stored DNS 140 | // names and writes them to the stored directory path. 141 | func (renewer *Renewer) renewCertificate() (*Certificate, error) { 142 | if renewer == nil { 143 | return nil, errors.Errorf("invalid nil renewer") 144 | } 145 | 146 | renewer.logger.Info("renewing certificate") 147 | 148 | certificate, err := NewCertificate(renewer.dnsNames) 149 | if err != nil { 150 | return nil, errors.WrapWithDetails( 151 | err, 152 | "creating new certificate for DNS names failed", 153 | "dnsNames", renewer.dnsNames, 154 | ) 155 | } 156 | 157 | err = certificate.Write(renewer.certificateDirectoryPath) 158 | if err != nil { 159 | return nil, errors.WrapWithDetails( 160 | err, 161 | "writing renewed certificate to directory failed", 162 | "directoryPath", renewer.certificateDirectoryPath, 163 | ) 164 | } 165 | 166 | return certificate, nil 167 | } 168 | 169 | // verifyCertificate verifies whether the underlying certificate is still valid. 170 | func (renewer *Renewer) verifyCertificate() (isValid bool, certificate *Certificate, err error) { 171 | if renewer == nil { 172 | return false, nil, errors.Errorf("invalid nil renewer") 173 | } 174 | 175 | renewer.logger.Info("verifying certificate") 176 | 177 | certificate, err = NewCertificateFromDirectory(renewer.certificateDirectoryPath) 178 | if err != nil { 179 | return false, nil, errors.WrapWithDetails( 180 | err, 181 | "loading certificate from directory failed", 182 | "directoryPath", renewer.certificateDirectoryPath, 183 | ) 184 | } 185 | 186 | renewer.logger.Info("verifying certificate") 187 | 188 | dnsNames := renewer.dnsNames 189 | if len(dnsNames) == 0 { 190 | dnsNames = append(dnsNames, "") 191 | } 192 | 193 | isValid = true 194 | 195 | for _, dnsName := range dnsNames { 196 | isValid = isValid && certificate.Verify(dnsName, time.Now().AddDate(0, 6, 0)) 197 | if !isValid { 198 | break 199 | } 200 | } 201 | 202 | return isValid, certificate, nil 203 | } 204 | 205 | // WithAfterCheckFunctions adds appends the specified functions to the existing 206 | // after check function chain. 207 | func (renewer *Renewer) WithAfterCheckFunctions(afterCheckFunctions ...AfterCheckFunctionType) { 208 | if renewer == nil { 209 | return 210 | } 211 | 212 | renewer.afterCheckFunctions = append(renewer.afterCheckFunctions, afterCheckFunctions...) 213 | } 214 | 215 | // WithDNSNames sets the DNS names used by the renewer in the certificate. 216 | func (renewer *Renewer) WithDNSNames(dnsNames ...string) { 217 | if renewer == nil { 218 | return 219 | } 220 | 221 | renewer.dnsNames = dnsNames 222 | } 223 | -------------------------------------------------------------------------------- /pkg/clustermeta/distribution.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package clustermeta 16 | 17 | import ( 18 | "context" 19 | "errors" 20 | 21 | corev1 "k8s.io/api/core/v1" 22 | "sigs.k8s.io/controller-runtime/pkg/client" 23 | ) 24 | 25 | var knownDistributions = []IsDistribution{ 26 | IsEKS, 27 | IsPKE, 28 | IsGKE, 29 | IsAKS, 30 | IsKIND, 31 | IsIKS, 32 | IsOpenShift, 33 | IsRKE, 34 | IsK3S, 35 | } 36 | 37 | type IsDistribution func(ctx context.Context, client client.Client, node *corev1.Node) (bool, string, error) 38 | 39 | type UnknownDistributionError struct{} 40 | 41 | func (UnknownDistributionError) Error() string { 42 | return "unknown distribution" 43 | } 44 | 45 | func IsUnknownDistributionError(err error) bool { 46 | return errors.As(err, &UnknownDistributionError{}) 47 | } 48 | 49 | func DetectDistribution(ctx context.Context, client client.Client, node *corev1.Node) (string, error) { 50 | for _, f := range knownDistributions { 51 | select { 52 | case <-ctx.Done(): 53 | return "", UnknownDistributionError{} 54 | default: 55 | if ok, distributionName, err := f(ctx, client, node); err != nil { 56 | return "", err 57 | } else if ok { 58 | return distributionName, nil 59 | } 60 | } 61 | } 62 | 63 | return "", UnknownDistributionError{} 64 | } 65 | -------------------------------------------------------------------------------- /pkg/clustermeta/distribution_test.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package clustermeta_test 16 | 17 | import ( 18 | "context" 19 | "io/ioutil" 20 | "testing" 21 | 22 | corev1 "k8s.io/api/core/v1" 23 | "k8s.io/apimachinery/pkg/runtime/serializer/json" 24 | "sigs.k8s.io/controller-runtime/pkg/client/fake" 25 | 26 | "github.com/cisco-open/cluster-registry-controller/pkg/clustermeta" 27 | ) 28 | 29 | const testdataDir = "testdata/" 30 | 31 | func TestDistributionDetector(t *testing.T) { 32 | t.Parallel() 33 | 34 | files, err := ioutil.ReadDir(testdataDir) 35 | if err != nil { 36 | t.Fatal(err) 37 | } 38 | 39 | testFiles := make(map[string][]byte) 40 | for _, file := range files { 41 | testFiles[file.Name()] = ReadFile(t, testdataDir+file.Name()) 42 | } 43 | 44 | testCases := []struct { 45 | filename string 46 | distribution string 47 | }{ 48 | { 49 | filename: "amazon-eks.yaml", 50 | distribution: "EKS", 51 | }, 52 | { 53 | filename: "amazon-pke.yaml", 54 | distribution: "PKE", 55 | }, 56 | { 57 | filename: "azure-aks.yaml", 58 | distribution: "AKS", 59 | }, 60 | { 61 | filename: "azure-pke.yaml", 62 | distribution: "PKE", 63 | }, 64 | { 65 | filename: "gcp-gke.yaml", 66 | distribution: "GKE", 67 | }, 68 | { 69 | filename: "vsphere-pke.yaml", 70 | distribution: "PKE", 71 | }, 72 | { 73 | filename: "kind-kind.yaml", 74 | distribution: "KIND", 75 | }, 76 | { 77 | filename: "cisco-iks.yaml", 78 | distribution: "IKS", 79 | }, 80 | { 81 | filename: "amazon-openshift.yaml", 82 | distribution: "OPENSHIFT", 83 | }, 84 | { 85 | filename: "unknown-rke.yaml", 86 | distribution: "RKE", 87 | }, 88 | { 89 | filename: "unknown-k3s.yaml", 90 | distribution: "K3S", 91 | }, 92 | } 93 | 94 | // with mocked client 95 | for _, tc := range testCases { 96 | tc := tc 97 | t.Run("withclient-"+tc.filename, func(t *testing.T) { 98 | t.Parallel() 99 | 100 | s := json.NewYAMLSerializer(json.DefaultMetaFactory, scheme, scheme) 101 | o, _, err := s.Decode(testFiles[tc.filename], nil, nil) 102 | if err != nil { 103 | t.Fatal(err) 104 | } 105 | 106 | nodeListerClient := fake.NewFakeClientWithScheme(scheme, o) 107 | foundDistribution, err := clustermeta.DetectDistribution(context.Background(), nodeListerClient, nil) 108 | if err != nil { 109 | t.Fatal(err) 110 | } 111 | if foundDistribution != tc.distribution { 112 | t.Fatalf("%s detected as '%s' and not '%s'", tc.filename, foundDistribution, tc.distribution) 113 | } 114 | }) 115 | } 116 | 117 | // with node instane 118 | for _, tc := range testCases { 119 | tc := tc 120 | t.Run(tc.filename, func(t *testing.T) { 121 | t.Parallel() 122 | s := json.NewYAMLSerializer(json.DefaultMetaFactory, scheme, scheme) 123 | o, _, err := s.Decode(testFiles[tc.filename], nil, nil) 124 | if err != nil { 125 | t.Fatal(err) 126 | } 127 | nodeListerClient := fake.NewFakeClientWithScheme(scheme, o) 128 | nodes := &corev1.NodeList{} 129 | err = nodeListerClient.List(context.Background(), nodes) 130 | if err != nil { 131 | t.Fatal(err) 132 | } 133 | foundDistribution, err := clustermeta.DetectDistribution(context.Background(), nil, &nodes.Items[0]) 134 | if err != nil { 135 | t.Fatal(err) 136 | } 137 | if foundDistribution != tc.distribution { 138 | t.Fatalf("%s detected as '%s' and not '%s'", tc.filename, foundDistribution, tc.distribution) 139 | } 140 | }) 141 | } 142 | } 143 | 144 | func TestUnknownDistributionDetector(t *testing.T) { 145 | t.Parallel() 146 | 147 | files, err := ioutil.ReadDir(testdataDir) 148 | if err != nil { 149 | t.Fatal(err) 150 | } 151 | 152 | testFiles := make(map[string][]byte) 153 | for _, file := range files { 154 | testFiles[file.Name()] = ReadFile(t, testdataDir+file.Name()) 155 | } 156 | 157 | testCases := []struct { 158 | filename string 159 | provider string 160 | }{{ 161 | filename: "unknown-distribution.yaml", 162 | provider: "unknown", 163 | }} 164 | 165 | for _, tc := range testCases { 166 | tc := tc 167 | t.Run(tc.filename, func(t *testing.T) { 168 | t.Parallel() 169 | 170 | s := json.NewYAMLSerializer(json.DefaultMetaFactory, scheme, scheme) 171 | o, _, err := s.Decode(testFiles[tc.filename], nil, nil) 172 | if err != nil { 173 | t.Fatal(err) 174 | } 175 | 176 | nodeListerClient := fake.NewFakeClientWithScheme(scheme, o) 177 | d, err := clustermeta.DetectDistribution(context.Background(), nodeListerClient, nil) 178 | if err == nil { 179 | t.Log(d) 180 | t.Fatal("unknown distribution detection ran without error") 181 | } 182 | if !clustermeta.IsUnknownDistributionError(err) { 183 | t.Fatalf("invalid error: %v", err) 184 | } 185 | }) 186 | } 187 | } 188 | -------------------------------------------------------------------------------- /pkg/clustermeta/helper_test.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package clustermeta_test 16 | 17 | import ( 18 | "io/ioutil" 19 | "testing" 20 | 21 | "k8s.io/apimachinery/pkg/runtime" 22 | clientgoscheme "k8s.io/client-go/kubernetes/scheme" 23 | ) 24 | 25 | var scheme = runtime.NewScheme() 26 | 27 | func init() { 28 | _ = clientgoscheme.AddToScheme(scheme) 29 | } 30 | 31 | // ReadFile reads the content of the given file or fails the test if an error is encountered. 32 | func ReadFile(tb testing.TB, file string) []byte { 33 | tb.Helper() 34 | content, err := ioutil.ReadFile(file) 35 | if err != nil { 36 | tb.Fatalf(err.Error()) 37 | } 38 | 39 | return content 40 | } 41 | -------------------------------------------------------------------------------- /pkg/clustermeta/metadata.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package clustermeta 16 | 17 | import ( 18 | "context" 19 | "strings" 20 | 21 | corev1 "k8s.io/api/core/v1" 22 | "sigs.k8s.io/controller-runtime/pkg/client" 23 | 24 | "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 25 | ) 26 | 27 | func GetClusterMetadata(ctx context.Context, client client.Client) (v1alpha1.ClusterMetadata, error) { 28 | md := v1alpha1.ClusterMetadata{} 29 | 30 | nodes := &corev1.NodeList{} 31 | if err := client.List(ctx, nodes); err != nil { 32 | return md, err 33 | } 34 | 35 | if len(nodes.Items) == 0 { 36 | return md, nil 37 | } 38 | 39 | provider, err := DetectProvider(ctx, client, &nodes.Items[0]) 40 | if err != nil && !IsUnknownProviderError(err) { 41 | return md, err 42 | } 43 | 44 | distribution, err := DetectDistribution(ctx, client, &nodes.Items[0]) 45 | if err != nil && !IsUnknownDistributionError(err) { 46 | return md, err 47 | } 48 | 49 | md.Provider = provider 50 | md.Distribution = distribution 51 | 52 | kubeProxyVersions := make(map[string]struct{}) 53 | kubeletVersions := make(map[string]struct{}) 54 | regions := make(map[string]struct{}) 55 | zones := make(map[string]struct{}) 56 | 57 | for _, node := range nodes.Items { 58 | kubeProxyVersions[node.Status.NodeInfo.KubeProxyVersion] = struct{}{} 59 | kubeletVersions[node.Status.NodeInfo.KubeletVersion] = struct{}{} 60 | if len(node.Labels) > 0 { 61 | if v := node.Labels[corev1.LabelZoneRegionStable]; v != "" { 62 | regions[v] = struct{}{} 63 | } 64 | if v := node.Labels[corev1.LabelZoneFailureDomainStable]; v != "" { 65 | zones[v] = struct{}{} 66 | } 67 | } 68 | } 69 | 70 | if len(kubeProxyVersions) > 0 { 71 | for v := range kubeProxyVersions { 72 | md.KubeProxyVersions = append(md.KubeProxyVersions, v) 73 | } 74 | 75 | md.Version = md.KubeProxyVersions[0] 76 | } 77 | 78 | for v := range kubeletVersions { 79 | md.KubeletVersions = append(md.KubeletVersions, v) 80 | } 81 | 82 | if len(regions) > 0 || len(zones) > 0 { 83 | md.Locality = &v1alpha1.Locality{ 84 | Regions: []string{}, 85 | Zones: []string{}, 86 | } 87 | } 88 | 89 | if len(regions) > 0 { 90 | for v := range regions { 91 | md.Locality.Regions = append(md.Locality.Regions, v) 92 | } 93 | md.Locality.Region = strings.Join(md.Locality.Regions, ", ") 94 | } 95 | 96 | for v := range zones { 97 | md.Locality.Zones = append(md.Locality.Zones, v) 98 | } 99 | 100 | return md, nil 101 | } 102 | -------------------------------------------------------------------------------- /pkg/clustermeta/provider.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package clustermeta 16 | 17 | import ( 18 | "context" 19 | "errors" 20 | "net/url" 21 | 22 | corev1 "k8s.io/api/core/v1" 23 | "sigs.k8s.io/controller-runtime/pkg/client" 24 | ) 25 | 26 | var knownProviders = []IsProvider{ 27 | IsAmazon, 28 | IsAzure, 29 | IsGoogle, 30 | IsVsphere, 31 | IsKind, 32 | IsCisco, 33 | } 34 | 35 | type IsProvider func(ctx context.Context, client client.Client, node *corev1.Node) (bool, string, error) 36 | 37 | type UnknownProviderError struct{} 38 | 39 | func (UnknownProviderError) Error() string { 40 | return "unknown provider" 41 | } 42 | 43 | func IsUnknownProviderError(err error) bool { 44 | return errors.As(err, &UnknownProviderError{}) 45 | } 46 | 47 | func DetectProvider(ctx context.Context, client client.Client, node *corev1.Node) (string, error) { 48 | for _, f := range knownProviders { 49 | select { 50 | case <-ctx.Done(): 51 | return "", UnknownProviderError{} 52 | default: 53 | if ok, providerName, err := f(ctx, client, node); err != nil { 54 | return "", err 55 | } else if ok { 56 | return providerName, nil 57 | } 58 | } 59 | } 60 | 61 | return "", UnknownProviderError{} 62 | } 63 | 64 | func getK8sNode(ctx context.Context, client client.Client) (*corev1.Node, bool, error) { 65 | nodes := &corev1.NodeList{} 66 | 67 | if err := client.List(ctx, nodes); err != nil { 68 | return nil, false, err 69 | } 70 | 71 | if len(nodes.Items) == 0 { 72 | return nil, false, nil 73 | } 74 | 75 | return &nodes.Items[0], true, nil 76 | } 77 | 78 | func detectNodeByProviderID(ctx context.Context, client client.Client, node *corev1.Node, scheme string) (bool, *corev1.Node, error) { 79 | var found bool 80 | var err error 81 | 82 | if node == nil { 83 | node, found, err = getK8sNode(ctx, client) 84 | if err != nil { 85 | return false, nil, err 86 | } 87 | if !found { 88 | return false, nil, nil 89 | } 90 | } 91 | 92 | if node.Spec.ProviderID != "" { 93 | u, err := url.Parse(node.Spec.ProviderID) 94 | if err != nil { 95 | return false, nil, err 96 | } 97 | 98 | if u.Scheme == scheme { 99 | return true, node, nil 100 | } 101 | } 102 | 103 | return false, node, nil 104 | } 105 | -------------------------------------------------------------------------------- /pkg/clustermeta/provider_test.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package clustermeta_test 16 | 17 | import ( 18 | "context" 19 | "io/ioutil" 20 | "testing" 21 | 22 | corev1 "k8s.io/api/core/v1" 23 | "k8s.io/apimachinery/pkg/runtime/serializer/json" 24 | "sigs.k8s.io/controller-runtime/pkg/client/fake" 25 | 26 | "github.com/cisco-open/cluster-registry-controller/pkg/clustermeta" 27 | ) 28 | 29 | func TestProviderDetector(t *testing.T) { 30 | t.Parallel() 31 | 32 | files, err := ioutil.ReadDir(testdataDir) 33 | if err != nil { 34 | t.Fatal(err) 35 | } 36 | 37 | testFiles := make(map[string][]byte) 38 | for _, file := range files { 39 | testFiles[file.Name()] = ReadFile(t, testdataDir+file.Name()) 40 | } 41 | 42 | testCases := []struct { 43 | filename string 44 | provider string 45 | }{ 46 | { 47 | filename: "amazon-eks.yaml", 48 | provider: "amazon", 49 | }, 50 | { 51 | filename: "amazon-pke.yaml", 52 | provider: "amazon", 53 | }, 54 | { 55 | filename: "azure-aks.yaml", 56 | provider: "azure", 57 | }, 58 | { 59 | filename: "azure-pke.yaml", 60 | provider: "azure", 61 | }, 62 | { 63 | filename: "gcp-gke.yaml", 64 | provider: "google", 65 | }, 66 | { 67 | filename: "vsphere-pke.yaml", 68 | provider: "vsphere", 69 | }, 70 | { 71 | filename: "kind-kind.yaml", 72 | provider: "kind", 73 | }, 74 | { 75 | filename: "cisco-iks.yaml", 76 | provider: "cisco", 77 | }, 78 | } 79 | 80 | // with mocked client 81 | for _, tc := range testCases { 82 | tc := tc 83 | t.Run("withclient-"+tc.filename, func(t *testing.T) { 84 | s := json.NewYAMLSerializer(json.DefaultMetaFactory, scheme, scheme) 85 | o, _, err := s.Decode(testFiles[tc.filename], nil, nil) 86 | if err != nil { 87 | t.Fatal(err) 88 | } 89 | 90 | nodeListerClient := fake.NewFakeClientWithScheme(scheme, o) 91 | foundProvider, err := clustermeta.DetectProvider(context.Background(), nodeListerClient, nil) 92 | if err != nil { 93 | t.Fatal(err) 94 | } 95 | if foundProvider != tc.provider { 96 | t.Fatalf("%s not detected as '%s' and not '%s'", tc.filename, foundProvider, tc.provider) 97 | } 98 | }) 99 | } 100 | 101 | // with node instane 102 | for _, tc := range testCases { 103 | tc := tc 104 | t.Run(tc.filename, func(t *testing.T) { 105 | t.Parallel() 106 | s := json.NewYAMLSerializer(json.DefaultMetaFactory, scheme, scheme) 107 | o, _, err := s.Decode(testFiles[tc.filename], nil, nil) 108 | if err != nil { 109 | t.Fatal(err) 110 | } 111 | 112 | nodeListerClient := fake.NewFakeClientWithScheme(scheme, o) 113 | nodes := &corev1.NodeList{} 114 | err = nodeListerClient.List(context.Background(), nodes) 115 | if err != nil { 116 | t.Fatal(err) 117 | } 118 | foundProvider, err := clustermeta.DetectProvider(context.Background(), nil, &nodes.Items[0]) 119 | if err != nil { 120 | t.Fatal(err) 121 | } 122 | if foundProvider != tc.provider { 123 | t.Fatalf("%s not detected as '%s' and not '%s'", tc.filename, foundProvider, tc.provider) 124 | } 125 | }) 126 | } 127 | } 128 | 129 | func TestUnknownProviderDetector(t *testing.T) { 130 | t.Parallel() 131 | dirName := "testdata/" 132 | files, err := ioutil.ReadDir(dirName) 133 | if err != nil { 134 | t.Fatal(err) 135 | } 136 | 137 | testFiles := make(map[string][]byte) 138 | for _, file := range files { 139 | testFiles[file.Name()] = ReadFile(t, dirName+file.Name()) 140 | } 141 | 142 | testCases := []struct { 143 | filename string 144 | provider string 145 | }{{ 146 | filename: "unknown-provider.yaml", 147 | provider: "unknown", 148 | }} 149 | 150 | for _, tc := range testCases { 151 | tc := tc 152 | t.Run(tc.filename, func(t *testing.T) { 153 | t.Parallel() 154 | 155 | s := json.NewYAMLSerializer(json.DefaultMetaFactory, scheme, scheme) 156 | o, _, err := s.Decode(testFiles[tc.filename], nil, nil) 157 | if err != nil { 158 | t.Fatal(err) 159 | } 160 | 161 | nodeListerClient := fake.NewFakeClientWithScheme(scheme, o) 162 | _, err = clustermeta.DetectProvider(context.Background(), nodeListerClient, nil) 163 | if err == nil { 164 | t.Fatal("unknown provider detection ran without error") 165 | } 166 | if !clustermeta.IsUnknownProviderError(err) { 167 | t.Fatalf("invalid error: %v", err) 168 | } 169 | }) 170 | } 171 | } 172 | -------------------------------------------------------------------------------- /pkg/clustermeta/providers.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package clustermeta 16 | 17 | import ( 18 | "context" 19 | 20 | corev1 "k8s.io/api/core/v1" 21 | "sigs.k8s.io/controller-runtime/pkg/client" 22 | ) 23 | 24 | const ( 25 | AMAZON = "amazon" 26 | AZURE = "azure" 27 | GOOGLE = "google" 28 | VSPHERE = "vsphere" 29 | KINDP = "kind" 30 | CISCO = "cisco" 31 | ) 32 | 33 | func IsAmazon(ctx context.Context, client client.Client, node *corev1.Node) (bool, string, error) { 34 | ok, _, err := detectNodeByProviderID(ctx, client, node, "aws") 35 | if err != nil { 36 | return ok, "", err 37 | } 38 | 39 | return ok, AMAZON, nil 40 | } 41 | 42 | func IsAzure(ctx context.Context, client client.Client, node *corev1.Node) (bool, string, error) { 43 | ok, _, err := detectNodeByProviderID(ctx, client, node, "azure") 44 | if err != nil { 45 | return ok, "", err 46 | } 47 | 48 | return ok, AZURE, nil 49 | } 50 | 51 | func IsGoogle(ctx context.Context, client client.Client, node *corev1.Node) (bool, string, error) { 52 | ok, _, err := detectNodeByProviderID(ctx, client, node, "gce") 53 | if err != nil { 54 | return ok, "", err 55 | } 56 | 57 | return ok, GOOGLE, nil 58 | } 59 | 60 | func IsVsphere(ctx context.Context, client client.Client, node *corev1.Node) (bool, string, error) { 61 | var ok bool 62 | var err error 63 | 64 | ok, node, err = detectNodeByProviderID(ctx, client, node, "vsphere") 65 | if err != nil { 66 | return ok, "", err 67 | } 68 | 69 | if node.Labels == nil { 70 | node.Labels = make(map[string]string) 71 | } 72 | 73 | if value, ok := node.Labels["iks.intersight.cisco.com/version"]; ok && value != "" { 74 | return false, "", nil 75 | } 76 | 77 | return ok, VSPHERE, nil 78 | } 79 | 80 | func IsKind(ctx context.Context, client client.Client, node *corev1.Node) (bool, string, error) { 81 | ok, _, err := detectNodeByProviderID(ctx, client, node, "kind") 82 | if err != nil { 83 | return ok, "", err 84 | } 85 | 86 | return ok, KINDP, nil 87 | } 88 | 89 | func IsCisco(ctx context.Context, client client.Client, node *corev1.Node) (bool, string, error) { 90 | ok, node, err := detectNodeByProviderID(ctx, client, node, "vsphere") 91 | if err != nil { 92 | return ok, "", err 93 | } 94 | 95 | if value, ok := node.Labels["iks.intersight.cisco.com/version"]; !ok || value == "" { 96 | return false, "", nil 97 | } 98 | 99 | return ok, CISCO, nil 100 | } 101 | -------------------------------------------------------------------------------- /pkg/clustermeta/testdata/amazon-eks.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Node 3 | metadata: 4 | annotations: 5 | node.alpha.kubernetes.io/ttl: "0" 6 | nodepool.banzaicloud.io/managed-labels: '["node.banzaicloud.io/cpu","node.banzaicloud.io/instanceTypeCategory","node.banzaicloud.io/memory","node.banzaicloud.io/networkPerfCategory","node.banzaicloud.io/ondemand","nodepool.banzaicloud.io/name"]' 7 | volumes.kubernetes.io/controller-managed-attach-detach: "true" 8 | creationTimestamp: "2021-03-08T23:04:47Z" 9 | labels: 10 | beta.kubernetes.io/arch: amd64 11 | beta.kubernetes.io/instance-type: t2.medium 12 | beta.kubernetes.io/os: linux 13 | failure-domain.beta.kubernetes.io/region: eu-central-1 14 | failure-domain.beta.kubernetes.io/zone: eu-central-1a 15 | kubernetes.io/arch: amd64 16 | kubernetes.io/hostname: ip-192-168-70-196.eu-central-1.compute.internal 17 | kubernetes.io/os: linux 18 | node.banzaicloud.io/cpu: "2" 19 | node.banzaicloud.io/instanceTypeCategory: General_purpose 20 | node.banzaicloud.io/memory: "4" 21 | node.banzaicloud.io/networkPerfCategory: low 22 | node.banzaicloud.io/ondemand: "false" 23 | node.kubernetes.io/instance-type: t2.medium 24 | nodepool.banzaicloud.io/name: pool1 25 | nodepool.banzaicloud.io/version: c67ce6aa812f6399dde59c9dc680db1a75f78275 26 | topology.kubernetes.io/region: eu-central-1 27 | topology.kubernetes.io/zone: eu-central-1a 28 | name: ip-192-168-70-196.eu-central-1.compute.internal 29 | resourceVersion: "1882" 30 | selfLink: /api/v1/nodes/ip-192-168-70-196.eu-central-1.compute.internal 31 | uid: 7ede515a-3032-4740-b59d-ab297f448b72 32 | spec: 33 | providerID: aws:///eu-central-1a/i-088b4f07708408cc0 34 | status: 35 | addresses: 36 | - address: 192.168.70.196 37 | type: InternalIP 38 | - address: 18.197.127.88 39 | type: ExternalIP 40 | - address: ip-192-168-70-196.eu-central-1.compute.internal 41 | type: Hostname 42 | - address: ip-192-168-70-196.eu-central-1.compute.internal 43 | type: InternalDNS 44 | - address: ec2-18-197-127-88.eu-central-1.compute.amazonaws.com 45 | type: ExternalDNS 46 | allocatable: 47 | attachable-volumes-aws-ebs: "39" 48 | cpu: 1930m 49 | ephemeral-storage: "47233297124" 50 | hugepages-2Mi: "0" 51 | memory: 3482572Ki 52 | pods: "17" 53 | capacity: 54 | attachable-volumes-aws-ebs: "39" 55 | cpu: "2" 56 | ephemeral-storage: 52416492Ki 57 | hugepages-2Mi: "0" 58 | memory: 4037580Ki 59 | pods: "17" 60 | conditions: 61 | - lastHeartbeatTime: "2021-03-08T23:06:48Z" 62 | lastTransitionTime: "2021-03-08T23:04:44Z" 63 | message: kubelet has sufficient memory available 64 | reason: KubeletHasSufficientMemory 65 | status: "False" 66 | type: MemoryPressure 67 | - lastHeartbeatTime: "2021-03-08T23:06:48Z" 68 | lastTransitionTime: "2021-03-08T23:04:44Z" 69 | message: kubelet has no disk pressure 70 | reason: KubeletHasNoDiskPressure 71 | status: "False" 72 | type: DiskPressure 73 | - lastHeartbeatTime: "2021-03-08T23:06:48Z" 74 | lastTransitionTime: "2021-03-08T23:04:44Z" 75 | message: kubelet has sufficient PID available 76 | reason: KubeletHasSufficientPID 77 | status: "False" 78 | type: PIDPressure 79 | - lastHeartbeatTime: "2021-03-08T23:06:48Z" 80 | lastTransitionTime: "2021-03-08T23:05:17Z" 81 | message: kubelet is posting ready status 82 | reason: KubeletReady 83 | status: "True" 84 | type: Ready 85 | daemonEndpoints: 86 | kubeletEndpoint: 87 | Port: 10250 88 | images: 89 | - names: 90 | - 602401143452.dkr.ecr.eu-central-1.amazonaws.com/amazon-k8s-cni@sha256:f310c918ee2b4ebced76d2d64a2ec128dde3b364d1b495f0ae73011f489d474d 91 | - 602401143452.dkr.ecr.eu-central-1.amazonaws.com/amazon-k8s-cni:v1.7.5-eksbuild.1 92 | sizeBytes: 312076970 93 | - names: 94 | - 602401143452.dkr.ecr.eu-central-1.amazonaws.com/amazon-k8s-cni-init@sha256:d96d712513464de6ce94e422634a25546565418f20d1b28d3bce399d578f3296 95 | - 602401143452.dkr.ecr.eu-central-1.amazonaws.com/amazon-k8s-cni-init:v1.7.5-eksbuild.1 96 | sizeBytes: 287782202 97 | - names: 98 | - 602401143452.dkr.ecr.eu-central-1.amazonaws.com/eks/kube-proxy@sha256:cbb2c85cbaa3d29d244eaec6ec5a8bbf765cc651590078ae30e9d210bac0c92a 99 | - 602401143452.dkr.ecr.eu-central-1.amazonaws.com/eks/kube-proxy:v1.17.9-eksbuild.1 100 | sizeBytes: 130676901 101 | - names: 102 | - k8s.gcr.io/autoscaling/cluster-autoscaler@sha256:cfc4c57fc3d6262457bb7da2b1bf20f23eafc1decacf34683d6acf8281f886e6 103 | - k8s.gcr.io/autoscaling/cluster-autoscaler:v1.17.4 104 | sizeBytes: 90458506 105 | - names: 106 | - traefik@sha256:5ec34caf19d114f8f0ed76f9bc3dad6ba8cf6d13a1575c4294b59b77709def39 107 | - traefik:1.7.20 108 | sizeBytes: 85706953 109 | - names: 110 | - ghcr.io/banzaicloud/instance-termination-handler@sha256:b9c415bf387ff2a19bab9f18c04525563e36796733454462fa037028d36c9e5a 111 | - ghcr.io/banzaicloud/instance-termination-handler:0.1.1 112 | sizeBytes: 46114957 113 | - names: 114 | - 602401143452.dkr.ecr.eu-central-1.amazonaws.com/eks/coredns@sha256:476c154960a843ac498376556fe5c42baad2f3ac690806b9989862064ab547c2 115 | - 602401143452.dkr.ecr.eu-central-1.amazonaws.com/eks/coredns:v1.6.6-eksbuild.1 116 | sizeBytes: 40859174 117 | - names: 118 | - ghcr.io/banzaicloud/nodepool-labels-operator@sha256:f46ab8c4a327f51dfb8cb47188a832d862531201d489c885f0c2798700f46f74 119 | - ghcr.io/banzaicloud/nodepool-labels-operator:v0.1.1 120 | sizeBytes: 39647988 121 | - names: 122 | - 602401143452.dkr.ecr.eu-central-1.amazonaws.com/eks/pause@sha256:1cb4ab85a3480446f9243178395e6bee7350f0d71296daeb6a9fdd221e23aea6 123 | - 602401143452.dkr.ecr.eu-central-1.amazonaws.com/eks/pause:3.1-eksbuild.1 124 | sizeBytes: 682696 125 | nodeInfo: 126 | architecture: amd64 127 | bootID: c2459348-a9de-41be-8654-661985cbd0ba 128 | containerRuntimeVersion: docker://19.3.6 129 | kernelVersion: 4.14.198-152.320.amzn2.x86_64 130 | kubeProxyVersion: v1.17.11-eks-cfdc40 131 | kubeletVersion: v1.17.11-eks-cfdc40 132 | machineID: 373e4bb11ccf4ee0a68b006c4826aa19 133 | operatingSystem: linux 134 | osImage: Amazon Linux 2 135 | systemUUID: EC2E70D2-A258-A245-AA55-A98555B8AB42 136 | -------------------------------------------------------------------------------- /pkg/clustermeta/testdata/amazon-pke.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Node 3 | metadata: 4 | annotations: 5 | io.cilium.network.ipv4-cilium-host: 10.20.0.203 6 | io.cilium.network.ipv4-health-ip: 10.20.0.133 7 | io.cilium.network.ipv4-pod-cidr: 10.20.0.0/24 8 | kubeadm.alpha.kubernetes.io/cri-socket: unix:///run/containerd/containerd.sock 9 | node.alpha.kubernetes.io/ttl: "0" 10 | nodepool.banzaicloud.io/managed-labels: '["nodepool.banzaicloud.io/name","node.banzaicloud.io/cpu","node.banzaicloud.io/instanceTypeCategory","node.banzaicloud.io/memory","node.banzaicloud.io/networkPerfCategory","node.banzaicloud.io/ondemand"]' 11 | volumes.kubernetes.io/controller-managed-attach-detach: "true" 12 | creationTimestamp: "2021-03-08T14:31:58Z" 13 | labels: 14 | beta.kubernetes.io/arch: amd64 15 | beta.kubernetes.io/instance-type: c5.large 16 | beta.kubernetes.io/os: linux 17 | failure-domain.beta.kubernetes.io/region: eu-central-1 18 | failure-domain.beta.kubernetes.io/zone: eu-central-1a 19 | kubernetes.io/arch: amd64 20 | kubernetes.io/hostname: ip-192-168-2-35.eu-central-1.compute.internal 21 | kubernetes.io/os: linux 22 | node-role.kubernetes.io/master: "" 23 | node.banzaicloud.io/cpu: "2" 24 | node.banzaicloud.io/instanceTypeCategory: Compute_optimized 25 | node.banzaicloud.io/memory: "4" 26 | node.banzaicloud.io/networkPerfCategory: high 27 | node.banzaicloud.io/ondemand: "false" 28 | node.kubernetes.io/instance-type: c5.large 29 | nodepool.banzaicloud.io/name: master 30 | topology.kubernetes.io/region: eu-central-1 31 | topology.kubernetes.io/zone: eu-central-1a 32 | name: ip-192-168-2-35.eu-central-1.compute.internal 33 | resourceVersion: "107595" 34 | selfLink: /api/v1/nodes/ip-192-168-2-35.eu-central-1.compute.internal 35 | uid: 27dddba5-1257-4258-8bf8-ecf866f30cf0 36 | spec: 37 | podCIDR: 10.20.0.0/24 38 | podCIDRs: 39 | - 10.20.0.0/24 40 | providerID: aws:///eu-central-1a/i-09ef514e292105377 41 | taints: 42 | - effect: NoSchedule 43 | key: node-role.kubernetes.io/master 44 | status: 45 | addresses: 46 | - address: 192.168.2.35 47 | type: InternalIP 48 | - address: 52.28.173.241 49 | type: ExternalIP 50 | - address: ip-192-168-2-35.eu-central-1.compute.internal 51 | type: Hostname 52 | - address: ip-192-168-2-35.eu-central-1.compute.internal 53 | type: InternalDNS 54 | - address: ec2-52-28-173-241.eu-central-1.compute.amazonaws.com 55 | type: ExternalDNS 56 | allocatable: 57 | attachable-volumes-aws-ebs: "25" 58 | cpu: 1850m 59 | ephemeral-storage: "44631645721" 60 | hugepages-1Gi: "0" 61 | hugepages-2Mi: "0" 62 | memory: 2662920Ki 63 | pods: "110" 64 | capacity: 65 | attachable-volumes-aws-ebs: "25" 66 | cpu: "2" 67 | ephemeral-storage: 50758604Ki 68 | hugepages-1Gi: "0" 69 | hugepages-2Mi: "0" 70 | memory: 3785224Ki 71 | pods: "110" 72 | conditions: 73 | - lastHeartbeatTime: "2021-03-08T14:33:06Z" 74 | lastTransitionTime: "2021-03-08T14:33:06Z" 75 | message: Cilium is running on this node 76 | reason: CiliumIsUp 77 | status: "False" 78 | type: NetworkUnavailable 79 | - lastHeartbeatTime: "2021-03-08T23:00:39Z" 80 | lastTransitionTime: "2021-03-08T14:31:54Z" 81 | message: kubelet has sufficient memory available 82 | reason: KubeletHasSufficientMemory 83 | status: "False" 84 | type: MemoryPressure 85 | - lastHeartbeatTime: "2021-03-08T23:00:39Z" 86 | lastTransitionTime: "2021-03-08T14:31:54Z" 87 | message: kubelet has no disk pressure 88 | reason: KubeletHasNoDiskPressure 89 | status: "False" 90 | type: DiskPressure 91 | - lastHeartbeatTime: "2021-03-08T23:00:39Z" 92 | lastTransitionTime: "2021-03-08T14:31:54Z" 93 | message: kubelet has sufficient PID available 94 | reason: KubeletHasSufficientPID 95 | status: "False" 96 | type: PIDPressure 97 | - lastHeartbeatTime: "2021-03-08T23:00:39Z" 98 | lastTransitionTime: "2021-03-08T14:33:02Z" 99 | message: kubelet is posting ready status. AppArmor enabled 100 | reason: KubeletReady 101 | status: "True" 102 | type: Ready 103 | daemonEndpoints: 104 | kubeletEndpoint: 105 | Port: 10250 106 | images: 107 | - names: 108 | - docker.io/banzaicloud/cilium@sha256:04734c47c93dd7dd7198bff555ab5b03943abd22a07124b37cdc44fe8a6ad731 109 | - docker.io/banzaicloud/cilium:v1.9.1 110 | sizeBytes: 152355763 111 | - names: 112 | - docker.io/banzaicloud/etcd@sha256:4198ba6f82f642dfd18ecf840ee37afb9df4b596f06eef20e44d0aec4ea27216 113 | - docker.io/banzaicloud/etcd:3.4.3-0 114 | sizeBytes: 100946002 115 | - names: 116 | - docker.io/banzaicloud/kube-apiserver@sha256:e9e916a58ccf6218055d95e5ca580d305d60e6e33b74dab47fa56a581213de47 117 | - docker.io/banzaicloud/kube-apiserver:v1.17.16 118 | sizeBytes: 50673258 119 | - names: 120 | - docker.io/banzaicloud/kube-proxy@sha256:55c953bef81858328811467acf6e3f7f1bc2634917df18770fc3874ff6f513f3 121 | - docker.io/banzaicloud/kube-proxy:v1.17.16 122 | sizeBytes: 49236344 123 | - names: 124 | - docker.io/banzaicloud/kube-controller-manager@sha256:b87d36909a94aabe84902338f8d4edd0f94e972403a5a3dd084bb9f9da8cf506 125 | - docker.io/banzaicloud/kube-controller-manager:v1.17.16 126 | sizeBytes: 48846956 127 | - names: 128 | - docker.io/banzaicloud/kube-scheduler@sha256:f38871fa02aff469d77d89c5212058389e763b21aeaf5139497097ee762af51a 129 | - docker.io/banzaicloud/kube-scheduler:v1.17.16 130 | sizeBytes: 33825234 131 | - names: 132 | - docker.io/banzaicloud/auto-approver@sha256:e96ae505c962540e19e68b4c218a50e7239123f6ae29a55f1fc3729d965a7ebc 133 | - docker.io/banzaicloud/auto-approver:0.1.0 134 | sizeBytes: 15960740 135 | - names: 136 | - docker.io/banzaicloud/cilium-operator@sha256:4b6c735761aecb1193542dcf359f9f70de7bca82d8966adf692176891247c17a 137 | - docker.io/banzaicloud/cilium-operator:v1.9.1 138 | sizeBytes: 15531407 139 | - names: 140 | - ghcr.io/banzaicloud/instance-termination-handler@sha256:b9c415bf387ff2a19bab9f18c04525563e36796733454462fa037028d36c9e5a 141 | - ghcr.io/banzaicloud/instance-termination-handler:0.1.1 142 | sizeBytes: 14640645 143 | - names: 144 | - docker.io/banzaicloud/coredns@sha256:608ac7ccba5ce41c6941fca13bc67059c1eef927fd968b554b790e21cc92543c 145 | - docker.io/banzaicloud/coredns:1.6.5 146 | sizeBytes: 13238295 147 | - names: 148 | - docker.io/banzaicloud/pause@sha256:fcaff905397ba63fd376d0c3019f1f1cb6e7506131389edbcb3d22719f1ae54d 149 | - docker.io/banzaicloud/pause:3.1 150 | sizeBytes: 325550 151 | nodeInfo: 152 | architecture: amd64 153 | bootID: 16c2a91f-8bde-4ee8-8fd3-de5a82535c0b 154 | containerRuntimeVersion: containerd://1.3.3 155 | kernelVersion: 5.4.0-1035-aws 156 | kubeProxyVersion: v1.17.16 157 | kubeletVersion: v1.17.16 158 | machineID: ec24eee07ebc803045c4af65883288fd 159 | operatingSystem: linux 160 | osImage: Ubuntu 20.04.1 LTS 161 | systemUUID: ec26490c-6060-dcc5-2ee7-245219eb0727 162 | -------------------------------------------------------------------------------- /pkg/clustermeta/testdata/cisco-iks.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Node 3 | metadata: 4 | annotations: 5 | alpha.kubernetes.io/provided-node-ip: 10.10.0.159 6 | csi.volume.kubernetes.io/nodeid: '{"csi.vsphere.vmware.com":"backend-wrkr-backe-7fba92b1c7"}' 7 | iks.intersight.cisco.com/cloud-init: success 8 | kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock 9 | node.alpha.kubernetes.io/ttl: "0" 10 | projectcalico.org/IPv4Address: 10.10.0.159/16 11 | projectcalico.org/IPv4IPIPTunnelAddr: 100.65.254.64 12 | volumes.kubernetes.io/controller-managed-attach-detach: "true" 13 | creationTimestamp: "2021-09-13T15:59:24Z" 14 | labels: 15 | beta.kubernetes.io/arch: amd64 16 | beta.kubernetes.io/instance-type: vsphere-vm.cpu-8.mem-24gb.os-ubuntu 17 | beta.kubernetes.io/os: linux 18 | iks-nodegroup: wrkr-backend 19 | iks.intersight.cisco.com/version: 60627cb27a6f722d30786563 20 | kubernetes.io/arch: amd64 21 | kubernetes.io/hostname: backend-wrkr-backe-7fba92b1c7 22 | kubernetes.io/os: linux 23 | name: backend-wrkr-backe-7fba92b1c7 24 | resourceVersion: "2635929" 25 | selfLink: /api/v1/nodes/backend-wrkr-backe-7fba92b1c7 26 | uid: 68903112-d4a0-4896-9b65-3e7e994e9e34 27 | spec: 28 | podCIDR: 100.65.3.0/24 29 | podCIDRs: 30 | - 100.65.3.0/24 31 | providerID: vsphere://421856ac-7d4a-9e14-863c-e0ed8f35fdf8 32 | status: 33 | addresses: 34 | - address: backend-wrkr-backe-7fba92b1c7 35 | type: Hostname 36 | - address: 10.10.0.159 37 | type: ExternalIP 38 | - address: 10.10.0.159 39 | type: InternalIP 40 | allocatable: 41 | cpu: "8" 42 | ephemeral-storage: "37509248962" 43 | hugepages-1Gi: "0" 44 | hugepages-2Mi: "0" 45 | memory: 24580028Ki 46 | pods: "110" 47 | capacity: 48 | cpu: "8" 49 | ephemeral-storage: 40700140Ki 50 | hugepages-1Gi: "0" 51 | hugepages-2Mi: "0" 52 | memory: 24682428Ki 53 | pods: "110" 54 | conditions: 55 | - lastHeartbeatTime: "2021-09-13T16:07:48Z" 56 | lastTransitionTime: "2021-09-13T16:07:48Z" 57 | message: Calico is running on this node 58 | reason: CalicoIsUp 59 | status: "False" 60 | type: NetworkUnavailable 61 | - lastHeartbeatTime: "2021-09-15T17:57:31Z" 62 | lastTransitionTime: "2021-09-13T15:59:23Z" 63 | message: kubelet has sufficient memory available 64 | reason: KubeletHasSufficientMemory 65 | status: "False" 66 | type: MemoryPressure 67 | - lastHeartbeatTime: "2021-09-15T17:57:31Z" 68 | lastTransitionTime: "2021-09-13T15:59:23Z" 69 | message: kubelet has no disk pressure 70 | reason: KubeletHasNoDiskPressure 71 | status: "False" 72 | type: DiskPressure 73 | - lastHeartbeatTime: "2021-09-15T17:57:31Z" 74 | lastTransitionTime: "2021-09-13T15:59:23Z" 75 | message: kubelet has sufficient PID available 76 | reason: KubeletHasSufficientPID 77 | status: "False" 78 | type: PIDPressure 79 | - lastHeartbeatTime: "2021-09-15T17:57:31Z" 80 | lastTransitionTime: "2021-09-13T16:07:52Z" 81 | message: kubelet is posting ready status. AppArmor enabled 82 | reason: KubeletReady 83 | status: "True" 84 | type: Ready 85 | daemonEndpoints: 86 | kubeletEndpoint: 87 | Port: 10250 88 | nodeInfo: 89 | architecture: amd64 90 | bootID: 5088bd84-d480-4162-9e5f-e81c77790896 91 | containerRuntimeVersion: docker://19.3.13 92 | kernelVersion: 4.15.0-134-generic 93 | kubeProxyVersion: v1.19.5 94 | kubeletVersion: v1.19.5 95 | machineID: a729ed1efc1f4387b985963182b9fde6 96 | operatingSystem: linux 97 | osImage: Ubuntu 18.04.5 LTS 98 | systemUUID: AC561842-4A7D-149E-863C-E0ED8F35FDF8 99 | -------------------------------------------------------------------------------- /pkg/clustermeta/testdata/kind-kind.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Node 3 | metadata: 4 | annotations: 5 | kubeadm.alpha.kubernetes.io/cri-socket: unix:///run/containerd/containerd.sock 6 | node.alpha.kubernetes.io/ttl: "0" 7 | volumes.kubernetes.io/controller-managed-attach-detach: "true" 8 | creationTimestamp: "2021-03-12T22:12:20Z" 9 | labels: 10 | beta.kubernetes.io/arch: amd64 11 | beta.kubernetes.io/os: linux 12 | kubernetes.io/arch: amd64 13 | kubernetes.io/hostname: kind-control-plane 14 | kubernetes.io/os: linux 15 | node-role.kubernetes.io/master: "" 16 | name: kind-control-plane 17 | resourceVersion: "463" 18 | selfLink: /api/v1/nodes/kind-control-plane 19 | uid: 2361a47c-c791-49a8-b803-ef568e5dd9af 20 | spec: 21 | podCIDR: 10.244.0.0/24 22 | podCIDRs: 23 | - 10.244.0.0/24 24 | providerID: kind://docker/kind/kind-control-plane 25 | status: 26 | addresses: 27 | - address: 172.18.0.2 28 | type: InternalIP 29 | - address: kind-control-plane 30 | type: Hostname 31 | allocatable: 32 | cpu: "4" 33 | ephemeral-storage: 41021664Ki 34 | hugepages-1Gi: "0" 35 | hugepages-2Mi: "0" 36 | memory: 4033432Ki 37 | pods: "110" 38 | capacity: 39 | cpu: "4" 40 | ephemeral-storage: 41021664Ki 41 | hugepages-1Gi: "0" 42 | hugepages-2Mi: "0" 43 | memory: 4033432Ki 44 | pods: "110" 45 | conditions: 46 | - lastHeartbeatTime: "2021-03-12T22:12:54Z" 47 | lastTransitionTime: "2021-03-12T22:12:17Z" 48 | message: kubelet has sufficient memory available 49 | reason: KubeletHasSufficientMemory 50 | status: "False" 51 | type: MemoryPressure 52 | - lastHeartbeatTime: "2021-03-12T22:12:54Z" 53 | lastTransitionTime: "2021-03-12T22:12:17Z" 54 | message: kubelet has no disk pressure 55 | reason: KubeletHasNoDiskPressure 56 | status: "False" 57 | type: DiskPressure 58 | - lastHeartbeatTime: "2021-03-12T22:12:54Z" 59 | lastTransitionTime: "2021-03-12T22:12:17Z" 60 | message: kubelet has sufficient PID available 61 | reason: KubeletHasSufficientPID 62 | status: "False" 63 | type: PIDPressure 64 | - lastHeartbeatTime: "2021-03-12T22:12:54Z" 65 | lastTransitionTime: "2021-03-12T22:12:54Z" 66 | message: kubelet is posting ready status 67 | reason: KubeletReady 68 | status: "True" 69 | type: Ready 70 | daemonEndpoints: 71 | kubeletEndpoint: 72 | Port: 10250 73 | images: 74 | - names: 75 | - k8s.gcr.io/etcd:3.4.3-0 76 | sizeBytes: 289997247 77 | - names: 78 | - k8s.gcr.io/kube-apiserver:v1.17.5 79 | sizeBytes: 144466737 80 | - names: 81 | - k8s.gcr.io/kube-proxy:v1.17.5 82 | sizeBytes: 132100222 83 | - names: 84 | - k8s.gcr.io/kube-controller-manager:v1.17.5 85 | sizeBytes: 131244355 86 | - names: 87 | - docker.io/kindest/kindnetd:0.5.4 88 | sizeBytes: 113207016 89 | - names: 90 | - k8s.gcr.io/kube-scheduler:v1.17.5 91 | sizeBytes: 111947057 92 | - names: 93 | - k8s.gcr.io/debian-base:v2.0.0 94 | sizeBytes: 53884301 95 | - names: 96 | - docker.io/rancher/local-path-provisioner:v0.0.12 97 | sizeBytes: 41994847 98 | - names: 99 | - k8s.gcr.io/coredns:1.6.5 100 | sizeBytes: 41705951 101 | - names: 102 | - k8s.gcr.io/pause:3.1 103 | sizeBytes: 746479 104 | nodeInfo: 105 | architecture: amd64 106 | bootID: dc84f3de-cdab-4c47-8328-efef8fdcf010 107 | containerRuntimeVersion: containerd://1.3.3-14-g449e9269 108 | kernelVersion: 4.19.121-linuxkit 109 | kubeProxyVersion: v1.17.5 110 | kubeletVersion: v1.17.5 111 | machineID: 8e7589a9984f4994883aafe8ebf3caef 112 | operatingSystem: linux 113 | osImage: Ubuntu 19.10 114 | systemUUID: 1bd32c91-d33f-4065-8e24-0fd56b19de91 115 | -------------------------------------------------------------------------------- /pkg/clustermeta/testdata/unknown-k3s.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Node 3 | metadata: 4 | annotations: 5 | flannel.alpha.coreos.com/backend-data: '{"VNI":1,"VtepMAC":""}' 6 | flannel.alpha.coreos.com/backend-type: vxlan 7 | flannel.alpha.coreos.com/kube-subnet-manager: "true" 8 | flannel.alpha.coreos.com/public-ip: 1.1.1.1 9 | k3s.io/internal-ip: 1.1.1.1 10 | k3s.io/node-args: '["agent"]' 11 | node.alpha.kubernetes.io/ttl: "0" 12 | volumes.kubernetes.io/controller-managed-attach-detach: "true" 13 | creationTimestamp: "2023-06-27T17:30:22Z" 14 | finalizers: 15 | - wrangler.cattle.io/node 16 | labels: 17 | beta.kubernetes.io/arch: amd64 18 | beta.kubernetes.io/instance-type: k3s 19 | beta.kubernetes.io/os: linux 20 | kubernetes.io/arch: amd64 21 | kubernetes.io/hostname: ip- 22 | kubernetes.io/os: linux 23 | node.kubernetes.io/instance-type: k3s 24 | name: test 25 | resourceVersion: "55091" 26 | uid: 5b31c507-9363-4124-bf73-c2a771cc7074 27 | spec: 28 | podCIDR: 10.42.1.0/24 29 | podCIDRs: 30 | - 10.42.1.0/24 31 | providerID: k3s://ip-1.2.3.4 32 | status: 33 | addresses: 34 | - address: 172.31.11.253 35 | type: InternalIP 36 | - address: ip-1.2.3.4 37 | type: Hostname 38 | allocatable: 39 | cpu: "8" 40 | ephemeral-storage: "39369928059" 41 | hugepages-1Gi: "0" 42 | hugepages-2Mi: "0" 43 | memory: 15911164Ki 44 | pods: "110" 45 | capacity: 46 | cpu: "8" 47 | ephemeral-storage: 40470732Ki 48 | hugepages-1Gi: "0" 49 | hugepages-2Mi: "0" 50 | memory: 15911164Ki 51 | pods: "110" 52 | conditions: 53 | - lastHeartbeatTime: "2023-06-27T19:03:36Z" 54 | lastTransitionTime: "2023-06-27T17:30:22Z" 55 | message: kubelet has sufficient memory available 56 | reason: KubeletHasSufficientMemory 57 | status: "False" 58 | type: MemoryPressure 59 | - lastHeartbeatTime: "2023-06-27T19:03:36Z" 60 | lastTransitionTime: "2023-06-27T17:30:22Z" 61 | message: kubelet has no disk pressure 62 | reason: KubeletHasNoDiskPressure 63 | status: "False" 64 | type: DiskPressure 65 | - lastHeartbeatTime: "2023-06-27T19:03:36Z" 66 | lastTransitionTime: "2023-06-27T17:30:22Z" 67 | message: kubelet has sufficient PID available 68 | reason: KubeletHasSufficientPID 69 | status: "False" 70 | type: PIDPressure 71 | - lastHeartbeatTime: "2023-06-27T19:03:36Z" 72 | lastTransitionTime: "2023-06-27T17:30:32Z" 73 | message: kubelet is posting ready status. AppArmor enabled 74 | reason: KubeletReady 75 | status: "True" 76 | type: Ready 77 | daemonEndpoints: 78 | kubeletEndpoint: 79 | Port: 10250 80 | images: 81 | - names: 82 | - docker.io/rancher/klipper-lb@sha256:2b963c02974155f7e9a51c54b91f09099e48b4550689aadb595e62118e045c10 83 | - docker.io/rancher/klipper-lb:v0.4.3 84 | sizeBytes: 4279717 85 | - names: 86 | - docker.io/rancher/mirrored-pause@sha256:74c4244427b7312c5b901fe0f67cbc53683d06f4f24c6faee65d4182bf0fa893 87 | - docker.io/rancher/mirrored-pause:3.6 88 | sizeBytes: 301463 89 | nodeInfo: 90 | architecture: amd64 91 | bootID: 541ccad7-f0ef-4d0f-a1ec-536b8ebc1735 92 | containerRuntimeVersion: containerd://1.7.1-k3s1 93 | kernelVersion: 5.15.0-1011-aws 94 | kubeProxyVersion: v1.24.14+k3s1 95 | kubeletVersion: v1.24.14+k3s1 96 | machineID: ec27585019c42c9186e2b6c677eb01ff 97 | operatingSystem: linux 98 | osImage: Ubuntu 22.04 LTS 99 | systemUUID: ec275850-19c4-2c91-86e2-b6c677eb01ff 100 | -------------------------------------------------------------------------------- /pkg/clustermeta/testdata/unknown-rke.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Node 3 | metadata: 4 | annotations: 5 | flannel.alpha.coreos.com/backend-data: '{"VNI":1,"VtepMAC":"66:52:b5:9b:39:6d"}' 6 | flannel.alpha.coreos.com/backend-type: vxlan 7 | flannel.alpha.coreos.com/kube-subnet-manager: "true" 8 | flannel.alpha.coreos.com/public-ip: 1.2.3.4 9 | node.alpha.kubernetes.io/ttl: "0" 10 | projectcalico.org/IPv4Address: 1.2.3.4/30 11 | projectcalico.org/IPv4IPIPTunnelAddr: 10.42.0.1 12 | rke.cattle.io/external-ip: 1.2.3.4 13 | rke.cattle.io/internal-ip: 1.2.3.4 14 | volumes.kubernetes.io/controller-managed-attach-detach: "true" 15 | creationTimestamp: "2023-06-28T10:23:54Z" 16 | labels: 17 | beta.kubernetes.io/arch: amd64 18 | beta.kubernetes.io/os: linux 19 | kubernetes.io/arch: amd64 20 | kubernetes.io/hostname: 1.2.3.4 21 | kubernetes.io/os: linux 22 | node-role.kubernetes.io/controlplane: "true" 23 | node-role.kubernetes.io/etcd: "true" 24 | node-role.kubernetes.io/worker: "true" 25 | name: 1.2.3.4 26 | resourceVersion: "4501" 27 | uid: 3591114e-12dd-4533-a8f1-673b7de61fd1 28 | spec: 29 | podCIDR: 10.42.0.0/24 30 | podCIDRs: 31 | - 10.42.0.0/24 32 | status: 33 | addresses: 34 | - address: 1.2.3.4 35 | type: InternalIP 36 | - address: 1.2.3.4 37 | type: Hostname 38 | allocatable: 39 | cpu: "4" 40 | ephemeral-storage: "6627384105" 41 | hugepages-1Gi: "0" 42 | hugepages-2Mi: "0" 43 | memory: 3910472Ki 44 | pods: "110" 45 | capacity: 46 | cpu: "4" 47 | ephemeral-storage: 7191172Ki 48 | hugepages-1Gi: "0" 49 | hugepages-2Mi: "0" 50 | memory: 4012872Ki 51 | pods: "110" 52 | conditions: 53 | - lastHeartbeatTime: "2023-06-30T18:00:03Z" 54 | lastTransitionTime: "2023-06-30T18:00:03Z" 55 | message: Flannel is running on this node 56 | reason: FlannelIsUp 57 | status: "False" 58 | type: NetworkUnavailable 59 | - lastHeartbeatTime: "2023-06-30T18:22:12Z" 60 | lastTransitionTime: "2023-06-28T10:23:54Z" 61 | message: kubelet has sufficient memory available 62 | reason: KubeletHasSufficientMemory 63 | status: "False" 64 | type: MemoryPressure 65 | - lastHeartbeatTime: "2023-06-30T18:22:12Z" 66 | lastTransitionTime: "2023-06-28T10:23:54Z" 67 | message: kubelet has no disk pressure 68 | reason: KubeletHasNoDiskPressure 69 | status: "False" 70 | type: DiskPressure 71 | - lastHeartbeatTime: "2023-06-30T18:22:12Z" 72 | lastTransitionTime: "2023-06-28T10:23:54Z" 73 | message: kubelet has sufficient PID available 74 | reason: KubeletHasSufficientPID 75 | status: "False" 76 | type: PIDPressure 77 | - lastHeartbeatTime: "2023-06-30T18:22:12Z" 78 | lastTransitionTime: "2023-06-30T17:59:47Z" 79 | message: kubelet is posting ready status. AppArmor enabled 80 | reason: KubeletReady 81 | status: "True" 82 | type: Ready 83 | daemonEndpoints: 84 | kubeletEndpoint: 85 | Port: 10250 86 | images: 87 | - names: 88 | - rancher/hyperkube@sha256:df1f498e106eff587fdae2ebf3c24019063392a7c1d16f84a64e2494b899eb1c 89 | - rancher/hyperkube:v1.25.6-rancher4 90 | sizeBytes: 2197934780 91 | - names: 92 | - rancher/rke-tools@sha256:c335ad107733ac03738fb69d98082caf88bdc29d8eb31191b75a44e5baa34afc 93 | - rancher/rke-tools:v0.1.87 94 | sizeBytes: 277342040 95 | - names: 96 | - rancher/nginx-ingress-controller@sha256:d47e59a054e3aee7a731b575e463ce05f29783945529d2116a0b28e0499f8152 97 | - rancher/nginx-ingress-controller:nginx-1.5.1-rancher2 98 | sizeBytes: 269799451 99 | - names: 100 | - rancher/mirrored-calico-node@sha256:d30a70114c8df718b957db1ffd07fbc53ba44e860ce2be19fcac95accd8026a8 101 | - rancher/mirrored-calico-node:v3.24.1 102 | sizeBytes: 222525166 103 | - names: 104 | - rancher/calico-cni@sha256:01265bdaecee7f38781e36d2154294725e72f418d9a8dde8187a0068b786a2c2 105 | - rancher/calico-cni:v3.24.1-rancher1 106 | sizeBytes: 201353594 107 | - names: 108 | - rancher/mirrored-coreos-etcd@sha256:c39e96f43b58fc6fe85c6712f0b0c4958ebdd57acb54e7a73965cf3c4d8a3320 109 | - rancher/mirrored-coreos-etcd:v3.5.6 110 | sizeBytes: 181461511 111 | - names: 112 | - rancher/mirrored-calico-kube-controllers@sha256:045647bf84a4e9d4849f8a1d11152b9e16db4127441e8777a77a9b19d6e88759 113 | - rancher/mirrored-calico-kube-controllers:v3.24.1 114 | sizeBytes: 71313096 115 | - names: 116 | - rancher/mirrored-metrics-server@sha256:16185c0d4d01f8919eca4779c69a374c184200cd9e6eded9ba53052fd73578df 117 | - rancher/mirrored-metrics-server:v0.6.2 118 | sizeBytes: 68892890 119 | - names: 120 | - rancher/mirrored-flannelcni-flannel@sha256:c9786f434d4663c924aeca1a2e479786d63df0d56c5d6bd62a64915f81d62ff0 121 | - rancher/mirrored-flannelcni-flannel:v0.19.2 122 | sizeBytes: 62313830 123 | - names: 124 | - ghcr.io/cisco-open/cluster-registry-controller@sha256:937eff91df1ec235db987826248ba64c040ad9bd9725a2c36c9cd53b23dc4b11 125 | - ghcr.io/cisco-open/cluster-registry-controller:v0.2.12 126 | sizeBytes: 50002021 127 | - names: 128 | - rancher/mirrored-coredns-coredns@sha256:823626055cba80e2ad6ff26e18df206c7f26964c7cd81a8ef57b4dc16c0eec61 129 | - rancher/mirrored-coredns-coredns:1.9.4 130 | sizeBytes: 49802873 131 | - names: 132 | - rancher/mirrored-cluster-proportional-autoscaler@sha256:d9333aded9b1a0526a0c756f9c72037abd742a08ed099575588175a8d9e29cee 133 | - rancher/mirrored-cluster-proportional-autoscaler:1.8.6 134 | sizeBytes: 47910609 135 | - names: 136 | - rancher/mirrored-ingress-nginx-kube-webhook-certgen@sha256:28197903d736aae74cbb1fa9e0ccbd11129395f0f65ad94281cc7fdfec020b25 137 | - rancher/mirrored-ingress-nginx-kube-webhook-certgen:v1.1.1 138 | sizeBytes: 47736388 139 | - names: 140 | - rancher/mirrored-pause@sha256:74c4244427b7312c5b901fe0f67cbc53683d06f4f24c6faee65d4182bf0fa893 141 | - rancher/mirrored-pause:3.6 142 | sizeBytes: 682696 143 | nodeInfo: 144 | architecture: amd64 145 | bootID: 535a110c-df41-43d1-b3fc-38ef3fab2d80 146 | containerRuntimeVersion: docker://20.10.24 147 | kernelVersion: 5.19.0-0.deb11.2-amd64 148 | kubeProxyVersion: v1.25.6 149 | kubeletVersion: v1.25.6 150 | machineID: 50ca20960ea94552bd5ef84a20ce7e47 151 | operatingSystem: linux 152 | osImage: Debian GNU/Linux 11 (bullseye) 153 | systemUUID: 50b00101-2870-0381-61f6-ab754bc3177c 154 | -------------------------------------------------------------------------------- /pkg/clusters/cluster_features.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package clusters 16 | 17 | import ( 18 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 19 | "k8s.io/apimachinery/pkg/labels" 20 | ) 21 | 22 | type ClusterFeature interface { 23 | GetUID() string 24 | GetName() string 25 | GetLabels() map[string]string 26 | } 27 | 28 | type clusterFeature struct { 29 | UID string 30 | Name string 31 | Labels map[string]string 32 | } 33 | 34 | func (f clusterFeature) GetUID() string { 35 | return f.UID 36 | } 37 | 38 | func (f clusterFeature) GetName() string { 39 | return f.Name 40 | } 41 | 42 | func (f clusterFeature) GetLabels() map[string]string { 43 | return f.Labels 44 | } 45 | 46 | func NewClusterFeature(uid, name string, labels map[string]string) ClusterFeature { 47 | return clusterFeature{ 48 | UID: uid, 49 | Name: name, 50 | Labels: labels, 51 | } 52 | } 53 | 54 | type ClusterFeatureRequirement struct { 55 | Name string 56 | MatchLabels map[string]string 57 | MatchExpressions []metav1.LabelSelectorRequirement 58 | } 59 | 60 | func (r ClusterFeatureRequirement) Match(features map[string]ClusterFeature) bool { 61 | for _, feature := range features { 62 | if r.Name != "" && r.Name != feature.GetName() { 63 | continue 64 | } 65 | 66 | if len(r.MatchLabels) == 0 && len(r.MatchExpressions) == 0 { 67 | return true 68 | } 69 | 70 | matcher, err := metav1.LabelSelectorAsSelector(&metav1.LabelSelector{ 71 | MatchLabels: r.MatchLabels, 72 | MatchExpressions: r.MatchExpressions, 73 | }) 74 | if err != nil { 75 | return false 76 | } 77 | if matcher.Matches(labels.Set(feature.GetLabels())) { 78 | return true 79 | } 80 | } 81 | 82 | return false 83 | } 84 | -------------------------------------------------------------------------------- /pkg/clusters/cluster_test.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package clusters_test 16 | 17 | import ( 18 | "context" 19 | "testing" 20 | 21 | "github.com/go-logr/logr" 22 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 23 | "k8s.io/client-go/rest" 24 | 25 | "github.com/cisco-open/cluster-registry-controller/pkg/clusters" 26 | ) 27 | 28 | func TestClusterFeatures(t *testing.T) { 29 | t.Parallel() 30 | 31 | cl, err := clusters.NewCluster(context.Background(), "test", &rest.Config{}, logr.Discard()) 32 | if err != nil { 33 | panic(err) 34 | } 35 | 36 | cf := clusters.NewClusterFeature("test-feature", "test-feature", 37 | map[string]string{ 38 | "testlabel": "testlabelvalue", 39 | "testlabelkey": "something", 40 | }, 41 | ) 42 | 43 | cf2 := clusters.NewClusterFeature("another-test-feature", "another-test-feature", nil) 44 | 45 | r := clusters.NewManagedReconciler("test", logr.Discard()) 46 | c := clusters.NewManagedController("test", r, logr.Discard(), clusters.WithRequiredClusterFeatures( 47 | clusters.ClusterFeatureRequirement{ 48 | Name: "test-feature", 49 | MatchLabels: map[string]string{ 50 | "testlabel": "testlabelvalue", 51 | }, 52 | MatchExpressions: []metav1.LabelSelectorRequirement{ 53 | { 54 | Key: "testlabelkey", 55 | Operator: metav1.LabelSelectorOpExists, 56 | }, 57 | }, 58 | }, 59 | clusters.ClusterFeatureRequirement{ 60 | Name: "another-test-feature", 61 | }, 62 | )) 63 | 64 | err = cl.AddController(c) 65 | if err != nil { 66 | panic(err) 67 | } 68 | 69 | if len(cl.GetControllers()) != 0 { 70 | t.Fatalf("controllers count != 0") 71 | } 72 | 73 | if len(cl.GetPendingControllers()) != 1 { 74 | t.Fatalf("pending controllers count != 1") 75 | } 76 | 77 | cl.AddFeature(cf) 78 | 79 | if len(cl.GetControllers()) != 0 { 80 | t.Fatalf("controllers count != 0 after the first feature is added") 81 | } 82 | if len(cl.GetPendingControllers()) != 1 { 83 | t.Fatalf("pending controllers != 1 after the first feature is added") 84 | } 85 | 86 | cl.AddFeature(cf2) 87 | if len(cl.GetControllers()) != 1 { 88 | t.Fatalf("controllers count != 1 after the second feature is added") 89 | } 90 | if len(cl.GetPendingControllers()) != 0 { 91 | t.Fatalf("pending controllers != 0 after the second feature is added") 92 | } 93 | 94 | cl.RemoveFeature(cf.GetUID()) 95 | if len(cl.GetControllers()) != 0 { 96 | t.Fatalf("controllers count != 0 after the feature is removed") 97 | } 98 | if len(cl.GetPendingControllers()) != 1 { 99 | t.Fatalf("pending controllers != 1 after the feature is removed") 100 | } 101 | 102 | cl.RemoveController(c) 103 | if len(cl.GetControllers()) != 0 { 104 | t.Fatalf("controllers count != 0 after the controller is removed") 105 | } 106 | if len(cl.GetPendingControllers()) != 0 { 107 | t.Fatalf("pending controllers != 0 after the controller is removed") 108 | } 109 | } 110 | -------------------------------------------------------------------------------- /pkg/clusters/managed_reconciler.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package clusters 16 | 17 | import ( 18 | "context" 19 | 20 | "github.com/go-logr/logr" 21 | "k8s.io/apimachinery/pkg/runtime" 22 | "k8s.io/client-go/tools/record" 23 | ctrl "sigs.k8s.io/controller-runtime" 24 | "sigs.k8s.io/controller-runtime/pkg/client" 25 | "sigs.k8s.io/controller-runtime/pkg/controller" 26 | "sigs.k8s.io/controller-runtime/pkg/reconcile" 27 | ) 28 | 29 | type ManagedReconciler interface { 30 | reconcile.Reconciler 31 | 32 | PreCheck(ctx context.Context, client client.Client) error 33 | DoCleanup() 34 | GetName() string 35 | GetManager() ctrl.Manager 36 | SetManager(mgr ctrl.Manager) 37 | GetRecorder() record.EventRecorder 38 | GetContext() context.Context 39 | SetContext(ctx context.Context) 40 | GetLogger() logr.Logger 41 | SetLogger(l logr.Logger) 42 | GetClient() client.Client 43 | SetClient(client client.Client) 44 | Start(ctx context.Context) error 45 | SetScheme(scheme *runtime.Scheme) 46 | SetupWithController(ctx context.Context, ctrl controller.Controller) error 47 | SetupWithManager(ctx context.Context, mgr ctrl.Manager) error 48 | } 49 | 50 | type ManagedReconcilerBase struct { 51 | name string 52 | log logr.Logger 53 | 54 | ctx context.Context 55 | mgr ctrl.Manager 56 | recorder record.EventRecorder 57 | scheme *runtime.Scheme 58 | client client.Client 59 | } 60 | 61 | func NewManagedReconciler(name string, log logr.Logger) ManagedReconciler { 62 | return &ManagedReconcilerBase{ 63 | name: name, 64 | log: log, 65 | } 66 | } 67 | 68 | func (r *ManagedReconcilerBase) GetClient() client.Client { 69 | return r.client 70 | } 71 | 72 | func (r *ManagedReconcilerBase) SetClient(client client.Client) { 73 | r.client = client 74 | } 75 | 76 | func (r *ManagedReconcilerBase) PreCheck(ctx context.Context, client client.Client) error { 77 | return nil 78 | } 79 | 80 | func (r *ManagedReconcilerBase) Start(ctx context.Context) error { 81 | return nil 82 | } 83 | 84 | func (r *ManagedReconcilerBase) DoCleanup() { 85 | r.ctx = nil 86 | r.mgr = nil 87 | r.recorder = nil 88 | r.scheme = nil 89 | } 90 | 91 | func (r *ManagedReconcilerBase) GetRecorder() record.EventRecorder { 92 | return r.recorder 93 | } 94 | 95 | func (r *ManagedReconcilerBase) GetName() string { 96 | return r.name 97 | } 98 | 99 | func (r *ManagedReconcilerBase) GetLogger() logr.Logger { 100 | return r.log 101 | } 102 | 103 | func (r *ManagedReconcilerBase) SetLogger(l logr.Logger) { 104 | r.log = l 105 | } 106 | 107 | func (r *ManagedReconcilerBase) GetContext() context.Context { 108 | return r.ctx 109 | } 110 | 111 | func (r *ManagedReconcilerBase) SetContext(ctx context.Context) { 112 | if r.ctx != nil { 113 | return 114 | } 115 | 116 | r.ctx = ctx 117 | } 118 | 119 | func (r *ManagedReconcilerBase) GetManager() ctrl.Manager { 120 | return r.mgr 121 | } 122 | 123 | func (r *ManagedReconcilerBase) SetManager(mgr ctrl.Manager) { 124 | if r.mgr != nil { 125 | return 126 | } 127 | 128 | r.mgr = mgr 129 | r.SetScheme(mgr.GetScheme()) 130 | r.recorder = mgr.GetEventRecorderFor(r.name) 131 | } 132 | 133 | func (r *ManagedReconcilerBase) SetScheme(scheme *runtime.Scheme) { 134 | r.scheme = scheme 135 | } 136 | 137 | func (r *ManagedReconcilerBase) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { 138 | return ctrl.Result{}, nil 139 | } 140 | 141 | func (r *ManagedReconcilerBase) SetupWithController(ctx context.Context, ctrl controller.Controller) error { 142 | r.SetContext(ctx) 143 | 144 | return nil 145 | } 146 | 147 | func (r *ManagedReconcilerBase) SetupWithManager(ctx context.Context, mgr ctrl.Manager) error { 148 | r.SetManager(mgr) 149 | 150 | return nil 151 | } 152 | -------------------------------------------------------------------------------- /pkg/clusters/manager.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package clusters 16 | 17 | import ( 18 | "context" 19 | "errors" 20 | "sync" 21 | "time" 22 | ) 23 | 24 | var ( 25 | ErrClusterNotFound = errors.New("cluster not found") 26 | ErrControllerNotFound = errors.New("controller not found") 27 | ) 28 | 29 | type ManagerOption func(m *Manager) 30 | 31 | type Manager struct { 32 | clusters map[string]*Cluster 33 | mu *sync.RWMutex 34 | ctx context.Context 35 | 36 | onBeforeAddFuncs map[string]func(c *Cluster) 37 | onBeforeDeleteFuncs map[string]func(c *Cluster) 38 | onAfterAddFuncs map[string]func(c *Cluster) 39 | onAfterDeleteFuncs map[string]func() 40 | } 41 | 42 | func WithOnBeforeAddFunc(f func(c *Cluster), ids ...string) ManagerOption { 43 | return func(m *Manager) { 44 | m.AddOnBeforeAddFunc(f, ids...) 45 | } 46 | } 47 | 48 | func WithOnAfterAddFunc(f func(c *Cluster), ids ...string) ManagerOption { 49 | return func(m *Manager) { 50 | m.AddOnAfterAddFunc(f, ids...) 51 | } 52 | } 53 | 54 | func WithOnBeforeDeleteFunc(f func(c *Cluster), ids ...string) ManagerOption { 55 | return func(m *Manager) { 56 | m.AddOnBeforeDeleteFunc(f, ids...) 57 | } 58 | } 59 | 60 | func WithOnAfterDeleteFunc(f func(), ids ...string) ManagerOption { 61 | return func(m *Manager) { 62 | m.AddOnAfterDeleteFunc(f, ids...) 63 | } 64 | } 65 | 66 | func NewManager(ctx context.Context, options ...ManagerOption) *Manager { 67 | mgr := &Manager{ 68 | clusters: make(map[string]*Cluster), 69 | mu: &sync.RWMutex{}, 70 | ctx: ctx, 71 | } 72 | 73 | for _, opt := range options { 74 | opt(mgr) 75 | } 76 | 77 | go mgr.waitForStop(ctx) 78 | 79 | return mgr 80 | } 81 | 82 | func (m *Manager) AddOnBeforeAddFunc(f func(c *Cluster), ids ...string) { 83 | m.mu.Lock() 84 | defer m.mu.Unlock() 85 | 86 | if m.onBeforeAddFuncs == nil { 87 | m.onBeforeAddFuncs = make(map[string]func(c *Cluster)) 88 | } 89 | 90 | id := time.Now().String() 91 | if len(ids) > 0 { 92 | id = ids[0] 93 | } 94 | m.onBeforeAddFuncs[id] = f 95 | } 96 | 97 | func (m *Manager) DeleteOnBeforeAddFunc(id string) { 98 | m.mu.Lock() 99 | defer m.mu.Unlock() 100 | 101 | delete(m.onBeforeAddFuncs, id) 102 | } 103 | 104 | func (m *Manager) AddOnAfterAddFunc(f func(c *Cluster), ids ...string) { 105 | m.mu.Lock() 106 | defer m.mu.Unlock() 107 | 108 | if m.onAfterAddFuncs == nil { 109 | m.onAfterAddFuncs = make(map[string]func(c *Cluster)) 110 | } 111 | id := time.Now().String() 112 | if len(ids) > 0 { 113 | id = ids[0] 114 | } 115 | m.onAfterAddFuncs[id] = f 116 | } 117 | 118 | func (m *Manager) DeleteOnAfterAddFunc(id string) { 119 | m.mu.Lock() 120 | defer m.mu.Unlock() 121 | 122 | delete(m.onAfterAddFuncs, id) 123 | } 124 | 125 | func (m *Manager) AddOnBeforeDeleteFunc(f func(c *Cluster), ids ...string) { 126 | m.mu.Lock() 127 | defer m.mu.Unlock() 128 | 129 | if m.onBeforeDeleteFuncs == nil { 130 | m.onBeforeDeleteFuncs = make(map[string]func(c *Cluster)) 131 | } 132 | id := time.Now().String() 133 | if len(ids) > 0 { 134 | id = ids[0] 135 | } 136 | m.onBeforeDeleteFuncs[id] = f 137 | } 138 | 139 | func (m *Manager) DeleteOnBeforeDeleteFunc(id string) { 140 | m.mu.Lock() 141 | defer m.mu.Unlock() 142 | 143 | delete(m.onBeforeDeleteFuncs, id) 144 | } 145 | 146 | func (m *Manager) AddOnAfterDeleteFunc(f func(), ids ...string) { 147 | m.mu.Lock() 148 | defer m.mu.Unlock() 149 | 150 | if m.onAfterDeleteFuncs == nil { 151 | m.onAfterDeleteFuncs = make(map[string]func()) 152 | } 153 | id := time.Now().String() 154 | if len(ids) > 0 { 155 | id = ids[0] 156 | } 157 | m.onAfterDeleteFuncs[id] = f 158 | } 159 | 160 | func (m *Manager) DeleteOnAfterDeleteFunc(id string) { 161 | m.mu.Lock() 162 | defer m.mu.Unlock() 163 | 164 | delete(m.onAfterDeleteFuncs, id) 165 | } 166 | 167 | func (m *Manager) Exists(name string) bool { 168 | m.mu.RLock() 169 | defer m.mu.RUnlock() 170 | 171 | _, ok := m.clusters[name] 172 | 173 | return ok 174 | } 175 | 176 | func (m *Manager) Get(name string) (*Cluster, error) { 177 | m.mu.RLock() 178 | defer m.mu.RUnlock() 179 | 180 | if cluster, ok := m.clusters[name]; ok { 181 | return cluster, nil 182 | } 183 | 184 | return nil, ErrClusterNotFound 185 | } 186 | 187 | func (m *Manager) GetAll() map[string]*Cluster { 188 | m.mu.RLock() 189 | defer m.mu.RUnlock() 190 | 191 | return m.clusters 192 | } 193 | 194 | func (m *Manager) GetAliveClustersByID() map[string]*Cluster { 195 | m.mu.RLock() 196 | defer m.mu.RUnlock() 197 | 198 | clusters := make(map[string]*Cluster) 199 | for _, cluster := range m.clusters { 200 | if cluster.IsAlive() { 201 | clusters[cluster.GetClusterID()] = cluster 202 | } 203 | } 204 | 205 | return clusters 206 | } 207 | 208 | func (m *Manager) Add(cluster *Cluster) error { 209 | m.mu.Lock() 210 | defer m.mu.Unlock() 211 | 212 | for _, f := range m.onBeforeAddFuncs { 213 | f(cluster) 214 | } 215 | 216 | m.clusters[cluster.GetName()] = cluster 217 | 218 | for _, f := range m.onAfterAddFuncs { 219 | f(cluster) 220 | } 221 | 222 | return nil 223 | } 224 | 225 | func (m *Manager) Remove(cluster *Cluster) error { 226 | if cluster == nil { 227 | return nil 228 | } 229 | 230 | if m.clusters[cluster.GetName()] == nil { 231 | return nil 232 | } 233 | 234 | m.mu.Lock() 235 | defer m.mu.Unlock() 236 | 237 | for _, f := range m.onBeforeDeleteFuncs { 238 | f(cluster) 239 | } 240 | 241 | cluster.Stop() 242 | 243 | delete(m.clusters, cluster.GetName()) 244 | 245 | for _, f := range m.onAfterDeleteFuncs { 246 | f() 247 | } 248 | 249 | return nil 250 | } 251 | 252 | func (m *Manager) Stopped() <-chan struct{} { 253 | return m.ctx.Done() 254 | } 255 | 256 | func (m *Manager) waitForStop(ctx context.Context) { 257 | <-ctx.Done() 258 | 259 | for _, c := range m.clusters { 260 | c.Stop() 261 | } 262 | } 263 | -------------------------------------------------------------------------------- /pkg/ratelimit/ratelimit.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package ratelimit 16 | 17 | import ( 18 | "emperror.dev/errors" 19 | "github.com/throttled/throttled" 20 | "github.com/throttled/throttled/store/memstore" 21 | ) 22 | 23 | var defaultRateQuota = throttled.RateQuota{MaxRate: throttled.PerSec(1), MaxBurst: 0} 24 | 25 | func NewRateLimiter(maxKeys int, quota *throttled.RateQuota) (throttled.RateLimiter, error) { 26 | var rateLimiter *throttled.GCRARateLimiter 27 | 28 | store, err := memstore.New(maxKeys) 29 | if err != nil { 30 | return nil, errors.WrapIf(err, "could not create memstore for rate limit") 31 | } 32 | 33 | if quota == nil { 34 | quota = &defaultRateQuota 35 | } 36 | 37 | rateLimiter, err = throttled.NewGCRARateLimiter(store, *quota) 38 | if err != nil { 39 | return nil, errors.WrapIf(err, "could not create rate limiter") 40 | } 41 | 42 | return rateLimiter, nil 43 | } 44 | -------------------------------------------------------------------------------- /pkg/signals/signals.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package signals 16 | 17 | import ( 18 | "context" 19 | "os" 20 | "os/signal" 21 | "syscall" 22 | ) 23 | 24 | func NotifyContext(parentContext context.Context) context.Context { 25 | ctx, cancel := context.WithCancel(parentContext) 26 | c := make(chan os.Signal, 1) 27 | signal.Notify(c, []os.Signal{os.Interrupt, syscall.SIGTERM}...) 28 | 29 | go func() { 30 | select { 31 | case <-c: 32 | cancel() 33 | <-c 34 | os.Exit(1) // second signal. Exit directly. 35 | case <-ctx.Done(): 36 | } 37 | }() 38 | 39 | return ctx 40 | } 41 | -------------------------------------------------------------------------------- /pkg/util/kubeconfig.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package util 16 | 17 | import ( 18 | "net/url" 19 | "strings" 20 | 21 | "emperror.dev/errors" 22 | "k8s.io/client-go/tools/clientcmd" 23 | k8sclientapiv1 "k8s.io/client-go/tools/clientcmd/api/v1" 24 | "sigs.k8s.io/yaml" 25 | 26 | clusterregistryv1alpha1 "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 27 | ) 28 | 29 | func GetKubeconfigWithSAToken(name, username, endpointURL string, caData []byte, saToken string) (string, error) { 30 | address, err := getURLWithHTTPSScheme(endpointURL) 31 | if err != nil { 32 | return "", errors.WithStack(err) 33 | } 34 | 35 | config := k8sclientapiv1.Config{ 36 | APIVersion: k8sclientapiv1.SchemeGroupVersion.Version, 37 | Kind: "Config", 38 | Clusters: []k8sclientapiv1.NamedCluster{ 39 | { 40 | Name: name, 41 | Cluster: k8sclientapiv1.Cluster{ 42 | CertificateAuthorityData: caData, 43 | Server: address, 44 | }, 45 | }, 46 | }, 47 | Contexts: []k8sclientapiv1.NamedContext{ 48 | { 49 | Name: name, 50 | Context: k8sclientapiv1.Context{ 51 | Cluster: name, 52 | AuthInfo: username, 53 | }, 54 | }, 55 | }, 56 | CurrentContext: name, 57 | AuthInfos: []k8sclientapiv1.NamedAuthInfo{ 58 | { 59 | Name: username, 60 | AuthInfo: k8sclientapiv1.AuthInfo{ 61 | Token: saToken, 62 | }, 63 | }, 64 | }, 65 | } 66 | 67 | y, err := yaml.Marshal(config) 68 | if err != nil { 69 | return "", err 70 | } 71 | 72 | return string(y), nil 73 | } 74 | 75 | func GetEndpointForClusterByNetwork(cluster *clusterregistryv1alpha1.Cluster, networkName string) clusterregistryv1alpha1.KubernetesAPIEndpoint { 76 | var endpoint clusterregistryv1alpha1.KubernetesAPIEndpoint 77 | 78 | for _, apiEndpoint := range cluster.Spec.KubernetesAPIEndpoints { 79 | if apiEndpoint.ClientNetwork == networkName { 80 | endpoint = apiEndpoint 81 | 82 | break 83 | } 84 | // use for every network if the endpoint is not network specific 85 | if apiEndpoint.ClientNetwork == "" { 86 | endpoint = apiEndpoint 87 | } 88 | } 89 | 90 | return endpoint 91 | } 92 | 93 | func GetKubeconfigOverridesForClusterByNetwork(cluster *clusterregistryv1alpha1.Cluster, networkName string) (*clientcmd.ConfigOverrides, error) { 94 | overrides := &clientcmd.ConfigOverrides{} 95 | 96 | if len(cluster.Spec.KubernetesAPIEndpoints) == 0 { 97 | return overrides, nil 98 | } 99 | 100 | endpoint := GetEndpointForClusterByNetwork(cluster, networkName) 101 | 102 | if endpoint.ServerAddress != "" { 103 | address, err := getURLWithHTTPSScheme(endpoint.ServerAddress) 104 | if err != nil { 105 | return overrides, errors.WithStack(err) 106 | } 107 | overrides.ClusterInfo.Server = address 108 | } 109 | 110 | if len(endpoint.CABundle) > 0 { 111 | overrides.ClusterInfo.CertificateAuthorityData = endpoint.CABundle 112 | } 113 | 114 | return overrides, nil 115 | } 116 | 117 | func getURLWithHTTPSScheme(address string) (string, error) { 118 | if !strings.Contains(address, "//") { 119 | address = "//" + address 120 | } 121 | u, err := url.Parse(address) 122 | if err != nil { 123 | return "", err 124 | } 125 | if u.Scheme == "" { 126 | u.Scheme = "https" 127 | } 128 | 129 | return u.String(), nil 130 | } 131 | -------------------------------------------------------------------------------- /pkg/util/overlaypatchtemplate.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package util 16 | 17 | import ( 18 | "bytes" 19 | "text/template" 20 | 21 | "github.com/Masterminds/sprig" 22 | 23 | "github.com/banzaicloud/operator-tools/pkg/resources" 24 | "github.com/banzaicloud/operator-tools/pkg/utils" 25 | ) 26 | 27 | func K8SResourceOverlayPatchExecuteTemplate(patch resources.K8SResourceOverlayPatch, data interface{}) (resources.K8SResourceOverlayPatch, error) { 28 | p := patch.DeepCopy() 29 | 30 | value := utils.PointerToString(p.Value) 31 | t, err := template.New("").Funcs(sprig.TxtFuncMap()).Parse(value) 32 | if err != nil { 33 | return resources.K8SResourceOverlayPatch{}, err 34 | } 35 | 36 | var tpl bytes.Buffer 37 | err = t.Execute(&tpl, data) 38 | if err != nil { 39 | return resources.K8SResourceOverlayPatch{}, err 40 | } 41 | 42 | p.Value = utils.StringPointer(tpl.String()) 43 | 44 | return *p, nil 45 | } 46 | 47 | func K8SResourceOverlayPatchExecuteTemplates(patches []resources.K8SResourceOverlayPatch, data interface{}) ([]resources.K8SResourceOverlayPatch, error) { 48 | result := make([]resources.K8SResourceOverlayPatch, 0) 49 | for _, p := range patches { 50 | p, err := K8SResourceOverlayPatchExecuteTemplate(p, data) 51 | if err != nil { 52 | return nil, err 53 | } 54 | result = append(result, p) 55 | } 56 | 57 | return result, nil 58 | } 59 | -------------------------------------------------------------------------------- /pkg/util/provision_local_cluster.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package util 16 | 17 | import ( 18 | "context" 19 | 20 | "emperror.dev/errors" 21 | "github.com/go-logr/logr" 22 | admissionregistrationv1 "k8s.io/api/admissionregistration/v1" 23 | corev1 "k8s.io/api/core/v1" 24 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 25 | "k8s.io/apimachinery/pkg/types" 26 | "sigs.k8s.io/controller-runtime/pkg/client" 27 | 28 | "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 29 | "github.com/cisco-open/cluster-registry-controller/internal/config" 30 | ) 31 | 32 | func ProvisionLocalClusterObject(c client.Client, log logr.Logger, configuration config.Configuration) error { 33 | clusterSpec := v1alpha1.Cluster{} 34 | 35 | err := c.Get(context.Background(), types.NamespacedName{ 36 | Name: configuration.ProvisionLocalCluster, 37 | Namespace: configuration.Namespace, 38 | }, &clusterSpec) 39 | 40 | if err == nil { 41 | log.Info("local cluster object already exists, skipping provisioning", "cluster_name", configuration.ProvisionLocalCluster) 42 | 43 | return nil 44 | } 45 | 46 | if configuration.ClusterValidatorWebhook.Enabled { 47 | webhook := &admissionregistrationv1.ValidatingWebhookConfiguration{} 48 | err := c.Get(context.Background(), types.NamespacedName{ 49 | Name: configuration.ClusterValidatorWebhook.Name, 50 | }, webhook) 51 | if err != nil { 52 | return errors.WrapIf(err, "could not get validating webhook configuration") 53 | } 54 | ignore := admissionregistrationv1.Ignore 55 | for i := range webhook.Webhooks { 56 | webhook.Webhooks[i].FailurePolicy = &ignore 57 | } 58 | err = c.Update(context.Background(), webhook) 59 | if err != nil { 60 | return errors.WrapIf(err, "could not update validating webhook configuration") 61 | } 62 | } 63 | 64 | newClusterSpec, err := NewLocalCluster(c, configuration.Namespace, configuration.ProvisionLocalCluster, configuration.APIServerEndpointAddress) 65 | if err != nil { 66 | return errors.WrapIf(err, "cannot create new local cluster object") 67 | } 68 | 69 | err = c.Create(context.Background(), newClusterSpec) 70 | if err != nil { 71 | return errors.WrapIf(err, "cannot create new local cluster object") 72 | } 73 | 74 | log.Info("provisioned local cluster configuration", "cluster_name", configuration.ProvisionLocalCluster) 75 | 76 | return nil 77 | } 78 | 79 | func NewLocalCluster(c client.Client, namespace, name, apiServerEndpointAddress string) (*v1alpha1.Cluster, error) { 80 | ns := &corev1.Namespace{} 81 | err := c.Get(context.Background(), types.NamespacedName{ 82 | Name: metav1.NamespaceSystem, 83 | }, ns) 84 | if err != nil { 85 | return nil, err 86 | } 87 | 88 | return &v1alpha1.Cluster{ 89 | TypeMeta: metav1.TypeMeta{ 90 | Kind: "Cluster", 91 | APIVersion: v1alpha1.GroupVersion.String(), 92 | }, 93 | ObjectMeta: metav1.ObjectMeta{ 94 | Name: name, 95 | }, 96 | Spec: v1alpha1.ClusterSpec{ 97 | ClusterID: ns.UID, 98 | AuthInfo: v1alpha1.AuthInfo{ 99 | SecretRef: v1alpha1.NamespacedName{ 100 | Name: name, 101 | Namespace: namespace, 102 | }, 103 | }, 104 | KubernetesAPIEndpoints: []v1alpha1.KubernetesAPIEndpoint{ 105 | { 106 | ServerAddress: apiServerEndpointAddress, 107 | }, 108 | }, 109 | }, 110 | }, nil 111 | } 112 | -------------------------------------------------------------------------------- /pkg/util/resources.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package util 16 | 17 | import ( 18 | "fmt" 19 | "strings" 20 | 21 | "k8s.io/apimachinery/pkg/runtime/schema" 22 | ) 23 | 24 | func GVKToString(gvk schema.GroupVersionKind) string { 25 | return fmt.Sprintf("%s.%s/%s", gvk.Kind, gvk.Group, gvk.Version) 26 | } 27 | 28 | func ParseGVKFromString(str string) *schema.GroupVersionKind { 29 | fp := strings.SplitN(str, ".", 2) 30 | if len(fp) != 2 { 31 | return nil 32 | } 33 | 34 | sp := strings.SplitN(fp[1], "/", 2) 35 | if len(sp) != 2 { 36 | return nil 37 | } 38 | 39 | return &schema.GroupVersionKind{ 40 | Group: sp[0], 41 | Kind: fp[0], 42 | Version: sp[1], 43 | } 44 | } 45 | -------------------------------------------------------------------------------- /pkg/util/util_test.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021, and 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package util_test 16 | 17 | import ( 18 | "fmt" 19 | "testing" 20 | 21 | corev1 "k8s.io/api/core/v1" 22 | v1 "k8s.io/apimachinery/pkg/apis/meta/v1" 23 | "k8s.io/apimachinery/pkg/runtime" 24 | 25 | "github.com/banzaicloud/operator-tools/pkg/resources" 26 | "github.com/banzaicloud/operator-tools/pkg/utils" 27 | clusterregistryv1alpha1 "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 28 | "github.com/cisco-open/cluster-registry-controller/pkg/util" 29 | ) 30 | 31 | func TestK8SResourceOverlayPatchExecuteTemplate(t *testing.T) { 32 | t.Parallel() 33 | 34 | tests := map[string]struct { 35 | patch resources.K8SResourceOverlayPatch 36 | object runtime.Object 37 | cluster *clusterregistryv1alpha1.Cluster 38 | wanted string 39 | }{ 40 | "working template": { 41 | patch: resources.K8SResourceOverlayPatch{ 42 | Path: utils.StringPointer("/name"), 43 | Type: resources.ReplaceOverlayPatchType, 44 | Value: utils.StringPointer(`{{ printf "%s-%s" .Object.GetName .Cluster.GetName }}`), 45 | }, 46 | object: &corev1.Service{ 47 | ObjectMeta: v1.ObjectMeta{ 48 | Name: "test-service", 49 | }, 50 | }, 51 | cluster: &clusterregistryv1alpha1.Cluster{ 52 | ObjectMeta: v1.ObjectMeta{ 53 | Name: "demo-cluster-1", 54 | }, 55 | }, 56 | wanted: "test-service-demo-cluster-1", 57 | }, 58 | } 59 | 60 | for _, test := range tests { 61 | result, err := util.K8SResourceOverlayPatchExecuteTemplate(test.patch, map[string]interface{}{ 62 | "Object": test.object, 63 | "Cluster": test.cluster, 64 | }) 65 | if err != nil { 66 | t.Fatal(err) 67 | } 68 | if utils.PointerToString(result.Value) != test.wanted { 69 | t.Fatal(fmt.Errorf("%s != %s", utils.PointerToString(result.Value), test.wanted)) // nolint:goerr113 70 | } 71 | } 72 | } 73 | -------------------------------------------------------------------------------- /pkg/webhooks/cluster_validator.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2022 Cisco and/or its affiliates. All rights reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | package webhooks 16 | 17 | import ( 18 | "context" 19 | "net/http" 20 | 21 | "emperror.dev/errors" 22 | ctrl "sigs.k8s.io/controller-runtime" 23 | "sigs.k8s.io/controller-runtime/pkg/webhook/admission" 24 | 25 | "github.com/go-logr/logr" 26 | 27 | clusterregistrycontrollerapiv1alpha1 "github.com/cisco-open/cluster-registry-controller/api/v1alpha1" 28 | ) 29 | 30 | // ClusterValidator validates cluster CRs of the cluster registry. 31 | type ClusterValidator struct { 32 | // logger is the log interface to use inside the validator. 33 | logger logr.Logger 34 | 35 | // manager is responsible for handling the communication between the 36 | // validator and the Kubernetes API server. 37 | manager ctrl.Manager 38 | 39 | // decoder is responsible for decoding the webhook request into structured 40 | // data. 41 | decoder *admission.Decoder 42 | } 43 | 44 | // NewClusterValidator instantiates a cluster CR validator set to work on the 45 | // specified Kubernetes API server. 46 | func NewClusterValidator(logger logr.Logger, manager ctrl.Manager) *ClusterValidator { 47 | return &ClusterValidator{ 48 | logger: logger, 49 | manager: manager, 50 | decoder: nil, 51 | } 52 | } 53 | 54 | // Handle handles the validator's admission requests and determines whether the 55 | // specified request can be allowed. 56 | func (validator *ClusterValidator) Handle(ctx context.Context, request admission.Request) admission.Response { 57 | validator.logger.Info("validating cluster CR", "request", request) 58 | 59 | newClusterCR := &clusterregistrycontrollerapiv1alpha1.Cluster{} 60 | 61 | err := validator.decoder.Decode(request, newClusterCR) 62 | if err != nil { 63 | err = errors.Wrap(err, "decoding admission request as new cluster CR failed") 64 | 65 | validator.logger.Error(err, "validating cluster CR failed", "request", request) 66 | 67 | return admission.Errored(http.StatusBadRequest, err) 68 | } 69 | 70 | k8sClient := validator.manager.GetClient() 71 | 72 | oldClusterCRs := &clusterregistrycontrollerapiv1alpha1.ClusterList{} 73 | err = k8sClient.List(ctx, oldClusterCRs) 74 | if err != nil { 75 | err = errors.Wrap(err, "listing existing cluster CRs failed") 76 | 77 | validator.logger.Error(err, "validating cluster CR failed") 78 | 79 | return admission.Errored(http.StatusConflict, err) 80 | } 81 | 82 | err = validator.validateUniqueLocalCluster(ctx, newClusterCR, oldClusterCRs) 83 | if err != nil { 84 | err = errors.Wrap(err, "validating unique local cluster CR failed") 85 | 86 | validator.logger.Error(err, "validating cluster CR failed", "newClusterCR", newClusterCR) 87 | 88 | return admission.Errored(http.StatusConflict, err) 89 | } 90 | 91 | validator.logger.Info("validating cluster CR succeeded", "request", request) 92 | 93 | return admission.Allowed("") 94 | } 95 | 96 | // validateUniqueLocalCluster returns an error if the specified new cluster CR 97 | // has a matching local clusterID among the old cluster CRs, nil otherwise. 98 | func (validator *ClusterValidator) validateUniqueLocalCluster( 99 | ctx context.Context, 100 | newClusterCR *clusterregistrycontrollerapiv1alpha1.Cluster, 101 | oldClusterCRs *clusterregistrycontrollerapiv1alpha1.ClusterList, 102 | ) error { 103 | // Note: the new cluster CR usually doesn't have a status.type set yet, so 104 | // we only return on explicit peer cluster status.type, we examine new 105 | // cluster CRs with local or nil status.type further. 106 | if newClusterCR.Status.Type == clusterregistrycontrollerapiv1alpha1.ClusterTypePeer { 107 | validator.logger.Info("skipping peer cluster unique local validation", "newClusterCR", newClusterCR) 108 | 109 | return nil // Note: for peer cluster CRs this validation is a no-operation. 110 | } 111 | 112 | for _, oldClusterCR := range oldClusterCRs.Items { 113 | if oldClusterCR.Status.Type == clusterregistrycontrollerapiv1alpha1.ClusterTypeLocal && 114 | oldClusterCR.Spec.ClusterID == newClusterCR.Spec.ClusterID && 115 | oldClusterCR.Name != newClusterCR.Name { // Note: in case a cluster CR is updated in place it should not be rejected because of the older version of itself. 116 | err := errors.Errorf( 117 | "a local cluster CR with name %s already exists with the same clusterID %s", 118 | oldClusterCR.ObjectMeta.Name, 119 | newClusterCR.Spec.ClusterID, 120 | ) 121 | 122 | validator.logger.Error( 123 | err, 124 | "validating unique local cluster CR failed", 125 | "newClusterCR", newClusterCR, 126 | "oldClusterCR", oldClusterCR, 127 | ) 128 | 129 | return err 130 | } 131 | } 132 | 133 | validator.logger.Info( 134 | "validating new cluster CR unique local cluster ID succeeded", 135 | "newClusterCRClusterID", newClusterCR.Spec.ClusterID, 136 | ) 137 | 138 | return nil 139 | } 140 | 141 | // InjectDecoder sets the cluster CR decoder object. 142 | func (validator *ClusterValidator) InjectDecoder(decoder *admission.Decoder) error { 143 | validator.decoder = decoder 144 | 145 | return nil 146 | } 147 | -------------------------------------------------------------------------------- /scripts/download-deps.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -euo pipefail 4 | 5 | controller_gen_version=0.6.2 6 | kuttl_version=0.8.0 7 | 8 | script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd)" 9 | binpath=${script_dir}/../bin 10 | 11 | mkdir -p "$binpath" 12 | 13 | function ensure-binary-version() { 14 | local bin_name=$1 15 | local bin_version=$2 16 | local download_location=$3 17 | 18 | local target_name=${bin_name}-proper-${bin_version} 19 | local link_path=${binpath}/${bin_name} 20 | 21 | if [ ! -L "${link_path}" ]; then 22 | rm -f "${link_path}" 23 | fi 24 | 25 | if [ ! -e "${binpath}/${target_name}" ]; then 26 | BUILD_DIR=$(mktemp -d) 27 | pushd "${BUILD_DIR}" 28 | GOBIN=${PWD} go install "${download_location}" 29 | mv "${bin_name}" "${binpath}/${target_name}" 30 | popd 31 | rm -rf "${BUILD_DIR}" 32 | fi 33 | 34 | ln -sf "${target_name}" "${link_path}" 35 | } 36 | 37 | ensure-binary-version controller-gen ${controller_gen_version} "sigs.k8s.io/controller-tools/cmd/controller-gen@v${controller_gen_version}" 38 | 39 | -------------------------------------------------------------------------------- /scripts/install_kubebuilder_tools.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright (c) 2022 Cisco and/or its affiliates. All rights reserved. 4 | 5 | set -euo pipefail 6 | 7 | os=$(go env GOOS) 8 | arch=$(go env GOARCH) 9 | 10 | # These are the current official versions for kubebuilder-tools (v1.19.2 or v1.22.x is recommended) 11 | # Reference: https://book.kubebuilder.io/reference/envtest.html#kubernetes-120-and-121-binary-issues 12 | verified_kubebuilder_tools_version=(1.19.2) 13 | 14 | 15 | FLOCK=`which flock 2> /dev/null || true` 16 | if [ -z "${FLOCK}" ] || [ ! -x "${FLOCK}" ]; then 17 | # On MacOS or any system that does not have the flock binary 18 | # let's not use any locking 19 | FLOCK="true" 20 | fi 21 | 22 | ( 23 | $FLOCK -x 200 24 | 25 | for i in "$@"; do 26 | case $i in 27 | --kubebuilder-tools-version=?*) 28 | kubebuilder_tools_version=${i#*=} 29 | ;; 30 | *) 31 | esac 32 | done 33 | 34 | if [ -z "${kubebuilder_tools_version:-}" ]; then 35 | echo "--kubebuilder-tools-version flag missing, possible options: ${verified_kubebuilder_tools_version[*]}" 36 | exit 1 37 | fi 38 | 39 | target_dir=bin/kubebuilder-tools/${kubebuilder_tools_version} 40 | 41 | mkdir -p "${target_dir}" 42 | 43 | if [[ ! -e "${target_dir}/complete" ]]; then 44 | echo "Installing kubebuilder-tools version v${kubebuilder_tools_version}..." 45 | echo "Downloading https://go.kubebuilder.io/test-tools/${kubebuilder_tools_version}/${os}/${arch}" 46 | # Validate kubebuilder-tools version 47 | if echo ${verified_kubebuilder_tools_version[@]} | grep -q -w ${kubebuilder_tools_version}; then 48 | curl -sSLo ${target_dir}/envtest-bins.tar.gz "https://go.kubebuilder.io/test-tools/${kubebuilder_tools_version}/${os}/${arch}" 49 | tar --strip-components=1 -zvxf ${target_dir}/envtest-bins.tar.gz -C "${target_dir}" 50 | rm ${target_dir}/envtest-bins.tar.gz 51 | touch "${target_dir}/complete" 52 | 53 | echo " The current kubebuilder-tools version is v${kubebuilder_tools_version}." 54 | else 55 | echo "The specified kubebuilder-tools version is not valid. Overriding failed." 56 | fi 57 | else 58 | echo "kubebuilder-tools already installed under ${target_dir}" 59 | fi 60 | 61 | ) 200> /tmp/install-kubebuilder-tools.lock --------------------------------------------------------------------------------