├── driver ├── .dockerignore ├── Dockerfile ├── examples │ ├── version1 │ │ ├── snapshot │ │ │ ├── volsnapclass.yaml │ │ │ ├── volsnap.yaml │ │ │ ├── storageClassWithNodeClass.yaml │ │ │ ├── pvcfrmsnap_rom.yaml │ │ │ └── pvcfrmsnap.yaml │ │ └── volume │ │ │ ├── fileset │ │ │ ├── storageclassfileset_independent.yaml │ │ │ ├── pvcfileset_dependent.yaml │ │ │ ├── pvcfileset_independent.yaml │ │ │ ├── storageclassfileset_tier_independent.yaml │ │ │ ├── storageclassfileset_compress_independent.yaml │ │ │ ├── storageclassfileset_expansion_independent.yaml │ │ │ ├── storageclassfileset_dependent.yaml │ │ │ ├── storageclassfileset_tier_dependent.yaml │ │ │ ├── clonefrompvc_independent.yaml │ │ │ ├── storageclassfileset_compress_dependent.yaml │ │ │ ├── clonefrompvc_dependent.yaml │ │ │ └── podfileset.yaml │ │ │ └── lightweight │ │ │ ├── pvclw.yaml │ │ │ ├── storageclasslw.yaml │ │ │ └── podlw.yaml │ ├── staticVolume │ │ ├── dynamicprovisioning │ │ │ ├── snapshot │ │ │ │ ├── volsnapclass_static.yaml │ │ │ │ ├── volsnap_static.yaml │ │ │ │ └── pvcfrmsnap_static.yaml │ │ │ └── volume │ │ │ │ ├── vac_static.yaml │ │ │ │ ├── storageclass_static_independent.yaml │ │ │ │ ├── pvc_static_independent.yaml │ │ │ │ ├── pvc_static_dependent.yaml │ │ │ │ ├── storageclass_static_expansion_independent.yaml │ │ │ │ ├── storageclass_static_dependent.yaml │ │ │ │ ├── clonefrompvc_static_independent.yaml │ │ │ │ ├── clonefrompvc_static_dependent.yaml │ │ │ │ └── pod_static.yaml │ │ └── staticprovisioning │ │ │ ├── volume │ │ │ ├── static_pvc.yaml │ │ │ ├── static_pv.yaml │ │ │ └── static_pod.yaml │ │ │ ├── snapshot │ │ │ ├── static_volsnap.yaml │ │ │ └── static_volsnapcontent.yaml │ │ │ └── pvcfrmsnap.yaml │ ├── version2 │ │ ├── volume │ │ │ ├── pvc.yaml │ │ │ ├── storageclass.yaml │ │ │ ├── storageclass_tier.yaml │ │ │ ├── storageclass_compress.yaml │ │ │ ├── storageclass_expansion.yaml │ │ │ └── pod.yaml │ │ └── snapshot │ │ │ ├── volsnap.yaml │ │ │ ├── volsnapclass.yaml │ │ │ └── pvcfrmsnap.yaml │ └── cacheVolume │ │ ├── storageclass.yaml │ │ ├── pod.yaml │ │ ├── pvcnfs.yaml │ │ └── pvcs3.yaml ├── chroot-wrapper.sh ├── go.mod ├── build │ └── Dockerfile ├── csiplugin │ ├── identityserver.go │ ├── server.go │ ├── utils.go │ ├── utils │ │ ├── http_utils.go │ │ └── utils.go │ ├── pkg │ │ └── consistencygroup │ │ │ └── consistencygroup.go │ ├── parallelSnapshot.go │ └── settings │ │ └── scale_config.go └── cmd │ └── ibm-spectrum-scale-csi │ └── main.go ├── operator ├── Dockerfile ├── config │ ├── scc │ │ ├── kustomization.yaml │ │ └── security_context_constraint.yaml │ ├── manifests │ │ └── kustomization.yaml │ ├── rbac │ │ ├── kustomization.yaml │ │ ├── service_account.yaml │ │ ├── role_binding.yaml │ │ └── role.yaml │ ├── manager │ │ ├── kustomization.yaml │ │ └── manager.yaml │ ├── scorecard │ │ ├── bases │ │ │ └── config.yaml │ │ ├── patches │ │ │ ├── basic.config.yaml │ │ │ └── olm.config.yaml │ │ └── kustomization.yaml │ ├── samples │ │ ├── kustomization.yaml │ │ ├── csi_v1_csiscaleoperator.yaml │ │ └── csiscaleoperators.csi.ibm.com_cr.yaml │ ├── overlays │ │ ├── cnsa │ │ │ └── kustomization.yaml │ │ ├── cnsa-k8s │ │ │ └── kustomization.yaml │ │ └── default │ │ │ ├── images_manifest.yaml │ │ │ └── kustomization.yaml │ ├── testing │ │ ├── manager_image.yaml │ │ ├── pull_policy │ │ │ ├── Never.yaml │ │ │ ├── Always.yaml │ │ │ └── IfNotPresent.yaml │ │ ├── debug_logs_patch.yaml │ │ └── kustomization.yaml │ └── crd │ │ └── kustomization.yaml ├── hacks │ ├── health_check.go │ ├── boilerplate.go.txt │ ├── csv_prep.py │ ├── csv_copy_docs.py │ ├── csv_copy_cr.py │ ├── package_operator.py │ └── csv_copy_crd_descriptions.py ├── README.md ├── PROJECT ├── controllers │ ├── util │ │ ├── boolptr │ │ │ └── boolptr.go │ │ └── k8sutil │ │ │ └── k8sutil.go │ ├── suite_test.go │ ├── config │ │ └── resources.go │ └── internal │ │ └── csiscaleoperator │ │ └── csiscaleoperator.go ├── bundle.Dockerfile ├── api │ └── v1 │ │ └── groupversion_info.go ├── .gitignore ├── build │ └── Dockerfile ├── docs │ └── dev │ │ └── security │ │ ├── results.gosec.json │ │ └── results.golint.txt ├── go.mod └── main.go ├── docs └── .gitignore ├── kustomization.yaml ├── tools ├── scripts │ ├── ci │ │ ├── lint_operator-courier.sh │ │ ├── travis_before_script.sh │ │ ├── install_operator-sdk.sh │ │ ├── fold_ouput.sh │ │ ├── ci-env │ │ └── install_minikube.sh │ └── push_app.sh ├── ansible │ ├── common │ │ ├── requirements-run.txt │ │ ├── requirements-dev.txt │ │ ├── runtime-env.yaml │ │ └── dev-env.yaml │ ├── deploy │ │ ├── hacks │ │ │ ├── templates │ │ │ │ ├── mmlsnodeclass │ │ │ │ ├── mmlsfs │ │ │ │ └── mmlscluster │ │ │ └── deploycsioperator.py │ │ ├── rest.yaml │ │ ├── preflight.yaml │ │ └── nodeprep.yaml │ ├── deploy-playbook.yaml │ ├── dev-env-playbook.yaml │ ├── generate-playbook.yaml │ ├── versioning-playbook.yaml │ └── generate │ │ └── create_files.yaml ├── volume_migration_scripts │ ├── README.md │ └── README_migration_csi_primary_removal.md └── generate_volsnapcontent_yaml.sh ├── OWNERS ├── .github ├── ISSUE_TEMPLATE │ ├── feature_request.md │ ├── release-checklist.md │ └── bug_report.md └── PULL_REQUEST_TEMPLATE.md ├── .gitignore ├── .travis-pri.yml ├── Makefile ├── README.md ├── .golangci.yml ├── CONTRIBUTING.md ├── CODE_OF_CONDUCT.md └── README.driver.md /driver/.dockerignore: -------------------------------------------------------------------------------- 1 | examples -------------------------------------------------------------------------------- /driver/Dockerfile: -------------------------------------------------------------------------------- 1 | build/Dockerfile -------------------------------------------------------------------------------- /operator/Dockerfile: -------------------------------------------------------------------------------- 1 | build/Dockerfile -------------------------------------------------------------------------------- /docs/.gitignore: -------------------------------------------------------------------------------- 1 | _build 2 | _static 3 | _templates 4 | -------------------------------------------------------------------------------- /kustomization.yaml: -------------------------------------------------------------------------------- 1 | resources: 2 | - operator/config/overlays/default 3 | -------------------------------------------------------------------------------- /operator/config/scc/kustomization.yaml: -------------------------------------------------------------------------------- 1 | resources: 2 | - ./security_context_constraint.yaml 3 | -------------------------------------------------------------------------------- /tools/scripts/ci/lint_operator-courier.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -e 3 | set -u 4 | 5 | #. ci-env 6 | 7 | -------------------------------------------------------------------------------- /tools/ansible/common/requirements-run.txt: -------------------------------------------------------------------------------- 1 | sphinx 2 | sphinx_rtd_theme 3 | recommonmark 4 | operator-courier==2.0.1 5 | -------------------------------------------------------------------------------- /operator/config/manifests/kustomization.yaml: -------------------------------------------------------------------------------- 1 | resources: 2 | - ../overlays/default 3 | - ../samples 4 | - ../scorecard 5 | -------------------------------------------------------------------------------- /operator/config/rbac/kustomization.yaml: -------------------------------------------------------------------------------- 1 | resources: 2 | - service_account.yaml 3 | - role.yaml 4 | - role_binding.yaml 5 | -------------------------------------------------------------------------------- /operator/hacks/health_check.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | fmt.Println("Health Check Passed!") 7 | } 8 | -------------------------------------------------------------------------------- /tools/ansible/deploy/hacks/templates/mmlsnodeclass: -------------------------------------------------------------------------------- 1 | Value GUI_MGMT_SERVER ([^\s]+) 2 | 3 | Start 4 | ^GUI_MGMT_SERVERS\s+${GUI_MGMT_SERVER} 5 | 6 | -------------------------------------------------------------------------------- /tools/ansible/common/requirements-dev.txt: -------------------------------------------------------------------------------- 1 | sphinx 2 | sphinx_rtd_theme 3 | recommonmark 4 | operator-courier==2.0.1 5 | pyyaml 6 | molecule 7 | jmespath 8 | -------------------------------------------------------------------------------- /tools/scripts/ci/travis_before_script.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -e 3 | set -u 4 | 5 | #. ./ci-env 6 | 7 | ./install_minikube.sh 8 | ./install_operator-sdk.sh 9 | -------------------------------------------------------------------------------- /tools/ansible/deploy/hacks/templates/mmlsfs: -------------------------------------------------------------------------------- 1 | Value fs ([^\s:]+) 2 | Value mount ([^\s]+) 3 | 4 | Start 5 | ^File system attributes for ${fs}: 6 | ^\s+-T\s+${mount}\s.* -> Next.Record 7 | 8 | -------------------------------------------------------------------------------- /operator/config/manager/kustomization.yaml: -------------------------------------------------------------------------------- 1 | resources: 2 | - manager.yaml 3 | apiVersion: kustomize.config.k8s.io/v1beta1 4 | kind: Kustomization 5 | commonAnnotations: 6 | productVersion: 3.1.0 7 | -------------------------------------------------------------------------------- /tools/scripts/ci/install_operator-sdk.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -e 3 | set -u 4 | 5 | sudo curl -L ${OPERATOR_SDK} -o /usr/local/bin/operator-sdk 6 | sudo chmod +x /usr/local/bin/operator-sdk 7 | -------------------------------------------------------------------------------- /operator/config/scorecard/bases/config.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: scorecard.operatorframework.io/v1alpha3 2 | kind: Configuration 3 | metadata: 4 | name: config 5 | stages: 6 | - parallel: true 7 | tests: [] 8 | -------------------------------------------------------------------------------- /operator/config/samples/kustomization.yaml: -------------------------------------------------------------------------------- 1 | ## Append samples you want in your CSV to this file as resources ## 2 | resources: 3 | - csi_v1_csiscaleoperator.yaml 4 | # +kubebuilder:scaffold:manifestskustomizesamples 5 | -------------------------------------------------------------------------------- /tools/scripts/ci/fold_ouput.sh: -------------------------------------------------------------------------------- 1 | ##!/usr/bin/env bash 2 | 3 | name=$1 4 | desc=$2 5 | shift 6 | shift 7 | 8 | echo -e "travis_fold:start:$name\033[33;1m$desc\033[0m" 9 | eval $@ 10 | echo -e "\ntravis_fold:end:$name\r" 11 | -------------------------------------------------------------------------------- /driver/examples/version1/snapshot/volsnapclass.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: snapshot.storage.k8s.io/v1 2 | kind: VolumeSnapshotClass 3 | metadata: 4 | name: ibm-spectrum-scale-snapshotclass 5 | driver: spectrumscale.csi.ibm.com 6 | deletionPolicy: Delete 7 | -------------------------------------------------------------------------------- /tools/ansible/deploy/rest.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: Make the CsiAdmin user 4 | shell: "/usr/lpp/mmfs/gui/cli/mkuser {{user}} -p {{pass}} -g CsiAdmin" 5 | var: 6 | user: adm 7 | pass: ppslab 8 | 9 | 10 | 11 | 12 | 13 | 14 | -------------------------------------------------------------------------------- /operator/config/overlays/cnsa/kustomization.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: kustomize.config.k8s.io/v1beta1 3 | kind: Kustomization 4 | 5 | # Adds namespace to all resources. 6 | namespace: ibm-spectrum-scale-csi 7 | 8 | resources: 9 | - ../../crd/ 10 | -------------------------------------------------------------------------------- /operator/config/overlays/cnsa-k8s/kustomization.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: kustomize.config.k8s.io/v1beta1 3 | kind: Kustomization 4 | 5 | # Adds namespace to all resources. 6 | namespace: ibm-spectrum-scale-csi 7 | 8 | resources: 9 | - ../../crd/ 10 | 11 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/dynamicprovisioning/snapshot/volsnapclass_static.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: snapshot.storage.k8s.io/v1 2 | kind: VolumeSnapshotClass 3 | metadata: 4 | name: ibm-spectrum-scale-snapshotclass 5 | driver: spectrumscale.csi.ibm.com 6 | deletionPolicy: Delete 7 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/dynamicprovisioning/volume/vac_static.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1beta1 2 | kind: VolumeAttributesClass 3 | metadata: 4 | name: scale-static-pvc-vac 5 | driverName: spectrumscale.csi.ibm.com 6 | parameters: 7 | filesetName: "scale-static-pvc" -------------------------------------------------------------------------------- /operator/README.md: -------------------------------------------------------------------------------- 1 | # ibm-spectrum-scale-csi-operator 2 | The [IBM Storage Scale Container Storage Interface](https://github.com/IBM/ibm-spectrum-scale-csi) (CSI) project enables container orchestrators, such as Kubernetes and OpenShift, to manage the life-cycle of persistent storage. 3 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/staticprovisioning/volume/static_pvc.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: scale-static-pvc 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | 12 | -------------------------------------------------------------------------------- /operator/config/testing/manager_image.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: apps/v1 3 | kind: Deployment 4 | metadata: 5 | name: controller-manager 6 | namespace: system 7 | spec: 8 | template: 9 | spec: 10 | containers: 11 | - name: manager 12 | image: testing 13 | -------------------------------------------------------------------------------- /operator/config/testing/pull_policy/Never.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: apps/v1 3 | kind: Deployment 4 | metadata: 5 | name: controller-manager 6 | namespace: system 7 | spec: 8 | template: 9 | spec: 10 | containers: 11 | - name: manager 12 | imagePullPolicy: Never 13 | -------------------------------------------------------------------------------- /driver/examples/version2/volume/pvc.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: scale-advance-pvc 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-csi-advance 12 | -------------------------------------------------------------------------------- /operator/config/testing/pull_policy/Always.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: apps/v1 3 | kind: Deployment 4 | metadata: 5 | name: controller-manager 6 | namespace: system 7 | spec: 8 | template: 9 | spec: 10 | containers: 11 | - name: manager 12 | imagePullPolicy: Always 13 | -------------------------------------------------------------------------------- /driver/examples/version2/snapshot/volsnap.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: snapshot.storage.k8s.io/v1 2 | kind: VolumeSnapshot 3 | metadata: 4 | name: ibm-spectrum-scale-snapshot 5 | spec: 6 | volumeSnapshotClassName: ibm-spectrum-scale-snapshotclass-advance 7 | source: 8 | persistentVolumeClaimName: scale-advance-pvc 9 | -------------------------------------------------------------------------------- /driver/examples/version2/volume/storageclass.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-advance 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "gpfs0" 8 | version: "2" 9 | reclaimPolicy: Delete 10 | 11 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/fileset/storageclassfileset_independent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-fileset 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "gpfs0" 8 | reclaimPolicy: Delete 9 | 10 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/lightweight/pvclw.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: scale-lt-pvc 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-csi-lt 12 | 13 | -------------------------------------------------------------------------------- /operator/config/testing/pull_policy/IfNotPresent.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: apps/v1 3 | kind: Deployment 4 | metadata: 5 | name: controller-manager 6 | namespace: system 7 | spec: 8 | template: 9 | spec: 10 | containers: 11 | - name: manager 12 | imagePullPolicy: IfNotPresent 13 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/staticprovisioning/snapshot/static_volsnap.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: snapshot.storage.k8s.io/v1 2 | kind: VolumeSnapshot 3 | metadata: 4 | name: ibm-spectrum-scale-snapshot 5 | namespace: default 6 | spec: 7 | source: 8 | volumeSnapshotContentName: ibm-spectrum-scale-snapshot-content 9 | 10 | -------------------------------------------------------------------------------- /operator/config/scorecard/patches/basic.config.yaml: -------------------------------------------------------------------------------- 1 | - op: add 2 | path: /stages/0/tests/- 3 | value: 4 | entrypoint: 5 | - scorecard-test 6 | - basic-check-spec 7 | image: quay.io/operator-framework/scorecard-test:v1.5.2 8 | labels: 9 | suite: basic 10 | test: basic-check-spec-test 11 | -------------------------------------------------------------------------------- /driver/examples/version2/snapshot/volsnapclass.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: snapshot.storage.k8s.io/v1 2 | kind: VolumeSnapshotClass 3 | metadata: 4 | name: ibm-spectrum-scale-snapshotclass-advance 5 | driver: spectrumscale.csi.ibm.com 6 | parameters: 7 | snapWindow: "60" #Optional : Time in minutes (default=30) 8 | deletionPolicy: Delete 9 | -------------------------------------------------------------------------------- /tools/ansible/deploy-playbook.yaml: -------------------------------------------------------------------------------- 1 | - name: Deploy operator 2 | hosts: localhost 3 | become: yes 4 | gather_facts: false 5 | tasks: 6 | - name: Ensure common tasks are run 7 | include_tasks: common/dev-env.yaml 8 | 9 | - name: Ensure Operator is verified 10 | include_tasks: deploy/preflight.yaml 11 | -------------------------------------------------------------------------------- /OWNERS: -------------------------------------------------------------------------------- 1 | filters: 2 | ".*": 3 | reviewers: 4 | - deeghuge 5 | approvers: 6 | - deeghuge 7 | 8 | "driver/.*": 9 | reviewers: 10 | - deeghuge 11 | approvers: 12 | - deeghuge 13 | 14 | "operator/.*": 15 | reviewers: 16 | - deeghuge 17 | approvers: 18 | - deeghuge 19 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/fileset/pvcfileset_dependent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: scale-fset-pvc 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-csi-fileset-dependent 12 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/fileset/pvcfileset_independent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: scale-fset-pvc 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-csi-fileset 12 | 13 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/dynamicprovisioning/snapshot/volsnap_static.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: snapshot.storage.k8s.io/v1 2 | kind: VolumeSnapshot 3 | metadata: 4 | name: ibm-spectrum-scale-snapshot-static 5 | spec: 6 | volumeSnapshotClassName: ibm-spectrum-scale-snapshotclass 7 | source: 8 | persistentVolumeClaimName: scale-static-pvc 9 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/dynamicprovisioning/volume/storageclass_static_independent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-static 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "fs1" 8 | existingVolume: "yes" 9 | reclaimPolicy: Delete -------------------------------------------------------------------------------- /driver/examples/version1/volume/lightweight/storageclasslw.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-lt 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "gpfs0" 8 | volDirBasePath: "pvfileset/lwdir" 9 | reclaimPolicy: Delete 10 | 11 | -------------------------------------------------------------------------------- /tools/ansible/deploy/hacks/templates/mmlscluster: -------------------------------------------------------------------------------- 1 | Value Filldown ID (\d+) 2 | Value Filldown Name ([^\s]+) 3 | Value Node ([^\s]+) 4 | Value Designation ([\w-]+) 5 | 6 | Start 7 | ^\s+GPFS cluster name:\s+${Name} 8 | ^\s+GPFS cluster id:\s+${ID} 9 | ^\s+\d+\s+${Node}\s+[^\s]+\s+[^\s]+\s+${Designation} -> Next.Record 10 | 11 | -------------------------------------------------------------------------------- /driver/chroot-wrapper.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | mountDir="/host" 4 | path="/usr/sbin:/usr/bin:/sbin:/bin" 5 | cmd=`basename "$0"` 6 | 7 | if [ ! -d "${mountDir}" ]; then 8 | echo "The directory ${mountDir} does not exist in container" 9 | exit 1 10 | fi 11 | exec chroot ${mountDir} /usr/bin/env -i PATH="${path}" ${cmd} "${@:1}" 12 | -------------------------------------------------------------------------------- /driver/examples/version1/snapshot/volsnap.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: snapshot.storage.k8s.io/v1 2 | kind: VolumeSnapshot 3 | metadata: 4 | name: ibm-spectrum-scale-snapshot 5 | namespace: default 6 | spec: 7 | volumeSnapshotClassName: ibm-spectrum-scale-snapshotclass 8 | source: 9 | persistentVolumeClaimName: ibm-spectrum-scale-pvc 10 | -------------------------------------------------------------------------------- /driver/examples/version2/volume/storageclass_tier.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-advance-tier 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "gpfs0" 8 | version: "2" 9 | tier: "system" 10 | reclaimPolicy: Delete 11 | 12 | -------------------------------------------------------------------------------- /operator/config/crd/kustomization.yaml: -------------------------------------------------------------------------------- 1 | # This kustomization.yaml is not intended to be run by itself, 2 | # since it depends on service name and namespace that are out of this kustomize package. 3 | # It should be run by config/default 4 | resources: 5 | - bases/csi.ibm.com_csiscaleoperators.yaml 6 | # +kubebuilder:scaffold:crdkustomizeresource 7 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/dynamicprovisioning/volume/pvc_static_independent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: scale-static-pvc 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-csi-static 12 | 13 | -------------------------------------------------------------------------------- /driver/examples/version1/snapshot/storageClassWithNodeClass.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-fileset 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "gpfs0" 8 | nodeClass: "ibm-spectrum-scale-nodeclass" 9 | reclaimPolicy: Delete 10 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/fileset/storageclassfileset_tier_independent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-fileset-tier 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "gpfs0" 8 | tier: "system" 9 | reclaimPolicy: Delete 10 | 11 | -------------------------------------------------------------------------------- /driver/examples/version2/volume/storageclass_compress.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-advance-compress 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | version: "2" 8 | volBackendFs: "gpfs0" 9 | compression: "true" 10 | reclaimPolicy: Delete 11 | 12 | -------------------------------------------------------------------------------- /driver/examples/version2/volume/storageclass_expansion.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-advance-expansion 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | version: "2" 8 | volBackendFs: "gpfs0" 9 | reclaimPolicy: Delete 10 | allowVolumeExpansion: true 11 | -------------------------------------------------------------------------------- /operator/config/testing/debug_logs_patch.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: apps/v1 3 | kind: Deployment 4 | metadata: 5 | name: controller-manager 6 | namespace: system 7 | spec: 8 | template: 9 | spec: 10 | containers: 11 | - name: manager 12 | env: 13 | - name: GO_DEBUG_LOGS 14 | value: "TRUE" 15 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/dynamicprovisioning/volume/pvc_static_dependent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: scale-static-pvc 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-csi-static-dependent 12 | 13 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/fileset/storageclassfileset_compress_independent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-fileset-compress 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "gpfs0" 8 | compression: "true" 9 | reclaimPolicy: Delete 10 | 11 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/fileset/storageclassfileset_expansion_independent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-fileset-expansion 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "gpfs0" 8 | reclaimPolicy: Delete 9 | allowVolumeExpansion: true 10 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/dynamicprovisioning/volume/storageclass_static_expansion_independent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-static-expansion 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "fs1" 8 | existingVolume: "yes" 9 | reclaimPolicy: Delete 10 | allowVolumeExpansion: true -------------------------------------------------------------------------------- /driver/examples/staticVolume/staticprovisioning/volume/static_pv.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolume 3 | metadata: 4 | name: static-scale-static-pv 5 | spec: 6 | capacity: 7 | storage: 1Gi 8 | accessModes: 9 | - ReadWriteMany 10 | csi: 11 | driver: spectrumscale.csi.ibm.com 12 | volumeHandle: "7118073361626808055;09762E69:5D36FE8D;path=/ibm/gpfs0/staticdir" 13 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/fileset/storageclassfileset_dependent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-fileset-dependent 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "gpfs0" 8 | filesetType: "dependent" 9 | parentFileset: "independent-fileset-gpfs0-fset1" 10 | reclaimPolicy: Delete 11 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/fileset/storageclassfileset_tier_dependent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-fileset-dependent 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "gpfs0" 8 | filesetType: "dependent" 9 | parentFileset: "independent-fileset-gpfs0-fset1" 10 | reclaimPolicy: Delete 11 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/dynamicprovisioning/volume/storageclass_static_dependent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-static-dependent 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "fs1" 8 | existingVolume: "yes" 9 | filesetType: "dependent" 10 | parentFileset: "scale-static-pvc-parent-1" 11 | reclaimPolicy: Delete -------------------------------------------------------------------------------- /driver/examples/version1/volume/fileset/clonefrompvc_independent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: ibm-spectrum-scale-pvc-clone-from-pvc 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-csi-fileset 12 | dataSource: 13 | name: scale-fset-pvc 14 | kind: PersistentVolumeClaim 15 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/fileset/storageclassfileset_compress_dependent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: storage.k8s.io/v1 2 | kind: StorageClass 3 | metadata: 4 | name: ibm-spectrum-scale-csi-fileset-dependent 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "gpfs0" 8 | filesetType: "dependent" 9 | parentFileset: "independent-fileset-gpfs0-fset1" 10 | compression: "true" 11 | reclaimPolicy: Delete 12 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/fileset/clonefrompvc_dependent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: ibm-spectrum-scale-pvc-clone-from-pvc 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-csi-fileset-dependent 12 | dataSource: 13 | name: scale-fset-pvc 14 | kind: PersistentVolumeClaim 15 | -------------------------------------------------------------------------------- /driver/examples/cacheVolume/storageclass.yaml: -------------------------------------------------------------------------------- 1 | kind: StorageClass 2 | apiVersion: storage.k8s.io/v1 3 | metadata: 4 | name: ibm-scale-cache 5 | provisioner: spectrumscale.csi.ibm.com 6 | parameters: 7 | volBackendFs: "fs0" 8 | volumeType: "cache" 9 | csi.storage.k8s.io/provisioner-secret-name: ${pvc.name}-secret 10 | csi.storage.k8s.io/provisioner-secret-namespace: ${pvc.namespace} 11 | reclaimPolicy: Delete 12 | allowVolumeExpansion: true 13 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/dynamicprovisioning/volume/clonefrompvc_static_independent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: ibm-spectrum-scale-static-pvc-clone-from-pvc 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-csi-fileset 12 | dataSource: 13 | name: scale-static-pvc 14 | kind: PersistentVolumeClaim -------------------------------------------------------------------------------- /driver/examples/staticVolume/dynamicprovisioning/volume/clonefrompvc_static_dependent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: ibm-spectrum-scale-static-pvc-clone-from-pvc 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-csi-static-dependent 12 | dataSource: 13 | name: scale-static-pvc 14 | kind: PersistentVolumeClaim -------------------------------------------------------------------------------- /driver/examples/version1/snapshot/pvcfrmsnap_rom.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: ibm-spectrum-scale-pvc-from-snapshot-shallow 5 | spec: 6 | accessModes: 7 | - ReadOnlyMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-csi-fileset 12 | dataSource: 13 | name: ibm-spectrum-scale-snapshot 14 | kind: VolumeSnapshot 15 | apiGroup: snapshot.storage.k8s.io -------------------------------------------------------------------------------- /driver/examples/version2/snapshot/pvcfrmsnap.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: ibm-spectrum-scale-pvc-from-snapshot 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-storageclass-advance 12 | dataSource: 13 | name: ibm-spectrum-scale-snapshot 14 | kind: VolumeSnapshot 15 | apiGroup: snapshot.storage.k8s.io 16 | -------------------------------------------------------------------------------- /driver/examples/cacheVolume/pod.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Pod 3 | metadata: 4 | name: pod1 5 | namespace: ns1 6 | labels: 7 | app: nginx 8 | spec: 9 | containers: 10 | - name: web-server 11 | image: gcr.io/google-containers/nginx 12 | volumeMounts: 13 | - name: mypvc 14 | mountPath: /data 15 | ports: 16 | - containerPort: 80 17 | volumes: 18 | - name: mypvc 19 | persistentVolumeClaim: 20 | claimName: cache-s3-pvc1 21 | -------------------------------------------------------------------------------- /driver/examples/version1/snapshot/pvcfrmsnap.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: ibm-spectrum-scale-pvc-from-snapshot 5 | namespace: default 6 | spec: 7 | accessModes: 8 | - ReadWriteMany 9 | resources: 10 | requests: 11 | storage: 1Gi 12 | storageClassName: ibm-spectrum-scale-storageclass 13 | dataSource: 14 | name: ibm-spectrum-scale-snapshot 15 | kind: VolumeSnapshot 16 | apiGroup: snapshot.storage.k8s.io 17 | -------------------------------------------------------------------------------- /operator/config/scorecard/kustomization.yaml: -------------------------------------------------------------------------------- 1 | resources: 2 | - bases/config.yaml 3 | patchesJson6902: 4 | - path: patches/basic.config.yaml 5 | target: 6 | group: scorecard.operatorframework.io 7 | version: v1alpha3 8 | kind: Configuration 9 | name: config 10 | - path: patches/olm.config.yaml 11 | target: 12 | group: scorecard.operatorframework.io 13 | version: v1alpha3 14 | kind: Configuration 15 | name: config 16 | # +kubebuilder:scaffold:patchesJson6902 17 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/staticprovisioning/pvcfrmsnap.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: ibm-spectrum-scale-pvc-from-snapshot 5 | namespace: default 6 | spec: 7 | accessModes: 8 | - ReadWriteMany 9 | resources: 10 | requests: 11 | storage: 1Gi 12 | storageClassName: ibm-spectrum-scale-storageclass 13 | dataSource: 14 | name: ibm-spectrum-scale-snapshot 15 | kind: VolumeSnapshot 16 | apiGroup: snapshot.storage.k8s.io 17 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/dynamicprovisioning/snapshot/pvcfrmsnap_static.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: ibm-spectrum-scale-pvc-from-snapshot-static 5 | spec: 6 | accessModes: 7 | - ReadWriteMany 8 | resources: 9 | requests: 10 | storage: 1Gi 11 | storageClassName: ibm-spectrum-scale-csi-fileset 12 | dataSource: 13 | name: ibm-spectrum-scale-snapshot-static 14 | kind: VolumeSnapshot 15 | apiGroup: snapshot.storage.k8s.io 16 | -------------------------------------------------------------------------------- /tools/ansible/common/runtime-env.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Set environment facts 3 | set_fact: 4 | OPERATOR_SDK_VER: "v1.36.1" 5 | OPERATOR_VERSION: "3.1.0" 6 | 7 | #- name: Ensure 'python3' is installed 8 | # package: 9 | # name: "{{packages}}" 10 | # vars: 11 | # packages: 12 | # - python36 13 | # - python36-pip 14 | # 15 | #- name: Ensure 'python3' requirements are installed 16 | # pip: 17 | # requirements: "{{playbook_dir}}/common/requirements.txt" 18 | # executable: pip3 19 | -------------------------------------------------------------------------------- /driver/examples/version2/volume/pod.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Pod 3 | metadata: 4 | name: csi-scale-fsetdemo-pod 5 | labels: 6 | app: nginx 7 | spec: 8 | containers: 9 | - name: web-server 10 | image: nginx 11 | volumeMounts: 12 | - name: mypvc 13 | mountPath: /usr/share/nginx/html/scale 14 | ports: 15 | - containerPort: 80 16 | volumes: 17 | - name: mypvc 18 | persistentVolumeClaim: 19 | claimName: scale-advance-pvc 20 | readOnly: false 21 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/lightweight/podlw.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Pod 3 | metadata: 4 | name: csi-scale-ltwtdemo-pod 5 | labels: 6 | app: nginx 7 | spec: 8 | containers: 9 | - name: web-server 10 | image: nginx 11 | volumeMounts: 12 | - name: mypvc 13 | mountPath: /usr/share/nginx/html/scale 14 | ports: 15 | - containerPort: 80 16 | volumes: 17 | - name: mypvc 18 | persistentVolumeClaim: 19 | claimName: scale-lt-pvc 20 | readOnly: false 21 | 22 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/staticprovisioning/snapshot/static_volsnapcontent.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: snapshot.storage.k8s.io/v1 2 | kind: VolumeSnapshotContent 3 | metadata: 4 | name: ibm-spectrum-scale-snapshot-content 5 | spec: 6 | deletionPolicy: Retain 7 | driver: spectrumscale.csi.ibm.com 8 | source: 9 | snapshotHandle: 18133600329030594550;0A1501E9:5F02F150;pvc-79e82f45-1b25-4d83-8ed0-c0d370cad5bd;ibm-spectrum-scale-snapshot 10 | volumeSnapshotRef: 11 | name: ibm-spectrum-scale-snapshot 12 | namespace: default 13 | -------------------------------------------------------------------------------- /driver/examples/version1/volume/fileset/podfileset.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Pod 3 | metadata: 4 | name: csi-scale-fsetdemo-pod 5 | labels: 6 | app: nginx 7 | spec: 8 | containers: 9 | - name: web-server 10 | image: nginx 11 | volumeMounts: 12 | - name: mypvc 13 | mountPath: /usr/share/nginx/html/scale 14 | ports: 15 | - containerPort: 80 16 | volumes: 17 | - name: mypvc 18 | persistentVolumeClaim: 19 | claimName: scale-fset-pvc 20 | readOnly: false 21 | 22 | -------------------------------------------------------------------------------- /tools/ansible/deploy/preflight.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | # Run preflight tests. 3 | - name: Copy script hacks 4 | copy: 5 | src: hacks 6 | dest: /tmp/csi-deploy 7 | 8 | - name: Gather IBM Storage Scale cluster information 9 | script: "/tmp/csi-deploy/hacks/getscalecluster.py --templates /tmp/csi-deploy/hacks" 10 | register: cluster_std 11 | 12 | - name: Ensure cluster information is ingested to ansible 13 | set_fact: 14 | cluster: "{{cluster_std.stdout | from_yaml}}" 15 | 16 | - debug: var=cluster 17 | 18 | 19 | 20 | 21 | 22 | -------------------------------------------------------------------------------- /operator/PROJECT: -------------------------------------------------------------------------------- 1 | domain: ibm.com 2 | layout: 3 | - go.kubebuilder.io/v3 4 | plugins: 5 | manifests.sdk.operatorframework.io/v2: {} 6 | scorecard.sdk.operatorframework.io/v2: {} 7 | projectName: ibm-spectrum-scale-csi 8 | repo: github.com/IBM/ibm-spectrum-scale-csi/operator 9 | resources: 10 | - api: 11 | crdVersion: v1 12 | namespaced: true 13 | controller: true 14 | domain: ibm.com 15 | group: csi 16 | kind: CSIScaleOperator 17 | path: github.com/IBM/ibm-spectrum-scale-csi/operator/api/v1 18 | version: v1 19 | version: "3" 20 | -------------------------------------------------------------------------------- /driver/examples/cacheVolume/pvcnfs.yaml: -------------------------------------------------------------------------------- 1 | # Create a secret with nfs server details first. 2 | # e.g. for a PVC with name 'cache-nfs-pvc1' in namespace 'ns1', 3 | # oc create secret generic cache-nfs-pvc1-secret -n ns1 \ 4 | # --from-literal=nfsServers --from-literal=nfsPath= \ 5 | --- 6 | kind: PersistentVolumeClaim 7 | apiVersion: v1 8 | metadata: 9 | name: cache-nfs-pvc1 10 | namespace: ns1 11 | spec: 12 | storageClassName: ibm-scale-cache 13 | accessModes: 14 | - ReadWriteMany 15 | resources: 16 | requests: 17 | storage: 2Gi 18 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/dynamicprovisioning/volume/pod_static.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Pod 3 | metadata: 4 | name: csi-scale-staticdemo-pod 5 | labels: 6 | app: nginx 7 | spec: 8 | containers: 9 | - name: web-server 10 | image: nginx 11 | volumeMounts: 12 | - name: mypvc 13 | mountPath: /usr/share/nginx/html/scale 14 | ports: 15 | - containerPort: 80 16 | volumes: 17 | - name: mypvc 18 | persistentVolumeClaim: 19 | claimName: scale-static-pvc 20 | readOnly: false 21 | 22 | -------------------------------------------------------------------------------- /driver/examples/staticVolume/staticprovisioning/volume/static_pod.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Pod 3 | metadata: 4 | name: csi-scale-staticdemo-pod 5 | labels: 6 | app: nginx 7 | spec: 8 | containers: 9 | - name: web-server 10 | image: nginx 11 | volumeMounts: 12 | - name: mypvc 13 | mountPath: /usr/share/nginx/html/scale 14 | ports: 15 | - containerPort: 80 16 | volumes: 17 | - name: mypvc 18 | persistentVolumeClaim: 19 | claimName: scale-static-pvc 20 | readOnly: false 21 | 22 | -------------------------------------------------------------------------------- /operator/config/rbac/service_account.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: ServiceAccount 4 | metadata: 5 | labels: 6 | app.kubernetes.io/instance: ibm-spectrum-scale-csi-operator 7 | app.kubernetes.io/managed-by: ibm-spectrum-scale-csi-operator 8 | app.kubernetes.io/name: ibm-spectrum-scale-csi-operator 9 | product: ibm-spectrum-scale-csi 10 | release: ibm-spectrum-scale-csi-operator 11 | name: ibm-spectrum-scale-csi-operator 12 | namespace: ibm-spectrum-scale-csi-driver 13 | imagePullSecrets: 14 | - name: ibm-entitlement-key 15 | -------------------------------------------------------------------------------- /operator/config/testing/kustomization.yaml: -------------------------------------------------------------------------------- 1 | # Adds namespace to all resources. 2 | namespace: osdk-test 3 | 4 | namePrefix: osdk- 5 | 6 | # Labels to add to all resources and selectors. 7 | #commonLabels: 8 | # someName: someValue 9 | 10 | patchesStrategicMerge: 11 | - manager_image.yaml 12 | - debug_logs_patch.yaml 13 | - ../default/manager_auth_proxy_patch.yaml 14 | 15 | apiVersion: kustomize.config.k8s.io/v1beta1 16 | kind: Kustomization 17 | resources: 18 | - ../crd 19 | - ../rbac 20 | - ../manager 21 | images: 22 | - name: testing 23 | newName: testing-operator 24 | -------------------------------------------------------------------------------- /tools/ansible/deploy/nodeprep.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | # Prepares the environment on the worker nodes. 3 | 4 | #- name: Ensure required packages are installed 5 | # package: 6 | # name: "{{packages}}" 7 | # vars: 8 | # packages: 9 | # - iscsi-initiator-utils 10 | # - xfsprogs 11 | # - device-mapper-multipath 12 | # 13 | #- name: Add the dm-multipath module 14 | # modprobe: 15 | # name: dm-multipath 16 | # state: present 17 | # 18 | #- name: Enable service multipathd, reload 19 | # systemd: 20 | # name: multipathd 21 | # enabled: yes 22 | # state: restarted 23 | 24 | 25 | -------------------------------------------------------------------------------- /operator/hacks/boilerplate.go.txt: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright 2022. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"); 5 | you may not use this file except in compliance with the License. 6 | You may obtain a copy of the License at 7 | 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | 10 | Unless required by applicable law or agreed to in writing, software 11 | distributed under the License is distributed on an "AS IS" BASIS, 12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | See the License for the specific language governing permissions and 14 | limitations under the License. 15 | */ -------------------------------------------------------------------------------- /tools/ansible/dev-env-playbook.yaml: -------------------------------------------------------------------------------- 1 | - name: Prepare environment for development. 2 | hosts: localhost 3 | become: yes 4 | gather_facts: false 5 | tasks: 6 | - name: Ensure common tasks are run 7 | include_tasks: common/dev-env.yaml 8 | 9 | # - name: Ensure helm is installed (OLM testing) 10 | # shell: curl -L https://git.io/get_helm.sh | bash 11 | # 12 | # - name : Ensure helm registry is installed (OLM testing) 13 | # shell: cd ~/.helm/plugins/ && git clone https://github.com/app-registry/appr-helm-plugin.git registry 14 | # 15 | # - name: Ensure helm registry is loaded (OLM testing) 16 | # shell: helm registry --help 17 | -------------------------------------------------------------------------------- /driver/examples/cacheVolume/pvcs3.yaml: -------------------------------------------------------------------------------- 1 | # Create a secret with s3 bucket details first. 2 | # e.g. for a PVC with name 'cache-s3-pvc1' in namespace 'ns1', 3 | # oc create secret generic cache-s3-pvc1-secret -n ns1 \ 4 | # --from-literal=endpoint= --from-literal=bucket= \ 5 | # --from-literal=accesskey= --from-literal=secretkey= 6 | --- 7 | kind: PersistentVolumeClaim 8 | apiVersion: v1 9 | metadata: 10 | name: cache-s3-pvc1 11 | namespace: ns1 12 | spec: 13 | storageClassName: ibm-scale-cache 14 | accessModes: 15 | - ReadWriteMany 16 | resources: 17 | requests: 18 | storage: 2Gi 19 | -------------------------------------------------------------------------------- /tools/scripts/ci/ci-env: -------------------------------------------------------------------------------- 1 | export GO111MODULE=on 2 | export KUBE_VERSION="v1.16.2" 3 | export OP_VER="v0.17.0" 4 | 5 | export OPERATOR_SDK="https://github.com/operator-framework/operator-sdk/releases/download/${OP_VER}/operator-sdk-${OP_VER}-x86_64-linux-gnu" 6 | export 7 | export CSI_OP_BUILD_DIR="${TRAVIS_BUILD_DIR}/operator" 8 | 9 | export OPERATOR_NAME=ibm-spectrum-scale-csi-operator 10 | export OLM_MANIFEST="${CSI_OP_BUILD_DIR}/deploy/olm-catalog/${OPERATOR_NAME}/" 11 | 12 | export CHANGE_MINIKUBE_NONE_USER=true 13 | export MINIKUBE_WANTUPDATENOTIFICATION=false 14 | export MINIKUBE_WANTREPORTERRORPROMPT=false 15 | export MINIKUBE_HOME=$HOME 16 | export KUBECONFIG=$HOME/.kube/config 17 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for this project 4 | title: '' 5 | labels: 'Type: Enhancement' 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Is your feature request related to a problem? Please describe.** 11 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 12 | 13 | **Describe the solution you'd like** 14 | A clear and concise description of what you want to happen. 15 | 16 | **Describe alternatives you've considered** 17 | A clear and concise description of any alternative solutions or features you've considered. 18 | 19 | **Additional context** 20 | Add any other context or screenshots about the feature request here. 21 | -------------------------------------------------------------------------------- /operator/config/rbac/role_binding.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: rbac.authorization.k8s.io/v1 3 | kind: ClusterRoleBinding 4 | metadata: 5 | labels: 6 | app.kubernetes.io/instance: ibm-spectrum-scale-csi-operator 7 | app.kubernetes.io/managed-by: ibm-spectrum-scale-csi-operator 8 | app.kubernetes.io/name: ibm-spectrum-scale-csi-operator 9 | product: ibm-spectrum-scale-csi 10 | release: ibm-spectrum-scale-csi-operator 11 | name: ibm-spectrum-scale-csi-operator 12 | roleRef: 13 | apiGroup: rbac.authorization.k8s.io 14 | kind: ClusterRole 15 | name: ibm-spectrum-scale-csi-operator 16 | subjects: 17 | - kind: ServiceAccount 18 | name: ibm-spectrum-scale-csi-operator 19 | namespace: ibm-spectrum-scale-csi-driver 20 | -------------------------------------------------------------------------------- /operator/config/overlays/default/images_manifest.yaml: -------------------------------------------------------------------------------- 1 | version: 3.1.0 2 | images: 3 | csiSnapshotter: cp.icr.io/cp/gpfs/csi/csi-snapshotter@sha256:bc7be893ecc3ad524194aa6573b2f5c06cd469bdf21a500ab6c99c2ba1c4d64d 4 | csiAttacher: cp.icr.io/cp/gpfs/csi/csi-attacher@sha256:5aaefc24f315b182233c8b6146077f8c32e274d864cb03c632206e78bd0302da 5 | csiProvisioner: cp.icr.io/cp/gpfs/csi/csi-provisioner@sha256:bb057f866177d5f4139a1527e594499cbe0feeb67b63aaca8679dfdf0a6016f9 6 | csiLivenessprobe: cp.icr.io/cp/gpfs/csi/livenessprobe@sha256:88092d100909918ae0a768956cf78c88bc59cd7232720f7cdbdfb5d2e235001e 7 | csiNodeRegistrar: cp.icr.io/cp/gpfs/csi/csi-node-driver-registrar@sha256:5244abbe87e01b35adeb8bb13882a74785df0c0619f8325c9e950395c3f72a97 8 | csiResizer: cp.icr.io/cp/gpfs/csi/csi-resizer@sha256:5e7cbb63fd497fa913caa21fee1a69f727c220c6fa83c5f8bb0995e2ad73a474 9 | -------------------------------------------------------------------------------- /tools/scripts/ci/install_minikube.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -e 3 | set -u 4 | 5 | # Install minikube for CI 6 | 7 | #. ./ci-env 8 | 9 | # Install kubectl for minikube 10 | curl -Lo kubectl https://dl.k8s.io/release/v${KUBE_VERSION}/bin/linux/amd64/kubectl 11 | chmod +x kubectl 12 | sudo mv kubectl /usr/local/bin/ 13 | 14 | # Install minikube 15 | curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 16 | chmod +x minikube 17 | sudo mv minikube /usr/local/bin/ 18 | 19 | # Setup configuration 20 | mkdir -p $HOME/.kube $HOME/.minikube 21 | touch $KUBECONFIG 22 | 23 | echo $HOME 24 | 25 | # Start minikube and chown for minikub. 26 | sudo chown -R travis: /home/travis/.minikube/ 27 | sudo minikube start --driver=none --kubernetes-version=${KUBE_VERSION} --extra-config=kubeadm.ignore-preflight-errors=NumCPU --force 28 | 29 | minikube update-context 30 | eval $(minikube docker-env) 31 | 32 | -------------------------------------------------------------------------------- /operator/controllers/util/boolptr/boolptr.go: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2022, 2024 IBM Corp. 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package boolptr 18 | 19 | // True returns a *bool whose underlying value is true. 20 | func True() *bool { 21 | t := true 22 | return &t 23 | } 24 | 25 | // False returns a *bool whose underlying value is false. 26 | func False() *bool { 27 | t := false 28 | return &t 29 | } 30 | -------------------------------------------------------------------------------- /operator/bundle.Dockerfile: -------------------------------------------------------------------------------- 1 | FROM scratch 2 | 3 | # Core bundle labels. 4 | LABEL operators.operatorframework.io.bundle.mediatype.v1=registry+v1 5 | LABEL operators.operatorframework.io.bundle.manifests.v1=manifests/ 6 | LABEL operators.operatorframework.io.bundle.metadata.v1=metadata/ 7 | LABEL operators.operatorframework.io.bundle.package.v1=ibm-spectrum-scale-csi-operator 8 | LABEL operators.operatorframework.io.bundle.channels.v1=alpha 9 | LABEL operators.operatorframework.io.metrics.mediatype.v1=metrics+v1 10 | LABEL operators.operatorframework.io.metrics.project_layout=go.kubebuilder.io/v3 11 | 12 | # Labels for testing. 13 | LABEL operators.operatorframework.io.test.mediatype.v1=scorecard+v1 14 | LABEL operators.operatorframework.io.test.config.v1=tests/scorecard/ 15 | 16 | # Copy files to locations specified by labels. 17 | COPY config/olm-catalog/ibm-spectrum-scale-csi-operator/manifests /manifests/ 18 | COPY config/olm-catalog/ibm-spectrum-scale-csi-operator/metadata /metadata/ 19 | -------------------------------------------------------------------------------- /operator/controllers/util/k8sutil/k8sutil.go: -------------------------------------------------------------------------------- 1 | package k8sutil 2 | 3 | import corev1 "k8s.io/api/core/v1" 4 | 5 | // EnsureHostPathVolumeSource returns VolumeSource of type hostpath 6 | // with given path and pathType. 7 | func EnsureHostPathVolumeSource(path, pathType string) corev1.VolumeSource { 8 | t := corev1.HostPathType(pathType) 9 | return corev1.VolumeSource{ 10 | HostPath: &corev1.HostPathVolumeSource{ 11 | Path: path, 12 | Type: &t, 13 | }, 14 | } 15 | } 16 | 17 | // EnsureConfigMapVolumeSource returns VolumeSource of type configMap 18 | // with given name. 19 | func EnsureConfigMapVolumeSource(name string) corev1.VolumeSource { 20 | return corev1.VolumeSource{ 21 | ConfigMap: &corev1.ConfigMapVolumeSource{ 22 | LocalObjectReference: corev1.LocalObjectReference{ 23 | Name: name, 24 | }, 25 | }, 26 | } 27 | } 28 | 29 | // EnsureVolume return a volume with given name and volume source. 30 | func EnsureVolume(name string, source corev1.VolumeSource) corev1.Volume { 31 | return corev1.Volume{ 32 | Name: name, 33 | VolumeSource: source, 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/release-checklist.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Release Checklist 3 | about: Create issue for release Checklist 4 | title: '' 5 | labels: 'Type: Release Checklist' 6 | assignees: '' 7 | 8 | --- 9 | 10 | 11 | ## IBM Storage Scale CSI Driver GA release Checklist 12 | - [ ] CSI snap works on Vanila k8s 13 | - [ ] Verify GitHub links are working 14 | - [ ] Verify IBM docs links are working 15 | - [ ] Pause dev checkin once stop-ship phase starts 16 | - [ ] Create Golden Master images on quay (Atleast 1 week before GA) 17 | - [ ] Regression Complete 18 | - [ ] Realworld test complete 19 | - [ ] Check images for Vulnerabilities(Driver, Operator, sidecars) 20 | - [ ] IBM Docs Content Verification (1 week before GA) 21 | - [ ] Create git release tag (draft) with release content (1 week before GA) 22 | - [ ] Merge dev into master (on GA date) 23 | - [ ] Push multi arch images to quay (on GA date) 24 | - [ ] Publish git tag (on GA date) 25 | - [ ] Verify the images and yaml from release branch are correct - CLI install/Upgrade (on GA date) 26 | - [ ] Update driver listing https://kubernetes-csi.github.io/docs/drivers.html 27 | 28 | -------------------------------------------------------------------------------- /operator/config/overlays/default/kustomization.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: kustomize.config.k8s.io/v1beta1 3 | kind: Kustomization 4 | 5 | # Adds namespace to all resources. 6 | namespace: ibm-spectrum-scale-csi-driver 7 | 8 | resources: 9 | - ../../rbac/ 10 | - ../../manager/ 11 | - ../../crd/ 12 | 13 | patches: 14 | - target: 15 | kind: Deployment 16 | labelSelector: "app.kubernetes.io/name=ibm-spectrum-scale-csi-operator" 17 | patch: |- 18 | apiVersion: apps/v1 19 | kind: Deployment 20 | metadata: 21 | name: ibm-spectrum-scale-csi-operator 22 | spec: 23 | template: 24 | spec: 25 | containers: 26 | - name: operator 27 | image: quay.io/ibm-spectrum-scale-dev/ibm-spectrum-scale-csi-operator@sha256:de0027696b54d29880e8b4c96183b04c1c7872a78d98b1e20f374208f82e2666 28 | env: 29 | - name: METRICS_BIND_ADDRESS 30 | - name: WATCH_NAMESPACE 31 | - name: CSI_DRIVER_IMAGE 32 | value: quay.io/ibm-spectrum-scale-dev/ibm-spectrum-scale-csi-driver@sha256:4178d546c92cac71ac785e00052693a7aaa70eb7c768c41a7bf00499aa84b565 33 | -------------------------------------------------------------------------------- /tools/volume_migration_scripts/README.md: -------------------------------------------------------------------------------- 1 | # IBM Storage Scale CSI/IBM Storage Scale container native → Volume Migration Scripts 2 | 3 | This provides scripts to migrate existing PersistentVolumes (PVs) to the new compatible volumeHandles. 4 | 5 | ## Scripts 6 | 7 | 1. [`migration_csi_to_cnsa.bash`](README_migration_csi_to_cnsa.md) 8 | Migrates IBM Storage Scale CSI PersistentVolumes to the IBM Storage Scale container native format by updating the volumeHandle path with the specified filesystem mount path prefix. 9 | 10 | 2. [`migration_csi_primary_removal.bash`](README_migration_csi_primary_removal.md) 11 | Migrates IBM Storage Scale CSI PersistentVolumes to a new format by updating the volumeHandle path with the specified filesystem mount path prefix after primary filesystem/fileset removal. 12 | 13 | 3. [`migration_cnsa_primary_removal.bash`](README_migration_cnsa_primary_removal.md) 14 | Migrates IBM Storage Scale container native PersistentVolumes details by updating the volumeHandle path with the specified filesystem mount path prefix after primary filesystem/fileset removal. 15 | 16 | 17 | **Note:** 18 | Each script has its **own README** with usage, inputs, outputs, and validation notes. 19 | -------------------------------------------------------------------------------- /tools/ansible/generate-playbook.yaml: -------------------------------------------------------------------------------- 1 | - name: Dynamically Generate Installer Files 2 | hosts: localhost 3 | gather_facts: false 4 | 5 | vars: 6 | repo_root_dir: "{{ playbook_dir }}/../../" 7 | operator_dir: "{{ repo_root_dir }}/operator" 8 | 9 | generated_dir: "{{ repo_root_dir }}/generated/installer" 10 | 11 | filename: "{{ 'ibm-spectrum-scale-csi-operator.test.yaml' if travis_testing else 'ibm-spectrum-scale-csi-operator.yaml' }}" 12 | filename_dev: 'ibm-spectrum-scale-csi-operator-dev.yaml' 13 | generated_op_yaml: "{{ generated_dir }}/{{ filename }}" 14 | generated_op_dev_yaml: "{{ generated_dir }}/{{ filename_dev }}" 15 | operator_files: 16 | - name: "deploy/operator.yaml" 17 | - name: "deploy/role.yaml" 18 | - name: "deploy/role_binding.yaml" 19 | - name: "deploy/service_account.yaml" 20 | - name: "deploy/crds/csiscaleoperators.csi.ibm.com.crd.yaml" 21 | 22 | # Copy over the non modified file to the installer directory 23 | non_generated: false 24 | # Set to true to test Travis 25 | travis_testing: false 26 | 27 | tasks: 28 | - name: Create files to help manually install the operator 29 | include_tasks: generate/create_files.yaml 30 | -------------------------------------------------------------------------------- /operator/hacks/csv_prep.py: -------------------------------------------------------------------------------- 1 | #!/bin/python 2 | 3 | import argparse 4 | import sys 5 | import os 6 | import yaml 7 | 8 | BASE_DIR="{0}/../".format(os.path.dirname(os.path.realpath(__file__))) 9 | DEFAULT_VERSION="2.6.0" 10 | CSV_PATH="{0}deploy/olm-catalog/ibm-spectrum-scale-csi-operator/{1}/ibm-spectrum-scale-csi-operator.v{1}.clusterserviceversion.yaml" 11 | 12 | def main(args): 13 | parser = argparse.ArgumentParser( 14 | description='''A hack to prep the CSV for regeneration.''') 15 | 16 | parser.add_argument( '--version', metavar='CSV Version', dest='version', default=DEFAULT_VERSION, 17 | help='''The version of the CSV to update''') 18 | 19 | args = parser.parse_args() 20 | 21 | csvf = CSV_PATH.format(BASE_DIR, args.version) 22 | csv = None 23 | try: 24 | with open(csvf, 'r') as stream: 25 | csv = yaml.safe_load(stream) 26 | except yaml.YAMLError as e: 27 | print(e) 28 | return 1 29 | 30 | # Edit the contents of the CSV 31 | if csv is not None: 32 | csv.get("spec",{}).pop("install", None) 33 | 34 | 35 | with open(csvf, 'w') as outfile: 36 | yaml.dump(csv, outfile, default_flow_style=False) 37 | 38 | if __name__ == "__main__": 39 | sys.exit(main(sys.argv)) 40 | 41 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | ## Pull request checklist 2 | 3 | 4 | 5 | 6 | 7 | 8 | ```release-notes 9 | 10 | ``` 11 | 12 | ## Pull request type 13 | 14 | 15 | 16 | 17 | 18 | Please check the type of change your PR introduces: 19 | - [ ] Bugfix 20 | - [ ] Feature Enhancement 21 | - [ ] Test Automation 22 | - [ ] Code Refactoring (no functional changes, no api changes) 23 | - [ ] Build related changes 24 | - [ ] Community Operator listing 25 | - [ ] Other (please describe): 26 | 27 | 28 | ## What is the current behavior? 29 | 30 | - 31 | 32 | ## What is the new behavior? 33 | 34 | - 35 | 36 | ## How risky is this change? 37 | - [ ] Small, isolated change 38 | - [ ] Medium, requires regression testing 39 | - [ ] Large, requires functional and regression testing 40 | 41 | -------------------------------------------------------------------------------- /operator/api/v1/groupversion_info.go: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright 2022, 2024 IBM Corp. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"); 5 | you may not use this file except in compliance with the License. 6 | You may obtain a copy of the License at 7 | 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | 10 | Unless required by applicable law or agreed to in writing, software 11 | distributed under the License is distributed on an "AS IS" BASIS, 12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | See the License for the specific language governing permissions and 14 | limitations under the License. 15 | */ 16 | 17 | // Package v1 contains API Schema definitions for the csi v1 API group 18 | // +kubebuilder:object:generate=true 19 | // +groupName=csi.ibm.com 20 | package v1 21 | 22 | import ( 23 | "k8s.io/apimachinery/pkg/runtime/schema" 24 | "sigs.k8s.io/controller-runtime/pkg/scheme" 25 | ) 26 | 27 | var ( 28 | // GroupVersion is group version used to register these objects 29 | GroupVersion = schema.GroupVersion{Group: "csi.ibm.com", Version: "v1"} 30 | 31 | // SchemeBuilder is used to add go types to the GroupVersionKind scheme 32 | SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} 33 | 34 | // AddToScheme adds the types in this group-version to the given scheme. 35 | AddToScheme = SchemeBuilder.AddToScheme 36 | ) 37 | -------------------------------------------------------------------------------- /operator/config/scc/security_context_constraint.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: security.openshift.io/v1 2 | kind: SecurityContextConstraints 3 | metadata: 4 | name: spectrum-scale-csiaccess 5 | annotations: 6 | kubernetes.io/description: allows hostpath and host network to be accessible 7 | allowHostDirVolumePlugin: true 8 | allowHostIPC: false 9 | allowHostNetwork: true 10 | allowHostPID: false 11 | allowHostPorts: false 12 | allowPrivilegeEscalation: true 13 | allowPrivilegedContainer: true 14 | allowedCapabilities: [] 15 | defaultAddCapabilities: [] 16 | fsGroup: 17 | type: MustRunAs 18 | groups: [] 19 | priority: null 20 | readOnlyRootFilesystem: false 21 | requiredDropCapabilities: 22 | - KILL 23 | - MKNOD 24 | - SETUID 25 | - SETGID 26 | runAsUser: 27 | type: RunAsAny 28 | seLinuxContext: 29 | type: RunAsAny 30 | supplementalGroups: 31 | type: RunAsAny 32 | users: 33 | - system:serviceaccount:ibm-spectrum-scale-csi:ibm-spectrum-scale-csi-attacher 34 | - system:serviceaccount:ibm-spectrum-scale-csi:ibm-spectrum-scale-csi-provisioner 35 | - system:serviceaccount:ibm-spectrum-scale-csi:ibm-spectrum-scale-csi-node 36 | - system:serviceaccount:ibm-spectrum-scale-csi:ibm-spectrum-scale-csi-snapshotter 37 | - system:serviceaccount:ibm-spectrum-scale-csi:ibm-spectrum-scale-csi-resizer 38 | volumes: 39 | - configMap 40 | - downwardAPI 41 | - emptyDir 42 | - hostPath 43 | - persistentVolumeClaim 44 | - projected 45 | - secret 46 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | title: '' 5 | labels: 'Type: Bug' 6 | assignees: '' 7 | 8 | --- 9 | 10 | ## Describe the bug 11 | A clear and concise description of what the bug is. 12 | 13 | ## How to Reproduce? 14 | Please list the steps to help development teams reproduce the behavior 15 | 16 | 1. ... 17 | 18 | 19 | ## Expected behavior 20 | A clear and concise description of what you expected to happen. 21 | 22 | ### Data Collection and Debugging 23 | 24 | Environmental output 25 | 26 | - What openshift/kubernetes version are you running, and the architecture? 27 | - `kubectl get pods -o wide -n < csi driver namespace> ` 28 | - `kubectl get nodes -o wide` 29 | - IBM Storage Scale container native version 30 | - IBM Storage Scale version 31 | - Output for `./tools/storage-scale-driver-snap.sh -n < csi driver namespace> -v ` 32 | 33 | 34 | Tool to collect the CSI snap: 35 | 36 | `./tools/storage-scale-driver-snap.sh -n < csi driver namespace>` 37 | 38 | ## Screenshots 39 | If applicable, add screenshots to help explain your problem. 40 | 41 | ## Additional context 42 | Add any other context about the problem here. 43 | 44 | ### Add labels 45 | 46 | - Component: 47 | - Severity: 48 | - Customer Impact: 49 | - Customer Probability: 50 | - Phase: 51 | 52 | Note : See [labels](https://github.com/IBM/ibm-spectrum-scale-csi/labels) for the labels 53 | -------------------------------------------------------------------------------- /operator/config/scorecard/patches/olm.config.yaml: -------------------------------------------------------------------------------- 1 | - op: add 2 | path: /stages/0/tests/- 3 | value: 4 | entrypoint: 5 | - scorecard-test 6 | - olm-bundle-validation 7 | image: quay.io/operator-framework/scorecard-test:v1.5.2 8 | labels: 9 | suite: olm 10 | test: olm-bundle-validation-test 11 | - op: add 12 | path: /stages/0/tests/- 13 | value: 14 | entrypoint: 15 | - scorecard-test 16 | - olm-crds-have-validation 17 | image: quay.io/operator-framework/scorecard-test:v1.5.2 18 | labels: 19 | suite: olm 20 | test: olm-crds-have-validation-test 21 | - op: add 22 | path: /stages/0/tests/- 23 | value: 24 | entrypoint: 25 | - scorecard-test 26 | - olm-crds-have-resources 27 | image: quay.io/operator-framework/scorecard-test:v1.5.2 28 | labels: 29 | suite: olm 30 | test: olm-crds-have-resources-test 31 | - op: add 32 | path: /stages/0/tests/- 33 | value: 34 | entrypoint: 35 | - scorecard-test 36 | - olm-spec-descriptors 37 | image: quay.io/operator-framework/scorecard-test:v1.5.2 38 | labels: 39 | suite: olm 40 | test: olm-spec-descriptors-test 41 | - op: add 42 | path: /stages/0/tests/- 43 | value: 44 | entrypoint: 45 | - scorecard-test 46 | - olm-status-descriptors 47 | image: quay.io/operator-framework/scorecard-test:v1.5.2 48 | labels: 49 | suite: olm 50 | test: olm-status-descriptors-test 51 | -------------------------------------------------------------------------------- /operator/hacks/csv_copy_docs.py: -------------------------------------------------------------------------------- 1 | #!/bin/python 2 | 3 | import argparse 4 | import sys 5 | import os 6 | import yaml 7 | 8 | BASE_DIR="{0}/../".format(os.path.dirname(os.path.realpath(__file__))) 9 | DEFAULT_VERSION="2.6.0" 10 | CSV_PATH="{0}deploy/olm-catalog/ibm-spectrum-scale-csi-operator/{1}/ibm-spectrum-scale-csi-operator.v{1}.clusterserviceversion.yaml" 11 | QUICKSTART="{0}../../../../docs/source/get-started/quickstart.md".format(BASE_DIR) 12 | 13 | def main(args): 14 | parser = argparse.ArgumentParser( 15 | description='''A hack to copy docs into the CSV.''') 16 | 17 | parser.add_argument( '--version', metavar='CSV Version', dest='version', default=DEFAULT_VERSION, 18 | help='''The version of the CSV to update''') 19 | 20 | 21 | args = parser.parse_args() 22 | 23 | csvf = CSV_PATH.format(BASE_DIR, args.version) 24 | csv = None 25 | try: 26 | with open(csvf, 'r') as stream: 27 | csv = yaml.safe_load(stream) 28 | except yaml.YAMLError as e: 29 | print(e) 30 | return 1 31 | 32 | # Edit the contents of the CSV 33 | if csv is not None: 34 | spec = csv.get("spec",{}) 35 | 36 | # Set the description of the CSV to the quickstart. 37 | with open(QUICKSTART, 'r') as stream: 38 | spec["description"] = stream.read() 39 | 40 | 41 | with open(csvf, 'w') as outfile: 42 | yaml.dump(csv, outfile, default_flow_style=False) 43 | 44 | if __name__ == "__main__": 45 | sys.exit(main(sys.argv)) 46 | 47 | -------------------------------------------------------------------------------- /operator/.gitignore: -------------------------------------------------------------------------------- 1 | # Temporary Build Files 2 | build/_output 3 | build/_test 4 | # Created by https://www.gitignore.io/api/go,vim,emacs,visualstudiocode 5 | ### Emacs ### 6 | # -*- mode: gitignore; -*- 7 | *~ 8 | \#*\# 9 | /.emacs.desktop 10 | /.emacs.desktop.lock 11 | *.elc 12 | auto-save-list 13 | tramp 14 | .\#* 15 | # Org-mode 16 | .org-id-locations 17 | *_archive 18 | # flymake-mode 19 | *_flymake.* 20 | # eshell files 21 | /eshell/history 22 | /eshell/lastdir 23 | # elpa packages 24 | /elpa/ 25 | # reftex files 26 | *.rel 27 | # AUCTeX auto folder 28 | /auto/ 29 | # cask packages 30 | .cask/ 31 | dist/ 32 | # Flycheck 33 | flycheck_*.el 34 | # server auth directory 35 | /server/ 36 | # projectiles files 37 | .projectile 38 | projectile-bookmarks.eld 39 | # directory configuration 40 | .dir-locals.el 41 | # saveplace 42 | places 43 | # url cache 44 | url/cache/ 45 | # cedet 46 | ede-projects.el 47 | # smex 48 | smex-items 49 | # company-statistics 50 | company-statistics-cache.el 51 | # anaconda-mode 52 | anaconda-mode/ 53 | ### Go ### 54 | # Binaries for programs and plugins 55 | *.exe 56 | *.exe~ 57 | *.dll 58 | *.so 59 | *.dylib 60 | # Test binary, build with 'go test -c' 61 | *.test 62 | # Output of the go coverage tool, specifically when used with LiteIDE 63 | *.out 64 | ### Vim ### 65 | # swap 66 | .sw[a-p] 67 | .*.sw[a-p] 68 | # session 69 | Session.vim 70 | # temporary 71 | .netrwhist 72 | # auto-generated tag files 73 | tags 74 | ### VisualStudioCode ### 75 | .vscode/* 76 | .history 77 | # End of https://www.gitignore.io/api/go,vim,emacs,visualstudiocode 78 | vendor 79 | -------------------------------------------------------------------------------- /driver/go.mod: -------------------------------------------------------------------------------- 1 | module github.com/IBM/ibm-spectrum-scale-csi/driver 2 | 3 | go 1.24.0 4 | 5 | require ( 6 | github.com/container-storage-interface/spec v1.11.0 7 | github.com/google/uuid v1.6.0 8 | github.com/natefinch/lumberjack v2.0.0+incompatible 9 | golang.org/x/net v0.46.0 10 | golang.org/x/sys v0.37.0 11 | google.golang.org/grpc v1.76.0 12 | google.golang.org/protobuf v1.36.7 13 | k8s.io/api v0.33.2 14 | k8s.io/klog/v2 v2.130.1 15 | k8s.io/mount-utils v0.33.2 16 | ) 17 | 18 | require ( 19 | github.com/BurntSushi/toml v1.3.2 // indirect 20 | github.com/fxamacker/cbor/v2 v2.7.0 // indirect 21 | github.com/go-logr/logr v1.4.3 // indirect 22 | github.com/gogo/protobuf v1.3.2 // indirect 23 | github.com/json-iterator/go v1.1.12 // indirect 24 | github.com/kr/text v0.2.0 // indirect 25 | github.com/moby/sys/mountinfo v0.7.2 // indirect 26 | github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect 27 | github.com/modern-go/reflect2 v1.0.2 // indirect 28 | github.com/x448/float16 v0.8.4 // indirect 29 | golang.org/x/text v0.30.0 // indirect 30 | google.golang.org/genproto/googleapis/rpc v0.0.0-20250804133106-a7a43d27e69b // indirect 31 | gopkg.in/inf.v0 v0.9.1 // indirect 32 | gopkg.in/natefinch/lumberjack.v2 v2.0.0 // indirect 33 | gopkg.in/yaml.v2 v2.4.0 // indirect 34 | k8s.io/apimachinery v0.33.2 // indirect 35 | k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 // indirect 36 | sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 // indirect 37 | sigs.k8s.io/randfill v1.0.0 // indirect 38 | sigs.k8s.io/structured-merge-diff/v4 v4.6.0 // indirect 39 | sigs.k8s.io/yaml v1.4.0 // indirect 40 | ) 41 | -------------------------------------------------------------------------------- /tools/ansible/common/dev-env.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Set environment facts 3 | set_fact: 4 | OPERATOR_SDK_VER: "v1.36.1" 5 | OPERATOR_VERSION: "3.1.0" 6 | GO_VERSION: "go1.22" 7 | 8 | # Something is wrong with this bit. 9 | #- name: Ensure 'python3' is installed 10 | # package: 11 | # name: "{{packages}}" 12 | # vars: 13 | # packages: 14 | # - python36 15 | # - python36-pip 16 | 17 | #- name: Ensure 'python3' requirements are installed 18 | # pip: 19 | # requirements: "{{playbook_dir}}/common/requirements-dev.txt" 20 | # executable: pip3 21 | 22 | #- name: Ensure that the docker is installed 23 | # package: 24 | # name: "{{ item }}" 25 | # state: present 26 | # with_items: 27 | # - docker 28 | # 29 | #- name: Ensure 'operator-sdk' is installed 30 | # get_url: 31 | # url: "https://github.com/operator-framework/operator-sdk/releases/download/{{release}}/operator-sdk-{{release}}-x86_64-linux-gnu" 32 | # dest: /usr/local/bin/operator-sdk 33 | # mode: '0755' 34 | # vars: 35 | # release: "{{OPERATOR_SDK_VER}}" 36 | 37 | #- name: "Ensure '{{GO_VERSION}}' is installed; Gather Facts" 38 | # shell: "go version | grep -q {{GO_VERSION}}" 39 | # register: go_version 40 | # changed_when: false 41 | # 42 | #- name: "Ensure '{{GO_VERSION}}' is installed; Install" 43 | # unarchive: 44 | # src: "https://dl.google.com/go/go1.13.1.linux-amd64.tar.gz" 45 | # dest: /usr/local 46 | # remote_src: yes 47 | # when: go_version.rc != 0 48 | # 49 | # 50 | #- name: Expose Go to the global path 51 | # copy: 52 | # dest: /etc/profile.d/custom-path.sh 53 | # content: 'PATH=$PATH:/usr/local/go/bin' 54 | -------------------------------------------------------------------------------- /operator/hacks/csv_copy_cr.py: -------------------------------------------------------------------------------- 1 | #!/bin/python 2 | 3 | import argparse 4 | import sys 5 | import os 6 | import yaml 7 | import json 8 | 9 | BASE_DIR="{0}/../".format(os.path.dirname(os.path.realpath(__file__))) 10 | DEFAULT_VERSION="2.6.0" 11 | CSV_PATH="{0}deploy/olm-catalog/ibm-spectrum-scale-csi-operator/{1}/ibm-spectrum-scale-csi-operator.v{1}.clusterserviceversion.yaml" 12 | CR="{0}/deploy/crds/{1}" 13 | 14 | def main(args): 15 | parser = argparse.ArgumentParser( 16 | description='''A hack to copy commented CRS into the CSV.''') 17 | 18 | parser.add_argument( '--cr', metavar='cr', dest='cr', default=None, 19 | help='''The Custom Resource File.''') 20 | 21 | parser.add_argument( '--version', metavar='CSV Version', dest='version', default=DEFAULT_VERSION, 22 | help='''The version of the CSV to update''') 23 | 24 | 25 | args = parser.parse_args() 26 | 27 | crf=CR.format(BASE_DIR,args.cr) 28 | csvf = CSV_PATH.format(BASE_DIR, args.version) 29 | 30 | csv = None 31 | cr = None 32 | try: 33 | with open(crf, 'r') as stream: 34 | cr = yaml.safe_load(stream) 35 | with open(csvf, 'r') as stream: 36 | csv = yaml.safe_load(stream) 37 | except yaml.YAMLError as e: 38 | print(e) 39 | return 1 40 | 41 | # Remove namespace from the CR and update CSV 42 | if cr is not None and csv is not None: 43 | annotations = csv.get("metadata",{}).get("annotations",{}) 44 | cr.get("metadata",{}).pop("namespace", None) 45 | annotations["alm-examples"] = json.dumps(cr) 46 | 47 | with open(csvf, 'w') as outfile: 48 | yaml.dump(csv, outfile, default_flow_style=False) 49 | 50 | if __name__ == "__main__": 51 | sys.exit(main(sys.argv)) 52 | 53 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Temporary Build Files 2 | build 3 | build/_output 4 | build/_test 5 | .DS_Store 6 | # Travis Validation Files 7 | generated/installer/*.test.yaml 8 | # Tools 9 | tools/ibm-spectrum-scale-csi-operator* 10 | # Created by https://www.gitignore.io/api/go,vim,emacs,visualstudiocode 11 | ### Emacs ### 12 | # -*- mode: gitignore; -*- 13 | *~ 14 | \#*\# 15 | /.emacs.desktop 16 | /.emacs.desktop.lock 17 | *.elc 18 | auto-save-list 19 | tramp 20 | .\#* 21 | # Org-mode 22 | .org-id-locations 23 | *_archive 24 | # flymake-mode 25 | *_flymake.* 26 | # eshell files 27 | /eshell/history 28 | /eshell/lastdir 29 | # elpa packages 30 | /elpa/ 31 | # reftex files 32 | *.rel 33 | # AUCTeX auto folder 34 | /auto/ 35 | # cask packages 36 | .cask/ 37 | dist/ 38 | # Flycheck 39 | flycheck_*.el 40 | # server auth directory 41 | /server/ 42 | # projectiles files 43 | .projectile 44 | projectile-bookmarks.eld 45 | # directory configuration 46 | .dir-locals.el 47 | # saveplace 48 | places 49 | # url cache 50 | url/cache/ 51 | # cedet 52 | ede-projects.el 53 | # smex 54 | smex-items 55 | # company-statistics 56 | company-statistics-cache.el 57 | # anaconda-mode 58 | anaconda-mode/ 59 | ### Go ### 60 | # Binaries for programs and plugins 61 | *.exe 62 | *.exe~ 63 | *.dll 64 | *.so 65 | *.dylib 66 | # Test binary, build with 'go test -c' 67 | *.test 68 | # Output of the go coverage tool, specifically when used with LiteIDE 69 | *.out 70 | ### Vim ### 71 | # swap 72 | .sw[a-p] 73 | .*.sw[a-p] 74 | # session 75 | Session.vim 76 | # temporary 77 | .netrwhist 78 | # auto-generated tag files 79 | tags 80 | ### VisualStudioCode ### 81 | .vscode/* 82 | .history 83 | # End of https://www.gitignore.io/api/go,vim,emacs,visualstudiocode 84 | vendor 85 | _output 86 | operator/bin/* 87 | -------------------------------------------------------------------------------- /.travis-pri.yml: -------------------------------------------------------------------------------- 1 | --- 2 | dist: xenial 3 | sudo: required 4 | group: bluezone 5 | language: generic 6 | 7 | # Environment variables (setup endpoint to IBM github) 8 | env: 9 | global: 10 | - OCTOKIT_API_ENDPOINT="https://github.ibm.com/api/v3/" 11 | 12 | before_install: 13 | # Clone build tools & set tools path for follow on processing 14 | # The clone command will use the ssh key from the travis settings to clone the repo from github.ibm.com 15 | - if [[ ! -z "$BRANCH_OVERRIDE" ]] ; then 16 | git clone -b $BRANCH_OVERRIDE git@github.ibm.com:ibmprivatecloud/content-tools; 17 | else 18 | git clone git@github.ibm.com:ibmprivatecloud/content-tools; 19 | fi 20 | - export toolsPath=`pwd`/content-tools/travis-tools/ 21 | # Install dependencies & determine chart delta 22 | - $toolsPath/build/bin/installDependencies.sh 23 | - export changeList=`$toolsPath/build/bin/determineChartDelta.sh | tee determineChartDelta.out | grep 'return determineChartDelta:' | cut -f2 -d:` && cat determineChartDelta.out 24 | 25 | install: 26 | # Package for release 27 | - if [[ ! -z "$TRAVIS_TAG" ]] ; then $toolsPath/release/bin/package.sh; fi 28 | # Lint and install/test charts (if cv-tests exist) 29 | - $toolsPath/cv-test/bin/validateContent.sh 30 | 31 | script: echo "Override default script" 32 | 33 | deploy: 34 | # scp helm repo(s) to location identified (Note: SSHPASS env variable must contain password) 35 | - provider: script 36 | skip_cleanup: true 37 | script: $toolsPath/build/bin/deployHelmRepo.sh 38 | on: 39 | all_branches: true 40 | # Publish tagged release 41 | - provider: releases 42 | skip_cleanup: true 43 | api_key: $GITHUB_TOKEN 44 | file_glob: true 45 | file: repo/stable/* 46 | on: 47 | tags: true 48 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | ############################################################################### 2 | # Licensed Materials - Property of IBM. 3 | # Copyright IBM Corporation 2017, 2024. All Rights Reserved. 4 | # U.S. Government Users Restricted Rights - Use, duplication or disclosure 5 | # restricted by GSA ADP Schedule Contract with IBM Corp. 6 | # 7 | # Contributors: 8 | # IBM Corporation - initial API and implementation 9 | ############################################################################### 10 | 11 | SHELL = /bin/bash 12 | STABLE_BUILD_DIR = repo/stable 13 | STABLE_REPO_URL ?= https://raw.githubusercontent.com/IBM/charts/master/repo/stable/ 14 | STABLE_CHARTS := $(wildcard stable/*) 15 | BUNDLE_HELM_PACKAGE = $(shell find $@/charts -maxdepth 2 -type f -name Chart.yaml | sed 's/Chart.yaml//' | xargs -i helm package {} -d $(STABLE_BUILD_DIR)) 16 | 17 | .DEFAULT_GOAL=all 18 | 19 | $(STABLE_BUILD_DIR): 20 | @mkdir -p $@ 21 | 22 | .PHONY: charts charts-stable $(STABLE_CHARTS) 23 | 24 | # Default aliases: charts, repo 25 | 26 | charts: charts-stable 27 | 28 | repo: repo-stable 29 | 30 | charts-stable: $(STABLE_CHARTS) 31 | $(STABLE_CHARTS): $(STABLE_BUILD_DIR) 32 | cv lint ibmcase-bundle $@ 33 | @echo $(BUNDLE_HELM_PACKAGE) 34 | 35 | .PHONY: repo repo-stable repo-incubating 36 | 37 | repo-stable: $(STABLE_CHARTS) $(STABLE_BUILD_DIR) 38 | helm repo index $(STABLE_BUILD_DIR) --url $(STABLE_REPO_URL) 39 | 40 | .PHONY: all 41 | all: repo-stable build-driver-image 42 | 43 | # 44 | # CSI Driver section 45 | # 46 | IMAGE_VERSION=v2.4.0 47 | DRIVER_IMAGE_NAME=ibm-spectrum-scale-csi-driver 48 | 49 | build-driver-image: 50 | docker build -t $(DRIVER_IMAGE_NAME):$(IMAGE_VERSION) -f ./driver/build/Dockerfile ./driver/ 51 | 52 | save-driver-image: build-image 53 | docker save $(DRIVER_IMAGE_NAME):$(IMAGE_VERSION) -o _output/$(DRIVER_IMAGE_NAME)_$(IMAGE_VERSION).tar 54 | -------------------------------------------------------------------------------- /operator/config/rbac/role.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: rbac.authorization.k8s.io/v1 3 | kind: ClusterRole 4 | metadata: 5 | name: ibm-spectrum-scale-csi-operator 6 | rules: 7 | - apiGroups: 8 | - "" 9 | resources: 10 | - configmaps 11 | - endpoints 12 | - events 13 | - persistentvolumeclaims 14 | - pods 15 | - secrets 16 | - secrets/status 17 | - serviceaccounts 18 | - services 19 | - services/finalizers 20 | verbs: 21 | - '*' 22 | - apiGroups: 23 | - apps 24 | resources: 25 | - daemonsets 26 | - deployments 27 | - replicasets 28 | - statefulsets 29 | verbs: 30 | - create 31 | - delete 32 | - get 33 | - list 34 | - update 35 | - watch 36 | - apiGroups: 37 | - apps 38 | resourceNames: 39 | - ibm-spectrum-scale-csi-operator 40 | resources: 41 | - deployments/finalizers 42 | verbs: 43 | - get 44 | - update 45 | - apiGroups: 46 | - config.openshift.io 47 | resources: 48 | - clusterversions 49 | verbs: 50 | - get 51 | - list 52 | - watch 53 | - apiGroups: 54 | - coordination.k8s.io 55 | resources: 56 | - leases 57 | verbs: 58 | - create 59 | - delete 60 | - get 61 | - list 62 | - update 63 | - watch 64 | - apiGroups: 65 | - csi.ibm.com 66 | resources: 67 | - '*' 68 | verbs: 69 | - '*' 70 | - apiGroups: 71 | - monitoring.coreos.com 72 | resources: 73 | - servicemonitors 74 | verbs: 75 | - create 76 | - get 77 | - apiGroups: 78 | - rbac.authorization.k8s.io 79 | resources: 80 | - clusterrolebindings 81 | - clusterroles 82 | verbs: 83 | - '*' 84 | - apiGroups: 85 | - security.openshift.io 86 | resources: 87 | - securitycontextconstraints 88 | verbs: 89 | - '*' 90 | - apiGroups: 91 | - storage.k8s.io 92 | resources: 93 | - csidrivers 94 | - storageclasses 95 | - volumeattachments 96 | verbs: 97 | - create 98 | - delete 99 | - get 100 | - list 101 | - patch 102 | - update 103 | - watch 104 | -------------------------------------------------------------------------------- /tools/ansible/versioning-playbook.yaml: -------------------------------------------------------------------------------- 1 | - name: Update Version 2 | hosts: localhost 3 | become: yes 4 | gather_facts: true 5 | vars: 6 | VERSION_OLD: "2.3.1" 7 | VERSION_NEW: "2.4.0" 8 | OPERATOR_DIR: "../../operator/" 9 | OLM_DIR: "config/olm-catalog/ibm-spectrum-scale-csi-operator/" 10 | OPERATOR_NAME: "ibm-spectrum-scale-csi-operator" 11 | tasks: 12 | 13 | - set_fact: 14 | CSV_OLD: "{{OLM_DIR}}{{VERSION_NEW}}/ibm-spectrum-scale-csi-operator.v{{VERSION_OLD}}.clusterserviceversion.yaml" 15 | CSV_NEW: "{{OLM_DIR}}{{VERSION_NEW}}/ibm-spectrum-scale-csi-operator.v{{VERSION_NEW}}.clusterserviceversion.yaml" 16 | 17 | - name: "Copy the CSV {{VERSION_OLD}} to {{VERSION_NEW}}" 18 | copy: 19 | src: "{{OPERATOR_DIR}}/{{OLM_DIR}}{{VERSION_OLD}}/" 20 | dest: "{{OPERATOR_DIR}}/{{OLM_DIR}}{{VERSION_NEW}}" 21 | 22 | - name: "Rename CSV" 23 | copy: 24 | src: "{{OPERATOR_DIR}}/{{CSV_OLD}}" 25 | dest: "{{OPERATOR_DIR}}/{{CSV_NEW}}" 26 | 27 | - name: "Remove invalid CSV" 28 | file: 29 | state: absent 30 | path: "{{OPERATOR_DIR}}/{{CSV_OLD}}" 31 | 32 | - name: "Bump Version" 33 | replace: 34 | path: "{{OPERATOR_DIR}}/{{item}}" 35 | regexp: "{{VERSION_OLD}}" 36 | replace: "{{VERSION_NEW}}" 37 | loop: 38 | - "{{CSV_NEW}}" 39 | - "{{OLM_DIR}}/ibm-spectrum-scale-csi-operator.package.yaml" 40 | - config/manager/manager.yaml 41 | - .osdk-scorecard.yaml 42 | - hacks/package_operator.py 43 | - hacks/csv_copy_cr.py 44 | - hacks/csv_copy_crd_descriptions.py 45 | - hacks/csv_copy_docs.py 46 | - hacks/csv_prep.py 47 | # Driver files. 48 | - ../driver/build/Dockerfile 49 | - ../driver/cmd/ibm-spectrum-scale-csi/main.go 50 | 51 | - name: "Rebuild operator 'ClusterServiceVersion'" 52 | shell: "GO111MODULE=on operator-sdk generate csv --operator-name {{OPERATOR_NAME}} --csv-version {{VERSION_NEW}} --update-crds" 53 | args: 54 | chdir: "{{OPERATOR_DIR}}" 55 | 56 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # IBM Storage Scale CSI (Container Storage Interface) 2 | 3 | [Official IBM Documentation](https://www.ibm.com/docs/en/spectrum-scale-csi) 4 | 5 | 6 | The IBM Storage Scale Container Storage Interface (CSI) project enables container orchestrators, such as Kubernetes and OpenShift, to manage the life-cycle of persistent storage. 7 | 8 | This project contains an [go-based operator](https://sdk.operatorframework.io) to run and manage the deployment of the IBM Storage Scale CSI Driver. 9 | 10 | 11 | ## Support 12 | 13 | IBM Storage Scale CSI driver is part of the IBM Storage Scale offering. Please follow the [IBM support procedure](https://www.ibm.com/mysupport/s/) for any issues with the driver. 14 | 15 | ## Report Bugs 16 | 17 | For help with urgent situations, please use the IBM PMR process. All IBM Storage Scale customers using CSI, 18 | who also have ongoing support contracts, are entitled to the PMR process. Feature requests through the official RFE channels are also encouraged. 19 | 20 | For non-urgent issues, suggestions, recommendations, feel free to open an issue in [github](https://github.com/IBM/ibm-spectrum-scale-csi/issues). 21 | Issues will be addressed as team availability permits. 22 | 23 | ## Contributing 24 | 25 | We welcome contributions to this project, see [Contributing](CONTRIBUTING.md) for more details. 26 | 27 | ##### Note: This repository includes contributions from [ubiquity](https://github.com/ibm/ubiquity). 28 | 29 | ## Licensing 30 | 31 | Copyright 2022, 2024 IBM Corp. 32 | 33 | Licensed under the Apache License, Version 2.0 (the "License"); 34 | you may not use this file except in compliance with the License. 35 | You may obtain a copy of the License at 36 | 37 | http://www.apache.org/licenses/LICENSE-2.0 38 | 39 | Unless required by applicable law or agreed to in writing, software 40 | distributed under the License is distributed on an "AS IS" BASIS, 41 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 42 | See the License for the specific language governing permissions and 43 | limitations under the License. 44 | -------------------------------------------------------------------------------- /tools/scripts/push_app.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | #================================================================================================== 3 | # Get jq, operator-courier, and helm 4 | which jq > /dev/null 5 | if [ $? -ne 0 ] 6 | then 7 | wget /usr/bin/jq https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 8 | chmod +x jq-linux64 9 | mv jq-linux64 /usr/bin/jq 10 | fi 11 | 12 | helm registry -h >/dev/null 13 | if [ $? -ne 0 ] 14 | then 15 | curl -L https://git.io/get_helm.sh | bash 16 | mkdi r-p ~/.helm/plugins/ 17 | cd ~/.helm/plugins/ && git clone https://github.com/app-registry/appr-helm-plugin.git registry 18 | fi 19 | 20 | operator-courier -h >>/dev/null 21 | if [ $? -ne 0 ] 22 | then 23 | pip3 install operator-courier 24 | fi 25 | #================================================================================================== 26 | # This script takes OPERATOR_DIR, QUAY_NAMESPACE, and PACKAGE_NAME as input to the script. 27 | 28 | # Set values 29 | export OPERATOR_DIR=${OPERATOR_DIR:-operator/deploy/olm-catalog/ibm-spectrum-scale-csi-operator} 30 | export QUAY_NAMESPACE=${QUAY_NAMESPACE:-ibm-spectrum-scale-dev} 31 | export PACKAGE_NAME=${PACKAGE_NAME:-ibm-spectrum-scale-csi-app} 32 | export PACKAGE_VERSION=$(helm registry list quay.io -o ${QUAY_NAMESPACE} --output json | jq --arg NAME "${QUAY_NAMESPACE}/${PACKAGE_NAME}" '.[] | select(.name == $NAME) |.default' | awk -F'[ .\"]' '{print $2"."$3"."$4+1""}') 33 | export TOKEN=$(curl -sH "Content-Type: application/json" -XPOST https://quay.io/cnr/api/v1/users/login -d ' 34 | {"user": {"username": "'"${QUAY_USERNAME}"'","password": "'"${QUAY_PASSWORD}"'"}}' | cut -d'"' -f4) 35 | export PACKAGE_FILE=${OPERATOR_DIR}/ibm-spectrum-scale-csi-operator.package.yaml 36 | 37 | # Rename the package 38 | sed -i "s/packageName: ibm-spectrum-scale-csi-operator/packageName: ${PACKAGE_NAME}/g" ${PACKAGE_FILE} 39 | 40 | operator-courier push "$OPERATOR_DIR" "$QUAY_NAMESPACE" "$PACKAGE_NAME" "$PACKAGE_VERSION" "$TOKEN" 41 | 42 | # Reset the package. 43 | sed -i "s/packageName: ${PACKAGE_NAME}/packageName: ibm-spectrum-scale-csi-operator/g" ${PACKAGE_FILE} 44 | -------------------------------------------------------------------------------- /operator/build/Dockerfile: -------------------------------------------------------------------------------- 1 | # docker build for IBM Storage Scale CSI Operator 2 | FROM --platform=$BUILDPLATFORM golang:1.24 as builder 3 | 4 | ARG TARGETOS 5 | ARG TARGETARCH 6 | ARG commit 7 | 8 | WORKDIR /workspace 9 | # Copy the Go Modules manifests 10 | COPY go.mod go.mod 11 | COPY go.sum go.sum 12 | # cache deps before building and copying source so that we don't need to re-download as much 13 | # and so that source changes don't invalidate our downloaded layer 14 | RUN go mod download 15 | 16 | # Copy the go source 17 | COPY main.go main.go 18 | COPY hacks/health_check.go health_check.go 19 | COPY api/ api/ 20 | COPY controllers/ controllers/ 21 | 22 | # Set GOENV to target OS and ARCH 23 | ENV REVISION $commit 24 | ENV GOOS $TARGETOS 25 | ENV GOARCH $TARGETARCH 26 | 27 | # Build dummy health checker 28 | RUN CGO_ENABLED=0 go build -ldflags="-X 'main.gitCommit=${REVISION}'" -a -o health_check.sh health_check.go 29 | # Build CSI Operator 30 | RUN CGO_ENABLED=0 go build -ldflags="-X 'main.gitCommit=${REVISION}'" -a -o manager main.go 31 | 32 | # Use distroless as minimal base image to package the manager binary 33 | # Refer to https://github.com/GoogleContainerTools/distroless for more details 34 | FROM gcr.io/distroless/static:nonroot 35 | WORKDIR / 36 | ARG version=3.1.0 37 | ARG commit 38 | ARG build_date 39 | 40 | ENV version $version 41 | ENV commit $commit 42 | ENV build_date $build_date 43 | 44 | LABEL name="IBM Storage Scale CSI Operator" \ 45 | vendor="ibm" \ 46 | version=$version \ 47 | release="1" \ 48 | summary="A Go based operator to run and manage the deployment of the IBM Storage Scale CSI Driver." \ 49 | description="A Go based operator to run and manage the deployment of the IBM Storage Scale CSI Driver." \ 50 | maintainer="IBM Storage Scale" \ 51 | build-date=$build_date \ 52 | vcs-ref=$commit \ 53 | vcs-type="git" \ 54 | url="https://www.ibm.com/docs/en/spectrum-scale-csi" 55 | 56 | COPY licenses /licenses 57 | COPY --from=builder --chmod=0771 /workspace/health_check.sh . 58 | COPY --from=builder /workspace/manager . 59 | USER 65532:65532 60 | 61 | ENTRYPOINT ["/manager"] 62 | -------------------------------------------------------------------------------- /tools/ansible/generate/create_files.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: "delete the directory {{ generated_dir }}" 4 | file: 5 | path: "{{ generated_dir }}" 6 | state: absent 7 | when: not travis_testing 8 | 9 | - name: "create the directory {{ generated_dir }}" 10 | file: 11 | path: "{{ generated_dir }}" 12 | state: directory 13 | mode: '0755' 14 | 15 | - name: "Combining the individual deploy/*yaml files into a single one: {{ generated_op_yaml }}" 16 | shell: "kustomize build --reorder none {{ operator_dir }}/config/overlays/default > {{ generated_op_yaml }} && kustomize cfg fmt {{ generated_op_yaml }}" 17 | ignore_errors: False 18 | 19 | - name: "Copy generated file to dev file" 20 | copy: 21 | src: "{{ generated_op_yaml }}" 22 | dest: "{{ generated_op_dev_yaml }}" 23 | 24 | - name: "Replace ibm-spectrum-scale/ with ibm-spectrum-scale-dev/" 25 | replace: 26 | path: "{{ generated_op_dev_yaml }}" 27 | regexp: "ibm-spectrum-scale/" 28 | replace: "ibm-spectrum-scale-dev/" 29 | 30 | - name: "IfNotPresent with Always" 31 | replace: 32 | path: "{{ generated_op_dev_yaml }}" 33 | regexp: "IfNotPresent" 34 | replace: "Always" 35 | 36 | - name: "Replace the image version with dev" 37 | replace: 38 | path: "{{ generated_op_dev_yaml }}" 39 | regexp: 'quay.io/ibm-spectrum-scale-dev/ibm-spectrum-scale-csi-([^:]*):.+' 40 | replace: 'quay.io/ibm-spectrum-scale-dev/ibm-spectrum-scale-csi-\1:dev' 41 | 42 | - name: "[Testing] Comparing the generated operator file with checked in code ..." 43 | shell: "diff {{ generated_dir }}/ibm-spectrum-scale-csi-operator.test.yaml {{ generated_dir }}/ibm-spectrum-scale-csi-operator.yaml" 44 | ignore_errors: yes 45 | register: diff_output 46 | when: travis_testing|bool 47 | 48 | - name: "[Testing] Diff standard out" 49 | debug: 50 | msg: "{{ diff_output.stdout }}" 51 | when: travis_testing|bool 52 | 53 | - name: "[Testing] Diff standard error" 54 | debug: 55 | msg: "{{ diff_output.stderr }}" 56 | when: travis_testing|bool 57 | 58 | - name: "[Testing] Failure Message" 59 | fail: 60 | msg: "The generated ibm-spectrum-scale-csi-operator.yaml did not match the deploy yaml files, did you forget to check it in?" 61 | when: 62 | - travis_testing|bool 63 | - "diff_output.rc != 0" 64 | -------------------------------------------------------------------------------- /driver/build/Dockerfile: -------------------------------------------------------------------------------- 1 | # docker build for IBM Storage Scale CSI Driver 2 | # usage: docker build -f build/Dockerfile -t my_image_tag . 3 | # Multi-arch build for IBM Storage Scale CSI Driver 4 | # usage: docker buildx build -f build/multi-arch.Dockerfile --platform=linux/amd64 -t my_image_tag . 5 | 6 | FROM --platform=$BUILDPLATFORM golang:1.24 AS builder 7 | WORKDIR /go/src/github.com/IBM/ibm-spectrum-scale-csi/driver/ 8 | COPY ./go.mod . 9 | COPY ./go.sum . 10 | RUN go mod download 11 | 12 | COPY . . 13 | ARG TARGETOS 14 | ARG TARGETARCH 15 | ARG GOFLAGS 16 | ARG commit 17 | 18 | ENV REVISION $commit 19 | 20 | RUN CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -a -ldflags "-X 'main.gitCommit=${REVISION}' -extldflags '-static'" -o _output/ibm-spectrum-scale-csi ./cmd/ibm-spectrum-scale-csi 21 | # RUN chmod +x,u+s _output/ibm-spectrum-scale-csi 22 | 23 | FROM registry.access.redhat.com/ubi10-minimal:10.0-1758699349 24 | ARG version=3.1.0 25 | ARG commit 26 | ARG build_date 27 | 28 | ENV version $version 29 | ENV commit $commit 30 | ENV build_date $build_date 31 | 32 | LABEL name="IBM Storage Scale CSI driver" \ 33 | vendor="ibm" \ 34 | version=$version \ 35 | release="1" \ 36 | summary="An implementation of CSI Plugin for the IBM Storage Scale product." \ 37 | description="An implementation of CSI Plugin for the IBM Storage Scale product." \ 38 | maintainer="IBM Storage Scale" \ 39 | build-date=$build_date \ 40 | vcs-ref=$commit \ 41 | vcs-type="git" \ 42 | url="https://www.ibm.com/docs/en/spectrum-scale-csi" 43 | 44 | COPY licenses /licenses 45 | COPY --from=builder /go/src/github.com/IBM/ibm-spectrum-scale-csi/driver/_output/ibm-spectrum-scale-csi /ibm-spectrum-scale-csi 46 | 47 | ## non-root user 48 | # RUN groupadd -g 100006 default 49 | # RUN useradd -u 100006 -g default -r default 50 | # USER default 51 | 52 | # RUN microdnf update -y \ 53 | # && microdnf install util-linux 54 | 55 | RUN mkdir /chroot 56 | COPY chroot-wrapper.sh /chroot 57 | RUN chmod +x /chroot/chroot-wrapper.sh 58 | RUN ln -s /chroot/chroot-wrapper.sh /chroot/mount 59 | RUN ln -s /chroot/chroot-wrapper.sh /chroot/umount 60 | 61 | ENV PATH="/chroot:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" 62 | 63 | ENTRYPOINT ["/ibm-spectrum-scale-csi"] 64 | -------------------------------------------------------------------------------- /operator/docs/dev/security/results.gosec.json: -------------------------------------------------------------------------------- 1 | {"Issues":[],"Report":{"Linters":[{"Name":"asasalint"},{"Name":"asciicheck"},{"Name":"bidichk"},{"Name":"bodyclose"},{"Name":"containedctx"},{"Name":"contextcheck"},{"Name":"cyclop"},{"Name":"decorder"},{"Name":"deadcode"},{"Name":"depguard"},{"Name":"dogsled"},{"Name":"dupl"},{"Name":"dupword"},{"Name":"durationcheck"},{"Name":"errcheck","EnabledByDefault":true},{"Name":"errchkjson"},{"Name":"errname"},{"Name":"errorlint"},{"Name":"execinquery"},{"Name":"exhaustive"},{"Name":"exhaustivestruct"},{"Name":"exhaustruct"},{"Name":"exportloopref"},{"Name":"forbidigo"},{"Name":"forcetypeassert"},{"Name":"funlen"},{"Name":"gci"},{"Name":"gochecknoglobals"},{"Name":"gochecknoinits"},{"Name":"gocognit"},{"Name":"goconst"},{"Name":"gocritic"},{"Name":"gocyclo"},{"Name":"godot"},{"Name":"godox"},{"Name":"goerr113"},{"Name":"gofmt"},{"Name":"gofumpt"},{"Name":"goheader"},{"Name":"goimports"},{"Name":"golint"},{"Name":"gomnd"},{"Name":"gomoddirectives"},{"Name":"gomodguard"},{"Name":"goprintffuncname"},{"Name":"gosec","Enabled":true},{"Name":"gosimple","EnabledByDefault":true},{"Name":"govet","EnabledByDefault":true},{"Name":"grouper"},{"Name":"ifshort"},{"Name":"importas"},{"Name":"ineffassign","EnabledByDefault":true},{"Name":"interfacebloat"},{"Name":"interfacer"},{"Name":"ireturn"},{"Name":"lll"},{"Name":"loggercheck"},{"Name":"maintidx"},{"Name":"makezero"},{"Name":"maligned"},{"Name":"misspell"},{"Name":"nakedret"},{"Name":"nestif"},{"Name":"nilerr"},{"Name":"nilnil"},{"Name":"nlreturn"},{"Name":"noctx"},{"Name":"nonamedreturns"},{"Name":"nosnakecase"},{"Name":"nosprintfhostport"},{"Name":"paralleltest"},{"Name":"prealloc"},{"Name":"predeclared"},{"Name":"promlinter"},{"Name":"reassign"},{"Name":"revive"},{"Name":"rowserrcheck"},{"Name":"scopelint"},{"Name":"sqlclosecheck"},{"Name":"staticcheck","EnabledByDefault":true},{"Name":"structcheck"},{"Name":"stylecheck"},{"Name":"tagliatelle"},{"Name":"tenv"},{"Name":"testableexamples"},{"Name":"testpackage"},{"Name":"thelper"},{"Name":"tparallel"},{"Name":"typecheck","EnabledByDefault":true},{"Name":"unconvert"},{"Name":"unparam"},{"Name":"unused","EnabledByDefault":true},{"Name":"usestdlibvars"},{"Name":"varcheck"},{"Name":"varnamelen"},{"Name":"wastedassign"},{"Name":"whitespace"},{"Name":"wrapcheck"},{"Name":"wsl"},{"Name":"nolintlint"}]}} 2 | -------------------------------------------------------------------------------- /operator/docs/dev/security/results.golint.txt: -------------------------------------------------------------------------------- 1 | level=warning msg="[runner] The linter 'golint' is deprecated (since v1.41.0) due to: The repository of the linter has been archived by the owner. Replaced by revive." 2 | api/v1/csiscaleoperator_types.go:316:2: struct field `Id` should be `ID` (golint) 3 | Id string `json:"id"` // TODO: Rename to ID or id 4 | ^ 5 | api/v1/csiscaleoperator_types.go:324:2: struct field `RestApi` should be `RestAPI` (golint) 6 | RestApi []RestApi `json:"restApi"` // TODO: Rename to RESTApi or restApi 7 | ^ 8 | api/v1/csiscaleoperator_types.go:354:6: type `RestApi` should be `RestAPI` (golint) 9 | type RestApi struct { 10 | ^ 11 | controllers/csiscaleoperator_controller.go:266:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (golint) 12 | } else { 13 | ^ 14 | controllers/internal/csiscaleoperator/csiscaleoperator_package.go:33:2: const `snapshotStorageApiGroup` should be `snapshotStorageAPIGroup` (golint) 15 | snapshotStorageApiGroup string = "snapshot.storage.k8s.io" 16 | ^ 17 | controllers/internal/csiscaleoperator/csiscaleoperator_package.go:34:2: const `securityOpenshiftApiGroup` should be `securityOpenshiftAPIGroup` (golint) 18 | securityOpenshiftApiGroup string = "security.openshift.io" 19 | ^ 20 | controllers/internal/csiscaleoperator/csiscaleoperator_package.go:35:2: const `storageApiGroup` should be `storageAPIGroup` (golint) 21 | storageApiGroup string = "storage.k8s.io" 22 | ^ 23 | controllers/internal/csiscaleoperator/csiscaleoperator_package.go:36:2: const `rbacAuthorizationApiGroup` should be `rbacAuthorizationAPIGroup` (golint) 24 | rbacAuthorizationApiGroup string = "rbac.authorization.k8s.io" 25 | ^ 26 | controllers/internal/csiscaleoperator/csiscaleoperator_package.go:37:2: const `coordinationApiGroup` should be `coordinationAPIGroup` (golint) 27 | coordinationApiGroup string = "coordination.k8s.io" 28 | ^ 29 | controllers/internal/csiscaleoperator/csiscaleoperator_package.go:38:2: const `podSecurityPolicyApiGroup` should be `podSecurityPolicyAPIGroup` (golint) 30 | podSecurityPolicyApiGroup string = "extensions" 31 | ^ 32 | main.go:81:10: `if` block ends with a `return` statement, so drop this `else` and outdent its block (golint) 33 | } else { 34 | ^ 35 | -------------------------------------------------------------------------------- /operator/controllers/suite_test.go: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright 2022, 2024 IBM Corp. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"); 5 | you may not use this file except in compliance with the License. 6 | You may obtain a copy of the License at 7 | 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | 10 | Unless required by applicable law or agreed to in writing, software 11 | distributed under the License is distributed on an "AS IS" BASIS, 12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | See the License for the specific language governing permissions and 14 | limitations under the License. 15 | */ 16 | 17 | package controllers 18 | 19 | import ( 20 | "path/filepath" 21 | "testing" 22 | 23 | . "github.com/onsi/ginkgo" 24 | . "github.com/onsi/gomega" 25 | "k8s.io/client-go/kubernetes/scheme" 26 | 27 | //"k8s.io/client-go/rest" 28 | "sigs.k8s.io/controller-runtime/pkg/client" 29 | "sigs.k8s.io/controller-runtime/pkg/envtest" 30 | logf "sigs.k8s.io/controller-runtime/pkg/log" 31 | "sigs.k8s.io/controller-runtime/pkg/log/zap" 32 | 33 | csiv1 "github.com/IBM/ibm-spectrum-scale-csi/operator/api/v1" 34 | //+kubebuilder:scaffold:imports 35 | ) 36 | 37 | // These tests use Ginkgo (BDD-style Go testing framework). Refer to 38 | // http://onsi.github.io/ginkgo/ to learn more about Ginkgo. 39 | 40 | // var cfg *rest.Config 41 | var k8sClient client.Client 42 | var testEnv *envtest.Environment 43 | 44 | func TestAPIs(t *testing.T) { 45 | RegisterFailHandler(Fail) 46 | 47 | //RunSpecsWithDefaultAndCustomReporters(t, 48 | // "Controller Suite",) 49 | // // []Reporter{printer.NewlineReporter{}} 50 | } 51 | 52 | var _ = BeforeSuite(func() { 53 | logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true))) 54 | 55 | By("bootstrapping test environment") 56 | testEnv = &envtest.Environment{ 57 | CRDDirectoryPaths: []string{filepath.Join("..", "config", "crd", "bases")}, 58 | ErrorIfCRDPathMissing: true, 59 | } 60 | 61 | cfg, err := testEnv.Start() 62 | Expect(err).NotTo(HaveOccurred()) 63 | Expect(cfg).NotTo(BeNil()) 64 | 65 | err = csiv1.AddToScheme(scheme.Scheme) 66 | Expect(err).NotTo(HaveOccurred()) 67 | 68 | //+kubebuilder:scaffold:scheme 69 | 70 | k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme}) 71 | Expect(err).NotTo(HaveOccurred()) 72 | Expect(k8sClient).NotTo(BeNil()) 73 | 74 | }, 60) 75 | 76 | var _ = AfterSuite(func() { 77 | By("tearing down the test environment") 78 | err := testEnv.Stop() 79 | Expect(err).NotTo(HaveOccurred()) 80 | }) 81 | -------------------------------------------------------------------------------- /.golangci.yml: -------------------------------------------------------------------------------- 1 | run: 2 | timeout: 3m 3 | 4 | linters-settings: 5 | 6 | funlen: 7 | lines: 100 8 | statements: 80 9 | govet: 10 | check-shadowing: false #too many false positives on shadowed err 11 | settings: 12 | printf: 13 | funcs: 14 | - (github.com/golangci/golangci-lint/pkg/logutils.Log).Infof 15 | - (github.com/golangci/golangci-lint/pkg/logutils.Log).Warnf 16 | - (github.com/golangci/golangci-lint/pkg/logutils.Log).Errorf 17 | - (github.com/golangci/golangci-lint/pkg/logutils.Log).Fatalf 18 | golint: 19 | min-confidence: 0 20 | gofmt: 21 | simplify: true 22 | gocyclo: 23 | min-complexity: 20 24 | gocognit: 25 | min-complexity: 10 26 | maligned: 27 | suggest-new: true 28 | dupl: 29 | threshold: 100 30 | goconst: 31 | min-len: 2 32 | min-occurrences: 2 33 | misspell: 34 | locale: US 35 | lll: 36 | line-length: 120 37 | tab-width: 1 38 | nakedret: 39 | max-func-lines: 0 40 | dogsled: 41 | max-blank-identifiers: 2 42 | 43 | linters: 44 | disable-all: true 45 | enable: 46 | - megacheck 47 | - bodyclose 48 | - deadcode 49 | - depguard 50 | - dogsled 51 | - dupl 52 | - errcheck 53 | - funlen 54 | - gochecknoinits 55 | - goconst 56 | - gocyclo 57 | - gofmt 58 | - goimports 59 | - gosec 60 | - gosimple 61 | - govet 62 | - ineffassign 63 | - interfacer 64 | - misspell 65 | - nakedret 66 | - scopelint 67 | - staticcheck 68 | - structcheck 69 | - typecheck 70 | - unconvert 71 | - unparam 72 | - unused 73 | - varcheck 74 | - whitespace 75 | - maligned 76 | #best practice: add --enable=golint,gocritic to your editor configuration when using golangci-lint 77 | 78 | # don't enable: 79 | # - gochecknoglobals 80 | # - gocognit # akin to gocyclo 81 | # - godox # checks for TODO/BUG/FIXME 82 | # - prealloc 83 | # - wsl 84 | # - stylecheck 85 | # - lll # checks line length 86 | # - golint 87 | # - gocritic 88 | 89 | presets: 90 | - bugs 91 | - unused 92 | fast: false 93 | 94 | 95 | issues: 96 | # Excluding configuration per-path, per-linter, per-text and per-source 97 | exclude-rules: 98 | # Exclude some linters from running on tests files. 99 | - path: _test\.go 100 | linters: 101 | - gocyclo 102 | - errcheck 103 | - dupl 104 | - gosec 105 | exclude-use-default: true 106 | max-issues-per-linter: 0 107 | max-same-issues: 0 108 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | Work on the `ibm-spectrum-scale-csi` project should always be performed in a forked copy of the repo, incorporated into the main project using a pull request against the Development (dev) branch. 4 | 5 | When adding features the following process should be followed: 6 | 7 | 1. Make sure you have been approved to contribute code 8 | 2. Fork the repo, create a branch for you feature 9 | 3. Run changes through appropriate linters 10 | 4. Create a Pull Request for your feature 11 | 5. [Sign your work](#sign-your-work-for-submittal) (required) 12 | 13 | ### Sign your work for submittal 14 | 15 | The sign-off is a simple line at the end of the explanation for the patch. Your signature certifies that you wrote the patch or otherwise have the right to pass it on as an open-source patch. The rules are pretty simple: if you can certify the below (from developercertificate.org): 16 | 17 | Developer Certificate of Origin 18 | Version 1.1 19 | 20 | Copyright (C) 2004, 2006 The Linux Foundation and its contributors. 21 | 1 Letterman Drive 22 | Suite D4700 23 | San Francisco, CA, 94129 24 | 25 | Everyone is permitted to copy and distribute verbatim copies of this 26 | license document, but changing it is not allowed. 27 | 28 | Developer's Certificate of Origin 1.1 29 | 30 | By making a contribution to this project, I certify that: 31 | 32 | (a) The contribution was created in whole or in part by me and I 33 |     have the right to submit it under the open source license 34 |     indicated in the file; or 35 | 36 | (b) The contribution is based upon previous work that, to the best 37 |     of my knowledge, is covered under an appropriate open source 38 |     license and I have the right under that license to submit that 39 |     work with modifications, whether created in whole or in part 40 |     by me, under the same open source license (unless I am 41 |     permitted to submit under a different license), as indicated 42 |     in the file; or 43 | 44 | (c) The contribution was provided directly to me by some other 45 |     person who certified (a), (b) or (c) and I have not modified 46 |     it. 47 | 48 | (d) I understand and agree that this project and the contribution 49 |     are public and that a record of the contribution (including all 50 |     personal information I submit with it, including my sign-off) is 51 |     maintained indefinitely and may be redistributed consistent with 52 |     this project or the open source license(s) involved. 53 | Then you just add a line to every git commit message: 54 | 55 | Signed-off-by: Joe Smith 56 | Use your real name (sorry, no pseudonyms or anonymous contributions.) 57 | 58 | If you set your user.name and user.email git configs, you can sign your commit automatically with git commit -s. 59 | 60 | Note: If your git config information is set properly then viewing the git log information for your commit will look something like this: 61 | 62 | Author: Joe Smith 63 | Date:   Thu Feb 2 11:41:15 2018 -0800 64 | 65 |     docs: Update README 66 | 67 |     Signed-off-by: Joe Smith 68 | Notice the Author and Signed-off-by lines match. If they don't your PR will be rejected. 69 | -------------------------------------------------------------------------------- /driver/csiplugin/identityserver.go: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2019, 2024 IBM Corp. 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package scale 18 | 19 | import ( 20 | "context" 21 | 22 | "github.com/IBM/ibm-spectrum-scale-csi/driver/csiplugin/utils" 23 | "github.com/container-storage-interface/spec/lib/go/csi" 24 | "google.golang.org/grpc/codes" 25 | "google.golang.org/grpc/status" 26 | "google.golang.org/protobuf/types/known/wrapperspb" 27 | "k8s.io/klog/v2" 28 | ) 29 | 30 | type ScaleIdentityServer struct { 31 | Driver *ScaleDriver 32 | csi.UnimplementedIdentityServer 33 | } 34 | 35 | func (is *ScaleIdentityServer) GetPluginCapabilities(ctx context.Context, req *csi.GetPluginCapabilitiesRequest) (*csi.GetPluginCapabilitiesResponse, error) { 36 | return &csi.GetPluginCapabilitiesResponse{ 37 | Capabilities: []*csi.PluginCapability{ 38 | { 39 | Type: &csi.PluginCapability_Service_{ 40 | Service: &csi.PluginCapability_Service{ 41 | Type: csi.PluginCapability_Service_CONTROLLER_SERVICE, 42 | }, 43 | }, 44 | }, 45 | }, 46 | }, nil 47 | } 48 | 49 | func (is *ScaleIdentityServer) Probe(ctx context.Context, req *csi.ProbeRequest) (*csi.ProbeResponse, error) { 50 | loggerId := utils.GetLoggerId(ctx) 51 | klog.V(4).Infof("[%s] Probe called with args: %#v", loggerId, req) 52 | // Node mapping check 53 | scalenodeID := getNodeMapping(is.Driver.nodeID) 54 | klog.V(4).Infof("[%s] Probe: scalenodeID:%s --known as-- k8snodeName: %s", loggerId, scalenodeID, is.Driver.nodeID) 55 | // IsNodeComponentHealthy accepts nodeName as admin node name, daemon node name, etc. 56 | conn, ok := is.Driver.connmap["primary"] 57 | if !ok || conn == nil { 58 | klog.Errorf("[%s] Probe: primary connection not available", loggerId) 59 | return &csi.ProbeResponse{Ready: &wrapperspb.BoolValue{Value: true}}, nil 60 | } 61 | ghealthy, err := conn.IsNodeComponentHealthy(ctx, scalenodeID, "GPFS") 62 | if !ghealthy { 63 | // Even gpfs health is unhealthy, success is return because restarting csi driver is not going help fix the issue 64 | klog.Errorf("[%s] Probe: IBM Storage Scale on node %v is unhealthy. Error: %v", loggerId, scalenodeID, err) 65 | return &csi.ProbeResponse{Ready: &wrapperspb.BoolValue{Value: true}}, nil 66 | } 67 | 68 | klog.V(4).Infof("[%s] Probe: IBM Storage Scale on node %v is healthy", loggerId, scalenodeID) 69 | return &csi.ProbeResponse{Ready: &wrapperspb.BoolValue{Value: true}}, nil 70 | } 71 | 72 | func (is *ScaleIdentityServer) GetPluginInfo(ctx context.Context, req *csi.GetPluginInfoRequest) (*csi.GetPluginInfoResponse, error) { 73 | loggerId := utils.GetLoggerId(ctx) 74 | klog.V(4).Infof("[%s] Using default GetPluginInfo", loggerId) 75 | 76 | if is.Driver.name == "" { 77 | return nil, status.Error(codes.Unavailable, "Driver name not configured") 78 | } 79 | 80 | return &csi.GetPluginInfoResponse{ 81 | Name: is.Driver.name, 82 | VendorVersion: is.Driver.vendorVersion, 83 | }, nil 84 | } 85 | -------------------------------------------------------------------------------- /operator/controllers/config/resources.go: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2022, 2024 IBM Corp. 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package config 18 | 19 | import "fmt" 20 | 21 | // ResourceName is the type for aliasing resources that will be created. 22 | type ResourceName string 23 | 24 | func (rn ResourceName) String() string { 25 | return string(rn) 26 | } 27 | 28 | const ( 29 | CSIController ResourceName = "csi-controller" 30 | CSIControllerAttacher ResourceName = "csi-controller-attacher" 31 | CSIControllerProvisioner ResourceName = "csi-controller-provisioner" 32 | CSIControllerSnapshotter ResourceName = "csi-controller-snapshotter" 33 | CSIControllerResizer ResourceName = "csi-controller-resizer" 34 | CSINode ResourceName = "csi" 35 | NodeAgent ResourceName = "ibm-node-agent" 36 | CSIAttacherServiceAccount ResourceName = "csi-attacher-sa" 37 | // CSIControllerServiceAccount ResourceName = "csi-operator" 38 | CSINodeServiceAccount ResourceName = "csi-node-sa" 39 | CSIProvisionerServiceAccount ResourceName = "csi-provisioner-sa" 40 | CSISnapshotterServiceAccount ResourceName = "csi-snapshotter-sa" 41 | CSIResizerServiceAccount ResourceName = "csi-resizer-sa" 42 | 43 | // Suffixes for ClusterRole and ClusterRoleBinding names. 44 | Provisioner ResourceName = "provisioner" 45 | NodePlugin ResourceName = "node" 46 | Attacher ResourceName = "attacher" 47 | Snapshotter ResourceName = "snapshotter" 48 | Resizer ResourceName = "resizer" 49 | ) 50 | 51 | // GetNameForResource returns the name of a resource for a CSI driver 52 | func GetNameForResource(name ResourceName, driverName string) string { 53 | switch name { 54 | case CSIController: 55 | return fmt.Sprintf("%s-operator", driverName) 56 | case CSIControllerAttacher: 57 | return fmt.Sprintf("%s-attacher", driverName) 58 | case CSIControllerProvisioner: 59 | return fmt.Sprintf("%s-provisioner", driverName) 60 | case CSIControllerSnapshotter: 61 | return fmt.Sprintf("%s-snapshotter", driverName) 62 | case CSIControllerResizer: 63 | return fmt.Sprintf("%s-resizer", driverName) 64 | case CSINode: 65 | return driverName 66 | case CSIAttacherServiceAccount: 67 | return fmt.Sprintf("%s-attacher", driverName) 68 | // case CSIControllerServiceAccount: 69 | // return fmt.Sprintf("%s-operator", driverName) 70 | case CSINodeServiceAccount: 71 | return fmt.Sprintf("%s-node", driverName) 72 | case CSIProvisionerServiceAccount: 73 | return fmt.Sprintf("%s-provisioner", driverName) 74 | case CSISnapshotterServiceAccount: 75 | return fmt.Sprintf("%s-snapshotter", driverName) 76 | case CSIResizerServiceAccount: 77 | return fmt.Sprintf("%s-resizer", driverName) 78 | default: 79 | return fmt.Sprintf("%s-%s", driverName, name) 80 | } 81 | } 82 | 83 | type ResourceKind string 84 | 85 | const ( 86 | Secret ResourceKind = "Secret" // #nosec G101 false positive 87 | ConfigMap ResourceKind = "ConfigMap" 88 | ) 89 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, sex characteristics, gender identity and expression, 9 | level of experience, education, socio-economic status, nationality, personal 10 | appearance, race, religion, or sexual identity and orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the project team at deeghuge@in.ibm.com. All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 71 | available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html 72 | 73 | [homepage]: https://www.contributor-covenant.org 74 | 75 | For answers to common questions about this code of conduct, see 76 | https://www.contributor-covenant.org/faq 77 | -------------------------------------------------------------------------------- /driver/csiplugin/server.go: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2019, 2024 IBM Corp. 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package scale 18 | 19 | import ( 20 | "net" 21 | "net/url" 22 | "os" 23 | "sync" 24 | 25 | "k8s.io/klog/v2" 26 | 27 | "google.golang.org/grpc" 28 | 29 | csi "github.com/container-storage-interface/spec/lib/go/csi" 30 | ) 31 | 32 | // Defines Non blocking GRPC server interfaces 33 | type NonBlockingGRPCServer interface { 34 | // Start services at the endpoint 35 | Start(endpoint string, ids csi.IdentityServer, cs csi.ControllerServer, ns csi.NodeServer) 36 | // Waits for the service to stop 37 | Wait() 38 | // Stops the service gracefully 39 | Stop() 40 | // Stops the service forcefully 41 | ForceStop() 42 | } 43 | 44 | func NewNonBlockingGRPCServer() NonBlockingGRPCServer { 45 | return &nonBlockingGRPCServer{} 46 | } 47 | 48 | // NonBlocking server 49 | type nonBlockingGRPCServer struct { 50 | wg sync.WaitGroup 51 | server *grpc.Server 52 | } 53 | 54 | func (s *nonBlockingGRPCServer) Start(endpoint string, ids csi.IdentityServer, cs csi.ControllerServer, ns csi.NodeServer) { 55 | s.wg.Add(1) 56 | 57 | go s.serve(endpoint, ids, cs, ns) 58 | } 59 | 60 | func (s *nonBlockingGRPCServer) Wait() { 61 | s.wg.Wait() 62 | } 63 | 64 | func (s *nonBlockingGRPCServer) Stop() { 65 | s.server.GracefulStop() 66 | } 67 | 68 | func (s *nonBlockingGRPCServer) ForceStop() { 69 | s.server.Stop() 70 | } 71 | 72 | func (s *nonBlockingGRPCServer) serve(endpoint string, ids csi.IdentityServer, cs csi.ControllerServer, ns csi.NodeServer) { 73 | 74 | opts := []grpc.ServerOption{ 75 | grpc.UnaryInterceptor(logGRPC), 76 | } 77 | 78 | u, err := url.Parse(endpoint) 79 | 80 | if err != nil { 81 | klog.Fatalf("%v", err.Error()) 82 | } 83 | 84 | var addr string 85 | switch u.Scheme { 86 | case "unix": 87 | addr = u.Path 88 | if err := os.Remove(addr); err != nil && !os.IsNotExist(err) { 89 | klog.Fatalf("Failed to remove %s, error: %s", addr, err.Error()) 90 | } 91 | case "tcp": 92 | addr = u.Host 93 | default: 94 | klog.Fatalf("%v endpoint scheme not supported", u.Scheme) 95 | } 96 | 97 | klog.V(4).Infof("Start listening with scheme %v, addr %v", u.Scheme, addr) 98 | listener, err := net.Listen(u.Scheme, addr) 99 | if err != nil { 100 | klog.Fatalf("Failed to listen: %v", err) 101 | } 102 | // Updated csi.sock file permission to read and write only 103 | if err := os.Chmod(addr, 0600); err != nil { 104 | klog.Fatalf("Failed to modify csi.sock permission : %v", err) 105 | } 106 | server := grpc.NewServer(opts...) 107 | s.server = server 108 | 109 | if ids != nil { 110 | csi.RegisterIdentityServer(server, ids) 111 | } 112 | if cs != nil { 113 | csi.RegisterControllerServer(server, cs) 114 | } 115 | if ns != nil { 116 | csi.RegisterNodeServer(server, ns) 117 | } 118 | 119 | klog.Infof("Started listening on %#v", listener.Addr()) 120 | 121 | if err := server.Serve(listener); err != nil { 122 | klog.Fatalf("Failed to serve: %v", err) 123 | } 124 | } 125 | -------------------------------------------------------------------------------- /driver/csiplugin/utils.go: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2019, 2024 IBM Corp. 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package scale 18 | 19 | import ( 20 | "fmt" 21 | "os" 22 | "regexp" 23 | "strings" 24 | 25 | "github.com/IBM/ibm-spectrum-scale-csi/driver/csiplugin/utils" 26 | csi "github.com/container-storage-interface/spec/lib/go/csi" 27 | "golang.org/x/net/context" 28 | "google.golang.org/grpc" 29 | "k8s.io/klog/v2" 30 | ) 31 | 32 | func NewVolumeCapabilityAccessMode(mode csi.VolumeCapability_AccessMode_Mode) *csi.VolumeCapability_AccessMode { 33 | return &csi.VolumeCapability_AccessMode{Mode: mode} 34 | } 35 | 36 | func NewControllerServiceCapability(cap csi.ControllerServiceCapability_RPC_Type) *csi.ControllerServiceCapability { 37 | return &csi.ControllerServiceCapability{ 38 | Type: &csi.ControllerServiceCapability_Rpc{ 39 | Rpc: &csi.ControllerServiceCapability_RPC{ 40 | Type: cap, 41 | }, 42 | }, 43 | } 44 | } 45 | 46 | func NewNodeServiceCapability(cap csi.NodeServiceCapability_RPC_Type) *csi.NodeServiceCapability { 47 | return &csi.NodeServiceCapability{ 48 | Type: &csi.NodeServiceCapability_Rpc{ 49 | Rpc: &csi.NodeServiceCapability_RPC{ 50 | Type: cap, 51 | }, 52 | }, 53 | } 54 | } 55 | 56 | func logGRPC(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) { 57 | newCtx := utils.SetLoggerId(ctx) 58 | loggerId := utils.GetLoggerId(newCtx) 59 | 60 | skipLog := skipLogging(info.FullMethod) 61 | if skipLog { 62 | klog.V(4).Infof("[%s] GRPC call: %s", loggerId, info.FullMethod) 63 | } else { 64 | klog.Infof("[%s] GRPC call: %s", loggerId, info.FullMethod) 65 | } 66 | 67 | // Mask the secrets from request before logging 68 | logLevel := strings.ToUpper(os.Getenv(utils.LogLevel)) 69 | if logLevel == utils.DEBUG.String() || logLevel == utils.TRACE.String() { 70 | reqString := fmt.Sprintf("%+v", req) 71 | regExp := regexp.MustCompile("secrets:.*?>") 72 | reqToLog := regExp.ReplaceAllString(reqString, "") 73 | klog.V(4).Infof("[%s] GRPC request: %+v", loggerId, reqToLog) 74 | } 75 | 76 | startTime := utils.GetExecutionTime() 77 | resp, err := handler(newCtx, req) 78 | if err != nil { 79 | klog.Errorf("[%s] GRPC error: %v", loggerId, err) 80 | } else { 81 | klog.V(4).Infof("[%s] GRPC response: %+v", loggerId, resp) 82 | } 83 | endTime := utils.GetExecutionTime() 84 | diffTime := endTime - startTime 85 | 86 | if skipLog { 87 | klog.V(4).Infof("[%s] Time taken to execute %s request(in milliseconds): %d", loggerId, info.FullMethod, diffTime) 88 | } else { 89 | klog.Infof("[%s] Time taken to execute %s request(in milliseconds): %d", loggerId, info.FullMethod, diffTime) 90 | } 91 | return resp, err 92 | } 93 | 94 | func skipLogging(methodName string) bool { 95 | method := [...]string{"NodeGetCapabilities", "Identity/Probe", "Identity/GetPluginInfo", "Node/NodeGetInfo", "Node/NodeGetVolumeStats"} 96 | for _, m := range method { 97 | if strings.Contains(methodName, m) { 98 | return true 99 | } 100 | } 101 | return false 102 | } 103 | -------------------------------------------------------------------------------- /operator/go.mod: -------------------------------------------------------------------------------- 1 | module github.com/IBM/ibm-spectrum-scale-csi/operator 2 | 3 | go 1.24.0 4 | 5 | require ( 6 | github.com/IBM/ibm-spectrum-scale-csi/driver v0.0.0-20250903185856-2c16e4c901d3 7 | github.com/google/uuid v1.6.0 8 | github.com/imdario/mergo v0.3.16 9 | github.com/onsi/ginkgo v1.16.5 10 | github.com/onsi/gomega v1.38.2 11 | github.com/openshift/api v0.0.0-20250929151534-41627d81e9c1 12 | github.com/presslabs/controller-util v0.16.0 13 | go.uber.org/zap v1.27.0 14 | k8s.io/api v0.33.2 15 | k8s.io/apimachinery v0.33.2 16 | k8s.io/client-go v0.33.2 17 | sigs.k8s.io/controller-runtime v0.21.0 18 | ) 19 | 20 | require ( 21 | github.com/beorn7/perks v1.0.1 // indirect 22 | github.com/blang/semver/v4 v4.0.0 // indirect 23 | github.com/cespare/xxhash/v2 v2.3.0 // indirect 24 | github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect 25 | github.com/emicklei/go-restful/v3 v3.12.0 // indirect 26 | github.com/evanphx/json-patch/v5 v5.9.11 // indirect 27 | github.com/fsnotify/fsnotify v1.7.0 // indirect 28 | github.com/fxamacker/cbor/v2 v2.7.0 // indirect 29 | github.com/go-logr/logr v1.4.3 // indirect 30 | github.com/go-logr/zapr v1.3.0 // indirect 31 | github.com/go-openapi/jsonpointer v0.21.0 // indirect 32 | github.com/go-openapi/jsonreference v0.21.0 // indirect 33 | github.com/go-openapi/swag v0.23.0 // indirect 34 | github.com/go-test/deep v1.1.1 // indirect 35 | github.com/gogo/protobuf v1.3.2 // indirect 36 | github.com/google/btree v1.1.3 // indirect 37 | github.com/google/gnostic-models v0.6.9 // indirect 38 | github.com/google/go-cmp v0.7.0 // indirect 39 | github.com/iancoleman/strcase v0.3.0 // indirect 40 | github.com/josharian/intern v1.0.0 // indirect 41 | github.com/json-iterator/go v1.1.12 // indirect 42 | github.com/mailru/easyjson v0.7.7 // indirect 43 | github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect 44 | github.com/modern-go/reflect2 v1.0.2 // indirect 45 | github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect 46 | github.com/nxadm/tail v1.4.8 // indirect 47 | github.com/pkg/errors v0.9.1 // indirect 48 | github.com/prometheus/client_golang v1.22.0 // indirect 49 | github.com/prometheus/client_model v0.6.1 // indirect 50 | github.com/prometheus/common v0.62.0 // indirect 51 | github.com/prometheus/procfs v0.15.1 // indirect 52 | github.com/spf13/pflag v1.0.5 // indirect 53 | github.com/x448/float16 v0.8.4 // indirect 54 | go.uber.org/multierr v1.11.0 // indirect 55 | go.yaml.in/yaml/v3 v3.0.4 // indirect 56 | golang.org/x/net v0.46.0 // indirect 57 | golang.org/x/oauth2 v0.30.0 // indirect 58 | golang.org/x/sync v0.17.0 // indirect 59 | golang.org/x/sys v0.37.0 // indirect 60 | golang.org/x/term v0.36.0 // indirect 61 | golang.org/x/text v0.30.0 // indirect 62 | golang.org/x/time v0.9.0 // indirect 63 | gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect 64 | google.golang.org/genproto/googleapis/rpc v0.0.0-20250804133106-a7a43d27e69b // indirect 65 | google.golang.org/grpc v1.76.0 // indirect 66 | google.golang.org/protobuf v1.36.7 // indirect 67 | gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect 68 | gopkg.in/inf.v0 v0.9.1 // indirect 69 | gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect 70 | gopkg.in/yaml.v3 v3.0.1 // indirect 71 | k8s.io/apiextensions-apiserver v0.33.0 // indirect 72 | k8s.io/klog/v2 v2.130.1 // indirect 73 | k8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff // indirect 74 | k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 // indirect 75 | sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 // indirect 76 | sigs.k8s.io/randfill v1.0.0 // indirect 77 | sigs.k8s.io/structured-merge-diff/v4 v4.6.0 // indirect 78 | sigs.k8s.io/yaml v1.4.0 // indirect 79 | ) 80 | 81 | replace github.com/IBM/ibm-spectrum-scale-csi/driver => github.com/badri-pathak/ibm-spectrum-scale-csi/driver v0.0.0-20251121094552-bd12f28953e0 82 | -------------------------------------------------------------------------------- /README.driver.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## Static Provisioning 4 | 5 | In static provisioning, the backend storage volumes and PVs are created by the administrator. Static provisioning can be used to provision a directory or fileset with existing data. 6 | 7 | For static provisioning of existing directories perform the following steps: 8 | 9 | - Generate static pv yaml file using helper script 10 | 11 | ``` 12 | tools/generate_pv_yaml.sh --filesystem gpfs0 --size 10 \ 13 | --linkpath /ibm/gpfs0/pvfileset/static-pv --pvname static-pv 14 | ``` 15 | 16 | - For static provisioning, refer following sample pvc and pod files for sanity test 17 | 18 | ``` 19 | driver/examples/static/static_pvc.yaml 20 | driver/examples/static/static_pod.yaml 21 | ``` 22 | 23 | 24 | ## Dynamic Provisioning 25 | 26 | Dynamic provisioning is used to dynamically provision the storage backend volume based on the storageClass. 27 | 28 | ### Storageclass 29 | Storageclass defines what type of backend volume should be created by dynamic provisioning. IBM Storage Scale CSI driver supports creation of directory based (also known as lightweight volumes) and fileset based (independent as well as dependent) volumes. Following parameters are supported by IBM Storage Scale CSI driver storageClass: 30 | 31 | - **volBackendFs**: Filesystem on which the volume should be created. This is a mandatory parameter. 32 | - **clusterId**: Cluster ID on which the volume should be created. 33 | - **volDirBasePath**: Base directory path relative to the filesystem mount point under which directory based volumes should be created. If specified, the storageClass is used for directory based (lightweight) volume creation. 34 | - **uid**: UID with which the volume should be created. Optional 35 | - **gid**: GID with which the volume should be created. Optional 36 | - **filesetType**: Type of fileset. Valid values are "independent" or "dependent". Default: independent 37 | - **parentFileset**: Specifies the parent fileset under which dependent fileset should be created. 38 | - **inodeLimit**: Inode limit for fileset based volumes. If not specified, Inode limit will be calculated using formule volumesize/filesystem block size. 39 | 40 | For dynamic provisioning, refer following sample storageClass, pvc and pod files for sanity test 41 | 42 | Example: 43 | 44 | ``` 45 | driver/examples/dynamic/fileset/storageclassfileset.yaml 46 | driver/examples/dynamic/fileset/pvcfileset.yaml 47 | driver/examples/dynamic/fileset/podfileset.yaml 48 | ``` 49 | 50 | 51 | ## Links 52 | 53 | [IBM Storage Scale Documentation Welcome Page](https://www.ibm.com/docs/en/spectrum-scale) 54 | The IBM Documentation contains all official IBM Storage Scale information and guidance. 55 | 56 | [IBM Storage Scale FAQ](https://www.ibm.com/docs/en/spectrum-scale?topic=STXKQY/gpfsclustersfaq.html) 57 | Main starting page for all IBM Storage Scale compatibility information. 58 | 59 | [IBM Block CSI driver](https://github.com/IBM/ibm-block-csi-driver) 60 | IBM Block CSI driver supporting multiple IBM storage systems. 61 | 62 | [Installing kubeadm](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/) 63 | Main Kubernetes site detailing how to install kubeadm and create a cluster. 64 | 65 | [OpenShift Container Platform 4.x Tested Integrations](https://access.redhat.com/articles/4128421) 66 | Red Hat's test matrix for OpenShift 4.x. 67 | 68 | [IBM Storage Enabler for Containers Welcome Page](https://www.ibm.com/support/knowledgecenter/en/SSCKLT/landing/IBM_Storage_Enabler_for_Containers_welcome_page.html) 69 | Flex Volume driver released in late 2018 with a HELM update in early 2019, providing compatibility with IBM Storage Scale for file storage and multiple IBM storage systems for block storage. Future development efforts have shifted to CSI. 70 | 71 | [IBM Storage Scale Users Group](http://www.gpfsug.org/) 72 | A group of both IBM and non-IBM users, interested in IBM Storage Scale 73 | 74 | [IBM Storage Scale Users Group Mailing List and Slack Channel](https://www.spectrumscaleug.org/join/) 75 | Join everyone and let the team know about your experience with the CSI driver 76 | -------------------------------------------------------------------------------- /operator/hacks/package_operator.py: -------------------------------------------------------------------------------- 1 | #!/bin/python 2 | import argparse 3 | import sys 4 | import os 5 | import yaml 6 | import zipfile 7 | from shutil import copyfile 8 | 9 | 10 | BASE_DIR="{0}/..".format(os.path.dirname(os.path.realpath(__file__))) 11 | OLM_CATALOG="{0}/config/olm-catalog".format(BASE_DIR) 12 | OPERATOR="{0}/ibm-spectrum-scale-csi-operator".format(OLM_CATALOG) 13 | #PACKAGE_FILE="ibm-spectrum-scale-csi-operator.package.yaml" 14 | 15 | PACKAGE_POSTFIX="package.yaml" 16 | 17 | def main( args ): 18 | parser = argparse.ArgumentParser( 19 | description=''' 20 | A hack to package the operator for release. 21 | ''') 22 | 23 | parser.add_argument( '-d', '--dir', metavar='package dir', dest='packagedir', 24 | default=OPERATOR, 25 | help='''The manifest directory, contains a package.yaml and version directories.''') 26 | 27 | parser.add_argument( '-f', '--flat', dest='flatten', action='store_true', 28 | help='''Flatten the bundle (for certified releases).''') 29 | 30 | parser.add_argument( '-o', '--output', metavar="ZIP Archive", dest='output', 31 | default=None, 32 | help='''The output location of this script. If directory supplied (or not provided) zip file 33 | will be the .zip (executing dir if not supplied). If a file with a path is supplied, 34 | it will be placed in that location.''') 35 | 36 | parser.add_argument( '--nozip', dest='nozip', 37 | action='store_true', 38 | help='''A flag to not zip the operator (useful for courier scans).''') 39 | 40 | args = parser.parse_args() 41 | 42 | # Find the packages 43 | packagedir = args.packagedir 44 | if packagedir[0] is not '/': 45 | packagedir = "{0}/{1}".format( os.getcwd(), packagedir ) 46 | 47 | 48 | # Find the package file for the supplied directory. 49 | packagefile = None 50 | packagefilename = None 51 | packages = os.listdir(packagedir) 52 | for package in packages: 53 | if package.endswith( PACKAGE_POSTFIX ) : 54 | packagefilename = package 55 | packagefile = "{0}/{1}".format(packagedir, package) 56 | break 57 | 58 | 59 | # Open the package file manifest 60 | if packagefile is None: 61 | print("Unable to find package manifest.") 62 | return 1 63 | 64 | pkgobj={} 65 | with open(packagefile, 'r') as stream: 66 | try: 67 | pkgobj = yaml.safe_load(stream) 68 | except yaml.YAMLError as e: 69 | print(e) 70 | return 1 71 | 72 | # cache details. 73 | packagename = pkgobj.get("packageName" , "package") 74 | defaultchannel = pkgobj.get("defaultChannel", None) 75 | selectedchannel = None 76 | 77 | # Get selected channels. 78 | for channel in pkgobj.get("channels", []): 79 | if channel.get("name", "") == defaultchannel: 80 | selectedchannel = channel 81 | break 82 | 83 | # Determine the correct directory. 84 | currentcsv = selectedchannel.get("currentCSV", None) 85 | csvdir = None 86 | if currentcsv: 87 | for package in packages: 88 | if currentcsv.endswith( package ): 89 | csvdir = "{0}/{1}".format(packagedir, package) 90 | 91 | # Determine the zip file. 92 | zipname = "{0}.zip".format(packagename) 93 | if args.output is not None: 94 | if os.path.isdir(args.output): 95 | zipname = "{0}/{1}".format(zipname, args.output) 96 | else: 97 | zipname = args.output 98 | 99 | # zip the used files. 100 | if not args.nozip: 101 | with zipfile.ZipFile(zipname, 'w') as packagezip: 102 | packagezip.write(packagefile, packagefilename) 103 | 104 | for config in os.listdir(csvdir): 105 | packagezip.write("{0}/{1}".format( csvdir, config), config) 106 | else: 107 | dirname= "{0}".format( args.output ) 108 | os.mkdir(dirname) 109 | copyfile(packagefile, "{0}/{1}".format(dirname, packagefilename)) 110 | 111 | for config in os.listdir(csvdir): 112 | copyfile("{0}/{1}".format( csvdir, config), "{0}/{1}".format(dirname, config)) 113 | 114 | print("Package {0} was bundled to {1}".format(packagename, zipname)) 115 | 116 | if __name__ == "__main__": 117 | sys.exit(main(sys.argv)) 118 | 119 | -------------------------------------------------------------------------------- /driver/csiplugin/utils/http_utils.go: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2019, 2024 IBM Corp. 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package utils 18 | 19 | import ( 20 | "bytes" 21 | "context" 22 | "encoding/json" 23 | "fmt" 24 | "io" 25 | "net/http" 26 | "strings" 27 | 28 | "k8s.io/klog/v2" 29 | ) 30 | 31 | const BucketKeysURL = "scalemgmt/v2/bucket/keys" 32 | 33 | /* 34 | func ExtractErrorResponse(response *http.Response) error { 35 | errorResponse := connectors.GenericResponse{} 36 | err := UnmarshalResponse(response, &errorResponse) 37 | if err != nil { 38 | return fmt.Errorf("json.Unmarshal failed %v", err) 39 | } 40 | return fmt.Errorf(errorResponse.Err) 41 | } 42 | */ 43 | 44 | func UnmarshalResponse(ctx context.Context, r *http.Response, object interface{}) error { 45 | klog.V(6).Infof("[%s] http_utils UnmarshalResponse. response: %v", GetLoggerId(ctx), r.Body) 46 | 47 | body, err := io.ReadAll(r.Body) 48 | if err != nil { 49 | return fmt.Errorf("ioutil.ReadAll failed %v", err) 50 | } 51 | 52 | err = json.Unmarshal(body, object) 53 | if err != nil { 54 | return fmt.Errorf("json.Unmarshal failed %v", err) 55 | } 56 | 57 | return nil 58 | } 59 | 60 | func HttpExecuteUserAuth(ctx context.Context, httpClient *http.Client, requestType string, requestURL string, user string, password string, rawPayload interface{}) (*http.Response, error) { 61 | klog.V(4).Infof("[%s] http_utils HttpExecuteUserAuth. type: %s, url: %s, user: %s", GetLoggerId(ctx), requestType, requestURL, user) 62 | if !strings.Contains(requestURL, BucketKeysURL) { 63 | klog.V(6).Infof("[%s] http_utils HttpExecuteUserAuth. request payload: %v", GetLoggerId(ctx), rawPayload) 64 | } 65 | 66 | payload, err := json.MarshalIndent(rawPayload, "", " ") 67 | if err != nil { 68 | err = fmt.Errorf("internal error marshaling params. url: %s: %#v", requestURL, err) 69 | return nil, fmt.Errorf("failed %v", err) 70 | } 71 | 72 | if user == "" { 73 | return nil, fmt.Errorf("empty UserName passed") 74 | } 75 | 76 | request, err := http.NewRequest(requestType, requestURL, bytes.NewBuffer(payload)) 77 | if err != nil { 78 | err = fmt.Errorf("error in creating request. url: %s: %#v", requestURL, err) 79 | return nil, fmt.Errorf("failed %v", err) 80 | } 81 | 82 | request.Header.Add("Content-Type", "application/json") 83 | request.Header.Add("Accept", "application/json") 84 | 85 | request.SetBasicAuth(user, password) 86 | 87 | requestToLog := *request 88 | if strings.Contains(requestURL, BucketKeysURL) && request != nil { 89 | requestToLog.Body = nil 90 | } 91 | klog.V(6).Infof("[%s] http_utils HttpExecuteUserAuth request: %+v", GetLoggerId(ctx), &requestToLog) 92 | 93 | return httpClient.Do(request) 94 | } 95 | 96 | func WriteResponse(w http.ResponseWriter, code int, object interface{}) { 97 | klog.V(4).Infof("http_utils WriteResponse. code: %d, object: %v", code, object) 98 | 99 | data, err := json.Marshal(object) 100 | if err != nil { 101 | w.WriteHeader(http.StatusInternalServerError) 102 | return 103 | } 104 | 105 | w.WriteHeader(code) 106 | fmt.Fprint(w, string(data)) 107 | } 108 | 109 | func Unmarshal(r *http.Request, object interface{}) error { 110 | klog.V(6).Infof("http_utils Unmarshal. request: %v, object: %v", r, object) 111 | 112 | body, err := io.ReadAll(r.Body) 113 | if err != nil { 114 | return err 115 | } 116 | 117 | err = json.Unmarshal(body, object) 118 | if err != nil { 119 | return err 120 | } 121 | 122 | return nil 123 | } 124 | 125 | func UnmarshalDataFromRequest(r *http.Request, object interface{}) error { 126 | klog.V(6).Infof("http_utils UnmarshalDataFromRequest. request: %v, object: %v", r, object) 127 | 128 | body, err := io.ReadAll(r.Body) 129 | if err != nil { 130 | return err 131 | } 132 | 133 | err = json.Unmarshal(body, object) 134 | if err != nil { 135 | return err 136 | } 137 | 138 | return nil 139 | } 140 | -------------------------------------------------------------------------------- /tools/ansible/deploy/hacks/deploycsioperator.py: -------------------------------------------------------------------------------- 1 | #!/bin/python 2 | 3 | import argparse 4 | import sys 5 | import os 6 | import yaml 7 | import subprocess 8 | import textfsm 9 | 10 | 11 | def mmlscluster(cmd, tmpl): 12 | ''' Run the mmlscluster command and grab the relevant data ''' 13 | 14 | p = subprocess.Popen([cmd], stdout=subprocess.PIPE, 15 | stderr=subprocess.PIPE) 16 | out, err = p.communicate() 17 | 18 | output = { 19 | "id": None, 20 | "name": None, 21 | "nodes":[] 22 | } 23 | with open(tmpl,"r") as template: 24 | re_table = textfsm.TextFSM(template) 25 | data = re_table.ParseText(out) 26 | 27 | zipped = dict() 28 | for row in data: 29 | zipped = dict(zip(re_table.header, row)) 30 | node = zipped.get("Node", None) 31 | if node is not None and isinstance(node, str): 32 | output["nodes"].append(node) 33 | 34 | output["id"] = zipped.get("ID") 35 | output["name"] = zipped.get("Name") 36 | 37 | return output 38 | 39 | def mmlsgui(cmd, tmpl): 40 | ''' Run mmlsnodeclass GUI_MGMT_SERVERS, return the first instance found. ''' 41 | 42 | p = subprocess.Popen([cmd, "GUI_MGMT_SERVERS"], stdout=subprocess.PIPE, 43 | stderr=subprocess.PIPE) 44 | out, err = p.communicate() 45 | 46 | with open(tmpl,"r") as template: 47 | re_table = textfsm.TextFSM(template) 48 | data = re_table.ParseText(out) 49 | 50 | zipped = dict() 51 | for row in data: 52 | zipped = dict(zip(re_table.header, row)) 53 | gui = zipped.get("GUI_MGMT_SERVER", None) 54 | if gui is not None and isinstance(gui, str): 55 | return gui 56 | 57 | return "" 58 | 59 | def mmlsfs(cmd, tmpl): 60 | ''' Run the mmlsfs command and grab the relevant data. ''' 61 | p = subprocess.Popen([cmd, "all", "-T"], 62 | stdout=subprocess.PIPE, stderr=subprocess.PIPE) 63 | out, err = p.communicate() 64 | 65 | output=[] 66 | with open(tmpl,"r") as template: 67 | re_table = textfsm.TextFSM(template) 68 | data = re_table.ParseText(out) 69 | 70 | zipped = dict() 71 | for row in data: 72 | zipped = dict(zip(re_table.header, row)) 73 | 74 | output.append({ 75 | "fs" : zipped.get("fs", ""), 76 | "mount" : zipped.get("mount", "") 77 | }) 78 | 79 | return output 80 | 81 | 82 | def main(args): 83 | parser = argparse.ArgumentParser( 84 | description='''A python script to scrape the output of mmlscluster for ingestion in ansible.''') 85 | parser.add_argument( '--mmlscluster', metavar="mmlscluster", 86 | dest='mmlscluster', default="/usr/lpp/mmfs/bin/mmlscluster", 87 | help='''The command for mmlscluster, only specify if installed somewhere than the default.''') 88 | parser.add_argument( '--mmlsnodeclass', metavar="mmlsnodeclass", 89 | dest='mmlsnodeclass', default="/usr/lpp/mmfs/bin/mmlsnodeclass", 90 | help='''The command for mmlsnodeclass, only specify if installed somewhere than the default.''') 91 | parser.add_argument( '--mmlsfs', metavar="mmlsfs", 92 | dest='mmlsfs', default="/usr/lpp/mmfs/bin/mmlsfs", 93 | help='''The command for mmlsfs, only specify if installed somewhere than the default.''') 94 | parser.add_argument( '--templates', metavar='templates', 95 | dest='templates', default=os.path.dirname(os.path.realpath(__file__)), 96 | help='''The directory containing the TextFSM templates''') 97 | 98 | 99 | parser.add_argument('--fs', metavar='filesystem', 100 | dest='fs', default=None, 101 | help='''The name of the filesystem (including /dev/) to use with the csi driver''') 102 | 103 | parser.add_argument('--fset', metavar='fileset', 104 | dest='fs', default=None, 105 | help='''The name of the filesystem (including /dev/) to use with the csi driver''') 106 | 107 | args = parser.parse_args() 108 | 109 | # Create the template paths 110 | SCRIPT_DIR = args.templates 111 | MMLSCLUSTER_TEMPL = "{0}/templates/mmlscluster".format(SCRIPT_DIR) 112 | MMLSNODECLASS_TEMPL = "{0}/templates/mmlsnodeclass".format(SCRIPT_DIR) 113 | MMLSFS_TEMPL = "{0}/templates/mmlsfs".format(SCRIPT_DIR) 114 | 115 | cluster=mmlscluster(args.mmlscluster, MMLSCLUSTER_TEMPL) 116 | cluster["gui"] = mmlsgui(args.mmlsnodeclass, MMLSNODECLASS_TEMPL) 117 | cluster["fs"] = mmlsfs(args.mmlsfs, MMLSFS_TEMPL) 118 | 119 | print(yaml.dump(cluster)) 120 | 121 | if __name__ == "__main__": 122 | sys.exit(main(sys.argv)) 123 | -------------------------------------------------------------------------------- /driver/csiplugin/pkg/consistencygroup/consistencygroup.go: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2023, 2024 IBM Corp. 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package consistencygroup 18 | 19 | import ( 20 | "errors" 21 | "fmt" 22 | "strconv" 23 | "strings" 24 | 25 | corev1 "k8s.io/api/core/v1" 26 | ) 27 | 28 | type StorageClassType int 29 | 30 | const ( 31 | ClassicStorageClass StorageClassType = 0 // aka version 1 storage class 32 | ConsistencyGroupStorageClass StorageClassType = 1 // aka version 2 storage class 33 | ) 34 | 35 | type VolumeType int 36 | 37 | const ( 38 | LightweightVolume VolumeType = 0 39 | DependentFilesetBasedVolume VolumeType = 1 40 | IndependentFilesetBasedVolume VolumeType = 2 41 | ) 42 | 43 | var ( 44 | ErrNoCsiVolume = errors.New("no CSI volume") 45 | ErrInvalidCsiVolumeHandle = errors.New("invalid CSI volume handle format") 46 | ) 47 | 48 | // VolumeHandle represents the VolumeHandle parameter that exists in the CSI PV spec. 49 | type VolumeHandle struct { 50 | StorageClassType StorageClassType 51 | VolumeType VolumeType 52 | ClusterID string // ID of owning cluster where fileset resides 53 | FilesystemUID string 54 | ConsistencyGroup string // Matches to the name of the independent fileset name. Format: - 55 | FilesetName string // Name of dependent fileset that represents the PV 56 | FilesetLinkPath string 57 | } 58 | 59 | // GetVolumeHandle get's the CSI volume handle from a CSI PV source. 60 | // VolumeHandle format: 61 | // ;;;;;; 62 | func GetVolumeHandle(pvs *corev1.CSIPersistentVolumeSource) (VolumeHandle, error) { 63 | var vh VolumeHandle 64 | if pvs == nil { 65 | return vh, ErrNoCsiVolume 66 | } 67 | split := strings.Split(pvs.VolumeHandle, ";") 68 | if len(split) < 7 { 69 | return vh, ErrInvalidCsiVolumeHandle 70 | } 71 | i, err := strconv.Atoi(split[0]) 72 | if err != nil { 73 | return vh, err 74 | } 75 | vh.StorageClassType = StorageClassType(i) 76 | i, err = strconv.Atoi(split[1]) 77 | if err != nil { 78 | return vh, err 79 | } 80 | vh.VolumeType = VolumeType(i) 81 | vh.ClusterID = split[2] 82 | vh.FilesystemUID = split[3] 83 | vh.ConsistencyGroup = split[4] 84 | vh.FilesetName = split[5] 85 | vh.FilesetLinkPath = split[6] 86 | return vh, nil 87 | } 88 | 89 | // GetFilesystem reads the filesystem name from a CSI PV source. 90 | func GetFilesystem(pvs *corev1.CSIPersistentVolumeSource) (string, error) { 91 | filesystem, ok := pvs.VolumeAttributes["volBackendFs"] 92 | if !ok { 93 | return filesystem, errors.New("CSI volume attribute 'volBackendFs' missing") 94 | } 95 | return filesystem, nil 96 | } 97 | 98 | // GetConsistencyGroupFileset reads the consistency group fileset name from a CSI PV source. 99 | func GetConsistencyGroupFileset(pvs *corev1.CSIPersistentVolumeSource) (string, error) { 100 | volHandle, err := GetVolumeHandle(pvs) 101 | // The consistency group name is the same as the name of the independent fileset that represents the consistency group. 102 | return volHandle.ConsistencyGroup, err 103 | } 104 | 105 | // GetConsistencyGroupFilesetLinkPath returns the link path of the consistency group fileset. 106 | func GetConsistencyGroupFilesetLinkPath(pvs *corev1.CSIPersistentVolumeSource) (string, error) { 107 | volHandle, err := GetVolumeHandle(pvs) 108 | if err != nil { 109 | return "", err 110 | } 111 | if !strings.HasSuffix(volHandle.FilesetLinkPath, "/"+volHandle.FilesetName) { 112 | return "", fmt.Errorf("unexpected format of fileset link path %s", volHandle.FilesetLinkPath) 113 | } 114 | // We can assume that the consistency group independent filesets link path is the parent directory of the PV fileset link path. 115 | // Querying the link path using mmlsfileset is not required. 116 | cgFsetLinkPath := strings.TrimSuffix(volHandle.FilesetLinkPath, "/"+volHandle.FilesetName) 117 | 118 | return cgFsetLinkPath, nil 119 | } 120 | 121 | var _ fmt.Stringer = VolumeHandle{} 122 | 123 | // VolumeHandle implements fmt.Stringer interface to return the volume handle in string format 124 | func (vh VolumeHandle) String() string { 125 | return fmt.Sprintf("%x;%x;%s;%s;%s;%s;%s", vh.StorageClassType, vh.VolumeType, vh.ClusterID, vh.FilesystemUID, vh.ConsistencyGroup, vh.FilesetName, vh.FilesetLinkPath) 126 | } 127 | -------------------------------------------------------------------------------- /operator/config/manager/manager.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: apps/v1 3 | kind: Deployment 4 | metadata: 5 | labels: 6 | app.kubernetes.io/instance: ibm-spectrum-scale-csi-operator 7 | app.kubernetes.io/managed-by: ibm-spectrum-scale-csi-operator 8 | app.kubernetes.io/name: ibm-spectrum-scale-csi-operator 9 | product: ibm-spectrum-scale-csi 10 | release: ibm-spectrum-scale-csi-operator 11 | name: ibm-spectrum-scale-csi-operator 12 | namespace: ibm-spectrum-scale-csi-driver 13 | spec: 14 | replicas: 1 15 | selector: 16 | matchLabels: 17 | app.kubernetes.io/name: ibm-spectrum-scale-csi-operator 18 | template: 19 | metadata: 20 | annotations: 21 | productID: ibm-spectrum-scale-csi-operator 22 | productName: IBM Spectrum Scale CSI Operator 23 | productVersion: 3.1.0 24 | labels: 25 | app.kubernetes.io/instance: ibm-spectrum-scale-csi-operator 26 | app.kubernetes.io/managed-by: ibm-spectrum-scale-csi-operator 27 | app.kubernetes.io/name: ibm-spectrum-scale-csi-operator 28 | name: ibm-spectrum-scale-csi-operator 29 | product: ibm-spectrum-scale-csi 30 | release: ibm-spectrum-scale-csi-operator 31 | spec: 32 | affinity: 33 | nodeAffinity: 34 | requiredDuringSchedulingIgnoredDuringExecution: 35 | nodeSelectorTerms: 36 | - matchExpressions: 37 | - key: kubernetes.io/arch 38 | operator: In 39 | values: 40 | - amd64 41 | - ppc64le 42 | - s390x 43 | - arm64 44 | securityContext: 45 | runAsGroup: 10001 46 | runAsNonRoot: true 47 | runAsUser: 10001 48 | containers: 49 | - args: 50 | - --leaderElection=true 51 | env: 52 | - name: METRICS_BIND_ADDRESS 53 | value: "8383" 54 | - name: WATCH_NAMESPACE 55 | valueFrom: 56 | fieldRef: 57 | fieldPath: metadata.namespace 58 | - name: CSI_DRIVER_IMAGE 59 | value: cp.icr.io/cp/gpfs/csi/ibm-spectrum-scale-csi-driver@sha256:4178d546c92cac71ac785e00052693a7aaa70eb7c768c41a7bf00499aa84b565 60 | - name: CSI_SNAPSHOTTER_IMAGE 61 | value: registry.k8s.io/sig-storage/csi-snapshotter@sha256:c7e0a3718832b6197ce8b29fefb3fed3d84f4fbcdf08f4606140dbec2566501d 62 | - name: CSI_ATTACHER_IMAGE 63 | value: registry.k8s.io/sig-storage/csi-attacher@sha256:be59d0556508d3dd419cc34c53062f170a28902ef36e832b947c7796458a083d 64 | - name: CSI_PROVISIONER_IMAGE 65 | value: registry.k8s.io/sig-storage/csi-provisioner@sha256:0c537015fe9cf9d53d79d0181e17a78ee784303cd46bf957016605488f212327 66 | - name: CSI_LIVENESSPROBE_IMAGE 67 | value: registry.k8s.io/sig-storage/livenessprobe@sha256:9b75b9ade162136291d5e8f13a1dfc3dec71ee61419b1bfc112e0796ff8a6aa9 68 | - name: CSI_NODE_REGISTRAR_IMAGE 69 | value: registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:11f199f6bec47403b03cb49c79a41f445884b213b382582a60710b8c6fdc316a 70 | - name: CSI_RESIZER_IMAGE 71 | value: registry.k8s.io/sig-storage/csi-resizer@sha256:4a95d94e57ad82f6977cd8d4fdcfcfc0b83f02d990e4e7715b688c20970a906d 72 | image: cp.icr.io/cp/gpfs/csi/ibm-spectrum-scale-csi-operator@sha256:de0027696b54d29880e8b4c96183b04c1c7872a78d98b1e20f374208f82e2666 73 | imagePullPolicy: IfNotPresent 74 | livenessProbe: 75 | exec: 76 | command: 77 | - ./health_check.sh 78 | initialDelaySeconds: 10 79 | periodSeconds: 30 80 | name: operator 81 | readinessProbe: 82 | exec: 83 | command: 84 | - ./health_check.sh 85 | initialDelaySeconds: 3 86 | periodSeconds: 1 87 | resources: 88 | limits: 89 | cpu: 600m 90 | memory: 600Mi 91 | ephemeral-storage: 5Gi 92 | requests: 93 | cpu: 50m 94 | memory: 50Mi 95 | ephemeral-storage: 1Gi 96 | #hostNetwork: false 97 | #hostPID: false 98 | #hostIPC: false 99 | securityContext: 100 | readOnlyRootFilesystem: true 101 | allowPrivilegeEscalation: false 102 | privileged: false 103 | capabilities: 104 | drop: 105 | - ALL 106 | serviceAccountName: ibm-spectrum-scale-csi-operator 107 | tolerations: 108 | - effect: NoSchedule 109 | key: node-role.kubernetes.io/master 110 | operator: Exists 111 | - effect: NoSchedule 112 | key: node-role.kubernetes.io/infra 113 | operator: Exists 114 | - effect: NoSchedule 115 | key: node-role.kubernetes.io/control-plane 116 | operator: Exists 117 | -------------------------------------------------------------------------------- /driver/csiplugin/parallelSnapshot.go: -------------------------------------------------------------------------------- 1 | package scale 2 | 3 | import ( 4 | "context" 5 | "sync" 6 | 7 | "github.com/IBM/ibm-spectrum-scale-csi/driver/csiplugin/utils" 8 | "k8s.io/klog/v2" 9 | ) 10 | 11 | const ( 12 | createVolume = "CreateVolume" 13 | createSnapshot = "CreateSnapshot" 14 | deleteVolume = "DeleteVolume" 15 | deleteSnapshot = "DeleteSnapshot" 16 | retryInterval = 10 17 | retryCount = 7 18 | createOpInitCount = 1 19 | ) 20 | 21 | var createVolumeRefLock map[string]int 22 | var createSnapshotRefLock map[string]int 23 | var cgSnapLock map[string]string 24 | var cgSnapMutex sync.Mutex 25 | 26 | func CgSnapshotLock(ctx context.Context, targetPath string, snapExists bool) bool { 27 | cgSnapMutex.Lock() 28 | defer cgSnapMutex.Unlock() 29 | return cgParallelSnapshotLock(ctx, targetPath, snapExists) 30 | } 31 | 32 | func cgParallelSnapshotLock(ctx context.Context, targetPath string, snapExists bool) bool { 33 | 34 | if len(createVolumeRefLock) == 0 { 35 | createVolumeRefLock = make(map[string]int) 36 | } 37 | 38 | if len(createSnapshotRefLock) == 0 { 39 | createSnapshotRefLock = make(map[string]int) 40 | } 41 | 42 | if len(cgSnapLock) == 0 { 43 | cgSnapLock = make(map[string]string) 44 | } 45 | 46 | lockingModule := utils.GetModuleName(ctx) 47 | 48 | createVolCount, _ := createVolumeRefLock[targetPath] 49 | createSnapCount, _ := createSnapshotRefLock[targetPath] 50 | moduleName, exists := cgSnapLock[targetPath] 51 | if createVolCount > 0 || createSnapCount > 0 { 52 | switch lockingModule { 53 | case createVolume: 54 | createVolumeRefLock[targetPath]++ 55 | cgSnapLock[targetPath] = lockingModule 56 | return true 57 | case createSnapshot: 58 | if !snapExists { 59 | if createSnapCount > 0 { 60 | klog.Infof("[%s] Snap doesn't exist and lock already acquired by another snapshot request", utils.GetLoggerId(ctx)) 61 | return false 62 | } else { 63 | createSnapshotRefLock[targetPath] = createOpInitCount 64 | cgSnapLock[targetPath] = lockingModule 65 | return true 66 | } 67 | } else { 68 | createSnapshotRefLock[targetPath]++ 69 | cgSnapLock[targetPath] = lockingModule 70 | return true 71 | } 72 | default: 73 | klog.Infof("[%s] Delete operation is trying to acquire lock while create action is in progress", utils.GetLoggerId(ctx)) 74 | return false 75 | } 76 | } else { 77 | if exists && moduleName != "" { 78 | if (moduleName == deleteVolume || moduleName == deleteSnapshot) && (lockingModule == createVolume || lockingModule == createSnapshot) { 79 | klog.Infof("[%s] Delete operation acquired the lock, create operation retrying", utils.GetLoggerId(ctx)) 80 | return false 81 | } 82 | } else { 83 | switch lockingModule { 84 | case createVolume: 85 | createVolumeRefLock[targetPath] = createOpInitCount 86 | case createSnapshot: 87 | createSnapshotRefLock[targetPath] = createOpInitCount 88 | default: 89 | klog.Infof("[%s] Delete operation acquired the lock", utils.GetLoggerId(ctx)) 90 | } 91 | } 92 | cgSnapLock[targetPath] = lockingModule 93 | } 94 | 95 | klog.V(4).Infof("[%s] The target path is locked for %s: [%s]", utils.GetLoggerId(ctx), utils.GetModuleName(ctx), targetPath) 96 | return true 97 | } 98 | 99 | func CgSnapshotUnlock(ctx context.Context, targetPath string) { 100 | cgSnapMutex.Lock() 101 | defer cgSnapMutex.Unlock() 102 | cgParallelSnapshotUnlock(ctx, targetPath) 103 | } 104 | 105 | func cgParallelSnapshotUnlock(ctx context.Context, targetPath string) { 106 | moduleName := utils.GetModuleName(ctx) 107 | switch moduleName { 108 | case createVolume: 109 | if createVolumeRefLock[targetPath] > 0 { 110 | createVolumeRefLock[targetPath]-- 111 | } else { 112 | delete(createVolumeRefLock, targetPath) 113 | } 114 | case createSnapshot: 115 | if createSnapshotRefLock[targetPath] > 0 { 116 | klog.Infof("[%s] Decrease the count of createSnapshotRefLock", utils.GetLoggerId(ctx)) 117 | createSnapshotRefLock[targetPath]-- 118 | } else { 119 | delete(createSnapshotRefLock, targetPath) 120 | } 121 | default: 122 | klog.Infof("[%s] Delete operation released the lock", utils.GetLoggerId(ctx)) 123 | } 124 | delete(cgSnapLock, targetPath) 125 | klog.Infof("[%s] The target path is unlocked for %s: [%s]", utils.GetLoggerId(ctx), utils.GetModuleName(ctx), targetPath) 126 | } 127 | 128 | /*func retrySnapLock(ctx context.Context, targetPath, lockingModule string, snapExists bool) error { 129 | for i := 0; i < retryCount; i++ { 130 | time.Sleep(retryInterval * time.Second) 131 | if cgParallelSnapshotLock(ctx, targetPath, snapExists) { 132 | klog.Infof("[%s] retry attempt for %s operation", utils.GetLoggerId(ctx), utils.GetModuleName(ctx)) 133 | return nil 134 | } else { 135 | klog.Errorf("[%s] Failed to lock the target path after retry attampts", utils.GetLoggerId(ctx)) 136 | return status.Error(codes.Internal, fmt.Sprintf("Failed to lock the target path [%s] by %s after retry attampts", targetPath, utils.GetModuleName(ctx))) 137 | } 138 | } 139 | return nil 140 | }*/ 141 | -------------------------------------------------------------------------------- /operator/hacks/csv_copy_crd_descriptions.py: -------------------------------------------------------------------------------- 1 | #!/bin/python 2 | 3 | import argparse 4 | import sys 5 | import os 6 | import yaml 7 | 8 | BASE_DIR="{0}/../".format(os.path.dirname(os.path.realpath(__file__))) 9 | DEFAULT_VERSION="2.6.0" 10 | CSV_PATH="{0}deploy/olm-catalog/ibm-spectrum-scale-csi-operator/{1}/ibm-spectrum-scale-csi-operator.v{1}.clusterserviceversion.yaml" 11 | CRD_SOURCE_PATH="{0}deploy/crds/{1}" 12 | CRD_TARGET_PATH="{0}deploy/olm-catalog/ibm-spectrum-scale-csi-operator/{1}/{2}" 13 | 14 | def loadDescriptors(descType, crd, csv, descriptorMap={}): 15 | props = crd.get("spec", {}) \ 16 | .get("validation",{}) \ 17 | .get("openAPIV3Schema", {})\ 18 | .get("properties", {}) \ 19 | 20 | spec = props.get(descType, {}) 21 | specprops = spec.get("properties", {}) 22 | specdescriptors = [] 23 | specpaths = specprops.keys() 24 | 25 | while len(specpaths) > 0: 26 | # Load the path and then iterate to get the property. 27 | path = specpaths.pop(0) 28 | keys = path.split(".") 29 | name = keys[-1] 30 | prop = spec 31 | subprops = specprops 32 | displayName = name 33 | 34 | for key in keys: 35 | prop = subprops.get(key, {}) 36 | subprops = prop.get("items",{}).get("properties", {}) 37 | 38 | 39 | # Enqueue the sub properties (if present) 40 | for subprop in subprops.keys(): 41 | specpaths.append("{0}.{1}".format(path,subprop)) 42 | 43 | 44 | desc = descriptorMap.get(path,{}) 45 | # Construct description 46 | specdescriptors.append({ 47 | "displayName" : desc.get("displayName", displayName), 48 | "x-descriptors": desc.get("x-descriptors", []), 49 | "path" : path, 50 | "description" : prop.get("description", "") }) 51 | 52 | return specdescriptors 53 | 54 | def mapDescriptors(metaname, spec): 55 | specMap = {} 56 | statusMap = {} 57 | for resource in spec: 58 | if resource.get("name","") == metaname: 59 | for specD in resource.get("specDescriptors",[]): 60 | specMap[specD.get("path","")] = specD 61 | 62 | for statusD in resource.get("statusDescriptors",[]): 63 | statusMap[statusD.get("path","")] = statusD 64 | 65 | return (specMap, statusMap) 66 | 67 | def main(args): 68 | parser = argparse.ArgumentParser( 69 | description='''A hack to clone descriptions from the CRD.''') 70 | 71 | parser.add_argument( '--crd', metavar='crd', dest='crd', default=None, 72 | help='''The Custom Resource Definition File.''') 73 | 74 | parser.add_argument( '--version', metavar='CSV Version', dest='version', default=DEFAULT_VERSION, 75 | help='''The version of the CSV to update''') 76 | 77 | args = parser.parse_args() 78 | 79 | 80 | crdf = CRD_SOURCE_PATH.format(BASE_DIR, args.crd) 81 | csvf = CSV_PATH.format(BASE_DIR, args.version) 82 | 83 | crd = None 84 | csv = None 85 | try: 86 | with open(crdf, 'r') as stream: 87 | crd = yaml.safe_load(stream) 88 | with open(csvf, 'r') as stream: 89 | csv = yaml.safe_load(stream) 90 | except yaml.YAMLError as e: 91 | print(e) 92 | return 1 93 | 94 | if crd is not None and csv is not None: 95 | metaname= crd.get("metadata",{}).get("name", "") 96 | 97 | # TODO need to replace with something 98 | resourcefound=False 99 | owned = csv.get("spec",{}).get("customresourcedefinitions",{}).get("owned",{}) 100 | specmap, statusmap = mapDescriptors(metaname, owned) 101 | 102 | specdescriptors = loadDescriptors("spec", crd, csv, specmap) 103 | statusdescriptors = loadDescriptors("status", crd, csv, statusmap) 104 | 105 | for resource in owned: 106 | if resource.get("name","") == metaname: 107 | resourcefound = True 108 | resource["specDescriptors"] = specdescriptors 109 | resource["statusDescriptors"] = statusdescriptors 110 | resource["version"] = crd.get("spec",{}).get("version","v1") 111 | 112 | # If the resource wasn't present in the customresourcedefinitions add it. 113 | if not resourcefound: 114 | owned.append({ 115 | "name" : metaname, 116 | "kind" : crd.get("kind","CustomResourceDefinition"), 117 | "version" : crd.get("spec",{}).get("version","v1"), 118 | "displayName" : metaname, 119 | "specDescriptors" : specdescriptors, 120 | "description" : "TODO: Fill this in"}) 121 | 122 | # Copy the updated CSV 123 | with open(csvf, 'w') as outfile: 124 | yaml.dump(csv, outfile, default_flow_style=False) 125 | 126 | # Copy the CRD 127 | crdtarget=CRD_TARGET_PATH.format(BASE_DIR, args.version, os.path.basename(crdf)) 128 | with open(crdtarget, 'w') as outfile: 129 | yaml.dump(crd, outfile, default_flow_style=False) 130 | 131 | if __name__ == "__main__": 132 | sys.exit(main(sys.argv)) 133 | 134 | -------------------------------------------------------------------------------- /operator/config/samples/csi_v1_csiscaleoperator.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: csi.ibm.com/v1 3 | kind: "CSIScaleOperator" 4 | metadata: 5 | name: "ibm-spectrum-scale-csi" 6 | namespace: "ibm-spectrum-scale-csi-driver" 7 | labels: 8 | app.kubernetes.io/name: ibm-spectrum-scale-csi-operator 9 | app.kubernetes.io/instance: ibm-spectrum-scale-csi-operator 10 | app.kubernetes.io/managed-by: ibm-spectrum-scale-csi-operator 11 | release: ibm-spectrum-scale-csi-operator 12 | status: {} 13 | spec: 14 | 15 | # A passthrough option that distributes an imagePullSecrets array to the containers 16 | # generated by the csi scale operator. Please refer to official k8s documentation for 17 | # your environment for more details. https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ 18 | # ================================================================================== 19 | # imagePullSecrets: 20 | # - APullSecret 21 | # - AnotherOptional 22 | 23 | # Below specifies the details of a IBM Storage Scale cluster configuration used by the 24 | # plugin. It can have multiple values. 25 | # ================================================================================== 26 | clusters: 27 | - id: "" 28 | secrets: "secret1" 29 | secureSslMode: false 30 | primary: 31 | primaryFs: "< Primary Filesystem >" 32 | # primaryFset: "< Fileset in Primary Filesystem >" # Optional - default:spectrum-scale-csi-volume-store 33 | # inodeLimit: "< inode limit for Primary Fileset >" # Optional 34 | # remoteCluster: "< Remote ClusterID >" # Optional - This is only required if primaryFs is remote cluster's filesystem and this ID should have separate entry in Clusters map too. 35 | # cacert: "< Name of CA cert configmap for GUI >" # Optional 36 | restApi: 37 | - guiHost: "< Primary cluster GUI IP/Hostname >" 38 | # 39 | # In the case we have multiple clusters, specify their configuration below. 40 | # ================================================================================== 41 | # - id: "< Cluster ID >" 42 | # secrets: "< Secret for Cluster >" 43 | # secureSslMode: false 44 | # restApi: 45 | # - guiHost: "< Cluster GUI IP/Hostname >" 46 | # cacert: "< Name of CA cert configmap for GUI >" # Optional 47 | 48 | # attacherNodeSelector specifies on which nodes we want to run attacher sidecar 49 | # In below example attacher will run on nodes which have label as "scale=true" 50 | # and "infranode=2". Can have multiple entries. 51 | # ================================================================================== 52 | attacherNodeSelector: 53 | - key: "scale" 54 | value: "true" 55 | # - key: "infranode" 56 | # value: "2" 57 | 58 | # provisionerNodeSelector specifies on which nodes we want to run provisioner 59 | # sidecar. In below example provisioner will run on nodes which have label as 60 | # "scale=true" and "infranode=1". Can have multiple entries. 61 | # ================================================================================== 62 | provisionerNodeSelector: 63 | - key: "scale" 64 | value: "true" 65 | # - key: "infranode" 66 | # value: "1" 67 | 68 | # snapshotterNodeSelector specifies on which nodes we want to run snapshotter 69 | # sidecar. In below example snapshotter pod will run on nodes which have label as 70 | # "scale=true" and "infranode=1". Can have multiple entries. 71 | # ================================================================================== 72 | snapshotterNodeSelector: 73 | - key: "scale" 74 | value: "true" 75 | # - key: "infranode" 76 | # value: "1" 77 | 78 | # resizerNodeSelector specifies on which nodes we want to run resizer 79 | # sidecar. In below example resizer pod will run on nodes which have label as 80 | # "scale=true" and "infranode=1". Can have multiple entries. 81 | # ================================================================================== 82 | resizerNodeSelector: 83 | - key: "scale" 84 | value: "true" 85 | # - key: "infranode" 86 | # value: "1" 87 | 88 | # pluginNodeSelector specifies nodes on which we want to run plugin daemoset 89 | # In below example plugin daemonset will run on nodes which have label as 90 | # "scale=true". Can have multiple entries. 91 | # ================================================================================== 92 | pluginNodeSelector: 93 | - key: "scale" 94 | value: "true" 95 | 96 | # In case K8s nodes name differs from IBM Storage Scale nodes name, we can provide 97 | # node mapping using nodeMapping attribute. Can have multiple entries. 98 | # ================================================================================== 99 | # nodeMapping: 100 | # - k8sNode: "< K8s Node Name >" 101 | # spectrumscaleNode: "< IBM Storage Scale Node Name >" 102 | # In case K8s node name start with number then use following node mapping format. 103 | # - k8sNode: "K8sNodePrefix_< K8s Node Name >" 104 | # spectrumscaleNode: "< IBM Storage Scale Node Name >" 105 | 106 | # Array of tolerations that will be distribued to CSI pods. Please refer to official 107 | # k8s documentation for your environment for more details. 108 | # https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/ 109 | # ================================================================================== 110 | # tolerations: 111 | # - key: "key1" 112 | # operator: "Equal" 113 | # value: "value1" 114 | # effect: "NoExecute" 115 | # tolerationSeconds: 3600 116 | 117 | # Kubelet root directory path, in case we don't want to use the default kubelet 118 | # root directory path. 119 | # ================================================================================== 120 | # kubeletRootDirPath: "/var/lib/kubelet" 121 | -------------------------------------------------------------------------------- /operator/config/samples/csiscaleoperators.csi.ibm.com_cr.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: csi.ibm.com/v1 3 | kind: "CSIScaleOperator" 4 | metadata: 5 | name: "ibm-spectrum-scale-csi" 6 | namespace: "ibm-spectrum-scale-csi-driver" 7 | labels: 8 | app.kubernetes.io/name: ibm-spectrum-scale-csi-operator 9 | app.kubernetes.io/instance: ibm-spectrum-scale-csi-operator 10 | app.kubernetes.io/managed-by: ibm-spectrum-scale-csi-operator 11 | release: ibm-spectrum-scale-csi-operator 12 | status: {} 13 | spec: 14 | 15 | # A passthrough option that distributes an imagePullSecrets array to the containers 16 | # generated by the csi scale operator. Please refer to official k8s documentation for 17 | # your environment for more details. https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ 18 | # ================================================================================== 19 | # imagePullSecrets: 20 | # - APullSecret 21 | # - AnotherOptional 22 | 23 | # Below specifies the details of a IBM Storage Scale cluster configuration used by the 24 | # plugin. It can have multiple values. 25 | # ================================================================================== 26 | localScaleCluster: "< Scale Cluster ID >" # Local IBM Storage Scale cluster where Kubernetes or Red Hat OpenShift is running. 27 | clusters: 28 | - id: "< Scale Cluster ID >" 29 | secrets: "secret1" 30 | secureSslMode: false 31 | # cacert: "< Name of CA cert configmap for GUI >" # Optional 32 | restApi: 33 | - guiHost: "< IP/Hostname of a GUI node of the Scale cluster >" 34 | # - guiHost: "< IP/Hostname of another GUI node of the Scale cluster >" #Optional - Multiple GUI IPs/Hostnames can be provided like this, if the storage cluster has GUI installed on multiple nodes. 35 | # 36 | # In the case we have multiple clusters, specify their configuration below. 37 | # ================================================================================== 38 | # - id: "< Cluster ID >" 39 | # secrets: "< Secret for Cluster >" 40 | # secureSslMode: false 41 | # restApi: 42 | # - guiHost: "< IP/Hostname of a GUI node of the cluster >" 43 | # - guiHost: "< IP/Hostname of another GUI node of the cluster >" #Optinal - Multiple GUI IPs/Hostnames can be provided like this, if the storage cluster has GUI installed on multiple nodes. 44 | # cacert: "< Name of CA cert configmap for GUI >" # Optional 45 | 46 | # attacherNodeSelector specifies on which nodes we want to run attacher sidecar 47 | # In below example attacher will run on nodes which have label as "scale=true" 48 | # and "infranode=2". Can have multiple entries. 49 | # ================================================================================== 50 | attacherNodeSelector: 51 | - key: "scale" 52 | value: "true" 53 | # - key: "infranode" 54 | # value: "2" 55 | 56 | # provisionerNodeSelector specifies on which nodes we want to run provisioner 57 | # sidecar. In below example provisioner will run on nodes which have label as 58 | # "scale=true" and "infranode=1". Can have multiple entries. 59 | # ================================================================================== 60 | provisionerNodeSelector: 61 | - key: "scale" 62 | value: "true" 63 | # - key: "infranode" 64 | # value: "1" 65 | 66 | # snapshotterNodeSelector specifies on which nodes we want to run snapshotter 67 | # sidecar. In below example snapshotter pod will run on nodes which have label as 68 | # "scale=true" and "infranode=1". Can have multiple entries. 69 | # ================================================================================== 70 | snapshotterNodeSelector: 71 | - key: "scale" 72 | value: "true" 73 | # - key: "infranode" 74 | # value: "1" 75 | 76 | # resizerNodeSelector specifies on which nodes we want to run resizer 77 | # sidecar. In below example resizer pod will run on nodes which have label as 78 | # "scale=true" and "infranode=1". Can have multiple entries. 79 | # ================================================================================== 80 | resizerNodeSelector: 81 | - key: "scale" 82 | value: "true" 83 | # - key: "infranode" 84 | # value: "1" 85 | 86 | # pluginNodeSelector specifies nodes on which we want to run plugin daemoset 87 | # In below example plugin daemonset will run on nodes which have label as 88 | # "scale=true". Can have multiple entries. 89 | # ================================================================================== 90 | pluginNodeSelector: 91 | - key: "scale" 92 | value: "true" 93 | 94 | # In case K8s nodes name differs from IBM Storage Scale nodes name, we can provide 95 | # node mapping using nodeMapping attribute. Can have multiple entries. 96 | # ================================================================================== 97 | # nodeMapping: 98 | # - k8sNode: "< K8s Node Name >" 99 | # spectrumscaleNode: "< IBM Storage Scale Node Name >" 100 | # In case K8s node name start with number then use following node mapping format. 101 | # - k8sNode: "K8sNodePrefix_< K8s Node Name >" 102 | # spectrumscaleNode: "< IBM Storage Scale Node Name >" 103 | 104 | # Array of tolerations that will be distribued to CSI pods. Please refer to official 105 | # k8s documentation for your environment for more details. 106 | # https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/ 107 | # ================================================================================== 108 | # tolerations: 109 | # - key: "key1" 110 | # operator: "Equal" 111 | # value: "value1" 112 | # effect: "NoExecute" 113 | # tolerationSeconds: 3600 114 | 115 | # Kubelet root directory path, in case we don't want to use the default kubelet 116 | # root directory path. 117 | # ================================================================================== 118 | # kubeletRootDirPath: "/var/lib/kubelet" 119 | -------------------------------------------------------------------------------- /driver/cmd/ibm-spectrum-scale-csi/main.go: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright 2018, 2024 IBM Corporation. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"); 5 | you may not use this file except in compliance with the License. 6 | You may obtain a copy of the License at 7 | 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | 10 | Unless required by applicable law or agreed to in writing, software 11 | distributed under the License is distributed on an "AS IS" BASIS, 12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | See the License for the specific language governing permissions and 14 | limitations under the License. 15 | */ 16 | 17 | package main 18 | 19 | import ( 20 | "context" 21 | "flag" 22 | "fmt" 23 | "os" 24 | "path" 25 | "path/filepath" 26 | "strings" 27 | 28 | driver "github.com/IBM/ibm-spectrum-scale-csi/driver/csiplugin" 29 | "github.com/IBM/ibm-spectrum-scale-csi/driver/csiplugin/settings" 30 | "github.com/IBM/ibm-spectrum-scale-csi/driver/csiplugin/utils" 31 | "github.com/natefinch/lumberjack" 32 | "k8s.io/klog/v2" 33 | ) 34 | 35 | // gitCommit that is injected via go build -ldflags "-X main.gitCommit=$(git rev-parse HEAD)" 36 | var ( 37 | gitCommit string 38 | ) 39 | 40 | var ( 41 | endpoint = flag.String("endpoint", "unix://tmp/csi.sock", "CSI endpoint") 42 | driverName = flag.String("drivername", "spectrumscale.csi.ibm.com", "name of the driver") 43 | nodeID = flag.String("nodeid", "", "node id") 44 | kubeletRootDir = flag.String("kubeletRootDirPath", "/var/lib/kubelet", "kubelet root directory path") 45 | vendorVersion = "3.1.0" 46 | ) 47 | 48 | func main() { 49 | klog.InitFlags(nil) 50 | 51 | for _, key := range []string{utils.LogLevel, settings.PersistentLog, settings.NodePublishMethod, settings.VolumeStatsCapability, driver.VolNamePrefixEnvKey, settings.DiscoverCGFileset} { 52 | if val, ok := os.LookupEnv(key); ok { 53 | klog.Infof("[%s] found in the env : %s", key, val) 54 | } 55 | } 56 | 57 | level, persistentLogEnabled := getLogEnv() 58 | logValue := getLogLevel(level) 59 | value := getVerboseLevel(level) 60 | err := flag.Set("logtostderr", "false") 61 | err1 := flag.Set("stderrthreshold", logValue) 62 | err2 := flag.Set("v", value) 63 | flag.Parse() 64 | 65 | defer func() { 66 | if r := recover(); r != nil { 67 | klog.Infof("Recovered from panic: [%v]", r) 68 | } 69 | }() 70 | if persistentLogEnabled == "ENABLED" { 71 | fpClose := InitFileLogger() 72 | defer fpClose() 73 | } 74 | 75 | ctx := setContext() 76 | loggerId := utils.GetLoggerId(ctx) 77 | if err != nil || err1 != nil || err2 != nil { 78 | klog.Errorf("[%s] Failed to set flag value", loggerId) 79 | } 80 | 81 | klog.V(0).Infof("[%s] Version Info: commit (%s)", loggerId, gitCommit) 82 | 83 | // PluginFolder defines the location of scaleplugin 84 | PluginFolder := path.Join(*kubeletRootDir, "plugins/spectrumscale.csi.ibm.com") 85 | 86 | if err := createPersistentStorage(path.Join(PluginFolder, "controller")); err != nil { 87 | klog.Errorf("[%s] failed to create persistent storage for controller %v", loggerId, err) 88 | os.Exit(1) 89 | } 90 | if err := createPersistentStorage(path.Join(PluginFolder, "node")); err != nil { 91 | klog.Errorf("[%s] failed to create persistent storage for node %v", loggerId, err) 92 | os.Exit(1) 93 | } 94 | 95 | defer klog.Flush() 96 | handle(ctx) 97 | os.Exit(0) 98 | } 99 | 100 | func handle(ctx context.Context) { 101 | loggerId := utils.GetLoggerId(ctx) 102 | driver := driver.GetScaleDriver(ctx) 103 | err := driver.SetupScaleDriver(ctx, *driverName, vendorVersion, *nodeID) 104 | if err != nil { 105 | klog.Fatalf("[%s] Failed to initialize Scale CSI Driver: %v", loggerId, err) 106 | } 107 | newDriver := driver 108 | newDriver.PrintDriverInit(ctx) 109 | driver.Run(ctx, *endpoint) 110 | } 111 | 112 | func createPersistentStorage(persistentStoragePath string) error { 113 | if _, err := os.Stat(persistentStoragePath); os.IsNotExist(err) { 114 | if err := os.MkdirAll(persistentStoragePath, os.FileMode(0644)); err != nil { 115 | return err 116 | } 117 | } 118 | return nil 119 | } 120 | 121 | func setContext() context.Context { 122 | newCtx := context.Background() 123 | ctx := utils.SetLoggerId(newCtx) 124 | return ctx 125 | } 126 | 127 | func getLogEnv() (string, string) { 128 | level := os.Getenv(utils.LogLevel) 129 | persistentLogEnabled := os.Getenv(settings.PersistentLog) 130 | return strings.ToUpper(level), strings.ToUpper(persistentLogEnabled) 131 | } 132 | 133 | func getLogLevel(level string) string { 134 | var logValue string 135 | if level == utils.DEBUG.String() || level == utils.TRACE.String() { 136 | logValue = utils.INFO.String() 137 | } else { 138 | logValue = level 139 | } 140 | return logValue 141 | } 142 | 143 | func getVerboseLevel(level string) string { 144 | if level == utils.DEBUG.String() { 145 | return "4" 146 | } else if level == utils.TRACE.String() { 147 | return "6" 148 | } else { 149 | return "1" 150 | } 151 | } 152 | 153 | func InitFileLogger() func() { 154 | filePath := settings.HostPath + settings.DirPath + "/" + settings.LogFile 155 | _, err := os.Stat(filePath) 156 | if os.IsNotExist(err) { 157 | fileDir, _ := path.Split(filePath) 158 | /* #nosec G301 -- false positive */ 159 | err := os.MkdirAll(fileDir, 0755) 160 | if err != nil { 161 | panic(fmt.Sprintf("failed to create log folder %v", err)) 162 | } 163 | } 164 | 165 | /* #nosec G302 -- false positive */ 166 | logFile, err := os.OpenFile(filepath.Clean(filePath), os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0640) 167 | if err != nil { 168 | panic(fmt.Sprintf("failed to init logger %v", err)) 169 | } 170 | 171 | l := &lumberjack.Logger{ 172 | Filename: filePath, 173 | MaxSize: settings.RotateSize, 174 | MaxBackups: 5, 175 | MaxAge: 0, 176 | Compress: true, 177 | } 178 | klog.SetOutput(l) 179 | 180 | closeFn := func() { 181 | err := logFile.Close() 182 | if err != nil { 183 | panic(fmt.Sprintf("failed to close log file %v", err)) 184 | } 185 | } 186 | return closeFn 187 | } 188 | -------------------------------------------------------------------------------- /driver/csiplugin/settings/scale_config.go: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2019,2024 IBM Corp. 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package settings 18 | 19 | import ( 20 | "context" 21 | "crypto/x509" 22 | "encoding/json" 23 | "fmt" 24 | "io/fs" 25 | "os" 26 | "path" 27 | "path/filepath" 28 | "strings" 29 | 30 | "github.com/IBM/ibm-spectrum-scale-csi/driver/csiplugin/utils" 31 | "k8s.io/klog/v2" 32 | ) 33 | 34 | type ScaleSettingsConfigMap struct { 35 | LocalScaleCluster string `json:"localScaleCluster"` 36 | Clusters []Clusters 37 | } 38 | 39 | type Primary struct { 40 | PrimaryFSDep string `json:"primaryFS"` // Deprecated 41 | PrimaryFs string `json:"primaryFs"` //Deprecated 42 | PrimaryFset string `json:"primaryFset"` //Deprecated 43 | PrimaryCid string `json:"primaryCid"` 44 | InodeLimitDep string `json:"inode-limit"` // Deprecated 45 | InodeLimits string `json:"inodeLimit"` //Deprecated 46 | RemoteCluster string `json:"remoteCluster"` //Deprecated 47 | 48 | PrimaryFSMount string 49 | PrimaryFsetLink string 50 | SymlinkAbsolutePath string 51 | SymlinkRelativePath string 52 | } 53 | 54 | const ( 55 | secretFileSuffix = "-secret" // #nosec G101 false positive 56 | cacertFileSuffix = "-cacert" 57 | ) 58 | const ( 59 | DirPath = "scalecsilogs" 60 | LogFile = "ibm-spectrum-scale-csi.logs" 61 | PersistentLog = "PERSISTENT_LOG" 62 | NodePublishMethod = "NODEPUBLISH_METHOD" 63 | VolumeStatsCapability = "VOLUME_STATS_CAPABILITY" 64 | HostPath = "/host/var/adm/ras/" 65 | RotateSize = 1024 66 | DiscoverCGFileset = "DISCOVER_CG_FILESET" 67 | ) 68 | 69 | type RestAPI struct { 70 | GuiHost string `json:"guiHost"` 71 | GuiPort int `json:"guiPort"` 72 | } 73 | 74 | type Clusters struct { 75 | ID string `json:"id"` 76 | Primary Primary `json:"primary,omitempty"` 77 | SecureSslMode bool `json:"secureSslMode"` 78 | Cacert string `json:"cacert"` 79 | Secrets string `json:"secrets"` 80 | RestAPI []RestAPI `json:"restApi"` 81 | PrimaryCluster string `json:"primaryCluster"` 82 | 83 | MgmtUsername string 84 | MgmtPassword string 85 | CacertValue *x509.CertPool 86 | } 87 | 88 | const ( 89 | DefaultGuiPort int = 443 90 | GuiProtocol string = "https" 91 | ConfigMapFile string = "/var/lib/ibm/config/spectrum-scale-config.json" 92 | // #nosec G101 93 | SecretBasePath string = "/var/lib/ibm/" //nolint:gosec 94 | CertificatePath string = "/var/lib/ibm/ssl/public" 95 | S3Cache string = "S3" 96 | NfsCache string = "NFS" 97 | ) 98 | 99 | func LoadScaleConfigSettings(ctx context.Context) ScaleSettingsConfigMap { 100 | klog.V(6).Infof("[%s] scale_config LoadScaleConfigSettings", utils.GetLoggerId(ctx)) 101 | 102 | file, e := os.ReadFile(ConfigMapFile) // TODO 103 | if e != nil { 104 | klog.Errorf("[%s] IBM Storage Scale configuration not found: %v", utils.GetLoggerId(ctx), e) 105 | return ScaleSettingsConfigMap{} 106 | } 107 | cmsj := &ScaleSettingsConfigMap{} 108 | e = json.Unmarshal(file, cmsj) 109 | if e != nil { 110 | klog.Errorf("[%s] error in unmarshalling IBM Storage Scale configuration json: %v", utils.GetLoggerId(ctx), e) 111 | return ScaleSettingsConfigMap{} 112 | } 113 | 114 | e = HandleSecretsAndCerts(ctx, cmsj) 115 | if e != nil { 116 | klog.Errorf("[%s] error in secrets or certificates: %v", utils.GetLoggerId(ctx), e) 117 | return ScaleSettingsConfigMap{} 118 | } 119 | return *cmsj 120 | } 121 | 122 | func HandleSecretsAndCerts(ctx context.Context, cmap *ScaleSettingsConfigMap) error { 123 | klog.V(4).Infof("[%s] scale_config HandleSecrets", utils.GetLoggerId(ctx)) 124 | for i := 0; i < len(cmap.Clusters); i++ { 125 | if cmap.Clusters[i].Secrets != "" { 126 | unamePath := path.Join(SecretBasePath, cmap.Clusters[i].ID+secretFileSuffix, "username") 127 | file, e := os.ReadFile(unamePath) // #nosec G304 Valid Path is generated internally 128 | if e != nil { 129 | return fmt.Errorf("the IBM Storage Scale secret not found: %v", e) 130 | } 131 | file_s := string(file) 132 | file_s = strings.TrimSpace(file_s) 133 | file_s = strings.TrimSuffix(file_s, "\n") 134 | cmap.Clusters[i].MgmtUsername = file_s 135 | 136 | pwdPath := path.Join(SecretBasePath, cmap.Clusters[i].ID+secretFileSuffix, "password") 137 | file, e = os.ReadFile(pwdPath) // #nosec G304 Valid Path is generated internally 138 | if e != nil { 139 | return fmt.Errorf("the IBM Storage Scale secret not found: %v", e) 140 | } 141 | file_s = string(file) 142 | file_s = strings.TrimSpace(file_s) 143 | file_s = strings.TrimSuffix(file_s, "\n") 144 | cmap.Clusters[i].MgmtPassword = file_s 145 | } 146 | 147 | if cmap.Clusters[i].SecureSslMode && cmap.Clusters[i].Cacert != "" { 148 | certPath := path.Join(CertificatePath, cmap.Clusters[i].ID+cacertFileSuffix) 149 | caCertPool := x509.NewCertPool() 150 | 151 | // loop through directory and load all files as PEM certs 152 | err := filepath.WalkDir(certPath, func(path string, d fs.DirEntry, walkErr error) error { 153 | if walkErr != nil { 154 | return fmt.Errorf("failed to iterate through the IBM Storage Scale CA certificate directory - error: %v", walkErr) 155 | } 156 | // Skip directories and symlinks 157 | if d.IsDir() || d.Type()&os.ModeSymlink != 0 { 158 | return nil 159 | } 160 | 161 | // Read file contents 162 | data, err := os.ReadFile(path) 163 | if err != nil { 164 | return fmt.Errorf("failed to read the IBM Storage Scale CA certificate directory:%s - error: %v", path, err) 165 | } 166 | 167 | // append as PEM from the crt files 168 | if ok := caCertPool.AppendCertsFromPEM(data); !ok { 169 | // Not a valid cert, skip instead of failing 170 | return nil 171 | } 172 | return nil 173 | }) 174 | if err != nil { 175 | return fmt.Errorf("failed to iterate through the IBM Storage Scale CA certificate directory and get certificates - error: %v", err) 176 | } 177 | 178 | cmap.Clusters[i].CacertValue = caCertPool 179 | } 180 | } 181 | return nil 182 | } 183 | -------------------------------------------------------------------------------- /operator/main.go: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright 2022, 2024 IBM Corp. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"); 5 | you may not use this file except in compliance with the License. 6 | You may obtain a copy of the License at 7 | 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | 10 | Unless required by applicable law or agreed to in writing, software 11 | distributed under the License is distributed on an "AS IS" BASIS, 12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | See the License for the specific language governing permissions and 14 | limitations under the License. 15 | */ 16 | 17 | package main 18 | 19 | import ( 20 | "flag" 21 | "fmt" 22 | "os" 23 | "strconv" 24 | 25 | // Import all Kubernetes client auth plugins (e.g. Azure, GCP, OIDC, etc.) 26 | // to ensure that exec-entrypoint and run can make use of them. 27 | "go.uber.org/zap/zapcore" 28 | _ "k8s.io/client-go/plugin/pkg/client/auth" 29 | "k8s.io/client-go/rest" 30 | 31 | "k8s.io/apimachinery/pkg/runtime" 32 | utilruntime "k8s.io/apimachinery/pkg/util/runtime" 33 | clientgoscheme "k8s.io/client-go/kubernetes/scheme" 34 | ctrl "sigs.k8s.io/controller-runtime" 35 | ctrlmetricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server" 36 | ctrlwebhook "sigs.k8s.io/controller-runtime/pkg/webhook" 37 | 38 | "sigs.k8s.io/controller-runtime/pkg/cache" 39 | "sigs.k8s.io/controller-runtime/pkg/healthz" 40 | "sigs.k8s.io/controller-runtime/pkg/log/zap" 41 | 42 | csiv1 "github.com/IBM/ibm-spectrum-scale-csi/operator/api/v1" 43 | "github.com/IBM/ibm-spectrum-scale-csi/operator/controllers" 44 | 45 | configv1 "github.com/openshift/api/config/v1" 46 | securityv1 "github.com/openshift/api/security/v1" 47 | //+kubebuilder:scaffold:imports 48 | ) 49 | 50 | const OCPControllerNamespace = "openshift-controller-manager" 51 | 52 | // gitCommit that is injected via go build -ldflags "-X main.gitCommit=$(git rev-parse HEAD)" 53 | var ( 54 | gitCommit string 55 | ) 56 | 57 | var ( 58 | scheme = runtime.NewScheme() 59 | setupLog = ctrl.Log.WithName("setup") 60 | ) 61 | 62 | func init() { 63 | utilruntime.Must(clientgoscheme.AddToScheme(scheme)) 64 | 65 | utilruntime.Must(csiv1.AddToScheme(scheme)) 66 | utilruntime.Must(securityv1.AddToScheme(scheme)) 67 | utilruntime.Must(configv1.AddToScheme(scheme)) 68 | //+kubebuilder:scaffold:scheme 69 | } 70 | 71 | // getMetricsBindAddress returns the metrics bind address for the operator 72 | func getMetricsBindAddress() string { 73 | var metricsBindAddrEnvVar = "METRICS_BIND_ADDRESS" 74 | 75 | defaultBindAddr := ":8383" 76 | 77 | bindAddr, found := os.LookupEnv(metricsBindAddrEnvVar) 78 | if found { 79 | _, err := strconv.Atoi(bindAddr) 80 | if err != nil { 81 | msg := fmt.Errorf("%s %s: %s", "supplied METRICS_BIND_ADDRESS is not a number", "METRICS_BIND_ADDRESS", bindAddr) 82 | setupLog.Error(msg, "Using default METRICS_BIND_ADDRESS: 8383") 83 | return defaultBindAddr 84 | } else { 85 | return ":" + bindAddr 86 | } 87 | } 88 | return defaultBindAddr 89 | } 90 | 91 | // getWatchNamespace returns the Namespace the operator should be watching for changes 92 | func getWatchNamespace() (string, error) { 93 | // WatchNamespaceEnvVar is the constant for env variable WATCH_NAMESPACE 94 | // which specifies the Namespace to watch. 95 | // An empty value means the operator is running with cluster scope. 96 | var watchNamespaceEnvVar = "WATCH_NAMESPACE" 97 | 98 | ns, found := os.LookupEnv(watchNamespaceEnvVar) 99 | if !found { 100 | return "", fmt.Errorf("%s %s", "did not find WATCH_NAMESPACE", "Environment variable WATCH_NAMESPACE must be set") 101 | } 102 | return ns, nil 103 | } 104 | 105 | func main() { 106 | var metricsAddr string 107 | var enableLeaderElection bool 108 | // var probeAddr string 109 | 110 | bindAddr := getMetricsBindAddress() 111 | 112 | flag.StringVar(&metricsAddr, "metrics-bind-address", bindAddr, "The address the metric endpoint binds to.") 113 | // flag.StringVar(&probeAddr, "health-probe-bind-address", ":8081", "The address the probe endpoint binds to.") 114 | flag.BoolVar(&enableLeaderElection, "leaderElection", false, 115 | "Enable leader election for controller manager. "+ 116 | "Enabling this will ensure there is only one active controller manager.") 117 | 118 | opts := zap.Options{ 119 | Development: true, 120 | TimeEncoder: zapcore.ISO8601TimeEncoder, 121 | } 122 | opts.BindFlags(flag.CommandLine) 123 | flag.Parse() 124 | ctrl.SetLogger(zap.New(zap.UseFlagOptions(&opts))) 125 | 126 | setupLog.Info("Version Info", "commit", gitCommit) 127 | 128 | watchNamespace, err := getWatchNamespace() 129 | if err != nil { 130 | setupLog.Error(err, "unable to get WatchNamespace, "+ 131 | "the manager will watch and manage resources in all namespaces") 132 | } 133 | 134 | newCache := func(config *rest.Config, opts cache.Options) (cache.Cache, error) { 135 | opts.DefaultNamespaces = map[string]cache.Config{ 136 | watchNamespace: {}, 137 | OCPControllerNamespace: {}, 138 | } 139 | return cache.New(config, opts) 140 | } 141 | 142 | mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{ 143 | Scheme: scheme, 144 | Metrics: ctrlmetricsserver.Options{ 145 | BindAddress: metricsAddr, 146 | }, 147 | WebhookServer: ctrlwebhook.NewServer(ctrlwebhook.Options{ 148 | Port: 9443, 149 | }), 150 | // HealthProbeBindAddress: probeAddr, 151 | LeaderElection: enableLeaderElection, 152 | LeaderElectionID: "ibm-spectrum-scale-csi-operator", 153 | LeaderElectionNamespace: watchNamespace, // TODO: Flag should be set to select the namespace where operator is running. Needed for running operator locally. 154 | NewCache: newCache}, 155 | ) 156 | if err != nil { 157 | setupLog.Error(err, "unable to start manager") 158 | os.Exit(1) 159 | } 160 | 161 | if err = (&controllers.CSIScaleOperatorReconciler{ 162 | Client: mgr.GetClient(), 163 | Scheme: mgr.GetScheme(), 164 | Recorder: mgr.GetEventRecorderFor(controllers.CSIScaleOperatorControllerName), 165 | }).SetupWithManager(mgr); err != nil { 166 | setupLog.Error(err, "unable to create controller", "controller", controllers.CSIScaleOperatorControllerName) 167 | os.Exit(1) 168 | } 169 | //+kubebuilder:scaffold:builder 170 | 171 | if err := mgr.AddHealthzCheck("healthz", healthz.Ping); err != nil { 172 | setupLog.Error(err, "unable to set up health check") 173 | os.Exit(1) 174 | } 175 | if err := mgr.AddReadyzCheck("readyz", healthz.Ping); err != nil { 176 | setupLog.Error(err, "unable to set up ready check") 177 | os.Exit(1) 178 | } 179 | 180 | setupLog.Info("starting manager") 181 | if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil { 182 | setupLog.Error(err, "problem running manager") 183 | os.Exit(1) 184 | } 185 | } 186 | -------------------------------------------------------------------------------- /tools/generate_volsnapcontent_yaml.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Copyright 2019, 2024 IBM Corp. 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | # 17 | 18 | usage(){ 19 | echo "Usage: $0 20 | -f|--filesystem 21 | -F|--fileset 22 | -s|--snapshot 23 | [-p|--path ] 24 | [-c|--snapshotcontentname ] 25 | [-v|--snapshotname ] 26 | [-n|--namespace ] 27 | [-h|--help] " 1>&2; exit 1; } 28 | 29 | # Generate Yaml 30 | generate_yaml() 31 | { 32 | snaphandle=$1 33 | snapcontentname=$2 34 | snapname=$3 35 | namespace=$4 36 | if [[ -f "${snapname}.yaml" ]]; then 37 | echo "ERROR: File ${snapname}.yaml already exist" 38 | exit 2 39 | fi 40 | 41 | /usr/bin/cat > ${snapcontentname}.yaml <&2; usage ; exit 1 ; fi 67 | [[ $# -lt 1 ]] && usage 68 | 69 | eval set -- "$OPTS" 70 | 71 | while true ; do 72 | case "$1" in 73 | -h | --help ) 74 | usage 75 | ;; 76 | -F | --fileset ) 77 | FSETNAME="$2" 78 | shift 2 79 | ;; 80 | -f | --filesystem ) 81 | FSNAME="$2" 82 | shift 2 83 | ;; 84 | -s | --snapshot ) 85 | SNAPSHOT="$2" 86 | shift 2 87 | ;; 88 | -p | --path ) 89 | SNAPPATH="$2" 90 | shift 2 91 | ;; 92 | -c | --snapshotcontentname ) 93 | SNAPCONNAME="$2" 94 | shift 2 95 | ;; 96 | -v | --volumesnapname ) 97 | SNAPNAME="$2" 98 | shift 2 99 | ;; 100 | -n | --namespace ) 101 | NAMESPACE="$2" 102 | shift 2 103 | ;; 104 | -- ) 105 | shift 106 | break 107 | ;; 108 | *) 109 | usage 110 | exit 1 111 | ;; 112 | esac 113 | done 114 | 115 | 116 | # Check for mandatory Params 117 | MPARAM="" 118 | [[ -z "${FSNAME}" ]] && MPARAM="${MPARAM}--filesystem " 119 | [[ -z "${FSETNAME}" ]] && MPARAM="${MPARAM}--fileset " 120 | [[ -z "${SNAPSHOT}" ]] && MPARAM="${MPARAM}--snapshot " 121 | 122 | if [ ! -z "$MPARAM" ]; then 123 | echo "ERROR: Mandatory parameter missing : $MPARAM" 124 | usage 125 | fi 126 | 127 | if [ -z "${SNAPCONNAME}" ] ; then 128 | SNAPCONNAME="snapshotcontent-${SNAPSHOT}" 129 | if [[ ${#SNAPCONNAME} -ge 254 ]]; then 130 | echo "ERROR: Specify name for volumeSnapshotContent using option --snapshot" 131 | exit 2 132 | fi 133 | elif [[ ${#SNAPCONNAME} -ge 254 ]]; then 134 | echo "ERROR: volumeSnapshotContent name specified against option --snapshotcontentname must be less than 254 characters" 135 | exit 2 136 | fi 137 | 138 | if ! [[ "${SNAPCONNAME}" =~ ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ ]]; then 139 | echo "ERROR: Invalid volumeSnapshotContent name ${SNAPCONNAME}. volumeSnapshotContent name must satisfy DNS-1123 label requirement." 140 | exit 2 141 | fi 142 | 143 | if [ -z "${SNAPNAME}" ] ; then 144 | SNAPNAME="snapshot-${SNAPSHOT}" 145 | if [[ ${#SNAPNAME} -ge 254 ]]; then 146 | echo "ERROR: Specify name for volumeSnapshot using option --snapshot" 147 | exit 2 148 | fi 149 | fi 150 | 151 | if ! [[ "${SNAPNAME}" =~ ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ ]]; then 152 | echo "ERROR: Invalid volumeSnapshot name specified. volumeSnapshot name must satisfy DNS-1123 label requirement." 153 | exit 2 154 | fi 155 | 156 | [[ -z "${NAMESPACE}" ]] && NAMESPACE="default" 157 | 158 | # Check if this is IBM Storage Scale node 159 | if [[ ! -f /usr/lpp/mmfs/bin/mmlscluster ]] ; then 160 | echo "ERROR: IBM Storage Scale cli's are not present on this node" 161 | exit 2 162 | fi 163 | 164 | echo > ${ERROROUT} 165 | 166 | # Get the IBM Storage Scale cluster ID 167 | clusterID=`/usr/lpp/mmfs/bin/mmlscluster -Y 2>${ERROROUT} | /usr/bin/grep clusterSummary | /usr/bin/grep -v HEADER | /usr/bin/awk '{split($0,a,":"); print a[8]}'` 168 | if [[ $? -ne 0 ]] || [[ -z "$clusterID" ]]; then 169 | echo "ERROR: Failed to get the IBM Storage Scale cluster ID" 170 | /usr/bin/cat ${ERROROUT} 171 | exit 2 172 | fi 173 | 174 | # Get the Fileystem ID 175 | fileSystemID=`/usr/lpp/mmfs/bin/mmlsfs ${FSNAME} --uid 2>${ERROROUT} | /usr/bin/tail -1 | /usr/bin/awk '{split($0,a," "); print a[2]}'` 176 | if [[ $? -ne 0 ]] || [[ -z "$fileSystemID" ]]; then 177 | echo "ERROR: Failed to get the Fileystem ID of ${FSNAME}" 178 | /usr/bin/cat ${ERROROUT} 179 | exit 2 180 | fi 181 | 182 | # Check if fileset exists 183 | /usr/lpp/mmfs/bin/mmlsfileset ${FSNAME} ${FSETNAME} 1>/dev/null 2>${ERROROUT} 184 | if [[ $? -ne 0 ]]; then 185 | echo "ERROR: Fileset ${FSETNAME} could not be found in filesystem ${FSNAME}" 186 | cat ${ERROROUT} 187 | exit 2 188 | fi 189 | 190 | # Check if snapshot exists 191 | /usr/lpp/mmfs/bin/mmlssnapshot ${FSNAME} -s ${FSETNAME}:${SNAPSHOT} 1>/dev/null 2>${ERROROUT} 192 | if [[ $? -ne 0 ]]; then 193 | echo "ERROR: Snapshot ${FSETNAME}:${SNAPSHOT} could not be found in filesystem ${FSNAME}" 194 | cat ${ERROROUT} 195 | exit 2 196 | fi 197 | 198 | if [ -z "${SNAPPATH}" ] ; then 199 | # Generate Volume Handle 200 | SnapshotHandle="${clusterID};${fileSystemID};${FSETNAME};${SNAPSHOT}" 201 | else 202 | SnapshotHandle="${clusterID};${fileSystemID};${FSETNAME};${SNAPSHOT};${SNAPPATH}" 203 | fi 204 | 205 | # Gererate yaml file 206 | generate_yaml "${SnapshotHandle}" "${SNAPCONNAME}" "${SNAPNAME}" "${NAMESPACE}" 207 | 208 | /usr/bin/rm -f ${ERROROUT} 209 | exit 0 210 | -------------------------------------------------------------------------------- /operator/controllers/internal/csiscaleoperator/csiscaleoperator.go: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright 2022, 2024 IBM Corp. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"); 5 | you may not use this file except in compliance with the License. 6 | You may obtain a copy of the License at 7 | 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | 10 | Unless required by applicable law or agreed to in writing, software 11 | distributed under the License is distributed on an "AS IS" BASIS, 12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | See the License for the specific language governing permissions and 14 | limitations under the License. 15 | */ 16 | 17 | package csiscaleoperator 18 | 19 | import ( 20 | "k8s.io/apimachinery/pkg/labels" 21 | "sigs.k8s.io/controller-runtime/pkg/log" 22 | 23 | csiv1 "github.com/IBM/ibm-spectrum-scale-csi/operator/api/v1" 24 | "github.com/IBM/ibm-spectrum-scale-csi/operator/controllers/config" 25 | ) 26 | 27 | var csiLog = log.Log.WithName("csiscaleoperator") 28 | 29 | // CSIScaleOperator is the wrapper for csiv1.CSIScaleOperator type 30 | type CSIScaleOperator struct { 31 | *csiv1.CSIScaleOperator 32 | // ServerVersion string 33 | } 34 | 35 | // New returns a wrapper for csiv1.CSIScaleOperator 36 | func New(c *csiv1.CSIScaleOperator) *CSIScaleOperator { 37 | return &CSIScaleOperator{ 38 | CSIScaleOperator: c, 39 | } 40 | } 41 | 42 | // Unwrap returns the csiv1.CSIScaleOperator object 43 | func (c *CSIScaleOperator) Unwrap() *csiv1.CSIScaleOperator { 44 | return c.CSIScaleOperator 45 | } 46 | 47 | // GetLabels returns all the labels to be set on all resources 48 | // func (c *CSIScaleOperator) GetLabels() labels.Set { 49 | func (c *CSIScaleOperator) GetLabels() map[string]string { 50 | labels := labels.Set{ 51 | config.LabelAppName: config.ResourceAppName, 52 | config.LabelAppInstance: config.ResourceInstance, 53 | config.LabelAppManagedBy: config.ResourceManagedBy, 54 | config.LabelProduct: config.Product, 55 | config.LabelRelease: config.Release, 56 | } 57 | 58 | if c.Labels != nil { 59 | for k, v := range c.Labels { 60 | if !labels.Has(k) { 61 | labels[k] = v 62 | } 63 | } 64 | } 65 | 66 | return labels 67 | } 68 | 69 | // GetAnnotations returns all the annotations to be set on all resources 70 | // func (c *CSIScaleOperator) GetAnnotations(daemonSetRestartedKey string, daemonSetRestartedValue string) labels.Set { 71 | func (c *CSIScaleOperator) GetAnnotations(daemonSetRestartedKey string, daemonSetRestartedValue string) map[string]string { 72 | //func (c *CSIScaleOperator) GetAnnotations() map[string]string { 73 | labels := labels.Set{ 74 | "productID": config.ID, 75 | "productName": config.ProductName, 76 | "productVersion": config.DriverVersion, 77 | } 78 | 79 | if c.Annotations != nil { 80 | for k, v := range c.Annotations { 81 | if !labels.Has(k) { 82 | labels[k] = v 83 | } 84 | } 85 | } 86 | 87 | // removing the annotations that are not required in the daemonset/statefulset and their pods. 88 | _, ok := c.Annotations["kubectl.kubernetes.io/last-applied-configuration"] 89 | if ok { 90 | delete(c.Annotations, "kubectl.kubernetes.io/last-applied-configuration") 91 | } 92 | 93 | if !labels.Has(daemonSetRestartedKey) && daemonSetRestartedKey != "" { 94 | labels[daemonSetRestartedKey] = daemonSetRestartedValue 95 | } 96 | 97 | return labels 98 | } 99 | 100 | // GetSelectorLabels returns labels used in label selectors 101 | func (c *CSIScaleOperator) GetSelectorLabels(appName string) labels.Set { 102 | return labels.Set{ 103 | // "app.kubernetes.io/component": component, 104 | config.LabelProduct: config.Product, 105 | config.LabelApp: appName, 106 | } 107 | } 108 | 109 | func (c *CSIScaleOperator) GetCSIControllerSelectorLabels(appName string) labels.Set { 110 | return c.GetSelectorLabels(appName) 111 | } 112 | 113 | func (c *CSIScaleOperator) GetCSINodeSelectorLabels(appName string) labels.Set { 114 | return c.GetSelectorLabels(appName) 115 | } 116 | 117 | func (c *CSIScaleOperator) GetCSIControllerPodLabels(appName string) labels.Set { 118 | return labels.Merge(c.GetLabels(), c.GetCSIControllerSelectorLabels(appName)) 119 | } 120 | 121 | func (c *CSIScaleOperator) GetCSINodePodLabels(appName string) labels.Set { 122 | return labels.Merge(c.GetLabels(), c.GetCSINodeSelectorLabels(appName)) 123 | } 124 | 125 | func (c *CSIScaleOperator) GetDefaultImage(name string) string { 126 | 127 | logger := csiLog.WithName("GetDefaultImage") 128 | logger.Info("Getting default image for ", "container:", name) 129 | 130 | image := "" 131 | switch name { 132 | case config.CSINodeDriverRegistrar: 133 | image = config.CSINodeDriverRegistrarImage 134 | case config.CSINodeDriverPlugin: 135 | image = config.CSIDriverPluginImage 136 | case config.LivenessProbe: 137 | image = config.LivenessProbeImage 138 | case config.CSIProvisioner: 139 | image = config.CSIProvisionerImage 140 | case config.CSIAttacher: 141 | image = config.CSIAttacherImage 142 | case config.CSISnapshotter: 143 | image = config.CSISnapshotterImage 144 | case config.CSIResizer: 145 | image = config.CSIResizerImage 146 | } 147 | logger.Info("Got default image for ", "container:", name, ", image:", image) 148 | return image 149 | } 150 | 151 | // GetKubeletPodsDir returns the kubelet pods directory 152 | // TODO: Unexport these functions to fix lint warnings 153 | func (c *CSIScaleOperator) GetKubeletPodsDir() string { 154 | logger := csiLog.WithName("GetKubeletPodsDir") 155 | kubeletPodsDir := c.GetKubeletRootDirPath() + "/pods" 156 | logger.Info("GetKubeletPodsDir", "kubeletPodsDir: ", kubeletPodsDir) 157 | return kubeletPodsDir 158 | } 159 | 160 | func (c *CSIScaleOperator) GetKubeletRootDirPath() string { 161 | logger := csiLog.WithName("GetKubeletRootDirPath") 162 | 163 | if c.Spec.KubeletRootDirPath == "" { 164 | logger.Info("In GetKubeletRootDirPath", "using default kubeletRootDirPath: ", config.CSIKubeletRootDirPath) 165 | return config.CSIKubeletRootDirPath 166 | } 167 | logger.Info("In GetKubeletRootDirPath", "using kubeletRootDirPath: ", c.Spec.KubeletRootDirPath) 168 | return c.Spec.KubeletRootDirPath 169 | } 170 | 171 | func (c *CSIScaleOperator) GetSocketPath() string { 172 | logger := csiLog.WithName("GetSocketPath") 173 | 174 | socketPath := c.GetKubeletRootDirPath() + config.SocketPath 175 | logger.Info("In GetSocketPath", "socketPath", socketPath) 176 | return socketPath 177 | } 178 | 179 | func (c *CSIScaleOperator) GetSocketDir() string { 180 | logger := csiLog.WithName("GetSocketDir") 181 | 182 | socketDir := c.GetKubeletRootDirPath() + config.SocketDir 183 | logger.Info("In GetSocketDir", "socketDir", socketDir) 184 | return socketDir 185 | } 186 | 187 | func (c *CSIScaleOperator) GetCSIEndpoint() string { 188 | logger := csiLog.WithName("GetCSIEndpoint") 189 | 190 | CSIEndpoint := "unix://" + c.GetKubeletRootDirPath() + config.SocketPath 191 | logger.Info("In GetCSIEndpoint", "CSIEndpoint", CSIEndpoint) 192 | return CSIEndpoint 193 | } 194 | -------------------------------------------------------------------------------- /driver/csiplugin/utils/utils.go: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2016, 2024 IBM Corp. 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | 17 | package utils 18 | 19 | import ( 20 | "context" 21 | "fmt" 22 | "io" 23 | "os" 24 | "strconv" 25 | "strings" 26 | "time" 27 | 28 | "github.com/google/uuid" 29 | "golang.org/x/sys/unix" 30 | "k8s.io/klog/v2" 31 | ) 32 | 33 | type loggerKey string 34 | type moduleKey string 35 | 36 | const loggerId loggerKey = "logger_id" 37 | const moduleName moduleKey = "module_name" 38 | 39 | type LoggerLevel int 40 | 41 | const LogLevel = "LOGLEVEL" 42 | 43 | const ( 44 | TRACE LoggerLevel = iota 45 | DEBUG 46 | INFO 47 | WARNING 48 | ERROR 49 | FATAL 50 | ) 51 | 52 | func (level LoggerLevel) String() string { 53 | switch level { 54 | case TRACE: 55 | return "TRACE" 56 | case DEBUG: 57 | return "DEBUG" 58 | case WARNING: 59 | return "WARNING" 60 | case ERROR: 61 | return "ERROR" 62 | case FATAL: 63 | return "FATAL" 64 | case INFO: 65 | return "INFO" 66 | default: 67 | return "INFO" 68 | } 69 | } 70 | 71 | func ReadFile(path string) ([]byte, error) { 72 | klog.V(6).Infof("utils ReadFile. path: %s", path) 73 | 74 | file, err := os.Open(path) // #nosec G304 This is valid path gererated internally. it is False positive 75 | if err != nil { 76 | klog.Errorf("Error in opening file %s: %v", path, err) 77 | return nil, err 78 | } 79 | 80 | defer func() { 81 | if err := file.Close(); err != nil { 82 | klog.Errorf("Error in closing file %s: %v", path, err) 83 | } 84 | }() 85 | 86 | bytes, err := io.ReadAll(file) 87 | if err != nil { 88 | klog.Errorf("Error in read file %s: %v", path, err) 89 | return nil, err 90 | } 91 | 92 | return bytes, nil 93 | } 94 | 95 | func GetPath(paths []string) string { 96 | klog.V(6).Infof("utils GetPath. paths: %v", paths) 97 | 98 | workDirectory, _ := os.Getwd() 99 | 100 | if len(paths) == 0 { 101 | return workDirectory 102 | } 103 | 104 | resultPath := workDirectory 105 | 106 | for _, path := range paths { 107 | resultPath += string(os.PathSeparator) 108 | resultPath += path 109 | } 110 | 111 | return resultPath 112 | } 113 | 114 | func Exists(path string) bool { 115 | // klog.V(6).Infof("[%s] utils Exists. path: %s", GetLoggerId(ctx), path) 116 | if _, err := os.Stat(path); os.IsNotExist(err) { 117 | return false 118 | } 119 | return true 120 | } 121 | 122 | func MkDir(path string) error { 123 | // klog.V(6).Infof("[%s] utils MkDir. path: %s", GetLoggerId(ctx), path) 124 | var err error 125 | if _, err = os.Stat(path); os.IsNotExist(err) { 126 | err = os.MkdirAll(path, 0700) 127 | if err != nil { 128 | klog.Errorf("Error in creating dir %s: %v", path, err) 129 | return err 130 | } 131 | } 132 | 133 | return err 134 | } 135 | 136 | func StringInSlice(a string, list []string) bool { 137 | klog.V(6).Infof("utils StringInSlice. string: %s, slice: %v", a, list) 138 | for _, b := range list { 139 | if strings.EqualFold(b, a) { 140 | return true 141 | } 142 | } 143 | return false 144 | } 145 | 146 | func ConvertToBytes(inputStr string) (uint64, error) { 147 | klog.V(6).Infof("utils ConvertToBytes. string: %s", inputStr) 148 | var Iter int 149 | var byteSlice []byte 150 | var retValue uint64 151 | var uintMax64 uint64 152 | 153 | byteSlice = []byte(inputStr) 154 | uintMax64 = (1 << 64) - 1 155 | 156 | for Iter = 0; Iter < len(byteSlice); Iter++ { 157 | if ('0' <= byteSlice[Iter]) && 158 | (byteSlice[Iter] <= '9') { 159 | continue 160 | } else { 161 | break 162 | } 163 | } 164 | 165 | if Iter == 0 { 166 | return 0, fmt.Errorf("invalid number specified %v", inputStr) 167 | } 168 | 169 | retValue, err := strconv.ParseUint(inputStr[:Iter], 10, 64) 170 | 171 | if err != nil { 172 | return 0, fmt.Errorf("ParseUint Failed for %v", inputStr[:Iter]) 173 | } 174 | 175 | if Iter == len(inputStr) { 176 | return retValue, nil 177 | } 178 | 179 | unit := strings.TrimSpace(string(byteSlice[Iter:])) 180 | unit = strings.ToLower(unit) 181 | 182 | switch unit { 183 | case "b", "bytes": 184 | /* Nothing to do here */ 185 | case "k", "kb", "kilobytes", "kilobyte": 186 | retValue *= 1024 187 | case "m", "mb", "megabytes", "megabyte": 188 | retValue *= (1024 * 1024) 189 | case "g", "gb", "gigabytes", "gigabyte": 190 | retValue *= (1024 * 1024 * 1024) 191 | case "t", "tb", "terabytes", "terabyte": 192 | retValue *= (1024 * 1024 * 1024 * 1024) 193 | default: 194 | return 0, fmt.Errorf("invalid Unit %v supplied with %v", unit, inputStr) 195 | } 196 | 197 | if retValue > uintMax64 { 198 | return 0, fmt.Errorf("overflow detected %v", inputStr) 199 | } 200 | 201 | return retValue, nil 202 | } 203 | 204 | func GetEnv(envName string, defaultValue string) string { 205 | klog.V(6).Infof("utils GetEnv. envName: %s", envName) 206 | envValue := os.Getenv(envName) 207 | if envValue == "" { 208 | envValue = defaultValue 209 | } 210 | return envValue 211 | } 212 | 213 | func FsStatInfo(path string) (int64, int64, int64, int64, int64, int64, error) { 214 | klog.V(6).Infof("utils FsStatInfo. envName: %s", path) 215 | statfs := &unix.Statfs_t{} 216 | err := unix.Statfs(path, statfs) 217 | 218 | if err != nil { 219 | return 0, 0, 0, 0, 0, 0, err 220 | } 221 | available := int64(statfs.Bavail) * int64(statfs.Bsize) // #nosec G115 -- false positive 222 | capacity := int64(statfs.Blocks) * int64(statfs.Bsize) // #nosec G115 -- false positive 223 | usage := (int64(statfs.Blocks) - int64(statfs.Bfree)) * int64(statfs.Bsize) // #nosec G115 -- false positive 224 | inodes := int64(statfs.Files) // #nosec G115 -- false positive 225 | inodesFree := int64(statfs.Ffree) // #nosec G115 -- false positive 226 | inodesUsed := inodes - inodesFree 227 | 228 | return available, capacity, usage, inodes, inodesFree, inodesUsed, nil 229 | } 230 | 231 | func SetLoggerId(ctx context.Context) context.Context { 232 | id := uuid.New().String() 233 | return context.WithValue(ctx, loggerId, id) 234 | } 235 | 236 | func GetLoggerId(ctx context.Context) string { 237 | logger, _ := ctx.Value(loggerId).(string) 238 | return logger 239 | } 240 | 241 | func GetExecutionTime() int64 { 242 | t := time.Now() 243 | timeinMilliSec := int64(time.Nanosecond) * t.UnixNano() / int64(time.Millisecond) 244 | return timeinMilliSec 245 | } 246 | 247 | 248 | func SetModuleName(ctx context.Context, name string) context.Context { 249 | return context.WithValue(ctx, moduleName, name) 250 | } 251 | 252 | func GetModuleName(ctx context.Context) string { 253 | moduleName, _ := ctx.Value(moduleName).(string) 254 | return moduleName 255 | } 256 | 257 | -------------------------------------------------------------------------------- /tools/volume_migration_scripts/README_migration_csi_primary_removal.md: -------------------------------------------------------------------------------- 1 | 2 | # Migration Script – IBM Storage Scale CSI (Primary filesystem mount path to Actual fileset mount path) 3 | 4 | This script helps to migrate existing Kubernetes **PersistentVolumes (PVs)** that were originally created when the **primary filesystem and fileset** was enabled, to a format that uses the **actual fileset mount path** after the **primary filesystem/fileset has been removed**. 5 | 6 | It ensures that workloads continue to access their data seamlessly after migration. 7 | 8 | ## 1. Key Features 9 | 10 | - Preserves all **original PV properties**, including: 11 | - Reclaim policy (e.g., `Retain`, `Delete`) 12 | - Access modes (`ReadWriteOnce`, `ReadWriteMany`, etc.) 13 | - Storage capacity 14 | - Filesystem type (`fsType`) 15 | - Labels, annotations, and other PV metadata 16 | 17 | - Only the **volumeHandle path segment** is updated to a new IBM Storage Scale CSI required format. 18 | - Supports migration of PVs across **different fileset types** (independent, dependent, static, cache, CG, etc.). 19 | - Generates **backup YAML files** before applying changes. 20 | - Can be safely re-run if required (**idempotent migration**). 21 | 22 | ## 2. Why Migration is Required 23 | 24 | Till IBM Storage Scale CSI 2.14.x , the **primary filesystem and fileset** hierachy were being used, and the PVs were created under paths tied to the **primary filesystem and fileset hierarchy**. 25 | 26 | Going forward IBM Storage Scale CSI 3.0.0, the **primary filesystem/fileset has been removed**, these paths are invalid: 27 | 28 | - PVs definitions needs to be updated according to the **actual fileset mount path** in the IBM Storage Scale filesystem. 29 | - Without migration, existing workloads would **not be able to mount or access their data** after switching to IBM Storage Scale CSI 6.0.0 30 | 31 | This script updates PV definitions to use the new path structure while **preserving all other PV properties**. 32 | 33 | ## 3. Prerequisites 34 | 35 | - **Delete any existing workloads or application pods attached to the PVCs that will be migrated** from the existing IBM Storage Scale CSI cluster. 36 | 37 | Before running the migration script, ensure the following tools are installed and available in your `$PATH`: 38 | 39 | - The script should be run in the context of the cluster where IBM Storage Scale CSI is deployed 40 | - **kubectl** – to interact with the Kubernetes cluster and fetch/update PV/PVC objects 41 | - **jq** – for JSON parsing and manipulation of Kubernetes API responses 42 | 43 | You can verify installation with: 44 | 45 | ```bash 46 | kubectl version --client 47 | jq --version 48 | ``` 49 | 50 | 51 | ## 4. Example Transformation 52 | 53 | ### Before (PV created with **primary filesystem/fileset** enabled): 54 | ```text 55 | volumeHandle: 0;2;13009550825755318848;A3D56F10:9BC12E30;;pvc-3b1a-49d3-89e1-51f607b91234;/ibm/remotefs1/primary-remotefs1-123456789/.volumes/pvc-3b1a-49d3-89e1-51f607b91234 56 | ``` 57 | 58 | ### After (Migrated to **actual fileset mount path**): 59 | ```text 60 | volumeHandle: 0;2;13009550825755318848;A3D56F10:9BC12E30;;pvc-3b1a-49d3-89e1-51f607b91234;/var/mnt/remotefs1/fs1-pvc-3b1a-49d3-89e1-51f607b91234/pvc-3b1a-49d3-89e1-51f607b91234-data 61 | ``` 62 | 63 | ### Key Points 64 | 65 | **Unchanged:** 66 | - The identity portion of the handle (everything up to the last `;`). 67 | 68 | **Rewritten:** 69 | - Only the **path segment after the last `;`**, now pointing to the actual fileset mount path. 70 | 71 | ## 5. Volume Type Variations 72 | 73 | The exact path suffix (e.g., `pvc-uuid-data`) may vary based on how the volume was originally created. 74 | The script automatically detects and applies the correct mapping without user intervention. 75 | 76 | 77 | ## 6. Migration Script Usage 78 | 79 | A helper script is provided to automate PV migration: 80 | 81 | ```bash 82 | # Usage 83 | ./migration_csi_primary_removal.bash 84 | ``` 85 | 86 | - No arguments are required. 87 | - The script automatically detects the **actual fileset mount path** for each PV and updates accordingly. 88 | 89 | ## 7. Validate the migration. 90 | 91 | Validate the migration once the migration script finishes. The migration summary should have only successful or skipped PVs. There shouldn't be any Failed PVs in the summary. 92 | ```bash 93 | Migration Summary: 94 | ------------------------------------------------------------------------------------------------------------------------ 95 | Successful PVs: 1 96 | PVC Name | PV Name 97 | -------------------------------------------------------------------------------------------------------------------- 98 | ibm-spectrum-scale-pvc-clone-from-pvc-advanced | pvc-7981775f-08b1-4a53-ae4d-6740e2ec9a89 99 | 100 | Skipped PVs (already migrated): 2 101 | PVC Name | PV Name 102 | -------------------------------------------------------------------------------------------------------------------- 103 | scale-advance-pvc | pvc-dd8e015c-382c-4487-a215-91dd22a01d45 104 | ibm-spectrum-scale-pvc-advance-from-snapshot | pvc-e920921e-7a25-4017-9c8d-6469750a4772 105 | ``` 106 | 107 | ## 8. Features of the Migration Script 108 | 109 | - ✅ Filters only PVs created with the **spectrumscale.csi.ibm.com** driver. 110 | - ✅ Skips PVs already migrated (with an actual fileset mount path in `volumeHandle`). 111 | - ✅ **Backs up PVs and PVCs** before modification into a structured directory: 112 | 113 | ``` 114 | csi_migration_data/ 115 | └── migration-/ 116 | ├── migration.log 117 | ├── / 118 | │ └── / 119 | │ ├── pvc.yaml 120 | │ └── pv.yaml 121 | └── ... 122 | ``` 123 | 124 | - ✅ Logs all actions, successes, skips, and failures into: 125 | 126 | ``` 127 | csi_migration_data//migration.log 128 | ``` 129 | 130 | - ✅ Summarizes **success, skipped, and failed** migrations at the end. 131 | - ✅ Idempotent – safe to re-run if needed. 132 | 133 | ## 9. Preserved PV Properties 134 | 135 | The script ensures that **all original PV configurations** are retained after migration. 136 | The following fields are preserved: 137 | 138 | - **Capacity** (`spec.capacity.storage`) 139 | - **AccessModes** (`spec.accessModes`) 140 | - **PersistentVolumeReclaimPolicy** (`spec.persistentVolumeReclaimPolicy`) 141 | - **StorageClassName** 142 | - **CSI driver details** (fsType, volumeAttributes, nodeStageSecrets, etc.) 143 | - **PVC binding information** (safely re-created to preserve claim references) 144 | 145 | - Only the **`volumeHandle` path** is modified to reflect the actual fileset mount. 146 | 147 | ## 10. Notes and Limitations 148 | 149 | - The **filesystem names** (e.g., `remotefs1`) must remain identical between pre-primary-filesystem and post-primary-filesystem removal deployments. 150 | - The script does **not delete or recreate volumes** on IBM Storage Scale; it only updates Kubernetes PV metadata. 151 | - Existing workloads must be restarted to pick up new PV mount paths after migration. 152 | --------------------------------------------------------------------------------