├── .gitignore ├── README.md ├── build_by_oneself ├── CodeBuild.yaml ├── README.md └── source-auth.png ├── buildspec-daily.yml ├── buildspec.yml ├── docs ├── container-mirror-usage-guide.md ├── docker-docker-compose-usage-guide.md ├── ecs-fargate-useage-guide.md ├── helm-chart-useage-guide.md ├── how-to-request-new-container-image.md ├── media │ ├── image-request-pr.png │ ├── mirror-PR-Merged.png │ ├── mirror-inprogress.png │ └── mirror-passing.png └── scripts │ └── busybox-demo.yaml ├── kustomize ├── README.md ├── deployment.yaml └── kustomization.yaml ├── mirror ├── ECRImageList.sh ├── ECR_Auth.sh ├── README.md ├── blacklist-images.txt ├── ignore-images.txt ├── mirror-base.sh ├── mirror-images-daily.sh ├── mirror-images.sh ├── mirrored-images.txt ├── policy.text ├── required-images-daily.txt └── required-images.txt ├── resources └── deployment.png └── webhook ├── README.md ├── api-gateway.yaml └── mutating-webhook.yaml /.gitignore: -------------------------------------------------------------------------------- 1 | /.project 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![](https://codebuild.ap-northeast-1.amazonaws.com/badges?uuid=eyJlbmNyeXB0ZWREYXRhIjoicjlSNndlSGg4ZkJPQXF0Z1hIQnJIaFZES2VvN2tmUllKTjNEemJGeDVKZU5UUUt5eWdWT0Jrd0NZc2xweHROZFV1dEdXNmJLOVZmUGF1Tnl3ZmRSd1ZBPSIsIml2UGFyYW1ldGVyU3BlYyI6Ik5rNkxrdTZnR21GLzl4YzkiLCJtYXRlcmlhbFNldFNlcmlhbCI6MX0%3D&branch=master) 2 | 3 | ## 声明 4 | 您理解并同意：本镜像站收集并供您下载的镜像文件是按“原样”提供的，即我们无法控制或修改镜像文件，可能会出现由于开发者未及时更新或该镜像文件本身存在异常导致该镜像文件损坏或其他不可用状态，我们也不提供有关文件（内容）的任何保证，不会对镜像文件及其相关的信息或文档的可用性、可靠性、正确性或更新、升级等提供任何明示或默示的承诺或保证，镜像文件的下载和使用完全由您自主决定并自行承担风险，由此带来的任何损失，您同意在法律允许的范围内放弃追究我们的责任。 5 | 6 | 7 | ## 项目介绍 8 | 本项目用于将[Docker Hub](https://hub.docker.com/)，[Google Container Registry](https://console.cloud.google.com/gcr/images/google-containers/GLOBAL?pli=1)和[Quay](https://quay.io/search)中常用的公共container image自动同步至AWS中国区的ECR内，使AWS用户能更方便快捷的获取这些常见的容器镜像。 9 | 如果您是container image的权利人，不允许相关container image同步到AWS中国区的ECR内，请发送邮件到nwcd_labs@nwcdcloud.cn。 10 | ![](./resources/deployment.png) 11 | 12 | ## Amazon ECR镜像路径 13 | 所有同步至ECR的镜像都放在048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn这一container registry内，ECR镜像路径规则如下 14 | * **Docker Hub** (*目前只支持[docker official images](https://github.com/docker-library/official-images)*) 15 | * 原始镜像路径: [library/]repo:tag 16 | * ECR镜像路径: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/[library/]repo:tag 17 | * **GCR** 18 | * 原始镜像路径: gcr.io/namespace/repo:tag 19 | * ECR镜像路径: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/namespace/repo:tag 20 | * 原始镜像路径: k8s.gcr.io/repo:tag 21 | * ECR镜像路径: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/google_containers/repo:tag 22 | * **Quay** 23 | * 原始镜像路径: quay.io/namespace/repo:tag 24 | * ECR镜像路径: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/quay/namespace/repo:tag 25 | * **Global ECR** 26 | * 原始镜像路径: 602401143452.dkr.ecr.us-west-2.amazonaws.com/repo:tag 27 | * ECR镜像路径: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/amazonecr/repo:tag 28 | 29 | 海外镜像复制到ECR后的路径转换示例如下： 30 | 31 | | 海外镜像 | ECR镜像 | 32 | |------------ |---------| 33 | | nginx:1.17.9 | 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/nginx:1.17.9 | 34 | | gcr.io/heptio-images/velero:v1.1.0 | 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/heptio-images/velero:v1.1.0 | 35 | | k8s.gcr.io/cluster-autoscaler:v1.2.2 | 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/google_containers/cluster-autoscaler:v1.2.2 | 36 | | quay.io/calico/node:v3.7.4 | 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/quay/calico/node:v3.7.4 | 37 | | 602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon-k8s-cni:v1.5.5 | 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/amazonecr/amazon-k8s-cni:v1.5.5 | 38 | 39 | ## 使用方法 40 | [使用方法帮助文档](docs/container-mirror-usage-guide.md) 41 | 42 | 1. 直接修改 kubernetes deployment yaml 文件中的 image 指向本项目 ECR 中相应镜像的路径。 43 | 2. 不方便修改 image 路径，或者想自动替换所有 Kubernetes Pod 中 image 到相应 ECR 路径，可以使用Kubernetes的[Mutating admission webhook](webhook/README.md) 实现自动替换。[点击查看示例](webhook/README.md) 44 | 3. 利用 Helm Charts 部署应用，并且chart template支持自定义Pod image，可以设置 chart 参数，指向本项目 ECR 中相应镜像的路径。[点击查看示例](docs/helm-chart-useage-guide.md) 45 | 4. 如果您的 kubernetes 集群直接使用 kubectl 部署，且kubectl版本在v1.14或以上，可以使用[kustomize](kustomize/README.md) 将原始 image 路径指向指向本项目 ECR 中相应镜像的路径。[点击查看示例](kustomize/README.md) 46 | 5. 直接 修改 ECS/Fargate 的 task defition yaml 文件，用于部署 ECS/Fargate Service和Task。[点击查看示例](docs/ecs-fargate-useage-guide.md) 47 | 6. Docker 和 docker-compose, 直接修改文件中的 image 指向本项目 ECR 中相应镜像的路径。[点击查看示例](docs/docker-docker-compose-usage-guide.md) 48 | 49 | ## 增加新的容器镜像 50 | 已有镜像列表放在[mirrored-images.txt](./mirror/mirrored-images.txt)。 51 | 如果您需要其他镜像, 请您编辑 [required-images.txt](./mirror/required-images.txt) ，这将会在您的GitHub账户中 fork 一个新的分支，之后您可以提交PR（pull request）。 52 | 后台管理员 Merge 您的PR会触发`CodeBuild` 去拉取 `required-images.txt` 中定义的镜像回 ECR库，拉取过程中，图标会变成`in progress`。 拉取完后，您可以看到图标从`in progress`变为`passing` 53 | ![](https://codebuild.ap-northeast-1.amazonaws.com/badges?uuid=eyJlbmNyeXB0ZWREYXRhIjoicjlSNndlSGg4ZkJPQXF0Z1hIQnJIaFZES2VvN2tmUllKTjNEemJGeDVKZU5UUUt5eWdWT0Jrd0NZc2xweHROZFV1dEdXNmJLOVZmUGF1Tnl3ZmRSd1ZBPSIsIml2UGFyYW1ldGVyU3BlYyI6Ik5rNkxrdTZnR21GLzl4YzkiLCJtYXRlcmlhbFNldFNlcmlhbCI6MX0%3D&branch=master) 54 | 55 | 详细说明请参考[增加新的容器镜像帮助文档](docs/how-to-request-new-container-image.md) 56 | 57 | ## 自动同步新镜像 58 | 在[required-images-daily.txt](./mirror/required-images-daily.txt)中的镜像，会自动同步指定image的新镜像，tag中包含alpha、beta的不同步。目前仅支持Docker Hub。 59 | 60 | ## ECR登录/docker login 61 | EKS、Kops on EC2用户可直接使用，无需 ECR登录/docker login。 62 | 对于docker用户，需要 ECR 登录/docker login 后才能使用。 63 | 64 | 1. 确定你执行命令的 IAM user / IAM role 拥有下面权限： 65 | ```json 66 | [ 67 | "ecr:GetDownloadUrlForLayer", 68 | "ecr:BatchGetImage", 69 | "ecr:GetAuthorizationToken", 70 | "ecr:BatchCheckLayerAvailability" 71 | ] 72 | ``` 73 | 74 | 2. 对于docker用户，执行 ECR 登录/docker login： 75 | ```bash 76 | pip install awscli --upgrade --user 77 | aws ecr get-login-password --region cn-northwest-1 | docker login --username AWS --password-stdin 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn 78 | ``` 79 | 80 | 如果AWS CLI版本低于v1.17.10，需运行以下脚本： 81 | ```bash 82 | aws ecr get-login --region cn-northwest-1 --registry-ids 048912060910 --no-include-email | sh 83 | ``` 84 | 85 | 您也可以使用[ecr-credential-helper](https://github.com/awslabs/amazon-ecr-credential-helper) 完成登录。 86 | -------------------------------------------------------------------------------- /build_by_oneself/CodeBuild.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: "2010-09-09" 2 | Resources: 3 | MirrorServiceRole: 4 | Type: "AWS::IAM::Role" 5 | Properties: 6 | RoleName: "codebuild-container-mirror-service-role" 7 | AssumeRolePolicyDocument: 8 | Version: "2012-10-17" 9 | Statement: 10 | - Effect: "Allow" 11 | Principal: 12 | Service: 13 | - "codebuild.amazonaws.com" 14 | Action: 15 | - "sts:AssumeRole" 16 | Path: "/service-role/" 17 | ManagedPolicyArns: 18 | - "arn:aws:iam::aws:policy/AmazonSSMReadOnlyAccess" 19 | - "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly" 20 | Policies: 21 | - PolicyName: "CodeBuildContainerMirror" 22 | PolicyDocument: 23 | Version: "2012-10-17" 24 | Statement: 25 | - Effect: "Allow" 26 | Action: 27 | - "logs:CreateLogGroup" 28 | - "logs:CreateLogStream" 29 | - "logs:PutLogEvents" 30 | Resource: 31 | - !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/codebuild/container-mirror' 32 | - !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/codebuild/container-mirror:*' 33 | - Effect: "Allow" 34 | Action: 35 | - "s3:PutObject" 36 | - "s3:GetObject" 37 | - "s3:GetObjectVersion" 38 | - "s3:GetBucketAcl" 39 | - "s3:GetBucketLocation" 40 | Resource: 41 | - "arn:aws:s3:::codepipeline-*" 42 | - Effect: "Allow" 43 | Action: 44 | - "codebuild:CreateReportGroup" 45 | - "codebuild:CreateReport" 46 | - "codebuild:UpdateReport" 47 | - "codebuild:BatchPutTestCases" 48 | Resource: 49 | - !Sub 'arn:aws:codebuild::${AWS::AccountId}:report-group/container-mirror-*' 50 | RootInstanceProfile: 51 | Type: "AWS::IAM::InstanceProfile" 52 | Properties: 53 | Path: "/" 54 | Roles: 55 | - Ref: "MirrorServiceRole" 56 | Project: 57 | Type: AWS::CodeBuild::Project 58 | Properties: 59 | Name: container-mirror 60 | ServiceRole: !GetAtt MirrorServiceRole.Arn 61 | BadgeEnabled: true 62 | Artifacts: 63 | Type: no_artifacts 64 | Environment: 65 | Type: LINUX_CONTAINER 66 | ComputeType: BUILD_GENERAL1_SMALL 67 | Image: aws/codebuild/amazonlinux2-x86_64-standard:3.0 68 | PrivilegedMode: true 69 | Source: 70 | Auth: 71 | Type: OAUTH 72 | Location: !Ref RepositoryURL 73 | Type: GITHUB 74 | BuildSpec: buildspec.yml 75 | Triggers: 76 | Webhook: true 77 | FilterGroups: 78 | - - Type: EVENT 79 | Pattern: PUSH 80 | TimeoutInMinutes: 480 81 | LogsConfig: 82 | CloudWatchLogs: 83 | Status: ENABLED 84 | StartCodeBuildRole: 85 | Type: "AWS::IAM::Role" 86 | Properties: 87 | RoleName: "codebuild-container-mirror-trigger-service-role" 88 | AssumeRolePolicyDocument: 89 | Version: "2012-10-17" 90 | Statement: 91 | - Effect: "Allow" 92 | Principal: 93 | Service: 94 | - "events.amazonaws.com" 95 | Action: 96 | - "sts:AssumeRole" 97 | Path: "/service-role/" 98 | Policies: 99 | - PolicyName: "CodeBuildContainerMirrorTrigger" 100 | PolicyDocument: 101 | Version: "2012-10-17" 102 | Statement: 103 | - Effect: "Allow" 104 | Action: 105 | - "codebuild:StartBuild" 106 | Resource: 107 | - !GetAtt Project.Arn 108 | AKParameter: 109 | Type: "AWS::SSM::Parameter" 110 | Properties: 111 | Name: "China-ECR-AK" 112 | Type: "String" 113 | Value: !Ref AWSChinaAK 114 | SKParameter: 115 | Type: "AWS::SSM::Parameter" 116 | Properties: 117 | Name: "China-ECR-SK" 118 | Type: "String" 119 | Value: !Ref AWSChinaSK 120 | Parameters: 121 | RepositoryURL: 122 | Description: GitHub Repository URL,https://github.com// 123 | Type: String 124 | MinLength: 20 125 | AWSChinaAK: 126 | NoEcho: true 127 | Description: "AWS China Access Key ID" 128 | Type: String 129 | MinLength: 20 130 | MaxLength: 20 131 | AWSChinaSK: 132 | NoEcho: true 133 | Description: "AWS China Secret Access Key" 134 | Type: String 135 | MinLength: 40 136 | MaxLength: 40 -------------------------------------------------------------------------------- /build_by_oneself/README.md: -------------------------------------------------------------------------------- 1 | ## 免责说明 2 | 建议测试过程中使用此方案，生产环境使用请自行考虑评估。 3 | 欢迎联系参与方案共建和提交方案需求, 也欢迎在 github 项目issue中留言反馈bugs。 4 | 5 | ## 介绍 6 | 本文介绍参考本项目进行自建镜像库。 7 | 8 | ## 前提条件 9 | 需要AWS China区域和AWS global区域账号。 10 | AWS global环境用于拉取容器镜像，然后推送到国内ECR。 11 | 12 | # 自建镜像库 13 | ## 步骤1-Fork本项目 14 | 登录GitHub后，Fork本项目。 15 | ## 步骤2-修改ECR推送地址 16 | 在fork后的项目中，打开[mirror/mirror-base.sh](../mirror/mirror-base.sh)，修改第4、5行，修改ECR_REGION和ECR_DN为自己的地址。 17 | ## 步骤3-生成AWS China区域访问密钥 18 | 如果已有访问密钥 ID(Access key ID)和私有访问密钥(Secret access key)可跳过本步骤。 19 | 登录AWS **China**后，访问[https://console.amazonaws.cn/iam/home?region=cn-northwest-1#/users$new?step=details](https://console.amazonaws.cn/iam/home?region=cn-northwest-1#/users$new?step=details)，用户名输入**ECR**，访问类型选中**编程访问**。点击**下一步:权限**，设置权限选择**直接附加现有策略**，筛选策略输入**AmazonEC2ContainerRegistryFullAccess**，然后在筛选出来的结果中，选中该策略前的复选框，点击**下一步:标签**，点击**下一步:审核**，点击**创建用户**，记录下**访问密钥 ID**和**私有访问密钥**。 20 | ## 步骤4-GitHub授权 21 | 登录AWS **Global**后，新建CodeBuild项目，直接点击连接：[https://ap-northeast-1.console.aws.amazon.com/codesuite/codebuild/project/new?region=ap-northeast-1](https://ap-northeast-1.console.aws.amazon.com/codesuite/codebuild/project/new?region=ap-northeast-1)，根据自己情况，可切换region。向下滚动到**源**，**源提供程序**选择**GitHub**，然后点击**连接到GitHub**，在弹出的页面中点击**Authorize aws-codesuite**进行授权。授权完毕后，不用再继续操作，步骤4会自动构建CodeBuild项目。 22 | ![](./source-auth.png) 23 | ## 步骤5-部署CodeBuild 24 | 在AWS **Global**打开CloudFormation，**创建堆栈**，**上传模板文件**，文件为[CodeBuild.yaml](CodeBuild.yaml)。过程中需要输入3个参数，分别是步骤2生成的**访问密钥 ID**(AWSChinaAK)、**私有访问密钥**(AWSChinaSK)和您Fork的地址(RepositoryURL)。然后等待创建。 25 | ## 步骤6-验证 26 | 在[required-images.txt](../mirror/required-images.txt)添加一个需要镜像的地址，push到GitHub后，CodeBuild会自动启动构建，点击项目，查看**构建历史记录**，可查看具体构建版本的日志。 -------------------------------------------------------------------------------- /build_by_oneself/source-auth.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nwcdlabs/container-mirror/d97b51be0113ec1b8b79c1970aba603bf87e8beb/build_by_oneself/source-auth.png -------------------------------------------------------------------------------- /buildspec-daily.yml: -------------------------------------------------------------------------------- 1 | version: 0.2 2 | 3 | env: 4 | variables: 5 | FOO: "bar" 6 | parameter-store: 7 | ecr_ak: "China-ECR-AK" 8 | ecr_sk: "China-ECR-SK" 9 | 10 | phases: 11 | install: 12 | runtime-versions: 13 | docker: 18 14 | pre_build: 15 | commands: 16 | - echo "authenticate to Amazon ECR(cn-northwest-1)" 17 | - bash ./mirror/ECR_Auth.sh 18 | build: 19 | commands: 20 | - cd mirror; bash mirror-images-daily.sh 21 | post_build: 22 | commands: 23 | - echo "post_build phase" 24 | artifacts: 25 | # files: 26 | discard-paths: yes 27 | -------------------------------------------------------------------------------- /buildspec.yml: -------------------------------------------------------------------------------- 1 | version: 0.2 2 | 3 | env: 4 | variables: 5 | FOO: "bar" 6 | parameter-store: 7 | ecr_ak: "China-ECR-AK" 8 | ecr_sk: "China-ECR-SK" 9 | 10 | phases: 11 | install: 12 | runtime-versions: 13 | docker: 18 14 | pre_build: 15 | commands: 16 | - echo "authenticate to Amazon ECR(cn-northwest-1)" 17 | - bash ./mirror/ECR_Auth.sh 18 | build: 19 | commands: 20 | - cd mirror; bash mirror-images.sh 21 | post_build: 22 | commands: 23 | - echo "post_build phase" 24 | artifacts: 25 | # files: 26 | discard-paths: yes 27 | -------------------------------------------------------------------------------- /docs/container-mirror-usage-guide.md: -------------------------------------------------------------------------------- 1 | # 容器镜像库使用指南 2 | 3 | 本文档用于分场景指导AWS用户如何使用[container-mirror](https://github.com/nwcdlabs/container-mirror)方便快捷的部署常见的容器镜像。 4 | 5 | 场景分类： 6 | 1. Docker 和 docker-compose, 直接修改文件中的 image 指向本项目 ECR 中相应镜像的路径。[点击查看示例](docker-docker-compose-usage-guide.md) 7 | 8 | 2. ECS/Fargate 9 | 修改 ECS/Fargate 的 task defition yaml 文件 或者 docker-compose.yml 中 image 参数，指向 ECR 中相应 image 的路径 [点击查看如何使用在 ECS/Fargate 使用本项目的 ECR 镜像地址](ecs-fargate-useage-guide.md)。 10 | 11 | 3. Kubernetes on EC2 / EKS 12 | 1. 使用 Mutating webhook 自动替换所有 Kubernetes Pod 中 image 路径 13 | 14 | 如果您使用了自动部署工具且不方便修改 image 路径，或者想自动替换所有 Kubernetes Pod 中 image 到相应 ECR 路径，可以使用 Kubernetes 的[Mutating admission webhook](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook)，[点击查看如何使用 Mutating webhook](webhook/README.md)。 15 | 16 | 2. 使用Helm Charts 17 | 18 | 利用 Helm Charts 部署应用，并且chart template支持自定义Pod image，可以设置 chart 参数，指向本项目 ECR 中相应镜像的路径，[点击查看如何使用示例](helm-chart-useage-guide.md)。 19 | 20 | 3. 直接修改 kubernetes deployment yaml 文件 21 | 22 | 如果您的项目可以直接修改引用到原始容器镜像的地方，如修改 kubernetes deployment yaml 文件中的 image 指向本项目 ECR 中相应镜像的路径 23 | ```yaml 24 | apiVersion: v1 25 | kind: Pod 26 | metadata: 27 | name: bosybox-ecr-demo 28 | spec: 29 | containers: 30 | - name: bosybox 31 | image: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/busybox:1.31.1 32 | command: ["/bin/sh"] 33 | args: 34 | [ 35 | '-c', 36 | 'i=0; while true; do echo "$i: $(date -u)"; i=$((i+1)); sleep 1; done' 37 | ] 38 | ``` 39 | ```bash 40 | kubectl apply -f busybox-demo.yaml 41 | kubectl get pod bosybox-ecr-demo -o=jsonpath='{.spec.containers[0].image}' 42 | kubectl logs --tail=30 bosybox-ecr-demo 43 | ``` 44 | 45 | 4. [使用kustomize](../kustomize/README.md) 46 | 47 | 如果您的 kubernetes 集群直接使用 kubectl 部署，且 kubectl 版本在v1.14或以上，可以使用 [kustomize](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/) 将原始 image 路径指向 本项目 ECR 响应镜像的路径，[点击查看使用kustomize示例](../kustomize/README.md)。 48 | 49 | 4. [如何增加新的容器镜像](how-to-request-new-container-image.md) 50 | 51 | 52 | -------------------------------------------------------------------------------- /docs/docker-docker-compose-usage-guide.md: -------------------------------------------------------------------------------- 1 | # docker 和 docker-compose 2 | 3 | 直接修改文件中的 image 指向本项目 ECR 中相应镜像的路径，这里以 Docker 部署 xwiki 为例 4 | 5 | ## ECR登录/docker login 6 | EKS、Kops on EC2用户可直接使用，无需 ECR登录/docker login。 7 | 8 | 确定你执行命令的IAM user / IAM role拥有下面权限： 9 | ```json 10 | [ 11 | "ecr:GetDownloadUrlForLayer", 12 | "ecr:BatchGetImage", 13 | "ecr:GetAuthorizationToken", 14 | "ecr:BatchCheckLayerAvailability" 15 | ] 16 | ``` 17 | 18 | 对于docker用户，需要 ECR 登录/docker login 后才能使用： 19 | ```bash 20 | pip install awscli --upgrade --user 21 | aws ecr get-login-password --region cn-northwest-1 | docker login --username AWS --password-stdin 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn 22 | ``` 23 | 24 | 如果AWS CLI版本低于v1.17.10，需运行以下脚本： 25 | ```bash 26 | aws ecr get-login --region cn-northwest-1 --registry-ids 048912060910 --no-include-email | sh 27 | ``` 28 | 29 | 您也可以使用[ecr-credential-helper](https://github.com/awslabs/amazon-ecr-credential-helper) 完成登录。 30 | 31 | ## 使用 docker-compose 部署 xwiki 32 | - 下载配置 33 | ```bash 34 | mkdir -p ~/workspace/docker-xwiki && cd ~/workspace/docker-xwiki 35 | wget https://raw.githubusercontent.com/xwiki-contrib/docker-xwiki/master/11/mysql-tomcat/mysql/xwiki.cnf 36 | 37 | wget https://raw.githubusercontent.com/xwiki-contrib/docker-xwiki/master/11/mysql-tomcat/mysql/init.sql 38 | 39 | wget -O docker-compose.yml https://raw.githubusercontent.com/xwiki-contrib/docker-xwiki/master/docker-compose-mysql.yml 40 | ``` 41 | 42 | - 修改 image 路径 43 | ```yaml 44 | version: '2' 45 | networks: 46 | bridge: 47 | driver: bridge 48 | services: 49 | web: 50 | image: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/xwiki:lts-mysql-tomcat 51 | container_name: xwiki-mysql-tomcat-web 52 | depends_on: 53 | - db 54 | ports: 55 | - "8080:8080" 56 | environment: 57 | - DB_USER=xwiki 58 | - DB_PASSWORD=xwiki 59 | - DB_HOST=xwiki-mysql-db 60 | volumes: 61 | - xwiki-data:/usr/local/xwiki 62 | networks: 63 | - bridge 64 | db: 65 | image: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/mysql:5.7 66 | container_name: xwiki-mysql-db 67 | volumes: 68 | - ./xwiki.cnf:/etc/mysql/conf.d/xwiki.cnf 69 | - mysql-data:/var/lib/mysql 70 | - ./init.sql:/docker-entrypoint-initdb.d/init.sql 71 | environment: 72 | - MYSQL_ROOT_PASSWORD=xwiki 73 | - MYSQL_USER=xwiki 74 | - MYSQL_PASSWORD=xwiki 75 | - MYSQL_DATABASE=xwiki 76 | networks: 77 | - bridge 78 | volumes: 79 | mysql-data: {} 80 | xwiki-data: {} 81 | ``` 82 | 83 | - 部署 84 | ``` 85 | docker-compose up 86 | docker logs --follow 87 | http://{instance-ip}:8080/bin/view/Main/ 88 | ``` 89 | 90 | ## 附录：如何安装 docker-compose 91 | ```bash 92 | sudo curl -L "https://github.com/docker/compose/releases/download/1.25.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose 93 | sudo chmod +x /usr/local/bin/docker-compose 94 | sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose 95 | docker-compose --version 96 | ``` 97 | -------------------------------------------------------------------------------- /docs/ecs-fargate-useage-guide.md: -------------------------------------------------------------------------------- 1 | # ECS and ECS Fargate usage guide 2 | 3 | 这里将展示如何利用本项目的 ECR 镜像地址部署 [ecsworkshop](https://ecsworkshop.com/introduction/) 演示用例, 这里以宁夏区 cn-northwest-1 ECS Fargate 为例 4 | 5 | # 安装ecs-cli，jq 6 | 7 | 依照 [ecscli 安装指南](https://ecsworkshop.com/prerequisites/software/#install-software-1) 安装 ecscli 8 | 9 | ```bash 10 | sudo curl -so /usr/local/bin/ecs-cli https://s3.amazonaws.com/amazon-ecs-cli/ecs-cli-linux-amd64-latest 11 | sudo chmod +x /usr/local/bin/ecs-cli 12 | sudo yum -y install jq gettext 13 | 14 | # Setting environment variables required to communicate with AWS API's via the cli tools 15 | export AWS_REGION=cn-northwest-1 16 | aws configure set default.region ${AWS_REGION} 17 | aws configure get default.region 18 | ``` 19 | 20 | # 部署示例应用 21 | ## 部署 ECS fargate 集群 22 | ```bash 23 | cd ~/workspace/ 24 | git clone https://github.com/brentley/container-demo 25 | aws cloudformation deploy --stack-name container-demo --template-file cluster-fargate-private-vpc.yml --capabilities CAPABILITY_IAM --region ${AWS_REGION} 26 | aws cloudformation deploy --stack-name container-demo-alb --template-file script/alb-external.yml --region ${AWS_REGION} 27 | ``` 28 | 29 | ## 部署前端服务 ecsdemo-frontend 30 | 1. git clone 前端服务 31 | ```bash 32 | cd ~/workspace/ 33 | git clone https://github.com/brentley/ecsdemo-frontend 34 | ``` 35 | 2. **修改镜像地址 ecsdemo-frontend/docker-compose.yml** 36 | ```yaml 37 | version: '3' 38 | services: 39 | ecsdemo-frontend: 40 | environment: 41 | - CRYSTAL_URL=http://ecsdemo-crystal.service:3000/crystal 42 | - NODEJS_URL=http://ecsdemo-nodejs.service:3000 43 | image: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/brentley/ecsdemo-frontend 44 | ports: 45 | - "3000:3000" 46 | logging: 47 | driver: awslogs 48 | options: 49 | awslogs-group: ecsdemo-frontend 50 | awslogs-region: ${AWS_REGION} 51 | awslogs-stream-prefix: ecsdemo-frontend 52 | ``` 53 | 54 | 3. 部署 ecsdemo-frontend 前端服务 55 | ```bash 56 | # Deploy service and task 57 | ecs-cli compose --region $AWS_REGION --project-name ecsdemo-frontend service up \ 58 | --create-log-groups \ 59 | --target-group-arn $target_group_arn \ 60 | --private-dns-namespace service \ 61 | --container-name ecsdemo-frontend \ 62 | --container-port 3000 \ 63 | --cluster-config container-demo \ 64 | --vpc $vpc 65 | INFO[0000] Using ECS task definition TaskDefinition="ecsdemo-frontend:3" 66 | ... 67 | INFO[0045] ECS Service has reached a stable state desiredCount=1 runningCount=1 serviceName=ecsdemo-frontend 68 | 69 | # View running container 70 | ecs-cli compose --project-name ecsdemo-frontend service ps \ 71 | --cluster-config container-demo --region $AWS_REGION 72 | Name State Ports TaskDefinition Health 73 | 7cb08af6-05cc-4c49-a5d2-0e5500301090/ecsdemo-frontend RUNNING 10.0.102.93:3000->3000/tcp ecsdemo-frontend:3 UNKNOWN 74 | 75 | # View task image url 76 | aws ecs describe-task-definition --task-definition ecsdemo-frontend:3 --query "taskDefinition.containerDefinitions[*].image" --region $AWS_REGION 77 | [ 78 | "048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/brentley/ecsdemo-frontend" 79 | ] 80 | 81 | # Check reachability 82 | alb_url=$(aws cloudformation describe-stacks --stack-name container-demo-alb --query 'Stacks[0].Outputs[?OutputKey==`ExternalUrl`].OutputValue' --output text --region $AWS_REGION) 83 | echo "Open $alb_url in your browser" 84 | 85 | # Scale the tasks 86 | ecs-cli compose --project-name ecsdemo-frontend service scale 3 \ 87 | --cluster-config container-demo --region $AWS_REGION 88 | INFO[0000] Updated ECS service successfully desiredCount=3 force-deployment=false service=ecsdemo-frontend 89 | ... 90 | INFO[0075] ECS Service has reached a stable state desiredCount=3 runningCount=3 serviceName=ecsdemo-frontend 91 | 92 | ecs-cli compose --project-name ecsdemo-frontend service ps \ 93 | --cluster-config container-demo --region $AWS_REGION 94 | Name State Ports TaskDefinition Health 95 | 1be97885-d9fa-41f2-9a2c-1f7b7e170aa7/ecsdemo-frontend RUNNING 10.0.100.48:3000->3000/tcp ecsdemo-frontend:3 UNKNOWN 96 | 25dfe72d-77e9-4971-8051-63b64913c7f6/ecsdemo-frontend RUNNING 10.0.101.174:3000->3000/tcp ecsdemo-frontend:3 UNKNOWN 97 | 7cb08af6-05cc-4c49-a5d2-0e5500301090/ecsdemo-frontend RUNNING 10.0.102.93:3000->3000/tcp ecsdemo-frontend:3 UNKNOWN 98 | ``` 99 | 100 | ## 部署后端服务 ecsdemo-nodejs 101 | ```bash 102 | git clone https://github.com/brentley/ecsdemo-nodejs 103 | ``` 104 | ```yaml 105 | version: '3' 106 | services: 107 | ecsdemo-nodejs: 108 | image: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/brentley/ecsdemo-nodejs 109 | ports: 110 | - "3000:3000" 111 | logging: 112 | driver: awslogs 113 | options: 114 | awslogs-group: ecsdemo-nodejs 115 | awslogs-region: ${AWS_REGION} 116 | awslogs-stream-prefix: ecsdemo-nodejs 117 | ``` 118 | 119 | ## 部署后端服务 ecsdemo-crystal 120 | ``` 121 | git clone https://github.com/brentley/ecsdemo-crystal 122 | ``` 123 | 124 | ```yaml 125 | version: '3' 126 | services: 127 | ecsdemo-crystal: 128 | image: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/brentley/ecsdemo-crystal 129 | ports: 130 | - "3000:3000" 131 | logging: 132 | driver: awslogs 133 | options: 134 | awslogs-group: ecsdemo-crystal 135 | awslogs-region: ${AWS_REGION} 136 | awslogs-stream-prefix: ecsdemo-crystal 137 | ``` 138 | 139 | ## 最终前端效果 140 | ```bash 141 | alb_url=$(aws cloudformation describe-stacks --stack-name container-demo-alb --query 'Stacks[0].Outputs[?OutputKey==`ExternalUrl`].OutputValue' --output text --region $AWS_REGION) 142 | echo "Open $alb_url in your browser" 143 | ``` -------------------------------------------------------------------------------- /docs/helm-chart-useage-guide.md: -------------------------------------------------------------------------------- 1 | # Helm Chat usage guide 2 | 3 | 利用 Helm Charts 部署应用，并且chart template支持自定义Pod image，可以设置 chart 参数，指向本项目 ECR 中相应镜像的路径， 4 | 5 | ## Helm 部署 cluster-autoscaler 示例 6 | 7 | 通过 `--set image.repository` 和 `--set image.tag` 设置 chart 参数 8 | 9 | ```bash 10 | # 部署 11 | helm search repo cluster-autoscaler 12 | 13 | # Helm 3 example 14 | helm install my-ca-helm stable/cluster-autoscaler --namespace kube-system \ 15 | --set autoDiscovery.clusterName=${CLUSTER_NAME} --set cloudProvider=aws --set awsRegion=${AWS_REGION} \ 16 | --set image.repository=048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/google_containers/cluster-autoscaler --set image.tag=v1.14.7 17 | # Helm 2 --name my-ca-helm 18 | helm install --name my-ca-helm stable/cluster-autoscaler --namespace kube-system \ 19 | --set autoDiscovery.clusterName=${CLUSTER_NAME} --set cloudProvider=aws --set awsRegion=${AWS_REGION} \ 20 | --set image.repository=048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/google_containers/cluster-autoscaler --set image.tag=v1.14.7 21 | 22 | # 检查是否成功 23 | kubectl get pod $(kubectl get pods -n kube-system | egrep -o "my-ca-helm[a-zA-Z0-9-]+") -n kube-system 24 | 25 | # 获取image地址 26 | kubectl get pod $(kubectl get pods -n kube-system | egrep -o "my-ca-helm[a-zA-Z0-9-]+") -n kube-system -o=jsonpath='{.spec.containers[0].image}' 27 | 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/google_containers/cluster-autoscaler:v1.14.7 28 | 29 | # 清理 30 | helm delete my-ca-helm --namespace kube-system 31 | ``` 32 | 33 | ## Helm 部署 wordpress 示例 34 | 35 | ```bash 36 | # 部署 37 | helm search repo wordpress 38 | 39 | kubectl create namespace wordpress 40 | helm install wordpress stable/wordpress --namespace wordpress \ 41 | --set image.repository=wordpress --set image.tag=5.4 \ 42 | --set mariadb.enabled=false --set externalDatabase.host=myexternalhost --set externalDatabase.user=myuser \ 43 | --set externalDatabase.password=mypassword --set externalDatabase.database=mydatabase --set externalDatabase.port=3306 44 | 45 | # 检查是否成功和获取image地址 46 | kubectl get pods -n wordpress 47 | kubectl get pod $(kubectl get pods -n wordpress | egrep -o "wordpress[a-zA-Z0-9-]+" | head -1) -n wordpress -o=jsonpath='{.spec.containers[0].image}' 48 | 49 | # 访问 50 | export SERVICE_IP=$(kubectl get svc --namespace wordpress wordpress --template "{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}") 51 | echo "WordPress URL: http://$SERVICE_IP/" 52 | echo "WordPress Admin URL: http://$SERVICE_IP/admin" 53 | 54 | echo Username: user 55 | echo Password: $(kubectl get secret --namespace wordpress wordpress -o jsonpath="{.data.wordpress-password}" | base64 --decode) 56 | 57 | # 清理 58 | helm delete wordpress --namespace wordpress 59 | ``` 60 | 61 | -------------------------------------------------------------------------------- /docs/how-to-request-new-container-image.md: -------------------------------------------------------------------------------- 1 | # 如何增加新的容器镜像 2 | 3 | ## 首先请查看已有镜像列表[mirrored-images.txt](../mirror/mirrored-images.txt)。 4 | 5 | ## 如果您需要其他镜像, 请您编辑 [required-images.txt](../mirror/required-images.txt) ，这将会在您的GitHub账户中 fork 一个新的分支，之后您可以提交 PR（pull request）。 6 | 7 | ![image-request-pr](media/image-request-pr.png) 8 | 9 | ## 后台管理员 Merge 您的PR会触发 `CodeBuild` 去拉取 `required-images.txt` 中定义的镜像回ECR库。 10 | 11 | ![mirror-PR-Merged](media/mirror-PR-Merged.png) 12 | 13 | ## 拉取过程中，图标会变成`in progress` 14 | 15 | ![mirror-inprogress](media/mirror-inprogress.png) 16 | 17 | ## 拉取完后，您可以看到图标从`in progress`变为`passing` 18 | ![](https://codebuild.ap-northeast-1.amazonaws.com/badges?uuid=eyJlbmNyeXB0ZWREYXRhIjoicjlSNndlSGg4ZkJPQXF0Z1hIQnJIaFZES2VvN2tmUllKTjNEemJGeDVKZU5UUUt5eWdWT0Jrd0NZc2xweHROZFV1dEdXNmJLOVZmUGF1Tnl3ZmRSd1ZBPSIsIml2UGFyYW1ldGVyU3BlYyI6Ik5rNkxrdTZnR21GLzl4YzkiLCJtYXRlcmlhbFNldFNlcmlhbCI6MX0%3D&branch=master) 19 | 20 | ![mirror-passing](media/mirror-passing.png) -------------------------------------------------------------------------------- /docs/media/image-request-pr.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nwcdlabs/container-mirror/d97b51be0113ec1b8b79c1970aba603bf87e8beb/docs/media/image-request-pr.png -------------------------------------------------------------------------------- /docs/media/mirror-PR-Merged.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nwcdlabs/container-mirror/d97b51be0113ec1b8b79c1970aba603bf87e8beb/docs/media/mirror-PR-Merged.png -------------------------------------------------------------------------------- /docs/media/mirror-inprogress.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nwcdlabs/container-mirror/d97b51be0113ec1b8b79c1970aba603bf87e8beb/docs/media/mirror-inprogress.png -------------------------------------------------------------------------------- /docs/media/mirror-passing.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nwcdlabs/container-mirror/d97b51be0113ec1b8b79c1970aba603bf87e8beb/docs/media/mirror-passing.png -------------------------------------------------------------------------------- /docs/scripts/busybox-demo.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Pod 3 | metadata: 4 | name: bosybox-ecr-demo 5 | spec: 6 | containers: 7 | - name: bosybox 8 | image: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/busybox:1.31.1 9 | command: ["/bin/sh"] 10 | args: 11 | [ 12 | '-c', 13 | 'i=0; while true; do echo "$i: $(date -u)"; i=$((i+1)); sleep 1; done' 14 | ] 15 | -------------------------------------------------------------------------------- /kustomize/README.md: -------------------------------------------------------------------------------- 1 | # 使用kubectl kustomize更换image路径 2 | 3 | 首先确认kubectl版本在v1.14或以上。本文以替换nginx image路径为例来介绍。 4 | 5 | ## 步骤1.正常编写deployment.yaml文件 6 | 首先正常编写deployment.yaml文件。 7 | ``` 8 | apiVersion: apps/v1 9 | kind: Deployment 10 | metadata: 11 | name: my-nginx 12 | spec: 13 | selector: 14 | matchLabels: 15 | run: my-nginx 16 | replicas: 2 17 | template: 18 | metadata: 19 | labels: 20 | run: my-nginx 21 | spec: 22 | containers: 23 | - name: my-nginx 24 | image: nginx 25 | ports: 26 | - containerPort: 80 27 | ``` 28 | 29 | ## 步骤2.编写kustomization.yaml文件 30 | 在resources里指定要处理的yaml文件；对要替换的image，需要逐个编写，指定实际需要指向的路径。 31 | ``` 32 | apiVersion: kustomize.config.k8s.io/v1beta1 33 | kind: Kustomization 34 | resources: 35 | - deployment.yaml 36 | images: 37 | - name: nginx 38 | newName: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/nginx 39 | newTag: "1.17" 40 | ``` 41 | 42 | ## 步骤3.查看image已更新(可选) 43 | 运行kubectl kustomize ./ 以确认image已更新，正常情况下，可看到**image**已替换为新的路径 44 | ``` 45 | apiVersion: apps/v1 46 | kind: Deployment 47 | metadata: 48 | name: my-nginx 49 | spec: 50 | replicas: 2 51 | selector: 52 | matchLabels: 53 | run: my-nginx 54 | template: 55 | metadata: 56 | labels: 57 | run: my-nginx 58 | spec: 59 | containers: 60 | - image: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/nginx:1.17 61 | name: my-nginx 62 | ports: 63 | - containerPort: 80 64 | ``` 65 | 66 | ## 步骤4.实际部署 67 | 运行kubectl apply -k ./ 部署。 68 | 69 | ## 步骤5.验证 70 | 运行kubectl get pod查看pod运行情况。 71 | 运行kubectl get pod my-nginx-xxxxxx-xxxx -o=jsonpath='{.spec.containers[0].image}' 72 | 结果应该为048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/nginx:1.17 -------------------------------------------------------------------------------- /kustomize/deployment.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: my-nginx 5 | spec: 6 | selector: 7 | matchLabels: 8 | run: my-nginx 9 | replicas: 2 10 | template: 11 | metadata: 12 | labels: 13 | run: my-nginx 14 | spec: 15 | containers: 16 | - name: my-nginx 17 | image: nginx 18 | ports: 19 | - containerPort: 80 20 | -------------------------------------------------------------------------------- /kustomize/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | resources: 4 | - deployment.yaml 5 | images: 6 | - name: nginx 7 | newName: 048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/nginx 8 | newTag: "1.17" 9 | -------------------------------------------------------------------------------- /mirror/ECRImageList.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ECR_REGION='cn-northwest-1' 4 | ECR_DN="048912060910.dkr.ecr.${ECR_REGION_FROM}.amazonaws.com.cn" 5 | 6 | # list all existing repos 7 | allEcrRepos=$(aws --profile=ChinaECR --region $ECR_REGION ecr describe-repositories --query 'repositories[*].repositoryName' --output text) 8 | #allEcrRepos="dockerhub/redis dockerhub/kope/dns-controller" 9 | #echo "allEcrRepos:$allEcrRepos" 10 | 11 | function replaceDomainName(){ 12 | URI="$1" 13 | URI=${URI/#quay/quay.io} 14 | URI=${URI/#gcr\/google_containers/k8s.gcr.io} 15 | URI=${URI/#gcr/gcr.io} 16 | URI=${URI/#amazonecr/602401143452.dkr.ecr.us-west-2.amazonaws.com} 17 | URI=${URI/#dockerhub\//} 18 | } 19 | 20 | allEcrRepos=$(echo $allEcrRepos | tr " " "\n" | sort) 21 | for repo in $allEcrRepos 22 | do 23 | tags=$(aws --profile ChinaECR --region $ECR_REGION ecr list-images --repository-name $repo |jq -r ".imageIds[]|.imageTag") 24 | tags=$(echo $tags | tr " " "\n" | sort) 25 | for tag in $tags 26 | do 27 | if [ "$tag" != "null" ]; then 28 | replaceDomainName "${repo}:${tag}" 29 | echo $URI >> mirrored-images.txt 30 | fi 31 | done 32 | done 33 | -------------------------------------------------------------------------------- /mirror/ECR_Auth.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | env 3 | 4 | aws configure --profile=ChinaECR set aws_access_key_id $ecr_ak 5 | aws configure --profile=ChinaECR set aws_secret_access_key $ecr_sk 6 | aws configure --profile=ChinaECR set default.region cn-northwest-1 -------------------------------------------------------------------------------- /mirror/README.md: -------------------------------------------------------------------------------- 1 | # 脚本说明 2 | ## mirror-base.sh 3 | 主要脚本，从Global同步到国内 4 | 5 | ## mirror-images.sh 6 | 需要同步的数据从required-images.txt获取，调用mirror-base.sh。 7 | 8 | ## mirror-images-daily.sh 9 | 需要配置trigger触发，从Global同步到国内，需要同步的数据从required-images-daily.txt获取。 10 | 11 | ## ECRImageList.sh 12 | 获取国内ECR现有image清单，需要修改第3、4行为自己的ECR地址。 13 | 14 | ## ECR_Auth.sh 15 | mirror-images.sh运行前，ECR授权 -------------------------------------------------------------------------------- /mirror/blacklist-images.txt: -------------------------------------------------------------------------------- 1 | #只填repo名称，不支持tag 2 | -------------------------------------------------------------------------------- /mirror/ignore-images.txt: -------------------------------------------------------------------------------- 1 | #需要指定具体tag 2 | -------------------------------------------------------------------------------- /mirror/mirror-base.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # set -x 3 | 4 | ECR_REGION='cn-northwest-1' 5 | ECR_DN="048912060910.dkr.ecr.${ECR_REGION}.amazonaws.com.cn" 6 | IMAGES_BLACKLIST='blacklist-images.txt' 7 | IMAGES_IGNORE_LIST='ignore-images.txt' 8 | 9 | declare -A DOMAIN_MAP 10 | DOMAIN_MAP["quayio"]="quay" 11 | DOMAIN_MAP["quay.io"]="quay" 12 | DOMAIN_MAP["gcr.io"]="gcr" 13 | DOMAIN_MAP["asia.gcr.io"]="gcr" 14 | DOMAIN_MAP["us.gcr.io"]="gcr" 15 | DOMAIN_MAP["k8s.gcr.io"]="gcr/google_containers" 16 | DOMAIN_MAP["602401143452.dkr.ecr.us-west-2.amazonaws.com"]="amazonecr" 17 | DOMAIN_MAP["docker.io"]="dockerhub" 18 | 19 | function replaceDomainName(){ 20 | math_mirror=False 21 | URI="$1" 22 | for key in ${!DOMAIN_MAP[*]};do 23 | if [[ $URI == ${key}* ]]; then 24 | math_mirror=True 25 | URI=${URI/#${key}/${DOMAIN_MAP[$key]}} 26 | break 27 | fi 28 | done 29 | if [[ $math_mirror == False ]] ; then 30 | URI="dockerhub/${URI}" 31 | fi 32 | } 33 | 34 | function createEcrRepo() { 35 | if inArray "$2" "$blacklist" 36 | then 37 | echo "repo: $2 on the blacklist" 38 | else 39 | if inArray "$1" "$allEcrRepos" 40 | then 41 | echo "repo: $1 already exists" 42 | else 43 | echo "creating repo: $1" 44 | aws --profile=ChinaECR --region ${ECR_REGION} ecr create-repository --repository-name "$1" 45 | attachPolicy "$1" 46 | fi 47 | fi 48 | } 49 | 50 | function attachPolicy() { 51 | echo "attaching public-read policy on ECR repo: $1" 52 | aws --profile ChinaECR --region $ECR_REGION ecr set-repository-policy --policy-text file://policy.text --repository-name "$1" 53 | } 54 | 55 | function deleteEcrRepo() { 56 | if inArray "$1" "$allEcrRepos" 57 | then 58 | echo "deleting repo: $1" 59 | aws --profile=ChinaECR --region ${ECR_REGION} ecr delete-repository --repository-name "$1" --force 60 | fi 61 | } 62 | 63 | function isRemoteImageExists(){ 64 | # is_remote_image_exists repositoryName:Tag Digests 65 | fullrepo=${1#*/} 66 | repoName=${fullrepo%%:*} 67 | tag=${fullrepo##*:} 68 | res=$(aws --profile ChinaECR --region $ECR_REGION ecr describe-images --repository-name "$repoName" --query "imageDetails[?(@.imageDigest=='$2')].contains(@.imageTags, '$tag') | [0]") 69 | 70 | if [ "$res" == "true" ]; then 71 | return 0 72 | else 73 | return 1 74 | fi 75 | } 76 | 77 | function getLocalImageDigests(){ 78 | x=$(docker image inspect --format='{{index .RepoDigests 0}}' "$1") 79 | echo ${x##*@} 80 | # docker images --digests --no-trunc -q "$1" 81 | } 82 | 83 | function inArray() { 84 | local list=$2 85 | local elem=$1 86 | for i in ${list[@]} 87 | do 88 | if [ "$i" == "${elem}" ] ; then 89 | return 0 90 | fi 91 | done 92 | return 1 93 | } 94 | 95 | function loginEcr() { 96 | aws --profile=ChinaECR ecr --region cn-northwest-1 get-login --no-include-email | sh 97 | #aws --profile=ChinaECR ecr --region cn-north-1 get-login --no-include-email | sh 98 | aws ecr get-login --region us-west-2 --registry-ids 602401143452 894847497797 --no-include-email | sh 99 | } 100 | 101 | function pullAndPush(){ 102 | origimg="$1" 103 | echo "------origimg:${origimg}------" 104 | repo=`echo ${origimg}|cut -d: -f1` 105 | if inArray "${repo}" "$blacklist" 106 | then 107 | echo "repo: $repo on the blacklist" 108 | else 109 | if inArray "${origimg}" "$ignoreImages" 110 | then 111 | echo "ignore images:${origimg}" 112 | else 113 | docker pull $origimg 114 | 115 | replaceDomainName $origimg 116 | targetImg="$ECR_DN/${URI}" 117 | 118 | echo "tagging $origimg to $targetImg" 119 | docker tag $origimg $targetImg 120 | 121 | #echo "getting the digests on $targetImg..." 122 | #digests=$(getLocalImageDigests $targetImg) 123 | #echo "digests:$digests" 124 | #echo "checking if remote image exists" 125 | 126 | #去掉检查 127 | #if isRemoteImageExists $targetImg $digests;then 128 | # echo "[SKIP] image already exists, skip" 129 | #else 130 | #echo "[PUSH] remote image not exists or digests not match, pushing $targetImg" 131 | docker push $targetImg 132 | #fi 133 | fi 134 | fi 135 | } 136 | 137 | # list all existing repos 138 | allEcrRepos=$(aws --profile=ChinaECR --region $ECR_REGION ecr describe-repositories --query 'repositories[*].repositoryName' --output text) 139 | echo "allEcrRepos:$allEcrRepos" 140 | 141 | blacklist=$(grep -v ^# $IMAGES_BLACKLIST | cut -d: -f1 | sort -u) 142 | for blackrepo in ${blacklist[@]} 143 | do 144 | replaceDomainName $blackrepo 145 | deleteEcrRepo $URI 146 | done 147 | 148 | ignoreImages=$(grep -v ^# $IMAGES_IGNORE_LIST | sort -u) 149 | 150 | 151 | # ecr login for the once 152 | loginEcr 153 | 154 | -------------------------------------------------------------------------------- /mirror/mirror-images-daily.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | source mirror-base.sh 3 | 4 | IMAGES_DAILY_FILE_LIST="required-images-daily.txt" 5 | images=$(grep -v ^# $IMAGES_DAILY_FILE_LIST) 6 | #images="golang" 7 | 8 | count=0 9 | for image in ${images[@]} 10 | do 11 | count=$[$count + 1] 12 | done 13 | echo "count:${count}" 14 | mytime=$(date "+%H") 15 | mytime="$((10#$mytime))" 16 | split_index=$[$mytime / 3] 17 | echo "split_index:${split_index}" 18 | split_size=$[$[$count / 8] + 1] 19 | echo "split_size:${split_size}" 20 | begin=$[split_index * split_size] 21 | end=$[$[split_index + 1] * split_size ] 22 | echo "begin:${begin}" 23 | echo "end:${end}" 24 | current_index=0 25 | for image in ${images[@]} 26 | do 27 | if [ $current_index -lt $begin ] 28 | then 29 | current_index=$[$current_index + 1] 30 | continue 31 | fi 32 | 33 | if [ $current_index == $end ] 34 | then 35 | break 36 | fi 37 | 38 | current_index=$[$current_index + 1] 39 | 40 | repo=`echo ${image}|cut -d: -f1` 41 | echo "************begin pull ${repo} all tag************" 42 | date 43 | if inArray "${repo}" "$blacklist" 44 | then 45 | echo "repo: $repo on the blacklist" 46 | else 47 | replaceDomainName $repo 48 | createEcrRepo $URI $repo 49 | 50 | #把现有tag和digest放入到map 51 | declare -A ECR_MAP=() 52 | ecrTagsData=$(aws --profile ChinaECR --region $ECR_REGION ecr list-images --repository-name $URI) 53 | ecrTags=$(echo $ecrTagsData|jq ".imageIds[].imageTag" -r) 54 | ecrDigest=$(echo $ecrTagsData|jq ".imageIds[].imageDigest" -r) 55 | arrDigest=($ecrDigest) 56 | index=0 57 | for existTag in $ecrTags ;do 58 | ECR_MAP_Key=$existTag 59 | ECR_MAP_Value=${arrDigest[$index]} 60 | #echo "${ECR_MAP_Key}_${ECR_MAP_Value}" 61 | ECR_MAP[${ECR_MAP_Key}]=${ECR_MAP_Value} 62 | index=`expr $index + 1` 63 | done 64 | 65 | if [[ ${repo} =~ / ]];then 66 | request_repo=$repo 67 | else 68 | request_repo="library/${repo}" 69 | fi 70 | url="https://hub.docker.com/v2/repositories/${request_repo}/tags" 71 | while true 72 | do 73 | echo "请求url:${url}" 74 | allTagsData=`wget -q $url -O -` 75 | allTags=$(echo $allTagsData | jq ".results[]" -c) 76 | #由于某些tag没有linux amd64版本，可能存在过滤后下标不对应问题，因此这里不能使用下标定位 77 | #schema1 manifest format的没有digest值 78 | for allTag in $allTags ;do 79 | tag=$(echo $allTag | jq -r ".name") 80 | if [[ ${tag} =~ beta || ${tag} =~ alpha || ${tag} =~ windowsservercore || ${tag} =~ nanoserver ]];then 81 | #echo "skip ${tag}===============" 82 | continue 83 | fi 84 | digest=$(echo $allTag | jq -r '.images[]|select(.architecture=="amd64")|select(.os=="linux")|.digest') 85 | #schema1 manifest format的没有digest值，如果ECR有值就不再处理 86 | if [[ "$digest" == "" && "${ECR_MAP[$tag]}" != "" ]] ; then 87 | #echo "skip ${tag}===============" 88 | continue 89 | elif [[ "$digest" != "" && "$digest" == "${ECR_MAP[$tag]}" ]] ; then 90 | #echo "skip ${tag}===============" 91 | continue 92 | elif [[ "$digest" != "" && "${ECR_MAP[$tag]}" != "" && "$digest" != "${ECR_MAP[$tag]}" ]] ; then 93 | #这种情况可能是tag内容变化 94 | #也有可能是有些tag的digest在docker hub和ECR上不一致 95 | #比如golang:1.10.6-alpine3.7 96 | #更新时间超过10天，且digest不一致的，不更新 97 | last_updated=$(echo $allTag | jq -r ".last_updated") 98 | last_date=$(date -d "$last_updated" +%s) 99 | current_date=$(date +%s) 100 | interval=$(expr $current_date - $last_date) 101 | if [ ${interval} -gt 864000 ] ; then 102 | #echo "skip ${tag}===============${last_updated}" 103 | continue 104 | else 105 | echo "${repo}:${tag}存在，需更新,last_updated:${last_updated},interval:${interval}" 106 | fi 107 | fi 108 | pullAndPush ${repo}:${tag} 109 | done 110 | url=$(echo $allTagsData | jq -r ".next") 111 | if [ "${url}" == "null" ]; then 112 | break 113 | fi 114 | done 115 | fi 116 | done 117 | date 118 | -------------------------------------------------------------------------------- /mirror/mirror-images.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | source mirror-base.sh 3 | 4 | IMAGES_FILE_LIST='required-images.txt' 5 | 6 | repos=$(grep -v ^# $IMAGES_FILE_LIST | cut -d: -f1 | sort -u) 7 | for repo in ${repos[@]} 8 | do 9 | replaceDomainName $repo 10 | createEcrRepo $URI $repo 11 | done 12 | 13 | images=$(grep -v ^# $IMAGES_FILE_LIST) 14 | for image in ${images[@]} 15 | do 16 | pullAndPush $image 17 | done 18 | -------------------------------------------------------------------------------- /mirror/policy.text: -------------------------------------------------------------------------------- 1 | { 2 | "Version": "2008-10-17", 3 | "Statement": [ 4 | { 5 | "Sid": "allow-all-pull", 6 | "Effect": "Allow", 7 | "Principal": "*", 8 | "Action": [ 9 | "ecr:GetDownloadUrlForLayer", 10 | "ecr:BatchGetImage", 11 | "ecr:BatchCheckLayerAvailability" 12 | ] 13 | } 14 | ] 15 | } 16 | -------------------------------------------------------------------------------- /mirror/required-images-daily.txt: -------------------------------------------------------------------------------- 1 | # So far, only Docker Hub is supported. 2 | amazon/aws-alb-ingress-controller 3 | amazon/aws-ebs-csi-driver 4 | amazon/aws-efs-csi-driver 5 | amazon/aws-for-fluent-bit 6 | amazon/aws-node-termination-handler 7 | amazon/cloudwatch-agent 8 | adminer 9 | couchbase 10 | redis 11 | ubuntu 12 | node 13 | mysql 14 | postgres 15 | registry 16 | hello-world 17 | traefik 18 | mongo 19 | nginx 20 | httpd 21 | busybox 22 | alpine 23 | mariadb 24 | memcached 25 | consul 26 | docker 27 | golang 28 | influxdb 29 | aerospike 30 | openjdk 31 | centos 32 | rabbitmq 33 | ruby 34 | debian 35 | wordpress 36 | haproxy 37 | elasticsearch 38 | nextcloud 39 | php 40 | maven 41 | tomcat 42 | crate 43 | sonarqube 44 | amazonlinux 45 | telegraf 46 | eclipse-mosquitto 47 | java 48 | logstash 49 | cassandra 50 | ghost 51 | kong 52 | vault 53 | gradle 54 | couchdb 55 | kibana 56 | jenkins 57 | bash 58 | joomla 59 | buildpack-deps 60 | drupal 61 | nats 62 | solr 63 | percona 64 | zookeeper 65 | neo4j 66 | matomo 67 | teamspeak 68 | flink 69 | rocket.chat 70 | swarm 71 | groovy 72 | fedora 73 | nats-streaming 74 | mongo-express 75 | owncloud 76 | rethinkdb 77 | jruby 78 | pypy 79 | kapacitor 80 | redmine 81 | websphere-liberty 82 | jetty 83 | composer 84 | erlang 85 | mono 86 | elixir 87 | odoo 88 | mediawiki 89 | swift 90 | znc 91 | rust 92 | iojs 93 | oraclelinux 94 | haxe 95 | arangodb 96 | xwiki 97 | piwik 98 | django 99 | clojure 100 | tomee 101 | gcc 102 | ros 103 | ibmjava 104 | ubuntu-debootstrap 105 | hylang 106 | rails 107 | neurodebian 108 | orientdb 109 | adoptopenjdk 110 | opensuse 111 | julia 112 | yourls 113 | irssi 114 | open-liberty 115 | kaazing-gateway 116 | notary 117 | photon 118 | convertigo 119 | backdrop 120 | cirros 121 | plone 122 | fsharp 123 | eggdrop 124 | r-base 125 | bonita 126 | lightstreamer 127 | php-zendserver 128 | geonetwork 129 | haskell 130 | clearlinux 131 | gazebo 132 | celery 133 | nuxeo 134 | storm 135 | swipl 136 | fluentd 137 | express-gateway 138 | docker-dev 139 | rakudo-star 140 | spiped 141 | rapidoid 142 | thrift 143 | ubuntu-upstart 144 | postfixadmin 145 | varnish 146 | hello-seattle 147 | mageia 148 | hola-mundo 149 | glassfish 150 | known 151 | silverpeas 152 | crux 153 | archlinux 154 | sourcemage 155 | sl 156 | euleros 157 | clefos 158 | alt 159 | hipache 160 | sapmachine 161 | friendica 162 | jobber 163 | scratch 164 | caddy 165 | cheers 166 | fluxcd/helm-controller 167 | fluxcd/image-automation-controller 168 | fluxcd/image-reflector-controller 169 | fluxcd/kustomize-controller 170 | fluxcd/notification-controller 171 | fluxcd/source-controller 172 | langgenius/dify-api 173 | langgenius/dify-web 174 | langgenius/dify-sandbox 175 | vllm/vllm-openai 176 | lmsysorg/sglang 177 | -------------------------------------------------------------------------------- /mirror/required-images.txt: -------------------------------------------------------------------------------- 1 | # 2 | # 1. check required-images-mirrored.txt for mirrored image list 3 | # 4 | # 2. please add additional required images below 5 | # 6 | # 3. delete old images in this txt 7 | 8 | # Example 9 | ## name - aws/node-termination-handler for Amazon EKS 10 | ## see - https://github.com/aws/aws-node-termination-handler 11 | ## images defined - https://github.com/aws/aws-node-termination-handler/blob/79bf81f618de474f86636120a40438a47a4e2869/config/helm/aws-node-termination-handler/values.yaml#L5-L7 12 | #amazon/aws-node-termination-handler:v1.3.1 13 | # 此行以及以上内容请勿删除。新images在此行下添加。提交新的images时，请删除旧的images 14 | 15 | langgenius/dify-api:1.3.1 16 | langgenius/dify-api:1.4.0 17 | langgenius/dify-web:1.3.1 18 | langgenius/dify-web:1.4.0 19 | langgenius/dify-sandbox:0.2.11 20 | langgenius/dify-sandbox:0.2.12 21 | langgenius/dify-plugin-daemon:0.0.6-local 22 | langgenius/dify-plugin-daemon:0.0.7-local 23 | langgenius/dify-plugin-daemon:0.0.8-local 24 | langgenius/dify-plugin-daemon:0.0.9-local 25 | langgenius/dify-plugin-daemon:0.1.0-local 26 | -------------------------------------------------------------------------------- /resources/deployment.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nwcdlabs/container-mirror/d97b51be0113ec1b8b79c1970aba603bf87e8beb/resources/deployment.png -------------------------------------------------------------------------------- /webhook/README.md: -------------------------------------------------------------------------------- 1 | # 如何使用Kubernetes mutating admission webhook自动更换Kubernetes Pod的容器镜像 2 | 3 | 本项目基于[通过 Amazon API Gateway 实现 Kubernetes mutation admission webhook](https://github.com/aws-samples/amazon-api-gateway-mutating-webhook-for-k8) 的原理，根据预先定义的映射规则自动修改 Kubernetes Pod 内的镜像路径到本项目对应的 ECR 镜像仓库。 4 | 5 | ## 如何部署使用 webhook 6 | ### 前提条件 7 | - 如果您在使用Amazon EKS，可跳过检查步骤直接到部署环节。 8 | - 如果您是自己搭建的Kubernetes，请确保 9 | - 您的Kubernetes集群版本为1.9或以上. 10 | - Kubernetes的MutatingAdmissionWebhook admission controllers 功能已打开 11 | - admissionregistration.k8s.io/v1 API 启用. 12 | 13 | ### 部署 webhook 14 | ### 方法1：Kubernetes mutating admission webhook 直接使用本项目托管 Amazon API Gateway 15 | 1. 本项目已经部署了一个托管 Amazon API Gateway，使用以下命令即可直接部署 WebHook，并指向托管 Amazon API Gateway。 16 | ```bash 17 | kubectl apply -f mutating-webhook.yaml 18 | #kubectl apply -f https://raw.githubusercontent.com/nwcdlabs/container-mirror/master/webhook/mutating-webhook.yaml 19 | ``` 20 | 21 | 2. 验证 pod 详细信息中的image 已经替换为本项目对应的 ECR 镜像仓库。 22 | ```bash 23 | kubectl run test --image=k8s.gcr.io/coredns:1.3.1 24 | kubectl get pod test -o=jsonpath='{.spec.containers[0].image}' 25 | # 结果应显示为048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/google_containers/coredns:1.3.1 26 | 27 | # 清理 28 | kubectl delete pod test 29 | ``` 30 | 31 | ### 方法2：自己部署Amazon API Gateway 32 | 如果您期望部署自己的 Amazon API Gateway 用于 webhook, 按照下列步骤： 33 | 1. 使用CloudFormation模板文件 api-gateway.yaml 在 AWS CloudFormation Console 上部署 Amazon API Gateway 以及相关资源，CloudFormation Stack 使用默认参数即可。 34 | 2. 创建 Kubernetes Mutating Webhook Configuration资源 35 | - 在第一步创建的CloudFormation stack完成后，在输出结果中找到 APIGateWayURL。 36 | - 修改 mutating-webhook.yaml，将 webhooks.clientConfig.url 的值替换为上面找到的APIGateWayURL值。 37 | - 创建 Kubernetes resource: 38 | ```bash 39 | kubectl apply -f mutating-webhook.yaml 40 | ``` 41 | 42 | ### 其他方法，例如 sam 部署方式 43 | 详情参考 [amazon-api-gateway-mutating-webhook-for-k8](https://github.com/aws-samples/amazon-api-gateway-mutating-webhook-for-k8) 44 | 45 | ## 使用 WebHook之后，期望部分 Image 依旧强制回源，引用绝对地址 46 | 使用WebHook后，会把相关地址都转化为ECR仓库地址，如果期望部分 Image 依旧强制回源，或者由于 ECR 仓库没有同步所有的image，期望回源地址下载。可以在 image 路径使用特殊标识(**direct.to/**)。 47 | 比如 48 | ```yaml 49 | image: direct.to/busybox:latest 50 | ``` 51 | 返回 busybox:latest 52 | 53 | ```yaml 54 | image: direct.to/gcr.io/google_containers/pause-amd64:3.0 55 | ``` 56 | 返回 gcr.io/google_containers/pause-amd64:3.0 57 | -------------------------------------------------------------------------------- /webhook/api-gateway.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: "2010-09-09" 2 | Description: "K8S image path mutation webhook" 3 | 4 | Resources: 5 | apiGateway: 6 | Type: AWS::ApiGateway::RestApi 7 | Properties: 8 | Name: k8s-image-path-webhook 9 | Description: K8S image path mutation webhook 10 | EndpointConfiguration: 11 | Types: [ REGIONAL ] 12 | 13 | apiGatewayRootMethod: 14 | Type: AWS::ApiGateway::Method 15 | Properties: 16 | AuthorizationType: NONE 17 | HttpMethod: POST 18 | Integration: 19 | IntegrationHttpMethod: POST 20 | Type: AWS_PROXY 21 | Uri: !Sub 22 | - "arn:${AWS::Partition}:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${lambdaArn}/invocations" 23 | - lambdaArn: !GetAtt lambdaFunction.Arn 24 | ResourceId: !GetAtt apiGateway.RootResourceId 25 | RestApiId: !Ref apiGateway 26 | 27 | apiGatewayDeployment: 28 | Type: AWS::ApiGateway::Deployment 29 | DependsOn: 30 | - apiGatewayRootMethod 31 | Properties: 32 | RestApiId: !Ref apiGateway 33 | StageName: call 34 | 35 | lambdaFunction: 36 | Type: AWS::Lambda::Function 37 | Properties: 38 | Code: 39 | ZipFile: | 40 | import json 41 | import base64 42 | from functools import reduce 43 | 44 | image_mirrors = { 45 | "k8s.gcr.io/": "048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/google_containers/", 46 | "gcr.io/google-containers/": "048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/google_containers/", 47 | "gcr.io/": "048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/", 48 | "asia.gcr.io/": "048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/", 49 | "eu.gcr.io/": "048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/", 50 | "us.gcr.io/": "048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/gcr/", 51 | "quay.io/": "048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/quay/", 52 | "602401143452.dkr.ecr.us-west-2.amazonaws.com/": "048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/amazonecr/", 53 | "/": "048912060910.dkr.ecr.cn-northwest-1.amazonaws.com.cn/dockerhub/", 54 | "direct.to/": "" 55 | } 56 | 57 | def handler(event, context): 58 | request_body = json.loads(event["body"]) 59 | json_patch = [] 60 | 61 | # get initContainers from request and replace image path with JSON Patch 62 | initContainers = dict_get(request_body, "request.object.spec.initContainers") 63 | if initContainers: 64 | json_patch += image_patch(initContainers, "/spec/initContainers") 65 | 66 | # get containters from request and replace image path with JSON Patch 67 | containers = dict_get(request_body, "request.object.spec.containers") 68 | if containers: 69 | json_patch += image_patch(containers, "/spec/containers") 70 | 71 | print(json.dumps(json_patch)) 72 | # set response body 73 | patch_b64 = str(base64.b64encode(json.dumps(json_patch).encode("utf-8")), "utf-8") 74 | response_body = { 75 | "apiVersion": request_body["apiVersion"], 76 | "kind": "AdmissionReview", 77 | "response": { 78 | "uid": request_body["request"]["uid"], 79 | "allowed": True, 80 | "patch": patch_b64, 81 | "patchType": "JSONPatch" 82 | } 83 | } 84 | 85 | return { 86 | "body": json.dumps(response_body), 87 | "headers": { 88 | "Content-Type": "application/json" 89 | }, 90 | "statusCode": 200 91 | } 92 | 93 | def dict_get(dictionary, keys, default=None): 94 | return reduce(lambda d, key: d.get(key, default) if isinstance(d, dict) else default, keys.split("."), dictionary) 95 | 96 | def image_patch(containers, path_prefix): 97 | json_patch = [] 98 | for idx, container in enumerate(containers): 99 | image = container["image"] 100 | math_mirror=False 101 | for orig_image, mirror_image in image_mirrors.items(): 102 | if image.startswith(orig_image): 103 | math_mirror=True 104 | image = mirror_image + image[len(orig_image):] 105 | break 106 | if "/" in image_mirrors and math_mirror==False: 107 | if image.startswith("docker.io/"): 108 | image = image_mirrors["/"] + image[len("docker.io/"):] 109 | elif ("." not in image.split("/")[0]) or ("/" not in image): 110 | image = image_mirrors["/"] + image 111 | json_patch.append({"op": "replace", "path": "%s/%d/image" % (path_prefix, idx), "value": image}) 112 | return json_patch 113 | 114 | Handler: index.handler 115 | MemorySize: 128 116 | Role: !GetAtt lambdaIAMRole.Arn 117 | Runtime: python3.7 118 | Timeout: 60 119 | 120 | lambdaApiGatewayInvoke: 121 | Type: AWS::Lambda::Permission 122 | Properties: 123 | Action: lambda:InvokeFunction 124 | FunctionName: !GetAtt lambdaFunction.Arn 125 | Principal: apigateway.amazonaws.com 126 | SourceArn: !Sub "arn:${AWS::Partition}:execute-api:${AWS::Region}:${AWS::AccountId}:${apiGateway}/*/POST/" 127 | 128 | lambdaIAMRole: 129 | Type: AWS::IAM::Role 130 | Properties: 131 | AssumeRolePolicyDocument: 132 | Statement: 133 | - Effect: Allow 134 | Principal: 135 | Service: ['lambda.amazonaws.com'] 136 | Action: ['sts:AssumeRole'] 137 | ManagedPolicyArns: 138 | - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole 139 | 140 | Outputs: 141 | APIGateWayURL: 142 | Value: !Sub "https://${apiGateway}.execute-api.${AWS::Region}.${AWS::URLSuffix}/call" -------------------------------------------------------------------------------- /webhook/mutating-webhook.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: admissionregistration.k8s.io/v1 2 | kind: MutatingWebhookConfiguration 3 | metadata: 4 | name: image-mutating 5 | webhooks: 6 | - name: image.mutating.webhook 7 | failurePolicy: Ignore 8 | clientConfig: 9 | url: https://cnn2vome4a.execute-api.cn-northwest-1.amazonaws.com.cn/call 10 | caBundle: "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUQ3ekNDQXRlZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBRENCbURFTE1Ba0dBMVVFQmhNQ1ZWTXgKRURBT0JnTlZCQWdUQjBGeWFYcHZibUV4RXpBUkJnTlZCQWNUQ2xOamIzUjBjMlJoYkdVeEpUQWpCZ05WQkFvVApIRk4wWVhKbWFXVnNaQ0JVWldOb2JtOXNiMmRwWlhNc0lFbHVZeTR4T3pBNUJnTlZCQU1UTWxOMFlYSm1hV1ZzClpDQlRaWEoyYVdObGN5QlNiMjkwSUVObGNuUnBabWxqWVhSbElFRjFkR2h2Y21sMGVTQXRJRWN5TUI0WERUQTUKTURrd01UQXdNREF3TUZvWERUTTNNVEl6TVRJek5UazFPVm93Z1pneEN6QUpCZ05WQkFZVEFsVlRNUkF3RGdZRApWUVFJRXdkQmNtbDZiMjVoTVJNd0VRWURWUVFIRXdwVFkyOTBkSE5rWVd4bE1TVXdJd1lEVlFRS0V4eFRkR0Z5ClptbGxiR1FnVkdWamFHNXZiRzluYVdWekxDQkpibU11TVRzd09RWURWUVFERXpKVGRHRnlabWxsYkdRZ1UyVnkKZG1salpYTWdVbTl2ZENCRFpYSjBhV1pwWTJGMFpTQkJkWFJvYjNKcGRIa2dMU0JITWpDQ0FTSXdEUVlKS29aSQpodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU5VTU9zUXErVTdpOWI0WmwxK09pRk94SHovTHo1OGdFMjBwCk9zZ1BmVHozYTNZNFk5azJZS2liWGx3QWdMSXZXWC8yaC9rbFE0Ym5hUnRTbXBEaGNlUFlMUTFPYi9iSVNkbTIKOHhwV3JpdTJkQlRyei9zbTR4cTZIWll1YWp0WWxJbEhWdjhsb0pOd1U0UGFoSFFVdzJlZUJHZzYzNDVBV2gxSwpUczlEa1R2blZ0WUFjTXRTN250OXJqcm52REg1UmZiQ1lNOFRXUUlyZ013MFI5KzUzcEJsYlFMUExKR21wdWZlCmhSaEpmR1pPb3pwdHFiWHVOQzY2RFFPNE05OUg2N0ZyalNYWm04NkIwVVZHTXBad2g5NENEa2xEaGJac2M3dGsKNm1GQnJNblVWTitITDhjaXNpYk1uMWxVYUovOHZpb3Z4RlVjZFVCZ0Y0VUNWVG1MZndVQ0F3RUFBYU5DTUVBdwpEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU9CZ05WSFE4QkFmOEVCQU1DQVFZd0hRWURWUjBPQkJZRUZKeGZBTitxCkFkY3dLemlJb3JodFNwenlFWkdETUEwR0NTcUdTSWIzRFFFQkN3VUFBNElCQVFCTE5xYUVkMm5kT3htZlp5TUkKYnc1aHlmMkUzRi9ZTm9ITjJCdEJMWjlnM2NjYWFOblJib2JoaUNQUEU5NUR6K0kwc3dTZEh5blZ2L2hleU5YQgp2ZTZTYnpKMDhwR0NMNzJDUW5xdEtyY2dmVTI4ZWxVU3doWHF2ZmRxbFM1c2RKL1BITFR5eFFHamhkQnlQcTF6CnF3dWJkUXh0UmJlT2xLeVdON1dnMEk4VlJ3N2o2SVBkai8zdlFRRjN6Q2VwWW9VejhqY0k3M0hQZHdiZXlCa2QKaUVEUGZVWWQveDdINGM3L0k5dkcrbzFWVHFrQzUwY1JSajcwL2IxN0tTYTdxV0ZpTnlpMkxTcjJFSVpreVhDbgowcTIzS1hCNTZqemFZeVdmL1dpM01PeHcrM1dLdDIxZ1o3SWV5TG5wMktodkFvdG5EVTBtVjNIYUlQekJTbENOCnNTaTYKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQ==" 11 | rules: 12 | - operations: [ "CREATE", "UPDATE" ] 13 | apiGroups: [""] 14 | apiVersions: ["v1"] 15 | resources: ["pods"] 16 | admissionReviewVersions: ["v1"] 17 | sideEffects: None --------------------------------------------------------------------------------