├── .gitignore ├── 01. gitlab-in-docker └── README.md ├── 02. gitlab-in-docker-compose ├── README.md └── docker-compose.yaml ├── 03. gitlab-runner-with-shell-executor ├── GitLab-networking ├── README.md ├── config.toml ├── docker-compose.yml ├── executor.png ├── gitlab-ci.yml └── runner.png ├── 04. gitlab-runner-with-docker-executor-socket-binding ├── Dockerfile ├── GitLab-networking ├── README.md ├── config.toml ├── docker-compose.yml ├── executor.png ├── gitlab-ci.yml └── runner.png ├── 05. gitlab-runner-with-docker-executor-dind ├── Dockerfile ├── GitLab-networking ├── README.md ├── config.toml ├── docker-compose.yml ├── executor.png ├── gitlab-ci.yml └── runner.png ├── 06. gitlab-runner-with-kubernetes-executor ├── README.md ├── config.toml ├── docker-compose.yml ├── kind-cluster-config.yaml └── kind-service.yaml ├── 07. auto-register-gitlab-runner-with-docker-executor ├── README.md ├── docker-compose.yml └── gitlab-ci.yml ├── 08. build-docker-images-using-kaniko ├── .gitlab-ci.yml ├── Dockerfile ├── README.md ├── config.toml ├── docker-compose.yml └── src │ ├── .gitkeep │ └── mydockerfile ├── 09. setup-container-registry ├── .gitlab-ci.yml ├── Dockerfile ├── README.md ├── config.toml └── docker-compose.yml ├── 10. scan-container-images-in-registry ├── .gitlab-ci.yml ├── Dockerfile ├── README.md ├── config.toml └── docker-compose.yml ├── 11. scan-dependencies-in-gitlab-ci ├── .gitlab-ci.yml ├── README.md └── requirements.txt └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | gitlab 2 | *.pem 3 | 4 | -------------------------------------------------------------------------------- /01. gitlab-in-docker/README.md: -------------------------------------------------------------------------------- 1 | ### Dockerized GitLab: How to Easily Set Up Your Own GitLab Server 2 | 3 | 4 | GitLab official docker image on docker hub 5 | https://hub.docker.com/r/gitlab/gitlab-ce 6 | 7 | ``` 8 | docker run -p 8000:80 gitlab/gitlab-ce 9 | ``` 10 | 11 | Wait a couple of mins and then visit 12 | http://localhost:8000 13 | 14 | default user is `root` 15 | 16 | ##### Get root password? 17 | ``` 18 | docker ps --latest # copy GitLab container id from here 19 | docker exec -it {CONTAINER_ID} cat /etc/gitlab/initial_root_password 20 | ``` 21 | 22 | Login and create a test repo 23 | 24 | ##### Stop GitLab container 25 | ``` 26 | docker stop {CONTAINER_ID} 27 | ``` 28 | 29 | ##### Start GitLab container 30 | ``` 31 | docker start --attach {CONTAINER_ID} 32 | ``` 33 | 34 | Repo is there as it was before 35 | 36 | ##### What if i remove GitLab container at all 37 | ``` 38 | docker stop {CONTAINER_ID} 39 | docker rm {CONTAINER_ID} 40 | ``` 41 | 42 | And create GitLab container again 43 | ``` 44 | docker run --port 8000:80 gitlab/gitlab-ce 45 | ``` 46 | 47 | ###### What! repos are lost? 48 | because we deleted container and everythig within that container is lost 49 | So how to retain GitLab data? 50 | First, lets see where GitLab keep its data and configs 51 | ``` 52 | docker exec -it {CONTAINER_ID} ls -l /etc/gitlab 53 | ``` 54 | This ^ keep password, secrets and other configurations 55 | 56 | ``` 57 | docker exec -it {CONTAINER_ID} ls -l /var/opt/gitlab 58 | ``` 59 | 60 | This keep actual GitLab data including redis and postgres 61 | 62 | So we need to make these 2 dirs persistent from container into our host machine using 63 | docker volumes 64 | 65 | Keep old password: 16Gf2RkJBRnIKJe4kHH++klHtR53X7f1WOpI5/FCrYQ= 66 | to be compared wih new password 67 | 68 | ##### Using volumes 69 | ``` 70 | docker run -p 8000:80 -v ./gitlab/config:/etc/gitlab -v ./gitlab/data:/var/opt/gitlab gitlab/gitlab-ce 71 | ``` 72 | `./gitlab/config` is a dir on my host machine inside my current folder 73 | `/etc/gitlab` is a dir inside container 74 | 75 | Check `./gitlab` dir in current folder for both configs and data 76 | 77 | delete GitLab container and re-create. Everything should be there as it was 78 | ``` 79 | docker stop {CONTAINER_ID} 80 | docker rm {CONTAINER_ID} 81 | docker run -p 8000:80 -v ./gitlab/config:/etc/gitlab -v ./gitlab/data:/var/opt/gitlab gitlab/gitlab-ce 82 | ``` 83 | 84 | All these commands seems like too much manual commands running. How we can combine all these 85 | into a single file and a single command? 86 | Here comes docker compose. [See here](../2.%20gitlab-in-docker-compose) 87 | -------------------------------------------------------------------------------- /02. gitlab-in-docker-compose/README.md: -------------------------------------------------------------------------------- 1 | ### GitLab Setup Using Docker Compose: A Beginner’s Guide 2 | 3 | 4 | We'll be creating exactly those same things we did in prev video to setup gitlab server in docker container 5 | BUT this time instead of running and managing everything using docker commands and passing flags using command line, we'll be handling it using docker compose. Which makes it easy to because it has yaml file where you can 6 | instruct docker compose what to do including docker image, ports, container name, volumes etc 7 | 8 | GitLab official docker image on docker hub 9 | https://hub.docker.com/r/gitlab/gitlab-ce 10 | 11 | ``` 12 | docker compose up 13 | ``` 14 | 15 | Wait a couple of mins and then visit 16 | http://localhost:8088 17 | 18 | Login and create a test repo 19 | 20 | ##### Stop GitLab container 21 | ``` 22 | CTRL + C 23 | ``` 24 | 25 | OR 26 | 27 | in a separate terminal 28 | ``` 29 | docker compose stop 30 | ``` 31 | 32 | ##### Start GitLab container 33 | ``` 34 | docker compose up 35 | ``` 36 | 37 | Repo is there as it was before 38 | 39 | ##### What if i remove GitLab container at all 40 | ``` 41 | docker compose stop 42 | ``` 43 | ``` 44 | docker ps -a 45 | ``` 46 | Container should be there as stopped 47 | 48 | ``` 49 | docker compose down 50 | ``` 51 | ``` 52 | docker ps -a 53 | ``` 54 | No container anymore... 55 | 56 | And create GitLab container again 57 | ``` 58 | docker compose up 59 | ``` 60 | 61 | ###### What! repos are lost? 62 | because we deleted container and everythig within that container is lost 63 | So how to retain GitLab data? 64 | First, lets see where GitLab keep its data and configs 65 | ``` 66 | docker exec -it {CONTAINER_ID} ls -l /etc/gitlab 67 | ``` 68 | This ^ keep password, secrets and other configurations 69 | 70 | ``` 71 | docker exec -it {CONTAINER_ID} ls -l /var/opt/gitlab 72 | ``` 73 | 74 | This keep actual GitLab data including redis and postgres 75 | 76 | So we need to make these 2 dirs persistent from container into our host machine using 77 | docker volumes 78 | 79 | 80 | ##### Using volumes 81 | ``` 82 | docker compose up 83 | ``` 84 | `./gitlab/config` is a dir on my host machine inside my current folder 85 | `/etc/gitlab` is a dir inside container 86 | 87 | Check `./gitlab` dir in current folder for both configs and data 88 | 89 | delete GitLab container and re-create. Everything should be there as it was 90 | ``` 91 | docker compose stop 92 | ``` 93 | ``` 94 | docker compose down 95 | ``` 96 | ``` 97 | docker compose up 98 | ``` 99 | 100 | That was all about GitLab server in a container using docker compose. 101 | In the next part, i'll show you how to setup GitLab runner, connect it with GitLab server 102 | and register an executor to run CI/CDs jobs and pipelines 103 | [See here](../3.%20setup-gitlab-runner-with-docker-executor/runner-instance.md) 104 | -------------------------------------------------------------------------------- /02. gitlab-in-docker-compose/docker-compose.yaml: -------------------------------------------------------------------------------- 1 | version: '3.8' 2 | services: 3 | 4 | gitlab-server: 5 | image: 'gitlab/gitlab-ce:latest' 6 | container_name: gitlab-server 7 | environment: 8 | GITLAB_ROOT_EMAIL: "admin@BuildWithLal.com" 9 | GITLAB_ROOT_PASSWORD: "Abcd@0123456789" 10 | GITLAB_OMNIBUS_CONFIG: | 11 | puma['worker_processes'] = 0 # disable cluster mode to avoid more memory usage 12 | volumes: 13 | - ./gitlab/config:/etc/gitlab 14 | - ./gitlab/logs:/var/log/gitlab 15 | - ./gitlab/data:/var/opt/gitlab 16 | ports: 17 | - '8088:80' 18 | -------------------------------------------------------------------------------- /03. gitlab-runner-with-shell-executor/GitLab-networking: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BuildWithLal/gitlab-in-docker/a5b8ed80aac3fa2933226c684e6ba9edb1bc37f7/03. gitlab-runner-with-shell-executor/GitLab-networking -------------------------------------------------------------------------------- /03. gitlab-runner-with-shell-executor/README.md: -------------------------------------------------------------------------------- 1 | 2 | #### Runner Registeration Command for Docker in Docker 3 | ``` 4 | gitlab-runner register --url http://localhost:8000 \ 5 | --token glrt-ydjvGwY6HqXrtBwz9Myh 6 | ``` 7 | 8 | #### GitLab Networking 9 | ![Screenshot from 2024-09-11 15-08-50](https://github.com/user-attachments/assets/f6353038-44e6-433e-8a77-423f62f02840) 10 | 11 | -------------------------------------------------------------------------------- /03. gitlab-runner-with-shell-executor/config.toml: -------------------------------------------------------------------------------- 1 | concurrent = 1 2 | check_interval = 0 3 | connection_max_age = "15m0s" 4 | shutdown_timeout = 0 5 | 6 | [session_server] 7 | session_timeout = 1800 8 | 9 | [[runners]] 10 | name = "shell" 11 | url = "http://localhost:8000" 12 | id = 1 13 | token = "glrt-nyi1ULL8W9VHyA81ssSz" 14 | token_obtained_at = 2024-09-10T16:56:21Z 15 | token_expires_at = 0001-01-01T00:00:00Z 16 | executor = "shell" 17 | [runners.custom_build_dir] 18 | [runners.cache] 19 | MaxUploadedArchiveSize = 0 20 | [runners.cache.s3] 21 | [runners.cache.gcs] 22 | [runners.cache.azure] 23 | -------------------------------------------------------------------------------- /03. gitlab-runner-with-shell-executor/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3.8' 2 | services: 3 | 4 | gitlab-server: 5 | image: 'gitlab/gitlab-ce:latest' 6 | container_name: gitlab-server 7 | environment: 8 | GITLAB_ROOT_EMAIL: "admin@buildwithlal.com" 9 | GITLAB_ROOT_PASSWORD: "Abcd@0123456789" 10 | GITLAB_OMNIBUS_CONFIG: | 11 | external_url 'http://localhost:8000' 12 | nginx['listen_port'] = 8000 13 | ports: 14 | - '8000:8000' 15 | volumes: 16 | - ./gitlab/config:/etc/gitlab 17 | - ./gitlab/data:/var/opt/gitlab 18 | 19 | gitlab-runner: 20 | image: gitlab/gitlab-runner:alpine 21 | container_name: gitlab-runner 22 | network_mode: 'host' 23 | 24 | -------------------------------------------------------------------------------- /03. gitlab-runner-with-shell-executor/executor.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BuildWithLal/gitlab-in-docker/a5b8ed80aac3fa2933226c684e6ba9edb1bc37f7/03. gitlab-runner-with-shell-executor/executor.png -------------------------------------------------------------------------------- /03. gitlab-runner-with-shell-executor/gitlab-ci.yml: -------------------------------------------------------------------------------- 1 | build with shell executor: 2 | stage: build 3 | tags: 4 | - shell 5 | script: 6 | - date # print current date 7 | - cat /etc/os-release # print os version for Linux 8 | -------------------------------------------------------------------------------- /03. gitlab-runner-with-shell-executor/runner.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BuildWithLal/gitlab-in-docker/a5b8ed80aac3fa2933226c684e6ba9edb1bc37f7/03. gitlab-runner-with-shell-executor/runner.png -------------------------------------------------------------------------------- /04. gitlab-runner-with-docker-executor-socket-binding/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.10-alpine 2 | 3 | RUN python --version 4 | -------------------------------------------------------------------------------- /04. gitlab-runner-with-docker-executor-socket-binding/GitLab-networking: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BuildWithLal/gitlab-in-docker/a5b8ed80aac3fa2933226c684e6ba9edb1bc37f7/04. gitlab-runner-with-docker-executor-socket-binding/GitLab-networking -------------------------------------------------------------------------------- /04. gitlab-runner-with-docker-executor-socket-binding/README.md: -------------------------------------------------------------------------------- 1 | 2 | #### Runner Registeration Command for Docker using host's docker socket 3 | ``` 4 | gitlab-runner register --url http://localhost:8000 \ 5 | --token glrt-qL_FTjkAGqy7SHcYYStx \ 6 | --executor docker \ 7 | --name "Docker Runner" \ 8 | --docker-image "python:3.10-alpine" \ 9 | --docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \ 10 | --docker-network-mode "host" 11 | ``` 12 | 13 | #### GitLab Networking 14 | ![Screenshot from 2024-09-11 15-08-50](https://github.com/user-attachments/assets/f6353038-44e6-433e-8a77-423f62f02840) 15 | 16 | #### How GitLab runner use /var/run/docker.sock for container creation 17 | ![image](https://github.com/user-attachments/assets/1566848a-57a7-44e9-8a2d-9e98d7525e5c) 18 | 19 | 20 | -------------------------------------------------------------------------------- /04. gitlab-runner-with-docker-executor-socket-binding/config.toml: -------------------------------------------------------------------------------- 1 | concurrent = 1 2 | check_interval = 0 3 | connection_max_age = "15m0s" 4 | shutdown_timeout = 0 5 | 6 | [session_server] 7 | session_timeout = 1800 8 | 9 | [[runners]] 10 | name = "docker" 11 | url = "http://localhost:8000" 12 | id = 2 13 | token = "glrt-NUABKuyozAsRdsZQyjia" 14 | token_obtained_at = 2024-09-10T16:58:14Z 15 | token_expires_at = 0001-01-01T00:00:00Z 16 | executor = "docker" 17 | [runners.custom_build_dir] 18 | [runners.cache] 19 | MaxUploadedArchiveSize = 0 20 | [runners.cache.s3] 21 | [runners.cache.gcs] 22 | [runners.cache.azure] 23 | [runners.docker] 24 | tls_verify = false 25 | image = "python:3.10-alpine" 26 | privileged = false 27 | disable_entrypoint_overwrite = false 28 | oom_kill_disable = false 29 | disable_cache = false 30 | volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"] 31 | network_mode = "host" 32 | shm_size = 0 33 | network_mtu = 0 34 | 35 | -------------------------------------------------------------------------------- /04. gitlab-runner-with-docker-executor-socket-binding/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3.8' 2 | services: 3 | 4 | gitlab-server: 5 | image: 'gitlab/gitlab-ce:latest' 6 | container_name: gitlab-server 7 | environment: 8 | GITLAB_ROOT_EMAIL: "admin@buildwithlal.com" 9 | GITLAB_ROOT_PASSWORD: "Abcd@0123456789" 10 | GITLAB_OMNIBUS_CONFIG: | 11 | external_url 'http://localhost:8000' 12 | nginx['listen_port'] = 8000 13 | ports: 14 | - '8000:8000' 15 | volumes: 16 | - ./gitlab/config:/etc/gitlab 17 | - ./gitlab/data:/var/opt/gitlab 18 | 19 | gitlab-runner: 20 | image: gitlab/gitlab-runner:alpine 21 | container_name: gitlab-runner 22 | network_mode: 'host' 23 | volumes: 24 | - /var/run/docker.sock:/var/run/docker.sock 25 | -------------------------------------------------------------------------------- /04. gitlab-runner-with-docker-executor-socket-binding/executor.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BuildWithLal/gitlab-in-docker/a5b8ed80aac3fa2933226c684e6ba9edb1bc37f7/04. gitlab-runner-with-docker-executor-socket-binding/executor.png -------------------------------------------------------------------------------- /04. gitlab-runner-with-docker-executor-socket-binding/gitlab-ci.yml: -------------------------------------------------------------------------------- 1 | build with docker executor: 2 | stage: build 3 | tags: 4 | - docker 5 | image: docker:24.0.5 6 | 7 | script: 8 | - docker ps 9 | - docker run -d --rm --name nested-container1-in-pipelinejob alpine sleep 20 10 | - docker ps 11 | - docker run --rm --name nested-container2-in-pipelinejob alpine sleep 20 12 | 13 | build with docker executor default image: # default python image 14 | stage: build 15 | tags: 16 | - docker 17 | script: 18 | - python --version 19 | - sleep 10 20 | -------------------------------------------------------------------------------- /04. gitlab-runner-with-docker-executor-socket-binding/runner.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BuildWithLal/gitlab-in-docker/a5b8ed80aac3fa2933226c684e6ba9edb1bc37f7/04. gitlab-runner-with-docker-executor-socket-binding/runner.png -------------------------------------------------------------------------------- /05. gitlab-runner-with-docker-executor-dind/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.10-alpine 2 | 3 | RUN python --version 4 | -------------------------------------------------------------------------------- /05. gitlab-runner-with-docker-executor-dind/GitLab-networking: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BuildWithLal/gitlab-in-docker/a5b8ed80aac3fa2933226c684e6ba9edb1bc37f7/05. gitlab-runner-with-docker-executor-dind/GitLab-networking -------------------------------------------------------------------------------- /05. gitlab-runner-with-docker-executor-dind/README.md: -------------------------------------------------------------------------------- 1 | 2 | #### Runner Registeration Command for Docker in Docker 3 | ``` 4 | gitlab-runner register --url http://localhost:8000 \ 5 | --token glrt-qL_FTjkAGqy7SHcYYStx \ 6 | --executor docker \ 7 | --name "Docker in Docker Runner" \ 8 | --docker-image "docker:27.2.0" \ 9 | --docker-privileged \ 10 | --docker-volumes "/certs/client" \ 11 | --docker-network-mode "gitlab-in-docker" \ 12 | --clone-url "http://gitlab-server:8000" 13 | ``` 14 | 15 | #### GitLab Networking 16 | ![Screenshot from 2024-09-11 15-08-50](https://github.com/user-attachments/assets/f6353038-44e6-433e-8a77-423f62f02840) 17 | 18 | #### How GitLab runner use /var/run/docker.sock for container creation 19 | ![image](https://github.com/user-attachments/assets/1566848a-57a7-44e9-8a2d-9e98d7525e5c) 20 | 21 | #### How GitLab runner use Docker-in-Docker service for container creation 22 | ![image](https://github.com/user-attachments/assets/54cdbd27-ce02-4727-9040-9c3792757b7b) 23 | 24 | 25 | -------------------------------------------------------------------------------- /05. gitlab-runner-with-docker-executor-dind/config.toml: -------------------------------------------------------------------------------- 1 | concurrent = 1 2 | check_interval = 0 3 | connection_max_age = "15m0s" 4 | shutdown_timeout = 0 5 | 6 | [session_server] 7 | session_timeout = 1800 8 | 9 | [[runners]] 10 | name = "Docker in Docker Runner" 11 | url = "http://localhost:8000" 12 | id = 4 13 | token = "glrt-qL_FTjkAGqy7SHcYYStx" 14 | token_obtained_at = 2024-09-11T09:23:50Z 15 | token_expires_at = 0001-01-01T00:00:00Z 16 | executor = "docker" 17 | clone_url = "http://gitlab-server:8000" 18 | [runners.custom_build_dir] 19 | [runners.cache] 20 | MaxUploadedArchiveSize = 0 21 | [runners.cache.s3] 22 | [runners.cache.gcs] 23 | [runners.cache.azure] 24 | [runners.docker] 25 | tls_verify = false 26 | image = "docker:27.2.0" 27 | privileged = true 28 | disable_entrypoint_overwrite = false 29 | oom_kill_disable = false 30 | disable_cache = false 31 | volumes = ["/certs/client", "/cache"] 32 | network_mode = "gitlab-in-docker" 33 | shm_size = 0 34 | network_mtu = 0 35 | -------------------------------------------------------------------------------- /05. gitlab-runner-with-docker-executor-dind/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3.8' 2 | services: 3 | 4 | gitlab-server: 5 | image: 'gitlab/gitlab-ce:latest' 6 | container_name: gitlab-server 7 | environment: 8 | GITLAB_ROOT_EMAIL: "admin@buildwithlal.com" 9 | GITLAB_ROOT_PASSWORD: "Abcd@0123456789" 10 | GITLAB_OMNIBUS_CONFIG: | 11 | external_url 'http://localhost:8000' 12 | nginx['listen_port'] = 8000 13 | ports: 14 | - '8000:8000' 15 | volumes: 16 | - ./gitlab/config:/etc/gitlab 17 | - ./gitlab/data:/var/opt/gitlab 18 | networks: 19 | - gitlab-in-docker 20 | 21 | gitlab-runner: 22 | image: gitlab/gitlab-runner:alpine 23 | container_name: gitlab-runner 24 | network_mode: 'host' 25 | volumes: 26 | - /var/run/docker.sock:/var/run/docker.sock 27 | 28 | networks: 29 | gitlab-in-docker: 30 | name: gitlab-in-docker 31 | driver: bridge 32 | -------------------------------------------------------------------------------- /05. gitlab-runner-with-docker-executor-dind/executor.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BuildWithLal/gitlab-in-docker/a5b8ed80aac3fa2933226c684e6ba9edb1bc37f7/05. gitlab-runner-with-docker-executor-dind/executor.png -------------------------------------------------------------------------------- /05. gitlab-runner-with-docker-executor-dind/gitlab-ci.yml: -------------------------------------------------------------------------------- 1 | build with docker in docker: 2 | stage: build 3 | image: docker:27.2.0 4 | services: 5 | - docker:27.2.0-dind 6 | variables: 7 | DOCKER_HOST: tcp://docker:2376 8 | DOCKER_TLS_CERTDIR: "/certs" 9 | 10 | tags: 11 | - docker-in-docker 12 | script: 13 | - docker ps 14 | - docker run -d --rm --name nested-container1-in-pipelinejob alpine sleep 20 15 | - docker run -d --rm --name nested-container2-in-pipelinejob alpine sleep 20 16 | - docker ps 17 | - sleep 20 18 | -------------------------------------------------------------------------------- /05. gitlab-runner-with-docker-executor-dind/runner.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BuildWithLal/gitlab-in-docker/a5b8ed80aac3fa2933226c684e6ba9edb1bc37f7/05. gitlab-runner-with-docker-executor-dind/runner.png -------------------------------------------------------------------------------- /06. gitlab-runner-with-kubernetes-executor/README.md: -------------------------------------------------------------------------------- 1 | ``` 2 | kind create cluster --config kind-cluster-config.yaml 3 | ``` 4 | 5 | ``` 6 | kubectl apply -f kind-service.yaml 7 | ``` 8 | -------------------------------------------------------------------------------- /06. gitlab-runner-with-kubernetes-executor/config.toml: -------------------------------------------------------------------------------- 1 | concurrent = 1 2 | check_interval = 0 3 | connection_max_age = "15m0s" 4 | shutdown_timeout = 0 5 | 6 | [session_server] 7 | session_timeout = 1800 8 | 9 | 10 | [[runners]] 11 | name = "kubernetes" 12 | url = "http://localhost:8000" 13 | id = 2 14 | token = "glrt-cgAaJpyx1zks4FZKQcde" 15 | token_obtained_at = 2024-09-15T16:35:49Z 16 | token_expires_at = 0001-01-01T00:00:00Z 17 | executor = "kubernetes" 18 | clone_url = "http://gitlab-server:8000" 19 | [runners.custom_build_dir] 20 | [runners.cache] 21 | MaxUploadedArchiveSize = 0 22 | [runners.cache.s3] 23 | [runners.cache.gcs] 24 | [runners.cache.azure] 25 | [runners.kubernetes] 26 | host = "https://localhost:46811" 27 | cert_file = "/etc/gitlab-runner/kubernetes/cert/apiserver-kubelet-client.crt" 28 | key_file = "/etc/gitlab-runner/kubernetes/cert/apiserver-kubelet-client.key" 29 | ca_file = "/etc/gitlab-runner/kubernetes/cert/ca.crt" 30 | privileged = true 31 | bearer_token_overwrite_allowed = false 32 | image = "docker:latest" 33 | namespace = "default" 34 | namespace_per_job = false 35 | network_mode = "host" 36 | [runners.kubernetes.init_permissions_container_security_context] 37 | [runners.kubernetes.init_permissions_container_security_context.capabilities] 38 | [runners.kubernetes.build_container_security_context] 39 | [runners.kubernetes.build_container_security_context.capabilities] 40 | [runners.kubernetes.helper_container_security_context] 41 | [runners.kubernetes.helper_container_security_context.capabilities] 42 | [runners.kubernetes.service_container_security_context] 43 | [runners.kubernetes.service_container_security_context.capabilities] 44 | [runners.kubernetes.volumes] 45 | [runners.kubernetes.dns_config] 46 | -------------------------------------------------------------------------------- /06. gitlab-runner-with-kubernetes-executor/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3.8' 2 | services: 3 | 4 | gitlab-server: 5 | image: 'gitlab/gitlab-ce:latest' 6 | container_name: gitlab-server 7 | environment: 8 | GITLAB_ROOT_EMAIL: "admin@buildwithlal.com" 9 | GITLAB_ROOT_PASSWORD: "Abcd@0123456789" 10 | GITLAB_OMNIBUS_CONFIG: | 11 | external_url 'http://localhost:8000' 12 | nginx['listen_port'] = 8000 13 | gitlab_rails['registry_enabled'] = true 14 | registry_external_url 'http://dockerhost:5001' 15 | ports: 16 | - '8000:8000' 17 | - '5001:5001' 18 | volumes: 19 | - ./gitlab/config:/etc/gitlab 20 | - ./gitlab/data:/var/opt/gitlab 21 | - ./gitlab/ssl:/etc/gitlab/ssl/ 22 | networks: 23 | - kind 24 | 25 | gitlab-runner: 26 | image: gitlab/gitlab-runner:alpine 27 | container_name: gitlab-runner 28 | network_mode: 'host' 29 | volumes: 30 | - /var/run/docker.sock:/var/run/docker.sock 31 | - ./gitlab/kubernetes:/etc/gitlab-runner/kubernetes 32 | 33 | networks: 34 | kind: 35 | name: kind 36 | driver: bridge 37 | -------------------------------------------------------------------------------- /06. gitlab-runner-with-kubernetes-executor/kind-cluster-config.yaml: -------------------------------------------------------------------------------- 1 | kind: Cluster 2 | apiVersion: kind.x-k8s.io/v1alpha4 3 | name: gitlab-ci-cluster 4 | nodes: 5 | - role: control-plane 6 | extraPortMappings: 7 | - containerPort: 80 8 | hostPort: 8088 9 | 10 | -------------------------------------------------------------------------------- /06. gitlab-runner-with-kubernetes-executor/kind-service.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: Endpoints 4 | metadata: 5 | name: dockerhost 6 | subsets: 7 | - addresses: 8 | - ip: 172.22.0.1 # this is the gateway IP in the "bridge" docker network 9 | --- 10 | apiVersion: v1 11 | kind: Service 12 | metadata: 13 | name: dockerhost 14 | spec: 15 | clusterIP: None 16 | -------------------------------------------------------------------------------- /07. auto-register-gitlab-runner-with-docker-executor/README.md: -------------------------------------------------------------------------------- 1 | 2 | ---------- 3 | Existing 4 | ---------- 5 | 6 | Add gitlab service without health check and runner token to compose 7 | Start GitLab server. 8 | Show runners. There shouldn't be any 9 | 10 | ----------- 11 | New 12 | ----------- 13 | Add GitLab runner service in docker compose 14 | 15 | Add runner shared token as env var to gitlab server service in docker compose. 16 | this shared token will be used when registering gitlab runner from gitlab runner container 17 | 18 | 19 | adding entrypoint so /bin/sh works 20 | 21 | ``` 22 | entrypoint: [""] 23 | ``` 24 | 25 | 26 | add runner registeration command and re-run gitlab runner to load new config 27 | 28 | ``` 29 | command: ["/bin/sh", "-c", "gitlab-runner register \ 30 | --non-interactive \ 31 | --url 'http://localhost:8088' \ 32 | --registration-token 'r3g1str4t10n' \ 33 | --executor 'docker' \ 34 | --docker-network-mode 'host' \ 35 | --docker-image 'python:alpine' \ 36 | && gitlab-runner run --user=gitlab-runner --working-directory=/etc/gitlab-runner"] 37 | ``` 38 | 39 | Also this command will only work if the gitlab-server is running and accepting requests. 40 | so we need to make runner container dependent server container 41 | 42 | we need to add health check to gitlab server 43 | ``` 44 | healthcheck: 45 | test: curl --fail http://localhost:8088/users/sign_in || exit 1 46 | interval: 60s 47 | timeout: 3s 48 | retries: 5 49 | ``` 50 | 51 | and add depends_on to runner container so runner will only starts once server container is started and 52 | gitlab server is ready to accept requests 53 | ``` 54 | depends_on: 55 | gitlab-server: 56 | condition: service_healthy 57 | ``` 58 | -------------------------------------------------------------------------------- /07. auto-register-gitlab-runner-with-docker-executor/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3.8' 2 | services: 3 | 4 | gitlab-server: 5 | image: gitlab/gitlab-ce:latest 6 | container_name: gitlab-server 7 | environment: 8 | GITLAB_ROOT_EMAIL: "admin@buildwithlal.com" 9 | GITLAB_ROOT_PASSWORD: "Abcd@0123456789" 10 | GITLAB_OMNIBUS_CONFIG: | 11 | external_url 'http://localhost:8000' 12 | nginx['listen_port'] = 8000 13 | GITLAB_SHARED_RUNNERS_REGISTRATION_TOKEN: r3g1str4t10n 14 | volumes: 15 | - ./gitlab/config:/etc/gitlab 16 | - ./gitlab/logs:/var/log/gitlab 17 | - ./gitlab/data:/var/opt/gitlab 18 | ports: 19 | - '8000:8000' 20 | healthcheck: 21 | test: curl --fail http://localhost:8000/users/sign_in || exit 1 22 | interval: 60s 23 | timeout: 3s 24 | retries: 5 25 | 26 | gitlab-runner: 27 | image: gitlab/gitlab-runner:latest 28 | container_name: gitlab-runner 29 | entrypoint: [""] 30 | command: ["/bin/sh", "-c", "gitlab-runner register \ 31 | --non-interactive \ 32 | --url 'http://localhost:8000' \ 33 | --registration-token 'r3g1str4t10n' \ 34 | --executor 'docker' \ 35 | --docker-network-mode 'host' \ 36 | --docker-image 'python:alpine' \ 37 | && gitlab-runner run --user=gitlab-runner --working-directory=/etc/gitlab-runner"] 38 | volumes: 39 | - /var/run/docker.sock:/var/run/docker.sock 40 | network_mode: 'host' 41 | depends_on: 42 | gitlab-server: 43 | condition: service_healthy 44 | -------------------------------------------------------------------------------- /07. auto-register-gitlab-runner-with-docker-executor/gitlab-ci.yml: -------------------------------------------------------------------------------- 1 | build: 2 | image: python:alpine 3 | script: 4 | - python --version 5 | -------------------------------------------------------------------------------- /08. build-docker-images-using-kaniko/.gitlab-ci.yml: -------------------------------------------------------------------------------- 1 | build with kaniko: 2 | stage: build 3 | tags: 4 | - kaniko 5 | image: 6 | name: gcr.io/kaniko-project/executor:v1.23.2-debug 7 | entrypoint: [""] 8 | before_script: 9 | # registry URL: https://index.docker.io/v1/ 10 | - echo "{\"auths\":{\"https://index.docker.io/v1/\":{\"auth\":\"$(printf "%s:%s" "REGISTRY_USERNAME" "REGISTRY_PASSWORD" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json 11 | script: 12 | 13 | - echo ${CI_PROJECT_DIR} 14 | 15 | # conventional project with Dockerfile inside root dir 16 | - /kaniko/executor --no-push 17 | 18 | # Dockerfile is in some other directory than root dir 19 | - /kaniko/executor --no-push --dockerfile "${CI_PROJECT_DIR}/src/mydockerfile" 20 | 21 | - /kaniko/executor 22 | --dockerfile "${CI_PROJECT_DIR}/src/Dockerfile" 23 | --destination "REGISTRY_USERNAME/REGISTRY_REPO_NAME:${CI_COMMIT_TAG}" 24 | 25 | 26 | 27 | -------------------------------------------------------------------------------- /08. build-docker-images-using-kaniko/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.10-alpine 2 | 3 | RUN python --version -------------------------------------------------------------------------------- /08. build-docker-images-using-kaniko/README.md: -------------------------------------------------------------------------------- 1 | #### Job executor using /var/run/docker.sock from host docker engine 2 | ![image](https://github.com/user-attachments/assets/fa5ef2e3-e5ec-4f32-b90f-49f029c18aa3) 3 | 4 |
5 | 6 | #### Job executor using Docker-in-Docker service 7 | ![image](https://github.com/user-attachments/assets/ccb7f92b-09a3-4e4f-9f6d-47eda0f15521) 8 | 9 | 10 |
11 | 12 | #### Register Kaniko Runner 13 | ``` 14 | gitlab-runner register \ 15 | --url http://localhost:8000 \ 16 | --token glrt-Y9BCAyZZyFhrrkezJbC8 \ 17 | --executor docker \ 18 | --docker-image "gcr.io/kaniko-project/executor:v1.23.2-debug" \ 19 | --docker-network-mode "host" 20 | ``` 21 | -------------------------------------------------------------------------------- /08. build-docker-images-using-kaniko/config.toml: -------------------------------------------------------------------------------- 1 | concurrent = 1 2 | check_interval = 0 3 | connection_max_age = "15m0s" 4 | shutdown_timeout = 0 5 | 6 | [session_server] 7 | session_timeout = 1800 8 | 9 | [[runners]] 10 | name = "docker" 11 | url = "http://localhost:8000" 12 | id = 2 13 | token = "glrt-NUABKuyozAsRdsZQyjia" 14 | token_obtained_at = 2024-09-10T16:58:14Z 15 | token_expires_at = 0001-01-01T00:00:00Z 16 | executor = "docker" 17 | [runners.custom_build_dir] 18 | [runners.cache] 19 | MaxUploadedArchiveSize = 0 20 | [runners.cache.s3] 21 | [runners.cache.gcs] 22 | [runners.cache.azure] 23 | [runners.docker] 24 | tls_verify = false 25 | image = "python:3.10-alpine" 26 | privileged = false 27 | disable_entrypoint_overwrite = false 28 | oom_kill_disable = false 29 | disable_cache = false 30 | volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"] 31 | network_mode = "host" 32 | shm_size = 0 33 | network_mtu = 0 34 | -------------------------------------------------------------------------------- /08. build-docker-images-using-kaniko/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3.8' 2 | services: 3 | 4 | gitlab-server: 5 | image: 'gitlab/gitlab-ce:latest' 6 | container_name: gitlab-server 7 | environment: 8 | GITLAB_ROOT_EMAIL: "admin@buildwithlal.com" 9 | GITLAB_ROOT_PASSWORD: "Abcd@0123456789" 10 | GITLAB_OMNIBUS_CONFIG: | 11 | external_url 'http://localhost:8000' 12 | nginx['listen_port'] = 8000 13 | ports: 14 | - '8000:8000' 15 | volumes: 16 | - ./gitlab/config:/etc/gitlab 17 | - ./gitlab/data:/var/opt/gitlab 18 | 19 | gitlab-runner: 20 | image: gitlab/gitlab-runner:alpine 21 | container_name: gitlab-runner 22 | network_mode: 'host' 23 | volumes: 24 | - /var/run/docker.sock:/var/run/docker.sock 25 | -------------------------------------------------------------------------------- /08. build-docker-images-using-kaniko/src/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BuildWithLal/gitlab-in-docker/a5b8ed80aac3fa2933226c684e6ba9edb1bc37f7/08. build-docker-images-using-kaniko/src/.gitkeep -------------------------------------------------------------------------------- /08. build-docker-images-using-kaniko/src/mydockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.9-alpine 2 | 3 | RUN python --version -------------------------------------------------------------------------------- /09. setup-container-registry/.gitlab-ci.yml: -------------------------------------------------------------------------------- 1 | build: 2 | stage: build 3 | image: docker:latest 4 | tags: 5 | - docker 6 | script: 7 | - echo "$CI_REGISTRY_PASSWORD" | docker login $CI_REGISTRY -u $CI_REGISTRY_USER --password-stdin 8 | - docker build -t "$CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:8}" . 9 | - docker push "$CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:8}" 10 | -------------------------------------------------------------------------------- /09. setup-container-registry/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.10-alpine 2 | 3 | RUN python --version -------------------------------------------------------------------------------- /09. setup-container-registry/README.md: -------------------------------------------------------------------------------- 1 | #### Enable GitLab Container Registry 2 | 3 | **Update `docker-compose.yml`** 4 | 5 | Add these lines to enable and expose the registry. 6 | 7 | ```yaml 8 | gitlab_rails['registry_enabled'] = true 9 | registry_external_url 'http://localhost:5001' 10 | ``` 11 | 12 | **Restart Containers** 13 | 14 | Run `docker compose up --build --force-recreate` 15 | 16 | 17 | #### Setup a New Repository 18 | 19 | 1. **Login to GitLab**: Create a new repository and access the Container Registry section. 20 | 2. **Add a Dockerfile** to the project root for testing purposes. 21 | 22 | 23 | #### Configure Pipeline for Image Build and Push 24 | 25 | 1. **Register a GitLab Runner** as a Docker executor (if not done already). 26 | 2. **Setup `gitlab-ci.yml` Pipeline**: 27 | ```yaml 28 | build: 29 | stage: build 30 | image: docker:latest 31 | tags: 32 | - docker 33 | script: 34 | - echo "$CI_REGISTRY_PASSWORD" | docker login $CI_REGISTRY -u $CI_REGISTRY_USER --password-stdin 35 | - docker build -t "$CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:8}" . 36 | - docker push "$CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:8}" 37 | ``` 38 | This pipeline logs into the registry, builds the Docker image, tags it, and pushes it to GitLab’s Container Registry. 39 | 40 | 41 | #### Pull Image from Registry 42 | 43 | 1. **Copy the Image Path** from the registry and run: 44 | ```bash 45 | docker pull localhost:5001/root/build-with-lal: 46 | ``` 47 | 2. **Authenticate if Needed**: 48 | ```bash 49 | docker login --username --password localhost:5001 50 | ``` 51 | 52 | 53 | #### Container Registry Storage Backends 54 | 55 | - **File System** (default): `registry_path` can be customized in `gitlab.rb`. 56 | - **Other Options**: Azure, Google Cloud Storage, and S3. 57 | 58 | 59 | #### Notes 60 | 61 | - **Third-Party Registries** are no longer supported as of GitLab 16.0. 62 | - **Registry Authentication**: GitLab manages authentication for secure image access. 63 | -------------------------------------------------------------------------------- /09. setup-container-registry/config.toml: -------------------------------------------------------------------------------- 1 | concurrent = 1 2 | check_interval = 0 3 | connection_max_age = "15m0s" 4 | shutdown_timeout = 0 5 | 6 | [session_server] 7 | session_timeout = 1800 8 | 9 | [[runners]] 10 | name = "docker" 11 | url = "http://localhost:8000" 12 | id = 2 13 | token = "glrt-NUABKuyozAsRdsZQyjia" 14 | token_obtained_at = 2024-09-10T16:58:14Z 15 | token_expires_at = 0001-01-01T00:00:00Z 16 | executor = "docker" 17 | [runners.custom_build_dir] 18 | [runners.cache] 19 | MaxUploadedArchiveSize = 0 20 | [runners.cache.s3] 21 | [runners.cache.gcs] 22 | [runners.cache.azure] 23 | [runners.docker] 24 | tls_verify = false 25 | image = "docker" 26 | privileged = false 27 | disable_entrypoint_overwrite = false 28 | oom_kill_disable = false 29 | disable_cache = false 30 | volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"] 31 | network_mode = "host" 32 | shm_size = 0 33 | network_mtu = 0 34 | -------------------------------------------------------------------------------- /09. setup-container-registry/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3.8' 2 | services: 3 | 4 | gitlab-server: 5 | image: 'gitlab/gitlab-ce:latest' 6 | container_name: gitlab-server 7 | environment: 8 | GITLAB_ROOT_EMAIL: "admin@buildwithlal.com" 9 | GITLAB_ROOT_PASSWORD: "Abcd@0123456789" 10 | GITLAB_OMNIBUS_CONFIG: | 11 | external_url 'http://localhost:8000' 12 | nginx['listen_port'] = 8000 13 | 14 | # new changes for enabling container registry 15 | gitlab_rails['registry_enabled'] = true 16 | 17 | # docker CLI from inside the pipeline's job will use this URL to push docker images 18 | registry_external_url 'http://localhost:5001' 19 | ports: 20 | - '8000:8000' 21 | 22 | # expose new port for registry hosting 23 | - '5001:5001' 24 | volumes: 25 | - ./gitlab/config:/etc/gitlab 26 | - ./gitlab/data:/var/opt/gitlab 27 | 28 | gitlab-runner: 29 | image: gitlab/gitlab-runner:alpine 30 | container_name: gitlab-runner 31 | network_mode: 'host' 32 | volumes: 33 | - /var/run/docker.sock:/var/run/docker.sock 34 | -------------------------------------------------------------------------------- /10. scan-container-images-in-registry/.gitlab-ci.yml: -------------------------------------------------------------------------------- 1 | include: 2 | - template: Jobs/Container-Scanning.gitlab-ci.yml 3 | 4 | container_scanning: 5 | allow_failure: false 6 | variables: 7 | CS_SEVERITY_THRESHOLD: 'medium' # low, medium, high anything below medium severity to not be reported by the analyzer 8 | SECURE_LOG_LEVEL: 'debug' # debug, info, warn, error, fatal 9 | before_script: 10 | - sudo apt-get update -y && sudo apt-get install -y jq 11 | - export CS_IMAGE="$CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:8}" 12 | script: 13 | - gtcs scan 14 | - HIGH_SEVERITY_COUNT=$(jq '.vulnerabilities | map(select(.severity=="High")) | length' gl-container-scanning-report.json ) 15 | - > 16 | if [[ $HIGH_SEVERITY_COUNT -gt 0 ]] 17 | then 18 | echo "High severity found"; exit 1 19 | else 20 | echo "No High severity found" 21 | fi 22 | 23 | build: 24 | stage: build 25 | image: docker:latest 26 | script: 27 | - echo "$CI_REGISTRY_PASSWORD" | docker login $CI_REGISTRY -u $CI_REGISTRY_USER --password-stdin 28 | - docker build -t "$CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:8}" . 29 | - docker push "$CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:8}" 30 | 31 | deploy: 32 | stage: deploy 33 | image: alpine 34 | script: 35 | - echo "Deployed" 36 | -------------------------------------------------------------------------------- /10. scan-container-images-in-registry/Dockerfile: -------------------------------------------------------------------------------- 1 | # low vulnerability docker image 2 | FROM python:3.10-alpine 3 | RUN python --version 4 | 5 | 6 | # High and critical vulnerability docker image 7 | # FROM vulnerables/web-dvwa 8 | # RUN echo "scan me !" 9 | -------------------------------------------------------------------------------- /10. scan-container-images-in-registry/README.md: -------------------------------------------------------------------------------- 1 | ##### Scan Docker images for security issues in external libraries. Uses Trivy to identify vulnerabilities in dependencies. 2 | ___ 3 | 4 | ![image](https://github.com/user-attachments/assets/89909746-8d1d-4761-a914-efdb6f59fce7) 5 | 6 | 7 | GitLab shows vulnerabilities in merge requests and saves a downloadable report. 8 | 9 | #### Requirements 10 | - Add a `test` stage in `.gitlab-ci.yml`. 11 | - Use a GitLab Runner with Docker or Kubernetes. 12 | - Push the Docker image to the project’s container registry. 13 | - If using a third-party registry, set `CS_REGISTRY_USER` and `CS_REGISTRY_PASSWORD`. 14 | 15 | #### Enable the Scanner 16 | - Add this to `.gitlab-ci.yml`: 17 | ```yaml 18 | include: 19 | - template: Jobs/Container-Scanning.gitlab-ci.yml 20 | ``` 21 | 22 | #### Set up Image Build & Push 23 | - Define a `build` stage to build and push your Docker image: 24 | ```yaml 25 | build: 26 | stage: build 27 | image: docker:latest 28 | script: 29 | - echo "$CI_REGISTRY_PASSWORD" | docker login $CI_REGISTRY -u $CI_REGISTRY_USER --password-stdin 30 | - docker build -t "$CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:8}" . 31 | - docker push "$CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:8}" 32 | ``` 33 | 34 | #### Specify Scanned Image 35 | - Update the `container_scanning` job to target the built image: 36 | ```yaml 37 | container_scanning: 38 | before_script: 39 | - export CS_IMAGE="$CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:8}" 40 | ``` 41 | 42 | #### Customization 43 | 44 | - **Severity Threshold**: 45 | - Set minimum severity level to report (e.g., `medium` or `high`): 46 | ```yaml 47 | container_scanning: 48 | variables: 49 | CS_SEVERITY_THRESHOLD: 'medium' 50 | ``` 51 | 52 | - **Fail Pipeline on High Severity**: 53 | - Use `jq` to parse results and fail the job if high-severity issues are found: 54 | ```yaml 55 | container_scanning: 56 | allow_failure: false 57 | before_script: 58 | - sudo apt-get update -y && sudo apt-get install -y jq 59 | - export CS_IMAGE="$CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:8}" 60 | script: 61 | - gtcs scan 62 | - HIGH_SEVERITY_COUNT=$(jq '.vulnerabilities | map(select(.severity=="High")) | length' gl-container-scanning-report.json) 63 | - if [[ $HIGH_SEVERITY_COUNT -gt 0 ]]; then exit 1; fi 64 | ``` 65 | -------------------------------------------------------------------------------- /10. scan-container-images-in-registry/config.toml: -------------------------------------------------------------------------------- 1 | concurrent = 1 2 | check_interval = 0 3 | connection_max_age = "15m0s" 4 | shutdown_timeout = 0 5 | 6 | [session_server] 7 | session_timeout = 1800 8 | 9 | [[runners]] 10 | name = "docker" 11 | url = "http://localhost:8000" 12 | id = 2 13 | token = "glrt-NUABKuyozAsRdsZQyjia" 14 | token_obtained_at = 2024-09-10T16:58:14Z 15 | token_expires_at = 0001-01-01T00:00:00Z 16 | executor = "docker" 17 | [runners.custom_build_dir] 18 | [runners.cache] 19 | MaxUploadedArchiveSize = 0 20 | [runners.cache.s3] 21 | [runners.cache.gcs] 22 | [runners.cache.azure] 23 | [runners.docker] 24 | tls_verify = false 25 | image = "python:3.10-alpine" 26 | privileged = false 27 | disable_entrypoint_overwrite = false 28 | oom_kill_disable = false 29 | disable_cache = false 30 | volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"] 31 | network_mode = "host" 32 | shm_size = 0 33 | network_mtu = 0 34 | 35 | -------------------------------------------------------------------------------- /10. scan-container-images-in-registry/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3.8' 2 | services: 3 | 4 | gitlab-server: 5 | image: 'gitlab/gitlab-ce:latest' 6 | container_name: gitlab-server 7 | environment: 8 | GITLAB_ROOT_EMAIL: "admin@buildwithlal.com" 9 | GITLAB_ROOT_PASSWORD: "Abcd@0123456789" 10 | GITLAB_OMNIBUS_CONFIG: | 11 | puma['worker_processes'] = 0 # disable cluster mode to avoid more memory usage 12 | external_url 'http://localhost:8000' 13 | nginx['listen_port'] = 8000 14 | gitlab_rails['registry_enabled'] = true 15 | registry_external_url 'http://localhost:5001' 16 | ports: 17 | - '8000:8000' 18 | - '5001:5001' 19 | volumes: 20 | - ./gitlab/config:/etc/gitlab 21 | - ./gitlab/data:/var/opt/gitlab 22 | 23 | gitlab-runner: 24 | image: gitlab/gitlab-runner:alpine 25 | container_name: gitlab-runner 26 | network_mode: 'host' 27 | volumes: 28 | - /var/run/docker.sock:/var/run/docker.sock 29 | -------------------------------------------------------------------------------- /11. scan-dependencies-in-gitlab-ci/.gitlab-ci.yml: -------------------------------------------------------------------------------- 1 | include: 2 | - template: Jobs/Dependency-Scanning.gitlab-ci.yml 3 | 4 | build: 5 | stage: build 6 | image: alpine 7 | script: 8 | - echo "Build project" 9 | 10 | gemnasium-python-dependency_scanning: 11 | allow_failure: false # force pipeline to fail if this job fails 12 | 13 | before_script: 14 | - apt-get update -y && apt-get install -y jq 15 | 16 | script: 17 | - /analyzer run 18 | - HIGH_SEVERITY_COUNT=$(jq '.vulnerabilities | map(select(.severity=="High")) | length' gl-dependency-scanning-report.json ) 19 | - > 20 | if [[ $HIGH_SEVERITY_COUNT -gt 0 ]] 21 | then 22 | echo "High severity found"; exit 1 23 | else 24 | echo "No High severity found" 25 | fi 26 | deploy: 27 | stage: deploy 28 | image: alpine 29 | script: 30 | - echo "Deployed" -------------------------------------------------------------------------------- /11. scan-dependencies-in-gitlab-ci/README.md: -------------------------------------------------------------------------------- 1 | ##### Scans your project’s external libraries for security issues, including nested (transitive) dependencies. 2 | 3 | ___ 4 | 5 | ![image](https://github.com/user-attachments/assets/61818fe4-b81e-4b46-b3b9-b6d9ba5b98cf) 6 | 7 | 8 | #### Requirements 9 | - GitLab Ultimate (GitLab.com, Self-managed, or Dedicated). 10 | - Add a `test` stage in your `.gitlab-ci.yml`. 11 | - A GitLab Runner with Docker or Kubernetes executor. 12 | 13 | #### Steps 14 | - In `.gitlab-ci.yml`, add this to enable the Dependency Scanning template: 15 | ```yaml 16 | include: 17 | - template: Jobs/Dependency-Scanning.gitlab-ci.yml 18 | ``` 19 | - Use specific dependency files like `requirements.txt` (up to 2 directories deep) to trigger the scan. 20 | 21 | #### Example 22 | - Add a test dependency (e.g., `requests==2.19.1` in `requirements.txt`) to check for vulnerabilities. 23 | - Run the pipeline to see scan results in logs and as a downloadable artifact. 24 | 25 | #### Fail on High Severity 26 | - Use `jq` to parse the scan report and stop the pipeline if high-severity issues are found, blocking deployment if any vulnerabilities are detected. 27 | -------------------------------------------------------------------------------- /11. scan-dependencies-in-gitlab-ci/requirements.txt: -------------------------------------------------------------------------------- 1 | # package without vulnerabilites 2 | # requests 3 | 4 | # package with vulnerabilites 5 | requests==2.19.1 6 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | This repo provides a detailed, step-by-step guide for setting up a custom GitLab server and GitLab runner using Docker containers. 2 | 3 | #### Features: 4 | * Step-by-Step Instructions: Clear and concise steps for setting up GitLab server and runner. 5 | * Docker Containers: Simplified setup using Docker for easy management and scalability. 6 | * Practical Examples: Real-world scenarios and configurations to get you up and running quickly. 7 | 8 | #### Topics Covered: 9 | * Dockerized GitLab: How to Easily Set Up Your Own GitLab Server 10 | * GitLab Setup Using Docker Compose: A Beginner’s Guide 11 | * Dockerized GitLab CI: Setting Up and Connecting Your GitLab Runner 12 | * Dockerized GitLab CI: Register Docker Executor as a GitLab Runner 13 | * Auto register docker executor with GitLab to run pipelines 14 | * Dockerized GitLab CI: Docker Executor as a Docker in Docker (docker:dind) 15 | * Dockerized GitLab CI: Build Docker Images using Google Kaniko 16 | * Setup GitLab Container Registry  -  GitLab Managed 17 | * Automating Security in GitLab CI: Set Up Container Scanning 18 | * Automating Security in GitLab CI: Set Up Dependency Scanning 19 | * Dockerize Next.JS app for Local development and Production 20 | * Automate Next.js Deployment to AWS EC2 with GitLab CI 21 | 22 | #### How to Use: 23 | * Clone the Repository: `git clone github.com/BuildWithLal/gitlab-in-docker.git` 24 | * Navigate to Topics: Each folder contains step-by-step guides and a README with the YouTube link. 25 | * Follow Along: Use the guides and watch the videos to set up and configure your GitLab server and runner. 26 | 27 | --------------------------------------------------------------------------------