├── .gitignore ├── .vscode └── settings.json ├── README.md ├── Vagrantfile ├── _include_gitlab_api.sh ├── configure-hyperv-guest.sh ├── configure-hyperv-vm.ps1 ├── create-example-jenkins-to-gitlab-configuration.sh ├── create-example-repositories.sh ├── create-example-users.sh ├── gitlab.rb-active-directory-ldap.patch ├── gitlab.rb-nginx-status.patch ├── provision-certificates.sh ├── provision-dns-server.sh ├── provision-examples.sh ├── provision-gitlab-cli.sh ├── provision-mailpit.sh ├── provision-openbao.sh ├── provision-resize-disk.sh ├── provision.sh ├── renovate.json5 ├── renovate.sh ├── set-example-groups-users.py └── summary.sh /.gitignore: -------------------------------------------------------------------------------- 1 | .vagrant/ 2 | tmp/ 3 | secrets* 4 | -------------------------------------------------------------------------------- /.vscode/settings.json: -------------------------------------------------------------------------------- 1 | { 2 | "cSpell.words": [ 3 | "openbao", 4 | "pwsh" 5 | ] 6 | } -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Environment 2 | 3 | This [vagrant](https://www.vagrantup.com/) environment configures a basic [GitLab Community Edition](https://gitlab.com/gitlab-org/gitlab-foss) installation using the [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab) package. 4 | 5 | After launching this environment, you can test GitLab CI by launching the [rgl/gitlab-ci-vagrant](https://github.com/rgl/gitlab-ci-vagrant) environment. 6 | 7 | [Nginx](http://nginx.org/en/) ([HTTP/2](https://en.wikipedia.org/wiki/HTTP/2) enabled) is configured with a self-signed certificate at: 8 | 9 | > https://gitlab.example.com/ 10 | 11 | [PostgreSQL](http://www.postgresql.org/) is configured to allow (and trust) any connection from the host. For example, you can use [pgAdmin III](http://www.pgadmin.org/) with these settings: 12 | 13 | Host: gitlab.example.com 14 | Port: 5432 15 | Maintenance DB: postgres 16 | Username: gitlab-psql 17 | 18 | GitLab is also configured to use the optional `ldaps://dc.example.com` Active Directory LDAP endpoint as configured by [rgl/windows-domain-controller-vagrant](https://github.com/rgl/windows-domain-controller-vagrant). 19 | 20 | Some example repositories are automatically installed, if you do not want that, comment the line that calls [`create-example-repositories.sh`](create-example-repositories.sh) inside the [`provision.sh` file](provision.sh) before running `vagrant up`. 21 | 22 | Email notifications are sent to a local [mailpit](https://github.com/axllent/mailpit) SMTP server running at localhost:1025 and you can browse them at [http://gitlab.example.com:8025](http://gitlab.example.com:8025). 23 | 24 | Prometheus is available at http://gitlab.example.com:9090/. 25 | 26 | 27 | # Usage 28 | 29 | Install the [Ubuntu 22.04 UEFI Base Box](https://github.com/rgl/ubuntu-vagrant). 30 | 31 | Start the environment: 32 | 33 | ```bash 34 | vagrant up --no-destroy-on-error 35 | ``` 36 | 37 | Configure your host system to resolve the `gitlab.example.com` and `vault.gitlab.example.com` domains to this vagrant environment IP address, e.g.: 38 | 39 | ```bash 40 | sudo tee -a /etc/hosts <<'EOF' 41 | 10.10.9.99 gitlab.example.com 42 | 10.10.9.99 vault.gitlab.example.com 43 | EOF 44 | ``` 45 | 46 | Sign In into GitLab using the `root` username and the `HeyH0Password` password at: 47 | 48 | > https://gitlab.example.com/users/sign_in 49 | 50 | When using the default LDAP settings you can also login with LDAP credentials as the following users: 51 | 52 | | Username | Password | 53 | |---------------|-----------------| 54 | | `john.doe` | `HeyH0Password` | 55 | | `jane.doe` | `HeyH0Password` | 56 | 57 | After login, you should add your [public SSH key](https://git-scm.com/book/en/v2/Git-on-the-Server-Generating-Your-SSH-Public-Key), for that open the SSH Keys page at: 58 | 59 | > https://gitlab.example.com/-/user_settings/ssh_keys 60 | 61 | Add a new SSH key with your SSH public key, for that, just copy the contents of 62 | your `id_rsa.pub` file. Get its contents with, e.g.: 63 | 64 | ```bash 65 | cat ~/.ssh/id_rsa.pub 66 | ``` 67 | 68 | Create a new repository named `hello` at: 69 | 70 | > https://gitlab.example.com/projects/new 71 | 72 | You can now clone that repository with SSH or HTTPS: 73 | 74 | ```bash 75 | git clone git@gitlab.example.com:root/hello.git 76 | git clone https://root@gitlab.example.com/root/hello.git 77 | ``` 78 | 79 | **NB** This vagrant environment certificates are signed by a private CA, as 80 | such, to prevent TLS verification errors, you must trust it by importing its 81 | certificate from the `tmp/gitlab-ca/gitlab-ca-crt.pem` file, or, not 82 | recommended, temporarily, disable all the certificate verifications, by 83 | setting the [`GIT_SSL_NO_VERIFY` environment variable](https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables) 84 | with `export GIT_SSL_NO_VERIFY=true`. 85 | 86 | Make some changes to the cloned repository and push them: 87 | 88 | ```bash 89 | cd hello 90 | echo '# Hello World' >> README.md 91 | git add README.md 92 | git commit -m 'some change' 93 | git push 94 | ``` 95 | 96 | List this repository dependencies (and which have newer versions): 97 | 98 | ```bash 99 | GITHUB_COM_TOKEN='YOUR_GITHUB_PERSONAL_TOKEN' ./renovate.sh 100 | ``` 101 | 102 | 103 | ## Hyper-V 104 | 105 | Create the required virtual switches: 106 | 107 | ```bash 108 | pwsh -NoLogo -NoProfile -ExecutionPolicy Bypass <<'EOF' 109 | @( 110 | @{Name='gitlab'; IpAddress='10.10.9.1'} 111 | ) | ForEach-Object { 112 | $switchName = $_.Name 113 | $switchIpAddress = $_.IpAddress 114 | $networkAdapterName = "vEthernet ($switchName)" 115 | $networkAdapterIpAddress = $switchIpAddress 116 | $networkAdapterIpPrefixLength = 24 117 | 118 | # create the vSwitch. 119 | New-VMSwitch -Name $switchName -SwitchType Internal | Out-Null 120 | 121 | # assign it an host IP address. 122 | $networkAdapter = Get-NetAdapter $networkAdapterName 123 | $networkAdapter | New-NetIPAddress ` 124 | -IPAddress $networkAdapterIpAddress ` 125 | -PrefixLength $networkAdapterIpPrefixLength ` 126 | | Out-Null 127 | } 128 | 129 | # remove all virtual switches from the windows firewall. 130 | Set-NetFirewallProfile ` 131 | -DisabledInterfaceAliases ( 132 | Get-NetAdapter -name "vEthernet*" | Where-Object {$_.ifIndex} 133 | ).InterfaceAlias 134 | EOF 135 | ``` 136 | 137 | 138 | # Git Large File Storage (LFS) 139 | 140 | You can also use Git Large File Storage (LFS). As this is an external Git plugin, 141 | you need to [install git-lfs](https://git-lfs.github.com/) before you continue. 142 | 143 | **NB** `git-lfs` needs to be on your `PATH`. Normally the installer configures 144 | your system `PATH`, but **you still need to restart your shell or Git Client 145 | application** for it to pick it up. 146 | 147 | Give it a try by cloning the example repository (created by 148 | [create-example-repositories.sh](create-example-repositories.sh)): 149 | 150 | ```bash 151 | git clone https://root:HeyH0Password@gitlab.example.com/example/use-git-lfs.git 152 | ``` 153 | 154 | **NB** `git-lfs` always uses an `https` endpoint (even when you clone with `ssh`). 155 | 156 | Lets get familiar with `git-lfs` by running some commands. 157 | 158 | See the available lfs commands: 159 | 160 | ```bash 161 | git lfs 162 | ``` 163 | 164 | Which file patterns are currently being tracked: 165 | 166 | ```bash 167 | git lfs track 168 | ``` 169 | 170 | **NB** do not forget, only the tracked files are put outside the git repository. So don't forget to 171 | track. e.g., with `git lfs track "*.iso"`. 172 | 173 | See which files are actually tracked: 174 | 175 | ```bash 176 | git lfs ls-files 177 | ``` 178 | 179 | See the `git-lfs` environment: 180 | 181 | ```bash 182 | git lfs env 183 | ``` 184 | 185 | For more information [read the tutorial](https://github.com/github/git-lfs/wiki/Tutorial) 186 | and [the documentation](https://git-lfs.github.com/). 187 | 188 | 189 | # Troubleshoot 190 | 191 | Watch the logs: 192 | 193 | ```bash 194 | sudo su -l 195 | tail -f /var/log/gitlab/gitlab-rails/*.log 196 | ``` 197 | 198 | Do a self-check: 199 | 200 | ```bash 201 | sudo gitlab-rake --trace gitlab:env:info 202 | sudo gitlab-rake --trace gitlab:check SANITIZE=true 203 | ``` 204 | 205 | 206 | # Monitorization 207 | 208 | By default Prometheus is configured to scrap the metric targets every 15 seconds and to store them for 15 days. 209 | 210 | You can see the current targets at: 211 | 212 | http://gitlab.example.com:9090/targets 213 | 214 | **WARNING** prometheus is configured to listen at `0.0.0.0`, you probably want to change this. 215 | 216 | 217 | # Command Line Interface 218 | 219 | GitLab has an [API](https://docs.gitlab.com/api/) which can be used from different applications, one of those, is the [`gitlab` cli application](https://python-gitlab.readthedocs.io/en/stable/cli-usage.html), which is already installed in the vagrant environment (see [provision-gitlab-cli.sh](provision-gitlab-cli.sh)) and can be used as: 220 | 221 | ```bash 222 | vagrant ssh 223 | sudo su -l 224 | 225 | # list all users. 226 | gitlab -o yaml -f id,name,email user list --get-all 227 | 228 | # list all groups and projects. 229 | gitlab -o yaml -f id,visibility,full_path,web_url group list --get-all 230 | gitlab -o yaml -f id,visibility,tag_list,path_with_namespace,web_url project list --get-all 231 | 232 | # list all the projects protected branches, tags, members. 233 | gitlab -o json -f id,visibility,tag_list,web_url project list --get-all >projects.json 234 | jq '.[].id' projects.json | xargs -L1 gitlab project-protected-branch list --get-all --project-id 235 | jq '.[].id' projects.json | xargs -L1 gitlab project-protected-tag list --get-all --project-id 236 | jq '.[].id' projects.json | xargs -L1 gitlab project-member list --get-all --project-id 237 | ``` 238 | 239 | # Python Interface 240 | 241 | [python-gitlab](https://github.com/python-gitlab/python-gitlab) is also available as the `gitlab` python library, which can be used as: 242 | 243 | ```python 244 | import gitlab 245 | 246 | gl = gitlab.Gitlab.from_config() 247 | 248 | # list all users. 249 | for user in gl.users.list(all=True): 250 | print(f'{user.id}\t{user.name}\t{user.email}') 251 | 252 | # list all groups and projects. 253 | for group in gl.groups.list(all=True): 254 | print(f'{group.id}\t{group.visibility}\t{group.full_path}\t{group.web_url}') 255 | for project in gl.projects.list(all=True): 256 | print(f'{project.id}\t{project.visibility}\t{project.tag_list}\t{project.path_with_namespace}\t{project.web_url}') 257 | 258 | # list project protected branches. 259 | for project in gl.projects.list(all=True): 260 | has_i = False 261 | for i in project.protectedbranches.list(all=True): 262 | print(f'{project.web_url}\t{i.name}') 263 | has_i = True 264 | if not has_i: 265 | print(project.web_url) 266 | 267 | # list project members. 268 | # NB these members do not include the ones added to the group. 269 | for project in gl.projects.list(all=True): 270 | has_member = False 271 | for member in project.members.list(all=True): 272 | # NB the member object does not contain the email attribute, so we also fetch the user. 273 | user = gl.users.get(id=member.id) 274 | print(f'{project.web_url}\t{user.username}\t{user.email}') 275 | has_member = True 276 | if not has_member: 277 | print(project.web_url) 278 | 279 | # see more examples at https://python-gitlab.readthedocs.io/en/stable/api-objects.html 280 | ``` 281 | 282 | Also check the [set-example-groups-users.py](set-example-groups-users.py) script to see how you could add users to all groups. 283 | -------------------------------------------------------------------------------- /Vagrantfile: -------------------------------------------------------------------------------- 1 | # to be able to configure the hyper-v vm. 2 | ENV['VAGRANT_EXPERIMENTAL'] = 'typed_triggers' 3 | 4 | # NB execute apt-cache madison gitlab-ce to list the available versions. 5 | # see https://gitlab.com/gitlab-org/gitlab-foss/-/tags 6 | # renovate: datasource=gitlab-tags depName=gitlab-org/gitlab-foss 7 | config_gitlab_version = '18.0.1' 8 | CONFIG_GITLAB_VERSION = "#{config_gitlab_version}-ce.0" 9 | CONFIG_DISK_SIZE_GB = 32 10 | 11 | CONFIG_GITLAB_FQDN = 'gitlab.example.com' 12 | CONFIG_GITLAB_IP = '10.10.9.99' 13 | CONFIG_VAULT_FQDN = "vault.#{CONFIG_GITLAB_FQDN}" 14 | CONFIG_VAULT_IP = '10.10.9.99' 15 | CONFIG_UBUNTU_FQDN = "ubuntu.#{CONFIG_GITLAB_FQDN}" 16 | CONFIG_UBUNTU_IP = '10.10.9.98' 17 | CONFIG_INCUS_FQDN = "incus.#{CONFIG_GITLAB_FQDN}" 18 | CONFIG_INCUS_IP = '10.10.9.97' 19 | CONFIG_LXD_FQDN = "lxd.#{CONFIG_GITLAB_FQDN}" 20 | CONFIG_LXD_IP = '10.10.9.96' 21 | CONFIG_WINDOWS_FQDN = "windows.#{CONFIG_GITLAB_FQDN}" 22 | CONFIG_WINDOWS_IP = '10.10.9.95' 23 | 24 | Vagrant.configure(2) do |config| 25 | config.vm.box = "ubuntu-22.04-uefi-amd64" 26 | 27 | config.vm.hostname = CONFIG_GITLAB_FQDN 28 | 29 | config.vm.network "private_network", ip: CONFIG_GITLAB_IP, libvirt__forward_mode: 'route', libvirt__dhcp_enabled: false, hyperv__bridge: 'gitlab' 30 | 31 | config.vm.provider 'libvirt' do |lv, config| 32 | lv.memory = 6*1024 33 | lv.cpus = 4 34 | lv.cpu_mode = 'host-passthrough' 35 | lv.keymap = 'pt' 36 | lv.machine_virtual_size = CONFIG_DISK_SIZE_GB 37 | config.vm.synced_folder '.', '/vagrant', type: 'nfs', nfs_version: '4.2', nfs_udp: false 38 | end 39 | 40 | config.vm.provider 'hyperv' do |hv, config| 41 | hv.linked_clone = true 42 | hv.memory = 6*1024 43 | hv.cpus = 4 44 | hv.enable_virtualization_extensions = false # nested virtualization. 45 | hv.vlan_id = ENV['HYPERV_VLAN_ID'] 46 | # see https://github.com/hashicorp/vagrant/issues/7915 47 | # see https://github.com/hashicorp/vagrant/blob/10faa599e7c10541f8b7acf2f8a23727d4d44b6e/plugins/providers/hyperv/action/configure.rb#L21-L35 48 | config.vm.network :private_network, bridge: ENV['HYPERV_SWITCH_NAME'] if ENV['HYPERV_SWITCH_NAME'] 49 | config.vm.synced_folder '.', '/vagrant', 50 | type: 'smb', 51 | smb_username: ENV['VAGRANT_SMB_USERNAME'] || ENV['USER'], 52 | smb_password: ENV['VAGRANT_SMB_PASSWORD'] 53 | # further configure the VM (e.g. manage the network adapters). 54 | config.trigger.before :'VagrantPlugins::HyperV::Action::StartInstance', type: :action do |trigger| 55 | trigger.ruby do |env, machine| 56 | # see https://github.com/hashicorp/vagrant/blob/v2.2.10/lib/vagrant/machine.rb#L13 57 | # see https://github.com/hashicorp/vagrant/blob/v2.2.10/plugins/kernel_v2/config/vm.rb#L716 58 | bridges = machine.config.vm.networks.select{|type, options| type == :private_network && options.key?(:hyperv__bridge)}.map do |type, options| 59 | mac_address_spoofing = false 60 | mac_address_spoofing = options[:hyperv__mac_address_spoofing] if options.key?(:hyperv__mac_address_spoofing) 61 | [options[:hyperv__bridge], mac_address_spoofing] 62 | end 63 | system( 64 | 'pwsh', 65 | '-NoLogo', 66 | '-NoProfile', 67 | '-ExecutionPolicy', 68 | 'Bypass', 69 | '-File', 70 | 'configure-hyperv-vm.ps1', 71 | machine.id, 72 | bridges.to_json 73 | ) or raise "failed to configure hyper-v vm with exit code #{$?.exitstatus}" 74 | end 75 | end 76 | end 77 | 78 | config.trigger.before :up do |trigger| 79 | ldap_ca_cert_path = '../windows-domain-controller-vagrant/tmp/ExampleEnterpriseRootCA.der' 80 | trigger.run = {inline: "sh -c 'mkdir -p tmp && cp #{ldap_ca_cert_path} tmp'"} if File.file? ldap_ca_cert_path 81 | end 82 | 83 | config.vm.provision "shell", inline: "apt-get update" 84 | config.vm.provision "shell", path: "provision-resize-disk.sh" 85 | config.vm.provision "shell", path: "configure-hyperv-guest.sh", args: [CONFIG_GITLAB_IP] 86 | config.vm.provision "shell", path: "provision-dns-server.sh", args: [ 87 | CONFIG_GITLAB_IP, 88 | CONFIG_VAULT_FQDN, 89 | CONFIG_VAULT_IP, 90 | CONFIG_UBUNTU_FQDN, 91 | CONFIG_UBUNTU_IP, 92 | CONFIG_INCUS_FQDN, 93 | CONFIG_INCUS_IP, 94 | CONFIG_LXD_FQDN, 95 | CONFIG_LXD_IP, 96 | CONFIG_WINDOWS_FQDN, 97 | CONFIG_WINDOWS_IP] 98 | config.vm.provision "shell", path: "provision-certificates.sh" 99 | config.vm.provision "shell", path: "provision-mailpit.sh" 100 | config.vm.provision "shell", path: "provision.sh", args: [CONFIG_GITLAB_VERSION] 101 | config.vm.provision "shell", path: "provision-openbao.sh", args: [CONFIG_GITLAB_FQDN, CONFIG_VAULT_FQDN] 102 | config.vm.provision "shell", path: "provision-gitlab-cli.sh" 103 | config.vm.provision "shell", path: "provision-examples.sh" 104 | config.vm.provision "shell", path: "summary.sh" 105 | end 106 | -------------------------------------------------------------------------------- /_include_gitlab_api.sh: -------------------------------------------------------------------------------- 1 | domain=$(hostname --fqdn) 2 | 3 | # e.g. gitlab-psql -t -c "select id,name from personal_access_tokens where user_id=1 and name='vagrant'" 4 | function gitlab-psql { 5 | sudo -sHu gitlab-psql \ 6 | /opt/gitlab/embedded/bin/psql \ 7 | -h /var/opt/gitlab/postgresql \ 8 | -d gitlabhq_production \ 9 | "$@" 10 | } 11 | 12 | gitlab_private_token=$(cat /vagrant/tmp/gitlab-root-personal-access-token.txt) 13 | 14 | # see https://docs.gitlab.com/api/rest/ 15 | function gitlab-api { 16 | local method=$1; shift 17 | local path=$1; shift 18 | http \ 19 | --ignore-stdin \ 20 | $method \ 21 | "https://$domain/api/v4$path" \ 22 | "Private-Token:$gitlab_private_token" \ 23 | "$@" 24 | } 25 | 26 | function gitlab-wait-for-ready { 27 | set +x 28 | echo 'Waiting for GitLab to be ready...' 29 | while true; do 30 | body=$( 31 | http \ 32 | --ignore-stdin \ 33 | GET \ 34 | "https://$domain/api/v4/version" \ 35 | "Private-Token:$gitlab_private_token") 36 | if jq -e . >/dev/null 2>&1 <<<"$body"; then 37 | version=$(jq -r .version <<<"$body") 38 | if [[ -n "$version" ]]; then 39 | echo "GitLab $version is ready!" 40 | break 41 | fi 42 | fi 43 | sleep 5 44 | done 45 | set -x 46 | } 47 | 48 | function gitlab-create-group { 49 | local name=$1 50 | 51 | local group="$(gitlab-api GET "/groups/$name?with_projects=false" --check-status)" 52 | local id="$(echo "$group" | jq -r .id)" 53 | 54 | [ "$id" != 'null' ] && echo "$group" || gitlab-api POST /groups name=$name path=$name visibility=private --check-status 55 | } 56 | 57 | # see https://docs.gitlab.com/api/groups/#list-groups 58 | function gitlab-get-groups { 59 | gitlab-api GET /groups --check-status 60 | } 61 | 62 | # see https://docs.gitlab.com/api/members/#add-a-member-to-a-group-or-project 63 | function gitlab-add-user-to-all-groups { 64 | local user_id=$1 65 | local access_level=$2 66 | 67 | for id in $(gitlab-get-groups | jq '.[].id'); do 68 | gitlab-api POST "/groups/$id/members" \ 69 | "user_id=$user_id" \ 70 | "access_level=$access_level" \ 71 | --check-status 72 | done 73 | } 74 | 75 | function gitlab-create-project { 76 | local name="$1" 77 | local namespaceId="$2" 78 | 79 | # NB we need to retry this call because sometimes it fails with: 80 | # HTTP 502 Bad Gateway 81 | # GitLab is not responding 82 | set +x 83 | while true; do 84 | body="$(gitlab-api POST /projects "name=$name" "namespace_id=$namespaceId" visibility=private)" 85 | if jq -e . >/dev/null 2>&1 <<<"$body"; then 86 | id=$(jq -r .id <<<"$body") 87 | if [[ -n "$id" ]]; then 88 | echo "$body" 89 | break 90 | fi 91 | fi 92 | sleep 5 93 | done 94 | set -x 95 | } 96 | 97 | # creates a new GitLab project from an existing git repository. 98 | # NB GitLab CE does not support mirroring a git repository. 99 | function gitlab-create-project-and-import { 100 | local sourceGitUrl=$1 101 | local destinationProjectName=$2 102 | local destinationNamespaceId=$3 103 | local destinationNamespaceFullPath="$(gitlab-api GET /namespaces/$destinationNamespaceId --check-status | jq -r .full_path)" 104 | 105 | gitlab-create-project $destinationProjectName $destinationNamespaceId 106 | 107 | git \ 108 | clone --mirror \ 109 | $sourceGitUrl \ 110 | $destinationProjectName 111 | 112 | pushd $destinationProjectName 113 | git \ 114 | push --mirror \ 115 | git@$domain:$destinationNamespaceFullPath/$destinationProjectName.git 116 | popd 117 | 118 | rm -rf $destinationProjectName 119 | } 120 | 121 | # see https://docs.gitlab.com/api/users/#create-a-user 122 | function gitlab-create-user { 123 | local username=$1 124 | local name=$2 125 | local email=$3 126 | local password=$4 127 | 128 | gitlab-api POST /users \ 129 | "username=$username" \ 130 | "name=$name" \ 131 | "email=$email" \ 132 | "password=$password" \ 133 | skip_confirmation=true \ 134 | --check-status 135 | } 136 | 137 | # see https://docs.gitlab.com/api/users/#as-a-regular-user 138 | function gitlab-get-user { 139 | local username=$1 140 | 141 | gitlab-api GET /users \ 142 | "username=$username" \ 143 | --check-status 144 | } 145 | 146 | # see https://docs.gitlab.com/api/user_tokens/#create-an-impersonation-token 147 | function gitlab-create-user-impersonation-token { 148 | local user_id="$1" 149 | local name="$2" 150 | local scopes="$3" 151 | 152 | # NB we need to retry this call because sometimes it fails with: 153 | # HTTP 502 Bad Gateway 154 | # GitLab is not responding 155 | set +x 156 | while true; do 157 | body="$( 158 | gitlab-api POST "/users/$user_id/impersonation_tokens" \ 159 | "user_id=$username" \ 160 | "name=$name" \ 161 | "scopes:=$scopes" 162 | )" 163 | if jq -e . >/dev/null 2>&1 <<<"$body"; then 164 | token=$(jq -r .token <<<"$body") 165 | if [[ -n "$token" ]]; then 166 | echo "$body" 167 | break 168 | fi 169 | fi 170 | sleep 5 171 | done 172 | set -x 173 | } 174 | 175 | # trust our own SSH server. 176 | if [ -z "$(ssh-keygen -F $domain 2>/dev/null)" ]; then 177 | install -d -m 700 ~/.ssh 178 | ssh-keyscan -H $domain >> ~/.ssh/known_hosts 179 | fi 180 | 181 | # generate a new ssh key for the current user account and add it to gitlab. 182 | # NB the public keys should end-up in /var/opt/gitlab/.ssh/authorized_keys. 183 | # NB that happens asynchronously via a sidekick job, as such, it might 184 | # not be immediately visible. 185 | # see https://docs.gitlab.com/user/ssh/ 186 | # see https://docs.gitlab.com/administration/raketasks/maintenance/#rebuild-authorized_keys-file 187 | # TODO consider switching to https://docs.gitlab.com/administration/operations/fast_ssh_key_lookup/ 188 | if [ ! -f ~/.ssh/id_rsa ]; then 189 | ssh-keygen -f ~/.ssh/id_rsa -t rsa -b 2048 -C "$USER@$domain" -N '' 190 | gitlab-api POST /user/keys "title=$USER@$domain" key=@~/.ssh/id_rsa.pub --check-status 191 | 192 | # force the update of the authorized_keys to workaround the 193 | # bug introduced in 12.9.0. 194 | # NB this also happens when adding the key in the UI. 195 | # see https://gitlab.com/gitlab-org/gitlab/-/issues/212297 196 | yes yes | gitlab-rake gitlab:shell:setup 197 | 198 | # wait for the key to be asynchronously added to the authorized_keys file. 199 | # NB this just checks whether the file length is above 500B and 200 | # assumes the ssh was successfully added, which is enough for 201 | # this use-case. 202 | while [ "$(stat --printf="%s" /var/opt/gitlab/.ssh/authorized_keys)" -le 500 ]; do 203 | sleep 5 204 | done 205 | 206 | # test the ssh connection. 207 | # NB this will eventually show "Welcome to GitLab, @root!" 208 | ssh -o BatchMode=yes -Tv git@$domain /etc/network/interfaces <>/etc/network/interfaces </etc/netplan/02-hyperv.yaml <>/etc/netplan/02-hyperv.yaml < Developer 14 | 15 | # create a impersonation token. 16 | token_json="$(gitlab-create-user-impersonation-token "$user_id" 'jenkins' '["api"]')" 17 | echo "$token_json" | jq -j '.token' >/vagrant/tmp/gitlab-jenkins-impersonation-token.txt 18 | 19 | # allow requests to the local network from hooks and services. 20 | gitlab-api PUT /application/settings allow_local_requests_from_hooks_and_services:=true --check-status 21 | -------------------------------------------------------------------------------- /create-example-repositories.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -euxo pipefail 3 | 4 | source /vagrant/_include_gitlab_api.sh 5 | 6 | VAULT_ADDR="https://vault.$domain:8200" 7 | 8 | # create the example group. 9 | example_group_name='example' 10 | example_group_id="$(gitlab-create-group $example_group_name | jq -r .id)" 11 | 12 | # import some existing git repositories. 13 | gitlab-create-project-and-import https://github.com/rgl/gitlab-ci-validate-jwt.git gitlab-ci-validate-jwt $example_group_id 14 | gitlab-create-project-and-import https://github.com/rgl/gitlab-vagrant.git gitlab-vagrant $example_group_id 15 | gitlab-create-project-and-import https://github.com/rgl/ubuntu-vagrant.git ubuntu-vagrant $example_group_id 16 | gitlab-create-project-and-import https://github.com/rgl/MailBounceDetector.git MailBounceDetector $example_group_id 17 | gitlab-create-project-and-import https://github.com/rgl/hello-puppeteer-windows-container.git hello-puppeteer-windows-container $example_group_id 18 | gitlab-create-project-and-import https://github.com/rgl/ubuntu-docker-compose-example.git ubuntu-docker-compose-example $example_group_id 19 | gitlab-create-project-and-import https://github.com/rgl/windows-docker-compose-example.git windows-docker-compose-example $example_group_id 20 | 21 | # configure the git client. 22 | git config --global user.name "Root Doe" 23 | git config --global user.email root@$domain 24 | git config --global push.default simple 25 | 26 | # install git-lfs. 27 | # see https://docs.gitlab.com/administration/lfs/ 28 | # see https://docs.gitlab.com/topics/git/lfs/ 29 | # see https://github.com/github/git-lfs/wiki/Installation 30 | curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash 31 | sudo apt-get install -y git-lfs 32 | 33 | # create a new repository with git-lfs support. 34 | # 35 | # NB when you push to this GitLab server, the git repository will be stored 36 | # somewhere inside: 37 | # /var/opt/gitlab/git-data/repositories/ 38 | # and the lfs objects inside: 39 | # /var/opt/gitlab/gitlab-rails/shared/lfs-objects/ 40 | pushd /tmp 41 | gitlab-create-project use-git-lfs $example_group_id 42 | git clone https://root:HeyH0Password@$domain/$example_group_name/use-git-lfs.git use-git-lfs && cd use-git-lfs 43 | git lfs install 44 | git lfs track '*.md' 45 | echo 'This file is in lfs' >in-lfs.md 46 | echo 'This file is in git repo db' >not-in-lfs.txt 47 | git add .gitattributes # NB git lfs uses this file to track the lfs file patterns. 48 | git add * 49 | git commit -m 'init' 50 | git push 51 | popd 52 | 53 | # create an example repository to show how to use openbao vault secrets. 54 | # see https://docs.gitlab.com/ci/secrets/id_token_authentication/ 55 | # see https://docs.gitlab.com/ci/secrets/hashicorp_vault/ 56 | # see https://gitlab.example.com/oauth/discovery/keys 57 | # see https://gitlab.example.com/.well-known/openid-configuration 58 | # see https://github.com/rgl/gitlab-ci-validate-jwt 59 | pushd /tmp 60 | gitlab-create-project example-gitlab-vault-secrets $example_group_id 61 | git clone https://root:HeyH0Password@$domain/$example_group_name/example-gitlab-vault-secrets.git example-gitlab-vault-secrets && cd example-gitlab-vault-secrets 62 | # add the .gitlab-ci.yml file. 63 | cat >.gitlab-ci.yml <<'EOF' 64 | test: 65 | tags: 66 | - ubuntu 67 | - shell 68 | variables: 69 | VAULT_ADDR: @@VAULT_ADDR@@ 70 | id_tokens: 71 | VAULT_ID_TOKEN: 72 | aud: $VAULT_ADDR 73 | script: 74 | - | 75 | set -euo pipefail 76 | if [ -z "$VAULT_ID_TOKEN" ]; then 77 | echo "ERROR: The VAULT_ID_TOKEN environment variable MUST NOT be empty." 78 | exit 1 79 | fi 80 | echo "Exchanging the GitLab CI ID Token with the OpenBao Vault Token..." 81 | export VAULT_TOKEN="$( 82 | echo -n "$VAULT_ID_TOKEN" \ 83 | | docker run \ 84 | --rm \ 85 | --env VAULT_ADDR \ 86 | --interactive \ 87 | --volume /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:ro \ 88 | ghcr.io/openbao/openbao:@@OPENBAO_VERSION@@ \ 89 | bao write \ 90 | -field=token \ 91 | auth/jwt/login \ 92 | role=example-gitlab-vault-secrets-main \ 93 | jwt=-)" 94 | if [ -z "$VAULT_TOKEN" ]; then 95 | echo "ERROR: The VAULT_TOKEN environment variable MUST NOT be empty." 96 | exit 1 97 | fi 98 | echo "Getting the example-gitlab-vault-secrets/main/user secret from OpenBao Vault..." 99 | docker run \ 100 | --rm \ 101 | --env VAULT_ADDR \ 102 | --env VAULT_TOKEN \ 103 | --volume /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:ro \ 104 | ghcr.io/openbao/openbao:@@OPENBAO_VERSION@@ \ 105 | bao kv get \ 106 | -format=json \ 107 | secret/example-gitlab-vault-secrets/main/user 108 | EOF 109 | # renovate: datasource=github-releases depName=openbao/openbao 110 | openbao_version='2.2.2' 111 | sed -i -E \ 112 | -e "s,@@OPENBAO_VERSION@@,$openbao_version,g" \ 113 | -e "s,@@VAULT_ADDR@@,$VAULT_ADDR,g" \ 114 | .gitlab-ci.yml 115 | git add * .gitlab-ci.yml 116 | git commit -m 'init' 117 | git push 118 | popd 119 | # configure the example-gitlab-vault-secrets project vault secrets. 120 | echo -n 'opensesame' \ 121 | | bao kv put secret/example-gitlab-vault-secrets/main/user \ 122 | username=alibaba \ 123 | password=- 124 | bao policy write example-gitlab-vault-secrets-main - <<'EOF' 125 | path "secret/data/example-gitlab-vault-secrets/main/*" { 126 | capabilities = ["read"] 127 | } 128 | EOF 129 | bao write auth/jwt/role/example-gitlab-vault-secrets-main - <cr-eol-terminators.md 151 | # add a file with LF eol terminators to see whether they are preserved. 152 | printf '1. This line ends with line feed (LF)\n2. This line ends with line feed (LF)\n3. This line ends with line feed (LF)\n' >lf-eol-terminators.md 153 | # add a file with CRLF eol terminators to see whether they are preserved. 154 | printf '1. This line ends with carriage return and line feed (CRLF)\r\n2. This line ends with carriage return and line feed (CRLF)\r\n3. This line ends with carriage return and line feed (CRLF)\r\n' >crlf-eol-terminators.md 155 | # add a file with mixed eol terminators to see whether they are preserved. 156 | printf '1. This line ends with carriage return (CR)\r2. This line ends with line feed (LF)\n3. This line ends with carriage return and line feed (CRLF)\r\n' >mixed-eol-terminators.md 157 | # add the .gitlab-ci.yml file. 158 | cat >.gitlab-ci.yml <<'EOF' 159 | default: 160 | before_script: 161 | # show the shell executable. 162 | - echo "$SHELL" 163 | # show the shell command line arguments. 164 | - cat "/proc/$$/cmdline" | tr \\0 \\n 165 | # enable the shell strict mode. 166 | - set -euo pipefail 167 | # show the shell options. 168 | - set -o 169 | info: 170 | tags: 171 | - ubuntu 172 | - shell 173 | script: 174 | - cat /proc/self/cgroup | sort 175 | - cat /etc/os-release 176 | - uname -a 177 | - dpkg-query -W -f='${binary:Package}\n' | sort 178 | - id 179 | - pwd 180 | - env | sort 181 | - mount | sort 182 | - ps axuww 183 | - file *-eol-terminators.md 184 | docker-info: 185 | tags: 186 | - ubuntu 187 | - shell 188 | script: 189 | - docker compose version 190 | - docker info 191 | - docker run --rm hello-world 192 | EOF 193 | git add * .gitlab-ci.yml 194 | git commit -m 'init' 195 | git push 196 | popd 197 | 198 | # create a new repository with a .gitlab-ci.yml file to show 199 | # information about the ubuntu docker gitlab-runner environment. 200 | pushd /tmp 201 | gitlab-create-project gitlab-runner-environment-info-ubuntu-docker $example_group_id 202 | git clone https://root:HeyH0Password@$domain/$example_group_name/gitlab-runner-environment-info-ubuntu-docker.git gitlab-runner-environment-info-ubuntu-docker && cd gitlab-runner-environment-info-ubuntu-docker 203 | # add a file with CR eol terminators to see whether they are preserved. 204 | printf '1. This line ends with carriage return (CR)\r2. This line ends with carriage return (CR)\r3. This line ends with carriage return (CR)\r' >cr-eol-terminators.md 205 | # add a file with LF eol terminators to see whether they are preserved. 206 | printf '1. This line ends with line feed (LF)\n2. This line ends with line feed (LF)\n3. This line ends with line feed (LF)\n' >lf-eol-terminators.md 207 | # add a file with CRLF eol terminators to see whether they are preserved. 208 | printf '1. This line ends with carriage return and line feed (CRLF)\r\n2. This line ends with carriage return and line feed (CRLF)\r\n3. This line ends with carriage return and line feed (CRLF)\r\n' >crlf-eol-terminators.md 209 | # add a file with mixed eol terminators to see whether they are preserved. 210 | printf '1. This line ends with carriage return (CR)\r2. This line ends with line feed (LF)\n3. This line ends with carriage return and line feed (CRLF)\r\n' >mixed-eol-terminators.md 211 | # add the .gitlab-ci.yml file. 212 | cat >.gitlab-ci.yml <<'EOF' 213 | default: 214 | before_script: 215 | # show the shell executable. 216 | - echo "$SHELL" 217 | # show the shell command line arguments. 218 | - cat "/proc/$$/cmdline" | tr \\0 \\n 219 | # enable the shell strict mode. 220 | - set -euo pipefail 221 | # show the shell options. 222 | - set -o 223 | info: 224 | tags: 225 | - ubuntu 226 | - docker 227 | script: 228 | - cat /proc/self/cgroup | sort 229 | - cat /etc/os-release 230 | - uname -a 231 | - dpkg-query -W -f='${binary:Package}\n' | sort 232 | - apt-get update && apt-get install -y file procps 233 | - id 234 | - pwd 235 | - env | sort 236 | - mount | sort 237 | - ps axuww 238 | - file *-eol-terminators.md 239 | EOF 240 | git add * .gitlab-ci.yml 241 | git commit -m 'init' 242 | git push 243 | popd 244 | 245 | # create a new repository with a .gitlab-ci.yml file to show 246 | # information about the ubuntu incus gitlab-runner environment. 247 | pushd /tmp 248 | gitlab-create-project gitlab-runner-environment-info-ubuntu-incus $example_group_id 249 | git clone https://root:HeyH0Password@$domain/$example_group_name/gitlab-runner-environment-info-ubuntu-incus.git gitlab-runner-environment-info-ubuntu-incus && cd gitlab-runner-environment-info-ubuntu-incus 250 | # add a file with CR eol terminators to see whether they are preserved. 251 | printf '1. This line ends with carriage return (CR)\r2. This line ends with carriage return (CR)\r3. This line ends with carriage return (CR)\r' >cr-eol-terminators.md 252 | # add a file with LF eol terminators to see whether they are preserved. 253 | printf '1. This line ends with line feed (LF)\n2. This line ends with line feed (LF)\n3. This line ends with line feed (LF)\n' >lf-eol-terminators.md 254 | # add a file with CRLF eol terminators to see whether they are preserved. 255 | printf '1. This line ends with carriage return and line feed (CRLF)\r\n2. This line ends with carriage return and line feed (CRLF)\r\n3. This line ends with carriage return and line feed (CRLF)\r\n' >crlf-eol-terminators.md 256 | # add a file with mixed eol terminators to see whether they are preserved. 257 | printf '1. This line ends with carriage return (CR)\r2. This line ends with line feed (LF)\n3. This line ends with carriage return and line feed (CRLF)\r\n' >mixed-eol-terminators.md 258 | # add the .gitlab-ci.yml file. 259 | cat >.gitlab-ci.yml <<'EOF' 260 | default: 261 | before_script: 262 | # show the shell executable. 263 | - echo "$SHELL" 264 | # show the shell command line arguments. 265 | - cat "/proc/$$/cmdline" | tr \\0 \\n 266 | # enable the shell strict mode. 267 | - set -euo pipefail 268 | # show the shell options. 269 | - set -o 270 | info: 271 | tags: 272 | - ubuntu 273 | - incus 274 | script: 275 | - cat /proc/self/cgroup | sort 276 | - cat /etc/os-release 277 | - uname -a 278 | - dpkg-query -W -f='${binary:Package}\n' | sort 279 | - apt-get update && apt-get install -y file procps 280 | - id 281 | - pwd 282 | - env | sort 283 | - mount | sort 284 | - ps axuww 285 | - file *-eol-terminators.md 286 | docker-info: 287 | tags: 288 | - ubuntu 289 | - incus 290 | script: 291 | - docker compose version 292 | - docker info 293 | - docker run --rm hello-world 294 | EOF 295 | git add * .gitlab-ci.yml 296 | git commit -m 'init' 297 | git push 298 | popd 299 | 300 | # create a new repository with a .gitlab-ci.yml file to show 301 | # information about the ubuntu lxd gitlab-runner environment. 302 | pushd /tmp 303 | gitlab-create-project gitlab-runner-environment-info-ubuntu-lxd $example_group_id 304 | git clone https://root:HeyH0Password@$domain/$example_group_name/gitlab-runner-environment-info-ubuntu-lxd.git gitlab-runner-environment-info-ubuntu-lxd && cd gitlab-runner-environment-info-ubuntu-lxd 305 | # add a file with CR eol terminators to see whether they are preserved. 306 | printf '1. This line ends with carriage return (CR)\r2. This line ends with carriage return (CR)\r3. This line ends with carriage return (CR)\r' >cr-eol-terminators.md 307 | # add a file with LF eol terminators to see whether they are preserved. 308 | printf '1. This line ends with line feed (LF)\n2. This line ends with line feed (LF)\n3. This line ends with line feed (LF)\n' >lf-eol-terminators.md 309 | # add a file with CRLF eol terminators to see whether they are preserved. 310 | printf '1. This line ends with carriage return and line feed (CRLF)\r\n2. This line ends with carriage return and line feed (CRLF)\r\n3. This line ends with carriage return and line feed (CRLF)\r\n' >crlf-eol-terminators.md 311 | # add a file with mixed eol terminators to see whether they are preserved. 312 | printf '1. This line ends with carriage return (CR)\r2. This line ends with line feed (LF)\n3. This line ends with carriage return and line feed (CRLF)\r\n' >mixed-eol-terminators.md 313 | # add the .gitlab-ci.yml file. 314 | cat >.gitlab-ci.yml <<'EOF' 315 | default: 316 | before_script: 317 | # show the shell executable. 318 | - echo "$SHELL" 319 | # show the shell command line arguments. 320 | - cat "/proc/$$/cmdline" | tr \\0 \\n 321 | # enable the shell strict mode. 322 | - set -euo pipefail 323 | # show the shell options. 324 | - set -o 325 | info: 326 | tags: 327 | - ubuntu 328 | - lxd 329 | script: 330 | - cat /proc/self/cgroup | sort 331 | - cat /etc/os-release 332 | - uname -a 333 | - dpkg-query -W -f='${binary:Package}\n' | sort 334 | - apt-get update && apt-get install -y file procps 335 | - id 336 | - pwd 337 | - env | sort 338 | - mount | sort 339 | - ps axuww 340 | - file *-eol-terminators.md 341 | docker-info: 342 | tags: 343 | - ubuntu 344 | - lxd 345 | script: 346 | - docker compose version 347 | - docker info 348 | - docker run --rm hello-world 349 | EOF 350 | git add * .gitlab-ci.yml 351 | git commit -m 'init' 352 | git push 353 | popd 354 | 355 | # create a new repository with a .gitlab-ci.yml file to show 356 | # information about the k8s gitlab-runner environment. 357 | pushd /tmp 358 | gitlab-create-project gitlab-runner-environment-info-k8s $example_group_id 359 | git clone https://root:HeyH0Password@$domain/$example_group_name/gitlab-runner-environment-info-k8s.git gitlab-runner-environment-info-k8s && cd gitlab-runner-environment-info-k8s 360 | # add a file with CR eol terminators to see whether they are preserved. 361 | printf '1. This line ends with carriage return (CR)\r2. This line ends with carriage return (CR)\r3. This line ends with carriage return (CR)\r' >cr-eol-terminators.md 362 | # add a file with LF eol terminators to see whether they are preserved. 363 | printf '1. This line ends with line feed (LF)\n2. This line ends with line feed (LF)\n3. This line ends with line feed (LF)\n' >lf-eol-terminators.md 364 | # add a file with CRLF eol terminators to see whether they are preserved. 365 | printf '1. This line ends with carriage return and line feed (CRLF)\r\n2. This line ends with carriage return and line feed (CRLF)\r\n3. This line ends with carriage return and line feed (CRLF)\r\n' >crlf-eol-terminators.md 366 | # add a file with mixed eol terminators to see whether they are preserved. 367 | printf '1. This line ends with carriage return (CR)\r2. This line ends with line feed (LF)\n3. This line ends with carriage return and line feed (CRLF)\r\n' >mixed-eol-terminators.md 368 | # add the .gitlab-ci.yml file. 369 | cat >.gitlab-ci.yml <<'EOF' 370 | default: 371 | before_script: 372 | # show the shell executable. 373 | - echo "$SHELL" 374 | # show the shell command line arguments. 375 | - cat "/proc/$$/cmdline" | tr \\0 \\n 376 | # enable the shell strict mode. 377 | - set -euo pipefail 378 | # show the shell options. 379 | - set -o 380 | info: 381 | tags: 382 | - k8s 383 | script: 384 | - cat /proc/self/cgroup | sort 385 | - cat /etc/os-release 386 | - uname -a 387 | - dpkg-query -W -f='${binary:Package}\n' | sort 388 | - apt-get update && apt-get install -y file procps 389 | - id 390 | - pwd 391 | - env | sort 392 | - mount | sort 393 | - ps axuww 394 | - file *-eol-terminators.md 395 | EOF 396 | git add * .gitlab-ci.yml 397 | git commit -m 'init' 398 | git push 399 | popd 400 | 401 | # create a new repository with a .gitlab-ci.yml file to show 402 | # information about the windows pwsh gitlab-runner environment. 403 | pushd /tmp 404 | gitlab-create-project gitlab-runner-environment-info-windows-pwsh $example_group_id 405 | git clone https://root:HeyH0Password@$domain/$example_group_name/gitlab-runner-environment-info-windows-pwsh.git gitlab-runner-environment-info-windows-pwsh && cd gitlab-runner-environment-info-windows-pwsh 406 | # add a file with CR eol terminators to see whether they are preserved. 407 | printf '1. This line ends with carriage return (CR)\r2. This line ends with carriage return (CR)\r3. This line ends with carriage return (CR)\r' >cr-eol-terminators.md 408 | # add a file with LF eol terminators to see whether they are preserved. 409 | printf '1. This line ends with line feed (LF)\n2. This line ends with line feed (LF)\n3. This line ends with line feed (LF)\n' >lf-eol-terminators.md 410 | # add a file with CRLF eol terminators to see whether they are preserved. 411 | printf '1. This line ends with carriage return and line feed (CRLF)\r\n2. This line ends with carriage return and line feed (CRLF)\r\n3. This line ends with carriage return and line feed (CRLF)\r\n' >crlf-eol-terminators.md 412 | # add a file with mixed eol terminators to see whether they are preserved. 413 | printf '1. This line ends with carriage return (CR)\r2. This line ends with line feed (LF)\n3. This line ends with carriage return and line feed (CRLF)\r\n' >mixed-eol-terminators.md 414 | # add the .gitlab-ci.yml file. 415 | cat >.gitlab-ci.yml <<'EOF' 416 | default: 417 | before_script: 418 | - | 419 | $FormatEnumerationLimit = -1 420 | function Write-Title($title) { 421 | Write-Output "#`n# $title`n#" 422 | } 423 | # wrap the docker command (to make sure this script aborts when it fails). 424 | - | 425 | function docker { 426 | docker.exe @Args | Out-String -Stream -Width ([int]::MaxValue) 427 | if ($LASTEXITCODE) { 428 | throw "$(@('docker')+$Args | ConvertTo-Json -Compress) failed with exit code $LASTEXITCODE" 429 | } 430 | } 431 | info: 432 | tags: 433 | - windows 434 | - pwsh 435 | script: 436 | - | 437 | $FormatEnumerationLimit = -1 438 | function Write-Title($title) { 439 | Write-Output "#`n# $title`n#" 440 | } 441 | - | 442 | Write-Title 'Current user permissions' 443 | whoami.exe /all 444 | - | 445 | Write-Title 'Environment Variables' 446 | dir env: ` 447 | | Sort-Object -Property Name ` 448 | | Format-Table -AutoSize ` 449 | | Out-String -Stream -Width ([int]::MaxValue) ` 450 | | ForEach-Object {$_.TrimEnd()} 451 | - | 452 | Write-Title 'PowerShell Info' 453 | $PSVersionTable 454 | docker-info: 455 | tags: 456 | - windows 457 | - pwsh 458 | script: 459 | - | 460 | Write-Title 'Docker Compose Version' 461 | docker compose version 462 | - | 463 | Write-Title 'Docker Info' 464 | docker info 465 | - | 466 | Write-Title 'Docker hello-world' 467 | docker run --rm hello-world 468 | EOF 469 | git add * .gitlab-ci.yml 470 | git commit -m 'init' 471 | git push 472 | popd 473 | 474 | # create a new repository with a .gitlab-ci.yml file to show 475 | # information about the windows docker gitlab-runner environment. 476 | pushd /tmp 477 | gitlab-create-project gitlab-runner-environment-info-windows-docker $example_group_id 478 | git clone https://root:HeyH0Password@$domain/$example_group_name/gitlab-runner-environment-info-windows-docker.git gitlab-runner-environment-info-windows-docker && cd gitlab-runner-environment-info-windows-docker 479 | # add a file with CR eol terminators to see whether they are preserved. 480 | printf '1. This line ends with carriage return (CR)\r2. This line ends with carriage return (CR)\r3. This line ends with carriage return (CR)\r' >cr-eol-terminators.md 481 | # add a file with LF eol terminators to see whether they are preserved. 482 | printf '1. This line ends with line feed (LF)\n2. This line ends with line feed (LF)\n3. This line ends with line feed (LF)\n' >lf-eol-terminators.md 483 | # add a file with CRLF eol terminators to see whether they are preserved. 484 | printf '1. This line ends with carriage return and line feed (CRLF)\r\n2. This line ends with carriage return and line feed (CRLF)\r\n3. This line ends with carriage return and line feed (CRLF)\r\n' >crlf-eol-terminators.md 485 | # add a file with mixed eol terminators to see whether they are preserved. 486 | printf '1. This line ends with carriage return (CR)\r2. This line ends with line feed (LF)\n3. This line ends with carriage return and line feed (CRLF)\r\n' >mixed-eol-terminators.md 487 | # add the .gitlab-ci.yml file. 488 | cat >.gitlab-ci.yml <<'EOF' 489 | info: 490 | tags: 491 | - windows 492 | - docker 493 | script: 494 | - | 495 | $FormatEnumerationLimit = -1 496 | function Write-Title($title) { 497 | Write-Output "#`n# $title`n#" 498 | } 499 | - | 500 | Write-Title 'Current user permissions' 501 | whoami.exe /all 502 | - | 503 | Write-Title 'Environment Variables' 504 | dir env: ` 505 | | Sort-Object -Property Name ` 506 | | Format-Table -AutoSize ` 507 | | Out-String -Stream -Width ([int]::MaxValue) ` 508 | | ForEach-Object {$_.TrimEnd()} 509 | EOF 510 | git add * .gitlab-ci.yml 511 | git commit -m 'init' 512 | git push 513 | popd 514 | 515 | # create a new repository with a .gitlab-ci.yml file to test building a docker 516 | # image on the windows docker gitlab-runner environment. 517 | # NB as of docker 24.0.7, --chown does not work in windows containers. 518 | # see https://github.com/rgl/windows-dockerfile-copy-with-chown-test 519 | # see https://github.com/moby/moby/issues/35507 520 | # see https://github.com/moby/moby/issues/41776 521 | pushd /tmp 522 | gitlab-create-project gitlab-runner-environment-build-windows-container-image $example_group_id 523 | git clone https://root:HeyH0Password@$domain/$example_group_name/gitlab-runner-environment-build-windows-container-image.git gitlab-runner-environment-build-windows-container-image && cd gitlab-runner-environment-build-windows-container-image 524 | # add a file with CR eol terminators to see whether they are preserved. 525 | printf '1. This line ends with carriage return (CR)\r2. This line ends with carriage return (CR)\r3. This line ends with carriage return (CR)\r' >cr-eol-terminators.md 526 | # add a file with LF eol terminators to see whether they are preserved. 527 | printf '1. This line ends with line feed (LF)\n2. This line ends with line feed (LF)\n3. This line ends with line feed (LF)\n' >lf-eol-terminators.md 528 | # add a file with CRLF eol terminators to see whether they are preserved. 529 | printf '1. This line ends with carriage return and line feed (CRLF)\r\n2. This line ends with carriage return and line feed (CRLF)\r\n3. This line ends with carriage return and line feed (CRLF)\r\n' >crlf-eol-terminators.md 530 | # add a file with mixed eol terminators to see whether they are preserved. 531 | printf '1. This line ends with carriage return (CR)\r2. This line ends with line feed (LF)\n3. This line ends with carriage return and line feed (CRLF)\r\n' >mixed-eol-terminators.md 532 | # add test script. 533 | cat >test.ps1 <<'EOF' 534 | function Write-Title($title) { 535 | Write-Output "`n#`n# $title`n#`n" 536 | } 537 | 538 | Write-Title "chown-guests.txt" 539 | Get-Acl chown-guests.txt | Format-List 540 | 541 | Write-Title "chown-administrator.txt" 542 | Get-Acl chown-administrator.txt | Format-List 543 | 544 | Write-Title "chown-containeruser.txt" 545 | Get-Acl chown-containeruser.txt | Format-List 546 | EOF 547 | # add test files. 548 | echo 'Administrator' >chown-administrator.txt 549 | echo 'Guests' >chown-guests.txt 550 | echo 'ContainerUser' >chown-containeruser.txt 551 | # add Dockefile. 552 | cat >Dockerfile <<'EOF' 553 | # escape=` 554 | #FROM mcr.microsoft.com/windows/nanoserver:ltsc2022 555 | FROM mcr.microsoft.com/powershell:7.4-windowsservercore-ltsc2022 556 | ENTRYPOINT ["pwsh.exe", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue'; $FormatEnumerationLimit = -1; "] 557 | SHELL ["pwsh.exe", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue'; $FormatEnumerationLimit = -1; "] 558 | RUN mkdir C:\test | Out-Null 559 | WORKDIR C:\test 560 | COPY test.ps1 ./ 561 | COPY --chown=Guests chown-guests.txt ./ 562 | COPY --chown=Administrator chown-administrator.txt ./ 563 | COPY --chown=ContainerUser chown-containeruser.txt ./ 564 | CMD ["./test.ps1"] 565 | EOF 566 | # add the .gitlab-ci.yml file. 567 | cat >.gitlab-ci.yml <<'EOF' 568 | default: 569 | before_script: 570 | # enable strict mode and fail the job when there is an unhandled exception. 571 | - | 572 | Set-StrictMode -Version Latest 573 | $FormatEnumerationLimit = -1 574 | $ErrorActionPreference = 'Stop' 575 | $ProgressPreference = 'SilentlyContinue' 576 | trap { 577 | Write-Host "ERROR: $_" 578 | ($_.ScriptStackTrace -split '\r?\n') -replace '^(.*)$','ERROR: $1' | Write-Host 579 | ($_.Exception.ToString() -split '\r?\n') -replace '^(.*)$','ERROR EXCEPTION: $1' | Write-Host 580 | Exit 1 581 | } 582 | # wrap the docker command (to make sure this script aborts when it fails). 583 | - | 584 | function docker { 585 | docker.exe @Args | Out-String -Stream -Width ([int]::MaxValue) 586 | if ($LASTEXITCODE) { 587 | throw "$(@('docker')+$Args | ConvertTo-Json -Compress) failed with exit code $LASTEXITCODE" 588 | } 589 | } 590 | build: 591 | tags: 592 | - windows 593 | - pwsh 594 | script: 595 | - | 596 | docker compose version 597 | - | 598 | docker info 599 | - | 600 | docker build --iidfile image-id.txt . 601 | $imageId = Get-Content -Raw image-id.txt 602 | - | 603 | docker run --rm --tty $imageId 604 | EOF 605 | git add * .gitlab-ci.yml 606 | git commit -m 'init' 607 | git push 608 | popd 609 | -------------------------------------------------------------------------------- /create-example-users.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -eux 3 | 4 | source /vagrant/_include_gitlab_api.sh 5 | 6 | email_domain=$(hostname --domain) 7 | 8 | users=( 9 | 'alice.doe Alice Doe' 10 | 'bob.doe Bob Doe' 11 | 'carol.doe Carol Doe' 12 | 'dave.doe Dave Doe' 13 | 'eve.doe Eve Doe' 14 | 'frank.doe Frank Doe' 15 | 'grace.doe Grace Doe' 16 | 'henry.doe Henry Doe' 17 | ) 18 | for user in "${users[@]}"; do 19 | username="$(echo "$user" | perl -n -e '/(.+?)\s+(.+)/ && print $1')" 20 | name="$(echo "$user" | perl -n -e '/(.+?)\s+(.+)/ && print $2')" 21 | 22 | gitlab-create-user "$username" "$name" "$username@$email_domain" HeyH0Password 23 | done 24 | -------------------------------------------------------------------------------- /gitlab.rb-active-directory-ldap.patch: -------------------------------------------------------------------------------- 1 | --- gitlab.rb.orig 2024-03-22 08:43:51.808098072 +0000 2 | +++ gitlab.rb 2024-03-22 08:49:58.698780373 +0000 3 | @@ -505,57 +505,36 @@ 4 | ###! **Be careful not to break the indentation in the ldap_servers block. It is 5 | ###! in yaml format and the spaces must be retained. Using tabs will not work.** 6 | 7 | -# gitlab_rails['ldap_enabled'] = false 8 | +gitlab_rails['ldap_enabled'] = true 9 | # gitlab_rails['prevent_ldap_sign_in'] = false 10 | 11 | ###! **remember to close this block with 'EOS' below** 12 | -# gitlab_rails['ldap_servers'] = YAML.load <<-'EOS' 13 | -# main: # 'main' is the GitLab 'provider ID' of this LDAP server 14 | -# label: 'LDAP' 15 | -# host: '_your_ldap_server' 16 | -# port: 389 17 | -# uid: 'sAMAccountName' 18 | -# bind_dn: '_the_full_dn_of_the_user_you_will_bind_with' 19 | -# password: '_the_password_of_the_bind_user' 20 | -# encryption: 'plain' # "start_tls" or "simple_tls" or "plain" 21 | -# verify_certificates: true 22 | -# smartcard_auth: false 23 | -# active_directory: true 24 | -# smartcard_ad_cert_field: 'altSecurityIdentities' 25 | -# smartcard_ad_cert_format: null # 'issuer_and_serial_number', 'issuer_and_subject' , 'principal_name' 26 | -# allow_username_or_email_login: false 27 | -# lowercase_usernames: false 28 | -# block_auto_created_users: false 29 | -# base: '' 30 | -# user_filter: '' 31 | -# ## EE only 32 | -# group_base: '' 33 | -# admin_group: '' 34 | -# sync_ssh_keys: false 35 | -# 36 | -# secondary: # 'secondary' is the GitLab 'provider ID' of second LDAP server 37 | -# label: 'LDAP' 38 | -# host: '_your_ldap_server' 39 | -# port: 389 40 | -# uid: 'sAMAccountName' 41 | -# bind_dn: '_the_full_dn_of_the_user_you_will_bind_with' 42 | -# password: '_the_password_of_the_bind_user' 43 | -# encryption: 'plain' # "start_tls" or "simple_tls" or "plain" 44 | -# verify_certificates: true 45 | -# smartcard_auth: false 46 | -# active_directory: true 47 | -# smartcard_ad_cert_field: 'altSecurityIdentities' 48 | -# smartcard_ad_cert_format: null # 'issuer_and_serial_number', 'issuer_and_subject' , 'principal_name' 49 | -# allow_username_or_email_login: false 50 | -# lowercase_usernames: false 51 | -# block_auto_created_users: false 52 | -# base: '' 53 | -# user_filter: '' 54 | -# ## EE only 55 | -# group_base: '' 56 | -# admin_group: '' 57 | -# sync_ssh_keys: false 58 | -# EOS 59 | +gitlab_rails['ldap_servers'] = YAML.load <<-'EOS' 60 | + main: # 'main' is the GitLab 'provider ID' of this LDAP server 61 | + label: 'LDAP' 62 | + host: 'dc.example.com' 63 | + port: 636 64 | + encryption: 'simple_tls' 65 | + verify_certificates: true 66 | + smartcard_auth: false 67 | + ca_file: '/etc/ssl/certs/ca-certificates.crt' 68 | + #ssl_version: '' 69 | + uid: 'sAMAccountName' 70 | + bind_dn: 'jane.doe@example.com' 71 | + password: 'HeyH0Password' 72 | + active_directory: true 73 | + allow_username_or_email_login: false 74 | + lowercase_usernames: false 75 | + block_auto_created_users: false 76 | + base: 'CN=Users,DC=example,DC=com' 77 | + user_filter: '(&(objectClass=person)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))' 78 | + attributes: 79 | + username: ['uid', 'userid', 'sAMAccountName'] 80 | + email: ['mail', 'email', 'userPrincipalName'] 81 | + name: 'displayName' 82 | + first_name: 'givenName' 83 | + last_name: 'sn' 84 | +EOS 85 | 86 | ### Smartcard authentication settings 87 | ###! Docs: https://docs.gitlab.com/ee/administration/auth/smartcard.html 88 | -------------------------------------------------------------------------------- /gitlab.rb-nginx-status.patch: -------------------------------------------------------------------------------- 1 | --- gitlab.rb.orig 2020-11-08 15:47:07.886114149 +0000 2 | +++ gitlab.rb 2020-11-08 16:01:08.737971127 +0000 3 | @@ -1353,19 +1353,19 @@ 4 | # nginx['request_buffering_off_path_regex'] = "/api/v\\d/jobs/\\d+/artifacts$|\\.git/git-receive-pack$|\\.git/gitlab-lfs/objects|\\.git/info/lfs/objects/batch$" 5 | 6 | ### Nginx status 7 | -# nginx['status'] = { 8 | -# "enable" => true, 9 | -# "listen_addresses" => ["127.0.0.1"], 10 | -# "fqdn" => "dev.example.com", 11 | -# "port" => 9999, 12 | -# "vts_enable" => true, 13 | -# "options" => { 14 | -# "server_tokens" => "off", # Don't show the version of NGINX 15 | -# "access_log" => "off", # Disable logs for stats 16 | -# "allow" => "127.0.0.1", # Only allow access from localhost 17 | -# "deny" => "all" # Deny access to anyone else 18 | -# } 19 | -# } 20 | +nginx['status'] = { 21 | + "enable" => true, 22 | + "listen_addresses" => ["127.0.0.1"], 23 | + "fqdn" => "localhost", 24 | + "port" => 8060, 25 | + "vts_enable" => true, 26 | + "options" => { 27 | + "server_tokens" => "off", # Don't show the version of NGINX 28 | + "access_log" => "off", # Disable logs for stats 29 | + "allow" => "127.0.0.1", # Only allow access from localhost 30 | + "deny" => "all" # Deny access to anyone else 31 | + } 32 | +} 33 | 34 | ################################################################################ 35 | ## GitLab Logging 36 | -------------------------------------------------------------------------------- /provision-certificates.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -euxo pipefail 3 | 4 | domain="$(hostname --fqdn)" 5 | 6 | ca_file_name='gitlab-ca' 7 | ca_common_name='GitLab CA' 8 | 9 | # copy from host. 10 | if [ -d "/vagrant/tmp/$ca_file_name" ]; then 11 | rsync \ 12 | --archive \ 13 | --no-owner \ 14 | --no-group \ 15 | --delete \ 16 | "/vagrant/tmp/$ca_file_name/" \ 17 | /etc/ssl/private 18 | fi 19 | 20 | # go into the CA data directory. 21 | cd /etc/ssl/private 22 | 23 | # create the CA. 24 | if [ ! -f "$ca_file_name-crt.pem" ]; then 25 | openssl genrsa \ 26 | -out "$ca_file_name-key.pem" \ 27 | 2048 \ 28 | 2>/dev/null 29 | chmod 400 "$ca_file_name-key.pem" 30 | openssl req -new \ 31 | -sha256 \ 32 | -subj "/CN=$ca_common_name" \ 33 | -key "$ca_file_name-key.pem" \ 34 | -out "$ca_file_name-csr.pem" 35 | openssl x509 -req -sha256 \ 36 | -signkey "$ca_file_name-key.pem" \ 37 | -extensions a \ 38 | -extfile <(echo "[a] 39 | basicConstraints=critical,CA:TRUE,pathlen:0 40 | keyUsage=critical,digitalSignature,keyCertSign,cRLSign 41 | ") \ 42 | -days 365 \ 43 | -in "$ca_file_name-csr.pem" \ 44 | -out "$ca_file_name-crt.pem" 45 | openssl x509 \ 46 | -in "$ca_file_name-crt.pem" \ 47 | -outform der \ 48 | -out "$ca_file_name-crt.der" 49 | # dump the certificate contents (for logging purposes). 50 | #openssl x509 -noout -text -in "$ca_file_name-crt.pem" 51 | fi 52 | 53 | # trust the CA. 54 | if [ ! -f "/usr/local/share/ca-certificates/$ca_file_name.crt" ]; then 55 | install "$ca_file_name-crt.pem" "/usr/local/share/ca-certificates/$ca_file_name.crt" 56 | update-ca-certificates -v 57 | fi 58 | 59 | # create the server certificates. 60 | function create-server-certificate { 61 | local domain="$1" 62 | if [ -f "$domain-crt.pem" ]; then 63 | return 64 | fi 65 | openssl genrsa \ 66 | -out "$domain-key.pem" \ 67 | 2048 \ 68 | 2>/dev/null 69 | chmod 400 "$domain-key.pem" 70 | openssl req -new \ 71 | -sha256 \ 72 | -subj "/CN=$domain" \ 73 | -key "$domain-key.pem" \ 74 | -out "$domain-csr.pem" 75 | openssl x509 -req -sha256 \ 76 | -CA "$ca_file_name-crt.pem" \ 77 | -CAkey "$ca_file_name-key.pem" \ 78 | -CAcreateserial \ 79 | -extensions a \ 80 | -extfile <(echo "[a] 81 | subjectAltName=DNS:$domain 82 | extendedKeyUsage=critical,serverAuth 83 | ") \ 84 | -days 365 \ 85 | -in "$domain-csr.pem" \ 86 | -out "$domain-crt.pem" 87 | openssl pkcs12 -export \ 88 | -keyex \ 89 | -inkey "$domain-key.pem" \ 90 | -in "$domain-crt.pem" \ 91 | -certfile "$domain-crt.pem" \ 92 | -passout pass: \ 93 | -out "$domain-key.p12" 94 | # dump the certificate contents (for logging purposes). 95 | #openssl x509 -noout -text -in "$domain-crt.pem" 96 | #openssl pkcs12 -info -nodes -passin pass: -in "$domain-key.p12" 97 | } 98 | create-server-certificate "$domain" 99 | create-server-certificate "vault.$domain" 100 | create-server-certificate "ubuntu.$domain" 101 | create-server-certificate "windows.$domain" 102 | 103 | # create the client certificates. 104 | function create-client-certificate { 105 | local name="$1" 106 | if [ -f "$name-crt.pem" ]; then 107 | return 108 | fi 109 | openssl genrsa \ 110 | -out "$name-key.pem" \ 111 | 2048 \ 112 | 2>/dev/null 113 | chmod 400 "$name-key.pem" 114 | openssl req -new \ 115 | -sha256 \ 116 | -subj "/CN=$name" \ 117 | -key "$name-key.pem" \ 118 | -out "$name-csr.pem" 119 | openssl x509 -req -sha256 \ 120 | -CA "$ca_file_name-crt.pem" \ 121 | -CAkey "$ca_file_name-key.pem" \ 122 | -CAcreateserial \ 123 | -extensions a \ 124 | -extfile <(echo "[a] 125 | extendedKeyUsage=critical,clientAuth 126 | ") \ 127 | -days 365 \ 128 | -in "$name-csr.pem" \ 129 | -out "$name-crt.pem" 130 | openssl pkcs12 -export \ 131 | -keyex \ 132 | -inkey "$name-key.pem" \ 133 | -in "$name-crt.pem" \ 134 | -certfile "$name-crt.pem" \ 135 | -passout pass: \ 136 | -out "$name-key.p12" 137 | # dump the certificate contents (for logging purposes). 138 | #openssl x509 -noout -text -in "$name-crt.pem" 139 | #openssl pkcs12 -info -nodes -passin pass: -in "$name-key.p12" 140 | } 141 | create-client-certificate ubuntu 142 | create-client-certificate incus 143 | create-client-certificate lxd 144 | create-client-certificate windows 145 | 146 | # copy to host. 147 | mkdir -p "/vagrant/tmp/$ca_file_name" 148 | rsync \ 149 | --archive \ 150 | --no-owner \ 151 | --no-group \ 152 | --delete \ 153 | /etc/ssl/private/ \ 154 | "/vagrant/tmp/$ca_file_name" 155 | -------------------------------------------------------------------------------- /provision-dns-server.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -euxo pipefail 3 | 4 | config_gitlab_fqdn="$(hostname --fqdn)" 5 | config_gitlab_ip="$1" 6 | config_vault_fqdn="$2" 7 | config_vault_ip="$3" 8 | config_ubuntu_runner_fqdn="$4" 9 | config_ubuntu_runner_ip="$5" 10 | config_incus_runner_fqdn="$6" 11 | config_incus_runner_ip="$7" 12 | config_lxd_runner_fqdn="$8" 13 | config_lxd_runner_ip="$9" 14 | config_windows_runner_fqdn="${10}" 15 | config_windows_runner_ip="${11}" 16 | 17 | # provision a recursive DNS server as a workaround for being able to access 18 | # our custom domain from a windows container. 19 | # see ../gitlab-ci-vagrant/windows/provision-gitlab-runner.ps1 20 | # see http://www.thekelleys.org.uk/dnsmasq/docs/setup.html 21 | # see http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html 22 | apt-get install -y dnsmasq 23 | cat >/etc/dnsmasq.d/local.conf <>~/.bashrc 15 | export PATH="$PATH:~/.local/bin" 16 | 17 | # configure gitlab-cli with the root token. 18 | cat >~/.python-gitlab.cfg </etc/systemd/system/mailpit.service <<'EOF' 29 | [Unit] 30 | Description=mailpit 31 | After=network.target 32 | 33 | [Service] 34 | Type=simple 35 | User=mailpit 36 | Group=mailpit 37 | ExecStart=/opt/mailpit/bin/mailpit \ 38 | --db-file /opt/mailpit/data/mailpit.db \ 39 | --smtp-auth-accept-any \ 40 | --smtp-auth-allow-insecure 41 | WorkingDirectory=/opt/mailpit 42 | Restart=always 43 | 44 | [Install] 45 | WantedBy=multi-user.target 46 | EOF 47 | systemctl enable mailpit 48 | systemctl start mailpit 49 | 50 | # configure the system to use mailpit as a smarthost. 51 | # these anwsers were obtained (after installing nullmailer) with: 52 | # #sudo debconf-show nullmailer 53 | # sudo apt-get install debconf-utils 54 | # # this way you can see the comments: 55 | # sudo debconf-get-selections 56 | # # this way you can just see the values needed for debconf-set-selections: 57 | # sudo debconf-get-selections | grep -E '^nullmailer\s+' | sort 58 | debconf-set-selections <&1 || true) \ 23 | | awk '/^ HTTP/{print $2}')" 24 | case "$status_code" in 25 | "$desired_state") 26 | return 0 27 | ;; 28 | *) 29 | sleep 5 30 | ;; 31 | esac 32 | done 33 | } 34 | 35 | 36 | # install dependencies. 37 | apt-get install -y httpie jq 38 | 39 | 40 | # install openbao. 41 | # see https://developer.hashicorp.com/vault/docs/concepts/production-hardening 42 | # see https://openbao.org/docs/internals/security/ 43 | # see https://github.com/openbao/openbao 44 | # renovate: datasource=github-releases depName=openbao/openbao 45 | openbao_version='2.2.2' 46 | url="https://github.com/openbao/openbao/releases/download/v$openbao_version/bao_${openbao_version}_linux_amd64.deb" 47 | p="$(basename "$url")" 48 | wget -qO "$p" "$url" 49 | dpkg -i "$p" 50 | rm -rf "$p" 51 | bao -v 52 | 53 | # configure the service to auto-unseal openbao. 54 | install -d /etc/systemd/system/openbao.service.d 55 | cat >/etc/systemd/system/openbao.service.d/override.conf <<'EOF' 56 | [Service] 57 | ExecStartPost= 58 | ExecStartPost=+/opt/openbao/auto-unseal/unseal 59 | EOF 60 | systemctl daemon-reload 61 | 62 | # configure. 63 | export VAULT_ADDR="https://$domain:8200" 64 | echo export VAULT_ADDR="https://$domain:8200" >>~/.bash_login 65 | install -o openbao -g openbao -m 770 -d /etc/openbao 66 | install -o root -g openbao -m 710 -d /opt/openbao 67 | install -o openbao -g openbao -m 700 -d /opt/openbao/data 68 | install -o root -g openbao -m 750 -d /opt/openbao/tls 69 | install -o root -g openbao -m 440 /vagrant/tmp/gitlab-ca/$domain-crt.pem /opt/openbao/tls 70 | install -o root -g openbao -m 440 /vagrant/tmp/gitlab-ca/$domain-key.pem /opt/openbao/tls 71 | cat >/etc/openbao/openbao.hcl </opt/openbao/auto-unseal/unseal 111 | 112 | # disable swap. 113 | swapoff --all 114 | sed -i -E 's,^(\s*[^#].+\sswap.+),#\1,g' /etc/fstab 115 | 116 | # start openbao. 117 | systemctl enable openbao 118 | systemctl start openbao 119 | wait-for-state 501 # wait for the not-initialized state. 120 | journalctl -u openbao 121 | 122 | # init openbao. 123 | # NB openbao-operator-init-result.txt will have something like: 124 | # Unseal Key 1: sXiqMfCPiRNGvo+tEoHVGy+FHFW092H7vfOY0wPrzpYh 125 | # Unseal Key 2: dCm5+NhacPcX6GwI0IMMK+CM0xL6wif5/k0LJ0XTPHhy 126 | # Unseal Key 3: YjbM3TANam0dO9FTa0y/2wj7nxnlDyct7oVMksHs7trE 127 | # Unseal Key 4: CxWG0yrF75cIYsKvWQBku8klN9oPaPJDWqO7l7LNWX2A 128 | # Unseal Key 5: C+ttQv3KeViOkIxVZH7gXuZ7iZPKi0va1/lUBSiMeyLz 129 | # Initial Root Token: d2bb2175-2264-d18b-e8d8-18b1d8b61278 130 | # 131 | # Vault initialized with 5 keys and a key threshold of 3. Please 132 | # securely distribute the above keys. When the openbao is re-sealed, 133 | # restarted, or stopped, you must provide at least 3 of these keys 134 | # to unseal it again. 135 | # 136 | # Vault does not store the master key. Without at least 3 keys, 137 | # your openbao will remain permanently sealed. 138 | pushd ~ 139 | install -o root -g root -m 600 /dev/null openbao-operator-init-result.txt 140 | install -o root -g root -m 600 /dev/null /opt/openbao/auto-unseal/unseal-keys.txt 141 | install -o root -g root -m 600 /dev/null .vault-token 142 | bao operator init >openbao-operator-init-result.txt 143 | awk '/Unseal Key [0-9]+: /{print $4}' openbao-operator-init-result.txt | head -3 >/opt/openbao/auto-unseal/unseal-keys.txt 144 | awk '/Initial Root Token: /{print $4}' openbao-operator-init-result.txt | tr -d '\n' >.vault-token 145 | cp .vault-token /vagrant/tmp/vault-root-token.txt 146 | popd 147 | cat >/opt/openbao/auto-unseal/unseal <&1 || true) \ 163 | | awk '/^ HTTP/{print \$2}')" 164 | case "\$status_code" in 165 | # openbao is sealed. break the loop, and unseal it. 166 | 503) 167 | break 168 | ;; 169 | # for some odd reason openbao is already unsealed. anyways, its 170 | # ready and unsealed, so exit this script. 171 | 200) 172 | exit 0 173 | ;; 174 | # otherwise, wait a bit, then retry the health check. 175 | *) 176 | sleep 5 177 | ;; 178 | esac 179 | done 180 | KEYS=\$(cat /opt/openbao/auto-unseal/unseal-keys.txt) 181 | for key in \$KEYS; do 182 | /usr/bin/bao operator unseal \$key 183 | done 184 | EOF 185 | /opt/openbao/auto-unseal/unseal 186 | 187 | # restart openbao to verify that the automatic unseal is working. 188 | systemctl restart openbao 189 | wait-for-state 200 # wait for the unsealed state. 190 | journalctl -u openbao 191 | bao status 192 | 193 | # install command line autocomplete. 194 | bao -autocomplete-install 195 | 196 | # show the openbao tls certificate. 197 | openssl s_client -connect $domain:8200 -servername $domain /dev/null | openssl x509 -noout -text 198 | 199 | # show information about our own token. 200 | # see https://openbao.org/api-docs/auth/token/#lookup-a-token-self 201 | bao token lookup 202 | http $VAULT_ADDR/v1/auth/token/lookup-self \ 203 | "X-Vault-Token: $(cat ~/.vault-token)" \ 204 | | jq .data 205 | 206 | # enable auditing to stdout (use journalctl -u openbao to see it). 207 | # see https://openbao.org/docs/commands/audit/ 208 | # see https://openbao.org/docs/commands/audit/enable/ 209 | bao audit enable file file_path=stdout log_raw=true 210 | bao audit list 211 | 212 | # enable the jwt authentication method. 213 | bao auth enable jwt 214 | 215 | # configure the openbao vault jwt authentication method to use gitlab. 216 | bao write auth/jwt/config \ 217 | "oidc_discovery_url=https://$gitlab_domain" \ 218 | "bound_issuer=https://$gitlab_domain" 219 | 220 | # enable the kv 2 secrets engine. 221 | bao secrets enable -version=2 -path=secret kv 222 | 223 | # list enabled authentication methods. 224 | bao auth list 225 | 226 | # list the active secret backends. 227 | bao secrets list 228 | 229 | # show the default policy. 230 | # see https://openbao.org/docs/concepts/policies/ 231 | bao read sys/policy/default 232 | 233 | # list the active authentication backends. 234 | # see https://openbao.org/docs/auth/ 235 | # see https://openbao.org/api-docs/system/auth/ 236 | bao path-help sys/auth 237 | http $VAULT_ADDR/v1/sys/auth "X-Vault-Token: $(cat ~/.vault-token)" \ 238 | | jq -r 'keys[] | select(endswith("/"))' 239 | -------------------------------------------------------------------------------- /provision-resize-disk.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -euo pipefail 3 | 4 | partition_device="$(findmnt -no SOURCE /)" 5 | partition_number="$(echo "$partition_device" | perl -ne '/(\d+)$/ && print $1')" 6 | disk_device="$(echo "$partition_device" | perl -ne '/(.+?)\d+$/ && print $1')" 7 | 8 | # resize the partition table. 9 | # Warning: Not all of the space available to /dev/sda appears to be used, you can fix the GPT to use all of the space (an extra 50331648 blocks) or continue with the current setting? 10 | # Fix/Ignore? Fix 11 | # Partition number? 2 12 | # Warning: Partition /dev/sda2 is being used. Are you sure you want to continue? 13 | # Yes/No? 14 | # Yes 15 | # End? [8589MB]? 16 | # 100% 17 | parted ---pretend-input-tty "$disk_device" </etc/vim/vimrc.local<<'EOF' 19 | syntax on 20 | set background=dark 21 | set esckeys 22 | set ruler 23 | set laststatus=2 24 | set nobackup 25 | autocmd BufNewFile,BufRead Vagrantfile set ft=ruby 26 | EOF 27 | 28 | # set the initial shell history. 29 | cat >~/.bash_history <<'EOF' 30 | tail -f /var/log/gitlab/gitlab-rails/*.log 31 | gitlab-ctl reconfigure 32 | vim /etc/gitlab/gitlab.rb 33 | vim /etc/hosts 34 | netstat -antp 35 | EOF 36 | 37 | # install the gitlab deb repository. 38 | apt-get install -y --no-install-recommends curl 39 | wget -qO- https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | bash -ex 40 | 41 | # install gitlab with the omnibus package. 42 | apt-get install -y --no-install-recommends "gitlab-ce=$gitlab_version" 43 | 44 | # configure the gitlab certificate. 45 | install -m 700 -o root -g root -d /etc/gitlab/ssl 46 | ln -s "/etc/ssl/private/$domain-key.pem" "/etc/gitlab/ssl/$domain.key" 47 | ln -s "/etc/ssl/private/$domain-crt.pem" "/etc/gitlab/ssl/$domain.crt" 48 | sed -i -E "s,^(external_url\s+).+,\1'https://$domain'," /etc/gitlab/gitlab.rb 49 | sed -i -E "s,^(\s*#\s*)?(nginx\['redirect_http_to_https'\]\s+).+,\2= true," /etc/gitlab/gitlab.rb 50 | 51 | # show the changes we've made to gitlab.rb. 52 | diff -u /opt/gitlab/etc/gitlab.rb.template /etc/gitlab/gitlab.rb || test $? = 1 53 | 54 | # configure nginx status. 55 | # see https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2857 56 | [ -n "$(which patch)" ] || apt-get install -y patch 57 | patch --batch --quiet /etc/gitlab/gitlab.rb /vagrant/gitlab.rb-nginx-status.patch 58 | 59 | # configure gitlab to use the dc.example.com Active Directory LDAP. 60 | # NB this assumes you are running https://github.com/rgl/windows-domain-controller-vagrant. 61 | if [ -f /vagrant/tmp/ExampleEnterpriseRootCA.der ]; then 62 | openssl x509 -inform der -in /vagrant/tmp/ExampleEnterpriseRootCA.der -out /usr/local/share/ca-certificates/ExampleEnterpriseRootCA.crt 63 | update-ca-certificates --verbose 64 | echo '192.168.56.2 dc.example.com' >>/etc/hosts 65 | patch --batch --quiet /etc/gitlab/gitlab.rb /vagrant/gitlab.rb-active-directory-ldap.patch 66 | fi 67 | 68 | # configure gitlab. 69 | gitlab-ctl reconfigure 70 | 71 | # set the gitlab root user password and create a personal access token. 72 | # see https://gitlab.com/gitlab-org/gitlab-foss/blob/v18.0.1/app/models/user.rb 73 | # see https://gitlab.com/gitlab-org/gitlab-foss/blob/v18.0.1/app/models/personal_access_token.rb 74 | gitlab-rails runner -e production - <<'EOF' 75 | u = User.first 76 | u.password_automatically_set = false 77 | u.password = 'HeyH0Password' 78 | u.password_confirmation = 'HeyH0Password' 79 | u.save! 80 | t = PersonalAccessToken.new({ 81 | user: u, 82 | name: 'vagrant', 83 | scopes: ['api', 'read_user', 'sudo'], 84 | expires_at: PersonalAccessToken::MAX_PERSONAL_ACCESS_TOKEN_LIFETIME_IN_DAYS.days.from_now}) 85 | t.save! 86 | File.write( 87 | '/tmp/gitlab-root-personal-access-token.txt', 88 | t.token) 89 | EOF 90 | mkdir -p /vagrant/tmp 91 | mv /tmp/gitlab-root-personal-access-token.txt /vagrant/tmp 92 | 93 | # set the gitlab sign in page title and description. 94 | # NB since gitlab 12.7 this can also be done with the appearance api. 95 | # see https://docs.gitlab.com/api/appearance/. 96 | # see https://gitlab.com/gitlab-org/gitlab-foss/blob/v18.0.1/app/models/appearance.rb 97 | gitlab-rails runner -e production - <<'EOF' 98 | a = Appearance.first_or_initialize 99 | a.title = 'GitLab Community Edition' 100 | a.description = 'Sign in on the right or [explore the public projects](/explore/projects).' 101 | a.save! 102 | EOF 103 | 104 | # include the gitlab api functions. 105 | apt-get install -y jq 106 | source /vagrant/_include_gitlab_api.sh 107 | 108 | # disable telemetry. 109 | # see https://gitlab.example.com/admin/application_settings/metrics_and_profiling 110 | # see https://docs.gitlab.com/api/settings/ 111 | gitlab-api PUT /application/settings \ 112 | usage_ping_enabled:=false \ 113 | version_check_enabled:=false \ 114 | gitlab_product_usage_data_enabled:=false \ 115 | snowplow_enabled:=false \ 116 | --check-status \ 117 | --print '' 118 | 119 | # disable user signup. 120 | gitlab-api PUT /application/settings signup_enabled:=false --check-status 121 | 122 | # disable the creation of public groups and repositories. 123 | # NB administrators can still create them, but regular users cannot. 124 | # see https://docs.gitlab.com/user/public_access/ 125 | # see https://docs.gitlab.com/api/settings/ 126 | gitlab-api PUT /application/settings restricted_visibility_levels:='["public"]' --check-status --print '' 127 | 128 | # set the maximum artifacts size to 1 GB (default is 100 MB; gitlab.com default is 1 GB). 129 | # NB this can be set at the instance level (like in this example), at group level, and project level. 130 | # NB this can also be set in the UI at Admin Area | CI/CD | Continuous Integration and Deployment | Maximum artifacts size (MB). 131 | # see https://docs.gitlab.com/api/settings/ 132 | # see https://gitlab.example.com/admin/application_settings/ci_cd 133 | # see https://gitlab.example.com/help/user/admin_area/settings/continuous_integration#maximum-artifacts-size 134 | # see https://gitlab.example.com/help/user/gitlab_com/index.md#gitlab-cicd 135 | gitlab-api PUT /application/settings max_artifacts_size:=1024 --check-status --print '' 136 | 137 | # do not keep the latest artifacts for all jobs in the latest successful pipelines. 138 | # NB all artifacts will be erased after they expire. 139 | gitlab-api PUT /application/settings keep_latest_artifact:=false --check-status --print '' 140 | 141 | # set default artifacts expiration to 3d (default is 30d; gitlab.com default is 30d). 142 | # see https://gitlab.example.com/help/user/admin_area/settings/continuous_integration#default-artifacts-expiration 143 | gitlab-api PUT /application/settings default_artifacts_expire_in=3d --check-status --print '' 144 | 145 | # archive the jobs after 3d (default is to never archive them; gitlab.com default is 3mo). 146 | # see https://gitlab.example.com/help/user/admin_area/settings/continuous_integration#archive-jobs 147 | gitlab-api PUT /application/settings archive_builds_in_human_readable=3d --check-status --print '' 148 | 149 | # enable prometheus metrics. 150 | # see https://gitlab.example.com/help/administration/monitoring/prometheus/gitlab_metrics#gitlab-prometheus-metrics 151 | # see https://docs.gitlab.com/api/settings/ 152 | sed -i -E "s,^(\s*#\s*)?(prometheus\['listen_address'\]).+,\2 = '0.0.0.0:9090'," /etc/gitlab/gitlab.rb 153 | gitlab-api PUT /application/settings prometheus_metrics_enabled:=true --check-status 154 | gitlab-ctl reconfigure 155 | gitlab-wait-for-ready 156 | 157 | # configure postgres to allow the host (e.g. pgAdmin III) to easily connect. 158 | if $testing; then 159 | echo 'host all all 10.10.9.0/24 trust' >> /var/opt/gitlab/postgresql/data/pg_hba.conf 160 | sed -i -E "s,^(\s*#\s*)?(listen_addresses\s+).+,\2= '*'," /var/opt/gitlab/postgresql/data/postgresql.conf 161 | gitlab-ctl restart postgresql 162 | fi 163 | 164 | # create artifacts that need to be shared with the other nodes. 165 | mkdir -p /vagrant/tmp 166 | pushd /vagrant/tmp 167 | find \ 168 | /etc/ssh \ 169 | -name 'ssh_host_*_key.pub' \ 170 | -exec sh -c "(echo -n '$domain '; cat {})" \; \ 171 | >$domain.ssh_known_hosts 172 | rm -f gitlab-runner-authentication-token-*.json 173 | # register the ubuntu 22.04 shell runner. 174 | # see https://docs.gitlab.com/api/runners/#create-a-runner 175 | # see https://docs.gitlab.com/api/users/#create-a-runner-linked-to-a-user 176 | # see https://docs.gitlab.com/runner/executors/shell/ 177 | gitlab-api POST /user/runners \ 178 | runner_type=instance_type \ 179 | tag_list='shell,linux,ubuntu,ubuntu-22.04' \ 180 | description='Shell / Ubuntu 22.04' \ 181 | --check-status \ 182 | >gitlab-runner-authentication-token-ubuntu-22.04-shell.json 183 | # register the ubuntu 22.04 docker runner. 184 | # see https://docs.gitlab.com/runner/executors/docker/ 185 | gitlab-api POST /user/runners \ 186 | runner_type=instance_type \ 187 | tag_list='docker,linux,ubuntu,ubuntu-22.04' \ 188 | description='Docker / Ubuntu 22.04' \ 189 | --check-status \ 190 | >gitlab-runner-authentication-token-ubuntu-22.04-docker.json 191 | # register the ubuntu 22.04 incus runner. 192 | # see https://docs.gitlab.com/runner/executors/custom/ 193 | # see https://docs.gitlab.com/runner/executors/custom_examples/lxd/ 194 | gitlab-api POST /user/runners \ 195 | runner_type=instance_type \ 196 | tag_list='incus,linux,ubuntu,ubuntu-22.04' \ 197 | description='Incus / Ubuntu 22.04' \ 198 | --check-status \ 199 | >gitlab-runner-authentication-token-ubuntu-22.04-incus.json 200 | # register the ubuntu 22.04 lxd runner. 201 | # see https://docs.gitlab.com/runner/executors/custom/ 202 | # see https://docs.gitlab.com/runner/executors/custom_examples/lxd/ 203 | gitlab-api POST /user/runners \ 204 | runner_type=instance_type \ 205 | tag_list='lxd,linux,ubuntu,ubuntu-22.04' \ 206 | description='LXD / Ubuntu 22.04' \ 207 | --check-status \ 208 | >gitlab-runner-authentication-token-ubuntu-22.04-lxd.json 209 | # register the k3s k8s runner. 210 | # see https://docs.gitlab.com/runner/executors/kubernetes/ 211 | gitlab-api POST /user/runners \ 212 | runner_type=instance_type \ 213 | tag_list='k3s,k8s' \ 214 | description='K3S / Kubernetes' \ 215 | --check-status \ 216 | >gitlab-runner-authentication-token-kubernetes-k3s.json 217 | # register the windows 2022 shell runner. 218 | # see https://docs.gitlab.com/runner/executors/shell/ 219 | gitlab-api POST /user/runners \ 220 | runner_type=instance_type \ 221 | tag_list='pwsh,shell,vs2022,windows,windows-2022' \ 222 | description='Shell / Windows 2022' \ 223 | --check-status \ 224 | >gitlab-runner-authentication-token-windows-2022-shell.json 225 | # register the windows 2022 docker runner. 226 | # see https://docs.gitlab.com/runner/executors/docker/ 227 | gitlab-api POST /user/runners \ 228 | runner_type=instance_type \ 229 | tag_list='docker,windows,windows-2022' \ 230 | description='Docker / Windows 2022' \ 231 | --check-status \ 232 | >gitlab-runner-authentication-token-windows-2022-docker.json 233 | popd 234 | -------------------------------------------------------------------------------- /renovate.json5: -------------------------------------------------------------------------------- 1 | // see https://docs.renovatebot.com/templates/ 2 | // see https://docs.renovatebot.com/modules/manager/ 3 | // see https://docs.renovatebot.com/modules/manager/regex/ 4 | // see https://docs.renovatebot.com/configuration-options/ 5 | { 6 | "$schema": "https://docs.renovatebot.com/renovate-schema.json", 7 | "regexManagers": [ 8 | // default datasources. 9 | { 10 | "fileMatch": [ 11 | "Vagrantfile", 12 | "\\.sh$", 13 | ], 14 | "matchStrings": [ 15 | "# renovate: datasource=(?[^:]+?) depName=(?.+?)( versioning=(?.+?))?( extractVersion=(?.+?))?( registryUrl=(?.+?))?\\s.+?[:=]\\s*[\"']?(?.+?)[\"']?\\s" 16 | ], 17 | "versioningTemplate": "{{#if versioning}}{{{versioning}}}{{else}}semver-coerced{{/if}}", 18 | "extractVersionTemplate": "{{#if extractVersion}}{{{extractVersion}}}{{else}}^v?(?.+)${{/if}}" 19 | }, 20 | ] 21 | } -------------------------------------------------------------------------------- /renovate.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -euo pipefail 3 | 4 | # this executes renovate against the local repository. 5 | # NB this uses a temporary gitea instance because running renovate against a 6 | # local directory not (yet?) supported. 7 | # see https://github.com/renovatebot/renovate/issues/3609 8 | 9 | export RENOVATE_USERNAME='renovate' 10 | export RENOVATE_NAME='Renovate Bot' 11 | export RENOVATE_PASSWORD='password' 12 | gitea_container_name="$(basename "$(dirname "$(realpath "${BASH_SOURCE[0]}")")")-renovate-gitea" 13 | 14 | # see https://hub.docker.com/r/gitea/gitea/tags 15 | # renovate: datasource=docker depName=gitea/gitea 16 | gitea_version='1.23.8' 17 | 18 | # see https://hub.docker.com/r/renovate/renovate/tags 19 | # renovate: datasource=docker depName=renovate/renovate 20 | renovate_version='40.36.8' 21 | 22 | # clean. 23 | echo 'Deleting existing Gitea...' 24 | docker rm --force "$gitea_container_name" >/dev/null 2>&1 25 | echo 'Deleting existing temporary files...' 26 | rm -f tmp/renovate-* 27 | install -d tmp 28 | 29 | # start gitea in background. 30 | # see https://docs.gitea.io/en-us/config-cheat-sheet/ 31 | # see https://github.com/go-gitea/gitea/releases 32 | # see https://github.com/go-gitea/gitea/blob/v1.23.8/docker/root/etc/s6/gitea/setup 33 | echo 'Starting Gitea...' 34 | docker run \ 35 | --detach \ 36 | --name "$gitea_container_name" \ 37 | -v /etc/timezone:/etc/timezone:ro \ 38 | -v /etc/localtime:/etc/localtime:ro \ 39 | -e SECRET_KEY=opensesame \ 40 | -p 3000 \ 41 | "gitea/gitea:$gitea_version" \ 42 | >/dev/null 43 | gitea_addr="$(docker port "$gitea_container_name" 3000 | head -1)" 44 | gitea_url="http://$gitea_addr" 45 | export RENOVATE_ENDPOINT="$gitea_url" 46 | export GIT_PUSH_REPOSITORY="http://$RENOVATE_USERNAME:$RENOVATE_PASSWORD@$gitea_addr/$RENOVATE_USERNAME/test.git" 47 | 48 | # wait for gitea to be ready. 49 | echo "Waiting for Gitea to be ready at $gitea_url..." 50 | GITEA_URL="$gitea_url" bash -euc 'while [ -z "$(wget -qO- "$GITEA_URL/api/v1/version" | jq -r ".version | select(.!=null)")" ]; do sleep 5; done' 51 | 52 | # create user in gitea. 53 | echo "Creating Gitea $RENOVATE_USERNAME user..." 54 | docker exec --user git "$gitea_container_name" gitea admin user create \ 55 | --admin \ 56 | --email "$RENOVATE_USERNAME@example.com" \ 57 | --username "$RENOVATE_USERNAME" \ 58 | --password "$RENOVATE_PASSWORD" 59 | curl \ 60 | --silent \ 61 | --show-error \ 62 | --fail-with-body \ 63 | -u "$RENOVATE_USERNAME:$RENOVATE_PASSWORD" \ 64 | -X 'PATCH' \ 65 | -H 'Accept: application/json' \ 66 | -H 'Content-Type: application/json' \ 67 | -d "{\"full_name\":\"$RENOVATE_NAME\"}" \ 68 | "$gitea_url/api/v1/user/settings" \ 69 | | jq \ 70 | > /dev/null 71 | 72 | # create the user personal access token. 73 | # see https://docs.gitea.io/en-us/api-usage/ 74 | # see https://docs.gitea.io/en-us/oauth2-provider/#scopes 75 | # see https://try.gitea.io/api/swagger#/user/userCreateToken 76 | echo "Creating Gitea $RENOVATE_USERNAME user personal access token..." 77 | curl \ 78 | --silent \ 79 | --show-error \ 80 | --fail-with-body \ 81 | -u "$RENOVATE_USERNAME:$RENOVATE_PASSWORD" \ 82 | -X POST \ 83 | -H "Content-Type: application/json" \ 84 | -d '{"name": "renovate", "scopes": ["read:user", "write:issue", "write:repository"]}' \ 85 | "$gitea_url/api/v1/users/$RENOVATE_USERNAME/tokens" \ 86 | | jq -r .sha1 \ 87 | >tmp/renovate-gitea-token.txt 88 | 89 | # try the token. 90 | echo "Trying the Gitea $RENOVATE_USERNAME user personal access token..." 91 | RENOVATE_TOKEN="$(cat tmp/renovate-gitea-token.txt)" 92 | export RENOVATE_TOKEN 93 | curl \ 94 | --silent \ 95 | --show-error \ 96 | --fail-with-body \ 97 | -H "Authorization: token $RENOVATE_TOKEN" \ 98 | -H 'Accept: application/json' \ 99 | "$gitea_url/api/v1/version" \ 100 | | jq \ 101 | > /dev/null 102 | 103 | # create remote repository in gitea. 104 | echo "Creating Gitea $RENOVATE_USERNAME test repository..." 105 | curl \ 106 | --silent \ 107 | --show-error \ 108 | --fail-with-body \ 109 | -u "$RENOVATE_USERNAME:$RENOVATE_PASSWORD" \ 110 | -X POST \ 111 | -H 'Accept: application/json' \ 112 | -H 'Content-Type: application/json' \ 113 | -d '{"name": "test"}' \ 114 | "$gitea_url/api/v1/user/repos" \ 115 | | jq \ 116 | > /dev/null 117 | 118 | # push the code to local gitea repository. 119 | # NB running renovate locally is not yet supported. 120 | # see https://github.com/renovatebot/renovate/issues/3609 121 | echo "Pushing local repository to Gitea $RENOVATE_USERNAME test repository..." 122 | git push --force "$GIT_PUSH_REPOSITORY" 123 | 124 | # see https://docs.renovatebot.com/modules/platform/gitea/ 125 | # see https://docs.renovatebot.com/self-hosted-configuration/#dryrun 126 | # see https://github.com/renovatebot/renovate/blob/main/docs/usage/examples/self-hosting.md 127 | # see https://github.com/renovatebot/renovate/tree/main/lib/modules/datasource 128 | # see https://github.com/renovatebot/renovate/tree/main/lib/modules/versioning 129 | RENOVATE_TOKEN="$(cat tmp/renovate-gitea-token.txt)" 130 | export RENOVATE_TOKEN 131 | # NB these can also be passed as raw positional arguments to docker run. 132 | export RENOVATE_REPOSITORIES="$RENOVATE_USERNAME/test" 133 | # see https://docs.github.com/en/rest/rate-limit#get-rate-limit-status-for-the-authenticated-user 134 | # see https://github.com/settings/tokens 135 | # NB this is only used for authentication. the token should not have any scope enabled. 136 | #export GITHUB_COM_TOKEN='TODO-YOUR-TOKEN' 137 | # let renovate create all the required pull requests. 138 | # see https://docs.renovatebot.com/configuration-options/#prhourlylimit 139 | # see https://docs.renovatebot.com/configuration-options/#prconcurrentlimit 140 | export RENOVATE_PR_HOURLY_LIMIT='0' 141 | export RENOVATE_PR_CONCURRENT_LIMIT='0' 142 | echo 'Running renovate...' 143 | # NB to capture the traffic using mitmproxy, start mitmweb in a different 144 | # shell, then enable the following if (i.e. true). 145 | docker_extra_args=() 146 | if false; then 147 | docker_extra_args+=( 148 | --env http_proxy=http://127.0.0.1:8080 149 | --env https_proxy=http://127.0.0.1:8080 150 | --env no_proxy= 151 | --env SSL_CERT_FILE=/usr/local/shared/ca-certificates/mitmproxy-ca.crt 152 | --volume "$HOME/.mitmproxy/mitmproxy-ca-cert.pem:/usr/local/shared/ca-certificates/mitmproxy-ca.crt:ro" 153 | ) 154 | fi 155 | # NB use --dry-run=lookup for not modifying the repository (e.g. for not 156 | # creating pull requests). 157 | docker run \ 158 | --rm \ 159 | --tty \ 160 | --interactive \ 161 | --net host \ 162 | --env GITHUB_COM_TOKEN \ 163 | --env RENOVATE_ENDPOINT \ 164 | --env RENOVATE_TOKEN \ 165 | --env RENOVATE_REPOSITORIES \ 166 | --env RENOVATE_PR_HOURLY_LIMIT \ 167 | --env RENOVATE_PR_CONCURRENT_LIMIT \ 168 | --env LOG_LEVEL=debug \ 169 | --env LOG_FORMAT=json \ 170 | "${docker_extra_args[@]}" \ 171 | "renovate/renovate:$renovate_version" \ 172 | --platform=gitea \ 173 | --git-url=endpoint \ 174 | >tmp/renovate-log.json 175 | 176 | echo 'Getting results...' 177 | # extract the errors. 178 | jq 'select(.err)' tmp/renovate-log.json >tmp/renovate-errors.json 179 | # extract the result from the renovate log. 180 | jq 'select(.msg == "packageFiles with updates") | .config' tmp/renovate-log.json >tmp/renovate-result.json 181 | # extract all the dependencies. 182 | jq 'to_entries[].value[] | {packageFile,dep:.deps[]}' tmp/renovate-result.json >tmp/renovate-dependencies.json 183 | # extract the dependencies that have updates. 184 | jq 'select((.dep.updates | length) > 0)' tmp/renovate-dependencies.json >tmp/renovate-dependencies-updates.json 185 | 186 | # helpers. 187 | function show-title { 188 | echo 189 | echo '#' 190 | echo "# $1" 191 | echo '#' 192 | echo 193 | } 194 | 195 | # show errors. 196 | if [ "$(jq --slurp length tmp/renovate-errors.json)" -ne '0' ]; then 197 | show-title errors 198 | jq . tmp/renovate-errors.json 199 | fi 200 | 201 | # show dependencies. 202 | function show-dependencies { 203 | show-title "$1" 204 | ( 205 | printf 'packageFile\tdatasource\tdepName\tcurrentValue\tnewVersions\tskipReason\twarnings\n' 206 | jq \ 207 | -r \ 208 | '[ 209 | .packageFile, 210 | .dep.datasource, 211 | .dep.depName, 212 | .dep.currentValue, 213 | (.dep | select(.updates) | .updates | map(.newVersion) | join(" | ")), 214 | .dep.skipReason, 215 | (.dep | select(.warnings) | .warnings | map(.message) | join(" | ")) 216 | ] | @tsv' \ 217 | "$2" \ 218 | | sort 219 | ) | column -t -s "$(printf \\t)" 220 | } 221 | show-dependencies 'Dependencies' tmp/renovate-dependencies.json 222 | show-dependencies 'Dependencies Updates' tmp/renovate-dependencies-updates.json 223 | 224 | # show the gitea project. 225 | show-title "See PRs at $gitea_url/$RENOVATE_USERNAME/test/pulls (you can login as $RENOVATE_USERNAME:$RENOVATE_PASSWORD)" 226 | -------------------------------------------------------------------------------- /set-example-groups-users.py: -------------------------------------------------------------------------------- 1 | import gitlab 2 | 3 | # this script will make sure the following usernames have the given role in all root groups (it ignores sub-groups). 4 | # NB usernames are case sensitive. 5 | users_role = { 6 | 'alice.doe': 'Owner', 7 | 'bob.doe': 'Maintainer', 8 | 'carol.doe': 'Developer', 9 | 'dave.doe': 'Reporter', 10 | 'eve.doe': 'Guest', 11 | } 12 | 13 | access_levels = { 14 | 'Guest': gitlab.const.AccessLevel.GUEST, 15 | 'Reporter': gitlab.const.AccessLevel.REPORTER, 16 | 'Developer': gitlab.const.AccessLevel.DEVELOPER, 17 | 'Maintainer': gitlab.const.AccessLevel.MAINTAINER, 18 | 'Owner': gitlab.const.AccessLevel.OWNER, 19 | } 20 | access_level_names = {v: k for k, v in access_levels.items()} 21 | 22 | if True: 23 | # manually configure from a user token. 24 | import requests 25 | session = requests.Session() 26 | session.verify = '/vagrant/tmp/gitlab.example.com-crt.pem' # or just export the CA_BUNDLE environment variable. 27 | private_token = '' 28 | with open('/vagrant/tmp/gitlab-root-personal-access-token.txt', 'r') as f: 29 | private_token = f.read().strip() 30 | gl = gitlab.Gitlab('https://gitlab.example.com', session=session, private_token=private_token) 31 | else: 32 | # automatically configure from configuration file (~/.python-gitlab.cfg or /etc/python-gitlab.cfg). 33 | gl = gitlab.Gitlab.from_config() 34 | 35 | for group in gl.groups.list(all=True): 36 | print(f'group {group.full_path}') 37 | if '/' in group.full_path: 38 | # ignore sub-groups. 39 | continue 40 | missing_users = set(users_role.keys()) 41 | for member in group.members.list(all=True): 42 | username = member.username 43 | user_role = users_role.get(username, None) 44 | if not user_role: 45 | continue 46 | missing_users.remove(username) 47 | expected_access_level = access_levels[users_role[username]] 48 | if member.access_level == expected_access_level: 49 | continue 50 | print(f'{group.full_path} group: updating user {username} role from {access_level_names[member.access_level]} to {user_role}...') 51 | member.access_level = expected_access_level 52 | member.save() 53 | for username in missing_users: 54 | user_role = users_role[username] 55 | access_level = access_levels[user_role] 56 | user = gl.users.list(username=username)[0] 57 | print(f'{group.full_path} group: adding user {username} as {user_role}...') 58 | group.members.create({ 59 | 'user_id': user.id, 60 | 'access_level': access_level, 61 | }) 62 | -------------------------------------------------------------------------------- /summary.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -euxo pipefail 3 | 4 | domain=$(hostname --fqdn) 5 | 6 | # see the gitlab services status. 7 | while ! gitlab-ctl status; do sleep 5; done 8 | 9 | # show the gitlab environment info. 10 | gitlab-rake gitlab:env:info 11 | 12 | # show software versions. 13 | dpkg-query --showformat '${Package} ${Version}\n' --show gitlab-ce 14 | /opt/gitlab/embedded/bin/git --version 15 | /opt/gitlab/embedded/bin/ruby -v 16 | gitlab-rails --version 17 | gitlab-psql --version 18 | /opt/gitlab/embedded/bin/redis-server --version 19 | /opt/gitlab/embedded/sbin/nginx -v 20 | 21 | # list projects using the gitlab-cli tool. 22 | echo 'GitLab projects:' 23 | gitlab -o yaml -f id,web_url project list --get-all 24 | 25 | # show GitLab address and root credentials. 26 | echo "GitLab is running at https://$domain" 27 | echo 'Sign in with the root user and the HeyH0Password password' 28 | --------------------------------------------------------------------------------