├── vars ├── main.yml ├── unsupported.yml ├── os_Rocky_8.yml ├── os_Rocky_9.yml ├── os_SLES_12.yml ├── os_SLES_15.yml ├── os_Ubuntu_22.yml ├── os_Ubuntu_20.yml ├── os_Ubuntu_18.yml ├── os_RedHat_7.yml ├── os_CentOS_7.yml ├── os_Ubuntu_16.yml ├── os_CentOS_8.yml └── os_RedHat_8.yml ├── tasks ├── ece-bootstrap │ ├── primary │ │ ├── main.yml │ │ └── install_stack.yml │ ├── secondary │ │ ├── main.yml │ │ └── install_stack.yml │ ├── upgrade.yml │ └── main.yml ├── base │ ├── SLES-12 │ │ ├── main.yml │ │ └── install_docker.yml │ ├── SLES-15 │ │ ├── main.yml │ │ └── install_docker.yml │ ├── Ubuntu-16 │ │ ├── main.yml │ │ ├── install_dependencies.yml │ │ └── install_docker.yml │ ├── Ubuntu-18 │ │ ├── main.yml │ │ ├── install_dependencies.yml │ │ └── install_docker.yml │ ├── Ubuntu-20 │ │ ├── main.yml │ │ ├── install_dependencies.yml │ │ └── install_docker.yml │ ├── RedHat-7 │ │ ├── install_dependencies.yml │ │ ├── main.yml │ │ └── install_docker.yml │ ├── CentOS-7 │ │ ├── install_dependencies.yml │ │ ├── main.yml │ │ └── install_docker.yml │ ├── RedHat-8 │ │ ├── install_dependencies.yml │ │ ├── main.yml │ │ └── install_docker.yml │ ├── Ubuntu-22 │ │ ├── main.yml │ │ ├── install_dependencies.yml │ │ ├── check_pre_requisites.yml │ │ └── install_docker.yml │ ├── Rocky-8 │ │ ├── install_dependencies.yml │ │ ├── main.yml │ │ └── install_podman.yml │ ├── CentOS-8 │ │ ├── main.yml │ │ ├── install_dependencies.yml │ │ └── install_docker.yml │ ├── general │ │ ├── kernel_modules.yml │ │ ├── update_grub_docker.yml │ │ ├── dependencies.yml │ │ ├── setup_mount_permissions.yml │ │ ├── sysctl_scripts.yml │ │ ├── set_limits.yml │ │ ├── configure_docker.yml │ │ ├── configure_podman.yml │ │ └── make_user.yml │ ├── Rocky-9 │ │ ├── install_dependencies.yml │ │ ├── main.yml │ │ └── install_podman.yml │ └── main.yml ├── main.yml ├── direct-install │ ├── main.yml │ └── setup_xfs.yml ├── vmimage │ └── main.yml └── diagnostics │ └── main.yml ├── .gitignore ├── templates ├── elastic.cfg.j2 ├── podman.conf ├── docker.conf └── format-drives.j2 ├── meta └── main.yml ├── LICENSE ├── defaults └── main.yml ├── CONTRIBUTING.md └── README.md /vars/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | -------------------------------------------------------------------------------- /vars/unsupported.yml: -------------------------------------------------------------------------------- 1 | --- 2 | unsupported_version: True -------------------------------------------------------------------------------- /tasks/ece-bootstrap/primary/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: install_stack.yml -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | *.retry 3 | *.local.yml 4 | *.local.sh 5 | *.local.json 6 | .vscode 7 | .DS_Store 8 | -------------------------------------------------------------------------------- /tasks/base/SLES-12/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: install_docker.yml 3 | tags: [install_docker, destructive] -------------------------------------------------------------------------------- /tasks/base/SLES-15/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: install_docker.yml 3 | tags: [install_docker, destructive] -------------------------------------------------------------------------------- /templates/elastic.cfg.j2: -------------------------------------------------------------------------------- 1 | #cloud-config 2 | system_info: 3 | default_user: 4 | name: {{ image_user }} 5 | -------------------------------------------------------------------------------- /tasks/base/Ubuntu-16/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: install_dependencies.yml 3 | - include_tasks: install_docker.yml 4 | tags: [install_docker, destructive] -------------------------------------------------------------------------------- /tasks/base/Ubuntu-18/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: install_dependencies.yml 3 | - include_tasks: install_docker.yml 4 | tags: [install_docker, destructive] -------------------------------------------------------------------------------- /tasks/base/Ubuntu-20/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: install_dependencies.yml 3 | - include_tasks: install_docker.yml 4 | tags: [install_docker, destructive] -------------------------------------------------------------------------------- /tasks/base/RedHat-7/install_dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Install base dependencies 3 | package: 4 | name: "{{ item }}" 5 | state: present 6 | with_items: 7 | - lvm2 -------------------------------------------------------------------------------- /tasks/base/CentOS-7/install_dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Install base dependencies 3 | package: 4 | name: "{{ item }}" 5 | state: present 6 | with_items: 7 | - lvm2 8 | - mdadm 9 | -------------------------------------------------------------------------------- /tasks/base/RedHat-8/install_dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Install base dependencies 3 | package: 4 | name: "{{ item }}" 5 | state: present 6 | with_items: 7 | - lvm2 8 | - iptables 9 | -------------------------------------------------------------------------------- /tasks/base/Ubuntu-22/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: install_dependencies.yml 3 | - include_tasks: check_pre_requisites.yml 4 | - include_tasks: install_docker.yml 5 | tags: [install_docker, destructive] 6 | -------------------------------------------------------------------------------- /templates/podman.conf: -------------------------------------------------------------------------------- 1 | [containers] 2 | userns = "keep-id" 3 | 4 | [engine] 5 | remote = true 6 | active_service = "elastic" 7 | [engine.service_destinations] 8 | [engine.service_destinations.elastic] 9 | uri = "unix:///run/podman/podman.sock" -------------------------------------------------------------------------------- /tasks/base/Rocky-8/install_dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Install base dependencies 3 | package: 4 | name: "{{ item }}" 5 | state: present 6 | with_items: 7 | - lvm2 8 | - iptables 9 | - sysstat 10 | - net-tools 11 | - policycoreutils-python-utils 12 | 13 | -------------------------------------------------------------------------------- /tasks/base/CentOS-8/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Disable firewalld 3 | systemd: 4 | name: firewalld 5 | state: stopped 6 | enabled: no 7 | ignore_errors: true 8 | 9 | - include_tasks: install_dependencies.yml 10 | - include_tasks: install_docker.yml 11 | tags: [install_docker, destructive] -------------------------------------------------------------------------------- /tasks/base/RedHat-7/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Disable firewalld 3 | systemd: 4 | name: firewalld 5 | state: stopped 6 | enabled: no 7 | ignore_errors: true 8 | 9 | - include_tasks: install_dependencies.yml 10 | - include_tasks: install_docker.yml 11 | tags: [install_docker, destructive] -------------------------------------------------------------------------------- /tasks/base/RedHat-8/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Disable firewalld 3 | systemd: 4 | name: firewalld 5 | state: stopped 6 | enabled: no 7 | ignore_errors: true 8 | 9 | - include_tasks: install_dependencies.yml 10 | - include_tasks: install_docker.yml 11 | tags: [install_docker, destructive] -------------------------------------------------------------------------------- /tasks/base/general/kernel_modules.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Enable kernel modules 3 | copy: 4 | content: "{{item.kernel_module}}" 5 | dest: "/etc/modules-load.d/{{item.name}}.conf" 6 | with_items: 7 | - { name: 'conntrack', kernel_module: 'ip_conntrack' } 8 | - { name: 'overlay', kernel_module: 'overlay' } -------------------------------------------------------------------------------- /tasks/base/Rocky-9/install_dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Install base dependencies 3 | package: 4 | name: "{{ item }}" 5 | state: present 6 | with_items: 7 | - lvm2 8 | - iptables 9 | - sysstat 10 | - net-tools 11 | - containernetworking-plugins 12 | - policycoreutils-python-utils 13 | -------------------------------------------------------------------------------- /vars/os_Rocky_8.yml: -------------------------------------------------------------------------------- 1 | --- 2 | bootloader_update_command: grub2-mkconfig -o /etc/grub2.cfg 3 | conntrack_module: ip_conntrack 4 | container_engine: Podman 5 | 6 | # Podman version mapping 7 | podman_version_map: 8 | "4.9.4": 9 | name: "podman" 10 | package: 11 | - podman-4.9.4 12 | - podman-remote-4.9.4 13 | -------------------------------------------------------------------------------- /vars/os_Rocky_9.yml: -------------------------------------------------------------------------------- 1 | --- 2 | bootloader_update_command: grub2-mkconfig -o /etc/grub2.cfg 3 | conntrack_module: ip_conntrack 4 | container_engine: Podman 5 | 6 | # Podman version mapping 7 | podman_version_map: 8 | "4.9.4": 9 | name: "podman" 10 | package: 11 | - podman-4.9.4 12 | - podman-remote-4.9.4 13 | -------------------------------------------------------------------------------- /tasks/base/Ubuntu-18/install_dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Install base dependencies 3 | apt: 4 | name: "{{ packages }}" 5 | update_cache: yes 6 | install_recommends: yes 7 | state: present 8 | register: apt_res 9 | retries: 3 10 | until: apt_res is success 11 | vars: 12 | packages: 13 | - xfsprogs 14 | -------------------------------------------------------------------------------- /vars/os_SLES_12.yml: -------------------------------------------------------------------------------- 1 | --- 2 | docker_unit_after: "network.target docker.socket" 3 | docker_storage_driver: overlay2 4 | bootloader_update_command: update-bootloader 5 | conntrack_module: ip_conntrack 6 | container_engine: Docker 7 | 8 | # Docker version mapping 9 | docker_version_map: 10 | "19.03": 11 | package: docker-19.03.15_ce 12 | -------------------------------------------------------------------------------- /vars/os_SLES_15.yml: -------------------------------------------------------------------------------- 1 | --- 2 | docker_unit_after: "network.target docker.socket" 3 | docker_storage_driver: overlay2 4 | bootloader_update_command: update-bootloader 5 | conntrack_module: ip_conntrack 6 | container_engine: Docker 7 | 8 | # Docker version mapping 9 | docker_version_map: 10 | "20.10": 11 | package: docker-20.10.12_ce 12 | -------------------------------------------------------------------------------- /tasks/base/Ubuntu-20/install_dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Install base dependencies 3 | apt: 4 | name: "{{ packages }}" 5 | update_cache: yes 6 | install_recommends: yes 7 | state: present 8 | register: apt_res 9 | retries: 3 10 | until: apt_res is success 11 | vars: 12 | packages: 13 | - xfsprogs 14 | - acl 15 | -------------------------------------------------------------------------------- /tasks/base/Ubuntu-22/install_dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Install base dependencies 3 | apt: 4 | name: "{{ packages }}" 5 | update_cache: yes 6 | install_recommends: yes 7 | state: present 8 | register: apt_res 9 | retries: 3 10 | until: apt_res is success 11 | vars: 12 | packages: 13 | - xfsprogs 14 | - acl 15 | -------------------------------------------------------------------------------- /tasks/base/Ubuntu-16/install_dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Install base dependencies 3 | apt: 4 | name: "{{ packages }}" 5 | update_cache: yes 6 | install_recommends: yes 7 | state: present 8 | register: apt_res 9 | retries: 3 10 | until: apt_res is success 11 | vars: 12 | packages: 13 | - linux-generic-lts-xenial 14 | - xfsprogs 15 | -------------------------------------------------------------------------------- /tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - import_tasks: base/main.yml 3 | tags: [base] 4 | 5 | - import_tasks: vmimage/main.yml 6 | tags: [never, vmimage] 7 | 8 | - import_tasks: direct-install/main.yml 9 | tags: [ece] 10 | 11 | - import_tasks: ece-bootstrap/main.yml 12 | tags: [ece, bootstrap] 13 | 14 | - import_tasks: diagnostics/main.yml 15 | when: fetch_diagnostics == true 16 | -------------------------------------------------------------------------------- /tasks/base/general/update_grub_docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Modify GRUB_CMDLINE_LINUX 3 | lineinfile: 4 | state: present 5 | dest: /etc/default/grub 6 | backrefs: yes 7 | regexp: '^(GRUB_CMDLINE_LINUX=\"[^\"]*)(\".*)' 8 | line: '\1 cgroup_enable=memory swapaccount=1 cgroup.memory=nokmem\2' 9 | 10 | - name: Run bootloader update 11 | command: "{{ bootloader_update_command }}" -------------------------------------------------------------------------------- /tasks/ece-bootstrap/secondary/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - set_fact: 3 | primary_hostname: "{{ hostvars[groups['primary'][0]].primary_hostname }}" 4 | when: primary_hostname is undefined 5 | 6 | - set_fact: 7 | adminconsole_root_password: "{{ hostvars[groups['primary'][0]].adminconsole_root_password }}" 8 | when: adminconsole_root_password is undefined 9 | 10 | - include_tasks: install_stack.yml -------------------------------------------------------------------------------- /tasks/base/Rocky-8/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Disable firewalld 3 | ansible.builtin.systemd: 4 | name: firewalld 5 | state: stopped 6 | enabled: no 7 | ignore_errors: true 8 | 9 | - ansible.builtin.include_tasks: install_dependencies.yml 10 | - ansible.builtin.include_tasks: install_podman.yml 11 | tags: [install_docker, destructive] 12 | when: container_engine == "Podman" 13 | 14 | 15 | -------------------------------------------------------------------------------- /tasks/base/Rocky-9/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Disable firewalld 3 | ansible.builtin.systemd: 4 | name: firewalld 5 | state: stopped 6 | enabled: no 7 | ignore_errors: true 8 | 9 | - ansible.builtin.include_tasks: install_dependencies.yml 10 | - ansible.builtin.include_tasks: install_podman.yml 11 | tags: [install_docker, destructive] 12 | when: container_engine == "Podman" 13 | 14 | 15 | -------------------------------------------------------------------------------- /tasks/direct-install/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: setup_xfs.yml 3 | tags: [setup_filesystem, destructive] 4 | 5 | - name: Reboot the machine with all defaults 6 | reboot: 7 | msg: "Reboot for changes to take effect initiated by Ansible" 8 | post_reboot_delay: 10 9 | tags: [setup_filesystem] 10 | 11 | - include_tasks: ../base/general/setup_mount_permissions.yml 12 | tags: [setup_filesystem, setup_fs_permissions] 13 | -------------------------------------------------------------------------------- /templates/docker.conf: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Docker Service 3 | After={{ docker_unit_after }} 4 | 5 | [Service] 6 | EnvironmentFile= 7 | Environment="DOCKER_OPTS=-H unix:///var/run/docker.sock --data-root {{ data_dir }}/docker --storage-driver={{ docker_storage_driver }} --bip={{ docker_bridge_ip }} --raw-logs --icc=false" 8 | ExecStart= 9 | ExecStart=/usr/bin/dockerd $DOCKER_OPTS 10 | Restart=on-failure 11 | RestartSec=1s 12 | TimeoutSec=20 13 | -------------------------------------------------------------------------------- /meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | galaxy_info: 3 | author: Vincent Barth 4 | description: Elastic Cloud Enterprise Reference preparation and installation 5 | company: "Elastic.co" 6 | license: "license (Apache)" 7 | min_ansible_version: 2.4 8 | platforms: 9 | - name: Ubuntu 10 | versions: 11 | - 16.04 12 | - 22.04 13 | - name: SLES 14 | versions: 15 | - name: Rocky 16 | versions: 17 | - 8 18 | - 9 19 | 20 | dependencies: [] 21 | -------------------------------------------------------------------------------- /tasks/ece-bootstrap/upgrade.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Execute upgrade 3 | shell: | 4 | /home/elastic/elastic-cloud-enterprise.sh \ 5 | upgrade \ 6 | --cloud-enterprise-version {{ ece_version }} \ 7 | --docker-registry {{ ece_docker_registry }} \ 8 | --ece-docker-repository {{ ece_docker_repository }} \ 9 | --user admin \ 10 | --pass {{ adminconsole_root_password }} 11 | become: yes 12 | become_method: sudo 13 | become_user: elastic 14 | -------------------------------------------------------------------------------- /tasks/base/general/dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # As per the documentation. 3 | - name: Remove nscd 4 | package: 5 | name: "{{ packages }}" 6 | state: absent 7 | vars: 8 | packages: 9 | - nscd 10 | register: remove_packages 11 | retries: 10 12 | delay: 30 13 | until: remove_packages is success 14 | 15 | - name: Install common base dependencies 16 | package: 17 | name: "{{ item }}" 18 | state: present 19 | with_items: 20 | - cloud-init 21 | -------------------------------------------------------------------------------- /tasks/base/Rocky-8/install_podman.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: disable SELinux 3 | selinux: 4 | state: disabled 5 | 6 | - name: Install Podman 7 | package: 8 | name: "{{ podman_version_map[podman_version]['package'] }}" 9 | state: present 10 | loop: 11 | - "{{ podman_version_map[podman_version]['package'] }}" 12 | 13 | - name: Verify that fs.may_detach_mounts is enabled 14 | lineinfile: 15 | path: /etc/sysctl.conf 16 | regexp: '^fs.may_detach_mounts' 17 | line: 'fs.may_detach_mounts = 1' 18 | create: yes 19 | -------------------------------------------------------------------------------- /tasks/base/Rocky-9/install_podman.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: disable SELinux 3 | selinux: 4 | state: disabled 5 | 6 | - name: Install Podman 7 | package: 8 | name: "{{ podman_version_map[podman_version]['package'] }}" 9 | state: present 10 | loop: 11 | - "{{ podman_version_map[podman_version]['package'] }}" 12 | 13 | - name: Verify that fs.may_detach_mounts is enabled 14 | lineinfile: 15 | path: /etc/sysctl.conf 16 | regexp: '^fs.may_detach_mounts' 17 | line: 'fs.may_detach_mounts = 1' 18 | create: yes 19 | -------------------------------------------------------------------------------- /vars/os_Ubuntu_22.yml: -------------------------------------------------------------------------------- 1 | --- 2 | docker_unit_after: "multi-user.target" 3 | docker_storage_driver: overlay2 4 | bootloader_update_command: update-grub 5 | conntrack_module: xt_conntrack 6 | container_engine: Docker 7 | 8 | # Docker version mapping 9 | docker_version_map: 10 | "24.0": 11 | package: 12 | - docker-ce=5:24.0.* 13 | - docker-ce-cli=5:24.0.* 14 | - containerd.io 15 | repo: deb https://download.docker.com/linux/ubuntu focal stable 16 | keys: 17 | server: https://download.docker.com/linux/ubuntu/gpg 18 | id: 0EBFCD88 19 | -------------------------------------------------------------------------------- /vars/os_Ubuntu_20.yml: -------------------------------------------------------------------------------- 1 | --- 2 | docker_unit_after: "multi-user.target" 3 | docker_storage_driver: overlay2 4 | bootloader_update_command: update-grub 5 | conntrack_module: xt_conntrack 6 | container_engine: Docker 7 | 8 | # Docker version mapping 9 | docker_version_map: 10 | "20.10": 11 | package: 12 | - docker-ce=5:20.10.14* 13 | - docker-ce-cli=5:20.10.14* 14 | - containerd.io 15 | repo: deb https://download.docker.com/linux/ubuntu focal stable 16 | keys: 17 | server: https://download.docker.com/linux/ubuntu/gpg 18 | id: 0EBFCD88 19 | -------------------------------------------------------------------------------- /vars/os_Ubuntu_18.yml: -------------------------------------------------------------------------------- 1 | --- 2 | docker_unit_after: "multi-user.target" 3 | docker_storage_driver: overlay2 4 | bootloader_update_command: update-grub 5 | conntrack_module: xt_conntrack 6 | container_engine: Docker 7 | 8 | # Docker version mapping 9 | docker_version_map: 10 | "19.03": 11 | package: 12 | - docker-ce=5:19.03.15* 13 | - docker-ce-cli=5:19.03.15* 14 | - containerd.io=1.4.3-1* 15 | repo: deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable 16 | keys: 17 | server: https://download.docker.com/linux/ubuntu/gpg 18 | id: 0EBFCD88 19 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2019 Elasticsearch 2 | 3 | Licensed under the Apache License, Version 2.0 (the "License"); 4 | you may not use this file except in compliance with the License. 5 | You may obtain a copy of the License at 6 | 7 | http://www.apache.org/licenses/LICENSE-2.0 8 | 9 | Unless required by applicable law or agreed to in writing, software 10 | distributed under the License is distributed on an "AS IS" BASIS, 11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | See the License for the specific language governing permissions and 13 | limitations under the License. -------------------------------------------------------------------------------- /tasks/base/Ubuntu-16/install_docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Add docker repository key 3 | apt_key: 4 | url: "{{ docker_version_map[docker_version]['keys']['server'] }}" 5 | id: "{{ docker_version_map[docker_version]['keys']['id'] }}" 6 | 7 | - name: Add docker repository 8 | apt_repository: 9 | repo: "{{ docker_version_map[docker_version]['repo'] }}" 10 | state: present 11 | 12 | - name: Install docker 13 | apt: 14 | name: "{{ docker_version_map[docker_version]['package'] }}" 15 | update_cache: yes 16 | state: present 17 | 18 | - name: Pin docker-engine packet 19 | shell: echo "docker-engine hold" | sudo dpkg --set-selections 20 | -------------------------------------------------------------------------------- /tasks/base/Ubuntu-18/install_docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Add docker repository key 3 | apt_key: 4 | url: "{{ docker_version_map[docker_version]['keys']['server'] }}" 5 | id: "{{ docker_version_map[docker_version]['keys']['id'] }}" 6 | 7 | - name: Add docker repository 8 | apt_repository: 9 | repo: "{{ docker_version_map[docker_version]['repo'] }}" 10 | state: present 11 | 12 | - name: Install docker 13 | apt: 14 | name: "{{ docker_version_map[docker_version]['package'] }}" 15 | update_cache: yes 16 | state: present 17 | 18 | - name: Pin docker-engine packet 19 | shell: echo "docker-engine hold" | sudo dpkg --set-selections 20 | -------------------------------------------------------------------------------- /tasks/base/Ubuntu-20/install_docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Add docker repository key 3 | apt_key: 4 | url: "{{ docker_version_map[docker_version]['keys']['server'] }}" 5 | id: "{{ docker_version_map[docker_version]['keys']['id'] }}" 6 | 7 | - name: Add docker repository 8 | apt_repository: 9 | repo: "{{ docker_version_map[docker_version]['repo'] }}" 10 | state: present 11 | 12 | - name: Install docker 13 | apt: 14 | name: "{{ docker_version_map[docker_version]['package'] }}" 15 | update_cache: yes 16 | state: present 17 | 18 | - name: Pin docker-engine packet 19 | shell: echo "docker-engine hold" | sudo dpkg --set-selections 20 | -------------------------------------------------------------------------------- /tasks/base/Ubuntu-22/check_pre_requisites.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Check kernel version 3 | vars: 4 | kernel_version: "{{ ansible_kernel.split('-')|first }}" 5 | fail: 6 | msg: '{{ ansible_kernel }} is below the recommendation of 4.15' 7 | when: 8 | - kernel_version is version('4.15', operator='lt', strict=True) 9 | ignore_errors: True 10 | 11 | 12 | - name: Get cgroup version 13 | shell: "grep cgroup /proc/filesystems" 14 | register: cgroup_version 15 | - name: Check cgroup is supported 16 | fail: 17 | msg: '{{cgroup_version}} is not supported, ECE only support cgroup v1' 18 | when: 19 | - "'nodev\tcgroup' not in cgroup_version.stdout_lines" 20 | -------------------------------------------------------------------------------- /tasks/base/general/setup_mount_permissions.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Change owner of {{ data_dir }} 3 | file: 4 | path: "{{ data_dir }}" 5 | owner: elastic 6 | group: elastic 7 | mode: 0700 8 | state: directory 9 | 10 | - name: Change owner and permissions of {{ data_dir }}/elastic 11 | file: 12 | path: "{{ data_dir }}/elastic" 13 | state: directory 14 | owner: elastic 15 | group: elastic 16 | mode: 0700 17 | 18 | - name: Change owner and permissions of {{ data_dir }}/docker 19 | file: 20 | path: "{{ data_dir }}/docker" 21 | state: directory 22 | owner: elastic 23 | group: elastic 24 | mode: 0700 25 | when: container_engine == "Docker" 26 | -------------------------------------------------------------------------------- /tasks/base/SLES-12/install_docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Remove docker and docker-runc 3 | zypper: 4 | name: "{{ packages }}" 5 | state: absent 6 | vars: 7 | packages: 8 | - docker 9 | - docker-runc 10 | register: remove_packages 11 | retries: 10 12 | delay: 30 13 | until: remove_packages is success 14 | 15 | - name: Install docker 16 | command: "zypper install -y --force-resolution --replacefiles {{ docker_version_map[docker_version]['package'] }} acl" 17 | args: 18 | warn: false 19 | 20 | # Workaround for https://github.com/elastic/ansible-elastic-cloud-enterprise/issues/155#issuecomment-1117069430 21 | - name: Uninstall nscd 22 | zypper: 23 | name: nscd 24 | state: absent 25 | -------------------------------------------------------------------------------- /tasks/base/SLES-15/install_docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Remove docker and docker-runc 3 | zypper: 4 | name: "{{ packages }}" 5 | state: absent 6 | vars: 7 | packages: 8 | - docker 9 | - docker-runc 10 | register: remove_packages 11 | retries: 10 12 | delay: 30 13 | until: remove_packages is success 14 | 15 | - name: Install docker 16 | command: "zypper install -y --force-resolution --replacefiles {{ docker_version_map[docker_version]['package'] }} acl" 17 | args: 18 | warn: false 19 | 20 | # Workaround for https://github.com/elastic/ansible-elastic-cloud-enterprise/issues/155#issuecomment-1117069430 21 | - name: Uninstall nscd 22 | zypper: 23 | name: nscd 24 | state: absent 25 | -------------------------------------------------------------------------------- /vars/os_RedHat_7.yml: -------------------------------------------------------------------------------- 1 | --- 2 | docker_unit_after: "multi-user.target" 3 | docker_storage_driver: overlay2 4 | bootloader_update_command: grub2-mkconfig -o /etc/grub2.cfg 5 | conntrack_module: ip_conntrack 6 | container_engine: Docker 7 | 8 | # Docker version mapping 9 | docker_version_map: 10 | "1.13": 11 | name: 'Docker' 12 | package: docker-2:1.13.1-109* 13 | repo: "Red Hat Enterprise Linux*7*Extra*RPMs)" 14 | "20.10": 15 | name: 'Docker-CE' 16 | package: 17 | - docker-ce-20.10.8 18 | - docker-ce-cli-20.10.8 19 | - containerd.io-1.4.3 20 | repo: https://download.docker.com/linux/centos/7/x86_64/stable 21 | keys: 22 | server: https://download.docker.com/linux/centos/gpg 23 | id: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35 24 | -------------------------------------------------------------------------------- /tasks/base/CentOS-7/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Disable firewalld 3 | systemd: 4 | name: firewalld 5 | state: stopped 6 | enabled: no 7 | ignore_errors: true 8 | 9 | - name: ensure dhcp dns is set 10 | lineinfile: 11 | path: /etc/sysconfig/network-scripts/ifcfg-eth0 12 | line: "{{ item }}" 13 | with_items: 14 | - 'PeerDNS=yes' 15 | - 'NM_CONTROLLED=yes' 16 | 17 | - name: set locale 18 | lineinfile: 19 | path: /etc/environment 20 | line: "{{ item }}" 21 | with_items: 22 | - 'LANG=en_US.utf8' 23 | - 'LC_CTYPE=en_US.utf8' 24 | 25 | - name: set path 26 | lineinfile: 27 | path: /etc/profile.d/path.sh 28 | line: "export PATH=$PATH:/usr/sbin" 29 | create: yes 30 | 31 | - include_tasks: install_dependencies.yml 32 | - include_tasks: install_docker.yml 33 | tags: [install_docker, destructive] 34 | -------------------------------------------------------------------------------- /tasks/vmimage/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: stop and disable docker 3 | systemd: 4 | name: docker 5 | state: stopped 6 | enabled: no 7 | 8 | - include_tasks: ../base/general/setup_mount_permissions.yml 9 | 10 | - name: Place per instance disk formater 11 | template: 12 | src: "format-drives.j2" 13 | dest: "/var/lib/cloud/scripts/per-instance/00-format-drives" 14 | mode: 755 15 | vars: 16 | systemd_os: ansible_service_mgr == 'systemd' 17 | image_user: elastic 18 | 19 | # Because disk structure can change, we start docker daemon only after disks has been formated 20 | - name: Place per instance enable docker 21 | copy: 22 | content: | 23 | #!/bin/bash 24 | set -x 25 | systemctl enable docker 26 | systemctl start docker 27 | dest: /var/lib/cloud/scripts/per-instance/01-enable-docker 28 | mode: 755 29 | -------------------------------------------------------------------------------- /tasks/base/CentOS-8/install_dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: fixing Centos Appstream (https://github.com/elastic/ansible-elastic-cloud-enterprise/issues/153) 3 | block: 4 | - name: Find all of the files inside this directory 5 | find: 6 | paths: "/etc/yum.repos.d/" 7 | patterns: "CentOS-*.repo" 8 | register: repos 9 | - replace: 10 | path: "{{ item.path }}" 11 | regexp: 'mirrorlist\=' 12 | replace: '#mirrorlist=' 13 | with_items: "{{ repos.files }}" 14 | - replace: 15 | path: "{{ item.path }}" 16 | regexp: '#baseurl=http://mirror\.centos\.org' 17 | replace: 'baseurl=http://mirror.centos.org' 18 | with_items: "{{ repos.files }}" 19 | 20 | - name: Install base dependencies 21 | yum: 22 | name: 23 | - lvm2 24 | - mdadm 25 | - iptables 26 | state: present 27 | update_cache: yes 28 | -------------------------------------------------------------------------------- /vars/os_CentOS_7.yml: -------------------------------------------------------------------------------- 1 | --- 2 | docker_unit_after: "network.target docker.socket" 3 | docker_storage_driver: overlay2 4 | bootloader_update_command: grub2-mkconfig -o /etc/grub2.cfg 5 | conntrack_module: ip_conntrack 6 | container_engine: Docker 7 | 8 | # Docker version mapping 9 | docker_version_map: 10 | "18.09": 11 | name: 'Docker-CE' 12 | package: 13 | - docker-ce-18.09.9 14 | - docker-ce-cli-18.09.9 15 | - containerd.io-1.4.3 16 | repo: https://download.docker.com/linux/centos/7/x86_64/stable 17 | keys: 18 | server: https://download.docker.com/linux/centos/gpg 19 | "20.10": 20 | name: 'Docker-CE' 21 | package: 22 | - docker-ce-20.10.8 23 | - docker-ce-cli-20.10.8 24 | - containerd.io-1.4.3 25 | repo: https://download.docker.com/linux/centos/7/x86_64/stable 26 | keys: 27 | server: https://download.docker.com/linux/centos/gpg 28 | -------------------------------------------------------------------------------- /vars/os_Ubuntu_16.yml: -------------------------------------------------------------------------------- 1 | --- 2 | docker_unit_after: "multi-user.target" 3 | docker_storage_driver: overlay2 4 | bootloader_update_command: update-grub 5 | conntrack_module: ip_conntrack 6 | container_engine: Docker 7 | 8 | # Docker version mapping 9 | docker_version_map: 10 | "18.09": 11 | package: 12 | - docker-ce=5:18.09.9* 13 | - docker-ce-cli=5:18.09.9* 14 | - containerd.io=1.4.3-1* 15 | repo: deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable 16 | keys: 17 | server: https://download.docker.com/linux/ubuntu/gpg 18 | id: 0EBFCD88 19 | "19.03": 20 | package: 21 | - docker-ce=5:19.03.15* 22 | - docker-ce-cli=5:19.03.15* 23 | - containerd.io=1.4.3-1* 24 | repo: deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable 25 | keys: 26 | server: https://download.docker.com/linux/ubuntu/gpg 27 | id: 0EBFCD88 28 | -------------------------------------------------------------------------------- /tasks/base/CentOS-7/install_docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Remove docker 3 | package: 4 | name: "{{ packages }}" 5 | state: absent 6 | vars: 7 | packages: 8 | - docker 9 | register: remove_packages 10 | retries: 10 11 | delay: 30 12 | until: remove_packages is success 13 | 14 | - name: Add Docker GPG Key 15 | rpm_key: 16 | key: "{{ docker_version_map[docker_version]['keys']['server']}}" 17 | state: present 18 | 19 | - name: Add docker repository 20 | yum_repository: 21 | name: "{{ docker_version_map[docker_version]['name'] }}" 22 | description: "Docker repository" 23 | file: docker-ce 24 | baseurl: "{{ docker_version_map[docker_version]['repo'] }}" 25 | enabled: yes 26 | gpgcheck: no 27 | register: repo_installed 28 | retries: 10 29 | delay: 30 30 | until: repo_installed is success 31 | 32 | - name: Install docker 33 | package: 34 | name: "{{ docker_version_map[docker_version]['package'] }}" 35 | state: present 36 | -------------------------------------------------------------------------------- /tasks/diagnostics/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Create path /tmp/elastic 3 | local_action: 4 | module: file 5 | path: /tmp/elastic/ece-support-diagnostics 6 | state: directory 7 | mode: 0755 8 | 9 | - name: Download ece support diagnostics 10 | local_action: 11 | module: get_url 12 | url: "{{ ece_supportdiagnostics_url }}" 13 | dest: /tmp/elastic/ece-support-diagnostics.tar.gz 14 | mode: 0755 15 | 16 | - name: Unzip downloaded ece support diagnostics 17 | local_action: 18 | module: unarchive 19 | src: /tmp/elastic/ece-support-diagnostics.tar.gz 20 | dest: /tmp/elastic/ 21 | 22 | - name: Run ece support diagnostics 23 | script: /tmp/elastic/ece-support-diagnostics-1.1/diagnostics.sh -s -d 24 | 25 | - name: Download diagnostic bundles to ansible host and save under {{ ece_supportdiagnostics_result_path }} 26 | fetch: 27 | src: "/tmp/ece_diag-{{ ansible_eth0.ipv4.address }}-.tar.gz" 28 | dest: "{{ ece_supportdiagnostics_result_path }}" 29 | -------------------------------------------------------------------------------- /vars/os_CentOS_8.yml: -------------------------------------------------------------------------------- 1 | --- 2 | docker_unit_after: "multi-user.target" 3 | docker_storage_driver: overlay2 4 | bootloader_update_command: grub2-mkconfig -o /etc/grub2.cfg 5 | conntrack_module: ip_conntrack 6 | container_engine: Docker 7 | 8 | # Docker version mapping 9 | docker_version_map: 10 | "19.03": 11 | name: 'Docker-CE' 12 | package: 13 | - docker-ce-19.03.15 14 | - docker-ce-cli-19.03.15 15 | - containerd.io-1.4.3 16 | repo: https://download.docker.com/linux/centos/8/x86_64/stable 17 | keys: 18 | server: https://download.docker.com/linux/centos/gpg 19 | id: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35 20 | "20.10": 21 | name: 'Docker-CE' 22 | package: 23 | - docker-ce-20.10.8 24 | - docker-ce-cli-20.10.8 25 | - containerd.io-1.4.3 26 | repo: https://download.docker.com/linux/centos/8/x86_64/stable 27 | keys: 28 | server: https://download.docker.com/linux/centos/gpg 29 | id: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35 30 | -------------------------------------------------------------------------------- /vars/os_RedHat_8.yml: -------------------------------------------------------------------------------- 1 | --- 2 | docker_unit_after: "multi-user.target" 3 | docker_storage_driver: overlay2 4 | bootloader_update_command: grub2-mkconfig -o /etc/grub2.cfg 5 | conntrack_module: ip_conntrack 6 | container_engine: Docker 7 | 8 | # Docker version mapping 9 | docker_version_map: 10 | "19.03": 11 | name: 'Docker-CE' 12 | package: 13 | - docker-ce-19.03.13 14 | - docker-ce-cli-19.03.13 15 | - containerd.io-1.5.11 16 | repo: https://download.docker.com/linux/centos/docker-ce.repo 17 | keys: 18 | server: https://download.docker.com/linux/centos/gpg 19 | id: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35 20 | "20.10": 21 | name: 'Docker-CE' 22 | package: 23 | - docker-ce-20.10.8 24 | - docker-ce-cli-20.10.8 25 | - containerd.io-1.5.11 26 | repo: https://download.docker.com/linux/centos/docker-ce.repo 27 | keys: 28 | server: https://download.docker.com/linux/centos/gpg 29 | id: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35 30 | -------------------------------------------------------------------------------- /tasks/base/RedHat-8/install_docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Remove docker 3 | package: 4 | name: "{{ packages }}" 5 | state: absent 6 | vars: 7 | packages: 8 | - docker 9 | register: remove_packages 10 | retries: 10 11 | delay: 30 12 | until: remove_packages is success 13 | 14 | - name: disable SELinux 15 | selinux: 16 | state: disabled 17 | 18 | - name: Add Docker GPG Key 19 | rpm_key: 20 | key: "{{ docker_version_map[docker_version]['keys']['server'] }}" 21 | state: present 22 | 23 | - name: Add RHEL8 Extras repository 24 | shell: dnf config-manager --add-repo="{{ docker_version_map[docker_version]['repo'] }}" 25 | register: repo_installed 26 | retries: 10 27 | delay: 30 28 | until: repo_installed is success 29 | 30 | - name: Install docker 31 | package: 32 | name: "{{ docker_version_map[docker_version]['package'] }}" 33 | state: present 34 | 35 | - name: Verify that fs.may_detach_mounts is enabled 36 | lineinfile: 37 | path: /etc/sysctl.conf 38 | regexp: '^fs.may_detach_mounts' 39 | line: 'fs.may_detach_mounts = 1' 40 | create: yes 41 | -------------------------------------------------------------------------------- /tasks/base/Ubuntu-22/install_docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # As per the doc 3 | # https://docs.docker.com/engine/install/ubuntu/#uninstall-old-versions 4 | - name: Remove docker 5 | package: 6 | name: "{{ packages }}" 7 | state: absent 8 | vars: 9 | packages: 10 | - docker.io 11 | - docker-doc 12 | - docker-compose 13 | - docker-compose-v2 14 | - podman-docker 15 | - containerd 16 | - runc 17 | register: remove_packages 18 | retries: 10 19 | delay: 30 20 | until: remove_packages is success 21 | - name: Add docker repository key 22 | apt_key: 23 | url: "{{ docker_version_map[docker_version]['keys']['server'] }}" 24 | id: "{{ docker_version_map[docker_version]['keys']['id'] }}" 25 | 26 | - name: Add docker repository 27 | apt_repository: 28 | repo: "{{ docker_version_map[docker_version]['repo'] }}" 29 | state: present 30 | 31 | - name: Install docker 32 | apt: 33 | name: "{{ docker_version_map[docker_version]['package'] }}" 34 | update_cache: yes 35 | state: present 36 | 37 | - name: Pin docker-engine packet 38 | shell: echo "docker-engine hold" | sudo dpkg --set-selections 39 | -------------------------------------------------------------------------------- /tasks/base/general/sysctl_scripts.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: sysctl_scripts.yml || load {{ conntrack_module }} if needed 3 | modprobe: 4 | name: "{{ conntrack_module }}" 5 | state: present 6 | 7 | - name: Create sysctl settings file 8 | file: 9 | path: "{{ sysctl_settings_file }}" 10 | state: touch 11 | 12 | - name: sysctl_scripts.yml || set sysctl items 13 | sysctl: 14 | name: "{{ item.name }}" 15 | value: "{{ item.value }}" 16 | reload: yes 17 | state: present 18 | sysctl_set: yes 19 | sysctl_file: "{{ sysctl_settings_file }}" 20 | with_items: 21 | - { name: 'net.ipv4.tcp_max_syn_backlog', value: '65536' } 22 | - { name: 'net.core.somaxconn', value: '32768' } 23 | - { name: 'net.core.netdev_max_backlog', value: '32768' } 24 | - { name: 'vm.max_map_count', value: '262144' } 25 | - { name: 'vm.swappiness', value: '1' } 26 | - { name: 'net.ipv4.tcp_keepalive_time', value: '1800' } 27 | - { name: 'net.netfilter.nf_conntrack_tcp_timeout_established', value: '7200' } 28 | - { name: 'net.netfilter.nf_conntrack_max', value: '262140' } 29 | - { name: 'net.ipv4.ip_forward', value: '1'} 30 | -------------------------------------------------------------------------------- /tasks/base/CentOS-8/install_docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Remove docker 3 | package: 4 | name: "{{ packages }}" 5 | state: absent 6 | vars: 7 | packages: 8 | - docker 9 | register: remove_packages 10 | retries: 10 11 | delay: 30 12 | until: remove_packages is success 13 | 14 | - name: disable SELinux 15 | selinux: 16 | state: disabled 17 | 18 | - name: Add Docker GPG Key 19 | rpm_key: 20 | key: https://download.docker.com/linux/centos/gpg 21 | state: present 22 | when: docker_version == '19.03' 23 | 24 | - name: Add docker repository 25 | yum_repository: 26 | name: "{{ docker_version_map[docker_version]['name'] }}" 27 | description: "Docker repository" 28 | file: docker-ce 29 | baseurl: "{{ docker_version_map[docker_version]['repo'] }}" 30 | enabled: yes 31 | gpgcheck: no 32 | register: repo_installed 33 | retries: 10 34 | delay: 30 35 | until: repo_installed is success 36 | 37 | - name: Install docker 38 | package: 39 | name: "{{ docker_version_map[docker_version]['package'] }}" 40 | state: present 41 | 42 | - name: Verify that fs.may_detach_mounts is enabled 43 | lineinfile: 44 | path: /etc/sysctl.conf 45 | regexp: '^fs.may_detach_mounts' 46 | line: 'fs.may_detach_mounts = 1' 47 | create: yes 48 | -------------------------------------------------------------------------------- /defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # General Elastic Cloud Enterprise relevant settings 3 | ece_version: 4.0.3 4 | ece_docker_registry: docker.elastic.co 5 | ece_docker_repository: cloud-enterprise 6 | docker_config: "" 7 | podman_config: "templates/podman.conf" 8 | ece_installer_url: "https://download.elastic.co/cloud/elastic-cloud-enterprise.sh" 9 | ece_runner_id: "{{ ansible_default_ipv4.address }}" 10 | 11 | # Overall setup variables (like package versions) 12 | device_name: xvdb 13 | data_dir: /mnt/data 14 | force_xfc: false 15 | 16 | # Misc. variables (like sysctl settings file, etc.) 17 | sysctl_settings_file: "/etc/sysctl.d/70-cloudenterprise.conf" 18 | system_limits_file: "/etc/security/limits.d/70-cloudenterprise.conf" 19 | 20 | # Memory settings 21 | memory: 22 | runner: 1G 23 | allocator: 4G 24 | proxy: 8G 25 | zookeeper: 4G 26 | director: 1G 27 | constructor: 4G 28 | adminconsole: 4G 29 | 30 | # Elastic Cloud Enterprise additional installer arguments 31 | extra_installer_args: "" 32 | 33 | 34 | # Elastic Cloud Enterprise - Support Diagnostics Settings 35 | ece_supportdiagnostics_url: "https://github.com/elastic/ece-support-diagnostics/archive/v1.3.tar.gz" 36 | ece_supportdiagnostics_result_path: "/tmp/ece-support-diagnostics" 37 | fetch_diagnostics: false 38 | 39 | # General settings for docker environment 40 | docker_bridge_ip: "172.17.42.1/16" 41 | 42 | 43 | # User and group id, uncomment if needed to override. 44 | # elastic_uid: 1234 45 | # elastic_gid: 1234 46 | -------------------------------------------------------------------------------- /tasks/base/general/set_limits.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Create system limits settings file 3 | file: 4 | path: "{{ system_limits_file }}" 5 | state: touch 6 | 7 | - name: Modify pam limits 8 | pam_limits: 9 | domain: "{{ item.domain }}" 10 | limit_type: "{{ item.limit_type }}" 11 | limit_item: "{{ item.limit_item }}" 12 | value: "{{ item.value }}" 13 | dest: "{{ system_limits_file }}" 14 | with_items: 15 | - { domain: '*', limit_type: 'soft', limit_item: 'nofile', value: '1024000' } 16 | - { domain: '*', limit_type: 'hard', limit_item: 'nofile', value: '1024000' } 17 | - { domain: '*', limit_type: 'soft', limit_item: 'memlock', value: 'unlimited' } 18 | - { domain: '*', limit_type: 'hard', limit_item: 'memlock', value: 'unlimited' } 19 | - { domain: 'elastic', limit_type: 'soft', limit_item: 'nofile', value: '1024000' } 20 | - { domain: 'elastic', limit_type: 'hard', limit_item: 'nofile', value: '1024000' } 21 | - { domain: 'elastic', limit_type: 'soft', limit_item: 'memlock', value: 'unlimited' } 22 | - { domain: 'elastic', limit_type: 'hard', limit_item: 'memlock', value: 'unlimited' } 23 | - { domain: 'root', limit_type: 'soft', limit_item: 'nofile', value: '1024000' } 24 | - { domain: 'root', limit_type: 'hard', limit_item: 'nofile', value: '1024000' } 25 | - { domain: 'root', limit_type: 'soft', limit_item: 'memlock', value: 'unlimited' } 26 | 27 | - name: Delete /etc/security/limits.d/20-nproc.conf 28 | file: 29 | path: /etc/security/limits.d/20-nproc.conf 30 | state: absent 31 | -------------------------------------------------------------------------------- /tasks/base/general/configure_docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Stop the docker service 3 | systemd: 4 | name: docker 5 | state: stopped 6 | 7 | - name: Ensures /etc/systemd/system/docker.service.d dir exists 8 | file: 9 | path: /etc/systemd/system/docker.service.d 10 | state: directory 11 | when: docker_version != '1.13' 12 | 13 | - name: Create service.d docker.conf 14 | template: 15 | src: docker.conf 16 | dest: /etc/systemd/system/docker.service.d/docker.conf 17 | when: docker_version != '1.13' 18 | 19 | - name: set docker storage options 20 | lineinfile: 21 | path: /etc/sysconfig/docker 22 | regexp: "^OPTIONS='(.*)'" 23 | line: "OPTIONS='-g {{ data_dir }}/docker \\1'" 24 | backrefs: yes 25 | create: yes 26 | when: docker_version == '1.13' 27 | 28 | - name: set docker network options 29 | lineinfile: 30 | path: /etc/sysconfig/docker-network 31 | regexp: '^DOCKER_NETWORK_OPTIONS=' 32 | line: 'DOCKER_NETWORK_OPTIONS="--bip={{ docker_bridge_ip }}"' 33 | create: yes 34 | when: docker_version == '1.13' 35 | 36 | - name: set docker storage driver 37 | lineinfile: 38 | path: /etc/sysconfig/docker-storage-setup 39 | regexp: '^DOCKER_NETWORK_OPTIONS=' 40 | line: 'STORAGE_DRIVER={{ docker_storage_driver }}' 41 | create: yes 42 | when: docker_version == '1.13' 43 | 44 | - name: Remove var/lib/docker 45 | file: 46 | path: /var/lib/docker 47 | state: absent 48 | force: yes 49 | 50 | - name: Docker daemon is enabled and systemd has read all changes 51 | systemd: 52 | name: docker 53 | enabled: yes 54 | daemon_reload: yes 55 | -------------------------------------------------------------------------------- /tasks/direct-install/setup_xfs.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Make sure {{ data_dir }} exists 3 | file: 4 | path: "{{ data_dir }}" 5 | state: directory 6 | 7 | - name: Create the volume 8 | lvg: 9 | vg: lxc 10 | pvs: /dev/{{ device_name }} 11 | force: true 12 | 13 | - name: Define size of swap 14 | shell: SWAP_MAX_SIZE=$(sudo vgdisplay --units M lxc | grep "VG Size" | awk '{ print mem=int(0.07*$3) }'); grep MemTotal /proc/meminfo | awk -v MAXMEM=${SWAP_MAX_SIZE} '{ mem=int($2/(2*1024)); if(mem>MAXMEM) mem=MAXMEM; print mem; }' 15 | register: swap_size 16 | 17 | - name: Create swap volume 18 | lvol: 19 | vg: lxc 20 | lv: swap 21 | size: "{{swap_size.stdout}}m" 22 | force: true 23 | 24 | - name: Create data volume 25 | lvol: 26 | vg: lxc 27 | lv: data 28 | size: 100%FREE 29 | shrink: no 30 | force: true 31 | 32 | - name: Create swap filesystem on /dev/lxc/swap 33 | filesystem: 34 | fstype: swap 35 | dev: /dev/lxc/swap 36 | 37 | - name: Create xfs filesystem on /dev/lxc/data 38 | filesystem: 39 | fstype: xfs 40 | dev: /dev/lxc/data 41 | opts: "-n ftype=1" 42 | 43 | 44 | - name: Mount /dev/lxc/data 45 | mount: 46 | path: "{{ data_dir }}" 47 | state: mounted 48 | src: /dev/lxc/data 49 | fstype: xfs 50 | opts: "defaults,pquota,prjquota,x-systemd.automount" 51 | 52 | - name: Mount /dev/lxc/swap 53 | mount: 54 | src: "/dev/lxc/swap" 55 | name: "swap" 56 | fstype: "swap" 57 | opts: "swap" 58 | dump: "0" 59 | passno: "0" 60 | state: "present" 61 | 62 | - name: Enable all swap devices 63 | command: swapon -a 64 | -------------------------------------------------------------------------------- /tasks/ece-bootstrap/secondary/install_stack.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Set default ece role if not defined 3 | set_fact: 4 | ece_roles: [director, coordinator, proxy, allocator] 5 | when: ece_roles is undefined 6 | 7 | - name: Get the roles token 8 | uri: 9 | url: "https://{{primary_hostname}}:12443/api/v1/platform/configuration/security/enrollment-tokens" 10 | method: POST 11 | user: admin 12 | password: "{{ adminconsole_root_password }}" 13 | force_basic_auth: yes 14 | return_content: yes 15 | validate_certs: no 16 | headers: 17 | Content-Type: "application/json" 18 | body_format: json 19 | body: '{ "persistent": false, "roles": {{ ece_roles }} }' 20 | register: roles_token 21 | 22 | - name: Execute installation 23 | shell: /home/elastic/elastic-cloud-enterprise.sh 24 | --coordinator-host {{ primary_hostname }} 25 | --roles-token '{{ roles_token.json.token }}' 26 | --roles '{{ ece_roles | join(',') }}' 27 | --availability-zone {{ availability_zone }} 28 | --cloud-enterprise-version {{ ece_version }} 29 | --docker-registry {{ ece_docker_registry }} 30 | --ece-docker-repository {{ ece_docker_repository }} 31 | --host-storage-path {{ data_dir }}/elastic 32 | --memory-settings '{{ memory_settings }}' 33 | --runner-id {{ ece_runner_id }}{% if capacity is defined %} 34 | --capacity {{ capacity }}{% endif %} 35 | {% if container_engine == "Podman" %} 36 | --podman 37 | --host-docker-host /run/podman/podman.sock 38 | --force 39 | {% endif %} 40 | {{ extra_installer_args }} 41 | become: yes 42 | become_method: sudo 43 | become_user: elastic 44 | register: installation 45 | -------------------------------------------------------------------------------- /tasks/ece-bootstrap/primary/install_stack.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # This task may run for a long time and may experience SSH timeouts. 3 | # The async and poll attributes below allow for the task to be launched and checked for completion. 4 | # If this behavior is unwanted in your environment, comment out line 25-34. 5 | 6 | - name: Execute the primary installation 7 | ansible.builtin.shell: /home/elastic/elastic-cloud-enterprise.sh 8 | install 9 | --availability-zone {{ availability_zone }} 10 | --cloud-enterprise-version {{ ece_version }} 11 | --docker-registry {{ ece_docker_registry }} 12 | --ece-docker-repository {{ ece_docker_repository }} 13 | --memory-settings '{{ memory_settings }}' 14 | --runner-id {{ ece_runner_id }} 15 | --host-storage-path {{ data_dir }}/elastic 16 | {% if container_engine == "Podman" %} 17 | --podman 18 | --host-docker-host /run/podman/podman.sock 19 | --force 20 | {% endif %} 21 | {{ extra_installer_args }} 22 | become: yes 23 | become_method: sudo 24 | become_user: elastic 25 | async: 960 26 | poll: 0 27 | 28 | - name: Monitor installer progress 29 | ansible.builtin.shell: 30 | cmd: tail -n 5 /mnt/data/elastic/logs/bootstrap-logs/bootstrap.log 31 | until: "'[no.found.util.LogApplicationExit$] Application is exiting {}' in ece_installer_result_text.stdout" 32 | register: ece_installer_result_text 33 | retries: 192 34 | delay: 5 35 | 36 | - name: Remember the bootstrap secrets 37 | command: cat {{ data_dir }}/elastic/bootstrap-state/bootstrap-secrets.json 38 | register: secrets 39 | 40 | - name: Fetch the bootstrap secrets 41 | fetch: 42 | src: "{{ data_dir }}/elastic/bootstrap-state/bootstrap-secrets.json" 43 | dest: bootstrap-secrets.local.json 44 | flat: yes 45 | 46 | - set_fact: 47 | install_secrets: "{{ secrets.stdout|from_json }}" 48 | 49 | - set_fact: 50 | adminconsole_root_password: "{{ install_secrets.adminconsole_root_password }}" 51 | 52 | - set_fact: 53 | primary_hostname: "{{ inventory_hostname }}" 54 | -------------------------------------------------------------------------------- /tasks/base/RedHat-7/install_docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Remove docker 3 | package: 4 | name: "{{ packages }}" 5 | state: absent 6 | vars: 7 | packages: 8 | - docker 9 | register: remove_packages 10 | retries: 10 11 | delay: 30 12 | until: remove_packages is success 13 | 14 | - name: disable SELinux 15 | selinux: 16 | state: disabled 17 | 18 | - name: Add RHEL7 Extras repository 19 | shell: yum-config-manager --enable "{{ docker_version_map[docker_version]['repo'] }}" 20 | register: repo_installed 21 | retries: 10 22 | delay: 30 23 | until: repo_installed is success 24 | 25 | # Fixing https://stackoverflow.com/questions/65878769/cannot-install-docker-in-a-rhel-server 26 | - name: Add Centos 7 Extras repository 27 | yum_repository: 28 | name: "Centos-extras-x86_64" 29 | description: "Centos Extra repository" 30 | file: docker-ce 31 | baseurl: "http://mirror.centos.org/centos/7/extras/x86_64" 32 | enabled: yes 33 | gpgcheck: no 34 | register: repo_installed 35 | retries: 10 36 | delay: 30 37 | until: repo_installed is success 38 | when: docker_version != '1.13' 39 | 40 | - name: Add Docker GPG Key 41 | rpm_key: 42 | key: "{{ docker_version_map[docker_version]['keys']['server'] }}" 43 | state: present 44 | when: docker_version != '1.13' 45 | 46 | - name: Add docker repository 47 | yum_repository: 48 | name: "{{ docker_version_map[docker_version]['name'] }}" 49 | description: "Docker repository" 50 | file: docker-ce 51 | baseurl: "{{ docker_version_map[docker_version]['repo'] }}" 52 | enabled: yes 53 | gpgcheck: no 54 | register: repo_installed 55 | retries: 10 56 | delay: 30 57 | until: repo_installed is success 58 | when: docker_version != '1.13' 59 | 60 | - name: Install docker 61 | package: 62 | name: "{{ docker_version_map[docker_version]['package'] }}" 63 | state: present 64 | 65 | - name: Verify that fs.may_detach_mounts is enabled 66 | lineinfile: 67 | path: /etc/sysctl.conf 68 | regexp: '^fs.may_detach_mounts' 69 | line: 'fs.may_detach_mounts = 1' 70 | create: yes 71 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | ## Adding a new distribution 4 | 5 | Ansible automatically determines on which distribution it is executing a playbook. 6 | 7 | In `tasks/base/main.yml` tasks and variables are dynamically included depending on the distribution of the host they are run on. 8 | 9 | ```yaml 10 | - name: Include OS specific vars 11 | include_vars: "{{ item }}" 12 | with_first_found: 13 | - os_{{ ansible_distribution }}_{{ ansible_distribution_major_version }}.yml 14 | - unsupported.yml 15 | 16 | - name: execute os specific tasks 17 | include_tasks: "{{ item }}" 18 | with_first_found: 19 | - "{{ ansible_distribution }}-{{ ansible_distribution_major_version}}/main.yml" 20 | - unsupported.yml 21 | ``` 22 | This means: 23 | - All distribution specific *variables* go into `vars/os_DISTRIBUTION_MAJORVERSION.yml` (e.g. `os_Ubuntu_16.yml`) 24 | - All distribution specific *tasks* go in `tasks/base/DISTRIBUTION-MAJORVERSION/` 25 | 26 | Distribution specific tasks are executed prior to all general tasks and include e.g. installing specific packages. 27 | 28 | Therefore to add a new distribution the following steps need to be done: 29 | 30 | **1)** Add a file to `vars` with the naming scheme of `os_DISTRIBUTION_MAJORVERSION.yml` 31 | Inside this file at least the following needs to be specified (example from `vars/os_SLES_12.yml`): 32 | 33 | ```yaml 34 | --- 35 | # The following variables are used to populate templates/docker19.03.conf for the sysctl configuration 36 | --- 37 | docker_unit_after: "network.target docker.socket" 38 | docker_storage_driver: overlay 39 | bootloader_update_command: update-bootloader 40 | 41 | # Docker version mapping 42 | docker_version_map: 43 | "19.03": 44 | package: docker-19.03.14_ce 45 | ``` 46 | 47 | See `vars/os_Ubuntu_16.yml` as an example. 48 | 49 | **2)** Add a folder `DISTRIBUTION-MAJORVERSION` to `tasks/base/` 50 | 51 | This folder must at least contain a file `main.yml`. Normally `main.yml` only includes playbooks which then contain the specific tasks. 52 | The specific tasks must include installing docker and other required packages (see [tasks/base/Ubuntu-16](tasks/base/Ubuntu-16)). 53 | -------------------------------------------------------------------------------- /tasks/base/general/configure_podman.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Stop the Podman service 3 | systemd: 4 | name: podman 5 | state: stopped 6 | 7 | # Not strictly needed, allows containers to run after user logout in some situations. 8 | - name: Enable lingering for Elastic user 9 | ansible.builtin.shell: loginctl enable-linger elastic 10 | 11 | # Default socket is created with restrictive permissions 12 | - name: Remove Podman Socket 13 | ansible.builtin.file: 14 | path: /run/podman 15 | state: absent 16 | force: true 17 | 18 | - name: Ensure /etc/tmpfiles.d/podman.conf exists 19 | ansible.builtin.file: 20 | path: /etc/tmpfiles.d/podman.conf 21 | state: touch 22 | 23 | # Allows non-root user to interact with root socket 24 | - name: Set Podman Socket Dir Permissions 25 | ansible.builtin.lineinfile: 26 | path: /etc/tmpfiles.d/podman.conf 27 | line: d /run/podman 0777 podman podman - 28 | 29 | # Allows non-root user to interact with root socket 30 | - name: Set Podman Socket Mode 31 | ansible.builtin.lineinfile: 32 | path: /usr/lib/systemd/system/podman.socket 33 | line: SocketMode=0770 34 | regexp: ^SocketMode.* 35 | 36 | # Allows non-root user to interact with root socket 37 | - name: Set Podman Socket ACL 38 | ansible.builtin.lineinfile: 39 | path: /usr/lib/systemd/system/podman.socket 40 | line: "{{ item }}" 41 | insertafter: ^SocketMode=.* 42 | loop: 43 | - SocketGroup=podman 44 | - DirectoryMode=0777 45 | 46 | # Not strictly needed, included out of caution for some configurations. 47 | # Containers spawned should retain elastic's uid 48 | - name: Add subuid and subgid to elastic user 49 | ansible.builtin.lineinfile: 50 | path: "{{ item.path }}" 51 | line: "{{ item.line }}" 52 | regexp: ^elastic\:.* 53 | loop: 54 | - path: /etc/subuid 55 | line: elastic:100000:65536 56 | - path: /etc/subgid 57 | line: elastic:100000:65536 58 | 59 | - name: Podman daemon is enabled and systemd has read all changes 60 | systemd: 61 | name: "{{ item }}" 62 | enabled: yes 63 | daemon_reload: yes 64 | loop: 65 | - podman.service 66 | - podman.socket -------------------------------------------------------------------------------- /tasks/base/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Include OS specific vars 3 | include_vars: "{{ item }}" 4 | with_first_found: 5 | - os_{{ ansible_distribution }}_{{ ansible_distribution_major_version }}.yml 6 | - unsupported.yml 7 | 8 | - name: Check that OS is supported 9 | fail: 10 | msg: "ERROR: OS {{ ansible_distribution }} {{ ansible_distribution_major_version}} is not supported!" 11 | when: unsupported_version is defined and unsupported_version 12 | 13 | - name: Set Docker version 14 | set_fact: 15 | docker_version: "{{ docker_version | default(docker_version_map.keys()|list|last) }}" 16 | when: container_engine == "Docker" 17 | 18 | - name: Set Podman version 19 | set_fact: 20 | podman_version: "{{ podman_version | default(podman_version_map.keys()|list|last) }}" 21 | when: container_engine == "Podman" 22 | 23 | - name: Assert Docker version is supported 24 | assert: 25 | that: "docker_version in docker_version_map.keys()" 26 | msg: "Docker version must be one of {{ docker_version_map.keys() }}" 27 | when: container_engine == "Docker" 28 | 29 | - name: Assert Podman version is supported 30 | assert: 31 | that: "podman_version in podman_version_map.keys()" 32 | msg: "Podman version must be one of {{ podman_version_map.keys() }}" 33 | when: container_engine == "Podman" 34 | 35 | - name: execute os specific tasks 36 | include_tasks: "{{ ansible_distribution }}-{{ ansible_distribution_major_version}}/main.yml" 37 | 38 | - include_tasks: general/dependencies.yml 39 | 40 | - include_tasks: general/make_user.yml 41 | - include_tasks: general/set_limits.yml 42 | tags: [setup_filesystem, destructive] 43 | when: ansible_lvm['vgs']['lxc'] is not defined or force_xfc == true 44 | - include_tasks: general/update_grub_docker.yml 45 | tags: [setup_filesystem, destructive] 46 | - include_tasks: general/configure_docker.yml 47 | tags: [install_docker, destructive] 48 | when: container_engine == "Docker" 49 | - include_tasks: general/configure_podman.yml 50 | tags: [install_podman, destructive] 51 | when: container_engine == "Podman" 52 | - include_tasks: general/sysctl_scripts.yml 53 | - include_tasks: general/kernel_modules.yml 54 | 55 | - name: skip automatic ephemeral mount 56 | copy: 57 | dest: /etc/cloud/cloud.cfg.d/01-mounts.cfg 58 | content: | 59 | mounts: 60 | - [ ephemeral0, null ] 61 | -------------------------------------------------------------------------------- /tasks/base/general/make_user.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Determine Available groups 3 | getent: 4 | database: group 5 | 6 | - name: Is there an elastic User 7 | getent: 8 | key: "elastic" 9 | database: passwd 10 | fail_key: false 11 | 12 | # Needed when elastic user created manually and used for Ansible 13 | - name: Set elastic_uid from ansible_user 14 | ansible.builtin.set_fact: 15 | elastic_uid: "{{ getent_passwd[ansible_user].1 }}" 16 | cacheable: true 17 | when: ansible_user == "elastic" 18 | 19 | # Needed when elastic user created manually and used for Ansible 20 | - name: Set elastic_gid from ansible_user 21 | ansible.builtin.set_fact: 22 | elastic_gid: "{{ getent_passwd[ansible_user].2 }}" 23 | cacheable: true 24 | when: ansible_user == "elastic" 25 | 26 | - name: Add group elastic 27 | group: 28 | name: elastic 29 | state: present 30 | gid: "{{ elastic_gid | default(1234) }}" 31 | when: "'elastic' not in ansible_facts.getent_group" 32 | 33 | - name: Add group {{ container_engine }} 34 | group: 35 | name: "{{ container_engine | lower }}" 36 | state: present 37 | when: container_engine | lower not in ansible_facts.getent_group 38 | 39 | - name: Add user elastic to {{ container_engine }} group 40 | user: 41 | name: elastic 42 | group: elastic 43 | groups: "{{ container_engine | lower }}" 44 | uid: "{{ elastic_uid | default(1234) }}" 45 | append: yes 46 | state: present 47 | generate_ssh_key: true 48 | when: getent_passwd["elastic"] == none 49 | 50 | - name: Modify user elastic to be included in groups 51 | user: 52 | name: elastic 53 | groups: "{{ container_engine | lower }}" 54 | append: yes 55 | when: getent_passwd["elastic"] != none 56 | 57 | - name: Check for user elastic's authorized_keys file 58 | stat: 59 | path: ~elastic/.ssh/authorized_keys 60 | register: es_authorized_keys 61 | 62 | # If elastic_authorized_keys_file is not set, copy the authorized keys from default ansible user 63 | - name: Copy keys from default user to elastic user 64 | copy: 65 | src: "~{{ ansible_ssh_user }}/.ssh/authorized_keys" 66 | dest: ~elastic/.ssh/ 67 | remote_src: yes 68 | when: es_authorized_keys.stat.exists == false and elastic_authorized_keys_file is not defined 69 | 70 | # If elastic_authorized_keys_file is defined, use that (local) path to copy the keys from 71 | - name: Copy local keys to elastic user 72 | copy: 73 | src: "{{elastic_authorized_keys_file}}" 74 | dest: ~elastic/.ssh/ 75 | when: elastic_authorized_keys_file is defined 76 | 77 | - name: Set pwd policy 78 | lineinfile: 79 | path: /etc/sudoers.d/99-ece-users 80 | line: 'elastic ALL=(ALL) NOPASSWD:ALL' 81 | state: present 82 | create: true 83 | 84 | - name: set boot user 85 | template: 86 | src: elastic.cfg.j2 87 | dest: /etc/cloud/cloud.cfg.d/00-elastic.cfg 88 | vars: 89 | image_user: elastic -------------------------------------------------------------------------------- /tasks/ece-bootstrap/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Copy ece installer 3 | copy: 4 | src: "{{ ece_installer_path }}" 5 | dest: /home/elastic/elastic-cloud-enterprise.sh 6 | mode: 0755 7 | when: ece_installer_path is defined 8 | 9 | - name: Download ece installer 10 | get_url: 11 | url: "{{ ece_installer_url }}" 12 | dest: /home/elastic/elastic-cloud-enterprise.sh 13 | mode: 0755 14 | when: ece_installer_path is not defined 15 | 16 | - name: Ensure ~/.docker is present 17 | file: 18 | path: /home/elastic/.docker/ 19 | state: directory 20 | owner: elastic 21 | when: container_engine == "Docker" 22 | 23 | - name: Ensure ~/.config/containers/containers.conf is present 24 | file: 25 | path: /home/elastic/.config/containers/ 26 | state: directory 27 | owner: elastic 28 | when: container_engine == "Podman" 29 | 30 | - name: Copy local Docker config 31 | copy: 32 | src: "{{ docker_config }}" 33 | dest: /home/elastic/.docker/config.json 34 | owner: elastic 35 | when: docker_config != "" and container_engine == "Docker" 36 | 37 | - name: Copy local Podman config 38 | copy: 39 | src: "{{ podman_config }}" 40 | dest: /home/elastic/.config/containers/containers.conf 41 | owner: elastic 42 | when: podman_config != "" and container_engine == "Podman" 43 | 44 | - name: Ensure the Docker service is started 45 | systemd: 46 | name: docker 47 | state: started 48 | when: container_engine == "Docker" 49 | 50 | - name: Ensure the Podman service is started 51 | systemd: 52 | name: podman 53 | state: started 54 | when: container_engine == "Podman" 55 | 56 | - name: Check if an installation or upgrade should be performed 57 | shell: '{{ container_engine | lower }} ps -a -f name=frc-runners-runner --format {%raw%}"{{.Image}}"{%endraw%}' 58 | register: existing_runner 59 | # become: true 60 | # become_user: root 61 | 62 | - name: Create memory settings 63 | set_fact: 64 | memory_settings: ' {"runner":{"xms":"{{memory.runner}}","xmx":"{{memory.runner}}"},"proxy":{"xms":"{{memory.proxy}}","xmx":"{{memory.proxy}}"},"zookeeper":{"xms":"{{memory.zookeeper}}","xmx":"{{memory.zookeeper}}"},"director":{"xms":"{{memory.director}}","xmx":"{{memory.director}}"},"constructor":{"xms":"{{memory.constructor}}","xmx":"{{memory.constructor}}"},"admin-console":{"xms":"{{memory.adminconsole}}","xmx":"{{memory.adminconsole}}"}}' 65 | 66 | - name: Install Elastic Cloud Enterprise 67 | block: 68 | - include_tasks: primary/main.yml 69 | when: ece_primary is defined and ece_primary 70 | 71 | - include_tasks: secondary/main.yml 72 | when: ece_primary is undefined or not ece_primary 73 | 74 | - debug: 75 | msg: "Adminconsole is reachable at: https://{{ primary_hostname }}:12443" 76 | - debug: 77 | msg: "Adminconsole password is: {{ adminconsole_root_password }}" 78 | when: existing_runner.stdout == "" 79 | 80 | - include_tasks: upgrade.yml 81 | when: existing_runner.stdout != "" 82 | -------------------------------------------------------------------------------- /templates/format-drives.j2: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -x 4 | 5 | export IMAGE_USER={{ image_user }} 6 | 7 | TARGET_DISK=/dev/md0 8 | MDADM_CONF=/etc/mdadm/mdadm.conf 9 | LVG=lxc 10 | drives="" 11 | 12 | # We want to detect all available disks (without the already mounted root disk) 13 | # unfortunately we cannot use `lsblk --json` because of the following bug: 14 | # https://bugs.launchpad.net/ubuntu/+source/util-linux/+bug/1811812 15 | all_disks=$(lsblk -d -n --raw -o NAME) 16 | for disk in ${all_disks}; do 17 | echo "Probing ${disk} .." 18 | set +e 19 | device_path="/dev/${disk}" 20 | if [ -b ${device_path} ]; then 21 | echo "Detected target disk: ${device_path}" 22 | has_mounted_partition=$(lsblk --raw -n -o NAME,MOUNTPOINT /dev/$disk | awk 'NF>=2' | wc -l) 23 | if [[ $has_mounted_partition -eq 0 ]]; then 24 | drives="${drives} ${device_path}" 25 | dd if=/dev/zero of=${device_path} bs=4096 count=1024 26 | else 27 | echo "Target disk $disk has already mounted partitions. skipping" 28 | fi 29 | else 30 | echo "Target disk ${disk}, ${device_path} is not present. skipping" 31 | fi 32 | set -e 33 | done 34 | 35 | drive_count=$(set -- ${drives}; echo $#) 36 | if [[ "${drive_count}" -eq 0 ]]; then 37 | echo "No data drives detected" 38 | elif [[ "${drive_count}" -eq 1 ]]; then 39 | echo "A single target disk detected." 40 | TARGET_DISK=${drives} 41 | else 42 | partprobe || true 43 | mdadm --create --verbose ${TARGET_DISK} --name=${TARGET_DISK##*/} --level=0 -c256 --raid-devices=${drive_count} ${drives} 44 | 45 | [[ -e ${MDADM_CONF} ]] && cp ${MDADM_CONF} ${MDADM_CONF}.backup 46 | install -Dv /dev/null ${MDADM_CONF} 47 | blockdev --setra 65536 ${TARGET_DISK} 48 | mdadm --examine --scan | tee -a ${MDADM_CONF} 49 | echo $((30*1024)) > /proc/sys/dev/raid/speed_limit_min 50 | fi 51 | 52 | echo "using target devices: ${drives}" 53 | pvcreate ${TARGET_DISK} 54 | vgcreate ${LVG} ${TARGET_DISK} 55 | 56 | ## create a swap volume with preferred size = total_memory/2, but 57 | ## dont use more than 7% of the available ephemeral storage 58 | SWAP_MAX_SIZE=$(vgdisplay --units M ${LVG} | grep "VG Size" | awk '{ print mem=int(0.07*$3); }') 59 | lvcreate -n swap -L $(grep MemTotal /proc/meminfo | awk -v MAXMEM=${SWAP_MAX_SIZE} '{ mem=int($2/(2*1024)); if(mem>MAXMEM) mem=MAXMEM; print mem; }')m ${LVG} 60 | lvcreate -n data -l 100%FREE ${LVG} 61 | 62 | echo "Formatting LVM volumes..." 63 | mkswap /dev/${LVG}/swap 64 | mkfs.xfs -K /dev/${LVG}/data 65 | 66 | DATA_MOUNT_OPTIONS="noatime,defaults,pquota,prjquota" 67 | 68 | {% if systemd_os %} 69 | DATA_MOUNT_OPTIONS="${DATA_MOUNT_OPTIONS},x-systemd.requires=cloud-init.service,x-systemd.requires" 70 | {% endif %} 71 | 72 | # TODO (mat): maybe remove this after testing discard behavior in fio/rally 73 | DATA_MOUNT_OPTIONS="${DATA_MOUNT_OPTIONS},discard" 74 | 75 | echo "/dev/${LVG}/swap swap swap swap 0 0" >> /etc/fstab 76 | echo "/dev/${LVG}/data {{ data_dir }} xfs ${DATA_MOUNT_OPTIONS} 0 0" >> /etc/fstab 77 | 78 | echo "Mounting LVM volumes..." 79 | mount -a 80 | swapon -a 81 | chown -R ${IMAGE_USER}:${IMAGE_USER} {{ data_dir }} 82 | install -d -m 0700 -o ${IMAGE_USER} -g docker {{ data_dir }}/docker 83 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ansible-elastic-cloud-enterprise 2 | 3 | Ansible role for installing [Elastic Cloud Enterprise](https://www.elastic.co/products/ece) and preparing hosts for it. 4 | 5 | Please note that the ECE Ansible playbook is a community project supported by Elastic. Elastic welcomes all community contributions to the repository and will validate any changes on a best-effort basis. 6 | 7 | ## Requirements 8 | 9 | This role is tested against Ansible 2.8.7. 10 | 11 | ## Contents of this role 12 | 13 | A minimal example of a [small playbook](https://www.elastic.co/guide/en/cloud-enterprise/current/ece-install-small-cloud.html) might look like this: 14 | 15 | ```yaml 16 | --- 17 | - hosts: primary 18 | gather_facts: true 19 | roles: 20 | - ansible-elastic-cloud-enterprise 21 | vars: 22 | ece_primary: true 23 | 24 | - hosts: secondary 25 | gather_facts: true 26 | roles: 27 | - ansible-elastic-cloud-enterprise 28 | vars: 29 | ece_roles: [director, coordinator, proxy, allocator] 30 | 31 | - hosts: tertiary 32 | gather_facts: true 33 | roles: 34 | - ansible-elastic-cloud-enterprise 35 | vars: 36 | ece_roles: [director, coordinator, proxy, allocator] 37 | ``` 38 | 39 | At least three hosts are needed for this example, a primary, a secondary, and tertiary host. The example above would execute the following high level steps on the defined hosts: 40 | - On all hosts: 41 | - Remove an existing docker installation 42 | - Install required general packages 43 | - Install a current, supported docker version 44 | - Create required users and set limits for them 45 | - Create a xfs partition and configure it 46 | - Configure docker 47 | 48 | More information about the prerequisites can be found in the following [page](https://www.elastic.co/guide/en/cloud-enterprise/current/ece-prereqs.html). 49 | - On the primary host: 50 | - Make the primary installation of Elastic Cloud Enterprise 51 | - On the secondary host: 52 | - Install Elastic Cloud Enterprise to join the existing installation with the given ece_roles 53 | - On the tertiary host: 54 | - Install Elastic Cloud Enterprise to join the existing installation with the given ece_roles 55 | 56 | There is a set of variables and tags available to further define the behaviour of this role, or exclude certain steps. 57 | 58 | For example in many cases you might want to install Elastic Coud Enterprise without running any of the potentially destructive system prerequisites like removing existing docker installations and setting up a filesystem. This can be done by specifying `--skip-tags destructive` on your ansible run - or if you want to only install Elastic Coud Enterprise without any system tasks before `--tags bootstrap`. 59 | 60 | 61 | ## Role Variables 62 | 63 | The following variables are avaible: 64 | 65 | - `device_name`: The name of the device on which the xfs partition should be created 66 | - **Required** unless filesystem tasks are skipped via tags 67 | - Default: xvdb 68 | - `ece_primary`: Whether this host should be the primary (first) host where Elastic Cloud Enterprise is installed 69 | - **Required** on a single host 70 | - `data_dir`: Which directory to mount the xfs partition under 71 | - Default: `/mnt/data` 72 | - `ece_roles`: Elastic Cloud Enterprise roles that successive hosts should assume 73 | - Default: [director, coordinator, proxy, allocator] 74 | - `capacity`: [Amount of memory to grant to the allocator](https://www.elastic.co/guide/en/cloud-enterprise/current/ece-manage-capacity.html#ece-alloc-memory) 75 | - Default: left empty, installer default behavior will be applied 76 | - `availability_zone`: The availability zone this group of hosts belongs to 77 | - `ece_version`: The Elastic Cloud Enterprise version that should get installed 78 | - Default: 2.8.1 79 | - `ece_docker_registry`: The docker registry from where to pull the Elastic Cloud Enterprise images. This is only relevant if you have a private mirror 80 | - Default: docker.elastic.co 81 | - `ece_docker_repository`: The docker repository in the given registry. This is only relevant if you have a private mirror 82 | - Default: cloud-enterprise 83 | - `ece_installer_url`: The url of the installation script to download. 84 | - Default: `https://download.elastic.co/cloud/elastic-cloud-enterprise.sh` 85 | - This will use the local script if existing in `/home/elastic/elastic-cloud-enterprise.sh` 86 | - `ece_installer_path`: The location of the installation script on the controller machine. It will be copied to remote host. 87 | - Default: left empty, it will download it from internet (cf. `ece_installer_url`) 88 | - `docker_config`: If specified as a path to a docker config, copies it to the target hosts 89 | - [Supported Docker Versions](https://www.elastic.co/guide/en/cloud-enterprise/2.7/ece-software-prereq.html#ece-linux-docker) 90 | - `docker_version`: Last supported version on Centos 7/8 and RHEL 7/8 is 20.0, Ubuntu 16, Ubuntu 18 and SLES 12 is 19.03. 91 | - `docker_bridge_ip `: The default IP of the docker bridge. Configurable to avoid overlapping with the current host subnet. 92 | - `force_xfc`: By default if the `lxc` xfc volume already exists, the `setup_xfc` step is skipped, if this is set to true, creation of the volume is forced 93 | - Default: false 94 | - `elastic_authorized_keys_file`: Defines a local path to an `authorized_keys` file that should be copied to the `elastic` user. If not set, the keys from the default user that is used with ansible will be copied over. 95 | - `memory`: Defines the JVM heap size to be used for different services running in ece. See https://www.elastic.co/guide/en/cloud-enterprise/2.7/ece-jvm.html for example values and [defaults/main.yml](defaults/main.yml) for the default values. 96 | 97 | - `fetch_diagnostics`: Determines if Elastic Cloud Enterprise Support Diagnostics should be downloaded and executed 98 | - `ece_supportdiagnostics_url`: THe location of the diagnostics tool. Can be a local file for offline installation. 99 | - Default: `https://github.com/elastic/ece-support-diagnostics/archive/v1.1.tar.gz` 100 | - `ece_supportdiagnostics_result_path`: The localtion where to store the diagnostic bundles on ansible host. 101 | - Default: `/tmp/ece-support-diagnostics` 102 | - `ece_runner_id`: Assigns an arbitrary ID to the host (runner) that you are installing Elastic Cloud Enterprise on 103 | - Default: `ansible_default_ipv4.address` 104 | 105 | If more hosts should join an Elastic Cloud Enterpise installation when a primary host was already installed previously there are two more variables that are required: 106 | - `primary_hostname`: The (reachable) hostname of the primary host 107 | - `adminconsole_root_password`: The adminconsole root password 108 | 109 | 110 | ## Role Tags 111 | 112 | The following tags are available to limit the execution, due to the nature of tags in ansible you should only use `--skip-tags` with these to skip certain parts instead of using `--tags` to limit the execution. 113 | 114 | - `base` Determines the execution of all tasks that setup the system (everything except the actual installation of Elastic Cloud Enterprise) 115 | - `setup_filesystem` If system tasks are executed, this determines if the filesystem tasks should get executed - includes creating the partitions for xfs and mount points 116 | - `install_docker` If system tasks are executed, this determines if existing docker packages should get removed and the current, supported version should get installed and configured 117 | - `destructive` This tag indicates whether a task is potentially destructive, like removing packages or doing filesystem partitioning 118 | - `ece` Determines if Elastic Cloud Enterprise should get installed 119 | - `vmimage` Prepare the system for building a Virtual Machine Image (Amazon AMI, ...). This will install a cloud-init script which will auto-discover and mount disk selected when an instance is launched with this image. 120 | - `bootstrap` This tags should be picked for only installing Elastic Cloud Entreprise itself (no prerequistes) 121 | 122 | By default, all tags are applied, except `vmimage`, which means that it will install all prerequisites and Elastic Cloud Entreprise. 123 | In order to use this ansible playbook for building a VM image, the following tags should be selected: `--tags base,vmimage` (this won't install Elastic Cloud Enterprise) 124 | 125 | ## Examples and use cases 126 | 127 | ### Medium sized first installation of Elastic Cloud Enterprise 128 | 129 | This example installs Elastic Cloud Enterprise as detailed in "A medium installation with separate management services" [in the official documentation](https://www.elastic.co/guide/en/cloud-enterprise/current/ece-install-medium-cloud.html) and brings you up to *step 5 - Modify the first host you installed Elastic Cloud Enterprise on* 130 | 131 | `site.yml`: 132 | ```yaml 133 | - hosts: primary 134 | roles: 135 | - ansible-elastic-cloud-enterprise 136 | vars: 137 | ece_primary: true 138 | 139 | - hosts: director_coordinator 140 | roles: 141 | - ansible-elastic-cloud-enterprise 142 | vars: 143 | ece_roles: [director, coordinator, proxy] 144 | 145 | - hosts: allocator 146 | roles: 147 | - ansible-elastic-cloud-enterprise 148 | vars: 149 | ece_roles: [allocator] 150 | ``` 151 | 152 | Assuming all hosts have the device name in common the `inventory.yml` could look like this: 153 | ```yaml 154 | all: 155 | vars: 156 | ansible_become: yes 157 | device_name: sdb 158 | children: 159 | primary: 160 | hosts: 161 | host1: 162 | availability_zone: zone-1 163 | director_coordinator: 164 | hosts: 165 | host2: 166 | availability_zone: zone-2 167 | host3: 168 | availability_zone: zone-3 169 | allocator: 170 | hosts: 171 | host4: 172 | availability_zone: zone-1 173 | host5: 174 | availability_zone: zone-2 175 | host6: 176 | availability_zone: zone-3 177 | ``` 178 | 179 | ### Adding hosts to an existing installation 180 | 181 | Assuming you already have an existing installation of Elastic Cloud Enterprise and you want to add more allocators to it you need to specify two additional variables: 182 | - `primary_hostname`: The (reachable) hostname of the primary host 183 | - `adminconsole_root_password`: The adminconsole root password 184 | 185 | The corresponding `site.yml` could then look like: 186 | 187 | ```yaml 188 | - hosts: allocator 189 | roles: 190 | - ansible-elastic-cloud-enterprise 191 | vars: 192 | ece_roles: [allocator] 193 | primary_hostname: host1 194 | adminconsole_root_password: secret_password 195 | ``` 196 | 197 | With the `inventory.yml` 198 | ```yaml 199 | all: 200 | vars: 201 | ansible_become: yes 202 | device_name: sdb 203 | children: 204 | allocator: 205 | hosts: 206 | host7: 207 | availability_zone: zone-1 208 | host8: 209 | availability_zone: zone-2 210 | host9: 211 | availability_zone: zone-3 212 | ``` 213 | 214 | ### Performing an upgrade 215 | 216 | You only need to run the upgrade on a single host, it will then automatically propagate to all other hosts. 217 | An upgrade is usually performed on the first host you installed Elastic Cloud Enterprise on, but it can also be run from any host that holds the director role. 218 | 219 | Assuming you have an installation of Elastic Cloud Enterprise 3.7.3 and want to upgrade to 3.8.0 `site.yml` could then look like: 220 | ```yaml 221 | - hosts: upgradehost 222 | roles: 223 | - ansible-elastic-cloud-enterprise 224 | vars: 225 | ece_version: 3.8.0 226 | adminconsole_root_password: secret_password 227 | ``` 228 | 229 | - `ece_version`: The target version you want to upgrade to 230 | - `adminconsole_root_password`: The adminconsole root password 231 | 232 | with `inventory.yml` 233 | ```yaml 234 | all: 235 | children: 236 | upgradehost: 237 | hosts: 238 | host1: 239 | ``` 240 | 241 | It is important that you then specify `--tags bootstrap` when you run the playbook in order to only perform the Elastic Cloud Enterprise update and no other tasks, especially when the initial installation was not done with this role. 242 | ```bash 243 | ansible-playbook -i inventory.yml site.yml --tags bootstrap 244 | ``` 245 | 246 | ### Building a base Virtual Machine Image 247 | Building a Virtual Machine Images depends on the tools and platform you are using. Once a base instance is running, you can use a playbook like the following: 248 | ```yaml 249 | - hosts: all 250 | become: true 251 | roles: 252 | - ansible-elastic-cloud-enterprise 253 | ``` 254 | 255 | And ansible should be run with `--tags base,vmimage`, this will install prerequisites for Elastic Cloud Entreprise, but not Elastic Cloud Entreprise. 256 | Finally, you will be able to save the instance as VM image (depending on your cloud provider) 257 | 258 | Once the image is ready, you can use it as a base to install Elastic Cloud Entreprise, either from the boostraper script, or with ansible, using `--tags bootstrap` (this will install only Elastic Cloud Entreprise) 259 | 260 | ## Extending and Contributing 261 | 262 | See [CONTRIBUTING.md](CONTRIBUTING.md) for more details on how to contribute and extend the Elastic Cloud Enterprise role. 263 | --------------------------------------------------------------------------------