├── LICENSE ├── README.md ├── defaults └── main.yml ├── handlers └── main.yml ├── meta └── main.yml ├── tasks ├── firewalld.yml ├── insecure.yml ├── main.yml └── secure.yml ├── templates ├── insecure │ ├── etcd │ │ └── etcd.conf.j2 │ ├── flannel │ │ └── flanneld-conf.json.j2 │ ├── kubernetes │ │ ├── apiserver.j2 │ │ ├── config.j2 │ │ ├── controller-manager.j2 │ │ ├── proxy.j2 │ │ └── scheduler.j2 │ └── sysconfig │ │ ├── docker-storage-setup.j2 │ │ └── flanneld.j2 └── secure │ ├── etcd │ └── etcd.conf.j2 │ ├── flannel │ └── flanneld-conf.json.j2 │ ├── kubernetes │ ├── apiserver.j2 │ ├── config.j2 │ ├── controller-manager.j2 │ ├── controller-manager.kubeconfig.j2 │ ├── proxy.j2 │ ├── proxy.kubeconfig.j2 │ ├── scheduler.j2 │ └── scheduler.kubeconfig.j2 │ ├── openssl │ └── openssl.conf.j2 │ └── sysconfig │ ├── docker-storage-setup.j2 │ └── flanneld.j2 └── vars └── main.yml /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2016 Andrew J Huffman 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![Ansible Role](https://img.shields.io/ansible/role/d/9306) 2 | 3 | # ahuffman.k8s-master 4 | 5 | An ansible role to configure a kubernetes master on a Red Hat Enterprise Linux based system. 6 | 7 | This role can deploy either a SSL Secured kubernetes master (kube-apiserver) or an insecured kubernetes master. See example playbooks for the minimum required variables to deploy a cluster in a insecure or secure fashion. It will also open the required system firewall ports with firewalld (default RHEL7/CentOS7). 8 | 9 | ## Requirements 10 | 11 | If the kubernetes master will also be a cluster member (node/minion), you will want to make use of the [`k8s-node`](https://galaxy.ansible.com/ahuffman/k8s-node/) role. 12 | It is critical that you properly set the hostname and domain of the kubernetes master properly. You may want to use the [`ahuffman.hosts`](https://galaxy.ansible.com/ahuffman/hosts/) role, or an alternative to accomplish this. 13 | 14 | 15 | ## Role Variables 16 | 17 | ### Defaults: 18 | Found in [`defaults/main.yml`](defaults/main.yml) 19 | #### k8s-master Role settings: 20 | `k8s_cockpit`: true - Required if you'd like cockpit and the cockpit-kubernetes plugin to be installed 21 | `k8s_mst_is_node`: false - Change to `true` if you plan on making the master a cluster member (node/minion) as well. You'll also need to make use of the [`k8s-node`](https://galaxy.ansible.com/ahuffman/k8s-node/) role to properly configure your node. 22 | `k8s_secure_master`: false - Change to `true` if you'd like to generate certificates and communicate over secured channels. 23 | `k8s_firewalld`: true - Whether or not to open the required firewall ports with firewalld. If you're managing your system's firewall ports outside of this role, set to false. 24 | `k8s_docker_storage_setup`: false - Whether or not to configure docker's container storage pool. See settings below. 25 | 26 | 27 | #### Etcd/flannel Network Settings: 28 | `etcd_port`: 2379 - The port of your etcd server. As mentioned above, you most likely won't need to change this setting. 29 | `etcd_key`: /kube01/network - The key where your cluster's network settings will be stored in etcd. Change as needed per cluster. 30 | `flannel_backend_network`: 172.16.0.0/12 - The network for backend pod/kube-proxy communications. 31 | `flannel_subnet_length`: '24' - The kubernetes node backend subnet length (i.e. The size of network slices assigned to kubernetes nodes.) 32 | 33 | #### Kube-apiserver Settings: 34 | `k8s_kubelet_port`: '10250' - The port on which kubernetes kubelets are expected to communicate with the apiserver. 35 | `k8s_service_network`: 192.168.22.0/24 - Modify this setting to define what range of IPs your deployed kubernetes services will serve on. 36 | `k8s_admission_control`: 'NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota' - Modify this if you wish to not use one of the default kubernetes admission controllers. 37 | 38 | #### Optional Advanced kube-apiserver Settings: 39 | `k8s_auth_mode`: '' - Use this setting to apply one of the authorization modes to the apiserver's configuration. 40 | `k8s_auth_policy_file`: '' - The location of your authorization method's policy file. 41 | `k8s_token_auth_file`: '' - The locaion of your token authorization file. 42 | `k8s_apiserver_additional_args`: '' - Any additional kube-apiserver options you wish to apply to the kube-apiserver service. See `man kube-apiserver` for all available options. 43 | `k8s_controller_manager_additional_args`: '' - Any additional kube-controller-manager options you wish to apply to the kube-controller-manager service. See `man kube-controller-manager` for all available options. 44 | `k8s_kube_proxy_additional_args`: '' - Any additional kube-proxy options you wish to apply to the kube-proxy service. See `man kube-proxy` for all available options. 45 | `k8s_scheduler_additional_args`: - Any additional kube-scheduler options you wish to apply to the kube-scheduler service. See `man kube-scheduler` for all available options. 46 | 47 | #### Kubernetes General Settings: 48 | `k8s_apiserver_insecure_port`: 8080 - The port where insecure communication with the kube-apiserver will take place. 49 | `k8s_allow_privileged`: false - Whether or not to allow the execution of privileged containers. 50 | `k8s_log_level`: 0 - The verbosity of kubernetes logs. 51 | `k8s_logtostderr`: true - Whether or not to log to standard error. 52 | 53 | #### Docker storage setup options: 54 | `k8s_docker_storage_disk`: '' - Used with the k8s_docker_storage_setup option above. Provide an unformatted device such as '/dev/sdb'. This assumes it is a clean server deployment. If you've already started the docker engine, then you'll have to cleanup the default storage pool. 55 | `k8s_docker_storage_vg`: vg_docker - The volume group to use/create for docker storage. 56 | `k8s_docker_storage_options`: 57 | - AUTO_EXTEND_POOL = true 58 | - POOL_AUTOEXTEND_THRESHOLD 59 | See 'man docker-storage-setup` for all available options. You can add whichever best suit your environment, but the defaults here should work well for you. 60 | 61 | #### Secure kube-apiserver Settings: - only applies if `k8s_secure_master: true`. 62 | `k8s_apiserver_secure_port`: 6443 - Port to server kube-apiserver secured communications. 63 | `k8s_apiserver_cert_path`: /etc/kubernetes/certs - Where to store the server's certificates and keys. 64 | `k8s_apiserver_ca_key_filename`: ca.key - What filename to name your CA key. 65 | `k8s_apiserver_ca_cert_filename`: ca.crt - What filename to name your CA certificate. 66 | `k8s_apiserver_server_key_filename`: server.key - What filename to name your kube-apiserver's key. 67 | `k8s_apiserver_server_csr_filename`: server.csr - What filename to name your kube-apiserver's certificate signing request. 68 | `k8s_apiserver_server_cert_filename`: server.cert - What filename to name your kube-apiserver's certificate. 69 | `k8s_proxy_kubeconfig_filename`: proxy.kubeconfig - What filename to name your kube-proxy service's kubeconfig. 70 | `k8s_scheduler_kubeconfig_filename`: scheduler.kubeconfig - What filename to name your kube-scheduler service's kubeconfig. 71 | `k8s_controller_manager_kubeconfig_filename`: controller-manager.kubeconfig - What filename to name your kube-controller-manager service's kubeconfig. 72 | `k8s_apiserver_dns_names`: 73 | - kubernetes 74 | - kubernetes.default 75 | - kubernetes.default.svc 76 | - kubernetes.default.svc.cluster.local 77 | 78 | `k8s_apiserver_dns_names` is a list of the Subject Alternative Names to use during certificate generation. You should override this variable at the `host_vars` level, and list any possible DNS name that the server will need to serve with. It's recommended to append your additional server names to this list. 79 | 80 | `k8s_apiserver_additional_ips`: [] - List to add any additional IP address the kube-apiserver will serve on. This will be used to create IP Subject Alternative Names during certificate generation. By default the `k8s-master` role will make use of the `ansible_default_ipv4` IP Address, so only add additional addresses other than the `ansible_default_ipv4` address. 81 | 82 | ### Variables: 83 | Found in [`vars/main.yml`](vars/main.yml) 84 | 85 | k8s_mst_packages: 86 | - etcd 87 | - kubernetes-master 88 | - kubernetes-node 89 | - flannel 90 | - openssl #for certificate generation 91 | 92 | cockpit_kubernetes_pkg: 93 | - cockpit-kubernetes 94 | - cockpit 95 | 96 | For ripping and replacing RHEL cockpit due to dependencies: 97 | 98 | cockpit_default: 99 | - cockpit-shell 100 | - cockpit-bridge 101 | - cockpit-ws 102 | - cockpit 103 | - cockpit-networkmanager 104 | - cockpit-storaged 105 | - cockpit-docker 106 | 107 | For opening the appropriate firewall ports on the master server (based upon secure or insecure config). As you'll see, these refer back to what is set in (or over-rided) [`defaults/main.yml`](defaults/main.yml). 108 | 109 | k8s_firewall_ports_secure: 110 | - '{{ etcd_port }}/tcp' #etcd 111 | - '{{ k8s_apiserver_secure_port }}/tcp' #kube-apiserver 112 | - '{{ k8s_kubelet_port }}/tcp' #kubelet 113 | 114 | k8s_firewall_ports_insecure: 115 | - '{{ etcd_port }}/tcp' #etcd 116 | - '{{ k8s_apiserver_insecure_port }}/tcp' #kube-apiserver 117 | - '{{ k8s_kubelet_port }}/tcp' #kubelet 118 | 119 | 120 | ### Viewing your Kubernetes Cluster in Cockpit 121 | If you set `k8s_cockpit: true` the Cockpit service and `cockpit-kubernetes` Cockpit plug-in will be installed on your master. 122 | 123 | You can access your Cockpit via a web-browser by visiting: https://YOUR_MASTER_HOSTNAME:9090 124 | 125 | Once you've logged into the Cockpit service, you can click on the "Cluster" tab at the top of the page to view information about your cluster. 126 | 127 | 128 | Example Playbooks 129 | ---------------- 130 | ### Insecured kubernetes master 131 | 132 | - hosts: kubernetes_master 133 | roles: 134 | - ahuffman.k8s-master 135 | 136 | ### Insecured kubernetes master, running a node on the master as well (applied [`k8s-node`](https://galaxy.ansible.com/ahuffman/k8s-node/) role) 137 | 138 | - hosts: kubernetes_master 139 | vars: 140 | k8s_mst_is_node: true 141 | roles: 142 | - ahuffman.k8s-master 143 | 144 | - hosts: kubernetes_master 145 | vars: 146 | k8s_docker_storage_setup: true 147 | k8s_docker_storage_disk: /dev/sdb 148 | k8s_master_fqdn: kubmst01.foobar.com 149 | roles: 150 | - ahuffman.k8s-node 151 | 152 | ### Secured kubernetes master 153 | 154 | - hosts: kubernetes_master 155 | vars: 156 | k8s_secure_master: true 157 | k8s_apiserver_dns_names: 158 | - kubernetes 159 | - kubernetes.default 160 | - kubernetes.default.svc 161 | - kubernetes.default.svc.cluster.local 162 | #Adding additional SANs 163 | - foobar01 164 | - foobar01.foobar.org 165 | - foobar 166 | - foobar.foobar.org 167 | roles: 168 | - ahuffman.k8s-master 169 | 170 | ### Secured kubernetes master, running a secured node on the master as well (applied [`k8s-node`](https://galaxy.ansible.com/ahuffman/k8s-node/) role) 171 | 172 | - hosts: kubernetes_master 173 | vars: 174 | k8s_secure_master: true 175 | k8s_mst_is_node: true 176 | k8s_apiserver_dns_names: 177 | - kubernetes 178 | - kubernetes.default 179 | - kubernetes.default.svc 180 | - kubernetes.default.svc.cluster.local 181 | #Adding additional SANs 182 | - foobar01 183 | - foobar01.foobar.org 184 | - foobar 185 | - foobar.foobar.org 186 | roles: 187 | - ahuffman.k8s-master 188 | 189 | - hosts: kubernetes_master 190 | vars: 191 | #k8s-node settings see k8s-node - https://galaxy.ansible.com/ahuffman/k8s-node/ for more details on settings 192 | k8s_secure_node: true 193 | k8s_docker_storage_setup: true 194 | k8s_docker_storage_disk: /dev/sdb 195 | k8s_master_fqdn: kubmst01.foobar.com 196 | roles: 197 | - ahuffman.k8s-node 198 | 199 | 200 | License 201 | ------- 202 | 203 | [MIT](LICENSE) 204 | 205 | Author Information 206 | ------------------ 207 | 208 | [Andrew J. Huffman](https://github.com/ahuffman) 209 | -------------------------------------------------------------------------------- /defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | #*****Variables that apply to how this role will operate***** 3 | #install cockpit and kubernetes plugin? 4 | k8s_cockpit: true 5 | 6 | #is the master also a node/minion? 7 | #for compatibility with k8s-node role: https://galaxy.ansible.com/ahuffman/k8s-node 8 | k8s_mst_is_node: false 9 | 10 | #Deploy secure cluster or insecure? 11 | k8s_secure_master: false 12 | #set to false to ensure we don't hurt any v1.x role users upgrading to this ver. 13 | 14 | #Open firewall ports with firewalld? 15 | k8s_firewalld: true 16 | 17 | #Configure docker storage? 18 | k8s_docker_storage_setup: false 19 | 20 | #*****etcd/flannel network settings****** 21 | etcd_port: 2379 22 | etcd_key: /kube01/network 23 | flannel_backend_network: 172.16.0.0/12 24 | flannel_subnet_length: '24' #applies to kubernetes node backend subnet length 25 | 26 | 27 | #*****kube-apiserver settings***** 28 | k8s_kubelet_port: '10250' #default 10250 and usually commented out in config 29 | k8s_service_network: 192.168.22.0/24 #Network range to serve kubernetes services on 30 | k8s_admission_control: 'NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota' 31 | 32 | 33 | #*****Optional advanced apiserver settings***** 34 | k8s_auth_mode: '' 35 | k8s_auth_policy_file: '' 36 | k8s_token_auth_file: '' 37 | k8s_apiserver_additional_args: '' #pass more apiserver switches here as a string 38 | k8s_controller_manager_additional_args: '' 39 | k8s_kube_proxy_additional_args: '' 40 | k8s_scheduler_additional_args: '' 41 | 42 | 43 | #*****kubernetes general settings***** 44 | k8s_apiserver_insecure_port: 8080 45 | k8s_allow_privileged: false 46 | k8s_log_level: 0 47 | k8s_logtostderr: true 48 | 49 | #*****docker-storage-setup options 50 | k8s_docker_storage_disk: '' 51 | k8s_docker_storage_vg: vg_docker 52 | k8s_docker_storage_options: 53 | - AUTO_EXTEND_POOL=true 54 | - POOL_AUTOEXTEND_THRESHOLD=80 55 | 56 | #*****Variables that apply to a secure kubernetes cluster***** 57 | k8s_apiserver_secure_port: '6443' 58 | k8s_apiserver_cert_path: /etc/kubernetes/certs 59 | k8s_apiserver_ca_key_filename: ca.key 60 | k8s_apiserver_ca_cert_filename: ca.crt 61 | k8s_apiserver_server_key_filename: server.key 62 | k8s_apiserver_server_csr_filename: server.csr 63 | k8s_apiserver_server_cert_filename: server.cert 64 | k8s_proxy_kubeconfig_filename: proxy.kubeconfig 65 | k8s_scheduler_kubeconfig_filename: scheduler.kubeconfig 66 | k8s_controller_manager_kubeconfig_filename: controller-manager.kubeconfig 67 | #we'll generate certs with this variable. 68 | #Override this variable list all the Subject Alternate Names the master will be known as 69 | k8s_apiserver_dns_names: 70 | - kubernetes 71 | - kubernetes.default 72 | - kubernetes.default.svc 73 | - kubernetes.default.svc.cluster.local 74 | #we'll generate certs with this variable. 75 | #Override this list with any other ip's required to validate the certificate 76 | #openssl.conf.j2 template grabs ansible_default_ipv4.address by default. Add any other IPs you'll serve on 77 | k8s_apiserver_additional_ips: [] 78 | -------------------------------------------------------------------------------- /handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | #**************Cockpit**************** 4 | #Remediation for install cockpit-kubernetes on RHEL system 5 | - name: Remove default cockpit packages 6 | yum: name='{{ item }}' state=removed 7 | with_items: '{{ cockpit_default }}' 8 | when: k8s_cockpit == True 9 | 10 | #Remediation for Ensure cockpit-kubernetes 11 | - name: Setup CentOS7-Extras Repository for cockpit-kubernetes 12 | yum_repository: 13 | name: centos7-extras 14 | description: CentOS7 - Extras 15 | baseurl: http://mirror.centos.org/centos/7/extras/x86_64/ 16 | enabled: yes 17 | gpgcheck: yes 18 | gpgkey: http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-7 19 | includepkgs: cockpit-kubernetes cockpit-shell cockpit-bridge cockpit cockpit-ws cockpit-networkmanager cockpit-storaged cockpit-docker 20 | when: k8s_cockpit == True 21 | 22 | - name: Install cockpit-kubernetes 23 | yum: name='{{ item }}' state=latest 24 | with_items: '{{ cockpit_kubernetes_pkg }}' 25 | when: k8s_cockpit == True 26 | 27 | - name: Disable CentOS7-Extras Repository post cockpit-kubernetes install 28 | yum_repository: 29 | name: centos7-extras 30 | description: CentOS7 - Extras 31 | baseurl: http://mirror.centos.org/centos/7/extras/x86_64/ 32 | enabled: no 33 | gpgcheck: yes 34 | gpgkey: http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-7 35 | includepkgs: cockpit-kubernetes cockpit-shell cockpit-bridge cockpit cockpit-ws cockpit-networkmanager cockpit-storaged cockpit-docker 36 | when: k8s_cockpit == True 37 | 38 | - name: Enable Cockpit Service in firewalld 39 | firewalld: service=cockpit permanent=true state=enabled zone=public immediate=yes 40 | when: k8s_cockpit == True and k8s_firewalld == True 41 | 42 | 43 | #*****************Services********************* 44 | #Service Restarts 45 | - name: Restart kube-scheduler 46 | service: name=kube-scheduler state=restarted 47 | 48 | - name: Restart kube-controller-manager 49 | service: name=kube-controller-manager state=restarted 50 | 51 | - name: Restart kube-proxy 52 | service: name=kube-proxy state=restarted 53 | when: k8s_mst_is_node == False 54 | 55 | - name: Restart flanneld 56 | service: name=flanneld state=restarted 57 | when: k8s_mst_is_node == False 58 | 59 | - name: Restart kube-apiserver 60 | service: name=kube-apiserver state=restarted 61 | 62 | - name: Enable cockpit 63 | service: name=cockpit.socket enabled=yes 64 | when: k8s_cockpit == True 65 | 66 | - name: Restart cockpit 67 | service: name=cockpit state=restarted 68 | when: k8s_cockpit == True 69 | 70 | - name: Restart etcd 71 | service: name=etcd state=restarted 72 | #******************CA************************* 73 | #Retrieve apiserver certificates 74 | - name: Get CA key 75 | slurp: 76 | src: "{{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_key_filename }}" 77 | register: ssl_ca_key 78 | 79 | - name: Get CA cert 80 | slurp: 81 | src: "{{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }}" 82 | register: ssl_ca_cert 83 | 84 | - name: Store CA key 85 | shell: echo "{{ ssl_ca_key.content | b64decode }}" > files/k8s-master-certs/{{ ansible_hostname }}.{{ ansible_domain }}/{{ k8s_apiserver_ca_key_filename }} 86 | delegate_to: localhost 87 | 88 | - name: Store CA cert 89 | shell: echo "{{ ssl_ca_cert.content | b64decode }}" > files/k8s-master-certs/{{ ansible_hostname }}.{{ ansible_domain }}/{{ k8s_apiserver_ca_cert_filename }} 90 | delegate_to: localhost 91 | -------------------------------------------------------------------------------- /meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | galaxy_info: 3 | author: Andrew J. Huffman 4 | description: An ansible role to configure a kubernetes master on a Red Hat Enterprise Linux based system. 5 | # If the issue tracker for your role is not on github, uncomment the 6 | # next line and provide a value 7 | # issue_tracker_url: http://example.com/issue/tracker 8 | # Some suggested licenses: 9 | # - BSD (default) 10 | # - MIT 11 | # - GPLv2 12 | # - GPLv3 13 | # - Apache 14 | # - CC-BY 15 | license: MIT 16 | min_ansible_version: 1.2 17 | # 18 | # Below are all platforms currently available. Just uncomment 19 | # the ones that apply to your role. If you don't see your 20 | # platform on this list, let us know and we'll get it added! 21 | # 22 | platforms: 23 | - name: EL 24 | versions: 25 | # - all 26 | # - 5 27 | # - 6 28 | - 7 29 | #- name: GenericUNIX 30 | # versions: 31 | # - all 32 | # - any 33 | #- name: Solaris 34 | # versions: 35 | # - all 36 | # - 10 37 | # - 11.0 38 | # - 11.1 39 | # - 11.2 40 | # - 11.3 41 | #- name: Fedora 42 | # versions: 43 | # - all 44 | # - 16 45 | # - 17 46 | # - 18 47 | # - 19 48 | # - 20 49 | # - 21 50 | # - 22 51 | # - 23 52 | #- name: opensuse 53 | # versions: 54 | # - all 55 | # - 12.1 56 | # - 12.2 57 | # - 12.3 58 | # - 13.1 59 | # - 13.2 60 | #- name: IOS 61 | # versions: 62 | # - all 63 | # - any 64 | #- name: SmartOS 65 | # versions: 66 | # - all 67 | # - any 68 | #- name: eos 69 | # versions: 70 | # - all 71 | # - Any 72 | #- name: Windows 73 | # versions: 74 | # - all 75 | # - 2012R2 76 | #- name: Amazon 77 | # versions: 78 | # - all 79 | # - 2013.03 80 | # - 2013.09 81 | #- name: GenericBSD 82 | # versions: 83 | # - all 84 | # - any 85 | #- name: Junos 86 | # versions: 87 | # - all 88 | # - any 89 | #- name: FreeBSD 90 | # versions: 91 | # - all 92 | # - 10.0 93 | # - 10.1 94 | # - 10.2 95 | # - 8.0 96 | # - 8.1 97 | # - 8.2 98 | # - 8.3 99 | # - 8.4 100 | # - 9.0 101 | # - 9.1 102 | # - 9.1 103 | # - 9.2 104 | # - 9.3 105 | #- name: Ubuntu 106 | # versions: 107 | # - all 108 | # - lucid 109 | # - maverick 110 | # - natty 111 | # - oneiric 112 | # - precise 113 | # - quantal 114 | # - raring 115 | # - saucy 116 | # - trusty 117 | # - utopic 118 | # - vivid 119 | # - wily 120 | # - xenial 121 | #- name: SLES 122 | # versions: 123 | # - all 124 | # - 10SP3 125 | # - 10SP4 126 | # - 11 127 | # - 11SP1 128 | # - 11SP2 129 | # - 11SP3 130 | #- name: GenericLinux 131 | # versions: 132 | # - all 133 | # - any 134 | #- name: NXOS 135 | # versions: 136 | # - all 137 | # - any 138 | #- name: Debian 139 | # versions: 140 | # - all 141 | # - etch 142 | # - jessie 143 | # - lenny 144 | # - sid 145 | # - squeeze 146 | # - stretch 147 | # - wheezy 148 | # 149 | # Below are all categories currently available. Just as with 150 | # the platforms above, uncomment those that apply to your role. 151 | # 152 | categories: 153 | - cloud 154 | #- cloud:ec2 155 | #- cloud:gce 156 | #- cloud:rax 157 | - clustering 158 | #- database 159 | #- database:nosql 160 | #- database:sql 161 | - development 162 | #- monitoring 163 | #- networking 164 | #- packaging 165 | - system 166 | #- web 167 | dependencies: [] 168 | # List your role dependencies here, one per line. 169 | # Be sure to remove the '[]' above if you add dependencies 170 | # to this list. 171 | -------------------------------------------------------------------------------- /tasks/firewalld.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Enable Kubernetes Services in firewalld - Secure 3 | firewalld: port='{{ item }}' permanent=true state=enabled zone=public immediate=yes 4 | with_items: '{{ k8s_firewall_ports_secure }}' 5 | when: k8s_secure_master == True 6 | 7 | - name: Enable Kubernetes Services in firewalld - Insecure 8 | firewalld: port='{{ item }}' permanent=true state=enabled zone=public immediate=yes 9 | with_items: '{{ k8s_firewall_ports_insecure }}' 10 | when: k8s_secure_master == False 11 | -------------------------------------------------------------------------------- /tasks/insecure.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | #***install required packages 4 | - name: Ensure required kubernetes master packages are installed 5 | yum: name='{{ item }}' state=latest 6 | with_items: '{{ k8s_mst_packages }}' 7 | 8 | - name: Ensure cockpit-kubernetes is installed 9 | yum: name=cockpit-kubernetes state=latest 10 | when: k8s_cockpit == True 11 | notify: 12 | - Remove default cockpit packages 13 | - Setup CentOS7-Extras Repository for cockpit-kubernetes 14 | - Install cockpit-kubernetes 15 | - Disable CentOS7-Extras Repository post cockpit-kubernetes install 16 | - Enable cockpit 17 | - Restart cockpit 18 | - Enable Cockpit Service in firewalld 19 | 20 | #Configure Docker Storage 21 | - name: Configure Docker Storage 22 | template: src=insecure/sysconfig/docker-storage-setup.j2 dest=/etc/sysconfig/docker-storage-setup 23 | when: 24 | - k8s_mst_is_node == False 25 | - k8s_docker_storage_setup == True 26 | notify: Restart flanneld 27 | 28 | #****Configure general kubernetes settings 29 | #config 30 | - name: Configure etcd settings 31 | template: src=insecure/etcd/etcd.conf.j2 dest=/etc/etcd/etcd.conf 32 | notify: 33 | - Restart etcd 34 | - Restart kube-apiserver 35 | - Restart flanneld 36 | - Restart kube-proxy 37 | 38 | - name: Configure kubernetes general settings 39 | template: src=insecure/kubernetes/config.j2 dest=/etc/kubernetes/config 40 | notify: 41 | - Restart kube-scheduler 42 | - Restart kube-controller-manager 43 | - Restart kube-proxy 44 | - Restart flanneld 45 | 46 | #scheduler 47 | - name: Configure kube-scheduler settings 48 | template: src=insecure/kubernetes/scheduler.j2 dest=/etc/kubernetes/scheduler 49 | notify: Restart kube-scheduler 50 | 51 | #controller-manager 52 | - name: Configure kube-controller-manager settings 53 | template: src=insecure/kubernetes/controller-manager.j2 dest=/etc/kubernetes/controller-manager 54 | notify: Restart kube-controller-manager 55 | 56 | #***Configure kubernetes apiserver settings 57 | - name: Configure kube-apiserver settings 58 | template: src=insecure/kubernetes/apiserver.j2 dest=/etc/kubernetes/apiserver 59 | notify: 60 | - Restart kube-apiserver 61 | - Restart kube-scheduler 62 | - Restart kube-controller-manager 63 | - Restart kube-proxy 64 | - Restart flanneld 65 | 66 | #***Enable and Startup core services 67 | 68 | #etcd 69 | - name: Enable and Start etcd service 70 | service: name=etcd enabled=yes state=started 71 | 72 | #apiserver 73 | - name: Enable and Start kube-apiserver service 74 | service: name=kube-apiserver enabled=yes state=started 75 | 76 | #controller-manager 77 | - name: Enable and Start kube-controller-manager service 78 | service: name=kube-controller-manager enabled=yes state=started 79 | 80 | #scheduler 81 | - name: Enable and Start kube-scheduler service 82 | service: name=kube-scheduler enabled=yes state=started 83 | 84 | #***Setup flanneld backend network 85 | 86 | #check etcd key 87 | - name: Check to see if flanneld network config exists in etcd 88 | command: /usr/bin/etcdctl ls '{{ etcd_key }}'/config 89 | register: etcdctl_result 90 | changed_when: False 91 | ignore_errors: True 92 | 93 | - name: Move flanneld network config to master 94 | template: src=insecure/flannel/flanneld-conf.json.j2 dest=/etc/kubernetes/flanneld-conf.json 95 | 96 | - name: Configure flanneld network key in etcd 97 | shell: etcdctl --endpoint http://{{ ansible_hostname }}.{{ ansible_domain }}:{{ etcd_port }} set {{ etcd_key }}/config /etc/kubernetes/etcdctl-out.txt 103 | register: flannel_diff 104 | changed_when: False 105 | failed_when: "flannel_diff.rc != 0" 106 | 107 | - name: Verify flanneld network key 108 | shell: sed -i '$ d' /etc/kubernetes/etcdctl-out.txt && diff /etc/kubernetes/etcdctl-out.txt /etc/kubernetes/flanneld-conf.json 109 | changed_when: False 110 | 111 | #***Remaining Configuration 112 | #Configure local flannel settings 113 | - name: Configure local flanneld settings 114 | template: src=insecure/sysconfig/flanneld.j2 dest=/etc/sysconfig/flanneld 115 | when: k8s_mst_is_node == False 116 | 117 | #Configure kube-proxy 118 | - name: Configure kube-proxy 119 | template: src=insecure/kubernetes/proxy.j2 dest=/etc/kubernetes/proxy 120 | when: k8s_mst_is_node == False 121 | 122 | #***Remaining Services 123 | #Enable and startup flanneld 124 | - name: Enable flanneld service 125 | service: name=flanneld enabled=yes 126 | notify: Restart flanneld 127 | when: k8s_mst_is_node == False 128 | 129 | #Enable and startup kube-proxy 130 | - name: Enable kube-proxy service 131 | service: name=kube-proxy enabled=yes 132 | notify: Restart kube-proxy 133 | when: k8s_mst_is_node == False 134 | 135 | #docker - make sure disabled, when this is not a node 136 | - name: Ensure docker is not enabled 137 | service: name=docker enabled=no state=stopped 138 | when: k8s_mst_is_node == False 139 | -------------------------------------------------------------------------------- /tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | #Choose your own adventure 3 | - name: Include appropriate tasks for Secure Cluster Configuration 4 | include: tasks/secure.yml 5 | when: k8s_secure_master == True 6 | 7 | - name: Include appropriate tasks for Insecure Cluster Configuration 8 | include: tasks/insecure.yml 9 | when: k8s_secure_master == False 10 | 11 | #firewalld 12 | - name: Include firewalld tasks 13 | include: tasks/firewalld.yml 14 | when: k8s_firewalld == True 15 | -------------------------------------------------------------------------------- /tasks/secure.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | #***install required packages 4 | - name: Ensure required kubernetes master packages are installed 5 | yum: name='{{ item }}' state=latest 6 | with_items: '{{ k8s_mst_packages }}' 7 | 8 | - name: Ensure cockpit-kubernetes is installed 9 | yum: name=cockpit-kubernetes state=latest 10 | when: k8s_cockpit == True 11 | notify: 12 | - Remove default cockpit packages 13 | - Setup CentOS7-Extras Repository for cockpit-kubernetes 14 | - Install cockpit-kubernetes 15 | - Disable CentOS7-Extras Repository post cockpit-kubernetes install 16 | - Enable cockpit 17 | - Restart cockpit 18 | - Enable Cockpit Service in firewalld 19 | 20 | #Configure Docker Storage 21 | - name: Configure Docker Storage 22 | template: src=secure/sysconfig/docker-storage-setup.j2 dest=/etc/sysconfig/docker-storage-setup 23 | when: 24 | - k8s_mst_is_node == False 25 | - k8s_docker_storage_setup == True 26 | notify: Restart flanneld 27 | 28 | #***Configure general kubernetes settings 29 | #config 30 | - name: Configure etcd settings 31 | template: src=secure/etcd/etcd.conf.j2 dest=/etc/etcd/etcd.conf 32 | notify: 33 | - Restart etcd 34 | - Restart kube-apiserver 35 | - Restart flanneld 36 | - Restart kube-proxy 37 | 38 | - name: Configure kubernetes general settings 39 | template: src=secure/kubernetes/config.j2 dest=/etc/kubernetes/config 40 | notify: 41 | - Restart kube-scheduler 42 | - Restart kube-controller-manager 43 | - Restart kube-proxy 44 | - Restart flanneld 45 | 46 | #scheduler 47 | - name: Configure kube-scheduler settings 48 | template: src=secure/kubernetes/scheduler.j2 dest=/etc/kubernetes/scheduler 49 | notify: Restart kube-scheduler 50 | 51 | #controller-manager 52 | - name: Configure kube-controller-manager settings 53 | template: src=secure/kubernetes/controller-manager.j2 dest=/etc/kubernetes/controller-manager 54 | notify: Restart kube-controller-manager 55 | 56 | #***Configure secured apiserver and services 57 | #Check presence of apiserver certificate path 58 | - name: Check for presence of kube-apiserver certificate path 59 | stat: path="{{ k8s_apiserver_cert_path }}" 60 | register: apiserver_cert_path 61 | 62 | - name: Create kube-apiserver certificate path 63 | file: path="{{ k8s_apiserver_cert_path }}" state=directory mode=655 group=root owner=root 64 | 65 | #Check presence of apiserver certificates 66 | - name: Check for presence of kube-apiserver CA key 67 | stat: path="{{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_key_filename }}" 68 | register: apiserver_ca_key 69 | 70 | - name: Check for presence of kube-apiserver CA certificate 71 | stat: path="{{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }}" 72 | register: apiserver_ca_cert 73 | 74 | - name: Check for presence of kube-apiserver server key 75 | stat: path="{{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }}" 76 | register: apiserver_server_key 77 | 78 | - name: Check for presence of kube-apiserver server certificate signing request 79 | stat: path="{{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_csr_filename }}" 80 | register: apiserver_server_csr 81 | 82 | - name: Check for presence of kube-apiserver server certificate 83 | stat: path="{{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_cert_filename }}" 84 | register: apiserver_server_cert 85 | 86 | #Create openssl config 87 | - name: Create Custom OpenSSL Config for kubernetes master certificates 88 | template: src=secure/openssl/openssl.conf.j2 dest={{ k8s_apiserver_cert_path }}/openssl.conf 89 | 90 | #Generate apiserver certificates ref: http://kubernetes.io/docs/admin/authentication/ 91 | - name: Create local certificate path 92 | file: 93 | path: files/k8s-master-certs/{{ ansible_hostname }}.{{ ansible_domain }} 94 | state: directory 95 | delegate_to: localhost 96 | 97 | - name: Generate kube-apiserver CA key 98 | shell: openssl genrsa -out {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_key_filename }} 2048 99 | when: "apiserver_ca_key.stat.exists != True" 100 | notify: 101 | - Get CA key 102 | - Store CA key 103 | - Restart kube-apiserver 104 | - Restart kube-scheduler 105 | - Restart kube-controller-manager 106 | - Restart kube-proxy 107 | 108 | - name: Generate kube-apiserver CA certificate 109 | shell: openssl req -x509 -new -nodes -key {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_key_filename }} -subj "/CN='{{ ansible_hostname }}'" -days 10000 -out {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }} -config {{ k8s_apiserver_cert_path }}/openssl.conf 110 | when: "apiserver_ca_cert.stat.exists != True" 111 | notify: 112 | - Get CA cert 113 | - Store CA cert 114 | - Restart kube-apiserver 115 | - Restart kube-scheduler 116 | - Restart kube-controller-manager 117 | - Restart kube-proxy 118 | 119 | - name: Generate kube-apiserver server key 120 | shell: openssl genrsa -out {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }} 2048 121 | when: "apiserver_server_key.stat.exists != True" 122 | notify: 123 | - Restart kube-apiserver 124 | - Restart kube-scheduler 125 | - Restart kube-controller-manager 126 | - Restart kube-proxy 127 | 128 | - name: Generate kube-apiserver server certificate signing request 129 | shell: openssl req -new -key {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }} -subj "/CN='{{ ansible_hostname }}'" -out {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_csr_filename }} 130 | when: "apiserver_server_csr.stat.exists != True" 131 | notify: 132 | - Restart kube-apiserver 133 | - Restart kube-scheduler 134 | - Restart kube-controller-manager 135 | - Restart kube-proxy 136 | 137 | - name: Generate kube-apiserver server certificate 138 | shell: openssl x509 -req -in {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_csr_filename }} -CA {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }} -CAkey {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_key_filename }} -CAcreateserial -out {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_cert_filename }} -days 10000 -extensions v3_req -extfile {{ k8s_apiserver_cert_path }}/openssl.conf 139 | when: "apiserver_server_cert.stat.exists != True" 140 | notify: 141 | - Restart kube-apiserver 142 | - Restart kube-scheduler 143 | - Restart kube-controller-manager 144 | - Restart kube-proxy 145 | 146 | - name: Ensure proper permissions on apiserver certificates 147 | file: dest="{{ k8s_apiserver_cert_path }}/{{ item }}" mode=644 owner=root group=root state=file 148 | with_items: 149 | - '{{ k8s_apiserver_ca_cert_filename }}' 150 | - '{{ k8s_apiserver_server_key_filename }}' 151 | - '{{ k8s_apiserver_server_csr_filename }}' 152 | - '{{ k8s_apiserver_server_cert_filename }}' 153 | notify: 154 | - Restart kube-apiserver 155 | - Restart kube-scheduler 156 | - Restart kube-controller-manager 157 | - Restart flanneld 158 | - Restart kube-proxy 159 | 160 | - name: Ensure proper permissions on CA key 161 | file: dest="{{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_key_filename }}" mode=600 owner=root group=root state=file 162 | 163 | 164 | #Update CA Trust 165 | - name: Copy CA Certificate to Trusted Sources 166 | shell: cp {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }} /etc/pki/ca-trust/source/anchors/{{ k8s_apiserver_ca_cert_filename }} 167 | when: "apiserver_ca_cert.stat.exists != True" 168 | 169 | - name: Update CA Trust 170 | shell: update-ca-trust extract 171 | when: "apiserver_ca_cert.stat.exists != True" 172 | 173 | - name: Restart etcd for kube-apiserver CA change 174 | service: name=etcd state=restarted 175 | when: "apiserver_ca_cert.stat.exists != True" 176 | 177 | #***Configure kubernetes components with kubeconfigs 178 | #proxy 179 | - name: Configure kube-proxy with secure kubeconfig 180 | template: src=secure/kubernetes/proxy.kubeconfig.j2 dest=/etc/kubernetes/{{ k8s_proxy_kubeconfig_filename }} 181 | notify: 182 | - Restart flanneld 183 | - Restart kube-proxy 184 | when: k8s_mst_is_node == False 185 | 186 | #scheduler 187 | - name: Configure kube-scheduler with secure kubeconfig 188 | template: src=secure/kubernetes/scheduler.kubeconfig.j2 dest=/etc/kubernetes/{{ k8s_scheduler_kubeconfig_filename }} 189 | notify: 190 | - Restart kube-scheduler 191 | 192 | #controller-manager 193 | - name: Configure kube-controller-manager with secure kubeconfig 194 | template: src=secure/kubernetes/controller-manager.kubeconfig.j2 dest=/etc/kubernetes/{{ k8s_controller_manager_kubeconfig_filename }} 195 | notify: 196 | - Restart kube-controller-manager 197 | 198 | #***Configure kubernetes apiserver settings 199 | - name: Configure kube-apiserver settings 200 | template: src=secure/kubernetes/apiserver.j2 dest=/etc/kubernetes/apiserver 201 | notify: 202 | - Restart kube-apiserver 203 | - Restart kube-scheduler 204 | - Restart kube-controller-manager 205 | - Restart kube-proxy 206 | - Restart flanneld 207 | 208 | #***Enable and Startup core services 209 | #etcd 210 | - name: Enable and Start etcd service 211 | service: name=etcd enabled=yes state=started 212 | 213 | #apiserver 214 | - name: Enable and Start kube-apiserver service 215 | service: name=kube-apiserver enabled=yes state=started 216 | 217 | #controller-manager 218 | - name: Enable and Start kube-controller-manager service 219 | service: name=kube-controller-manager enabled=yes state=started 220 | 221 | #scheduler 222 | - name: Enable and Start kube-scheduler service 223 | service: name=kube-scheduler enabled=yes state=started 224 | 225 | #***Setup flanneld backend network 226 | #check etcd key 227 | - name: Check to see if flanneld network config exists in etcd 228 | command: etcdctl --cert-file {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_cert_filename }} --key-file {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }} --ca-file {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }} --endpoint=https://{{ ansible_hostname }}.{{ ansible_domain }}:{{ etcd_port }} ls '{{ etcd_key }}'/config 229 | register: etcdctl_result 230 | changed_when: False 231 | ignore_errors: True 232 | 233 | - name: Move flanneld network config to master 234 | template: src=secure/flannel/flanneld-conf.json.j2 dest=/etc/kubernetes/flanneld-conf.json 235 | 236 | - name: Configure flanneld network key in etcd secure 237 | shell: etcdctl --cert-file {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_cert_filename }} --key-file {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }} --ca-file {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }} --endpoint=https://{{ ansible_hostname }}.{{ ansible_domain }}:{{ etcd_port }} set {{ etcd_key }}/config /etc/kubernetes/etcdctl-out.txt 244 | changed_when: False 245 | 246 | - name: Verify flanneld network key 247 | shell: sed -i '$ d' /etc/kubernetes/etcdctl-out.txt && diff /etc/kubernetes/etcdctl-out.txt /etc/kubernetes/flanneld-conf.json 248 | register: flannel_diff 249 | changed_when: False 250 | failed_when: "flannel_diff.rc != 0" 251 | 252 | #***Remaining Configuration 253 | #Configure local flannel settings 254 | - name: Configure local flanneld settings 255 | template: src=secure/sysconfig/flanneld.j2 dest=/etc/sysconfig/flanneld 256 | when: k8s_mst_is_node == False 257 | notify: 258 | - Restart flanneld 259 | 260 | #Configure kube-proxy 261 | - name: Configure kube-proxy 262 | template: src=secure/kubernetes/proxy.j2 dest=/etc/kubernetes/proxy 263 | when: k8s_mst_is_node == False 264 | notify: 265 | - Restart flanneld 266 | - Restart kube-proxy 267 | 268 | #***Remaining Services 269 | #Enable and startup flanneld 270 | - name: Enable flanneld service 271 | service: name=flanneld enabled=yes 272 | notify: Restart flanneld 273 | when: k8s_mst_is_node == False 274 | 275 | #Enable and startup kube-proxy 276 | - name: Enable kube-proxy service 277 | service: name=kube-proxy enabled=yes 278 | notify: Restart kube-proxy 279 | when: k8s_mst_is_node == False 280 | 281 | #docker - make sure disabled, when master is not a node 282 | - name: Ensure docker is not enabled 283 | service: name=docker enabled=no state=stopped 284 | when: k8s_mst_is_node == False 285 | -------------------------------------------------------------------------------- /templates/insecure/etcd/etcd.conf.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # [member] 3 | ETCD_NAME=default 4 | ETCD_DATA_DIR="/var/lib/etcd/default.etcd" 5 | #ETCD_SNAPSHOT_COUNTER="10000" 6 | #ETCD_HEARTBEAT_INTERVAL="100" 7 | #ETCD_ELECTION_TIMEOUT="1000" 8 | #ETCD_LISTEN_PEER_URLS="http://localhost:2380" 9 | ETCD_LISTEN_CLIENT_URLS="http://localhost:{{ etcd_port }},http://{{ ansible_hostname }}.{{ ansible_domain }}:{{ etcd_port }}" 10 | #ETCD_MAX_SNAPSHOTS="5" 11 | #ETCD_MAX_WALS="5" 12 | #ETCD_CORS="" 13 | # 14 | #[cluster] 15 | #ETCD_INITIAL_ADVERTISE_PEER_URLS="http://localhost:2380" 16 | # if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..." 17 | #ETCD_INITIAL_CLUSTER="default=http://localhost:2380" 18 | #ETCD_INITIAL_CLUSTER_STATE="new" 19 | #ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" 20 | ETCD_ADVERTISE_CLIENT_URLS="http://localhost:{{ etcd_port }},http://{{ ansible_hostname }}.{{ ansible_domain }}:{{ etcd_port }}" 21 | #ETCD_DISCOVERY="" 22 | #ETCD_DISCOVERY_SRV="" 23 | #ETCD_DISCOVERY_FALLBACK="proxy" 24 | #ETCD_DISCOVERY_PROXY="" 25 | # 26 | #[proxy] 27 | #ETCD_PROXY="off" 28 | # 29 | #[security] 30 | #ETCD_CERT_FILE="" 31 | #ETCD_KEY_FILE="" 32 | #ETCD_CLIENT_CERT_AUTH="false" 33 | #ETCD_TRUSTED_CA_FILE="" 34 | #ETCD_PEER_CERT_FILE="" 35 | #ETCD_PEER_KEY_FILE="" 36 | #ETCD_PEER_CLIENT_CERT_AUTH="false" 37 | #ETCD_PEER_TRUSTED_CA_FILE="" 38 | # 39 | #[logging] 40 | #ETCD_DEBUG="false" 41 | # examples for -log-package-levels etcdserver=WARNING,security=DEBUG 42 | #ETCD_LOG_PACKAGE_LEVELS="" 43 | -------------------------------------------------------------------------------- /templates/insecure/flannel/flanneld-conf.json.j2: -------------------------------------------------------------------------------- 1 | { 2 | "Network": "{{ flannel_backend_network }}", 3 | "SubnetLen": {{ flannel_subnet_length }}, 4 | "Backend": { 5 | "Type": "vxlan" 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /templates/insecure/kubernetes/apiserver.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | ### 3 | # kubernetes system config 4 | # 5 | # The following values are used to configure the kube-apiserver 6 | # 7 | 8 | # The address on the local server to listen to. 9 | KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0" 10 | 11 | # The port on the local server to listen on. 12 | # KUBE_API_PORT="--port=8080" 13 | {% if k8s_apiserver_insecure_port %} 14 | KUBE_API_PORT="--insecure-port={{ k8s_apiserver_insecure_port }}" 15 | {% endif %} 16 | 17 | # Port minions listen on 18 | # KUBELET_PORT="--kubelet_port=10250" 19 | {% if k8s_kubelet_port %} 20 | KUBELET_PORT="--kubelet_port={{ k8s_kubelet_port }}" 21 | {% endif %} 22 | 23 | # Comma separated list of nodes in the etcd cluster 24 | KUBE_ETCD_SERVERS="--etcd_servers=http://{{ ansible_hostname }}.{{ ansible_domain }}:{{ etcd_port }}" 25 | 26 | # Address range to use for services 27 | KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range={{ k8s_service_network }}" 28 | 29 | # default admission control policies 30 | KUBE_ADMISSION_CONTROL="--admission_control={{ k8s_admission_control }}" 31 | 32 | # Add your own! 33 | KUBE_API_ARGS="{% if k8s_secure_master == False %}--secure-port=0{% endif %}{% if k8s_auth_mode %} --authorization_mode={{ k8s_auth_mode }}{% endif %}{% if k8s_auth_policy_file %} --authorization_policy_file={{ k8s_auth_policy_file }}{% endif %}{% if k8s_token_auth_file %} --token_auth_file={{ k8s_token_auth_file }}{% endif %}{% if k8s_apiserver_additional_args %} {{ k8s_apiserver_additional_args }}{% endif %}" 34 | -------------------------------------------------------------------------------- /templates/insecure/kubernetes/config.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | ### 3 | # kubernetes system config 4 | # 5 | # The following values are used to configure various aspects of all 6 | # kubernetes services, including 7 | # 8 | # kube-apiserver.service 9 | # kube-controller-manager.service 10 | # kube-scheduler.service 11 | # kubelet.service 12 | # kube-proxy.service 13 | # logging to stderr means we get it in the systemd journal 14 | KUBE_LOGTOSTDERR="--logtostderr={{ k8s_logtostderr }}" 15 | 16 | # journal message level, 0 is debug 17 | KUBE_LOG_LEVEL="--v={{ k8s_log_level }}" 18 | 19 | # Should this cluster be allowed to run privileged docker containers 20 | KUBE_ALLOW_PRIV="--allow_privileged={{ k8s_allow_privileged }}" 21 | 22 | # How the controller-manager, scheduler, and proxy find the apiserver 23 | KUBE_MASTER="--master=http://{{ ansible_hostname }}.{{ ansible_domain }}:{{ k8s_apiserver_insecure_port }}" 24 | -------------------------------------------------------------------------------- /templates/insecure/kubernetes/controller-manager.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | ### 3 | # The following values are used to configure the kubernetes controller-manager 4 | 5 | # defaults from config and apiserver should be adequate 6 | 7 | # Add your own! 8 | KUBE_CONTROLLER_MANAGER_ARGS="{% if k8s_controller_manager_additional_args %}{{ k8s_controller_manager_additional_args }}{% endif %}" 9 | -------------------------------------------------------------------------------- /templates/insecure/kubernetes/proxy.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | ### 3 | # kubernetes proxy config 4 | 5 | # default config should be adequate 6 | 7 | # Add your own! 8 | KUBE_PROXY_ARGS="{% if k8s_kube_proxy_additional_args %}{{ k8s_kube_proxy_additional_args }}{% endif %}" 9 | -------------------------------------------------------------------------------- /templates/insecure/kubernetes/scheduler.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | ### 3 | # kubernetes scheduler config 4 | 5 | # default config should be adequate 6 | 7 | # Add your own! 8 | KUBE_SCHEDULER_ARGS="{% if k8s_scheduler_additional_args %}{{ k8s_scheduler_additional_args }}{% endif %}" 9 | -------------------------------------------------------------------------------- /templates/insecure/sysconfig/docker-storage-setup.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # Edit this file to override any configuration options specified in 3 | # /usr/lib/docker-storage-setup/docker-storage-setup. 4 | # 5 | # For more details refer to "man docker-storage-setup" 6 | DEVS={{ k8s_docker_storage_disk }} 7 | VG={{ k8s_docker_storage_vg }} 8 | {% for i in k8s_docker_storage_options %} 9 | {{ i }} 10 | {% endfor %} 11 | -------------------------------------------------------------------------------- /templates/insecure/sysconfig/flanneld.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # Flanneld configuration options 3 | 4 | # Any additional options that you want to pass 5 | FLANNEL_OPTIONS="-etcd-prefix {{ etcd_key }} -etcd-endpoints http://{{ ansible_hostname }}.{{ ansible_domain }}:{{ etcd_port }}" 6 | -------------------------------------------------------------------------------- /templates/secure/etcd/etcd.conf.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # [member] 3 | ETCD_NAME=default 4 | ETCD_DATA_DIR="/var/lib/etcd/default.etcd" 5 | #ETCD_SNAPSHOT_COUNTER="10000" 6 | #ETCD_HEARTBEAT_INTERVAL="100" 7 | #ETCD_ELECTION_TIMEOUT="1000" 8 | ETCD_LISTEN_PEER_URLS="http://localhost:2380" 9 | ETCD_LISTEN_CLIENT_URLS="https://localhost:{{ etcd_port }},https://{{ ansible_hostname }}.{{ ansible_domain }}:{{ etcd_port }}" 10 | #ETCD_MAX_SNAPSHOTS="5" 11 | #ETCD_MAX_WALS="5" 12 | #ETCD_CORS="" 13 | # 14 | #[cluster] 15 | #ETCD_INITIAL_ADVERTISE_PEER_URLS="http://localhost:2380" 16 | # if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..." 17 | #ETCD_INITIAL_CLUSTER="default=http://localhost:2380" 18 | #ETCD_INITIAL_CLUSTER_STATE="new" 19 | #ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" 20 | ETCD_ADVERTISE_CLIENT_URLS="https://localhost:{{ etcd_port }},https://{{ ansible_hostname }}.{{ ansible_domain }}:{{ etcd_port }}" 21 | #ETCD_DISCOVERY="" 22 | #ETCD_DISCOVERY_SRV="" 23 | #ETCD_DISCOVERY_FALLBACK="proxy" 24 | #ETCD_DISCOVERY_PROXY="" 25 | # 26 | #[proxy] 27 | #ETCD_PROXY="off" 28 | # 29 | #[security] 30 | ETCD_CERT_FILE="{% if k8s_apiserver_cert_path and k8s_apiserver_server_cert_filename %}{{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_cert_filename }}"{% endif %} 31 | 32 | ETCD_KEY_FILE="{% if k8s_apiserver_cert_path and k8s_apiserver_server_key_filename %}{{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }}{% endif %}" 33 | ETCD_CLIENT_CERT_AUTH="true" 34 | ETCD_TRUSTED_CA_FILE="{% if k8s_apiserver_cert_path and k8s_apiserver_ca_cert_filename %}{{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }}{% endif %}" 35 | #ETCD_PEER_CERT_FILE="" 36 | #ETCD_PEER_KEY_FILE="" 37 | #ETCD_PEER_CLIENT_CERT_AUTH="false" 38 | #ETCD_PEER_TRUSTED_CA_FILE="" 39 | # 40 | #[logging] 41 | #ETCD_DEBUG="false" 42 | # examples for -log-package-levels etcdserver=WARNING,security=DEBUG 43 | #ETCD_LOG_PACKAGE_LEVELS="" 44 | -------------------------------------------------------------------------------- /templates/secure/flannel/flanneld-conf.json.j2: -------------------------------------------------------------------------------- 1 | { 2 | "Network": "{{ flannel_backend_network }}", 3 | "SubnetLen": {{ flannel_subnet_length }}, 4 | "Backend": { 5 | "Type": "vxlan" 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /templates/secure/kubernetes/apiserver.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | ### 3 | # kubernetes system config 4 | # 5 | # The following values are used to configure the kube-apiserver 6 | # 7 | 8 | # The address on the local server to listen to. 9 | KUBE_API_ADDRESS="--bind-address={{ ansible_default_ipv4.address }}" 10 | 11 | # The port on the local server to listen on. 12 | # KUBE_API_PORT="--port=8080" 13 | {% if k8s_apiserver_insecure_port %} 14 | KUBE_API_PORT="--insecure-port={{ k8s_apiserver_insecure_port }}" 15 | {% endif %} 16 | 17 | # Port minions listen on 18 | # KUBELET_PORT="--kubelet_port=10250" 19 | {% if k8s_kubelet_port %} 20 | KUBELET_PORT="--kubelet_port={{ k8s_kubelet_port }}" 21 | {% endif %} 22 | 23 | # Comma separated list of nodes in the etcd cluster 24 | KUBE_ETCD_SERVERS="--etcd_servers=https://{{ ansible_hostname }}.{{ ansible_domain }}:{{ etcd_port }}{% if k8s_apiserver_cert_path and k8s_apiserver_ca_cert_filename %} --etcd-cafile={{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }}{% endif %}{% if k8s_apiserver_cert_path and k8s_apiserver_server_cert_filename %} --etcd-certfile={{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_cert_filename }}{% endif %}{% if k8s_apiserver_cert_path and k8s_apiserver_server_key_filename %} --etcd-keyfile={{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }}{% endif %}" 25 | 26 | # Address range to use for services 27 | {% if k8s_service_network %} 28 | KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range={{ k8s_service_network }}" 29 | {% else %} 30 | KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=" 31 | {% endif %} 32 | 33 | # default admission control policies 34 | {% if k8s_admission_control %} 35 | KUBE_ADMISSION_CONTROL="--admission_control={{ k8s_admission_control }}" 36 | {% else %} 37 | KUBE_ADMISSION_CONTROL="--admission_control=" 38 | {% endif %} 39 | 40 | # Add your own! 41 | KUBE_API_ARGS="{% if k8s_auth_mode %} --authorization_mode={{ k8s_auth_mode }}{% endif %}{% if k8s_auth_policy_file %} --authorization_policy_file={{ k8s_auth_policy_file }}{% endif %}{% if k8s_token_auth_file %} --token_auth_file={{ k8s_token_auth_file }}{% endif %}{% if k8s_apiserver_secure_port %} --secure-port={{ k8s_apiserver_secure_port }}{% endif %}{% if k8s_apiserver_cert_path and k8s_apiserver_server_cert_filename %} --tls_cert_file={{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_cert_filename }}{% endif %}{% if k8s_apiserver_cert_path and k8s_apiserver_server_key_filename %} --tls_private_key_file={{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }}{% endif %}{% if k8s_apiserver_cert_path and k8s_apiserver_ca_cert_filename %} --client_ca_file={{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }}{% endif %}{% if k8s_apiserver_cert_path and k8s_apiserver_server_key_filename %} --service_account_key_file={{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }}{% endif %}{% if k8s_apiserver_additional_args %} {{ k8s_apiserver_additional_args }}{% endif %}" 42 | -------------------------------------------------------------------------------- /templates/secure/kubernetes/config.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | ### 3 | # kubernetes system config 4 | # 5 | # The following values are used to configure various aspects of all 6 | # kubernetes services, including 7 | # 8 | # kube-apiserver.service 9 | # kube-controller-manager.service 10 | # kube-scheduler.service 11 | # kubelet.service 12 | # kube-proxy.service 13 | # logging to stderr means we get it in the systemd journal 14 | KUBE_LOGTOSTDERR="--logtostderr={{ k8s_logtostderr }}" 15 | 16 | # journal message level, 0 is debug 17 | KUBE_LOG_LEVEL="--v={{ k8s_log_level }}" 18 | 19 | # Should this cluster be allowed to run privileged docker containers 20 | KUBE_ALLOW_PRIV="--allow_privileged={{ k8s_allow_privileged }}" 21 | 22 | # How the controller-manager, scheduler, and proxy find the apiserver 23 | KUBE_MASTER="--master=https://{{ ansible_hostname }}.{{ ansible_domain }}:{{ k8s_apiserver_secure_port }}" 24 | -------------------------------------------------------------------------------- /templates/secure/kubernetes/controller-manager.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | ### 3 | # The following values are used to configure the kubernetes controller-manager 4 | 5 | # defaults from config and apiserver should be adequate 6 | 7 | # Add your own! 8 | KUBE_CONTROLLER_MANAGER_ARGS="--service_account_private_key_file={{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }} --kubeconfig=/etc/kubernetes/{{ k8s_controller_manager_kubeconfig_filename }}{% if k8s_controller_manager_additional_args %} {{ k8s_controller_manager_additional_args }}{% endif %}" 9 | -------------------------------------------------------------------------------- /templates/secure/kubernetes/controller-manager.kubeconfig.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | apiVersion: v1 3 | kind: Config 4 | clusters: 5 | - cluster: 6 | certificate-authority: {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }} 7 | server: https://{{ ansible_hostname }}.{{ ansible_domain }}:{{ k8s_apiserver_secure_port }} 8 | name: kubernetes 9 | contexts: 10 | - context: 11 | cluster: kubernetes 12 | user: controller-manager 13 | name: controller-manager-ctx 14 | current-context: controller-manager-ctx 15 | users: 16 | - name: controller-manager 17 | user: 18 | client-certificate: {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_cert_filename }} 19 | client-key: {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }} 20 | -------------------------------------------------------------------------------- /templates/secure/kubernetes/proxy.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | ### 3 | # kubernetes proxy config 4 | 5 | # default config should be adequate 6 | 7 | # Add your own! 8 | KUBE_PROXY_ARGS="--kubeconfig=/etc/kubernetes/{{ k8s_proxy_kubeconfig_filename }}{% if k8s_kube_proxy_additional_args %} {{ k8s_kube_proxy_additional_args }}{% endif %}" 9 | -------------------------------------------------------------------------------- /templates/secure/kubernetes/proxy.kubeconfig.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | apiVersion: v1 3 | kind: Config 4 | clusters: 5 | - cluster: 6 | certificate-authority: {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }} 7 | server: https://{{ ansible_hostname }}.{{ ansible_domain }}:{{ k8s_apiserver_secure_port }} 8 | name: kubernetes 9 | contexts: 10 | - context: 11 | cluster: kubernetes 12 | user: proxy 13 | name: proxy-ctx 14 | current-context: proxy-ctx 15 | users: 16 | - name: proxy 17 | user: 18 | client-certificate: {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_cert_filename }} 19 | client-key: {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }} 20 | -------------------------------------------------------------------------------- /templates/secure/kubernetes/scheduler.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | ### 3 | # kubernetes scheduler config 4 | 5 | # default config should be adequate 6 | 7 | # Add your own! 8 | KUBE_SCHEDULER_ARGS="--kubeconfig=/etc/kubernetes/{{ k8s_scheduler_kubeconfig_filename }}{% if k8s_scheduler_additional_args %} {{ k8s_scheduler_additional_args }}{% endif %}" 9 | -------------------------------------------------------------------------------- /templates/secure/kubernetes/scheduler.kubeconfig.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | apiVersion: v1 3 | kind: Config 4 | clusters: 5 | - cluster: 6 | certificate-authority: {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }} 7 | server: https://{{ ansible_hostname }}.{{ ansible_domain }}:{{ k8s_apiserver_secure_port }} 8 | name: kubernetes 9 | contexts: 10 | - context: 11 | cluster: kubernetes 12 | user: scheduler 13 | name: scheduler-ctx 14 | current-context: scheduler-ctx 15 | users: 16 | - name: scheduler 17 | user: 18 | client-certificate: {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_cert_filename }} 19 | client-key: {{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }} 20 | -------------------------------------------------------------------------------- /templates/secure/openssl/openssl.conf.j2: -------------------------------------------------------------------------------- 1 | [req] 2 | req_extensions = v3_req 3 | distinguished_name = req_distinguished_name 4 | [req_distinguished_name] 5 | 6 | [v3_req] 7 | 8 | # Extensions to add to a certificate request 9 | basicConstraints = CA:FALSE 10 | keyUsage = nonRepudiation, digitalSignature, keyEncipherment 11 | subjectAltName = @alt_names 12 | 13 | [alt_names] 14 | {% if k8s_apiserver_dns_names is defined %} 15 | {% set dns_count = 1 %} 16 | {% for name in k8s_apiserver_dns_names %} 17 | DNS.{{ dns_count }} = {{ name }} 18 | {% set dns_count = dns_count +1 %} 19 | {% endfor %} 20 | {% endif %} 21 | IP.1 = {{ ansible_default_ipv4.address }} 22 | {% if k8s_apiserver_additional_ips is defined %} 23 | {% set ip_count = 2 %} 24 | {% for ip in k8s_apiserver_additional_ips %} 25 | IP.{{ ip_count }} = {{ ip }} 26 | {% set ip_count = ip_count +1 %} 27 | {% endfor %} 28 | {% endif %} 29 | -------------------------------------------------------------------------------- /templates/secure/sysconfig/docker-storage-setup.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # Edit this file to override any configuration options specified in 3 | # /usr/lib/docker-storage-setup/docker-storage-setup. 4 | # 5 | # For more details refer to "man docker-storage-setup" 6 | DEVS={{ k8s_docker_storage_disk }} 7 | VG={{ k8s_docker_storage_vg }} 8 | {% for i in k8s_docker_storage_options %} 9 | {{ i }} 10 | {% endfor %} 11 | -------------------------------------------------------------------------------- /templates/secure/sysconfig/flanneld.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # Flanneld configuration options 3 | 4 | FLANNEL_OPTIONS="-etcd-prefix {{ etcd_key }} -etcd-endpoints https://{{ ansible_hostname }}.{{ ansible_domain }}:{{ etcd_port }}{% if k8s_apiserver_cert_path and k8s_apiserver_ca_cert_filename %} -etcd-cafile={{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_ca_cert_filename }}{% endif %}{% if k8s_apiserver_cert_path and k8s_apiserver_server_cert_filename %} -etcd-certfile={{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_cert_filename }}{% endif %}{% if k8s_apiserver_cert_path and k8s_apiserver_server_key_filename %} -etcd-keyfile={{ k8s_apiserver_cert_path }}/{{ k8s_apiserver_server_key_filename }}{% endif %}" 5 | -------------------------------------------------------------------------------- /vars/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | k8s_mst_packages: 4 | - etcd 5 | - kubernetes-master 6 | - kubernetes-node 7 | - flannel 8 | - openssl 9 | 10 | cockpit_kubernetes_pkg: 11 | - cockpit-kubernetes 12 | - cockpit 13 | 14 | #For ripping and replacing RHEL cockpit due to dependencies 15 | cockpit_default: 16 | - cockpit-shell 17 | - cockpit-bridge 18 | - cockpit-ws 19 | - cockpit 20 | - cockpit-networkmanager 21 | - cockpit-storaged 22 | - cockpit-docker 23 | 24 | #firewall services to open 25 | k8s_firewall_ports_secure: 26 | - '{{ etcd_port }}/tcp' #etcd 27 | - '{{ k8s_apiserver_secure_port }}/tcp' #kube-apiserver 28 | - '{{ k8s_kubelet_port }}/tcp' #kubelet 29 | 30 | k8s_firewall_ports_insecure: 31 | - '{{ etcd_port }}/tcp' #etcd 32 | - '{{ k8s_apiserver_insecure_port }}/tcp' #kube-apiserver 33 | - '{{ k8s_kubelet_port }}/tcp' #kubelet 34 | --------------------------------------------------------------------------------