├── README.md ├── centos-7-base.json ├── centos7 ├── main.tf ├── metadata.json ├── network_config.cfg ├── output.tf ├── userdata.tpl └── variables.tf ├── floppy └── centos7.ks ├── iso └── .gitkeep └── provision └── centos-7-install-package.sh /README.md: -------------------------------------------------------------------------------- 1 | # create-and-deploy-esxi 2 | Creating an image using Packer and deploying to an esxi server using Terraform 3 | ## Build for CentOS VMs on ESXI. 4 | ### 1) Configure ESXI server 5 | First, configure the ESXI host to allow Packer to find IP addresses of VMs: 6 | ```sh 7 | esxcli system settings advanced set -o /Net/GuestIPHack -i 1 8 | ``` 9 | Next, open VNC ports on the ESXI host firewall: 10 | ```sh 11 | chmod 644 /etc/vmware/firewall/service.xml 12 | chmod +t /etc/vmware/firewall/service.xml 13 | vi /etc/vmware/firewall/service.xml 14 | ``` 15 | Add this to bottom of the file (above ): 16 | ```sh 17 | 18 | packer-vnc 19 | 20 | inbound 21 | tcp 22 | dst 23 | 24 | 5900 25 | 6000 26 | 27 | 28 | true 29 | true 30 | 31 | ``` 32 | Reload the firewall: 33 | ```sh 34 | chmod 444 /etc/vmware/firewall/service.xml 35 | esxcli network firewall refresh 36 | ``` 37 | For the ISO, either add it to the ./iso directory or let Packer pull it remotely. 38 | 39 | ### Virtual machine where the build and deployment will take place 40 | ```sh 41 | centos 7 x86_64 minimal (selinux disabled) 42 | yum install unzip git -y 43 | curl -O https://releases.hashicorp.com/packer/1.5.5/packer_1.5.5_linux_amd64.zip 44 | unzip packer_1.5.5_linux_amd64.zip -d /usr/bin && rm -rf packer_1.5.5_linux_amd64.zip 45 | packer version 46 | Packer v1.3.5 47 | ``` 48 | ### Troubleshooting 49 | >On some RedHat-based Linux distributions there is another tool named packer installed by default. You can check for this >using which -a packer. If you get an error like this it indicates there is a name conflict. 50 | ```sh 51 | which -a packer 52 | /usr/sbin/packer 53 | ``` 54 | >To fix this, you can create a symlink to packer that uses a different name like packer.io, or invoke the packer binary you >want using its absolute path, e.g. /usr/bin/packer. 55 | ### Download and install ovftool https://www.vmware.com/support/developer/ovf/ 56 | ```sh 57 | install ovftoos 58 | chmod +x VMware-ovftool-4.4.0-15722219-lin.x86_64.bundle 59 | ./VMware-ovftool-4.4.0-15722219-lin.x86_64.bundle 60 | Extracting VMware Installer...done. 61 | You must accept the VMware OVF Tool component for Linux End User 62 | License Agreement to continue. Press Enter to proceed. 63 | VMWARE END USER LICENSE AGREEMENT 64 | Do you agree? [yes/no]:yes 65 | The product is ready to be installed. Press Enter to begin 66 | installation or Ctrl-C to cancel. 67 | Installing VMware OVF Tool component for Linux 4.4.0 68 | Configuring... 69 | [######################################################################] 100% 70 | Installation was successful. 71 | ``` 72 | 73 | # Build 74 | ```sh 75 | git clone https://github.com/letnab/create-and-deploy-esxi.git && cd create-and-deploy-esxi 76 | packer build centos-7-base.json 77 | ``` 78 | If for some reason it does not work, you need to replace 79 | ```sh 80 | centos-7-base.json 81 | - "boot_command": [ 82 | - " text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/floppy/centos7.ks" 83 | - ] 84 | 85 | + "boot_command": [ 86 | + " text biosdevname=0 net.ifnames=0 ks=hd:fd0:/centos7.ks" 87 | + ], 88 | + "floppy_files": [ 89 | + "floppy/centos7.ks" 90 | + ] 91 | ``` 92 | #### If successful, in the output-packer-centos-7-x86_64/ folder.ova/ will be ova file, packer-centos-7-x86_64.ova 93 | ### 2) Provider preparation and build for ESXI 94 | #### Download terraform 95 | ```sh 96 | curl -O https://releases.hashicorp.com/terraform/0.12.24/terraform_0.12.24_linux_amd64.zip 97 | unzip terraform_0.12.24_linux_amd64.zip -d /usr/bin/ && rm -rf terraform_0.12.24_linux_amd64.zip 98 | terraform version 99 | Terraform v0.12.24 100 | ``` 101 | #### Install golang 102 | ```sh 103 | cd /tmp 104 | curl -O https://dl.google.com/go/go1.14.2.linux-amd64.tar.gz 105 | tar -C /usr/local -xzf go1.14.2.linux-amd64.tar.gz && rm -rf go1.14.2.linux-amd64.tar.gz 106 | export PATH=$PATH:/usr/local/go/bin 107 | go version 108 | go version go1.14.2 linux/amd64 109 | ``` 110 | #### Build provider from ESXi 111 | ```sh 112 | go get -u -v golang.org/x/crypto/ssh 113 | go get -u -v github.com/hashicorp/terraform 114 | go get -u -v github.com/josenk/terraform-provider-esxi 115 | export GOPATH="$HOME/go" 116 | cd $GOPATH/src/github.com/josenk/terraform-provider-esxi 117 | CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -ldflags '-w -extldflags "-static"' -o terraform-provider-esxi_`cat version` 118 | cp terraform-provider-esxi_`cat version` /usr/bin 119 | ``` 120 | ### Usage 121 | ```sh 122 | cd /root/create-and-deploy-esxi/centos7 123 | #change file variables.tf 124 | #change file network_config.cfg 125 | #run auto replace metadata.json 126 | sed -i -e '2d' -e '3i "network": "'$(gzip < network_config.cfg| base64 | tr -d '\n')'",' metadata.json 127 | terraform init 128 | Initializing the backend... 129 | 130 | Initializing provider plugins... 131 | 132 | The following providers do not have any version constraints in configuration, 133 | so the latest version was installed. 134 | 135 | To prevent automatic upgrades to new major versions that may contain breaking 136 | changes, it is recommended to add version = "..." constraints to the 137 | corresponding provider blocks in configuration, with the constraint strings 138 | suggested below. 139 | 140 | * provider.esxi: version = "~> 1.6" 141 | * provider.template: version = "~> 2.1" 142 | 143 | Terraform has been successfully initialized! 144 | 145 | You may now begin working with Terraform. Try running "terraform plan" to see 146 | any changes that are required for your infrastructure. All Terraform commands 147 | should now work. 148 | 149 | If you ever set or change modules or backend configuration for Terraform, 150 | rerun this command to reinitialize your working directory. If you forget, other 151 | commands will detect it and remind you to do so if necessary. 152 | 153 | terraform plan 154 | 155 | terraforn apply 156 | ``` 157 | #### if everything is configured correctly, the virtual machine will be deployed on the esxi host in 2-3 minutes 158 | -------------------------------------------------------------------------------- /centos-7-base.json: -------------------------------------------------------------------------------- 1 | { 2 | "builders": [ 3 | { 4 | "boot_command": [ 5 | " text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/floppy/centos7.ks" 6 | ], 7 | "boot_wait": "7s", 8 | "communicator": "ssh", 9 | "disk_size": 8192, 10 | "disk_type_id": "zeroedthick", 11 | "format": "ova", 12 | "guest_os_type": "rhel7-64", 13 | "headless": true, 14 | "http_directory": ".", 15 | "iso_checksum": "9a2c47d97b9975452f7d582264e9fc16d108ed8252ac6816239a3b58cef5c53d", 16 | "iso_checksum_type": "sha256", 17 | "iso_urls": [ 18 | "./iso/CentOS-7-x86_64-Minimal-1908.iso", 19 | "https://mirror.yandex.ru/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-1810.iso" 20 | ], 21 | "keep_registered": true, 22 | "name": "packer-centos7-x86_64", 23 | "remote_datastore": "{{user `esxi_datastore`}}", 24 | "remote_host": "{{user `esxi_host`}}", 25 | "remote_password": "{{user `esxi_password`}}", 26 | "remote_type": "esx5", 27 | "remote_username": "{{user `esxi_username`}}", 28 | "shutdown_command": "sudo -S /usr/sbin/shutdown -h now", 29 | "skip_compaction": true, 30 | "ssh_password": "123456", 31 | "ssh_pty": "true", 32 | "memory": 1024, 33 | "ssh_timeout": "15m", 34 | "ssh_username": "root", 35 | "tools_upload_flavor": "linux", 36 | "type": "vmware-iso", 37 | "vm_name": "packer-centos7-x86_64", 38 | "vmx_data": { 39 | "ethernet0.networkName": "VM Network" 40 | }, 41 | "vnc_disable_password": true 42 | } 43 | ], 44 | "provisioners": [ 45 | { 46 | "script": "provision/centos-7-install-package.sh", 47 | "type": "shell" 48 | } 49 | ], 50 | "variables": { 51 | "esxi_datastore": "datastore1", 52 | "esxi_host": "10.10.10.10.", 53 | "esxi_password": "passwd", 54 | "esxi_username": "root" 55 | } 56 | } 57 | -------------------------------------------------------------------------------- /centos7/main.tf: -------------------------------------------------------------------------------- 1 | provider "esxi" { 2 | esxi_hostname = var.esxi_hostname 3 | esxi_hostport = var.esxi_hostport 4 | esxi_username = var.esxi_username 5 | esxi_password = var.esxi_password 6 | } 7 | # Template for initial configuration bash script template_file is a great way to pass variables to cloud-init 8 | 9 | data "template_file" "Default" { 10 | template = "${file("userdata.tpl")}" 11 | } 12 | 13 | data "template_file" "network_config" { 14 | template = "${file("metadata.json")}" 15 | vars = { 16 | HOSTNAME = "${var.vm_hostname}" 17 | } 18 | } 19 | # ESXI Guest resource 20 | resource "esxi_guest" "Default" { 21 | guest_name = var.vm_hostname 22 | disk_store = var.disk_store 23 | ovf_source = "/root/create-and-deploy-esxi/output-packer-centos7-x86_64/packer-centos7-x86_64.ova" 24 | memsize = "1024" 25 | network_interfaces { 26 | virtual_network = var.virtual_network 27 | } 28 | 29 | guestinfo = { 30 | "userdata.encoding" = "gzip+base64" 31 | "userdata" = base64gzip(data.template_file.Default.rendered) 32 | "metadata.encoding" = "gzip+base64" 33 | "metadata" = base64gzip(data.template_file.network_config.rendered) 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /centos7/metadata.json: -------------------------------------------------------------------------------- 1 | { 2 | "network": "gzip+base64", 3 | "network.encoding": "gzip+base64", 4 | "local-hostname": "${HOSTNAME}", 5 | "instance-id": "cloud-vm" 6 | } 7 | -------------------------------------------------------------------------------- /centos7/network_config.cfg: -------------------------------------------------------------------------------- 1 | version: 1 2 | config: 3 | - type: physical 4 | name: ens32 5 | subnets: 6 | - type: static 7 | address: 10.10.0.37/24 8 | gateway: 10.10.0.254 9 | dns_nameservers: 10 | - 8.8.8.8 11 | dns_search: 12 | - local.domain 13 | -------------------------------------------------------------------------------- /centos7/output.tf: -------------------------------------------------------------------------------- 1 | output "ip" { 2 | value = ["${esxi_guest.Default.ip_address}"] 3 | } 4 | -------------------------------------------------------------------------------- /centos7/userdata.tpl: -------------------------------------------------------------------------------- 1 | #cloud-config 2 | 3 | users: 4 | - name: admin 5 | sudo: ALL=(ALL) NOPASSWD:ALL 6 | ssh_import_id: None 7 | lock_passwd: true 8 | ssh_authorized_keys: 9 | - ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyxXkYdu/5QF6BJZw8+xUb1Pr3h98T3WO9Mtxb1e4Sq+Og0zhOfp5fjvReTnzv/seQrcAw+5NMoJEA74XEw7fsPiNDBukO9cCdcOC7NGZPyfA09Llq3Ut62HJSGWUtYKOiHHe5ZxkryGIear0VXKZ4ZCfmNjqODCdVjvRC+HBSgxvp062EquETeNozKVbUmA4r8QGFI/CnlGyotnlSSftkA4ioT893TAl/vMAkauQk1XGYWD4pO2LdG64rYwidwb4GzWPMxH2wlBw3bNrU4uSUvfURBXEN2+ZyMMK7AMpoKEHvgDeJXPptqFEYfneqFv9353D62rAAF8JVRDezAuMZQ== root@ibmserv 10 | -------------------------------------------------------------------------------- /centos7/variables.tf: -------------------------------------------------------------------------------- 1 | variable "esxi_hostname" { 2 | default = "10.10.0.10" 3 | } 4 | variable "esxi_hostport" { 5 | default = "22" 6 | } 7 | variable "esxi_username" { 8 | default = "root" 9 | } 10 | variable "esxi_password" { 11 | default = "passwd-to-esxi-server" 12 | } 13 | 14 | variable "virtual_network" { 15 | default = "VM Network" 16 | } 17 | variable "disk_store" { 18 | default = "datastore1" 19 | } 20 | 21 | variable "vm_hostname" { 22 | default = "centos7-test" 23 | } 24 | -------------------------------------------------------------------------------- /floppy/centos7.ks: -------------------------------------------------------------------------------- 1 | #version=DEVEL 2 | # System authorization information 3 | auth --enableshadow --passalgo=sha512 4 | # Use CDROM installation media 5 | cdrom 6 | # Use graphical install 7 | #graphical 8 | text 9 | # Run the Setup Agent on first boot 10 | firstboot --enable 11 | ignoredisk --only-use=sda 12 | # Keyboard layouts 13 | keyboard --vckeymap=us --xlayouts='us' 14 | # System language 15 | lang en_US.UTF-8 16 | 17 | # Network information 18 | network --bootproto=static --device=eth0 --gateway=10.10.0.254 --ip=10.10.0.22 --nameserver=10.10.0.254 --netmask=255.255.255.0 --ipv6=auto --activate 19 | network --hostname=cloud-vm 20 | 21 | # Root password 22 | rootpw --iscrypted $6$GL4F80.CKLU67lTu$Bwqq6BJ3YjltnpUVBhnN85K3aWQ0f130t0IHJloWUTnXHHjH36PamVvIdAcPEjoG3Obxu965BrdZleW7OX/4L1 23 | # System services 24 | services --disabled="chronyd" 25 | # System timezone 26 | timezone Europe/Moscow --isUtc --nontp 27 | # System bootloader configuration 28 | bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=sda 29 | autopart --type=lvm 30 | # Partition clearing information 31 | clearpart --none --initlabel 32 | 33 | %packages 34 | @^minimal 35 | @core 36 | kexec-tools 37 | 38 | %end 39 | 40 | %addon com_redhat_kdump --enable --reserve-mb='auto' 41 | 42 | %end 43 | 44 | %anaconda 45 | pwpolicy root --minlen=6 --minquality=1 --notstrict --nochanges --notempty 46 | pwpolicy user --minlen=6 --minquality=1 --notstrict --nochanges --emptyok 47 | pwpolicy luks --minlen=6 --minquality=1 --notstrict --nochanges --notempty 48 | %end 49 | 50 | reboot 51 | -------------------------------------------------------------------------------- /iso/.gitkeep: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /provision/centos-7-install-package.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | yum install epel-release -y ; 3 | yum -y install open-vm-tools python-ldap ; 4 | yum install -y https://github.com/vmware/cloud-init-vmware-guestinfo/releases/download/v1.1.0/cloud-init-vmware-guestinfo-1.1.0-1.el7.noarch.rpm ; 5 | cloud-init clean; 6 | 7 | # Force logs rotate and remove old logs 8 | /usr/sbin/logrotate -f /etc/logrotate.conf 9 | /bin/rm -f /var/log/*-???????? /var/log/*.gz 10 | /bin/rm -f /var/log/dmesg.old 11 | /bin/rm -rf /var/log/anaconda 12 | 13 | # Truncate the audit logs 14 | /bin/cat /dev/null > /var/log/audit/audit.log 15 | /bin/cat /dev/null > /var/log/wtmp 16 | /bin/cat /dev/null > /var/log/lastlog 17 | /bin/cat /dev/null > /var/log/grubby 18 | 19 | # Remove udev persistent rules 20 | /bin/rm -f /etc/udev/rules.d/70* 21 | 22 | # Remove mac address and uuids from any interface 23 | /bin/sed -i '/^(HWADDR|UUID)=/d' /etc/sysconfig/network-scripts/ifcfg-* 24 | 25 | # Clean /tmp 26 | /bin/rm -rf /tmp/* 27 | /bin/rm -rf /var/tmp/* 28 | 29 | # Remove ssh keys 30 | /bin/rm -f /etc/ssh/*key* 31 | 32 | # Remove root's SSH history and other cruft 33 | /bin/rm -rf ~root/.ssh/ 34 | /bin/rm -f ~root/anaconda-ks.cfg 35 | 36 | # Remove root's and users history 37 | /bin/rm -f ~root/.bash_history 38 | /bin/rm -f /home/*/.bash_history 39 | unset HISTFILE 40 | --------------------------------------------------------------------------------