├── .gitignore ├── Dockerfile ├── Dockerfile.net ├── README.md ├── bin ├── cloudconfigserver.py ├── make_dnsmasq_dhcp_options.py ├── make_dnsmasq_dhcp_reservations.py ├── pipework ├── reconfig.sh └── startup.sh ├── default.yaml.sample ├── jpetazzo_pxe_README.md ├── pxe_hosts.json.sample └── utilities └── manage_pxe_container /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM alpine:latest 2 | RUN apk add --update dnsmasq wget python && rm -rf /var/cache/apk/* 3 | RUN mkdir -p /cloudconfigserver/bin /cloudconfigserver/data 4 | COPY bin/* /cloudconfigserver/bin/ 5 | RUN chmod +x /cloudconfigserver/bin/* 6 | WORKDIR /cloudconfigserver/data 7 | ENTRYPOINT /cloudconfigserver/bin/startup.sh 8 | -------------------------------------------------------------------------------- /Dockerfile.net: -------------------------------------------------------------------------------- 1 | FROM alpine:latest 2 | RUN apk add --update dnsmasq wget python && rm -rf /var/cache/apk/* 3 | RUN mkdir -p /cloudconfigserver/bin /cloudconfigserver/data 4 | WORKDIR /cloudconfigserver/bin 5 | RUN wget https://raw.githubusercontent.com/avlis/pxe_coreos/master/bin/pipework 6 | RUN wget https://raw.githubusercontent.com/avlis/pxe_coreos/master/bin/make_dnsmasq_dhcp_reservations.py 7 | RUN wget https://raw.githubusercontent.com/avlis/pxe_coreos/master/bin/make_dnsmasq_dhcp_options.py 8 | RUN wget https://raw.githubusercontent.com/avlis/pxe_coreos/master/bin/cloudconfigserver.py 9 | RUN wget https://raw.githubusercontent.com/avlis/pxe_coreos/master/bin/startup.sh 10 | RUN wget https://raw.githubusercontent.com/avlis/pxe_coreos/master/bin/reconfig.sh 11 | RUN chmod +x * 12 | WORKDIR /cloudconfigserver/data 13 | ENTRYPOINT /cloudconfigserver/bin/startup.sh 14 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # container to help manage a cluster of coreOS machines that boot via PXE 2 | My first container, learning and work in progress! 3 | 4 | Forked from the nice work at [jpetazzo/pxe](https://github.com/jpetazzo/pxe). 5 | 6 | ## Notes: 7 | - developed using IBM blades, with a serial console on ttyS1, and a private VLAN on ethernet 2, that is used for DHCP/PXE and coreOS comms only. 8 | - "public" IP address of hosts (eth0) is set by cloud-config (provided by this container, customised for each host). 9 | - DHCP subnet is currently hardcoded as a /24, from .10 to .250; but it should/could be even smaller, as we use reservations?... 10 | - Yes, it sounds silly to read the config files on each request on the httpserver that serves cloud-configs, but this way you can change the config files on the data folder and not have to reload the container... Also, this reloading this should only happen when a host boots. 11 | 12 | ## Enhancements over the original project: 13 | - IP address and subnet for DHCP server and range are extracted from the IP address set on eth1 via pipework. 14 | - boot different OSs based on configuration options. 15 | - includes "smart" webserver to provide a personalised cloud-config to each host. 16 | 17 | ## TODO: 18 | - currently glued to the host that contains the data folder. find better, more independent storage option. 19 | - don't run more than one of these on the same data folder... 20 | - automatic process to update the coreos pxe files? 21 | - web server should be able to provide static files also. 22 | 23 | ## HOW DOES IT WORK: 24 | When the container starts, it processes the *pxe_hosts.json* file, to create a *dhcp_reservations* and *dhcp_options* files for dnsmasq. 25 | pxelinux is configured to offer two config files, stable or beta, based on the channel you choose for your host. 26 | 27 | It will also get the latest coreos release (stable and beta) and a pxelinux.0, if any of these files is not present on the data folder. 28 | 29 | When the host boots, it will request a cloud-config file from this same container, provided by a python web server script running on port 8080. 30 | That script gets the client private IP address of the request, and replaces the variables on the *default.yaml* file, or from a custom yaml template, (represented by a `$`) for that host. 31 | 32 | So you just have customise your *default.yaml* (or create multiple .yaml files if needed) for your hardware, and add hosts to your *pxe_hosts.json*. 33 | 34 | As the original project, you need to setup on the host a bridge that has access to the pxe network, and then connect it to the container via pipework. If you see the *default.yaml.sample* file, the networking stuff there already sets a coreOS host with bridges in front of both ethernet interfaces of these IBM blades. 35 | 36 | ## TO BUILD: 37 | two options: git clone this repo: 38 | ``` 39 | $ git clone https://github.com/avlis/pxe_coreos pxe_coreos 40 | $ cd pxe_coreos 41 | ``` 42 | or just download the Dockerfile.net into an empty folder (and rename it to Dockerfile): 43 | ``` 44 | $ mkdir pxe_coreos && cd pxe_coreos 45 | $ wget https://github.com/avlis/pxe_coreos/Dockerfile.net -O Dockerfile 46 | ``` 47 | 48 | then: 49 | ``` 50 | $ docker build -t pxe_coreos . 51 | ``` 52 | 53 | ## TO LAUNCH: 54 | 55 | make sure you have a proper *default.yaml* and a *pxe_hosts.json* file on your data folder! 56 | (in this example, it's in /home/core/pxe_coreos.data). Two samples are provided in this repo. 57 | 58 | you can use a script similar to the one below (or the [manage_pxe_container](https://github.com/avlis/pxe_coreos/blob/master/utilities/manage_pxe_container) one) to start the container: 59 | ```#!/bin/bash 60 | VLAN_ADDR=192.168.90.2 61 | PXECID=$(docker run --cap-add NET_ADMIN -v /home/core/pxe_coreos.data:/cloudconfigserver/data -d pxe_coreos) 62 | if [ -z "$PXECID" ]; then 63 | echo something bad happened, container not launched. 64 | exit 1 65 | fi 66 | sudo /home/core/pxe_coreos/bin/pipework br-internal $PXECID $VLAN_ADDR/24 67 | ``` 68 | 69 | ## MANAGEMENT & TROUBLESHOOTING: 70 | 71 | check out the [manage_pxe_container](https://github.com/avlis/pxe_coreos/blob/master/utilities/manage_pxe_container) shell script on the utilites folder (on git, not on the container). 72 | 73 | - make sure you have properly indented *.yaml* files, and a properly written (validate to be sure) *pxe_hosts.json* copied to the data dir. 74 | - if running on a vm (e.g. virtualbox) make sure the bridged interface is in promiscuous mode so non bound ips are recieveing tftp requests. 75 | - a while after the container starts, you should have 5 files on /tftp: pxelinux.0 and 4 coreosfiles: coreos_[beta|stable]_pxe[.vmlinuz|_image.cpio.gz] 76 | - you can manually update those files and reboot your host, no need to restart the pxe_coreos container. 77 | - check the output of the startup script with the docker logs command. 78 | - check the content of */dnsmasq/dhcp_leases*, to see if dnsmasq is giving IPs to hosts; 79 | get those mac addresses from that file and copy to new host entries on *pxe_hosts.json* 80 | - you can check the cloud-config file from somewhere else, just add an ?override_ipv4=`` to the url: 81 | `http://:8080/?override_ipv4=` 82 | - for pxe stuff... tcpdump & wireshark are your friends... 83 | - the httpserver reloads the config files at each request, but dnsmasq doesn't: you have to restart the container, or 84 | enter the container and execute the */cloudconfigserver/bin/reconfig.sh* script. 85 | ``` 86 | docker exec -it $PXECID /bin/sh /cloudconfigserver/bin/reconfig.sh 87 | ``` 88 | 89 | ## PROTIP 90 | 91 | Get a coreos container up and ready for running pxe_coreos using this cloud config. 92 | 93 | ``` 94 | #cloud-config 95 | 96 | coreos: 97 | units: 98 | - name: br-internal.netdev 99 | runtime: true 100 | content: | 101 | [NetDev] 102 | Name=br-internal 103 | Kind=bridge 104 | - name: eth0.network 105 | runtime: true 106 | content: | 107 | [Match] 108 | Name=eth0 109 | 110 | [Network] 111 | Bridge=br-internal 112 | - name: br-internal.network 113 | runtime: true 114 | content: | 115 | [Match] 116 | Name=br-internal 117 | 118 | [Network] 119 | DNS=1.2.3.4 120 | Address=10.0.2.2/24 121 | ``` 122 | -------------------------------------------------------------------------------- /bin/cloudconfigserver.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import time 4 | import BaseHTTPServer 5 | import json 6 | import re 7 | from urlparse import urlparse, parse_qs 8 | import os.path 9 | 10 | HOST_NAME = '0.0.0.0' # 11 | PORT_NUMBER = 8080 # Maybe set this to 9000. 12 | 13 | 14 | class MyHandler(BaseHTTPServer.BaseHTTPRequestHandler): 15 | 16 | def load_file(self,p_file, p_replacements): 17 | # Read contents from file as a single string 18 | file_handle = open(p_file, 'r') 19 | file_string = file_handle.read() 20 | file_handle.close() 21 | for r in p_replacements: 22 | file_string = (re.sub(r[0], r[1], file_string)) 23 | return file_string 24 | 25 | def do_HEAD(s): 26 | s.send_response(200) 27 | s.send_header("Content-type", "text/plain") 28 | s.end_headers() 29 | def do_GET(s): 30 | """Respond to a GET request.""" 31 | hosts_data={} 32 | if not os.path.isfile('pxe_hosts.json'): 33 | s.wfile.write("#cloud-config\n\n#Please check your configuration folder, can't find the file [pxe_hosts.json]\n") 34 | return 35 | 36 | with open('pxe_hosts.json') as data_file: 37 | hosts_data=json.load(data_file) 38 | 39 | myClient='' 40 | query_components = parse_qs(urlparse(s.path).query) 41 | if 'override_ipv4' in query_components.keys(): 42 | myClient=query_components['override_ipv4'][0][:16] 43 | 44 | isValidIPV4 = re.compile("\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}") 45 | if not isValidIPV4.match(myClient): 46 | myClient=s.client_address[0][:16] 47 | 48 | if myClient not in hosts_data['hosts'].keys(): 49 | s.send_response(404) 50 | s.send_header("Content-type", "text/plain") 51 | s.end_headers() 52 | s.wfile.write("#cloud-config\n\n#Please add %s to the file [pxe_hosts.json].\n" % myClient) 53 | else: 54 | s.send_response(200) 55 | s.send_header("Content-type", "text/plain") 56 | s.end_headers() 57 | 58 | myReplacements=[] 59 | myReplacements.append( ('\$private_ipv4',myClient) ) 60 | for k in hosts_data['common']: 61 | if k[:1]=="$": 62 | newrep=('\\'+k, hosts_data['common'][k]) 63 | myReplacements.append( newrep ) 64 | for k in hosts_data['hosts'][myClient]: 65 | if k[:1]=="$": 66 | newrep=('\\'+k, hosts_data['hosts'][myClient][k]) 67 | myReplacements.append( newrep ) 68 | if 'template' in hosts_data['hosts'][myClient]: 69 | templateFile=hosts_data['hosts'][myClient]['template'] 70 | else: 71 | templateFile='default.yaml' 72 | if os.path.isfile(templateFile): 73 | myBuffer=s.load_file(templateFile,myReplacements) 74 | s.wfile.write(myBuffer) 75 | else: 76 | s.wfile.write("#cloud-config\n\n#Please check the entry for the host %s on pxe_hosts.json, I can't find the template file [%s].\n" % (myClient,templateFile)) 77 | 78 | if __name__ == '__main__': 79 | server_class = BaseHTTPServer.HTTPServer 80 | httpd = server_class((HOST_NAME, PORT_NUMBER), MyHandler) 81 | print time.asctime(), "Server Starts - %s:%s" % (HOST_NAME, PORT_NUMBER) 82 | try: 83 | httpd.serve_forever() 84 | except KeyboardInterrupt: 85 | pass 86 | httpd.server_close() 87 | print time.asctime(), "Server Stops - %s:%s" % (HOST_NAME, PORT_NUMBER) -------------------------------------------------------------------------------- /bin/make_dnsmasq_dhcp_options.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import json 4 | from pprint import pprint 5 | 6 | with open('pxe_hosts.json') as data_file: 7 | hosts_data = json.load(data_file) 8 | 9 | for hi in hosts_data['hosts']: 10 | print "%s,209,\"pxelinux.cfg/%s\"" % (hosts_data['hosts'][hi]['$hostname'],hosts_data['hosts'][hi]['channel']) -------------------------------------------------------------------------------- /bin/make_dnsmasq_dhcp_reservations.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import json 4 | from pprint import pprint 5 | 6 | with open('pxe_hosts.json') as data_file: 7 | hosts_data = json.load(data_file) 8 | 9 | for hi in hosts_data['hosts']: 10 | print "%s,set:%s,%s,%s" % (hosts_data['hosts'][hi]['macaddr'],hosts_data['hosts'][hi]['channel'],hi ,hosts_data['hosts'][hi]['$hostname']) -------------------------------------------------------------------------------- /bin/pipework: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | set -e 3 | 4 | case "$1" in 5 | --wait) 6 | while ! grep -q ^up$ /sys/class/net/eth1/operstate 2>/dev/null 7 | do sleep 1 8 | done 9 | exit 0 10 | ;; 11 | esac 12 | 13 | IFNAME=$1 14 | GUESTNAME=docker-$2 15 | IPADDR=$3 16 | MACADDR=$4 17 | 18 | [ "$IPADDR" ] || { 19 | echo "Syntax:" 20 | echo "pipework /[@default_gateway] [macaddr]" 21 | echo "pipework dhcp [macaddr]" 22 | echo "pipework --wait" 23 | exit 1 24 | } 25 | 26 | # First step: determine type of first argument (bridge, physical interface...) 27 | if [ -d /sys/class/net/$IFNAME ] 28 | then 29 | if [ -d /sys/class/net/$IFNAME/bridge ] 30 | then IFTYPE=bridge 31 | else IFTYPE=phys 32 | fi 33 | else 34 | case "$IFNAME" in 35 | br*) 36 | IFTYPE=bridge 37 | ;; 38 | *) 39 | echo "I do not know how to setup interface $IFNAME." 40 | exit 1 41 | ;; 42 | esac 43 | fi 44 | 45 | # Second step: find the guest (for now, we only support LXC containers) 46 | while read dev mnt fstype options dump fsck 47 | do 48 | [ "$fstype" != "cgroup" ] && continue 49 | echo $options | grep -qw devices || continue 50 | CGROUPMNT=$mnt 51 | done < /proc/mounts 52 | 53 | [ "$CGROUPMNT" ] || { 54 | echo "Could not locate cgroup mount point." 55 | exit 1 56 | } 57 | 58 | N=$(find "$CGROUPMNT" -name "$GUESTNAME*" | wc -l) 59 | case "$N" in 60 | 0) 61 | echo "Could not find any container matching $GUESTNAME." 62 | exit 1 63 | ;; 64 | 1) 65 | true 66 | ;; 67 | *) 68 | echo "Found more than one container matching $GUESTNAME." 69 | exit 1 70 | ;; 71 | esac 72 | 73 | if [ "$IPADDR" = "dhcp" ] 74 | then 75 | # We use udhcpc to obtain the DHCP lease, make sure it's installed. 76 | which udhcpc >/dev/null || { 77 | echo "You asked for DHCP; please install udhcpc first." 78 | exit 1 79 | } 80 | else 81 | # Check if a subnet mask was provided. 82 | echo $IPADDR | grep -q / || { 83 | echo "The IP address should include a netmask." 84 | echo "Maybe you meant $IPADDR/24 ?" 85 | exit 1 86 | } 87 | # Check if a gateway address was provided. 88 | if echo $IPADDR | grep -q @ 89 | then 90 | GATEWAY=$(echo $IPADDR | cut -d@ -f2) 91 | IPADDR=$(echo $IPADDR | cut -d@ -f1) 92 | else 93 | GATEWAY= 94 | fi 95 | fi 96 | 97 | NSPID=$(head -n 1 $(find "$CGROUPMNT" -name "$GUESTNAME*" | head -n 1)/tasks) 98 | [ "$NSPID" ] || { 99 | echo "Could not find a process inside container $GUESTNAME." 100 | exit 1 101 | } 102 | mkdir -p /var/run/netns 103 | rm -f /var/run/netns/$NSPID 104 | ln -s /proc/$NSPID/ns/net /var/run/netns/$NSPID 105 | 106 | 107 | # Check if we need to create a bridge. 108 | [ $IFTYPE = bridge ] && [ ! -d /sys/class/net/$IFNAME ] && { 109 | ip link add $IFNAME type bridge 110 | ip link set $IFNAME up 111 | } 112 | 113 | # If it's a bridge, we need to create a veth pair 114 | [ $IFTYPE = bridge ] && { 115 | LOCAL_IFNAME=vethl$NSPID 116 | GUEST_IFNAME=vethg$NSPID 117 | ip link add name $LOCAL_IFNAME type veth peer name $GUEST_IFNAME 118 | ip link set $LOCAL_IFNAME master $IFNAME 119 | ip link set $LOCAL_IFNAME up 120 | } 121 | 122 | # If it's a physical interface, create a macvlan subinterface 123 | [ $IFTYPE = phys ] && { 124 | GUEST_IFNAME=macvlan$NSPID 125 | ip link add link $IFNAME dev $GUEST_IFNAME type macvlan mode bridge 126 | ip link set $IFNAME up 127 | } 128 | 129 | ip link set $GUEST_IFNAME netns $NSPID 130 | ip netns exec $NSPID ip link set $GUEST_IFNAME name eth1 131 | [ "$MACADDR" ] && ip netns exec $NSPID ip link set eth1 address $MACADDR 132 | if [ "$IPADDR" = "dhcp" ] 133 | then 134 | ip netns exec $NSPID udhcpc -qi eth1 135 | else 136 | ip netns exec $NSPID ip addr add $IPADDR dev eth1 137 | ip netns exec $NSPID ip link set eth1 up 138 | [ "$GATEWAY" ] && { 139 | ip netns exec $NSPID ip route replace default via $GATEWAY 140 | } 141 | fi 142 | -------------------------------------------------------------------------------- /bin/reconfig.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | cd /cloudconfigserver/data 3 | /cloudconfigserver/bin/make_dnsmasq_dhcp_reservations.py > dnsmasq/dhcp_reservations 4 | /cloudconfigserver/bin/make_dnsmasq_dhcp_options.py > dnsmasq/dhcp_options 5 | kill -HUP $(pidof dnsmasq) -------------------------------------------------------------------------------- /bin/startup.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | echo Waiting for pipework to give us the eth1 interface... 3 | /cloudconfigserver/bin/pipework --wait 4 | myIP=$(ip addr show dev eth1 | awk -F '[ /]+' '/global/ {print $3}') 5 | mySUBNET=$(echo $myIP | cut -d '.' -f 1,2,3) 6 | echo Got IP $myIP, on subnet $mySUBNET 7 | echo making sure the necessary data folders exist 8 | mkdir -p /cloudconfigserver/data/tftp/pxelinux.cfg 9 | mkdir -p /cloudconfigserver/data/dnsmasq 10 | cd /cloudconfigserver/data/tftp 11 | echo downloading pxe if needed 12 | [ ! -f pxelinux.0 ] && wget http://ftp.debian.org/debian/dists/stable/main/installer-amd64/current/images/netboot/pxelinux.0 13 | echo downloading coreos if needed 14 | [ ! -f coreos_stable_pxe.vmlinuz ] && wget http://stable.release.core-os.net/amd64-usr/current/coreos_production_pxe.vmlinuz -O coreos_stable_pxe.vmlinuz 15 | [ ! -f coreos_stable_pxe_image.cpio.gz ] && wget http://stable.release.core-os.net/amd64-usr/current/coreos_production_pxe_image.cpio.gz -O coreos_stable_pxe_image.cpio.gz 16 | [ ! -f coreos_beta_pxe.vmlinuz ] && wget http://beta.release.core-os.net/amd64-usr/current/coreos_production_pxe.vmlinuz -O coreos_beta_pxe.vmlinuz 17 | [ ! -f coreos_beta_pxe_image.cpio.gz ] && wget http://beta.release.core-os.net/amd64-usr/current/coreos_production_pxe_image.cpio.gz -O coreos_beta_pxe_image.cpio.gz 18 | echo making pxe menus if needed 19 | [ ! -f pxelinux.cfg/stable ] && printf "DEFAULT coreos-stable\nlabel coreos-stable\n\tmenu coreos\n\tkernel coreos_stable_pxe.vmlinuz\n\tappend initrd=coreos_stable_pxe_image.cpio.gz console=ttyS1,19200n8 console=tty0 coreos.autologin=ttyS1 coreos.autologin=tty0 cloud-config-url=http://$myIP:8080\n" >pxelinux.cfg/stable 20 | [ ! -f pxelinux.cfg/beta ] && printf "DEFAULT coreos-beta\nlabel coreos-beta\n\tmenu coreos\n\tkernel coreos_beta_pxe.vmlinuz\n\tappend initrd=coreos_beta_pxe_image.cpio.gz console=ttyS1,19200n8 console=tty0 coreos.autologin=ttyS1 coreos.autologin=tty0 cloud-config-url=http://$myIP:8080\n" >pxelinux.cfg/beta 21 | echo generating dnsmask config files from pxe_hosts.json file... 22 | cd /cloudconfigserver/data 23 | /cloudconfigserver/bin/make_dnsmasq_dhcp_reservations.py > dnsmasq/dhcp_reservations 24 | /cloudconfigserver/bin/make_dnsmasq_dhcp_options.py > dnsmasq/dhcp_options 25 | echo Starting DHCP+TFTP server... 26 | dnsmasq --interface=eth1 \ 27 | --dhcp-hostsfile=/cloudconfigserver/data/dnsmasq/dhcp_reservations \ 28 | --dhcp-optsfile=/cloudconfigserver/data/dnsmasq/dhcp_options \ 29 | --dhcp-leasefile=/cloudconfigserver/data/dnsmasq/dhcp_leases \ 30 | --dhcp-option-force=tag:stable,209,"pxelinux.cfg/stable" \ 31 | --dhcp-option-force=tag:beta,209,"pxelinux.cfg/beta" \ 32 | --dhcp-range=$mySUBNET.10,$mySUBNET.250,255.255.255.0,1h \ 33 | --dhcp-boot=pxelinux.0,pxeserver,$myIP \ 34 | --pxe-service=x86PC,"coreos",pxelinux \ 35 | --enable-tftp --tftp-root=/cloudconfigserver/data/tftp 36 | /cloudconfigserver/bin/cloudconfigserver.py 37 | -------------------------------------------------------------------------------- /default.yaml.sample: -------------------------------------------------------------------------------- 1 | #cloud-config 2 | 3 | hostname: $hostname 4 | 5 | coreos: 6 | update: 7 | reboot-strategy: etcd-lock 8 | 9 | etcd2: 10 | # generate a new token for each unique cluster from https://discovery.etcd.io/new?size=3 11 | # specify the intial size of your cluster with ?size=X 12 | discovery: https://discovery.etcd.io/ 13 | advertise-client-urls: http://$private_ipv4:2379 14 | initial-advertise-peer-urls: http://$private_ipv4:2380 15 | listen-client-urls: http://0.0.0.0:2379 16 | listen-peer-urls: http://$private_ipv4:2380 17 | 18 | units: 19 | - name: etcd2.service 20 | command: start 21 | - name: fleet.service 22 | command: start 23 | - name: 01_br-internal.netdev 24 | runtime: true 25 | content: | 26 | [NetDev] 27 | Name=br-internal 28 | Kind=bridge 29 | - name: 11_br-internal.network 30 | runtime: true 31 | content: | 32 | [Match] 33 | Name=br-internal 34 | [Network] 35 | Address=$private_ipv4/24 36 | 37 | - name: 02_br-external.netdev 38 | runtime: true 39 | content: | 40 | [NetDev] 41 | Name=br-external 42 | Kind=bridge 43 | - name: 12_br-external.network 44 | runtime: true 45 | content: | 46 | [Match] 47 | Name=br-external 48 | [Network] 49 | Address=$public_ipv4/24 50 | Gateway=$public_ipv4_gw 51 | DNS=$public_ipv4_dns1 52 | DNS=$public_ipv4_dns2 53 | 54 | - name: 03_br-hostonly.netdev 55 | runtime: true 56 | content: | 57 | [NetDev] 58 | Name=br-hostonly 59 | Kind=bridge 60 | - name: 13_br-hostonly.network 61 | runtime: true 62 | content: | 63 | [Match] 64 | Name=br-hostonly 65 | [Network] 66 | Address=192.168.1.1/24 67 | 68 | - name: 31_enp4s0.network 69 | runtime: true 70 | content: | 71 | [Match] 72 | Name=enp4s0 73 | 74 | [Network] 75 | Bridge=br-external 76 | 77 | - name: 32_enp6s0.network 78 | runtime: true 79 | content: | 80 | [Match] 81 | Name=enp6s0 82 | 83 | [Network] 84 | Bridge=br-internal 85 | 86 | ssh_authorized_keys: 87 | - -------------------------------------------------------------------------------- /jpetazzo_pxe_README.md: -------------------------------------------------------------------------------- 1 | # My other PXE server is a container 2 | 3 | This is a Dockerfile to build a container running a PXE server, 4 | pre-configured to serve a Debian netinstall kernel and initrd. 5 | 6 | ## Quick start 7 | 8 | 1. Of course you need Docker first! 9 | 1. Clone this repo and `cd` into the repo checkout. 10 | 1. Build the container with `docker build -t pxe .` 11 | 1. Run the container with `PXECID=$(docker run -d pxe)` 12 | 1. Give it an extra network interface with `./pipework br0 $PXECID 192.168.242.1/24` 13 | 1. Put the network interface connected to your machines on the same bridge 14 | with e.g. `brctl addif br0 eth0` (don't forget to move `eth0` IP address 15 | to `br0` if there is one). 16 | 1. You can now boot PXE machines on the network connected to `eth0`! 17 | Alternatively, you can put VMs on `br0` and achieve the same result. 18 | 19 | 20 | ### Why and how do we move eth0 IP address to br0? 21 | 22 | The Linux network stack has the notion of master and slave interfaces. 23 | They are used in many places, including bridges and bonding (when 24 | multiple physical interfaces are grouped together to form a single 25 | logical link, for increased throughput or reliability). When using 26 | Linux bridges, the bridge is the master interface, and all the ports 27 | of the bridge are slave interfaces. 28 | 29 | Now is the tricky part: with interfaces like bridges and bonding 30 | groups, only the master should have IP addresses; not the slaves. 31 | If an IP address is configured on a slave interface, it will misbehave 32 | in seemingly random ways. For instance, it can stop working if 33 | the interface is down (but the master interface is still up). 34 | Or it might handle some protocols like ARP only for packets 35 | inbound on this interface. 36 | 37 | Therefore, when changing the configuration of an existing interface 38 | to place it inside a bridge (or bonding group), you should 39 | deconfigure its IP address, and assign it to the master interface 40 | instead. I recommend the following steps: 41 | 42 | 1. Check the IP address of the interface (with e.g. `ip addr ls eth0`). 43 | Carefully note the IP address *and its subnet mask*, e.g. 44 | 192.168.1.4/24. There can be multiple addresses; in that case, 45 | note all of them. 46 | 2. Check if there are special routes going through that interface. 47 | Chances are, that there is a default route, and you will have to 48 | take care of it; otherwise you will lose internet connectivity. 49 | The easiest way is to do `ip route ls dev eth0`. You will almost 50 | certainly see an entry with `proto kernel scope link`, which 51 | is the automatic entry corresponding to the subnet directly 52 | connected to this interface. You can ignore this one. However, 53 | if you see something like `default via 192.168.1.1`, note it. 54 | 3. Deconfigure the IP address. In that case, we would do 55 | `ip addr del 192.168.1.4/24 dev eth0`. You don't havea to 56 | deconfigure the routes: they will be automatically removed 57 | as the address is withdrawn. 58 | 4. Configure the IP address on the bridge. In our example, that would 59 | be `ip addr add 192.168.1.4/24 dev br0`. 60 | 5. Last but not least, re-add the routes on the bridge. Here, we 61 | would do `ip route add default via 192.168.1.1`. 62 | 63 | If you want to do that automatically at boot, you can do it through 64 | the `/etc/network/interfaces` file (on Debian/Ubuntu). 65 | 66 | It will look like this (assuming the same IP addresses than our 67 | previous example): 68 | 69 | ``` 70 | auto br0 71 | iface br0 inet static 72 | address 192.168.1.4 73 | netmask 255.255.255.0 74 | network 192.168.1.0 75 | broadcast 192.168.1.255 76 | gateway 192.168.1.1 77 | bridge_ports eth0 78 | bridge_stp off 79 | bridge_fd 0 80 | ``` 81 | 82 | Don't forget to disable the section related to `eth0` then! 83 | 84 | 85 | ## I want to netboot something else! 86 | 87 | Left as an exercise for the reader. Check the Dockerfile and rebuild; 88 | it should be easy enough. 89 | 90 | 91 | ## Can I change the IP address, 192.168.242.1...? 92 | 93 | Yes, if you also change it in the Dockerfile. 94 | 95 | 96 | ## Can I *not* use pipework? 97 | 98 | Yes, but it will be more complicated. You will have to: 99 | 100 | - make sure that Docker UDP can handle broadcast packets (since PXE/DHCP 101 | uses broadcast packets); 102 | - make sure that UDP ports are correctly mapped; 103 | - auto-detect the gateway address and DNS server, instead of using the 104 | container as a router+DNS server; 105 | - maybe something else that I overlooked. 106 | 107 | 108 | ## I want MOAR fun! 109 | 110 | Let's have a game! 111 | 112 | 1. Burn a [boot2docker](https://github.com/steeve/boot2docker) ISO on 113 | a blank CD. 114 | 1. With that CD, boot a physical machine. 115 | 1. Run the PXE container on Docker on the physical machine. 116 | 1. Pull the ubuntu container, start it in privileged mode, apt-get install 117 | QEMU in it, and start a QEMU VM, mapping its hard disks to the real 118 | hard disk of the machine, and bridging it with the PXE container. 119 | 1. The QEMU VM will netboot from the PXE container. Install Debian. 120 | 1. Reboot the physical machine -- it now boots on Debian. 121 | 1. Repeat steps but install Windows for trolling purposes. 122 | -------------------------------------------------------------------------------- /pxe_hosts.json.sample: -------------------------------------------------------------------------------- 1 | { 2 | "common": { 3 | "$public_ipv4_gw": "10.100.10.5", 4 | "$public_ipv4_dns1": "10.100.10.9", 5 | "$public_ipv4_dns2": "10.100.10.9" 6 | }, 7 | "hosts": { 8 | "192.168.90.101": { 9 | "macaddr": "e2:29:a2:da:43:b1", 10 | "$hostname": "docker-101", 11 | "$public_ipv4": "10.100.10.101", 12 | "channel": "stable", 13 | }, 14 | "192.168.90.102": { 15 | "macaddr": "e2:29:a2:da:43:b2", 16 | "$hostname": "docker-102", 17 | "$public_ipv4": "10.100.10.102", 18 | "channel": "stable" 19 | }, 20 | "192.168.90.103": { 21 | "macaddr": "78:31:c1:c3:d3:0e", 22 | "$hostname": "docker-103", 23 | "$public_ipv4": "10.100.10.103", 24 | "channel": "beta", 25 | "template": "etcd2.yaml" 26 | } 27 | } 28 | } 29 | -------------------------------------------------------------------------------- /utilities/manage_pxe_container: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | [ -z "${PXE_VLAN_ADDR}" ] && PXE_VLAN_ADDR=192.168.90.2 3 | [ -z "${PXE_IMAGE_NAME}" ] && PXE_IMAGE_NAME=pxe_coreos 4 | PXECID=$(docker ps --no-trunc | grep "${PXE_IMAGE_NAME}:latest" | awk '{print $1}') 5 | case "$1" in 6 | '') 7 | echo "I need one parameter: start | stop | status | enter | logs" 8 | ;; 9 | start) 10 | if [ ! -z "${PXECID}" ]; then 11 | echo it looks like there is already a pxe container running... 12 | exit 1 13 | fi 14 | mkdir -p /home/core/${PXE_IMAGE_NAME}.data 15 | PXECID=$(docker run --cap-add NET_ADMIN -v /home/core/${PXE_IMAGE_NAME}.data:/cloudconfigserver/data -d ${PXE_IMAGE_NAME}) 16 | if [ -z "${PXECID}" ]; then 17 | echo something bad happened, container not launched. 18 | exit 1 19 | fi 20 | sudo /root/bin/pipework br-internal ${PXECID} ${PXE_VLAN_ADDR}/24 21 | ;; 22 | enter) 23 | if [ -z "${PXECID}" ]; then 24 | echo could not find pxe container. 25 | exit 1 26 | fi 27 | [ -z "$2" ] && PXESHELL=bash || PXESHELL="$2" 28 | docker exec -i -t ${PXECID} ${PXESHELL} 29 | ;; 30 | stop) 31 | if [ -z "${PXECID}" ]; then 32 | echo could not find pxe container. 33 | exit 1 34 | fi 35 | docker stop ${PXECID} 36 | [ "$2" == "rm" ] && docker rm ${PXECID} 37 | ;; 38 | status) 39 | docker ps -a | grep -e "CONTAINER\|${PXE_IMAGE_NAME}" 40 | ;; 41 | *) 42 | if [ -z "${PXECID}" ]; then 43 | echo could not find pxe container. 44 | exit 1 45 | fi 46 | docker $1 ${PXECID} 47 | ;; 48 | esac 49 | --------------------------------------------------------------------------------