├── README.md └── docs ├── _config.yml └── index.md /README.md: -------------------------------------------------------------------------------- 1 | # Home Lab 2 | 3 | # Status: This has been replaced with https://github.com/clayshek/homelab-monorepo 4 | 5 | ## Overview 6 | 7 | This (WIP) page describes a home lab environment for evaluation and testing of various technologies. Basic provisioning & configuration of both supporting infrastructure and additional products is documented here - mostly so I remember how I did stuff. 8 | 9 | ## Goals 10 | - A stable base platform of hypervisors & container hosts on physical hardware, on which further virtualized or containerized products can be easily deployed without impact to the base platform. 11 | - Simplicity (as much as possible) 12 | - Raspberry Pis always on, power-hungry servers powered on as needed - so any "critical" roles (dynamic DNS updater, etc) should reside on a Raspberry Pi. 13 | - Totally separate lab env from home (don't want tinkering to impact "home" WiFi, DNS, storage, etc in any way). 14 | - Codified & documented config leading to trivial re/deployments. 15 | - Learning 16 | 17 | ## Software 18 | - 3-node [Proxmox VE](https://www.proxmox.com/) cluster for KVM based virtual machines and LXC containers. 19 | - 4-node Raspberry Pi [K3s](https://k3s.io/) / [Ubuntu Server](https://ubuntu.com/download/server) cluster for ARM-compatible containerized workloads 20 | - Lots of [Ansible](https://www.ansible.com/) for automation of provisioning and configuration 21 | 22 | ## Gear / Roles 23 | - Servers 24 | - 5x Dell R610 1U rack servers. 25 | - Ea: 96 GB RAM, 2x 73 GB HDD (RAID-1 for OS), 4x 450 GB HDD (local data storage) 26 | - Roles: 3x [Proxmox VE](https://www.proxmox.com/) hypervisors, 1x cold standby, 1x spare parts 27 | - Rasberry Pis 28 | - 4x Model 3B, 1x Model 3B+ 29 | - Ea: 1 GB RAM, 1x 32 GB MicroSD 30 | - Roles: 4x [K3s](https://k3s.io/) cluster members, 1x standalone running [Docker](https://www.docker.com/) and serving as an Ansible [control node](https://docs.ansible.com/ansible/2.5/network/getting_started/basic_concepts.html#control-node), all running [Ubuntu Server](ubuntu.com/download/raspberry-pi) 31 | - Switches, Routers, APs 32 | - 1x Ubiquiti [EdgeRouter X](https://www.ui.com/edgemax/edgerouter-x/). Provides routing, firewall, DHCP, DNS to lab, as well as inbound VPN 33 | - 1x Netgear JGS524E 24-port managed switch 34 | - 1x Netgear 8-port unmanaged switch 35 | - 1x Ubiquiti [Unifi AP AC Pro](ui.com/unifi/unifi-ap-ac-pro/) 36 | - Storage 37 | - 1x Buffalo 500 GB NAS (backups, image storage, etc). Old, and requires SMB v1, target for replacement. 38 | - Otherwise locally attached storage (R610 RAID controller limitation not allowing JBOD passthrough restricts ability to use Ceph and other cluster storage technologies) 39 | - Power 40 | - 1x APC BX1500M 1500VA UPS 41 | 42 | ## Config 43 | - Network 44 | - LAN: `192.168.2.0/24` 45 | - Gateway: `192.168.2.1` 46 | - DHCP: Range `192.168.2.150-.199`, provided by EgdeRouterX 47 | - DNS Resolver (default): EdgeRouterX to upstream ISP router to OpenDNS 48 | - Managed switch: currently no special config, but will likely implement VLANs in the future 49 | - DNS Zones 50 | - layer8sys.com (Root zone. Authoritative DNS servers: Google DNS) 51 | - int.layer8sys.com (Purpose: private IP space / internal resource access by FQDN. Authoritative DNS: Primary home router) 52 | - ad.layer8sys.com (Purpose: Windows Active Directory. Authoritative DNS: AD domain controller VMs) 53 | - lab.layer8sys.com (TBD) 54 | 55 | - Wireless 56 | 57 | 58 | ### **Raspberry Pi Provisioning & Config** 59 | 60 | Raspberry Pis are each configured with an Ansible playbook, pulled at OS install from another of my GitHub repos: https://github.com/clayshek/raspi-ubuntu-ansible 61 | 62 | Requires flashing SD card(s) with Ubuntu, and copying in the customizable CloudInit user-data file (included in repo) to the boot partition before inserting into and starting each Pi. After a few minutes, based on defined inventory role, provisioning is complete and ready for any further config. K3s cluster is provisioned with [Rancher's Ansible playbook](https://github.com/rancher/k3s-ansible). 63 | 64 | ### **Proxmox Hypervisor Provisioning & Config** 65 | 66 | Proxmox configuration requires installation of [Proxmox VE](https://www.proxmox.com/en/downloads) on each node, followed by running https://github.com/clayshek/ansible-proxmox-config Ansible playbook (after customization). Once complete, manually create cluster on one node, join other nodes to cluster, and configure cluster data storage specific to implementation details. 67 | 68 | ## Metrics, Monitoring & Logging 69 | - Prometheus / Grafana 70 | - UPS power status & consumption monitoring 71 | - ELK - Logzio? 72 | - UptimeRobot for remote network monitoring 73 | 74 | 75 | ## Proxmox VM Templates 76 | 77 | VM deployments based on a [template](https://pve.proxmox.com/wiki/VM_Templates_and_Clones) are much faster than running through a new install. The following repos use Ansible to create Proxmox template images (and handle OS / package updates) for my most frequently used VM operating systems. These templates are used for later infrastructure provisioning. 78 | 79 | - Windows Server 2019: https://github.com/clayshek/ans-pve-win-templ 80 | - Ubuntu Server 20.04: 81 | 82 | ## Lab Environment & Deployed Apps 83 | 84 | ### Microsoft Windows Server Lab: 85 | - 2x Active Directory Domain Controllers (Proxmox VMs) 86 | - 4-node Microsoft Hyper-V Cluster (Proxmox VMs) 87 | - System Center Virtual Machine Manager (Proxmox VM) 88 | - Windows Admin Center (Proxmox VM) 89 | 90 | The base VMs for the Windows Server lab are provisioned (from the [Server 2019 template](https://github.com/clayshek/ans-pve-win-templ) above), using https://github.com/clayshek/ans-pve-win-provision. 91 | Once online, role assignment and final configuration is done using https://github.com/clayshek/ansible-lab-config 92 | 93 | ### Apache CloudStack (Proxmox VMs) 94 | 95 | ### Kubernetes cluster, incl Windows worker node (Proxmox VMs) 96 | 97 | ### GitLab ([Proxmox Turnkey Linux Container](https://www.turnkeylinux.org/gitlab)) 98 | 99 | ### InfoBlox Eval (Proxmox VM) 100 | 101 | ### [Caddy](https://caddyserver.com/)-based Lab Dashboard / Portal (K3s container) 102 | 103 | ### Dynamic DNS Updaters for Google & OpenDNS (K3s container) 104 | - Google Domains dynamic DNS updater deployed onto Ras Pi K3s cluster to keep my dynamic home IP mapped to a custom FQDN. Deployed as documented here: https://github.com/clayshek/google-ddns-updater 105 | 106 | ### Unifi Network Controller (K3s container) 107 | 108 | ### Prometheus (K3s container) 109 | 110 | ### Grafana (Proxmox VM) 111 | 112 | ### Consul (K3s containers) 113 | 114 | ### APC UPS Monitor (K3s container) 115 | 116 | ## General To-Dos 117 | - [ ] Identify better NAS storage solution, potentially with iSCSI, also providing persistent K3s storage. 118 | - [ ] Update [Proxmox config repo](https://github.com/clayshek/ansible-proxmox-config) to automate cluster creation/join & storage setup. Possibly change to auto playbook pull? 119 | - [ ] Check out the Ras Pi model 4 120 | - [ ] Maybe switch all this from Ansible to Salt 121 | 122 | ## Diagram & Photos 123 | -------------------------------------------------------------------------------- /docs/_config.yml: -------------------------------------------------------------------------------- 1 | theme: jekyll-theme-slate -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- 1 | ## Welcome to GitHub Pages 2 | 3 | You can use the [editor on GitHub](https://github.com/clayshek/homelab/edit/master/docs/index.md) to maintain and preview the content for your website in Markdown files. 4 | 5 | Whenever you commit to this repository, GitHub Pages will run [Jekyll](https://jekyllrb.com/) to rebuild the pages in your site, from the content in your Markdown files. 6 | 7 | ### Markdown 8 | 9 | Markdown is a lightweight and easy-to-use syntax for styling your writing. It includes conventions for 10 | 11 | ```markdown 12 | Syntax highlighted code block 13 | 14 | # Header 1 15 | ## Header 2 16 | ### Header 3 17 | 18 | - Bulleted 19 | - List 20 | 21 | 1. Numbered 22 | 2. List 23 | 24 | **Bold** and _Italic_ and `Code` text 25 | 26 | [Link](url) and ![Image](src) 27 | ``` 28 | 29 | For more details see [GitHub Flavored Markdown](https://guides.github.com/features/mastering-markdown/). 30 | 31 | ### Jekyll Themes 32 | 33 | Your Pages site will use the layout and styles from the Jekyll theme you have selected in your [repository settings](https://github.com/clayshek/homelab/settings). The name of this theme is saved in the Jekyll `_config.yml` configuration file. 34 | 35 | ### Support or Contact 36 | 37 | Having trouble with Pages? Check out our [documentation](https://docs.github.com/categories/github-pages-basics/) or [contact support](https://support.github.com/contact) and we’ll help you sort it out. 38 | --------------------------------------------------------------------------------