├── docs ├── user │ ├── providers.md │ ├── reference.md │ ├── faq.md │ └── quickstart.md ├── dev │ ├── providers.md │ └── releases.md └── index.md ├── .gitignore ├── contrib └── do │ ├── size │ ├── 16gb │ ├── 1gb │ ├── 2gb │ ├── 32gb │ ├── 48gb │ ├── 4gb │ ├── 512mb │ ├── 64gb │ └── 8gb │ ├── region │ ├── london1 │ ├── newyork1 │ ├── newyork2 │ ├── newyork3 │ ├── sanfran1 │ ├── amsterdam1 │ ├── amsterdam2 │ ├── amsterdam3 │ └── singapore1 │ └── image │ ├── centos_6 │ ├── centos_7 │ ├── coreos_alpha │ ├── coreos_beta │ ├── fedora_21 │ ├── coreos_stable │ ├── ubuntu_12.04 │ └── ubuntu_14.04 ├── providers.go ├── env_test.go ├── Makefile ├── mkdocs.yml ├── ip_test.go ├── circle.yml ├── ip.go ├── down_test.go ├── ls.go ├── down.go ├── env.go ├── up.go ├── LICENSE ├── ssh.go ├── up_test.go ├── ls_test.go ├── README.md ├── hostctl.go ├── providers └── providers.go ├── scale.go ├── scale_test.go ├── util.go ├── digitalocean └── provider.go ├── hostctl_test.go └── aws └── provider.go /docs/user/providers.md: -------------------------------------------------------------------------------- 1 | # Providers 2 | -------------------------------------------------------------------------------- /docs/user/reference.md: -------------------------------------------------------------------------------- 1 | # Reference 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | hostctl 2 | build 3 | release 4 | -------------------------------------------------------------------------------- /contrib/do/size/16gb: -------------------------------------------------------------------------------- 1 | export HOSTCTL_FLAVOR=16gb 2 | -------------------------------------------------------------------------------- /contrib/do/size/1gb: -------------------------------------------------------------------------------- 1 | export HOSTCTL_FLAVOR=1gb 2 | -------------------------------------------------------------------------------- /contrib/do/size/2gb: -------------------------------------------------------------------------------- 1 | export HOSTCTL_FLAVOR=2gb 2 | -------------------------------------------------------------------------------- /contrib/do/size/32gb: -------------------------------------------------------------------------------- 1 | export HOSTCTL_FLAVOR=32gb 2 | -------------------------------------------------------------------------------- /contrib/do/size/48gb: -------------------------------------------------------------------------------- 1 | export HOSTCTL_FLAVOR=48gb 2 | -------------------------------------------------------------------------------- /contrib/do/size/4gb: -------------------------------------------------------------------------------- 1 | export HOSTCTL_FLAVOR=4gb 2 | -------------------------------------------------------------------------------- /contrib/do/size/512mb: -------------------------------------------------------------------------------- 1 | export HOSTCTL_FLAVOR=512mb 2 | -------------------------------------------------------------------------------- /contrib/do/size/64gb: -------------------------------------------------------------------------------- 1 | export HOSTCTL_FLAVOR=64gb 2 | -------------------------------------------------------------------------------- /contrib/do/size/8gb: -------------------------------------------------------------------------------- 1 | export HOSTCTL_FLAVOR=8gb 2 | -------------------------------------------------------------------------------- /docs/user/faq.md: -------------------------------------------------------------------------------- 1 | # Frequently Asked Questions 2 | -------------------------------------------------------------------------------- /contrib/do/region/london1: -------------------------------------------------------------------------------- 1 | export HOSTCTL_REGION=lon1 2 | -------------------------------------------------------------------------------- /contrib/do/region/newyork1: -------------------------------------------------------------------------------- 1 | export HOSTCTL_REGION=nyc1 2 | -------------------------------------------------------------------------------- /contrib/do/region/newyork2: -------------------------------------------------------------------------------- 1 | export HOSTCTL_REGION=nyc2 2 | -------------------------------------------------------------------------------- /contrib/do/region/newyork3: -------------------------------------------------------------------------------- 1 | export HOSTCTL_REGION=nyc3 2 | -------------------------------------------------------------------------------- /contrib/do/region/sanfran1: -------------------------------------------------------------------------------- 1 | export HOSTCTL_REGION=sfo1 2 | -------------------------------------------------------------------------------- /docs/dev/providers.md: -------------------------------------------------------------------------------- 1 | # Contributing Cloud Providers 2 | -------------------------------------------------------------------------------- /contrib/do/region/amsterdam1: -------------------------------------------------------------------------------- 1 | export HOSTCTL_REGION=ams1 2 | -------------------------------------------------------------------------------- /contrib/do/region/amsterdam2: -------------------------------------------------------------------------------- 1 | export HOSTCTL_REGION=ams2 2 | -------------------------------------------------------------------------------- /contrib/do/region/amsterdam3: -------------------------------------------------------------------------------- 1 | export HOSTCTL_REGION=ams3 2 | -------------------------------------------------------------------------------- /contrib/do/region/singapore1: -------------------------------------------------------------------------------- 1 | export HOSTCTL_REGION=sgp1 2 | -------------------------------------------------------------------------------- /contrib/do/image/centos_6: -------------------------------------------------------------------------------- 1 | export HOSTCTL_IMAGE=centos-6-5-x64 2 | -------------------------------------------------------------------------------- /contrib/do/image/centos_7: -------------------------------------------------------------------------------- 1 | export HOSTCTL_IMAGE=centos-7-0-x64 2 | -------------------------------------------------------------------------------- /contrib/do/image/coreos_alpha: -------------------------------------------------------------------------------- 1 | export HOSTCTL_IMAGE=coreos-alpha 2 | -------------------------------------------------------------------------------- /contrib/do/image/coreos_beta: -------------------------------------------------------------------------------- 1 | export HOSTCTL_IMAGE=coreos-beta 2 | -------------------------------------------------------------------------------- /contrib/do/image/fedora_21: -------------------------------------------------------------------------------- 1 | export HOSTCTL_IMAGE=fedora-21-x64 2 | -------------------------------------------------------------------------------- /contrib/do/image/coreos_stable: -------------------------------------------------------------------------------- 1 | export HOSTCTL_IMAGE=coreos-stable 2 | -------------------------------------------------------------------------------- /contrib/do/image/ubuntu_12.04: -------------------------------------------------------------------------------- 1 | export HOSTCTL_IMAGE=ubuntu-12-04-x64 2 | -------------------------------------------------------------------------------- /contrib/do/image/ubuntu_14.04: -------------------------------------------------------------------------------- 1 | export HOSTCTL_IMAGE=ubuntu-14-04-x64 2 | -------------------------------------------------------------------------------- /providers.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | _ "github.com/gliderlabs/hostctl/aws" 5 | _ "github.com/gliderlabs/hostctl/digitalocean" 6 | ) 7 | -------------------------------------------------------------------------------- /docs/dev/releases.md: -------------------------------------------------------------------------------- 1 | # Staging Releases 2 | 3 | Don't wait for maintainers to cut a release! You can stage a release at any time 4 | using GitHub. Just open a PR against the release branch from master. If merged, 5 | a new release will automatically be cut. 6 | 7 | Please be sure to bump the version and update CHANGELOG.md and include your 8 | changelog text in the PR body. 9 | -------------------------------------------------------------------------------- /env_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/facebookgo/ensure" 7 | ) 8 | 9 | func TestEnvCmd(t *testing.T) { 10 | t.Parallel() 11 | stdout, stderr := testRunCmd(t, "hostctl env", 0, nil, nil) 12 | ensure.StringContains(t, stdout.String(), "HOSTCTL_PROVIDER=") 13 | ensure.StringContains(t, stdout.String(), "HOSTCTL_NAMESPACE=") 14 | ensure.DeepEqual(t, stderr.String(), "") 15 | } 16 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | NAME=hostctl 2 | ARCH=$(shell uname -m) 3 | VERSION=0.1.0dev 4 | 5 | .PHONY: build release docs 6 | 7 | build: 8 | glu build darwin,linux 9 | 10 | test: 11 | go test -v 12 | 13 | deps: 14 | go get github.com/gliderlabs/glu 15 | go get -d . 16 | 17 | release: 18 | glu release v$(VERSION) 19 | 20 | docs: 21 | boot2docker ssh "sync; sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'" || true 22 | docker run --rm -it -p 8000:8000 -v $(PWD):/work gliderlabs/pagebuilder mkdocs serve 23 | -------------------------------------------------------------------------------- /mkdocs.yml: -------------------------------------------------------------------------------- 1 | site_name: Hostctl 2 | site_url: https://gliderlabs.com/hostctl 3 | repo_url: https://github.com/gliderlabs/hostctl 4 | dev_addr: 0.0.0.0:8000 5 | theme_dir: /pagebuilder/theme 6 | google_analytics: ['UA-58928488-1', 'auto'] 7 | pages: 8 | - 'Readme': index.md 9 | - 'User Guide': 10 | - 'Quickstart': user/quickstart.md 11 | - 'Reference': user/reference.md 12 | - 'FAQ': user/faq.md 13 | - 'Developer Guide': 14 | - 'Adding Providers': dev/providers.md 15 | - 'Staging Releases': dev/releases.md 16 | -------------------------------------------------------------------------------- /ip_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/facebookgo/ensure" 7 | "github.com/gliderlabs/hostctl/providers" 8 | ) 9 | 10 | func TestIpCmd(t *testing.T) { 11 | t.Parallel() 12 | provider := new(providers.TestProvider) 13 | provider.Create(providers.Host{ 14 | Name: "test1", 15 | IP: "127.0.0.1", 16 | }) 17 | 18 | stdout, stderr := testRunCmd(t, "hostctl ip test1", 0, provider, nil) 19 | ensure.DeepEqual(t, stdout.String(), "127.0.0.1\n") 20 | ensure.DeepEqual(t, stderr.String(), "") 21 | } 22 | -------------------------------------------------------------------------------- /circle.yml: -------------------------------------------------------------------------------- 1 | machine: 2 | services: 3 | - docker 4 | 5 | dependencies: 6 | pre: 7 | - docker run --rm gliderlabs/glu | tar xC /home/ubuntu/bin 8 | - glu circleci 9 | - glu container up 10 | override: 11 | - glu build linux,darwin 12 | 13 | test: 14 | pre: 15 | - go get -d -t 16 | override: 17 | - go test -v -race 18 | 19 | deployment: 20 | master: 21 | branch: master 22 | commands: 23 | - eval $(docker run gliderlabs/pagebuilder circleci-cmd) 24 | release: 25 | branch: release 26 | commands: 27 | - make release 28 | -------------------------------------------------------------------------------- /ip.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | 7 | "github.com/gliderlabs/hostctl/providers" 8 | "github.com/spf13/cobra" 9 | ) 10 | 11 | func init() { 12 | Hostctl.AddCommand(ipCmd) 13 | } 14 | 15 | var ipCmd = &cobra.Command{ 16 | Use: "ip ", 17 | Short: "Show IP for host", 18 | Run: func(cmd *cobra.Command, args []string) { 19 | if len(args) < 1 && defaultName == "" { 20 | cmd.Usage() 21 | os.Exit(1) 22 | } 23 | provider, err := providers.Get(providerName, true) 24 | fatal(err) 25 | host := provider.Get(namespace + optArg(args, 0, defaultName)) 26 | if host == nil { 27 | os.Exit(1) 28 | } 29 | fmt.Println(host.IP) 30 | }, 31 | } 32 | -------------------------------------------------------------------------------- /down_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/facebookgo/ensure" 7 | "github.com/gliderlabs/hostctl/providers" 8 | ) 9 | 10 | func TestDownCmd(t *testing.T) { 11 | t.Parallel() 12 | provider := new(providers.TestProvider) 13 | provider.Create(providers.Host{ 14 | Name: "test1", 15 | }) 16 | provider.Create(providers.Host{ 17 | Name: "test2", 18 | }) 19 | 20 | stdout, stderr := testRunCmd(t, "hostctl down test1", 0, provider, nil) 21 | ensure.DeepEqual(t, stdout.String(), "") 22 | ensure.DeepEqual(t, stderr.String(), "\n") 23 | 24 | ensure.DeepEqual(t, provider.Get("test1"), (*providers.Host)(nil)) 25 | ensure.NotDeepEqual(t, provider.Get("test2"), (*providers.Host)(nil)) 26 | } 27 | -------------------------------------------------------------------------------- /ls.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "strings" 6 | 7 | "github.com/gliderlabs/hostctl/providers" 8 | "github.com/spf13/cobra" 9 | ) 10 | 11 | var fullNames bool 12 | 13 | func init() { 14 | listCmd.Flags().BoolVarP(&fullNames, 15 | "full", "f", false, "show full names with namespace") 16 | Hostctl.AddCommand(listCmd) 17 | } 18 | 19 | var listCmd = &cobra.Command{ 20 | Use: "ls [pattern]", 21 | Short: "List hosts", 22 | Run: func(cmd *cobra.Command, args []string) { 23 | pattern := "*" 24 | if len(args) > 0 { 25 | pattern = args[0] 26 | } 27 | provider, err := providers.Get(providerName, true) 28 | fatal(err) 29 | for _, host := range provider.List(namespace + pattern) { 30 | if fullNames { 31 | fmt.Println(host.Name) 32 | } else { 33 | fmt.Println(strings.TrimPrefix(host.Name, namespace)) 34 | } 35 | } 36 | }, 37 | } 38 | -------------------------------------------------------------------------------- /down.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "os" 5 | "sync" 6 | 7 | "github.com/gliderlabs/hostctl/providers" 8 | "github.com/spf13/cobra" 9 | ) 10 | 11 | func init() { 12 | Hostctl.AddCommand(downCmd) 13 | } 14 | 15 | var downCmd = &cobra.Command{ 16 | Use: "down [...]", 17 | Short: "Terminate host", 18 | Run: func(cmd *cobra.Command, args []string) { 19 | if len(args) < 1 && defaultName == "" { 20 | cmd.Usage() 21 | os.Exit(1) 22 | } 23 | if len(args) == 0 { 24 | args = []string{defaultName} 25 | } 26 | provider, err := providers.Get(providerName, true) 27 | fatal(err) 28 | finished := progressBar(".", 1) 29 | parallelWait(args, func(_ int, arg string, wg *sync.WaitGroup) { 30 | defer wg.Done() 31 | name := namespace + arg 32 | if provider.Get(name) == nil { 33 | return 34 | } 35 | fatal(provider.Destroy(name)) 36 | }) 37 | finished() 38 | }, 39 | } 40 | -------------------------------------------------------------------------------- /env.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "os" 5 | 6 | "github.com/MattAitchison/env" 7 | "github.com/gliderlabs/hostctl/providers" 8 | "github.com/spf13/cobra" 9 | ) 10 | 11 | var ( 12 | exportMode bool 13 | secretsMode bool 14 | ) 15 | 16 | func init() { 17 | envCmd.Flags().BoolVarP(&exportMode, 18 | "export", "e", false, "export vars for sourcing later") 19 | envCmd.Flags().BoolVarP(&secretsMode, 20 | "secrets", "s", false, "show secrets or include in export") 21 | Hostctl.AddCommand(envCmd) 22 | } 23 | 24 | var envCmd = &cobra.Command{ 25 | Use: "env", 26 | Short: "Show relevant environment", 27 | Run: func(cmd *cobra.Command, args []string) { 28 | env.PrintEnv(os.Stdout, exportMode, secretsMode) 29 | provider, _ := providers.Get(providerName, false) 30 | if provider != nil && provider.Env() != nil { 31 | provider.Env().PrintEnv(os.Stdout, exportMode, secretsMode) 32 | } 33 | }, 34 | } 35 | -------------------------------------------------------------------------------- /up.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "os" 5 | "sync" 6 | 7 | "github.com/gliderlabs/hostctl/providers" 8 | "github.com/spf13/cobra" 9 | ) 10 | 11 | func init() { 12 | Hostctl.AddCommand(upCmd) 13 | } 14 | 15 | var upCmd = &cobra.Command{ 16 | Use: "up [...]", 17 | Short: "Provision host, wait until ready", 18 | Run: func(cmd *cobra.Command, args []string) { 19 | if len(args) < 1 && defaultName == "" { 20 | cmd.Usage() 21 | os.Exit(1) 22 | } 23 | loadStdinUserdata() 24 | if len(args) == 0 { 25 | args = []string{defaultName} 26 | } 27 | provider, err := providers.Get(providerName, true) 28 | fatal(err) 29 | finished := progressBar(".", 2) 30 | parallelWait(args, func(_ int, arg string, wg *sync.WaitGroup) { 31 | defer wg.Done() 32 | name := namespace + arg 33 | if provider.Get(name) != nil { 34 | return 35 | } 36 | fatal(provider.Create(newHost(name))) 37 | }) 38 | finished() 39 | }, 40 | } 41 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Glider Labs 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /ssh.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | "os/exec" 7 | "syscall" 8 | 9 | "github.com/gliderlabs/hostctl/providers" 10 | "github.com/spf13/cobra" 11 | ) 12 | 13 | func init() { 14 | Hostctl.AddCommand(sshCmd) 15 | } 16 | 17 | var sshCmd = &cobra.Command{ 18 | Use: "ssh [--] [...]", 19 | Short: "SSH to host", 20 | Run: func(cmd *cobra.Command, args []string) { 21 | if len(args) < 1 && defaultName == "" { 22 | cmd.Usage() 23 | os.Exit(1) 24 | } 25 | name, sshCmd := sshParseArgs(args) 26 | provider, err := providers.Get(providerName, true) 27 | fatal(err) 28 | host := provider.Get(namespace + name) 29 | if host == nil { 30 | os.Exit(1) 31 | } 32 | fatal(sshExec(host.IP, sshCmd)) 33 | }, 34 | } 35 | 36 | func sshExec(ip string, cmd []string) error { 37 | binary, err := exec.LookPath("ssh") 38 | if err != nil { 39 | return fmt.Errorf("Unable to find ssh") 40 | } 41 | args := []string{"ssh", "-A", "-l", sshUser, ip} 42 | return syscall.Exec(binary, append(args, cmd...), os.Environ()) 43 | } 44 | 45 | func sshParseArgs(args []string) (string, []string) { 46 | var name string 47 | var sshCmd []string 48 | if len(args) == 0 || args[0] == "--" { 49 | name = defaultName 50 | sshCmd = args 51 | } else { 52 | name = args[0] 53 | sshCmd = args[1:] 54 | } 55 | if optArg(sshCmd, 0, "") == "--" { 56 | sshCmd = sshCmd[1:] 57 | } 58 | return name, sshCmd 59 | } 60 | -------------------------------------------------------------------------------- /up_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/facebookgo/ensure" 7 | "github.com/gliderlabs/hostctl/providers" 8 | ) 9 | 10 | func TestUpSimple(t *testing.T) { 11 | t.Parallel() 12 | provider := new(providers.TestProvider) 13 | 14 | stdout, stderr := testRunCmd(t, "hostctl up test1", 0, provider, nil) 15 | ensure.DeepEqual(t, stderr.String(), "\n") 16 | ensure.DeepEqual(t, stdout.String(), "") 17 | 18 | ensure.NotDeepEqual(t, provider.Get("test1"), (*providers.Host)(nil)) 19 | } 20 | 21 | func TestUpExists(t *testing.T) { 22 | t.Parallel() 23 | provider := new(providers.TestProvider) 24 | provider.Create(providers.Host{ 25 | Name: "test1", 26 | }) 27 | 28 | stdout, stderr := testRunCmd(t, "hostctl up test1", 0, provider, nil) 29 | ensure.DeepEqual(t, stderr.String(), "\n") 30 | ensure.DeepEqual(t, stdout.String(), "") 31 | 32 | ensure.NotDeepEqual(t, provider.Get("test1"), (*providers.Host)(nil)) 33 | } 34 | 35 | func TestUpMultiple(t *testing.T) { 36 | t.Parallel() 37 | provider := new(providers.TestProvider) 38 | 39 | stdout, stderr := testRunCmd(t, "hostctl up test1 test2 test3", 0, provider, nil) 40 | ensure.DeepEqual(t, stderr.String(), "\n") 41 | ensure.DeepEqual(t, stdout.String(), "") 42 | 43 | ensure.NotDeepEqual(t, provider.Get("test1"), (*providers.Host)(nil)) 44 | ensure.NotDeepEqual(t, provider.Get("test2"), (*providers.Host)(nil)) 45 | ensure.NotDeepEqual(t, provider.Get("test3"), (*providers.Host)(nil)) 46 | } 47 | -------------------------------------------------------------------------------- /ls_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/facebookgo/ensure" 7 | "github.com/gliderlabs/hostctl/providers" 8 | ) 9 | 10 | func TestListBasic(t *testing.T) { 11 | t.Parallel() 12 | provider := new(providers.TestProvider) 13 | provider.Create(providers.Host{ 14 | Name: "test1", 15 | }) 16 | provider.Create(providers.Host{ 17 | Name: "test2", 18 | }) 19 | 20 | stdout, stderr := testRunCmd(t, "hostctl ls", 0, provider, nil) 21 | ensure.DeepEqual(t, stdout.String(), "test1\ntest2\n") 22 | ensure.DeepEqual(t, stderr.String(), "") 23 | } 24 | 25 | func TestListPatternSingle(t *testing.T) { 26 | t.Parallel() 27 | provider := new(providers.TestProvider) 28 | provider.Create(providers.Host{ 29 | Name: "test1", 30 | }) 31 | provider.Create(providers.Host{ 32 | Name: "test2", 33 | }) 34 | 35 | stdout, stderr := testRunCmd(t, "hostctl ls test1", 0, provider, nil) 36 | ensure.DeepEqual(t, stdout.String(), "test1\n") 37 | ensure.DeepEqual(t, stderr.String(), "") 38 | } 39 | 40 | func TestListPatternPartial(t *testing.T) { 41 | t.Parallel() 42 | provider := new(providers.TestProvider) 43 | provider.Create(providers.Host{ 44 | Name: "test1", 45 | }) 46 | provider.Create(providers.Host{ 47 | Name: "test2", 48 | }) 49 | provider.Create(providers.Host{ 50 | Name: "foobar", 51 | }) 52 | 53 | stdout, stderr := testRunCmd(t, "hostctl ls te*", 0, provider, nil) 54 | ensure.DeepEqual(t, stdout.String(), "test1\ntest2\n") 55 | ensure.DeepEqual(t, stderr.String(), "") 56 | } 57 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # hostctl 2 | 3 | Hostctl is an opinionated command line tool for easily provisioning cloud VMs. 4 | 5 | Hostctl is ideal for spinning up VMs for development or personal use. It does 6 | nothing more than manage VM hosts, so if you need anything else you should look 7 | at cloud provider specific tools. It's not intended for managing production 8 | clusters, as you should be using a tool like [Terraform](https://terraform.io/) instead. 9 | 10 | ## Getting hostctl 11 | 12 | Until the first release, you can get hostctl with `go get`: 13 | 14 | $ go get github.com/progrium/hostctl 15 | 16 | ## Usage 17 | 18 | ``` 19 | Usage: 20 | hostctl [command] 21 | 22 | Available Commands: 23 | down Terminate host 24 | env Show relevant environment 25 | ip Show IP for host 26 | ls List hosts 27 | scale Resize host cluster 28 | ssh SSH to host 29 | up Provision host, wait until ready 30 | help Help about any command 31 | ``` 32 | 33 | ## Configuration 34 | 35 | ``` 36 | HOSTCTL_PROVIDER # what provider backend (digitalocean, ec2) 37 | HOSTCTL_IMAGE # vm image 38 | HOSTCTL_FLAVOR # vm flavor 39 | HOSTCTL_REGION # vm region 40 | HOSTCTL_KEYNAME # vm keyname 41 | HOSTCTL_USERDATA # vm userdata 42 | HOSTCTL_NAMESPACE # optional namespace for names 43 | HOSTCTL_NAME # optional default name 44 | HOSTCTL_USER # ssh user 45 | ``` 46 | 47 | ## Todo 48 | 49 | * move to GL, project infrastructure 50 | * tests 51 | * docs 52 | 53 | ## License 54 | 55 | MIT 56 | -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- 1 | # Hostctl 2 | 3 | The simplest way to boot and manage cloud VMs. 4 | 5 | [![Circle CI](https://circleci.com/gh/gliderlabs/hostctl.png?style=shield)](https://circleci.com/gh/gliderlabs/hostctl) 6 | [![IRC Channel](https://img.shields.io/badge/irc-%23gliderlabs-blue.svg)](https://kiwiirc.com/client/irc.freenode.net/#gliderlabs) 7 |

8 | 9 | Hostctl is an opinionated CLI tool for basic cloud VM operations, ideal for 10 | development and personal use. It does nothing more than manage VM hosts, so if 11 | you need anything else you should look at cloud provider specific tools. Hostctl 12 | supports pluggable cloud providers, currently including DigitalOcean and Amazon 13 | EC2. 14 | 15 | ## Getting Hostctl 16 | 17 | You can install the Hostctl binary right into a directory in your `$PATH`: 18 | ``` 19 | $ curl https://dl.gliderlabs.com/gh/hostctl/latest/$(uname -sm|tr \ _).tgz \ 20 | | tar -zxC /usr/local/bin 21 | ``` 22 | ## Using Hostctl 23 | 24 | The quickest way to see Hostctl in action is our 25 | [Quickstart](user/quickstart.md) tutorial. After [configuring for a particular 26 | provider](user/providers.md), spinning up a VM is as simple as: 27 | 28 | $ hostctl up 29 | 30 | For a full list of command, see the [Command Reference](user/reference.md) in 31 | the User Guide. 32 | 33 | ## Contributing 34 | 35 | Pull requests are welcome! We recommend getting feedback before starting by 36 | opening a [GitHub issue](https://github.com/gliderlabs/hostctl/issues) or 37 | discussing in [Slack](http://glider-slackin.herokuapp.com/). 38 | 39 | Also check out our Developer Guide on [Contributing Providers](dev/providers.md) 40 | and [Staging Releases](dev/releases.md). 41 | 42 | ## License 43 | 44 | MIT 45 | 46 | 47 | -------------------------------------------------------------------------------- /hostctl.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "os" 5 | 6 | "github.com/MattAitchison/env" 7 | "github.com/gliderlabs/pkg/usage" 8 | "github.com/spf13/cobra" 9 | ) 10 | 11 | var ( 12 | Version string 13 | versionChecker = usage.NewChecker("hostctl", Version) 14 | 15 | providerName string 16 | defaultName string 17 | namespace string 18 | 19 | hostImage string 20 | hostFlavor string 21 | hostRegion string 22 | hostKeyname string 23 | hostUserdata string 24 | 25 | sshUser string 26 | 27 | profile string 28 | ) 29 | 30 | func readEnv() { 31 | env.Clear() 32 | providerName = env.String("HOSTCTL_PROVIDER", "digitalocean", "cloud provider") 33 | defaultName = env.String("HOSTCTL_NAME", "", "optional default name") 34 | namespace = env.String("HOSTCTL_NAMESPACE", "", "optional namespace for names") 35 | hostImage = env.String("HOSTCTL_IMAGE", "", "vm image") 36 | hostFlavor = env.String("HOSTCTL_FLAVOR", "", "vm flavor") 37 | hostRegion = env.String("HOSTCTL_REGION", "", "vm region") 38 | hostKeyname = env.String("HOSTCTL_KEYNAME", "", "vm keyname") 39 | hostUserdata = env.String("HOSTCTL_USERDATA", "", "optional vm user data") 40 | sshUser = env.String("HOSTCTL_USER", os.Getenv("USER"), "ssh user") 41 | } 42 | 43 | func init() { 44 | readEnv() 45 | Hostctl.PersistentFlags().StringVarP(&profile, "profile", "p", "", "profile to source") 46 | Hostctl.AddCommand(versionCmd) 47 | } 48 | 49 | func main() { 50 | fatal(Hostctl.Execute()) 51 | } 52 | 53 | var Hostctl = &cobra.Command{ 54 | Use: "hostctl", 55 | Short: "An opinionated tool for provisioning cloud VMs", 56 | Run: func(cmd *cobra.Command, args []string) { 57 | cmd.Help() 58 | }, 59 | PersistentPreRun: func(cmd *cobra.Command, args []string) { 60 | if exists(expandHome("~/.hostctl")) { 61 | fatal(source(expandHome("~/.hostctl"))) 62 | readEnv() 63 | } 64 | if profile != "" { 65 | fatal(source(profile)) 66 | readEnv() 67 | } 68 | }, 69 | } 70 | 71 | var versionCmd = &cobra.Command{ 72 | Use: "version", 73 | Short: "Show version", 74 | Run: func(cmd *cobra.Command, args []string) { 75 | versionChecker.PrintVersion() 76 | }, 77 | } 78 | -------------------------------------------------------------------------------- /providers/providers.go: -------------------------------------------------------------------------------- 1 | package providers 2 | 3 | import ( 4 | "fmt" 5 | "path/filepath" 6 | "sync" 7 | 8 | "github.com/MattAitchison/env" 9 | ) 10 | 11 | var providers = make(map[string]HostProvider) 12 | 13 | func Register(provider HostProvider, name string) { 14 | providers[name] = provider 15 | } 16 | 17 | func Get(name string, setup bool) (HostProvider, error) { 18 | p, found := providers[name] 19 | if !found { 20 | return nil, fmt.Errorf("Provider not registered: %s", name) 21 | } 22 | if setup { 23 | return p, p.Setup() 24 | } else { 25 | return p, nil 26 | } 27 | } 28 | 29 | type HostProvider interface { 30 | Setup() error 31 | Create(host Host) error 32 | Destroy(name string) error 33 | List(pattern string) []Host 34 | Get(name string) *Host 35 | Env() *env.EnvSet 36 | } 37 | 38 | type Host struct { 39 | Name string 40 | IP string 41 | Region string 42 | Image string 43 | Keyname string 44 | Flavor string 45 | Userdata string 46 | } 47 | 48 | type TestProvider struct { 49 | mu sync.Mutex 50 | Hosts []Host 51 | } 52 | 53 | func (p *TestProvider) Setup() error { 54 | return nil 55 | } 56 | 57 | func (p *TestProvider) Create(host Host) error { 58 | p.mu.Lock() 59 | defer p.mu.Unlock() 60 | p.Hosts = append(p.Hosts, host) 61 | return nil 62 | } 63 | 64 | func (p *TestProvider) Destroy(name string) error { 65 | p.mu.Lock() 66 | defer p.mu.Unlock() 67 | var hosts []Host 68 | for i := range p.Hosts { 69 | if p.Hosts[i].Name != name { 70 | hosts = append(hosts, p.Hosts[i]) 71 | } 72 | } 73 | p.Hosts = hosts 74 | return nil 75 | } 76 | 77 | func (p *TestProvider) List(pattern string) []Host { 78 | p.mu.Lock() 79 | defer p.mu.Unlock() 80 | var hosts []Host 81 | for i := range p.Hosts { 82 | if ok, _ := filepath.Match(pattern, p.Hosts[i].Name); ok { 83 | hosts = append(hosts, p.Hosts[i]) 84 | } 85 | } 86 | return hosts 87 | } 88 | 89 | func (p *TestProvider) Get(name string) *Host { 90 | p.mu.Lock() 91 | defer p.mu.Unlock() 92 | for i := range p.Hosts { 93 | if p.Hosts[i].Name == name { 94 | return &p.Hosts[i] 95 | } 96 | } 97 | return nil 98 | } 99 | 100 | func (p *TestProvider) Env() *env.EnvSet { 101 | return nil 102 | } 103 | -------------------------------------------------------------------------------- /scale.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | "strconv" 7 | "sync" 8 | 9 | "github.com/gliderlabs/hostctl/providers" 10 | "github.com/spf13/cobra" 11 | ) 12 | 13 | func init() { 14 | Hostctl.AddCommand(scaleCmd) 15 | } 16 | 17 | var scaleCmd = &cobra.Command{ 18 | Use: "scale ", 19 | Short: "Resize host cluster", 20 | Run: func(cmd *cobra.Command, args []string) { 21 | if (len(args) < 2 && defaultName == "") || 22 | (len(args) < 1 && defaultName != "") { 23 | cmd.Usage() 24 | os.Exit(1) 25 | } 26 | var name, count string 27 | if len(args) == 1 { 28 | name = defaultName 29 | count = args[0] 30 | } else { 31 | name = args[0] 32 | count = args[1] 33 | } 34 | loadStdinUserdata() 35 | provider, err := providers.Get(providerName, true) 36 | fatal(err) 37 | existing := existingHosts(provider, name) 38 | desired := desiredHosts(name, count) 39 | hosts := append(strSet(existing, desired), namespace+name) 40 | finished := progressBar(".", 2) 41 | parallelWait(hosts, func(_ int, host string, wg *sync.WaitGroup) { 42 | defer wg.Done() 43 | if !strIn(host, desired) { 44 | fatal(provider.Destroy(host)) 45 | return 46 | } 47 | if strIn(host, desired) && !strIn(host, existing) { 48 | fatal(provider.Create(newHost(host))) 49 | return 50 | } 51 | }) 52 | finished() 53 | }, 54 | } 55 | 56 | func desiredHosts(name, count string) []string { 57 | c, err := strconv.Atoi(count) 58 | fatal(err) 59 | var hosts []string 60 | for i := 0; i < c; i++ { 61 | hosts = append(hosts, fmt.Sprintf("%s%s.%v", namespace, name, i)) 62 | } 63 | return hosts 64 | } 65 | 66 | func existingHosts(provider providers.HostProvider, name string) []string { 67 | var hosts []string 68 | for _, h := range provider.List(namespace + name + ".*") { 69 | hosts = append(hosts, h.Name) 70 | } 71 | return hosts 72 | } 73 | 74 | func strIn(str string, list []string) bool { 75 | for i := range list { 76 | if str == list[i] { 77 | return true 78 | } 79 | } 80 | return false 81 | } 82 | 83 | func strSet(strs ...[]string) []string { 84 | m := make(map[string]bool) 85 | for i := range strs { 86 | for _, str := range strs[i] { 87 | m[str] = true 88 | } 89 | } 90 | var set []string 91 | for k := range m { 92 | set = append(set, k) 93 | } 94 | return set 95 | } 96 | -------------------------------------------------------------------------------- /scale_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/facebookgo/ensure" 7 | "github.com/gliderlabs/hostctl/providers" 8 | ) 9 | 10 | func TestScaleUpFromZero(t *testing.T) { 11 | t.Parallel() 12 | provider := new(providers.TestProvider) 13 | 14 | stdout, stderr := testRunCmd(t, "hostctl scale test 3", 0, provider, nil) 15 | ensure.DeepEqual(t, stderr.String(), "\n") 16 | ensure.DeepEqual(t, stdout.String(), "") 17 | 18 | ensure.NotDeepEqual(t, provider.Get("test.0"), (*providers.Host)(nil)) 19 | ensure.NotDeepEqual(t, provider.Get("test.1"), (*providers.Host)(nil)) 20 | ensure.NotDeepEqual(t, provider.Get("test.2"), (*providers.Host)(nil)) 21 | } 22 | 23 | func TestScaleDownToZero(t *testing.T) { 24 | t.Parallel() 25 | provider := new(providers.TestProvider) 26 | provider.Create(providers.Host{ 27 | Name: "test.0", 28 | }) 29 | provider.Create(providers.Host{ 30 | Name: "test.1", 31 | }) 32 | provider.Create(providers.Host{ 33 | Name: "test.2", 34 | }) 35 | 36 | stdout, stderr := testRunCmd(t, "hostctl scale test 0", 0, provider, nil) 37 | ensure.DeepEqual(t, stderr.String(), "\n") 38 | ensure.DeepEqual(t, stdout.String(), "") 39 | 40 | ensure.DeepEqual(t, provider.Get("test.2"), (*providers.Host)(nil)) 41 | ensure.DeepEqual(t, provider.Get("test.1"), (*providers.Host)(nil)) 42 | ensure.DeepEqual(t, provider.Get("test.0"), (*providers.Host)(nil)) 43 | } 44 | 45 | func TestScaleDownToOne(t *testing.T) { 46 | t.Parallel() 47 | provider := new(providers.TestProvider) 48 | provider.Create(providers.Host{ 49 | Name: "test.0", 50 | }) 51 | provider.Create(providers.Host{ 52 | Name: "test.1", 53 | }) 54 | provider.Create(providers.Host{ 55 | Name: "test.2", 56 | }) 57 | 58 | stdout, stderr := testRunCmd(t, "hostctl scale test 1", 0, provider, nil) 59 | ensure.DeepEqual(t, stderr.String(), "\n") 60 | ensure.DeepEqual(t, stdout.String(), "") 61 | 62 | ensure.NotDeepEqual(t, provider.Get("test.0"), (*providers.Host)(nil)) 63 | ensure.DeepEqual(t, provider.Get("test.1"), (*providers.Host)(nil)) 64 | ensure.DeepEqual(t, provider.Get("test.2"), (*providers.Host)(nil)) 65 | } 66 | 67 | func TestScaleUpFromOne(t *testing.T) { 68 | t.Parallel() 69 | provider := new(providers.TestProvider) 70 | provider.Create(providers.Host{ 71 | Name: "test.0", 72 | }) 73 | 74 | stdout, stderr := testRunCmd(t, "hostctl scale test 3", 0, provider, nil) 75 | ensure.DeepEqual(t, stderr.String(), "\n") 76 | ensure.DeepEqual(t, stdout.String(), "") 77 | 78 | ensure.NotDeepEqual(t, provider.Get("test.0"), (*providers.Host)(nil)) 79 | ensure.NotDeepEqual(t, provider.Get("test.1"), (*providers.Host)(nil)) 80 | ensure.NotDeepEqual(t, provider.Get("test.2"), (*providers.Host)(nil)) 81 | } 82 | -------------------------------------------------------------------------------- /util.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "io/ioutil" 6 | "os" 7 | "os/exec" 8 | "path/filepath" 9 | "strings" 10 | "sync" 11 | "time" 12 | 13 | "github.com/gliderlabs/hostctl/providers" 14 | "github.com/mitchellh/go-homedir" 15 | "golang.org/x/crypto/ssh/terminal" 16 | ) 17 | 18 | func newHost(name string) providers.Host { 19 | return providers.Host{ 20 | Name: name, 21 | Flavor: hostFlavor, 22 | Image: hostImage, 23 | Region: hostRegion, 24 | Keyname: hostKeyname, 25 | Userdata: hostUserdata, 26 | } 27 | } 28 | 29 | func loadStdinUserdata() { 30 | if !terminal.IsTerminal(int(os.Stdin.Fd())) { 31 | data, err := ioutil.ReadAll(os.Stdin) 32 | fatal(err) 33 | hostUserdata = string(data) 34 | } 35 | } 36 | 37 | func parallelWait(items []string, fn func(int, string, *sync.WaitGroup)) { 38 | var wg sync.WaitGroup 39 | for i := 0; i < len(items); i++ { 40 | wg.Add(1) 41 | go fn(i, items[i], &wg) 42 | } 43 | wg.Wait() 44 | } 45 | 46 | func fatal(err error) { 47 | if err != nil { 48 | fmt.Println("!!", err) 49 | os.Exit(1) 50 | } 51 | } 52 | 53 | func exists(path ...string) bool { 54 | _, err := os.Stat(filepath.Join(path...)) 55 | if err == nil { 56 | return true 57 | } 58 | if os.IsNotExist(err) { 59 | return false 60 | } 61 | fatal(err) 62 | return true 63 | } 64 | 65 | func expandHome(path string) string { 66 | if len(path) > 1 && path[:2] == "~/" { 67 | path, _ = homedir.Expand(path) 68 | } 69 | return path 70 | } 71 | 72 | func optArg(args []string, i int, default_ string) string { 73 | if i+1 > len(args) { 74 | return default_ 75 | } 76 | return args[i] 77 | } 78 | 79 | func progressBar(unit string, interval time.Duration) func() { 80 | finished := make(chan bool) 81 | go func() { 82 | for { 83 | select { 84 | case <-finished: 85 | return 86 | case <-time.After(interval * time.Second): 87 | fmt.Fprint(os.Stderr, unit) 88 | } 89 | } 90 | }() 91 | return func() { 92 | finished <- true 93 | fmt.Fprintln(os.Stderr) 94 | } 95 | } 96 | 97 | func source(filepath string) error { 98 | file, err := ioutil.ReadFile(filepath) 99 | if err != nil { 100 | return err 101 | } 102 | 103 | cmdStr := fmt.Sprintf("source %s 1>&2; env", filepath) 104 | out, err := exec.Command("bash", "-c", cmdStr).Output() 105 | if err != nil { 106 | return err 107 | } 108 | 109 | fileStr := string(file) 110 | outLines := strings.Split(string(out), "\n") 111 | for _, line := range outLines { 112 | lineSplit := strings.SplitN(line, "=", 2) 113 | if len(lineSplit) != 2 { 114 | continue 115 | } 116 | if strings.Contains(fileStr, lineSplit[0]) { 117 | os.Setenv(lineSplit[0], lineSplit[1]) 118 | } 119 | } 120 | return nil 121 | } 122 | -------------------------------------------------------------------------------- /digitalocean/provider.go: -------------------------------------------------------------------------------- 1 | package digitalocean 2 | 3 | import ( 4 | "fmt" 5 | "path/filepath" 6 | "strconv" 7 | "strings" 8 | "time" 9 | 10 | "github.com/MattAitchison/env" 11 | "github.com/digitalocean/godo" 12 | "github.com/gliderlabs/hostctl/providers" 13 | "golang.org/x/oauth2" 14 | ) 15 | 16 | var envSet = env.NewEnvSet("digitalocean") 17 | 18 | func init() { 19 | readEnv() 20 | providers.Register(new(digitalOceanProvider), "digitalocean") 21 | } 22 | 23 | func readEnv() { 24 | envSet.Clear() 25 | envSet.Secret("DO_TOKEN", "token for DigitalOcean API v2") 26 | } 27 | 28 | type digitalOceanProvider struct { 29 | client *godo.Client 30 | } 31 | 32 | func (p *digitalOceanProvider) Setup() error { 33 | readEnv() 34 | token := envSet.Var("DO_TOKEN").Value.Get().(string) 35 | if token == "" { 36 | return fmt.Errorf("DO_TOKEN required for Digital Ocean provider") 37 | } 38 | tokenSource := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: token}) 39 | oauthClient := oauth2.NewClient(oauth2.NoContext, tokenSource) 40 | p.client = godo.NewClient(oauthClient) 41 | _, _, err := p.client.Account.Get() 42 | return err 43 | } 44 | 45 | func (p *digitalOceanProvider) Env() *env.EnvSet { 46 | readEnv() 47 | return envSet 48 | } 49 | 50 | func (p *digitalOceanProvider) Create(host providers.Host) error { 51 | var sshKey godo.DropletCreateSSHKey 52 | if strings.Contains(host.Keyname, ":") { 53 | sshKey.Fingerprint = host.Keyname 54 | } else { 55 | id, err := strconv.Atoi(host.Keyname) 56 | if err != nil { 57 | return err 58 | } 59 | sshKey.ID = id 60 | } 61 | droplet, _, err := p.client.Droplets.Create(&godo.DropletCreateRequest{ 62 | Name: host.Name, 63 | Region: host.Region, 64 | Size: host.Flavor, 65 | Image: godo.DropletCreateImage{ 66 | Slug: host.Image, 67 | }, 68 | SSHKeys: []godo.DropletCreateSSHKey{sshKey}, 69 | UserData: host.Userdata, 70 | }) 71 | if err != nil { 72 | return err 73 | } 74 | for { 75 | droplet, _, err = p.client.Droplets.Get(droplet.ID) 76 | if droplet != nil && droplet.Status == "active" { 77 | return nil 78 | } 79 | if err != nil { 80 | return err 81 | } 82 | time.Sleep(1 * time.Second) 83 | } 84 | } 85 | 86 | func (p *digitalOceanProvider) Destroy(name string) error { 87 | droplets, _, err := p.client.Droplets.List(nil) 88 | if err != nil { 89 | return err 90 | } 91 | for i := range droplets { 92 | if droplets[i].Name == name { 93 | _, err := p.client.Droplets.Delete(droplets[i].ID) 94 | if err != nil { 95 | return err 96 | } 97 | // TODO timeout 98 | for p.Get(name) != nil { 99 | time.Sleep(1 * time.Second) 100 | } 101 | return nil 102 | } 103 | } 104 | return nil 105 | } 106 | 107 | func (p *digitalOceanProvider) List(pattern string) []providers.Host { 108 | droplets, _, err := p.client.Droplets.List(nil) 109 | if err != nil { 110 | return nil 111 | } 112 | var hosts []providers.Host 113 | for i := range droplets { 114 | if ok, _ := filepath.Match(pattern, droplets[i].Name); ok { 115 | hosts = append(hosts, providers.Host{ 116 | Name: droplets[i].Name, 117 | }) 118 | } 119 | } 120 | return hosts 121 | } 122 | 123 | func (p *digitalOceanProvider) Get(name string) *providers.Host { 124 | droplets, _, err := p.client.Droplets.List(nil) 125 | if err != nil { 126 | return nil 127 | } 128 | for i := range droplets { 129 | if droplets[i].Name == name { 130 | var ip string 131 | if droplets[i].Networks != nil { 132 | if len(droplets[i].Networks.V4) > 0 { 133 | ip = droplets[i].Networks.V4[0].IPAddress 134 | } 135 | } 136 | return &providers.Host{ 137 | Name: name, 138 | IP: ip, 139 | } 140 | } 141 | } 142 | return nil 143 | } 144 | -------------------------------------------------------------------------------- /hostctl_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bytes" 5 | "encoding/gob" 6 | "fmt" 7 | "io/ioutil" 8 | "os" 9 | "os/exec" 10 | "runtime" 11 | "strings" 12 | "syscall" 13 | "testing" 14 | 15 | "github.com/facebookgo/ensure" 16 | "github.com/gliderlabs/hostctl/providers" 17 | "github.com/gliderlabs/pkg/usage" 18 | ) 19 | 20 | func testRunCmd(t *testing.T, cmdline string, statusExpected int, provider *providers.TestProvider, setupFn func()) (*bytes.Buffer, *bytes.Buffer) { 21 | if os.Getenv("TEST_CMD") != "" { 22 | if setupFn != nil { 23 | setupFn() 24 | } 25 | p := registerTestProvider() 26 | os.Args = strings.Split(os.Getenv("TEST_CMD"), " ") 27 | fatal(Hostctl.Execute()) 28 | returnTestProvider(p) 29 | os.Exit(0) 30 | } 31 | pc, _, _, _ := runtime.Caller(1) 32 | callerPath := strings.Split(runtime.FuncForPC(pc).Name(), ".") 33 | testName := callerPath[len(callerPath)-1] 34 | var stdout, stderr bytes.Buffer 35 | cmd := exec.Command(os.Args[0], "-test.run="+testName) 36 | cmd.Env = append(os.Environ(), "TEST_CMD="+cmdline) 37 | cmd.Stdout = &stdout 38 | cmd.Stderr = &stderr 39 | f := writeTestProvider(provider, testName) 40 | if f != nil { 41 | cmd.Env = append(cmd.Env, "TEST_PROVIDER="+f.Name()) 42 | } 43 | err := cmd.Run() 44 | status := 0 45 | if exiterr, ok := err.(*exec.ExitError); ok { 46 | if s, ok := exiterr.Sys().(syscall.WaitStatus); ok { 47 | status = s.ExitStatus() 48 | } 49 | } 50 | if status != statusExpected { 51 | fmt.Println(stderr.String()) 52 | ensure.DeepEqual(t, status, statusExpected) 53 | } 54 | readTestProvider(provider, f) 55 | return &stdout, &stderr 56 | } 57 | 58 | func registerTestProvider() *providers.TestProvider { 59 | if os.Getenv("TEST_PROVIDER") != "" { 60 | providerFile, err := os.Open(os.Getenv("TEST_PROVIDER")) 61 | if err != nil { 62 | panic(err) 63 | } 64 | dec := gob.NewDecoder(providerFile) 65 | var testProvider providers.TestProvider 66 | err = dec.Decode(&testProvider) 67 | if err != nil { 68 | panic(err) 69 | } 70 | providerFile.Close() 71 | providerName = "test" 72 | providers.Register(&testProvider, "test") 73 | return &testProvider 74 | } 75 | return nil 76 | } 77 | 78 | func returnTestProvider(provider *providers.TestProvider) { 79 | if os.Getenv("TEST_PROVIDER") != "" { 80 | providerFile, err := os.Create(os.Getenv("TEST_PROVIDER")) 81 | if err != nil { 82 | panic(err) 83 | } 84 | enc := gob.NewEncoder(providerFile) 85 | err = enc.Encode(provider) 86 | if err != nil { 87 | panic(err) 88 | } 89 | providerFile.Close() 90 | } 91 | } 92 | 93 | func writeTestProvider(provider *providers.TestProvider, prefix string) *os.File { 94 | if provider != nil { 95 | providerFile, err := ioutil.TempFile("", prefix) 96 | if err != nil { 97 | panic(err) 98 | } 99 | enc := gob.NewEncoder(providerFile) 100 | err = enc.Encode(provider) 101 | if err != nil { 102 | panic(err) 103 | } 104 | providerFile.Close() 105 | return providerFile 106 | } 107 | return nil 108 | } 109 | 110 | func readTestProvider(provider *providers.TestProvider, f *os.File) { 111 | if provider != nil { 112 | providerFile, err := os.Open(f.Name()) 113 | if err != nil { 114 | panic(err) 115 | } 116 | dec := gob.NewDecoder(providerFile) 117 | var testProvider providers.TestProvider 118 | err = dec.Decode(&testProvider) 119 | if err != nil { 120 | panic(err) 121 | } 122 | providerFile.Close() 123 | os.Remove(f.Name()) 124 | *provider = testProvider 125 | } 126 | } 127 | 128 | func TestHostctlCmd(t *testing.T) { 129 | t.Parallel() 130 | stdout, stderr := testRunCmd(t, "hostctl", 0, nil, nil) 131 | ensure.StringContains(t, stdout.String(), Hostctl.Short) 132 | ensure.StringContains(t, stdout.String(), "Usage:") 133 | ensure.StringContains(t, stdout.String(), "Available Commands:") 134 | ensure.StringContains(t, stdout.String(), "Flags:") 135 | ensure.DeepEqual(t, stderr.String(), "") 136 | } 137 | 138 | func TestVersionCmd(t *testing.T) { 139 | t.Parallel() 140 | stdout, stderr := testRunCmd(t, "hostctl version", 0, nil, func() { 141 | Version = "0" 142 | versionChecker = usage.NewChecker("hostctl", Version) 143 | }) 144 | ensure.DeepEqual(t, stdout.String(), "0\n") 145 | ensure.DeepEqual(t, stderr.String(), "") 146 | } 147 | -------------------------------------------------------------------------------- /aws/provider.go: -------------------------------------------------------------------------------- 1 | package aws 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | "time" 7 | 8 | "github.com/MattAitchison/env" 9 | "github.com/aws/aws-sdk-go/aws" 10 | "github.com/aws/aws-sdk-go/service/ec2" 11 | 12 | "github.com/gliderlabs/hostctl/providers" 13 | ) 14 | 15 | func init() { 16 | providers.Register(&awsProvider{}, "aws") 17 | } 18 | 19 | type awsProvider struct { 20 | client *ec2.EC2 21 | } 22 | 23 | // Setup ec2.Client using aws supported credentials (env, credential file) 24 | func (p *awsProvider) Setup() error { 25 | region := os.Getenv("HOSTCTL_REGION") 26 | if region == "" { 27 | return fmt.Errorf("HOSTCTL_REGION required") 28 | } 29 | config := aws.NewConfig().WithRegion(region) 30 | p.client = ec2.New(config) 31 | return nil 32 | } 33 | 34 | func (p *awsProvider) Env() *env.EnvSet { 35 | var envSet = env.NewEnvSet("aws") 36 | envSet.Secret("AWS_ACCESS_KEY", "access key for AWS") 37 | envSet.Secret("AWS_SECRET_KEY", "secret key for AWS") 38 | envSet.String("AWS_AVAILABILITY_ZONE", "", "availability zone for AWS; eg: us-west-2a") 39 | return envSet 40 | } 41 | 42 | // Create an instance based on a Host, poll until instance.Status=running 43 | func (p *awsProvider) Create(host providers.Host) error { 44 | zone := os.Getenv("AWS_AVAILABILITY_ZONE") 45 | res, err := p.client.RunInstances(&ec2.RunInstancesInput{ 46 | ImageId: aws.String(host.Image), 47 | MaxCount: aws.Int64(1), 48 | MinCount: aws.Int64(1), 49 | UserData: aws.String(host.Userdata), 50 | KeyName: aws.String(host.Keyname), 51 | InstanceType: aws.String(host.Flavor), 52 | Placement: &ec2.Placement{ 53 | AvailabilityZone: aws.String(zone), 54 | }, 55 | }) 56 | if err != nil { 57 | return err 58 | } 59 | if res == nil || len(res.Instances) == 0 { 60 | return fmt.Errorf("no instances created") 61 | } 62 | instanceID := res.Instances[0].InstanceId 63 | _, err = p.client.CreateTags(&ec2.CreateTagsInput{ 64 | Resources: []*string{instanceID}, 65 | Tags: []*ec2.Tag{ 66 | { 67 | Key: aws.String("Name"), 68 | Value: aws.String(host.Name), 69 | }, 70 | }, 71 | }) 72 | if err != nil { 73 | return err 74 | } 75 | return p.pollFor(*instanceID, "running") 76 | } 77 | 78 | // Destroy the first instance with tag:Name=name 79 | func (p *awsProvider) Destroy(name string) error { 80 | for id, host := range p.list(name) { 81 | if host.Name == name { 82 | return p.destroy(id) 83 | } 84 | } 85 | return nil 86 | } 87 | 88 | // destroy instance by ID polling until instance.State=terminated 89 | func (p *awsProvider) destroy(id string) error { 90 | _, err := p.client.TerminateInstances(&ec2.TerminateInstancesInput{ 91 | InstanceIds: []*string{aws.String(id)}, 92 | }) 93 | if err != nil { 94 | return err 95 | } 96 | return p.pollFor(id, "terminated") 97 | } 98 | 99 | func (p *awsProvider) List(pattern string) []providers.Host { 100 | var hosts []providers.Host 101 | for _, host := range p.list(pattern) { 102 | hosts = append(hosts, host) 103 | } 104 | return hosts 105 | } 106 | 107 | // list all instances with tag:Name matching pattern. 108 | // Will NOT return any instances where PublicIpAddress is nil. 109 | // Map key will be set to the instanceID. 110 | func (p *awsProvider) list(pattern string) map[string]providers.Host { 111 | res, err := p.client.DescribeInstances(&ec2.DescribeInstancesInput{ 112 | Filters: []*ec2.Filter{ 113 | { 114 | Name: aws.String("tag:Name"), 115 | Values: []*string{aws.String(pattern)}, 116 | }, 117 | }, 118 | }) 119 | if err != nil { 120 | return nil 121 | } 122 | hosts := make(map[string]providers.Host) 123 | for i := range res.Reservations { 124 | for _, instance := range res.Reservations[i].Instances { 125 | // Note: an instance that is terminated/terminating will not have a public IP. 126 | if instance != nil && instance.PublicIpAddress != nil { 127 | id := *instance.InstanceId 128 | hosts[id] = providers.Host{ 129 | Name: nametag(instance), 130 | IP: *instance.PublicIpAddress, 131 | Region: *instance.Placement.AvailabilityZone, 132 | Image: *instance.ImageId, 133 | Keyname: *instance.KeyName, 134 | Flavor: *instance.InstanceType, 135 | } 136 | } 137 | } 138 | } 139 | return hosts 140 | } 141 | 142 | // Get the first instance with tag:Name=name 143 | func (p *awsProvider) Get(name string) *providers.Host { 144 | hosts := p.List(name) 145 | for _, host := range hosts { 146 | if host.Name == name { 147 | return &host 148 | } 149 | } 150 | return nil 151 | } 152 | 153 | // pollFor will poll every 2 seconds for an instance by ID, 154 | // until instance.State=state, timeout or an error. 155 | func (p *awsProvider) pollFor(id string, state string) error { 156 | timeout := time.After(1 * time.Minute) 157 | for { 158 | select { 159 | case <-timeout: 160 | return fmt.Errorf("aws provider: timed out wating for state: %s", state) 161 | default: 162 | } 163 | res, err := p.client.DescribeInstances(&ec2.DescribeInstancesInput{ 164 | InstanceIds: []*string{aws.String(id)}, 165 | }) 166 | if err != nil { 167 | return err 168 | } 169 | for i := range res.Reservations { 170 | for _, instance := range res.Reservations[i].Instances { 171 | if *instance.State.Name == state { 172 | return nil 173 | } 174 | } 175 | } 176 | time.Sleep(2 * time.Second) 177 | } 178 | } 179 | 180 | func nametag(instance *ec2.Instance) string { 181 | for _, tag := range instance.Tags { 182 | if *tag.Key == "Name" { 183 | return *tag.Value 184 | } 185 | } 186 | return "" 187 | } 188 | -------------------------------------------------------------------------------- /docs/user/quickstart.md: -------------------------------------------------------------------------------- 1 | # Quickstart 2 | 3 | This is a short, simple tutorial intended to get you started with Hostctl as 4 | quickly as possible. Alternatively, you can skip ahead to the [Command Reference](reference.md). 5 | 6 | ## Overview 7 | 8 | Hostctl lets you spin up and down VMs with cloud providers like DigitalOcean and EC2. You can also use Hostctl to easily get their IP, SSH to them, and scale them. The goal is to make it easier when doing development and experiments. 9 | 10 | In this tutorial, we're going to use Hostctl with DigitalOcean to tour all the functionality it provides. 11 | 12 | ## Installing 13 | 14 | If you haven't already, you want to download Hostctl for your platform. You can find the latest version on the [releases page](https://github.com/gliderlabs/hostctl/releases), or you can install from this URL providing your platform. Using `curl` and `tar` you can install `hostctl` right into a directory in your `$PATH`: 15 | ``` 16 | $ curl https://dl.gliderlabs.com/qs/hostctl/latest/$(uname -sm|tr \ _).tgz \ 17 | | tar -zxC /usr/local/bin 18 | ``` 19 | You can change `/usr/local/bin` as necessary. Now you should be able to run `hostctl` and see usage and available commands: 20 | 21 | $ hostctl 22 | 23 | ## Environment 24 | 25 | VMs require a lot of parameters, which is one of the annoying parts of booting a VM that slows you down. Hostctl addresses this by letting you define parameters upfront in your environment, which you can put into loadable profiles. 26 | 27 | You can see everything you can configure in the environment with `hostctl env`: 28 | ``` 29 | $ hostctl env 30 | HOSTCTL_IMAGE="" # vm image 31 | HOSTCTL_REGION="" # vm region 32 | HOSTCTL_KEYNAME="" # vm keyname 33 | HOSTCTL_USER="progrium" # ssh user 34 | HOSTCTL_PROVIDER="digitalocean" # cloud provider 35 | HOSTCTL_NAME="" # optional default name 36 | HOSTCTL_NAMESPACE="" # optional namespace for names 37 | HOSTCTL_FLAVOR="" # vm flavor 38 | HOSTCTL_USERDATA="" # vm user data 39 | DO_TOKEN="" # token for DigitalOcean API v2 40 | ``` 41 | We see they're mostly empty except for some defaults. `HOSTCTL_USER` defaults to your system's logged in user. `HOSTCTL_PROVIDER` defaults to DigitalOcean. 42 | 43 | ## Cloud Provider Setup 44 | 45 | The values you want for the rest are going to depend on the provider. We're going to focus on DigitalOcean. You can see our [Provider Reference](providers.md) to see what values you want for EC2, for example. 46 | 47 | There's two values you'll have to lookup for this to work: `DO_TOKEN` which is a personal access token for the API, and `HOSTCTL_KEYNAME` which is typically the fingerprint (or ID) of the public key you want to use. 48 | 49 | If you don't have a personal access token, or to make a new one, you can go to [Applications & API](https://cloud.digitalocean.com/settings/applications) once logged into DigitalOcean. The fingerprint values for your SSH keys are under [Security](https://cloud.digitalocean.com/settings/security) in your account settings. 50 | 51 | It's easiest to write these to the global Hostctl profile, a simple shell config script at `~/.hostctl` always sourced by `hostctl`. We'll write two lines to it: 52 | 53 | $ echo "export DO_TOKEN=your-token" >> ~/.hostctl 54 | $ echo "export HOSTCTL_KEYNAME=your-ssh-key-fingerprint" >> ~/.hostctl 55 | 56 | ## Base VM Attributes 57 | 58 | The rest of the required environment defines your VM. `HOSTCTL_IMAGE`, `HOSTCTL_FLAVOR`, and `HOSTCTL_REGION`. For DigitalOcean, these are slug values from the API. In our case, we'll just use these: 59 | 60 | $ export HOSTCTL_IMAGE=ubuntu 61 | $ export HOSTCTL_FLAVOR=512mb 62 | $ export HOSTCTL_REGION=nyc1 63 | $ export HOSTCTL_USER=root 64 | 65 | We also set `HOSTCTL_USER` since DigitalOcean's default user is `root`. We're setting these in our terminal session environment, but they could also be defined anywhere else in your environment. Later we'll see how we can make them into loadable profiles. 66 | 67 | We can run `hostctl env` again to see its current configuration. 68 | 69 | ## Provisioning 70 | 71 | Let's make a VM called `demo`: 72 | 73 | $ hostctl up demo 74 | 75 | The command will wait until the VM is ready to go, showing a progress bar while you wait. Take a moment to think about how easy that was. 76 | 77 | ## Everything Else 78 | 79 | We can list our VMs and see `demo`, as well as any other VMs you might have running on this account: 80 | 81 | $ hostctl ls 82 | demo 83 | 84 | We can get the IP for `demo` very easily: 85 | 86 | $ hostctl ip demo 87 | 198.5.101.164 88 | 89 | We could use this with SSH to connect to it by name: 90 | 91 | $ ssh root@$(hostctl ip demo) 92 | 93 | Although it's easier to use the builtin convenience command: 94 | 95 | $ hostctl ssh demo 96 | 97 | We can boot more VMs with `hostctl up` or we could create a cluster with `hostctl scale`: 98 | 99 | $ hostctl scale node 3 100 | .......................................... 101 | $ hostctl ls 102 | demo 103 | node.0 104 | node.1 105 | node.2 106 | 107 | We can see it made 3 hosts called `node`. We could use scale again to resize the cluster or just scale to nothing: 108 | 109 | $ hostctl scale node 0 110 | .......................................... 111 | $ hostctl ls 112 | demo 113 | 114 | But we can also just shutdown any hosts with `hostctl down`: 115 | 116 | $ hostctl down demo 117 | 118 | ## Profiles 119 | 120 | With our VM attributes in the environment, we can iteratively change them. Maybe I want the same VM, but using the `docker` image instead of `ubuntu`. Just set it: 121 | 122 | $ export HOSTCTL_IMAGE=docker 123 | $ hostctl up docker-vm1 124 | 125 | However, if we close this session and come back tomorrow, we won't have this environment. We'd have to set it all up again. But we can write our current environment to a profile that we can use later. We do this with `hostctl env --export`: 126 | 127 | $ hostctl env --export > docker.profile 128 | 129 | If you look at the contents of `docker.profile`, it's a shell script that exports everything that was in your current hostctl environment. Now tomorrow we can use the profile without setting any environment: 130 | 131 | $ hostctl -p docker.profile up docker-vm2 132 | 133 | Profiles make it easy to iteratively build up a VM configuration and then save it to a file you can use later. You can also write profiles from scratch. They're just shell scripts, so the above is basically the same as: 134 | 135 | $ source docker.profile 136 | $ hostctl up docker-vm2 137 | 138 | ## Next Steps 139 | 140 | And that's not all. Check out the [Command Reference](reference.md) for what else you can do with these commands, or [Provider Reference](providers.md) to try EC2. 141 | --------------------------------------------------------------------------------