├── .circleci └── config.yml ├── .codeflow.yml ├── .gitignore ├── CONTRIBUTING.md ├── Dockerfile ├── LICENSE ├── README.md ├── assets ├── odin-deploy.gif ├── odin-logo.png └── sm.png ├── aws ├── alarms │ ├── alarm_input.go │ ├── alarm_input_test.go │ └── policy_input.go ├── alb │ ├── target_group.go │ └── target_group_test.go ├── ami │ ├── image.go │ └── image_test.go ├── asg │ ├── asg.go │ ├── asg_input.go │ ├── asg_input_test.go │ └── asg_test.go ├── aws.go ├── elb │ ├── elb.go │ └── elb_test.go ├── iam │ ├── iam.go │ └── iam_test.go ├── instances.go ├── instances_test.go ├── lc │ ├── launch_configuration.go │ ├── launch_configuration_input.go │ └── launch_configuration_input_test.go ├── mocks │ ├── clients.go │ ├── mock_alb.go │ ├── mock_asg.go │ ├── mock_cw.go │ ├── mock_ec2.go │ ├── mock_elb.go │ ├── mock_iam.go │ └── mock_sns.go ├── pg │ ├── placement_group.go │ └── placement_group_test.go ├── sg │ ├── security_group.go │ └── security_group_test.go ├── sns │ └── sns.go └── subnet │ ├── subnet_test.go │ └── subnets.go ├── client ├── client.go ├── client_test.go ├── deploy.go ├── deploy_test.go ├── fails.go ├── halt.go └── halt_test.go ├── deployer ├── fuzz_test.go ├── handlers.go ├── handlers_test.go ├── helpers_test.go ├── intergration_test.go ├── machine.go └── models │ ├── autoscaling.go │ ├── autoscaling_test.go │ ├── lifecycle.go │ ├── lifecycle_test.go │ ├── mocks.go │ ├── policy.go │ ├── policy_test.go │ ├── release.go │ ├── release_parsing.go │ ├── release_parsing_test.go │ ├── release_resources.go │ ├── release_resources_test.go │ ├── release_safe.go │ ├── release_safe_test.go │ ├── release_test.go │ ├── service.go │ ├── service_resources.go │ ├── service_resources_test.go │ ├── service_test.go │ ├── strategy.go │ └── strategy_test.go ├── go.mod ├── go.sum ├── odin.go ├── releases ├── deploy-test-release.json ├── deploy-test-release.json.userdata ├── null-release.json └── null-release.json.userdata ├── resources ├── deploy-test-resources.rb ├── odin.rb ├── odin_assumed_policy.json.erb ├── odin_lambda_policy.json.erb └── vpc-resources.rb └── scripts ├── bootstrap ├── bootstrap_deployer ├── build_lambda_zip ├── deploy_deployer ├── deploy_test_deploy ├── deploy_test_halt ├── geo └── graph /.circleci/config.yml: -------------------------------------------------------------------------------- 1 | version: 2 2 | jobs: 3 | build: 4 | docker: 5 | - image: circleci/golang:1.14 6 | working_directory: /go/src/github.com/coinbase/odin 7 | steps: 8 | - checkout 9 | - run: export GO111MODULE=on && go mod download 10 | - run: export GO111MODULE=on && go test ./... 11 | 12 | -------------------------------------------------------------------------------- /.codeflow.yml: -------------------------------------------------------------------------------- 1 | deploy: 2 | engine: Step 3 | operate: 4 | slack_channels: 5 | - "#infra-deploys" 6 | secure: 7 | required_reviews: 1 8 | upstream_repository: coinbase/odin 9 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | step.zip 3 | step 4 | lambda 5 | lambda.zip 6 | tmp 7 | coverage.out 8 | coverage.html 9 | vendor 10 | odin 11 | odin2 12 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to Odin 2 | 3 | ## Code of Conduct 4 | 5 | All interactions with this project follow our [Code of Conduct][code-of-conduct]. 6 | By participating, you are expected to honor this code. Violators can be banned 7 | from further participation in this project, or potentially all Coinbase projects. 8 | 9 | [code-of-conduct]: https://github.com/coinbase/code-of-conduct 10 | 11 | ## Bug Reports 12 | 13 | * Ensure your issue [has not already been reported][1]. It may already be fixed! 14 | * Include the steps you carried out to produce the problem. 15 | * Include the behavior you observed along with the behavior you expected, and 16 | why you expected it. 17 | * Include any relevant stack traces or debugging output. 18 | 19 | ## Feature Requests 20 | 21 | We welcome feedback with or without pull requests. If you have an idea for how 22 | to improve the project, great! All we ask is that you take the time to write a 23 | clear and concise explanation of what need you are trying to solve. If you have 24 | thoughts on _how_ it can be solved, include those too! 25 | 26 | The best way to see a feature added, however, is to submit a pull request. 27 | 28 | ## Pull Requests 29 | 30 | * Before creating your pull request, it's usually worth asking if the code 31 | you're planning on writing will actually be considered for merging. You can 32 | do this by [opening an issue][1] and asking. It may also help give the 33 | maintainers context for when the time comes to review your code. 34 | 35 | * Ensure your [commit messages are well-written][2]. This can double as your 36 | pull request message, so it pays to take the time to write a clear message. 37 | 38 | * Add tests for your feature. You should be able to look at other tests for 39 | examples. If you're unsure, don't hesitate to [open an issue][1] and ask! 40 | 41 | * Submit your pull request! 42 | 43 | ## Support Requests 44 | 45 | For security reasons, any communication referencing support tickets for Coinbase 46 | products will be ignored. The request will have its content redacted and will 47 | be locked to prevent further discussion. 48 | 49 | All support requests must be made via [our support team][3]. 50 | 51 | [1]: https://github.com/coinbase/step/issues 52 | [2]: https://medium.com/brigade-engineering/the-secrets-to-great-commit-messages-106fc0a92a25 53 | [3]: https://support.coinbase.com/customer/en/portal/articles/2288496-how-can-i-contact-coinbase-support- 54 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM golang@sha256:6486ea568f95953b86c9687c1e656f4297d9b844481e645a00c0602f26fee136 2 | 3 | # Install Zip 4 | RUN apt-get update && apt-get upgrade -y && apt-get install -y zip 5 | 6 | WORKDIR /go/src/github.com/coinbase/odin 7 | 8 | ENV GO111MODULE on 9 | ENV GOPATH /go 10 | 11 | COPY go.mod go.sum ./ 12 | RUN go mod download 13 | 14 | COPY . . 15 | 16 | RUN go build && go install 17 | 18 | # builds lambda.zip 19 | RUN ./scripts/build_lambda_zip 20 | RUN shasum -a 256 lambda.zip | awk '{print $1}' > lambda.zip.sha256 21 | 22 | RUN mv lambda.zip.sha256 lambda.zip / 23 | RUN odin json > /state_machine.json 24 | 25 | CMD ["odin"] 26 | -------------------------------------------------------------------------------- /assets/odin-deploy.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coinbase/odin/b58e2672dd575d6fa65ca3cbec6154f486ab034b/assets/odin-deploy.gif -------------------------------------------------------------------------------- /assets/odin-logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coinbase/odin/b58e2672dd575d6fa65ca3cbec6154f486ab034b/assets/odin-logo.png -------------------------------------------------------------------------------- /assets/sm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coinbase/odin/b58e2672dd575d6fa65ca3cbec6154f486ab034b/assets/sm.png -------------------------------------------------------------------------------- /aws/alarms/alarm_input.go: -------------------------------------------------------------------------------- 1 | package alarms 2 | 3 | import ( 4 | "fmt" 5 | "strings" 6 | 7 | "github.com/aws/aws-sdk-go/service/cloudwatch" 8 | "github.com/coinbase/odin/aws" 9 | ) 10 | 11 | // AlarmInput struct 12 | type AlarmInput struct { 13 | *cloudwatch.PutMetricAlarmInput 14 | } 15 | 16 | // Create the alarm input 17 | func (alarm *AlarmInput) Create(cwc aws.CWAPI) (*cloudwatch.PutMetricAlarmOutput, error) { 18 | return cwc.PutMetricAlarm(alarm.PutMetricAlarmInput) 19 | } 20 | 21 | // SetAlarmDescription takes the alarms values to build a description 22 | func (alarm *AlarmInput) SetAlarmDescription() { 23 | desc := []string{} 24 | 25 | if alarm.AlarmName != nil { 26 | desc = append(desc, fmt.Sprintf("Scale-%v", *alarm.AlarmName)) 27 | } 28 | 29 | if alarm.Statistic != nil && alarm.MetricName != nil { 30 | desc = append(desc, fmt.Sprintf(" if %v %v", *alarm.Statistic, *alarm.MetricName)) 31 | } 32 | 33 | if alarm.ComparisonOperator != nil && alarm.Threshold != nil { 34 | desc = append(desc, fmt.Sprintf(" is %v %v%v", *alarm.ComparisonOperator, *alarm.Threshold, "%")) 35 | } 36 | 37 | if alarm.Period != nil && alarm.EvaluationPeriods != nil { 38 | desc = append(desc, fmt.Sprintf(" for %v seconds %v times in a row", *alarm.Period, *alarm.EvaluationPeriods)) 39 | } 40 | 41 | descstr := strings.Join(desc, "\n") 42 | 43 | alarm.AlarmDescription = &descstr 44 | } 45 | -------------------------------------------------------------------------------- /aws/alarms/alarm_input_test.go: -------------------------------------------------------------------------------- 1 | package alarms 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/aws/aws-sdk-go/service/cloudwatch" 7 | fuzz "github.com/google/gofuzz" 8 | ) 9 | 10 | func Test_SetAlarmDescription_Fuzz(t *testing.T) { 11 | // Making sure the descrption never panics 12 | for i := 0; i < 50; i++ { 13 | f := fuzz.New() 14 | aii := cloudwatch.PutMetricAlarmInput{} 15 | ai := AlarmInput{&aii} 16 | f.Fuzz(&aii) 17 | ai.SetAlarmDescription() 18 | } 19 | } 20 | -------------------------------------------------------------------------------- /aws/alarms/policy_input.go: -------------------------------------------------------------------------------- 1 | package alarms 2 | 3 | import ( 4 | "github.com/aws/aws-sdk-go/service/autoscaling" 5 | "github.com/coinbase/odin/aws" 6 | ) 7 | 8 | // PolicyInput struct 9 | type PolicyInput struct { 10 | *autoscaling.PutScalingPolicyInput 11 | } 12 | 13 | // Create a policy 14 | func (a *PolicyInput) Create(asgc aws.ASGAPI) (*autoscaling.PutScalingPolicyOutput, error) { 15 | return asgc.PutScalingPolicy(a.PutScalingPolicyInput) 16 | } 17 | -------------------------------------------------------------------------------- /aws/alb/target_group.go: -------------------------------------------------------------------------------- 1 | package alb 2 | 3 | import ( 4 | "fmt" 5 | "strconv" 6 | 7 | "github.com/aws/aws-sdk-go/service/elbv2" 8 | "github.com/coinbase/odin/aws" 9 | "github.com/coinbase/step/utils/to" 10 | ) 11 | 12 | // TargetGroup struct 13 | type TargetGroup struct { 14 | ProjectNameTag *string 15 | ConfigNameTag *string 16 | ServiceNameTag *string 17 | AllowedServiceTag *string 18 | TargetGroupArn *string 19 | TargetGroupName *string 20 | SlowStartDuration int 21 | } 22 | 23 | // ProjectName returns tag 24 | func (s *TargetGroup) ProjectName() *string { 25 | return s.ProjectNameTag 26 | } 27 | 28 | // ConfigName returns tag 29 | func (s *TargetGroup) ConfigName() *string { 30 | return s.ConfigNameTag 31 | } 32 | 33 | // ServiceName returns tag 34 | func (s *TargetGroup) ServiceName() *string { 35 | return s.ServiceNameTag 36 | } 37 | 38 | // Name returns tag 39 | func (s *TargetGroup) Name() *string { 40 | return s.TargetGroupName 41 | } 42 | 43 | // AllowedService returns which service is allowed to attach to it 44 | func (s *TargetGroup) AllowedService() *string { 45 | if s.ProjectNameTag == nil || s.ConfigNameTag == nil || s.ServiceNameTag == nil { 46 | return to.Strp("no services allowed") 47 | } 48 | if s.AllowedServiceTag == nil { 49 | return to.Strp(fmt.Sprintf("%s::%s::%s", *s.ProjectName(), *s.ConfigName(), *s.ServiceName())) 50 | } 51 | return s.AllowedServiceTag 52 | } 53 | 54 | ////// 55 | // Healthy 56 | ////// 57 | 58 | // GetInstances return instances on the target group 59 | func GetInstances(albc aws.ALBAPI, arn *string, instances []string) (aws.Instances, error) { 60 | healthOutput, err := albc.DescribeTargetHealth(createDescribeTargetHealthInput(arn, instances)) 61 | 62 | if err != nil { 63 | return nil, err 64 | } 65 | 66 | tgInstances := aws.Instances{} 67 | for _, thd := range healthOutput.TargetHealthDescriptions { 68 | tgInstances.AddTargetGroupInstance(thd) 69 | } 70 | 71 | return tgInstances, nil 72 | } 73 | 74 | func createDescribeTargetHealthInput(arn *string, instances []string) *elbv2.DescribeTargetHealthInput { 75 | awsInstances := []*elbv2.TargetDescription{} 76 | for _, id := range instances { 77 | awsInstances = append(awsInstances, &elbv2.TargetDescription{Id: to.Strp(id)}) 78 | } 79 | 80 | return &elbv2.DescribeTargetHealthInput{ 81 | TargetGroupArn: arn, 82 | Targets: awsInstances, 83 | } 84 | } 85 | 86 | ////// 87 | // Find 88 | ////// 89 | 90 | // FindAll returns all target groups in a list 91 | func FindAll(albc aws.ALBAPI, names []*string) ([]*TargetGroup, error) { 92 | tgs := []*TargetGroup{} 93 | for _, name := range names { 94 | tg, err := find(albc, name) 95 | if err != nil { 96 | return nil, err 97 | } 98 | tgs = append(tgs, tg) 99 | } 100 | 101 | return tgs, nil 102 | } 103 | 104 | func find(alb aws.ALBAPI, targetGroupName *string) (*TargetGroup, error) { 105 | awsTarget, err := findByName(alb, targetGroupName) 106 | if err != nil { 107 | return nil, err 108 | } 109 | 110 | awsTags, err := findTagsByName(alb, awsTarget.TargetGroupArn) 111 | if err != nil { 112 | return nil, err 113 | } 114 | 115 | slowStartDuration := findSlowStartDuration(alb, awsTarget.TargetGroupArn) 116 | 117 | return &TargetGroup{ 118 | ProjectNameTag: aws.FetchELBV2Tag(awsTags, to.Strp("ProjectName")), 119 | ConfigNameTag: aws.FetchELBV2Tag(awsTags, to.Strp("ConfigName")), 120 | ServiceNameTag: aws.FetchELBV2Tag(awsTags, to.Strp("ServiceName")), 121 | AllowedServiceTag: aws.FetchELBV2Tag(awsTags, to.Strp("AllowedService")), 122 | TargetGroupArn: awsTarget.TargetGroupArn, 123 | TargetGroupName: targetGroupName, 124 | SlowStartDuration: slowStartDuration, 125 | }, nil 126 | } 127 | 128 | func findByName(alb aws.ALBAPI, targetGroupName *string) (*elbv2.TargetGroup, error) { 129 | elbsOutput, err := alb.DescribeTargetGroups(&elbv2.DescribeTargetGroupsInput{ 130 | Names: []*string{targetGroupName}, 131 | }) 132 | 133 | if err != nil { 134 | return nil, err 135 | } 136 | 137 | if len(elbsOutput.TargetGroups) != 1 { 138 | return nil, fmt.Errorf("LoadBalancer Not Found") 139 | } 140 | 141 | if *elbsOutput.TargetGroups[0].TargetGroupName != *targetGroupName { 142 | return nil, fmt.Errorf("LoadBalancer Not Found") 143 | } 144 | 145 | return elbsOutput.TargetGroups[0], nil 146 | } 147 | 148 | func findTagsByName(alb aws.ALBAPI, targetGroupARN *string) ([]*elbv2.Tag, error) { 149 | tagsOutput, err := alb.DescribeTags(&elbv2.DescribeTagsInput{ 150 | ResourceArns: []*string{targetGroupARN}, 151 | }) 152 | 153 | if err != nil { 154 | return nil, err 155 | } 156 | 157 | if len(tagsOutput.TagDescriptions) != 1 { 158 | return nil, fmt.Errorf("TargetGroup Not Found") 159 | } 160 | 161 | if *tagsOutput.TagDescriptions[0].ResourceArn != *targetGroupARN { 162 | return nil, fmt.Errorf("TargetGroup Not Found") 163 | } 164 | 165 | return tagsOutput.TagDescriptions[0].Tags, nil 166 | } 167 | 168 | func findSlowStartDuration(alb aws.ALBAPI, targetGroupARN *string) int { 169 | output, err := alb.DescribeTargetGroupAttributes(&elbv2.DescribeTargetGroupAttributesInput{TargetGroupArn: targetGroupARN}) 170 | if err != nil { 171 | return 0 172 | } 173 | for _, attribute := range output.Attributes { 174 | if attribute.Key == nil || attribute.Value == nil { 175 | continue 176 | } 177 | 178 | if *attribute.Key == "slow_start.duration_seconds" { 179 | duration, err := strconv.Atoi(*attribute.Value) 180 | if err != nil { 181 | continue 182 | } 183 | return duration 184 | } 185 | } 186 | return 0 187 | } 188 | -------------------------------------------------------------------------------- /aws/alb/target_group_test.go: -------------------------------------------------------------------------------- 1 | package alb 2 | 3 | import ( 4 | "sort" 5 | "testing" 6 | 7 | "github.com/coinbase/odin/aws/mocks" 8 | "github.com/coinbase/step/utils/to" 9 | "github.com/stretchr/testify/assert" 10 | ) 11 | 12 | func Test_AllowedService_ExplicitValue(t *testing.T) { 13 | explicitService := "other/project::other-config::other-service" 14 | 15 | tg := TargetGroup{ 16 | ProjectNameTag: to.Strp("project"), 17 | ConfigNameTag: to.Strp("config"), 18 | ServiceNameTag: to.Strp("service"), 19 | AllowedServiceTag: to.Strp(explicitService), 20 | } 21 | service := tg.AllowedService() 22 | assert.Equal(t, *service, explicitService) 23 | } 24 | 25 | func Test_AllowedService_ImplicitValue(t *testing.T) { 26 | tg := TargetGroup{ 27 | ProjectNameTag: to.Strp("project"), 28 | ConfigNameTag: to.Strp("config"), 29 | ServiceNameTag: to.Strp("service"), 30 | } 31 | service := tg.AllowedService() 32 | assert.Equal(t, *service, "project::config::service") 33 | } 34 | 35 | func Test_FindAll_Empty(t *testing.T) { 36 | albc := &mocks.ALBClient{} 37 | am, err := FindAll(albc, []*string{}) 38 | assert.NoError(t, err) 39 | assert.Equal(t, 0, len(am)) 40 | } 41 | 42 | func Test_FindAll_NotFound(t *testing.T) { 43 | albc := &mocks.ALBClient{} 44 | _, err := FindAll(albc, []*string{to.Strp("tg_name")}) 45 | assert.Error(t, err) 46 | } 47 | 48 | func Test_FindAll_Found(t *testing.T) { 49 | albc := &mocks.ALBClient{} 50 | albc.AddTargetGroup(mocks.MockTargetGroup{}) 51 | albc.AddTargetGroup(mocks.MockTargetGroup{Name: "tg_other_name"}) 52 | am, err := FindAll(albc, []*string{to.Strp("tg_name"), to.Strp("tg_other_name")}) 53 | 54 | assert.NoError(t, err) 55 | assert.Equal(t, 2, len(am)) 56 | assert.Equal(t, *am[0].TargetGroupArn, "tg_name") 57 | assert.Equal(t, *am[1].TargetGroupArn, "tg_other_name") 58 | } 59 | 60 | func Test_GetInstances(t *testing.T) { 61 | albc := &mocks.ALBClient{} 62 | albc.AddTargetGroup(mocks.MockTargetGroup{}) 63 | 64 | instances, err := GetInstances(albc, to.Strp("tg_name"), []string{"InstanceId"}) 65 | assert.NoError(t, err) 66 | assert.Equal(t, 1, len(instances)) 67 | } 68 | 69 | func Test_createDescribeTargetHealthInput(t *testing.T) { 70 | name := "" 71 | 72 | in := createDescribeTargetHealthInput(&name, []string{}) 73 | assert.Equal(t, len(in.Targets), 0) 74 | 75 | in = createDescribeTargetHealthInput(&name, []string{"a"}) 76 | assert.Equal(t, len(in.Targets), 1) 77 | assert.Equal(t, *in.Targets[0].Id, "a") 78 | 79 | in = createDescribeTargetHealthInput(&name, []string{"a", "b"}) 80 | tgsIDs := []string{} 81 | for _, tg := range in.Targets { 82 | tgsIDs = append(tgsIDs, *tg.Id) 83 | } 84 | 85 | sort.Strings(tgsIDs) // Sort not Guaranteed by map 86 | 87 | assert.Equal(t, len(tgsIDs), 2) 88 | assert.Equal(t, tgsIDs[0], "a") 89 | assert.Equal(t, tgsIDs[1], "b") 90 | } 91 | -------------------------------------------------------------------------------- /aws/ami/image.go: -------------------------------------------------------------------------------- 1 | package ami 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/aws/aws-sdk-go/service/ec2" 7 | "github.com/coinbase/odin/aws" 8 | "github.com/coinbase/step/utils/to" 9 | ) 10 | 11 | // Image struct 12 | type Image struct { 13 | ImageID *string 14 | DeployWithTag *string 15 | } 16 | 17 | func isID(name string) bool { 18 | if len(name) < 5 { 19 | return false 20 | } 21 | 22 | return (name)[0:4] == "ami-" 23 | } 24 | 25 | // Find takes either a ID or a Tag of an ami e.g. ubuntu or ami-00000000 26 | func Find(ec2c aws.EC2API, nameTagOrID *string) (*Image, error) { 27 | if nameTagOrID == nil { 28 | return nil, fmt.Errorf("AMI Image nil") 29 | } 30 | if isID(*nameTagOrID) { 31 | return findByID(ec2c, nameTagOrID) 32 | } 33 | return findByTag(ec2c, nameTagOrID) 34 | } 35 | 36 | func findByID(ec2c aws.EC2API, id *string) (*Image, error) { 37 | return find(ec2c, &ec2.DescribeImagesInput{ImageIds: []*string{id}}) 38 | } 39 | 40 | func findByTag(ec2c aws.EC2API, nameTag *string) (*Image, error) { 41 | filters := []*ec2.Filter{ 42 | &ec2.Filter{ 43 | Name: to.Strp("tag:Name"), 44 | Values: []*string{nameTag}, 45 | }, 46 | } 47 | 48 | return find(ec2c, &ec2.DescribeImagesInput{Filters: filters}) 49 | } 50 | 51 | func find(ec2c aws.EC2API, in *ec2.DescribeImagesInput) (*Image, error) { 52 | output, err := ec2c.DescribeImages(in) 53 | 54 | if err != nil { 55 | return nil, err 56 | } 57 | 58 | switch len(output.Images) { 59 | case 0: 60 | return nil, nil 61 | case 1: 62 | im := output.Images[0] 63 | if im == nil { 64 | return nil, fmt.Errorf("AMI Image nil") 65 | } 66 | return &Image{ 67 | im.ImageId, 68 | aws.FetchEc2Tag(im.Tags, to.Strp("DeployWith")), 69 | }, nil 70 | default: 71 | return nil, fmt.Errorf("Must be exactly 1 Image with tag Name, there are %v", len(output.Images)) 72 | } 73 | } 74 | -------------------------------------------------------------------------------- /aws/ami/image_test.go: -------------------------------------------------------------------------------- 1 | package ami 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/odin/aws/mocks" 7 | "github.com/coinbase/step/utils/to" 8 | "github.com/stretchr/testify/assert" 9 | ) 10 | 11 | func Test_isID(t *testing.T) { 12 | assert.True(t, isID("ami-asfasf")) 13 | assert.False(t, isID("ubuntu")) 14 | assert.False(t, isID("amigdshiunet")) 15 | assert.False(t, isID("ami")) 16 | } 17 | 18 | func Test_Find_ID(t *testing.T) { 19 | ec2c := &mocks.EC2Client{} 20 | ec2c.AddImage("ubuntu", "ami-000000") 21 | img, err := Find(ec2c, to.Strp("ami-000000")) 22 | assert.NoError(t, err) 23 | assert.Equal(t, "ami-000000", *img.ImageID) 24 | } 25 | 26 | func Test_Find_Tag(t *testing.T) { 27 | ec2c := &mocks.EC2Client{} 28 | ec2c.AddImage("ubuntu", "ami-000000") 29 | img, err := Find(ec2c, to.Strp("ubuntu")) 30 | assert.NoError(t, err) 31 | assert.Equal(t, "ami-000000", *img.ImageID) 32 | } 33 | -------------------------------------------------------------------------------- /aws/asg/asg_input.go: -------------------------------------------------------------------------------- 1 | package asg 2 | 3 | import ( 4 | "github.com/aws/aws-sdk-go/service/autoscaling" 5 | "github.com/coinbase/odin/aws" 6 | "github.com/coinbase/step/utils/to" 7 | ) 8 | 9 | // Input input struct 10 | type Input struct { 11 | *autoscaling.CreateAutoScalingGroupInput 12 | } 13 | 14 | // Create calls to create an ASG 15 | func (s *Input) Create(asgc aws.ASGAPI) error { 16 | if err := s.Validate(); err != nil { 17 | return err 18 | } 19 | 20 | _, err := asgc.CreateAutoScalingGroup(s.CreateAutoScalingGroupInput) 21 | 22 | if err != nil { 23 | return err 24 | } 25 | 26 | return nil 27 | } 28 | 29 | // SetDefaults assigns default values 30 | func (s *Input) SetDefaults() { 31 | if s.MinSize == nil { 32 | s.MinSize = to.Int64p(1) 33 | } 34 | 35 | if s.MaxSize == nil { 36 | s.MaxSize = to.Int64p(1) 37 | } 38 | 39 | if s.DesiredCapacity == nil { 40 | s.DesiredCapacity = to.Int64p(1) 41 | } 42 | 43 | if s.DefaultCooldown == nil { 44 | s.DefaultCooldown = to.Int64p(300) 45 | } 46 | 47 | if s.HealthCheckGracePeriod == nil { 48 | s.HealthCheckGracePeriod = to.Int64p(300) 49 | } 50 | 51 | if s.LaunchConfigurationName == nil { 52 | s.LaunchConfigurationName = s.AutoScalingGroupName // Makes the name the same 53 | } 54 | 55 | s.HealthCheckType = to.Strp("EC2") 56 | if len(s.LoadBalancerNames) > 0 || len(s.TargetGroupARNs) > 0 { 57 | s.HealthCheckType = to.Strp("ELB") // If there are any ELBs set the health check to that 58 | } 59 | 60 | if len(s.TerminationPolicies) == 0 { 61 | s.TerminationPolicies = []*string{to.Strp("ClosestToNextInstanceHour")} 62 | } 63 | } 64 | 65 | // AddTag adds a tag to the input 66 | func (s *Input) AddTag(key string, value *string) { 67 | if s.Tags == nil { 68 | s.Tags = []*autoscaling.Tag{} 69 | } 70 | 71 | for _, tag := range s.Tags { 72 | if *tag.Key == key { 73 | tag.Value = value 74 | return // Found the tag key already 75 | } 76 | } 77 | 78 | // Add new Tag 79 | s.Tags = append(s.Tags, &autoscaling.Tag{Key: &key, Value: value, PropagateAtLaunch: to.Boolp(true)}) 80 | } 81 | 82 | // ToASG returns ASG object 83 | func (s *Input) ToASG() *ASG { 84 | return &ASG{ 85 | AutoScalingGroupName: s.AutoScalingGroupName, 86 | DesiredCapacity: s.DesiredCapacity, 87 | MinSize: s.MinSize, 88 | } 89 | } 90 | -------------------------------------------------------------------------------- /aws/asg/asg_input_test.go: -------------------------------------------------------------------------------- 1 | package asg 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/aws/aws-sdk-go/service/autoscaling" 7 | fuzz "github.com/google/gofuzz" 8 | ) 9 | 10 | func Test_Defaults(t *testing.T) { 11 | for i := 0; i < 50; i++ { 12 | f := fuzz.New() 13 | aii := autoscaling.CreateAutoScalingGroupInput{} 14 | ai := Input{&aii} 15 | f.Fuzz(&aii) 16 | ai.SetDefaults() 17 | } 18 | } 19 | -------------------------------------------------------------------------------- /aws/asg/asg_test.go: -------------------------------------------------------------------------------- 1 | package asg 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/aws/aws-sdk-go/service/autoscaling" 7 | "github.com/coinbase/odin/aws/mocks" 8 | "github.com/coinbase/step/utils/to" 9 | "github.com/stretchr/testify/assert" 10 | ) 11 | 12 | func Test_GetInstances(t *testing.T) { 13 | //func GetInstances(asgc aws.ASGAPI, asg_name *string) (aws.Instances, error) { 14 | asgc := &mocks.ASGClient{} 15 | _, _, err := GetInstances(asgc, to.Strp("asd")) 16 | assert.Error(t, err) // Not Found 17 | 18 | name := asgc.AddPreviousRuntimeResources("project", "config", "service", "release") 19 | ins, _, err := GetInstances(asgc, to.Strp(name)) 20 | assert.NoError(t, err) 21 | assert.Equal(t, 1, len(ins)) 22 | } 23 | 24 | func Test_ForProjectConfigNotReleaseIDServiceMap(t *testing.T) { 25 | // func ForProjectConfigNotReleaseIDServiceMap(asgc aws.ASGAPI, project_name *string, config_name *string, release_uuid *string) (map[string]*ASG, error) { 26 | asgc := &mocks.ASGClient{} 27 | services, err := ForProjectConfigNotReleaseIDServiceMap(asgc, to.Strp("project"), to.Strp("config"), to.Strp("release")) 28 | assert.NoError(t, err) 29 | assert.Equal(t, 0, len(services)) 30 | 31 | asgc.AddPreviousRuntimeResources("project", "config", "service1", "release") 32 | asgc.AddPreviousRuntimeResources("project", "config", "service2", "release") 33 | asgc.AddPreviousRuntimeResources("project", "config", "service3", "not_release") 34 | asgc.AddPreviousRuntimeResources("not_project", "config", "service4", "release") 35 | asgc.AddPreviousRuntimeResources("project", "not_config", "service5", "release") 36 | 37 | services, err = ForProjectConfigNotReleaseIDServiceMap(asgc, to.Strp("project"), to.Strp("config"), to.Strp("release")) 38 | assert.NoError(t, err) 39 | assert.Equal(t, 1, len(services)) 40 | assert.Equal(t, "service3", *services["service3"].ServiceName()) 41 | } 42 | 43 | func Test_ForProjectConfigNotReleaseIDServiceMap_Error(t *testing.T) { 44 | // func ForProjectConfigNotReleaseIDServiceMap(asgc aws.ASGAPI, project_name *string, config_name *string, release_uuid *string) (map[string]*ASG, error) { 45 | asgc := &mocks.ASGClient{} 46 | asgc.AddPreviousRuntimeResources("project", "config", "service1", "release") 47 | asgc.AddPreviousRuntimeResources("project", "config", "service1", "release") 48 | 49 | _, err := ForProjectConfigNotReleaseIDServiceMap(asgc, to.Strp("project"), to.Strp("config"), to.Strp("not_release")) 50 | assert.Error(t, err) 51 | 52 | } 53 | 54 | func Test_ForProjectConfigNOTReleaseID(t *testing.T) { 55 | // func ForProjectConfigNOTReleaseID(asgc aws.ASGAPI, project_name *string, config_name *string, release_uuid *string) ([]*ASG, error) { 56 | asgc := &mocks.ASGClient{} 57 | asgs, err := ForProjectConfigNOTReleaseID(asgc, to.Strp("project"), to.Strp("config"), to.Strp("release")) 58 | assert.NoError(t, err) 59 | assert.Equal(t, 0, len(asgs)) 60 | 61 | asgc.AddPreviousRuntimeResources("project", "config", "service1", "release") 62 | asgc.AddPreviousRuntimeResources("project", "config", "service2", "release") 63 | asgc.AddPreviousRuntimeResources("project", "config", "service3", "not_release") 64 | asgc.AddPreviousRuntimeResources("not_project", "config", "service4", "release") 65 | asgc.AddPreviousRuntimeResources("project", "not_config", "service5", "release") 66 | 67 | asgs, err = ForProjectConfigNOTReleaseID(asgc, to.Strp("project"), to.Strp("config"), to.Strp("release")) 68 | assert.NoError(t, err) 69 | assert.Equal(t, 1, len(asgs)) 70 | } 71 | 72 | func Test_ForProjectConfigReleaseID(t *testing.T) { 73 | // func ForProjectConfigReleaseID(asgc aws.ASGAPI, project_name *string, config_name *string, release_uuid *string) ([]*ASG, error) { 74 | asgc := &mocks.ASGClient{} 75 | asgs, err := ForProjectConfigReleaseID(asgc, to.Strp("project"), to.Strp("config"), to.Strp("release")) 76 | assert.NoError(t, err) 77 | assert.Equal(t, 0, len(asgs)) 78 | 79 | asgc.AddPreviousRuntimeResources("project", "config", "service1", "release") 80 | asgc.AddPreviousRuntimeResources("project", "config", "service2", "release") 81 | asgc.AddPreviousRuntimeResources("project", "config", "service3", "not_release") 82 | asgc.AddPreviousRuntimeResources("not_project", "config", "service4", "release") 83 | asgc.AddPreviousRuntimeResources("project", "not_config", "service5", "release") 84 | 85 | asgs, err = ForProjectConfigReleaseID(asgc, to.Strp("project"), to.Strp("config"), to.Strp("release")) 86 | assert.NoError(t, err) 87 | assert.Equal(t, 2, len(asgs)) 88 | } 89 | 90 | func Test_Teardown(t *testing.T) { 91 | // func (s *ASG) Teardown(asgc aws.ASGAPI, cwc aws.CWAPI) error { 92 | asgc := &mocks.ASGClient{} 93 | cwc := &mocks.CWClient{} 94 | 95 | asgc.AddPreviousRuntimeResources("project", "config", "service1", "not_release") 96 | asgs, err := ForProjectConfigNOTReleaseID(asgc, to.Strp("project"), to.Strp("config"), to.Strp("release")) 97 | assert.NoError(t, err) 98 | assert.Equal(t, 1, len(asgs)) 99 | 100 | err = asgs[0].Teardown(asgc, cwc) 101 | assert.NoError(t, err) 102 | } 103 | 104 | func Test_AttachedLBs(t *testing.T) { 105 | asgc := &mocks.ASGClient{} 106 | 107 | asgc.DescribeLoadBalancerTargetGroupsOutput = &autoscaling.DescribeLoadBalancerTargetGroupsOutput{ 108 | LoadBalancerTargetGroups: []*autoscaling.LoadBalancerTargetGroupState{ 109 | &autoscaling.LoadBalancerTargetGroupState{ 110 | LoadBalancerTargetGroupARN: to.Strp("arn"), 111 | State: to.Strp("aaa"), 112 | }, 113 | }, 114 | } 115 | 116 | asgc.DescribeLoadBalancersOutput = &autoscaling.DescribeLoadBalancersOutput{ 117 | LoadBalancers: []*autoscaling.LoadBalancerState{ 118 | &autoscaling.LoadBalancerState{ 119 | LoadBalancerName: to.Strp("arn"), 120 | State: to.Strp("aaa"), 121 | }, 122 | }, 123 | } 124 | 125 | asgc.AddPreviousRuntimeResources("project", "config", "service1", "not_release") 126 | asgs, err := ForProjectConfigNOTReleaseID(asgc, to.Strp("project"), to.Strp("config"), to.Strp("release")) 127 | 128 | attached, err := asgs[0].AttachedLBs(asgc) 129 | assert.NoError(t, err) 130 | assert.Equal(t, 2, len(attached)) 131 | 132 | asgc.DescribeLoadBalancersOutput = &autoscaling.DescribeLoadBalancersOutput{ 133 | LoadBalancers: []*autoscaling.LoadBalancerState{ 134 | &autoscaling.LoadBalancerState{ 135 | LoadBalancerName: to.Strp("arn"), 136 | State: to.Strp("Removed"), 137 | }, 138 | }, 139 | } 140 | 141 | asgc.DescribeLoadBalancerTargetGroupsOutput = &autoscaling.DescribeLoadBalancerTargetGroupsOutput{ 142 | LoadBalancerTargetGroups: []*autoscaling.LoadBalancerTargetGroupState{ 143 | &autoscaling.LoadBalancerTargetGroupState{ 144 | LoadBalancerTargetGroupARN: to.Strp("arn"), 145 | State: to.Strp("Removed"), 146 | }, 147 | }, 148 | } 149 | 150 | attached, err = asgs[0].AttachedLBs(asgc) 151 | assert.NoError(t, err) 152 | assert.Equal(t, 0, len(attached)) 153 | } 154 | -------------------------------------------------------------------------------- /aws/aws.go: -------------------------------------------------------------------------------- 1 | package aws 2 | 3 | import ( 4 | "fmt" 5 | "github.com/aws/aws-sdk-go/service/autoscaling" 6 | "github.com/aws/aws-sdk-go/service/autoscaling/autoscalingiface" 7 | "github.com/aws/aws-sdk-go/service/cloudwatch" 8 | "github.com/aws/aws-sdk-go/service/cloudwatch/cloudwatchiface" 9 | "github.com/aws/aws-sdk-go/service/dynamodb" 10 | "github.com/aws/aws-sdk-go/service/dynamodb/dynamodbiface" 11 | "github.com/aws/aws-sdk-go/service/ec2" 12 | "github.com/aws/aws-sdk-go/service/ec2/ec2iface" 13 | "github.com/aws/aws-sdk-go/service/elb" 14 | "github.com/aws/aws-sdk-go/service/elb/elbiface" 15 | "github.com/aws/aws-sdk-go/service/elbv2" 16 | "github.com/aws/aws-sdk-go/service/elbv2/elbv2iface" 17 | "github.com/aws/aws-sdk-go/service/iam" 18 | "github.com/aws/aws-sdk-go/service/iam/iamiface" 19 | "github.com/aws/aws-sdk-go/service/s3" 20 | "github.com/aws/aws-sdk-go/service/s3/s3iface" 21 | "github.com/aws/aws-sdk-go/service/sfn" 22 | "github.com/aws/aws-sdk-go/service/sfn/sfniface" 23 | "github.com/aws/aws-sdk-go/service/sns" 24 | "github.com/aws/aws-sdk-go/service/sns/snsiface" 25 | ar "github.com/coinbase/step/aws" 26 | ) 27 | 28 | // FetchEc2Tag extracts tags 29 | func FetchEc2Tag(tags []*ec2.Tag, tagKey *string) *string { 30 | if tagKey == nil { 31 | return nil 32 | } 33 | 34 | for _, tag := range tags { 35 | if tag.Key == nil { 36 | continue 37 | } 38 | if *tag.Key == *tagKey { 39 | return tag.Value 40 | } 41 | } 42 | 43 | return nil 44 | } 45 | 46 | // FetchELBTag extracts tags 47 | func FetchELBTag(tags []*elb.Tag, tagKey *string) *string { 48 | if tagKey == nil { 49 | return nil 50 | } 51 | 52 | for _, tag := range tags { 53 | if tag.Key == nil { 54 | continue 55 | } 56 | if *tag.Key == *tagKey { 57 | return tag.Value 58 | } 59 | } 60 | 61 | return nil 62 | } 63 | 64 | // FetchELBV2Tag extracts tags 65 | func FetchELBV2Tag(tags []*elbv2.Tag, tagKey *string) *string { 66 | if tagKey == nil { 67 | return nil 68 | } 69 | 70 | for _, tag := range tags { 71 | if tag.Key == nil { 72 | continue 73 | } 74 | if *tag.Key == *tagKey { 75 | return tag.Value 76 | } 77 | } 78 | 79 | return nil 80 | } 81 | 82 | // FetchASGTag extracts tags 83 | func FetchASGTag(tags []*autoscaling.TagDescription, tagKey *string) *string { 84 | if tagKey == nil { 85 | return nil 86 | } 87 | 88 | for _, tag := range tags { 89 | if tag.Key == nil { 90 | continue 91 | } 92 | if *tag.Key == *tagKey { 93 | return tag.Value 94 | } 95 | } 96 | 97 | return nil 98 | } 99 | 100 | // HasAllValue checks for the _all value tag 101 | func HasAllValue(tag *string) bool { 102 | if tag == nil { 103 | return false 104 | } 105 | return "_all" == *tag 106 | } 107 | 108 | // HasProjectName checks value 109 | func HasProjectName(r interface { 110 | ProjectName() *string 111 | }, projectName *string) bool { 112 | if r.ProjectName() == nil || projectName == nil { 113 | return false 114 | } 115 | return *r.ProjectName() == *projectName 116 | } 117 | 118 | // HasConfigName checks value 119 | func HasConfigName(r interface { 120 | ConfigName() *string 121 | }, configName *string) bool { 122 | if r.ConfigName() == nil || configName == nil { 123 | return false 124 | } 125 | return *r.ConfigName() == *configName 126 | } 127 | 128 | // HasServiceName checks value 129 | func HasServiceName(r interface { 130 | ServiceName() *string 131 | }, serviceName *string) bool { 132 | if r.ServiceName() == nil || serviceName == nil { 133 | return false 134 | } 135 | return *r.ServiceName() == *serviceName 136 | } 137 | 138 | func MatchesAllowedService(r interface { 139 | AllowedService() *string 140 | }, projectName *string, configName *string, serviceName *string) bool { 141 | if projectName == nil || configName == nil || serviceName == nil || r.AllowedService() == nil { 142 | return false 143 | } 144 | return *r.AllowedService() == fmt.Sprintf("%s::%s::%s", *projectName, *configName, *serviceName) 145 | } 146 | 147 | // HasReleaseID checks value 148 | func HasReleaseID(r interface { 149 | ReleaseID() *string 150 | }, releaseID *string) bool { 151 | if r.ReleaseID() == nil || releaseID == nil { 152 | return false 153 | } 154 | return *r.ReleaseID() == *releaseID 155 | } 156 | 157 | // S3API aws API 158 | type S3API s3iface.S3API 159 | 160 | // ASGAPI aws API 161 | type ASGAPI autoscalingiface.AutoScalingAPI 162 | 163 | // ELBAPI aws API 164 | type ELBAPI elbiface.ELBAPI 165 | 166 | // EC2API aws API 167 | type EC2API ec2iface.EC2API 168 | 169 | // ALBAPI aws API 170 | type ALBAPI elbv2iface.ELBV2API 171 | 172 | // CWAPI aws API 173 | type CWAPI cloudwatchiface.CloudWatchAPI 174 | 175 | // IAMAPI aws API 176 | type IAMAPI iamiface.IAMAPI 177 | 178 | // SNSAPI aws API 179 | type SNSAPI snsiface.SNSAPI 180 | 181 | // SFNAPI aws API 182 | type SFNAPI sfniface.SFNAPI 183 | 184 | // DynamoDBAPI aws API 185 | type DynamoDBAPI dynamodbiface.DynamoDBAPI 186 | 187 | // Clients for AWS 188 | type Clients interface { 189 | S3Client(region *string, accountID *string, role *string) S3API 190 | ASGClient(region *string, accountID *string, role *string) ASGAPI 191 | ELBClient(region *string, accountID *string, role *string) ELBAPI 192 | EC2Client(region *string, accountID *string, role *string) EC2API 193 | ALBClient(region *string, accountID *string, role *string) ALBAPI 194 | CWClient(region *string, accountID *string, role *string) CWAPI 195 | IAMClient(region *string, accountID *string, role *string) IAMAPI 196 | SNSClient(region *string, accountID *string, role *string) SNSAPI 197 | SFNClient(region *string, accountID *string, role *string) SFNAPI 198 | DynamoDBClient(region *string, accountID *string, role *string) DynamoDBAPI 199 | } 200 | 201 | // ClientsStr implementation 202 | type ClientsStr struct { 203 | ar.Clients 204 | } 205 | 206 | // S3Client returns client for region account and role 207 | func (awsc *ClientsStr) S3Client(region *string, accountID *string, role *string) S3API { 208 | return s3.New(awsc.Session(), awsc.Config(region, accountID, role)) 209 | } 210 | 211 | // ASGClient returns client for region account and role 212 | func (awsc *ClientsStr) ASGClient(region *string, accountID *string, role *string) ASGAPI { 213 | return autoscaling.New(awsc.Session(), awsc.Config(region, accountID, role)) 214 | } 215 | 216 | // ELBClient returns client for region account and role 217 | func (awsc *ClientsStr) ELBClient(region *string, accountID *string, role *string) ELBAPI { 218 | return elb.New(awsc.Session(), awsc.Config(region, accountID, role)) 219 | } 220 | 221 | // EC2Client returns client for region account and role 222 | func (awsc *ClientsStr) EC2Client(region *string, accountID *string, role *string) EC2API { 223 | return ec2.New(awsc.Session(), awsc.Config(region, accountID, role)) 224 | } 225 | 226 | // ALBClient returns client for region account and role 227 | func (awsc *ClientsStr) ALBClient(region *string, accountID *string, role *string) ALBAPI { 228 | return elbv2.New(awsc.Session(), awsc.Config(region, accountID, role)) 229 | } 230 | 231 | // CWClient returns client for region account and role 232 | func (awsc *ClientsStr) CWClient(region *string, accountID *string, role *string) CWAPI { 233 | return cloudwatch.New(awsc.Session(), awsc.Config(region, accountID, role)) 234 | } 235 | 236 | // IAMClient returns client for region account and role 237 | func (awsc *ClientsStr) IAMClient(region *string, accountID *string, role *string) IAMAPI { 238 | return iam.New(awsc.Session(), awsc.Config(region, accountID, role)) 239 | } 240 | 241 | // SNSClient returns client for region account and role 242 | func (awsc *ClientsStr) SNSClient(region *string, accountID *string, role *string) SNSAPI { 243 | return sns.New(awsc.Session(), awsc.Config(region, accountID, role)) 244 | } 245 | 246 | // SFNClient returns client for region account and role 247 | func (awsc *ClientsStr) SFNClient(region *string, accountID *string, role *string) SFNAPI { 248 | return sfn.New(awsc.Session(), awsc.Config(region, accountID, role)) 249 | } 250 | 251 | // DynamoDBClient returns client for region account and role 252 | func (awsc *ClientsStr) DynamoDBClient(region *string, account_id *string, role *string) DynamoDBAPI { 253 | return dynamodb.New(awsc.Session(), awsc.Config(region, account_id, role)) 254 | } 255 | -------------------------------------------------------------------------------- /aws/elb/elb.go: -------------------------------------------------------------------------------- 1 | package elb 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/aws/aws-sdk-go/aws/awserr" 7 | aws_elb "github.com/aws/aws-sdk-go/service/elb" 8 | "github.com/coinbase/odin/aws" 9 | "github.com/coinbase/step/utils/to" 10 | ) 11 | 12 | // LoadBalancer struct 13 | type LoadBalancer struct { 14 | ProjectNameTag *string 15 | ConfigNameTag *string 16 | ServiceNameTag *string 17 | LoadBalancerName *string 18 | } 19 | 20 | // ProjectName returns tag 21 | func (s *LoadBalancer) ProjectName() *string { 22 | return s.ProjectNameTag 23 | } 24 | 25 | // ConfigName returns tag 26 | func (s *LoadBalancer) ConfigName() *string { 27 | return s.ConfigNameTag 28 | } 29 | 30 | // ServiceName returns tag 31 | func (s *LoadBalancer) ServiceName() *string { 32 | return s.ServiceNameTag 33 | } 34 | 35 | // Name returns tag 36 | func (s *LoadBalancer) Name() *string { 37 | return s.LoadBalancerName 38 | } 39 | 40 | // AllowedService returns tag 41 | func (s *LoadBalancer) AllowedService() *string { 42 | return to.Strp(fmt.Sprintf("%s::%s::%s", *s.ProjectName(), *s.ConfigName(), *s.ServiceName())) 43 | } 44 | 45 | /////// 46 | // Healthy 47 | /////// 48 | 49 | // GetInstances returns a list of specific instances on the ELB 50 | func GetInstances(elbc aws.ELBAPI, name *string, instances []string) (aws.Instances, error) { 51 | instanceStates, err := instanceStates(elbc, name, instances) 52 | 53 | if err != nil { 54 | return nil, err 55 | } 56 | 57 | elbInstances := aws.Instances{} 58 | for _, is := range instanceStates { 59 | elbInstances.AddELBInstance(is) 60 | } 61 | 62 | return elbInstances, nil 63 | } 64 | 65 | func createDescribeInstanceHealthInput(name *string, instances []string) *aws_elb.DescribeInstanceHealthInput { 66 | awsInstances := []*aws_elb.Instance{} 67 | for _, id := range instances { 68 | awsInstances = append(awsInstances, &aws_elb.Instance{InstanceId: to.Strp(id)}) 69 | } 70 | 71 | return &aws_elb.DescribeInstanceHealthInput{ 72 | LoadBalancerName: name, 73 | Instances: awsInstances, 74 | } 75 | } 76 | 77 | func instanceStates(elbc aws.ELBAPI, name *string, instances []string) ([]*aws_elb.InstanceState, error) { 78 | 79 | healthOutput, err := elbc.DescribeInstanceHealth(createDescribeInstanceHealthInput(name, instances)) 80 | 81 | if err != nil { 82 | if aerr, ok := err.(awserr.Error); ok { 83 | switch aerr.Code() { 84 | case aws_elb.ErrCodeInvalidEndPointException: 85 | // Error occurs if instance is not yet (or no longer) attached to the ELB 86 | return []*aws_elb.InstanceState{}, nil 87 | } 88 | } 89 | return nil, err 90 | } 91 | 92 | return healthOutput.InstanceStates, nil 93 | } 94 | 95 | /////// 96 | // Find 97 | /////// 98 | 99 | // FindAll returns ELBs with names 100 | func FindAll(elbc aws.ELBAPI, names []*string) ([]*LoadBalancer, error) { 101 | elbs := []*LoadBalancer{} 102 | for _, name := range names { 103 | elb, err := find(elbc, name) 104 | if err != nil { 105 | return nil, err 106 | } 107 | elbs = append(elbs, elb) 108 | } 109 | 110 | return elbs, nil 111 | } 112 | 113 | func find(elbc aws.ELBAPI, name *string) (*LoadBalancer, error) { 114 | 115 | elbDesc, err := findAwsByName(elbc, name) 116 | 117 | if err != nil { 118 | return nil, err 119 | } 120 | 121 | tags, err := findAwsTagsByName(elbc, name) 122 | 123 | if err != nil { 124 | return nil, err 125 | } 126 | 127 | return &LoadBalancer{ 128 | ProjectNameTag: aws.FetchELBTag(tags, to.Strp("ProjectName")), 129 | ConfigNameTag: aws.FetchELBTag(tags, to.Strp("ConfigName")), 130 | ServiceNameTag: aws.FetchELBTag(tags, to.Strp("ServiceName")), 131 | LoadBalancerName: elbDesc.LoadBalancerName, 132 | }, nil 133 | } 134 | 135 | func findAwsByName(elbc aws.ELBAPI, name *string) (*aws_elb.LoadBalancerDescription, error) { 136 | elbsOutput, err := elbc.DescribeLoadBalancers(&aws_elb.DescribeLoadBalancersInput{ 137 | LoadBalancerNames: []*string{name}, 138 | }) 139 | 140 | if err != nil { 141 | return nil, err 142 | } 143 | 144 | if len(elbsOutput.LoadBalancerDescriptions) != 1 { 145 | return nil, fmt.Errorf("LoadBalancer Not Found") 146 | } 147 | 148 | if *elbsOutput.LoadBalancerDescriptions[0].LoadBalancerName != *name { 149 | return nil, fmt.Errorf("LoadBalancer Not Found") 150 | } 151 | 152 | return elbsOutput.LoadBalancerDescriptions[0], nil 153 | } 154 | 155 | func findAwsTagsByName(elbc aws.ELBAPI, name *string) ([]*aws_elb.Tag, error) { 156 | tagsOutput, err := elbc.DescribeTags(&aws_elb.DescribeTagsInput{ 157 | LoadBalancerNames: []*string{name}, 158 | }) 159 | 160 | if err != nil { 161 | return nil, err 162 | } 163 | 164 | if len(tagsOutput.TagDescriptions) != 1 { 165 | return nil, fmt.Errorf("LoadBalancer Not Found") 166 | } 167 | 168 | if *tagsOutput.TagDescriptions[0].LoadBalancerName != *name { 169 | return nil, fmt.Errorf("LoadBalancer Not Found") 170 | } 171 | 172 | return tagsOutput.TagDescriptions[0].Tags, nil 173 | 174 | } 175 | -------------------------------------------------------------------------------- /aws/elb/elb_test.go: -------------------------------------------------------------------------------- 1 | package elb 2 | 3 | import ( 4 | "sort" 5 | "testing" 6 | 7 | "github.com/coinbase/odin/aws/mocks" 8 | "github.com/coinbase/step/utils/to" 9 | "github.com/stretchr/testify/assert" 10 | ) 11 | 12 | func Test_GetInstances(t *testing.T) { 13 | //func GetInstances(elbc aws.ELBAPI, name *string, instances []string) (aws.Instances, error) { 14 | elbc := &mocks.ELBClient{} 15 | _, err := GetInstances(elbc, to.Strp("asd"), []string{"asd"}) 16 | assert.Error(t, err) 17 | 18 | elbc.AddELB("asd", "project", "config", "service") 19 | ins, err := GetInstances(elbc, to.Strp("asd"), []string{"asd"}) 20 | assert.NoError(t, err) 21 | assert.Equal(t, 1, len(ins)) 22 | } 23 | 24 | func Test_FindAll(t *testing.T) { 25 | //func FindAll(elbc aws.ELBAPI, names []*string) ([]*LoadBalancer, error) { 26 | elbc := &mocks.ELBClient{} 27 | _, err := FindAll(elbc, []*string{to.Strp("asd")}) 28 | assert.Error(t, err) 29 | 30 | elbc.AddELB("asd", "project", "config", "service") 31 | elbc.AddELB("das", "project", "config", "service") 32 | 33 | elbs, err := FindAll(elbc, []*string{to.Strp("asd"), to.Strp("das")}) 34 | 35 | assert.NoError(t, err) 36 | assert.Equal(t, 2, len(elbs)) 37 | } 38 | 39 | func Test_createDescribeInstanceHealthInput(t *testing.T) { 40 | name := "" 41 | 42 | in := createDescribeInstanceHealthInput(&name, []string{}) 43 | assert.Equal(t, len(in.Instances), 0) 44 | 45 | in = createDescribeInstanceHealthInput(&name, []string{"a"}) 46 | assert.Equal(t, len(in.Instances), 1) 47 | assert.Equal(t, *in.Instances[0].InstanceId, "a") 48 | 49 | in = createDescribeInstanceHealthInput(&name, []string{"a", "b"}) 50 | 51 | elbsIDs := []string{} 52 | for _, lb := range in.Instances { 53 | elbsIDs = append(elbsIDs, *lb.InstanceId) 54 | } 55 | 56 | sort.Strings(elbsIDs) // Sort not Guaranteed by map 57 | 58 | assert.Equal(t, len(elbsIDs), 2) 59 | assert.Equal(t, elbsIDs[0], "a") 60 | assert.Equal(t, elbsIDs[1], "b") 61 | } 62 | -------------------------------------------------------------------------------- /aws/iam/iam.go: -------------------------------------------------------------------------------- 1 | package iam 2 | 3 | import ( 4 | "github.com/aws/aws-sdk-go/service/iam" 5 | "github.com/coinbase/odin/aws" 6 | ) 7 | 8 | ////// 9 | // PROFILE 10 | ////// 11 | 12 | // Profile struct 13 | type Profile struct { 14 | Path *string 15 | Arn *string 16 | } 17 | 18 | // Find returns profile with name 19 | func Find(iamClient aws.IAMAPI, profileName *string) (*Profile, error) { 20 | profileOutput, err := iamClient.GetInstanceProfile(&iam.GetInstanceProfileInput{ 21 | InstanceProfileName: profileName, 22 | }) 23 | 24 | if err != nil { 25 | return nil, err 26 | } 27 | 28 | awsProfile := profileOutput.InstanceProfile 29 | return &Profile{ 30 | Path: awsProfile.Path, 31 | Arn: awsProfile.Arn, 32 | }, nil 33 | } 34 | 35 | ////// 36 | // ROLE 37 | ////// 38 | 39 | // RoleExists returns whether profile exists 40 | func RoleExists(iamc aws.IAMAPI, roleName *string) error { 41 | _, err := iamc.GetRole(&iam.GetRoleInput{ 42 | RoleName: roleName, 43 | }) 44 | 45 | return err 46 | } 47 | -------------------------------------------------------------------------------- /aws/iam/iam_test.go: -------------------------------------------------------------------------------- 1 | package iam 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/odin/aws/mocks" 7 | "github.com/coinbase/step/utils/to" 8 | "github.com/stretchr/testify/assert" 9 | ) 10 | 11 | func Test_Find(t *testing.T) { 12 | //func FindAll(elbc aws.ELBAPI, names []*string) ([]*LoadBalancer, error) { 13 | iamc := &mocks.IAMClient{} 14 | _, err := Find(iamc, to.Strp("asd")) 15 | assert.Error(t, err) 16 | 17 | iamc.AddGetInstanceProfile("asd", "/path/") 18 | profile, err := Find(iamc, to.Strp("asd")) 19 | assert.NoError(t, err) 20 | assert.Equal(t, "/path/", *profile.Path) 21 | } 22 | -------------------------------------------------------------------------------- /aws/instances.go: -------------------------------------------------------------------------------- 1 | package aws 2 | 3 | import ( 4 | "github.com/aws/aws-sdk-go/service/autoscaling" 5 | "github.com/aws/aws-sdk-go/service/elb" 6 | "github.com/aws/aws-sdk-go/service/elbv2" 7 | ) 8 | 9 | const terminating = "terminating" 10 | const unhealthy = "unhealthy" 11 | const healthy = "healthy" 12 | 13 | // Instances Map of instance id to state 14 | type Instances map[string]string 15 | 16 | // AddTargetGroupInstance add a target group instances 17 | func (all Instances) AddTargetGroupInstance(thd *elbv2.TargetHealthDescription) { 18 | state := unhealthy 19 | if *thd.TargetHealth.State == "healthy" { 20 | state = healthy 21 | } 22 | all[*thd.Target.Id] = state 23 | } 24 | 25 | // AddASGInstance add a ASG instances 26 | func (all Instances) AddASGInstance(i *autoscaling.Instance) { 27 | if i == nil || i.LifecycleState == nil { 28 | return 29 | } 30 | 31 | state := unhealthy 32 | 33 | if i.HealthStatus != nil && *i.HealthStatus == "Healthy" && *i.LifecycleState == "InService" { 34 | state = healthy 35 | } 36 | 37 | if (*i.LifecycleState)[0:4] == "Term" { 38 | state = terminating 39 | } 40 | 41 | all[*i.InstanceId] = state 42 | } 43 | 44 | // AddELBInstance add a ELB instances 45 | func (all Instances) AddELBInstance(is *elb.InstanceState) { 46 | state := unhealthy 47 | if *is.State == "InService" { 48 | state = healthy 49 | } 50 | all[*is.InstanceId] = state 51 | } 52 | 53 | // HealthyUnhealthyTerming returns the numbers of states 54 | func (all Instances) HealthyUnhealthyTerming() (int, int, int) { 55 | healthyc := 0 56 | unhealthyc := 0 57 | termingc := 0 58 | 59 | for _, state := range all { 60 | switch state { 61 | case healthy: 62 | healthyc++ 63 | case unhealthy: 64 | unhealthyc++ 65 | case terminating: 66 | termingc++ 67 | } 68 | } 69 | 70 | return healthyc, unhealthyc, termingc 71 | } 72 | 73 | // InstanceIDs list of instance IDs 74 | func (all Instances) InstanceIDs() []string { 75 | ids := []string{} 76 | 77 | for id := range all { 78 | ids = append(ids, id) 79 | } 80 | 81 | return ids 82 | } 83 | 84 | // HealthyIDs list of instances terminating 85 | func (all Instances) UnhealthyIDs() []string { 86 | ids := []string{} 87 | for id, state := range all { 88 | if state == unhealthy { 89 | ids = append(ids, id) 90 | } 91 | } 92 | return ids 93 | } 94 | 95 | // HealthyIDs list of instances terminating 96 | func (all Instances) HealthyIDs() []string { 97 | ids := []string{} 98 | for id, state := range all { 99 | if state == healthy { 100 | ids = append(ids, id) 101 | } 102 | } 103 | return ids 104 | } 105 | 106 | // TerminatingIDs list of instances terminating 107 | func (all Instances) TerminatingIDs() []string { 108 | ids := []string{} 109 | for id, state := range all { 110 | if state == terminating { 111 | ids = append(ids, id) 112 | } 113 | } 114 | return ids 115 | } 116 | 117 | // MergeInstances merge new set of instances returns new set 118 | func (all Instances) MergeInstances(update Instances) Instances { 119 | ret := Instances{} 120 | for id, state := range all { 121 | ret[id] = stateCompare(state, update[id]) 122 | } 123 | 124 | return ret 125 | } 126 | 127 | func stateCompare(s1 string, s2 string) string { 128 | // terminating > unhealthy > healthy 129 | if s1 == healthy && s2 == healthy { 130 | // Both Healthy Return Healthy 131 | return healthy 132 | } 133 | 134 | if s1 == terminating || s2 == terminating { 135 | // Either Terming Return term 136 | return terminating 137 | } 138 | // Otherwise Unhealthy 139 | return unhealthy 140 | } 141 | -------------------------------------------------------------------------------- /aws/instances_test.go: -------------------------------------------------------------------------------- 1 | package aws 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/stretchr/testify/assert" 7 | ) 8 | 9 | func Test_MergeInstances(t *testing.T) { 10 | i1 := Instances{"i": healthy} 11 | i2 := Instances{"i": healthy} 12 | assert.Equal(t, healthy, i1.MergeInstances(i2)["i"]) 13 | 14 | i2 = Instances{"i": unhealthy} 15 | assert.Equal(t, unhealthy, i1.MergeInstances(i2)["i"]) 16 | 17 | i2 = Instances{"i": terminating} 18 | assert.Equal(t, terminating, i1.MergeInstances(i2)["i"]) 19 | 20 | // inverse 21 | i2 = Instances{"i": healthy} 22 | assert.Equal(t, healthy, i2.MergeInstances(i1)["i"]) 23 | 24 | i2 = Instances{"i": unhealthy} 25 | assert.Equal(t, unhealthy, i2.MergeInstances(i1)["i"]) 26 | 27 | i2 = Instances{"i": terminating} 28 | assert.Equal(t, terminating, i2.MergeInstances(i1)["i"]) 29 | } 30 | -------------------------------------------------------------------------------- /aws/lc/launch_configuration.go: -------------------------------------------------------------------------------- 1 | package lc 2 | 3 | import ( 4 | "github.com/aws/aws-sdk-go/service/autoscaling" 5 | 6 | "github.com/coinbase/odin/aws" 7 | ) 8 | 9 | // Teardown deleted launch configuration 10 | func Teardown(asgc aws.ASGAPI, name *string) error { 11 | _, err := asgc.DeleteLaunchConfiguration(&autoscaling.DeleteLaunchConfigurationInput{ 12 | LaunchConfigurationName: name, 13 | }) 14 | 15 | if err != nil { 16 | return err 17 | } 18 | 19 | return nil 20 | } 21 | -------------------------------------------------------------------------------- /aws/lc/launch_configuration_input.go: -------------------------------------------------------------------------------- 1 | package lc 2 | 3 | import ( 4 | "github.com/aws/aws-sdk-go/service/autoscaling" 5 | "github.com/coinbase/odin/aws" 6 | "github.com/coinbase/step/utils/to" 7 | ) 8 | 9 | var ebsOptimizedInstances = map[string]bool{ 10 | "c4.large": true, 11 | "c4.xlarge": true, 12 | "c4.2xlarge": true, 13 | "c4.4xlarge": true, 14 | "c4.8xlarge": true, 15 | "c5.large": true, 16 | "c5.xlarge": true, 17 | "c5.2xlarge": true, 18 | "c5.4xlarge": true, 19 | "c5.9xlarge": true, 20 | "c5.18xlarge": true, 21 | "i3.large": true, 22 | "i3.xlarge": true, 23 | "i3.2xlarge": true, 24 | "i3.4xlarge": true, 25 | "i3.8xlarge": true, 26 | "i3.16xlarge": true, 27 | "m4.large": true, 28 | "m4.xlarge": true, 29 | "m4.2xlarge": true, 30 | "m4.4xlarge": true, 31 | "m4.10xlarge": true, 32 | "m4.16xlarge": true, 33 | "m5.large": true, 34 | "m5.xlarge": true, 35 | "m5.2xlarge": true, 36 | "m5.4xlarge": true, 37 | "m5.12xlarge": true, 38 | "m5.24xlarge": true, 39 | "r4.large": true, 40 | "r4.xlarge": true, 41 | "r4.2xlarge": true, 42 | "r4.4xlarge": true, 43 | "r4.8xlarge": true, 44 | "r4.16xlarge": true, 45 | } 46 | 47 | // LaunchConfigInput input struct 48 | type LaunchConfigInput struct { 49 | *autoscaling.CreateLaunchConfigurationInput 50 | } 51 | 52 | // Create tryes to create the launch configuration 53 | func (s *LaunchConfigInput) Create(asgc aws.ASGAPI) error { 54 | if err := s.Validate(); err != nil { 55 | return err 56 | } 57 | 58 | _, err := asgc.CreateLaunchConfiguration(s.CreateLaunchConfigurationInput) 59 | 60 | if err != nil { 61 | return err 62 | } 63 | 64 | return nil 65 | } 66 | 67 | // AddBlockDevice adds an EBS block device to the LC 68 | func (s *LaunchConfigInput) AddBlockDevice(ebsVolumeSize *int64, ebsVolumeType *string, ebsDeviceType *string) { 69 | if ebsVolumeSize == nil { 70 | return 71 | } 72 | 73 | if ebsVolumeType == nil { 74 | ebsVolumeType = to.Strp("gp2") 75 | } 76 | 77 | if ebsDeviceType == nil { 78 | ebsDeviceType = to.Strp("/dev/xvda") 79 | } 80 | 81 | block := &autoscaling.BlockDeviceMapping{ 82 | DeviceName: ebsDeviceType, 83 | Ebs: &autoscaling.Ebs{ 84 | VolumeSize: ebsVolumeSize, 85 | VolumeType: ebsVolumeType, 86 | }, 87 | } 88 | 89 | if s.BlockDeviceMappings == nil { 90 | s.BlockDeviceMappings = []*autoscaling.BlockDeviceMapping{} 91 | } 92 | 93 | s.BlockDeviceMappings = append(s.BlockDeviceMappings, block) 94 | } 95 | 96 | // SetDefaults assigns values 97 | func (s *LaunchConfigInput) SetDefaults() { 98 | if s.InstanceType == nil { 99 | s.InstanceType = to.Strp("t2.nano") 100 | } 101 | 102 | if s.InstanceMonitoring == nil { 103 | s.InstanceMonitoring = &autoscaling.InstanceMonitoring{Enabled: to.Boolp(false)} 104 | } 105 | 106 | if s.EbsOptimized == nil { 107 | opt := ebsOptimizedInstances[*s.InstanceType] 108 | s.EbsOptimized = to.Boolp(opt) 109 | } 110 | } 111 | -------------------------------------------------------------------------------- /aws/lc/launch_configuration_input_test.go: -------------------------------------------------------------------------------- 1 | package lc 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/aws/aws-sdk-go/service/autoscaling" 7 | "github.com/coinbase/step/utils/to" 8 | ) 9 | 10 | func Test_AddBlockDevice(t *testing.T) { 11 | input := &LaunchConfigInput{&autoscaling.CreateLaunchConfigurationInput{}} 12 | 13 | input.AddBlockDevice(to.Int64p(10), nil, nil) 14 | input.AddBlockDevice(to.Int64p(10), to.Strp("asd"), nil) 15 | input.AddBlockDevice(to.Int64p(10), nil, to.Strp("asd")) 16 | 17 | } 18 | -------------------------------------------------------------------------------- /aws/mocks/clients.go: -------------------------------------------------------------------------------- 1 | package mocks 2 | 3 | import ( 4 | "github.com/coinbase/odin/aws" 5 | "github.com/coinbase/step/aws/mocks" 6 | ) 7 | 8 | // MockClients struct 9 | type MockClients struct { 10 | S3 *mocks.MockS3Client 11 | ASG *ASGClient 12 | ELB *ELBClient 13 | EC2 *EC2Client 14 | ALB *ALBClient 15 | CW *CWClient 16 | IAM *IAMClient 17 | SNS *SNSClient 18 | SFN *mocks.MockSFNClient 19 | DynamoDB *mocks.MockDynamoDBClient 20 | } 21 | 22 | // MockAWS mock clients 23 | func MockAWS() *MockClients { 24 | return &MockClients{ 25 | S3: &mocks.MockS3Client{}, 26 | ASG: &ASGClient{}, 27 | ELB: &ELBClient{}, 28 | EC2: &EC2Client{}, 29 | ALB: &ALBClient{}, 30 | CW: &CWClient{}, 31 | IAM: &IAMClient{}, 32 | SNS: &SNSClient{}, 33 | SFN: &mocks.MockSFNClient{}, 34 | DynamoDB: &mocks.MockDynamoDBClient{}, 35 | } 36 | } 37 | 38 | // S3Client returns 39 | func (a *MockClients) S3Client(*string, *string, *string) aws.S3API { 40 | return a.S3 41 | } 42 | 43 | // ASGClient returns 44 | func (a *MockClients) ASGClient(*string, *string, *string) aws.ASGAPI { 45 | return a.ASG 46 | } 47 | 48 | // ELBClient returns 49 | func (a *MockClients) ELBClient(*string, *string, *string) aws.ELBAPI { 50 | return a.ELB 51 | } 52 | 53 | // EC2Client returns 54 | func (a *MockClients) EC2Client(*string, *string, *string) aws.EC2API { 55 | return a.EC2 56 | } 57 | 58 | // ALBClient returns 59 | func (a *MockClients) ALBClient(*string, *string, *string) aws.ALBAPI { 60 | return a.ALB 61 | } 62 | 63 | // CWClient returns 64 | func (a *MockClients) CWClient(*string, *string, *string) aws.CWAPI { 65 | return a.CW 66 | } 67 | 68 | // IAMClient returns 69 | func (a *MockClients) IAMClient(*string, *string, *string) aws.IAMAPI { 70 | return a.IAM 71 | } 72 | 73 | // SNSClient returns 74 | func (a *MockClients) SNSClient(*string, *string, *string) aws.SNSAPI { 75 | return a.SNS 76 | } 77 | 78 | // SFNClient returns 79 | func (a *MockClients) SFNClient(*string, *string, *string) aws.SFNAPI { 80 | return a.SFN 81 | } 82 | 83 | // DynamoDBClient returns 84 | func (a *MockClients) DynamoDBClient(*string, *string, *string) aws.DynamoDBAPI { 85 | return a.DynamoDB 86 | } 87 | -------------------------------------------------------------------------------- /aws/mocks/mock_alb.go: -------------------------------------------------------------------------------- 1 | package mocks 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/aws/aws-sdk-go/aws/awserr" 7 | "github.com/aws/aws-sdk-go/service/elbv2" 8 | "github.com/coinbase/odin/aws" 9 | "github.com/coinbase/step/utils/to" 10 | ) 11 | 12 | // ALBClient return 13 | type ALBClient struct { 14 | aws.ALBAPI 15 | DescribeTargetGroupsResp map[string]*DescribeTargetGroupsResponse 16 | DescribeTagsResp map[string]*DescribeV2TagsResponse 17 | DescribeTargetHealthResp map[string]*DescribeTargetHealthResponse 18 | DescribeTargetGroupAttributesResp map[string]*DescribeTargetGroupAttributesResponse 19 | } 20 | 21 | // DescribeTargetGroupsResponse return 22 | type DescribeTargetGroupsResponse struct { 23 | Resp *elbv2.DescribeTargetGroupsOutput 24 | Error error 25 | } 26 | 27 | // DescribeV2TagsResponse return 28 | type DescribeV2TagsResponse struct { 29 | Resp *elbv2.DescribeTagsOutput 30 | Error error 31 | } 32 | 33 | // DescribeTargetHealthResponse return 34 | type DescribeTargetHealthResponse struct { 35 | Resp *elbv2.DescribeTargetHealthOutput 36 | Error error 37 | } 38 | 39 | // DescribeTargetGroupAttributesResponse return 40 | type DescribeTargetGroupAttributesResponse struct { 41 | Resp *elbv2.DescribeTargetGroupAttributesOutput 42 | Error error 43 | } 44 | 45 | // MockTargetGroup configuration struct, with defaults 46 | type MockTargetGroup struct { 47 | Name string 48 | ProjectName string 49 | ConfigName string 50 | ServiceName string 51 | AllowedService string 52 | } 53 | 54 | func (tg MockTargetGroup) allowedService() string { 55 | if tg.AllowedService == "" { 56 | return fmt.Sprintf("%s::%s::%s", tg.ProjectName, tg.ConfigName, tg.ServiceName) 57 | } 58 | return tg.AllowedService 59 | } 60 | 61 | func (tg *MockTargetGroup) init() { 62 | if tg.Name == "" { 63 | tg.Name = "tg_name" 64 | } 65 | if tg.ProjectName == "" { 66 | tg.ProjectName = "project_name" 67 | } 68 | if tg.ConfigName == "" { 69 | tg.ConfigName = "config_name" 70 | } 71 | if tg.ServiceName == "" { 72 | tg.ServiceName = "service_name" 73 | } 74 | } 75 | 76 | // AWSTargetGroupNotFoundError return 77 | func AWSTargetGroupNotFoundError() error { 78 | return awserr.New(elbv2.ErrCodeTargetGroupNotFoundException, "TargetGroupNotFound", nil) 79 | } 80 | 81 | func (m *ALBClient) init() { 82 | if m.DescribeTargetGroupsResp == nil { 83 | m.DescribeTargetGroupsResp = map[string]*DescribeTargetGroupsResponse{} 84 | } 85 | 86 | if m.DescribeTagsResp == nil { 87 | m.DescribeTagsResp = map[string]*DescribeV2TagsResponse{} 88 | } 89 | 90 | if m.DescribeTargetHealthResp == nil { 91 | m.DescribeTargetHealthResp = map[string]*DescribeTargetHealthResponse{} 92 | } 93 | 94 | if m.DescribeTargetGroupAttributesResp == nil { 95 | m.DescribeTargetGroupAttributesResp = map[string]*DescribeTargetGroupAttributesResponse{} 96 | } 97 | } 98 | 99 | // AddTargetGroup return 100 | func (m *ALBClient) AddTargetGroup(parameters MockTargetGroup) { 101 | m.init() 102 | parameters.init() 103 | 104 | name := parameters.Name 105 | m.DescribeTargetGroupsResp[name] = &DescribeTargetGroupsResponse{ 106 | Resp: &elbv2.DescribeTargetGroupsOutput{ 107 | TargetGroups: []*elbv2.TargetGroup{ 108 | &elbv2.TargetGroup{TargetGroupName: &name, TargetGroupArn: &name}, 109 | }, 110 | }, 111 | } 112 | 113 | m.DescribeTagsResp[name] = &DescribeV2TagsResponse{ 114 | Resp: &elbv2.DescribeTagsOutput{ 115 | TagDescriptions: []*elbv2.TagDescription{ 116 | &elbv2.TagDescription{ 117 | ResourceArn: &name, 118 | Tags: []*elbv2.Tag{ 119 | &elbv2.Tag{Key: to.Strp("ProjectName"), Value: to.Strp(parameters.ProjectName)}, 120 | &elbv2.Tag{Key: to.Strp("ConfigName"), Value: to.Strp(parameters.ConfigName)}, 121 | &elbv2.Tag{Key: to.Strp("ServiceName"), Value: to.Strp(parameters.ServiceName)}, 122 | &elbv2.Tag{Key: to.Strp("AllowedService"), Value: to.Strp(parameters.allowedService())}, 123 | }, 124 | }, 125 | }, 126 | }, 127 | } 128 | 129 | m.DescribeTargetHealthResp[name] = &DescribeTargetHealthResponse{ 130 | Resp: &elbv2.DescribeTargetHealthOutput{ 131 | TargetHealthDescriptions: []*elbv2.TargetHealthDescription{ 132 | &elbv2.TargetHealthDescription{ 133 | Target: &elbv2.TargetDescription{Id: to.Strp("InstanceId1")}, 134 | TargetHealth: &elbv2.TargetHealth{State: to.Strp("healthy")}, 135 | }, 136 | }, 137 | }, 138 | } 139 | 140 | m.DescribeTargetGroupAttributesResp[name] = &DescribeTargetGroupAttributesResponse{ 141 | Resp: &elbv2.DescribeTargetGroupAttributesOutput{ 142 | Attributes: []*elbv2.TargetGroupAttribute{ 143 | &elbv2.TargetGroupAttribute{ 144 | Key: to.Strp("slow_start.duration_seconds"), 145 | Value: to.Strp("42"), 146 | }, 147 | }, 148 | }, 149 | } 150 | } 151 | 152 | // DescribeTargetGroups return 153 | func (m *ALBClient) DescribeTargetGroups(in *elbv2.DescribeTargetGroupsInput) (*elbv2.DescribeTargetGroupsOutput, error) { 154 | m.init() 155 | lbName := in.Names[0] 156 | resp := m.DescribeTargetGroupsResp[*lbName] 157 | if resp == nil { 158 | return nil, AWSTargetGroupNotFoundError() 159 | } 160 | return resp.Resp, resp.Error 161 | } 162 | 163 | // DescribeTags return 164 | func (m *ALBClient) DescribeTags(in *elbv2.DescribeTagsInput) (*elbv2.DescribeTagsOutput, error) { 165 | m.init() 166 | lbName := in.ResourceArns[0] 167 | resp := m.DescribeTagsResp[*lbName] 168 | if resp == nil { 169 | return nil, AWSTargetGroupNotFoundError() 170 | } 171 | return resp.Resp, resp.Error 172 | } 173 | 174 | // DescribeTargetHealth return 175 | func (m *ALBClient) DescribeTargetHealth(in *elbv2.DescribeTargetHealthInput) (*elbv2.DescribeTargetHealthOutput, error) { 176 | m.init() 177 | lbName := in.TargetGroupArn 178 | resp := m.DescribeTargetHealthResp[*lbName] 179 | if resp == nil { 180 | return nil, AWSTargetGroupNotFoundError() 181 | } 182 | 183 | if resp.Resp == nil { 184 | return &elbv2.DescribeTargetHealthOutput{}, nil 185 | } 186 | 187 | return resp.Resp, resp.Error 188 | } 189 | 190 | // DescribeTargetGroupAttributes return 191 | func (m *ALBClient) DescribeTargetGroupAttributes(in *elbv2.DescribeTargetGroupAttributesInput) (*elbv2.DescribeTargetGroupAttributesOutput, error) { 192 | m.init() 193 | arn := in.TargetGroupArn 194 | resp := m.DescribeTargetGroupAttributesResp[*arn] 195 | if resp == nil { 196 | return nil, AWSTargetGroupNotFoundError() 197 | } 198 | return resp.Resp, resp.Error 199 | } 200 | -------------------------------------------------------------------------------- /aws/mocks/mock_asg.go: -------------------------------------------------------------------------------- 1 | package mocks 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/aws/aws-sdk-go/service/autoscaling" 7 | "github.com/coinbase/odin/aws" 8 | "github.com/coinbase/step/utils/to" 9 | ) 10 | 11 | // DescribeAutoScalingGroupResponse returns 12 | type DescribeAutoScalingGroupResponse struct { 13 | Resp *autoscaling.DescribeAutoScalingGroupsOutput 14 | Error error 15 | } 16 | 17 | // DescribeLaunchConfigurationsResponse returns 18 | type DescribeLaunchConfigurationsResponse struct { 19 | Resp *autoscaling.DescribeLaunchConfigurationsOutput 20 | Error error 21 | } 22 | 23 | // DescribePoliciesResponse returns 24 | type DescribePoliciesResponse struct { 25 | Resp *autoscaling.DescribePoliciesOutput 26 | Error error 27 | } 28 | 29 | // ASGClient returns 30 | type ASGClient struct { 31 | aws.ASGAPI 32 | DescribeAutoScalingGroupsPageResp []DescribeAutoScalingGroupResponse 33 | DescribeLaunchConfigurationsResp map[string]*DescribeLaunchConfigurationsResponse 34 | DescribePoliciesResp map[string]*DescribePoliciesResponse 35 | 36 | DescribeLoadBalancerTargetGroupsOutput *autoscaling.DescribeLoadBalancerTargetGroupsOutput 37 | DescribeLoadBalancersOutput *autoscaling.DescribeLoadBalancersOutput 38 | 39 | UpdateAutoScalingGroupLastInput *autoscaling.UpdateAutoScalingGroupInput 40 | DetachLoadBalancersError error 41 | } 42 | 43 | func (m *ASGClient) init() { 44 | if m.DescribeAutoScalingGroupsPageResp == nil { 45 | m.DescribeAutoScalingGroupsPageResp = []DescribeAutoScalingGroupResponse{} 46 | } 47 | 48 | if m.DescribeLaunchConfigurationsResp == nil { 49 | m.DescribeLaunchConfigurationsResp = map[string]*DescribeLaunchConfigurationsResponse{} 50 | } 51 | 52 | if m.DescribePoliciesResp == nil { 53 | m.DescribePoliciesResp = map[string]*DescribePoliciesResponse{} 54 | } 55 | } 56 | 57 | // MakeMockASG returns 58 | func MakeMockASG(name string, projetName string, configName string, serviceName string, releaseID string) *autoscaling.Group { 59 | return &autoscaling.Group{ 60 | AutoScalingGroupName: to.Strp(name), 61 | Instances: MakeMockASGInstances(1, 0, 0), 62 | LoadBalancerNames: []*string{to.Strp("elb")}, 63 | TargetGroupARNs: []*string{to.Strp("tg")}, 64 | 65 | MinSize: to.Int64p(1), 66 | MaxSize: to.Int64p(3), 67 | DesiredCapacity: to.Int64p(1), 68 | Tags: []*autoscaling.TagDescription{ 69 | &autoscaling.TagDescription{Key: to.Strp("ProjectName"), Value: to.Strp(projetName)}, 70 | &autoscaling.TagDescription{Key: to.Strp("ConfigName"), Value: to.Strp(configName)}, 71 | &autoscaling.TagDescription{Key: to.Strp("ServiceName"), Value: to.Strp(serviceName)}, 72 | &autoscaling.TagDescription{Key: to.Strp("ReleaseID"), Value: to.Strp(releaseID)}, 73 | }, 74 | } 75 | } 76 | 77 | // MakeMockASGInstances returns 78 | func MakeMockASGInstances(healthy int, unhealthy int, terming int) []*autoscaling.Instance { 79 | ins := []*autoscaling.Instance{} 80 | x := 0 81 | for i := 0; i < healthy; i++ { 82 | x++ 83 | ins = append(ins, &autoscaling.Instance{ 84 | InstanceId: to.Strp(fmt.Sprintf("InstanceId%v", x)), 85 | HealthStatus: to.Strp("Healthy"), 86 | LifecycleState: to.Strp("InService"), 87 | }) 88 | } 89 | 90 | for i := 0; i < unhealthy; i++ { 91 | x++ 92 | ins = append(ins, &autoscaling.Instance{ 93 | InstanceId: to.Strp(fmt.Sprintf("InstanceId%v", x)), 94 | HealthStatus: to.Strp("Unhealthy"), 95 | LifecycleState: to.Strp("Waiting"), 96 | }) 97 | } 98 | 99 | for i := 0; i < terming; i++ { 100 | x++ 101 | ins = append(ins, &autoscaling.Instance{ 102 | InstanceId: to.Strp(fmt.Sprintf("InstanceId%v", x)), 103 | HealthStatus: to.Strp("Terminating"), 104 | LifecycleState: to.Strp("Terminating"), 105 | }) 106 | } 107 | return ins 108 | } 109 | 110 | // AddASG returns 111 | func (m *ASGClient) AddASG(asg *autoscaling.Group) { 112 | m.init() 113 | m.DescribeAutoScalingGroupsPageResp = append(m.DescribeAutoScalingGroupsPageResp, 114 | DescribeAutoScalingGroupResponse{ 115 | Resp: &autoscaling.DescribeAutoScalingGroupsOutput{ 116 | AutoScalingGroups: []*autoscaling.Group{ 117 | asg, 118 | }, 119 | }, 120 | }, 121 | ) 122 | } 123 | 124 | // AddPreviousRuntimeResources returns 125 | func (m *ASGClient) AddPreviousRuntimeResources(projectName string, configName string, serviceName string, releaseID string) string { 126 | m.init() 127 | 128 | name := fmt.Sprintf("%v-%v-%v-%v", projectName, configName, serviceName, releaseID) 129 | 130 | m.AddASG(MakeMockASG(name, projectName, configName, serviceName, releaseID)) 131 | 132 | m.DescribeLaunchConfigurationsResp[name] = &DescribeLaunchConfigurationsResponse{ 133 | Resp: &autoscaling.DescribeLaunchConfigurationsOutput{ 134 | LaunchConfigurations: []*autoscaling.LaunchConfiguration{ 135 | &autoscaling.LaunchConfiguration{LaunchConfigurationName: to.Strp(name)}, 136 | }, 137 | }, 138 | } 139 | 140 | m.DescribePoliciesResp[name] = &DescribePoliciesResponse{ 141 | Resp: &autoscaling.DescribePoliciesOutput{ 142 | ScalingPolicies: []*autoscaling.ScalingPolicy{ 143 | &autoscaling.ScalingPolicy{ 144 | Alarms: []*autoscaling.Alarm{ 145 | &autoscaling.Alarm{ 146 | AlarmName: to.Strp("VeryEmbeddedAlarm"), 147 | }, 148 | }, 149 | }, 150 | }, 151 | }, 152 | } 153 | 154 | return name 155 | } 156 | 157 | // DescribeAutoScalingGroupsPages returns 158 | func (m *ASGClient) DescribeAutoScalingGroupsPages(input *autoscaling.DescribeAutoScalingGroupsInput, fn func(*autoscaling.DescribeAutoScalingGroupsOutput, bool) bool) error { 159 | m.init() 160 | // Loop through all autoscaling groups, 1 per page 161 | var cont bool 162 | for _, page := range m.DescribeAutoScalingGroupsPageResp { 163 | if page.Error != nil { 164 | return page.Error 165 | } 166 | 167 | cont = fn(page.Resp, false) 168 | 169 | if !cont { 170 | return fmt.Errorf("Should always end here") 171 | } 172 | } 173 | 174 | cont = fn(&autoscaling.DescribeAutoScalingGroupsOutput{}, true) 175 | 176 | if cont { 177 | return fmt.Errorf("Should always end here") 178 | } 179 | 180 | return nil 181 | } 182 | 183 | // DeleteAutoScalingGroup returns 184 | func (m *ASGClient) DeleteAutoScalingGroup(input *autoscaling.DeleteAutoScalingGroupInput) (*autoscaling.DeleteAutoScalingGroupOutput, error) { 185 | return nil, nil 186 | } 187 | 188 | // CreateAutoScalingGroup returns 189 | func (m *ASGClient) CreateAutoScalingGroup(input *autoscaling.CreateAutoScalingGroupInput) (*autoscaling.CreateAutoScalingGroupOutput, error) { 190 | return nil, nil 191 | } 192 | 193 | // DescribeLaunchConfigurations returns 194 | func (m *ASGClient) DescribeLaunchConfigurations(in *autoscaling.DescribeLaunchConfigurationsInput) (*autoscaling.DescribeLaunchConfigurationsOutput, error) { 195 | m.init() 196 | lcName := in.LaunchConfigurationNames[0] 197 | resp := m.DescribeLaunchConfigurationsResp[*lcName] 198 | if resp == nil { 199 | return &autoscaling.DescribeLaunchConfigurationsOutput{LaunchConfigurations: []*autoscaling.LaunchConfiguration{}}, nil 200 | } 201 | return resp.Resp, resp.Error 202 | } 203 | 204 | // CreateLaunchConfiguration returns 205 | func (m *ASGClient) CreateLaunchConfiguration(input *autoscaling.CreateLaunchConfigurationInput) (*autoscaling.CreateLaunchConfigurationOutput, error) { 206 | return nil, nil 207 | } 208 | 209 | // DeleteLaunchConfiguration returns 210 | func (m *ASGClient) DeleteLaunchConfiguration(input *autoscaling.DeleteLaunchConfigurationInput) (*autoscaling.DeleteLaunchConfigurationOutput, error) { 211 | return nil, nil 212 | } 213 | 214 | // DescribePolicies returns 215 | func (m *ASGClient) DescribePolicies(in *autoscaling.DescribePoliciesInput) (*autoscaling.DescribePoliciesOutput, error) { 216 | m.init() 217 | resp := m.DescribePoliciesResp[*in.AutoScalingGroupName] 218 | if resp == nil { 219 | return &autoscaling.DescribePoliciesOutput{}, nil 220 | } 221 | return resp.Resp, resp.Error 222 | } 223 | 224 | // EnableMetricsCollection returns 225 | func (m *ASGClient) EnableMetricsCollection(input *autoscaling.EnableMetricsCollectionInput) (*autoscaling.EnableMetricsCollectionOutput, error) { 226 | return nil, nil 227 | } 228 | 229 | // PutScalingPolicy returns 230 | func (m *ASGClient) PutScalingPolicy(input *autoscaling.PutScalingPolicyInput) (*autoscaling.PutScalingPolicyOutput, error) { 231 | return &autoscaling.PutScalingPolicyOutput{PolicyARN: to.Strp("arn")}, nil 232 | } 233 | 234 | func (m *ASGClient) DetachLoadBalancers(input *autoscaling.DetachLoadBalancersInput) (*autoscaling.DetachLoadBalancersOutput, error) { 235 | return nil, m.DetachLoadBalancersError 236 | } 237 | 238 | func (m *ASGClient) DetachLoadBalancerTargetGroups(input *autoscaling.DetachLoadBalancerTargetGroupsInput) (*autoscaling.DetachLoadBalancerTargetGroupsOutput, error) { 239 | return nil, nil 240 | } 241 | 242 | func (m *ASGClient) DescribeLoadBalancerTargetGroups(input *autoscaling.DescribeLoadBalancerTargetGroupsInput) (*autoscaling.DescribeLoadBalancerTargetGroupsOutput, error) { 243 | if m.DescribeLoadBalancerTargetGroupsOutput != nil { 244 | return m.DescribeLoadBalancerTargetGroupsOutput, nil 245 | } 246 | return &autoscaling.DescribeLoadBalancerTargetGroupsOutput{}, nil 247 | } 248 | 249 | func (m *ASGClient) DescribeLoadBalancers(input *autoscaling.DescribeLoadBalancersInput) (*autoscaling.DescribeLoadBalancersOutput, error) { 250 | if m.DescribeLoadBalancersOutput != nil { 251 | return m.DescribeLoadBalancersOutput, nil 252 | } 253 | return &autoscaling.DescribeLoadBalancersOutput{}, nil 254 | } 255 | 256 | func (m *ASGClient) UpdateAutoScalingGroup(input *autoscaling.UpdateAutoScalingGroupInput) (*autoscaling.UpdateAutoScalingGroupOutput, error) { 257 | m.UpdateAutoScalingGroupLastInput = input 258 | return nil, nil 259 | } 260 | -------------------------------------------------------------------------------- /aws/mocks/mock_cw.go: -------------------------------------------------------------------------------- 1 | package mocks 2 | 3 | import ( 4 | "github.com/aws/aws-sdk-go/service/cloudwatch" 5 | "github.com/coinbase/odin/aws" 6 | ) 7 | 8 | // CWClient struct 9 | type CWClient struct { 10 | aws.CWAPI 11 | } 12 | 13 | // DeleteAlarms returns 14 | func (m *CWClient) DeleteAlarms(input *cloudwatch.DeleteAlarmsInput) (*cloudwatch.DeleteAlarmsOutput, error) { 15 | return nil, nil 16 | } 17 | 18 | // PutMetricAlarm returns 19 | func (m *CWClient) PutMetricAlarm(input *cloudwatch.PutMetricAlarmInput) (*cloudwatch.PutMetricAlarmOutput, error) { 20 | return nil, nil 21 | } 22 | -------------------------------------------------------------------------------- /aws/mocks/mock_ec2.go: -------------------------------------------------------------------------------- 1 | package mocks 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/aws/aws-sdk-go/service/ec2" 7 | "github.com/coinbase/odin/aws" 8 | "github.com/coinbase/step/utils/to" 9 | ) 10 | 11 | // DescribeSubnetsResponse returns 12 | type DescribeSubnetsResponse struct { 13 | Resp *ec2.DescribeSubnetsOutput 14 | Error error 15 | } 16 | 17 | // DescribeImagesResponse returns 18 | type DescribeImagesResponse struct { 19 | Resp *ec2.DescribeImagesOutput 20 | Error error 21 | } 22 | 23 | // DescribeSecurityGroupsResponse returns 24 | type DescribeSecurityGroupsResponse struct { 25 | Resp *ec2.DescribeSecurityGroupsOutput 26 | Error error 27 | } 28 | 29 | // EC2Client returns 30 | type EC2Client struct { 31 | aws.EC2API 32 | DescribeSecurityGroupsResp map[string]*DescribeSecurityGroupsResponse 33 | DescribeSubnetsResp *DescribeSubnetsResponse 34 | DescribeImagesResp *DescribeImagesResponse 35 | PlacementGroups []*ec2.PlacementGroup 36 | } 37 | 38 | func (m *EC2Client) init() { 39 | if m.DescribeSecurityGroupsResp == nil { 40 | m.DescribeSecurityGroupsResp = map[string]*DescribeSecurityGroupsResponse{} 41 | } 42 | if m.PlacementGroups == nil { 43 | m.PlacementGroups = []*ec2.PlacementGroup{} 44 | } 45 | } 46 | 47 | // AddSecurityGroup returns 48 | func (m *EC2Client) AddSecurityGroup(name string, projectName string, configName string, serviceName string, err error) { 49 | m.init() 50 | m.DescribeSecurityGroupsResp[name] = &DescribeSecurityGroupsResponse{ 51 | Resp: &ec2.DescribeSecurityGroupsOutput{ 52 | SecurityGroups: []*ec2.SecurityGroup{ 53 | MakeMockSecurityGroup(name, projectName, configName, serviceName), 54 | }, 55 | }, 56 | Error: err, 57 | } 58 | } 59 | 60 | // AddImage returns 61 | func (m *EC2Client) AddImage(nameTag string, id string) { 62 | m.DescribeImagesResp = &DescribeImagesResponse{ 63 | Resp: &ec2.DescribeImagesOutput{ 64 | Images: []*ec2.Image{ 65 | &ec2.Image{ 66 | ImageId: to.Strp(id), 67 | Tags: []*ec2.Tag{ 68 | &ec2.Tag{Key: to.Strp("Name"), Value: to.Strp(nameTag)}, 69 | &ec2.Tag{Key: to.Strp("DeployWith"), Value: to.Strp("odin")}, 70 | }, 71 | }, 72 | }, 73 | }, 74 | } 75 | } 76 | 77 | // AddSubnet returns 78 | func (m *EC2Client) AddSubnet(nameTag string, id string) { 79 | m.DescribeSubnetsResp = &DescribeSubnetsResponse{ 80 | Resp: &ec2.DescribeSubnetsOutput{ 81 | Subnets: []*ec2.Subnet{ 82 | &ec2.Subnet{ 83 | SubnetId: to.Strp(id), 84 | Tags: []*ec2.Tag{ 85 | &ec2.Tag{Key: to.Strp("Name"), Value: to.Strp(nameTag)}, 86 | &ec2.Tag{Key: to.Strp("DeployWith"), Value: to.Strp("odin")}, 87 | }, 88 | }, 89 | }, 90 | }, 91 | } 92 | } 93 | 94 | // DescribeSecurityGroups returns 95 | func (m *EC2Client) DescribeSecurityGroups(in *ec2.DescribeSecurityGroupsInput) (*ec2.DescribeSecurityGroupsOutput, error) { 96 | m.init() 97 | sgName := in.Filters[0].Values[0] 98 | resp := m.DescribeSecurityGroupsResp[*sgName] 99 | if resp == nil { 100 | return &ec2.DescribeSecurityGroupsOutput{SecurityGroups: []*ec2.SecurityGroup{}}, nil 101 | } 102 | return resp.Resp, resp.Error 103 | } 104 | 105 | // MakeMockSecurityGroup returns 106 | func MakeMockSecurityGroup(name string, projectName string, configName string, serviceName string) *ec2.SecurityGroup { 107 | return &ec2.SecurityGroup{ 108 | GroupId: to.Strp("group-id"), 109 | Tags: []*ec2.Tag{ 110 | &ec2.Tag{Key: to.Strp("Name"), Value: to.Strp(name)}, 111 | &ec2.Tag{Key: to.Strp("ProjectName"), Value: to.Strp(projectName)}, 112 | &ec2.Tag{Key: to.Strp("ConfigName"), Value: to.Strp(configName)}, 113 | &ec2.Tag{Key: to.Strp("ServiceName"), Value: to.Strp(serviceName)}, 114 | }, 115 | } 116 | } 117 | 118 | // DescribeSubnets returns 119 | func (m *EC2Client) DescribeSubnets(in *ec2.DescribeSubnetsInput) (*ec2.DescribeSubnetsOutput, error) { 120 | if m.DescribeSubnetsResp == nil { 121 | return nil, fmt.Errorf("Add Subnets") 122 | } 123 | 124 | return m.DescribeSubnetsResp.Resp, m.DescribeSubnetsResp.Error 125 | } 126 | 127 | // DescribeImages returns 128 | func (m *EC2Client) DescribeImages(in *ec2.DescribeImagesInput) (*ec2.DescribeImagesOutput, error) { 129 | if m.DescribeImagesResp == nil { 130 | return nil, fmt.Errorf("Add Image") 131 | } 132 | 133 | return m.DescribeImagesResp.Resp, m.DescribeImagesResp.Error 134 | } 135 | 136 | func (m *EC2Client) DescribePlacementGroups(in *ec2.DescribePlacementGroupsInput) (*ec2.DescribePlacementGroupsOutput, error) { 137 | m.init() 138 | return &ec2.DescribePlacementGroupsOutput{ 139 | PlacementGroups: m.PlacementGroups, 140 | }, nil 141 | } 142 | 143 | func (m *EC2Client) CreatePlacementGroup(in *ec2.CreatePlacementGroupInput) (*ec2.CreatePlacementGroupOutput, error) { 144 | m.init() 145 | m.PlacementGroups = append(m.PlacementGroups, &ec2.PlacementGroup{ 146 | GroupName: in.GroupName, 147 | PartitionCount: in.PartitionCount, 148 | Strategy: in.Strategy, 149 | State: to.Strp("available"), 150 | }) 151 | 152 | return nil, nil 153 | } 154 | -------------------------------------------------------------------------------- /aws/mocks/mock_elb.go: -------------------------------------------------------------------------------- 1 | package mocks 2 | 3 | import ( 4 | "github.com/aws/aws-sdk-go/aws/awserr" 5 | "github.com/aws/aws-sdk-go/service/elb" 6 | "github.com/coinbase/odin/aws" 7 | "github.com/coinbase/step/utils/to" 8 | ) 9 | 10 | // DescribeLoadBalancersResponse returns 11 | type DescribeLoadBalancersResponse struct { 12 | Resp *elb.DescribeLoadBalancersOutput 13 | Error error 14 | } 15 | 16 | // DescribeTagsResponse returns 17 | type DescribeTagsResponse struct { 18 | Resp *elb.DescribeTagsOutput 19 | Error error 20 | } 21 | 22 | // DescribeInstanceHealthResponse returns 23 | type DescribeInstanceHealthResponse struct { 24 | Resp *elb.DescribeInstanceHealthOutput 25 | Error error 26 | } 27 | 28 | // ELBClient returns 29 | type ELBClient struct { 30 | aws.ELBAPI 31 | DescribeLoadBalancersResp map[string]*DescribeLoadBalancersResponse 32 | DescribeTagsResp map[string]*DescribeTagsResponse 33 | DescribeInstanceHealthResp map[string]*DescribeInstanceHealthResponse 34 | } 35 | 36 | // AWSELBNotFoundError returns 37 | func AWSELBNotFoundError() error { 38 | return awserr.New(elb.ErrCodeAccessPointNotFoundException, "LoadBalancerNotFound", nil) 39 | } 40 | 41 | func (m *ELBClient) init() { 42 | if m.DescribeLoadBalancersResp == nil { 43 | m.DescribeLoadBalancersResp = map[string]*DescribeLoadBalancersResponse{} 44 | } 45 | 46 | if m.DescribeTagsResp == nil { 47 | m.DescribeTagsResp = map[string]*DescribeTagsResponse{} 48 | } 49 | 50 | if m.DescribeInstanceHealthResp == nil { 51 | m.DescribeInstanceHealthResp = map[string]*DescribeInstanceHealthResponse{} 52 | } 53 | } 54 | 55 | // AddELB returns 56 | func (m *ELBClient) AddELB(name string, projectName string, configName string, serviceName string) { 57 | m.init() 58 | m.DescribeLoadBalancersResp[name] = &DescribeLoadBalancersResponse{ 59 | Resp: &elb.DescribeLoadBalancersOutput{ 60 | LoadBalancerDescriptions: []*elb.LoadBalancerDescription{ 61 | &elb.LoadBalancerDescription{LoadBalancerName: &name}, 62 | }, 63 | }, 64 | } 65 | 66 | m.DescribeTagsResp[name] = &DescribeTagsResponse{ 67 | Resp: &elb.DescribeTagsOutput{ 68 | TagDescriptions: []*elb.TagDescription{ 69 | &elb.TagDescription{ 70 | LoadBalancerName: &name, 71 | Tags: []*elb.Tag{ 72 | &elb.Tag{Key: to.Strp("ProjectName"), Value: to.Strp(projectName)}, 73 | &elb.Tag{Key: to.Strp("ConfigName"), Value: to.Strp(configName)}, 74 | &elb.Tag{Key: to.Strp("ServiceName"), Value: to.Strp(serviceName)}, 75 | }, 76 | }, 77 | }, 78 | }, 79 | } 80 | 81 | m.DescribeInstanceHealthResp[name] = &DescribeInstanceHealthResponse{ 82 | Resp: &elb.DescribeInstanceHealthOutput{ 83 | InstanceStates: []*elb.InstanceState{ 84 | &elb.InstanceState{ 85 | InstanceId: to.Strp("InstanceId1"), 86 | State: to.Strp("InService"), 87 | }, 88 | }, 89 | }, 90 | } 91 | 92 | } 93 | 94 | // DescribeLoadBalancers returns 95 | func (m *ELBClient) DescribeLoadBalancers(in *elb.DescribeLoadBalancersInput) (*elb.DescribeLoadBalancersOutput, error) { 96 | m.init() 97 | lbName := in.LoadBalancerNames[0] 98 | resp := m.DescribeLoadBalancersResp[*lbName] 99 | if resp == nil { 100 | return nil, AWSELBNotFoundError() 101 | } 102 | return resp.Resp, resp.Error 103 | } 104 | 105 | // DescribeTags returns 106 | func (m *ELBClient) DescribeTags(in *elb.DescribeTagsInput) (*elb.DescribeTagsOutput, error) { 107 | m.init() 108 | lbName := in.LoadBalancerNames[0] 109 | resp := m.DescribeTagsResp[*lbName] 110 | if resp == nil { 111 | return nil, AWSELBNotFoundError() 112 | } 113 | return resp.Resp, resp.Error 114 | } 115 | 116 | // DescribeInstanceHealth returns 117 | func (m *ELBClient) DescribeInstanceHealth(in *elb.DescribeInstanceHealthInput) (*elb.DescribeInstanceHealthOutput, error) { 118 | m.init() 119 | lbName := in.LoadBalancerName 120 | resp := m.DescribeInstanceHealthResp[*lbName] 121 | if resp == nil { 122 | return nil, AWSELBNotFoundError() 123 | } 124 | if resp.Resp == nil { 125 | return &elb.DescribeInstanceHealthOutput{}, nil 126 | } 127 | return resp.Resp, resp.Error 128 | } 129 | -------------------------------------------------------------------------------- /aws/mocks/mock_iam.go: -------------------------------------------------------------------------------- 1 | package mocks 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/aws/aws-sdk-go/aws/awserr" 7 | "github.com/aws/aws-sdk-go/service/iam" 8 | "github.com/coinbase/odin/aws" 9 | "github.com/coinbase/step/utils/to" 10 | ) 11 | 12 | // GetInstanceProfileResponse returns 13 | type GetInstanceProfileResponse struct { 14 | Resp *iam.GetInstanceProfileOutput 15 | Error error 16 | } 17 | 18 | // GetRoleResponse returns 19 | type GetRoleResponse struct { 20 | Resp *iam.GetRoleOutput 21 | Error error 22 | } 23 | 24 | // IAMClient returns 25 | type IAMClient struct { 26 | aws.IAMAPI 27 | GetInstanceProfileResp map[string]*GetInstanceProfileResponse 28 | GetRoleResp map[string]*GetRoleResponse 29 | } 30 | 31 | func (m *IAMClient) init() { 32 | if m.GetInstanceProfileResp == nil { 33 | m.GetInstanceProfileResp = map[string]*GetInstanceProfileResponse{} 34 | } 35 | 36 | if m.GetRoleResp == nil { 37 | m.GetRoleResp = map[string]*GetRoleResponse{} 38 | } 39 | } 40 | 41 | // AWSProfileNotFoundError returns 42 | func AWSProfileNotFoundError() error { 43 | return awserr.New(iam.ErrCodeNoSuchEntityException, "NoSuchEntity", nil) 44 | } 45 | 46 | // AddGetInstanceProfile returns 47 | func (m *IAMClient) AddGetInstanceProfile(profileName string, path string) { 48 | m.init() 49 | m.GetInstanceProfileResp[profileName] = &GetInstanceProfileResponse{ 50 | Resp: &iam.GetInstanceProfileOutput{ 51 | InstanceProfile: &iam.InstanceProfile{ 52 | Arn: to.Strp(fmt.Sprintf("%v%v", path, profileName)), 53 | Path: to.Strp(path), 54 | }, 55 | }, 56 | } 57 | } 58 | 59 | // AddGetRole returns 60 | func (m *IAMClient) AddGetRole(roleName string) { 61 | m.init() 62 | m.GetRoleResp[roleName] = &GetRoleResponse{ 63 | Resp: &iam.GetRoleOutput{ 64 | Role: &iam.Role{ 65 | Arn: to.Strp(roleName), 66 | }, 67 | }, 68 | } 69 | } 70 | 71 | // GetInstanceProfile returns 72 | func (m *IAMClient) GetInstanceProfile(in *iam.GetInstanceProfileInput) (*iam.GetInstanceProfileOutput, error) { 73 | m.init() 74 | resp := m.GetInstanceProfileResp[*in.InstanceProfileName] 75 | if resp == nil { 76 | return nil, AWSProfileNotFoundError() 77 | } 78 | return resp.Resp, resp.Error 79 | } 80 | 81 | // GetRole returns 82 | func (m *IAMClient) GetRole(in *iam.GetRoleInput) (*iam.GetRoleOutput, error) { 83 | m.init() 84 | resp := m.GetRoleResp[*in.RoleName] 85 | if resp == nil { 86 | return nil, AWSProfileNotFoundError() 87 | } 88 | return resp.Resp, resp.Error 89 | } 90 | -------------------------------------------------------------------------------- /aws/mocks/mock_sns.go: -------------------------------------------------------------------------------- 1 | package mocks 2 | 3 | import ( 4 | "github.com/aws/aws-sdk-go/service/sns" 5 | "github.com/coinbase/odin/aws" 6 | ) 7 | 8 | // SNSClient returns 9 | type SNSClient struct { 10 | aws.SNSAPI 11 | } 12 | 13 | // GetTopicAttributes returns 14 | func (m *SNSClient) GetTopicAttributes(in *sns.GetTopicAttributesInput) (*sns.GetTopicAttributesOutput, error) { 15 | return nil, nil 16 | } 17 | -------------------------------------------------------------------------------- /aws/pg/placement_group.go: -------------------------------------------------------------------------------- 1 | package pg 2 | 3 | import ( 4 | "fmt" 5 | "strings" 6 | 7 | "github.com/aws/aws-sdk-go/service/ec2" 8 | "github.com/coinbase/odin/aws" 9 | "github.com/coinbase/step/utils/to" 10 | ) 11 | 12 | // FindOrCreatePartitionGroup will find a partition group by name. 13 | // If it doenst exist: 14 | // - It will create one with the provided detail (error if detail is nil) 15 | // If one does exist: 16 | // - It will fetch that Partition Group and Error if any provided detail is missing 17 | func FindOrCreatePartitionGroup(ec2c aws.EC2API, prefix string, groupName *string, partitionCount *int64, strategy *string) error { 18 | if groupName == nil { 19 | return fmt.Errorf("PlacementGroupError: groupName nil") 20 | } 21 | 22 | pg, err := findPlacementGroup(ec2c, groupName) 23 | if err != nil { 24 | return err 25 | } 26 | 27 | // If no pg is found create a new one 28 | // otherwise validate that the found one equals the correct values 29 | if pg == nil { 30 | // If we are creating the Placement Group we restrict the name to prefix 31 | if !strings.HasPrefix(*groupName, prefix) { 32 | return fmt.Errorf("PlacementGroupError(%s): If this is a new PlacementGroup it must have %q prefix", to.Strs(groupName), prefix) 33 | } 34 | 35 | return createNewPlacementGroup(ec2c, groupName, partitionCount, strategy) 36 | } 37 | 38 | return validatePlacementGroup(pg, groupName, partitionCount, strategy) 39 | } 40 | 41 | // findPlacementGroup will search for a placement group, if none are found it will return (nil, nil) 42 | func findPlacementGroup(ec2c aws.EC2API, groupName *string) (*ec2.PlacementGroup, error) { 43 | out, err := ec2c.DescribePlacementGroups(&ec2.DescribePlacementGroupsInput{ 44 | Filters: []*ec2.Filter{ 45 | &ec2.Filter{ 46 | Name: to.Strp("group-name"), 47 | Values: []*string{groupName}, 48 | }, 49 | }, 50 | }) 51 | 52 | if err != nil { 53 | return nil, err 54 | } 55 | 56 | for _, checkPG := range out.PlacementGroups { 57 | if to.Strs(checkPG.GroupName) == to.Strs(groupName) { 58 | return checkPG, nil 59 | } 60 | } 61 | 62 | return nil, nil 63 | } 64 | 65 | func createNewPlacementGroup(ec2c aws.EC2API, groupName *string, partitionCount *int64, strategy *string) error { 66 | _, err := ec2c.CreatePlacementGroup(&ec2.CreatePlacementGroupInput{ 67 | GroupName: groupName, 68 | PartitionCount: partitionCount, 69 | Strategy: strategy, 70 | }) 71 | return err 72 | } 73 | 74 | func validatePlacementGroup(pg *ec2.PlacementGroup, groupName *string, partitionCount *int64, strategy *string) error { 75 | // pending | available | deleting | deleted 76 | if to.Strs(pg.State) != "available" { 77 | return fmt.Errorf("PlacementGroupError(%s): PG in invalid state %s", to.Strs(groupName), to.Strs(pg.State)) 78 | } 79 | 80 | if to.Strs(groupName) != to.Strs(pg.GroupName) { 81 | return fmt.Errorf("PlacementGroupError(%s): PG in invalid name %s", to.Strs(groupName), to.Strs(pg.GroupName)) 82 | } 83 | 84 | if to.Strs(strategy) != to.Strs(pg.Strategy) { 85 | return fmt.Errorf("PlacementGroupError(%s): PG in invalid strategy expected %s, got %s", to.Strs(groupName), to.Strs(pg.Strategy), to.Strs(strategy)) 86 | } 87 | 88 | // Partition count should be equal only if strategy is 'partition' 89 | if *strategy != "partition" { 90 | return nil 91 | } 92 | 93 | if partitionCount == nil || pg.PartitionCount == nil { 94 | // We should never get here, but checking is easy 95 | return fmt.Errorf("PlacementGroupError(%s): PG has nil PartitionCount", to.Strs(groupName)) 96 | } 97 | 98 | if *partitionCount != *pg.PartitionCount { 99 | return fmt.Errorf("PlacementGroupError(%s): PG in invalid strategy expected %q, got %q", to.Strs(groupName), *pg.PartitionCount, *partitionCount) 100 | } 101 | 102 | return nil 103 | } 104 | -------------------------------------------------------------------------------- /aws/pg/placement_group_test.go: -------------------------------------------------------------------------------- 1 | package pg 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/stretchr/testify/assert" 7 | 8 | "github.com/coinbase/odin/aws/mocks" 9 | "github.com/coinbase/step/utils/to" 10 | ) 11 | 12 | func Test_FindOrCreatePartitionGroup(t *testing.T) { 13 | //func Find(ec2Client aws.EC2API, name_tags_or_ids []*string) ([]*Subnet, error) { 14 | ec2c := &mocks.EC2Client{} 15 | 16 | // This will creates a placement group 17 | err := FindOrCreatePartitionGroup(ec2c, "i/i", to.Strp("i/i/groupName"), to.Int64p(10), to.Strp("cluster")) 18 | assert.NoError(t, err) 19 | 20 | // Finds the already created group 21 | err = FindOrCreatePartitionGroup(ec2c, "i/i", to.Strp("i/i/groupName"), to.Int64p(10), to.Strp("cluster")) 22 | assert.NoError(t, err) 23 | 24 | // This will error because the strategy is incorrect 25 | err = FindOrCreatePartitionGroup(ec2c, "i/i", to.Strp("i/i/groupName"), to.Int64p(10), to.Strp("wrong_stratgy")) 26 | assert.Error(t, err) 27 | 28 | err = FindOrCreatePartitionGroup(ec2c, "bad_prefix", to.Strp("i/i/groupName"), to.Int64p(10), to.Strp("wrong_stratgy")) 29 | assert.Error(t, err) 30 | } 31 | -------------------------------------------------------------------------------- /aws/sg/security_group.go: -------------------------------------------------------------------------------- 1 | package sg 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/aws/aws-sdk-go/service/ec2" 7 | "github.com/coinbase/odin/aws" 8 | "github.com/coinbase/step/utils/to" 9 | ) 10 | 11 | // SecurityGroup struct 12 | type SecurityGroup struct { 13 | NameTag *string 14 | ProjectNameTag *string 15 | ConfigNameTag *string 16 | ServiceNameTag *string 17 | GroupID *string 18 | } 19 | 20 | // ProjectName returns tag 21 | func (s *SecurityGroup) ProjectName() *string { 22 | return s.ProjectNameTag 23 | } 24 | 25 | // ConfigName returns tag 26 | func (s *SecurityGroup) ConfigName() *string { 27 | return s.ConfigNameTag 28 | } 29 | 30 | // ServiceName returns tag 31 | func (s *SecurityGroup) ServiceName() *string { 32 | return s.ServiceNameTag 33 | } 34 | 35 | // Name returns tag 36 | func (s *SecurityGroup) Name() *string { 37 | return s.NameTag 38 | } 39 | 40 | // AllowedService returns which service is allowed to attach to it 41 | func (s *SecurityGroup) AllowedService() *string { 42 | return to.Strp(fmt.Sprintf("%s::%s::%s", *s.ProjectName(), *s.ConfigName(), *s.ServiceName())) 43 | } 44 | 45 | // Find returns the security groups with tags 46 | func Find(ec2Client aws.EC2API, nameTags []*string) ([]*SecurityGroup, error) { 47 | output, err := ec2Client.DescribeSecurityGroups(&ec2.DescribeSecurityGroupsInput{ 48 | Filters: []*ec2.Filter{ 49 | &ec2.Filter{ 50 | Name: to.Strp("tag:Name"), 51 | Values: nameTags, 52 | }}}) 53 | 54 | if err != nil { 55 | return nil, err 56 | } 57 | 58 | sgs := newSGs(output.SecurityGroups) 59 | 60 | // Need to validate that each Name tag matches Exactly one Security Group 61 | for _, nameTag := range nameTags { 62 | matches := 0 63 | for _, sg := range sgs { 64 | if sg.NameTag == nil { 65 | return nil, fmt.Errorf("SecurityGroup '%v': incorrect Name Tag", *nameTag) 66 | } 67 | 68 | if *sg.NameTag == *nameTag { 69 | matches += 1 70 | } 71 | } 72 | 73 | switch matches { 74 | case 0: 75 | return nil, fmt.Errorf("SecurityGroup '%v': not found", *nameTag) 76 | case 1: 77 | // Do nothing 78 | default: 79 | return nil, fmt.Errorf("SecurityGroup '%v': too many found", *nameTag) 80 | } 81 | } 82 | 83 | if len(sgs) != len(nameTags) { 84 | // Last assurance that no additional security groups were found 85 | return nil, fmt.Errorf("SecurityGroup: found %v required %v", len(sgs), len(nameTags)) 86 | } 87 | 88 | return sgs, nil 89 | } 90 | 91 | func newSGs(output []*ec2.SecurityGroup) []*SecurityGroup { 92 | sgs := []*SecurityGroup{} 93 | 94 | for _, sg := range output { 95 | sgs = append(sgs, &SecurityGroup{ 96 | GroupID: sg.GroupId, 97 | NameTag: aws.FetchEc2Tag(sg.Tags, to.Strp("Name")), 98 | ProjectNameTag: aws.FetchEc2Tag(sg.Tags, to.Strp("ProjectName")), 99 | ConfigNameTag: aws.FetchEc2Tag(sg.Tags, to.Strp("ConfigName")), 100 | ServiceNameTag: aws.FetchEc2Tag(sg.Tags, to.Strp("ServiceName")), 101 | }) 102 | } 103 | 104 | return sgs 105 | } 106 | -------------------------------------------------------------------------------- /aws/sg/security_group_test.go: -------------------------------------------------------------------------------- 1 | package sg 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/odin/aws/mocks" 7 | "github.com/coinbase/step/utils/to" 8 | "github.com/stretchr/testify/assert" 9 | ) 10 | 11 | func Test_Find(t *testing.T) { 12 | //func Find(ec2Client aws.EC2API, name_tags []*string) ([]*SecurityGroup, error) { 13 | ec2c := &mocks.EC2Client{} 14 | _, err := Find(ec2c, []*string{to.Strp("sg1")}) 15 | assert.Error(t, err) 16 | 17 | ec2c.AddSecurityGroup("sg1", "project_name", "config_name", "service_name", nil) 18 | 19 | sgs, err := Find(ec2c, []*string{to.Strp("sg1")}) 20 | assert.NoError(t, err) 21 | assert.Equal(t, 1, len(sgs)) 22 | } 23 | -------------------------------------------------------------------------------- /aws/sns/sns.go: -------------------------------------------------------------------------------- 1 | package sns 2 | 3 | import ( 4 | "github.com/aws/aws-sdk-go/service/sns" 5 | "github.com/coinbase/odin/aws" 6 | ) 7 | 8 | // TopicExists errors if SNS topic doesn't exists 9 | func TopicExists(snsc aws.SNSAPI, topicARN *string) error { 10 | _, err := snsc.GetTopicAttributes(&sns.GetTopicAttributesInput{ 11 | TopicArn: topicARN, 12 | }) 13 | 14 | return err 15 | } 16 | -------------------------------------------------------------------------------- /aws/subnet/subnet_test.go: -------------------------------------------------------------------------------- 1 | package subnet 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/odin/aws/mocks" 7 | "github.com/coinbase/step/utils/to" 8 | "github.com/stretchr/testify/assert" 9 | ) 10 | 11 | func Test_Find_IDs_and_Tags(t *testing.T) { 12 | //func Find(ec2Client aws.EC2API, name_tags_or_ids []*string) ([]*Subnet, error) { 13 | ec2c := &mocks.EC2Client{} 14 | _, err := Find(ec2c, []*string{to.Strp("private-subnet")}) 15 | assert.Error(t, err) 16 | 17 | ec2c.AddSubnet("private-subnet1", "subnet-asd1") 18 | 19 | sgs, err := Find(ec2c, []*string{to.Strp("private-subnet1")}) 20 | assert.NoError(t, err) 21 | assert.Equal(t, 1, len(sgs)) 22 | 23 | sgs, err = Find(ec2c, []*string{to.Strp("subnet-asd1")}) 24 | assert.NoError(t, err) 25 | assert.Equal(t, 1, len(sgs)) 26 | } 27 | 28 | func Test_isID(t *testing.T) { 29 | assert.True(t, isID("subnet-asfasf")) 30 | assert.False(t, isID("ubuntu")) 31 | assert.False(t, isID("subnetgfjosd")) 32 | } 33 | 34 | func Test_splitIDsTags(t *testing.T) { 35 | ids := []*string{to.Strp("subnet-asfasf"), to.Strp("subnet-aasdasdf")} 36 | tags := []*string{to.Strp("privatea"), to.Strp("privateb")} 37 | 38 | unclean := []*string{to.Strp("privatea"), to.Strp("subnet-aasdasdf")} 39 | 40 | i, ts := splitIDsTags(ids) 41 | assert.Equal(t, 2, len(i)) 42 | assert.Equal(t, 0, len(ts)) 43 | 44 | i, ts = splitIDsTags(tags) 45 | assert.Equal(t, 0, len(i)) 46 | assert.Equal(t, 2, len(ts)) 47 | 48 | i, ts = splitIDsTags(unclean) 49 | assert.Equal(t, 1, len(i)) 50 | assert.Equal(t, 1, len(ts)) 51 | } 52 | -------------------------------------------------------------------------------- /aws/subnet/subnets.go: -------------------------------------------------------------------------------- 1 | package subnet 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/aws/aws-sdk-go/service/ec2" 7 | "github.com/coinbase/odin/aws" 8 | "github.com/coinbase/step/utils/to" 9 | ) 10 | 11 | // Subnet struct 12 | type Subnet struct { 13 | SubnetID *string 14 | DeployWithTag *string 15 | } 16 | 17 | // Find returns a list of subnets for either ids or tags NO MIXING , e.g. subnet-00000000 OR privatea 18 | func Find(ec2Client aws.EC2API, nameTagsOrIDs []*string) ([]*Subnet, error) { 19 | ids, tags := splitIDsTags(nameTagsOrIDs) 20 | 21 | subnets := []*Subnet{} 22 | 23 | if len(ids) > 0 { 24 | sns, err := findByID(ec2Client, ids) 25 | if err != nil { 26 | return nil, err 27 | } 28 | subnets = append(subnets, sns...) 29 | } 30 | 31 | if len(tags) > 0 { 32 | sns, err := findByTag(ec2Client, tags) 33 | if err != nil { 34 | return nil, err 35 | } 36 | subnets = append(subnets, sns...) 37 | } 38 | 39 | if len(subnets) != len(nameTagsOrIDs) { 40 | return nil, fmt.Errorf("Incorrect Number of Subnets Found. Found %v, Required %v", len(subnets), len(nameTagsOrIDs)) 41 | } 42 | 43 | return subnets, nil 44 | } 45 | 46 | // isID sees if a string is 47 | func isID(name string) bool { 48 | if len(name) < 8 { 49 | return false 50 | } 51 | 52 | return (name)[0:7] == "subnet-" 53 | } 54 | 55 | // splitIDsTags returns list of ids, and list of tags 56 | func splitIDsTags(nameTagsOrIDs []*string) ([]*string, []*string) { 57 | ids := []*string{} 58 | tags := []*string{} 59 | for _, sn := range nameTagsOrIDs { 60 | if isID(*sn) { 61 | ids = append(ids, sn) 62 | } else { 63 | tags = append(tags, sn) 64 | } 65 | } 66 | 67 | return ids, tags 68 | } 69 | 70 | func findByID(ec2Client aws.EC2API, ids []*string) ([]*Subnet, error) { 71 | return find(ec2Client, &ec2.DescribeSubnetsInput{SubnetIds: ids}) 72 | } 73 | 74 | func findByTag(ec2Client aws.EC2API, nameTags []*string) ([]*Subnet, error) { 75 | filters := []*ec2.Filter{ 76 | &ec2.Filter{ 77 | Name: to.Strp("tag:Name"), 78 | Values: nameTags, 79 | }, 80 | } 81 | 82 | return find(ec2Client, &ec2.DescribeSubnetsInput{Filters: filters}) 83 | } 84 | 85 | func find(ec2Client aws.EC2API, in *ec2.DescribeSubnetsInput) ([]*Subnet, error) { 86 | output, err := ec2Client.DescribeSubnets(in) 87 | 88 | if err != nil { 89 | return nil, err 90 | } 91 | 92 | subnets := []*Subnet{} 93 | for _, subnet := range output.Subnets { 94 | subnets = append(subnets, &Subnet{ 95 | subnet.SubnetId, 96 | aws.FetchEc2Tag(subnet.Tags, to.Strp("DeployWith")), 97 | }) 98 | } 99 | 100 | return subnets, nil 101 | } 102 | -------------------------------------------------------------------------------- /client/client.go: -------------------------------------------------------------------------------- 1 | package client 2 | 3 | import ( 4 | "encoding/json" 5 | "fmt" 6 | "io/ioutil" 7 | "math" 8 | "sort" 9 | "strings" 10 | "time" 11 | 12 | "github.com/coinbase/odin/deployer/models" 13 | "github.com/coinbase/step/execution" 14 | "github.com/coinbase/step/utils/is" 15 | "github.com/coinbase/step/utils/to" 16 | ) 17 | 18 | // executionPrefix returns 19 | func executionPrefix(release *models.Release) string { 20 | pn := strings.Replace(*release.ProjectName, "/", "-", -1) 21 | return fmt.Sprintf("deploy-%v-%v-", pn, *release.ConfigName) 22 | } 23 | 24 | // executionName returns 25 | func executionName(release *models.Release) *string { 26 | return to.TimeUUID(executionPrefix(release)) 27 | } 28 | 29 | // validateClientAttributes returns 30 | func validateClientAttributes(release *models.Release) error { 31 | if release == nil { 32 | // Extra paranoid 33 | return fmt.Errorf("Release is nil") 34 | } 35 | 36 | if is.EmptyStr(release.ProjectName) { 37 | return fmt.Errorf("ProjectName must be defined") 38 | } 39 | 40 | if is.EmptyStr(release.ConfigName) { 41 | return fmt.Errorf("ConfigName must be defined") 42 | } 43 | 44 | if is.EmptyStr(release.Bucket) { 45 | return fmt.Errorf("Bucket must be defined") 46 | } 47 | 48 | return nil 49 | } 50 | 51 | func prepareRelease(release *models.Release, region *string, accountID *string) { 52 | release.Release.SetDefaults(region, accountID, "coinbase-odin-") 53 | release.UUID = nil // Remove UUID 54 | 55 | release.ReleaseID = to.TimeUUID("release-") 56 | release.CreatedAt = to.Timep(time.Now()) 57 | } 58 | 59 | func parseRelease(releaseFile string) (*models.Release, error) { 60 | rawRelease, err := ioutil.ReadFile(releaseFile) 61 | if err != nil { 62 | return nil, err 63 | } 64 | 65 | var release models.Release 66 | if err := json.Unmarshal(rawRelease, &release); err != nil { 67 | return nil, err 68 | } 69 | 70 | return &release, nil 71 | } 72 | 73 | func parseUserData(releaseFile string) (*string, error) { 74 | userdataFile := fmt.Sprintf("%v.userdata", releaseFile) 75 | rawUserData, err := ioutil.ReadFile(userdataFile) 76 | 77 | if err != nil { 78 | return nil, err 79 | } 80 | 81 | return to.Strp(string(rawUserData)), nil 82 | } 83 | 84 | func releaseFromFile(releaseFile *string, region *string, accountID *string) (*models.Release, error) { 85 | release, err := parseRelease(*releaseFile) 86 | if err != nil { 87 | return nil, err 88 | } 89 | 90 | userdata, err := parseUserData(*releaseFile) 91 | if err != nil { 92 | return nil, err 93 | } 94 | 95 | release.SetUserData(userdata) 96 | release.UserDataSHA256 = to.Strp(to.SHA256Str(userdata)) 97 | 98 | prepareRelease(release, region, accountID) 99 | 100 | if err := validateClientAttributes(release); err != nil { 101 | return nil, err 102 | } 103 | 104 | return release, nil 105 | } 106 | 107 | func stateName(sd *execution.StateDetails) string { 108 | stateName := "" 109 | if sd.LastTaskName != nil { 110 | stateName = *sd.LastTaskName 111 | } else if sd.LastStateName != nil { 112 | stateName = *sd.LastStateName 113 | } 114 | return stateName 115 | } 116 | 117 | func waiter(ed *execution.Execution, sd *execution.StateDetails, err error) error { 118 | if err != nil { 119 | return fmt.Errorf("Unexpected Error %v", err.Error()) 120 | } 121 | 122 | spinnerCounter++ 123 | 124 | ws, err := waiterStr(ed.Status, sd) 125 | if err != nil { 126 | return err 127 | } 128 | 129 | fmt.Printf("\r%v ", ws) 130 | 131 | return nil 132 | } 133 | 134 | func waiterStr(status *string, sd *execution.StateDetails) (string, error) { 135 | newLine := fmt.Sprintf("%s(%s)", *status, stateName(sd)) 136 | 137 | var release models.Release 138 | if sd.LastOutput != nil { 139 | if err := json.Unmarshal([]byte(*sd.LastOutput), &release); err != nil { 140 | return "", err 141 | } 142 | } 143 | // Checks it has correctly unmarshalled 144 | if release.ProjectName != nil { 145 | if release.Error != nil { 146 | newLine = fmt.Sprintf("%v Error %v(%v)", newLine, *release.Error.Error, *release.Error.Cause) 147 | } else { 148 | sh := []string{} 149 | for name, service := range release.Services { 150 | st := serviceStr(name, service) 151 | if st != "" { 152 | sh = append(sh, st) 153 | } 154 | } 155 | if len(sh) > 0 { 156 | sort.Strings(sh) 157 | newLine = fmt.Sprintf("%v %v", newLine, strings.Join(sh, " ")) 158 | } 159 | } 160 | } 161 | 162 | return fmt.Sprintf("%v%v", spinner(), newLine), nil 163 | } 164 | 165 | var spinnerCounter = 0 166 | var spinnerChar = "/-\\|" 167 | 168 | func spinner() string { 169 | return string(spinnerChar[int(math.Mod(float64(spinnerCounter), 4))]) 170 | } 171 | 172 | func serviceStr(name string, service *models.Service) string { 173 | RED := "\x1b[0;31m" 174 | GRAY := "\x1b[1;37m" 175 | GREEN := "\x1b[0;32m" 176 | YELLOW := "\x1b[1;33m" 177 | NC := "\x1b[0m" // No Color 178 | 179 | if service.HealthReport != nil { 180 | dots := []string{} 181 | barAt := int(*service.HealthReport.TargetHealthy) 182 | // There might have been a termination now number of instances are above desired capacity 183 | numberOfDots := int(math.Max(float64(*service.HealthReport.TargetLaunched), float64(*service.HealthReport.Launching))) 184 | 185 | numberOfGreenDots := *service.HealthReport.Healthy 186 | numberOfRedDots := *service.HealthReport.Terminating 187 | numberOfYellowDots := *service.HealthReport.Launching - numberOfGreenDots - numberOfRedDots 188 | 189 | for i := 0; i < numberOfDots; i++ { 190 | if i == barAt { 191 | dots = append(dots, fmt.Sprintf("%v|%v", GRAY, NC)) 192 | } 193 | if i < numberOfGreenDots { 194 | dots = append(dots, fmt.Sprintf("%v.%v", GREEN, NC)) 195 | } else if i < (numberOfGreenDots + numberOfYellowDots) { 196 | dots = append(dots, fmt.Sprintf("%v.%v", YELLOW, NC)) 197 | } else if i < (numberOfGreenDots + numberOfYellowDots + numberOfRedDots) { 198 | dots = append(dots, fmt.Sprintf("%v.%v", RED, NC)) 199 | } else { 200 | dots = append(dots, fmt.Sprintf("%v.%v", GRAY, NC)) 201 | } 202 | } 203 | return fmt.Sprintf("%s: %v", name, strings.Join(dots, "")) 204 | } 205 | 206 | return "" 207 | } 208 | -------------------------------------------------------------------------------- /client/client_test.go: -------------------------------------------------------------------------------- 1 | package client 2 | 3 | import ( 4 | "encoding/json" 5 | "testing" 6 | 7 | "github.com/coinbase/odin/deployer/models" 8 | "github.com/coinbase/step/execution" 9 | "github.com/coinbase/step/utils/to" 10 | "github.com/stretchr/testify/assert" 11 | ) 12 | 13 | func minimalRelease(t *testing.T) *models.Release { 14 | var r models.Release 15 | err := json.Unmarshal([]byte(` 16 | { 17 | "release_id": "rr", 18 | "project_name": "project", 19 | "config_name": "config", 20 | "ami": "ami-123456", 21 | "subnets": ["subnet-1"], 22 | "services": { 23 | "web": { 24 | "instance_type": "t2.small", 25 | "security_groups": ["web-sg"] 26 | } 27 | } 28 | } 29 | `), &r) 30 | 31 | assert.NoError(t, err) 32 | return &r 33 | } 34 | 35 | func createStateDetails(release *models.Release, tn string) *execution.StateDetails { 36 | lo, _ := to.PrettyJSON((release)) 37 | return &execution.StateDetails{ 38 | LastOutput: &lo, 39 | LastTaskName: &tn, 40 | } 41 | } 42 | 43 | func waiterStrTest(t *testing.T, r *models.Release) string { 44 | spinnerCounter = 5 45 | str, err := waiterStr(to.Strp("RUNNING"), createStateDetails(r, "TaskName")) 46 | assert.NoError(t, err) 47 | return str 48 | } 49 | 50 | func Test_waiterStr(t *testing.T) { 51 | r := minimalRelease(t) 52 | assert.Equal(t, "-RUNNING(TaskName)", waiterStrTest(t, r)) 53 | 54 | r.Services["web"].HealthReport = &models.HealthReport{ 55 | TargetHealthy: to.Int64p(3), 56 | TargetLaunched: to.Int64p(5), 57 | Healthy: to.Intp(1), 58 | Launching: to.Intp(5), 59 | Terminating: to.Intp(0), 60 | } 61 | 62 | waiterStrTest(t, r) // Checks errors 63 | } 64 | -------------------------------------------------------------------------------- /client/deploy.go: -------------------------------------------------------------------------------- 1 | package client 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/aws/aws-sdk-go/service/sfn/sfniface" 7 | "github.com/coinbase/odin/aws" 8 | "github.com/coinbase/odin/deployer/models" 9 | "github.com/coinbase/step/aws/s3" 10 | "github.com/coinbase/step/execution" 11 | "github.com/coinbase/step/utils/to" 12 | ) 13 | 14 | // Deploy attempts to deploy release 15 | func Deploy(step_fn *string, releaseFile *string) error { 16 | region, accountID := to.RegionAccount() 17 | release, err := releaseFromFile(releaseFile, region, accountID) 18 | if err != nil { 19 | return err 20 | } 21 | 22 | deployerARN := to.StepArn(region, accountID, step_fn) 23 | 24 | return deploy(&aws.ClientsStr{}, release, deployerARN) 25 | } 26 | 27 | func kMSKey() *string { 28 | // TODO: allow customization of the KMS key from the command line utility 29 | return to.Strp("alias/aws/s3") 30 | } 31 | 32 | func deploy(awsc aws.Clients, release *models.Release, deployerARN *string) error { 33 | // Uploading the Release to S3 to match SHAs 34 | if err := s3.PutStruct(awsc.S3Client(nil, nil, nil), release.Bucket, release.ReleasePath(), release); err != nil { 35 | return err 36 | } 37 | 38 | // Uploading the encrypted Userdata to S3 39 | if err := s3.PutSecure(awsc.S3Client(nil, nil, nil), release.Bucket, release.UserDataPath(), release.UserData(), kMSKey()); err != nil { 40 | return err 41 | } 42 | 43 | exec, err := findOrCreateExec(awsc.SFNClient(nil, nil, nil), deployerARN, release) 44 | if err != nil { 45 | return err 46 | } 47 | 48 | // Execute every second 49 | exec.WaitForExecution(awsc.SFNClient(nil, nil, nil), 1, waiter) 50 | fmt.Println("") 51 | return nil 52 | } 53 | 54 | func findOrCreateExec(sfnc sfniface.SFNAPI, deployer *string, release *models.Release) (*execution.Execution, error) { 55 | exec, err := execution.FindExecution(sfnc, deployer, release.ExecutionPrefix()) 56 | if err != nil { 57 | return nil, err 58 | } 59 | 60 | if exec != nil { 61 | return exec, nil 62 | } 63 | 64 | return execution.StartExecution(sfnc, deployer, release.ExecutionName(), release) 65 | } 66 | -------------------------------------------------------------------------------- /client/deploy_test.go: -------------------------------------------------------------------------------- 1 | package client 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/odin/aws/mocks" 7 | "github.com/coinbase/step/utils/to" 8 | "github.com/stretchr/testify/assert" 9 | ) 10 | 11 | func Test_Deploy(t *testing.T) { 12 | awsc := mocks.MockAWS() 13 | r := minimalRelease(t) 14 | r.Release.SetDefaults(to.Strp("region"), to.Strp("accountid"), "") 15 | r.SetUserData(to.Strp("#cloud_config")) 16 | 17 | err := deploy(awsc, r, to.Strp("deployerARN")) 18 | assert.NoError(t, err) 19 | } 20 | -------------------------------------------------------------------------------- /client/fails.go: -------------------------------------------------------------------------------- 1 | package client 2 | 3 | import ( 4 | "encoding/json" 5 | "fmt" 6 | "time" 7 | 8 | "github.com/coinbase/odin/aws" 9 | "github.com/coinbase/step/bifrost" 10 | "github.com/coinbase/step/execution" 11 | "github.com/coinbase/step/utils/to" 12 | ) 13 | 14 | // Release is the Data Structure passed between Client to Deployer 15 | type FailedRelease struct { 16 | // Useful information from AWS 17 | AwsAccountID *string `json:"aws_account_id,omitempty"` 18 | AwsRegion *string `json:"aws_region,omitempty"` 19 | 20 | ProjectName *string `json:"project_name,omitempty"` 21 | ConfigName *string `json:"config_name,omitempty"` 22 | 23 | // Where the previous Catch Error should be located 24 | Error *bifrost.ReleaseError `json:"error,omitempty"` 25 | } 26 | 27 | // List the recent failures and their causes 28 | func Failures(step_fn *string) error { 29 | region, accountID := to.RegionAccount() 30 | 31 | deployerARN := to.StepArn(region, accountID, step_fn) 32 | 33 | awsc := &aws.ClientsStr{} 34 | 35 | return failures(awsc.SFNClient(nil, nil, nil), deployerARN) 36 | } 37 | 38 | func failures(sfnc aws.SFNAPI, arn *string) error { 39 | execs, err := execution.ExecutionsAfter(sfnc, arn, to.Strp("FAILED"), time.Now().Add((-3*24)*time.Hour)) 40 | 41 | if err != nil { 42 | return err 43 | } 44 | 45 | for _, e := range execs { 46 | sd, err := e.GetStateDetails(sfnc) 47 | if err != nil { 48 | return err 49 | } 50 | 51 | release := FailedRelease{} 52 | if err := json.Unmarshal([]byte(*sd.LastOutput), &release); err != nil { 53 | j, _ := to.PrettyJSON(*sd.LastOutput) 54 | fmt.Println(j) 55 | continue 56 | } 57 | 58 | cause := "" 59 | err_json := map[string]string{} 60 | 61 | if release.Error != nil { 62 | err = json.Unmarshal([]byte(*release.Error.Cause), &err_json) 63 | 64 | if err != nil { 65 | fmt.Println(err) 66 | cause = *release.Error.Cause 67 | } else { 68 | cause = err_json["errorMessage"] 69 | } 70 | } 71 | 72 | fmt.Println(fmt.Printf("%v -- %v -- %q", *sd.LastStateName, *e.Name, cause)) 73 | } 74 | 75 | return nil 76 | } 77 | -------------------------------------------------------------------------------- /client/halt.go: -------------------------------------------------------------------------------- 1 | package client 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/coinbase/odin/aws" 7 | "github.com/coinbase/odin/deployer/models" 8 | "github.com/coinbase/step/execution" 9 | "github.com/coinbase/step/utils/to" 10 | ) 11 | 12 | // Halt attempts to halt release 13 | func Halt(step_fn *string, releaseFile *string) error { 14 | region, accountID := to.RegionAccount() 15 | release, err := releaseFromFile(releaseFile, region, accountID) 16 | if err != nil { 17 | return err 18 | } 19 | 20 | deployerARN := to.StepArn(region, accountID, step_fn) 21 | 22 | return halt(&aws.ClientsStr{}, release, deployerARN) 23 | } 24 | 25 | func halt(awsc aws.Clients, release *models.Release, deployerARN *string) error { 26 | exec, err := execution.FindExecution(awsc.SFNClient(nil, nil, nil), deployerARN, release.ExecutionPrefix()) 27 | if err != nil { 28 | return err 29 | } 30 | 31 | if exec == nil { 32 | return fmt.Errorf("Cannot find current execution of release with prefix %q", release.ExecutionPrefix()) 33 | } 34 | 35 | if err := release.Halt(awsc.S3Client(nil, nil, nil), to.Strp("Odin client Halted deploy")); err != nil { 36 | return err 37 | } 38 | 39 | exec.WaitForExecution(awsc.SFNClient(nil, nil, nil), 1, waiter) 40 | fmt.Println("") 41 | return nil 42 | } 43 | -------------------------------------------------------------------------------- /client/halt_test.go: -------------------------------------------------------------------------------- 1 | package client 2 | 3 | import ( 4 | "testing" 5 | "time" 6 | 7 | "github.com/aws/aws-sdk-go/service/sfn" 8 | "github.com/coinbase/odin/aws/mocks" 9 | "github.com/coinbase/step/utils/to" 10 | "github.com/stretchr/testify/assert" 11 | ) 12 | 13 | func Test_Halt(t *testing.T) { 14 | awsc := mocks.MockAWS() 15 | r := minimalRelease(t) 16 | 17 | r.Release.SetDefaults(to.Strp("region"), to.Strp("accountid"), "") 18 | 19 | awsc.SFN.ListExecutionsResp = &sfn.ListExecutionsOutput{ 20 | Executions: []*sfn.ExecutionListItem{ 21 | &sfn.ExecutionListItem{ 22 | Name: r.ExecutionName(), 23 | ExecutionArn: to.Strp("arn"), 24 | StartDate: to.Timep(time.Now()), 25 | }, 26 | }, 27 | } 28 | 29 | err := halt(awsc, r, to.Strp("deployerARN")) 30 | assert.NoError(t, err) 31 | } 32 | -------------------------------------------------------------------------------- /deployer/fuzz_test.go: -------------------------------------------------------------------------------- 1 | package deployer 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/odin/deployer/models" 7 | "github.com/coinbase/step/utils/to" 8 | fuzz "github.com/google/gofuzz" 9 | "github.com/stretchr/testify/assert" 10 | ) 11 | 12 | func Test_Release_Basic_Fuzz(t *testing.T) { 13 | for i := 0; i < 50; i++ { 14 | f := fuzz.New() 15 | var release models.Release 16 | f.Fuzz(&release) 17 | 18 | assertNoPanic(t, &release) 19 | } 20 | } 21 | 22 | func Test_Release_Basic_Service_Fuzz(t *testing.T) { 23 | for i := 0; i < 50; i++ { 24 | f := fuzz.New() 25 | release := models.MockRelease(t) 26 | f.Fuzz(release.Services["web"]) 27 | 28 | assertNoPanic(t, release) 29 | } 30 | } 31 | 32 | func Test_Release_Basic_Autoscaling_Fuzz(t *testing.T) { 33 | for i := 0; i < 50; i++ { 34 | f := fuzz.New() 35 | release := models.MockRelease(t) 36 | f.Fuzz(release.Services["web"].Autoscaling) 37 | 38 | assertNoPanic(t, release) 39 | } 40 | } 41 | 42 | func Test_Release_Basic_Policies_Fuzz(t *testing.T) { 43 | for i := 0; i < 25; i++ { 44 | f := fuzz.New() 45 | release := models.MockRelease(t) 46 | f.Fuzz(release.Services["web"].Autoscaling.Policies[0]) 47 | release.Services["web"].Autoscaling.Policies[0].Type = to.Strp("cpu_scale_up") 48 | assertNoPanic(t, release) 49 | } 50 | 51 | for i := 0; i < 25; i++ { 52 | f := fuzz.New() 53 | release := models.MockRelease(t) 54 | f.Fuzz(release.Services["web"].Autoscaling.Policies[0]) 55 | release.Services["web"].Autoscaling.Policies[0].Type = to.Strp("cpu_scale_down") 56 | assertNoPanic(t, release) 57 | } 58 | 59 | } 60 | 61 | func Test_Release_Basic_LifeCycle_Fuzz(t *testing.T) { 62 | for i := 0; i < 50; i++ { 63 | f := fuzz.New() 64 | release := models.MockRelease(t) 65 | f.Fuzz(release.LifeCycleHooks["TermHook"]) 66 | 67 | assertNoPanic(t, release) 68 | } 69 | } 70 | 71 | func assertNoPanic(t *testing.T, release *models.Release) { 72 | release.AwsAccountID = to.Strp("0000000") 73 | stateMachine := createTestStateMachine(t, models.MockAwsClients(release)) 74 | 75 | exec, err := stateMachine.Execute(release) 76 | if err != nil { 77 | assert.NotRegexp(t, "Panic", err.Error()) 78 | } 79 | 80 | assert.NotRegexp(t, "Panic", exec.LastOutputJSON) 81 | } 82 | -------------------------------------------------------------------------------- /deployer/handlers_test.go: -------------------------------------------------------------------------------- 1 | package deployer 2 | 3 | import ( 4 | "fmt" 5 | "testing" 6 | 7 | "github.com/aws/aws-sdk-go/service/autoscaling" 8 | "github.com/coinbase/odin/aws/mocks" 9 | "github.com/coinbase/odin/deployer/models" 10 | "github.com/coinbase/step/utils/to" 11 | "github.com/stretchr/testify/assert" 12 | ) 13 | 14 | // Test that validate resources fetches the correct resources 15 | func Test_ValidateResources_FetchesCorrectResources(t *testing.T) { 16 | release := models.MockRelease(t) 17 | models.MockPrepareRelease(release) 18 | 19 | awsc := models.MockAwsClients(release) 20 | rel, err := ValidateResources(awsc)(nil, release) 21 | assert.NoError(t, err) 22 | res := rel.Services["web"].Resources 23 | assert.Equal(t, "ami-123456", *res.Image) 24 | assert.Equal(t, "/odin/project/config/web/web-profile", *res.Profile) 25 | assert.Equal(t, "project-config-web-old-release", *res.PrevASG) 26 | assert.Equal(t, []string{"group-id"}, to.StrSlice(res.SecurityGroups)) 27 | assert.Equal(t, []string{"web-elb"}, to.StrSlice(res.ELBs)) 28 | assert.Equal(t, []string{"web-elb-target"}, to.StrSlice(res.TargetGroups)) 29 | assert.Equal(t, []string{"subnet-1"}, to.StrSlice(res.Subnets)) 30 | } 31 | 32 | // Test that validate resources fails is security group does not exist, or has wrong tags 33 | func Test_ValidateResources_BadSG(t *testing.T) { 34 | release := models.MockRelease(t) 35 | models.MockPrepareRelease(release) 36 | 37 | awsc := models.MockAwsClients(release) 38 | awsc.EC2.AddSecurityGroup("web-sg", *release.ProjectName, *release.ConfigName, "noop", nil) 39 | _, err := ValidateResources(awsc)(nil, release) 40 | assert.Error(t, err) 41 | } 42 | 43 | // Test that validate resources fails if ELB or target Group has wrong tags 44 | func Test_ValidateResources_BadELB(t *testing.T) { 45 | release := models.MockRelease(t) 46 | models.MockPrepareRelease(release) 47 | 48 | awsc := models.MockAwsClients(release) 49 | awsc.ELB.AddELB("web-elb", *release.ProjectName, *release.ConfigName, "noop") 50 | _, err := ValidateResources(awsc)(nil, release) 51 | assert.Error(t, err) 52 | } 53 | 54 | // Test that validate resources fails if IAM role has wrong path 55 | func Test_ValidateResources_BadProfile(t *testing.T) { 56 | release := models.MockRelease(t) 57 | models.MockPrepareRelease(release) 58 | 59 | awsc := models.MockAwsClients(release) 60 | awsc.IAM.AddGetInstanceProfile("web-profile", fmt.Sprintf("/%v/%v/webnoop/", *release.ProjectName, *release.ConfigName)) 61 | _, err := ValidateResources(awsc)(nil, release) 62 | assert.Error(t, err) 63 | } 64 | 65 | func Test_ValidateResources_BadTG(t *testing.T) { 66 | release := models.MockRelease(t) 67 | models.MockPrepareRelease(release) 68 | 69 | awsc := models.MockAwsClients(release) 70 | awsc.ALB.AddTargetGroup(mocks.MockTargetGroup{ 71 | Name: "web-elb-target", 72 | ProjectName: *release.ProjectName, 73 | ConfigName: *release.ConfigName, 74 | ServiceName: "noop", 75 | }) 76 | _, err := ValidateResources(awsc)(nil, release) 77 | assert.Error(t, err) 78 | } 79 | 80 | func Test_ValidateResources_AllowedServiceTg(t *testing.T) { 81 | release := models.MockRelease(t) 82 | release.Services["web"].TargetGroups = []*string{to.Strp("other-project-target")} 83 | 84 | models.MockPrepareRelease(release) 85 | 86 | awsc := models.MockAwsClients(release) 87 | awsc.ALB.AddTargetGroup(mocks.MockTargetGroup{ 88 | Name: "other-project-target", 89 | ProjectName: "other/project", 90 | ServiceName: "some-service", 91 | AllowedService: "project::config::web", 92 | }) 93 | rel, err := ValidateResources(awsc)(nil, release) 94 | assert.NoError(t, err) 95 | res := rel.Services["web"].Resources 96 | assert.Equal(t, []string{"other-project-target"}, to.StrSlice(res.TargetGroups)) 97 | } 98 | 99 | // Test Check Healthy 100 | func Test_CheckHealthy_CorrectReport(t *testing.T) { 101 | release := models.MockRelease(t) 102 | models.MockPrepareRelease(release) 103 | release.Services["web"].Resources = &models.ServiceResourceNames{} 104 | release.Services["web"].CreatedASG = to.Strp("asd") 105 | 106 | awsc := mocks.MockAWS() 107 | awsc.ASG.AddASG(&autoscaling.Group{ 108 | MinSize: to.Int64p(1), 109 | DesiredCapacity: to.Int64p(1), 110 | Instances: mocks.MakeMockASGInstances(2, 3, 0), 111 | }) 112 | 113 | assert.Equal(t, false, *release.Healthy) 114 | 115 | res, err := CheckHealthy(awsc)(nil, release) 116 | assert.NoError(t, err) 117 | 118 | assert.Equal(t, true, *res.Healthy) 119 | hr := res.Services["web"].HealthReport 120 | assert.EqualValues(t, 1, *hr.TargetHealthy) 121 | assert.EqualValues(t, 1, *hr.TargetLaunched) 122 | assert.Equal(t, 2, *hr.Healthy) 123 | assert.Equal(t, 5, *hr.Launching) 124 | assert.Equal(t, 0, *hr.Terminating) 125 | } 126 | 127 | // Test Check Healthy halts if terming 128 | func Test_CheckHealthy_Terming(t *testing.T) { 129 | release := models.MockRelease(t) 130 | models.MockPrepareRelease(release) 131 | release.Services["web"].Resources = &models.ServiceResourceNames{} 132 | release.Services["web"].CreatedASG = to.Strp("asd") 133 | 134 | awsc := mocks.MockAWS() 135 | awsc.ASG.AddASG(&autoscaling.Group{ 136 | MinSize: to.Int64p(1), 137 | DesiredCapacity: to.Int64p(1), 138 | Instances: mocks.MakeMockASGInstances(2, 3, 1), 139 | }) 140 | 141 | _, err := CheckHealthy(awsc)(nil, release) 142 | assert.Error(t, err) 143 | } 144 | -------------------------------------------------------------------------------- /deployer/helpers_test.go: -------------------------------------------------------------------------------- 1 | package deployer 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/odin/aws" 7 | "github.com/coinbase/odin/aws/mocks" 8 | "github.com/coinbase/odin/deployer/models" 9 | "github.com/coinbase/step/machine" 10 | "github.com/coinbase/step/utils/to" 11 | "github.com/stretchr/testify/assert" 12 | ) 13 | 14 | func assertSuccessfulExecution(t *testing.T, release *models.Release) { 15 | awsc := models.MockAwsClients(release) 16 | 17 | assertSuccessfulExecutionWithAWS(t, release, awsc) 18 | } 19 | 20 | func assertSuccessfulExecutionWithAWS(t *testing.T, release *models.Release, awsc *mocks.MockClients) { 21 | 22 | stateMachine := createTestStateMachine(t, awsc) 23 | 24 | previousRelease := models.MockRelease(t) 25 | previousRelease.ReleaseID = to.Strp("old-release") 26 | models.AddReleaseS3Objects(awsc, previousRelease) 27 | 28 | exec, err := stateMachine.Execute(release) 29 | output := exec.Output 30 | 31 | assert.NoError(t, err) 32 | assert.Equal(t, true, output["success"]) 33 | assert.NotRegexp(t, "error", exec.LastOutputJSON) 34 | 35 | assert.Equal(t, exec.Path(), []string{ 36 | "Validate", 37 | "Lock", 38 | "ValidateResources", 39 | "Deploy", 40 | "WaitForDeploy", 41 | "WaitForHealthy", 42 | "CheckHealthy", 43 | "Healthy?", 44 | "WaitForDetach", 45 | "DetachForSuccess", 46 | "WaitDetachForSuccess", 47 | "CleanUpSuccess", 48 | "Success", 49 | }) 50 | } 51 | 52 | ////////// 53 | // CREATING THE STATE MACHINE 54 | ////////// 55 | 56 | func createTestStateMachine(t *testing.T, awsc aws.Clients) *machine.StateMachine { 57 | stateMachine, err := StateMachine() 58 | assert.NoError(t, err) 59 | 60 | err = stateMachine.SetTaskFnHandlers(CreateTaskFunctinons(awsc)) 61 | assert.NoError(t, err) 62 | 63 | return stateMachine 64 | } 65 | -------------------------------------------------------------------------------- /deployer/machine.go: -------------------------------------------------------------------------------- 1 | package deployer 2 | 3 | // StateMachine returns the StateMachine 4 | import ( 5 | "github.com/coinbase/odin/aws" 6 | "github.com/coinbase/step/handler" 7 | "github.com/coinbase/step/machine" 8 | ) 9 | 10 | // StateMachine returns 11 | func StateMachine() (*machine.StateMachine, error) { 12 | stateMachine, err := machine.FromJSON([]byte(`{ 13 | "Comment": "ASG Deployer", 14 | "StartAt": "Validate", 15 | "States": { 16 | "Validate": { 17 | "Type": "TaskFn", 18 | "Resource": "arn:aws:lambda:{{aws_region}}:{{aws_account}}:function:{{lambda_name}}", 19 | "Comment": "Validate and Set Defaults", 20 | "Next": "Lock", 21 | "Catch": [ 22 | { 23 | "Comment": "Bad Input, straight to Failure Clean, dont pass go dont collect $200", 24 | "ErrorEquals": ["States.ALL"], 25 | "ResultPath": "$.error", 26 | "Next": "FailureClean" 27 | } 28 | ] 29 | }, 30 | "Lock": { 31 | "Type": "TaskFn", 32 | "Resource": "arn:aws:lambda:{{aws_region}}:{{aws_account}}:function:{{lambda_name}}", 33 | "Comment": "Grab Lock", 34 | "Next": "ValidateResources", 35 | "Catch": [ 36 | { 37 | "Comment": "Bad Input, straight to Failure Clean", 38 | "ErrorEquals": ["LockExistsError"], 39 | "ResultPath": "$.error", 40 | "Next": "FailureClean" 41 | }, 42 | { 43 | "Comment": "Release Lock if you created it", 44 | "ErrorEquals": ["States.ALL"], 45 | "ResultPath": "$.error", 46 | "Next": "ReleaseLockFailure" 47 | } 48 | ] 49 | }, 50 | "ValidateResources": { 51 | "Type": "TaskFn", 52 | "Resource": "arn:aws:lambda:{{aws_region}}:{{aws_account}}:function:{{lambda_name}}", 53 | "Comment": "Validate Resources", 54 | "Next": "Deploy", 55 | "Catch": [ 56 | { 57 | "Comment": "Try to Release Locks", 58 | "ErrorEquals": ["States.ALL"], 59 | "ResultPath": "$.error", 60 | "Next": "ReleaseLockFailure" 61 | } 62 | ] 63 | }, 64 | "Deploy": { 65 | "Type": "TaskFn", 66 | "Resource": "arn:aws:lambda:{{aws_region}}:{{aws_account}}:function:{{lambda_name}}", 67 | "Comment": "Create Resources", 68 | "Next": "WaitForDeploy", 69 | "Catch": [ 70 | { 71 | "Comment": "Try to Release Locks", 72 | "ErrorEquals": ["HaltError"], 73 | "ResultPath": "$.error", 74 | "Next": "ReleaseLockFailure" 75 | }, 76 | { 77 | "Comment": "Try to Release Locks and Cleanup any created Resources", 78 | "ErrorEquals": ["States.ALL"], 79 | "ResultPath": "$.error", 80 | "Next": "DetachForFailure" 81 | } 82 | ] 83 | }, 84 | "WaitForDeploy": { 85 | "Comment": "Give the Deploy time to boot instances", 86 | "Type": "Wait", 87 | "Seconds" : 90, 88 | "Next": "WaitForHealthy" 89 | }, 90 | "WaitForHealthy": { 91 | "Type": "Wait", 92 | "SecondsPath" : "$.wait_for_healthy", 93 | "Next": "CheckHealthy" 94 | }, 95 | "CheckHealthy": { 96 | "Type": "TaskFn", 97 | "Resource": "arn:aws:lambda:{{aws_region}}:{{aws_account}}:function:{{lambda_name}}", 98 | "Comment": "Is the new deploy healthy? Should we continue checking? Also, scale the instances according to the strategy.", 99 | "Next": "Healthy?", 100 | "Retry": [{ 101 | "Comment": "Do not retry on HaltError", 102 | "ErrorEquals": ["HaltError"], 103 | "MaxAttempts": 0 104 | }, 105 | { 106 | "Comment": "Errors might occur, just retry a few times", 107 | "ErrorEquals": ["States.ALL"], 108 | "MaxAttempts": 3, 109 | "IntervalSeconds": 15 110 | }], 111 | "Catch": [{ 112 | "Comment": "Immediately Clean up on Error", 113 | "ErrorEquals": ["States.ALL"], 114 | "ResultPath": "$.error", 115 | "Next": "DetachForFailure" 116 | }] 117 | }, 118 | "Healthy?": { 119 | "Comment": "Check the release is $.healthy", 120 | "Type": "Choice", 121 | "Choices": [ 122 | { 123 | "Variable": "$.healthy", 124 | "BooleanEquals": true, 125 | "Next": "WaitForDetach" 126 | }, 127 | { 128 | "Variable": "$.healthy", 129 | "BooleanEquals": false, 130 | "Next": "WaitForHealthy" 131 | } 132 | ], 133 | "Default": "DetachForFailure" 134 | }, 135 | "WaitForDetach": { 136 | "Type": "Wait", 137 | "SecondsPath" : "$.wait_for_detach", 138 | "Next": "DetachForSuccess" 139 | }, 140 | "DetachForSuccess": { 141 | "Type": "TaskFn", 142 | "Resource": "arn:aws:lambda:{{aws_region}}:{{aws_account}}:function:{{lambda_name}}", 143 | "Comment": "Detach Old ASGs", 144 | "Next": "WaitDetachForSuccess", 145 | "Retry": [{ 146 | "Comment": "Retry on Detach Error, for 10 minutes", 147 | "ErrorEquals": ["DetachError"], 148 | "MaxAttempts": 60, 149 | "IntervalSeconds": 10, 150 | "BackoffRate": 1.0 151 | },{ 152 | "Comment": "Keep trying to Clean", 153 | "ErrorEquals": ["States.ALL"], 154 | "MaxAttempts": 3, 155 | "IntervalSeconds": 60 156 | }], 157 | "Catch": [{ 158 | "Comment": "Force the deletion rather than fail", 159 | "ErrorEquals": ["States.ALL"], 160 | "ResultPath": "$.error", 161 | "Next": "WaitDetachForSuccess" 162 | }] 163 | }, 164 | "WaitDetachForSuccess": { 165 | "Comment": "Give detach a little time to do what it does", 166 | "Type": "Wait", 167 | "Seconds" : 5, 168 | "Next": "CleanUpSuccess" 169 | }, 170 | "CleanUpSuccess": { 171 | "Type": "TaskFn", 172 | "Resource": "arn:aws:lambda:{{aws_region}}:{{aws_account}}:function:{{lambda_name}}", 173 | "Comment": "Promote New Resources & Delete Old Resources", 174 | "Next": "Success", 175 | "Retry": [{ 176 | "Comment": "Keep trying to Clean", 177 | "ErrorEquals": ["States.ALL"], 178 | "MaxAttempts": 3, 179 | "IntervalSeconds": 60, 180 | "BackoffRate": 1.0 181 | }], 182 | "Catch": [{ 183 | "ErrorEquals": ["States.ALL"], 184 | "ResultPath": "$.error", 185 | "Next": "FailureDirty" 186 | }] 187 | }, 188 | "DetachForFailure": { 189 | "Type": "TaskFn", 190 | "Resource": "arn:aws:lambda:{{aws_region}}:{{aws_account}}:function:{{lambda_name}}", 191 | "Comment": "Detach Old ASGs", 192 | "Next": "WaitDetachForFailure", 193 | "Retry": [{ 194 | "Comment": "Keep trying to Clean", 195 | "ErrorEquals": ["States.ALL"], 196 | "MaxAttempts": 3, 197 | "IntervalSeconds": 60 198 | }], 199 | "Catch": [{ 200 | "Comment": "Force the deletion rather than fail", 201 | "ErrorEquals": ["States.ALL"], 202 | "ResultPath": "$.error", 203 | "Next": "WaitDetachForFailure" 204 | }] 205 | }, 206 | "WaitDetachForFailure": { 207 | "Comment": "Give detach a little time to do what it does", 208 | "Type": "Wait", 209 | "Seconds" : 60, 210 | "Next": "CleanUpFailure" 211 | }, 212 | "CleanUpFailure": { 213 | "Type": "TaskFn", 214 | "Resource": "arn:aws:lambda:{{aws_region}}:{{aws_account}}:function:{{lambda_name}}", 215 | "Comment": "Delete New Resources", 216 | "Next": "ReleaseLockFailure", 217 | "Retry": [{ 218 | "Comment": "Keep trying to Clean", 219 | "ErrorEquals": ["States.ALL"], 220 | "MaxAttempts": 3, 221 | "IntervalSeconds": 30 222 | }], 223 | "Catch": [{ 224 | "ErrorEquals": ["States.ALL"], 225 | "ResultPath": "$.error", 226 | "Next": "FailureDirty" 227 | }] 228 | }, 229 | "ReleaseLockFailure": { 230 | "Type": "TaskFn", 231 | "Resource": "arn:aws:lambda:{{aws_region}}:{{aws_account}}:function:{{lambda_name}}", 232 | "Comment": "Delete New Resources", 233 | "Next": "FailureClean", 234 | "Retry": [ { 235 | "Comment": "Keep trying to Clean", 236 | "ErrorEquals": ["States.ALL"], 237 | "MaxAttempts": 3, 238 | "IntervalSeconds": 30 239 | }], 240 | "Catch": [{ 241 | "ErrorEquals": ["States.ALL"], 242 | "ResultPath": "$.error", 243 | "Next": "FailureDirty" 244 | }] 245 | }, 246 | "FailureClean": { 247 | "Comment": "Deploy Failed, but no bad resources left behind", 248 | "Type": "Fail", 249 | "Error": "FailureClean" 250 | }, 251 | "FailureDirty": { 252 | "Comment": "Deploy Failed, Resources left in Bad State, ALERT!", 253 | "Type": "Fail", 254 | "Error": "FailureDirty" 255 | }, 256 | "Success": { 257 | "Type": "Succeed" 258 | } 259 | } 260 | }`)) 261 | if err != nil { 262 | return nil, err 263 | } 264 | 265 | return stateMachine, nil 266 | } 267 | 268 | // TaskHandlers returns 269 | func TaskHandlers() *handler.TaskHandlers { 270 | return CreateTaskFunctinons(&aws.ClientsStr{}) 271 | } 272 | 273 | // CreateTaskFunctinons returns 274 | func CreateTaskFunctinons(awsc aws.Clients) *handler.TaskHandlers { 275 | tm := handler.TaskHandlers{} 276 | tm["Validate"] = Validate(awsc) 277 | tm["Lock"] = Lock(awsc) 278 | tm["ValidateResources"] = ValidateResources(awsc) 279 | tm["Deploy"] = Deploy(awsc) 280 | tm["CheckHealthy"] = CheckHealthy(awsc) 281 | 282 | // success 283 | tm["DetachForSuccess"] = DetachForSuccess(awsc) 284 | tm["CleanUpSuccess"] = CleanUpSuccess(awsc) 285 | 286 | // Failure 287 | tm["DetachForFailure"] = DetachForFailure(awsc) 288 | tm["CleanUpFailure"] = CleanUpFailure(awsc) 289 | tm["ReleaseLockFailure"] = ReleaseLockFailure(awsc) 290 | return &tm 291 | } 292 | -------------------------------------------------------------------------------- /deployer/models/autoscaling.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/coinbase/step/utils/is" 7 | "github.com/coinbase/step/utils/to" 8 | ) 9 | 10 | // AutoScalingConfig struct 11 | type AutoScalingConfig struct { 12 | MinSize *int64 `json:"min_size,omitempty"` 13 | MaxSize *int64 `json:"max_size,omitempty"` 14 | MaxTerminations *int64 `json:"max_terms,omitempty"` 15 | DefaultCooldown *int64 `json:"default_cooldown,omitempty"` 16 | HealthCheckGracePeriod *int64 `json:"health_check_grace_period,omitempty"` 17 | Spread *float64 `json:"spread,omitempty"` 18 | Policies []*Policy `json:"policies,omitempty"` 19 | 20 | Strategy *string `json:"strategy,omitempty"` 21 | } 22 | 23 | // ValidateAttributes validates attributes 24 | func (a *AutoScalingConfig) ValidateAttributes() error { 25 | if a.Strategy == nil { 26 | return fmt.Errorf("Autoscaling Strategy nil") 27 | } 28 | 29 | if !containsStr(STRATEGIES, *a.Strategy) { 30 | return fmt.Errorf("Autoscaling Strategy is %s but must be in %s", *a.Strategy, STRATEGIES) 31 | } 32 | 33 | if a.MinSize == nil { 34 | return fmt.Errorf("Autoscaling MinSize is nil") 35 | } 36 | 37 | if a.MaxSize == nil { 38 | return fmt.Errorf("Autoscaling MaxSize is nil") 39 | } 40 | 41 | if a.Spread == nil { 42 | return fmt.Errorf("Autoscaling Spread is nil") 43 | } 44 | 45 | if *a.MinSize > *a.MaxSize { 46 | return fmt.Errorf("Autoscaling MinSize is Greater than MaxSize") 47 | } 48 | 49 | if *a.Spread < 0 || *a.Spread > 1 { 50 | return fmt.Errorf("Spread must be between 0 and 1") 51 | } 52 | 53 | policyNames := []*string{} 54 | 55 | for _, p := range a.Policies { 56 | if p == nil { 57 | return fmt.Errorf("Policy nil") 58 | } 59 | 60 | if err := p.ValidateAttributes(); err != nil { 61 | return err 62 | } 63 | 64 | policyNames = append(policyNames, p.Name()) 65 | } 66 | 67 | if !is.UniqueStrp(policyNames) { 68 | return fmt.Errorf("Policy Names not Unique") 69 | } 70 | 71 | return nil 72 | } 73 | 74 | // SetDefaults assigns values 75 | func (a *AutoScalingConfig) SetDefaults(serviceID *string, timeout *int) error { 76 | 77 | if a.Strategy == nil { 78 | a.Strategy = to.Strp("AllAtOnce") 79 | } 80 | 81 | if a.MinSize == nil { 82 | a.MinSize = to.Int64p(1) 83 | } 84 | 85 | if a.MaxSize == nil { 86 | a.MaxSize = to.Int64p(1) 87 | } 88 | 89 | if a.Spread == nil { 90 | a.Spread = to.Float64p(0) 91 | } 92 | 93 | if a.MaxTerminations == nil { 94 | a.MaxTerminations = to.Int64p(0) 95 | } 96 | 97 | if a.HealthCheckGracePeriod == nil && timeout != nil { 98 | // Increase the HealthCheckGracePeriod from default to timeout if not specified 99 | // This ensures instaces are not terminated early while we are waiting for healthy status 100 | // Downside: instances might not be terminated after the deploy finished due to bad health 101 | a.HealthCheckGracePeriod = to.Int64p(int64(*timeout)) 102 | } else if a.HealthCheckGracePeriod != nil && timeout != nil { 103 | // There is no reason for HealthCheckGracePeriod to be above timeout 104 | // It could cause a successful deploy to not term unhealthy instances after deployer 105 | // For unsuccessful deploys it makes no difference 106 | a.HealthCheckGracePeriod = to.Int64p( 107 | min( 108 | *a.HealthCheckGracePeriod, 109 | int64(*timeout), 110 | )) 111 | } 112 | 113 | for _, p := range a.Policies { 114 | if p != nil { 115 | p.SetDefaults(serviceID) 116 | } 117 | } 118 | 119 | return nil 120 | } 121 | 122 | func containsStr(s []string, e string) bool { 123 | for _, a := range s { 124 | if a == e { 125 | return true 126 | } 127 | } 128 | return false 129 | } 130 | -------------------------------------------------------------------------------- /deployer/models/autoscaling_test.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/step/utils/to" 7 | "github.com/stretchr/testify/assert" 8 | ) 9 | 10 | func Test_Autoscaling_Valid(t *testing.T) { 11 | asg := &AutoScalingConfig{} 12 | asg.SetDefaults(nil, nil) 13 | assert.NoError(t, asg.ValidateAttributes()) 14 | } 15 | 16 | func Test_PolicyNames_Uniq(t *testing.T) { 17 | asg := &AutoScalingConfig{ 18 | Policies: []*Policy{ 19 | &Policy{Type: to.Strp("cpu_scale_down")}, 20 | &Policy{Type: to.Strp("cpu_scale_up")}, 21 | }, 22 | } 23 | asg.SetDefaults(to.Strp("service_id"), nil) 24 | assert.NoError(t, asg.ValidateAttributes()) 25 | 26 | asg.Policies[0].Type = to.Strp("cpu_scale_up") 27 | assert.Error(t, asg.ValidateAttributes()) 28 | 29 | asg.Policies[0].NameVal = to.Strp("override_name") 30 | assert.NoError(t, asg.ValidateAttributes()) 31 | } 32 | 33 | func Test_Autoscaling_HealthCheckGracePeriod(t *testing.T) { 34 | asg := &AutoScalingConfig{} 35 | assert.Nil(t, asg.HealthCheckGracePeriod) 36 | 37 | // Default to timeout 38 | asg.SetDefaults(nil, to.Intp(10)) 39 | assert.Equal(t, *asg.HealthCheckGracePeriod, int64(10)) 40 | 41 | // Min to timeout 42 | asg.HealthCheckGracePeriod = to.Int64p(100) 43 | asg.SetDefaults(nil, to.Intp(20)) 44 | assert.Equal(t, *asg.HealthCheckGracePeriod, int64(20)) 45 | 46 | // min to HealthCheck 47 | asg.HealthCheckGracePeriod = to.Int64p(100) 48 | asg.SetDefaults(nil, to.Intp(2000)) 49 | assert.Equal(t, *asg.HealthCheckGracePeriod, int64(100)) 50 | } 51 | -------------------------------------------------------------------------------- /deployer/models/lifecycle.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/aws/aws-sdk-go/service/autoscaling" 7 | "github.com/coinbase/odin/aws" 8 | "github.com/coinbase/odin/aws/iam" 9 | "github.com/coinbase/odin/aws/sns" 10 | "github.com/coinbase/step/utils/is" 11 | "github.com/coinbase/step/utils/to" 12 | ) 13 | 14 | // LifeCycleHook struct 15 | type LifeCycleHook struct { 16 | Transistion *string `json:"transition,omitempty"` 17 | SNS *string `json:"sns,omitempty"` 18 | Role *string `json:"role,omitempty"` 19 | HeartbeatTimeout *int64 `json:"heartbeat_timeout,omitempty"` 20 | 21 | RoleARN *string `json:"role_arn,omitempty"` 22 | NotificationTargetARN *string `json:"notification_target_arn,omitempty"` 23 | Name *string `json:"name,omitempty"` 24 | } 25 | 26 | // ToLifecycleHookSpecification returns Specification 27 | func (lc *LifeCycleHook) ToLifecycleHookSpecification() *autoscaling.LifecycleHookSpecification { 28 | return &autoscaling.LifecycleHookSpecification{ 29 | LifecycleHookName: lc.Name, 30 | HeartbeatTimeout: lc.HeartbeatTimeout, 31 | 32 | LifecycleTransition: lc.Transistion, 33 | 34 | NotificationTargetARN: lc.NotificationTargetARN, 35 | RoleARN: lc.RoleARN, 36 | } 37 | } 38 | 39 | // FetchResources validates resources exist 40 | func (lc *LifeCycleHook) FetchResources(iamc aws.IAMAPI, snsc aws.SNSAPI) error { 41 | err := iam.RoleExists(iamc, lc.Role) 42 | if err != nil { 43 | return err 44 | } 45 | 46 | if lc.SNS != nil { 47 | if err := sns.TopicExists(snsc, lc.NotificationTargetARN); err != nil { 48 | return fmt.Errorf("SNS topic does not exist %v", err.Error()) 49 | } 50 | } 51 | 52 | return nil 53 | } 54 | 55 | // SetDefaults assigns default values 56 | func (lc *LifeCycleHook) SetDefaults(region *string, accountID *string, name string) { 57 | lc.Name = to.Strp(name) 58 | 59 | if lc.Role != nil && lc.RoleARN == nil { 60 | lc.RoleARN = to.Strp(fmt.Sprintf("arn:aws:iam::%v:role/%v", *accountID, *lc.Role)) 61 | } 62 | 63 | if lc.SNS != nil && lc.NotificationTargetARN == nil { 64 | lc.NotificationTargetARN = to.Strp(fmt.Sprintf("arn:aws:sns:%v:%v:%v", *region, *accountID, *lc.SNS)) 65 | } 66 | } 67 | 68 | // ValidateAttributes validates attributes 69 | func (lc *LifeCycleHook) ValidateAttributes() error { 70 | // Quick nil check 71 | if err := lc.ToLifecycleHookSpecification().Validate(); err != nil { 72 | return err 73 | } 74 | 75 | if is.EmptyStr(lc.RoleARN) { 76 | return fmt.Errorf("Lifecycle RoleARN nil") 77 | } 78 | 79 | if is.EmptyStr(lc.NotificationTargetARN) { 80 | return fmt.Errorf("Lifecycle NotificationTargetARN nil") 81 | } 82 | 83 | if *lc.Transistion != "autoscaling:EC2_INSTANCE_LAUNCHING" && *lc.Transistion != "autoscaling:EC2_INSTANCE_TERMINATING" { 84 | return fmt.Errorf("Transistion must equal either 'autoscaling:EC2_INSTANCE_LAUNCHING' or 'autoscaling:EC2_INSTANCE_TERMINATING'") 85 | } 86 | 87 | return nil 88 | } 89 | -------------------------------------------------------------------------------- /deployer/models/lifecycle_test.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/step/utils/to" 7 | "github.com/stretchr/testify/assert" 8 | ) 9 | 10 | func Test_Lifecycle_Valid(t *testing.T) { 11 | lc := &LifeCycleHook{ 12 | Transistion: to.Strp("autoscaling:EC2_INSTANCE_LAUNCHING"), 13 | Role: to.Strp("role"), 14 | SNS: to.Strp("sns"), 15 | } 16 | 17 | lc.SetDefaults(to.Strp("region"), to.Strp("accountID"), "name") 18 | assert.NoError(t, lc.ValidateAttributes()) 19 | } 20 | -------------------------------------------------------------------------------- /deployer/models/mocks.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "encoding/json" 5 | "fmt" 6 | "testing" 7 | "time" 8 | 9 | "github.com/coinbase/odin/aws/mocks" 10 | "github.com/coinbase/step/utils/to" 11 | "github.com/stretchr/testify/assert" 12 | ) 13 | 14 | ////////// 15 | // Mock AWS Clients 16 | ////////// 17 | 18 | // MockPrepareRelease mocks 19 | func MockPrepareRelease(release *Release) { 20 | release.Release.SetDefaults(to.Strp("region"), to.Strp("account"), "") 21 | release.SetDefaults() 22 | if release.UserData() == nil { 23 | release.SetUserData(to.Strp("#cloud_config")) 24 | } 25 | 26 | release.UserDataSHA256 = to.Strp(to.SHA256Str(release.UserData())) 27 | } 28 | 29 | // MockAwsClients mocks 30 | func MockAwsClients(release *Release) *mocks.MockClients { 31 | awsc := mocks.MockAWS() 32 | 33 | if release.ProjectName != nil && release.ConfigName != nil { 34 | awsc.ASG.AddPreviousRuntimeResources(*release.ProjectName, *release.ConfigName, "web", "old-release") 35 | 36 | awsc.EC2.AddSecurityGroup("web-sg", *release.ProjectName, *release.ConfigName, "web", nil) 37 | awsc.EC2.AddImage("ubuntu", "ami-123456") 38 | awsc.EC2.AddSubnet("private-subnet", "subnet-1") 39 | 40 | awsc.ELB.AddELB("web-elb", *release.ProjectName, *release.ConfigName, "web") 41 | awsc.ALB.AddTargetGroup(mocks.MockTargetGroup{ 42 | Name: "web-elb-target", 43 | ProjectName: *release.ProjectName, 44 | ConfigName: *release.ConfigName, 45 | ServiceName: "web", 46 | }) 47 | 48 | awsc.IAM.AddGetInstanceProfile("web-profile", fmt.Sprintf("/odin/%v/%v/web/", *release.ProjectName, *release.ConfigName)) 49 | awsc.IAM.AddGetRole("sns_role") 50 | 51 | // Upload items to S3 52 | if release.ReleaseID == nil { 53 | release.ReleaseID = to.Strp("rr") 54 | } 55 | 56 | AddReleaseS3Objects(awsc, release) 57 | } 58 | 59 | return awsc 60 | } 61 | 62 | func AddReleaseS3Objects(awsc *mocks.MockClients, release *Release) { 63 | if release.UserData() == nil { 64 | release.SetUserData(to.Strp("#cloud_config")) 65 | } 66 | 67 | awsc.S3.AddGetObject(*release.UserDataPath(), *release.UserData(), nil) 68 | release.UserDataSHA256 = to.Strp(to.SHA256Str(release.UserData())) 69 | 70 | raw, _ := json.Marshal(release) 71 | awsc.S3.AddGetObject(*release.ReleasePath(), string(raw), nil) 72 | } 73 | 74 | ////////// 75 | // MockObjects 76 | ////////// 77 | 78 | // MockMinimalRelease mocks 79 | func MockMinimalRelease(t *testing.T) *Release { 80 | var r Release 81 | err := json.Unmarshal([]byte(` 82 | { 83 | "aws_account_id": "000000", 84 | "release_id": "rr", 85 | "project_name": "project", 86 | "config_name": "config", 87 | "ami": "ami-123456", 88 | "subnets": ["subnet-1"], 89 | "services": { 90 | "web": { 91 | "instance_type": "t2.small", 92 | "security_groups": ["web-sg"] 93 | } 94 | } 95 | } 96 | `), &r) 97 | 98 | assert.NoError(t, err) 99 | r.CreatedAt = to.Timep(time.Now()) 100 | 101 | return &r 102 | } 103 | 104 | // MockRelease mocks 105 | func MockRelease(t *testing.T) *Release { 106 | var r Release 107 | err := json.Unmarshal([]byte(` 108 | { 109 | "aws_account_id": "000000", 110 | "aws_region": "us-east-1", 111 | "release_id": "1", 112 | "project_name": "project", 113 | "config_name": "config", 114 | "bucket": "bucket", 115 | "ami": "ubuntu", 116 | "subnets": ["private-subnet"], 117 | "timeout": 1, 118 | "lifecycle": { 119 | "TermHook" : { 120 | "transition": "autoscaling:EC2_INSTANCE_TERMINATING", 121 | "role": "sns_role", 122 | "sns": "target", 123 | "heartbeat_timeout": 300 124 | } 125 | }, 126 | "services": { 127 | "web": { 128 | "instance_type": "t2.small", 129 | "security_groups": ["web-sg"], 130 | "elbs": ["web-elb"], 131 | "target_groups": ["web-elb-target"], 132 | "profile" : "web-profile", 133 | "ebs_volume_size": 120, 134 | "placement_group_name": "odin/project/config/moonbase-partition", 135 | "placement_group_partition_count": 5, 136 | "placement_group_strategy": "partition", 137 | "tags": { 138 | "custom": "tag" 139 | }, 140 | "autoscaling": { 141 | "min_size": 1, 142 | "max_size": 1, 143 | "max_terms": 0, 144 | "spread": 0.5, 145 | "strategy": "AllAtOnce", 146 | "default_cooldown": 10, 147 | "health_check_grace_period": 10, 148 | "policies": [ 149 | { 150 | "name": "asd", 151 | "type": "cpu_scale_up", 152 | "scaling_adjustment": 5, 153 | "threshold" : 25, 154 | "period": 2, 155 | "evaluation_periods": 10 156 | }, 157 | { 158 | "type": "cpu_scale_down", 159 | "scaling_adjustment": -1, 160 | "threshold" : 15 161 | } 162 | ] 163 | } 164 | } 165 | } 166 | } 167 | `), &r) 168 | 169 | assert.NoError(t, err) 170 | 171 | r.CreatedAt = to.Timep(time.Now()) 172 | 173 | return &r 174 | } 175 | -------------------------------------------------------------------------------- /deployer/models/policy.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/aws/aws-sdk-go/service/autoscaling" 7 | "github.com/aws/aws-sdk-go/service/cloudwatch" 8 | "github.com/coinbase/odin/aws" 9 | "github.com/coinbase/odin/aws/alarms" 10 | "github.com/coinbase/step/utils/to" 11 | ) 12 | 13 | const cpuScaleDown = "cpu_scale_down" 14 | const cpuScaleUp = "cpu_scale_up" 15 | 16 | // Policy struct 17 | type Policy struct { 18 | serviceID *string 19 | 20 | NameVal *string `json:"name,omitempty"` 21 | Type *string `json:"type,omitempty"` 22 | ScalingAdjustmentVal *int64 `json:"scaling_adjustment,omitempty"` 23 | ThresholdVal *float64 `json:"threshold,omitempty"` 24 | PeriodVal *int64 `json:"period,omitempty"` 25 | EvaluationPeriodsVal *int64 `json:"evaluation_periods,omitempty"` 26 | CooldownVal *int64 `json:"cooldown,omitempty"` 27 | } 28 | 29 | func (a *Policy) Name() *string { 30 | if a.NameVal != nil { 31 | return to.Strp(fmt.Sprintf("%v-%v-%v", *a.serviceID, *a.Type, *a.NameVal)) 32 | } 33 | 34 | return to.Strp(fmt.Sprintf("%v-%v", *a.serviceID, *a.Type)) 35 | } 36 | 37 | // ScalingAdjustment returns up or down adjustment 38 | func (a *Policy) ScalingAdjustment() *int64 { 39 | if a.ScalingAdjustmentVal != nil { 40 | return a.ScalingAdjustmentVal 41 | } 42 | 43 | switch *a.Type { 44 | case cpuScaleDown: 45 | return to.Int64p(-1) // default scale down one 46 | case cpuScaleUp: 47 | return to.Int64p(1) // default scale up one 48 | } 49 | 50 | return to.Int64p(1) 51 | } 52 | 53 | // Threshold returns threshold 54 | func (a *Policy) Threshold() *float64 { 55 | if a.ThresholdVal != nil { 56 | return a.ThresholdVal 57 | } 58 | 59 | switch *a.Type { 60 | case cpuScaleDown: 61 | return to.Float64p(20) 62 | case cpuScaleUp: 63 | return to.Float64p(50) 64 | } 65 | 66 | return to.Float64p(50) 67 | } 68 | 69 | // Period returns period 70 | func (a *Policy) Period() *int64 { 71 | if a.PeriodVal != nil { 72 | return a.PeriodVal 73 | } 74 | return to.Int64p(300) 75 | } 76 | 77 | // EvaluationPeriods returns eval periods 78 | func (a *Policy) EvaluationPeriods() *int64 { 79 | if a.EvaluationPeriodsVal != nil { 80 | return a.EvaluationPeriodsVal 81 | } 82 | 83 | return to.Int64p(2) 84 | } 85 | 86 | // Cooldown returns cooldown 87 | func (a *Policy) Cooldown() *int64 { 88 | if a.CooldownVal != nil { 89 | return a.CooldownVal 90 | } 91 | return to.Int64p(60) 92 | } 93 | 94 | // Create attempts to create alarm and policy 95 | func (a *Policy) Create(asgc aws.ASGAPI, cwc aws.CWAPI, asgName *string) error { 96 | 97 | policyInput := a.createPutScalingPolicyInput(asgName) 98 | output, err := policyInput.Create(asgc) 99 | if err != nil { 100 | return err 101 | } 102 | 103 | alarmInput := a.createMetricAlarmInput(asgName, output.PolicyARN) 104 | _, err = alarmInput.Create(cwc) 105 | 106 | if err != nil { 107 | return err 108 | } 109 | 110 | return nil 111 | } 112 | 113 | // ValidateAttributes validates attributes 114 | func (a *Policy) ValidateAttributes() error { 115 | if a.Type == nil { 116 | return fmt.Errorf("Policy(?): Type nil") 117 | } 118 | 119 | if *a.Type != cpuScaleDown && *a.Type != cpuScaleUp { 120 | return fmt.Errorf("Policy(%v): Unsupported Type %v", *a.Name(), *a.Type) 121 | } 122 | 123 | if err := a.createMetricAlarmInput(to.Strp("asgName"), nil).Validate(); err != nil { 124 | return fmt.Errorf("Policy(%v): %v", *a.Name(), err.Error()) 125 | } 126 | 127 | if err := a.createPutScalingPolicyInput(to.Strp("asgName")).Validate(); err != nil { 128 | return fmt.Errorf("Policy(%v): %v", *a.Name(), err.Error()) 129 | } 130 | 131 | return nil 132 | } 133 | 134 | // SetDefaults assigns default values 135 | func (a *Policy) SetDefaults(serviceID *string) error { 136 | a.serviceID = serviceID 137 | 138 | return nil 139 | } 140 | 141 | func (a *Policy) createMetricAlarmInput(asgName *string, policyARN *string) *alarms.AlarmInput { 142 | alarm := &alarms.AlarmInput{&cloudwatch.PutMetricAlarmInput{}} 143 | alarm.MetricName = to.Strp("CPUUtilization") 144 | alarm.Namespace = to.Strp("AWS/EC2") 145 | alarm.Statistic = to.Strp("Average") 146 | alarm.ActionsEnabled = to.Boolp(true) 147 | alarm.Period = a.Period() 148 | alarm.EvaluationPeriods = a.EvaluationPeriods() 149 | alarm.AlarmName = a.Name() 150 | alarm.Threshold = a.Threshold() 151 | alarm.Dimensions = []*cloudwatch.Dimension{ 152 | &cloudwatch.Dimension{Name: to.Strp("AutoScalingGroupName"), Value: asgName}, 153 | } 154 | 155 | if policyARN != nil { 156 | alarm.AlarmActions = []*string{policyARN} 157 | } 158 | 159 | switch *a.Type { 160 | case cpuScaleUp: 161 | alarm.ComparisonOperator = to.Strp("GreaterThanThreshold") 162 | case cpuScaleDown: 163 | alarm.ComparisonOperator = to.Strp("LessThanThreshold") 164 | } 165 | 166 | alarm.SetAlarmDescription() 167 | 168 | return alarm 169 | } 170 | 171 | func (a *Policy) createPutScalingPolicyInput(asgName *string) *alarms.PolicyInput { 172 | return &alarms.PolicyInput{&autoscaling.PutScalingPolicyInput{ 173 | AutoScalingGroupName: asgName, 174 | PolicyName: a.Name(), 175 | ScalingAdjustment: a.ScalingAdjustment(), 176 | AdjustmentType: to.Strp("ChangeInCapacity"), 177 | Cooldown: a.Cooldown(), 178 | }} 179 | } 180 | -------------------------------------------------------------------------------- /deployer/models/policy_test.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/step/utils/to" 7 | "github.com/stretchr/testify/assert" 8 | ) 9 | 10 | func Test_Policy_Valid(t *testing.T) { 11 | pol := &Policy{ 12 | Type: to.Strp("cpu_scale_down"), 13 | } 14 | 15 | pol.SetDefaults(to.Strp("service_id")) 16 | 17 | assert.NoError(t, pol.ValidateAttributes()) 18 | } 19 | 20 | func Test_Policy_Name_Prefix(t *testing.T) { 21 | pol := &Policy{ 22 | Type: to.Strp("cpu_scale_down"), 23 | } 24 | 25 | pol.SetDefaults(to.Strp("service_id")) 26 | 27 | assert.Equal(t, *pol.Name(), "service_id-cpu_scale_down") 28 | 29 | pol.NameVal = to.Strp("boom") 30 | assert.Equal(t, *pol.Name(), "service_id-cpu_scale_down-boom") 31 | } 32 | -------------------------------------------------------------------------------- /deployer/models/release.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/coinbase/odin/aws" 7 | "github.com/coinbase/step/aws/s3" 8 | "github.com/coinbase/step/bifrost" 9 | "github.com/coinbase/step/utils/is" 10 | "github.com/coinbase/step/utils/to" 11 | ) 12 | 13 | // Release is the Data Structure passed between Client to Deployer 14 | type Release struct { 15 | bifrost.Release 16 | 17 | SafeRelease bool `json:"safe_release,omitempty"` 18 | 19 | Subnets []*string `json:"subnets,omitempty"` 20 | 21 | Image *string `json:"ami,omitempty"` 22 | 23 | userdata *string // Not serialized 24 | UserDataSHA256 *string `json:"user_data_sha256,omitempty"` 25 | 26 | // LifeCycleHooks 27 | LifeCycleHooks map[string]*LifeCycleHook `json:"lifecycle,omitempty"` 28 | 29 | // Maintain a Log to look at what has happened 30 | Healthy *bool `json:"healthy,omitempty"` 31 | 32 | WaitForHealthy *int `json:"wait_for_healthy,omitempty"` 33 | 34 | // AWS Service is Downloaded 35 | Services map[string]*Service `json:"services,omitempty"` // Downloaded From S3 36 | 37 | // DetachStrategy can be "Detach"(default) | "SkipDetach" || "SkipDetachCheck" 38 | DetachStrategy *string `json:"detach_strategy,omitempty"` 39 | 40 | WaitForDetach *int `json:"wait_for_detach,omitempty"` 41 | } 42 | 43 | ////////// 44 | // Getters 45 | ////////// 46 | 47 | // UserDataPath returns 48 | func (release *Release) UserDataPath() *string { 49 | s := fmt.Sprintf("%v/userdata", *release.ReleaseDir()) 50 | return &s 51 | } 52 | 53 | ////////// 54 | // Setters 55 | ////////// 56 | 57 | // SetDefaultsWithUserData sets the default values including userdata fetched from S3 58 | func (release *Release) SetDefaultsWithUserData(s3c aws.S3API) error { 59 | release.SetDefaults() 60 | err := release.DownloadUserData(s3c) 61 | if err != nil { 62 | return err 63 | } 64 | 65 | for _, service := range release.Services { 66 | if service != nil { 67 | service.SetUserData(release.UserData()) 68 | } 69 | } 70 | 71 | return nil 72 | } 73 | 74 | // SetDefaults assigns default values 75 | func (release *Release) SetDefaults() { 76 | // Overwrite WaitForHealthy to be Min 15 seconds, Max 5 minutes 77 | waitForHealthy := 120 78 | 79 | if release.Timeout == nil { 80 | release.Timeout = to.Intp(600) 81 | } 82 | 83 | switch { 84 | case *release.Timeout < 1800: 85 | // Under 30 mins check every 15 seconds 86 | waitForHealthy = 15 87 | case *release.Timeout < 7200: 88 | // Under 2 hour check every 60 seconds 89 | waitForHealthy = 60 90 | } 91 | 92 | release.WaitForHealthy = to.Intp(waitForHealthy) 93 | 94 | if release.WaitForDetach == nil { 95 | release.WaitForDetach = to.Intp(0) 96 | } 97 | 98 | if release.Healthy == nil { 99 | release.Healthy = to.Boolp(false) 100 | } 101 | 102 | if release.LifeCycleHooks == nil { 103 | release.LifeCycleHooks = map[string]*LifeCycleHook{} 104 | } 105 | 106 | if release.DetachStrategy == nil { 107 | release.DetachStrategy = to.Strp("Detach") 108 | } 109 | 110 | for name, lc := range release.LifeCycleHooks { 111 | if lc != nil { 112 | lc.SetDefaults(release.AwsRegion, release.AwsAccountID, name) 113 | } 114 | } 115 | 116 | for name, service := range release.Services { 117 | if service != nil { 118 | service.SetDefaults(release, name) 119 | } 120 | } 121 | } 122 | 123 | ////////// 124 | // Validate 125 | ////////// 126 | 127 | // Validate returns 128 | func (release *Release) Validate(s3c aws.S3API) error { 129 | if err := release.Release.Validate(s3c, &Release{}); err != nil { 130 | return err 131 | } 132 | 133 | // Max timeout is 48 hours (for now) 134 | if *release.Timeout > 172800 { 135 | // 48 hours of timeout means the WaitForHealthy of 120 will work 136 | return fmt.Errorf("%v Max timeout is 172800 (48 hours)", release.ErrorPrefix()) 137 | } 138 | 139 | if (5.0/float64(*release.WaitForHealthy))*(float64(*release.Timeout)) > 10000.0 { 140 | // There are 5 state transitions per health check 141 | // (5/WaitForHealthy) * Timeout is about equal to the max state transistions 142 | // Due to limitations on StepFucntions History Events the max state transistions is about 10k 143 | // So (5/WaitForHealthy) * Timeout < 10k as a rule of thumb 144 | return fmt.Errorf("%v Rule of Thumb (5/WaitForHealthy) * Timeout < 10k", release.ErrorPrefix()) 145 | } 146 | 147 | // DetachStrategy 148 | if release.DetachStrategy == nil { 149 | return fmt.Errorf("%v %v", release.ErrorPrefix(), "DetachStrategy must be provided") 150 | } 151 | switch *release.DetachStrategy { 152 | case "Detach", "SkipDetach", "SkipDetachCheck": 153 | //skip 154 | default: 155 | return fmt.Errorf("%v %v", release.ErrorPrefix(), "DetachStrategy must be either 'Detach', 'SkipDetach', 'SkipDetachCheck'") 156 | } 157 | 158 | if release.Image == nil { 159 | return fmt.Errorf("%v %v", release.ErrorPrefix(), "AMI image must be provided") 160 | } 161 | 162 | if err := release.ValidateUserDataSHA(s3c); err != nil { 163 | return fmt.Errorf("%v %v", release.ErrorPrefix(), err.Error()) 164 | } 165 | 166 | if err := release.ValidateServices(); err != nil { 167 | return fmt.Errorf("%v %v", release.ErrorPrefix(), err.Error()) 168 | } 169 | 170 | return nil 171 | } 172 | 173 | // ValidateUserDataSHA validates the userdata has the correct SHA for the release 174 | func (release *Release) ValidateUserDataSHA(s3c aws.S3API) error { 175 | if is.EmptyStr(release.UserDataSHA256) { 176 | return fmt.Errorf("UserDataSHA256 must be defined") 177 | } 178 | 179 | err := release.DownloadUserData(s3c) 180 | 181 | if err != nil { 182 | return fmt.Errorf("Error Getting UserData with %v", err.Error()) 183 | } 184 | 185 | userdataSha := to.SHA256Str(release.UserData()) 186 | if userdataSha != *release.UserDataSHA256 { 187 | return fmt.Errorf("UserData SHA incorrect expected %v, got %v", userdataSha, *release.UserDataSHA256) 188 | } 189 | 190 | return nil 191 | } 192 | 193 | // UserData returns user data 194 | func (release *Release) UserData() *string { 195 | return release.userdata 196 | } 197 | 198 | // DownloadUserData fetches and populates the User data from S3 199 | func (release *Release) DownloadUserData(s3c aws.S3API) error { 200 | userdataBytes, err := s3.Get(s3c, release.Bucket, release.UserDataPath()) 201 | 202 | if err != nil { 203 | return err 204 | } 205 | 206 | release.SetUserData(to.Strp(string(*userdataBytes))) 207 | return nil 208 | } 209 | 210 | // SetUserData sets the User data 211 | func (release *Release) SetUserData(userdata *string) { 212 | release.userdata = userdata 213 | } 214 | 215 | // ValidateServices returns 216 | func (release *Release) ValidateServices() error { 217 | if release.Services == nil { 218 | return fmt.Errorf("Services nil") 219 | } 220 | 221 | if len(release.Services) == 0 { 222 | return fmt.Errorf("Services empty") 223 | } 224 | 225 | for name, service := range release.Services { 226 | if service == nil { 227 | return fmt.Errorf("Service %v is nil", name) 228 | } 229 | 230 | err := service.Validate() 231 | if err != nil { 232 | return err 233 | } 234 | } 235 | 236 | return nil 237 | } 238 | -------------------------------------------------------------------------------- /deployer/models/release_parsing.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "bytes" 5 | "encoding/json" 6 | ) 7 | 8 | // The goal here is to raise an error if a key is sent that is not supported. 9 | // This should stop many dangerous problems, like misspelling a parameter. 10 | type XRelease Release 11 | 12 | // But the problem is that there are exceptions that we have 13 | type XReleaseExceptions struct { 14 | XRelease 15 | Task *string // Do not include the Task because that can be implemented 16 | } 17 | 18 | // UnmarshalJSON should error if there is something unexpected 19 | func (release *Release) UnmarshalJSON(data []byte) error { 20 | var releaseE XReleaseExceptions 21 | dec := json.NewDecoder(bytes.NewReader(data)) 22 | dec.DisallowUnknownFields() // Force 23 | 24 | if err := dec.Decode(&releaseE); err != nil { 25 | return err 26 | } 27 | 28 | *release = Release(releaseE.XRelease) 29 | return nil 30 | } 31 | -------------------------------------------------------------------------------- /deployer/models/release_parsing_test.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "encoding/json" 5 | "testing" 6 | 7 | "github.com/stretchr/testify/assert" 8 | ) 9 | 10 | func Test_Parsing_Errors(t *testing.T) { 11 | var r Release 12 | assert.NoError(t, json.Unmarshal([]byte(`{}`), &r)) 13 | 14 | assert.NoError(t, json.Unmarshal([]byte(`{"release_id" : "1"}`), &r)) 15 | 16 | assert.Error(t, json.Unmarshal([]byte(`{"release_ids" : "1"}`), &r)) 17 | } 18 | -------------------------------------------------------------------------------- /deployer/models/release_resources_test.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/step/utils/to" 7 | "github.com/stretchr/testify/assert" 8 | ) 9 | 10 | func Test_Release_FetchResources_Works(t *testing.T) { 11 | // func (release *Release) FetchResources(asgc aws.ASGAPI, ec2 aws.EC2API, elbc aws.ELBAPI, albc aws.ALBAPI, iamc aws.IAMAPI, snsc aws.SNSAPI) (map[string]*ServiceResources, error) 12 | r := MockRelease(t) 13 | MockPrepareRelease(r) 14 | 15 | awsc := MockAwsClients(r) 16 | 17 | resources, err := r.FetchResources(awsc.ASG, awsc.EC2, awsc.ELB, awsc.ALB, awsc.IAM, awsc.SNS) 18 | assert.NoError(t, err) 19 | 20 | assert.Equal(t, 1, len(resources.ServiceResources)) 21 | } 22 | 23 | func Test_Release_ValidateResources_Works(t *testing.T) { 24 | // func (release *Release) ValidateResources(resources map[string]*ServiceResources) error { 25 | r := MockRelease(t) 26 | MockPrepareRelease(r) 27 | 28 | awsc := MockAwsClients(r) 29 | 30 | sm, err := r.FetchResources(awsc.ASG, awsc.EC2, awsc.ELB, awsc.ALB, awsc.IAM, awsc.SNS) 31 | assert.NoError(t, err) 32 | 33 | assert.NoError(t, r.ValidateResources(sm)) 34 | } 35 | 36 | func Test_Release_UpdateWithResources_Works(t *testing.T) { 37 | // func (release *Release) UpdateWithResources(resources map[string]*ServiceResources) { 38 | r := MockRelease(t) 39 | MockPrepareRelease(r) 40 | 41 | awsc := MockAwsClients(r) 42 | 43 | sm, err := r.FetchResources(awsc.ASG, awsc.EC2, awsc.ELB, awsc.ALB, awsc.IAM, awsc.SNS) 44 | assert.NoError(t, err) 45 | 46 | r.UpdateWithResources(sm) 47 | } 48 | 49 | func Test_Release_FetchResources_Stores_WaitForDetach(t *testing.T) { 50 | r := MockRelease(t) 51 | MockPrepareRelease(r) 52 | awsc := MockAwsClients(r) 53 | r.FetchResources(awsc.ASG, awsc.EC2, awsc.ELB, awsc.ALB, awsc.IAM, awsc.SNS) 54 | assert.Equal(t, 42, *r.WaitForDetach) 55 | } 56 | 57 | func Test_Release_CreateResources_Works(t *testing.T) { 58 | // func (release *Release) CreateResources(asgc aws.ASGAPI, cwc aws.CWAPI) error { 59 | r := MockRelease(t) 60 | MockPrepareRelease(r) 61 | 62 | awsc := MockAwsClients(r) 63 | assert.NoError(t, r.CreateResources(awsc.ASG, awsc.CW)) 64 | } 65 | 66 | func Test_Release_UpdateHealthy_Works(t *testing.T) { 67 | // func (release *Release) UpdateHealthy(asgc aws.ASGAPI, elbc aws.ELBAPI, albc aws.ALBAPI) error { 68 | r := MockRelease(t) 69 | MockPrepareRelease(r) 70 | 71 | awsc := MockAwsClients(r) 72 | 73 | assert.NoError(t, r.CreateResources(awsc.ASG, awsc.CW)) 74 | assert.NoError(t, r.UpdateHealthy(awsc.ASG, awsc.ELB, awsc.ALB)) 75 | } 76 | 77 | func Test_Release_SuccessfulTearDown_Works(t *testing.T) { 78 | // func (release *Release) SuccessfulTearDown(asgc aws.ASGAPI, cwc aws.CWAPI) error { 79 | r := MockRelease(t) 80 | MockPrepareRelease(r) 81 | 82 | awsc := MockAwsClients(r) 83 | assert.NoError(t, r.SuccessfulTearDown(awsc.ASG, awsc.CW)) 84 | } 85 | 86 | func Test_Release_UnsuccessfulTearDown_Works(t *testing.T) { 87 | // func (release *Release) UnsuccessfulTearDown(asgc aws.ASGAPI, cwc aws.CWAPI) error { 88 | r := MockRelease(t) 89 | MockPrepareRelease(r) 90 | 91 | awsc := MockAwsClients(r) 92 | assert.NoError(t, r.UnsuccessfulTearDown(awsc.ASG, awsc.CW)) 93 | } 94 | 95 | func Test_Release_ResetDesiredCapacity_Works(t *testing.T) { 96 | // func (release *Release) ResetDesiredCapacity(asgc aws.ASGAPI) error { 97 | r := MockRelease(t) 98 | MockPrepareRelease(r) 99 | 100 | awsc := MockAwsClients(r) 101 | s := r.Services["web"] 102 | s.CreatedASG = to.Strp("name") 103 | 104 | s.PreviousDesiredCapacity = to.Int64p(6) 105 | r.SetDefaults() 106 | 107 | a := s.Autoscaling 108 | 109 | a.MinSize = to.Int64p(int64(4)) 110 | a.MaxSize = to.Int64p(int64(10)) 111 | a.Spread = to.Float64p(float64(0.8)) 112 | 113 | r.SetDefaults() 114 | assert.NoError(t, r.ResetDesiredCapacity(awsc.ASG)) 115 | 116 | assert.Equal(t, int64(6), *awsc.ASG.UpdateAutoScalingGroupLastInput.DesiredCapacity) 117 | 118 | } 119 | -------------------------------------------------------------------------------- /deployer/models/release_safe_test.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/odin/aws/asg" 7 | "github.com/coinbase/step/utils/to" 8 | "github.com/stretchr/testify/assert" 9 | ) 10 | 11 | func Test_Release_ValidateSafeRelease_Works(t *testing.T) { 12 | release := MockRelease(t) 13 | MockPrepareRelease(release) 14 | 15 | awsc := MockAwsClients(release) 16 | 17 | previousRelease := MockRelease(t) 18 | previousRelease.ReleaseID = to.Strp("prevReleaseID") 19 | 20 | // Add release to S3 Mock 21 | AddReleaseS3Objects(awsc, previousRelease) 22 | 23 | err := release.ValidateSafeRelease(awsc.S3, &ReleaseResources{ 24 | PreviousReleaseID: previousRelease.ReleaseID, 25 | PreviousASGs: map[string]*asg.ASG{"a": nil}, 26 | ServiceResources: map[string]*ServiceResources{"a": nil}, 27 | }) 28 | 29 | assert.NoError(t, err) 30 | } 31 | 32 | func Test_Release_validateSafeRelease_Works(t *testing.T) { 33 | release := MockRelease(t) 34 | previousRelease := MockRelease(t) 35 | 36 | err := release.validateSafeRelease(previousRelease) 37 | assert.NoError(t, err) 38 | } 39 | 40 | func Test_Release_validateSafeRelease_Subnet_Image(t *testing.T) { 41 | // Subnet 42 | release := MockRelease(t) 43 | release.Subnets = []*string{to.Strp("not")} 44 | 45 | validateSafeErrorTest(t, release, "Subnet") 46 | 47 | release = MockRelease(t) 48 | release.Services = map[string]*Service{} 49 | 50 | validateSafeErrorTest(t, release, "Services") 51 | } 52 | 53 | func Test_Release_validateSafeRelease_Service(t *testing.T) { 54 | // ELB 55 | release := MockRelease(t) 56 | release.Services["web"].ELBs = []*string{to.Strp("not")} 57 | 58 | validateSafeErrorTest(t, release, "ELB") 59 | 60 | // TargetGroup 61 | release = MockRelease(t) 62 | release.Services["web"].TargetGroups = []*string{to.Strp("not")} 63 | 64 | validateSafeErrorTest(t, release, "TargetGroup") 65 | 66 | //Instance Type 67 | release = MockRelease(t) 68 | release.Services["web"].InstanceType = to.Strp("not") 69 | 70 | validateSafeErrorTest(t, release, "InstanceType") 71 | 72 | // Security Group 73 | release = MockRelease(t) 74 | release.Services["web"].SecurityGroups = []*string{to.Strp("not")} 75 | 76 | validateSafeErrorTest(t, release, "SecurityGroup") 77 | 78 | // Profile 79 | release = MockRelease(t) 80 | release.Services["web"].Profile = to.Strp("not") 81 | 82 | validateSafeErrorTest(t, release, "Profile") 83 | } 84 | 85 | func Test_Release_validateSafeRelease_Autoscaling(t *testing.T) { 86 | // Autoscaling 87 | 88 | // MinSize 89 | release := MockRelease(t) 90 | release.Services["web"].Autoscaling.MinSize = to.Int64p(64) 91 | 92 | validateSafeErrorTest(t, release, "MinSize") 93 | 94 | // MaxSize 95 | release = MockRelease(t) 96 | release.Services["web"].Autoscaling.MaxSize = to.Int64p(64) 97 | 98 | validateSafeErrorTest(t, release, "MaxSize") 99 | } 100 | 101 | func Test_Release_validateSafeRelease_MultipleErrors(t *testing.T) { 102 | // Multiple Errors 103 | release := MockRelease(t) 104 | release.Subnets = []*string{to.Strp("not")} 105 | release.Services["web"].Profile = to.Strp("not") 106 | release.Services["web"].Autoscaling.MinSize = to.Int64p(64) 107 | 108 | previousRelease := MockRelease(t) 109 | 110 | err := release.validateSafeRelease(previousRelease) 111 | assert.Error(t, err) 112 | if err != nil { 113 | assert.Regexp(t, "Subnet", err.Error()) 114 | assert.Regexp(t, "Profile", err.Error()) 115 | assert.Regexp(t, "MinSize", err.Error()) 116 | } 117 | } 118 | 119 | func Test_Release_safe_serviceMapKeys(t *testing.T) { 120 | // ELB 121 | s := serviceMapKeys(map[string]*Service{"web": nil, "angry": nil}) 122 | // Convert to string slice 123 | ss := to.StrSlice(s) 124 | 125 | assert.Equal(t, len(ss), 2) 126 | assert.Contains(t, ss, "web") 127 | assert.Contains(t, ss, "angry") 128 | } 129 | 130 | func Test_Release_ValidateSafeRelease_NewServiceWorks(t *testing.T) { 131 | release := MockRelease(t) 132 | MockPrepareRelease(release) 133 | 134 | release.Services["new_service"] = release.Services["web"] 135 | 136 | awsc := MockAwsClients(release) 137 | previousRelease := MockRelease(t) 138 | previousRelease.ReleaseID = to.Strp("prevReleaseID") 139 | 140 | // Add release to S3 Mock 141 | AddReleaseS3Objects(awsc, previousRelease) 142 | 143 | err := release.ValidateSafeRelease(awsc.S3, &ReleaseResources{ 144 | PreviousReleaseID: previousRelease.ReleaseID, 145 | PreviousASGs: map[string]*asg.ASG{"a": nil}, 146 | ServiceResources: map[string]*ServiceResources{"a": nil}, 147 | }) 148 | 149 | assert.Error(t, err) 150 | if err != nil { 151 | assert.Regexp(t, "No previous service", err.Error()) 152 | } 153 | } 154 | 155 | // Test Util 156 | func validateSafeErrorTest(t *testing.T, release *Release, errStr string) { 157 | previousRelease := MockRelease(t) 158 | 159 | err := release.validateSafeRelease(previousRelease) 160 | assert.Error(t, err) 161 | if err != nil { 162 | assert.Regexp(t, errStr, err.Error()) 163 | } 164 | } 165 | -------------------------------------------------------------------------------- /deployer/models/release_test.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/step/utils/to" 7 | "github.com/stretchr/testify/assert" 8 | ) 9 | 10 | func Test_Release_Validate_Works(t *testing.T) { 11 | r := MockRelease(t) 12 | awsc := MockAwsClients(r) 13 | r.ReleaseSHA256 = to.SHA256Struct(r) 14 | 15 | MockPrepareRelease(r) 16 | 17 | assert.NoError(t, r.Validate(awsc.S3)) 18 | } 19 | 20 | func Test_Release_ValidateServices_Works(t *testing.T) { 21 | r := MockRelease(t) 22 | MockPrepareRelease(r) 23 | 24 | assert.NoError(t, r.ValidateServices()) 25 | } 26 | 27 | func Test_SetDefaults_Sets_WaitForHealthy(t *testing.T) { 28 | r := MockRelease(t) 29 | MockPrepareRelease(r) 30 | 31 | assert.Equal(t, 15, *r.WaitForHealthy) 32 | 33 | r = MockRelease(t) 34 | r.Timeout = to.Intp(3600) 35 | MockPrepareRelease(r) 36 | assert.Equal(t, 60, *r.WaitForHealthy) 37 | 38 | r = MockRelease(t) 39 | r.Timeout = to.Intp(8000) 40 | MockPrepareRelease(r) 41 | assert.Equal(t, 120, *r.WaitForHealthy) 42 | } 43 | -------------------------------------------------------------------------------- /deployer/models/service_resources_test.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/coinbase/odin/aws/alb" 7 | "github.com/coinbase/odin/aws/ami" 8 | "github.com/coinbase/odin/aws/asg" 9 | "github.com/coinbase/odin/aws/elb" 10 | "github.com/coinbase/odin/aws/iam" 11 | "github.com/coinbase/odin/aws/sg" 12 | "github.com/coinbase/odin/aws/subnet" 13 | "github.com/coinbase/step/utils/to" 14 | "github.com/stretchr/testify/assert" 15 | ) 16 | 17 | type MockService struct{} 18 | 19 | func (*MockService) ProjectName() *string { 20 | return to.Strp("project") 21 | } 22 | 23 | func (*MockService) ConfigName() *string { 24 | return to.Strp("config") 25 | } 26 | 27 | func (*MockService) Name() *string { 28 | return to.Strp("servicename") 29 | } 30 | 31 | func (*MockService) ReleaseID() *string { 32 | return to.Strp("releaseid") 33 | } 34 | 35 | func Test_Service_ValidateImage(t *testing.T) { 36 | // func ValidateImage(service *Service, im *ami.Image) error { 37 | assert.Error(t, ValidateImage(&MockService{}, &ami.Image{ImageID: to.Strp("image")})) 38 | assert.NoError(t, ValidateImage(&MockService{}, &ami.Image{ 39 | ImageID: to.Strp("image"), 40 | DeployWithTag: to.Strp("odin"), 41 | })) 42 | } 43 | 44 | func Test_Service_ValidateSubnet(t *testing.T) { 45 | // func ValidateSubnet(service *Service, subnet *subnet.Subnet) error { 46 | assert.Error(t, ValidateSubnet(&MockService{}, &subnet.Subnet{SubnetID: to.Strp("subnet")})) 47 | 48 | assert.NoError(t, ValidateSubnet(&MockService{}, &subnet.Subnet{ 49 | SubnetID: to.Strp("subnet"), 50 | DeployWithTag: to.Strp("odin"), 51 | })) 52 | } 53 | 54 | func Test_Service_ValidatePrevASG(t *testing.T) { 55 | // func ValidatePrevASG(service *Service, as *asg.ASG) error { 56 | assert.Error(t, ValidatePrevASG(&MockService{}, &asg.ASG{})) 57 | 58 | // Project Name 59 | assert.Error(t, ValidatePrevASG(&MockService{}, &asg.ASG{ 60 | ProjectNameTag: to.Strp("notproject"), 61 | ConfigNameTag: to.Strp("config"), 62 | ServiceNameTag: to.Strp("servicename"), 63 | ReleaseIDTag: to.Strp("new_releaseid"), 64 | })) 65 | 66 | // Config Name 67 | assert.Error(t, ValidatePrevASG(&MockService{}, &asg.ASG{ 68 | ProjectNameTag: to.Strp("project"), 69 | ConfigNameTag: to.Strp("notconfig"), 70 | ServiceNameTag: to.Strp("servicename"), 71 | ReleaseIDTag: to.Strp("new_releaseid"), 72 | })) 73 | // Service Name 74 | assert.Error(t, ValidatePrevASG(&MockService{}, &asg.ASG{ 75 | ProjectNameTag: to.Strp("project"), 76 | ConfigNameTag: to.Strp("config"), 77 | ServiceNameTag: to.Strp("notservicename"), 78 | ReleaseIDTag: to.Strp("new_releaseid"), 79 | })) 80 | // ReleaseID the same 81 | assert.Error(t, ValidatePrevASG(&MockService{}, &asg.ASG{ 82 | ProjectNameTag: to.Strp("project"), 83 | ConfigNameTag: to.Strp("config"), 84 | ServiceNameTag: to.Strp("servicename"), 85 | ReleaseIDTag: to.Strp("releaseid"), 86 | })) 87 | 88 | assert.NoError(t, ValidatePrevASG(&MockService{}, &asg.ASG{ 89 | ProjectNameTag: to.Strp("project"), 90 | ConfigNameTag: to.Strp("config"), 91 | ServiceNameTag: to.Strp("servicename"), 92 | ReleaseIDTag: to.Strp("new_releaseid"), 93 | })) 94 | } 95 | 96 | func Test_Service_ValidateIAMProfile(t *testing.T) { 97 | // func ValidateIAMProfile(service *Service, profile *iam.Profile) error { 98 | assert.Error(t, ValidateIAMProfile(&MockService{}, &iam.Profile{})) 99 | 100 | assert.NoError(t, ValidateIAMProfile(&MockService{}, &iam.Profile{ 101 | Path: to.Strp("/odin/project/config/servicename/"), 102 | })) 103 | 104 | assert.NoError(t, ValidateIAMProfile(&MockService{}, &iam.Profile{ 105 | Path: to.Strp("/odin/project/config/_all/"), 106 | })) 107 | 108 | assert.NoError(t, ValidateIAMProfile(&MockService{}, &iam.Profile{ 109 | Path: to.Strp("/odin/project/_all/_all/"), 110 | })) 111 | 112 | assert.NoError(t, ValidateIAMProfile(&MockService{}, &iam.Profile{ 113 | Path: to.Strp("/odin/_all/_all/_all/"), 114 | })) 115 | 116 | assert.Error(t, ValidateIAMProfile(&MockService{}, &iam.Profile{ 117 | Path: to.Strp("/notodin/_all/_all/_all/"), 118 | })) 119 | } 120 | 121 | func Test_Service_ValidateSecurityGroup(t *testing.T) { 122 | // func ValidateSecurityGroup(service *Service, sc *sg.SecurityGroup) error { 123 | assert.Error(t, ValidateSecurityGroup(&MockService{}, &sg.SecurityGroup{})) 124 | 125 | // Project Name 126 | assert.Error(t, ValidateSecurityGroup(&MockService{}, &sg.SecurityGroup{ 127 | ProjectNameTag: to.Strp("notproject"), 128 | ConfigNameTag: to.Strp("config"), 129 | ServiceNameTag: to.Strp("servicename"), 130 | })) 131 | 132 | // Config Name 133 | assert.Error(t, ValidateSecurityGroup(&MockService{}, &sg.SecurityGroup{ 134 | ProjectNameTag: to.Strp("project"), 135 | ConfigNameTag: to.Strp("notconfig"), 136 | ServiceNameTag: to.Strp("servicename"), 137 | })) 138 | 139 | // Service Name 140 | assert.Error(t, ValidateSecurityGroup(&MockService{}, &sg.SecurityGroup{ 141 | ProjectNameTag: to.Strp("project"), 142 | ConfigNameTag: to.Strp("config"), 143 | ServiceNameTag: to.Strp("notservicename"), 144 | })) 145 | 146 | assert.NoError(t, ValidateSecurityGroup(&MockService{}, &sg.SecurityGroup{ 147 | ProjectNameTag: to.Strp("project"), 148 | ConfigNameTag: to.Strp("config"), 149 | ServiceNameTag: to.Strp("servicename"), 150 | })) 151 | 152 | assert.NoError(t, ValidateSecurityGroup(&MockService{}, &sg.SecurityGroup{ 153 | ProjectNameTag: to.Strp("_all"), 154 | ConfigNameTag: to.Strp("config"), 155 | ServiceNameTag: to.Strp("servicename"), 156 | })) 157 | 158 | assert.NoError(t, ValidateSecurityGroup(&MockService{}, &sg.SecurityGroup{ 159 | ProjectNameTag: to.Strp("_all"), 160 | ConfigNameTag: to.Strp("config"), 161 | ServiceNameTag: to.Strp("servicename"), 162 | })) 163 | 164 | assert.NoError(t, ValidateSecurityGroup(&MockService{}, &sg.SecurityGroup{ 165 | ProjectNameTag: to.Strp("_all"), 166 | ConfigNameTag: to.Strp("_all"), 167 | ServiceNameTag: to.Strp("_all"), 168 | })) 169 | } 170 | 171 | func Test_Service_ValidateELB(t *testing.T) { 172 | // func ValidateELB(service *Service, lb *elb.LoadBalancer) error { 173 | assert.Error(t, ValidateELB(&MockService{}, &elb.LoadBalancer{})) 174 | 175 | // Project Name 176 | assert.Error(t, ValidateELB(&MockService{}, &elb.LoadBalancer{ 177 | ProjectNameTag: to.Strp("notproject"), 178 | ConfigNameTag: to.Strp("config"), 179 | })) 180 | 181 | // Config Name 182 | assert.Error(t, ValidateELB(&MockService{}, &elb.LoadBalancer{ 183 | ProjectNameTag: to.Strp("project"), 184 | ConfigNameTag: to.Strp("notconfig"), 185 | })) 186 | 187 | assert.NoError(t, ValidateELB(&MockService{}, &elb.LoadBalancer{ 188 | ProjectNameTag: to.Strp("project"), 189 | ConfigNameTag: to.Strp("config"), 190 | ServiceNameTag: to.Strp("servicename"), 191 | })) 192 | } 193 | 194 | func Test_Service_ValidateTargetGroup(t *testing.T) { 195 | // func ValidateTargetGroup(service *Service, tg *alb.TargetGroup) error { 196 | assert.Error(t, ValidateTargetGroup(&MockService{}, &alb.TargetGroup{})) 197 | 198 | // Project Name 199 | assert.Error(t, ValidateTargetGroup(&MockService{}, &alb.TargetGroup{ 200 | ProjectNameTag: to.Strp("notproject"), 201 | ConfigNameTag: to.Strp("config"), 202 | ServiceNameTag: to.Strp("servicename"), 203 | })) 204 | 205 | // Config Name 206 | assert.Error(t, ValidateTargetGroup(&MockService{}, &alb.TargetGroup{ 207 | ProjectNameTag: to.Strp("project"), 208 | ConfigNameTag: to.Strp("notconfig"), 209 | ServiceNameTag: to.Strp("servicename"), 210 | })) 211 | 212 | // Service Name 213 | assert.Error(t, ValidateTargetGroup(&MockService{}, &alb.TargetGroup{ 214 | ProjectNameTag: to.Strp("project"), 215 | ConfigNameTag: to.Strp("config"), 216 | ServiceNameTag: to.Strp("notservicename"), 217 | })) 218 | 219 | assert.NoError(t, ValidateTargetGroup(&MockService{}, &alb.TargetGroup{ 220 | ProjectNameTag: to.Strp("project"), 221 | ConfigNameTag: to.Strp("config"), 222 | ServiceNameTag: to.Strp("servicename"), 223 | })) 224 | } 225 | -------------------------------------------------------------------------------- /deployer/models/service_test.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "fmt" 5 | "testing" 6 | 7 | "github.com/coinbase/odin/aws/asg" 8 | "github.com/coinbase/odin/aws/mocks" 9 | "github.com/coinbase/step/utils/to" 10 | "github.com/stretchr/testify/assert" 11 | ) 12 | 13 | func Test_Service_SetGetUserdata(t *testing.T) { 14 | release := MockMinimalRelease(t) 15 | 16 | service := Service{} 17 | service.SetUserData(to.Strp("{{RELEASE_ID}}\n{{PROJECT_NAME}}\n{{CONFIG_NAME}}\n{{SERVICE_NAME}}\n")) 18 | service.SetDefaults(release, "web") 19 | 20 | assert.Equal(t, fmt.Sprintf("%v\n%v\n%v\nweb\n", *release.ReleaseID, *release.ProjectName, *release.ConfigName), *service.UserData()) 21 | } 22 | 23 | func Test_Service_CreateInput_HealthCheckGracePeriod(t *testing.T) { 24 | release := MockMinimalRelease(t) 25 | release.Timeout = to.Intp(10) 26 | 27 | service := Service{} 28 | service.SetUserData(to.Strp("{{RELEASE_ID}}\n{{PROJECT_NAME}}\n{{CONFIG_NAME}}\n{{SERVICE_NAME}}\n")) 29 | service.SetDefaults(release, "web") 30 | 31 | input := service.createInput() 32 | assert.Equal(t, *input.HealthCheckGracePeriod, int64(10)) 33 | } 34 | 35 | func Test_Service_PlacementgroupValidation(t *testing.T) { 36 | // bad strat 37 | service := Service{ 38 | PlacementGroupName: to.Strp("asd"), 39 | PlacementGroupStrategy: to.Strp("asd"), 40 | } 41 | 42 | err := service.validatePlacementGroupAttributes() 43 | assert.Error(t, err) 44 | assert.Contains(t, err.Error(), "PlacementGroupStrategy") 45 | 46 | // need PartitionCount 47 | service = Service{ 48 | PlacementGroupName: to.Strp("asd"), 49 | PlacementGroupStrategy: to.Strp("partition"), 50 | } 51 | 52 | err = service.validatePlacementGroupAttributes() 53 | assert.Error(t, err) 54 | assert.Contains(t, err.Error(), "PlacementGroupPartitionCount") 55 | 56 | // need only if partitionPartitionCount 57 | service = Service{ 58 | PlacementGroupName: to.Strp("asd"), 59 | PlacementGroupStrategy: to.Strp("spread"), 60 | PlacementGroupPartitionCount: to.Int64p(10), 61 | } 62 | 63 | err = service.validatePlacementGroupAttributes() 64 | assert.Error(t, err) 65 | assert.Contains(t, err.Error(), "PlacementGroupPartitionCount") 66 | 67 | // need PartitionCount 68 | service = Service{ 69 | PlacementGroupName: to.Strp("asd"), 70 | PlacementGroupStrategy: to.Strp("partition"), 71 | PlacementGroupPartitionCount: to.Int64p(10), 72 | } 73 | 74 | err = service.validatePlacementGroupAttributes() 75 | assert.NoError(t, err) 76 | 77 | } 78 | 79 | func Test_Service_ResetDesiredCapacity_Works(t *testing.T) { 80 | service := &Service{ 81 | Autoscaling: &AutoScalingConfig{ 82 | MinSize: to.Int64p(int64(4)), 83 | MaxSize: to.Int64p(int64(10)), 84 | Spread: to.Float64p(float64(0.8)), 85 | }, 86 | PreviousDesiredCapacity: to.Int64p(6), 87 | } 88 | 89 | service.SetDefaults(&Release{}, "asd") 90 | 91 | awsc := mocks.MockAWS() 92 | 93 | assert.NoError(t, service.ResetDesiredCapacity(awsc.ASG)) 94 | assert.Equal(t, int64(6), *awsc.ASG.UpdateAutoScalingGroupLastInput.DesiredCapacity) 95 | } 96 | 97 | func Test_Service_CapacityValues(t *testing.T) { 98 | service := &Service{ 99 | Autoscaling: &AutoScalingConfig{ 100 | MinSize: to.Int64p(int64(10)), 101 | MaxSize: to.Int64p(int64(50)), 102 | Spread: to.Float64p(float64(0.8)), 103 | }, 104 | PreviousDesiredCapacity: to.Int64p(20), 105 | } 106 | 107 | service.SetDefaults(&Release{}, "asd") 108 | 109 | assert.EqualValues(t, 10, service.strategy.TargetHealthy()) // The number of instances we want healthy 110 | assert.EqualValues(t, 20, service.strategy.DesiredCapacity()) // The final number of instances 111 | assert.EqualValues(t, 36, service.strategy.TargetCapacity()) // The number of launched instances 112 | } 113 | 114 | func Test_Service_SafeSetMinDesiredCapacity_Works(t *testing.T) { 115 | awsc := mocks.MockAWS() 116 | service := &Service{} 117 | group := &asg.ASG{MinSize: to.Int64p(2), DesiredCapacity: to.Int64p(2)} 118 | 119 | // if min and dc are the same dont call 120 | assert.NoError(t, service.SafeSetMinDesiredCapacity(awsc.ASG, group, 2, 2)) 121 | assert.Nil(t, awsc.ASG.UpdateAutoScalingGroupLastInput) 122 | 123 | // if min and dc are the same dont call 124 | assert.NoError(t, service.SafeSetMinDesiredCapacity(awsc.ASG, group, 1, 1)) 125 | assert.Nil(t, awsc.ASG.UpdateAutoScalingGroupLastInput) 126 | 127 | // When called asks for the correct values 128 | assert.NoError(t, service.SafeSetMinDesiredCapacity(awsc.ASG, group, 2, 3)) 129 | assert.Equal(t, int64(3), *awsc.ASG.UpdateAutoScalingGroupLastInput.DesiredCapacity) 130 | assert.Equal(t, int64(2), *awsc.ASG.UpdateAutoScalingGroupLastInput.MinSize) 131 | } 132 | -------------------------------------------------------------------------------- /deployer/models/strategy.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import ( 4 | "github.com/coinbase/odin/aws" 5 | "github.com/coinbase/step/utils/to" 6 | ) 7 | 8 | var STRATEGIES = []string{ 9 | "AllAtOnce", 10 | "OneThenAllWithCanary", 11 | "25PercentStepRolloutNoCanary", 12 | "10PercentStepRolloutNoCanary", 13 | "10AtATimeNoCanary", 14 | "20AtATimeNoCanary", 15 | } 16 | 17 | type StrategyType string 18 | 19 | const ( 20 | AllAtOnce StrategyType = "AllAtOnce" 21 | Canary = "Canary" 22 | Percent = "Percent" 23 | Increment = "Increment" 24 | ) 25 | 26 | func NewStrategy(autoscaling *AutoScalingConfig, previousDesiredCapacity *int64) *Strategy { 27 | // Defaults 28 | s := &Strategy{ 29 | name: *autoscaling.Strategy, 30 | sType: AllAtOnce, 31 | minSize: int64(1), 32 | maxSize: int64(1), 33 | maxTerminations: int64(0), 34 | spread: 0.2, 35 | previousDesiredCapacity: previousDesiredCapacity, 36 | } 37 | 38 | // Get Info from autoscaling 39 | if autoscaling.Spread != nil { 40 | s.spread = *autoscaling.Spread 41 | } 42 | 43 | if autoscaling.MinSize != nil { 44 | s.minSize = *autoscaling.MinSize 45 | } 46 | 47 | if autoscaling.MaxSize != nil { 48 | s.maxSize = *autoscaling.MaxSize 49 | } 50 | 51 | if autoscaling.MaxTerminations != nil { 52 | s.maxTerminations = *autoscaling.MaxTerminations 53 | } 54 | 55 | // Define the Strategy properties 56 | switch s.name { 57 | case "OneThenAllWithCanary": 58 | s.sType = Canary 59 | case "25PercentStepRolloutNoCanary": 60 | s.sType = Percent 61 | // 25% means release is divided into 4 steps 62 | s.rollOutSteps = 4 63 | case "10PercentStepRolloutNoCanary": 64 | s.sType = Percent 65 | // 10% means release divided into 10 stages 66 | s.rollOutSteps = 10 67 | case "10AtATimeNoCanary": 68 | s.sType = Increment 69 | s.rollOutSteps = float64(s.TargetCapacity()) / float64(10) 70 | case "20AtATimeNoCanary": 71 | s.sType = Increment 72 | s.rollOutSteps = float64(s.TargetCapacity()) / float64(20) 73 | } 74 | 75 | return s 76 | } 77 | 78 | // Strategy describes the way in which Odin brings up instances in an Autoscaling Group 79 | // pulling it out into this struct helps isolate code from the rest of the service 80 | type Strategy struct { 81 | name string 82 | sType StrategyType 83 | minSize int64 84 | maxSize int64 85 | maxTerminations int64 86 | spread float64 87 | previousDesiredCapacity *int64 // This can be nil 88 | 89 | // For Percent and Increment types 90 | // This is the number of steps used to rollout all instances 91 | rollOutSteps float64 92 | } 93 | 94 | //// 95 | // Health Report Methods 96 | //// 97 | 98 | // TargetCapacity is the number of launched instances including the spread 99 | func (strategy *Strategy) TargetCapacity() int64 { 100 | maxSize := strategy.maxSize 101 | dc := strategy.DesiredCapacity() 102 | spread := strategy.spread 103 | 104 | tc := percent(dc, (1 + spread)) 105 | return min(maxSize, tc) 106 | } 107 | 108 | // TargetHealthy is the number of instances the service needs to be Healthy 109 | func (strategy *Strategy) TargetHealthy() int64 { 110 | minSize := strategy.minSize 111 | dc := strategy.DesiredCapacity() 112 | spread := strategy.spread 113 | th := percent(dc, (1 - spread)) 114 | 115 | return max(minSize, th) 116 | } 117 | 118 | // DesiredCapacity is the REAL amount of instances we want. 119 | // This is later altered for practicality by spread 120 | func (strategy *Strategy) DesiredCapacity() int64 { 121 | minSize := strategy.minSize 122 | maxSize := strategy.maxSize 123 | previousDesiredCapacity := strategy.previousDesiredCapacity 124 | pc := int64(-1) 125 | if previousDesiredCapacity != nil { 126 | pc = *previousDesiredCapacity 127 | } 128 | 129 | // Scale down desired capacity if new max is lower than the previous dc 130 | desiredCapacity := min(pc, maxSize) 131 | // Scale up desired capacity, if the new min is higher than the previous dc 132 | return max(desiredCapacity, minSize) 133 | } 134 | 135 | //// 136 | // Init Methods 137 | //// 138 | 139 | func (strategy *Strategy) InitialMinSize() *int64 { 140 | switch strategy.sType { 141 | case Canary: 142 | // "OneThenAllWithCanary" starts with 1 143 | return to.Int64p(1) 144 | case Percent, Increment: 145 | // no instances yet 146 | return to.Int64p(fastRolloutRate(0, strategy.minSize, strategy.rollOutSteps)) 147 | } 148 | 149 | // default case "AllAtOnce" is minSize 150 | return &strategy.minSize 151 | } 152 | 153 | func (strategy *Strategy) InitialDesiredCapacity() *int64 { 154 | switch strategy.sType { 155 | case Canary: 156 | // "OneThenAllWithCanary" starts with 1 157 | return to.Int64p(1) 158 | case Percent, Increment: 159 | return to.Int64p(fastRolloutRate(0, strategy.TargetCapacity(), strategy.rollOutSteps)) 160 | } 161 | 162 | // default case "AllAtOnce" is target capacity 163 | return to.Int64p(strategy.TargetCapacity()) 164 | } 165 | 166 | //// 167 | // Flow Methods 168 | //// 169 | 170 | func (strategy *Strategy) ReachedMaxTerminations(instances aws.Instances) bool { 171 | maxTermingInstances := strategy.maxTerminations 172 | 173 | switch strategy.sType { 174 | case Canary: 175 | // OneThenAllWithCanary during the canary it will exit if one terminates, otherwise default 176 | canarying := len(instances) <= 1 177 | if canarying { 178 | maxTermingInstances = 0 179 | } 180 | } 181 | 182 | // Non Canaries just use maxTerms by default 183 | // If there are more terminating instances than allowed return true 184 | return int64(len(instances.TerminatingIDs())) > maxTermingInstances 185 | } 186 | 187 | func (strategy *Strategy) CalculateMinDesired(instances aws.Instances) (int64, int64) { 188 | switch strategy.sType { 189 | case Canary: 190 | // "OneThenAllWithCanary" if there is only one instance and it is healthy proceed 191 | canarying := len(instances) <= 1 192 | if !canarying { 193 | break 194 | } // Only continue if canarying 195 | 196 | canaryIsHealthy := len(instances.HealthyIDs()) == 1 197 | if canaryIsHealthy { 198 | break 199 | } // return default amounts if the canary is Healthy 200 | 201 | return 1, 1 202 | case Percent, Increment: 203 | // Percent will continually add 1/strategy.rollOutSteps additional instances to those that are launching 204 | // until InitialMinSize and InitialDesiredCapacity 205 | minSize := fastRolloutRate(len(instances), strategy.minSize, strategy.rollOutSteps) 206 | dc := fastRolloutRate(len(instances), strategy.TargetCapacity(), strategy.rollOutSteps) 207 | return minSize, dc 208 | } 209 | 210 | // default case "AllAtOnce" is init values 211 | return strategy.minSize, strategy.TargetCapacity() 212 | } 213 | 214 | //// 215 | // MATH 216 | //// 217 | 218 | func min(x int64, y int64) int64 { 219 | if x < y { 220 | return x 221 | } 222 | return y 223 | } 224 | 225 | func max(x int64, y int64) int64 { 226 | if x > y { 227 | return x 228 | } 229 | return y 230 | } 231 | 232 | func percent(x int64, percent float64) int64 { 233 | return (x * int64(percent*100)) / 100 234 | } 235 | 236 | // 25PercentStepRolloutNoCanary and 10PercentStepRolloutNoCanary 237 | 238 | func fastRolloutRate(instanceCount int, baseAmount int64, denominator float64) int64 { 239 | // 1. Always return greater than 1 240 | // 2. Always return less than baseAmount 241 | // 3. return the instanceCount + 1/4 the baseAmount 242 | 243 | // find the additional amount, always return more than 1 244 | additionalInstances := max(1, int64(float64(baseAmount)/denominator)) 245 | 246 | // core return value 247 | amount := int64(instanceCount) + additionalInstances 248 | 249 | // Always return greater than 1, and less than baseAmount 250 | return max(1, min(amount, baseAmount)) 251 | } 252 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/coinbase/odin 2 | 3 | require ( 4 | github.com/aws/aws-lambda-go v1.17.0 5 | github.com/aws/aws-sdk-go v1.31.9 6 | github.com/coinbase/step v1.0.2 7 | github.com/davecgh/go-spew v1.1.1 8 | github.com/google/gofuzz v0.0.0-20170612174753-24818f796faf 9 | github.com/jmespath/go-jmespath v0.3.0 10 | github.com/pmezard/go-difflib v1.0.0 11 | github.com/stretchr/testify v1.5.1 12 | ) 13 | 14 | go 1.13 15 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= 2 | github.com/DATA-DOG/go-sqlmock v1.4.1/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM= 3 | github.com/DataDog/datadog-lambda-go v0.6.0 h1://2QePQGtIQAyFbsv/Bew4EX8VVBUaXltPyxp7rHkZo= 4 | github.com/DataDog/datadog-lambda-go v0.6.0/go.mod h1:8IH+3AngDt+on4Fc7qeFAxj2h6oPuIgsXs5lEPFImto= 5 | github.com/aws/aws-lambda-go v1.8.0 h1:YMCzi9FP7MNVVj9AkGpYyaqh/mvFOjhqiDtnNlWtKTg= 6 | github.com/aws/aws-lambda-go v1.8.0/go.mod h1:zUsUQhAUjYzR8AuduJPCfhBuKWUaDbQiPOG+ouzmE1A= 7 | github.com/aws/aws-lambda-go v1.11.1 h1:wuOnhS5aqzPOWns71FO35PtbtBKHr4MYsPVt5qXLSfI= 8 | github.com/aws/aws-lambda-go v1.11.1/go.mod h1:Rr2SMTLeSMKgD45uep9V/NP8tnbCcySgu04cx0k/6cw= 9 | github.com/aws/aws-lambda-go v1.17.0 h1:Ogihmi8BnpmCNktKAGpNwSiILNNING1MiosnKUfU8m0= 10 | github.com/aws/aws-lambda-go v1.17.0/go.mod h1:FEwgPLE6+8wcGBTe5cJN3JWurd1Ztm9zN4jsXsjzKKw= 11 | github.com/aws/aws-sdk-go v1.15.90/go.mod h1:es1KtYUFs7le0xQ3rOihkuoVD90z7D0fR2Qm4S00/gU= 12 | github.com/aws/aws-sdk-go v1.16.3 h1:esEQzoR8SVXtwg42nRoR/YLftI4ktsZg6Qwr7jnDXy8= 13 | github.com/aws/aws-sdk-go v1.16.3/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= 14 | github.com/aws/aws-sdk-go v1.17.12/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= 15 | github.com/aws/aws-sdk-go v1.20.2 h1:/BBeW8F4PPmvJ5jpFvgkCK4RJQXErNndVRnNhO2qEkQ= 16 | github.com/aws/aws-sdk-go v1.20.2/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= 17 | github.com/aws/aws-sdk-go v1.31.8 h1:qbA8nsLYcqtGjMGDogqykuO0LyUONkP9YlsKu1SVV5M= 18 | github.com/aws/aws-sdk-go v1.31.8/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0= 19 | github.com/aws/aws-sdk-go v1.31.9 h1:n+b34ydVfgC30j0Qm69yaapmjejQPW2BoDBX7Uy/tLI= 20 | github.com/aws/aws-sdk-go v1.31.9/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0= 21 | github.com/aws/aws-xray-sdk-go v1.0.0-rc.9/go.mod h1:XtMKdBQfpVut+tJEwI7+dJFRxxRdxHDyVNp2tHXRq04= 22 | github.com/aws/aws-xray-sdk-go v1.0.1 h1:En3DuQ3fAIlNPKoMcAY7bv0lINCJPV0lElK8kEEXsKM= 23 | github.com/aws/aws-xray-sdk-go v1.0.1/go.mod h1:tmxq1c+yeEbMh39OmRFuXOrse5ajRlMmDXJ6LrCVsIs= 24 | github.com/cenkalti/backoff v2.1.1+incompatible/go.mod h1:90ReRw6GdpyfrHakVjL/QHaoyV4aDUVVkXQJJJ3NXXM= 25 | github.com/cenkalti/backoff v2.2.1+incompatible h1:tNowT99t7UNflLxfYYSlKYsBpXdEet03Pg2g16Swow4= 26 | github.com/cenkalti/backoff v2.2.1+incompatible/go.mod h1:90ReRw6GdpyfrHakVjL/QHaoyV4aDUVVkXQJJJ3NXXM= 27 | github.com/cihub/seelog v0.0.0-20170130134532-f561c5e57575/go.mod h1:9d6lWj8KzO/fd/NrVaLscBKmPigpZpn5YawRPw+e3Yo= 28 | github.com/coinbase/step v0.0.0-20190408131218-9f799639d07c h1:1Eu05EjTKaiEhGQpJPB+piCXqGRu1TGQuobv+i5CLdA= 29 | github.com/coinbase/step v0.0.0-20190408131218-9f799639d07c/go.mod h1:XU8bLMHgXSiQu4+RZ2Gcgq5dlfPDBYQK4gvKJKnff6k= 30 | github.com/coinbase/step v0.0.0-20190722103144-9138b048645f h1:GgNbLO2mgyPg8s3N2Rfc2ktHe/ZzCu8MjPI0TanpriI= 31 | github.com/coinbase/step v0.0.0-20190722103144-9138b048645f/go.mod h1:KofWhfd3TU0FFqZjUgwzIJesq92rmoRDU4fD50hVlWw= 32 | github.com/coinbase/step v0.0.0-20190913195516-3d78393342f3 h1:he31TRwB5Mcu+kZ8FCTCQy1q+7BdDeVEaGDlEkiZ1x0= 33 | github.com/coinbase/step v0.0.0-20190913195516-3d78393342f3/go.mod h1:KofWhfd3TU0FFqZjUgwzIJesq92rmoRDU4fD50hVlWw= 34 | github.com/coinbase/step v0.0.0-20200212195241-a6141d79bdd2 h1:d7hPr204gp/1D7XMw4/YLj++fyoUpXxUgpqtolSq+yw= 35 | github.com/coinbase/step v0.0.0-20200212195241-a6141d79bdd2/go.mod h1:KofWhfd3TU0FFqZjUgwzIJesq92rmoRDU4fD50hVlWw= 36 | github.com/coinbase/step v0.0.0-20200602093335-cbb46a27a28f h1:nDYlYyKn14xaxv2JynLbVGceL4z4tRRpogSrT6jb/I8= 37 | github.com/coinbase/step v0.0.0-20200602093335-cbb46a27a28f/go.mod h1:KofWhfd3TU0FFqZjUgwzIJesq92rmoRDU4fD50hVlWw= 38 | github.com/coinbase/step v1.0.2 h1:YFTSx3tv+KCHsazWwmOLyQbJgDcdbeXI+mR0Sp4RsBk= 39 | github.com/coinbase/step v1.0.2/go.mod h1:Fj5jyW0N5uGKkxofpIlK04ZHgv9lvbpHbvwW0G1TP9o= 40 | github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= 41 | github.com/davecgh/go-spew v0.0.0-20160907170601-6d212800a42e/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 42 | github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 43 | github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= 44 | github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 45 | github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg= 46 | github.com/google/gofuzz v0.0.0-20170612174753-24818f796faf h1:+RRA9JqSOZFfKrOeqr2z77+8R2RKyh8PG66dcu1V0ck= 47 | github.com/google/gofuzz v0.0.0-20170612174753-24818f796faf/go.mod h1:HP5RmnzzSNb993RKQDq4+1A4ia9nllfqcQFTQJedwGI= 48 | github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= 49 | github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af h1:pmfjZENx5imkbgOkpRUYLnmbU7UEFbjtDA2hxJ1ichM= 50 | github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= 51 | github.com/jmespath/go-jmespath v0.3.0 h1:OS12ieG61fsCg5+qLJ+SsW9NicxNkg3b25OyT2yCeUc= 52 | github.com/jmespath/go-jmespath v0.3.0/go.mod h1:9QtRXoHjLGCJ5IBSaohpXITPlowMeeYCZ7fLUTSywik= 53 | github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 54 | github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= 55 | github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 56 | github.com/pmezard/go-difflib v0.0.0-20151028094244-d8ed2627bdf0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= 57 | github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= 58 | github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= 59 | github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= 60 | github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= 61 | github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= 62 | github.com/stretchr/testify v1.1.4/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= 63 | github.com/stretchr/testify v1.2.1/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= 64 | github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w= 65 | github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= 66 | github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= 67 | github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= 68 | github.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4= 69 | github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= 70 | github.com/urfave/cli/v2 v2.1.1/go.mod h1:SE9GqnLQmjVa0iPEY0f1w3ygNIYcIJ0OKPMoW2caLfQ= 71 | golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= 72 | golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 73 | golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 74 | golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 75 | golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 76 | golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= 77 | gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 78 | gopkg.in/urfave/cli.v1 v1.20.0/go.mod h1:vuBzUtMdQeixQj8LVd+/98pzhxNGQoyuPBlsXHOQNO0= 79 | gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw= 80 | gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 81 | -------------------------------------------------------------------------------- /odin.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | 7 | "github.com/coinbase/odin/client" 8 | "github.com/coinbase/odin/deployer" 9 | "github.com/coinbase/step/utils/is" 10 | "github.com/coinbase/step/utils/run" 11 | "github.com/coinbase/step/utils/to" 12 | ) 13 | 14 | func main() { 15 | var arg, command string 16 | switch len(os.Args) { 17 | case 1: 18 | fmt.Println("Starting Lambda") 19 | run.LambdaTasks(deployer.TaskHandlers()) 20 | case 2: 21 | command = os.Args[1] 22 | arg = "" 23 | case 3: 24 | command = os.Args[1] 25 | arg = os.Args[2] 26 | default: 27 | printUsage() // Print how to use and exit 28 | } 29 | 30 | stepFn := to.Strp(os.Getenv("ODIN_STEP")) 31 | 32 | if is.EmptyStr(stepFn) { 33 | stepFn = to.Strp("coinbase-odin") 34 | } 35 | 36 | switch command { 37 | case "json": 38 | run.JSON(deployer.StateMachine()) 39 | case "deploy": 40 | // Send Configuration to the deployer 41 | // arg is a filename 42 | err := client.Deploy(stepFn, &arg) 43 | if err != nil { 44 | fmt.Println(err.Error()) 45 | os.Exit(1) 46 | } 47 | case "fails": 48 | // List the recent failures and their causes 49 | err := client.Failures(stepFn) 50 | if err != nil { 51 | fmt.Println(err.Error()) 52 | os.Exit(1) 53 | } 54 | case "halt": 55 | err := client.Halt(stepFn, &arg) 56 | if err != nil { 57 | fmt.Println(err.Error()) 58 | os.Exit(1) 59 | } 60 | default: 61 | printUsage() // Print how to use and exit 62 | } 63 | } 64 | 65 | func printUsage() { 66 | fmt.Println("Usage: odin (No args starts Lambda)") 67 | os.Exit(0) 68 | } 69 | -------------------------------------------------------------------------------- /releases/deploy-test-release.json: -------------------------------------------------------------------------------- 1 | { 2 | "project_name": "coinbase/deploy-test", 3 | "config_name": "development", 4 | "timeout": 240, 5 | "subnets": [ 6 | "test_private_subnet_a", 7 | "test_private_subnet_b" 8 | ], 9 | "ami": "ubuntu", 10 | "lifecycle": { 11 | "termhook" : { 12 | "transition": "autoscaling:EC2_INSTANCE_TERMINATING", 13 | "role": "asg_lifecycle_hooks", 14 | "sns": "asg_lifecycle_hooks", 15 | "heartbeat_timeout": 300 16 | } 17 | }, 18 | "services": { 19 | "worker": { 20 | "instance_type": "t2.nano", 21 | "security_groups": ["ec2::default"], 22 | "profile": "default-profile", 23 | "autoscaling": { 24 | "policies": [ 25 | { 26 | "name": "cpu_scale_up", 27 | "type": "cpu_scale_up", 28 | "threshold" : 25, 29 | "scaling_adjustment": 2 30 | }, 31 | { 32 | "name": "cpu_scale_down", 33 | "type": "cpu_scale_down", 34 | "threshold" : 15, 35 | "scaling_adjustment": -1 36 | } 37 | ] 38 | } 39 | }, 40 | "web": { 41 | "instance_type": "t2.nano", 42 | "security_groups": ["ec2::default", "ec2::coinbase/deploy-test::development"], 43 | "elbs": [ 44 | "coinbase-deploy-test-web-elb" 45 | ], 46 | "profile": "coinbase-deploy-test", 47 | "target_groups": [ 48 | "coinbase-deploy-test-web-tg" 49 | ], 50 | "ebs_volume_size": 20, 51 | "ebs_volume_type": "gp2", 52 | "ebs_device_name": "/dev/sda1", 53 | "autoscaling": { 54 | "min_size": 3, 55 | "max_size": 5, 56 | "spread": 0.8, 57 | "max_terms": 1, 58 | "policies": [ 59 | { 60 | "name": "cpu_scale_up", 61 | "type": "cpu_scale_up", 62 | "threshold" : 25, 63 | "scaling_adjustment": 2 64 | }, 65 | { 66 | "name": "cpu_scale_down", 67 | "type": "cpu_scale_down", 68 | "threshold" : 15, 69 | "scaling_adjustment": -1 70 | } 71 | ] 72 | } 73 | } 74 | } 75 | } 76 | 77 | 78 | -------------------------------------------------------------------------------- /releases/deploy-test-release.json.userdata: -------------------------------------------------------------------------------- 1 | #cloud-config 2 | repo_update: true 3 | repo_upgrade: all 4 | 5 | packages: 6 | - docker.io 7 | 8 | write_files: 9 | - path: / 10 | content: | 11 | {{RELEASE_ID}} 12 | {{SERVICE_NAME}} 13 | {{PROJECT_NAME}} 14 | {{CONFIG_NAME}} 15 | 16 | runcmd: 17 | - docker run -d --restart always --name test_server -p 8000:80 nginx 18 | -------------------------------------------------------------------------------- /releases/null-release.json: -------------------------------------------------------------------------------- 1 | { 2 | "project_name": "coinbase/null", 3 | "config_name": "development", 4 | "subnets": ["test_private_subnet_a"], 5 | "ami": "ubuntu", 6 | "services": { 7 | "null": { 8 | "instance_type": "t2.nano", 9 | "security_groups": ["ec2::default"] 10 | } 11 | } 12 | } 13 | 14 | 15 | -------------------------------------------------------------------------------- /releases/null-release.json.userdata: -------------------------------------------------------------------------------- 1 | #cloud_config 2 | -------------------------------------------------------------------------------- /resources/deploy-test-resources.rb: -------------------------------------------------------------------------------- 1 | ######################################## 2 | ### ENVIRONMENT ### 3 | ######################################## 4 | require_relative './vpc-resources' 5 | env = environment('development') 6 | 7 | project = project('coinbase', 'deploy-test') { 8 | environments 'development' 9 | tags { 10 | ProjectName "coinbase/deploy-test" 11 | ConfigName "development" 12 | ServiceName "web" 13 | self[:org] = "coinbase" 14 | self[:project] = "deploy-test" 15 | } 16 | } 17 | 18 | # SECURITY GROUPS 19 | elb_sg = project.resource("aws_security_group", "elb-web-app") { 20 | name "coinbase-deploy-test-development-web-elb-sg" 21 | description "Security Group for Web ELBs" 22 | vpc_id env.vpc_id 23 | 24 | ingress { 25 | from_port 8000 26 | to_port 8000 27 | protocol "tcp" 28 | cidr_blocks ["10.0.0.0/16"] 29 | } 30 | 31 | egress { 32 | from_port 0 33 | to_port 0 34 | protocol "-1" 35 | cidr_blocks ["0.0.0.0/0"] 36 | } 37 | 38 | tags { 39 | Name "elb::coinbase/deploy-test::development" 40 | } 41 | } 42 | 43 | project.resource("aws_security_group", "web-app") { 44 | name "coinbase-deploy-test-development-web-ec2-sg" 45 | description "Security Group for Web EC2 instances" 46 | vpc_id env.vpc_id 47 | 48 | ingress { 49 | from_port 8000 50 | to_port 8000 51 | protocol "tcp" 52 | security_groups [elb_sg] 53 | } 54 | 55 | ingress { 56 | from_port 22 57 | to_port 22 58 | protocol "tcp" 59 | cidr_blocks ["10.0.0.0/16"] 60 | } 61 | 62 | egress { 63 | from_port 0 64 | to_port 0 65 | protocol "-1" 66 | cidr_blocks ["0.0.0.0/0"] 67 | } 68 | 69 | tags { 70 | Name "ec2::coinbase/deploy-test::development" 71 | } 72 | } 73 | 74 | project.resource("aws_security_group", "default") { 75 | name "coinbase-deploy-test-development-ec2-default" 76 | description "Default Security Group" 77 | vpc_id env.vpc_id 78 | 79 | tags { 80 | Name "ec2::default" 81 | ProjectName "_all" 82 | ConfigName "development" 83 | ServiceName "_all" 84 | } 85 | } 86 | 87 | # ELB 88 | project.resource("aws_elb", "web-app") { 89 | name "coinbase-deploy-test-web-elb" 90 | internal true 91 | security_groups [elb_sg] 92 | subnets [env.public_subnet_a_id, env.public_subnet_b_id] 93 | listener { 94 | instance_port 8000 95 | instance_protocol "http" 96 | lb_port 80 97 | lb_protocol "http" 98 | } 99 | 100 | health_check { 101 | target "HTTP:8000/" 102 | healthy_threshold 2 103 | unhealthy_threshold 5 104 | interval 30 105 | timeout 10 106 | } 107 | 108 | tags { 109 | Name "elb:coinbase-deploy-test" 110 | } 111 | } 112 | 113 | # ALB 114 | alb = project.resource("aws_lb", "web-app") { 115 | name "coinbase-deploy-test-web-alb" 116 | internal true 117 | security_groups [elb_sg] 118 | subnets [env.public_subnet_a_id, env.public_subnet_b_id] 119 | 120 | lifecycle { 121 | ignore_changes ["enable_cross_zone_load_balancing", "enable_http2"] 122 | } 123 | 124 | tags { 125 | Name "alb:coinbase-deploy-test" 126 | } 127 | } 128 | 129 | 130 | target_group = project.resource('aws_alb_target_group', "alb_tg") { 131 | name "coinbase-deploy-test-web-tg" 132 | port 8000 133 | protocol 'HTTP' 134 | vpc_id env.vpc_id 135 | 136 | health_check { 137 | timeout 10 138 | unhealthy_threshold 5 139 | healthy_threshold 2 140 | interval 30 141 | path '/' 142 | } 143 | 144 | lifecycle { 145 | ignore_changes ["proxy_protocol_v2"] 146 | } 147 | 148 | tags { 149 | Name "alb:coinbase-deploy-test:tg" 150 | } 151 | } 152 | 153 | project.resource("aws_alb_listener", 'alb_ln') { 154 | _load_balancer_name alb.name 155 | load_balancer_arn alb.to_ref("arn") 156 | port 80 157 | protocol "HTTP" 158 | 159 | default_action { 160 | target_group_arn target_group.to_id_or_ref 161 | self["type"] = "forward" 162 | } 163 | } 164 | 165 | # Instance Roles 166 | role = project.resource('aws_iam_role', 'coinbase-deploy-test-dns') { 167 | name 'coinbase-deploy-test' 168 | path '/odin/coinbase/deploy-test/development/web/' 169 | assume_role_policy(%({ 170 | "Version": "2012-10-17", 171 | "Statement": [ 172 | { 173 | "Effect": "Allow", 174 | "Principal": { 175 | "Service": "ec2.amazonaws.com" 176 | }, 177 | "Action": "sts:AssumeRole" 178 | } 179 | ] 180 | })) 181 | } 182 | 183 | project.resource('aws_iam_instance_profile', 'coinbase-deploy-test') { 184 | name 'coinbase-deploy-test' 185 | path '/odin/coinbase/deploy-test/development/web/' 186 | role role 187 | } 188 | 189 | 190 | default_role = project.resource('aws_iam_role', 'default-profile') { 191 | name 'default-profile' 192 | path '/odin/_all/_all/_all/' 193 | assume_role_policy(%({ 194 | "Version": "2012-10-17", 195 | "Statement": [ 196 | { 197 | "Effect": "Allow", 198 | "Principal": { 199 | "Service": "ec2.amazonaws.com" 200 | }, 201 | "Action": "sts:AssumeRole" 202 | } 203 | ] 204 | })) 205 | } 206 | 207 | project.resource('aws_iam_instance_profile', 'default-profile') { 208 | name 'default-profile' 209 | path '/odin/_all/_all/_all/' 210 | role default_role 211 | } 212 | 213 | policy = project.resource('aws_iam_policy', 'az-policy') { 214 | name 'coinbase-deploy-test' 215 | policy '{ 216 | "Version": "2012-10-17", 217 | "Statement": [ 218 | { 219 | "Effect": "Allow", 220 | "Action": [ 221 | "ec2:DescribeAvailabilityZones" 222 | ], 223 | "Resource": ["*"] 224 | } 225 | ] 226 | }' 227 | } 228 | 229 | project.resource('aws_iam_policy_attachment', 'default-profile') { 230 | name 'default-profile' 231 | _policy policy 232 | roles [role, default_role] 233 | } 234 | -------------------------------------------------------------------------------- /resources/odin.rb: -------------------------------------------------------------------------------- 1 | # GeoEngineer Resources For Step Function Deployer 2 | # GEO_ENV=development bundle exec geo apply resources/odin.rb 3 | 4 | ######################################## 5 | ### ENVIRONMENT ### 6 | ######################################## 7 | 8 | env = environment('development') { 9 | region ENV.fetch('AWS_REGION') 10 | account_id ENV.fetch('AWS_ACCOUNT_ID') 11 | } 12 | 13 | ######################################## 14 | ### PROJECT ### 15 | ######################################## 16 | project = project('coinbase', 'odin') { 17 | environments 'development' 18 | tags { 19 | ProjectName "coinbase/odin" 20 | ConfigName "development" 21 | DeployWith "step-deployer" 22 | self[:org] = "coinbase" 23 | self[:project] = "odin" 24 | } 25 | } 26 | 27 | context = { 28 | assumed_role_name: "coinbase-odin-assumed", 29 | assumable_from: [ ENV['AWS_ACCOUNT_ID'] ], 30 | assumed_policy_file: "#{__dir__}/odin_assumed_policy.json.erb" 31 | } 32 | 33 | project.from_template('bifrost_deployer', 'odin', { 34 | lambda_policy_file: "#{__dir__}/odin_lambda_policy.json.erb", 35 | lambda_policy_context: context 36 | }) 37 | 38 | # The assumed role exists in all environments 39 | project.from_template('step_assumed', 'coinbase-odin-assumed', context) 40 | -------------------------------------------------------------------------------- /resources/odin_assumed_policy.json.erb: -------------------------------------------------------------------------------- 1 | { 2 | "Version": "2012-10-17", 3 | "Statement": [ 4 | { 5 | "Effect": "Allow", 6 | "Action": [ 7 | "iam:GetRole", 8 | "iam:PassRole", 9 | "iam:GetInstanceProfile", 10 | "ec2:DescribeImages", 11 | "ec2:RunInstances", 12 | "ec2:DescribeSubnets", 13 | "ec2:DescribeSecurityGroups", 14 | "elasticloadbalancing:DescribeLoadBalancerAttributes", 15 | "elasticloadbalancing:DescribeLoadBalancers", 16 | "elasticloadbalancing:DescribeTargetGroupAttributes", 17 | "elasticloadbalancing:DescribeTags", 18 | "elasticloadbalancing:DescribeTargetHealth", 19 | "elasticloadbalancing:DescribeTargetGroups", 20 | "elasticloadbalancing:DescribeLoadBalancerPolicies", 21 | "elasticloadbalancing:DescribeLoadBalancerPolicyTypes", 22 | "elasticloadbalancing:DescribeInstanceHealth", 23 | "cloudwatch:PutMetricAlarm", 24 | "cloudwatch:DeleteAlarms", 25 | "cloudwatch:DescribeAlarms", 26 | "sns:GetTopicAttributes", 27 | "autoscaling:*" 28 | ], 29 | "Resource": "*", 30 | "Condition": { 31 | "Bool": { 32 | "aws:SecureTransport": "true" 33 | } 34 | } 35 | } 36 | ] 37 | } 38 | -------------------------------------------------------------------------------- /resources/odin_lambda_policy.json.erb: -------------------------------------------------------------------------------- 1 | { 2 | "Version": "2012-10-17", 3 | "Statement": [ 4 | { 5 | "Effect": "Allow", 6 | "Resource": "arn:aws:iam::*:role/<%= assumed_role_name %>", 7 | "Action": "sts:AssumeRole" 8 | }, 9 | { 10 | "Effect": "Allow", 11 | "Action": [ 12 | "s3:GetObject*", 13 | "s3:PutObject*", 14 | "s3:DeleteObject*", 15 | "s3:ListBucket" 16 | ], 17 | "Resource": [ 18 | "arn:aws:s3:::<%= s3_bucket_name %>/*", 19 | "arn:aws:s3:::<%= s3_bucket_name %>" 20 | ] 21 | }, 22 | { 23 | "Effect": "Deny", 24 | "Action": [ 25 | "s3:*" 26 | ], 27 | "NotResource": [ 28 | "arn:aws:s3:::<%= s3_bucket_name %>/*", 29 | "arn:aws:s3:::<%= s3_bucket_name %>" 30 | ] 31 | } 32 | ] 33 | } 34 | -------------------------------------------------------------------------------- /resources/vpc-resources.rb: -------------------------------------------------------------------------------- 1 | env = environment('development') { 2 | region ENV.fetch('AWS_REGION') 3 | account_id ENV.fetch('AWS_ACCOUNT_ID') 4 | } 5 | 6 | vpc = env.resource('aws_vpc', "main") { 7 | cidr_block "10.0.0.0/16" 8 | tags { 9 | Name "test_vpc" 10 | } 11 | } 12 | 13 | public_subnet_a = env.resource("aws_subnet", "public_subnet_a") { 14 | vpc_id vpc.to_ref 15 | cidr_block "10.0.10.0/24" 16 | availability_zone "#{env.region}a" 17 | map_public_ip_on_launch false 18 | 19 | tags { 20 | Name "test_public_subnet_a" 21 | } 22 | } 23 | 24 | public_subnet_b = env.resource("aws_subnet", "public_subnet_b") { 25 | vpc_id vpc.to_ref 26 | cidr_block "10.0.11.0/24" 27 | availability_zone "#{env.region}b" 28 | map_public_ip_on_launch false 29 | 30 | tags { 31 | Name "test_public_subnet_b" 32 | } 33 | } 34 | 35 | private_subnet_a = env.resource("aws_subnet", "private_subnet_a") { 36 | vpc_id vpc.to_ref 37 | cidr_block "10.0.20.0/24" 38 | availability_zone "#{env.region}a" 39 | 40 | tags { 41 | Name "test_private_subnet_a" 42 | DeployWith "odin" 43 | } 44 | } 45 | 46 | private_subnet_b = env.resource("aws_subnet", "private_subnet_b") { 47 | vpc_id vpc.to_ref 48 | cidr_block "10.0.21.0/24" 49 | availability_zone "#{env.region}b" 50 | 51 | tags { 52 | Name "test_private_subnet_b" 53 | DeployWith "odin" 54 | } 55 | } 56 | 57 | ig = env.resource("aws_internet_gateway", "internet_gateway") { 58 | vpc_id vpc.to_ref 59 | tags { 60 | Name "test_internet_gateway" 61 | } 62 | } 63 | 64 | public_routetable = env.resource("aws_route_table", "public_routetable") { 65 | vpc_id vpc.to_ref 66 | 67 | route { 68 | cidr_block "0.0.0.0/0" 69 | gateway_id ig.to_ref 70 | } 71 | 72 | tags { 73 | Name "test_public_routetable" 74 | } 75 | } 76 | 77 | eip = env.resource("aws_eip", "eip_4_nat") { 78 | tags { 79 | Name "test_eip" 80 | } 81 | } 82 | 83 | nat = env.resource("aws_nat_gateway", "nat") { 84 | allocation_id eip.to_ref 85 | subnet_id public_subnet_a.to_ref 86 | tags { 87 | Name "test_nat_gateway" 88 | } 89 | } 90 | 91 | private_routetable = env.resource("aws_route_table", "private_routetable") { 92 | vpc_id vpc.to_ref 93 | 94 | route { 95 | cidr_block "0.0.0.0/0" 96 | nat_gateway_id nat.to_ref 97 | } 98 | 99 | tags { 100 | Name "test_private_routetable" 101 | } 102 | } 103 | 104 | env.resource("aws_route_table_association", "public_subnet_a") { 105 | subnet public_subnet_a 106 | route_table public_routetable 107 | } 108 | 109 | env.resource("aws_route_table_association", "public_subnet_b") { 110 | subnet public_subnet_b 111 | route_table public_routetable 112 | } 113 | 114 | env.resource("aws_route_table_association", "private_subnet_a") { 115 | subnet private_subnet_a 116 | route_table private_routetable 117 | } 118 | 119 | env.resource("aws_route_table_association", "private_subnet_b") { 120 | subnet private_subnet_b 121 | route_table private_routetable 122 | } 123 | 124 | env.vpc_id = vpc.to_ref 125 | env.private_subnet_a_id = private_subnet_a.to_ref 126 | env.public_subnet_a_id = public_subnet_a.to_ref 127 | env.private_subnet_b_id = private_subnet_b.to_ref 128 | env.public_subnet_b_id = public_subnet_b.to_ref 129 | 130 | ##### 131 | # LIFECYCLE HOOK RESOURCES 132 | ##### 133 | 134 | env.resource('aws_sns_topic', 'asg_lifecycle_hooks') { 135 | display_name 'asg_lifecycle_hooks' 136 | name 'asg_lifecycle_hooks' 137 | } 138 | 139 | life_cycle_role = env.resource('aws_iam_role', 'asg_lifecycle_hooks') { 140 | name 'asg_lifecycle_hooks' 141 | assume_role_policy '{ 142 | "Version": "2012-10-17", 143 | "Statement": [ 144 | { 145 | "Sid": "", 146 | "Effect": "Allow", 147 | "Principal": { 148 | "Service": "autoscaling.amazonaws.com" 149 | }, 150 | "Action": "sts:AssumeRole" 151 | } 152 | ] 153 | }' 154 | } 155 | 156 | life_cycle_policy = env.resource('aws_iam_policy', 'asg_lifecycle_hooks') { 157 | name 'asg_lifecycle_hooks' 158 | policy '{ 159 | "Version": "2012-10-17", 160 | "Statement": [{ 161 | "Effect": "Allow", 162 | "Resource": "*", 163 | "Action": [ 164 | "sqs:SendMessage", 165 | "sqs:GetQueueUrl", 166 | "sns:Publish" 167 | ] 168 | } 169 | ] 170 | }' 171 | } 172 | 173 | env.resource('aws_iam_policy_attachment', 'asg_lifecycle_hooks') { 174 | name 'asg_lifecycle_hooks' 175 | _policy life_cycle_policy 176 | roles [life_cycle_role] 177 | } 178 | -------------------------------------------------------------------------------- /scripts/bootstrap: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # bootstrap odin 3 | # assume-role to the correct account 4 | set -e 5 | 6 | function ensure { 7 | if ! [ -x "$(command -v $1)" ]; then 8 | echo "Error: $1 is not installed." >&2 9 | exit 1 10 | fi 11 | } 12 | 13 | # ruby bundle terraform zip and go are required 14 | ensure ruby 15 | ensure bundle 16 | ensure terraform 17 | ensure zip 18 | ensure go 19 | 20 | bundle install 21 | 22 | ./scripts/geo apply resources/odin.rb 23 | 24 | ./scripts/bootstrap_deployer 25 | -------------------------------------------------------------------------------- /scripts/bootstrap_deployer: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # bootstrap odin 3 | # assume-role to the correct account 4 | set -e 5 | 6 | ./scripts/build_lambda_zip 7 | 8 | PROJECT_NAME=${PROJECT_NAME:-coinbase/odin} 9 | STEP_NAME=$(echo $PROJECT_NAME | sed 's/\//\-/') 10 | echo $PROJECT_NAME 11 | echo $STEP_NAME 12 | 13 | step bootstrap \ 14 | -lambda $STEP_NAME \ 15 | -step $STEP_NAME \ 16 | -states "$(go run odin.go json)" \ 17 | -project $PROJECT_NAME \ 18 | -config "development" 19 | 20 | rm lambda.zip 21 | -------------------------------------------------------------------------------- /scripts/build_lambda_zip: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Build Lambda Zip 3 | set -e 4 | 5 | # Build step (called lambda) for linux lambda 6 | GOOS=linux go build -o lambda 7 | zip lambda.zip lambda 8 | rm lambda 9 | -------------------------------------------------------------------------------- /scripts/deploy_deployer: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # deploy odin 3 | # assume-role to the correct account 4 | set -e 5 | 6 | go build . # Build step for your operating system 7 | 8 | ./scripts/build_lambda_zip 9 | 10 | step deploy \ 11 | -lambda "coinbase-odin" \ 12 | -step "coinbase-odin" \ 13 | -states "$(./odin json)"\ 14 | -project "coinbase/odin"\ 15 | -config "development" 16 | 17 | rm lambda.zip 18 | -------------------------------------------------------------------------------- /scripts/deploy_test_deploy: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # deploy deploy test project 3 | set -e 4 | 5 | go build && go install # Build step for your operating system 6 | 7 | odin deploy releases/deploy-test-release.json 8 | -------------------------------------------------------------------------------- /scripts/deploy_test_halt: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # halt deploy test project 3 | set -e 4 | 5 | go build && go intsall # Build step for your operating system 6 | 7 | odin halt releases/deploy-test-release.json 8 | -------------------------------------------------------------------------------- /scripts/geo: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | export GEO_ENV=step 3 | bundle exec geo $@ 4 | -------------------------------------------------------------------------------- /scripts/graph: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | step dot -states "$(go run odin.go json)" | dot -Tpng -o assets/sm.png; open assets/sm.png --------------------------------------------------------------------------------