├── .github └── PULL_REQUEST_TEMPLATE.md ├── CODE_OF_CONDUCT.md ├── README.md ├── LICENSE ├── CONTRIBUTING.md └── sources └── templates ├── dummy_VPC.yaml ├── outbound-proxy-Launch-Template-CF.yml └── outbound-proxy-CF.yaml /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | *Issue #, if available:* 2 | 3 | *Description of changes:* 4 | 5 | 6 | By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. 7 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Outbound VPC proxy 2 | 3 | Outbound VPC proxy with domain whitelisting and content filtering 4 | 5 | For more information see AWS Security Blog [How to set up an outbound VPC proxy with domain whitelisting and content filtering](https://aws.amazon.com/blogs/security/how-to-set-up-an-outbound-vpc-proxy-with-domain-whitelisting-and-content-filtering) 6 | 7 | ## License Summary 8 | 9 | This sample code is made available under the MIT-0 license. See the LICENSE file. 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | this software and associated documentation files (the "Software"), to deal in 5 | the Software without restriction, including without limitation the rights to 6 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check [existing open](https://github.com/aws-samples/outbound-vpc-filtering-proxy/issues), or [recently closed](https://github.com/aws-samples/outbound-vpc-filtering-proxy/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *master* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws-samples/outbound-vpc-filtering-proxy/labels/help%20wanted) issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](https://github.com/aws-samples/outbound-vpc-filtering-proxy/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | 61 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. 62 | -------------------------------------------------------------------------------- /sources/templates/dummy_VPC.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | AWSTemplateFormatVersion: '2010-09-09' 4 | Description: Create VPC for proxy demo 5 | Parameters: 6 | 7 | VPCCidrBlock: 8 | Type: String 9 | Description: CIDR block for VPC to deploy. (X.X.X.X/X) 10 | AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(/([0-9]|[1-2][0-9]|3[0-2]))$ 11 | Default: 10.0.0.0/16 12 | PublicSubnetCIDR1: 13 | Type: String 14 | Description: First public subnet CIDR 15 | AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(/([0-9]|[1-2][0-9]|3[0-2]))$ 16 | Default: 10.0.1.0/24 17 | PublicSubnetCIDR2: 18 | Type: String 19 | Description: Second public subnet CIDR 20 | AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(/([0-9]|[1-2][0-9]|3[0-2]))$ 21 | Default: 10.0.2.0/24 22 | PublicSubnetCIDR3: 23 | Type: String 24 | Description: Third public subnet CIDR 25 | AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(/([0-9]|[1-2][0-9]|3[0-2]))$ 26 | Default: 10.0.3.0/24 27 | PrivateSubnetCIDR1: 28 | Type: String 29 | Description: First private subnet CIDR 30 | AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(/([0-9]|[1-2][0-9]|3[0-2]))$ 31 | Default: 10.0.4.0/24 32 | PrivateSubnetCIDR2: 33 | Type: String 34 | Description: Second private subnet CIDR 35 | AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(/([0-9]|[1-2][0-9]|3[0-2]))$ 36 | Default: 10.0.5.0/24 37 | PrivateSubnetCIDR3: 38 | Type: String 39 | Description: Thirs private subnet CIDR 40 | AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(/([0-9]|[1-2][0-9]|3[0-2]))$ 41 | Default: 10.0.6.0/24 42 | Resources: 43 | VPC: 44 | Type: AWS::EC2::VPC 45 | Properties: 46 | CidrBlock: !Ref 'VPCCidrBlock' 47 | EnableDnsSupport: 'true' 48 | EnableDnsHostnames: 'true' 49 | InstanceTenancy: default 50 | Tags: 51 | - Key: Name 52 | Value: !Sub vpc-${AWS::StackName} 53 | 54 | InternetGateway: 55 | Type: AWS::EC2::InternetGateway 56 | Properties: 57 | Tags: 58 | - Key: Name 59 | Value: Internet Gateway 60 | InternetGatewayAttachment: 61 | Type: AWS::EC2::VPCGatewayAttachment 62 | Properties: 63 | InternetGatewayId: !Ref 'InternetGateway' 64 | VpcId: !Ref 'VPC' 65 | PublicSubnet1: 66 | Type: AWS::EC2::Subnet 67 | Properties: 68 | CidrBlock: !Ref 'PublicSubnetCIDR1' 69 | AvailabilityZone: !Select 70 | - '0' 71 | - !GetAZs '' 72 | MapPublicIpOnLaunch: 'false' 73 | Tags: 74 | - Key: Name 75 | Value: Public subnet 1 76 | 77 | VpcId: !Ref 'VPC' 78 | PublicSubnet2: 79 | Type: AWS::EC2::Subnet 80 | Properties: 81 | CidrBlock: !Ref 'PublicSubnetCIDR2' 82 | AvailabilityZone: !Select 83 | - '1' 84 | - !GetAZs '' 85 | MapPublicIpOnLaunch: 'false' 86 | Tags: 87 | - Key: Name 88 | Value: Public subnet 2 89 | VpcId: !Ref 'VPC' 90 | PublicSubnet3: 91 | Type: AWS::EC2::Subnet 92 | Properties: 93 | CidrBlock: !Ref 'PublicSubnetCIDR3' 94 | AvailabilityZone: !Select 95 | - '2' 96 | - !GetAZs '' 97 | MapPublicIpOnLaunch: 'false' 98 | Tags: 99 | - Key: Name 100 | Value: Pubic subnet 3 101 | 102 | VpcId: !Ref 'VPC' 103 | PublicRouteTable: 104 | Type: AWS::EC2::RouteTable 105 | Properties: 106 | VpcId: !Ref 'VPC' 107 | Tags: 108 | - Key: Name 109 | Value: Public Route Table 110 | AttachPublicSubnet1RouteTable: 111 | Type: AWS::EC2::SubnetRouteTableAssociation 112 | Properties: 113 | RouteTableId: !Ref 'PublicRouteTable' 114 | SubnetId: !Ref 'PublicSubnet1' 115 | AttachPublicSubnet2RouteTable: 116 | Type: AWS::EC2::SubnetRouteTableAssociation 117 | Properties: 118 | RouteTableId: !Ref 'PublicRouteTable' 119 | SubnetId: !Ref 'PublicSubnet2' 120 | AttachPublicSubnet3RouteTable: 121 | Type: AWS::EC2::SubnetRouteTableAssociation 122 | Properties: 123 | RouteTableId: !Ref 'PublicRouteTable' 124 | SubnetId: !Ref 'PublicSubnet3' 125 | PublicRoutetoInternet: 126 | Type: AWS::EC2::Route 127 | Properties: 128 | DestinationCidrBlock: '0.0.0.0/0' 129 | GatewayId: !Ref 'InternetGateway' 130 | RouteTableId: !Ref 'PublicRouteTable' 131 | 132 | PrivateSubnet1: 133 | Type: AWS::EC2::Subnet 134 | Properties: 135 | CidrBlock: !Ref 'PrivateSubnetCIDR1' 136 | AvailabilityZone: !Select 137 | - '0' 138 | - !GetAZs '' 139 | MapPublicIpOnLaunch: 'false' 140 | Tags: 141 | - Key: Name 142 | Value: Private Subnet 1 143 | VpcId: !Ref 'VPC' 144 | PrivateSubnet2: 145 | Type: AWS::EC2::Subnet 146 | Properties: 147 | CidrBlock: !Ref 'PrivateSubnetCIDR2' 148 | AvailabilityZone: !Select 149 | - '1' 150 | - !GetAZs '' 151 | MapPublicIpOnLaunch: 'false' 152 | Tags: 153 | - Key: Name 154 | Value: Private Subnet 2 155 | VpcId: !Ref 'VPC' 156 | PrivateSubnet3: 157 | Type: AWS::EC2::Subnet 158 | Properties: 159 | CidrBlock: !Ref 'PrivateSubnetCIDR3' 160 | AvailabilityZone: !Select 161 | - '2' 162 | - !GetAZs '' 163 | MapPublicIpOnLaunch: 'false' 164 | Tags: 165 | - Key: Name 166 | Value: Private Subnet 3 167 | VpcId: !Ref 'VPC' 168 | 169 | PrivateRouteTable1: 170 | Type: AWS::EC2::RouteTable 171 | Properties: 172 | VpcId: !Ref 'VPC' 173 | Tags: 174 | - Key: Name 175 | Value: Private Route Table 1 176 | AttachPrivateSubnet1RouteTable1: 177 | Type: AWS::EC2::SubnetRouteTableAssociation 178 | Properties: 179 | RouteTableId: !Ref 'PrivateRouteTable1' 180 | SubnetId: !Ref 'PrivateSubnet1' 181 | AttachPrivateSubnet2RouteTable1: 182 | Type: AWS::EC2::SubnetRouteTableAssociation 183 | Properties: 184 | RouteTableId: !Ref 'PrivateRouteTable1' 185 | SubnetId: !Ref 'PrivateSubnet2' 186 | AttachPrivateSubnet3RouteTable1: 187 | Type: AWS::EC2::SubnetRouteTableAssociation 188 | Properties: 189 | RouteTableId: !Ref 'PrivateRouteTable1' 190 | SubnetId: !Ref 'PrivateSubnet3' 191 | 192 | 193 | Outputs: 194 | VPCID: 195 | Description: VPC ID 196 | Value: !Ref 'VPC' 197 | Export: 198 | Name: !Sub 'VPC' 199 | VPCCidrBlock: 200 | Description: VPC Cidr Block 201 | Value: !Ref 'VPCCidrBlock' 202 | Export: 203 | Name: !Sub '${AWS::StackName}-VPCCidrBlock' 204 | PublicSubnet1: 205 | Description: Public Subnet 1 206 | Value: !Ref 'PublicSubnet1' 207 | Export: 208 | Name: !Sub '${AWS::StackName}-public-subnet-1' 209 | PublicSubnet2: 210 | Description: Public Subnet 2 211 | Value: !Ref 'PublicSubnet2' 212 | Export: 213 | Name: !Sub '${AWS::StackName}-public-subnet-2' 214 | PublicSubnet3: 215 | Description: Public Subnet 3 216 | Value: !Ref 'PublicSubnet3' 217 | Export: 218 | Name: !Sub '${AWS::StackName}-public-subnet-3' 219 | PrivateSubnet1: 220 | Description: Private Subnet 1 221 | Value: !Ref 'PrivateSubnet1' 222 | Export: 223 | Name: !Sub '${AWS::StackName}-private-subnet-1' 224 | PrivateSubnet2: 225 | Description: Private Subnet 2 226 | Value: !Ref 'PrivateSubnet2' 227 | Export: 228 | Name: !Sub '${AWS::StackName}-private-subnet-2' 229 | PrivateSubnet3: 230 | Description: Private Subnet 3 231 | Value: !Ref 'PrivateSubnet3' 232 | Export: 233 | Name: !Sub '${AWS::StackName}-private-subnet-3' 234 | -------------------------------------------------------------------------------- /sources/templates/outbound-proxy-Launch-Template-CF.yml: -------------------------------------------------------------------------------- 1 | Description: Outbound filtering proxy 2 | 3 | Parameters: 4 | WhitelistedDomains: 5 | Type: String 6 | Default: .amazonaws.com, .debian.org 7 | Description: Whitelisted domains comma separated 8 | 9 | CustomDNS: 10 | Type: String 11 | Default: default 12 | Description: Provide optional a DNS server for domain filtering, like OpenDNS (comma separated, like 8.8.8.8,8.8.8.7) 13 | 14 | KeyName: 15 | Type: "String" 16 | Description: Name of RSA key for EC2 access for testing only. 17 | Default: 'Hossam-PDC' 18 | 19 | ProxyPort: 20 | Type: String 21 | Default: 3128 22 | Description: Port Proxy 23 | 24 | VpcId: 25 | Description: VPC ID Where the Proxy will be installed 26 | Type: "AWS::EC2::VPC::Id" 27 | 28 | PrivateSubnetIDs: 29 | Description: Private SubnetIDs where the Network LoadBalancer will be placed (Select min 2 max 3) 30 | Type: "List" 31 | 32 | PublicSubnetIDs: 33 | Description: Public SubnetIDs where the proxy will be placed (Select min 2 max 3) 34 | Type: "List" 35 | 36 | InstanceType: 37 | Description: WebServer EC2 instance type 38 | Type: String 39 | Default: t3.medium 40 | AllowedValues: 41 | - t3.nano 42 | - t3.micro 43 | - t3.small 44 | - t3.medium 45 | - t3.large 46 | - m3.medium 47 | - m3.large 48 | - m3.xlarge 49 | - m3.2xlarge 50 | - m4.large 51 | - m4.xlarge 52 | - m4.2xlarge 53 | - m5.large 54 | - m5.xlarge 55 | - m5.2xlarge 56 | - c3.large 57 | - c3.xlarge 58 | - c4.large 59 | ConstraintDescription: must be a valid EC2 instance type. 60 | 61 | NetworkAllowedCIDR: 62 | Description: CIDR allowed in Proxy Security Group. The allowed block size is between a /32 netmask and /8 netmask 63 | Type: String 64 | Default: 10.0.0.0/16 65 | AllowedPattern: ^[.0-9]*\/([89]|[12][0-9]|3[0-2])$ 66 | 67 | LatestAmiId: 68 | Type: 'AWS::SSM::Parameter::Value' 69 | Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2' 70 | Description: AMI ID pointer in SSM. Default latest AMI Amazon Linux2. 71 | 72 | Metadata: 73 | 'AWS::CloudFormation::Interface': 74 | ParameterGroups: 75 | - Label: 76 | default: Proxy parameter 77 | Parameters: 78 | - WhitelistedDomains 79 | - CustomDNS 80 | - ProxyPort 81 | - InstanceType 82 | - LatestAmiId 83 | - KeyName 84 | - Label: 85 | default: Network parameter 86 | Parameters: 87 | - VpcId 88 | - PrivateSubnetIDs 89 | - PublicSubnetIDs 90 | - NetworkAllowedCIDR 91 | 92 | ParameterLabels: 93 | WhitelistedDomains: 94 | default: Allowed domains (whitelisted) 95 | CustomDNS: 96 | default: Custom DNS servers 97 | ProxyPort: 98 | default: Proxy Port 99 | InstanceType: 100 | default: Instance Type 101 | LatestAmiId: 102 | default: AMI ID 103 | KeyName: 104 | default: SSH Key name 105 | VpcId: 106 | default: VPC ID 107 | PrivateSubnetIDs: 108 | default: Private Subnet IDs 109 | PublicSubnetIDs: 110 | default: Public Subnet IDs 111 | NetworkAllowedCIDR: 112 | default: Allowed client CIRD 113 | 114 | Conditions: 115 | AddSSHKey: !Not 116 | - !Equals 117 | - '' 118 | - !Ref KeyName 119 | Resources: 120 | OutboundProxyRole: 121 | Type: AWS::IAM::Role 122 | Properties: 123 | RoleName: !Sub "Outbound-proxy-${AWS::StackName}" 124 | AssumeRolePolicyDocument: 125 | Version: '2012-10-17' 126 | Statement: 127 | - Effect: Allow 128 | Principal: 129 | Service: 130 | - ec2.amazonaws.com 131 | Action: 132 | - sts:AssumeRole 133 | Path: "/" 134 | Policies: 135 | - PolicyName: LogRolePolicy 136 | PolicyDocument: 137 | Version: '2012-10-17' 138 | Statement: 139 | - Effect: Allow 140 | Action: 141 | - logs:CreateLogGroup 142 | - logs:CreateLogStream 143 | - logs:PutLogEvents 144 | - logs:DescribeLogStreams 145 | Resource: 146 | - !GetAtt OutboundProxyLogGroup.Arn 147 | - PolicyName: AssociateEIP 148 | PolicyDocument: 149 | Version: '2012-10-17' 150 | Statement: 151 | - Effect: Allow 152 | Action: 153 | - ec2:AssociateAddress 154 | - ec2:Describe* 155 | Resource: 156 | - "*" 157 | - PolicyName: RevokeAuthorizeSG 158 | PolicyDocument: 159 | Version: '2012-10-17' 160 | Statement: 161 | - Effect: Allow 162 | Action: 163 | - ec2:RevokeSecurityGroupIngress 164 | - ec2:AuthorizeSecurityGroupIngress 165 | - ec2:Describe* 166 | Resource: 167 | - "*" 168 | - PolicyName: GetSecret 169 | PolicyDocument: 170 | Version: '2012-10-17' 171 | Statement: 172 | - Effect: Allow 173 | Action: 174 | - secretsmanager:GetSecretValue 175 | Resource: 176 | - !Ref WhitelistedSitesSecret 177 | - PolicyName: CloudWatchMetric 178 | PolicyDocument: 179 | Version: '2012-10-17' 180 | Statement: 181 | - Effect: Allow 182 | Action: 183 | - cloudwatch:PutMetricData 184 | Resource: 185 | - "*" 186 | 187 | WhitelistedSitesSecret: 188 | Type: 'AWS::SecretsManager::Secret' 189 | Properties: 190 | Name: Proxy-Domains-Whitelisting 191 | Description: This secret contains the proxy whitelisted domains 192 | SecretString: !Ref WhitelistedDomains 193 | 194 | FixedEIPa: 195 | Type: AWS::EC2::EIP 196 | Properties: 197 | Domain: vpc 198 | 199 | FixedEIPb: 200 | Type: AWS::EC2::EIP 201 | Properties: 202 | Domain: vpc 203 | 204 | FixedEIPc: 205 | Type: AWS::EC2::EIP 206 | Properties: 207 | Domain: vpc 208 | 209 | FixedEIPd: 210 | Type: AWS::EC2::EIP 211 | Properties: 212 | Domain: vpc 213 | 214 | LoadBalancer: 215 | Type: AWS::ElasticLoadBalancingV2::LoadBalancer 216 | Properties: 217 | Scheme: internal 218 | Type: network 219 | Name: OutboundProxyLoadBalancer 220 | Subnets: !Ref PrivateSubnetIDs 221 | 222 | NetworkLoadBalancerTargetGroup: 223 | Type: AWS::ElasticLoadBalancingV2::TargetGroup 224 | Properties: 225 | Name: OutboundProxyTargetGroup 226 | Port: !Ref ProxyPort 227 | Protocol: TCP 228 | VpcId: !Ref VpcId 229 | TargetGroupAttributes: 230 | - Key: deregistration_delay.timeout_seconds 231 | Value: 60 232 | Tags: 233 | - Key: Name 234 | Value: SMARTProxyTargetGroup 235 | 236 | LoadBalancerListenerHTTPS: 237 | Type: AWS::ElasticLoadBalancingV2::Listener 238 | Properties: 239 | DefaultActions: 240 | - Type: forward 241 | TargetGroupArn: !Ref NetworkLoadBalancerTargetGroup 242 | LoadBalancerArn: !Ref LoadBalancer 243 | Port: !Ref ProxyPort 244 | Protocol: TCP 245 | OutboundProxyProfile: 246 | Type: AWS::IAM::InstanceProfile 247 | Properties: 248 | Path: "/" 249 | InstanceProfileName: !Sub "Proxy-EC2-${AWS::StackName}" 250 | Roles: 251 | - !Ref OutboundProxyRole 252 | 253 | OutboundProxySecurityGroup: 254 | Type: AWS::EC2::SecurityGroup 255 | Properties: 256 | GroupDescription: Allow access to Outbound Proxy 257 | VpcId: !Ref VpcId 258 | SecurityGroupIngress: 259 | - CidrIp: !Ref NetworkAllowedCIDR 260 | FromPort: !Ref ProxyPort 261 | ToPort: !Ref ProxyPort 262 | IpProtocol: tcp 263 | 264 | OutboundProxyLaunchTemplate: 265 | Type: AWS::EC2::LaunchTemplate 266 | Metadata: 267 | AWS::CloudFormation::Init: 268 | config: 269 | files: 270 | "/root/update-dns.sh": 271 | content: !Sub | 272 | # DNS List comma delimited 273 | dns_list="${CustomDNS}" 274 | # 275 | # check if default 276 | if [[ $dns_list == "default" ]]; then 277 | exit 278 | fi 279 | # 280 | # split to list 281 | array=(${!dns_list//,/ }) 282 | int_list=`ls /etc/sysconfig/network-scripts/ifcfg-* | grep -v "\-lo$\|old$"` 283 | 284 | # for all interfaces except lookback 285 | for int in ${!int_list[@]} 286 | do 287 | # remove spaces 288 | $int=${!int//[[:blank:]]/} 289 | echo "working on $int" 290 | # make tmp file without DNS settings 291 | grep -ve "PEERDNS=\|DNS.=" $int > ./tmp.int.conf 292 | grep -v "nameserver" /etc/resolv.conf > ./tmp.resolv.conf 293 | echo "PEERDNS=yes" >> ./tmp.int.conf 294 | counter=1 295 | for i in ${!array[@]} 296 | do 297 | echo "DNS${!counter}=${!i}" >> ./tmp.int.conf 298 | echo "nameserver ${!i}" >> ./tmp.resolv.conf 299 | ((counter++)) 300 | done 301 | # update the interface config 302 | mv $int ${!int}.old 303 | cp ./tmp.int.conf $int 304 | done 305 | # update the resolv.conf 306 | mv /etc/resolv.conf /etc/resolv.conf.old 307 | cp ./tmp.resolv.conf /etc/resolv.conf 308 | echo "done" 309 | mode: '000755' 310 | owner: "root" 311 | group: "root" 312 | "/etc/awslogs/awscli.conf": 313 | content: !Sub | 314 | [plugins] 315 | cwlogs = cwlogs 316 | [default] 317 | region = ${AWS::Region} 318 | mode: '000755' 319 | owner: "root" 320 | group: "root" 321 | "/root/fetch-config-cron.sh": 322 | content: !Sub | 323 | aws secretsmanager get-secret-value --secret-id ${WhitelistedSitesSecret} --region ${AWS::Region} > ~/.tmp.hosts 324 | upstreamVersion=$(grep VersionId ~/.tmp.hosts) 325 | hostVersion=$(cat ~/configVersion) || hostVersion="0" 326 | # update if config 327 | if [[ $upstreamVersion != $hostVersion ]]; then 328 | mv /etc/squid/squid.allowed.sites.txt /etc/squid/squid.allowed.sites.txt.old 329 | grep SecretString ~/.tmp.hosts | sed 's/^.*SecretString\": \"\(.*\)\"\,/\1/' | tr -d " " | tr "," "\n" > /etc/squid/squid.allowed.sites.txt 330 | grep VersionId ~/.tmp.hosts > ~/configVersion 331 | systemctl restart squid 332 | echo "Squid config updated" 333 | logger "Squid config updated by cron-job from AWS secret store ${WhitelistedSitesSecret}" 334 | fi 335 | mode: '000755' 336 | owner: "root" 337 | group: "root" 338 | # [Previous file definitions continue...] 339 | Properties: 340 | LaunchTemplateName: !Sub "${AWS::StackName}-launch-template" 341 | LaunchTemplateData: 342 | ImageId: !Ref LatestAmiId 343 | InstanceType: !Ref InstanceType 344 | KeyName: !If 345 | - AddSSHKey 346 | - !Ref KeyName 347 | - !Ref "AWS::NoValue" 348 | SecurityGroupIds: 349 | - !Ref OutboundProxySecurityGroup 350 | IamInstanceProfile: 351 | Name: !Ref OutboundProxyProfile 352 | BlockDeviceMappings: 353 | - DeviceName: /dev/xvda 354 | Ebs: 355 | VolumeSize: 8 356 | VolumeType: gp3 357 | DeleteOnTermination: true 358 | UserData: 359 | Fn::Base64: !Sub | 360 | #!/bin/bash -xe 361 | yum -y install python-pip 362 | yum -y install python-setuptools 363 | yum install -y awscli 364 | # install squid 365 | yum install -y squid 366 | # install the CloudWatch Logs agent 367 | yum install -y awslogs 368 | # Get the latest CloudFormation package 369 | easy_install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz 370 | # Start cfn-init 371 | /opt/aws/bin/cfn-init -s ${AWS::StackId} -r OutboundProxyLaunchTemplate --region ${AWS::Region} || error_exit 'Failed to run cfn-init' 372 | # Start up the cfn-hup daemon to listen for changes to the launch configuration metadata 373 | /opt/aws/bin/cfn-hup || error_exit 'Failed to start cfn-hup' 374 | # start the cloud watch agent 375 | systemctl start awslogsd 376 | # get the IP allocation id 377 | EIPs=(${FixedEIPa.AllocationId} ${FixedEIPb.AllocationId} ${FixedEIPc.AllocationId} ${FixedEIPd.AllocationId}) 378 | for i in ${!EIPs[@]}; do 379 | out=$(aws ec2 describe-addresses --region ${AWS::Region} --allocation-ids $i) 380 | if [[ $out != *AssociationId* ]]; then 381 | freeEIP=$i 382 | break 383 | fi 384 | done 385 | # bind the address 386 | echo "binding EIP" 387 | aws ec2 associate-address --region ${AWS::Region} --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) --allocation-id $freeEIP --allow-reassociation || error_exit 'Failed to Associate Elastic IP' 388 | # generate dummy certificate 389 | openssl req -x509 -newkey rsa:4096 -keyout /etc/squid/cert.pem -out /etc/squid/cert.pem -days 3650 -subj "/C=XX/ST=XX/L=squid/O=squid/CN=squid" -nodes 390 | # get the whitelisted domain 391 | /root/fetch-config-cron.sh 392 | # start squit 393 | systemctl start squid 394 | # cron to update whitelist if needed every 5 min 395 | echo "*/5 * * * * /root/fetch-config-cron.sh" | crontab - 396 | # cron to to get and push proxy stats 397 | (crontab -l ; echo "*/5 * * * * /root/get-stats-cron.sh") | crontab - 398 | # set up DNS if needed 399 | if [[ ${CustomDNS} != "default" ]]; then 400 | /root/update-dns.sh 401 | fi 402 | # All done so signal success 403 | /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource OutboundProxyASG --region ${AWS::Region} 404 | echo "User data done" 405 | 406 | OutboundProxyASG: 407 | Type: AWS::AutoScaling::AutoScalingGroup 408 | Properties: 409 | VPCZoneIdentifier: !Ref PublicSubnetIDs 410 | Cooldown: 120 411 | LaunchTemplate: 412 | LaunchTemplateId: !Ref OutboundProxyLaunchTemplate 413 | Version: !GetAtt OutboundProxyLaunchTemplate.LatestVersionNumber 414 | MaxSize: 3 415 | MinSize: 1 416 | TargetGroupARNs: 417 | - Ref: "NetworkLoadBalancerTargetGroup" 418 | TerminationPolicies: 419 | - OldestInstance 420 | Tags: 421 | - Key: Name 422 | PropagateAtLaunch: 'true' 423 | Value: outbound-proxy 424 | - Key: AppVersion 425 | PropagateAtLaunch: 'true' 426 | Value: 1.0.0 427 | - Key: ApplicationID 428 | PropagateAtLaunch: 'true' 429 | Value: outbound-proxy 430 | ScaleOutPolicy: 431 | Type: AWS::AutoScaling::ScalingPolicy 432 | Properties: 433 | AdjustmentType: ChangeInCapacity 434 | AutoScalingGroupName: 435 | Ref: OutboundProxyASG 436 | Cooldown: '90' 437 | ScalingAdjustment: '1' 438 | 439 | CPUAlarmHigh: 440 | Type: AWS::CloudWatch::Alarm 441 | Properties: 442 | EvaluationPeriods: '1' 443 | Statistic: Average 444 | Threshold: '80' 445 | AlarmDescription: Alarm if CPU too high (50%) or metric disappears indicating instance 446 | is down 447 | Period: '60' 448 | AlarmActions: 449 | - Ref: ScaleOutPolicy 450 | Namespace: AWS/EC2 451 | Dimensions: 452 | - Name: AutoScalingGroupName 453 | Value: 454 | Ref: OutboundProxyASG 455 | ComparisonOperator: GreaterThanThreshold 456 | MetricName: CPUUtilization 457 | 458 | ScaleInPolicy: 459 | Type: AWS::AutoScaling::ScalingPolicy 460 | Properties: 461 | AdjustmentType: ChangeInCapacity 462 | AutoScalingGroupName: 463 | Ref: OutboundProxyASG 464 | Cooldown: '90' 465 | ScalingAdjustment: '-1' 466 | 467 | CPUAlarmLow: 468 | Type: AWS::CloudWatch::Alarm 469 | Properties: 470 | EvaluationPeriods: '1' 471 | Statistic: Average 472 | Threshold: '10' 473 | AlarmDescription: Alarm if CPU low (10%) or metric disappears indicating instance 474 | is down 475 | Period: '60' 476 | AlarmActions: 477 | - Ref: ScaleInPolicy 478 | Namespace: AWS/EC2 479 | Dimensions: 480 | - Name: AutoScalingGroupName 481 | Value: 482 | Ref: OutboundProxyASG 483 | ComparisonOperator: LessThanThreshold 484 | MetricName: CPUUtilization 485 | 486 | OutboundProxyLogGroup: 487 | Type: AWS::Logs::LogGroup 488 | Properties: 489 | RetentionInDays: 30 490 | LogGroupName: !Sub "Proxy-${AWS::StackName}" 491 | 492 | Outputs: 493 | CloudWatchLogGroupName: 494 | Description: The name of the CloudWatch log group for outbound proxy 495 | Value: !Ref OutboundProxyLogGroup 496 | Export: 497 | Name: Proxy-CloudWatchLogGroupName 498 | 499 | OutboundProxyDomain: 500 | Description: Proxy DNS name to be used in the clients 501 | Value: !GetAtt LoadBalancer.DNSName 502 | Export: 503 | Name: Proxy-Domain 504 | 505 | OutboundProxyPort: 506 | Description: Port of the Proxy 507 | Value: !Ref ProxyPort 508 | Export: 509 | Name: Proxy-Port 510 | 511 | EgressIP1: 512 | Description: Outbound Proxy source IP (first) 513 | Value: !Ref FixedEIPa 514 | Export: 515 | Name: Proxy-Egress-IP-1 516 | 517 | EgressIP2: 518 | Description: Outbound Proxy source IP (second) 519 | Value: !Ref FixedEIPb 520 | Export: 521 | Name: Proxy-Egress-IP-2 522 | 523 | EgressIP3: 524 | Description: Outbound Proxy source IP (third) 525 | Value: !Ref FixedEIPc 526 | Export: 527 | Name: Proxy-Egress-IP-3 528 | 529 | EgressIP4: 530 | Description: Outbound Proxy source IP (fourth) 531 | Value: !Ref FixedEIPd 532 | Export: 533 | Name: Proxy-Egress-IP-4 534 | 535 | SecurityGroupProxy: 536 | Description: Proxy security group 537 | Value: SecurityGroup_Proxy 538 | Export: 539 | Name: Proxy-SecurityGroup 540 | 541 | LinuxProxySettings: 542 | Description: Linux proxy settings. Copy and paste to your shell to set the proxy 543 | Value: !Sub "export http_proxy=http://${LoadBalancer.DNSName}:${ProxyPort} && export https_proxy=$http_proxy" 544 | Export: 545 | Name: LinuxProxySettings 546 | -------------------------------------------------------------------------------- /sources/templates/outbound-proxy-CF.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | Description: Outbound filtering proxy 5 | 6 | Parameters: 7 | 8 | WhitelistedDomains: 9 | Type: String 10 | Default: .amazonaws.com, .debian.org 11 | Description: Whitelisted domains comma separated 12 | 13 | CustomDNS: 14 | Type: String 15 | Default: default 16 | Description: Provide optional a DNS server for domain filtering, like OpenDNS (comma separated, like 8.8.8.8,8.8.8.7) 17 | 18 | KeyName: 19 | Type: "String" 20 | Description: Name of RSA key for EC2 access for testing only. 21 | Default: '' 22 | 23 | ProxyPort: 24 | Type: String 25 | Default: 3128 26 | Description: Port Proxy 27 | 28 | 29 | VpcId: 30 | Description: VPC ID Where the Proxy will be installed 31 | Type: "AWS::EC2::VPC::Id" 32 | 33 | PrivateSubnetIDs: 34 | Description: Private SubnetIDs where the Network LoadBalancer will be placed (Select min 2 max 3) 35 | Type: "List" 36 | 37 | PublicSubnetIDs: 38 | Description: Public SubnetIDs where the proxy will be placed (Select min 2 max 3) 39 | Type: "List" 40 | 41 | InstanceType: 42 | Description: WebServer EC2 instance type 43 | Type: String 44 | Default: t3.medium 45 | AllowedValues: 46 | - t3.nano 47 | - t3.micro 48 | - t3.small 49 | - t3.medium 50 | - t3.large 51 | - m3.medium 52 | - m3.large 53 | - m3.xlarge 54 | - m3.2xlarge 55 | - m4.large 56 | - m4.xlarge 57 | - m4.2xlarge 58 | - m5.large 59 | - m5.xlarge 60 | - m5.2xlarge 61 | - c3.large 62 | - c3.xlarge 63 | - c4.large 64 | ConstraintDescription: must be a valid EC2 instance type. 65 | 66 | NetworkAllowedCIDR: 67 | Description: CIDR allowed in Proxy Security Group. The allowed block size is between a /32 netmask and /8 netmask 68 | Type: String 69 | Default: 172.31.0.0/16 70 | AllowedPattern: ^[.0-9]*\/([89]|[12][0-9]|3[0-2])$ 71 | 72 | LatestAmiId: 73 | Type: 'AWS::SSM::Parameter::Value' 74 | Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2' 75 | Description: AMI ID pointer in SSM. Default latest AMI Amazon Linux2. 76 | 77 | Metadata: 78 | 'AWS::CloudFormation::Interface': 79 | ParameterGroups: 80 | - Label: 81 | default: Proxy parameter 82 | Parameters: 83 | - WhitelistedDomains 84 | - CustomDNS 85 | - ProxyPort 86 | - InstanceType 87 | - LatestAmiId 88 | - KeyName 89 | 90 | - Label: 91 | default: Network parameter 92 | Parameters: 93 | - VpcId 94 | - PrivateSubnetIDs 95 | - PublicSubnetIDs 96 | - NetworkAllowedCIDR 97 | 98 | ParameterLabels: 99 | WhitelistedDomains: 100 | default: Allowed domains (whitelisted) 101 | CustomDNS: 102 | default: Custom DNS servers 103 | ProxyPort: 104 | default: Proxy Port 105 | InstanceType: 106 | default: Instance Type 107 | LatestAmiId: 108 | default: AMI ID 109 | KeyName: 110 | default: SSH Key name 111 | VpcId: 112 | default: VPC ID 113 | PrivateSubnetIDs: 114 | default: Private Subnet IDs 115 | PublicSubnetIDs: 116 | default: Public Subnet IDs 117 | NetworkAllowedCIDR: 118 | default: Allowed client CIRD 119 | 120 | 121 | Conditions: 122 | 123 | AddSSHKey: !Not 124 | - !Equals 125 | - '' 126 | - !Ref KeyName 127 | 128 | Resources: 129 | 130 | OutboundProxyRole: 131 | Type: AWS::IAM::Role 132 | Properties: 133 | RoleName: !Sub "Outbound-proxy-${AWS::StackName}" 134 | AssumeRolePolicyDocument: 135 | Version: '2012-10-17' 136 | Statement: 137 | - Effect: Allow 138 | Principal: 139 | Service: 140 | - ec2.amazonaws.com 141 | Action: 142 | - sts:AssumeRole 143 | Path: "/" 144 | Policies: 145 | - PolicyName: LogRolePolicy 146 | PolicyDocument: 147 | Version: '2012-10-17' 148 | Statement: 149 | - Effect: Allow 150 | Action: 151 | - logs:CreateLogGroup 152 | - logs:CreateLogStream 153 | - logs:PutLogEvents 154 | - logs:DescribeLogStreams 155 | Resource: 156 | - !GetAtt OutboundProxyLogGroup.Arn 157 | - PolicyName: AssociateEIP 158 | PolicyDocument: 159 | Version: '2012-10-17' 160 | Statement: 161 | - Effect: Allow 162 | Action: 163 | - ec2:AssociateAddress 164 | - ec2:Describe* 165 | Resource: 166 | - "*" 167 | - PolicyName: RevokeAuthorizeSG 168 | PolicyDocument: 169 | Version: '2012-10-17' 170 | Statement: 171 | - Effect: Allow 172 | Action: 173 | - ec2:RevokeSecurityGroupIngress 174 | - ec2:AuthorizeSecurityGroupIngress 175 | - ec2:Describe* 176 | Resource: 177 | - "*" 178 | - PolicyName: GetSecret 179 | PolicyDocument: 180 | Version: '2012-10-17' 181 | Statement: 182 | - Effect: Allow 183 | Action: 184 | - secretsmanager:GetSecretValue 185 | Resource: 186 | - !Ref WhitelistedSitesSecret 187 | - PolicyName: CloudWatchMetric 188 | PolicyDocument: 189 | Version: '2012-10-17' 190 | Statement: 191 | - Effect: Allow 192 | Action: 193 | - cloudwatch:PutMetricData 194 | Resource: 195 | - "*" 196 | 197 | WhitelistedSitesSecret: 198 | Type: 'AWS::SecretsManager::Secret' 199 | Properties: 200 | Name: Proxy-Domains-Whitelisting 201 | Description: This secret contains the proxy whitelisted domains 202 | SecretString: !Ref WhitelistedDomains 203 | 204 | FixedEIPa: 205 | Type: AWS::EC2::EIP 206 | Properties: 207 | Domain: vpc 208 | 209 | FixedEIPb: 210 | Type: AWS::EC2::EIP 211 | Properties: 212 | Domain: vpc 213 | 214 | FixedEIPc: 215 | Type: AWS::EC2::EIP 216 | Properties: 217 | Domain: vpc 218 | 219 | FixedEIPd: 220 | Type: AWS::EC2::EIP 221 | Properties: 222 | Domain: vpc 223 | 224 | LoadBalancer: 225 | Type: AWS::ElasticLoadBalancingV2::LoadBalancer 226 | Properties: 227 | Scheme: internal 228 | Type: network 229 | Name: OutboundProxyLoadBalancer 230 | Subnets: !Ref PrivateSubnetIDs 231 | 232 | NetworkLoadBalancerTargetGroup: 233 | Type: AWS::ElasticLoadBalancingV2::TargetGroup 234 | Properties: 235 | Name: OutboundProxyTargetGroup 236 | Port: !Ref ProxyPort 237 | Protocol: TCP 238 | VpcId: !Ref VpcId 239 | TargetGroupAttributes: 240 | - Key: deregistration_delay.timeout_seconds 241 | Value: 60 242 | Tags: 243 | - Key: Name 244 | Value: SMARTProxyTargetGroup 245 | LoadBalancerListenerHTTPS: 246 | Type: AWS::ElasticLoadBalancingV2::Listener 247 | Properties: 248 | DefaultActions: 249 | - Type: forward 250 | TargetGroupArn: !Ref NetworkLoadBalancerTargetGroup 251 | LoadBalancerArn: !Ref LoadBalancer 252 | Port: !Ref ProxyPort 253 | Protocol: TCP 254 | 255 | OutboundProxyProfile: 256 | Type: AWS::IAM::InstanceProfile 257 | Properties: 258 | Path: "/" 259 | InstanceProfileName: !Sub "Proxy-EC2-${AWS::StackName}" 260 | Roles: 261 | - !Ref OutboundProxyRole 262 | 263 | OutboundProxySecurityGroup: 264 | Type: AWS::EC2::SecurityGroup 265 | Properties: 266 | GroupDescription: Allow access to Outbound Proxy 267 | VpcId: !Ref VpcId 268 | SecurityGroupIngress: 269 | - CidrIp: !Ref NetworkAllowedCIDR 270 | FromPort: !Ref ProxyPort 271 | ToPort: !Ref ProxyPort 272 | IpProtocol: tcp 273 | 274 | OutboundProxyASG: 275 | Type: AWS::AutoScaling::AutoScalingGroup 276 | Properties: 277 | VPCZoneIdentifier: !Ref PublicSubnetIDs 278 | Cooldown: 120 279 | LaunchConfigurationName: !Ref OutboundProxyLaunchConfig 280 | MaxSize: 3 281 | MinSize: 1 282 | TargetGroupARNs: 283 | - Ref: "NetworkLoadBalancerTargetGroup" 284 | TerminationPolicies: 285 | - OldestInstance 286 | Tags: 287 | - Key: Name 288 | PropagateAtLaunch: 'true' 289 | Value: outbound-proxy 290 | - Key: AppVersion 291 | PropagateAtLaunch: 'true' 292 | Value: 1.0.0 293 | - Key: ApplicationID 294 | PropagateAtLaunch: 'true' 295 | Value: outbound-proxy 296 | CreationPolicy: 297 | ResourceSignal: 298 | Timeout: PT15M 299 | Count: '1' 300 | UpdatePolicy: 301 | AutoScalingScheduledAction: 302 | IgnoreUnmodifiedGroupSizeProperties: true 303 | AutoScalingRollingUpdate: 304 | MinInstancesInService: 1 305 | MaxBatchSize: 1 306 | PauseTime: PT15M 307 | WaitOnResourceSignals: 'true' 308 | SuspendProcesses: 309 | - ScheduledActions 310 | 311 | ScaleOutPolicy: 312 | Type: AWS::AutoScaling::ScalingPolicy 313 | Properties: 314 | AdjustmentType: ChangeInCapacity 315 | AutoScalingGroupName: 316 | Ref: OutboundProxyASG 317 | Cooldown: '90' 318 | ScalingAdjustment: '1' 319 | 320 | CPUAlarmHigh: 321 | Type: AWS::CloudWatch::Alarm 322 | Properties: 323 | EvaluationPeriods: '1' 324 | Statistic: Average 325 | Threshold: '80' 326 | AlarmDescription: Alarm if CPU too high (50%) or metric disappears indicating instance 327 | is down 328 | Period: '60' 329 | AlarmActions: 330 | - Ref: ScaleOutPolicy 331 | Namespace: AWS/EC2 332 | Dimensions: 333 | - Name: AutoScalingGroupName 334 | Value: 335 | Ref: OutboundProxyASG 336 | ComparisonOperator: GreaterThanThreshold 337 | MetricName: CPUUtilization 338 | 339 | ScaleInPolicy: 340 | Type: AWS::AutoScaling::ScalingPolicy 341 | Properties: 342 | AdjustmentType: ChangeInCapacity 343 | AutoScalingGroupName: 344 | Ref: OutboundProxyASG 345 | Cooldown: '90' 346 | ScalingAdjustment: '-1' 347 | 348 | CPUAlarmLow: 349 | Type: AWS::CloudWatch::Alarm 350 | Properties: 351 | EvaluationPeriods: '1' 352 | Statistic: Average 353 | Threshold: '10' 354 | AlarmDescription: Alarm if CPU low (10%) or metric disappears indicating instance 355 | is down 356 | Period: '60' 357 | AlarmActions: 358 | - Ref: ScaleInPolicy 359 | Namespace: AWS/EC2 360 | Dimensions: 361 | - Name: AutoScalingGroupName 362 | Value: 363 | Ref: OutboundProxyASG 364 | ComparisonOperator: LessThanThreshold 365 | MetricName: CPUUtilization 366 | 367 | OutboundProxyLaunchConfig: 368 | Type: AWS::AutoScaling::LaunchConfiguration 369 | Metadata: 370 | Comment: Configures Outbound Proxy 371 | AWS::CloudFormation::Init: 372 | config: 373 | files: 374 | "/root/update-dns.sh": 375 | content: !Sub | 376 | # DNS List comma delimited 377 | dns_list="${CustomDNS}" 378 | # 379 | # check if default 380 | if [[ $dns_list == "default" ]]; then 381 | exit 382 | fi 383 | # 384 | # split to list 385 | array=(${!dns_list//,/ }) 386 | int_list=`ls /etc/sysconfig/network-scripts/ifcfg-* | grep -v "\-lo$\|old$"` 387 | 388 | # for all interfaces except lookback 389 | for int in ${!int_list[@]} 390 | do 391 | # remove spaces 392 | $int=${!int//[[:blank:]]/} 393 | echo "working on $int" 394 | # make tmp file without DNS settings 395 | grep -ve "PEERDNS=\|DNS.=" $int > ./tmp.int.conf 396 | grep -v "nameserver" /etc/resolv.conf > ./tmp.resolv.conf 397 | echo "PEERDNS=yes" >> ./tmp.int.conf 398 | counter=1 399 | for i in ${!array[@]} 400 | do 401 | echo "DNS${!counter}=${!i}" >> ./tmp.int.conf 402 | echo "nameserver ${!i}" >> ./tmp.resolv.conf 403 | ((counter++)) 404 | done 405 | # update the interface config 406 | mv $int ${!int}.old 407 | cp ./tmp.int.conf $int 408 | done 409 | # update the resolv.conf 410 | mv /etc/resolv.conf /etc/resolv.conf.old 411 | cp ./tmp.resolv.conf /etc/resolv.conf 412 | # clear squid cache if squid is running. Relevant for dns content filtering 413 | # systemctl status squid && systemctl stop squid && rm -rf /var/spool/squid/ && squid -z && systemctl start squid 414 | echo "done" 415 | mode: '000755' 416 | owner: "root" 417 | group: "root" 418 | "/etc/awslogs/awscli.conf": 419 | content: !Sub | 420 | [plugins] 421 | cwlogs = cwlogs 422 | [default] 423 | region = ${AWS::Region} 424 | mode: '000755' 425 | owner: "root" 426 | group: "root" 427 | "/root/fetch-config-cron.sh": 428 | content: !Sub | 429 | aws secretsmanager get-secret-value --secret-id ${WhitelistedSitesSecret} --region ${AWS::Region} > ~/.tmp.hosts 430 | upstreamVersion=$(grep VersionId ~/.tmp.hosts) 431 | hostVersion=$(cat ~/configVersion) || hostVersion="0" 432 | # update if config 433 | if [[ $upstreamVersion != $hostVersion ]]; then 434 | mv /etc/squid/squid.allowed.sites.txt /etc/squid/squid.allowed.sites.txt.old 435 | grep SecretString ~/.tmp.hosts | sed 's/^.*SecretString\": \"\(.*\)\"\,/\1/' | tr -d " " | tr "," "\n" > /etc/squid/squid.allowed.sites.txt 436 | grep VersionId ~/.tmp.hosts > ~/configVersion 437 | systemctl restart squid 438 | echo "Squid config updated" 439 | logger "Squid config updated by cron-job from AWS secret store ${WhitelistedSitesSecret}" 440 | fi 441 | mode: '000755' 442 | owner: "root" 443 | group: "root" 444 | "/root/get-stats-cron.sh": 445 | content: !Sub | 446 | #!/bin/bash 447 | # 448 | # Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved. 449 | # SPDX-License-Identifier: MIT-0# 450 | # 451 | # gets statistics from squid proxy and pushes them to CloudWatch 452 | # 453 | ### 454 | region=`curl --silent http://169.254.169.254/latest/dynamic/instance-identity/document | grep region | cut -f 4 -d '"'` 455 | instanceId=`curl --silent http://169.254.169.254/latest/meta-data/instance-id` 456 | 457 | squidclient -h localhost cache_object://localhost/ mgr:5min | grep "client_http.request\|client_http.hits\|client_http.errors\|client_http.kbytes_in\|client_http.kbytes_out\|server.all." | while read line ; do 458 | name=`echo $line | cut -d "=" -f 1` 459 | value=`echo $line | cut -d "=" -f 2 | sed "s/[^0-9\.]*//g" ` 460 | aws cloudwatch put-metric-data --metric-name "$name" --namespace Proxy --dimensions InstanceID="$instanceId" --value "$value" --region $region 461 | done 462 | mode: '000755' 463 | owner: "root" 464 | group: "root" 465 | "/etc/squid/squid.allowed.sites.txt": 466 | content: | 467 | .amazon.com 468 | mode: '000400' 469 | owner: "root" 470 | group: "root" 471 | "/etc/squid/squid.conf": 472 | content: !Sub | 473 | # Recommended minimum configuration: 474 | # 475 | 476 | # Example rule allowing access from your local networks. 477 | # Adapt to list your (internal) IP networks from where browsing 478 | # should be allowed 479 | acl localnet src 10.0.0.0/8 # RFC1918 possible internal network 480 | acl localnet src 172.16.0.0/12 # RFC1918 possible internal network 481 | acl localnet src 192.168.0.0/16 # RFC1918 possible internal network 482 | acl localnet src fc00::/7 # RFC 4193 local private network range 483 | acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines 484 | acl localnet src 127.0.0.1 485 | 486 | # The Instance Metadata Service 487 | # (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html#instance-metadata-limiting-access) 488 | acl imds dst 169.254.169.254 489 | 490 | 491 | acl SSL_ports port 443 492 | acl Safe_ports port 80 # http 493 | acl Safe_ports port 21 # ftp 494 | acl Safe_ports port 443 # https 495 | acl Safe_ports port 70 # gopher 496 | acl Safe_ports port 210 # wais 497 | acl Safe_ports port 1025-65535 # unregistered ports 498 | acl Safe_ports port 280 # http-mgmt 499 | acl Safe_ports port 488 # gss-http 500 | acl Safe_ports port 591 # filemaker 501 | acl Safe_ports port 777 # multiling http 502 | acl CONNECT method CONNECT 503 | 504 | # 505 | # Recommended minimum Access Permission configuration: 506 | # 507 | # Deny requests to the Instance Metadata Service 508 | http_access deny imds 509 | 510 | # Deny requests to certain unsafe ports 511 | http_access deny !Safe_ports 512 | 513 | # Deny CONNECT to other than secure SSL ports 514 | http_access deny CONNECT !SSL_ports 515 | 516 | # Only allow cachemgr access from localhost 517 | http_access allow localhost manager 518 | http_access deny manager 519 | 520 | # Deny requests to services running on localhost 521 | http_access deny to_localhost 522 | 523 | # 524 | # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS 525 | # 526 | 527 | # Example rule allowing access from your local networks. 528 | # Adapt localnet in the ACL section to list your (internal) IP networks 529 | # from where browsing should be allowed 530 | acl allowed_http_sites dstdomain "/etc/squid/squid.allowed.sites.txt" 531 | http_access allow allowed_http_sites 532 | #http_access allow localnet 533 | #http_access allow localhost 534 | 535 | # And finally deny all other access to this proxy 536 | http_access deny all 537 | 538 | # Squid normally listens to port 3128, but needs to be parametrized here 539 | http_port 0.0.0.0:${ProxyPort} ssl-bump cert=/etc/squid/cert.pem 540 | acl allowed_https_sites ssl::server_name "/etc/squid/squid.allowed.sites.txt" 541 | acl step1 at_step SslBump1 542 | acl step2 at_step SslBump2 543 | acl step3 at_step SslBump3 544 | ssl_bump peek step1 all 545 | ssl_bump peek step2 allowed_https_sites 546 | ssl_bump splice step3 allowed_https_sites 547 | ssl_bump terminate step2 all 548 | 549 | # Uncomment and adjust the following to add a disk cache directory. 550 | #cache_dir ufs /var/spool/squid 100 16 256 551 | 552 | # Leave coredumps in the first cache dir 553 | coredump_dir /var/spool/squid 554 | # 555 | # Add any of your own refresh_pattern entries above these. 556 | # 557 | refresh_pattern ^ftp: 1440 20% 10080 558 | refresh_pattern ^gopher: 1440 0% 1440 559 | refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 560 | refresh_pattern . 0 20% 4320 561 | "/etc/awslogs/awslogs.conf": 562 | content: !Sub | 563 | [general] 564 | state_file = /var/lib/awslogs/agent-state 565 | [/var/log/squid/access.log] 566 | file = /var/log/squid/access.log 567 | log_group_name = ${OutboundProxyLogGroup} 568 | log_stream_name = {instance_id}/squid_access.log 569 | #datetime_format = %d/%b/%Y:%H:%M:%S 570 | mode: '000400' 571 | owner: "root" 572 | group: "root" 573 | "/etc/cfn/cfn-hup.conf": 574 | content: !Sub | 575 | [main] 576 | stack= ${AWS::StackId} 577 | region=${AWS::Region} 578 | interval=5 579 | mode: "000400" 580 | owner: "root" 581 | group: "root" 582 | "/etc/cfn/hooks.d/cfn-auto-reloader.conf": 583 | content: !Sub | 584 | [cfn-auto-reloader-hook] 585 | triggers=post.update 586 | path=Resources.OutboundProxyLaunchConfig.Metadata.AWS::CloudFormation::Init 587 | action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource OutboundProxyLaunchConfig --region ${AWS::Region} 588 | runas=root 589 | mode: "000400" 590 | owner: "root" 591 | group: "root" 592 | Properties: 593 | AssociatePublicIpAddress: True 594 | ImageId: !Ref LatestAmiId 595 | InstanceType: !Ref InstanceType 596 | KeyName: !If 597 | - AddSSHKey 598 | - !Ref KeyName 599 | - !Ref "AWS::NoValue" 600 | SecurityGroups: 601 | - !Ref OutboundProxySecurityGroup 602 | 603 | IamInstanceProfile: 604 | Ref: OutboundProxyProfile 605 | UserData: 606 | Fn::Base64: !Sub | 607 | #!/bin/bash -xe 608 | yum -y install python-pip 609 | yum -y install python-setuptools 610 | yum install -y awscli 611 | # install squid 612 | yum install -y squid 613 | # install the CloudWatch Logs agent 614 | yum install -y awslogs 615 | # Get the latest CloudFormation package 616 | easy_install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz 617 | # Start cfn-init 618 | /opt/aws/bin/cfn-init -s ${AWS::StackId} -r OutboundProxyLaunchConfig --region ${AWS::Region} || error_exit 'Failed to run cfn-init' 619 | # Start up the cfn-hup daemon to listen for changes to the launch configuration metadata 620 | /opt/aws/bin/cfn-hup || error_exit 'Failed to start cfn-hup' 621 | # start the cloud watch agent 622 | systemctl start awslogsd 623 | # get the IP allocation id 624 | EIPs=(${FixedEIPa.AllocationId} ${FixedEIPb.AllocationId} ${FixedEIPc.AllocationId} ${FixedEIPd.AllocationId}) 625 | for i in ${!EIPs[@]}; do 626 | out=$(aws ec2 describe-addresses --region ${AWS::Region} --allocation-ids $i) 627 | if [[ $out != *AssociationId* ]]; then 628 | freeEIP=$i 629 | break 630 | fi 631 | done 632 | # bind the address 633 | echo "binding EIP" 634 | aws ec2 associate-address --region ${AWS::Region} --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) --allocation-id $freeEIP --allow-reassociation || error_exit 'Failed to Associate Elastic IP' 635 | # generate dummy certificate 636 | openssl req -x509 -newkey rsa:4096 -keyout /etc/squid/cert.pem -out /etc/squid/cert.pem -days 3650 -subj "/C=XX/ST=XX/L=squid/O=squid/CN=squid" -nodes 637 | # get the whitelisted domain 638 | /root/fetch-config-cron.sh 639 | # start squit 640 | systemctl start squid 641 | # cron to update whitelist if needed every 5 min 642 | echo "*/5 * * * * /root/fetch-config-cron.sh" | crontab - 643 | # cron to to get and push proxy stats 644 | (crontab -l ; echo "*/5 * * * * /root/get-stats-cron.sh") | crontab - 645 | # set up DNS if needed 646 | if [[ ${CustomDNS} != "default" ]]; then 647 | /root/update-dns.sh 648 | fi 649 | # All done so signal success 650 | /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource OutboundProxyASG --region ${AWS::Region} 651 | echo "User data done" 652 | 653 | OutboundProxyLogGroup: 654 | Type: AWS::Logs::LogGroup 655 | Properties: 656 | RetentionInDays: 30 657 | LogGroupName: !Sub "Proxy-${AWS::StackName}" 658 | 659 | Outputs: 660 | CloudWatchLogGroupName: 661 | Description: The name of the CloudWatch log group for outbound proxy 662 | Value: !Ref OutboundProxyLogGroup 663 | Export: 664 | Name: Proxy-CloudWatchLogGroupName 665 | 666 | OutboundProxyDomain: 667 | Description: Proxy DNS name to be used in the clients 668 | Value: !GetAtt LoadBalancer.DNSName 669 | Export: 670 | Name: Proxy-Domain 671 | 672 | OutboundProxyPort: 673 | Description: Port of the Proxy 674 | Value: !Ref ProxyPort 675 | Export: 676 | Name: Proxy-Port 677 | 678 | EgressIP1: 679 | Description: Outbound Proxy source IP (first) 680 | Value: !Ref FixedEIPa 681 | Export: 682 | Name: Proxy-Egress-IP-1 683 | EgressIP2: 684 | Description: Outbound Proxy source IP (second) 685 | Value: !Ref FixedEIPb 686 | Export: 687 | Name: Proxy-Egress-IP-2 688 | EgressIP3: 689 | Description: Outbound Proxy source IP (third) 690 | Value: !Ref FixedEIPc 691 | Export: 692 | Name: Proxy-Egress-IP-3 693 | EgressIP4: 694 | Description: Outbound Proxy source IP (fourth) 695 | Value: !Ref FixedEIPd 696 | Export: 697 | Name: Proxy-Egress-IP-4 698 | 699 | SecurityGroupProxy: 700 | Description: Proxy security group 701 | Value: SecurityGroup_Proxy 702 | Export: 703 | Name: Proxy-SecurityGroup 704 | 705 | LinuxProxySettings: 706 | Description: Linux proxy settings. Copy and paste to your shell to set the proxy 707 | Value: !Sub "export http_proxy=http://${LoadBalancer.DNSName}:${ProxyPort} && export https_proxy=$http_proxy" 708 | Export: 709 | Name: LinuxProxySettings 710 | 711 | 712 | 713 | 714 | 715 | --------------------------------------------------------------------------------