├── .gitignore ├── images ├── mirroring.png └── splitting-mirroring.png ├── CODE_OF_CONDUCT.md ├── LICENSE ├── THIRD-PARTY-NOTICE ├── README.md ├── CONTRIBUTING.md ├── main.go └── replay-handler-cloudformation.yaml /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | **/.DS_Store -------------------------------------------------------------------------------- /images/mirroring.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/http-requests-mirroring/HEAD/images/mirroring.png -------------------------------------------------------------------------------- /images/splitting-mirroring.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/http-requests-mirroring/HEAD/images/splitting-mirroring.png -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2012 Google, Inc. All rights reserved. 2 | Copyright (c) 2009-2011 Andreas Krennmair. All rights reserved. 3 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | 5 | Redistribution and use in source and binary forms, with or without 6 | modification, are permitted provided that the following conditions are 7 | met: 8 | 9 | * Redistributions of source code must retain the above copyright 10 | notice, this list of conditions and the following disclaimer. 11 | * Redistributions in binary form must reproduce the above 12 | copyright notice, this list of conditions and the following disclaimer 13 | in the documentation and/or other materials provided with the 14 | distribution. 15 | * Neither the name of Andreas Krennmair, Google, nor the names of its 16 | contributors may be used to endorse or promote products derived from 17 | this software without specific prior written permission. 18 | 19 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 20 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 21 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 22 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 23 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 24 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 25 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 26 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 27 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 28 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -------------------------------------------------------------------------------- /THIRD-PARTY-NOTICE: -------------------------------------------------------------------------------- 1 | ** @gopacket -- 2 | https://github.com/google/gopacket 3 | Copyright (c) 2012 Google, Inc. All rights reserved. 4 | Copyright (c) 2009-2011 Andreas Krennmair. All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are 8 | met: 9 | 10 | * Redistributions of source code must retain the above copyright 11 | notice, this list of conditions and the following disclaimer. 12 | * Redistributions in binary form must reproduce the above 13 | copyright notice, this list of conditions and the following disclaimer 14 | in the documentation and/or other materials provided with the 15 | distribution. 16 | * Neither the name of Andreas Krennmair, Google, nor the names of its 17 | contributors may be used to endorse or promote products derived from 18 | this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 21 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 22 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 23 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 24 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 25 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 26 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 27 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 28 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 29 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 30 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 31 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Mirror production traffic to test environment with VPC Traffic Mirroring 2 | 3 | This repository contains the artifacts for the AWS blog post [Mirror production traffic to test environment with VPC Traffic Mirroring](https://aws.amazon.com/blogs/networking-and-content-delivery/mirror-production-traffic-to-test-environment-with-vpc-traffic-mirroring/). 4 | 5 | ![The diagram shows how requests for the production environment are replied to the test environment](images/mirroring.png) 6 | 7 | ## Additional Considerations 8 | 9 | #### Parameters 10 | 11 | When creating the stack, you can optionally specify additional parameters. For example, you can use the parameter “ForwardPercentage” to define the percentage of requests that are replicated (by default, this is 100%). You can even choose to only replicate requests coming from a percentage of header values or remote addresses - for example, to mirror all requests that come from only a percentage of users (rather than a percentage of requests from all users). To do that, set the parameter “PercentageBy” to “header” or “remoteaddr”. When “PercentageBy” is set to “header”, you need to provide the header name in the parameter “PercentageByHeader”. 12 | 13 | #### X-Forwarded headers 14 | 15 | When the replay handler generates new requests, it manupulates the following headers: 16 | - X-Forwarded-For: appends the IP of the client or the IP of the latest proxy. 17 | - X-Forwarded-Port: sets it to the outermost port from the chain of client and proxies. 18 | - X-Forwarded-Proto: sets it to the outermost protocol from the chain of client and proxies. 19 | - X-Forwarded-Host: sets it to the outermost host from the chain of client and proxies. 20 | 21 | #### Protocols support 22 | 23 | The only protocol supported is HTTP. HTTPS is not supported. Therefore, SSL offloading should happen before the traffic reaches the EC2 instances in the production environment. 24 | 25 | #### Scaling up the EC2 instances in the replay handler 26 | 27 | If you increase the number of instances in the autoscaling group, traffic may get unbalanced in some cases due to how Network Load Balancer flow hash algorithm works. This may happen during scale out operations in the replay handler. To prevent this from happening, when a scale out action is needed from n to m instances (e.g. from 3 to 4), you can scale out to n+m first (e.g. to 3+4=7) and then scale in to m (e.g. 4). You can do this operation with two subsequent updates of the "InstanceNumber" parameter of the CloudFormation Stack. The CloudFormation template provided is already configured to remove the oldest instances first, so that traffic is re-distributed equally to the newer instances. 28 | 29 | ## Security 30 | 31 | See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. 32 | 33 | ## License 34 | 35 | This library is licensed under the BSD-3-Clause License. See the LICENSE file. 36 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *main* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | -------------------------------------------------------------------------------- /main.go: -------------------------------------------------------------------------------- 1 | // Original Copyright 2012 Google, Inc. All rights reserved. 2 | // 3 | // Use of this source code is governed by a BSD-style license 4 | // that can be found in the LICENSE file in the root of the source 5 | // tree. 6 | 7 | // Modification Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 8 | // SPDX-License-Identifier: BSD-3-Clause 9 | 10 | package main 11 | 12 | import ( 13 | "bufio" 14 | "bytes" 15 | crypto_rand "crypto/rand" 16 | "encoding/binary" 17 | "flag" 18 | "fmt" 19 | "hash/crc64" 20 | "io" 21 | "io/ioutil" 22 | "log" 23 | math_rand "math/rand" 24 | "net" 25 | "net/http" 26 | "os" 27 | "time" 28 | 29 | "github.com/google/gopacket" 30 | "github.com/google/gopacket/examples/util" 31 | "github.com/google/gopacket/layers" 32 | "github.com/google/gopacket/pcap" 33 | "github.com/google/gopacket/tcpassembly" 34 | "github.com/google/gopacket/tcpassembly/tcpreader" 35 | ) 36 | 37 | var fwdDestination = flag.String("destination", "", "Destination of the forwarded requests.") 38 | var fwdPerc = flag.Float64("percentage", 100, "Must be between 0 and 100.") 39 | var fwdBy = flag.String("percentage-by", "", "Can be empty. Otherwise, valid values are: header, remoteaddr.") 40 | var fwdHeader = flag.String("percentage-by-header", "", "If percentage-by is header, then specify the header here.") 41 | var reqPort = flag.Int("filter-request-port", 80, "Must be between 0 and 65535.") 42 | var keepHostHeader = flag.Bool("keep-host-header", false, "Keep Host header from original request.") 43 | 44 | // Build a simple HTTP request parser using tcpassembly.StreamFactory and tcpassembly.Stream interfaces 45 | 46 | // httpStreamFactory implements tcpassembly.StreamFactory 47 | type httpStreamFactory struct{} 48 | 49 | // httpStream will handle the actual decoding of http requests. 50 | type httpStream struct { 51 | net, transport gopacket.Flow 52 | r tcpreader.ReaderStream 53 | } 54 | 55 | func (h *httpStreamFactory) New(net, transport gopacket.Flow) tcpassembly.Stream { 56 | hstream := &httpStream{ 57 | net: net, 58 | transport: transport, 59 | r: tcpreader.NewReaderStream(), 60 | } 61 | go hstream.run() // Important... we must guarantee that data from the reader stream is read. 62 | 63 | // ReaderStream implements tcpassembly.Stream, so we can return a pointer to it. 64 | return &hstream.r 65 | } 66 | 67 | func (h *httpStream) run() { 68 | buf := bufio.NewReader(&h.r) 69 | for { 70 | req, err := http.ReadRequest(buf) 71 | if err == io.EOF { 72 | // We must read until we see an EOF... very important! 73 | return 74 | } else if err != nil { 75 | log.Println("Error reading stream", h.net, h.transport, ":", err) 76 | } else { 77 | reqSourceIP := h.net.Src().String() 78 | reqDestionationPort := h.transport.Dst().String() 79 | body, bErr := ioutil.ReadAll(req.Body) 80 | if bErr != nil { 81 | return 82 | } 83 | req.Body.Close() 84 | go forwardRequest(req, reqSourceIP, reqDestionationPort, body) 85 | } 86 | } 87 | } 88 | 89 | func forwardRequest(req *http.Request, reqSourceIP string, reqDestionationPort string, body []byte) { 90 | 91 | // if percentage flag is not 100, then a percentage of requests is skipped 92 | if *fwdPerc != 100 { 93 | var uintForSeed uint64 94 | 95 | if *fwdBy == "" { 96 | // if percentage-by is empty, then forward only a certain percentage of requests 97 | var b [8]byte 98 | _, err := crypto_rand.Read(b[:]) 99 | if err != nil { 100 | log.Println("Error generating crypto random unit for seed", ":", err) 101 | return 102 | } 103 | // uintForSeed is random 104 | uintForSeed = binary.LittleEndian.Uint64(b[:]) 105 | } else { 106 | // if percentage-by is not empty, then forward only requests from a certain percentage of headers/remoteaddresses 107 | strForSeed := "" 108 | if *fwdBy == "header" { 109 | strForSeed = req.Header.Get(*fwdHeader) 110 | } else { 111 | strForSeed = reqSourceIP 112 | } 113 | crc64Table := crc64.MakeTable(0xC96C5795D7870F42) 114 | // uintForSeed is derived from strForSeed 115 | uintForSeed = crc64.Checksum([]byte(strForSeed), crc64Table) 116 | } 117 | 118 | // generate a consistent random number from the variable uintForSeed 119 | math_rand.Seed(int64(uintForSeed)) 120 | randomPercent := math_rand.Float64() * 100 121 | // skip a percentage of requests 122 | if randomPercent > *fwdPerc { 123 | return 124 | } 125 | } 126 | 127 | // create a new url from the raw RequestURI sent by the client 128 | url := fmt.Sprintf("%s%s", string(*fwdDestination), req.RequestURI) 129 | 130 | // create a new HTTP request 131 | forwardReq, err := http.NewRequest(req.Method, url, bytes.NewReader(body)) 132 | if err != nil { 133 | return 134 | } 135 | 136 | // add headers to the new HTTP request 137 | for header, values := range req.Header { 138 | for _, value := range values { 139 | forwardReq.Header.Add(header, value) 140 | } 141 | } 142 | 143 | // Append to X-Forwarded-For the IP of the client or the IP of the latest proxy (if any proxies are in between) 144 | // https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For 145 | forwardReq.Header.Add("X-Forwarded-For", reqSourceIP) 146 | // The three following headers should contain 1 value only, i.e. the outermost port, protocol, and host 147 | // https://tools.ietf.org/html/rfc7239#section-5.4 148 | if forwardReq.Header.Get("X-Forwarded-Port") == "" { 149 | forwardReq.Header.Set("X-Forwarded-Port", reqDestionationPort) 150 | } 151 | if forwardReq.Header.Get("X-Forwarded-Proto") == "" { 152 | forwardReq.Header.Set("X-Forwarded-Proto", "http") 153 | } 154 | if forwardReq.Header.Get("X-Forwarded-Host") == "" { 155 | forwardReq.Header.Set("X-Forwarded-Host", req.Host) 156 | } 157 | 158 | if *keepHostHeader { 159 | forwardReq.Host = req.Host 160 | } 161 | 162 | // Execute the new HTTP request 163 | httpClient := &http.Client{} 164 | resp, rErr := httpClient.Do(forwardReq) 165 | if rErr != nil { 166 | // log.Println("Forward request error", ":", err) 167 | return 168 | } 169 | 170 | defer resp.Body.Close() 171 | } 172 | 173 | // Listen for incoming connections. 174 | func openTCPClient() { 175 | ln, err := net.Listen("tcp", ":4789") 176 | if err != nil { 177 | // If TCP listener cannot be established, NLB health checks would fail 178 | // For this reason, we OS.exit 179 | log.Println("Error listening on TCP", ":", err) 180 | os.Exit(1) 181 | } 182 | log.Println("Listening on TCP 4789") 183 | for { 184 | // Listen for an incoming connection and close it immediately. 185 | conn, _ := ln.Accept() 186 | conn.Close() 187 | } 188 | } 189 | 190 | func main() { 191 | defer util.Run()() 192 | var handle *pcap.Handle 193 | var err error 194 | 195 | flag.Parse() 196 | //labels validation 197 | if *fwdPerc > 100 || *fwdPerc < 0 { 198 | err = fmt.Errorf("Flag percentage is not between 0 and 100. Value: %f.", *fwdPerc) 199 | } else if *fwdBy != "" && *fwdBy != "header" && *fwdBy != "remoteaddr" { 200 | err = fmt.Errorf("Flag percentage-by (%s) is not valid.", *fwdBy) 201 | } else if *fwdBy == "header" && *fwdHeader == "" { 202 | err = fmt.Errorf("Flag percentage-by is set to header, but percentage-by-header is empty.") 203 | } else if *reqPort > 65535 || *reqPort < 0 { 204 | err = fmt.Errorf("Flag filter-request-port is not between 0 and 65535. Value: %f.", *fwdPerc) 205 | } 206 | if err != nil { 207 | log.Fatal(err) 208 | } 209 | 210 | // Set up pcap packet capture 211 | log.Printf("Starting capture on interface vxlan0") 212 | handle, err = pcap.OpenLive("vxlan0", 8951, true, pcap.BlockForever) 213 | if err != nil { 214 | log.Fatal(err) 215 | } 216 | 217 | // Set up BPF filter 218 | BPFFilter := fmt.Sprintf("%s%d", "tcp and dst port ", *reqPort) 219 | if err := handle.SetBPFFilter(BPFFilter); err != nil { 220 | log.Fatal(err) 221 | } 222 | 223 | // Set up assembly 224 | streamFactory := &httpStreamFactory{} 225 | streamPool := tcpassembly.NewStreamPool(streamFactory) 226 | assembler := tcpassembly.NewAssembler(streamPool) 227 | 228 | log.Println("reading in packets") 229 | // Read in packets, pass to assembler. 230 | packetSource := gopacket.NewPacketSource(handle, handle.LinkType()) 231 | packets := packetSource.Packets() 232 | ticker := time.Tick(time.Minute) 233 | 234 | //Open a TCP Client, for NLB Health Checks only 235 | go openTCPClient() 236 | 237 | for { 238 | select { 239 | case packet := <-packets: 240 | // A nil packet indicates the end of a pcap file. 241 | if packet == nil { 242 | return 243 | } 244 | if packet.NetworkLayer() == nil || packet.TransportLayer() == nil || packet.TransportLayer().LayerType() != layers.LayerTypeTCP { 245 | log.Println("Unusable packet") 246 | continue 247 | } 248 | tcp := packet.TransportLayer().(*layers.TCP) 249 | assembler.AssembleWithTimestamp(packet.NetworkLayer().NetworkFlow(), tcp, packet.Metadata().Timestamp) 250 | 251 | case <-ticker: 252 | // Every minute, flush connections that haven't seen activity in the past 1 minute. 253 | assembler.FlushOlderThan(time.Now().Add(time.Minute * -1)) 254 | } 255 | } 256 | } 257 | -------------------------------------------------------------------------------- /replay-handler-cloudformation.yaml: -------------------------------------------------------------------------------- 1 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: BSD-3-Clause 3 | 4 | AWSTemplateFormatVersion: '2010-09-09' 5 | 6 | Description: Replay handler of HTTP requests. 7 | 8 | Parameters: 9 | HandlerVpcId: 10 | Type: 'AWS::EC2::VPC::Id' 11 | Description: Specify the VPC for the replay handler. 12 | ConstraintDescription: must be the VPC Id of an existing Virtual Private Cloud. 13 | HandlerSubnetIds: 14 | Type: 'List' 15 | Description: >- 16 | Specify the subnets for the replay handler. Use subnets that have the following features: 17 | a route to and from the test and production environments; 18 | a route to a NAT Gateway (so that the EC2 instances of the replay handler can download the handler script from the internet). 19 | For high availability use at least two subnets in separate availability zones. 20 | ConstraintDescription: >- 21 | must be a list of at least one existing subnets. They should be residing in 22 | the selected Virtual Private Cloud. it is recommended to provide at least two 23 | subnets in separate availability zones, for high availability purposes. 24 | InstanceType: 25 | Description: EC2 instance type for the replay handler. 26 | Type: String 27 | Default: t4g.small 28 | AllowedValues: [ 29 | t4g.nano, t4g.micro, t4g.small, t4g.medium, t4g.large, t4g.xlarge, t4g.2xlarge, 30 | t3.nano, t3.micro, t3.small, t3.medium, t3.large, t3.xlarge, t3.2xlarge, 31 | 32 | m6g.medium, m6g.large, m6g.xlarge, m6g.2xlarge, m6g.4xlarge, m6g.8xlarge, m6g.12xlarge, m6g.16xlarge, 33 | m5.large, m5.xlarge, m5.2xlarge, m5.4xlarge, m5.8xlarge, m5.12xlarge, m5.16xlarge, m5.24xlarge, 34 | m5n.large, m5n.xlarge, m5n.2xlarge, m5n.4xlarge, m5n.8xlarge, m5n.12xlarge, m5n.16xlarge, m5n.24xlarge, 35 | 36 | c6g.medium, c6g.large, c6g.xlarge, c6g.2xlarge, c6g.4xlarge, c6g.8xlarge, c6g.12xlarge, c6g.16xlarge, 37 | c5.large, c5.xlarge, c5.2xlarge, c5.4xlarge, c5.9xlarge, c5.12xlarge, c5.18xlarge, c5.24xlarge, 38 | c5n.large, c5n.xlarge, c5n.2xlarge, c5n.4xlarge, c5n.9xlarge, c5n.18xlarge, 39 | 40 | r6g.medium, r6g.large, r6g.xlarge, r6g.2xlarge, r6g.4xlarge, r6g.8xlarge, r6g.12xlarge, r6g.16xlarge, 41 | r5.large, r5.xlarge, r5.2xlarge, r5.4xlarge, r5.8xlarge, r5.12xlarge, r5.16xlarge, r5.24xlarge, 42 | r5n.large, r5n.xlarge, r5n.2xlarge, r5n.4xlarge, r5n.8xlarge, r5n.12xlarge, r5n.16xlarge, r5n.24xlarge, 43 | ] 44 | ConstraintDescription: must be a valid EC2 instance type. 45 | InstanceNumber: 46 | Type: Number 47 | Default: 2 48 | Description: Number of EC2 instances in the replay handler. 49 | KeyName: 50 | Description: This parameter is optional. The name of the EC2 Key Pair to allow SSH access to the instances. 51 | Type: 'String' 52 | ConstraintDescription: This parameter is optional. If provided, must be the name of an existing EC2 KeyPair. 53 | SSHLocation: 54 | Description: This parameter is optional. The IP address range that can be used to SSH to the EC2 instances in the replay handler. 55 | Type: String 56 | MinLength: '0' 57 | MaxLength: '18' 58 | AllowedPattern: '((\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2}))?' 59 | ConstraintDescription: This parameter is optional. If provided, must be a valid IP CIDR range of the form x.x.x.x/x. 60 | FilterByDestinationCidrBlock: 61 | Description: Filter the mirrored requests by destination IP. 62 | Type: String 63 | MinLength: '9' 64 | MaxLength: '18' 65 | AllowedPattern: '(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2})' 66 | ConstraintDescription: must be a valid IP CIDR range of the form x.x.x.x/x. 67 | Default: '0.0.0.0/0' 68 | FilterByDestinationPort: 69 | Description: Filter the mirrored requests by destination port. 70 | Type: Number 71 | MinValue: 0 72 | MaxValue: 65535 73 | ConstraintDescription: must be a valid port. 74 | Default: 80 75 | FilterBySourceCidrBlock: 76 | Description: Filter the mirrored requests by source IP. 77 | Type: String 78 | MinLength: '9' 79 | MaxLength: '18' 80 | AllowedPattern: '(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2})' 81 | ConstraintDescription: must be a valid IP CIDR range of the form x.x.x.x/x. 82 | Default: '0.0.0.0/0' 83 | TrafficMirroringVNI: 84 | Description: The VXLAN ID for traffic mirroring session. 85 | Type: Number 86 | MinValue: 1 87 | MaxValue: 16777215 88 | ConstraintDescription: must be an integer between 1 and 16777215. 89 | LatestAmiIdX86: 90 | Description: DO NOT change this parameter. This refers to the latest Amazon Linux 2 AMI. 91 | Type: 'AWS::SSM::Parameter::Value' 92 | Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2' 93 | ConstraintDescription: 'only use /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2' 94 | LatestAmiIdArm: 95 | Description: DO NOT change this parameter. This refers to the latest Amazon Linux 2 AMI. 96 | Type: 'AWS::SSM::Parameter::Value' 97 | Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-arm64-gp2' 98 | ConstraintDescription: 'only use /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-arm64-gp2' 99 | RequestsForwardDestination: 100 | Description: >- 101 | The destination endpoint of the replayed HTTP requests. The endpoint can be an IP address or a DNS name. 102 | Enter the endpoint AND its protocol, like: http://172.0.0.1 or https://www.example.com. 103 | Type: String 104 | ConstraintDescription: 'Enter the endpoint AND its protocol, like: http://172.0.0.1 or https://www.example.com.' 105 | ForwardPercentage: 106 | Description: The percentage of traffic that gets replicated. Enter a number between 0 and 100. 107 | Type: Number 108 | Default: 100 109 | MinValue: 0 110 | MaxValue: 100 111 | ConstraintDescription: 'Enter a number between 0 and 100.' 112 | PercentageBy: 113 | Description: >- 114 | How traffic percentage is determined. If empty, then forward only a certain percentage of requests. 115 | If header/remoteaddr, then forward only requests from a certain percentage of headers/remoteaddresses. 116 | Type: String 117 | AllowedValues: 118 | - '' 119 | - 'header' 120 | - 'remoteaddr' 121 | PercentageByHeader: 122 | Description: If the parameter PercentageBy is set to 'header', then enter the name of the header. Otherwise, leave it empty. 123 | Type: String 124 | KeepOriginalHostHeader: 125 | Description: Copy the original Host header from the mirrored request. Keep empty to set the Host header to RequestsForwardDestination. 126 | Type: String 127 | Default: '' 128 | AllowedValues: 129 | - '' 130 | - '-keep-host-header' 131 | 132 | Metadata: 133 | AWS::CloudFormation::Interface: 134 | ParameterGroups: 135 | - 136 | Label: 137 | default: "Infrastructure" 138 | Parameters: 139 | - HandlerVpcId 140 | - HandlerSubnetIds 141 | - RequestsForwardDestination 142 | - TrafficMirroringVNI 143 | - InstanceType 144 | - InstanceNumber 145 | - KeyName 146 | - SSHLocation 147 | - 148 | Label: 149 | default: "Filters of mirroring" 150 | Parameters: 151 | - ForwardPercentage 152 | - PercentageBy 153 | - PercentageByHeader 154 | - FilterBySourceCidrBlock 155 | - FilterByDestinationCidrBlock 156 | - FilterByDestinationPort 157 | - 158 | Label: 159 | default: "Do NOT change these" 160 | Parameters: 161 | - LatestAmiIdX86 162 | - LatestAmiIdArm 163 | 164 | Mappings: 165 | AmiMap: 166 | { 167 | t4g.nano: {arch: arm}, t4g.micro: {arch: arm}, t4g.small: {arch: arm}, t4g.medium: {arch: arm}, t4g.large: {arch: arm}, t4g.xlarge: {arch: arm}, t4g.2xlarge: {arch: arm}, m6g.medium: {arch: arm}, m6g.large: {arch: arm}, m6g.xlarge: {arch: arm}, m6g.2xlarge: {arch: arm}, m6g.4xlarge: {arch: arm}, m6g.8xlarge: {arch: arm}, m6g.12xlarge: {arch: arm}, m6g.16xlarge: {arch: arm}, c6g.medium: {arch: arm}, c6g.large: {arch: arm}, c6g.xlarge: {arch: arm}, c6g.2xlarge: {arch: arm}, c6g.4xlarge: {arch: arm}, c6g.8xlarge: {arch: arm}, c6g.12xlarge: {arch: arm}, c6g.16xlarge: {arch: arm}, r6g.medium: {arch: arm}, r6g.large: {arch: arm}, r6g.xlarge: {arch: arm}, r6g.2xlarge: {arch: arm}, r6g.4xlarge: {arch: arm}, r6g.8xlarge: {arch: arm}, r6g.12xlarge: {arch: arm}, r6g.16xlarge: {arch: arm} 168 | ,t3.nano: {arch: x86}, t3.micro: {arch: x86}, t3.small: {arch: x86}, t3.medium: {arch: x86}, t3.large: {arch: x86}, t3.xlarge: {arch: x86}, t3.2xlarge: {arch: x86}, m5.large: {arch: x86}, m5.xlarge: {arch: x86}, m5.2xlarge: {arch: x86}, m5.4xlarge: {arch: x86}, m5.8xlarge: {arch: x86}, m5.12xlarge: {arch: x86}, m5.16xlarge: {arch: x86}, m5.24xlarge: {arch: x86}, m5n.large: {arch: x86}, m5n.xlarge: {arch: x86}, m5n.2xlarge: {arch: x86}, m5n.4xlarge: {arch: x86}, m5n.8xlarge: {arch: x86}, m5n.12xlarge: {arch: x86}, m5n.16xlarge: {arch: x86}, m5n.24xlarge: {arch: x86}, c5.large: {arch: x86}, c5.xlarge: {arch: x86}, c5.2xlarge: {arch: x86}, c5.4xlarge: {arch: x86}, c5.9xlarge: {arch: x86}, c5.12xlarge: {arch: x86}, c5.18xlarge: {arch: x86}, c5.24xlarge: {arch: x86}, c5n.large: {arch: x86}, c5n.xlarge: {arch: x86}, c5n.2xlarge: {arch: x86}, c5n.4xlarge: {arch: x86}, c5n.9xlarge: {arch: x86}, c5n.18xlarge: {arch: x86}, r5.large: {arch: x86}, r5.xlarge: {arch: x86}, r5.2xlarge: {arch: x86}, r5.4xlarge: {arch: x86}, r5.8xlarge: {arch: x86}, r5.12xlarge: {arch: x86}, r5.16xlarge: {arch: x86}, r5.24xlarge: {arch: x86}, r5n.large: {arch: x86}, r5n.xlarge: {arch: x86}, r5n.2xlarge: {arch: x86}, r5n.4xlarge: {arch: x86}, r5n.8xlarge: {arch: x86}, r5n.12xlarge: {arch: x86}, r5n.16xlarge: {arch: x86}, r5n.24xlarge: {arch: x86} 169 | } 170 | 171 | Conditions: 172 | UseArm: !Equals [!FindInMap [AmiMap, !Ref "InstanceType", "arch"], arm] 173 | KeyNameEmpty: !Equals [!Ref KeyName, ''] 174 | HasSSHLocation: !Not [!Equals [!Ref SSHLocation, '']] 175 | 176 | Resources: 177 | InstancesGroup: 178 | Type: 'AWS::AutoScaling::AutoScalingGroup' 179 | Properties: 180 | VPCZoneIdentifier: !Ref HandlerSubnetIds 181 | LaunchTemplate: 182 | LaunchTemplateId: !Ref LaunchTemplate 183 | Version: !GetAtt LaunchTemplate.LatestVersionNumber 184 | MinSize: !Ref InstanceNumber 185 | MaxSize: !Ref InstanceNumber 186 | HealthCheckType: ELB 187 | HealthCheckGracePeriod: 180 188 | # AutoScalingGroupName: ASG-ReplayHandler 189 | TargetGroupARNs: 190 | - !Ref NLBTargetGroup 191 | TerminationPolicies: 192 | - OldestInstance 193 | UpdatePolicy: 194 | AutoScalingRollingUpdate: 195 | MinInstancesInService: 0 196 | LaunchTemplate: 197 | Type: 'AWS::EC2::LaunchTemplate' 198 | Properties: 199 | LaunchTemplateName: LT-ReplayHandler 200 | LaunchTemplateData: 201 | KeyName: !If [KeyNameEmpty, !Ref 'AWS::NoValue', !Ref KeyName] 202 | ImageId: !If [UseArm, !Ref LatestAmiIdArm, !Ref LatestAmiIdX86] 203 | SecurityGroupIds: 204 | - !Ref InstanceSecurityGroup 205 | InstanceType: !Ref InstanceType 206 | UserData: !Base64 207 | 'Fn::Join': 208 | - '' 209 | - - | 210 | #!/bin/bash -xe 211 | # 212 | # 213 | ### update and install ### 214 | # 215 | export HOME=~ 216 | yum update -y 217 | yum install git -y 218 | yum install go -y 219 | # dependency of github.com/google/gopacket 220 | yum install libpcap-devel -y 221 | # 222 | # 223 | ### dependency of main.go ### 224 | go env GOPATH 225 | echo 'export GOPATH=$HOME/go' >>~/.bash_profile 226 | source ~/.bash_profile 227 | # 228 | # 229 | # 230 | ### create a virtual network interface that gets decapsulated VXLAN packets 231 | #### compile & run go script ### 232 | go install github.com/aws-samples/http-requests-mirroring@latest 233 | - 'Fn::Sub': | 234 | sudo ip link add vxlan0 type vxlan id ${TrafficMirroringVNI} dev eth0 dstport 4789 235 | sudo ip link set vxlan0 up 236 | $GOPATH"/bin/http-requests-mirroring" -destination "${RequestsForwardDestination}" -percentage "${ForwardPercentage}" -percentage-by "${PercentageBy}" -percentage-by-header "${PercentageByHeader}" -filter-request-port "${FilterByDestinationPort}" ${KeepOriginalHostHeader} 237 | NetworkLoadBalancer: 238 | Type: 'AWS::ElasticLoadBalancingV2::LoadBalancer' 239 | Properties: 240 | Scheme: internal 241 | Subnets: !Ref HandlerSubnetIds 242 | Type: network 243 | LoadBalancerAttributes: 244 | - Key: load_balancing.cross_zone.enabled 245 | Value: "true" 246 | NLBListener: 247 | Type: 'AWS::ElasticLoadBalancingV2::Listener' 248 | Properties: 249 | DefaultActions: 250 | - Type: forward 251 | TargetGroupArn: !Ref NLBTargetGroup 252 | LoadBalancerArn: !Ref NetworkLoadBalancer 253 | Port: 4789 254 | Protocol: UDP 255 | NLBTargetGroup: 256 | Type: 'AWS::ElasticLoadBalancingV2::TargetGroup' 257 | Properties: 258 | Port: 4789 259 | Protocol: UDP 260 | VpcId: !Ref HandlerVpcId 261 | InstanceSecurityGroup: 262 | Type: 'AWS::EC2::SecurityGroup' 263 | Properties: 264 | GroupDescription: Enable SSH access, VXLAN UDP (for traffic), and VXLAN TCP (for health checks) from the load balancer only 265 | SecurityGroupIngress: 266 | - IpProtocol: udp 267 | FromPort: 4789 268 | ToPort: 4789 269 | CidrIp: !Ref FilterByDestinationCidrBlock 270 | - IpProtocol: tcp 271 | FromPort: 4789 272 | ToPort: 4789 273 | CidrIp: !GetAtt VpcInfo.CidrBlock 274 | VpcId: !Ref HandlerVpcId 275 | 276 | InstanceSecurityGroupSSHIngress: 277 | Type: 'AWS::EC2::SecurityGroupIngress' 278 | Condition: HasSSHLocation 279 | Properties: 280 | GroupId: !Ref InstanceSecurityGroup 281 | IpProtocol: tcp 282 | FromPort: 22 283 | ToPort: 22 284 | CidrIp: !Ref SSHLocation 285 | 286 | TrafficMirrorTarget: 287 | Type: 'AWS::EC2::TrafficMirrorTarget' 288 | Properties: 289 | NetworkLoadBalancerArn: !Ref NetworkLoadBalancer 290 | 291 | TrafficMirrorFilter: 292 | Type: 'AWS::EC2::TrafficMirrorFilter' 293 | 294 | TrafficMirrorFilterRule: 295 | Type: 'AWS::EC2::TrafficMirrorFilterRule' 296 | Properties: 297 | DestinationCidrBlock: !Ref FilterByDestinationCidrBlock 298 | DestinationPortRange: 299 | FromPort: !Ref FilterByDestinationPort 300 | ToPort: !Ref FilterByDestinationPort 301 | Protocol: 6 302 | RuleAction: accept 303 | RuleNumber: 100 304 | SourceCidrBlock: !Ref FilterBySourceCidrBlock 305 | TrafficDirection: ingress 306 | TrafficMirrorFilterId: !Ref TrafficMirrorFilter 307 | 308 | LambdaExecutionRole: 309 | Type: AWS::IAM::Role 310 | Properties: 311 | AssumeRolePolicyDocument: 312 | Version: '2012-10-17' 313 | Statement: 314 | - Effect: Allow 315 | Principal: 316 | Service: 317 | - lambda.amazonaws.com 318 | Action: 319 | - sts:AssumeRole 320 | Path: "/" 321 | Policies: 322 | - PolicyName: root 323 | PolicyDocument: 324 | Version: '2012-10-17' 325 | Statement: 326 | - Effect: Allow 327 | Action: 328 | - logs:CreateLogGroup 329 | - logs:CreateLogStream 330 | - logs:PutLogEvents 331 | Resource: !Sub 'arn:${AWS::Partition}:logs:*:*:*' 332 | - Effect: Allow 333 | Action: 334 | - ec2:DescribeVpcs 335 | Resource: '*' 336 | 337 | # NOTE: Pay special attention to the indentatiion in the Python code below. 338 | # Lines that appear blank are likely not blank, but have leading spaces. 339 | GetAttFromParam: 340 | Type: AWS::Lambda::Function 341 | Properties: 342 | Description: Look up info from a VPC or subnet ID 343 | Handler: index.handler 344 | MemorySize: 128 345 | Role: !GetAtt LambdaExecutionRole.Arn 346 | Runtime: "python3.12" 347 | Timeout: 30 348 | Code: 349 | ZipFile: | 350 | import json 351 | import boto3 352 | import cfnresponse 353 | import logging 354 | 355 | def handler(event, context): 356 | logger = logging.getLogger() 357 | logger.setLevel(logging.INFO) 358 | 359 | # initialize our responses, assume failure by default 360 | 361 | response_data = {} 362 | response_status = cfnresponse.FAILED 363 | 364 | logger.info('Received event: {}'.format(json.dumps(event))) 365 | 366 | if event['RequestType'] == 'Delete': 367 | response_status = cfnresponse.SUCCESS 368 | cfnresponse.send(event, context, response_status, response_data) 369 | 370 | try: 371 | ec2=boto3.client('ec2') 372 | except Exception as e: 373 | logger.info('boto3.client failure: {}'.format(e)) 374 | cfnresponse.send(event, context, response_status, response_data) 375 | 376 | name_filter = event['ResourceProperties']['NameFilter'] 377 | name_filter_parts = name_filter.split('-') 378 | resource_type=name_filter_parts[0] 379 | 380 | if resource_type == "vpc": 381 | try: 382 | vpcs=ec2.describe_vpcs(VpcIds=[name_filter]) 383 | except Exception as e: 384 | logger.info('ec2.describe_vpcs failure: {}'.format(e)) 385 | cfnresponse.send(event, context, response_status, response_data) 386 | 387 | number_of_vpcs = len(vpcs['Vpcs']) 388 | logger.info('number of vpcs returned: {}'.format(number_of_vpcs)) 389 | 390 | if number_of_vpcs == 1: 391 | CidrBlock = vpcs['Vpcs'][0]['CidrBlock'] 392 | response_data['CidrBlock'] = CidrBlock 393 | logger.info('vpc CidrBlock {}'.format(CidrBlock)) 394 | response_status = cfnresponse.SUCCESS 395 | cfnresponse.send(event, context, response_status, response_data) 396 | 397 | elif number_of_vpcs == 0: 398 | logger.info('no matching vpcs for filter {}'.format(name_filter)) 399 | cfnresponse.send(event, context, response_status, response_data) 400 | 401 | else: 402 | logger.info('multiple matching vpcs for filter {}'.format(name_filter)) 403 | cfnresponse.send(event, context, response_status, response_data) 404 | 405 | elif resource_type == "subnet": 406 | try: 407 | subnets = ec2.describe_subnets(SubnetIds=[name_filter]) 408 | except Exception as e: 409 | logger.info('ec2.describe_subnets failure: {}'.format(e)) 410 | cfnresponse.send(event, context, response_status, response_data) 411 | 412 | number_of_subnets = len(subnets['Subnets']) 413 | logger.info('number of subnets returned: {}'.format(number_of_subnets)) 414 | 415 | if number_of_subnets == 1: 416 | CidrBlock = subnets['Subnets'][0]['CidrBlock'] 417 | VpcId = subnets['Subnets'][0]['VpcId'] 418 | AvailabilityZone = subnets['Subnets'][0]['AvailabilityZone'] 419 | response_data['AvailabilityZone'] = AvailabilityZone 420 | response_data['CidrBlock'] = CidrBlock 421 | response_data['VpcId'] = VpcId 422 | 423 | logger.info('subnet AvailabilityZone {}'.format(AvailabilityZone)) 424 | logger.info('subnet CidrBlock {}'.format(CidrBlock)) 425 | logger.info('subnet VpcId {}'.format(VpcId)) 426 | 427 | response_status = cfnresponse.SUCCESS 428 | cfnresponse.send(event, context, response_status, response_data) 429 | 430 | elif number_of_subnets == 0: 431 | logger.info('no matching subnet for filter {}'.format(name_filter)) 432 | cfnresponse.send(event, context, response_status, response_data) 433 | 434 | else: 435 | logger.info('multiple matching subnets for filter {}'.format(name_filter)) 436 | cfnresponse.send(event, context, response_status, response_data) 437 | 438 | else: 439 | logger.info('invalid resource type {}'.resource_type) 440 | cfnresponse.send(event, context, response_status, response_data) 441 | 442 | VpcInfo: 443 | Type: Custom::VpcInfo 444 | Properties: 445 | ServiceToken: !GetAtt GetAttFromParam.Arn 446 | NameFilter: !Ref HandlerVpcId 447 | 448 | Outputs: 449 | TrafficMirrorTarget: 450 | Description: The ID of the Traffic Mirror target. 451 | Value: !Ref TrafficMirrorTarget 452 | TrafficMirrorFilter: 453 | Description: The ID of the Traffic Mirror filter. 454 | Value: !Ref TrafficMirrorFilter 455 | --------------------------------------------------------------------------------