├── .github └── PULL_REQUEST_TEMPLATE.md ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── CheckMachine.py ├── Cleanup.py ├── CloudEndure.py ├── LICENSE ├── LaunchMachine.py ├── Machine.py ├── README.md ├── StatusCheck.py ├── UpdateBlueprint.py ├── UpdateProject.py ├── config-cutover.yml ├── config-projects.yml └── config-test.yml /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | *Issue #, if available:* 2 | 3 | *Description of changes:* 4 | 5 | 6 | By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. 7 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check [existing open](https://github.com/aws-samples/automating-aws-migration-with-cloudendure-scripts/issues), or [recently closed](https://github.com/aws-samples/automating-aws-migration-with-cloudendure-scripts/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *master* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws-samples/automating-aws-migration-with-cloudendure-scripts/labels/help%20wanted) issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](https://github.com/aws-samples/automating-aws-migration-with-cloudendure-scripts/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | 61 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. 62 | -------------------------------------------------------------------------------- /CheckMachine.py: -------------------------------------------------------------------------------- 1 | #Copyright 2008-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | #Permission is hereby granted, free of charge, to any person obtaining a copy of this 4 | #software and associated documentation files (the "Software"), to deal in the Software 5 | #without restriction, including without limitation the rights to use, copy, modify, 6 | #merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 7 | #permit persons to whom the Software is furnished to do so. 8 | 9 | #THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 10 | #INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 11 | #PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 12 | #HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 13 | #OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 14 | #SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | from __future__ import print_function 17 | import sys 18 | import requests 19 | import json 20 | import yaml 21 | import os 22 | import datetime 23 | 24 | def status(session, headers, endpoint, HOST, project_id, configfile, launchtype, dryrun): 25 | if launchtype == "test" or launchtype == "cutover": 26 | with open(os.path.join(sys.path[0], configfile), 'r') as ymlfile: 27 | config = yaml.load(ymlfile) 28 | machine_status = 0 29 | m = requests.get(HOST + endpoint.format('projects/{}/machines').format(project_id), headers=headers, cookies=session) 30 | for i in range(1, config["project"]["machinecount"]+1): 31 | index = "machine" + str(i) 32 | machine_exist = False 33 | for machine in json.loads(m.text)["items"]: 34 | if config[index]["machineName"] == machine['sourceProperties']['name']: 35 | machine_exist = True 36 | # Check if replication is done 37 | if 'lastConsistencyDateTime' not in machine['replicationInfo']: 38 | print("ERROR: Machine: " + machine['sourceProperties']['name'] + " replication in progress, please wait for a few minutes....") 39 | sys.exit(1) 40 | else: 41 | # check replication lag 42 | a = int(machine['replicationInfo']['lastConsistencyDateTime'][11:13]) 43 | b = int(machine['replicationInfo']['lastConsistencyDateTime'][14:16]) 44 | x = int(datetime.datetime.utcnow().isoformat()[11:13]) 45 | y = int(datetime.datetime.utcnow().isoformat()[14:16]) 46 | result = (x - a) * 60 + (y - b) 47 | if result > 180: 48 | print("ERROR: Machine: " + machine['sourceProperties']['name'] + " replication lag is more than 180 minutes....") 49 | print("- Current Replication lag for " + machine['sourceProperties']['name'] + " is: " + str(result) + " minutes....") 50 | sys.exit(6) 51 | else: 52 | # Check dryrun flag and skip the rest of checks 53 | if dryrun == "Yes": 54 | machine_status += 1 55 | else: 56 | # Check if the target machine has been tested already 57 | if launchtype == "test": 58 | if 'lastTestLaunchDateTime' not in machine["lifeCycle"] and 'lastCutoverDateTime' not in machine["lifeCycle"]: 59 | machine_status += 1 60 | else: 61 | print("ERROR: Machine: " + machine['sourceProperties']['name'] + " has been tested already....") 62 | sys.exit(2) 63 | # Check if the target machine has been migrated to PROD already 64 | elif launchtype == "cutover": 65 | if 'lastTestLaunchDateTime' in machine["lifeCycle"]: 66 | if 'lastCutoverDateTime' not in machine["lifeCycle"]: 67 | machine_status += 1 68 | else: 69 | print("ERROR: Machine: " + machine['sourceProperties']['name'] + " has been migrated already....") 70 | sys.exit(3) 71 | else: 72 | print("ERROR: Machine: " + machine['sourceProperties']['name'] + " has not been tested....") 73 | sys.exit(4) 74 | if machine_exist == False: 75 | print("ERROR: Machine: " + config[index]["machineName"] + " does not exist....") 76 | sys.exit(7) 77 | 78 | if machine_status == config["project"]["machinecount"]: 79 | print("All Machines in the config file are ready....") 80 | else: 81 | print("ERROR: some machines in the config file are not ready....") 82 | sys.exit(5) -------------------------------------------------------------------------------- /Cleanup.py: -------------------------------------------------------------------------------- 1 | #Copyright 2008-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | #Permission is hereby granted, free of charge, to any person obtaining a copy of this 4 | #software and associated documentation files (the "Software"), to deal in the Software 5 | #without restriction, including without limitation the rights to use, copy, modify, 6 | #merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 7 | #permit persons to whom the Software is furnished to do so. 8 | 9 | #THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 10 | #INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 11 | #PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 12 | #HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 13 | #OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 14 | #SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | from __future__ import print_function 17 | import sys 18 | import requests 19 | import json 20 | import yaml 21 | import os 22 | 23 | def remove(session, headers, endpoint, HOST, projectname, configfile): 24 | with open(os.path.join(sys.path[0], configfile), 'r') as ymlfile: 25 | config = yaml.load(ymlfile) 26 | 27 | # Get Projects 28 | r = requests.get(HOST + endpoint.format('projects'), headers=headers, cookies=session) 29 | if r.status_code != 200: 30 | print("ERROR: Failed to fetch the project....") 31 | sys.exit(1) 32 | try: 33 | # Get Project ID 34 | projects = json.loads(r.text)["items"] 35 | project_exist = False 36 | for project in projects: 37 | if project["name"] == projectname: 38 | project_id = project["id"] 39 | project_exist = True 40 | if project_exist == False: 41 | print("ERROR: Project Name does not exist....") 42 | sys.exit(2) 43 | except: 44 | print(sys.exc_info()) 45 | sys.exit(6) 46 | 47 | m = requests.get(HOST + endpoint.format('projects/{}/machines').format(project_id), headers=headers, cookies=session) 48 | machine_status = 0 49 | for i in range(1, config["project"]["machinecount"]+1): 50 | index = "machine" + str(i) 51 | machine_exist = False 52 | for machine in json.loads(m.text)["items"]: 53 | if config[index]["machineName"] == machine['sourceProperties']['name']: 54 | machine_exist = True 55 | if 'lastCutoverDateTime' in machine["lifeCycle"]: 56 | machine_data = {'machineIDs': [machine['id']]} 57 | remove = requests.delete(HOST + endpoint.format('projects/{}/machines').format(project_id), data = json.dumps(machine_data), headers=headers, cookies=session) 58 | if remove.status_code == 204: 59 | print("Machine: " + machine['sourceProperties']['name'] + " has been removed from CloudEndure....") 60 | machine_status += 1 61 | else: 62 | print("Machine: " + machine['sourceProperties']['name'] + " cleanup failed....") 63 | else: 64 | print("ERROR: Machine: " + machine['sourceProperties']['name'] + " has not been migrated to PROD environment....") 65 | sys.exit(4) 66 | if machine_exist == False: 67 | print("ERROR: Machine: " + config[index]["machineName"] + " does not exist....") 68 | 69 | if machine_status == config["project"]["machinecount"]: 70 | print("All Machines in the config file have been removed from CloudEndure....") 71 | else: 72 | print("ERROR: Some machines in the config file do not exist or have NOT been migrated to the PROD environment....") 73 | sys.exit(5) -------------------------------------------------------------------------------- /CloudEndure.py: -------------------------------------------------------------------------------- 1 | #Copyright 2008-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | #Permission is hereby granted, free of charge, to any person obtaining a copy of this 4 | #software and associated documentation files (the "Software"), to deal in the Software 5 | #without restriction, including without limitation the rights to use, copy, modify, 6 | #merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 7 | #permit persons to whom the Software is furnished to do so. 8 | 9 | #THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 10 | #INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 11 | #PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 12 | #HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 13 | #OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 14 | #SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | from __future__ import print_function 17 | import sys 18 | import argparse 19 | import Machine 20 | import UpdateProject 21 | import requests 22 | import json 23 | import StatusCheck 24 | import Cleanup 25 | 26 | HOST = 'https://console.cloudendure.com' 27 | headers = {'Content-Type': 'application/json'} 28 | session = {} 29 | endpoint = '/api/latest/{}' 30 | 31 | def login(userapitoken, endpoint): 32 | login_data = {'userApiToken': userapitoken} 33 | r = requests.post(HOST + endpoint.format('login'), 34 | data=json.dumps(login_data), headers=headers) 35 | if r.status_code != 200 and r.status_code != 307: 36 | if r.status_code == 401: 37 | print("ERROR: The login credentials provided cannot be authenticated....") 38 | elif r.status_code == 402: 39 | print("ERROR: There is no active license configured for this account....") 40 | elif r.status_code == 429: 41 | print("ERROR: Authentication failure limit has been reached. The service will become available for additional requests after a timeout....") 42 | sys.exit(1) 43 | 44 | # check if need to use a different API entry point 45 | if r.history: 46 | endpoint = '/' + '/'.join(r.url.split('/')[3:-1]) + '/{}' 47 | r = requests.post(HOST + endpoint.format('login'), 48 | data=json.dumps(login_data), headers=headers) 49 | 50 | session['session'] = r.cookies['session'] 51 | try: 52 | headers['X-XSRF-TOKEN'] = r.cookies['XSRF-TOKEN'] 53 | except: 54 | pass 55 | 56 | def main(arguments): 57 | 58 | parser = argparse.ArgumentParser( 59 | description=__doc__, 60 | formatter_class=argparse.RawDescriptionHelpFormatter) 61 | parser.add_argument('--userapitoken', required=True) 62 | parser.add_argument('--configfile', required=True) 63 | parser.add_argument('--launchtype') 64 | parser.add_argument('--projectname', required=True) 65 | parser.add_argument('--updateproject', default="No") 66 | parser.add_argument('--statuscheck', default="No") 67 | parser.add_argument('--cleanup', default="No") 68 | parser.add_argument('--dryrun', default="No") 69 | args = parser.parse_args(arguments) 70 | 71 | print("************************") 72 | print("* Login to CloudEndure *") 73 | print("************************") 74 | login(args.userapitoken, endpoint) 75 | 76 | 77 | if args.updateproject == "Yes": 78 | UpdateProject.update(session, headers, endpoint, HOST, args.projectname, args.configfile) 79 | sys.exit(4) 80 | 81 | if args.cleanup == "Yes": 82 | Cleanup.remove(session, headers, endpoint, HOST, args.projectname, args.configfile) 83 | sys.exit(2) 84 | 85 | if args.dryrun != "No" and args.dryrun != "Yes": 86 | print("ERROR: Please type '--dryrun Yes' if you want to validate your production YAML file....") 87 | sys.exit(3) 88 | 89 | if args.launchtype == "test" or args.launchtype =="cutover": 90 | if args.statuscheck == "No": 91 | Machine.execute(args.launchtype, session, headers, endpoint, HOST, args.projectname, args.configfile, args.dryrun) 92 | else: 93 | if args.statuscheck =="Yes": 94 | StatusCheck.check(args.launchtype, session, headers, endpoint, HOST, args.projectname, args.configfile) 95 | else: 96 | print("ERROR: Please type '--statuscheck Yes' if you want to check migration status....") 97 | else: 98 | print("ERROR: Please use the valid launch type: test|cutover....") 99 | 100 | 101 | if __name__ == '__main__': 102 | sys.exit(main(sys.argv[1:])) -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | this software and associated documentation files (the "Software"), to deal in 5 | the Software without restriction, including without limitation the rights to 6 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | -------------------------------------------------------------------------------- /LaunchMachine.py: -------------------------------------------------------------------------------- 1 | #Copyright 2008-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | #Permission is hereby granted, free of charge, to any person obtaining a copy of this 4 | #software and associated documentation files (the "Software"), to deal in the Software 5 | #without restriction, including without limitation the rights to use, copy, modify, 6 | #merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 7 | #permit persons to whom the Software is furnished to do so. 8 | 9 | #THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 10 | #INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 11 | #PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 12 | #HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 13 | #OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 14 | #SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | from __future__ import print_function 17 | import sys 18 | import requests 19 | import json 20 | import yaml 21 | import os 22 | 23 | def launch(launchtype, session, headers, endpoint, HOST, project_id, configfile): 24 | if launchtype == "test" or launchtype == "cutover": 25 | with open(os.path.join(sys.path[0], configfile), 'r') as ymlfile: 26 | config = yaml.load(ymlfile) 27 | m = requests.get(HOST + endpoint.format('projects/{}/machines').format(project_id), headers=headers, cookies=session) 28 | machine_ids = [] 29 | machine_names = [] 30 | for i in range(1, config["project"]["machinecount"]+1): 31 | index = "machine" + str(i) 32 | for machine in json.loads(m.text)["items"]: 33 | if config[index]["machineName"] == machine['sourceProperties']['name']: 34 | machine_ids.append({"machineId": machine['id']}) 35 | machine_names.append(machine['sourceProperties']['name']) 36 | if launchtype == "test": 37 | machine_data = {'items': machine_ids, "launchType": "TEST"} 38 | elif launchtype == "cutover": 39 | machine_data = {'items': machine_ids, "launchType": "CUTOVER"} 40 | else: 41 | print("ERROR: Invalid Launch Type....") 42 | result = requests.post(HOST + endpoint.format('projects/{}/launchMachines').format(project_id), data = json.dumps(machine_data), headers = headers, cookies = session) 43 | # Response code translate to message 44 | if result.status_code == 202: 45 | if launchtype == "test": 46 | print("Test Job created for machine: ") 47 | for machine in machine_names: 48 | print("****** " + machine + " ******") 49 | if launchtype == "cutover": 50 | print("Cutover Job created for machine: ") 51 | for machine in machine_names: 52 | print("****** " + machine + " ******") 53 | elif result.status_code == 409: 54 | print("ERROR: Source machines have job in progress....") 55 | elif result.status_code == 402: 56 | print("ERROR: Project license has expired....") 57 | else: 58 | print(result.text) 59 | print("ERROR: Launch target machine failed....") -------------------------------------------------------------------------------- /Machine.py: -------------------------------------------------------------------------------- 1 | #Copyright 2008-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | #Permission is hereby granted, free of charge, to any person obtaining a copy of this 4 | #software and associated documentation files (the "Software"), to deal in the Software 5 | #without restriction, including without limitation the rights to use, copy, modify, 6 | #merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 7 | #permit persons to whom the Software is furnished to do so. 8 | 9 | #THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 10 | #INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 11 | #PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 12 | #HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 13 | #OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 14 | #SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | from __future__ import print_function 17 | import sys 18 | import UpdateBlueprint 19 | import CheckMachine 20 | import LaunchMachine 21 | import requests 22 | import json 23 | 24 | def execute(launchtype, session, headers, endpoint, HOST, projectname, configfile, dryrun): 25 | r = requests.get(HOST + endpoint.format('projects'), headers=headers, cookies=session) 26 | if r.status_code != 200: 27 | print("ERROR: Failed to fetch the project....") 28 | sys.exit(1) 29 | try: 30 | # Get Project ID 31 | projects = json.loads(r.text)["items"] 32 | project_exist = False 33 | for project in projects: 34 | if project["name"] == projectname: 35 | project_id = project["id"] 36 | project_exist = True 37 | if project_exist == False: 38 | print("ERROR: Project Name does not exist....") 39 | sys.exit(2) 40 | 41 | # Get Machine List 42 | m = requests.get(HOST + endpoint.format('projects/{}/machines').format(project_id), headers=headers, cookies=session) 43 | if "sourceProperties" not in m.text: 44 | print("ERROR: Failed to fetch the machines....") 45 | sys.exit(3) 46 | machinelist = {} 47 | for machine in json.loads(m.text)["items"]: 48 | print('Machine name:{}, Machine ID:{}'.format(machine['sourceProperties']['name'], machine['id'])) 49 | machinelist[machine['id']] = machine['sourceProperties']['name'] 50 | 51 | # Check Target Machines 52 | print("****************************") 53 | print("* Checking Target machines *") 54 | print("****************************") 55 | CheckMachine.status(session, headers, endpoint, HOST, project_id, configfile, launchtype, dryrun) 56 | 57 | # Update Machine Blueprint 58 | print("**********************") 59 | print("* Updating Blueprint *") 60 | print("**********************") 61 | UpdateBlueprint.update(launchtype, session, headers, endpoint, HOST, project_id, machinelist, configfile, dryrun) 62 | 63 | 64 | # Launch Target machines 65 | if dryrun == "No": 66 | print("*****************************") 67 | print("* Launching target machines *") 68 | print("*****************************") 69 | LaunchMachine.launch(launchtype, session, headers, endpoint, HOST, project_id, configfile) 70 | 71 | except: 72 | print(sys.exc_info()) 73 | sys.exit(6) -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Automating AWS Migration with CloudEndure Scripts 2 | 3 | Automating AWS Migration with CloudEndure Scripts to accelerate the migration and reduce migration errors 4 | 5 | ## License Summary 6 | 7 | This sample code is made available under the MIT-0 license. See the LICENSE file. 8 | -------------------------------------------------------------------------------- /StatusCheck.py: -------------------------------------------------------------------------------- 1 | #Copyright 2008-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | #Permission is hereby granted, free of charge, to any person obtaining a copy of this 4 | #software and associated documentation files (the "Software"), to deal in the Software 5 | #without restriction, including without limitation the rights to use, copy, modify, 6 | #merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 7 | #permit persons to whom the Software is furnished to do so. 8 | 9 | #THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 10 | #INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 11 | #PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 12 | #HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 13 | #OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 14 | #SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | from __future__ import print_function 17 | import sys 18 | import requests 19 | import json 20 | import yaml 21 | import os 22 | 23 | def check(launchtype, session, headers, endpoint, HOST, projectname, configfile): 24 | if launchtype == "test" or launchtype == "cutover": 25 | with open(os.path.join(sys.path[0], configfile), 'r') as ymlfile: 26 | config = yaml.load(ymlfile) 27 | 28 | r = requests.get(HOST + endpoint.format('projects'), headers=headers, cookies=session) 29 | if r.status_code != 200: 30 | print("ERROR: Failed to fetch the project....") 31 | sys.exit(1) 32 | try: 33 | # Get Project ID 34 | projects = json.loads(r.text)["items"] 35 | project_exist = False 36 | for project in projects: 37 | if project["name"] == projectname: 38 | project_id = project["id"] 39 | project_exist = True 40 | if project_exist == False: 41 | print("ERROR: Project Name does not exist....") 42 | sys.exit(2) 43 | except: 44 | print(sys.exc_info()) 45 | sys.exit(6) 46 | 47 | machine_status = 0 48 | m = requests.get(HOST + endpoint.format('projects/{}/machines').format(project_id), headers=headers, cookies=session) 49 | for i in range(1, config["project"]["machinecount"]+1): 50 | index = "machine" + str(i) 51 | machine_exist = False 52 | for machine in json.loads(m.text)["items"]: 53 | if config[index]["machineName"] == machine['sourceProperties']['name']: 54 | machine_exist = True 55 | if 'lastConsistencyDateTime' not in machine['replicationInfo']: 56 | print("Machine: " + machine['sourceProperties']['name'] + " replication in progress, please wait for a few minutes....") 57 | else: 58 | if launchtype == "test": 59 | if 'lastTestLaunchDateTime' in machine["lifeCycle"]: 60 | machine_status += 1 61 | print("Machine: " + machine['sourceProperties']['name'] + " has been migrated to the TEST environment....") 62 | else: 63 | print("Machine: " + machine['sourceProperties']['name'] + " has NOT been migrated to the TEST environment, please wait....") 64 | elif launchtype == "cutover": 65 | if 'lastCutoverDateTime' in machine["lifeCycle"]: 66 | machine_status += 1 67 | print("Machine: " + machine['sourceProperties']['name'] + " has been migrated to the PROD environment....") 68 | else: 69 | print("Machine: " + machine['sourceProperties']['name'] + " has NOT been migrated to the PROD environment....") 70 | if machine_exist == False: 71 | print("ERROR: Machine: " + config[index]["machineName"] + " does not exist....") 72 | 73 | if machine_status == config["project"]["machinecount"]: 74 | if launchtype == "test": 75 | print("All Machines in the config file have been migrated to the TEST environment....") 76 | if launchtype == "cutover": 77 | print("All Machines in the config file have been migrated to the PROD environment....") 78 | else: 79 | if launchtype == "test": 80 | print("*WARNING*: Some machines in the config file have NOT been migrated to the TEST environment....") 81 | if launchtype == "cutover": 82 | print("*WARNING*: Some machines in the config file have NOT been migrated to the PROD environment....") -------------------------------------------------------------------------------- /UpdateBlueprint.py: -------------------------------------------------------------------------------- 1 | #Copyright 2008-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | #Permission is hereby granted, free of charge, to any person obtaining a copy of this 4 | #software and associated documentation files (the "Software"), to deal in the Software 5 | #without restriction, including without limitation the rights to use, copy, modify, 6 | #merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 7 | #permit persons to whom the Software is furnished to do so. 8 | 9 | #THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 10 | #INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 11 | #PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 12 | #HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 13 | #OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 14 | #SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | from __future__ import print_function 17 | import sys 18 | import requests 19 | import json 20 | import yaml 21 | import os 22 | 23 | def update(launchtype, session, headers, endpoint, HOST, projectId, machinelist, configfile, dryrun): 24 | if launchtype == "test" or launchtype == "cutover": 25 | with open(os.path.join(sys.path[0], configfile), 'r') as ymlfile: 26 | config = yaml.load(ymlfile) 27 | else: 28 | print("Invalid Launch Type !") 29 | try: 30 | b = requests.get(HOST + endpoint.format('projects/{}/blueprints').format(projectId), headers=headers, cookies=session) 31 | for blueprint in json.loads(b.text)["items"]: 32 | machineName = machinelist[blueprint["machineId"]] 33 | for i in range(1, config["project"]["machinecount"]+1): 34 | index = "machine" + str(i) 35 | if config[index]["machineName"] == machineName: 36 | url = endpoint.format('projects/{}/blueprints/').format(projectId) + blueprint['id'] 37 | blueprint["instanceType"] = config[index]["instanceType"] 38 | blueprint["tenancy"] = config[index]["tenancy"] 39 | if config[index]["iamRole"].lower() != "none": 40 | blueprint["iamRole"] = config[index]["iamRole"] 41 | for disk in blueprint["disks"]: 42 | blueprint["disks"] = [{"type":"SSD", "name":disk["name"]}] 43 | existing_subnetId = blueprint["subnetIDs"] 44 | existing_SecurityGroupIds = blueprint["securityGroupIDs"] 45 | blueprint["subnetIDs"] = config[index]["subnetIDs"] 46 | blueprint["securityGroupIDs"] = config[index]["securitygroupIDs"] 47 | blueprint["publicIPAction"] = 'DONT_ALLOCATE' 48 | blueprint["privateIPAction"] = 'CREATE_NEW' 49 | tags = [] 50 | # Update machine tags 51 | for i in range(1, config[index]["tags"]["count"]+1): 52 | keytag = "key" + str(i) 53 | valuetag = "value" + str(i) 54 | tag = {"key":config[index]["tags"][keytag], "value":config[index]["tags"][valuetag]} 55 | tags.append(tag) 56 | existing_tag = blueprint["tags"] 57 | blueprint["tags"] = tags 58 | result = requests.patch(HOST + url, data=json.dumps(blueprint), headers=headers, cookies=session) 59 | if result.status_code != 200: 60 | print("ERROR: Updating blueprint failed for machine: " + machineName +", invalid blueprint config....") 61 | if dryrun == "Yes": 62 | print("ERROR: YAML validation failed, please fix the errors in the cutover YAML file") 63 | sys.exit(4) 64 | machinelist[blueprint["machineId"]] = "updated" 65 | print("Blueprint for machine: " + machineName + " updated....") 66 | if dryrun == "Yes": 67 | blueprint["subnetIDs"] = existing_subnetId 68 | blueprint["securityGroupIDs"] = existing_SecurityGroupIds 69 | blueprint["tags"] = existing_tag 70 | result = requests.patch(HOST + url, data=json.dumps(blueprint), headers=headers, cookies=session) 71 | if result.status_code != 200: 72 | print("ERROR: Failed to roll back subnet,SG and tags for machine: " + machineName +"....") 73 | sys.exit(5) 74 | else: 75 | print("Dryrun was successful for machine: " + machineName +"...." ) 76 | 77 | except: 78 | print("ERROR: Updating blueprint task failed....") 79 | print(sys.exc_info()) 80 | sys.exit(3) -------------------------------------------------------------------------------- /UpdateProject.py: -------------------------------------------------------------------------------- 1 | #Copyright 2008-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | #Permission is hereby granted, free of charge, to any person obtaining a copy of this 4 | #software and associated documentation files (the "Software"), to deal in the Software 5 | #without restriction, including without limitation the rights to use, copy, modify, 6 | #merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 7 | #permit persons to whom the Software is furnished to do so. 8 | 9 | #THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 10 | #INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 11 | #PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 12 | #HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 13 | #OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 14 | #SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | from __future__ import print_function 17 | import sys 18 | import requests 19 | import json 20 | import yaml 21 | import os 22 | import base64 23 | 24 | def update(session, headers, endpoint, HOST, projectname, configfile): 25 | with open(os.path.join(sys.path[0], configfile), 'r') as ymlfile: 26 | config = yaml.load(ymlfile) 27 | r = requests.get(HOST + endpoint.format('projects'), headers=headers, cookies=session) 28 | if r.status_code != 200: 29 | print("ERROR: Failed to fetch the project....") 30 | sys.exit(1) 31 | try: 32 | # Get Project ID 33 | projects = json.loads(r.text)["items"] 34 | project_exist = False 35 | for project in projects: 36 | if project["name"] == projectname: 37 | project_id = project["id"] 38 | project_exist = True 39 | if project_exist == False: 40 | print("ERROR: Project Name does not exist....") 41 | sys.exit(2) 42 | 43 | # Update encryption key and replication server 44 | rep = requests.get(HOST + endpoint.format('projects/{}/replicationConfigurations').format(project_id), headers=headers, cookies=session) 45 | for replication in json.loads(rep.text)["items"]: 46 | url = endpoint.format('projects/{}/replicationConfigurations/').format(project_id) + replication['id'] 47 | replication["volumeEncryptionKey"] = config["replication"]["encryptionkey"] 48 | replication["volumeEncryptionAllowed"] = True 49 | replication["subnetId"] = config["replication"]["subnetID"] 50 | replication["replicatorSecurityGroupIDs"] = config["replication"]["securitygroupIDs"] 51 | eresult = requests.patch(HOST + url, data=json.dumps(replication), headers=headers, cookies=session) 52 | if eresult.status_code == 200: 53 | print("Encryption Key and replication server updated for project: " + projectname + "....") 54 | else: 55 | print("ERROR: Updating Encryption key or replication server failed....") 56 | 57 | except: 58 | print("ERROR: Updating project failed....") 59 | print(sys.exc_info()) 60 | sys.exit(3) -------------------------------------------------------------------------------- /config-cutover.yml: -------------------------------------------------------------------------------- 1 | project: 2 | waves : poc 3 | machinecount : 1 4 | machine1: 5 | machineName : Test-03 6 | subnetIDs : [subnet-xxxxxx] 7 | securitygroupIDs : [sg-xxxxxx] 8 | instanceType : t2.micro 9 | iamRole : None 10 | tenancy : SHARED 11 | tags : 12 | count : 8 13 | key1 : Name 14 | value1 : Test-02 15 | key2 : ami_short_name 16 | value2 : win2012-Base 17 | key3 : application 18 | value3 : Test Web Server 19 | key4 : application_tier 20 | value4 : APS 21 | key5 : cost_center 22 | value5 : '112233' 23 | key6 : environment 24 | value6 : Prod 25 | key7 : operating_system 26 | value7 : 'W2K12' 27 | key8 : migrated 28 | value8 : 'true' 29 | -------------------------------------------------------------------------------- /config-projects.yml: -------------------------------------------------------------------------------- 1 | replication: 2 | encryptionkey : arn:aws:kms:us-east-1:111122223333:key/xxxxxxxx 3 | subnetID : subnet-xxxxxx 4 | securitygroupIDs : [sg-xxxxxx] -------------------------------------------------------------------------------- /config-test.yml: -------------------------------------------------------------------------------- 1 | project: 2 | waves : wave1 3 | machinecount : 2 4 | machine1: 5 | machineName : TESTXXX01 6 | subnetIDs : [subnet-XXXXX] 7 | securitygroupIDs : [sg-XXXXX] 8 | instanceType : t2.micro 9 | iamRole : None 10 | tenancy : SHARED 11 | tags : 12 | count : 3 13 | key1 : Name 14 | value1 : TESTXXX01 15 | key2 : migrated 16 | value2 : 'true' 17 | key3 : environment 18 | value3 : PROD 19 | machine2: 20 | machineName : TESTXXX02 21 | subnetIDs : [subnet-XXXXX] 22 | securitygroupIDs : [sg-XXXXX] 23 | instanceType : t2.micro 24 | iamRole : None 25 | tags : 26 | count : 3 27 | key1 : Name 28 | value1 : TESTXXX01 29 | key2 : migrated 30 | value2 : 'true' 31 | key3 : environment 32 | value3 : QA 33 | --------------------------------------------------------------------------------