├── .github └── PULL_REQUEST_TEMPLATE.md ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── Lab-Part1.md ├── Lab-Part2.md ├── Lab-Part3.md ├── Lab-Part4.md ├── Lab-Part5.md ├── Lab-Part6.md ├── Lab-Part7.md ├── README.md ├── cloudformation └── MigrationCloudFormation.json └── img ├── .DS_Store ├── lab-2 ├── lab2-image1.png ├── lab2-image2.png ├── lab2-image3.png ├── lab2-image4.png ├── lab2-image5.png ├── lab2-image6.png ├── lab2-image7.png ├── lab2-image8.png └── lab2-image9.png ├── lab-3 ├── lab3-image1.png ├── lab3-image2.png ├── lab3-image3.png ├── lab3-image4.png ├── lab3-image5.png ├── lab3-image6.png ├── lab3-image7.png └── lab3-image8.png ├── lab-4 ├── lab4-image1.png ├── lab4-image2.png ├── lab4-image3.png ├── lab4-image4.png └── sql-icon.png ├── lab-5 ├── lab5-image1.png ├── lab5-image2.png ├── lab5-image3.png ├── lab5-image4.png ├── lab5-image5.png ├── lab5-image6.png ├── lab5-image7.png └── lab5-image8.png ├── lab-6 ├── lab6-image1.png ├── lab6-image10.png ├── lab6-image11.png ├── lab6-image12.png ├── lab6-image13.png ├── lab6-image14.png ├── lab6-image15.png ├── lab6-image16.png ├── lab6-image17.png ├── lab6-image18.png ├── lab6-image19.png ├── lab6-image2.png ├── lab6-image20.png ├── lab6-image21.png ├── lab6-image22.png ├── lab6-image23.png ├── lab6-image24.png ├── lab6-image25.png ├── lab6-image3.png ├── lab6-image4.png ├── lab6-image5.png ├── lab6-image6.png ├── lab6-image7.png ├── lab6-image8.png ├── lab6-image9.png └── putty.png ├── lab-arch.jpg ├── lab-arch.png ├── lab1-image01.png ├── lab1-image03.png ├── lab1-image1.png ├── lab1-image2.png ├── lab1-image3.png ├── lab1-image4.png ├── lab1-image5.png ├── lab1-image6.png └── lab2-image1.png /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | *Issue #, if available:* 2 | 3 | *Description of changes:* 4 | 5 | 6 | By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. 7 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check [existing open](https://github.com/aws-samples/amazon-redshift-data-warehouse-migration/issues), or [recently closed](https://github.com/aws-samples/amazon-redshift-data-warehouse-migration/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *master* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws-samples/amazon-redshift-data-warehouse-migration/labels/help%20wanted) issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](https://github.com/aws-samples/amazon-redshift-data-warehouse-migration/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | 61 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. 62 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of this 4 | software and associated documentation files (the "Software"), to deal in the Software 5 | without restriction, including without limitation the rights to use, copy, modify, 6 | merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 7 | permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 10 | INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 11 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 12 | HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 13 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 14 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | -------------------------------------------------------------------------------- /Lab-Part1.md: -------------------------------------------------------------------------------- 1 | # Setting up the Infrastructure for the lab 2 | 3 | In order to setup the infrastructure for this lab, we will be using an AWS CloudFormation template. 4 | 5 | The CloudFormation template for this lab is available in the directory - cloudformation. 6 | 7 | ### **Important:** 8 | **Note:** - This lab should be run in the N. Virginia region. Please make sure you select Singapore region : **us-east-1** 9 | 10 | The CloudFormation script can be launched only from N. Virginia region only and doesn't work on any other region. 11 | 12 | ## Creating the stack using the CloudFormation template 13 | 14 | 1. Login to the AWS Management Console and select CloudFormation under Management tools. 15 | 16 | ![](img/lab1-image1.png) 17 | 18 | 2. Once you are on the CloudFormation page, select the “Create Stack” option to create a new stack. 19 | 20 | ![](img/lab1-image01.png) 21 |
22 | 23 | 3. Click Choose File and select Upload a template to Amazon S3 and Browse to where you downloaded the Zip file. Select the CloudFormation template called MigrationCloudFormation.json under the cloudformation directory and then click Next. 24 | 25 | ![](img/lab1-image03.png) 26 |
27 | 28 | 4. Populate the form with the values specified below and then click Next. 29 | 30 | > **Stack Name:** Any name you would like to give the stack. but recommended name to be unique like DWMigration2019. Please append your initials if multiple people are running the lab in the same account. 31 | 32 | > **KeyName:** Select a key in N. Virginia region from the dropdown. If you created a key in step 1 then select that key or else select another key already existing in your account. 33 | 34 |
35 | 36 | 5. On the Options page, put the following values for Tags and click Next. All other values can remain as default. 37 | > Key = Environment 38 | > Value = DWMigration2019 39 | 40 |
41 | 42 | 6. On the Review page, At the bottom of the screen please make sure you check the box ‘I acknowledge that AWS CloudFormation might create IAM resources with custom names.’ and Click Create. 43 | 44 | ![CloudFormation-Acknowledgement](img/lab1-image5.png) 45 |
46 | 47 | 7. You can observe the execution progress of CloudFormation as show below. 48 | ![CloudFormation-Progress](img/lab1-image6.png) 49 |
50 | 51 | 8. At this point, you will be directed back to the CloudFormation console and will see a status of CREATE_IN_PROGRESS. Do not continue until the status changes to CREATE_COMPLETE. 52 |
53 | 54 | 9. Once CloudFormation has completed building the stack, the status will change to create_complete. Make sure to note the values for the following keys in the OUTPUTS tab. These will be needed as input for SCT and DMS setup. 55 | 56 | | Output Field Name | Description | 57 | | --- | --- | 58 | | Reinvent2018ANT371RedshiftEndpoint | End point for the Redshift cluster | 59 | | Reinvent2018ANT371VpcId | VPC ID of the newly created VPC | 60 | | Reinvent2018ANT371OracleEndpoint | End point of the source RDS database | 61 | | Reinvent2018ANT371SCTDNS | End point for the EC2 instance with SCT and drivers installed. | 62 | | Reinvent2018ANT371AGENT1PublicDNS | Public end point for first EC2 instance with SCT Extraction agent. | 63 | | Reinvent2018ANT371AGENT1PrivateIP | Private IP for first EC2 instance with SCT Extraction agent | 64 | | Reinvent2018ANT371AGENT2PublicDNS | End point for second EC2 instance with SCT Extraction agent. | 65 | | Reinvent2018ANT371AGENT2PrivateIP | VPrivate IP for second EC2 instance with SCT Extraction agent. | 66 | | Reinvent2018ANT371S3Bucket | S3 Bucket name. | 67 | | CredDatabaseDetails | Credentials for Oracle database and Amazon Redshift | 68 | | CredSCTDetails | Password for login to Windows SCT server. | 69 | 70 |
71 | 72 | This lab will conclude once the CloudFormation completes successfully and the user identifies the AWS resources properties in the output tab in CloudFormation Console. Please proceed to the next lab. 73 | -------------------------------------------------------------------------------- /Lab-Part2.md: -------------------------------------------------------------------------------- 1 | # Connect to your environment 2 | 3 | Now that the environment has been built, the next step is to connect to the EC2 instance. 4 | 5 | 1. Once CloudFormation has completed building the stack, go to the EC2 console located at: [link to AWS Management console ](https://console.aws.amazon.com/ec2/v2/home). Then click Instances in the left column. 6 | 7 | ![EC2 Console](img/lab-2/lab2-image1.png) 8 | 9 | 2. Select the instance with Reinvent2018ANT371-SCTWinEC2 (or whatever name you gave your stack) in its name and obtain the Public IP Address from the instance. 10 | 11 | ![EC2 Console](img/lab-2/lab2-image2.png) 12 | 13 | 3. Click on the Connect button to popup the window as shown below and click on Download Remote Desktop File button to download the RDP file. (Note that you do not need to use the ‘Get Password’ button for this lab. The password is provided in step 4.) 14 | 15 | ![EC2 Console](img/lab-2/lab2-image3.png) 16 |
17 | 4. Using the RDP tool in Mac/Windows, open the RDP file downloaded. Use the username ‘Administrator’ if it’s blank. For password, kindly refer to Cloud Formation output section for the Windows SCT machine. 18 | 19 | ## Install AWS schema Conversion Tool and verify connectivity to Oracle RDS and Amazon Redshift from AWS SCT 20 | 21 | 1. Once connected launch SCT installation msi from the `aws-schema-conversion-tool-1.0.latest` folder on the desktop of the EC2 instance. You will find it under the `aws-schema-conversion-tool-1.0.latest` folder as ``AWS Schema Conversion Tool-1.0.645``. Double click it to launch Schema Conversion Tool installation. 22 | 23 | 2. After clicking AWS Schema Conversion Tool, wait for a minute for the tool to launch, hit Run and install AWS Schema Conversion Tool. Once installed, be aware there will be no confirmation message of the installation. Click on Windows Start button, and click on Down Arrow to list Apps, drag the bar to the right until you see AWS Schema Conversion Tool as seen below: 24 | 25 | > ![EC2 Console](img/lab-2/lab2-image4.png) 26 | 27 | 3. Click on Schema Conversion Tool to open the application. You may see a message that asks to download new version, click Not now button and proceed. If you do upgrade then your screen shots may vary from what we have in the lab document. 28 | 29 | > ![EC2 Console](img/lab-2/lab2-image5.png) 30 | 31 | 4. A New Database Migration Project page opens. If a new project does not open then click on ** File ** and ** New Project.** Provide a Project Name of your choice (recommended **Reinvent2018ANT371**) and leave Location default. Make sure you select **Data Warehouse (OLAP)** and Source Database Engine as **Oracle DW** and the Target Database Engine as **Amazon Redshift.** Click OK. 32 | 33 | > ![EC2 Console](img/lab-2/lab2-image6.png) 34 | 35 | 5. Click on Connect to Oracle DW on the top menu and put in the following details and click Test Connection. It should come back with a message ‘Connection successful’. Then click OK. Update screen shot 36 | > 37 | Type | SID 38 | ---- | ---- 39 | Server name | Take this from the output of the CloudFormation template in Step 8. It is the Reinvent2018ANT371OracleEndpoint output field. 40 | Server port | 1521 41 | Oracle SID | DMSDW 42 | User name | dms_user 43 | Password | dms_user 44 | Oracle Driver Path | Specify the Oracle driver path as C:\Users\Administrator\Desktop\ANT371\OracleDrivers\ojdbc8.jar 45 | 46 | > ![EC2 Console](img/lab-2/lab2-image7.png) 47 | 48 | 6. Now you should see the Oracle DW schema on your left as shown below. 49 | 50 | ![EC2 Console](img/lab-2/lab2-image8.png) 51 | 52 | 7. Click on the Connect to Amazon Redshift on the top menu and put in the following details and click Test Connection. It should come back with a message ‘Connection successful’. Then click OK. 53 | 54 | Type | SID 55 | ---- | ---- 56 | Server name | Take this from the output of the CloudFormation template in Step 8. It is the Reinvent2018-ANT371-Redshift-Endpoint output field. 57 | Server port | 5439 58 | Database | dwtarget 59 | User name | dms_sample_dw 60 | Password | 61 | Amazon Redshift Driver Path | C:\Users\Administrator\Desktop\ANT371\RedshiftDrivers\RedshiftJDBC42-1.2.16.1027.jar 62 | 63 | ![EC2 Console](img/lab-2/lab2-image9.png) 64 | -------------------------------------------------------------------------------- /Lab-Part3.md: -------------------------------------------------------------------------------- 1 | # Use AWS SCT to convert schema from source Oracle to target Amazon Redshift 2 | 3 | 1. Now you should see Redshift on the right hand side. Click on the dms_sample_dw schema on the left hand side in the Oracle DW. You may see a message saying loading metadata. Once that is done your screen should look like below. 4 |
5 | 6 | ![EC2 Console](img/lab-3/lab3-image1.png) 7 |
8 | 9 | 10 | 2. Right click on the dms_sample_dw schema in the left pane and click Create Report. This will create an assessement report. 11 | 12 | > You will get a message saying that ‘Your Optimization Settings require run-time statitics information from the source database. Would you like to load statistics?’ Click Continue. This will take about 30-40 secs and your Database Migration Assessement Report will be ready. 13 | 14 | > The AWS SCT creates a database migration assessment report to help you convert your schema. The database migration assessment report provides important information about the conversion of the schema from your source database to your target database instance. The report summarizes all of the schema conversion tasks and details the action items for schema that can't be converted to the DB engine of your target DB instance. The report also includes estimates of the amount of effort that it will take to write the equivalent code in your target DB instance that can't be converted automatically. 15 | 16 | > ![EC2 Console](img/lab-3/lab3-image2.png) 17 |
18 | 19 | > ![EC2 Console](img/lab-3/lab3-image3.png) 20 | 21 |
22 | 3. Choose the Action Items tab. This tab will show all the items that need to be fixed before they can be migrated. E.g. data types that mismatch, objects not supported by Redshift like stored procedures, materialised views etc. 23 | 24 |
25 | 26 | > ![EC2 Console](img/lab-3/lab3-image4.png) 27 | 28 | >> Some of the few issues depending on the source schema, which our customers may encountered during the schema conversion . 29 | 30 | >> **Unable to convert datatypes** – This is for an CLOB datatype which is converted to VARCHAR. For a list of target data types see [here](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Reference.Target.Redshift.DataTypes.html). SCT also specifies using S3 as storage for this datatype. 31 | 32 | >> **Amazon redshift supports stored procedures** – The existing stored procedure will need to be manually converted or rewritten per Amazon Redshift standards. 33 | 34 | >> **Amazon Redshift doesn’t support sequences** – This will need to be manually converted. All the materialised views need to be converted to tables in Redshift and data loading scripts need to be defined separately to populate them. 35 | 36 | >> In a real life scenario you will need to work through these issues and resolve them before moving on. 37 | 38 |
39 | 40 | 4. Open the context (right-click) menu for the dms_sample_dw item in the Schemas list in the Oracle DW pane on the left, and then choose Collect Statistics. AWS SCT analyzes the source data to recommend the best keys for the target Amazon Redshift database. For more information, see [Collecting or Uploading Statistics for the AWS Schema Conversion Tool](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_SchemaConversionTool.DW.Statistics.html). 41 | 42 | ![EC2 Console](img/lab-3/lab3-image5.png) 43 | 44 | 5. To optimise how the AWS Schema Conversion Tool (AWS SCT) converts your data warehouse schema, you can choose the strategies and rules you want the tool to use. After converting your schema, and reviewing the suggested keys, you can adjust your rules or change your strategy to get the results you want. Choose Settings, and then choose Project Settings. The Current project settings dialog box appears. In the left pane, choose optimisation Strategies. The optimisation strategies appear in the right pane with the defaults selected. For optimisation Strategies, choose the optimisation strategy you want to use. 45 | 46 | * You can choose from the following: 47 | * Use metadata, ignore statistical information 48 | * Ignore metadata, use statistical information 49 | * Use metadata and use statistical information 50 |
51 | 52 | * ![EC2 Console](img/lab-3/lab3-image8.png) 65 | 66 | 7. In the Amazon Redshift view, open the context (right-click) menu for the dms_sample_dw schema, and then choose Apply to database to apply the schema scripts to the target Amazon Redshift instance. 67 | 68 | > When complete, open the context (right-click) menu for the dms_sample_dw schema, and then choose Refresh from Database to refresh from the target database. Click Yes to confirm. The database schema has now been converted and imported from source to target. 69 | 70 | > ![EC2 Console](img/lab-3/lab3-image7.png) 71 | -------------------------------------------------------------------------------- /Lab-Part4.md: -------------------------------------------------------------------------------- 1 | # Validate the schema conversion and work through conversion issues 2 | 3 | 1. To validate the schema conversion, you compare the objects found in the Oracle and Amazon Redshift databases using SQL Workbench/J. 4 | 5 | > Double click the SQL Workbench shortcut ![EC2 Console](img/lab-4/sql-icon.png) in the desktop folder on your EC2 instance. 6 | 7 | > ![EC2 Console](img/lab-4/lab4-image1.png) 8 | 9 | 2. Click on File, Connect Window and a Select Connection Profile screen will pop up. 10 | 11 | > ![EC2 Console](img/lab-4/lab4-image2.png) 12 | 13 | 3. Create two connections, OracleDW and RedshiftDW with the following information. 14 | 15 | > 16 | Name | OracleDW 17 | ---- | ---- 18 | Driver | Choose the Oracle driver which is installed on your EC2 instance. You can specify the drivers path by clicking on Manage Drivers and pointing the path to C:\Users\Administrator\Desktop\drivers\ojdbc8.jar . 19 | URL | Jdbc url as follows - jdbc:oracle:thin:@OracleEndpoint:1521:DMSDW Get the Reinvent2018ANT371OracleEndpoint from step 8 20 | User name | dms_sample 21 | Password | Refer to the Cloud Formation output for the database password 22 | 23 | Check AutoCommit and click OK. OracleDW will connect to the Source Oracle database. 24 |
25 | 26 | 4. The RedshiftDW will connect to the target Redshift cluster. 27 | 28 | > 29 | Name | RedshiftDW 30 | ---- | ---- 31 | Driver | Choose the Redshift driver which is installed on your EC2 instance. You can specify the drivers path by clicking on Manage Drivers Drivers and pointing the path to C:\Users\Administrator\Desktop\drivers\RedshiftJDBC42-1.2.43.1067.jar . 32 | URL | Jdbc url can be found on Redshift cluster page on the AWS Management Console. Its format is as follows: jdbc:redshift://Redshift Endpoint:5439/dmsdwtarget 33 | User name | dms_sample_dw 34 | Password | Refer to the Cloud Formation output for the database password 35 | 36 | 5. In SQL Workbench/J, choose File, then choose Connect window. Choose the RedshiftConnection you created in an earlier step. Choose OK. 37 | 38 | 6. Run the following script to verify the number of object types and count in dms_sample_dw schema in the target Amazon Redshift database. These values should match the number of objects in the source Oracle database. 39 | 40 | > 41 | ``` 42 | SELECT 'TABLE' AS OBJECT_TYPE, 43 | TABLE_NAME AS OBJECT_NAME, 44 | TABLE_SCHEMA AS OBJECT_SCHEMA 45 | FROM information_schema.TABLES 46 | WHERE TABLE_TYPE = 'BASE TABLE' 47 | AND OBJECT_SCHEMA = 'dms_sample_dw' 48 | ``` 49 | The output from this query should be similar to the following. 50 | 51 | > 52 | ![EC2 Console](img/lab-4/lab4-image3.png) 53 | 54 | 7. Verify the sort and distributions keys that are created in the Amazon Redshift cluster by using the following query. 55 | 56 | > 57 | ``` 58 | SET search_path TO '$user', 'public', 'dms_sample_dw'; 59 | SELECT tablename, 60 | "column", 61 | TYPE, 62 | encoding, 63 | distkey, 64 | sortkey, 65 | "notnull" 66 | FROM pg_table_def 67 | WHERE (distkey = TRUE OR sortkey <> 0); 68 | ``` 69 | The results of the query reflect the distribution key (distkey) and sort key (sortkey) choices made by using AWS SCT key management. 70 | 71 | > 72 | ![EC2 Console](img/lab-4/lab4-image4.png) 73 | -------------------------------------------------------------------------------- /Lab-Part5.md: -------------------------------------------------------------------------------- 1 | # Configuring SCT for AWS data extraction agents 2 | 3 | ## Create IAM Access Key and Access Secret Key for SCT Authentication 4 | 5 | SCT requires IAM Access Key and Access Secret Key for authentication. Best practice is to create a minimum privilege IAM user for this purpose, which you can later delete. 6 | 7 | You will start with creating two IAM policies. One which will allow SCT to list your bucket for the lab, and the other to Delete, Put, and Get objects in the bucket. Next you will create a new IAM user, attach these two policies to the user, then save the Access Key and Access Secret Key to be used in SCT. 8 | 9 | 1. Select IAM in the AWS Console. Select Policies and Create Policy. 10 | 11 | 2. In the Create policy page, select JSON, and copy/paste the following policy. Ensure to replace **YOURBUCKETNAME** with the S3 Bucket name created in CloudFormation. Select Review Policy. 12 | > 13 | ``` 14 | { 15 | "Version": "2012-10-17", 16 | "Statement": [ 17 | { 18 | "Sid": "Stmt1497651415000", 19 | "Effect": "Allow", 20 | "Action": [ 21 | "s3:ListBucket" 22 | ], 23 | "Resource": [ 24 | "arn:aws:s3:::YOURBUCKETNAME" 25 | ] 26 | } 27 | ] 28 | } 29 | ``` 30 | 31 | > ![EC2 Console](img/lab-5/lab5-image1.png) 32 | 33 | 3. In the Review Policy page, provide a policy name, such as ANT371ListBucket and select Create Policy. 34 | 35 | 4. You will return to the IAM Policies page. You need to create one more policy. Select Create Policy, select the JSON tab, and use the following policy. Ensure to replace **YOURBUCKETNAME** with the S3 Bucket name created in CloudFormation. Select Review Policy. 36 | 37 | > 38 | ``` 39 | { 40 | "Version": "2012-10-17", 41 | "Statement": [ 42 | { 43 | "Sid": "Stmt1497631447000", 44 | "Effect": "Allow", 45 | "Action": [ 46 | "s3:DeleteObject", 47 | "s3:GetObject", 48 | "s3:PutObject" 49 | ], 50 | "Resource": [ 51 | "" 52 | ] 53 | } 54 | ] 55 | } 56 | ``` 57 | 58 | 5. In the Review Policy page, provide a policy name, such as ANT371ReadWriteBucket and select Create Policy. 59 | 60 | 6. You now need to create an IAM user. In IAM, go to Users and select Add User. 61 | 62 | 7. In the Add user page, provide a name, such as ANT371User, and select the Programmatic Access checkbox. Select Next. 63 | 64 | > ![EC2 Console](img/lab-5/lab5-image2.png) 65 | 66 | 8. In the Set permissions page, select Attach existing policies directly. In the Filter policies field, enter ANT371. You should see both policies created. Select both and select Next. 67 | 68 | > ![EC2 Console](img/lab-5/lab5-image3.png) 69 | 70 | 71 | 9. In the Review screen select Create User. 72 | 73 | > ![EC2 Console](img/lab-5/lab5-image4.png) 74 | 75 | 10. The final screen shows the IAM Access key and Secret access key. Select Show to view the secret key. 76 | 77 | #### Ensure you save these credentials for SCT. Once you select close the Secret key will no longer be available. 78 | 79 | > ![EC2 Console](img/lab-5/lab5-image5.png) 80 | 81 | 82 | ## AWS SCT Global Settings and Amazon S3 Bucket Configuration 83 | 84 | #### Steps to Create a Global Profile: 85 | 86 | 1. Click on ‘Settings’, ‘Global Settings’ and then select ‘AWS Service Profiles’. 87 | 88 | > ![EC2 Console](img/lab-5/lab5-image6.png) 89 | 90 | 2. Click on “Add a new AWS Service Profile” 91 | 92 | 3. Enter details such as Profile Name, AWS Access Key and AWS Secret Key 93 | 94 | > **NOTE:** You will need an AWS Access Key and AWS Secret Access key in AWS IAM for connecting to the AWS Services. 95 | 96 | 4. Select the region that’s applicable or selected during the execution of CloudFormation script, for example: we suggested to use **Singapore** as your region. 97 | 98 | 5. For the “S3 bucket folder”, use the “Reinvent2018ANT371S3Bucket” name from the output tab of the CloudFormation console. 99 | 100 | > ![EC2 Console](img/lab-5/lab5-image7.png) 101 | 102 | 6. The AWS profile screen with your inputs provided should look similar to the screenshot given below: 103 | > ![EC2 Console](img/lab-5/lab5-image8.png) 104 | 105 | 7. Click Apply button to save the profile and click OK. 106 | -------------------------------------------------------------------------------- /Lab-Part6.md: -------------------------------------------------------------------------------- 1 | # Working with AWS SCT Extraction Agents 2 | 3 | ## Installing and Configuring the AWS SCT Extraction Agents 4 | 5 | In this step, we will install and configure the AWS SCT Extraction Agents on two Linux hosts. SSH to the Linux host using the IP address and private key as mentioned in the below steps. 6 | 7 | ### Steps for SSH login to Agent Extractor hosts 8 | 9 | If you are using windows, please follow the instructions here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/putty.html 10 | 11 | 1. Login to AWS Management Console and note the public IP address of Agent1 instance. 12 | 13 | ![EC2 Console](img/lab-6/lab6-image1.png) 14 | 15 | 2. Open the “Terminal” application ![SSH](img/lab-6/putty.png) in Mac and go to the location where you saved the EC2 Private Key (.pem) file 16 | 17 | > Execute below SSH command to connect to AWS SCT Extraction Agent EC2 host. Please remember to replace with your Keypair file Name and the Public IP address of your EC2 instances. 18 | 19 | > ``` ssh -i ec2-user@ ``` 20 | 21 | 3. Once you are connected to Agent Host using SSH, perform the below steps from the terminal to install and configure the AWS SCT Extraction Agent. 22 | 23 | > **NOTE:** The agent rpm executables and JDBC driver files have been copied to appropriate locations. 24 | 25 | 4. Run below commands from the terminal to install the agent 26 | 27 | > 28 | ``` 29 | cd /home/ec2-user/agents 30 | sudo rpm -ivh aws-schema-conversion-tool-extractor-1.0.620-1.x86_64.rpm 31 | ``` 32 | 33 | 5. Execute the following command to configure the SCT Extraction Agent 34 | 35 | > 36 | ``` 37 | /usr/share/aws/sct-extractor/bin/sct-extractor-setup.sh -configuration 38 | ``` 39 | Specify values for the properties as highlighted below: 40 | 41 | ![EC2 Console](img/lab-6/lab6-image2.png) 42 | 43 | **NOTE:** We selected “no” for “Enable SSL communication” option while configuring the AWS SCT Extraction Agent. This is ONLY for the lab purposes. When you are migrating the production data warehouses on AWS, it is highly recommended to enable SSL communication between the Extraction Agents, AWS SCT, and the source and target database. 44 | 45 | 6. Run the following command to start the Extractor Agent on Host 1 46 | 47 | ``` sudo service sct-extractor start ``` 48 | 49 | 50 | ### Steps for Installing and Configuring the SCT Extraction Agents on Host 2 51 | 52 | ***Please repeat the previous steps (1 to 5) to configure the SCT extractor agent on host 2.*** 53 | 54 | ### Registering Extraction Agents with AWS SCT tool 55 | 56 | Now that we have SCT Extraction Agent installed, configured and started on both the hosts successfully, we can register them with the AWS SCT tool. 57 | 58 | Please follow below steps to register Agents with AWS SCT: 59 | 1. Connect to Windows server using RDP client and open AWS Schema Conversion Tool. 60 | 61 | 2. Select View -> Data Migration View 62 | 63 | ![EC2 Console](img/lab-6/lab6-image3.png) 64 | 65 | 3. From the central panel, click “Register” button under “Agents” tab. 66 | 67 | ![EC2 Console](img/lab-6/lab6-image4.png) 68 | 69 | 4. Fill-in the required details for Agent registration as shown below: 70 | 71 | ![EC2 Console](img/lab-6/lab6-image5.png) 72 | 73 | **Note:** that Hostname is the Private IP Address of your Hosts on which SCT Extractor Agent is running. This address can be obtained from Amazon EC2 console UI or from “Reinvent2018ANT371AGENT1PrivateIP” and “Reinvent2018ANT371AGENT2PrivateIP” outputs in the CloudFormation console. 74 | 75 | 5. Click on “Test Connection” 76 | 77 | ![EC2 Console](img/lab-6/lab6-image6.png) 78 | 79 | 6. Click on “Register” to register the agent with the AWS SCT tool 80 | 81 | ![EC2 Console](img/lab-6/lab6-image7.png) 82 | 83 | 7. Register another agent running on host2 with the AWS SCT tool using the same steps as above. You should see two SCT extraction agents registered in AWS SCT console as given below: 84 | 85 | ![EC2 Console](img/lab-6/lab6-image8.png) 86 | 87 | Note that, in order to connect to AWS SCT Extractors Agents from AWS SCT GUI, entries are already made in the Security Groups associated with Agent hosts in CloudFormation template. 88 | 89 | ## Creating, Running and Monitoring the AWS SCT Data Extraction Tasks 90 | 91 | You can select one or more tables to be migrated at a time when creating Data Extraction Tasks. You can choose all tables, but we recommend against that for performance reasons. We recommend that you create multiple tasks for multiple tables based on the size of the tables in your data warehouse. 92 | 93 | 1. In the AWS SCT application UI, for the current reinventANT371 project in the source oracle database tab, select all the tables under the ***“DMS_SAMPLE_DW”.*** 94 | 95 | Now ***uncheck*** only the ***“DIM_PERSON”*** tables. This table is very large and we will be using Virtual Partitioning feature to migrate these tables to Amazon Redshift. Overall the final table selection for migration would look like the screenshot below: 96 | 97 | ![EC2 Console](img/lab-6/lab6-image9.png) 98 | 99 | 2. Right Click the “Tables” node and select “Create Local Task”. If the “Create Local Task” is not enabled, then click on “Connect to Oracle” button on top of the AWS SCT Window. 100 | 101 | ![EC2 Console](img/lab-6/lab6-image10.png) 102 | 103 | 3. In the “Create local task” window, fill-in the required details as shown below: 104 | 105 | ![EC2 Console](img/lab-6/lab6-image11.png) 106 | 107 | For Migration Mode, you can choose one of the following: 108 | * Extract Only – Extract your data and save the data to your local working folders. 109 | * Extract and Upload – Extract your data and upload your data to Amazon S3. 110 | * Extract, Upload and Copy – Extract your data, upload your data to Amazon S3, and copy it into your Amazon Redshift data warehouse. 111 | 112 | In this lab, we will select the third option, i.e. “Extract, upload and copy”. 113 | 114 | This will extract the data from Oracle data warehouse, move it to Amazon S3 and load the data into Amazon Redshift using COPY command. 115 | 116 | 4. Click ‘Test Task’. ‘Test Validation Passed’ message is displayed. 117 | 118 | ![EC2 Console](img/lab-6/lab6-image12.png) 119 | 120 | 5. Go through various options available under “Advanced” and “AWS S3 settings” tabs. 121 | 122 | ![EC2 Console](img/lab-6/lab6-image13.png) 123 | 124 | ![EC2 Console](img/lab-6/lab6-image14.png) 125 | 126 | 6. Click on ‘Create’ button to create the extraction task. 127 | 128 | ![EC2 Console](img/lab-6/lab6-image15.png) 129 | 130 | 7. Select the task in AWS SCT and press “Start” to start the migration from Oracle data warehouse to Amazon Redshift. 131 | 132 | ![EC2 Console](img/lab-6/lab6-image16.png) 133 | 134 | ![EC2 Console](img/lab-6/lab6-image17.png) 135 | 136 | 8. Observe that the “Extract” and “Upload” steps are executed in parallel. 137 | 138 | 9. Click on each step (Extract, Upload, Copy) in the AWS SCT UI and select “Subtasks” tab. 139 | Observe that for each table a separate subtask is created. 140 | 141 | ![EC2 Console](img/lab-6/lab6-image18.png) 142 | 143 | The subtasks have been distributed between the two extractor agents we had registered with the AWS SCT. 144 | 145 | ## Migrating Large Tables using the Virtual Partitioning Feature 146 | 147 | In AWS SCT, you can create virtual partitions for migrating your data. There are three partition types, which work with specific data types: 148 | • The RANGE partition type works with numeric, date, and time data types. 149 | • The LIST partition type works with numeric, character, and date and time data types. 150 | • The DATE AUTO SPLIT partition type works with date and time data types. 151 | 152 | For more information on Virtual Partitioning, please refer to AWS Documentation at this [page](https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_Agents.DW.html): 153 | 154 | #### Using RANGE Virtual Partitioning to Migrate the PERSON Table 155 | 156 | Steps: 157 | 1. Uncheck all objects from DMS_SCHEMA_DW schema from source side in the AWS SCT UI 158 | 159 | 2. Select ‘DIM_PERSON’ table, right click and select ‘Add Virtual Partitioning’. 160 | 161 | ![EC2 Console](img/lab-6/lab6-image19.png) 162 | 163 | 3. Select ‘Range’ for the ‘Partition Type’ option. 164 | 165 | ![EC2 Console](img/lab-6/lab6-image20.png) 166 | 167 | 4. Select the ID Column, Enter Value as ‘1000000’ and click on ‘+’ icon or press enter. 168 | 169 | 5. Similarly add the 2000000, 3000000 values and press enter. 170 | 171 | ![EC2 Console](img/lab-6/lab6-image21.png) 172 | 173 | **Note** the details displayed under ‘Virtual Partitions’ tab in SCT. 174 | 175 | ![EC2 Console](img/lab-6/lab6-image22.png) 176 | 177 | We will now migrate the data in ‘RANGE – Virtual Partitioned’ table ‘DIM_PERSON’ to Amazon Redshift. 178 | 179 | 6. Right Click the “DIM_PERSON’” table in AWS SCT and select “Create Local Task”. The required steps would be similar to the ones for migrating the other tables in the previous section. Please select ONLY the ‘DIM_PERSON’ table. 180 | 181 | ![EC2 Console](img/lab-6/lab6-image23.png) 182 | 183 | ![EC2 Console](img/lab-6/lab6-image24.png) 184 | 185 | 7. Click on the “Start” button above to start the AWS SCT Extractor Agent Task from the tool. 186 | 187 | 8. Click on a specific Task (such as Extract or Upload) in SCT UI and navigate the “SubTasks” tab. 188 | 189 | ![EC2 Console](img/lab-6/lab6-image25.png) 190 | 191 | You will see that there are four threads running in parallel to extract and move the data to the Amazon S3 bucket. This is because we have created four virtual partitions for the “DIM_PERSON” table. These 4 concurrent threads are divided equally between 2 agents. 192 | 193 | **NOTE:** To move data into Amazon Redshift directly, you would select “Extract, Upload and Copy’ option while creating the task. 194 | -------------------------------------------------------------------------------- /Lab-Part7.md: -------------------------------------------------------------------------------- 1 | # Verify that your data migration completed successfully 2 | 3 | After the migration is complete, connect to SQL Workbench and your Redshift cluster and verify counts in the following tables 4 | 5 | Query | Count 6 | ---- | ---- 7 | select count(*) from dim_person; | 7055276 8 | select count(*) from dim_player; | 5157 9 | select count(*) from dim_sport_location_seats; | 3565082 10 | select count(*) from dim_sport_team;| 62 11 | select count(*) from dim_sporting_event; | 1142 12 | select count(*) from fact_sport_event_ticket_info; | 56530154 13 | 14 | ## Conclusion 15 | 16 | You have completed the workshop. In this workshop you have created a simulated on-premise Oracle source database and used AWS SCT to convert your schema and migrate your data. 17 | 18 | ## Delete all resources 19 | 20 | Make sure you delete the Replication instance. Once the replication instance is deleted you should be able to delete the CloudFormation stack, which will in turn delete all resources. 21 | 22 | **Note: If you do not delete resources they will keep incurring charges.** 23 | 24 | 1. Open the CloudFormation console at this [link](https://console.aws.amazon.com/cloudformation/home) 25 | 26 | 2. Select the checkbox for the stack you created for this lab. 27 | 28 | 3. Select Actions and Delete Stack. Select Yes, Delete. CloudFormation will delete all resources created. 29 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Last Updated - Dec -2020 2 | 3 | ## Data Warehouse Migration to Amazon Redshift 4 | 5 | In few hours, quickly learn how to effectively migrate an existing oracle data warehouse workload to Amazon Redshift using AWS Schema Conversion Tool (SCT) and SCT data extractors. In this session, we will demonstrate the detailed approach and embrace some of the advance features and best practices around various AWS services like AWS Schema Conversion Tool (SCT) , SCT data extractors agents and Amazon Redshift. 6 | 7 | See the diagram below for a depiction of the complete architecture. 8 | 9 | ![Migration Workshop Architecture](img/lab-arch.jpg) 10 | 11 | As shown in the diagram above, multiple SCT extraction agents extract your data from the source data warehouse and upload it to Amazon S3 in parallel. Then, you can use AWS SCT to copy the data automatically to Amazon Redshift, or you can manually load the data from Amazon S3 into Amazon Redshift at a later point in time. 12 | 13 | ## Prerequisites 14 | 15 | Pre-requisites to be completed before creating the stack 16 | 17 | a. **Amazon EC2 Key Pair** - The CloudFormation template requires an Amazon EC2 Key Pair as an input. This key pair is required to access the EC2 instances on which the SCT extraction agents will run. If you do not have an existing key pair in the region, please create one before launching the CloudFormation template. 18 | 19 | **Note: - This lab should be run in the Singapore region. Please make sure you select Singapore region before you go to the next steps outlined below.** 20 | 21 | If you do not have a key in the Singapore region, please follow these steps 22 | 23 | 1. Open the AWS Management console at https://console.aws.amazon.com/ec2/v2/home 24 | 25 | 2. Change the Region to Singapore. 26 | 27 | 3. Click on Key Pairs on the left hand menu and then click on Create Key Pair. Give your key pair a name e.g. Works371 and click Create. A dialog box will open asking you to save the key file. Click on Save File to save the key. Remember where you saved the key or move it to another folder of your choice. You will need this later. 28 | 29 | 30 | b. **IAM Permissions** – You need IAM permissions to create resources such as Amazon VPC, Amazon Redshift cluster, Amazon RDS instances, Amazon EC2 instances and Amazon S3 buckets. 31 | 32 | c. **AWS Limits** - Ensure that you will not exceed the number of EC2 instances allowed by default in a given region (CloudFormation will create three EC2 instances). Ensure that you will not exceed number of VPCs allowed by default in a given region (CloudFormation will create one new VPC). 33 | 34 | # Verify the Source Data Warehouse before Migration 35 | 36 | For the purpose of this workshop, we have used an Amazon RDS for Oracle instance to host the Oracle Data Warehouse. In a real-life situation, this Oracle data warehouse could be on-premises, on Amazon EC2 or on Amazon RDS itself. 37 | 38 | Once the CloudFormation Stack is created, an Amazon RDS instance will be provisioned along with the proper security group, DB instance type, storage type configuration and size as specified in the CloudFormation template. 39 | 40 | **NOTE:** To keep things simple for this lab, we have kept most of the configuration static in the CloudFormation template. 41 | 42 | This Amazon RDS instance is pre-loaded with a sample schema (dms_sample). The sample schema mimics a simple sporting event ticketing application. The following diagram illustrates the data model we will be working in the lab. 43 | 44 | ![CloudFormation](img/lab2-image1.png) 45 | 46 | # Labs 47 | This workshop is broken into multiple labs. You must complete each Lab before proceeding to the next. 48 | 49 | 1. [Lab 1 - Setting up the Lab](Lab-Part1.md) 50 | 51 | 2. [Lab 2 - Connect to your environment](Lab-Part2.md) 52 | 53 | 3. [Lab 3 - Using AWS SCT to convert the Schema](Lab-Part3.md) 54 | 55 | 4. [Lab 4 - Validate the Converted Schema](Lab-Part4.md) 56 | 57 | 5. [Lab 5 - Configuring SCT for AWS data extraction agents](Lab-Part5.md) 58 | 59 | 6. [Lab 6 - Working with AWS SCT Extraction Agents](Lab-Part6.md) 60 | 61 | 7. [Lab 7 - Conclusion](Lab-Part7.md) 62 | 63 | 64 | ## License Summary 65 | 66 | This sample code is made available under a modified MIT license. See the LICENSE file. 67 | -------------------------------------------------------------------------------- /cloudformation/MigrationCloudFormation.json: -------------------------------------------------------------------------------- 1 | { 2 | "AWSTemplateFormatVersion":"2010-09-09", 3 | "Description":"This template creates resources necessary for Migrating Your Oracle Warehouse to Amazon Redshift Using AWS DMS and SCT.", 4 | "Parameters":{ 5 | "KeyName":{ 6 | "Description":"Enter the name of your AWS key pair", 7 | "Type":"AWS::EC2::KeyPair::KeyName" 8 | } 9 | }, 10 | "Mappings" : { 11 | "RegionMap" : { 12 | "us-east-1" : { "32" : "ami-08c0a7138df957552"}, 13 | "eu-west-1" : { "32" : "ami-0f12fdfe4d059c55a"}, 14 | "ap-southeast-1" : { "32" : "ami-0ed1a352c91feff22"} 15 | }, 16 | "SourceRDSSnapshot" : { 17 | "us-east-1" : { "snapshot" : "arn:aws:rds:us-east-1:045618575699:snapshot:dmsdatawarehouse-2018"}, 18 | "ap-southeast-1" : { "snapshot" : "arn:aws:rds:ap-southeast-1:045618575699:snapshot:dmsdatawarehouse111418sgp"}, 19 | "eu-west-1" : { "snapshot" : "arn:aws:rds:eu-west-1:045618575699:snapshot:dmsdatawarehouse111418ire"} 20 | }, 21 | "AgentInstanceRegionMap" : { 22 | "us-east-1" : { "32" : "ami-0ca42ba68f37ccefe"}, 23 | "ap-southeast-1" : { "32" : "ami-0cef60108967e82d3"}, 24 | "eu-west-1" : { "32" : "ami-022941a59f26b6778"} 25 | } 26 | }, 27 | "Resources":{ 28 | "DmsVpc":{ 29 | "Type":"AWS::EC2::VPC", 30 | "Properties":{ 31 | "CidrBlock":"172.31.0.0/16", 32 | "EnableDnsSupport":"true", 33 | "EnableDnsHostnames":"true", 34 | "InstanceTenancy":"default", 35 | "Tags":[ 36 | { 37 | "Key":"Name", 38 | "Value":{ 39 | "Fn::Join":[ 40 | "-", 41 | [ 42 | { 43 | "Ref":"AWS::StackName" 44 | }, 45 | "DmsVpc" 46 | ] 47 | ] 48 | } 49 | } 50 | ] 51 | } 52 | }, 53 | "DMSInternetGateway":{ 54 | "Type":"AWS::EC2::InternetGateway", 55 | "Properties":{ 56 | "Tags":[ 57 | { 58 | "Key":"Name", 59 | "Value":{ 60 | "Fn::Join":[ 61 | "-", 62 | [ 63 | { 64 | "Ref":"AWS::StackName" 65 | }, 66 | "DMSInternetGateway" 67 | ] 68 | ] 69 | } 70 | } 71 | ] 72 | } 73 | }, 74 | "DMSAttachGateway":{ 75 | "Type":"AWS::EC2::VPCGatewayAttachment", 76 | "Properties":{ 77 | "VpcId":{ 78 | "Ref":"DmsVpc" 79 | }, 80 | "InternetGatewayId":{ 81 | "Ref":"DMSInternetGateway" 82 | } 83 | } 84 | }, 85 | "DmsSubnet1":{ 86 | "Type":"AWS::EC2::Subnet", 87 | "Properties":{ 88 | "VpcId":{ 89 | "Ref":"DmsVpc" 90 | }, 91 | "MapPublicIpOnLaunch":"true", 92 | "CidrBlock":"172.31.0.0/24", 93 | "AvailabilityZone":{ 94 | "Fn::Select" : [ "0", { "Fn::GetAZs" : "" } ] 95 | }, 96 | "Tags":[ 97 | { 98 | "Key":"Name", 99 | "Value":{ 100 | "Fn::Join":[ 101 | "-", 102 | [ 103 | { 104 | "Ref":"AWS::StackName" 105 | }, 106 | "DmsSubnet1" 107 | ] 108 | ] 109 | } 110 | } 111 | ] 112 | } 113 | }, 114 | "DmsSubnet2":{ 115 | "Type":"AWS::EC2::Subnet", 116 | "Properties":{ 117 | "VpcId":{ 118 | "Ref":"DmsVpc" 119 | }, 120 | "MapPublicIpOnLaunch":"true", 121 | "CidrBlock":"172.31.8.0/24", 122 | "AvailabilityZone":{ 123 | "Fn::Select" : [ "1", { "Fn::GetAZs" : "" } ] 124 | }, 125 | "Tags":[ 126 | { 127 | "Key":"Name", 128 | "Value":{ 129 | "Fn::Join":[ 130 | "-", 131 | [ 132 | { 133 | "Ref":"AWS::StackName" 134 | }, 135 | "DmsSubnet2" 136 | ] 137 | ] 138 | } 139 | } 140 | ] 141 | } 142 | }, 143 | "DmsSubnet3":{ 144 | "Type":"AWS::EC2::Subnet", 145 | "Properties":{ 146 | "VpcId":{ 147 | "Ref":"DmsVpc" 148 | }, 149 | "MapPublicIpOnLaunch":"true", 150 | "CidrBlock":"172.31.16.0/24", 151 | "AvailabilityZone":{ 152 | "Fn::Select" : [ "2", { "Fn::GetAZs" : "" } ] 153 | }, 154 | "Tags":[ 155 | { 156 | "Key":"Name", 157 | "Value":{ 158 | "Fn::Join":[ 159 | "-", 160 | [ 161 | { 162 | "Ref":"AWS::StackName" 163 | }, 164 | "DmsSubnet3" 165 | ] 166 | ] 167 | } 168 | } 169 | ] 170 | } 171 | }, 172 | "DmsDbSubnetGroup":{ 173 | "Type":"AWS::RDS::DBSubnetGroup", 174 | "Properties":{ 175 | "DBSubnetGroupDescription":"DMS DB Subnet Group", 176 | "SubnetIds":[ 177 | { 178 | "Ref":"DmsSubnet1" 179 | }, 180 | { 181 | "Ref":"DmsSubnet2" 182 | }, 183 | { 184 | "Ref":"DmsSubnet3" 185 | } 186 | ], 187 | "Tags":[ 188 | { 189 | "Key":"Name", 190 | "Value":{ 191 | "Fn::Join":[ 192 | "-", 193 | [ 194 | { 195 | "Ref":"AWS::StackName" 196 | }, 197 | "DmsDbSubnetGroup" 198 | ] 199 | ] 200 | } 201 | } 202 | ] 203 | }, 204 | "DependsOn" : "DmsPublicRoute" 205 | }, 206 | "DmsClusterSubnetGroup" : { 207 | "Type" : "AWS::Redshift::ClusterSubnetGroup", 208 | "Properties" : { 209 | "Description" : "DMS Redshift Cluster Subnet Group", 210 | "SubnetIds":[ 211 | { 212 | "Ref":"DmsSubnet1" 213 | }, 214 | { 215 | "Ref":"DmsSubnet2" 216 | }, 217 | { 218 | "Ref":"DmsSubnet3" 219 | } 220 | ], 221 | "Tags":[ 222 | { 223 | "Key":"Name", 224 | "Value":{ 225 | "Fn::Join":[ 226 | "-", 227 | [ 228 | { 229 | "Ref":"AWS::StackName" 230 | }, 231 | "DmsClusterSubnetGroup" 232 | ] 233 | ] 234 | } 235 | } 236 | ] 237 | } 238 | }, 239 | "DmsPublicRouteTable":{ 240 | "Type":"AWS::EC2::RouteTable", 241 | "Properties":{ 242 | "VpcId":{ 243 | "Ref":"DmsVpc" 244 | }, 245 | "Tags":[ 246 | { 247 | "Key":"Name", 248 | "Value":{ 249 | "Fn::Join":[ 250 | "-", 251 | [ 252 | { 253 | "Ref":"AWS::StackName" 254 | }, 255 | "DmsPublicRouteTable" 256 | ] 257 | ] 258 | } 259 | } 260 | ] 261 | }, 262 | "DependsOn" : "DMSAttachGateway" 263 | }, 264 | "DmsPublicRoute":{ 265 | "Type":"AWS::EC2::Route", 266 | "Properties":{ 267 | "RouteTableId":{ 268 | "Ref":"DmsPublicRouteTable" 269 | }, 270 | "DestinationCidrBlock":"0.0.0.0/0", 271 | "GatewayId":{ 272 | "Ref":"DMSInternetGateway" 273 | } 274 | }, 275 | "DependsOn" : "DMSAttachGateway" 276 | }, 277 | "DmsSubnet1RouteTableAssociation":{ 278 | "Type":"AWS::EC2::SubnetRouteTableAssociation", 279 | "Properties":{ 280 | "SubnetId":{ 281 | "Ref":"DmsSubnet1" 282 | }, 283 | "RouteTableId":{ 284 | "Ref":"DmsPublicRouteTable" 285 | } 286 | } 287 | }, 288 | "DmsSubnet2RouteTableAssociation":{ 289 | "Type":"AWS::EC2::SubnetRouteTableAssociation", 290 | "Properties":{ 291 | "SubnetId":{ 292 | "Ref":"DmsSubnet2" 293 | }, 294 | "RouteTableId":{ 295 | "Ref":"DmsPublicRouteTable" 296 | } 297 | } 298 | }, 299 | "DmsSubnet3RouteTableAssociation":{ 300 | "Type":"AWS::EC2::SubnetRouteTableAssociation", 301 | "Properties":{ 302 | "SubnetId":{ 303 | "Ref":"DmsSubnet3" 304 | }, 305 | "RouteTableId":{ 306 | "Ref":"DmsPublicRouteTable" 307 | } 308 | } 309 | }, 310 | "OracleRDSSecurityGroup":{ 311 | "Type":"AWS::EC2::SecurityGroup", 312 | "Properties":{ 313 | "VpcId":{ 314 | "Ref":"DmsVpc" 315 | }, 316 | "GroupDescription":"Enable access into Oracle RDS DB", 317 | "SecurityGroupIngress":[ 318 | { 319 | "IpProtocol":"tcp", 320 | "FromPort":"1521", 321 | "ToPort":"1521", 322 | "CidrIp":"172.31.0.0/16" 323 | }, 324 | { 325 | "IpProtocol":"tcp", 326 | "FromPort":"1630", 327 | "ToPort":"1630", 328 | "CidrIp":"172.31.0.0/16" 329 | } 330 | ] 331 | } 332 | }, 333 | "RedshiftSecurityGroup":{ 334 | "Type":"AWS::EC2::SecurityGroup", 335 | "Properties":{ 336 | "VpcId":{ 337 | "Ref":"DmsVpc" 338 | }, 339 | "GroupDescription":"Enable access into Redshift cluster", 340 | "SecurityGroupIngress":[ 341 | { 342 | "IpProtocol":"tcp", 343 | "FromPort":"5439", 344 | "ToPort":"5439", 345 | "CidrIp":"172.31.0.0/16" 346 | } 347 | ] 348 | } 349 | }, 350 | "Reinvent2018ANT371InstanceSecurityGroup":{ 351 | "Type":"AWS::EC2::SecurityGroup", 352 | "Properties":{ 353 | "VpcId":{ 354 | "Ref":"DmsVpc" 355 | }, 356 | "GroupDescription":"Enable RDP and Database access into the Instance", 357 | "SecurityGroupIngress":[ 358 | { 359 | "IpProtocol":"tcp", 360 | "FromPort":"3389", 361 | "ToPort":"3389", 362 | "CidrIp":"0.0.0.0/0" 363 | }, 364 | { 365 | "IpProtocol":"tcp", 366 | "FromPort":"1521", 367 | "ToPort":"1521", 368 | "CidrIp":"172.31.0.0/16" 369 | }, 370 | { 371 | "IpProtocol":"tcp", 372 | "FromPort":"8192", 373 | "ToPort":"8192", 374 | "CidrIp":"172.31.0.0/16" 375 | }, 376 | { 377 | "IpProtocol":"tcp", 378 | "FromPort":"5439", 379 | "ToPort":"5439", 380 | "CidrIp":"172.31.0.0/16" 381 | }, 382 | { 383 | "IpProtocol":"tcp", 384 | "FromPort":"22", 385 | "ToPort":"22", 386 | "CidrIp":"0.0.0.0/0" 387 | } 388 | ] 389 | } 390 | }, 391 | "RedshiftS3Role": { 392 | "Type": "AWS::IAM::Role", 393 | "Properties": { 394 | "RoleName": { 395 | "Fn::Join":[ 396 | "-", 397 | [ 398 | { 399 | "Ref":"AWS::StackName" 400 | }, 401 | "redshift-dms-s3" 402 | ] 403 | ] 404 | }, 405 | "AssumeRolePolicyDocument": { 406 | "Version": "2012-10-17", 407 | "Statement": [{ 408 | "Effect": "Allow", 409 | "Principal": {"Service": "redshift.amazonaws.com"}, 410 | "Action": ["sts:AssumeRole"] 411 | }] 412 | }, 413 | "Policies": [{ 414 | "PolicyName": "redshift-dms-s3-role", 415 | "PolicyDocument": { 416 | "Version": "2012-10-17", 417 | "Statement": [ 418 | { 419 | "Effect": "Allow", 420 | "Action": "s3:*", 421 | "Resource": [ 422 | {"Fn::Join": ["", ["arn:aws:s3:::", { "Ref" : "S3Bucket" }]]}, 423 | {"Fn::Join": ["", ["arn:aws:s3:::", { "Ref" : "S3Bucket" },"/*"]]} 424 | ]} 425 | ] 426 | } 427 | }] 428 | } 429 | }, 430 | "OracleSourceDB":{ 431 | "Type" : "AWS::RDS::DBInstance", 432 | "Properties" : { 433 | "DBInstanceClass" : "db.m4.xlarge", 434 | "DBName" : "DMSDW", 435 | "AllocatedStorage" : "100", 436 | "LicenseModel" : "bring-your-own-license", 437 | "Engine" : "oracle-ee", 438 | "EngineVersion" : "12.1.0.2.v13", 439 | "StorageType" : "io1", 440 | "Iops" : "3000", 441 | "Tags" : [ {"Key" : "Application", "Value" : "Oracle RDS instance for ANT317" } ], 442 | "DBSnapshotIdentifier" : { "Fn::FindInMap" : [ "SourceRDSSnapshot", { "Ref" : "AWS::Region" }, "snapshot"]}, 443 | "VPCSecurityGroups" : [ { "Ref" : "OracleRDSSecurityGroup" } ], 444 | "DBInstanceIdentifier" : 445 | { 446 | "Fn::Join":[ 447 | "-", 448 | [ 449 | { 450 | "Ref":"AWS::StackName" 451 | }, 452 | "OracleInstance" 453 | ] 454 | ] 455 | }, 456 | "DBSubnetGroupName":{ 457 | "Ref":"DmsDbSubnetGroup" 458 | }, 459 | "PubliclyAccessible":"true" 460 | } 461 | }, 462 | "RedshiftTargetCluster":{ 463 | "Type": "AWS::Redshift::Cluster", 464 | "Properties": { 465 | "ClusterType": "single-node", 466 | "NodeType": "dc1.large", 467 | "DBName": "dmsdwtarget", 468 | "IamRoles" : [{ 469 | "Fn::GetAtt":[ 470 | "RedshiftS3Role", 471 | "Arn" 472 | ] 473 | }], 474 | "MasterUsername": "dms_sample_dw", 475 | "MasterUserPassword": "Password1", 476 | "VpcSecurityGroupIds" : [ { "Ref" : "RedshiftSecurityGroup" } ], 477 | "ClusterSubnetGroupName":{ 478 | "Ref":"DmsClusterSubnetGroup" 479 | }, 480 | "PubliclyAccessible": "true" 481 | } 482 | }, 483 | "Reinvent2018ANT371Instance":{ 484 | "Type":"AWS::EC2::Instance", 485 | "Properties":{ 486 | "SubnetId":{ 487 | "Ref":"DmsSubnet1" 488 | }, 489 | "InstanceType":"m5.large", 490 | "SecurityGroupIds":[ 491 | { 492 | "Ref":"Reinvent2018ANT371InstanceSecurityGroup" 493 | } 494 | ], 495 | "KeyName":{ 496 | "Ref":"KeyName" 497 | }, 498 | "Tags":[ 499 | { 500 | "Key":"Name", 501 | "Value":{ 502 | "Fn::Join":[ 503 | "-", 504 | [ 505 | { 506 | "Ref":"AWS::StackName" 507 | }, 508 | "SCTWinEC2" 509 | ] 510 | ] 511 | } 512 | } 513 | ], 514 | "BlockDeviceMappings":[ 515 | { 516 | "DeviceName":"/dev/sda1", 517 | "Ebs":{ 518 | "DeleteOnTermination":"true", 519 | "Iops":"7500", 520 | "VolumeSize":"150", 521 | "VolumeType":"io1" 522 | } 523 | } 524 | ], 525 | "ImageId": { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "32"]} 526 | }, 527 | "DependsOn" : "DMSInternetGateway" 528 | }, 529 | "Reinvent2018ANT371AgentInstance1":{ 530 | "Type":"AWS::EC2::Instance", 531 | "Properties":{ 532 | "SubnetId":{ 533 | "Ref":"DmsSubnet1" 534 | }, 535 | "InstanceType":"t2.xlarge", 536 | "SecurityGroupIds":[ 537 | { 538 | "Ref":"Reinvent2018ANT371InstanceSecurityGroup" 539 | } 540 | ], 541 | "KeyName":{ 542 | "Ref":"KeyName" 543 | }, 544 | "Tags":[ 545 | { 546 | "Key":"Name", 547 | "Value":{ 548 | "Fn::Join":[ 549 | "-", 550 | [ 551 | { 552 | "Ref":"AWS::StackName" 553 | }, 554 | "Reinvent2018ANT371AGENTINSTANCE-1" 555 | ] 556 | ] 557 | } 558 | } 559 | ], 560 | "BlockDeviceMappings":[ 561 | { 562 | "DeviceName":"/dev/sda1", 563 | "Ebs":{ 564 | "DeleteOnTermination":"true", 565 | "VolumeSize":"100", 566 | "VolumeType":"gp2" 567 | } 568 | } 569 | ], 570 | "ImageId": { "Fn::FindInMap" : [ "AgentInstanceRegionMap", { "Ref" : "AWS::Region" }, "32"]} 571 | }, 572 | "DependsOn" : "DMSInternetGateway" 573 | }, 574 | "Reinvent2018ANT371AgentInstance2":{ 575 | "Type":"AWS::EC2::Instance", 576 | "Properties":{ 577 | "SubnetId":{ 578 | "Ref":"DmsSubnet1" 579 | }, 580 | "InstanceType":"t2.xlarge", 581 | "SecurityGroupIds":[ 582 | { 583 | "Ref":"Reinvent2018ANT371InstanceSecurityGroup" 584 | } 585 | ], 586 | "KeyName":{ 587 | "Ref":"KeyName" 588 | }, 589 | "Tags":[ 590 | { 591 | "Key":"Name", 592 | "Value":{ 593 | "Fn::Join":[ 594 | "-", 595 | [ 596 | { 597 | "Ref":"AWS::StackName" 598 | }, 599 | "Reinvent2018ANT371AGENTINSTANCE-2" 600 | ] 601 | ] 602 | } 603 | } 604 | ], 605 | "BlockDeviceMappings":[ 606 | { 607 | "DeviceName":"/dev/sda1", 608 | "Ebs":{ 609 | "DeleteOnTermination":"true", 610 | "VolumeSize":"100", 611 | "VolumeType":"gp2" 612 | } 613 | } 614 | ], 615 | "ImageId": { "Fn::FindInMap" : [ "AgentInstanceRegionMap", { "Ref" : "AWS::Region" }, "32"]} 616 | }, 617 | "DependsOn" : "DMSInternetGateway" 618 | }, 619 | "S3Bucket":{ 620 | "Type" : "AWS::S3::Bucket" 621 | } 622 | }, 623 | "Outputs":{ 624 | "Reinvent2018ANT371SCTDNS":{ 625 | "Description":"Public DNS Name of the Reinvent2018ANT371 SCT Windows instance", 626 | "Value":{ 627 | "Fn::GetAtt":[ 628 | "Reinvent2018ANT371Instance", 629 | "PublicDnsName" 630 | ] 631 | } 632 | }, 633 | "Reinvent2018ANT371AGENT1PublicDNS":{ 634 | "Description":"Public DNS Name of the Reinvent2018ANT371 SCT Agent1 EC2 linux instance", 635 | "Value":{ 636 | "Fn::GetAtt":[ 637 | "Reinvent2018ANT371AgentInstance1", 638 | "PublicDnsName" 639 | ] 640 | } 641 | }, 642 | "Reinvent2018ANT371AGENT1PrivateIP":{ 643 | "Description":"Private IP of the Reinvent2018ANT371 SCT Agent1 EC2 linux instance", 644 | "Value":{ 645 | "Fn::GetAtt":[ 646 | "Reinvent2018ANT371AgentInstance1", 647 | "PrivateIp" 648 | ] 649 | } 650 | }, 651 | "Reinvent2018ANT371AGENT2PublicDNS":{ 652 | "Description":"Public DNS Name of the Reinvent2018ANT371 SCT Agent2 EC2 linux instance", 653 | "Value":{ 654 | "Fn::GetAtt":[ 655 | "Reinvent2018ANT371AgentInstance2", 656 | "PublicDnsName" 657 | ] 658 | } 659 | }, 660 | "Reinvent2018ANT371AGENT2PrivateIP":{ 661 | "Description":"Private IP of the Reinvent2018ANT371 SCT Agent2 EC2 linux instance", 662 | "Value":{ 663 | "Fn::GetAtt":[ 664 | "Reinvent2018ANT371AgentInstance2", 665 | "PrivateIp" 666 | ] 667 | } 668 | }, 669 | "Reinvent2018ANT371OracleEndpoint":{ 670 | "Description":"Endpoint of source Oracle RDS DB instance", 671 | "Value":{ 672 | "Fn::GetAtt":[ 673 | "OracleSourceDB", 674 | "Endpoint.Address" 675 | ] 676 | } 677 | }, 678 | "Reinvent2018ANT371RedshiftEndpoint":{ 679 | "Description":"Endpoint of target Redshift instance", 680 | "Value":{ 681 | "Fn::GetAtt":[ 682 | "RedshiftTargetCluster", 683 | "Endpoint.Address" 684 | ] 685 | } 686 | }, 687 | "Reinvent2018ANT371VpcId":{ 688 | "Description":"ANT371 workshop VPC Identifier", 689 | "Value":{ 690 | "Ref":"DmsVpc" 691 | } 692 | }, 693 | "Reinvent2018ANT371S3Bucket":{ 694 | "Description":"ANT371 workshop S3 bucket", 695 | "Value":{ 696 | "Ref":"S3Bucket" 697 | } 698 | }, 699 | "CredDatabaseDetails":{ 700 | "Description":"DB Cluster Login", 701 | "Value": "Password1" 702 | }, 703 | "CredSCTDetails":{ 704 | "Description":"SCT Windows Login", 705 | "Value": "“DWMigration@1" 706 | } 707 | } 708 | } 709 | -------------------------------------------------------------------------------- /img/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/.DS_Store -------------------------------------------------------------------------------- /img/lab-2/lab2-image1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-2/lab2-image1.png -------------------------------------------------------------------------------- /img/lab-2/lab2-image2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-2/lab2-image2.png -------------------------------------------------------------------------------- /img/lab-2/lab2-image3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-2/lab2-image3.png -------------------------------------------------------------------------------- /img/lab-2/lab2-image4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-2/lab2-image4.png -------------------------------------------------------------------------------- /img/lab-2/lab2-image5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-2/lab2-image5.png -------------------------------------------------------------------------------- /img/lab-2/lab2-image6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-2/lab2-image6.png -------------------------------------------------------------------------------- /img/lab-2/lab2-image7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-2/lab2-image7.png -------------------------------------------------------------------------------- /img/lab-2/lab2-image8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-2/lab2-image8.png -------------------------------------------------------------------------------- /img/lab-2/lab2-image9.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-2/lab2-image9.png -------------------------------------------------------------------------------- /img/lab-3/lab3-image1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-3/lab3-image1.png -------------------------------------------------------------------------------- /img/lab-3/lab3-image2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-3/lab3-image2.png -------------------------------------------------------------------------------- /img/lab-3/lab3-image3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-3/lab3-image3.png -------------------------------------------------------------------------------- /img/lab-3/lab3-image4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-3/lab3-image4.png -------------------------------------------------------------------------------- /img/lab-3/lab3-image5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-3/lab3-image5.png -------------------------------------------------------------------------------- /img/lab-3/lab3-image6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-3/lab3-image6.png -------------------------------------------------------------------------------- /img/lab-3/lab3-image7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-3/lab3-image7.png -------------------------------------------------------------------------------- /img/lab-3/lab3-image8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-3/lab3-image8.png -------------------------------------------------------------------------------- /img/lab-4/lab4-image1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-4/lab4-image1.png -------------------------------------------------------------------------------- /img/lab-4/lab4-image2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-4/lab4-image2.png -------------------------------------------------------------------------------- /img/lab-4/lab4-image3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-4/lab4-image3.png -------------------------------------------------------------------------------- /img/lab-4/lab4-image4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-4/lab4-image4.png -------------------------------------------------------------------------------- /img/lab-4/sql-icon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-4/sql-icon.png -------------------------------------------------------------------------------- /img/lab-5/lab5-image1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-5/lab5-image1.png -------------------------------------------------------------------------------- /img/lab-5/lab5-image2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-5/lab5-image2.png -------------------------------------------------------------------------------- /img/lab-5/lab5-image3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-5/lab5-image3.png -------------------------------------------------------------------------------- /img/lab-5/lab5-image4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-5/lab5-image4.png -------------------------------------------------------------------------------- /img/lab-5/lab5-image5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-5/lab5-image5.png -------------------------------------------------------------------------------- /img/lab-5/lab5-image6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-5/lab5-image6.png -------------------------------------------------------------------------------- /img/lab-5/lab5-image7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-5/lab5-image7.png -------------------------------------------------------------------------------- /img/lab-5/lab5-image8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-5/lab5-image8.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image1.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image10.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image11.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image12.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image12.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image13.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image13.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image14.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image14.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image15.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image15.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image16.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image16.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image17.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image17.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image18.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image18.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image19.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image19.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image2.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image20.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image20.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image21.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image21.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image22.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image22.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image23.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image23.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image24.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image24.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image25.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image25.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image3.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image4.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image5.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image6.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image7.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image8.png -------------------------------------------------------------------------------- /img/lab-6/lab6-image9.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/lab6-image9.png -------------------------------------------------------------------------------- /img/lab-6/putty.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-6/putty.png -------------------------------------------------------------------------------- /img/lab-arch.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-arch.jpg -------------------------------------------------------------------------------- /img/lab-arch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab-arch.png -------------------------------------------------------------------------------- /img/lab1-image01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab1-image01.png -------------------------------------------------------------------------------- /img/lab1-image03.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab1-image03.png -------------------------------------------------------------------------------- /img/lab1-image1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab1-image1.png -------------------------------------------------------------------------------- /img/lab1-image2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab1-image2.png -------------------------------------------------------------------------------- /img/lab1-image3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab1-image3.png -------------------------------------------------------------------------------- /img/lab1-image4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab1-image4.png -------------------------------------------------------------------------------- /img/lab1-image5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab1-image5.png -------------------------------------------------------------------------------- /img/lab1-image6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab1-image6.png -------------------------------------------------------------------------------- /img/lab2-image1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-redshift-data-warehouse-migration/5ea113d8386928c739bdb0e18ed450a78a3c95d3/img/lab2-image1.png --------------------------------------------------------------------------------