├── LICENSE ├── README.md ├── cloudformation ├── glue_workshop_cfn.yml └── vpc_s3_stack.yml ├── notebooks ├── module1 │ ├── 1_Building_a_DataLake_using_AWS_Glue.ipynb │ └── README.md ├── module2 │ ├── 1_Verify_Source_and_Target_Databases_and_the_Ingestion_Pipeline.ipynb │ ├── 2_Execute_Incremental_Processing_Job_with_AWS_Glue.ipynb │ ├── README.md │ ├── redshift-schema.sql │ └── redshift_utils.py ├── module3 │ ├── 1_Using_AWS_Glue_Python_Shell_Jobs.ipynb │ └── README.md └── resources │ ├── Glue_Workflow.png │ ├── README.md │ ├── activity_flow_1.png │ ├── denormalize.png │ ├── glue_logs.png │ ├── module2_architecture_diagram.png │ └── module3_architecture_diagram.png └── scripts ├── Build_Top_Flight_Delays_Report.py ├── Load_CUSTOMER_DIM.py ├── Load_PRODUCT_DIM.py ├── Load_SALES_ORDER_FACT.py ├── Load_SUPPLIER_DIM.py ├── athena-sql-script.sql ├── incr_load_SALES_ORDER_FACT.py ├── sagemaker-glue-get-stack-info.py └── sagemaker_glue_lifecycle.sh /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/README.md -------------------------------------------------------------------------------- /cloudformation/glue_workshop_cfn.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/cloudformation/glue_workshop_cfn.yml -------------------------------------------------------------------------------- /cloudformation/vpc_s3_stack.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/cloudformation/vpc_s3_stack.yml -------------------------------------------------------------------------------- /notebooks/module1/1_Building_a_DataLake_using_AWS_Glue.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/notebooks/module1/1_Building_a_DataLake_using_AWS_Glue.ipynb -------------------------------------------------------------------------------- /notebooks/module1/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /notebooks/module2/1_Verify_Source_and_Target_Databases_and_the_Ingestion_Pipeline.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/notebooks/module2/1_Verify_Source_and_Target_Databases_and_the_Ingestion_Pipeline.ipynb -------------------------------------------------------------------------------- /notebooks/module2/2_Execute_Incremental_Processing_Job_with_AWS_Glue.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/notebooks/module2/2_Execute_Incremental_Processing_Job_with_AWS_Glue.ipynb -------------------------------------------------------------------------------- /notebooks/module2/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /notebooks/module2/redshift-schema.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/notebooks/module2/redshift-schema.sql -------------------------------------------------------------------------------- /notebooks/module2/redshift_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/notebooks/module2/redshift_utils.py -------------------------------------------------------------------------------- /notebooks/module3/1_Using_AWS_Glue_Python_Shell_Jobs.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/notebooks/module3/1_Using_AWS_Glue_Python_Shell_Jobs.ipynb -------------------------------------------------------------------------------- /notebooks/module3/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /notebooks/resources/Glue_Workflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/notebooks/resources/Glue_Workflow.png -------------------------------------------------------------------------------- /notebooks/resources/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /notebooks/resources/activity_flow_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/notebooks/resources/activity_flow_1.png -------------------------------------------------------------------------------- /notebooks/resources/denormalize.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/notebooks/resources/denormalize.png -------------------------------------------------------------------------------- /notebooks/resources/glue_logs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/notebooks/resources/glue_logs.png -------------------------------------------------------------------------------- /notebooks/resources/module2_architecture_diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/notebooks/resources/module2_architecture_diagram.png -------------------------------------------------------------------------------- /notebooks/resources/module3_architecture_diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/notebooks/resources/module3_architecture_diagram.png -------------------------------------------------------------------------------- /scripts/Build_Top_Flight_Delays_Report.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/scripts/Build_Top_Flight_Delays_Report.py -------------------------------------------------------------------------------- /scripts/Load_CUSTOMER_DIM.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/scripts/Load_CUSTOMER_DIM.py -------------------------------------------------------------------------------- /scripts/Load_PRODUCT_DIM.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/scripts/Load_PRODUCT_DIM.py -------------------------------------------------------------------------------- /scripts/Load_SALES_ORDER_FACT.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/scripts/Load_SALES_ORDER_FACT.py -------------------------------------------------------------------------------- /scripts/Load_SUPPLIER_DIM.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/scripts/Load_SUPPLIER_DIM.py -------------------------------------------------------------------------------- /scripts/athena-sql-script.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/scripts/athena-sql-script.sql -------------------------------------------------------------------------------- /scripts/incr_load_SALES_ORDER_FACT.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/scripts/incr_load_SALES_ORDER_FACT.py -------------------------------------------------------------------------------- /scripts/sagemaker-glue-get-stack-info.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/scripts/sagemaker-glue-get-stack-info.py -------------------------------------------------------------------------------- /scripts/sagemaker_glue_lifecycle.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emrspecialistsamer/aws-glue-workshop/HEAD/scripts/sagemaker_glue_lifecycle.sh --------------------------------------------------------------------------------