├── 01-Python └── readme.md ├── 02-Big Data (Hadoop) ├── 00_BigData Intro.md ├── 01_Hadoop Installation.md ├── 02_linux.md ├── 03_hdfs.md ├── 04_Hive │ ├── backup │ │ └── 04_Hive_not_required.md │ ├── h00_intro.md │ ├── h01_Hive_Metastore.md │ ├── h02_Hive_Tables.md │ ├── h03_Hive_Operations.md │ ├── h04_Hadoop File Formats and its Types.md │ ├── h05_Different ways to connecting hive.md │ ├── h06_Partitioning.md │ └── h07_Bucketing.md ├── 05_sqoop.md ├── 1.Assignment-Setting up Hadoop Environment.pptx └── 1.Hadoop 1.x.pptx ├── 03-Azure-Data-Factory(ADF) ├── ADF01_Introduction.md ├── ADF02_Azure Free Account.md ├── ADF03_Storage Account.md ├── ADF04_Azure Sql Database.md ├── ADF05_Azure Data Factory.md ├── ADF06_Integration runtimes.md ├── ADF07_Linked Service.md ├── ADF08_datasets.md ├── ADF09_Pipenine from Blob to Azure Sql Database.md ├── ADF10_Pipenine from Blob to Blob.md ├── ADF11_Pipenine from Blob to MS Sql Server.md ├── ADF12_Pipenine from Azure SQL DB to MS Sql Server DB.md ├── ADF13_0_Pipeline to Load multiple sheets from excel to Azure SQL DB.md ├── ADF13_1_Get Excel Number of Sheets Dynamically and load it.md ├── ADF13_2_Target_file_name_with_timestamp.md ├── ADF13_3_Load_file_if_available.md ├── ADF13_4_Branch Files using Switch.md ├── ADF14_Pipeline to Incrementally load data.md ├── ADF15_Pipeline Scheduling.md ├── ADF16_Execute Pipeline activity.md ├── ADF17_Create Dataflow equal to sql query.md ├── ADF18_Replicate DataFlows logic in copy activity.md ├── ADF19_Split Transformation.md ├── ADF20_Exists_Transformation.md ├── ADF21_0_Lookup_Transformation.md ├── ADF21_UNION_transformation.md ├── ADF22_Derived_Column_Transformation.md ├── ADF23_Data flow libraries.md ├── ADF24_1_Transformations in Stored Procedure.md ├── ADF24_2_Transformations in Stored Procedure.md ├── ADF25_Transformations_in_Azure_FUnctions.md ├── ADF26_Transformations_in_Batch_Account.md ├── ADF27_Logic Apps.md ├── ADF28_git.md ├── ADF29_Azure DevOps.md ├── ADF30_Recursive Files.md ├── ADF31_Change Data Capture(CDC).md ├── ADF31_Start or stop triggers.md ├── archive │ └── ADF Notebook.md ├── cosmos-db │ ├── cdb01. cosmos_db_using_python.md │ ├── cdb02. cosmosdb.ipynb │ ├── cdb03.ADF_pipeline_from_blob_to_cosbosdb.md │ └── image.png ├── labdata │ ├── ParameterFile.json │ ├── ParameterFile1.json │ ├── emp10.csv │ └── working Days.xlsx └── more │ ├── 01.paginated_api.md │ ├── How to Resolve Issues with Commas in CSV File Records.md │ ├── Sharepoint Online to Blob.md │ └── read parameterfile using lookup.md ├── 03_1. Azure Synapse ├── 00. Course Content.md ├── 01. Create an Azure Synapse Workspace.md ├── 02. Azure Synapse Architecture.md ├── 03. ServerLess SQL Pool.md ├── 100. Interview.md └── labdata │ ├── 2019.csv │ ├── 2019.snappy.parquet │ ├── 2020.csv │ ├── 2020.snappy.parquet │ ├── 2021.csv │ ├── 2021.snappy.parquet │ ├── SO43700.json │ ├── SO43701.json │ ├── SO43703.json │ ├── SO43704.json │ ├── SO43705.json │ └── sales.csv ├── 04-Data Bricks ├── 01. PySpark.md ├── 02. Databricks Introduction.md ├── 03. Create DataFrame.md ├── 04. DBFS.md ├── 05. Dataframe Operations.md ├── 06. Write Mode.md ├── 07. Read Modes.md ├── 08. Join Dataframes.md ├── 09.Spark DF Functions.md ├── 10.Spark SQL.md ├── 11. Create Table With UI.md ├── 12. Create Table in Notebook.ipynb ├── 13. Medallion Architecture.md ├── 14. Create Azure Databricks.md ├── 15. Working with Azure Blob Files Sizes.ipynb ├── 16. Delta Lake.ipynb └── archive │ ├── Azure Blob Mount.md │ └── pyspark and DataBricks.md ├── 05. dataFabric └── readme.md ├── ADE Demo.md ├── Lab Data ├── Double Quote enabling.md ├── Load_multiple_sheets_of_excel.xlsx ├── department.csv ├── dept.csv ├── emp.csv ├── employee.csv ├── ford.json ├── input.json ├── iris.csv ├── moviesDB.csv ├── orders.csv ├── people.json ├── retail_db │ ├── categories │ │ └── part-00000 │ ├── customers │ │ └── part-00000 │ ├── departments │ │ └── part-00000 │ ├── order_items │ │ └── part-00000 │ ├── orders │ │ └── part-00000 │ └── products │ │ └── part-00000 ├── retail_db_json │ ├── categories │ │ └── part-r-00000-ce1d8208-178d-48d3-bfb2-1a97d9c05094 │ ├── customers │ │ └── part-r-00000-70554560-527b-44f6-9e80-4e2031af5994 │ ├── departments │ │ └── part-r-00000-3db7cfae-3ad2-4fc7-88ff-afe0ec709f49 │ └── orders │ │ └── part-r-00000-990f5773-9005-49ba-b670-631286032674 ├── salgrade.csv ├── sample-data │ ├── _images │ │ ├── photo1.jpg │ │ ├── photo2.jpg │ │ ├── photo3.jpg │ │ ├── photo4.jpg │ │ └── photo5.jpg │ ├── avro │ │ ├── README.txt │ │ ├── userdata.avsc │ │ ├── userdata1.avro │ │ ├── userdata2.avro │ │ ├── userdata3.avro │ │ ├── userdata4.avro │ │ └── userdata5.avro │ ├── csv │ │ ├── README.txt │ │ ├── userdata1.csv │ │ ├── userdata2.csv │ │ ├── userdata3.csv │ │ ├── userdata4.csv │ │ └── userdata5.csv │ ├── json │ │ ├── books1.json │ │ └── colors.json │ ├── orc │ │ ├── README.txt │ │ ├── userdata1_orc │ │ ├── userdata2_orc │ │ ├── userdata3_orc │ │ ├── userdata4_orc │ │ └── userdata5_orc │ └── parquet │ │ ├── README.txt │ │ ├── userdata1.parquet │ │ ├── userdata2.parquet │ │ ├── userdata3.parquet │ │ ├── userdata4.parquet │ │ └── userdata5.parquet ├── sample.txt ├── students_nested_json_data.json ├── test.csv └── working Days2.xlsx ├── README.md └── miscellaneous ├── 01. Interview.md ├── 02.Incorta Interview Questions.md ├── 03. Interview.md ├── 04.Interview.ipynb ├── ADF01-create-azure-storage-account.md ├── ADF01_Set-up Azure Data Factory.md ├── Azure Data Engineering Interview Questions.md ├── KeyVault └── 01. KeyVault.md ├── Monitoring_for_dummy_file.md ├── SQL ├── Functions.md ├── Stored procedures.md └── stored procedure 2.md ├── notebooks ├── ADF 01 of 01 Copy Data tool.md ├── ADF 01 of 02 Copy activity in pipeline.md ├── ADF 01 of 03 Copy data from on-premises to the cloud.md ├── ADF 01 of 04 Load multiple sheets from excel to azure sql database.md ├── ADF 01 of 05 Run_Python_code_and_load_into_blob.md ├── ADF 01 of 06 Amazon S3 to ADLS2.md ├── ADF 02 of 01 Incrementally load data from Azure SQL Database to Azure Blob storage.md ├── ADF 02 of 02 Incrementally load data from multiple tables in SQL Server to a database in Azure SQL Database.md ├── ADF 02 of 03 Incrementally copy new and changed files based on LastModifiedDate.md ├── ADF 02 of 04 Incrementally copy new files based on time partitioned file name.md ├── ADF 03 of 01 Branching and chaining activities.md ├── databricks │ ├── Spark Sql.ipynb │ └── temp.txt ├── dataflows │ └── adf 01 of 06 Filter and Aggregate Activity.md ├── emp.csv ├── kql │ └── kqlreadme.md └── misc │ └── Azure CLI Scripts.txt └── ref.md /01-Python/readme.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/01-Python/readme.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/00_BigData Intro.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/00_BigData Intro.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/01_Hadoop Installation.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/01_Hadoop Installation.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/02_linux.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/02_linux.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/03_hdfs.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/03_hdfs.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/04_Hive/backup/04_Hive_not_required.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/04_Hive/backup/04_Hive_not_required.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/04_Hive/h00_intro.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/04_Hive/h00_intro.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/04_Hive/h01_Hive_Metastore.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/04_Hive/h01_Hive_Metastore.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/04_Hive/h02_Hive_Tables.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/04_Hive/h02_Hive_Tables.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/04_Hive/h03_Hive_Operations.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/04_Hive/h03_Hive_Operations.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/04_Hive/h04_Hadoop File Formats and its Types.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/04_Hive/h04_Hadoop File Formats and its Types.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/04_Hive/h05_Different ways to connecting hive.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/04_Hive/h05_Different ways to connecting hive.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/04_Hive/h06_Partitioning.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/04_Hive/h06_Partitioning.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/04_Hive/h07_Bucketing.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/04_Hive/h07_Bucketing.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/05_sqoop.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/05_sqoop.md -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/1.Assignment-Setting up Hadoop Environment.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/1.Assignment-Setting up Hadoop Environment.pptx -------------------------------------------------------------------------------- /02-Big Data (Hadoop)/1.Hadoop 1.x.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/02-Big Data (Hadoop)/1.Hadoop 1.x.pptx -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF01_Introduction.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF01_Introduction.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF02_Azure Free Account.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF02_Azure Free Account.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF03_Storage Account.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF03_Storage Account.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF04_Azure Sql Database.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF04_Azure Sql Database.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF05_Azure Data Factory.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF05_Azure Data Factory.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF06_Integration runtimes.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF06_Integration runtimes.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF07_Linked Service.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF07_Linked Service.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF08_datasets.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF08_datasets.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF09_Pipenine from Blob to Azure Sql Database.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF09_Pipenine from Blob to Azure Sql Database.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF10_Pipenine from Blob to Blob.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF10_Pipenine from Blob to Blob.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF11_Pipenine from Blob to MS Sql Server.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF11_Pipenine from Blob to MS Sql Server.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF12_Pipenine from Azure SQL DB to MS Sql Server DB.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF12_Pipenine from Azure SQL DB to MS Sql Server DB.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF13_0_Pipeline to Load multiple sheets from excel to Azure SQL DB.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF13_0_Pipeline to Load multiple sheets from excel to Azure SQL DB.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF13_1_Get Excel Number of Sheets Dynamically and load it.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF13_1_Get Excel Number of Sheets Dynamically and load it.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF13_2_Target_file_name_with_timestamp.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF13_2_Target_file_name_with_timestamp.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF13_3_Load_file_if_available.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF13_3_Load_file_if_available.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF13_4_Branch Files using Switch.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF13_4_Branch Files using Switch.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF14_Pipeline to Incrementally load data.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF14_Pipeline to Incrementally load data.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF15_Pipeline Scheduling.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF15_Pipeline Scheduling.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF16_Execute Pipeline activity.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF16_Execute Pipeline activity.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF17_Create Dataflow equal to sql query.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF17_Create Dataflow equal to sql query.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF18_Replicate DataFlows logic in copy activity.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF18_Replicate DataFlows logic in copy activity.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF19_Split Transformation.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF19_Split Transformation.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF20_Exists_Transformation.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF20_Exists_Transformation.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF21_0_Lookup_Transformation.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF21_0_Lookup_Transformation.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF21_UNION_transformation.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF21_UNION_transformation.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF22_Derived_Column_Transformation.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF22_Derived_Column_Transformation.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF23_Data flow libraries.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF23_Data flow libraries.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF24_1_Transformations in Stored Procedure.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF24_1_Transformations in Stored Procedure.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF24_2_Transformations in Stored Procedure.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF24_2_Transformations in Stored Procedure.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF25_Transformations_in_Azure_FUnctions.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF25_Transformations_in_Azure_FUnctions.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF26_Transformations_in_Batch_Account.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF26_Transformations_in_Batch_Account.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF27_Logic Apps.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF27_Logic Apps.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF28_git.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF28_git.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF29_Azure DevOps.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF29_Azure DevOps.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF30_Recursive Files.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF30_Recursive Files.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF31_Change Data Capture(CDC).md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF31_Change Data Capture(CDC).md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/ADF31_Start or stop triggers.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/ADF31_Start or stop triggers.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/archive/ADF Notebook.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/archive/ADF Notebook.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/cosmos-db/cdb01. cosmos_db_using_python.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/cosmos-db/cdb01. cosmos_db_using_python.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/cosmos-db/cdb02. cosmosdb.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/cosmos-db/cdb02. cosmosdb.ipynb -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/cosmos-db/cdb03.ADF_pipeline_from_blob_to_cosbosdb.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/cosmos-db/cdb03.ADF_pipeline_from_blob_to_cosbosdb.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/cosmos-db/image.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/cosmos-db/image.png -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/labdata/ParameterFile.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/labdata/ParameterFile.json -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/labdata/ParameterFile1.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/labdata/ParameterFile1.json -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/labdata/emp10.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/labdata/emp10.csv -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/labdata/working Days.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/labdata/working Days.xlsx -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/more/01.paginated_api.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/more/01.paginated_api.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/more/How to Resolve Issues with Commas in CSV File Records.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/more/How to Resolve Issues with Commas in CSV File Records.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/more/Sharepoint Online to Blob.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/more/Sharepoint Online to Blob.md -------------------------------------------------------------------------------- /03-Azure-Data-Factory(ADF)/more/read parameterfile using lookup.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03-Azure-Data-Factory(ADF)/more/read parameterfile using lookup.md -------------------------------------------------------------------------------- /03_1. Azure Synapse/00. Course Content.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/00. Course Content.md -------------------------------------------------------------------------------- /03_1. Azure Synapse/01. Create an Azure Synapse Workspace.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/01. Create an Azure Synapse Workspace.md -------------------------------------------------------------------------------- /03_1. Azure Synapse/02. Azure Synapse Architecture.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/02. Azure Synapse Architecture.md -------------------------------------------------------------------------------- /03_1. Azure Synapse/03. ServerLess SQL Pool.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/03. ServerLess SQL Pool.md -------------------------------------------------------------------------------- /03_1. Azure Synapse/100. Interview.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/100. Interview.md -------------------------------------------------------------------------------- /03_1. Azure Synapse/labdata/2019.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/labdata/2019.csv -------------------------------------------------------------------------------- /03_1. Azure Synapse/labdata/2019.snappy.parquet: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/labdata/2019.snappy.parquet -------------------------------------------------------------------------------- /03_1. Azure Synapse/labdata/2020.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/labdata/2020.csv -------------------------------------------------------------------------------- /03_1. Azure Synapse/labdata/2020.snappy.parquet: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/labdata/2020.snappy.parquet -------------------------------------------------------------------------------- /03_1. Azure Synapse/labdata/2021.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/labdata/2021.csv -------------------------------------------------------------------------------- /03_1. Azure Synapse/labdata/2021.snappy.parquet: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/labdata/2021.snappy.parquet -------------------------------------------------------------------------------- /03_1. Azure Synapse/labdata/SO43700.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/labdata/SO43700.json -------------------------------------------------------------------------------- /03_1. Azure Synapse/labdata/SO43701.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/labdata/SO43701.json -------------------------------------------------------------------------------- /03_1. Azure Synapse/labdata/SO43703.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/labdata/SO43703.json -------------------------------------------------------------------------------- /03_1. Azure Synapse/labdata/SO43704.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/labdata/SO43704.json -------------------------------------------------------------------------------- /03_1. Azure Synapse/labdata/SO43705.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/labdata/SO43705.json -------------------------------------------------------------------------------- /03_1. Azure Synapse/labdata/sales.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/03_1. Azure Synapse/labdata/sales.csv -------------------------------------------------------------------------------- /04-Data Bricks/01. PySpark.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/01. PySpark.md -------------------------------------------------------------------------------- /04-Data Bricks/02. Databricks Introduction.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/02. Databricks Introduction.md -------------------------------------------------------------------------------- /04-Data Bricks/03. Create DataFrame.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/03. Create DataFrame.md -------------------------------------------------------------------------------- /04-Data Bricks/04. DBFS.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/04. DBFS.md -------------------------------------------------------------------------------- /04-Data Bricks/05. Dataframe Operations.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/05. Dataframe Operations.md -------------------------------------------------------------------------------- /04-Data Bricks/06. Write Mode.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/06. Write Mode.md -------------------------------------------------------------------------------- /04-Data Bricks/07. Read Modes.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/07. Read Modes.md -------------------------------------------------------------------------------- /04-Data Bricks/08. Join Dataframes.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/08. Join Dataframes.md -------------------------------------------------------------------------------- /04-Data Bricks/09.Spark DF Functions.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/09.Spark DF Functions.md -------------------------------------------------------------------------------- /04-Data Bricks/10.Spark SQL.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/10.Spark SQL.md -------------------------------------------------------------------------------- /04-Data Bricks/11. Create Table With UI.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/11. Create Table With UI.md -------------------------------------------------------------------------------- /04-Data Bricks/12. Create Table in Notebook.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/12. Create Table in Notebook.ipynb -------------------------------------------------------------------------------- /04-Data Bricks/13. Medallion Architecture.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/13. Medallion Architecture.md -------------------------------------------------------------------------------- /04-Data Bricks/14. Create Azure Databricks.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/14. Create Azure Databricks.md -------------------------------------------------------------------------------- /04-Data Bricks/15. Working with Azure Blob Files Sizes.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/15. Working with Azure Blob Files Sizes.ipynb -------------------------------------------------------------------------------- /04-Data Bricks/16. Delta Lake.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/16. Delta Lake.ipynb -------------------------------------------------------------------------------- /04-Data Bricks/archive/Azure Blob Mount.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/archive/Azure Blob Mount.md -------------------------------------------------------------------------------- /04-Data Bricks/archive/pyspark and DataBricks.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/04-Data Bricks/archive/pyspark and DataBricks.md -------------------------------------------------------------------------------- /05. dataFabric/readme.md: -------------------------------------------------------------------------------- 1 | https://github.com/rritec/dataFabric/tree/main 2 | -------------------------------------------------------------------------------- /ADE Demo.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/ADE Demo.md -------------------------------------------------------------------------------- /Lab Data/Double Quote enabling.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/Double Quote enabling.md -------------------------------------------------------------------------------- /Lab Data/Load_multiple_sheets_of_excel.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/Load_multiple_sheets_of_excel.xlsx -------------------------------------------------------------------------------- /Lab Data/department.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/department.csv -------------------------------------------------------------------------------- /Lab Data/dept.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/dept.csv -------------------------------------------------------------------------------- /Lab Data/emp.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/emp.csv -------------------------------------------------------------------------------- /Lab Data/employee.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/employee.csv -------------------------------------------------------------------------------- /Lab Data/ford.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/ford.json -------------------------------------------------------------------------------- /Lab Data/input.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/input.json -------------------------------------------------------------------------------- /Lab Data/iris.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/iris.csv -------------------------------------------------------------------------------- /Lab Data/moviesDB.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/moviesDB.csv -------------------------------------------------------------------------------- /Lab Data/orders.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/orders.csv -------------------------------------------------------------------------------- /Lab Data/people.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/people.json -------------------------------------------------------------------------------- /Lab Data/retail_db/categories/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/retail_db/categories/part-00000 -------------------------------------------------------------------------------- /Lab Data/retail_db/customers/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/retail_db/customers/part-00000 -------------------------------------------------------------------------------- /Lab Data/retail_db/departments/part-00000: -------------------------------------------------------------------------------- 1 | 2|Fitness 2 | 3|Footwear 3 | 4|Apparel 4 | 5|Golf 5 | 6|Outdoors 6 | 7|Fan Shop 7 | -------------------------------------------------------------------------------- /Lab Data/retail_db/order_items/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/retail_db/order_items/part-00000 -------------------------------------------------------------------------------- /Lab Data/retail_db/orders/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/retail_db/orders/part-00000 -------------------------------------------------------------------------------- /Lab Data/retail_db/products/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/retail_db/products/part-00000 -------------------------------------------------------------------------------- /Lab Data/retail_db_json/categories/part-r-00000-ce1d8208-178d-48d3-bfb2-1a97d9c05094: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/retail_db_json/categories/part-r-00000-ce1d8208-178d-48d3-bfb2-1a97d9c05094 -------------------------------------------------------------------------------- /Lab Data/retail_db_json/customers/part-r-00000-70554560-527b-44f6-9e80-4e2031af5994: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/retail_db_json/customers/part-r-00000-70554560-527b-44f6-9e80-4e2031af5994 -------------------------------------------------------------------------------- /Lab Data/retail_db_json/departments/part-r-00000-3db7cfae-3ad2-4fc7-88ff-afe0ec709f49: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/retail_db_json/departments/part-r-00000-3db7cfae-3ad2-4fc7-88ff-afe0ec709f49 -------------------------------------------------------------------------------- /Lab Data/retail_db_json/orders/part-r-00000-990f5773-9005-49ba-b670-631286032674: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/retail_db_json/orders/part-r-00000-990f5773-9005-49ba-b670-631286032674 -------------------------------------------------------------------------------- /Lab Data/salgrade.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/salgrade.csv -------------------------------------------------------------------------------- /Lab Data/sample-data/_images/photo1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/_images/photo1.jpg -------------------------------------------------------------------------------- /Lab Data/sample-data/_images/photo2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/_images/photo2.jpg -------------------------------------------------------------------------------- /Lab Data/sample-data/_images/photo3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/_images/photo3.jpg -------------------------------------------------------------------------------- /Lab Data/sample-data/_images/photo4.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/_images/photo4.jpg -------------------------------------------------------------------------------- /Lab Data/sample-data/_images/photo5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/_images/photo5.jpg -------------------------------------------------------------------------------- /Lab Data/sample-data/avro/README.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/avro/README.txt -------------------------------------------------------------------------------- /Lab Data/sample-data/avro/userdata.avsc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/avro/userdata.avsc -------------------------------------------------------------------------------- /Lab Data/sample-data/avro/userdata1.avro: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/avro/userdata1.avro -------------------------------------------------------------------------------- /Lab Data/sample-data/avro/userdata2.avro: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/avro/userdata2.avro -------------------------------------------------------------------------------- /Lab Data/sample-data/avro/userdata3.avro: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/avro/userdata3.avro -------------------------------------------------------------------------------- /Lab Data/sample-data/avro/userdata4.avro: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/avro/userdata4.avro -------------------------------------------------------------------------------- /Lab Data/sample-data/avro/userdata5.avro: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/avro/userdata5.avro -------------------------------------------------------------------------------- /Lab Data/sample-data/csv/README.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/csv/README.txt -------------------------------------------------------------------------------- /Lab Data/sample-data/csv/userdata1.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/csv/userdata1.csv -------------------------------------------------------------------------------- /Lab Data/sample-data/csv/userdata2.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/csv/userdata2.csv -------------------------------------------------------------------------------- /Lab Data/sample-data/csv/userdata3.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/csv/userdata3.csv -------------------------------------------------------------------------------- /Lab Data/sample-data/csv/userdata4.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/csv/userdata4.csv -------------------------------------------------------------------------------- /Lab Data/sample-data/csv/userdata5.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/csv/userdata5.csv -------------------------------------------------------------------------------- /Lab Data/sample-data/json/books1.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/json/books1.json -------------------------------------------------------------------------------- /Lab Data/sample-data/json/colors.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/json/colors.json -------------------------------------------------------------------------------- /Lab Data/sample-data/orc/README.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/orc/README.txt -------------------------------------------------------------------------------- /Lab Data/sample-data/orc/userdata1_orc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/orc/userdata1_orc -------------------------------------------------------------------------------- /Lab Data/sample-data/orc/userdata2_orc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/orc/userdata2_orc -------------------------------------------------------------------------------- /Lab Data/sample-data/orc/userdata3_orc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/orc/userdata3_orc -------------------------------------------------------------------------------- /Lab Data/sample-data/orc/userdata4_orc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/orc/userdata4_orc -------------------------------------------------------------------------------- /Lab Data/sample-data/orc/userdata5_orc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/orc/userdata5_orc -------------------------------------------------------------------------------- /Lab Data/sample-data/parquet/README.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/parquet/README.txt -------------------------------------------------------------------------------- /Lab Data/sample-data/parquet/userdata1.parquet: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/parquet/userdata1.parquet -------------------------------------------------------------------------------- /Lab Data/sample-data/parquet/userdata2.parquet: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/parquet/userdata2.parquet -------------------------------------------------------------------------------- /Lab Data/sample-data/parquet/userdata3.parquet: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/parquet/userdata3.parquet -------------------------------------------------------------------------------- /Lab Data/sample-data/parquet/userdata4.parquet: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/parquet/userdata4.parquet -------------------------------------------------------------------------------- /Lab Data/sample-data/parquet/userdata5.parquet: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample-data/parquet/userdata5.parquet -------------------------------------------------------------------------------- /Lab Data/sample.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/sample.txt -------------------------------------------------------------------------------- /Lab Data/students_nested_json_data.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/students_nested_json_data.json -------------------------------------------------------------------------------- /Lab Data/test.csv: -------------------------------------------------------------------------------- 1 | emno,ename 2 | 11,ram 3 | -------------------------------------------------------------------------------- /Lab Data/working Days2.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/Lab Data/working Days2.xlsx -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/README.md -------------------------------------------------------------------------------- /miscellaneous/01. Interview.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/01. Interview.md -------------------------------------------------------------------------------- /miscellaneous/02.Incorta Interview Questions.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/02.Incorta Interview Questions.md -------------------------------------------------------------------------------- /miscellaneous/03. Interview.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/03. Interview.md -------------------------------------------------------------------------------- /miscellaneous/04.Interview.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/04.Interview.ipynb -------------------------------------------------------------------------------- /miscellaneous/ADF01-create-azure-storage-account.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/ADF01-create-azure-storage-account.md -------------------------------------------------------------------------------- /miscellaneous/ADF01_Set-up Azure Data Factory.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/ADF01_Set-up Azure Data Factory.md -------------------------------------------------------------------------------- /miscellaneous/Azure Data Engineering Interview Questions.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/Azure Data Engineering Interview Questions.md -------------------------------------------------------------------------------- /miscellaneous/KeyVault/01. KeyVault.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/KeyVault/01. KeyVault.md -------------------------------------------------------------------------------- /miscellaneous/Monitoring_for_dummy_file.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/Monitoring_for_dummy_file.md -------------------------------------------------------------------------------- /miscellaneous/SQL/Functions.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/SQL/Functions.md -------------------------------------------------------------------------------- /miscellaneous/SQL/Stored procedures.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/SQL/Stored procedures.md -------------------------------------------------------------------------------- /miscellaneous/SQL/stored procedure 2.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/SQL/stored procedure 2.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/ADF 01 of 01 Copy Data tool.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/ADF 01 of 01 Copy Data tool.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/ADF 01 of 02 Copy activity in pipeline.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/ADF 01 of 02 Copy activity in pipeline.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/ADF 01 of 03 Copy data from on-premises to the cloud.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/ADF 01 of 03 Copy data from on-premises to the cloud.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/ADF 01 of 04 Load multiple sheets from excel to azure sql database.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/ADF 01 of 04 Load multiple sheets from excel to azure sql database.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/ADF 01 of 05 Run_Python_code_and_load_into_blob.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/ADF 01 of 05 Run_Python_code_and_load_into_blob.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/ADF 01 of 06 Amazon S3 to ADLS2.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/ADF 01 of 06 Amazon S3 to ADLS2.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/ADF 02 of 01 Incrementally load data from Azure SQL Database to Azure Blob storage.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/ADF 02 of 01 Incrementally load data from Azure SQL Database to Azure Blob storage.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/ADF 02 of 02 Incrementally load data from multiple tables in SQL Server to a database in Azure SQL Database.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/ADF 02 of 02 Incrementally load data from multiple tables in SQL Server to a database in Azure SQL Database.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/ADF 02 of 03 Incrementally copy new and changed files based on LastModifiedDate.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/ADF 02 of 03 Incrementally copy new and changed files based on LastModifiedDate.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/ADF 02 of 04 Incrementally copy new files based on time partitioned file name.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/ADF 02 of 04 Incrementally copy new files based on time partitioned file name.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/ADF 03 of 01 Branching and chaining activities.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/ADF 03 of 01 Branching and chaining activities.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/databricks/Spark Sql.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/databricks/Spark Sql.ipynb -------------------------------------------------------------------------------- /miscellaneous/notebooks/databricks/temp.txt: -------------------------------------------------------------------------------- 1 | sa 2 | -------------------------------------------------------------------------------- /miscellaneous/notebooks/dataflows/adf 01 of 06 Filter and Aggregate Activity.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/dataflows/adf 01 of 06 Filter and Aggregate Activity.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/emp.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/emp.csv -------------------------------------------------------------------------------- /miscellaneous/notebooks/kql/kqlreadme.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/kql/kqlreadme.md -------------------------------------------------------------------------------- /miscellaneous/notebooks/misc/Azure CLI Scripts.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rritec/Azure-Cloud-Data-Engineering/HEAD/miscellaneous/notebooks/misc/Azure CLI Scripts.txt -------------------------------------------------------------------------------- /miscellaneous/ref.md: -------------------------------------------------------------------------------- 1 | https://github.com/MicrosoftLearning/DP-203-Data-Engineer 2 | --------------------------------------------------------------------------------