├── README.md ├── Section 01 - Introduction about the Course └── Free Download - Course Material ├── Section 02 - Setting up Environment Using AWS Cloud9 └── [Commands] Setup Jupyter Lab on Cloud9 ├── Section 03 - Setting up Environment - Overview of GCP and Provision Ubuntu VM ├── Links │ └── Links └── Setting up Environment │ └── Setup Ubuntu VM on GCP │ ├── 02_signing_up_for_gcp.ipynb │ ├── 03_understanding_gcp_web_console.ipynb │ ├── 04_overview_of_gcp_pricing.ipynb │ ├── 05_provision_ubuntu_18.04_vm.ipynb │ ├── 06_setup_docker.ipynb │ ├── 07_validating_python.ipynb │ └── 08_setup_jupyter_lab.ipynb ├── Section 05 - Setting up Environment - Setup Postgres Database on Ubuntu VM ├── Links │ └── Links ├── Scripts │ ├── create_db_tables_pg.sql │ └── load_db_tables_pg.sql └── Setup Postgres Database │ ├── 02_docker_cheat_sheet.ipynb │ ├── 03_setup_postgres_using_docker.ipynb │ ├── 04_accessing_postgres_using_docker_cli.ipynb │ ├── 05_create_database_and_user.ipynb │ ├── 06_execute_sql_scripts.ipynb │ ├── 08_sql_workbench_and_postgres.ipynb │ ├── 11_jupyter_lab_and_postgresql.ipynb │ └── 12_jupyter_lab_and_postgresql_on_ubuntu_vm.ipynb ├── Section 06 - Database Essentials - Getting Started ├── Getting Started │ ├── 02_connecting_to_database.ipynb │ ├── 03_using_psql.ipynb │ ├── 04_setup_postgres_using_docker.ipynb │ ├── 05_setup_sql_workbench.ipynb │ ├── 06_sql_workbench_and_postgres.ipynb │ ├── 07_sql_workbench_features.ipynb │ ├── 08_data_loading_utilities.ipynb │ └── 09_loading_data_postgres_in_docker.ipynb ├── Links │ └── Links └── Script │ └── 01 - Create Table.sql ├── Section 07 - Database Essentials - Database Operations ├── DML or CRUD Operations │ ├── 04_overview_of_database_operations.ipynb │ ├── 05_crud_operations.ipynb │ ├── 06_creating_table.ipynb │ ├── 07_inserting_data.ipynb │ ├── 08_updating_data.ipynb │ ├── 09_deleting_data.ipynb │ ├── 10_overview_of_transactions.ipynb │ └── 11_exercises_database_operations.ipynb ├── Exercises │ ├── 01 - Create Table.sql │ ├── 02 - Insert Into.sql │ ├── 03 - Updating Data.sql │ ├── 04 - Deleting Data.sql │ └── 05 - Selecting.sql ├── Links │ └── Links └── Scripts │ ├── 01 - Information Schema Tables.sql │ ├── 02 - Information Schema Columns.sql │ ├── 03 - Users.sql │ ├── 04 - Truncate Table.sql │ ├── 05 - Insert Into.sql │ ├── 06 - Update.sql │ └── 07 - Delete.sql ├── Section 08 - Database Essentials - Writing Basic SQL Queries ├── Exercises │ ├── 01 - Customer Order Count.sql │ ├── 02 - Dormant Customers.sql │ ├── 03 - Revenue Per Customer.sql │ ├── 04 - Revenue Per Category.sql │ └── 05 - Product Count Per Department.sql ├── Links │ └── Links ├── Scripts │ ├── 01 - Selecting.sql │ ├── 02 - Distinct.sql │ ├── 03 - Filtering.sql │ ├── 04 - Inner Join.sql │ ├── 05 - Outher Join.sql │ ├── 06 - Aggregations.sql │ ├── 07 - Sorting Data.sql │ └── 08 - Daily.sql └── Writing Basic SQL Queries │ ├── 02_standard_transformations.ipynb │ ├── 03_overview_of_data_model.ipynb │ ├── 04_define_problem_statement.ipynb │ ├── 05_preparing_tables.ipynb │ ├── 06_selecting_or_projecting_data.ipynb │ ├── 07_filtering_data.ipynb │ ├── 08_joining_tables_inner.ipynb │ ├── 09_joining_tables_outer.ipynb │ ├── 10_performing_aggregations.ipynb │ ├── 11_sorting_data.ipynb │ ├── 12_solution_daily_product_revenue.ipynb │ └── 13_exercises_basic_sql_queries.ipynb ├── Section 09 - Database Essentials - Creating Tables and Indexes ├── Creating Tables and Indexes │ ├── 02_data_definition_language.ipynb │ ├── 03_overview_of_data_types.ipynb │ ├── 04_adding_or_modifying_columns.ipynb │ ├── 05_different_types_of_constraints.ipynb │ ├── 06_managing_constraints.ipynb │ ├── 07_indexes_on_tables.ipynb │ ├── 08_indexes_for_constraints.ipynb │ ├── 09_overview_of_sequences.ipynb │ ├── 10_truncating_tables.ipynb │ ├── 11_dropping_tables.ipynb │ ├── 12_Exercises_Managing_Database_Objects_using_Postgresql.ipynb │ └── 13_Solutions_Managing_Database_Objects_in_Postgresql.ipynb ├── Links │ └── Links └── Scripts │ ├── 01 - DDL.sql │ ├── 02 - DT.sql │ ├── 03 - Adding or Modifying Columns.sql │ ├── 04 - Managing Constraints.sql │ ├── 05 - Indexes for Constraints.sql │ ├── 06 - Overview of Sequences.sql │ ├── 07 - Truncating Tables.sql │ └── 08 - Dropping Tables.sql ├── Section 10 - Database Essentials - Partitioning Tables and Indexes ├── Exercises │ ├── 01 - Partitioned tables.sql │ └── 02 - Load Partitioned Tables.sql ├── Links │ └── Links ├── Partitioning Tables and Indexes │ ├── 02_overview_of_partitioning.ipynb │ ├── 03_list_partitioning.ipynb │ ├── 04_managing_partitions_list.ipynb │ ├── 05_manipulating_data.ipynb │ ├── 06_range_partitioning.ipynb │ ├── 07_managing_partitions_range.ipynb │ ├── 08_repartitioning_range.ipynb │ ├── 09_hash_partitioning.ipynb │ ├── 10_managing_partitions_hash.ipynb │ ├── 11_usage_scenarios.ipynb │ ├── 12_sub_partitioning.ipynb │ └── 13_exercises_partitioning_tables.ipynb └── Scripts │ ├── 01 - List Partitioning.sql │ ├── 02 - Managing Partitions - List.sql │ ├── 03 - Manipulating Data.sql │ ├── 04 - Range Partitioning.sql │ ├── 05 - Managing Partitions - Range.sql │ ├── 06 - Repartitioning - Range.sql │ ├── 07 - Hash Partitioning.sql │ ├── 08 - Managing Partitions - Hash.sql │ ├── 09 - Usage Scenarios.sql │ └── 10 - Sub Partitioning.sql ├── Section 11 - Database Essentials - Predefined Functions ├── Links │ └── Links ├── Predefined Functions │ ├── 02_overview_of_predefined_functions.ipynb │ ├── 03_string_manipulation_functions.ipynb │ ├── 04_date_manipulation_functions.ipynb │ ├── 05_overview_of_numeric_functions.ipynb │ ├── 06_data_type_conversion.ipynb │ ├── 07_handling_null_values.ipynb │ └── 08_using_case_and_when.ipynb └── Scripts │ ├── 01 - Overview of Pre-Defined Functions.sql │ ├── 02 - Case Conversion and Length.sql │ ├── 03 - Extracting Data - substr and split_part.sql │ ├── 04 - Using position or strpos.sql │ ├── 05 - Trimming and Padding Functions.sql │ ├── 06 - Reverse and Concatenating multiple strings.sql │ ├── 07 - String Replacement.sql │ ├── 08 - Getting Current Date and Timestamp.sql │ ├── 09 - Date Arithmetic.sql │ ├── 10 - Beginning Date or Time - date_trunc.sql │ ├── 11 - Using to_char and to_date.sql │ ├── 12 - Extracting information - extract.sql │ ├── 13 - Dealing with Unix Timestamp.sql │ ├── 14 - Overview of Numeric Functions.sql │ ├── 15 - Data Type Conversion.sql │ ├── 16 - Handling NULL Values.sql │ └── 17 - Using CASE and WHEN.sql ├── Section 12 - Database Essentials - Writing Advanced SQL Queries ├── Links │ └── Links ├── Scripts │ ├── 01 - Overview of Views.sql │ ├── 02 - Named Queries - Using WITH Clause.sql │ ├── 03 - Overview of Sub Queries.sql │ ├── 04 - CTAS - Create Table as Select.sql │ ├── 05 - Advanced DML Operations.sql │ ├── 06 - Merging or Upserting Data.sql │ ├── 07 - Pivoting Rows into Columns.sql │ ├── 08 - Overview of Analytic Functions.sql │ ├── 09 - Analytic Functions – Aggregations.sql │ ├── 10 - Cumulative or Moving Aggregations.sql │ ├── 11 - Analytic Functions – Windowing.sql │ ├── 12 - Analytic Functions – Ranking.sql │ ├── 13 - Analytic Functions - Filtering.sql │ └── 14 - Ranking and Filtering - Recap.sql └── Writing Advanced SQL Queries │ ├── 02_overview_of_views.ipynb │ ├── 03_named_queries_using_with_clause.ipynb │ ├── 04_overview_of_sub_queries.ipynb │ ├── 05_create_table_as_select.ipynb │ ├── 06_advanced_dml_operations.ipynb │ ├── 07_merging_or_upserting_data.ipynb │ ├── 08_pivoting_rows_into_columns.ipynb │ ├── 09_overview_of_analytic_functions.ipynb │ ├── 10_analytic_functions_aggregations.ipynb │ ├── 11_cumulative_or_moving_aggregations.ipynb │ ├── 12_analytic_functions_windowing.ipynb │ ├── 13_analytic_functions_ranking.ipynb │ ├── 14_analytic_funcions_filtering.ipynb │ ├── 15_ranking_and_filtering_recap.ipynb │ ├── 16_Exercises_SQL_Analytics_or_Windowing_Functions_using_Postgresql.ipynb │ └── 17_Solutions_SQL_Analytics_or_Windowing_Functions_using_Postgres.ipynb ├── Section 13 - Programming Essentials Using Python - Perform Database Operations ├── Exercises │ ├── 01 - Create Table.sql │ ├── 02 - Insert Into.sql │ ├── 03 - Updating Data.sql │ ├── 04 - Deleting Data.sql │ └── 05 - Selecting.sql ├── Links │ └── Links ├── Postgres Database Operations │ ├── 02_overview_of_sql.ipynb │ ├── 03_create_database.ipynb │ ├── 04_ddl_data_definition_language.ipynb │ ├── 05_dml.ipynb │ ├── 06_dql_data_query_language.ipynb │ ├── 07_crud_operations_dml_and_dql.ipynb │ ├── 08_tcl.ipynb │ ├── 09_example_data_engineering.ipynb │ ├── 10_example_web_application.ipynb │ └── 11_exercise_database_operations.ipynb └── Scripts │ ├── 01 - Create Database and Users Table.sql │ ├── 02 - DDL – Data Definition Language.sql │ ├── 03 - Dml Data Manipulation Language.sql │ └── 04 - DQL – Data Query Language.sql ├── Section 14 - Programming Essentials Using Python - Getting Started with Python └── Links ├── Section 15 - Programming Essentials Using Python - Basic Programming Constructs ├── Links │ └── Links └── Python Code │ ├── 01_Basic_Programming_Constructs.ipynb │ ├── 02_getting_help.ipynb │ ├── 03_variables_and_objects.ipynb │ ├── 04_data_types_commonly_used.ipynb │ ├── 05_operators_in_python.ipynb │ ├── 06_tasks_data_types_and_operators.ipynb │ ├── 07_conditionals.ipynb │ ├── 08_all_about_for_loops.ipynb │ ├── 09_running_os_commands.ipynb │ ├── 10 - Exercises - Basic Programming Constructs.ipynb │ └── 11 - Eval and Exec.ipynb ├── Section 16 - Programming Essentials Using Python - Predefined Functions ├── Links │ └── Links └── Python Code │ ├── 01 - Pre-defined Functions.ipynb │ ├── 02_overview_of_pre-defined_functions.ipynb │ ├── 03_numeric_functions.ipynb │ ├── 04_overview_of_strings.ipynb │ ├── 05_string_manipulation_functions.ipynb │ ├── 06_formatting_strings.ipynb │ ├── 07_print_and_input_functions.ipynb │ ├── 08_date_manipulation_functions.ipynb │ ├── 09_special_functions.ipynb │ └── 10_exercises_pre_defined_functions.ipynb ├── Section 17 - Programming Essentials Using Python - User Defined Functions ├── Links │ └── Links └── Python Code │ ├── 01_user_defined_functions.ipynb │ ├── 02_defining_functions.ipynb │ ├── 03_doc_strings.ipynb │ ├── 04_returning_values.ipynb │ ├── 05_function_parameters_and_arguments.ipynb │ ├── 06_varying_arguments.ipynb │ ├── 07_keyword_arguments.ipynb │ ├── 08_recap_of_user_defined_functions.ipynb │ ├── 09_passing_functions_as_arguments.ipynb │ ├── 10_lambda_functions.ipynb │ ├── 11_usage_of_lambda_functions.ipynb │ └── 12_exercise_user_defined_functions.ipynb ├── Section 18 - Programming Essentials Using Python - Overview of Collections - List and Set ├── Links │ └── Links └── Python Code │ ├── 01_Overview of Collections - list and set.ipynb │ ├── 02_overview.ipynb │ ├── 03_common_operations.ipynb │ ├── 04_accessing_elements_from_list.ipynb │ ├── 05_adding_elements.ipynb │ ├── 06_updating_and_deleting_elements_list.ipynb │ ├── 07_other_list_operations.ipynb │ ├── 08_adding_and_deleting_elements_set.ipynb │ ├── 09_typical_set_operations.ipynb │ ├── 10_validating_set.ipynb │ ├── 11_list_and_set_usage.ipynb │ ├── 12_Overview of Strings.ipynb │ ├── 13_exercises_list_and_set.ipynb │ ├── 14_List_of_Delimited_strings.ipynb │ ├── 15_sorting_data_in_lists_and_tuples.ipynb │ ├── 16_sorting_list_of_delimete_strings.ipynb │ └── 17_Exercises_sorting_lists_and_sets.ipynb ├── Section 19 - Programming Essentials Using Python - Overview of Collections - Dict and Tuple ├── Links │ └── Links └── Python Code │ ├── 01_manipulating_collections_using_loops.ipynb │ ├── 02_overview_of_dict_and_tuple.ipynb │ ├── 03_common_operations.ipynb │ ├── 04_accessing_elements_tuples.ipynb │ ├── 05_accessing_elements_dict.ipynb │ ├── 06_manipulating_dict.ipynb │ ├── 07_common_examples_dict.ipynb │ ├── 08_list_of_tuples.ipynb │ ├── 09_list_of_dicts.ipynb │ ├── 10_process_dict_values.ipynb │ ├── 11_process_dict_items.ipynb │ ├── 12_sorting_dict_items.ipynb │ └── 13_exercises.ipynb ├── Section 20 - Programming Essentials Using Python - Manipulating Collections using Loops ├── Links │ └── Links └── Python Code │ ├── 01_manipulating_collections_using_loops.ipynb │ ├── 02_reading_files_into_collections.ipynb │ ├── 03_overview_of_standard_transformations.ipynb │ ├── 04_row_level_transformations.ipynb │ ├── 05_getting_unique_elements.ipynb │ ├── 06_filtering_data.ipynb │ ├── 07_preparing_data_sets.ipynb │ ├── 08_quick_recap_of_dict_operations.ipynb │ ├── 09_performing_total_aggregations.ipynb │ ├── 10_manipulate_collections_using_comprehensions.ipynb │ ├── 11_joining_data_sets.ipynb │ ├── 12_limitations_of_using_loops.ipynb │ ├── 13_overview_of_grouped_aggregations.ipynb │ ├── 14_get_order_count_by_status.ipynb │ ├── 15_get_revenue_details_per_order.ipynb │ ├── 16_get_order_count_by_month.ipynb │ ├── 17_list_comprehensions.ipynb │ ├── 18_set_comprehensions.ipynb │ ├── 19_dict_comprehensions.ipynb │ └── 20_exercises_manipulating_collections_using_loops.ipynb ├── Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs ├── Links │ └── Links └── Python Code │ ├── 01_development_of_map_reduce_apis.ipynb │ ├── 02_develop_myFilter.ipynb │ ├── 03_validate_myFilter.ipynb │ ├── 04_develop_myMap.ipynb │ ├── 05_validate_myMap.ipynb │ ├── 06_develop_myReduce.ipynb │ ├── 07_validate_myReduce.ipynb │ ├── 08_develop_myReduceByKey.ipynb │ ├── 09_validate_myReduceByKey.ipynb │ ├── 10_develop_myJoin.ipynb │ ├── 11_validate_myJoin.ipynb │ └── 12_exercises_custom_map_reduce_functions.ipynb ├── Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries ├── 01 - Preparing Data Sets.ipynb ├── 02 - Filtering Data using Filter.ipynb ├── 03 - Projecting Data using Map.ipynb ├── 04 - Row Level Transformations using Map.ipynb ├── 05 - Aggregations using Reduce.ipynb ├── 06 - Get Revenue for a Given Product Id.ipynb ├── 07 - Get total items sold and revenue.ipynb ├── 08 - Get Total Commission Amount.ipynb ├── 09 - Overview of Itertools.ipynb ├── 10 - Cumulative Operations using itertools.ipynb ├── 11 - Using itertools starmap.ipynb ├── 12 - Overview of itertools groupby.ipynb ├── 13 - Order Count by Status using itertools.ipynb ├── 14 - Revenue per Order using itertools.ipynb ├── 15 - Limitations of Map Reduce Libraries.ipynb └── 16 - Exercises - Map Reduce Functions.ipynb ├── Section 23 - Programming Essentials Using Python - Basics of File IO using Python ├── 01 - Overview of File IO.ipynb ├── 02 - Folders and Files.ipynb ├── 03 - File Paths and Names.ipynb ├── 04 - Ovewrview of Retail Data.ipynb ├── 05 - Read text file into string.ipynb ├── 06 - Write string to text file.ipynb ├── 07 - Overview of modes to write into files.ipynb ├── 08 - Overview of Delimited Strings.ipynb ├── 09 - Read csv into list of strings.ipynb ├── 10 - Basics of File IO using Python.ipynb ├── 11 - Write Strings to File in Append Mode.ipynb └── 12 - Managing Folders and Files using Python.ipynb ├── Section 24 - Programming Essentials Using Python - Delimited Files and Collections ├── 01 - Overview of Delimited Text Files.ipynb ├── 02 - Recap of Basic File IO.ipynb ├── 03 - Read Delimited Files into list of tuples.ipynb ├── 04 - Write Delimited Strings into Files.ipynb ├── 05 - Overview of CSV Module.ipynb ├── 06 - Read Delimited data using CSV.ipynb ├── 07 - Delimited File and Collections.ipynb ├── 08 - Write iterables to files using CSV.ipynb ├── 09 - Advantages of using CSV.ipynb └── 10 - Apply Schema on the lists from files.ipynb ├── Section 25 - Programming Essentials Using Python - Overview of Pandas Libraries ├── Links │ └── Links └── Python Code │ ├── 01_overview_of_pandas_libraries.ipynb │ ├── 02_pandas_data_structures_overview.ipynb │ ├── 03_overview_of_series.ipynb │ ├── 04_creating_data_frames_from_lists.ipynb │ ├── 05_data_frames_basic_operations.ipynb │ ├── 06_csv_to_pandas_data_frame.ipynb │ ├── 07_projecting_and_filtering.ipynb │ ├── 08_performing_total_aggregations.ipynb │ ├── 09_performing_grouped_aggregations.ipynb │ ├── 10_writing_data_frames_to_files.ipynb │ └── 11_joining_data_frames.ipynb ├── Section 26 - Programming Essentials Using Python - Database Programming CRUD Operations ├── Links │ └── Links ├── Postgre Code │ ├── 01 - Creating Table.sql │ ├── 02 - Inserting Data into Table.sql │ ├── 03 - Updating Existing Table Data.sql │ ├── 04 - Deleting Data from Table.sql │ └── 05 - Querying Data from Table.sql └── Python Code │ ├── 01_database_programming_crud_operations.ipynb │ ├── 02_overview_of_database_programming.ipynb │ ├── 03_recap_of_rdbms_concepts.ipynb │ ├── 04_setup_database_client_libraries.ipynb │ ├── 05_function_get_database_connection.ipynb │ ├── 06_creating_database_table.ipynb │ ├── 07_inserting_data_into_table.ipynb │ ├── 08_updating_existing_table_data.ipynb │ ├── 09_deleting_data_from_table.ipynb │ ├── 10_querying_data_from_table.ipynb │ └── 11_crud.ipynb ├── Section 27 - Programming Essentials Using Python - Database Programming Bath Operation ├── Links │ └── Links ├── Postgre Code │ ├── 01 - Insert.sql │ └── 02 - Select.sql └── Python Code │ ├── 01_database_programming_batch_operations.ipynb │ ├── 02_overview_of_database_programming.ipynb │ ├── 03_recap_of_rdbms_concepts.ipynb │ ├── 04_recap_of_insert.ipynb │ ├── 05_function_get_database_connection.ipynb │ ├── 06_creating_database_table.ipynb │ ├── 07_inserting_data_into_table.ipynb │ ├── 08_updating_existing_table_data.ipynb │ ├── 09_deleting_data_from_table.ipynb │ ├── 10_querying_data_from_table.ipynb │ ├── 11_crud.ipynb │ ├── 12_preparing_database.ipynb │ ├── 13_reading_data_from_file.ipynb │ ├── 14 - Batch Loading of data.ipynb │ └── 15 - Best Practices - Batch Loading.ipynb ├── Section 28 - Programming Essentials Using Python - Processing JSON Data ├── 01 - Overview of JSON.ipynb ├── 02 - JSON Data Types.ipynb ├── 03 - Create JSON String.ipynb ├── 04 - Process JSON String.ipynb ├── 05 - Single JSON in Files.ipynb ├── 06 - Multiple JSON Documents in files.ipynb ├── 07 - Process JSON using Pandas.ipynb ├── 08 - Differente JSON Formats supported by Pandas.ipynb ├── 09 - Common Use Cases for JSON.ipynb ├── 10 - Processing JSON Data.ipynb ├── 11 - Write to JSON Files using JSON.ipynb └── 12 - Write to JSON files using pandas.ipynb ├── Section 29 - Programming Essentials Using Python - Processing REST Payloads ├── 01_Overview_of_REST_APIs.ipynb ├── 02_Using_curl_command.ipynb ├── 03_Overview_of_Postman.ipynb ├── 04_Getting_Started_with_requests.ipynb ├── 05_Convert_REST_Payload_to_Python_Objects.ipynb ├── 06_Process_REST_Payload_using_Collection_Operations.ipynb └── 07_Process_REST_Payload_using_pandas.ipynb ├── Section 30 - Understanding Python Virtual Environments ├── 01_Accessing_Python_through_PowerShell.txt ├── 02_Create_Virtual_Environment_for_Web_Application.txt ├── 03_Reviewing_Dependencies_Virtual_Environment.txt ├── 04_Installing_Dependencies_Web_applications.txt ├── 05_Getting_Details_About_PIP.txt ├── 06_Uninstall_Packages_Using_PIP.txt ├── 07_Cleanup_Virtual_Environment.txt ├── 08_Recreate_and_Activate_Virtual_Environment_Web_Application.txt ├── 09_Define_Requeriments_File_Web_Application.txt ├── 10_Install_Dependencies_Web_Application.txt ├── 11_Create_Virtual_Environment_Data_Engineering.txt └── 12_Install_Dependencias_data_Engineering_Application.txt ├── Section 31 - Overview of Pycharm for Python Application Development ├── Link │ └── 01_Installation_of_Pycharm_Windowns.txt └── gettingStarted │ └── test.py ├── Section 32 - Data Copier - Getting Started ├── CMD Codes │ ├── 01_Codes_CMD │ ├── 02_Listing_the_images │ └── 03_Checking_all_docker_commands └── Manual │ ├── 01_Getting_Started.ipynb │ ├── 02_Problem_Statement_Data_Copier.ipynb │ ├── 03_Setup_Docker.ipynb │ ├── 04_Quick_Overview_of_Docker.ipynb │ ├── 05_Prepare_Dataset.ipynb │ ├── 06_Setup_Postgres_Database.ipynb │ ├── 07_Overview_of_Postgres_Database.ipynb │ ├── 08_Setup_Project_using_PyCharm.ipynb │ ├── 09_Managing_Dependencies.ipynb │ └── 10_Create_GitHub_Repository.ipynb ├── Section 33 - Data Copier - Reading Data using Pandas ├── 01_Reading_Data_using_Pandas_Introduction.ipynb ├── 02_Overview_of_Retail_Data.ipynb ├── 03_Adding_Pandas_to_Project.ipynb ├── 04_Reading_Data_using_Pandas.ipynb ├── 05_Previewing_Data_using_Pandas.ipynb ├── 06_Reading_Data_in Chunks_using_Pandas.ipynb └── 07_Dynamically_Read_Files.ipynb ├── Section 34 - Data Copier - Database Programming using Pandas ├── 01_Database_Programming_using_Pandas.ipynb ├── 02_Validate_Postgres_Setup_using_Docker.ipynb ├── 03_Add_Required_Dependencies.ipynb ├── 04_Create_Users_Table_in_retail_db_Database.ipynb ├── 05_Populating_Sample_Data_into_Users_Table.ipynb ├── 06_Reading_Data_from_Table_using_Pandas.ipynb ├── 07_Truncate_Users_Table.ipynb ├── 08_Writing_Pandas_Dataframe_to_Table.ipynb ├── 09_Validating_Users_Data.ipynb └── 10_Dropping_Users_Table.ipynb ├── Section 35 - Data Copier - Loading Data from Files to Tables ├── 01_Loading_Data_from_Files_to_Tables.ipynb ├── 02_Populating_Departments_Table.ipynb ├── 03_Validate_Departments_Table.ipynb ├── 04_Populating_Orders_Table.ipynb ├── 05_Validate_Orders_Table_in_Database.ipynb └── 06_Validate_Orders_Table_using_Pandas.ipynb ├── Section 36 - Data Copier - Modularizing the Application ├── Manual │ ├── 01_Modularizing_Data_Copier.ipynb │ ├── 02_Overview_of_main_Function.ipynb │ ├── 03_Overview_of_Environment_Variables.ipynb │ ├── 04_Using_os_Module_for_Environment_Variables.ipynb │ ├── 05_Passing_Environment_Variables_using_Pycharm.ipynb │ ├── 06_Read_Logic_using_Pandas.ipynb │ ├── 07_Validate_Read_Logic.ipynb │ ├── 08_Write_Logic_using_Pandas.ipynb │ ├── 09_Validate_Write_Logic.ipynb │ ├── 10_Integrate_Read_and_Write_Logic.ipynb │ ├── 11_Validate_Integration_Logic.ipynb │ ├── 12_Develop_Logic_for_ultiple_Tables.ipynb │ └── 13_Validate_Logic_for_Multiple_Tables.ipynb └── Python Code │ ├── app.py │ ├── read.py │ ├── requirements.txt │ └── write.py ├── Section 37 - Data Copier - Dockerizing the Application ├── Manual │ ├── 01_Prepare_database_for_validation.txt │ ├── 02_Pull_and_validate_python_image.txt │ ├── 03_Create_and_attach_network_to_database.txt │ ├── 04_Quick_recap_about_docker_conatiners.txt │ ├── 05_Deploying_application_and_installing_docker │ ├── 06_Copy_source_data_file_into_container │ ├── 07_Add_data_copier_container │ ├── 08_Installing_os_libraries_docker │ ├── 09_Validate_network_connectivity │ ├── 10_Running_application_docker │ └── 11_Delete_docker_container └── Python Code │ ├── app.py │ ├── read.py │ ├── requirements.txt │ └── write.py ├── Section 38 - Data Copier - Using Custom Docker Image ├── Manual │ ├── 01_Getting_started_with_docker_image.txt │ ├── 02_Install_OS_modules.txt │ ├── 03_Copying_source_code.txt │ ├── 04_Adding_dependencies.txt │ ├── 05_Unsderstanding_docker.txt │ ├── 06_Mounting_data_folders.txt │ ├── 07_Add_data_copier_container.txt │ └── 08_Run_application_using_docker.txt └── Python Code │ └── app │ ├── Dockerfile.txt │ ├── app.py │ ├── read.py │ ├── requirements.txt │ └── write.py ├── Section 39 - Data Copier - Deploy and Validate Application on Remote Server ├── 01_Clone_application_on_remote.txt ├── 02_Setup_network_and_database.txt ├── 03_Setup_database_and tables_docker.txt ├── 04_Building_custom_docker.txt └── 05_Run_and_validate_dockerized_application.txt ├── Section 40 - Setup Single Node Hadoop and Spark Cluster or Lab using Docker ├── 01_setup_single_node_hadoop_cluster │ ├── 01_introduction_to_single_node_hadoop_cluster.ipynb │ ├── 02_setup_prerequisites.ipynb │ ├── 03_setup_passwordless_login.ipynb │ ├── 04_download_and_install_hadoop.ipynb │ ├── 05_configure_hadoop_hdfs.ipynb │ ├── 06_start_and_validate_hdfs.ipynb │ ├── 07_configure_hadoop_yarn.ipynb │ ├── 08_start_and_validate_yarn.ipynb │ ├── 09_managing_single_node_hadoop.ipynb │ └── 10_accessing_hadoop_uris.ipynb ├── 02_setup_hive_and_spark │ ├── 02_setup_data_sets_for_practice.ipynb │ ├── 03_download_and_install_hive.ipynb │ ├── 04_setup_database_for_hive_metastore.ipynb │ ├── 05_configure_and_setup_hive_metastore.ipynb │ ├── 06_launch_and_validate_hive.ipynb │ ├── 07_scripts_to_manage_single_node_cluster.ipynb │ ├── 08_download_and_install_spark2.ipynb │ ├── 09_configure_spark2.ipynb │ ├── 10_validate_spark2_using_clis.ipynb │ ├── 11_validate_jupyterlab_setup.ipynb │ ├── 12_integrate_spark2_with_jupyterlab.ipynb │ ├── 13_download_and_install_spark3.ipynb │ ├── 14_configure_spark3.ipynb │ ├── 15_validate_spark3_clis.ipynb │ └── 16_integrate_spark3_with_jupyterlab.ipynb └── 03_setup_single_node_kafka_cluster │ ├── 02_download_and_install_kafka.ipynb │ ├── 03_configure_and_start_zookeeper.ipynb │ ├── 04_configure_and_start_kafka_broker.ipynb │ ├── 05_scripts_to_manage_single_node_cluster.ipynb │ ├── 06_overview_of_kafka_cli.ipynb │ ├── 07_setup_retail_log_generator.ipynb │ └── 08_redirecting_logs_to_kafka_topic.ipynb ├── Section 41 - Introduction to Hadoop Eco System - Overview of HDFS ├── 03_getting_help_or_usage.ipynb ├── 04_listing_hdfs_files.ipynb ├── 05_managing_hdfs_directories.ipynb ├── 06_copying_files_from_local_to_hdfs.ipynb ├── 07_copying_files_from_hdfs_to_local.ipynb ├── 09_previewing_data_in_hdfs_files.ipynb ├── 10_getting_file_metadata.ipynb ├── 11_hdfs_blocksize.ipynb ├── 12_hdfs_replication_factor.ipynb ├── 13_getting_hdfs_storage_usage.ipynb ├── 14_using_hdfs_stat_command.ipynb ├── 15_hdfs_file_permissions.ipynb └── 16_overriding_properties.ipynb ├── Section 42 - Data Engineering using Spark SQL - Getting Started ├── 01_getting_started.ipynb ├── 02_overview_of_spark_documentation.ipynb ├── 03_overview_of_spark_sql_cli.ipynb ├── 04_overview_of_spark_sql_properties.ipynb ├── 05_running_os_commands.ipynb ├── 06_understanding_warehouse_directory.ipynb ├── 07_managing_spark_metastore_databases.ipynb ├── 08_managing_spark_metastore_tables.ipynb ├── 09_retrieve_metadata_of_tables.ipynb ├── 10_role_of_spark_or_hive_metastore.ipynb └── 11_exercise_getting_started.ipynb ├── Section 43 - Data Engineering using Spark SQL - Basic Transformations ├── 01_basic_transformations.ipynb ├── 02_spark_sql_overview.ipynb ├── 03_define_problem_statement.ipynb ├── 04_preparing_tables.ipynb ├── 05_projecting_data.ipynb ├── 06_filtering_data.ipynb ├── 07_joining_tables_inner.ipynb ├── 08_joining_tables_outer.ipynb ├── 09_aggregating_data.ipynb ├── 10_sorting_data.ipynb └── 11_conclusion_final_solution.ipynb ├── Section 44 - Data Engineering using Spark SQL - Basic DDL and DML ├── 01_basic_ddl_and_dml.ipynb ├── 02_create_spark_metastore_tables.ipynb ├── 03_overview_of_data_types.ipynb ├── 04_adding_comments.ipynb ├── 05_loading_data_into_tables_local.ipynb ├── 06_loading_data_into_tables_hdfs.ipynb ├── 07_loading_data_append_and_overwrite.ipynb ├── 08_creating_external_tables.ipynb ├── 09_managed_vs_external_tables.ipynb ├── 10_overview_of_file_formats.ipynb ├── 11_dropping_tables_and_databases.ipynb ├── 12_truncating_tables.ipynb └── 13_exercises_managing_tables.ipynb ├── Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning ├── 01_dml_and_partitioning.ipynb ├── 02_introduction_to_partitioning.ipynb ├── 03_creating_tables_using_parquet.ipynb ├── 04_load_vs_insert.ipynb ├── 05_inserting_data_using_stage_table.ipynb ├── 06_creating_partitioned_tables.ipynb ├── 07_adding_partitions_to_tables.ipynb ├── 08_loading_data_into_partitions.ipynb ├── 09_inserting_data_into_partitions.ipynb ├── 10_using_dynamic_partition_mode.ipynb └── 11_exercises_partitioned_tables.ipynb ├── Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions ├── 01_predefined_functions.ipynb ├── 02_overview_of_functions.ipynb ├── 03_validating_functions.ipynb ├── 04_string_manipulation_functions.ipynb ├── 05_date_manipulation_functions.ipynb ├── 06_overview_of_numeric_functions.ipynb ├── 07_data_type_conversion.ipynb ├── 08_handling_null_values.ipynb ├── 09_using_case_and_when.ipynb └── 10_query_example_word_count.ipynb ├── Section 47 - Data Engineering using Spark SQL - Windowing Functions ├── 01_windowing_functions.ipynb ├── 02_prepare_database.ipynb ├── 03_overview_of_windowing_functions.ipynb ├── 04_aggregations_using_windowing_functions.ipynb ├── 05_using_lead_or_lag.ipynb ├── 06_getting_first_and_last_values.ipynb ├── 07_ranking_using_windowing_functions.ipynb ├── 08_order_of_execution_of_sql.ipynb ├── 09_overview_of_sub_queries.ipynb └── 10_filtering_window_function_results.ipynb ├── Section 48 - Apache Spark using Python - Data Processing Overview ├── 03_starting_spark_context.ipynb ├── 04_overview_of_spark_read_apis.ipynb ├── 05_understand_airlines_data.ipynb ├── 06_inferring_schema.ipynb ├── 07_previewing_airlines_data.ipynb ├── 08_overview_of_dataframe_apis.ipynb ├── 09_overview_of_functions.ipynb └── 10_overview_of_spark_write_apis.ipynb ├── Section 49 - Apache Spark using Python - Processing Column Data ├── 02_predefined_functions.ipynb ├── 03_create_dummy_dataframes.ipynb ├── 04_categories_of_functions.ipynb ├── 05_special_functions_col_and_lit.ipynb ├── 06_common_string_manipulation_functions.ipynb ├── 07_extracting_strings_using_substring.ipynb ├── 08_extracting_strings_using_split.ipynb ├── 09_padding_characters_around_strings.ipynb ├── 10_trimming_characters_from_strings.ipynb ├── 11_date_and_time_manipulation_functions.ipynb ├── 12_date_and_time_arithmetic.ipynb ├── 13_using_date_and_time_trunc_functions.ipynb ├── 14_date_and_time_extract_functions.ipynb ├── 15_using_to_date_and_to_timestamp.ipynb ├── 16_using_date_format_function.ipynb ├── 17_dealing_with_unix_timestamp.ipynb ├── 18_dealing_with_nulls.ipynb └── 19_using_case_and_when.ipynb ├── Section 50 - Apache Spark using Python - Basic Transformations ├── 02_overview_of_basic_transformations.ipynb ├── 03_data_frame_for_basic_transformations.ipynb ├── 04_basic_filtering_of_data.ipynb ├── 05_filtering_example_using_dates.ipynb ├── 06_boolean_operators.ipynb ├── 07_using_in_operator_or_isin_function.ipynb ├── 08_using_like_operator_or_function.ipynb ├── 09_using_between_operator.ipynb ├── 10_dealing_with_nulls_while_filtering.ipynb ├── 11_total_aggregations.ipynb ├── 12_aggregate_data_using_groupby.ipynb ├── 13_aggregate_data_using_rollup.ipynb ├── 14_aggregate_data_using_cube.ipynb ├── 15_overview_of_sorting_data_frames.ipynb ├── 16_solution_problem_1.ipynb └── 17_solution_problem_2.ipynb ├── Section 51 - Apache Spark using Python - Joining Data Sets ├── 02_preparing_data_sets_for_joins.ipynb ├── 03_analyze_data_sets_for_joins.ipynb ├── 04_problem_statements_for_joins.ipynb ├── 05_overview_of_joins.ipynb ├── 06_using_inner_joins.ipynb ├── 07_left_or_right_outer_join.ipynb ├── 08_solutions_problem_1.ipynb ├── 09_solutions_problem_2.ipynb ├── 10_solutions_problem_3.ipynb ├── 11_solutions_problem_4.ipynb ├── 12_solutions_problem_5.ipynb ├── 13_solutions_problem_6.ipynb ├── 14_solutions_problem_7.ipynb └── 15_solutions_problem_8.ipynb ├── Section 52 - Apache Spark using Python - Spark Metastore ├── 02_overview_of_spark_metastore.ipynb ├── 03_exploring_spark_catalog.ipynb ├── 04_creating_metastore_tables_using_catalog.ipynb ├── 05_inferring_schema_for_tables.ipynb ├── 06_define_schema_for_tables_using_structtype.ipynb ├── 07_inserting_into_existing_tables.ipynb ├── 08_read_and_process_data_from_metastore_tables.ipynb ├── 09_creating_partitioned_tables.ipynb ├── 10_saving_as_partitioned_tables.ipynb ├── 11_creating_temp_views.ipynb └── 12_using_spark_sql.ipynb ├── Section 53 - Apache Spark - Development Life Cycle using Python ├── Getting Started ├── Process data using Spark APIs ├── Productionize Code ├── Read data from files ├── Write data to files └── set up a virtual environment └── Section 54 - Spark Application Execution Life Cycle and Spark UI ├── Develop Shell Script to run Spark Application ├── Develop Word Count Application └── Setup Data Set for Word Count Application /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/README.md -------------------------------------------------------------------------------- /Section 01 - Introduction about the Course/Free Download - Course Material: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 01 - Introduction about the Course/Free Download - Course Material -------------------------------------------------------------------------------- /Section 02 - Setting up Environment Using AWS Cloud9/[Commands] Setup Jupyter Lab on Cloud9: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 02 - Setting up Environment Using AWS Cloud9/[Commands] Setup Jupyter Lab on Cloud9 -------------------------------------------------------------------------------- /Section 03 - Setting up Environment - Overview of GCP and Provision Ubuntu VM/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 03 - Setting up Environment - Overview of GCP and Provision Ubuntu VM/Links/Links -------------------------------------------------------------------------------- /Section 05 - Setting up Environment - Setup Postgres Database on Ubuntu VM/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 05 - Setting up Environment - Setup Postgres Database on Ubuntu VM/Links/Links -------------------------------------------------------------------------------- /Section 05 - Setting up Environment - Setup Postgres Database on Ubuntu VM/Scripts/create_db_tables_pg.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 05 - Setting up Environment - Setup Postgres Database on Ubuntu VM/Scripts/create_db_tables_pg.sql -------------------------------------------------------------------------------- /Section 05 - Setting up Environment - Setup Postgres Database on Ubuntu VM/Scripts/load_db_tables_pg.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 05 - Setting up Environment - Setup Postgres Database on Ubuntu VM/Scripts/load_db_tables_pg.sql -------------------------------------------------------------------------------- /Section 05 - Setting up Environment - Setup Postgres Database on Ubuntu VM/Setup Postgres Database/02_docker_cheat_sheet.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 05 - Setting up Environment - Setup Postgres Database on Ubuntu VM/Setup Postgres Database/02_docker_cheat_sheet.ipynb -------------------------------------------------------------------------------- /Section 05 - Setting up Environment - Setup Postgres Database on Ubuntu VM/Setup Postgres Database/06_execute_sql_scripts.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 05 - Setting up Environment - Setup Postgres Database on Ubuntu VM/Setup Postgres Database/06_execute_sql_scripts.ipynb -------------------------------------------------------------------------------- /Section 06 - Database Essentials - Getting Started/Getting Started/02_connecting_to_database.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 06 - Database Essentials - Getting Started/Getting Started/02_connecting_to_database.ipynb -------------------------------------------------------------------------------- /Section 06 - Database Essentials - Getting Started/Getting Started/03_using_psql.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 06 - Database Essentials - Getting Started/Getting Started/03_using_psql.ipynb -------------------------------------------------------------------------------- /Section 06 - Database Essentials - Getting Started/Getting Started/04_setup_postgres_using_docker.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 06 - Database Essentials - Getting Started/Getting Started/04_setup_postgres_using_docker.ipynb -------------------------------------------------------------------------------- /Section 06 - Database Essentials - Getting Started/Getting Started/05_setup_sql_workbench.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 06 - Database Essentials - Getting Started/Getting Started/05_setup_sql_workbench.ipynb -------------------------------------------------------------------------------- /Section 06 - Database Essentials - Getting Started/Getting Started/06_sql_workbench_and_postgres.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 06 - Database Essentials - Getting Started/Getting Started/06_sql_workbench_and_postgres.ipynb -------------------------------------------------------------------------------- /Section 06 - Database Essentials - Getting Started/Getting Started/07_sql_workbench_features.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 06 - Database Essentials - Getting Started/Getting Started/07_sql_workbench_features.ipynb -------------------------------------------------------------------------------- /Section 06 - Database Essentials - Getting Started/Getting Started/08_data_loading_utilities.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 06 - Database Essentials - Getting Started/Getting Started/08_data_loading_utilities.ipynb -------------------------------------------------------------------------------- /Section 06 - Database Essentials - Getting Started/Getting Started/09_loading_data_postgres_in_docker.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 06 - Database Essentials - Getting Started/Getting Started/09_loading_data_postgres_in_docker.ipynb -------------------------------------------------------------------------------- /Section 06 - Database Essentials - Getting Started/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 06 - Database Essentials - Getting Started/Links/Links -------------------------------------------------------------------------------- /Section 06 - Database Essentials - Getting Started/Script/01 - Create Table.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 06 - Database Essentials - Getting Started/Script/01 - Create Table.sql -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/04_overview_of_database_operations.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/04_overview_of_database_operations.ipynb -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/05_crud_operations.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/05_crud_operations.ipynb -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/06_creating_table.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/06_creating_table.ipynb -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/07_inserting_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/07_inserting_data.ipynb -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/08_updating_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/08_updating_data.ipynb -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/09_deleting_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/09_deleting_data.ipynb -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/10_overview_of_transactions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/10_overview_of_transactions.ipynb -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/11_exercises_database_operations.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/DML or CRUD Operations/11_exercises_database_operations.ipynb -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Exercises/01 - Create Table.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/Exercises/01 - Create Table.sql -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Exercises/02 - Insert Into.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/Exercises/02 - Insert Into.sql -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Exercises/03 - Updating Data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/Exercises/03 - Updating Data.sql -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Exercises/04 - Deleting Data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/Exercises/04 - Deleting Data.sql -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Exercises/05 - Selecting.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/Exercises/05 - Selecting.sql -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/Links/Links -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Scripts/01 - Information Schema Tables.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/Scripts/01 - Information Schema Tables.sql -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Scripts/02 - Information Schema Columns.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/Scripts/02 - Information Schema Columns.sql -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Scripts/03 - Users.sql: -------------------------------------------------------------------------------- 1 | SELECT 2 | * 3 | FROM 4 | users; -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Scripts/04 - Truncate Table.sql: -------------------------------------------------------------------------------- 1 | TRUNCATE TABLE users; -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Scripts/05 - Insert Into.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/Scripts/05 - Insert Into.sql -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Scripts/06 - Update.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/Scripts/06 - Update.sql -------------------------------------------------------------------------------- /Section 07 - Database Essentials - Database Operations/Scripts/07 - Delete.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 07 - Database Essentials - Database Operations/Scripts/07 - Delete.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Exercises/01 - Customer Order Count.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Exercises/01 - Customer Order Count.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Exercises/02 - Dormant Customers.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Exercises/02 - Dormant Customers.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Exercises/03 - Revenue Per Customer.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Exercises/03 - Revenue Per Customer.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Exercises/04 - Revenue Per Category.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Exercises/04 - Revenue Per Category.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Exercises/05 - Product Count Per Department.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Exercises/05 - Product Count Per Department.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Links/Links -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/01 - Selecting.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/01 - Selecting.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/02 - Distinct.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/02 - Distinct.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/03 - Filtering.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/03 - Filtering.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/04 - Inner Join.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/04 - Inner Join.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/05 - Outher Join.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/05 - Outher Join.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/06 - Aggregations.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/06 - Aggregations.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/07 - Sorting Data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/07 - Sorting Data.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/08 - Daily.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Scripts/08 - Daily.sql -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/02_standard_transformations.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/02_standard_transformations.ipynb -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/03_overview_of_data_model.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/03_overview_of_data_model.ipynb -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/04_define_problem_statement.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/04_define_problem_statement.ipynb -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/05_preparing_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/05_preparing_tables.ipynb -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/06_selecting_or_projecting_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/06_selecting_or_projecting_data.ipynb -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/07_filtering_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/07_filtering_data.ipynb -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/08_joining_tables_inner.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/08_joining_tables_inner.ipynb -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/09_joining_tables_outer.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/09_joining_tables_outer.ipynb -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/10_performing_aggregations.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/10_performing_aggregations.ipynb -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/11_sorting_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/11_sorting_data.ipynb -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/12_solution_daily_product_revenue.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/12_solution_daily_product_revenue.ipynb -------------------------------------------------------------------------------- /Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/13_exercises_basic_sql_queries.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 08 - Database Essentials - Writing Basic SQL Queries/Writing Basic SQL Queries/13_exercises_basic_sql_queries.ipynb -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/02_data_definition_language.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/02_data_definition_language.ipynb -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/03_overview_of_data_types.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/03_overview_of_data_types.ipynb -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/04_adding_or_modifying_columns.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/04_adding_or_modifying_columns.ipynb -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/06_managing_constraints.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/06_managing_constraints.ipynb -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/07_indexes_on_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/07_indexes_on_tables.ipynb -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/08_indexes_for_constraints.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/08_indexes_for_constraints.ipynb -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/09_overview_of_sequences.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/09_overview_of_sequences.ipynb -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/10_truncating_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/10_truncating_tables.ipynb -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/11_dropping_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Creating Tables and Indexes/11_dropping_tables.ipynb -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Links/Links -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/01 - DDL.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/01 - DDL.sql -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/02 - DT.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/02 - DT.sql -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/03 - Adding or Modifying Columns.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/03 - Adding or Modifying Columns.sql -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/04 - Managing Constraints.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/04 - Managing Constraints.sql -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/05 - Indexes for Constraints.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/05 - Indexes for Constraints.sql -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/06 - Overview of Sequences.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/06 - Overview of Sequences.sql -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/07 - Truncating Tables.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/07 - Truncating Tables.sql -------------------------------------------------------------------------------- /Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/08 - Dropping Tables.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 09 - Database Essentials - Creating Tables and Indexes/Scripts/08 - Dropping Tables.sql -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Exercises/01 - Partitioned tables.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Exercises/01 - Partitioned tables.sql -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Exercises/02 - Load Partitioned Tables.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Exercises/02 - Load Partitioned Tables.sql -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Links/Links -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Partitioning Tables and Indexes/03_list_partitioning.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Partitioning Tables and Indexes/03_list_partitioning.ipynb -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Partitioning Tables and Indexes/05_manipulating_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Partitioning Tables and Indexes/05_manipulating_data.ipynb -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Partitioning Tables and Indexes/06_range_partitioning.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Partitioning Tables and Indexes/06_range_partitioning.ipynb -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Partitioning Tables and Indexes/09_hash_partitioning.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Partitioning Tables and Indexes/09_hash_partitioning.ipynb -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Partitioning Tables and Indexes/11_usage_scenarios.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Partitioning Tables and Indexes/11_usage_scenarios.ipynb -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Partitioning Tables and Indexes/12_sub_partitioning.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Partitioning Tables and Indexes/12_sub_partitioning.ipynb -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/01 - List Partitioning.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/01 - List Partitioning.sql -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/02 - Managing Partitions - List.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/02 - Managing Partitions - List.sql -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/03 - Manipulating Data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/03 - Manipulating Data.sql -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/04 - Range Partitioning.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/04 - Range Partitioning.sql -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/05 - Managing Partitions - Range.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/05 - Managing Partitions - Range.sql -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/06 - Repartitioning - Range.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/06 - Repartitioning - Range.sql -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/07 - Hash Partitioning.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/07 - Hash Partitioning.sql -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/08 - Managing Partitions - Hash.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/08 - Managing Partitions - Hash.sql -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/09 - Usage Scenarios.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/09 - Usage Scenarios.sql -------------------------------------------------------------------------------- /Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/10 - Sub Partitioning.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 10 - Database Essentials - Partitioning Tables and Indexes/Scripts/10 - Sub Partitioning.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Links/Links -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Predefined Functions/02_overview_of_predefined_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Predefined Functions/02_overview_of_predefined_functions.ipynb -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Predefined Functions/03_string_manipulation_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Predefined Functions/03_string_manipulation_functions.ipynb -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Predefined Functions/04_date_manipulation_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Predefined Functions/04_date_manipulation_functions.ipynb -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Predefined Functions/05_overview_of_numeric_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Predefined Functions/05_overview_of_numeric_functions.ipynb -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Predefined Functions/06_data_type_conversion.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Predefined Functions/06_data_type_conversion.ipynb -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Predefined Functions/07_handling_null_values.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Predefined Functions/07_handling_null_values.ipynb -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Predefined Functions/08_using_case_and_when.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Predefined Functions/08_using_case_and_when.ipynb -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/01 - Overview of Pre-Defined Functions.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/01 - Overview of Pre-Defined Functions.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/02 - Case Conversion and Length.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/02 - Case Conversion and Length.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/03 - Extracting Data - substr and split_part.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/03 - Extracting Data - substr and split_part.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/04 - Using position or strpos.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/04 - Using position or strpos.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/05 - Trimming and Padding Functions.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/05 - Trimming and Padding Functions.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/06 - Reverse and Concatenating multiple strings.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/06 - Reverse and Concatenating multiple strings.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/07 - String Replacement.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/07 - String Replacement.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/08 - Getting Current Date and Timestamp.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/08 - Getting Current Date and Timestamp.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/09 - Date Arithmetic.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/09 - Date Arithmetic.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/10 - Beginning Date or Time - date_trunc.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/10 - Beginning Date or Time - date_trunc.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/11 - Using to_char and to_date.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/11 - Using to_char and to_date.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/12 - Extracting information - extract.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/12 - Extracting information - extract.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/13 - Dealing with Unix Timestamp.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/13 - Dealing with Unix Timestamp.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/14 - Overview of Numeric Functions.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/14 - Overview of Numeric Functions.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/15 - Data Type Conversion.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/15 - Data Type Conversion.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/16 - Handling NULL Values.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/16 - Handling NULL Values.sql -------------------------------------------------------------------------------- /Section 11 - Database Essentials - Predefined Functions/Scripts/17 - Using CASE and WHEN.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 11 - Database Essentials - Predefined Functions/Scripts/17 - Using CASE and WHEN.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Links/Links -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/01 - Overview of Views.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/01 - Overview of Views.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/02 - Named Queries - Using WITH Clause.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/02 - Named Queries - Using WITH Clause.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/03 - Overview of Sub Queries.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/03 - Overview of Sub Queries.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/04 - CTAS - Create Table as Select.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/04 - CTAS - Create Table as Select.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/05 - Advanced DML Operations.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/05 - Advanced DML Operations.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/06 - Merging or Upserting Data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/06 - Merging or Upserting Data.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/07 - Pivoting Rows into Columns.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/07 - Pivoting Rows into Columns.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/08 - Overview of Analytic Functions.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/08 - Overview of Analytic Functions.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/09 - Analytic Functions – Aggregations.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/09 - Analytic Functions – Aggregations.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/10 - Cumulative or Moving Aggregations.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/10 - Cumulative or Moving Aggregations.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/11 - Analytic Functions – Windowing.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/11 - Analytic Functions – Windowing.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/12 - Analytic Functions – Ranking.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/12 - Analytic Functions – Ranking.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/13 - Analytic Functions - Filtering.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/13 - Analytic Functions - Filtering.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/14 - Ranking and Filtering - Recap.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Scripts/14 - Ranking and Filtering - Recap.sql -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Writing Advanced SQL Queries/02_overview_of_views.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Writing Advanced SQL Queries/02_overview_of_views.ipynb -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Writing Advanced SQL Queries/04_overview_of_sub_queries.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Writing Advanced SQL Queries/04_overview_of_sub_queries.ipynb -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Writing Advanced SQL Queries/05_create_table_as_select.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Writing Advanced SQL Queries/05_create_table_as_select.ipynb -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Writing Advanced SQL Queries/06_advanced_dml_operations.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Writing Advanced SQL Queries/06_advanced_dml_operations.ipynb -------------------------------------------------------------------------------- /Section 12 - Database Essentials - Writing Advanced SQL Queries/Writing Advanced SQL Queries/07_merging_or_upserting_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 12 - Database Essentials - Writing Advanced SQL Queries/Writing Advanced SQL Queries/07_merging_or_upserting_data.ipynb -------------------------------------------------------------------------------- /Section 13 - Programming Essentials Using Python - Perform Database Operations/Exercises/01 - Create Table.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 13 - Programming Essentials Using Python - Perform Database Operations/Exercises/01 - Create Table.sql -------------------------------------------------------------------------------- /Section 13 - Programming Essentials Using Python - Perform Database Operations/Exercises/02 - Insert Into.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 13 - Programming Essentials Using Python - Perform Database Operations/Exercises/02 - Insert Into.sql -------------------------------------------------------------------------------- /Section 13 - Programming Essentials Using Python - Perform Database Operations/Exercises/03 - Updating Data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 13 - Programming Essentials Using Python - Perform Database Operations/Exercises/03 - Updating Data.sql -------------------------------------------------------------------------------- /Section 13 - Programming Essentials Using Python - Perform Database Operations/Exercises/04 - Deleting Data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 13 - Programming Essentials Using Python - Perform Database Operations/Exercises/04 - Deleting Data.sql -------------------------------------------------------------------------------- /Section 13 - Programming Essentials Using Python - Perform Database Operations/Exercises/05 - Selecting.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 13 - Programming Essentials Using Python - Perform Database Operations/Exercises/05 - Selecting.sql -------------------------------------------------------------------------------- /Section 13 - Programming Essentials Using Python - Perform Database Operations/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 13 - Programming Essentials Using Python - Perform Database Operations/Links/Links -------------------------------------------------------------------------------- /Section 13 - Programming Essentials Using Python - Perform Database Operations/Postgres Database Operations/05_dml.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 13 - Programming Essentials Using Python - Perform Database Operations/Postgres Database Operations/05_dml.ipynb -------------------------------------------------------------------------------- /Section 13 - Programming Essentials Using Python - Perform Database Operations/Postgres Database Operations/08_tcl.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 13 - Programming Essentials Using Python - Perform Database Operations/Postgres Database Operations/08_tcl.ipynb -------------------------------------------------------------------------------- /Section 13 - Programming Essentials Using Python - Perform Database Operations/Scripts/01 - Create Database and Users Table.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 13 - Programming Essentials Using Python - Perform Database Operations/Scripts/01 - Create Database and Users Table.sql -------------------------------------------------------------------------------- /Section 13 - Programming Essentials Using Python - Perform Database Operations/Scripts/02 - DDL – Data Definition Language.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 13 - Programming Essentials Using Python - Perform Database Operations/Scripts/02 - DDL – Data Definition Language.sql -------------------------------------------------------------------------------- /Section 13 - Programming Essentials Using Python - Perform Database Operations/Scripts/03 - Dml Data Manipulation Language.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 13 - Programming Essentials Using Python - Perform Database Operations/Scripts/03 - Dml Data Manipulation Language.sql -------------------------------------------------------------------------------- /Section 13 - Programming Essentials Using Python - Perform Database Operations/Scripts/04 - DQL – Data Query Language.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 13 - Programming Essentials Using Python - Perform Database Operations/Scripts/04 - DQL – Data Query Language.sql -------------------------------------------------------------------------------- /Section 14 - Programming Essentials Using Python - Getting Started with Python/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 14 - Programming Essentials Using Python - Getting Started with Python/Links -------------------------------------------------------------------------------- /Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Links/Links -------------------------------------------------------------------------------- /Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/02_getting_help.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/02_getting_help.ipynb -------------------------------------------------------------------------------- /Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/03_variables_and_objects.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/03_variables_and_objects.ipynb -------------------------------------------------------------------------------- /Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/04_data_types_commonly_used.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/04_data_types_commonly_used.ipynb -------------------------------------------------------------------------------- /Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/05_operators_in_python.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/05_operators_in_python.ipynb -------------------------------------------------------------------------------- /Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/07_conditionals.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/07_conditionals.ipynb -------------------------------------------------------------------------------- /Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/08_all_about_for_loops.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/08_all_about_for_loops.ipynb -------------------------------------------------------------------------------- /Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/09_running_os_commands.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/09_running_os_commands.ipynb -------------------------------------------------------------------------------- /Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/11 - Eval and Exec.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 15 - Programming Essentials Using Python - Basic Programming Constructs/Python Code/11 - Eval and Exec.ipynb -------------------------------------------------------------------------------- /Section 16 - Programming Essentials Using Python - Predefined Functions/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 16 - Programming Essentials Using Python - Predefined Functions/Links/Links -------------------------------------------------------------------------------- /Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/01 - Pre-defined Functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/01 - Pre-defined Functions.ipynb -------------------------------------------------------------------------------- /Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/02_overview_of_pre-defined_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/02_overview_of_pre-defined_functions.ipynb -------------------------------------------------------------------------------- /Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/03_numeric_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/03_numeric_functions.ipynb -------------------------------------------------------------------------------- /Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/04_overview_of_strings.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/04_overview_of_strings.ipynb -------------------------------------------------------------------------------- /Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/05_string_manipulation_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/05_string_manipulation_functions.ipynb -------------------------------------------------------------------------------- /Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/06_formatting_strings.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/06_formatting_strings.ipynb -------------------------------------------------------------------------------- /Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/07_print_and_input_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/07_print_and_input_functions.ipynb -------------------------------------------------------------------------------- /Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/08_date_manipulation_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/08_date_manipulation_functions.ipynb -------------------------------------------------------------------------------- /Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/09_special_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/09_special_functions.ipynb -------------------------------------------------------------------------------- /Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/10_exercises_pre_defined_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 16 - Programming Essentials Using Python - Predefined Functions/Python Code/10_exercises_pre_defined_functions.ipynb -------------------------------------------------------------------------------- /Section 17 - Programming Essentials Using Python - User Defined Functions/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 17 - Programming Essentials Using Python - User Defined Functions/Links/Links -------------------------------------------------------------------------------- /Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/01_user_defined_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/01_user_defined_functions.ipynb -------------------------------------------------------------------------------- /Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/02_defining_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/02_defining_functions.ipynb -------------------------------------------------------------------------------- /Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/03_doc_strings.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/03_doc_strings.ipynb -------------------------------------------------------------------------------- /Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/04_returning_values.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/04_returning_values.ipynb -------------------------------------------------------------------------------- /Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/06_varying_arguments.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/06_varying_arguments.ipynb -------------------------------------------------------------------------------- /Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/07_keyword_arguments.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/07_keyword_arguments.ipynb -------------------------------------------------------------------------------- /Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/08_recap_of_user_defined_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/08_recap_of_user_defined_functions.ipynb -------------------------------------------------------------------------------- /Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/09_passing_functions_as_arguments.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/09_passing_functions_as_arguments.ipynb -------------------------------------------------------------------------------- /Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/10_lambda_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/10_lambda_functions.ipynb -------------------------------------------------------------------------------- /Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/11_usage_of_lambda_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/11_usage_of_lambda_functions.ipynb -------------------------------------------------------------------------------- /Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/12_exercise_user_defined_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 17 - Programming Essentials Using Python - User Defined Functions/Python Code/12_exercise_user_defined_functions.ipynb -------------------------------------------------------------------------------- /Section 18 - Programming Essentials Using Python - Overview of Collections - List and Set/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 18 - Programming Essentials Using Python - Overview of Collections - List and Set/Links/Links -------------------------------------------------------------------------------- /Section 18 - Programming Essentials Using Python - Overview of Collections - List and Set/Python Code/02_overview.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 18 - Programming Essentials Using Python - Overview of Collections - List and Set/Python Code/02_overview.ipynb -------------------------------------------------------------------------------- /Section 18 - Programming Essentials Using Python - Overview of Collections - List and Set/Python Code/05_adding_elements.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 18 - Programming Essentials Using Python - Overview of Collections - List and Set/Python Code/05_adding_elements.ipynb -------------------------------------------------------------------------------- /Section 18 - Programming Essentials Using Python - Overview of Collections - List and Set/Python Code/10_validating_set.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 18 - Programming Essentials Using Python - Overview of Collections - List and Set/Python Code/10_validating_set.ipynb -------------------------------------------------------------------------------- /Section 19 - Programming Essentials Using Python - Overview of Collections - Dict and Tuple/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 19 - Programming Essentials Using Python - Overview of Collections - Dict and Tuple/Links/Links -------------------------------------------------------------------------------- /Section 19 - Programming Essentials Using Python - Overview of Collections - Dict and Tuple/Python Code/08_list_of_tuples.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 19 - Programming Essentials Using Python - Overview of Collections - Dict and Tuple/Python Code/08_list_of_tuples.ipynb -------------------------------------------------------------------------------- /Section 19 - Programming Essentials Using Python - Overview of Collections - Dict and Tuple/Python Code/09_list_of_dicts.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 19 - Programming Essentials Using Python - Overview of Collections - Dict and Tuple/Python Code/09_list_of_dicts.ipynb -------------------------------------------------------------------------------- /Section 19 - Programming Essentials Using Python - Overview of Collections - Dict and Tuple/Python Code/13_exercises.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 19 - Programming Essentials Using Python - Overview of Collections - Dict and Tuple/Python Code/13_exercises.ipynb -------------------------------------------------------------------------------- /Section 20 - Programming Essentials Using Python - Manipulating Collections using Loops/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 20 - Programming Essentials Using Python - Manipulating Collections using Loops/Links/Links -------------------------------------------------------------------------------- /Section 20 - Programming Essentials Using Python - Manipulating Collections using Loops/Python Code/06_filtering_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 20 - Programming Essentials Using Python - Manipulating Collections using Loops/Python Code/06_filtering_data.ipynb -------------------------------------------------------------------------------- /Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Links/Links -------------------------------------------------------------------------------- /Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/02_develop_myFilter.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/02_develop_myFilter.ipynb -------------------------------------------------------------------------------- /Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/03_validate_myFilter.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/03_validate_myFilter.ipynb -------------------------------------------------------------------------------- /Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/04_develop_myMap.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/04_develop_myMap.ipynb -------------------------------------------------------------------------------- /Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/05_validate_myMap.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/05_validate_myMap.ipynb -------------------------------------------------------------------------------- /Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/06_develop_myReduce.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/06_develop_myReduce.ipynb -------------------------------------------------------------------------------- /Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/07_validate_myReduce.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/07_validate_myReduce.ipynb -------------------------------------------------------------------------------- /Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/08_develop_myReduceByKey.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/08_develop_myReduceByKey.ipynb -------------------------------------------------------------------------------- /Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/10_develop_myJoin.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/10_develop_myJoin.ipynb -------------------------------------------------------------------------------- /Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/11_validate_myJoin.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 21 - Programming Essentials Using Python - Development of Map Reduce APIs/Python Code/11_validate_myJoin.ipynb -------------------------------------------------------------------------------- /Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/01 - Preparing Data Sets.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/01 - Preparing Data Sets.ipynb -------------------------------------------------------------------------------- /Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/02 - Filtering Data using Filter.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/02 - Filtering Data using Filter.ipynb -------------------------------------------------------------------------------- /Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/03 - Projecting Data using Map.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/03 - Projecting Data using Map.ipynb -------------------------------------------------------------------------------- /Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/05 - Aggregations using Reduce.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/05 - Aggregations using Reduce.ipynb -------------------------------------------------------------------------------- /Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/08 - Get Total Commission Amount.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/08 - Get Total Commission Amount.ipynb -------------------------------------------------------------------------------- /Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/09 - Overview of Itertools.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/09 - Overview of Itertools.ipynb -------------------------------------------------------------------------------- /Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/11 - Using itertools starmap.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 22 - Programming Essentials Using Python - Understanding Map Reduce Libraries/11 - Using itertools starmap.ipynb -------------------------------------------------------------------------------- /Section 23 - Programming Essentials Using Python - Basics of File IO using Python/01 - Overview of File IO.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 23 - Programming Essentials Using Python - Basics of File IO using Python/01 - Overview of File IO.ipynb -------------------------------------------------------------------------------- /Section 23 - Programming Essentials Using Python - Basics of File IO using Python/02 - Folders and Files.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 23 - Programming Essentials Using Python - Basics of File IO using Python/02 - Folders and Files.ipynb -------------------------------------------------------------------------------- /Section 23 - Programming Essentials Using Python - Basics of File IO using Python/03 - File Paths and Names.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 23 - Programming Essentials Using Python - Basics of File IO using Python/03 - File Paths and Names.ipynb -------------------------------------------------------------------------------- /Section 23 - Programming Essentials Using Python - Basics of File IO using Python/04 - Ovewrview of Retail Data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 23 - Programming Essentials Using Python - Basics of File IO using Python/04 - Ovewrview of Retail Data.ipynb -------------------------------------------------------------------------------- /Section 23 - Programming Essentials Using Python - Basics of File IO using Python/05 - Read text file into string.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 23 - Programming Essentials Using Python - Basics of File IO using Python/05 - Read text file into string.ipynb -------------------------------------------------------------------------------- /Section 23 - Programming Essentials Using Python - Basics of File IO using Python/06 - Write string to text file.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 23 - Programming Essentials Using Python - Basics of File IO using Python/06 - Write string to text file.ipynb -------------------------------------------------------------------------------- /Section 23 - Programming Essentials Using Python - Basics of File IO using Python/08 - Overview of Delimited Strings.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 23 - Programming Essentials Using Python - Basics of File IO using Python/08 - Overview of Delimited Strings.ipynb -------------------------------------------------------------------------------- /Section 23 - Programming Essentials Using Python - Basics of File IO using Python/09 - Read csv into list of strings.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 23 - Programming Essentials Using Python - Basics of File IO using Python/09 - Read csv into list of strings.ipynb -------------------------------------------------------------------------------- /Section 23 - Programming Essentials Using Python - Basics of File IO using Python/10 - Basics of File IO using Python.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 23 - Programming Essentials Using Python - Basics of File IO using Python/10 - Basics of File IO using Python.ipynb -------------------------------------------------------------------------------- /Section 24 - Programming Essentials Using Python - Delimited Files and Collections/02 - Recap of Basic File IO.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 24 - Programming Essentials Using Python - Delimited Files and Collections/02 - Recap of Basic File IO.ipynb -------------------------------------------------------------------------------- /Section 24 - Programming Essentials Using Python - Delimited Files and Collections/05 - Overview of CSV Module.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 24 - Programming Essentials Using Python - Delimited Files and Collections/05 - Overview of CSV Module.ipynb -------------------------------------------------------------------------------- /Section 24 - Programming Essentials Using Python - Delimited Files and Collections/06 - Read Delimited data using CSV.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 24 - Programming Essentials Using Python - Delimited Files and Collections/06 - Read Delimited data using CSV.ipynb -------------------------------------------------------------------------------- /Section 24 - Programming Essentials Using Python - Delimited Files and Collections/07 - Delimited File and Collections.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 24 - Programming Essentials Using Python - Delimited Files and Collections/07 - Delimited File and Collections.ipynb -------------------------------------------------------------------------------- /Section 24 - Programming Essentials Using Python - Delimited Files and Collections/09 - Advantages of using CSV.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 24 - Programming Essentials Using Python - Delimited Files and Collections/09 - Advantages of using CSV.ipynb -------------------------------------------------------------------------------- /Section 25 - Programming Essentials Using Python - Overview of Pandas Libraries/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 25 - Programming Essentials Using Python - Overview of Pandas Libraries/Links/Links -------------------------------------------------------------------------------- /Section 25 - Programming Essentials Using Python - Overview of Pandas Libraries/Python Code/03_overview_of_series.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 25 - Programming Essentials Using Python - Overview of Pandas Libraries/Python Code/03_overview_of_series.ipynb -------------------------------------------------------------------------------- /Section 25 - Programming Essentials Using Python - Overview of Pandas Libraries/Python Code/11_joining_data_frames.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 25 - Programming Essentials Using Python - Overview of Pandas Libraries/Python Code/11_joining_data_frames.ipynb -------------------------------------------------------------------------------- /Section 26 - Programming Essentials Using Python - Database Programming CRUD Operations/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 26 - Programming Essentials Using Python - Database Programming CRUD Operations/Links/Links -------------------------------------------------------------------------------- /Section 26 - Programming Essentials Using Python - Database Programming CRUD Operations/Postgre Code/01 - Creating Table.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 26 - Programming Essentials Using Python - Database Programming CRUD Operations/Postgre Code/01 - Creating Table.sql -------------------------------------------------------------------------------- /Section 26 - Programming Essentials Using Python - Database Programming CRUD Operations/Python Code/11_crud.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 26 - Programming Essentials Using Python - Database Programming CRUD Operations/Python Code/11_crud.ipynb -------------------------------------------------------------------------------- /Section 27 - Programming Essentials Using Python - Database Programming Bath Operation/Links/Links: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 27 - Programming Essentials Using Python - Database Programming Bath Operation/Links/Links -------------------------------------------------------------------------------- /Section 27 - Programming Essentials Using Python - Database Programming Bath Operation/Postgre Code/01 - Insert.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 27 - Programming Essentials Using Python - Database Programming Bath Operation/Postgre Code/01 - Insert.sql -------------------------------------------------------------------------------- /Section 27 - Programming Essentials Using Python - Database Programming Bath Operation/Postgre Code/02 - Select.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 27 - Programming Essentials Using Python - Database Programming Bath Operation/Postgre Code/02 - Select.sql -------------------------------------------------------------------------------- /Section 27 - Programming Essentials Using Python - Database Programming Bath Operation/Python Code/04_recap_of_insert.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 27 - Programming Essentials Using Python - Database Programming Bath Operation/Python Code/04_recap_of_insert.ipynb -------------------------------------------------------------------------------- /Section 27 - Programming Essentials Using Python - Database Programming Bath Operation/Python Code/11_crud.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 27 - Programming Essentials Using Python - Database Programming Bath Operation/Python Code/11_crud.ipynb -------------------------------------------------------------------------------- /Section 28 - Programming Essentials Using Python - Processing JSON Data/01 - Overview of JSON.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 28 - Programming Essentials Using Python - Processing JSON Data/01 - Overview of JSON.ipynb -------------------------------------------------------------------------------- /Section 28 - Programming Essentials Using Python - Processing JSON Data/02 - JSON Data Types.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 28 - Programming Essentials Using Python - Processing JSON Data/02 - JSON Data Types.ipynb -------------------------------------------------------------------------------- /Section 28 - Programming Essentials Using Python - Processing JSON Data/03 - Create JSON String.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 28 - Programming Essentials Using Python - Processing JSON Data/03 - Create JSON String.ipynb -------------------------------------------------------------------------------- /Section 28 - Programming Essentials Using Python - Processing JSON Data/04 - Process JSON String.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 28 - Programming Essentials Using Python - Processing JSON Data/04 - Process JSON String.ipynb -------------------------------------------------------------------------------- /Section 28 - Programming Essentials Using Python - Processing JSON Data/05 - Single JSON in Files.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 28 - Programming Essentials Using Python - Processing JSON Data/05 - Single JSON in Files.ipynb -------------------------------------------------------------------------------- /Section 28 - Programming Essentials Using Python - Processing JSON Data/06 - Multiple JSON Documents in files.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 28 - Programming Essentials Using Python - Processing JSON Data/06 - Multiple JSON Documents in files.ipynb -------------------------------------------------------------------------------- /Section 28 - Programming Essentials Using Python - Processing JSON Data/07 - Process JSON using Pandas.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 28 - Programming Essentials Using Python - Processing JSON Data/07 - Process JSON using Pandas.ipynb -------------------------------------------------------------------------------- /Section 28 - Programming Essentials Using Python - Processing JSON Data/09 - Common Use Cases for JSON.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 28 - Programming Essentials Using Python - Processing JSON Data/09 - Common Use Cases for JSON.ipynb -------------------------------------------------------------------------------- /Section 28 - Programming Essentials Using Python - Processing JSON Data/10 - Processing JSON Data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 28 - Programming Essentials Using Python - Processing JSON Data/10 - Processing JSON Data.ipynb -------------------------------------------------------------------------------- /Section 28 - Programming Essentials Using Python - Processing JSON Data/11 - Write to JSON Files using JSON.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 28 - Programming Essentials Using Python - Processing JSON Data/11 - Write to JSON Files using JSON.ipynb -------------------------------------------------------------------------------- /Section 28 - Programming Essentials Using Python - Processing JSON Data/12 - Write to JSON files using pandas.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 28 - Programming Essentials Using Python - Processing JSON Data/12 - Write to JSON files using pandas.ipynb -------------------------------------------------------------------------------- /Section 29 - Programming Essentials Using Python - Processing REST Payloads/01_Overview_of_REST_APIs.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 29 - Programming Essentials Using Python - Processing REST Payloads/01_Overview_of_REST_APIs.ipynb -------------------------------------------------------------------------------- /Section 29 - Programming Essentials Using Python - Processing REST Payloads/02_Using_curl_command.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 29 - Programming Essentials Using Python - Processing REST Payloads/02_Using_curl_command.ipynb -------------------------------------------------------------------------------- /Section 29 - Programming Essentials Using Python - Processing REST Payloads/03_Overview_of_Postman.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 29 - Programming Essentials Using Python - Processing REST Payloads/03_Overview_of_Postman.ipynb -------------------------------------------------------------------------------- /Section 29 - Programming Essentials Using Python - Processing REST Payloads/04_Getting_Started_with_requests.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 29 - Programming Essentials Using Python - Processing REST Payloads/04_Getting_Started_with_requests.ipynb -------------------------------------------------------------------------------- /Section 29 - Programming Essentials Using Python - Processing REST Payloads/05_Convert_REST_Payload_to_Python_Objects.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 29 - Programming Essentials Using Python - Processing REST Payloads/05_Convert_REST_Payload_to_Python_Objects.ipynb -------------------------------------------------------------------------------- /Section 29 - Programming Essentials Using Python - Processing REST Payloads/07_Process_REST_Payload_using_pandas.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 29 - Programming Essentials Using Python - Processing REST Payloads/07_Process_REST_Payload_using_pandas.ipynb -------------------------------------------------------------------------------- /Section 30 - Understanding Python Virtual Environments/01_Accessing_Python_through_PowerShell.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 30 - Understanding Python Virtual Environments/01_Accessing_Python_through_PowerShell.txt -------------------------------------------------------------------------------- /Section 30 - Understanding Python Virtual Environments/02_Create_Virtual_Environment_for_Web_Application.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 30 - Understanding Python Virtual Environments/02_Create_Virtual_Environment_for_Web_Application.txt -------------------------------------------------------------------------------- /Section 30 - Understanding Python Virtual Environments/03_Reviewing_Dependencies_Virtual_Environment.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 30 - Understanding Python Virtual Environments/03_Reviewing_Dependencies_Virtual_Environment.txt -------------------------------------------------------------------------------- /Section 30 - Understanding Python Virtual Environments/04_Installing_Dependencies_Web_applications.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 30 - Understanding Python Virtual Environments/04_Installing_Dependencies_Web_applications.txt -------------------------------------------------------------------------------- /Section 30 - Understanding Python Virtual Environments/05_Getting_Details_About_PIP.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 30 - Understanding Python Virtual Environments/05_Getting_Details_About_PIP.txt -------------------------------------------------------------------------------- /Section 30 - Understanding Python Virtual Environments/06_Uninstall_Packages_Using_PIP.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 30 - Understanding Python Virtual Environments/06_Uninstall_Packages_Using_PIP.txt -------------------------------------------------------------------------------- /Section 30 - Understanding Python Virtual Environments/07_Cleanup_Virtual_Environment.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 30 - Understanding Python Virtual Environments/07_Cleanup_Virtual_Environment.txt -------------------------------------------------------------------------------- /Section 30 - Understanding Python Virtual Environments/08_Recreate_and_Activate_Virtual_Environment_Web_Application.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 30 - Understanding Python Virtual Environments/08_Recreate_and_Activate_Virtual_Environment_Web_Application.txt -------------------------------------------------------------------------------- /Section 30 - Understanding Python Virtual Environments/09_Define_Requeriments_File_Web_Application.txt: -------------------------------------------------------------------------------- 1 | # Creating a vi file 2 | vi requirements.txt -------------------------------------------------------------------------------- /Section 30 - Understanding Python Virtual Environments/10_Install_Dependencies_Web_Application.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 30 - Understanding Python Virtual Environments/10_Install_Dependencies_Web_Application.txt -------------------------------------------------------------------------------- /Section 30 - Understanding Python Virtual Environments/11_Create_Virtual_Environment_Data_Engineering.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 30 - Understanding Python Virtual Environments/11_Create_Virtual_Environment_Data_Engineering.txt -------------------------------------------------------------------------------- /Section 30 - Understanding Python Virtual Environments/12_Install_Dependencias_data_Engineering_Application.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 30 - Understanding Python Virtual Environments/12_Install_Dependencias_data_Engineering_Application.txt -------------------------------------------------------------------------------- /Section 31 - Overview of Pycharm for Python Application Development/Link/01_Installation_of_Pycharm_Windowns.txt: -------------------------------------------------------------------------------- 1 | # Link 2 | https://www.jetbrains.com/pycharm/download/#section=windows -------------------------------------------------------------------------------- /Section 31 - Overview of Pycharm for Python Application Development/gettingStarted/test.py: -------------------------------------------------------------------------------- 1 | print("Hello World Pycharm!!") 2 | -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/CMD Codes/01_Codes_CMD: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/CMD Codes/01_Codes_CMD -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/CMD Codes/02_Listing_the_images: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/CMD Codes/02_Listing_the_images -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/CMD Codes/03_Checking_all_docker_commands: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/CMD Codes/03_Checking_all_docker_commands -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/Manual/01_Getting_Started.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/Manual/01_Getting_Started.ipynb -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/Manual/02_Problem_Statement_Data_Copier.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/Manual/02_Problem_Statement_Data_Copier.ipynb -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/Manual/03_Setup_Docker.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/Manual/03_Setup_Docker.ipynb -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/Manual/04_Quick_Overview_of_Docker.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/Manual/04_Quick_Overview_of_Docker.ipynb -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/Manual/05_Prepare_Dataset.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/Manual/05_Prepare_Dataset.ipynb -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/Manual/06_Setup_Postgres_Database.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/Manual/06_Setup_Postgres_Database.ipynb -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/Manual/07_Overview_of_Postgres_Database.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/Manual/07_Overview_of_Postgres_Database.ipynb -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/Manual/08_Setup_Project_using_PyCharm.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/Manual/08_Setup_Project_using_PyCharm.ipynb -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/Manual/09_Managing_Dependencies.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/Manual/09_Managing_Dependencies.ipynb -------------------------------------------------------------------------------- /Section 32 - Data Copier - Getting Started/Manual/10_Create_GitHub_Repository.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 32 - Data Copier - Getting Started/Manual/10_Create_GitHub_Repository.ipynb -------------------------------------------------------------------------------- /Section 33 - Data Copier - Reading Data using Pandas/01_Reading_Data_using_Pandas_Introduction.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 33 - Data Copier - Reading Data using Pandas/01_Reading_Data_using_Pandas_Introduction.ipynb -------------------------------------------------------------------------------- /Section 33 - Data Copier - Reading Data using Pandas/02_Overview_of_Retail_Data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 33 - Data Copier - Reading Data using Pandas/02_Overview_of_Retail_Data.ipynb -------------------------------------------------------------------------------- /Section 33 - Data Copier - Reading Data using Pandas/03_Adding_Pandas_to_Project.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 33 - Data Copier - Reading Data using Pandas/03_Adding_Pandas_to_Project.ipynb -------------------------------------------------------------------------------- /Section 33 - Data Copier - Reading Data using Pandas/04_Reading_Data_using_Pandas.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 33 - Data Copier - Reading Data using Pandas/04_Reading_Data_using_Pandas.ipynb -------------------------------------------------------------------------------- /Section 33 - Data Copier - Reading Data using Pandas/05_Previewing_Data_using_Pandas.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 33 - Data Copier - Reading Data using Pandas/05_Previewing_Data_using_Pandas.ipynb -------------------------------------------------------------------------------- /Section 33 - Data Copier - Reading Data using Pandas/06_Reading_Data_in Chunks_using_Pandas.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 33 - Data Copier - Reading Data using Pandas/06_Reading_Data_in Chunks_using_Pandas.ipynb -------------------------------------------------------------------------------- /Section 33 - Data Copier - Reading Data using Pandas/07_Dynamically_Read_Files.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 33 - Data Copier - Reading Data using Pandas/07_Dynamically_Read_Files.ipynb -------------------------------------------------------------------------------- /Section 34 - Data Copier - Database Programming using Pandas/01_Database_Programming_using_Pandas.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 34 - Data Copier - Database Programming using Pandas/01_Database_Programming_using_Pandas.ipynb -------------------------------------------------------------------------------- /Section 34 - Data Copier - Database Programming using Pandas/02_Validate_Postgres_Setup_using_Docker.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 34 - Data Copier - Database Programming using Pandas/02_Validate_Postgres_Setup_using_Docker.ipynb -------------------------------------------------------------------------------- /Section 34 - Data Copier - Database Programming using Pandas/03_Add_Required_Dependencies.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 34 - Data Copier - Database Programming using Pandas/03_Add_Required_Dependencies.ipynb -------------------------------------------------------------------------------- /Section 34 - Data Copier - Database Programming using Pandas/04_Create_Users_Table_in_retail_db_Database.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 34 - Data Copier - Database Programming using Pandas/04_Create_Users_Table_in_retail_db_Database.ipynb -------------------------------------------------------------------------------- /Section 34 - Data Copier - Database Programming using Pandas/05_Populating_Sample_Data_into_Users_Table.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 34 - Data Copier - Database Programming using Pandas/05_Populating_Sample_Data_into_Users_Table.ipynb -------------------------------------------------------------------------------- /Section 34 - Data Copier - Database Programming using Pandas/06_Reading_Data_from_Table_using_Pandas.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 34 - Data Copier - Database Programming using Pandas/06_Reading_Data_from_Table_using_Pandas.ipynb -------------------------------------------------------------------------------- /Section 34 - Data Copier - Database Programming using Pandas/07_Truncate_Users_Table.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 34 - Data Copier - Database Programming using Pandas/07_Truncate_Users_Table.ipynb -------------------------------------------------------------------------------- /Section 34 - Data Copier - Database Programming using Pandas/08_Writing_Pandas_Dataframe_to_Table.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 34 - Data Copier - Database Programming using Pandas/08_Writing_Pandas_Dataframe_to_Table.ipynb -------------------------------------------------------------------------------- /Section 34 - Data Copier - Database Programming using Pandas/09_Validating_Users_Data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 34 - Data Copier - Database Programming using Pandas/09_Validating_Users_Data.ipynb -------------------------------------------------------------------------------- /Section 34 - Data Copier - Database Programming using Pandas/10_Dropping_Users_Table.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 34 - Data Copier - Database Programming using Pandas/10_Dropping_Users_Table.ipynb -------------------------------------------------------------------------------- /Section 35 - Data Copier - Loading Data from Files to Tables/01_Loading_Data_from_Files_to_Tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 35 - Data Copier - Loading Data from Files to Tables/01_Loading_Data_from_Files_to_Tables.ipynb -------------------------------------------------------------------------------- /Section 35 - Data Copier - Loading Data from Files to Tables/02_Populating_Departments_Table.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 35 - Data Copier - Loading Data from Files to Tables/02_Populating_Departments_Table.ipynb -------------------------------------------------------------------------------- /Section 35 - Data Copier - Loading Data from Files to Tables/03_Validate_Departments_Table.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 35 - Data Copier - Loading Data from Files to Tables/03_Validate_Departments_Table.ipynb -------------------------------------------------------------------------------- /Section 35 - Data Copier - Loading Data from Files to Tables/04_Populating_Orders_Table.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 35 - Data Copier - Loading Data from Files to Tables/04_Populating_Orders_Table.ipynb -------------------------------------------------------------------------------- /Section 35 - Data Copier - Loading Data from Files to Tables/05_Validate_Orders_Table_in_Database.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 35 - Data Copier - Loading Data from Files to Tables/05_Validate_Orders_Table_in_Database.ipynb -------------------------------------------------------------------------------- /Section 35 - Data Copier - Loading Data from Files to Tables/06_Validate_Orders_Table_using_Pandas.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 35 - Data Copier - Loading Data from Files to Tables/06_Validate_Orders_Table_using_Pandas.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/01_Modularizing_Data_Copier.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/01_Modularizing_Data_Copier.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/02_Overview_of_main_Function.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/02_Overview_of_main_Function.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/03_Overview_of_Environment_Variables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/03_Overview_of_Environment_Variables.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/04_Using_os_Module_for_Environment_Variables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/04_Using_os_Module_for_Environment_Variables.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/05_Passing_Environment_Variables_using_Pycharm.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/05_Passing_Environment_Variables_using_Pycharm.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/06_Read_Logic_using_Pandas.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/06_Read_Logic_using_Pandas.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/07_Validate_Read_Logic.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/07_Validate_Read_Logic.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/08_Write_Logic_using_Pandas.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/08_Write_Logic_using_Pandas.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/09_Validate_Write_Logic.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/09_Validate_Write_Logic.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/10_Integrate_Read_and_Write_Logic.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/10_Integrate_Read_and_Write_Logic.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/11_Validate_Integration_Logic.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/11_Validate_Integration_Logic.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/12_Develop_Logic_for_ultiple_Tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/12_Develop_Logic_for_ultiple_Tables.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Manual/13_Validate_Logic_for_Multiple_Tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Manual/13_Validate_Logic_for_Multiple_Tables.ipynb -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Python Code/app.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Python Code/app.py -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Python Code/read.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Python Code/read.py -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Python Code/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Python Code/requirements.txt -------------------------------------------------------------------------------- /Section 36 - Data Copier - Modularizing the Application/Python Code/write.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 36 - Data Copier - Modularizing the Application/Python Code/write.py -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Manual/01_Prepare_database_for_validation.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Manual/01_Prepare_database_for_validation.txt -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Manual/02_Pull_and_validate_python_image.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Manual/02_Pull_and_validate_python_image.txt -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Manual/03_Create_and_attach_network_to_database.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Manual/03_Create_and_attach_network_to_database.txt -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Manual/04_Quick_recap_about_docker_conatiners.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Manual/04_Quick_recap_about_docker_conatiners.txt -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Manual/05_Deploying_application_and_installing_docker: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Manual/05_Deploying_application_and_installing_docker -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Manual/06_Copy_source_data_file_into_container: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Manual/06_Copy_source_data_file_into_container -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Manual/07_Add_data_copier_container: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Manual/07_Add_data_copier_container -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Manual/08_Installing_os_libraries_docker: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Manual/08_Installing_os_libraries_docker -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Manual/09_Validate_network_connectivity: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Manual/09_Validate_network_connectivity -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Manual/10_Running_application_docker: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Manual/10_Running_application_docker -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Manual/11_Delete_docker_container: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Manual/11_Delete_docker_container -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Python Code/app.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Python Code/app.py -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Python Code/read.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Python Code/read.py -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Python Code/requirements.txt: -------------------------------------------------------------------------------- 1 | psycopg2-binary==3.0.8 2 | SQLAlchemy==1.4.31 3 | pandas==2.0.0 4 | -------------------------------------------------------------------------------- /Section 37 - Data Copier - Dockerizing the Application/Python Code/write.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 37 - Data Copier - Dockerizing the Application/Python Code/write.py -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Manual/01_Getting_started_with_docker_image.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 38 - Data Copier - Using Custom Docker Image/Manual/01_Getting_started_with_docker_image.txt -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Manual/02_Install_OS_modules.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 38 - Data Copier - Using Custom Docker Image/Manual/02_Install_OS_modules.txt -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Manual/03_Copying_source_code.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 38 - Data Copier - Using Custom Docker Image/Manual/03_Copying_source_code.txt -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Manual/04_Adding_dependencies.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 38 - Data Copier - Using Custom Docker Image/Manual/04_Adding_dependencies.txt -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Manual/05_Unsderstanding_docker.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 38 - Data Copier - Using Custom Docker Image/Manual/05_Unsderstanding_docker.txt -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Manual/06_Mounting_data_folders.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 38 - Data Copier - Using Custom Docker Image/Manual/06_Mounting_data_folders.txt -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Manual/07_Add_data_copier_container.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 38 - Data Copier - Using Custom Docker Image/Manual/07_Add_data_copier_container.txt -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Manual/08_Run_application_using_docker.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 38 - Data Copier - Using Custom Docker Image/Manual/08_Run_application_using_docker.txt -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Python Code/app/Dockerfile.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 38 - Data Copier - Using Custom Docker Image/Python Code/app/Dockerfile.txt -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Python Code/app/app.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 38 - Data Copier - Using Custom Docker Image/Python Code/app/app.py -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Python Code/app/read.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 38 - Data Copier - Using Custom Docker Image/Python Code/app/read.py -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Python Code/app/requirements.txt: -------------------------------------------------------------------------------- 1 | psycopg2-binary==3.0.8 2 | SQLAlchemy==1.4.31 3 | pandas==2.0.0 4 | -------------------------------------------------------------------------------- /Section 38 - Data Copier - Using Custom Docker Image/Python Code/app/write.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 38 - Data Copier - Using Custom Docker Image/Python Code/app/write.py -------------------------------------------------------------------------------- /Section 39 - Data Copier - Deploy and Validate Application on Remote Server/01_Clone_application_on_remote.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 39 - Data Copier - Deploy and Validate Application on Remote Server/01_Clone_application_on_remote.txt -------------------------------------------------------------------------------- /Section 39 - Data Copier - Deploy and Validate Application on Remote Server/02_Setup_network_and_database.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 39 - Data Copier - Deploy and Validate Application on Remote Server/02_Setup_network_and_database.txt -------------------------------------------------------------------------------- /Section 39 - Data Copier - Deploy and Validate Application on Remote Server/03_Setup_database_and tables_docker.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 39 - Data Copier - Deploy and Validate Application on Remote Server/03_Setup_database_and tables_docker.txt -------------------------------------------------------------------------------- /Section 39 - Data Copier - Deploy and Validate Application on Remote Server/04_Building_custom_docker.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 39 - Data Copier - Deploy and Validate Application on Remote Server/04_Building_custom_docker.txt -------------------------------------------------------------------------------- /Section 39 - Data Copier - Deploy and Validate Application on Remote Server/05_Run_and_validate_dockerized_application.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 39 - Data Copier - Deploy and Validate Application on Remote Server/05_Run_and_validate_dockerized_application.txt -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/03_getting_help_or_usage.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/03_getting_help_or_usage.ipynb -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/04_listing_hdfs_files.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/04_listing_hdfs_files.ipynb -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/05_managing_hdfs_directories.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/05_managing_hdfs_directories.ipynb -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/06_copying_files_from_local_to_hdfs.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/06_copying_files_from_local_to_hdfs.ipynb -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/07_copying_files_from_hdfs_to_local.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/07_copying_files_from_hdfs_to_local.ipynb -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/09_previewing_data_in_hdfs_files.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/09_previewing_data_in_hdfs_files.ipynb -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/10_getting_file_metadata.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/10_getting_file_metadata.ipynb -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/11_hdfs_blocksize.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/11_hdfs_blocksize.ipynb -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/12_hdfs_replication_factor.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/12_hdfs_replication_factor.ipynb -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/13_getting_hdfs_storage_usage.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/13_getting_hdfs_storage_usage.ipynb -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/14_using_hdfs_stat_command.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/14_using_hdfs_stat_command.ipynb -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/15_hdfs_file_permissions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/15_hdfs_file_permissions.ipynb -------------------------------------------------------------------------------- /Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/16_overriding_properties.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 41 - Introduction to Hadoop Eco System - Overview of HDFS/16_overriding_properties.ipynb -------------------------------------------------------------------------------- /Section 42 - Data Engineering using Spark SQL - Getting Started/01_getting_started.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 42 - Data Engineering using Spark SQL - Getting Started/01_getting_started.ipynb -------------------------------------------------------------------------------- /Section 42 - Data Engineering using Spark SQL - Getting Started/02_overview_of_spark_documentation.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 42 - Data Engineering using Spark SQL - Getting Started/02_overview_of_spark_documentation.ipynb -------------------------------------------------------------------------------- /Section 42 - Data Engineering using Spark SQL - Getting Started/03_overview_of_spark_sql_cli.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 42 - Data Engineering using Spark SQL - Getting Started/03_overview_of_spark_sql_cli.ipynb -------------------------------------------------------------------------------- /Section 42 - Data Engineering using Spark SQL - Getting Started/04_overview_of_spark_sql_properties.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 42 - Data Engineering using Spark SQL - Getting Started/04_overview_of_spark_sql_properties.ipynb -------------------------------------------------------------------------------- /Section 42 - Data Engineering using Spark SQL - Getting Started/05_running_os_commands.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 42 - Data Engineering using Spark SQL - Getting Started/05_running_os_commands.ipynb -------------------------------------------------------------------------------- /Section 42 - Data Engineering using Spark SQL - Getting Started/06_understanding_warehouse_directory.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 42 - Data Engineering using Spark SQL - Getting Started/06_understanding_warehouse_directory.ipynb -------------------------------------------------------------------------------- /Section 42 - Data Engineering using Spark SQL - Getting Started/07_managing_spark_metastore_databases.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 42 - Data Engineering using Spark SQL - Getting Started/07_managing_spark_metastore_databases.ipynb -------------------------------------------------------------------------------- /Section 42 - Data Engineering using Spark SQL - Getting Started/08_managing_spark_metastore_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 42 - Data Engineering using Spark SQL - Getting Started/08_managing_spark_metastore_tables.ipynb -------------------------------------------------------------------------------- /Section 42 - Data Engineering using Spark SQL - Getting Started/09_retrieve_metadata_of_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 42 - Data Engineering using Spark SQL - Getting Started/09_retrieve_metadata_of_tables.ipynb -------------------------------------------------------------------------------- /Section 42 - Data Engineering using Spark SQL - Getting Started/10_role_of_spark_or_hive_metastore.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 42 - Data Engineering using Spark SQL - Getting Started/10_role_of_spark_or_hive_metastore.ipynb -------------------------------------------------------------------------------- /Section 42 - Data Engineering using Spark SQL - Getting Started/11_exercise_getting_started.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 42 - Data Engineering using Spark SQL - Getting Started/11_exercise_getting_started.ipynb -------------------------------------------------------------------------------- /Section 43 - Data Engineering using Spark SQL - Basic Transformations/01_basic_transformations.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 43 - Data Engineering using Spark SQL - Basic Transformations/01_basic_transformations.ipynb -------------------------------------------------------------------------------- /Section 43 - Data Engineering using Spark SQL - Basic Transformations/02_spark_sql_overview.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 43 - Data Engineering using Spark SQL - Basic Transformations/02_spark_sql_overview.ipynb -------------------------------------------------------------------------------- /Section 43 - Data Engineering using Spark SQL - Basic Transformations/03_define_problem_statement.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 43 - Data Engineering using Spark SQL - Basic Transformations/03_define_problem_statement.ipynb -------------------------------------------------------------------------------- /Section 43 - Data Engineering using Spark SQL - Basic Transformations/04_preparing_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 43 - Data Engineering using Spark SQL - Basic Transformations/04_preparing_tables.ipynb -------------------------------------------------------------------------------- /Section 43 - Data Engineering using Spark SQL - Basic Transformations/05_projecting_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 43 - Data Engineering using Spark SQL - Basic Transformations/05_projecting_data.ipynb -------------------------------------------------------------------------------- /Section 43 - Data Engineering using Spark SQL - Basic Transformations/06_filtering_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 43 - Data Engineering using Spark SQL - Basic Transformations/06_filtering_data.ipynb -------------------------------------------------------------------------------- /Section 43 - Data Engineering using Spark SQL - Basic Transformations/07_joining_tables_inner.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 43 - Data Engineering using Spark SQL - Basic Transformations/07_joining_tables_inner.ipynb -------------------------------------------------------------------------------- /Section 43 - Data Engineering using Spark SQL - Basic Transformations/08_joining_tables_outer.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 43 - Data Engineering using Spark SQL - Basic Transformations/08_joining_tables_outer.ipynb -------------------------------------------------------------------------------- /Section 43 - Data Engineering using Spark SQL - Basic Transformations/09_aggregating_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 43 - Data Engineering using Spark SQL - Basic Transformations/09_aggregating_data.ipynb -------------------------------------------------------------------------------- /Section 43 - Data Engineering using Spark SQL - Basic Transformations/10_sorting_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 43 - Data Engineering using Spark SQL - Basic Transformations/10_sorting_data.ipynb -------------------------------------------------------------------------------- /Section 43 - Data Engineering using Spark SQL - Basic Transformations/11_conclusion_final_solution.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 43 - Data Engineering using Spark SQL - Basic Transformations/11_conclusion_final_solution.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/01_basic_ddl_and_dml.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/01_basic_ddl_and_dml.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/02_create_spark_metastore_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/02_create_spark_metastore_tables.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/03_overview_of_data_types.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/03_overview_of_data_types.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/04_adding_comments.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/04_adding_comments.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/05_loading_data_into_tables_local.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/05_loading_data_into_tables_local.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/06_loading_data_into_tables_hdfs.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/06_loading_data_into_tables_hdfs.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/07_loading_data_append_and_overwrite.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/07_loading_data_append_and_overwrite.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/08_creating_external_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/08_creating_external_tables.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/09_managed_vs_external_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/09_managed_vs_external_tables.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/10_overview_of_file_formats.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/10_overview_of_file_formats.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/11_dropping_tables_and_databases.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/11_dropping_tables_and_databases.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/12_truncating_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/12_truncating_tables.ipynb -------------------------------------------------------------------------------- /Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/13_exercises_managing_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 44 - Data Engineering using Spark SQL - Basic DDL and DML/13_exercises_managing_tables.ipynb -------------------------------------------------------------------------------- /Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/01_dml_and_partitioning.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/01_dml_and_partitioning.ipynb -------------------------------------------------------------------------------- /Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/02_introduction_to_partitioning.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/02_introduction_to_partitioning.ipynb -------------------------------------------------------------------------------- /Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/04_load_vs_insert.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/04_load_vs_insert.ipynb -------------------------------------------------------------------------------- /Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/06_creating_partitioned_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/06_creating_partitioned_tables.ipynb -------------------------------------------------------------------------------- /Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/07_adding_partitions_to_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/07_adding_partitions_to_tables.ipynb -------------------------------------------------------------------------------- /Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/08_loading_data_into_partitions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/08_loading_data_into_partitions.ipynb -------------------------------------------------------------------------------- /Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/10_using_dynamic_partition_mode.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/10_using_dynamic_partition_mode.ipynb -------------------------------------------------------------------------------- /Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/11_exercises_partitioned_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 45 - Data Engineering using Spark SQL - Managing Tables - DML and Partitioning/11_exercises_partitioned_tables.ipynb -------------------------------------------------------------------------------- /Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/01_predefined_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/01_predefined_functions.ipynb -------------------------------------------------------------------------------- /Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/02_overview_of_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/02_overview_of_functions.ipynb -------------------------------------------------------------------------------- /Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/03_validating_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/03_validating_functions.ipynb -------------------------------------------------------------------------------- /Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/04_string_manipulation_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/04_string_manipulation_functions.ipynb -------------------------------------------------------------------------------- /Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/05_date_manipulation_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/05_date_manipulation_functions.ipynb -------------------------------------------------------------------------------- /Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/06_overview_of_numeric_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/06_overview_of_numeric_functions.ipynb -------------------------------------------------------------------------------- /Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/07_data_type_conversion.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/07_data_type_conversion.ipynb -------------------------------------------------------------------------------- /Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/08_handling_null_values.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/08_handling_null_values.ipynb -------------------------------------------------------------------------------- /Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/09_using_case_and_when.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/09_using_case_and_when.ipynb -------------------------------------------------------------------------------- /Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/10_query_example_word_count.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 46 - Data Engineering using Spark SQL - Overview of Spark SQL Functions/10_query_example_word_count.ipynb -------------------------------------------------------------------------------- /Section 47 - Data Engineering using Spark SQL - Windowing Functions/01_windowing_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 47 - Data Engineering using Spark SQL - Windowing Functions/01_windowing_functions.ipynb -------------------------------------------------------------------------------- /Section 47 - Data Engineering using Spark SQL - Windowing Functions/02_prepare_database.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 47 - Data Engineering using Spark SQL - Windowing Functions/02_prepare_database.ipynb -------------------------------------------------------------------------------- /Section 47 - Data Engineering using Spark SQL - Windowing Functions/03_overview_of_windowing_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 47 - Data Engineering using Spark SQL - Windowing Functions/03_overview_of_windowing_functions.ipynb -------------------------------------------------------------------------------- /Section 47 - Data Engineering using Spark SQL - Windowing Functions/04_aggregations_using_windowing_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 47 - Data Engineering using Spark SQL - Windowing Functions/04_aggregations_using_windowing_functions.ipynb -------------------------------------------------------------------------------- /Section 47 - Data Engineering using Spark SQL - Windowing Functions/05_using_lead_or_lag.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 47 - Data Engineering using Spark SQL - Windowing Functions/05_using_lead_or_lag.ipynb -------------------------------------------------------------------------------- /Section 47 - Data Engineering using Spark SQL - Windowing Functions/06_getting_first_and_last_values.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 47 - Data Engineering using Spark SQL - Windowing Functions/06_getting_first_and_last_values.ipynb -------------------------------------------------------------------------------- /Section 47 - Data Engineering using Spark SQL - Windowing Functions/07_ranking_using_windowing_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 47 - Data Engineering using Spark SQL - Windowing Functions/07_ranking_using_windowing_functions.ipynb -------------------------------------------------------------------------------- /Section 47 - Data Engineering using Spark SQL - Windowing Functions/08_order_of_execution_of_sql.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 47 - Data Engineering using Spark SQL - Windowing Functions/08_order_of_execution_of_sql.ipynb -------------------------------------------------------------------------------- /Section 47 - Data Engineering using Spark SQL - Windowing Functions/09_overview_of_sub_queries.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 47 - Data Engineering using Spark SQL - Windowing Functions/09_overview_of_sub_queries.ipynb -------------------------------------------------------------------------------- /Section 47 - Data Engineering using Spark SQL - Windowing Functions/10_filtering_window_function_results.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 47 - Data Engineering using Spark SQL - Windowing Functions/10_filtering_window_function_results.ipynb -------------------------------------------------------------------------------- /Section 48 - Apache Spark using Python - Data Processing Overview/03_starting_spark_context.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 48 - Apache Spark using Python - Data Processing Overview/03_starting_spark_context.ipynb -------------------------------------------------------------------------------- /Section 48 - Apache Spark using Python - Data Processing Overview/04_overview_of_spark_read_apis.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 48 - Apache Spark using Python - Data Processing Overview/04_overview_of_spark_read_apis.ipynb -------------------------------------------------------------------------------- /Section 48 - Apache Spark using Python - Data Processing Overview/05_understand_airlines_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 48 - Apache Spark using Python - Data Processing Overview/05_understand_airlines_data.ipynb -------------------------------------------------------------------------------- /Section 48 - Apache Spark using Python - Data Processing Overview/06_inferring_schema.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 48 - Apache Spark using Python - Data Processing Overview/06_inferring_schema.ipynb -------------------------------------------------------------------------------- /Section 48 - Apache Spark using Python - Data Processing Overview/07_previewing_airlines_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 48 - Apache Spark using Python - Data Processing Overview/07_previewing_airlines_data.ipynb -------------------------------------------------------------------------------- /Section 48 - Apache Spark using Python - Data Processing Overview/08_overview_of_dataframe_apis.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 48 - Apache Spark using Python - Data Processing Overview/08_overview_of_dataframe_apis.ipynb -------------------------------------------------------------------------------- /Section 48 - Apache Spark using Python - Data Processing Overview/09_overview_of_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 48 - Apache Spark using Python - Data Processing Overview/09_overview_of_functions.ipynb -------------------------------------------------------------------------------- /Section 48 - Apache Spark using Python - Data Processing Overview/10_overview_of_spark_write_apis.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 48 - Apache Spark using Python - Data Processing Overview/10_overview_of_spark_write_apis.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/02_predefined_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/02_predefined_functions.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/03_create_dummy_dataframes.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/03_create_dummy_dataframes.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/04_categories_of_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/04_categories_of_functions.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/05_special_functions_col_and_lit.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/05_special_functions_col_and_lit.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/06_common_string_manipulation_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/06_common_string_manipulation_functions.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/07_extracting_strings_using_substring.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/07_extracting_strings_using_substring.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/08_extracting_strings_using_split.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/08_extracting_strings_using_split.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/09_padding_characters_around_strings.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/09_padding_characters_around_strings.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/10_trimming_characters_from_strings.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/10_trimming_characters_from_strings.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/11_date_and_time_manipulation_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/11_date_and_time_manipulation_functions.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/12_date_and_time_arithmetic.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/12_date_and_time_arithmetic.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/13_using_date_and_time_trunc_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/13_using_date_and_time_trunc_functions.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/14_date_and_time_extract_functions.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/14_date_and_time_extract_functions.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/15_using_to_date_and_to_timestamp.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/15_using_to_date_and_to_timestamp.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/16_using_date_format_function.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/16_using_date_format_function.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/17_dealing_with_unix_timestamp.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/17_dealing_with_unix_timestamp.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/18_dealing_with_nulls.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/18_dealing_with_nulls.ipynb -------------------------------------------------------------------------------- /Section 49 - Apache Spark using Python - Processing Column Data/19_using_case_and_when.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 49 - Apache Spark using Python - Processing Column Data/19_using_case_and_when.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/02_overview_of_basic_transformations.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/02_overview_of_basic_transformations.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/03_data_frame_for_basic_transformations.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/03_data_frame_for_basic_transformations.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/04_basic_filtering_of_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/04_basic_filtering_of_data.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/05_filtering_example_using_dates.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/05_filtering_example_using_dates.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/06_boolean_operators.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/06_boolean_operators.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/07_using_in_operator_or_isin_function.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/07_using_in_operator_or_isin_function.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/08_using_like_operator_or_function.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/08_using_like_operator_or_function.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/09_using_between_operator.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/09_using_between_operator.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/10_dealing_with_nulls_while_filtering.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/10_dealing_with_nulls_while_filtering.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/11_total_aggregations.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/11_total_aggregations.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/12_aggregate_data_using_groupby.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/12_aggregate_data_using_groupby.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/13_aggregate_data_using_rollup.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/13_aggregate_data_using_rollup.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/14_aggregate_data_using_cube.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/14_aggregate_data_using_cube.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/15_overview_of_sorting_data_frames.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/15_overview_of_sorting_data_frames.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/16_solution_problem_1.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/16_solution_problem_1.ipynb -------------------------------------------------------------------------------- /Section 50 - Apache Spark using Python - Basic Transformations/17_solution_problem_2.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 50 - Apache Spark using Python - Basic Transformations/17_solution_problem_2.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/02_preparing_data_sets_for_joins.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/02_preparing_data_sets_for_joins.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/03_analyze_data_sets_for_joins.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/03_analyze_data_sets_for_joins.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/04_problem_statements_for_joins.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/04_problem_statements_for_joins.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/05_overview_of_joins.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/05_overview_of_joins.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/06_using_inner_joins.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/06_using_inner_joins.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/07_left_or_right_outer_join.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/07_left_or_right_outer_join.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/08_solutions_problem_1.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/08_solutions_problem_1.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/09_solutions_problem_2.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/09_solutions_problem_2.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/10_solutions_problem_3.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/10_solutions_problem_3.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/11_solutions_problem_4.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/11_solutions_problem_4.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/12_solutions_problem_5.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/12_solutions_problem_5.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/13_solutions_problem_6.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/13_solutions_problem_6.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/14_solutions_problem_7.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/14_solutions_problem_7.ipynb -------------------------------------------------------------------------------- /Section 51 - Apache Spark using Python - Joining Data Sets/15_solutions_problem_8.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 51 - Apache Spark using Python - Joining Data Sets/15_solutions_problem_8.ipynb -------------------------------------------------------------------------------- /Section 52 - Apache Spark using Python - Spark Metastore/02_overview_of_spark_metastore.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 52 - Apache Spark using Python - Spark Metastore/02_overview_of_spark_metastore.ipynb -------------------------------------------------------------------------------- /Section 52 - Apache Spark using Python - Spark Metastore/03_exploring_spark_catalog.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 52 - Apache Spark using Python - Spark Metastore/03_exploring_spark_catalog.ipynb -------------------------------------------------------------------------------- /Section 52 - Apache Spark using Python - Spark Metastore/04_creating_metastore_tables_using_catalog.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 52 - Apache Spark using Python - Spark Metastore/04_creating_metastore_tables_using_catalog.ipynb -------------------------------------------------------------------------------- /Section 52 - Apache Spark using Python - Spark Metastore/05_inferring_schema_for_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 52 - Apache Spark using Python - Spark Metastore/05_inferring_schema_for_tables.ipynb -------------------------------------------------------------------------------- /Section 52 - Apache Spark using Python - Spark Metastore/06_define_schema_for_tables_using_structtype.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 52 - Apache Spark using Python - Spark Metastore/06_define_schema_for_tables_using_structtype.ipynb -------------------------------------------------------------------------------- /Section 52 - Apache Spark using Python - Spark Metastore/07_inserting_into_existing_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 52 - Apache Spark using Python - Spark Metastore/07_inserting_into_existing_tables.ipynb -------------------------------------------------------------------------------- /Section 52 - Apache Spark using Python - Spark Metastore/08_read_and_process_data_from_metastore_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 52 - Apache Spark using Python - Spark Metastore/08_read_and_process_data_from_metastore_tables.ipynb -------------------------------------------------------------------------------- /Section 52 - Apache Spark using Python - Spark Metastore/09_creating_partitioned_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 52 - Apache Spark using Python - Spark Metastore/09_creating_partitioned_tables.ipynb -------------------------------------------------------------------------------- /Section 52 - Apache Spark using Python - Spark Metastore/10_saving_as_partitioned_tables.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 52 - Apache Spark using Python - Spark Metastore/10_saving_as_partitioned_tables.ipynb -------------------------------------------------------------------------------- /Section 52 - Apache Spark using Python - Spark Metastore/11_creating_temp_views.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 52 - Apache Spark using Python - Spark Metastore/11_creating_temp_views.ipynb -------------------------------------------------------------------------------- /Section 52 - Apache Spark using Python - Spark Metastore/12_using_spark_sql.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 52 - Apache Spark using Python - Spark Metastore/12_using_spark_sql.ipynb -------------------------------------------------------------------------------- /Section 53 - Apache Spark - Development Life Cycle using Python/Getting Started: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 53 - Apache Spark - Development Life Cycle using Python/Getting Started -------------------------------------------------------------------------------- /Section 53 - Apache Spark - Development Life Cycle using Python/Process data using Spark APIs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 53 - Apache Spark - Development Life Cycle using Python/Process data using Spark APIs -------------------------------------------------------------------------------- /Section 53 - Apache Spark - Development Life Cycle using Python/Productionize Code: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 53 - Apache Spark - Development Life Cycle using Python/Productionize Code -------------------------------------------------------------------------------- /Section 53 - Apache Spark - Development Life Cycle using Python/Read data from files: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 53 - Apache Spark - Development Life Cycle using Python/Read data from files -------------------------------------------------------------------------------- /Section 53 - Apache Spark - Development Life Cycle using Python/Write data to files: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 53 - Apache Spark - Development Life Cycle using Python/Write data to files -------------------------------------------------------------------------------- /Section 53 - Apache Spark - Development Life Cycle using Python/set up a virtual environment: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 53 - Apache Spark - Development Life Cycle using Python/set up a virtual environment -------------------------------------------------------------------------------- /Section 54 - Spark Application Execution Life Cycle and Spark UI/Develop Shell Script to run Spark Application: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 54 - Spark Application Execution Life Cycle and Spark UI/Develop Shell Script to run Spark Application -------------------------------------------------------------------------------- /Section 54 - Spark Application Execution Life Cycle and Spark UI/Develop Word Count Application: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 54 - Spark Application Execution Life Cycle and Spark UI/Develop Word Count Application -------------------------------------------------------------------------------- /Section 54 - Spark Application Execution Life Cycle and Spark UI/Setup Data Set for Word Count Application: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/romulovieira777/Data_Engineering_Essentials_Hands_on_SQL_Python_and_Spark/HEAD/Section 54 - Spark Application Execution Life Cycle and Spark UI/Setup Data Set for Word Count Application --------------------------------------------------------------------------------