├── Chapter 1 - Getting Started with Databricks ├── Demo Notebook.ipynb └── Setup.ipynb ├── Chapter 2 - Managing Data with Delta Lake └── 2.1 - Delta Lake.sql ├── Chapter 3 - Mastering Relational Entities in Databricks ├── 3.1 - Databases and Tables.sql ├── 3.2A - Views.sql └── 3.2B - Views (Session 2).sql ├── Chapter 4 - Transforming Data with Apache Spark ├── 4.1 - Querying Files.sql ├── 4.2 - Writing to Tables.sql ├── 4.3 - Advanced Transformations.sql ├── 4.4 - Higher Order Functions.sql └── 4.5 - SQL UDFs.sql ├── Chapter 5 - Processing Incremental Data ├── 5.1 - Structured Streaming.py ├── 5.2 - Auto Loader.py └── 5.3 - Medallion Architecture.py ├── Chapter 6 - Building Production Pipelines ├── 6.1 - Delta Live Tables.sql ├── 6.2 - Output Exploration.sql ├── 6.3 - Land New Data.py └── 6.4 - CDC Pipeline.sql ├── Chapter 7 - Exploring Databricks SQL ├── Dashboard Data Source.sql └── Example SQL Query.sql ├── Chapter 8 - Implementing Data Governance ├── Managing Permissions.sql └── Unity Catalog in Action.sql ├── Exports └── book_materials.dbc ├── Includes ├── Images │ ├── book_cover.jpg │ └── school_schema.png └── School-Setup.py └── README.md /Chapter 1 - Getting Started with Databricks/Demo Notebook.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 1 - Getting Started with Databricks/Demo Notebook.ipynb -------------------------------------------------------------------------------- /Chapter 1 - Getting Started with Databricks/Setup.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 1 - Getting Started with Databricks/Setup.ipynb -------------------------------------------------------------------------------- /Chapter 2 - Managing Data with Delta Lake/2.1 - Delta Lake.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 2 - Managing Data with Delta Lake/2.1 - Delta Lake.sql -------------------------------------------------------------------------------- /Chapter 3 - Mastering Relational Entities in Databricks/3.1 - Databases and Tables.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 3 - Mastering Relational Entities in Databricks/3.1 - Databases and Tables.sql -------------------------------------------------------------------------------- /Chapter 3 - Mastering Relational Entities in Databricks/3.2A - Views.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 3 - Mastering Relational Entities in Databricks/3.2A - Views.sql -------------------------------------------------------------------------------- /Chapter 3 - Mastering Relational Entities in Databricks/3.2B - Views (Session 2).sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 3 - Mastering Relational Entities in Databricks/3.2B - Views (Session 2).sql -------------------------------------------------------------------------------- /Chapter 4 - Transforming Data with Apache Spark/4.1 - Querying Files.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 4 - Transforming Data with Apache Spark/4.1 - Querying Files.sql -------------------------------------------------------------------------------- /Chapter 4 - Transforming Data with Apache Spark/4.2 - Writing to Tables.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 4 - Transforming Data with Apache Spark/4.2 - Writing to Tables.sql -------------------------------------------------------------------------------- /Chapter 4 - Transforming Data with Apache Spark/4.3 - Advanced Transformations.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 4 - Transforming Data with Apache Spark/4.3 - Advanced Transformations.sql -------------------------------------------------------------------------------- /Chapter 4 - Transforming Data with Apache Spark/4.4 - Higher Order Functions.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 4 - Transforming Data with Apache Spark/4.4 - Higher Order Functions.sql -------------------------------------------------------------------------------- /Chapter 4 - Transforming Data with Apache Spark/4.5 - SQL UDFs.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 4 - Transforming Data with Apache Spark/4.5 - SQL UDFs.sql -------------------------------------------------------------------------------- /Chapter 5 - Processing Incremental Data/5.1 - Structured Streaming.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 5 - Processing Incremental Data/5.1 - Structured Streaming.py -------------------------------------------------------------------------------- /Chapter 5 - Processing Incremental Data/5.2 - Auto Loader.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 5 - Processing Incremental Data/5.2 - Auto Loader.py -------------------------------------------------------------------------------- /Chapter 5 - Processing Incremental Data/5.3 - Medallion Architecture.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 5 - Processing Incremental Data/5.3 - Medallion Architecture.py -------------------------------------------------------------------------------- /Chapter 6 - Building Production Pipelines/6.1 - Delta Live Tables.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 6 - Building Production Pipelines/6.1 - Delta Live Tables.sql -------------------------------------------------------------------------------- /Chapter 6 - Building Production Pipelines/6.2 - Output Exploration.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 6 - Building Production Pipelines/6.2 - Output Exploration.sql -------------------------------------------------------------------------------- /Chapter 6 - Building Production Pipelines/6.3 - Land New Data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 6 - Building Production Pipelines/6.3 - Land New Data.py -------------------------------------------------------------------------------- /Chapter 6 - Building Production Pipelines/6.4 - CDC Pipeline.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 6 - Building Production Pipelines/6.4 - CDC Pipeline.sql -------------------------------------------------------------------------------- /Chapter 7 - Exploring Databricks SQL/Dashboard Data Source.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 7 - Exploring Databricks SQL/Dashboard Data Source.sql -------------------------------------------------------------------------------- /Chapter 7 - Exploring Databricks SQL/Example SQL Query.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 7 - Exploring Databricks SQL/Example SQL Query.sql -------------------------------------------------------------------------------- /Chapter 8 - Implementing Data Governance/Managing Permissions.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 8 - Implementing Data Governance/Managing Permissions.sql -------------------------------------------------------------------------------- /Chapter 8 - Implementing Data Governance/Unity Catalog in Action.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Chapter 8 - Implementing Data Governance/Unity Catalog in Action.sql -------------------------------------------------------------------------------- /Exports/book_materials.dbc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Exports/book_materials.dbc -------------------------------------------------------------------------------- /Includes/Images/book_cover.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Includes/Images/book_cover.jpg -------------------------------------------------------------------------------- /Includes/Images/school_schema.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Includes/Images/school_schema.png -------------------------------------------------------------------------------- /Includes/School-Setup.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/Includes/School-Setup.py -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derar-alhussein/oreilly-databricks-dea/HEAD/README.md --------------------------------------------------------------------------------