├── .gitignore ├── Data engineering architecture.png ├── README.md ├── dags └── kafka_stream.py ├── docker-compose.yml ├── requirements.txt ├── script └── entrypoint.sh └── spark_stream.py /.gitignore: -------------------------------------------------------------------------------- 1 | venv 2 | __pycache__ 3 | .idea -------------------------------------------------------------------------------- /Data engineering architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/airscholar/e2e-data-engineering/HEAD/Data engineering architecture.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/airscholar/e2e-data-engineering/HEAD/README.md -------------------------------------------------------------------------------- /dags/kafka_stream.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/airscholar/e2e-data-engineering/HEAD/dags/kafka_stream.py -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/airscholar/e2e-data-engineering/HEAD/docker-compose.yml -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/airscholar/e2e-data-engineering/HEAD/requirements.txt -------------------------------------------------------------------------------- /script/entrypoint.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/airscholar/e2e-data-engineering/HEAD/script/entrypoint.sh -------------------------------------------------------------------------------- /spark_stream.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/airscholar/e2e-data-engineering/HEAD/spark_stream.py --------------------------------------------------------------------------------