├── README.md ├── airflow ├── Dockerfile ├── config │ └── airflow.cfg ├── requirements.txt └── script │ └── entrypoint.sh ├── docker-compose.yml └── sandbox-test ├── dags ├── test_hello_spark.py └── test_hello_world.py └── spark ├── app └── hello-world-spark.py └── resources └── data └── test.csv /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pyjaime/docker-airflow-spark/HEAD/README.md -------------------------------------------------------------------------------- /airflow/Dockerfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pyjaime/docker-airflow-spark/HEAD/airflow/Dockerfile -------------------------------------------------------------------------------- /airflow/config/airflow.cfg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pyjaime/docker-airflow-spark/HEAD/airflow/config/airflow.cfg -------------------------------------------------------------------------------- /airflow/requirements.txt: -------------------------------------------------------------------------------- 1 | celery 2 | kombu 3 | tornado 4 | werkzeug 5 | pyspark==3.2.1 6 | SQLAlchemy 7 | wtforms -------------------------------------------------------------------------------- /airflow/script/entrypoint.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pyjaime/docker-airflow-spark/HEAD/airflow/script/entrypoint.sh -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pyjaime/docker-airflow-spark/HEAD/docker-compose.yml -------------------------------------------------------------------------------- /sandbox-test/dags/test_hello_spark.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pyjaime/docker-airflow-spark/HEAD/sandbox-test/dags/test_hello_spark.py -------------------------------------------------------------------------------- /sandbox-test/dags/test_hello_world.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pyjaime/docker-airflow-spark/HEAD/sandbox-test/dags/test_hello_world.py -------------------------------------------------------------------------------- /sandbox-test/spark/app/hello-world-spark.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pyjaime/docker-airflow-spark/HEAD/sandbox-test/spark/app/hello-world-spark.py -------------------------------------------------------------------------------- /sandbox-test/spark/resources/data/test.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pyjaime/docker-airflow-spark/HEAD/sandbox-test/spark/resources/data/test.csv --------------------------------------------------------------------------------