├── .astro ├── config.yaml └── test_dag_integrity_default.py ├── .dockerignore ├── .env-example ├── .github └── workflows │ └── ci.yaml ├── .gitignore ├── .python-version ├── Dockerfile ├── README.md ├── airflow_settings.yaml ├── dags ├── .airflowignore └── dag_etl.py ├── data ├── eur_revenue.csv ├── usd_revenue.csv ├── wrong_data.csv └── yen_revenue.csv ├── dev-requirements.txt ├── docker-compose.yml ├── packages.txt ├── pics └── etl-diagram.png ├── pyproject.toml ├── requirements.txt ├── src ├── __init__.py ├── database.py ├── etl.py ├── google_drive.py ├── main.py ├── schema.py └── transform_utils.py └── tests ├── dags └── test_dag_example.py ├── test_schema_in.py └── test_schema_out.py /.astro/config.yaml: -------------------------------------------------------------------------------- 1 | project: 2 | name: etl-airflow 3 | -------------------------------------------------------------------------------- /.astro/test_dag_integrity_default.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/.astro/test_dag_integrity_default.py -------------------------------------------------------------------------------- /.dockerignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/.dockerignore -------------------------------------------------------------------------------- /.env-example: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/.env-example -------------------------------------------------------------------------------- /.github/workflows/ci.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/.github/workflows/ci.yaml -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/.gitignore -------------------------------------------------------------------------------- /.python-version: -------------------------------------------------------------------------------- 1 | 3.11.5 2 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/Dockerfile -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/README.md -------------------------------------------------------------------------------- /airflow_settings.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/airflow_settings.yaml -------------------------------------------------------------------------------- /dags/.airflowignore: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /dags/dag_etl.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/dags/dag_etl.py -------------------------------------------------------------------------------- /data/eur_revenue.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/data/eur_revenue.csv -------------------------------------------------------------------------------- /data/usd_revenue.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/data/usd_revenue.csv -------------------------------------------------------------------------------- /data/wrong_data.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/data/wrong_data.csv -------------------------------------------------------------------------------- /data/yen_revenue.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/data/yen_revenue.csv -------------------------------------------------------------------------------- /dev-requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/dev-requirements.txt -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/docker-compose.yml -------------------------------------------------------------------------------- /packages.txt: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /pics/etl-diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/pics/etl-diagram.png -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/pyproject.toml -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/requirements.txt -------------------------------------------------------------------------------- /src/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /src/database.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/src/database.py -------------------------------------------------------------------------------- /src/etl.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/src/etl.py -------------------------------------------------------------------------------- /src/google_drive.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/src/google_drive.py -------------------------------------------------------------------------------- /src/main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/src/main.py -------------------------------------------------------------------------------- /src/schema.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/src/schema.py -------------------------------------------------------------------------------- /src/transform_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/src/transform_utils.py -------------------------------------------------------------------------------- /tests/dags/test_dag_example.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/tests/dags/test_dag_example.py -------------------------------------------------------------------------------- /tests/test_schema_in.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/tests/test_schema_in.py -------------------------------------------------------------------------------- /tests/test_schema_out.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lealre/etl-airflow/HEAD/tests/test_schema_out.py --------------------------------------------------------------------------------