├── .gitignore ├── README.md ├── apache-airflow.yaml ├── dags ├── etl_pipeline_dag.py ├── helpers │ └── connections.py ├── initialize_databases.py ├── initialize_reference_table.py └── tutorial_data │ ├── Company.csv │ └── CompanyValues.csv ├── etl_databases.yml └── requirements.txt /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/egehanyorulmaz/reference_etl/HEAD/.gitignore -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/egehanyorulmaz/reference_etl/HEAD/README.md -------------------------------------------------------------------------------- /apache-airflow.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/egehanyorulmaz/reference_etl/HEAD/apache-airflow.yaml -------------------------------------------------------------------------------- /dags/etl_pipeline_dag.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/egehanyorulmaz/reference_etl/HEAD/dags/etl_pipeline_dag.py -------------------------------------------------------------------------------- /dags/helpers/connections.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/egehanyorulmaz/reference_etl/HEAD/dags/helpers/connections.py -------------------------------------------------------------------------------- /dags/initialize_databases.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/egehanyorulmaz/reference_etl/HEAD/dags/initialize_databases.py -------------------------------------------------------------------------------- /dags/initialize_reference_table.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/egehanyorulmaz/reference_etl/HEAD/dags/initialize_reference_table.py -------------------------------------------------------------------------------- /dags/tutorial_data/Company.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/egehanyorulmaz/reference_etl/HEAD/dags/tutorial_data/Company.csv -------------------------------------------------------------------------------- /dags/tutorial_data/CompanyValues.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/egehanyorulmaz/reference_etl/HEAD/dags/tutorial_data/CompanyValues.csv -------------------------------------------------------------------------------- /etl_databases.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/egehanyorulmaz/reference_etl/HEAD/etl_databases.yml -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | pandas~=1.2.4 2 | psycopg2 3 | numpy 4 | mysql-connector-python --------------------------------------------------------------------------------