├── .astro └── config.yaml ├── .dockerignore ├── .gitignore ├── 00_Setup.ipynb ├── 01_Data_Engineering.ipynb ├── 01_Ingest.ipynb ├── 02_Data_Marketplace.ipynb ├── 02_Data_Science-ARIMA-Baseline.ipynb ├── 02_Data_Science.ipynb ├── 03_ML_Engineering.ipynb ├── 04_ML_Ops.ipynb ├── 05_Airflow_Pipeline.ipynb ├── 06_Streamlit_App.ipynb ├── Dockerfile ├── README.md ├── airflow_settings.yml ├── dags ├── airflow_incremental_pipeline.py ├── airflow_setup_pipeline.py ├── airflow_tasks.py ├── cdc.py ├── elt.py ├── feature_engineering.py ├── ingest.py ├── mlops_monthly_pipeline.py ├── mlops_pipeline.py ├── mlops_setup_pipeline.py ├── mlops_tasks.py ├── model_eval.py ├── snowpark_connection.py └── station_train_predict.py ├── docker-compose.override.yml.TEMP ├── environment.yml ├── include ├── .DS_Store ├── images │ ├── apache_airflow.jpg │ ├── sagemaker_studio_lab.jpg │ ├── streamlit.jpg │ ├── weathersource_getdata.png │ └── weathersource_tile.png ├── pytorch_tabnet.zip ├── rexmex.zip ├── state.json └── streamlit_app.py ├── jupyter_env.yml ├── packages.txt ├── requirements.txt ├── setup ├── Dockerfile ├── app-image-config-input.json ├── build.sh ├── create-domain-input.json ├── environment.yml ├── snowpark_container_build.zip └── update-domain-input.json └── tests └── dags └── test_dag_integrity.py /.astro/config.yaml: -------------------------------------------------------------------------------- 1 | project: 2 | name: citibike_ml 3 | -------------------------------------------------------------------------------- /.dockerignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/.dockerignore -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/.gitignore -------------------------------------------------------------------------------- /00_Setup.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/00_Setup.ipynb -------------------------------------------------------------------------------- /01_Data_Engineering.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/01_Data_Engineering.ipynb -------------------------------------------------------------------------------- /01_Ingest.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/01_Ingest.ipynb -------------------------------------------------------------------------------- /02_Data_Marketplace.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/02_Data_Marketplace.ipynb -------------------------------------------------------------------------------- /02_Data_Science-ARIMA-Baseline.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/02_Data_Science-ARIMA-Baseline.ipynb -------------------------------------------------------------------------------- /02_Data_Science.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/02_Data_Science.ipynb -------------------------------------------------------------------------------- /03_ML_Engineering.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/03_ML_Engineering.ipynb -------------------------------------------------------------------------------- /04_ML_Ops.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/04_ML_Ops.ipynb -------------------------------------------------------------------------------- /05_Airflow_Pipeline.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/05_Airflow_Pipeline.ipynb -------------------------------------------------------------------------------- /06_Streamlit_App.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/06_Streamlit_App.ipynb -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/Dockerfile -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/README.md -------------------------------------------------------------------------------- /airflow_settings.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/airflow_settings.yml -------------------------------------------------------------------------------- /dags/airflow_incremental_pipeline.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/airflow_incremental_pipeline.py -------------------------------------------------------------------------------- /dags/airflow_setup_pipeline.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/airflow_setup_pipeline.py -------------------------------------------------------------------------------- /dags/airflow_tasks.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/airflow_tasks.py -------------------------------------------------------------------------------- /dags/cdc.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/cdc.py -------------------------------------------------------------------------------- /dags/elt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/elt.py -------------------------------------------------------------------------------- /dags/feature_engineering.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/feature_engineering.py -------------------------------------------------------------------------------- /dags/ingest.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/ingest.py -------------------------------------------------------------------------------- /dags/mlops_monthly_pipeline.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/mlops_monthly_pipeline.py -------------------------------------------------------------------------------- /dags/mlops_pipeline.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/mlops_pipeline.py -------------------------------------------------------------------------------- /dags/mlops_setup_pipeline.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/mlops_setup_pipeline.py -------------------------------------------------------------------------------- /dags/mlops_tasks.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/mlops_tasks.py -------------------------------------------------------------------------------- /dags/model_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/model_eval.py -------------------------------------------------------------------------------- /dags/snowpark_connection.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/snowpark_connection.py -------------------------------------------------------------------------------- /dags/station_train_predict.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/dags/station_train_predict.py -------------------------------------------------------------------------------- /docker-compose.override.yml.TEMP: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/docker-compose.override.yml.TEMP -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/environment.yml -------------------------------------------------------------------------------- /include/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/include/.DS_Store -------------------------------------------------------------------------------- /include/images/apache_airflow.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/include/images/apache_airflow.jpg -------------------------------------------------------------------------------- /include/images/sagemaker_studio_lab.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/include/images/sagemaker_studio_lab.jpg -------------------------------------------------------------------------------- /include/images/streamlit.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/include/images/streamlit.jpg -------------------------------------------------------------------------------- /include/images/weathersource_getdata.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/include/images/weathersource_getdata.png -------------------------------------------------------------------------------- /include/images/weathersource_tile.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/include/images/weathersource_tile.png -------------------------------------------------------------------------------- /include/pytorch_tabnet.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/include/pytorch_tabnet.zip -------------------------------------------------------------------------------- /include/rexmex.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/include/rexmex.zip -------------------------------------------------------------------------------- /include/state.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/include/state.json -------------------------------------------------------------------------------- /include/streamlit_app.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/include/streamlit_app.py -------------------------------------------------------------------------------- /jupyter_env.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/jupyter_env.yml -------------------------------------------------------------------------------- /packages.txt: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /setup/Dockerfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/setup/Dockerfile -------------------------------------------------------------------------------- /setup/app-image-config-input.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/setup/app-image-config-input.json -------------------------------------------------------------------------------- /setup/build.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/setup/build.sh -------------------------------------------------------------------------------- /setup/create-domain-input.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/setup/create-domain-input.json -------------------------------------------------------------------------------- /setup/environment.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/setup/environment.yml -------------------------------------------------------------------------------- /setup/snowpark_container_build.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/setup/snowpark_container_build.zip -------------------------------------------------------------------------------- /setup/update-domain-input.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/setup/update-domain-input.json -------------------------------------------------------------------------------- /tests/dags/test_dag_integrity.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Snowflake-Labs/sfguide-citibike-ml-snowpark-python/HEAD/tests/dags/test_dag_integrity.py --------------------------------------------------------------------------------