├── .DS_Store ├── .gitignore ├── .pre-commit-config.yaml ├── .python-version ├── Dockerfile ├── Makefile ├── README.md ├── app ├── ETL │ ├── __init__.py │ ├── handler_http.py │ ├── manager_aws.py │ ├── manager_polars.py │ └── manager_spotify.py ├── __init__.py ├── models │ ├── __init__.py │ └── spotify_models.py └── pipeline.py ├── data ├── beatles-data.csv ├── beatles-data.json └── beatles-data.parquet ├── docker-compose.yml ├── docs └── index.md ├── mkdocs.yml ├── poetry.lock ├── pyproject.toml └── static ├── LocalStack.png ├── log.png ├── mkdocs.png └── pipeline.png /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/.DS_Store -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/.gitignore -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/.pre-commit-config.yaml -------------------------------------------------------------------------------- /.python-version: -------------------------------------------------------------------------------- 1 | 3.11.3 2 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/Dockerfile -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/Makefile -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/README.md -------------------------------------------------------------------------------- /app/ETL/__init__.py: -------------------------------------------------------------------------------- 1 | """Essa módulo é o responsável por conter a ETL.""" 2 | -------------------------------------------------------------------------------- /app/ETL/handler_http.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/app/ETL/handler_http.py -------------------------------------------------------------------------------- /app/ETL/manager_aws.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/app/ETL/manager_aws.py -------------------------------------------------------------------------------- /app/ETL/manager_polars.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/app/ETL/manager_polars.py -------------------------------------------------------------------------------- /app/ETL/manager_spotify.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/app/ETL/manager_spotify.py -------------------------------------------------------------------------------- /app/__init__.py: -------------------------------------------------------------------------------- 1 | """Módulo Pipeline""" 2 | -------------------------------------------------------------------------------- /app/models/__init__.py: -------------------------------------------------------------------------------- 1 | """Módulo com os Models do Spotify.""" 2 | -------------------------------------------------------------------------------- /app/models/spotify_models.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/app/models/spotify_models.py -------------------------------------------------------------------------------- /app/pipeline.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/app/pipeline.py -------------------------------------------------------------------------------- /data/beatles-data.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/data/beatles-data.csv -------------------------------------------------------------------------------- /data/beatles-data.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/data/beatles-data.json -------------------------------------------------------------------------------- /data/beatles-data.parquet: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/data/beatles-data.parquet -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/docker-compose.yml -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/docs/index.md -------------------------------------------------------------------------------- /mkdocs.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/mkdocs.yml -------------------------------------------------------------------------------- /poetry.lock: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/poetry.lock -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/pyproject.toml -------------------------------------------------------------------------------- /static/LocalStack.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/static/LocalStack.png -------------------------------------------------------------------------------- /static/log.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/static/log.png -------------------------------------------------------------------------------- /static/mkdocs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/static/mkdocs.png -------------------------------------------------------------------------------- /static/pipeline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lvgalvao/datalake-format-explorer/HEAD/static/pipeline.png --------------------------------------------------------------------------------