├── .gitignore ├── AWS ├── EKS.md ├── EMR.md ├── README.md └── S3.md ├── Git.md ├── Helm ├── Jupyterhub.md └── README.md ├── Linux.md ├── PySpark.md ├── Python.md ├── PythonScript ├── json_load.py ├── re_date_time.py ├── read_With_custom_schema.py ├── read_from_gitlab.py ├── read_parquet_file.py └── read_part_file.py ├── README.md ├── mongodb.md ├── pyspark ├── encrypt_decryt.py ├── profiler.sh ├── pySparkApp │ ├── Makefile │ ├── README.md │ ├── dist │ │ └── foo.zip │ ├── foo │ │ ├── __init__.py │ │ └── foo.py │ └── main.py ├── read_hive_table.py └── when_otherwise.py └── terraform.md /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | .DS_Store -------------------------------------------------------------------------------- /AWS/EKS.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/AWS/EKS.md -------------------------------------------------------------------------------- /AWS/EMR.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/AWS/EMR.md -------------------------------------------------------------------------------- /AWS/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/AWS/README.md -------------------------------------------------------------------------------- /AWS/S3.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/AWS/S3.md -------------------------------------------------------------------------------- /Git.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/Git.md -------------------------------------------------------------------------------- /Helm/Jupyterhub.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/Helm/Jupyterhub.md -------------------------------------------------------------------------------- /Helm/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/Helm/README.md -------------------------------------------------------------------------------- /Linux.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/Linux.md -------------------------------------------------------------------------------- /PySpark.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/PySpark.md -------------------------------------------------------------------------------- /Python.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/Python.md -------------------------------------------------------------------------------- /PythonScript/json_load.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/PythonScript/json_load.py -------------------------------------------------------------------------------- /PythonScript/re_date_time.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/PythonScript/re_date_time.py -------------------------------------------------------------------------------- /PythonScript/read_With_custom_schema.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/PythonScript/read_With_custom_schema.py -------------------------------------------------------------------------------- /PythonScript/read_from_gitlab.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/PythonScript/read_from_gitlab.py -------------------------------------------------------------------------------- /PythonScript/read_parquet_file.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/PythonScript/read_parquet_file.py -------------------------------------------------------------------------------- /PythonScript/read_part_file.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/PythonScript/read_part_file.py -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/README.md -------------------------------------------------------------------------------- /mongodb.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/mongodb.md -------------------------------------------------------------------------------- /pyspark/encrypt_decryt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/pyspark/encrypt_decryt.py -------------------------------------------------------------------------------- /pyspark/profiler.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/pyspark/profiler.sh -------------------------------------------------------------------------------- /pyspark/pySparkApp/Makefile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/pyspark/pySparkApp/Makefile -------------------------------------------------------------------------------- /pyspark/pySparkApp/README.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /pyspark/pySparkApp/dist/foo.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/pyspark/pySparkApp/dist/foo.zip -------------------------------------------------------------------------------- /pyspark/pySparkApp/foo/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /pyspark/pySparkApp/foo/foo.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/pyspark/pySparkApp/foo/foo.py -------------------------------------------------------------------------------- /pyspark/pySparkApp/main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/pyspark/pySparkApp/main.py -------------------------------------------------------------------------------- /pyspark/read_hive_table.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /pyspark/when_otherwise.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/pyspark/when_otherwise.py -------------------------------------------------------------------------------- /terraform.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/arverma/TowardsDataEngineering/HEAD/terraform.md --------------------------------------------------------------------------------