├── .ipynb_checkpoints └── Pyspark Installation Testing Notebook-checkpoint.ipynb ├── 1_start ├── .ipynb_checkpoints │ ├── Basic Streaming Test-checkpoint.ipynb │ ├── First Spark Streaming Example-checkpoint.ipynb │ ├── First Twitter App-checkpoint.ipynb │ └── TweetRead-checkpoint.ipynb ├── First Twitter App.ipynb ├── TweetRead.ipynb └── TweetRead.py ├── 2_basics ├── .ipynb_checkpoints │ ├── 01_Queue of RDDs as a Stream-checkpoint.ipynb │ ├── 02_and_03_Basics of Transformations Demo-checkpoint.ipynb │ ├── 04_Basics of Transformations Exercise - Solution-checkpoint.ipynb │ ├── 04_Basics of Transformations Exercise-checkpoint.ipynb │ ├── 05_Transformation Operation Demo-checkpoint.ipynb │ ├── 06_Transformation Operation Exercise - Solution-checkpoint.ipynb │ ├── 06_Transformation Operation Exercise-checkpoint.ipynb │ ├── 07_Window Operations Demo-checkpoint.ipynb │ ├── 08_Window Operations Exercise - Solution-checkpoint.ipynb │ ├── 08_Window Operations Exercise-checkpoint.ipynb │ ├── 09_countByWindow transformation Demo-checkpoint.ipynb │ ├── 10_countByWindow transformation Exercise - Solution-checkpoint.ipynb │ ├── 10_countByWindow transformation Exercise-checkpoint.ipynb │ ├── 11_reduceByKeyAndWindow transformation Demo-checkpoint.ipynb │ ├── 12_reduceByKeyAndWindow transformation Exercise - Solution-checkpoint.ipynb │ ├── 12_reduceByKeyAndWindow transformation Exercise-checkpoint.ipynb │ ├── 13_countByValueAndWindow Transformation Demo-checkpoint.ipynb │ ├── 14_countByValueAndWindow Transformation Exercise - Solution-checkpoint.ipynb │ ├── 14_countByValueAndWindow Transformation Exercise-checkpoint.ipynb │ ├── 15_Output Operations on DStreams-checkpoint.ipynb │ ├── 16_foreachRDD Demo-checkpoint.ipynb │ ├── 17_foreachRDD Exercise - Solution-checkpoint.ipynb │ ├── 17_foreachRDD Exercise-checkpoint.ipynb │ ├── 18_SQL Operations Demo-checkpoint.ipynb │ ├── 19_SQL Operations Exercise - Solution-checkpoint.ipynb │ ├── 19_SQL Operations Exercise-checkpoint.ipynb │ ├── 20_Basics of Spark Streaming Section Exercise - Solution-checkpoint.ipynb │ ├── 20_Basics of Spark Streaming Section Exercise-checkpoint.ipynb │ ├── 7_Window Operations Demo-checkpoint.ipynb │ ├── 8_Window Operations Exercise-checkpoint.ipynb │ ├── Basic Streaming Test-checkpoint.ipynb │ ├── Basics of Transformations Demo 1-checkpoint.ipynb │ ├── Basics of Transformations Exercise-checkpoint.ipynb │ ├── Output Operations on DStreams-checkpoint.ipynb │ ├── Queue of RDDs as a Stream-checkpoint.ipynb │ ├── SQL Operations Demo-checkpoint.ipynb │ ├── SQL Operations Exercise-checkpoint.ipynb │ ├── Transformation Operation Demo-checkpoint.ipynb │ ├── Transformation Operation Exercise-checkpoint.ipynb │ ├── Untitled Notebook-checkpoint.ipynb │ ├── Window Operations Demo-checkpoint.ipynb │ ├── Window Operations Exercise-checkpoint.ipynb │ ├── countByValueAndWindow Transformation Demo-checkpoint.ipynb │ ├── countByValueAndWindow Transformation Exercise-checkpoint.ipynb │ ├── countByWindow transformation Demo-checkpoint.ipynb │ ├── countByWindow transformation Exercise-checkpoint.ipynb │ ├── foreachRDD Demo-checkpoint.ipynb │ ├── foreachRDD Exercise-checkpoint.ipynb │ ├── reduceByKeyAndWindow transformation Demo-checkpoint.ipynb │ └── reduceByKeyAndWindow transformation Exercise-checkpoint.ipynb ├── 01_Queue of RDDs as a Stream.ipynb ├── 02_and_03_Basics of Transformations Demo.ipynb ├── 04_Basics of Transformations Exercise - Solution.ipynb ├── 04_Basics of Transformations Exercise.ipynb ├── 05_Transformation Operation Demo.ipynb ├── 06_Transformation Operation Exercise - Solution.ipynb ├── 06_Transformation Operation Exercise.ipynb ├── 07_Window Operations Demo.ipynb ├── 08_Window Operations Exercise - Solution.ipynb ├── 08_Window Operations Exercise.ipynb ├── 09_countByWindow transformation Demo.ipynb ├── 10_countByWindow transformation Exercise - Solution.ipynb ├── 10_countByWindow transformation Exercise.ipynb ├── 11_reduceByKeyAndWindow transformation Demo.ipynb ├── 12_reduceByKeyAndWindow transformation Exercise - Solution.ipynb ├── 12_reduceByKeyAndWindow transformation Exercise.ipynb ├── 13_countByValueAndWindow Transformation Demo.ipynb ├── 14_countByValueAndWindow Transformation Exercise - Solution.ipynb ├── 14_countByValueAndWindow Transformation Exercise.ipynb ├── 15_Output Operations on DStreams.ipynb ├── 16_foreachRDD Demo.ipynb ├── 17_foreachRDD Exercise - Solution.ipynb ├── 17_foreachRDD Exercise.ipynb ├── 18_SQL Operations Demo.ipynb ├── 19_SQL Operations Exercise - Solution.ipynb ├── 19_SQL Operations Exercise.ipynb ├── 20_Basics of Spark Streaming Section Exercise - Solution.ipynb ├── 20_Basics of Spark Streaming Section Exercise.ipynb ├── Basic Streaming Test.ipynb ├── DrSeuss.text ├── __pycache__ │ └── apache_log_parser.cpython-35.pyc ├── access_log.log ├── apache_log_parser.py ├── checkpoint │ ├── .checkpoint-1516348666000.crc │ ├── .checkpoint-1516348744000.crc │ ├── .checkpoint-1518402455000.bk.crc │ ├── .checkpoint-1518402455000.crc │ ├── .checkpoint-1518402456000.bk.crc │ ├── .checkpoint-1518402456000.crc │ ├── .checkpoint-1518402457000.bk.crc │ ├── .checkpoint-1518402457000.crc │ ├── .checkpoint-1518402458000.bk.crc │ ├── .checkpoint-1518402458000.crc │ ├── .checkpoint-1518402459000.bk.crc │ ├── .checkpoint-1518402459000.crc │ ├── .checkpoint-1518557358000.bk.crc │ ├── .checkpoint-1518557358000.crc │ ├── .checkpoint-1518557359000.bk.crc │ ├── .checkpoint-1518557359000.crc │ ├── .checkpoint-1518557360000.bk.crc │ ├── .checkpoint-1518557360000.crc │ ├── .checkpoint-1518557361000.bk.crc │ ├── .checkpoint-1518557361000.crc │ ├── .checkpoint-1518557362000.bk.crc │ ├── .checkpoint-1518557362000.crc │ ├── .checkpoint-1519957956000.bk.crc │ ├── .checkpoint-1519957956000.crc │ ├── .checkpoint-1519957958000.crc │ ├── .checkpoint-1519957960000.crc │ ├── .checkpoint-1519957962000.bk.crc │ ├── .checkpoint-1519957962000.crc │ ├── .checkpoint-1519957964000.bk.crc │ ├── .checkpoint-1519957964000.crc │ ├── .checkpoint-1519957966000.bk.crc │ ├── .checkpoint-1519957966000.crc │ ├── 24932bca-b023-408e-a835-1a888a7d6aa6 │ │ ├── rdd-1927 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-1943 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-1959 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── _partitioner │ │ │ └── part-00000 │ │ ├── rdd-2281 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-2332 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-2347 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-2364 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── _partitioner │ │ │ └── part-00000 │ │ ├── rdd-2642 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-2728 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-2745 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-2762 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── _partitioner │ │ │ └── part-00000 │ │ ├── rdd-2948 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-3132 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-3147 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-3164 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── _partitioner │ │ │ └── part-00000 │ │ ├── rdd-3313 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-3538 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-3553 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-3571 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── _partitioner │ │ │ └── part-00000 │ │ ├── rdd-3627 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-3946 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-3961 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ ├── rdd-3978 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── _partitioner │ │ │ └── part-00000 │ │ └── rdd-3991 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ ├── 69275960-bc12-40cf-b14f-d2e148ae9b34 │ │ ├── rdd-384 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── _partitioner │ │ │ └── part-00000 │ │ └── rdd-579 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── _partitioner │ │ │ └── part-00000 │ ├── 9ec93a1d-eeec-432d-be82-ebd10c82043f │ │ ├── rdd-387 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ └── rdd-585 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ ├── cd444967-5cd9-488a-aeab-bbf22df3a984 │ │ ├── rdd-383 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ │ └── rdd-578 │ │ │ ├── ._partitioner.crc │ │ │ ├── .part-00000.crc │ │ │ ├── .part-00001.crc │ │ │ ├── .part-00002.crc │ │ │ ├── .part-00003.crc │ │ │ ├── _partitioner │ │ │ ├── part-00000 │ │ │ ├── part-00001 │ │ │ ├── part-00002 │ │ │ └── part-00003 │ ├── checkpoint-1518557358000 │ ├── checkpoint-1518557358000.bk │ ├── checkpoint-1518557359000 │ ├── checkpoint-1518557359000.bk │ ├── checkpoint-1518557360000 │ ├── checkpoint-1518557360000.bk │ ├── checkpoint-1518557361000 │ ├── checkpoint-1518557361000.bk │ ├── checkpoint-1518557362000 │ ├── checkpoint-1518557362000.bk │ ├── checkpoint-1519957956000 │ ├── checkpoint-1519957956000.bk │ ├── checkpoint-1519957958000 │ ├── checkpoint-1519957960000 │ ├── checkpoint-1519957962000 │ ├── checkpoint-1519957962000.bk │ ├── checkpoint-1519957964000 │ ├── checkpoint-1519957964000.bk │ ├── checkpoint-1519957966000 │ ├── checkpoint-1519957966000.bk │ └── receivedBlockMetadata │ │ ├── log-1518557323415-1518557383415 │ │ ├── log-1519957847369-1519957907369 │ │ └── log-1519957910693-1519957970693 ├── data │ ├── 2016-stack-overflow-survey-responses.csv │ ├── DrSeuss.text │ ├── RealEstate.csv │ ├── airports.text │ ├── greetings.txt │ ├── nasa_19950701.tsv │ ├── nasa_19950801.tsv │ ├── prime_nums.text │ ├── students.json │ ├── uk-makerspaces-identifiable-data.csv │ ├── uk-postcode.csv │ ├── uppercase.text │ └── word_count.text ├── logs │ └── access_log.log └── output │ ├── 001.txt │ ├── Counts-1518511080000 │ ├── ._SUCCESS.crc │ ├── .part-00000.crc │ ├── .part-00001.crc │ ├── _SUCCESS │ ├── part-00000 │ └── part-00001 │ ├── Counts-1518511085000 │ ├── ._SUCCESS.crc │ ├── .part-00000.crc │ ├── .part-00001.crc │ ├── _SUCCESS │ ├── part-00000 │ └── part-00001 │ ├── Counts-1518511090000 │ ├── ._SUCCESS.crc │ ├── .part-00000.crc │ ├── .part-00001.crc │ ├── _SUCCESS │ ├── part-00000 │ └── part-00001 │ ├── Counts-1518511095000 │ ├── ._SUCCESS.crc │ ├── .part-00000.crc │ ├── .part-00001.crc │ ├── _SUCCESS │ ├── part-00000 │ └── part-00001 │ ├── Counts-1518511100000 │ ├── ._SUCCESS.crc │ ├── .part-00000.crc │ ├── .part-00001.crc │ ├── _SUCCESS │ ├── part-00000 │ └── part-00001 │ ├── Counts-1518511105000 │ ├── ._SUCCESS.crc │ ├── .part-00000.crc │ ├── .part-00001.crc │ ├── _SUCCESS │ ├── part-00000 │ └── part-00001 │ ├── Counts-1518511110000 │ ├── ._SUCCESS.crc │ ├── .part-00000.crc │ ├── .part-00001.crc │ ├── _SUCCESS │ ├── part-00000 │ └── part-00001 │ ├── Counts-1518511115000 │ ├── ._SUCCESS.crc │ ├── .part-00000.crc │ ├── .part-00001.crc │ ├── _SUCCESS │ ├── part-00000 │ └── part-00001 │ └── Counts-1518511120000 │ ├── ._SUCCESS.crc │ ├── .part-00000.crc │ ├── .part-00001.crc │ ├── _SUCCESS │ ├── part-00000 │ └── part-00001 ├── 3_advanced ├── .ipynb_checkpoints │ ├── 1_Stream-Stream Join Demo-checkpoint.ipynb │ ├── 2_Stream-Dataset Join Demo-checkpoint.ipynb │ ├── 3_Join Operations Exercise - Solution-checkpoint.ipynb │ ├── 3_Join Operations Exercise-checkpoint.ipynb │ ├── 4_updateStateByKey Demo-checkpoint.ipynb │ ├── 5_updateStateByKey Exercise - Solution-checkpoint.ipynb │ ├── 5_updateStateByKey Exercise-checkpoint.ipynb │ ├── 6_Checkpointing Demo-checkpoint.ipynb │ ├── 7_Checkpointing Exercise - Solution-checkpoint.ipynb │ ├── 7_Checkpointing Exercise-Copy1-checkpoint.ipynb │ ├── 7_Checkpointing Exercise-checkpoint.ipynb │ ├── 8_Accumulators Demo-checkpoint.ipynb │ ├── 9_Accumulators Exercise - Solution-checkpoint.ipynb │ ├── 9_Accumulators Exercise-checkpoint.ipynb │ ├── Accumulators Demo-checkpoint.ipynb │ ├── Accumulators Exercise-checkpoint.ipynb │ ├── Checkpointing Demo-checkpoint.ipynb │ ├── Checkpointing Exercise-checkpoint.ipynb │ ├── Join Operations Exercise-checkpoint.ipynb │ ├── Stream-Dataset Join Demo-checkpoint.ipynb │ ├── Stream-Stream Join Demo-checkpoint.ipynb │ ├── updateStateByKey or mapWithState Demo-checkpoint.ipynb │ └── updateStateByKey or mapWithState Exercise-checkpoint.ipynb ├── 1_Stream-Stream Join Demo.ipynb ├── 2_Stream-Dataset Join Demo.ipynb ├── 3_Join Operations Exercise - Solution.ipynb ├── 3_Join Operations Exercise.ipynb ├── 4_updateStateByKey Demo.ipynb ├── 5_updateStateByKey Exercise - Solution.ipynb ├── 5_updateStateByKey Exercise.ipynb ├── 6_Checkpointing Demo.ipynb ├── 7_Checkpointing Exercise - Solution.ipynb ├── 7_Checkpointing Exercise.ipynb ├── 8_Accumulators Demo.ipynb ├── 9_Accumulators Exercise - Solution.ipynb ├── 9_Accumulators Exercise.ipynb ├── checkpoint │ ├── .checkpoint-1519965875000.crc │ ├── .checkpoint-1519965876000.crc │ ├── .checkpoint-1519965877000.crc │ ├── .checkpoint-1519965878000.crc │ ├── .checkpoint-1519965879000.crc │ ├── .checkpoint-1519965880000.crc │ ├── .checkpoint-1519965881000.crc │ ├── .checkpoint-1519965882000.crc │ ├── .checkpoint-1519965883000.crc │ ├── .checkpoint-1519965884000.crc │ ├── .checkpoint-1519968021000.bk.crc │ ├── .checkpoint-1519968021000.crc │ ├── .checkpoint-1519968022000.bk.crc │ ├── .checkpoint-1519968022000.crc │ ├── .checkpoint-1519968023000.bk.crc │ ├── .checkpoint-1519968023000.crc │ ├── .checkpoint-1519968024000.bk.crc │ ├── .checkpoint-1519968024000.crc │ ├── .checkpoint-1519968025000.bk.crc │ ├── .checkpoint-1519968025000.crc │ ├── .checkpoint-1519978126000.bk.crc │ ├── .checkpoint-1519978126000.crc │ ├── .checkpoint-1519978127000.bk.crc │ ├── .checkpoint-1519978127000.crc │ ├── .checkpoint-1519978128000.bk.crc │ ├── .checkpoint-1519978128000.crc │ ├── .checkpoint-1519978129000.bk.crc │ ├── .checkpoint-1519978129000.crc │ ├── .checkpoint-1519978130000.bk.crc │ ├── .checkpoint-1519978130000.crc │ ├── .checkpoint-1519979383000.crc │ ├── .checkpoint-1519979384000.bk.crc │ ├── .checkpoint-1519979384000.crc │ ├── .checkpoint-1519979385000.bk.crc │ ├── .checkpoint-1519979385000.crc │ ├── .checkpoint-1519979386000.bk.crc │ ├── .checkpoint-1519979386000.crc │ ├── .checkpoint-1519979387000.bk.crc │ ├── .checkpoint-1519979387000.crc │ ├── .checkpoint-1519979388000.crc │ ├── .checkpoint-1519983050000.bk.crc │ ├── .checkpoint-1519983050000.crc │ ├── .checkpoint-1519983052000.crc │ ├── .checkpoint-1519983054000.bk.crc │ ├── .checkpoint-1519983054000.crc │ ├── .checkpoint-1519983056000.bk.crc │ ├── .checkpoint-1519983056000.crc │ ├── .checkpoint-1519983068000.crc │ ├── .checkpoint-1519983070000.bk.crc │ ├── .checkpoint-1519983070000.crc │ ├── .checkpoint-1519983072000.bk.crc │ ├── .checkpoint-1519983072000.crc │ ├── .checkpoint-1519983074000.crc │ ├── .checkpoint-1519983076000.crc │ ├── .checkpoint-1519983078000.crc │ ├── .checkpoint-1519983170000.crc │ ├── .checkpoint-1519983172000.crc │ ├── .checkpoint-1519983174000.crc │ ├── .checkpoint-1519983176000.crc │ ├── .checkpoint-1519983178000.crc │ ├── .checkpoint-1519983180000.crc │ ├── .checkpoint-1519983182000.crc │ ├── .checkpoint-1519983184000.crc │ ├── .checkpoint-1519983186000.crc │ ├── .checkpoint-1519983188000.crc │ ├── checkpoint-1519968021000 │ ├── checkpoint-1519968021000.bk │ ├── checkpoint-1519968022000 │ ├── checkpoint-1519968022000.bk │ ├── checkpoint-1519968023000 │ ├── checkpoint-1519968023000.bk │ ├── checkpoint-1519968024000 │ ├── checkpoint-1519968024000.bk │ ├── checkpoint-1519968025000 │ ├── checkpoint-1519968025000.bk │ ├── checkpoint-1519983170000 │ ├── checkpoint-1519983172000 │ ├── checkpoint-1519983174000 │ ├── checkpoint-1519983176000 │ ├── checkpoint-1519983178000 │ ├── checkpoint-1519983180000 │ ├── checkpoint-1519983182000 │ ├── checkpoint-1519983184000 │ ├── checkpoint-1519983186000 │ ├── checkpoint-1519983188000 │ └── receivedBlockMetadata │ │ └── log-1519968004287-1519968064287 ├── data │ ├── 2016-stack-overflow-survey-responses.csv │ ├── RealEstate.csv │ ├── airports.text │ ├── ip_file1.txt │ ├── ip_file2.txt │ ├── nasa_19950701.tsv │ ├── nasa_19950801.tsv │ ├── prime_nums.text │ ├── uk-makerspaces-identifiable-data.csv │ ├── uk-postcode.csv │ ├── uppercase.text │ └── word_count.text └── output │ ├── accum.text │ └── accum.txt ├── 4_morestream ├── .ipynb_checkpoints │ ├── 1_Integration with Kafka Demo take1-checkpoint.ipynb │ ├── 1_Integration with Kafka Demo take2-checkpoint.ipynb │ ├── 1_Integration with Kafka Demo-checkpoint.ipynb │ ├── 1_Integration_with_Kafka_Demo-checkpoint.ipynb │ ├── 2_Integration with Kinesis Demo-checkpoint.ipynb │ ├── 2_Integration_with_Kinesis_Demo-checkpoint.ipynb │ ├── Integration with Kafka Demo-checkpoint.ipynb │ ├── Integration with Kafka Test-checkpoint.ipynb │ └── Integration with Kinesis Demo-checkpoint.ipynb ├── 1_Integration_with_Kafka_Demo.ipynb ├── 1_Integration_with_Kafka_Demo.py ├── 2_Integration_with_Kinesis_Demo.ipynb ├── 2_Integration_with_Kinesis_Demo.py ├── amazon-kinesis-client-1.7.4.jar ├── data │ ├── 2016-stack-overflow-survey-responses.csv │ ├── RealEstate.csv │ ├── airports.text │ ├── nasa_19950701.tsv │ ├── nasa_19950801.tsv │ ├── prime_nums.text │ ├── uk-makerspaces-identifiable-data.csv │ ├── uk-postcode.csv │ ├── uppercase.text │ └── word_count.text └── kinesisExercise.py ├── 5_structured ├── .ipynb_checkpoints │ ├── 1_Integration with Kafka Demo-checkpoint.ipynb │ ├── 1_Structured Streaming Demo-checkpoint.ipynb │ ├── 1_Structured_Streaming_Demo-checkpoint.ipynb │ ├── 2_Operations on Streaming Dataframes-Datasets Demo-checkpoint.ipynb │ ├── 2_Operations_on_Streaming_Dataframes-Datasets_Demo-checkpoint.ipynb │ ├── 3_Operations on Streaming Dataframes-Datasets Exercise-checkpoint.ipynb │ ├── 3_Operations_on_Streaming_Dataframes-Datasets_Exercise - Solution-checkpoint.ipynb │ ├── 3_Operations_on_Streaming_Dataframes-Datasets_Exercise-checkpoint.ipynb │ ├── 4_Window Operations Demo-checkpoint.ipynb │ ├── 4_Window_Operations_Demo-checkpoint.ipynb │ ├── 5_Window Operations Exercise - Solution-checkpoint.ipynb │ ├── 5_Window Operations Exercise-checkpoint.ipynb │ ├── Operations on Streaming Dataframes-Datasets Demo-checkpoint.ipynb │ ├── Operations on Streaming Dataframes-Datasets Exercise-checkpoint.ipynb │ ├── Structured Streaming Demo-checkpoint.ipynb │ ├── Window Operations Demo-checkpoint.ipynb │ └── Window Operations Exercise-checkpoint.ipynb ├── 1_Structured_Streaming_Demo.ipynb ├── 2_Operations_on_Streaming_Dataframes-Datasets_Demo.ipynb ├── 3_Operations_on_Streaming_Dataframes-Datasets_Exercise - Solution.ipynb ├── 3_Operations_on_Streaming_Dataframes-Datasets_Exercise.ipynb ├── 4_Window_Operations_Demo.ipynb ├── 5_Window Operations Exercise - Solution.ipynb ├── 5_Window Operations Exercise.ipynb ├── README.md ├── applicationHistory │ ├── .metadata.crc │ ├── commits │ │ └── 0 │ ├── local-1522024432571.inprogress │ ├── local-1522024548123.inprogress │ ├── local-1522054335049.inprogress │ ├── local-1522054516947.inprogress │ ├── local-1522054716297.inprogress │ ├── metadata │ ├── offsets │ │ └── 0 │ └── sources │ │ └── 0 │ │ └── 0 ├── buys.csv ├── data ├── monitoring_data │ └── 1.csv ├── split-dataset │ ├── 1.csv │ ├── 10.csv │ ├── 100.csv │ ├── 1000.csv │ ├── 1001.csv │ ├── 1002.csv │ ├── 1003.csv │ ├── 1004.csv │ ├── 1005.csv │ ├── 1006.csv │ ├── 1007.csv │ ├── 1008.csv │ ├── 1009.csv │ ├── 101.csv │ ├── 1010.csv │ ├── 1011.csv │ ├── 1012.csv │ ├── 1013.csv │ ├── 1014.csv │ ├── 1015.csv │ ├── 1016.csv │ ├── 1017.csv │ ├── 1018.csv │ ├── 1019.csv │ ├── 102.csv │ ├── 1020.csv │ ├── 1021.csv │ ├── 1022.csv │ ├── 1023.csv │ ├── 1024.csv │ ├── 1025.csv │ ├── 1026.csv │ ├── 1027.csv │ ├── 1028.csv │ ├── 1029.csv │ ├── 103.csv │ ├── 1030.csv │ ├── 1031.csv │ ├── 1032.csv │ ├── 1033.csv │ ├── 1034.csv │ ├── 1035.csv │ ├── 1036.csv │ ├── 1037.csv │ ├── 1038.csv │ ├── 1039.csv │ ├── 104.csv │ ├── 1040.csv │ ├── 1041.csv │ ├── 1042.csv │ ├── 1043.csv │ ├── 1044.csv │ ├── 1045.csv │ ├── 1046.csv │ ├── 1047.csv │ ├── 1048.csv │ ├── 1049.csv │ ├── 105.csv │ ├── 1050.csv │ ├── 1051.csv │ ├── 1052.csv │ ├── 1053.csv │ ├── 1054.csv │ ├── 1055.csv │ ├── 1056.csv │ ├── 1057.csv │ ├── 1058.csv │ ├── 1059.csv │ ├── 106.csv │ ├── 1060.csv │ ├── 1061.csv │ ├── 1062.csv │ ├── 1063.csv │ ├── 1064.csv │ ├── 1065.csv │ ├── 1066.csv │ ├── 1067.csv │ ├── 1068.csv │ ├── 1069.csv │ ├── 107.csv │ ├── 1070.csv │ ├── 1071.csv │ ├── 1072.csv │ ├── 1073.csv │ ├── 1074.csv │ ├── 1075.csv │ ├── 1076.csv │ ├── 1077.csv │ ├── 1078.csv │ ├── 1079.csv │ ├── 108.csv │ ├── 1080.csv │ ├── 1081.csv │ ├── 1082.csv │ ├── 1083.csv │ ├── 1084.csv │ ├── 1085.csv │ ├── 1086.csv │ ├── 1087.csv │ ├── 1088.csv │ ├── 1089.csv │ ├── 109.csv │ ├── 1090.csv │ ├── 1091.csv │ ├── 1092.csv │ ├── 1093.csv │ ├── 1094.csv │ ├── 1095.csv │ ├── 1096.csv │ ├── 1097.csv │ ├── 1098.csv │ ├── 1099.csv │ ├── 11.csv │ ├── 110.csv │ ├── 1100.csv │ ├── 1101.csv │ ├── 1102.csv │ ├── 1103.csv │ ├── 1104.csv │ ├── 1105.csv │ ├── 1106.csv │ ├── 1107.csv │ ├── 1108.csv │ ├── 1109.csv │ ├── 111.csv │ ├── 1110.csv │ ├── 1111.csv │ ├── 1112.csv │ ├── 1113.csv │ ├── 1114.csv │ ├── 1115.csv │ ├── 1116.csv │ ├── 1117.csv │ ├── 1118.csv │ ├── 1119.csv │ ├── 112.csv │ ├── 1120.csv │ ├── 1121.csv │ ├── 1122.csv │ ├── 1123.csv │ ├── 1124.csv │ ├── 1125.csv │ ├── 1126.csv │ ├── 1127.csv │ ├── 113.csv │ ├── 114.csv │ ├── 115.csv │ ├── 116.csv │ ├── 117.csv │ ├── 118.csv │ ├── 119.csv │ ├── 12.csv │ ├── 120.csv │ ├── 121.csv │ ├── 122.csv │ ├── 123.csv │ ├── 124.csv │ ├── 125.csv │ ├── 126.csv │ ├── 127.csv │ ├── 128.csv │ ├── 129.csv │ ├── 13.csv │ ├── 130.csv │ ├── 131.csv │ ├── 132.csv │ ├── 133.csv │ ├── 134.csv │ ├── 135.csv │ ├── 136.csv │ ├── 137.csv │ ├── 138.csv │ ├── 139.csv │ ├── 14.csv │ ├── 140.csv │ ├── 141.csv │ ├── 142.csv │ ├── 143.csv │ ├── 144.csv │ ├── 145.csv │ ├── 146.csv │ ├── 147.csv │ ├── 148.csv │ ├── 149.csv │ ├── 15.csv │ ├── 150.csv │ ├── 151.csv │ ├── 152.csv │ ├── 153.csv │ ├── 154.csv │ ├── 155.csv │ ├── 156.csv │ ├── 157.csv │ ├── 158.csv │ ├── 159.csv │ ├── 16.csv │ ├── 160.csv │ ├── 161.csv │ ├── 162.csv │ ├── 163.csv │ ├── 164.csv │ ├── 165.csv │ ├── 166.csv │ ├── 167.csv │ ├── 168.csv │ ├── 169.csv │ ├── 17.csv │ ├── 170.csv │ ├── 171.csv │ ├── 172.csv │ ├── 173.csv │ ├── 174.csv │ ├── 175.csv │ ├── 176.csv │ ├── 177.csv │ ├── 178.csv │ ├── 179.csv │ ├── 18.csv │ ├── 180.csv │ ├── 181.csv │ ├── 182.csv │ ├── 183.csv │ ├── 184.csv │ ├── 185.csv │ ├── 186.csv │ ├── 187.csv │ ├── 188.csv │ ├── 189.csv │ ├── 19.csv │ ├── 190.csv │ ├── 191.csv │ ├── 192.csv │ ├── 193.csv │ ├── 194.csv │ ├── 195.csv │ ├── 196.csv │ ├── 197.csv │ ├── 198.csv │ ├── 199.csv │ ├── 2.csv │ ├── 20.csv │ ├── 200.csv │ ├── 201.csv │ ├── 202.csv │ ├── 203.csv │ ├── 204.csv │ ├── 205.csv │ ├── 206.csv │ ├── 207.csv │ ├── 208.csv │ ├── 209.csv │ ├── 21.csv │ ├── 210.csv │ ├── 211.csv │ ├── 212.csv │ ├── 213.csv │ ├── 214.csv │ ├── 215.csv │ ├── 216.csv │ ├── 217.csv │ ├── 218.csv │ ├── 219.csv │ ├── 22.csv │ ├── 220.csv │ ├── 221.csv │ ├── 222.csv │ ├── 223.csv │ ├── 224.csv │ ├── 225.csv │ ├── 226.csv │ ├── 227.csv │ ├── 228.csv │ ├── 229.csv │ ├── 23.csv │ ├── 230.csv │ ├── 231.csv │ ├── 232.csv │ ├── 233.csv │ ├── 234.csv │ ├── 235.csv │ ├── 236.csv │ ├── 237.csv │ ├── 238.csv │ ├── 239.csv │ ├── 24.csv │ ├── 240.csv │ ├── 241.csv │ ├── 242.csv │ ├── 243.csv │ ├── 244.csv │ ├── 245.csv │ ├── 246.csv │ ├── 247.csv │ ├── 248.csv │ ├── 249.csv │ ├── 25.csv │ ├── 250.csv │ ├── 251.csv │ ├── 252.csv │ ├── 253.csv │ ├── 254.csv │ ├── 255.csv │ ├── 256.csv │ ├── 257.csv │ ├── 258.csv │ ├── 259.csv │ ├── 26.csv │ ├── 260.csv │ ├── 261.csv │ ├── 262.csv │ ├── 263.csv │ ├── 264.csv │ ├── 265.csv │ ├── 266.csv │ ├── 267.csv │ ├── 268.csv │ ├── 269.csv │ ├── 27.csv │ ├── 270.csv │ ├── 271.csv │ ├── 272.csv │ ├── 273.csv │ ├── 274.csv │ ├── 275.csv │ ├── 276.csv │ ├── 277.csv │ ├── 278.csv │ ├── 279.csv │ ├── 28.csv │ ├── 280.csv │ ├── 281.csv │ ├── 282.csv │ ├── 283.csv │ ├── 284.csv │ ├── 285.csv │ ├── 286.csv │ ├── 287.csv │ ├── 288.csv │ ├── 289.csv │ ├── 29.csv │ ├── 290.csv │ ├── 291.csv │ ├── 292.csv │ ├── 293.csv │ ├── 294.csv │ ├── 295.csv │ ├── 296.csv │ ├── 297.csv │ ├── 298.csv │ ├── 299.csv │ ├── 3.csv │ ├── 30.csv │ ├── 300.csv │ ├── 301.csv │ ├── 302.csv │ ├── 303.csv │ ├── 304.csv │ ├── 305.csv │ ├── 306.csv │ ├── 307.csv │ ├── 308.csv │ ├── 309.csv │ ├── 31.csv │ ├── 310.csv │ ├── 311.csv │ ├── 312.csv │ ├── 313.csv │ ├── 314.csv │ ├── 315.csv │ ├── 316.csv │ ├── 317.csv │ ├── 318.csv │ ├── 319.csv │ ├── 32.csv │ ├── 320.csv │ ├── 321.csv │ ├── 322.csv │ ├── 323.csv │ ├── 324.csv │ ├── 325.csv │ ├── 326.csv │ ├── 327.csv │ ├── 328.csv │ ├── 329.csv │ ├── 33.csv │ ├── 330.csv │ ├── 331.csv │ ├── 332.csv │ ├── 333.csv │ ├── 334.csv │ ├── 335.csv │ ├── 336.csv │ ├── 337.csv │ ├── 338.csv │ ├── 339.csv │ ├── 34.csv │ ├── 340.csv │ ├── 341.csv │ ├── 342.csv │ ├── 343.csv │ ├── 344.csv │ ├── 345.csv │ ├── 346.csv │ ├── 347.csv │ ├── 348.csv │ ├── 349.csv │ ├── 35.csv │ ├── 350.csv │ ├── 351.csv │ ├── 352.csv │ ├── 353.csv │ ├── 354.csv │ ├── 355.csv │ ├── 356.csv │ ├── 357.csv │ ├── 358.csv │ ├── 359.csv │ ├── 36.csv │ ├── 360.csv │ ├── 361.csv │ ├── 362.csv │ ├── 363.csv │ ├── 364.csv │ ├── 365.csv │ ├── 366.csv │ ├── 367.csv │ ├── 368.csv │ ├── 369.csv │ ├── 37.csv │ ├── 370.csv │ ├── 371.csv │ ├── 372.csv │ ├── 373.csv │ ├── 374.csv │ ├── 375.csv │ ├── 376.csv │ ├── 377.csv │ ├── 378.csv │ ├── 379.csv │ ├── 38.csv │ ├── 380.csv │ ├── 381.csv │ ├── 382.csv │ ├── 383.csv │ ├── 384.csv │ ├── 385.csv │ ├── 386.csv │ ├── 387.csv │ ├── 388.csv │ ├── 389.csv │ ├── 39.csv │ ├── 390.csv │ ├── 391.csv │ ├── 392.csv │ ├── 393.csv │ ├── 394.csv │ ├── 395.csv │ ├── 396.csv │ ├── 397.csv │ ├── 398.csv │ ├── 399.csv │ ├── 4.csv │ ├── 40.csv │ ├── 400.csv │ ├── 401.csv │ ├── 402.csv │ ├── 403.csv │ ├── 404.csv │ ├── 405.csv │ ├── 406.csv │ ├── 407.csv │ ├── 408.csv │ ├── 409.csv │ ├── 41.csv │ ├── 410.csv │ ├── 411.csv │ ├── 412.csv │ ├── 413.csv │ ├── 414.csv │ ├── 415.csv │ ├── 416.csv │ ├── 417.csv │ ├── 418.csv │ ├── 419.csv │ ├── 42.csv │ ├── 420.csv │ ├── 421.csv │ ├── 422.csv │ ├── 423.csv │ ├── 424.csv │ ├── 425.csv │ ├── 426.csv │ ├── 427.csv │ ├── 428.csv │ ├── 429.csv │ ├── 43.csv │ ├── 430.csv │ ├── 431.csv │ ├── 432.csv │ ├── 433.csv │ ├── 434.csv │ ├── 435.csv │ ├── 436.csv │ ├── 437.csv │ ├── 438.csv │ ├── 439.csv │ ├── 44.csv │ ├── 440.csv │ ├── 441.csv │ ├── 442.csv │ ├── 443.csv │ ├── 444.csv │ ├── 445.csv │ ├── 446.csv │ ├── 447.csv │ ├── 448.csv │ ├── 449.csv │ ├── 45.csv │ ├── 450.csv │ ├── 451.csv │ ├── 452.csv │ ├── 453.csv │ ├── 454.csv │ ├── 455.csv │ ├── 456.csv │ ├── 457.csv │ ├── 458.csv │ ├── 459.csv │ ├── 46.csv │ ├── 460.csv │ ├── 461.csv │ ├── 462.csv │ ├── 463.csv │ ├── 464.csv │ ├── 465.csv │ ├── 466.csv │ ├── 467.csv │ ├── 468.csv │ ├── 469.csv │ ├── 47.csv │ ├── 470.csv │ ├── 471.csv │ ├── 472.csv │ ├── 473.csv │ ├── 474.csv │ ├── 475.csv │ ├── 476.csv │ ├── 477.csv │ ├── 478.csv │ ├── 479.csv │ ├── 48.csv │ ├── 480.csv │ ├── 481.csv │ ├── 482.csv │ ├── 483.csv │ ├── 484.csv │ ├── 485.csv │ ├── 486.csv │ ├── 487.csv │ ├── 488.csv │ ├── 489.csv │ ├── 49.csv │ ├── 490.csv │ ├── 491.csv │ ├── 492.csv │ ├── 493.csv │ ├── 494.csv │ ├── 495.csv │ ├── 496.csv │ ├── 497.csv │ ├── 498.csv │ ├── 499.csv │ ├── 5.csv │ ├── 50.csv │ ├── 500.csv │ ├── 501.csv │ ├── 502.csv │ ├── 503.csv │ ├── 504.csv │ ├── 505.csv │ ├── 506.csv │ ├── 507.csv │ ├── 508.csv │ ├── 509.csv │ ├── 51.csv │ ├── 510.csv │ ├── 511.csv │ ├── 512.csv │ ├── 513.csv │ ├── 514.csv │ ├── 515.csv │ ├── 516.csv │ ├── 517.csv │ ├── 518.csv │ ├── 519.csv │ ├── 52.csv │ ├── 520.csv │ ├── 521.csv │ ├── 522.csv │ ├── 523.csv │ ├── 524.csv │ ├── 525.csv │ ├── 526.csv │ ├── 527.csv │ ├── 528.csv │ ├── 529.csv │ ├── 53.csv │ ├── 530.csv │ ├── 531.csv │ ├── 532.csv │ ├── 533.csv │ ├── 534.csv │ ├── 535.csv │ ├── 536.csv │ ├── 537.csv │ ├── 538.csv │ ├── 539.csv │ ├── 54.csv │ ├── 540.csv │ ├── 541.csv │ ├── 542.csv │ ├── 543.csv │ ├── 544.csv │ ├── 545.csv │ ├── 546.csv │ ├── 547.csv │ ├── 548.csv │ ├── 549.csv │ ├── 55.csv │ ├── 550.csv │ ├── 551.csv │ ├── 552.csv │ ├── 553.csv │ ├── 554.csv │ ├── 555.csv │ ├── 556.csv │ ├── 557.csv │ ├── 558.csv │ ├── 559.csv │ ├── 56.csv │ ├── 560.csv │ ├── 561.csv │ ├── 562.csv │ ├── 563.csv │ ├── 564.csv │ ├── 565.csv │ ├── 566.csv │ ├── 567.csv │ ├── 568.csv │ ├── 569.csv │ ├── 57.csv │ ├── 570.csv │ ├── 571.csv │ ├── 572.csv │ ├── 573.csv │ ├── 574.csv │ ├── 575.csv │ ├── 576.csv │ ├── 577.csv │ ├── 578.csv │ ├── 579.csv │ ├── 58.csv │ ├── 580.csv │ ├── 581.csv │ ├── 582.csv │ ├── 583.csv │ ├── 584.csv │ ├── 585.csv │ ├── 586.csv │ ├── 587.csv │ ├── 588.csv │ ├── 589.csv │ ├── 59.csv │ ├── 590.csv │ ├── 591.csv │ ├── 592.csv │ ├── 593.csv │ ├── 594.csv │ ├── 595.csv │ ├── 596.csv │ ├── 597.csv │ ├── 598.csv │ ├── 599.csv │ ├── 6.csv │ ├── 60.csv │ ├── 600.csv │ ├── 601.csv │ ├── 602.csv │ ├── 603.csv │ ├── 604.csv │ ├── 605.csv │ ├── 606.csv │ ├── 607.csv │ ├── 608.csv │ ├── 609.csv │ ├── 61.csv │ ├── 610.csv │ ├── 611.csv │ ├── 612.csv │ ├── 613.csv │ ├── 614.csv │ ├── 615.csv │ ├── 616.csv │ ├── 617.csv │ ├── 618.csv │ ├── 619.csv │ ├── 62.csv │ ├── 620.csv │ ├── 621.csv │ ├── 622.csv │ ├── 623.csv │ ├── 624.csv │ ├── 625.csv │ ├── 626.csv │ ├── 627.csv │ ├── 628.csv │ ├── 629.csv │ ├── 63.csv │ ├── 630.csv │ ├── 631.csv │ ├── 632.csv │ ├── 633.csv │ ├── 634.csv │ ├── 635.csv │ ├── 636.csv │ ├── 637.csv │ ├── 638.csv │ ├── 639.csv │ ├── 64.csv │ ├── 640.csv │ ├── 641.csv │ ├── 642.csv │ ├── 643.csv │ ├── 644.csv │ ├── 645.csv │ ├── 646.csv │ ├── 647.csv │ ├── 648.csv │ ├── 649.csv │ ├── 65.csv │ ├── 650.csv │ ├── 651.csv │ ├── 652.csv │ ├── 653.csv │ ├── 654.csv │ ├── 655.csv │ ├── 656.csv │ ├── 657.csv │ ├── 658.csv │ ├── 659.csv │ ├── 66.csv │ ├── 660.csv │ ├── 661.csv │ ├── 662.csv │ ├── 663.csv │ ├── 664.csv │ ├── 665.csv │ ├── 666.csv │ ├── 667.csv │ ├── 668.csv │ ├── 669.csv │ ├── 67.csv │ ├── 670.csv │ ├── 671.csv │ ├── 672.csv │ ├── 673.csv │ ├── 674.csv │ ├── 675.csv │ ├── 676.csv │ ├── 677.csv │ ├── 678.csv │ ├── 679.csv │ ├── 68.csv │ ├── 680.csv │ ├── 681.csv │ ├── 682.csv │ ├── 683.csv │ ├── 684.csv │ ├── 685.csv │ ├── 686.csv │ ├── 687.csv │ ├── 688.csv │ ├── 689.csv │ ├── 69.csv │ ├── 690.csv │ ├── 691.csv │ ├── 692.csv │ ├── 693.csv │ ├── 694.csv │ ├── 695.csv │ ├── 696.csv │ ├── 697.csv │ ├── 698.csv │ ├── 699.csv │ ├── 7.csv │ ├── 70.csv │ ├── 700.csv │ ├── 701.csv │ ├── 702.csv │ ├── 703.csv │ ├── 704.csv │ ├── 705.csv │ ├── 706.csv │ ├── 707.csv │ ├── 708.csv │ ├── 709.csv │ ├── 71.csv │ ├── 710.csv │ ├── 711.csv │ ├── 712.csv │ ├── 713.csv │ ├── 714.csv │ ├── 715.csv │ ├── 716.csv │ ├── 717.csv │ ├── 718.csv │ ├── 719.csv │ ├── 72.csv │ ├── 720.csv │ ├── 721.csv │ ├── 722.csv │ ├── 723.csv │ ├── 724.csv │ ├── 725.csv │ ├── 726.csv │ ├── 727.csv │ ├── 728.csv │ ├── 729.csv │ ├── 73.csv │ ├── 730.csv │ ├── 731.csv │ ├── 732.csv │ ├── 733.csv │ ├── 734.csv │ ├── 735.csv │ ├── 736.csv │ ├── 737.csv │ ├── 738.csv │ ├── 739.csv │ ├── 74.csv │ ├── 740.csv │ ├── 741.csv │ ├── 742.csv │ ├── 743.csv │ ├── 744.csv │ ├── 745.csv │ ├── 746.csv │ ├── 747.csv │ ├── 748.csv │ ├── 749.csv │ ├── 75.csv │ ├── 750.csv │ ├── 751.csv │ ├── 752.csv │ ├── 753.csv │ ├── 754.csv │ ├── 755.csv │ ├── 756.csv │ ├── 757.csv │ ├── 758.csv │ ├── 759.csv │ ├── 76.csv │ ├── 760.csv │ ├── 761.csv │ ├── 762.csv │ ├── 763.csv │ ├── 764.csv │ ├── 765.csv │ ├── 766.csv │ ├── 767.csv │ ├── 768.csv │ ├── 769.csv │ ├── 77.csv │ ├── 770.csv │ ├── 771.csv │ ├── 772.csv │ ├── 773.csv │ ├── 774.csv │ ├── 775.csv │ ├── 776.csv │ ├── 777.csv │ ├── 778.csv │ ├── 779.csv │ ├── 78.csv │ ├── 780.csv │ ├── 781.csv │ ├── 782.csv │ ├── 783.csv │ ├── 784.csv │ ├── 785.csv │ ├── 786.csv │ ├── 787.csv │ ├── 788.csv │ ├── 789.csv │ ├── 79.csv │ ├── 790.csv │ ├── 791.csv │ ├── 792.csv │ ├── 793.csv │ ├── 794.csv │ ├── 795.csv │ ├── 796.csv │ ├── 797.csv │ ├── 798.csv │ ├── 799.csv │ ├── 8.csv │ ├── 80.csv │ ├── 800.csv │ ├── 801.csv │ ├── 802.csv │ ├── 803.csv │ ├── 804.csv │ ├── 805.csv │ ├── 806.csv │ ├── 807.csv │ ├── 808.csv │ ├── 809.csv │ ├── 81.csv │ ├── 810.csv │ ├── 811.csv │ ├── 812.csv │ ├── 813.csv │ ├── 814.csv │ ├── 815.csv │ ├── 816.csv │ ├── 817.csv │ ├── 818.csv │ ├── 819.csv │ ├── 82.csv │ ├── 820.csv │ ├── 821.csv │ ├── 822.csv │ ├── 823.csv │ ├── 824.csv │ ├── 825.csv │ ├── 826.csv │ ├── 827.csv │ ├── 828.csv │ ├── 829.csv │ ├── 83.csv │ ├── 830.csv │ ├── 831.csv │ ├── 832.csv │ ├── 833.csv │ ├── 834.csv │ ├── 835.csv │ ├── 836.csv │ ├── 837.csv │ ├── 838.csv │ ├── 839.csv │ ├── 84.csv │ ├── 840.csv │ ├── 841.csv │ ├── 842.csv │ ├── 843.csv │ ├── 844.csv │ ├── 845.csv │ ├── 846.csv │ ├── 847.csv │ ├── 848.csv │ ├── 849.csv │ ├── 85.csv │ ├── 850.csv │ ├── 851.csv │ ├── 852.csv │ ├── 853.csv │ ├── 854.csv │ ├── 855.csv │ ├── 856.csv │ ├── 857.csv │ ├── 858.csv │ ├── 859.csv │ ├── 86.csv │ ├── 860.csv │ ├── 861.csv │ ├── 862.csv │ ├── 863.csv │ ├── 864.csv │ ├── 865.csv │ ├── 866.csv │ ├── 867.csv │ ├── 868.csv │ ├── 869.csv │ ├── 87.csv │ ├── 870.csv │ ├── 871.csv │ ├── 872.csv │ ├── 873.csv │ ├── 874.csv │ ├── 875.csv │ ├── 876.csv │ ├── 877.csv │ ├── 878.csv │ ├── 879.csv │ ├── 88.csv │ ├── 880.csv │ ├── 881.csv │ ├── 882.csv │ ├── 883.csv │ ├── 884.csv │ ├── 885.csv │ ├── 886.csv │ ├── 887.csv │ ├── 888.csv │ ├── 889.csv │ ├── 89.csv │ ├── 890.csv │ ├── 891.csv │ ├── 892.csv │ ├── 893.csv │ ├── 894.csv │ ├── 895.csv │ ├── 896.csv │ ├── 897.csv │ ├── 898.csv │ ├── 899.csv │ ├── 9.csv │ ├── 90.csv │ ├── 900.csv │ ├── 901.csv │ ├── 902.csv │ ├── 903.csv │ ├── 904.csv │ ├── 905.csv │ ├── 906.csv │ ├── 907.csv │ ├── 908.csv │ ├── 909.csv │ ├── 91.csv │ ├── 910.csv │ ├── 911.csv │ ├── 912.csv │ ├── 913.csv │ ├── 914.csv │ ├── 915.csv │ ├── 916.csv │ ├── 917.csv │ ├── 918.csv │ ├── 919.csv │ ├── 92.csv │ ├── 920.csv │ ├── 921.csv │ ├── 922.csv │ ├── 923.csv │ ├── 924.csv │ ├── 925.csv │ ├── 926.csv │ ├── 927.csv │ ├── 928.csv │ ├── 929.csv │ ├── 93.csv │ ├── 930.csv │ ├── 931.csv │ ├── 932.csv │ ├── 933.csv │ ├── 934.csv │ ├── 935.csv │ ├── 936.csv │ ├── 937.csv │ ├── 938.csv │ ├── 939.csv │ ├── 94.csv │ ├── 940.csv │ ├── 941.csv │ ├── 942.csv │ ├── 943.csv │ ├── 944.csv │ ├── 945.csv │ ├── 946.csv │ ├── 947.csv │ ├── 948.csv │ ├── 949.csv │ ├── 95.csv │ ├── 950.csv │ ├── 951.csv │ ├── 952.csv │ ├── 953.csv │ ├── 954.csv │ ├── 955.csv │ ├── 956.csv │ ├── 957.csv │ ├── 958.csv │ ├── 959.csv │ ├── 96.csv │ ├── 960.csv │ ├── 961.csv │ ├── 962.csv │ ├── 963.csv │ ├── 964.csv │ ├── 965.csv │ ├── 966.csv │ ├── 967.csv │ ├── 968.csv │ ├── 969.csv │ ├── 97.csv │ ├── 970.csv │ ├── 971.csv │ ├── 972.csv │ ├── 973.csv │ ├── 974.csv │ ├── 975.csv │ ├── 976.csv │ ├── 977.csv │ ├── 978.csv │ ├── 979.csv │ ├── 98.csv │ ├── 980.csv │ ├── 981.csv │ ├── 982.csv │ ├── 983.csv │ ├── 984.csv │ ├── 985.csv │ ├── 986.csv │ ├── 987.csv │ ├── 988.csv │ ├── 989.csv │ ├── 99.csv │ ├── 990.csv │ ├── 991.csv │ ├── 992.csv │ ├── 993.csv │ ├── 994.csv │ ├── 995.csv │ ├── 996.csv │ ├── 997.csv │ ├── 998.csv │ └── 999.csv └── twitterIDs.csv ├── 6_finishing_up ├── 1_Introductory Lecture.pptx ├── 1_Introductory Lecture │ ├── Slide1.PNG │ ├── Slide2.PNG │ ├── Slide3.PNG │ └── Slide4.PNG ├── 2_Promotional Video.pptx ├── 3_Final Lecture.pptx └── 3_Final Lecture │ ├── Slide1.PNG │ ├── Slide2.PNG │ ├── Slide3.PNG │ ├── Slide4.PNG │ └── Slide5.PNG ├── Basic Streaming Test.ipynb └── README.md /1_start/TweetRead.py: -------------------------------------------------------------------------------- 1 | import tweepy 2 | from tweepy import OAuthHandler 3 | from tweepy import Stream 4 | from tweepy.streaming import StreamListener 5 | import socket 6 | import json 7 | 8 | 9 | # Set up your credentials 10 | consumer_key = 'pNYC6DkaEzm0M5CyIrgiW08K4' 11 | consumer_secret = 'PxpJPBxALtkDmUb8LVcJNkHlb5HXBpqWeqmPJAjYzWIzM165qo' 12 | access_token = '2988835149-CKcWUdG5BeoqA93wA3gBAwEdWeKk3Oru4NqiFM4' 13 | access_secret = 'V7eSQ06uZ8w8byspmLlm2u2iOLZaTY4jYHFxKiqF9074o' 14 | 15 | 16 | class TweetsListener(StreamListener): 17 | 18 | def __init__(self, csocket): 19 | self.client_socket = csocket 20 | 21 | def on_data(self, data): 22 | try: 23 | msg = json.loads( data ) 24 | print(msg['text'].encode('utf-8')) 25 | self.client_socket.send( msg['text'].encode('utf-8')) 26 | return True 27 | except BaseException as e: 28 | print("Error on_data: %s" % str(e)) 29 | return True 30 | 31 | def on_error(self, status): 32 | print(status) 33 | return True 34 | 35 | def sendData(c_socket): 36 | auth = OAuthHandler(consumer_key, consumer_secret) 37 | auth.set_access_token(access_token, access_secret) 38 | 39 | twitter_stream = Stream(auth, TweetsListener(c_socket)) 40 | twitter_stream.filter(track=['football']) 41 | 42 | if __name__ == "__main__": 43 | s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # Create a socket object 44 | host = "127.0.0.1" # Get local machine name 45 | port = 5555 # Reserve a port for your service. 46 | s.bind((host, port)) # Bind to the port 47 | 48 | print("Listening on port: %s" % str(port)) 49 | 50 | s.listen(5) # Now wait for client connection. 51 | c, addr = s.accept() # Establish connection with client. 52 | 53 | print("Received request from: " + str(addr)) 54 | 55 | sendData(c) 56 | -------------------------------------------------------------------------------- /2_basics/.ipynb_checkpoints/17_foreachRDD Exercise-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# foreachRDD Exercise" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Explain foreachRDD and the basic usage about foreachRDD" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "### Exercise" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [ 30 | "import findspark\n", 31 | "# TODO: your path will likely not have 'matthew' in it. Change it to reflect your path.\n", 32 | "findspark.init('/home/matthew/spark-2.1.0-bin-hadoop2.7')" 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": null, 38 | "metadata": {}, 39 | "outputs": [], 40 | "source": [ 41 | "import pyspark\n", 42 | "import pyspark.streaming\n", 43 | "from pyspark import SparkConf, SparkContext\n", 44 | "from pyspark.streaming import StreamingContext" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": { 51 | "collapsed": true 52 | }, 53 | "outputs": [], 54 | "source": [ 55 | "sc = SparkContext(\"local[2]\", \"CalculateSumSquares\")\n", 56 | "ssc = StreamingContext(sc, 1)\n", 57 | "\n", 58 | "lines = ssc.socketTextStream('localhost',9999)\n", 59 | "\n", 60 | "def sum_of_squares(lines):\n", 61 | " \n", 62 | " nbrs = lines.flatMap(lambda line: line.split(\" \"))\n", 63 | " result = nbrs.map(lambda x:int(x)**2).reduce(lambda x,y,:x+y)\n", 64 | " return result\n", 65 | "\n", 66 | "result = sum_of_squares(lines)\n", 67 | "result.pprint()\n", 68 | "\n", 69 | "# TODO: using `foreachRDD`, write a lambda function that outputs the stream to a text file.\n", 70 | "\n", 71 | "\n", 72 | "# End of Exercise section" 73 | ] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "execution_count": null, 78 | "metadata": {}, 79 | "outputs": [], 80 | "source": [ 81 | "ssc.start()\n", 82 | "# ssc.awaitTermination()" 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": null, 88 | "metadata": {}, 89 | "outputs": [], 90 | "source": [ 91 | "ssc.stop(stopSparkContext=True, stopGraceFully=True)" 92 | ] 93 | }, 94 | { 95 | "cell_type": "markdown", 96 | "metadata": {}, 97 | "source": [ 98 | "## References\n", 99 | "1. https://spark.apache.org/docs/latest/streaming-programming-guide.html#design-patterns-for-using-foreachrdd" 100 | ] 101 | }, 102 | { 103 | "cell_type": "markdown", 104 | "metadata": {}, 105 | "source": [ 106 | " " 107 | ] 108 | } 109 | ], 110 | "metadata": { 111 | "kernelspec": { 112 | "display_name": "Python 3", 113 | "language": "python", 114 | "name": "python3" 115 | }, 116 | "language_info": { 117 | "codemirror_mode": { 118 | "name": "ipython", 119 | "version": 3 120 | }, 121 | "file_extension": ".py", 122 | "mimetype": "text/x-python", 123 | "name": "python", 124 | "nbconvert_exporter": "python", 125 | "pygments_lexer": "ipython3", 126 | "version": "3.5.2" 127 | } 128 | }, 129 | "nbformat": 4, 130 | "nbformat_minor": 2 131 | } 132 | -------------------------------------------------------------------------------- /2_basics/.ipynb_checkpoints/Queue of RDDs as a Stream-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Overview of Discretized Streams" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "**Discretized Streams** (or **DStreams**) are the basic abstraction provided by Spark Streaming. These are continuous streams of data. The DStream could be the input coming from a source, or the output data that was generated by performing functions on the input. DStreams are basically continuous series of RDDs, which is Spark’s abstraction of an immutable, distributed dataset. \n", 15 | "\n", 16 | "One of the consequences of this is that any operation applied on a DStream translates to operations on the underlying RDDs. For example, in the earlier example of converting a stream of lines to words, the flatMap operation is applied on each RDD in the lines DStream to generate the RDDs of the words DStream.\n" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "metadata": {}, 22 | "source": [ 23 | "### Demo\n", 24 | "For testing a Spark Streaming application with test data, we are going to create a DStream based on a queue of RDDs, using `streamingContext.queueStream(queueOfRDDs)`. Each RDD pushed into the queue will be treated as a batch of data in the DStream, and processed like a stream." 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": null, 30 | "metadata": { 31 | "collapsed": true 32 | }, 33 | "outputs": [], 34 | "source": [ 35 | "import time\n", 36 | "from pyspark import SparkContext\n", 37 | "from pyspark.streaming import StreamingContext\n", 38 | "\n", 39 | "\n", 40 | "if __name__ == \"__main__\":\n", 41 | " sc = SparkContext(appName=\"PythonStreamingQueueStream\")\n", 42 | " ssc = StreamingContext(sc, 1)\n", 43 | " \n", 44 | " \n", 45 | " rddQueue = []\n", 46 | " for i in range(5):\n", 47 | " rddQueue += [ssc.sparkContext.parallelize([j for j in range(1, 1001)], 10)]\n", 48 | " \n", 49 | " inputStream = ssc.queueStream(rddQueue)\n", 50 | " mappedStream = inputStream.map(lambda x: (x % 10, 1))\n", 51 | " reducedStream = mappedStream.reduceByKey(lambda a, b: a + b)\n", 52 | " reducedStream.pprint()\n", 53 | " \n", 54 | " ssc.start()\n", 55 | " time.sleep(6)\n", 56 | " ssc.stop(stopSparkContext=True, stopGraceFully=True)" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "### References\n", 64 | "1. https://spark.apache.org/docs/latest/streaming-programming-guide.html#basic-sources\n", 65 | "2. https://spark.apache.org/docs/latest/streaming-programming-guide.html#discretized-streams-dstreams\n", 66 | "3. https://spark.apache.org/docs/latest/api/python/pyspark.streaming.html#pyspark.streaming.StreamingContext" 67 | ] 68 | }, 69 | { 70 | "cell_type": "markdown", 71 | "metadata": {}, 72 | "source": [ 73 | " " 74 | ] 75 | } 76 | ], 77 | "metadata": { 78 | "kernelspec": { 79 | "display_name": "Python 3", 80 | "language": "python", 81 | "name": "python3" 82 | }, 83 | "language_info": { 84 | "codemirror_mode": { 85 | "name": "ipython", 86 | "version": 3 87 | }, 88 | "file_extension": ".py", 89 | "mimetype": "text/x-python", 90 | "name": "python", 91 | "nbconvert_exporter": "python", 92 | "pygments_lexer": "ipython3", 93 | "version": "3.6.3" 94 | } 95 | }, 96 | "nbformat": 4, 97 | "nbformat_minor": 2 98 | } 99 | -------------------------------------------------------------------------------- /2_basics/.ipynb_checkpoints/Transformation Operation Demo-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Transformation Operation Demo" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "The `transform` operation (along with its variations like `transformWith`) allows arbitrary RDD-to-RDD functions to be applied on a DStream. It can be used to apply any RDD operation that is not exposed in the DStream API. For example, the functionality of joining every batch in a data stream with another dataset is not directly exposed in the DStream API. However, you can easily use `transform to do` this. This enables very powerful possibilities. For example, one can do real-time data cleaning by joining the input data stream with precomputed spam information (maybe generated with Spark as well) and then filtering based on it.\n", 15 | "```python\n", 16 | "spamInfoRDD = sc.pickleFile(...) # RDD containing spam information\n", 17 | "\n", 18 | "# join data stream with spam information to do data cleaning\n", 19 | "cleanedDStream = wordCounts.transform(lambda rdd: rdd.join(spamInfoRDD).filter(...))\n", 20 | "```\n", 21 | "Note that the supplied function gets called in every batch interval. This allows you to do time-varying RDD operations, that is, RDD operations, number of partitions, broadcast variables, etc. can be changed between batches.\n", 22 | "\n", 23 | "What is the benefit of it?\n" 24 | ] 25 | }, 26 | { 27 | "cell_type": "markdown", 28 | "metadata": {}, 29 | "source": [ 30 | "### Demo" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": null, 36 | "metadata": { 37 | "collapsed": true 38 | }, 39 | "outputs": [], 40 | "source": [] 41 | }, 42 | { 43 | "cell_type": "markdown", 44 | "metadata": {}, 45 | "source": [ 46 | "## References\n", 47 | "1. https://spark.apache.org/docs/latest/streaming-programming-guide.html#transform-operation\n" 48 | ] 49 | }, 50 | { 51 | "cell_type": "markdown", 52 | "metadata": {}, 53 | "source": [ 54 | " " 55 | ] 56 | } 57 | ], 58 | "metadata": { 59 | "kernelspec": { 60 | "display_name": "Python 3", 61 | "language": "python", 62 | "name": "python3" 63 | }, 64 | "language_info": { 65 | "codemirror_mode": { 66 | "name": "ipython", 67 | "version": 3 68 | }, 69 | "file_extension": ".py", 70 | "mimetype": "text/x-python", 71 | "name": "python", 72 | "nbconvert_exporter": "python", 73 | "pygments_lexer": "ipython3", 74 | "version": "3.6.1" 75 | } 76 | }, 77 | "nbformat": 4, 78 | "nbformat_minor": 2 79 | } 80 | -------------------------------------------------------------------------------- /2_basics/.ipynb_checkpoints/Transformation Operation Exercise-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Transformation Operation Exercise" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "The `transform` operation (along with its variations like `transformWith`) allows arbitrary RDD-to-RDD functions to be applied on a DStream. It can be used to apply any RDD operation that is not exposed in the DStream API. For example, the functionality of joining every batch in a data stream with another dataset is not directly exposed in the DStream API. However, you can easily use `transform to do` this. This enables very powerful possibilities. For example, one can do real-time data cleaning by joining the input data stream with precomputed spam information (maybe generated with Spark as well) and then filtering based on it.\n", 15 | "```python\n", 16 | "spamInfoRDD = sc.pickleFile(...) # RDD containing spam information\n", 17 | "\n", 18 | "# join data stream with spam information to do data cleaning\n", 19 | "cleanedDStream = wordCounts.transform(lambda rdd: rdd.join(spamInfoRDD).filter(...))\n", 20 | "```\n", 21 | "Note that the supplied function gets called in every batch interval. This allows you to do time-varying RDD operations, that is, RDD operations, number of partitions, broadcast variables, etc. can be changed between batches.\n", 22 | "\n", 23 | "What is the benefit of it?\n" 24 | ] 25 | }, 26 | { 27 | "cell_type": "markdown", 28 | "metadata": {}, 29 | "source": [ 30 | "### Exercise" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": null, 36 | "metadata": { 37 | "collapsed": true 38 | }, 39 | "outputs": [], 40 | "source": [] 41 | }, 42 | { 43 | "cell_type": "markdown", 44 | "metadata": {}, 45 | "source": [ 46 | "## References\n", 47 | "1. https://spark.apache.org/docs/latest/streaming-programming-guide.html#transform-operation\n" 48 | ] 49 | }, 50 | { 51 | "cell_type": "markdown", 52 | "metadata": {}, 53 | "source": [ 54 | " " 55 | ] 56 | } 57 | ], 58 | "metadata": { 59 | "kernelspec": { 60 | "display_name": "Python 3", 61 | "language": "python", 62 | "name": "python3" 63 | }, 64 | "language_info": { 65 | "codemirror_mode": { 66 | "name": "ipython", 67 | "version": 3 68 | }, 69 | "file_extension": ".py", 70 | "mimetype": "text/x-python", 71 | "name": "python", 72 | "nbconvert_exporter": "python", 73 | "pygments_lexer": "ipython3", 74 | "version": "3.6.1" 75 | } 76 | }, 77 | "nbformat": 4, 78 | "nbformat_minor": 2 79 | } 80 | -------------------------------------------------------------------------------- /2_basics/.ipynb_checkpoints/Untitled Notebook-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Untitled Notebook" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "This Notebook Contains no content. It is being used as a placeholder notebook" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "### Demo" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "metadata": { 28 | "collapsed": true 29 | }, 30 | "outputs": [], 31 | "source": [] 32 | } 33 | ], 34 | "metadata": { 35 | "kernelspec": { 36 | "display_name": "Python 3", 37 | "language": "python", 38 | "name": "python3" 39 | }, 40 | "language_info": { 41 | "codemirror_mode": { 42 | "name": "ipython", 43 | "version": 3 44 | }, 45 | "file_extension": ".py", 46 | "mimetype": "text/x-python", 47 | "name": "python", 48 | "nbconvert_exporter": "python", 49 | "pygments_lexer": "ipython3", 50 | "version": "3.6.1" 51 | } 52 | }, 53 | "nbformat": 4, 54 | "nbformat_minor": 2 55 | } 56 | -------------------------------------------------------------------------------- /2_basics/.ipynb_checkpoints/foreachRDD Exercise-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# foreachRDD Exercise" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Explain foreachRDD and the basic usage about foreachRDD" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "### Exercise" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "metadata": { 28 | "collapsed": true 29 | }, 30 | "outputs": [], 31 | "source": [ 32 | "import pyspark\n", 33 | "import pyspark.streaming\n", 34 | "from pyspark.streaming import SparkContext\n", 35 | "from pyspark.streaming import StreamingContext\n", 36 | "import utils\n", 37 | "import twitter_app\n", 38 | "\n", 39 | "\n", 40 | "twitter_app()\n", 41 | "\n", 42 | "ssc = StreamingContext(\"local[*]\", \"SaveTweets\", Seconds(1))\n", 43 | "\n", 44 | "tweets = TwitterUtils.createStream(ssc, None)\n", 45 | " \n", 46 | "# Now extract the text of each status update into RDD's using map()\n", 47 | "statuses = tweets.map(lambda status: status.getText())\n", 48 | "\n", 49 | "totalTweets = int(0)\n", 50 | " \n", 51 | "def twitterStatus(rdd, time):\n", 52 | " \n", 53 | " if rdd.count() > 0:\n", 54 | " \n", 55 | " repartitionedRDD = rdd.repartition(1).cache()\n", 56 | " repartitionedRDD.saveAsTextFile(\"Tweets_\" + time.milliseconds.toString)\n", 57 | " \n", 58 | " totalTweets += repartitionedRDD.count()\n", 59 | " print(\"Tweet count: \" + totalTweets)\n", 60 | " if totalTweets > 1000:\n", 61 | " sys.exit(0)\n", 62 | "\n", 63 | "# TODO: use ForeachRDD to process the 'twitterStatus()' function\n", 64 | "statuses.foreachRDD(twitterStatus(rdd, time))\n", 65 | " \n", 66 | "ssc.start()\n", 67 | "ssc.awaitTermination()" 68 | ] 69 | }, 70 | { 71 | "cell_type": "markdown", 72 | "metadata": {}, 73 | "source": [ 74 | "## References\n", 75 | "1. https://spark.apache.org/docs/latest/streaming-programming-guide.html#design-patterns-for-using-foreachrdd" 76 | ] 77 | }, 78 | { 79 | "cell_type": "markdown", 80 | "metadata": {}, 81 | "source": [ 82 | " " 83 | ] 84 | } 85 | ], 86 | "metadata": { 87 | "kernelspec": { 88 | "display_name": "Python 3", 89 | "language": "python", 90 | "name": "python3" 91 | }, 92 | "language_info": { 93 | "codemirror_mode": { 94 | "name": "ipython", 95 | "version": 3 96 | }, 97 | "file_extension": ".py", 98 | "mimetype": "text/x-python", 99 | "name": "python", 100 | "nbconvert_exporter": "python", 101 | "pygments_lexer": "ipython3", 102 | "version": "3.6.3" 103 | } 104 | }, 105 | "nbformat": 4, 106 | "nbformat_minor": 2 107 | } 108 | -------------------------------------------------------------------------------- /2_basics/17_foreachRDD Exercise.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# foreachRDD Exercise" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Explain foreachRDD and the basic usage about foreachRDD" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "### Exercise" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [ 30 | "import findspark\n", 31 | "# TODO: your path will likely not have 'matthew' in it. Change it to reflect your path.\n", 32 | "findspark.init('/home/matthew/spark-2.1.0-bin-hadoop2.7')" 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": null, 38 | "metadata": {}, 39 | "outputs": [], 40 | "source": [ 41 | "import pyspark\n", 42 | "import pyspark.streaming\n", 43 | "from pyspark import SparkConf, SparkContext\n", 44 | "from pyspark.streaming import StreamingContext" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": { 51 | "collapsed": true 52 | }, 53 | "outputs": [], 54 | "source": [ 55 | "sc = SparkContext(\"local[2]\", \"CalculateSumSquares\")\n", 56 | "ssc = StreamingContext(sc, 1)\n", 57 | "\n", 58 | "lines = ssc.socketTextStream('localhost',9999)\n", 59 | "\n", 60 | "def sum_of_squares(lines):\n", 61 | " \n", 62 | " nbrs = lines.flatMap(lambda line: line.split(\" \"))\n", 63 | " result = nbrs.map(lambda x:int(x)**2).reduce(lambda x,y,:x+y)\n", 64 | " return result\n", 65 | "\n", 66 | "result = sum_of_squares(lines)\n", 67 | "result.pprint()\n", 68 | "\n", 69 | "# TODO: using `foreachRDD`, write a lambda function that outputs the stream to a text file.\n", 70 | "\n", 71 | "\n", 72 | "# End of Exercise section" 73 | ] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "execution_count": null, 78 | "metadata": {}, 79 | "outputs": [], 80 | "source": [ 81 | "ssc.start()\n", 82 | "# ssc.awaitTermination()" 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": null, 88 | "metadata": {}, 89 | "outputs": [], 90 | "source": [ 91 | "ssc.stop(stopSparkContext=True, stopGraceFully=True)" 92 | ] 93 | }, 94 | { 95 | "cell_type": "markdown", 96 | "metadata": {}, 97 | "source": [ 98 | "## References\n", 99 | "1. https://spark.apache.org/docs/latest/streaming-programming-guide.html#design-patterns-for-using-foreachrdd" 100 | ] 101 | }, 102 | { 103 | "cell_type": "markdown", 104 | "metadata": {}, 105 | "source": [ 106 | " " 107 | ] 108 | } 109 | ], 110 | "metadata": { 111 | "kernelspec": { 112 | "display_name": "Python 3", 113 | "language": "python", 114 | "name": "python3" 115 | }, 116 | "language_info": { 117 | "codemirror_mode": { 118 | "name": "ipython", 119 | "version": 3 120 | }, 121 | "file_extension": ".py", 122 | "mimetype": "text/x-python", 123 | "name": "python", 124 | "nbconvert_exporter": "python", 125 | "pygments_lexer": "ipython3", 126 | "version": "3.5.2" 127 | } 128 | }, 129 | "nbformat": 4, 130 | "nbformat_minor": 2 131 | } 132 | -------------------------------------------------------------------------------- /2_basics/__pycache__/apache_log_parser.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/__pycache__/apache_log_parser.cpython-35.pyc -------------------------------------------------------------------------------- /2_basics/apache_log_parser.py: -------------------------------------------------------------------------------- 1 | import re 2 | 3 | class ApacheAccessLog(): 4 | """ 5 | This class represents an Apache access log line. 6 | See http://httpd.apache.org/docs/2.2/logs.html for more details. 7 | """ 8 | 9 | # Example Apache log line: 10 | # 127.0.0.1 - - [21/Jul/2014:9:55:27 -0800] "GET /home.html HTTP/1.1" 200 2048 11 | # 1:IP 2:client 3:user 4:datetime 5:method 6:req 7:proto 8:respcode 9:size 12 | # LOG_ENTRY_PATTERN = "^(\\S+) (\\S+) (\\S+) \\[([\\w:/]+\\s[+\\-]\\d{4})\\] \"(\\S+) (\\S+) (\\S+)\" (\\d{3}) (\\d+)" 13 | log_entry_regex = '(.*?) (.*?) (.*?) \[(.*?)\] "(.*?) (.*?) (.*?)" (\d{3}) (.*?)' 14 | 15 | def __init__(self, ip, client_identd, user_id, dateTime, method, endpoint, protocol, response_code, content_size): 16 | self.ip = ip 17 | self.client_identd = client_identd 18 | self.user_id = user_id 19 | self.date_time_string = dateTime 20 | self.method = method 21 | self.endpoint = endpoint 22 | self.protocol = protocol 23 | self.response_code = response_code 24 | self.content_size = content_size 25 | 26 | @staticmethod 27 | def parse_from_log_line(logline): 28 | m = re.search(ApacheAccessLog.log_entry_regex, logline) 29 | if m is None: 30 | print("Cannot parse logline" + logline) 31 | return None 32 | return ApacheAccessLog(m.group(1), m.group(2), m.group(3), m.group(4), m.group(5), m.group(6), m.group(7), m.group(8), m.group(9)) 33 | 34 | def __repr__(self): 35 | return "{} {} {} [{}] \"{} {} {}\" {} {}".format(self.ip, self.client_identd, self.user_id, self.date_time_string, self.method, self.endpoint,self.protocol, self.response_code, self.content_size) -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1516348666000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1516348666000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1516348744000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1516348744000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518402455000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518402455000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518402455000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518402455000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518402456000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518402456000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518402456000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518402456000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518402457000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518402457000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518402457000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518402457000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518402458000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518402458000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518402458000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518402458000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518402459000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518402459000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518402459000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518402459000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518557358000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518557358000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518557358000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518557358000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518557359000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518557359000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518557359000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518557359000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518557360000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518557360000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518557360000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518557360000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518557361000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518557361000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518557361000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518557361000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518557362000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518557362000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1518557362000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1518557362000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1519957956000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1519957956000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1519957956000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1519957956000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1519957958000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1519957958000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1519957960000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1519957960000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1519957962000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1519957962000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1519957962000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1519957962000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1519957964000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1519957964000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1519957964000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1519957964000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1519957966000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1519957966000.bk.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/.checkpoint-1519957966000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/.checkpoint-1519957966000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1927/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1943/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1959/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1959/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1959/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1959/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1959/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1959/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1959/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-1959/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2281/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2332/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2347/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2364/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2364/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2364/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2364/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2364/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2364/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2364/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2364/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2642/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2728/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2745/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2762/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2762/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2762/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2762/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2762/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2762/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2762/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2762/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-2948/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3132/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3147/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3164/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3164/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3164/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3164/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3164/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3164/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3164/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3164/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3313/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3538/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3553/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3571/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3571/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3571/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3571/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3571/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3571/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3571/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3571/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3627/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3946/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3961/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3978/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3978/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3978/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3978/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3978/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3978/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3978/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3978/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/24932bca-b023-408e-a835-1a888a7d6aa6/rdd-3991/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-384/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-384/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-384/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-384/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-384/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-384/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-384/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-384/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-579/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-579/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-579/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-579/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-579/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-579/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-579/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/69275960-bc12-40cf-b14f-d2e148ae9b34/rdd-579/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-387/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/9ec93a1d-eeec-432d-be82-ebd10c82043f/rdd-585/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-383/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/._partitioner.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/._partitioner.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/.part-00002.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/.part-00002.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/.part-00003.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/.part-00003.crc -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/_partitioner: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/_partitioner -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/part-00000 -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/part-00001 -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/part-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/part-00002 -------------------------------------------------------------------------------- /2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/part-00003: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/cd444967-5cd9-488a-aeab-bbf22df3a984/rdd-578/part-00003 -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1518557358000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1518557358000 -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1518557358000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1518557358000.bk -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1518557359000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1518557359000 -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1518557359000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1518557359000.bk -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1518557360000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1518557360000 -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1518557360000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1518557360000.bk -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1518557361000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1518557361000 -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1518557361000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1518557361000.bk -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1518557362000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1518557362000 -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1518557362000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1518557362000.bk -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1519957956000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1519957956000 -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1519957956000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1519957956000.bk -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1519957958000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1519957958000 -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1519957960000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1519957960000 -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1519957962000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1519957962000 -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1519957962000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1519957962000.bk -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1519957964000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1519957964000 -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1519957964000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1519957964000.bk -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1519957966000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1519957966000 -------------------------------------------------------------------------------- /2_basics/checkpoint/checkpoint-1519957966000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/checkpoint-1519957966000.bk -------------------------------------------------------------------------------- /2_basics/checkpoint/receivedBlockMetadata/log-1518557323415-1518557383415: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/receivedBlockMetadata/log-1518557323415-1518557383415 -------------------------------------------------------------------------------- /2_basics/checkpoint/receivedBlockMetadata/log-1519957847369-1519957907369: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/receivedBlockMetadata/log-1519957847369-1519957907369 -------------------------------------------------------------------------------- /2_basics/checkpoint/receivedBlockMetadata/log-1519957910693-1519957970693: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/checkpoint/receivedBlockMetadata/log-1519957910693-1519957970693 -------------------------------------------------------------------------------- /2_basics/data/greetings.txt: -------------------------------------------------------------------------------- 1 | Good Morning 2 | Good Evening 3 | Good Day 4 | Happy Birthday 5 | Happy New Year -------------------------------------------------------------------------------- /2_basics/data/prime_nums.text: -------------------------------------------------------------------------------- 1 | 2 3 5 7 11 13 17 19 23 29 2 | 31 37 41 43 47 53 59 61 67 71 3 | 73 79 83 89 97 101 103 107 109 113 4 | 127 131 137 139 149 151 157 163 167 173 5 | 179 181 191 193 197 199 211 223 227 229 6 | 233 239 241 251 257 263 269 271 277 281 7 | 283 293 307 311 313 317 331 337 347 349 8 | 353 359 367 373 379 383 389 397 401 409 9 | 419 421 431 433 439 443 449 457 461 463 10 | 467 479 487 491 499 503 509 521 523 541 11 | -------------------------------------------------------------------------------- /2_basics/data/students.json: -------------------------------------------------------------------------------- 1 | {"id":1234, "event":"update", "name":"mike", "age":20} 2 | {"id":1235, "event":"update", "name":"jack", "age":30} 3 | {"id":1236, "event":"update", "name":"lucy", "age":20} 4 | {"id":1237, "event":"update", "name":"ann", "age":30} -------------------------------------------------------------------------------- /2_basics/output/001.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/001.txt -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511080000/._SUCCESS.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511080000/.part-00000.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511080000/.part-00001.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511080000/_SUCCESS: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511080000/_SUCCESS -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511080000/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511080000/part-00000 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511080000/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511080000/part-00001 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511085000/._SUCCESS.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511085000/.part-00000.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511085000/.part-00001.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511085000/_SUCCESS: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511085000/_SUCCESS -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511085000/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511085000/part-00000 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511085000/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511085000/part-00001 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511090000/._SUCCESS.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511090000/.part-00000.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511090000/.part-00001.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511090000/_SUCCESS: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511090000/_SUCCESS -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511090000/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511090000/part-00000 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511090000/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511090000/part-00001 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511095000/._SUCCESS.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511095000/.part-00000.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511095000/.part-00001.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511095000/_SUCCESS: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511095000/_SUCCESS -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511095000/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511095000/part-00000 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511095000/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511095000/part-00001 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511100000/._SUCCESS.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511100000/.part-00000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511100000/.part-00000.crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511100000/.part-00001.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511100000/.part-00001.crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511100000/_SUCCESS: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511100000/_SUCCESS -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511105000/._SUCCESS.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511105000/.part-00000.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511105000/.part-00001.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511105000/_SUCCESS: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511105000/_SUCCESS -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511105000/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511105000/part-00000 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511105000/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511105000/part-00001 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511110000/._SUCCESS.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511110000/.part-00000.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511110000/.part-00001.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511110000/_SUCCESS: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511110000/_SUCCESS -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511110000/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511110000/part-00000 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511110000/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511110000/part-00001 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511115000/._SUCCESS.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511115000/.part-00000.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511115000/.part-00001.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511115000/_SUCCESS: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511115000/_SUCCESS -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511115000/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511115000/part-00000 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511115000/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511115000/part-00001 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511120000/._SUCCESS.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511120000/.part-00000.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511120000/.part-00001.crc: -------------------------------------------------------------------------------- 1 | crc -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511120000/_SUCCESS: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511120000/_SUCCESS -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511120000/part-00000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511120000/part-00000 -------------------------------------------------------------------------------- /2_basics/output/Counts-1518511120000/part-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/2_basics/output/Counts-1518511120000/part-00001 -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519965875000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519965875000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519965876000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519965876000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519965877000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519965877000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519965878000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519965878000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519965879000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519965879000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519965880000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519965880000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519965881000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519965881000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519965882000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519965882000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519965883000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519965883000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519965884000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519965884000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519968021000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519968021000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519968021000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519968021000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519968022000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519968022000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519968022000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519968022000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519968023000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519968023000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519968023000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519968023000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519968024000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519968024000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519968024000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519968024000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519968025000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519968025000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519968025000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519968025000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519978126000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519978126000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519978126000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519978126000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519978127000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519978127000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519978127000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519978127000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519978128000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519978128000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519978128000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519978128000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519978129000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519978129000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519978129000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519978129000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519978130000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519978130000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519978130000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519978130000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519979383000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519979383000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519979384000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519979384000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519979384000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519979384000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519979385000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519979385000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519979385000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519979385000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519979386000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519979386000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519979386000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519979386000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519979387000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519979387000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519979387000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519979387000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519979388000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519979388000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983050000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983050000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983050000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983050000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983052000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983052000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983054000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983054000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983054000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983054000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983056000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983056000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983056000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983056000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983068000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983068000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983070000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983070000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983070000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983070000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983072000.bk.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983072000.bk.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983072000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983072000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983074000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983074000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983076000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983076000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983078000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983078000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983170000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983170000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983172000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983172000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983174000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983174000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983176000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983176000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983178000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983178000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983180000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983180000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983182000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983182000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983184000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983184000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983186000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983186000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/.checkpoint-1519983188000.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/.checkpoint-1519983188000.crc -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519968021000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519968021000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519968021000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519968021000.bk -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519968022000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519968022000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519968022000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519968022000.bk -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519968023000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519968023000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519968023000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519968023000.bk -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519968024000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519968024000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519968024000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519968024000.bk -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519968025000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519968025000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519968025000.bk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519968025000.bk -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519983170000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519983170000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519983172000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519983172000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519983174000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519983174000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519983176000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519983176000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519983178000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519983178000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519983180000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519983180000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519983182000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519983182000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519983184000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519983184000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519983186000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519983186000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/checkpoint-1519983188000: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/checkpoint-1519983188000 -------------------------------------------------------------------------------- /3_advanced/checkpoint/receivedBlockMetadata/log-1519968004287-1519968064287: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/checkpoint/receivedBlockMetadata/log-1519968004287-1519968064287 -------------------------------------------------------------------------------- /3_advanced/data/ip_file1.txt: -------------------------------------------------------------------------------- 1 | 1.2.3.4 2 | 10.20.30.40 3 | -------------------------------------------------------------------------------- /3_advanced/data/ip_file2.txt: -------------------------------------------------------------------------------- 1 | 5.6.7.8 2 | 50.60.70.80 3 | -------------------------------------------------------------------------------- /3_advanced/data/prime_nums.text: -------------------------------------------------------------------------------- 1 | 2 3 5 7 11 13 17 19 23 29 2 | 31 37 41 43 47 53 59 61 67 71 3 | 73 79 83 89 97 101 103 107 109 113 4 | 127 131 137 139 149 151 157 163 167 173 5 | 179 181 191 193 197 199 211 223 227 229 6 | 233 239 241 251 257 263 269 271 277 281 7 | 283 293 307 311 313 317 331 337 347 349 8 | 353 359 367 373 379 383 389 397 401 409 9 | 419 421 431 433 439 443 449 457 461 463 10 | 467 479 487 491 499 503 509 521 523 541 11 | -------------------------------------------------------------------------------- /3_advanced/output/accum.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/3_advanced/output/accum.txt -------------------------------------------------------------------------------- /4_morestream/.ipynb_checkpoints/1_Integration with Kafka Demo take1-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Integration with Kafka Demo" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Performance Tuning\n", 15 | "Reference: https://spark.apache.org/docs/latest/streaming-programming-guide.html#performance-tuning\n", 16 | "Reducing the Batch Processing Times\n", 17 | "Level of Parallelism in Data Receiving\n", 18 | "Level of Parallelism in Data Processing\n", 19 | "Data Serialization\n", 20 | "Task Launching Overheads\n", 21 | "Setting the Right Batch Interval\n", 22 | "Memory Tuning\n", 23 | "Integration with Kafka\n", 24 | "Introduction to Kafka\n", 25 | "Why integrate with Kafka\n", 26 | "DEMO: Demo\n" 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "metadata": {}, 32 | "source": [ 33 | "### Demo" 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": null, 39 | "metadata": {}, 40 | "outputs": [], 41 | "source": [ 42 | "import findspark\n", 43 | "# TODO: your path will likely not have 'matthew' in it. Change it to reflect your path.\n", 44 | "findspark.init('/home/matthew/spark-2.1.0-bin-hadoop2.7')" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": { 51 | "collapsed": true 52 | }, 53 | "outputs": [], 54 | "source": [ 55 | "import sys\n", 56 | "from pyspark import SparkContext\n", 57 | "from pyspark.streaming import StreamingContext\n", 58 | "from pyspark.streaming.kafka import KafkaUtils\n", 59 | "\n", 60 | "sc = SparkContext(appName=\"PythonStreamingKafkaWordCount\")\n", 61 | "ssc = StreamingContext(sc, 1)\n", 62 | "zkQuorum, topic = sys.argv[1:]\n", 63 | "kvs = KafkaUtils.createStream(ssc, zkQuorum, \"spark-streaming-consumer\", {topic: 1})\n", 64 | "lines = kvs.map(lambda x: x[1])\n", 65 | "counts = lines.flatMap(lambda line: line.split(\" \")).map(lambda word: (word, 1)).reduceByKey(lambda a, b: a+b)\n", 66 | "counts.pprint()\n", 67 | "\n", 68 | "ssc.start()\n", 69 | "ssc.awaitTermination()" 70 | ] 71 | }, 72 | { 73 | "cell_type": "markdown", 74 | "metadata": {}, 75 | "source": [ 76 | "## References\n", 77 | "1. https://spark.apache.org/docs/latest/streaming-kafka-integration.html\n", 78 | "2. https://spark.apache.org/docs/latest/streaming-programming-guide.html#performance-tuning\n", 79 | "3. https://apache.googlesource.com/spark/+/master/examples/src/main/python/streaming/kafka_wordcount.py" 80 | ] 81 | }, 82 | { 83 | "cell_type": "markdown", 84 | "metadata": {}, 85 | "source": [ 86 | " " 87 | ] 88 | } 89 | ], 90 | "metadata": { 91 | "kernelspec": { 92 | "display_name": "Python 3", 93 | "language": "python", 94 | "name": "python3" 95 | }, 96 | "language_info": { 97 | "codemirror_mode": { 98 | "name": "ipython", 99 | "version": 3 100 | }, 101 | "file_extension": ".py", 102 | "mimetype": "text/x-python", 103 | "name": "python", 104 | "nbconvert_exporter": "python", 105 | "pygments_lexer": "ipython3", 106 | "version": "3.5.2" 107 | } 108 | }, 109 | "nbformat": 4, 110 | "nbformat_minor": 2 111 | } 112 | -------------------------------------------------------------------------------- /4_morestream/.ipynb_checkpoints/1_Integration with Kafka Demo-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Integration with Kafka Demo" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Performance Tuning\n", 15 | "Reference: https://spark.apache.org/docs/latest/streaming-programming-guide.html#performance-tuning\n", 16 | "Reducing the Batch Processing Times\n", 17 | "Level of Parallelism in Data Receiving\n", 18 | "Level of Parallelism in Data Processing\n", 19 | "Data Serialization\n", 20 | "Task Launching Overheads\n", 21 | "Setting the Right Batch Interval\n", 22 | "Memory Tuning\n", 23 | "Integration with Kafka\n", 24 | "Introduction to Kafka\n", 25 | "Why integrate with Kafka\n", 26 | "DEMO: Demo\n" 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "metadata": {}, 32 | "source": [ 33 | "### Demo" 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": null, 39 | "metadata": { 40 | "collapsed": true 41 | }, 42 | "outputs": [], 43 | "source": [ 44 | "import sys\n", 45 | "from pyspark import SparkContext\n", 46 | "from pyspark.streaming import StreamingContext\n", 47 | "from pyspark.streaming.kafka import KafkaUtils\n", 48 | "\n", 49 | "sc = SparkContext(appName=\"PythonStreamingKafkaWordCount\")\n", 50 | "ssc = StreamingContext(sc, 1)\n", 51 | "zkQuorum, topic = sys.argv[1:]\n", 52 | "kvs = KafkaUtils.createStream(ssc, zkQuorum, \"spark-streaming-consumer\", {topic: 1})\n", 53 | "lines = kvs.map(lambda x: x[1])\n", 54 | "counts = lines.flatMap(lambda line: line.split(\" \")).map(lambda word: (word, 1)).reduceByKey(lambda a, b: a+b)\n", 55 | "counts.pprint()\n", 56 | "\n", 57 | "ssc.start()\n", 58 | "ssc.awaitTermination()" 59 | ] 60 | }, 61 | { 62 | "cell_type": "markdown", 63 | "metadata": {}, 64 | "source": [ 65 | "## References\n", 66 | "1. https://spark.apache.org/docs/latest/streaming-kafka-integration.html\n", 67 | "2. https://spark.apache.org/docs/latest/streaming-programming-guide.html#performance-tuning\n", 68 | "3. https://apache.googlesource.com/spark/+/master/examples/src/main/python/streaming/kafka_wordcount.py" 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": {}, 74 | "source": [ 75 | " " 76 | ] 77 | } 78 | ], 79 | "metadata": { 80 | "kernelspec": { 81 | "display_name": "Python 3", 82 | "language": "python", 83 | "name": "python3" 84 | }, 85 | "language_info": { 86 | "codemirror_mode": { 87 | "name": "ipython", 88 | "version": 3 89 | }, 90 | "file_extension": ".py", 91 | "mimetype": "text/x-python", 92 | "name": "python", 93 | "nbconvert_exporter": "python", 94 | "pygments_lexer": "ipython3", 95 | "version": "3.6.3" 96 | } 97 | }, 98 | "nbformat": 4, 99 | "nbformat_minor": 2 100 | } 101 | -------------------------------------------------------------------------------- /4_morestream/.ipynb_checkpoints/Integration with Kafka Test-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Integration with Kafka Test" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Performance Tuning\n", 15 | "Reference: https://spark.apache.org/docs/latest/streaming-programming-guide.html#performance-tuning\n", 16 | "Reducing the Batch Processing Times\n", 17 | "Level of Parallelism in Data Receiving\n", 18 | "Level of Parallelism in Data Processing\n", 19 | "Data Serialization\n", 20 | "Task Launching Overheads\n", 21 | "Setting the Right Batch Interval\n", 22 | "Memory Tuning\n", 23 | "Integration with Kafka\n", 24 | "Introduction to Kafka\n", 25 | "Why integrate with Kafka\n", 26 | "DEMO: Demo\n" 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "metadata": {}, 32 | "source": [ 33 | "### Demo" 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": null, 39 | "metadata": {}, 40 | "outputs": [], 41 | "source": [ 42 | "import findspark\n", 43 | "# TODO: your path will likely not have 'matthew' in it. Change it to reflect your path.\n", 44 | "findspark.init('/home/matthew/spark-2.1.0-bin-hadoop2.7')" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": { 51 | "collapsed": true 52 | }, 53 | "outputs": [], 54 | "source": [ 55 | "import sys\n", 56 | "from pyspark import SparkContext\n", 57 | "from pyspark.streaming import StreamingContext\n", 58 | "from pyspark.streaming.kafka import KafkaUtils\n", 59 | "\n", 60 | "sc = SparkContext(appName=\"PythonStreamingKafkaWordCount\")\n", 61 | "ssc = StreamingContext(sc, 1)\n", 62 | "zkQuorum, topic = sys.argv[1:]\n", 63 | "kvs = KafkaUtils.createStream(ssc, zkQuorum, \"spark-streaming-consumer\", {topic: 1})\n", 64 | "lines = kvs.map(lambda x: x[1])\n", 65 | "counts = lines.flatMap(lambda line: line.split(\" \")).map(lambda word: (word, 1)).reduceByKey(lambda a, b: a+b)\n", 66 | "counts.pprint()\n", 67 | "\n", 68 | "ssc.start()\n", 69 | "ssc.awaitTermination()" 70 | ] 71 | }, 72 | { 73 | "cell_type": "markdown", 74 | "metadata": {}, 75 | "source": [ 76 | "## References\n", 77 | "1. https://spark.apache.org/docs/latest/streaming-kafka-integration.html\n", 78 | "2. https://spark.apache.org/docs/latest/streaming-programming-guide.html#performance-tuning\n", 79 | "3. https://apache.googlesource.com/spark/+/master/examples/src/main/python/streaming/kafka_wordcount.py" 80 | ] 81 | }, 82 | { 83 | "cell_type": "markdown", 84 | "metadata": {}, 85 | "source": [ 86 | " " 87 | ] 88 | } 89 | ], 90 | "metadata": { 91 | "kernelspec": { 92 | "display_name": "Python 3", 93 | "language": "python", 94 | "name": "python3" 95 | }, 96 | "language_info": { 97 | "codemirror_mode": { 98 | "name": "ipython", 99 | "version": 3 100 | }, 101 | "file_extension": ".py", 102 | "mimetype": "text/x-python", 103 | "name": "python", 104 | "nbconvert_exporter": "python", 105 | "pygments_lexer": "ipython3", 106 | "version": "3.5.2" 107 | } 108 | }, 109 | "nbformat": 4, 110 | "nbformat_minor": 2 111 | } 112 | -------------------------------------------------------------------------------- /4_morestream/1_Integration_with_Kafka_Demo.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # # Integration with Kafka Demo 5 | 6 | # Donwload this notebook as a .py file 7 | # 8 | # For setting up the stream, First run the following scripts to create the zookeeper server: 9 | # ``` 10 | # ~/kafka_2.11-0.11.0.0/bin/zookeeper-server-start.sh ~/kafka_2.11-0.11.0.0/config/zookeeper.properties 11 | # ``` 12 | # Then setup the kafka server with this shell command: 13 | # ``` 14 | # ~/kafka_2.11-0.11.0.0/bin/kafka-server-start.sh ~/kafka_2.11-0.11.0.0/config/server.properties 15 | # ``` 16 | # to create the topic, use the following commands 17 | # ``` 18 | # ~/kafka_2.11-0.11.0.0/bin/kafka-topics.sh --create --zookeeper localhost:2181 --topic pyspark-kafka-demo --replication-factor 1 --partitions 3 19 | # ``` 20 | # 21 | # Donwload this notebook as a .py file and run the following: 22 | # ``` 23 | # python3 pyspark-streaming/4_morestream/1_Integration_with_Kafka_Demo.py 24 | # ``` 25 | # Finally, start the producer: 26 | # ``` 27 | # ~/kafka_2.11-0.11.0.0/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic pyspark-kafka-demo 28 | # ``` 29 | # 30 | # 31 | 32 | # ### Demo 33 | 34 | # In[ ]: 35 | 36 | 37 | import findspark 38 | # TODO: your path will likely not have 'matthew' in it. Change it to reflect your path. 39 | findspark.init('/home/matthew/spark-2.3.0-bin-hadoop2.7') 40 | 41 | import os 42 | os.environ['PYSPARK_SUBMIT_ARGS'] = '--packages org.apache.spark:spark-streaming-kafka-0-8_2.11:2.3.0 pyspark-shell' 43 | 44 | import sys 45 | import time 46 | from pyspark import SparkContext, SparkConf 47 | from pyspark.streaming import StreamingContext 48 | from pyspark.streaming.kafka import KafkaUtils 49 | 50 | 51 | n_secs = 1 52 | topic = "pyspark-kafka-demo" 53 | 54 | conf = SparkConf().setAppName("KafkaStreamProcessor").setMaster("local[*]") 55 | sc = SparkContext(conf=conf) 56 | sc.setLogLevel("WARN") 57 | ssc = StreamingContext(sc, n_secs) 58 | 59 | kafkaStream = KafkaUtils.createDirectStream(ssc, [topic], { 60 | 'bootstrap.servers':'localhost:9092', 61 | 'group.id':'video-group', 62 | 'fetch.message.max.bytes':'15728640', 63 | 'auto.offset.reset':'largest'}) 64 | # Group ID is completely arbitrary 65 | 66 | lines = kafkaStream.map(lambda x: x[1]) 67 | counts = lines.flatMap(lambda line: line.split(" ")).map(lambda word: (word, 1)).reduceByKey(lambda a, b: a+b) 68 | counts.pprint() 69 | 70 | ssc.start() 71 | time.sleep(600) # Run stream for 10 minutes just in case no detection of producer 72 | # ssc.awaitTermination() 73 | ssc.stop(stopSparkContext=True,stopGraceFully=True) 74 | 75 | 76 | # ## References 77 | # 1. https://spark.apache.org/docs/latest/streaming-kafka-integration.html 78 | # 2. https://spark.apache.org/docs/latest/streaming-programming-guide.html#performance-tuning 79 | # 3. https://apache.googlesource.com/spark/+/master/examples/src/main/python/streaming/kafka_wordcount.py 80 | 81 | # 82 | -------------------------------------------------------------------------------- /4_morestream/2_Integration_with_Kinesis_Demo.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # # Integration with Kinesis Demo 5 | 6 | # More information here: https://docs.aws.amazon.com/streams/latest/dev/before-you-begin.html 7 | # 8 | # and here: https://spark.apache.org/docs/latest/streaming-kinesis-integration.html 9 | 10 | # Kinesis Data Generator: https://awslabs.github.io/amazon-kinesis-data-generator/ 11 | # 12 | # After going through the setup, download this file as a .py file, go into the terminal, and run the following command: 13 | # ``` 14 | # python3 pyspark-streaming/4_morestream/2_Integration_with_Kinesis_Demo.py 15 | # ``` 16 | # 17 | 18 | # ### Demo 19 | 20 | # In[ ]: 21 | 22 | 23 | import findspark 24 | # TODO: your path will likely not have 'matthew' in it. Change it to reflect your path. 25 | findspark.init('/home/matthew/spark-2.3.0-bin-hadoop2.7') 26 | 27 | import os 28 | os.environ['PYSPARK_SUBMIT_ARGS'] = '--packages org.apache.spark:spark-streaming-kinesis-asl_2.11:2.3.0 pyspark-shell' 29 | 30 | 31 | import sys 32 | import json 33 | import time 34 | from pyspark import SparkContext 35 | from pyspark.streaming import StreamingContext 36 | from pyspark.streaming.kinesis import KinesisUtils, InitialPositionInStream 37 | 38 | appName="PythonKinesisApp" 39 | sc = SparkContext(appName=appName) 40 | ssc = StreamingContext(sc, 1) 41 | 42 | 43 | streamName = 'DemoStream' 44 | endpointUrl = 'https://kinesis.us-east-1.amazonaws.com' 45 | regionName = 'us-east-1' 46 | AWS_ACCESS_KEY_ID = '' 47 | SECRET_ACCESS_KEY = '' 48 | checkpointInterval = 5 49 | kinesisstream = KinesisUtils.createStream(ssc, appName, 50 | streamName, endpointUrl, regionName, 51 | InitialPositionInStream.LATEST, 52 | checkpointInterval, 53 | awsAccessKeyId=AWS_ACCESS_KEY_ID, 54 | awsSecretKey=SECRET_ACCESS_KEY) 55 | lines = kinesisstream.map(lambda x: x[1]) 56 | counts = lines.flatMap(lambda line: line.split(" ")).map(lambda word: (word, 1)).reduceByKey(lambda a, b: a+b) 57 | counts.pprint() 58 | 59 | ssc.start() 60 | time.sleep(600) # Run stream for 10 minutes just in case no detection of producer 61 | # ssc.awaitTermination() 62 | ssc.stop(stopSparkContext=True,stopGraceFully=True) 63 | 64 | 65 | # ## References 66 | # 1. https://spark.apache.org/docs/latest/streaming-kinesis-integration.html 67 | # 2. https://spark.apache.org/docs/latest/streaming-programming-guide.html#performance-tuning 68 | 69 | # 70 | -------------------------------------------------------------------------------- /4_morestream/amazon-kinesis-client-1.7.4.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/4_morestream/amazon-kinesis-client-1.7.4.jar -------------------------------------------------------------------------------- /4_morestream/data/prime_nums.text: -------------------------------------------------------------------------------- 1 | 2 3 5 7 11 13 17 19 23 29 2 | 31 37 41 43 47 53 59 61 67 71 3 | 73 79 83 89 97 101 103 107 109 113 4 | 127 131 137 139 149 151 157 163 167 173 5 | 179 181 191 193 197 199 211 223 227 229 6 | 233 239 241 251 257 263 269 271 277 281 7 | 283 293 307 311 313 317 331 337 347 349 8 | 353 359 367 373 379 383 389 397 401 409 9 | 419 421 431 433 439 443 449 457 461 463 10 | 467 479 487 491 499 503 509 521 523 541 11 | -------------------------------------------------------------------------------- /4_morestream/kinesisExercise.py: -------------------------------------------------------------------------------- 1 | # This script will read 100 lines of data from PARQUET datasource stored in S3 and stream them into an AWS Kinesis stream. 2 | 3 | from __future__ import print_function 4 | import logging 5 | import json 6 | from pyspark import SparkContext 7 | from pyspark.sql import SparkSession 8 | from boto import kinesis 9 | 10 | logger = logging.getLogger('py4j') 11 | 12 | kinesisStreamName='stream' 13 | 14 | def write_partition(partition): 15 | # Access the Kinesis client object 16 | kinesisClient = kinesis.connect_to_region("us-east-1") 17 | 18 | # Iterate over rows 19 | for row in partition: 20 | # Send the row as a JSON string into the Kinesis stream 21 | kinesisClient.put_record(kinesisStreamName, json.dumps(row),"partitionKey") 22 | 23 | if __name__ == "__main__": 24 | appName='Send2KinesisStream' 25 | 26 | sc = SparkContext() 27 | 28 | # Connect to the hive context of our spark context. 29 | sqlContext = SparkSession.builder.enableHiveSupport().getOrCreate(); 30 | 31 | # Define an external hive table from the PARQUET files stored in S3 to be used as source to read data from. 32 | sqlContext.sql("CREATE EXTERNAL TABLE IF NOT EXISTS yellow_trips_parquet(" + 33 | "pickup_timestamp BIGINT, vendor_id STRING, rate_code STRING, payment_type STRING) " + 34 | #"ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' " + 35 | "STORED AS parquet " + 36 | "LOCATION 's3://awskrug-jjy-athena/parquet/'") 37 | 38 | # Create an RDD containing 100 items from the external table defined above 39 | #lines=sqlContext.sql("select pickup_timestamp, vendor_id, rate_code, payment_type from yellow_trips_parquet limit 300") 40 | lines=sqlContext.sql("select * from yellow_trips_parquet limit 300") 41 | 42 | # Iterate over data 43 | lines.foreachPartition(write_partition) -------------------------------------------------------------------------------- /5_structured/.ipynb_checkpoints/Operations on Streaming Dataframes-Datasets Demo-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Operations on Streaming Dataframes/Datasets Demo" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Operations on streaming DataFrames/Datasets\n", 15 | "https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#operations-on-streaming-dataframesdatasets\n", 16 | "DEMO: DO a demo:\n", 17 | "EXERCISE: Prepare an excise " 18 | ] 19 | }, 20 | { 21 | "cell_type": "markdown", 22 | "metadata": {}, 23 | "source": [ 24 | "### Demo" 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": null, 30 | "metadata": { 31 | "collapsed": true 32 | }, 33 | "outputs": [], 34 | "source": [] 35 | } 36 | ], 37 | "metadata": { 38 | "kernelspec": { 39 | "display_name": "Python 3", 40 | "language": "python", 41 | "name": "python3" 42 | }, 43 | "language_info": { 44 | "codemirror_mode": { 45 | "name": "ipython", 46 | "version": 3 47 | }, 48 | "file_extension": ".py", 49 | "mimetype": "text/x-python", 50 | "name": "python", 51 | "nbconvert_exporter": "python", 52 | "pygments_lexer": "ipython3", 53 | "version": "3.6.1" 54 | } 55 | }, 56 | "nbformat": 4, 57 | "nbformat_minor": 2 58 | } 59 | -------------------------------------------------------------------------------- /5_structured/.ipynb_checkpoints/Operations on Streaming Dataframes-Datasets Exercise-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Operations on Streaming Dataframes/Datasets Exercise" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Operations on streaming DataFrames/Datasets\n", 15 | "https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#operations-on-streaming-dataframesdatasets\n", 16 | "DEMO: DO a demo:\n", 17 | "EXERCISE: Prepare an excise \n" 18 | ] 19 | }, 20 | { 21 | "cell_type": "markdown", 22 | "metadata": {}, 23 | "source": [ 24 | "### Exercise" 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": null, 30 | "metadata": { 31 | "collapsed": true 32 | }, 33 | "outputs": [], 34 | "source": [] 35 | } 36 | ], 37 | "metadata": { 38 | "kernelspec": { 39 | "display_name": "Python 3", 40 | "language": "python", 41 | "name": "python3" 42 | }, 43 | "language_info": { 44 | "codemirror_mode": { 45 | "name": "ipython", 46 | "version": 3 47 | }, 48 | "file_extension": ".py", 49 | "mimetype": "text/x-python", 50 | "name": "python", 51 | "nbconvert_exporter": "python", 52 | "pygments_lexer": "ipython3", 53 | "version": "3.6.1" 54 | } 55 | }, 56 | "nbformat": 4, 57 | "nbformat_minor": 2 58 | } 59 | -------------------------------------------------------------------------------- /5_structured/.ipynb_checkpoints/Structured Streaming Demo-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Structured Streaming Demo" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Introduction about Structured Streaming\n", 15 | "* Overview of Structured Streaming\n", 16 | "* [The Benefit of structured streaming](https://drive.google.com/file/d/0Bym8DZ5hyGifM2VOYlJVQ3NwaTg/view?usp=drive_web)\n", 17 | "* Basic Concepts about Spark streaming\n", 18 | "* https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#basic-concepts\n", 19 | "* DEMO: A quick demo about an structured streaming example.\n", 20 | "* https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#quick-example\n" 21 | ] 22 | }, 23 | { 24 | "cell_type": "markdown", 25 | "metadata": {}, 26 | "source": [ 27 | "### Demo" 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": null, 33 | "metadata": { 34 | "collapsed": true 35 | }, 36 | "outputs": [], 37 | "source": [] 38 | } 39 | ], 40 | "metadata": { 41 | "kernelspec": { 42 | "display_name": "Python 3", 43 | "language": "python", 44 | "name": "python3" 45 | }, 46 | "language_info": { 47 | "codemirror_mode": { 48 | "name": "ipython", 49 | "version": 3 50 | }, 51 | "file_extension": ".py", 52 | "mimetype": "text/x-python", 53 | "name": "python", 54 | "nbconvert_exporter": "python", 55 | "pygments_lexer": "ipython3", 56 | "version": "3.6.1" 57 | } 58 | }, 59 | "nbformat": 4, 60 | "nbformat_minor": 2 61 | } 62 | -------------------------------------------------------------------------------- /5_structured/README.md: -------------------------------------------------------------------------------- 1 | # Spark Structured Streaming with a Static Input 2 | An application that takes as input the list of twitter user IDs and every 5 seconds, it emits the number of tweet actions of a user if it is present in the input list. 3 | An inner join on the staticDF consisting of twitterIDs and the input stream DF was performed and grouped on the userIDs. 4 | 5 | ## Dataset 6 | This application was developed to analyze the Higgs Twitter Dataset. The Higgs dataset has been built after monitoring the spreading processes on Twitter before, during and after the announcement of the discovery of a new particle with the features of the Higgs boson. Each row in this dataset is of the format where interactions can be retweets (RT), mention (MT) and reply (RE). 7 | We have split the dataset into a number of small files so that we can use the dataset to emulate streaming data. Download the split dataset onto your master VM. 8 | 9 | ## streamer.sh 10 | This script emulates twitter stream by doing the following : 11 | * Copies the entire split dataset to the HDFS. This would be the staging directory. 12 | * Creates a monitoring directory on the HDFS that this application listens to. This would be the directory this streaming application is listening to. 13 | * Periodically, moves the split dataset files from the staging directory to the monitoring directory using the hadoop fs -mv command. 14 | 15 | ## Usage 16 | Submit this spark job by using the following command : 17 | 18 | `spark-submit --verbose tweetactions.py ` -------------------------------------------------------------------------------- /5_structured/applicationHistory/.metadata.crc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/5_structured/applicationHistory/.metadata.crc -------------------------------------------------------------------------------- /5_structured/applicationHistory/commits/0: -------------------------------------------------------------------------------- 1 | v1 2 | {} -------------------------------------------------------------------------------- /5_structured/applicationHistory/metadata: -------------------------------------------------------------------------------- 1 | {"id":"b4f1d3c7-b6a7-43eb-9d8e-366a0e859aa0"} -------------------------------------------------------------------------------- /5_structured/applicationHistory/offsets/0: -------------------------------------------------------------------------------- 1 | v1 2 | {"batchWatermarkMs":0,"batchTimestampMs":1522025410224,"conf":{"spark.sql.shuffle.partitions":"200","spark.sql.streaming.stateStore.providerClass":"org.apache.spark.sql.execution.streaming.state.HDFSBackedStateStoreProvider"}} 3 | {"logOffset":0} -------------------------------------------------------------------------------- /5_structured/applicationHistory/sources/0/0: -------------------------------------------------------------------------------- 1 | v1 2 | {"path":"file:///home/matthew/pyspark-streaming/5_structured/monitoring_data/5.csv","timestamp":1515756794000,"batchId":0} 3 | {"path":"file:///home/matthew/pyspark-streaming/5_structured/monitoring_data/4.csv","timestamp":1515756794000,"batchId":0} 4 | {"path":"file:///home/matthew/pyspark-streaming/5_structured/monitoring_data/3.csv","timestamp":1515756794000,"batchId":0} 5 | {"path":"file:///home/matthew/pyspark-streaming/5_structured/monitoring_data/8.csv","timestamp":1515756794000,"batchId":0} 6 | {"path":"file:///home/matthew/pyspark-streaming/5_structured/monitoring_data/6.csv","timestamp":1515756794000,"batchId":0} 7 | {"path":"file:///home/matthew/pyspark-streaming/5_structured/monitoring_data/7.csv","timestamp":1515756794000,"batchId":0} 8 | {"path":"file:///home/matthew/pyspark-streaming/5_structured/monitoring_data/1.csv","timestamp":1515756794000,"batchId":0} 9 | {"path":"file:///home/matthew/pyspark-streaming/5_structured/monitoring_data/2.csv","timestamp":1515756794000,"batchId":0} 10 | {"path":"file:///home/matthew/pyspark-streaming/5_structured/monitoring_data/9.csv","timestamp":1515756794000,"batchId":0} -------------------------------------------------------------------------------- /5_structured/split-dataset/1127.csv: -------------------------------------------------------------------------------- 1 | 338046,122469,2012-07-07 05:04:16,RT 2 | 90121,1852,2012-07-04 02:21:07,RT 3 | 313219,326,2012-07-04 14:56:00,RT 4 | 396613,26180,2012-07-04 11:53:15,RT 5 | 175410,519,2012-07-03 06:58:13,MT 6 | 175117,175117,2012-07-04 16:59:28,MT 7 | 240926,27861,2012-07-05 23:18:47,MT 8 | 267163,88,2012-07-04 01:51:25,RE 9 | 51513,37799,2012-07-05 03:04:45,RE 10 | 354151,37561,2012-07-04 06:38:31,RT 11 | 187214,31650,2012-07-04 23:47:47,RT 12 | 139188,88,2012-07-05 06:27:46,MT 13 | 165520,12801,2012-07-05 13:03:23,RT 14 | 359974,13833,2012-07-04 04:44:31,RT 15 | 89351,4393,2012-07-04 11:36:51,RT 16 | 110141,86525,2012-07-07 09:49:07,RT 17 | 372344,59907,2012-07-04 00:58:10,RT 18 | 175737,960,2012-07-05 08:47:22,RT 19 | 85795,88,2012-07-03 23:55:44,MT 20 | 77496,4720,2012-07-04 01:02:15,MT 21 | 99783,13758,2012-07-04 21:52:30,RT 22 | 133200,233354,2012-07-03 18:29:31,MT 23 | 300528,88,2012-07-05 07:46:02,MT 24 | 365649,14615,2012-07-04 09:05:32,RT 25 | 275005,11991,2012-07-04 00:54:47,MT 26 | 296593,3393,2012-07-04 17:44:05,RT 27 | 21802,43580,2012-07-04 01:40:36,RT 28 | 378950,58028,2012-07-04 06:40:49,RT 29 | 1902,308,2012-07-02 14:00:57,MT 30 | 249903,468,2012-07-04 20:42:44,RT 31 | 45911,3998,2012-07-04 00:39:04,RT 32 | 110468,14093,2012-07-05 05:45:20,RT 33 | 313711,110868,2012-07-04 04:57:04,RT 34 | 5461,920,2012-07-05 00:40:27,RT 35 | 382502,138,2012-07-04 07:13:11,RT 36 | 218885,677,2012-07-04 02:01:03,RT 37 | 224067,4821,2012-07-04 02:35:33,RT 38 | 265704,265704,2012-07-04 10:57:21,MT 39 | 412144,30231,2012-07-04 03:09:07,MT 40 | 73863,6897,2012-07-04 03:16:45,RT 41 | 270830,16,2012-07-05 11:13:21,RT 42 | 3109,542,2012-07-06 16:47:54,MT 43 | 128459,126341,2012-07-02 17:32:16,MT 44 | 17607,3571,2012-07-04 12:39:32,RT 45 | 101136,7533,2012-07-03 23:58:40,MT 46 | 142723,15158,2012-07-06 06:24:39,MT 47 | 3290,45823,2012-07-04 16:24:58,MT 48 | 22410,16715,2012-07-04 04:25:11,RT 49 | 114104,63931,2012-07-01 15:20:50,RT 50 | 327456,689,2012-07-04 01:30:17,RT 51 | 224022,319889,2012-07-04 12:18:40,RE 52 | 307830,23727,2012-07-04 04:15:16,RT 53 | 219452,3808,2012-07-04 02:34:05,RT 54 | 190209,5018,2012-07-04 05:52:04,RT 55 | 322547,73343,2012-07-04 02:48:10,RT 56 | 322893,67549,2012-07-04 19:28:16,RT 57 | 303777,1988,2012-07-04 02:14:24,RT 58 | 4368,89805,2012-07-04 00:57:33,RT 59 | 337867,30222,2012-07-04 11:11:51,MT 60 | 55786,5508,2012-07-04 07:05:58,RT 61 | 249768,211,2012-07-04 22:12:05,RT 62 | 50151,13820,2012-07-02 14:24:50,MT 63 | 391398,370,2012-07-04 06:08:17,MT 64 | 107065,88696,2012-07-05 07:12:16,MT 65 | 7468,502,2012-07-05 09:38:18,MT 66 | 443110,373106,2012-07-06 23:08:16,RT 67 | 177472,220,2012-07-04 08:36:09,RT 68 | 180220,2014,2012-07-05 13:11:47,MT 69 | 295879,5121,2012-07-04 05:56:46,MT 70 | -------------------------------------------------------------------------------- /5_structured/twitterIDs.csv: -------------------------------------------------------------------------------- 1 | 330734 2 | 223125 3 | 42318 4 | 38011 5 | 78359 6 | 4543 7 | 305032 8 | 69958 9 | 12345 10 | 67890 11 | -------------------------------------------------------------------------------- /6_finishing_up/1_Introductory Lecture.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/6_finishing_up/1_Introductory Lecture.pptx -------------------------------------------------------------------------------- /6_finishing_up/1_Introductory Lecture/Slide1.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/6_finishing_up/1_Introductory Lecture/Slide1.PNG -------------------------------------------------------------------------------- /6_finishing_up/1_Introductory Lecture/Slide2.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/6_finishing_up/1_Introductory Lecture/Slide2.PNG -------------------------------------------------------------------------------- /6_finishing_up/1_Introductory Lecture/Slide3.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/6_finishing_up/1_Introductory Lecture/Slide3.PNG -------------------------------------------------------------------------------- /6_finishing_up/1_Introductory Lecture/Slide4.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/6_finishing_up/1_Introductory Lecture/Slide4.PNG -------------------------------------------------------------------------------- /6_finishing_up/2_Promotional Video.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/6_finishing_up/2_Promotional Video.pptx -------------------------------------------------------------------------------- /6_finishing_up/3_Final Lecture.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/6_finishing_up/3_Final Lecture.pptx -------------------------------------------------------------------------------- /6_finishing_up/3_Final Lecture/Slide1.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/6_finishing_up/3_Final Lecture/Slide1.PNG -------------------------------------------------------------------------------- /6_finishing_up/3_Final Lecture/Slide2.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/6_finishing_up/3_Final Lecture/Slide2.PNG -------------------------------------------------------------------------------- /6_finishing_up/3_Final Lecture/Slide3.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/6_finishing_up/3_Final Lecture/Slide3.PNG -------------------------------------------------------------------------------- /6_finishing_up/3_Final Lecture/Slide4.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/6_finishing_up/3_Final Lecture/Slide4.PNG -------------------------------------------------------------------------------- /6_finishing_up/3_Final Lecture/Slide5.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jleetutorial/python-spark-streaming/c7f4e5d3c84e6606180c37a67247680f81fc0d74/6_finishing_up/3_Final Lecture/Slide5.PNG -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Python Spark Streaming 2 | 3 | ### Overview 4 | 5 | Project source code for James Lee's Aparch Spark with Python (Pyspark) course. 6 | 7 | ### Description 8 | 9 | Tools like spark are incredibly useful for processing data that is continuously appended. The python bindings for Pyspark not only allow you to do that, but also allow you to combine spark streaming with other Python tools for Data Science and Machine learning. This course goes through some of the basics of using Apache Spark, as well as more advanced concepts like accumulators, combining Pyspark with Apache Kafka, using Pyspark with AWS tools like Kinesis, streaming data from sources like Twitter, and how to get the most out of the Structured Streaming paradigm in the recently-released Spark 2.3.0. 10 | 11 | This course is a one-stop-shop for all your pyspark streaming education needs. 12 | 13 | 14 | ### What's in this Repo? 15 | 16 | In this repo are the notebooks, data files, exercise files, and everything else you need to learn how to use the streaming capabilities of Pyspark. 17 | 18 | ### More content like this 19 | 20 | Check out the full list of DevOps and Big Data courses that James and Tao teach [here](https://www.level-up.one/courses/) 21 | --------------------------------------------------------------------------------