├── .gitignore ├── .idea └── vcs.xml ├── .udacity-pa └── projects.py ├── CODEOWNERS ├── README.md ├── RNN_project.ipynb ├── datasets ├── apple_prices.csv ├── holmes.txt ├── monthly-milk-production-pounds-p.csv ├── my_test_output.txt ├── normalized_apple_prices.csv └── pg1661.txt ├── environment_setup_instructions.txt ├── images ├── RNN_graph_model.png ├── apple_RNN_prediction.png ├── dnn2rnn.png ├── dog_speech.png ├── sample_grid_a_square.png ├── spectrogram_creation.png ├── text_windowing.gif ├── text_windowing_training.gif ├── time_windowing.gif ├── timeseries_windowing_training.gif └── wright_bros_face_detect.png ├── model_weights ├── best_RNN_large_textdata_weights.hdf5 ├── best_RNN_small_textdata_weights.hdf5 └── best_RNN_weights.hdf5 ├── my_answers.py ├── my_test_output.txt ├── requirements.txt └── text_gen_output └── RNN_large_textdata_output.txt /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Jupyter Notebook 10 | .ipynb_checkpoints/ 11 | 12 | # dotenv 13 | .env 14 | 15 | # virtualenv 16 | .venv 17 | venv/ 18 | ENV/ 19 | -------------------------------------------------------------------------------- /.idea/vcs.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | -------------------------------------------------------------------------------- /.udacity-pa/projects.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import shutil 3 | import os 4 | import subprocess as sp 5 | from udacity_pa import udacity 6 | 7 | nanodegree = 'nd889' 8 | projects = ['rnn'] 9 | filenames = ['my_answers.py', 'RNN_project.ipynb', 'RNN_project.html'] 10 | 11 | def submit(args): 12 | 13 | sp.call(['jupyter', 'nbconvert', '--to', 'html', 'RNN_project.ipynb']) 14 | 15 | udacity.submit(nanodegree, projects[0], filenames, 16 | environment = args.environment, 17 | jwt_path = args.jwt_path) 18 | -------------------------------------------------------------------------------- /CODEOWNERS: -------------------------------------------------------------------------------- 1 | # This is a comment. 2 | # Each line is a file pattern followed by one or more owners. 3 | 4 | # These owners will be the default owners for everything in 5 | # the repo. 6 | * @cgearhart 7 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Recurrent Neural Networks course project: time series prediction and text generation 2 | 3 | ## Accelerating the Training Process 4 | 5 | If your code is taking too long to run, you will need to either reduce the complexity of your chosen RNN architecture or switch to running your code on a GPU. If you'd like to use a GPU, you have two options: 6 | 7 | #### Build your Own Deep Learning Workstation 8 | 9 | If you have access to a GPU, you should follow the Keras instructions for [running Keras on GPU](https://keras.io/getting-started/faq/#how-can-i-run-keras-on-gpu). 10 | 11 | #### Amazon Web Services 12 | 13 | Instead of a local GPU, you could use Amazon Web Services to launch an EC2 GPU instance. (This costs money.) 14 | 15 | 16 | ## Rubric items 17 | 18 | #### Files Submitted 19 | 20 | | Criteria | Meets Specifications | 21 | |:---------------------:|:---------------------------------------------------------:| 22 | | Submission Files | RNN_project.ipynb, my_answers.py --> both the completed notebook RNN_project.ipynb as well as all completed python functions requested in the main notebook RNN_project.ipynb (TODO items) should be copied into this python script and submitted for grading. | 23 | 24 | #### Step 1: Implement a function to window time series 25 | | Criteria | Meets Specifications | 26 | |:---------------------:|:---------------------------------------------------------:| 27 | | Window time series data. | The submission returns the proper windowed version of input time series of proper dimension listed in the notebook. | 28 | 29 | 30 | #### Step 2: Create a simple RNN model using keras to perform regression 31 | 32 | | Criteria | Meets Specifications | 33 | |:---------------------:|:---------------------------------------------------------:| 34 | | Build an RNN model to perform regression. | The submission constructs an RNN model in keras with LSTM module of dimension defined in the notebook. | 35 | 36 | 37 | #### Step 3: Clean up a large text corpus 38 | 39 | | Criteria | Meets Specifications | 40 | |:---------------------:|:---------------------------------------------------------:| 41 | | Find and remove all non-english or punctuation characters from input text data. The submission removes all non-english / non-punctuation characters. | 42 | 43 | 44 | #### Step 4: Implement a function to window a large text corpus 45 | 46 | | Criteria | Meets Specifications | 47 | |:---------------------:|:---------------------------------------------------------:| 48 | | Implement a function to window input text data| The submission returns the proper windowed version of input text of proper dimension listed in the notebook. | 49 | 50 | 51 | #### Step 5: Create a simple RNN model using keras to perform multiclass classification 52 | 53 | | Criteria | Meets Specifications | 54 | |:---------------------:|:---------------------------------------------------------:| 55 | | Build an RNN model to perform multiclass classification. | The submission constructs an RNN model in keras with LSTM module of dimension defined in the notebook. | 56 | 57 | 58 | #### Step 6: Generate text using a fully trained RNN model and a variety of input sequences 59 | | Criteria | Meets Specifications | 60 | |:---------------------:|:---------------------------------------------------------:| 61 | | Generate text using a trained RNN classifier. | The submission presents examples of generated text from a trained RNN module. The majority of this generated text should consist of real english words. | 62 | 63 | ## Submission 64 | Before submitting your solution to a reviewer, you are required to submit your project to Udacity's Project Assistant, which will provide some initial feedback. 65 | 66 | The setup is simple. If you have not installed the client tool already, then you may do so with the command `pip install udacity-pa`. 67 | 68 | To submit your code to the project assistant, run `udacity submit` from within the top-level directory of this project. You will be prompted for a username and password. If you login using google or facebook, visit [this link](https://project-assistant.udacity.com/auth_tokens/jwt_login) for alternate login instructions. 69 | 70 | This process will create a zipfile in your top-level directory named rnn-.zip. This is the file that you should submit to the Udacity reviews system. 71 | -------------------------------------------------------------------------------- /datasets/apple_prices.csv: -------------------------------------------------------------------------------- 1 | 590.140015 2 | 572.800049 3 | 525.339966 4 | 521.900024 5 | 528.02002 6 | 539.230042 7 | 538.420044 8 | 527.699951 9 | 528.360046 10 | 523.419983 11 | 523.150024 12 | 546.0 13 | 518.659973 14 | 502.610016 15 | 550.070007 16 | 540.98999 17 | 529.910034 18 | 537.450012 19 | 557.460022 20 | 568.0 21 | 555.02002 22 | 560.900024 23 | 558.0 24 | 521.02002 25 | 524.98999 26 | 519.98999 27 | 521.099976 28 | 529.039978 29 | 511.77002 30 | 489.830017 31 | 486.559967 32 | 477.25 33 | 496.100006 34 | 460.999969 35 | 505.000031 36 | 493.099976 37 | 500.75 38 | 504.339966 39 | 456.860016 40 | 464.690002 41 | 440.799988 42 | 429.459991 43 | 425.01001 44 | 420.109985 45 | 402.690002 46 | 407.399994 47 | 431.440002 48 | 444.72998 49 | 450.72998 50 | 449.900024 51 | 431.910004 52 | 451.509979 53 | 455.709961 54 | 420.450012 55 | 392.639984 56 | 427.0 57 | 424.849976 58 | 441.899994 59 | 464.690002 60 | 441.450012 61 | 429.75 62 | 427.799988 63 | 453.850006 64 | 461.100006 65 | 476.5 66 | 453.909973 67 | 437.829987 68 | 504.559998 69 | 502.680023 70 | 522.0 71 | 510.529968 72 | 520.350037 73 | 508.929993 74 | 525.0 75 | 593.650024 76 | 575.900024 77 | 540.710022 78 | 554.149963 79 | 583.52002 80 | 594.880005 81 | 612.419983 82 | 632.350037 83 | 646.880005 84 | 671.159973 85 | 686.859985 86 | 699.349976 87 | 680.450012 88 | 665.76001 89 | 679.98999 90 | 650.01001 91 | 623.390015 92 | 617.290039 93 | 590.920044 94 | 594.399963 95 | 605.119995 96 | 605.299988 97 | 584.730042 98 | 577.299988 99 | 570.959961 100 | 587.719971 101 | 561.5 102 | 570.900024 103 | 534.5 104 | 562.570007 105 | 561.5 106 | 597.799988 107 | 570.609985 108 | 610.059998 109 | 626.130005 110 | 601.830017 111 | 599.790039 112 | 598.369995 113 | 548.97998 114 | 545.420044 115 | 521.309998 116 | 506.880005 117 | 499.529999 118 | 458.380005 119 | 445.709991 120 | 422.669983 121 | 424.199982 122 | 425.5 123 | 409.399994 124 | 403.099976 125 | 382.470001 126 | 391.679993 127 | 393.48999 128 | 372.349976 129 | 370.400024 130 | 383.519989 131 | 399.910004 132 | 402.419983 133 | 396.179993 134 | 421.740021 135 | 379.090027 136 | 380.369995 137 | 399.859985 138 | 419.639984 -------------------------------------------------------------------------------- /datasets/monthly-milk-production-pounds-p.csv: -------------------------------------------------------------------------------- 1 | "Month","Monthly milk production: pounds per cow. Jan 62 ? Dec 75" 2 | "1962-01",589 3 | "1962-02",561 4 | "1962-03",640 5 | "1962-04",656 6 | "1962-05",727 7 | "1962-06",697 8 | "1962-07",640 9 | "1962-08",599 10 | "1962-09",568 11 | "1962-10",577 12 | "1962-11",553 13 | "1962-12",582 14 | "1963-01",600 15 | "1963-02",566 16 | "1963-03",653 17 | "1963-04",673 18 | "1963-05",742 19 | "1963-06",716 20 | "1963-07",660 21 | "1963-08",617 22 | "1963-09",583 23 | "1963-10",587 24 | "1963-11",565 25 | "1963-12",598 26 | "1964-01",628 27 | "1964-02",618 28 | "1964-03",688 29 | "1964-04",705 30 | "1964-05",770 31 | "1964-06",736 32 | "1964-07",678 33 | "1964-08",639 34 | "1964-09",604 35 | "1964-10",611 36 | "1964-11",594 37 | "1964-12",634 38 | "1965-01",658 39 | "1965-02",622 40 | "1965-03",709 41 | "1965-04",722 42 | "1965-05",782 43 | "1965-06",756 44 | "1965-07",702 45 | "1965-08",653 46 | "1965-09",615 47 | "1965-10",621 48 | "1965-11",602 49 | "1965-12",635 50 | "1966-01",677 51 | "1966-02",635 52 | "1966-03",736 53 | "1966-04",755 54 | "1966-05",811 55 | "1966-06",798 56 | "1966-07",735 57 | "1966-08",697 58 | "1966-09",661 59 | "1966-10",667 60 | "1966-11",645 61 | "1966-12",688 62 | "1967-01",713 63 | "1967-02",667 64 | "1967-03",762 65 | "1967-04",784 66 | "1967-05",837 67 | "1967-06",817 68 | "1967-07",767 69 | "1967-08",722 70 | "1967-09",681 71 | "1967-10",687 72 | "1967-11",660 73 | "1967-12",698 74 | "1968-01",717 75 | "1968-02",696 76 | "1968-03",775 77 | "1968-04",796 78 | "1968-05",858 79 | "1968-06",826 80 | "1968-07",783 81 | "1968-08",740 82 | "1968-09",701 83 | "1968-10",706 84 | "1968-11",677 85 | "1968-12",711 86 | "1969-01",734 87 | "1969-02",690 88 | "1969-03",785 89 | "1969-04",805 90 | "1969-05",871 91 | "1969-06",845 92 | "1969-07",801 93 | "1969-08",764 94 | "1969-09",725 95 | "1969-10",723 96 | "1969-11",690 97 | "1969-12",734 98 | "1970-01",750 99 | "1970-02",707 100 | "1970-03",807 101 | "1970-04",824 102 | "1970-05",886 103 | "1970-06",859 104 | "1970-07",819 105 | "1970-08",783 106 | "1970-09",740 107 | "1970-10",747 108 | "1970-11",711 109 | "1970-12",751 110 | "1971-01",804 111 | "1971-02",756 112 | "1971-03",860 113 | "1971-04",878 114 | "1971-05",942 115 | "1971-06",913 116 | "1971-07",869 117 | "1971-08",834 118 | "1971-09",790 119 | "1971-10",800 120 | "1971-11",763 121 | "1971-12",800 122 | "1972-01",826 123 | "1972-02",799 124 | "1972-03",890 125 | "1972-04",900 126 | "1972-05",961 127 | "1972-06",935 128 | "1972-07",894 129 | "1972-08",855 130 | "1972-09",809 131 | "1972-10",810 132 | "1972-11",766 133 | "1972-12",805 134 | "1973-01",821 135 | "1973-02",773 136 | "1973-03",883 137 | "1973-04",898 138 | "1973-05",957 139 | "1973-06",924 140 | "1973-07",881 141 | "1973-08",837 142 | "1973-09",784 143 | "1973-10",791 144 | "1973-11",760 145 | "1973-12",802 146 | "1974-01",828 147 | "1974-02",778 148 | "1974-03",889 149 | "1974-04",902 150 | "1974-05",969 151 | "1974-06",947 152 | "1974-07",908 153 | "1974-08",867 154 | "1974-09",815 155 | "1974-10",812 156 | "1974-11",773 157 | "1974-12",813 158 | "1975-01",834 159 | "1975-02",782 160 | "1975-03",892 161 | "1975-04",903 162 | "1975-05",966 163 | "1975-06",937 164 | "1975-07",896 165 | "1975-08",858 166 | "1975-09",817 167 | "1975-10",827 168 | "1975-11",797 169 | "1975-12",843 170 | 171 | Monthly milk production: pounds per cow. Jan 62 ? Dec 75 172 | 173 | -------------------------------------------------------------------------------- /datasets/my_test_output.txt: -------------------------------------------------------------------------------- 1 | this is only a test 2 | the value of x is 2 3 | -------------------------------------------------------------------------------- /datasets/normalized_apple_prices.csv: -------------------------------------------------------------------------------- 1 | -7.006233945278093067e-01 2 | -8.208848439047651269e-01 3 | -9.393830524103554680e-01 4 | -9.471651967287715301e-01 5 | -6.878552698496820383e-01 6 | -8.432590195361995278e-01 7 | -8.053201783108936418e-01 8 | -8.205807307732939648e-01 9 | -9.202312393102278776e-01 10 | -1.000000000000000000e+00 11 | -9.881443849549484959e-01 12 | -8.596141093220164286e-01 13 | -8.706188046502583155e-01 14 | -9.266151162107476580e-01 15 | -8.011858533422127060e-01 16 | -7.628820447433897201e-01 17 | -6.649947770778212863e-01 18 | -6.728988244388007800e-01 19 | -6.822011452976286527e-01 20 | -5.421189968740289800e-01 21 | -4.650859167780025949e-01 22 | -2.148959182702663284e-01 23 | -1.702082327709231180e-01 24 | -8.247456439817346663e-02 25 | 6.411336396851030628e-02 26 | 8.575760485291095137e-02 27 | 3.860465375596104920e-01 28 | 3.946803372690568246e-01 29 | 4.070833060951475346e-01 30 | 5.548260727516387547e-01 31 | 4.571211975735445243e-01 32 | 2.172670023675826734e-01 33 | 3.825809222188305547e-01 34 | 1.618787285915157526e-01 35 | 1.683843200560799502e-01 36 | -2.279982092837862240e-03 37 | 2.190304256375150693e-01 38 | 1.618787285915157526e-01 39 | 3.212949002041503022e-01 40 | 2.193948397353775448e-01 41 | 2.579419011436732134e-01 42 | 3.031162746605300384e-01 43 | 4.281805640755953490e-01 44 | 4.270862152306991177e-01 45 | 3.619089325782907096e-01 46 | 3.407511912328837766e-01 47 | 5.010795016015081593e-01 48 | 5.381670643928226916e-01 49 | 7.000153628233394265e-01 50 | 8.822922096064025332e-01 51 | 7.957746107225456278e-01 52 | 8.850891213992340134e-01 53 | 1.000000000000000444e+00 54 | 9.240614511474380954e-01 55 | 8.286061279011831537e-01 56 | 6.809850818886880042e-01 57 | 5.926435702899879310e-01 58 | 4.714697936785232635e-01 59 | 3.648275650151182603e-01 60 | 2.957593986820223897e-01 61 | 1.171908546136526397e-01 62 | 3.547665512351283468e-02 63 | 2.494301868753581175e-01 64 | 3.573493392697018045e-01 65 | -6.003952844474014228e-02 66 | -1.577444036228339286e-01 67 | -8.831108143770105556e-02 68 | -1.480166320255307255e-01 69 | -7.827938518744614882e-02 70 | -1.957439227715709329e-01 71 | -1.843137645449477446e-01 72 | -5.900290449046785568e-01 73 | -4.922634978831066377e-01 74 | -3.549172124518196192e-01 75 | -4.485484405846635880e-01 76 | -4.926280943795364742e-01 77 | -6.510109598678401888e-01 78 | -6.391549800256544067e-01 79 | -5.680194657696739924e-01 80 | -4.267214363357010320e-01 81 | -5.652835966974087967e-01 82 | -6.689468919575944916e-01 83 | -6.558748487064680610e-01 84 | -8.647821052121629215e-01 85 | -6.956984629686155941e-01 86 | -4.813196567969098005e-01 87 | -5.068553467975576154e-01 88 | -6.260222588510968578e-01 89 | -5.166437963182923809e-01 90 | -5.115977034707088755e-01 91 | -5.480774169561208886e-01 92 | -6.288798485673590122e-01 93 | -7.750419492385272058e-01 94 | -8.036784756849577604e-01 95 | -6.977658108914996937e-01 96 | -6.679738928796070674e-01 97 | -6.409182208970194417e-01 98 | -5.719715806494476418e-01 99 | -4.267214363357010320e-01 100 | -4.743273773148324324e-01 101 | -1.856515485978849078e-01 102 | -2.074783704482801028e-01 103 | -2.539901510610338420e-01 104 | -1.816383849175937648e-01 105 | -4.491566607676533707e-01 106 | -2.357501119197609185e-01 107 | -3.503572482661434506e-01 108 | -2.937530934797036863e-01 109 | -2.738713456325414519e-01 110 | -1.404771750810285269e-01 111 | -3.547665512351239059e-02 112 | -8.375148812911215046e-02 113 | -9.050015000458122572e-02 114 | -6.010038876673817754e-02 115 | -8.423761679101815503e-02 116 | 1.405988957250254856e-01 117 | 1.582309031618285289e-01 118 | 1.224807596263159937e-01 119 | 2.013984182007120260e-01 120 | 1.373158552702875568e-01 121 | 1.565594999691644063e-02 122 | -3.018675619080157801e-02 123 | 3.717884719436037599e-02 124 | 9.238491696147166365e-02 125 | -1.961695619885666098e-01 126 | -9.858658985303581090e-02 127 | 6.763946875420145943e-02 128 | -7.128729418388823902e-02 129 | -6.964595635508707971e-02 130 | -3.961060921510606292e-02 131 | -4.362395529396456695e-02 132 | 2.155369823552977238e-02 133 | 2.647844739615612397e-02 134 | -4.167795105803762112e-02 135 | -7.888723449334911209e-02 136 | -5.797255139894330611e-02 137 | 2.305824869067012450e-01 138 | 3.360086521611651555e-01 139 | -------------------------------------------------------------------------------- /environment_setup_instructions.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/environment_setup_instructions.txt -------------------------------------------------------------------------------- /images/RNN_graph_model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/images/RNN_graph_model.png -------------------------------------------------------------------------------- /images/apple_RNN_prediction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/images/apple_RNN_prediction.png -------------------------------------------------------------------------------- /images/dnn2rnn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/images/dnn2rnn.png -------------------------------------------------------------------------------- /images/dog_speech.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/images/dog_speech.png -------------------------------------------------------------------------------- /images/sample_grid_a_square.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/images/sample_grid_a_square.png -------------------------------------------------------------------------------- /images/spectrogram_creation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/images/spectrogram_creation.png -------------------------------------------------------------------------------- /images/text_windowing.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/images/text_windowing.gif -------------------------------------------------------------------------------- /images/text_windowing_training.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/images/text_windowing_training.gif -------------------------------------------------------------------------------- /images/time_windowing.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/images/time_windowing.gif -------------------------------------------------------------------------------- /images/timeseries_windowing_training.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/images/timeseries_windowing_training.gif -------------------------------------------------------------------------------- /images/wright_bros_face_detect.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/images/wright_bros_face_detect.png -------------------------------------------------------------------------------- /model_weights/best_RNN_large_textdata_weights.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/model_weights/best_RNN_large_textdata_weights.hdf5 -------------------------------------------------------------------------------- /model_weights/best_RNN_small_textdata_weights.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/model_weights/best_RNN_small_textdata_weights.hdf5 -------------------------------------------------------------------------------- /model_weights/best_RNN_weights.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoninithomas/Time-series-prediction-and-text-generation/91e393c9b1e3be1982165dcd818a134d40a31ab6/model_weights/best_RNN_weights.hdf5 -------------------------------------------------------------------------------- /my_answers.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | from keras.models import Sequential 4 | from keras.layers import Dense 5 | from keras.layers import LSTM 6 | from keras.layers import Dropout 7 | from keras.layers import Activation 8 | import keras 9 | import string 10 | 11 | # TODO: fill out the function below that transforms the input series 12 | # and window-size into a set of input/output pairs for use with our RNN model 13 | def window_transform_series(series, window_size): 14 | # containers for input/output pairs 15 | X = [] 16 | y = [] 17 | 18 | # Let's calculate how many series we can make 19 | nb_of_pairs = len(series) - window_size 20 | print(nb_of_pairs) 21 | for i in range(nb_of_pairs): 22 | 23 | # init_val = p at which we began 24 | init_val = i 25 | 26 | end_val = window_size + i 27 | 28 | y_val = end_val 29 | 30 | X.append(series[init_val:end_val]) 31 | y.append(series[y_val]) 32 | 33 | # reshape each 34 | X = np.asarray(X) 35 | X.shape = (np.shape(X)[0:2]) 36 | y = np.asarray(y) 37 | y.shape = (len(y),1) 38 | 39 | print(X) 40 | print(y) 41 | 42 | 43 | return X,y 44 | 45 | # TODO: build an RNN to perform regression on our time series input/output data 46 | def build_part1_RNN(window_size): 47 | 48 | model = Sequential() 49 | #layer 1 uses an LSTM module with 5 hidden units (note here the input_shape = (window_size,1)) 50 | model.add(LSTM(5, 51 | input_shape = (window_size, 1))) 52 | #layer 2 uses a fully connected module with one unit 53 | model.add(Dense(1)) 54 | 55 | return model 56 | 57 | 58 | ### TODO: return the text input with only ascii lowercase and the punctuation given below included. 59 | def cleaned_text(text): 60 | punctuation = ['!', ',', '.', ':', ';', '?'] 61 | 62 | text_char = ''.join(punctuation) 63 | text_char = text_char + string.ascii_lowercase 64 | 65 | # Lowercase 66 | text = text.lower() 67 | 68 | for t in text: 69 | if not t in text_char: 70 | text = text.replace(t, ' ') 71 | 72 | return text 73 | 74 | ### TODO: fill out the function below that transforms the input text and window-size into a set of input/output pairs for use with our RNN model 75 | def window_transform_text(text, window_size, step_size): 76 | # containers for input/output pairs 77 | inputs = [] 78 | outputs = [] 79 | 80 | # Let's calculate how many series we can make 81 | nb_of_pairs = len(text) - window_size 82 | 83 | index = 0 84 | i = 0 85 | for i in range(0, nb_of_pairs, step_size): 86 | 87 | # init_val = p at which we began 88 | init_val = i 89 | end_val = init_val + window_size 90 | y_val = end_val 91 | 92 | inputs.append(text[init_val:end_val]) 93 | outputs.append(text[y_val]) 94 | 95 | return inputs,outputs 96 | 97 | 98 | # TODO build the required RNN model: 99 | # a single LSTM hidden layer with softmax activation, categorical_crossentropy loss 100 | def build_part2_RNN(window_size, num_chars): 101 | model = Sequential() 102 | 103 | # layer 1 should be an LSTM module with 200 hidden units --> note this should have input_shape = (window_size,len(chars)) where len(chars) = number of unique characters in your cleaned text 104 | model.add(LSTM(200, 105 | input_shape = (window_size, num_chars))) 106 | 107 | # layer 2 should be a linear module, fully connected, with len(chars) hidden units --> where len(chars) = number of unique characters in your cleaned text 108 | model.add(Dense(num_chars)) 109 | 110 | #layer 3 should be a softmax activation ( since we are solving a multiclass classification) Use the categorical_crossentropy loss 111 | model.add(Activation("softmax")) 112 | 113 | return model 114 | 115 | 116 | 117 | 118 | 119 | 120 | -------------------------------------------------------------------------------- /my_test_output.txt: -------------------------------------------------------------------------------- 1 | this is only a test 2 | the value of x is 2 3 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | opencv-python==3.2.0.6 2 | matplotlib==2.0.0 3 | numpy==1.12.0 4 | scipy==0.18.1 5 | keras==2.0.2 6 | scikit-learn==0.18.1 7 | tensorflow==1.0.0 8 | -------------------------------------------------------------------------------- /text_gen_output/RNN_large_textdata_output.txt: -------------------------------------------------------------------------------- 1 | ------------------- 2 | input chars = 3 | e eclipses and predominates the whole of her sex. it was not that he felt any emotion akin to love f" 4 | predicted chars = 5 | or a claad of the carige. is no doubt that i may groute, and sourne, he said, and where the case upo" 6 | ------------------- 7 | input chars = 8 | difficulties, if you are to understand the situation. i am following you closely, i answered. i was" 9 | predicted chars = 10 | weakn of their. weyen he come to see me what i was ally unow that there was no some allosing a smal" 11 | ------------------- 12 | input chars = 13 | notice the carriage go up any. come, cried the inspector, laughing; its a very pretty diversity of o" 14 | predicted chars = 15 | thers. and the evering a small bearine it allars where are remarked hor had been day no tonsing that" 16 | --------------------------------------------------------------------------------