├── .DS_Store ├── README.md └── keras-master ├── .gitignore ├── .travis.yml ├── Keras.egg-info ├── PKG-INFO ├── SOURCES.txt ├── dependency_links.txt ├── requires.txt └── top_level.txt ├── LICENSE ├── docs ├── README.md ├── mkdocs.yml └── sources │ ├── activations.md │ ├── callbacks.md │ ├── constraints.md │ ├── datasets.md │ ├── documentation.md │ ├── examples.md │ ├── index.md │ ├── initializations.md │ ├── layers │ ├── advanced_activations.md │ ├── containers.md │ ├── convolutional.md │ ├── core.md │ ├── embeddings.md │ ├── noise.md │ ├── normalization.md │ └── recurrent.md │ ├── models.md │ ├── objectives.md │ ├── optimizers.md │ ├── preprocessing │ ├── image.md │ ├── sequence.md │ └── text.md │ ├── regularizers.md │ └── utils │ └── visualization.md ├── examples ├── SimpleNet.py ├── TempRecSys .py ├── TempRecSys.py ├── cifar10_cnn.py ├── console.out ├── decodeDoc.py ├── imdb_lstm.py ├── kaggle_otto_nn.py ├── lstm_text_generation.py ├── mnist_cnn.py ├── mnist_irnn.py ├── mnist_mlp.py ├── movieLens100k.cos.batch.py ├── movieLens100k.cos.py ├── movieLens100k.maxdoty.py ├── movieLens100k.mixedmembership.max.py ├── movieLens100k.mixedmembership.py ├── movieLens100k.mixedmembership_singleitemmodel.py ├── movieLens1M.cos - Copy (2).py ├── movieLens1M.cos.py ├── movieLens1M.hidden.py ├── movielens_100K.py ├── movieslensDSSM.py ├── out.err ├── reuters_autoEncoder.py ├── reuters_factorization.py ├── reuters_mlp.py ├── reuters_mlp_using_factors.py ├── skipgram_word_embeddings.py ├── tempRec - Copy.py ├── tempRec.py ├── tempRecTest.nolstm.py ├── tempRecTest.py ├── tempRecTest_nouser.py ├── tempRec_nouser.py ├── tempRec_nouser_no_seq.py └── tempRecomnedation.batch.py ├── keras ├── __init__.py ├── activations.py ├── callbacks.py ├── constraints.py ├── datasets │ ├── __init__.py │ ├── cifar.py │ ├── cifar10.py │ ├── cifar100.py │ ├── data_utils.py │ ├── imdb.py │ ├── mnist.py │ └── reuters.py ├── initializations.py ├── layers │ ├── __init__.py │ ├── advanced_activations.py │ ├── containers.py │ ├── convolutional.py │ ├── core.py │ ├── embeddings.py │ ├── noise.py │ ├── normalization.py │ └── recurrent.py ├── models.py ├── objectives.py ├── optimizers.py ├── preprocessing │ ├── __init__.py │ ├── image.py │ ├── sequence.py │ └── text.py ├── regularizers.py ├── utils │ ├── __init__.py │ ├── dot_utils.py │ ├── generic_utils.py │ ├── io_utils.py │ ├── np_utils.py │ ├── test_utils.py │ └── theano_utils.py └── wrappers │ ├── __init__.py │ └── scikit_learn.py ├── setup.cfg ├── setup.py └── tests ├── __init__.py ├── auto ├── keras │ ├── test_activations.py │ └── test_constraints.py ├── test_embeddings.py ├── test_graph_model.py ├── test_loss_weighting.py ├── test_regularizers.py └── test_tasks.py └── manual ├── __init__.py ├── check_autoencoder.py ├── check_callbacks.py ├── check_constraints.py ├── check_dot_utils.py ├── check_masked_recurrent.py ├── check_models.py ├── check_save_weights.py └── check_wrappers.py /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sonyisme/keras-recommendation/e84fab6706faf8405f4bfc75cf50ecff6433eb77/.DS_Store -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Keras Implementation of Recommender Systems 2 | 3 | This library contains a modified version of Keras (mostly in the layers/core.py) to implement various recommender systems, including the Deep Structured Semantic Model (DSSM), Multi-View DSSM (MV-DSSM), Temporal DSSM (TDSSM) and matrix factorization (MF). 4 | 5 | The examples can be found in the examples/ folder. 6 | 7 | Contact: Yang Song (sonyisme AT google dot com) 8 | 9 | Homepage: http://sonyis.me 10 | 11 | ## Examples 12 | 13 | ### Temporal DSSM 14 | ```python 15 | staticmodel=Sequential() 16 | staticmodel.add(Dense(300,300)) 17 | staticmodel.add(Activation('tanh')) 18 | tempmodel=Sequential() 19 | tempmodel.add(LSTM(tempFea,300)) 20 | model=Sequential() 21 | model.add(Merge([staticmodel, tempmodel],mode='concat')) 22 | model.add(Dense(300+300,300)) 23 | model.add(Activation('tanh')) 24 | ``` 25 | 26 | ### DSSM and Multi-view DSSM 27 | ```python 28 | userModel = Sequential() 29 | userModel.add(Dense(3883, 300)) 30 | userModel.add(Activation('tanh')) 31 | userModel.add(Dropout(0.4)) 32 | userModel.add(Dense(300, 300)) 33 | userModel.add(Activation('tanh')) 34 | 35 | itemModel = Sequential() 36 | itemModel.add(TimeDistributedDense(6039, 300)) 37 | itemModel.add(Activation('tanh')) 38 | itemModel.add(Dropout(0.4)) 39 | itemModel.add(TimeDistributedDense(300, 300)) 40 | itemModel.add(Activation('tanh')) 41 | 42 | model=Sequential() 43 | model.add(Cosine([userModel,itemModel])) #should output 2 values 44 | model.add(Reshape(2)) 45 | ``` 46 | ### Matrix Factorization on MovieLens 47 | ```python 48 | serModel = Sequential() 49 | userModel.add(Dense(1682, 500)) 50 | userModel.add(Activation('tanh')) 51 | userModel.add(Dropout(0.4)) 52 | userModel.add(Dense(500, 500)) 53 | userModel.add(Activation('tanh')) 54 | 55 | itemModel = Sequential() 56 | itemModel.add(TimeDistributedDense(943, 500)) 57 | itemModel.add(Activation('tanh')) 58 | itemModel.add(Dropout(0.4)) 59 | itemModel.add(TimeDistributedDense(500, 500)) 60 | itemModel.add(Activation('tanh')) 61 | model=Sequential() 62 | model.add(ElementMul([userModel,itemModel])) #should output 2 values 63 | model.add(TimeDistributedDense(500, 1)) 64 | model.add(Reshape(2)) 65 | y_score= model.get_output(train=False) 66 | x_test=model.get_input(train=False) 67 | model.add(Activation('softmax')) 68 | ``` 69 | 70 | ## References 71 | [1] Yang Song, Ali Elkahky, and Xiaodong He, Multi-Rate Deep Learning for Temporal Recommendation, in SIGIR 2016. 72 | 73 | [2] Ali Mamdouh Elkahky, Yang Song, and Xiaodong He, A Multi-View Deep Learning Approach for User Modeling in Recommendation Systems, in WWW 2015. 74 | 75 | [3] Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero and Larry Heck, Learning Deep Structured Semantic Models for Web Search using Clickthrough Data, in CIKM 2013. 76 | 77 | -------------------------------------------------------------------------------- /keras-master/.gitignore: -------------------------------------------------------------------------------- 1 | *.DS_Store 2 | *.pyc 3 | *.swp 4 | temp/* 5 | dist/* 6 | build/* 7 | keras/datasets/data/* 8 | keras/datasets/temp/* 9 | docs/site/* 10 | docs/theme/* -------------------------------------------------------------------------------- /keras-master/.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | # Setup anaconda 3 | before_install: 4 | - wget http://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh -O miniconda.sh 5 | - chmod +x miniconda.sh 6 | - ./miniconda.sh -b 7 | - export PATH=/home/travis/miniconda/bin:$PATH 8 | - conda update --yes conda 9 | # The next couple lines fix a crash with multiprocessing on Travis and are not specific to using Miniconda 10 | - sudo rm -rf /dev/shm 11 | - sudo ln -s /run/shm /dev/shm 12 | python: 13 | - "3.4" 14 | # command to install dependencies 15 | install: 16 | - conda install --yes python=$TRAVIS_PYTHON_VERSION numpy scipy matplotlib pandas pytest 17 | # Coverage packages are on my binstar channel 18 | - python setup.py install 19 | # command to run tests 20 | script: py.test -------------------------------------------------------------------------------- /keras-master/Keras.egg-info/PKG-INFO: -------------------------------------------------------------------------------- 1 | Metadata-Version: 1.1 2 | Name: Keras 3 | Version: 0.1.1 4 | Summary: Theano-based Deep Learning library 5 | Home-page: https://github.com/fchollet/keras 6 | Author: Francois Chollet 7 | Author-email: francois.chollet@gmail.com 8 | License: MIT 9 | Download-URL: https://github.com/fchollet/keras/tarball/0.1.1 10 | Description: UNKNOWN 11 | Platform: UNKNOWN 12 | -------------------------------------------------------------------------------- /keras-master/Keras.egg-info/SOURCES.txt: -------------------------------------------------------------------------------- 1 | setup.cfg 2 | setup.py 3 | Keras.egg-info/PKG-INFO 4 | Keras.egg-info/SOURCES.txt 5 | Keras.egg-info/dependency_links.txt 6 | Keras.egg-info/requires.txt 7 | Keras.egg-info/top_level.txt 8 | keras/__init__.py 9 | keras/activations.py 10 | keras/callbacks.py 11 | keras/constraints.py 12 | keras/initializations.py 13 | keras/models.py 14 | keras/objectives.py 15 | keras/optimizers.py 16 | keras/regularizers.py 17 | keras/datasets/__init__.py 18 | keras/datasets/cifar.py 19 | keras/datasets/cifar10.py 20 | keras/datasets/cifar100.py 21 | keras/datasets/data_utils.py 22 | keras/datasets/imdb.py 23 | keras/datasets/mnist.py 24 | keras/datasets/reuters.py 25 | keras/layers/__init__.py 26 | keras/layers/advanced_activations.py 27 | keras/layers/containers.py 28 | keras/layers/convolutional.py 29 | keras/layers/core.py 30 | keras/layers/embeddings.py 31 | keras/layers/noise.py 32 | keras/layers/normalization.py 33 | keras/layers/recurrent.py 34 | keras/preprocessing/__init__.py 35 | keras/preprocessing/image.py 36 | keras/preprocessing/sequence.py 37 | keras/preprocessing/text.py 38 | keras/utils/__init__.py 39 | keras/utils/dot_utils.py 40 | keras/utils/generic_utils.py 41 | keras/utils/io_utils.py 42 | keras/utils/np_utils.py 43 | keras/utils/test_utils.py 44 | keras/utils/theano_utils.py 45 | keras/wrappers/__init__.py 46 | keras/wrappers/scikit_learn.py 47 | tests/__init__.py 48 | tests/manual/__init__.py 49 | tests/manual/check_autoencoder.py 50 | tests/manual/check_callbacks.py 51 | tests/manual/check_constraints.py 52 | tests/manual/check_dot_utils.py 53 | tests/manual/check_masked_recurrent.py 54 | tests/manual/check_models.py 55 | tests/manual/check_save_weights.py 56 | tests/manual/check_wrappers.py -------------------------------------------------------------------------------- /keras-master/Keras.egg-info/dependency_links.txt: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /keras-master/Keras.egg-info/requires.txt: -------------------------------------------------------------------------------- 1 | theano 2 | -------------------------------------------------------------------------------- /keras-master/Keras.egg-info/top_level.txt: -------------------------------------------------------------------------------- 1 | keras 2 | tests 3 | -------------------------------------------------------------------------------- /keras-master/LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /keras-master/docs/README.md: -------------------------------------------------------------------------------- 1 | # Keras Documentation 2 | 3 | The source for Keras documentation is in this directory under `sources/`. 4 | Our documentation uses extended Markdown, as implemented by [MkDocs](http://mkdocs.org). 5 | 6 | ## Building the documentation 7 | 8 | - install MkDocs: `sudo pip install mkdocs` 9 | - `cd` to the `docs/` folder and run: `mkdocs serve` -------------------------------------------------------------------------------- /keras-master/docs/mkdocs.yml: -------------------------------------------------------------------------------- 1 | site_name: Keras Documentation 2 | #theme: readthedocs 3 | docs_dir: sources 4 | repo_url: http://github.com/fchollet/keras 5 | site_url: http://keras.io/ 6 | theme_dir: theme 7 | site_description: Documentation for fast and lightweight Keras Deep Learning library. 8 | 9 | dev_addr: '0.0.0.0:8000' 10 | google_analytics: ['UA-61785484-1', 'keras.io'] 11 | 12 | 13 | pages: 14 | - Home: index.md 15 | - Index: documentation.md 16 | - Examples: examples.md 17 | - Optimizers: optimizers.md 18 | - Objectives: objectives.md 19 | - Models: models.md 20 | - Activations: activations.md 21 | - Initializations: initializations.md 22 | - Regularizers: regularizers.md 23 | - Constraints: constraints.md 24 | - Callbacks: callbacks.md 25 | - Datasets: datasets.md 26 | - Layers: 27 | - Core Layers: layers/core.md 28 | - Convolutional Layers: layers/convolutional.md 29 | - Recurrent Layers: layers/recurrent.md 30 | - Advanced Activations Layers: layers/advanced_activations.md 31 | - Normalization Layers: layers/normalization.md 32 | - Embedding Layers: layers/embeddings.md 33 | - Noise layers: layers/noise.md 34 | - Containers: layers/containers.md 35 | - Preprocessing: 36 | - Sequence Preprocessing: preprocessing/sequence.md 37 | - Text Preprocessing: preprocessing/text.md 38 | - Image Preprocessing: preprocessing/image.md 39 | - Utils: 40 | - Visualization Utilities: utils/visualization.md 41 | 42 | -------------------------------------------------------------------------------- /keras-master/docs/sources/activations.md: -------------------------------------------------------------------------------- 1 | 2 | ## Usage of activations 3 | 4 | Activations can either be used through an `Activation` layer, or through the `activation` argument supported by all forward layers: 5 | 6 | ```python 7 | from keras.layers.core import Activation, Dense 8 | 9 | model.add(Dense(64, 64, init='uniform')) 10 | model.add(Activation('tanh')) 11 | ``` 12 | is equivalent to: 13 | ```python 14 | model.add(Dense(20, 64, init='uniform', activation='tanh')) 15 | ``` 16 | 17 | You can also pass an element-wise Theano function as an activation: 18 | 19 | ```python 20 | def tanh(x): 21 | return theano.tensor.tanh(x) 22 | 23 | model.add(Dense(20, 64, init='uniform', activation=tanh)) 24 | model.add(Activation(tanh)) 25 | ``` 26 | 27 | ## Available activations 28 | 29 | - __softmax__: Softmax applied across inputs last dimension. Expects shape either `(nb_samples, nb_timesteps, nb_dims)` or `(nb_samples, nb_dims)`. 30 | - __softplus__ 31 | - __relu__ 32 | - __tanh__ 33 | - __sigmoid__ 34 | - __hard_sigmoid__ 35 | - __linear__ 36 | 37 | ## On Advanced Activations 38 | 39 | Activations that are more complex than a simple Theano function (eg. learnable activations, configurable activations, etc.) are available as [Advanced Activation layers](layers/advanced_activations.md), and can be found in the module `keras.layers.advanced_activations`. These include PReLU and LeakyReLU. 40 | -------------------------------------------------------------------------------- /keras-master/docs/sources/callbacks.md: -------------------------------------------------------------------------------- 1 | ## Usage of callbacks 2 | 3 | A callback is a set of functions to be applied at given stages of the training procedure. You can use callbacks to get a view on internal states and statistics of the model during training. You can pass a list of callback (as the keyword argument `callbacks`) to the `.fit()` method of the `Sequential` model. The relevant methods of the callbacks will then be called at each stage of the training. 4 | 5 | --- 6 | 7 | ## Base class 8 | 9 | ```python 10 | keras.callbacks.Callback() 11 | ``` 12 | - __Properties__: 13 | - __params__: dict. Training parameters (eg. verbosity, batch size, number of epochs...). 14 | - __model__: `keras.models.Model`. Reference of the model being trained. 15 | - __Methods__: 16 | - __on_train_begin__(logs={}): Method called at the beginning of training. 17 | - __on_train_end__(logs={}): Method called at the end of training. 18 | - __on_epoch_begin__(epoch, logs={}): Method called at the beginning of epoch `epoch`. 19 | - __on_epoch_end__(epoch, logs={}): Method called at the end of epoch `epoch`. 20 | - __on_batch_begin__(batch, logs={}): Method called at the beginning of batch `batch`. 21 | - __on_batch_end__(batch, logs={}): Method called at the end of batch `batch`. 22 | 23 | The `logs` dictionary will contain keys for quantities relevant to the current batch or epoch. Currently, the `.fit()` method of the `Sequential` model class will include the following quantities in the `logs` that it passes to its callbacks: 24 | - __on_epoch_end__: logs optionally include `val_loss` (if validation is enabled in `fit`), and `val_accuracy` (if validation and accuracy monitoring are enabled). 25 | - __on_batch_begin__: logs include `size`, the number of samples in the current batch. 26 | - __on_batch_end__: logs include `loss`, and optionally `accuracy` (if accuracy monitoring is enabled). 27 | 28 | --- 29 | 30 | ## Available callbacks 31 | 32 | ```python 33 | keras.callbacks.ModelCheckpoint(filepath, verbose=0, save_best_only=False) 34 | ``` 35 | 36 | Save the model after every epoch. If `save_best_only=True`, the latest best model according to the validation loss will not be overwritten. 37 | 38 | 39 | ```python 40 | keras.callbacks.EarlyStopping(monitor='val_loss', patience=0, verbose=0) 41 | ``` 42 | 43 | Stop training after no improvement of the metric `monitor` is seen for `patience` epochs. 44 | 45 | --- 46 | 47 | 48 | ## Create a callback 49 | 50 | You can create a custom callback by extending the base class `keras.callbacks.Callback`. A callback has access to its associated model through the class property `self.model`. 51 | 52 | Here's a simple example saving a list of losses over each batch during training: 53 | ```python 54 | class LossHistory(keras.callbacks.Callback): 55 | def on_train_begin(self): 56 | self.losses = [] 57 | 58 | def on_batch_end(self, batch, logs={}): 59 | self.losses.append(logs.get('loss')) 60 | ``` 61 | 62 | --- 63 | 64 | ### Example to record the loss history 65 | 66 | ```python 67 | class LossHistory(keras.callbacks.Callback): 68 | def on_train_begin(self, logs={}): 69 | self.losses = [] 70 | 71 | def on_batch_end(self, batch, logs={}): 72 | self.losses.append(logs.get('loss')) 73 | 74 | model = Sequential() 75 | model.add(Dense(784, 10, init='uniform')) 76 | model.add(Activation('softmax')) 77 | model.compile(loss='categorical_crossentropy', optimizer='rmsprop') 78 | 79 | history = LossHistory() 80 | model.fit(X_train, Y_train, batch_size=128, nb_epoch=20, verbose=0, callbacks=[history]) 81 | 82 | print history.losses 83 | # outputs 84 | ''' 85 | [0.66047596406559383, 0.3547245744908703, ..., 0.25953155204159617, 0.25901699725311789] 86 | ''' 87 | ``` 88 | 89 | --- 90 | 91 | ### Example to checkpoint models 92 | 93 | ```python 94 | from keras.callbacks import ModelCheckpoint 95 | 96 | model = Sequential() 97 | model.add(Dense(784, 10, init='uniform')) 98 | model.add(Activation('softmax')) 99 | model.compile(loss='categorical_crossentropy', optimizer='rmsprop') 100 | 101 | ''' 102 | saves the model weights after each epoch if the validation loss decreased 103 | ''' 104 | checkpointer = ModelCheckpoint(filepath="/tmp/weights.hdf5", verbose=1, save_best_only=True) 105 | model.fit(X_train, Y_train, batch_size=128, nb_epoch=20, verbose=0, validation_data=(X_test, Y_test), callbacks=[checkpointer]) 106 | 107 | ``` 108 | 109 | -------------------------------------------------------------------------------- /keras-master/docs/sources/constraints.md: -------------------------------------------------------------------------------- 1 | ## Usage of constraints 2 | 3 | Functions from the `constraints` module allow setting constraints (eg. non-negativity) on network parameters during optimization. 4 | 5 | The penalties are applied on a per-layer basis. The exact API will depend on the layer, but the layers `Dense`, `TimeDistributedDense`, `MaxoutDense`, `Convolution1D` and `Convolution2D` have a unified API. 6 | 7 | These layers expose 2 keyword arguments: 8 | 9 | - `W_constraint` for the main weights matrix 10 | - `b_constraint` for the bias. 11 | 12 | 13 | ```python 14 | from keras.constraints import maxnorm 15 | model.add(Dense(64, 64, W_constraint = maxnorm(2))) 16 | ``` 17 | 18 | ## Available constraints 19 | 20 | - __maxnorm__(m=2): maximum-norm constraint 21 | - __nonneg__(): non-negativity constraint 22 | - __unitnorm__(): unit-norm constraint, enforces the matrix to have unit norm along the last axis -------------------------------------------------------------------------------- /keras-master/docs/sources/datasets.md: -------------------------------------------------------------------------------- 1 | # Datasets 2 | 3 | ## CIFAR10 small image classification 4 | 5 | `keras.datasets.cifar10` 6 | 7 | Dataset of 50,000 32x32 color training images, labeled over 10 categories, and 10,000 test images. 8 | 9 | ### Usage: 10 | 11 | ```python 12 | (X_train, y_train), (X_test, y_test) = cifar10.load_data() 13 | ``` 14 | 15 | - __Return:__ 16 | - 2 tuples: 17 | - __X_train, X_test__: uint8 array of RGB image data with shape (nb_samples, 3, 32, 32). 18 | - __y_train, y_test__: uint8 array of category labels (integers in range 0-9) with shape (nb_samples,). 19 | 20 | --- 21 | 22 | ## CIFAR100 small image classification 23 | 24 | `keras.datasets.cifar100` 25 | 26 | Dataset of 50,000 32x32 color training images, labeled over 100 categories, and 10,000 test images. 27 | 28 | ### Usage: 29 | 30 | ```python 31 | (X_train, y_train), (X_test, y_test) = cifar100.load_data(label_mode='fine') 32 | ``` 33 | 34 | - __Return:__ 35 | - 2 tuples: 36 | - __X_train, X_test__: uint8 array of RGB image data with shape (nb_samples, 3, 32, 32). 37 | - __y_train, y_test__: uint8 array of category labels with shape (nb_samples,). 38 | 39 | - __Arguments:__ 40 | 41 | - __label_mode__: "fine" or "coarse". 42 | 43 | --- 44 | 45 | ## IMDB Movie reviews sentiment classification 46 | 47 | `keras.datasets.imdb` 48 | 49 | Dataset of 25,000 movies reviews from IMDB, labeled by sentiment (positive/negative). Reviews have been preprocessed, and each review is encoded as a [sequence](preprocessing/sequence.md) of word indexes (integers). For convenience, words are indexed by overall frequency in the dataset, so that for instance the integer "3" encodes the 3rd most frequent word in the data. This allows for quick filtering operations such as: "only consider the top 10,000 most common words, but eliminate the top 20 most common words". 50 | 51 | As a convention, "0" does not stand for a specific word, but instead is used to encode any unknown word. 52 | 53 | ### Usage: 54 | 55 | ```python 56 | (X_train, y_train), (X_test, y_test) = imdb.load_data(path="imdb.pkl", \ 57 | nb_words=None, skip_top=0, maxlen=None, test_split=0.1, seed=113) 58 | ``` 59 | - __Return:__ 60 | - 2 tuples: 61 | - __X_train, X_test__: list of sequences, which are lists of indexes (integers). If the nb_words argument was specific, the maximum possible index value is nb_words-1. If the maxlen argument was specified, the largest possible sequence length is maxlen. 62 | - __y_train, y_test__: list of integer labels (1 or 0). 63 | 64 | - __Arguments:__ 65 | 66 | - __path__: if you do have the data locally (at `'~/.keras/datasets/' + path`), if will be downloaded to this location (in cPickle format). 67 | - __nb_words__: integer or None. Top most frequent words to consider. Any less frequent word will appear as 0 in the sequence data. 68 | - __skip_top__: integer. Top most frequent words to ignore (they will appear as 0s in the sequence data). 69 | - __maxlen__: int. Maximum sequence length. Any longer sequence will be truncated. 70 | - __test_split__: float. Fraction of the dataset to be used as test data. 71 | - __seed__: int. Seed for reproducible data shuffling. 72 | 73 | --- 74 | 75 | ## Reuters newswire topics classification 76 | 77 | `keras.datasets.reuters` 78 | 79 | Dataset of 11,228 newswires from Reuters, labeled over 46 topics. As with the IMDB dataset, each wire is encoded as a sequence of word indexes (same conventions). 80 | 81 | ### Usage: 82 | 83 | ```python 84 | (X_train, y_train), (X_test, y_test) = reuters.load_data(path="reuters.pkl", \ 85 | nb_words=None, skip_top=0, maxlen=None, test_split=0.1, seed=113) 86 | ``` 87 | 88 | The specifications are the same as that of the IMDB dataset. 89 | 90 | This dataset also makes available the word index used for encoding the sequences: 91 | 92 | ```python 93 | word_index = reuters.get_word_index(path="reuters_word_index.pkl") 94 | ``` 95 | 96 | - __Return:__ A dictionary where key are words (str) and values are indexes (integer). eg. `word_index["giraffe"]` might return `1234`. 97 | 98 | - __Arguments:__ 99 | 100 | - __path__: if you do have the index file locally (at `'~/.keras/datasets/' + path`), if will be downloaded to this location (in cPickle format). 101 | 102 | ## MNIST database of handwritten digits 103 | 104 | `keras.datasets.mnist` 105 | 106 | Dataset of 60,000 28x28 grayscale images of the 10 digits, along with a test set of 10,000 images. 107 | 108 | ### Usage: 109 | 110 | ```python 111 | (X_train, y_train), (X_test, y_test) = mnist.load_data() 112 | ``` 113 | 114 | - __Return:__ 115 | - 2 tuples: 116 | - __X_train, X_test__: uint8 array of grayscale image data with shape (nb_samples, 28, 28). 117 | - __y_train, y_test__: uint8 array of digit labels (integers in range 0-9) with shape (nb_samples,). 118 | 119 | - __Arguments:__ 120 | 121 | - __path__: if you do have the index file locally (at `'~/.keras/datasets/' + path`), if will be downloaded to this location (in cPickle format). 122 | -------------------------------------------------------------------------------- /keras-master/docs/sources/documentation.md: -------------------------------------------------------------------------------- 1 | # Keras Documentation Index 2 | 3 | ## Introduction 4 | 5 | - [Home](index.md) 6 | - [Index](documentation.md) 7 | - [Examples](examples.md) 8 | 9 | --- 10 | 11 | ## Base functionality 12 | 13 | - [Optimizers](optimizers.md) 14 | - [Objectives](objectives.md) 15 | - [Models](models.md) 16 | - [Activations](activations.md) 17 | - [Initializations](initializations.md) 18 | - [Datasets](datasets.md) 19 | 20 | --- 21 | 22 | ## Layers 23 | - [Core](layers/core.md) 24 | - [Convolutional](layers/convolutional.md) 25 | - [Recurrent](layers/recurrent.md) 26 | - [Advanced Activations](layers/advanced_activations.md) 27 | - [Normalization](layers/normalization.md) 28 | - [Embeddings](layers/embeddings.md) 29 | 30 | --- 31 | 32 | ## Preprocessing 33 | - [Sequence](preprocessing/sequence.md) 34 | - [Text](preprocessing/text.md) 35 | - [Image](preprocessing/image.md) 36 | 37 | --- 38 | 39 | ## Utils 40 | - [Visualization](utils/visualization.md) -------------------------------------------------------------------------------- /keras-master/docs/sources/examples.md: -------------------------------------------------------------------------------- 1 | 2 | Here are a few examples to get you started! 3 | 4 | ### Multilayer Perceptron (MLP) 5 | 6 | ```python 7 | from keras.models import Sequential 8 | from keras.layers.core import Dense, Dropout, Activation 9 | from keras.optimizers import SGD 10 | 11 | model = Sequential() 12 | model.add(Dense(20, 64, init='uniform')) 13 | model.add(Activation('tanh')) 14 | model.add(Dropout(0.5)) 15 | model.add(Dense(64, 64, init='uniform')) 16 | model.add(Activation('tanh')) 17 | model.add(Dropout(0.5)) 18 | model.add(Dense(64, 2, init='uniform')) 19 | model.add(Activation('softmax')) 20 | 21 | sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True) 22 | model.compile(loss='mean_squared_error', optimizer=sgd) 23 | 24 | model.fit(X_train, y_train, nb_epoch=20, batch_size=16) 25 | score = model.evaluate(X_test, y_test, batch_size=16) 26 | ``` 27 | 28 | --- 29 | 30 | ### Alternative implementation of MLP 31 | 32 | ```python 33 | model = Sequential() 34 | model.add(Dense(20, 64, init='uniform', activation='tanh')) 35 | model.add(Dropout(0.5)) 36 | model.add(Dense(64, 64, init='uniform', activation='tanh')) 37 | model.add(Dropout(0.5)) 38 | model.add(Dense(64, 2, init='uniform', activation='softmax')) 39 | 40 | sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True) 41 | model.compile(loss='mean_squared_error', optimizer=sgd) 42 | ``` 43 | 44 | --- 45 | 46 | ### VGG-like convnet 47 | 48 | ```python 49 | from keras.models import Sequential 50 | from keras.layers.core import Dense, Dropout, Activation, Flatten 51 | from keras.layers.convolutional import Convolution2D, MaxPooling2D 52 | from keras.optimizers import SGD 53 | 54 | model = Sequential() 55 | model.add(Convolution2D(32, 3, 3, 3, border_mode='full')) 56 | model.add(Activation('relu')) 57 | model.add(Convolution2D(32, 32, 3, 3)) 58 | model.add(Activation('relu')) 59 | model.add(MaxPooling2D(poolsize=(2, 2))) 60 | model.add(Dropout(0.25)) 61 | 62 | model.add(Convolution2D(64, 32, 3, 3, border_mode='full')) 63 | model.add(Activation('relu')) 64 | model.add(Convolution2D(64, 64, 3, 3)) 65 | model.add(Activation('relu')) 66 | model.add(MaxPooling2D(poolsize=(2, 2))) 67 | model.add(Dropout(0.25)) 68 | 69 | model.add(Flatten()) 70 | model.add(Dense(64*8*8, 256)) 71 | model.add(Activation('relu')) 72 | model.add(Dropout(0.5)) 73 | 74 | model.add(Dense(256, 10)) 75 | model.add(Activation('softmax')) 76 | 77 | sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True) 78 | model.compile(loss='categorical_crossentropy', optimizer=sgd) 79 | 80 | model.fit(X_train, Y_train, batch_size=32, nb_epoch=1) 81 | 82 | ``` 83 | 84 | --- 85 | 86 | ### Sequence classification with LSTM 87 | 88 | ```python 89 | from keras.models import Sequential 90 | from keras.layers.core import Dense, Dropout, Activation 91 | from keras.layers.embeddings import Embedding 92 | from keras.layers.recurrent import LSTM 93 | 94 | model = Sequential() 95 | model.add(Embedding(max_features, 256)) 96 | model.add(LSTM(256, 128, activation='sigmoid', inner_activation='hard_sigmoid')) 97 | model.add(Dropout(0.5)) 98 | model.add(Dense(128, 1)) 99 | model.add(Activation('sigmoid')) 100 | 101 | model.compile(loss='binary_crossentropy', optimizer='rmsprop') 102 | 103 | model.fit(X_train, Y_train, batch_size=16, nb_epoch=10) 104 | score = model.evaluate(X_test, Y_test, batch_size=16) 105 | ``` 106 | 107 | --- 108 | 109 | ### Image captioning 110 | 111 | Architecture for learning image captions with a convnet and a Gated Recurrent Unit (word-level embedding, caption of maximum length 16 words). 112 | 113 | Note that getting this to actually "work" will require using a bigger convnet, initialized with pre-trained weights. 114 | Displaying readable results will also require an embedding decoder. 115 | 116 | ```python 117 | max_caption_len = 16 118 | 119 | model = Sequential() 120 | model.add(Convolution2D(32, 3, 3, 3, border_mode='full')) 121 | model.add(Activation('relu')) 122 | model.add(Convolution2D(32, 32, 3, 3)) 123 | model.add(Activation('relu')) 124 | model.add(MaxPooling2D(poolsize=(2, 2))) 125 | 126 | model.add(Convolution2D(64, 32, 3, 3, border_mode='full')) 127 | model.add(Activation('relu')) 128 | model.add(Convolution2D(64, 64, 3, 3)) 129 | model.add(Activation('relu')) 130 | model.add(MaxPooling2D(poolsize=(2, 2))) 131 | 132 | model.add(Convolution2D(128, 64, 3, 3, border_mode='full')) 133 | model.add(Activation('relu')) 134 | model.add(Convolution2D(128, 128, 3, 3)) 135 | model.add(Activation('relu')) 136 | model.add(MaxPooling2D(poolsize=(2, 2))) 137 | 138 | model.add(Flatten()) 139 | model.add(Dense(128*4*4, 256)) 140 | model.add(Activation('relu')) 141 | model.add(Dropout(0.5)) 142 | 143 | model.add(Repeat(max_caption_len)) 144 | # the GRU below returns sequences of max_caption_len vectors of size 256 (our word embedding size) 145 | model.add(GRU(256, 256, return_sequences=True)) 146 | 147 | model.compile(loss='mean_squared_error', optimizer='rmsprop') 148 | 149 | # "images" is a numpy array of shape (nb_samples, nb_channels=3, width, height) 150 | # "captions" is a numpy array of shape (nb_samples, max_caption_len=16, embedding_dim=256) 151 | # captions are supposed already embedded (dense vectors). 152 | model.fit(images, captions, batch_size=16, nb_epoch=100) 153 | 154 | ``` 155 | 156 | --- 157 | 158 | In the [examples folder](https://github.com/fchollet/keras/tree/master/examples), you will find example models for real datasets: 159 | 160 | - CIFAR10 small images classification: Convnet with realtime data augmentation 161 | - IMDB movie review sentiment classification: LSTM over sequences of words 162 | - Reuters newswires topic classification: Multilayer Perceptron 163 | -------------------------------------------------------------------------------- /keras-master/docs/sources/initializations.md: -------------------------------------------------------------------------------- 1 | 2 | ## Usage of initializations 3 | 4 | Initializations define the probability distribution used to set the initial random weights of Keras layers. 5 | 6 | The keyword arguments used for passing initializations to layers will depend on the layer. Usually it is simply `init`: 7 | 8 | ```python 9 | model.add(Dense(64, 64, init='uniform')) 10 | ``` 11 | 12 | ## Available initializations 13 | 14 | - __uniform__ 15 | - __lecun_uniform__: Uniform initialization scaled by the square root of the number of inputs (LeCun 98). 16 | - __normal__ 17 | - __identity__: Use with square 2D layers (`shape[0] == shape[1]`). 18 | - __orthogonal__: Use with square 2D layers (`shape[0] == shape[1]`). 19 | - __zero__ 20 | - __glorot_normal__: Gaussian initialization scaled by fan_in + fan_out (Glorot 2010) 21 | - __glorot_uniform__ 22 | - __he_normal__: Gaussian initialization scaled by fan_in (He et al., 2014) 23 | - __he_uniform__ 24 | -------------------------------------------------------------------------------- /keras-master/docs/sources/layers/advanced_activations.md: -------------------------------------------------------------------------------- 1 | 2 | ## LeakyReLU 3 | 4 | ```python 5 | keras.layers.advanced_activations.LeakyReLU(alpha=0.3) 6 | ``` 7 | 8 | Special version of a Rectified Linear Unit that allows a small gradient when the unit is not active (`f(x) = alpha*x for x < 0`). 9 | 10 | - __Input shape__: This layer does not assume a specific input shape. As a result, it cannot be used as the first layer in a model. 11 | 12 | - __Output shape__: Same as input. 13 | 14 | - __Arguments__: 15 | - __alpha__: float >= 0. Negative slope coefficient. 16 | 17 | --- 18 | 19 | ## PReLU 20 | 21 | ```python 22 | keras.layers.advanced_activations.PReLU(input_shape) 23 | ``` 24 | 25 | Parametrized linear unit. Similar to a LeakyReLU, where each input unit has its alpha coefficient, and where these coefficients are learned during training. 26 | 27 | - __Input shape__: Same as `input_shape`. This layer cannot be used as first layer in a model. 28 | 29 | - __Output shape__: Same as input. 30 | 31 | - __Arguments__: 32 | - __input_shape__: tuple. 33 | 34 | - __References__: 35 | - [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](http://arxiv.org/pdf/1502.01852v1.pdf) -------------------------------------------------------------------------------- /keras-master/docs/sources/layers/containers.md: -------------------------------------------------------------------------------- 1 | Containers are ensembles of layers that can be interacted with through the same API as `Layer` objects. 2 | 3 | ## Sequential 4 | 5 | ```python 6 | keras.layers.containers.Sequential(layers=[]) 7 | ``` 8 | 9 | The Sequential container is a linear stack of layers. Apart from the `add` methods and the `layers` constructor argument, the API is identical to that of the `Layer` class. 10 | 11 | This class is also the basis for the `keras.models.Sequential` architecture. 12 | 13 | The `layers` constructor argument is a list of Layer instances. 14 | 15 | __Methods__: 16 | 17 | ```python 18 | add(layer) 19 | ``` 20 | 21 | Add a new layer to the stack. -------------------------------------------------------------------------------- /keras-master/docs/sources/layers/embeddings.md: -------------------------------------------------------------------------------- 1 | 2 | ## Embedding 3 | 4 | ```python 5 | keras.layers.embeddings.Embedding(input_dim, output_dim, init='uniform', weights=None, W_regularizer=None, W_constraint=None) 6 | ``` 7 | 8 | Turn positive integers (indexes) into denses vectors of fixed size, 9 | eg. `[[4], [20]] -> [[0.25, 0.1], [0.6, -0.2]]` 10 | 11 | - __Input shape__: 2D tensor with shape: `(nb_samples, maxlen)`. 12 | 13 | - __Output shape__: 3D tensor with shape: `(nb_samples, maxlen, output_dim)`. 14 | 15 | - __Arguments__: 16 | 17 | - __input_dim__: int >= 0. Size of the vocabulary, ie. 1+maximum integer index occuring in the input data. 18 | - __output_dim__: int >= 0. Dimension of the dense embedding. 19 | - __init__: name of initialization function for the weights of the layer (see: [initializations](../initializations.md)), or alternatively, Theano function to use for weights initialization. This parameter is only relevant if you don't pass a `weights` argument. 20 | - __weights__: list of numpy arrays to set as initial weights. The list should have 1 element, of shape `(input_dim, output_dim)`. 21 | - __W_regularizer__: instance of the [regularizers](../regularizers.md) module (eg. L1 or L2 regularization), applied to the embedding matrix. 22 | - __W_constraint__: instance of the [constraints](../constraints.md) module (eg. maxnorm, nonneg), applied to the embedding matrix. 23 | 24 | 25 | ## WordContextProduct 26 | 27 | ```python 28 | keras.layers.embeddings.WordContextProduct(input_dim, proj_dim=128, 29 | init='uniform', activation='sigmoid', weights=None) 30 | ``` 31 | 32 | This layer turns a pair of words (a pivot word + a context word, ie. a word from the same context as a pivot, or a random, out-of-context word), indentified by their indices in a vocabulary, into two dense reprensentations (word representation and context representation). 33 | 34 | Then it returns `activation(dot(pivot_embedding, context_embedding))`, which can be trained to encode the probability of finding the context word in the context of the pivot word (or reciprocally depending on your training procedure). 35 | 36 | For more context, see Mikolov et al.: [Efficient Estimation of Word reprensentations in Vector Space](http://arxiv.org/pdf/1301.3781v3.pdf) 37 | 38 | - __Input shape__: 2D tensor with shape: `(nb_samples, 2)`. 39 | 40 | - __Output shape__: 2D tensor with shape: `(nb_samples, 1)`. 41 | 42 | - __Arguments__: 43 | 44 | - __input_dim__: int >= 0. Size of the vocabulary, ie. 1+maximum integer index occuring in the input data. 45 | - __proj_dim__: int >= 0. Dimension of the dense embedding used internally. 46 | - __init__: name of initialization function for the embeddings (see: [initializations](../initializations.md)), or alternatively, Theano function to use for weights initialization. This parameter is only relevant if you don't pass a `weights` argument. 47 | - __activation__: name of activation function to use (see: [activations](../activations.md)), or alternatively, elementwise Theano function. 48 | - __weights__: list of numpy arrays to set as initial weights. The list should have 2 element, both of shape `(input_dim, proj_dim)`. The first element is the word embedding weights, the second one is the context embedding weights. 49 | -------------------------------------------------------------------------------- /keras-master/docs/sources/layers/noise.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## GaussianNoise 4 | ```python 5 | keras.layers.noise.GaussianNoise(sigma) 6 | ``` 7 | Apply to the input an additive zero-centred gaussian noise with standard deviation `sigma`. This is useful to mitigate overfitting (you could see it as a kind of random data augmentation). Gaussian Noise (GS) is a natural choice as corruption process for real valued inputs. 8 | 9 | The Gaussian noise is only added at training time. 10 | 11 | - __Input shape__: This layer does not assume a specific input shape. 12 | 13 | - __Output shape__: Same as input. 14 | 15 | - __Arguments__: 16 | 17 | - __sigma__: float, standard deviation of the noise distribution. 18 | 19 | --- 20 | 21 | ## GaussianDropout 22 | ```python 23 | keras.layers.noise.GaussianDropout(p) 24 | ``` 25 | Apply to the input an multiplicative one-centred gaussian noise with standard deviation `sqrt(p/(1-p))`. p refers to drop probability to match Dropout layer syntax. 26 | 27 | http://www.cs.toronto.edu/~rsalakhu/papers/srivastava14a.pdf 28 | 29 | The Gaussian noise is only used at training time. 30 | 31 | - __Input shape__: This layer does not assume a specific input shape. 32 | 33 | - __Output shape__: Same as input. 34 | 35 | - __Arguments__: 36 | 37 | - __p__: float, drop probability as with Dropout. 38 | 39 | -------------------------------------------------------------------------------- /keras-master/docs/sources/layers/normalization.md: -------------------------------------------------------------------------------- 1 | 2 | ## BatchNormalization 3 | 4 | ```python 5 | keras.layers.normalization.BatchNormalization(input_shape, epsilon=1e-6, weights=None) 6 | ``` 7 | 8 | Normalize the activations of the previous layer at each batch. 9 | 10 | - __Input shape__: Same as `input_shape`. This layer cannot be used as first layer in a model. 11 | 12 | - __Output shape__: Same as input. 13 | 14 | - __Arguments__: 15 | - __input_shape__: tuple. 16 | - __epsilon__: small float > 0. Fuzz parameter. 17 | - __weights__: Initialization weights. List of 2 numpy arrays, with shapes: `[(input_shape,), (input_shape,)]` 18 | 19 | - __References__: 20 | - [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](http://arxiv.org/pdf/1502.03167v3.pdf) -------------------------------------------------------------------------------- /keras-master/docs/sources/objectives.md: -------------------------------------------------------------------------------- 1 | 2 | ## Usage of objectives 3 | 4 | An objective function (or loss function, or optimization score function) is one of the two parameters required to compile a model: 5 | 6 | ```python 7 | model.compile(loss='mean_squared_error', optimizer='sgd') 8 | ``` 9 | 10 | You can either pass the name of an existing objective, or pass a Theano symbolic function that returns a scalar and takes the following two arguments: 11 | 12 | - __y_true__: True labels. Theano tensor. 13 | - __y_pred__: Predictions. Theano tensor of the same shape as y_true. 14 | 15 | For a few examples of such functions, check out the [objectives source](https://github.com/fchollet/keras/blob/master/keras/objectives.py). 16 | 17 | ## Available objectives 18 | 19 | - __mean_squared_error__ / __mse__ 20 | - __mean_absolute_error__ / __mae__ 21 | - __mean_absolute_percentage_error__ / __mape__ 22 | - __mean_squared_logarithmic_error__ / __msle__ 23 | - __squared_hinge__ 24 | - __hinge__ 25 | - __binary_crossentropy__: Also known as logloss. 26 | - __categorical_crossentropy__: Also known as multiclass logloss. __Note__: using this objective requires that your labels are binary arrays of shape `(nb_samples, nb_classes)`. -------------------------------------------------------------------------------- /keras-master/docs/sources/optimizers.md: -------------------------------------------------------------------------------- 1 | 2 | ## Usage of optimizers 3 | 4 | An optimizer is one of the two arguments required for compiling a Keras model: 5 | 6 | ```python 7 | model = Sequential() 8 | model.add(Dense(20, 64, init='uniform')) 9 | model.add(Activation('tanh')) 10 | model.add(Activation('softmax')) 11 | 12 | sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True) 13 | model.compile(loss='mean_squared_error', optimizer=sgd) 14 | ``` 15 | 16 | You can either instantiate an optimizer before passing it to `model.compile()` , as in the above example, or you can call it by its name. In the latter case, the default parameters for the optimizer will be used. 17 | 18 | ```python 19 | # pass optimizer by name: default parameters will be used 20 | model.compile(loss='mean_squared_error', optimizer='sgd') 21 | ``` 22 | 23 | --- 24 | 25 | ## Base class 26 | 27 | ```python 28 | keras.optimizers.Optimizer(**kwargs) 29 | ``` 30 | 31 | All optimizers descended from this class support the following keyword argument: 32 | 33 | - __clipnorm__: float >= 0. 34 | 35 | Note: this is base class for building optimizers, not an actual optimizer that can be used for training models. 36 | 37 | --- 38 | 39 | ## SGD 40 | 41 | ```python 42 | keras.optimizers.SGD(lr=0.01, momentum=0., decay=0., nesterov=False) 43 | ``` 44 | 45 | __Arguments__: 46 | 47 | - __lr__: float >= 0. Learning rate. 48 | - __momentum__: float >= 0. Parameter updates momentum. 49 | - __decay__: float >= 0. Learning rate decay over each update. 50 | - __nesterov__: boolean. Whether to apply Nesterov momentum. 51 | 52 | --- 53 | 54 | ## Adagrad 55 | 56 | ```python 57 | keras.optimizers.Adagrad(lr=0.01, epsilon=1e-6) 58 | ``` 59 | 60 | It is recommended to leave the parameters of this optimizer at their default values. 61 | 62 | __Arguments__: 63 | 64 | - __lr__: float >= 0. Learning rate. 65 | - __epsilon__: float >= 0. 66 | 67 | --- 68 | 69 | ## Adadelta 70 | 71 | ```python 72 | keras.optimizers.Adadelta(lr=1.0, rho=0.95, epsilon=1e-6) 73 | ``` 74 | 75 | It is recommended to leave the parameters of this optimizer at their default values. 76 | 77 | __Arguments__: 78 | 79 | - __lr__: float >= 0. Learning rate. It is recommended to leave it at the default value. 80 | - __rho__: float >= 0. 81 | - __epsilon__: float >= 0. Fuzz factor. 82 | 83 | For more info, see *"Adadelta: an adaptive learning rate method"* by Matthew Zeiler. 84 | 85 | --- 86 | 87 | ## RMSprop 88 | 89 | ```python 90 | keras.optimizers.RMSprop(lr=0.001, rho=0.9, epsilon=1e-6) 91 | ``` 92 | 93 | It is recommended to leave the parameters of this optimizer at their default values. 94 | 95 | __Arguments__: 96 | 97 | - __lr__: float >= 0. Learning rate. 98 | - __rho__: float >= 0. 99 | - __epsilon__: float >= 0. Fuzz factor. 100 | 101 | --- 102 | 103 | ## Adam 104 | 105 | ```python 106 | keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-8, kappa=1-1e-8) 107 | ``` 108 | 109 | Adam optimizer, proposed by Kingma and Lei Ba in [Adam: A Method For Stochastic Optimization](http://arxiv.org/pdf/1412.6980v4.pdf). Default parameters are those suggested in the paper. The parameter "lambda" from the paper has been renamed kappa, for syntactic reasons. 110 | 111 | __Arguments__: 112 | 113 | - __lr__: float >= 0. Learning rate. 114 | - __beta_1__, __beta_2__: floats, 0 < beta < 1. Generally close to 1. 115 | - __epsilon__: float >= 0. Fuzz factor. 116 | - __kappa__: float 0 < kappa < 1. Lambda parameter in the original paper. 117 | 118 | --- -------------------------------------------------------------------------------- /keras-master/docs/sources/preprocessing/image.md: -------------------------------------------------------------------------------- 1 | 2 | ## ImageDataGenerator 3 | 4 | ```python 5 | keras.preprocessing.image.ImageDataGenerator(featurewise_center=True, 6 | samplewise_center=False, 7 | featurewise_std_normalization=True, 8 | samplewise_std_normalization=False, 9 | zca_whitening=False, 10 | rotation_range=0., 11 | width_shift_range=0., 12 | height_shift_range=0., 13 | horizontal_flip=False, 14 | vertical_flip=False) 15 | ``` 16 | 17 | Generate batches of tensor image data with real-time data augmentation. 18 | 19 | - __Arguments__: 20 | - __featurewise_center__: Boolean. Set input mean to 0 over the dataset. 21 | - __samplewise_center__: Boolean. Set each sample mean to 0. 22 | - __featurewise_std_normalization__: Boolean. Divide inputs by std of the dataset. 23 | - __samplewise_std_normalization__: Boolean. Divide each input by its std. 24 | - __zca_whitening__: Boolean. Apply ZCA whitening. 25 | - __rotation_range__: Int. Degree range for random rotations. 26 | - __width_shift_range__: Float (fraction of total width). Range for random horizontal shifts. 27 | - __height_shift_range__: Float (fraction of total height). Range for random vertical shifts. 28 | - __horizontal_flip__: Boolean. Randomly flip inputs horizontally. 29 | - __vertical_flip__: Boolean. Randomly flip inputs vertically. 30 | 31 | - __Methods__: 32 | - __fit(X)__: Required if featurewise_center or featurewise_std_normalization or zca_whitening. Compute necessary quantities on some sample data. 33 | - __Arguments__: 34 | - __X__: sample data. 35 | - __augment__: Boolean (default: False). Whether to fit on randomly augmented samples. 36 | - __rounds__: int (default: 1). If augment, how many augmentation passes over the data to use. 37 | - __flow(X, y)__: 38 | - __Arguments__: 39 | - __X__: data. 40 | - __y__: labels. 41 | - __batch_size__: int (default: 32). 42 | - __shuffle__: boolean (defaut: False). 43 | - __save_to_dir__: None or str. This allows you to optimally specify a directory to which to save the augmented pictures being generated (useful for visualizing what you are doing). 44 | - __save_prefix__: str. Prefix to use for filenames of saved pictures. 45 | - __save_format__: one of "png", jpeg". 46 | 47 | - __Example__: 48 | ```python 49 | (X_train, y_train), (X_test, y_test) = cifar10.load_data(test_split=0.1) 50 | Y_train = np_utils.to_categorical(y_train, nb_classes) 51 | Y_test = np_utils.to_categorical(y_test, nb_classes) 52 | 53 | datagen = ImageDataGenerator( 54 | featurewise_center=True, 55 | featurewise_std_normalization=True, 56 | rotation_range=20, 57 | width_shift_range=0.2, 58 | height_shift_range=0.2, 59 | horizontal_flip=True) 60 | 61 | # compute quantities required for featurewise normalization 62 | # (std, mean, and principal components if ZCA whitening is applied) 63 | datagen.fit(X_train) 64 | 65 | for e in range(nb_epoch): 66 | print 'Epoch', e 67 | # batch train with realtime data augmentation 68 | for X_batch, Y_batch in datagen.flow(X_train, Y_train): 69 | loss = model.train(X_batch, Y_batch) 70 | ``` -------------------------------------------------------------------------------- /keras-master/docs/sources/preprocessing/sequence.md: -------------------------------------------------------------------------------- 1 | ## pad_sequences 2 | 3 | ```python 4 | keras.preprocessing.sequence.pad_sequences(sequences, maxlen=None, dtype='int32') 5 | ``` 6 | 7 | Transform a list of `nb_samples sequences` (lists of scalars) into a 2D numpy array of shape `(nb_samples, nb_timesteps)`. `nb_timesteps` is either the `maxlen` argument if provided, or the length of the longest sequence otherwise. Sequences that are shorter than `nb_timesteps` are padded with zeros at the end. 8 | 9 | - __Return__: 2D numpy array of shape `(nb_samples, nb_timesteps)`. 10 | 11 | - __Arguments__: 12 | - __sequences__: List of lists of int or float. 13 | - __maxlen__: None or int. Maximum sequence length, longer sequences are truncated and shorter sequences are padded with zeros at the end. 14 | - __dtype__: datatype of the numpy array returned. 15 | 16 | --- 17 | 18 | ## skipgrams 19 | 20 | ```python 21 | keras.preprocessing.sequence.skipgrams(sequence, vocabulary_size, 22 | window_size=4, negative_samples=1., shuffle=True, 23 | categorical=False, sampling_table=None) 24 | ``` 25 | 26 | Transforms a sequence of word indexes (list of int) into couples of the form: 27 | 28 | - (word, word in the same window), with label 1 (positive samples). 29 | - (word, random word from the vocabulary), with label 0 (negative samples). 30 | 31 | Read more about Skipgram in this gnomic paper by Mikolov et al.: [Efficient Estimation of Word Representations in 32 | Vector Space](http://arxiv.org/pdf/1301.3781v3.pdf) 33 | 34 | - __Return__: tuple `(couples, labels)`. 35 | - `couples` is a list of 2-elements lists of int: `[word_index, other_word_index]`. 36 | - `labels` is a list of 0 and 1, where 1 indicates that `other_word_index` was found in the same window as `word_index`, and 0 indicates that `other_word_index` was random. 37 | - if categorical is set to True, the labels are categorical, ie. 1 becomes [0,1], and 0 becomes [1, 0]. 38 | 39 | - __Arguments__: 40 | - __sequence__: list of int indexes. If using a sampling_table, the index of a word should be its the rank in the dataset (starting at 1). 41 | - __vocabulary_size__: int. 42 | - __window_size__: int. maximum distance between two words in a positive couple. 43 | - __negative_samples__: float >= 0. 0 for no negative (=random) samples. 1 for same number as positive samples. etc. 44 | - __shuffle__: boolean. Whether to shuffle the samples. 45 | - __categorical__: boolean. Whether to make the returned labels categorical. 46 | - __sampling_table__: numpy array of shape `(vocabulary_size,)` where `sampling_table[i]` is the probability of sampling the word with index i (assumed to be i-th most common word in the dataset). 47 | 48 | 49 | --- 50 | 51 | ## make_sampling_table 52 | 53 | ```python 54 | keras.preprocessing.sequence.make_sampling_table(size, sampling_factor=1e-5) 55 | ``` 56 | 57 | Used for generating the `sampling_table` argument for `skipgrams`. `sampling_table[i]` is the probability of sampling the word i-th most common word in a dataset (more common words should be sampled less frequently, for balance). 58 | 59 | - __Return__: numpy array of shape `(size,)`. 60 | 61 | - __Arguments__: 62 | - __size__: size of the vocabulary considered. 63 | - __sampling_factor__: lower values result in a longer probability decay (common words will be sampled less frequently). If set to 1, no subsampling will be performed (all sampling probabilities will be 1). 64 | -------------------------------------------------------------------------------- /keras-master/docs/sources/preprocessing/text.md: -------------------------------------------------------------------------------- 1 | 2 | ## text_to_word_sequence 3 | 4 | ```python 5 | keras.preprocessing.text.text_to_word_sequence(text, 6 | filters=base_filter(), lower=True, split=" ") 7 | ``` 8 | 9 | Split a sentence into a list of words. 10 | 11 | - __Return__: List of words (str). 12 | 13 | - __Arguments__: 14 | - __text__: str. 15 | - __filters__: list (or concatenation) of characters to filter out, such as punctuation. Default: base_filter(), includes basic punctuation, tabs, and newlines. 16 | - __lower__: boolean. Whether to set the text to lowercase. 17 | - __split__: str. Separator for word splitting. 18 | 19 | ## one_hot 20 | 21 | ```python 22 | keras.preprocessing.text.one_hot(text, n, 23 | filters=base_filter(), lower=True, split=" ") 24 | ``` 25 | 26 | One-hot encode a text into a list of word indexes in a vocabulary of size n. 27 | 28 | - __Return__: List of integers in [1, n]. Each integer encodes a word (unicity non-guaranteed). 29 | 30 | - __Arguments__: Same as `text_to_word_sequence` above. 31 | - __n__: int. Size of vocabulary. 32 | 33 | ## Tokenizer 34 | 35 | ```python 36 | keras.preprocessing.text.Tokenizer(nb_words=None, filters=base_filter(), 37 | lower=True, split=" ") 38 | ``` 39 | 40 | Class for vectorizing texts, or/and turning texts into sequences (=list of word indexes, where the word of rank i in the dataset (starting at 1) has index i). 41 | 42 | - __Arguments__: Same as `text_to_word_sequence` above. 43 | - __nb_words__: None or int. Maximum number of words to work with (if set, tokenization will be restricted to the top nb_words most common words in the dataset). 44 | 45 | - __Methods__: 46 | 47 | - __fit_on_texts(texts)__: 48 | - __Arguments__: 49 | - __texts__: list of texts to train on. 50 | 51 | - __texts_to_sequences(texts)__ 52 | - __Arguments__: 53 | - __texts__: list of texts to turn to sequences. 54 | - __Return__: list of sequences (one per text input). 55 | 56 | - __texts_to_sequences_generator(texts)__: generator version of the above. 57 | - __Return__: yield one sequence per input text. 58 | 59 | - __texts_to_matrix(texts)__: 60 | - __Return__: numpy array of shape `(len(texts), nb_words)`. 61 | - __Arguments__: 62 | - __texts__: list of texts to vectorize. 63 | - __mode__: one of "binary", "count", "tfidf", "freq" (default: "binary"). 64 | 65 | - __fit_on_sequences(sequences)__: 66 | - __Arguments__: 67 | - __sequences__: list of sequences to train on. 68 | 69 | - __sequences_to_matrix(sequences)__: 70 | - __Return__: numpy array of shape `(len(sequences), nb_words)`. 71 | - __Arguments__: 72 | - __sequences__: list of sequences to vectorize. 73 | - __mode__: one of "binary", "count", "tfidf", "freq" (default: "binary"). 74 | 75 | - __Attributes__: 76 | - __word_counts__: dictionary mapping words (str) to the number of times they appeared on during fit. Only set after fit_on_texts was called. 77 | - __word_docs__: dictionary mapping words (str) to the number of documents/texts they appeared on during fit. Only set after fit_on_texts was called. 78 | - __word_index__: dictionary mapping words (str) to their rank/index (int). Only set after fit_on_texts was called. 79 | - __document_count__: int. Number of documents (texts/sequences) the tokenizer was trained on. Only set after fit_on_texts or fit_on_sequences was called. 80 | 81 | 82 | -------------------------------------------------------------------------------- /keras-master/docs/sources/regularizers.md: -------------------------------------------------------------------------------- 1 | ## Usage of regularizers 2 | 3 | Regularizers allow to apply penalties on layer parameters or layer activity during optimization. These penalties are incorporated in the loss function that the network optimizes. 4 | 5 | The penalties are applied on a per-layer basis. The exact API will depend on the layer, but the layers `Dense`, `TimeDistributedDense`, `MaxoutDense`, `Convolution1D` and `Convolution2D` have a unified API. 6 | 7 | These layers expose 3 keyword arguments: 8 | 9 | - `W_regularizer`: instance of `keras.regularizers.WeightRegularizer` 10 | - `b_regularizer`: instance of `keras.regularizers.WeightRegularizer` 11 | - `activity_regularizer`: instance of `keras.regularizers.ActivityRegularizer` 12 | 13 | 14 | ## Example 15 | 16 | ```python 17 | from keras.regularizers import l2, activity_l2 18 | model.add(Dense(64, 64, W_regularizer=l2(0.01), activity_regularizer=activity_l2(0.01))) 19 | ``` 20 | 21 | ## Available penalties 22 | 23 | ```python 24 | keras.regularizers.WeightRegularizer(l1=0., l2=0.) 25 | ``` 26 | 27 | ```python 28 | keras.regularizers.ActivityRegularizer(l1=0., l2=0.) 29 | ``` 30 | 31 | ## Shortcuts 32 | 33 | These are shortcut functions available in `keras.regularizers`. 34 | 35 | - __l1__(l=0.01): L1 weight regularization penalty, also known as LASSO 36 | - __l2__(l=0.01): L2 weight regularization penalty, also known as weight decay, or Ridge 37 | - __l1l2__(l1=0.01, l2=0.01): L1-L2 weight regularization penalty, also known as ElasticNet 38 | - __activity_l1__(l=0.01): L1 activity regularization 39 | - __activity_l2__(l=0.01): L2 activity regularization 40 | - __activity_l1l2__(l1=0.01, l2=0.01): L1+L2 activity regularization 41 | -------------------------------------------------------------------------------- /keras-master/docs/sources/utils/visualization.md: -------------------------------------------------------------------------------- 1 | ## Grapher 2 | 3 | Creates a visualization of the model structure using `pydot`. 4 | 5 | ```python 6 | grapher = keras.utils.dot_utils.Grapher() 7 | ``` 8 | - __Methods__: 9 | - __plot__(model, to_file): creates a graph visualizing the structure of `model` and writes it to `to_file`. 10 | - __Arguments__: 11 | - __model__: an instance of a Keras model (e.g. `Sequential`) 12 | - __to_file__: the filename to save the visualization png to. 13 | 14 | __Examples__: 15 | 16 | ```python 17 | from keras.models import Sequential 18 | from keras.layers.core import Dense, Activation 19 | from keras.utils.dot_utils import Grapher 20 | 21 | grapher = Grapher() 22 | 23 | model = Sequential() 24 | model.add(Dense(64, 2, init='uniform')) 25 | model.add(Activation('softmax')) 26 | grapher.plot(model, 'model.png') 27 | ``` 28 | 29 | -------------------------------------------------------------------------------- /keras-master/examples/SimpleNet.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import numpy as np 4 | np.random.seed(1337) # for reproducibility 5 | 6 | from keras.datasets import reuters 7 | from keras.models import Sequential 8 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul 9 | from keras.layers.normalization import BatchNormalization 10 | from keras.utils import np_utils 11 | from keras.preprocessing.text import Tokenizer 12 | 13 | 14 | 15 | import theano 16 | 17 | def inspect_inputs(i, node, fn): 18 | print (i) 19 | print (node) 20 | print ("input(s) value(s):") 21 | print ([input[0] for input in fn.inputs]) 22 | 23 | def inspect_outputs(i, node, fn): 24 | print( "output(s) value(s):", [output[0] for output in fn.outputs]) 25 | 26 | 27 | def load_dataset(): 28 | #user= np.array([[1,0],[1,0],[0,1]]) 29 | #y_train=np.array([[1,0],[1,0],[1,0]]) 30 | #Items=np.array( [ [[1,0],[0,1]] , [[.5,0],[0,1]],[[-1,1],[1,0]] ]) 31 | user= np.array([[1,1,1],[1,3,1],[0,1,0],[0,2,-1]]) 32 | y_train=np.array([[1,0],[1,0],[1,0],[1,0]]) 33 | Items=np.array( [ [[1,2,0],[0,2,0]] , [[2,2,1],[2,0,2]],[[0,1,2],[1,0,0]],[[1,3,3],[1,3,-1]] ]) 34 | #user= np.array([[0,1]]) 35 | #y_train=np.array([[1,0]]) 36 | #Items=np.array( [[[-1,1],[1,0]]]) 37 | # The inputs come as vectors, we reshape them to monochrome 2D images, 38 | # according to the shape convention: (examples, channels, rows, columns) 39 | user.reshape(-1,3); 40 | # We just return all the arrays in order, as expected in main(). 41 | # (It doesn't matter how we do this as long as we can read them again.) 42 | return (user ,Items, y_train) 43 | 44 | 45 | print("Loading data...") 46 | user ,Items, y_train = load_dataset() 47 | print(len(user), 'train sequences') 48 | 49 | print('user_train shape:', user.shape) 50 | print('Item shape:', Items.shape) 51 | userModel = Sequential() 52 | userModel.add(Dense(3, 3)) 53 | userModel.add(Activation('tanh')) 54 | userModel.add(Dropout(0.1)) 55 | userModel.add(Dense(3, 2)) 56 | userModel.add(Activation('tanh')) 57 | 58 | itemModel = Sequential() 59 | itemModel.add(TimeDistributedDense(3, 3)) 60 | itemModel.add(Activation('tanh')) 61 | itemModel.add(Dropout(0.1)) 62 | itemModel.add(TimeDistributedDense(3, 2)) 63 | itemModel.add(Activation('tanh')) 64 | ##itemModel.add(Reshape(4)) 65 | ##itemModel.add(Dense(4, 2)) 66 | model=Sequential() 67 | model.add(ElementMul([userModel,itemModel])) #should output 2 values 68 | model.add(TimeDistributedDense(2, 1)) 69 | ##model.add(Activation('normalization')) 70 | model.add(Reshape(2)) 71 | y_score= model.get_output(train=False) 72 | x_test=model.get_input(train=False) 73 | model.add(Activation('softmax')) 74 | ##model.add(Merge([userModel, itemModel], mode='sum')) 75 | 76 | 77 | print('done model construction') 78 | model.compile(loss='categorical_crossentropy', optimizer='Adadelta') 79 | print('done complie') 80 | scoring= theano.function(x_test,y_score, 81 | allow_input_downcast=True, mode=None) 82 | history = model.fit([user ,Items] ,y_train, nb_epoch=5, batch_size=1024, verbose=2, show_accuracy=True) 83 | 84 | #history = model.train_on_batch([user ,Items] ,y_train,accuracy=True)# nb_epoch=10, batch_size=1024, verbose=2, show_accuracy=True) 85 | print('done training') 86 | #user_test ,Items_test, y_test = load_dataset(r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.userstest100k.centered",r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.itemstest100k",r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.itemstest100k.fakeneg",50781) 87 | y_p=model.predict([user,Items]) 88 | y_pp=model.custom_predict([user,Items],scoring) 89 | print('done score compile') 90 | 91 | pfile=open(r"C:\Users\t-alie\Downloads\movieLens_1M\yp","w") 92 | for y in y_p: 93 | pfile.write("%s\n" %y ) 94 | 95 | for y in y_pp: 96 | pfile.write("%s\n" %y ) 97 | 98 | 99 | pfile.close() 100 | 101 | print('done prediction') 102 | #model.save_weights(r'f:\1b.model') -------------------------------------------------------------------------------- /keras-master/examples/TempRecSys .py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import numpy as np 4 | from itertools import repeat 5 | import csv 6 | np.random.seed(1337) # for reproducibility 7 | 8 | from keras.preprocessing import sequence 9 | from keras.optimizers import SGD, RMSprop, Adagrad 10 | from keras.utils import np_utils 11 | from keras.models import Sequential 12 | from keras.layers.core import Dense, Dropout, Activation 13 | from keras.layers.core import TimeDistributedDense 14 | from keras.layers.embeddings import Embedding 15 | from keras.layers.recurrent import LSTM, GRU 16 | from keras.datasets import imdb 17 | 18 | #input is 4*embeddingdim 19 | #output is embeddingdim 20 | def loadData(file=r'c:\f\f\cortana_lstm_new_same_length.txt', num_features=61): 21 | x_train = [] 22 | y_train = [] 23 | current_x = [] 24 | current_y = [] 25 | current_index = 1 26 | with open(file, 'rb') as input_file: 27 | input = csv.reader(input_file, delimiter='\t') 28 | for row in input: 29 | print(row[0]) 30 | if int(row[0])!=current_index: 31 | # a new training instance 32 | print("new") 33 | x_train.append(current_x) 34 | y_train.append(current_y) 35 | current_index=int(row[0]) 36 | current_x = [] 37 | current_y = [] 38 | else: 39 | print("old") 40 | current_x.append(map(int, row[2:2+num_features])) 41 | current_y.append(int(row[1])) 42 | # don't forget the last instance 43 | x_train.append(current_x) 44 | y_train.append((current_y)) 45 | return x_train, y_train 46 | 47 | 48 | 49 | if __name__ == "__main__": 50 | numfeatures = 61 # the input feature size 51 | youtput = 1 # the output prediction (only predict 0 or 1) 52 | embeddingdim=300 53 | x_train, y_train= loadData(num_features=numfeatures) 54 | print(len(x_train), 'train sequences') 55 | #print(x_train[0]) 56 | #print(x_train[1]) 57 | # padding sequence 58 | print("padding sequences") 59 | #x_train = sequence.pad_sequences(x_train,maxlen= 20) 60 | x_train = np.array(x_train) 61 | y_train = np.array(y_train) 62 | print(x_train.shape, ' x_train shape') 63 | print(y_train.shape, ' y_train shape') 64 | print(y_train) 65 | print('Build model...') 66 | 67 | model_user_Item_globalItem = Sequential() 68 | model_user_Item_globalItem.add(Dense(embeddingdim*3, embeddingdim)) 69 | model_temp_userItem = Sequential() 70 | model_temp_userItem.add(LSTM(embeddingdim, embeddingdim, return_sequences=False)) # try using a GRU instead, for fun 71 | #model.add(Dropout(0.5)) 72 | 73 | model=Sequential() 74 | model.add(Merge([model_user_Item_globalItem, model_temp_userItem], mode='concat')) 75 | model.add(model.add(Dense(2*embeddingdim, embeddingdim))) #embedding *4 x embedding 76 | # try using different optimizers and different optimizer configs 77 | model.compile(loss='cos_loss', optimizer='adam') # migth need to implement cosine loss (1-T.dot( y_prd ,y_true) / T.dot(y_prd,y_prd) 78 | 79 | print("Train...") 80 | model.fit(x_train, y_train, batch_size=1, nb_epoch=4, validation_data=(x_train, y_train), show_accuracy=True) 81 | #score, acc = model.evaluate(X_test, y_test, batch_size=batch_size, show_accuracy=True) 82 | 83 | -------------------------------------------------------------------------------- /keras-master/examples/TempRecSys.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import numpy as np 4 | from itertools import repeat 5 | import csv 6 | np.random.seed(1337) # for reproducibility 7 | 8 | from keras.preprocessing import sequence 9 | from keras.optimizers import SGD, RMSprop, Adagrad 10 | from keras.utils import np_utils 11 | from keras.models import Sequential 12 | from keras.layers.core import Dense, Dropout, Activation 13 | from keras.layers.core import TimeDistributedDense 14 | from keras.layers.embeddings import Embedding 15 | from keras.layers.recurrent import LSTM, GRU 16 | from keras.datasets import imdb 17 | 18 | #input is 4*embeddingdim 19 | #output is embeddingdim 20 | def loadData(file=r'c:\f\f\cortana_lstm_new_same_length.txt', num_features=61): 21 | x_train = [] 22 | y_train = [] 23 | current_x = [] 24 | current_y = [] 25 | current_index = 1 26 | with open(file, 'rb') as input_file: 27 | input = csv.reader(input_file, delimiter='\t') 28 | for row in input: 29 | print(row[0]) 30 | if int(row[0])!=current_index: 31 | # a new training instance 32 | print("new") 33 | x_train.append(current_x) 34 | y_train.append(current_y) 35 | current_index=int(row[0]) 36 | current_x = [] 37 | current_y = [] 38 | else: 39 | print("old") 40 | current_x.append(map(int, row[2:2+num_features])) 41 | current_y.append(int(row[1])) 42 | # don't forget the last instance 43 | x_train.append(current_x) 44 | y_train.append((current_y)) 45 | return x_train, y_train 46 | 47 | 48 | 49 | if __name__ == "__main__": 50 | numfeatures = 61 # the input feature size 51 | youtput = 1 # the output prediction (only predict 0 or 1) 52 | embeddingdim=300 53 | x_train, y_train= loadData(num_features=numfeatures) 54 | print(len(x_train), 'train sequences') 55 | #print(x_train[0]) 56 | #print(x_train[1]) 57 | # padding sequence 58 | print("padding sequences") 59 | #x_train = sequence.pad_sequences(x_train,maxlen= 20) 60 | x_train = np.array(x_train) 61 | y_train = np.array(y_train) 62 | print(x_train.shape, ' x_train shape') 63 | print(y_train.shape, ' y_train shape') 64 | print(y_train) 65 | print('Build model...') 66 | 67 | model_user_Item_globalItem = Sequential() 68 | model_user_Item_globalItem.add(Dense(embeddingdim*3, embeddingdim)) 69 | model_temp_userItem = Sequential() 70 | model_temp_userItem.add(LSTM(embeddingdim, embeddingdim, return_sequences=False)) # try using a GRU instead, for fun 71 | #model.add(Dropout(0.5)) 72 | 73 | model=Sequential() 74 | model.add(Merge([model_user_Item_globalItem, model_temp_userItem], mode='concat')) 75 | model.add(model.add(Dense(2*embeddingdim, embeddingdim))) #embedding *4 x embedding 76 | # try using different optimizers and different optimizer configs 77 | model.compile(loss='cos_loss', optimizer='adam') # migth need to implement cosine loss (1-T.dot( y_prd ,y_true) / T.dot(y_prd,y_prd) 78 | 79 | print("Train...") 80 | model.fit(x_train, y_train, batch_size=1, nb_epoch=4, validation_data=(x_train, y_train), show_accuracy=True) 81 | #score, acc = model.evaluate(X_test, y_test, batch_size=batch_size, show_accuracy=True) 82 | 83 | -------------------------------------------------------------------------------- /keras-master/examples/cifar10_cnn.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | from keras.datasets import cifar10 4 | from keras.preprocessing.image import ImageDataGenerator 5 | from keras.models import Sequential 6 | from keras.layers.core import Dense, Dropout, Activation, Flatten 7 | from keras.layers.convolutional import Convolution2D, MaxPooling2D 8 | from keras.optimizers import SGD, Adadelta, Adagrad 9 | from keras.utils import np_utils, generic_utils 10 | from six.moves import range 11 | 12 | ''' 13 | Train a (fairly simple) deep CNN on the CIFAR10 small images dataset. 14 | 15 | GPU run command: 16 | THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python cifar10_cnn.py 17 | 18 | It gets down to 0.65 test logloss in 25 epochs, and down to 0.55 after 50 epochs. 19 | (it's still underfitting at that point, though). 20 | 21 | Note: the data was pickled with Python 2, and some encoding issues might prevent you 22 | from loading it in Python 3. You might have to load it in Python 2, 23 | save it in a different format, load it in Python 3 and repickle it. 24 | ''' 25 | 26 | batch_size = 32 27 | nb_classes = 10 28 | nb_epoch = 200 29 | data_augmentation = True 30 | 31 | # the data, shuffled and split between tran and test sets 32 | (X_train, y_train), (X_test, y_test) = cifar10.load_data() 33 | print('X_train shape:', X_train.shape) 34 | print(X_train.shape[0], 'train samples') 35 | print(X_test.shape[0], 'test samples') 36 | 37 | # convert class vectors to binary class matrices 38 | Y_train = np_utils.to_categorical(y_train, nb_classes) 39 | Y_test = np_utils.to_categorical(y_test, nb_classes) 40 | 41 | model = Sequential() 42 | 43 | model.add(Convolution2D(32, 3, 3, 3, border_mode='full')) 44 | model.add(Activation('relu')) 45 | model.add(Convolution2D(32, 32, 3, 3)) 46 | model.add(Activation('relu')) 47 | model.add(MaxPooling2D(poolsize=(2, 2))) 48 | model.add(Dropout(0.25)) 49 | 50 | model.add(Convolution2D(64, 32, 3, 3, border_mode='full')) 51 | model.add(Activation('relu')) 52 | model.add(Convolution2D(64, 64, 3, 3)) 53 | model.add(Activation('relu')) 54 | model.add(MaxPooling2D(poolsize=(2, 2))) 55 | model.add(Dropout(0.25)) 56 | 57 | model.add(Flatten()) 58 | model.add(Dense(64*8*8, 512)) 59 | model.add(Activation('relu')) 60 | model.add(Dropout(0.5)) 61 | 62 | model.add(Dense(512, nb_classes)) 63 | model.add(Activation('softmax')) 64 | 65 | # let's train the model using SGD + momentum (how original). 66 | sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True) 67 | model.compile(loss='categorical_crossentropy', optimizer=sgd) 68 | 69 | if not data_augmentation: 70 | print("Not using data augmentation or normalization") 71 | 72 | X_train = X_train.astype("float32") 73 | X_test = X_test.astype("float32") 74 | X_train /= 255 75 | X_test /= 255 76 | model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch) 77 | score = model.evaluate(X_test, Y_test, batch_size=batch_size) 78 | print('Test score:', score) 79 | 80 | else: 81 | print("Using real time data augmentation") 82 | 83 | # this will do preprocessing and realtime data augmentation 84 | datagen = ImageDataGenerator( 85 | featurewise_center=True, # set input mean to 0 over the dataset 86 | samplewise_center=False, # set each sample mean to 0 87 | featurewise_std_normalization=True, # divide inputs by std of the dataset 88 | samplewise_std_normalization=False, # divide each input by its std 89 | zca_whitening=False, # apply ZCA whitening 90 | rotation_range=20, # randomly rotate images in the range (degrees, 0 to 180) 91 | width_shift_range=0.2, # randomly shift images horizontally (fraction of total width) 92 | height_shift_range=0.2, # randomly shift images vertically (fraction of total height) 93 | horizontal_flip=True, # randomly flip images 94 | vertical_flip=False) # randomly flip images 95 | 96 | # compute quantities required for featurewise normalization 97 | # (std, mean, and principal components if ZCA whitening is applied) 98 | datagen.fit(X_train) 99 | 100 | for e in range(nb_epoch): 101 | print('-'*40) 102 | print('Epoch', e) 103 | print('-'*40) 104 | print("Training...") 105 | # batch train with realtime data augmentation 106 | progbar = generic_utils.Progbar(X_train.shape[0]) 107 | for X_batch, Y_batch in datagen.flow(X_train, Y_train): 108 | loss = model.train_on_batch(X_batch, Y_batch) 109 | progbar.add(X_batch.shape[0], values=[("train loss", loss)]) 110 | 111 | print("Testing...") 112 | # test time! 113 | progbar = generic_utils.Progbar(X_test.shape[0]) 114 | for X_batch, Y_batch in datagen.flow(X_test, Y_test): 115 | score = model.test_on_batch(X_batch, Y_batch) 116 | progbar.add(X_batch.shape[0], values=[("test loss", score)]) 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | -------------------------------------------------------------------------------- /keras-master/examples/console.out: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sonyisme/keras-recommendation/e84fab6706faf8405f4bfc75cf50ecff6433eb77/keras-master/examples/console.out -------------------------------------------------------------------------------- /keras-master/examples/decodeDoc.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import numpy as np 4 | import random 5 | np.random.seed(1337) # for reproducibility 6 | import theano 7 | from keras.datasets import reuters 8 | from keras.models import Sequential 9 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul 10 | from keras.layers.normalization import BatchNormalization 11 | from keras.utils import np_utils 12 | from keras.preprocessing.text import Tokenizer 13 | 14 | ''' 15 | Train and evaluate a simple MLP on the Reuters newswire topic classification task. 16 | GPU run command: 17 | THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python examples/reuters_mlp.py 18 | CPU run command: 19 | python examples/reuters_mlp.py 20 | ''' 21 | 22 | max_words = 1000 23 | batch_size = 1000 24 | nb_epoch = 15 25 | 26 | print("Loading data...") 27 | (X_train, y_train), (X_test, y_test) = reuters.load_data(nb_words=max_words, test_split=0.2) 28 | print(len(X_train), 'train sequences') 29 | print(len(X_test), 'test sequences') 30 | 31 | nb_classes = np.max(y_train)+1 32 | print(nb_classes, 'classes') 33 | 34 | print("Vectorizing sequence data...") 35 | tokenizer = Tokenizer(nb_words=max_words) 36 | X_train = tokenizer.sequences_to_matrix(X_train, mode="binary") 37 | X_test = tokenizer.sequences_to_matrix(X_test, mode="binary") 38 | xw=X_train.transpose() 39 | 40 | 41 | userfea=1000 42 | itemfea=8982 43 | 44 | print("Building model...") 45 | userModel = Sequential() 46 | userModel.add(Dense(userfea, 700)) 47 | userModel.add(Activation('tanh')) 48 | userModel.add(Dropout(0.4)) 49 | userModel.add(Dense(700, 500)) 50 | userModel.add(Activation('tanh')) 51 | 52 | itemModel = Sequential() 53 | itemModel.add(TimeDistributedDense(itemfea, 1000)) 54 | itemModel.add(Activation('tanh')) 55 | itemModel.add(Dropout(0.4)) 56 | itemModel.add(TimeDistributedDense(1000, 500)) 57 | itemModel.add(Activation('tanh')) 58 | ##itemModel.add(Reshape(4)) 59 | ##itemModel.add(Dense(4, 2)) 60 | itm=itemModel.get_input(train=False) 61 | usr=userModel.get_input(train=False) 62 | itemrep=itemModel.get_output(train=False) 63 | userrep=userModel.get_output(train=False) 64 | model=Sequential() 65 | model.add(Cosine([userModel,itemModel])) #should output 2 values 66 | #model.add(TimeDistributedDense(300, 1)) 67 | ##model.add(Activation('normalization')) 68 | model.add(Reshape(2)) 69 | y_score= model.get_output(train=False) 70 | x_test=model.get_input(train=False) 71 | model.add(Activation('softmax')) 72 | print("Complie model...") 73 | model.compile(loss='categorical_crossentropy', optimizer='adam') 74 | print("Complie outs...") 75 | outv1= theano.function([usr],userrep,allow_input_downcast=True, mode=None) 76 | outv2= theano.function([itm],itemrep,allow_input_downcast=True, mode=None) 77 | print("load W...") 78 | model.load_weights(r'c:\users\t-alie\txtfactorization.model') 79 | print("start predicting...") 80 | df=open(r'c:\users\t-alie\docrep.txt','w') 81 | wf=open(r'c:\users\t-alie\wordrep.txt','w') 82 | for d in range(0,8982): 83 | dh=userModel.custom_predict([X_train[d]],outv1) 84 | df.write("%s\n" %dh) 85 | 86 | #, nb_epoch=nb_epoch, batch_size=batch_size, verbose=1, show_accuracy=True, validation_split=0.1) 87 | #score = model.evaluate(X_test, Y_test, batch_size=batch_size, verbose=1, show_accuracy=True) 88 | -------------------------------------------------------------------------------- /keras-master/examples/imdb_lstm.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import numpy as np 4 | np.random.seed(1337) # for reproducibility 5 | 6 | from keras.preprocessing import sequence 7 | from keras.optimizers import SGD, RMSprop, Adagrad 8 | from keras.utils import np_utils 9 | from keras.models import Sequential 10 | from keras.layers.core import Dense, Dropout, Activation 11 | from keras.layers.embeddings import Embedding 12 | from keras.layers.recurrent import LSTM, GRU 13 | from keras.datasets import imdb 14 | 15 | ''' 16 | Train a LSTM on the IMDB sentiment classification task. 17 | 18 | The dataset is actually too small for LSTM to be of any advantage 19 | compared to simpler, much faster methods such as TF-IDF+LogReg. 20 | 21 | Notes: 22 | 23 | - RNNs are tricky. Choice of batch size is important, 24 | choice of loss and optimizer is critical, etc. 25 | Some configurations won't converge. 26 | 27 | - LSTM loss decrease patterns during training can be quite different 28 | from what you see with CNNs/MLPs/etc. 29 | 30 | GPU command: 31 | THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python imdb_lstm.py 32 | ''' 33 | 34 | max_features = 20000 35 | maxlen = 100 # cut texts after this number of words (among top max_features most common words) 36 | batch_size = 32 37 | 38 | print("Loading data...") 39 | (X_train, y_train), (X_test, y_test) = imdb.load_data(nb_words=max_features, test_split=0.2) 40 | print(len(X_train), 'train sequences') 41 | print(len(X_test), 'test sequences') 42 | 43 | print("Pad sequences (samples x time)") 44 | X_train = sequence.pad_sequences(X_train, maxlen=maxlen) 45 | X_test = sequence.pad_sequences(X_test, maxlen=maxlen) 46 | print('X_train shape:', X_train.shape) 47 | print('X_test shape:', X_test.shape) 48 | 49 | print('Build model...') 50 | model = Sequential() 51 | model.add(Embedding(max_features, 128)) 52 | model.add(LSTM(128, 128)) # try using a GRU instead, for fun 53 | model.add(Dropout(0.5)) 54 | model.add(Dense(128, 1)) 55 | model.add(Activation('sigmoid')) 56 | 57 | # try using different optimizers and different optimizer configs 58 | model.compile(loss='binary_crossentropy', optimizer='adam', class_mode="binary") 59 | 60 | print("Train...") 61 | model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=4, validation_data=(X_test, y_test), show_accuracy=True) 62 | score, acc = model.evaluate(X_test, y_test, batch_size=batch_size, show_accuracy=True) 63 | print('Test score:', score) 64 | print('Test accuracy:', acc) 65 | -------------------------------------------------------------------------------- /keras-master/examples/kaggle_otto_nn.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | 4 | import numpy as np 5 | import pandas as pd 6 | np.random.seed(1337) # for reproducibility 7 | 8 | from keras.models import Sequential 9 | from keras.layers.core import Dense, Dropout, Activation 10 | from keras.layers.normalization import BatchNormalization 11 | from keras.layers.advanced_activations import PReLU 12 | from keras.utils import np_utils, generic_utils 13 | 14 | from sklearn.preprocessing import LabelEncoder 15 | from sklearn.preprocessing import StandardScaler 16 | 17 | ''' 18 | This demonstrates how to reach a score of 0.4890 (local validation) 19 | on the Kaggle Otto challenge, with a deep net using Keras. 20 | 21 | Compatible Python 2.7-3.4. Requires Scikit-Learn and Pandas. 22 | 23 | Recommended to run on GPU: 24 | Command: THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python kaggle_otto_nn.py 25 | On EC2 g2.2xlarge instance: 19s/epoch. 6-7 minutes total training time. 26 | 27 | Best validation score at epoch 21: 0.4881 28 | 29 | Try it at home: 30 | - with/without BatchNormalization (BatchNormalization helps!) 31 | - with ReLU or with PReLU (PReLU helps!) 32 | - with smaller layers, largers layers 33 | - with more layers, less layers 34 | - with different optimizers (SGD+momentum+decay is probably better than Adam!) 35 | 36 | Get the data from Kaggle: https://www.kaggle.com/c/otto-group-product-classification-challenge/data 37 | ''' 38 | 39 | def load_data(path, train=True): 40 | df = pd.read_csv(path) 41 | X = df.values.copy() 42 | if train: 43 | np.random.shuffle(X) # https://youtu.be/uyUXoap67N8 44 | X, labels = X[:, 1:-1].astype(np.float32), X[:, -1] 45 | return X, labels 46 | else: 47 | X, ids = X[:, 1:].astype(np.float32), X[:, 0].astype(str) 48 | return X, ids 49 | 50 | def preprocess_data(X, scaler=None): 51 | if not scaler: 52 | scaler = StandardScaler() 53 | scaler.fit(X) 54 | X = scaler.transform(X) 55 | return X, scaler 56 | 57 | def preprocess_labels(labels, encoder=None, categorical=True): 58 | if not encoder: 59 | encoder = LabelEncoder() 60 | encoder.fit(labels) 61 | y = encoder.transform(labels).astype(np.int32) 62 | if categorical: 63 | y = np_utils.to_categorical(y) 64 | return y, encoder 65 | 66 | def make_submission(y_prob, ids, encoder, fname): 67 | with open(fname, 'w') as f: 68 | f.write('id,') 69 | f.write(','.join([str(i) for i in encoder.classes_])) 70 | f.write('\n') 71 | for i, probs in zip(ids, y_prob): 72 | probas = ','.join([i] + [str(p) for p in probs.tolist()]) 73 | f.write(probas) 74 | f.write('\n') 75 | print("Wrote submission to file {}.".format(fname)) 76 | 77 | 78 | print("Loading data...") 79 | X, labels = load_data('train.csv', train=True) 80 | X, scaler = preprocess_data(X) 81 | y, encoder = preprocess_labels(labels) 82 | 83 | X_test, ids = load_data('test.csv', train=False) 84 | X_test, _ = preprocess_data(X_test, scaler) 85 | 86 | nb_classes = y.shape[1] 87 | print(nb_classes, 'classes') 88 | 89 | dims = X.shape[1] 90 | print(dims, 'dims') 91 | 92 | print("Building model...") 93 | 94 | model = Sequential() 95 | model.add(Dense(dims, 512, init='glorot_uniform')) 96 | model.add(PReLU((512,))) 97 | model.add(BatchNormalization((512,))) 98 | model.add(Dropout(0.5)) 99 | 100 | model.add(Dense(512, 512, init='glorot_uniform')) 101 | model.add(PReLU((512,))) 102 | model.add(BatchNormalization((512,))) 103 | model.add(Dropout(0.5)) 104 | 105 | model.add(Dense(512, 512, init='glorot_uniform')) 106 | model.add(PReLU((512,))) 107 | model.add(BatchNormalization((512,))) 108 | model.add(Dropout(0.5)) 109 | 110 | model.add(Dense(512, nb_classes, init='glorot_uniform')) 111 | model.add(Activation('softmax')) 112 | 113 | model.compile(loss='categorical_crossentropy', optimizer="adam") 114 | 115 | print("Training model...") 116 | 117 | model.fit(X, y, nb_epoch=20, batch_size=128, validation_split=0.15) 118 | 119 | print("Generating submission...") 120 | 121 | proba = model.predict_proba(X_test) 122 | make_submission(proba, ids, encoder, fname='keras-otto.csv') 123 | -------------------------------------------------------------------------------- /keras-master/examples/lstm_text_generation.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | from keras.models import Sequential 3 | from keras.layers.core import Dense, Activation, Dropout 4 | from keras.layers.recurrent import LSTM 5 | from keras.datasets.data_utils import get_file 6 | import numpy as np 7 | import random, sys 8 | 9 | ''' 10 | Example script to generate text from Nietzsche's writings. 11 | 12 | At least 20 epochs are required before the generated text 13 | starts sounding coherent. 14 | 15 | It is recommended to run this script on GPU, as recurrent 16 | networks are quite computationally intensive. 17 | 18 | If you try this script on new data, make sure your corpus 19 | has at least ~100k characters. ~1M is better. 20 | ''' 21 | 22 | path = get_file('nietzsche.txt', origin="https://s3.amazonaws.com/text-datasets/nietzsche.txt") 23 | text = open(path).read().lower() 24 | print('corpus length:', len(text)) 25 | 26 | chars = set(text) 27 | print('total chars:', len(chars)) 28 | char_indices = dict((c, i) for i, c in enumerate(chars)) 29 | indices_char = dict((i, c) for i, c in enumerate(chars)) 30 | 31 | # cut the text in semi-redundant sequences of maxlen characters 32 | maxlen = 20 33 | step = 3 34 | sentences = [] 35 | next_chars = [] 36 | for i in range(0, len(text) - maxlen, step): 37 | sentences.append(text[i : i + maxlen]) 38 | next_chars.append(text[i + maxlen]) 39 | print('nb sequences:', len(sentences)) 40 | 41 | print('Vectorization...') 42 | X = np.zeros((len(sentences), maxlen, len(chars)), dtype=np.bool) 43 | y = np.zeros((len(sentences), len(chars)), dtype=np.bool) 44 | for i, sentence in enumerate(sentences): 45 | for t, char in enumerate(sentence): 46 | X[i, t, char_indices[char]] = 1 47 | y[i, char_indices[next_chars[i]]] = 1 48 | 49 | 50 | # build the model: 2 stacked LSTM 51 | print('Build model...') 52 | model = Sequential() 53 | model.add(LSTM(len(chars), 512, return_sequences=True)) 54 | model.add(Dropout(0.2)) 55 | model.add(LSTM(512, 512, return_sequences=False)) 56 | model.add(Dropout(0.2)) 57 | model.add(Dense(512, len(chars))) 58 | model.add(Activation('softmax')) 59 | 60 | model.compile(loss='categorical_crossentropy', optimizer='rmsprop') 61 | 62 | # helper function to sample an index from a probability array 63 | def sample(a, diversity=0.75): 64 | if random.random() > diversity: 65 | return np.argmax(a) 66 | while 1: 67 | i = random.randint(0, len(a)-1) 68 | if a[i] > random.random(): 69 | return i 70 | 71 | # train the model, output generated text after each iteration 72 | for iteration in range(1, 60): 73 | print() 74 | print('-' * 50) 75 | print('Iteration', iteration) 76 | model.fit(X, y, batch_size=128, nb_epoch=1) 77 | 78 | start_index = random.randint(0, len(text) - maxlen - 1) 79 | 80 | for diversity in [0.2, 0.4, 0.6, 0.8]: 81 | print() 82 | print('----- diversity:', diversity) 83 | 84 | generated = '' 85 | sentence = text[start_index : start_index + maxlen] 86 | generated += sentence 87 | print('----- Generating with seed: "' + sentence + '"') 88 | sys.stdout.write(generated) 89 | 90 | for iteration in range(400): 91 | x = np.zeros((1, maxlen, len(chars))) 92 | for t, char in enumerate(sentence): 93 | x[0, t, char_indices[char]] = 1. 94 | 95 | preds = model.predict(x, verbose=0)[0] 96 | next_index = sample(preds, diversity) 97 | next_char = indices_char[next_index] 98 | 99 | generated += next_char 100 | sentence = sentence[1:] + next_char 101 | 102 | sys.stdout.write(next_char) 103 | sys.stdout.flush() 104 | print() -------------------------------------------------------------------------------- /keras-master/examples/mnist_cnn.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import numpy as np 4 | np.random.seed(1337) # for reproducibility 5 | 6 | from keras.datasets import mnist 7 | from keras.models import Sequential 8 | from keras.layers.core import Dense, Dropout, Activation, Flatten 9 | from keras.layers.convolutional import Convolution2D, MaxPooling2D 10 | from keras.utils import np_utils 11 | 12 | ''' 13 | Train a simple convnet on the MNIST dataset. 14 | 15 | Run on GPU: THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python mnist_cnn.py 16 | 17 | Get to 99.25% test accuracy after 12 epochs (there is still a lot of margin for parameter tuning). 18 | 16 seconds per epoch on a GRID K520 GPU. 19 | ''' 20 | 21 | batch_size = 128 22 | nb_classes = 10 23 | nb_epoch = 12 24 | 25 | # the data, shuffled and split between tran and test sets 26 | (X_train, y_train), (X_test, y_test) = mnist.load_data() 27 | 28 | X_train = X_train.reshape(X_train.shape[0], 1, 28, 28) 29 | X_test = X_test.reshape(X_test.shape[0], 1, 28, 28) 30 | X_train = X_train.astype("float32") 31 | X_test = X_test.astype("float32") 32 | X_train /= 255 33 | X_test /= 255 34 | print('X_train shape:', X_train.shape) 35 | print(X_train.shape[0], 'train samples') 36 | print(X_test.shape[0], 'test samples') 37 | 38 | # convert class vectors to binary class matrices 39 | Y_train = np_utils.to_categorical(y_train, nb_classes) 40 | Y_test = np_utils.to_categorical(y_test, nb_classes) 41 | 42 | model = Sequential() 43 | 44 | model.add(Convolution2D(32, 1, 3, 3, border_mode='full')) 45 | model.add(Activation('relu')) 46 | model.add(Convolution2D(32, 32, 3, 3)) 47 | model.add(Activation('relu')) 48 | model.add(MaxPooling2D(poolsize=(2, 2))) 49 | model.add(Dropout(0.25)) 50 | 51 | model.add(Flatten()) 52 | model.add(Dense(32*196, 128)) 53 | model.add(Activation('relu')) 54 | model.add(Dropout(0.5)) 55 | 56 | model.add(Dense(128, nb_classes)) 57 | model.add(Activation('softmax')) 58 | 59 | model.compile(loss='categorical_crossentropy', optimizer='adadelta') 60 | 61 | model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch, show_accuracy=True, verbose=1, validation_data=(X_test, Y_test)) 62 | score = model.evaluate(X_test, Y_test, show_accuracy=True, verbose=0) 63 | print('Test score:', score[0]) 64 | print('Test accuracy:', score[1]) 65 | -------------------------------------------------------------------------------- /keras-master/examples/mnist_irnn.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import numpy as np 4 | np.random.seed(1337) # for reproducibility 5 | 6 | from keras.datasets import mnist 7 | from keras.models import Sequential 8 | from keras.layers.core import Dense, Activation 9 | from keras.initializations import normal, identity 10 | from keras.layers.recurrent import SimpleRNN, LSTM 11 | from keras.optimizers import RMSprop 12 | from keras.utils import np_utils 13 | 14 | ''' 15 | This is a reproduction of the IRNN experiment 16 | with pixel-by-pixel sequential MNIST in 17 | "A Simple Way to Initialize Recurrent Networks of Rectified Linear Units " 18 | by Quoc V. Le, Navdeep Jaitly, Geoffrey E. Hinton 19 | 20 | arXiv:1504.00941v2 [cs.NE] 7 Apr 201 21 | http://arxiv.org/pdf/1504.00941v2.pdf 22 | 23 | Optimizer is replaced with RMSprop which yields more stable and steady 24 | improvement. 25 | 26 | Reaches 0.93 train/test accuracy after 900 epochs (which roughly corresponds 27 | to 1687500 steps in the original paper.) 28 | ''' 29 | 30 | batch_size = 32 31 | nb_classes = 10 32 | nb_epochs = 200 33 | hidden_units = 100 34 | 35 | learning_rate = 1e-6 36 | clip_norm = 1.0 37 | BPTT_truncate = 28*28 38 | 39 | # the data, shuffled and split between train and test sets 40 | (X_train, y_train), (X_test, y_test) = mnist.load_data() 41 | 42 | X_train = X_train.reshape(X_train.shape[0], -1, 1) 43 | X_test = X_test.reshape(X_test.shape[0], -1, 1) 44 | X_train = X_train.astype("float32") 45 | X_test = X_test.astype("float32") 46 | X_train /= 255 47 | X_test /= 255 48 | print('X_train shape:', X_train.shape) 49 | print(X_train.shape[0], 'train samples') 50 | print(X_test.shape[0], 'test samples') 51 | 52 | # convert class vectors to binary class matrices 53 | Y_train = np_utils.to_categorical(y_train, nb_classes) 54 | Y_test = np_utils.to_categorical(y_test, nb_classes) 55 | 56 | print('Evaluate IRNN...') 57 | model = Sequential() 58 | model.add(SimpleRNN(input_dim=1, output_dim=hidden_units, 59 | init=lambda shape: normal(shape, scale=0.001), 60 | inner_init=lambda shape: identity(shape, scale=1.0), 61 | activation='relu', truncate_gradient=BPTT_truncate)) 62 | model.add(Dense(hidden_units, nb_classes)) 63 | model.add(Activation('softmax')) 64 | rmsprop = RMSprop(lr=learning_rate) 65 | model.compile(loss='categorical_crossentropy', optimizer=rmsprop) 66 | 67 | model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epochs, 68 | show_accuracy=True, verbose=1, validation_data=(X_test, Y_test)) 69 | 70 | scores = model.evaluate(X_test, Y_test, show_accuracy=True, verbose=0) 71 | print('IRNN test score:', scores[0]) 72 | print('IRNN test accuracy:', scores[1]) 73 | 74 | print('Compare to LSTM...') 75 | model = Sequential() 76 | model.add(LSTM(1, hidden_units)) 77 | model.add(Dense(hidden_units, nb_classes)) 78 | model.add(Activation('softmax')) 79 | rmsprop = RMSprop(lr=learning_rate) 80 | model.compile(loss='categorical_crossentropy', optimizer=rmsprop) 81 | 82 | model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epochs, 83 | show_accuracy=True, verbose=1, validation_data=(X_test, Y_test)) 84 | 85 | scores = model.evaluate(X_test, Y_test, show_accuracy=True, verbose=0) 86 | print('LSTM test score:', scores[0]) 87 | print('LSTM test accuracy:', scores[1]) 88 | -------------------------------------------------------------------------------- /keras-master/examples/mnist_mlp.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import numpy as np 4 | np.random.seed(1337) # for reproducibility 5 | 6 | from keras.datasets import mnist 7 | from keras.models import Sequential 8 | from keras.layers.core import Dense, Dropout, Activation 9 | from keras.optimizers import SGD, Adam, RMSprop 10 | from keras.utils import np_utils 11 | 12 | ''' 13 | Train a simple deep NN on the MNIST dataset. 14 | 15 | Get to 98.30% test accuracy after 20 epochs (there is *a lot* of margin for parameter tuning). 16 | 2 seconds per epoch on a GRID K520 GPU. 17 | ''' 18 | 19 | batch_size = 128 20 | nb_classes = 10 21 | nb_epoch = 20 22 | 23 | 24 | 25 | # the data, shuffled and split between tran and test sets 26 | (X_train, y_train), (X_test, y_test) = mnist.load_data() 27 | 28 | X_train = X_train.reshape(60000, 784) 29 | X_test = X_test.reshape(10000, 784) 30 | X_train = X_train.astype("float32") 31 | X_test = X_test.astype("float32") 32 | X_train /= 255 33 | X_test /= 255 34 | print(X_train.shape[0], 'train samples') 35 | print(X_test.shape[0], 'test samples') 36 | 37 | # convert class vectors to binary class matrices 38 | Y_train = np_utils.to_categorical(y_train, nb_classes) 39 | Y_test = np_utils.to_categorical(y_test, nb_classes) 40 | 41 | model = Sequential() 42 | model.add(Dense(784, 128)) 43 | model.add(Activation('relu')) 44 | model.add(Dropout(0.2)) 45 | model.add(Dense(128, 128)) 46 | model.add(Activation('relu')) 47 | model.add(Dropout(0.2)) 48 | model.add(Dense(128, 10)) 49 | model.add(Activation('softmax')) 50 | 51 | rms = RMSprop() 52 | model.compile(loss='categorical_crossentropy', optimizer=rms) 53 | 54 | model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch, show_accuracy=True, verbose=2, validation_data=(X_test, Y_test)) 55 | score = model.evaluate(X_test, Y_test, show_accuracy=True, verbose=0) 56 | print('Test score:', score[0]) 57 | print('Test accuracy:', score[1]) 58 | -------------------------------------------------------------------------------- /keras-master/examples/movieLens100k.cos.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | from __future__ import absolute_import 5 | from __future__ import print_function 6 | import numpy as np 7 | import theano 8 | 9 | np.random.seed(1337) # for reproducibility 10 | 11 | from keras.datasets import reuters 12 | from keras.models import Sequential 13 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul 14 | from keras.layers.normalization import BatchNormalization 15 | from keras.utils import np_utils 16 | from keras.preprocessing.text import Tokenizer 17 | 18 | 19 | def inspect_inputs(i, node, fn): 20 | print( "Beging intput:") 21 | print (i) 22 | print (node) 23 | print ("input(s) value(s):") 24 | print ([input[0].shape for input in fn.inputs]) 25 | theano.printing.debugprint(node) 26 | print( "End input:") 27 | 28 | def inspect_outputs(i, node, fn): 29 | print( "Beging output:") 30 | print( "output(s) :", [output[0].shape for output in fn.outputs]) 31 | print( "End output:") 32 | 33 | def load_dataset(userFile,posFile,negFile,n): 34 | fitems = open(posFile, 'r') 35 | fitemsN = open(negFile, 'r') 36 | fusers = open(userFile, 'r') 37 | 38 | lusers=fusers.readline() 39 | user= np.zeros([n,1682]) 40 | items= np.zeros([n,2,943]) 41 | y_train=np.zeros([n,2]) 42 | 43 | i=0 44 | for lusers in fusers: 45 | if i==n: 46 | break 47 | litems=fitems.readline() 48 | litemsN=fitemsN.readline() 49 | 50 | feats=lusers.split(" ") 51 | pfeats=litems.split(" ") 52 | nfeats=litemsN.split(" ") 53 | 54 | for fea in feats: 55 | if ':' in fea: 56 | x=fea.split(":") 57 | id=int(unicode(x[0], errors='ignore'))-1 58 | user[i][id]=float(unicode(x[1], errors='ignore') ) 59 | y_train[i][0]=1 60 | 61 | 62 | for fea in pfeats: 63 | if ':' in fea: 64 | x=fea.split(":") 65 | id=int(unicode(x[0], errors='ignore'))-1 66 | items[i][0][id]=float(unicode(x[1], errors='ignore') ) 67 | 68 | for fea in nfeats: 69 | if ':' in fea: 70 | x=fea.split(":") 71 | id=int(unicode(x[0], errors='ignore'))-1 72 | items[i][1][id]=float(unicode(x[1], errors='ignore') ) 73 | 74 | i=i+1 75 | 76 | #user= np.array([[1,0],[1,0],[0,1]]) 77 | #y_train=np.array([[1,0],[1,0],[1,0]]) 78 | #Items=np.array( [ [[1,0],[0,1]] , [[.5,0],[0,1]],[[-1,1],[1,0]] ]) 79 | #user= np.array([[1,1,1],[1,3,1],[0,1,0],[0,2,-1]]) 80 | #y_train=np.array([[1,0],[1,0],[1,0],[1,0]]) 81 | #Items=np.array( [ [[1,2,0],[0,2,0]] , [[2,2,1],[2,0,2]],[[0,1,2],[1,0,0]],[[1,3,3],[1,3,-1]] ]) 82 | #user= np.array([[0,1]]) 83 | #y_train=np.array([[1,0]]) 84 | #Items=np.array( [[[-1,1],[1,0]]]) 85 | # The inputs come as vectors, we reshape them to monochrome 2D images, 86 | # according to the shape convention: (examples, channels, rows, columns) 87 | #user.reshape(-1,3); 88 | # We just return all the arrays in order, as expected in main(). 89 | # (It doesn't matter how we do this as long as we can read them again.) 90 | return (user ,items, y_train) 91 | 92 | 93 | print("Loading data...") 94 | user ,Items, y_train = load_dataset(r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.users100k",r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.items_pos100k",r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.items_neg100k0",45915) 95 | 96 | #print(len(user), 'train sequences',r"f:\1b.items.n0",) 97 | 98 | print('user_train shape:', user.shape) 99 | print('Item shape:', Items.shape) 100 | userModel = Sequential() 101 | userModel.add(Dense(1682, 500)) 102 | userModel.add(Activation('tanh')) 103 | userModel.add(Dropout(0.4)) 104 | userModel.add(Dense(500, 500)) 105 | userModel.add(Activation('tanh')) 106 | 107 | itemModel = Sequential() 108 | itemModel.add(TimeDistributedDense(943, 500)) 109 | itemModel.add(Activation('tanh')) 110 | itemModel.add(Dropout(0.4)) 111 | itemModel.add(TimeDistributedDense(500, 500)) 112 | itemModel.add(Activation('tanh')) 113 | ##itemModel.add(Reshape(4)) 114 | ##itemModel.add(Dense(4, 2)) 115 | model=Sequential() 116 | model.add(ElementMul([userModel,itemModel])) #should output 2 values 117 | model.add(TimeDistributedDense(500, 1)) 118 | ##model.add(Activation('normalization')) 119 | model.add(Reshape(2)) 120 | y_score= model.get_output(train=False) 121 | x_test=model.get_input(train=False) 122 | model.add(Activation('softmax')) 123 | ##model.add(Merge([userModel, itemModel], mode='sum')) 124 | 125 | 126 | print('done model construction') 127 | model.compile(loss='categorical_crossentropy', optimizer='Adadelta') 128 | print('done complie') 129 | scoring= theano.function(x_test,y_score,allow_input_downcast=True, mode=None) 130 | history = model.fit([user ,Items] ,y_train, nb_epoch=7, batch_size=2048, verbose=2, show_accuracy=True) 131 | 132 | #history = model.train_on_batch([user ,Items] ,y_train,accuracy=True)# nb_epoch=10, batch_size=1024, verbose=2, show_accuracy=True) 133 | print('done training') 134 | user_test ,Items_test, y_test = load_dataset(r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.userstest100k.centered",r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.itemstest100k",r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.itemstest100k.fakeneg",45915) 135 | y_p=model.custom_predict([user_test,Items_test],scoring) 136 | #y_pp=model.predict([user_test,Items_test]) 137 | pfile=open(r"C:\Users\t-alie\Downloads\movieLens_1M\yp_cos_relu_c","w") 138 | for y in y_p: 139 | pfile.write("%s\n" %y) 140 | pfile.close() 141 | #pfile1=open(r"C:\Users\t-alie\Downloads\movieLens_1M\yp1","w") 142 | #for y in y_pp: 143 | # pfile1.write("%s\n" %y) 144 | 145 | #pfile1.close() 146 | print('done prediction') 147 | model.save_weights(r'f:\1b.model') 148 | #print('done saving') -------------------------------------------------------------------------------- /keras-master/examples/movieLens100k.maxdoty.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | from __future__ import absolute_import 5 | from __future__ import print_function 6 | import numpy as np 7 | import theano 8 | 9 | np.random.seed(1337) # for reproducibility 10 | 11 | from keras.datasets import reuters 12 | from keras.models import Sequential 13 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul,MaxDot 14 | from keras.layers.normalization import BatchNormalization 15 | from keras.utils import np_utils 16 | from keras.preprocessing.text import Tokenizer 17 | 18 | 19 | def inspect_inputs(i, node, fn): 20 | print( "Beging intput:") 21 | print (i) 22 | print (node) 23 | print ("input(s) value(s):") 24 | print ([input[0].shape for input in fn.inputs]) 25 | theano.printing.debugprint(node) 26 | print( "End input:") 27 | 28 | def inspect_outputs(i, node, fn): 29 | print( "Beging output:") 30 | print( "output(s) :", [output[0].shape for output in fn.outputs]) 31 | print( "End output:") 32 | 33 | def load_dataset(userFile,posFile,negFile,n): 34 | fitems = open(posFile, 'r') 35 | fitemsN = open(negFile, 'r') 36 | fusers = open(userFile, 'r') 37 | 38 | lusers=fusers.readline() 39 | user= np.zeros([n,1682]) 40 | items= np.zeros([n,2,943]) 41 | y_train=np.zeros([n,2]) 42 | 43 | i=0 44 | for lusers in fusers: 45 | if i==n: 46 | break 47 | litems=fitems.readline() 48 | litemsN=fitemsN.readline() 49 | 50 | feats=lusers.split(" ") 51 | pfeats=litems.split(" ") 52 | nfeats=litemsN.split(" ") 53 | 54 | for fea in feats: 55 | if ':' in fea: 56 | x=fea.split(":") 57 | id=int(unicode(x[0], errors='ignore'))-1 58 | user[i][id]=float(unicode(x[1], errors='ignore') ) 59 | y_train[i][0]=1 60 | 61 | 62 | for fea in pfeats: 63 | if ':' in fea: 64 | x=fea.split(":") 65 | id=int(unicode(x[0], errors='ignore'))-1 66 | items[i][0][id]=float(unicode(x[1], errors='ignore') ) 67 | 68 | for fea in nfeats: 69 | if ':' in fea: 70 | x=fea.split(":") 71 | id=int(unicode(x[0], errors='ignore'))-1 72 | items[i][1][id]=float(unicode(x[1], errors='ignore') ) 73 | 74 | i=i+1 75 | 76 | #user= np.array([[1,0],[1,0],[0,1]]) 77 | #y_train=np.array([[1,0],[1,0],[1,0]]) 78 | #Items=np.array( [ [[1,0],[0,1]] , [[.5,0],[0,1]],[[-1,1],[1,0]] ]) 79 | #user= np.array([[1,1,1],[1,3,1],[0,1,0],[0,2,-1]]) 80 | #y_train=np.array([[1,0],[1,0],[1,0],[1,0]]) 81 | #Items=np.array( [ [[1,2,0],[0,2,0]] , [[2,2,1],[2,0,2]],[[0,1,2],[1,0,0]],[[1,3,3],[1,3,-1]] ]) 82 | #user= np.array([[0,1]]) 83 | #y_train=np.array([[1,0]]) 84 | #Items=np.array( [[[-1,1],[1,0]]]) 85 | # The inputs come as vectors, we reshape them to monochrome 2D images, 86 | # according to the shape convention: (examples, channels, rows, columns) 87 | #user.reshape(-1,3); 88 | # We just return all the arrays in order, as expected in main(). 89 | # (It doesn't matter how we do this as long as we can read them again.) 90 | return (user ,items, y_train) 91 | 92 | 93 | print("Loading data...") 94 | user ,Items, y_train = load_dataset(r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.users100k",r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.items_pos100k",r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.items_neg100k0",45915) 95 | 96 | #print(len(user), 'train sequences',r"f:\1b.items.n0",) 97 | 98 | print('user_train shape:', user.shape) 99 | print('Item shape:', Items.shape) 100 | userModel = Sequential() 101 | userModel.add(Dense(1682, 500)) 102 | userModel.add(Activation('tanh')) 103 | userModel.add(Dropout(0.4)) 104 | userModel.add(Dense(500, 500)) 105 | userModel.add(Activation('tanh')) 106 | 107 | itemModel = Sequential() 108 | itemModel.add(TimeDistributedDense(943, 500)) 109 | itemModel.add(Activation('tanh')) 110 | itemModel.add(Dropout(0.4)) 111 | itemModel.add(TimeDistributedDense(500, 500)) 112 | itemModel.add(Activation('tanh')) 113 | ##itemModel.add(Reshape(4)) 114 | ##itemModel.add(Dense(4, 2)) 115 | model=Sequential() 116 | model.add(MaxDot([userModel,itemModel])) #should output 2 values 117 | #model.add(TimeDistributedDense(300, 1)) 118 | ##model.add(Activation('normalization')) 119 | model.add(Reshape(2)) 120 | y_score= model.get_output(train=False) 121 | x_test=model.get_input(train=False) 122 | model.add(Activation('softmax')) 123 | ##model.add(Merge([userModel, itemModel], mode='sum')) 124 | 125 | 126 | print('done model construction') 127 | model.compile(loss='categorical_crossentropy', optimizer='Adadelta') 128 | print('done complie') 129 | scoring= theano.function(x_test,y_score,allow_input_downcast=True, mode=None) 130 | history = model.fit([user ,Items] ,y_train, nb_epoch=100, batch_size=2048, verbose=2, show_accuracy=True) 131 | 132 | #history = model.train_on_batch([user ,Items] ,y_train,accuracy=True)# nb_epoch=10, batch_size=1024, verbose=2, show_accuracy=True) 133 | print('done training') 134 | user_test ,Items_test, y_test = load_dataset(r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.userstest100k.centered",r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.itemstest100k",r"C:\Users\t-alie\Downloads\movieLens_1M\movielens.itemstest100k.fakeneg",50781) 135 | y_p=model.custom_predict([user_test,Items_test],scoring) 136 | #y_pp=model.predict([user_test,Items_test]) 137 | pfile=open(r"C:\Users\t-alie\Downloads\movieLens_1M\yp_max","w") 138 | for y in y_p: 139 | pfile.write("%s\n" %y) 140 | pfile.close() 141 | #pfile1=open(r"C:\Users\t-alie\Downloads\movieLens_1M\yp1","w") 142 | #for y in y_pp: 143 | # pfile1.write("%s\n" %y) 144 | 145 | #pfile1.close() 146 | print('done prediction') 147 | #model.save_weights(r'f:\1b.model') 148 | #print('done saving') -------------------------------------------------------------------------------- /keras-master/examples/movieslensDSSM.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | from __future__ import absolute_import 5 | from __future__ import print_function 6 | import numpy as np 7 | np.random.seed(1337) # for reproducibility 8 | 9 | from keras.datasets import reuters 10 | from keras.models import Sequential 11 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape 12 | from keras.layers.normalization import BatchNormalization 13 | from keras.utils import np_utils 14 | from keras.preprocessing.text import Tokenizer 15 | 16 | def load_dataset(): 17 | fitems = open(r"F:\1b.items.p", 'r') 18 | fitemsN = open(r"f:\1b.items.n0", 'r') 19 | fusers = open(r"f:\1b.users", 'r') 20 | 21 | lusers=fusers.readline() 22 | user= np.zeros([1024,3883]) 23 | items= np.zeros([1024,2,6039]) 24 | y_train=np.zeros([1024,2]) 25 | 26 | i=0 27 | for lusers in fusers: 28 | 29 | litems=fitems.readline() 30 | litemsN=fitemsN.readline() 31 | 32 | feats=lusers.split(" ") 33 | pfeats=litems.split(" ") 34 | nfeats=litemsN.split(" ") 35 | 36 | for fea in feats: 37 | if ':' in fea: 38 | x=fea.split(":") 39 | id=int(unicode(x[0], errors='ignore'))-1 40 | user[i][id]=float(unicode(x[1], errors='ignore') ) 41 | y_train[i][0]=1 42 | 43 | 44 | for fea in pfeats: 45 | if ':' in fea: 46 | x=fea.split(":") 47 | id=int(unicode(x[0], errors='ignore'))-1 48 | items[i][0][id]=float(unicode(x[1], errors='ignore') ) 49 | 50 | for fea in nfeats: 51 | if ':' in fea: 52 | x=fea.split(":") 53 | id=int(unicode(x[0], errors='ignore'))-1 54 | items[i][1][id]=float(unicode(x[1], errors='ignore') ) 55 | 56 | i=i+1 57 | 58 | #user= np.array([[1,0],[1,0],[0,1]]) 59 | #y_train=np.array([[1,0],[1,0],[1,0]]) 60 | #Items=np.array( [ [[1,0],[0,1]] , [[.5,0],[0,1]],[[-1,1],[1,0]] ]) 61 | #user= np.array([[1,1,1],[1,3,1],[0,1,0],[0,2,-1]]) 62 | #y_train=np.array([[1,0],[1,0],[1,0],[1,0]]) 63 | #Items=np.array( [ [[1,2,0],[0,2,0]] , [[2,2,1],[2,0,2]],[[0,1,2],[1,0,0]],[[1,3,3],[1,3,-1]] ]) 64 | #user= np.array([[0,1]]) 65 | #y_train=np.array([[1,0]]) 66 | #Items=np.array( [[[-1,1],[1,0]]]) 67 | # The inputs come as vectors, we reshape them to monochrome 2D images, 68 | # according to the shape convention: (examples, channels, rows, columns) 69 | #user.reshape(-1,3); 70 | # We just return all the arrays in order, as expected in main(). 71 | # (It doesn't matter how we do this as long as we can read them again.) 72 | return (user ,items, y_train) 73 | 74 | 75 | print("Loading data...") 76 | user ,Items, y_train = load_dataset() 77 | print(len(user), 'train sequences') 78 | 79 | print('user_train shape:', user.shape) 80 | print('Item shape:', Items.shape) 81 | userModel = Sequential() 82 | userModel.add(Dense(3883, 300)) 83 | userModel.add(Activation('tanh')) 84 | userModel.add(Dropout(0.4)) 85 | userModel.add(Dense(300, 300)) 86 | userModel.add(Activation('tanh')) 87 | 88 | itemModel = Sequential() 89 | itemModel.add(TimeDistributedDense(6039, 300)) 90 | itemModel.add(Activation('tanh')) 91 | itemModel.add(Dropout(0.4)) 92 | itemModel.add(TimeDistributedDense(300, 300)) 93 | itemModel.add(Activation('tanh')) 94 | ##itemModel.add(Reshape(4)) 95 | ##itemModel.add(Dense(4, 2)) 96 | model=Sequential() 97 | model.add(Cosine([userModel,itemModel])) #should output 2 values 98 | ##model.add(Activation('normalization')) 99 | model.add(Reshape(2)) 100 | ##model.add(Merge([userModel, itemModel], mode='sum')) 101 | 102 | 103 | print('done model construction') 104 | model.compile(loss='categorical_crossentropy', optimizer='Adadelta') 105 | print('done complie') 106 | history = model.fit([user ,Items] ,y_train, nb_epoch=10, batch_size=1, verbose=2, show_accuracy=True) 107 | print('done training') 108 | #model.save_weights(r'f:\1b.model') 109 | #print('done saving') -------------------------------------------------------------------------------- /keras-master/examples/out.err: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sonyisme/keras-recommendation/e84fab6706faf8405f4bfc75cf50ecff6433eb77/keras-master/examples/out.err -------------------------------------------------------------------------------- /keras-master/examples/reuters_autoEncoder.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import numpy as np 4 | import random 5 | np.random.seed(1337) # for reproducibility 6 | from keras.layers import containers 7 | 8 | from keras.datasets import reuters 9 | from keras.models import Sequential 10 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul,AutoEncoder 11 | from keras.layers.normalization import BatchNormalization 12 | from keras.utils import np_utils 13 | from keras.preprocessing.text import Tokenizer 14 | 15 | ''' 16 | Train and evaluate a simple MLP on the Reuters newswire topic classification task. 17 | GPU run command: 18 | THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python examples/reuters_mlp.py 19 | CPU run command: 20 | python examples/reuters_mlp.py 21 | ''' 22 | 23 | max_words = 1000 24 | batch_size = 1000 25 | nb_epoch = 15 26 | 27 | print("Loading data...") 28 | (X_train, y_train), (X_test, y_test) = reuters.load_data(nb_words=max_words, test_split=0.2) 29 | print(len(X_train), 'train sequences') 30 | print(len(X_test), 'test sequences') 31 | 32 | nb_classes = np.max(y_train)+1 33 | print(nb_classes, 'classes') 34 | 35 | print("Vectorizing sequence data...") 36 | tokenizer = Tokenizer(nb_words=max_words) 37 | X_train = tokenizer.sequences_to_matrix(X_train, mode="binary") 38 | X_test = tokenizer.sequences_to_matrix(X_test, mode="binary") 39 | xw=X_train.transpose() 40 | 41 | 42 | userfea=1000 43 | itemfea=8982 44 | 45 | print("Building model...") 46 | encoder = containers.Sequential([Dense(1000, 700), Dense(700, 500)]) 47 | decoder = containers.Sequential([Dense(500, 700), Dense(700, 1000)]) 48 | 49 | model = Sequential() 50 | model.add(AutoEncoder(encoder=encoder, decoder=decoder, output_reconstruction=False)) 51 | 52 | model.compile(loss='mean_squared_error', optimizer='adam') 53 | 54 | 55 | model.fit(X_train, X_train, nb_epoch=15, batch_size=1024, verbose=1, show_accuracy=True, validation_split=0.1) 56 | df=open(r'f:\autoencoderrep.txt') 57 | dh= model.predict(X_train) 58 | for doc in dh: 59 | for v in doc: 60 | df.write("%s " %v) 61 | df.write("\n") 62 | df.close() 63 | #, nb_epoch=nb_epoch, batch_size=batch_size, verbose=1, show_accuracy=True, validation_split=0.1) 64 | #score = model.evaluate(X_test, Y_test, batch_size=batch_size, verbose=1, show_accuracy=True) 65 | 66 | model.save_weights(r'c:\users\t-alie\txtfactorization.Auto.model') -------------------------------------------------------------------------------- /keras-master/examples/reuters_factorization.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import numpy as np 4 | import random 5 | np.random.seed(1337) # for reproducibility 6 | 7 | from keras.datasets import reuters 8 | from keras.models import Sequential 9 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul 10 | from keras.layers.normalization import BatchNormalization 11 | from keras.utils import np_utils 12 | from keras.preprocessing.text import Tokenizer 13 | 14 | ''' 15 | Train and evaluate a simple MLP on the Reuters newswire topic classification task. 16 | GPU run command: 17 | THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python examples/reuters_mlp.py 18 | CPU run command: 19 | python examples/reuters_mlp.py 20 | ''' 21 | 22 | max_words = 1000 23 | batch_size = 1000 24 | nb_epoch = 15 25 | 26 | print("Loading data...") 27 | (X_train, y_train), (X_test, y_test) = reuters.load_data(nb_words=max_words, test_split=0.2) 28 | print(len(X_train), 'train sequences') 29 | print(len(X_test), 'test sequences') 30 | 31 | nb_classes = np.max(y_train)+1 32 | print(nb_classes, 'classes') 33 | 34 | print("Vectorizing sequence data...") 35 | tokenizer = Tokenizer(nb_words=max_words) 36 | X_train = tokenizer.sequences_to_matrix(X_train, mode="binary") 37 | X_test = tokenizer.sequences_to_matrix(X_test, mode="binary") 38 | xw=X_train.transpose() 39 | 40 | 41 | userfea=1000 42 | itemfea=8982 43 | 44 | print("Building model...") 45 | userModel = Sequential() 46 | userModel.add(Dense(userfea, 700)) 47 | userModel.add(Activation('tanh')) 48 | userModel.add(Dropout(0.4)) 49 | userModel.add(Dense(700, 2)) 50 | userModel.add(Activation('tanh')) 51 | 52 | itemModel = Sequential() 53 | itemModel.add(TimeDistributedDense(itemfea, 1000)) 54 | itemModel.add(Activation('tanh')) 55 | itemModel.add(Dropout(0.4)) 56 | itemModel.add(TimeDistributedDense(1000, 2)) 57 | itemModel.add(Activation('tanh')) 58 | ##itemModel.add(Reshape(4)) 59 | ##itemModel.add(Dense(4, 2)) 60 | model=Sequential() 61 | model.add(Cosine([userModel,itemModel])) #should output 2 values 62 | #model.add(TimeDistributedDense(300, 1)) 63 | ##model.add(Activation('normalization')) 64 | model.add(Reshape(2)) 65 | y_score= model.get_output(train=False) 66 | x_test=model.get_input(train=False) 67 | model.add(Activation('softmax')) 68 | 69 | model.compile(loss='categorical_crossentropy', optimizer='adam') 70 | 71 | for itr in range(0,15): 72 | print(itr) 73 | n=3000 74 | traindoc=np.zeros([n,1000]) 75 | ytrain=np.zeros([n,2]) 76 | trainword=np.zeros([n,2,8982]) 77 | cnt=0 78 | avg=0 79 | nb=0 80 | for d in range(0,8982): 81 | for w in range(0,1000): 82 | if cnt==n: # this will ignore last batch 83 | avg=avg+( model.train_on_batch([traindoc, trainword], ytrain)) 84 | nb=nb+1 85 | traindoc=np.zeros([n,1000]) 86 | ytrain=np.zeros([n,2]) 87 | trainword=np.zeros([n,2,8982]) 88 | cnt=0 89 | if X_train[d,w]==1: 90 | traindoc[cnt]=X_train[d] 91 | trainword[cnt,0]=xw[w] 92 | while True: 93 | wn=random.randint(0,999) 94 | if X_train[d,wn]==0: 95 | break 96 | trainword[cnt,1]=xw[wn] 97 | ytrain[cnt,0]=1 98 | cnt=cnt+1 99 | print(avg/nb) 100 | model.save_weights(r'c:\users\t-alie\textfactorization.model.2.'+`itr`) 101 | #, nb_epoch=nb_epoch, batch_size=batch_size, verbose=1, show_accuracy=True, validation_split=0.1) 102 | #score = model.evaluate(X_test, Y_test, batch_size=batch_size, verbose=1, show_accuracy=True) 103 | 104 | model.save_weights(r'c:\users\t-alie\txtfactorization.model.2') -------------------------------------------------------------------------------- /keras-master/examples/reuters_mlp.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import numpy as np 4 | np.random.seed(1337) # for reproducibility 5 | 6 | from keras.datasets import reuters 7 | from keras.models import Sequential 8 | from keras.layers.core import Dense, Dropout, Activation 9 | from keras.layers.normalization import BatchNormalization 10 | from keras.utils import np_utils 11 | from keras.preprocessing.text import Tokenizer 12 | 13 | ''' 14 | Train and evaluate a simple MLP on the Reuters newswire topic classification task. 15 | GPU run command: 16 | THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python examples/reuters_mlp.py 17 | CPU run command: 18 | python examples/reuters_mlp.py 19 | ''' 20 | 21 | max_words = 1000 22 | batch_size = 1024 23 | nb_epoch = 50 24 | 25 | print("Loading data...") 26 | (X_train, y_train), (X_test, y_test) = reuters.load_data(nb_words=max_words, test_split=0.2) 27 | print(len(X_train), 'train sequences') 28 | print(len(X_test), 'test sequences') 29 | 30 | nb_classes = np.max(y_train)+1 31 | print(nb_classes, 'classes') 32 | 33 | print("Vectorizing sequence data...") 34 | tokenizer = Tokenizer(nb_words=max_words) 35 | X_train = tokenizer.sequences_to_matrix(X_train, mode="binary") 36 | X_test = tokenizer.sequences_to_matrix(X_test, mode="binary") 37 | print('X_train shape:', X_train.shape) 38 | print('X_test shape:', X_test.shape) 39 | 40 | print("Convert class vector to binary class matrix (for use with categorical_crossentropy)") 41 | Y_train = np_utils.to_categorical(y_train, nb_classes) 42 | Y_test = np_utils.to_categorical(y_test, nb_classes) 43 | print('Y_train shape:', Y_train.shape) 44 | print('Y_test shape:', Y_test.shape) 45 | 46 | print("Building model...") 47 | model = Sequential() 48 | model.add(Dense(max_words, 512)) 49 | model.add(Activation('relu')) 50 | model.add(Dropout(0.5)) 51 | model.add(Dense(512, nb_classes)) 52 | model.add(Activation('softmax')) 53 | 54 | model.compile(loss='categorical_crossentropy', optimizer='adam') 55 | 56 | history = model.fit(X_train, Y_train, nb_epoch=nb_epoch, batch_size=batch_size, verbose=1, show_accuracy=True, validation_split=0.1) 57 | score = model.evaluate(X_test, Y_test, batch_size=batch_size, verbose=1, show_accuracy=True) 58 | print('Test score:', score[0]) 59 | print('Test accuracy:', score[1]) 60 | -------------------------------------------------------------------------------- /keras-master/examples/reuters_mlp_using_factors.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import numpy as np 4 | np.random.seed(1337) # for reproducibility 5 | 6 | from keras.datasets import reuters 7 | from keras.models import Sequential 8 | from keras.layers.core import Dense, Dropout, Activation 9 | from keras.layers.normalization import BatchNormalization 10 | from keras.utils import np_utils 11 | from keras.preprocessing.text import Tokenizer 12 | 13 | ''' 14 | Train and evaluate a simple MLP on the Reuters newswire topic classification task. 15 | GPU run command: 16 | THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python examples/reuters_mlp.py 17 | CPU run command: 18 | python examples/reuters_mlp.py 19 | ''' 20 | 21 | max_words = 1000 22 | batch_size = 1024 23 | nb_epoch = 50 24 | 25 | print("Loading data...") 26 | (X_train, y_train), (X_test, y_test) = reuters.load_data(nb_words=max_words, test_split=0.2) 27 | print(len(X_train), 'train sequences') 28 | print(len(X_test), 'test sequences') 29 | 30 | nb_classes = np.max(y_train)+1 31 | print(nb_classes, 'classes') 32 | 33 | print("Vectorizing sequence data...") 34 | tokenizer = Tokenizer(nb_words=max_words) 35 | X_train = tokenizer.sequences_to_matrix(X_train, mode="binary") 36 | X_test = tokenizer.sequences_to_matrix(X_test, mode="binary") 37 | print('X_train shape:', X_train.shape) 38 | print('X_test shape:', X_test.shape) 39 | 40 | 41 | ftrain=open(r'f:\docrep.txt') 42 | xtrain=np.zeros([8982,100]) 43 | cnt=0 44 | for line in ftrain.readlines(): 45 | strfea= line.split(' ') 46 | fid=0 47 | for sf in strfea: 48 | if fid <100: 49 | xtrain[cnt,fid]=float(sf) 50 | fid=fid+1 51 | cnt=cnt+1 52 | 53 | ftest=open(r'f:\docrep.test.txt') 54 | xtest=np.zeros([2246,100]) 55 | cnt=0 56 | for line in ftest.readlines(): 57 | strfea= line.split(' ') 58 | for sf in strfea: 59 | if fid <100: 60 | xtest[cnt,fid]=float(sf) 61 | fid=fid+1 62 | cnt=cnt+1 63 | 64 | 65 | print("Convert class vector to binary class matrix (for use with categorical_crossentropy)") 66 | Y_train = np_utils.to_categorical(y_train, nb_classes) 67 | Y_test = np_utils.to_categorical(y_test, nb_classes) 68 | print('Y_train shape:', Y_train.shape) 69 | print('Y_test shape:', Y_test.shape) 70 | 71 | print("Building model...") 72 | model = Sequential() 73 | model.add(Dense(100, 100)) 74 | model.add(Dense(100, 46)) 75 | model.add(Activation('softmax')) 76 | 77 | model.compile(loss='categorical_crossentropy', optimizer='adam') 78 | 79 | history = model.fit(xtrain, Y_train, nb_epoch=nb_epoch, batch_size=batch_size, verbose=1, show_accuracy=True, validation_split=0.1) 80 | score = model.evaluate(xtest, Y_test, batch_size=batch_size, verbose=1, show_accuracy=True) 81 | print('Test score:', score[0]) 82 | print('Test accuracy:', score[1]) 83 | -------------------------------------------------------------------------------- /keras-master/examples/tempRec - Copy.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | from __future__ import absolute_import 5 | from __future__ import print_function 6 | import numpy as np 7 | import theano 8 | #import random 9 | np.random.seed(1337) # for reproducibility 10 | 11 | from keras.datasets import reuters 12 | from keras.models import Sequential 13 | from keras.layers.recurrent import GRU,LSTM 14 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul 15 | from keras.layers.normalization import BatchNormalization 16 | from keras.utils import np_utils 17 | from keras.preprocessing.text import Tokenizer 18 | 19 | 20 | batchsize=3681 21 | staticFea=300 22 | tempFea=600 23 | 24 | 25 | 26 | 27 | def readbatch(): 28 | global staticFile,tempFile,lblFile,batchsize 29 | static= np.zeros([batchsize,staticFea]) 30 | temp= np.zeros([batchsize,14,tempFea]) 31 | y_train=np.zeros([batchsize,300]) 32 | i=0 33 | while True: 34 | s=staticFile.readline() 35 | t=tempFile.readline() 36 | l=lblFile.readline() 37 | s=unicode(s, errors='ignore') 38 | t=unicode(t, errors='ignore') 39 | l=unicode(l, errors='ignore') 40 | if not s: 41 | break 42 | st= s.split('\t') 43 | j=0 44 | for si in st: 45 | if not si or si=='\n': 46 | j=j-1 47 | else: 48 | static[i, j]=float(si) 49 | j=j+1 50 | 51 | tf=t.split('\t') 52 | j=0 53 | tff=np.zeros(14*600) 54 | for ti in tf: 55 | if not ti or ti=='\n': 56 | j=j-1 57 | else: 58 | tff[ j]=float(ti) 59 | j=j+1 60 | tff=tff.reshape([14,600]) 61 | temp[i]=tff 62 | lbls=l.split('\t') 63 | j=0 64 | for si in lbls: 65 | if not si or si=='\n': 66 | j=j-1 67 | else: 68 | y_train[i, j]=float(si) 69 | j=j+1 70 | 71 | i=i+1 72 | if i==batchsize: 73 | break 74 | if i==batchsize: 75 | hasmore=1 76 | else: 77 | hasmore=0 78 | 79 | return static[0:i-1], temp[0:i-1],y_train[0:i-1],hasmore 80 | 81 | 82 | 83 | staticmodel=Sequential() 84 | staticmodel.add(Dense(300,300)) 85 | staticmodel.add(Activation('tanh')) 86 | tempmodel=Sequential() 87 | tempmodel.add(LSTM(tempFea,300)) 88 | model=Sequential() 89 | model.add(Merge([staticmodel, tempmodel],mode='concat')) 90 | model.add(Dense(300+300,300)) 91 | model.add(Activation('tanh')) 92 | 93 | 94 | 95 | 96 | print('done model construction') 97 | model.compile(loss='mean_squared_error', optimizer='Adadelta') 98 | print('done complie') 99 | for i in range(0,10): 100 | print("itr",i) 101 | staticFile=open(r"\\ZW5338456\F$\newTempOut1\fea.static") 102 | tempFile=open(r"\\ZW5338456\F$\newTempOut1\fea.Temp") 103 | lblFile=open(r"\\ZW5338456\F$\newTempOut1\fea.lbl") 104 | j=0 105 | while True: 106 | print("batch",j) 107 | j=j+1 108 | staticinput ,tempinput, y_train,hasmore = readbatch() 109 | history = model.train_on_batch([staticinput ,tempinput] ,y_train,accuracy=True)# nb_epoch=10, batch_size=1024, verbose=2, show_accuracy=True) 110 | if not hasmore : 111 | staticFile.close() 112 | tempFile.close() 113 | lblFile.close() 114 | break 115 | model.save_weights(r'\\ZW5338456\f$\temprepdiction.model.lstm.'+`i`) 116 | 117 | 118 | model.save_weights(r'\\ZW5338456\f$\temprepdiction.model.lstm.') 119 | -------------------------------------------------------------------------------- /keras-master/examples/tempRec.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | from __future__ import absolute_import 5 | from __future__ import print_function 6 | import numpy as np 7 | import theano 8 | #import random 9 | np.random.seed(1337) # for reproducibility 10 | 11 | from keras.datasets import reuters 12 | from keras.models import Sequential 13 | from keras.layers.recurrent import GRU,LSTM 14 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul 15 | from keras.layers.normalization import BatchNormalization 16 | from keras.utils import np_utils 17 | from keras.preprocessing.text import Tokenizer 18 | 19 | 20 | batchsize=3681 21 | staticFea=300 22 | tempFea=600 23 | 24 | 25 | 26 | 27 | def readbatch(): 28 | global staticFile,tempFile,lblFile,batchsize 29 | static= np.zeros([batchsize,staticFea]) 30 | temp= np.zeros([batchsize,14,tempFea]) 31 | y_train=np.zeros([batchsize,300]) 32 | i=0 33 | while True: 34 | s=staticFile.readline() 35 | t=tempFile.readline() 36 | l=lblFile.readline() 37 | s=unicode(s, errors='ignore') 38 | t=unicode(t, errors='ignore') 39 | l=unicode(l, errors='ignore') 40 | if not s: 41 | break 42 | st= s.split('\t') 43 | j=0 44 | for si in st: 45 | if not si or si=='\n': 46 | j=j-1 47 | else: 48 | static[i, j]=float(si) 49 | j=j+1 50 | 51 | tf=t.split('\t') 52 | j=0 53 | tff=np.zeros(14*600) 54 | for ti in tf: 55 | if not ti or ti=='\n': 56 | j=j-1 57 | else: 58 | tff[ j]=float(ti) 59 | j=j+1 60 | tff=tff.reshape([14,600]) 61 | temp[i]=tff 62 | lbls=l.split('\t') 63 | j=0 64 | for si in lbls: 65 | if not si or si=='\n': 66 | j=j-1 67 | else: 68 | y_train[i, j]=float(si) 69 | j=j+1 70 | 71 | i=i+1 72 | if i==batchsize: 73 | break 74 | if i==batchsize: 75 | hasmore=1 76 | else: 77 | hasmore=0 78 | 79 | return static[0:i-1], temp[0:i-1],y_train[0:i-1],hasmore 80 | 81 | 82 | 83 | staticmodel=Sequential() 84 | staticmodel.add(Dense(300,300)) 85 | staticmodel.add(Activation('tanh')) 86 | tempmodel=Sequential() 87 | tempmodel.add(LSTM(tempFea,300)) 88 | model=Sequential() 89 | model.add(Merge([staticmodel, tempmodel],mode='concat')) 90 | model.add(Dense(300+300,300)) 91 | model.add(Activation('tanh')) 92 | 93 | 94 | 95 | 96 | print('done model construction') 97 | model.compile(loss='mean_squared_error', optimizer='Adadelta') 98 | print('done complie') 99 | for i in range(0,10): 100 | print("itr",i) 101 | staticFile=open(r"\\ZW5338456\F$\newTempOut1\fea.static") 102 | tempFile=open(r"\\ZW5338456\F$\newTempOut1\fea.Temp") 103 | lblFile=open(r"\\ZW5338456\F$\newTempOut1\fea.lbl") 104 | j=0 105 | while True: 106 | print("batch",j) 107 | j=j+1 108 | staticinput ,tempinput, y_train,hasmore = readbatch() 109 | history = model.train_on_batch([staticinput ,tempinput] ,y_train,accuracy=True)# nb_epoch=10, batch_size=1024, verbose=2, show_accuracy=True) 110 | if not hasmore : 111 | staticFile.close() 112 | tempFile.close() 113 | lblFile.close() 114 | break 115 | model.save_weights(r'\\ZW5338456\f$\temprepdiction.model.lstm.'+`i`) 116 | 117 | 118 | model.save_weights(r'\\ZW5338456\f$\temprepdiction.model.lstm.') 119 | -------------------------------------------------------------------------------- /keras-master/examples/tempRecTest.nolstm.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | from __future__ import absolute_import 5 | from __future__ import print_function 6 | import numpy as np 7 | import theano 8 | #import random 9 | np.random.seed(1337) # for reproducibility 10 | 11 | from keras.datasets import reuters 12 | from keras.models import Sequential 13 | from keras.layers.recurrent import GRU 14 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul 15 | from keras.layers.normalization import BatchNormalization 16 | from keras.utils import np_utils 17 | from keras.preprocessing.text import Tokenizer 18 | 19 | 20 | batchsize=3681 21 | staticFea=300 22 | tempFea=300 23 | 24 | 25 | 26 | 27 | 28 | def readbatch(): 29 | global staticFile,lblFile,batchsize 30 | static= np.zeros([batchsize,staticFea]) 31 | #temp= np.zeros([batchsize,7,tempFea]) 32 | 33 | i=0 34 | while True: 35 | s=staticFile.readline() 36 | # t=tempFile.readline() 37 | 38 | s=unicode(s, errors='ignore') 39 | # t=unicode(t, errors='ignore') 40 | 41 | if not s: 42 | break 43 | st= s.split('\t') 44 | j=0 45 | for si in st: 46 | if not si or si=='\n': 47 | j=j-1 48 | else: 49 | static[i, j]=float(si) 50 | j=j+1 51 | 52 | #tf=t.split('\t') 53 | #j=0 54 | #tff=np.zeros(7*300) 55 | #for ti in tf: 56 | # if not ti or ti=='\n': 57 | # j=j-1 58 | # else: 59 | # tff[ j]=float(ti) 60 | # j=j+1 61 | #tff=tff.reshape([7,300]) 62 | #temp[i]=tff 63 | 64 | 65 | i=i+1 66 | if i==batchsize: 67 | break 68 | if i==batchsize: 69 | hasmore=1 70 | else: 71 | hasmore=0 72 | 73 | return static[0:i-1],hasmore 74 | 75 | 76 | staticFile=open(r"\\ZW5338456\F$\newTempOut1\fea_no_user_test.static") 77 | 78 | 79 | outfile=open(r"\\ZW5338456\F$\newTempOut1\featest.nolstm.out",'w') 80 | 81 | #staticinput ,tempinput,hasmore = readbatch() 82 | model=Sequential() 83 | model.add(Dense(300,300)) 84 | model.add(Activation('tanh')) 85 | 86 | 87 | 88 | 89 | print('done model construction') 90 | model.compile(loss='mean_squared_error', optimizer='Adadelta') 91 | print('done complie') 92 | model.load_weights(r'\\ZW5338456\f$\temprepdiction_no_user_.model.nolstm') 93 | 94 | j=0 95 | while True: 96 | print("batch",j) 97 | j=j+1 98 | staticinput ,hasmore = readbatch() 99 | ys = model.predict([staticinput ])# ,y_train,accuracy=True)# nb_epoch=10, batch_size=1024, verbose=2, show_accuracy=True) 100 | for y in ys: 101 | for yi in y: 102 | outfile.write("%s\t" %yi) 103 | outfile.write("\n") 104 | outfile.flush() 105 | if hasmore==0 : 106 | staticFile.close() 107 | # tempFile.close() 108 | 109 | break 110 | 111 | 112 | outfile.close() 113 | 114 | -------------------------------------------------------------------------------- /keras-master/examples/tempRecTest.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | from __future__ import absolute_import 5 | from __future__ import print_function 6 | import numpy as np 7 | import theano 8 | #import random 9 | np.random.seed(1337) # for reproducibility 10 | 11 | from keras.datasets import reuters 12 | from keras.models import Sequential 13 | from keras.layers.recurrent import GRU 14 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul 15 | from keras.layers.normalization import BatchNormalization 16 | from keras.utils import np_utils 17 | from keras.preprocessing.text import Tokenizer 18 | 19 | 20 | batchsize=3681 21 | staticFea=300 22 | tempFea=600 23 | 24 | 25 | 26 | 27 | def readbatch(): 28 | global staticFile,tempFile,lblFile,batchsize 29 | static= np.zeros([batchsize,staticFea]) 30 | temp= np.zeros([batchsize,14,tempFea]) 31 | 32 | i=0 33 | while True: 34 | s=staticFile.readline() 35 | t=tempFile.readline() 36 | 37 | s=unicode(s, errors='ignore') 38 | t=unicode(t, errors='ignore') 39 | 40 | if not s: 41 | break 42 | st= s.split('\t') 43 | j=0 44 | for si in st: 45 | static[i, j]=float(si) 46 | j=j+1 47 | 48 | tf=t.split('\t') 49 | j=0 50 | tff=np.zeros(14*600) 51 | for ti in tf: 52 | if not ti or ti=='\n': 53 | j=j-1 54 | else: 55 | tff[ j]=float(ti) 56 | j=j+1 57 | tff=tff.reshape([14,600]) 58 | temp[i]=tff 59 | i=i+1 60 | if i==batchsize: 61 | break 62 | if i==batchsize: 63 | hasmore=1 64 | else: 65 | hasmore=0 66 | print(hasmore) 67 | return static[0:i-1], temp[0:i-1],hasmore 68 | 69 | 70 | staticFile=open(r"\\ZW5338456\F$\newTempOut1\featest.static") 71 | tempFile=open(r"\\ZW5338456\F$\newTempOut1\featest.Temp") 72 | 73 | outfile=open(r"\\ZW5338456\F$\newTempOut1\featest.out",'w') 74 | 75 | #staticinput ,tempinput,hasmore = readbatch() 76 | staticmodel=Sequential() 77 | staticmodel.add(Dense(300,300)) 78 | tempmodel=Sequential() 79 | tempmodel.add(GRU(tempFea,300)) 80 | model=Sequential() 81 | model.add(Merge([staticmodel, tempmodel],mode='concat')) 82 | model.add(Dense(300+300,300)) 83 | model.add(Activation('tanh')) 84 | 85 | 86 | 87 | 88 | print('done model construction') 89 | model.compile(loss='mean_squared_error', optimizer='Adadelta') 90 | print('done complie') 91 | model.load_weights(r'\\ZW5338456\f$\temprepdiction.model') 92 | 93 | j=0 94 | while True: 95 | print("batch",j) 96 | j=j+1 97 | staticinput ,tempinput,hasmore = readbatch() 98 | ys = model.predict([staticinput ,tempinput])# ,y_train,accuracy=True)# nb_epoch=10, batch_size=1024, verbose=2, show_accuracy=True) 99 | for y in ys: 100 | for yi in y: 101 | outfile.write("%s\t" %yi) 102 | outfile.write("\n") 103 | outfile.flush() 104 | if hasmore==0 : 105 | staticFile.close() 106 | tempFile.close() 107 | 108 | break 109 | 110 | 111 | outfile.close() 112 | 113 | -------------------------------------------------------------------------------- /keras-master/examples/tempRecTest_nouser.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | from __future__ import absolute_import 5 | from __future__ import print_function 6 | import numpy as np 7 | import theano 8 | #import random 9 | np.random.seed(1337) # for reproducibility 10 | 11 | from keras.datasets import reuters 12 | from keras.models import Sequential 13 | from keras.layers.recurrent import GRU,LSTM 14 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul 15 | from keras.layers.normalization import BatchNormalization 16 | from keras.utils import np_utils 17 | from keras.preprocessing.text import Tokenizer 18 | 19 | 20 | batchsize=3681 21 | staticFea=300 22 | tempFea=300 23 | 24 | 25 | 26 | 27 | def readbatch(): 28 | global staticFile,tempFile,lblFile,batchsize 29 | static= np.zeros([batchsize,staticFea]) 30 | temp= np.zeros([batchsize,7,tempFea]) 31 | 32 | i=0 33 | while True: 34 | s=staticFile.readline() 35 | t=tempFile.readline() 36 | 37 | s=unicode(s, errors='ignore') 38 | t=unicode(t, errors='ignore') 39 | 40 | if not s: 41 | break 42 | st= s.split('\t') 43 | j=0 44 | for si in st: 45 | if not si or si=='\n': 46 | j=j-1 47 | else: 48 | static[i, j]=float(si) 49 | j=j+1 50 | 51 | tf=t.split('\t') 52 | j=0 53 | tff=np.zeros(7*300) 54 | for ti in tf: 55 | if not ti or ti=='\n': 56 | j=j-1 57 | else: 58 | tff[ j]=float(ti) 59 | j=j+1 60 | tff=tff.reshape([7,300]) 61 | temp[i]=tff 62 | i=i+1 63 | if i==batchsize: 64 | break 65 | if i==batchsize: 66 | hasmore=1 67 | else: 68 | hasmore=0 69 | print(hasmore) 70 | return static[0:i-1], temp[0:i-1],hasmore 71 | 72 | 73 | staticFile=open(r"\\ZW5338456\F$\newTempOut1\fea_no_user_test.static") 74 | tempFile=open(r"\\ZW5338456\F$\newTempOut1\fea_no_user_test.Temp") 75 | 76 | outfile=open(r"\\ZW5338456\F$\newTempOut1\fea_no_user_test.out",'w') 77 | 78 | #staticinput ,tempinput,hasmore = readbatch() 79 | staticmodel=Sequential() 80 | staticmodel.add(Dense(300,300)) 81 | staticmodel.add(Activation('tanh')) 82 | tempmodel=Sequential() 83 | tempmodel.add(LSTM(tempFea,300)) 84 | model=Sequential() 85 | model.add(Merge([staticmodel, tempmodel],mode='concat')) 86 | model.add(Dense(300+300,300)) 87 | model.add(Activation('tanh')) 88 | 89 | 90 | 91 | 92 | 93 | print('done model construction') 94 | model.compile(loss='mean_squared_error', optimizer='Adadelta') 95 | print('done complie') 96 | model.load_weights(r'\\ZW5338456\f$\temprepdiction_no_user_.model.lstm') 97 | 98 | j=0 99 | while True: 100 | print("batch",j) 101 | j=j+1 102 | staticinput ,tempinput,hasmore = readbatch() 103 | ys = model.predict([staticinput ,tempinput])# ,y_train,accuracy=True)# nb_epoch=10, batch_size=1024, verbose=2, show_accuracy=True) 104 | for y in ys: 105 | for yi in y: 106 | outfile.write("%s\t" %yi) 107 | outfile.write("\n") 108 | outfile.flush() 109 | if hasmore==0 : 110 | staticFile.close() 111 | tempFile.close() 112 | 113 | break 114 | 115 | 116 | outfile.close() 117 | 118 | -------------------------------------------------------------------------------- /keras-master/examples/tempRec_nouser.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | from __future__ import absolute_import 5 | from __future__ import print_function 6 | import numpy as np 7 | import theano 8 | #import random 9 | np.random.seed(1337) # for reproducibility 10 | 11 | from keras.datasets import reuters 12 | from keras.models import Sequential 13 | from keras.layers.recurrent import GRU,LSTM 14 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul 15 | from keras.layers.normalization import BatchNormalization 16 | from keras.utils import np_utils 17 | from keras.preprocessing.text import Tokenizer 18 | 19 | 20 | batchsize=3681 21 | staticFea=300 22 | tempFea=300 23 | 24 | 25 | 26 | 27 | def readbatch(): 28 | global staticFile,tempFile,lblFile,batchsize 29 | static= np.zeros([batchsize,staticFea]) 30 | temp= np.zeros([batchsize,7,tempFea]) 31 | y_train=np.zeros([batchsize,300]) 32 | i=0 33 | while True: 34 | s=staticFile.readline() 35 | t=tempFile.readline() 36 | l=lblFile.readline() 37 | s=unicode(s, errors='ignore') 38 | t=unicode(t, errors='ignore') 39 | l=unicode(l, errors='ignore') 40 | if not s: 41 | break 42 | st= s.split('\t') 43 | j=0 44 | for si in st: 45 | if not si or si=='\n': 46 | j=j-1 47 | else: 48 | static[i, j]=float(si) 49 | j=j+1 50 | 51 | tf=t.split('\t') 52 | j=0 53 | tff=np.zeros(7*300) 54 | for ti in tf: 55 | if not ti or ti=='\n': 56 | j=j-1 57 | else: 58 | tff[ j]=float(ti) 59 | j=j+1 60 | tff=tff.reshape([7,300]) 61 | temp[i]=tff 62 | lbls=l.split('\t') 63 | j=0 64 | for si in lbls: 65 | if not si or si=='\n': 66 | j=j-1 67 | else: 68 | y_train[i, j]=float(si) 69 | j=j+1 70 | 71 | i=i+1 72 | if i==batchsize: 73 | break 74 | if i==batchsize: 75 | hasmore=1 76 | else: 77 | hasmore=0 78 | 79 | return static[0:i-1], temp[0:i-1],y_train[0:i-1],hasmore 80 | 81 | 82 | 83 | staticmodel=Sequential() 84 | staticmodel.add(Dense(300,300)) 85 | staticmodel.add(Activation('tanh')) 86 | tempmodel=Sequential() 87 | tempmodel.add(LSTM(tempFea,300)) 88 | model=Sequential() 89 | model.add(Merge([staticmodel, tempmodel],mode='concat')) 90 | model.add(Dense(300+300,300)) 91 | model.add(Activation('tanh')) 92 | 93 | 94 | 95 | 96 | print('done model construction') 97 | model.compile(loss='mean_squared_error', optimizer='Adadelta') 98 | model.load_weights(r'\\ZW5338456\f$\temprepdiction_no_user_.model.lstm.') 99 | print('done complie') 100 | for i in range(10,30): 101 | print("itr",i) 102 | staticFile=open(r"\\ZW5338456\F$\newTempOut1\fea_no_user_.static") 103 | tempFile=open(r"\\ZW5338456\F$\newTempOut1\fea_no_user_.Temp") 104 | lblFile=open(r"\\ZW5338456\F$\newTempOut1\fea_no_user_.lbl") 105 | j=0 106 | while True: 107 | print("batch",j) 108 | j=j+1 109 | staticinput ,tempinput, y_train,hasmore = readbatch() 110 | history = model.train_on_batch([staticinput ,tempinput] ,y_train,accuracy=True)# nb_epoch=10, batch_size=1024, verbose=2, show_accuracy=True) 111 | if not hasmore : 112 | staticFile.close() 113 | tempFile.close() 114 | lblFile.close() 115 | break 116 | model.save_weights(r'\\ZW5338456\f$\temprepdiction_no_user_.model1.lstm.'+`i`) 117 | 118 | 119 | model.save_weights(r'\\ZW5338456\f$\temprepdiction_no_user_.model1.lstm.') 120 | -------------------------------------------------------------------------------- /keras-master/examples/tempRec_nouser_no_seq.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | from __future__ import absolute_import 5 | from __future__ import print_function 6 | import numpy as np 7 | import theano 8 | #import random 9 | np.random.seed(1337) # for reproducibility 10 | 11 | from keras.datasets import reuters 12 | from keras.models import Sequential 13 | from keras.layers.recurrent import GRU,LSTM 14 | from keras.layers.core import Dense, Dropout, Activation,TimeDistributedDense,Cosine,Merge,Reshape,ElementMul 15 | from keras.layers.normalization import BatchNormalization 16 | from keras.utils import np_utils 17 | from keras.preprocessing.text import Tokenizer 18 | 19 | 20 | batchsize=3681 21 | staticFea=300 22 | tempFea=300 23 | 24 | 25 | 26 | 27 | def readbatch(): 28 | global staticFile,lblFile,batchsize 29 | static= np.zeros([batchsize,staticFea]) 30 | #temp= np.zeros([batchsize,7,tempFea]) 31 | y_train=np.zeros([batchsize,300]) 32 | i=0 33 | while True: 34 | s=staticFile.readline() 35 | # t=tempFile.readline() 36 | l=lblFile.readline() 37 | s=unicode(s, errors='ignore') 38 | # t=unicode(t, errors='ignore') 39 | l=unicode(l, errors='ignore') 40 | if not s: 41 | break 42 | st= s.split('\t') 43 | j=0 44 | for si in st: 45 | if not si or si=='\n': 46 | j=j-1 47 | else: 48 | static[i, j]=float(si) 49 | j=j+1 50 | 51 | #tf=t.split('\t') 52 | #j=0 53 | #tff=np.zeros(7*300) 54 | #for ti in tf: 55 | # if not ti or ti=='\n': 56 | # j=j-1 57 | # else: 58 | # tff[ j]=float(ti) 59 | # j=j+1 60 | #tff=tff.reshape([7,300]) 61 | #temp[i]=tff 62 | lbls=l.split('\t') 63 | j=0 64 | for si in lbls: 65 | if not si or si=='\n': 66 | j=j-1 67 | else: 68 | y_train[i, j]=float(si) 69 | j=j+1 70 | 71 | i=i+1 72 | if i==batchsize: 73 | break 74 | if i==batchsize: 75 | hasmore=1 76 | else: 77 | hasmore=0 78 | 79 | return static[0:i-1],y_train[0:i-1],hasmore 80 | 81 | 82 | 83 | model=Sequential() 84 | model.add(Dense(300,300)) 85 | model.add(Activation('tanh')) 86 | 87 | 88 | 89 | 90 | 91 | print('done model construction') 92 | model.compile(loss='mean_squared_error', optimizer='Adadelta') 93 | print('done complie') 94 | for i in range(0,10): 95 | print("itr",i) 96 | staticFile=open(r"\\ZW5338456\F$\newTempOut1\fea_no_user_.static") 97 | #tempFile=open(r"\\ZW5338456\F$\newTempOut1\fea_no_user_.Temp") 98 | lblFile=open(r"\\ZW5338456\F$\newTempOut1\fea_no_user_.lbl") 99 | j=0 100 | while True: 101 | print("batch",j) 102 | j=j+1 103 | staticinput , y_train,hasmore = readbatch() 104 | history = model.train_on_batch([staticinput ] ,y_train,accuracy=True)# nb_epoch=10, batch_size=1024, verbose=2, show_accuracy=True) 105 | if not hasmore : 106 | staticFile.close() 107 | # tempFile.close() 108 | lblFile.close() 109 | break 110 | model.save_weights(r'\\ZW5338456\f$\temprepdiction_no_user_.model.nolstm.'+`i`) 111 | 112 | 113 | model.save_weights(r'\\ZW5338456\f$\temprepdiction_no_user_.model.nolstm') 114 | -------------------------------------------------------------------------------- /keras-master/keras/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sonyisme/keras-recommendation/e84fab6706faf8405f4bfc75cf50ecff6433eb77/keras-master/keras/__init__.py -------------------------------------------------------------------------------- /keras-master/keras/activations.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import theano.tensor as T 3 | 4 | def softmax(x): 5 | return T.nnet.softmax(x.reshape((-1, x.shape[-1]))).reshape(x.shape) 6 | def normalize_x(x): 7 | x = T.tanh(x) 8 | return x/T.sqrt(T.dot(x,x)) 9 | 10 | def time_distributed_softmax(x): 11 | import warnings 12 | warnings.warn("time_distributed_softmax is deprecated. Just use softmax!", DeprecationWarning) 13 | return softmax(x) 14 | 15 | def softplus(x): 16 | return T.nnet.softplus(x) 17 | 18 | def relu(x): 19 | return (x + abs(x)) / 2.0 20 | 21 | def tanh(x): 22 | return T.tanh(x) 23 | 24 | def sigmoid(x): 25 | return T.nnet.sigmoid(x) 26 | 27 | def hard_sigmoid(x): 28 | return T.nnet.hard_sigmoid(x) 29 | 30 | def linear(x): 31 | ''' 32 | The function returns the variable that is passed in, so all types work 33 | ''' 34 | return x 35 | 36 | from .utils.generic_utils import get_from_module 37 | def get(identifier): 38 | return get_from_module(identifier, globals(), 'activation function') 39 | -------------------------------------------------------------------------------- /keras-master/keras/constraints.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import theano 3 | import theano.tensor as T 4 | import numpy as np 5 | 6 | class Constraint(object): 7 | def __call__(self, p): 8 | return p 9 | 10 | class MaxNorm(Constraint): 11 | def __init__(self, m=2): 12 | self.m = m 13 | 14 | def __call__(self, p): 15 | norms = T.sqrt(T.sum(T.sqr(p), axis=0)) 16 | desired = T.clip(norms, 0, self.m) 17 | p = p * (desired / (1e-7 + norms)) 18 | return p 19 | 20 | class NonNeg(Constraint): 21 | def __call__(self, p): 22 | p *= T.ge(p, 0) 23 | return p 24 | 25 | class UnitNorm(Constraint): 26 | def __call__(self, p): 27 | return p / T.sqrt(T.sum(p**2, axis=-1, keepdims=True)) 28 | 29 | identity = Constraint 30 | maxnorm = MaxNorm 31 | nonneg = NonNeg 32 | unitnorm = UnitNorm 33 | -------------------------------------------------------------------------------- /keras-master/keras/datasets/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sonyisme/keras-recommendation/e84fab6706faf8405f4bfc75cf50ecff6433eb77/keras-master/keras/datasets/__init__.py -------------------------------------------------------------------------------- /keras-master/keras/datasets/cifar.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | from __future__ import absolute_import 3 | import sys 4 | import six.moves.cPickle 5 | from six.moves import range 6 | 7 | def load_batch(fpath, label_key='labels'): 8 | f = open(fpath, 'rb') 9 | if sys.version_info < (3,): 10 | d = six.moves.cPickle.load(f) 11 | else: 12 | d = six.moves.cPickle.load(f, encoding="bytes") 13 | # decode utf8 14 | for k, v in d.items(): 15 | del(d[k]) 16 | d[k.decode("utf8")] = v 17 | f.close() 18 | data = d["data"] 19 | labels = d[label_key] 20 | 21 | data = data.reshape(data.shape[0], 3, 32, 32) 22 | return data, labels 23 | -------------------------------------------------------------------------------- /keras-master/keras/datasets/cifar10.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from .cifar import load_batch 3 | from .data_utils import get_file 4 | import numpy as np 5 | import os 6 | 7 | def load_data(): 8 | dirname = "cifar-10-batches-py" 9 | origin = "http://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz" 10 | path = get_file(dirname, origin=origin, untar=True) 11 | 12 | nb_test_samples = 10000 13 | nb_train_samples = 50000 14 | 15 | X_train = np.zeros((nb_train_samples, 3, 32, 32), dtype="uint8") 16 | y_train = np.zeros((nb_train_samples,), dtype="uint8") 17 | 18 | for i in range(1, 6): 19 | fpath = os.path.join(path, 'data_batch_' + str(i)) 20 | data, labels = load_batch(fpath) 21 | X_train[(i-1)*10000:i*10000, :, :, :] = data 22 | y_train[(i-1)*10000:i*10000] = labels 23 | 24 | fpath = os.path.join(path, 'test_batch') 25 | X_test, y_test = load_batch(fpath) 26 | 27 | y_train = np.reshape(y_train, (len(y_train), 1)) 28 | y_test = np.reshape(y_test, (len(y_test), 1)) 29 | 30 | return (X_train, y_train), (X_test, y_test) -------------------------------------------------------------------------------- /keras-master/keras/datasets/cifar100.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from .cifar import load_batch 3 | from .data_utils import get_file 4 | import numpy as np 5 | import os 6 | 7 | def load_data(label_mode='fine'): 8 | if label_mode not in ['fine', 'coarse']: 9 | raise Exception('label_mode must be one of "fine" "coarse".') 10 | 11 | dirname = "cifar-100-python" 12 | origin = "http://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz" 13 | path = get_file(dirname, origin=origin, untar=True) 14 | 15 | nb_test_samples = 10000 16 | nb_train_samples = 50000 17 | 18 | fpath = os.path.join(path, 'train') 19 | X_train, y_train = load_batch(fpath, label_key=label_mode+'_labels') 20 | 21 | fpath = os.path.join(path, 'test') 22 | X_test, y_test = load_batch(fpath, label_key=label_mode+'_labels') 23 | 24 | y_train = np.reshape(y_train, (len(y_train), 1)) 25 | y_test = np.reshape(y_test, (len(y_test), 1)) 26 | 27 | return (X_train, y_train), (X_test, y_test) -------------------------------------------------------------------------------- /keras-master/keras/datasets/data_utils.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import tarfile, inspect, os 4 | from six.moves.urllib.request import urlretrieve 5 | from ..utils.generic_utils import Progbar 6 | 7 | def get_file(fname, origin, untar=False): 8 | datadir = os.path.expanduser(os.path.join('~', '.keras', 'datasets')) 9 | if not os.path.exists(datadir): 10 | os.makedirs(datadir) 11 | 12 | if untar: 13 | untar_fpath = os.path.join(datadir, fname) 14 | fpath = untar_fpath + '.tar.gz' 15 | else: 16 | fpath = os.path.join(datadir, fname) 17 | 18 | try: 19 | f = open(fpath) 20 | except: 21 | print('Downloading data from', origin) 22 | 23 | global progbar 24 | progbar = None 25 | def dl_progress(count, block_size, total_size): 26 | global progbar 27 | if progbar is None: 28 | progbar = Progbar(total_size) 29 | else: 30 | progbar.update(count*block_size) 31 | 32 | urlretrieve(origin, fpath, dl_progress) 33 | progbar = None 34 | 35 | if untar: 36 | if not os.path.exists(untar_fpath): 37 | print('Untaring file...') 38 | tfile = tarfile.open(fpath, 'r:gz') 39 | tfile.extractall(path=datadir) 40 | tfile.close() 41 | return untar_fpath 42 | 43 | return fpath 44 | 45 | -------------------------------------------------------------------------------- /keras-master/keras/datasets/imdb.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import six.moves.cPickle 3 | import gzip 4 | from .data_utils import get_file 5 | import random 6 | from six.moves import zip 7 | import numpy as np 8 | 9 | def load_data(path="imdb.pkl", nb_words=None, skip_top=0, maxlen=None, test_split=0.2, seed=113, 10 | start_char=1, oov_char=2, index_from=3): 11 | 12 | path = get_file(path, origin="https://s3.amazonaws.com/text-datasets/imdb.pkl") 13 | 14 | if path.endswith(".gz"): 15 | f = gzip.open(path, 'rb') 16 | else: 17 | f = open(path, 'rb') 18 | 19 | X, labels = six.moves.cPickle.load(f) 20 | f.close() 21 | 22 | np.random.seed(seed) 23 | np.random.shuffle(X) 24 | np.random.seed(seed) 25 | np.random.shuffle(labels) 26 | 27 | if start_char is not None: 28 | X = [[start_char] + [w + index_from for w in x] for x in X] 29 | elif index_from: 30 | X = [[w + index_from for w in x] for x in X] 31 | 32 | if maxlen: 33 | new_X = [] 34 | new_labels = [] 35 | for x, y in zip(X, labels): 36 | if len(x) < maxlen: 37 | new_X.append(x) 38 | new_labels.append(y) 39 | X = new_X 40 | labels = new_labels 41 | 42 | if not nb_words: 43 | nb_words = max([max(x) for x in X]) 44 | 45 | # by convention, use 2 as OOV word 46 | # reserve 'index_from' (=3 by default) characters: 0 (padding), 1 (start), 2 (OOV) 47 | if oov_char is not None: 48 | X = [[oov_char if (w >= nb_words or w < skip_top) else w for w in x] for x in X] 49 | else: 50 | nX = [] 51 | for x in X: 52 | nx = [] 53 | for w in x: 54 | if (w >= nb_words or w < skip_top): 55 | nx.append(w) 56 | nX.append(nx) 57 | X = nX 58 | 59 | X_train = X[:int(len(X)*(1-test_split))] 60 | y_train = labels[:int(len(X)*(1-test_split))] 61 | 62 | X_test = X[int(len(X)*(1-test_split)):] 63 | y_test = labels[int(len(X)*(1-test_split)):] 64 | 65 | return (X_train, y_train), (X_test, y_test) 66 | 67 | -------------------------------------------------------------------------------- /keras-master/keras/datasets/mnist.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import gzip 3 | from .data_utils import get_file 4 | import six.moves.cPickle 5 | import sys 6 | 7 | def load_data(path="mnist.pkl.gz"): 8 | path = get_file(path, origin="https://s3.amazonaws.com/img-datasets/mnist.pkl.gz") 9 | 10 | if path.endswith(".gz"): 11 | f = gzip.open(path, 'rb') 12 | else: 13 | f = open(path, 'rb') 14 | 15 | if sys.version_info < (3,): 16 | data = six.moves.cPickle.load(f) 17 | else: 18 | data = six.moves.cPickle.load(f, encoding="bytes") 19 | 20 | f.close() 21 | 22 | return data # (X_train, y_train), (X_test, y_test) 23 | -------------------------------------------------------------------------------- /keras-master/keras/datasets/reuters.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | from __future__ import absolute_import 3 | from __future__ import print_function 4 | from .data_utils import get_file 5 | import string 6 | import random 7 | import os 8 | import six.moves.cPickle 9 | from six.moves import zip 10 | import numpy as np 11 | 12 | def make_reuters_dataset(path=os.path.join('datasets', 'temp', 'reuters21578'), min_samples_per_topic=15): 13 | import re 14 | from ..preprocessing.text import Tokenizer 15 | 16 | wire_topics = [] 17 | topic_counts = {} 18 | wire_bodies = [] 19 | 20 | for fname in os.listdir(path): 21 | if 'sgm' in fname: 22 | s = open(path + fname).read() 23 | tag = '' 24 | while tag in s: 25 | s = s[s.find(tag)+len(tag):] 26 | topics = s[:s.find('' in topics: 29 | topic = topics.replace('', '').replace('', '') 30 | wire_topics.append(topic) 31 | topic_counts[topic] = topic_counts.get(topic, 0) + 1 32 | else: 33 | continue 34 | 35 | bodytag = '' 36 | body = s[s.find(bodytag)+len(bodytag):] 37 | body = body[:body.find('= min_samples_per_topic: 47 | kept_topics.add(x[0]) 48 | print('-') 49 | print('Kept topics:', len(kept_topics)) 50 | 51 | # filter wires with rare topics 52 | kept_wires = [] 53 | labels = [] 54 | topic_indexes = {} 55 | for t, b in zip(wire_topics, wire_bodies): 56 | if t in kept_topics: 57 | if t not in topic_indexes: 58 | topic_index = len(topic_indexes) 59 | topic_indexes[t] = topic_index 60 | else: 61 | topic_index = topic_indexes[t] 62 | 63 | labels.append(topic_index) 64 | kept_wires.append(b) 65 | 66 | # vectorize wires 67 | tokenizer = Tokenizer() 68 | tokenizer.fit_on_texts(kept_wires) 69 | X = tokenizer.texts_to_sequences(kept_wires) 70 | 71 | print('Sanity check:') 72 | for w in ["banana", "oil", "chocolate", "the", "dsft"]: 73 | print('...index of', w, ':', tokenizer.word_index.get(w)) 74 | 75 | dataset = (X, labels) 76 | print('-') 77 | print('Saving...') 78 | six.moves.cPickle.dump(dataset, open(os.path.join('datasets', 'data', 'reuters.pkl'), 'w')) 79 | six.moves.cPickle.dump(tokenizer.word_index, open(os.path.join('datasets','data', 'reuters_word_index.pkl'), 'w')) 80 | 81 | 82 | 83 | def load_data(path="reuters.pkl", nb_words=None, skip_top=0, maxlen=None, test_split=0.2, seed=113, 84 | start_char=1, oov_char=2, index_from=3): 85 | 86 | path = get_file(path, origin="https://s3.amazonaws.com/text-datasets/reuters.pkl") 87 | f = open(path, 'rb') 88 | 89 | X, labels = six.moves.cPickle.load(f) 90 | f.close() 91 | 92 | np.random.seed(seed) 93 | np.random.shuffle(X) 94 | np.random.seed(seed) 95 | np.random.shuffle(labels) 96 | 97 | if start_char is not None: 98 | X = [[start_char] + [w + index_from for w in x] for x in X] 99 | elif index_from: 100 | X = [[w + index_from for w in x] for x in X] 101 | 102 | if maxlen: 103 | new_X = [] 104 | new_labels = [] 105 | for x, y in zip(X, labels): 106 | if len(x) < maxlen: 107 | new_X.append(x) 108 | new_labels.append(y) 109 | X = new_X 110 | labels = new_labels 111 | 112 | if not nb_words: 113 | nb_words = max([max(x) for x in X]) 114 | 115 | # by convention, use 2 as OOV word 116 | # reserve 'index_from' (=3 by default) characters: 0 (padding), 1 (start), 2 (OOV) 117 | if oov_char is not None: 118 | X = [[oov_char if (w >= nb_words or w < skip_top) else w for w in x] for x in X] 119 | else: 120 | nX = [] 121 | for x in X: 122 | nx = [] 123 | for w in x: 124 | if (w >= nb_words or w < skip_top): 125 | nx.append(w) 126 | nX.append(nx) 127 | X = nX 128 | 129 | X_train = X[:int(len(X)*(1-test_split))] 130 | y_train = labels[:int(len(X)*(1-test_split))] 131 | 132 | X_test = X[int(len(X)*(1-test_split)):] 133 | y_test = labels[int(len(X)*(1-test_split)):] 134 | 135 | return (X_train, y_train), (X_test, y_test) 136 | 137 | 138 | def get_word_index(path="reuters_word_index.pkl"): 139 | path = get_file(path, origin="https://s3.amazonaws.com/text-datasets/reuters_word_index.pkl") 140 | f = open(path, 'rb') 141 | return six.moves.cPickle.load(f) 142 | 143 | 144 | if __name__ == "__main__": 145 | make_reuters_dataset() 146 | (X_train, y_train), (X_test, y_test) = load_data() 147 | -------------------------------------------------------------------------------- /keras-master/keras/initializations.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import theano 3 | import theano.tensor as T 4 | import numpy as np 5 | 6 | from .utils.theano_utils import sharedX, shared_zeros, shared_ones 7 | 8 | def get_fans(shape): 9 | fan_in = shape[0] if len(shape) == 2 else np.prod(shape[1:]) 10 | fan_out = shape[1] if len(shape) == 2 else shape[0] 11 | return fan_in, fan_out 12 | 13 | 14 | def uniform(shape, scale=0.05): 15 | return sharedX(np.random.uniform(low=-scale, high=scale, size=shape)) 16 | 17 | def normal(shape, scale=0.05): 18 | return sharedX(np.random.randn(*shape) * scale) 19 | 20 | def lecun_uniform(shape): 21 | ''' Reference: LeCun 98, Efficient Backprop 22 | http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf 23 | ''' 24 | fan_in, fan_out = get_fans(shape) 25 | scale = 1./np.sqrt(fan_in) 26 | return uniform(shape, scale) 27 | 28 | def glorot_normal(shape): 29 | ''' Reference: Glorot & Bengio, AISTATS 2010 30 | ''' 31 | fan_in, fan_out = get_fans(shape) 32 | s = np.sqrt(2. / (fan_in + fan_out)) 33 | return normal(shape, s) 34 | 35 | def glorot_uniform(shape): 36 | fan_in, fan_out = get_fans(shape) 37 | s = np.sqrt(2. / (fan_in + fan_out)) 38 | return uniform(shape, s) 39 | 40 | def he_normal(shape): 41 | ''' Reference: He et al., http://arxiv.org/abs/1502.01852 42 | ''' 43 | fan_in, fan_out = get_fans(shape) 44 | s = np.sqrt(2. / fan_in) 45 | return normal(shape, s) 46 | 47 | def he_uniform(shape): 48 | fan_in, fan_out = get_fans(shape) 49 | s = np.sqrt(2. / fan_in) 50 | return uniform(shape, s) 51 | 52 | def orthogonal(shape, scale=1.1): 53 | ''' From Lasagne 54 | ''' 55 | flat_shape = (shape[0], np.prod(shape[1:])) 56 | a = np.random.normal(0.0, 1.0, flat_shape) 57 | u, _, v = np.linalg.svd(a, full_matrices=False) 58 | q = u if u.shape == flat_shape else v # pick the one with the correct shape 59 | q = q.reshape(shape) 60 | return sharedX(scale * q[:shape[0], :shape[1]]) 61 | 62 | def identity(shape, scale=1): 63 | if len(shape) != 2 or shape[0] != shape[1]: 64 | raise Exception("Identity matrix initialization can only be used for 2D square matrices") 65 | else: 66 | return sharedX(scale * np.identity(shape[0])) 67 | 68 | def zero(shape): 69 | return shared_zeros(shape) 70 | 71 | def one(shape): 72 | return shared_ones(shape) 73 | 74 | 75 | from .utils.generic_utils import get_from_module 76 | def get(identifier): 77 | return get_from_module(identifier, globals(), 'initialization') 78 | -------------------------------------------------------------------------------- /keras-master/keras/layers/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sonyisme/keras-recommendation/e84fab6706faf8405f4bfc75cf50ecff6433eb77/keras-master/keras/layers/__init__.py -------------------------------------------------------------------------------- /keras-master/keras/layers/advanced_activations.py: -------------------------------------------------------------------------------- 1 | from ..layers.core import Layer 2 | from ..utils.theano_utils import shared_zeros 3 | 4 | class LeakyReLU(Layer): 5 | def __init__(self, alpha=0.3): 6 | super(LeakyReLU,self).__init__() 7 | self.alpha = alpha 8 | 9 | def get_output(self, train): 10 | X = self.get_input(train) 11 | return ((X + abs(X)) / 2.0) + self.alpha * ((X - abs(X)) / 2.0) 12 | 13 | def get_config(self): 14 | return {"name":self.__class__.__name__, 15 | "alpha":self.alpha} 16 | 17 | 18 | class PReLU(Layer): 19 | ''' 20 | Reference: 21 | Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification 22 | http://arxiv.org/pdf/1502.01852v1.pdf 23 | ''' 24 | def __init__(self, input_shape): 25 | super(PReLU,self).__init__() 26 | self.alphas = shared_zeros(input_shape) 27 | self.params = [self.alphas] 28 | self.input_shape = input_shape 29 | 30 | def get_output(self, train): 31 | X = self.get_input(train) 32 | pos = ((X + abs(X)) / 2.0) 33 | neg = self.alphas * ((X - abs(X)) / 2.0) 34 | return pos + neg 35 | 36 | def get_config(self): 37 | return {"name":self.__class__.__name__, 38 | "input_shape":self.input_shape} 39 | -------------------------------------------------------------------------------- /keras-master/keras/layers/embeddings.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import theano 3 | import theano.tensor as T 4 | 5 | from .. import activations, initializations 6 | from ..layers.core import Layer, MaskedLayer 7 | from ..utils.theano_utils import sharedX 8 | 9 | from ..constraints import unitnorm 10 | 11 | 12 | class Embedding(Layer): 13 | ''' 14 | Turn positive integers (indexes) into denses vectors of fixed size. 15 | eg. [[4], [20]] -> [[0.25, 0.1], [0.6, -0.2]] 16 | 17 | @input_dim: size of vocabulary (highest input integer + 1) 18 | @out_dim: size of dense representation 19 | ''' 20 | def __init__(self, input_dim, output_dim, init='uniform', 21 | W_regularizer=None, activity_regularizer=None, W_constraint=None, 22 | mask_zero=False, weights=None): 23 | 24 | super(Embedding,self).__init__() 25 | self.init = initializations.get(init) 26 | self.input_dim = input_dim 27 | self.output_dim = output_dim 28 | 29 | self.input = T.imatrix() 30 | self.W = self.init((self.input_dim, self.output_dim)) 31 | self.mask_zero = mask_zero 32 | 33 | self.params = [self.W] 34 | self.constraints = [W_constraint] 35 | 36 | self.regularizers = [] 37 | if W_regularizer: 38 | W_regularizer.set_param(self.W) 39 | self.regularizers.append(W_regularizer) 40 | if activity_regularizer: 41 | activity_regularizer.set_layer(self) 42 | self.regularizers.append(activity_regularizer) 43 | 44 | if weights is not None: 45 | self.set_weights(weights) 46 | 47 | def get_output_mask(self, train=None): 48 | X = self.get_input(train) 49 | if not self.mask_zero: 50 | return None 51 | else: 52 | return T.ones_like(X) * (1 - T.eq(X,0)) 53 | 54 | def get_output(self, train=False): 55 | X = self.get_input(train) 56 | out = self.W[X] 57 | return out 58 | 59 | def get_config(self): 60 | return {"name":self.__class__.__name__, 61 | "input_dim":self.input_dim, 62 | "output_dim":self.output_dim, 63 | "init":self.init.__name__} 64 | 65 | 66 | class WordContextProduct(Layer): 67 | ''' 68 | This layer turns a pair of words (a pivot word + a context word, 69 | ie. a word from the same context, or a random, out-of-context word), 70 | indentified by their index in a vocabulary, into two dense reprensentations 71 | (word representation and context representation). 72 | 73 | Then it returns activation(dot(pivot_embedding, context_embedding)), 74 | which can be trained to encode the probability 75 | of finding the context word in the context of the pivot word 76 | (or reciprocally depending on your training procedure). 77 | 78 | The layer ingests integer tensors of shape: 79 | (nb_samples, 2) 80 | and outputs a float tensor of shape 81 | (nb_samples, 1) 82 | 83 | The 2nd dimension encodes (pivot, context). 84 | input_dim is the size of the vocabulary. 85 | 86 | For more context, see Mikolov et al.: 87 | Efficient Estimation of Word reprensentations in Vector Space 88 | http://arxiv.org/pdf/1301.3781v3.pdf 89 | ''' 90 | def __init__(self, input_dim, proj_dim=128, 91 | init='uniform', activation='sigmoid', weights=None): 92 | super(WordContextProduct,self).__init__() 93 | self.input_dim = input_dim 94 | self.proj_dim = proj_dim 95 | self.init = initializations.get(init) 96 | self.activation = activations.get(activation) 97 | 98 | self.input = T.imatrix() 99 | # two different embeddings for pivot word and its context 100 | # because p(w|c) != p(c|w) 101 | self.W_w = self.init((input_dim, proj_dim)) 102 | self.W_c = self.init((input_dim, proj_dim)) 103 | 104 | self.params = [self.W_w, self.W_c] 105 | 106 | if weights is not None: 107 | self.set_weights(weights) 108 | 109 | 110 | def get_output(self, train=False): 111 | X = self.get_input(train) 112 | w = self.W_w[X[:, 0]] # nb_samples, proj_dim 113 | c = self.W_c[X[:, 1]] # nb_samples, proj_dim 114 | 115 | dot = T.sum(w * c, axis=1) 116 | dot = theano.tensor.reshape(dot, (X.shape[0], 1)) 117 | return self.activation(dot) 118 | 119 | def get_config(self): 120 | return {"name":self.__class__.__name__, 121 | "input_dim":self.input_dim, 122 | "proj_dim":self.proj_dim, 123 | "init":self.init.__name__, 124 | "activation":self.activation.__name__} 125 | 126 | -------------------------------------------------------------------------------- /keras-master/keras/layers/noise.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from .core import srng, MaskedLayer 3 | import theano 4 | 5 | class GaussianNoise(MaskedLayer): 6 | ''' 7 | Corruption process with GaussianNoise 8 | ''' 9 | def __init__(self, sigma): 10 | super(GaussianNoise, self).__init__() 11 | self.sigma = sigma 12 | 13 | def get_output(self, train=False): 14 | X = self.get_input(train) 15 | if not train or self.sigma == 0: 16 | return X 17 | else: 18 | return X + srng.normal(size=X.shape, avg=0.0, std=self.sigma, 19 | dtype=theano.config.floatX) 20 | 21 | def get_config(self): 22 | return {"name":self.__class__.__name__, 23 | "sigma":self.sigma} 24 | 25 | class GaussianDropout(Layer): 26 | ''' 27 | Multiplicative Gaussian Noise 28 | Reference: 29 | Dropout: A Simple Way to Prevent Neural Networks from Overfitting 30 | Srivastava, Hinton, et al. 2014 31 | http://www.cs.toronto.edu/~rsalakhu/papers/srivastava14a.pdf 32 | ''' 33 | def __init__(self, p): 34 | super(GaussianDropout,self).__init__() 35 | self.p = p 36 | 37 | def get_output(self, train): 38 | X = self.get_input(train) 39 | if train: 40 | # self.p refers to drop probability rather than retain probability (as in paper) to match Dropout layer syntax 41 | X *= srng.normal(size=X.shape, avg=1.0, std=T.sqrt(self.p / (1.0 - self.p)), dtype=theano.config.floatX) 42 | return X 43 | 44 | def get_config(self): 45 | return {"name":self.__class__.__name__, 46 | "p":self.p} 47 | -------------------------------------------------------------------------------- /keras-master/keras/layers/normalization.py: -------------------------------------------------------------------------------- 1 | from ..layers.core import Layer 2 | from ..utils.theano_utils import shared_zeros 3 | from .. import initializations 4 | 5 | import theano.tensor as T 6 | 7 | class BatchNormalization(Layer): 8 | ''' 9 | Reference: 10 | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift 11 | http://arxiv.org/pdf/1502.03167v3.pdf 12 | 13 | mode: 0 -> featurewise normalization 14 | 1 -> samplewise normalization (may sometimes outperform featurewise mode) 15 | 16 | momentum: momentum term in the computation of a running estimate of the mean and std of the data 17 | ''' 18 | def __init__(self, input_shape, epsilon=1e-6, mode=0, momentum=0.9, weights=None): 19 | super(BatchNormalization,self).__init__() 20 | self.init = initializations.get("uniform") 21 | self.input_shape = input_shape 22 | self.epsilon = epsilon 23 | self.mode = mode 24 | self.momentum = momentum 25 | 26 | self.gamma = self.init((self.input_shape)) 27 | self.beta = shared_zeros(self.input_shape) 28 | 29 | self.running_mean = None 30 | self.running_std = None 31 | 32 | self.params = [self.gamma, self.beta] 33 | if weights is not None: 34 | self.set_weights(weights) 35 | 36 | def get_output(self, train): 37 | X = self.get_input(train) 38 | 39 | if self.mode == 0: 40 | if train: 41 | m = X.mean(axis=0) 42 | # manual computation of std to prevent NaNs 43 | std = T.mean((X-m)**2 + self.epsilon, axis=0) ** 0.5 44 | X_normed = (X - m) / (std + self.epsilon) 45 | 46 | if self.running_mean is None: 47 | self.running_mean = m 48 | self.running_std = std 49 | else: 50 | self.running_mean *= self.momentum 51 | self.running_mean += (1-self.momentum) * m 52 | self.running_std *= self.momentum 53 | self.running_std += (1-self.momentum) * std 54 | else: 55 | X_normed = (X - self.running_mean) / (self.running_std + self.epsilon) 56 | 57 | elif self.mode == 1: 58 | m = X.mean(axis=-1, keepdims=True) 59 | std = X.std(axis=-1, keepdims=True) 60 | X_normed = (X - m) / (std + self.epsilon) 61 | 62 | out = self.gamma * X_normed + self.beta 63 | return out 64 | 65 | def get_config(self): 66 | return {"name":self.__class__.__name__, 67 | "input_shape":self.input_shape, 68 | "epsilon":self.epsilon, 69 | "mode":self.mode} 70 | 71 | 72 | class LRN2D(Layer): 73 | """ 74 | This code is adapted from pylearn2. 75 | License at: https://github.com/lisa-lab/pylearn2/blob/master/LICENSE.txt 76 | """ 77 | 78 | def __init__(self, alpha=1e-4, k=2, beta=0.75, n=5): 79 | if n % 2 == 0: 80 | raise NotImplementedError("LRN2D only works with odd n. n provided: " + str(n)) 81 | super(LRN2D, self).__init__() 82 | self.alpha = alpha 83 | self.k = k 84 | self.beta = beta 85 | self.n = n 86 | 87 | def get_output(self, train): 88 | X = self.get_input(train) 89 | b, ch, r, c = X.shape 90 | half_n = self.n // 2 91 | input_sqr = T.sqr(X) 92 | extra_channels = T.alloc(0., b, ch + 2*half_n, r, c) 93 | input_sqr = T.set_subtensor(extra_channels[:, half_n:half_n+ch, :, :], input_sqr) 94 | scale = self.k 95 | for i in range(self.n): 96 | scale += self.alpha * input_sqr[:, i:i+ch, :, :] 97 | scale = scale ** self.beta 98 | return X / scale 99 | 100 | def get_config(self): 101 | return {"name":self.__class__.__name__, 102 | "alpha":self.alpha, 103 | "k":self.k, 104 | "beta":self.beta, 105 | "n": self.n} -------------------------------------------------------------------------------- /keras-master/keras/objectives.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import theano 3 | import theano.tensor as T 4 | import numpy as np 5 | from six.moves import range 6 | 7 | epsilon = 1.0e-9 8 | 9 | def mean_squared_error(y_true, y_pred): 10 | return T.sqr(y_pred - y_true).mean(axis=-1) 11 | 12 | def mean_absolute_error(y_true, y_pred): 13 | return T.abs_(y_pred - y_true).mean(axis=-1) 14 | 15 | def mean_absolute_percentage_error(y_true, y_pred): 16 | return T.abs_((y_true - y_pred) / T.clip(T.abs_(y_true), epsilon, np.inf)).mean(axis=-1) * 100. 17 | 18 | def mean_squared_logarithmic_error(y_true, y_pred): 19 | return T.sqr(T.log(T.clip(y_pred, epsilon, np.inf) + 1.) - T.log(T.clip(y_true, epsilon, np.inf) + 1.)).mean(axis=-1) 20 | 21 | def squared_hinge(y_true, y_pred): 22 | return T.sqr(T.maximum(1. - y_true * y_pred, 0.)).mean(axis=-1) 23 | 24 | def hinge(y_true, y_pred): 25 | return T.maximum(1. - y_true * y_pred, 0.).mean(axis=-1) 26 | 27 | def cos_loss(y_true, y_pred): 28 | return 1-T.dot(y_true , y_pred)/T.dot(y_pred,y_pred) 29 | 30 | def categorical_crossentropy(y_true, y_pred): 31 | '''Expects a binary class matrix instead of a vector of scalar classes 32 | ''' 33 | y_pred = T.clip(y_pred, epsilon, 1.0 - epsilon) 34 | # scale preds so that the class probas of each sample sum to 1 35 | y_pred /= y_pred.sum(axis=-1, keepdims=True) 36 | cce = T.nnet.categorical_crossentropy(y_pred, y_true) 37 | return cce 38 | 39 | def binary_crossentropy(y_true, y_pred): 40 | y_pred = T.clip(y_pred, epsilon, 1.0 - epsilon) 41 | bce = T.nnet.binary_crossentropy(y_pred, y_true) 42 | return bce 43 | 44 | # aliases 45 | mse = MSE = mean_squared_error 46 | mae = MAE = mean_absolute_error 47 | mape = MAPE = mean_absolute_percentage_error 48 | msle = MSLE = mean_squared_logarithmic_error 49 | 50 | from .utils.generic_utils import get_from_module 51 | def get(identifier): 52 | return get_from_module(identifier, globals(), 'objective') 53 | 54 | -------------------------------------------------------------------------------- /keras-master/keras/preprocessing/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sonyisme/keras-recommendation/e84fab6706faf8405f4bfc75cf50ecff6433eb77/keras-master/keras/preprocessing/__init__.py -------------------------------------------------------------------------------- /keras-master/keras/preprocessing/sequence.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | # -*- coding: utf-8 -*- 3 | import numpy as np 4 | import random 5 | from six.moves import range 6 | 7 | def pad_sequences(sequences, maxlen=None, dtype='int32', padding='pre', truncating='pre', value=0.): 8 | """ 9 | Pad each sequence to the same length: 10 | the length of the longuest sequence. 11 | 12 | If maxlen is provided, any sequence longer 13 | than maxlen is truncated to maxlen. Truncation happens off either the beginning (default) or 14 | the end of the sequence. 15 | 16 | Supports post-padding and pre-padding (default). 17 | 18 | """ 19 | lengths = [len(s) for s in sequences] 20 | 21 | nb_samples = len(sequences) 22 | if maxlen is None: 23 | maxlen = np.max(lengths) 24 | 25 | x = (np.ones((nb_samples, maxlen)) * value).astype(dtype) 26 | for idx, s in enumerate(sequences): 27 | if truncating == 'pre': 28 | trunc = s[-maxlen:] 29 | elif truncating == 'post': 30 | trunc = s[:maxlen] 31 | else: 32 | raise ValueError("Truncating type '%s' not understood" % padding) 33 | 34 | if padding == 'post': 35 | x[idx, :len(trunc)] = trunc 36 | elif padding == 'pre': 37 | x[idx, -len(trunc):] = trunc 38 | else: 39 | raise ValueError("Padding type '%s' not understood" % padding) 40 | return x 41 | 42 | 43 | def make_sampling_table(size, sampling_factor=1e-5): 44 | ''' 45 | This generates an array where the ith element 46 | is the probability that a word of rank i would be sampled, 47 | according to the sampling distribution used in word2vec. 48 | 49 | The word2vec formula is: 50 | p(word) = min(1, sqrt(word.frequency/sampling_factor) / (word.frequency/sampling_factor)) 51 | 52 | We assume that the word frequencies follow Zipf's law (s=1) to derive 53 | a numerical approximation of frequency(rank): 54 | frequency(rank) ~ 1/(rank * (log(rank) + gamma) + 1/2 - 1/(12*rank)) 55 | where gamma is the Euler-Mascheroni constant. 56 | ''' 57 | gamma = 0.577 58 | rank = np.array(list(range(size))) 59 | rank[0] = 1 60 | inv_fq = rank * (np.log(rank) + gamma) + 0.5 - 1./(12.*rank) 61 | f = sampling_factor * inv_fq 62 | return np.minimum(1., f / np.sqrt(f)) 63 | 64 | 65 | def skipgrams(sequence, vocabulary_size, 66 | window_size=4, negative_samples=1., shuffle=True, 67 | categorical=False, sampling_table=None): 68 | ''' 69 | Take a sequence (list of indexes of words), 70 | returns couples of [word_index, other_word index] and labels (1s or 0s), 71 | where label = 1 if 'other_word' belongs to the context of 'word', 72 | and label=0 if 'other_word' is ramdomly sampled 73 | 74 | @param vocabulary_size: int. maximum possible word index + 1 75 | @param window_size: int. actually half-window. The window of a word wi will be [i-window_size, i+window_size+1] 76 | @param negative_samples: float >= 0. 0 for no negative (=random) samples. 1 for same number as positive samples. etc. 77 | @param categorical: bool. if False, labels will be integers (eg. [0, 1, 1 .. ]), 78 | if True labels will be categorical eg. [[1,0],[0,1],[0,1] .. ] 79 | 80 | Note: by convention, index 0 in the vocabulary is a non-word and will be skipped. 81 | ''' 82 | couples = [] 83 | labels = [] 84 | for i, wi in enumerate(sequence): 85 | if not wi: 86 | continue 87 | if sampling_table is not None: 88 | if sampling_table[wi] < random.random(): 89 | continue 90 | 91 | window_start = max(0, i-window_size) 92 | window_end = min(len(sequence), i+window_size+1) 93 | for j in range(window_start, window_end): 94 | if j != i: 95 | wj = sequence[j] 96 | if not wj: 97 | continue 98 | couples.append([wi, wj]) 99 | if categorical: 100 | labels.append([0,1]) 101 | else: 102 | labels.append(1) 103 | 104 | if negative_samples > 0: 105 | nb_negative_samples = int(len(labels) * negative_samples) 106 | words = [c[0] for c in couples] 107 | random.shuffle(words) 108 | 109 | couples += [[words[i%len(words)], random.randint(1, vocabulary_size-1)] for i in range(nb_negative_samples)] 110 | if categorical: 111 | labels += [[1,0]]*nb_negative_samples 112 | else: 113 | labels += [0]*nb_negative_samples 114 | 115 | if shuffle: 116 | seed = random.randint(0,10e6) 117 | random.seed(seed) 118 | random.shuffle(couples) 119 | random.seed(seed) 120 | random.shuffle(labels) 121 | 122 | return couples, labels 123 | -------------------------------------------------------------------------------- /keras-master/keras/regularizers.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import theano.tensor as T 3 | 4 | class Regularizer(object): 5 | def set_param(self, p): 6 | self.p = p 7 | 8 | def set_layer(self, layer): 9 | self.layer = layer 10 | 11 | def __call__(self, loss): 12 | return loss 13 | 14 | 15 | class WeightRegularizer(Regularizer): 16 | def __init__(self, l1=0., l2=0.): 17 | self.l1 = l1 18 | self.l2 = l2 19 | 20 | def set_param(self, p): 21 | self.p = p 22 | 23 | def __call__(self, loss): 24 | loss += T.sum(abs(self.p)) * self.l1 25 | loss += T.sum(self.p ** 2) * self.l2 26 | return loss 27 | 28 | 29 | class ActivityRegularizer(Regularizer): 30 | def __init__(self, l1=0., l2=0.): 31 | self.l1 = l1 32 | self.l2 = l2 33 | 34 | def set_layer(self, layer): 35 | self.layer = layer 36 | 37 | def __call__(self, loss): 38 | loss += self.l1 * T.sum(T.mean(abs(self.layer.get_output(True)), axis=0)) 39 | loss += self.l2 * T.sum(T.mean(self.layer.get_output(True) ** 2, axis=0)) 40 | return loss 41 | 42 | 43 | def l1(l=0.01): 44 | return WeightRegularizer(l1=l) 45 | 46 | def l2(l=0.01): 47 | return WeightRegularizer(l2=l) 48 | 49 | def l1l2(l1=0.01, l2=0.01): 50 | return WeightRegularizer(l1=l1, l2=l2) 51 | 52 | def activity_l1(l=0.01): 53 | return ActivityRegularizer(l1=l) 54 | 55 | def activity_l2(l=0.01): 56 | return ActivityRegularizer(l2=l) 57 | 58 | def activity_l1l2(l1=0.01, l2=0.01): 59 | return ActivityRegularizer(l1=l1, l2=l2) 60 | 61 | identity = Regularizer 62 | -------------------------------------------------------------------------------- /keras-master/keras/utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sonyisme/keras-recommendation/e84fab6706faf8405f4bfc75cf50ecff6433eb77/keras-master/keras/utils/__init__.py -------------------------------------------------------------------------------- /keras-master/keras/utils/dot_utils.py: -------------------------------------------------------------------------------- 1 | import pydot 2 | from keras.layers.core import Merge 3 | from keras.models import Model 4 | from collections import Counter 5 | 6 | class Grapher(object): 7 | 8 | def __init__(self): 9 | self.names = {} 10 | self.class_counts = Counter() 11 | 12 | def get_name(self, model): 13 | """ 14 | returns the name of the model instance. If model does not have a `name` attribute, then it will be assigned 15 | a generic (and unique) identifier based on its class 16 | """ 17 | if hasattr(model, 'name'): 18 | return model.name 19 | clz = model.__class__.__name__ 20 | if model not in self.names: 21 | self.class_counts[clz] += 1 22 | self.names[model] = clz + str(self.class_counts[clz]) 23 | return self.names[model] 24 | 25 | def add_edge(self, f, t, graph): 26 | if f: graph.add_edge(pydot.Edge(f, t)) 27 | return t 28 | 29 | def add_model(self, model, graph, parent=None): 30 | """ 31 | Recursively adds `model` and its components to the pydot graph 32 | """ 33 | this = self.get_name(model) 34 | if isinstance(model, Model): 35 | parent = self.add_edge(parent, this, graph) 36 | for child in reversed(model.layers): 37 | parent = self.add_model(child, graph, parent) 38 | elif isinstance(model, Merge): 39 | for child in model.models: 40 | self.add_model(child, graph, this) 41 | return self.add_edge(parent, this, graph) 42 | else: 43 | return self.add_edge(parent, this, graph) 44 | 45 | def plot(self, model, to_file): 46 | """ 47 | creates a graph visualizing the structure of `model` and writes it to `to_file` 48 | """ 49 | graph = pydot.Dot(graph_type='graph') 50 | self.add_model(model, graph) 51 | graph.write_png(to_file) 52 | -------------------------------------------------------------------------------- /keras-master/keras/utils/generic_utils.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import numpy as np 3 | import time 4 | import sys 5 | import six 6 | 7 | def get_from_module(identifier, module_params, module_name, instantiate=False): 8 | if isinstance(identifier, six.string_types): 9 | res = module_params.get(identifier) 10 | if not res: 11 | raise Exception('Invalid ' + str(module_name) + ': ' + str(identifier)) 12 | if instantiate: 13 | return res() 14 | else: 15 | return res 16 | return identifier 17 | 18 | def make_tuple(*args): 19 | return args 20 | 21 | def printv(v, prefix=''): 22 | if type(v) == dict: 23 | if 'name' in v: 24 | print(prefix + '#' + v['name']) 25 | del v['name'] 26 | prefix += '...' 27 | for nk, nv in v.items(): 28 | if type(nv) in [dict, list]: 29 | print(prefix + nk + ':') 30 | printv(nv, prefix) 31 | else: 32 | print(prefix + nk + ':' + str(nv)) 33 | elif type(v) == list: 34 | prefix += '...' 35 | for i, nv in enumerate(v): 36 | print(prefix + '#' + str(i)) 37 | printv(nv, prefix) 38 | else: 39 | prefix += '...' 40 | print(prefix + str(v)) 41 | 42 | class Progbar(object): 43 | def __init__(self, target, width=30, verbose=1): 44 | ''' 45 | @param target: total number of steps expected 46 | ''' 47 | self.width = width 48 | self.target = target 49 | self.sum_values = {} 50 | self.unique_values = [] 51 | self.start = time.time() 52 | self.total_width = 0 53 | self.seen_so_far = 0 54 | self.verbose = verbose 55 | 56 | def update(self, current, values=[]): 57 | ''' 58 | @param current: index of current step 59 | @param values: list of tuples (name, value_for_last_step). 60 | The progress bar will display averages for these values. 61 | ''' 62 | for k, v in values: 63 | if k not in self.sum_values: 64 | self.sum_values[k] = [v * (current-self.seen_so_far), current-self.seen_so_far] 65 | self.unique_values.append(k) 66 | else: 67 | self.sum_values[k][0] += v * (current-self.seen_so_far) 68 | self.sum_values[k][1] += (current-self.seen_so_far) 69 | self.seen_so_far = current 70 | 71 | now = time.time() 72 | if self.verbose == 1: 73 | prev_total_width = self.total_width 74 | sys.stdout.write("\b" * prev_total_width) 75 | sys.stdout.write("\r") 76 | 77 | numdigits = int(np.floor(np.log10(self.target))) + 1 78 | barstr = '%%%dd/%%%dd [' % (numdigits, numdigits) 79 | bar = barstr % (current, self.target) 80 | prog = float(current)/self.target 81 | prog_width = int(self.width*prog) 82 | if prog_width > 0: 83 | bar += ('='*(prog_width-1)) 84 | if current < self.target: 85 | bar += '>' 86 | else: 87 | bar += '=' 88 | bar += ('.'*(self.width-prog_width)) 89 | bar += ']' 90 | sys.stdout.write(bar) 91 | self.total_width = len(bar) 92 | 93 | if current: 94 | time_per_unit = (now - self.start) / current 95 | else: 96 | time_per_unit = 0 97 | eta = time_per_unit*(self.target - current) 98 | info = '' 99 | if current < self.target: 100 | info += ' - ETA: %ds' % eta 101 | else: 102 | info += ' - %ds' % (now - self.start) 103 | for k in self.unique_values: 104 | info += ' - %s: %.4f' % (k, self.sum_values[k][0]/ max(1, self.sum_values[k][1])) 105 | 106 | self.total_width += len(info) 107 | if prev_total_width > self.total_width: 108 | info += ((prev_total_width-self.total_width) * " ") 109 | 110 | sys.stdout.write(info) 111 | sys.stdout.flush() 112 | 113 | if current >= self.target: 114 | sys.stdout.write("\n") 115 | 116 | if self.verbose == 2: 117 | if current >= self.target: 118 | info = '%ds' % (now - self.start) 119 | for k in self.unique_values: 120 | info += ' - %s: %.4f' % (k, self.sum_values[k][0]/ max(1, self.sum_values[k][1])) 121 | sys.stdout.write(info + "\n") 122 | 123 | 124 | def add(self, n, values=[]): 125 | self.update(self.seen_so_far+n, values) 126 | -------------------------------------------------------------------------------- /keras-master/keras/utils/io_utils.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import h5py 3 | import numpy as np 4 | from collections import defaultdict 5 | 6 | class HDF5Matrix: 7 | 8 | refs = defaultdict(int) 9 | 10 | def __init__(self, datapath, dataset, start, end, normalizer=None): 11 | if datapath not in list(self.refs.keys()): 12 | f = h5py.File(datapath) 13 | self.refs[datapath] = f 14 | else: 15 | f = self.refs[datapath] 16 | self.start = start 17 | self.end = end 18 | self.data = f[dataset] 19 | self.normalizer = normalizer 20 | 21 | def __len__(self): 22 | return self.end - self.start 23 | 24 | def __getitem__(self, key): 25 | if isinstance(key, slice): 26 | if key.stop + self.start <= self.end: 27 | idx = slice(key.start+self.start, key.stop + self.start) 28 | else: 29 | raise IndexError 30 | elif isinstance(key, int): 31 | if key + self.start < self.end: 32 | idx = key+self.start 33 | else: 34 | raise IndexError 35 | elif isinstance(key, np.ndarray): 36 | if np.max(key) + self.start < self.end: 37 | idx = (self.start + key).tolist() 38 | else: 39 | raise IndexError 40 | elif isinstance(key, list): 41 | if max(key) + self.start < self.end: 42 | idx = [x + self.start for x in key] 43 | else: 44 | raise IndexError 45 | if self.normalizer is not None: 46 | return self.normalizer(self.data[idx]) 47 | else: 48 | return self.data[idx] 49 | 50 | @property 51 | def shape(self): 52 | return tuple([self.end - self.start, self.data.shape[1]]) 53 | 54 | 55 | def save_array(array, name): 56 | import tables 57 | f = tables.open_file(name, 'w') 58 | atom = tables.Atom.from_dtype(array.dtype) 59 | ds = f.createCArray(f.root, 'data', atom, array.shape) 60 | ds[:] = array 61 | f.close() 62 | 63 | def load_array(name): 64 | import tables 65 | f = tables.open_file(name) 66 | array = f.root.data 67 | a=np.empty(shape=array.shape, dtype=array.dtype) 68 | a[:]=array[:] 69 | f.close() 70 | return a 71 | -------------------------------------------------------------------------------- /keras-master/keras/utils/np_utils.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import numpy as np 3 | import scipy as sp 4 | from six.moves import range 5 | from six.moves import zip 6 | 7 | def to_categorical(y, nb_classes=None): 8 | '''Convert class vector (integers from 0 to nb_classes) 9 | to binary class matrix, for use with categorical_crossentropy 10 | ''' 11 | y = np.asarray(y, dtype='int32') 12 | if not nb_classes: 13 | nb_classes = np.max(y)+1 14 | Y = np.zeros((len(y), nb_classes)) 15 | for i in range(len(y)): 16 | Y[i, y[i]] = 1. 17 | return Y 18 | 19 | def normalize(a, axis=-1, order=2): 20 | l2 = np.atleast_1d(np.linalg.norm(a, order, axis)) 21 | l2[l2==0] = 1 22 | return a / np.expand_dims(l2, axis) 23 | 24 | def binary_logloss(p, y): 25 | epsilon = 1e-15 26 | p = sp.maximum(epsilon, p) 27 | p = sp.minimum(1-epsilon, p) 28 | res = sum(y*sp.log(p) + sp.subtract(1,y)*sp.log(sp.subtract(1,p))) 29 | res *= -1.0/len(y) 30 | return res 31 | 32 | def multiclass_logloss(P, Y): 33 | score = 0. 34 | npreds = [P[i][Y[i]-1] for i in range(len(Y))] 35 | score = -(1./len(Y)) * np.sum(np.log(npreds)) 36 | return score 37 | 38 | def accuracy(p, y): 39 | return np.mean([a==b for a, b in zip(p, y)]) 40 | 41 | def probas_to_classes(y_pred): 42 | if len(y_pred.shape) > 1 and y_pred.shape[1] > 1: 43 | return categorical_probas_to_classes(y_pred) 44 | return np.array([1 if p > 0.5 else 0 for p in y_pred]) 45 | 46 | def categorical_probas_to_classes(p): 47 | return np.argmax(p, axis=1) 48 | -------------------------------------------------------------------------------- /keras-master/keras/utils/test_utils.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | def get_test_data(nb_train=1000, nb_test=500, input_shape=(10,), output_shape=(2,), 4 | classification=True, nb_class=2): 5 | ''' 6 | classification=True overrides output_shape 7 | (i.e. output_shape is set to (1,)) and the output 8 | consists in integers in [0, nb_class-1]. 9 | 10 | Otherwise: float output with shape output_shape. 11 | ''' 12 | nb_sample = nb_train + nb_test 13 | if classification: 14 | y = np.random.randint(0, nb_class, size=(nb_sample, 1)) 15 | X = np.zeros((nb_sample,) + input_shape) 16 | for i in range(nb_sample): 17 | X[i] = np.random.normal(loc=y[i], scale=1.0, size=input_shape) 18 | else: 19 | y_loc = np.random.random((nb_sample,)) 20 | X = np.zeros((nb_sample,) + input_shape) 21 | y = np.zeros((nb_sample,) + output_shape) 22 | for i in range(nb_sample): 23 | X[i] = np.random.normal(loc=y_loc[i], scale=1.0, size=input_shape) 24 | y[i] = np.random.normal(loc=y_loc[i], scale=1.0, size=output_shape) 25 | 26 | return (X[:nb_train], y[:nb_train]), (X[nb_train:], y[nb_train:]) 27 | -------------------------------------------------------------------------------- /keras-master/keras/utils/theano_utils.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import numpy as np 3 | import theano 4 | import theano.tensor as T 5 | 6 | def floatX(X): 7 | return np.asarray(X, dtype=theano.config.floatX) 8 | 9 | def sharedX(X, dtype=theano.config.floatX, name=None): 10 | return theano.shared(np.asarray(X, dtype=dtype), name=name) 11 | 12 | def shared_zeros(shape, dtype=theano.config.floatX, name=None): 13 | return sharedX(np.zeros(shape), dtype=dtype, name=name) 14 | 15 | def shared_scalar(val=0., dtype=theano.config.floatX, name=None): 16 | return theano.shared(np.cast[dtype](val)) 17 | 18 | def shared_ones(shape, dtype=theano.config.floatX, name=None): 19 | return sharedX(np.ones(shape), dtype=dtype, name=name) 20 | 21 | def alloc_zeros_matrix(*dims): 22 | return T.alloc(np.cast[theano.config.floatX](0.), *dims) 23 | 24 | -------------------------------------------------------------------------------- /keras-master/keras/wrappers/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sonyisme/keras-recommendation/e84fab6706faf8405f4bfc75cf50ecff6433eb77/keras-master/keras/wrappers/__init__.py -------------------------------------------------------------------------------- /keras-master/setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | description-file = README.md -------------------------------------------------------------------------------- /keras-master/setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | from setuptools import find_packages 3 | 4 | setup(name = 'Keras', 5 | version = '0.1.1', 6 | description = 'Theano-based Deep Learning library', 7 | author = 'Francois Chollet', 8 | author_email = 'francois.chollet@gmail.com', 9 | url = 'https://github.com/fchollet/keras', 10 | download_url = 'https://github.com/fchollet/keras/tarball/0.1.1', 11 | license = 'MIT', 12 | install_requires = ['theano'], 13 | packages = find_packages(), 14 | ) -------------------------------------------------------------------------------- /keras-master/tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sonyisme/keras-recommendation/e84fab6706faf8405f4bfc75cf50ecff6433eb77/keras-master/tests/__init__.py -------------------------------------------------------------------------------- /keras-master/tests/auto/keras/test_activations.py: -------------------------------------------------------------------------------- 1 | import math 2 | 3 | import keras 4 | import theano 5 | import theano.tensor as T 6 | 7 | import numpy 8 | 9 | def list_assert_equal(a, b, round_to=7): 10 | ''' 11 | This will do a pairwise, rounded equality test across two lists of 12 | numbers. 13 | ''' 14 | pairs = zip(a, b) 15 | for i, j in pairs: 16 | assert round(i, round_to) == round(j, round_to) 17 | 18 | def get_standard_values(): 19 | ''' 20 | These are just a set of floats used for testing the activation 21 | functions, and are useful in multiple tests. 22 | ''' 23 | 24 | return [0,0.1,0.5,0.9,1.0] 25 | 26 | def test_softmax(): 27 | 28 | from keras.activations import softmax as s 29 | 30 | # Test using a reference implementation of softmax 31 | def softmax(values): 32 | m = max(values) 33 | values = numpy.array(values) 34 | e = numpy.exp(values - m) 35 | dist = list(e / numpy.sum(e)) 36 | 37 | return dist 38 | 39 | x = T.vector() 40 | exp = s(x) 41 | f = theano.function([x], exp) 42 | test_values=get_standard_values() 43 | 44 | result = f(test_values) 45 | expected = softmax(test_values) 46 | 47 | print(str(result)) 48 | print(str(expected)) 49 | 50 | list_assert_equal(result, expected) 51 | 52 | def test_relu(): 53 | ''' 54 | Relu implementation doesn't depend on the value being 55 | a theano variable. Testing ints, floats and theano tensors. 56 | ''' 57 | 58 | from keras.activations import relu as r 59 | 60 | assert r(5) == 5 61 | assert r(-5) == 0 62 | assert r(-0.1) == 0 63 | assert r(0.1) == 0.1 64 | 65 | x = T.vector() 66 | exp = r(x) 67 | f = theano.function([x], exp) 68 | 69 | test_values = get_standard_values() 70 | result = f(test_values) 71 | 72 | list_assert_equal(result, test_values) # because no negatives in test values 73 | 74 | 75 | def test_tanh(): 76 | 77 | from keras.activations import tanh as t 78 | test_values = get_standard_values() 79 | 80 | x = T.vector() 81 | exp = t(x) 82 | f = theano.function([x], exp) 83 | 84 | result = f(test_values) 85 | expected = [math.tanh(v) for v in test_values] 86 | 87 | print(result) 88 | print(expected) 89 | 90 | list_assert_equal(result, expected) 91 | 92 | 93 | def test_linear(): 94 | ''' 95 | This function does no input validation, it just returns the thing 96 | that was passed in. 97 | ''' 98 | 99 | from keras.activations import linear as l 100 | 101 | xs = [1, 5, True, None, 'foo'] 102 | 103 | for x in xs: 104 | assert x == l(x) 105 | -------------------------------------------------------------------------------- /keras-master/tests/auto/keras/test_constraints.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | import numpy as np 3 | from numpy.testing import assert_allclose 4 | from theano import tensor as T 5 | 6 | 7 | class TestConstraints(unittest.TestCase): 8 | def setUp(self): 9 | self.some_values = [0.1, 0.5, 3, 8, 1e-7] 10 | np.random.seed(3537) 11 | self.example_array = np.random.random((100, 100)) * 100. - 50. 12 | self.example_array[0, 0] = 0. # 0 could possibly cause trouble 13 | 14 | def test_maxnorm(self): 15 | from keras.constraints import maxnorm 16 | 17 | for m in self.some_values: 18 | norm_instance = maxnorm(m) 19 | normed = norm_instance(self.example_array) 20 | assert (np.all(normed.eval() < m)) 21 | 22 | # a more explicit example 23 | norm_instance = maxnorm(2.0) 24 | x = np.array([[0, 0, 0], [1.0, 0, 0], [3, 0, 0], [3, 3, 3]]).T 25 | x_normed_target = np.array([[0, 0, 0], [1.0, 0, 0], [2.0, 0, 0], [2./np.sqrt(3), 2./np.sqrt(3), 2./np.sqrt(3)]]).T 26 | x_normed_actual = norm_instance(x).eval() 27 | assert_allclose(x_normed_actual, x_normed_target) 28 | 29 | def test_nonneg(self): 30 | from keras.constraints import nonneg 31 | 32 | nonneg_instance = nonneg() 33 | 34 | normed = nonneg_instance(self.example_array) 35 | assert (np.all(np.min(normed.eval(), axis=1) == 0.)) 36 | 37 | def test_identity(self): 38 | from keras.constraints import identity 39 | 40 | identity_instance = identity() 41 | 42 | normed = identity_instance(self.example_array) 43 | assert (np.all(normed == self.example_array)) 44 | 45 | def test_identity_oddballs(self): 46 | """ 47 | test the identity constraint on some more exotic input. 48 | this does not need to pass for the desired real life behaviour, 49 | but it should in the current implementation. 50 | """ 51 | from keras.constraints import identity 52 | identity_instance = identity() 53 | 54 | oddball_examples = ["Hello", [1], -1, None] 55 | assert(oddball_examples == identity_instance(oddball_examples)) 56 | 57 | def test_unitnorm(self): 58 | from keras.constraints import unitnorm 59 | unitnorm_instance = unitnorm() 60 | 61 | normalized = unitnorm_instance(self.example_array) 62 | 63 | norm_of_normalized = np.sqrt(np.sum(normalized.eval()**2, axis=1)) 64 | difference = norm_of_normalized - 1. #in the unit norm constraint, it should be equal to 1. 65 | largest_difference = np.max(np.abs(difference)) 66 | self.assertAlmostEqual(largest_difference, 0.) 67 | 68 | if __name__ == '__main__': 69 | unittest.main() 70 | -------------------------------------------------------------------------------- /keras-master/tests/auto/test_embeddings.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | import numpy as np 3 | from keras.models import Sequential 4 | from keras.layers.core import Merge, Dense, Activation, Flatten 5 | from keras.layers.embeddings import Embedding 6 | from theano import function 7 | from keras.constraints import unitnorm 8 | 9 | class TestConcatenation(unittest.TestCase): 10 | 11 | def setUp(self): 12 | self.X1 = np.array([[1], [2]], dtype='int32') 13 | self.W1 = np.array([[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]], dtype='float32') 14 | 15 | def test_unitnorm_constraint(self): 16 | lookup = Sequential() 17 | lookup.add(Embedding(3, 2, weights=[self.W1], W_constraint=unitnorm())) 18 | lookup.add(Flatten()) 19 | lookup.add(Dense(2, 1)) 20 | lookup.add(Activation('sigmoid')) 21 | lookup.compile(loss='binary_crossentropy', optimizer='sgd', class_mode='binary') 22 | lookup.train(self.X1, np.array([[1], [0]], dtype='int32')) 23 | norm = np.linalg.norm(lookup.params[0].get_value(), axis=1) 24 | self.assertTrue(np.allclose(norm, np.ones_like(norm).astype('float32'))) 25 | 26 | if __name__ == '__main__': 27 | unittest.main() -------------------------------------------------------------------------------- /keras-master/tests/auto/test_loss_weighting.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | from keras.datasets import mnist 4 | from keras.models import Sequential 5 | from keras.layers.core import Dense, Activation 6 | from keras.utils import np_utils 7 | import numpy as np 8 | import unittest 9 | 10 | nb_classes = 10 11 | batch_size = 128 12 | nb_epoch = 5 13 | weighted_class = 9 14 | standard_weight = 1 15 | high_weight = 5 16 | max_train_samples = 5000 17 | max_test_samples = 1000 18 | 19 | np.random.seed(1337) # for reproducibility 20 | 21 | # the data, shuffled and split between tran and test sets 22 | (X_train, y_train), (X_test, y_test) = mnist.load_data() 23 | X_train = X_train.reshape(60000, 784)[:max_train_samples] 24 | X_test = X_test.reshape(10000, 784)[:max_test_samples] 25 | X_train = X_train.astype("float32") / 255 26 | X_test = X_test.astype("float32") / 255 27 | 28 | # convert class vectors to binary class matrices 29 | y_train = y_train[:max_train_samples] 30 | y_test = y_test[:max_test_samples] 31 | Y_train = np_utils.to_categorical(y_train, nb_classes) 32 | Y_test = np_utils.to_categorical(y_test, nb_classes) 33 | test_ids = np.where(y_test == np.array(weighted_class))[0] 34 | 35 | def create_model(): 36 | model = Sequential() 37 | model.add(Dense(784, 50)) 38 | model.add(Activation('relu')) 39 | model.add(Dense(50, 10)) 40 | model.add(Activation('softmax')) 41 | return model 42 | 43 | def test_weights(model, class_weight=None, sample_weight=None): 44 | model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch, verbose=0, \ 45 | class_weight=class_weight, sample_weight=sample_weight) 46 | score = model.evaluate(X_test[test_ids, :], Y_test[test_ids, :], verbose=0) 47 | return score 48 | 49 | class TestConcatenation(unittest.TestCase): 50 | 51 | def test_loss_weighting(self): 52 | class_weight = dict([(i, standard_weight) for i in range(nb_classes)]) 53 | class_weight[weighted_class] = high_weight 54 | 55 | sample_weight = np.ones((y_train.shape[0])) * standard_weight 56 | sample_weight[y_train == weighted_class] = high_weight 57 | 58 | for loss in ['mae', 'mse', 'categorical_crossentropy']: 59 | print('loss:', loss) 60 | # no weights: reference point 61 | model = create_model() 62 | model.compile(loss='categorical_crossentropy', optimizer='rmsprop') 63 | standard_score = test_weights(model) 64 | # test class_weight 65 | model = create_model() 66 | model.compile(loss=loss, optimizer='rmsprop') 67 | score = test_weights(model, class_weight=class_weight) 68 | print('score:', score, ' vs.', standard_score) 69 | self.assertTrue(score < standard_score) 70 | # test sample_weight 71 | model = create_model() 72 | model.compile(loss=loss, optimizer='rmsprop') 73 | score = test_weights(model, sample_weight=sample_weight) 74 | print('score:', score, ' vs.', standard_score) 75 | self.assertTrue(score < standard_score) 76 | 77 | if __name__ == '__main__': 78 | print('Test class_weight and sample_weight') 79 | unittest.main() -------------------------------------------------------------------------------- /keras-master/tests/auto/test_regularizers.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | import numpy as np 3 | np.random.seed(1337) # for reproducibility 4 | 5 | from keras.models import Sequential 6 | from keras.layers.core import Merge, Dense, Activation, Flatten, ActivityRegularization 7 | from keras.layers.embeddings import Embedding 8 | from keras.datasets import mnist 9 | from keras.utils import np_utils 10 | from keras import regularizers 11 | 12 | nb_classes = 10 13 | batch_size = 128 14 | nb_epoch = 5 15 | weighted_class = 9 16 | standard_weight = 1 17 | high_weight = 5 18 | max_train_samples = 5000 19 | max_test_samples = 1000 20 | 21 | # the data, shuffled and split between tran and test sets 22 | (X_train, y_train), (X_test, y_test) = mnist.load_data() 23 | X_train = X_train.reshape(60000, 784)[:max_train_samples] 24 | X_test = X_test.reshape(10000, 784)[:max_test_samples] 25 | X_train = X_train.astype("float32") / 255 26 | X_test = X_test.astype("float32") / 255 27 | 28 | # convert class vectors to binary class matrices 29 | y_train = y_train[:max_train_samples] 30 | y_test = y_test[:max_test_samples] 31 | Y_train = np_utils.to_categorical(y_train, nb_classes) 32 | Y_test = np_utils.to_categorical(y_test, nb_classes) 33 | test_ids = np.where(y_test == np.array(weighted_class))[0] 34 | 35 | def create_model(weight_reg=None, activity_reg=None): 36 | model = Sequential() 37 | model.add(Dense(784, 50)) 38 | model.add(Activation('relu')) 39 | model.add(Dense(50, 10, W_regularizer=weight_reg, activity_regularizer=activity_reg)) 40 | model.add(Activation('softmax')) 41 | return model 42 | 43 | 44 | class TestRegularizers(unittest.TestCase): 45 | def test_W_reg(self): 46 | for reg in [regularizers.identity(), regularizers.l1(), regularizers.l2(), regularizers.l1l2()]: 47 | model = create_model(weight_reg=reg) 48 | model.compile(loss='categorical_crossentropy', optimizer='rmsprop') 49 | model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch, verbose=0) 50 | model.evaluate(X_test[test_ids, :], Y_test[test_ids, :], verbose=0) 51 | 52 | def test_A_reg(self): 53 | for reg in [regularizers.activity_l1(), regularizers.activity_l2()]: 54 | model = create_model(activity_reg=reg) 55 | model.compile(loss='categorical_crossentropy', optimizer='rmsprop') 56 | model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch, verbose=0) 57 | model.evaluate(X_test[test_ids, :], Y_test[test_ids, :], verbose=0) 58 | 59 | if __name__ == '__main__': 60 | print('Test weight and activity regularizers') 61 | unittest.main() 62 | -------------------------------------------------------------------------------- /keras-master/tests/manual/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sonyisme/keras-recommendation/e84fab6706faf8405f4bfc75cf50ecff6433eb77/keras-master/tests/manual/__init__.py -------------------------------------------------------------------------------- /keras-master/tests/manual/check_constraints.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | import keras 4 | from keras.datasets import mnist 5 | import keras.models 6 | from keras.models import Sequential 7 | from keras.layers.core import Dense, Dropout, Activation 8 | from keras.regularizers import l2, l1 9 | from keras.constraints import maxnorm, nonneg 10 | from keras.optimizers import SGD, Adam, RMSprop 11 | from keras.utils import np_utils, generic_utils 12 | import theano 13 | import theano.tensor as T 14 | import numpy as np 15 | import scipy 16 | 17 | batch_size = 100 18 | nb_classes = 10 19 | nb_epoch = 10 20 | 21 | # the data, shuffled and split between tran and test sets 22 | (X_train, y_train), (X_test, y_test) = mnist.load_data() 23 | X_train=X_train.reshape(60000,784) 24 | X_test=X_test.reshape(10000,784) 25 | X_train = X_train.astype("float32") 26 | X_test = X_test.astype("float32") 27 | X_train /= 255 28 | X_test /= 255 29 | 30 | # convert class vectors to binary class matrices 31 | Y_train = np_utils.to_categorical(y_train, nb_classes) 32 | Y_test = np_utils.to_categorical(y_test, nb_classes) 33 | 34 | model = Sequential() 35 | model.add(Dense(784, 20, W_constraint=maxnorm(1))) 36 | model.add(Activation('relu')) 37 | model.add(Dropout(0.1)) 38 | model.add(Dense(20, 20, W_constraint=nonneg())) 39 | model.add(Activation('relu')) 40 | model.add(Dropout(0.1)) 41 | model.add(Dense(20, 10, W_constraint=maxnorm(1))) 42 | model.add(Activation('softmax')) 43 | 44 | 45 | rms = RMSprop() 46 | model.compile(loss='categorical_crossentropy', optimizer=rms) 47 | 48 | model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch, show_accuracy=True, verbose=0) 49 | 50 | a=model.params[0].eval() 51 | if np.isclose(np.max(np.sqrt(np.sum(a**2, axis=0))),1): 52 | print('Maxnorm test passed') 53 | else: 54 | raise ValueError('Maxnorm test failed!') 55 | 56 | b=model.params[2].eval() 57 | if np.min(b)==0 and np.min(a)!=0: 58 | print('Nonneg test passed') 59 | else: 60 | raise ValueError('Nonneg test failed!') 61 | 62 | 63 | model = Sequential() 64 | model.add(Dense(784, 20)) 65 | model.add(Activation('relu')) 66 | model.add(Dense(20, 20, W_regularizer=l1(.01))) 67 | model.add(Activation('relu')) 68 | model.add(Dense(20, 10)) 69 | model.add(Activation('softmax')) 70 | 71 | 72 | rms = RMSprop() 73 | model.compile(loss='categorical_crossentropy', optimizer=rms) 74 | 75 | model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=20, show_accuracy=True, verbose=0) 76 | 77 | a=model.params[2].eval().reshape(400) 78 | (D, p1) = scipy.stats.kurtosistest(a) 79 | 80 | model = Sequential() 81 | model.add(Dense(784, 20)) 82 | model.add(Activation('relu')) 83 | model.add(Dense(20, 20, W_regularizer=l2(.01))) 84 | model.add(Activation('relu')) 85 | model.add(Dense(20, 10)) 86 | model.add(Activation('softmax')) 87 | 88 | 89 | rms = RMSprop() 90 | model.compile(loss='categorical_crossentropy', optimizer=rms) 91 | 92 | model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=20, show_accuracy=True, verbose=0) 93 | 94 | a=model.params[2].eval().reshape(400) 95 | (D, p2) = scipy.stats.kurtosistest(a) 96 | 97 | if p1<.01 and p2>.01: 98 | print('L1 and L2 regularization tests passed') 99 | else: 100 | raise ValueError('L1 and L2 regularization tests failed!') -------------------------------------------------------------------------------- /keras-master/tests/manual/check_dot_utils.py: -------------------------------------------------------------------------------- 1 | from keras.utils.dot_utils import Grapher 2 | 3 | from keras.models import Sequential 4 | from keras.layers.core import Dense, Activation, Merge, Flatten 5 | from keras.layers.embeddings import Embedding 6 | from keras.layers.recurrent import GRU 7 | 8 | ent_lookup = Sequential() 9 | ent_lookup.add(Embedding(10, 2)) 10 | ent_lookup.add(Flatten()) 11 | 12 | rel_lookup = Sequential() 13 | rel_lookup.add(Embedding(20, 2)) 14 | rel_lookup.add(Flatten()) 15 | 16 | word_sequence = Sequential() 17 | word_sequence.add(Embedding(10, 5)) 18 | word_sequence.add(GRU(5, 2)) 19 | 20 | model = Sequential() 21 | model.add(Merge([word_sequence, ent_lookup, rel_lookup], mode='concat')) 22 | model.add(Activation('relu')) 23 | model.add(Dense(6, 2)) 24 | model.add(Activation('softmax')) 25 | 26 | g = Grapher() 27 | g.plot(model, 'mymodel.png') -------------------------------------------------------------------------------- /keras-master/tests/manual/check_masked_recurrent.py: -------------------------------------------------------------------------------- 1 | # Dummy test data as input to RNN. This input is 3 timesteps long where the third timestep always matches the 2 | # first. Without masking it should be able to learn it, with masking it should fail. 3 | 4 | import numpy as np 5 | from keras.utils.theano_utils import sharedX 6 | from keras.models import Sequential 7 | from keras.layers.core import Dense, Activation, Merge, Dropout, TimeDistributedDense 8 | from keras.layers.embeddings import Embedding 9 | from keras.layers.recurrent import SimpleRNN, SimpleDeepRNN, LSTM, GRU 10 | import theano 11 | 12 | theano.config.exception_verbosity = 'high' 13 | 14 | # (nb_samples, timesteps, dimensions) 15 | X = np.random.random_integers(1, 4, size=(500000, 15)) 16 | 17 | print("About to compile the first model") 18 | model = Sequential() 19 | model.add(Embedding(5, 4, mask_zero=True)) 20 | model.add(TimeDistributedDense(4, 4)) # obviously this is redundant. Just testing. 21 | model.add(SimpleRNN(4, 4, activation='relu', return_sequences=True)) 22 | model.add(Dropout(0.5)) 23 | model.add(SimpleDeepRNN(4, 4, depth=2, activation='relu')) 24 | model.add(Dropout(0.5)) 25 | model.add(Dense(4, 4, activation='softmax')) 26 | model.compile(loss='categorical_crossentropy', 27 | optimizer='rmsprop', theano_mode=theano.compile.mode.FAST_RUN) 28 | print("Compiled model") 29 | 30 | W = model.get_weights() # We'll save these so we can reset it later 31 | 32 | X[:, : 10] = 0 33 | Xmask0 = X.copy() 34 | Xmask0[:, 10] = 0 35 | 36 | Xmask12 = X.copy() 37 | Xmask12[:, 11] = 0 38 | Xmask12[:, 12] = 0 39 | 40 | X0_onehot = np.zeros((X.shape[0], 4)) 41 | X1_onehot = np.zeros((X.shape[0], 4)) 42 | for i, row in enumerate(X): 43 | X0_onehot[i, row[10] - 1] = 1 44 | X1_onehot[i, row[11] - 1] = 1 45 | 46 | # Uniform score: 4 options = ln(4) nats (2 bits) 47 | # we should not do better than this when we mask out the part of the input 48 | # that gives us the correct answer 49 | uniform_score = np.log(4) 50 | batch_size=512 51 | 52 | # Train it to guess 0th dim 53 | model.fit(X, X0_onehot, nb_epoch=1, batch_size=batch_size) 54 | score = model.evaluate(X, X0_onehot, batch_size=batch_size) 55 | if score > uniform_score * 0.9: 56 | raise Exception('Failed to learn to copy timestep 0, score %f' % score) 57 | 58 | 59 | model.set_weights(W) 60 | 61 | # Train without showing it the 0th dim to learn 1st dim 62 | model.fit(X[: , 1:], X1_onehot, nb_epoch=1, batch_size=batch_size) 63 | score = model.evaluate(X[:, 1:], X1_onehot, batch_size=batch_size) 64 | if score > uniform_score * 0.9: 65 | raise Exception('Failed to learn to copy timestep 1, score %f' % score) 66 | 67 | model.set_weights(W) 68 | 69 | # Train to guess 0th dim when 0th dim has been masked (should fail) 70 | model.fit(Xmask0, X0_onehot, nb_epoch=1, batch_size=batch_size) 71 | score = model.evaluate(Xmask0, X0_onehot, batch_size=batch_size) 72 | if score < uniform_score * 0.9: 73 | raise Exception('Somehow learned to copy timestep 0 despite mask, score %f' % score) 74 | 75 | model.set_weights(W) 76 | 77 | # Train to guess 1st dim when 0th dim has been masked (should succeed) 78 | model.fit(Xmask0, X1_onehot, nb_epoch=1, batch_size=batch_size) 79 | score = model.evaluate(Xmask0, X1_onehot, batch_size=batch_size) 80 | if score > uniform_score * 0.9: 81 | raise Exception('Failed to learn to copy timestep 1 in masked model, score %f' % score) 82 | 83 | model.set_weights(W) 84 | 85 | # Finally, make sure the mask is actually blocking input, mask out timesteps 1 and 2, and see if 86 | # it can learn timestep 0 (should fail) 87 | model.fit(Xmask12, X0_onehot, nb_epoch=1, batch_size=batch_size) 88 | 89 | score = model.evaluate(Xmask12, X0_onehot, batch_size=batch_size) 90 | if score < uniform_score * 0.9: 91 | raise Exception('Somehow learned to copy timestep 0 despite masking 1, score %f' % score) 92 | 93 | # Another testing approach, just initialize models and make sure that prepending zeros doesn't affect 94 | # their output 95 | print("About to compile the second model") 96 | model2 = Sequential() 97 | model2.add(Embedding(5, 4, mask_zero=True)) 98 | model2.add(TimeDistributedDense(4, 4)) 99 | model2.add(Activation('time_distributed_softmax')) 100 | model2.add(LSTM(4, 4, return_sequences=True)) 101 | model2.add(Activation('tanh')) 102 | model2.add(GRU(4, 4, activation='softmax', return_sequences=True)) 103 | model2.add(SimpleDeepRNN(4, 4, depth=2, activation='relu', return_sequences=True)) 104 | model2.add(SimpleRNN(4, 4, activation='relu', return_sequences=True)) 105 | model2.compile(loss='categorical_crossentropy', 106 | optimizer='rmsprop', theano_mode=theano.compile.mode.FAST_RUN) 107 | print("Compiled model2") 108 | 109 | X2 = np.random.random_integers(1, 4, size=(2, 5)) 110 | y2 = np.random.random((X2.shape[0], X2.shape[1], 4)) 111 | 112 | ref = model2.predict(X2) 113 | ref_eval = model2.evaluate(X2, y2) 114 | mask = np.ones((y2.shape[0], y2.shape[1], 1)) 115 | 116 | for pre_zeros in range(1, 10): 117 | padded_X2 = np.concatenate((np.zeros((X2.shape[0], pre_zeros)), X2), axis=1) 118 | padded_mask = np.concatenate((np.zeros((mask.shape[0], pre_zeros, mask.shape[2])), mask), axis=1) 119 | padded_y2 = np.concatenate((np.zeros((y2.shape[0], pre_zeros, y2.shape[2])), y2), axis=1) 120 | 121 | pred = model2.predict(padded_X2) 122 | if not np.allclose(ref[:, -1, :], pred[:, -1, :]): 123 | raise Exception("Different result after left-padding %d zeros. Ref: %s, Pred: %s" % (pre_zeros, ref, pred)) 124 | 125 | pad_eval = model2.evaluate(padded_X2, padded_y2, weights=padded_mask) 126 | if not np.allclose([pad_eval], [ref_eval]): 127 | raise Exception("Got dissimilar categorical_crossentropy after left-padding %d zeros. Ref: %f, Pred %f" %\ 128 | (pref_eval, pred_val)) 129 | 130 | 131 | -------------------------------------------------------------------------------- /keras-master/tests/manual/check_save_weights.py: -------------------------------------------------------------------------------- 1 | from keras.models import Sequential 2 | from keras.layers.core import Dense, Dropout, Activation 3 | from keras.optimizers import SGD 4 | 5 | import sys 6 | sys.setrecursionlimit(10000) # to be able to pickle Theano compiled functions 7 | 8 | import pickle, numpy 9 | 10 | def create_model(): 11 | model = Sequential() 12 | model.add(Dense(256, 2048, init='uniform', activation='relu')) 13 | model.add(Dropout(0.5)) 14 | model.add(Dense(2048, 2048, init='uniform', activation='relu')) 15 | model.add(Dropout(0.5)) 16 | model.add(Dense(2048, 2048, init='uniform', activation='relu')) 17 | model.add(Dropout(0.5)) 18 | model.add(Dense(2048, 2048, init='uniform', activation='relu')) 19 | model.add(Dropout(0.5)) 20 | model.add(Dense(2048, 256, init='uniform', activation='linear')) 21 | return model 22 | 23 | model = create_model() 24 | sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True) 25 | model.compile(loss='mse', optimizer=sgd) 26 | 27 | pickle.dump(model, open('/tmp/model.pkl', 'wb')) 28 | model.save_weights('/tmp/model_weights.hdf5') 29 | 30 | model_loaded = create_model() 31 | model_loaded.load_weights('/tmp/model_weights.hdf5') 32 | 33 | for k in range(len(model.layers)): 34 | weights_orig = model.layers[k].get_weights() 35 | weights_loaded = model_loaded.layers[k].get_weights() 36 | for x, y in zip(weights_orig, weights_loaded): 37 | if numpy.any(x != y): 38 | raise ValueError('Loaded weights are different from pickled weights!') 39 | 40 | 41 | -------------------------------------------------------------------------------- /keras-master/tests/manual/check_wrappers.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | from __future__ import print_function 3 | from keras.datasets import mnist 4 | from keras.models import Sequential 5 | from keras.layers.core import Dense, Activation 6 | from keras.utils import np_utils 7 | from keras.wrappers.scikit_learn import KerasClassifier 8 | import numpy as np 9 | 10 | nb_classes = 10 11 | batch_size = 128 12 | nb_epoch = 1 13 | 14 | max_train_samples = 5000 15 | max_test_samples = 1000 16 | 17 | np.random.seed(1337) # for reproducibility 18 | 19 | # the data, shuffled and split between tran and test sets 20 | (X_train, y_train), (X_test, y_test) = mnist.load_data() 21 | 22 | X_train = X_train.reshape(60000,784)[:max_train_samples] 23 | X_test = X_test.reshape(10000,784)[:max_test_samples] 24 | X_train = X_train.astype('float32') 25 | X_test = X_test.astype('float32') 26 | X_train /= 255 27 | X_test /= 255 28 | 29 | Y_train = np_utils.to_categorical(y_train, nb_classes)[:max_train_samples] 30 | Y_test = np_utils.to_categorical(y_test, nb_classes)[:max_test_samples] 31 | 32 | ############################# 33 | # scikit-learn wrapper test # 34 | ############################# 35 | print('Beginning scikit-learn wrapper test') 36 | 37 | print('Defining model') 38 | model = Sequential() 39 | model.add(Dense(784, 50)) 40 | model.add(Activation('relu')) 41 | model.add(Dense(50, 10)) 42 | model.add(Activation('softmax')) 43 | 44 | print('Creating wrapper') 45 | classifier = KerasClassifier(model) 46 | 47 | print('Fitting model') 48 | classifier.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch) 49 | 50 | print('Testing score function') 51 | score = classifier.score(X_train, Y_train) 52 | print('Score: ', score) 53 | 54 | print('Testing predict function') 55 | preds = classifier.predict(X_test) 56 | print('Preds.shape: ', preds.shape) 57 | 58 | print('Testing predict proba function') 59 | proba = classifier.predict_proba(X_test) 60 | print('Proba.shape: ', proba.shape) 61 | 62 | print('Testing get params') 63 | print(classifier.get_params()) 64 | 65 | print('Testing set params') 66 | classifier.set_params(optimizer='sgd', loss='mse') 67 | print(classifier.get_params()) 68 | 69 | print('Testing attributes') 70 | print('Classes') 71 | print(classifier.classes_) 72 | print('Config') 73 | print(classifier.config_) 74 | print('Weights') 75 | print(classifier.weights_) 76 | 77 | print('Test script complete.') 78 | --------------------------------------------------------------------------------