├── LICENSE
├── README.md
└── data
└── lstm_ptb.csv
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2018 MITAutoML
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Neural Network Performance Prediction Dataset
2 |
3 |
4 | This repository contains datasets for neural network performance prediction. Each CSV file in the "data" directory contains architecture details, training hyperparamters, and per-epoch validation accuracy for a variety of neural networks trained on image classification and language modeling datasets. Please cite the following paper if you use this dataset:
5 |
6 | **[Accelerating Neural Architecture Search using Performance Prediction](https://openreview.net/pdf?id=BJypUGZ0Z)**
7 | *Bowen Baker\*, Otkrist Gupta\*, Ramesh Raskar, Nikhil Naik*
8 | *International Conference on Learning Representations (ICLR) Workshop 2018*
9 | (*\* denotes equal contribution*)
10 |
11 | In this paper, we showed that simple regression models can predict the final performance of partially trained model configurations using features based on network architectures, hyperparameters, and partially-trained time-series validation performance data. Using these prediction models, we implemented an early stopping method for hyperparameter optimization and neural architecture search, which obtained a speedup of a factor up to 6x. Our early stopping method can be seamlessly incorporated into both reinforcement learning-based architecture search algorithms and bandit-based hyperparameter search.
12 |
13 | We hope that this dataset encourages further research in neural network performance prediction.
14 |
15 | ## Dataset Description
16 |
17 | ### 1. MetaQNN CNNs - CIFAR10 (metaqnn\_cifar10.csv) and MetaQNN CNN - SVHN (metaqnn\_svhn.csv)
18 | We sampled CNN architectures from the search space detailed in the [MetaQNN](https://openreview.net/pdf?id=S1c2cvqee) paper by Baker et al. (2017), which allows for varying the numbers and orderings of convolution, pooling, and fully connected layers. The model depths were between 1 and 12 layers for the SVHN experiment and between 1 and 18 layers for the CIFAR-10 experiment. Each model was trained for a total of 20 epochs with the Adam optimizer with β1 = 0.9, β2 = 0.999, and ε= 10-8. The batch size was set to 128. We reduced the learning rate by a factor of 0.2 every 5 epochs. The CSV files contain the following fields:
19 |
20 | **net**: This is a string description of the trained neural network. For CNNs, *C(n, f, l)* denotes a Convolutional layer with *n* filters, receptive field size *f*, and stride *l*. *P(f, l)* denotes a Pooling layer with receptive field size *f* and stride *l*. *SM(n)* denotes a *softmax* layer with *n* outputs. *GAP(n)* denotes a Global Average Pooling layer with *n* outputs.
21 |
22 | **num\_params**: The total number of parameters in the neural network
23 |
24 | **depth**: The total number of layers in the neural network, excluding GAP and SM layers
25 |
26 | **lr**: Initial learning rate
27 |
28 | **acc\_n**: Validation set accuracy at epoch *n*
29 |
30 | ### 2. Deep ResNets - TinyImageNet: (deep\_resnets\_tinyimagenet.csv)
31 | We sampled ResNet architectures and trained them on the [TinyImageNet](https://tiny-imagenet.herokuapp.com/) dataset (containing 200 classes with 500 training images of 32×32 pixels) for 140 epochs. We varied depths, filter sizes and number of convolutional filter block outputs. Each network was trained for 140 epochs, using Nesterov optimizer. For all models, learning rate reduction and momentum were set to 0.1 and 0.9 respectively. The CSV files contain the following fields:
32 |
33 | **num\_blocks**: Number of ResNet blocks in the model. Each ResNet block is composed of three convolutional layers followed by batch normalization and summation layers. We vary the number of blocks from 2 to 18, giving us networks with depths between 14 and 110 (since *depth = 6\*num\_blocks + 2*)
34 |
35 | **num\_filters**: Number of filters in each convolutional layer. Sampled from {2, 3, ..., 22}
36 |
37 | **filter_size**: Filter size for each convolutional layer. Sampled from {3, 5, 7}
38 |
39 | **lr**: Initial learning rate
40 |
41 | **ss**: Learning rate reduction step size
42 |
43 | **acc\_n**: Validation set accuracy at epoch *n*. Note that *acc_0* refers to the accuracy of a randomly initialized model.
44 |
45 |
46 |
47 | ### 3. Deep ResNets - CIFAR10: (deep\_resnets\_cifar10.csv)
48 | We sampled ResNet architectures from a search space similar to [Zoph & Le (2017)](https://openreview.net/pdf?id=r1Ue8Hcxg). Each architecture consists of 39 layers: 12 *conv*, a 2x2 *max pool*, 9 *conv*, a 2x2 *max pool*, 15 *conv*, and *softmax*. Each *conv* layer is followed by batch normalization and *ReLU*. A ResNet block contains 3 *conv* layers, which share the same kernel width, kernel height, and number of learnable kernels. We varied these parameters across architectures. Each network was trained on the CIFAR-10 dataset for 50 epochs using the RMSProp optimizer, with
49 | variable initial learning rate, weight decay of 10−4, and a learning rate reduction to 10−5 at epoch 30. The CSV files contain the following fields:
50 |
51 | **num\_params**: The total number of parameters in the neural network.
52 |
53 | **lr**: Initial learning rate
54 |
55 | **acc\_n**: Validation set accuracy at epoch *n*. Note that *acc_0* refers to the accuracy of a randomly initialized model.
56 |
57 |
58 |
59 | ### 4. CudaConvnet - CIFAR10 (cudaconvnet\_cifar10.csv) and CudaConvet - SVHN (metaqnn\_svhn.csv)
60 | We trained the [Cuda-Convnet](https://code.google.com/p/cuda-convnet/) architecture with varying values of initial learning rate, learning rate reduction step size, weight decay for convolutional and fully connected layers, and scale and power of local response normalization layers. We trained models with CIFAR-10 for 60 epochs and with SVHN for 12 epochs. The CSV files contain the following fields:
61 |
62 | **lr**: Initial learning rate
63 |
64 | **ss**: Learning rate reduction step size
65 |
66 | **cn_l2**: L_2 penalty of Conv"n" layer
67 |
68 | **f4_l2**: L_2 penalty for FC4 layer
69 |
70 | **weight_decay**: Weight decay parameter
71 |
72 | **lrn_scale**: Response normalization scale
73 |
74 | **lrn_pow**: Response normalization power
75 |
76 | **acc\_n**: Validation set accuracy at epoch *n*.
77 |
78 |
79 |
80 | ### 5. LSTM - Penn Treebank (lstm\_ptb.csv)
81 | We sampled LSTM models and trained them on the Penn Treebank dataset for 60 epochs. Both the number of hidden layer inputs and number of LSTM cells were varied between 10 to 1400 in steps of 20. Each network was trained with SGD for 60 epochs with batch size of 50 and dropout ratio of 0.5. We used a dictionary size of 400 words to generate embeddings when vectorizing the data. The CSV files contain the following fields:
82 |
83 | **n_inputs**: number of LSTM cells
84 |
85 | **n_layers**: depth of neural network
86 |
87 | **base_lr**: Initial learning rate
88 |
89 | **ss**: Learning rate reduction step size
90 |
91 | **acc\_n**: Validation set perplexity at epoch *n*. Note that *acc_0* refers to the perplexity of a randomly initialized model.
92 |
93 |
94 |
95 | ### 6. AlexNet - ImageNet (alexnet\_imagenet.csv)
96 |
97 | We trained the AlexNet model on the original ImageNet (ILSVRC12) dataset. To
98 | compensate for our limited computation resources, we randomly sampled 10% of dataset, trained each
99 | configuration for 500 epochs, and only varied initial learning rate and learning rate reduction step size across models.
100 |
101 | **lr**: Initial learning rate
102 |
103 | **ss**: Learning rate reduction step size
104 |
105 | **acc\_n**: Validation set accuracy at epoch *n*. Note that *acc_0* refers to the accuracy of a randomly initialized model.
106 |
107 |
108 |
--------------------------------------------------------------------------------
/data/lstm_ptb.csv:
--------------------------------------------------------------------------------
1 | base_model,training_dataset,n_layer,n_inputs,base_lr,ss,ep_0,ep_1,ep_2,ep_3,ep_4,ep_5,ep_6,ep_7,ep_8,ep_9,ep_10,ep_11,ep_12,ep_13,ep_14,ep_15,ep_16,ep_17,ep_18,ep_19,ep_20,ep_21,ep_22,ep_23,ep_24,ep_25,ep_26,ep_27,ep_28,ep_29,ep_30,ep_31,ep_32,ep_33,ep_34,ep_35,ep_36,ep_37,ep_38,ep_39,ep_40,ep_41,ep_42,ep_43,ep_44,ep_45,ep_46,ep_47,ep_48,ep_49,ep_50,ep_51,ep_52,ep_53,ep_54,ep_55,ep_56,ep_57,ep_58,ep_59
2 | lstm,penntreebank,2,90,0.1,15000,296,204.5,177.4,160.1,143.8,138.4,134.1,124.3,119.9,119.4,115.4,114.1,111,108.6,107.3,107.4,108.3,105.4,104.6,101.8,101.8,103.6,104.5,101.9,100.3,99.51,98.8,97.6,97.79,97.6,98.05,96.76,97.22,95.52,95.79,97.04,96.16,94.85,94.78,95.79,89.64,89.32,89.05,88.83,88.65,88.5,88.34,88.23,88.11,87.99,87.87,87.76,87.66,87.56,87.47,87.38,87.29,87.2,87.1,87.01
3 | lstm,penntreebank,2,330,0.1,15000,286.3,196.9,159,146.9,132.1,126.2,118.7,112.9,108.5,106.1,105.1,100.9,99.46,98.78,96.37,96.43,95.85,95.52,92.11,91.61,93.22,91.65,90.82,89.67,89.03,88.47,89.55,88.79,87.4,87.78,86.57,86.45,87.48,85.8,85.32,87.45,80.99,80.53,80.26,80.06,79.88,79.7,79.52,79.38,79.24,79.12,79.01,78.92,78.82,78.73,78.64,78.56,78.48,78.4,78.32,78.24,78.16,78.08,78.01,77.93
4 | lstm,penntreebank,2,570,0.1,15000,292.4,189,156.6,141.4,129.8,120.2,113.7,108.3,104,100.5,99.72,96.82,94.75,93.37,91.79,91.66,90.82,89.92,88.79,88.05,89.72,87.93,88.06,85.89,85.19,84.19,85.21,84.88,84.16,83.5,84.05,82.7,82.39,82.11,82.38,82.46,81.98,82.13,81.49,81.19,81,80.25,80.97,79.99,80.33,80.29,79.53,79.75,80,80.72,75.24,74.84,74.64,74.48,74.34,74.22,74.09,73.99,73.88,73.78
5 | lstm,penntreebank,2,810,0.1,15000,274.2,194.8,157.7,140,125.4,117.1,110.9,106.8,102.2,100.7,96.91,94.91,92.73,91.18,90.92,89.06,88.83,87.42,86.49,87.53,87.44,84.86,87.41,83.22,82.54,81.85,82.21,83.14,81.28,81.13,80.94,80.18,80.72,80.17,79.68,80.93,79.89,79.22,78.47,78,78.47,77.98,78.45,78.67,78.21,77.31,77.31,77.56,77.84,77.92,78.18,73.01,72.63,72.4,72.2,72.03,71.87,71.75,71.65,71.54
6 | lstm,penntreebank,2,1050,0.1,15000,288.6,190.1,160.3,137.2,125.5,116.9,110.3,104.5,101.5,97.86,96.01,93.13,91.06,89.42,89.91,88.1,87.48,86.71,84.94,84.53,84.61,84.11,83.42,82.21,82.02,81.26,81.16,81.56,80.4,79.81,79,78.86,79.31,78.9,78.35,79.95,74.04,73.48,73.15,72.91,72.73,72.57,72.44,72.32,72.21,72.11,72,71.89,71.79,71.67,71.57,71.48,71.4,71.32,71.24,71.16,71.09,71.02,70.95,70.88
7 | lstm,penntreebank,2,1290,0.1,15000,292.9,197,160.5,138.3,125.3,115.4,109.7,103.6,100.9,98.39,95.18,95.4,91.42,89.37,87.6,88.61,84.96,84.62,84.15,83.96,84.11,83.28,81.53,81.74,80.93,79.83,79.57,80.07,78.53,78.77,78.68,77.71,78.45,78.1,77.25,77.78,76.82,77.05,76.5,75.99,75.8,77.19,76.13,75.72,76.17,74.87,75.35,75.4,74.91,74.78,76.21,70.69,70.29,70.02,69.81,69.67,69.54,69.41,69.3,69.21
8 | lstm,penntreebank,3,130,0.1,15000,375.2,233.6,195.5,168.6,155.6,145.7,137.7,132,125.6,122.4,121.5,116.4,114.8,112.7,110.7,110.9,109.4,107.6,105.9,104.4,108.5,105.5,107.6,101.4,101.8,100.2,104.7,95.07,94.54,94.14,93.8,93.55,93.31,93.11,92.91,92.71,92.53,92.35,92.18,92.01,91.83,91.66,91.51,91.35,91.2,91.06,90.92,90.8,90.68,90.56,90.45,90.34,90.23,90.12,90.02,89.92,89.82,89.73,89.63,89.54
9 | lstm,penntreebank,3,370,0.1,15000,330.1,214,172.1,152.4,143.2,131.6,124.9,116.1,110.4,107.5,107.5,101.7,100.4,98.41,97.31,97.15,96.03,93.87,92.61,91.18,93.69,91.3,90.65,89.49,88.58,88.14,88.09,89.26,86.17,86.47,85.87,85.41,86.63,84.45,85.74,84.8,84.32,83.58,83.24,83.27,83.41,82.4,82.28,81.98,82,82.45,81.39,81.38,81.28,80.92,81.01,80.68,81.52,76.57,76.25,76.01,75.82,75.65,75.51,75.39
10 | lstm,penntreebank,3,610,0.1,15000,339.6,215.7,167.7,151.3,132.8,123.9,117,112.5,105.4,105.2,100.3,99.98,95.99,93.17,91.69,91.22,89.04,89.06,89.06,86.32,88.41,84.77,85.1,84.03,83.32,82.18,81.92,82.14,80.8,80.4,80.07,80.2,80.25,79.26,79.68,78.98,79.12,78.68,78.72,78.16,77.93,77.14,77.29,78.29,77.34,76.36,76.2,76.1,76.47,75.72,75.37,75.88,75.86,75.16,75.06,74.9,74.89,74.25,75.11,74.68
11 | lstm,penntreebank,3,850,0.1,15000,337,218.9,167.3,151,135.5,123,116.4,109,104.2,99.98,98.32,94.6,92.75,90.47,88.84,88.11,89.26,85.81,84.51,83.23,83.51,82.18,81.99,83.21,80.22,79.49,80.64,79.19,78.33,77.97,77.74,77.05,77.11,76.73,76.04,77.01,76.95,75.92,75.39,75.06,76.03,75.64,76.39,70.59,70.23,69.97,69.77,69.62,69.46,69.33,69.2,69.08,68.97,68.87,68.78,68.67,68.58,68.49,68.41,68.34
12 | lstm,penntreebank,3,1090,0.1,15000,349.6,216.2,180,147.9,131.8,121.4,115.9,107.4,100.8,100.3,96.49,93.09,91,89.17,87.35,87.09,84.9,85.38,82.67,81.81,82.55,81.23,80.82,79.26,79.51,78.27,78.93,77.89,76.76,76.17,76.39,75.74,76.94,74.99,75.04,75.57,74.77,74.62,74.02,73.82,73.75,72.78,72.81,72.39,72.82,72.69,72.58,72,71.85,71.72,72.77,67.88,67.53,67.29,67.13,67,66.88,66.77,66.68,66.6
13 | lstm,penntreebank,3,1330,0.1,15000,350,222.3,175,148,133.1,122.7,112.2,107.9,101.3,101.4,97.65,92.76,90.75,88.61,86.29,86.16,84.43,83.38,83.03,80.9,81.76,80.06,79.34,78.71,77.99,77.11,76.58,77.22,75.79,75.6,74.98,74.89,75.05,75.71,74.52,74.11,73.79,73.18,73.48,72.59,72.65,71.89,71.96,72.23,72.94,71.54,70.88,70.93,70.78,70.9,71.91,67.27,66.84,66.57,66.35,66.19,66.05,65.93,65.81,65.69
14 | lstm,penntreebank,4,170,0.1,15000,493.6,281.6,216.6,194.2,165.8,158.7,151.3,138.6,130.6,127.3,125.9,127,118.3,114.7,113.6,112.7,111.2,112.5,109.1,105.6,107.2,107,105.8,105.3,101.8,101.7,101.7,100.9,101.6,99.23,99.23,99.96,99.97,98.05,99.03,99.25,92.08,91.61,91.24,90.99,90.76,90.56,90.37,90.19,90.04,89.89,89.75,89.61,89.48,89.36,89.22,89.11,89,88.89,88.79,88.68,88.58,88.47,88.37,88.28
15 | lstm,penntreebank,4,410,0.1,15000,453.7,269.5,210,171.5,154.6,141.3,133.8,123,116.7,113.8,112.3,106.7,104.8,101.8,99.89,99.88,97.78,96.96,94.46,93.3,93.35,92.39,92.07,90.24,89.51,89.4,88.51,88.91,87.22,87.17,86.46,86.79,87.77,85.23,85.43,85.84,83.76,83.65,84.14,84.04,82.94,82.45,82.68,81.96,83.45,82.07,81.78,81.66,81.84,81.3,81.71,80.8,80.79,81.36,80.67,80.27,80.86,79.98,80,79.47
16 | lstm,penntreebank,4,650,0.1,15000,498.7,263,200.6,172.3,153.1,137.3,129.6,122.1,112.6,111.7,106.5,102,99.15,97.81,94.98,93.78,94.01,92.77,89.91,87.96,90.19,88.19,86.24,85.17,84.37,84.09,83.64,83.88,82.18,81.77,81.59,80.97,81.21,79.84,79.86,79.97,79.71,78.69,78.79,78.85,78.37,77.82,77.09,77.29,77.2,77.21,76.39,76.51,76.51,76.88,77.22,71.89,71.52,71.26,71.06,70.9,70.77,70.64,70.51,70.39
17 | lstm,penntreebank,4,890,0.1,15000,426.7,289.2,194.8,169.9,145.1,135.6,124.7,115.7,109.8,105.8,104.4,99.39,96.35,95.75,92.11,92.09,89.13,88.23,85.9,84.62,85.77,84.83,82.74,82.44,81.28,80.54,80.14,80.71,79.21,78.97,78.22,77.85,77.82,77.5,76.98,78.09,76.6,76.06,76.09,75.32,76.04,75.09,74.59,74.46,74.71,74.44,73.48,73.63,73.84,73.09,74.28,72.91,73.3,72.99,72.53,71.99,72.45,72.32,72.33,72.39
18 | lstm,penntreebank,4,1130,0.1,15000,455.9,262.7,199,169.6,149.2,137.7,127,117.5,111.1,107,103.7,99.07,97.17,93.94,93.27,91.27,88.88,87.15,85.9,84.87,85.37,83.75,84.52,81.82,80.53,80.02,79.58,79.18,78,78.09,77.61,76.63,77.49,75.91,76.65,76.28,76.52,74.87,74.27,74.09,74.13,73.42,73.84,73.37,73.25,73.31,72.81,72.12,72.28,71.94,72.46,71.29,72.54,71.79,71.89,71.06,71.37,71.42,67.27,66.83
19 | lstm,penntreebank,4,1370,0.1,15000,484.7,260.3,208.3,168.1,149.9,133.4,123.6,113.9,108.4,107.3,101,97.83,94.29,91.99,89.7,89.07,86.8,85.78,84.28,83.28,82.99,81.43,80.43,80.1,78.85,78.11,77.7,77.66,76.72,76.24,75.95,75.39,75.27,74.37,74.99,75.28,74.79,73.39,73.68,73.21,73.42,72.72,72.89,72.43,72.83,72.4,71.72,71.59,71.38,71.19,71.66,70.5,70.66,70.74,70.49,70.18,70.59,71.06,66.81,66.41
20 | lstm,penntreebank,5,210,0.1,15000,725.3,700.6,641.3,436.1,295.9,256.4,223.7,183.1,167,154.1,151.6,139.1,136.6,130.5,126.9,125.7,122.2,123.2,116.4,115.8,115.4,114.3,110.2,110.6,107.9,106.4,106.7,105.3,104.3,104.4,102.9,101.9,105,100.8,100.7,99.98,98.8,99.42,98.09,100.1,98.82,96.91,98.11,96.88,97.63,97.39,95.5,95.92,97.8,90.03,89.58,89.26,89,88.8,88.6,88.46,88.34,88.24,88.15,88.07
21 | lstm,penntreebank,5,450,0.1,15000,729.2,648.5,406.9,293.5,224.1,191.1,170.1,149.3,139.6,131.2,127.4,119,113.7,111.9,112.9,107,106.2,104.5,100.6,99.46,102,98.5,96.93,94.63,93.63,93.05,93.22,93.69,91.84,90.67,90.22,89.93,90.22,88.76,88.51,88.93,87.2,87.05,86.99,86.41,85.9,85.04,85.4,88.4,80.06,79.59,79.27,78.99,78.77,78.58,78.42,78.26,78.11,77.98,77.86,77.74,77.62,77.51,77.41,77.32
22 | lstm,penntreebank,5,690,0.1,15000,729.3,498.9,301.5,241,193.8,172.5,156.9,143.4,131.1,121.3,119.4,112.1,108.9,108,101.7,100.4,98.66,96.94,93.7,94.81,94.04,92.15,89.92,88.55,87.66,86.49,86.92,85.84,85.26,83.75,84.33,82.97,83.31,83.22,82.41,84.01,77.25,76.71,76.41,76.13,75.89,75.65,75.44,75.23,75.03,74.85,74.68,74.52,74.38,74.24,74.1,73.97,73.84,73.71,73.59,73.47,73.36,73.25,73.15,73.05
23 | lstm,penntreebank,5,930,0.1,15000,727.4,578.2,471,251.3,191.3,167,157.8,141.4,130.6,125.1,118,111.8,108.5,104.9,101,99.57,96.53,97.18,93.28,92.03,93.09,90.03,92.33,87.21,85.72,84.83,84.88,83.77,82.63,82.48,81.57,80.96,81.33,79.92,79.97,80.42,79.6,78.39,78.2,78.32,78.29,77.94,77.28,76.61,77.15,76.04,75.69,75.39,75.38,75.46,75.71,74.75,74.43,74.98,74.47,73.76,74.11,73.75,74.07,73.51
24 | lstm,penntreebank,5,1170,0.1,15000,749.2,473.8,285.4,234.5,192.1,168.5,150.8,138.4,131.6,123.3,116.8,110.4,105.8,103,99.39,97.95,95.36,94.89,91.11,90.42,89.2,88.66,86.42,85.45,84.12,83.24,83.01,82.92,81.41,80.76,80.5,80.09,80.09,79,78.91,79.47,78.05,77.17,77.47,76.77,76.54,75.98,75.71,74.83,75.05,74.99,74.71,74.41,74.27,73.75,75.14,69.84,69.44,69.16,68.95,68.76,68.61,68.48,68.35,68.22
25 | lstm,penntreebank,2,10,0.1,15000,374.2,258.1,223.5,218.7,196,186.9,187.1,171.8,167.8,167.2,164.8,159.3,158.6,154.9,152,153.3,156.7,149,146.8,145.9,151.7,146.2,146.5,149,143.1,146.2,136.6,136,135.5,135,134.7,134.4,134.1,133.8,133.6,133.3,133.1,132.9,132.7,132.6,132.4,132.3,132.1,131.9,131.8,131.7,131.5,131.3,131.2,131.1,130.9,130.8,130.7,130.5,130.4,130.3,130.2,130.1,130,129.9
26 | lstm,penntreebank,2,250,0.1,15000,289.8,197.8,161.8,149.2,134.5,126.9,120.3,116.9,111.6,111.8,107,103.3,102.7,100.3,97.84,98.86,96.65,99.32,94.8,95.08,97.58,94.9,92.43,91.87,91.76,92,90.84,90.44,89.46,90.53,88.83,89.67,89.39,88.91,88.67,91.43,83.31,82.85,82.56,82.34,82.16,82,81.85,81.69,81.56,81.45,81.33,81.22,81.11,81.01,80.91,80.82,80.73,80.65,80.56,80.49,80.41,80.34,80.27,80.21
27 | lstm,penntreebank,2,490,0.1,15000,286.7,190.4,159.5,142.9,131.8,122.7,113.5,110.2,105.7,104,100.9,97.99,96.7,94.55,93.88,92.83,92.38,94.2,89.66,89.28,88.87,88.46,87.93,86.56,86.08,85.05,84.75,85.04,84.52,84.57,83.98,83.68,83.56,82.85,82.8,84.12,82.46,82.26,81.6,81.36,81.63,80.9,81.01,82.05,81.48,80.55,81.01,80.23,80.68,80.89,80.55,79.51,82.59,75.58,75.18,74.91,74.73,74.56,74.42,74.3
28 | lstm,penntreebank,2,730,0.1,15000,287.5,192,157.3,143.6,125.3,118.8,113.6,106.6,102.1,103,97.76,95.08,94.74,92.55,90.79,90.44,90.74,88.79,86.3,87.82,85.69,85.48,84.87,84.26,83.18,82.83,82.93,82.38,82.08,81.8,81.28,81.34,81.78,80.71,80.27,80.74,80.26,79.79,79.75,79.7,79.22,78.93,78.64,78.17,78,78.37,78.38,77.33,77.84,77.91,77.54,77.16,77.85,77.58,73.12,72.77,72.49,72.29,72.15,72.01
29 | lstm,penntreebank,2,970,0.1,15000,301,189,155.7,140,127.1,118.1,110.5,105.4,101.9,99.7,97.46,93.33,92.53,91.18,88.98,89.2,88.16,86.72,85.42,84.06,87.97,84.87,83.1,82.4,81.95,81.33,81.02,81.91,80.18,80.11,79.49,78.93,80.03,78.75,79.39,79.35,78.65,78.1,78.52,77.6,77.92,78.14,77.14,76.99,76.87,76.89,76.68,76.39,76.31,76.62,76.36,76.14,76.57,75.69,75.45,75.17,75.35,75.25,75.85,75.23
30 | lstm,penntreebank,2,1210,0.1,15000,280.4,197.2,156.7,138.8,127.4,115.9,110.8,103.3,100.9,99.39,95.19,92,90.46,90.84,87.51,87.44,87.68,85.84,83.77,84.68,83.87,82.1,82.16,80.86,80.38,80.97,80.38,80.52,79.09,78.46,78.51,77.79,78.38,78.21,77.37,77.85,77.86,76.87,76.53,76.44,76.36,76.16,75.81,75.66,76.28,76.32,75.28,75.5,75.21,75.3,75.25,74.39,75.48,70.9,70.52,70.25,70.03,69.88,69.74,69.59
31 | lstm,penntreebank,3,50,0.1,15000,372.5,255.9,216.2,190.2,168.8,164.9,159.9,149.6,145.2,144.4,138.9,134.8,133.1,130.8,129.8,129.8,126.7,127.5,125,123,124,124,120.8,120.6,119.4,117.7,117.9,118.8,119.3,115.9,117.6,114.3,114.7,114.1,115.3,117.2,108.3,107.8,107.5,107.2,107,106.8,106.6,106.5,106.3,106.2,106,105.9,105.8,105.7,105.5,105.4,105.3,105.2,105.1,105,104.9,104.8,104.7,104.6
32 | lstm,penntreebank,3,290,0.1,15000,360.1,219.1,171.4,157.8,140.7,134.1,127.7,118.4,112.9,111.3,109.4,105.4,102.6,100.7,99.87,98.89,98.48,99.58,93.85,93.43,95.78,93.5,91.44,91.19,90.66,89.4,89.13,92.04,88.89,89.05,87.71,87.65,88.88,87.59,87.44,87.82,87.2,85.96,85.91,85.28,85.77,84.9,85.54,84.07,85.01,85.32,79.89,79.57,79.34,79.15,78.98,78.84,78.72,78.58,78.47,78.38,78.28,78.19,78.11,78.02
33 | lstm,penntreebank,3,530,0.1,15000,358.1,228.8,170.3,151.9,133.9,127.7,119.3,113.8,106.9,106.8,103,99.23,98.29,95.73,93.45,92.5,91.11,91.22,89.86,89.02,87.49,87.11,86.43,86.16,84.63,84.15,84.95,84.28,83.21,82.06,81.85,81.88,82.03,80.82,81.35,82.15,76.31,75.87,75.63,75.37,75.18,75.01,74.84,74.68,74.53,74.39,74.26,74.13,74.01,73.9,73.8,73.71,73.62,73.53,73.44,73.34,73.26,73.17,73.09,73
34 | lstm,penntreebank,3,770,0.1,15000,356.8,209.5,165.4,150.6,134.1,124,115.8,109.5,103.4,102.6,98.31,96.57,95.93,90.84,89.59,88.96,88.25,87.07,85.46,84.73,84.84,83.81,83.25,82.18,81.49,80.76,80.69,80.83,79.35,79.07,78.73,78.17,78.28,78.13,78.16,78.21,77.03,77.22,77.24,76.19,76.3,75.5,75.48,75.21,75.32,74.49,74.8,74.59,74.39,74.51,74.37,73.81,74.13,74.24,73.9,73.51,74.14,72.77,73.85,69.27
35 | lstm,penntreebank,3,1010,0.1,15000,329.3,218.5,167.4,148,132.4,121.3,114.9,107.2,102.5,99.89,96.76,93.59,92.62,89.19,87.75,87.51,85.22,84.49,83.9,82.75,83.25,81.99,80.69,80.56,79.46,78.55,78.39,78.72,77.96,77.26,76.59,76.43,76.77,77.64,75.69,76.33,75.25,74.65,74.91,74.45,74.38,73.23,73.85,73.93,74.18,72.72,72.7,72.64,72.68,72.47,73.19,71.94,72.77,68.12,67.77,67.58,67.39,67.22,67.08,66.96
36 | lstm,penntreebank,3,1250,0.1,15000,367.6,212.4,169.7,148.6,131.2,121.1,114.5,105.6,101.5,98.16,95.89,92.19,90.68,88.39,87.09,86.32,85.2,83.75,82.74,81.49,81.67,81.23,79.82,78.31,78.12,76.85,77.1,76.37,76.12,76.1,75.59,75.15,74.9,74.82,74.6,75.14,74.4,73.04,73.39,72.65,73.06,72.14,72.23,72.61,72.31,71.98,71.5,70.9,71.05,71.03,71.61,70.75,70.66,70.36,70.74,70.21,70.04,69.95,70.29,69.76
37 | lstm,penntreebank,4,90,0.1,15000,521.8,313.1,253.5,215.4,185.9,180.2,164.6,152.9,146.7,143.7,142.8,134.7,134.2,128.2,128.4,127.4,127.6,125.5,120.2,120.2,120.4,119.9,118.3,116.7,114.7,113.7,113.8,113.6,113.6,112.2,111.2,110.7,113.3,110.6,109.6,111.4,110.7,111.2,103.4,102.7,102.4,102.1,101.9,101.6,101.5,101.2,101.1,100.9,100.8,100.6,100.5,100.3,100.2,100.1,99.94,99.83,99.71,99.6,99.51,99.42
38 | lstm,penntreebank,4,330,0.1,15000,498.3,279.6,213.2,182.2,161.7,145,134.8,126.8,124.7,116.3,117.2,110.2,109.9,106.1,104.2,103.5,103.5,103.1,99.43,99.3,98.43,98.03,94.45,93.83,92.89,92.11,92.55,92.57,90.33,89.84,89.16,89.11,89.76,87.65,88.24,88.31,87.8,87.79,87.37,86.67,86.5,85.34,86.11,84.63,85.53,85.52,84.7,84.66,84.37,83.59,85.22,79.68,79.33,79.06,78.86,78.73,78.6,78.47,78.35,78.26
39 | lstm,penntreebank,4,570,0.1,15000,447.3,267.8,195.2,172.9,147.8,139.4,128.4,118.2,112.9,109.3,107.6,103.1,101.8,98.02,95.43,95.94,94.48,92.69,92.48,90.17,90.24,88.18,87.72,86.06,85.58,85.56,84.27,85.14,83.24,82.73,82.99,82.83,81.92,81.28,81.61,81.39,80.58,80.82,79.99,79.64,79.63,79.06,79,78.5,79.03,78.62,77.61,77.38,77.33,77.82,78.03,76.99,76.47,76.28,76.43,75.83,76.72,75.77,75.41,76.14
40 | lstm,penntreebank,4,810,0.1,15000,498.2,273.2,200.3,168.5,147.1,134.9,125.4,116.4,111.3,108.6,103.3,100.4,96.99,94.39,92.34,92.1,90.91,89.82,88.76,86.47,86.98,84.8,83.59,82.45,81.93,80.9,81.85,80.68,79.66,79.45,78.79,79.09,79.21,78.98,78.33,77.97,77.94,76.93,76.63,75.89,76.38,75.44,76.38,74.85,75.87,75.03,74.25,74.59,73.96,74.35,75.16,69.89,69.52,69.25,69.03,68.86,68.7,68.57,68.45,68.34
41 | lstm,penntreebank,4,1050,0.1,15000,420.1,260.7,197,165.6,146.4,131.5,122.4,113.9,107.9,105,101.6,97.51,96.88,93.25,90.79,89.89,88.38,86.9,87.18,84.63,83.77,83.11,83.09,80.98,81.1,79.99,79.66,79.05,78.22,77.77,77.12,77.81,77.76,76.24,76.3,75.79,75.1,75.03,74.75,74.42,74.74,73.74,73.58,73.44,73.29,72.8,72.73,72.53,72.18,72.44,72.47,72.91,68.44,68.08,67.82,67.61,67.45,67.29,67.17,67.05
42 | lstm,penntreebank,4,1290,0.1,15000,414.3,248.8,188.6,166.7,147.4,132.2,123.5,115.1,107.8,106.1,103.4,96.4,93.96,92.11,90.29,89.81,87.16,86.22,84.58,84.2,83.94,81.83,81,80.36,79.64,78.98,79.14,78.39,77.03,76.83,76.58,75.98,76.11,75.74,75.42,76.2,74.57,74.21,74.3,73.49,73.74,72.94,73,73.5,72.69,72.92,71.93,71.66,71.62,71.67,72.18,71.3,71.84,71.18,71.23,70.86,71.17,71.03,70.68,70.19
43 | lstm,penntreebank,5,130,0.1,15000,731.5,698.8,691.3,592.6,397.8,310.5,239,205.8,182.9,173.3,165.4,153.1,150.3,143.2,140,137.1,136.3,134.8,127.9,124.1,126.2,124.4,122.7,122.5,119.3,116.5,115.5,115.8,114.5,114.3,112.1,111.3,113.6,110.8,111.8,111.2,109.4,111.6,102.9,102.4,102,101.8,101.5,101.2,101,100.8,100.6,100.5,100.3,100.1,99.97,99.83,99.69,99.57,99.44,99.31,99.2,99.08,98.95,98.82
44 | lstm,penntreebank,5,370,0.1,15000,729.8,650.5,409.4,305.2,232.2,204.7,172.9,156.6,145.7,141.5,133.7,125.5,121.1,119.7,114.2,112.6,109.9,112.7,106.2,107.2,105,101.5,102.7,100.7,97.91,96.57,96.92,97.63,95.42,94.12,94.03,92.78,93.87,91.02,92.24,91.83,90.49,90.87,90.48,89.49,90.07,88.14,88.63,87.79,87.82,87.3,87.29,86.29,86.8,86.59,86.49,85.71,85.65,86.38,80.71,80.26,80.03,79.85,79.72,79.58
45 | lstm,penntreebank,5,610,0.1,15000,728.2,520.1,342.1,276.2,202.2,175.5,154.9,140.7,133.5,130.4,123.3,114.7,112.3,109,105.4,105.1,102,103.3,97.28,96.9,97.7,95.57,92.92,91.31,91.18,90.26,89.54,89.74,87.99,87.43,87.36,85.89,86.35,84.98,84.88,83.73,84.46,83.42,82.35,82.91,82.63,81.78,81.32,81.12,81.32,80.81,80.77,80.02,79.99,79.89,80.57,79.43,80.45,79.5,79.29,79.4,78.36,78.38,78.19,77.95
46 | lstm,penntreebank,5,850,0.1,15000,726.7,427.1,283.7,232.3,187.7,165.1,150.5,135.5,125.8,119.9,117.3,110.2,107.3,102.3,100.2,99.51,98.12,97.4,93.65,91.18,90.34,89.38,88.89,87.94,86.48,85.26,85.53,84.85,83.09,83.32,82.37,81.39,82.53,81.09,81.2,81.44,80.19,81.55,74.74,74.31,73.98,73.73,73.5,73.3,73.11,72.95,72.79,72.62,72.47,72.33,72.19,72.05,71.92,71.79,71.66,71.54,71.42,71.3,71.19,71.09
47 | lstm,penntreebank,5,1090,0.1,15000,750.4,410.6,283.3,222.7,184.8,159.3,149,135.6,124.4,117.7,114.5,108.6,105.7,102.1,98.68,97.1,94.92,93.5,91.02,89.27,89.51,86.63,86.59,84.9,84.29,83.7,83.31,83.54,81.91,81.28,81.02,79.94,79.11,78.47,78.54,78.37,78.45,78.53,76.9,76.45,76.6,75.37,75.85,76.35,75.62,75.32,74.47,74.45,74.17,74.03,74.93,74.13,73.86,73.87,73.64,73.41,73.07,72.91,72.77,72.55
48 | lstm,penntreebank,5,1330,0.1,15000,729.1,438.7,314,220,187.9,162.2,145.6,132,122.9,121.4,112.9,107.2,103.5,100.9,97.31,97.52,93.69,91.07,89.66,88.37,88.13,86.68,85.76,83.47,83.24,82.05,81.49,81.09,80.53,79.71,79.01,79.38,78.31,77.96,77.41,79.98,72.9,72.44,72.07,71.73,71.46,71.25,71.07,70.89,70.71,70.54,70.38,70.22,70.07,69.93,69.8,69.68,69.57,69.46,69.34,69.24,69.13,69.02,68.92,68.82
49 | lstm,penntreebank,2,110,0.1,15000,300.8,201.7,170.7,156.9,141.7,135.3,130.5,125.6,118.2,119.1,117.2,110.7,109.1,107.8,106.9,106.3,104.9,103.1,101.4,100.8,103,100.7,99.81,98.18,97.56,98.09,97.06,96.84,97.03,96,95.24,94.5,95.84,93.76,94.68,95.08,93.86,94.5,93.32,93.41,93.27,92.18,92.35,91.96,93.35,87.48,87.12,86.86,86.66,86.5,86.35,86.19,86.07,85.98,85.88,85.79,85.7,85.61,85.52,85.44
50 | lstm,penntreebank,2,350,0.1,15000,287.4,195.1,164.2,142.2,129.2,123.7,117,113.4,108,104.9,103.8,101.5,100.9,98.28,97.33,95.9,94.89,95.1,92.25,92.42,94.15,90.72,91.09,89.8,89.41,88.66,87.8,87.92,87.97,87.85,86.19,86.02,87.33,84.69,85.86,86.6,85.17,84.4,84.36,84.88,79.57,79.24,78.96,78.78,78.62,78.47,78.31,78.17,78.04,77.9,77.78,77.68,77.58,77.49,77.39,77.31,77.23,77.16,77.08,77.01
51 | lstm,penntreebank,2,590,0.1,15000,301,201.2,158.2,142.4,129.8,121.3,114.7,107.9,104.5,101.9,100.1,97.02,96.48,93.96,93.69,92.06,90.72,89.68,89.34,88.81,89.67,86.91,86.28,85.69,85.41,84.63,83.92,84.92,83.6,83.6,82.62,81.99,84.33,77.84,77.42,77.16,76.96,76.72,76.5,76.34,76.2,76.07,75.94,75.83,75.71,75.6,75.5,75.4,75.31,75.21,75.12,75.03,74.96,74.89,74.82,74.75,74.69,74.62,74.55,74.49
52 | lstm,penntreebank,2,830,0.1,15000,281.7,191.6,165.4,142,131.1,116.5,111.8,105.6,102.1,98.79,97.46,94.36,93.56,91.66,91.35,89.54,89.1,88.84,86.58,85.31,87.01,84.8,84.89,83.28,83.24,81.64,83,83.15,81.14,80.89,81.36,79.81,80.18,79.7,80.07,80.4,80.26,78.59,78.92,78.64,78.77,78.39,77.99,77.99,79.05,73.63,73.17,72.89,72.72,72.55,72.4,72.28,72.17,72.07,71.96,71.87,71.78,71.69,71.61,71.54
53 | lstm,penntreebank,2,1070,0.1,15000,293.2,194.8,158.3,139.1,132,116.5,111.2,104.6,100.3,98.58,95.5,92.51,90.96,91.82,88.18,87.63,87.31,86.01,84.06,85.06,85.64,84.08,82.44,81.61,81.72,81.05,80.76,82.06,79.64,79.39,79.19,78.91,79.58,78.49,77.88,79.88,73.68,73.18,72.83,72.6,72.43,72.28,72.13,72,71.9,71.8,71.7,71.62,71.55,71.46,71.38,71.3,71.22,71.14,71.06,70.99,70.92,70.86,70.79,70.72
54 | lstm,penntreebank,2,1310,0.1,15000,285.4,196.5,154.8,139.3,124.7,118.2,109.8,104.3,99.33,99.5,96.02,92.23,90.77,89.48,87.96,87.32,86.2,85.72,84.74,83.76,83.88,83.31,81.5,81.25,81.71,79.74,80.38,80.63,79.2,78.69,78.17,77.97,78.1,78.49,77.76,78.02,77.4,77.6,76.49,76.42,76.89,75.75,75.74,75.36,75.27,75.58,75.21,74.85,75.35,75.08,75.01,74.28,74.43,74.75,73.83,73.83,74.08,73.62,74.42,69.84
55 | lstm,penntreebank,3,150,0.1,15000,375.4,236.7,186.1,168.2,153.5,143.7,138.3,129.6,124.3,121.8,120.2,114.4,112.7,110.4,109,109,107,106.2,103.9,104.4,104.9,102.5,101.9,101.9,100.2,99.02,99.54,99.75,99.82,97.5,96.38,96.07,98.47,95.72,96.38,96.14,95.79,96.35,89.93,89.48,89.2,88.97,88.78,88.62,88.46,88.3,88.16,88.03,87.92,87.81,87.69,87.59,87.5,87.4,87.31,87.21,87.13,87.04,86.97,86.88
56 | lstm,penntreebank,3,390,0.1,15000,358,216,171.6,155.7,139.7,130.5,122,121.1,111.7,107.9,106.1,102.8,99.79,98.5,96.5,96.07,94.93,94.17,91.89,91.68,92.12,90.97,90.23,89.02,88.3,87.1,87.25,86.44,86.14,86,84.96,84.78,85.64,87.37,79.54,79.11,78.8,78.59,78.39,78.2,78.03,77.88,77.75,77.62,77.5,77.39,77.28,77.18,77.08,76.97,76.87,76.76,76.66,76.57,76.47,76.38,76.29,76.2,76.12,76.04
57 | lstm,penntreebank,3,630,0.1,15000,348.7,214.4,175,153.3,136,125.3,119.8,110.9,106.2,104.6,100.7,97.41,95.54,93.19,91.21,91,88.94,89.51,86.89,88.62,86.92,86.11,83.79,83.76,82.65,83.07,82.08,81.2,81.33,80.45,80.02,79.57,79.42,78.84,78.9,80.19,80.25,74.22,73.82,73.59,73.35,73.14,72.94,72.75,72.59,72.45,72.31,72.18,72.06,71.94,71.84,71.75,71.66,71.57,71.49,71.41,71.34,71.26,71.19,71.12
58 | lstm,penntreebank,3,870,0.1,15000,331.6,217.8,175.2,150.3,132.9,122.6,115.8,108.7,103.3,101,97.85,94.02,91.55,90.36,89.28,88.27,86.66,85.93,84.02,83.57,84.14,83.07,81.6,80.52,80.35,79.38,79.61,79.52,78.25,77.99,77.36,76.95,77.08,76.67,76.39,76.5,76.33,75.81,75.22,75.15,75.29,74.33,74.38,75.46,74.01,74.51,73.25,73.62,73.82,73.64,73.63,72.99,73.45,69.1,68.76,68.57,68.39,68.24,68.12,68
59 | lstm,penntreebank,3,1110,0.1,15000,344.2,214,169.5,148.6,133.8,122.9,114.1,107.1,104.1,99.03,97.35,93.79,91.35,89.41,88.07,86.64,85.58,85.36,83.91,82.19,83.08,80.51,79.5,78.92,78.61,78.1,77.74,77.35,76.66,76.52,77.04,75.51,75.65,75.06,75.2,75.49,73.87,73.69,73.72,73.59,73.74,72.89,72.78,72.58,72.67,72.54,71.69,71.99,71.49,71.91,72.1,71.13,71.93,67.54,67.15,66.92,66.76,66.61,66.5,66.39
60 | lstm,penntreebank,3,1350,0.1,15000,340.8,219.1,173.8,146.1,131,118.8,113.2,105.1,101.1,97.8,94.92,91.49,89.5,87.95,85.96,85.61,83.54,82.5,82.09,80.44,80.36,79.43,79.27,78.06,77.4,77.36,76.55,76.37,75.55,75.06,74.8,74.6,75.08,74.03,73.77,73.7,74.11,72.57,72.82,72.21,71.9,71.3,71.46,71.44,71.24,71.12,70.71,70.53,70.37,70.22,71.36,66.88,66.5,66.24,66.04,65.87,65.74,65.63,65.53,65.44
61 | lstm,penntreebank,4,190,0.1,15000,527.4,313.9,228.5,198.4,169.2,157,147.7,139.2,134,126.1,125.2,118.9,116.4,113.7,110.7,111.1,109,112.9,107.3,104.7,109.1,106.8,104.5,102.2,100.7,99.95,99.18,99.7,98.23,97.67,97.74,97.14,97.69,95.89,96.66,96.74,95.52,96.19,94.79,94.03,94.31,93.75,93.94,93.14,93.93,94.07,88.09,87.67,87.39,87.17,86.96,86.79,86.66,86.54,86.41,86.28,86.17,86.05,85.95,85.85
62 | lstm,penntreebank,4,430,0.1,15000,436.5,265.2,203.4,178.8,155.4,140.5,130.5,121.8,116,111.6,111.5,105.6,102.9,101.1,99.38,98.48,97.4,98.51,93.11,93.83,92.23,93.02,90.73,90.18,88.42,87.98,88.38,87.13,87.33,87.07,85.64,85.26,85.68,84.11,84.68,84.19,83.16,83.75,82.96,83.43,83.21,81.9,83.01,81.54,82.3,80.87,81.3,80.75,80.94,80.46,81.51,80.38,81.64,75.84,75.46,75.21,75.04,74.88,74.76,74.65
63 | lstm,penntreebank,4,670,0.1,15000,469,260.5,201.1,172.2,148.9,138.7,130.6,121.6,112.3,108.8,106.7,102.8,98.72,96.63,95.43,93.62,93.05,91.99,89.39,88.04,89.99,86.38,88.15,84.54,84.07,83.16,84.17,82.34,81.83,81.81,82.66,80.4,81.06,80.12,79.96,79.72,79.79,78.34,78.29,78.71,77.67,76.94,77.45,76.68,77.85,76.7,76.25,76.34,76.77,75.91,75.91,74.8,75.87,74.89,75.64,74.48,74.71,74.4,75.12,70.54
64 | lstm,penntreebank,4,910,0.1,15000,457.6,257.2,189.2,164.7,145.3,134.3,122.6,114.1,108.2,105.8,104.7,98.37,95.53,94.09,91.24,90.82,89.77,88.29,86.03,84.77,86.04,84.05,82.68,81.76,80.77,79.99,80.54,80.22,80.15,78.43,78.59,77.51,77.93,77.22,76.84,77.79,76.71,76.07,76.2,75.8,75.58,75.08,74.38,74.28,74.68,74.13,73.84,73.45,73.63,73.63,73.91,73.11,73.05,72.67,72.67,72.74,72.28,71.97,71.97,72.56
65 | lstm,penntreebank,4,1150,0.1,15000,493,257.5,195.1,166.4,144.3,132.4,122.6,115.2,107.8,107.2,102.6,97.16,95.94,94.07,90.52,89.8,88.26,87.21,84.82,83.59,83.84,82,81.73,80.48,79.84,80.03,79.28,78.71,78.12,77.16,77.15,76.46,76.7,75.76,75.71,75.66,75.97,74.48,74.45,73.65,73.92,73.98,73.22,73.4,72.73,73.26,72.24,71.89,72.24,71.66,72.3,71.95,71.58,71.31,71.03,70.95,70.81,70.96,70.44,70.56
66 | lstm,penntreebank,4,1390,0.1,15000,551.4,256.9,197.5,170.7,144.8,134.8,123.4,115.3,110.1,104.2,101,96.97,95.04,92.3,90.7,89.3,87.59,86.58,84.5,83.43,82.62,82.6,80.96,80.34,79.18,79.5,77.82,77.64,76.82,76.97,76.6,75.52,76.08,74.92,75.07,75.22,73.86,74.04,73.83,73.77,73.57,72.9,73.01,73.2,72.61,72.43,71.76,71.87,71.73,71.46,71.97,71,71.45,71.48,71.05,70.73,70.74,70.73,70.48,70.53
67 | lstm,penntreebank,5,230,0.1,15000,723.5,700.6,612.9,427.2,294,254.8,209.3,176.9,163.5,154.5,147.8,137.5,133.4,127.8,124.4,124,123.9,119.6,114.6,113.3,114.4,111.9,108.5,108,105.8,105.6,104.9,104,105.8,102.2,101.4,101,101.5,99.04,100.4,99.97,98.23,98.92,98.17,97.85,96.85,95.66,95.87,95.36,96.22,95.6,94.67,94.21,93.96,93.69,94.26,93.11,93.48,92.96,92.52,92.25,92.23,92.71,93.43,87
68 | lstm,penntreebank,5,470,0.1,15000,724.1,498.5,333,272.8,206.8,186.9,167.4,148.8,139.1,135.6,128.1,119.1,115.5,111.1,108.7,107.9,106.1,103.7,100.4,99.67,100.9,97.71,95.71,94.64,93.56,93.01,92.65,91.21,90.73,90.61,89.55,88.64,89.26,87.83,87.61,88.24,87.39,86.67,86.14,86.22,85.08,84.68,84.56,83.64,84.22,84.37,83.23,84.02,83.63,82.83,84.51,78.05,77.7,77.44,77.21,77.05,76.91,76.78,76.66,76.55
69 | lstm,penntreebank,5,710,0.1,15000,723.8,604.1,329.5,258.7,199.1,176.5,155,138.8,128.8,122.9,118.8,111.3,107.4,104.8,100.9,101.4,101.4,96.7,93.71,93.42,93.84,91.54,90.15,89.92,88.13,86.81,87.08,86.09,85.09,84.82,83.77,83.22,83.63,82.37,82.29,82.75,81.24,80.69,80.92,80.34,79.71,79.75,79.05,78.73,79.22,78.86,77.7,78.05,78.21,77.45,78.2,78.19,78.57,72.75,72.36,72.12,71.92,71.75,71.6,71.47
70 | lstm,penntreebank,5,950,0.1,15000,713.1,373.8,271.1,222.7,177.2,163.8,145.6,134.6,124.6,121.3,114.8,107.7,107.5,102.1,98.93,98.27,95.63,94.91,91.55,90.67,90.81,88.08,88.39,85.89,86.01,84.43,83.93,82.97,82.81,81.79,81.28,80.6,81.51,79.73,80.31,80.14,79.67,78.51,78.94,78.33,77.81,77.05,77.43,76.69,76.8,75.9,75.69,75.94,75.89,75.63,75.95,74.81,74.91,75.28,74.78,75.01,74.3,74.08,73.76,74.12
71 | lstm,penntreebank,5,1190,0.1,15000,725.4,432.5,287.7,235.4,179.4,159.4,146.4,132.5,123.1,119.4,112.9,106.2,102.5,99.33,96.67,95.93,93.28,91.26,90.24,89.71,88.17,86.64,85.27,84.4,83.4,82.66,82.39,81.64,80.9,80.11,79.87,78.94,78.98,78.96,78.29,77.86,77.23,77.36,76.87,76.53,76.22,75.58,75.59,75.53,75.25,74.75,74.55,74.12,74.12,73.44,73.66,73.49,73.63,73.6,73.4,72.57,73.04,73.04,72.84,72.18
72 | lstm,penntreebank,2,30,0.1,15000,340.7,221.7,188.1,175.7,159.5,154.2,149.3,141.8,135.6,137,133.5,129.6,128.4,124.8,123.2,122.9,122.9,120.4,118.6,118.2,122.7,118.5,115.8,115.1,114.6,114.8,113.8,114.1,115.4,114.1,111.8,111.5,113.4,110.6,112.2,112.6,110.6,110.9,111.7,104.5,104,103.9,103.7,103.6,103.4,103.3,103.1,103,102.8,102.7,102.6,102.5,102.3,102.2,102.1,102,101.9,101.8,101.7,101.7
73 | lstm,penntreebank,2,270,0.1,15000,303.3,195.7,160.7,148.2,137.1,126.3,120.5,116.1,112,107.3,107.8,103.2,103.6,101.4,99.86,98.44,97.12,99.69,94.53,93.92,93.62,94.12,94.38,91.42,91.41,90.53,90.39,90.43,89.52,89.59,88.51,88.24,88.28,87.32,88.28,90.27,82.52,82.06,81.77,81.52,81.32,81.15,80.99,80.85,80.7,80.56,80.45,80.33,80.22,80.11,80.01,79.93,79.85,79.77,79.7,79.62,79.55,79.47,79.41,79.36
74 | lstm,penntreebank,2,510,0.1,15000,299,196.5,161,142.1,131.6,121.8,114.8,111,106.5,102.9,100.4,97.91,96.39,95.53,93.6,93.04,93.65,91.76,89.75,88.44,90.39,88.59,86.98,86.42,86.33,85.37,85.97,86.83,84.64,84.23,83.97,83.58,83.88,82.65,82.11,83.12,82.98,81.65,81.83,81.79,82.02,80.5,81.31,80.8,80.98,80.9,79.9,79.63,81.05,75.72,75.32,75.05,74.85,74.72,74.59,74.49,74.4,74.31,74.21,74.13
75 | lstm,penntreebank,2,750,0.1,15000,290.2,199.1,158.4,142.4,128.9,118.5,112.2,106,102.4,103.3,99.24,96.77,95.84,93.14,91.26,90.05,88.89,88.46,87.3,86.26,87.44,85.75,84.34,84.41,84.39,83.93,83.46,82.51,81.6,82.02,81.09,80.8,81.51,80.86,80.86,81.39,80.19,79.83,79.04,78.95,78.83,78.65,78.55,78.43,78.69,78.97,74.06,73.66,73.37,73.18,73,72.87,72.75,72.63,72.52,72.42,72.33,72.26,72.18,72.09
76 | lstm,penntreebank,2,990,0.1,15000,286.1,195.5,155.9,137.8,126.7,117.5,110.9,105,100.8,97.22,96.33,95.46,93.41,89.9,88.42,88.76,87.25,86.5,86.14,85.02,85.57,83.06,82.57,81.95,81.6,81.25,82.35,80.64,79.34,80.17,79.84,79.71,79.21,78.13,78.21,78.68,78.62,77.59,77.55,77.41,76.78,76.39,77.07,76.43,77.36,76.24,75.83,75.9,75.85,75.83,76.35,75.99,76.77,71.29,70.9,70.61,70.39,70.24,70.1,69.99
77 | lstm,penntreebank,2,1230,0.1,15000,289.1,209.1,157.6,140.5,126.4,115.2,110.7,104.7,101.3,97.54,95.25,91.87,91.25,88.64,87.2,86.73,85.93,85.15,84.78,84.38,83.65,81.51,81.89,80.6,80.63,81.69,80.66,79.73,79.35,78.57,78.47,78.6,78.76,77.9,77.24,77.94,77.77,76.69,76.99,76.92,76.92,75.68,75.97,75.59,75.8,75.65,75.05,74.88,75.48,75.22,75.26,75,75.17,70.82,70.41,70.2,70,69.84,69.71,69.58
78 | lstm,penntreebank,3,70,0.1,15000,369.3,258.3,201.3,182.9,172.3,158.5,154.1,140.9,139.7,133.1,132.7,126.9,124.4,123.8,121.6,120.3,121,120,115.7,115.3,118.1,115.4,112.3,112.9,111.1,110.5,109.8,112.4,110.1,108.1,107.5,109.3,111.2,102.1,101.6,101.3,101.1,100.8,100.6,100.4,100.3,100.1,99.92,99.77,99.62,99.48,99.34,99.21,99.09,98.97,98.85,98.75,98.66,98.56,98.47,98.39,98.3,98.23,98.15,98.06
79 | lstm,penntreebank,3,310,0.1,15000,385.9,228.8,172.2,157.5,146.7,133.2,126.4,118.3,112.9,110.3,108.4,104.1,102.5,100.6,100.3,98.01,97.95,98.45,94.04,93.05,95.23,92.55,91.21,90.83,90.04,89.68,89.69,89.61,88.08,88.1,87.46,87.5,87.17,86.47,86.78,87.25,85.39,84.9,85.4,85.7,85.14,84.93,84.23,83.23,83.63,84.06,83.29,82.89,82.84,82.39,85,78.4,77.98,77.71,77.51,77.37,77.22,77.09,76.98,76.88
80 | lstm,penntreebank,3,550,0.1,15000,329.1,218,172.4,149.2,137.6,124.9,117.1,111.6,106.3,103.2,101.2,97.97,96.51,93.96,92.84,91.87,90.52,91.75,87.83,88.76,88.66,87.22,85.99,85.58,84.21,83.34,83.69,84.19,82.04,81.21,80.95,80.54,80.63,79.97,80.37,80.1,79.56,78.68,78.88,78.95,78.65,77.78,78.41,77.94,78.22,79.55,73.45,73.03,72.76,72.56,72.4,72.27,72.16,72.05,71.95,71.84,71.73,71.63,71.55,71.47
81 | lstm,penntreebank,3,790,0.1,15000,331.1,210.6,171.3,149.7,135,123.1,114.8,107.1,102.7,102.9,98.57,94.46,91.88,90.74,89.41,88.71,86.82,86.03,84.45,83.86,84.68,83.67,81.96,81.04,80.71,79.77,79.85,79.41,79.23,78.68,78.11,77.59,78.54,77.05,76.98,77.58,76.69,76.57,76.45,75.59,75.78,75.96,76.18,76.36,74.52,74.5,74,74.34,74.28,74.15,74.11,73.32,74.12,69.38,69,68.76,68.59,68.45,68.32,68.21
82 | lstm,penntreebank,3,1030,0.1,15000,347.7,210.6,175.2,147.6,133,122.4,114,107,102.6,98.55,96.57,94.23,92.23,89.91,88.12,86.75,85.44,84.79,83.89,82.5,83.09,81.85,80.47,79.75,79.25,78.28,78.46,77.89,78,78.3,76.65,76.62,76.7,75.59,75.86,75.73,75.75,74.51,74.4,74.2,74.11,73.41,73.85,73.1,73.38,72.89,72.53,72.49,72.51,72.55,72.57,73.04,68.31,67.99,67.77,67.58,67.42,67.29,67.18,67.07
83 | lstm,penntreebank,3,1270,0.1,15000,356.4,218.3,164.8,146.4,130.4,119.9,112.7,106.2,105.8,98.42,95.1,92.24,89.85,87.98,85.54,85.41,84.57,83.45,81.15,80.85,81.19,80.08,78.59,78.28,77.26,76.73,77.06,77.02,75.8,75.51,75.18,74.3,75.32,74.26,73.9,74.61,73.89,73.14,73.12,72.71,72.17,72,71.52,72.03,72.13,72.18,71.18,70.72,71.19,70.44,71.06,70.59,70.7,70.89,66.68,66.32,66.05,65.89,65.75,65.63
84 | lstm,penntreebank,4,110,0.1,15000,513,311.3,246,209.2,180.9,169.3,160.1,147.7,141.2,137.8,136.5,129.2,127.8,124,120.7,122,120.6,120.2,115.9,114.5,119.4,113.7,113,112,111.2,109.7,114.8,103.6,102.8,102.5,102.1,101.7,101.4,101.1,100.8,100.6,100.3,100.1,99.87,99.66,99.46,99.26,99.07,98.88,98.71,98.55,98.39,98.24,98.09,97.94,97.8,97.66,97.53,97.4,97.28,97.16,97.04,96.92,96.82,96.72
85 | lstm,penntreebank,4,350,0.1,15000,539.1,274.6,208.5,180.5,164.4,146,137.3,127,120,116.8,114,108.8,107.1,104,103.2,100.6,101.4,99.06,95.97,95.72,96.85,97.76,94.45,92.51,91.96,91.2,90.4,91.32,89.98,90.43,88.73,88.58,88.19,87.25,88.71,88.1,86.8,87.84,85.81,85.67,86.23,84.52,85.77,86.92,80.12,79.73,79.43,79.19,79,78.82,78.65,78.51,78.39,78.3,78.19,78.11,78.03,77.94,77.84,77.75
86 | lstm,penntreebank,4,590,0.1,15000,443.4,261.8,196.4,171.2,149.2,138.6,129.8,120.1,112.7,109.2,107.9,102.9,100.2,98.9,95.12,95.03,93.96,92,90.52,89.5,90.74,89.44,88.03,86.21,85.75,84.54,84.06,85.46,82.98,83.35,82.75,82.13,83.16,81.81,81.19,82.73,80.53,80.28,79.88,80.35,79.43,78.67,78.9,78.79,78.61,78.32,77.92,77.57,77.49,77.14,78.55,77.01,77.04,77,76.61,76.02,77.11,75.87,77.01,76.24
87 | lstm,penntreebank,4,830,0.1,15000,452.3,266.4,214.7,168.8,156,134.9,124.7,116.6,110,106.5,104.9,99.73,97.42,94.46,92.82,91.94,89.59,89.25,87.18,87.6,86.16,84.95,84.16,82.94,82.59,82.4,81.02,82.02,79.78,79.73,79.31,78.91,79.03,78.55,78.04,78.5,78.02,77.25,76.47,76.28,75.93,75.46,75.81,75.4,75.12,74.76,74.62,74.46,74.02,75.14,74.49,73.55,74.35,73.78,73.47,73.12,73.44,73.25,73.37,73.53
88 | lstm,penntreebank,4,1070,0.1,15000,445.8,256,200.8,171.2,144.2,134.9,125.9,116.7,109.2,109.1,102.5,98.42,97.48,93.92,91.88,89.71,89.25,87.8,85.95,84.69,84.9,83.44,82.73,81.76,81.48,79.27,78.93,79.06,77.58,77.48,77.49,76.24,76.92,75.61,76.13,76.38,75.85,74.11,74.76,74.04,73.85,72.93,73.47,73.03,73.6,72.97,72.21,72.56,71.89,72.94,68.05,67.65,67.4,67.21,67.06,66.91,66.77,66.63,66.52,66.42
89 | lstm,penntreebank,4,1310,0.1,15000,444.1,262.4,191.5,165.3,145.6,132.4,121.6,113,107.2,108.3,100.9,95.35,93.59,91.56,89.6,88.69,86.43,85.58,86.16,83.86,83.37,82.94,80.39,80.62,79.57,78.26,78.15,77.35,77.15,76.7,76.12,75.79,76.13,74.63,74.44,74.89,73.66,73.66,73.54,73.26,72.9,72.65,72.89,72.34,72.94,71.76,71.59,71.31,71.26,70.96,71.97,70.68,71.52,71.09,70.77,70.23,70.49,70.37,70.13,70.06
90 | lstm,penntreebank,5,150,0.1,15000,724.2,700,544.3,394.2,283.5,238,208,180.1,166,163.7,153.3,143.3,138.3,136.2,130.9,129.9,131.4,125.4,124.1,119.9,122.7,120.5,116.3,117.2,115.1,112.8,115.1,113.2,111.2,111.3,110.6,108.7,111.8,107.5,107.9,111.1,106.8,106.9,106.4,106.4,106.2,104.6,104.9,103.2,104.8,105,103.2,102.5,102.1,103,104,96.09,95.75,95.45,95.18,95,94.84,94.67,94.54,94.44
91 | lstm,penntreebank,5,390,0.1,15000,741.6,700.1,690.7,488.2,308.7,241.5,199.3,172.5,150.7,142.3,140.1,128.4,125.3,119,116.1,114.9,111.3,110,106.4,103.4,104.9,102.1,100.9,99.03,96.82,95.86,95.38,94.97,94.13,93.76,92.84,91.66,92.45,91.85,91.21,90.83,89.68,89.05,89.25,88.65,88.88,87.54,87.28,86.68,86.98,87.12,85.46,85.24,84.96,85.4,85.15,85.01,84.82,85.09,84.42,83.49,83.86,84.15,84.14,83.2
92 | lstm,penntreebank,5,630,0.1,15000,728.8,527.8,312.7,239.2,190.6,172.6,152.3,139.6,127.1,121.7,117.4,111.5,108.5,104.9,101.5,100.8,99.1,98.96,94.68,93.16,95.35,92.8,90.5,90.74,89.07,87.56,87.59,87.84,85.75,86.18,84.44,84.44,84.43,83.91,83.17,83.64,83,82.56,82.28,82,81.77,80.41,80.61,81.67,80.42,79.59,78.94,79.24,79.16,80.35,79.23,78.34,78.42,79.03,73.92,73.57,73.29,73.03,72.85,72.72
93 | lstm,penntreebank,5,870,0.1,15000,727.7,562,346.1,238.8,186.1,170.4,152.1,139,127.6,121.4,117.6,110.3,107.1,104.2,100.5,98.53,98.06,95.45,92.66,91.77,93.97,90.42,89.69,87.8,86.37,85.33,84.99,84.32,83.13,83.27,81.78,82.37,82.08,80.57,81.14,81.12,79.35,78.83,79.68,78.47,78.64,77.7,77.8,77.41,77.62,76.65,76.58,75.98,76.27,76.05,76.63,75.68,75.39,75.68,75.58,74.79,74.79,74.77,74.6,74.45
94 | lstm,penntreebank,5,1110,0.1,15000,726,412.6,298,216,180.8,159.2,145.7,130.6,123.7,117.2,112,107.6,104.1,100.6,98.57,97.89,94,93.01,90.07,88.77,89.43,87.78,87.46,85.15,85.23,82.73,82.9,83.62,80.95,80.58,79.68,79.38,79.96,78.42,78.56,79.91,77.77,77.16,77.04,76.55,77.28,75.48,76.28,74.99,75.64,75.04,74.81,74.69,74.43,74.13,75.12,70.17,69.72,69.42,69.18,69,68.83,68.68,68.53,68.38
95 | lstm,penntreebank,5,1350,0.1,15000,772.4,384.5,266.6,211.4,172.5,153.2,140.7,128.7,120.4,118.4,110.9,106.5,103.3,100.2,97.3,94.55,93.01,91.6,88.83,88.29,88.87,85.94,84.68,84,83.33,81.47,81.83,81,80,79.52,78.52,78.46,78.21,77.45,77.68,77.61,76.75,76.12,76.14,75.8,75.62,74.95,74.94,74.68,74.97,74.35,74.19,73.54,73.37,73.36,74.23,72.74,72.72,73.71,69.15,68.69,68.41,68.23,68.06,67.94
96 | lstm,penntreebank,2,130,0.1,15000,304.9,212.1,168.4,159,148.6,134.1,128,120.8,119,114.6,113.5,109.6,107,106.1,104.1,104.1,103.1,103.9,99.87,101.3,101.7,99.74,98.07,97.42,96.68,95.82,95.12,96.71,94.89,95.07,93.74,94.27,94.46,92.33,93.14,94.99,87.89,87.53,87.22,87.02,86.84,86.68,86.52,86.38,86.26,86.15,86.02,85.9,85.79,85.69,85.59,85.51,85.43,85.36,85.29,85.21,85.15,85.08,85.02,84.95
97 | lstm,penntreebank,2,370,0.1,15000,281.3,194.2,158.5,148.2,134,124.2,117.2,111.9,106.2,106.2,102.1,99.84,99.09,96.97,95.79,93.97,93.97,93.04,90.27,90.14,92.01,90.25,90.98,87.87,87.58,86.85,87.34,86.47,86.11,85.68,86.81,85.42,86.12,84.01,84.16,85.36,84.17,83.94,83.57,84.11,78.85,78.45,78.2,77.97,77.81,77.66,77.56,77.43,77.31,77.2,77.1,77,76.92,76.84,76.76,76.68,76.61,76.54,76.47,76.41
98 | lstm,penntreebank,2,610,0.1,15000,287.2,194.2,157.2,140.7,127.4,119,113.5,109.3,105.1,104.9,99.56,96.83,96,93.3,91.66,91.77,90.75,89,88.09,87.08,90.63,86.79,86.28,84.88,84.55,83.77,83.38,84.31,83.33,82.63,81.92,81.66,82.68,81.38,81.29,82.55,81.97,76.42,75.96,75.7,75.48,75.31,75.18,75.06,74.94,74.83,74.74,74.63,74.52,74.41,74.31,74.2,74.11,74.03,73.95,73.87,73.8,73.73,73.66,73.59
99 | lstm,penntreebank,2,850,0.1,15000,278.6,189.5,161.4,139.7,126.1,118.4,112.4,106.1,101.1,102.9,98.24,93.87,92.82,92.06,92.03,89.46,89.59,87.48,86.23,86.51,86.69,84.7,84.24,83.1,82.4,82.53,82.09,81.82,80.88,80.79,81.12,81.17,81.39,80.99,79.65,80.98,75.13,74.67,74.38,74.12,73.9,73.71,73.55,73.41,73.27,73.14,73.01,72.9,72.79,72.71,72.61,72.53,72.44,72.36,72.27,72.19,72.12,72.04,71.97,71.9
100 | lstm,penntreebank,2,1090,0.1,15000,290.4,197.1,156.7,141.4,126.6,117,109.7,103.5,99.65,97.51,95.9,92.98,90.82,90.67,88.16,87.19,87,85.44,84.58,83.75,84.94,82.53,81.89,82.57,80.91,80.39,80.21,80.6,79.32,79.05,78.75,78.53,79.4,78.64,77.84,78.42,78.86,73.39,72.94,72.66,72.45,72.28,72.11,71.94,71.79,71.67,71.56,71.47,71.37,71.27,71.17,71.08,71,70.91,70.84,70.77,70.71,70.64,70.57,70.51
101 | lstm,penntreebank,2,1330,0.1,15000,299.7,190.8,155.3,137.3,124.3,116.8,109,103.4,98.67,96.37,94.31,92.25,91.21,89.36,86.81,87.25,85.55,85.32,83.3,83.1,83.41,82.61,81.72,80.65,80.56,79.34,79.74,79.65,78.97,78.77,78.41,77.73,78.08,77.51,77.77,77.71,77.16,76.16,76.44,75.83,75.79,75.75,75.74,74.91,76.42,71.33,70.94,70.71,70.5,70.32,70.17,70.04,69.94,69.82,69.7,69.61,69.52,69.42,69.34,69.26
102 | lstm,penntreebank,3,170,0.1,15000,384.9,221.6,188.3,164.2,149.1,140.6,135,127.2,121.8,121.3,116,112.9,111.5,110,106.6,108.1,105.3,107.2,101.9,101.5,102,101.1,99.75,99.95,98.21,97.21,96.7,96.64,96.16,95.95,95.74,95.31,96.34,94.36,94.76,94.97,93.98,93.47,92.95,94.82,87.66,87.21,86.85,86.61,86.4,86.24,86.12,86,85.89,85.77,85.66,85.56,85.45,85.35,85.26,85.17,85.09,85,84.92,84.85
103 | lstm,penntreebank,3,410,0.1,15000,338,228.1,170.2,152.8,140.1,128.6,121.4,114.5,110.4,106.2,104.4,101,98.73,98.89,95.52,95.3,94.93,93.77,92.14,89.61,91.83,89.59,87.75,88.08,86.46,86.08,85.81,86.67,84.95,84.5,84.37,83.57,84.85,83.71,83.23,84.05,84.4,78.11,77.74,77.42,77.15,76.96,76.8,76.65,76.51,76.38,76.25,76.13,76.02,75.92,75.82,75.72,75.62,75.52,75.43,75.34,75.26,75.17,75.09,75.02
104 | lstm,penntreebank,3,650,0.1,15000,344.5,221.9,168.2,148.7,138.1,123.6,116.9,111.1,104.4,101,100.6,96.28,95.23,91.69,90.88,90.19,88.58,87.86,86.86,85.4,86.36,85.4,84.03,83.01,82.16,81.97,81.2,81.31,80.69,79.78,79.56,79.04,79.8,78.49,78.49,79.08,78.46,77.23,77.34,76.62,76.75,76.44,76.22,76.09,76.09,75.94,76.7,71.67,71.3,71.06,70.89,70.74,70.6,70.46,70.35,70.24,70.15,70.05,69.95,69.86
105 | lstm,penntreebank,3,890,0.1,15000,342.5,211.9,164.6,148.2,132.3,121.9,116.7,107.8,102.3,100.1,97.5,94.57,92.73,92.58,88.34,88.26,87.19,86.71,83.93,83.32,84.38,82.81,82.03,80.6,80.56,79.44,79.59,79.16,78.29,78.71,77.72,77.14,78.39,76.37,76.72,77.14,76.83,75.82,75.94,75.2,75.08,75.57,74.26,75.11,74.79,74.15,73.63,73.37,73.06,73.51,74.2,72.8,72.99,73.13,72.44,73.02,72.27,71.93,73.35,68.15
106 | lstm,penntreebank,3,1130,0.1,15000,338.2,218.1,167.7,148.2,132.9,121.4,113,106.5,101.4,98.33,95.72,93.06,90.71,90.26,87.83,85.89,84.63,84.77,82.34,81.25,81.37,80.67,80.82,79.11,79.04,77.64,78.19,77.25,76.45,76.15,75.62,75.4,75.15,75.32,76.1,75.73,74.75,73.84,73.41,73.97,73.34,73.52,72.55,72.51,72.78,72.56,72.08,71.95,72.01,71.5,71.98,71.16,71.58,71.65,71.15,71.11,70.81,70.23,70.83,70.73
107 | lstm,penntreebank,3,1370,0.1,15000,329.5,222.7,171.9,149.7,134.1,122.2,114.2,106.7,102.3,100.9,95.69,91.43,89.52,88,86.27,85.6,84.54,82.98,81.76,81.17,80,80.05,79.33,78,77.36,77.02,77.04,76.7,75.53,75.28,74.44,74.54,75.12,74.15,73.94,74.3,73.16,72.83,72.73,72.22,72.43,72.28,72.06,71.45,71.56,71.26,70.89,70.98,71.02,71.16,71.54,66.9,66.55,66.32,66.15,66.01,65.86,65.74,65.62,65.52
108 | lstm,penntreebank,4,210,0.1,15000,544,292.6,225.2,196,166.1,154.2,147.4,136.1,129.4,124.4,123.7,117.3,116.6,113.7,115.2,110.7,108.7,107.4,104.4,104.8,104.9,103.3,100.8,101.7,99.12,98.37,98.47,97.96,97.8,97.38,96.49,95.53,96.46,95.08,96.31,96.04,94.62,94.6,93.14,95.12,88.32,87.97,87.7,87.46,87.27,87.12,86.97,86.81,86.65,86.5,86.35,86.21,86.09,85.97,85.86,85.75,85.65,85.54,85.45,85.36
109 | lstm,penntreebank,4,450,0.1,15000,526.9,285.5,205.1,180.4,152.8,144.3,134.4,123.2,117.4,111.9,111.2,106.2,104.1,102.1,99.54,99.64,97.68,96.7,95.45,93.58,95.44,92.82,90.37,90.13,88.5,88.45,87.83,88.66,87.11,85.51,85.89,85.92,86.21,85.24,85.3,86.74,79.58,79.12,78.84,78.58,78.31,78.11,77.92,77.75,77.6,77.45,77.3,77.15,77.03,76.89,76.76,76.63,76.51,76.39,76.28,76.17,76.06,75.95,75.85,75.75
110 | lstm,penntreebank,4,690,0.1,15000,442.2,273.8,197.3,169.9,148.8,137.7,127.4,117.2,112,109.5,104.7,101.4,97.97,97.08,93.38,92.73,91.66,91.47,88.44,88.1,88.96,86.93,85.3,86.02,83.61,82.65,82.94,82.42,81.62,81.26,80.66,80.56,81.46,79.75,79.77,79.92,78.97,79.19,78.29,77.96,77.81,78.43,77.33,76.49,77.46,77.05,76.29,76.04,75.47,76.06,76.23,74.93,75.14,75.51,75.2,74.25,74.35,73.74,74.59,74.01
111 | lstm,penntreebank,4,930,0.1,15000,465.9,260.9,195.4,169.8,148,134.8,124.8,116.4,110.8,109.7,103.4,98.78,96.31,93.66,92.24,91.19,89.59,88.82,86.55,86.42,86.34,84.57,83.79,82.6,81.53,81.03,80.63,79.84,79.48,78.93,78.51,77.53,78.27,76.91,77.57,77.31,76.83,76.14,76.13,76.07,75.47,75.05,74.83,74.54,74.97,73.73,73.56,73.19,73.06,73.24,73.76,72.9,72.58,72.89,72.47,72.03,72.2,72.45,72.19,72.04
112 | lstm,penntreebank,4,1170,0.1,15000,446.2,262.3,213.7,166.8,143.2,132.9,124.8,114.5,109,103.5,101.8,97.23,94.6,92.58,90.4,88.87,90.8,86.29,85.26,84.12,84.02,83.18,81.37,80.44,80.01,79.4,78.91,79.27,77.81,77.63,76.75,76.98,76.84,75.58,75.66,75.91,75.44,74.33,74.39,74.02,73.35,73.38,73.29,73.24,73.02,73.05,72.73,71.75,71.45,71.72,72.69,71.55,71.48,71.3,70.7,70.63,70.89,70.53,70.84,70.31
113 | lstm,penntreebank,5,10,0.1,15000,723,699.1,691.9,711.7,702.9,707.4,719,697.8,696.6,680,680.1,680.1,680.2,680.3,680.3,680.3,680.4,680.4,680.5,680.5,680.5,680.5,680.6,680.6,680.6,680.6,680.6,680.6,680.7,680.7,680.7,680.7,680.8,680.8,680.8,680.8,680.9,680.9,681,681.1,681.1,681.1,681.2,681.2,681.2,681.2,681.2,681.2,681.2,681.2,681.3,681.3,681.3,681.3,681.3,681.3,681.3,681.3,681.3,681.3
114 | lstm,penntreebank,5,250,0.1,15000,725.5,700.1,655.8,462.4,295.1,255.9,211.2,178.2,161.1,150.8,147.1,136.6,132.6,127,123.5,122.7,119.9,118.3,114.5,111.3,113.9,110.7,109.2,107.5,105.2,103.4,102.2,101.9,101.5,100.5,99.72,98.59,100.2,97.67,97.96,98.73,98.39,96.7,96.81,95.47,97.2,94.24,95.49,94.31,94.12,93.62,93.39,92.79,92.28,92.34,92.88,91.6,92.36,91.29,91.5,90.57,92.01,92.1,85.85,85.48
115 | lstm,penntreebank,5,490,0.1,15000,742.5,552.3,329.4,254.1,203.5,181.7,164.7,146.7,133.7,127.6,124.8,117.5,115,110.7,107.6,106.3,104.2,102.8,99.41,98.54,98.42,96.57,95.06,94.27,92.2,90.97,90.35,90.43,89.39,88.64,88.71,87.54,88.22,87.71,86.69,87.63,85.57,86.36,85.44,84.61,84.34,84.13,84.49,83.75,83.14,83.68,82.26,81.98,81.85,82.26,82.77,81.77,81.5,81.81,82.06,76.34,75.89,75.61,75.41,75.26
116 | lstm,penntreebank,5,730,0.1,15000,739,434.3,281.3,229.2,184.2,163.6,147.3,135.7,126.6,122.1,119.2,111.1,107.7,104.4,101,100.4,99.4,99.04,93.84,93.58,95.15,90.22,89.54,88.75,87.9,86.79,86.29,85.73,84.89,84.61,83.62,82.85,82.67,82.01,81.8,81.58,81.68,81.34,81.73,79.99,79.91,79.94,79.19,79.09,79.25,78.54,78.3,78.08,77.77,77.48,77.27,76.99,76.77,77.41,76.3,76.63,75.96,76.96,76.27,75.71
117 | lstm,penntreebank,5,970,0.1,15000,715.5,445.1,290,231.7,187.2,165.6,151,136.4,124.8,118.8,115.8,112.8,105.8,101.8,99.47,98.34,96.46,94.25,91.98,89.79,90.28,89.21,87.26,85.69,85.09,83.61,83.63,83.11,82.11,81.81,82.55,80.31,81.1,80.17,79.82,79.91,79.01,78.65,78.13,77.58,77.97,77.68,77.1,76.73,76.57,76.21,75.8,74.95,75.11,75.39,75.15,74.52,74.47,74.62,74.92,73.75,73.95,73.44,73.65,73.12
118 | lstm,penntreebank,5,1210,0.1,15000,688.2,384.1,274.9,219.8,179.5,159.9,145.9,133.6,124.1,117.7,111.8,107.3,104.3,100.5,96.87,96.2,95.81,92.84,90.09,88.61,89.45,87.36,86.08,84.63,83.34,82.75,81.95,81.62,81.3,80.18,79.78,79.85,79.42,78.88,78.8,78.56,77.89,76.94,77.86,76.62,76.59,75.16,76.23,75.2,74.85,74.6,74.37,74.18,74.21,73.75,74.61,73.7,73.08,73.85,72.69,72.83,72.83,72.53,72.83,73
119 | lstm,penntreebank,2,50,0.1,15000,297.2,214.9,180.3,169.4,153.8,146.5,138.6,136.4,127.7,128.7,123.6,118.8,118.1,115.5,114,114.4,113.7,110.5,109.4,109.1,118.6,103.2,102.5,101.9,101.5,101.1,100.8,100.6,100.3,100,99.82,99.61,99.41,99.22,99.03,98.84,98.66,98.49,98.32,98.15,98,97.85,97.71,97.57,97.43,97.3,97.16,97.03,96.89,96.77,96.65,96.53,96.42,96.31,96.21,96.11,96.01,95.92,95.82,95.73
120 | lstm,penntreebank,2,290,0.1,15000,285.6,198.3,160,147.4,134.1,128.2,120,112.5,108.4,105.7,105.7,101.9,100.8,99.02,97.66,97.12,96.02,97.35,93.47,92.98,94.89,92.69,91.62,90.52,89.92,89.63,90.3,89.55,88.06,87.67,87.58,87.7,87.57,88.14,86.75,87.48,86.47,85.86,85.86,85.85,85.74,84.37,84.45,84.18,84.49,84.3,84.34,84.18,83.91,83.96,84.59,79.51,79.15,78.88,78.69,78.55,78.43,78.3,78.2,78.1
121 | lstm,penntreebank,2,530,0.1,15000,306.2,191.6,158.8,142.6,127.7,121.6,116.9,108.1,103.9,101.2,100.5,97.08,96.13,94.22,94.32,91.77,91.03,90.15,88.6,88.21,90.55,87.89,86.63,86.38,85.09,84.52,84.63,85.73,82.86,83.54,83.36,82.18,83.13,82.34,82.34,82.34,83.86,77.12,76.74,76.47,76.26,76.08,75.91,75.72,75.56,75.43,75.32,75.22,75.12,75.02,74.92,74.83,74.74,74.66,74.59,74.51,74.45,74.38,74.3,74.23
122 | lstm,penntreebank,2,770,0.1,15000,291,201.2,157.1,140.1,129.5,117.6,114.7,105.7,102.9,99.33,99.46,95.17,94.95,91.07,90.43,89.41,88.85,87.19,86.29,85.69,88.05,85.12,84.2,83.67,84.15,82.52,82.6,82.04,81.61,81.16,80.78,80.11,81.5,80.09,79.81,80.36,79.85,79.19,79.64,78.36,79.01,78.25,77.92,78.28,79.09,79.17,73.83,73.41,73.14,72.91,72.74,72.6,72.48,72.37,72.27,72.17,72.08,72,71.92,71.85
123 | lstm,penntreebank,2,1010,0.1,15000,282.3,196.6,159.8,139.8,128.3,115.8,111.7,104.5,100.6,98.51,94.9,93.11,91.4,90.39,88.28,88.87,86.73,86.11,85.92,84.05,85.1,84.39,83.56,81.52,81.99,81.29,81.17,82.43,80.3,79.77,79.45,78.97,79.08,78.3,77.92,78.41,78.37,77.61,78.43,76.85,76.5,76.58,77.02,76.31,76.83,76.33,75.96,75.65,76.25,76.36,71.55,71.21,70.97,70.79,70.62,70.48,70.35,70.23,70.12,70.03
124 | lstm,penntreebank,2,1250,0.1,15000,313.4,202.7,159.5,141.5,128.2,117.8,111.4,103.9,100.7,96.86,96.4,92.53,90.64,89.67,87.6,87.89,86.17,85.84,83.75,83.35,86.34,82.32,82.28,81.63,80.38,79.73,81.52,80.65,78.46,79.17,78.56,78.05,77.86,77.47,77.73,78.14,77.27,77.01,76.39,76.64,76.34,75.81,75.62,75.57,75.53,75.26,75.15,75,75.4,74.97,75.56,70.77,70.36,70.12,69.96,69.81,69.67,69.54,69.42,69.31
125 | lstm,penntreebank,3,90,0.1,15000,376.5,237.6,192.6,180.9,162.8,151.7,147.4,135,134.1,127.1,126.4,121.9,118.8,117.1,116.3,116.5,115.9,115.2,110.8,110.6,111.2,110.8,108.3,107.4,106.2,106.5,105.3,107.9,105.3,104.4,103.9,104.7,104.9,103.5,103.6,104,103,103.3,102.4,101.7,101.8,100.1,100.4,101.3,101.9,101.2,99.38,99.28,98.58,100.3,94.04,93.6,93.32,93.14,92.99,92.84,92.7,92.59,92.48,92.4
126 | lstm,penntreebank,3,330,0.1,15000,369.4,220,174,159.6,139.8,133,125.3,116.8,113.8,108.9,109.4,103.7,101.7,100.2,98.21,98.24,97.59,95.71,92.78,93.29,95.84,92.73,91.41,90.38,89.54,88.87,89.59,88.54,88.15,88.02,86.51,86.24,87.7,85.44,86.04,87.31,84.68,84.8,84.71,84.03,83.95,83.09,83.22,82.44,83.56,83.47,82.49,82.94,81.92,81.84,82.49,77.44,77.12,76.93,76.77,76.62,76.51,76.4,76.32,76.23
127 | lstm,penntreebank,3,570,0.1,15000,361.5,213.7,169.9,151.6,135.7,126.4,117.4,111.1,106.6,106.5,102.5,98.6,96.97,94.09,92.76,91.58,90.81,89.33,88.25,89,89.11,86.59,90.02,85.1,83.82,83.33,83.02,83.54,81.71,81.8,81.23,80.47,81.15,79.69,80.58,80.82,79,79.2,78.83,77.82,77.77,77.85,77.66,78.78,77.09,77.49,76.41,76.39,76.35,76.85,77.46,72.48,72.16,71.93,71.75,71.61,71.46,71.34,71.25,71.17
128 | lstm,penntreebank,3,810,0.1,15000,361.9,204.2,168.6,146.4,132.4,121.2,114.1,107.9,102,99.16,97.81,94.26,93.17,91.18,88.78,88.17,86.62,85.58,84.01,83.4,85.04,82.8,82.96,81.43,80.26,80.33,79.56,79.43,78.97,78.08,77.74,77.38,77.69,76.51,77.31,76.8,77.25,75.74,75.33,75.57,75.24,74.86,75,74.64,75.21,74.08,74.22,73.91,73.63,73.59,73.62,73.48,73.39,73.3,73.3,72.78,72.75,73.16,72.72,72.16
129 | lstm,penntreebank,3,1050,0.1,15000,323.6,210.8,166.8,144.7,131.9,120.2,114.1,106.5,101.3,97.71,96.14,92.95,90.64,88.57,86.97,87.62,86.65,84.62,82.88,81.91,82.79,80.95,80.82,79.7,78.97,78.54,77.64,77.46,77.58,76.75,76.19,76.01,76.19,74.89,74.92,75.11,74.63,74.28,73.86,74.05,74.27,73.31,73.07,73.12,73.12,72.84,72.28,72.2,71.99,72.24,72.11,71.5,71.75,71.86,71.86,70.99,71.33,71.2,70.61,70.72
130 | lstm,penntreebank,3,1290,0.1,15000,344.6,213.6,166,149.4,132.2,121.4,113,106.7,101,101.4,96.07,92.14,90.54,88.51,86.77,85.92,84.89,83.6,81.7,85.01,81.57,79.76,78.89,78.67,78.76,77.22,76.97,77,75.73,75.14,74.92,74,74.45,73.69,73.94,73.94,73.4,72.77,72.76,72.65,72.74,71.71,71.84,71.56,72.24,71.59,71.03,71.15,70.91,71.11,71,70.62,70.85,70.25,70.54,70.02,69.84,69.91,69.61,69.92
131 | lstm,penntreebank,4,130,0.1,15000,498.1,304,237.9,199.3,176.9,166.4,155.8,145.3,137.7,136,131.4,126.1,124.7,121.8,120.9,119.5,117.3,117.7,113,111.5,111.3,111.7,111.6,108.8,107.4,106.1,106.2,106.6,105.2,104.5,103.5,104.4,104.6,103.6,103.9,102.2,103.5,96.24,95.84,95.54,95.29,95.13,94.95,94.8,94.65,94.5,94.36,94.21,94.07,93.95,93.82,93.7,93.59,93.48,93.37,93.27,93.17,93.07,92.98,92.89
132 | lstm,penntreebank,4,370,0.1,15000,509.5,282.8,220,177.5,157.1,144.6,133.5,125.8,119.5,114.4,113.1,112.3,106.5,103.1,102.6,100.8,99.43,99.2,96.31,95.37,96.72,93.26,91.64,92.22,90.54,90.45,90.26,89.17,89.59,87.86,86.9,87.31,89.01,85.84,86.17,86.84,86.7,86.12,85.04,84.8,85.3,83.63,83.9,84.18,83.92,83.21,82.41,82.5,82.6,81.79,83.79,77.99,77.62,77.35,77.14,76.99,76.87,76.78,76.67,76.58
133 | lstm,penntreebank,4,610,0.1,15000,496,262.6,202.4,169.5,148.4,137.1,129.4,119.3,113.4,110.2,106.9,102.4,100.4,97.62,95.65,95.58,93,94.02,90.04,88.94,90.61,89.07,86.46,85.98,84.9,84.56,83.99,83.62,83.02,82.65,82.64,81.68,82.15,80.86,81.17,80.96,80.9,79.58,79.97,79.26,79.07,78.37,78.69,77.95,77.9,77.53,77.14,77.01,77.1,77.46,77.04,77.2,76.81,76.43,77.53,71.92,71.56,71.31,71.13,70.98
134 | lstm,penntreebank,4,850,0.1,15000,483.5,278.7,196.7,169.9,146.1,136.2,125.4,116.7,110.6,106.7,104.2,99.54,97.79,94.7,96.84,91.28,89.97,89.37,87.22,86.49,87.94,84.99,84.42,83.02,82.03,81.6,81.35,81.43,80.19,79.81,78.93,78.55,79.48,78.06,77.64,78.68,77.65,76.91,76.56,76.14,76.72,75.74,76.04,75.37,75.86,75.28,74.54,74.33,74.12,73.91,74.53,73.66,74.02,73.38,73.13,72.58,73.33,73.04,73.4,72.76
135 | lstm,penntreebank,4,1090,0.1,15000,462.3,268.7,197.7,168.5,146.8,133.2,123.8,115.5,110.1,105.6,102,98.23,95.96,93.66,91,90.27,88.64,88.42,85.48,84.08,84.57,83.95,82.46,80.98,80.62,79.82,79.49,78.68,78.11,77.74,77.17,76.96,76.9,76.32,76.4,75.84,75.51,74.92,74.65,74.24,73.71,73.6,73.45,74.01,73.8,72.94,72.76,72.46,72.94,72.34,72.56,71.9,71.87,72.06,71.99,71.36,71.41,71.15,71.06,70.55
136 | lstm,penntreebank,4,1330,0.1,15000,491.5,252.5,188.5,165.1,148.6,133.9,123.2,115.4,108,105,101.3,96.89,95.15,93.88,90.05,88.59,87.16,86.19,86.23,83.74,83.26,82,81.07,80.91,80.18,78.68,78.23,78.42,77.3,76.8,76.35,75.8,76.22,74.92,75.34,75.45,74.61,74.03,73.83,73.63,73.84,72.34,72.53,72.33,72.1,72.18,71.62,71.74,71.61,71.45,72.05,71.07,71.56,72.2,67.19,66.8,66.55,66.39,66.27,66.15
137 | lstm,penntreebank,5,170,0.1,15000,756,699.2,524.3,404.5,275.8,241,203.6,179.1,162.9,158.9,154.2,140,136.2,131,129.6,128.5,124.4,121.9,118.3,116.4,117.6,116.5,113.1,112.1,110.1,110.1,108.6,108.7,108,107.3,106,105.5,106,104.7,104.4,105.7,102.7,103,103,104.8,96.33,95.87,95.57,95.3,95.1,94.91,94.73,94.6,94.47,94.33,94.17,94.02,93.86,93.72,93.59,93.47,93.35,93.23,93.11,93
138 | lstm,penntreebank,5,410,0.1,15000,758.5,577.9,401.5,305.2,226.1,197.7,168.4,153.2,142.3,136.1,132.3,123.8,119,114.2,111.3,109.9,113.5,106.6,104.7,101.1,101.4,99.87,99.45,97.32,95.09,94.98,94.89,95.05,92.32,92.16,91.01,90.62,90.7,89.92,90.93,90.85,88.24,89.09,87.97,87.93,88.25,86.34,87.07,85.79,87.06,87.5,84.91,85.02,84.75,85.43,85.68,84.35,84.82,85.25,79.09,78.63,78.41,78.23,78.08,77.97
139 | lstm,penntreebank,5,650,0.1,15000,729,550.3,384.6,272.3,207.4,184.4,163.3,146.6,135.7,127.8,125.9,115.4,112.2,108.5,105.3,104.8,101.4,101.6,98.83,95.57,97.48,96.07,92.27,92.36,91.3,89.12,89.07,88.17,87.2,86.95,86.64,85.3,86.52,84.35,84.99,85.24,84.15,83.08,82.71,83.53,82.28,81.28,81.48,81.23,81.27,80.7,79.64,80.49,79.7,79.42,79.44,80.38,78.53,79.41,79.03,77.37,78.38,77.96,77.6,78.87
140 | lstm,penntreebank,5,890,0.1,15000,736.8,496.9,321.4,249.2,192.7,172.7,154.9,140.5,128.6,124,119,112.1,108.3,105,103.3,99.72,97.85,96.62,92.58,93.04,91.72,90.05,89.53,87.15,86.06,85.55,87.56,84.4,83.62,83.2,82.63,82.33,81.83,81.02,80.81,80.7,79.68,79.06,79.27,78.85,78.8,77.84,78.48,77.18,77.86,77.06,76.02,76.04,76.57,76.58,77.14,75.41,75.55,75.46,74.72,73.89,74.61,74.16,74.33,74.08
141 | lstm,penntreebank,5,1130,0.1,15000,621,355.6,262.4,209.9,173.5,156.7,143.1,131,122.1,118.5,113.5,107.4,103.1,100.2,97.02,98.18,95,92.75,89.99,89.71,88.66,87.82,86.54,85.34,84.74,84.15,82.36,82.3,81.52,80.77,80.37,79.43,79.77,78.5,78.54,79.35,77.58,77.08,77.07,76.6,77.2,75.98,75.54,75.54,76.12,75.5,74.87,74.64,74.58,74.2,74.72,73.97,73.72,73.53,73.25,73.4,73.24,73.13,72.87,72.44
142 | lstm,penntreebank,5,1370,0.1,15000,709.9,408.7,285.2,224.1,179.4,161.7,145.3,137.9,121.5,119.9,113.6,107.1,103.9,100,97.17,95.64,93.61,91.55,89.1,87.56,87.99,86.27,84.15,83.63,83.24,81.95,81.32,81.36,79.66,79.62,78.63,78.61,78.56,77.84,77.97,77.66,76.96,76.31,76.78,75.61,75.57,75.21,75.08,74.59,75.24,74.43,73.9,73.72,73.77,74.16,73.64,73.18,73.23,73.62,73.15,73.02,73.54,73.01,72.55,72.29
143 | lstm,penntreebank,2,150,0.1,15000,297.8,197.5,165.7,150.1,140.4,132.4,126.6,119.7,114.1,113.1,111.2,107.6,106.2,104.3,102.7,102.2,102,99.64,99.17,99.33,101.9,100.2,96.15,95.33,94.55,95.84,94.23,96.85,93.28,92.37,91.71,91.53,92.53,91.7,91.75,92.75,86.79,86.4,86.11,85.87,85.65,85.45,85.31,85.16,85,84.86,84.74,84.64,84.53,84.43,84.33,84.25,84.16,84.08,84,83.92,83.85,83.79,83.72,83.66
144 | lstm,penntreebank,2,390,0.1,15000,291.2,196.5,158,142.7,129.2,122.4,117.2,111.5,107.1,107.3,102.5,100.3,99.43,96.63,95.3,93.93,94.17,93.52,90.87,90.33,92.92,90.01,89.14,87.95,87.37,86.99,88.09,89.15,86.05,85.78,85.39,85.82,85.7,84.68,85.18,85.12,84.58,83.18,83.22,83.06,83.76,82.3,83.37,83.45,77.91,77.54,77.29,77.07,76.89,76.76,76.65,76.54,76.45,76.37,76.29,76.22,76.13,76.06,75.98,75.91
145 | lstm,penntreebank,2,630,0.1,15000,280.9,192.8,155.2,140.5,127.7,119.6,113.1,108,103.1,100.7,99.93,96.55,95.48,94.35,91.3,91.61,89.85,89.35,88.18,87.53,89.36,86.61,85.75,85.13,84.09,85.87,83.59,84.04,83.3,82.72,82.22,81.87,82.37,81.58,82.09,82.77,76.83,76.34,76.12,75.9,75.74,75.6,75.47,75.34,75.21,75.08,74.96,74.85,74.75,74.64,74.55,74.45,74.37,74.28,74.19,74.12,74.04,73.96,73.9,73.83
146 | lstm,penntreebank,2,870,0.1,15000,294.1,191.9,155.3,138.2,127.7,116.6,111.5,105.9,101,99.49,97.2,94.6,93.16,90.41,88.82,89.88,87.95,86.3,86.37,85.51,86.75,84.08,83.72,82.66,83.29,81.59,82.94,83.09,81.4,80.45,80.12,79.74,79.7,79.98,79.51,80.34,79.52,78.77,77.89,78.91,77.65,77.47,77.91,77.2,77.63,76.76,76.65,76.84,76.92,76.82,76.63,76.8,72.12,71.72,71.47,71.27,71.13,70.99,70.88,70.79
147 | lstm,penntreebank,2,1110,0.1,15000,286,191.7,157.9,140.5,124.9,116.6,111,106.8,100.2,101.6,95.68,92.94,91.58,89.68,88.42,88.41,86.6,85.6,86.13,84.74,85.47,83.37,84.36,81.61,81.25,80.78,80.26,80.11,79.03,79.96,79.34,78.5,79.99,77.55,78.46,79.23,73.42,72.92,72.64,72.38,72.16,72.01,71.87,71.75,71.63,71.53,71.44,71.34,71.25,71.15,71.06,70.97,70.88,70.8,70.72,70.65,70.57,70.5,70.42,70.35
148 | lstm,penntreebank,2,1350,0.1,15000,295.6,215,159.2,139.6,123.4,115.2,108.8,103,99.06,97.84,93.92,91.98,90.76,88.99,87.31,86.28,84.66,85.51,83.8,83.35,83.92,82.22,81.48,80.64,80.02,79.5,79.86,79.8,78.4,78.18,78.5,77.35,77.87,77.35,77.78,78.51,72.82,72.41,72.11,71.87,71.66,71.49,71.33,71.19,71.05,70.94,70.83,70.72,70.61,70.52,70.42,70.34,70.26,70.18,70.11,70.03,69.95,69.88,69.8,69.73
149 | lstm,penntreebank,3,190,0.1,15000,362.5,238,179.2,162.2,151.4,137.3,135.3,124.4,120.4,118,116.5,111,108.9,106.8,106.3,105.3,106.8,103.8,99.62,99,99.97,99.42,97.33,96.92,95.86,95.22,94.89,95.6,94,93.86,93.33,93.35,94.25,92.44,93.06,94.86,86.95,86.51,86.17,85.93,85.73,85.55,85.41,85.27,85.15,85,84.87,84.74,84.62,84.5,84.39,84.29,84.19,84.1,84.01,83.92,83.83,83.74,83.66,83.58
150 | lstm,penntreebank,3,430,0.1,15000,363.2,216.2,177.1,151.5,145.9,127.6,119.9,114.4,108.9,106,105.6,100.8,98.83,97.02,95.83,95.46,95.03,92.23,90.32,91.73,90.33,90.23,88.33,86.85,88,85.23,87.3,84.95,84.39,83.68,84.69,82.57,84.08,83.09,82.92,82.26,83.63,81.22,81.46,81.69,80.83,80.36,80.34,80.34,80.58,79.94,79.09,79.93,79.47,78.7,79.68,78.91,79.91,74.46,74.13,73.93,73.73,73.6,73.49,73.39
151 | lstm,penntreebank,3,670,0.1,15000,343.5,227.1,174.2,150.4,134.4,126.3,116.7,110.6,105.1,103.9,100.6,96.5,94.54,92.03,90.87,90.12,88.92,89.96,85.81,85.21,85.53,85.36,83.87,83.36,82.21,81.7,82.02,81.86,80.3,80.29,79.25,78.68,79.26,78.55,78.93,78.3,78.06,77.34,76.91,76.65,76.8,76.11,76.37,75.93,76.18,75.86,74.99,74.96,74.93,75.45,76.18,70.91,70.5,70.25,70.08,69.93,69.78,69.66,69.55,69.45
152 | lstm,penntreebank,3,910,0.1,15000,356.9,207.1,166.4,149.9,132.3,123.6,114.6,107.5,102.3,99.74,96.95,93.49,91.87,89.57,88.29,87.6,85.48,85.6,83.65,83.3,83.27,82.88,81.7,80.26,79.97,79.21,80.44,78.84,78.33,77.58,77.61,76.95,77.31,76.54,76.43,76.43,76.41,75.07,75.16,74.94,74.82,74.38,74.96,74.07,73.58,73.49,73.45,73.25,73.13,72.89,74.19,68.93,68.61,68.33,68.13,67.97,67.83,67.71,67.58,67.48
153 | lstm,penntreebank,3,1150,0.1,15000,339.6,217.6,171.7,151.8,131.5,121.1,113.1,105.3,101.4,98.31,97.21,93.29,90.34,88.01,86.46,85.61,85.24,83.1,82.08,81.25,81.01,80.81,79.77,78.71,78.88,77.93,77.88,76.78,75.92,75.76,75.23,75.11,75.02,74.77,74.4,74.59,74.06,73.53,73.23,72.86,73.33,72.19,72.54,72.22,72.43,71.5,71.43,71.53,71.55,71.43,72.07,70.62,71.29,70.56,70.58,70.04,70.37,70.78,66.7,66.34
154 | lstm,penntreebank,3,1390,0.1,15000,339.7,211.5,169.8,148.4,133.4,119.6,112.9,105.5,100.5,96.9,95.19,91.77,89.6,87.53,86.26,85.4,84.36,83.31,81.68,80.54,80.65,81.57,79.45,78.47,77.42,76.42,76.53,77.43,75.97,75.01,74.03,74.27,74.92,73.68,73.57,73.44,72.83,72.43,72.43,72.19,72.15,71.85,71.59,71.68,71.43,71.3,70.85,70.54,70.14,70.63,71.15,70.33,70.36,70.57,66.36,65.94,65.65,65.48,65.31,65.19
155 | lstm,penntreebank,4,230,0.1,15000,463.9,280,217,185.7,160.9,149.9,141.5,132.2,126,125.6,120.2,115.2,112.9,111,108.8,109.5,106.8,106.5,102.4,103.4,104.5,101.7,100.7,98.79,97.27,96.96,95.9,96.03,94.92,95.79,96.44,96.07,95.96,88.74,88.29,87.91,87.63,87.4,87.15,86.93,86.74,86.57,86.42,86.28,86.15,86.01,85.88,85.74,85.61,85.48,85.36,85.25,85.13,85.03,84.93,84.83,84.74,84.64,84.54,84.45
156 | lstm,penntreebank,4,470,0.1,15000,472.4,285.9,203,175.1,153.3,143.2,134,123.5,116.9,115.4,110.2,105.1,102.8,100.7,99.71,98.27,96.47,94.79,93.94,91.96,92.52,92,89.93,90.19,89.1,88.98,87.48,87.8,86.79,85.82,84.93,85.23,85.21,83.79,84.09,84.2,83.56,83.53,83.16,81.93,82.75,81.83,82.35,80.84,81.47,80.88,80.23,80.2,79.67,79.27,80.39,79.11,80.43,74.98,74.61,74.35,74.15,74.02,73.93,73.81
157 | lstm,penntreebank,4,710,0.1,15000,461,249.1,200.4,166.9,147.4,135.5,126.9,117.5,111.3,109,106.3,101.2,98.89,96.08,94.56,93.34,91.14,90.58,88.17,86.57,88.18,88.04,85.96,84.42,84.9,82.76,82.89,82.14,82.06,80.77,79.99,79.74,79.83,78.43,79.02,80.12,77.71,77.73,77.29,77.79,77.23,77.03,76.6,78.02,72.4,72.04,71.79,71.55,71.32,71.12,70.96,70.83,70.69,70.57,70.46,70.36,70.26,70.17,70.08,69.99
158 | lstm,penntreebank,4,950,0.1,15000,476.2,261,195.4,170.6,145.6,135.1,125.6,118,109.6,110.1,103.1,99.52,96.72,93.5,92.07,90.72,90.6,88.01,86.51,85.35,85.13,84.12,82.85,82.56,81.46,80,80.17,79.46,78.62,78.62,78.18,77.8,77.76,77.29,77.35,79.32,71.99,71.51,71.13,70.86,70.65,70.45,70.29,70.13,69.98,69.84,69.7,69.56,69.44,69.31,69.19,69.08,68.97,68.87,68.77,68.68,68.59,68.49,68.4,68.31
159 | lstm,penntreebank,4,1190,0.1,15000,426.4,258.6,208.1,163.8,144.6,130.3,124.3,114.4,110.2,103.8,102,97.16,96.57,92.48,90.88,89.7,87.92,86.86,86,85.59,83.99,83.73,82.29,81.71,80.6,79.76,79.34,79.3,77.63,77.36,77.37,76.68,76.96,78.52,75.73,76.11,75.22,74.49,74.74,74.63,74.26,73.39,73.76,73.47,73.35,73.69,73.52,72.53,72.88,72.34,71.86,71.84,71.73,71.95,71.97,71.27,71.51,71.28,70.97,70.72
160 | lstm,penntreebank,5,30,0.1,15000,725.2,699.4,690.7,709.3,703.3,706.4,715.5,697.4,696.5,679.9,680,680.2,680.3,680.3,680.4,680.4,680.4,680.4,680.4,680.4,680.4,680.5,680.5,680.5,680.5,680.5,680.5,680.5,680.5,680.6,680.6,680.6,680.6,680.7,680.7,680.7,680.8,680.8,680.8,680.8,680.7,680.8,680.8,680.8,680.8,680.9,680.8,680.9,680.9,680.9,680.9,680.9,680.9,681,681,681,681,681,681,681.1
161 | lstm,penntreebank,5,270,0.1,15000,731.4,700.5,432,318.1,243.3,211.5,184.8,161.4,150.7,141,140.2,130.9,127.4,123.9,118.5,118.8,117,114.1,111,108.9,109.2,108.1,103.7,104.8,102.1,100.9,102,99.19,99.51,98.48,97.57,97.07,98.44,95.99,95.94,97.69,95.71,95.79,94.44,94.59,95.5,93.82,93,92.75,92.66,91.89,91.79,92.13,91.1,90.68,92.06,90.55,92.1,85.54,85.17,84.95,84.72,84.53,84.37,84.23
162 | lstm,penntreebank,5,510,0.1,15000,734.7,700,474.1,351,269.6,218.9,187.6,159.8,147.7,140.1,131.8,121.6,116.9,115.1,110.6,108.5,105.2,108.9,103.6,101.7,103,98.18,95.66,95.09,93.33,93.91,91.14,90.84,90.79,90.13,89.35,88.02,89.23,87.62,87.06,86.2,86.14,85.69,85.28,84.73,85.06,84.43,83.99,83.87,83.46,82.78,82.43,82.09,82.13,83.04,81.64,81.14,82.73,76.56,76.26,76,75.76,75.55,75.37,75.23
163 | lstm,penntreebank,5,750,0.1,15000,724.7,458.8,286.5,229.1,186.6,169.3,154.2,138.6,128.9,126.9,120.7,112.2,108.9,105.5,103.2,103.2,98.68,97.38,95.33,93.75,94.18,92.12,90.46,89.08,88.37,86.61,86.33,85.77,84.56,84.73,83.92,83.19,83.69,81.82,82.15,83.55,81.35,80.83,80.16,80.6,79.73,79.42,78.93,78.7,78.76,78.35,78.34,77.7,77.94,77.37,77.99,76.65,77.19,76.68,76.16,76.26,76.47,75.88,76.63,75.19
164 | lstm,penntreebank,5,990,0.1,15000,658.4,366.1,255.8,213.7,179.5,158.1,145.4,133,122.7,121.5,113.4,107,104,100.4,98.08,96.51,94.33,94.05,91,89.3,90.42,87.81,85.91,85.71,84.9,83.52,83.4,83.76,81.9,81.2,80.73,80.45,81.25,79.21,79.39,79.64,78.46,77.88,77.4,77.73,77.2,77.38,76.53,76.11,76.67,75.35,75.43,75.13,75.09,74.83,75.28,74.26,74.6,74.77,73.54,73.45,73.76,73.24,73.06,73.33
165 | lstm,penntreebank,5,1230,0.1,15000,724.7,508.8,338.6,247,208.1,168.2,153.9,139.5,125.5,122.2,116.3,109.2,105.1,101.8,99.27,97.62,95.03,93.56,91.31,90.17,88.56,86.89,86,85.45,83.67,83.29,82.63,81.33,80.7,80.63,79.81,79.57,79.35,78.27,78.4,77.79,77.15,76.83,76.89,76.15,76.65,76.52,75.41,75.18,74.77,74.63,74,74.62,73.75,73.42,74.19,73.26,73.3,74,69.29,68.89,68.62,68.44,68.26,68.11
166 | lstm,penntreebank,2,70,0.1,15000,292.1,204,177.6,160.4,146.8,140.7,134.4,127.5,122.8,122.9,121.5,115.2,112.8,110.5,110.3,110.2,109,110.3,105.1,105.8,105.1,104.6,106.4,103.5,101.6,101.7,100.1,100.4,99.83,99.96,99.08,99.18,101.4,93.64,93.17,92.83,92.57,92.38,92.21,92.07,91.93,91.82,91.69,91.57,91.45,91.34,91.23,91.14,91.04,90.95,90.86,90.78,90.69,90.6,90.52,90.44,90.35,90.27,90.19,90.12
167 | lstm,penntreebank,2,310,0.1,15000,300.5,194.9,168.2,147.1,133.6,126.8,119.4,113.4,112.6,105.4,105.3,104.7,100.3,98.39,96.79,96.68,96.65,96.03,93.16,92.31,94.05,92.62,91.37,91.02,89.72,88.74,89.66,89.51,87.99,88.08,86.82,87.21,88.58,86.14,87.19,86.5,85.68,86.14,85.23,84.66,85,83.84,84.88,84.24,84.28,83.81,83.84,83.43,82.77,83.47,84.51,79.05,78.61,78.36,78.14,77.96,77.83,77.7,77.59,77.49
168 | lstm,penntreebank,2,550,0.1,15000,281.4,189.2,156.8,140.4,127.1,119.8,115.2,109.2,104.7,101.3,99.93,96.87,94.91,93.61,92.74,91.94,90.98,90.36,88.99,87.64,90.08,87.29,86.96,85.68,84.72,83.84,84.92,84.55,84.37,83.1,83.19,82.27,83.15,82.83,82.77,83.71,77.35,76.91,76.64,76.41,76.21,76.03,75.88,75.75,75.62,75.51,75.39,75.29,75.19,75.09,74.99,74.9,74.81,74.73,74.65,74.58,74.51,74.44,74.38,74.32
169 | lstm,penntreebank,2,790,0.1,15000,291.9,190.1,165.1,142.6,124.4,116.9,113.4,106.5,101.4,103.3,97.75,95.37,92.99,90.8,89.47,90.48,88.07,88.15,86.38,85.58,87.77,85.41,84.5,83.31,83.28,82.21,82.43,83.29,81.33,81.63,81,81,80.42,79.87,79.91,80.59,80.4,80.17,78.94,79.32,79.17,78.81,77.82,79.6,77.97,77.8,77.41,77.89,77.87,73.53,73.17,72.93,72.7,72.54,72.4,72.28,72.17,72.05,71.94,71.84
170 | lstm,penntreebank,2,1030,0.1,15000,282.8,204.1,154.2,141.2,127.2,117.1,109.9,104.8,101.1,101,95.57,93.51,92.28,89.76,88.29,87.97,87.37,85.94,84.96,83.92,85.53,84.3,82.52,82.41,82.17,80.68,80.74,80.13,80.55,79.48,79.8,78.69,78.92,78.48,79.97,79.35,78.36,78.66,77.35,77.1,77.74,77.16,77.41,76.62,77.64,72.52,72.14,71.91,71.7,71.53,71.39,71.28,71.18,71.08,70.99,70.91,70.82,70.74,70.66,70.58
171 | lstm,penntreebank,2,1270,0.1,15000,290.6,191.8,152.8,140.7,124,115.3,109.8,103,99.4,95.31,95.13,91.55,90.26,88.36,87.06,86.56,85.91,85.3,83.89,83.73,84.77,82.45,81.22,80.92,80.6,80.63,80.39,79.19,79.13,78.93,78.22,78.12,79.08,77.76,78.01,77.6,77.14,76.65,76.91,76.28,76.9,75.89,75.64,77.03,71.85,71.46,71.12,70.85,70.66,70.53,70.4,70.29,70.2,70.12,70.03,69.93,69.85,69.78,69.71,69.64
172 | lstm,penntreebank,3,110,0.1,15000,355.4,240.2,188.9,173.2,153.5,149.3,142.4,134.6,128.5,128,124.9,119.4,117.9,115.4,113.5,112.6,112.7,109.8,108,108.2,110.1,106.6,105.8,106.6,104.7,106.3,103.2,103.3,101.9,102,100.7,101.3,102.9,100.2,100.8,100.5,99.97,99.61,99.04,97.83,98.95,97.61,97.81,98.98,98.33,98.02,92.28,91.89,91.63,91.41,91.25,91.11,90.97,90.84,90.73,90.63,90.55,90.47,90.39,90.32
173 | lstm,penntreebank,3,350,0.1,15000,381.5,213.2,176.1,153.3,142.4,130.3,125.1,116.4,111.2,108.8,107.4,103.5,101.4,99.52,97.84,97.35,96.61,96.06,94.5,92.78,94.88,91.91,91.31,90.94,88.8,88.55,88.49,87.85,87.92,86.89,86.94,85.75,86.23,88.02,80.6,80.19,79.94,79.72,79.5,79.31,79.14,78.97,78.82,78.68,78.54,78.4,78.26,78.12,78,77.88,77.76,77.65,77.55,77.45,77.36,77.27,77.19,77.11,77.03,76.95
174 | lstm,penntreebank,3,590,0.1,15000,350.4,211.9,171,152.8,133.3,125.5,119.1,110.8,107.4,110.5,101.4,99.61,95.39,95,92.28,91.68,91.62,89.66,88.31,86.83,89.35,86.26,86.33,84.4,84.06,83.77,83.25,84,81.99,81.44,80.98,81,80.89,79.97,79.84,82.25,75.23,74.8,74.49,74.2,73.96,73.77,73.6,73.46,73.32,73.2,73.08,72.97,72.87,72.76,72.65,72.55,72.45,72.35,72.25,72.15,72.06,71.97,71.89,71.81
175 | lstm,penntreebank,3,830,0.1,15000,327.5,209.6,165.5,149.5,132.5,122.2,115.3,108.5,103.6,101.4,98.71,96.66,92.41,90.7,90.52,88.21,89.21,86.48,84.63,84.03,84.22,82.98,81.83,81.84,81.07,79.61,79.71,79.45,78.98,78.63,78.13,77.51,78.04,77.03,77.07,77.62,77.87,76.21,76.14,76,75.45,75.42,75.64,75.49,75.59,74.25,74.04,74.01,73.82,74.12,74.35,73.61,73.61,73.44,73.77,73.55,72.96,72.56,73.09,72.39
176 | lstm,penntreebank,3,1070,0.1,15000,338.6,210,168.3,148.7,132.3,124.3,114.5,106.7,101.6,101,96.67,94.07,90.83,89.28,88.16,86.55,85.36,85.18,82.79,81.86,82.65,81.38,80.65,79.27,79.09,78.38,77.84,77.71,76.68,76.82,76.23,76.08,75.96,75.22,75.2,75.33,73.99,73.85,73.92,73.61,73.45,72.78,73.72,72.97,73.67,72.37,72.44,71.95,72.59,72.03,72.88,68.15,67.74,67.48,67.26,67.1,66.97,66.85,66.74,66.65
177 | lstm,penntreebank,3,1310,0.1,15000,355.8,216.6,166,151.4,133.5,120.5,113,105.5,102.3,100.3,95.06,91.72,90.3,88.06,86.67,86.91,86.34,83.13,81.97,80.9,81.33,80.81,79.3,77.87,77.76,76.42,76.87,75.82,75.32,74.88,74.54,74.39,74.26,74.18,73.96,74.01,73.63,72.55,72.64,72.22,72.35,72.29,71.67,71.7,71.59,71.17,70.74,70.72,70.33,71.17,71,70.31,70.82,66.73,66.34,66.09,65.88,65.75,65.62,65.49
178 | lstm,penntreebank,4,150,0.1,15000,588,302.9,224.5,199.2,176.5,161.9,154.2,142.4,135.6,130.6,132.2,124.2,124.5,118.8,116,115.5,115.8,112.7,110,109.5,111.1,109.6,106.6,107,105.5,103.7,103.2,102.5,103.7,102.2,101.2,100.4,102.8,100.4,101.6,103.1,94.18,93.68,93.31,93.05,92.85,92.65,92.49,92.33,92.2,92.06,91.92,91.77,91.63,91.5,91.37,91.26,91.15,91.05,90.95,90.86,90.75,90.66,90.58,90.5
179 | lstm,penntreebank,4,390,0.1,15000,535.2,277,211.4,180.5,154.3,144.4,134.9,123.5,119,113.8,112.3,107.9,105.7,102.6,101,100.4,98.9,99.78,96.76,93.8,94.4,94.05,92.59,91.47,91.82,89.32,89.14,89.44,90.44,88.26,87.98,87.92,87.68,86.46,86.01,86.67,84.98,85.06,84.89,85.02,84.55,83.29,83.63,82.77,83.49,83.2,81.99,81.97,82.48,82.23,84.44,77.52,77.18,76.99,76.81,76.63,76.46,76.31,76.18,76.06
180 | lstm,penntreebank,4,630,0.1,15000,475.2,263.8,199.3,168.1,147.2,135.9,127.9,118.5,112.8,112.7,105.8,101.5,99.24,98.8,94.98,95.62,93.37,91.69,90.91,91.4,89.92,89.4,87.3,85.55,85.17,84.87,83.89,83.31,82.57,82.43,82.27,81.95,82.96,80.47,81.07,80.85,80.41,79.29,78.9,78.68,79,77.67,78.17,77.72,78.31,77.67,77.09,76.99,76.88,76.74,78.35,72.45,72.01,71.7,71.46,71.28,71.12,70.99,70.87,70.76
181 | lstm,penntreebank,4,870,0.1,15000,454.9,269.5,197,168.1,149.6,133.9,125.4,116.3,110.4,106.5,103,98.77,96.45,93.86,92.23,92.12,89.75,88.92,86.8,85.7,86.6,84.66,83.54,82.2,81.82,80.78,81.39,80.33,79.19,78.73,78.49,78,78.84,78.1,77.17,77.31,76.88,76.42,75.59,75.84,75.62,75.12,75.23,74.79,75.11,75.11,73.57,73.48,74.21,73.5,73.7,73.23,74.3,69.09,68.72,68.5,68.29,68.14,68.02,67.94
182 | lstm,penntreebank,4,1110,0.1,15000,470.9,261.4,197.9,169,155,131.6,123,113.1,108.2,105.4,100.3,96.94,94.72,95.81,90.49,89.81,87.35,87.64,84.88,83.45,83.75,83.16,81.43,80.82,79.98,79.04,79.08,78.62,77.89,77.93,76.92,76.6,76.67,75.71,75.77,75.94,75.02,74.52,74.39,74.1,73.98,73.24,73.17,73.25,73.85,72.7,72.28,72.18,71.73,71.66,72.37,71.35,71.83,71.74,71.17,70.76,71.28,71.09,70.46,70.39
183 | lstm,penntreebank,4,1350,0.1,15000,454.5,251.9,198.4,168.8,149.6,133.9,122.3,117.7,108.8,104.9,101.1,97.77,95.21,92.61,90.55,89.4,87.7,86.54,84.11,83.27,83.13,82.6,81.52,80.05,79.3,78.85,78.12,78.1,76.88,76.67,76.18,76.51,75.68,76.6,75.07,75.4,74.01,74.04,73.91,73.3,73.47,72.82,72.95,72.52,72.83,72.43,71.74,71.66,72.1,72.32,72.55,67.56,67.18,66.9,66.7,66.54,66.39,66.26,66.14,66.03
184 | lstm,penntreebank,5,190,0.1,15000,725.9,698.8,690.6,711.3,494.9,368.9,284.8,219.8,188.7,174,166.7,152.1,146,140.2,134.2,133.6,130.1,131.2,123,118.5,122.1,117.7,118.7,112.7,111.4,109.5,109.4,109.1,108.9,107.5,106.2,105.7,105.6,103.4,104.6,104.8,103.2,105.1,101.5,102,101.3,99.9,101,100.1,101.3,99.58,97.72,97.17,98.1,98.98,98.11,96.85,97.82,91.47,91.06,90.86,90.64,90.46,90.32,90.17
185 | lstm,penntreebank,5,430,0.1,15000,747.9,659.6,499.8,305.3,226,198.2,179.4,150.8,141.1,132.3,130.6,120.9,119.6,116.7,112.3,109.1,107.5,106.5,102.1,101.5,102.6,100.4,98.54,97.29,94.73,94.08,94.27,94.25,92.8,91.6,94.24,90.28,93.33,89.73,90.66,90.11,88.39,88.2,87.94,86.96,86.84,86.43,87.3,87.11,87.34,85.29,84.72,84.79,84.43,84.24,85.63,84.7,84.42,84.66,82.91,82.4,83.93,85.46,78.21,77.83
186 | lstm,penntreebank,5,670,0.1,15000,738.7,633.3,356.2,275.4,214.7,185.8,163.3,144.9,133.8,126.1,121.4,115.3,111.6,106.9,103.2,103.1,101,98.76,96.17,95.66,95.71,92.61,91.66,91.38,89.14,87.93,87.38,87.65,86.21,85.87,84.6,84.5,84.59,82.76,83.41,83.74,81.98,82.36,81.47,81.53,81.22,80.06,80.54,80.34,80.23,79.11,78.58,78.52,78.41,78.8,78.26,77.5,78.39,78.12,76.89,77.23,77.08,76.7,76.99,76.03
187 | lstm,penntreebank,5,910,0.1,15000,729.9,376.9,258.2,222.2,180.9,165.1,145.3,133.2,125.8,121.4,115.1,108.9,106.1,103.3,99.65,98.7,96.76,95.46,92.78,95.23,91,89.57,88.16,86.6,85.53,86.41,84.1,83.86,83.1,82.43,81.98,80.63,82.04,80.23,80.72,79.83,79.64,78.74,78.11,78.33,77.92,77.3,77.42,77.45,78.24,72.43,71.97,71.65,71.41,71.17,70.98,70.83,70.7,70.58,70.46,70.33,70.22,70.12,70.02,69.93
188 | lstm,penntreebank,5,1150,0.1,15000,724.6,451.3,289.9,233.2,185.2,164.1,146.5,133,124.2,118.1,112.1,107.4,103.7,100.3,97.49,95.72,94.07,91.86,89.76,89.57,89.46,87.88,85.63,84.96,84.15,82.9,82.67,82.1,81.3,80.89,80.17,80.07,79.76,78.29,78.83,79.8,77.84,78.05,76.94,77.11,76.67,75.85,75.86,75.44,76.06,75.12,74.78,74.6,74.69,74.28,74.85,74.31,73.99,73.83,73.87,72.98,72.98,73.33,72.57,72.92
189 | lstm,penntreebank,5,1390,0.1,15000,747.2,375.2,270.7,215.7,176,159.6,146.6,130.7,123.4,119.6,113.8,105.7,102.7,100.1,97.48,95.23,93.77,92.1,89.35,89.22,88.94,86.36,84.58,84.21,83.57,82.5,81.6,80.72,80.61,79.64,79.38,78.79,78.6,77.87,78.04,77.67,76.82,76.08,76.68,75.79,75.9,74.51,75.23,74.62,75.45,74.45,73.7,73.8,73.38,73.65,74.39,73.47,72.85,73.44,73.08,72.49,72.82,72.56,72.83,72.22
190 | lstm,penntreebank,2,170,0.1,15000,298.7,201,164.1,152,139.5,129.7,126.5,117.7,113.5,112.2,112.6,106.7,108.6,103.2,101.6,101.1,102.8,102,98.29,97.45,96.4,97.93,94.88,97.03,94.22,93.15,93.34,92.75,93.09,91.98,91.52,91.4,92.14,90.2,91.3,90.97,90.55,89.65,90.3,89.76,89.73,88.94,90.07,88.28,89.23,88.22,88.21,88.3,87.25,87.43,88.2,86.84,87.17,86.87,86.53,86.41,86.89,86.56,87.65,82.06
191 | lstm,penntreebank,2,410,0.1,15000,272.2,190.9,159.2,144.4,128.2,122.8,117.6,110.3,106.7,105.6,102,100,99.93,97.14,93.8,94.84,92.83,91.56,91.29,90.74,93.79,89.88,89.04,87.79,87.28,85.93,86.13,86.66,85.21,85.14,84.75,84.33,85.8,84.17,84.18,84.84,83.95,84.27,83.42,83.45,82.78,82.91,82.05,82.45,81.52,82.36,81.22,81.81,81.02,81.64,81.94,76.92,76.53,76.28,76.04,75.86,75.72,75.59,75.47,75.36
192 | lstm,penntreebank,2,650,0.1,15000,295.9,193.1,165.1,141.5,126.2,118.4,112.1,107.1,105.1,100.6,99.7,99.33,94.56,92.6,91.54,90.79,90.5,88.76,87.7,89.97,89.04,86.99,85.75,85.32,84.98,83.97,84.42,83.52,82.47,82.28,81.9,81.47,83.07,81.99,81.32,83.48,76.62,76.21,75.96,75.76,75.61,75.45,75.29,75.15,75.01,74.87,74.73,74.62,74.51,74.42,74.34,74.26,74.18,74.11,74.04,73.97,73.9,73.84,73.77,73.72
193 | lstm,penntreebank,2,890,0.1,15000,296.7,190.5,156.1,139.1,128.7,117.7,111,105.4,101.1,99.92,97.7,93.81,93.33,90.62,89.98,89.15,88.57,87.12,85.1,89.1,86.35,84.4,84.06,82.45,82.26,83.16,81.04,81.95,80.61,80.37,80.02,79.09,79.53,79.06,78.9,81.34,74.55,74.12,73.82,73.59,73.39,73.26,73.13,73.02,72.9,72.79,72.7,72.6,72.51,72.42,72.32,72.24,72.16,72.08,72,71.92,71.85,71.77,71.7,71.63
194 | lstm,penntreebank,2,1130,0.1,15000,276.7,196.8,155.5,136.3,127.8,115.6,110.1,103.4,99.38,101.9,94.8,93.12,92.66,89.42,87.12,88.09,86.3,84.99,85.77,84,85.06,83.27,82.7,81.62,80.84,80.09,80.54,81.87,79.6,79.88,79.11,79.11,78.64,78.5,78.1,78.97,78.62,77.37,77.79,77.07,77.87,76.61,76.19,76.79,76.84,77.23,71.98,71.61,71.42,71.21,71.01,70.86,70.69,70.58,70.46,70.34,70.24,70.13,70.04,69.96
195 | lstm,penntreebank,2,1370,0.1,15000,295.5,196.7,155.1,139.4,127,115.2,109.6,101.9,99.87,96.34,94.45,91.29,90.35,88.24,86.62,86.23,85.45,83.91,83.28,82.95,84.87,81.91,81.19,80.88,80.1,79.95,79.09,79.92,78.61,78.66,77.87,77.5,78.47,77.45,77.36,77.66,76.91,76.02,75.77,75.57,75.95,75.03,75.36,76.26,71.23,70.77,70.5,70.28,70.12,69.97,69.82,69.7,69.6,69.52,69.43,69.36,69.27,69.18,69.1,69.02
196 | lstm,penntreebank,3,210,0.1,15000,383.9,226.1,186.6,165.1,148,139.4,131.1,122.8,118.9,116.8,114.1,110.1,108.5,106.2,104.9,103,102.4,102.7,98.94,97.7,98.48,98.39,96.18,95.72,94.95,93.57,94.35,93.03,92.49,92.26,92.26,91.27,91.58,90.37,91.71,91.25,91.21,90.25,89.55,89.01,89.47,88.49,88.77,87.33,88.38,88.54,87.36,86.89,87.45,86.08,87.8,82.48,82.16,81.9,81.7,81.52,81.37,81.23,81.12,81.03
197 | lstm,penntreebank,3,450,0.1,15000,383.5,220.6,169.2,152.6,141.5,127.6,120.9,114.7,109.1,109,103.5,100.1,98.44,97.89,94.88,94.92,93.59,94.05,90.03,89.58,91.23,89.06,87.55,87.77,86.85,85.56,87.3,84.9,84.85,84.4,83.12,83.66,83.21,82.24,82.84,83.35,81.65,82.03,81.19,80.77,80.67,80.25,80.35,81.21,79.75,81.11,75.41,75.02,74.77,74.59,74.43,74.3,74.17,74.04,73.92,73.83,73.73,73.64,73.55,73.48
198 | lstm,penntreebank,3,690,0.1,15000,347.9,209,171.4,148,131.3,123.9,117.5,110.1,104.8,101.7,101,96.02,94.33,92.1,90.5,90.62,87.87,87.82,85.27,85.22,85.84,84.15,83.86,82.76,81.61,80.8,80.67,81.8,79.82,80.22,80.08,79.16,78.69,79.47,79.12,78.14,78.02,77.38,77.36,76.61,77.52,76.18,76.03,75.74,76.17,75.64,75.26,74.93,75.43,75.37,74.76,74.53,74.53,74.49,74.54,73.51,75.11,70.04,69.67,69.43
199 | lstm,penntreebank,3,930,0.1,15000,332.8,216.7,166,146.6,130.4,120.8,114.1,106.7,102,98.46,97.57,93.91,92.31,89.62,88.51,87.37,86.35,85.8,83.73,82.64,82.71,81.85,81.56,79.95,79.61,78.7,78.69,78.78,77.46,77.22,76.96,76.41,76.25,76.02,75.73,76.47,75.86,74.55,74.6,74.32,74.03,74.07,73.99,73.13,73.96,73.28,73.03,72.39,72.68,72.6,72.68,71.76,72.26,72.13,72.09,71.57,71.65,70.99,71.26,71.31
200 | lstm,penntreebank,3,1170,0.1,15000,339.5,210.5,172.5,145.2,133,122.1,114.5,106.1,101.9,103,96.27,92.25,90.51,88.69,87.23,86.33,84.73,83.84,82.13,81.02,81.88,80.04,79.32,79.38,78.87,77.49,77.24,77.32,76.28,76.43,75.57,75.07,75.53,74.57,74.93,74.52,74.37,74.01,73.21,73.06,73.65,72.59,72.23,72.24,72.13,72.41,71.38,71.83,71.65,71.98,72.14,67.66,67.33,67.08,66.83,66.64,66.48,66.35,66.24,66.14
201 | lstm,penntreebank,4,10,0.1,15000,873,499.8,446.1,387.6,374,369,378.2,342.7,350.7,326.2,325.8,312.1,312.8,300.5,292.3,321.3,312.7,290.1,283.8,282.4,290.3,297.6,270.4,269.7,274.5,270.6,268.9,268.2,267.1,267,266.4,266.2,265.1,264.6,263.8,263.5,263.4,263,262.9,263.1,263.3,263,263.2,263.5,263.5,262.9,262.5,262.5,262.2,263.6,262.6,262,261.8,261.9,261.8,261.8,261.8,261.6,261.4,261.6
202 | lstm,penntreebank,4,250,0.1,15000,480.5,280.5,210.9,181.7,160.2,149.2,142,129.7,123.6,121.9,118.8,114.2,110.9,108.5,106.7,107.1,105.3,104.1,101.1,101.8,103,98.66,98.21,96.71,96.39,95.81,95.58,95.46,94.09,93.7,92.72,92.88,94.4,91.92,92.45,92.47,90.59,91.53,91.03,90.57,91.16,88.97,89.74,88.78,88.77,89.12,87.79,87.83,88.01,87.4,89.71,83.02,82.71,82.49,82.28,82.12,81.97,81.84,81.72,81.61
203 | lstm,penntreebank,4,490,0.1,15000,499.3,261.3,197.3,171.5,149.1,141.4,130.9,119.5,114.3,111.9,108.7,106.4,101.8,99.85,98.19,97.14,95.69,93.72,92.83,91.46,91.7,90.88,88.95,88.19,88.09,86.07,86.01,86.03,84.77,84.55,84.84,83.6,84.32,82.86,83.09,82.28,82.53,81.65,81.8,81.61,81.59,80.22,80.61,80.21,80.24,79.86,79.79,79.36,79.37,78.34,79.32,78.68,78.97,78.82,78.8,77.67,77.74,77.81,78.07,78.21
204 | lstm,penntreebank,4,730,0.1,15000,457.8,264.8,196.5,170.2,149.6,137.2,128.7,117.4,111.5,108.4,104.9,101.6,99.44,96.57,93.91,92.84,92.03,90.49,89.13,88.41,89.16,86.66,85.42,84.49,83.78,82.6,83.27,82.17,81.31,80.87,79.91,80.01,80.86,79.5,79.85,79.9,79.7,77.7,78.03,77.64,78.84,77.62,77.47,78.75,72.64,72.23,71.94,71.71,71.51,71.34,71.17,71.02,70.88,70.76,70.64,70.53,70.43,70.33,70.24,70.14
205 | lstm,penntreebank,4,970,0.1,15000,477.2,262.2,196.5,170.8,146.9,134.8,127.3,116.6,110.1,109,103.6,98.43,98.04,93.98,91.91,91.46,90.18,87.46,87.81,85.64,85.87,84.65,83.02,81.83,81.88,80.3,80.59,80.08,79.24,78.15,77.73,77.64,77.73,76.74,77.12,77.37,77.3,75.61,75.82,75.16,75.06,74.16,74.18,74.54,74.69,73.81,73.65,73.22,73.33,73,73.51,72.78,73.4,72.91,72.84,71.72,71.83,71.8,71.89,71.34
206 | lstm,penntreebank,4,1210,0.1,15000,481,261.3,192.7,168.7,157.9,133.2,124.7,114.6,108.6,107.8,102.9,97.74,94.95,92.85,90.74,90.28,88.01,87.58,84.9,83.99,83.82,82.66,81.51,81.6,80.97,78.67,79.04,78.28,77.17,77.24,76.88,75.96,76.22,75.85,75.33,75.28,74.77,74.69,74,73.25,74.24,72.64,73.82,72.28,72.69,72.59,71.87,71.57,71.58,71.76,71.79,71.1,71.22,71.89,67.3,66.89,66.62,66.4,66.24,66.1
207 | lstm,penntreebank,5,50,0.1,15000,727.6,699.8,691.9,535.1,385.5,324.9,278.3,254,222.6,215.6,210.1,189,181.8,176.8,171.3,173.8,167.5,163.9,158.7,158.3,154.2,158.8,153,152.2,149.5,153.5,148,148.7,145.9,145.4,146.3,144.7,147.7,143,143.7,143.5,142,143.8,143.6,142.6,141.2,139.3,139.5,141.1,139.7,139.3,138.8,138,137,141.3,129.9,129.6,129.3,129.1,128.9,128.8,128.6,128.5,128.4,128.3
208 | lstm,penntreebank,5,290,0.1,15000,727.8,701.3,590.8,401.7,278.4,247.3,198.3,176.3,155,151.8,147,134.3,129.6,125.1,122.5,120,117.5,120.4,113.9,109.2,109.9,106.8,111,104.7,102.5,101.7,100.7,101.1,99.14,99.14,97.34,97.13,97.21,96.07,95.57,96.35,93.48,95.57,93.76,93.99,93.86,91.83,93.59,86.88,86.47,86.18,85.96,85.75,85.55,85.38,85.21,85.04,84.88,84.74,84.59,84.47,84.33,84.21,84.09,83.97
209 | lstm,penntreebank,5,530,0.1,15000,729.6,703.3,423,302.5,216.7,190.5,172.7,149.5,136.9,132.5,126.6,119.6,114.3,111.6,108.1,106.4,103.6,103.5,98.73,98.24,100.1,96.04,93.93,92.98,92.08,90.82,90.48,91.26,88.93,88.41,88.11,87.55,86.96,86.09,85.8,85.84,85.31,84.39,84.15,85.86,84.74,83.45,83.48,82.44,82.61,82.62,81.18,81.67,80.93,81.07,81.82,80.22,80.83,80.27,80.21,79.8,79.74,80.3,74.99,74.7
210 | lstm,penntreebank,5,770,0.1,15000,722.7,386.8,270.5,222.6,182,161.2,148.6,132.4,124.5,120.7,116.7,110.1,105.8,103.1,102.6,100,97.65,95.75,93.1,92.78,93.88,91.07,89.69,88.38,87.94,86.09,85.55,84.7,84.79,83.41,82.92,83.1,82.94,81.79,81.55,81.58,81.13,80.1,79.76,80.6,79.81,78.48,78.55,78.51,78.7,77.55,77.03,77.14,77.46,78.09,77.6,76.34,77.08,72.13,71.74,71.41,71.2,71.02,70.89,70.78
211 | lstm,penntreebank,5,1010,0.1,15000,726.4,398.3,273.3,215,177.1,165.1,145.7,133.8,124.6,121,114.4,108.6,104.3,101,99.39,97.41,95.62,93.26,91.21,90.4,89.44,89.19,86.97,85.4,84.85,84.66,83.99,83.02,82.1,81.96,80.98,80.1,80.23,79.61,79.67,79.58,79.45,77.74,77.53,77.48,77.33,76.44,76.45,76.5,76.53,76.32,75.66,75.18,74.86,75.27,75.95,74.71,73.95,74.48,73.92,74.15,73.89,73.55,73.39,73.44
212 | lstm,penntreebank,5,1250,0.1,15000,736.8,476.4,316.3,236.9,192.6,168.1,152,135.4,126.9,119.1,116.4,108.8,108,103.2,99.27,96.85,95.32,92.97,90.57,90.59,89.85,88.42,86.98,85.09,84.62,83.17,82.74,82.12,81.16,80.67,80.08,79.45,79.08,78.39,79.77,78.39,77.37,77.15,76.94,76.81,76.64,75.46,75.61,76.47,75.89,74.66,74.2,73.92,73.9,73.81,74.62,73.33,73.53,73.36,72.81,72.83,72.78,72.69,72.38,72.24
213 | lstm,penntreebank,2,190,0.1,15000,292.9,206.9,165.2,151.6,139.1,133.1,124.9,118.9,113.6,110.8,110,108.8,105.2,103.6,100.7,100.7,99.4,99.3,96.73,97.1,95.9,97.18,94.05,93.34,93.32,92.37,92.02,92.56,91.88,91.45,91.1,91.37,90.98,90.24,90.02,91.05,90.15,89.64,88.67,88.73,88.79,87.58,88.03,87.56,87.7,87.22,86.91,86.33,86.59,87.01,88.58,82.08,81.76,81.51,81.28,81.11,80.99,80.88,80.77,80.68
214 | lstm,penntreebank,2,430,0.1,15000,291.4,191.2,158.6,143.1,135.6,122.1,115.9,112.1,106.2,104.7,102.5,99.75,97.99,96.64,95,95.24,93.08,92.38,91.42,91.04,92.74,90.53,89.54,88.24,86.65,87.05,86.45,86.12,85.75,85.59,85.15,85.05,85.45,83.87,84.55,84.03,84,83.51,83.41,82.42,83.04,82,81.95,81.8,82.19,81.3,81.73,81.59,81.01,81.04,81.64,80.45,82.3,76.33,75.93,75.75,75.58,75.42,75.3,75.19
215 | lstm,penntreebank,2,670,0.1,15000,293.9,192,157.6,142.8,129,120.1,113.8,107.5,102.9,101.7,99.91,99.31,94.27,92.98,92.07,91.58,90.59,92.35,87.84,88.49,88.22,86.6,85.17,84.78,84.72,83.79,83.02,83.05,83.25,82.36,82.31,81.33,82.28,83.39,76.75,76.43,76.15,75.92,75.74,75.55,75.39,75.23,75.09,74.97,74.85,74.74,74.63,74.54,74.45,74.36,74.27,74.19,74.12,74.05,73.97,73.9,73.83,73.76,73.69,73.62
216 | lstm,penntreebank,2,910,0.1,15000,298.6,195.1,159.3,139.9,126.5,119.2,110.1,105.5,100.5,101.2,95.92,93.79,92.08,91.01,89.07,88.57,87.24,86.18,85.49,85.97,86.43,83.79,82.73,82.21,82.09,81.83,81.45,81.9,79.91,80.16,79.8,79.28,80.76,80.06,79.38,79.75,78.68,78.43,78.37,77.78,77.62,77.33,76.91,77.99,77.27,77.51,76.71,76.49,76.39,77.56,72.31,71.99,71.73,71.56,71.39,71.26,71.14,71.04,70.96,70.87
217 | lstm,penntreebank,2,1150,0.1,15000,292.9,207.4,155.5,139.3,125.5,116,110.4,103.3,100,96.91,95.04,92.81,91.31,88.74,88.58,87.31,87.07,85.54,84.5,84.21,85.54,82.29,82.05,80.84,81.75,80.34,80.39,80.29,79.85,79.96,79.08,78.47,78.43,77.78,78.13,78.2,77.16,76.84,76.79,76.61,76.28,76.01,76.72,76.53,76.32,75.9,75.67,75.24,75.05,75.18,74.91,74.95,75.74,71.04,70.58,70.33,70.16,69.98,69.8,69.67
218 | lstm,penntreebank,2,1390,0.1,15000,292.6,189.7,154.9,139.4,123.9,115.3,108.3,103.4,98.94,97.07,94.65,92,90.43,87.83,87.89,86.76,85.25,85.66,83.88,83.4,82.48,81.55,81.44,81.25,80.59,79.27,79.49,79.38,78.57,78.74,77.5,77.56,78.99,77.73,76.94,79.18,72.67,72.22,71.93,71.7,71.52,71.35,71.22,71.11,71.01,70.9,70.8,70.71,70.62,70.52,70.43,70.34,70.26,70.19,70.12,70.04,69.96,69.89,69.82,69.75
219 | lstm,penntreebank,3,230,0.1,15000,342.9,216.8,179.2,161.7,144.2,137.7,128.8,122.9,116.5,116.7,113.1,108.2,105.9,105.3,102.4,102.6,101.6,102.3,99.03,96.95,99.18,97.09,95.13,95.21,94.47,93.48,95.87,92.28,92.21,92.01,91.44,90.83,91.6,89.5,90.22,90.62,90.36,88.63,88.54,88.59,88.68,87.18,87.45,86.74,87.59,88.04,86.52,86.48,86.26,86.41,86.78,81.65,81.3,81.06,80.84,80.68,80.53,80.4,80.32,80.23
220 | lstm,penntreebank,3,470,0.1,15000,353,215,174.6,152.6,134.8,126.4,121.1,113.3,107.9,107.9,104.2,99.71,97.92,95.73,94.62,94.22,93.59,92.76,89.23,88.61,90.84,88.7,86.65,86.15,85.64,86.8,85.54,85.18,84.12,84.23,83.11,82.43,83.61,81.62,81.88,82.46,82.21,81.5,80.41,80.95,80.94,79.67,80.6,79.79,80.26,79.22,78.55,79.53,78.67,78.5,78.98,78.28,78.1,78.31,77.91,77.4,78.44,78.13,77.39,78.42
221 | lstm,penntreebank,3,710,0.1,15000,367.1,209.2,166.2,150.2,134.9,124.3,115.1,109.8,104.8,103.1,99.31,95.5,94.14,91.65,90.91,90.4,88.9,87.27,87.07,84.34,85.58,84.05,83.61,82.23,81.52,81.25,81.53,80.76,80.37,79.83,79.44,78.55,79.3,78.58,77.62,80.84,73.67,73.21,72.86,72.59,72.39,72.22,72.07,71.94,71.8,71.67,71.54,71.43,71.31,71.2,71.1,70.99,70.89,70.79,70.69,70.61,70.52,70.44,70.36,70.29
222 | lstm,penntreebank,3,950,0.1,15000,342.1,210.7,166.7,147.5,135.4,120.7,114.3,107.7,103,99.76,97.65,93.93,92.28,89.91,89.19,87.44,86.35,85.47,84.19,83.06,83.41,82.28,81.76,81.13,79.68,79.04,79.94,78.48,78.35,77.51,78.77,76.91,77.16,76.86,76.07,76.71,76.12,75.46,75.16,74.74,74.62,74.71,74.64,73.73,74.55,73.55,73.35,73.31,72.93,73.49,73.61,72.46,72.39,73.4,68.63,68.26,68.01,67.83,67.67,67.53
223 | lstm,penntreebank,3,1190,0.1,15000,355.4,217.1,167.2,150.7,129.6,122.3,114,106.8,101.7,98.56,96.83,92.48,90.05,88.72,87.15,86.06,85.16,83.49,83.14,82.02,83.74,80.62,79.9,79.31,78.71,77.92,77.47,77.16,76.6,75.83,75.78,75.3,75.62,74.97,75.11,75.49,75.6,73.54,74.09,73.14,73.43,72.67,72.93,72.32,72.83,72.46,71.85,71.63,71.36,71.68,72.17,71.13,71.78,71.76,67.32,66.99,66.7,66.48,66.32,66.2
224 | lstm,penntreebank,4,30,0.1,15000,601.2,393.3,301.6,266.8,238.3,219.9,228.6,197.1,192.6,186.1,187.2,174.2,171.4,168.2,166,169.1,171.5,169.9,160.6,160.3,161,161.2,155,154.2,154,153.1,152.3,153.7,151,150.6,150.8,150.8,156,141.8,141.2,140.7,140.4,140.1,139.8,139.6,139.4,139.2,139,138.8,138.6,138.4,138.3,138.1,138,137.8,137.7,137.5,137.4,137.3,137.2,137.1,137,136.9,136.8,136.7
225 | lstm,penntreebank,4,270,0.1,15000,551.3,285.7,225.7,184.7,162.9,149.8,141.1,132.4,123.8,121.5,118.9,114,110.1,107.2,106.1,105.9,104.8,105.4,100.6,98.85,101.4,99.81,97.83,96.57,96.06,94.4,96.04,94.63,93.14,93.45,92.2,91.67,94.01,91.1,91.52,92.37,90.64,90.43,90.13,89.39,89.81,89.06,89.94,88.47,88.63,88.8,87.19,87.85,86.97,86.97,87.68,86.86,88.95,82.04,81.61,81.32,81.13,80.98,80.84,80.73
226 | lstm,penntreebank,4,510,0.1,15000,453.3,282,209.5,175.4,153.3,138.9,130.7,121.5,114,113.4,107.7,103.6,101.1,99.03,98,96.32,95.76,94.4,93.2,90.47,92.82,89.88,88.51,87.87,86.72,85.76,86.74,86.92,85.22,85.28,83.58,83.25,84.86,82.27,81.82,85.17,82.13,83.22,81.22,80.92,80.02,79.72,80.17,79.47,79.6,79.65,78.66,78.54,78.3,79.35,79.37,78.59,78.67,73.94,73.54,73.29,73.09,72.97,72.84,72.71
227 | lstm,penntreebank,4,750,0.1,15000,438.8,284.8,203.4,168.5,148.6,135.1,126.2,117.6,111,110.4,105,100.6,98.62,96.15,93.24,92.6,90.79,89.66,87.94,87.14,87.52,86.72,85.93,83.99,82.74,83.19,82.22,82,80.39,80.54,80.18,79.62,79.36,78.52,78.59,78.92,78.32,77.65,77.23,77.2,77.24,75.88,76.56,75.85,76.39,75.39,76.6,74.83,75.51,75.27,75.92,70.7,70.31,70.06,69.85,69.69,69.53,69.41,69.31,69.21
228 | lstm,penntreebank,4,990,0.1,15000,511.9,262.9,201.9,171.9,147.6,135.8,124.9,122,110,108.9,102.4,97.36,96.21,92.88,90.94,90.85,88.62,87.74,85.92,84.84,84.48,83.71,82.47,81.23,80.51,79.48,79.4,79.28,78.34,77.85,77.56,77.13,77.44,75.99,76.54,77.73,76.27,75.38,75.6,74.58,75.02,73.95,73.64,74.26,73.75,73.6,73.01,73.22,72.43,72.51,72.61,72,72.83,72.42,72.57,68.07,67.67,67.44,67.23,67.09
229 | lstm,penntreebank,4,1230,0.1,15000,494.2,264.5,193.8,170,147,135,125.9,116.1,109.4,109.1,102,99.19,95.53,93.26,90.09,89.43,87.53,86.31,84.83,83.2,82.98,82.53,81.73,80.67,79.84,79.18,77.94,77.71,77.45,76.79,76.41,75.79,76.26,75.16,75.3,74.91,74.4,73.74,74.03,73.54,73.25,72.82,72.82,72.43,72.44,72.3,71.53,71.6,71.26,71.84,71.84,71.33,71.19,71.67,67.38,66.97,66.75,66.58,66.42,66.27
230 | lstm,penntreebank,5,70,0.1,15000,751.1,698.9,691.1,711.1,703.1,706.3,568.5,410.1,322.5,275.1,246.5,212.9,203.6,189.3,177.8,173.7,173.1,160,155.2,153.1,157.9,150.6,151.6,144.9,143.9,139.9,146,139.3,138.7,135.1,134.5,132.4,140.2,125.6,124.8,124.2,123.8,123.4,123.1,122.7,122.4,122,121.7,121.4,121.1,120.9,120.7,120.4,120.2,120,119.8,119.6,119.4,119.3,119.1,118.9,118.8,118.6,118.5,118.3
231 | lstm,penntreebank,5,310,0.1,15000,731.7,699.9,475.4,343.1,254.6,229.3,192.4,167.8,152.4,144.2,143.1,135.1,125.3,120.8,118.3,117.8,115.4,112.2,108.2,107.1,110.4,107.7,103.8,103.1,102.2,100.8,98.63,99.28,97.89,96.86,96.62,95.37,98.14,96.77,94.27,94.61,92.99,92.78,93.66,92.34,91.41,90.2,91.29,89.53,90.22,89.66,88.56,89.26,89.86,88.88,89.37,87.75,89.23,83.28,82.8,82.52,82.33,82.16,82.02,81.88
232 | lstm,penntreebank,5,550,0.1,15000,740.9,538.3,336.8,266.5,208,185,160.4,144.8,133.2,131.1,125.5,116.7,112.6,109.4,106.3,105.8,104.3,102.1,98.45,97.85,98.93,96.35,94.4,92.86,91.57,91.3,89.94,90.45,89.85,88.84,87.7,87.35,88.85,86.76,85.71,87.28,86.35,85.55,84.56,84.79,84.21,82.69,83.23,82.27,82.93,83.41,81.56,81.58,81.24,81.02,81.43,82.28,76.13,75.7,75.44,75.25,75.07,74.89,74.75,74.62
233 | lstm,penntreebank,5,790,0.1,15000,723.8,460.9,282.5,234.7,182.5,166.2,153.3,139.2,129.5,124.8,119.5,111.7,108.6,104.7,102.5,100.7,101.1,97.75,93.66,93.59,93.67,90.46,89.51,89.04,87.84,86.47,86.37,85.63,84.41,84.27,83.61,82.48,82.56,82.94,81.77,81.84,81.41,80.8,80.07,79.21,79.77,79.33,79.22,78.42,78.74,78.53,78.59,77.52,77.6,77.44,77.73,76.81,77.01,76.51,76.57,75.53,75.98,75.5,75.58,75.55
234 | lstm,penntreebank,5,1030,0.1,15000,745.3,505.9,321,238.8,188,167.7,151.2,135.6,128.3,123.5,119.3,110.6,106.4,103,99.74,98.56,96.13,94.77,91.95,90.18,90.49,88.81,86.96,86.79,86.04,84.2,83.66,84.12,82.11,82.29,81.81,80.93,80.25,79.59,80.74,79.53,80.14,77.94,77.92,77.25,77.5,76.9,76.72,76.26,76.14,75.7,74.85,75,75.06,75.35,75.03,74.21,73.62,74.11,73.8,73.35,73.94,73.49,73.55,72.92
235 | lstm,penntreebank,5,1270,0.1,15000,734.6,407.5,260.4,214.3,172.9,155.8,143.7,130.6,121.3,118.7,112.5,105.9,103.6,100.1,97.5,96.17,94.05,93.46,89.44,88.1,88.46,86.67,85.53,84.69,83.49,82.42,81.72,82,80.41,79.49,79.67,79.08,79.11,77.88,77.97,77.54,76.87,76.5,76.47,76.06,75.6,75.25,74.72,74.66,75.33,74.81,74.16,74,73.84,73.62,73.98,73.28,73.6,72.85,72.98,72.29,72.91,72.33,72.64,72.58
236 | lstm,penntreebank,2,210,0.1,15000,279.7,195.1,163.2,150.8,135.9,128.2,123.3,117.2,113,110.1,109.1,104.8,104.1,101.9,100.3,99.45,99.66,97.32,96.36,95.94,99.59,95.95,96.45,93.26,94.76,92.22,93.66,92.27,91.26,90.62,89.94,89.96,90.33,89.19,89.06,89.04,88.62,88.27,88.83,88.07,88.49,87.96,88.21,88.07,87.8,86.27,86.43,86.55,86.45,86.54,87.34,85.56,87.03,81.45,81.02,80.78,80.53,80.34,80.2,80.11
237 | lstm,penntreebank,2,450,0.1,15000,289.6,197,157.6,144,130.6,121.8,116.5,109.4,106.2,104.7,101.6,101,96.79,95.53,94.66,93.2,92.45,91.87,90.82,90.57,92.65,88.87,90.64,86.99,87.19,86.34,85.76,86.36,86.27,84.67,84.98,83.62,84.42,84.03,83.59,85.34,79,78.52,78.19,77.93,77.71,77.53,77.37,77.23,77.09,76.97,76.87,76.76,76.67,76.57,76.47,76.38,76.29,76.2,76.1,76.02,75.94,75.86,75.79,75.71
238 | lstm,penntreebank,2,690,0.1,15000,287.4,200.3,155.1,141.4,126.1,118.6,113.6,106.5,103.6,101.6,97.59,95.53,94.37,94.49,90.89,90.72,88.9,88.57,87.2,86.7,88.27,85.84,86.41,85.49,83.94,82.93,83.3,83.53,82.13,82.1,83.47,81.65,82.13,81.23,80.36,83.53,76.28,75.86,75.54,75.28,75.1,74.92,74.77,74.61,74.47,74.35,74.24,74.12,74.01,73.91,73.8,73.7,73.6,73.51,73.42,73.33,73.26,73.19,73.12,73.05
239 | lstm,penntreebank,2,930,0.1,15000,281.7,190.3,157.1,138.2,128.8,118.4,111.1,105.6,101.1,98.72,99.06,94.27,92.78,90.24,88.95,89.31,86.85,86.91,85.53,85.88,86.03,83.88,84.67,82.75,82.72,81.64,82,81.47,80.92,81,81.38,80.5,80.92,79.27,79.22,80.15,79.8,78.67,78.63,78.17,78.71,77.58,77.62,77.78,77.88,77.37,76.7,76.43,76.75,77.19,77.86,72.51,72.14,71.87,71.64,71.47,71.34,71.21,71.11,71.01
240 | lstm,penntreebank,2,1170,0.1,15000,282.9,196.8,155.9,139.4,124.4,117,109.3,104,100.4,99.09,95.39,92.79,90.82,90.04,88.41,87.4,86.14,85.21,83.92,84.45,83.93,83.16,83.55,81.76,81.34,82.12,80.16,80.33,79.59,79.1,78.86,78.8,79.53,77.58,78.2,77.94,77.62,77.15,76.81,76.89,77.36,75.94,76.33,76.43,76.11,75.88,75.59,75.42,76.06,71.02,70.6,70.32,70.11,69.95,69.82,69.72,69.62,69.53,69.43,69.35
241 | lstm,penntreebank,3,10,0.1,15000,499.8,375.2,318.4,319.2,279.7,269.9,273.1,256.9,246.2,242,246.1,235.9,231.5,229.5,228.2,240.3,245.5,214.2,213,211.7,210.8,210.1,209.6,208.6,208,207.7,207,206.5,206,205.6,205.2,204.9,204.7,204.7,204.6,204.4,204.3,204.3,205.5,206,207,211.6,218.4,221.9,223.1,223.9,224.6,223,222,221.1,220.7,219.7,218.9,217.8,216.7,216.3,214.7,213.5,212.4,211.4
242 | lstm,penntreebank,3,250,0.1,15000,359.7,218.8,183.7,163.3,142.6,137.5,129.2,121.8,115.3,113.9,112.4,107.6,104.4,108,101.6,101.3,99.48,100.7,99.01,95.71,98.14,96.98,94.96,93.5,93.36,93.11,92.29,92.61,91.39,92.32,90.62,90.33,91.13,92.01,89.32,90.7,88.83,88.4,87.82,90.14,88.4,86.31,87.72,86.03,87.49,86.91,85.49,85.89,85.06,85.93,85.86,85.11,85,86.27,80.46,80.12,79.9,79.7,79.53,79.4
243 | lstm,penntreebank,3,490,0.1,15000,377,218.8,171,151.2,136,126.7,119,112.6,108.9,106.1,102.7,99.28,97.24,95.22,93.49,93.86,92.46,93.2,89.9,88.33,91.78,87.44,86.92,86.02,85.14,84.15,84.94,83.87,83.01,83.13,82.75,82.52,81.91,81.46,81,81.83,81.23,79.89,79.91,79.82,79.58,79.2,79.36,79.15,79.04,78.54,78.01,78.06,77.94,77.63,78.54,77.76,77.12,77.73,78.68,73.21,72.75,72.53,72.37,72.21
244 | lstm,penntreebank,3,730,0.1,15000,331.7,216.1,170.7,148.2,131.3,122.2,116.4,109.8,103.8,102.6,101.1,95.97,94.24,91.29,90.7,88.79,88.3,87.59,85.36,84.87,86.07,83.76,83.09,81.97,81.64,80.3,80.3,81.23,79.64,79.2,78.95,78.14,78.68,77.26,77.49,78.12,76.76,76.94,76.85,76.33,75.89,75.92,75.77,76.03,75.45,74.98,74.74,74.51,74.86,75.6,74.63,75.16,70.11,69.75,69.53,69.36,69.19,69.04,68.93,68.83
245 | lstm,penntreebank,3,970,0.1,15000,350.9,211.7,169.2,148.8,134.5,123,116,107.9,102.5,99.15,98.59,96.13,92.15,89.73,89.01,87.76,86.36,85.25,83.71,83.48,83.22,82.46,81.27,80.07,79.36,78.85,78.65,78.41,77.26,77.17,77.02,76.64,77.43,75.88,76.04,76.49,75.6,75.11,75.03,74.52,74.37,73.78,73.88,73.48,73.62,73.32,72.71,72.47,72.56,73.19,72.81,72.06,72.72,68.34,67.89,67.67,67.49,67.34,67.23,67.11
246 | lstm,penntreebank,3,1210,0.1,15000,337.8,205.5,165.1,146.2,133.1,120.3,114.1,105.6,103.9,101.6,95.67,92.11,90.09,88.49,86.85,85.87,84.5,83.12,82.12,81.79,81.58,80.3,79.56,78.87,78.66,77.37,78.41,76.31,75.87,75.6,75.26,75.72,75.13,74.36,74.18,74.56,73.7,73.05,73.32,72.57,72.96,72.66,71.97,72.41,72.33,71.72,71.35,71.19,71.09,70.67,71.5,70.54,71.08,70.81,70.48,70.03,70.06,70,69.9,69.99
247 | lstm,penntreebank,4,50,0.1,15000,597.4,372.6,280.1,245.4,213.7,200.9,181.4,171,168.5,158.8,158.9,152.5,149.4,147.9,143.3,144.5,146.6,140.8,138.6,136.5,140.1,136.9,134.5,133.3,131.8,130.4,133.3,131.2,130.7,130.7,129.2,128.5,129.9,127.8,128.6,128.6,125.2,125.5,126,128.5,118.7,118,117.5,117.2,117,116.8,116.7,116.5,116.4,116.3,116.1,116,115.8,115.7,115.6,115.5,115.4,115.3,115.1,115
248 | lstm,penntreebank,4,290,0.1,15000,511.2,276.2,208.6,184.6,157.2,149.5,140.8,130.1,122.4,120.8,117,113.2,108.9,110.7,105.5,106.3,103.2,103,100,98.54,100.1,98.67,96.09,95.66,94.61,93.96,93.5,92.7,92.37,92.37,90.97,90.7,91.16,89.64,90.35,91.41,88.97,91.18,83.96,83.49,83.19,83,82.84,82.66,82.49,82.33,82.18,82.04,81.92,81.81,81.71,81.6,81.49,81.39,81.3,81.19,81.09,80.99,80.9,80.81
249 | lstm,penntreebank,4,530,0.1,15000,414.1,288.8,201.6,175.2,155.7,141.4,130.7,119.7,112.8,109.7,107.4,103,101,98.36,96.69,96.46,95.09,94.24,91.23,90.13,91.77,89.67,88.87,86.94,87.37,85.2,85.78,85.02,84.35,84.28,83.43,83.16,82.74,82.07,83.17,82.31,81.94,81.24,80.52,80.44,80.68,79.37,79.84,79.73,79.2,79.53,79.37,78.34,77.94,78.91,78.85,77.5,78.53,77.75,77.98,73.38,73.06,72.83,72.63,72.47
250 | lstm,penntreebank,4,770,0.1,15000,521,270.1,199.7,173.8,150.3,134.3,125.1,115.8,111.7,107,103.7,99.78,96.71,95.22,93.04,92.07,90.23,88.62,87.66,86.73,87.32,85.85,84.52,83.32,83.02,82.38,82.43,82.02,80.45,80.07,79.49,78.73,79.74,78.78,78.31,78.63,77.63,78.01,77.15,76.76,77.95,76.36,76.38,75.74,75.49,75.91,75.06,74.76,74.55,75.34,75.31,74.21,74.58,74.81,70.25,69.83,69.57,69.36,69.23,69.11
251 | lstm,penntreebank,4,1010,0.1,15000,458.1,263,201.8,167.6,145.9,132.8,123.9,115.4,109.8,109,102.6,98.82,96.94,94.11,91.47,90.86,88.69,87.2,88.84,85.4,85.05,84.04,82.57,82.15,81.4,80.24,80.14,80.02,78.95,78.49,77.97,77.5,77.9,78.05,76.85,77.09,76.44,75.8,75.08,74.53,75.29,74.32,73.86,73.83,74.17,74.06,73.21,73.63,72.77,73.72,73.33,72.67,72.55,72.33,71.78,71.59,71.57,71.7,71.96,71.52
252 | lstm,penntreebank,4,1250,0.1,15000,442.1,258.3,192.4,168.9,144.1,133.1,123.8,116.2,108.3,104.5,102.1,97.85,94.42,92.06,89.98,89.61,87.89,86.62,84.13,85.03,84.18,82.37,81.11,80.5,79.55,79.19,78.08,78.62,77.44,77.15,76.63,75.7,75.97,75.01,75.18,75.58,74.68,73.86,73.8,73.3,73.2,73.61,72.72,72.29,72.66,72.46,72.21,71.46,71.72,71.41,72.38,67.72,67.29,67.01,66.82,66.64,66.47,66.34,66.21,66.1
253 | lstm,penntreebank,5,90,0.1,15000,746,699.5,691.6,711.3,447.6,359.4,306.3,241.3,211.4,200.7,189.6,172.5,167.6,160.8,154.9,152.8,153.3,148.5,140.7,139.1,143.8,138.6,137.9,135.1,133.1,129.3,135.2,129.3,128.5,125.9,123.7,123.3,126.6,122.7,123.6,121.9,120.9,122.5,122.7,120.2,120.4,118.6,120.2,121.9,111.3,111.1,110.7,110.4,110.2,109.9,109.7,109.5,109.3,109.2,109,108.9,108.8,108.6,108.5,108.4
254 | lstm,penntreebank,5,330,0.1,15000,725.7,654.8,413.3,305.7,230.4,201.4,177,158.1,145.3,138.7,136.9,126.4,121.3,118.7,114.2,114.3,111.9,113.7,105.8,104.9,105.4,104.4,105.3,100.5,98.68,97.91,96.84,96.2,97.23,95.23,94.46,93.41,95.23,94.21,93.31,94.68,92.07,92.53,91.18,91.1,91.4,89.74,90.5,89.05,90.21,89.01,88.48,87.81,88.32,87.56,89.01,89.05,82.38,82.04,81.78,81.58,81.42,81.25,81.11,81
255 | lstm,penntreebank,5,570,0.1,15000,738.2,449.9,307.7,249.9,192.5,174,159.9,141.3,132.2,125.8,122.7,115.6,110.7,108.5,105.7,103.9,102.7,102.2,97.42,96.2,98.24,95.29,93.59,91.63,91.35,90.43,89.46,89.17,88.2,87.95,87.21,86.57,87.57,84.5,84.82,85.45,83.82,83.29,83.48,82.43,83.31,81.78,81.51,80.94,81.51,81.01,80.25,80.24,80.55,79.58,80.22,79.37,80.31,75.03,74.59,74.32,74.13,73.95,73.82,73.69
256 | lstm,penntreebank,5,810,0.1,15000,721.9,364.9,275.7,222.5,185,163.7,148.2,135,126,121.7,117.8,111.4,108,104.2,100.2,99.86,99.27,95.43,93.11,91.09,91.39,89.56,89.03,87.62,86.45,85.35,85.65,85.15,83.38,83.45,82.19,82.01,82.47,81.53,81.14,81.89,80.39,80.14,79.43,78.98,80.27,77.99,78.5,79.2,78.48,77.88,77.94,77.2,76.63,76.63,77.09,76.53,76.03,76.62,75.93,75.86,75.61,75.02,75.08,75.05
257 | lstm,penntreebank,5,1050,0.1,15000,723.1,408.3,272.9,224.3,183.7,163.8,147.8,136.8,124.1,118.5,115.4,108.5,105,101.7,99.68,97.48,95.33,92.86,90.46,90.67,89.72,88.22,86.82,86.58,85.77,84.23,83.49,83.8,81.41,80.44,81.21,79.6,80.06,79.7,79.4,79.04,78.84,77.81,77.68,77.57,77.16,76.21,77.29,76.05,76.07,74.8,75.03,74.64,74.72,75.06,75.01,74.23,73.68,73.97,73.42,73.34,73.14,73.2,73.16,72.4
258 | lstm,penntreebank,5,1290,0.1,15000,737.1,431.7,267.1,214.1,176.8,159.9,144,131.8,122.8,119.8,113.1,108.5,104.2,101.4,97.7,96.88,93.98,93.05,91.73,89.87,88.6,87.18,86.39,85.67,83.96,82.86,83.05,81.88,80.91,80.28,80.3,79.13,79.95,78.78,78.9,78.21,77.16,76.97,76.75,76.08,76.54,75.48,75.16,75.38,75.4,74.31,74.25,73.7,73.96,74.21,74.38,73.86,73.6,73.9,69.17,68.76,68.49,68.32,68.15,68.01
259 | lstm,penntreebank,2,230,0.1,15000,301.8,198.6,161.6,150.1,136.6,128.4,123,115.9,113.5,112.5,107.3,104.4,102.4,100.7,99.43,98.84,98.58,96.94,95.72,95.34,96.47,93.47,93.36,91.95,91.35,91.06,90.78,93.39,90.04,89.43,89.49,88.87,89.49,88.45,88.28,88.18,89.09,88.38,87.15,86.58,86.74,86.18,86.65,86.36,86.82,87.37,81.44,80.94,80.67,80.45,80.29,80.16,80.03,79.93,79.84,79.75,79.65,79.56,79.49,79.41
260 | lstm,penntreebank,2,470,0.1,15000,303.8,207.9,159.3,142.4,129.4,120.4,116.6,108.6,105.4,102.2,101.7,97.43,96.43,95.14,93.52,93.48,91.5,90.87,89.12,88.71,90.57,89.23,87.63,86.24,86.6,86.18,84.65,85.62,84.43,83.94,83.75,83.62,83.39,86.38,78.57,78.17,77.87,77.61,77.42,77.26,77.1,76.95,76.83,76.7,76.59,76.48,76.37,76.27,76.18,76.09,76,75.91,75.83,75.74,75.67,75.59,75.52,75.44,75.37,75.3
261 | lstm,penntreebank,2,710,0.1,15000,285.2,189.4,161.1,140.1,129.6,117.9,112.4,105.5,103,100.4,98.68,96.58,95.91,93.13,91.18,91.23,89.86,88.62,87.31,88.27,89.36,86.61,85.69,84.31,83.44,83.49,84.01,84.07,82.86,82.26,81.54,81.68,81.39,81.07,80.68,82.64,76.08,75.68,75.4,75.17,74.97,74.79,74.63,74.5,74.4,74.29,74.19,74.09,74,73.92,73.83,73.75,73.67,73.6,73.52,73.45,73.38,73.31,73.24,73.19
262 | lstm,penntreebank,2,950,0.1,15000,297.2,195.1,158.1,143.2,128.6,118,111.2,107.4,101.1,101.2,96.36,94.23,91.69,90.38,89.17,88.51,88.11,85.89,85.52,85.75,87.75,84.36,83.88,82.47,82.67,83.9,81.91,81.63,81.22,80.81,80.27,79.68,80.1,79.48,79.25,79.07,79.07,77.82,78.6,77.37,77.92,77.63,76.85,77.39,77.26,77.76,73.11,72.8,72.54,72.29,72.08,71.93,71.78,71.66,71.54,71.44,71.34,71.25,71.16,71.06
263 | lstm,penntreebank,2,1190,0.1,15000,299.4,199.7,156.9,139.7,132,116.6,109.7,104.2,101.1,96.73,95.91,91.52,90.51,89.08,89.58,87.16,86.03,85.07,85.07,84.62,84.13,82.15,82.85,80.94,80.93,80.52,81.05,79.83,79.29,79.01,78.47,78.33,78.24,78.29,77.76,78.18,78.35,77.21,77.18,76.17,76.58,76.93,76.82,75.58,76.03,76.02,75.45,75.16,75.14,75.01,75.99,71.06,70.61,70.34,70.15,69.96,69.81,69.69,69.6,69.51
264 | lstm,penntreebank,3,30,0.1,15000,423.4,276.8,234.1,210.2,190.1,180.6,176.1,168.5,160.3,155.5,155.7,150.1,148.8,146.1,144.5,147.5,145.7,142.2,139.8,139.6,140.2,139,135.5,134.6,133.7,131.9,131.8,133.5,132.8,131,131.1,129.9,132.1,123.1,122.7,122.4,122,121.8,121.5,121.3,121.1,120.9,120.9,120.7,120.5,120.3,120.2,120,119.9,119.8,119.6,119.5,119.4,119.3,119.2,119.1,119,118.9,118.8,118.7
265 | lstm,penntreebank,3,270,0.1,15000,365.3,213.2,174.8,155.1,140.4,135.5,125.4,119.5,114.2,111.3,109.8,106.8,104.6,103,101.2,100.5,99.29,103.2,96.66,95.6,97.46,95.62,93.35,92.96,92.61,91.34,93.3,91.1,89.92,90.13,88.92,89.38,91.29,91.71,83.9,83.5,83.14,82.82,82.59,82.38,82.19,82.02,81.83,81.69,81.55,81.41,81.27,81.14,81.02,80.89,80.77,80.65,80.54,80.43,80.33,80.23,80.14,80.05,79.95,79.87
266 | lstm,penntreebank,3,510,0.1,15000,377.3,227.4,169.1,153,136.1,127.9,120.1,117.5,108.2,106.7,102,99.42,97.64,95.38,98.16,93.06,91.61,92.7,89.98,88.02,90.59,88.22,86.59,85.77,84.79,85.09,84.58,83.42,83.12,83.13,82.04,81.54,81.62,81.27,80.65,81.98,82.35,76.24,75.78,75.45,75.17,74.97,74.8,74.62,74.47,74.32,74.18,74.05,73.93,73.82,73.7,73.59,73.5,73.4,73.31,73.22,73.14,73.05,72.96,72.88
267 | lstm,penntreebank,3,750,0.1,15000,341.8,207.1,166.4,150.7,134.8,124.5,116,109.2,104.2,102.3,99.02,95.54,93.31,91.03,90.43,89.72,87.72,87.02,85.34,84.5,85.03,82.95,83.62,81.78,81.78,80.65,80.38,79.9,79.12,78.73,79.3,78.94,79.48,78.13,77.52,77.62,77.54,76.89,76.28,76.18,76.13,75.52,75.54,74.84,75.01,74.89,74.6,74.16,74.25,74.03,74.48,73.64,73.97,73.59,73.3,73.64,72.79,72.93,73.04,72.62
268 | lstm,penntreebank,3,990,0.1,15000,358.7,212.1,168,148.3,131,120.6,114.2,107,101.4,98.86,97.19,93,91.55,89.78,87.71,87.26,85.75,85.19,83.23,82.7,83.55,81.81,80.93,80.17,79.75,78.53,78.76,79.23,77.65,77.39,76.32,76.28,76.3,75.88,75.53,75.7,75.43,76.25,74.76,74.99,74.93,73.65,73.45,74.87,74.01,73.2,73.06,72.54,72.78,72.89,72.88,72.22,73.35,68.36,67.96,67.68,67.48,67.32,67.17,67.05
269 | lstm,penntreebank,3,1230,0.1,15000,330,217,192.3,147.4,130.5,121.6,112.5,106,101.5,97.75,95.41,92.1,90.06,87.78,86.65,85.71,84.36,83.45,81.87,81.19,81.82,79.91,79.47,78.95,77.77,76.95,76.68,76.29,76.1,75.68,75.02,74.8,74.92,73.73,74.31,74.06,73.98,73.25,72.7,72.6,72.26,71.93,72.33,71.88,71.75,71.62,70.98,71.17,71.09,71.89,67.57,67.2,66.93,66.68,66.46,66.3,66.16,66.04,65.93,65.82
270 | lstm,penntreebank,4,70,0.1,15000,649.2,331.5,256.7,221,193.1,181.3,176,157.1,150.7,151,147.4,140.7,139.5,134.2,133,133.4,133.3,133.4,125.4,124.8,124,127.3,124.4,123.5,120.8,119.6,119.2,119.3,119.2,117.5,116.8,116.4,117.8,115.6,117.3,116.5,116.3,116.4,108.3,107.9,107.6,107.4,107.2,107,106.9,106.7,106.5,106.4,106.3,106.1,106,105.9,105.8,105.7,105.6,105.5,105.4,105.3,105.2,105.1
271 | lstm,penntreebank,4,310,0.1,15000,539.5,280.2,212.5,182.1,156.9,150.3,139,128.8,121.7,119.8,116.8,111.7,109,106.5,103.6,103.3,102.2,100.9,99.56,97.86,98.91,97.21,96.43,95.25,93.49,91.95,93.69,92.49,92.39,91.89,90.5,89.76,91.04,89.19,90.22,89.66,89.16,88.28,88.86,87.81,88.31,86.25,86.85,86.85,87.71,85.96,85.31,85.16,85.24,86.38,80.53,80.09,79.85,79.63,79.46,79.32,79.19,79.05,78.94,78.84
272 | lstm,penntreebank,4,550,0.1,15000,494.8,264.5,196,171,154,139.1,129.6,121,113.7,109.1,108.8,104,100.4,97.94,95.49,96.03,94.61,92.58,90.37,90.11,90.74,89.34,87.68,86.28,86.17,84.94,85.46,84.67,83.46,83.76,83.24,82.65,83.18,85.1,77.18,76.71,76.38,76.12,75.94,75.77,75.6,75.43,75.28,75.13,74.99,74.85,74.72,74.59,74.48,74.36,74.25,74.14,74.03,73.93,73.83,73.74,73.64,73.55,73.46,73.39
273 | lstm,penntreebank,4,790,0.1,15000,440.1,251,196.8,168.2,146.7,135.4,124.2,116.6,111.2,106.8,104.6,100,98.84,95.53,93.09,91.77,90.8,90.8,87.62,87.21,88.72,85.92,85.22,83.83,82.54,82.13,81.32,81.71,80.3,80.22,79.95,79.17,79.35,78.56,78.78,79.54,77.9,77.9,77.15,76.53,76.67,75.83,77.19,75.35,75.02,74.94,74.65,74.42,74.82,74.3,75.29,70.22,69.86,69.59,69.39,69.22,69.07,68.94,68.84,68.74
274 | lstm,penntreebank,4,1030,0.1,15000,468.2,263.2,197.2,169.5,144,135,123.8,115.8,109.3,106.4,101.6,98.05,95.91,93.11,90.97,90.47,88.94,88.74,85.72,85.12,85.06,83.52,82.6,81.42,80.86,80.14,80.46,79.22,78.52,78.36,77.61,77.36,77.76,76.89,76.45,76.27,75.5,75.18,74.89,74.48,75.05,73.99,73.92,74.06,74.1,73.71,73.08,72.85,72.72,73.27,72.88,72.57,72.53,73.02,68.27,67.92,67.65,67.44,67.25,67.09
275 | lstm,penntreebank,4,1270,0.1,15000,466.9,263.2,194.3,167.3,143.2,134.4,125.3,115.2,108.2,105,102.5,97.54,94.32,91.83,90.42,89.42,87.48,86.56,84.91,83.55,82.82,82.18,81.45,80.36,79.79,78.94,79.44,77.91,77.19,76.8,76.05,75.64,76.53,75.07,74.68,75.35,74.32,73.98,74.03,73.49,73.8,73.41,73.51,73.58,72.83,72.4,71.91,71.8,71.52,71.23,71.76,71.24,71.05,71.23,70.46,70.21,70.24,70.39,70.27,70.15
276 | lstm,penntreebank,5,110,0.1,15000,749.1,699.2,641.2,495.8,354.5,293.5,249.1,207.4,183.4,173.1,168.1,162.1,152.1,146.9,143,145.2,140.3,140.3,132.8,129.2,130.8,129.9,124.5,124.9,122.4,120.9,121.3,119.4,118.9,119.4,117.4,116,119.1,116.1,115.2,118.2,115.1,114,113.5,113.8,113,111.6,112.4,111.4,111.9,112.7,109.8,109.7,109.9,108.9,110.4,107.7,107.8,108.2,109.3,107.1,107.2,108.3,101.4,101
277 | lstm,penntreebank,5,350,0.1,15000,728.9,671.4,423.4,306.1,238.9,211.9,179.5,162.7,148.1,140.8,136.8,126.4,122.7,118.4,115.4,114.1,112.3,111.6,105.7,104.6,104.1,103,102.1,100.6,99.41,97.31,96.96,96.28,95.95,94.66,94.48,93.42,94.45,92.62,92.11,93.78,91.51,90.7,90.31,89.97,90.51,88.94,88.7,88.5,88.65,88.36,87.91,87.28,87.32,86.91,88.21,86.18,86.56,85.52,86.19,85.5,85.54,85.86,86.34,80.48
278 | lstm,penntreebank,5,590,0.1,15000,728.5,474.2,325.9,252.1,196,172.7,162,142,132.7,130.8,123.3,115,111.9,107.9,105.6,103,102.6,100,97.65,96.9,97.83,95.16,93.16,92.2,90.64,89.52,89.77,89.67,88.16,87.75,87.03,85.47,86.07,85.13,85.11,84.99,84.81,83.6,83.77,83.73,84.05,83.73,81.93,82.07,81.79,81.92,80.23,81.19,80.71,79.89,80.77,79.64,81.38,75.13,74.73,74.48,74.29,74.17,74.07,73.97
279 | lstm,penntreebank,5,830,0.1,15000,726.1,483.2,312.7,233.4,190.4,170.5,150.8,138.5,127,122,116.7,110,107.9,104.4,101.1,99.83,97.05,94.94,93.48,91.83,91.14,90.87,88.06,87.49,86.43,85.44,85.51,84.76,84.18,82.94,82.41,81.97,82.25,81.13,81.15,81.46,80.4,79.34,79.52,78.3,79.23,77.8,78.29,77.76,78.08,77.19,77.49,76.77,77.44,76.31,77.14,75.48,76.49,76.08,75.26,74.68,75.35,74.43,75.4,74.39
280 | lstm,penntreebank,5,1070,0.1,15000,727.3,414.8,272.2,226,180.3,162,149,137.7,126.6,118.5,114.6,108.4,105.7,101.3,98.58,96.74,95.73,93.22,90.78,89.77,89.4,87.89,87.9,85.5,84.97,83.83,84.46,83.11,82.01,81.57,81.07,80.33,80.43,80.09,79.75,79.55,78.48,77.83,77.84,77.27,77.5,76.88,77,76,76.86,75.59,75.18,74.72,74.75,74.66,74.94,74.07,74.06,74.34,74.28,73.84,73.88,73.11,73.13,74.01
281 | lstm,penntreebank,5,1310,0.1,15000,728.4,509.7,310.8,233.2,187.4,164.4,151.1,135.4,123.7,116.8,113.1,106.7,103.4,99.68,98.53,95.05,92.75,92.91,89.18,88.39,88.12,86.93,85.08,84.23,83.45,82.32,81.25,81.57,80.37,79.83,78.99,78.8,79.15,78.15,78.01,77.41,76.92,76.39,76.45,76.29,75.84,74.97,75.19,74.93,74.61,74.86,73.77,73.91,73.67,73.37,74.1,73.4,73.32,73.78,69.15,68.73,68.46,68.25,68.1,67.95
282 |
--------------------------------------------------------------------------------