└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # deeplearning-default-value 2 | **WELCOME FOR YOUR PR!!** 3 | 4 | ## Functions 5 | 6 | | Object names in Chainer | Parameter names in Chainer | Chainer | PyTorch | TensorFlow | 7 | |:----------------------- |:---------------------------|:-----------:|:-----------:|:--------------:| 8 | | dropout | ratio | [0.5](https://docs.chainer.org/en/stable/reference/generated/chainer.functions.dropout.html)
(drop ratio)| [0.5](https://pytorch.org/docs/stable/nn.html?highlight=dropou#torch.nn.functional.dropout)
(drop ratio)| [Required](https://www.tensorflow.org/api_docs/python/tf/nn/dropout)
(keep ratio)| 9 | 10 | ## Links 11 | 12 | | Object names in Chainer | Parameter names in Chainer | Chainer | PyTorch | TensorFlow | 13 | |:----------------------- |:---------------------------|:-----------:|:-----------:|:--------------:| 14 | |Link | initilizer | [LecunNormal](https://docs.chainer.org/en/stable/reference/initializers.html#weight-initializers)| Each link has
a different
initializer.| [GlorotUniform](https://www.tensorflow.org/api_docs/python/tf/get_variable)| 15 | 16 | ## Initializers 17 | - PyTorch initializers are defined in each layer implementation, so skip it. 18 | 19 | | Object names in Chainer | Parameter names in Chainer | Chainer | TensorFlow | 20 | |:----------------------- |:---------------------------|:-----------:|:--------------:| 21 | | GlorotUniform | scale | [1.0](https://docs.chainer.org/en/stable/reference/generated/chainer.initializers.GlorotNormal.html)| No args ([1.0](https://www.tensorflow.org/api_docs/python/tf/keras/initializers/glorot_normal))| 22 | | LecunNormal | scale | [1.0](https://docs.chainer.org/en/stable/reference/generated/chainer.initializers.LeCunNormal.html)| No args ([1.0](https://www.tensorflow.org/api_docs/python/tf/keras/initializers/lecun_normal))| 23 | | Normal | scale | [0.05](https://docs.chainer.org/en/stable/reference/generated/chainer.initializers.Normal.html)| [1.0](https://www.tensorflow.org/api_docs/python/tf/initializers/random_normal) or [0.05](https://www.tensorflow.org/api_docs/python/tf/keras/initializers/RandomNormal)| 24 | | Uniform | scale | [0.05](https://docs.chainer.org/en/stable/reference/generated/chainer.initializers.Uniform.html)|min, max = [(0.0, 1.0)](https://www.tensorflow.org/api_docs/python/tf/initializers/random_uniform) or [(-0.05, 0.05)](https://www.tensorflow.org/api_docs/python/tf/keras/initializers/RandomUniform)| 25 | 26 | ## Optimizers 27 | 28 | | Object names in Chainer | Parameter names in Chainer | Chainer | PyTorch | TensorFlow | 29 | |:----------------------- |:---------------------------|:-----------:|:-----------:|:--------------:| 30 | | SGD | lr | [0.01](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.SGD.html) | [Required](https://pytorch.org/docs/stable/optim.html#torch.optim.SGD) |[0.01](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/SGD)| 31 | | Adam | alpha | [0.001](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [0.001](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [0.001](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)| 32 | | | beta1 | [0.9](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [0.9](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [0.9](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)| 33 | | | beta2 | [0.999](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [0.999](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [0.999](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)| 34 | | | eps | [1e-08](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [1e-08](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [1e-08](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)| 35 | | | eta | [1.0](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [No args](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [No args](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)| 36 | | | weight_decay | [0.0](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [0.0](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [No args](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)| 37 | | | amsgrad | [False](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [False](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [No args](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)| 38 | 39 | ## Normalization methods 40 | 41 | | Object names in Chainer | Parameter names in Chainer | Chainer | PyTorch | TensorFlow | 42 | |:----------------------- |:---------------------------|:-----------:|:-----------:|:--------------:| 43 | | BatchNormalization | eps | [2e-05](https://docs.chainer.org/en/stable/reference/generated/chainer.links.BatchNormalization.html) | [1e-05](https://pytorch.org/docs/stable/nn.html#batchnorm2d) |[1e-03](https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization)| 44 | | LayerNormalization | eps | [1e-06](https://docs.chainer.org/en/stable/reference/generated/chainer.links.LayerNormalization.html) | [1e-05](https://pytorch.org/docs/stable/nn.html#layernorm) | No args ([1e-12](https://github.com/tensorflow/tensorflow/blob/a6d8ffae097d0132989ae4688d224121ec6d8f35/tensorflow/contrib/layers/python/layers/layers.py#L2314)) | 45 | | GroupNormalization | eps | [1e-05](https://docs.chainer.org/en/stable/reference/generated/chainer.links.GroupNormalization.html) | [1e-05](https://pytorch.org/docs/stable/nn.html#groupnorm) | [1e-06](https://www.tensorflow.org/api_docs/python/tf/contrib/layers/group_norm)| 46 | | | groups | [Required](https://docs.chainer.org/en/stable/reference/generated/chainer.links.GroupNormalization.html) | [Required](https://pytorch.org/docs/stable/nn.html#groupnorm) | [32](https://www.tensorflow.org/api_docs/python/tf/contrib/layers/group_norm)| 47 | | local_response_normalization | n | [5](https://docs.chainer.org/en/stable/reference/generated/chainer.functions.local_response_normalization.html#chainer.functions.local_response_normalization) | [Required](https://pytorch.org/docs/stable/nn.html?highlight=local_response_norm#torch.nn.LocalResponseNorm) | [5](https://www.tensorflow.org/api_docs/python/tf/nn/local_response_normalization)| 48 | | | k | [2](https://docs.chainer.org/en/stable/reference/generated/chainer.functions.local_response_normalization.html#chainer.functions.local_response_normalization) | [1](https://pytorch.org/docs/stable/nn.html?highlight=local_response_norm#torch.nn.LocalResponseNorm) | [1](https://www.tensorflow.org/api_docs/python/tf/nn/local_response_normalization)| 49 | | | alpha | [1e-04](https://docs.chainer.org/en/stable/reference/generated/chainer.functions.local_response_normalization.html#chainer.functions.local_response_normalization) | [1e-04](https://pytorch.org/docs/stable/nn.html?highlight=local_response_norm#torch.nn.LocalResponseNorm) | [1](https://www.tensorflow.org/api_docs/python/tf/nn/local_response_normalization)| 50 | | | beta | [0.75](https://docs.chainer.org/en/stable/reference/generated/chainer.functions.local_response_normalization.html#chainer.functions.local_response_normalization) | [0.75](https://pytorch.org/docs/stable/nn.html?highlight=local_response_norm#torch.nn.LocalResponseNorm) | [0.5](https://www.tensorflow.org/api_docs/python/tf/nn/local_response_normalization)| 51 | 52 | ## Rules 53 | ### Kinds of symbols 54 | - None: default value is None. 55 | - x: No implementations. 56 | - No args: Magic number. We can not specify the value. 57 | - Required: No default value. We should specify the value. 58 | --------------------------------------------------------------------------------