└── README.md
/README.md:
--------------------------------------------------------------------------------
1 | # deeplearning-default-value
2 | **WELCOME FOR YOUR PR!!**
3 |
4 | ## Functions
5 |
6 | | Object names in Chainer | Parameter names in Chainer | Chainer | PyTorch | TensorFlow |
7 | |:----------------------- |:---------------------------|:-----------:|:-----------:|:--------------:|
8 | | dropout | ratio | [0.5](https://docs.chainer.org/en/stable/reference/generated/chainer.functions.dropout.html)
(drop ratio)| [0.5](https://pytorch.org/docs/stable/nn.html?highlight=dropou#torch.nn.functional.dropout)
(drop ratio)| [Required](https://www.tensorflow.org/api_docs/python/tf/nn/dropout)
(keep ratio)|
9 |
10 | ## Links
11 |
12 | | Object names in Chainer | Parameter names in Chainer | Chainer | PyTorch | TensorFlow |
13 | |:----------------------- |:---------------------------|:-----------:|:-----------:|:--------------:|
14 | |Link | initilizer | [LecunNormal](https://docs.chainer.org/en/stable/reference/initializers.html#weight-initializers)| Each link has
a different
initializer.| [GlorotUniform](https://www.tensorflow.org/api_docs/python/tf/get_variable)|
15 |
16 | ## Initializers
17 | - PyTorch initializers are defined in each layer implementation, so skip it.
18 |
19 | | Object names in Chainer | Parameter names in Chainer | Chainer | TensorFlow |
20 | |:----------------------- |:---------------------------|:-----------:|:--------------:|
21 | | GlorotUniform | scale | [1.0](https://docs.chainer.org/en/stable/reference/generated/chainer.initializers.GlorotNormal.html)| No args ([1.0](https://www.tensorflow.org/api_docs/python/tf/keras/initializers/glorot_normal))|
22 | | LecunNormal | scale | [1.0](https://docs.chainer.org/en/stable/reference/generated/chainer.initializers.LeCunNormal.html)| No args ([1.0](https://www.tensorflow.org/api_docs/python/tf/keras/initializers/lecun_normal))|
23 | | Normal | scale | [0.05](https://docs.chainer.org/en/stable/reference/generated/chainer.initializers.Normal.html)| [1.0](https://www.tensorflow.org/api_docs/python/tf/initializers/random_normal) or [0.05](https://www.tensorflow.org/api_docs/python/tf/keras/initializers/RandomNormal)|
24 | | Uniform | scale | [0.05](https://docs.chainer.org/en/stable/reference/generated/chainer.initializers.Uniform.html)|min, max = [(0.0, 1.0)](https://www.tensorflow.org/api_docs/python/tf/initializers/random_uniform) or [(-0.05, 0.05)](https://www.tensorflow.org/api_docs/python/tf/keras/initializers/RandomUniform)|
25 |
26 | ## Optimizers
27 |
28 | | Object names in Chainer | Parameter names in Chainer | Chainer | PyTorch | TensorFlow |
29 | |:----------------------- |:---------------------------|:-----------:|:-----------:|:--------------:|
30 | | SGD | lr | [0.01](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.SGD.html) | [Required](https://pytorch.org/docs/stable/optim.html#torch.optim.SGD) |[0.01](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/SGD)|
31 | | Adam | alpha | [0.001](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [0.001](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [0.001](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)|
32 | | | beta1 | [0.9](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [0.9](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [0.9](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)|
33 | | | beta2 | [0.999](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [0.999](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [0.999](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)|
34 | | | eps | [1e-08](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [1e-08](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [1e-08](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)|
35 | | | eta | [1.0](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [No args](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [No args](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)|
36 | | | weight_decay | [0.0](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [0.0](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [No args](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)|
37 | | | amsgrad | [False](https://docs.chainer.org/en/stable/reference/generated/chainer.optimizers.Adam.html) | [False](https://pytorch.org/docs/stable/optim.html#torch.optim.Adam) | [No args](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)|
38 |
39 | ## Normalization methods
40 |
41 | | Object names in Chainer | Parameter names in Chainer | Chainer | PyTorch | TensorFlow |
42 | |:----------------------- |:---------------------------|:-----------:|:-----------:|:--------------:|
43 | | BatchNormalization | eps | [2e-05](https://docs.chainer.org/en/stable/reference/generated/chainer.links.BatchNormalization.html) | [1e-05](https://pytorch.org/docs/stable/nn.html#batchnorm2d) |[1e-03](https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization)|
44 | | LayerNormalization | eps | [1e-06](https://docs.chainer.org/en/stable/reference/generated/chainer.links.LayerNormalization.html) | [1e-05](https://pytorch.org/docs/stable/nn.html#layernorm) | No args ([1e-12](https://github.com/tensorflow/tensorflow/blob/a6d8ffae097d0132989ae4688d224121ec6d8f35/tensorflow/contrib/layers/python/layers/layers.py#L2314)) |
45 | | GroupNormalization | eps | [1e-05](https://docs.chainer.org/en/stable/reference/generated/chainer.links.GroupNormalization.html) | [1e-05](https://pytorch.org/docs/stable/nn.html#groupnorm) | [1e-06](https://www.tensorflow.org/api_docs/python/tf/contrib/layers/group_norm)|
46 | | | groups | [Required](https://docs.chainer.org/en/stable/reference/generated/chainer.links.GroupNormalization.html) | [Required](https://pytorch.org/docs/stable/nn.html#groupnorm) | [32](https://www.tensorflow.org/api_docs/python/tf/contrib/layers/group_norm)|
47 | | local_response_normalization | n | [5](https://docs.chainer.org/en/stable/reference/generated/chainer.functions.local_response_normalization.html#chainer.functions.local_response_normalization) | [Required](https://pytorch.org/docs/stable/nn.html?highlight=local_response_norm#torch.nn.LocalResponseNorm) | [5](https://www.tensorflow.org/api_docs/python/tf/nn/local_response_normalization)|
48 | | | k | [2](https://docs.chainer.org/en/stable/reference/generated/chainer.functions.local_response_normalization.html#chainer.functions.local_response_normalization) | [1](https://pytorch.org/docs/stable/nn.html?highlight=local_response_norm#torch.nn.LocalResponseNorm) | [1](https://www.tensorflow.org/api_docs/python/tf/nn/local_response_normalization)|
49 | | | alpha | [1e-04](https://docs.chainer.org/en/stable/reference/generated/chainer.functions.local_response_normalization.html#chainer.functions.local_response_normalization) | [1e-04](https://pytorch.org/docs/stable/nn.html?highlight=local_response_norm#torch.nn.LocalResponseNorm) | [1](https://www.tensorflow.org/api_docs/python/tf/nn/local_response_normalization)|
50 | | | beta | [0.75](https://docs.chainer.org/en/stable/reference/generated/chainer.functions.local_response_normalization.html#chainer.functions.local_response_normalization) | [0.75](https://pytorch.org/docs/stable/nn.html?highlight=local_response_norm#torch.nn.LocalResponseNorm) | [0.5](https://www.tensorflow.org/api_docs/python/tf/nn/local_response_normalization)|
51 |
52 | ## Rules
53 | ### Kinds of symbols
54 | - None: default value is None.
55 | - x: No implementations.
56 | - No args: Magic number. We can not specify the value.
57 | - Required: No default value. We should specify the value.
58 |
--------------------------------------------------------------------------------