└── README.md


/README.md:
--------------------------------------------------------------------------------
 1 | * `requires_grad` VS `volatile`
 2 |     * > Volatile differs from requires_grad in how the flag propagates. If there’s even a single volatile input to an operation, its output is also going to be volatile. Volatility spreads accross the graph much easier than non-requiring gradient - you only need a single volatile leaf to have a volatile output, while you need all leaves to not require gradient to have an output the doesn’t require gradient. Using volatile flag you don’t need to change any settings of your model parameters to use it for inference. It’s enough to create a volatile input, and this will ensure that no intermediate states are saved.
 3 |     * `Variable` newly created (by default):
 4 |         ```py
 5 |         requires_grad = False
 6 |         volatile = False
 7 |         ```
 8 |     * `Variable` output from any `nn.Module` (by default):
 9 |         ```py
10 |         requires_grad = True
11 |         volatile = False
12 |         ```
13 | 
14 | * `vb = vb.detach()` VS `vb = Variable(vb.data)`
15 |     * after `vb = vb.detach()`:
16 |         ```py
17 |         requires_grad = False
18 |         volatile = Unchanged
19 |         ```
20 |     * after `vb = Variable(vb.data)`:
21 |         ```py
22 |         requires_grad = False
23 |         volatile = False
24 |         ```
25 | 
26 | * inherited classes of `nn.Module`:
27 |     * all submodules must be either `nn.` or `nn.ModuleList`, otherwise they will not be handled correctly. e.g. the following example will give errors since `self.list` is not put onto `cuda` by `self.type(torch.cuda.FloatTensor)`. Thus, `torch.save(model.state_dict(), "...")` and `model.load_state_dict(torch.load("..."))` will also not work correctly for `model.list`
28 |         ```py
29 |         import torch
30 |         import torch.nn as nn
31 |         from torch.autograd import Variable
32 | 
33 |         class Model(nn.Module):
34 |             def __init__(self):
35 |                 super(Model, self).__init__()
36 |                 self.layer = nn.Linear(10, 10)
37 |                 self.modulelist = nn.ModuleList([nn.Linear(10, 10) for i in range(10)])
38 |                 self.list = [nn.Linear(10, 10) for i in range(10)]
39 | 
40 |                 self.type(torch.cuda.FloatTensor)
41 | 
42 |             def forward(self, input_vb):
43 |                 x_vb = self.layer(input_vb)
44 |                 print("passed through layer")
45 |                 for i in range(len(self.modulelist)):
46 |                     x_vb = self.modulelist[i](x_vb)
47 |                 print("passed through modulelist")
48 |                 for i in range(len(self.list)):
49 |                     x_vb = self.list[i](x_vb)
50 |                 print("passed through list")
51 |                 return x_vb
52 | 
53 |         model = Model()
54 |         input_vb = Variable(torch.randn(3, 10))
55 |         output_vb = model(input_vb.cuda())
56 |         ```
57 | 


--------------------------------------------------------------------------------