├── doc
└── lenet
│ ├── lenet.jpeg
│ ├── lenet_arch_1.png
│ ├── lenet_arch_2.png
│ ├── lenet_arch_3.png
│ ├── lenet_arch_4.png
│ ├── lenet_arch_5.png
│ └── lenet_arch_6.webp
├── pyproject.toml
├── requirements.txt
├── models
└── lenet.py
├── LICENSE
├── utils
└── dataset.py
├── train.py
├── .gitignore
└── README.md
/doc/lenet/lenet.jpeg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kadirnar/Pyorch-LeNet5/HEAD/doc/lenet/lenet.jpeg
--------------------------------------------------------------------------------
/doc/lenet/lenet_arch_1.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kadirnar/Pyorch-LeNet5/HEAD/doc/lenet/lenet_arch_1.png
--------------------------------------------------------------------------------
/doc/lenet/lenet_arch_2.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kadirnar/Pyorch-LeNet5/HEAD/doc/lenet/lenet_arch_2.png
--------------------------------------------------------------------------------
/doc/lenet/lenet_arch_3.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kadirnar/Pyorch-LeNet5/HEAD/doc/lenet/lenet_arch_3.png
--------------------------------------------------------------------------------
/doc/lenet/lenet_arch_4.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kadirnar/Pyorch-LeNet5/HEAD/doc/lenet/lenet_arch_4.png
--------------------------------------------------------------------------------
/doc/lenet/lenet_arch_5.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kadirnar/Pyorch-LeNet5/HEAD/doc/lenet/lenet_arch_5.png
--------------------------------------------------------------------------------
/doc/lenet/lenet_arch_6.webp:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kadirnar/Pyorch-LeNet5/HEAD/doc/lenet/lenet_arch_6.webp
--------------------------------------------------------------------------------
/pyproject.toml:
--------------------------------------------------------------------------------
1 | [tool.black]
2 | line-length = 120
3 |
4 | [tool.isort]
5 | line_length = 120
6 | profile = "black"
--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | torch==1.12.0
2 | torchvision==0.13.0
3 | black==21.7b0
4 | flake8==3.9.2
5 | isort==5.9.2
6 | click==8.0.4
--------------------------------------------------------------------------------
/models/lenet.py:
--------------------------------------------------------------------------------
1 | import torch.nn as nn
2 |
3 |
4 | class LeNet5(nn.Module):
5 | def __init__(self):
6 | super(LeNet5, self).__init__()
7 |
8 | self.conv1 = nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5, stride=1, padding=0)
9 | self.conv2 = nn.Conv2d(in_channels=6, out_channels=16, kernel_size=5, stride=1, padding=0)
10 | self.conv3 = nn.Conv2d(in_channels=16, out_channels=120, kernel_size=5, stride=1, padding=0)
11 |
12 | self.linear = nn.Linear(in_features=120, out_features=84)
13 | self.linear2 = nn.Linear(in_features=84, out_features=10)
14 | self.tanh = nn.Tanh()
15 | self.avgpool = nn.AvgPool2d(kernel_size=2, stride=2)
16 |
17 | def forward(self, x):
18 | x = self.conv1(x)
19 | x = self.tanh(x)
20 | x = self.avgpool(x)
21 | x = self.conv2(x)
22 | x = self.tanh(x)
23 | x = self.avgpool(x)
24 | x = self.conv3(x)
25 | x = self.tanh(x)
26 |
27 | x = x.reshape(x.shape[0], -1)
28 | x = self.linear(x)
29 | x = self.tanh(x)
30 | x = self.linear2(x)
31 | return x
32 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2022 Kadir Nar
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/utils/dataset.py:
--------------------------------------------------------------------------------
1 | from torch.utils.data import DataLoader, Dataset
2 | from torchvision import datasets, transforms
3 |
4 |
5 | class MnistDataset(Dataset):
6 | def __init__(
7 | self,
8 | image_size: int = 32,
9 | batch_size: int = 64,
10 | ):
11 | self.image_size = image_size
12 | self.batch_size = batch_size
13 |
14 | def __getitem__(self, train=True):
15 | if train:
16 | train_transform = transforms.Compose(
17 | [
18 | transforms.ToTensor(),
19 | transforms.Resize(size=(self.image_size, self.image_size)),
20 | transforms.Normalize((0.1307,), (0.3081,)),
21 | ]
22 | )
23 | train_dataset = datasets.MNIST(root="dataset", train=True, transform=train_transform, download=True)
24 | data_loader = DataLoader(train_dataset, batch_size=self.batch_size, shuffle=True)
25 |
26 | else:
27 | test_transform = transforms.Compose(
28 | [
29 | transforms.ToTensor(),
30 | transforms.Resize(size=(self.image_size, self.image_size)),
31 | transforms.Normalize((0.1307,), (0.3081,)),
32 | ]
33 | )
34 | test_dataset = datasets.MNIST(root="dataset", train=False, transform=test_transform, download=True)
35 | data_loader = DataLoader(test_dataset, batch_size=self.batch_size, shuffle=False)
36 |
37 | return data_loader
38 |
39 | def __len__(self, train=True):
40 | if train:
41 | return len(self.__getitem__(train=True))
42 | else:
43 | return len(self.__getitem__(train=False))
44 |
--------------------------------------------------------------------------------
/train.py:
--------------------------------------------------------------------------------
1 | import torch
2 |
3 | from utils.dataset import MnistDataset
4 | from models.lenet import LeNet5
5 |
6 | class ModelTrainer:
7 | def config(self):
8 | self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
9 | self.model = LeNet5().to(self.device)
10 | self.optimizer = torch.optim.Adam(self.model.parameters(), lr=0.001)
11 | self.criterion = torch.nn.CrossEntropyLoss()
12 | self.num_epochs = 5
13 | self.batch_size = 64
14 |
15 |
16 | def train(self):
17 | train_data_loader = MnistDataset().__getitem__(train=True)
18 | for epoch in range(self.num_epochs):
19 | for i, (images, labels) in enumerate(train_data_loader):
20 | images = images.to(self.device)
21 | labels = labels.to(self.device)
22 | self.optimizer.zero_grad()
23 | outputs = self.model(images)
24 | loss = self.criterion(outputs, labels)
25 | loss.backward()
26 | self.optimizer.step()
27 | if (i + 1) % 100 == 0:
28 | print(
29 | "Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}".format(
30 | epoch + 1, self.num_epochs, i + 1, MnistDataset().__len__(train=True), loss.item()
31 | )
32 | )
33 |
34 |
35 | test_data_loader = MnistDataset().__getitem__(train=False)
36 | correct = 0
37 | total = 0
38 | for images, labels in test_data_loader:
39 | images = images.to(self.device)
40 | labels = labels.to(self.device)
41 | outputs = self.model(images)
42 | _, predicted = torch.max(outputs.data, 1)
43 | total += labels.size(0)
44 | correct += (predicted == labels).sum().item()
45 |
46 | acc = 100 * correct / float(total)
47 | print("Accuracy of the network on the 10000 test images: {} %".format(acc))
48 |
49 | if __name__ == "__main__":
50 | trainer = ModelTrainer()
51 | trainer.config()
52 | trainer.train()
53 |
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
1 | # Byte-compiled / optimized / DLL files
2 | __pycache__/
3 | *.py[cod]
4 | *$py.class
5 |
6 | # C extensions
7 | *.so
8 |
9 | # Distribution / packaging
10 | .Python
11 | build/
12 | develop-eggs/
13 | dist/
14 | downloads/
15 | eggs/
16 | .eggs/
17 | lib/
18 | lib64/
19 | parts/
20 | sdist/
21 | var/
22 | wheels/
23 | pip-wheel-metadata/
24 | share/python-wheels/
25 | *.egg-info/
26 | .installed.cfg
27 | *.egg
28 | MANIFEST
29 |
30 | # PyInstaller
31 | # Usually these files are written by a python script from a template
32 | # before PyInstaller builds the exe, so as to inject date/other infos into it.
33 | *.manifest
34 | *.spec
35 |
36 | # Installer logs
37 | pip-log.txt
38 | pip-delete-this-directory.txt
39 |
40 | # Unit test / coverage reports
41 | htmlcov/
42 | .tox/
43 | .nox/
44 | .coverage
45 | .coverage.*
46 | .cache
47 | nosetests.xml
48 | coverage.xml
49 | *.cover
50 | *.py,cover
51 | .hypothesis/
52 | .pytest_cache/
53 |
54 | # Translations
55 | *.mo
56 | *.pot
57 |
58 | # Django stuff:
59 | *.log
60 | local_settings.py
61 | db.sqlite3
62 | db.sqlite3-journal
63 |
64 | # Flask stuff:
65 | instance/
66 | .webassets-cache
67 |
68 | # Scrapy stuff:
69 | .scrapy
70 |
71 | # Sphinx documentation
72 | docs/_build/
73 |
74 | # PyBuilder
75 | target/
76 |
77 | # Jupyter Notebook
78 | .ipynb_checkpoints
79 |
80 | # IPython
81 | profile_default/
82 | ipython_config.py
83 |
84 | # pyenv
85 | .python-version
86 |
87 | # pipenv
88 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
89 | # However, in case of collaboration, if having platform-specific dependencies or dependencies
90 | # having no cross-platform support, pipenv may install dependencies that don't work, or not
91 | # install all needed dependencies.
92 | #Pipfile.lock
93 |
94 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow
95 | __pypackages__/
96 |
97 | # Celery stuff
98 | celerybeat-schedule
99 | celerybeat.pid
100 |
101 | # SageMath parsed files
102 | *.sage.py
103 |
104 | # Environments
105 | .env
106 | .venv
107 | env/
108 | venv/
109 | ENV/
110 | env.bak/
111 | venv.bak/
112 |
113 | # Spyder project settings
114 | .spyderproject
115 | .spyproject
116 |
117 | # Rope project settings
118 | .ropeproject
119 |
120 | # mkdocs documentation
121 | /site
122 |
123 | # mypy
124 | .mypy_cache/
125 | .dmypy.json
126 | dmypy.json
127 |
128 | # Pyre type checker
129 | .pyre/
130 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 |
2 |
3 | PyTorch implementation of LeNet5
4 |
5 |
6 |
32 |
33 | The input to this model is a 32 X 32 grayscale image hence the number of channels is one.
34 |
35 | 
36 |
37 | We then apply the first convolution operation with the filter size 5X5 and we have 6 such filters. As a result, we get a feature map of size 28X28X6. Here the number of channels is equal to the number of filters applied..
38 |
39 | 
40 |
41 | We then apply the first convolution operation with the filter size 5X5 and we have 6 such filters. As a result, we get a feature map of size 28X28X6. Here the number of channels is equal to the number of filters applied.
42 |
43 | 
44 |
45 | Next, we have a convolution layer with sixteen filters of size 5X5. Again the feature map changed it is 10X10X16. The output size is calculated in a similar manner. After this, we again applied an average pooling or subsampling layer, which again reduce the size of the feature map by half i.e 5X5X16.
46 |
47 | 
48 |
49 | Then we have a final convolution layer of size 5X5 with 120 filters. As shown in the above image. Leaving the feature map size 1X1X120. After which flatten result is 120 values.
50 |
51 | After these convolution layers, we have a fully connected layer with eighty-four neurons. At last, we have an output layer with ten neurons since the data have ten classes.
52 |
53 | 
54 |
55 | Here is the final architecture of the Lenet-5 model.
56 |
57 | * Input: 32x32x1 grayscale image
58 | * Conv1: 5x5 convolution, 6 outputs (28x28x6)
59 | * Pool1: 2x2 pooling, outputs (14x14x6)
60 | * Conv2: 5x5 convolution, 16 outputs (10x10x16)
61 | * Pool2: 2x2 pooling, outputs (5x5x16)
62 | * FC1: 120 outputs (1x1x120)
63 | * FC2: 84 outputs (1x1x84)
64 | * Output: 10 outputs (1x1x10)
65 |
66 | References:
67 | - [LeNet5](https://www.datasciencecentral.com/lenet-5-a-classic-cnn-architecture/)
68 | - [Paper](http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf)
69 |
--------------------------------------------------------------------------------