├── .gitignore ├── CIFAR10 ├── Pytorch CIFAR-10分类 模板.ipynb ├── Pytorch CIFAR-10分类 （SENet）.ipynb ├── Pytorch CIFAR-10分类 （ShuffleNetv2）.ipynb ├── Pytorch CIFAR-10分类 （ShuffleNet）.ipynb ├── Pytorch CIFAR-10分类 （Swin Transformer）.ipynb ├── Pytorch CIFAR-10分类 （ViT+）.ipynb ├── Pytorch CIFAR-10分类 （ViT）.ipynb ├── Pytorch CIFAR-10分类（AlexNet）.ipynb ├── Pytorch CIFAR-10分类（DenseNet）.ipynb ├── Pytorch CIFAR-10分类（DenseNet）Improve.ipynb ├── Pytorch CIFAR-10分类（DenseNet）Improve2.ipynb ├── Pytorch CIFAR-10分类（EfficientNet）.ipynb ├── Pytorch CIFAR-10分类（GoogLeNet）.ipynb ├── Pytorch CIFAR-10分类（LeNet5）.ipynb ├── Pytorch CIFAR-10分类（MobileNetv1).ipynb ├── Pytorch CIFAR-10分类（MobileNetv2）.ipynb ├── Pytorch CIFAR-10分类（MobileNetv3）.ipynb ├── Pytorch CIFAR-10分类（ResNet34）.ipynb ├── Pytorch CIFAR-10分类（VGG16）.ipynb ├── Pytorch CIFAR-10分类（ZFNet）.ipynb ├── Pytorch CIFAR-10分类（自定义网络）.ipynb ├── README.md ├── airplane.jpg ├── dataloader.py ├── eval.py ├── model │ └── README.md ├── nets │ ├── AlexNet.py │ ├── DenseNet.py │ ├── EfficientNet.py │ ├── LeNet5.py │ ├── MobileNetv1.py │ ├── MobileNetv2.py │ ├── RegNet.py │ ├── ResNet.py │ ├── SENet.py │ ├── ShuffleNet.py │ ├── ShuffleNetv2.py │ ├── VGG.py │ ├── pnasnet.py │ ├── preact_resnet.py │ ├── squeezeNet.py │ └── wideresidual.py ├── requirements.txt ├── train.py ├── utils.py └── utils2.py ├── ConvNeXt ├── dataloader.py ├── eval.py ├── nets │ └── ConvNeXt.py ├── train.py ├── utils.py ├── 大数据驱动的多种深度模型在图像分类中的对比（ConvNeXt篇）.doc └── 大数据驱动的多种深度模型在图像分类中的对比（ConvNeXt篇）.pdf ├── MAE ├── MAE_In_CIFAR.ipynb ├── README.md ├── logs │ └── README.md ├── mae_pretrain.py ├── model.py ├── pic │ └── mae-cifar10-reconstruction.png ├── requirements.txt ├── train_classifier.py ├── utils.py ├── vit-t-classifier-from_pretrained.pth ├── vit-t-classifier-from_scratch.pth └── vit-t-mae.pth ├── Parper-link └── README.md ├── QR.png ├── README.md ├── TransferLearning ├── README.md ├── Transfer_Learning.ipynb ├── Transfer_models.py ├── train.py └── utils.py ├── docs ├── Pytorch CIFAR10图像分类 AlexNet篇.md ├── Pytorch CIFAR10图像分类 DenseNet篇.md ├── Pytorch CIFAR10图像分类 EfficientNetv1篇.md ├── Pytorch CIFAR10图像分类 GoogLeNet篇.md ├── Pytorch CIFAR10图像分类 LeNet篇.md ├── Pytorch CIFAR10图像分类 MobileNetv1篇.md ├── Pytorch CIFAR10图像分类 MobileNetv2篇.md ├── Pytorch CIFAR10图像分类 ResNet篇.md ├── Pytorch CIFAR10图像分类 SENet篇.md ├── Pytorch CIFAR10图像分类 ShuffleNetv2篇.md ├── Pytorch CIFAR10图像分类 ShuffleNet篇.md ├── Pytorch CIFAR10图像分类 Swin Transformer篇.md ├── Pytorch CIFAR10图像分类 VGG篇.md ├── Pytorch CIFAR10图像分类 ViT篇.md ├── Pytorch CIFAR10图像分类 ZFNet篇.md ├── Pytorch CIFAR10图像分类 工具函数utils更新v2.0篇.md ├── Pytorch CIFAR10图像分类 工具函数utils篇.md ├── Pytorch CIFAR10图像分类 数据加载篇.md ├── Pytorch CIFAR10图像分类 自定义网络篇.md ├── Pytorch&Keras CIFAR10图像分类.md └── README.md ├── markdown ├── Pytorch CIFAR10图像分类 AlexNet篇.md ├── Pytorch CIFAR10图像分类 DenseNet篇.md ├── Pytorch CIFAR10图像分类 GoogLeNet篇.md ├── Pytorch CIFAR10图像分类 LeNet篇.md ├── Pytorch CIFAR10图像分类 MobileNetv1篇.md ├── Pytorch CIFAR10图像分类 MobileNetv2篇.md ├── Pytorch CIFAR10图像分类 ResNet篇.md ├── Pytorch CIFAR10图像分类 SENet篇.md ├── Pytorch CIFAR10图像分类 VGG篇.md ├── Pytorch CIFAR10图像分类 ViT篇.md ├── Pytorch CIFAR10图像分类 工具函数utils更新v2.0篇.md ├── Pytorch CIFAR10图像分类 工具函数utils篇.md ├── Pytorch CIFAR10图像分类 数据加载篇.md ├── Pytorch CIFAR10图像分类 自定义网络篇.md ├── Pytorch&Keras CIFAR10图像分类.md ├── README.md └── img │ ├── AlexNet.png │ ├── BasicBlock.png │ ├── Bottleneck.png │ ├── DenseNet_Explained.png │ ├── Dense_bottleneck.png │ ├── GooLeNet2.png │ ├── Inception.jpg │ ├── Inception.png │ ├── Inception2.png │ ├── Inception3.png │ ├── Inceptionv2.png │ ├── LeNet5.png │ ├── Original-ResNet-18-Architecture.png │ ├── ResNet-34.png │ ├── Residual3.png │ ├── cifar.png │ ├── dense1.png │ ├── dense2.png │ ├── dense3.png │ ├── dense_cifar.jpg │ ├── densenet.gif │ ├── densenet_table.png │ ├── gooLenet.png │ ├── residual.png │ ├── residual2.png │ ├── resnet_table.png │ ├── vgg.png │ └── vgg19.png └── requirements.txt /.gitignore: -------------------------------------------------------------------------------- 1 | *.pkl 2 | *.jpg 3 | *.png 4 | *.mp4 5 | *.pyc 6 | *.pth 7 | __pycache__ 8 | *.h5 9 | *.avi 10 | *.wav 11 | *.pyc 12 | *.mkv 13 | *.gif 14 | *.webm 15 | *.mp3 16 | *.zip 17 | CIFAR10-Keras/ -------------------------------------------------------------------------------- /CIFAR10/README.md: -------------------------------------------------------------------------------- 1 | ## Finish 2 | 3 | - [x] LetNet 4 | - [x] AlexNet 5 | - [x] VGG 6 | - [x] ResNet 7 | - [x] GoogLeNet 8 | - [x] DenseNet 9 | 10 | ## Comming soon 11 | 12 | - [x] MobileNetv1 Model 13 | 14 | - [x] MobileNetv2 Model 15 | 16 | - [ ] ShuffleNetv1 Model 17 | 18 | - [ ] ShuffleNetv2 Model 19 | 20 | - [x] ResNeXt Model 21 | 22 | - [ ] ZFNet Model 23 | 24 | - [ ] SeNet Model 25 | 26 | - [ ] Efficientent Model 27 | 28 | - [x] ViT Model 29 | 30 | 31 | 32 | ## Acknowledgments 33 | 34 | - [kuangliu/pytorch-cifar](https://github.com/kuangliu/pytorch-cifar) 35 | 36 | - [pytorch-cifar100](https://github.com/weiaicunzai/pytorch-cifar100) 37 | 38 | - [bearpaw/pytorch-classification](https://github.com/bearpaw/pytorch-classification) 39 | 40 | - [timgaripov/swa](https://github.com/timgaripov/swa) 41 | 42 | - [xgastaldi/shake-shake](https://github.com/xgastaldi/shake-shake) 43 | 44 | - [uoguelph-mlrg/Cutout](https://github.com/uoguelph-mlrg/Cutout) 45 | 46 | - [facebookresearch/mixup-cifar10](https://github.com/facebookresearch/mixup-cifar10) 47 | 48 | - [BIGBALLON/cifar-10-cnn](https://github.com/BIGBALLON/cifar-10-cnn) 49 | 50 | - [BayesWatch/pytorch-GENet](https://github.com/BayesWatch/pytorch-GENet/) 51 | 52 | - [Jongchan/attention-module](https://github.com/Jongchan/attention-module) 53 | 54 | - [pppLang/SKNet](https://github.com/pppLang/SKNet) 55 | 56 | 57 | 58 | 59 | 60 | -------------------------------------------------------------------------------- /CIFAR10/airplane.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/CIFAR10/airplane.jpg -------------------------------------------------------------------------------- /CIFAR10/dataloader.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torchvision 3 | import torchvision.transforms as transforms 4 | 5 | # Data 6 | def get_training_dataloader(batch_size = 64, num_workers = 4, shuffle = True): 7 | print('==> Preparing Train data..') 8 | transform_train = transforms.Compose([ 9 | transforms.RandomCrop(32, padding=4), 10 | transforms.RandomHorizontalFlip(), 11 | transforms.ToTensor(), 12 | transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)), 13 | ]) 14 | trainset = torchvision.datasets.CIFAR10( 15 | root='./data', train=True, download=True, transform=transform_train) 16 | trainloader = torch.utils.data.DataLoader( 17 | trainset, batch_size=batch_size, shuffle=shuffle, num_workers= num_workers) 18 | return trainloader 19 | 20 | def get_test_dataloader(batch_size = 64, num_workers = 4, shuffle = True): 21 | print('==> Preparing Test data..') 22 | transform_test = transforms.Compose([ 23 | transforms.ToTensor(), 24 | transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)), 25 | ]) 26 | 27 | testset = torchvision.datasets.CIFAR10( 28 | root='./data', train=False, download=True, transform=transform_test) 29 | testloader = torch.utils.data.DataLoader( 30 | testset, batch_size=batch_size, shuffle=shuffle, num_workers= num_workers) 31 | return testloader 32 | 33 | -------------------------------------------------------------------------------- /CIFAR10/eval.py: -------------------------------------------------------------------------------- 1 | 2 | import os 3 | import torch 4 | import torch.nn as nn 5 | import argparse 6 | import numpy as np 7 | from tqdm import tqdm 8 | from dataloader import get_test_dataloader 9 | import torch.backends.cudnn as cudnn 10 | 11 | def eval_top1(outputs, label): 12 | total = outputs.shape[0] 13 | outputs = torch.softmax(outputs, dim=-1) 14 | _, pred_y = outputs.data.max(dim=1) # 得到概率 15 | correct = (pred_y == label).sum().data 16 | return correct / total 17 | 18 | def eval_top5(outputs, label): 19 | total = outputs.shape[0] 20 | outputs = torch.softmax(outputs, dim=-1) 21 | pred_y = np.argsort(-outputs.cpu().numpy()) 22 | pred_y_top5 = pred_y[:,:5] 23 | correct = 0 24 | for i in range(total): 25 | if label[i].cpu().numpy() in pred_y_top5[i]: 26 | correct += 1 27 | return correct / total 28 | 29 | 30 | if __name__ == '__main__': 31 | parser = argparse.ArgumentParser(description='PyTorch CIFAR10 Test') 32 | parser.add_argument('--cuda', action='store_true', default=False, help =' use GPU?') 33 | parser.add_argument('--batch-size', default=64, type=int, help = "Batch Size for Test") 34 | parser.add_argument('--num-workers', default=2, type=int, help = 'num-workers') 35 | parser.add_argument('--net', type = str, choices=['LeNet5', 'AlexNet', 'VGG16','VGG19','ResNet18','ResNet34', 36 | 'DenseNet','MobileNetv1','MobileNetv2'], default='MobileNetv1', help='net type') 37 | args = parser.parse_args() 38 | testloader = get_test_dataloader() 39 | 40 | # Model 41 | print('==> Building model..') 42 | if args.net == 'VGG16': 43 | from nets.VGG import VGG 44 | net = VGG('VGG16') 45 | elif args.net == 'VGG19': 46 | from nets.VGG import VGG 47 | net = VGG('VGG19') 48 | elif args.net == 'ResNet18': 49 | from nets.ResNet import ResNet18 50 | net = ResNet18() 51 | elif args.net == 'ResNet34': 52 | from nets.ResNet import ResNet34 53 | net = ResNet34() 54 | elif args.net == 'LeNet5': 55 | from nets.LeNet5 import LeNet5 56 | net = LeNet5() 57 | elif args.net == 'AlexNet': 58 | from nets.AlexNet import AlexNet 59 | net = AlexNet() 60 | elif args.net == 'DenseNet': 61 | from nets.DenseNet import densenet_cifar 62 | net = densenet_cifar() 63 | elif args.net == 'MobileNetv1': 64 | from nets.MobileNetv1 import MobileNet 65 | net = MobileNet() 66 | elif args.net == 'MobileNetv2': 67 | from nets.MobileNetv2 import MobileNetV2 68 | net = MobileNetV2() 69 | 70 | if args.cuda and torch.cuda.is_available(): 71 | device = 'cuda' 72 | net = torch.nn.DataParallel(net) 73 | cudnn.benchmark = True 74 | else: 75 | device = 'cpu' 76 | 77 | criterion = nn.CrossEntropyLoss() 78 | 79 | # Load checkpoint. 80 | print('==> Resuming from checkpoint..') 81 | assert os.path.isdir('checkpoint'), 'Error: no checkpoint directory found!' 82 | checkpoint = torch.load('./checkpoint/{}_ckpt.pth'.format(args.net)) 83 | net.load_state_dict(checkpoint['net']) 84 | 85 | 86 | epoch_step_test = len(testloader) 87 | if epoch_step_test == 0: 88 | raise ValueError("数据集过小，无法进行训练，请扩充数据集，或者减小batchsize") 89 | 90 | net.eval() 91 | test_acc_top1 = 0 92 | test_acc_top5 = 0 93 | print('Start Test') 94 | #-------------------------------- 95 | # 相同方法，同train 96 | #-------------------------------- 97 | with tqdm(total=epoch_step_test,desc=f'Test Acc',postfix=dict,mininterval=0.3) as pbar2: 98 | for step,(im,label) in enumerate(testloader,start=0): 99 | im = im.to(device) 100 | label = label.to(device) 101 | with torch.no_grad(): 102 | if step >= epoch_step_test: 103 | break 104 | 105 | # 释放内存 106 | if hasattr(torch.cuda, 'empty_cache'): 107 | torch.cuda.empty_cache() 108 | #----------------------# 109 | # 前向传播 110 | #----------------------# 111 | outputs = net(im) 112 | loss = criterion(outputs,label) 113 | test_acc_top1 += eval_top1(outputs,label) 114 | test_acc_top5 += eval_top5(outputs,label) 115 | pbar2.set_postfix(**{'Test Acc Top1': test_acc_top1.item()/(step+1), 116 | 'Test Acc Top5': test_acc_top5 / (step + 1)}) 117 | pbar2.update(1) 118 | 119 | top1 = test_acc_top1.item()/ len(testloader) 120 | top5 = test_acc_top5 / len(testloader) 121 | print("top-1 accuracy = %.2f%%" % (top1*100)) 122 | print("top-5 accuracy = %.2f%%" % (top5*100)) -------------------------------------------------------------------------------- /CIFAR10/model/README.md: -------------------------------------------------------------------------------- 1 | # 权重文件 2 | 3 | 所有的权重文件在release中有，可以下载相应的版本去对自己的文件进行测试 4 | 5 | [v1.0.0](https://github.com/Dreaming-future/Pytorch-Image-Classification/releases/tag/v1.0.0) -------------------------------------------------------------------------------- /CIFAR10/nets/AlexNet.py: -------------------------------------------------------------------------------- 1 | ''' 2 | AlexNet in Pytorch 3 | ''' 4 | 5 | import torch 6 | import torch.nn as nn 7 | 8 | # 定义2012的AlexNet 9 | class AlexNet(nn.Module): 10 | def __init__(self,num_classes=10): 11 | super(AlexNet,self).__init__() 12 | # 五个卷积层 输入 32 * 32 * 3 13 | self.conv1 = nn.Sequential( 14 | nn.Conv2d(in_channels=3, out_channels=6, kernel_size=3, stride=1, padding=1), # (32-3+2)/1+1 = 32 15 | nn.ReLU(), 16 | nn.MaxPool2d(kernel_size=2, stride=2, padding=0) # (32-2)/2+1 = 16 17 | ) 18 | self.conv2 = nn.Sequential( # 输入 16 * 16 * 6 19 | nn.Conv2d(in_channels=6, out_channels=16, kernel_size=3, stride=1, padding=1), # (16-3+2)/1+1 = 16 20 | nn.ReLU(), 21 | nn.MaxPool2d(kernel_size=2, stride=2, padding=0) # (16-2)/2+1 = 8 22 | ) 23 | self.conv3 = nn.Sequential( # 输入 8 * 8 * 16 24 | nn.Conv2d(in_channels=16, out_channels=32, kernel_size=3, stride=1, padding=1), # (8-3+2)/1+1 = 8 25 | nn.ReLU(), 26 | nn.MaxPool2d(kernel_size=2, stride=2, padding=0) # (8-2)/2+1 = 4 27 | ) 28 | self.conv4 = nn.Sequential( # 输入 4 * 4 * 64 29 | nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3, stride=1, padding=1), # (4-3+2)/1+1 = 4 30 | nn.ReLU(), 31 | nn.MaxPool2d(kernel_size=2, stride=2, padding=0) # (4-2)/2+1 = 2 32 | ) 33 | self.conv5 = nn.Sequential( # 输入 2 * 2 * 128 34 | nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, stride=1, padding=1),# (2-3+2)/1+1 = 2 35 | nn.ReLU(), 36 | nn.MaxPool2d(kernel_size=2, stride=2, padding=0) # (2-2)/2+1 = 1 37 | ) # 最后一层卷积层，输出 1 * 1 * 128 38 | # 全连接层 39 | self.fc = nn.Sequential( 40 | nn.Linear(128,120), 41 | nn.ReLU(), 42 | nn.Linear(120,84), 43 | nn.ReLU(), 44 | nn.Linear(84,num_classes) 45 | ) 46 | 47 | def forward(self,x): 48 | x = self.conv1(x) 49 | x = self.conv2(x) 50 | x = self.conv3(x) 51 | x = self.conv4(x) 52 | x = self.conv5(x) 53 | x = x.view(x.size()[0],-1) 54 | x = self.fc(x) 55 | return x 56 | 57 | def test(): 58 | net = AlexNet() 59 | x = torch.randn(2,3,32,32) 60 | y = net(x) 61 | print(y.size()) 62 | from torchinfo import summary 63 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 64 | net = net.to(device) 65 | summary(net,(3,32,32)) -------------------------------------------------------------------------------- /CIFAR10/nets/DenseNet.py: -------------------------------------------------------------------------------- 1 | 2 | """ 3 | DenseNet in pytorch 4 | see the details in papaer 5 | [1] Gao Huang, Zhuang Liu, Laurens van der Maaten, Kilian Q. Weinberger. 6 | Densely Connected Convolutional Networks 7 | https://arxiv.org/abs/1608.06993v5 8 | """ 9 | import torch 10 | import torch.nn as nn 11 | 12 | class Bottleneck(nn.Module): 13 | """ 14 | Dense Block 15 | 这里的growth_rate=out_channels, 就是每个Block自己输出的通道数。 16 | 先通过1x1卷积层，将通道数缩小为4 * growth_rate，然后再通过3x3卷积层降低到growth_rate。 17 | """ 18 | # 通常1×1卷积的通道数为GrowthRate的4倍 19 | expansion = 4 20 | 21 | def __init__(self, in_channels, growth_rate): 22 | super(Bottleneck, self).__init__() 23 | zip_channels = self.expansion * growth_rate 24 | self.features = nn.Sequential( 25 | nn.BatchNorm2d(in_channels), 26 | nn.ReLU(True), 27 | nn.Conv2d(in_channels, zip_channels, kernel_size=1, bias=False), 28 | nn.BatchNorm2d(zip_channels), 29 | nn.ReLU(True), 30 | nn.Conv2d(zip_channels, growth_rate, kernel_size=3, padding=1, bias=False) 31 | ) 32 | 33 | def forward(self, x): 34 | out = self.features(x) 35 | out = torch.cat([out, x], 1) 36 | return out 37 | 38 | 39 | class Transition(nn.Module): 40 | """ 41 | 改变维数的Transition层 具体包括BN、ReLU、1×1卷积（Conv）、2×2平均池化操作 42 | 先通过1x1的卷积层减少channels，再通过2x2的平均池化层缩小feature-map 43 | """ 44 | # 1×1卷积的作用是降维，起到压缩模型的作用，而平均池化则是降低特征图的尺寸。 45 | def __init__(self, in_channels, out_channels): 46 | super(Transition, self).__init__() 47 | self.features = nn.Sequential( 48 | nn.BatchNorm2d(in_channels), 49 | nn.ReLU(True), 50 | nn.Conv2d(in_channels, out_channels, kernel_size=1, bias=False), 51 | nn.AvgPool2d(2) 52 | ) 53 | 54 | def forward(self, x): 55 | out = self.features(x) 56 | return out 57 | # DesneNet-BC 58 | # B 代表 bottleneck layer(BN-RELU-CONV(1x1)-BN-RELU-CONV(3x3)) 59 | # C 代表压缩系数(0<=theta<=1) 60 | import math 61 | class DenseNet(nn.Module): 62 | """ 63 | Dense Net 64 | paper中growth_rate取12，维度压缩的参数θ，即reduction取0.5 65 | 且初始化方法为kaiming_normal() 66 | num_blocks为每段网络中的DenseBlock数量 67 | DenseNet和ResNet一样也是六段式网络（一段卷积+四段Dense+平均池化层），最后FC层。 68 | 第一段将维数从3变到2 * growth_rate 69 | 70 | (3, 32, 32) -> [Conv2d] -> (24, 32, 32) -> [layer1] -> (48, 16, 16) -> [layer2] 71 | ->(96, 8, 8) -> [layer3] -> (192, 4, 4) -> [layer4] -> (384, 4, 4) -> [AvgPool] 72 | ->(384, 1, 1) -> [Linear] -> (10) 73 | 74 | """ 75 | def __init__(self, num_blocks, growth_rate=12, reduction=0.5, num_classes=10, init_weights=True): 76 | super(DenseNet, self).__init__() 77 | self.growth_rate = growth_rate 78 | self.reduction = reduction 79 | 80 | num_channels = 2 * growth_rate 81 | 82 | self.features = nn.Conv2d(3, num_channels, kernel_size=3, padding=1, bias=False) 83 | self.layer1, num_channels = self._make_dense_layer(num_channels, num_blocks[0]) 84 | self.layer2, num_channels = self._make_dense_layer(num_channels, num_blocks[1]) 85 | self.layer3, num_channels = self._make_dense_layer(num_channels, num_blocks[2]) 86 | self.layer4, num_channels = self._make_dense_layer(num_channels, num_blocks[3], transition=False) 87 | self.avg_pool = nn.Sequential( 88 | nn.BatchNorm2d(num_channels), 89 | nn.ReLU(True), 90 | nn.AvgPool2d(4), 91 | ) 92 | self.classifier = nn.Linear(num_channels, num_classes) 93 | 94 | if init_weights: 95 | self._initialize_weights() 96 | 97 | def _make_dense_layer(self, in_channels, nblock, transition=True): 98 | layers = [] 99 | for i in range(nblock): 100 | layers += [Bottleneck(in_channels, self.growth_rate)] 101 | in_channels += self.growth_rate 102 | out_channels = in_channels 103 | if transition: 104 | out_channels = int(math.floor(in_channels * self.reduction)) 105 | layers += [Transition(in_channels, out_channels)] 106 | return nn.Sequential(*layers), out_channels 107 | 108 | def _initialize_weights(self): 109 | for m in self.modules(): 110 | if isinstance(m, nn.Conv2d): 111 | nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu') 112 | if m.bias is not None: 113 | nn.init.constant_(m.bias, 0) 114 | elif isinstance(m, nn.BatchNorm2d): 115 | nn.init.constant_(m.weight, 1) 116 | nn.init.constant_(m.bias, 0) 117 | elif isinstance(m, nn.Linear): 118 | nn.init.normal_(m.weight, 0, 0.01) 119 | nn.init.constant_(m.bias, 0) 120 | 121 | def forward(self, x): 122 | out = self.features(x) 123 | out = self.layer1(out) 124 | out = self.layer2(out) 125 | out = self.layer3(out) 126 | out = self.layer4(out) 127 | out = self.avg_pool(out) 128 | out = out.view(out.size(0), -1) 129 | out = self.classifier(out) 130 | return out 131 | 132 | def DenseNet121(): 133 | return DenseNet([6,12,24,16], growth_rate=32) 134 | 135 | def DenseNet169(): 136 | return DenseNet([6,12,32,32], growth_rate=32) 137 | 138 | def DenseNet201(): 139 | return DenseNet([6,12,48,32], growth_rate=32) 140 | 141 | def DenseNet161(): 142 | return DenseNet([6,12,36,24], growth_rate=48) 143 | 144 | def densenet_cifar(): 145 | return DenseNet([6,12,24,16], growth_rate=12) 146 | 147 | 148 | def test(): 149 | net = densenet_cifar() 150 | x = torch.randn(1,3,32,32) 151 | y = net(x) 152 | print(y.size()) 153 | from torchinfo import summary 154 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 155 | net = net.to(device) 156 | summary(net,(1,3,32,32)) 157 | 158 | # test() -------------------------------------------------------------------------------- /CIFAR10/nets/LeNet5.py: -------------------------------------------------------------------------------- 1 | ''' 2 | LetNet in Pytorch 3 | ''' 4 | import torch 5 | import torch.nn as nn 6 | 7 | class LeNet5(nn.Module): 8 | def __init__(self, num_classes = 10): 9 | super(LeNet5,self).__init__() 10 | self.conv1 = nn.Sequential( 11 | # 输入 32x32x3 -> 28x28x6 (32-5)/1 + 1=28 12 | nn.Conv2d(in_channels=3,out_channels=6,kernel_size=5,stride=1), 13 | nn.ReLU(), 14 | # 经过2x2的maxpool，变成14x14 (28-2)/2+1 15 | nn.MaxPool2d(kernel_size=2,stride=2) 16 | ) 17 | 18 | self.conv2 = nn.Sequential( 19 | # 输入 14x14x6 -> 10x10x16 (14-5)/1 + 1 = 10 20 | nn.Conv2d(in_channels=6,out_channels=16,kernel_size=5,stride=1), 21 | nn.ReLU(), 22 | # (10-2)/2 + 1 = 5 23 | nn.MaxPool2d(kernel_size=2,stride=2) 24 | ) 25 | 26 | self.fc = nn.Sequential( 27 | nn.Linear(5*5*16,120), 28 | nn.ReLU(), 29 | nn.Linear(120,84), 30 | nn.ReLU(), 31 | nn.Linear(84,num_classes) 32 | ) 33 | 34 | def forward(self,x): 35 | x = self.conv1(x) 36 | x = self.conv2(x) 37 | # 要把多维度的tensor展平成一维 38 | x = x.view(x.size()[0],-1) 39 | x = self.fc(x) 40 | return x 41 | 42 | 43 | 44 | def test(): 45 | net = LeNet5() 46 | x = torch.randn(2,3,32,32) 47 | y = net(x) 48 | print(y.size()) 49 | from torchinfo import summary 50 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 51 | net = net.to(device) 52 | summary(net,(2,3,32,32)) 53 | 54 | -------------------------------------------------------------------------------- /CIFAR10/nets/MobileNetv1.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MobileNetv1 in pytorch 3 | 4 | See the paper "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications" 5 | for more details. 6 | ''' 7 | import torch 8 | import torch.nn as nn 9 | 10 | class Block(nn.Module): 11 | '''Depthwise conv + Pointwise conv''' 12 | def __init__(self,in_channels,out_channels,stride=1): 13 | super(Block,self).__init__() 14 | # groups参数就是深度可分离卷积的关键 15 | self.conv1 = nn.Conv2d(in_channels,in_channels,kernel_size=3,stride=stride, 16 | padding=1,groups=in_channels,bias=False) 17 | self.bn1 = nn.BatchNorm2d(in_channels) 18 | self.relu1 = nn.ReLU() 19 | self.conv2 = nn.Conv2d(in_channels,out_channels,kernel_size=1,stride=1,padding=0,bias=False) 20 | self.bn2 = nn.BatchNorm2d(out_channels) 21 | self.relu2 = nn.ReLU() 22 | def forward(self,x): 23 | x = self.relu1(self.bn1(self.conv1(x))) 24 | x = self.relu2(self.bn2(self.conv2(x))) 25 | return x 26 | 27 | # 深度可分离卷积 DepthWise Separable Convolution 28 | class MobileNet(nn.Module): 29 | # (128,2) means conv channel=128, conv stride=2, by default conv stride=1 30 | cfg = [64,(128,2),128,(256,2),256,(512,2),512,512,512,512,512,(1024,2),1024] 31 | 32 | def __init__(self, num_classes=10,alpha=1.0,beta=1.0,init_weights=True): 33 | super(MobileNet,self).__init__() 34 | self.conv1 = nn.Sequential( 35 | nn.Conv2d(3,32,kernel_size=3,stride=1,bias=False), 36 | nn.BatchNorm2d(32), 37 | nn.ReLU() 38 | ) 39 | self.avg = nn.AvgPool2d(kernel_size=2) 40 | self.layers = self._make_layers(in_channels=32) 41 | self.linear = nn.Linear(1024,num_classes) 42 | 43 | def _make_layers(self, in_channels): 44 | layers = [] 45 | for x in self.cfg: 46 | out_channels = x if isinstance(x,int) else x[0] 47 | stride = 1 if isinstance(x,int) else x[1] 48 | layers.append(Block(in_channels,out_channels,stride)) 49 | in_channels = out_channels 50 | return nn.Sequential(*layers) 51 | 52 | if init_weights: 53 | self._initialize_weights() 54 | 55 | def forward(self,x): 56 | x = self.conv1(x) 57 | x = self.layers(x) 58 | x = self.avg(x) 59 | x = x.view(x.size()[0],-1) 60 | x = self.linear(x) 61 | return x 62 | 63 | def _initialize_weights(self): 64 | for w in self.modules(): 65 | if isinstance(w, nn.Conv2d): 66 | nn.init.kaiming_normal_(w.weight, mode='fan_out') 67 | if w.bias is not None: 68 | nn.init.zeros_(w.bias) 69 | elif isinstance(w, nn.BatchNorm2d): 70 | nn.init.ones_(w.weight) 71 | nn.init.zeros_(w.bias) 72 | elif isinstance(w, nn.Linear): 73 | nn.init.normal_(w.weight, 0, 0.01) 74 | nn.init.zeros_(w.bias) 75 | 76 | 77 | def test(): 78 | net = MobileNet() 79 | x = torch.randn(2,3,32,32) 80 | y = net(x) 81 | print(y.size()) 82 | from torchinfo import summary 83 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 84 | net = net.to(device) 85 | summary(net,(2,3,32,32)) 86 | 87 | # test() -------------------------------------------------------------------------------- /CIFAR10/nets/MobileNetv2.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MobileNetV2 in PyTorch. 3 | 4 | See the paper "Inverted Residuals and Linear Bottlenecks: 5 | Mobile Networks for Classification, Detection and Segmentation" for more details. 6 | ''' 7 | import torch 8 | import torch.nn as nn 9 | 10 | 11 | class Block(nn.Module): 12 | # 使用了inverted residuals 13 | '''expand + depthwise + pointwise''' 14 | def __init__(self, in_channels, out_channels, expansion, stride): 15 | super(Block, self).__init__() 16 | self.stride = stride 17 | channels = expansion * in_channels # 倒残差结构先升维 再降维 18 | self.conv1 = nn.Conv2d(in_channels, channels, kernel_size = 1, stride=1, padding=0, bias=False) 19 | self.bn1 = nn.BatchNorm2d(channels) 20 | self.conv2 = nn.Conv2d(channels,channels,kernel_size=3,stride=stride,padding=1, groups=channels, bias=False) 21 | self.bn2 = nn.BatchNorm2d(channels) 22 | self.conv3 = nn.Conv2d(channels, out_channels, kernel_size=1,stride=1, padding = 0, bias=False) 23 | self.bn3 = nn.BatchNorm2d(out_channels) 24 | 25 | self.shortcut = nn.Sequential() 26 | if stride == 1 and in_channels != out_channels: 27 | self.shortcut = nn.Sequential( 28 | nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=1, padding=0, bias=False), 29 | nn.BatchNorm2d(out_channels) 30 | ) 31 | self.relu6 = nn.ReLU6() 32 | def forward(self, x): 33 | out = self.relu6(self.bn1(self.conv1(x))) 34 | out = self.relu6(self.bn2(self.conv2(out))) 35 | out = self.bn3(self.conv3(out)) 36 | out = out + self.shortcut(x) if self.stride == 1 else out 37 | 38 | return out 39 | 40 | class MobileNetV2(nn.Module): 41 | # (expansion, out_channels, num_blocks, stride) 42 | cfg = [(1, 16, 1, 1), 43 | (6, 24, 2, 1), # NOTE: change stride 2 -> 1 for CIFAR10 44 | (6, 32, 3, 2), 45 | (6, 64, 4, 2), 46 | (6, 96, 3, 1), 47 | (6, 160, 3, 2), 48 | (6, 320, 1, 1)] 49 | 50 | def __init__(self, num_classes=10): 51 | super(MobileNetV2, self).__init__() 52 | # NOTE: change conv1 stride 2 -> 1 for CIFAR10 53 | self.conv1 = nn.Conv2d(3, 32, kernel_size=3, stride=1, padding=1, bias=False) 54 | self.bn1 = nn.BatchNorm2d(32) 55 | self.layers = self._make_layers(in_channels=32) 56 | self.conv2 = nn.Conv2d(320, 1280, kernel_size=1, stride=1, padding=0, bias=False) 57 | self.bn2 = nn.BatchNorm2d(1280) 58 | self.avgpool = nn.AdaptiveAvgPool2d(1) 59 | self.linear = nn.Linear(1280, num_classes) 60 | self.relu6 = nn.ReLU6() 61 | 62 | def _make_layers(self, in_channels): 63 | layers = [] 64 | for expansion, out_channels, num_block, stride in self.cfg: 65 | strides = [stride] + [1]*(num_block-1) 66 | for stride in strides: 67 | layers.append(Block(in_channels, out_channels, expansion, stride)) 68 | in_channels = out_channels 69 | return nn.Sequential(*layers) 70 | 71 | 72 | def forward(self, x): 73 | out = self.relu6(self.bn1(self.conv1(x))) 74 | out = self.layers(out) 75 | out = self.relu6(self.bn2(self.conv2(out))) 76 | out = self.avgpool(out) 77 | out = out.view(out.size(0),-1) 78 | out = self.linear(out) 79 | return out 80 | 81 | 82 | def test(): 83 | net = MobileNetV2() 84 | x = torch.randn(2,3,32,32) 85 | y = net(x) 86 | print(y.size()) 87 | from torchinfo import summary 88 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 89 | net = net.to(device) 90 | summary(net,(2,3,32,32)) 91 | 92 | # test() -------------------------------------------------------------------------------- /CIFAR10/nets/RegNet.py: -------------------------------------------------------------------------------- 1 | '''RegNet in PyTorch. 2 | Paper: "Designing Network Design Spaces". 3 | Reference: https://github.com/keras-team/keras-applications/blob/master/keras_applications/efficientnet.py 4 | ''' 5 | import torch 6 | import torch.nn as nn 7 | import torch.nn.functional as F 8 | 9 | 10 | class SE(nn.Module): 11 | '''Squeeze-and-Excitation block.''' 12 | 13 | def __init__(self, in_planes, se_planes): 14 | super(SE, self).__init__() 15 | self.se1 = nn.Conv2d(in_planes, se_planes, kernel_size=1, bias=True) 16 | self.se2 = nn.Conv2d(se_planes, in_planes, kernel_size=1, bias=True) 17 | 18 | def forward(self, x): 19 | out = F.adaptive_avg_pool2d(x, (1, 1)) 20 | out = F.relu(self.se1(out)) 21 | out = self.se2(out).sigmoid() 22 | out = x * out 23 | return out 24 | 25 | 26 | class Block(nn.Module): 27 | def __init__(self, w_in, w_out, stride, group_width, bottleneck_ratio, se_ratio): 28 | super(Block, self).__init__() 29 | # 1x1 30 | w_b = int(round(w_out * bottleneck_ratio)) 31 | self.conv1 = nn.Conv2d(w_in, w_b, kernel_size=1, bias=False) 32 | self.bn1 = nn.BatchNorm2d(w_b) 33 | # 3x3 34 | num_groups = w_b // group_width 35 | self.conv2 = nn.Conv2d(w_b, w_b, kernel_size=3, 36 | stride=stride, padding=1, groups=num_groups, bias=False) 37 | self.bn2 = nn.BatchNorm2d(w_b) 38 | # se 39 | self.with_se = se_ratio > 0 40 | if self.with_se: 41 | w_se = int(round(w_in * se_ratio)) 42 | self.se = SE(w_b, w_se) 43 | # 1x1 44 | self.conv3 = nn.Conv2d(w_b, w_out, kernel_size=1, bias=False) 45 | self.bn3 = nn.BatchNorm2d(w_out) 46 | 47 | self.shortcut = nn.Sequential() 48 | if stride != 1 or w_in != w_out: 49 | self.shortcut = nn.Sequential( 50 | nn.Conv2d(w_in, w_out, 51 | kernel_size=1, stride=stride, bias=False), 52 | nn.BatchNorm2d(w_out) 53 | ) 54 | 55 | def forward(self, x): 56 | out = F.relu(self.bn1(self.conv1(x))) 57 | out = F.relu(self.bn2(self.conv2(out))) 58 | if self.with_se: 59 | out = self.se(out) 60 | out = self.bn3(self.conv3(out)) 61 | out += self.shortcut(x) 62 | out = F.relu(out) 63 | return out 64 | 65 | 66 | class RegNet(nn.Module): 67 | def __init__(self, cfg, num_classes=10): 68 | super(RegNet, self).__init__() 69 | self.cfg = cfg 70 | self.in_planes = 64 71 | self.conv1 = nn.Conv2d(3, 64, kernel_size=3, 72 | stride=1, padding=1, bias=False) 73 | self.bn1 = nn.BatchNorm2d(64) 74 | self.layer1 = self._make_layer(0) 75 | self.layer2 = self._make_layer(1) 76 | self.layer3 = self._make_layer(2) 77 | self.layer4 = self._make_layer(3) 78 | self.linear = nn.Linear(self.cfg['widths'][-1], num_classes) 79 | 80 | def _make_layer(self, idx): 81 | depth = self.cfg['depths'][idx] 82 | width = self.cfg['widths'][idx] 83 | stride = self.cfg['strides'][idx] 84 | group_width = self.cfg['group_width'] 85 | bottleneck_ratio = self.cfg['bottleneck_ratio'] 86 | se_ratio = self.cfg['se_ratio'] 87 | 88 | layers = [] 89 | for i in range(depth): 90 | s = stride if i == 0 else 1 91 | layers.append(Block(self.in_planes, width, 92 | s, group_width, bottleneck_ratio, se_ratio)) 93 | self.in_planes = width 94 | return nn.Sequential(*layers) 95 | 96 | def forward(self, x): 97 | out = F.relu(self.bn1(self.conv1(x))) 98 | out = self.layer1(out) 99 | out = self.layer2(out) 100 | out = self.layer3(out) 101 | out = self.layer4(out) 102 | out = F.adaptive_avg_pool2d(out, (1, 1)) 103 | out = out.view(out.size(0), -1) 104 | out = self.linear(out) 105 | return out 106 | 107 | 108 | def RegNetX_200MF(): 109 | cfg = { 110 | 'depths': [1, 1, 4, 7], 111 | 'widths': [24, 56, 152, 368], 112 | 'strides': [1, 1, 2, 2], 113 | 'group_width': 8, 114 | 'bottleneck_ratio': 1, 115 | 'se_ratio': 0, 116 | } 117 | return RegNet(cfg) 118 | 119 | 120 | def RegNetX_400MF(): 121 | cfg = { 122 | 'depths': [1, 2, 7, 12], 123 | 'widths': [32, 64, 160, 384], 124 | 'strides': [1, 1, 2, 2], 125 | 'group_width': 16, 126 | 'bottleneck_ratio': 1, 127 | 'se_ratio': 0, 128 | } 129 | return RegNet(cfg) 130 | 131 | 132 | def RegNetY_400MF(): 133 | cfg = { 134 | 'depths': [1, 2, 7, 12], 135 | 'widths': [32, 64, 160, 384], 136 | 'strides': [1, 1, 2, 2], 137 | 'group_width': 16, 138 | 'bottleneck_ratio': 1, 139 | 'se_ratio': 0.25, 140 | } 141 | return RegNet(cfg) 142 | 143 | 144 | def test(): 145 | net = RegNetX_200MF() 146 | print(net) 147 | x = torch.randn(2, 3, 32, 32) 148 | y = net(x) 149 | print(y.shape) 150 | 151 | 152 | if __name__ == '__main__': 153 | test() -------------------------------------------------------------------------------- /CIFAR10/nets/ResNet.py: -------------------------------------------------------------------------------- 1 | ''' 2 | ResNet in PyTorch. 3 | Reference: 4 | [1] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun 5 | Deep Residual Learning for Image Recognition. arXiv:1512.03385 6 | ''' 7 | import torch 8 | import torch.nn as nn 9 | class BasicBlock(nn.Module): 10 | """ 11 | 对于浅层网络，如ResNet-18/34等，用基本的Block 12 | 基础模块没有压缩,所以expansion=1 13 | """ 14 | expansion = 1 15 | def __init__(self, in_channels, out_channels, stride=1): 16 | super(BasicBlock,self).__init__() 17 | self.features = nn.Sequential( 18 | nn.Conv2d(in_channels, out_channels, kernel_size=3, stride=stride, padding=1, bias=False), 19 | nn.BatchNorm2d(out_channels), 20 | nn.ReLU(True), 21 | nn.Conv2d(out_channels,out_channels, kernel_size=3, stride=1, padding=1, bias=False), 22 | nn.BatchNorm2d(out_channels) 23 | ) 24 | # 如果输入输出维度不等，则使用1x1卷积层来改变维度 25 | self.shortcut = nn.Sequential() 26 | if stride != 1 or in_channels != self.expansion * out_channels: 27 | self.shortcut = nn.Sequential( 28 | nn.Conv2d(in_channels, self.expansion * out_channels, kernel_size=1, stride=stride, bias=False), 29 | nn.BatchNorm2d(self.expansion * out_channels), 30 | ) 31 | def forward(self, x): 32 | out = self.features(x) 33 | # print(out.shape) 34 | out += self.shortcut(x) 35 | out = torch.relu(out) 36 | return out 37 | 38 | 39 | class Bottleneck(nn.Module): 40 | """ 41 | 对于深层网络，我们使用BottleNeck，论文中提出其拥有近似的计算复杂度，但能节省很多资源 42 | zip_channels: 压缩后的维数，最后输出的维数是 expansion * zip_channels 43 | 针对ResNet50/101/152的网络结构,主要是因为第三层是第二层的4倍的关系所以expansion=4 44 | """ 45 | expansion = 4 46 | 47 | def __init__(self, in_channels, zip_channels, stride=1): 48 | super(Bottleneck, self).__init__() 49 | out_channels = self.expansion * zip_channels 50 | self.features = nn.Sequential( 51 | nn.Conv2d(in_channels, zip_channels, kernel_size=1, bias=False), 52 | nn.BatchNorm2d(zip_channels), 53 | nn.ReLU(inplace=True), 54 | nn.Conv2d(zip_channels, zip_channels, kernel_size=3, stride=stride, padding=1, bias=False), 55 | nn.BatchNorm2d(zip_channels), 56 | nn.ReLU(inplace=True), 57 | nn.Conv2d(zip_channels, out_channels, kernel_size=1, bias=False), 58 | nn.BatchNorm2d(out_channels) 59 | ) 60 | self.shortcut = nn.Sequential() 61 | if stride != 1 or in_channels != out_channels: 62 | self.shortcut = nn.Sequential( 63 | nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=stride, bias=False), 64 | nn.BatchNorm2d(out_channels) 65 | ) 66 | 67 | def forward(self, x): 68 | out = self.features(x) 69 | # print(out.shape) 70 | out += self.shortcut(x) 71 | out = torch.relu(out) 72 | return out 73 | 74 | class ResNet(nn.Module): 75 | """ 76 | 不同的ResNet架构都是统一的一层特征提取、四层残差，不同点在于每层残差的深度。 77 | 对于cifar10，feature map size的变化如下： 78 | (32, 32, 3) -> [Conv2d] -> (32, 32, 64) -> [Res1] -> (32, 32, 64) -> [Res2] 79 | -> (16, 16, 128) -> [Res3] -> (8, 8, 256) ->[Res4] -> (4, 4, 512) -> [AvgPool] 80 | -> (1, 1, 512) -> [Reshape] -> (512) -> [Linear] -> (10) 81 | """ 82 | def __init__(self, block, num_blocks, num_classes=10, verbose = False, init_weights=True): 83 | super(ResNet, self).__init__() 84 | self.verbose = verbose 85 | self.in_channels = 64 86 | self.features = nn.Sequential( 87 | nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False), 88 | nn.BatchNorm2d(64), 89 | nn.ReLU(inplace=True) 90 | ) 91 | #使用_make_layer函数生成上表对应的conv2_x, conv3_x, conv4_x, conv5_x的结构 92 | self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1) 93 | self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2) 94 | self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2) 95 | self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2) 96 | # cifar10经过上述结构后，到这里的feature map size是 4 x 4 x 512 x expansion 97 | # 所以这里用了 4 x 4 的平均池化 98 | self.avg_pool = nn.AvgPool2d(kernel_size=4) 99 | self.classifer = nn.Linear(512 * block.expansion, num_classes) 100 | 101 | if init_weights: 102 | self._initialize_weights() 103 | 104 | 105 | def _make_layer(self, block, out_channels, num_blocks, stride): 106 | # 第一个block要进行降采样 107 | strides = [stride] + [1] * (num_blocks - 1) 108 | layers = [] 109 | for stride in strides: 110 | layers.append(block(self.in_channels, out_channels, stride)) 111 | # 如果是Bottleneck Block的话需要对每层输入的维度进行压缩，压缩后再增加维数 112 | # 所以每层的输入维数也要跟着变 113 | self.in_channels = out_channels * block.expansion 114 | return nn.Sequential(*layers) 115 | 116 | def forward(self, x): 117 | out = self.features(x) 118 | if self.verbose: 119 | print('block 1 output: {}'.format(out.shape)) 120 | out = self.layer1(out) 121 | if self.verbose: 122 | print('block 2 output: {}'.format(out.shape)) 123 | out = self.layer2(out) 124 | if self.verbose: 125 | print('block 3 output: {}'.format(out.shape)) 126 | out = self.layer3(out) 127 | if self.verbose: 128 | print('block 4 output: {}'.format(out.shape)) 129 | out = self.layer4(out) 130 | if self.verbose: 131 | print('block 5 output: {}'.format(out.shape)) 132 | out = self.avg_pool(out) 133 | out = out.view(out.size(0), -1) 134 | out = self.classifer(out) 135 | return out 136 | 137 | def _initialize_weights(self): 138 | for m in self.modules(): 139 | if isinstance(m, nn.Conv2d): 140 | nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu') 141 | if m.bias is not None: 142 | nn.init.constant_(m.bias, 0) 143 | elif isinstance(m, nn.BatchNorm2d): 144 | nn.init.constant_(m.weight, 1) 145 | nn.init.constant_(m.bias, 0) 146 | elif isinstance(m, nn.Linear): 147 | nn.init.normal_(m.weight, 0, 0.01) 148 | nn.init.constant_(m.bias, 0) 149 | def ResNet18(verbose=False): 150 | return ResNet(BasicBlock, [2,2,2,2],verbose=verbose) 151 | 152 | def ResNet34(verbose=False): 153 | return ResNet(BasicBlock, [3,4,6,3],verbose=verbose) 154 | 155 | def ResNet50(verbose=False): 156 | return ResNet(Bottleneck, [3,4,6,3],verbose=verbose) 157 | 158 | def ResNet101(verbose=False): 159 | return ResNet(Bottleneck, [3,4,23,3],verbose=verbose) 160 | 161 | def ResNet152(verbose=False): 162 | return ResNet(Bottleneck, [3,8,36,3],verbose=verbose) 163 | 164 | def test(): 165 | net = ResNet34() 166 | x = torch.randn(2,3,32,32) 167 | y = net(x) 168 | print(y.size()) 169 | from torchinfo import summary 170 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 171 | net = net.to(device) 172 | summary(net,(2,3,32,32)) 173 | 174 | # test() -------------------------------------------------------------------------------- /CIFAR10/nets/SENet.py: -------------------------------------------------------------------------------- 1 | ''' 2 | SENet in PyTorch. 3 | SENet is the winner of ImageNet-2017. The paper is not released yet. 4 | ''' 5 | import torch 6 | import torch.nn as nn 7 | import torch.nn.functional as F 8 | 9 | 10 | class BasicBlock(nn.Module): 11 | def __init__(self, in_channels, channels, stride=1): 12 | super(BasicBlock, self).__init__() 13 | self.conv1 = nn.Conv2d(in_channels, channels, kernel_size=3, stride=stride, padding=1, bias=False) 14 | self.bn1 = nn.BatchNorm2d(channels) 15 | self.conv2 = nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1, bias=False) 16 | self.bn2 = nn.BatchNorm2d(channels) 17 | 18 | self.shortcut = nn.Sequential() 19 | if stride != 1 or in_channels != channels: 20 | self.shortcut = nn.Sequential( 21 | nn.Conv2d(in_channels, channels, kernel_size=1, stride=stride, bias=False), 22 | nn.BatchNorm2d(channels) 23 | ) 24 | 25 | # SE layers 26 | self.fc1 = nn.Conv2d(channels, channels//16, kernel_size=1) # Use nn.Conv2d instead of nn.Linear 27 | self.fc2 = nn.Conv2d(channels//16, channels, kernel_size=1) 28 | 29 | def forward(self, x): 30 | out = F.relu(self.bn1(self.conv1(x))) 31 | out = self.bn2(self.conv2(out)) 32 | 33 | # Squeeze 34 | w = F.avg_pool2d(out, out.size(2)) 35 | w = F.relu(self.fc1(w)) 36 | w = F.sigmoid(self.fc2(w)) 37 | # Excitation 38 | out = out * w 39 | 40 | out += self.shortcut(x) 41 | out = F.relu(out) 42 | return out 43 | 44 | 45 | class PreActBlock(nn.Module): 46 | def __init__(self, in_channels, channels, stride=1): 47 | super(PreActBlock, self).__init__() 48 | self.bn1 = nn.BatchNorm2d(in_channels) 49 | self.conv1 = nn.Conv2d(in_channels, channels, kernel_size=3, stride=stride, padding=1, bias=False) 50 | self.bn2 = nn.BatchNorm2d(channels) 51 | self.conv2 = nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1, bias=False) 52 | 53 | if stride != 1 or in_channels != channels: 54 | self.shortcut = nn.Sequential( 55 | nn.Conv2d(in_channels, channels, kernel_size=1, stride=stride, bias=False) 56 | ) 57 | 58 | # SE layers 59 | self.fc1 = nn.Conv2d(channels, channels//16, kernel_size=1) 60 | self.fc2 = nn.Conv2d(channels//16, channels, kernel_size=1) 61 | 62 | def forward(self, x): 63 | out = F.relu(self.bn1(x)) 64 | shortcut = self.shortcut(out) if hasattr(self, 'shortcut') else x 65 | out = self.conv1(out) 66 | out = self.conv2(F.relu(self.bn2(out))) 67 | 68 | # Squeeze 69 | w = F.avg_pool2d(out, out.size(2)) 70 | w = F.relu(self.fc1(w)) 71 | w = F.sigmoid(self.fc2(w)) 72 | # Excitation 73 | out = out * w 74 | 75 | out += shortcut 76 | return out 77 | 78 | 79 | class SENet(nn.Module): 80 | def __init__(self, block, num_blocks, num_classes=10): 81 | super(SENet, self).__init__() 82 | self.in_channels = 64 83 | 84 | self.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False) 85 | self.bn1 = nn.BatchNorm2d(64) 86 | self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1) 87 | self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2) 88 | self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2) 89 | self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2) 90 | self.linear = nn.Linear(512, num_classes) 91 | 92 | def _make_layer(self, block, channels, num_blocks, stride): 93 | strides = [stride] + [1]*(num_blocks-1) 94 | layers = [] 95 | for stride in strides: 96 | layers.append(block(self.in_channels, channels, stride)) 97 | self.in_channels = channels 98 | return nn.Sequential(*layers) 99 | 100 | def forward(self, x): 101 | out = F.relu(self.bn1(self.conv1(x))) 102 | out = self.layer1(out) 103 | out = self.layer2(out) 104 | out = self.layer3(out) 105 | out = self.layer4(out) 106 | out = F.avg_pool2d(out, 4) 107 | out = out.view(out.size(0), -1) 108 | out = self.linear(out) 109 | return out 110 | 111 | 112 | def SENet18(): 113 | return SENet(PreActBlock, [2,2,2,2]) 114 | 115 | 116 | def test(): 117 | net = SENet18() 118 | y = net(torch.randn(1,3,32,32)) 119 | print(y.size()) 120 | 121 | # test() -------------------------------------------------------------------------------- /CIFAR10/nets/ShuffleNet.py: -------------------------------------------------------------------------------- 1 | ''' 2 | ShuffleNet in PyTorch. 3 | See the paper "ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices" for more details. 4 | ''' 5 | 6 | import torch 7 | import torch.nn as nn 8 | import torch.nn.functional as F 9 | 10 | 11 | class ShuffleBlock(nn.Module): 12 | def __init__(self, groups): 13 | super(ShuffleBlock, self).__init__() 14 | self.groups = groups 15 | 16 | def forward(self, x): 17 | '''Channel shuffle: [N,C,H,W] -> [N,g,C/g,H,W] -> [N,C/g,g,H,w] -> [N,C,H,W]''' 18 | N,C,H,W = x.size() 19 | g = self.groups 20 | return x.view(N,g,C//g,H,W).permute(0,2,1,3,4).reshape(N,C,H,W) 21 | 22 | 23 | class Bottleneck(nn.Module): 24 | def __init__(self, in_channels, out_channels, stride, groups): 25 | super(Bottleneck, self).__init__() 26 | self.stride = stride 27 | 28 | mid_channels = out_channels/4 29 | g = 1 if in_channels==24 else groups 30 | self.conv1 = nn.Conv2d(in_channels, mid_channels, kernel_size=1, groups=g, bias=False) 31 | self.bn1 = nn.BatchNorm2d(mid_channels) 32 | self.shuffle1 = ShuffleBlock(groups=g) 33 | self.conv2 = nn.Conv2d(mid_channels, mid_channels, kernel_size=3, stride=stride, padding=1, groups=mid_channels, bias=False) 34 | self.bn2 = nn.BatchNorm2d(mid_channels) 35 | self.conv3 = nn.Conv2d(mid_channels, out_channels, kernel_size=1, groups=groups, bias=False) 36 | self.bn3 = nn.BatchNorm2d(out_channels) 37 | 38 | self.shortcut = nn.Sequential() 39 | if stride == 2: 40 | self.shortcut = nn.Sequential(nn.AvgPool2d(3, stride=2, padding=1)) 41 | 42 | def forward(self, x): 43 | out = F.relu(self.bn1(self.conv1(x))) 44 | out = self.shuffle1(out) 45 | out = F.relu(self.bn2(self.conv2(out))) 46 | out = self.bn3(self.conv3(out)) 47 | res = self.shortcut(x) 48 | out = F.relu(torch.cat([out,res], 1)) if self.stride==2 else F.relu(out+res) 49 | return out 50 | 51 | 52 | class ShuffleNet(nn.Module): 53 | def __init__(self, cfg): 54 | super(ShuffleNet, self).__init__() 55 | out_channels = cfg['out_channels'] 56 | num_blocks = cfg['num_blocks'] 57 | groups = cfg['groups'] 58 | 59 | self.conv1 = nn.Conv2d(3, 24, kernel_size=1, bias=False) 60 | self.bn1 = nn.BatchNorm2d(24) 61 | self.in_channels = 24 62 | self.layer1 = self._make_layer(out_channels[0], num_blocks[0], groups) 63 | self.layer2 = self._make_layer(out_channels[1], num_blocks[1], groups) 64 | self.layer3 = self._make_layer(out_channels[2], num_blocks[2], groups) 65 | self.linear = nn.Linear(out_channels[2], 10) 66 | 67 | def _make_layer(self, out_channels, num_blocks, groups): 68 | layers = [] 69 | for i in range(num_blocks): 70 | stride = 2 if i == 0 else 1 71 | cat_channels = self.in_channels if i == 0 else 0 72 | layers.append(Bottleneck(self.in_channels, out_channels-cat_channels, stride=stride, groups=groups)) 73 | self.in_channels = out_channels 74 | return nn.Sequential(*layers) 75 | 76 | def forward(self, x): 77 | out = F.relu(self.bn1(self.conv1(x))) 78 | out = self.layer1(out) 79 | out = self.layer2(out) 80 | out = self.layer3(out) 81 | out = F.avg_pool2d(out, 4) 82 | out = out.view(out.size(0), -1) 83 | out = self.linear(out) 84 | return out 85 | 86 | 87 | def ShuffleNetG2(): 88 | cfg = { 89 | 'out_channels': [200,400,800], 90 | 'num_blocks': [4,8,4], 91 | 'groups': 2 92 | } 93 | return ShuffleNet(cfg) 94 | 95 | def ShuffleNetG3(): 96 | cfg = { 97 | 'out_channels': [240,480,960], 98 | 'num_blocks': [4,8,4], 99 | 'groups': 3 100 | } 101 | return ShuffleNet(cfg) 102 | 103 | 104 | def test(): 105 | net = ShuffleNetG2() 106 | x = torch.randn(1,3,32,32) 107 | y = net(x) 108 | print(y) 109 | 110 | # test() -------------------------------------------------------------------------------- /CIFAR10/nets/ShuffleNetv2.py: -------------------------------------------------------------------------------- 1 | '''ShuffleNetV2 in PyTorch. 2 | See the paper "ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design" for more details. 3 | ''' 4 | import torch 5 | import torch.nn as nn 6 | import torch.nn.functional as F 7 | 8 | 9 | class ShuffleBlock(nn.Module): 10 | def __init__(self, groups=2): 11 | super(ShuffleBlock, self).__init__() 12 | self.groups = groups 13 | 14 | def forward(self, x): 15 | '''Channel shuffle: [N,C,H,W] -> [N,g,C/g,H,W] -> [N,C/g,g,H,w] -> [N,C,H,W]''' 16 | N, C, H, W = x.size() 17 | g = self.groups 18 | return x.view(N, g, C//g, H, W).permute(0, 2, 1, 3, 4).reshape(N, C, H, W) 19 | 20 | 21 | class SplitBlock(nn.Module): 22 | def __init__(self, ratio): 23 | super(SplitBlock, self).__init__() 24 | self.ratio = ratio 25 | 26 | def forward(self, x): 27 | c = int(x.size(1) * self.ratio) 28 | return x[:, :c, :, :], x[:, c:, :, :] 29 | 30 | 31 | class BasicBlock(nn.Module): 32 | def __init__(self, in_channels, split_ratio=0.5): 33 | super(BasicBlock, self).__init__() 34 | self.split = SplitBlock(split_ratio) 35 | in_channels = int(in_channels * split_ratio) 36 | self.conv1 = nn.Conv2d(in_channels, in_channels, 37 | kernel_size=1, bias=False) 38 | self.bn1 = nn.BatchNorm2d(in_channels) 39 | self.conv2 = nn.Conv2d(in_channels, in_channels, 40 | kernel_size=3, stride=1, padding=1, groups=in_channels, bias=False) 41 | self.bn2 = nn.BatchNorm2d(in_channels) 42 | self.conv3 = nn.Conv2d(in_channels, in_channels, 43 | kernel_size=1, bias=False) 44 | self.bn3 = nn.BatchNorm2d(in_channels) 45 | self.shuffle = ShuffleBlock() 46 | 47 | def forward(self, x): 48 | x1, x2 = self.split(x) 49 | out = F.relu(self.bn1(self.conv1(x2))) 50 | out = self.bn2(self.conv2(out)) 51 | out = F.relu(self.bn3(self.conv3(out))) 52 | out = torch.cat([x1, out], 1) 53 | out = self.shuffle(out) 54 | return out 55 | 56 | 57 | class DownBlock(nn.Module): 58 | def __init__(self, in_channels, out_channels): 59 | super(DownBlock, self).__init__() 60 | mid_channels = out_channels // 2 61 | # left 62 | self.conv1 = nn.Conv2d(in_channels, in_channels, 63 | kernel_size=3, stride=2, padding=1, groups=in_channels, bias=False) 64 | self.bn1 = nn.BatchNorm2d(in_channels) 65 | self.conv2 = nn.Conv2d(in_channels, mid_channels, 66 | kernel_size=1, bias=False) 67 | self.bn2 = nn.BatchNorm2d(mid_channels) 68 | # right 69 | self.conv3 = nn.Conv2d(in_channels, mid_channels, 70 | kernel_size=1, bias=False) 71 | self.bn3 = nn.BatchNorm2d(mid_channels) 72 | self.conv4 = nn.Conv2d(mid_channels, mid_channels, 73 | kernel_size=3, stride=2, padding=1, groups=mid_channels, bias=False) 74 | self.bn4 = nn.BatchNorm2d(mid_channels) 75 | self.conv5 = nn.Conv2d(mid_channels, mid_channels, 76 | kernel_size=1, bias=False) 77 | self.bn5 = nn.BatchNorm2d(mid_channels) 78 | 79 | self.shuffle = ShuffleBlock() 80 | 81 | def forward(self, x): 82 | # left 83 | out1 = self.bn1(self.conv1(x)) 84 | out1 = F.relu(self.bn2(self.conv2(out1))) 85 | # right 86 | out2 = F.relu(self.bn3(self.conv3(x))) 87 | out2 = self.bn4(self.conv4(out2)) 88 | out2 = F.relu(self.bn5(self.conv5(out2))) 89 | # concat 90 | out = torch.cat([out1, out2], 1) 91 | out = self.shuffle(out) 92 | return out 93 | 94 | 95 | class ShuffleNetV2(nn.Module): 96 | def __init__(self, net_size): 97 | super(ShuffleNetV2, self).__init__() 98 | out_channels = configs[net_size]['out_channels'] 99 | num_blocks = configs[net_size]['num_blocks'] 100 | 101 | self.conv1 = nn.Conv2d(3, 24, kernel_size=3, 102 | stride=1, padding=1, bias=False) 103 | self.bn1 = nn.BatchNorm2d(24) 104 | self.in_channels = 24 105 | self.layer1 = self._make_layer(out_channels[0], num_blocks[0]) 106 | self.layer2 = self._make_layer(out_channels[1], num_blocks[1]) 107 | self.layer3 = self._make_layer(out_channels[2], num_blocks[2]) 108 | self.conv2 = nn.Conv2d(out_channels[2], out_channels[3], 109 | kernel_size=1, stride=1, padding=0, bias=False) 110 | self.bn2 = nn.BatchNorm2d(out_channels[3]) 111 | self.linear = nn.Linear(out_channels[3], 10) 112 | 113 | def _make_layer(self, out_channels, num_blocks): 114 | layers = [DownBlock(self.in_channels, out_channels)] 115 | for i in range(num_blocks): 116 | layers.append(BasicBlock(out_channels)) 117 | self.in_channels = out_channels 118 | return nn.Sequential(*layers) 119 | 120 | def forward(self, x): 121 | out = F.relu(self.bn1(self.conv1(x))) 122 | # out = F.max_pool2d(out, 3, stride=2, padding=1) 123 | out = self.layer1(out) 124 | out = self.layer2(out) 125 | out = self.layer3(out) 126 | out = F.relu(self.bn2(self.conv2(out))) 127 | out = F.avg_pool2d(out, 4) 128 | out = out.view(out.size(0), -1) 129 | out = self.linear(out) 130 | return out 131 | 132 | 133 | configs = { 134 | 0.5: { 135 | 'out_channels': (48, 96, 192, 1024), 136 | 'num_blocks': (3, 7, 3) 137 | }, 138 | 139 | 1: { 140 | 'out_channels': (116, 232, 464, 1024), 141 | 'num_blocks': (3, 7, 3) 142 | }, 143 | 1.5: { 144 | 'out_channels': (176, 352, 704, 1024), 145 | 'num_blocks': (3, 7, 3) 146 | }, 147 | 2: { 148 | 'out_channels': (224, 488, 976, 2048), 149 | 'num_blocks': (3, 7, 3) 150 | } 151 | } 152 | 153 | 154 | def test(): 155 | net = ShuffleNetV2(net_size=0.5) 156 | x = torch.randn(3, 3, 32, 32) 157 | y = net(x) 158 | print(y.shape) 159 | 160 | 161 | # test() -------------------------------------------------------------------------------- /CIFAR10/nets/VGG.py: -------------------------------------------------------------------------------- 1 | ''' 2 | VGG11,13,16,19 in pytorch 3 | ''' 4 | 5 | from turtle import forward 6 | import torch 7 | import torch.nn as nn 8 | cfg = { 9 | 'VGG11': [64, 'M', 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'], 10 | 'VGG13': [64, 64, 'M', 128, 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'], 11 | 'VGG16': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 'M', 512, 512, 512, 'M', 512, 512, 512, 'M'], 12 | 'VGG19': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 256, 'M', 512, 512, 512, 512, 'M', 512, 512, 512, 512, 'M'], 13 | } 14 | 15 | class VGG(nn.Module): 16 | def __init__(self, vggname = 'VGG16',num_classes=10, init_weights=True): 17 | super(VGG,self).__init__() 18 | self.features = self._make_layers(cfg[vggname]) 19 | self.classifier = nn.Linear(512,num_classes) 20 | 21 | def _make_layers(self,cfg): 22 | layers = [] 23 | in_channels = 3 24 | for x in cfg: 25 | if x == 'M': # 最大池化层 26 | layers += [nn.MaxPool2d(kernel_size=2,stride=2)] 27 | else: 28 | layers += [nn.Conv2d(in_channels,out_channels=x,kernel_size=3,padding=1), 29 | nn.BatchNorm2d(x), 30 | nn.ReLU(inplace=True)] 31 | in_channels = x 32 | layers += [nn.AvgPool2d(kernel_size=1,stride=1)] 33 | return nn.Sequential(*layers) 34 | if init_weights: 35 | self. _initialize_weight() 36 | def forward(self,x): 37 | x = self.features(x) 38 | x = x.view(x.size()[0],-1) 39 | x = self.classifier(x) 40 | return x 41 | # 初始化参数 42 | def _initialize_weight(self): 43 | for m in self.modules(): 44 | if isinstance(m, nn.Conv2d): 45 | # xavier is used in VGG's paper 46 | nn.init.xavier_normal_(m.weight.data) 47 | if m.bias is not None: 48 | m.bias.data.zero_() 49 | elif isinstance(m, nn.BatchNorm2d): 50 | m.weight.data.fill_(1) 51 | m.bias.data.zero_() 52 | elif isinstance(m, nn.Linear): 53 | m.weight.data.normal_(0, 0.01) 54 | m.bias.data.zero_() 55 | def test(): 56 | net = VGG('VGG19') 57 | x = torch.randn(2,3,32,32) 58 | y = net(x) 59 | print(y.size()) 60 | from torchinfo import summary 61 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 62 | net = net.to(device) 63 | summary(net,(2,3,32,32)) 64 | 65 | # test() -------------------------------------------------------------------------------- /CIFAR10/nets/pnasnet.py: -------------------------------------------------------------------------------- 1 | '''PNASNet in PyTorch. 2 | Paper: Progressive Neural Architecture Search 3 | ''' 4 | import torch 5 | import torch.nn as nn 6 | import torch.nn.functional as F 7 | 8 | 9 | class SepConv(nn.Module): 10 | '''Separable Convolution.''' 11 | def __init__(self, in_planes, out_planes, kernel_size, stride): 12 | super(SepConv, self).__init__() 13 | self.conv1 = nn.Conv2d(in_planes, out_planes, 14 | kernel_size, stride, 15 | padding=(kernel_size-1)//2, 16 | bias=False, groups=in_planes) 17 | self.bn1 = nn.BatchNorm2d(out_planes) 18 | 19 | def forward(self, x): 20 | return self.bn1(self.conv1(x)) 21 | 22 | 23 | class CellA(nn.Module): 24 | def __init__(self, in_planes, out_planes, stride=1): 25 | super(CellA, self).__init__() 26 | self.stride = stride 27 | self.sep_conv1 = SepConv(in_planes, out_planes, kernel_size=7, stride=stride) 28 | if stride==2: 29 | self.conv1 = nn.Conv2d(in_planes, out_planes, kernel_size=1, stride=1, padding=0, bias=False) 30 | self.bn1 = nn.BatchNorm2d(out_planes) 31 | 32 | def forward(self, x): 33 | y1 = self.sep_conv1(x) 34 | y2 = F.max_pool2d(x, kernel_size=3, stride=self.stride, padding=1) 35 | if self.stride==2: 36 | y2 = self.bn1(self.conv1(y2)) 37 | return F.relu(y1+y2) 38 | 39 | class CellB(nn.Module): 40 | def __init__(self, in_planes, out_planes, stride=1): 41 | super(CellB, self).__init__() 42 | self.stride = stride 43 | # Left branch 44 | self.sep_conv1 = SepConv(in_planes, out_planes, kernel_size=7, stride=stride) 45 | self.sep_conv2 = SepConv(in_planes, out_planes, kernel_size=3, stride=stride) 46 | # Right branch 47 | self.sep_conv3 = SepConv(in_planes, out_planes, kernel_size=5, stride=stride) 48 | if stride==2: 49 | self.conv1 = nn.Conv2d(in_planes, out_planes, kernel_size=1, stride=1, padding=0, bias=False) 50 | self.bn1 = nn.BatchNorm2d(out_planes) 51 | # Reduce channels 52 | self.conv2 = nn.Conv2d(2*out_planes, out_planes, kernel_size=1, stride=1, padding=0, bias=False) 53 | self.bn2 = nn.BatchNorm2d(out_planes) 54 | 55 | def forward(self, x): 56 | # Left branch 57 | y1 = self.sep_conv1(x) 58 | y2 = self.sep_conv2(x) 59 | # Right branch 60 | y3 = F.max_pool2d(x, kernel_size=3, stride=self.stride, padding=1) 61 | if self.stride==2: 62 | y3 = self.bn1(self.conv1(y3)) 63 | y4 = self.sep_conv3(x) 64 | # Concat & reduce channels 65 | b1 = F.relu(y1+y2) 66 | b2 = F.relu(y3+y4) 67 | y = torch.cat([b1,b2], 1) 68 | return F.relu(self.bn2(self.conv2(y))) 69 | 70 | class PNASNet(nn.Module): 71 | def __init__(self, cell_type, num_cells, num_planes): 72 | super(PNASNet, self).__init__() 73 | self.in_planes = num_planes 74 | self.cell_type = cell_type 75 | 76 | self.conv1 = nn.Conv2d(3, num_planes, kernel_size=3, stride=1, padding=1, bias=False) 77 | self.bn1 = nn.BatchNorm2d(num_planes) 78 | 79 | self.layer1 = self._make_layer(num_planes, num_cells=6) 80 | self.layer2 = self._downsample(num_planes*2) 81 | self.layer3 = self._make_layer(num_planes*2, num_cells=6) 82 | self.layer4 = self._downsample(num_planes*4) 83 | self.layer5 = self._make_layer(num_planes*4, num_cells=6) 84 | 85 | self.linear = nn.Linear(num_planes*4, 10) 86 | 87 | def _make_layer(self, planes, num_cells): 88 | layers = [] 89 | for _ in range(num_cells): 90 | layers.append(self.cell_type(self.in_planes, planes, stride=1)) 91 | self.in_planes = planes 92 | return nn.Sequential(*layers) 93 | 94 | def _downsample(self, planes): 95 | layer = self.cell_type(self.in_planes, planes, stride=2) 96 | self.in_planes = planes 97 | return layer 98 | 99 | def forward(self, x): 100 | out = F.relu(self.bn1(self.conv1(x))) 101 | out = self.layer1(out) 102 | out = self.layer2(out) 103 | out = self.layer3(out) 104 | out = self.layer4(out) 105 | out = self.layer5(out) 106 | out = F.avg_pool2d(out, 8) 107 | out = self.linear(out.view(out.size(0), -1)) 108 | return out 109 | 110 | 111 | def PNASNetA(): 112 | return PNASNet(CellA, num_cells=6, num_planes=44) 113 | 114 | def PNASNetB(): 115 | return PNASNet(CellB, num_cells=6, num_planes=32) 116 | 117 | 118 | def test(): 119 | net = PNASNetB() 120 | x = torch.randn(1,3,32,32) 121 | y = net(x) 122 | print(y) 123 | 124 | # test() -------------------------------------------------------------------------------- /CIFAR10/nets/preact_resnet.py: -------------------------------------------------------------------------------- 1 | '''Pre-activation ResNet in PyTorch. 2 | Reference: 3 | [1] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun 4 | Identity Mappings in Deep Residual Networks. arXiv:1603.05027 5 | ''' 6 | import torch 7 | import torch.nn as nn 8 | import torch.nn.functional as F 9 | 10 | 11 | class PreActBlock(nn.Module): 12 | '''Pre-activation version of the BasicBlock.''' 13 | expansion = 1 14 | 15 | def __init__(self, in_planes, planes, stride=1): 16 | super(PreActBlock, self).__init__() 17 | self.bn1 = nn.BatchNorm2d(in_planes) 18 | self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) 19 | self.bn2 = nn.BatchNorm2d(planes) 20 | self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=1, padding=1, bias=False) 21 | 22 | if stride != 1 or in_planes != self.expansion*planes: 23 | self.shortcut = nn.Sequential( 24 | nn.Conv2d(in_planes, self.expansion*planes, kernel_size=1, stride=stride, bias=False) 25 | ) 26 | 27 | def forward(self, x): 28 | out = F.relu(self.bn1(x)) 29 | shortcut = self.shortcut(out) if hasattr(self, 'shortcut') else x 30 | out = self.conv1(out) 31 | out = self.conv2(F.relu(self.bn2(out))) 32 | out += shortcut 33 | return out 34 | 35 | 36 | class PreActBottleneck(nn.Module): 37 | '''Pre-activation version of the original Bottleneck module.''' 38 | expansion = 4 39 | 40 | def __init__(self, in_planes, planes, stride=1): 41 | super(PreActBottleneck, self).__init__() 42 | self.bn1 = nn.BatchNorm2d(in_planes) 43 | self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=1, bias=False) 44 | self.bn2 = nn.BatchNorm2d(planes) 45 | self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) 46 | self.bn3 = nn.BatchNorm2d(planes) 47 | self.conv3 = nn.Conv2d(planes, self.expansion*planes, kernel_size=1, bias=False) 48 | 49 | if stride != 1 or in_planes != self.expansion*planes: 50 | self.shortcut = nn.Sequential( 51 | nn.Conv2d(in_planes, self.expansion*planes, kernel_size=1, stride=stride, bias=False) 52 | ) 53 | 54 | def forward(self, x): 55 | out = F.relu(self.bn1(x)) 56 | shortcut = self.shortcut(out) if hasattr(self, 'shortcut') else x 57 | out = self.conv1(out) 58 | out = self.conv2(F.relu(self.bn2(out))) 59 | out = self.conv3(F.relu(self.bn3(out))) 60 | out += shortcut 61 | return out 62 | 63 | 64 | class PreActResNet(nn.Module): 65 | def __init__(self, block, num_blocks, num_classes=10): 66 | super(PreActResNet, self).__init__() 67 | self.in_planes = 64 68 | 69 | self.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False) 70 | self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1) 71 | self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2) 72 | self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2) 73 | self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2) 74 | self.linear = nn.Linear(512*block.expansion, num_classes) 75 | 76 | def _make_layer(self, block, planes, num_blocks, stride): 77 | strides = [stride] + [1]*(num_blocks-1) 78 | layers = [] 79 | for stride in strides: 80 | layers.append(block(self.in_planes, planes, stride)) 81 | self.in_planes = planes * block.expansion 82 | return nn.Sequential(*layers) 83 | 84 | def forward(self, x): 85 | out = self.conv1(x) 86 | out = self.layer1(out) 87 | out = self.layer2(out) 88 | out = self.layer3(out) 89 | out = self.layer4(out) 90 | out = F.avg_pool2d(out, 4) 91 | out = out.view(out.size(0), -1) 92 | out = self.linear(out) 93 | return out 94 | 95 | 96 | def PreActResNet18(): 97 | return PreActResNet(PreActBlock, [2,2,2,2]) 98 | 99 | def PreActResNet34(): 100 | return PreActResNet(PreActBlock, [3,4,6,3]) 101 | 102 | def PreActResNet50(): 103 | return PreActResNet(PreActBottleneck, [3,4,6,3]) 104 | 105 | def PreActResNet101(): 106 | return PreActResNet(PreActBottleneck, [3,4,23,3]) 107 | 108 | def PreActResNet152(): 109 | return PreActResNet(PreActBottleneck, [3,8,36,3]) 110 | 111 | 112 | def test(): 113 | net = PreActResNet18() 114 | y = net((torch.randn(1,3,32,32))) 115 | print(y.size()) 116 | 117 | # test() -------------------------------------------------------------------------------- /CIFAR10/nets/squeezeNet.py: -------------------------------------------------------------------------------- 1 | """squeezenet in pytorch 2 | [1] Song Han, Jeff Pool, John Tran, William J. Dally 3 | squeezenet: Learning both Weights and Connections for Efficient Neural Networks 4 | https://arxiv.org/abs/1506.02626 5 | """ 6 | 7 | import torch 8 | import torch.nn as nn 9 | 10 | 11 | class Fire(nn.Module): 12 | 13 | def __init__(self, in_channel, out_channel, squzee_channel): 14 | 15 | super().__init__() 16 | self.squeeze = nn.Sequential( 17 | nn.Conv2d(in_channel, squzee_channel, 1), 18 | nn.BatchNorm2d(squzee_channel), 19 | nn.ReLU(inplace=True) 20 | ) 21 | 22 | self.expand_1x1 = nn.Sequential( 23 | nn.Conv2d(squzee_channel, int(out_channel / 2), 1), 24 | nn.BatchNorm2d(int(out_channel / 2)), 25 | nn.ReLU(inplace=True) 26 | ) 27 | 28 | self.expand_3x3 = nn.Sequential( 29 | nn.Conv2d(squzee_channel, int(out_channel / 2), 3, padding=1), 30 | nn.BatchNorm2d(int(out_channel / 2)), 31 | nn.ReLU(inplace=True) 32 | ) 33 | 34 | def forward(self, x): 35 | 36 | x = self.squeeze(x) 37 | x = torch.cat([ 38 | self.expand_1x1(x), 39 | self.expand_3x3(x) 40 | ], 1) 41 | 42 | return x 43 | 44 | class SqueezeNet(nn.Module): 45 | 46 | """mobile net with simple bypass""" 47 | def __init__(self, class_num=100): 48 | 49 | super().__init__() 50 | self.stem = nn.Sequential( 51 | nn.Conv2d(3, 96, 3, padding=1), 52 | nn.BatchNorm2d(96), 53 | nn.ReLU(inplace=True), 54 | nn.MaxPool2d(2, 2) 55 | ) 56 | 57 | self.fire2 = Fire(96, 128, 16) 58 | self.fire3 = Fire(128, 128, 16) 59 | self.fire4 = Fire(128, 256, 32) 60 | self.fire5 = Fire(256, 256, 32) 61 | self.fire6 = Fire(256, 384, 48) 62 | self.fire7 = Fire(384, 384, 48) 63 | self.fire8 = Fire(384, 512, 64) 64 | self.fire9 = Fire(512, 512, 64) 65 | 66 | self.conv10 = nn.Conv2d(512, class_num, 1) 67 | self.avg = nn.AdaptiveAvgPool2d(1) 68 | self.maxpool = nn.MaxPool2d(2, 2) 69 | 70 | def forward(self, x): 71 | x = self.stem(x) 72 | 73 | f2 = self.fire2(x) 74 | f3 = self.fire3(f2) + f2 75 | f4 = self.fire4(f3) 76 | f4 = self.maxpool(f4) 77 | 78 | f5 = self.fire5(f4) + f4 79 | f6 = self.fire6(f5) 80 | f7 = self.fire7(f6) + f6 81 | f8 = self.fire8(f7) 82 | f8 = self.maxpool(f8) 83 | 84 | f9 = self.fire9(f8) 85 | c10 = self.conv10(f9) 86 | 87 | x = self.avg(c10) 88 | x = x.view(x.size(0), -1) 89 | 90 | return x 91 | 92 | def squeezenet(class_num=100): 93 | return SqueezeNet(class_num=class_num) -------------------------------------------------------------------------------- /CIFAR10/nets/wideresidual.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | 4 | 5 | class WideBasic(nn.Module): 6 | 7 | def __init__(self, in_channels, out_channels, stride=1): 8 | super().__init__() 9 | self.residual = nn.Sequential( 10 | nn.BatchNorm2d(in_channels), 11 | nn.ReLU(inplace=True), 12 | nn.Conv2d( 13 | in_channels, 14 | out_channels, 15 | kernel_size=3, 16 | stride=stride, 17 | padding=1 18 | ), 19 | nn.BatchNorm2d(out_channels), 20 | nn.ReLU(inplace=True), 21 | nn.Dropout(), 22 | nn.Conv2d( 23 | out_channels, 24 | out_channels, 25 | kernel_size=3, 26 | stride=1, 27 | padding=1 28 | ) 29 | ) 30 | 31 | self.shortcut = nn.Sequential() 32 | 33 | if in_channels != out_channels or stride != 1: 34 | self.shortcut = nn.Sequential( 35 | nn.Conv2d(in_channels, out_channels, 1, stride=stride) 36 | ) 37 | 38 | def forward(self, x): 39 | 40 | residual = self.residual(x) 41 | shortcut = self.shortcut(x) 42 | 43 | return residual + shortcut 44 | 45 | class WideResNet(nn.Module): 46 | def __init__(self, num_classes, block, depth=50, widen_factor=1): 47 | super().__init__() 48 | 49 | self.depth = depth 50 | k = widen_factor 51 | l = int((depth - 4) / 6) 52 | self.in_channels = 16 53 | self.init_conv = nn.Conv2d(3, self.in_channels, 3, 1, padding=1) 54 | self.conv2 = self._make_layer(block, 16 * k, l, 1) 55 | self.conv3 = self._make_layer(block, 32 * k, l, 2) 56 | self.conv4 = self._make_layer(block, 64 * k, l, 2) 57 | self.bn = nn.BatchNorm2d(64 * k) 58 | self.relu = nn.ReLU(inplace=True) 59 | self.avg_pool = nn.AdaptiveAvgPool2d((1, 1)) 60 | self.linear = nn.Linear(64 * k, num_classes) 61 | 62 | def forward(self, x): 63 | x = self.init_conv(x) 64 | x = self.conv2(x) 65 | x = self.conv3(x) 66 | x = self.conv4(x) 67 | x = self.bn(x) 68 | x = self.relu(x) 69 | x = self.avg_pool(x) 70 | x = x.view(x.size(0), -1) 71 | x = self.linear(x) 72 | 73 | return x 74 | 75 | def _make_layer(self, block, out_channels, num_blocks, stride): 76 | """make resnet layers(by layer i didnt mean this 'layer' was the 77 | same as a neuron netowork layer, ex. conv layer), one layer may 78 | contain more than one residual block 79 | Args: 80 | block: block type, basic block or bottle neck block 81 | out_channels: output depth channel number of this layer 82 | num_blocks: how many blocks per layer 83 | stride: the stride of the first block of this layer 84 | Return: 85 | return a resnet layer 86 | """ 87 | 88 | # we have num_block blocks per layer, the first block 89 | # could be 1 or 2, other blocks would always be 1 90 | strides = [stride] + [1] * (num_blocks - 1) 91 | layers = [] 92 | for stride in strides: 93 | layers.append(block(self.in_channels, out_channels, stride)) 94 | self.in_channels = out_channels 95 | 96 | return nn.Sequential(*layers) 97 | 98 | 99 | # Table 9: Best WRN performance over various datasets, single run results. 100 | def wideresnet(depth=40, widen_factor=10): 101 | net = WideResNet(100, WideBasic, depth=depth, widen_factor=widen_factor) 102 | return net 103 | 104 | import math 105 | import torch 106 | import torch.nn as nn 107 | import torch.nn.functional as F 108 | 109 | 110 | class BasicBlock(nn.Module): 111 | def __init__(self, in_planes, out_planes, stride, drop_rate=0.0): 112 | super(BasicBlock, self).__init__() 113 | self.bn1 = nn.BatchNorm2d(in_planes) 114 | self.relu1 = nn.ReLU(inplace=True) 115 | self.conv1 = nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride, 116 | padding=1, bias=False) 117 | self.bn2 = nn.BatchNorm2d(out_planes) 118 | self.relu2 = nn.ReLU(inplace=True) 119 | self.conv2 = nn.Conv2d(out_planes, out_planes, kernel_size=3, stride=1, 120 | padding=1, bias=False) 121 | self.droprate = drop_rate 122 | self.equalInOut = (in_planes == out_planes) 123 | self.convShortcut = (not self.equalInOut) and nn.Conv2d(in_planes, out_planes, kernel_size=1, stride=stride, 124 | padding=0, bias=False) or None 125 | 126 | def forward(self, x): 127 | if not self.equalInOut: 128 | x = self.relu1(self.bn1(x)) 129 | else: 130 | out = self.relu1(self.bn1(x)) 131 | 132 | out = self.relu2(self.bn2(self.conv1(out if self.equalInOut else x))) 133 | if self.droprate > 0: 134 | out = F.dropout(out, p=self.droprate, training=self.training) 135 | out = self.conv2(out) 136 | return torch.add(x if self.equalInOut else self.convShortcut(x), out) 137 | 138 | 139 | class NetworkBlock(nn.Module): 140 | def __init__(self, nb_layers, in_planes, out_planes, block, stride, dropRate=0.0): 141 | super(NetworkBlock, self).__init__() 142 | self.layer = self._make_layer(block, in_planes, out_planes, nb_layers, stride, dropRate) 143 | 144 | @staticmethod 145 | def _make_layer(block, in_planes, out_planes, nb_layers, stride, dropRate): 146 | layers = [] 147 | for i in range(nb_layers): 148 | layers.append(block(i == 0 and in_planes or out_planes, out_planes, i == 0 and stride or 1, dropRate)) 149 | return nn.Sequential(*layers) 150 | 151 | def forward(self, x): 152 | return self.layer(x) 153 | 154 | 155 | class WideResNet(nn.Module): 156 | def __init__(self, depth, num_classes, widen_factor=1, drop_rate=0.0): 157 | super(WideResNet, self).__init__() 158 | n_channels = [16, 16 * widen_factor, 32 * widen_factor, 64 * widen_factor] 159 | assert ((depth - 4) % 6 == 0) 160 | n = int((depth - 4) / 6) 161 | block = BasicBlock 162 | # 1st conv before any network block 163 | self.conv1 = nn.Conv2d(3, n_channels[0], kernel_size=3, stride=1, 164 | padding=1, bias=False) 165 | # 1st block 166 | self.block1 = NetworkBlock(n, n_channels[0], n_channels[1], block, 1, drop_rate) 167 | # 2nd block 168 | self.block2 = NetworkBlock(n, n_channels[1], n_channels[2], block, 2, drop_rate) 169 | # 3rd block 170 | self.block3 = NetworkBlock(n, n_channels[2], n_channels[3], block, 2, drop_rate) 171 | # global average pooling and classifier 172 | self.bn1 = nn.BatchNorm2d(n_channels[3]) 173 | self.relu = nn.ReLU(inplace=True) 174 | self.fc = nn.Linear(n_channels[3], num_classes) 175 | self.nChannels = n_channels[3] 176 | 177 | for m in self.modules(): 178 | if isinstance(m, nn.Conv2d): 179 | n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels 180 | m.weight.data.normal_(0, math.sqrt(2. / n)) 181 | elif isinstance(m, nn.BatchNorm2d): 182 | m.weight.data.fill_(1) 183 | m.bias.data.zero_() 184 | elif isinstance(m, nn.Linear): 185 | m.bias.data.zero_() 186 | 187 | def forward(self, x): 188 | out = self.conv1(x) 189 | out = self.block1(out) 190 | out = self.block2(out) 191 | out = self.block3(out) 192 | out = self.relu(self.bn1(out)) 193 | out = F.avg_pool2d(out, 8) 194 | out = out.view(-1, self.nChannels) 195 | return self.fc(out) -------------------------------------------------------------------------------- /CIFAR10/requirements.txt: -------------------------------------------------------------------------------- 1 | torch 2 | torchvision 3 | numpy 4 | matplotlib 5 | requests 6 | pillow 7 | tqdm 8 | torchinfo 9 | tensorboardX 10 | setuptools==59.5.0 -------------------------------------------------------------------------------- /CIFAR10/train.py: -------------------------------------------------------------------------------- 1 | '''Train CIFAR10 with PyTorch.''' 2 | import torch 3 | import torch.nn as nn 4 | import torch.optim as optim 5 | import torch.backends.cudnn as cudnn 6 | import torchvision 7 | import torchvision.transforms as transforms 8 | 9 | import os 10 | import argparse 11 | from utils import get_acc,EarlyStopping 12 | from dataloader import get_test_dataloader, get_training_dataloader 13 | from tqdm import tqdm 14 | 15 | 16 | classes = ('plane', 'car', 'bird', 'cat', 'deer', 17 | 'dog', 'frog', 'horse', 'ship', 'truck') 18 | 19 | if __name__ == '__main__': 20 | parser = argparse.ArgumentParser(description='PyTorch CIFAR10 Training') 21 | parser.add_argument('--lr', default=0.1, type=float, help='learning rate') 22 | parser.add_argument('--cuda', action='store_true', default=False, help =' use GPU?') 23 | parser.add_argument('--batch-size', default=64, type=int, help = "Batch Size for Training") 24 | parser.add_argument('--num-workers', default=2, type=int, help = 'num-workers') 25 | parser.add_argument('--net', type = str, choices=['LeNet5', 'AlexNet', 'VGG16','VGG19','ResNet18','ResNet34', 26 | 'DenseNet','MobileNetv1','MobileNetv2'], default='MobileNetv1', help='net type') 27 | parser.add_argument('--epochs', type = int, default=20, help = 'Epochs') 28 | parser.add_argument('--resume', '-r', action='store_true', help='resume from checkpoint') 29 | parser.add_argument('--patience', '-p', type = int, default=7, help='patience for Early stop') 30 | parser.add_argument('--optim','-o',type = str, choices = ['sgd','adam','adamw'], default = 'adamw', help = 'choose optimizer') 31 | 32 | args = parser.parse_args() 33 | 34 | print(args) 35 | best_acc = 0 # best test accuracy 36 | start_epoch = 0 # start from epoch 0 or last checkpoint epoch 37 | 38 | # Train Data 39 | trainloader = get_training_dataloader(batch_size = args.batch_size, num_workers = args.num_workers) 40 | testloader = get_test_dataloader(batch_size = args.batch_size, num_workers = args.num_workers, shuffle=False) 41 | # Model 42 | print('==> Building model..') 43 | if args.net == 'VGG16': 44 | from nets.VGG import VGG 45 | net = VGG('VGG16') 46 | elif args.net == 'VGG19': 47 | from nets.VGG import VGG 48 | net = VGG('VGG19') 49 | elif args.net == 'ResNet18': 50 | from nets.ResNet import ResNet18 51 | net = ResNet18() 52 | elif args.net == 'ResNet34': 53 | from nets.ResNet import ResNet34 54 | net = ResNet34() 55 | elif args.net == 'LeNet5': 56 | from nets.LeNet5 import LeNet5 57 | net = LeNet5() 58 | elif args.net == 'AlexNet': 59 | from nets.AlexNet import AlexNet 60 | net = AlexNet() 61 | elif args.net == 'DenseNet': 62 | from nets.DenseNet import densenet_cifar 63 | net = densenet_cifar() 64 | elif args.net == 'MobileNetv1': 65 | from nets.MobileNetv1 import MobileNet 66 | net = MobileNet() 67 | elif args.net == 'MobileNetv2': 68 | from nets.MobileNetv2 import MobileNetV2 69 | net = MobileNetV2() 70 | 71 | if args.cuda: 72 | device = 'cuda' 73 | net = torch.nn.DataParallel(net) 74 | # 当计算图不会改变的时候（每次输入形状相同，模型不改变）的情况下可以提高性能，反之则降低性能 75 | torch.backends.cudnn.benchmark = True 76 | else: 77 | device = 'cpu' 78 | 79 | 80 | if args.resume: 81 | # Load checkpoint. 82 | print('==> Resuming from checkpoint..') 83 | assert os.path.isdir('checkpoint'), 'Error: no checkpoint directory found!' 84 | checkpoint = torch.load('./checkpoint/{}_ckpt.pth'.format(args.net)) 85 | net.load_state_dict(checkpoint['net']) 86 | best_acc = checkpoint['acc'] 87 | start_epoch = checkpoint['epoch'] 88 | args.lr = checkpoint['lr'] 89 | 90 | early_stopping = EarlyStopping(patience = args.patience, verbose=True) 91 | criterion = nn.CrossEntropyLoss() 92 | if args.optim == 'adamw': 93 | optimizer = optim.AdamW(net.parameters(), lr=args.lr) 94 | elif args.optim == 'adam': 95 | optimizer = optim.Adam(net.parameters(), lr=args.lr) 96 | else: 97 | optimizer = optim.SGD(net.parameters(), lr=args.lr, 98 | momentum=0.9, weight_decay=5e-4) 99 | scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, 'min', factor=0.94,verbose=True,patience = 1,min_lr = 0.000001) # 动态更新学习率 100 | 101 | epochs = args.epochs 102 | def train(epoch): 103 | epoch_step = len(trainloader) 104 | if epoch_step == 0: 105 | raise ValueError("数据集过小，无法进行训练，请扩充数据集，或者减小batchsize") 106 | net.train() 107 | train_loss = 0 108 | train_acc = 0 109 | print('Start Train') 110 | with tqdm(total=epoch_step,desc=f'Epoch {epoch + 1}/{epochs}',postfix=dict,mininterval=0.3) as pbar: 111 | for step,(im,label) in enumerate(trainloader,start=0): 112 | im = im.to(device) 113 | label = label.to(device) 114 | #--------------------- 115 | # 释放内存 116 | #--------------------- 117 | if hasattr(torch.cuda, 'empty_cache'): 118 | torch.cuda.empty_cache() 119 | #----------------------# 120 | # 清零梯度 121 | #----------------------# 122 | optimizer.zero_grad() 123 | #----------------------# 124 | # 前向传播forward 125 | #----------------------# 126 | outputs = net(im) 127 | #----------------------# 128 | # 计算损失 129 | #----------------------# 130 | loss = criterion(outputs,label) 131 | train_loss += loss.data 132 | train_acc += get_acc(outputs,label) 133 | #----------------------# 134 | # 反向传播 135 | #----------------------# 136 | # backward 137 | loss.backward() 138 | # 更新参数 139 | optimizer.step() 140 | lr = optimizer.param_groups[0]['lr'] 141 | pbar.set_postfix(**{'Train Loss' : train_loss.item()/(step+1), 142 | 'Train Acc' :train_acc.item()/(step+1), 143 | 'Lr' : lr}) 144 | pbar.update(1) 145 | # train_loss = train_loss.item() / len(trainloader) 146 | # train_acc = train_acc.item() * 100 / len(trainloader) 147 | scheduler.step(train_loss) 148 | print('Finish Train') 149 | def test(epoch): 150 | global best_acc 151 | epoch_step_test = len(testloader) 152 | if epoch_step_test == 0: 153 | raise ValueError("数据集过小，无法进行训练，请扩充数据集，或者减小batchsize") 154 | 155 | net.eval() 156 | test_loss = 0 157 | test_acc = 0 158 | print('Start Test') 159 | #-------------------------------- 160 | # 相同方法，同train 161 | #-------------------------------- 162 | with tqdm(total=epoch_step_test,desc=f'Epoch {epoch + 1}/{epochs}',postfix=dict,mininterval=0.3) as pbar2: 163 | for step,(im,label) in enumerate(testloader,start=0): 164 | im = im.to(device) 165 | label = label.to(device) 166 | with torch.no_grad(): 167 | if step >= epoch_step_test: 168 | break 169 | 170 | # 释放内存 171 | if hasattr(torch.cuda, 'empty_cache'): 172 | torch.cuda.empty_cache() 173 | #----------------------# 174 | # 前向传播 175 | #----------------------# 176 | outputs = net(im) 177 | loss = criterion(outputs,label) 178 | test_loss += loss.data 179 | test_acc += get_acc(outputs,label) 180 | 181 | pbar2.set_postfix(**{'Test Acc': test_acc.item()/(step+1), 182 | 'Test Loss': test_loss.item() / (step + 1)}) 183 | pbar2.update(1) 184 | lr = optimizer.param_groups[0]['lr'] 185 | test_acc = test_acc.item() * 100 / len(testloader) 186 | # Save checkpoint. 187 | if test_acc > best_acc: 188 | print('Saving..') 189 | state = { 190 | 'net': net.state_dict(), 191 | 'acc': test_acc, 192 | 'epoch': epoch, 193 | 'lr': lr, 194 | } 195 | if not os.path.isdir('checkpoint'): 196 | os.mkdir('checkpoint') 197 | torch.save(state, './checkpoint/{}_ckpt.pth'.format(args.net)) 198 | best_acc = test_acc 199 | 200 | print('Finish Test') 201 | 202 | early_stopping(test_loss, net) 203 | # 若满足 early stopping 要求 204 | if early_stopping.early_stop: 205 | print("Early stopping") 206 | # 结束模型训练 207 | exit() 208 | 209 | for epoch in range(start_epoch, epochs): 210 | train(epoch) 211 | test(epoch) 212 | 213 | torch.cuda.empty_cache() 214 | -------------------------------------------------------------------------------- /CIFAR10/utils.py: -------------------------------------------------------------------------------- 1 | import os 2 | import torch 3 | import matplotlib.pyplot as plt 4 | import numpy as np 5 | import random 6 | from tqdm import tqdm # 在脚本中使用较好 7 | # from tqdm import tqdm_notebook as tqdm # 在jupyter中使用 8 | 9 | 10 | def setup_seed(seed): 11 | np.random.seed(seed) 12 | random.seed(seed) 13 | torch.manual_seed(seed) 14 | torch.cuda.manual_seed(seed) 15 | # When running on the CuDNN backend, two further options must be set 16 | torch.backends.cudnn.deterministic = True 17 | torch.backends.cudnn.benchmark = False 18 | # torch.backends.cudnn.benchmark = True 可以加速卷积的运算 19 | # Set a fixed value for the hash seed 20 | os.environ["PYTHONHASHSEED"] = str(seed) 21 | print(f"Random seed set as {seed}") 22 | 23 | 24 | def get_mean_and_std(dataset): 25 | '''Compute the mean and std value of dataset.''' 26 | dataloader = torch.utils.data.DataLoader(dataset, batch_size=1, shuffle=True, num_workers=2) 27 | mean = torch.zeros(3) 28 | std = torch.zeros(3) 29 | print('==> Computing mean and std..') 30 | for inputs, targets in dataloader: 31 | for i in range(3): 32 | mean[i] += inputs[:,i,:,:].mean() 33 | std[i] += inputs[:,i,:,:].std() 34 | mean.div_(len(dataset)) 35 | std.div_(len(dataset)) 36 | return mean, std 37 | 38 | 39 | def get_acc(outputs, label): 40 | total = outputs.shape[0] 41 | probs, pred_y = outputs.data.max(dim=1) # 得到概率 42 | correct = (pred_y == label).sum().data 43 | return torch.div(correct, total) 44 | 45 | 46 | class EarlyStopping: 47 | """Early stops the training if validation loss doesn't improve after a given patience.""" 48 | def __init__(self, patience=7, verbose=False, delta=0): 49 | """ 50 | Args: 51 | patience (int): How long to wait after last time validation loss improved. 52 | Default: 7 53 | verbose (bool): If True, prints a message for each validation loss improvement. 54 | Default: False 55 | delta (float): Minimum change in the monitored quantity to qualify as an improvement. 56 | Default: 0 57 | """ 58 | self.patience = patience 59 | self.verbose = verbose 60 | self.counter = 0 61 | self.best_score = None 62 | self.early_stop = False 63 | self.val_loss_min = np.Inf 64 | self.delta = delta 65 | 66 | def __call__(self, val_loss, model): 67 | score = -val_loss 68 | if self.best_score is None: 69 | self.best_score = score 70 | self.save_checkpoint(val_loss, model) 71 | elif score < self.best_score + self.delta: 72 | self.counter += 1 73 | print(f'EarlyStopping counter: {self.counter} out of {self.patience}') 74 | if self.counter >= self.patience: 75 | self.early_stop = True 76 | else: 77 | self.best_score = score 78 | self.save_checkpoint(val_loss, model) 79 | self.counter = 0 80 | 81 | def save_checkpoint(self, val_loss, model): 82 | '''Saves model when validation loss decrease.''' 83 | if self.verbose: 84 | print(f'Validation loss decreased ({self.val_loss_min:.6f} --> {val_loss:.6f}). Saving model ...') 85 | # torch.save(model.state_dict(), 'checkpoint.pt') # 这里会存储迄今最优模型的参数 86 | self.val_loss_min = val_loss 87 | 88 | def train(net, trainloader, testloader, epochs, optimizer , criterion, scheduler , path = './model.pth', writer = None ,verbose = False): 89 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 90 | best_acc = 0 91 | train_acc_list, test_acc_list = [],[] 92 | train_loss_list, test_loss_list = [],[] 93 | lr_list = [] 94 | for i in range(epochs): 95 | train_loss = 0 96 | train_acc = 0 97 | test_loss = 0 98 | test_acc = 0 99 | if torch.cuda.is_available(): 100 | net = net.to(device) 101 | net.train() 102 | train_step = len(trainloader) 103 | with tqdm(total=train_step,desc=f'Train Epoch {i + 1}/{epochs}',postfix=dict,mininterval=0.3) as pbar: 104 | for step,data in enumerate(trainloader,start=0): 105 | im,label = data 106 | im = im.to(device) 107 | label = label.to(device) 108 | 109 | optimizer.zero_grad() 110 | # 释放内存 111 | if hasattr(torch.cuda, 'empty_cache'): 112 | torch.cuda.empty_cache() 113 | # formard 114 | outputs = net(im) 115 | loss = criterion(outputs,label) 116 | # backward 117 | loss.backward() 118 | # 更新参数 119 | optimizer.step() 120 | # 累计损失 121 | train_loss += loss.item() 122 | train_acc += get_acc(outputs,label).item() 123 | pbar.set_postfix(**{'Train Acc' : train_acc/(step+1), 124 | 'Train Loss' :train_loss/(step+1)}) 125 | pbar.update(1) 126 | pbar.close() 127 | train_loss = train_loss / len(trainloader) 128 | train_acc = train_acc * 100 / len(trainloader) 129 | if verbose: 130 | train_acc_list.append(train_acc) 131 | train_loss_list.append(train_loss) 132 | # 记录学习率 133 | lr = optimizer.param_groups[0]['lr'] 134 | if verbose: 135 | lr_list.append(lr) 136 | # 更新学习率 137 | scheduler.step(train_loss) 138 | if testloader is not None: 139 | net.eval() 140 | test_step = len(testloader) 141 | with torch.no_grad(): 142 | with tqdm(total=test_step,desc=f'Test Epoch {i + 1}/{epochs}',postfix=dict,mininterval=0.3) as pbar: 143 | for step,data in enumerate(testloader,start=0): 144 | im,label = data 145 | im = im.to(device) 146 | label = label.to(device) 147 | # 释放内存 148 | if hasattr(torch.cuda, 'empty_cache'): 149 | torch.cuda.empty_cache() 150 | outputs = net(im) 151 | loss = criterion(outputs,label) 152 | test_loss += loss.item() 153 | test_acc += get_acc(outputs,label).item() 154 | pbar.set_postfix(**{'Test Acc' : test_acc/(step+1), 155 | 'Test Loss' :test_loss/(step+1)}) 156 | pbar.update(1) 157 | pbar.close() 158 | test_loss = test_loss / len(testloader) 159 | test_acc = test_acc * 100 / len(testloader) 160 | if verbose: 161 | test_loss_list.append(test_loss) 162 | test_acc_list.append(test_acc) 163 | print( 164 | 'Epoch [{:>3d}/{:>3d}] Train Loss:{:>.6f} Train Acc:{:>3.2f}% Test Loss:{:>.6f} Test Acc:{:>3.2f}% Learning Rate:{:>.6f}'.format( 165 | i + 1, epochs, train_loss, train_acc, test_loss, test_acc,lr)) 166 | else: 167 | print('Epoch [{:>3d}/{:>3d}] Train Loss:{:>.6f} Train Acc:{:>3.2f}% Learning Rate:{:>.6f}'.format(i+1,epochs,train_loss,train_acc,lr)) 168 | 169 | # ====================== 使用 tensorboard ================== 170 | if writer is not None: 171 | writer.add_scalars('Loss', {'train': train_loss, 172 | 'test': test_loss}, i+1) 173 | writer.add_scalars('Acc', {'train': train_acc , 174 | 'test': test_acc}, i+1) 175 | writer.add_scalar('Learning Rate',lr,i+1) 176 | # ========================================================= 177 | # 如果取得更好的准确率，就保存模型 178 | if test_acc > best_acc: 179 | torch.save(net,path) 180 | best_acc = test_acc 181 | Acc = {} 182 | Loss = {} 183 | Acc['train_acc'] = train_acc_list 184 | Acc['test_acc'] = test_acc_list 185 | Loss['train_loss'] = train_loss_list 186 | Loss['test_loss'] = test_loss_list 187 | Lr = lr_list 188 | return Acc, Loss, Lr 189 | 190 | def plot_history(epochs, Acc, Loss, lr): 191 | plt.rcParams['figure.figsize'] = (12.0, 8.0) # set default size of plots 192 | plt.rcParams['image.interpolation'] = 'nearest' 193 | plt.rcParams['image.cmap'] = 'gray' 194 | 195 | epoch_list = range(1,epochs + 1) 196 | plt.plot(epoch_list, Loss['train_loss']) 197 | plt.plot(epoch_list, Loss['test_loss']) 198 | plt.xlabel('epoch') 199 | plt.ylabel('Loss Value') 200 | plt.legend(['train', 'test'], loc='upper left') 201 | plt.show() 202 | 203 | plt.plot(epoch_list, Acc['train_acc']) 204 | plt.plot(epoch_list, Acc['test_acc']) 205 | plt.xlabel('epoch') 206 | plt.ylabel('Acc Value') 207 | plt.legend(['train', 'test'], loc='upper left') 208 | plt.show() 209 | 210 | plt.plot(epoch_list, lr) 211 | plt.xlabel('epoch') 212 | plt.ylabel('Train LR') 213 | plt.show() 214 | 215 | -------------------------------------------------------------------------------- /CIFAR10/utils2.py: -------------------------------------------------------------------------------- 1 | 2 | ''' 3 | 对训练函数进行更新 4 | 可视化更加方便，更加直观 5 | ''' 6 | import os 7 | import matplotlib.pyplot as plt 8 | from tqdm import tqdm 9 | import torch 10 | def get_acc(outputs, label): 11 | total = outputs.shape[0] 12 | probs, pred_y = outputs.data.max(dim=1) # 得到概率 13 | correct = (pred_y == label).sum().data 14 | return correct / total 15 | 16 | def plot_history(epochs, Acc = None, Loss=None, lr=None): 17 | plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots 18 | plt.style.use('seaborn') 19 | 20 | if Acc or Loss or lr: 21 | if not os.path.isdir('vis'): 22 | os.mkdir('vis') 23 | epoch_list = range(1,epochs + 1) 24 | 25 | if Loss: 26 | plt.plot(epoch_list, Loss['train_loss']) 27 | plt.plot(epoch_list, Loss['val_loss']) 28 | plt.xlabel('epoch') 29 | plt.ylabel('Loss Value') 30 | plt.legend(['train', 'val'], loc='upper left') 31 | plt.savefig('vis/history_Loss.png') 32 | plt.show() 33 | 34 | if Acc: 35 | plt.plot(epoch_list, Acc['train_acc']) 36 | plt.plot(epoch_list, Acc['val_acc']) 37 | plt.xlabel('epoch') 38 | plt.ylabel('Acc Value') 39 | plt.legend(['train', 'val'], loc='upper left') 40 | plt.savefig('vis/history_Acc.png') 41 | plt.show() 42 | 43 | if lr: 44 | plt.plot(epoch_list, lr) 45 | plt.xlabel('epoch') 46 | plt.ylabel('Train LR') 47 | plt.savefig('vis/history_Lr.png') 48 | plt.show() 49 | 50 | 51 | def train(epoch, epochs, model, dataloader, criterion, optimizer, scheduler = None): 52 | 53 | ''' 54 | Function used to train the model over a single epoch and update it according to the 55 | calculated gradients. 56 | 57 | Args: 58 | model: Model supplied to the function 59 | dataloader: DataLoader supplied to the function 60 | criterion: Criterion used to calculate loss 61 | optimizer: Optimizer used update the model 62 | scheduler: Scheduler used to update the learing rate for faster convergence 63 | (Commented out due to poor results) 64 | resnet_features: Model to get Resnet Features for the hybrid architecture (Default=None) 65 | 66 | Output: 67 | running_loss: Training Loss (Float) 68 | running_accuracy: Training Accuracy (Float) 69 | ''' 70 | running_loss = 0.0 71 | running_accuracy = 0.0 72 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 73 | 74 | model.train() 75 | train_step = len(dataloader) 76 | with tqdm(total=train_step,desc=f'Train Epoch {epoch + 1}/{epochs}',postfix=dict,mininterval=0.3) as pbar: 77 | for step,(data, target) in enumerate(dataloader): 78 | data = data.to(device) 79 | target = target.to(device) 80 | #--------------------- 81 | # 释放内存 82 | #--------------------- 83 | if hasattr(torch.cuda, 'empty_cache'): 84 | torch.cuda.empty_cache() 85 | optimizer.zero_grad() 86 | 87 | output = model(data) 88 | loss = criterion(output, target) 89 | 90 | 91 | acc = get_acc(output,target) 92 | running_accuracy += acc.item() 93 | running_loss += loss.item() 94 | loss.backward() 95 | optimizer.step() 96 | 97 | pbar.set_postfix(**{'Train Acc' : running_accuracy/(step+1), 98 | 'Train Loss' :running_loss/(step+1)}) 99 | pbar.update(1) 100 | if scheduler: 101 | scheduler.step(running_loss) 102 | running_loss, running_accuracy = running_loss/len(dataloader), running_accuracy/len(dataloader) 103 | return running_loss, running_accuracy 104 | 105 | 106 | def evaluation(epoch, epochs, model, dataloader, criterion): 107 | ''' 108 | Function used to evaluate the model on the test dataset. 109 | 110 | Args: 111 | model: Model supplied to the function 112 | dataloader: DataLoader supplied to the function 113 | criterion: Criterion used to calculate loss 114 | resnet_features: Model to get Resnet Features for the hybrid architecture (Default=None) 115 | 116 | Output: 117 | test_loss: Testing Loss (Float) 118 | test_accuracy: Testing Accuracy (Float) 119 | ''' 120 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 121 | eval_step = len(dataloader) 122 | with torch.no_grad(): 123 | test_accuracy = 0.0 124 | test_loss = 0.0 125 | with tqdm(total=eval_step,desc=f'Evaluation Epoch {epoch + 1}/{epochs}',postfix=dict,mininterval=0.3) as pbar: 126 | for step,(data, target) in enumerate(dataloader): 127 | data = data.to(device) 128 | target = target.to(device) 129 | #--------------------- 130 | # 释放内存 131 | #--------------------- 132 | if hasattr(torch.cuda, 'empty_cache'): 133 | torch.cuda.empty_cache() 134 | output = model(data) 135 | 136 | loss = criterion(output, target) 137 | acc = get_acc(output,target) 138 | 139 | test_accuracy += acc.item() 140 | test_loss += loss.item() 141 | 142 | pbar.set_postfix(**{'Eval Acc' : test_accuracy/(step+1), 143 | 'Eval Loss' :test_loss/(step+1)}) 144 | pbar.update(1) 145 | 146 | test_loss, test_accuracy = test_loss/eval_step, test_accuracy/eval_step 147 | return test_loss, test_accuracy 148 | 149 | def test(model, dataloader): 150 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 151 | correct = 0 # 定义预测正确的图片数，初始化为0 152 | total = 0 # 总共参与测试的图片数，也初始化为0 153 | model.eval() 154 | with torch.no_grad(): 155 | for data in dataloader: # 循环每一个batch 156 | images, labels = data 157 | images = images.to(device) 158 | labels = labels.to(device) 159 | model.eval() # 把模型转为test模式 160 | if hasattr(torch.cuda, 'empty_cache'): 161 | torch.cuda.empty_cache() 162 | outputs = model(images) # 输入网络进行测试 163 | 164 | # outputs.data是一个4x10张量，将每一行的最大的那一列的值和序号各自组成一个一维张量返回，第一个是值的张量，第二个是序号的张量。 165 | _, predicted = torch.max(outputs.data, 1) 166 | total += labels.size(0) # 更新测试图片的数量 167 | correct += (predicted == labels).sum() # 更新正确分类的图片的数量 168 | 169 | print('Accuracy of the network on the %d test images: %.2f %%' % (total, 100 * correct / total)) 170 | 171 | 172 | def test_precls(model, dataloader, classes): 173 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 174 | # 定义2个存储每类中测试正确的个数的 列表，初始化为0 175 | class_correct = list(0. for i in range(10)) 176 | class_total = list(0. for i in range(10)) 177 | # testloader = torch.utils.data.DataLoader(testset, batch_size=64,shuffle=True, num_workers=2) 178 | model.eval() 179 | with torch.no_grad(): 180 | for data in dataloader: 181 | images, labels = data 182 | images = images.to(device) 183 | labels = labels.to(device) 184 | if hasattr(torch.cuda, 'empty_cache'): 185 | torch.cuda.empty_cache() 186 | outputs = model(images) 187 | 188 | _, predicted = torch.max(outputs.data, 1) 189 | 190 | c = (predicted == labels).squeeze() 191 | for i in range(len(images)): # 因为每个batch都有4张图片，所以还需要一个4的小循环 192 | label = labels[i] # 对各个类的进行各自累加 193 | class_correct[label] += c[i] 194 | class_total[label] += 1 195 | 196 | 197 | for i in range(10): 198 | print('Accuracy of %5s : %.2f %%' % (classes[i], 100 * class_correct[i] / class_total[i])) -------------------------------------------------------------------------------- /ConvNeXt/dataloader.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torchvision 3 | import torchvision.transforms as transforms 4 | 5 | # Data 6 | def get_training_dataloader(batch_size = 64, num_workers = 4, shuffle = True, resize = 32): 7 | print('==> Preparing Train data..') 8 | transform_train = transforms.Compose([ 9 | transforms.Resize(resize), 10 | # transforms.RandomCrop(32, padding=4), 11 | # transforms.RandomHorizontalFlip(), 12 | transforms.ToTensor(), 13 | transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)), 14 | ]) 15 | trainset = torchvision.datasets.CIFAR10( 16 | root='./data', train=True, download=True, transform=transform_train) 17 | trainloader = torch.utils.data.DataLoader( 18 | trainset, batch_size=batch_size, shuffle=shuffle, num_workers= num_workers) 19 | return trainloader 20 | 21 | def get_test_dataloader(batch_size = 64, num_workers = 4, shuffle = True, resize = 32): 22 | print('==> Preparing Test data..') 23 | transform_test = transforms.Compose([ 24 | transforms.Resize(resize), 25 | transforms.ToTensor(), 26 | transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)), 27 | ]) 28 | 29 | testset = torchvision.datasets.CIFAR10( 30 | root='./data', train=False, download=True, transform=transform_test) 31 | testloader = torch.utils.data.DataLoader( 32 | testset, batch_size=batch_size, shuffle=shuffle, num_workers= num_workers) 33 | return testloader 34 | 35 | -------------------------------------------------------------------------------- /ConvNeXt/eval.py: -------------------------------------------------------------------------------- 1 | 2 | import os 3 | import torch 4 | import torch.nn as nn 5 | import argparse 6 | import numpy as np 7 | from tqdm import tqdm 8 | from dataloader import get_test_dataloader 9 | import torch.backends.cudnn as cudnn 10 | 11 | def eval_top1(outputs, label): 12 | total = outputs.shape[0] 13 | outputs = torch.softmax(outputs, dim=-1) 14 | _, pred_y = outputs.data.max(dim=1) # 得到概率 15 | correct = (pred_y == label).sum().data 16 | return correct / total 17 | 18 | def eval_top5(outputs, label): 19 | total = outputs.shape[0] 20 | outputs = torch.softmax(outputs, dim=-1) 21 | pred_y = np.argsort(-outputs.cpu().numpy()) 22 | pred_y_top5 = pred_y[:,:5] 23 | correct = 0 24 | for i in range(total): 25 | if label[i].cpu().numpy() in pred_y_top5[i]: 26 | correct += 1 27 | return correct / total 28 | 29 | 30 | if __name__ == '__main__': 31 | parser = argparse.ArgumentParser(description='PyTorch CIFAR10 Test') 32 | parser.add_argument('--cuda', action='store_true', default=False, help =' use GPU?') 33 | parser.add_argument('--batch-size', default=64, type=int, help = "Batch Size for Test") 34 | parser.add_argument('--num-workers', default=2, type=int, help = 'num-workers') 35 | parser.add_argument('--net', type = str, choices=['LeNet5', 'AlexNet', 'VGG16','VGG19','ResNet18','ResNet34', 36 | 'DenseNet','MobileNetv1','MobileNetv2','ResNeXt','ConvNeXt-T','ConvNeXt-S','ConvNeXt-B'], default='MobileNetv1', help='net type') 37 | parser.add_argument('--resize',type=int,default=32) 38 | args = parser.parse_args() 39 | testloader = get_test_dataloader(batch_size = args.batch_size, num_workers = args.num_workers, shuffle=False, resize = args.resize) 40 | 41 | # Model 42 | print('==> Building model..') 43 | if args.net == 'VGG16': 44 | from nets.VGG import VGG 45 | net = VGG('VGG16') 46 | elif args.net == 'VGG19': 47 | from nets.VGG import VGG 48 | net = VGG('VGG19') 49 | elif args.net == 'ResNet18': 50 | from nets.ResNet import ResNet18 51 | net = ResNet18() 52 | elif args.net == 'ResNet34': 53 | from nets.ResNet import ResNet34 54 | net = ResNet34() 55 | elif args.net == 'LeNet5': 56 | from nets.LeNet5 import LeNet5 57 | net = LeNet5() 58 | elif args.net == 'AlexNet': 59 | from nets.AlexNet import AlexNet 60 | net = AlexNet() 61 | elif args.net == 'DenseNet': 62 | from nets.DenseNet import densenet_cifar 63 | net = densenet_cifar() 64 | elif args.net == 'MobileNetv1': 65 | from nets.MobileNetv1 import MobileNet 66 | net = MobileNet() 67 | elif args.net == 'MobileNetv2': 68 | from nets.MobileNetv2 import MobileNetV2 69 | net = MobileNetV2() 70 | elif args.net == 'ResNeXt': 71 | from nets.ResNeXt import ResNeXt50 72 | net = ResNeXt50(10) 73 | elif args.net == 'ConvNeXt-T': 74 | from nets.ConvNeXt import convnext_tiny 75 | net = convnext_tiny(10) 76 | elif args.net == 'ConvNeXt-S': 77 | from nets.ConvNeXt import convnext_small 78 | net = convnext_small(10) 79 | elif args.net == 'ConvNeXt-B': 80 | from nets.ConvNeXt import convnext_base 81 | net = convnext_base(10) 82 | 83 | from torchinfo import summary 84 | summary(net,(2,3,224,224)) 85 | 86 | criterion = nn.CrossEntropyLoss() 87 | 88 | # Load checkpoint. 89 | print('==> Resuming from checkpoint..') 90 | assert os.path.isdir('checkpoint'), 'Error: no checkpoint directory found!' 91 | checkpoint = torch.load('./checkpoint/{}_ckpt.pth'.format(args.net)) 92 | 93 | weights_dict = {} 94 | for k, v in checkpoint['net'].items(): 95 | new_k = k.replace('module.', '') if 'module' in k else k 96 | weights_dict[new_k] = v 97 | 98 | net.load_state_dict(weights_dict) 99 | 100 | if args.cuda and torch.cuda.is_available(): 101 | device = 'cuda' 102 | net = torch.nn.DataParallel(net) 103 | cudnn.benchmark = True 104 | else: 105 | device = 'cpu' 106 | 107 | epoch_step_test = len(testloader) 108 | if epoch_step_test == 0: 109 | raise ValueError("数据集过小，无法进行训练，请扩充数据集，或者减小batchsize") 110 | 111 | net.eval() 112 | test_acc_top1 = 0 113 | test_acc_top5 = 0 114 | print('Start Test') 115 | #-------------------------------- 116 | # 相同方法，同train 117 | #-------------------------------- 118 | with tqdm(total=epoch_step_test,desc=f'Test Acc',postfix=dict,mininterval=0.3) as pbar2: 119 | for step,(im,label) in enumerate(testloader,start=0): 120 | im = im.to(device) 121 | label = label.to(device) 122 | with torch.no_grad(): 123 | if step >= epoch_step_test: 124 | break 125 | 126 | # 释放内存 127 | if hasattr(torch.cuda, 'empty_cache'): 128 | torch.cuda.empty_cache() 129 | #----------------------# 130 | # 前向传播 131 | #----------------------# 132 | outputs = net(im) 133 | loss = criterion(outputs,label) 134 | test_acc_top1 += eval_top1(outputs,label) 135 | test_acc_top5 += eval_top5(outputs,label) 136 | pbar2.set_postfix(**{'Test Acc Top1': test_acc_top1.item()/(step+1), 137 | 'Test Acc Top5': test_acc_top5 / (step + 1)}) 138 | pbar2.update(1) 139 | 140 | top1 = test_acc_top1.item()/ len(testloader) 141 | top5 = test_acc_top5 / len(testloader) 142 | print("top-1 accuracy = %.2f%%" % (top1*100)) 143 | print("top-5 accuracy = %.2f%%" % (top5*100)) -------------------------------------------------------------------------------- /ConvNeXt/大数据驱动的多种深度模型在图像分类中的对比（ConvNeXt篇）.doc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/ConvNeXt/大数据驱动的多种深度模型在图像分类中的对比（ConvNeXt篇）.doc -------------------------------------------------------------------------------- /ConvNeXt/大数据驱动的多种深度模型在图像分类中的对比（ConvNeXt篇）.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/ConvNeXt/大数据驱动的多种深度模型在图像分类中的对比（ConvNeXt篇）.pdf -------------------------------------------------------------------------------- /MAE/README.md: -------------------------------------------------------------------------------- 1 | ## 基于CIFAR10 MAE的实现 2 | 3 | 由于可用资源有限，我们仅在 cifar10 上测试模型。我们主要想重现这样的结果：**使用 MAE 预训练 ViT 可以比直接使用标签进行监督学习训练获得更好的结果**。这应该是**自我监督学习比监督学习更有效的数据**的证据。 4 | 5 | 我们主要遵循论文中的实现细节。但是，由于 Cifar10 和 ImageNet 的区别，我们做了一些修改： 6 | 7 | - 我们使用 vit-tiny 而不是 vit-base。 8 | - 由于 Cifar10 只有 50k 训练数据，我们将 pretraining epoch 从 400 增加到 2000，将 warmup epoch 从 40 增加到 200。我们注意到，在 2000 epoch 之后损失仍在减少。 9 | - 我们将训练分类器的批量大小从 1024 减少到 512 以减轻过度拟合。 10 | 11 | ### Install 12 | 13 | `pip install -r requirements.txt` 14 | 15 | ### Run 16 | 17 | 首先进行预训练 18 | 19 | ```python 20 | # pretrained with mae 21 | python mae_pretrain.py 22 | ``` 23 | 24 | 训练未用MAE的分类器，也就是从头开始训练分类器 25 | 26 | ``` 27 | # train classifier from scratch 28 | python train_classifier.py 29 | ``` 30 | 31 | 利用训练好的MAE的encoder作为输入，构建的分类模型作为分类器 32 | 33 | ```python 34 | # train classifier from pretrained model 35 | python train_classifier.py --pretrained_model_path vit-t-mae.pth --output_model_path vit-t-classifier-from_pretrained.pth 36 | ``` 37 | 38 | 集成了tensorboerd 39 | 40 | ``` 41 | tensorboard --logdir logs 42 | ``` 43 | 44 | 可以查看结果 45 | 46 | 47 | 48 | ### Result 49 | 50 | |Model|Validation Acc| 51 | |-----|--------------| 52 | |ViT-T w/o pretrain|74.13| 53 | |ViT-T w/ pretrain|**89.77**| 54 | 55 | 可视化CIFAR10前16张的图片，也可以在TensorBoard中查看 56 | 57 | 58 | 59 | ![avatar](pic/mae-cifar10-reconstruction.png) 60 | 61 | 、 62 | 63 | 由于TensorBoard的文件太大了，所以没有传上去，不过我上传了，大家可以自由查看https://tensorboard.dev/experiment/GIv9UzukQ5yFalbykPVMFQ/ -------------------------------------------------------------------------------- /MAE/logs/README.md: -------------------------------------------------------------------------------- 1 | 存放logs的tensorboard日志文件 2 | 3 | 已经上传到tensorboard上了，可以自己查看https://tensorboard.dev/experiment/GIv9UzukQ5yFalbykPVMFQ/ -------------------------------------------------------------------------------- /MAE/mae_pretrain.py: -------------------------------------------------------------------------------- 1 | import os 2 | import argparse 3 | import math 4 | import torch 5 | import torchvision 6 | from torch.utils.tensorboard import SummaryWriter 7 | from torchvision.transforms import ToTensor, Compose, Normalize 8 | from tqdm import tqdm 9 | 10 | from model import * 11 | from utils import setup_seed 12 | 13 | if __name__ == '__main__': 14 | parser = argparse.ArgumentParser() 15 | parser.add_argument('--seed', type=int, default=42) 16 | parser.add_argument('-bs','--batch_size', type=int, default=4096) 17 | parser.add_argument('--max_device_batch_size', type=int, default=128) 18 | parser.add_argument('--base_learning_rate', type=float, default=1.5e-4) 19 | parser.add_argument('--weight_decay', type=float, default=0.05) 20 | parser.add_argument('--mask_ratio', type=float, default=0.75) 21 | parser.add_argument('--total_epoch', type=int, default=2000) 22 | parser.add_argument('--warmup_epoch', type=int, default=200) 23 | parser.add_argument('--model_path', type=str, default='vit-t-mae.pth') 24 | 25 | args = parser.parse_args() 26 | 27 | setup_seed(args.seed) 28 | 29 | batch_size = args.batch_size 30 | load_batch_size = min(args.max_device_batch_size, batch_size) 31 | 32 | assert batch_size % load_batch_size == 0 33 | steps_per_update = batch_size // load_batch_size 34 | 35 | train_dataset = torchvision.datasets.CIFAR10('data', train=True, download=True, transform=Compose([ToTensor(), Normalize(0.5, 0.5)])) 36 | val_dataset = torchvision.datasets.CIFAR10('data', train=False, download=True, transform=Compose([ToTensor(), Normalize(0.5, 0.5)])) 37 | dataloader = torch.utils.data.DataLoader(train_dataset, load_batch_size, shuffle=True, num_workers=4) 38 | writer = SummaryWriter(os.path.join('logs', 'cifar10', 'mae-pretrain')) 39 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 40 | 41 | model = MAE_ViT(mask_ratio=args.mask_ratio).to(device) 42 | if device == 'cuda': 43 | net = torch.nn.DataParallel(model) 44 | 45 | optim = torch.optim.AdamW(model.parameters(), lr=args.base_learning_rate * args.batch_size / 256, betas=(0.9, 0.95), weight_decay=args.weight_decay) 46 | lr_func = lambda epoch: min((epoch + 1) / (args.warmup_epoch + 1e-8), 0.5 * (math.cos(epoch / args.total_epoch * math.pi) + 1)) 47 | lr_scheduler = torch.optim.lr_scheduler.LambdaLR(optim, lr_lambda=lr_func, verbose=True) 48 | 49 | step_count = 0 50 | optim.zero_grad() 51 | for e in range(args.total_epoch): 52 | model.train() 53 | losses = [] 54 | train_step = len(dataloader) 55 | with tqdm(total=train_step,desc=f'Epoch {e+1}/{args.total_epoch}',postfix=dict,mininterval=0.3) as pbar: 56 | for img, label in iter(dataloader): 57 | step_count += 1 58 | img = img.to(device) 59 | predicted_img, mask = model(img) 60 | loss = torch.mean((predicted_img - img) ** 2 * mask) / args.mask_ratio 61 | loss.backward() 62 | if step_count % steps_per_update == 0: 63 | optim.step() 64 | optim.zero_grad() 65 | losses.append(loss.item()) 66 | pbar.set_postfix(**{'Loss' : np.mean(losses)}) 67 | pbar.update(1) 68 | lr_scheduler.step() 69 | avg_loss = sum(losses) / len(losses) 70 | writer.add_scalar('mae_loss', avg_loss, global_step=e) 71 | # print(f'In epoch {e}, average traning loss is {avg_loss}.') 72 | 73 | ''' visualize the first 16 predicted images on val dataset''' 74 | model.eval() 75 | with torch.no_grad(): 76 | val_img = torch.stack([val_dataset[i][0] for i in range(16)]) 77 | val_img = val_img.to(device) 78 | predicted_val_img, mask = model(val_img) 79 | predicted_val_img = predicted_val_img * mask + val_img * (1 - mask) 80 | img = torch.cat([val_img * (1 - mask), predicted_val_img, val_img], dim=0) 81 | img = rearrange(img, '(v h1 w1) c h w -> c (h1 h) (w1 v w)', w1=2, v=3) 82 | writer.add_image('mae_image', (img + 1) / 2, global_step=e) 83 | 84 | ''' save model ''' 85 | torch.save(model, args.model_path) -------------------------------------------------------------------------------- /MAE/model.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import timm 3 | import numpy as np 4 | 5 | from einops import repeat, rearrange 6 | from einops.layers.torch import Rearrange 7 | 8 | 9 | # 这里可以用两个timm模型进行构建我们的结果 10 | from timm.models.layers import trunc_normal_ 11 | from timm.models.vision_transformer import Block 12 | 13 | def random_indexes(size : int): 14 | forward_indexes = np.arange(size) 15 | np.random.shuffle(forward_indexes) # 打乱index 16 | backward_indexes = np.argsort(forward_indexes) # 得到原来index的位置，方便进行还原 17 | return forward_indexes, backward_indexes 18 | 19 | def take_indexes(sequences, indexes): 20 | return torch.gather(sequences, 0, repeat(indexes, 't b -> t b c', c=sequences.shape[-1])) 21 | 22 | class PatchShuffle(torch.nn.Module): 23 | def __init__(self, ratio) -> None: 24 | super().__init__() 25 | self.ratio = ratio 26 | 27 | def forward(self, patches : torch.Tensor): 28 | T, B, C = patches.shape # length, batch, dim 29 | remain_T = int(T * (1 - self.ratio)) 30 | 31 | indexes = [random_indexes(T) for _ in range(B)] 32 | forward_indexes = torch.as_tensor(np.stack([i[0] for i in indexes], axis=-1), dtype=torch.long).to(patches.device) 33 | backward_indexes = torch.as_tensor(np.stack([i[1] for i in indexes], axis=-1), dtype=torch.long).to(patches.device) 34 | 35 | patches = take_indexes(patches, forward_indexes) # 随机打乱了数据的patch，这样所有的patch都被打乱了 36 | patches = patches[:remain_T] #得到未mask的pacth [T*0.25, B, C] 37 | 38 | return patches, forward_indexes, backward_indexes 39 | 40 | class MAE_Encoder(torch.nn.Module): 41 | def __init__(self, 42 | image_size=32, 43 | patch_size=2, 44 | emb_dim=192, 45 | num_layer=12, 46 | num_head=3, 47 | mask_ratio=0.75, 48 | ) -> None: 49 | super().__init__() 50 | 51 | self.cls_token = torch.nn.Parameter(torch.zeros(1, 1, emb_dim)) 52 | self.pos_embedding = torch.nn.Parameter(torch.zeros((image_size // patch_size) ** 2, 1, emb_dim)) 53 | 54 | # 对patch进行shuffle 和 mask 55 | self.shuffle = PatchShuffle(mask_ratio) 56 | 57 | # 这里得到一个 (3, dim, patch, patch) 58 | self.patchify = torch.nn.Conv2d(3, emb_dim, patch_size, patch_size) 59 | 60 | self.transformer = torch.nn.Sequential(*[Block(emb_dim, num_head) for _ in range(num_layer)]) 61 | 62 | # ViT的laynorm 63 | self.layer_norm = torch.nn.LayerNorm(emb_dim) 64 | 65 | self.init_weight() 66 | 67 | # 初始化类别编码和向量编码 68 | def init_weight(self): 69 | trunc_normal_(self.cls_token, std=.02) 70 | trunc_normal_(self.pos_embedding, std=.02) 71 | 72 | def forward(self, img): 73 | patches = self.patchify(img) 74 | patches = rearrange(patches, 'b c h w -> (h w) b c') 75 | patches = patches + self.pos_embedding 76 | 77 | patches, forward_indexes, backward_indexes = self.shuffle(patches) 78 | 79 | patches = torch.cat([self.cls_token.expand(-1, patches.shape[1], -1), patches], dim=0) 80 | patches = rearrange(patches, 't b c -> b t c') 81 | features = self.layer_norm(self.transformer(patches)) 82 | features = rearrange(features, 'b t c -> t b c') 83 | 84 | return features, backward_indexes 85 | 86 | class MAE_Decoder(torch.nn.Module): 87 | def __init__(self, 88 | image_size=32, 89 | patch_size=2, 90 | emb_dim=192, 91 | num_layer=4, 92 | num_head=3, 93 | ) -> None: 94 | super().__init__() 95 | 96 | self.mask_token = torch.nn.Parameter(torch.zeros(1, 1, emb_dim)) 97 | self.pos_embedding = torch.nn.Parameter(torch.zeros((image_size // patch_size) ** 2 + 1, 1, emb_dim)) 98 | 99 | self.transformer = torch.nn.Sequential(*[Block(emb_dim, num_head) for _ in range(num_layer)]) 100 | 101 | self.head = torch.nn.Linear(emb_dim, 3 * patch_size ** 2) 102 | self.patch2img = Rearrange('(h w) b (c p1 p2) -> b c (h p1) (w p2)', p1=patch_size, p2=patch_size, h=image_size//patch_size) 103 | 104 | self.init_weight() 105 | 106 | def init_weight(self): 107 | trunc_normal_(self.mask_token, std=.02) 108 | trunc_normal_(self.pos_embedding, std=.02) 109 | 110 | def forward(self, features, backward_indexes): 111 | T = features.shape[0] 112 | backward_indexes = torch.cat([torch.zeros(1, backward_indexes.shape[1]).to(backward_indexes), backward_indexes + 1], dim=0) 113 | features = torch.cat([features, self.mask_token.expand(backward_indexes.shape[0] - features.shape[0], features.shape[1], -1)], dim=0) 114 | features = take_indexes(features, backward_indexes) 115 | features = features + self.pos_embedding # 加上了位置编码的信息 116 | 117 | features = rearrange(features, 't b c -> b t c') 118 | features = self.transformer(features) 119 | features = rearrange(features, 'b t c -> t b c') 120 | features = features[1:] # remove global feature 去掉全局信息，得到图像信息 121 | 122 | patches = self.head(features) # 用head得到patchs 123 | mask = torch.zeros_like(patches) 124 | mask[T:] = 1 # mask其他的像素全部设为 1 125 | mask = take_indexes(mask, backward_indexes[1:] - 1) 126 | img = self.patch2img(patches) # 得到 重构之后的 img 127 | mask = self.patch2img(mask) 128 | 129 | return img, mask 130 | 131 | class MAE_ViT(torch.nn.Module): 132 | def __init__(self, 133 | image_size=32, 134 | patch_size=2, 135 | emb_dim=192, 136 | encoder_layer=12, 137 | encoder_head=3, 138 | decoder_layer=4, 139 | decoder_head=3, 140 | mask_ratio=0.75, 141 | ) -> None: 142 | super().__init__() 143 | 144 | self.encoder = MAE_Encoder(image_size, patch_size, emb_dim, encoder_layer, encoder_head, mask_ratio) 145 | self.decoder = MAE_Decoder(image_size, patch_size, emb_dim, decoder_layer, decoder_head) 146 | 147 | def forward(self, img): 148 | features, backward_indexes = self.encoder(img) 149 | predicted_img, mask = self.decoder(features, backward_indexes) 150 | return predicted_img, mask 151 | 152 | class ViT_Classifier(torch.nn.Module): 153 | def __init__(self, encoder : MAE_Encoder, num_classes=10) -> None: 154 | super().__init__() 155 | self.cls_token = encoder.cls_token 156 | self.pos_embedding = encoder.pos_embedding 157 | self.patchify = encoder.patchify 158 | self.transformer = encoder.transformer 159 | self.layer_norm = encoder.layer_norm 160 | self.head = torch.nn.Linear(self.pos_embedding.shape[-1], num_classes) 161 | 162 | def forward(self, img): 163 | patches = self.patchify(img) 164 | patches = rearrange(patches, 'b c h w -> (h w) b c') 165 | patches = patches + self.pos_embedding 166 | patches = torch.cat([self.cls_token.expand(-1, patches.shape[1], -1), patches], dim=0) 167 | patches = rearrange(patches, 't b c -> b t c') 168 | features = self.layer_norm(self.transformer(patches)) 169 | features = rearrange(features, 'b t c -> t b c') 170 | logits = self.head(features[0]) 171 | return logits 172 | 173 | 174 | if __name__ == '__main__': 175 | shuffle = PatchShuffle(0.75) 176 | a = torch.rand(16, 2, 10) 177 | b, forward_indexes, backward_indexes = shuffle(a) 178 | print(b.shape) 179 | 180 | img = torch.rand(2, 3, 32, 32) 181 | encoder = MAE_Encoder() 182 | decoder = MAE_Decoder() 183 | features, backward_indexes = encoder(img) 184 | print(forward_indexes.shape) 185 | predicted_img, mask = decoder(features, backward_indexes) 186 | print(predicted_img.shape) 187 | loss = torch.mean((predicted_img - img) ** 2 * mask / 0.75) -------------------------------------------------------------------------------- /MAE/pic/mae-cifar10-reconstruction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/MAE/pic/mae-cifar10-reconstruction.png -------------------------------------------------------------------------------- /MAE/requirements.txt: -------------------------------------------------------------------------------- 1 | torch 2 | torchvision 3 | tensorboard 4 | einops 5 | timm 6 | tqdm 7 | torchinfo 8 | numpy 9 | matplotlib 10 | -------------------------------------------------------------------------------- /MAE/train_classifier.py: -------------------------------------------------------------------------------- 1 | import os 2 | import argparse 3 | import math 4 | import torch 5 | import torchvision 6 | from torch.utils.tensorboard import SummaryWriter 7 | from torchvision.transforms import ToTensor, Compose, Normalize 8 | from tqdm import tqdm 9 | 10 | from model import * 11 | from utils import setup_seed 12 | 13 | if __name__ == '__main__': 14 | parser = argparse.ArgumentParser() 15 | parser.add_argument('--seed', type=int, default=2022) 16 | parser.add_argument('--batch_size', type=int, default=128) 17 | parser.add_argument('--max_device_batch_size', type=int, default=256) 18 | parser.add_argument('--base_learning_rate', type=float, default=1e-3) 19 | parser.add_argument('--weight_decay', type=float, default=0.05) 20 | parser.add_argument('--total_epoch', type=int, default=100) 21 | parser.add_argument('--warmup_epoch', type=int, default=5) 22 | parser.add_argument('--pretrained_model_path', type=str, default=None) 23 | parser.add_argument('--output_model_path', type=str, default='vit-t-classifier-from_scratch.pth') 24 | 25 | args = parser.parse_args() 26 | 27 | setup_seed(args.seed) 28 | 29 | batch_size = args.batch_size 30 | load_batch_size = min(args.max_device_batch_size, batch_size) 31 | 32 | assert batch_size % load_batch_size == 0 33 | steps_per_update = batch_size // load_batch_size 34 | 35 | train_dataset = torchvision.datasets.CIFAR10('data', train=True, download=True, transform=Compose([ToTensor(), Normalize(0.5, 0.5)])) 36 | val_dataset = torchvision.datasets.CIFAR10('data', train=False, download=True, transform=Compose([ToTensor(), Normalize(0.5, 0.5)])) 37 | train_dataloader = torch.utils.data.DataLoader(train_dataset, load_batch_size, shuffle=True, num_workers=4) 38 | val_dataloader = torch.utils.data.DataLoader(val_dataset, load_batch_size, shuffle=False, num_workers=4) 39 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 40 | 41 | if args.pretrained_model_path is not None: 42 | model = torch.load(args.pretrained_model_path, map_location='cpu') 43 | writer = SummaryWriter(os.path.join('logs', 'cifar10', 'pretrain-cls')) 44 | else: 45 | model = MAE_ViT() 46 | writer = SummaryWriter(os.path.join('logs', 'cifar10', 'scratch-cls')) 47 | model = ViT_Classifier(model.encoder, num_classes=10).to(device) 48 | 49 | if device == 'cuda': 50 | net = torch.nn.DataParallel(model) 51 | loss_fn = torch.nn.CrossEntropyLoss() 52 | acc_fn = lambda logit, label: torch.mean((logit.argmax(dim=-1) == label).float()) 53 | 54 | optim = torch.optim.AdamW(model.parameters(), lr=args.base_learning_rate * args.batch_size / 256, betas=(0.9, 0.999), weight_decay=args.weight_decay) 55 | lr_func = lambda epoch: min((epoch + 1) / (args.warmup_epoch + 1e-8), 0.5 * (math.cos(epoch / args.total_epoch * math.pi) + 1)) 56 | lr_scheduler = torch.optim.lr_scheduler.LambdaLR(optim, lr_lambda=lr_func, verbose=True) 57 | 58 | best_val_acc = 0 59 | step_count = 0 60 | optim.zero_grad() 61 | for e in range(args.total_epoch): 62 | model.train() 63 | losses = [] 64 | acces = [] 65 | train_step = len(train_dataloader) 66 | with tqdm(total=train_step,desc=f'Train Epoch {e+1}/{args.total_epoch}',postfix=dict,mininterval=0.3) as pbar: 67 | for img, label in iter(train_dataloader): 68 | step_count += 1 69 | img = img.to(device) 70 | label = label.to(device) 71 | logits = model(img) 72 | loss = loss_fn(logits, label) 73 | acc = acc_fn(logits, label) 74 | loss.backward() 75 | if step_count % steps_per_update == 0: 76 | optim.step() 77 | optim.zero_grad() 78 | losses.append(loss.item()) 79 | acces.append(acc.item()) 80 | 81 | pbar.set_postfix(**{'Train Loss' : np.mean(losses), 82 | 'Tran accs': np.mean(acces)}) 83 | pbar.update(1) 84 | 85 | lr_scheduler.step() 86 | avg_train_loss = sum(losses) / len(losses) 87 | avg_train_acc = sum(acces) / len(acces) 88 | # print(f'In epoch {e}, average training loss is {avg_train_loss}, average training acc is {avg_train_acc}.') 89 | 90 | model.eval() 91 | with torch.no_grad(): 92 | losses = [] 93 | acces = [] 94 | val_step = len(val_dataloader) 95 | with tqdm(total=val_step,desc=f'Val Epoch {e+1}/{args.total_epoch}',postfix=dict,mininterval=0.3) as pbar2: 96 | for img, label in iter(val_dataloader): 97 | img = img.to(device) 98 | label = label.to(device) 99 | logits = model(img) 100 | loss = loss_fn(logits, label) 101 | acc = acc_fn(logits, label) 102 | losses.append(loss.item()) 103 | acces.append(acc.item()) 104 | 105 | pbar2.set_postfix(**{'Val Loss' : np.mean(losses), 106 | 'Val accs': np.mean(acces)}) 107 | pbar2.update(1) 108 | avg_val_loss = sum(losses) / len(losses) 109 | avg_val_acc = sum(acces) / len(acces) 110 | # print(f'In epoch {e}, average validation loss is {avg_val_loss}, average validation acc is {avg_val_acc}.') 111 | 112 | if avg_val_acc > best_val_acc: 113 | best_val_acc = avg_val_acc 114 | print(f'saving best model with acc {best_val_acc} at {e} epoch!') 115 | torch.save(model, args.output_model_path) 116 | 117 | writer.add_scalars('cls/loss', {'train' : avg_train_loss, 'val' : avg_val_loss}, global_step=e) 118 | writer.add_scalars('cls/acc', {'train' : avg_train_acc, 'val' : avg_val_acc}, global_step=e) -------------------------------------------------------------------------------- /MAE/utils.py: -------------------------------------------------------------------------------- 1 | import random 2 | import torch 3 | import numpy as np 4 | 5 | def setup_seed(seed=2022): 6 | torch.manual_seed(seed) 7 | torch.cuda.manual_seed_all(seed) 8 | np.random.seed(seed) 9 | random.seed(seed) 10 | torch.backends.cudnn.deterministic = True -------------------------------------------------------------------------------- /MAE/vit-t-classifier-from_pretrained.pth: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/MAE/vit-t-classifier-from_pretrained.pth -------------------------------------------------------------------------------- /MAE/vit-t-classifier-from_scratch.pth: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/MAE/vit-t-classifier-from_scratch.pth -------------------------------------------------------------------------------- /MAE/vit-t-mae.pth: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/MAE/vit-t-mae.pth -------------------------------------------------------------------------------- /Parper-link/README.md: -------------------------------------------------------------------------------- 1 | # ClassicNetwork 图像分类网络论文链接汇总 2 | 3 | Classical network implemented by pytorch 4 | 5 | **LeNet:** 6 | 7 | - **LeNet:** LeNet-5, convolutional neural networks 8 | 9 | [http://yann.lecun.com/exdb/lenet/index.html](http://yann.lecun.com/exdb/lenet/index.html) 10 | 11 | **AlexNet:** 12 | 13 | - ImageNet Classification with Deep Convolutional Neural Networks, Alex Krizhevsky, 2012 14 | 15 | [http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf](http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf) 16 | 17 | **VGG:** 18 | 19 | - Very Deep Convolutional Networks for Large-Scale Image Recognition,Karen Simonyan,2014 20 | 21 | [https://arxiv.org/abs/1409.1556](https://arxiv.org/abs/1409.1556) 22 | 23 | **ResNet:** 24 | 25 | - Deep Residual Learning for Image Recognition, He-Kaiming, 2016 26 | 27 | [https://arxiv.org/abs/1512.03385](https://arxiv.org/abs/1512.03385) 28 | 29 | **Batch Normalization** 30 | 31 | - Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,2015 32 | 33 | [https://arxiv.org/abs/1502.03167](https://arxiv.org/abs/1502.03167) 34 | 35 | **ZFNet** 36 | 37 | - Visualizing and Understanding Convolutional Networks,2013 38 | 39 | [https://arxiv.org/abs/1311.2901](https://arxiv.org/abs/1311.2901) 40 | 41 | **Inception系列** 42 | 43 | - **InceptionV1:** Going deeper with convolutions , Christian Szegedy , 2014 44 | 45 | [https://arxiv.org/abs/1409.4842](https://arxiv.org/abs/1409.4842) 46 | 47 | - **InceptionV2 and InceptionV3:** Rethinking the Inception Architecture for Computer Vision , Christian Szegedy ,2015 48 | 49 | [https://arxiv.org/abs/1512.00567](https://arxiv.org/abs/1512.00567) 50 | 51 | - **InceptionV4 and Inception-ResNet:** Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , Christian Szegedy ,2016 52 | 53 | [https://arxiv.org/abs/1602.07261](https://arxiv.org/abs/1602.07261) 54 | 55 | **DenseNet:** 56 | 57 | - Densely Connected Convolutional Networks, 2017 58 | 59 | [https://arxiv.org/abs/1608.06993](https://arxiv.org/abs/1608.06993) 60 | 61 | **ResNeXt:** 62 | 63 | - Aggregated Residual Transformations for Deep Neural Networks,2017 64 | 65 | [https://arxiv.org/abs/1611.05431](https://arxiv.org/abs/1611.05431) 66 | 67 | **NASNet:** 68 | 69 | - Learning Transferable Architectures for Scalable Image Recognition 70 | 71 | [https://arxiv.org/abs/1707.07012](https://arxiv.org/abs/1707.07012) 72 | 73 | **SENet** 74 | 75 | - Squeeze-and-Excitation Networks 76 | 77 | [https://arxiv.org/abs/1709.01507](https://arxiv.org/abs/1709.01507) 78 | 79 | **MobileNet:** 80 | 81 | - **MobileNet(v1)** : MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications 82 | 83 | [https://arxiv.org/abs/1704.04861](https://arxiv.org/abs/1704.04861) 84 | 85 | - **MobileNet(v2)** : MobileNetV2: Inverted Residuals and Linear Bottlenecks 86 | 87 | [https://arxiv.org/abs/1801.04381](https://arxiv.org/abs/1801.04381) 88 | 89 | - **MobileNet(v3)** : Searching for MobileNetV3 90 | 91 | [https://arxiv.org/abs/1905.02244](https://arxiv.org/abs/1905.02244) 92 | 93 | 94 | 95 | **ShuffleNet:** 96 | 97 | - **ShuffleNet(v1)** :ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices 98 | 99 | [https://arxiv.org/abs/1707.01083](https://arxiv.org/abs/1707.01083) 100 | 101 | - **ShuffleNet(v2)** :ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design 102 | 103 | [https://arxiv.org/abs/1807.11164](https://arxiv.org/abs/1807.11164) 104 | 105 | **EfficientNet:** 106 | 107 | - EfficientNet(v1) :EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks 108 | 109 | [https://arxiv.org/abs/1905.11946](https://arxiv.org/abs/1905.11946) 110 | 111 | - EfficientNet(v2) :EfficientNetV2: Smaller Models and Faster Training 112 | 113 | [https://arxiv.org/abs/2104.00298](https://arxiv.org/abs/2104.00298) -------------------------------------------------------------------------------- /QR.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/QR.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ✨Pytorch&Keras CIFAR10图像分类 2 | 3 | 4 | 5 | - [✨Pytorch&Keras CIFAR10图像分类](#pytorchkeras-cifar10图像分类) 6 | - [🧑‍🎓适用人群](#‍适用人群) 7 | - [📚︎博客汇总](#︎博客汇总) 8 | - [💻︎ Pytorch CIFAR10 图像分类](#︎-pytorch-cifar10-图像分类) 9 | - [⁉ 数据处理以及工具函数（网络篇前必看）](#⁉-数据处理以及工具函数网络篇前必看) 10 | - [❕ 网络篇](#-网络篇) 11 | - [🖥︎ Keras CIFAR10 图像分类](#🖥︎-keras-cifar10-图像分类) 12 | - [💝 有趣的项目和尝试](#-有趣的项目和尝试) 13 | - [📅 Comming soon 更新计划](#-comming-soon-更新计划) 14 | - [🧰 使用方法](#🧰-使用方法) 15 | - [📚 参考](#-参考) 16 | 17 | 18 | ![Keras vs PyTorch vs Caffe - Comparing the Implementation of CNN](https://149695847.v2.pressablecdn.com/wp-content/uploads/2020/08/create-machine-learning-and-deep-learning-models-using-pytorch-and-tensorflow.jpg#pic_center) 19 | 20 | ## 💪专栏介绍 21 | 22 | 一开始写这个专栏的初衷是，**有时候有些代码找的太繁琐了，并且找到了还运行不出来，或者运行了，但是没有一个很好的理解**，所以基于此，我写了这个CIFAR10图像分类的专栏，借此希望，大家都能通过我这个专栏，找到自己想要的模型或者学习深度学习的代码。 23 | 24 | 由于现在深度学习也在热度中，很多时候我们难免需要遇见深度学习的时候，**在一些课程和项目的要求下，我们会发现，一个好的代码和一个好的可视化和清晰解释是能够节省特别特别多的时间的**，基于此，我写出了这个专栏，这个专栏下的所有项目，都是**可运行无差错的。**如果遇见了问题，也可以留言跟我沟通 25 | 26 | --- 27 | 28 | ## 🧑‍🎓适用人群 29 | 30 | 他很适合大家**初入深度学习或者是Pytorch和Keras**，希望这能够帮助初学深度学习的同学一个入门Pytorch或者Keras的项目和在这之中更加了解Pytorch&Keras和各个图像分类的模型。 31 | 32 | 他有比较清晰的可视化结构和架构，除此之外，我是**用jupyter写的，所以说在文章整体架构可以说是非常清晰**，可以帮助你快速学习到**各个模块的知识**，而不是通过python脚本一行一行的看，这样的方式是符合初学者的。 33 | 34 | 除此之外，如果你需要变成脚本形式，也是很简单的。 35 | 36 | --- 37 | 38 | ## 📚︎博客汇总 39 | 40 | 为了使得文章体系结构更加清晰，这里给出**Pytorch&Keras对于CIFAR10图像分类的所有资料汇总**，也就是我的博客汇总，也告诉大家，我做了什么工作，如果大家有兴趣订阅我的专栏亦或者是有什么其他模型的想法，也可以评论留言，我也可以进行去学习的。 41 | 42 | 这部分也方便大家看到介绍，并且快速找到自己所需要的代码进行学习和了解 43 | 44 | --- 45 | 46 | ### 💻︎ Pytorch CIFAR10 图像分类 47 | 48 | 在看网络篇的时候，可以先看前三个，前三个说明的如何加载数据包括数据的预处理以及进行可视化，工具函数篇介绍了如何构建训练的函数，并且有时候会遇到一部分utils.py的错误，在这里面都有很好的解释和进行学习。 49 | 50 | #### ⁉ 数据处理以及工具函数（网络篇前必看） 51 | 52 | - [Pytorch CIFAR10图像分类 数据加载与可视化篇](https://blog.csdn.net/weixin_45508265/article/details/119285113) 53 | - [Pytorch CIFAR10图像分类 工具函数utils篇](https://redamancy.blog.csdn.net/article/details/121589217) 54 | - [Pytorch CIFAR10图像分类 工具函数utils更新v2.0篇](https://redamancy.blog.csdn.net/article/details/127856569) 55 | 56 | #### ❕ 网络篇 57 | 58 | - [Pytorch CIFAR10图像分类 自定义网络篇](https://blog.csdn.net/weixin_45508265/article/details/119305277) 59 | - [Pytorch CIFAR10图像分类 LeNet5篇](https://blog.csdn.net/weixin_45508265/article/details/119305673) 60 | - [Pytorch CIFAR10图像分类 AlexNet篇](https://blog.csdn.net/weixin_45508265/article/details/119305848) 61 | - [Pytorch CIFAR10图像分类 VGG篇](https://blog.csdn.net/weixin_45508265/article/details/119332904) 62 | - [Pytorch CIFAR10图像分类 GoogLeNet篇](https://blog.csdn.net/weixin_45508265/article/details/119399239) 63 | - [Pytorch CIFAR10图像分类 ResNet篇](https://blog.csdn.net/weixin_45508265/article/details/119532143) 64 | - [Pytorch CIFAR10图像分类 DenseNet篇](https://blog.csdn.net/weixin_45508265/article/details/119648036) 65 | - [Pytorch CIFAR10图像分类 MobieNetv1篇](https://redamancy.blog.csdn.net/article/details/124636103) 66 | - [Pytorch CIFAR10图像分类 MobileNetv2篇](https://redamancy.blog.csdn.net/article/details/127946431) 67 | - [Pytorch CIFAR10图像分类 ResNeXt篇](https://redamancy.blog.csdn.net/article/details/126655797) 68 | - [Pytorch CIFAR10图像分类 ZFNet篇](https://blog.csdn.net/weixin_45508265/article/details/128560595) 69 | - [Pytorch CIFAR10图像分类 SENet篇](https://blog.csdn.net/weixin_45508265/article/details/130938341) 70 | - [Pytorch CIFAR10 图像分类 Vision Transformer篇](https://redamancy.blog.csdn.net/article/details/126751948) 71 | - [Pytorch CIFAR10图像分类 EfficientNet篇](https://blog.csdn.net/weixin_45508265/article/details/128585354) 72 | - [Pytorch CIFAR10图像分类 ShuffleNet篇](https://blog.csdn.net/weixin_45508265/article/details/130945031) 73 | - [Pytorch CIFAR10图像分类 ShuffleNetv2篇](https://blog.csdn.net/weixin_45508265/article/details/130938341) 74 | - [Pytorch CIFAR10图像分类 Swin Tansformer篇](https://blog.csdn.net/weixin_45508265/article/details/134773753) 75 | 76 | > 具体的详情可以关注[Pytorch CIFAR10图像分类汇总篇](https://redamancy.blog.csdn.net/article/details/119285255) 77 | 78 | --- 79 | 80 | ### 🖥︎ Keras CIFAR10 图像分类 81 | 82 | - [Keras CIFAR-10分类 SVM 分类器篇][1] 83 | - [Keras CIFAR-10分类 LeNet-5篇][2] 84 | - [Keras CIFAR-10分类 AlexNet篇][3] 85 | - [Keras CIFAR-10分类 GoogleNet篇][4] 86 | - [Keras CIFAR-10分类 VGG篇][5] 87 | - [Keras CIFAR-10分类 ResNet篇][6] 88 | - [Keras CIFAR-10分类 DenseNet篇][7] 89 | 90 | > 具体的详情可以关注[Keras CIFAR-10 分类汇总篇](https://blog.csdn.net/weixin_45508265/article/details/127859003) 91 | 92 | --- 93 | 94 | ### 💝 有趣的项目和尝试 95 | 96 | - [MAE实现及预训练可视化 （CIFAR-Pytorch）][MAE] 97 | 98 | --- 99 | 100 | ## 📅 Comming soon 更新计划 101 | 102 | - [x] LetNet 103 | - [x] AlexNet 104 | - [x] VGG 105 | - [x] ResNet 106 | - [x] GoogLeNet 107 | - [x] DenseNet 108 | - [x] ResNeXt 109 | - [x] MobileNetv1 110 | - [x] MobileNetv2 111 | - [x] ZFNet 112 | - [x] SeNet 113 | - [x] Efficientent 114 | - [x] ViT 115 | - [x] ShuffleNetv1 116 | - [x] ShuffleNetv2 117 | - [x] Swin-Transformer 118 | - [ ] ConvNeXt 119 | - [ ] ConvNeXtv2 120 | 121 | --- 122 | 123 | ## 🧰 使用方法 124 | 125 | 下载`CIFAR10`里所有文件，直接运行ipynb即可，由于我是利用一个工具函数进行训练的，所以**切记utils.py是必不可少的。** 126 | 127 | 运行ipynb文件即可，对于网络的py文件会持续更新，之后会利用一个函数来选取对应的网络进行训练得到结果。 128 | 129 | --- 130 | 131 | ## 📚 参考 132 | 133 | 除此之外，我还为图像分类这个专栏录了一下我的视频讲解，感兴趣的小伙伴可以来我的B站看视频进行学习，啃代码的时候，可能听一下也会有更多的感触哦 134 | [https://space.bilibili.com/241286257/channel/seriesdetail?sid=2075039](https://space.bilibili.com/241286257/channel/seriesdetail?sid=2075039) 135 | 136 | --- 137 | 138 | 最后这个是我写的一个pytorch的基础的介绍，[Pytorch Note 快乐星球](https://blog.csdn.net/weixin_45508265/article/details/117809512)，从0开始的完整的介绍pytorch和pytorch的简单语法，并且里面有一些项目和学习，还是很不错的哦，可以查看，除此之外，有什么想法可以加我wx: `pikachu2biubiu`聊哦，需要什么帮助也可以付费聊咨询。 139 | 140 | ![二维码](QR.png) 141 | 142 | 143 | 144 | ## Star History 145 | 146 | [![Star History Chart](https://api.star-history.com/svg?repos=Kedreamix/Pytorch-Image-Classification&type=Date)](https://star-history.com/#Kedreamix/Pytorch-Image-Classification&Date) 147 | 148 | 149 | 150 | [1]: https://redamancy.blog.csdn.net/article/details/126445778 151 | [2]: https://redamancy.blog.csdn.net/article/details/126446810 152 | [3]: https://redamancy.blog.csdn.net/article/details/126590621 153 | [4]: https://redamancy.blog.csdn.net/article/details/126591761 154 | [5]: https://redamancy.blog.csdn.net/article/details/126669709 155 | [6]: https://redamancy.blog.csdn.net/article/details/127827641 156 | [7]: https://redamancy.blog.csdn.net/article/details/127828318 157 | [MAE]: https://redamancy.blog.csdn.net/article/details/126863995 158 | 159 | -------------------------------------------------------------------------------- /TransferLearning/README.md: -------------------------------------------------------------------------------- 1 | # Train your data in Pytorch 2 | 3 | ## Taining 4 | 5 | 实际上使用方法是很简单的，如果下载了数据集之后，就可以直接python train.py 6 | 7 | ```python 8 | start training with: 9 | python train.py 10 | 11 | You can manually resume the training with: 12 | python train.py --data './cats_and_dogs_filtered/' 13 | --model 'ResNet' # choices ['AlexNet','VGG','ResNet','MobileNet','ShuffleNet','DenseNet','MnasNet'] 14 | --classes 2 # set classes 15 | --epochs 10 # set epochs 16 | --lr 0.02 # set learning rate 17 | --optimizer 'Adam' # 优化器 如Adam,AdamW,SGD 18 | --batch-size 64 # batch size 19 | --verbose # 可视化 20 | --logs # 日志，每次都保存模型 21 | ``` 22 | 23 | ## 使用自己的数据集 24 | 25 | 这里要注意，对于自己的数据集，需要按照一定的排列格式，可以按照你的数据集排列方式，最后在运行的时候改成python train.py --data './data'即可，除此之外，多个类别需要设置classes参数，也就是python train.py --data './data' --classes 类别 26 | 27 | ```txt 28 | data 29 | ----train 30 | ----dogs 31 | ----cats 32 | ----... 33 | ----validation 34 | ----dogs 35 | ----cats 36 | ----... 37 | ----test(放任意图片，可以在命令行设置，没有默认0) 38 | ``` 39 | 40 | -------------------------------------------------------------------------------- /TransferLearning/Transfer_models.py: -------------------------------------------------------------------------------- 1 | import torchvision.models as models 2 | import torch.nn as nn 3 | import torch 4 | 5 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 6 | 7 | def AlexNet(classes=1000): 8 | # 导入Pytorch封装的AlexNet网络模型 9 | alexnet = models.alexnet(pretrained=True) 10 | # 固定卷积层参数 11 | for param in alexnet.parameters(): 12 | param.requires_grad = False 13 | 14 | # 获取最后一个全连接层的输入通道数 15 | num_input = alexnet.classifier[6].in_features 16 | 17 | # 获取全连接层的网络结构 18 | feature_model = list(alexnet.classifier.children()) 19 | 20 | # 去掉原来的最后一层 21 | feature_model.pop() 22 | 23 | # 添加上适用于自己数据集的全连接层 24 | feature_model.append(nn.Linear(num_input, classes)) 25 | # 仿照这里的方法，可以修改网络的结构，不仅可以修改最后一个全连接层 26 | # 还可以为网络添加新的层 27 | # 重新生成网络的后半部分 28 | alexnet.classifier = nn.Sequential(*feature_model) 29 | for param in alexnet.classifier.parameters(): 30 | param.requires_grad = True 31 | alexnet = alexnet.to(device) 32 | # 打印一下 33 | # print(alexnet) 34 | return alexnet 35 | 36 | def VGG16(classes=1000): 37 | vgg16 = models.vgg16_bn(pretrained=True) 38 | # 固定模型权重 39 | for param in vgg16.parameters(): 40 | param.requires_grad = False 41 | 42 | # 最后加一个分类器 43 | vgg16.classifier[6] = nn.Sequential(nn.Linear(4096, classes)) 44 | for param in vgg16.classifier[6].parameters(): 45 | param.requires_grad = True 46 | 47 | vgg16 = vgg16.to(device) 48 | return vgg16 49 | 50 | 51 | def ResNet18(classes=1000): 52 | resnet18 = models.resnet18(pretrained=True) 53 | 54 | for param in resnet18.parameters(): 55 | param.requires_grad = False 56 | 57 | inchannel = resnet18.fc.in_features 58 | resnet18.fc = nn.Linear(inchannel, classes) 59 | for param in resnet18.fc.parameters(): 60 | param.requires_grad = True 61 | 62 | resnet18 = resnet18.to(device) 63 | return resnet18 64 | 65 | 66 | def DenseNet121(classes=1000): 67 | densenet121 = models.densenet121(pretrained=True) 68 | 69 | for param in densenet121.parameters(): 70 | param.requires_grad = False 71 | 72 | inchannel = densenet121.classifier.in_features 73 | densenet121.classifier = nn.Linear(inchannel, classes) 74 | for param in densenet121.classifier.parameters(): 75 | param.requires_grad = True 76 | 77 | densenet121 = densenet121.to(device) 78 | return densenet121 79 | 80 | 81 | def MobileNetv2(classes=1000): 82 | mobilenet = models.mobilenet_v2(pretrained=True) 83 | 84 | for param in mobilenet.parameters(): 85 | param.requires_grad = False 86 | 87 | # 最后加一个分类器 88 | mobilenet.classifier[1] = nn.Sequential(nn.Linear(1280, classes)) 89 | for param in mobilenet.classifier[1].parameters(): 90 | param.requires_grad = True 91 | 92 | mobilenet = mobilenet.to(device) 93 | return mobilenet 94 | 95 | def ShuffleNetv2(classes=1000): 96 | shufflenet = models.shufflenet_v2_x1_0(pretrained=True) 97 | 98 | for param in shufflenet.parameters(): 99 | param.requires_grad = False 100 | 101 | # 最后加一个分类器 102 | inchannel = shufflenet.fc.in_features 103 | shufflenet.fc = nn.Linear(inchannel, classes) 104 | for param in shufflenet.fc.parameters(): 105 | param.requires_grad = True 106 | 107 | shufflenet = shufflenet.to(device) 108 | return shufflenet 109 | 110 | def MnasNet(classes=1000): 111 | mnasnet = models.mnasnet1_0(pretrained=True) 112 | for param in mnasnet.parameters(): 113 | param.requires_grad = False 114 | 115 | # 最后加一个分类器 116 | mnasnet.classifier[1] = nn.Sequential(nn.Dropout(p=0.2),nn.Linear(1280, classes)) 117 | for param in mnasnet.classifier[1].parameters(): 118 | param.requires_grad = True 119 | 120 | mnasnet = mnasnet.to(device) 121 | return mnasnet 122 | -------------------------------------------------------------------------------- /TransferLearning/train.py: -------------------------------------------------------------------------------- 1 | import torch.nn as nn 2 | import torchvision 3 | import os 4 | from utils import * 5 | from Transfer_models import * 6 | import numpy as np 7 | import argparse 8 | 9 | def main(opt): 10 | transform = transforms.Compose([ 11 | transforms.Resize(256),# Resize成256x256 12 | transforms.CenterCrop(224),# 随机裁剪为224x224 13 | transforms.ToTensor(),# 转化为向量 14 | transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])]) # 进行标准化 15 | 16 | root = opt.data 17 | # root = '../cats_and_dogs_filtered/' # 把文件夹放在当前目录下 18 | trainset = torchvision.datasets.ImageFolder(root + 'train',transform=transform) 19 | valset = torchvision.datasets.ImageFolder(root + 'validation',transform=transform) 20 | 21 | trainloader = torch.utils.data.DataLoader(trainset, batch_size=opt.batch_size,shuffle=True,num_workers=opt.workers) 22 | valloader = torch.utils.data.DataLoader(valset, batch_size=opt.batch_size,shuffle=False,num_workers=opt.workers) 23 | 24 | print('{} DATA has loaded'.format(root)) 25 | train_size = len(trainset) 26 | val_size = len(valset) 27 | test_size = 1000 # 测试集文件夹下的图片数量 28 | print(u"训练集个数:", len(trainset)) 29 | print(u"验证集个数:", len(valset)) 30 | 31 | if opt.save_path != '': 32 | # 模型文件 33 | if not os.path.exists('./model/'): 34 | os.mkdir('./model') 35 | if opt.logs: 36 | # 日志文件，以防中断以后继续加载模型训练 37 | if not os.path.exists('./logs/'): 38 | os.mkdir('./logs') 39 | if opt.verbose: 40 | if not os.path.exists('./vis/'): 41 | os.mkdir('./vis') 42 | CLASSES = opt.classes # 类别 43 | if opt.model == "AlexNet": 44 | model = AlexNet(CLASSES) 45 | elif opt.model == "VGG":model = VGG16(CLASSES) 46 | elif opt.model == "ResNet":model = ResNet18(CLASSES) 47 | elif opt.model == "DenseNet":model = DenseNet121(CLASSES) 48 | elif opt.model == "MobileNet":model = MobileNetv2(CLASSES) 49 | elif opt.model == "ShuffleNet":model = ShuffleNetv2(CLASSES) 50 | elif opt.model == "MnasNet":model = MnasNet(CLASSES) 51 | 52 | print("You Choose {} model".format(opt.model)) 53 | 54 | #---------------------------------------# 55 | # 定义优化器，损失函数以及学习策略 56 | #---------------------------------------# 57 | if opt.optimizer == "SGD": 58 | optimizer = torch.optim.SGD(model.parameters(), lr=opt.lr, momentum=0.9,nesterov=True) 59 | elif opt.optimizer == "Adam": 60 | optimizer = torch.optim.Adam(model.parameters(),lr=opt.lr) 61 | elif opt.optimizer == "AdamW": 62 | optimizer = torch.optim.AdamW(model.parameters(),lr=opt.lr) 63 | print("We use {} optimizer".format(opt.optimizer)) 64 | criterion = nn.CrossEntropyLoss() 65 | scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer,patience=3,factor=0.5,min_lr=1e-6) 66 | 67 | #-------------------------------------------------# 68 | # 进行训练，默认10个epochs，保存路径是model.pth, 69 | # 不进行tensorboard可视化，记录损失， 70 | # 存在日志记录，没有预训练模型 71 | #-------------------------------------------------# 72 | 73 | if opt.verbose: 74 | Acc, Loss, Lr = train(model,trainloader, valloader, optimizer , criterion, scheduler , epochs = opt.epochs , path = opt.save_path , writer = False, verbose = opt.verbose, logs = opt.logs, pretrain = opt.weights) 75 | plot_history(opt.epochs, Acc, Loss, Lr) 76 | n = np.random.randint(0,test_size) 77 | test(root + 'test/%d.jpg'% n ,model) 78 | else: 79 | train(model,trainloader, valloader, optimizer , criterion, scheduler , epochs = opt.epochs , path = opt.save_path , verbose = opt.verbose, logs = opt.logs, pretrain = opt.weights) 80 | 81 | if __name__ == '__main__': 82 | # python train.py 83 | parser = argparse.ArgumentParser() 84 | parser.add_argument('--epochs', type=int, default=10,help='epochs for training') 85 | parser.add_argument('--classes', type=int, default=2,help='the number of classes') 86 | parser.add_argument('--model', type=str,choices=['AlexNet','VGG','ResNet','MobileNet','ShuffleNet','DenseNet','MnasNet'], default='MnasNet', help='select model to train') 87 | parser.add_argument('--data', type=str, default='./cats_and_dogs_filtered/', help='data 文件路径') 88 | parser.add_argument('--batch-size', type=int, default=64, help='total batch size for all GPUs') 89 | parser.add_argument('--workers', type=int, default=0, help='maximum number of dataloader workers') 90 | parser.add_argument('--optimizer', type=str, choices=['SGD', 'Adam', 'AdamW'], default='SGD', help='optimizer') 91 | parser.add_argument('--lr',type=int,default=0.02,help='Learning Rate') 92 | parser.add_argument('--weights',type=str,default='',help='initial weights path') 93 | parser.add_argument('--save-path',type=str,default='model.pth',help='save model') 94 | parser.add_argument('--verbose',action='store_true',help='可视化训练结果') 95 | parser.add_argument('--logs',action='store_true',help='每次迭代后都保存模型') 96 | parser.add_argument('--test-size',type=int,default=0,help='the number for picture in data/test') 97 | # parser.add_argument('--device', default='', help='cuda device, i.e. 0 or 0,1,2,3 or cpu') # 可以自动选择GPU，但是这里我就让程序自动判断 98 | 99 | opt = parser.parse_args() 100 | print(opt) 101 | main(opt) 102 | -------------------------------------------------------------------------------- /TransferLearning/utils.py: -------------------------------------------------------------------------------- 1 | from tqdm import tqdm 2 | import torch 3 | import torch.nn as nn 4 | import matplotlib.pyplot as plt 5 | from PIL import Image 6 | from torchvision import transforms 7 | 8 | def plot_history(epochs, Acc, Loss, lr): 9 | plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots 10 | plt.rcParams['image.interpolation'] = 'nearest' 11 | plt.rcParams['image.cmap'] = 'gray' 12 | 13 | epoch_list = range(1,epochs + 1) 14 | plt.plot(epoch_list, Loss['train_loss']) 15 | plt.plot(epoch_list, Loss['val_loss']) 16 | plt.xlabel('epoch') 17 | plt.ylabel('Loss Value') 18 | plt.legend(['train', 'val'], loc='upper left') 19 | plt.savefig('vis/history_Loss.png') 20 | plt.show() 21 | 22 | plt.plot(epoch_list, Acc['train_acc']) 23 | plt.plot(epoch_list, Acc['val_acc']) 24 | plt.xlabel('epoch') 25 | plt.ylabel('Acc Value') 26 | plt.legend(['train', 'val'], loc='upper left') 27 | plt.savefig('vis/history_Acc.png') 28 | plt.show() 29 | 30 | plt.plot(epoch_list, lr) 31 | plt.xlabel('epoch') 32 | plt.ylabel('Train LR') 33 | plt.savefig('vis/history_Lr.png') 34 | plt.show() 35 | 36 | def test(path, model): 37 | # 读取要预测的图片 38 | img = Image.open(path).convert('RGB') # 读取图像 39 | 40 | data_transform = transforms.Compose([transforms.Resize(256), 41 | transforms.CenterCrop(224), 42 | transforms.ToTensor(), 43 | transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])]) 44 | class_indict = ["cat", "dog"] 45 | plt.imshow(img) 46 | img = data_transform(img) 47 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 48 | img = img.to(device) 49 | img = torch.unsqueeze(img, dim=0) 50 | model.eval() 51 | with torch.no_grad(): 52 | # predict class 53 | output = torch.squeeze(model(img)) 54 | predict = torch.softmax(output, dim=0) 55 | predict_cla = torch.argmax(predict).data.cpu().numpy() 56 | print(class_indict[predict_cla], predict[predict_cla].data.cpu().numpy()) 57 | plt.savefig('{}_{:2.2f}.png'.format(class_indict[predict_cla],100*predict[predict_cla].data.cpu().numpy())) 58 | plt.show() 59 | 60 | def get_acc(outputs, label): 61 | total = outputs.shape[0] 62 | probs, pred_y = outputs.data.max(dim=1) # 得到概率 63 | correct = (pred_y == label).sum().data 64 | return correct / total 65 | 66 | # optimizer = torch.optim.SGD(model.parameters(), lr=0.01, momentum=0.9) 67 | # criterion = nn.CrossEntropyLoss() 68 | # scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer,patience=3,factor=0.5,min_lr=1e-6) 69 | def train(model,trainloader, valloader, optimizer , criterion, scheduler , epochs = 10 , path = './model.pth', writer = False, verbose = False, logs = False, pretrain = ''): 70 | 71 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 72 | if pretrain != '': 73 | print('Load weights {}.'.format(pretrain)) 74 | model.load_state_dict(torch.load(pretrain)) 75 | 76 | best_acc = 0 77 | train_acc_list, val_acc_list = [],[] 78 | train_loss_list, val_loss_list = [],[] 79 | lr_list = [] 80 | epoch_step = len(trainloader) 81 | epoch_step_val = len(valloader) 82 | 83 | if epoch_step == 0 or epoch_step_val == 0: 84 | raise ValueError("数据集过小，无法进行训练，请扩充数据集，或者减小batchsize") 85 | 86 | for epoch in range(epochs): 87 | train_loss = 0 88 | train_acc = 0 89 | val_loss = 0 90 | val_acc = 0 91 | if torch.cuda.is_available(): 92 | model = model.to(device) 93 | model.train() 94 | print('Start Train') 95 | with tqdm(total=epoch_step,desc=f'Epoch {epoch + 1}/{epochs}',postfix=dict,mininterval=0.3) as pbar: 96 | for step,data in enumerate(trainloader,start=0): 97 | im,label = data 98 | im = im.to(device) 99 | label = label.to(device) 100 | #--------------------- 101 | # 释放内存 102 | #--------------------- 103 | if hasattr(torch.cuda, 'empty_cache'): 104 | torch.cuda.empty_cache() 105 | #----------------------# 106 | # 清零梯度 107 | #----------------------# 108 | optimizer.zero_grad() 109 | #----------------------# 110 | # 前向传播forward 111 | #----------------------# 112 | outputs = model(im) 113 | #----------------------# 114 | # 计算损失 115 | #----------------------# 116 | loss = criterion(outputs,label) 117 | train_loss += loss.data 118 | train_acc += get_acc(outputs,label) 119 | #----------------------# 120 | # 反向传播 121 | #----------------------# 122 | # backward 123 | loss.backward() 124 | # 更新参数 125 | optimizer.step() 126 | lr = optimizer.param_groups[0]['lr'] 127 | pbar.set_postfix(**{'Train Loss' : train_loss.item()/(step+1), 128 | 'Train Acc' :train_acc.item()/(step+1), 129 | 'Lr' : lr}) 130 | pbar.update(1) 131 | train_loss = train_loss.item() / len(trainloader) 132 | train_acc = train_acc.item() * 100 / len(trainloader) 133 | if verbose: 134 | train_acc_list.append(train_acc) 135 | train_loss_list.append(train_loss) 136 | 137 | # 记录学习率 138 | lr = optimizer.param_groups[0]['lr'] 139 | if verbose: 140 | lr_list.append(lr) 141 | 142 | # 更新学习率 143 | scheduler.step(train_loss) 144 | 145 | print('Finish Train') 146 | 147 | model.eval() 148 | print('Start Validation') 149 | #-------------------------------- 150 | # 相同方法，同train 151 | #-------------------------------- 152 | with tqdm(total=epoch_step_val,desc=f'Epoch {epoch + 1}/{epochs}',postfix=dict,mininterval=0.3) as pbar2: 153 | for step,data in enumerate(valloader,start=0): 154 | im,label = data 155 | im = im.to(device) 156 | label = label.to(device) 157 | with torch.no_grad(): 158 | if step >= epoch_step_val: 159 | break 160 | 161 | # 释放内存 162 | if hasattr(torch.cuda, 'empty_cache'): 163 | torch.cuda.empty_cache() 164 | #----------------------# 165 | # 前向传播 166 | #----------------------# 167 | outputs = model(im) 168 | loss = criterion(outputs,label) 169 | val_loss += loss.data 170 | # probs, pred_y = outputs.data.max(dim=1) # 得到概率 171 | # test_acc += (pred_y==label).sum().item() 172 | # total += label.size(0) 173 | val_acc += get_acc(outputs,label) 174 | 175 | pbar2.set_postfix(**{'Val Acc': val_acc.item()/(step+1), 176 | 'Val Loss': val_loss.item() / (step + 1)}) 177 | pbar2.update(1) 178 | 179 | val_loss = val_loss.item() / len(valloader) 180 | val_acc = val_acc.item() * 100 / len(valloader) 181 | if verbose: 182 | val_loss_list.append(val_loss) 183 | val_acc_list.append(val_acc) 184 | print('Finish Validation') 185 | 186 | print("Epoch [{:>3d}/{:>3d}] Train Acc: {:>3.2f}% Train Loss: {:>.6f} || Val Acc: {:>3.2f}% Val Loss: {:>.6f} || Learning Rate:{:>.6f}" 187 | .format(epoch+1,epochs,train_acc,train_loss,val_acc,val_loss,lr)) 188 | 189 | # ====================== 使用 tensorboard ==================: 190 | if writer: 191 | writer.add_scalars('Loss', {'train': train_loss.item(), 192 | 'valid': val_loss.item()}, epoch+1) 193 | writer.add_scalars('Acc', {'train': train_acc.item() , 194 | 'valid': val_acc.item()}, epoch+1) 195 | # writer.add_scalars('Learning Rate',lr,i+1) 196 | # ========================================================= 197 | 198 | #------------------------------------------------------------# 199 | # 保存日志模型，中断以后可以继续训练 200 | #------------------------------------------------------------# 201 | if logs: 202 | torch.save(model.state_dict(), 'logs/ep%03d-loss%.3f-val_loss%.3f.pth' % (epoch + 1, train_loss, val_loss)) 203 | 204 | #-----------------------------------------------------------# 205 | # 如果取得更好的准确率，就保存模型 206 | #-----------------------------------------------------------# 207 | if path != '': 208 | if val_acc > best_acc: 209 | torch.save(model,'./model/'+path) 210 | best_acc = val_acc 211 | torch.cuda.empty_cache() 212 | if verbose: 213 | Acc = {} 214 | Loss = {} 215 | Acc['train_acc'] = train_acc_list 216 | Acc['val_acc'] = val_acc_list 217 | Loss['train_loss'] = train_loss_list 218 | Loss['val_loss'] = val_loss_list 219 | Lr = lr_list 220 | return Acc, Loss, Lr -------------------------------------------------------------------------------- /docs/Pytorch CIFAR10图像分类 工具函数utils更新v2.0篇.md: -------------------------------------------------------------------------------- 1 | # Pytorch CIFAR10图像分类 工具函数utils更新v2.0篇 2 | 3 | [toc] 4 | 5 | 这里贴一下汇总篇：[汇总篇](https://blog.csdn.net/weixin_45508265/article/details/119285255) 6 | 7 | 对于上一版的工具函数utils.py，我认为可能来说，可视化的感觉还是不是很好，所以我就修改了一下我新的训练函数，为了兼容，参数基本相同，但是加入了tqdm来可视化进度条，这样也会更加的好看和直观，并且统一了一些代码的格式，使得代码稍微好看点，之前有时候有点乱，除此为了兼容一些情况，修改了部分代码，但是意义相同。 8 | 9 | 如果想看上一版的工具函数utils.py，可以查看这篇博客。[PytorchCIFAR10图像分类 工具函数utils篇](https://redamancy.blog.csdn.net/article/details/121589217) 10 | 11 | 12 | 13 | ## 设置随机种子（保证可复现性） 14 | 15 | 为了保证训练结果的可复现性，同时加了设置随机种子的代码 16 | 17 | ```python 18 | def setup_seed(seed): 19 | np.random.seed(seed) 20 | random.seed(seed) 21 | torch.manual_seed(seed) 22 | torch.cuda.manual_seed(seed) 23 | torch.cuda.manual_seed_all(seed) 24 | # torch.backends.cudnn.benchmark = True #for accelerating the running 25 | ``` 26 | 27 | 28 | 29 | ## 得到分类的精度（适配高版本） 30 | 31 | 首先对于这个函数来说，我变化了一下最后一行的 32 | 33 | ```python 34 | return correct / total 35 | ``` 36 | 37 | 因为这一行在多数时候是正确的，但是在一些版本的torch中是不兼容的，这时候需要使用torch.div或者torch.true_div 38 | 39 | ```python 40 | def get_acc(outputs, label): 41 | total = outputs.shape[0] 42 | probs, pred_y = outputs.data.max(dim=1) # 得到概率 43 | correct = (pred_y == label).sum().data 44 | return torch.div(correct, total) 45 | ``` 46 | 47 | 48 | 49 | ## 训练函数最新版本（增加tensorboard函数和tqdm可视化进度条） 50 | 51 | 实际上，为了保证版本的兼容，所以参数都是一样的，不同的地方是，我利用了tqdm来可视化我们的进度条 52 | 53 | 这样以后，我们能更好的看到自己的训练进度和时间，并且在训练过程中就可以看到自己的结果，这样就更好的保证了可视化，在训练过程中也可以看到准确率和损失的变化。 54 | 55 | **参数介绍** 56 | 57 | - **net : 所选的网络模型** 58 | - **trainloader： 训练集加载器** 59 | - **testloader： 测试集加载器** 60 | - **epoches：训练次数** 61 | - **optimizer：优化器** 62 | - **criterion：损失函数** 63 | - **writer：是否使用tensorboard可视化，默认为None** 64 | - **verbose：是否使用记录准确率，损失值，学习率，默认为True** 65 | - **scheduler：学习率调整策略** 66 | - **path：保存迭代次数中最优的模型的权重，默认为model.pth** 67 | 68 | ```python 69 | def train(net, trainloader, testloader, epochs, optimizer , criterion, scheduler , path = './model.pth', writer = None ,verbose = False): 70 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 71 | best_acc = 0 72 | train_acc_list, test_acc_list = [],[] 73 | train_loss_list, test_loss_list = [],[] 74 | lr_list = [] 75 | for i in range(epochs): 76 | train_loss = 0 77 | train_acc = 0 78 | test_loss = 0 79 | test_acc = 0 80 | if torch.cuda.is_available(): 81 | net = net.to(device) 82 | net.train() 83 | train_step = len(trainloader) 84 | with tqdm(total=train_step,desc=f'Train Epoch {i + 1}/{epochs}',postfix=dict,mininterval=0.3) as pbar: 85 | for step,data in enumerate(trainloader,start=0): 86 | im,label = data 87 | im = im.to(device) 88 | label = label.to(device) 89 | 90 | optimizer.zero_grad() 91 | # 释放内存 92 | if hasattr(torch.cuda, 'empty_cache'): 93 | torch.cuda.empty_cache() 94 | # formard 95 | outputs = net(im) 96 | loss = criterion(outputs,label) 97 | # backward 98 | loss.backward() 99 | # 更新参数 100 | optimizer.step() 101 | # 累计损失 102 | train_loss += loss.item() 103 | train_acc += get_acc(outputs,label).item() 104 | pbar.set_postfix(**{'Train Acc' : train_acc/(step+1), 105 | 'Train Loss' :train_loss/(step+1)}) 106 | pbar.update(1) 107 | pbar.close() 108 | train_loss = train_loss / len(trainloader) 109 | train_acc = train_acc * 100 / len(trainloader) 110 | if verbose: 111 | train_acc_list.append(train_acc) 112 | train_loss_list.append(train_loss) 113 | # 记录学习率 114 | lr = optimizer.param_groups[0]['lr'] 115 | if verbose: 116 | lr_list.append(lr) 117 | # 更新学习率 118 | scheduler.step(train_loss) 119 | if testloader is not None: 120 | net.eval() 121 | test_step = len(testloader) 122 | with torch.no_grad(): 123 | with tqdm(total=test_step,desc=f'Test Epoch {i + 1}/{epochs}',postfix=dict,mininterval=0.3) as pbar: 124 | for step,data in enumerate(testloader,start=0): 125 | im,label = data 126 | im = im.to(device) 127 | label = label.to(device) 128 | # 释放内存 129 | if hasattr(torch.cuda, 'empty_cache'): 130 | torch.cuda.empty_cache() 131 | outputs = net(im) 132 | loss = criterion(outputs,label) 133 | test_loss += loss.item() 134 | test_acc += get_acc(outputs,label).item() 135 | pbar.set_postfix(**{'Test Acc' : test_acc/(step+1), 136 | 'Test Loss' :test_loss/(step+1)}) 137 | pbar.update(1) 138 | pbar.close() 139 | test_loss = test_loss / len(testloader) 140 | test_acc = test_acc * 100 / len(testloader) 141 | if verbose: 142 | test_loss_list.append(test_loss) 143 | test_acc_list.append(test_acc) 144 | print( 145 | 'Epoch [{:>3d}/{:>3d}] Train Loss:{:>.6f} Train Acc:{:>3.2f}% Test Loss:{:>.6f} Test Acc:{:>3.2f}% Learning Rate:{:>.6f}'.format( 146 | i + 1, epochs, train_loss, train_acc, test_loss, test_acc,lr)) 147 | else: 148 | print('Epoch [{:>3d}/{:>3d}] Train Loss:{:>.6f} Train Acc:{:>3.2f}% Learning Rate:{:>.6f}'.format(i+1,epochs,train_loss,train_acc,lr)) 149 | 150 | # ====================== 使用 tensorboard ================== 151 | if writer is not None: 152 | writer.add_scalars('Loss', {'train': train_loss, 153 | 'test': test_loss}, i+1) 154 | writer.add_scalars('Acc', {'train': train_acc , 155 | 'test': test_acc}, i+1) 156 | writer.add_scalar('Learning Rate',lr,i+1) 157 | # ========================================================= 158 | # 如果取得更好的准确率，就保存模型 159 | if test_acc > best_acc: 160 | torch.save(net,path) 161 | best_acc = test_acc 162 | Acc = {} 163 | Loss = {} 164 | Acc['train_acc'] = train_acc_list 165 | Acc['test_acc'] = test_acc_list 166 | Loss['train_loss'] = train_loss_list 167 | Loss['test_loss'] = test_loss_list 168 | Lr = lr_list 169 | return Acc, Loss, Lr 170 | ``` 171 | 172 | 173 | 174 | ## 可视化准确率、损失、学习率变化 175 | 176 | 这一部分用原来的，几乎没有什么变化，就更新了一下图片的大小 177 | 178 | 以下函数可以可视化准确率、损失、学习率随着迭代次数的变化 179 | 180 | ```python 181 | def plot_history(epochs, Acc, Loss, lr): 182 | plt.rcParams['figure.figsize'] = (12.0, 8.0) # set default size of plots 183 | plt.rcParams['image.interpolation'] = 'nearest' 184 | plt.rcParams['image.cmap'] = 'gray' 185 | 186 | epoch_list = range(1,epochs + 1) 187 | plt.plot(epoch_list, Loss['train_loss']) 188 | plt.plot(epoch_list, Loss['test_loss']) 189 | plt.xlabel('epoch') 190 | plt.ylabel('Loss Value') 191 | plt.legend(['train', 'test'], loc='upper left') 192 | plt.show() 193 | 194 | plt.plot(epoch_list, Acc['train_acc']) 195 | plt.plot(epoch_list, Acc['test_acc']) 196 | plt.xlabel('epoch') 197 | plt.ylabel('Acc Value') 198 | plt.legend(['train', 'test'], loc='upper left') 199 | plt.show() 200 | 201 | plt.plot(epoch_list, lr) 202 | plt.xlabel('epoch') 203 | plt.ylabel('Train LR') 204 | plt.show() 205 | ``` 206 | 207 | -------------------------------------------------------------------------------- /docs/Pytorch CIFAR10图像分类 数据加载篇.md: -------------------------------------------------------------------------------- 1 | # Pytorch CIFAR10图像分类 数据加载与可视化篇 2 | 3 | [toc] 4 | 5 | 这里贴一下汇总篇：[汇总篇](https://blog.csdn.net/weixin_45508265/article/details/119285255) 6 | 7 | **Pytorch一般有以下几个流程** 8 | 9 | 1. 数据读取 10 | 2. 数据处理 11 | 3. 搭建网络 12 | 4. 模型训练 13 | 5. 模型上线 14 | 15 | 这里会先讲一下关于CIFAR10的数据加载和图片可视化，之后的模型篇会对网络进行介绍和实现。 16 | 17 | ### 1.数据读取 18 | 19 | CIFAR-10 是由 Hinton 的学生 Alex Krizhevsky 和 Ilya Sutskever 整理的一个用于识别普适物体的小型数据集。一共包含 10 个类别的 RGB 彩色图 片：飞机（ arplane ）、汽车（ automobile ）、鸟类（ bird ）、猫（ cat ）、鹿（ deer ）、狗（ dog ）、蛙类（ frog ）、马（ horse ）、船（ ship ）和卡车（ truck ）。图片的尺寸为 32×32 ，数据集中一共有 50000 张训练圄片和 10000 张测试图片。 20 | 21 | 与 MNIST 数据集中目比， CIFAR-10 具有以下不同点： 22 | 23 | - CIFAR-10 是 3 通道的彩色 RGB 图像，而 MNIST 是灰度图像。 24 | - CIFAR-10 的图片尺寸为 32×32， 而 MNIST 的图片尺寸为 28×28，比 MNIST 稍大。 25 | - 相比于手写字符， CIFAR-10 含有的是现实世界中真实的物体，不仅噪声很大，而且物体的比例、 特征都不尽相同，这为识别带来很大困难。 26 | 27 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/16f85f24a70e452e8659a1874616420f.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NTUwODI2NQ==,size_16,color_FFFFFF,t_70#pic_center) 28 | 29 | 30 | 31 | 首先使用`torchvision`加载和归一化我们的训练数据和测试数据。 32 | 33 | a、`torchvision`这个东西，实现了常用的一些深度学习的相关的图像数据的加载功能，比如cifar10、Imagenet、Mnist等等的，保存在`torchvision.datasets`模块中。 34 | 35 | b、同时，也封装了一些处理数据的方法。保存在`torchvision.transforms`模块中 36 | 37 | c、还封装了一些模型和工具封装在相应模型中,比如`torchvision.models`当中就包含了AlexNet，VGG，ResNet，SqueezeNet等模型。 38 | 39 | 40 | 41 | **由于torchvision的datasets的输出是[0,1]的PILImage，所以我们先先归一化为[-1,1]的Tensor** 42 | 43 | 首先定义了一个变换transform，利用的是上面提到的transforms模块中的Compose( )把多个变换组合在一起，可以看到这里面组合了ToTensor和Normalize这两个变换 44 | 45 | `transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))`前面的（0.5，0.5，0.5） 是 R G B 三个通道上的均值， 后面(0.5, 0.5, 0.5)是三个通道的标准差，注意通道顺序是 R G B ，用过opencv的同学应该知道openCV读出来的图像是 BRG顺序。这两个tuple数据是用来对RGB 图像做归一化的，如其名称 Normalize 所示这里都取0.5只是一个近似的操作，实际上其均值和方差并不是这么多，但是就这个示例而言 影响可不计。精确值是通过分别计算R,G,B三个通道的数据算出来的。 46 | 47 | ```python 48 | transform = transforms.Compose([ 49 | # transforms.CenterCrop(224), 50 | transforms.RandomCrop(32,padding=4), # 数据增广 51 | transforms.RandomHorizontalFlip(), # 数据增广 52 | transforms.ToTensor(), 53 | transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)) 54 | ]) 55 | ``` 56 | 57 | `trainloader`其实是一个比较重要的东西，我们后面就是通过`trainloader`把数据传入网络，当然这里的`trainloader`其实是个变量名，可以随便取，重点是他是由后面的`torch.utils.data.DataLoader()`定义的，这个东西来源于`torch.utils.data`模块 58 | 59 | ```python 60 | Batch_Size = 256 61 | ``` 62 | 63 | ```python 64 | trainset = datasets.CIFAR10(root='./data', train=True,download=True, transform=transform) 65 | testset = datasets.CIFAR10(root='./data',train=False,download=True,transform=transform) 66 | trainloader = torch.utils.data.DataLoader(trainset, batch_size=Batch_Size,shuffle=True, num_workers=2) 67 | testloader = torch.utils.data.DataLoader(testset, batch_size=Batch_Size,shuffle=True, num_workers=2) 68 | classes = ('plane', 'car', 'bird', 'cat','deer', 'dog', 'frog', 'horse', 'ship', 'truck') 69 | ``` 70 | 71 | > ```python 72 | > Files already downloaded and verified 73 | > Files already downloaded and verified 74 | > ``` 75 | 76 | ### 2. 查看数据（格式，大小，形状） 77 | 78 | 首先可以查看类别 79 | 80 | ```python 81 | classes = trainset.classes 82 | classes 83 | ``` 84 | 85 | > ```python 86 | > ['airplane', 87 | > 'automobile', 88 | > 'bird', 89 | > 'cat', 90 | > 'deer', 91 | > 'dog', 92 | > 'frog', 93 | > 'horse', 94 | > 'ship', 95 | > 'truck'] 96 | > ``` 97 | 98 | ```python 99 | trainset.class_to_idx 100 | ``` 101 | 102 | > ```python 103 | > {'airplane': 0, 104 | > 'automobile': 1, 105 | > 'bird': 2, 106 | > 'cat': 3, 107 | > 'deer': 4, 108 | > 'dog': 5, 109 | > 'frog': 6, 110 | > 'horse': 7, 111 | > 'ship': 8, 112 | > 'truck': 9} 113 | > ``` 114 | 115 | 也可以查看一下训练集的数据 116 | 117 | ```python 118 | trainset.data.shape #50000是图片数量，32x32是图片大小，3是通道数量RGB 119 | ``` 120 | 121 | > ```python 122 | > (50000, 32, 32, 3) 123 | > ``` 124 | 125 | 查看数据类型 126 | 127 | ```python 128 | #查看数据类型 129 | print(type(trainset.data)) 130 | print(type(trainset)) 131 | ``` 132 | 133 | > ```python 134 | > 135 | > 136 | > ``` 137 | > 138 | > 139 | 140 | **总结：** 141 | 142 | `trainset.data.shape`是标准的numpy.ndarray类型，其中50000是图片数量，32x32是图片大小，3是通道数量RGB； 143 | `trainset`是标准的？？类型，其中50000为图片数量，0表示取前面的数据，2表示3通道数RGB，32*32表示图片大小 144 | 145 | ### 3. 查看图片 146 | 147 | 接下来我们对图片进行可视化 148 | 149 | ```python 150 | import numpy as np 151 | import matplotlib.pyplot as plt 152 | plt.imshow(trainset.data[0]) 153 | im,label = iter(trainloader).next() 154 | ``` 155 | 156 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/d2733fe9714446caa0f6ff0d8501adcd.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NTUwODI2NQ==,size_16,color_FFFFFF,t_70) 157 | 158 | #### np.ndarray转为torch.Tensor 159 | 160 | 在深度学习中，原始图像需要转换为深度学习框架自定义的数据格式，在pytorch中，需要转为`torch.Tensor`。 161 | pytorch提供了`torch.Tensor` 与`numpy.ndarray`转换为接口： 162 | 163 | | 方法名 | 作用 | 164 | | ----------------------- | ------------------------------- | 165 | | `torch.from_numpy(xxx)` | `numpy.ndarray`转为torch.Tensor | 166 | | `tensor1.numpy()` | 获取tensor1对象的numpy格式数据 | 167 | 168 | `torch.Tensor` 高维矩阵的表示： N x C x H x W 169 | 170 | `numpy.ndarray` 高维矩阵的表示：N x H x W x C 171 | 172 | 因此在两者转换的时候需要使用`numpy.transpose( )` 方法 。 173 | 174 | ```python 175 | def imshow(img): 176 | img = img / 2 + 0.5 177 | img = np.transpose(img.numpy(),(1,2,0)) 178 | plt.imshow(img) 179 | ``` 180 | 181 | ```python 182 | imshow(im[0]) 183 | ``` 184 | 185 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/bc61a5f1af4b45f696e2a6d8bfc7b223.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NTUwODI2NQ==,size_16,color_FFFFFF,t_70) 186 | 187 | 我们也可以批量可视化图片，不过这里需要用到`make_grid` 188 | 189 | ```python 190 | plt.figure(figsize=(8,12)) 191 | imshow(torchvision.utils.make_grid(im[:32])) 192 | ``` 193 | 194 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/b9beb6f0fe4b4fcb9ef26f300bbb242b.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NTUwODI2NQ==,size_16,color_FFFFFF,t_70) -------------------------------------------------------------------------------- /docs/Pytorch&Keras CIFAR10图像分类.md: -------------------------------------------------------------------------------- 1 | # ✨Pytorch&Keras CIFAR10图像分类 2 | 3 | 4 | 5 | - [✨Pytorch&Keras CIFAR10图像分类](#pytorchkeras-cifar10图像分类) 6 | - [🧑‍🎓适用人群](#‍适用人群) 7 | - [📚︎博客汇总](#︎博客汇总) 8 | - [💻︎ Pytorch CIFAR10 图像分类](#︎-pytorch-cifar10-图像分类) 9 | - [⁉ 数据处理以及工具函数（网络篇前必看）](#⁉-数据处理以及工具函数网络篇前必看) 10 | - [❕ 网络篇](#-网络篇) 11 | - [🖥︎ Keras CIFAR10 图像分类](#🖥︎-keras-cifar10-图像分类) 12 | - [💝 有趣的项目和尝试](#-有趣的项目和尝试) 13 | - [📅 Comming soon 更新计划](#-comming-soon-更新计划) 14 | - [🧰 使用方法](#🧰-使用方法) 15 | - [📚 参考](#-参考) 16 | 17 | 18 | ![Keras vs PyTorch vs Caffe - Comparing the Implementation of CNN](https://149695847.v2.pressablecdn.com/wp-content/uploads/2020/08/create-machine-learning-and-deep-learning-models-using-pytorch-and-tensorflow.jpg#pic_center) 19 | 20 | ## 💪专栏介绍 21 | 22 | 一开始写这个专栏的初衷是，**有时候有些代码找的太繁琐了，并且找到了还运行不出来，或者运行了，但是没有一个很好的理解**，所以基于此，我写了这个CIFAR10图像分类的专栏，借此希望，大家都能通过我这个专栏，找到自己想要的模型或者学习深度学习的代码。 23 | 24 | 由于现在深度学习也在热度中，很多时候我们难免需要遇见深度学习的时候，**在一些课程和项目的要求下，我们会发现，一个好的代码和一个好的可视化和清晰解释是能够节省特别特别多的时间的**，基于此，我写出了这个专栏，这个专栏下的所有项目，都是**可运行无差错的。**如果遇见了问题，也可以留言跟我沟通 25 | 26 | --- 27 | 28 | ## 🧑‍🎓适用人群 29 | 30 | 他很适合大家**初入深度学习或者是Pytorch和Keras**，希望这能够帮助初学深度学习的同学一个入门Pytorch或者Keras的项目和在这之中更加了解Pytorch&Keras和各个图像分类的模型。 31 | 32 | 他有比较清晰的可视化结构和架构，除此之外，我是**用jupyter写的，所以说在文章整体架构可以说是非常清晰**，可以帮助你快速学习到**各个模块的知识**，而不是通过python脚本一行一行的看，这样的方式是符合初学者的。 33 | 34 | 除此之外，如果你需要变成脚本形式，也是很简单的。 35 | 36 | --- 37 | 38 | ## 📚︎博客汇总 39 | 40 | 为了使得文章体系结构更加清晰，这里给出**Pytorch&Keras对于CIFAR10图像分类的所有资料汇总**，也就是我的博客汇总，也告诉大家，我做了什么工作，如果大家有兴趣订阅我的专栏亦或者是有什么其他模型的想法，也可以评论留言，我也可以进行去学习的。 41 | 42 | 这部分也方便大家看到介绍，并且快速找到自己所需要的代码进行学习和了解 43 | 44 | --- 45 | 46 | ### 💻︎ Pytorch CIFAR10 图像分类 47 | 48 | 在看网络篇的时候，可以先看前三个，前三个说明的如何加载数据包括数据的预处理以及进行可视化，工具函数篇介绍了如何构建训练的函数，并且有时候会遇到一部分utils.py的错误，在这里面都有很好的解释和进行学习。 49 | 50 | #### ⁉ 数据处理以及工具函数（网络篇前必看） 51 | 52 | - [Pytorch CIFAR10图像分类 数据加载与可视化篇](https://blog.csdn.net/weixin_45508265/article/details/119285113) 53 | - [Pytorch CIFAR10图像分类 工具函数utils篇](https://redamancy.blog.csdn.net/article/details/121589217) 54 | - [Pytorch CIFAR10图像分类 工具函数utils更新v2.0篇](https://redamancy.blog.csdn.net/article/details/127856569) 55 | 56 | #### ❕ 网络篇 57 | 58 | - [Pytorch CIFAR10图像分类 自定义网络篇](https://blog.csdn.net/weixin_45508265/article/details/119305277) 59 | - [Pytorch CIFAR10图像分类 LeNet5篇](https://blog.csdn.net/weixin_45508265/article/details/119305673) 60 | - [Pytorch CIFAR10图像分类 AlexNet篇](https://blog.csdn.net/weixin_45508265/article/details/119305848) 61 | - [Pytorch CIFAR10图像分类 VGG篇](https://blog.csdn.net/weixin_45508265/article/details/119332904) 62 | - [Pytorch CIFAR10图像分类 GoogLeNet篇](https://blog.csdn.net/weixin_45508265/article/details/119399239) 63 | - [Pytorch CIFAR10图像分类 ResNet篇](https://blog.csdn.net/weixin_45508265/article/details/119532143) 64 | - [Pytorch CIFAR10图像分类 DenseNet篇](https://blog.csdn.net/weixin_45508265/article/details/119648036) 65 | - [Pytorch CIFAR10图像分类 MobieNetv1篇](https://redamancy.blog.csdn.net/article/details/124636103) 66 | - [Pytorch CIFAR10图像分类 MobileNetv2篇](https://redamancy.blog.csdn.net/article/details/127946431) 67 | - [Pytorch CIFAR10图像分类 ResNeXt篇](https://redamancy.blog.csdn.net/article/details/126655797) 68 | - [Pytorch CIFAR10图像分类 ZFNet篇](https://blog.csdn.net/weixin_45508265/article/details/128560595) 69 | - [Pytorch CIFAR10图像分类 SENet篇](https://blog.csdn.net/weixin_45508265/article/details/130938341) 70 | - [Pytorch CIFAR10 图像分类 Vision Transformer篇](https://redamancy.blog.csdn.net/article/details/126751948) 71 | - [Pytorch CIFAR10图像分类 EfficientNet篇](https://blog.csdn.net/weixin_45508265/article/details/128585354) 72 | - [Pytorch CIFAR10图像分类 ShuffleNet篇](https://blog.csdn.net/weixin_45508265/article/details/130945031) 73 | - [Pytorch CIFAR10图像分类 ShuffleNetv2篇](https://blog.csdn.net/weixin_45508265/article/details/130938341) 74 | - [Pytorch CIFAR10图像分类 Swin Tansformer篇](https://blog.csdn.net/weixin_45508265/article/details/134773753) 75 | 76 | > 具体的详情可以关注[Pytorch CIFAR10图像分类汇总篇](https://redamancy.blog.csdn.net/article/details/119285255) 77 | 78 | --- 79 | 80 | ### 🖥︎ Keras CIFAR10 图像分类 81 | 82 | - [Keras CIFAR-10分类 SVM 分类器篇][1] 83 | - [Keras CIFAR-10分类 LeNet-5篇][2] 84 | - [Keras CIFAR-10分类 AlexNet篇][3] 85 | - [Keras CIFAR-10分类 GoogleNet篇][4] 86 | - [Keras CIFAR-10分类 VGG篇][5] 87 | - [Keras CIFAR-10分类 ResNet篇][6] 88 | - [Keras CIFAR-10分类 DenseNet篇][7] 89 | 90 | > 具体的详情可以关注[Keras CIFAR-10 分类汇总篇](https://blog.csdn.net/weixin_45508265/article/details/127859003) 91 | 92 | --- 93 | 94 | ### 💝 有趣的项目和尝试 95 | 96 | - [MAE实现及预训练可视化 （CIFAR-Pytorch）][MAE] 97 | 98 | --- 99 | 100 | ## 📅 Comming soon 更新计划 101 | 102 | - [x] LetNet 103 | - [x] AlexNet 104 | - [x] VGG 105 | - [x] ResNet 106 | - [x] GoogLeNet 107 | - [x] DenseNet 108 | - [x] ResNeXt 109 | - [x] MobileNetv1 110 | - [x] MobileNetv2 111 | - [x] ZFNet 112 | - [x] SeNet 113 | - [x] Efficientent 114 | - [x] ViT 115 | - [x] ShuffleNetv1 116 | - [x] ShuffleNetv2 117 | - [x] Swin-Transformer 118 | - [ ] ConvNeXt 119 | - [ ] ConvNeXtv2 120 | 121 | --- 122 | 123 | ## 🧰 使用方法 124 | 125 | 下载`CIFAR10`里所有文件，直接运行ipynb即可，由于我是利用一个工具函数进行训练的，所以**切记utils.py是必不可少的。** 126 | 127 | 运行ipynb文件即可，对于网络的py文件会持续更新，之后会利用一个函数来选取对应的网络进行训练得到结果。 128 | 129 | --- 130 | 131 | ## 📚 参考 132 | 133 | 除此之外，我还为图像分类这个专栏录了一下我的视频讲解，感兴趣的小伙伴可以来我的B站看视频进行学习，啃代码的时候，可能听一下也会有更多的感触哦 134 | [https://space.bilibili.com/241286257/channel/seriesdetail?sid=2075039](https://space.bilibili.com/241286257/channel/seriesdetail?sid=2075039) 135 | 136 | --- 137 | 138 | 最后这个是我写的一个pytorch的基础的介绍，[Pytorch Note 快乐星球](https://blog.csdn.net/weixin_45508265/article/details/117809512)，从0开始的完整的介绍pytorch和pytorch的简单语法，并且里面有一些项目和学习，还是很不错的哦，可以查看，除此之外，有什么想法可以加我wx: `pikachu2biubiu`聊哦，需要什么帮助也可以付费聊咨询。 139 | 140 | ![二维码](C:\Users\86137\Documents\GitHub\Pytorch-Image-Classification\QR.png) 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | 150 | 151 | 152 | [1]: https://redamancy.blog.csdn.net/article/details/126445778 153 | [2]: https://redamancy.blog.csdn.net/article/details/126446810 154 | [3]: https://redamancy.blog.csdn.net/article/details/126590621 155 | [4]: https://redamancy.blog.csdn.net/article/details/126591761 156 | [5]: https://redamancy.blog.csdn.net/article/details/126669709 157 | [6]: https://redamancy.blog.csdn.net/article/details/127827641 158 | [7]: https://redamancy.blog.csdn.net/article/details/127828318 159 | [MAE]: https://redamancy.blog.csdn.net/article/details/126863995 160 | 161 | -------------------------------------------------------------------------------- /docs/README.md: -------------------------------------------------------------------------------- 1 | # Pytorch CIFAR10 图像分类篇 汇总 2 | 3 | 4 | 5 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/f37d635b66334021a12521d6bed88d87.png#pic_center) 6 | 7 | 接下来我会分别利用深度学习的方法，用Pytorch实现我们的CIFAR10的图像分类 8 | 9 | 大概预计的模型有LeNet，AlexNet，VGG，GoogLeNet，ResNet，DenseNet，MobileNet，Vision Transformer， ResNeXt等，除此之外也会陆续补充 10 | 11 | 希望这能够帮助初学机器学习的同学一个入门Pytorch的项目和在这之中更加了解Pytorch和各个图像分类的模型。 12 | 13 | - [Pytorch CIFAR10图像分类 数据加载与可视化篇](https://blog.csdn.net/weixin_45508265/article/details/119285113) [B站视频](https://www.bilibili.com/video/BV1FP4y1g7sc) 14 | - [Pytorch CIFAR10图像分类 工具函数utils篇](https://redamancy.blog.csdn.net/article/details/121589217) [Online Demo](https://drive.google.com/file/d/1HESbXuEb__9eXqq4tAl8owsb1FKhpO2i/view?usp=sharing) 15 | - [Pytorch CIFAR10图像分类 自定义网络篇](https://blog.csdn.net/weixin_45508265/article/details/119305277) [B站视频](https://www.bilibili.com/video/BV1MF41147gZ) [Colab Demo](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) for 自定义网络 [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) 16 | - [Pytorch CIFAR10图像分类 LeNet5篇](https://blog.csdn.net/weixin_45508265/article/details/119305673) [B站视频](https://www.bilibili.com/video/BV1FL411K7VJ) [Colab Demo](https://colab.research.google.com/drive/15B0HBssfRzQk8mJyYF-v5fwAdPtNqf3H?usp=sharing) for LeNet5 [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/15B0HBssfRzQk8mJyYF-v5fwAdPtNqf3H?usp=sharing) 17 | - [Pytorch CIFAR10图像分类 AlexNet篇](https://blog.csdn.net/weixin_45508265/article/details/119305848) [B站视频](https://www.bilibili.com/video/BV1xu411B75x) [Colab Demo](https://colab.research.google.com/drive/1d6CTYzyWeB03xiSlT8mzsZ_LtH9TlPvs?usp=sharing) for AlexNet [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1d6CTYzyWeB03xiSlT8mzsZ_LtH9TlPvs?usp=sharing) 18 | - [Pytorch CIFAR10图像分类 VGG篇](https://blog.csdn.net/weixin_45508265/article/details/119332904) [B站视频](https://www.bilibili.com/video/BV12L4y1u7WH) [Colab Demo](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) for VGG16 [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) 19 | - [Pytorch CIFAR10图像分类 GoogLeNet篇](https://blog.csdn.net/weixin_45508265/article/details/119399239) [B站视频](https://www.bilibili.com/video/BV1RS4y1274A) [Colab Demo](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) for GoogLeNet Inceptionv1 [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1o8lfWHvr4WoyTA5Y9b4mSCSw2TEbXJb7?usp=sharing) 20 | - [Pytorch CIFAR10图像分类 ResNet篇](https://blog.csdn.net/weixin_45508265/article/details/119532143) [B站视频](https://www.bilibili.com/video/BV1Wu411v72u) [Colab Demo](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) for ResNet [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1W6d-eTY89bvGEL_QoMq4kw7m0dP9lHkS?usp=sharing) 21 | - [Pytorch CIFAR10图像分类 DenseNet篇](https://blog.csdn.net/weixin_45508265/article/details/119648036) [B站视频](https://www.bilibili.com/video/BV1ar4y1J77T) [Colab Demo](https://colab.research.google.com/drive/1vkCEiDaOAb7TCfIErAekEikiN1Ll9IaS?usp=sharing) for DenseNet [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1vkCEiDaOAb7TCfIErAekEikiN1Ll9IaS?usp=sharing) 22 | - [Pytorch CIFAR10图像分类 MobieNetv1篇](https://redamancy.blog.csdn.net/article/details/124636103) [Colab Demo](https://colab.research.google.com/drive/1r2umC8IoWfM5Qk0P8yCaUT6lyNKevMkK?usp=sharing) for MobileNetv1 [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1r2umC8IoWfM5Qk0P8yCaUT6lyNKevMkK?usp=sharing) 23 | - [Pytorch CIFAR10图像分类 ResNeXt篇](https://redamancy.blog.csdn.net/article/details/126655797) [Colab Demo](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) for ResNeXt [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1vkCEiDaOAb7TCfIErAekEikiN1Ll9IaS?usp=sharing) 24 | - [Pytorch CIFAR10 图像分类 Vision Transformer篇](https://redamancy.blog.csdn.net/article/details/126751948) for ViT 25 | - 26 | 27 | 除此之外，所有的模型权重都在release之中，可以选择相对应的权重文件进行下载[模型权重](https://github.com/Dreaming-future/Pytorch-Image-Classification/releases/tag/v1.0.0) 28 | 29 | - [Transer Learning](https://redamancy.blog.csdn.net/article/details/120213598) [Colab Demo](https://colab.research.google.com/drive/1j7rg9eDbWnn8KJQvMVEaOgDiLUJqFxRh?usp=sharing)[![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1j7rg9eDbWnn8KJQvMVEaOgDiLUJqFxRh?usp=sharing) 30 | 31 | 数据集也可以从[release](https://github.com/Dreaming-future/Pytorch-Image-Classification/releases/tag/v1.1.0)中获取 32 | 33 | 对于无法上github的同学，我们还可以通过Gitee来下载我们的代码和结果 34 | 35 | https://github.com/Dreaming-future/Pytorch-Image-Classification/releases/tag/v1.0.0 36 | 37 | ## 📅 Comming soon 更新计划 38 | 39 | - [x] LetNet 40 | - [x] AlexNet 41 | - [x] VGG 42 | - [x] ResNet 43 | - [x] GoogLeNet 44 | - [x] DenseNet 45 | - [x] ResNeXt Model 46 | - [x] MobileNetv1 47 | - [x] MobileNetv2 48 | - [x] ShuffleNetv1 49 | - [x] ShuffleNetv2 50 | - [x] ZFNet 51 | - [x] SeNet 52 | - [x] Efficiententv1 53 | - [x] ViT 54 | - [x] Swin-Transformer 55 | - [ ] ConvNeXt 56 | 57 | --- 58 | 59 | ## 🧰 使用方法 60 | 61 | 下载`CIFAR10`里所有文件，直接运行ipynb即可，由于我是利用一个工具函数进行训练的，所以**切记utils.py是必不可少的。** 62 | 63 | 运行ipynb文件即可，对于网络的py文件会持续更新，之后会利用一个函数来选取对应的网络进行训练得到结果。 64 | 65 | --- 66 | 67 | ## 📚 参考 68 | 69 | 除此之外，我还为图像分类这个专栏录了一下我的视频讲解，感兴趣的小伙伴可以来我的B站看视频进行学习，啃代码的时候，可能听一下也会有更多的感触哦 70 | [https://space.bilibili.com/241286257/channel/seriesdetail?sid=2075039](https://space.bilibili.com/241286257/channel/seriesdetail?sid=2075039) 71 | 72 | --- 73 | 74 | 最后这个是我写的一个pytorch的基础的介绍，[Pytorch Note 快乐星球](https://blog.csdn.net/weixin_45508265/article/details/117809512)，从0开始的完整的介绍pytorch和pytorch的简单语法，并且里面有一些项目和学习，还是很不错的哦，可以查看，除此之外，有什么想法可以加我wx: `pikachu2biubiu`聊哦，需要什么帮助也可以付费聊咨询。 75 | 76 | ![二维码](../QR.png) 77 | 78 | 79 | 80 | 81 | 82 | -------------------------------------------------------------------------------- /markdown/Pytorch CIFAR10图像分类 工具函数utils更新v2.0篇.md: -------------------------------------------------------------------------------- 1 | # Pytorch CIFAR10图像分类 工具函数utils更新v2.0篇 2 | 3 | [toc] 4 | 5 | 这里贴一下汇总篇：[汇总篇](https://blog.csdn.net/weixin_45508265/article/details/119285255) 6 | 7 | 对于上一版的工具函数utils.py，我认为可能来说，可视化的感觉还是不是很好，所以我就修改了一下我新的训练函数，为了兼容，参数基本相同，但是加入了tqdm来可视化进度条，这样也会更加的好看和直观，并且统一了一些代码的格式，使得代码稍微好看点，之前有时候有点乱，除此为了兼容一些情况，修改了部分代码，但是意义相同。 8 | 9 | 如果想看上一版的工具函数utils.py，可以查看这篇博客。[PytorchCIFAR10图像分类 工具函数utils篇](https://redamancy.blog.csdn.net/article/details/121589217) 10 | 11 | 12 | 13 | ## 设置随机种子（保证可复现性） 14 | 15 | 为了保证训练结果的可复现性，同时加了设置随机种子的代码 16 | 17 | ```python 18 | def setup_seed(seed): 19 | np.random.seed(seed) 20 | random.seed(seed) 21 | torch.manual_seed(seed) 22 | torch.cuda.manual_seed(seed) 23 | torch.cuda.manual_seed_all(seed) 24 | # torch.backends.cudnn.benchmark = True #for accelerating the running 25 | ``` 26 | 27 | 28 | 29 | ## 得到分类的精度（适配高版本） 30 | 31 | 首先对于这个函数来说，我变化了一下最后一行的 32 | 33 | ```python 34 | return correct / total 35 | ``` 36 | 37 | 因为这一行在多数时候是正确的，但是在一些版本的torch中是不兼容的，这时候需要使用torch.div或者torch.true_div 38 | 39 | ```python 40 | def get_acc(outputs, label): 41 | total = outputs.shape[0] 42 | probs, pred_y = outputs.data.max(dim=1) # 得到概率 43 | correct = (pred_y == label).sum().data 44 | return torch.div(correct, total) 45 | ``` 46 | 47 | 48 | 49 | ## 训练函数最新版本（增加tensorboard函数和tqdm可视化进度条） 50 | 51 | 实际上，为了保证版本的兼容，所以参数都是一样的，不同的地方是，我利用了tqdm来可视化我们的进度条 52 | 53 | 这样以后，我们能更好的看到自己的训练进度和时间，并且在训练过程中就可以看到自己的结果，这样就更好的保证了可视化，在训练过程中也可以看到准确率和损失的变化。 54 | 55 | **参数介绍** 56 | 57 | - **net : 所选的网络模型** 58 | - **trainloader： 训练集加载器** 59 | - **testloader： 测试集加载器** 60 | - **epoches：训练次数** 61 | - **optimizer：优化器** 62 | - **criterion：损失函数** 63 | - **writer：是否使用tensorboard可视化，默认为None** 64 | - **verbose：是否使用记录准确率，损失值，学习率，默认为True** 65 | - **scheduler：学习率调整策略** 66 | - **path：保存迭代次数中最优的模型的权重，默认为model.pth** 67 | 68 | ```python 69 | def train(net, trainloader, testloader, epochs, optimizer , criterion, scheduler , path = './model.pth', writer = None ,verbose = False): 70 | device = 'cuda' if torch.cuda.is_available() else 'cpu' 71 | best_acc = 0 72 | train_acc_list, test_acc_list = [],[] 73 | train_loss_list, test_loss_list = [],[] 74 | lr_list = [] 75 | for i in range(epochs): 76 | train_loss = 0 77 | train_acc = 0 78 | test_loss = 0 79 | test_acc = 0 80 | if torch.cuda.is_available(): 81 | net = net.to(device) 82 | net.train() 83 | train_step = len(trainloader) 84 | with tqdm(total=train_step,desc=f'Train Epoch {i + 1}/{epochs}',postfix=dict,mininterval=0.3) as pbar: 85 | for step,data in enumerate(trainloader,start=0): 86 | im,label = data 87 | im = im.to(device) 88 | label = label.to(device) 89 | 90 | optimizer.zero_grad() 91 | # 释放内存 92 | if hasattr(torch.cuda, 'empty_cache'): 93 | torch.cuda.empty_cache() 94 | # formard 95 | outputs = net(im) 96 | loss = criterion(outputs,label) 97 | # backward 98 | loss.backward() 99 | # 更新参数 100 | optimizer.step() 101 | # 累计损失 102 | train_loss += loss.item() 103 | train_acc += get_acc(outputs,label).item() 104 | pbar.set_postfix(**{'Train Acc' : train_acc/(step+1), 105 | 'Train Loss' :train_loss/(step+1)}) 106 | pbar.update(1) 107 | pbar.close() 108 | train_loss = train_loss / len(trainloader) 109 | train_acc = train_acc * 100 / len(trainloader) 110 | if verbose: 111 | train_acc_list.append(train_acc) 112 | train_loss_list.append(train_loss) 113 | # 记录学习率 114 | lr = optimizer.param_groups[0]['lr'] 115 | if verbose: 116 | lr_list.append(lr) 117 | # 更新学习率 118 | scheduler.step(train_loss) 119 | if testloader is not None: 120 | net.eval() 121 | test_step = len(testloader) 122 | with torch.no_grad(): 123 | with tqdm(total=test_step,desc=f'Test Epoch {i + 1}/{epochs}',postfix=dict,mininterval=0.3) as pbar: 124 | for step,data in enumerate(testloader,start=0): 125 | im,label = data 126 | im = im.to(device) 127 | label = label.to(device) 128 | # 释放内存 129 | if hasattr(torch.cuda, 'empty_cache'): 130 | torch.cuda.empty_cache() 131 | outputs = net(im) 132 | loss = criterion(outputs,label) 133 | test_loss += loss.item() 134 | test_acc += get_acc(outputs,label).item() 135 | pbar.set_postfix(**{'Test Acc' : test_acc/(step+1), 136 | 'Test Loss' :test_loss/(step+1)}) 137 | pbar.update(1) 138 | pbar.close() 139 | test_loss = test_loss / len(testloader) 140 | test_acc = test_acc * 100 / len(testloader) 141 | if verbose: 142 | test_loss_list.append(test_loss) 143 | test_acc_list.append(test_acc) 144 | print( 145 | 'Epoch [{:>3d}/{:>3d}] Train Loss:{:>.6f} Train Acc:{:>3.2f}% Test Loss:{:>.6f} Test Acc:{:>3.2f}% Learning Rate:{:>.6f}'.format( 146 | i + 1, epochs, train_loss, train_acc, test_loss, test_acc,lr)) 147 | else: 148 | print('Epoch [{:>3d}/{:>3d}] Train Loss:{:>.6f} Train Acc:{:>3.2f}% Learning Rate:{:>.6f}'.format(i+1,epochs,train_loss,train_acc,lr)) 149 | 150 | # ====================== 使用 tensorboard ================== 151 | if writer is not None: 152 | writer.add_scalars('Loss', {'train': train_loss, 153 | 'test': test_loss}, i+1) 154 | writer.add_scalars('Acc', {'train': train_acc , 155 | 'test': test_acc}, i+1) 156 | writer.add_scalar('Learning Rate',lr,i+1) 157 | # ========================================================= 158 | # 如果取得更好的准确率，就保存模型 159 | if test_acc > best_acc: 160 | torch.save(net,path) 161 | best_acc = test_acc 162 | Acc = {} 163 | Loss = {} 164 | Acc['train_acc'] = train_acc_list 165 | Acc['test_acc'] = test_acc_list 166 | Loss['train_loss'] = train_loss_list 167 | Loss['test_loss'] = test_loss_list 168 | Lr = lr_list 169 | return Acc, Loss, Lr 170 | ``` 171 | 172 | 173 | 174 | ## 可视化准确率、损失、学习率变化 175 | 176 | 这一部分用原来的，几乎没有什么变化，就更新了一下图片的大小 177 | 178 | 以下函数可以可视化准确率、损失、学习率随着迭代次数的变化 179 | 180 | ```python 181 | def plot_history(epochs, Acc, Loss, lr): 182 | plt.rcParams['figure.figsize'] = (12.0, 8.0) # set default size of plots 183 | plt.rcParams['image.interpolation'] = 'nearest' 184 | plt.rcParams['image.cmap'] = 'gray' 185 | 186 | epoch_list = range(1,epochs + 1) 187 | plt.plot(epoch_list, Loss['train_loss']) 188 | plt.plot(epoch_list, Loss['test_loss']) 189 | plt.xlabel('epoch') 190 | plt.ylabel('Loss Value') 191 | plt.legend(['train', 'test'], loc='upper left') 192 | plt.show() 193 | 194 | plt.plot(epoch_list, Acc['train_acc']) 195 | plt.plot(epoch_list, Acc['test_acc']) 196 | plt.xlabel('epoch') 197 | plt.ylabel('Acc Value') 198 | plt.legend(['train', 'test'], loc='upper left') 199 | plt.show() 200 | 201 | plt.plot(epoch_list, lr) 202 | plt.xlabel('epoch') 203 | plt.ylabel('Train LR') 204 | plt.show() 205 | ``` 206 | 207 | -------------------------------------------------------------------------------- /markdown/Pytorch CIFAR10图像分类 数据加载篇.md: -------------------------------------------------------------------------------- 1 | # Pytorch CIFAR10图像分类 数据加载与可视化篇 2 | 3 | [toc] 4 | 5 | 这里贴一下汇总篇：[汇总篇](https://blog.csdn.net/weixin_45508265/article/details/119285255) 6 | 7 | **Pytorch一般有以下几个流程** 8 | 9 | 1. 数据读取 10 | 2. 数据处理 11 | 3. 搭建网络 12 | 4. 模型训练 13 | 5. 模型上线 14 | 15 | 这里会先讲一下关于CIFAR10的数据加载和图片可视化，之后的模型篇会对网络进行介绍和实现。 16 | 17 | ### 1.数据读取 18 | 19 | CIFAR-10 是由 Hinton 的学生 Alex Krizhevsky 和 Ilya Sutskever 整理的一个用于识别普适物体的小型数据集。一共包含 10 个类别的 RGB 彩色图 片：飞机（ arplane ）、汽车（ automobile ）、鸟类（ bird ）、猫（ cat ）、鹿（ deer ）、狗（ dog ）、蛙类（ frog ）、马（ horse ）、船（ ship ）和卡车（ truck ）。图片的尺寸为 32×32 ，数据集中一共有 50000 张训练圄片和 10000 张测试图片。 20 | 21 | 与 MNIST 数据集中目比， CIFAR-10 具有以下不同点： 22 | 23 | - CIFAR-10 是 3 通道的彩色 RGB 图像，而 MNIST 是灰度图像。 24 | - CIFAR-10 的图片尺寸为 32×32， 而 MNIST 的图片尺寸为 28×28，比 MNIST 稍大。 25 | - 相比于手写字符， CIFAR-10 含有的是现实世界中真实的物体，不仅噪声很大，而且物体的比例、 特征都不尽相同，这为识别带来很大困难。 26 | 27 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/16f85f24a70e452e8659a1874616420f.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NTUwODI2NQ==,size_16,color_FFFFFF,t_70#pic_center) 28 | 29 | 30 | 31 | 首先使用`torchvision`加载和归一化我们的训练数据和测试数据。 32 | 33 | a、`torchvision`这个东西，实现了常用的一些深度学习的相关的图像数据的加载功能，比如cifar10、Imagenet、Mnist等等的，保存在`torchvision.datasets`模块中。 34 | 35 | b、同时，也封装了一些处理数据的方法。保存在`torchvision.transforms`模块中 36 | 37 | c、还封装了一些模型和工具封装在相应模型中,比如`torchvision.models`当中就包含了AlexNet，VGG，ResNet，SqueezeNet等模型。 38 | 39 | 40 | 41 | **由于torchvision的datasets的输出是[0,1]的PILImage，所以我们先先归一化为[-1,1]的Tensor** 42 | 43 | 首先定义了一个变换transform，利用的是上面提到的transforms模块中的Compose( )把多个变换组合在一起，可以看到这里面组合了ToTensor和Normalize这两个变换 44 | 45 | `transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))`前面的（0.5，0.5，0.5） 是 R G B 三个通道上的均值， 后面(0.5, 0.5, 0.5)是三个通道的标准差，注意通道顺序是 R G B ，用过opencv的同学应该知道openCV读出来的图像是 BRG顺序。这两个tuple数据是用来对RGB 图像做归一化的，如其名称 Normalize 所示这里都取0.5只是一个近似的操作，实际上其均值和方差并不是这么多，但是就这个示例而言 影响可不计。精确值是通过分别计算R,G,B三个通道的数据算出来的。 46 | 47 | ```python 48 | transform = transforms.Compose([ 49 | # transforms.CenterCrop(224), 50 | transforms.RandomCrop(32,padding=4), # 数据增广 51 | transforms.RandomHorizontalFlip(), # 数据增广 52 | transforms.ToTensor(), 53 | transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)) 54 | ]) 55 | ``` 56 | 57 | `trainloader`其实是一个比较重要的东西，我们后面就是通过`trainloader`把数据传入网络，当然这里的`trainloader`其实是个变量名，可以随便取，重点是他是由后面的`torch.utils.data.DataLoader()`定义的，这个东西来源于`torch.utils.data`模块 58 | 59 | ```python 60 | Batch_Size = 256 61 | ``` 62 | 63 | ```python 64 | trainset = datasets.CIFAR10(root='./data', train=True,download=True, transform=transform) 65 | testset = datasets.CIFAR10(root='./data',train=False,download=True,transform=transform) 66 | trainloader = torch.utils.data.DataLoader(trainset, batch_size=Batch_Size,shuffle=True, num_workers=2) 67 | testloader = torch.utils.data.DataLoader(testset, batch_size=Batch_Size,shuffle=True, num_workers=2) 68 | classes = ('plane', 'car', 'bird', 'cat','deer', 'dog', 'frog', 'horse', 'ship', 'truck') 69 | ``` 70 | 71 | > ```python 72 | > Files already downloaded and verified 73 | > Files already downloaded and verified 74 | > ``` 75 | 76 | ### 2. 查看数据（格式，大小，形状） 77 | 78 | 首先可以查看类别 79 | 80 | ```python 81 | classes = trainset.classes 82 | classes 83 | ``` 84 | 85 | > ```python 86 | > ['airplane', 87 | > 'automobile', 88 | > 'bird', 89 | > 'cat', 90 | > 'deer', 91 | > 'dog', 92 | > 'frog', 93 | > 'horse', 94 | > 'ship', 95 | > 'truck'] 96 | > ``` 97 | 98 | ```python 99 | trainset.class_to_idx 100 | ``` 101 | 102 | > ```python 103 | > {'airplane': 0, 104 | > 'automobile': 1, 105 | > 'bird': 2, 106 | > 'cat': 3, 107 | > 'deer': 4, 108 | > 'dog': 5, 109 | > 'frog': 6, 110 | > 'horse': 7, 111 | > 'ship': 8, 112 | > 'truck': 9} 113 | > ``` 114 | 115 | 也可以查看一下训练集的数据 116 | 117 | ```python 118 | trainset.data.shape #50000是图片数量，32x32是图片大小，3是通道数量RGB 119 | ``` 120 | 121 | > ```python 122 | > (50000, 32, 32, 3) 123 | > ``` 124 | 125 | 查看数据类型 126 | 127 | ```python 128 | #查看数据类型 129 | print(type(trainset.data)) 130 | print(type(trainset)) 131 | ``` 132 | 133 | > ```python 134 | > 135 | > 136 | > ``` 137 | > 138 | > 139 | 140 | **总结：** 141 | 142 | `trainset.data.shape`是标准的numpy.ndarray类型，其中50000是图片数量，32x32是图片大小，3是通道数量RGB； 143 | `trainset`是标准的？？类型，其中50000为图片数量，0表示取前面的数据，2表示3通道数RGB，32*32表示图片大小 144 | 145 | ### 3. 查看图片 146 | 147 | 接下来我们对图片进行可视化 148 | 149 | ```python 150 | import numpy as np 151 | import matplotlib.pyplot as plt 152 | plt.imshow(trainset.data[0]) 153 | im,label = iter(trainloader).next() 154 | ``` 155 | 156 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/d2733fe9714446caa0f6ff0d8501adcd.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NTUwODI2NQ==,size_16,color_FFFFFF,t_70) 157 | 158 | #### np.ndarray转为torch.Tensor 159 | 160 | 在深度学习中，原始图像需要转换为深度学习框架自定义的数据格式，在pytorch中，需要转为`torch.Tensor`。 161 | pytorch提供了`torch.Tensor` 与`numpy.ndarray`转换为接口： 162 | 163 | | 方法名 | 作用 | 164 | | ----------------------- | ------------------------------- | 165 | | `torch.from_numpy(xxx)` | `numpy.ndarray`转为torch.Tensor | 166 | | `tensor1.numpy()` | 获取tensor1对象的numpy格式数据 | 167 | 168 | `torch.Tensor` 高维矩阵的表示： N x C x H x W 169 | 170 | `numpy.ndarray` 高维矩阵的表示：N x H x W x C 171 | 172 | 因此在两者转换的时候需要使用`numpy.transpose( )` 方法 。 173 | 174 | ```python 175 | def imshow(img): 176 | img = img / 2 + 0.5 177 | img = np.transpose(img.numpy(),(1,2,0)) 178 | plt.imshow(img) 179 | ``` 180 | 181 | ```python 182 | imshow(im[0]) 183 | ``` 184 | 185 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/bc61a5f1af4b45f696e2a6d8bfc7b223.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NTUwODI2NQ==,size_16,color_FFFFFF,t_70) 186 | 187 | 我们也可以批量可视化图片，不过这里需要用到`make_grid` 188 | 189 | ```python 190 | plt.figure(figsize=(8,12)) 191 | imshow(torchvision.utils.make_grid(im[:32])) 192 | ``` 193 | 194 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/b9beb6f0fe4b4fcb9ef26f300bbb242b.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NTUwODI2NQ==,size_16,color_FFFFFF,t_70) -------------------------------------------------------------------------------- /markdown/Pytorch&Keras CIFAR10图像分类.md: -------------------------------------------------------------------------------- 1 | # ✨Pytorch&Keras CIFAR10图像分类 2 | 3 | 4 | 5 | - [✨Pytorch&Keras CIFAR10图像分类](#pytorchkeras-cifar10图像分类) 6 | - [🧑‍🎓适用人群](#‍适用人群) 7 | - [📚︎博客汇总](#︎博客汇总) 8 | - [💻︎ Pytorch CIFAR10 图像分类](#︎-pytorch-cifar10-图像分类) 9 | - [⁉ 数据处理以及工具函数（网络篇前必看）](#⁉-数据处理以及工具函数网络篇前必看) 10 | - [❕ 网络篇](#-网络篇) 11 | - [🖥︎ Keras CIFAR10 图像分类](#🖥︎-keras-cifar10-图像分类) 12 | - [💝 有趣的项目和尝试](#-有趣的项目和尝试) 13 | - [📅 Comming soon 更新计划](#-comming-soon-更新计划) 14 | - [🧰 使用方法](#🧰-使用方法) 15 | - [📚 参考](#-参考) 16 | 17 | 18 | ![Keras vs PyTorch vs Caffe - Comparing the Implementation of CNN](https://149695847.v2.pressablecdn.com/wp-content/uploads/2020/08/create-machine-learning-and-deep-learning-models-using-pytorch-and-tensorflow.jpg#pic_center) 19 | 20 | ## 💪专栏介绍 21 | 22 | 一开始写这个专栏的初衷是，**有时候有些代码找的太繁琐了，并且找到了还运行不出来，或者运行了，但是没有一个很好的理解**，所以基于此，我写了这个CIFAR10图像分类的专栏，借此希望，大家都能通过我这个专栏，找到自己想要的模型或者学习深度学习的代码。 23 | 24 | 由于现在深度学习也在热度中，很多时候我们难免需要遇见深度学习的时候，**在一些课程和项目的要求下，我们会发现，一个好的代码和一个好的可视化和清晰解释是能够节省特别特别多的时间的**，基于此，我写出了这个专栏，这个专栏下的所有项目，都是**可运行无差错的。**如果遇见了问题，也可以留言跟我沟通 25 | 26 | --- 27 | 28 | ## 🧑‍🎓适用人群 29 | 30 | 他很适合大家**初入深度学习或者是Pytorch和Keras**，希望这能够帮助初学深度学习的同学一个入门Pytorch或者Keras的项目和在这之中更加了解Pytorch&Keras和各个图像分类的模型。 31 | 32 | 他有比较清晰的可视化结构和架构，除此之外，我是**用jupyter写的，所以说在文章整体架构可以说是非常清晰**，可以帮助你快速学习到**各个模块的知识**，而不是通过python脚本一行一行的看，这样的方式是符合初学者的。 33 | 34 | 除此之外，如果你需要变成脚本形式，也是很简单的。 35 | 36 | --- 37 | 38 | ## 📚︎博客汇总 39 | 40 | 为了使得文章体系结构更加清晰，这里给出**Pytorch&Keras对于CIFAR10图像分类的所有资料汇总**，也就是我的博客汇总，也告诉大家，我做了什么工作，如果大家有兴趣订阅我的专栏亦或者是有什么其他模型的想法，也可以评论留言，我也可以进行去学习的。 41 | 42 | 这部分也方便大家看到介绍，并且快速找到自己所需要的代码进行学习和了解 43 | 44 | --- 45 | 46 | ### 💻︎ Pytorch CIFAR10 图像分类 47 | 48 | 在看网络篇的时候，可以先看前三个，前三个说明的如何加载数据包括数据的预处理以及进行可视化，工具函数篇介绍了如何构建训练的函数，并且有时候会遇到一部分utils.py的错误，在这里面都有很好的解释和进行学习。 49 | 50 | #### ⁉ 数据处理以及工具函数（网络篇前必看） 51 | 52 | - [Pytorch CIFAR10图像分类 数据加载与可视化篇](https://blog.csdn.net/weixin_45508265/article/details/119285113) 53 | - [Pytorch CIFAR10图像分类 工具函数utils篇](https://redamancy.blog.csdn.net/article/details/121589217) 54 | - [Pytorch CIFAR10图像分类 工具函数utils更新v2.0篇](https://redamancy.blog.csdn.net/article/details/127856569) 55 | 56 | #### ❕ 网络篇 57 | 58 | - [Pytorch CIFAR10图像分类 自定义网络篇](https://blog.csdn.net/weixin_45508265/article/details/119305277) 59 | - [Pytorch CIFAR10图像分类 LeNet5篇](https://blog.csdn.net/weixin_45508265/article/details/119305673) 60 | - [Pytorch CIFAR10图像分类 AlexNet篇](https://blog.csdn.net/weixin_45508265/article/details/119305848) 61 | - [Pytorch CIFAR10图像分类 VGG篇](https://blog.csdn.net/weixin_45508265/article/details/119332904) 62 | - [Pytorch CIFAR10图像分类 GoogLeNet篇](https://blog.csdn.net/weixin_45508265/article/details/119399239) 63 | - [Pytorch CIFAR10图像分类 ResNet篇](https://blog.csdn.net/weixin_45508265/article/details/119532143) 64 | - [Pytorch CIFAR10图像分类 DenseNet篇](https://blog.csdn.net/weixin_45508265/article/details/119648036) 65 | 66 | - [Pytorch CIFAR10图像分类 MobieNetv1篇](https://redamancy.blog.csdn.net/article/details/124636103) 67 | - [Pytorch CIFAR10图像分类 MobileNetv2篇](https://redamancy.blog.csdn.net/article/details/127946431) 68 | - [Pytorch CIFAR10图像分类 ResNeXt篇](https://redamancy.blog.csdn.net/article/details/126655797) 69 | - [Pytorch CIFAR10图像分类 ZFNet篇](https://blog.csdn.net/weixin_45508265/article/details/128560595) 70 | - [Pytorch CIFAR10图像分类 SENet篇](https://blog.csdn.net/weixin_45508265/article/details/130938341) 71 | - [Pytorch CIFAR10 图像分类 Vision Transformer篇](https://redamancy.blog.csdn.net/article/details/126751948) 72 | - [Pytorch CIFAR10图像分类 EfficientNet篇](https://blog.csdn.net/weixin_45508265/article/details/128585354) 73 | 74 | > 具体的详情可以关注[Pytorch CIFAR10图像分类汇总篇](https://redamancy.blog.csdn.net/article/details/119285255) 75 | 76 | --- 77 | 78 | ### 🖥︎ Keras CIFAR10 图像分类 79 | 80 | - [Keras CIFAR-10分类 SVM 分类器篇][1] 81 | - [Keras CIFAR-10分类 LeNet-5篇][2] 82 | - [Keras CIFAR-10分类 AlexNet篇][3] 83 | - [Keras CIFAR-10分类 GoogleNet篇][4] 84 | - [Keras CIFAR-10分类 VGG篇][5] 85 | - [Keras CIFAR-10分类 ResNet篇][6] 86 | - [Keras CIFAR-10分类 DenseNet篇][7] 87 | 88 | > 具体的详情可以关注[Keras CIFAR-10 分类汇总篇](https://blog.csdn.net/weixin_45508265/article/details/127859003) 89 | 90 | --- 91 | 92 | ### 💝 有趣的项目和尝试 93 | 94 | - [MAE实现及预训练可视化 （CIFAR-Pytorch）][MAE] 95 | 96 | --- 97 | 98 | ## 📅 Comming soon 更新计划 99 | 100 | - [x] LetNet 101 | - [x] AlexNet 102 | - [x] VGG 103 | - [x] ResNet 104 | - [x] GoogLeNet 105 | - [x] DenseNet 106 | - [x] ResNeXt 107 | - [x] MobileNetv1 108 | - [x] MobileNetv2 109 | - [x] ZFNet 110 | - [x] SeNet 111 | - [x] Efficientent 112 | - [x] ViT 113 | - [ ] ShuffleNetv1 114 | - [ ] ShuffleNetv2 115 | - [ ] Swin-Transformer 116 | - [ ] ConvNeXt 117 | - [ ] ConvNeXtv2 118 | 119 | --- 120 | 121 | ## 🧰 使用方法 122 | 123 | 下载`CIFAR10`里所有文件，直接运行ipynb即可，由于我是利用一个工具函数进行训练的，所以**切记utils.py是必不可少的。** 124 | 125 | 运行ipynb文件即可，对于网络的py文件会持续更新，之后会利用一个函数来选取对应的网络进行训练得到结果。 126 | 127 | --- 128 | 129 | ## 📚 参考 130 | 131 | 除此之外，我还为图像分类这个专栏录了一下我的视频讲解，感兴趣的小伙伴可以来我的B站看视频进行学习，啃代码的时候，可能听一下也会有更多的感触哦 132 | [https://space.bilibili.com/241286257/channel/seriesdetail?sid=2075039](https://space.bilibili.com/241286257/channel/seriesdetail?sid=2075039) 133 | 134 | --- 135 | 136 | 最后这个是我写的一个pytorch的基础的介绍，[Pytorch Note 快乐星球](https://blog.csdn.net/weixin_45508265/article/details/117809512)，从0开始的完整的介绍pytorch和pytorch的简单语法，并且里面有一些项目和学习，还是很不错的哦，可以查看，除此之外，有什么想法可以加我wx: `pikachu2biubiu`聊哦，需要什么帮助也可以付费聊咨询。 137 | 138 | ![二维码](C:\Users\86137\Documents\GitHub\Pytorch-Image-Classification\QR.png) 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | 150 | [1]: https://redamancy.blog.csdn.net/article/details/126445778 151 | [2]: https://redamancy.blog.csdn.net/article/details/126446810 152 | [3]: https://redamancy.blog.csdn.net/article/details/126590621 153 | [4]: https://redamancy.blog.csdn.net/article/details/126591761 154 | [5]: https://redamancy.blog.csdn.net/article/details/126669709 155 | [6]: https://redamancy.blog.csdn.net/article/details/127827641 156 | [7]: https://redamancy.blog.csdn.net/article/details/127828318 157 | [MAE]: https://redamancy.blog.csdn.net/article/details/126863995 158 | 159 | -------------------------------------------------------------------------------- /markdown/README.md: -------------------------------------------------------------------------------- 1 | # Pytorch CIFAR10 图像分类篇 汇总 2 | 3 | 4 | 5 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/f37d635b66334021a12521d6bed88d87.png#pic_center) 6 | 7 | 接下来我会分别利用深度学习的方法，用Pytorch实现我们的CIFAR10的图像分类 8 | 9 | 大概预计的模型有LeNet，AlexNet，VGG，GoogLeNet，ResNet，DenseNet，MobileNet，Vision Transformer， ResNeXt等，除此之外也会陆续补充 10 | 11 | 希望这能够帮助初学机器学习的同学一个入门Pytorch的项目和在这之中更加了解Pytorch和各个图像分类的模型。 12 | 13 | - [Pytorch CIFAR10图像分类 数据加载与可视化篇](https://blog.csdn.net/weixin_45508265/article/details/119285113) [B站视频](https://www.bilibili.com/video/BV1FP4y1g7sc) 14 | - [Pytorch CIFAR10图像分类 工具函数utils篇](https://redamancy.blog.csdn.net/article/details/121589217) [Online Demo](https://drive.google.com/file/d/1HESbXuEb__9eXqq4tAl8owsb1FKhpO2i/view?usp=sharing) 15 | - [Pytorch CIFAR10图像分类 自定义网络篇](https://blog.csdn.net/weixin_45508265/article/details/119305277) [B站视频](https://www.bilibili.com/video/BV1MF41147gZ) [Colab Demo](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) for 自定义网络 [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) 16 | - [Pytorch CIFAR10图像分类 LeNet5篇](https://blog.csdn.net/weixin_45508265/article/details/119305673) [B站视频](https://www.bilibili.com/video/BV1FL411K7VJ) [Colab Demo](https://colab.research.google.com/drive/15B0HBssfRzQk8mJyYF-v5fwAdPtNqf3H?usp=sharing) for LeNet5 [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/15B0HBssfRzQk8mJyYF-v5fwAdPtNqf3H?usp=sharing) 17 | - [Pytorch CIFAR10图像分类 AlexNet篇](https://blog.csdn.net/weixin_45508265/article/details/119305848) [B站视频](https://www.bilibili.com/video/BV1xu411B75x) [Colab Demo](https://colab.research.google.com/drive/1d6CTYzyWeB03xiSlT8mzsZ_LtH9TlPvs?usp=sharing) for AlexNet [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1d6CTYzyWeB03xiSlT8mzsZ_LtH9TlPvs?usp=sharing) 18 | - [Pytorch CIFAR10图像分类 VGG篇](https://blog.csdn.net/weixin_45508265/article/details/119332904) [B站视频](https://www.bilibili.com/video/BV12L4y1u7WH) [Colab Demo](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) for VGG16 [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) 19 | - [Pytorch CIFAR10图像分类 GoogLeNet篇](https://blog.csdn.net/weixin_45508265/article/details/119399239) [B站视频](https://www.bilibili.com/video/BV1RS4y1274A) [Colab Demo](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) for GoogLeNet Inceptionv1 [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1o8lfWHvr4WoyTA5Y9b4mSCSw2TEbXJb7?usp=sharing) 20 | - [Pytorch CIFAR10图像分类 ResNet篇](https://blog.csdn.net/weixin_45508265/article/details/119532143) [B站视频](https://www.bilibili.com/video/BV1Wu411v72u) [Colab Demo](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) for ResNet [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1W6d-eTY89bvGEL_QoMq4kw7m0dP9lHkS?usp=sharing) 21 | - [Pytorch CIFAR10图像分类 DenseNet篇](https://blog.csdn.net/weixin_45508265/article/details/119648036) [B站视频](https://www.bilibili.com/video/BV1ar4y1J77T) [Colab Demo](https://colab.research.google.com/drive/1vkCEiDaOAb7TCfIErAekEikiN1Ll9IaS?usp=sharing) for DenseNet [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1vkCEiDaOAb7TCfIErAekEikiN1Ll9IaS?usp=sharing) 22 | - [Pytorch CIFAR10图像分类 MobieNetv1篇](https://redamancy.blog.csdn.net/article/details/124636103) [Colab Demo](https://colab.research.google.com/drive/1r2umC8IoWfM5Qk0P8yCaUT6lyNKevMkK?usp=sharing) for MobileNetv1 [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1r2umC8IoWfM5Qk0P8yCaUT6lyNKevMkK?usp=sharing) 23 | - [Pytorch CIFAR10图像分类 ResNeXt篇](https://redamancy.blog.csdn.net/article/details/126655797) [Colab Demo](https://colab.research.google.com/drive/1BO0wSY3w3xma-oATLyIQRq19qjX1FxF7?usp=sharing) for ResNeXt [![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1vkCEiDaOAb7TCfIErAekEikiN1Ll9IaS?usp=sharing) 24 | - [Pytorch CIFAR10 图像分类 Vision Transformer篇](https://redamancy.blog.csdn.net/article/details/126751948) for ViT 25 | - 26 | 27 | 除此之外，所有的模型权重都在release之中，可以选择相对应的权重文件进行下载[模型权重](https://github.com/Dreaming-future/Pytorch-Image-Classification/releases/tag/v1.0.0) 28 | 29 | - [Transer Learning](https://redamancy.blog.csdn.net/article/details/120213598) [Colab Demo](https://colab.research.google.com/drive/1j7rg9eDbWnn8KJQvMVEaOgDiLUJqFxRh?usp=sharing)[![在这里插入图片描述](https://img-blog.csdnimg.cn/47fbca1712ba49719240c6dc3258ddc7.png)](https://colab.research.google.com/drive/1j7rg9eDbWnn8KJQvMVEaOgDiLUJqFxRh?usp=sharing) 30 | 31 | 数据集也可以从[release](https://github.com/Dreaming-future/Pytorch-Image-Classification/releases/tag/v1.1.0)中获取 32 | 33 | 对于无法上github的同学，我们还可以通过Gitee来下载我们的代码和结果 34 | 35 | https://github.com/Dreaming-future/Pytorch-Image-Classification/releases/tag/v1.0.0 36 | 37 | ## 📅 Comming soon 更新计划 38 | 39 | - [x] LetNet 40 | - [x] AlexNet 41 | - [x] VGG 42 | - [x] ResNet 43 | - [x] GoogLeNet 44 | - [x] DenseNet 45 | - [x] ResNeXt Model 46 | - [x] MobileNetv1 47 | - [x] MobileNetv2 48 | - [ ] ShuffleNetv1 49 | - [ ] ShuffleNetv2 50 | - [ ] ZFNet 51 | - [ ] SeNet 52 | - [ ] Efficiententv1 53 | - [x] ViT 54 | - [ ] Swin-Transformer 55 | - [ ] ConvNeXt 56 | 57 | --- 58 | 59 | ## 🧰 使用方法 60 | 61 | 下载`CIFAR10`里所有文件，直接运行ipynb即可，由于我是利用一个工具函数进行训练的，所以**切记utils.py是必不可少的。** 62 | 63 | 运行ipynb文件即可，对于网络的py文件会持续更新，之后会利用一个函数来选取对应的网络进行训练得到结果。 64 | 65 | --- 66 | 67 | ## 📚 参考 68 | 69 | 除此之外，我还为图像分类这个专栏录了一下我的视频讲解，感兴趣的小伙伴可以来我的B站看视频进行学习，啃代码的时候，可能听一下也会有更多的感触哦 70 | [https://space.bilibili.com/241286257/channel/seriesdetail?sid=2075039](https://space.bilibili.com/241286257/channel/seriesdetail?sid=2075039) 71 | 72 | --- 73 | 74 | 最后这个是我写的一个pytorch的基础的介绍，[Pytorch Note 快乐星球](https://blog.csdn.net/weixin_45508265/article/details/117809512)，从0开始的完整的介绍pytorch和pytorch的简单语法，并且里面有一些项目和学习，还是很不错的哦，可以查看，除此之外，有什么想法可以加我wx: `pikachu2biubiu`聊哦，需要什么帮助也可以付费聊咨询。 75 | 76 | ![二维码](../QR.png) 77 | 78 | 79 | 80 | 81 | 82 | -------------------------------------------------------------------------------- /markdown/img/AlexNet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/AlexNet.png -------------------------------------------------------------------------------- /markdown/img/BasicBlock.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/BasicBlock.png -------------------------------------------------------------------------------- /markdown/img/Bottleneck.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/Bottleneck.png -------------------------------------------------------------------------------- /markdown/img/DenseNet_Explained.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/DenseNet_Explained.png -------------------------------------------------------------------------------- /markdown/img/Dense_bottleneck.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/Dense_bottleneck.png -------------------------------------------------------------------------------- /markdown/img/GooLeNet2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/GooLeNet2.png -------------------------------------------------------------------------------- /markdown/img/Inception.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/Inception.jpg -------------------------------------------------------------------------------- /markdown/img/Inception.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/Inception.png -------------------------------------------------------------------------------- /markdown/img/Inception2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/Inception2.png -------------------------------------------------------------------------------- /markdown/img/Inception3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/Inception3.png -------------------------------------------------------------------------------- /markdown/img/Inceptionv2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/Inceptionv2.png -------------------------------------------------------------------------------- /markdown/img/LeNet5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/LeNet5.png -------------------------------------------------------------------------------- /markdown/img/Original-ResNet-18-Architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/Original-ResNet-18-Architecture.png -------------------------------------------------------------------------------- /markdown/img/ResNet-34.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/ResNet-34.png -------------------------------------------------------------------------------- /markdown/img/Residual3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/Residual3.png -------------------------------------------------------------------------------- /markdown/img/cifar.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/cifar.png -------------------------------------------------------------------------------- /markdown/img/dense1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/dense1.png -------------------------------------------------------------------------------- /markdown/img/dense2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/dense2.png -------------------------------------------------------------------------------- /markdown/img/dense3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/dense3.png -------------------------------------------------------------------------------- /markdown/img/dense_cifar.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/dense_cifar.jpg -------------------------------------------------------------------------------- /markdown/img/densenet.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/densenet.gif -------------------------------------------------------------------------------- /markdown/img/densenet_table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/densenet_table.png -------------------------------------------------------------------------------- /markdown/img/gooLenet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/gooLenet.png -------------------------------------------------------------------------------- /markdown/img/residual.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/residual.png -------------------------------------------------------------------------------- /markdown/img/residual2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/residual2.png -------------------------------------------------------------------------------- /markdown/img/resnet_table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/resnet_table.png -------------------------------------------------------------------------------- /markdown/img/vgg.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/vgg.png -------------------------------------------------------------------------------- /markdown/img/vgg19.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kedreamix/Pytorch-Image-Classification/47a9a1e7ddbe626eaffde72975057dc11ba3cab4/markdown/img/vgg19.png -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | torch 2 | torchvision 3 | torchsummary 4 | torchinfo 5 | numpy 6 | matplotlib 7 | requests 8 | pillow 9 | tqdm 10 | tensorboardX --------------------------------------------------------------------------------