├── LICENSE
├── README.md
├── chapter03_Python_image_classification
├── README.md
├── client.py
├── datasets.py
├── figures
│ ├── fig2.png
│ └── fig31.png
├── main.py
├── models.py
├── server.py
└── utils
│ └── conf.json
├── chapter05_FATE_HFL
├── README.md
├── breast_1_train.csv
├── breast_2_train.csv
├── breast_eval.csv
├── figures
│ ├── FATE_logo.jpg
│ └── local_2_dtable.png
├── split_dataset.py
├── test_homolr_train_job_conf.json
└── test_homolr_train_job_dsl.json
├── chapter06_FATE_VFL
├── README.md
├── figures
│ ├── FATE_logo.jpg
│ ├── fig1.png
│ ├── fig10.png
│ ├── fig11.png
│ ├── fig2.png
│ ├── fig3.png
│ ├── fig4.png
│ ├── fig5.png
│ ├── fig6.png
│ ├── fig7.png
│ ├── fig8.png
│ ├── fig9.png
│ └── local_2_dtable.png
├── housing_1_eval.csv
├── housing_1_train.csv
├── housing_2_eval.csv
├── housing_2_train.csv
├── split_eval_dataset.py
├── split_training_dataset.py
├── test_hetero_linr_train_eval_job_conf.json
├── test_hetero_linr_train_eval_job_dsl.json
├── test_hetero_linr_train_job_conf.json
└── test_hetero_linr_train_job_dsl.json
├── chapter09_Recommendation
├── README.md
└── figures
│ ├── cent_recom.png
│ ├── fig1.png
│ ├── fig10.png
│ ├── fig11.png
│ ├── fig12.png
│ ├── fig13.png
│ ├── fig14.png
│ ├── fig2.png
│ ├── fig3.png
│ ├── fig4.png
│ ├── fig5.png
│ ├── fig6.png
│ ├── fig7.png
│ ├── fig8.png
│ └── fig9.png
├── chapter10_Computer_Vision
├── Dataset_description.md
├── LICENSE
├── README.md
├── config
│ ├── coco.data
│ ├── create_custom_model.sh
│ ├── custom.data
│ ├── yolov3-custom-street.cfg
│ ├── yolov3-tiny.cfg
│ └── yolov3.cfg
├── data
│ ├── __init__.py
│ ├── custom
│ │ ├── classes.names
│ │ ├── train.txt
│ │ └── valid.txt
│ ├── data_utils.py
│ ├── dataset.py
│ ├── generate_task_json.py
│ ├── util.py
│ └── voc_dataset.py
├── experiments
│ └── log_formatter.py
├── figures
│ ├── FL_flow.png
│ ├── fig10.png
│ ├── loss.png
│ └── map.png
├── fl_client.py
├── fl_server.py
├── model
│ ├── __init__.py
│ ├── faster_rcnn.py
│ ├── faster_rcnn_trainer.py
│ ├── faster_rcnn_vgg16.py
│ ├── model_wrapper.py
│ ├── region_proposal_network.py
│ ├── roi_module.py
│ ├── utils
│ │ ├── __init__.py
│ │ ├── bbox_tools.py
│ │ ├── creator_tool.py
│ │ ├── nms
│ │ │ ├── __init__.py
│ │ │ ├── _nms_gpu_post.pyx
│ │ │ ├── _nms_gpu_post_py.py
│ │ │ ├── build.py
│ │ │ └── non_maximum_suppression.py
│ │ └── roi_cupy.py
│ └── yolo.py
├── requirements.txt
├── run.sh
├── run_server.sh
├── stop.sh
├── utils
│ ├── __init__.py
│ ├── array_tool.py
│ ├── augmentations.py
│ ├── config.py
│ ├── datasets.py
│ ├── eval_tool.py
│ ├── model_dump.py
│ ├── parse_config.py
│ ├── pvoc2coco.py
│ ├── utils.py
│ └── vis_tool.py
└── weights
│ └── download_weights.sh
├── chapter15_Attack_and_Defense
├── README.md
└── figures
│ ├── attack_defense.jpeg
│ └── summary.png
├── chapter15_Backdoor_Attack
├── README.md
├── client.py
├── datasets.py
├── images
│ ├── fl_backdoor.png
│ ├── normal_image.png
│ ├── normal_image_1.png
│ ├── poison_image.png
│ └── target.png
├── main.py
├── models.py
├── server.py
└── utils
│ └── conf.json
├── chapter15_Compression
├── README.md
├── client.py
├── datasets.py
├── figures
│ └── fig1.png
├── main.py
├── models.py
├── server.py
└── utils
│ └── conf.json
├── chapter15_Differential_Privacy
├── README.md
├── client.py
├── datasets.py
├── figures
│ ├── fig1.png
│ ├── fig13.png
│ ├── fig14.png
│ ├── fig2.png
│ └── fig3.png
├── main.py
├── models.py
├── server.py
└── utils
│ └── conf.json
├── chapter15_Homomorphic_Encryption
├── README.md
├── client.py
├── data
│ └── breast.csv
├── encoding.py
├── figures
│ ├── fig12.png
│ └── fig2.png
├── main.py
├── models.py
├── paillier.py
├── server.py
├── util.py
└── utils
│ └── conf.json
├── chapter15_Sparsity
├── README.md
├── client.py
├── datasets.py
├── figures
│ ├── fig1.png
│ └── fig33.png
├── main.py
├── models.py
├── server.py
└── utils
│ └── conf.json
├── errata
├── README.md
└── figures
│ ├── 3_1.png
│ ├── 3_2.png
│ └── 5_1.png
└── figures
├── cover.jpg
├── cover2.png
└── link.md
/README.md:
--------------------------------------------------------------------------------
1 | # 联邦学习实战 (Practicing-Federated-Learning)
2 |
3 | [](https://github.com/innovation-cat/Awesome-Federated-Machine-Learning) [](https://item.jd.com/13206070.html)
4 |
5 |
6 |
7 | 联邦学习是一种新型的、基于数据隐私保护技术实现的分布式计算范式，自提出以来，就受到学术界和工业界的广泛关注。近年来，随着联邦学习的飞速发展，使得其成为解决数据孤岛和用户隐私问题的首选方案，但当前市面上这方面的实战书籍却尚不多见。本书是第一本权威的联邦学习实战书籍，结合联邦学习案例，有助于读者更深入的理解联邦学习这一新兴的学科。
8 |
9 | **本项目将长期维护和更新《联邦学习实战》书籍对应的章节代码。书或代码中遇到的问题，可以邮件联系：huanganbu@gmail.com**
10 |
11 | 由于受到出版刊物限制，本书不能在纸质书页面上放置网络链接，本书的链接对应可查看这里（[书中链接](figures/link.md)）。
12 |
13 | 书中可能存在印刷或撰写的错误，勘误列表读者可点击：[勘误列表](errata/README.md)，同时也欢迎读者反馈书中的文字错误问题，以便我们改进。
14 |
15 |
16 |
17 | ## 联邦学习材料
18 |
19 | - [联邦学习最新研究论文、书籍、代码、视频等详细资料汇总 (Everything about Federated Learning)](https://github.com/innovation-cat/Awesome-Federated-Machine-Learning)
20 |
21 | - [香港科技大学“联邦学习”课程](https://ising.cse.ust.hk/fl/index.html)
22 |
23 |
24 |
25 | ## 简 介
26 |
27 | 本书是联邦学习系列书籍的第二本，共由五大部分共19章构成。既有理论知识的系统性总结，又有详细的案例分析，本书的组织结构如下：
28 |
29 | - 第一部分简要回顾联邦学习的理论知识点，包括联邦学习的分类、定义；联邦学习常见的安全机制等。
30 |
31 | - 第二部分介绍如何使用Python和FATE进行简单的联邦学习建模；同时，我们也对当前一些常见的联邦学习平台进行总结。
32 |
33 | - 第三部分是联邦学习的案例分析，本部分我们挑选了包括视觉、个性化推荐、金融保险、攻防、医疗等领域的案例，我们将探讨联邦学习如何在这些领域中进行应用和落地。
34 |
35 | - 第四部分主要介绍和联邦学习相关的高级知识点，包括联邦学习的架构和训练的加速方法；联邦学习与区块链、split learning、边缘计算的异同。
36 |
37 | - 第五部分是对本书的总结，以及对联邦学习技术的未来展望。
38 |
39 |
40 |
41 | 本书可以与《联邦学习》书籍配套使用，书的链接地址：
42 |
43 | - 联邦学习实战：[https://item.jd.com/13206070.html](https://item.jd.com/13206070.html)
44 |
45 | - 联邦学习：[https://item.jd.com/12649191.html](https://item.jd.com/12649191.html)
46 |
47 |
48 |
49 | 注：本项目含有部分数学公式，为了不影响阅读效果，在Github网页中能正常显示公式，需要读者首先下载相应的浏览器插件，如MathJax插件（[MathJax Plugin for Github](https://chrome.google.com/webstore/detail/mathjax-plugin-for-github/ioemnmodlmafdkllaclgeombjnmnbima)）
50 |
51 |
52 |
53 | ## 推荐语
54 |
55 | 本书的编写也得到了来自学术界、金融投资界和工业界的知名人士推荐，在此深表感谢：
56 |
57 | > "国务院在2020年将数据作为新型生产要素写入法律文件中，与土地、劳动力、资本、技术并列为五个生产要素。这意味着一方面，个人数据隐私将受到法律的严格保护；另一方面，数据与其它生产要素一样，可以进行开放、共享和交易。如何有效解决数据隐私与数据共享之间的矛盾成为了当前人工智能领域的研究热点问题。联邦学习作为一种新型的分布式机器学习训练和应用范式，从提出以来就备受广泛的关注，也被认为是当前产业界解决数据隐私与数据共享之间矛盾的一种有效方案。同时作为可信计算的新成员，书中特别提到，联邦学习还可以与区块链进行强强联合，例如借助区块链记录的不可篡改特性，帮助对联邦学习可能面临的恶意攻击进行追溯；借助区块链的共识机制和智能合约，对联邦学习创造的价值进行利益分配等。《联邦学习实战》一书，对联邦学习的理论和应用案例做了系统性的阐述和分析，相信能够为广大的科研工作者、企业开发人员提供有效的指导和参考。"
58 | > — 陈 纯，中国工程院院士
59 |
60 |
61 |
62 | > "人工智能时代的到来已经不可逆转，在算力算法和机器学习蓬勃发展的大背景下，数据资产成为了非常重要的技术资源，服务企业的长远发展。如何利用好、保护好数据资产是人工智能能否创造出更大经济和社会价值的关键因子。联邦学习理念的提出和发展，在这个方面探索出一条可行之路并做出了重要的示范。杨强教授是世界级人工智能研究专家，在学术和产业两端都对这一领域有非常深的造诣，希望他这本《联邦学习实战》可以为更多业内人士和机器学习的从业者与爱好者带来更多的启发与思考。"
63 | >
64 | > — 沈南鹏，红杉资本全球执行合伙人
65 |
66 |
67 |
68 | > "数据资产化是实现人工智能产业价值的核心环节，而联邦学习是其中的关键技术。书中严谨而深入浅出阐述为读者们提供了非常有效的工具。"
69 | >
70 | > — 陆 奇，奇绩创坛创始人
71 |
72 |
73 |
74 | > "为了互联网更好的未来，我们需要建立负责任的数据经济体系，使我们既能充分实现数据价值，又能很好的保护用户的数据隐私，并能够公平分配数据创造的价值。联邦学习正是支撑这一愿景的重要技术。本书描述了该领域的实际应用案例，为将联邦学习付诸实践提供了重要的指导意义。"
75 | >
76 | > — Dawn Song，美国加州大学伯克利分校教授
77 |
78 |
79 |
80 |
81 | ## 代码章节目录
82 |
83 | * [第3章：用Python从零实现横向联邦图像分类](chapter03_Python_image_classification)
84 | * [第5章：用FATE从零实现横向逻辑回归](chapter05_FATE_HFL)
85 | * [第6章：用FATE从零实现纵向线性回归](chapter06_FATE_VFL)
86 | * [第9章：个性化推荐案例实战](chapter09_Recommendation)
87 | * [第10章：视觉案例实战](chapter10_Computer_Vision)
88 | * [第15章：联邦学习攻防实战](chapter15_Attack_and_Defense)
89 | * [第15章：攻防实战 - 后门攻击](chapter15_Backdoor_Attack)
90 | * [第15章：攻防实战 - 差分隐私](chapter15_Differential_Privacy)
91 | * [第15章：攻防实战 - 同态加密](chapter15_Homomorphic_Encryption)
92 | * [第15章：攻防实战 - 稀疏化](chapter15_Sparsity)
93 | * [第15章：攻防实战 - 模型压缩](chapter15_Compression)
94 |
95 |
--------------------------------------------------------------------------------
/chapter03_Python_image_classification/client.py:
--------------------------------------------------------------------------------
1 |
2 | import models, torch, copy
3 | class Client(object):
4 |
5 | def __init__(self, conf, model, train_dataset, id = -1):
6 |
7 | self.conf = conf
8 |
9 | self.local_model = models.get_model(self.conf["model_name"])
10 |
11 | self.client_id = id
12 |
13 | self.train_dataset = train_dataset
14 |
15 | all_range = list(range(len(self.train_dataset)))
16 | data_len = int(len(self.train_dataset) / self.conf['no_models'])
17 | train_indices = all_range[id * data_len: (id + 1) * data_len]
18 |
19 | self.train_loader = torch.utils.data.DataLoader(self.train_dataset, batch_size=conf["batch_size"],
20 | sampler=torch.utils.data.sampler.SubsetRandomSampler(train_indices))
21 |
22 |
23 | def local_train(self, model):
24 |
25 | for name, param in model.state_dict().items():
26 | self.local_model.state_dict()[name].copy_(param.clone())
27 |
28 | #print(id(model))
29 | optimizer = torch.optim.SGD(self.local_model.parameters(), lr=self.conf['lr'],
30 | momentum=self.conf['momentum'])
31 | #print(id(self.local_model))
32 | self.local_model.train()
33 | for e in range(self.conf["local_epochs"]):
34 |
35 | for batch_id, batch in enumerate(self.train_loader):
36 | data, target = batch
37 |
38 | if torch.cuda.is_available():
39 | data = data.cuda()
40 | target = target.cuda()
41 |
42 | optimizer.zero_grad()
43 | output = self.local_model(data)
44 | loss = torch.nn.functional.cross_entropy(output, target)
45 | loss.backward()
46 |
47 | optimizer.step()
48 | print("Epoch %d done." % e)
49 | diff = dict()
50 | for name, data in self.local_model.state_dict().items():
51 | diff[name] = (data - model.state_dict()[name])
52 | #print(diff[name])
53 |
54 | return diff
55 |
--------------------------------------------------------------------------------
/chapter03_Python_image_classification/datasets.py:
--------------------------------------------------------------------------------
1 |
2 |
3 | import torch
4 | from torchvision import datasets, transforms
5 |
6 | def get_dataset(dir, name):
7 |
8 | if name=='mnist':
9 | train_dataset = datasets.MNIST(dir, train=True, download=True, transform=transforms.ToTensor())
10 | eval_dataset = datasets.MNIST(dir, train=False, transform=transforms.ToTensor())
11 |
12 | elif name=='cifar':
13 | transform_train = transforms.Compose([
14 | transforms.RandomCrop(32, padding=4),
15 | transforms.RandomHorizontalFlip(),
16 | transforms.ToTensor(),
17 | transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)),
18 | ])
19 |
20 | transform_test = transforms.Compose([
21 | transforms.ToTensor(),
22 | transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)),
23 | ])
24 |
25 | train_dataset = datasets.CIFAR10(dir, train=True, download=True,
26 | transform=transform_train)
27 | eval_dataset = datasets.CIFAR10(dir, train=False, transform=transform_test)
28 |
29 |
30 | return train_dataset, eval_dataset
--------------------------------------------------------------------------------
/chapter03_Python_image_classification/figures/fig2.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/FederatedAI/Practicing-Federated-Learning/8090c9acfc584571e622807ac5ba4079cdc2b0c8/chapter03_Python_image_classification/figures/fig2.png
--------------------------------------------------------------------------------
/chapter03_Python_image_classification/figures/fig31.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/FederatedAI/Practicing-Federated-Learning/8090c9acfc584571e622807ac5ba4079cdc2b0c8/chapter03_Python_image_classification/figures/fig31.png
--------------------------------------------------------------------------------
/chapter03_Python_image_classification/main.py:
--------------------------------------------------------------------------------
1 | import argparse, json
2 | import datetime
3 | import os
4 | import logging
5 | import torch, random
6 |
7 | from server import *
8 | from client import *
9 | import models, datasets
10 |
11 |
12 |
13 | if __name__ == '__main__':
14 |
15 | parser = argparse.ArgumentParser(description='Federated Learning')
16 | parser.add_argument('-c', '--conf', dest='conf')
17 | args = parser.parse_args()
18 |
19 |
20 | with open(args.conf, 'r') as f:
21 | conf = json.load(f)
22 |
23 |
24 | train_datasets, eval_datasets = datasets.get_dataset("./data/", conf["type"])
25 |
26 | server = Server(conf, eval_datasets)
27 | clients = []
28 |
29 | for c in range(conf["no_models"]):
30 | clients.append(Client(conf, server.global_model, train_datasets, c))
31 |
32 | print("\n\n")
33 | for e in range(conf["global_epochs"]):
34 |
35 | candidates = random.sample(clients, conf["k"])
36 |
37 | weight_accumulator = {}
38 |
39 | for name, params in server.global_model.state_dict().items():
40 | weight_accumulator[name] = torch.zeros_like(params)
41 |
42 | for c in candidates:
43 | diff = c.local_train(server.global_model)
44 |
45 | for name, params in server.global_model.state_dict().items():
46 | weight_accumulator[name].add_(diff[name])
47 |
48 |
49 | server.model_aggregate(weight_accumulator)
50 |
51 | acc, loss = server.model_eval()
52 |
53 | print("Epoch %d, acc: %f, loss: %f\n" % (e, acc, loss))
54 |
55 |
56 |
57 |
58 |
59 |
60 |
61 |
--------------------------------------------------------------------------------
/chapter03_Python_image_classification/models.py:
--------------------------------------------------------------------------------
1 |
2 | import torch
3 | from torchvision import models
4 |
5 | def get_model(name="vgg16", pretrained=True):
6 | if name == "resnet18":
7 | model = models.resnet18(pretrained=pretrained)
8 | elif name == "resnet50":
9 | model = models.resnet50(pretrained=pretrained)
10 | elif name == "densenet121":
11 | model = models.densenet121(pretrained=pretrained)
12 | elif name == "alexnet":
13 | model = models.alexnet(pretrained=pretrained)
14 | elif name == "vgg16":
15 | model = models.vgg16(pretrained=pretrained)
16 | elif name == "vgg19":
17 | model = models.vgg19(pretrained=pretrained)
18 | elif name == "inception_v3":
19 | model = models.inception_v3(pretrained=pretrained)
20 | elif name == "googlenet":
21 | model = models.googlenet(pretrained=pretrained)
22 |
23 | if torch.cuda.is_available():
24 | return model.cuda()
25 | else:
26 | return model
--------------------------------------------------------------------------------
/chapter03_Python_image_classification/server.py:
--------------------------------------------------------------------------------
1 |
2 | import models, torch
3 |
4 |
5 | class Server(object):
6 |
7 | def __init__(self, conf, eval_dataset):
8 |
9 | self.conf = conf
10 |
11 | self.global_model = models.get_model(self.conf["model_name"])
12 |
13 | self.eval_loader = torch.utils.data.DataLoader(eval_dataset, batch_size=self.conf["batch_size"], shuffle=True)
14 |
15 |
16 | def model_aggregate(self, weight_accumulator):
17 | for name, data in self.global_model.state_dict().items():
18 |
19 | update_per_layer = weight_accumulator[name] * self.conf["lambda"]
20 |
21 | if data.type() != update_per_layer.type():
22 | data.add_(update_per_layer.to(torch.int64))
23 | else:
24 | data.add_(update_per_layer)
25 |
26 | def model_eval(self):
27 | self.global_model.eval()
28 |
29 | total_loss = 0.0
30 | correct = 0
31 | dataset_size = 0
32 | for batch_id, batch in enumerate(self.eval_loader):
33 | data, target = batch
34 | dataset_size += data.size()[0]
35 |
36 | if torch.cuda.is_available():
37 | data = data.cuda()
38 | target = target.cuda()
39 |
40 |
41 | output = self.global_model(data)
42 |
43 | total_loss += torch.nn.functional.cross_entropy(output, target,
44 | reduction='sum').item() # sum up batch loss
45 | pred = output.data.max(1)[1] # get the index of the max log-probability
46 | correct += pred.eq(target.data.view_as(pred)).cpu().sum().item()
47 |
48 | acc = 100.0 * (float(correct) / float(dataset_size))
49 | total_l = total_loss / dataset_size
50 |
51 | return acc, total_l
--------------------------------------------------------------------------------
/chapter03_Python_image_classification/utils/conf.json:
--------------------------------------------------------------------------------
1 | {
2 |
3 | "model_name" : "resnet18",
4 |
5 | "no_models" : 10,
6 |
7 | "type" : "cifar",
8 |
9 | "global_epochs" : 20,
10 |
11 | "local_epochs" : 3,
12 |
13 | "k" : 5,
14 |
15 | "batch_size" : 32,
16 |
17 | "lr" : 0.001,
18 |
19 | "momentum" : 0.0001,
20 |
21 | "lambda" : 0.1
22 | }
--------------------------------------------------------------------------------
/chapter05_FATE_HFL/README.md:
--------------------------------------------------------------------------------
1 | # 第5章：用FATE从零实现横向逻辑回归
2 |
3 | 在第三章我们介绍了如何使用Python构建简单的横向联邦学习模型。但应该注意到，联邦学习的开发，特别是工业级的产品开发，涉及的工程量却远不止于此，一个功能完备的联邦学习框架的设计细节是相当复杂的，幸运的是，随着联邦学习的发展，当前市面上已经出现了越来越多的开源平台。
4 |
5 | 本章我们将介绍使用FATE从零开始构建一个简单的横向逻辑回归模型，经过本章的学习，读者能够了解利用FATE进行横向建模的基本流程。鉴于本书的篇幅有限，以及本书的写作目的，我们不对FATE的具体原理实现进行详细的讲解。
6 |
7 |
8 |
9 | **注：由于FATE平台在不断迭代和更新的过程中，本章的内容撰写截稿时间较早（2020年9月截稿），可能会因版本变化而导致配置方式和运行方式发生改变，因此，强烈建议读者如果想进一步了解FATE的最新使用和安装，可以直接参考FATE官方文档教程：**
10 |
11 | * [FATE的安装部署](https://github.com/FederatedAI/DOC-CHN/tree/master/%E9%83%A8%E7%BD%B2)
12 | * [FATE的官方文档](https://github.com/FederatedAI/DOC-CHN)
13 |
14 | **如果FATE的安装和使用遇到任何问题，可以添加FATE小助手，有专门的工程团队人员帮忙解决。**
15 |
16 |
17 |
18 |
80 |
81 |
6 |
7 |
13 |
14 |
41 |
42 |
46 |
47 |
55 |
56 |
63 |
64 |
69 |
70 |
75 |
76 |
95 |
96 |
107 |
108 |
116 |
117 |
124 |
125 |
130 |
131 |
136 |
137 |
142 |
143 |
}{area(B_p&space;\cup&space;B_{g_t})}$$ "$$a_o=\frac{area(B_p \cap B_{g_t})}{area(B_p \cup B_{g_t})}$$")
53 | when 
denotes the intersection of the predicted and ground truth bounding boxes and 
their union.
54 | Average Precision is calculated for each class respectively.
55 | 
56 | where n is the number of total object in given class.
57 | For $k$ classes, mAP is the average of APs.
58 | 
59 |
--------------------------------------------------------------------------------
/chapter10_Computer_Vision/README.md:
--------------------------------------------------------------------------------
1 | # 第10章：联邦视觉案例
2 |
3 | 本章是《联邦学习实战》的第十章，我们将介绍在联邦学习场景下，利用分散在各摄像头的数据，构建一个联邦视觉系统，如下图所示：
4 |
5 |
6 |
7 |
25 |
26 |
105 |
106 |
111 |
112 |
8 |
9 |
21 |
22 |
38 |
39 |
40 | | 原始数据集 |
41 | 毒化数据集 |
42 | 目标 |
43 |
44 |
45 |  |
46 | |
47 |  |
48 |
49 |
50 |
51 |
52 |
53 |
54 | - 第二种方式：在原始图片中，人为添加某种特征信息，如下图所示，我们在原始图片添加红色的条纹，我们期望模型将带有红色条纹的图片数据都识别为小鸟。
55 |
56 |
57 |
58 |
59 | | 原始数据集 |
60 | 毒化数据集 |
61 | 目标 |
62 |
63 |
64 |  |
65 |  |
66 | |
67 |
68 |
69 |
70 |
71 |
72 |
73 | ## 15.1.4 客户端本地训练
74 |
75 | 对于带有后门攻击的联邦学习训练，其客户端可以分为恶意客户端和正常客户端。不同类型的客户端，其本地训练策略各不相同。正常客户端的训练策略如下所示，其执行过程就是常规的梯度下降过程。
76 |
77 | * 对于正常的客户端，其本地训练过程是一个常规的梯度下降过程
78 |
79 | ```python
80 | def local_train(self, model):
81 | for name, param in model.state_dict().items():
82 | self.local_model.state_dict()[name].copy_(param.clone())
83 |
84 |
85 | optimizer = torch.optim.SGD(self.local_model.parameters(), lr=self.conf['lr'],
86 | momentum=self.conf['momentum'])
87 |
88 | self.local_model.train()
89 | for e in range(self.conf["local_epochs"]):
90 |
91 | for batch_id, batch in enumerate(self.train_loader):
92 | data, target = batch
93 |
94 | if torch.cuda.is_available():
95 | data = data.cuda()
96 | target = target.cuda()
97 |
98 | optimizer.zero_grad()
99 | output = self.local_model(data)
100 | loss = torch.nn.functional.cross_entropy(output, target)
101 | loss.backward()
102 |
103 | optimizer.step()
104 | print("Epoch %d done." % e)
105 | diff = dict()
106 | for name, data in self.local_model.state_dict().items():
107 | diff[name] = (data - model.state_dict()[name])
108 |
109 | return diff
110 | ```
111 |
112 | 我们注意到，在最后模型上传的时候，客户端$C_i$上传的参数为：
113 |
114 | $$L_i^{t+1} - G^{t}$$
115 |
116 | 对于恶意客户端，其本地训练需要保证一方面模型训练后在毒化的数据集和正常的数据集上都能取得好的效果；另一方面，为了防止模型出现太大的偏差，需要保证当前训练的本地模型不会过于偏离全局模型：
117 |
118 | - 对于恶意客户端，首先是损失函数需要重新设计，恶意客户端的损失函数有两部分构成，分别是：
119 |
120 | 1. 类别损失$L_{class}$：要求模型在正常数据集和毒化数据集上都能取得好的效果；_
121 | 2. 距离损失$L_{distance}$：确保本地模型与全局模型不会产生太大的距离。
122 |
123 | ```python
124 | def local_train_malicious(self, model):
125 | for name, param in model.state_dict().items():
126 | self.local_model.state_dict()[name].copy_(param.clone())
127 |
128 | optimizer = torch.optim.SGD(self.local_model.parameters(), lr=self.conf['lr'],
129 | momentum=self.conf['momentum'])
130 | pos = []
131 | for i in range(2, 28):
132 | pos.append([i, 3])
133 | pos.append([i, 4])
134 | pos.append([i, 5])
135 | self.local_model.train()
136 | for e in range(self.conf["local_epochs"]):
137 | for batch_id, batch in enumerate(self.train_loader):
138 | data, target = batch
139 | for k in range(self.conf["poisoning_per_batch"]):
140 | img = data[k].numpy()
141 | for i in range(0,len(pos)):
142 | img[0][pos[i][0]][pos[i][1]] = 1.0
143 | img[1][pos[i][0]][pos[i][1]] = 0
144 | img[2][pos[i][0]][pos[i][1]] = 0
145 |
146 | target[k] = self.conf['poison_label']
147 | if torch.cuda.is_available():
148 | data = data.cuda()
149 | target = target.cuda()
150 |
151 | optimizer.zero_grad()
152 | output = self.local_model(data)
153 |
154 | class_loss = torch.nn.functional.cross_entropy(output, target)
155 | dist_loss = models.model_norm(self.local_model, model)
156 | loss = self.conf["alpha"]*class_loss + (1-self.conf["alpha"])*dist_loss
157 | loss.backward()
158 |
159 | optimizer.step()
160 | print("Epoch %d done." % e)
161 |
162 | diff = dict()
163 | for name, data in self.local_model.state_dict().items():
164 | diff[name] = self.conf["eta"]*(data - model.state_dict()[name])+model.state_dict()[name]
165 |
166 | return diff
167 |
168 | ```
169 |
170 | 我们同样注意到，在最后模型上传的时候，客户端$C_i$上传的参数为：
171 |
172 | $$\lambda*(L_m^{t+1} - G^{t})+G^t$$
173 |
174 | 其中$\lambda$是一个大于1的数值，读者可以在配置文件中自行设置。
175 |
176 | ## 15.1.5 参考文献
177 |
178 | - [How to backdoor federated learning](https://arxiv.org/pdf/1807.00459.pdf)
179 |
180 | - [DBA: Distributed Backdoor Attacks against Federated Learning](https://openreview.net/pdf?id=rkgyS0VFvr)
181 |
182 |
183 |
--------------------------------------------------------------------------------
/chapter15_Backdoor_Attack/client.py:
--------------------------------------------------------------------------------
1 |
2 | import models, torch, copy
3 | import numpy as np
4 | import matplotlib.pyplot as plt
5 |
6 | class Client(object):
7 |
8 | def __init__(self, conf, model, train_dataset, id = -1):
9 |
10 | self.conf = conf
11 |
12 | self.local_model = models.get_model(self.conf["model_name"])
13 |
14 | self.client_id = id
15 |
16 | self.train_dataset = train_dataset
17 |
18 | all_range = list(range(len(self.train_dataset)))
19 | data_len = int(len(self.train_dataset) / self.conf['no_models'])
20 | train_indices = all_range[id * data_len: (id + 1) * data_len]
21 |
22 | self.train_loader = torch.utils.data.DataLoader(self.train_dataset, batch_size=conf["batch_size"],
23 | sampler=torch.utils.data.sampler.SubsetRandomSampler(train_indices))
24 |
25 |
26 |
27 | def local_train(self, model):
28 |
29 | for name, param in model.state_dict().items():
30 | self.local_model.state_dict()[name].copy_(param.clone())
31 |
32 |
33 | optimizer = torch.optim.SGD(self.local_model.parameters(), lr=self.conf['lr'],
34 | momentum=self.conf['momentum'])
35 |
36 | self.local_model.train()
37 | for e in range(self.conf["local_epochs"]):
38 |
39 | for batch_id, batch in enumerate(self.train_loader):
40 | data, target = batch
41 |
42 | if torch.cuda.is_available():
43 | data = data.cuda()
44 | target = target.cuda()
45 |
46 | optimizer.zero_grad()
47 | output = self.local_model(data)
48 | loss = torch.nn.functional.cross_entropy(output, target)
49 | loss.backward()
50 |
51 | optimizer.step()
52 | print("Epoch %d done." % e)
53 | diff = dict()
54 | for name, data in self.local_model.state_dict().items():
55 | diff[name] = (data - model.state_dict()[name])
56 |
57 | return diff
58 |
59 | def local_train_malicious(self, model):
60 |
61 | for name, param in model.state_dict().items():
62 | self.local_model.state_dict()[name].copy_(param.clone())
63 |
64 | optimizer = torch.optim.SGD(self.local_model.parameters(), lr=self.conf['lr'],
65 | momentum=self.conf['momentum'])
66 | pos = []
67 | for i in range(2, 28):
68 | pos.append([i, 3])
69 | pos.append([i, 4])
70 | pos.append([i, 5])
71 |
72 | self.local_model.train()
73 | for e in range(self.conf["local_epochs"]):
74 |
75 | for batch_id, batch in enumerate(self.train_loader):
76 | data, target = batch
77 |
78 | for k in range(self.conf["poisoning_per_batch"]):
79 | img = data[k].numpy()
80 | for i in range(0,len(pos)):
81 | img[0][pos[i][0]][pos[i][1]] = 1.0
82 | img[1][pos[i][0]][pos[i][1]] = 0
83 | img[2][pos[i][0]][pos[i][1]] = 0
84 |
85 | target[k] = self.conf['poison_label']
86 | #for k in range(32):
87 | # img = data[k].numpy()
88 | #
89 | # img = np.transpose(img, (1, 2, 0))
90 | # plt.imshow(img)
91 | # plt.show()
92 | if torch.cuda.is_available():
93 | data = data.cuda()
94 | target = target.cuda()
95 |
96 | optimizer.zero_grad()
97 | output = self.local_model(data)
98 |
99 | class_loss = torch.nn.functional.cross_entropy(output, target)
100 | dist_loss = models.model_norm(self.local_model, model)
101 | loss = self.conf["alpha"]*class_loss + (1-self.conf["alpha"])*dist_loss
102 | loss.backward()
103 |
104 | optimizer.step()
105 | print("Epoch %d done." % e)
106 |
107 | diff = dict()
108 | for name, data in self.local_model.state_dict().items():
109 | diff[name] = self.conf["eta"]*(data - model.state_dict()[name])+model.state_dict()[name]
110 |
111 | return diff
112 |
--------------------------------------------------------------------------------
/chapter15_Backdoor_Attack/datasets.py:
--------------------------------------------------------------------------------
1 |
2 |
3 | import torch
4 | from torchvision import datasets, transforms
5 |
6 | def get_dataset(dir, name):
7 |
8 | if name=='mnist':
9 | train_dataset = datasets.MNIST(dir, train=True, download=True, transform=transforms.ToTensor())
10 | eval_dataset = datasets.MNIST(dir, train=False, transform=transforms.ToTensor())
11 |
12 | elif name=='cifar':
13 | transform_train = transforms.Compose([
14 | transforms.RandomCrop(32, padding=4),
15 | transforms.RandomHorizontalFlip(),
16 | transforms.ToTensor(),
17 | transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)),
18 | ])
19 |
20 | transform_test = transforms.Compose([
21 | transforms.ToTensor(),
22 | transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)),
23 | ])
24 |
25 | train_dataset = datasets.CIFAR10(dir, train=True, download=True,
26 | transform=transform_train)
27 | eval_dataset = datasets.CIFAR10(dir, train=False, transform=transform_test)
28 |
29 |
30 | return train_dataset, eval_dataset
--------------------------------------------------------------------------------
/chapter15_Backdoor_Attack/images/fl_backdoor.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/FederatedAI/Practicing-Federated-Learning/8090c9acfc584571e622807ac5ba4079cdc2b0c8/chapter15_Backdoor_Attack/images/fl_backdoor.png
--------------------------------------------------------------------------------
/chapter15_Backdoor_Attack/images/normal_image.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/FederatedAI/Practicing-Federated-Learning/8090c9acfc584571e622807ac5ba4079cdc2b0c8/chapter15_Backdoor_Attack/images/normal_image.png
--------------------------------------------------------------------------------
/chapter15_Backdoor_Attack/images/normal_image_1.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/FederatedAI/Practicing-Federated-Learning/8090c9acfc584571e622807ac5ba4079cdc2b0c8/chapter15_Backdoor_Attack/images/normal_image_1.png
--------------------------------------------------------------------------------
/chapter15_Backdoor_Attack/images/poison_image.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/FederatedAI/Practicing-Federated-Learning/8090c9acfc584571e622807ac5ba4079cdc2b0c8/chapter15_Backdoor_Attack/images/poison_image.png
--------------------------------------------------------------------------------
/chapter15_Backdoor_Attack/images/target.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/FederatedAI/Practicing-Federated-Learning/8090c9acfc584571e622807ac5ba4079cdc2b0c8/chapter15_Backdoor_Attack/images/target.png
--------------------------------------------------------------------------------
/chapter15_Backdoor_Attack/main.py:
--------------------------------------------------------------------------------
1 | import argparse, json
2 | import datetime
3 | import os
4 | import logging
5 | import torch, random
6 |
7 | from server import *
8 | from client import *
9 | import models, datasets
10 |
11 |
12 |
13 | if __name__ == '__main__':
14 |
15 | parser = argparse.ArgumentParser(description='Federated Learning')
16 | parser.add_argument('-c', '--conf', dest='conf')
17 | args = parser.parse_args()
18 |
19 |
20 | with open(args.conf, 'r') as f:
21 | conf = json.load(f)
22 |
23 |
24 | train_datasets, eval_datasets = datasets.get_dataset("./data/", conf["type"])
25 |
26 | server = Server(conf, eval_datasets)
27 | clients = []
28 |
29 | for c in range(conf["no_models"]):
30 | clients.append(Client(conf, server.global_model, train_datasets, c))
31 |
32 | print("\n\n")
33 | for e in range(conf["global_epochs"]):
34 |
35 | candidates = random.sample(clients, conf["k"])
36 |
37 | weight_accumulator = {}
38 |
39 | for name, params in server.global_model.state_dict().items():
40 | weight_accumulator[name] = torch.zeros_like(params)
41 |
42 | for c in candidates:
43 | if c.client_id == 1:
44 | print("malicious client")
45 | diff = c.local_train_malicious(server.global_model)
46 | else:
47 | diff = c.local_train(server.global_model)
48 |
49 | for name, params in server.global_model.state_dict().items():
50 | weight_accumulator[name].add_(diff[name])
51 |
52 |
53 | server.model_aggregate(weight_accumulator)
54 |
55 | acc, loss = server.model_eval()
56 |
57 | print("Epoch %d, acc: %f, loss: %f\n" % (e, acc, loss))
58 |
59 |
60 |
61 |
62 |
63 |
64 |
65 |
--------------------------------------------------------------------------------
/chapter15_Backdoor_Attack/models.py:
--------------------------------------------------------------------------------
1 |
2 | import torch
3 | from torchvision import models
4 | import math
5 |
6 | def get_model(name="vgg16", pretrained=True):
7 | if name == "resnet18":
8 | model = models.resnet18(pretrained=pretrained)
9 | elif name == "resnet50":
10 | model = models.resnet50(pretrained=pretrained)
11 | elif name == "densenet121":
12 | model = models.densenet121(pretrained=pretrained)
13 | elif name == "alexnet":
14 | model = models.alexnet(pretrained=pretrained)
15 | elif name == "vgg16":
16 | model = models.vgg16(pretrained=pretrained)
17 | elif name == "vgg19":
18 | model = models.vgg19(pretrained=pretrained)
19 | elif name == "inception_v3":
20 | model = models.inception_v3(pretrained=pretrained)
21 | elif name == "googlenet":
22 | model = models.googlenet(pretrained=pretrained)
23 |
24 | if torch.cuda.is_available():
25 | return model.cuda()
26 | else:
27 | return model
28 |
29 | def model_norm(model_1, model_2):
30 | squared_sum = 0
31 | for name, layer in model_1.named_parameters():
32 | # print(torch.mean(layer.data), torch.mean(model_2.state_dict()[name].data))
33 | squared_sum += torch.sum(torch.pow(layer.data - model_2.state_dict()[name].data, 2))
34 | return math.sqrt(squared_sum)
--------------------------------------------------------------------------------
/chapter15_Backdoor_Attack/server.py:
--------------------------------------------------------------------------------
1 |
2 | import models, torch
3 |
4 |
5 | class Server(object):
6 |
7 | def __init__(self, conf, eval_dataset):
8 |
9 | self.conf = conf
10 |
11 | self.global_model = models.get_model(self.conf["model_name"])
12 |
13 | self.eval_loader = torch.utils.data.DataLoader(eval_dataset, batch_size=self.conf["batch_size"], shuffle=True)
14 |
15 |
16 | def model_aggregate(self, weight_accumulator):
17 | for name, data in self.global_model.state_dict().items():
18 |
19 | update_per_layer = weight_accumulator[name] * self.conf["lambda"]
20 |
21 | if data.type() != update_per_layer.type():
22 | data.add_(update_per_layer.to(torch.int64))
23 | else:
24 | data.add_(update_per_layer)
25 |
26 | def model_eval(self):
27 | self.global_model.eval()
28 |
29 | total_loss = 0.0
30 | correct = 0
31 | dataset_size = 0
32 | for batch_id, batch in enumerate(self.eval_loader):
33 | data, target = batch
34 | dataset_size += data.size()[0]
35 |
36 | if torch.cuda.is_available():
37 | data = data.cuda()
38 | target = target.cuda()
39 |
40 |
41 | output = self.global_model(data)
42 |
43 | total_loss += torch.nn.functional.cross_entropy(output, target,
44 | reduction='sum').item() # sum up batch loss
45 | pred = output.data.max(1)[1] # get the index of the max log-probability
46 | correct += pred.eq(target.data.view_as(pred)).cpu().sum().item()
47 |
48 | acc = 100.0 * (float(correct) / float(dataset_size))
49 | total_l = total_loss / dataset_size
50 |
51 | return acc, total_l
--------------------------------------------------------------------------------
/chapter15_Backdoor_Attack/utils/conf.json:
--------------------------------------------------------------------------------
1 | {
2 |
3 | "model_name" : "resnet18",
4 |
5 | "no_models" : 10,
6 |
7 | "type" : "cifar",
8 |
9 | "global_epochs" : 20,
10 |
11 | "local_epochs" : 3,
12 |
13 | "k" : 3,
14 |
15 | "batch_size" : 32,
16 |
17 | "lr" : 0.001,
18 |
19 | "momentum" : 0.0001,
20 |
21 | "lambda" : 0.3,
22 |
23 | "eta" : 2,
24 |
25 | "alpha" : 1.0,
26 |
27 | "poison_label" : 2,
28 |
29 | "poisoning_per_batch" : 4
30 | }
--------------------------------------------------------------------------------
/chapter15_Compression/README.md:
--------------------------------------------------------------------------------
1 | # 15.3：模型压缩
2 |
3 | 研究表明，大部分深度神经网络模型，在模型训练的过程中，都存在着权重参数冗余的问题，即在所有参数中，对模型性能起到重要作用的，往往只有很少的一部分（比如5%）权值。基于这个思想，在深度学习中，我们常常使用剪枝、量化、蒸漏等模型压缩手段来压缩模型，以达到模型性能与模型复杂度之间的平衡。
4 |
5 | 联邦学习在训练的过程中，影响其训练效率的一大因素就是服务端与客户端之间的模型参数交换。因此，我们可以利用模型压缩的思想，在传输的过程中，只传输部分的参数数据，一方面随着传输数据量的减少，能够有效降低网络传输的带宽消耗；另一方面，可以防止模型参数被窃取，由于只传输了部分参数数据，这样即使攻击者获取了这部分数据，由于没有全局信息，因此，也很难利用模型反演攻击来反推出原始数据，从而有效提升了系统的安全性。
6 |
7 |
8 | ## 15.3.1 代码使用
9 |
10 | 在本目录下，在命令行中执行下面的命令：
11 |
12 | ```
13 | python main.py -c ./utils/conf.json
14 | ```
15 |
16 | ## 15.3.2 方法介绍
17 |
18 | 本小节我们考虑这样的一种压缩策略：模型在本地训练过程中，训练前后每一层的变化，选取前后变化大的层传输，为此，我们首先定义层敏感度的概念：
19 |
20 | > 定义：设当前的模型表示为$G = \\{g_1, g_2, · · · , g_L\\}$，这里的$g_i$表示模型的第i层。设当前处在第$t$轮，客户端$C_j$进行联邦学习本地训练时，模型将从：
21 | >
22 | > $$G_t = \\{g^{t}\_{1,j} ,g^{t}\_{2,j} , · · · , g^{t}\_{L,j} \\} $$
23 | >
24 | > 变为：
25 | >
26 | > $$L^{t+1}_j = \\{g^{t+1}\_{1,j} ,g^{t+1}\_{2,j} , · · · , g^{t+1}\_{L,j} \\}$$
27 | >
28 | > 我们将第$i$层的变化记为：
29 | >
30 | > $$\delta^t_{i,j}=|mean(g^{t}\_{i,j}) - mean(g^{t+1}\_{i,j})|$$
31 | >
32 | > 我们将此每一层的参数均值变化量$\delta$，称为层敏感度。
33 |
34 | 基于按层敏感度的定义，对任意被挑选的客户端$C_j$，在模型本地训练结束后，按照上式计算模型的每一层均值变化量$\Delta^t_{i,j}=\\{\delta^t\_{1,j}, \delta^t\_{2,j}, ... , \delta^t\_{L,j} \\}$, 将每一层的敏感度变化量从大到小进行排序，变化越大，说明该层重要性越高。
35 |
36 | ## 15.3.3 配置文件
37 |
38 | 我们首先在配置文件中添加一个字段“rate”，用来控制传输比例，比如0.95表示传输最重要的前95%的层权重参数。
39 | ```python
40 | {
41 | ...,
42 |
43 | rate : 0.95
44 | }
45 | ```
46 |
47 | ## 15.3.4 客户端训练
48 | 其本地训练代码如下所示，训练的过程与常规的本地迭代训练一致：
49 |
50 | ```python
51 | for name, param in model.state_dict().items():
52 | self.local_model.state_dict()[name].copy_(param.clone())
53 | optimizer = torch.optim.SGD(self.local_model.parameters(), lr=self.conf['lr'],
54 | momentum=self.conf['momentum'])
55 |
56 | self.local_model.train()
57 | for e in range(self.conf["local_epochs"]):
58 | for batch_id, batch in enumerate(self.train_loader):
59 | data, target = batch
60 | if torch.cuda.is_available():
61 | data = data.cuda()
62 | target = target.cuda()
63 |
64 | optimizer.zero_grad()
65 | output = self.local_model(data)
66 | loss = torch.nn.functional.cross_entropy(output, target)
67 | loss.backward()
68 | optimizer.step()
69 | print("Epoch %d done." % e)
70 | ```
71 | 训练完毕后，求取每一层的变化值，并对变化值按大小从高到低进行排序，最后按照配置文件中设置的“rate”值，上传最重要的参数权重即可：
72 | ```python
73 | diff = dict()
74 | for name, data in self.local_model.state_dict().items():
75 | diff[name] = (data - model.state_dict()[name])
76 | diff = sorted(diff.items(), key=lambda item:abs(torch.mean(item[1].float())), reverse=True)
77 | sum1, sum2 = 0, 0
78 | for id, (name, data) in enumerate(diff):
79 | if id < 304:
80 | sum1 += torch.prod(torch.tensor(data.size()))
81 | else:
82 | sum2 += torch.prod(torch.tensor(data.size()))
83 |
84 | ret_size = int(self.conf["rate"]*len(diff))
85 |
86 | return dict(diff[:ret_size])
87 | ```
88 |
89 | ## 15.3.5 服务端
90 |
91 | 在服务端中进行聚合的时候，由于每一个客户端上传的只是部分的权重参数，需要根据不同的层来进行聚合，如下所示，在Pytorch中，我们可以通过权重的名字来实现这一过程：
92 |
93 | ```python
94 | def model_aggregate(self, weight_accumulator, cnt):
95 | for name, data in self.global_model.state_dict().items():
96 | if name in weight_accumulator and cnt[name] > 0:
97 | update_per_layer = weight_accumulator[name] * (1.0 / cnt[name])
98 |
99 | if data.type() != update_per_layer.type():
100 | data.add_(update_per_layer.to(torch.int64))
101 | else:
102 | data.add_(update_per_layer)
103 | ```
104 |
105 | ## 15.3.6 效果分析
106 |
107 | 我们根据上传的参数比例不同，比较了在不同的比例配置下，模型性能之间的差别，从下图可以看到，选取重要的模型参数进行传输（top 90%），其性能与完整的参数传输（100%）相比，几乎没有下降。
108 |
109 |
110 |
111 |
10 |
11 |
19 |
20 |
44 |
45 |
92 |
93 |
6 |
7 |
176 |
177 |
12 |
13 |
95 |
96 |
10 |
11 |
16 |
17 |
26 |
27 |