├── .github └── FUNDING.yml ├── KerasFederated ├── Console │ ├── README.md │ ├── client1.py │ ├── client2.py │ └── server.py ├── README.md ├── client1.py ├── client2.py └── server.py ├── README.md ├── TutorialProject ├── Part1 │ ├── README.md │ ├── client.py │ └── server.py ├── Part2 │ ├── README.md │ ├── client1.py │ ├── client2.py │ └── server.py ├── Part3 │ ├── README.md │ ├── client1.py │ ├── client2.py │ └── server.py └── Part4 │ ├── README.md │ ├── client1.py │ ├── client2.py │ └── server.py ├── client1.py ├── client2.py ├── requirements.txt └── server.py /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | # These are supported funding model platforms 2 | 3 | github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2] 4 | # paypal: http://paypal.me/ahmedfgad # Replace with a single Patreon username 5 | open_collective: pygad 6 | ko_fi: # Replace with a single Ko-fi username 7 | tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel 8 | community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry 9 | liberapay: # Replace with a single Liberapay username 10 | issuehunt: # Replace with a single IssueHunt username 11 | otechie: # Replace with a single Otechie username 12 | custom: ['https://donate.stripe.com/eVa5kO866elKgM0144', 'http://paypal.me/ahmedfgad'] # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2'] 13 | -------------------------------------------------------------------------------- /KerasFederated/Console/README.md: -------------------------------------------------------------------------------- 1 | # Federated Learning using Keras and PyGAD 2 | 3 | Training a Keras model using the genetic algorithm ([PyGAD](https://pygad.readthedocs.io)) using federated learning of multiple clients. 4 | 5 | To know more about training Keras models using [PyGAD](https://pygad.readthedocs.io), please read this tutorial: [How To Train Keras Models Using the Genetic Algorithm with PyGAD](https://blog.paperspace.com/train-keras-models-using-genetic-algorithm-with-pygad) 6 | 7 | # Project Files 8 | 9 | The project has the following files: 10 | 11 | - `server.py`: The server app. It creates a Keras model that is trained on the clients' devices using FL with PyGAD. 12 | - `client1.py`: The client app which trains the Keras model sent by the server using just 2 samples of the XOR problem. 13 | - `client2.py`: Another client app that trains the server's Keras model using the other 2 samples in the XOR problem. 14 | 15 | # Install PyGAD 16 | 17 | Before running the project, the [PyGAD](https://pygad.readthedocs.io) library must be installed. 18 | 19 | ``` 20 | pip install pygad 21 | ``` 22 | 23 | # Running the Project 24 | 25 | Start the project by running the [`server.py`](https://github.com/ahmedfgad/FederatedLearning/blob/master/KerasFederated/Console/server.py) file. Please use appropriate IPv4 and port number. 26 | 27 | After running the server, next is to run one or more clients. The project creates 2 clients but you can add more. The only expected change among the different clients is the data being used for training the model sent by the server. 28 | 29 | For [`client1.py`](https://github.com/ahmedfgad/FederatedLearning/blob/master/KerasFederated/client1.py), here is the training data (2 samples of the XOR problem): 30 | 31 | ```python 32 | # Preparing the NumPy array of the inputs. 33 | data_inputs = numpy.array([[0, 1], 34 | [0, 0]]) 35 | 36 | # Preparing the NumPy array of the outputs. 37 | data_outputs = numpy.array([[0, 1], 38 | [1, 0]]) 39 | ``` 40 | 41 | Here is the training data (other 2 samples of the XOR problem) for the other client ([`client2.py`](https://github.com/ahmedfgad/FederatedLearning/blob/master/KerasFederated/Console/client2.py)): 42 | 43 | ```python 44 | # Preparing the NumPy array of the inputs. 45 | data_inputs = numpy.array([[1, 0], 46 | [1, 1]]) 47 | 48 | # Preparing the NumPy array of the outputs. 49 | data_outputs = numpy.array([[0, 1], 50 | [1, 0]]) 51 | ``` 52 | 53 | # For More Information 54 | 55 | There are a number of resources to get started with federated learning and Kivy. 56 | 57 | ## Tutorial: [Introduction to Federated Learning](https://heartbeat.fritz.ai/introduction-to-federated-learning-40eb122754a2) 58 | 59 | This tutorial describes the pipeline of training a machine learning model using federated learning. 60 | 61 | [![](https://miro.medium.com/max/3240/1*6gRmlrDPp5J42HR3QWLYew.jpeg)](https://heartbeat.fritz.ai/introduction-to-federated-learning-40eb122754a2) 62 | 63 | ## Tutorial: [How To Train Keras Models Using the Genetic Algorithm with PyGAD](https://blog.paperspace.com/train-keras-models-using-genetic-algorithm-with-pygad) 64 | 65 | Use [PyGAD](https://pygad.readthedocs.io), a Python 3 easy-to-use genetic algorithm library, to train Keras models using the genetic algorithm. The tutorial is detailed to explain all the steps needed to build and train the model. 66 | 67 | [![](https://user-images.githubusercontent.com/16560492/101267295-c74c0180-375f-11eb-9ad0-f8e37bd796ce.png)](https://blog.paperspace.com/train-keras-models-using-genetic-algorithm-with-pygad) 68 | 69 | ## Tutorial: [Breaking Privacy in Federated Learning](https://heartbeat.fritz.ai/breaking-privacy-in-federated-learning-77fa08ccac9a) 70 | 71 | Even that federated learning does not disclose the private user data, there are some cases in which the privacy of federated learning can be broken. 72 | 73 | [![](https://miro.medium.com/max/3240/1*nZQg-E4a1wOvIH2AmkUUsQ.jpeg)](https://heartbeat.fritz.ai/breaking-privacy-in-federated-learning-77fa08ccac9a) 74 | 75 | ## Tutorial: [Python for Android: Start Building Kivy Cross-Platform Applications](https://www.linkedin.com/pulse/python-android-start-building-kivy-cross-platform-applications-gad) 76 | 77 | This tutorial titled [Python for Android: Start Building Kivy Cross-Platform Applications](https://www.linkedin.com/pulse/python-android-start-building-kivy-cross-platform-applications-gad) covers the steps for creating an Android app out of the Kivy app. 78 | 79 | [![Kivy-Tutorial](https://user-images.githubusercontent.com/16560492/86205332-dfdd3d80-bb69-11ea-91fb-cb0143cb1e5e.png)](https://www.linkedin.com/pulse/python-android-start-building-kivy-cross-platform-applications-gad) 80 | 81 | ## Book: [Building Android Apps in Python Using Kivy with Android Studio](https://www.amazon.com/Building-Android-Python-Using-Studio/dp/1484250303) 82 | 83 | To get started with Kivy app development and how to built Android apps out of the Kivy app, check the book titled [Building Android Apps in Python Using Kivy with Android Studio](https://www.amazon.com/Building-Android-Python-Using-Studio/dp/1484250303) 84 | 85 | [![kivy-book](https://user-images.githubusercontent.com/16560492/86205093-575e9d00-bb69-11ea-82f7-23fef487ce3c.jpg)](https://www.amazon.com/Building-Android-Python-Using-Studio/dp/1484250303) 86 | 87 | # Contact Us 88 | 89 | - E-mail: [ahmed.f.gad@gmail.com](mailto:ahmed.f.gad@gmail.com) 90 | - [LinkedIn](https://www.linkedin.com/in/ahmedfgad) 91 | - [Amazon Author Page](https://amazon.com/author/ahmedgad) 92 | - [Heartbeat](https://heartbeat.fritz.ai/@ahmedfgad) 93 | - [Paperspace](https://blog.paperspace.com/author/ahmed) 94 | - [KDnuggets](https://kdnuggets.com/author/ahmed-gad) 95 | - [TowardsDataScience](https://towardsdatascience.com/@ahmedfgad) 96 | - [GitHub](https://github.com/ahmedfgad) -------------------------------------------------------------------------------- /KerasFederated/Console/client1.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import numpy 4 | import threading 5 | 6 | import tensorflow.keras 7 | import pygad.kerasga 8 | import pygad 9 | 10 | def close_socket(*args): 11 | soc.close() 12 | print("Socket Closed") 13 | 14 | def fitness_func(solution, sol_idx): 15 | global keras_ga, data_inputs, data_outputs 16 | 17 | model = keras_ga.model 18 | 19 | predictions = pygad.kerasga.predict(model, 20 | solution, 21 | data_inputs) 22 | 23 | bce = tensorflow.keras.losses.BinaryCrossentropy() 24 | solution_fitness = 1.0 / (bce(data_outputs, predictions).numpy() + 0.00000001) 25 | 26 | return solution_fitness 27 | 28 | def prepare_GA(server_data): 29 | global keras_ga 30 | 31 | population_weights = server_data["population_weights"] 32 | model_json = server_data["model_json"] 33 | num_solutions = server_data["num_solutions"] 34 | 35 | model = tensorflow.keras.models.model_from_json(model_json) 36 | keras_ga = pygad.kerasga.KerasGA(model=model, 37 | num_solutions=num_solutions) 38 | 39 | keras_ga.population_weights = population_weights 40 | 41 | ga_instance = pygad.GA(num_generations=150, 42 | num_parents_mating=4, 43 | initial_population=keras_ga.population_weights.copy(), 44 | fitness_func=fitness_func) 45 | 46 | return ga_instance 47 | 48 | # Preparing the NumPy array of the inputs. 49 | data_inputs = numpy.array([[0, 1], 50 | [0, 0]]) 51 | 52 | # Preparing the NumPy array of the outputs. 53 | data_outputs = numpy.array([[0, 1], 54 | [1, 0]]) 55 | 56 | class RecvThread(threading.Thread): 57 | 58 | def __init__(self, buffer_size, recv_timeout): 59 | threading.Thread.__init__(self) 60 | self.buffer_size = buffer_size 61 | self.recv_timeout = recv_timeout 62 | 63 | def recv(self): 64 | received_data = b"" 65 | while True: 66 | try: 67 | soc.settimeout(self.recv_timeout) 68 | received_data += soc.recv(self.buffer_size) 69 | 70 | try: 71 | pickle.loads(received_data) 72 | print("All data ({data_len} bytes) is received from the server.".format(data_len=len(received_data)), end="\n") 73 | # If the previous pickle.loads() statement is passed, this means all the data is received. 74 | # Thus, no need to continue the loop and a break statement should be excuted. 75 | break 76 | except BaseException: 77 | # An exception is expected when the data is not 100% received. 78 | pass 79 | 80 | except socket.timeout: 81 | print("A socket.timeout exception occurred because the server did not send any data for {recv_timeout} seconds.".format(recv_timeout=self.recv_timeout)) 82 | print("{recv_timeout} Seconds of Inactivity. socket.timeout Exception Occurred".format(recv_timeout=self.recv_timeout)) 83 | return None, 0 84 | except BaseException as e: 85 | return None, 0 86 | print("Error While Receiving Data from the Server: {msg}.".format(msg=e)) 87 | 88 | try: 89 | received_data = pickle.loads(received_data) 90 | except BaseException as e: 91 | print("Error Decoding the Data: {msg}.\n".format(msg=e)) 92 | return None, 0 93 | 94 | return received_data, 1 95 | 96 | def run(self): 97 | global server_data 98 | 99 | subject = "echo" 100 | server_data = None 101 | best_sol_idx = -1 102 | best_model_weights_vector = None 103 | 104 | while True: 105 | data_dict = {"best_model_weights_vector": best_model_weights_vector} 106 | data = {"subject": subject, "data": data_dict} 107 | 108 | # data = {"subject": subject, "data": keras_ga, "best_solution_idx": best_sol_idx} 109 | data_byte = pickle.dumps(data) 110 | 111 | print("Sending a Message of Type {subject} to the Server".format(subject=subject)) 112 | try: 113 | soc.sendall(data_byte) 114 | except BaseException as e: 115 | print("Error Connecting to the Server. The server might has been closed: {msg}".format(msg=e)) 116 | break 117 | 118 | print("Receiving Reply from the Server") 119 | received_data, status = self.recv() 120 | if status == 0: 121 | print("Nothing Received from the Server") 122 | break 123 | else: 124 | print("New Message from the Server with subject {sub}".format(sub=received_data["subject"])) 125 | 126 | subject = received_data["subject"] 127 | if subject == "model": 128 | server_data = received_data["data"] 129 | elif subject == "done": 130 | print("Model is Trained") 131 | break 132 | else: 133 | print("Unrecognized Message Type: {subject}".format(subject=subject)) 134 | break 135 | 136 | ga_instance = prepare_GA(server_data) 137 | 138 | ga_instance.run() 139 | 140 | subject = "model" 141 | best_sol_idx = ga_instance.best_solution(ga_instance.last_generation_fitness)[2] 142 | best_model_weights_vector = ga_instance.population[best_sol_idx, :] 143 | 144 | predictions = keras_ga.model.predict(data_inputs) 145 | ba = tensorflow.keras.metrics.BinaryAccuracy() 146 | ba.update_state(data_outputs, predictions) 147 | accuracy = ba.result().numpy() 148 | print("Accuracy {acc}".format(acc=accuracy), end="\n\n") 149 | 150 | soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 151 | print("Socket Created") 152 | 153 | try: 154 | soc.connect(("192.168.0.10", int("5000"))) 155 | print("Successful Connection to the Server") 156 | except BaseException as e: 157 | print("Error Connecting to the Server: {msg}".format(msg=e)) 158 | 159 | recvThread = RecvThread(buffer_size=1024, recv_timeout=10) 160 | recvThread.start() 161 | 162 | -------------------------------------------------------------------------------- /KerasFederated/Console/client2.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import numpy 4 | import threading 5 | 6 | import tensorflow.keras 7 | import pygad.kerasga 8 | import pygad 9 | 10 | def close_socket(*args): 11 | soc.close() 12 | print("Socket Closed") 13 | 14 | def fitness_func(solution, sol_idx): 15 | global keras_ga, data_inputs, data_outputs 16 | 17 | model = keras_ga.model 18 | 19 | predictions = pygad.kerasga.predict(model, 20 | solution, 21 | data_inputs) 22 | 23 | bce = tensorflow.keras.losses.BinaryCrossentropy() 24 | solution_fitness = 1.0 / (bce(data_outputs, predictions).numpy() + 0.00000001) 25 | 26 | return solution_fitness 27 | 28 | def prepare_GA(server_data): 29 | global keras_ga 30 | 31 | population_weights = server_data["population_weights"] 32 | model_json = server_data["model_json"] 33 | num_solutions = server_data["num_solutions"] 34 | 35 | model = tensorflow.keras.models.model_from_json(model_json) 36 | keras_ga = pygad.kerasga.KerasGA(model=model, 37 | num_solutions=num_solutions) 38 | 39 | keras_ga.population_weights = population_weights 40 | 41 | ga_instance = pygad.GA(num_generations=150, 42 | num_parents_mating=4, 43 | initial_population=keras_ga.population_weights.copy(), 44 | fitness_func=fitness_func) 45 | 46 | return ga_instance 47 | 48 | # Preparing the NumPy array of the inputs. 49 | data_inputs = numpy.array([[1, 0], 50 | [1, 1]]) 51 | 52 | # Preparing the NumPy array of the outputs. 53 | data_outputs = numpy.array([[0, 1], 54 | [1, 0]]) 55 | 56 | class RecvThread(threading.Thread): 57 | 58 | def __init__(self, buffer_size, recv_timeout): 59 | threading.Thread.__init__(self) 60 | self.buffer_size = buffer_size 61 | self.recv_timeout = recv_timeout 62 | 63 | def recv(self): 64 | received_data = b"" 65 | while True: 66 | try: 67 | soc.settimeout(self.recv_timeout) 68 | received_data += soc.recv(self.buffer_size) 69 | 70 | try: 71 | pickle.loads(received_data) 72 | print("All data ({data_len} bytes) is received from the server.".format(data_len=len(received_data)), end="\n") 73 | # If the previous pickle.loads() statement is passed, this means all the data is received. 74 | # Thus, no need to continue the loop and a break statement should be excuted. 75 | break 76 | except BaseException: 77 | # An exception is expected when the data is not 100% received. 78 | pass 79 | 80 | except socket.timeout: 81 | print("A socket.timeout exception occurred because the server did not send any data for {recv_timeout} seconds.".format(recv_timeout=self.recv_timeout)) 82 | print("{recv_timeout} Seconds of Inactivity. socket.timeout Exception Occurred".format(recv_timeout=self.recv_timeout)) 83 | return None, 0 84 | except BaseException as e: 85 | return None, 0 86 | print("Error While Receiving Data from the Server: {msg}.".format(msg=e)) 87 | 88 | try: 89 | received_data = pickle.loads(received_data) 90 | except BaseException as e: 91 | print("Error Decoding the Data: {msg}.\n".format(msg=e)) 92 | return None, 0 93 | 94 | return received_data, 1 95 | 96 | def run(self): 97 | global server_data 98 | 99 | subject = "echo" 100 | server_data = None 101 | best_sol_idx = -1 102 | best_model_weights_vector = None 103 | 104 | while True: 105 | data_dict = {"best_model_weights_vector": best_model_weights_vector} 106 | data = {"subject": subject, "data": data_dict} 107 | 108 | # data = {"subject": subject, "data": keras_ga, "best_solution_idx": best_sol_idx} 109 | data_byte = pickle.dumps(data) 110 | 111 | print("Sending a Message of Type {subject} to the Server".format(subject=subject)) 112 | try: 113 | soc.sendall(data_byte) 114 | except BaseException as e: 115 | print("Error Connecting to the Server. The server might has been closed: {msg}".format(msg=e)) 116 | break 117 | 118 | print("Receiving Reply from the Server") 119 | received_data, status = self.recv() 120 | if status == 0: 121 | print("Nothing Received from the Server") 122 | break 123 | else: 124 | print("New Message from the Server with subject {sub}".format(sub=received_data["subject"])) 125 | 126 | subject = received_data["subject"] 127 | if subject == "model": 128 | server_data = received_data["data"] 129 | elif subject == "done": 130 | print("Model is Trained") 131 | break 132 | else: 133 | print("Unrecognized Message Type: {subject}".format(subject=subject)) 134 | break 135 | 136 | ga_instance = prepare_GA(server_data) 137 | 138 | ga_instance.run() 139 | 140 | subject = "model" 141 | best_sol_idx = ga_instance.best_solution(ga_instance.last_generation_fitness)[2] 142 | best_model_weights_vector = ga_instance.population[best_sol_idx, :] 143 | 144 | predictions = keras_ga.model.predict(data_inputs) 145 | ba = tensorflow.keras.metrics.BinaryAccuracy() 146 | ba.update_state(data_outputs, predictions) 147 | accuracy = ba.result().numpy() 148 | print("Accuracy {acc}".format(acc=accuracy), end="\n\n") 149 | 150 | soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 151 | print("Socket Created") 152 | 153 | try: 154 | soc.connect(("192.168.0.10", int("5000"))) 155 | print("Successful Connection to the Server") 156 | except BaseException as e: 157 | print("Error Connecting to the Server: {msg}".format(msg=e)) 158 | 159 | recvThread = RecvThread(buffer_size=1024, recv_timeout=10) 160 | recvThread.start() 161 | -------------------------------------------------------------------------------- /KerasFederated/Console/server.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import threading 4 | import time 5 | import numpy 6 | 7 | import tensorflow.keras 8 | import pygad.kerasga 9 | 10 | def close_socket(*args): 11 | soc.close() 12 | print("Socket Closed") 13 | 14 | class SocketThread(threading.Thread): 15 | 16 | def __init__(self, connection, client_info, buffer_size=1024, recv_timeout=5): 17 | threading.Thread.__init__(self) 18 | self.connection = connection 19 | self.client_info = client_info 20 | self.buffer_size = buffer_size 21 | self.recv_timeout = recv_timeout 22 | 23 | def recv(self): 24 | all_data_received_flag = False 25 | received_data = b"" 26 | while True: 27 | try: 28 | data = self.connection.recv(self.buffer_size) 29 | received_data += data 30 | 31 | try: 32 | pickle.loads(received_data) 33 | # If the previous pickle.loads() statement is passed, this means all the data is received. 34 | # Thus, no need to continue the loop. The flag all_data_received_flag is set to True to signal all data is received. 35 | all_data_received_flag = True 36 | except BaseException: 37 | # An exception is expected when the data is not 100% received. 38 | pass 39 | 40 | if data == b'': # Nothing received from the client. 41 | received_data = b"" 42 | # If still nothing received for a number of seconds specified by the recv_timeout attribute, return with status 0 to close the connection. 43 | if (time.time() - self.recv_start_time) > self.recv_timeout: 44 | return None, 0 # 0 means the connection is no longer active and it should be closed. 45 | 46 | elif all_data_received_flag: 47 | print("All data ({data_len} bytes) Received from {client_info}.".format(client_info=self.client_info, data_len=len(received_data))) 48 | 49 | if len(received_data) > 0: 50 | try: 51 | # Decoding the data (bytes). 52 | received_data = pickle.loads(received_data) 53 | # Returning the decoded data. 54 | return received_data, 1 55 | 56 | except BaseException as e: 57 | print("Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 58 | return None, 0 59 | 60 | else: 61 | # In case data is received from the client, update the recv_start_time to the current time to reset the timeout counter. 62 | self.recv_start_time = time.time() 63 | 64 | except BaseException as e: 65 | print("Error Receiving Data from the Client: {msg}.\n".format(msg=e)) 66 | return None, 0 67 | 68 | def model_averaging(self, model, best_model_weights_matrix): 69 | model_weights_vector = pygad.kerasga.model_weights_as_vector(model=model) 70 | model_weights_matrix = pygad.kerasga.model_weights_as_matrix(model=model, 71 | weights_vector=model_weights_vector) 72 | 73 | # new_weights = numpy.array(model_weights_matrix + best_model_weights_matrix)/2 74 | new_weights = model_weights_matrix 75 | for idx, arr in enumerate(new_weights): 76 | new_weights[idx] = new_weights[idx] + best_model_weights_matrix[idx] 77 | new_weights[idx] = new_weights[idx] / 2 78 | 79 | model.set_weights(weights=new_weights) 80 | 81 | def reply(self, received_data): 82 | global keras_ga, data_inputs, data_outputs, model 83 | if (type(received_data) is dict): 84 | if (("data" in received_data.keys()) and ("subject" in received_data.keys())): 85 | subject = received_data["subject"] 86 | msg_model = received_data["data"] 87 | print("Client's Message Subject is {subject}.".format(subject=subject)) 88 | 89 | print("Replying to the Client.") 90 | if subject == "echo": 91 | if msg_model is None: 92 | data_dict = {"population_weights": keras_ga.population_weights, 93 | "model_json": model.to_json(), 94 | "num_solutions": keras_ga.num_solutions} 95 | data = {"subject": "model", "data": data_dict} 96 | else: 97 | predictions = model.predict(data_inputs) 98 | ba = tensorflow.keras.metrics.BinaryAccuracy() 99 | ba.update_state(data_outputs, predictions) 100 | accuracy = ba.result().numpy() 101 | 102 | # In case a client sent a model to the server despite that the model accuracy is 1.0. In this case, no need to make changes in the model. 103 | if accuracy == 1.0: 104 | data = {"subject": "done", "data": None} 105 | else: 106 | data_dict = {"population_weights": keras_ga.population_weights, 107 | "model_json": model.to_json(), 108 | "num_solutions": keras_ga.num_solutions} 109 | data = {"subject": "model", "data": data_dict} 110 | try: 111 | response = pickle.dumps(data) 112 | except BaseException as e: 113 | print("Error Encoding the Message: {msg}.\n".format(msg=e)) 114 | elif subject == "model": 115 | try: 116 | best_model_weights_vector = received_data["data"]["best_model_weights_vector"] 117 | 118 | best_model_weights_matrix = pygad.kerasga.model_weights_as_matrix(model=model, 119 | weights_vector=best_model_weights_vector) 120 | if model is None: 121 | print("Model is None") 122 | else: 123 | new_model = tensorflow.keras.models.clone_model(model) 124 | new_model.set_weights(weights=best_model_weights_matrix) 125 | predictions = model.predict(data_inputs) 126 | 127 | ba = tensorflow.keras.metrics.BinaryAccuracy() 128 | ba.update_state(data_outputs, predictions) 129 | accuracy = ba.result().numpy() 130 | 131 | # In case a client sent a model to the server despite that the model accuracy is 1.0. In this case, no need to make changes in the model. 132 | if accuracy == 1.0: 133 | data = {"subject": "done", "data": None} 134 | response = pickle.dumps(data) 135 | return 136 | 137 | self.model_averaging(model, best_model_weights_matrix) 138 | 139 | predictions = model.predict(data_inputs) 140 | print("Model Predictions: {predictions}".format(predictions=predictions)) 141 | 142 | ba = tensorflow.keras.metrics.BinaryAccuracy() 143 | ba.update_state(data_outputs, predictions) 144 | accuracy = ba.result().numpy() 145 | print("Accuracy = {accuracy}\n".format(accuracy=accuracy)) 146 | 147 | if accuracy != 1.0: 148 | data_dict = {"population_weights": keras_ga.population_weights, 149 | "model_json": model.to_json(), 150 | "num_solutions": keras_ga.num_solutions} 151 | data = {"subject": "model", "data": data_dict} 152 | response = pickle.dumps(data) 153 | else: 154 | data = {"subject": "done", "data": None} 155 | response = pickle.dumps(data) 156 | 157 | except BaseException as e: 158 | print("reply(): Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 159 | else: 160 | response = pickle.dumps("Response from the Server") 161 | 162 | try: 163 | self.connection.sendall(response) 164 | except BaseException as e: 165 | print("Error Sending Data to the Client: {msg}.\n".format(msg=e)) 166 | 167 | else: 168 | print("The received dictionary from the client must have the 'subject' and 'data' keys available. The existing keys are {d_keys}.".format(d_keys=received_data.keys())) 169 | print("Error Parsing Received Dictionary") 170 | else: 171 | print("A dictionary is expected to be received from the client but {d_type} received.".format(d_type=type(received_data))) 172 | 173 | def run(self): 174 | print("Running a Thread for the Connection with {client_info}.".format(client_info=self.client_info)) 175 | 176 | # This while loop allows the server to wait for the client to send data more than once within the same connection. 177 | while True: 178 | self.recv_start_time = time.time() 179 | time_struct = time.gmtime() 180 | date_time = "Waiting to Receive Data Starting from {day}/{month}/{year} {hour}:{minute}:{second} GMT".format(year=time_struct.tm_year, month=time_struct.tm_mon, day=time_struct.tm_mday, hour=time_struct.tm_hour, minute=time_struct.tm_min, second=time_struct.tm_sec) 181 | print(date_time) 182 | received_data, status = self.recv() 183 | if status == 0: 184 | self.connection.close() 185 | print("Connection Closed with {client_info} either due to inactivity for {recv_timeout} seconds or due to an error.".format(client_info=self.client_info, recv_timeout=self.recv_timeout), end="\n\n") 186 | break 187 | 188 | # print(received_data) 189 | self.reply(received_data) 190 | 191 | class ListenThread(threading.Thread): 192 | 193 | def __init__(self): 194 | threading.Thread.__init__(self) 195 | 196 | def run(self): 197 | while True: 198 | try: 199 | connection, client_info = soc.accept() 200 | print("\nNew Connection from {client_info}".format(client_info=client_info)) 201 | socket_thread = SocketThread(connection=connection, 202 | client_info=client_info, 203 | buffer_size=1024, 204 | recv_timeout=10) 205 | socket_thread.start() 206 | except BaseException as e: 207 | soc.close() 208 | print("Error in the run() of the ListenThread class: {msg}.\n".format(msg=e)) 209 | print("Socket is No Longer Accepting Connections") 210 | break 211 | 212 | model = None 213 | 214 | # Preparing the NumPy array of the inputs. 215 | data_inputs = numpy.array([[1, 1], 216 | [1, 0], 217 | [0, 1], 218 | [0, 0]]) 219 | 220 | # Preparing the NumPy array of the outputs. 221 | data_outputs = numpy.array([[1, 0], 222 | [0, 1], 223 | [0, 1], 224 | [1, 0]]) 225 | 226 | num_classes = 2 227 | num_inputs = 2 228 | 229 | # Build the keras model using the functional API. 230 | input_layer = tensorflow.keras.layers.Input(num_inputs) 231 | dense_layer = tensorflow.keras.layers.Dense(4, activation="sigmoid")(input_layer) 232 | output_layer = tensorflow.keras.layers.Dense(num_classes, activation="softmax")(dense_layer) 233 | 234 | model = tensorflow.keras.Model(inputs=input_layer, outputs=output_layer) 235 | 236 | num_solutions = 10 237 | # Create an instance of the pygad.kerasga.KerasGA class to build the initial population. 238 | keras_ga = pygad.kerasga.KerasGA(model=model, 239 | num_solutions=num_solutions) 240 | 241 | soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 242 | print("Socket Created") 243 | 244 | ipv4_address = "192.168.0.10" 245 | port_number = "5000" 246 | soc.bind((ipv4_address, int(port_number))) 247 | print("Socket Bound to IPv4 & Port Number") 248 | 249 | soc.listen(1) 250 | print("Socket is Listening for Connections") 251 | 252 | listenThread = ListenThread() 253 | listenThread.start() 254 | -------------------------------------------------------------------------------- /KerasFederated/README.md: -------------------------------------------------------------------------------- 1 | # Federated Learning using Keras and PyGAD 2 | 3 | Training a Keras model using the genetic algorithm ([PyGAD](https://pygad.readthedocs.io)) using federated learning of multiple clients. 4 | 5 | To know more about training Keras models using [PyGAD](https://pygad.readthedocs.io), please read this tutorial: [How To Train Keras Models Using the Genetic Algorithm with PyGAD](https://blog.paperspace.com/train-keras-models-using-genetic-algorithm-with-pygad) 6 | 7 | # Project Files 8 | 9 | The project has the following files: 10 | 11 | - `server.py`: The server Kivy app. It creates a Keras model that is trained on the clients' devices using FL with PyGAD. 12 | - `client1.py`: The client Kivy app which trains the Keras model sent by the server using just 2 samples of the XOR problem. 13 | - `client2.py`: Another client Kivy app that trains the server's Keras model using the other 2 samples in the XOR problem. 14 | 15 | # Install PyGAD 16 | 17 | Before running the project, the [PyGAD](https://pygad.readthedocs.io) library must be installed. 18 | 19 | ``` 20 | pip install pygad 21 | ``` 22 | 23 | # Running the Project 24 | 25 | Start the project by running the [`server.py`](https://github.com/ahmedfgad/FederatedLearning/blob/master/KerasFederated/server.py) file. The GUI of the server Kivy app is shown below. Follow these steps to make sure the server is running and listening for connections. 26 | 27 | * Click on the **Create Socket** button to create a socket. 28 | 29 | * Enter the IPv4 address and port number of the server's socket. `localhost` is used if both the server and the clients are running on the same machine. This is just for testing purposes. Practically, they run on different machines. Thus, the user need to specify the IPv4 address (e.g. 192.168.1.4). 30 | * Click on the **Bind Socket** button to bind the create socket to the entered IPv4 address and port number. 31 | * Click on the **Listen to Connections** button to start listening and accepting incoming connections. Each connected device receives the current model to be trained by its local data. Once the model is trained, then no more models will be sent to the connected devices. 32 | 33 | ![Fig01](https://user-images.githubusercontent.com/16560492/86205885-5af32380-bb6b-11ea-9ca6-149c0170e82b.png) 34 | 35 | After running the server, next is to run one or more clients. The project creates 2 clients but you can add more. The only expected change among the different clients is the data being used for training the model sent by the server. 36 | 37 | For [`client1.py`](https://github.com/ahmedfgad/FederatedLearning/blob/master/KerasFederated/client1.py), here is the training data (2 samples of the XOR problem): 38 | 39 | ```python 40 | # Preparing the NumPy array of the inputs. 41 | data_inputs = numpy.array([[0, 1], 42 | [0, 0]]) 43 | 44 | # Preparing the NumPy array of the outputs. 45 | data_outputs = numpy.array([[0, 1], 46 | [1, 0]]) 47 | ``` 48 | 49 | Here is the training data (other 2 samples of the XOR problem) for the other client ([`client2.py`](https://github.com/ahmedfgad/FederatedLearning/blob/master/KerasFederated/client2.py)): 50 | 51 | ```python 52 | # Preparing the NumPy array of the inputs. 53 | data_inputs = numpy.array([[1, 0], 54 | [1, 1]]) 55 | 56 | # Preparing the NumPy array of the outputs. 57 | data_outputs = numpy.array([[0, 1], 58 | [1, 0]]) 59 | ``` 60 | 61 | Just run any client and a GUI will appear like that. You can either run the client at a desktop or a mobile device. 62 | 63 | Follow these steps to run the client: 64 | 65 | * Click on the **Create Socket** button to create a socket. 66 | 67 | * Enter the IPv4 address and port number of the server's socket. If both the client and the server are running on the same machine, just use `localhost` for the IPv4 address. Otherwise, specify the IPv4 address (e.g. 192.168.1.4).s 68 | * Click on the **Connect to Server** button to create a TCP connection with the server. 69 | * Click on the **Receive & Train Model** button to ask the server to send its current ML model. The model will be trained by the client's local private data. The updated model will be sent back to the server. Once the model is trained, the message **Model is Trained** will appear. 70 | 71 | ![Fig03](https://user-images.githubusercontent.com/16560492/86206222-292e8c80-bb6c-11ea-9311-1ef4bb467188.jpg) 72 | 73 | # For More Information 74 | 75 | There are a number of resources to get started with federated learning and Kivy. 76 | 77 | ## Tutorial: [Introduction to Federated Learning](https://heartbeat.fritz.ai/introduction-to-federated-learning-40eb122754a2) 78 | 79 | This tutorial describes the pipeline of training a machine learning model using federated learning. 80 | 81 | [![](https://miro.medium.com/max/3240/1*6gRmlrDPp5J42HR3QWLYew.jpeg)](https://heartbeat.fritz.ai/introduction-to-federated-learning-40eb122754a2) 82 | 83 | ## Tutorial: [How To Train Keras Models Using the Genetic Algorithm with PyGAD](https://blog.paperspace.com/train-keras-models-using-genetic-algorithm-with-pygad) 84 | 85 | Use [PyGAD](https://pygad.readthedocs.io), a Python 3 easy-to-use genetic algorithm library, to train Keras models using the genetic algorithm. The tutorial is detailed to explain all the steps needed to build and train the model. 86 | 87 | [![](https://user-images.githubusercontent.com/16560492/101267295-c74c0180-375f-11eb-9ad0-f8e37bd796ce.png)](https://blog.paperspace.com/train-keras-models-using-genetic-algorithm-with-pygad) 88 | 89 | ## Tutorial: [Breaking Privacy in Federated Learning](https://heartbeat.fritz.ai/breaking-privacy-in-federated-learning-77fa08ccac9a) 90 | 91 | Even that federated learning does not disclose the private user data, there are some cases in which the privacy of federated learning can be broken. 92 | 93 | [![](https://miro.medium.com/max/3240/1*nZQg-E4a1wOvIH2AmkUUsQ.jpeg)](https://heartbeat.fritz.ai/breaking-privacy-in-federated-learning-77fa08ccac9a) 94 | 95 | ## Tutorial: [Python for Android: Start Building Kivy Cross-Platform Applications](https://www.linkedin.com/pulse/python-android-start-building-kivy-cross-platform-applications-gad) 96 | 97 | This tutorial titled [Python for Android: Start Building Kivy Cross-Platform Applications](https://www.linkedin.com/pulse/python-android-start-building-kivy-cross-platform-applications-gad) covers the steps for creating an Android app out of the Kivy app. 98 | 99 | [![Kivy-Tutorial](https://user-images.githubusercontent.com/16560492/86205332-dfdd3d80-bb69-11ea-91fb-cb0143cb1e5e.png)](https://www.linkedin.com/pulse/python-android-start-building-kivy-cross-platform-applications-gad) 100 | 101 | ## Book: [Building Android Apps in Python Using Kivy with Android Studio](https://www.amazon.com/Building-Android-Python-Using-Studio/dp/1484250303) 102 | 103 | To get started with Kivy app development and how to built Android apps out of the Kivy app, check the book titled [Building Android Apps in Python Using Kivy with Android Studio](https://www.amazon.com/Building-Android-Python-Using-Studio/dp/1484250303) 104 | 105 | [![kivy-book](https://user-images.githubusercontent.com/16560492/86205093-575e9d00-bb69-11ea-82f7-23fef487ce3c.jpg)](https://www.amazon.com/Building-Android-Python-Using-Studio/dp/1484250303) 106 | 107 | # Contact Us 108 | 109 | - E-mail: [ahmed.f.gad@gmail.com](mailto:ahmed.f.gad@gmail.com) 110 | - [LinkedIn](https://www.linkedin.com/in/ahmedfgad) 111 | - [Amazon Author Page](https://amazon.com/author/ahmedgad) 112 | - [Heartbeat](https://heartbeat.fritz.ai/@ahmedfgad) 113 | - [Paperspace](https://blog.paperspace.com/author/ahmed) 114 | - [KDnuggets](https://kdnuggets.com/author/ahmed-gad) 115 | - [TowardsDataScience](https://towardsdatascience.com/@ahmedfgad) 116 | - [GitHub](https://github.com/ahmedfgad) 117 | -------------------------------------------------------------------------------- /KerasFederated/client1.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import numpy 4 | import threading 5 | 6 | import tensorflow.keras 7 | import pygad.kerasga 8 | import pygad 9 | 10 | import kivy.app 11 | import kivy.uix.button 12 | import kivy.uix.label 13 | import kivy.uix.boxlayout 14 | import kivy.uix.textinput 15 | 16 | class ClientApp(kivy.app.App): 17 | 18 | def __init__(self): 19 | super().__init__() 20 | 21 | def create_socket(self, *args): 22 | self.soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 23 | self.label.text = "Socket Created" 24 | 25 | self.create_socket_btn.disabled = True 26 | self.connect_btn.disabled = False 27 | self.close_socket_btn.disabled = False 28 | 29 | def connect(self, *args): 30 | try: 31 | self.soc.connect((self.server_ip.text, int(self.server_port.text))) 32 | self.label.text = "Successful Connection to the Server" 33 | 34 | self.connect_btn.disabled = True 35 | self.recv_train_model_btn.disabled = False 36 | 37 | except BaseException as e: 38 | self.label.text = "Error Connecting to the Server" 39 | print("Error Connecting to the Server: {msg}".format(msg=e)) 40 | 41 | self.connect_btn.disabled = False 42 | self.recv_train_model_btn.disabled = True 43 | 44 | def recv_train_model(self, *args): 45 | global keras_ga 46 | 47 | self.recv_train_model_btn.disabled = True 48 | recvThread = RecvThread(kivy_app=self, buffer_size=1024, recv_timeout=10) 49 | recvThread.start() 50 | 51 | def close_socket(self, *args): 52 | self.soc.close() 53 | self.label.text = "Socket Closed" 54 | 55 | self.create_socket_btn.disabled = False 56 | self.connect_btn.disabled = True 57 | self.recv_train_model_btn.disabled = True 58 | self.close_socket_btn.disabled = True 59 | 60 | def build(self): 61 | self.create_socket_btn = kivy.uix.button.Button(text="Create Socket") 62 | self.create_socket_btn.bind(on_press=self.create_socket) 63 | 64 | self.server_ip = kivy.uix.textinput.TextInput(hint_text="Server IPv4 Address", text="localhost") 65 | self.server_port = kivy.uix.textinput.TextInput(hint_text="Server Port Number", text="10000") 66 | 67 | self.server_info_boxlayout = kivy.uix.boxlayout.BoxLayout(orientation="horizontal") 68 | self.server_info_boxlayout.add_widget(self.server_ip) 69 | self.server_info_boxlayout.add_widget(self.server_port) 70 | 71 | self.connect_btn = kivy.uix.button.Button(text="Connect to Server", disabled=True) 72 | self.connect_btn.bind(on_press=self.connect) 73 | 74 | self.recv_train_model_btn = kivy.uix.button.Button(text="Receive & Train Model", disabled=True) 75 | self.recv_train_model_btn.bind(on_press=self.recv_train_model) 76 | 77 | self.close_socket_btn = kivy.uix.button.Button(text="Close Socket", disabled=True) 78 | self.close_socket_btn.bind(on_press=self.close_socket) 79 | 80 | self.label = kivy.uix.label.Label(text="Socket Status") 81 | 82 | self.box_layout = kivy.uix.boxlayout.BoxLayout(orientation="vertical") 83 | self.box_layout.add_widget(self.create_socket_btn) 84 | self.box_layout.add_widget(self.server_info_boxlayout) 85 | self.box_layout.add_widget(self.connect_btn) 86 | self.box_layout.add_widget(self.recv_train_model_btn) 87 | self.box_layout.add_widget(self.close_socket_btn) 88 | self.box_layout.add_widget(self.label) 89 | 90 | return self.box_layout 91 | 92 | def fitness_func(solution, sol_idx): 93 | global keras_ga, data_inputs, data_outputs 94 | 95 | model = keras_ga.model 96 | 97 | model_weights_matrix = pygad.kerasga.model_weights_as_matrix(model=model, 98 | weights_vector=solution) 99 | model.set_weights(weights=model_weights_matrix) 100 | predictions = model.predict(data_inputs) 101 | bce = tensorflow.keras.losses.BinaryCrossentropy() 102 | solution_fitness = 1.0 / (bce(data_outputs, predictions).numpy() + 0.00000001) 103 | 104 | return solution_fitness 105 | 106 | """ 107 | def callback_generation(ga_instance): 108 | global GANN_instance, last_fitness 109 | 110 | population_matrices = pygad.gann.population_as_matrices(population_networks=GANN_instance.population_networks, 111 | population_vectors=ga_instance.population) 112 | 113 | GANN_instance.update_population_trained_weights(population_trained_weights=population_matrices) 114 | 115 | # print("Generation = {generation}".format(generation=ga_instance.generations_completed)) 116 | # print("Fitness = {fitness}".format(fitness=ga_instance.best_solution()[1])) 117 | # print("Change = {change}".format(change=ga_instance.best_solution()[1] - last_fitness)) 118 | """ 119 | 120 | #last_fitness = 0 121 | 122 | def prepare_GA(server_data): 123 | global keras_ga 124 | 125 | population_weights = server_data["population_weights"] 126 | model_json = server_data["model_json"] 127 | num_solutions = server_data["num_solutions"] 128 | 129 | model = tensorflow.keras.models.model_from_json(model_json) 130 | keras_ga = pygad.kerasga.KerasGA(model=model, 131 | num_solutions=num_solutions) 132 | 133 | keras_ga.population_weights = population_weights 134 | 135 | population_vectors = keras_ga.population_weights 136 | 137 | # To prepare the initial population, there are 2 ways: 138 | # 1) Prepare it yourself and pass it to the initial_population parameter. This way is useful when the user wants to start the genetic algorithm with a custom initial population. 139 | # 2) Assign valid integer values to the sol_per_pop and num_genes parameters. If the initial_population parameter exists, then the sol_per_pop and num_genes parameters are useless. 140 | initial_population = population_vectors.copy() 141 | 142 | num_parents_mating = 4 # Number of solutions to be selected as parents in the mating pool. 143 | 144 | num_generations = 50 # Number of generations. 145 | 146 | mutation_percent_genes = 5 # Percentage of genes to mutate. This parameter has no action if the parameter mutation_num_genes exists. 147 | 148 | ga_instance = pygad.GA(num_generations=num_generations, 149 | num_parents_mating=num_parents_mating, 150 | initial_population=initial_population, 151 | fitness_func=fitness_func, 152 | mutation_percent_genes=mutation_percent_genes) 153 | 154 | return ga_instance 155 | 156 | # Preparing the NumPy array of the inputs. 157 | data_inputs = numpy.array([[0, 1], 158 | [0, 0]]) 159 | 160 | # Preparing the NumPy array of the outputs. 161 | data_outputs = numpy.array([[0, 1], 162 | [1, 0]]) 163 | 164 | class RecvThread(threading.Thread): 165 | 166 | def __init__(self, kivy_app, buffer_size, recv_timeout): 167 | threading.Thread.__init__(self) 168 | self.kivy_app = kivy_app 169 | self.buffer_size = buffer_size 170 | self.recv_timeout = recv_timeout 171 | 172 | def recv(self): 173 | received_data = b"" 174 | while True: # str(received_data)[-2] != '.': 175 | try: 176 | self.kivy_app.soc.settimeout(self.recv_timeout) 177 | received_data += self.kivy_app.soc.recv(self.buffer_size) 178 | 179 | try: 180 | pickle.loads(received_data) 181 | self.kivy_app.label.text = "All data is received from the server." 182 | print("All data is received from the server.") 183 | # If the previous pickle.loads() statement is passed, this means all the data is received. 184 | # Thus, no need to continue the loop and a break statement should be excuted. 185 | break 186 | except BaseException: 187 | # An exception is expected when the data is not 100% received. 188 | pass 189 | 190 | except socket.timeout: 191 | print("A socket.timeout exception occurred because the server did not send any data for {recv_timeout} seconds.".format(recv_timeout=self.recv_timeout)) 192 | self.kivy_app.label.text = "{recv_timeout} Seconds of Inactivity. socket.timeout Exception Occurred".format(recv_timeout=self.recv_timeout) 193 | return None, 0 194 | except BaseException as e: 195 | return None, 0 196 | print("Error While Receiving Data from the Server: {msg}.".format(msg=e)) 197 | self.kivy_app.label.text = "Error While Receiving Data from the Server" 198 | 199 | try: 200 | received_data = pickle.loads(received_data) 201 | except BaseException as e: 202 | print("Error Decoding the Data: {msg}.\n".format(msg=e)) 203 | self.kivy_app.label.text = "Error Decoding the Client's Data" 204 | return None, 0 205 | 206 | return received_data, 1 207 | 208 | def run(self): 209 | global server_data 210 | 211 | subject = "echo" 212 | server_data = None 213 | best_sol_idx = -1 214 | best_model_weights_vector = None 215 | 216 | while True: 217 | data_dict = {"best_model_weights_vector": best_model_weights_vector} 218 | data = {"subject": subject, "data": data_dict} 219 | 220 | # data = {"subject": subject, "data": keras_ga, "best_solution_idx": best_sol_idx} 221 | data_byte = pickle.dumps(data) 222 | 223 | self.kivy_app.label.text = "Sending a Message of Type {subject} to the Server".format(subject=subject) 224 | try: 225 | self.kivy_app.soc.sendall(data_byte) 226 | except BaseException as e: 227 | self.kivy_app.label.text = "Error Connecting to the Server. The server might has been closed." 228 | print("Error Connecting to the Server: {msg}".format(msg=e)) 229 | break 230 | 231 | self.kivy_app.label.text = "Receiving Reply from the Server" 232 | received_data, status = self.recv() 233 | if status == 0: 234 | self.kivy_app.label.text = "Nothing Received from the Server" 235 | break 236 | else: 237 | self.kivy_app.label.text = "New Message from the Server" 238 | 239 | subject = received_data["subject"] 240 | if subject == "model": 241 | server_data = received_data["data"] 242 | elif subject == "done": 243 | self.kivy_app.label.text = "Model is Trained" 244 | break 245 | else: 246 | self.kivy_app.label.text = "Unrecognized Message Type: {subject}".format(subject=subject) 247 | break 248 | 249 | ga_instance = prepare_GA(server_data) 250 | 251 | ga_instance.run() 252 | 253 | subject = "model" 254 | best_sol_idx = ga_instance.best_solution()[2] 255 | best_model_weights_vector = ga_instance.population[best_sol_idx, :] 256 | 257 | clientApp = ClientApp() 258 | clientApp.title = "Client App" 259 | clientApp.run() 260 | -------------------------------------------------------------------------------- /KerasFederated/client2.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import numpy 4 | import threading 5 | 6 | import tensorflow.keras 7 | import pygad.kerasga 8 | import pygad 9 | 10 | import kivy.app 11 | import kivy.uix.button 12 | import kivy.uix.label 13 | import kivy.uix.boxlayout 14 | import kivy.uix.textinput 15 | 16 | class ClientApp(kivy.app.App): 17 | 18 | def __init__(self): 19 | super().__init__() 20 | 21 | def create_socket(self, *args): 22 | self.soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 23 | self.label.text = "Socket Created" 24 | 25 | self.create_socket_btn.disabled = True 26 | self.connect_btn.disabled = False 27 | self.close_socket_btn.disabled = False 28 | 29 | def connect(self, *args): 30 | try: 31 | self.soc.connect((self.server_ip.text, int(self.server_port.text))) 32 | self.label.text = "Successful Connection to the Server" 33 | 34 | self.connect_btn.disabled = True 35 | self.recv_train_model_btn.disabled = False 36 | 37 | except BaseException as e: 38 | self.label.text = "Error Connecting to the Server" 39 | print("Error Connecting to the Server: {msg}".format(msg=e)) 40 | 41 | self.connect_btn.disabled = False 42 | self.recv_train_model_btn.disabled = True 43 | 44 | def recv_train_model(self, *args): 45 | global keras_ga 46 | 47 | self.recv_train_model_btn.disabled = True 48 | recvThread = RecvThread(kivy_app=self, buffer_size=1024, recv_timeout=10) 49 | recvThread.start() 50 | 51 | def close_socket(self, *args): 52 | self.soc.close() 53 | self.label.text = "Socket Closed" 54 | 55 | self.create_socket_btn.disabled = False 56 | self.connect_btn.disabled = True 57 | self.recv_train_model_btn.disabled = True 58 | self.close_socket_btn.disabled = True 59 | 60 | def build(self): 61 | self.create_socket_btn = kivy.uix.button.Button(text="Create Socket") 62 | self.create_socket_btn.bind(on_press=self.create_socket) 63 | 64 | self.server_ip = kivy.uix.textinput.TextInput(hint_text="Server IPv4 Address", text="localhost") 65 | self.server_port = kivy.uix.textinput.TextInput(hint_text="Server Port Number", text="10000") 66 | 67 | self.server_info_boxlayout = kivy.uix.boxlayout.BoxLayout(orientation="horizontal") 68 | self.server_info_boxlayout.add_widget(self.server_ip) 69 | self.server_info_boxlayout.add_widget(self.server_port) 70 | 71 | self.connect_btn = kivy.uix.button.Button(text="Connect to Server", disabled=True) 72 | self.connect_btn.bind(on_press=self.connect) 73 | 74 | self.recv_train_model_btn = kivy.uix.button.Button(text="Receive & Train Model", disabled=True) 75 | self.recv_train_model_btn.bind(on_press=self.recv_train_model) 76 | 77 | self.close_socket_btn = kivy.uix.button.Button(text="Close Socket", disabled=True) 78 | self.close_socket_btn.bind(on_press=self.close_socket) 79 | 80 | self.label = kivy.uix.label.Label(text="Socket Status") 81 | 82 | self.box_layout = kivy.uix.boxlayout.BoxLayout(orientation="vertical") 83 | self.box_layout.add_widget(self.create_socket_btn) 84 | self.box_layout.add_widget(self.server_info_boxlayout) 85 | self.box_layout.add_widget(self.connect_btn) 86 | self.box_layout.add_widget(self.recv_train_model_btn) 87 | self.box_layout.add_widget(self.close_socket_btn) 88 | self.box_layout.add_widget(self.label) 89 | 90 | return self.box_layout 91 | 92 | def fitness_func(solution, sol_idx): 93 | global keras_ga, data_inputs, data_outputs 94 | 95 | model = keras_ga.model 96 | 97 | model_weights_matrix = pygad.kerasga.model_weights_as_matrix(model=model, 98 | weights_vector=solution) 99 | model.set_weights(weights=model_weights_matrix) 100 | predictions = model.predict(data_inputs) 101 | bce = tensorflow.keras.losses.BinaryCrossentropy() 102 | solution_fitness = 1.0 / (bce(data_outputs, predictions).numpy() + 0.00000001) 103 | 104 | return solution_fitness 105 | 106 | """ 107 | def callback_generation(ga_instance): 108 | global GANN_instance, last_fitness 109 | 110 | population_matrices = pygad.gann.population_as_matrices(population_networks=GANN_instance.population_networks, 111 | population_vectors=ga_instance.population) 112 | 113 | GANN_instance.update_population_trained_weights(population_trained_weights=population_matrices) 114 | 115 | # print("Generation = {generation}".format(generation=ga_instance.generations_completed)) 116 | # print("Fitness = {fitness}".format(fitness=ga_instance.best_solution()[1])) 117 | # print("Change = {change}".format(change=ga_instance.best_solution()[1] - last_fitness)) 118 | """ 119 | 120 | #last_fitness = 0 121 | 122 | def prepare_GA(server_data): 123 | global keras_ga 124 | 125 | population_weights = server_data["population_weights"] 126 | model_json = server_data["model_json"] 127 | num_solutions = server_data["num_solutions"] 128 | 129 | model = tensorflow.keras.models.model_from_json(model_json) 130 | keras_ga = pygad.kerasga.KerasGA(model=model, 131 | num_solutions=num_solutions) 132 | 133 | keras_ga.population_weights = population_weights 134 | 135 | population_vectors = keras_ga.population_weights 136 | 137 | # To prepare the initial population, there are 2 ways: 138 | # 1) Prepare it yourself and pass it to the initial_population parameter. This way is useful when the user wants to start the genetic algorithm with a custom initial population. 139 | # 2) Assign valid integer values to the sol_per_pop and num_genes parameters. If the initial_population parameter exists, then the sol_per_pop and num_genes parameters are useless. 140 | initial_population = population_vectors.copy() 141 | 142 | num_parents_mating = 4 # Number of solutions to be selected as parents in the mating pool. 143 | 144 | num_generations = 50 # Number of generations. 145 | 146 | mutation_percent_genes = 5 # Percentage of genes to mutate. This parameter has no action if the parameter mutation_num_genes exists. 147 | 148 | ga_instance = pygad.GA(num_generations=num_generations, 149 | num_parents_mating=num_parents_mating, 150 | initial_population=initial_population, 151 | fitness_func=fitness_func, 152 | mutation_percent_genes=mutation_percent_genes) 153 | 154 | return ga_instance 155 | 156 | # Preparing the NumPy array of the inputs. 157 | data_inputs = numpy.array([[1, 0], 158 | [1, 1]]) 159 | 160 | # Preparing the NumPy array of the outputs. 161 | data_outputs = numpy.array([[0, 1], 162 | [1, 0]]) 163 | 164 | class RecvThread(threading.Thread): 165 | 166 | def __init__(self, kivy_app, buffer_size, recv_timeout): 167 | threading.Thread.__init__(self) 168 | self.kivy_app = kivy_app 169 | self.buffer_size = buffer_size 170 | self.recv_timeout = recv_timeout 171 | 172 | def recv(self): 173 | received_data = b"" 174 | while True: # str(received_data)[-2] != '.': 175 | try: 176 | self.kivy_app.soc.settimeout(self.recv_timeout) 177 | received_data += self.kivy_app.soc.recv(self.buffer_size) 178 | 179 | try: 180 | pickle.loads(received_data) 181 | self.kivy_app.label.text = "All data is received from the server." 182 | print("All data is received from the server.") 183 | # If the previous pickle.loads() statement is passed, this means all the data is received. 184 | # Thus, no need to continue the loop and a break statement should be excuted. 185 | break 186 | except BaseException: 187 | # An exception is expected when the data is not 100% received. 188 | pass 189 | 190 | except socket.timeout: 191 | print("A socket.timeout exception occurred because the server did not send any data for {recv_timeout} seconds.".format(recv_timeout=self.recv_timeout)) 192 | self.kivy_app.label.text = "{recv_timeout} Seconds of Inactivity. socket.timeout Exception Occurred".format(recv_timeout=self.recv_timeout) 193 | return None, 0 194 | except BaseException as e: 195 | return None, 0 196 | print("Error While Receiving Data from the Server: {msg}.".format(msg=e)) 197 | self.kivy_app.label.text = "Error While Receiving Data from the Server" 198 | 199 | try: 200 | received_data = pickle.loads(received_data) 201 | except BaseException as e: 202 | print("Error Decoding the Data: {msg}.\n".format(msg=e)) 203 | self.kivy_app.label.text = "Error Decoding the Client's Data" 204 | return None, 0 205 | 206 | return received_data, 1 207 | 208 | def run(self): 209 | global server_data 210 | 211 | subject = "echo" 212 | server_data = None 213 | best_sol_idx = -1 214 | best_model_weights_vector = None 215 | 216 | while True: 217 | data_dict = {"best_model_weights_vector": best_model_weights_vector} 218 | data = {"subject": subject, "data": data_dict} 219 | 220 | # data = {"subject": subject, "data": keras_ga, "best_solution_idx": best_sol_idx} 221 | data_byte = pickle.dumps(data) 222 | 223 | self.kivy_app.label.text = "Sending a Message of Type {subject} to the Server".format(subject=subject) 224 | try: 225 | self.kivy_app.soc.sendall(data_byte) 226 | except BaseException as e: 227 | self.kivy_app.label.text = "Error Connecting to the Server. The server might has been closed." 228 | print("Error Connecting to the Server: {msg}".format(msg=e)) 229 | break 230 | 231 | self.kivy_app.label.text = "Receiving Reply from the Server" 232 | received_data, status = self.recv() 233 | if status == 0: 234 | self.kivy_app.label.text = "Nothing Received from the Server" 235 | break 236 | else: 237 | self.kivy_app.label.text = "New Message from the Server" 238 | 239 | subject = received_data["subject"] 240 | if subject == "model": 241 | server_data = received_data["data"] 242 | elif subject == "done": 243 | self.kivy_app.label.text = "Model is Trained" 244 | break 245 | else: 246 | self.kivy_app.label.text = "Unrecognized Message Type: {subject}".format(subject=subject) 247 | break 248 | 249 | ga_instance = prepare_GA(server_data) 250 | 251 | ga_instance.run() 252 | 253 | subject = "model" 254 | best_sol_idx = ga_instance.best_solution()[2] 255 | best_model_weights_vector = ga_instance.population[best_sol_idx, :] 256 | 257 | clientApp = ClientApp() 258 | clientApp.title = "Client App" 259 | clientApp.run() 260 | -------------------------------------------------------------------------------- /KerasFederated/server.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import threading 4 | import time 5 | import numpy 6 | 7 | import tensorflow.keras 8 | import pygad.kerasga 9 | 10 | import kivy.app 11 | import kivy.uix.button 12 | import kivy.uix.label 13 | import kivy.uix.textinput 14 | import kivy.uix.boxlayout 15 | 16 | class ServerApp(kivy.app.App): 17 | 18 | def __init__(self): 19 | super().__init__() 20 | 21 | def create_socket(self, *args): 22 | self.soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 23 | self.label.text = "Socket Created" 24 | 25 | self.create_socket_btn.disabled = True 26 | self.bind_btn.disabled = False 27 | self.close_socket_btn.disabled = False 28 | 29 | def bind_socket(self, *args): 30 | ipv4_address = self.server_ip.text 31 | port_number = self.server_port.text 32 | self.soc.bind((ipv4_address, int(port_number))) 33 | self.label.text = "Socket Bound to IPv4 & Port Number" 34 | 35 | self.bind_btn.disabled = True 36 | self.listen_btn.disabled = False 37 | 38 | def listen_accept(self, *args): 39 | self.soc.listen(1) 40 | self.label.text = "Socket is Listening for Connections" 41 | 42 | self.listen_btn.disabled = True 43 | 44 | self.listenThread = ListenThread(kivy_app=self) 45 | self.listenThread.start() 46 | 47 | def close_socket(self, *args): 48 | self.soc.close() 49 | self.label.text = "Socket Closed" 50 | 51 | self.create_socket_btn.disabled = False 52 | self.bind_btn.disabled = True 53 | self.listen_btn.disabled = True 54 | self.close_socket_btn.disabled = True 55 | 56 | def build(self): 57 | self.create_socket_btn = kivy.uix.button.Button(text="Create Socket", disabled=False) 58 | self.create_socket_btn.bind(on_press=self.create_socket) 59 | 60 | self.server_ip = kivy.uix.textinput.TextInput(hint_text="IPv4 Address", text="localhost") 61 | self.server_port = kivy.uix.textinput.TextInput(hint_text="Port Number", text="10000") 62 | 63 | self.server_socket_box_layout = kivy.uix.boxlayout.BoxLayout(orientation="horizontal") 64 | self.server_socket_box_layout.add_widget(self.server_ip) 65 | self.server_socket_box_layout.add_widget(self.server_port) 66 | 67 | self.bind_btn = kivy.uix.button.Button(text="Bind Socket", disabled=True) 68 | self.bind_btn.bind(on_press=self.bind_socket) 69 | 70 | self.listen_btn = kivy.uix.button.Button(text="Listen to Connections", disabled=True) 71 | self.listen_btn.bind(on_press=self.listen_accept) 72 | 73 | self.close_socket_btn = kivy.uix.button.Button(text="Close Socket", disabled=True) 74 | self.close_socket_btn.bind(on_press=self.close_socket) 75 | 76 | self.label = kivy.uix.label.Label(text="Socket Status") 77 | 78 | self.box_layout = kivy.uix.boxlayout.BoxLayout(orientation="vertical") 79 | 80 | self.box_layout.add_widget(self.create_socket_btn) 81 | self.box_layout.add_widget(self.server_socket_box_layout) 82 | self.box_layout.add_widget(self.bind_btn) 83 | self.box_layout.add_widget(self.listen_btn) 84 | self.box_layout.add_widget(self.close_socket_btn) 85 | self.box_layout.add_widget(self.label) 86 | 87 | return self.box_layout 88 | 89 | model = None 90 | 91 | # Preparing the NumPy array of the inputs. 92 | data_inputs = numpy.array([[1, 1], 93 | [1, 0], 94 | [0, 1], 95 | [0, 0]]) 96 | 97 | # Preparing the NumPy array of the outputs. 98 | data_outputs = numpy.array([[1, 0], 99 | [0, 1], 100 | [0, 1], 101 | [1, 0]]) 102 | 103 | num_classes = 2 104 | num_inputs = 2 105 | 106 | # Build the keras model using the functional API. 107 | input_layer = tensorflow.keras.layers.Input(num_inputs) 108 | dense_layer = tensorflow.keras.layers.Dense(4, activation="relu")(input_layer) 109 | output_layer = tensorflow.keras.layers.Dense(num_classes, activation="softmax")(dense_layer) 110 | 111 | model = tensorflow.keras.Model(inputs=input_layer, outputs=output_layer) 112 | 113 | num_solutions = 10 114 | # Create an instance of the pygad.kerasga.KerasGA class to build the initial population. 115 | keras_ga = pygad.kerasga.KerasGA(model=model, 116 | num_solutions=num_solutions) 117 | 118 | class SocketThread(threading.Thread): 119 | 120 | def __init__(self, connection, client_info, kivy_app, buffer_size=1024, recv_timeout=5): 121 | threading.Thread.__init__(self) 122 | self.connection = connection 123 | self.client_info = client_info 124 | self.buffer_size = buffer_size 125 | self.recv_timeout = recv_timeout 126 | self.kivy_app = kivy_app 127 | 128 | def recv(self): 129 | all_data_received_flag = False 130 | received_data = b"" 131 | while True: 132 | try: 133 | data = self.connection.recv(self.buffer_size) 134 | received_data += data 135 | 136 | try: 137 | pickle.loads(received_data) 138 | # If the previous pickle.loads() statement is passed, this means all the data is received. 139 | # Thus, no need to continue the loop. The flag all_data_received_flag is set to True to signal all data is received. 140 | all_data_received_flag = True 141 | except BaseException: 142 | # An exception is expected when the data is not 100% received. 143 | pass 144 | 145 | if data == b'': # Nothing received from the client. 146 | received_data = b"" 147 | # If still nothing received for a number of seconds specified by the recv_timeout attribute, return with status 0 to close the connection. 148 | if (time.time() - self.recv_start_time) > self.recv_timeout: 149 | return None, 0 # 0 means the connection is no longer active and it should be closed. 150 | 151 | elif all_data_received_flag: 152 | print("All data ({data_len} bytes) Received from {client_info}.".format(client_info=self.client_info, data_len=len(received_data))) 153 | self.kivy_app.label.text = "All data ({data_len} bytes) Received from {client_info}.".format(client_info=self.client_info, data_len=len(received_data)) 154 | 155 | if len(received_data) > 0: 156 | try: 157 | # Decoding the data (bytes). 158 | received_data = pickle.loads(received_data) 159 | # Returning the decoded data. 160 | return received_data, 1 161 | 162 | except BaseException as e: 163 | print("Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 164 | self.kivy_app.label.text = "Error Decoding the Client's Data" 165 | return None, 0 166 | 167 | else: 168 | # In case data are received from the client, update the recv_start_time to the current time to reset the timeout counter. 169 | self.recv_start_time = time.time() 170 | 171 | except BaseException as e: 172 | print("Error Receiving Data from the Client: {msg}.\n".format(msg=e)) 173 | self.kivy_app.label.text = "Error Receiving Data from the Client" 174 | return None, 0 175 | 176 | def model_averaging(self, model, best_model_weights_matrix): 177 | model_weights_vector = pygad.kerasga.model_weights_as_vector(model=model) 178 | model_weights_matrix = pygad.kerasga.model_weights_as_matrix(model=model, 179 | weights_vector=model_weights_vector) 180 | 181 | # new_weights = numpy.array(model_weights_matrix + best_model_weights_matrix)/2 182 | new_weights = model_weights_matrix 183 | for idx, arr in enumerate(new_weights): 184 | new_weights[idx] = new_weights[idx] + best_model_weights_matrix[idx] 185 | new_weights[idx] = new_weights[idx] / 2 186 | 187 | # for idx, layer in enumerate(model.layers): 188 | # print(new_weights[idx].shape, model.weights[idx].shape) 189 | 190 | model.set_weights(weights=new_weights) 191 | 192 | def reply(self, received_data): 193 | global keras_ga, data_inputs, data_outputs, model 194 | if (type(received_data) is dict): 195 | if (("data" in received_data.keys()) and ("subject" in received_data.keys())): 196 | subject = received_data["subject"] 197 | msg_model = received_data["data"] 198 | print("Client's Message Subject is {subject}.".format(subject=subject)) 199 | self.kivy_app.label.text = "Client's Message Subject is {subject}".format(subject=subject) 200 | 201 | print("Replying to the Client.") 202 | self.kivy_app.label.text = "Replying to the Client" 203 | if subject == "echo": 204 | if msg_model is None: 205 | data_dict = {"population_weights": keras_ga.population_weights, 206 | "model_json": model.to_json(), 207 | "num_solutions": keras_ga.num_solutions} 208 | data = {"subject": "model", "data": data_dict} 209 | else: 210 | predictions = model.predict(data_inputs) 211 | ba = tensorflow.keras.metrics.BinaryAccuracy() 212 | ba.update_state(data_outputs, predictions) 213 | accuracy = ba.result().numpy() 214 | 215 | # In case a client sent a model to the server despite that the model accuracy is 1.0. In this case, no need to make changes in the model. 216 | if accuracy == 1.0: 217 | data = {"subject": "done", "data": None} 218 | else: 219 | data_dict = {"population_weights": keras_ga.population_weights, 220 | "model_json": model.to_json(), 221 | "num_solutions": keras_ga.num_solutions} 222 | data = {"subject": "model", "data": data_dict} 223 | try: 224 | response = pickle.dumps(data) 225 | except BaseException as e: 226 | print("Error Encoding the Message: {msg}.\n".format(msg=e)) 227 | self.kivy_app.label.text = "Error Encoding the Message" 228 | elif subject == "model": 229 | try: 230 | best_model_weights_vector = received_data["data"]["best_model_weights_vector"] 231 | # keras_ga.population_weights = population_weights 232 | # keras_ga = received_data["data"] 233 | # best_model_idx = received_data["best_solution_idx"] 234 | 235 | # best_model_weights_vector = keras_ga.population_weights[best_model_idx] 236 | best_model_weights_matrix = pygad.kerasga.model_weights_as_matrix(model=model, 237 | weights_vector=best_model_weights_vector) 238 | if model is None: 239 | print("Model is None") 240 | else: 241 | new_model = tensorflow.keras.models.clone_model(model) 242 | new_model.set_weights(weights=best_model_weights_matrix) 243 | predictions = model.predict(data_inputs) 244 | 245 | ba = tensorflow.keras.metrics.BinaryAccuracy() 246 | ba.update_state(data_outputs, predictions) 247 | accuracy = ba.result().numpy() 248 | 249 | # In case a client sent a model to the server despite that the model accuracy is 1.0. In this case, no need to make changes in the model. 250 | if accuracy == 1.0: 251 | data = {"subject": "done", "data": None} 252 | response = pickle.dumps(data) 253 | return 254 | 255 | self.model_averaging(model, best_model_weights_matrix) 256 | 257 | # print(best_model.trained_weights) 258 | # print(model.trained_weights) 259 | 260 | predictions = model.predict(data_inputs) 261 | print("Model Predictions: {predictions}".format(predictions=predictions)) 262 | 263 | ba = tensorflow.keras.metrics.BinaryAccuracy() 264 | ba.update_state(data_outputs, predictions) 265 | accuracy = ba.result().numpy() 266 | print("Accuracy = {accuracy}\n".format(accuracy=accuracy)) 267 | self.kivy_app.label.text = "Accuracy = {accuracy}".format(accuracy=accuracy) 268 | 269 | if accuracy != 1.0: 270 | data_dict = {"population_weights": keras_ga.population_weights, 271 | "model_json": model.to_json(), 272 | "num_solutions": keras_ga.num_solutions} 273 | data = {"subject": "model", "data": data_dict} 274 | response = pickle.dumps(data) 275 | else: 276 | data = {"subject": "done", "data": None} 277 | response = pickle.dumps(data) 278 | 279 | except BaseException as e: 280 | print("reply(): Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 281 | self.kivy_app.label.text = "reply(): Error Decoding the Client's Data" 282 | else: 283 | response = pickle.dumps("Response from the Server") 284 | 285 | try: 286 | self.connection.sendall(response) 287 | except BaseException as e: 288 | print("Error Sending Data to the Client: {msg}.\n".format(msg=e)) 289 | self.kivy_app.label.text = "Error Sending Data to the Client: {msg}".format(msg=e) 290 | 291 | else: 292 | print("The received dictionary from the client must have the 'subject' and 'data' keys available. The existing keys are {d_keys}.".format(d_keys=received_data.keys())) 293 | self.kivy_app.label.text = "Error Parsing Received Dictionary" 294 | else: 295 | print("A dictionary is expected to be received from the client but {d_type} received.".format(d_type=type(received_data))) 296 | self.kivy_app.label.text = "A dictionary is expected but {d_type} received.".format(d_type=type(received_data)) 297 | 298 | def run(self): 299 | print("Running a Thread for the Connection with {client_info}.".format(client_info=self.client_info)) 300 | self.kivy_app.label.text = "Running a Thread for the Connection with {client_info}.".format(client_info=self.client_info) 301 | 302 | # This while loop allows the server to wait for the client to send data more than once within the same connection. 303 | while True: 304 | self.recv_start_time = time.time() 305 | time_struct = time.gmtime() 306 | date_time = "Waiting to Receive Data Starting from {day}/{month}/{year} {hour}:{minute}:{second} GMT".format(year=time_struct.tm_year, month=time_struct.tm_mon, day=time_struct.tm_mday, hour=time_struct.tm_hour, minute=time_struct.tm_min, second=time_struct.tm_sec) 307 | print(date_time) 308 | received_data, status = self.recv() 309 | if status == 0: 310 | self.connection.close() 311 | self.kivy_app.label.text = "Connection Closed with {client_info}".format(client_info=self.client_info) 312 | print("Connection Closed with {client_info} either due to inactivity for {recv_timeout} seconds or due to an error.".format(client_info=self.client_info, recv_timeout=self.recv_timeout), end="\n\n") 313 | break 314 | 315 | # print(received_data) 316 | self.reply(received_data) 317 | 318 | class ListenThread(threading.Thread): 319 | 320 | def __init__(self, kivy_app): 321 | threading.Thread.__init__(self) 322 | self.kivy_app = kivy_app 323 | 324 | def run(self): 325 | while True: 326 | try: 327 | connection, client_info = self.kivy_app.soc.accept() 328 | self.kivy_app.label.text = "New Connection from {client_info}".format(client_info=client_info) 329 | socket_thread = SocketThread(connection=connection, 330 | client_info=client_info, 331 | kivy_app=self.kivy_app, 332 | buffer_size=1024, 333 | recv_timeout=10) 334 | socket_thread.start() 335 | except BaseException as e: 336 | self.kivy_app.soc.close() 337 | print("Error in the run() of the ListenThread class: {msg}.\n".format(msg=e)) 338 | self.kivy_app.label.text = "Socket is No Longer Accepting Connections" 339 | self.kivy_app.create_socket_btn.disabled = False 340 | self.kivy_app.close_socket_btn.disabled = True 341 | break 342 | 343 | serverApp = ServerApp() 344 | serverApp.title="Server App" 345 | serverApp.run() -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Federated Learning Demo in Python using Socket Programming 2 | 3 | This is a demo project for applying the concepts of federated learning (FL) in Python using socket programming by building and training machine learning (ML) models using FL. The ML model is trained using [PyGAD](https://pygad.readthedocs.io) which trains ML models using the genetic algorithm (GA). The problem used to demonstrate how things work is XOR. 4 | 5 | The project builds GUI for the server and the client using [Kivy](https://kivy.org). This has a number of benefits. 6 | 7 | - Easy way to manage the client application. 8 | - Ability to make the client available in mobile devices because Kivy supports deploying its desktop apps into mobile apps. As a result, machine learning models could be trained using federated learning by the massive private data available in mobile devices. 9 | 10 | # Project Files 11 | 12 | The project has the following files: 13 | 14 | - `server.py`: The server Kivy app. It creates a model that is trained on the clients' devices using FL. 15 | - `client1.py`: The client Kivy app which trains the model sent by the server using just 2 samples of the XOR problem. 16 | - `client2.py`: Another client Kivy app that trains the server's model using the other 2 samples in the XOR problem. 17 | 18 | # Install PyGAD 19 | 20 | Before running the project, the [PyGAD](https://pygad.readthedocs.io/) library must be installed. 21 | 22 | ``` 23 | pip install pygad 24 | ``` 25 | 26 | For Linux and Mac, use `pip3`: 27 | 28 | ``` 29 | pip3 install pygad 30 | ``` 31 | 32 | # Running the Project 33 | 34 | Start the project by running the [`server.py`](https://github.com/ahmedfgad/FederatedLearning/blob/master/server.py) file. The GUI of the server Kivy app is shown below. Follow these steps to make sure the server is running and listening for connections. 35 | 36 | * Click on the **Create Socket** button to create a socket. 37 | 38 | * Enter the IPv4 address and port number of the server's socket. `localhost` is used if both the server and the clients are running on the same machine. This is just for testing purposes. Practically, they run on different machines. Thus, the user need to specify the IPv4 address (e.g. 192.168.1.4). 39 | * Click on the **Bind Socket** button to bind the create socket to the entered IPv4 address and port number. 40 | * Click on the **Listen to Connections** button to start listening and accepting incoming connections. Each connected device receives the current model to be trained by its local data. Once the model is trained, then no more models will be sent to the connected devices. 41 | 42 | ![Fig01](https://user-images.githubusercontent.com/16560492/86205885-5af32380-bb6b-11ea-9ca6-149c0170e82b.png) 43 | 44 | After running the server, next is to run one or more clients. The project creates 2 clients but you can add more. The only expected change among the different clients is the data being used for training the model sent by the server. 45 | 46 | For [`client1.py`](https://github.com/ahmedfgad/FederatedLearning/blob/master/client1.py), here is the training data (2 samples of the XOR problem): 47 | 48 | ```python 49 | # Preparing the NumPy array of the inputs. 50 | data_inputs = numpy.array([[0, 1], 51 | [0, 0]]) 52 | 53 | # Preparing the NumPy array of the outputs. 54 | data_outputs = numpy.array([1, 55 | 0]) 56 | ``` 57 | 58 | Here is the training data (other 2 samples of the XOR problem) for the other client ([`client2.py`](https://github.com/ahmedfgad/FederatedLearning/blob/master/client2.py)): 59 | 60 | ```python 61 | # Preparing the NumPy array of the inputs. 62 | data_inputs = numpy.array([[1, 0], 63 | [1, 1]]) 64 | 65 | # Preparing the NumPy array of the outputs. 66 | data_outputs = numpy.array([1, 67 | 0]) 68 | ``` 69 | 70 | Just run any client and a GUI will appear like that. You can either run the client at a desktop or a mobile device. 71 | 72 | Follow these steps to run the client: 73 | 74 | * Click on the **Create Socket** button to create a socket. 75 | 76 | * Enter the IPv4 address and port number of the server's socket. If both the client and the server are running on the same machine, just use `localhost` for the IPv4 address. Otherwise, specify the IPv4 address (e.g. 192.168.1.4).s 77 | * Click on the **Connect to Server** button to create a TCP connection with the server. 78 | * Click on the **Receive & Train Model** button to ask the server to send its current ML model. The model will be trained by the client's local private data. The updated model will be sent back to the server. Once the model is trained, the message **Model is Trained** will appear. 79 | 80 | ![Fig03](https://user-images.githubusercontent.com/16560492/86206222-292e8c80-bb6c-11ea-9311-1ef4bb467188.jpg) 81 | 82 | # Download APKs 83 | 84 | The links for the APK files of both the server and the client Android apps are given below: 85 | 86 | - [Server](https://github.com/ahmedfgad/FederatedLearning/releases/download/0.1/FL-server-android.apk): https://github.com/ahmedfgad/FederatedLearning/releases/download/0.1/FL-server-android.apk 87 | - [Client](https://github.com/ahmedfgad/FederatedLearning/releases/download/0.1/FL-client-android.apk): https://github.com/ahmedfgad/FederatedLearning/releases/download/0.1/FL-client-android.apk 88 | 89 | # For More Information 90 | 91 | There are a number of resources to get started with federated learning and Kivy. 92 | 93 | ## Tutorial: [Introduction to Federated Learning](https://heartbeat.fritz.ai/introduction-to-federated-learning-40eb122754a2) 94 | 95 | This tutorial describes the pipeline of training a machine learning model using federated learning. 96 | 97 | [![](https://miro.medium.com/max/3240/1*6gRmlrDPp5J42HR3QWLYew.jpeg)](https://heartbeat.fritz.ai/introduction-to-federated-learning-40eb122754a2) 98 | 99 | ## Tutorial: [Breaking Privacy in Federated Learning](https://heartbeat.fritz.ai/breaking-privacy-in-federated-learning-77fa08ccac9a) 100 | 101 | Even that federated learning does not disclose the private user data, there are some cases in which the privacy of federated learning can be broken. 102 | 103 | [![](https://miro.medium.com/max/3240/1*nZQg-E4a1wOvIH2AmkUUsQ.jpeg)](https://heartbeat.fritz.ai/breaking-privacy-in-federated-learning-77fa08ccac9a) 104 | 105 | ## Tutorial: [Python for Android: Start Building Kivy Cross-Platform Applications](https://www.linkedin.com/pulse/python-android-start-building-kivy-cross-platform-applications-gad) 106 | 107 | This tutorial titled [Python for Android: Start Building Kivy Cross-Platform Applications](https://www.linkedin.com/pulse/python-android-start-building-kivy-cross-platform-applications-gad) covers the steps for creating an Android app out of the Kivy app. 108 | 109 | [![Kivy-Tutorial](https://user-images.githubusercontent.com/16560492/86205332-dfdd3d80-bb69-11ea-91fb-cb0143cb1e5e.png)](https://www.linkedin.com/pulse/python-android-start-building-kivy-cross-platform-applications-gad) 110 | 111 | ## Book: [Building Android Apps in Python Using Kivy with Android Studio](https://www.amazon.com/Building-Android-Python-Using-Studio/dp/1484250303) 112 | 113 | To get started with Kivy app development and how to built Android apps out of the Kivy app, check the book titled [Building Android Apps in Python Using Kivy with Android Studio](https://www.amazon.com/Building-Android-Python-Using-Studio/dp/1484250303) 114 | 115 | [![kivy-book](https://user-images.githubusercontent.com/16560492/86205093-575e9d00-bb69-11ea-82f7-23fef487ce3c.jpg)](https://www.amazon.com/Building-Android-Python-Using-Studio/dp/1484250303) 116 | 117 | # Citing PyGAD - Bibtex Formatted Citation 118 | 119 | If you used PyGAD, please consider adding a citation to the following paper about PyGAD: 120 | 121 | ``` 122 | @misc{gad2021pygad, 123 | title={PyGAD: An Intuitive Genetic Algorithm Python Library}, 124 | author={Ahmed Fawzy Gad}, 125 | year={2021}, 126 | eprint={2106.06158}, 127 | archivePrefix={arXiv}, 128 | primaryClass={cs.NE} 129 | } 130 | ``` 131 | 132 | # Contact Us 133 | 134 | - E-mail: [ahmed.f.gad@gmail.com](mailto:ahmed.f.gad@gmail.com) 135 | - [LinkedIn](https://www.linkedin.com/in/ahmedfgad) 136 | - [Amazon Author Page](https://amazon.com/author/ahmedgad) 137 | - [Heartbeat](https://heartbeat.fritz.ai/@ahmedfgad) 138 | - [Paperspace](https://blog.paperspace.com/author/ahmed) 139 | - [KDnuggets](https://kdnuggets.com/author/ahmed-gad) 140 | - [TowardsDataScience](https://towardsdatascience.com/@ahmedfgad) 141 | - [GitHub](https://github.com/ahmedfgad) 142 | -------------------------------------------------------------------------------- /TutorialProject/Part1/README.md: -------------------------------------------------------------------------------- 1 | # Federated Learning Demo in Python using Socket Programming: Part 1 2 | 3 | This is [Part 1](https://github.com/ahmedfgad/FederatedLearning/tree/master/TutorialProject/Part1) of the federated learning (FL) demo project in Python using socket programming. In this part, a simple client-server app is created to send and receive text messages. 4 | 5 | In [Part 2](https://github.com/ahmedfgad/FederatedLearning/tree/master/TutorialProject/Part2), the server app will be extended to allow accepting multiple connections at the same time in addition to sending and receiving multiple messages within the same connection. 6 | 7 | # Project Files 8 | 9 | The project has the following files: 10 | 11 | - `server.py`: The server app. The server receives a text message from the client and replies with another text message. 12 | - `client.py`: The client app which sends a text message trains to the server and receives a text response. 13 | 14 | # Install PyGAD 15 | 16 | The project uses the [PyGAD](https://pypi.org/project/pygad) library for building and training the ML model. To install [PyGAD](https://pypi.org/project/pygad), simply use pip to download and install the library from [PyPI](https://pypi.org/project/pygad) (Python Package Index). The library lives a PyPI at this page https://pypi.org/project/pygad. 17 | 18 | For Windows, issue the following command: 19 | 20 | ``` 21 | pip install pygad 22 | ``` 23 | 24 | For Linux and Mac, replace `pip` by use `pip3` because the library only supports Python 3. 25 | 26 | ``` 27 | pip3 install pygad 28 | ``` 29 | 30 | PyGAD is developed in Python 3.7.3 and depends on NumPy for creating and manipulating arrays and Matplotlib for creating figures. The exact NumPy version used in developing PyGAD is 1.16.4. For Matplotlib, the version is 3.1.0. 31 | 32 | To get started with PyGAD, please read the documentation at [Read The Docs](https://pygad.readthedocs.io/) [https://pygad.readthedocs.io](https://pygad.readthedocs.io/). 33 | 34 | # Running the Project 35 | 36 | Start the project by running the `server.py` file from the terminal using the following command: 37 | 38 | ``` 39 | python server.py 40 | ``` 41 | 42 | For Mac/Linux, use `python3` rather than `python`: 43 | 44 | ``` 45 | python3 server.py 46 | ``` 47 | 48 | After running the server, next is to run the client using the `client.py` script. 49 | 50 | ``` 51 | python client.py 52 | ``` 53 | 54 | For Mac/Linux, use `python3` rather than `python`: 55 | 56 | ``` 57 | python3 client.py 58 | ``` 59 | 60 | # Contact Us 61 | 62 | - E-mail: [ahmed.f.gad@gmail.com](mailto:ahmed.f.gad@gmail.com) 63 | - [LinkedIn](https://www.linkedin.com/in/ahmedfgad) 64 | - [Amazon Author Page](https://amazon.com/author/ahmedgad) 65 | - [Heartbeat](https://heartbeat.fritz.ai/@ahmedfgad) 66 | - [Paperspace](https://blog.paperspace.com/author/ahmed) 67 | - [KDnuggets](https://kdnuggets.com/author/ahmed-gad) 68 | - [TowardsDataScience](https://towardsdatascience.com/@ahmedfgad) 69 | - [GitHub](https://github.com/ahmedfgad) -------------------------------------------------------------------------------- /TutorialProject/Part1/client.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | 4 | soc = socket.socket() 5 | print("Socket is created.") 6 | 7 | soc.connect(("localhost", 10000)) 8 | print("Connected to the server.") 9 | 10 | msg = "A message from the client." 11 | msg = pickle.dumps(msg) 12 | soc.sendall(msg) 13 | print("Client sent a message to the server.") 14 | 15 | received_data = b'' 16 | while str(received_data)[-2] != '.': 17 | data = soc.recv(8) 18 | received_data += data 19 | 20 | received_data = pickle.loads(received_data) 21 | print("Received data from the client: {received_data}".format(received_data=received_data)) 22 | 23 | soc.close() 24 | print("Socket is closed.") -------------------------------------------------------------------------------- /TutorialProject/Part1/server.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | 4 | soc = socket.socket() 5 | print("Socket is created.") 6 | 7 | soc.bind(("localhost", 10000)) 8 | print("Socket is bound to an address & port number.") 9 | 10 | soc.listen(1) 11 | print("Listening for incoming connection ...") 12 | 13 | connected = False 14 | accept_timeout = 10 15 | soc.settimeout(accept_timeout) 16 | try: 17 | connection, address = soc.accept() 18 | print("Connected to a client: {client_info}.".format(client_info=address)) 19 | connected = True 20 | except socket.timeout: 21 | print("A socket.timeout exception occurred because the server did not receive any connection for {accept_timeout} seconds.".format(accept_timeout=accept_timeout)) 22 | 23 | received_data = b'' 24 | if connected: 25 | while str(received_data)[-2] != '.': 26 | data = connection.recv(8) 27 | received_data += data 28 | received_data = pickle.loads(received_data) 29 | print("Received data from the client: {received_data}".format(received_data=received_data)) 30 | 31 | msg = "Reply from the server." 32 | msg = pickle.dumps(msg) 33 | connection.sendall(msg) 34 | print("Server sent a message to the client.") 35 | 36 | connection.close() 37 | print("Connection is closed with: {client_info}.".format(client_info=address)) 38 | 39 | soc.close() 40 | print("Socket is closed.") 41 | -------------------------------------------------------------------------------- /TutorialProject/Part2/README.md: -------------------------------------------------------------------------------- 1 | # Federated Learning Demo in Python using Socket Programming: Part 2 2 | 3 | This is [Part 2](https://github.com/ahmedfgad/FederatedLearning/tree/master/TutorialProject/Part2) of the federated learning (FL) demo project in Python using socket programming. In this part, the server app created in [Part 1](https://github.com/ahmedfgad/FederatedLearning/tree/master/TutorialProject/Part1) is extended so that it can: 4 | 5 | * Accept multiple connections at the same time using threading. 6 | * Send and receive multiple messages within the same connection. 7 | 8 | In [Part 3](https://github.com/ahmedfgad/FederatedLearning/tree/master/TutorialProject/Part3), a machine learning model is created using [PyGAD](https://pygad.readthedocs.io) at the server and then sent to the clients over the socket to be trained using the genetic algorithm. The trained model at the client is then sent back to the server. 9 | 10 | # Project Files 11 | 12 | The project has the following files: 13 | 14 | - `server.py`: The server app. The server receives a text message from the client and replies with another text message. 15 | - `client1.py`: A client app which sends a text message trains to the server and receives a text response. 16 | - `client2.py`: A client app which sends a text message trains to the server and receives a text response. 17 | 18 | # Install PyGAD 19 | 20 | The project uses the [PyGAD](https://pypi.org/project/pygad) library for building and training the ML model. To install [PyGAD](https://pypi.org/project/pygad), simply use pip to download and install the library from [PyPI](https://pypi.org/project/pygad) (Python Package Index). The library lives a PyPI at this page https://pypi.org/project/pygad. 21 | 22 | For Windows, issue the following command: 23 | 24 | ``` 25 | pip install pygad 26 | ``` 27 | 28 | For Linux and Mac, replace `pip` by use `pip3` because the library only supports Python 3. 29 | 30 | ``` 31 | pip3 install pygad 32 | ``` 33 | 34 | PyGAD is developed in Python 3.7.3 and depends on NumPy for creating and manipulating arrays and Matplotlib for creating figures. The exact NumPy version used in developing PyGAD is 1.16.4. For Matplotlib, the version is 3.1.0. 35 | 36 | To get started with PyGAD, please read the documentation at [Read The Docs](https://pygad.readthedocs.io/) [https://pygad.readthedocs.io](https://pygad.readthedocs.io/). 37 | 38 | # Running the Project 39 | 40 | Start the project by running the `server.py` file from the terminal using the following command: 41 | 42 | ``` 43 | python server.py 44 | ``` 45 | 46 | For Mac/Linux, use `python3` rather than `python`: 47 | 48 | ``` 49 | python3 server.py 50 | ``` 51 | 52 | After running the server, next is to run one or more clients. The project creates 2 clients but you can add more. 53 | 54 | To run a client, simply issue the following terminal command while replacing `.py` by the client's script name. 55 | 56 | ``` 57 | python .py 58 | ``` 59 | 60 | For Mac/Linux, use `python3` rather than `python`: 61 | 62 | ``` 63 | python3 .py 64 | ``` 65 | 66 | # Contact Us 67 | 68 | - E-mail: [ahmed.f.gad@gmail.com](mailto:ahmed.f.gad@gmail.com) 69 | - [LinkedIn](https://www.linkedin.com/in/ahmedfgad) 70 | - [Amazon Author Page](https://amazon.com/author/ahmedgad) 71 | - [Heartbeat](https://heartbeat.fritz.ai/@ahmedfgad) 72 | - [Paperspace](https://blog.paperspace.com/author/ahmed) 73 | - [KDnuggets](https://kdnuggets.com/author/ahmed-gad) 74 | - [TowardsDataScience](https://towardsdatascience.com/@ahmedfgad) 75 | - [GitHub](https://github.com/ahmedfgad) -------------------------------------------------------------------------------- /TutorialProject/Part2/client1.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | 4 | soc = socket.socket() 5 | print("Socket is created.") 6 | 7 | soc.connect(("localhost", 10000)) 8 | print("Connected to the server.") 9 | 10 | msg = "A message from the client." 11 | msg = pickle.dumps(msg) 12 | soc.sendall(msg) 13 | print("Client sent a message to the server.") 14 | 15 | received_data = b'' 16 | while str(received_data)[-2] != '.': 17 | data = soc.recv(8) 18 | received_data += data 19 | 20 | received_data = pickle.loads(received_data) 21 | print("Received data from the client: {received_data}".format(received_data=received_data)) 22 | 23 | msg = "Another message from the client." 24 | msg = pickle.dumps(msg) 25 | soc.sendall(msg) 26 | print("Client sent a message to the server.") 27 | 28 | received_data = b'' 29 | while str(received_data)[-2] != '.': 30 | data = soc.recv(8) 31 | received_data += data 32 | 33 | received_data = pickle.loads(received_data) 34 | print("Received data from the client: {received_data}".format(received_data=received_data)) 35 | 36 | soc.close() 37 | print("Socket is closed.") -------------------------------------------------------------------------------- /TutorialProject/Part2/client2.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | 4 | soc = socket.socket() 5 | print("Socket is created.") 6 | 7 | soc.connect(("localhost", 10000)) 8 | print("Connected to the server.") 9 | 10 | msg = "A message from the client." 11 | msg = pickle.dumps(msg) 12 | soc.sendall(msg) 13 | print("Client sent a message to the server.") 14 | 15 | received_data = b'' 16 | while str(received_data)[-2] != '.': 17 | data = soc.recv(8) 18 | received_data += data 19 | 20 | received_data = pickle.loads(received_data) 21 | print("Received data from the client: {received_data}".format(received_data=received_data)) 22 | 23 | msg = "Another message from the client." 24 | msg = pickle.dumps(msg) 25 | soc.sendall(msg) 26 | print("Client sent a message to the server.") 27 | 28 | received_data = b'' 29 | while str(received_data)[-2] != '.': 30 | data = soc.recv(8) 31 | received_data += data 32 | 33 | received_data = pickle.loads(received_data) 34 | print("Received data from the client: {received_data}".format(received_data=received_data)) 35 | 36 | soc.close() 37 | print("Socket is closed.") -------------------------------------------------------------------------------- /TutorialProject/Part2/server.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import time 4 | import threading 5 | 6 | class SocketThread(threading.Thread): 7 | 8 | def __init__(self, connection, client_info, buffer_size=1024, recv_timeout=5): 9 | threading.Thread.__init__(self) 10 | self.connection = connection 11 | self.client_info = client_info 12 | self.buffer_size = buffer_size 13 | self.recv_timeout = recv_timeout 14 | 15 | def recv(self): 16 | received_data = b"" 17 | while True: 18 | try: 19 | data = connection.recv(self.buffer_size) 20 | received_data += data 21 | 22 | if data == b'': # Nothing received from the client. 23 | received_data = b"" 24 | # If still nothing received for a number of seconds specified by the recv_timeout attribute, return with status 0 to close the connection. 25 | if (time.time() - self.recv_start_time) > self.recv_timeout: 26 | return None, 0 # 0 means the connection is no longer active and it should be closed. 27 | elif str(data)[-2] == '.': 28 | print("All data ({data_len} bytes) Received from {client_info}.".format(client_info=self.client_info, data_len=len(received_data))) 29 | 30 | if len(received_data) > 0: 31 | try: 32 | # Decoding the data (bytes). 33 | received_data = pickle.loads(received_data) 34 | # Returning the decoded data. 35 | return received_data, 1 36 | 37 | except BaseException as e: 38 | print("Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 39 | return None, 0 40 | else: 41 | # In case data are received from the client, update the recv_start_time to the current time to reset the timeout counter. 42 | self.recv_start_time = time.time() 43 | 44 | except BaseException as e: 45 | print("Error Receiving Data from the Client: {msg}.\n".format(msg=e)) 46 | return None, 0 47 | 48 | def run(self): 49 | while True: 50 | self.recv_start_time = time.time() 51 | time_struct = time.gmtime() 52 | date_time = "Waiting to Receive Data Starting from {day}/{month}/{year} {hour}:{minute}:{second} GMT".format(year=time_struct.tm_year, month=time_struct.tm_mon, day=time_struct.tm_mday, hour=time_struct.tm_hour, minute=time_struct.tm_min, second=time_struct.tm_sec) 53 | print(date_time) 54 | received_data, status = self.recv() 55 | if status == 0: 56 | self.connection.close() 57 | print("Connection Closed with {client_info} either due to inactivity for {recv_timeout} seconds or due to an error.".format(client_info=self.client_info, recv_timeout=self.recv_timeout), end="\n\n") 58 | break 59 | 60 | msg = "Reply from the server." 61 | msg = pickle.dumps(msg) 62 | connection.sendall(msg) 63 | print("Server sent a message to the client.") 64 | 65 | soc = socket.socket() 66 | print("Socket is created.") 67 | 68 | soc.bind(("localhost", 10000)) 69 | print("Socket is bound to an address & port number.") 70 | 71 | soc.listen(1) 72 | print("Listening for incoming connection ...") 73 | 74 | while True: 75 | try: 76 | connection, client_info = soc.accept() 77 | print("New Connection from {client_info}.".format(client_info=client_info)) 78 | socket_thread = SocketThread(connection=connection, 79 | client_info=client_info, 80 | buffer_size=1024, 81 | recv_timeout=10) 82 | socket_thread.start() 83 | except: 84 | soc.close() 85 | print("(Timeout) Socket Closed Because no Connections Received.\n") 86 | break 87 | -------------------------------------------------------------------------------- /TutorialProject/Part3/README.md: -------------------------------------------------------------------------------- 1 | # Federated Learning Demo in Python using Socket Programming: Part 3 2 | 3 | This is [Part 3](https://github.com/ahmedfgad/FederatedLearning/tree/master/TutorialProject/Part3) of the federated learning (FL) demo project in Python using socket programming. In this part, [PyGAD](https://pygad.readthedocs.io) is used to create a ML model at the server which is then sent to the clients to be trained using the genetic algorithm (GA). The problem used to demonstrate how things work is XOR. 4 | 5 | In [Part 4](https://github.com/ahmedfgad/FederatedLearning/tree/master/TutorialProject/Part4), a GUI is created using Kivy for both the server and the client apps. Moreover, both the server and client apps will be made available for Android. 6 | 7 | # Project Files 8 | 9 | The project has the following files: 10 | 11 | - `server.py`: The server app. It creates a model that is trained on the clients' devices using FL. 12 | - `client1.py`: A client app which trains the model sent by the server using just 2 samples of the XOR problem. 13 | - `client2.py`: Another client app that trains the server's model using the other 2 samples in the XOR problem. 14 | 15 | # Install PyGAD 16 | 17 | The project uses the [PyGAD](https://pypi.org/project/pygad) library for building and training the ML model. To install [PyGAD](https://pypi.org/project/pygad), simply use pip to download and install the library from [PyPI](https://pypi.org/project/pygad) (Python Package Index). The library lives a PyPI at this page https://pypi.org/project/pygad. 18 | 19 | For Windows, issue the following command: 20 | 21 | ``` 22 | pip install pygad 23 | ``` 24 | 25 | For Linux and Mac, replace `pip` by use `pip3` because the library only supports Python 3. 26 | 27 | ``` 28 | pip3 install pygad 29 | ``` 30 | 31 | PyGAD is developed in Python 3.7.3 and depends on NumPy for creating and manipulating arrays and Matplotlib for creating figures. The exact NumPy version used in developing PyGAD is 1.16.4. For Matplotlib, the version is 3.1.0. 32 | 33 | To get started with PyGAD, please read the documentation at [Read The Docs](https://pygad.readthedocs.io/) [https://pygad.readthedocs.io](https://pygad.readthedocs.io/). 34 | 35 | # Running the Project 36 | 37 | Start the project by running the `server.py` file from the terminal using the following command: 38 | 39 | ``` 40 | python server.py 41 | ``` 42 | 43 | For Mac/Linux, use `python3` rather than `python`: 44 | 45 | ``` 46 | python3 server.py 47 | ``` 48 | 49 | After running the server, next is to run one or more clients. The project creates 2 clients but you can add more. The only expected change among the different clients is the data being used for training the model sent by the server. 50 | 51 | For `client1.py`, here is the training data (2 samples of the XOR problem): 52 | 53 | ```python 54 | # Preparing the NumPy array of the inputs. 55 | data_inputs = numpy.array([[0, 1], 56 | [0, 0]]) 57 | 58 | # Preparing the NumPy array of the outputs. 59 | data_outputs = numpy.array([1, 60 | 0]) 61 | ``` 62 | 63 | Here is the training data (other 2 samples of the XOR problem) for the other client (`client2.py`): 64 | 65 | ```python 66 | # Preparing the NumPy array of the inputs. 67 | data_inputs = numpy.array([[1, 0], 68 | [1, 1]]) 69 | 70 | # Preparing the NumPy array of the outputs. 71 | data_outputs = numpy.array([1, 72 | 0]) 73 | ``` 74 | 75 | To run a client, simply issue the following terminal command while replacing `.py` by the client's script name. 76 | 77 | ``` 78 | python .py 79 | ``` 80 | 81 | For Mac/Linux, use `python3` rather than `python`: 82 | 83 | ``` 84 | python3 .py 85 | ``` 86 | 87 | # For More Information 88 | 89 | There are a number of resources to get started with federated learning . 90 | 91 | ## Tutorial: [Introduction to Federated Learning](https://heartbeat.fritz.ai/introduction-to-federated-learning-40eb122754a2) 92 | 93 | This tutorial describes the pipeline of training a machine learning model using federated learning. 94 | 95 | [![](https://miro.medium.com/max/3240/1*6gRmlrDPp5J42HR3QWLYew.jpeg)](https://heartbeat.fritz.ai/introduction-to-federated-learning-40eb122754a2) 96 | 97 | ## Tutorial: [Breaking Privacy in Federated Learning](https://heartbeat.fritz.ai/breaking-privacy-in-federated-learning-77fa08ccac9a) 98 | 99 | Even that federated learning does not disclose the private user data, there are some cases in which the privacy of federated learning can be broken. 100 | 101 | [![](https://miro.medium.com/max/3240/1*nZQg-E4a1wOvIH2AmkUUsQ.jpeg)](https://heartbeat.fritz.ai/breaking-privacy-in-federated-learning-77fa08ccac9a) 102 | 103 | # Contact Us 104 | 105 | - E-mail: [ahmed.f.gad@gmail.com](mailto:ahmed.f.gad@gmail.com) 106 | - [LinkedIn](https://www.linkedin.com/in/ahmedfgad) 107 | - [Amazon Author Page](https://amazon.com/author/ahmedgad) 108 | - [Heartbeat](https://heartbeat.fritz.ai/@ahmedfgad) 109 | - [Paperspace](https://blog.paperspace.com/author/ahmed) 110 | - [KDnuggets](https://kdnuggets.com/author/ahmed-gad) 111 | - [TowardsDataScience](https://towardsdatascience.com/@ahmedfgad) 112 | - [GitHub](https://github.com/ahmedfgad) -------------------------------------------------------------------------------- /TutorialProject/Part3/client1.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import numpy 4 | 5 | import pygad 6 | import pygad.nn 7 | import pygad.gann 8 | 9 | def fitness_func(solution, sol_idx): 10 | global GANN_instance, data_inputs, data_outputs 11 | 12 | predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[sol_idx], 13 | data_inputs=data_inputs) 14 | correct_predictions = numpy.where(predictions == data_outputs)[0].size 15 | solution_fitness = (correct_predictions/data_outputs.size)*100 16 | 17 | return solution_fitness 18 | 19 | def callback_generation(ga_instance): 20 | global GANN_instance, last_fitness 21 | 22 | population_matrices = pygad.gann.population_as_matrices(population_networks=GANN_instance.population_networks, 23 | population_vectors=ga_instance.population) 24 | 25 | GANN_instance.update_population_trained_weights(population_trained_weights=population_matrices) 26 | 27 | print("Generation = {generation}".format(generation=ga_instance.generations_completed)) 28 | print("Fitness = {fitness}".format(fitness=ga_instance.best_solution()[1])) 29 | print("Change = {change}".format(change=ga_instance.best_solution()[1] - last_fitness)) 30 | 31 | last_fitness = ga_instance.best_solution()[1] 32 | 33 | last_fitness = 0 34 | 35 | def prepare_GA(GANN_instance): 36 | # population does not hold the numerical weights of the network instead it holds a list of references to each last layer of each network (i.e. solution) in the population. A solution or a network can be used interchangeably. 37 | # If there is a population with 3 solutions (i.e. networks), then the population is a list with 3 elements. Each element is a reference to the last layer of each network. Using such a reference, all details of the network can be accessed. 38 | population_vectors = pygad.gann.population_as_vectors(population_networks=GANN_instance.population_networks) 39 | 40 | # To prepare the initial population, there are 2 ways: 41 | # 1) Prepare it yourself and pass it to the initial_population parameter. This way is useful when the user wants to start the genetic algorithm with a custom initial population. 42 | # 2) Assign valid integer values to the sol_per_pop and num_genes parameters. If the initial_population parameter exists, then the sol_per_pop and num_genes parameters are useless. 43 | initial_population = population_vectors.copy() 44 | 45 | num_parents_mating = 4 # Number of solutions to be selected as parents in the mating pool. 46 | 47 | num_generations = 500 # Number of generations. 48 | 49 | mutation_percent_genes = 5 # Percentage of genes to mutate. This parameter has no action if the parameter mutation_num_genes exists. 50 | 51 | parent_selection_type = "sss" # Type of parent selection. 52 | 53 | crossover_type = "single_point" # Type of the crossover operator. 54 | 55 | mutation_type = "random" # Type of the mutation operator. 56 | 57 | keep_parents = 1 # Number of parents to keep in the next population. -1 means keep all parents and 0 means keep nothing. 58 | 59 | init_range_low = -2 60 | init_range_high = 5 61 | 62 | ga_instance = pygad.GA(num_generations=num_generations, 63 | num_parents_mating=num_parents_mating, 64 | initial_population=initial_population, 65 | fitness_func=fitness_func, 66 | mutation_percent_genes=mutation_percent_genes, 67 | init_range_low=init_range_low, 68 | init_range_high=init_range_high, 69 | parent_selection_type=parent_selection_type, 70 | crossover_type=crossover_type, 71 | mutation_type=mutation_type, 72 | keep_parents=keep_parents, 73 | callback_generation=callback_generation) 74 | 75 | return ga_instance 76 | 77 | # Preparing the NumPy array of the inputs. 78 | data_inputs = numpy.array([[0, 1], 79 | [0, 0]]) 80 | 81 | # Preparing the NumPy array of the outputs. 82 | data_outputs = numpy.array([1, 83 | 0]) 84 | 85 | def recv(soc, buffer_size=1024, recv_timeout=10): 86 | received_data = b"" 87 | while str(received_data)[-2] != '.': 88 | try: 89 | soc.settimeout(recv_timeout) 90 | received_data += soc.recv(buffer_size) 91 | except socket.timeout: 92 | print("A socket.timeout exception occurred because the server did not send any data for {recv_timeout} seconds. There may be an error or the model may be trained successfully.".format(recv_timeout=recv_timeout)) 93 | return None, 0 94 | except BaseException as e: 95 | return None, 0 96 | print("An error occurred while receiving data from the server {msg}.".format(msg=e)) 97 | 98 | try: 99 | received_data = pickle.loads(received_data) 100 | except BaseException as e: 101 | print("Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 102 | return None, 0 103 | 104 | return received_data, 1 105 | 106 | soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 107 | print("Socket Created.\n") 108 | 109 | try: 110 | soc.connect(("localhost", 10000)) 111 | print("Successful Connection to the Server.\n") 112 | except BaseException as e: 113 | print("Error Connecting to the Server: {msg}".format(msg=e)) 114 | soc.close() 115 | print("Socket Closed.") 116 | 117 | subject = "echo" 118 | GANN_instance = None 119 | best_sol_idx = -1 120 | 121 | while True: 122 | data = {"subject": subject, "data": GANN_instance, "best_solution_idx": best_sol_idx} 123 | data_byte = pickle.dumps(data) 124 | 125 | print("Sending the Model to the Server.\n") 126 | soc.sendall(data_byte) 127 | 128 | print("Receiving Reply from the Server.") 129 | received_data, status = recv(soc=soc, 130 | buffer_size=1024, 131 | recv_timeout=10) 132 | if status == 0: 133 | print("Nothing Received from the Server.") 134 | break 135 | else: 136 | print(received_data, end="\n\n") 137 | 138 | subject = received_data["subject"] 139 | if subject == "model": 140 | GANN_instance = received_data["data"] 141 | elif subject == "done": 142 | print("The server said the model is trained successfully and no need for further updates its parameters.") 143 | break 144 | else: 145 | print("Unrecognized message type.") 146 | break 147 | 148 | ga_instance = prepare_GA(GANN_instance) 149 | 150 | ga_instance.run() 151 | 152 | ga_instance.plot_result() 153 | 154 | subject = "model" 155 | best_sol_idx = ga_instance.best_solution()[2] 156 | 157 | # predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[best_sol_idx], data_inputs=data_inputs) 158 | 159 | soc.close() 160 | print("Socket Closed.\n") -------------------------------------------------------------------------------- /TutorialProject/Part3/client2.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import numpy 4 | 5 | import pygad 6 | import pygad.nn 7 | import pygad.gann 8 | 9 | def fitness_func(solution, sol_idx): 10 | global GANN_instance, data_inputs, data_outputs 11 | 12 | predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[sol_idx], 13 | data_inputs=data_inputs) 14 | correct_predictions = numpy.where(predictions == data_outputs)[0].size 15 | solution_fitness = (correct_predictions/data_outputs.size)*100 16 | 17 | return solution_fitness 18 | 19 | def callback_generation(ga_instance): 20 | global GANN_instance, last_fitness 21 | 22 | population_matrices = pygad.gann.population_as_matrices(population_networks=GANN_instance.population_networks, 23 | population_vectors=ga_instance.population) 24 | 25 | GANN_instance.update_population_trained_weights(population_trained_weights=population_matrices) 26 | 27 | print("Generation = {generation}".format(generation=ga_instance.generations_completed)) 28 | print("Fitness = {fitness}".format(fitness=ga_instance.best_solution()[1])) 29 | print("Change = {change}".format(change=ga_instance.best_solution()[1] - last_fitness)) 30 | 31 | last_fitness = ga_instance.best_solution()[1] 32 | 33 | last_fitness = 0 34 | 35 | def prepare_GA(GANN_instance): 36 | # population does not hold the numerical weights of the network instead it holds a list of references to each last layer of each network (i.e. solution) in the population. A solution or a network can be used interchangeably. 37 | # If there is a population with 3 solutions (i.e. networks), then the population is a list with 3 elements. Each element is a reference to the last layer of each network. Using such a reference, all details of the network can be accessed. 38 | population_vectors = pygad.gann.population_as_vectors(population_networks=GANN_instance.population_networks) 39 | 40 | # To prepare the initial population, there are 2 ways: 41 | # 1) Prepare it yourself and pass it to the initial_population parameter. This way is useful when the user wants to start the genetic algorithm with a custom initial population. 42 | # 2) Assign valid integer values to the sol_per_pop and num_genes parameters. If the initial_population parameter exists, then the sol_per_pop and num_genes parameters are useless. 43 | initial_population = population_vectors.copy() 44 | 45 | num_parents_mating = 4 # Number of solutions to be selected as parents in the mating pool. 46 | 47 | num_generations = 500 # Number of generations. 48 | 49 | mutation_percent_genes = 5 # Percentage of genes to mutate. This parameter has no action if the parameter mutation_num_genes exists. 50 | 51 | parent_selection_type = "sss" # Type of parent selection. 52 | 53 | crossover_type = "single_point" # Type of the crossover operator. 54 | 55 | mutation_type = "random" # Type of the mutation operator. 56 | 57 | keep_parents = 1 # Number of parents to keep in the next population. -1 means keep all parents and 0 means keep nothing. 58 | 59 | init_range_low = -2 60 | init_range_high = 5 61 | 62 | ga_instance = pygad.GA(num_generations=num_generations, 63 | num_parents_mating=num_parents_mating, 64 | initial_population=initial_population, 65 | fitness_func=fitness_func, 66 | mutation_percent_genes=mutation_percent_genes, 67 | init_range_low=init_range_low, 68 | init_range_high=init_range_high, 69 | parent_selection_type=parent_selection_type, 70 | crossover_type=crossover_type, 71 | mutation_type=mutation_type, 72 | keep_parents=keep_parents, 73 | callback_generation=callback_generation) 74 | 75 | return ga_instance 76 | 77 | # Preparing the NumPy array of the inputs. 78 | data_inputs = numpy.array([[1, 0], 79 | [1, 1]]) 80 | 81 | # Preparing the NumPy array of the outputs. 82 | data_outputs = numpy.array([1, 83 | 0]) 84 | 85 | def recv(soc, buffer_size=1024, recv_timeout=10): 86 | received_data = b"" 87 | while str(received_data)[-2] != '.': 88 | try: 89 | soc.settimeout(recv_timeout) 90 | received_data += soc.recv(buffer_size) 91 | except socket.timeout: 92 | print("A socket.timeout exception occurred because the server did not send any data for {recv_timeout} seconds. There may be an error or the model may be trained successfully.".format(recv_timeout=recv_timeout)) 93 | return None, 0 94 | except BaseException as e: 95 | return None, 0 96 | print("An error occurred while receiving data from the server {msg}.".format(msg=e)) 97 | 98 | try: 99 | received_data = pickle.loads(received_data) 100 | except BaseException as e: 101 | print("Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 102 | return None, 0 103 | 104 | return received_data, 1 105 | 106 | soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 107 | print("Socket Created.\n") 108 | 109 | try: 110 | soc.connect(("localhost", 10000)) 111 | print("Successful Connection to the Server.\n") 112 | except BaseException as e: 113 | print("Error Connecting to the Server: {msg}".format(msg=e)) 114 | soc.close() 115 | print("Socket Closed.") 116 | 117 | subject = "echo" 118 | GANN_instance = None 119 | best_sol_idx = -1 120 | 121 | while True: 122 | data = {"subject": subject, "data": GANN_instance, "best_solution_idx": best_sol_idx} 123 | data_byte = pickle.dumps(data) 124 | 125 | print("Sending the Model to the Server.\n") 126 | soc.sendall(data_byte) 127 | 128 | print("Receiving Reply from the Server.") 129 | received_data, status = recv(soc=soc, 130 | buffer_size=1024, 131 | recv_timeout=10) 132 | if status == 0: 133 | print("Nothing Received from the Server.") 134 | break 135 | else: 136 | print(received_data, end="\n\n") 137 | 138 | subject = received_data["subject"] 139 | if subject == "model": 140 | GANN_instance = received_data["data"] 141 | elif subject == "done": 142 | print("The server said the model is trained successfully and no need for further updates its parameters.") 143 | break 144 | else: 145 | print("Unrecognized message type.") 146 | break 147 | 148 | ga_instance = prepare_GA(GANN_instance) 149 | 150 | ga_instance.run() 151 | 152 | ga_instance.plot_result() 153 | 154 | subject = "model" 155 | best_sol_idx = ga_instance.best_solution()[2] 156 | 157 | # predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[best_sol_idx], data_inputs=data_inputs) 158 | 159 | soc.close() 160 | print("Socket Closed.\n") -------------------------------------------------------------------------------- /TutorialProject/Part3/server.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import threading 4 | import time 5 | 6 | import pygad 7 | import pygad.nn 8 | import pygad.gann 9 | import numpy 10 | 11 | model = None 12 | 13 | # Preparing the NumPy array of the inputs. 14 | data_inputs = numpy.array([[1, 1], 15 | [1, 0], 16 | [0, 1], 17 | [0, 0]]) 18 | 19 | # Preparing the NumPy array of the outputs. 20 | data_outputs = numpy.array([0, 21 | 1, 22 | 1, 23 | 0]) 24 | 25 | num_classes = 2 26 | num_inputs = 2 27 | 28 | num_solutions = 6 29 | GANN_instance = pygad.gann.GANN(num_solutions=num_solutions, 30 | num_neurons_input=num_inputs, 31 | num_neurons_hidden_layers=[2], 32 | num_neurons_output=num_classes, 33 | hidden_activations=["relu"], 34 | output_activation="softmax") 35 | 36 | class SocketThread(threading.Thread): 37 | 38 | def __init__(self, connection, client_info, buffer_size=1024, recv_timeout=5): 39 | threading.Thread.__init__(self) 40 | self.connection = connection 41 | self.client_info = client_info 42 | self.buffer_size = buffer_size 43 | self.recv_timeout = recv_timeout 44 | 45 | def recv(self): 46 | received_data = b"" 47 | while True: 48 | try: 49 | 50 | data = self.connection.recv(self.buffer_size) 51 | received_data += data 52 | 53 | if data == b'': # Nothing received from the client. 54 | received_data = b"" 55 | # If still nothing received for a number of seconds specified by the recv_timeout attribute, return with status 0 to close the connection. 56 | if (time.time() - self.recv_start_time) > self.recv_timeout: 57 | return None, 0 # 0 means the connection is no longer active and it should be closed. 58 | 59 | elif str(data)[-2] == '.': 60 | print("All data ({data_len} bytes) Received from {client_info}.".format(client_info=self.client_info, data_len=len(received_data))) 61 | 62 | if len(received_data) > 0: 63 | try: 64 | # Decoding the data (bytes). 65 | received_data = pickle.loads(received_data) 66 | # Returning the decoded data. 67 | return received_data, 1 68 | 69 | except BaseException as e: 70 | print("Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 71 | return None, 0 72 | 73 | else: 74 | # In case data are received from the client, update the recv_start_time to the current time to reset the timeout counter. 75 | self.recv_start_time = time.time() 76 | 77 | except BaseException as e: 78 | print("Error Receiving Data from the Client: {msg}.\n".format(msg=e)) 79 | return None, 0 80 | 81 | def model_averaging(self, model, other_model): 82 | model_weights = pygad.nn.layers_weights(last_layer=model, initial=False) 83 | other_model_weights = pygad.nn.layers_weights(last_layer=other_model, initial=False) 84 | 85 | new_weights = numpy.array(model_weights + other_model_weights)/2 86 | 87 | pygad.nn.update_layers_trained_weights(last_layer=model, final_weights=new_weights) 88 | 89 | def reply(self, received_data): 90 | global GANN_instance, data_inputs, data_outputs, model 91 | if (type(received_data) is dict): 92 | if (("data" in received_data.keys()) and ("subject" in received_data.keys())): 93 | subject = received_data["subject"] 94 | print("Client's Message Subject is {subject}.".format(subject=subject)) 95 | 96 | print("Replying to the Client.") 97 | if subject == "echo": 98 | if model is None: 99 | data = {"subject": "model", "data": GANN_instance} 100 | else: 101 | predictions = pygad.nn.predict(last_layer=model, data_inputs=data_inputs) 102 | error = numpy.sum(numpy.abs(predictions - data_outputs)) 103 | # In case a client sent a model to the server despite that the model error is 0.0. In this case, no need to make changes in the model. 104 | if error == 0: 105 | data = {"subject": "done", "data": None} 106 | print("The client asked for the model but it was already trained successfully. There is no need to send the model to the client for retraining.") 107 | else: 108 | data = {"subject": "model", "data": GANN_instance} 109 | 110 | try: 111 | response = pickle.dumps(data) 112 | except BaseException as e: 113 | print("Error Encoding the Message: {msg}.\n".format(msg=e)) 114 | elif subject == "model": 115 | try: 116 | GANN_instance = received_data["data"] 117 | best_model_idx = received_data["best_solution_idx"] 118 | 119 | best_model = GANN_instance.population_networks[best_model_idx] 120 | if model is None: 121 | model = best_model 122 | else: 123 | predictions = pygad.nn.predict(last_layer=model, data_inputs=data_inputs) 124 | 125 | error = numpy.sum(numpy.abs(predictions - data_outputs)) 126 | 127 | # In case a client sent a model to the server despite that the model error is 0.0. In this case, no need to make changes in the model. 128 | if error == 0: 129 | data = {"subject": "done", "data": None} 130 | response = pickle.dumps(data) 131 | print("The model is trained successfully and no need to send the model to the client for retraining.") 132 | return 133 | 134 | self.model_averaging(model, best_model) 135 | 136 | # print(best_model.trained_weights) 137 | # print(model.trained_weights) 138 | 139 | predictions = pygad.nn.predict(last_layer=model, data_inputs=data_inputs) 140 | print("Model Predictions: {predictions}".format(predictions=predictions)) 141 | 142 | error = numpy.sum(numpy.abs(predictions - data_outputs)) 143 | print("Error = {error}\n".format(error=error)) 144 | 145 | if error != 0: 146 | data = {"subject": "model", "data": GANN_instance} 147 | response = pickle.dumps(data) 148 | else: 149 | data = {"subject": "done", "data": None} 150 | response = pickle.dumps(data) 151 | print("\n*****The Model is Trained Successfully*****\n\n") 152 | 153 | except BaseException as e: 154 | print("Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 155 | else: 156 | response = pickle.dumps("Response from the Server") 157 | 158 | try: 159 | self.connection.sendall(response) 160 | except BaseException as e: 161 | print("Error Sending Data to the Client: {msg}.\n".format(msg=e)) 162 | 163 | else: 164 | print("The received dictionary from the client must have the 'subject' and 'data' keys available. The existing keys are {d_keys}.".format(d_keys=received_data.keys())) 165 | else: 166 | print("A dictionary is expected to be received from the client but {d_type} received.".format(d_type=type(received_data))) 167 | 168 | def run(self): 169 | print("Running a Thread for the Connection with {client_info}.".format(client_info=self.client_info)) 170 | 171 | # This while loop allows the server to wait for the client to send data more than once within the same connection. 172 | while True: 173 | self.recv_start_time = time.time() 174 | time_struct = time.gmtime() 175 | date_time = "\nWaiting to Receive Data from {client_info} Starting from {day}/{month}/{year} {hour}:{minute}:{second} GMT".format(year=time_struct.tm_year, month=time_struct.tm_mon, day=time_struct.tm_mday, hour=time_struct.tm_hour, minute=time_struct.tm_min, second=time_struct.tm_sec, client_info=self.client_info) 176 | print(date_time) 177 | received_data, status = self.recv() 178 | if status == 0: 179 | self.connection.close() 180 | print("\nConnection Closed with {client_info} either due to inactivity for {recv_timeout} seconds or due to an error.".format(client_info=self.client_info, recv_timeout=self.recv_timeout), end="\n\n") 181 | break 182 | 183 | # print(received_data) 184 | self.reply(received_data) 185 | 186 | soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 187 | print("Socket Created.\n") 188 | 189 | # Timeout after which the socket will be closed. 190 | # soc.settimeout(5) 191 | 192 | soc.bind(("localhost", 10000)) 193 | print("Socket Bound to IPv4 Address & Port Number.\n") 194 | 195 | soc.listen(1) 196 | print("Socket is Listening for Connections ....\n") 197 | 198 | all_data = b"" 199 | while True: 200 | try: 201 | connection, client_info = soc.accept() 202 | print("\nNew Connection from {client_info}.".format(client_info=client_info)) 203 | socket_thread = SocketThread(connection=connection, 204 | client_info=client_info, 205 | buffer_size=1024, 206 | recv_timeout=10) 207 | socket_thread.start() 208 | except: 209 | soc.close() 210 | print("(Timeout) Socket Closed Because no Connections Received.\n") 211 | break 212 | -------------------------------------------------------------------------------- /TutorialProject/Part4/README.md: -------------------------------------------------------------------------------- 1 | # Federated Learning Demo in Python using Socket Programming: Part 4 2 | 3 | This is [Part 4](https://github.com/ahmedfgad/FederatedLearning/tree/master/TutorialProject/Part4) of the federated learning (FL) demo project in Python using socket programming. In this part, a GUI is created for the server and client apps built in [Part 3](https://github.com/ahmedfgad/FederatedLearning/tree/master/TutorialProject/Part3). 4 | 5 | In [Part 3](https://github.com/ahmedfgad/FederatedLearning/tree/master/TutorialProject/Part3), the user can only interact with the server and the client apps using the terminal. In Part 4, the major change is building a GUI for the client using [Kivy](https://kivy.org). This has a number of benefits. 6 | 7 | - Easy way to manage the client application. 8 | - Ability to make the client available in mobile devices because Kivy supports deploying its desktop apps into mobile apps. As a result, machine learning models could be trained using federated learning by the massive private data available in mobile devices. 9 | 10 | Similar to building a GUI for the client app, a GUI can be built for the server app using Kivy. 11 | 12 | # Project Files 13 | 14 | The project has the following files: 15 | 16 | - `server.py`: The server Kivy app. It creates a model that is trained on the clients' devices using FL. 17 | - `client1.py`: A client Kivy app which trains the model sent by the server using just 2 samples of the XOR problem. 18 | - `client2.py`: Another client Kivy app that trains the server's model using the other 2 samples in the XOR problem. 19 | 20 | # Install PyGAD 21 | 22 | Before running the project, the [PyGAD](https://pygad.readthedocs.io/) library must be installed. 23 | 24 | ``` 25 | pip install pygad 26 | ``` 27 | 28 | For Linux and Mac, use `pip3`: 29 | 30 | ``` 31 | pip3 install pygad 32 | ``` 33 | 34 | # Running the Project 35 | 36 | Start the project by running the [`server.py`](https://github.com/ahmedfgad/FederatedLearning/blob/master/TutorialProject/Part4/server.py) file. The GUI of the server Kivy app is shown below. Follow these steps to make sure the server is running and listening for connections. 37 | 38 | * Click on the **Create Socket** button to create a socket. 39 | 40 | * Enter the IPv4 address and port number of the server's socket. `localhost` is used if both the server and the clients are running on the same machine. This is just for testing purposes. Practically, they run on different machines. Thus, the user need to specify the IPv4 address (e.g. 192.168.1.4). 41 | * Click on the **Bind Socket** button to bind the create socket to the entered IPv4 address and port number. 42 | * Click on the **Listen to Connections** button to start listening and accepting incoming connections. Each connected device receives the current model to be trained by its local data. Once the model is trained, then no more models will be sent to the connected devices. 43 | 44 | ![Fig01](https://user-images.githubusercontent.com/16560492/86205885-5af32380-bb6b-11ea-9ca6-149c0170e82b.png) 45 | 46 | After running the server, next is to run one or more clients. The project creates 2 clients but you can add more. The only expected change among the different clients is the data being used for training the model sent by the server. 47 | 48 | For [`client1.py`](https://github.com/ahmedfgad/FederatedLearning/blob/master/TutorialProject/Part4/client1.py), here is the training data (2 samples of the XOR problem): 49 | 50 | ```python 51 | # Preparing the NumPy array of the inputs. 52 | data_inputs = numpy.array([[0, 1], 53 | [0, 0]]) 54 | 55 | # Preparing the NumPy array of the outputs. 56 | data_outputs = numpy.array([1, 57 | 0]) 58 | ``` 59 | 60 | Here is the training data (other 2 samples of the XOR problem) for the other client ([`client2.py`](https://github.com/ahmedfgad/FederatedLearning/blob/master/TutorialProject/Part4/client2.py)): 61 | 62 | ```python 63 | # Preparing the NumPy array of the inputs. 64 | data_inputs = numpy.array([[1, 0], 65 | [1, 1]]) 66 | 67 | # Preparing the NumPy array of the outputs. 68 | data_outputs = numpy.array([1, 69 | 0]) 70 | ``` 71 | 72 | Just run any client and a GUI will appear like that. You can either run the client at a desktop or a mobile device. 73 | 74 | Follow these steps to run the client: 75 | 76 | * Click on the **Create Socket** button to create a socket. 77 | 78 | * Enter the IPv4 address and port number of the server's socket. If both the client and the server are running on the same machine, just use `localhost` for the IPv4 address. Otherwise, specify the IPv4 address (e.g. 192.168.1.4).s 79 | * Click on the **Connect to Server** button to create a TCP connection with the server. 80 | * Click on the **Receive & Train Model** button to ask the server to send its current ML model. The model will be trained by the client's local private data. The updated model will be sent back to the server. Once the model is trained, the message **Model is Trained** will appear. 81 | 82 | ![Fig03](https://user-images.githubusercontent.com/16560492/86206222-292e8c80-bb6c-11ea-9311-1ef4bb467188.jpg) 83 | 84 | # Download APKs 85 | 86 | The links for the APK files of both the server and the client Android apps are given below: 87 | 88 | - [Server](https://github.com/ahmedfgad/FederatedLearning/releases/download/0.1/FL-server-android.apk): https://github.com/ahmedfgad/FederatedLearning/releases/download/0.1/FL-server-android.apk 89 | - [Client](https://github.com/ahmedfgad/FederatedLearning/releases/download/0.1/FL-client-android.apk): https://github.com/ahmedfgad/FederatedLearning/releases/download/0.1/FL-client-android.apk 90 | 91 | # For More Information 92 | 93 | There are a number of resources to get started with federated learning and Kivy. 94 | 95 | ## Tutorial: [Introduction to Federated Learning](https://heartbeat.fritz.ai/introduction-to-federated-learning-40eb122754a2) 96 | 97 | This tutorial describes the pipeline of training a machine learning model using federated learning. 98 | 99 | [![](https://miro.medium.com/max/3240/1*6gRmlrDPp5J42HR3QWLYew.jpeg)](https://heartbeat.fritz.ai/introduction-to-federated-learning-40eb122754a2) 100 | 101 | ## Tutorial: [Breaking Privacy in Federated Learning](https://heartbeat.fritz.ai/breaking-privacy-in-federated-learning-77fa08ccac9a) 102 | 103 | Even that federated learning does not disclose the private user data, there are some cases in which the privacy of federated learning can be broken. 104 | 105 | [![](https://miro.medium.com/max/3240/1*nZQg-E4a1wOvIH2AmkUUsQ.jpeg)](https://heartbeat.fritz.ai/breaking-privacy-in-federated-learning-77fa08ccac9a) 106 | 107 | ## Tutorial: [Python for Android: Start Building Kivy Cross-Platform Applications](https://www.linkedin.com/pulse/python-android-start-building-kivy-cross-platform-applications-gad) 108 | 109 | This tutorial titled [Python for Android: Start Building Kivy Cross-Platform Applications](https://www.linkedin.com/pulse/python-android-start-building-kivy-cross-platform-applications-gad) covers the steps for creating an Android app out of the Kivy app. 110 | 111 | [![Kivy-Tutorial](https://user-images.githubusercontent.com/16560492/86205332-dfdd3d80-bb69-11ea-91fb-cb0143cb1e5e.png)](https://www.linkedin.com/pulse/python-android-start-building-kivy-cross-platform-applications-gad) 112 | 113 | ## Book: [Building Android Apps in Python Using Kivy with Android Studio](https://www.amazon.com/Building-Android-Python-Using-Studio/dp/1484250303) 114 | 115 | To get started with Kivy app development and how to built Android apps out of the Kivy app, check the book titled [Building Android Apps in Python Using Kivy with Android Studio](https://www.amazon.com/Building-Android-Python-Using-Studio/dp/1484250303) 116 | 117 | [![kivy-book](https://user-images.githubusercontent.com/16560492/86205093-575e9d00-bb69-11ea-82f7-23fef487ce3c.jpg)](https://www.amazon.com/Building-Android-Python-Using-Studio/dp/1484250303) 118 | 119 | # Contact Us 120 | 121 | - E-mail: [ahmed.f.gad@gmail.com](mailto:ahmed.f.gad@gmail.com) 122 | - [LinkedIn](https://www.linkedin.com/in/ahmedfgad) 123 | - [Amazon Author Page](https://amazon.com/author/ahmedgad) 124 | - [Heartbeat](https://heartbeat.fritz.ai/@ahmedfgad) 125 | - [Paperspace](https://blog.paperspace.com/author/ahmed) 126 | - [KDnuggets](https://kdnuggets.com/author/ahmed-gad) 127 | - [TowardsDataScience](https://towardsdatascience.com/@ahmedfgad) 128 | - [GitHub](https://github.com/ahmedfgad) -------------------------------------------------------------------------------- /TutorialProject/Part4/client1.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import numpy 4 | import threading 5 | 6 | import pygad 7 | import pygad.nn 8 | import pygad.gann 9 | 10 | import kivy.app 11 | import kivy.uix.button 12 | import kivy.uix.label 13 | import kivy.uix.boxlayout 14 | import kivy.uix.textinput 15 | 16 | class ClientApp(kivy.app.App): 17 | 18 | def __init__(self): 19 | super().__init__() 20 | 21 | def create_socket(self, *args): 22 | self.soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 23 | self.label.text = "Socket Created" 24 | 25 | self.create_socket_btn.disabled = True 26 | self.connect_btn.disabled = False 27 | self.close_socket_btn.disabled = False 28 | 29 | def connect(self, *args): 30 | try: 31 | self.soc.connect((self.server_ip.text, int(self.server_port.text))) 32 | self.label.text = "Successful Connection to the Server" 33 | 34 | self.connect_btn.disabled = True 35 | self.recv_train_model_btn.disabled = False 36 | 37 | except BaseException as e: 38 | self.label.text = "Error Connecting to the Server" 39 | print("Error Connecting to the Server: {msg}".format(msg=e)) 40 | 41 | self.connect_btn.disabled = False 42 | self.recv_train_model_btn.disabled = True 43 | 44 | def recv_train_model(self, *args): 45 | global GANN_instance 46 | 47 | self.recv_train_model_btn.disabled = True 48 | recvThread = RecvThread(kivy_app=self, buffer_size=1024, recv_timeout=10) 49 | recvThread.start() 50 | 51 | def close_socket(self, *args): 52 | self.soc.close() 53 | self.label.text = "Socket Closed" 54 | 55 | self.create_socket_btn.disabled = False 56 | self.connect_btn.disabled = True 57 | self.recv_train_model_btn.disabled = True 58 | self.close_socket_btn.disabled = True 59 | 60 | def build(self): 61 | self.create_socket_btn = kivy.uix.button.Button(text="Create Socket") 62 | self.create_socket_btn.bind(on_press=self.create_socket) 63 | 64 | self.server_ip = kivy.uix.textinput.TextInput(hint_text="Server IPv4 Address", text="localhost") 65 | self.server_port = kivy.uix.textinput.TextInput(hint_text="Server Port Number", text="10000") 66 | 67 | self.server_info_boxlayout = kivy.uix.boxlayout.BoxLayout(orientation="horizontal") 68 | self.server_info_boxlayout.add_widget(self.server_ip) 69 | self.server_info_boxlayout.add_widget(self.server_port) 70 | 71 | self.connect_btn = kivy.uix.button.Button(text="Connect to Server", disabled=True) 72 | self.connect_btn.bind(on_press=self.connect) 73 | 74 | self.recv_train_model_btn = kivy.uix.button.Button(text="Receive & Train Model", disabled=True) 75 | self.recv_train_model_btn.bind(on_press=self.recv_train_model) 76 | 77 | self.close_socket_btn = kivy.uix.button.Button(text="Close Socket", disabled=True) 78 | self.close_socket_btn.bind(on_press=self.close_socket) 79 | 80 | self.label = kivy.uix.label.Label(text="Socket Status") 81 | 82 | self.box_layout = kivy.uix.boxlayout.BoxLayout(orientation="vertical") 83 | self.box_layout.add_widget(self.create_socket_btn) 84 | self.box_layout.add_widget(self.server_info_boxlayout) 85 | self.box_layout.add_widget(self.connect_btn) 86 | self.box_layout.add_widget(self.recv_train_model_btn) 87 | self.box_layout.add_widget(self.close_socket_btn) 88 | self.box_layout.add_widget(self.label) 89 | 90 | return self.box_layout 91 | 92 | def fitness_func(solution, sol_idx): 93 | global GANN_instance, data_inputs, data_outputs 94 | 95 | predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[sol_idx], 96 | data_inputs=data_inputs) 97 | correct_predictions = numpy.where(predictions == data_outputs)[0].size 98 | solution_fitness = (correct_predictions/data_outputs.size)*100 99 | 100 | return solution_fitness 101 | 102 | def callback_generation(ga_instance): 103 | global GANN_instance, last_fitness 104 | 105 | population_matrices = pygad.gann.population_as_matrices(population_networks=GANN_instance.population_networks, 106 | population_vectors=ga_instance.population) 107 | 108 | GANN_instance.update_population_trained_weights(population_trained_weights=population_matrices) 109 | 110 | # print("Generation = {generation}".format(generation=ga_instance.generations_completed)) 111 | # print("Fitness = {fitness}".format(fitness=ga_instance.best_solution()[1])) 112 | # print("Change = {change}".format(change=ga_instance.best_solution()[1] - last_fitness)) 113 | 114 | #last_fitness = 0 115 | 116 | def prepare_GA(GANN_instance): 117 | # population does not hold the numerical weights of the network instead it holds a list of references to each last layer of each network (i.e. solution) in the population. A solution or a network can be used interchangeably. 118 | # If there is a population with 3 solutions (i.e. networks), then the population is a list with 3 elements. Each element is a reference to the last layer of each network. Using such a reference, all details of the network can be accessed. 119 | population_vectors = pygad.gann.population_as_vectors(population_networks=GANN_instance.population_networks) 120 | 121 | # To prepare the initial population, there are 2 ways: 122 | # 1) Prepare it yourself and pass it to the initial_population parameter. This way is useful when the user wants to start the genetic algorithm with a custom initial population. 123 | # 2) Assign valid integer values to the sol_per_pop and num_genes parameters. If the initial_population parameter exists, then the sol_per_pop and num_genes parameters are useless. 124 | initial_population = population_vectors.copy() 125 | 126 | num_parents_mating = 4 # Number of solutions to be selected as parents in the mating pool. 127 | 128 | num_generations = 500 # Number of generations. 129 | 130 | mutation_percent_genes = 5 # Percentage of genes to mutate. This parameter has no action if the parameter mutation_num_genes exists. 131 | 132 | parent_selection_type = "sss" # Type of parent selection. 133 | 134 | crossover_type = "single_point" # Type of the crossover operator. 135 | 136 | mutation_type = "random" # Type of the mutation operator. 137 | 138 | keep_parents = 1 # Number of parents to keep in the next population. -1 means keep all parents and 0 means keep nothing. 139 | 140 | init_range_low = -2 141 | init_range_high = 5 142 | 143 | ga_instance = pygad.GA(num_generations=num_generations, 144 | num_parents_mating=num_parents_mating, 145 | initial_population=initial_population, 146 | fitness_func=fitness_func, 147 | mutation_percent_genes=mutation_percent_genes, 148 | init_range_low=init_range_low, 149 | init_range_high=init_range_high, 150 | parent_selection_type=parent_selection_type, 151 | crossover_type=crossover_type, 152 | mutation_type=mutation_type, 153 | keep_parents=keep_parents, 154 | on_generation=callback_generation) 155 | 156 | return ga_instance 157 | 158 | # Preparing the NumPy array of the inputs. 159 | data_inputs = numpy.array([[0, 1], 160 | [0, 0]]) 161 | 162 | # Preparing the NumPy array of the outputs. 163 | data_outputs = numpy.array([1, 164 | 0]) 165 | 166 | class RecvThread(threading.Thread): 167 | 168 | def __init__(self, kivy_app, buffer_size, recv_timeout): 169 | threading.Thread.__init__(self) 170 | self.kivy_app = kivy_app 171 | self.buffer_size = buffer_size 172 | self.recv_timeout = recv_timeout 173 | 174 | def recv(self): 175 | received_data = b"" 176 | while True: # str(received_data)[-2] != '.': 177 | try: 178 | self.kivy_app.soc.settimeout(self.recv_timeout) 179 | received_data += self.kivy_app.soc.recv(self.buffer_size) 180 | 181 | try: 182 | pickle.loads(received_data) 183 | # If the previous pickle.loads() statement is passed, this means all the data is received. 184 | # Thus, no need to continue the loop and a break statement should be excuted. 185 | break 186 | except BaseException: 187 | # An exception is expected when the data is not 100% received. 188 | pass 189 | 190 | except socket.timeout: 191 | print("A socket.timeout exception occurred because the server did not send any data for {recv_timeout} seconds.".format(recv_timeout=self.recv_timeout)) 192 | self.kivy_app.label.text = "{recv_timeout} Seconds of Inactivity. socket.timeout Exception Occurred".format(recv_timeout=self.recv_timeout) 193 | return None, 0 194 | except BaseException as e: 195 | return None, 0 196 | print("Error While Receiving Data from the Server: {msg}.".format(msg=e)) 197 | self.kivy_app.label.text = "Error While Receiving Data from the Server" 198 | 199 | try: 200 | received_data = pickle.loads(received_data) 201 | except BaseException as e: 202 | print("Error Decoding the Data: {msg}.\n".format(msg=e)) 203 | self.kivy_app.label.text = "Error Decoding the Client's Data" 204 | return None, 0 205 | 206 | return received_data, 1 207 | 208 | def run(self): 209 | global GANN_instance 210 | 211 | subject = "echo" 212 | GANN_instance = None 213 | best_sol_idx = -1 214 | 215 | while True: 216 | data = {"subject": subject, "data": GANN_instance, "best_solution_idx": best_sol_idx} 217 | data_byte = pickle.dumps(data) 218 | 219 | self.kivy_app.label.text = "Sending a Message of Type {subject} to the Server".format(subject=subject) 220 | try: 221 | self.kivy_app.soc.sendall(data_byte) 222 | except BaseException as e: 223 | self.kivy_app.label.text = "Error Connecting to the Server. The server might has been closed." 224 | print("Error Connecting to the Server: {msg}".format(msg=e)) 225 | break 226 | 227 | self.kivy_app.label.text = "Receiving Reply from the Server" 228 | received_data, status = self.recv() 229 | if status == 0: 230 | self.kivy_app.label.text = "Nothing Received from the Server" 231 | break 232 | else: 233 | self.kivy_app.label.text = "New Message from the Server" 234 | 235 | subject = received_data["subject"] 236 | if subject == "model": 237 | GANN_instance = received_data["data"] 238 | elif subject == "done": 239 | self.kivy_app.label.text = "Model is Trained" 240 | break 241 | else: 242 | self.kivy_app.label.text = "Unrecognized Message Type: {subject}".format(subject=subject) 243 | break 244 | 245 | ga_instance = prepare_GA(GANN_instance) 246 | 247 | ga_instance.run() 248 | 249 | subject = "model" 250 | best_sol_idx = ga_instance.best_solution()[2] 251 | 252 | clientApp = ClientApp() 253 | clientApp.title = "Client App" 254 | clientApp.run() -------------------------------------------------------------------------------- /TutorialProject/Part4/client2.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import numpy 4 | import threading 5 | 6 | import pygad 7 | import pygad.nn 8 | import pygad.gann 9 | 10 | import kivy.app 11 | import kivy.uix.button 12 | import kivy.uix.label 13 | import kivy.uix.boxlayout 14 | import kivy.uix.textinput 15 | 16 | class ClientApp(kivy.app.App): 17 | 18 | def __init__(self): 19 | super().__init__() 20 | 21 | def create_socket(self, *args): 22 | self.soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 23 | self.label.text = "Socket Created" 24 | 25 | self.create_socket_btn.disabled = True 26 | self.connect_btn.disabled = False 27 | self.close_socket_btn.disabled = False 28 | 29 | def connect(self, *args): 30 | try: 31 | self.soc.connect((self.server_ip.text, int(self.server_port.text))) 32 | self.label.text = "Successful Connection to the Server" 33 | 34 | self.connect_btn.disabled = True 35 | self.recv_train_model_btn.disabled = False 36 | 37 | except BaseException as e: 38 | self.label.text = "Error Connecting to the Server" 39 | print("Error Connecting to the Server: {msg}".format(msg=e)) 40 | 41 | self.connect_btn.disabled = False 42 | self.recv_train_model_btn.disabled = True 43 | 44 | def recv_train_model(self, *args): 45 | global GANN_instance 46 | 47 | self.recv_train_model_btn.disabled = True 48 | recvThread = RecvThread(kivy_app=self, buffer_size=1024, recv_timeout=10) 49 | recvThread.start() 50 | 51 | def close_socket(self, *args): 52 | self.soc.close() 53 | self.label.text = "Socket Closed" 54 | 55 | self.create_socket_btn.disabled = False 56 | self.connect_btn.disabled = True 57 | self.recv_train_model_btn.disabled = True 58 | self.close_socket_btn.disabled = True 59 | 60 | def build(self): 61 | self.create_socket_btn = kivy.uix.button.Button(text="Create Socket") 62 | self.create_socket_btn.bind(on_press=self.create_socket) 63 | 64 | self.server_ip = kivy.uix.textinput.TextInput(hint_text="Server IPv4 Address", text="localhost") 65 | self.server_port = kivy.uix.textinput.TextInput(hint_text="Server Port Number", text="10000") 66 | 67 | self.server_info_boxlayout = kivy.uix.boxlayout.BoxLayout(orientation="horizontal") 68 | self.server_info_boxlayout.add_widget(self.server_ip) 69 | self.server_info_boxlayout.add_widget(self.server_port) 70 | 71 | self.connect_btn = kivy.uix.button.Button(text="Connect to Server", disabled=True) 72 | self.connect_btn.bind(on_press=self.connect) 73 | 74 | self.recv_train_model_btn = kivy.uix.button.Button(text="Receive & Train Model", disabled=True) 75 | self.recv_train_model_btn.bind(on_press=self.recv_train_model) 76 | 77 | self.close_socket_btn = kivy.uix.button.Button(text="Close Socket", disabled=True) 78 | self.close_socket_btn.bind(on_press=self.close_socket) 79 | 80 | self.label = kivy.uix.label.Label(text="Socket Status") 81 | 82 | self.box_layout = kivy.uix.boxlayout.BoxLayout(orientation="vertical") 83 | self.box_layout.add_widget(self.create_socket_btn) 84 | self.box_layout.add_widget(self.server_info_boxlayout) 85 | self.box_layout.add_widget(self.connect_btn) 86 | self.box_layout.add_widget(self.recv_train_model_btn) 87 | self.box_layout.add_widget(self.close_socket_btn) 88 | self.box_layout.add_widget(self.label) 89 | 90 | return self.box_layout 91 | 92 | def fitness_func(solution, sol_idx): 93 | global GANN_instance, data_inputs, data_outputs 94 | 95 | predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[sol_idx], 96 | data_inputs=data_inputs) 97 | correct_predictions = numpy.where(predictions == data_outputs)[0].size 98 | solution_fitness = (correct_predictions/data_outputs.size)*100 99 | 100 | return solution_fitness 101 | 102 | def callback_generation(ga_instance): 103 | global GANN_instance, last_fitness 104 | 105 | population_matrices = pygad.gann.population_as_matrices(population_networks=GANN_instance.population_networks, 106 | population_vectors=ga_instance.population) 107 | 108 | GANN_instance.update_population_trained_weights(population_trained_weights=population_matrices) 109 | 110 | # print("Generation = {generation}".format(generation=ga_instance.generations_completed)) 111 | # print("Fitness = {fitness}".format(fitness=ga_instance.best_solution()[1])) 112 | # print("Change = {change}".format(change=ga_instance.best_solution()[1] - last_fitness)) 113 | 114 | #last_fitness = 0 115 | 116 | def prepare_GA(GANN_instance): 117 | # population does not hold the numerical weights of the network instead it holds a list of references to each last layer of each network (i.e. solution) in the population. A solution or a network can be used interchangeably. 118 | # If there is a population with 3 solutions (i.e. networks), then the population is a list with 3 elements. Each element is a reference to the last layer of each network. Using such a reference, all details of the network can be accessed. 119 | population_vectors = pygad.gann.population_as_vectors(population_networks=GANN_instance.population_networks) 120 | 121 | # To prepare the initial population, there are 2 ways: 122 | # 1) Prepare it yourself and pass it to the initial_population parameter. This way is useful when the user wants to start the genetic algorithm with a custom initial population. 123 | # 2) Assign valid integer values to the sol_per_pop and num_genes parameters. If the initial_population parameter exists, then the sol_per_pop and num_genes parameters are useless. 124 | initial_population = population_vectors.copy() 125 | 126 | num_parents_mating = 4 # Number of solutions to be selected as parents in the mating pool. 127 | 128 | num_generations = 500 # Number of generations. 129 | 130 | mutation_percent_genes = 5 # Percentage of genes to mutate. This parameter has no action if the parameter mutation_num_genes exists. 131 | 132 | parent_selection_type = "sss" # Type of parent selection. 133 | 134 | crossover_type = "single_point" # Type of the crossover operator. 135 | 136 | mutation_type = "random" # Type of the mutation operator. 137 | 138 | keep_parents = 1 # Number of parents to keep in the next population. -1 means keep all parents and 0 means keep nothing. 139 | 140 | init_range_low = -2 141 | init_range_high = 5 142 | 143 | ga_instance = pygad.GA(num_generations=num_generations, 144 | num_parents_mating=num_parents_mating, 145 | initial_population=initial_population, 146 | fitness_func=fitness_func, 147 | mutation_percent_genes=mutation_percent_genes, 148 | init_range_low=init_range_low, 149 | init_range_high=init_range_high, 150 | parent_selection_type=parent_selection_type, 151 | crossover_type=crossover_type, 152 | mutation_type=mutation_type, 153 | keep_parents=keep_parents, 154 | on_generation=callback_generation) 155 | 156 | return ga_instance 157 | 158 | # Preparing the NumPy array of the inputs. 159 | data_inputs = numpy.array([[1, 1], 160 | [1, 0]]) 161 | 162 | # Preparing the NumPy array of the outputs. 163 | data_outputs = numpy.array([0, 164 | 1]) 165 | 166 | class RecvThread(threading.Thread): 167 | 168 | def __init__(self, kivy_app, buffer_size, recv_timeout): 169 | threading.Thread.__init__(self) 170 | self.kivy_app = kivy_app 171 | self.buffer_size = buffer_size 172 | self.recv_timeout = recv_timeout 173 | 174 | def recv(self): 175 | received_data = b"" 176 | while True: # str(received_data)[-2] != '.': 177 | try: 178 | self.kivy_app.soc.settimeout(self.recv_timeout) 179 | received_data += self.kivy_app.soc.recv(self.buffer_size) 180 | 181 | try: 182 | pickle.loads(received_data) 183 | # If the previous pickle.loads() statement is passed, this means all the data is received. 184 | # Thus, no need to continue the loop and a break statement should be excuted. 185 | break 186 | except BaseException: 187 | # An exception is expected when the data is not 100% received. 188 | pass 189 | 190 | except socket.timeout: 191 | print("A socket.timeout exception occurred because the server did not send any data for {recv_timeout} seconds.".format(recv_timeout=self.recv_timeout)) 192 | self.kivy_app.label.text = "{recv_timeout} Seconds of Inactivity. socket.timeout Exception Occurred".format(recv_timeout=self.recv_timeout) 193 | return None, 0 194 | except BaseException as e: 195 | return None, 0 196 | print("Error While Receiving Data from the Server: {msg}.".format(msg=e)) 197 | self.kivy_app.label.text = "Error While Receiving Data from the Server" 198 | 199 | try: 200 | received_data = pickle.loads(received_data) 201 | except BaseException as e: 202 | print("Error Decoding the Data: {msg}.\n".format(msg=e)) 203 | self.kivy_app.label.text = "Error Decoding the Client's Data" 204 | return None, 0 205 | 206 | return received_data, 1 207 | 208 | def run(self): 209 | global GANN_instance 210 | 211 | subject = "echo" 212 | GANN_instance = None 213 | best_sol_idx = -1 214 | 215 | while True: 216 | data = {"subject": subject, "data": GANN_instance, "best_solution_idx": best_sol_idx} 217 | data_byte = pickle.dumps(data) 218 | 219 | self.kivy_app.label.text = "Sending a Message of Type {subject} to the Server".format(subject=subject) 220 | try: 221 | self.kivy_app.soc.sendall(data_byte) 222 | except BaseException as e: 223 | self.kivy_app.label.text = "Error Connecting to the Server. The server might has been closed." 224 | print("Error Connecting to the Server: {msg}".format(msg=e)) 225 | break 226 | 227 | self.kivy_app.label.text = "Receiving Reply from the Server" 228 | received_data, status = self.recv() 229 | if status == 0: 230 | self.kivy_app.label.text = "Nothing Received from the Server" 231 | break 232 | else: 233 | self.kivy_app.label.text = "New Message from the Server" 234 | 235 | subject = received_data["subject"] 236 | if subject == "model": 237 | GANN_instance = received_data["data"] 238 | elif subject == "done": 239 | self.kivy_app.label.text = "Model is Trained" 240 | break 241 | else: 242 | self.kivy_app.label.text = "Unrecognized Message Type: {subject}".format(subject=subject) 243 | break 244 | 245 | ga_instance = prepare_GA(GANN_instance) 246 | 247 | ga_instance.run() 248 | 249 | subject = "model" 250 | best_sol_idx = ga_instance.best_solution()[2] 251 | 252 | clientApp = ClientApp() 253 | clientApp.title = "Client App" 254 | clientApp.run() -------------------------------------------------------------------------------- /TutorialProject/Part4/server.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import threading 4 | import time 5 | import numpy 6 | 7 | import pygad.nn 8 | import pygad.gann 9 | 10 | import kivy.app 11 | import kivy.uix.button 12 | import kivy.uix.label 13 | import kivy.uix.textinput 14 | import kivy.uix.boxlayout 15 | 16 | class ServerApp(kivy.app.App): 17 | 18 | def __init__(self): 19 | super().__init__() 20 | 21 | def create_socket(self, *args): 22 | self.soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 23 | self.label.text = "Socket Created" 24 | 25 | self.create_socket_btn.disabled = True 26 | self.bind_btn.disabled = False 27 | self.close_socket_btn.disabled = False 28 | 29 | def bind_socket(self, *args): 30 | ipv4_address = self.server_ip.text 31 | port_number = self.server_port.text 32 | self.soc.bind((ipv4_address, int(port_number))) 33 | self.label.text = "Socket Bound to IPv4 & Port Number" 34 | 35 | self.bind_btn.disabled = True 36 | self.listen_btn.disabled = False 37 | 38 | def listen_accept(self, *args): 39 | self.soc.listen(1) 40 | self.label.text = "Socket is Listening for Connections" 41 | 42 | self.listen_btn.disabled = True 43 | 44 | self.listenThread = ListenThread(kivy_app=self) 45 | self.listenThread.start() 46 | 47 | def close_socket(self, *args): 48 | self.soc.close() 49 | self.label.text = "Socket Closed" 50 | 51 | self.create_socket_btn.disabled = False 52 | self.bind_btn.disabled = True 53 | self.listen_btn.disabled = True 54 | self.close_socket_btn.disabled = True 55 | 56 | def build(self): 57 | self.create_socket_btn = kivy.uix.button.Button(text="Create Socket", disabled=False) 58 | self.create_socket_btn.bind(on_press=self.create_socket) 59 | 60 | self.server_ip = kivy.uix.textinput.TextInput(hint_text="IPv4 Address", text="localhost") 61 | self.server_port = kivy.uix.textinput.TextInput(hint_text="Port Number", text="10000") 62 | 63 | self.server_socket_box_layout = kivy.uix.boxlayout.BoxLayout(orientation="horizontal") 64 | self.server_socket_box_layout.add_widget(self.server_ip) 65 | self.server_socket_box_layout.add_widget(self.server_port) 66 | 67 | self.bind_btn = kivy.uix.button.Button(text="Bind Socket", disabled=True) 68 | self.bind_btn.bind(on_press=self.bind_socket) 69 | 70 | self.listen_btn = kivy.uix.button.Button(text="Listen to Connections", disabled=True) 71 | self.listen_btn.bind(on_press=self.listen_accept) 72 | 73 | self.close_socket_btn = kivy.uix.button.Button(text="Close Socket", disabled=True) 74 | self.close_socket_btn.bind(on_press=self.close_socket) 75 | 76 | self.label = kivy.uix.label.Label(text="Socket Status") 77 | 78 | self.box_layout = kivy.uix.boxlayout.BoxLayout(orientation="vertical") 79 | 80 | self.box_layout.add_widget(self.create_socket_btn) 81 | self.box_layout.add_widget(self.server_socket_box_layout) 82 | self.box_layout.add_widget(self.bind_btn) 83 | self.box_layout.add_widget(self.listen_btn) 84 | self.box_layout.add_widget(self.close_socket_btn) 85 | self.box_layout.add_widget(self.label) 86 | 87 | return self.box_layout 88 | 89 | model = None 90 | 91 | # Preparing the NumPy array of the inputs. 92 | data_inputs = numpy.array([[1, 1], 93 | [1, 0], 94 | [0, 1], 95 | [0, 0]]) 96 | 97 | # Preparing the NumPy array of the outputs. 98 | data_outputs = numpy.array([0, 99 | 1, 100 | 1, 101 | 0]) 102 | 103 | num_classes = 2 104 | num_inputs = 2 105 | 106 | num_solutions = 6 107 | GANN_instance = pygad.gann.GANN(num_solutions=num_solutions, 108 | num_neurons_input=num_inputs, 109 | num_neurons_hidden_layers=[2], 110 | num_neurons_output=num_classes, 111 | hidden_activations=["relu"], 112 | output_activation="softmax") 113 | 114 | class SocketThread(threading.Thread): 115 | 116 | def __init__(self, connection, client_info, kivy_app, buffer_size=1024, recv_timeout=5): 117 | threading.Thread.__init__(self) 118 | self.connection = connection 119 | self.client_info = client_info 120 | self.buffer_size = buffer_size 121 | self.recv_timeout = recv_timeout 122 | self.kivy_app = kivy_app 123 | 124 | def recv(self): 125 | all_data_received_flag = False 126 | received_data = b"" 127 | while True: 128 | try: 129 | 130 | data = self.connection.recv(self.buffer_size) 131 | received_data += data 132 | 133 | try: 134 | pickle.loads(received_data) 135 | # If the previous pickle.loads() statement is passed, this means all the data is received. 136 | # Thus, no need to continue the loop. The flag all_data_received_flag is set to True to signal all data is received. 137 | all_data_received_flag = True 138 | except BaseException: 139 | # An exception is expected when the data is not 100% received. 140 | pass 141 | 142 | if data == b'': # Nothing received from the client. 143 | received_data = b"" 144 | # If still nothing received for a number of seconds specified by the recv_timeout attribute, return with status 0 to close the connection. 145 | if (time.time() - self.recv_start_time) > self.recv_timeout: 146 | return None, 0 # 0 means the connection is no longer active and it should be closed. 147 | 148 | elif all_data_received_flag: 149 | print("All data ({data_len} bytes) Received from {client_info}.".format(client_info=self.client_info, data_len=len(received_data))) 150 | self.kivy_app.label.text = "All data ({data_len} bytes) Received from {client_info}.".format(client_info=self.client_info, data_len=len(received_data)) 151 | 152 | if len(received_data) > 0: 153 | try: 154 | # Decoding the data (bytes). 155 | received_data = pickle.loads(received_data) 156 | # Returning the decoded data. 157 | return received_data, 1 158 | 159 | except BaseException as e: 160 | print("Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 161 | self.kivy_app.label.text = "Error Decoding the Client's Data" 162 | return None, 0 163 | 164 | else: 165 | # In case data are received from the client, update the recv_start_time to the current time to reset the timeout counter. 166 | self.recv_start_time = time.time() 167 | 168 | except BaseException as e: 169 | print("Error Receiving Data from the Client: {msg}.\n".format(msg=e)) 170 | self.kivy_app.label.text = "Error Receiving Data from the Client" 171 | return None, 0 172 | 173 | def model_averaging(self, model, other_model): 174 | model_weights = pygad.nn.layers_weights(last_layer=model, initial=False) 175 | other_model_weights = pygad.nn.layers_weights(last_layer=other_model, initial=False) 176 | 177 | new_weights = numpy.array(model_weights + other_model_weights)/2 178 | 179 | pygad.nn.update_layers_trained_weights(last_layer=model, final_weights=new_weights) 180 | 181 | def reply(self, received_data): 182 | global GANN_instance, data_inputs, data_outputs, model 183 | if (type(received_data) is dict): 184 | if (("data" in received_data.keys()) and ("subject" in received_data.keys())): 185 | subject = received_data["subject"] 186 | print("Client's Message Subject is {subject}.".format(subject=subject)) 187 | self.kivy_app.label.text = "Client's Message Subject is {subject}".format(subject=subject) 188 | 189 | print("Replying to the Client.") 190 | self.kivy_app.label.text = "Replying to the Client" 191 | if subject == "echo": 192 | if model is None: 193 | data = {"subject": "model", "data": GANN_instance} 194 | else: 195 | predictions = pygad.nn.predict(last_layer=model, data_inputs=data_inputs) 196 | error = numpy.sum(numpy.abs(predictions - data_outputs)) 197 | # In case a client sent a model to the server despite that the model error is 0.0. In this case, no need to make changes in the model. 198 | if error == 0: 199 | data = {"subject": "done", "data": None} 200 | else: 201 | data = {"subject": "model", "data": GANN_instance} 202 | 203 | try: 204 | response = pickle.dumps(data) 205 | except BaseException as e: 206 | print("Error Encoding the Message: {msg}.\n".format(msg=e)) 207 | self.kivy_app.label.text = "Error Encoding the Message" 208 | elif subject == "model": 209 | try: 210 | GANN_instance = received_data["data"] 211 | best_model_idx = received_data["best_solution_idx"] 212 | 213 | best_model = GANN_instance.population_networks[best_model_idx] 214 | if model is None: 215 | model = best_model 216 | else: 217 | predictions = pygad.nn.predict(last_layer=model, data_inputs=data_inputs) 218 | 219 | error = numpy.sum(numpy.abs(predictions - data_outputs)) 220 | 221 | # In case a client sent a model to the server despite that the model error is 0.0. In this case, no need to make changes in the model. 222 | if error == 0: 223 | data = {"subject": "done", "data": None} 224 | response = pickle.dumps(data) 225 | return 226 | 227 | self.model_averaging(model, best_model) 228 | 229 | # print(best_model.trained_weights) 230 | # print(model.trained_weights) 231 | 232 | predictions = pygad.nn.predict(last_layer=model, data_inputs=data_inputs) 233 | print("Model Predictions: {predictions}".format(predictions=predictions)) 234 | 235 | error = numpy.sum(numpy.abs(predictions - data_outputs)) 236 | print("Prediction Error = {error}".format(error=error)) 237 | self.kivy_app.label.text = "Prediction Error = {error}".format(error=error) 238 | 239 | if error != 0: 240 | data = {"subject": "model", "data": GANN_instance} 241 | response = pickle.dumps(data) 242 | else: 243 | data = {"subject": "done", "data": None} 244 | response = pickle.dumps(data) 245 | 246 | except BaseException as e: 247 | print("Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 248 | self.kivy_app.label.text = "Error Decoding the Client's Data" 249 | else: 250 | response = pickle.dumps("Response from the Server") 251 | 252 | try: 253 | self.connection.sendall(response) 254 | except BaseException as e: 255 | print("Error Sending Data to the Client: {msg}.\n".format(msg=e)) 256 | self.kivy_app.label.text = "Error Sending Data to the Client: {msg}".format(msg=e) 257 | 258 | else: 259 | print("The received dictionary from the client must have the 'subject' and 'data' keys available. The existing keys are {d_keys}.".format(d_keys=received_data.keys())) 260 | self.kivy_app.label.text = "Error Parsing Received Dictionary" 261 | else: 262 | print("A dictionary is expected to be received from the client but {d_type} received.".format(d_type=type(received_data))) 263 | self.kivy_app.label.text = "A dictionary is expected but {d_type} received.".format(d_type=type(received_data)) 264 | 265 | def run(self): 266 | print("Running a Thread for the Connection with {client_info}.".format(client_info=self.client_info)) 267 | self.kivy_app.label.text = "Running a Thread for the Connection with {client_info}.".format(client_info=self.client_info) 268 | 269 | # This while loop allows the server to wait for the client to send data more than once within the same connection. 270 | while True: 271 | self.recv_start_time = time.time() 272 | time_struct = time.gmtime() 273 | date_time = "Waiting to Receive Data Starting from {day}/{month}/{year} {hour}:{minute}:{second} GMT".format(year=time_struct.tm_year, month=time_struct.tm_mon, day=time_struct.tm_mday, hour=time_struct.tm_hour, minute=time_struct.tm_min, second=time_struct.tm_sec) 274 | print(date_time) 275 | received_data, status = self.recv() 276 | if status == 0: 277 | self.connection.close() 278 | self.kivy_app.label.text = "Connection Closed with {client_info}".format(client_info=self.client_info) 279 | print("Connection Closed with {client_info} either due to inactivity for {recv_timeout} seconds or due to an error.".format(client_info=self.client_info, recv_timeout=self.recv_timeout), end="\n\n") 280 | break 281 | 282 | # print(received_data) 283 | self.reply(received_data) 284 | 285 | class ListenThread(threading.Thread): 286 | 287 | def __init__(self, kivy_app): 288 | threading.Thread.__init__(self) 289 | self.kivy_app = kivy_app 290 | 291 | def run(self): 292 | while True: 293 | try: 294 | connection, client_info = self.kivy_app.soc.accept() 295 | self.kivy_app.label.text = "New Connection from {client_info}".format(client_info=client_info) 296 | socket_thread = SocketThread(connection=connection, 297 | client_info=client_info, 298 | kivy_app=self.kivy_app, 299 | buffer_size=1024, 300 | recv_timeout=10) 301 | socket_thread.start() 302 | except BaseException as e: 303 | self.kivy_app.soc.close() 304 | print(e) 305 | self.kivy_app.label.text = "Socket is No Longer Accepting Connections" 306 | self.kivy_app.create_socket_btn.disabled = False 307 | self.kivy_app.close_socket_btn.disabled = True 308 | break 309 | 310 | serverApp = ServerApp() 311 | serverApp.title="Server App" 312 | serverApp.run() -------------------------------------------------------------------------------- /client1.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import numpy 4 | import threading 5 | 6 | import pygad 7 | import pygad.nn 8 | import pygad.gann 9 | 10 | import kivy.app 11 | import kivy.uix.button 12 | import kivy.uix.label 13 | import kivy.uix.boxlayout 14 | import kivy.uix.textinput 15 | 16 | class ClientApp(kivy.app.App): 17 | 18 | def __init__(self): 19 | super().__init__() 20 | 21 | def create_socket(self, *args): 22 | self.soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 23 | self.label.text = "Socket Created" 24 | 25 | self.create_socket_btn.disabled = True 26 | self.connect_btn.disabled = False 27 | self.close_socket_btn.disabled = False 28 | 29 | def connect(self, *args): 30 | try: 31 | self.soc.connect((self.server_ip.text, int(self.server_port.text))) 32 | self.label.text = "Successful Connection to the Server" 33 | 34 | self.connect_btn.disabled = True 35 | self.recv_train_model_btn.disabled = False 36 | 37 | except BaseException as e: 38 | self.label.text = "Error Connecting to the Server" 39 | print("Error Connecting to the Server: {msg}".format(msg=e)) 40 | 41 | self.connect_btn.disabled = False 42 | self.recv_train_model_btn.disabled = True 43 | 44 | def recv_train_model(self, *args): 45 | global GANN_instance 46 | 47 | self.recv_train_model_btn.disabled = True 48 | recvThread = RecvThread(kivy_app=self, buffer_size=1024, recv_timeout=10) 49 | recvThread.start() 50 | 51 | def close_socket(self, *args): 52 | self.soc.close() 53 | self.label.text = "Socket Closed" 54 | 55 | self.create_socket_btn.disabled = False 56 | self.connect_btn.disabled = True 57 | self.recv_train_model_btn.disabled = True 58 | self.close_socket_btn.disabled = True 59 | 60 | def build(self): 61 | self.create_socket_btn = kivy.uix.button.Button(text="Create Socket") 62 | self.create_socket_btn.bind(on_press=self.create_socket) 63 | 64 | self.server_ip = kivy.uix.textinput.TextInput(hint_text="Server IPv4 Address", text="localhost") 65 | self.server_port = kivy.uix.textinput.TextInput(hint_text="Server Port Number", text="10000") 66 | 67 | self.server_info_boxlayout = kivy.uix.boxlayout.BoxLayout(orientation="horizontal") 68 | self.server_info_boxlayout.add_widget(self.server_ip) 69 | self.server_info_boxlayout.add_widget(self.server_port) 70 | 71 | self.connect_btn = kivy.uix.button.Button(text="Connect to Server", disabled=True) 72 | self.connect_btn.bind(on_press=self.connect) 73 | 74 | self.recv_train_model_btn = kivy.uix.button.Button(text="Receive & Train Model", disabled=True) 75 | self.recv_train_model_btn.bind(on_press=self.recv_train_model) 76 | 77 | self.close_socket_btn = kivy.uix.button.Button(text="Close Socket", disabled=True) 78 | self.close_socket_btn.bind(on_press=self.close_socket) 79 | 80 | self.label = kivy.uix.label.Label(text="Socket Status") 81 | 82 | self.box_layout = kivy.uix.boxlayout.BoxLayout(orientation="vertical") 83 | self.box_layout.add_widget(self.create_socket_btn) 84 | self.box_layout.add_widget(self.server_info_boxlayout) 85 | self.box_layout.add_widget(self.connect_btn) 86 | self.box_layout.add_widget(self.recv_train_model_btn) 87 | self.box_layout.add_widget(self.close_socket_btn) 88 | self.box_layout.add_widget(self.label) 89 | 90 | return self.box_layout 91 | 92 | def fitness_func(solution, sol_idx): 93 | global GANN_instance, data_inputs, data_outputs 94 | 95 | predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[sol_idx], 96 | data_inputs=data_inputs) 97 | correct_predictions = numpy.where(predictions == data_outputs)[0].size 98 | solution_fitness = (correct_predictions/data_outputs.size)*100 99 | 100 | return solution_fitness 101 | 102 | def callback_generation(ga_instance): 103 | global GANN_instance, last_fitness 104 | 105 | population_matrices = pygad.gann.population_as_matrices(population_networks=GANN_instance.population_networks, 106 | population_vectors=ga_instance.population) 107 | 108 | GANN_instance.update_population_trained_weights(population_trained_weights=population_matrices) 109 | 110 | # print("Generation = {generation}".format(generation=ga_instance.generations_completed)) 111 | # print("Fitness = {fitness}".format(fitness=ga_instance.best_solution()[1])) 112 | # print("Change = {change}".format(change=ga_instance.best_solution()[1] - last_fitness)) 113 | 114 | #last_fitness = 0 115 | 116 | def prepare_GA(GANN_instance): 117 | # population does not hold the numerical weights of the network instead it holds a list of references to each last layer of each network (i.e. solution) in the population. A solution or a network can be used interchangeably. 118 | # If there is a population with 3 solutions (i.e. networks), then the population is a list with 3 elements. Each element is a reference to the last layer of each network. Using such a reference, all details of the network can be accessed. 119 | population_vectors = pygad.gann.population_as_vectors(population_networks=GANN_instance.population_networks) 120 | 121 | # To prepare the initial population, there are 2 ways: 122 | # 1) Prepare it yourself and pass it to the initial_population parameter. This way is useful when the user wants to start the genetic algorithm with a custom initial population. 123 | # 2) Assign valid integer values to the sol_per_pop and num_genes parameters. If the initial_population parameter exists, then the sol_per_pop and num_genes parameters are useless. 124 | initial_population = population_vectors.copy() 125 | 126 | num_parents_mating = 4 # Number of solutions to be selected as parents in the mating pool. 127 | 128 | num_generations = 500 # Number of generations. 129 | 130 | mutation_percent_genes = 5 # Percentage of genes to mutate. This parameter has no action if the parameter mutation_num_genes exists. 131 | 132 | parent_selection_type = "sss" # Type of parent selection. 133 | 134 | crossover_type = "single_point" # Type of the crossover operator. 135 | 136 | mutation_type = "random" # Type of the mutation operator. 137 | 138 | keep_parents = 1 # Number of parents to keep in the next population. -1 means keep all parents and 0 means keep nothing. 139 | 140 | init_range_low = -2 141 | init_range_high = 5 142 | 143 | ga_instance = pygad.GA(num_generations=num_generations, 144 | num_parents_mating=num_parents_mating, 145 | initial_population=initial_population, 146 | fitness_func=fitness_func, 147 | mutation_percent_genes=mutation_percent_genes, 148 | init_range_low=init_range_low, 149 | init_range_high=init_range_high, 150 | parent_selection_type=parent_selection_type, 151 | crossover_type=crossover_type, 152 | mutation_type=mutation_type, 153 | keep_parents=keep_parents, 154 | on_generation=callback_generation) 155 | 156 | return ga_instance 157 | 158 | # Preparing the NumPy array of the inputs. 159 | data_inputs = numpy.array([[0, 1], 160 | [0, 0]]) 161 | 162 | # Preparing the NumPy array of the outputs. 163 | data_outputs = numpy.array([1, 164 | 0]) 165 | 166 | class RecvThread(threading.Thread): 167 | 168 | def __init__(self, kivy_app, buffer_size, recv_timeout): 169 | threading.Thread.__init__(self) 170 | self.kivy_app = kivy_app 171 | self.buffer_size = buffer_size 172 | self.recv_timeout = recv_timeout 173 | 174 | def recv(self): 175 | received_data = b"" 176 | while True: # str(received_data)[-2] != '.': 177 | try: 178 | self.kivy_app.soc.settimeout(self.recv_timeout) 179 | received_data += self.kivy_app.soc.recv(self.buffer_size) 180 | 181 | try: 182 | pickle.loads(received_data) 183 | self.kivy_app.label.text = "All data is received from the server." 184 | print("All data is received from the server.") 185 | # If the previous pickle.loads() statement is passed, this means all the data is received. 186 | # Thus, no need to continue the loop and a break statement should be excuted. 187 | break 188 | except BaseException: 189 | # An exception is expected when the data is not 100% received. 190 | pass 191 | 192 | except socket.timeout: 193 | print("A socket.timeout exception occurred because the server did not send any data for {recv_timeout} seconds.".format(recv_timeout=self.recv_timeout)) 194 | self.kivy_app.label.text = "{recv_timeout} Seconds of Inactivity. socket.timeout Exception Occurred".format(recv_timeout=self.recv_timeout) 195 | return None, 0 196 | except BaseException as e: 197 | return None, 0 198 | print("Error While Receiving Data from the Server: {msg}.".format(msg=e)) 199 | self.kivy_app.label.text = "Error While Receiving Data from the Server" 200 | 201 | try: 202 | received_data = pickle.loads(received_data) 203 | except BaseException as e: 204 | print("Error Decoding the Data: {msg}.\n".format(msg=e)) 205 | self.kivy_app.label.text = "Error Decoding the Client's Data" 206 | return None, 0 207 | 208 | return received_data, 1 209 | 210 | def run(self): 211 | global GANN_instance 212 | 213 | subject = "echo" 214 | GANN_instance = None 215 | best_sol_idx = -1 216 | 217 | while True: 218 | data = {"subject": subject, "data": GANN_instance, "best_solution_idx": best_sol_idx} 219 | data_byte = pickle.dumps(data) 220 | 221 | self.kivy_app.label.text = "Sending a Message of Type {subject} to the Server".format(subject=subject) 222 | try: 223 | self.kivy_app.soc.sendall(data_byte) 224 | except BaseException as e: 225 | self.kivy_app.label.text = "Error Connecting to the Server. The server might has been closed." 226 | print("Error Connecting to the Server: {msg}".format(msg=e)) 227 | break 228 | 229 | self.kivy_app.label.text = "Receiving Reply from the Server" 230 | received_data, status = self.recv() 231 | if status == 0: 232 | self.kivy_app.label.text = "Nothing Received from the Server" 233 | break 234 | else: 235 | self.kivy_app.label.text = "New Message from the Server" 236 | 237 | subject = received_data["subject"] 238 | if subject == "model": 239 | GANN_instance = received_data["data"] 240 | elif subject == "done": 241 | self.kivy_app.label.text = "Model is Trained" 242 | break 243 | else: 244 | self.kivy_app.label.text = "Unrecognized Message Type: {subject}".format(subject=subject) 245 | break 246 | 247 | ga_instance = prepare_GA(GANN_instance) 248 | 249 | ga_instance.run() 250 | 251 | subject = "model" 252 | best_sol_idx = ga_instance.best_solution()[2] 253 | 254 | clientApp = ClientApp() 255 | clientApp.title = "Client App" 256 | clientApp.run() 257 | -------------------------------------------------------------------------------- /client2.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import numpy 4 | import threading 5 | 6 | import pygad 7 | import pygad.nn 8 | import pygad.gann 9 | 10 | import kivy.app 11 | import kivy.uix.button 12 | import kivy.uix.label 13 | import kivy.uix.boxlayout 14 | import kivy.uix.textinput 15 | 16 | class ClientApp(kivy.app.App): 17 | 18 | def __init__(self): 19 | super().__init__() 20 | 21 | def create_socket(self, *args): 22 | self.soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 23 | self.label.text = "Socket Created" 24 | 25 | self.create_socket_btn.disabled = True 26 | self.connect_btn.disabled = False 27 | self.close_socket_btn.disabled = False 28 | 29 | def connect(self, *args): 30 | try: 31 | self.soc.connect((self.server_ip.text, int(self.server_port.text))) 32 | self.label.text = "Successful Connection to the Server" 33 | 34 | self.connect_btn.disabled = True 35 | self.recv_train_model_btn.disabled = False 36 | 37 | except BaseException as e: 38 | self.label.text = "Error Connecting to the Server" 39 | print("Error Connecting to the Server: {msg}".format(msg=e)) 40 | 41 | self.connect_btn.disabled = False 42 | self.recv_train_model_btn.disabled = True 43 | 44 | def recv_train_model(self, *args): 45 | global GANN_instance 46 | 47 | self.recv_train_model_btn.disabled = True 48 | recvThread = RecvThread(kivy_app=self, buffer_size=1024, recv_timeout=10) 49 | recvThread.start() 50 | 51 | def close_socket(self, *args): 52 | self.soc.close() 53 | self.label.text = "Socket Closed" 54 | 55 | self.create_socket_btn.disabled = False 56 | self.connect_btn.disabled = True 57 | self.recv_train_model_btn.disabled = True 58 | self.close_socket_btn.disabled = True 59 | 60 | def build(self): 61 | self.create_socket_btn = kivy.uix.button.Button(text="Create Socket") 62 | self.create_socket_btn.bind(on_press=self.create_socket) 63 | 64 | self.server_ip = kivy.uix.textinput.TextInput(hint_text="Server IPv4 Address", text="localhost") 65 | self.server_port = kivy.uix.textinput.TextInput(hint_text="Server Port Number", text="10000") 66 | 67 | self.server_info_boxlayout = kivy.uix.boxlayout.BoxLayout(orientation="horizontal") 68 | self.server_info_boxlayout.add_widget(self.server_ip) 69 | self.server_info_boxlayout.add_widget(self.server_port) 70 | 71 | self.connect_btn = kivy.uix.button.Button(text="Connect to Server", disabled=True) 72 | self.connect_btn.bind(on_press=self.connect) 73 | 74 | self.recv_train_model_btn = kivy.uix.button.Button(text="Receive & Train Model", disabled=True) 75 | self.recv_train_model_btn.bind(on_press=self.recv_train_model) 76 | 77 | self.close_socket_btn = kivy.uix.button.Button(text="Close Socket", disabled=True) 78 | self.close_socket_btn.bind(on_press=self.close_socket) 79 | 80 | self.label = kivy.uix.label.Label(text="Socket Status") 81 | 82 | self.box_layout = kivy.uix.boxlayout.BoxLayout(orientation="vertical") 83 | self.box_layout.add_widget(self.create_socket_btn) 84 | self.box_layout.add_widget(self.server_info_boxlayout) 85 | self.box_layout.add_widget(self.connect_btn) 86 | self.box_layout.add_widget(self.recv_train_model_btn) 87 | self.box_layout.add_widget(self.close_socket_btn) 88 | self.box_layout.add_widget(self.label) 89 | 90 | return self.box_layout 91 | 92 | def fitness_func(solution, sol_idx): 93 | global GANN_instance, data_inputs, data_outputs 94 | 95 | predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[sol_idx], 96 | data_inputs=data_inputs) 97 | correct_predictions = numpy.where(predictions == data_outputs)[0].size 98 | solution_fitness = (correct_predictions/data_outputs.size)*100 99 | 100 | return solution_fitness 101 | 102 | def callback_generation(ga_instance): 103 | global GANN_instance, last_fitness 104 | 105 | population_matrices = pygad.gann.population_as_matrices(population_networks=GANN_instance.population_networks, 106 | population_vectors=ga_instance.population) 107 | 108 | GANN_instance.update_population_trained_weights(population_trained_weights=population_matrices) 109 | 110 | # print("Generation = {generation}".format(generation=ga_instance.generations_completed)) 111 | # print("Fitness = {fitness}".format(fitness=ga_instance.best_solution()[1])) 112 | # print("Change = {change}".format(change=ga_instance.best_solution()[1] - last_fitness)) 113 | 114 | #last_fitness = 0 115 | 116 | def prepare_GA(GANN_instance): 117 | # population does not hold the numerical weights of the network instead it holds a list of references to each last layer of each network (i.e. solution) in the population. A solution or a network can be used interchangeably. 118 | # If there is a population with 3 solutions (i.e. networks), then the population is a list with 3 elements. Each element is a reference to the last layer of each network. Using such a reference, all details of the network can be accessed. 119 | population_vectors = pygad.gann.population_as_vectors(population_networks=GANN_instance.population_networks) 120 | 121 | # To prepare the initial population, there are 2 ways: 122 | # 1) Prepare it yourself and pass it to the initial_population parameter. This way is useful when the user wants to start the genetic algorithm with a custom initial population. 123 | # 2) Assign valid integer values to the sol_per_pop and num_genes parameters. If the initial_population parameter exists, then the sol_per_pop and num_genes parameters are useless. 124 | initial_population = population_vectors.copy() 125 | 126 | num_parents_mating = 4 # Number of solutions to be selected as parents in the mating pool. 127 | 128 | num_generations = 500 # Number of generations. 129 | 130 | mutation_percent_genes = 5 # Percentage of genes to mutate. This parameter has no action if the parameter mutation_num_genes exists. 131 | 132 | parent_selection_type = "sss" # Type of parent selection. 133 | 134 | crossover_type = "single_point" # Type of the crossover operator. 135 | 136 | mutation_type = "random" # Type of the mutation operator. 137 | 138 | keep_parents = 1 # Number of parents to keep in the next population. -1 means keep all parents and 0 means keep nothing. 139 | 140 | init_range_low = -2 141 | init_range_high = 5 142 | 143 | ga_instance = pygad.GA(num_generations=num_generations, 144 | num_parents_mating=num_parents_mating, 145 | initial_population=initial_population, 146 | fitness_func=fitness_func, 147 | mutation_percent_genes=mutation_percent_genes, 148 | init_range_low=init_range_low, 149 | init_range_high=init_range_high, 150 | parent_selection_type=parent_selection_type, 151 | crossover_type=crossover_type, 152 | mutation_type=mutation_type, 153 | keep_parents=keep_parents, 154 | on_generation=callback_generation) 155 | 156 | return ga_instance 157 | 158 | # Preparing the NumPy array of the inputs. 159 | data_inputs = numpy.array([[1, 1], 160 | [1, 0]]) 161 | 162 | # Preparing the NumPy array of the outputs. 163 | data_outputs = numpy.array([0, 164 | 1]) 165 | 166 | class RecvThread(threading.Thread): 167 | 168 | def __init__(self, kivy_app, buffer_size, recv_timeout): 169 | threading.Thread.__init__(self) 170 | self.kivy_app = kivy_app 171 | self.buffer_size = buffer_size 172 | self.recv_timeout = recv_timeout 173 | 174 | def recv(self): 175 | received_data = b"" 176 | while True: # str(received_data)[-2] != '.': 177 | try: 178 | self.kivy_app.soc.settimeout(self.recv_timeout) 179 | received_data += self.kivy_app.soc.recv(self.buffer_size) 180 | 181 | try: 182 | pickle.loads(received_data) 183 | self.kivy_app.label.text = "All data is received from the server." 184 | print("All data is received from the server.") 185 | 186 | # If the previous pickle.loads() statement is passed, this means all the data is received. 187 | # Thus, no need to continue the loop and a break statement should be excuted. 188 | break 189 | except BaseException: 190 | # An exception is expected when the data is not 100% received. 191 | pass 192 | 193 | except socket.timeout: 194 | print("A socket.timeout exception occurred because the server did not send any data for {recv_timeout} seconds.".format(recv_timeout=self.recv_timeout)) 195 | self.kivy_app.label.text = "{recv_timeout} Seconds of Inactivity. socket.timeout Exception Occurred".format(recv_timeout=self.recv_timeout) 196 | return None, 0 197 | except BaseException as e: 198 | return None, 0 199 | print("Error While Receiving Data from the Server: {msg}.".format(msg=e)) 200 | self.kivy_app.label.text = "Error While Receiving Data from the Server" 201 | 202 | try: 203 | received_data = pickle.loads(received_data) 204 | except BaseException as e: 205 | print("Error Decoding the Data: {msg}.\n".format(msg=e)) 206 | self.kivy_app.label.text = "Error Decoding the Client's Data" 207 | return None, 0 208 | 209 | return received_data, 1 210 | 211 | def run(self): 212 | global GANN_instance 213 | 214 | subject = "echo" 215 | GANN_instance = None 216 | best_sol_idx = -1 217 | 218 | while True: 219 | data = {"subject": subject, "data": GANN_instance, "best_solution_idx": best_sol_idx} 220 | data_byte = pickle.dumps(data) 221 | 222 | self.kivy_app.label.text = "Sending a Message of Type {subject} to the Server".format(subject=subject) 223 | try: 224 | self.kivy_app.soc.sendall(data_byte) 225 | except BaseException as e: 226 | self.kivy_app.label.text = "Error Connecting to the Server. The server might has been closed." 227 | print("Error Connecting to the Server: {msg}".format(msg=e)) 228 | break 229 | 230 | self.kivy_app.label.text = "Receiving Reply from the Server" 231 | received_data, status = self.recv() 232 | if status == 0: 233 | self.kivy_app.label.text = "Nothing Received from the Server" 234 | break 235 | else: 236 | self.kivy_app.label.text = "New Message from the Server" 237 | 238 | subject = received_data["subject"] 239 | if subject == "model": 240 | GANN_instance = received_data["data"] 241 | elif subject == "done": 242 | self.kivy_app.label.text = "Model is Trained" 243 | break 244 | else: 245 | self.kivy_app.label.text = "Unrecognized Message Type: {subject}".format(subject=subject) 246 | break 247 | 248 | ga_instance = prepare_GA(GANN_instance) 249 | 250 | ga_instance.run() 251 | 252 | subject = "model" 253 | best_sol_idx = ga_instance.best_solution()[2] 254 | 255 | clientApp = ClientApp() 256 | clientApp.title = "Client App" 257 | clientApp.run() 258 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | pygad -------------------------------------------------------------------------------- /server.py: -------------------------------------------------------------------------------- 1 | import socket 2 | import pickle 3 | import threading 4 | import time 5 | import numpy 6 | 7 | import pygad.nn 8 | import pygad.gann 9 | 10 | import kivy.app 11 | import kivy.uix.button 12 | import kivy.uix.label 13 | import kivy.uix.textinput 14 | import kivy.uix.boxlayout 15 | 16 | class ServerApp(kivy.app.App): 17 | 18 | def __init__(self): 19 | super().__init__() 20 | 21 | def create_socket(self, *args): 22 | self.soc = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM) 23 | self.label.text = "Socket Created" 24 | 25 | self.create_socket_btn.disabled = True 26 | self.bind_btn.disabled = False 27 | self.close_socket_btn.disabled = False 28 | 29 | def bind_socket(self, *args): 30 | ipv4_address = self.server_ip.text 31 | port_number = self.server_port.text 32 | self.soc.bind((ipv4_address, int(port_number))) 33 | self.label.text = "Socket Bound to IPv4 & Port Number" 34 | 35 | self.bind_btn.disabled = True 36 | self.listen_btn.disabled = False 37 | 38 | def listen_accept(self, *args): 39 | self.soc.listen(1) 40 | self.label.text = "Socket is Listening for Connections" 41 | 42 | self.listen_btn.disabled = True 43 | 44 | self.listenThread = ListenThread(kivy_app=self) 45 | self.listenThread.start() 46 | 47 | def close_socket(self, *args): 48 | self.soc.close() 49 | self.label.text = "Socket Closed" 50 | 51 | self.create_socket_btn.disabled = False 52 | self.bind_btn.disabled = True 53 | self.listen_btn.disabled = True 54 | self.close_socket_btn.disabled = True 55 | 56 | def build(self): 57 | self.create_socket_btn = kivy.uix.button.Button(text="Create Socket", disabled=False) 58 | self.create_socket_btn.bind(on_press=self.create_socket) 59 | 60 | self.server_ip = kivy.uix.textinput.TextInput(hint_text="IPv4 Address", text="localhost") 61 | self.server_port = kivy.uix.textinput.TextInput(hint_text="Port Number", text="10000") 62 | 63 | self.server_socket_box_layout = kivy.uix.boxlayout.BoxLayout(orientation="horizontal") 64 | self.server_socket_box_layout.add_widget(self.server_ip) 65 | self.server_socket_box_layout.add_widget(self.server_port) 66 | 67 | self.bind_btn = kivy.uix.button.Button(text="Bind Socket", disabled=True) 68 | self.bind_btn.bind(on_press=self.bind_socket) 69 | 70 | self.listen_btn = kivy.uix.button.Button(text="Listen to Connections", disabled=True) 71 | self.listen_btn.bind(on_press=self.listen_accept) 72 | 73 | self.close_socket_btn = kivy.uix.button.Button(text="Close Socket", disabled=True) 74 | self.close_socket_btn.bind(on_press=self.close_socket) 75 | 76 | self.label = kivy.uix.label.Label(text="Socket Status") 77 | 78 | self.box_layout = kivy.uix.boxlayout.BoxLayout(orientation="vertical") 79 | 80 | self.box_layout.add_widget(self.create_socket_btn) 81 | self.box_layout.add_widget(self.server_socket_box_layout) 82 | self.box_layout.add_widget(self.bind_btn) 83 | self.box_layout.add_widget(self.listen_btn) 84 | self.box_layout.add_widget(self.close_socket_btn) 85 | self.box_layout.add_widget(self.label) 86 | 87 | return self.box_layout 88 | 89 | model = None 90 | 91 | # Preparing the NumPy array of the inputs. 92 | data_inputs = numpy.array([[1, 1], 93 | [1, 0], 94 | [0, 1], 95 | [0, 0]]) 96 | 97 | # Preparing the NumPy array of the outputs. 98 | data_outputs = numpy.array([0, 99 | 1, 100 | 1, 101 | 0]) 102 | 103 | num_classes = 2 104 | num_inputs = 2 105 | 106 | num_solutions = 6 107 | GANN_instance = pygad.gann.GANN(num_solutions=num_solutions, 108 | num_neurons_input=num_inputs, 109 | num_neurons_hidden_layers=[2], 110 | num_neurons_output=num_classes, 111 | hidden_activations=["relu"], 112 | output_activation="softmax") 113 | 114 | class SocketThread(threading.Thread): 115 | 116 | def __init__(self, connection, client_info, kivy_app, buffer_size=1024, recv_timeout=5): 117 | threading.Thread.__init__(self) 118 | self.connection = connection 119 | self.client_info = client_info 120 | self.buffer_size = buffer_size 121 | self.recv_timeout = recv_timeout 122 | self.kivy_app = kivy_app 123 | 124 | def recv(self): 125 | all_data_received_flag = False 126 | received_data = b"" 127 | while True: 128 | try: 129 | data = self.connection.recv(self.buffer_size) 130 | received_data += data 131 | 132 | try: 133 | pickle.loads(received_data) 134 | # If the previous pickle.loads() statement is passed, this means all the data is received. 135 | # Thus, no need to continue the loop. The flag all_data_received_flag is set to True to signal all data is received. 136 | all_data_received_flag = True 137 | except BaseException: 138 | # An exception is expected when the data is not 100% received. 139 | pass 140 | 141 | if data == b'': # Nothing received from the client. 142 | received_data = b"" 143 | # If still nothing received for a number of seconds specified by the recv_timeout attribute, return with status 0 to close the connection. 144 | if (time.time() - self.recv_start_time) > self.recv_timeout: 145 | return None, 0 # 0 means the connection is no longer active and it should be closed. 146 | 147 | elif all_data_received_flag: 148 | print("All data ({data_len} bytes) Received from {client_info}.".format(client_info=self.client_info, data_len=len(received_data))) 149 | self.kivy_app.label.text = "All data ({data_len} bytes) Received from {client_info}.".format(client_info=self.client_info, data_len=len(received_data)) 150 | 151 | if len(received_data) > 0: 152 | try: 153 | # Decoding the data (bytes). 154 | received_data = pickle.loads(received_data) 155 | # Returning the decoded data. 156 | return received_data, 1 157 | 158 | except BaseException as e: 159 | print("Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 160 | self.kivy_app.label.text = "Error Decoding the Client's Data" 161 | return None, 0 162 | else: 163 | # In case data are received from the client, update the recv_start_time to the current time to reset the timeout counter. 164 | self.recv_start_time = time.time() 165 | 166 | except BaseException as e: 167 | print("Error Receiving Data from the Client: {msg}.\n".format(msg=e)) 168 | self.kivy_app.label.text = "Error Receiving Data from the Client" 169 | return None, 0 170 | 171 | def model_averaging(self, model, other_model): 172 | model_weights = pygad.nn.layers_weights(last_layer=model, initial=False) 173 | other_model_weights = pygad.nn.layers_weights(last_layer=other_model, initial=False) 174 | 175 | new_weights = numpy.array(model_weights + other_model_weights)/2 176 | 177 | pygad.nn.update_layers_trained_weights(last_layer=model, final_weights=new_weights) 178 | 179 | def reply(self, received_data): 180 | global GANN_instance, data_inputs, data_outputs, model 181 | if (type(received_data) is dict): 182 | if (("data" in received_data.keys()) and ("subject" in received_data.keys())): 183 | subject = received_data["subject"] 184 | print("Client's Message Subject is {subject}.".format(subject=subject)) 185 | self.kivy_app.label.text = "Client's Message Subject is {subject}".format(subject=subject) 186 | 187 | print("Replying to the Client.") 188 | self.kivy_app.label.text = "Replying to the Client" 189 | if subject == "echo": 190 | if model is None: 191 | data = {"subject": "model", "data": GANN_instance} 192 | else: 193 | predictions = pygad.nn.predict(last_layer=model, data_inputs=data_inputs) 194 | error = numpy.sum(numpy.abs(predictions - data_outputs)) 195 | # In case a client sent a model to the server despite that the model error is 0.0. In this case, no need to make changes in the model. 196 | if error == 0: 197 | data = {"subject": "done", "data": None} 198 | else: 199 | data = {"subject": "model", "data": GANN_instance} 200 | try: 201 | response = pickle.dumps(data) 202 | except BaseException as e: 203 | print("Error Encoding the Message: {msg}.\n".format(msg=e)) 204 | self.kivy_app.label.text = "Error Encoding the Message" 205 | elif subject == "model": 206 | try: 207 | GANN_instance = received_data["data"] 208 | best_model_idx = received_data["best_solution_idx"] 209 | 210 | best_model = GANN_instance.population_networks[best_model_idx] 211 | if model is None: 212 | model = best_model 213 | else: 214 | predictions = pygad.nn.predict(last_layer=model, data_inputs=data_inputs) 215 | 216 | error = numpy.sum(numpy.abs(predictions - data_outputs)) 217 | 218 | # In case a client sent a model to the server despite that the model error is 0.0. In this case, no need to make changes in the model. 219 | if error == 0: 220 | data = {"subject": "done", "data": None} 221 | response = pickle.dumps(data) 222 | return 223 | 224 | self.model_averaging(model, best_model) 225 | 226 | # print(best_model.trained_weights) 227 | # print(model.trained_weights) 228 | 229 | predictions = pygad.nn.predict(last_layer=model, data_inputs=data_inputs) 230 | print("Model Predictions: {predictions}".format(predictions=predictions)) 231 | 232 | error = numpy.sum(numpy.abs(predictions - data_outputs)) 233 | print("Prediction Error = {error}".format(error=error)) 234 | self.kivy_app.label.text = "Prediction Error = {error}".format(error=error) 235 | 236 | if error != 0: 237 | data = {"subject": "model", "data": GANN_instance} 238 | response = pickle.dumps(data) 239 | else: 240 | data = {"subject": "done", "data": None} 241 | response = pickle.dumps(data) 242 | 243 | except BaseException as e: 244 | print("Error Decoding the Client's Data: {msg}.\n".format(msg=e)) 245 | self.kivy_app.label.text = "Error Decoding the Client's Data" 246 | else: 247 | response = pickle.dumps("Response from the Server") 248 | 249 | try: 250 | self.connection.sendall(response) 251 | except BaseException as e: 252 | print("Error Sending Data to the Client: {msg}.\n".format(msg=e)) 253 | self.kivy_app.label.text = "Error Sending Data to the Client: {msg}".format(msg=e) 254 | 255 | else: 256 | print("The received dictionary from the client must have the 'subject' and 'data' keys available. The existing keys are {d_keys}.".format(d_keys=received_data.keys())) 257 | self.kivy_app.label.text = "Error Parsing Received Dictionary" 258 | else: 259 | print("A dictionary is expected to be received from the client but {d_type} received.".format(d_type=type(received_data))) 260 | self.kivy_app.label.text = "A dictionary is expected but {d_type} received.".format(d_type=type(received_data)) 261 | 262 | def run(self): 263 | print("Running a Thread for the Connection with {client_info}.".format(client_info=self.client_info)) 264 | self.kivy_app.label.text = "Running a Thread for the Connection with {client_info}.".format(client_info=self.client_info) 265 | 266 | # This while loop allows the server to wait for the client to send data more than once within the same connection. 267 | while True: 268 | self.recv_start_time = time.time() 269 | time_struct = time.gmtime() 270 | date_time = "Waiting to Receive Data Starting from {day}/{month}/{year} {hour}:{minute}:{second} GMT".format(year=time_struct.tm_year, month=time_struct.tm_mon, day=time_struct.tm_mday, hour=time_struct.tm_hour, minute=time_struct.tm_min, second=time_struct.tm_sec) 271 | print(date_time) 272 | received_data, status = self.recv() 273 | if status == 0: 274 | self.connection.close() 275 | self.kivy_app.label.text = "Connection Closed with {client_info}".format(client_info=self.client_info) 276 | print("Connection Closed with {client_info} either due to inactivity for {recv_timeout} seconds or due to an error.".format(client_info=self.client_info, recv_timeout=self.recv_timeout), end="\n\n") 277 | break 278 | 279 | # print(received_data) 280 | self.reply(received_data) 281 | 282 | class ListenThread(threading.Thread): 283 | 284 | def __init__(self, kivy_app): 285 | threading.Thread.__init__(self) 286 | self.kivy_app = kivy_app 287 | 288 | def run(self): 289 | while True: 290 | try: 291 | connection, client_info = self.kivy_app.soc.accept() 292 | self.kivy_app.label.text = "New Connection from {client_info}".format(client_info=client_info) 293 | socket_thread = SocketThread(connection=connection, 294 | client_info=client_info, 295 | kivy_app=self.kivy_app, 296 | buffer_size=1024, 297 | recv_timeout=10) 298 | socket_thread.start() 299 | except BaseException as e: 300 | self.kivy_app.soc.close() 301 | print("Error in the run() of the ListenThread class: {msg}.\n".format(msg=e)) 302 | self.kivy_app.label.text = "Socket is No Longer Accepting Connections" 303 | self.kivy_app.create_socket_btn.disabled = False 304 | self.kivy_app.close_socket_btn.disabled = True 305 | break 306 | 307 | serverApp = ServerApp() 308 | serverApp.title="Server App" 309 | serverApp.run() --------------------------------------------------------------------------------