├── .gitignore ├── LICENSE ├── README.md ├── microlearn ├── __init__.py ├── ml_models │ ├── GaussianNB.py │ ├── KMeans.py │ ├── LinearDiscriminantAnalysis.py │ ├── LinearRegression.py │ ├── LogisticRegression.py │ ├── PCA.py │ ├── PassiveAggressiveClassifier.py │ ├── Perceptron.py │ ├── QuadraticDiscriminantAnalysis.py │ ├── SVC.py │ └── __init__.py └── offloader.py └── setup.py /.gitignore: -------------------------------------------------------------------------------- 1 | /build/ 2 | /dist/ 3 | /micro_learn.egg-info/ 4 | __pycache__* -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Adarsh Pal Singh 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![PyPI version](https://badge.fury.io/py/micro-learn.svg)](https://badge.fury.io/py/micro-learn) 2 | 3 | **Micro-learn** is a Python library for converting machine learning models trained using scikit-learn into inference code that can run on virtually any microcontroller in real time. 4 | 5 | Machine learning algorithms typically require heavy computing and memory resources in the training phase, far greater than what a typical constrained microcontroller can offer. However, post training, many of these algorithms boil down to simple parameters that require simple arithmetic and logical operations for inference. These can easily run on microcontrollers in real time. In a nutshell, train your ML models on a resource-abundant machine using scikit-learn and, with a single line of code, export that trained model into C++/Arduino code using micro-learn! 6 | 7 | All the inference algorithms are optimized for microcontrollers and require least possible arithmetic computations. For example, division operations have been converted to multiplications since the latter is much more computationally efficient in microcontrollers. Note that all the algorithms are exact and not approximate. Please refer to my [ACM paper](https://dl.acm.org/doi/abs/10.1145/3341105.3373967) for more insights. 8 | 9 | ## Installation 10 | 11 | ### Dependencies 12 | 13 | - Python (>= 3.6) 14 | - NumPy (>= 1.13.3) 15 | 16 | ### User Installation 17 | Use Python3 pip to install the latest micro-learn package. 18 | 19 | ```bash 20 | pip install micro-learn 21 | ``` 22 | 23 | ## Supported Algorithms 24 | The following scikit-learn models are supported as of now: 25 | 26 | #### Supervised Learning (Classification) 27 | - Perceptron 28 | - Logistic Regression (Logit) 29 | - Gaussian Naive Bayes (GNB) 30 | - Passive-Aggressive Classifier (PA) 31 | - Linear Discriminant Analysis (LDA) 32 | - Quadratic Discriminant Analysis (QDA) 33 | - Support Vector Machine (SVM) (Linear Kernel) 34 | 35 | *Note that only binary-class classification is supported as of now. Support for multi-class classification coming soon!* 36 | 37 | #### Supervised Learning (Regression) 38 | - Linear Regression (LR) 39 | 40 | #### Unsupervised Learning (Clustering) 41 | - KMeans 42 | 43 | #### Unsupervised Learning (Dimensionality Reduction) 44 | - Principal Component Analysis (PCA) 45 | 46 | *Support for other scikit-learn models coming soon!* 47 | 48 | ## Usage 49 | Train any of the supported machine learning models using scikit-learn and simply pass this trained model to micro-learn's *Offload()*. Example for *Gaussian Naive Bayes* is shown below. All other supported algorithms follow the exact same sequence. For SVM and PCA (scale-variant algorithms), an optional *StandardScaler* object can be passed as a second argument to *Offload()*. 50 | 51 | ```python 52 | >>> from sklearn.naive_bayes import GaussianNB 53 | >>> gnb = GaussianNB() 54 | >>> gnb.fit(X, Y) 55 | >>> from microlearn.offloader import Offload 56 | >>> off = Offload(gnb) #Simply pass your trained model! 57 | >>> off.export_to_arduino('/home/adarsh1001/gnb.ino') 58 | ``` 59 | 60 | And that's it! The output Arduino template will have the corresponding ML inference code along with all the trained parameters. After exporting, open the .ino file and edit the data section and the output class label section as per your need. And of course, since the Arduino programming language is a derivative of C/C++, you can directly edit the template and convert it into a generic .c or .cpp code. For a more detailed guide, you can check out my [medium post](https://medium.com/analytics-vidhya/micro-learn-getting-started-with-machine-learning-on-arduino-52167bc34c1d) on this. 61 | 62 | ## Project History 63 | I started working on embedded machine learning, both from a theoretical as well as practical perspective, in August 2019 as part of my MS. In November 2019, my paper on a related topic was accepted in ACM SAC. The work on coding a unified library for offloading trained ML models to microcontroller code (micro-learn) was started in May 2020. 64 | 65 | ### Citation 66 | If you use micro-learn in a scientific publication, please do cite my [paper](https://dl.acm.org/doi/abs/10.1145/3341105.3373967). 67 | -------------------------------------------------------------------------------- /microlearn/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adarsh1001/micro-learn/60cf9da31ff1ee489b28d2cf88236a0622fb5c42/microlearn/__init__.py -------------------------------------------------------------------------------- /microlearn/ml_models/GaussianNB.py: -------------------------------------------------------------------------------- 1 | import math 2 | import numpy as np 3 | 4 | class OffloadGNB: 5 | def __init__(self, model): 6 | self.class0 = model.__dict__['classes_'][0] 7 | self.class1 = model.__dict__['classes_'][1] 8 | self.prior0 = model.class_prior_[0] 9 | self.prior1 = model.class_prior_[1] 10 | self.variance_vector0 = model.sigma_[0] 11 | self.variance_vector1 = model.sigma_[1] 12 | self.dim = len(self.variance_vector0) 13 | 14 | self.p0 = self.get_inv_variance(0) 15 | self.p1 = self.get_inv_variance(1) 16 | self.c = self.get_constant() 17 | self.u0 = model.theta_[0] 18 | self.u1 = model.theta_[1] 19 | 20 | def get_inv_variance(self, label): 21 | if label == 0: 22 | return np.array([1/x for x in self.variance_vector0]) 23 | else: 24 | return np.array([1/x for x in self.variance_vector1]) 25 | 26 | def get_constant(self): 27 | numerator = 1.0 28 | denominator = 1.0 29 | for term_num,term_den in zip(self.variance_vector0, self.variance_vector1): 30 | numerator *= (1/(math.sqrt(term_num))) 31 | denominator *= (1/(math.sqrt(term_den))) 32 | numerator *= self.prior0 33 | denominator *= self.prior1 34 | 35 | return 2*math.log(numerator/denominator) 36 | 37 | def get_params(self): 38 | return {'Inverse_Variances_0':self.p0, 'Inverse_Variances_1':self.p1, 'Means_0':self.u0, 'Means_1':self.u1, 'Constant':self.c} 39 | 40 | def get_arduino_code(self): 41 | str_u0 = ', '.join([str(x) for x in self.u0]) 42 | str_u1 = ', '.join([str(x) for x in self.u1]) 43 | str_p0 = ', '.join([str(x) for x in self.p0]) 44 | str_p1 = ', '.join([str(x) for x in self.p1]) 45 | 46 | code = f"""double u0[] = {{{str_u0}}}; 47 | double u1[] = {{{str_u1}}}; 48 | 49 | double p0[] = {{{str_p0}}}; 50 | double p1[] = {{{str_p1}}}; 51 | 52 | double c = {str(self.c)}; 53 | 54 | void setup() {{ 55 | Serial.begin(9600); 56 | 57 | }} 58 | 59 | void loop() {{ 60 | //Data Section: To Be Coded Manually 61 | 62 | float data[{str(self.dim)}]; //This is your feature vector. Retrive your data into this array. 63 | 64 | //ML Inference Section 65 | 66 | double term1 = 0.0; 67 | double term2 = 0.0; 68 | 69 | for(int i=0; i<{str(self.dim)}; i++) 70 | {{ 71 | term1 += p0[i] * (data[i] - u0[i]) * (data[i] - u0[i]); 72 | term2 += p1[i] * (data[i] - u1[i]) * (data[i] - u1[i]); 73 | }} 74 | 75 | double temp = term1 - term2; 76 | 77 | if(temp >= c) 78 | {{ 79 | //Do something for class label {str(self.class1)}. 80 | Serial.println("{str(self.class1)}"); 81 | }} 82 | else 83 | {{ 84 | //Do something for class label {str(self.class0)}. 85 | Serial.println("{str(self.class0)}"); 86 | }} 87 | 88 | delay(1000); 89 | }}""" 90 | return code 91 | -------------------------------------------------------------------------------- /microlearn/ml_models/KMeans.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | class OffloadKMeans: 4 | def __init__(self, model): 5 | self.num_clusters = model.n_clusters 6 | self.cluster_centers = model.cluster_centers_ 7 | self.dim = len(self.cluster_centers[0]) 8 | 9 | def get_params(self): 10 | return {'Num_Clusters':self.num_clusters, 'Cluster_Centers':self.cluster_centers} 11 | 12 | def get_arduino_code(self): 13 | cc_terms = [] 14 | for term in self.cluster_centers: 15 | temp = '{' + ', '.join([str(x) for x in term]) + '}' 16 | cc_terms.append(temp) 17 | str_cc = ', '.join(cc_terms) 18 | 19 | code = f"""double CC[{str(self.num_clusters)}][{str(self.dim)}] = {{{str_cc}}}; 20 | 21 | double euclidean_sq(float A[], double B[]) 22 | {{ 23 | double dist = 0.0; 24 | for(int i=0; i<{str(self.dim)}; i++) 25 | {{ 26 | double temp = A[i]-B[i]; 27 | dist += temp*temp; 28 | }} 29 | return dist; 30 | }} 31 | 32 | void setup() {{ 33 | Serial.begin(9600); 34 | 35 | }} 36 | 37 | void loop() {{ 38 | //Data Section: To Be Coded Manually 39 | 40 | float data[{str(self.dim)}]; //This is your feature vector. Retrive your data into this array. 41 | 42 | //ML Inference Section 43 | 44 | double min_dist = euclidean_sq(data, CC[0]); 45 | int label = 0; 46 | for(int i=1; i<{str(self.num_clusters)}; i++) 47 | {{ 48 | double temp = euclidean_sq(data, CC[i]); 49 | if(temp < min_dist) 50 | {{ 51 | min_dist = temp; 52 | label = i; 53 | }} 54 | }} 55 | 56 | //Do something with the cluster label prediction. 57 | Serial.println(label); 58 | 59 | delay(1000); 60 | }}""" 61 | return code 62 | -------------------------------------------------------------------------------- /microlearn/ml_models/LinearDiscriminantAnalysis.py: -------------------------------------------------------------------------------- 1 | import math 2 | import numpy as np 3 | 4 | class OffloadLDA: 5 | def __init__(self, model): 6 | self.class0 = model.__dict__['classes_'][0] 7 | self.class1 = model.__dict__['classes_'][1] 8 | try: 9 | self.cov_matrix = model.covariance_ 10 | except AttributeError: 11 | print("LDA must be trained with 'store_covariance=True' parameter.") 12 | raise 13 | self.inv_cov_matrix = np.linalg.inv(self.cov_matrix) 14 | self.mean_vector0 = model.means_[0] 15 | self.mean_vector1 = model.means_[1] 16 | self.prior0 = model.priors_[0] 17 | self.prior1 = model.priors_[1] 18 | self.dim = len(self.mean_vector0) 19 | 20 | self.w = self.get_weights() 21 | self.c = self.get_intercept() 22 | 23 | def get_weights(self): 24 | return np.matmul(self.inv_cov_matrix, (self.mean_vector1 - self.mean_vector0)) 25 | 26 | def get_intercept(self): 27 | term1 = math.log(self.prior0/self.prior1) 28 | term2 = 0.5*(np.matmul(np.matmul(self.mean_vector0, self.inv_cov_matrix), self.mean_vector0) - np.matmul(np.matmul(self.mean_vector1, self.inv_cov_matrix), self.mean_vector1)) 29 | return term1 - term2 30 | 31 | def get_params(self): 32 | return {'Weight_Vector':self.w, 'Intercept_Constant':self.c} 33 | 34 | def get_arduino_code(self): 35 | str_w = ', '.join([str(x) for x in self.w]) 36 | 37 | code = f"""double w[] = {{{str_w}}}; 38 | double c = {str(self.c)}; 39 | 40 | void setup() {{ 41 | Serial.begin(9600); 42 | 43 | }} 44 | 45 | void loop() {{ 46 | //Data Section: To Be Coded Manually 47 | 48 | float data[{str(self.dim)}]; //This is your feature vector. Retrive your data into this array. 49 | 50 | //ML Inference Section 51 | 52 | double temp = 0.0; 53 | for(int i=0; i<{str(self.dim)}; i++) 54 | {{ 55 | temp += data[i]*w[i]; 56 | }} 57 | 58 | if(temp >= c) 59 | {{ 60 | //Do something for class label {str(self.class1)}. 61 | Serial.println("{str(self.class1)}"); 62 | }} 63 | else 64 | {{ 65 | //Do something for class label {str(self.class0)}. 66 | Serial.println("{str(self.class0)}"); 67 | }} 68 | 69 | delay(1000); 70 | }}""" 71 | return code 72 | -------------------------------------------------------------------------------- /microlearn/ml_models/LinearRegression.py: -------------------------------------------------------------------------------- 1 | import math 2 | import numpy as np 3 | 4 | class OffloadLR: 5 | def __init__(self, model): 6 | self.w = model.coef_ 7 | self.c = model.intercept_ 8 | self.dim = len(self.w) 9 | 10 | def get_params(self): 11 | return {'Weight_Vector':self.w, 'Intercept_Constant':self.c} 12 | 13 | def get_arduino_code(self): 14 | str_w = ', '.join([str(x) for x in self.w]) 15 | 16 | code = f"""double w[] = {{{str_w}}}; 17 | double c = {str(self.c)}; 18 | 19 | void setup() {{ 20 | Serial.begin(9600); 21 | 22 | }} 23 | 24 | void loop() {{ 25 | //Data Section: To Be Coded Manually 26 | 27 | float data[{str(self.dim)}]; //This is your feature vector. Retrive your data into this array. 28 | 29 | //ML Inference Section 30 | 31 | double prediction = 0.0; 32 | for(int i=0; i<{str(self.dim)}; i++) 33 | {{ 34 | prediction += data[i]*w[i]; 35 | }} 36 | prediction += c; 37 | 38 | //Do something with the prediction. 39 | Serial.println(prediction); 40 | 41 | delay(1000); 42 | }}""" 43 | return code 44 | -------------------------------------------------------------------------------- /microlearn/ml_models/LogisticRegression.py: -------------------------------------------------------------------------------- 1 | import math 2 | import numpy as np 3 | 4 | class OffloadLogit: 5 | def __init__(self, model): 6 | self.class0 = model.__dict__['classes_'][0] 7 | self.class1 = model.__dict__['classes_'][1] 8 | self.w = model.coef_[0] 9 | self.b = model.intercept_[0] 10 | self.dim = len(self.w) 11 | 12 | def get_params(self): 13 | return {'Weight_Vector':self.w, 'Bias_Constant':self.b} 14 | 15 | def get_arduino_code(self): 16 | str_w = ', '.join([str(x) for x in self.w]) 17 | 18 | code = f"""#include 19 | 20 | double w[] = {{{str_w}}}; 21 | double b = {str(self.b)}; 22 | 23 | double sigmoid(double x) 24 | {{ 25 | return (1 / (1 + exp((-1) * x))); 26 | }} 27 | 28 | void setup() {{ 29 | Serial.begin(9600); 30 | 31 | }} 32 | 33 | void loop() {{ 34 | //Data Section: To Be Coded Manually 35 | 36 | float data[{str(self.dim)}]; //This is your feature vector. Retrive your data into this array. 37 | 38 | //ML Inference Section 39 | 40 | double temp = 0.0; 41 | for(int i=0; i<{str(self.dim)}; i++) 42 | {{ 43 | temp += data[i]*w[i]; 44 | }} 45 | 46 | if(round(sigmoid(temp + b)) == 1) 47 | {{ 48 | //Do something for class label {str(self.class1)}. 49 | Serial.println("{str(self.class1)}"); 50 | }} 51 | else 52 | {{ 53 | //Do something for class label {str(self.class0)}. 54 | Serial.println("{str(self.class0)}"); 55 | }} 56 | delay(1000); 57 | }}""" 58 | return code 59 | -------------------------------------------------------------------------------- /microlearn/ml_models/PCA.py: -------------------------------------------------------------------------------- 1 | import math 2 | import numpy as np 3 | 4 | class OffloadPCA: 5 | def __init__(self, model, scaler): 6 | self.scaler = scaler 7 | self.n_components = model.n_components_ 8 | self.mean_vector = model.mean_ 9 | self.pca_components = model.components_.T 10 | self.dim = len(self.mean_vector) 11 | if scaler: 12 | self.u = scaler.mean_ 13 | self.p = np.reciprocal(scaler.scale_) 14 | 15 | def get_params(self): 16 | return {'Number_PCA_Components':self.n_components, 'Mean_Vector':self.mean_vector, 'PCA_Components':self.pca_components} 17 | 18 | def get_pca_params_string(self): 19 | pca_components_terms = [] 20 | for term in self.pca_components: 21 | temp = '{' + ', '.join([str(x) for x in term]) + '}' 22 | pca_components_terms.append(temp) 23 | str_pca_components = ', '.join(pca_components_terms) 24 | str_mean_vector = ', '.join([str(x) for x in self.mean_vector]) 25 | return str_pca_components, str_mean_vector 26 | 27 | def get_scaling_params_string(self): 28 | str_u = ', '.join([str(x) for x in self.u]) 29 | str_p = ', '.join([str(x) for x in self.p]) 30 | return str_u, str_p 31 | 32 | def unscaled_pca_arduino_code(self): 33 | str_pca_components, str_mean_vector = self.get_pca_params_string() 34 | 35 | code = f"""double pca_components[{str(self.dim)}][{str(self.n_components)}] = {{{str_pca_components}}}; 36 | double mean_vector[] = {{{str_mean_vector}}}; 37 | 38 | void setup() {{ 39 | Serial.begin(9600); 40 | 41 | }} 42 | 43 | void loop() {{ 44 | //Data Section: To Be Coded Manually 45 | 46 | float data[{str(self.dim)}]; //This is your feature vector. Retrive your data into this array. 47 | 48 | //ML Inference Section 49 | 50 | for(int i=0; i<{str(self.dim)}; i++) 51 | {{ 52 | data[i] = data[i] - mean_vector[i]; //Center the feature vector. 53 | }} 54 | 55 | double data_pca_transformed[{str(self.n_components)}] = {{ 0.0 }}; 56 | for(int col=0; col<{str(self.n_components)}; col++) 57 | {{ 58 | double temp = 0.0; 59 | for(int i=0; i<{str(self.dim)}; i++) 60 | {{ 61 | data_pca_transformed[col] += data[i] * pca_components[i][col]; 62 | }} 63 | }} 64 | 65 | //Do something with the PCA transformed feature vector: data_pca_transformed. 66 | Serial.println("PCA Transformation Complete"); 67 | 68 | delay(1000); 69 | }}""" 70 | return code 71 | 72 | 73 | def scaled_pca_arduino_code(self): 74 | str_pca_components, str_mean_vector = self.get_pca_params_string() 75 | str_u, str_p = self.get_scaling_params_string() 76 | 77 | code = f"""double pca_components[{str(self.dim)}][{str(self.n_components)}] = {{{str_pca_components}}}; 78 | double mean_vector[] = {{{str_mean_vector}}}; 79 | double u[] = {{{str_u}}}; 80 | double p[] = {{{str_p}}}; 81 | 82 | void setup() {{ 83 | Serial.begin(9600); 84 | 85 | }} 86 | 87 | void loop() {{ 88 | //Data Section: To Be Coded Manually 89 | 90 | float data[{str(self.dim)}]; //This is your feature vector. Retrive your data into this array. 91 | 92 | //ML Inference Section 93 | 94 | for(int i=0; i<{str(self.dim)}; i++) 95 | {{ 96 | data[i] = (((data[i] - u[i]) * p[i]) - mean_vector[i]); //Standard Scaling and Centering. 97 | }} 98 | 99 | double data_pca_transformed[{str(self.n_components)}] = {{ 0.0 }}; 100 | for(int col=0; col<{str(self.n_components)}; col++) 101 | {{ 102 | double temp = 0.0; 103 | for(int i=0; i<{str(self.dim)}; i++) 104 | {{ 105 | data_pca_transformed[col] += data[i] * pca_components[i][col]; 106 | }} 107 | }} 108 | 109 | //Do something with the PCA transformed feature vector: data_pca_transformed. 110 | Serial.println("PCA Transformation Complete"); 111 | 112 | delay(1000); 113 | }}""" 114 | return code 115 | 116 | 117 | def get_arduino_code(self): 118 | if self.scaler: 119 | return self.scaled_pca_arduino_code() 120 | return self.unscaled_pca_arduino_code() 121 | -------------------------------------------------------------------------------- /microlearn/ml_models/PassiveAggressiveClassifier.py: -------------------------------------------------------------------------------- 1 | import math 2 | import numpy as np 3 | 4 | class OffloadPA: 5 | def __init__(self, model): 6 | self.class0 = model.__dict__['classes_'][0] 7 | self.class1 = model.__dict__['classes_'][1] 8 | self.w = model.coef_[0] 9 | self.b = model.intercept_[0] 10 | self.dim = len(self.w) 11 | 12 | def get_params(self): 13 | return {'Weight_Vector':self.w, 'Bias_Constant':self.b} 14 | 15 | def get_arduino_code(self): 16 | str_w = ', '.join([str(x) for x in self.w]) 17 | 18 | code = f"""double w[] = {{{str_w}}}; 19 | double b = {str(self.b)}; 20 | 21 | void setup() {{ 22 | Serial.begin(9600); 23 | 24 | }} 25 | 26 | void loop() {{ 27 | //Data Section: To Be Coded Manually 28 | 29 | float data[{str(self.dim)}]; //This is your feature vector. Retrive your data into this array. 30 | 31 | //ML Inference Section 32 | 33 | double temp = 0.0; 34 | for(int i=0; i<{str(self.dim)}; i++) 35 | {{ 36 | temp += data[i]*w[i]; 37 | }} 38 | 39 | if(temp + b >= 0) 40 | {{ 41 | //Do something for class label {str(self.class1)}. 42 | Serial.println("{str(self.class1)}"); 43 | }} 44 | else 45 | {{ 46 | //Do something for class label {str(self.class0)}. 47 | Serial.println("{str(self.class0)}"); 48 | }} 49 | 50 | delay(1000); 51 | }}""" 52 | return code 53 | -------------------------------------------------------------------------------- /microlearn/ml_models/Perceptron.py: -------------------------------------------------------------------------------- 1 | import math 2 | import numpy as np 3 | 4 | class OffloadPerceptron: 5 | def __init__(self, model): 6 | self.class0 = model.__dict__['classes_'][0] 7 | self.class1 = model.__dict__['classes_'][1] 8 | self.w = model.coef_[0] 9 | self.b = model.intercept_[0] 10 | self.dim = len(self.w) 11 | 12 | def get_params(self): 13 | return {'Weight_Vector':self.w, 'Bias_Constant':self.b} 14 | 15 | def get_arduino_code(self): 16 | str_w = ', '.join([str(x) for x in self.w]) 17 | 18 | code = f"""double w[] = {{{str_w}}}; 19 | double b = {str(self.b)}; 20 | 21 | void setup() {{ 22 | Serial.begin(9600); 23 | 24 | }} 25 | 26 | void loop() {{ 27 | //Data Section: To Be Coded Manually 28 | 29 | float data[{str(self.dim)}]; //This is your feature vector. Retrive your data into this array. 30 | 31 | //ML Inference Section 32 | 33 | double temp = 0.0; 34 | for(int i=0; i<{str(self.dim)}; i++) 35 | {{ 36 | temp += data[i]*w[i]; 37 | }} 38 | 39 | if(temp + b >= 0) 40 | {{ 41 | //Do something for class label {str(self.class1)}. 42 | Serial.println("{str(self.class1)}"); 43 | }} 44 | else 45 | {{ 46 | //Do something for class label {str(self.class0)}. 47 | Serial.println("{str(self.class0)}"); 48 | }} 49 | 50 | delay(1000); 51 | }}""" 52 | return code 53 | -------------------------------------------------------------------------------- /microlearn/ml_models/QuadraticDiscriminantAnalysis.py: -------------------------------------------------------------------------------- 1 | import math 2 | import numpy as np 3 | 4 | class OffloadQDA: 5 | def __init__(self, model): 6 | self.class0 = model.__dict__['classes_'][0] 7 | self.class1 = model.__dict__['classes_'][1] 8 | try: 9 | self.cov_matrix0 = model.covariance_[0] 10 | self.cov_matrix1 = model.covariance_[1] 11 | except AttributeError: 12 | print("QDA must be trained with 'store_covariances=True' parameter.") 13 | raise 14 | self.inv_cov_matrix0 = np.linalg.inv(self.cov_matrix0) 15 | self.inv_cov_matrix1 = np.linalg.inv(self.cov_matrix1) 16 | self.mean_vector0 = model.means_[0] 17 | self.mean_vector1 = model.means_[1] 18 | self.prior0 = model.priors_[0] 19 | self.prior1 = model.priors_[1] 20 | self.dim = len(self.mean_vector0) 21 | 22 | self.w = self.get_weights() 23 | self.c = self.get_constant() 24 | self.invEdiff = self.get_cov_matrix_diff() 25 | 26 | def get_weights(self): 27 | term1 = 2*np.matmul(self.inv_cov_matrix1, self.mean_vector1) 28 | term2 = 2*np.matmul(self.inv_cov_matrix0, self.mean_vector0) 29 | return term1 - term2 30 | 31 | def get_cov_matrix_diff(self): 32 | return self.inv_cov_matrix0 - self.inv_cov_matrix1 33 | 34 | def get_constant(self): 35 | term1_num = self.prior0*math.sqrt(np.linalg.det(self.cov_matrix1)) 36 | term1_den = self.prior1*math.sqrt(np.linalg.det(self.cov_matrix0)) 37 | term1 = 2*math.log(term1_num/term1_den) 38 | 39 | term2 = np.matmul(np.matmul(self.mean_vector1, self.inv_cov_matrix1), self.mean_vector1) 40 | 41 | term3 = np.matmul(np.matmul(self.mean_vector0, self.inv_cov_matrix0), self.mean_vector0) 42 | 43 | return term1 + term2 - term3 44 | 45 | def get_params(self): 46 | return {'Inverse_CovMatrix_Diff':self.invEdiff, 'Weight_Vector':self.w, 'Constant':self.c} 47 | 48 | def get_arduino_code(self): 49 | invEdiff_terms = [] 50 | for term in self.invEdiff: 51 | temp = '{' + ', '.join([str(x) for x in term]) + '}' 52 | invEdiff_terms.append(temp) 53 | str_invEdiff = ', '.join(invEdiff_terms) 54 | 55 | str_w = ', '.join([str(x) for x in self.w]) 56 | 57 | code = f"""double invEdiff[{str(self.dim)}][{str(self.dim)}] = {{{str_invEdiff}}}; 58 | double w[] = {{{str_w}}}; 59 | double c = {str(self.c)}; 60 | 61 | void setup() {{ 62 | Serial.begin(9600); 63 | 64 | }} 65 | 66 | void loop() {{ 67 | //Data Section: To Be Coded Manually 68 | 69 | float data[{str(self.dim)}]; //This is your feature vector. Retrive your data into this array. 70 | 71 | //ML Inference Section 72 | 73 | double term = 0.0; 74 | for(int col=0; col<{str(self.dim)}; col++) 75 | {{ 76 | double temp = 0.0; 77 | for(int i=0; i<{str(self.dim)}; i++) 78 | {{ 79 | temp += data[i] * invEdiff[i][col]; 80 | }} 81 | term += (temp + w[col]) * data[col]; 82 | }} 83 | 84 | if(term >= c) 85 | {{ 86 | //Do something for class label {str(self.class1)}. 87 | Serial.println("{str(self.class1)}"); 88 | }} 89 | else 90 | {{ 91 | //Do something for class label {str(self.class0)}. 92 | Serial.println("{str(self.class0)}"); 93 | }} 94 | 95 | delay(1000); 96 | }}""" 97 | return code 98 | -------------------------------------------------------------------------------- /microlearn/ml_models/SVC.py: -------------------------------------------------------------------------------- 1 | import math 2 | import numpy as np 3 | 4 | class OffloadSVM: 5 | def __init__(self, model, scaler): 6 | self.scaler = scaler 7 | self.class0 = model.__dict__['classes_'][0] 8 | self.class1 = model.__dict__['classes_'][1] 9 | if model.__repr__().split('(')[0] == 'SVC' and model.__dict__['kernel'] != 'linear': 10 | raise TypeError("Only linear SVM is supported! Pass kernel = 'linear' to SVC or use LinearSVC.") 11 | self.w = model.coef_[0] 12 | self.c = -1*model.intercept_[0] 13 | self.dim = len(self.w) 14 | if scaler: 15 | self.u = scaler.mean_ 16 | self.p = np.reciprocal(scaler.scale_) 17 | 18 | def get_params(self): 19 | return {'Weight_Vector':self.w, 'Negative_Intercept_Constant':self.c} 20 | 21 | def get_svm_params_string(self): 22 | str_w = ', '.join([str(x) for x in self.w]) 23 | return str_w 24 | 25 | def get_scaling_params_string(self): 26 | str_u = ', '.join([str(x) for x in self.u]) 27 | str_p = ', '.join([str(x) for x in self.p]) 28 | return str_u, str_p 29 | 30 | def unscaled_svm_arduino_code(self): 31 | str_w = self.get_svm_params_string() 32 | 33 | code = f"""double w[] = {{{str_w}}}; 34 | 35 | double c = {str(self.c)}; 36 | 37 | void setup() {{ 38 | Serial.begin(9600); 39 | 40 | }} 41 | 42 | void loop() {{ 43 | //Data Section: To Be Coded Manually 44 | 45 | float data[{str(self.dim)}]; //This is your feature vector. Retrive your data into this array. 46 | 47 | //ML Inference Section 48 | 49 | double temp = 0.0; 50 | for(int i=0; i<{str(self.dim)}; i++) 51 | {{ 52 | temp += (data[i] * w[i]); 53 | }} 54 | 55 | if(temp >= c) 56 | {{ 57 | //Do something for class label {str(self.class1)}. 58 | Serial.println("{str(self.class1)}"); 59 | }} 60 | else 61 | {{ 62 | //Do something for class label {str(self.class0)}. 63 | Serial.println("{str(self.class0)}"); 64 | }} 65 | 66 | delay(1000); 67 | }}""" 68 | return code 69 | 70 | 71 | def scaled_svm_arduino_code(self): 72 | str_w = self.get_svm_params_string() 73 | str_u, str_p = self.get_scaling_params_string() 74 | 75 | code = f"""double w[] = {{{str_w}}}; 76 | double u[] = {{{str_u}}}; 77 | double p[] = {{{str_p}}}; 78 | 79 | double c = {str(self.c)}; 80 | 81 | void setup() {{ 82 | Serial.begin(9600); 83 | 84 | }} 85 | 86 | void loop() {{ 87 | //Data Section: To Be Coded Manually 88 | 89 | float data[{str(self.dim)}]; //This is your feature vector. Retrive your data into this array. 90 | 91 | //ML Inference Section 92 | 93 | double temp = 0.0; 94 | for(int i=0; i<{str(self.dim)}; i++) 95 | {{ 96 | temp += (data[i]-u[i]) * p[i] * w[i]; 97 | }} 98 | 99 | if(temp >= c) 100 | {{ 101 | //Do something for class label {str(self.class1)}. 102 | Serial.println("{str(self.class1)}"); 103 | }} 104 | else 105 | {{ 106 | //Do something for class label {str(self.class0)}. 107 | Serial.println("{str(self.class0)}"); 108 | }} 109 | 110 | delay(1000); 111 | }}""" 112 | return code 113 | 114 | 115 | def get_arduino_code(self): 116 | if self.scaler: 117 | return self.scaled_svm_arduino_code() 118 | return self.unscaled_svm_arduino_code() 119 | -------------------------------------------------------------------------------- /microlearn/ml_models/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adarsh1001/micro-learn/60cf9da31ff1ee489b28d2cf88236a0622fb5c42/microlearn/ml_models/__init__.py -------------------------------------------------------------------------------- /microlearn/offloader.py: -------------------------------------------------------------------------------- 1 | from .ml_models.SVC import OffloadSVM 2 | from .ml_models.PCA import OffloadPCA 3 | from .ml_models.KMeans import OffloadKMeans 4 | from .ml_models.GaussianNB import OffloadGNB 5 | from .ml_models.LinearRegression import OffloadLR 6 | from .ml_models.Perceptron import OffloadPerceptron 7 | from .ml_models.LogisticRegression import OffloadLogit 8 | from .ml_models.LinearDiscriminantAnalysis import OffloadLDA 9 | from .ml_models.PassiveAggressiveClassifier import OffloadPA 10 | from .ml_models.QuadraticDiscriminantAnalysis import OffloadQDA 11 | 12 | class Offload: 13 | supported_algorithms = ["LinearDiscriminantAnalysis", "QuadraticDiscriminantAnalysis", "GaussianNB", "SVC", "LinearSVC", "Perceptron", "LinearRegression", "LogisticRegression", "PassiveAggressiveClassifier", "KMeans", "PCA"] 14 | 15 | def __init__(self, model, optional=None): 16 | self.optional = optional 17 | self.check_model_validity(model) 18 | self.model = model 19 | self.algorithm = self.get_algorithm(model) 20 | self.offloader = self.get_offloader() 21 | 22 | def get_algorithm(self, model): 23 | return model.__repr__().split('(')[0] 24 | 25 | def is_algorithm_supported(self, model): 26 | try: 27 | model.__getstate__() 28 | except AttributeError: 29 | return False 30 | if "_sklearn_version" not in model.__getstate__(): 31 | return False 32 | return self.get_algorithm(model) in self.supported_algorithms 33 | 34 | def is_model_trained(self, model): 35 | if self.get_algorithm(model) == "StandardScaler": 36 | return "n_samples_seen_" in model.__dict__ 37 | elif self.get_algorithm(model) == "LinearRegression": 38 | return "singular_" in model.__dict__ 39 | elif self.get_algorithm(model) == "KMeans": 40 | return "cluster_centers_" in model.__dict__ 41 | elif self.get_algorithm(model) == "PCA": 42 | return "n_components_" in model.__dict__ 43 | else: 44 | return "classes_" in model.__dict__ 45 | 46 | def is_model_binary(self, model): 47 | if self.get_algorithm(model) == "LinearRegression" or self.get_algorithm(model) == "KMeans" or self.get_algorithm(model) == "PCA": 48 | return True 49 | else: 50 | return len(model.__dict__["classes_"]) == 2 51 | 52 | def get_offloader(self): 53 | if self.algorithm == self.supported_algorithms[0]: #LDA 54 | return OffloadLDA(self.model) 55 | elif self.algorithm == self.supported_algorithms[1]: #QDA 56 | return OffloadQDA(self.model) 57 | elif self.algorithm == self.supported_algorithms[2]: #GNB 58 | return OffloadGNB(self.model) 59 | elif self.algorithm == self.supported_algorithms[3] or self.algorithm == self.supported_algorithms[4]: #SVM 60 | return OffloadSVM(self.model, self.optional) 61 | elif self.algorithm == self.supported_algorithms[5]: #Perceptron 62 | return OffloadPerceptron(self.model) 63 | elif self.algorithm == self.supported_algorithms[6]: #LR 64 | return OffloadLR(self.model) 65 | elif self.algorithm == self.supported_algorithms[7]: #Logit 66 | return OffloadLogit(self.model) 67 | elif self.algorithm == self.supported_algorithms[8]: #PA 68 | return OffloadPA(self.model) 69 | elif self.algorithm == self.supported_algorithms[9]: #KMeans 70 | return OffloadKMeans(self.model) 71 | elif self.algorithm == self.supported_algorithms[10]: #PCA 72 | return OffloadPCA(self.model, self.optional) 73 | 74 | def check_model_validity(self, model): 75 | if not self.is_algorithm_supported(model): 76 | raise TypeError("Input ML model not supported! Only LDA, QDA, GNB, LR, Logit, SVM, PA, KMeans, PCA and Perceptron of scikit-learn are supported.") 77 | 78 | if not self.is_model_trained(model): 79 | raise TypeError("Input ML model not trained on a dataset! First .fit() on a dataset and then offload.") 80 | 81 | if not self.is_model_binary(model): 82 | raise TypeError("Input ML model trained on a multiclass dataset! Only binary-class models are supported.") 83 | 84 | if self.optional: 85 | if self.get_algorithm(self.optional) != "StandardScaler": 86 | raise TypeError("Only StandardScaler is supported as the second argument.") 87 | elif not self.is_model_trained(self.optional): 88 | raise TypeError("First fit StandardScaler on the training dataset and then offload.") 89 | 90 | def get_params(self): 91 | return self.offloader.get_params() 92 | 93 | def export_to_arduino(self, path): 94 | preamble = "//This code was autogenerated using micro-learn.\n//Use the dummy variable array 'data' to interact with the ML inference.\n\n" 95 | code = preamble + self.offloader.get_arduino_code() 96 | 97 | f = open(path, 'a') 98 | f.write(code) 99 | f.close() 100 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import setuptools 2 | 3 | with open("README.md", "r") as fh: 4 | long_description = fh.read() 5 | 6 | setuptools.setup( 7 | name="micro-learn", 8 | version="0.0.4", 9 | author="Adarsh Pal Singh", 10 | author_email="adarshpalsingh1996@gmail.com", 11 | description="A Python library to export scikit-learn models to microcontrollers", 12 | long_description=long_description, 13 | long_description_content_type="text/markdown", 14 | url="https://github.com/adarsh1001/micro-learn", 15 | packages=setuptools.find_packages(), 16 | classifiers=[ 17 | "Programming Language :: Python :: 3", 18 | "License :: OSI Approved :: MIT License", 19 | "Operating System :: OS Independent", 20 | ], 21 | python_requires='>=3.6', 22 | ) 23 | --------------------------------------------------------------------------------