├── README.md ├── SVM_fully from scratch (Helper) .pdf ├── SVM_implementation.ipynb └── evaluate.py /README.md: -------------------------------------------------------------------------------- 1 | # 🎓 Support Vector Machines (SVM) Implementations 2 | 3 | Welcome to our repository where we delve into the implementation of Support Vector Machines (SVM) for educational purposes in our Machine Learning course. This repository is designed to facilitate a deep understanding of SVM through practical examples and custom implementations. 4 | 5 | SVM Illustration 6 | 7 | ## 📄 Overview 8 | 9 | In this repository, you will find two main types of SVM implementations: 10 | 11 | 1. **SVM with scikit-learn**: Utilize the powerful scikit-learn library to implement standard SVM models quickly and efficiently. 12 | 2. **SVM from Scratch**: Challenge yourself by building SVM models from the ground up, gaining a deeper understanding of the underlying mechanics. You will also become familiar with the CVXOPT library. 13 | 14 | ## 🌟 Features 15 | 16 | We cover a range of scenarios and advanced topics in SVM, including: 17 | 18 | - **Kernel SVMs**: Explore the use of different kernels such as linear, polynomial, and radial basis function (RBF) to understand how they influence the decision boundaries of the SVM. 19 | - **Multiclass Classification**: Learn how to extend the binary classification capability of SVM to handle multiple classes, which is essential for dealing with more complex datasets. 20 | 21 | ## 📚 Contents 22 | 23 | The repository is organized as follows: 24 | 25 | - `README.md`: This file providing an overview of the repository. 26 | - `SVM_fully from scratch (Helper).pdf`: A detailed PDF guide to understanding and implementing SVM from scratch. 27 | - `SVM_implementation.ipynb`: Jupyter notebook demonstrating the implementation of SVM using scikit-learn and from scratch. 28 | - `evaluate.py`: A Python script to evaluate the performance of the SVM models. 29 | 30 | ## 🚀 Getting Started 31 | 32 | To get started with the code in this repository, follow these steps: 33 | 34 | 1. **Clone the repository**: 35 | ```sh 36 | git clone https://github.com/shining0611armor/svm-implementations.git 37 | cd svm-implementations 38 | ``` 39 | 40 | 41 | 2. **Explore the Jupyter notebooks**: 42 | ```sh 43 | jupyter notebook 44 | ``` 45 | Open and run the `SVM_implementation.ipynb` notebook to see the SVM implementations in action. 46 | 47 | ## 📈 Usage 48 | 49 | - **SVM with scikit-learn**: 50 | - Quickly implement SVM models with minimal code. 51 | - Experiment with different kernels and hyperparameters. 52 | 53 | - **SVM from Scratch**: 54 | - Gain a deep understanding of the mathematical foundations of SVM. 55 | - Learn how to solve the quadratic optimization problem using CVXOPT. 56 | 57 | ## 🎯 Goals 58 | 59 | By the end of this course, you will: 60 | - Have a strong understanding of SVM and its applications in machine learning. 61 | - Be able to implement SVM models using scikit-learn for quick and efficient solutions. 62 | - Gain the ability to build SVM models from scratch, enhancing your problem-solving and coding skills. 63 | 64 | 65 | ## 👩‍🏫 About the Instructor 66 | 67 | I am Mehran Tamjidi, a passionate educator and researcher in the field of machine learning and artificial intelligence. This repository is a part of my efforts to provide comprehensive and practical knowledge to students and enthusiasts. 68 | 69 | ## 📫 Contact 70 | 71 | Feel free to reach out if you have any questions or suggestions: 72 | - **Email**: mehrant.0611@gmail.com 73 | - **GitHub**: [shining0611armor](https://github.com/shining0611armor) 74 | 75 | 76 | 77 | Happy Learning! 😊 78 | -------------------------------------------------------------------------------- /SVM_fully from scratch (Helper) .pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shining0611armor/SVM-Implementation/d1123df34731073600819888772c86769495154f/SVM_fully from scratch (Helper) .pdf -------------------------------------------------------------------------------- /evaluate.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from sklearn.metrics import confusion_matrix, precision_score, recall_score, f1_score 3 | import seaborn as sns 4 | import matplotlib.pyplot as plt 5 | 6 | def calculate_metrics_and_plot(y_true, y_pred, labels=None): 7 | # Calculate confusion matrix 8 | cm = confusion_matrix(y_true, y_pred, labels=labels) 9 | 10 | # Calculate precision, recall, and F1-score 11 | precision = precision_score(y_true, y_pred, average=None, labels=labels) 12 | recall = recall_score(y_true, y_pred, average=None, labels=labels) 13 | f1 = f1_score(y_true, y_pred, average=None, labels=labels) 14 | 15 | # Calculate accuracy 16 | accuracy = np.sum(np.diag(cm)) / np.sum(cm) 17 | 18 | # Print precision, recall, F1-score, and accuracy 19 | for i in range(len(labels)): 20 | print(f"Class {labels[i]} - Precision: {precision[i]:.4f}, Recall: {recall[i]:.4f}, F1-score: {f1[i]:.4f}") 21 | 22 | print(f"Accuracy: {accuracy:.4f}") 23 | 24 | # Set custom color map and font size 25 | sns.set(font_scale=1.2) 26 | sns.set_style("whitegrid") 27 | 28 | # Plot the confusion matrix as a heatmap 29 | plt.figure(figsize=(8, 6)) 30 | heatmap = sns.heatmap(cm, annot=True, fmt='d', cmap='BuGn', xticklabels=labels, yticklabels=labels, annot_kws={"size": 14}) 31 | 32 | # Set the title font 33 | heatmap.set_title('Confusion Matrix', fontdict={'fontsize': 16, 'family': 'serif'}) 34 | 35 | plt.xlabel('Predicted') 36 | plt.ylabel('True') 37 | plt.show() 38 | 39 | if __name__ == "__main__": 40 | y_true = np.array([1, 0, 1, 1, 2, 1, 0, 2]) 41 | y_pred = np.array([1, 0, 1, 1, 1, 2, 0, 2]) 42 | calculate_metrics_and_plot(y_true, y_pred, labels=[0, 1, 2]) 43 | --------------------------------------------------------------------------------