├── .github ├── ISSUE_TEMPLATE │ ├── bug_report.md │ ├── feature_request.md │ └── new-layer-algorithm.md └── pull_request_template.md ├── .gitignore ├── CONTRIBUTING.md ├── CONTRIBUTORS.md ├── DL_algorithms ├── Adam Optimizer │ └── Adam_optimizer_from_scratch.ipynb ├── Convolutional Neural Network │ ├── CNN_from_scratch.ipynb │ └── README.md ├── MultiLayerPerceptron │ ├── MultilayerPerceptron.ipynb │ └── README.md ├── scratch code for single layer perceptron │ └── perceptron_scratch_code.ipynb └── word2vec │ ├── README.md │ ├── assets │ └── word2vec.jpg │ ├── word2vec.ipynb │ └── word2vec.py ├── LICENSE ├── README.md ├── requirements.txt └── traditional_ML_algorithms ├── Decision Trees ├── .ipynb_checkpoints │ └── Decision_Tree_from_scratch-checkpoint.ipynb └── Decision_Tree_from_scratch.ipynb ├── K-Means ├── K-Means.ipynb └── README.md ├── KNN ├── iphone_purchase_records.csv ├── knn_scratch.ipynb └── readme.md ├── LogisticRegression ├── ChurnData.csv └── Logistic_Regression.ipynb ├── Multiple Linear Regression ├── Multiple_Linear_Regression.ipynb ├── README.md └── data.csv ├── Polynomial Regression └── Polynomial_reg_from_scratch.ipynb ├── Principal Component Analysis ├── PCA_FaceRecognition │ ├── Eigenfaces_Face_Recognition.pdf │ ├── PCA_Face_Recognition.ipynb │ ├── archive.zip │ ├── readme.md │ ├── test1.jpg │ ├── test10.jpg │ ├── test2.jpg │ ├── test3.jpg │ ├── test4.jpg │ ├── test5.jpg │ ├── test6.jpg │ ├── test7.jpg │ ├── test8.jpg │ └── test9.jpg ├── PCA_Face_Recognition │ ├── Eigenfaces_Face_Recognition.pdf │ ├── PCA_Face_Recognition.ipynb │ ├── archive.zip │ ├── test1.jpg │ ├── test10.jpg │ ├── test2.jpg │ ├── test3.jpg │ ├── test4.jpg │ ├── test5.jpg │ ├── test6.jpg │ ├── test7.jpg │ ├── test8.jpg │ └── test9.jpg ├── Principal_Component_Analysis.ipynb ├── README.md └── data.csv ├── SimpleLinearRegression ├── README.md ├── SimpleLinearRegression.ipynb ├── data.csv └── model.py └── svm algotrithm └── svm_classifier_scratch_code.ipynb /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | title: '' 5 | labels: bug 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Describe the bug** 11 | A clear and concise description of what the bug is. 12 | 13 | **To Reproduce** 14 | Steps to reproduce the behavior: 15 | 1. Go to '...' 16 | 2. Click on '....' 17 | 3. Scroll down to '....' 18 | 4. See error 19 | 20 | **Expected behavior** 21 | A clear and concise description of what you expected to happen. 22 | 23 | **Screenshots** 24 | If applicable, add screenshots to help explain your problem. 25 | 26 | **Desktop (please complete the following information):** 27 | - OS: [e.g. iOS] 28 | - Browser [e.g. chrome, safari] 29 | - Version [e.g. 22] 30 | 31 | **Package Versions** 32 | List the packages and their versions you used related to your issue. ex:(tensorflow - 2.2.0, numpy - 1.8.2, etc.) 33 | 34 | **Additional context** 35 | Add any other context about the problem here. 36 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for this project 4 | title: '' 5 | labels: enhancement 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Is your feature request related to a problem? Please describe.** 11 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 12 | 13 | **Describe the solution you'd like** 14 | A clear and concise description of what you want to happen. 15 | 16 | **Describe alternatives you've considered** 17 | A clear and concise description of any alternative solutions or features you've considered. 18 | 19 | **Additional context** 20 | Add any other context or screenshots about the feature request here. 21 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/new-layer-algorithm.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: New layer/algorithm 3 | about: Submit new layer/algorithm design 4 | title: '' 5 | labels: new layer/algo 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Name** 11 | Provide the name of layer/algorithm you want to contribute. 12 | 13 | **Packages used** 14 | List all the packages and versions used. 15 | 16 | **Brief explanation** 17 | Describe what this algorithm/layer does in brief 18 | -------------------------------------------------------------------------------- /.github/pull_request_template.md: -------------------------------------------------------------------------------- 1 | fixes #issue_number (if any, otherwise delete this line) 2 | 3 | # Description 4 | 5 | > :memo: Please include a summary of the change. 6 | > 7 | > * Please also include relevant motivation and context. 8 | > * List any dependencies that are required for this change. 9 | 10 | ## Type of change 11 | 12 | Note: Please delete options that are not relevant. 13 | 14 | - [ ] Bug fix (non-breaking change which fixes an issue) 15 | - [ ] Documentation update 16 | - [ ] New feature (non-breaking change which adds functionality) 17 | - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) 18 | - [ ] New layer/algorithm 19 | - [ ] Other (Specify) 20 | 21 | ### Implementation details 22 | (only for `New layer/algorithm`) 23 | > * List all the packages and their versions used. 24 | > * Include test results such as: 25 | > * Implementation time for 1 instance of data. 26 | 27 | ## Tests 28 | 29 | > :memo: Please describe the tests that you ran to verify your changes. 30 | > 31 | > * Provide instructions so we can reproduce. 32 | > * Please also list any relevant details for your test configuration. 33 | 34 | ## Checklist 35 | 36 | - [ ] I have performed a self code_review of my own code. 37 | - [ ] I have commented my code, particularly in hard-to-understand areas. 38 | - [ ] I have added tests that prove my fix is effective or that my feature works. 39 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | .vscode 3 | .idea 4 | tempCodeRunnerFile.py -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # How to Contribute 2 | 3 | --- 4 | 5 | The ScratchAI project can only grow through the contributions of the community. We appreciate your enthusiasm and contribution in helping our initiative. 6 | 7 | 8 | ## Creating an issue 9 | 10 | --- 11 | * Make sure you are using appropriate template that corresponds to your issue. 12 | * If your **issue** is different from provided templates, start a plain issue without using the template. 13 | 14 | 15 | ## Submitting a pull request 16 | 17 | --- 18 | Here are a few things you can do that will increase the likelihood of your pull request being accepted: 19 | 20 | * For a new feature or function, please create an issue first to discuss it with us before submitting a pull request. 21 | * Keep your change as focused as possible. 22 | * If there are changes in multiple directories you would like to make that are not dependent upon each other, please submit them as separate pull requests. 23 | * The pull request should have a useful title and description. 24 | * Explain the rationale for your change in the pull request using a pull request template. 25 | 26 | * Here's a quick guid to create a pull request: 27 | * Fork the github project. 28 | * Clone the git repository 29 | ``` (bash) 30 | $ git clone https://github.com/YOUR-GITHUB-USERNAME/ScratchAI.git 31 | ``` 32 | * When your implementation is ready, [submit a pull request][pr]. Add some comments or screen shots to help us. 33 | * Wait for us to review your pull request. If something is wrong or if we want you to make some changes before the merge, we'll let you know through commit comments or pull request comments. 34 | -------------------------------------------------------------------------------- /CONTRIBUTORS.md: -------------------------------------------------------------------------------- 1 | ![Contributors](https://img.shields.io/github/contributors/Math-behind-AI/ScratchAI) 2 | 3 | 4 | # Contributors 5 | 6 | ## Special thanks to all the people who has helped this project so far: 7 | 8 | * [Vishesh Tripathi](https://github.com/Vishesht27) 9 | * [Mayuresh Aagashe](https://github.com/mayureshagashe2105) 10 | * [Rithuraj Nambiar](https://github.com/rithurajnambiar17) 11 | * [Hemanth Sai](https://github.com/HemanthSai7) 12 | 13 | ![GitHub Contributors Image](https://contrib.rocks/image?repo=Math-behind-AI/ScratchAI) 14 | 15 | > Note: Display Pics & the above list are not in order! 16 | 17 | ## Would like to join this list? Here is how to help the project! 18 | 19 | We're currently looking for contributions for the following: 20 | 21 | - [x] Bug fixes 22 | - [x] Submissions of new algorithm/layer 23 | - [x] Feature Request 24 | - [x] Enhancement of existing models 25 | 26 | For more information, please refer to our [CONTRIBUTING](CONTRIBUTING.md) guide. 27 | -------------------------------------------------------------------------------- /DL_algorithms/Convolutional Neural Network/README.md: -------------------------------------------------------------------------------- 1 | # Implementation of Convolutional Neural Network from Scratch 2 | 3 | ### Libraries used: 4 | `Numpy`
5 | `Matplotlib`
6 | `Pandas`
7 | `Tensorflow`
8 | 9 | #### :pencil2: A CNN, or Convolutional Neural Network, is a type of artificial neural network that is commonly used for image/object detection and classification. Deep Learning detects things in images by employing a CNN. 10 | 11 | #### :pencil2: CNNs play an important part in a variety of activities/functions such as image processing difficulties, computer vision tasks such as localization and segmentation, video analysis, recognising obstacles in self-driving cars, and speech recognition in natural language processing. 12 | 13 | #### :pencil2: Here we have built a ConvNet to identify Sign language digits from scratch along with in-depth implementation of each layer in neural network 14 | -------------------------------------------------------------------------------- /DL_algorithms/MultiLayerPerceptron/README.md: -------------------------------------------------------------------------------- 1 | Name: 2 | Multilayer Perceptron 3 | 4 | Packages used:
5 | `Numpy` : '1.23.1'
6 | `Pandas` : 1.4.3
7 | 8 | Brief explanation: 9 | Creating a Multilayer Perceptron model from scratch using numpy module for classification. 10 | -------------------------------------------------------------------------------- /DL_algorithms/word2vec/README.md: -------------------------------------------------------------------------------- 1 | ## Implementation of word2vec from scratch 2 | --- 3 | This is a simple implementation of word2vec from scratch. The code is written in Python 3.6. The code is partially based on the following paper: **[Efficient Estimation of Word Representations in Vector Space](https://arxiv.org/abs/1301.3781)** 4 | 5 | ![word2vec](./assets/word2vec.jpg) 6 | 7 | ## Words embeddings 8 | --- 9 | Word embeddings are a type of word representation that allows words with similar meaning to have a similar representation. The model learns to represent words in a way that words with similar meaning are represented by vectors that are close to each other in the vector space. 10 | 11 | ## Mathematical background 12 | --- 13 | The word2vec model is a neural network model that learns word embeddings. The model is trained to predict the context of a word given a set of words. The context of a word is defined as the set of words that appear close to the target word in a text. The model is trained using a **skip-gram** architecture. There are two different architectures for training word2vec models: **skip-gram** and **continuous bag of words** (CBOW). 14 | 15 | ```python 16 | def generate_training_data() 17 | ``` 18 | This function generates the training data. Primary purpose of this function is to generate the training data for the skip-gram model. It implements the following. 19 | - Preprocess the text sentence. 20 | - Split the text sentence into words and create lookup dictionaries of words and indices. 21 | ```python 22 | self.word2id={"word in sentence":index of the word} 23 | self.id2word={index of the word:"word in sentence"} 24 | ``` 25 | - Create a corpus of words which is a list of all the words in the text sentence. 26 | ```python 27 | corpus=["word1","word2","word3",...] 28 | ``` 29 | - Convert the target word to one-hot encoded vector. 30 | - Cycle through context window and create a training sample for each target word. 31 | 32 | ```python 33 | def one_hot_encode() 34 | ``` 35 | - Converts the target word to one-hot encoded vector. 36 | 37 | ```python 38 | def train() 39 | ``` 40 | - Randomly initialize the weights of the neural network. 41 | - Cycle through each training sample and perform forward and backward propagation. 42 | - Update the weights after each iteration. 43 | - Calculate loss after each iteration. 44 | 45 | ```python 46 | def forward_pass() 47 | ``` 48 | - Starting with the first epoch, the forward pass calculates the dot product of the input word vector and the hidden layer weights. This is then passed through the softmax function to get the output word vector. 49 | 50 | ```python 51 | def softmax() 52 | ``` 53 | - The softmax function is used to calculate the probability distribution of the output word vector. The output word vector is a one-hot encoded vector with the index of the target word set to 1 and the rest of the indices set to 0. The softmax function calculates the probability of each word in the vocabulary being the target word. 54 | 55 | ```python 56 | def backprop() 57 | ``` 58 | - The backpropagation function calculates the gradients of the loss function with respect to the weights of the neural network. The gradients are then used to update the weights of the neural network. 59 | 60 | ```python 61 | def word_vec() 62 | ``` 63 | - This function returns the word vector for a given word. 64 | 65 | ```python 66 | def vec_similarity() 67 | ``` 68 | - This function calculates the cosine similarity between two word vectors. 69 | 70 | > For better understanding of the algorithm and calculations , please refer to this awesome blog by Jay Alammar: [Illustrated Word2Vec](https://jalammar.github.io/illustrated-word2vec/) 71 | -------------------------------------------------------------------------------- /DL_algorithms/word2vec/assets/word2vec.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/DL_algorithms/word2vec/assets/word2vec.jpg -------------------------------------------------------------------------------- /DL_algorithms/word2vec/word2vec.py: -------------------------------------------------------------------------------- 1 | # Implementation of word2vec algorithm from scratch 2 | 3 | import numpy as np 4 | 5 | class Word2Vec: 6 | def __init__(self, vocab_size, embedding_size, window_size,epochs, learning_rate): 7 | self.vocab_size = vocab_size 8 | self.embedding_size = embedding_size 9 | self.window_size = window_size 10 | self.epochs = epochs 11 | self.learning_rate = learning_rate 12 | self.word2id = {} 13 | self.id2word = {} 14 | 15 | def generate_training_data(self,text): 16 | text = text.lower().replace('.', ' .') 17 | words = text.split(' ') 18 | for i, word in enumerate(words): 19 | self.word2id[word] = i 20 | self.id2word[i] = word 21 | corpus = [self.id2word[i] for i in range(len(self.id2word))] 22 | print(corpus) 23 | training_data=[] 24 | for i,word in enumerate(corpus): 25 | sent_len=len(corpus) 26 | w_target=self.one_hot_encode(corpus[i]) 27 | w_context=[] 28 | for j in range(i-self.window_size,i+self.window_size+1): 29 | if j!=1 and j<=sent_len-1 and j>=0: 30 | w_context.append(self.one_hot_encode(corpus[j])) 31 | training_data.append([w_target,w_context]) 32 | return np.array(training_data,dtype=object) 33 | 34 | def one_hot_encode(self,word): 35 | vector=np.zeros(self.vocab_size) 36 | vector[self.word2id[word]]=1 37 | return vector 38 | 39 | def train(self,training_data): 40 | self.W1 = np.random.uniform(-1, 1, size=(self.vocab_size, self.embedding_size)) 41 | self.W2 = np.random.uniform(-1, 1, size=(self.embedding_size, self.vocab_size)) 42 | for i in range(self.epochs): 43 | self.loss=0 44 | for w_target, w_context in training_data: 45 | y_pred,h,output_layer=self.forward_pass(w_target) 46 | EI=np.sum([np.subtract(y_pred,word) for word in w_context],axis=0) 47 | self.backprop(EI,h,w_target) 48 | self.loss+=-np.sum([output_layer[np.where(word==1)] for word in w_context])+len(w_context)*np.log(np.sum(np.exp(output_layer))) 49 | print(f"Epoch:{i},Loss:{self.loss}") 50 | 51 | def forward_pass(self,one_hot_target): 52 | h=np.dot(self.W1.T,one_hot_target) 53 | output_layer=np.dot(self.W2.T,h) 54 | y_context=self.softmax(output_layer) 55 | return y_context,h,output_layer 56 | 57 | def softmax(self,one_hot_target): 58 | exp=np.exp(one_hot_target) 59 | return exp/np.sum(exp) 60 | 61 | def backprop(self,EI,h,w_target): 62 | dL_dW2=np.outer(h,EI) 63 | dL_dW1=np.outer(w_target,np.dot(self.W2,EI.T)) 64 | self.W1=self.W1-(self.learning_rate*dL_dW1) 65 | self.W2=self.W2-(self.learning_rate*dL_dW2) 66 | 67 | def word_vec(self,word): 68 | w_index=self.word2id[word] 69 | v_word=self.W1[w_index] 70 | return v_word 71 | 72 | def vec_similarity(self,word,top_n): 73 | v_word1=self.word_vec(word) 74 | word_sim={} 75 | for i in range(self.vocab_size): 76 | v_target=self.W1[i] 77 | theta_sum=np.dot(v_word1,v_target) 78 | theta_den=np.linalg.norm(v_word1)*np.linalg.norm(v_target) 79 | theta=theta_sum/theta_den 80 | word=self.id2word[i] 81 | word_sim[word]=theta 82 | words_sorted=sorted(word_sim.items(),key=lambda x:x[1],reverse=True) 83 | for word,similarity in words_sorted[:top_n]: 84 | print(word,similarity) 85 | 86 | 87 | res=Word2Vec(17,2,2,10,0.01) 88 | text="Recurrent neural network based language model has been proposed to overcome certain limitations of the feedforward NNLM" 89 | training_data=res.generate_training_data(text) 90 | print(training_data) 91 | res.train(training_data) 92 | res.vec_similarity('limitations',3) -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![image](https://user-images.githubusercontent.com/75118658/193441891-f4e14df7-2213-4ac1-b9a7-c9811e6cf54a.png) 2 | 3 | # Scratch-AI [![Open Source Love](https://badges.frapsoft.com/os/v1/open-source.svg?v=103)](https://github.com/ellerbrock/open-source-badges/) 4 | 5 | --- 6 | 7 | 8 | ✨ **The Scratch-AI** is a repository with different algorithms of Machine Learning & Deep Learning implemented from scratch. 9 | 10 | 🗄 The main motto behind this repository is to grasp the mathematical aspect and intuition of different AI algorithms. 11 | 12 | 🤖 AI is being used everywhere nowadays, but people don't know about implementation of complex mathematical functions behind the scene. 13 | 14 | ## Table of Contents 15 | - [ScratchAI](#scratch-ai-) 16 | * [Machine Learning Algorithms](/traditional_ML_algorithms/) 17 | * [Decision Trees](/traditional_ML_algorithms/Decision%20Trees/) 18 | * [KNN](./traditional_ML_algorithms/KNN) 19 | * [K-Means](./traditional_ML_algorithms/K-Means) 20 | * [Linear Regression](./traditional_ML_algorithms/SimpleLinearRegression) 21 | * [Logistic Regression](./traditional_ML_algorithms/LogisticRegression) 22 | * [Multiple Linear Regression](/traditional_ML_algorithms/Multiple%20Linear%20Regression/) 23 | * [Polynomial Regression](/traditional_ML_algorithms/Polynomial%20Regression/) 24 | * [Principal Component Analysis](/traditional_ML_algorithms/Principal%20Component%20Analysis/) 25 | * [SVM](/traditional_ML_algorithms/svm%20algotrithm/) 26 | * [Deep Learning Algorithms](/deep_learning_algorithms/) 27 | * [Adam Optimizer](/DL_algorithms/Adam%20Optimizer/) 28 | * [Convolutional Neural Network](/DL_algorithms/Convolutional%20Neural%20Network/) 29 | * [Single Layer Perceptron](/DL_algorithms/scratch%20code%20for%20single%20layer%20perceptron/) 30 | * [Multi Layer Perceptron](/DL_algorithms/MultiLayerPerceptron/) 31 | * [Word2Vec](./DL_algorithms/word2vec/) 32 | * [Contributions](#contributions) 33 | * [Maintainers](#maintainers) 34 | 35 | ## Contributions 36 | --- 37 | 38 | We welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to 39 | your active involvement. 40 | 41 | Contributors can contribute to the project in the following ways: 42 | 43 | 1. Contributing by fixing existing issues. 44 | 2. Submitting a better layer design or algorithm than the existing one. 45 | 3. Submitting new layer design or algorithm. 46 | 47 | NOTE: For detailed instructions refer to [CONTRIBUTING.md](CONTRIBUTING.md) 48 | 49 | 50 | ## Maintainers 51 | 52 | --- 53 | 54 | 55 | | | | | 56 | | :---: | :---: | :---: | 57 | | **[Mayuresh Agashe](https://github.com/mayureshagashe2105)**| **[Vishesh Tripathi](https://github.com/Vishesht27)**| **[Hemanth Sai](https://github.com/HemanthSai7)**| 58 | | | | | 59 | 60 | 61 | 62 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | numpy 2 | pandas 3 | matplotlib 4 | scipy 5 | -------------------------------------------------------------------------------- /traditional_ML_algorithms/K-Means/README.md: -------------------------------------------------------------------------------- 1 | ## K-Means Clustering Algorithm 2 | 3 | ### Packages used: 4 | `Numpy` : 1.3.4 5 | `Pandas` : 1.21.4 6 | 7 | ### Brief explanation: 8 | Unsupervised algorithm to group data points into `k` different clusters. -------------------------------------------------------------------------------- /traditional_ML_algorithms/KNN/iphone_purchase_records.csv: -------------------------------------------------------------------------------- 1 | Gender,Age,Salary,Purchase Iphone 2 | Male,19,19000,0 3 | Male,35,20000,0 4 | Female,26,43000,0 5 | Female,27,57000,0 6 | Male,19,76000,0 7 | Male,27,58000,0 8 | Female,27,84000,0 9 | Female,32,150000,1 10 | Male,25,33000,0 11 | Female,35,65000,0 12 | Female,26,80000,0 13 | Female,26,52000,0 14 | Male,20,86000,0 15 | Male,32,18000,0 16 | Male,18,82000,0 17 | Male,29,80000,0 18 | Male,47,25000,1 19 | Male,45,26000,1 20 | Male,46,28000,1 21 | Female,48,29000,1 22 | Male,45,22000,1 23 | Female,47,49000,1 24 | Male,48,41000,1 25 | Female,45,22000,1 26 | Male,46,23000,1 27 | Male,47,20000,1 28 | Male,49,28000,1 29 | Female,47,30000,1 30 | Male,29,43000,0 31 | Male,31,18000,0 32 | Male,31,74000,0 33 | Female,27,137000,1 34 | Female,21,16000,0 35 | Female,28,44000,0 36 | Male,27,90000,0 37 | Male,35,27000,0 38 | Female,33,28000,0 39 | Male,30,49000,0 40 | Female,26,72000,0 41 | Female,27,31000,0 42 | Female,27,17000,0 43 | Female,33,51000,0 44 | Male,35,108000,0 45 | Male,30,15000,0 46 | Female,28,84000,0 47 | Male,23,20000,0 48 | Male,25,79000,0 49 | Female,27,54000,0 50 | Male,30,135000,1 51 | Female,31,89000,0 52 | Female,24,32000,0 53 | Female,18,44000,0 54 | Female,29,83000,0 55 | Female,35,23000,0 56 | Female,27,58000,0 57 | Female,24,55000,0 58 | Female,23,48000,0 59 | Male,28,79000,0 60 | Male,22,18000,0 61 | Female,32,117000,0 62 | Male,27,20000,0 63 | Male,25,87000,0 64 | Female,23,66000,0 65 | Male,32,120000,1 66 | Female,59,83000,0 67 | Male,24,58000,0 68 | Male,24,19000,0 69 | Female,23,82000,0 70 | Female,22,63000,0 71 | Female,31,68000,0 72 | Male,25,80000,0 73 | Female,24,27000,0 74 | Female,20,23000,0 75 | Female,33,113000,0 76 | Male,32,18000,0 77 | Male,34,112000,1 78 | Male,18,52000,0 79 | Female,22,27000,0 80 | Female,28,87000,0 81 | Female,26,17000,0 82 | Male,30,80000,0 83 | Male,39,42000,0 84 | Male,20,49000,0 85 | Male,35,88000,0 86 | Female,30,62000,0 87 | Female,31,118000,1 88 | Male,24,55000,0 89 | Female,28,85000,0 90 | Male,26,81000,0 91 | Male,35,50000,0 92 | Male,22,81000,0 93 | Female,30,116000,0 94 | Male,26,15000,0 95 | Female,29,28000,0 96 | Female,29,83000,0 97 | Female,35,44000,0 98 | Female,35,25000,0 99 | Male,28,123000,1 100 | Male,35,73000,0 101 | Female,28,37000,0 102 | Male,27,88000,0 103 | Male,28,59000,0 104 | Female,32,86000,0 105 | Female,33,149000,1 106 | Female,19,21000,0 107 | Male,21,72000,0 108 | Female,26,35000,0 109 | Male,27,89000,0 110 | Male,26,86000,0 111 | Female,38,80000,0 112 | Female,39,71000,0 113 | Female,37,71000,0 114 | Male,38,61000,0 115 | Male,37,55000,0 116 | Male,42,80000,0 117 | Male,40,57000,0 118 | Male,35,75000,0 119 | Male,36,52000,0 120 | Male,40,59000,0 121 | Male,41,59000,0 122 | Female,36,75000,0 123 | Male,37,72000,0 124 | Female,40,75000,0 125 | Male,35,53000,0 126 | Female,41,51000,0 127 | Female,39,61000,0 128 | Male,42,65000,0 129 | Male,26,32000,0 130 | Male,30,17000,0 131 | Female,26,84000,0 132 | Male,31,58000,0 133 | Male,33,31000,0 134 | Male,30,87000,0 135 | Female,21,68000,0 136 | Female,28,55000,0 137 | Male,23,63000,0 138 | Female,20,82000,0 139 | Male,30,107000,1 140 | Female,28,59000,0 141 | Male,19,25000,0 142 | Male,19,85000,0 143 | Female,18,68000,0 144 | Male,35,59000,0 145 | Male,30,89000,0 146 | Female,34,25000,0 147 | Female,24,89000,0 148 | Female,27,96000,1 149 | Female,41,30000,0 150 | Male,29,61000,0 151 | Male,20,74000,0 152 | Female,26,15000,0 153 | Male,41,45000,0 154 | Male,31,76000,0 155 | Female,36,50000,0 156 | Male,40,47000,0 157 | Female,31,15000,0 158 | Male,46,59000,0 159 | Male,29,75000,0 160 | Male,26,30000,0 161 | Female,32,135000,1 162 | Male,32,100000,1 163 | Male,25,90000,0 164 | Female,37,33000,0 165 | Male,35,38000,0 166 | Female,33,69000,0 167 | Female,18,86000,0 168 | Female,22,55000,0 169 | Female,35,71000,0 170 | Male,29,148000,1 171 | Female,29,47000,0 172 | Male,21,88000,0 173 | Male,34,115000,0 174 | Female,26,118000,0 175 | Female,34,43000,0 176 | Female,34,72000,0 177 | Female,23,28000,0 178 | Female,35,47000,0 179 | Male,25,22000,0 180 | Male,24,23000,0 181 | Female,31,34000,0 182 | Male,26,16000,0 183 | Female,31,71000,0 184 | Female,32,117000,1 185 | Male,33,43000,0 186 | Female,33,60000,0 187 | Male,31,66000,0 188 | Female,20,82000,0 189 | Female,33,41000,0 190 | Male,35,72000,0 191 | Male,28,32000,0 192 | Male,24,84000,0 193 | Female,19,26000,0 194 | Male,29,43000,0 195 | Male,19,70000,0 196 | Male,28,89000,0 197 | Male,34,43000,0 198 | Female,30,79000,0 199 | Female,20,36000,0 200 | Male,26,80000,0 201 | Male,35,22000,0 202 | Male,35,39000,0 203 | Male,49,74000,0 204 | Female,39,134000,1 205 | Female,41,71000,0 206 | Female,58,101000,1 207 | Female,47,47000,0 208 | Female,55,130000,1 209 | Female,52,114000,0 210 | Female,40,142000,1 211 | Female,46,22000,0 212 | Female,48,96000,1 213 | Male,52,150000,1 214 | Female,59,42000,0 215 | Male,35,58000,0 216 | Male,47,43000,0 217 | Female,60,108000,1 218 | Male,49,65000,0 219 | Male,40,78000,0 220 | Female,46,96000,0 221 | Male,59,143000,1 222 | Female,41,80000,0 223 | Male,35,91000,1 224 | Male,37,144000,1 225 | Male,60,102000,1 226 | Female,35,60000,0 227 | Male,37,53000,0 228 | Female,36,126000,1 229 | Male,56,133000,1 230 | Female,40,72000,0 231 | Female,42,80000,1 232 | Female,35,147000,1 233 | Male,39,42000,0 234 | Male,40,107000,1 235 | Male,49,86000,1 236 | Female,38,112000,0 237 | Male,46,79000,1 238 | Male,40,57000,0 239 | Female,37,80000,0 240 | Female,46,82000,0 241 | Female,53,143000,1 242 | Male,42,149000,1 243 | Male,38,59000,0 244 | Female,50,88000,1 245 | Female,56,104000,1 246 | Female,41,72000,0 247 | Female,51,146000,1 248 | Female,35,50000,0 249 | Female,57,122000,1 250 | Male,41,52000,0 251 | Female,35,97000,1 252 | Female,44,39000,0 253 | Male,37,52000,0 254 | Female,48,134000,1 255 | Female,37,146000,1 256 | Female,50,44000,0 257 | Female,52,90000,1 258 | Female,41,72000,0 259 | Male,40,57000,0 260 | Female,58,95000,1 261 | Female,45,131000,1 262 | Female,35,77000,0 263 | Male,36,144000,1 264 | Female,55,125000,1 265 | Female,35,72000,0 266 | Male,48,90000,1 267 | Female,42,108000,1 268 | Male,40,75000,0 269 | Male,37,74000,0 270 | Female,47,144000,1 271 | Male,40,61000,0 272 | Female,43,133000,0 273 | Female,59,76000,1 274 | Male,60,42000,1 275 | Male,39,106000,1 276 | Female,57,26000,1 277 | Male,57,74000,1 278 | Male,38,71000,0 279 | Male,49,88000,1 280 | Female,52,38000,1 281 | Female,50,36000,1 282 | Female,59,88000,1 283 | Male,35,61000,0 284 | Male,37,70000,1 285 | Female,52,21000,1 286 | Male,48,141000,0 287 | Female,37,93000,1 288 | Female,37,62000,0 289 | Female,48,138000,1 290 | Male,41,79000,0 291 | Female,37,78000,1 292 | Male,39,134000,1 293 | Male,49,89000,1 294 | Male,55,39000,1 295 | Male,37,77000,0 296 | Female,35,57000,0 297 | Female,36,63000,0 298 | Male,42,73000,1 299 | Female,43,112000,1 300 | Male,45,79000,0 301 | Male,46,117000,1 302 | Female,58,38000,1 303 | Male,48,74000,1 304 | Female,37,137000,1 305 | Male,37,79000,1 306 | Female,40,60000,0 307 | Male,42,54000,0 308 | Female,51,134000,0 309 | Female,47,113000,1 310 | Male,36,125000,1 311 | Female,38,50000,0 312 | Female,42,70000,0 313 | Male,39,96000,1 314 | Female,38,50000,0 315 | Female,49,141000,1 316 | Female,39,79000,0 317 | Female,39,75000,1 318 | Female,54,104000,1 319 | Male,35,55000,0 320 | Male,45,32000,1 321 | Male,36,60000,0 322 | Female,52,138000,1 323 | Female,53,82000,1 324 | Male,41,52000,0 325 | Female,48,30000,1 326 | Female,48,131000,1 327 | Female,41,60000,0 328 | Male,41,72000,0 329 | Female,42,75000,0 330 | Male,36,118000,1 331 | Female,47,107000,1 332 | Male,38,51000,0 333 | Female,48,119000,1 334 | Male,42,65000,0 335 | Male,40,65000,0 336 | Male,57,60000,1 337 | Female,36,54000,0 338 | Male,58,144000,1 339 | Male,35,79000,0 340 | Female,38,55000,0 341 | Male,39,122000,1 342 | Female,53,104000,1 343 | Male,35,75000,0 344 | Female,38,65000,0 345 | Female,47,51000,1 346 | Male,47,105000,1 347 | Female,41,63000,0 348 | Male,53,72000,1 349 | Female,54,108000,1 350 | Male,39,77000,0 351 | Male,38,61000,0 352 | Female,38,113000,1 353 | Male,37,75000,0 354 | Female,42,90000,1 355 | Female,37,57000,0 356 | Male,36,99000,1 357 | Male,60,34000,1 358 | Male,54,70000,1 359 | Female,41,72000,0 360 | Male,40,71000,1 361 | Male,42,54000,0 362 | Male,43,129000,1 363 | Female,53,34000,1 364 | Female,47,50000,1 365 | Female,42,79000,0 366 | Male,42,104000,1 367 | Female,59,29000,1 368 | Female,58,47000,1 369 | Male,46,88000,1 370 | Male,38,71000,0 371 | Female,54,26000,1 372 | Female,60,46000,1 373 | Male,60,83000,1 374 | Female,39,73000,0 375 | Male,59,130000,1 376 | Female,37,80000,0 377 | Female,46,32000,1 378 | Female,46,74000,0 379 | Female,42,53000,0 380 | Male,41,87000,1 381 | Female,58,23000,1 382 | Male,42,64000,0 383 | Male,48,33000,1 384 | Female,44,139000,1 385 | Male,49,28000,1 386 | Female,57,33000,1 387 | Male,56,60000,1 388 | Female,49,39000,1 389 | Male,39,71000,0 390 | Male,47,34000,1 391 | Female,48,35000,1 392 | Male,48,33000,1 393 | Male,47,23000,1 394 | Female,45,45000,1 395 | Male,60,42000,1 396 | Female,39,59000,0 397 | Female,46,41000,1 398 | Male,51,23000,1 399 | Female,50,20000,1 400 | Male,36,33000,0 401 | Female,49,36000,1 -------------------------------------------------------------------------------- /traditional_ML_algorithms/KNN/knn_scratch.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "a8712a48", 6 | "metadata": {}, 7 | "source": [ 8 | "# KNN" 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "id": "ccf2d425", 14 | "metadata": {}, 15 | "source": [ 16 | "### Import libraries and load data" 17 | ] 18 | }, 19 | { 20 | "cell_type": "code", 21 | "execution_count": 589, 22 | "id": "9e5efc38", 23 | "metadata": {}, 24 | "outputs": [ 25 | { 26 | "data": { 27 | "text/html": [ 28 | "
\n", 29 | "\n", 42 | "\n", 43 | " \n", 44 | " \n", 45 | " \n", 46 | " \n", 47 | " \n", 48 | " \n", 49 | " \n", 50 | " \n", 51 | " \n", 52 | " \n", 53 | " \n", 54 | " \n", 55 | " \n", 56 | " \n", 57 | " \n", 58 | " \n", 59 | " \n", 60 | " \n", 61 | " \n", 62 | " \n", 63 | " \n", 64 | " \n", 65 | " \n", 66 | " \n", 67 | " \n", 68 | " \n", 69 | " \n", 70 | " \n", 71 | " \n", 72 | " \n", 73 | " \n", 74 | " \n", 75 | " \n", 76 | " \n", 77 | " \n", 78 | " \n", 79 | " \n", 80 | " \n", 81 | " \n", 82 | " \n", 83 | " \n", 84 | " \n", 85 | " \n", 86 | " \n", 87 | " \n", 88 | " \n", 89 | "
GenderAgeSalaryPurchase Iphone
0019190000
1035200000
2126430000
3127570000
4019760000
\n", 90 | "
" 91 | ], 92 | "text/plain": [ 93 | " Gender Age Salary Purchase Iphone\n", 94 | "0 0 19 19000 0\n", 95 | "1 0 35 20000 0\n", 96 | "2 1 26 43000 0\n", 97 | "3 1 27 57000 0\n", 98 | "4 0 19 76000 0" 99 | ] 100 | }, 101 | "execution_count": 589, 102 | "metadata": {}, 103 | "output_type": "execute_result" 104 | } 105 | ], 106 | "source": [ 107 | "import numpy as np\n", 108 | "import pandas as pd \n", 109 | "dataset = pd.read_csv(\"iphone_purchase_records.csv\")\n", 110 | "X = dataset.iloc[:,:-1].values\n", 111 | "y = dataset.iloc[:, 3].values\n", 112 | "dataset.Gender=dataset.Gender.map({'Female':1,'Male':0})\n", 113 | "dataset.head(5)" 114 | ] 115 | }, 116 | { 117 | "cell_type": "markdown", 118 | "id": "525f8713", 119 | "metadata": {}, 120 | "source": [ 121 | "### Data pre-processing and selection" 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "execution_count": 590, 127 | "id": "987cf418", 128 | "metadata": {}, 129 | "outputs": [], 130 | "source": [ 131 | "from sklearn.preprocessing import LabelEncoder\n", 132 | "labelEncoder_gender = LabelEncoder()\n", 133 | "X[:,0] = labelEncoder_gender.fit_transform(X[:,0])\n", 134 | "\n", 135 | "\n", 136 | "import numpy as np\n", 137 | "X = np.vstack(X[:, :]).astype(np.float64)" 138 | ] 139 | }, 140 | { 141 | "cell_type": "markdown", 142 | "id": "cd629313", 143 | "metadata": {}, 144 | "source": [ 145 | "### Training and Testing data" 146 | ] 147 | }, 148 | { 149 | "cell_type": "code", 150 | "execution_count": 591, 151 | "id": "54b5a638", 152 | "metadata": {}, 153 | "outputs": [], 154 | "source": [ 155 | "train_size = int(dataset.shape[0]*0.60)" 156 | ] 157 | }, 158 | { 159 | "cell_type": "code", 160 | "execution_count": 592, 161 | "id": "b8269a8e", 162 | "metadata": {}, 163 | "outputs": [ 164 | { 165 | "name": "stdout", 166 | "output_type": "stream", 167 | "text": [ 168 | "Train_Shape: (240, 4)\n", 169 | "Test_Shape: (160, 4)\n" 170 | ] 171 | } 172 | ], 173 | "source": [ 174 | "train_df = dataset.iloc[:train_size,:] \n", 175 | "test_df = dataset.iloc[train_size:,:]\n", 176 | "train = dataset.values\n", 177 | "test = test_df.values\n", 178 | "y_true = test[:,-1]\n", 179 | "print('Train_Shape: ',train_df.shape)\n", 180 | "print('Test_Shape: ',test_df.shape)" 181 | ] 182 | }, 183 | { 184 | "cell_type": "markdown", 185 | "id": "9b5b1eca", 186 | "metadata": {}, 187 | "source": [ 188 | "### KNN in 3 Steps:\n", 189 | "1. Measure distance (Euclidean Distance or Manhattan Distance)\n", 190 | "2. Get nearest neighbours\n", 191 | "3. Predict Classifier" 192 | ] 193 | }, 194 | { 195 | "cell_type": "markdown", 196 | "id": "1eac0361", 197 | "metadata": {}, 198 | "source": [ 199 | "#### Step 1. Euclidian distance\n", 200 | "- Measuring Distance using Euclidean Distance:\n", 201 | " Mathematical formula √ (x2 − x1)2 + (y2 − y1)2" 202 | ] 203 | }, 204 | { 205 | "cell_type": "code", 206 | "execution_count": 593, 207 | "id": "b1513390", 208 | "metadata": {}, 209 | "outputs": [], 210 | "source": [ 211 | "from math import sqrt\n", 212 | "def euclidean_distance(x_test, x_train):\n", 213 | " distance = 0\n", 214 | " for i in range(len(x_test)-1):\n", 215 | " distance += (x_test[i]-x_train[i])**2\n", 216 | " return sqrt(distance)" 217 | ] 218 | }, 219 | { 220 | "cell_type": "markdown", 221 | "id": "ba1f5707", 222 | "metadata": {}, 223 | "source": [ 224 | "#### Step 2. Getting the nearest neighbours" 225 | ] 226 | }, 227 | { 228 | "cell_type": "code", 229 | "execution_count": 594, 230 | "id": "1f98eeb0", 231 | "metadata": {}, 232 | "outputs": [], 233 | "source": [ 234 | "def get_neighbors(x_test, x_train, num_neighbors):\n", 235 | " distances = []\n", 236 | " data = []\n", 237 | " for i in x_train:\n", 238 | " distances.append(euclidean_distance(x_test,i))\n", 239 | " data.append(i)\n", 240 | " distances = np.array(distances)\n", 241 | " data = np.array(data)\n", 242 | " sort_indexes = distances.argsort() #argsort() function returns indices by sorting distances data in ascending order\n", 243 | " data = data[sort_indexes] #modifying our data based on sorted indices, so that we can get the nearest neightbours\n", 244 | " return data[:num_neighbors] " 245 | ] 246 | }, 247 | { 248 | "cell_type": "markdown", 249 | "id": "ceba0d82", 250 | "metadata": {}, 251 | "source": [ 252 | "#### Step 3. Predicting the classifier of which our new data point belongs to." 253 | ] 254 | }, 255 | { 256 | "cell_type": "code", 257 | "execution_count": 595, 258 | "id": "47f2b306", 259 | "metadata": {}, 260 | "outputs": [], 261 | "source": [ 262 | "def prediction(x_test, x_train, num_neighbors):\n", 263 | " classes = []\n", 264 | " neighbors = get_neighbors(x_test, x_train, num_neighbors)\n", 265 | " for i in neighbors:\n", 266 | " classes.append(i[-1])\n", 267 | " predicted = max(classes, key=classes.count) #taking the most repeated class\n", 268 | " return predicted" 269 | ] 270 | }, 271 | { 272 | "cell_type": "markdown", 273 | "id": "e746c8f0", 274 | "metadata": {}, 275 | "source": [ 276 | "### Measuring the accuracy. So that we can know how accurate our model would predict new data samples" 277 | ] 278 | }, 279 | { 280 | "cell_type": "code", 281 | "execution_count": 596, 282 | "id": "79514b57", 283 | "metadata": {}, 284 | "outputs": [], 285 | "source": [ 286 | "def accuracy(y_true, y_pred):\n", 287 | " num_correct = 0\n", 288 | " for i in range(len(y_true)):\n", 289 | " if y_true[i]==y_pred[i]:\n", 290 | " num_correct+=1\n", 291 | " accuracy = num_correct/len(y_true)\n", 292 | " return accuracy" 293 | ] 294 | }, 295 | { 296 | "cell_type": "markdown", 297 | "id": "a234695f", 298 | "metadata": {}, 299 | "source": [ 300 | "### Predicting test data" 301 | ] 302 | }, 303 | { 304 | "cell_type": "code", 305 | "execution_count": 597, 306 | "id": "9f2a120a", 307 | "metadata": {}, 308 | "outputs": [], 309 | "source": [ 310 | "y_pred = []\n", 311 | "for i in test:\n", 312 | " y_pred.append(prediction(i, train, 4))\n", 313 | "#y_pred" 314 | ] 315 | }, 316 | { 317 | "cell_type": "code", 318 | "execution_count": 598, 319 | "id": "0540ffa2", 320 | "metadata": {}, 321 | "outputs": [], 322 | "source": [ 323 | "accuracy = accuracy(y_true, y_pred)" 324 | ] 325 | }, 326 | { 327 | "cell_type": "markdown", 328 | "id": "b1dbb18a", 329 | "metadata": {}, 330 | "source": [ 331 | "### Accuracy" 332 | ] 333 | }, 334 | { 335 | "cell_type": "code", 336 | "execution_count": 599, 337 | "id": "dc33f30b", 338 | "metadata": {}, 339 | "outputs": [ 340 | { 341 | "data": { 342 | "text/plain": [ 343 | "89.375" 344 | ] 345 | }, 346 | "execution_count": 599, 347 | "metadata": {}, 348 | "output_type": "execute_result" 349 | } 350 | ], 351 | "source": [ 352 | "accuracy*100" 353 | ] 354 | }, 355 | { 356 | "cell_type": "markdown", 357 | "id": "a1cb4888", 358 | "metadata": {}, 359 | "source": [ 360 | "### Sample Output" 361 | ] 362 | }, 363 | { 364 | "cell_type": "code", 365 | "execution_count": 601, 366 | "id": "dd8a5169", 367 | "metadata": {}, 368 | "outputs": [ 369 | { 370 | "data": { 371 | "text/html": [ 372 | "
\n", 373 | "\n", 386 | "\n", 387 | " \n", 388 | " \n", 389 | " \n", 390 | " \n", 391 | " \n", 392 | " \n", 393 | " \n", 394 | " \n", 395 | " \n", 396 | " \n", 397 | " \n", 398 | " \n", 399 | " \n", 400 | " \n", 401 | " \n", 402 | " \n", 403 | " \n", 404 | " \n", 405 | " \n", 406 | " \n", 407 | " \n", 408 | " \n", 409 | " \n", 410 | " \n", 411 | " \n", 412 | " \n", 413 | " \n", 414 | " \n", 415 | " \n", 416 | " \n", 417 | " \n", 418 | " \n", 419 | " \n", 420 | " \n", 421 | " \n", 422 | " \n", 423 | " \n", 424 | " \n", 425 | " \n", 426 | " \n", 427 | " \n", 428 | " \n", 429 | " \n", 430 | " \n", 431 | " \n", 432 | " \n", 433 | "
GenderAgeSalaryPurchase Iphone
267037740000
368038710000
3440471050001
338138550000
277049880001
\n", 434 | "
" 435 | ], 436 | "text/plain": [ 437 | " Gender Age Salary Purchase Iphone\n", 438 | "267 0 37 74000 0\n", 439 | "368 0 38 71000 0\n", 440 | "344 0 47 105000 1\n", 441 | "338 1 38 55000 0\n", 442 | "277 0 49 88000 1" 443 | ] 444 | }, 445 | "execution_count": 601, 446 | "metadata": {}, 447 | "output_type": "execute_result" 448 | } 449 | ], 450 | "source": [ 451 | "test_df.sample(5)" 452 | ] 453 | }, 454 | { 455 | "cell_type": "code", 456 | "execution_count": null, 457 | "id": "31c8e63d", 458 | "metadata": {}, 459 | "outputs": [], 460 | "source": [] 461 | } 462 | ], 463 | "metadata": { 464 | "kernelspec": { 465 | "display_name": "Python 3 (ipykernel)", 466 | "language": "python", 467 | "name": "python3" 468 | }, 469 | "language_info": { 470 | "codemirror_mode": { 471 | "name": "ipython", 472 | "version": 3 473 | }, 474 | "file_extension": ".py", 475 | "mimetype": "text/x-python", 476 | "name": "python", 477 | "nbconvert_exporter": "python", 478 | "pygments_lexer": "ipython3", 479 | "version": "3.9.12" 480 | } 481 | }, 482 | "nbformat": 4, 483 | "nbformat_minor": 5 484 | } 485 | -------------------------------------------------------------------------------- /traditional_ML_algorithms/KNN/readme.md: -------------------------------------------------------------------------------- 1 | Name: K-Nearest Neighbors 2 | 3 | Packages used:
4 | `Numpy`:1.21.5
5 | `Pandas`:1.4.2
6 | `sklearn`:1.0.2
7 | 8 | #### 📌 The k-nearest neighbors algorithm, also known as KNN or k-NN, is a non-parametric, supervised learning classifier, which uses proximity to make classifications or predictions about the grouping of an individual data point. 9 | 10 | #### 📌 Goal of this project is to predict if the customer will purchase an iPhone or not given their gender, age and salary. 11 | 12 | -------------------------------------------------------------------------------- /traditional_ML_algorithms/LogisticRegression/ChurnData.csv: -------------------------------------------------------------------------------- 1 | tenure,age,address,income,ed,employ,equip,callcard,wireless,longmon,tollmon,equipmon,cardmon,wiremon,longten,tollten,cardten,voice,pager,internet,callwait,confer,ebill,loglong,logtoll,lninc,custcat,churn 2 | 11.000,33.000,7.000,136.000,5.000,5.000,0.000,1.000,1.000,4.400,20.750,0.000,15.250,35.700,42.000,211.450,125.000,1.000,1.000,0.000,1.000,1.000,0.000,1.482,3.033,4.913,4.000,1.000 3 | 33.000,33.000,12.000,33.000,2.000,0.000,0.000,0.000,0.000,9.450,0.000,0.000,0.000,0.000,288.800,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,2.246,3.240,3.497,1.000,1.000 4 | 23.000,30.000,9.000,30.000,1.000,2.000,0.000,0.000,0.000,6.300,0.000,0.000,0.000,0.000,157.050,0.000,0.000,0.000,0.000,0.000,0.000,1.000,0.000,1.841,3.240,3.401,3.000,0.000 5 | 38.000,35.000,5.000,76.000,2.000,10.000,1.000,1.000,1.000,6.050,45.000,50.100,23.250,64.900,239.550,1873.050,880.000,1.000,1.000,1.000,1.000,1.000,1.000,1.800,3.807,4.331,4.000,0.000 6 | 7.000,35.000,14.000,80.000,2.000,15.000,0.000,1.000,0.000,7.100,22.000,0.000,23.750,0.000,47.450,166.100,145.000,1.000,0.000,0.000,1.000,1.000,0.000,1.960,3.091,4.382,3.000,0.000 7 | 68.000,52.000,17.000,120.000,1.000,24.000,0.000,1.000,0.000,20.700,0.000,0.000,22.000,0.000,1391.050,0.000,1505.000,0.000,0.000,0.000,0.000,0.000,0.000,3.030,3.240,4.787,1.000,0.000 8 | 42.000,40.000,7.000,37.000,2.000,8.000,1.000,1.000,1.000,8.250,23.500,36.900,28.000,37.400,399.150,950.650,1190.000,1.000,0.000,1.000,1.000,1.000,1.000,2.110,3.157,3.611,4.000,0.000 9 | 9.000,21.000,1.000,17.000,2.000,2.000,0.000,0.000,0.000,2.900,0.000,0.000,0.000,0.000,25.250,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,1.065,3.240,2.833,1.000,0.000 10 | 35.000,50.000,26.000,140.000,2.000,21.000,0.000,1.000,0.000,6.500,27.500,0.000,35.000,0.000,247.550,1068.250,1215.000,0.000,0.000,0.000,1.000,1.000,0.000,1.872,3.314,4.942,3.000,0.000 11 | 49.000,51.000,27.000,63.000,4.000,19.000,0.000,1.000,0.000,12.850,25.750,0.000,14.250,0.000,585.600,1278.450,635.000,0.000,0.000,1.000,1.000,0.000,1.000,2.553,3.248,4.143,2.000,0.000 12 | 56.000,52.000,28.000,49.000,2.000,12.000,0.000,1.000,0.000,24.750,0.000,0.000,22.250,0.000,1349.050,0.000,1215.000,0.000,0.000,0.000,0.000,0.000,0.000,3.209,3.240,3.892,2.000,0.000 13 | 47.000,40.000,16.000,127.000,4.000,12.000,1.000,1.000,0.000,19.700,0.000,28.150,14.750,0.000,909.900,0.000,680.000,0.000,0.000,1.000,0.000,0.000,1.000,2.981,3.240,4.844,2.000,0.000 14 | 56.000,50.000,1.000,80.000,2.000,24.000,0.000,1.000,1.000,28.800,55.500,0.000,21.250,61.100,1558.100,3119.450,1065.000,1.000,1.000,0.000,1.000,1.000,0.000,3.360,4.016,4.382,4.000,0.000 15 | 69.000,51.000,11.000,438.000,4.000,23.000,1.000,1.000,0.000,29.000,0.000,47.050,24.750,0.000,1815.400,0.000,1655.000,0.000,1.000,1.000,0.000,1.000,0.000,3.367,3.240,6.082,4.000,0.000 16 | 16.000,27.000,5.000,37.000,3.000,5.000,0.000,0.000,0.000,6.000,0.000,0.000,0.000,0.000,80.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,1.792,3.240,3.611,1.000,0.000 17 | 4.000,35.000,16.000,161.000,5.000,6.000,1.000,0.000,1.000,3.400,23.750,49.800,0.000,35.750,10.350,103.100,0.000,1.000,1.000,1.000,1.000,1.000,1.000,1.224,3.168,5.081,4.000,1.000 18 | 27.000,51.000,3.000,80.000,5.000,11.000,1.000,0.000,0.000,7.100,0.000,39.400,0.000,0.000,176.200,0.000,0.000,0.000,1.000,1.000,0.000,0.000,1.000,1.960,3.240,4.382,2.000,0.000 19 | 52.000,61.000,3.000,53.000,5.000,1.000,1.000,1.000,1.000,12.250,0.000,38.400,8.750,35.950,631.700,0.000,465.000,1.000,0.000,1.000,0.000,1.000,1.000,2.506,3.240,3.970,2.000,0.000 20 | 64.000,25.000,4.000,76.000,3.000,2.000,1.000,1.000,0.000,24.050,0.000,35.400,21.250,0.000,1536.550,0.000,1400.000,0.000,0.000,0.000,0.000,1.000,1.000,3.180,3.240,4.331,3.000,0.000 21 | 12.000,24.000,2.000,19.000,1.000,0.000,0.000,1.000,0.000,4.000,24.750,0.000,27.250,0.000,46.000,299.000,295.000,0.000,0.000,0.000,1.000,1.000,0.000,1.386,3.209,2.944,3.000,1.000 22 | 35.000,61.000,23.000,41.000,2.000,11.000,0.000,1.000,0.000,9.600,0.000,0.000,9.500,0.000,353.550,0.000,295.000,0.000,0.000,0.000,0.000,0.000,0.000,2.262,3.240,3.714,1.000,0.000 23 | 13.000,54.000,2.000,31.000,4.000,2.000,0.000,0.000,0.000,5.850,0.000,0.000,0.000,0.000,97.000,0.000,0.000,1.000,0.000,1.000,0.000,0.000,1.000,1.766,3.240,3.434,1.000,0.000 24 | 45.000,22.000,2.000,36.000,4.000,0.000,1.000,0.000,0.000,9.950,14.750,26.150,0.000,0.000,412.100,663.100,0.000,0.000,0.000,0.000,0.000,0.000,0.000,2.298,2.691,3.584,2.000,1.000 25 | 3.000,37.000,13.000,24.000,1.000,3.000,0.000,0.000,0.000,2.000,0.000,0.000,0.000,0.000,3.050,0.000,0.000,1.000,0.000,0.000,0.000,1.000,1.000,0.693,3.240,3.178,1.000,0.000 26 | 53.000,22.000,1.000,25.000,4.000,0.000,1.000,1.000,0.000,12.050,0.000,27.150,6.500,0.000,666.000,0.000,335.000,0.000,0.000,1.000,0.000,0.000,1.000,2.489,3.240,3.219,2.000,0.000 27 | 17.000,42.000,6.000,131.000,5.000,6.000,1.000,0.000,1.000,5.800,0.000,35.350,0.000,21.650,103.100,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.000,1.758,3.240,4.875,2.000,1.000 28 | 59.000,43.000,4.000,101.000,2.000,22.000,0.000,1.000,0.000,13.650,0.000,0.000,20.750,0.000,817.650,0.000,1195.000,0.000,0.000,0.000,0.000,1.000,0.000,2.614,3.240,4.615,2.000,0.000 29 | 57.000,37.000,11.000,108.000,4.000,9.000,1.000,1.000,0.000,21.800,27.250,42.000,8.000,0.000,1292.000,1492.100,425.000,0.000,0.000,1.000,1.000,1.000,0.000,3.082,3.305,4.682,3.000,0.000 30 | 3.000,24.000,2.000,20.000,2.000,3.000,0.000,1.000,0.000,3.350,22.500,0.000,9.250,0.000,7.550,49.550,15.000,1.000,0.000,1.000,1.000,1.000,0.000,1.209,3.114,2.996,3.000,0.000 31 | 4.000,47.000,5.000,123.000,4.000,11.000,1.000,1.000,0.000,2.500,0.000,25.300,13.750,0.000,9.250,0.000,40.000,0.000,0.000,1.000,0.000,0.000,1.000,0.916,3.240,4.812,1.000,0.000 32 | 29.000,26.000,7.000,34.000,1.000,7.000,0.000,1.000,0.000,6.500,0.000,0.000,16.000,0.000,198.700,0.000,490.000,0.000,0.000,0.000,1.000,0.000,0.000,1.872,3.240,3.526,3.000,0.000 33 | 64.000,55.000,28.000,104.000,1.000,26.000,0.000,1.000,0.000,15.000,0.000,0.000,39.250,0.000,960.950,0.000,2360.000,0.000,0.000,0.000,0.000,1.000,0.000,2.708,3.240,4.644,3.000,0.000 34 | 22.000,34.000,1.000,46.000,3.000,1.000,1.000,0.000,0.000,7.100,0.000,23.400,0.000,0.000,125.050,0.000,0.000,0.000,0.000,1.000,0.000,0.000,0.000,1.960,3.240,3.829,2.000,0.000 35 | 33.000,54.000,18.000,57.000,4.000,4.000,1.000,0.000,0.000,7.500,30.250,33.900,0.000,0.000,226.250,1105.750,0.000,0.000,0.000,1.000,1.000,0.000,0.000,2.015,3.409,4.043,3.000,1.000 36 | 18.000,69.000,11.000,58.000,3.000,8.000,1.000,1.000,1.000,6.350,27.250,53.950,11.750,63.000,118.050,454.050,175.000,1.000,1.000,1.000,1.000,1.000,1.000,1.848,3.305,4.060,4.000,0.000 37 | 65.000,65.000,27.000,128.000,3.000,24.000,0.000,1.000,0.000,21.200,24.000,0.000,14.500,0.000,1325.050,1483.550,940.000,0.000,0.000,0.000,1.000,1.000,0.000,3.054,3.178,4.852,3.000,0.000 38 | 39.000,24.000,2.000,26.000,2.000,4.000,0.000,1.000,0.000,10.400,0.000,0.000,12.500,0.000,403.500,0.000,480.000,0.000,0.000,0.000,0.000,0.000,1.000,2.342,3.240,3.258,1.000,1.000 39 | 28.000,29.000,4.000,23.000,3.000,5.000,0.000,0.000,0.000,3.700,0.000,0.000,0.000,0.000,91.950,0.000,0.000,0.000,0.000,1.000,0.000,1.000,1.000,1.308,3.240,3.135,2.000,0.000 40 | 46.000,42.000,9.000,52.000,4.000,7.000,0.000,1.000,0.000,14.250,0.000,0.000,21.000,0.000,611.650,0.000,985.000,0.000,0.000,0.000,0.000,0.000,0.000,2.657,3.240,3.951,2.000,0.000 41 | 43.000,43.000,3.000,55.000,4.000,18.000,1.000,1.000,1.000,9.100,40.750,48.400,16.000,50.600,422.700,1802.500,675.000,1.000,1.000,1.000,1.000,1.000,0.000,2.208,3.707,4.007,4.000,0.000 42 | 21.000,29.000,7.000,40.000,4.000,2.000,1.000,1.000,1.000,4.500,17.000,36.200,19.250,33.400,89.100,386.800,395.000,0.000,0.000,1.000,1.000,0.000,0.000,1.504,2.833,3.689,1.000,0.000 43 | 53.000,57.000,25.000,37.000,1.000,7.000,0.000,1.000,0.000,7.700,0.000,0.000,9.250,0.000,363.950,0.000,465.000,0.000,0.000,0.000,0.000,0.000,0.000,2.041,3.240,3.611,2.000,0.000 44 | 50.000,52.000,17.000,36.000,4.000,16.000,0.000,1.000,1.000,11.950,55.000,0.000,48.000,85.850,616.250,2662.100,2330.000,1.000,1.000,0.000,1.000,1.000,1.000,2.481,4.007,3.584,4.000,0.000 45 | 43.000,21.000,1.000,25.000,1.000,4.000,0.000,0.000,0.000,7.600,0.000,0.000,0.000,0.000,334.450,0.000,0.000,0.000,0.000,1.000,0.000,1.000,0.000,2.028,3.240,3.219,2.000,1.000 46 | 33.000,33.000,12.000,42.000,4.000,7.000,1.000,1.000,1.000,8.800,0.000,30.950,23.000,23.350,305.750,0.000,730.000,1.000,0.000,0.000,0.000,0.000,0.000,2.175,3.240,3.738,2.000,0.000 47 | 45.000,66.000,43.000,144.000,2.000,13.000,0.000,1.000,0.000,7.750,0.000,0.000,13.000,0.000,338.800,0.000,565.000,0.000,0.000,1.000,0.000,0.000,0.000,2.048,3.240,4.970,2.000,0.000 48 | 28.000,57.000,33.000,82.000,4.000,22.000,1.000,0.000,0.000,4.050,0.000,31.450,0.000,0.000,71.500,0.000,0.000,1.000,0.000,1.000,0.000,0.000,1.000,1.399,3.240,4.407,2.000,1.000 49 | 37.000,33.000,1.000,102.000,2.000,12.000,1.000,1.000,1.000,16.300,0.000,33.350,10.250,37.900,646.700,0.000,360.000,0.000,1.000,1.000,1.000,0.000,1.000,2.791,3.240,4.625,4.000,0.000 50 | 71.000,56.000,23.000,170.000,1.000,30.000,0.000,1.000,0.000,14.200,27.000,0.000,30.000,0.000,1001.200,1840.650,2140.000,1.000,1.000,0.000,1.000,1.000,0.000,2.653,3.296,5.136,4.000,0.000 51 | 58.000,58.000,10.000,96.000,2.000,17.000,0.000,0.000,0.000,12.500,23.000,0.000,0.000,0.000,710.800,1345.950,0.000,0.000,0.000,0.000,1.000,1.000,0.000,2.526,3.135,4.564,3.000,0.000 52 | 36.000,58.000,34.000,80.000,1.000,21.000,0.000,1.000,0.000,8.500,0.000,0.000,8.500,0.000,298.250,0.000,260.000,0.000,0.000,0.000,1.000,0.000,0.000,2.140,3.240,4.382,2.000,0.000 53 | 26.000,39.000,15.000,58.000,4.000,9.000,0.000,1.000,1.000,8.650,21.250,0.000,6.500,19.650,210.600,567.550,145.000,0.000,0.000,1.000,0.000,1.000,0.000,2.158,3.056,4.060,3.000,0.000 54 | 13.000,43.000,1.000,123.000,3.000,21.000,1.000,1.000,0.000,3.850,23.000,33.050,6.750,0.000,55.050,317.500,75.000,0.000,1.000,1.000,1.000,0.000,1.000,1.348,3.135,4.812,4.000,0.000 55 | 28.000,40.000,7.000,64.000,1.000,19.000,0.000,0.000,0.000,6.000,27.500,0.000,0.000,0.000,167.650,722.500,0.000,0.000,0.000,0.000,0.000,1.000,0.000,1.792,3.314,4.159,3.000,0.000 56 | 1.000,21.000,1.000,18.000,3.000,0.000,1.000,0.000,0.000,2.700,0.000,32.700,0.000,0.000,2.700,0.000,0.000,0.000,0.000,1.000,0.000,1.000,1.000,0.993,3.240,2.890,1.000,1.000 57 | 15.000,35.000,5.000,34.000,3.000,8.000,1.000,0.000,0.000,5.150,0.000,25.550,0.000,0.000,110.550,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.000,1.639,3.240,3.526,2.000,1.000 58 | 12.000,64.000,13.000,9.000,2.000,6.000,1.000,1.000,0.000,3.800,30.000,32.800,8.500,0.000,52.850,317.750,95.000,0.000,0.000,1.000,1.000,1.000,1.000,1.335,3.401,2.197,3.000,0.000 59 | 5.000,30.000,3.000,46.000,4.000,7.000,1.000,0.000,0.000,4.300,0.000,32.100,0.000,0.000,17.550,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.000,1.459,3.240,3.829,2.000,1.000 60 | 32.000,27.000,3.000,91.000,4.000,1.000,1.000,1.000,1.000,5.500,16.000,42.650,3.750,41.300,189.050,543.150,100.000,1.000,1.000,1.000,0.000,1.000,1.000,1.705,2.773,4.511,4.000,1.000 61 | 14.000,36.000,13.000,67.000,5.000,4.000,1.000,0.000,0.000,14.050,0.000,25.100,0.000,0.000,187.400,0.000,0.000,0.000,0.000,1.000,1.000,0.000,0.000,2.643,3.240,4.205,1.000,0.000 62 | 67.000,60.000,32.000,93.000,1.000,21.000,1.000,1.000,0.000,18.850,0.000,32.000,42.750,0.000,1200.700,0.000,2665.000,0.000,0.000,0.000,0.000,0.000,1.000,2.937,3.240,4.533,2.000,0.000 63 | 60.000,45.000,23.000,117.000,4.000,11.000,1.000,1.000,1.000,12.400,28.000,51.700,19.500,52.900,755.700,1721.250,1135.000,1.000,1.000,1.000,1.000,1.000,1.000,2.518,3.332,4.762,4.000,0.000 64 | 70.000,55.000,12.000,65.000,3.000,24.000,0.000,1.000,0.000,26.700,42.750,0.000,20.500,0.000,1874.250,2921.350,1465.000,1.000,0.000,0.000,1.000,1.000,0.000,3.285,3.755,4.174,3.000,0.000 65 | 6.000,24.000,2.000,28.000,1.000,5.000,0.000,1.000,0.000,4.400,9.000,0.000,28.250,0.000,35.100,57.800,155.000,0.000,0.000,0.000,1.000,1.000,0.000,1.482,2.197,3.332,3.000,1.000 66 | 38.000,33.000,3.000,38.000,4.000,0.000,1.000,1.000,0.000,10.950,0.000,27.200,12.250,0.000,412.800,0.000,465.000,0.000,0.000,1.000,0.000,0.000,1.000,2.393,3.240,3.638,2.000,0.000 67 | 53.000,35.000,15.000,59.000,3.000,5.000,0.000,1.000,0.000,16.850,23.250,0.000,18.500,0.000,888.450,1250.700,995.000,0.000,1.000,0.000,1.000,0.000,0.000,2.824,3.146,4.078,3.000,0.000 68 | 37.000,76.000,38.000,117.000,4.000,21.000,0.000,1.000,0.000,7.500,0.000,0.000,6.250,0.000,273.700,0.000,230.000,0.000,0.000,0.000,0.000,0.000,0.000,2.015,3.240,4.762,1.000,0.000 69 | 61.000,45.000,21.000,80.000,2.000,13.000,0.000,1.000,0.000,18.100,0.000,0.000,35.250,0.000,1114.750,0.000,2200.000,0.000,0.000,0.000,0.000,0.000,0.000,2.896,3.240,4.382,2.000,0.000 70 | 20.000,31.000,10.000,39.000,2.000,3.000,0.000,0.000,0.000,7.750,0.000,0.000,0.000,0.000,148.750,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,2.048,3.240,3.664,1.000,0.000 71 | 72.000,60.000,33.000,12.000,1.000,20.000,0.000,1.000,1.000,53.750,68.500,0.000,10.250,43.950,3776.100,4938.600,780.000,1.000,1.000,0.000,1.000,1.000,0.000,3.984,4.227,2.485,4.000,0.000 72 | 61.000,43.000,6.000,34.000,5.000,6.000,1.000,1.000,1.000,25.050,39.000,56.800,32.750,51.000,1541.900,2536.100,1980.000,1.000,1.000,1.000,1.000,1.000,1.000,3.221,3.664,3.526,4.000,1.000 73 | 54.000,27.000,3.000,27.000,2.000,6.000,1.000,1.000,1.000,12.400,35.000,48.300,5.750,43.500,660.850,1919.750,315.000,1.000,1.000,1.000,1.000,1.000,1.000,2.518,3.555,3.296,4.000,0.000 74 | 28.000,36.000,3.000,42.000,3.000,7.000,0.000,1.000,0.000,12.900,28.250,0.000,14.000,0.000,363.650,820.150,370.000,0.000,0.000,1.000,1.000,1.000,0.000,2.557,3.341,3.738,3.000,0.000 75 | 52.000,42.000,17.000,27.000,3.000,8.000,0.000,1.000,0.000,19.650,31.000,0.000,19.750,0.000,1008.800,1661.300,1050.000,0.000,0.000,1.000,1.000,1.000,0.000,2.978,3.434,3.296,3.000,0.000 76 | 43.000,27.000,3.000,21.000,2.000,1.000,0.000,1.000,0.000,12.650,0.000,0.000,16.500,0.000,558.850,0.000,760.000,0.000,0.000,0.000,0.000,0.000,0.000,2.538,3.240,3.045,2.000,0.000 77 | 46.000,45.000,12.000,96.000,3.000,17.000,1.000,1.000,0.000,10.950,0.000,24.500,9.750,0.000,504.300,0.000,410.000,0.000,0.000,1.000,0.000,0.000,1.000,2.393,3.240,4.564,2.000,0.000 78 | 39.000,59.000,20.000,1668.000,4.000,27.000,0.000,0.000,0.000,8.200,15.000,0.000,0.000,0.000,303.950,595.600,0.000,1.000,0.000,1.000,1.000,0.000,0.000,2.104,2.708,7.419,2.000,0.000 79 | 10.000,34.000,1.000,52.000,5.000,3.000,1.000,0.000,1.000,4.300,0.000,32.550,0.000,25.400,47.450,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.000,1.459,3.240,3.951,2.000,1.000 80 | 69.000,46.000,18.000,66.000,2.000,19.000,0.000,1.000,0.000,21.850,0.000,0.000,46.500,0.000,1589.100,0.000,3265.000,0.000,0.000,0.000,0.000,0.000,0.000,3.084,3.240,4.190,2.000,0.000 81 | 45.000,30.000,0.000,63.000,5.000,4.000,1.000,1.000,1.000,7.100,40.750,39.750,16.500,50.850,314.800,1849.500,775.000,1.000,1.000,1.000,1.000,1.000,1.000,1.960,3.707,4.143,4.000,1.000 82 | 52.000,62.000,23.000,36.000,4.000,17.000,0.000,1.000,0.000,13.350,18.500,0.000,18.500,0.000,709.250,978.850,915.000,0.000,0.000,0.000,0.000,1.000,0.000,2.592,2.918,3.584,1.000,0.000 83 | 42.000,36.000,14.000,44.000,2.000,11.000,0.000,0.000,1.000,7.000,26.500,0.000,0.000,26.100,321.950,1161.300,0.000,0.000,1.000,0.000,1.000,1.000,0.000,1.946,3.277,3.784,3.000,0.000 84 | 5.000,44.000,5.000,83.000,1.000,16.000,1.000,0.000,0.000,4.650,0.000,25.250,0.000,0.000,12.300,0.000,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.537,3.240,4.419,2.000,1.000 85 | 10.000,33.000,2.000,66.000,3.000,9.000,0.000,1.000,0.000,7.100,21.250,0.000,25.250,0.000,59.300,241.800,220.000,0.000,0.000,1.000,0.000,0.000,0.000,1.960,3.056,4.190,1.000,1.000 86 | 26.000,30.000,9.000,18.000,4.000,1.000,0.000,1.000,0.000,7.250,25.500,0.000,32.500,0.000,171.850,688.350,840.000,1.000,1.000,0.000,1.000,1.000,0.000,1.981,3.239,2.890,4.000,1.000 87 | 25.000,30.000,0.000,20.000,1.000,4.000,0.000,1.000,0.000,8.550,21.750,0.000,15.500,0.000,215.950,522.150,370.000,0.000,0.000,0.000,1.000,1.000,0.000,2.146,3.080,2.996,3.000,0.000 88 | 30.000,23.000,4.000,19.000,3.000,1.000,1.000,1.000,0.000,7.900,0.000,35.200,27.500,0.000,253.050,0.000,840.000,0.000,0.000,1.000,0.000,0.000,1.000,2.067,3.240,2.944,2.000,0.000 89 | 65.000,59.000,27.000,197.000,4.000,26.000,1.000,1.000,0.000,45.400,0.000,25.750,46.750,0.000,2973.550,0.000,3065.000,0.000,1.000,0.000,0.000,0.000,1.000,3.816,3.240,5.283,2.000,0.000 90 | 5.000,32.000,6.000,33.000,4.000,3.000,1.000,0.000,0.000,3.200,0.000,21.600,0.000,0.000,24.550,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.000,1.163,3.240,3.497,1.000,1.000 91 | 7.000,23.000,3.000,27.000,2.000,1.000,0.000,1.000,0.000,3.650,9.500,0.000,7.000,0.000,21.000,76.050,55.000,0.000,0.000,0.000,1.000,0.000,0.000,1.295,2.251,3.296,1.000,0.000 92 | 25.000,29.000,9.000,55.000,4.000,1.000,0.000,0.000,0.000,5.250,24.750,0.000,0.000,0.000,147.950,612.450,0.000,0.000,0.000,0.000,0.000,0.000,1.000,1.658,3.209,4.007,1.000,1.000 93 | 8.000,22.000,3.000,25.000,4.000,0.000,0.000,1.000,1.000,8.000,32.250,0.000,18.250,19.450,61.550,264.000,155.000,1.000,0.000,0.000,1.000,1.000,0.000,2.079,3.474,3.219,3.000,0.000 94 | 39.000,47.000,1.000,68.000,4.000,10.000,0.000,0.000,0.000,8.300,0.000,0.000,0.000,0.000,308.550,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.000,2.116,3.240,4.220,2.000,1.000 95 | 28.000,36.000,3.000,69.000,3.000,2.000,1.000,1.000,1.000,6.050,25.250,41.550,31.750,36.600,153.050,697.050,835.000,1.000,1.000,0.000,1.000,1.000,1.000,1.800,3.229,4.234,4.000,0.000 96 | 55.000,52.000,22.000,127.000,1.000,28.000,0.000,1.000,0.000,7.050,0.000,0.000,20.250,0.000,400.550,0.000,1150.000,1.000,0.000,0.000,0.000,0.000,1.000,1.953,3.240,4.844,2.000,0.000 97 | 11.000,63.000,9.000,41.000,3.000,3.000,1.000,0.000,0.000,4.150,0.000,29.900,0.000,0.000,46.350,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.000,1.423,3.240,3.714,1.000,1.000 98 | 51.000,48.000,27.000,58.000,1.000,18.000,0.000,1.000,0.000,19.200,25.750,0.000,9.250,0.000,964.850,1320.050,390.000,0.000,0.000,0.000,0.000,1.000,0.000,2.955,3.248,4.060,3.000,1.000 99 | 25.000,62.000,27.000,28.000,4.000,33.000,1.000,1.000,1.000,8.300,14.750,52.000,18.000,47.000,160.400,359.250,430.000,1.000,1.000,0.000,1.000,1.000,1.000,2.116,2.691,3.332,4.000,1.000 100 | 62.000,76.000,20.000,35.000,3.000,18.000,0.000,1.000,0.000,17.250,0.000,0.000,17.750,0.000,1045.700,0.000,1085.000,0.000,0.000,0.000,0.000,0.000,1.000,2.848,3.240,3.555,2.000,0.000 101 | 53.000,33.000,1.000,60.000,1.000,6.000,0.000,1.000,1.000,17.650,28.500,0.000,41.000,28.550,950.700,1502.200,2170.000,1.000,1.000,0.000,1.000,1.000,0.000,2.871,3.350,4.094,4.000,0.000 102 | 1.000,30.000,3.000,135.000,4.000,3.000,1.000,1.000,0.000,1.100,5.750,22.800,2.750,0.000,1.100,5.750,2.750,0.000,0.000,1.000,0.000,0.000,1.000,0.095,1.749,4.905,1.000,1.000 103 | 20.000,32.000,10.000,19.000,3.000,5.000,1.000,0.000,0.000,6.850,0.000,36.050,0.000,0.000,106.500,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.000,1.924,3.240,2.944,2.000,0.000 104 | 34.000,63.000,10.000,23.000,2.000,0.000,1.000,1.000,1.000,14.300,19.500,39.600,15.250,28.950,459.050,616.500,495.000,1.000,1.000,1.000,1.000,1.000,1.000,2.660,2.970,3.135,4.000,0.000 105 | 44.000,45.000,19.000,88.000,1.000,21.000,0.000,1.000,0.000,11.900,25.250,0.000,8.250,0.000,524.250,1108.400,305.000,0.000,0.000,0.000,1.000,1.000,0.000,2.477,3.229,4.477,3.000,0.000 106 | 12.000,23.000,2.000,24.000,4.000,0.000,0.000,1.000,1.000,3.200,41.000,0.000,15.750,35.400,35.500,502.000,210.000,1.000,1.000,1.000,1.000,1.000,0.000,1.163,3.714,3.178,4.000,0.000 107 | 39.000,34.000,4.000,20.000,5.000,3.000,1.000,1.000,1.000,9.550,21.250,59.200,10.000,79.200,375.400,872.500,385.000,1.000,1.000,1.000,1.000,1.000,1.000,2.257,3.056,2.996,4.000,0.000 108 | 19.000,26.000,2.000,48.000,3.000,0.000,0.000,1.000,0.000,7.550,0.000,0.000,6.500,0.000,156.150,0.000,105.000,1.000,0.000,0.000,1.000,0.000,0.000,2.022,3.240,3.871,3.000,0.000 109 | 7.000,27.000,3.000,39.000,3.000,2.000,0.000,0.000,0.000,3.550,0.000,0.000,0.000,0.000,19.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,1.267,3.240,3.664,1.000,1.000 110 | 32.000,34.000,0.000,38.000,1.000,10.000,0.000,1.000,0.000,7.350,0.000,0.000,19.250,0.000,217.450,0.000,600.000,0.000,0.000,0.000,0.000,0.000,1.000,1.995,3.240,3.638,2.000,0.000 111 | 35.000,34.000,7.000,78.000,4.000,10.000,0.000,1.000,0.000,13.900,0.000,0.000,4.500,0.000,556.200,0.000,135.000,0.000,0.000,0.000,0.000,0.000,0.000,2.632,3.240,4.357,1.000,0.000 112 | 56.000,42.000,10.000,24.000,2.000,5.000,0.000,0.000,0.000,33.650,0.000,0.000,0.000,0.000,1871.200,0.000,0.000,0.000,0.000,0.000,1.000,1.000,0.000,3.516,3.240,3.178,3.000,0.000 113 | 5.000,47.000,7.000,46.000,1.000,6.000,0.000,1.000,0.000,2.950,0.000,0.000,7.250,0.000,21.550,0.000,35.000,0.000,0.000,0.000,0.000,0.000,0.000,1.082,3.240,3.829,1.000,1.000 114 | 16.000,50.000,5.000,263.000,2.000,29.000,1.000,0.000,1.000,3.750,17.000,31.950,0.000,21.900,59.750,253.350,0.000,0.000,0.000,0.000,1.000,1.000,1.000,1.322,2.833,5.572,3.000,0.000 115 | 17.000,42.000,6.000,31.000,2.000,2.000,0.000,0.000,1.000,8.250,24.500,0.000,0.000,26.400,140.850,404.800,0.000,1.000,0.000,0.000,1.000,1.000,1.000,2.110,3.199,3.434,4.000,1.000 116 | 9.000,24.000,3.000,26.000,4.000,1.000,1.000,0.000,0.000,7.650,0.000,23.250,0.000,0.000,75.250,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,2.035,3.240,3.258,1.000,0.000 117 | 9.000,41.000,12.000,39.000,4.000,3.000,1.000,0.000,1.000,4.700,0.000,39.650,0.000,20.800,26.300,0.000,0.000,1.000,1.000,1.000,0.000,0.000,1.000,1.548,3.240,3.664,1.000,1.000 118 | 71.000,41.000,10.000,73.000,2.000,23.000,0.000,1.000,0.000,32.650,0.000,0.000,41.750,0.000,2412.600,0.000,3085.000,0.000,0.000,0.000,1.000,0.000,0.000,3.486,3.240,4.290,3.000,0.000 119 | 11.000,26.000,2.000,53.000,3.000,3.000,1.000,1.000,1.000,4.150,21.500,43.700,11.750,38.800,34.700,188.350,110.000,1.000,1.000,1.000,1.000,0.000,1.000,1.423,3.068,3.970,4.000,1.000 120 | 23.000,50.000,1.000,151.000,4.000,8.000,0.000,0.000,0.000,8.250,21.750,0.000,0.000,0.000,186.500,438.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,2.110,3.080,5.017,1.000,0.000 121 | 24.000,58.000,30.000,24.000,1.000,5.000,0.000,0.000,0.000,7.650,16.000,0.000,0.000,0.000,177.400,420.650,0.000,0.000,0.000,0.000,0.000,0.000,0.000,2.035,2.773,3.178,1.000,0.000 122 | 4.000,24.000,1.000,17.000,2.000,2.000,0.000,0.000,0.000,3.200,0.000,0.000,0.000,0.000,9.900,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,1.163,3.240,2.833,1.000,1.000 123 | 10.000,28.000,9.000,75.000,4.000,1.000,1.000,1.000,1.000,6.200,27.500,42.400,40.000,36.300,45.950,225.850,340.000,1.000,1.000,1.000,1.000,1.000,1.000,1.825,3.314,4.317,4.000,1.000 124 | 24.000,35.000,10.000,41.000,5.000,6.000,1.000,0.000,0.000,3.300,0.000,30.300,0.000,0.000,88.650,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.000,1.194,3.240,3.714,2.000,0.000 125 | 12.000,31.000,8.000,18.000,4.000,4.000,1.000,0.000,0.000,3.700,0.000,24.300,0.000,0.000,48.550,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.000,1.308,3.240,2.890,1.000,1.000 126 | 59.000,55.000,29.000,42.000,3.000,21.000,0.000,1.000,0.000,8.950,0.000,0.000,15.250,0.000,492.350,0.000,855.000,0.000,0.000,0.000,0.000,0.000,0.000,2.192,3.240,3.738,2.000,0.000 127 | 72.000,75.000,48.000,14.000,2.000,6.000,0.000,1.000,0.000,37.300,0.000,0.000,109.250,0.000,2686.250,0.000,7515.000,0.000,0.000,0.000,0.000,0.000,0.000,3.619,3.240,2.639,2.000,0.000 128 | 67.000,40.000,14.000,59.000,3.000,11.000,0.000,1.000,0.000,27.000,30.000,0.000,19.750,0.000,1722.500,1959.950,1245.000,0.000,0.000,0.000,1.000,1.000,0.000,3.296,3.401,4.078,3.000,0.000 129 | 10.000,40.000,6.000,22.000,3.000,6.000,0.000,0.000,0.000,5.050,0.000,0.000,0.000,0.000,59.950,0.000,0.000,0.000,0.000,1.000,0.000,0.000,0.000,1.619,3.240,3.091,1.000,1.000 130 | 30.000,28.000,1.000,20.000,1.000,8.000,0.000,0.000,0.000,12.400,0.000,0.000,0.000,0.000,369.450,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,2.518,3.240,2.996,1.000,1.000 131 | 24.000,46.000,12.000,43.000,2.000,6.000,0.000,1.000,1.000,12.050,48.250,0.000,33.000,28.150,307.750,1079.050,820.000,1.000,1.000,0.000,1.000,1.000,1.000,2.489,3.876,3.761,4.000,0.000 132 | 72.000,75.000,37.000,33.000,1.000,44.000,0.000,1.000,0.000,49.300,31.750,0.000,26.000,0.000,3417.400,2254.150,1835.000,0.000,0.000,0.000,0.000,1.000,0.000,3.898,3.458,3.497,2.000,0.000 133 | 26.000,43.000,23.000,51.000,5.000,4.000,1.000,0.000,0.000,7.900,0.000,21.950,0.000,0.000,267.600,0.000,0.000,0.000,0.000,1.000,0.000,0.000,0.000,2.067,3.240,3.932,2.000,1.000 134 | 36.000,45.000,22.000,117.000,3.000,15.000,0.000,0.000,0.000,10.500,0.000,0.000,0.000,0.000,429.100,0.000,0.000,0.000,0.000,0.000,0.000,0.000,1.000,2.351,3.240,4.762,2.000,0.000 135 | 16.000,54.000,20.000,147.000,1.000,29.000,0.000,1.000,0.000,6.950,23.250,0.000,6.500,0.000,98.500,357.950,90.000,0.000,0.000,0.000,1.000,1.000,0.000,1.939,3.146,4.990,3.000,0.000 136 | 54.000,42.000,0.000,55.000,4.000,2.000,1.000,1.000,1.000,14.300,55.500,62.150,19.250,83.700,695.200,3086.350,1050.000,1.000,1.000,1.000,1.000,1.000,1.000,2.660,4.016,4.007,4.000,1.000 137 | 72.000,62.000,35.000,163.000,5.000,31.000,1.000,1.000,1.000,28.350,32.750,45.150,39.750,56.550,2017.600,2276.300,2845.000,1.000,1.000,1.000,1.000,1.000,0.000,3.345,3.489,5.094,4.000,0.000 138 | 19.000,32.000,12.000,71.000,4.000,5.000,1.000,1.000,1.000,5.250,21.500,62.400,17.000,47.800,97.400,419.250,275.000,1.000,1.000,1.000,1.000,1.000,1.000,1.658,3.068,4.263,4.000,1.000 139 | 18.000,25.000,4.000,26.000,2.000,7.000,0.000,1.000,0.000,3.300,27.500,0.000,31.000,0.000,50.350,531.100,540.000,0.000,0.000,0.000,0.000,1.000,0.000,1.194,3.314,3.258,1.000,0.000 140 | 45.000,34.000,14.000,43.000,4.000,0.000,1.000,0.000,0.000,6.550,0.000,32.350,0.000,0.000,315.200,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.000,1.879,3.240,3.761,2.000,1.000 141 | 17.000,41.000,9.000,28.000,4.000,3.000,0.000,1.000,0.000,1.800,0.000,0.000,10.000,0.000,27.100,0.000,140.000,0.000,0.000,0.000,1.000,0.000,0.000,0.588,3.240,3.332,1.000,0.000 142 | 8.000,42.000,2.000,129.000,4.000,17.000,0.000,0.000,0.000,6.100,0.000,0.000,0.000,0.000,63.800,0.000,0.000,0.000,0.000,1.000,0.000,1.000,1.000,1.808,3.240,4.860,1.000,0.000 143 | 66.000,62.000,31.000,47.000,2.000,4.000,0.000,1.000,0.000,34.250,21.000,0.000,20.000,0.000,2298.250,1425.350,1355.000,0.000,0.000,0.000,0.000,0.000,0.000,3.534,3.045,3.850,2.000,0.000 144 | 60.000,57.000,18.000,72.000,5.000,30.000,1.000,1.000,1.000,20.350,29.000,50.250,20.500,55.900,1263.900,1806.850,1140.000,1.000,1.000,1.000,1.000,1.000,1.000,3.013,3.367,4.277,4.000,0.000 145 | 63.000,37.000,1.000,45.000,4.000,9.000,1.000,1.000,0.000,14.950,0.000,39.250,13.500,0.000,974.150,0.000,820.000,1.000,0.000,1.000,0.000,0.000,1.000,2.705,3.240,3.807,2.000,0.000 146 | 45.000,27.000,3.000,39.000,3.000,2.000,0.000,0.000,0.000,13.450,0.000,0.000,0.000,0.000,583.900,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,2.599,3.240,3.664,2.000,0.000 147 | 68.000,42.000,16.000,89.000,4.000,12.000,1.000,1.000,0.000,25.600,0.000,33.000,16.000,0.000,1759.600,0.000,1020.000,0.000,0.000,1.000,0.000,0.000,1.000,3.243,3.240,4.489,2.000,0.000 148 | 6.000,29.000,4.000,19.000,2.000,5.000,0.000,0.000,0.000,4.200,0.000,0.000,0.000,0.000,25.250,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,1.435,3.240,2.944,1.000,0.000 149 | 70.000,35.000,4.000,48.000,2.000,9.000,0.000,1.000,0.000,43.600,0.000,0.000,30.000,0.000,3040.800,0.000,2020.000,0.000,0.000,0.000,0.000,1.000,0.000,3.775,3.240,3.871,2.000,0.000 150 | 24.000,25.000,3.000,28.000,4.000,0.000,1.000,1.000,1.000,10.600,0.000,40.400,25.250,23.400,267.300,0.000,530.000,0.000,1.000,1.000,0.000,0.000,1.000,2.361,3.240,3.332,2.000,0.000 151 | 19.000,35.000,7.000,58.000,3.000,5.000,1.000,1.000,1.000,3.650,37.000,40.300,21.250,43.050,65.150,690.850,385.000,1.000,1.000,1.000,1.000,1.000,0.000,1.295,3.611,4.060,4.000,1.000 152 | 5.000,43.000,16.000,72.000,3.000,17.000,0.000,1.000,0.000,5.550,15.750,0.000,6.500,0.000,23.900,62.250,25.000,0.000,0.000,0.000,1.000,1.000,0.000,1.714,2.757,4.277,3.000,0.000 153 | 14.000,40.000,13.000,398.000,5.000,11.000,1.000,1.000,1.000,6.400,18.000,43.100,23.750,47.700,71.000,259.000,295.000,1.000,1.000,1.000,1.000,1.000,1.000,1.856,2.890,5.986,4.000,1.000 154 | 17.000,39.000,12.000,45.000,4.000,10.000,1.000,0.000,0.000,8.650,0.000,28.950,0.000,0.000,133.100,0.000,0.000,0.000,0.000,0.000,0.000,0.000,1.000,2.158,3.240,3.807,1.000,1.000 155 | 59.000,32.000,9.000,73.000,4.000,5.000,0.000,1.000,0.000,29.200,21.500,0.000,12.750,0.000,1726.600,1231.900,655.000,0.000,0.000,0.000,1.000,0.000,1.000,3.374,3.068,4.290,2.000,1.000 156 | 60.000,53.000,22.000,171.000,1.000,37.000,0.000,1.000,0.000,9.900,31.750,0.000,18.000,0.000,608.650,1972.100,1070.000,0.000,0.000,0.000,1.000,1.000,0.000,2.293,3.458,5.142,3.000,0.000 157 | 53.000,37.000,7.000,25.000,1.000,2.000,0.000,1.000,0.000,5.400,52.000,0.000,34.000,0.000,291.500,2794.950,1660.000,0.000,0.000,0.000,1.000,1.000,0.000,1.686,3.951,3.219,3.000,0.000 158 | 14.000,26.000,1.000,25.000,2.000,0.000,0.000,0.000,0.000,4.400,0.000,0.000,0.000,0.000,71.200,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,1.482,3.240,3.219,3.000,0.000 159 | 17.000,19.000,0.000,18.000,2.000,0.000,1.000,0.000,0.000,7.100,0.000,24.950,0.000,0.000,120.350,0.000,0.000,0.000,0.000,1.000,0.000,0.000,1.000,1.960,3.240,2.890,2.000,1.000 160 | 69.000,42.000,23.000,19.000,3.000,0.000,1.000,1.000,0.000,25.950,0.000,36.900,18.750,0.000,1812.150,0.000,1270.000,0.000,0.000,1.000,0.000,0.000,1.000,3.256,3.240,2.944,1.000,0.000 161 | 10.000,20.000,1.000,20.000,2.000,0.000,0.000,0.000,0.000,5.900,0.000,0.000,0.000,0.000,48.950,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,1.775,3.240,2.996,1.000,1.000 162 | 21.000,43.000,13.000,41.000,2.000,1.000,1.000,1.000,1.000,6.150,23.750,45.400,12.250,37.250,118.150,497.150,195.000,1.000,1.000,1.000,1.000,1.000,1.000,1.816,3.168,3.714,4.000,1.000 163 | 64.000,37.000,10.000,44.000,4.000,9.000,1.000,1.000,0.000,16.150,0.000,35.050,22.000,0.000,965.300,0.000,1350.000,1.000,0.000,0.000,0.000,0.000,1.000,2.782,3.240,3.784,2.000,0.000 164 | 60.000,56.000,19.000,51.000,4.000,11.000,1.000,1.000,1.000,15.250,0.000,57.450,28.500,67.000,873.700,0.000,1630.000,1.000,1.000,1.000,1.000,1.000,1.000,2.725,3.240,3.932,4.000,0.000 165 | 58.000,37.000,5.000,64.000,4.000,8.000,1.000,1.000,1.000,10.800,58.750,63.250,37.750,109.700,634.750,3525.550,2305.000,1.000,1.000,1.000,1.000,1.000,1.000,2.380,4.073,4.159,4.000,1.000 166 | 72.000,61.000,34.000,61.000,1.000,8.000,0.000,1.000,0.000,27.350,0.000,0.000,13.250,0.000,1980.150,0.000,895.000,0.000,0.000,0.000,0.000,0.000,0.000,3.309,3.240,4.111,2.000,0.000 167 | 18.000,34.000,4.000,42.000,2.000,14.000,0.000,1.000,0.000,6.450,19.000,0.000,17.000,0.000,104.050,318.750,240.000,1.000,1.000,0.000,1.000,1.000,0.000,1.864,2.944,3.738,4.000,0.000 168 | 70.000,36.000,8.000,50.000,1.000,15.000,0.000,1.000,0.000,20.550,0.000,0.000,20.750,0.000,1407.150,0.000,1370.000,0.000,0.000,0.000,0.000,0.000,0.000,3.023,3.240,3.912,2.000,0.000 169 | 7.000,38.000,4.000,70.000,4.000,4.000,1.000,1.000,1.000,3.850,19.250,38.350,7.000,17.700,26.650,127.500,40.000,1.000,1.000,1.000,1.000,0.000,1.000,1.348,2.958,4.248,4.000,1.000 170 | 71.000,53.000,29.000,48.000,4.000,0.000,0.000,1.000,0.000,34.950,28.750,0.000,36.000,0.000,2506.050,2074.750,2560.000,0.000,0.000,0.000,1.000,1.000,0.000,3.554,3.359,3.871,3.000,0.000 171 | 31.000,46.000,23.000,144.000,4.000,13.000,0.000,1.000,0.000,7.600,15.250,0.000,9.750,0.000,271.550,501.150,290.000,0.000,0.000,1.000,1.000,0.000,1.000,2.028,2.725,4.970,3.000,0.000 172 | 16.000,49.000,17.000,41.000,2.000,5.000,1.000,0.000,1.000,4.100,14.000,39.150,0.000,31.800,62.450,260.050,0.000,1.000,1.000,1.000,0.000,0.000,0.000,1.411,2.639,3.714,4.000,0.000 173 | 59.000,26.000,3.000,41.000,4.000,1.000,1.000,1.000,1.000,12.650,0.000,35.100,46.750,29.150,804.000,0.000,2710.000,0.000,0.000,1.000,0.000,0.000,0.000,2.538,3.240,3.714,2.000,1.000 174 | 9.000,40.000,13.000,38.000,4.000,7.000,1.000,1.000,1.000,3.350,21.000,42.400,17.500,59.550,37.700,179.800,125.000,1.000,1.000,1.000,1.000,1.000,1.000,1.209,3.045,3.638,4.000,1.000 175 | 12.000,55.000,13.000,36.000,1.000,5.000,1.000,0.000,0.000,5.950,0.000,26.550,0.000,0.000,73.350,0.000,0.000,0.000,0.000,1.000,0.000,0.000,0.000,1.783,3.240,3.584,2.000,1.000 176 | 3.000,32.000,4.000,58.000,2.000,11.000,1.000,1.000,1.000,2.750,15.750,29.500,9.250,28.550,5.700,49.650,15.000,1.000,1.000,0.000,0.000,0.000,1.000,1.012,2.757,4.060,4.000,1.000 177 | 52.000,39.000,6.000,119.000,3.000,18.000,0.000,1.000,0.000,10.500,25.000,0.000,8.250,0.000,556.500,1280.900,450.000,0.000,1.000,0.000,0.000,0.000,1.000,2.351,3.219,4.779,1.000,0.000 178 | 18.000,69.000,28.000,11.000,1.000,17.000,0.000,0.000,0.000,3.850,0.000,0.000,0.000,0.000,62.750,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,1.348,3.240,2.398,1.000,0.000 179 | 43.000,29.000,4.000,33.000,1.000,13.000,0.000,1.000,1.000,22.050,18.750,0.000,7.500,24.950,1042.950,830.750,345.000,0.000,0.000,0.000,1.000,1.000,0.000,3.093,2.931,3.497,3.000,0.000 180 | 37.000,33.000,4.000,41.000,3.000,8.000,1.000,0.000,0.000,9.200,0.000,30.450,0.000,0.000,347.100,0.000,0.000,0.000,0.000,0.000,0.000,0.000,1.000,2.219,3.240,3.714,2.000,1.000 181 | 51.000,46.000,8.000,107.000,2.000,21.000,0.000,1.000,1.000,17.300,25.750,0.000,23.000,37.400,840.000,1396.750,1155.000,0.000,1.000,0.000,1.000,1.000,0.000,2.851,3.248,4.673,3.000,0.000 182 | 56.000,53.000,23.000,100.000,5.000,14.000,1.000,1.000,0.000,14.150,0.000,34.200,9.500,0.000,752.900,0.000,560.000,1.000,0.000,1.000,0.000,0.000,1.000,2.650,3.240,4.605,2.000,0.000 183 | 72.000,55.000,24.000,82.000,3.000,25.000,1.000,1.000,0.000,62.300,0.000,35.650,65.250,0.000,4333.000,0.000,4915.000,0.000,0.000,0.000,1.000,0.000,1.000,4.132,3.240,4.407,2.000,0.000 184 | 32.000,44.000,10.000,201.000,2.000,24.000,0.000,1.000,0.000,7.650,28.000,0.000,18.500,0.000,237.850,895.050,595.000,0.000,1.000,0.000,1.000,1.000,0.000,2.035,3.332,5.303,3.000,0.000 185 | 51.000,49.000,29.000,45.000,1.000,16.000,0.000,1.000,0.000,15.700,0.000,0.000,8.500,0.000,815.050,0.000,400.000,0.000,0.000,0.000,1.000,0.000,0.000,2.754,3.240,3.807,1.000,0.000 186 | 26.000,55.000,13.000,61.000,1.000,26.000,0.000,1.000,0.000,4.250,29.000,0.000,12.000,0.000,102.450,729.150,300.000,0.000,0.000,0.000,1.000,1.000,0.000,1.447,3.367,4.111,3.000,0.000 187 | 34.000,40.000,21.000,23.000,4.000,9.000,1.000,1.000,1.000,5.950,25.750,45.400,16.750,37.700,207.700,805.850,540.000,0.000,1.000,1.000,1.000,1.000,1.000,1.783,3.248,3.135,4.000,0.000 188 | 20.000,25.000,4.000,33.000,4.000,0.000,0.000,1.000,1.000,4.550,16.000,0.000,19.250,27.050,115.650,288.750,365.000,0.000,0.000,1.000,0.000,0.000,1.000,1.515,2.773,3.497,1.000,0.000 189 | 58.000,36.000,13.000,39.000,2.000,8.000,0.000,1.000,1.000,16.400,38.250,0.000,55.500,57.050,933.100,2355.400,3145.000,1.000,1.000,1.000,1.000,1.000,1.000,2.797,3.644,3.664,4.000,1.000 190 | 25.000,38.000,19.000,56.000,1.000,19.000,1.000,1.000,1.000,10.550,0.000,31.950,32.750,23.650,290.250,0.000,770.000,1.000,0.000,0.000,0.000,1.000,0.000,2.356,3.240,4.025,3.000,0.000 191 | 66.000,50.000,2.000,333.000,5.000,24.000,0.000,1.000,0.000,10.300,0.000,0.000,14.250,0.000,659.550,0.000,975.000,0.000,1.000,1.000,0.000,0.000,0.000,2.332,3.240,5.808,2.000,0.000 192 | 71.000,48.000,25.000,288.000,3.000,19.000,0.000,1.000,0.000,30.900,0.000,0.000,19.750,0.000,2123.600,0.000,1305.000,0.000,0.000,0.000,0.000,0.000,0.000,3.431,3.240,5.663,1.000,0.000 193 | 6.000,20.000,0.000,25.000,2.000,0.000,0.000,1.000,0.000,1.900,20.750,0.000,15.500,0.000,19.300,114.400,95.000,0.000,0.000,0.000,1.000,1.000,0.000,0.642,3.033,3.219,3.000,0.000 194 | 26.000,30.000,4.000,76.000,3.000,7.000,1.000,0.000,0.000,9.450,0.000,29.200,0.000,0.000,214.700,0.000,0.000,0.000,1.000,0.000,0.000,0.000,1.000,2.246,3.240,4.331,1.000,0.000 195 | 61.000,52.000,21.000,82.000,1.000,18.000,0.000,1.000,0.000,12.100,0.000,0.000,15.750,0.000,751.750,0.000,910.000,0.000,0.000,0.000,0.000,1.000,0.000,2.493,3.240,4.407,3.000,0.000 196 | 57.000,60.000,20.000,14.000,2.000,27.000,0.000,1.000,0.000,16.100,14.000,0.000,11.750,0.000,938.650,822.350,630.000,0.000,0.000,0.000,1.000,1.000,0.000,2.779,2.639,2.639,3.000,0.000 197 | 55.000,44.000,24.000,83.000,1.000,23.000,0.000,1.000,0.000,17.350,24.500,0.000,14.250,0.000,973.100,1343.500,720.000,0.000,0.000,0.000,0.000,1.000,0.000,2.854,3.199,4.419,3.000,0.000 198 | 34.000,23.000,3.000,24.000,1.000,7.000,0.000,1.000,0.000,6.000,28.000,0.000,12.750,0.000,203.250,959.400,435.000,0.000,0.000,0.000,1.000,1.000,0.000,1.792,3.332,3.178,3.000,0.000 199 | 6.000,32.000,10.000,47.000,1.000,10.000,0.000,1.000,0.000,3.850,23.750,0.000,12.500,0.000,29.900,128.450,80.000,0.000,0.000,0.000,1.000,1.000,0.000,1.348,3.168,3.850,3.000,0.000 200 | 24.000,30.000,0.000,25.000,4.000,5.000,0.000,1.000,1.000,8.700,47.750,0.000,32.750,64.000,186.600,1152.900,780.000,1.000,1.000,1.000,1.000,1.000,1.000,2.163,3.866,3.219,4.000,1.000 201 | 61.000,50.000,16.000,190.000,2.000,22.000,1.000,1.000,1.000,16.850,0.000,42.550,26.500,44.100,1063.150,0.000,1600.000,0.000,0.000,1.000,0.000,0.000,1.000,2.824,3.240,5.247,2.000,0.000 202 | -------------------------------------------------------------------------------- /traditional_ML_algorithms/LogisticRegression/Logistic_Regression.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Logistic Regression" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "### Import Libraries" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": 32, 20 | "metadata": {}, 21 | "outputs": [], 22 | "source": [ 23 | "import numpy as np\n", 24 | "import pandas as pd\n", 25 | "import warnings\n", 26 | "# from sklearn import datasets\n", 27 | "# import matplotlib.pyplot as plt\n", 28 | "\n", 29 | "warnings.filterwarnings('ignore')\n", 30 | "\n", 31 | "%matplotlib inline" 32 | ] 33 | }, 34 | { 35 | "cell_type": "markdown", 36 | "metadata": {}, 37 | "source": [ 38 | "### Load data from CSV file" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": 33, 44 | "metadata": {}, 45 | "outputs": [ 46 | { 47 | "data": { 48 | "text/html": [ 49 | "
\n", 50 | "\n", 63 | "\n", 64 | " \n", 65 | " \n", 66 | " \n", 67 | " \n", 68 | " \n", 69 | " \n", 70 | " \n", 71 | " \n", 72 | " \n", 73 | " \n", 74 | " \n", 75 | " \n", 76 | " \n", 77 | " \n", 78 | " \n", 79 | " \n", 80 | " \n", 81 | " \n", 82 | " \n", 83 | " \n", 84 | " \n", 85 | " \n", 86 | " \n", 87 | " \n", 88 | " \n", 89 | " \n", 90 | " \n", 91 | " \n", 92 | " \n", 93 | " \n", 94 | " \n", 95 | " \n", 96 | " \n", 97 | " \n", 98 | " \n", 99 | " \n", 100 | " \n", 101 | " \n", 102 | " \n", 103 | " \n", 104 | " \n", 105 | " \n", 106 | " \n", 107 | " \n", 108 | " \n", 109 | " \n", 110 | " \n", 111 | " \n", 112 | " \n", 113 | " \n", 114 | " \n", 115 | " \n", 116 | " \n", 117 | " \n", 118 | " \n", 119 | " \n", 120 | " \n", 121 | " \n", 122 | " \n", 123 | " \n", 124 | " \n", 125 | " \n", 126 | " \n", 127 | " \n", 128 | " \n", 129 | " \n", 130 | " \n", 131 | " \n", 132 | " \n", 133 | " \n", 134 | " \n", 135 | " \n", 136 | " \n", 137 | " \n", 138 | " \n", 139 | " \n", 140 | " \n", 141 | " \n", 142 | " \n", 143 | " \n", 144 | " \n", 145 | " \n", 146 | " \n", 147 | " \n", 148 | " \n", 149 | " \n", 150 | " \n", 151 | " \n", 152 | " \n", 153 | " \n", 154 | " \n", 155 | " \n", 156 | " \n", 157 | " \n", 158 | " \n", 159 | " \n", 160 | " \n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | " \n", 170 | " \n", 171 | " \n", 172 | " \n", 173 | " \n", 174 | " \n", 175 | " \n", 176 | " \n", 177 | " \n", 178 | " \n", 179 | " \n", 180 | " \n", 181 | " \n", 182 | " \n", 183 | " \n", 184 | " \n", 185 | " \n", 186 | " \n", 187 | " \n", 188 | " \n", 189 | " \n", 190 | " \n", 191 | " \n", 192 | " \n", 193 | " \n", 194 | " \n", 195 | " \n", 196 | " \n", 197 | " \n", 198 | " \n", 199 | " \n", 200 | " \n", 201 | " \n", 202 | " \n", 203 | " \n", 204 | " \n", 205 | " \n", 206 | " \n", 207 | " \n", 208 | " \n", 209 | " \n", 210 | " \n", 211 | " \n", 212 | "
tenureageaddressincomeedemployequipcallcardwirelesslongmon...pagerinternetcallwaitconferebillloglonglogtolllninccustcatchurn
011.033.07.0136.05.05.00.01.01.04.40...1.00.01.01.00.01.4823.0334.9134.01.0
133.033.012.033.02.00.00.00.00.09.45...0.00.00.00.00.02.2463.2403.4971.01.0
223.030.09.030.01.02.00.00.00.06.30...0.00.00.01.00.01.8413.2403.4013.00.0
338.035.05.076.02.010.01.01.01.06.05...1.01.01.01.01.01.8003.8074.3314.00.0
47.035.014.080.02.015.00.01.00.07.10...0.00.01.01.00.01.9603.0914.3823.00.0
\n", 213 | "

5 rows × 28 columns

\n", 214 | "
" 215 | ], 216 | "text/plain": [ 217 | " tenure age address income ed employ equip callcard wireless \\\n", 218 | "0 11.0 33.0 7.0 136.0 5.0 5.0 0.0 1.0 1.0 \n", 219 | "1 33.0 33.0 12.0 33.0 2.0 0.0 0.0 0.0 0.0 \n", 220 | "2 23.0 30.0 9.0 30.0 1.0 2.0 0.0 0.0 0.0 \n", 221 | "3 38.0 35.0 5.0 76.0 2.0 10.0 1.0 1.0 1.0 \n", 222 | "4 7.0 35.0 14.0 80.0 2.0 15.0 0.0 1.0 0.0 \n", 223 | "\n", 224 | " longmon ... pager internet callwait confer ebill loglong logtoll \\\n", 225 | "0 4.40 ... 1.0 0.0 1.0 1.0 0.0 1.482 3.033 \n", 226 | "1 9.45 ... 0.0 0.0 0.0 0.0 0.0 2.246 3.240 \n", 227 | "2 6.30 ... 0.0 0.0 0.0 1.0 0.0 1.841 3.240 \n", 228 | "3 6.05 ... 1.0 1.0 1.0 1.0 1.0 1.800 3.807 \n", 229 | "4 7.10 ... 0.0 0.0 1.0 1.0 0.0 1.960 3.091 \n", 230 | "\n", 231 | " lninc custcat churn \n", 232 | "0 4.913 4.0 1.0 \n", 233 | "1 3.497 1.0 1.0 \n", 234 | "2 3.401 3.0 0.0 \n", 235 | "3 4.331 4.0 0.0 \n", 236 | "4 4.382 3.0 0.0 \n", 237 | "\n", 238 | "[5 rows x 28 columns]" 239 | ] 240 | }, 241 | "execution_count": 33, 242 | "metadata": {}, 243 | "output_type": "execute_result" 244 | } 245 | ], 246 | "source": [ 247 | "churn_df=pd.read_csv(\"./ChurnData.csv\")\n", 248 | "churn_df.head()" 249 | ] 250 | }, 251 | { 252 | "cell_type": "markdown", 253 | "metadata": {}, 254 | "source": [ 255 | "### Data pre-processing and selection" 256 | ] 257 | }, 258 | { 259 | "cell_type": "code", 260 | "execution_count": 34, 261 | "metadata": {}, 262 | "outputs": [ 263 | { 264 | "data": { 265 | "text/html": [ 266 | "
\n", 267 | "\n", 280 | "\n", 281 | " \n", 282 | " \n", 283 | " \n", 284 | " \n", 285 | " \n", 286 | " \n", 287 | " \n", 288 | " \n", 289 | " \n", 290 | " \n", 291 | " \n", 292 | " \n", 293 | " \n", 294 | " \n", 295 | " \n", 296 | " \n", 297 | " \n", 298 | " \n", 299 | " \n", 300 | " \n", 301 | " \n", 302 | " \n", 303 | " \n", 304 | " \n", 305 | " \n", 306 | " \n", 307 | " \n", 308 | " \n", 309 | " \n", 310 | " \n", 311 | " \n", 312 | " \n", 313 | " \n", 314 | " \n", 315 | " \n", 316 | " \n", 317 | " \n", 318 | " \n", 319 | " \n", 320 | " \n", 321 | " \n", 322 | " \n", 323 | " \n", 324 | " \n", 325 | " \n", 326 | " \n", 327 | " \n", 328 | " \n", 329 | " \n", 330 | " \n", 331 | " \n", 332 | " \n", 333 | " \n", 334 | " \n", 335 | " \n", 336 | " \n", 337 | " \n", 338 | " \n", 339 | " \n", 340 | " \n", 341 | " \n", 342 | " \n", 343 | " \n", 344 | " \n", 345 | " \n", 346 | " \n", 347 | " \n", 348 | " \n", 349 | " \n", 350 | " \n", 351 | " \n", 352 | " \n", 353 | " \n", 354 | " \n", 355 | " \n", 356 | " \n", 357 | " \n", 358 | " \n", 359 | " \n", 360 | " \n", 361 | " \n", 362 | " \n", 363 | "
tenureageaddressincomeedemployequipcallcardwirelesschurn
011.033.07.0136.05.05.00.01.01.01
133.033.012.033.02.00.00.00.00.01
223.030.09.030.01.02.00.00.00.00
338.035.05.076.02.010.01.01.01.00
47.035.014.080.02.015.00.01.00.00
\n", 364 | "
" 365 | ], 366 | "text/plain": [ 367 | " tenure age address income ed employ equip callcard wireless \\\n", 368 | "0 11.0 33.0 7.0 136.0 5.0 5.0 0.0 1.0 1.0 \n", 369 | "1 33.0 33.0 12.0 33.0 2.0 0.0 0.0 0.0 0.0 \n", 370 | "2 23.0 30.0 9.0 30.0 1.0 2.0 0.0 0.0 0.0 \n", 371 | "3 38.0 35.0 5.0 76.0 2.0 10.0 1.0 1.0 1.0 \n", 372 | "4 7.0 35.0 14.0 80.0 2.0 15.0 0.0 1.0 0.0 \n", 373 | "\n", 374 | " churn \n", 375 | "0 1 \n", 376 | "1 1 \n", 377 | "2 0 \n", 378 | "3 0 \n", 379 | "4 0 " 380 | ] 381 | }, 382 | "execution_count": 34, 383 | "metadata": {}, 384 | "output_type": "execute_result" 385 | } 386 | ], 387 | "source": [ 388 | "churn_df=churn_df[['tenure', 'age', 'address', 'income', 'ed', 'employ', 'equip', 'callcard', 'wireless','churn']]\n", 389 | "churn_df['churn']=churn_df['churn'].astype('int')\n", 390 | "churn_df.head()" 391 | ] 392 | }, 393 | { 394 | "cell_type": "code", 395 | "execution_count": 35, 396 | "metadata": {}, 397 | "outputs": [ 398 | { 399 | "name": "stdout", 400 | "output_type": "stream", 401 | "text": [ 402 | "[[ 11. 33. 7. 136. 5. 5. 0.]\n", 403 | " [ 33. 33. 12. 33. 2. 0. 0.]\n", 404 | " [ 23. 30. 9. 30. 1. 2. 0.]\n", 405 | " [ 38. 35. 5. 76. 2. 10. 1.]\n", 406 | " [ 7. 35. 14. 80. 2. 15. 0.]]\n", 407 | "(200, 7)\n" 408 | ] 409 | } 410 | ], 411 | "source": [ 412 | "X=np.asarray(churn_df[['tenure', 'age', 'address', 'income', 'ed', 'employ', 'equip']])\n", 413 | "print(X[0:5])\n", 414 | "print(X.shape)" 415 | ] 416 | }, 417 | { 418 | "cell_type": "code", 419 | "execution_count": 36, 420 | "metadata": {}, 421 | "outputs": [ 422 | { 423 | "data": { 424 | "text/plain": [ 425 | "array([1, 1, 0, 0, 0])" 426 | ] 427 | }, 428 | "execution_count": 36, 429 | "metadata": {}, 430 | "output_type": "execute_result" 431 | } 432 | ], 433 | "source": [ 434 | "y=np.asarray(churn_df['churn'])\n", 435 | "y[0:5]" 436 | ] 437 | }, 438 | { 439 | "cell_type": "markdown", 440 | "metadata": {}, 441 | "source": [ 442 | "### Normalising data" 443 | ] 444 | }, 445 | { 446 | "cell_type": "code", 447 | "execution_count": 37, 448 | "metadata": {}, 449 | "outputs": [ 450 | { 451 | "data": { 452 | "text/plain": [ 453 | "array([[0.00659472, 0.01978417, 0.00419664, 0.08153477, 0.0029976 ,\n", 454 | " 0.0029976 , 0. ],\n", 455 | " [0.01978417, 0.01978417, 0.00719424, 0.01978417, 0.00119904,\n", 456 | " 0. , 0. ],\n", 457 | " [0.01378897, 0.01798561, 0.00539568, 0.01798561, 0.00059952,\n", 458 | " 0.00119904, 0. ],\n", 459 | " [0.02278177, 0.02098321, 0.0029976 , 0.04556355, 0.00119904,\n", 460 | " 0.0059952 , 0.00059952],\n", 461 | " [0.00419664, 0.02098321, 0.00839329, 0.04796163, 0.00119904,\n", 462 | " 0.00899281, 0. ]])" 463 | ] 464 | }, 465 | "execution_count": 37, 466 | "metadata": {}, 467 | "output_type": "execute_result" 468 | } 469 | ], 470 | "source": [ 471 | "normalized_data=X.copy()\n", 472 | "normalized_data = (normalized_data-normalized_data.min()) /(normalized_data.max()-normalized_data.min())\n", 473 | "normalized_data[0:5]" 474 | ] 475 | }, 476 | { 477 | "cell_type": "markdown", 478 | "metadata": {}, 479 | "source": [ 480 | "#### Theta\n", 481 | "To perform a prediction we use a neural network like notaion. We have weights (w), inputs(x) and bias (b). \n", 482 | "$$ \n", 483 | "z= ( \\sum_{i=1}^{n} w_{i}x_{i} ) + b \n", 484 | "$$\n", 485 | "\n", 486 | "For simplicity we will represent (w,b) as $ \\theta $" 487 | ] 488 | }, 489 | { 490 | "cell_type": "markdown", 491 | "metadata": {}, 492 | "source": [ 493 | "#### Sigmoid Function\n", 494 | "$$\n", 495 | " \\sigma(z)= \\frac{1}{1+\\exp^{-z}}\n", 496 | "$$\n" 497 | ] 498 | }, 499 | { 500 | "cell_type": "markdown", 501 | "metadata": {}, 502 | "source": [ 503 | "#### Loss\n", 504 | "We need a loss function that expresses, for an obersvaton x, how close the classifier output $(\\hat{y} = \\sigma{(w.x+b)}) $ is to the correct output **y**. Here in this example we have used Cross entropy as our loss function. Cross entropy is basically a bernouli eqaution.\n", 505 | "\n", 506 | "Bernouli Equation in probability:\n", 507 | "$$ BE(p,q)=p^y(q)^{1-y} $$\n", 508 | "$$ L_{CE} (\\hat{y},y)= -\\frac {1}{m} \\sum_{i=1}^{m}y\\log({\\hat{y}})+(1-\\hat{y})\\log({1-\\hat{y}}) $$" 509 | ] 510 | }, 511 | { 512 | "cell_type": "markdown", 513 | "metadata": {}, 514 | "source": [ 515 | "#### Gradient descent and gradients \n", 516 | "\n", 517 | "\n", 518 | "Now, to calculate the gradients to optimize the weights using gradient descent, we must calculate the derivative of the loss function. That is partial derivative of the cross entropy formula\n", 519 | "$$ \\frac {\\partial L_{CE} (\\hat{y},y)} {\\partial w} = \\frac {1}{m}(\\hat{y}-y)x_{i}^T $$\n", 520 | "$$ \\frac {\\partial L_{CE} (\\hat{y},y)} {\\partial b} = \\frac {1}{m}(\\hat{y}-y) $$\n", 521 | "\n", 522 | "Now since we have our gradients, we can use it to update the weights and biases\n", 523 | "\n", 524 | "##### Derivation\n", 525 | "$$ \\frac{d}{dx}\\ln(x)=\\frac{1}{x} $$\n", 526 | "$$ \\frac{d\\sigma(z)}{dz}=\\sigma(z)(1-\\sigma(z)) $$\n", 527 | "By the chain rule of derivatives:-\n", 528 | "$$ \\frac{df}{dx}=\\frac{du}{dv}.\\frac{dv}{dx} $$\n", 529 | "\n", 530 | "First, we need to calcualte the derivative of the loss function with respect to a single weight $ w_{j} $ (that is we need to compute the derivative for each weight and bias)\n", 531 | "\n", 532 | "$$ \\frac{\\partial{L_{CE}}}{\\partial{w_{j}}}=\\frac{\\partial{}}{\\partial{w_{j}}}-[y\\log\\sigma(w.x+b)+(1-y)\\log(1-\\sigma(w.x+b))] $$\n", 533 | "$$ \n", 534 | " = -\\left[ \\frac{\\partial{}}{\\partial{w_{j}}}y\\log\\sigma(w.x+b)+\\frac{\\partial{}}{\\partial{w_{j}}} (1-y)\\log[1-\\sigma(w.x+b)] \\right ]\n", 535 | "$$\n", 536 | "Now we need to use the chain rule\n", 537 | "$$ \n", 538 | "\\frac{\\partial{L_{CE}}}{\\partial{w_{j}}} = -\\frac{y}{\\sigma(w.x+b)} \\frac{\\partial{}}{\\partial{w_{j}}}\\sigma(w.x+b)-\\frac{1-y}{1-\\sigma(w.x+b)}\\frac{\\partial{}}{\\partial{w_{j}}}(1-\\sigma(w.x+b))\n", 539 | "$$\n", 540 | "Simplifying the above equation we get,\n", 541 | "$$ \\frac{\\partial{L_{CE}}}{\\partial{w_{j}}}= -\\left [ \\frac{y}{\\sigma(w.x+b)}-\\frac{1-y}{1-\\sigma(w.x+b)} \\right]\\frac{\\partial{}}{\\partial{w_{j}}}\\sigma(w.x+b)\n", 542 | "$$\n", 543 | "\n", 544 | "Now substituting the derivative of the sigmoid function we get,\n", 545 | "$$ \\frac{\\partial{L_{CE}}}{\\partial{w_{j}}}= -\\left [ \\frac{y-\\sigma(w.x+b)}{\\sigma(w.x+b)[1-\\sigma(w.x+b)]} \\right ] \\sigma(w.x+b)[1-\\sigma(w.x+b)] \\frac{\\partial(w.x+b)}{\\partial{w_{j}}} $$\n", 546 | "$$ = -\\left [ \\frac{y-\\sigma(w.x+b)}{\\sigma(w.x+b)[1-\\sigma(w.x+b)]} \\right ] \\sigma(w.x+b)[1-\\sigma(w.x+b)] x_{j} $$\n", 547 | "$$ = -[y-\\sigma(w.x+b)]x_{j} $$\n", 548 | "$$ = [\\sigma(w.x+b)-y]x_{j} $$\n", 549 | "$$ = (\\hat{y}-y)x_{j} $$\n", 550 | "\n" 551 | ] 552 | }, 553 | { 554 | "cell_type": "code", 555 | "execution_count": 38, 556 | "metadata": {}, 557 | "outputs": [ 558 | { 559 | "data": { 560 | "text/plain": [ 561 | "array([[ 1., 11., 33., ..., 5., 5., 0.],\n", 562 | " [ 1., 33., 33., ..., 2., 0., 0.],\n", 563 | " [ 1., 23., 30., ..., 1., 2., 0.],\n", 564 | " ...,\n", 565 | " [ 1., 6., 32., ..., 1., 10., 0.],\n", 566 | " [ 1., 24., 30., ..., 4., 5., 0.],\n", 567 | " [ 1., 61., 50., ..., 2., 22., 1.]])" 568 | ] 569 | }, 570 | "execution_count": 38, 571 | "metadata": {}, 572 | "output_type": "execute_result" 573 | } 574 | ], 575 | "source": [ 576 | "np.concatenate((np.ones((X.shape[0],1)),X),axis=1)\n", 577 | "# X\n" 578 | ] 579 | }, 580 | { 581 | "cell_type": "code", 582 | "execution_count": 39, 583 | "metadata": {}, 584 | "outputs": [], 585 | "source": [ 586 | "class LogisticRegression:\n", 587 | " def __init__(self, lr, num_iter, fit_intercept=True, verbose=False):\n", 588 | " self.lr = lr #learning rate\n", 589 | " self.num_iter = num_iter #number of iterations\n", 590 | " self.fit_intercept = fit_intercept #intercept\n", 591 | " self.verbose = verbose # Verbose means that it will output messages which could be useful for debugging and for understanding how the training is doing\n", 592 | "\n", 593 | " #weights \n", 594 | " def add_intercept(self, X):\n", 595 | " intercept = np.ones((X.shape[0], 1))\n", 596 | " return np.concatenate((intercept, X), axis=1)\n", 597 | "\n", 598 | " #sigmoid function \n", 599 | " def sigmoid(self, z):\n", 600 | " return 1 / (1 + np.exp(-z))\n", 601 | "\n", 602 | " #cross entropy loss \n", 603 | " def loss(self, h, y):\n", 604 | " return (-y * np.log(h) - (1 - y) * np.log(1 - h)).mean()\n", 605 | "\n", 606 | " #fitting the model \n", 607 | " def fit(self, X, y):\n", 608 | " if self.fit_intercept: #bias initialization\n", 609 | " X = self.add_intercept(X)\n", 610 | "\n", 611 | " # weights initialization\n", 612 | " self.theta = np.zeros(X.shape[1]) #bias values\n", 613 | "\n", 614 | " for i in range(self.num_iter):\n", 615 | " z = np.dot(X, self.theta) #w.x+b\n", 616 | " h = self.sigmoid(z) \n", 617 | " gradient = np.dot(X.T, (h - y)) / y.size #following the formula above (y.size=m), x.T is calculating the transpose of the matrix for matrix multiplication\n", 618 | " self.theta -= self.lr * gradient #updating the weights by moving to the minimum by a small rate.\n", 619 | "\n", 620 | " if(self.verbose == True and i % 100000 == 0):\n", 621 | " z = np.dot(X, self.theta)\n", 622 | " h = self.sigmoid(z)\n", 623 | " print(f'loss: {self.loss(h, y)} \\t')\n", 624 | "\n", 625 | " #These two functions will predict the correct labels and show us the probabilty of correct labels predicted. \n", 626 | " def predict_prob(self, X):\n", 627 | " if self.fit_intercept:\n", 628 | " X = self.add_intercept(X)\n", 629 | "\n", 630 | " return self.sigmoid(np.dot(X, self.theta))\n", 631 | "\n", 632 | " def predict(self, X):\n", 633 | " return self.predict_prob(X).round()\n" 634 | ] 635 | }, 636 | { 637 | "cell_type": "code", 638 | "execution_count": 28, 639 | "metadata": {}, 640 | "outputs": [ 641 | { 642 | "name": "stdout", 643 | "output_type": "stream", 644 | "text": [ 645 | "loss: 0.6927042186454124 \t\n", 646 | "loss: 0.5910096716660708 \t\n", 647 | "loss: 0.5838561953745031 \t\n", 648 | "loss: 0.5779887445467037 \t\n", 649 | "loss: 0.5729061990167699 \t\n", 650 | "loss: 0.5683919821502402 \t\n", 651 | "loss: 0.564322571262061 \t\n", 652 | "loss: 0.5606177537705266 \t\n", 653 | "loss: 0.5572212812637848 \t\n", 654 | "loss: 0.5540915323955028 \t\n", 655 | "loss: 0.5511964079618044 \t\n", 656 | "loss: 0.5485103102805649 \t\n", 657 | "loss: 0.5460122548772475 \t\n", 658 | "loss: 0.5436846402124331 \t\n", 659 | "loss: 0.5415124191628589 \t\n", 660 | "loss: 0.5394825252637198 \t\n", 661 | "loss: 0.5375834654712491 \t\n", 662 | "loss: 0.5358050245217261 \t\n", 663 | "loss: 0.5341380456807027 \t\n", 664 | "loss: 0.5325742647612982 \t\n", 665 | "Wall time: 25.4 s\n" 666 | ] 667 | } 668 | ], 669 | "source": [ 670 | "model=LogisticRegression(lr=0.01,num_iter=2000000,verbose=True)\n", 671 | "%time model.fit(normalized_data,y)" 672 | ] 673 | }, 674 | { 675 | "cell_type": "code", 676 | "execution_count": 29, 677 | "metadata": {}, 678 | "outputs": [ 679 | { 680 | "data": { 681 | "text/plain": [ 682 | "array([[ 1., 11., 33., ..., 5., 5., 0.],\n", 683 | " [ 1., 33., 33., ..., 2., 0., 0.],\n", 684 | " [ 1., 23., 30., ..., 1., 2., 0.],\n", 685 | " ...,\n", 686 | " [ 1., 6., 32., ..., 1., 10., 0.],\n", 687 | " [ 1., 24., 30., ..., 4., 5., 0.],\n", 688 | " [ 1., 61., 50., ..., 2., 22., 1.]])" 689 | ] 690 | }, 691 | "execution_count": 29, 692 | "metadata": {}, 693 | "output_type": "execute_result" 694 | } 695 | ], 696 | "source": [ 697 | "model.add_intercept(X)" 698 | ] 699 | }, 700 | { 701 | "cell_type": "code", 702 | "execution_count": 30, 703 | "metadata": {}, 704 | "outputs": [ 705 | { 706 | "data": { 707 | "text/plain": [ 708 | "array([ 0.28961352, -29.03375129, -12.90644264, -9.13661946,\n", 709 | " -4.55567855, 1.52032046, -10.9023424 , 0.70800146])" 710 | ] 711 | }, 712 | "execution_count": 30, 713 | "metadata": {}, 714 | "output_type": "execute_result" 715 | } 716 | ], 717 | "source": [ 718 | "model.theta" 719 | ] 720 | }, 721 | { 722 | "cell_type": "code", 723 | "execution_count": 31, 724 | "metadata": {}, 725 | "outputs": [ 726 | { 727 | "data": { 728 | "text/plain": [ 729 | "0.71" 730 | ] 731 | }, 732 | "execution_count": 31, 733 | "metadata": {}, 734 | "output_type": "execute_result" 735 | } 736 | ], 737 | "source": [ 738 | "preds=model.predict(X)\n", 739 | "(preds==y).mean()" 740 | ] 741 | } 742 | ], 743 | "metadata": { 744 | "kernelspec": { 745 | "display_name": "Python 3.9.7 ('base')", 746 | "language": "python", 747 | "name": "python3" 748 | }, 749 | "language_info": { 750 | "codemirror_mode": { 751 | "name": "ipython", 752 | "version": 3 753 | }, 754 | "file_extension": ".py", 755 | "mimetype": "text/x-python", 756 | "name": "python", 757 | "nbconvert_exporter": "python", 758 | "pygments_lexer": "ipython3", 759 | "version": "3.9.7" 760 | }, 761 | "orig_nbformat": 4, 762 | "vscode": { 763 | "interpreter": { 764 | "hash": "b09ec625f77bf4fd762565a912b97636504ad6ec901eb2d0f4cf5a7de23e1ee5" 765 | } 766 | } 767 | }, 768 | "nbformat": 4, 769 | "nbformat_minor": 2 770 | } 771 | -------------------------------------------------------------------------------- /traditional_ML_algorithms/Multiple Linear Regression/Multiple_Linear_Regression.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "source": [ 6 | "## README\n", 7 | "To run the file, upload the data.csv file. In my case, I have uploaded it to sample_data folder in session storage. You can upload it elsewhere but remember to change the path mentioned in code afterwards." 8 | ], 9 | "metadata": { 10 | "id": "SoWKE7fEIMEi" 11 | } 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "source": [ 16 | "# Multiple Linear Regression\n", 17 | "### Using Gradient Descent" 18 | ], 19 | "metadata": { 20 | "id": "jQWnqkHCH6ln" 21 | } 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": null, 26 | "metadata": { 27 | "id": "pcN9YClQHU9i" 28 | }, 29 | "outputs": [], 30 | "source": [ 31 | "import numpy as np\n", 32 | "import pandas as pd" 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": null, 38 | "metadata": { 39 | "id": "OczzR2KGHU9l" 40 | }, 41 | "outputs": [], 42 | "source": [ 43 | "batch_size = 1000\n", 44 | "learning_rate = 0.000001\n", 45 | "num_of_epoch = 10000\n", 46 | "test_training_ratio = 0.2\n", 47 | "dataset_path = '/content/sample_data/data.csv'" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": null, 53 | "metadata": { 54 | "id": "TqoZARBKHU9m" 55 | }, 56 | "outputs": [], 57 | "source": [ 58 | "# Read the dataset\n", 59 | "df = pd.read_csv(dataset_path) \n", 60 | "df.head()" 61 | ] 62 | }, 63 | { 64 | "cell_type": "code", 65 | "execution_count": null, 66 | "metadata": { 67 | "id": "LWBsgFL2HU9o" 68 | }, 69 | "outputs": [], 70 | "source": [ 71 | "# Total_X is a matrix the size of 9568 X 4\n", 72 | "total_X = df.iloc[:, :-1].values \n", 73 | "\n", 74 | "# Total_Y is a vector the size of 9568\n", 75 | "total_y = df.iloc[:, -1].values " 76 | ] 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": null, 81 | "metadata": { 82 | "colab": { 83 | "base_uri": "https://localhost:8080/" 84 | }, 85 | "id": "an5C6G4GHU9p", 86 | "outputId": "78b44bdf-1185-4852-bb81-ed73c8c66e13" 87 | }, 88 | "outputs": [ 89 | { 90 | "output_type": "stream", 91 | "name": "stdout", 92 | "text": [ 93 | "[[1.00000e+00 8.34000e+00 4.07700e+01 1.01084e+03 9.00100e+01]\n", 94 | " [1.00000e+00 2.36400e+01 5.84900e+01 1.01140e+03 7.42000e+01]\n", 95 | " [1.00000e+00 2.97400e+01 5.69000e+01 1.00715e+03 4.19100e+01]\n", 96 | " [1.00000e+00 1.90700e+01 4.96900e+01 1.00722e+03 7.67900e+01]\n", 97 | " [1.00000e+00 1.18000e+01 4.06600e+01 1.01713e+03 9.72000e+01]]\n" 98 | ] 99 | } 100 | ], 101 | "source": [ 102 | "b = np.ones((total_X.shape[0],total_X.shape[1]+1)) # b is a 9568 X 5 matrix filled with ones.\n", 103 | "b[:, 1:] = total_X #b is the total_X matrix except with first column filled with ones\n", 104 | "total_X = b #reassign\n", 105 | "print(total_X[:5])" 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "source": [ 111 | "In order for us to verify the accuracy later, we will reserve about 20% of the whole dataset." 112 | ], 113 | "metadata": { 114 | "id": "rGcm0qsTsBYu" 115 | } 116 | }, 117 | { 118 | "cell_type": "code", 119 | "execution_count": null, 120 | "metadata": { 121 | "id": "3ZznhA1lHU9q" 122 | }, 123 | "outputs": [], 124 | "source": [ 125 | "num_of_testing_example = int(test_training_ratio*total_X.shape[0]) # (20/100) * 9568 = 1913.6. After flooring, 1913\n", 126 | "num_of_training_example = total_X.shape[0] - num_of_testing_example # 9568 - 1913 = 7655\n", 127 | "X = total_X[:num_of_training_example] #matrix of size 7655 X 4. input X (according to the pdf note)\n", 128 | "y = total_y[:num_of_training_example] #vector of size 7655. input y (according to the pdf note)\n", 129 | "X_test = total_X[num_of_training_example:] #matrix of size 1913 X 4\n", 130 | "y_test = total_y[num_of_training_example:] #vector of size 1913" 131 | ] 132 | }, 133 | { 134 | "cell_type": "code", 135 | "execution_count": null, 136 | "metadata": { 137 | "id": "4kmMoNR3HU9q" 138 | }, 139 | "outputs": [], 140 | "source": [ 141 | "theta = np.random.rand((X.shape[1])) #randomly initialize theta as a vector of size 5" 142 | ] 143 | }, 144 | { 145 | "cell_type": "markdown", 146 | "source": [ 147 | "Parameters :\n", 148 | "\n", 149 | " data_point : Batches of rows in the training set matrix\n", 150 | " theta: theta vector\n", 151 | "\n", 152 | " Process : matrix (data points) multiplies vector (theta) \n", 153 | "\n", 154 | " Output : prediction of the model, y_hat or h(X)" 155 | ], 156 | "metadata": { 157 | "id": "c1tjdBu4Icnr" 158 | } 159 | }, 160 | { 161 | "cell_type": "code", 162 | "execution_count": null, 163 | "metadata": { 164 | "id": "e_DkDC3SHU9r" 165 | }, 166 | "outputs": [], 167 | "source": [ 168 | "def find_y_hat(data_point,theta):\n", 169 | " return np.dot(data_point, theta) " 170 | ] 171 | }, 172 | { 173 | "cell_type": "code", 174 | "execution_count": null, 175 | "metadata": { 176 | "id": "MVhlFsvtHU9s" 177 | }, 178 | "outputs": [], 179 | "source": [ 180 | "def testing_with_MSE():\n", 181 | " \n", 182 | " diff = X_test.dot(theta) - y_test #calculate (h(x) - y)\n", 183 | " \n", 184 | " loss = (1/(2*X_test.shape[0]))*(diff.dot(diff)) #calculate the loss using the defined equation\n", 185 | " \n", 186 | " print(\"Testing loss with MSE is : \", loss)\n", 187 | " \n", 188 | "def testing_with_R2():\n", 189 | " \n", 190 | " diff = X_test.dot(theta) - y_test #calculates h(x) - y\n", 191 | "\n", 192 | " u = sum(np.square(diff)) #variance of the data from the prediction model\n", 193 | " v = sum(np.square(y_test - np.mean(y_test))) #variance of the original data\n", 194 | "\n", 195 | " print(\"The testing loss with R2 is :\", (1 - (u/v))) #print the R2 coefficient\n", 196 | "\n", 197 | "\n", 198 | "testing_with_MSE() #run the testing with MSE before the training begins\n", 199 | "testing_with_R2() #run the testing with R2 before the training begins" 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": null, 205 | "metadata": { 206 | "id": "x9DAlWRVHU9s" 207 | }, 208 | "outputs": [], 209 | "source": [ 210 | "for epoch in range(num_of_epoch):\n", 211 | " \n", 212 | " diff = X.dot(theta) - y #calculate (h(x) - y)\n", 213 | " \n", 214 | " loss = (1/(2*X.shape[0]))*(diff.dot(diff)) #calculate the loss using the defined equation\n", 215 | " \n", 216 | " for index_first in range(0, X.shape[0], batch_size): #loop through the dataset with the given batch size\n", 217 | " \n", 218 | " index_last = index_first + batch_size #index_first - index_last = batch size\n", 219 | " index_last = None if index_last > X.shape[0] else index_last #if index_last > total no. of data points, it will be set to None\n", 220 | " \n", 221 | " y_hat = find_y_hat(X[index_first:index_last], theta) #predicted output\n", 222 | " \n", 223 | " m = y_hat.shape[0] # number of data points i.e. batch size\n", 224 | " \n", 225 | " y_hat_diff = y_hat - y[index_first:index_last] #calculate (h(x) - y)\n", 226 | " \n", 227 | " \n", 228 | " for j in range(theta.shape[0]): #gradient descent\n", 229 | " \n", 230 | " x_j = X[index_first:index_last, j] #get the x_j vector for that specified batch size\n", 231 | " \n", 232 | " par_der = (1/m)*(y_hat_diff.dot(x_j)) #calculate the partial derivative\n", 233 | " \n", 234 | " theta[j] = theta[j] - learning_rate*par_der\n", 235 | " \n", 236 | " if epoch % 1000 == 0: #print loss for every 500 epoch starting from 0th epoch\n", 237 | " print(\"Epoch : \", epoch)\n", 238 | " print(\"Loss : \", loss)\n", 239 | " " 240 | ] 241 | }, 242 | { 243 | "cell_type": "code", 244 | "execution_count": null, 245 | "metadata": { 246 | "id": "NtVREv88HU9u" 247 | }, 248 | "outputs": [], 249 | "source": [ 250 | "print(theta)" 251 | ] 252 | }, 253 | { 254 | "cell_type": "code", 255 | "execution_count": null, 256 | "metadata": { 257 | "id": "jFHrWhu7HU9u" 258 | }, 259 | "outputs": [], 260 | "source": [ 261 | "testing_with_MSE() #as expected gradient descent optimization works slightly worse than the normal equation method.\n", 262 | "testing_with_R2()" 263 | ] 264 | }, 265 | { 266 | "cell_type": "code", 267 | "execution_count": null, 268 | "metadata": { 269 | "id": "AdhHZ-wDHU9v" 270 | }, 271 | "outputs": [], 272 | "source": [ 273 | "X_test.dot(theta)- y_test" 274 | ] 275 | } 276 | ], 277 | "metadata": { 278 | "kernelspec": { 279 | "display_name": "nasa", 280 | "language": "python", 281 | "name": "nasa" 282 | }, 283 | "language_info": { 284 | "codemirror_mode": { 285 | "name": "ipython", 286 | "version": 3 287 | }, 288 | "file_extension": ".py", 289 | "mimetype": "text/x-python", 290 | "name": "python", 291 | "nbconvert_exporter": "python", 292 | "pygments_lexer": "ipython3", 293 | "version": "3.5.2" 294 | }, 295 | "colab": { 296 | "provenance": [] 297 | } 298 | }, 299 | "nbformat": 4, 300 | "nbformat_minor": 0 301 | } -------------------------------------------------------------------------------- /traditional_ML_algorithms/Multiple Linear Regression/README.md: -------------------------------------------------------------------------------- 1 | ## Multiple Linear Regression 2 | 3 | ### Packages used: 4 | 5 | `Numpy` `Pandas` 6 | 7 | ### About: 8 | 9 | 1. Linear Regression is a machine learning algorithm. 10 | 11 | 2. It attempts to model the relationship between TWO variables by fitting 12 | a ”best-fit” line to the observed data points where the ”best-fit” line has 13 | the minimum sum of the squares of the vertical distance from each data 14 | point to the ”best-fit” line. 15 | 16 | 3. The method of minimizing the sum of the squares of the vertical distance 17 | from each data point to the line is known as the method of least-squares. 18 | 19 | 4. The variables in Linear Regression is known as dependent variable and 20 | independent variable. The idea is to derive the independent variable using 21 | the dependent variable. 22 | 23 | 5. In Multiple Linear Regression, there are more than one dependent variable 24 | and exactly one independent variable. 25 | -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/Eigenfaces_Face_Recognition.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/Eigenfaces_Face_Recognition.pdf -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/archive.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/archive.zip -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/readme.md: -------------------------------------------------------------------------------- 1 | # Implementation of Principal Component Analysis 2 | Face Recognition is an important implementation of PCA. It uses eigenvectors of the covariance vectors of the image matrix to detect and recognize faces. 3 | In this repo, I have provided the implementation along with a document for reference. 4 | -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test1.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test10.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test10.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test2.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test3.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test4.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test4.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test5.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test6.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test6.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test7.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test7.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test8.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test8.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test9.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_FaceRecognition/test9.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/Eigenfaces_Face_Recognition.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/Eigenfaces_Face_Recognition.pdf -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/PCA_Face_Recognition.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "source": [ 6 | "# README\n", 7 | "\n", 8 | "Import archive.zip, test1.jpg, test2.jpg, test3.jpg, test4.jpg, test5.jpg, test6.jpg, test7.jpg, test8.jpg, test9.jpg, and test10.jpg from the assignment folder before the colab file. \n", 9 | "\n", 10 | "I had imported it in sample_data. You can follow the same, or else can change the datapath in the corresponding code sections." 11 | ], 12 | "metadata": { 13 | "id": "IQ6_KflU86Q5" 14 | } 15 | }, 16 | { 17 | "cell_type": "code", 18 | "execution_count": null, 19 | "metadata": { 20 | "id": "H5_qr8Gwnvek" 21 | }, 22 | "outputs": [], 23 | "source": [ 24 | "# Import all the necessities for the project.\n", 25 | "import os, glob\n", 26 | "from sklearn import preprocessing\n", 27 | "import cv2\n", 28 | "import numpy as np\n", 29 | "import matplotlib.pyplot as plt\n", 30 | "import math" 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "source": [ 36 | "# Dataset\n", 37 | "The archive.zip contains the AT&T database of faces. It contains 400 images ( 10 images each of 40 persons). I have used the 9 image each of every person for training and 1 image each is left for the test set." 38 | ], 39 | "metadata": { 40 | "id": "mGi1PJg555Qx" 41 | } 42 | }, 43 | { 44 | "cell_type": "code", 45 | "source": [ 46 | "from zipfile import ZipFile\n", 47 | "file_name = '/content/sample_data/archive.zip'\n", 48 | "\n", 49 | "with ZipFile(file_name, 'r') as zip:\n", 50 | " zip.extractall('/content/sample_data/faces')\n", 51 | " print('Done')\n", 52 | "\n", 53 | "path = '/content/sample_data/faces' #path to the dataset" 54 | ], 55 | "metadata": { 56 | "colab": { 57 | "base_uri": "https://localhost:8080/" 58 | }, 59 | "id": "b0K0lEp1qWVs", 60 | "outputId": "646fa50c-e683-4883-a942-86f3fb8ef579" 61 | }, 62 | "execution_count": null, 63 | "outputs": [ 64 | { 65 | "output_type": "stream", 66 | "name": "stdout", 67 | "text": [ 68 | "Done\n" 69 | ] 70 | } 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": null, 76 | "metadata": { 77 | "id": "MlE20eqOnveo" 78 | }, 79 | "outputs": [], 80 | "source": [ 81 | "# This function plots the images\n", 82 | "def plot_images(images, titles, h, w, row, col):\n", 83 | " plt.figure(figsize=(2.5*col,2.5*row))\n", 84 | " plt.subplots_adjust(bottom=0, left=.01, right=.99, top=.90, hspace=.20)\n", 85 | " for i in range(row * col):\n", 86 | " plt.subplot(row, col, i + 1)\n", 87 | " plt.imshow(images[i].reshape((h, w)), cmap=plt.cm.gray)\n", 88 | " plt.title(titles[i])\n", 89 | " plt.xticks(())\n", 90 | " plt.yticks(())" 91 | ] 92 | }, 93 | { 94 | "cell_type": "code", 95 | "execution_count": null, 96 | "metadata": { 97 | "id": "FVBoqYUYnvep", 98 | "outputId": "3d4c8dc9-7eaa-4a1b-e135-eabba8e268c7", 99 | "colab": { 100 | "base_uri": "https://localhost:8080/" 101 | } 102 | }, 103 | "outputs": [ 104 | { 105 | "output_type": "stream", 106 | "name": "stdout", 107 | "text": [ 108 | "360\n" 109 | ] 110 | } 111 | ], 112 | "source": [ 113 | "# Counts the total number of images\n", 114 | "total_images = 0\n", 115 | "shape = None\n", 116 | "for images in glob.glob(path + '/**', recursive=True): # The glob module is used to retrieve files/pathnames matching a specified pattern\n", 117 | " if images[-3:] == 'pgm' or images[-3:] == 'jpg':\n", 118 | " total_images += 1\n", 119 | "\n", 120 | "# 40 images, i.e. 1 for each person is separated for testing purposes\n", 121 | "total_images = total_images - 40\n", 122 | "print(total_images)\n", 123 | "\n" 124 | ] 125 | }, 126 | { 127 | "cell_type": "markdown", 128 | "source": [ 129 | "The code section below plots the images in the training set." 130 | ], 131 | "metadata": { 132 | "id": "Fk2Fl-WC6VBK" 133 | } 134 | }, 135 | { 136 | "cell_type": "code", 137 | "execution_count": null, 138 | "metadata": { 139 | "id": "BpFos0BRnveq" 140 | }, 141 | "outputs": [], 142 | "source": [ 143 | "# Size of the images\n", 144 | "shape = (112,92) \n", 145 | "\n", 146 | "# Initialize the array which contains all the images\n", 147 | "train_images = np.zeros((total_images, shape[0], shape[1]) ,dtype='float64')\n", 148 | "test_images = np.zeros((40, shape[0], shape[1]) ,dtype='float64')\n", 149 | "test_images_dict = dict()\n", 150 | "id_train = list()\n", 151 | "id_test = list()\n", 152 | "i = 0\n", 153 | "j = 0\n", 154 | "# Traverse through the folder which contains all the faces\n", 155 | "for folder in glob.glob(path + '/*'):\n", 156 | " # Iterates through all the 10 images in folder and appends in id_train list()\n", 157 | " for q in range(10):\n", 158 | " if q == 9:\n", 159 | " id_test.append(folder[-3:].replace('/', ''))\n", 160 | " else:\n", 161 | " id_train.append(folder[-3:].replace('/', ''))\n", 162 | " # print(folder[-3:])\n", 163 | " \n", 164 | " # Iterates through all the images in folder and add in train_images\n", 165 | " k = 0\n", 166 | " for image in glob.glob(folder + '/*'):\n", 167 | " read_image = cv2.imread(image, cv2.IMREAD_GRAYSCALE)\n", 168 | " resized_image = cv2.resize(read_image, (shape[1], shape[0]))\n", 169 | " if k == 9:\n", 170 | " test_images[j] = np.array(resized_image)\n", 171 | " test_images_dict[j] = resized_image\n", 172 | " j = j + 1\n", 173 | " else:\n", 174 | " train_images[i] = np.array(resized_image)\n", 175 | " i = i + 1\n", 176 | " # print(train_images[i])\n", 177 | " k = k + 1\n", 178 | "\n", 179 | "# Plots row*col images (20 in this case), first 20 in training set\n", 180 | "plot_images(train_images, id_train, 112,92, 2, 10)\n", 181 | "\n", 182 | "# print(id_train, id_test)" 183 | ] 184 | }, 185 | { 186 | "cell_type": "markdown", 187 | "source": [ 188 | "The code section below plots the images in test set\n" 189 | ], 190 | "metadata": { 191 | "id": "vbmsQls_6di3" 192 | } 193 | }, 194 | { 195 | "cell_type": "code", 196 | "source": [ 197 | "# Plots row*col images (40 in this case, All in test set\n", 198 | "plot_images(test_images, id_test, 112, 92, 4, 10)" 199 | ], 200 | "metadata": { 201 | "id": "G96SkjqaD1gw" 202 | }, 203 | "execution_count": null, 204 | "outputs": [] 205 | }, 206 | { 207 | "cell_type": "markdown", 208 | "source": [ 209 | "# The Average Face\n", 210 | "The code section plots the mean face of the training set." 211 | ], 212 | "metadata": { 213 | "id": "LnjPj24k6kD3" 214 | } 215 | }, 216 | { 217 | "cell_type": "code", 218 | "execution_count": null, 219 | "metadata": { 220 | "id": "jblkl_6Ynveq" 221 | }, 222 | "outputs": [], 223 | "source": [ 224 | "# Convert the nxn image vectors inside train_images matrix to 1 x n^2 matrix and resize\n", 225 | "# Finally, F is of the order m x n^2 where m = number of images\n", 226 | "F = np.resize(train_images, (total_images, shape[0]*shape[1]))\n", 227 | "\n", 228 | "# The mean matrix is calculated by summing the elements of F and then dividing by total_images\n", 229 | "average_vector = np.sum(F, axis=0, dtype='float64')/total_images\n", 230 | "\n", 231 | "# Duplicate average vector into m x n^2 matrix to calculate differences\n", 232 | "average_matrix = np.tile(average_vector, (total_images, 1)) \n", 233 | "\n", 234 | "# Mean subtracted image matrix\n", 235 | "F_tilde = F - average_matrix\n", 236 | "\n", 237 | "# Plot the mean image\n", 238 | "plt.imshow(np.resize(average_vector, (shape[0],shape[1])), cmap='gray')\n", 239 | "plt.title('Average Face')\n", 240 | "plt.show()" 241 | ] 242 | }, 243 | { 244 | "cell_type": "markdown", 245 | "source": [ 246 | "# Mean Subtracted Image" 247 | ], 248 | "metadata": { 249 | "id": "kiordPue6xKC" 250 | } 251 | }, 252 | { 253 | "cell_type": "code", 254 | "execution_count": null, 255 | "metadata": { 256 | "id": "O_qPGdZxnver" 257 | }, 258 | "outputs": [], 259 | "source": [ 260 | "# F_tilde contains vectors of each mean subtracted image\n", 261 | "plot_images(F_tilde, id_train, 112,92, 2, 10)" 262 | ] 263 | }, 264 | { 265 | "cell_type": "markdown", 266 | "source": [ 267 | "# Eigenfaces" 268 | ], 269 | "metadata": { 270 | "id": "-NOZivDp6165" 271 | } 272 | }, 273 | { 274 | "cell_type": "code", 275 | "execution_count": null, 276 | "metadata": { 277 | "id": "8vd0eYWjnves" 278 | }, 279 | "outputs": [], 280 | "source": [ 281 | "# L is MxM matrix as mentioned in the paper and is used to find out eigenvectors\n", 282 | "L = (F_tilde.dot(F_tilde.T))/total_images\n", 283 | "print(\"L shape : \", L.shape)\n", 284 | "\n", 285 | "# Calculate eigenvalues and eigenvectors from L\n", 286 | "eigenvalues, eigenvectors = np.linalg.eig(L)\n", 287 | "\n", 288 | "# Sort the eigenvalues and eigenvectors in descending order to obtain highest\n", 289 | "# eigenvalue by using the indices\n", 290 | "idx = eigenvalues.argsort()[::-1]\n", 291 | "eigenvalues = eigenvalues[idx] \n", 292 | "eigenvectors = eigenvectors[:, idx]\n", 293 | "\n", 294 | "# Matrix multiplication by linear combinarion of each column of F_tilde\n", 295 | "# Resultant contains eigenvector of C in each column\n", 296 | "eigenvectors_C = F_tilde.T @ eigenvectors\n", 297 | "print(\"Eigenvectors shape : \", eigenvectors_C.shape)\n", 298 | "\n", 299 | "# Normalize eigenvectors to get eigenfaces\n", 300 | "eigenfaces = preprocessing.normalize(eigenvectors_C.T)\n", 301 | "print(\"Eigenfaces shape : \", eigenfaces.shape)\n", 302 | "\n", 303 | "# Plots first 20 eigenfaces\n", 304 | "eigenface_labels = [x for x in range(eigenfaces.shape[0])]\n", 305 | "plot_images(eigenfaces, eigenface_labels , 112,92, 2, 10) \n" 306 | ] 307 | }, 308 | { 309 | "cell_type": "markdown", 310 | "source": [ 311 | "# Mean Subtracted Images of the Test Set" 312 | ], 313 | "metadata": { 314 | "id": "vlC2-b_r65pR" 315 | } 316 | }, 317 | { 318 | "cell_type": "code", 319 | "execution_count": null, 320 | "metadata": { 321 | "id": "cXiCILZanveu" 322 | }, 323 | "outputs": [], 324 | "source": [ 325 | "# Stores all the vectors that represent image with respect to the eigenfaces\n", 326 | "omega = dict()\n", 327 | "\n", 328 | "# Stores the subtracted mean of the testing images\n", 329 | "mean_subtracted_testing_dict = dict()\n", 330 | "\n", 331 | "# The number of chosen eigenfaces\n", 332 | "q = 350\n", 333 | "\n", 334 | "def omega_mst(test_image):\n", 335 | " mean_subtracted_testing = np.reshape(test_image, (test_image.shape[0]*test_image.shape[1])) - average_vector\n", 336 | " omega = eigenfaces[:q].dot(mean_subtracted_testing)\n", 337 | " return (mean_subtracted_testing, omega)\n", 338 | " \n", 339 | "# Store the values\n", 340 | "for j in test_images_dict:\n", 341 | " mean_subtracted_testing_dict[j] = omega_mst(test_images_dict[j])[0]\n", 342 | " omega[j] = omega_mst(test_images_dict[j])[1]\n", 343 | " # print(omega.shape)\n", 344 | "\n", 345 | "# fig = plt.figure(figsize=(10, 7))\n", 346 | " \n", 347 | "# # setting values to rows and column variables\n", 348 | "# rows = 3\n", 349 | "# columns = 3\n", 350 | "\n", 351 | "# # Plot the mean subtracted test image\n", 352 | "for j in test_images_dict:\n", 353 | " plt.imshow(np.reshape(mean_subtracted_testing_dict[j], (112,92)), cmap='gray')\n", 354 | " plt.title(\"Mean Subtracted Test Image\")\n", 355 | " plt.show()\n", 356 | "\n", 357 | " # fig.add_subplot(rows, columns, j+1)\n", 358 | "\n", 359 | " # # showing image\n", 360 | " # plt.imshow(np.reshape(mean_subtracted_testing_dict[j], (112,92)), cmap='gray')\n", 361 | " # plt.axis('off')\n", 362 | " # plt.title(str(j))\n", 363 | "\n", 364 | "def mst_face(mst):\n", 365 | " plt.imshow(np.reshape(mst, (112,92)), cmap='gray')\n", 366 | " plt.title(\"Mean Subtracted Test Image\")\n", 367 | " plt.show()\n" 368 | ] 369 | }, 370 | { 371 | "cell_type": "markdown", 372 | "source": [ 373 | "# Reconstructed Images of the Test Set" 374 | ], 375 | "metadata": { 376 | "id": "q_qAXpsy6_1b" 377 | } 378 | }, 379 | { 380 | "cell_type": "code", 381 | "execution_count": null, 382 | "metadata": { 383 | "id": "67TO5fYznvev" 384 | }, 385 | "outputs": [], 386 | "source": [ 387 | "# To plot the reconstructed face image\n", 388 | "def face_reconstruct(omega_val):\n", 389 | " # Image reconstruction depends on q eigenfaces\n", 390 | " reconstructed = eigenfaces[:q].T.dot(omega_val) \n", 391 | " # print(reconstructed.shape)\n", 392 | "\n", 393 | " plt.imshow(np.reshape(reconstructed, (shape[0],shape[1])), cmap='gray')\n", 394 | " plt.title(\"Reconstructed image - \"+str(q)+\" eigenfaces\")\n", 395 | " plt.show()\n", 396 | "\n", 397 | "# Plot the reconstructed face image\n", 398 | "for i in omega:\n", 399 | " face_reconstruct(omega[i])\n", 400 | "\n", 401 | "# fig = plt.figure(figsize=(10, 7))\n", 402 | " \n", 403 | "# # setting values to rows and column variables\n", 404 | "# rows = 3\n", 405 | "# columns = 3\n", 406 | "\n", 407 | "# # Plot the mean subtracted test image\n", 408 | "# for j in range(9):\n", 409 | "# # plt.imshow(np.reshape(mean_subtracted_testing_dict[j], (112,92)), cmap='gray')\n", 410 | "# # plt.title(\"Mean Subtracted Test Image\")\n", 411 | "# # plt.show()\n", 412 | "\n", 413 | "# fig.add_subplot(rows, columns, j+1)\n", 414 | "\n", 415 | "# # showing image\n", 416 | "# reconstructed = eigenfaces[:q].T.dot(omega[j]) \n", 417 | "# # print(reconstructed.shape)\n", 418 | "# plt.imshow(np.reshape(reconstructed, (shape[0],shape[1])), cmap='gray')\n", 419 | "# plt.axis('off')\n", 420 | "# plt.title(str(j))\n" 421 | ] 422 | }, 423 | { 424 | "cell_type": "markdown", 425 | "source": [ 426 | "# Function for face detection\n", 427 | "Detects the face using omega and mean subtracted testing values of a particular image. The correct value of alpha_1 is obtained by trial and error. It is 3000 in this case." 428 | ], 429 | "metadata": { 430 | "id": "Nrc-Co2N7Kdh" 431 | } 432 | }, 433 | { 434 | "cell_type": "code", 435 | "execution_count": null, 436 | "metadata": { 437 | "id": "NjTF0xT6nvew" 438 | }, 439 | "outputs": [], 440 | "source": [ 441 | "def face_detect(omega_val, mean_subtracted_testing_val):\n", 442 | " # Threshold for face detection\n", 443 | " alpha_1 = 3000\n", 444 | "\n", 445 | " # nxn vector of the test face image represented as the linear combination \n", 446 | " # of the chosen eigenfaces\n", 447 | " projected_new_img_vector = eigenfaces[:q].T @ omega_val \n", 448 | "\n", 449 | " # Beta is the distance between the original face image vector and \n", 450 | " # the projected vector. Compared with the threshold.\n", 451 | " diff = mean_subtracted_testing_val - projected_new_img_vector \n", 452 | " beta = math.sqrt(diff.dot(diff)) \n", 453 | "\n", 454 | " if beta < alpha_1:\n", 455 | " print(\"Face detected! \", beta)\n", 456 | " else:\n", 457 | " print(\"No face detected! \", beta)\n", 458 | "\n", 459 | "\n", 460 | "for i in omega:\n", 461 | " face_detect(omega[i], mean_subtracted_testing_dict[i])" 462 | ] 463 | }, 464 | { 465 | "cell_type": "markdown", 466 | "source": [ 467 | "# Function for Face Recognition\n", 468 | "Recognizes faces using the omega value of a particular image. The correct value of alpha_2 is obtained by trial and error. It is 3200 in this case. It returns 4 things.Firstly, it returns all the matching indexes whose value is lower than the threshold. Then it selects the smallest of those values and return the index and the correponsding label/id." 469 | ], 470 | "metadata": { 471 | "id": "LWS8Gd687Wt9" 472 | } 473 | }, 474 | { 475 | "cell_type": "code", 476 | "execution_count": null, 477 | "metadata": { 478 | "id": "IJaMmeh1nvew" 479 | }, 480 | "outputs": [], 481 | "source": [ 482 | "def face_recognize(omega_val):\n", 483 | " # Threshold for face recognition\n", 484 | " alpha_2 = 3200\n", 485 | "\n", 486 | " # Keep track of the smallest value for face recognition\n", 487 | " smallest_value = None\n", 488 | "\n", 489 | " # The face image/class that produces the smallest value\n", 490 | " index = None \n", 491 | "\n", 492 | " # Indexes of matching images to the given test image \n", 493 | " matching_indexes = list()\n", 494 | "\n", 495 | " # Calculate pattern vector and distance to each know class (face image in the\n", 496 | " # traingin dataset)\n", 497 | " for k in range(total_images):\n", 498 | " omega_k = eigenfaces[:q].dot(F_tilde[k]) \n", 499 | " diff = omega_val - omega_k\n", 500 | " epsilon_k = math.sqrt(diff.dot(diff))\n", 501 | " if smallest_value == None:\n", 502 | " smallest_value = epsilon_k\n", 503 | " index = k\n", 504 | " if smallest_value > epsilon_k:\n", 505 | " smallest_value = epsilon_k\n", 506 | " index = k\n", 507 | " matching_indexes.append(k)\n", 508 | "\n", 509 | " if smallest_value < alpha_2:\n", 510 | " # index += 1\n", 511 | " # print(smallest_value, id_train[index])\n", 512 | " return (smallest_value, id_train[index], index, matching_indexes)\n", 513 | " else:\n", 514 | " # print(smallest_value, \"Unknown Face!\")\n", 515 | " return (smallest_value, \"Unknown Face!\", index, matching_indexes)\n", 516 | "\n", 517 | "print(\"Matching indexes: \")\n", 518 | "for i in omega:\n", 519 | " res = face_recognize(omega[i])\n", 520 | " if res[1] == \"Unknown Face\":\n", 521 | " print(str(i) + \". \" + \"Above Threshold Smallest index = \" + str(res[2]))\n", 522 | " print(\"Matching indexes = \" + str(res[3]))\n", 523 | " else:\n", 524 | " print(str(i) + \". \" + \"Below Threshold Smallest index = \" + str(res[2]))\n", 525 | " print(\"Matching indexes = \" + str(res[3]))\n", 526 | "\n", 527 | "print(\" \")\n", 528 | "print(\"Result: \")\n", 529 | "accuracy = 0\n", 530 | "for i in omega:\n", 531 | " res = face_recognize(omega[i])\n", 532 | " if res[1] == \"Unknown Face\":\n", 533 | " print(str(i) + \". \" + \"Actual-Id: \" + str(id_test[i]) + \" Result: Unkown Face! smallest_value: \" + str(res[0]))\n", 534 | " elif res[1] != id_test[i]:\n", 535 | " print(str(i) + \". \" + \"Actual-Id: \" + str(id_test[i]) + \" Result: \" + str(res[1]) + \" smallest_value: \" + str(res[0]))\n", 536 | " else:\n", 537 | " print(str(i) + \". \" + \"Actual-Id: \" + str(id_test[i]) + \" Result: \" + str(res[1]) + \" smallest_value: \" + str(res[0]))\n", 538 | " accuracy += 1\n", 539 | " \n", 540 | "percent_accuracy = (accuracy / 40) * 100\n", 541 | "print(\"Correctly recognized: \" + str(accuracy) + \" images out of 40 images.\")\n", 542 | "print(\"Accuracy: \" + str(percent_accuracy))\n", 543 | "\n" 544 | ] 545 | }, 546 | { 547 | "cell_type": "markdown", 548 | "source": [ 549 | "# Random Test Set\n", 550 | "After testing my algorithm on the test above, I tried using a random test set with very limited images, i.e., only 10 as of now to test the face detection and recogniton. These 10 images are not from any internationally recognized standard datasets. These are just some random images that I found on internet. The algorithm performs satifactorily in this case too. \n", 551 | "\n", 552 | "You can also see the mean subtracted images and the reconstructed images by running the code section present below." 553 | ], 554 | "metadata": { 555 | "id": "pCgmH7Ic72rJ" 556 | } 557 | }, 558 | { 559 | "cell_type": "code", 560 | "source": [ 561 | "for i in range(1, 11):\n", 562 | " test_img = cv2.imread('/content/sample_data/test'+ str(i)+'.jpg', cv2.IMREAD_GRAYSCALE) #testing image\n", 563 | " test_img = cv2.resize(test_img, (shape[1],shape[0])) #resize the testing image. cv2 resize by width and height.\n", 564 | " mean_subtracted_testing, omega = omega_mst(test_img)[0], omega_mst(test_img)[1]\n", 565 | " mst_face(mean_subtracted_testing)\n", 566 | " face_reconstruct(omega)\n", 567 | " face_detect(omega, mean_subtracted_testing)\n", 568 | " res = face_recognize(omega)\n", 569 | " print(res[0], res[1], res[2], res[3])" 570 | ], 571 | "metadata": { 572 | "id": "pGJye_KWb7um" 573 | }, 574 | "execution_count": null, 575 | "outputs": [] 576 | } 577 | ], 578 | "metadata": { 579 | "kernelspec": { 580 | "display_name": "Python 3", 581 | "name": "python3" 582 | }, 583 | "language_info": { 584 | "name": "python" 585 | }, 586 | "colab": { 587 | "provenance": [], 588 | "collapsed_sections": [] 589 | }, 590 | "accelerator": "GPU" 591 | }, 592 | "nbformat": 4, 593 | "nbformat_minor": 0 594 | } -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/archive.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/archive.zip -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test1.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test10.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test10.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test2.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test3.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test4.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test4.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test5.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test6.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test6.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test7.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test7.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test8.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test8.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test9.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Math-behind-AI/ScratchAI/0bde50d72f85e0a0a8ba0fcb925890fb4171d5c5/traditional_ML_algorithms/Principal Component Analysis/PCA_Face_Recognition/test9.jpg -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/Principal_Component_Analysis.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "source": [ 6 | "## README\n", 7 | "To run the file, upload the data.csv file. In my case, I have uploaded it to sample_data folder in session storage. You can upload it elsewhere but remember to change the path mentioned in code afterwards." 8 | ], 9 | "metadata": { 10 | "id": "YHDi1DNIb6zw" 11 | } 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "source": [ 16 | "# Principal Component Analysis\n", 17 | "Principal Component Analysis (PCA) is an algorithm that reduces the dimen\u0002sionality of a data set to a lower-dimensional linear subspace by linear projection\n", 18 | "in such a way that the reconstruction error made by the linear projection is as\n", 19 | "low as possible." 20 | ], 21 | "metadata": { 22 | "id": "aNiQmgXNbaxJ" 23 | } 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": 18, 28 | "metadata": { 29 | "id": "qmZwMukZKyXr" 30 | }, 31 | "outputs": [], 32 | "source": [ 33 | "import numpy as np\n", 34 | "import pandas as pd\n", 35 | "from sklearn.preprocessing import StandardScaler" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "metadata": { 42 | "id": "38pLsddIKyXw" 43 | }, 44 | "outputs": [], 45 | "source": [ 46 | "data = pd.read_csv('/content/sample_data/data.csv')\n", 47 | "data.head()" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": 20, 53 | "metadata": { 54 | "id": "M-5p83JXKyXy" 55 | }, 56 | "outputs": [], 57 | "source": [ 58 | "# X should be transposed to convert rows to features and columns to data points\n", 59 | "# 12 x 517 size matrix where there are 12 features and 517 data points\n", 60 | "X = np.transpose(data.iloc[:, :-1].values)\n", 61 | "\n", 62 | "# Get the last column of the matrix\n", 63 | "label = data.iloc[:, -1].values\n", 64 | "\n", 65 | "# X = StandardScaler().fit_transform(X) #standardize the data\n", 66 | "# X.shape" 67 | ] 68 | }, 69 | { 70 | "cell_type": "code", 71 | "execution_count": 21, 72 | "metadata": { 73 | "id": "Vqn7TLhkKyXz" 74 | }, 75 | "outputs": [], 76 | "source": [ 77 | "mean_vector = np.mean(X, axis=1)\n", 78 | "mean_matrix = np.tile(mean_vector, (X.shape[1],1))\n", 79 | "X = X - mean_matrix.T" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": 22, 85 | "metadata": { 86 | "id": "1dfX5uokKyX0" 87 | }, 88 | "outputs": [], 89 | "source": [ 90 | "# Covariance matrix of X\n", 91 | "Cov = (np.dot(X, X.T))/(X.shape[1])\n", 92 | "# Cov = np.cov(X)\n", 93 | "\n", 94 | "# Get the dimension of the covariance matrix\n", 95 | "n = Cov.shape[0] \n", 96 | "\n", 97 | "q = 5 #dimension of the lower-dimensional subspace\n", 98 | "# Cov.shape" 99 | ] 100 | }, 101 | { 102 | "cell_type": "markdown", 103 | "source": [ 104 | "# Reconstruction" 105 | ], 106 | "metadata": { 107 | "id": "knqvzxPZbvV8" 108 | } 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": 23, 113 | "metadata": { 114 | "id": "JvTs2QONKyX1" 115 | }, 116 | "outputs": [], 117 | "source": [ 118 | "# Corresponding Eigenvalues and Eigenvectors for Cov.\n", 119 | "eigenvalue, eigenvector = np.linalg.eig(Cov)\n", 120 | "\n", 121 | "# Order in decreasing order\n", 122 | "eigval_eigvec = list(zip(*sorted(zip(eigenvalue,eigenvector), reverse=True)))\n", 123 | "\n", 124 | "# Get eigenvectors in rows instead of columns\n", 125 | "principal_components = np.asarray(list(eigval_eigvec[1][:])).T\n", 126 | "\n", 127 | "P = principal_components[:q] # q x 12 matrix with q leading principal components in each row.\n", 128 | "\n", 129 | "# Transform X to get reconstructed data points\n", 130 | "Y = np.dot(P, X) \n", 131 | "# Y.shape" 132 | ] 133 | }, 134 | { 135 | "cell_type": "code", 136 | "execution_count": 24, 137 | "metadata": { 138 | "colab": { 139 | "base_uri": "https://localhost:8080/" 140 | }, 141 | "id": "q6TK8MmTKyX6", 142 | "outputId": "601a8db2-cdc1-4b27-90d6-7f894c6d959c" 143 | }, 144 | "outputs": [ 145 | { 146 | "output_type": "stream", 147 | "name": "stdout", 148 | "text": [ 149 | "PCA Loss : 0.037985574963925774 %\n" 150 | ] 151 | } 152 | ], 153 | "source": [ 154 | "# Total Loss\n", 155 | "variance_loss = sum(eigval_eigvec[0][q:]) \n", 156 | "\n", 157 | "# Total variance\n", 158 | "total_variance = sum(eigval_eigvec[0][:]) \n", 159 | "\n", 160 | "# Percentage loss\n", 161 | "print(\"PCA Loss : \", (variance_loss/total_variance)*100, \"%\") " 162 | ] 163 | }, 164 | { 165 | "cell_type": "code", 166 | "execution_count": null, 167 | "metadata": { 168 | "id": "ogv_58ZiKyX8" 169 | }, 170 | "outputs": [], 171 | "source": [ 172 | "# To compare the linear regression prediction on the original data points \n", 173 | "# and the reconstructed data points.\n", 174 | "from sklearn.linear_model import LinearRegression\n", 175 | "import datetime\n", 176 | "\n", 177 | "# Fit the original data with the label\n", 178 | "start_time = datetime.datetime.now()\n", 179 | "reg_original_data = LinearRegression().fit(X.T,label) \n", 180 | "\n", 181 | "# Calculate the R2 coefficient on the training data itself\n", 182 | "print(\"The R2 coefficient is : \", reg_original_data.score(X.T, label)) \n", 183 | "\n", 184 | "# Calculate total elapsed time\n", 185 | "end_time = datetime.datetime.now()\n", 186 | "print(\"Total elapsed time : \", end_time - start_time)" 187 | ] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "execution_count": null, 192 | "metadata": { 193 | "id": "io41xcGyKyX9" 194 | }, 195 | "outputs": [], 196 | "source": [ 197 | "# Fit the reconstructed data points with the label\n", 198 | "start_time = datetime.datetime.now()\n", 199 | "reg_pca_data = LinearRegression().fit(Y.T, label) \n", 200 | "\n", 201 | "# Calculate the R2 coefficient with the training data itself\n", 202 | "print(\"The R2 coefficient is : \", reg_pca_data.score(Y.T, label)) \n", 203 | "\n", 204 | "# Calculate total elapsed time\n", 205 | "end_time = datetime.datetime.now()\n", 206 | "print(\"Total elapsed time : \", end_time - start_time)" 207 | ] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "source": [ 212 | "# Observation\n", 213 | "It can be seen that even though the reconstructed data points has a lower R2 coefficient, it saves some time\n", 214 | "during the modelling of the linear regression. The time saved might be small for this example, but it makes a lot\n", 215 | "of difference when it comes to higher dimensional dataset and more complex modelling techniques. Linear Regression\n", 216 | "was just used for simplicity sake to show the working principles of PCA. That is, even though the dimension\n", 217 | "from the original data points has been reduced drastically, the reconstructed data points can still provide\n", 218 | "linear regression enough information to model the relationships between the dependent and independent variables." 219 | ], 220 | "metadata": { 221 | "id": "s_KrCSPwatJs" 222 | } 223 | }, 224 | { 225 | "cell_type": "code", 226 | "execution_count": null, 227 | "metadata": { 228 | "id": "Xm0_oRA_KyX-" 229 | }, 230 | "outputs": [], 231 | "source": [ 232 | "from sklearn.decomposition import PCA #using PCA from the sklearn \n", 233 | "pca = PCA(n_components=q)\n", 234 | "pca.fit(X.T)\n", 235 | "Y_2 = np.transpose(pca.transform(X.T))\n", 236 | "Y_2.shape" 237 | ] 238 | }, 239 | { 240 | "cell_type": "code", 241 | "execution_count": null, 242 | "metadata": { 243 | "id": "Q0F63r1EKyX_" 244 | }, 245 | "outputs": [], 246 | "source": [ 247 | "# Difference between the principal components (my implementation) and Sklearn's implementation\n", 248 | "diff = sum(sum(pca.components_) - sum(P)) \n", 249 | "print(diff)" 250 | ] 251 | }, 252 | { 253 | "cell_type": "markdown", 254 | "source": [ 255 | "## Differences\n", 256 | "The difference between the principal components are not that much. Sklearn's PCA uses normalizations on the data, hence provides a slightly different result than my implementation." 257 | ], 258 | "metadata": { 259 | "id": "QmIU3PpJbMxo" 260 | } 261 | } 262 | ], 263 | "metadata": { 264 | "kernelspec": { 265 | "display_name": "Python 3.10.6 64-bit", 266 | "language": "python", 267 | "name": "python3" 268 | }, 269 | "language_info": { 270 | "codemirror_mode": { 271 | "name": "ipython", 272 | "version": 3 273 | }, 274 | "file_extension": ".py", 275 | "mimetype": "text/x-python", 276 | "name": "python", 277 | "nbconvert_exporter": "python", 278 | "pygments_lexer": "ipython3", 279 | "version": "3.10.6" 280 | }, 281 | "vscode": { 282 | "interpreter": { 283 | "hash": "2a8dfe095fce2b5e88c64a2c3ee084c8e0e0d70b23e7b95b1cfb538be294c5c8" 284 | } 285 | }, 286 | "colab": { 287 | "provenance": [], 288 | "collapsed_sections": [] 289 | } 290 | }, 291 | "nbformat": 4, 292 | "nbformat_minor": 0 293 | } -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/README.md: -------------------------------------------------------------------------------- 1 | ## Principal Component Analysis 2 | 3 | ### Packages used: 4 | 5 | `Numpy` `Pandas` `Sklearn` `Matplotlib` 6 | 7 | ### Brief explanation: 8 | 9 | An algorithm that reduces the dimensionality of a data set to a lower-dimensional linear subspace by linear projection 10 | in such a way that the reconstruction error made by the linear projection is as 11 | low as possible. 12 | 13 | The implementation of Principal Component Analysis on Face Recognition is also present in the repository. -------------------------------------------------------------------------------- /traditional_ML_algorithms/Principal Component Analysis/data.csv: -------------------------------------------------------------------------------- 1 | X,Y,month,day,FFMC,DMC,DC,ISI,temp,RH,wind,rain,area 2 | 7,5,3,6,86.2,26.2,94.3,5.1,8.2,51,6.7,0,0 3 | 7,4,10,3,90.6,35.4,669.1,6.7,18,33,0.9,0,0 4 | 7,4,10,7,90.6,43.7,686.9,6.7,14.6,33,1.3,0,0 5 | 8,6,3,6,91.7,33.3,77.5,9,8.3,97,4,0.2,0 6 | 8,6,3,1,89.3,51.3,102.2,9.6,11.4,99,1.8,0,0 7 | 8,6,8,1,92.3,85.3,488,14.7,22.2,29,5.4,0,0 8 | 8,6,8,2,92.3,88.9,495.6,8.5,24.1,27,3.1,0,0 9 | 8,6,8,2,91.5,145.4,608.2,10.7,8,86,2.2,0,0 10 | 8,6,9,3,91,129.5,692.6,7,13.1,63,5.4,0,0 11 | 7,5,9,7,92.5,88,698.6,7.1,22.8,40,4,0,0 12 | 7,5,9,7,92.5,88,698.6,7.1,17.8,51,7.2,0,0 13 | 7,5,9,7,92.8,73.2,713,22.6,19.3,38,4,0,0 14 | 6,5,8,6,63.5,70.8,665.3,0.8,17,72,6.7,0,0 15 | 6,5,9,2,90.9,126.5,686.5,7,21.3,42,2.2,0,0 16 | 6,5,9,4,92.9,133.3,699.6,9.2,26.4,21,4.5,0,0 17 | 6,5,9,6,93.3,141.2,713.9,13.9,22.9,44,5.4,0,0 18 | 5,5,3,7,91.7,35.8,80.8,7.8,15.1,27,5.4,0,0 19 | 8,5,10,2,84.9,32.8,664.2,3,16.7,47,4.9,0,0 20 | 6,4,3,4,89.2,27.9,70.8,6.3,15.9,35,4,0,0 21 | 6,4,4,7,86.3,27.4,97.1,5.1,9.3,44,4.5,0,0 22 | 6,4,9,3,91,129.5,692.6,7,18.3,40,2.7,0,0 23 | 5,4,9,2,91.8,78.5,724.3,9.2,19.1,38,2.7,0,0 24 | 7,4,6,1,94.3,96.3,200,56.1,21,44,4.5,0,0 25 | 7,4,8,7,90.2,110.9,537.4,6.2,19.5,43,5.8,0,0 26 | 7,4,8,7,93.5,139.4,594.2,20.3,23.7,32,5.8,0,0 27 | 7,4,8,1,91.4,142.4,601.4,10.6,16.3,60,5.4,0,0 28 | 7,4,9,6,92.4,117.9,668,12.2,19,34,5.8,0,0 29 | 7,4,9,2,90.9,126.5,686.5,7,19.4,48,1.3,0,0 30 | 6,3,9,7,93.4,145.4,721.4,8.1,30.2,24,2.7,0,0 31 | 6,3,9,1,93.5,149.3,728.6,8.1,22.8,39,3.6,0,0 32 | 6,3,9,6,94.3,85.1,692.3,15.9,25.4,24,3.6,0,0 33 | 6,3,9,2,88.6,91.8,709.9,7.1,11.2,78,7.6,0,0 34 | 6,3,9,6,88.6,69.7,706.8,5.8,20.6,37,1.8,0,0 35 | 6,3,9,1,91.7,75.6,718.3,7.8,17.7,39,3.6,0,0 36 | 6,3,9,2,91.8,78.5,724.3,9.2,21.2,32,2.7,0,0 37 | 6,3,9,3,90.3,80.7,730.2,6.3,18.2,62,4.5,0,0 38 | 6,3,10,3,90.6,35.4,669.1,6.7,21.7,24,4.5,0,0 39 | 7,4,10,6,90,41.5,682.6,8.7,11.3,60,5.4,0,0 40 | 7,3,10,7,90.6,43.7,686.9,6.7,17.8,27,4,0,0 41 | 4,4,3,3,88.1,25.7,67.6,3.8,14.1,43,2.7,0,0 42 | 4,4,7,3,79.5,60.6,366.7,1.5,23.3,37,3.1,0,0 43 | 4,4,8,7,90.2,96.9,624.2,8.9,18.4,42,6.7,0,0 44 | 4,4,8,3,94.8,108.3,647.1,17,16.6,54,5.4,0,0 45 | 4,4,9,7,92.5,88,698.6,7.1,19.6,48,2.7,0,0 46 | 4,4,9,4,90.1,82.9,735.7,6.2,12.9,74,4.9,0,0 47 | 5,6,9,4,94.3,85.1,692.3,15.9,25.9,24,4,0,0 48 | 5,6,9,2,90.9,126.5,686.5,7,14.7,70,3.6,0,0 49 | 6,6,7,2,94.2,62.3,442.9,11,23,36,3.1,0,0 50 | 4,4,3,2,87.2,23.9,64.7,4.1,11.8,35,1.8,0,0 51 | 4,4,3,2,87.6,52.2,103.8,5,11,46,5.8,0,0 52 | 4,4,9,5,92.9,137,706.4,9.2,20.8,17,1.3,0,0 53 | 4,3,8,1,90.2,99.6,631.2,6.3,21.5,34,2.2,0,0 54 | 4,3,8,4,92.1,111.2,654.1,9.6,20.4,42,4.9,0,0 55 | 4,3,8,4,92.1,111.2,654.1,9.6,20.4,42,4.9,0,0 56 | 4,3,8,5,91.7,114.3,661.3,6.3,17.6,45,3.6,0,0 57 | 4,3,9,5,92.9,137,706.4,9.2,27.7,24,2.2,0,0 58 | 4,3,9,3,90.3,80.7,730.2,6.3,17.8,63,4.9,0,0 59 | 4,3,10,1,92.6,46.5,691.8,8.8,13.8,50,2.7,0,0 60 | 2,2,2,2,84,9.3,34,2.1,13.9,40,5.4,0,0 61 | 2,2,2,6,86.6,13.2,43,5.3,12.3,51,0.9,0,0 62 | 2,2,3,1,89.3,51.3,102.2,9.6,11.5,39,5.8,0,0 63 | 2,2,3,1,89.3,51.3,102.2,9.6,5.5,59,6.3,0,0 64 | 2,2,8,5,93,75.3,466.6,7.7,18.8,35,4.9,0,0 65 | 2,2,8,1,90.2,99.6,631.2,6.3,20.8,33,2.7,0,0 66 | 2,2,8,2,91.1,103.2,638.8,5.8,23.1,31,3.1,0,0 67 | 2,2,8,5,91.7,114.3,661.3,6.3,18.6,44,4.5,0,0 68 | 2,2,9,6,92.4,117.9,668,12.2,23,37,4.5,0,0 69 | 2,2,9,6,92.4,117.9,668,12.2,19.6,33,5.4,0,0 70 | 2,2,9,6,92.4,117.9,668,12.2,19.6,33,6.3,0,0 71 | 4,5,3,6,91.7,33.3,77.5,9,17.2,26,4.5,0,0 72 | 4,5,3,6,91.2,48.3,97.8,12.5,15.8,27,7.6,0,0 73 | 4,5,9,6,94.3,85.1,692.3,15.9,17.7,37,3.6,0,0 74 | 5,4,3,6,91.7,33.3,77.5,9,15.6,25,6.3,0,0 75 | 5,4,8,3,88.8,147.3,614.5,9,17.3,43,4.5,0,0 76 | 5,4,9,6,93.3,141.2,713.9,13.9,27.6,30,1.3,0,0 77 | 9,9,2,5,84.2,6.8,26.6,7.7,6.7,79,3.1,0,0 78 | 9,9,2,6,86.6,13.2,43,5.3,15.7,43,3.1,0,0 79 | 1,3,3,2,87.6,52.2,103.8,5,8.3,72,3.1,0,0 80 | 1,2,8,6,90.1,108,529.8,12.5,14.7,66,2.7,0,0 81 | 1,2,8,3,91,121.2,561.6,7,21.6,19,6.7,0,0 82 | 1,2,8,1,91.4,142.4,601.4,10.6,19.5,39,6.3,0,0 83 | 1,2,8,1,90.2,99.6,631.2,6.3,17.9,44,2.2,0,0 84 | 1,2,8,3,94.8,108.3,647.1,17,18.6,51,4.5,0,0 85 | 1,2,8,4,92.1,111.2,654.1,9.6,16.6,47,0.9,0,0 86 | 1,2,8,5,91.7,114.3,661.3,6.3,20.2,45,3.6,0,0 87 | 1,2,9,5,92.9,137,706.4,9.2,21.5,15,0.9,0,0 88 | 1,2,9,5,92.9,137,706.4,9.2,25.4,27,2.2,0,0 89 | 1,2,9,5,92.9,137,706.4,9.2,22.4,34,2.2,0,0 90 | 1,2,9,1,93.5,149.3,728.6,8.1,25.3,36,3.6,0,0 91 | 6,5,3,7,91.7,35.8,80.8,7.8,17.4,25,4.9,0,0 92 | 6,5,8,7,90.2,96.9,624.2,8.9,14.7,59,5.8,0,0 93 | 8,6,3,6,91.7,35.8,80.8,7.8,17.4,24,5.4,0,0 94 | 8,6,8,1,92.3,85.3,488,14.7,20.8,32,6.3,0,0 95 | 8,6,8,1,91.4,142.4,601.4,10.6,18.2,43,4.9,0,0 96 | 8,6,8,2,91.1,103.2,638.8,5.8,23.4,22,2.7,0,0 97 | 4,4,9,1,89.7,90,704.4,4.8,17.8,64,1.3,0,0 98 | 3,4,2,7,83.9,8,30.2,2.6,12.7,48,1.8,0,0 99 | 3,4,3,7,69,2.4,15.5,0.7,17.4,24,5.4,0,0 100 | 3,4,8,1,91.4,142.4,601.4,10.6,11.6,87,4.5,0,0 101 | 3,4,8,1,91.4,142.4,601.4,10.6,19.8,39,5.4,0,0 102 | 3,4,8,1,91.4,142.4,601.4,10.6,19.8,39,5.4,0,0 103 | 3,4,8,3,88.8,147.3,614.5,9,14.4,66,5.4,0,0 104 | 2,4,8,3,94.8,108.3,647.1,17,20.1,40,4,0,0 105 | 2,4,9,7,92.5,121.1,674.4,8.6,24.1,29,4.5,0,0 106 | 2,4,1,7,82.1,3.7,9.3,2.9,5.3,78,3.1,0,0 107 | 4,5,3,6,85.9,19.5,57.3,2.8,12.7,52,6.3,0,0 108 | 4,5,3,5,91.4,30.7,74.3,7.5,18.2,29,3.1,0,0 109 | 4,5,8,1,90.2,99.6,631.2,6.3,21.4,33,3.1,0,0 110 | 4,5,9,7,92.5,88,698.6,7.1,20.3,45,3.1,0,0 111 | 4,5,9,2,88.6,91.8,709.9,7.1,17.4,56,5.4,0,0 112 | 4,4,3,6,85.9,19.5,57.3,2.8,13.7,43,5.8,0,0 113 | 3,4,3,6,91.7,33.3,77.5,9,18.8,18,4.5,0,0 114 | 3,4,9,1,89.7,90,704.4,4.8,22.8,39,3.6,0,0 115 | 3,4,9,2,91.8,78.5,724.3,9.2,18.9,35,2.7,0,0 116 | 3,4,3,3,88.1,25.7,67.6,3.8,15.8,27,7.6,0,0 117 | 3,5,3,3,88.1,25.7,67.6,3.8,15.5,27,6.3,0,0 118 | 3,4,3,7,91.7,35.8,80.8,7.8,11.6,30,6.3,0,0 119 | 3,4,3,7,91.7,35.8,80.8,7.8,15.2,27,4.9,0,0 120 | 3,4,3,2,90.1,39.7,86.6,6.2,10.6,30,4,0,0 121 | 3,4,8,5,93,75.3,466.6,7.7,19.6,36,3.1,0,0 122 | 3,4,8,2,91.5,145.4,608.2,10.7,10.3,74,2.2,0,0 123 | 3,4,8,2,91.5,145.4,608.2,10.7,17.1,43,5.4,0,0 124 | 3,4,9,1,92.4,124.1,680.7,8.5,22.5,42,5.4,0,0 125 | 3,4,9,3,84.4,73.4,671.9,3.2,17.9,45,3.1,0,0 126 | 3,4,9,6,94.3,85.1,692.3,15.9,19.8,50,5.4,0,0 127 | 3,4,10,1,92.6,46.5,691.8,8.8,20.6,24,5.4,0,0 128 | 3,5,3,2,87.6,52.2,103.8,5,9,49,2.2,0,0 129 | 3,5,9,6,93.5,149.3,728.6,8.1,17.2,43,3.1,0,0 130 | 3,5,10,4,91.4,37.9,673.8,5.2,15.9,46,3.6,0,0 131 | 2,5,10,1,92.6,46.5,691.8,8.8,15.4,35,0.9,0,0 132 | 4,6,2,7,68.2,21.5,87.2,0.8,15.4,40,2.7,0,0 133 | 4,6,3,2,87.2,23.9,64.7,4.1,14,39,3.1,0,0 134 | 4,6,3,1,89.3,51.3,102.2,9.6,10.6,46,4.9,0,0 135 | 4,6,9,5,93.7,80.9,685.2,17.9,17.6,42,3.1,0,0 136 | 3,5,3,3,88.1,25.7,67.6,3.8,14.9,38,2.7,0,0 137 | 3,5,8,7,93.5,139.4,594.2,20.3,17.6,52,5.8,0,0 138 | 3,6,9,1,92.4,124.1,680.7,8.5,17.2,58,1.3,0,0 139 | 3,6,9,2,90.9,126.5,686.5,7,15.6,66,3.1,0,0 140 | 9,9,7,3,85.8,48.3,313.4,3.9,18,42,2.7,0,0.36 141 | 1,4,9,3,91,129.5,692.6,7,21.7,38,2.2,0,0.43 142 | 2,5,9,2,90.9,126.5,686.5,7,21.9,39,1.8,0,0.47 143 | 1,2,8,4,95.5,99.9,513.3,13.2,23.3,31,4.5,0,0.55 144 | 8,6,8,6,90.1,108,529.8,12.5,21.2,51,8.9,0,0.61 145 | 1,2,7,7,90,51.3,296.3,8.7,16.6,53,5.4,0,0.71 146 | 2,5,8,4,95.5,99.9,513.3,13.2,23.8,32,5.4,0,0.77 147 | 6,5,8,5,95.2,131.7,578.8,10.4,27.4,22,4,0,0.9 148 | 5,4,3,2,90.1,39.7,86.6,6.2,13.2,40,5.4,0,0.95 149 | 8,3,9,3,84.4,73.4,671.9,3.2,24.2,28,3.6,0,0.96 150 | 2,2,8,3,94.8,108.3,647.1,17,17.4,43,6.7,0,1.07 151 | 8,6,9,5,93.7,80.9,685.2,17.9,23.7,25,4.5,0,1.12 152 | 6,5,6,6,92.5,56.4,433.3,7.1,23.2,39,5.4,0,1.19 153 | 9,9,7,1,90.1,68.6,355.2,7.2,24.8,29,2.2,0,1.36 154 | 3,4,7,7,90.1,51.2,424.1,6.2,24.6,43,1.8,0,1.43 155 | 5,4,9,6,94.3,85.1,692.3,15.9,20.1,47,4.9,0,1.46 156 | 1,5,9,7,93.4,145.4,721.4,8.1,29.6,27,2.7,0,1.46 157 | 7,4,8,1,94.8,108.3,647.1,17,16.4,47,1.3,0,1.56 158 | 2,4,9,7,93.4,145.4,721.4,8.1,28.6,27,2.2,0,1.61 159 | 2,2,8,4,92.1,111.2,654.1,9.6,18.4,45,3.6,0,1.63 160 | 2,4,8,4,92.1,111.2,654.1,9.6,20.5,35,4,0,1.64 161 | 7,4,9,6,92.4,117.9,668,12.2,19,34,5.8,0,1.69 162 | 7,4,3,2,90.1,39.7,86.6,6.2,16.1,29,3.1,0,1.75 163 | 6,4,8,5,95.2,131.7,578.8,10.4,20.3,41,4,0,1.9 164 | 6,3,3,7,90.6,50.1,100.4,7.8,15.2,31,8.5,0,1.94 165 | 8,6,9,7,92.5,121.1,674.4,8.6,17.8,56,1.8,0,1.95 166 | 8,5,9,1,89.7,90,704.4,4.8,17.8,67,2.2,0,2.01 167 | 6,5,3,5,84.9,18.2,55,3,5.3,70,4.5,0,2.14 168 | 6,5,8,4,92.1,111.2,654.1,9.6,16.6,47,0.9,0,2.29 169 | 6,5,8,4,96,127.1,570.5,16.5,23.4,33,4.5,0,2.51 170 | 6,5,3,6,91.2,48.3,97.8,12.5,14.6,26,9.4,0,2.53 171 | 8,6,8,5,95.2,131.7,578.8,10.4,20.7,45,2.2,0,2.55 172 | 5,4,9,4,92.9,133.3,699.6,9.2,21.9,35,1.8,0,2.57 173 | 8,6,8,4,85.6,90.4,609.6,6.6,17.4,50,4,0,2.69 174 | 7,4,8,1,91.4,142.4,601.4,10.6,20.1,39,5.4,0,2.74 175 | 4,4,9,2,90.9,126.5,686.5,7,17.7,39,2.2,0,3.07 176 | 1,4,8,7,90.2,96.9,624.2,8.9,14.2,53,1.8,0,3.5 177 | 1,4,8,7,90.2,96.9,624.2,8.9,20.3,39,4.9,0,4.53 178 | 6,5,4,5,81.5,9.1,55.2,2.7,5.8,54,5.8,0,4.61 179 | 2,5,8,1,90.2,99.6,631.2,6.3,19.2,44,2.7,0,4.69 180 | 2,5,9,4,90.1,82.9,735.7,6.2,18.3,45,2.2,0,4.88 181 | 8,6,8,3,88.8,147.3,614.5,9,14.4,66,5.4,0,5.23 182 | 1,3,9,1,92.4,124.1,680.7,8.5,23.9,32,6.7,0,5.33 183 | 8,6,10,2,84.9,32.8,664.2,3,19.1,32,4,0,5.44 184 | 5,4,2,1,86.8,15.6,48.3,3.9,12.4,53,2.2,0,6.38 185 | 7,4,10,2,91.7,48.5,696.1,11.1,16.8,45,4.5,0,6.83 186 | 8,6,8,6,93.9,135.7,586.7,15.1,20.8,34,4.9,0,6.96 187 | 2,5,9,3,91,129.5,692.6,7,17.6,46,3.1,0,7.04 188 | 8,6,3,1,89.3,51.3,102.2,9.6,11.5,39,5.8,0,7.19 189 | 1,5,9,2,90.9,126.5,686.5,7,21,42,2.2,0,7.3 190 | 6,4,3,7,90.8,41.9,89.4,7.9,13.3,42,0.9,0,7.4 191 | 7,4,3,1,90.7,44,92.4,5.5,11.5,60,4,0,8.24 192 | 6,5,3,6,91.2,48.3,97.8,12.5,11.7,33,4,0,8.31 193 | 2,5,8,5,95.2,131.7,578.8,10.4,24.2,28,2.7,0,8.68 194 | 2,2,8,3,94.8,108.3,647.1,17,24.6,22,4.5,0,8.71 195 | 4,5,9,4,92.9,133.3,699.6,9.2,24.3,25,4,0,9.41 196 | 2,2,8,3,94.8,108.3,647.1,17,24.6,22,4.5,0,10.01 197 | 2,5,8,6,93.9,135.7,586.7,15.1,23.5,36,5.4,0,10.02 198 | 6,5,4,5,81.5,9.1,55.2,2.7,5.8,54,5.8,0,10.93 199 | 4,5,9,5,92.9,137,706.4,9.2,21.5,15,0.9,0,11.06 200 | 3,4,9,3,91,129.5,692.6,7,13.9,59,6.3,0,11.24 201 | 2,4,9,2,63.5,70.8,665.3,0.8,22.6,38,3.6,0,11.32 202 | 1,5,9,3,91,129.5,692.6,7,21.6,33,2.2,0,11.53 203 | 6,5,3,1,90.1,37.6,83.7,7.2,12.4,54,3.6,0,12.1 204 | 7,4,2,1,83.9,8.7,32.1,2.1,8.8,68,2.2,0,13.05 205 | 8,6,10,4,91.4,37.9,673.8,5.2,20.2,37,2.7,0,13.7 206 | 5,6,3,7,90.6,50.1,100.4,7.8,15.1,64,4,0,13.99 207 | 4,5,9,5,92.9,137,706.4,9.2,22.1,34,1.8,0,14.57 208 | 2,2,8,7,93.5,139.4,594.2,20.3,22.9,31,7.2,0,15.45 209 | 7,5,9,3,91,129.5,692.6,7,20.7,37,2.2,0,17.2 210 | 6,5,9,6,92.4,117.9,668,12.2,19.6,33,6.3,0,19.23 211 | 8,3,9,5,93.7,80.9,685.2,17.9,23.2,26,4.9,0,23.41 212 | 4,4,10,7,90.6,43.7,686.9,6.7,18.4,25,3.1,0,24.23 213 | 7,4,8,7,93.5,139.4,594.2,20.3,5.1,96,5.8,0,26 214 | 7,4,9,6,94.3,85.1,692.3,15.9,20.1,47,4.9,0,26.13 215 | 7,3,3,2,87.6,52.2,103.8,5,11,46,5.8,0,27.35 216 | 4,4,3,7,91.7,35.8,80.8,7.8,17,27,4.9,0,28.66 217 | 4,4,3,7,91.7,35.8,80.8,7.8,17,27,4.9,0,28.66 218 | 4,4,9,1,92.4,124.1,680.7,8.5,16.9,60,1.3,0,29.48 219 | 1,3,9,2,88.6,91.8,709.9,7.1,12.4,73,6.3,0,30.32 220 | 4,5,9,4,92.9,133.3,699.6,9.2,19.4,19,1.3,0,31.72 221 | 6,5,3,2,90.1,39.7,86.6,6.2,15.2,27,3.1,0,31.86 222 | 8,6,8,1,90.2,99.6,631.2,6.3,16.2,59,3.1,0,32.07 223 | 3,4,9,6,93.3,141.2,713.9,13.9,18.6,49,3.6,0,35.88 224 | 4,3,3,2,87.6,52.2,103.8,5,11,46,5.8,0,36.85 225 | 2,2,7,6,88.3,150.3,309.9,6.8,13.4,79,3.6,0,37.02 226 | 7,4,9,4,90.1,82.9,735.7,6.2,15.4,57,4.5,0,37.71 227 | 4,4,9,1,93.5,149.3,728.6,8.1,22.9,39,4.9,0,48.55 228 | 7,5,10,2,91.7,48.5,696.1,11.1,16.1,44,4,0,49.37 229 | 8,6,8,7,92.2,81.8,480.8,11.9,20.1,34,4.5,0,58.3 230 | 4,6,9,1,93.5,149.3,728.6,8.1,28.3,26,3.1,0,64.1 231 | 8,6,8,7,92.2,81.8,480.8,11.9,16.4,43,4,0,71.3 232 | 4,4,9,4,92.9,133.3,699.6,9.2,26.4,21,4.5,0,88.49 233 | 1,5,9,1,93.5,149.3,728.6,8.1,27.8,27,3.1,0,95.18 234 | 6,4,9,3,91,129.5,692.6,7,18.7,43,2.7,0,103.39 235 | 9,4,9,3,84.4,73.4,671.9,3.2,24.3,36,3.1,0,105.66 236 | 4,5,9,7,92.5,121.1,674.4,8.6,17.7,25,3.1,0,154.88 237 | 8,6,8,1,91.4,142.4,601.4,10.6,19.6,41,5.8,0,196.48 238 | 2,2,9,7,92.5,121.1,674.4,8.6,18.2,46,1.8,0,200.94 239 | 1,2,9,3,91,129.5,692.6,7,18.8,40,2.2,0,212.88 240 | 6,5,9,7,92.5,121.1,674.4,8.6,25.1,27,4,0,1090.84 241 | 7,5,4,1,81.9,3,7.9,3.5,13.4,75,1.8,0,0 242 | 6,3,4,4,88,17.2,43.5,3.8,15.2,51,2.7,0,0 243 | 4,4,4,6,83,23.3,85.3,2.3,16.7,20,3.1,0,0 244 | 2,4,8,1,94.2,122.3,589.9,12.9,15.4,66,4,0,10.13 245 | 7,4,8,1,91.8,175.1,700.7,13.8,21.9,73,7.6,1,0 246 | 2,4,8,1,91.8,175.1,700.7,13.8,22.4,54,7.6,0,2.87 247 | 3,4,8,1,91.8,175.1,700.7,13.8,26.8,38,6.3,0,0.76 248 | 5,4,8,1,91.8,175.1,700.7,13.8,25.7,39,5.4,0,0.09 249 | 2,4,8,4,92.2,91.6,503.6,9.6,20.7,70,2.2,0,0.75 250 | 8,6,8,4,93.1,157.3,666.7,13.5,28.7,28,2.7,0,0 251 | 3,4,8,4,93.1,157.3,666.7,13.5,21.7,40,0.4,0,2.47 252 | 8,5,8,4,93.1,157.3,666.7,13.5,26.8,25,3.1,0,0.68 253 | 8,5,8,4,93.1,157.3,666.7,13.5,24,36,3.1,0,0.24 254 | 6,5,8,4,93.1,157.3,666.7,13.5,22.1,37,3.6,0,0.21 255 | 7,4,8,5,91.9,109.2,565.5,8,21.4,38,2.7,0,1.52 256 | 6,3,8,5,91.6,138.1,621.7,6.3,18.9,41,3.1,0,10.34 257 | 2,5,8,5,87.5,77,694.8,5,22.3,46,4,0,0 258 | 8,6,8,7,94.2,117.2,581.1,11,23.9,41,2.2,0,8.02 259 | 4,3,8,7,94.2,117.2,581.1,11,21.4,44,2.7,0,0.68 260 | 3,4,8,7,91.8,170.9,692.3,13.7,20.6,59,0.9,0,0 261 | 7,4,8,7,91.8,170.9,692.3,13.7,23.7,40,1.8,0,1.38 262 | 2,4,8,2,93.6,97.9,542,14.4,28.3,32,4,0,8.85 263 | 3,4,8,6,91.6,112.4,573,8.9,11.2,84,7.6,0,3.3 264 | 2,4,8,6,91.6,112.4,573,8.9,21.4,42,3.1,0,4.25 265 | 6,3,8,6,91.1,141.1,629.1,7.1,19.3,39,3.6,0,1.56 266 | 4,4,8,6,94.3,167.6,684.4,13,21.8,53,3.1,0,6.54 267 | 4,4,8,3,93.7,102.2,550.3,14.6,22.1,54,7.6,0,0.79 268 | 6,5,8,3,94.3,131.7,607.1,22.7,19.4,55,4,0,0.17 269 | 2,2,8,3,92.1,152.6,658.2,14.3,23.7,24,3.1,0,0 270 | 3,4,8,3,92.1,152.6,658.2,14.3,21,32,3.1,0,0 271 | 4,4,8,3,92.1,152.6,658.2,14.3,19.1,53,2.7,0,4.4 272 | 2,2,8,3,92.1,152.6,658.2,14.3,21.8,56,3.1,0,0.52 273 | 8,6,8,3,92.1,152.6,658.2,14.3,20.1,58,4.5,0,9.27 274 | 2,5,8,3,92.1,152.6,658.2,14.3,20.2,47,4,0,3.09 275 | 4,6,12,1,84.4,27.2,353.5,6.8,4.8,57,8.5,0,8.98 276 | 8,6,12,4,84,27.8,354.6,5.3,5.1,61,8,0,11.19 277 | 4,6,12,5,84.6,26.4,352,2,5.1,61,4.9,0,5.38 278 | 4,4,12,2,85.4,25.4,349.7,2.6,4.6,21,8.5,0,17.85 279 | 3,4,12,2,85.4,25.4,349.7,2.6,4.6,21,8.5,0,10.73 280 | 4,4,12,2,85.4,25.4,349.7,2.6,4.6,21,8.5,0,22.03 281 | 4,4,12,2,85.4,25.4,349.7,2.6,4.6,21,8.5,0,9.77 282 | 4,6,12,6,84.7,26.7,352.6,4.1,2.2,59,4.9,0,9.27 283 | 6,5,12,3,85.4,25.4,349.7,2.6,5.1,24,8.5,0,24.77 284 | 6,3,2,1,84.9,27.5,353.5,3.4,4.2,51,4,0,0 285 | 3,4,2,4,86.9,6.6,18.7,3.2,8.8,35,3.1,0,1.1 286 | 5,4,2,6,85.2,4.9,15.8,6.3,7.5,46,8,0,24.24 287 | 2,5,7,1,93.9,169.7,411.8,12.3,23.4,40,6.3,0,0 288 | 7,6,7,4,91.2,183.1,437.7,12.5,12.6,90,7.6,0.2,0 289 | 7,4,7,7,91.6,104.2,474.9,9,22.1,49,2.7,0,0 290 | 7,4,7,7,91.6,104.2,474.9,9,24.2,32,1.8,0,0 291 | 7,4,7,7,91.6,104.2,474.9,9,24.3,30,1.8,0,0 292 | 2,5,7,7,91.6,104.2,474.9,9,18.7,53,1.8,0,0 293 | 9,4,7,7,91.6,104.2,474.9,9,25.3,39,0.9,0,8 294 | 4,5,7,6,91.6,100.2,466.3,6.3,22.9,40,1.3,0,2.64 295 | 7,6,7,3,93.1,180.4,430.8,11,26.9,28,5.4,0,86.45 296 | 8,6,7,3,92.3,88.8,440.9,8.5,17.1,67,3.6,0,6.57 297 | 7,5,6,1,93.1,180.4,430.8,11,22.2,48,1.3,0,0 298 | 6,4,6,1,90.4,89.5,290.8,6.4,14.3,46,1.8,0,0.9 299 | 8,6,6,1,90.4,89.5,290.8,6.4,15.4,45,2.2,0,0 300 | 8,6,6,4,91.2,147.8,377.2,12.7,19.6,43,4.9,0,0 301 | 6,5,6,7,53.4,71,233.8,0.4,10.6,90,2.7,0,0 302 | 6,5,6,2,90.4,93.3,298.1,7.5,20.7,25,4.9,0,0 303 | 6,5,6,2,90.4,93.3,298.1,7.5,19.1,39,5.4,0,3.52 304 | 3,6,6,6,91.1,94.1,232.1,7.1,19.2,38,4.5,0,0 305 | 3,6,6,6,91.1,94.1,232.1,7.1,19.2,38,4.5,0,0 306 | 6,5,5,7,85.1,28,113.8,3.5,11.3,94,4.9,0,0 307 | 1,4,9,1,89.6,84.1,714.3,5.7,19,52,2.2,0,0 308 | 7,4,9,1,89.6,84.1,714.3,5.7,17.1,53,5.4,0,0.41 309 | 3,4,9,1,89.6,84.1,714.3,5.7,23.8,35,3.6,0,5.18 310 | 2,4,9,1,92.4,105.8,758.1,9.9,16,45,1.8,0,0 311 | 2,4,9,1,92.4,105.8,758.1,9.9,24.9,27,2.2,0,0 312 | 7,4,9,1,92.4,105.8,758.1,9.9,25.3,27,2.7,0,0 313 | 6,3,9,1,92.4,105.8,758.1,9.9,24.8,28,1.8,0,14.29 314 | 2,4,9,1,50.4,46.2,706.6,0.4,12.2,78,6.3,0,0 315 | 6,5,9,4,92.6,115.4,777.1,8.8,24.3,27,4.9,0,0 316 | 4,4,9,4,92.6,115.4,777.1,8.8,19.7,41,1.8,0,1.58 317 | 3,4,9,4,91.2,134.7,817.5,7.2,18.5,30,2.7,0,0 318 | 4,5,9,5,92.4,96.2,739.4,8.6,18.6,24,5.8,0,0 319 | 4,4,9,5,92.4,96.2,739.4,8.6,19.2,24,4.9,0,3.78 320 | 6,5,9,5,92.8,119,783.5,7.5,21.6,27,2.2,0,0 321 | 5,4,9,5,92.8,119,783.5,7.5,21.6,28,6.3,0,4.41 322 | 6,3,9,5,92.8,119,783.5,7.5,18.9,34,7.2,0,34.36 323 | 1,4,9,5,92.8,119,783.5,7.5,16.8,28,4,0,7.21 324 | 6,5,9,5,92.8,119,783.5,7.5,16.8,28,4,0,1.01 325 | 3,5,9,5,90.7,136.9,822.8,6.8,12.9,39,2.7,0,2.18 326 | 6,5,9,5,88.1,53.3,726.9,5.4,13.7,56,1.8,0,4.42 327 | 1,4,9,7,92.2,102.3,751.5,8.4,24.2,27,3.1,0,0 328 | 5,4,9,7,92.2,102.3,751.5,8.4,24.1,27,3.1,0,0 329 | 6,5,9,7,92.2,102.3,751.5,8.4,21.2,32,2.2,0,0 330 | 6,5,9,7,92.2,102.3,751.5,8.4,19.7,35,1.8,0,0 331 | 4,3,9,7,92.2,102.3,751.5,8.4,23.5,27,4,0,3.33 332 | 3,3,9,7,92.2,102.3,751.5,8.4,24.2,27,3.1,0,6.58 333 | 7,4,9,7,91.2,124.4,795.3,8.5,21.5,28,4.5,0,15.64 334 | 4,4,9,7,91.2,124.4,795.3,8.5,17.1,41,2.2,0,11.22 335 | 1,4,9,2,92.1,87.7,721.1,9.5,18.1,54,3.1,0,2.13 336 | 2,3,9,2,91.6,108.4,764,6.2,18,51,5.4,0,0 337 | 4,3,9,2,91.6,108.4,764,6.2,9.8,86,1.8,0,0 338 | 7,4,9,2,91.6,108.4,764,6.2,19.3,44,2.2,0,0 339 | 6,3,9,2,91.6,108.4,764,6.2,23,34,2.2,0,56.04 340 | 8,6,9,2,91.6,108.4,764,6.2,22.7,35,2.2,0,7.48 341 | 2,4,9,2,91.6,108.4,764,6.2,20.4,41,1.8,0,1.47 342 | 2,5,9,2,91.6,108.4,764,6.2,19.3,44,2.2,0,3.93 343 | 8,6,9,2,91.9,111.7,770.3,6.5,15.7,51,2.2,0,0 344 | 6,3,9,2,91.5,130.1,807.1,7.5,20.6,37,1.8,0,0 345 | 8,6,9,2,91.5,130.1,807.1,7.5,15.9,51,4.5,0,2.18 346 | 6,3,9,2,91.5,130.1,807.1,7.5,12.2,66,4.9,0,6.1 347 | 2,2,9,2,91.5,130.1,807.1,7.5,16.8,43,3.1,0,5.83 348 | 1,4,9,2,91.5,130.1,807.1,7.5,21.3,35,2.2,0,28.19 349 | 5,4,9,6,92.1,99,745.3,9.6,10.1,75,3.6,0,0 350 | 3,4,9,6,92.1,99,745.3,9.6,17.4,57,4.5,0,0 351 | 5,4,9,6,92.1,99,745.3,9.6,12.8,64,3.6,0,1.64 352 | 5,4,9,6,92.1,99,745.3,9.6,10.1,75,3.6,0,3.71 353 | 4,4,9,6,92.1,99,745.3,9.6,15.4,53,6.3,0,7.31 354 | 7,4,9,6,92.1,99,745.3,9.6,20.6,43,3.6,0,2.03 355 | 7,4,9,6,92.1,99,745.3,9.6,19.8,47,2.7,0,1.72 356 | 7,4,9,6,92.1,99,745.3,9.6,18.7,50,2.2,0,5.97 357 | 4,4,9,6,92.1,99,745.3,9.6,20.8,35,4.9,0,13.06 358 | 4,4,9,6,92.1,99,745.3,9.6,20.8,35,4.9,0,1.26 359 | 6,3,9,6,92.5,122,789.7,10.2,15.9,55,3.6,0,0 360 | 6,3,9,6,92.5,122,789.7,10.2,19.7,39,2.7,0,0 361 | 1,4,9,6,92.5,122,789.7,10.2,21.1,39,2.2,0,8.12 362 | 6,5,9,6,92.5,122,789.7,10.2,18.4,42,2.2,0,1.09 363 | 4,3,9,6,92.5,122,789.7,10.2,17.3,45,4,0,3.94 364 | 7,4,9,6,88.2,55.2,732.3,11.6,15.2,64,3.1,0,0.52 365 | 4,3,9,3,91.9,111.7,770.3,6.5,15.9,53,2.2,0,2.93 366 | 6,5,9,3,91.9,111.7,770.3,6.5,21.1,35,2.7,0,5.65 367 | 6,5,9,3,91.9,111.7,770.3,6.5,19.6,45,3.1,0,20.03 368 | 4,5,9,3,91.1,132.3,812.1,12.5,15.9,38,5.4,0,1.75 369 | 4,5,9,3,91.1,132.3,812.1,12.5,16.4,27,3.6,0,0 370 | 6,5,9,7,91.2,94.3,744.4,8.4,16.8,47,4.9,0,12.64 371 | 4,5,9,1,91,276.3,825.1,7.1,13.8,77,7.6,0,0 372 | 7,4,9,1,91,276.3,825.1,7.1,13.8,77,7.6,0,11.06 373 | 3,4,7,4,91.9,133.6,520.5,8,14.2,58,4,0,0 374 | 4,5,8,1,92,203.2,664.5,8.1,10.4,75,0.9,0,0 375 | 5,4,8,5,94.8,222.4,698.6,13.9,20.3,42,2.7,0,0 376 | 6,5,9,6,90.3,290,855.3,7.4,10.3,78,4,0,18.3 377 | 6,5,9,7,91.2,94.3,744.4,8.4,15.4,57,4.9,0,39.35 378 | 8,6,8,2,92.1,207,672.6,8.2,21.1,54,2.2,0,0 379 | 2,2,8,7,93.7,231.1,715.1,8.4,21.9,42,2.2,0,174.63 380 | 6,5,3,5,90.9,18.9,30.6,8,8.7,51,5.8,0,0 381 | 4,5,1,1,18.7,1.1,171.4,0,5.2,100,0.9,0,0 382 | 5,4,7,4,93.7,101.3,458.8,11.9,19.3,39,7.2,0,7.73 383 | 8,6,8,5,90.7,194.1,643,6.8,16.2,63,2.7,0,16.33 384 | 8,6,8,4,95.2,217.7,690,18,28.2,29,1.8,0,5.86 385 | 9,6,8,5,91.6,248.4,753.8,6.3,20.5,58,2.7,0,42.87 386 | 8,4,8,7,91.6,273.8,819.1,7.7,21.3,44,4.5,0,12.18 387 | 2,4,8,1,91.6,181.3,613,7.6,20.9,50,2.2,0,16 388 | 3,4,9,1,90.5,96.7,750.5,11.4,20.6,55,5.4,0,24.59 389 | 5,5,3,5,90.9,18.9,30.6,8,11.6,48,5.4,0,0 390 | 6,4,8,6,94.8,227,706.7,12,23.3,34,3.1,0,28.74 391 | 7,4,8,6,94.8,227,706.7,12,23.3,34,3.1,0,0 392 | 7,4,2,2,84.7,9.5,58.3,4.1,7.5,71,6.3,0,9.96 393 | 8,6,9,6,91.1,91.3,738.1,7.2,20.7,46,2.7,0,30.18 394 | 1,3,9,1,91,276.3,825.1,7.1,21.9,43,4,0,70.76 395 | 2,4,3,3,93.4,15,25.6,11.4,15.2,19,7.6,0,0 396 | 6,5,2,2,84.1,4.6,46.7,2.2,5.3,68,1.8,0,0 397 | 4,5,2,1,85,9,56.9,3.5,10.1,62,1.8,0,51.78 398 | 4,3,9,1,90.5,96.7,750.5,11.4,20.4,55,4.9,0,3.64 399 | 5,6,8,1,91.6,181.3,613,7.6,24.3,33,3.6,0,3.63 400 | 1,2,8,7,93.7,231.1,715.1,8.4,25.9,32,3.1,0,0 401 | 9,5,6,4,93.3,49.5,297.7,14,28,34,4.5,0,0 402 | 9,5,6,4,93.3,49.5,297.7,14,28,34,4.5,0,8.16 403 | 3,4,9,5,91.1,88.2,731.7,8.3,22.8,46,4,0,4.95 404 | 9,9,8,6,94.8,227,706.7,12,25,36,4,0,0 405 | 8,6,8,5,90.7,194.1,643,6.8,21.3,41,3.6,0,0 406 | 2,4,9,4,87.9,84.8,725.1,3.7,21.8,34,2.2,0,6.04 407 | 2,2,8,3,94.6,212.1,680.9,9.5,27.9,27,2.2,0,0 408 | 6,5,9,7,87.1,291.3,860.6,4,17,67,4.9,0,3.95 409 | 4,5,2,7,84.7,8.2,55,2.9,14.2,46,4,0,0 410 | 4,3,9,6,90.3,290,855.3,7.4,19.9,44,3.1,0,7.8 411 | 1,4,7,3,92.3,96.2,450.2,12.1,23.4,31,5.4,0,0 412 | 6,3,2,6,84.1,7.3,52.8,2.7,14.7,42,2.7,0,0 413 | 7,4,2,6,84.6,3.2,43.6,3.3,8.2,53,9.4,0,4.62 414 | 9,4,7,2,92.3,92.1,442.1,9.8,22.8,27,4.5,0,1.63 415 | 7,5,8,7,93.7,231.1,715.1,8.4,26.4,33,3.6,0,0 416 | 5,4,8,1,93.6,235.1,723.1,10.1,24.1,50,4,0,0 417 | 8,6,8,5,94.8,222.4,698.6,13.9,27.5,27,4.9,0,746.28 418 | 6,3,7,3,92.7,164.1,575.8,8.9,26.3,39,3.1,0,7.02 419 | 6,5,3,4,93.4,17.3,28.3,9.9,13.8,24,5.8,0,0 420 | 2,4,8,1,92,203.2,664.5,8.1,24.9,42,5.4,0,2.44 421 | 2,5,8,1,91.6,181.3,613,7.6,24.8,36,4,0,3.05 422 | 8,8,8,4,91.7,191.4,635.9,7.8,26.2,36,4.5,0,185.76 423 | 2,4,8,4,95.2,217.7,690,18,30.8,19,4.5,0,0 424 | 8,6,7,1,88.9,263.1,795.9,5.2,29.3,27,3.6,0,6.3 425 | 1,3,9,7,91.2,94.3,744.4,8.4,22.3,48,4,0,0.72 426 | 8,6,8,7,93.7,231.1,715.1,8.4,26.9,31,3.6,0,4.96 427 | 2,2,8,5,91.6,248.4,753.8,6.3,20.4,56,2.2,0,0 428 | 8,6,8,5,91.6,248.4,753.8,6.3,20.4,56,2.2,0,0 429 | 2,4,8,2,92.1,207,672.6,8.2,27.9,33,2.2,0,2.35 430 | 1,3,8,5,94.8,222.4,698.6,13.9,26.2,34,5.8,0,0 431 | 3,4,8,1,91.6,181.3,613,7.6,24.6,44,4,0,3.2 432 | 7,4,9,5,89.7,287.2,849.3,6.8,19.4,45,3.6,0,0 433 | 1,3,8,7,92.1,178,605.3,9.6,23.3,40,4,0,6.36 434 | 8,6,8,5,94.8,222.4,698.6,13.9,23.9,38,6.7,0,0 435 | 2,4,8,1,93.6,235.1,723.1,10.1,20.9,66,4.9,0,15.34 436 | 1,4,8,6,90.6,269.8,811.2,5.5,22.2,45,3.6,0,0 437 | 2,5,7,7,90.8,84.7,376.6,5.6,23.8,51,1.8,0,0 438 | 8,6,8,2,92.1,207,672.6,8.2,26.8,35,1.3,0,0.54 439 | 8,6,8,7,89.4,253.6,768.4,9.7,14.2,73,2.7,0,0 440 | 2,5,8,7,93.7,231.1,715.1,8.4,23.6,53,4,0,6.43 441 | 1,3,9,6,91.1,91.3,738.1,7.2,19.1,46,2.2,0,0.33 442 | 5,4,9,6,90.3,290,855.3,7.4,16.2,58,3.6,0,0 443 | 8,6,8,2,92.1,207,672.6,8.2,25.5,29,1.8,0,1.23 444 | 6,5,4,2,87.9,24.9,41.6,3.7,10.9,64,3.1,0,3.35 445 | 1,2,7,6,90.7,80.9,368.3,16.8,14.8,78,8,0,0 446 | 2,5,9,6,90.3,290,855.3,7.4,16.2,58,3.6,0,9.96 447 | 5,5,8,1,94,47.9,100.7,10.7,17.3,80,4.5,0,0 448 | 6,5,8,1,92,203.2,664.5,8.1,19.1,70,2.2,0,0 449 | 3,4,3,4,93.4,17.3,28.3,9.9,8.9,35,8,0,0 450 | 7,4,9,4,89.7,284.9,844,10.1,10.5,77,4,0,0 451 | 7,4,8,1,91.6,181.3,613,7.6,19.3,61,4.9,0,0 452 | 4,5,8,4,95.2,217.7,690,18,23.4,49,5.4,0,6.43 453 | 1,4,8,6,90.5,196.8,649.9,16.3,11.8,88,4.9,0,9.71 454 | 7,4,8,2,91.5,238.2,730.6,7.5,17.7,65,4,0,0 455 | 4,5,8,5,89.4,266.2,803.3,5.6,17.4,54,3.1,0,0 456 | 3,4,8,5,91.6,248.4,753.8,6.3,16.8,56,3.1,0,0 457 | 3,4,7,2,94.6,160,567.2,16.7,17.9,48,2.7,0,0 458 | 2,4,8,5,91.6,248.4,753.8,6.3,16.6,59,2.7,0,0 459 | 1,4,8,4,91.7,191.4,635.9,7.8,19.9,50,4,0,82.75 460 | 8,6,8,7,93.7,231.1,715.1,8.4,18.9,64,4.9,0,3.32 461 | 7,4,8,7,91.6,273.8,819.1,7.7,15.5,72,8,0,1.94 462 | 2,5,8,7,93.7,231.1,715.1,8.4,18.9,64,4.9,0,0 463 | 8,6,8,7,93.7,231.1,715.1,8.4,18.9,64,4.9,0,0 464 | 1,4,9,1,91,276.3,825.1,7.1,14.5,76,7.6,0,3.71 465 | 6,5,2,3,75.1,4.4,16.2,1.9,4.6,82,6.3,0,5.39 466 | 6,4,2,3,75.1,4.4,16.2,1.9,5.1,77,5.4,0,2.14 467 | 2,2,2,7,79.5,3.6,15.3,1.8,4.6,59,0.9,0,6.84 468 | 6,5,3,2,87.2,15.1,36.9,7.1,10.2,45,5.8,0,3.18 469 | 3,4,3,4,90.2,18.5,41.1,7.3,11.2,41,5.4,0,5.55 470 | 6,5,3,5,91.3,20.6,43.5,8.5,13.3,27,3.6,0,6.61 471 | 6,3,4,1,91,14.6,25.6,12.3,13.7,33,9.4,0,61.13 472 | 5,4,4,1,91,14.6,25.6,12.3,17.6,27,5.8,0,0 473 | 4,3,5,6,89.6,25.4,73.7,5.7,18,40,4,0,38.48 474 | 8,3,6,2,88.2,96.2,229,4.7,14.3,79,4,0,1.94 475 | 9,4,6,7,90.5,61.1,252.6,9.4,24.5,50,3.1,0,70.32 476 | 4,3,6,5,93,103.8,316.7,10.8,26.4,35,2.7,0,10.08 477 | 2,5,6,5,93.7,121.7,350.2,18,22.7,40,9.4,0,3.19 478 | 4,3,7,5,93.5,85.3,395,9.9,27.2,28,1.3,0,1.76 479 | 4,3,7,1,93.7,101.3,423.4,14.7,26.1,45,4,0,7.36 480 | 7,4,7,1,93.7,101.3,423.4,14.7,18.2,82,4.5,0,2.21 481 | 7,4,7,2,89.2,103.9,431.6,6.4,22.6,57,4.9,0,278.53 482 | 9,9,7,5,93.2,114.4,560,9.5,30.2,25,4.5,0,2.75 483 | 4,3,7,5,93.2,114.4,560,9.5,30.2,22,4.9,0,0 484 | 3,4,8,1,94.9,130.3,587.1,14.1,23.4,40,5.8,0,1.29 485 | 8,6,8,1,94.9,130.3,587.1,14.1,31,27,5.4,0,0 486 | 2,5,8,1,94.9,130.3,587.1,14.1,33.1,25,4,0,26.43 487 | 2,4,8,2,95,135.5,596.3,21.3,30.6,28,3.6,0,2.07 488 | 5,4,8,3,95.1,141.3,605.8,17.7,24.1,43,6.3,0,2 489 | 5,4,8,3,95.1,141.3,605.8,17.7,26.4,34,3.6,0,16.4 490 | 4,4,8,3,95.1,141.3,605.8,17.7,19.4,71,7.6,0,46.7 491 | 4,4,8,4,95.1,141.3,605.8,17.7,20.6,58,1.3,0,0 492 | 4,4,8,4,95.1,141.3,605.8,17.7,28.7,33,4,0,0 493 | 4,4,8,5,95.8,152,624.1,13.8,32.4,21,4.5,0,0 494 | 1,3,8,6,95.9,158,633.6,11.3,32.4,27,2.2,0,0 495 | 1,3,8,6,95.9,158,633.6,11.3,27.5,29,4.5,0,43.32 496 | 6,6,8,7,96,164,643,14,30.8,30,4.9,0,8.59 497 | 6,6,8,2,96.2,175.5,661.8,16.8,23.9,42,2.2,0,0 498 | 4,5,8,2,96.2,175.5,661.8,16.8,32.6,26,3.1,0,2.77 499 | 3,4,8,3,96.1,181.1,671.2,14.3,32.3,27,2.2,0,14.68 500 | 6,5,8,3,96.1,181.1,671.2,14.3,33.3,26,2.7,0,40.54 501 | 7,5,8,3,96.1,181.1,671.2,14.3,27.3,63,4.9,6.4,10.82 502 | 8,6,8,3,96.1,181.1,671.2,14.3,21.6,65,4.9,0.8,0 503 | 7,5,8,3,96.1,181.1,671.2,14.3,21.6,65,4.9,0.8,0 504 | 4,4,8,3,96.1,181.1,671.2,14.3,20.7,69,4.9,0.4,0 505 | 2,4,8,4,94.5,139.4,689.1,20,29.2,30,4.9,0,1.95 506 | 4,3,8,4,94.5,139.4,689.1,20,28.9,29,4.9,0,49.59 507 | 1,2,8,5,91,163.2,744.4,10.1,26.7,35,1.8,0,5.8 508 | 1,2,8,6,91,166.9,752.6,7.1,18.5,73,8.5,0,0 509 | 2,4,8,6,91,166.9,752.6,7.1,25.9,41,3.6,0,0 510 | 1,2,8,6,91,166.9,752.6,7.1,25.9,41,3.6,0,0 511 | 5,4,8,6,91,166.9,752.6,7.1,21.1,71,7.6,1.4,2.17 512 | 6,5,8,6,91,166.9,752.6,7.1,18.2,62,5.4,0,0.43 513 | 8,6,8,1,81.6,56.7,665.6,1.9,27.8,35,2.7,0,0 514 | 4,3,8,1,81.6,56.7,665.6,1.9,27.8,32,2.7,0,6.44 515 | 2,4,8,1,81.6,56.7,665.6,1.9,21.9,71,5.8,0,54.29 516 | 7,4,8,1,81.6,56.7,665.6,1.9,21.2,70,6.7,0,11.16 517 | 1,4,8,7,94.4,146,614.7,11.3,25.6,42,4,0,0 518 | 6,3,11,3,79.5,3,106.7,1.1,11.8,31,4.5,0,0 519 | -------------------------------------------------------------------------------- /traditional_ML_algorithms/SimpleLinearRegression/README.md: -------------------------------------------------------------------------------- 1 | Name: 2 | Simple Linear Regression 3 | 4 | Packages used:
5 | `Numpy` : 1.19.5
6 | `Pandas` : 1.2.4
7 | 8 | Brief explanation: 9 | Creating a Simple Linear Regression to model from scratch to make prediction. 10 | -------------------------------------------------------------------------------- /traditional_ML_algorithms/SimpleLinearRegression/data.csv: -------------------------------------------------------------------------------- 1 | Temperature,Revenue 2 | 24.56688442,534.7990284 3 | 26.00519115,625.1901215 4 | 27.79055388,660.6322888 5 | 20.59533505,487.7069603 6 | 11.50349764,316.2401944 7 | 14.35251388,367.9407438 8 | 13.70777988,308.8945179 9 | 30.83398474,696.7166402 10 | 0.976869989,55.39033824 11 | 31.66946458,737.8008241 12 | 11.45525338,325.9684084 13 | 3.664669577,71.16015301 14 | 18.81182403,467.4467066 15 | 13.62450892,289.5409341 16 | 39.53990899,905.4776043 17 | 18.48314099,469.9090332 18 | 25.93537514,648.2099977 19 | 42.51528041,921.508275 20 | 29.58948056,649.5611747 21 | 21.77594799,534.6228653 22 | 25.45783637,612.1539491 23 | 15.21456942,353.3256334 24 | 22.61931574,524.2361154 25 | 16.25872074,374.231135 26 | 23.88172478,523.1245467 27 | 18.97830025,473.6043349 28 | 15.6614643,402.4553204 29 | 29.18504465,679.3177906 30 | 19.02461092,517.5340283 31 | 35.12015142,809.6720534 32 | 24.18393726,528.3804165 33 | 15.23119012,356.0980075 34 | 8.790952808,237.7639106 35 | 18.23322784,418.1372788 36 | 35.62892497,809.4634112 37 | 37.05754246,870.7659159 38 | 22.28455032,550.2785159 39 | 17.51707397,405.6614459 40 | 31.7379196,740.9356848 41 | 17.04973761,501.7329901 42 | 23.0034888,539.6880057 43 | 8.755553938,242.2362083 44 | 18.77535808,421.621505 45 | 14.10966102,358.0028493 46 | 18.63391286,467.631063 47 | 15.67648661,396.9356482 48 | 20.94791347,500.9250645 49 | 30.6353071,651.8615363 50 | 20.47359412,451.4507843 51 | 31.22898848,697.8339862 52 | 6.393834627,190.7109408 53 | 27.18581031,621.1897304 54 | 28.63373276,666.1368355 55 | 27.99922248,628.4532107 56 | 10.32638937,219.3039932 57 | 27.31281141,623.5988607 58 | 33.23567229,749.3671543 59 | 36.56911506,827.6848313 60 | 12.46293731,303.7343815 61 | 14.37969698,351.2888691 62 | 16.30255473,381.5641352 63 | 11.56964367,321.8482734 64 | 33.55141872,774.1080813 65 | 3.986523168,131.6570175 66 | 20.51163741,498.7570498 67 | 6.542514397,195.7357217 68 | 19.81753939,496.0112948 69 | 11.69453767,284.7727889 70 | 21.4881766,483.4897686 71 | 18.77353222,430.3439033 72 | 12.68842965,276.787086 73 | 27.88711086,627.2912952 74 | 26.9567197,643.6486011 75 | 27.37540101,623.2487008 76 | 24.10161613,586.150568 77 | 28.7901015,653.9867356 78 | 40.47398918,918.3912316 79 | 25.54596553,591.1733898 80 | 28.70127646,651.1862423 81 | 29.4637861,682.7528689 82 | 16.02097541,372.9906055 83 | 14.73955066,381.8030138 84 | 22.17119887,515.4591017 85 | 29.03573877,685.3623881 86 | 29.20971484,654.7474611 87 | 16.36494499,406.5792487 88 | 27.78049953,643.9443266 89 | 13.33060576,344.6887652 90 | 29.3050392,642.2272909 91 | 14.3840835,361.1191443 92 | 30.42779184,704.2814391 93 | 9.073838248,222.8723171 94 | 23.07061587,543.5995933 95 | 8.586948141,221.2232906 96 | 12.35208102,337.1190252 97 | 9.018860236,212.5917401 98 | 20.26501213,474.7493924 99 | 19.36315346,460.4025002 100 | 14.6859445,343.3629045 101 | 9.95435701,283.8343266 102 | 19.97746731,468.9751034 103 | 32.00416835,711.1740653 104 | 14.28719594,322.592741 105 | 17.65850231,401.4330183 106 | 26.59505405,627.9018411 107 | 17.26218112,415.8176744 108 | 23.76143592,553.4452906 109 | 15.58806128,362.5152155 110 | 28.43656665,643.7883311 111 | 27.72739922,651.5043041 112 | 25.41930193,603.0371179 113 | 18.4750345,427.2113597 114 | 24.24311263,565.8749999 115 | 31.66848495,733.215828 116 | 17.69003162,385.6725008 117 | 31.8914678,689.930778 118 | 25.925171,572.0812915 119 | 29.31201255,719.4717014 120 | 11.05909651,306.7499304 121 | 25.49662411,596.2366902 122 | 22.94031709,540.798122 123 | 12.90177331,341.8593529 124 | 28.26283094,655.4339792 125 | 30.56266124,702.9017171 126 | 12.57151377,319.3494624 127 | 19.05928653,450.4732071 128 | 15.99234716,382.0739541 129 | 26.18796989,674.1506442 130 | 31.41262876,731.5982226 131 | 32.29733128,751.0545702 132 | 21.6967827,496.0119175 133 | 20.47502254,417.3548387 134 | 19.43326763,448.4713348 135 | 20.12955049,477.2950539 136 | 6.093897205,158.8498064 137 | 22.84197048,516.5486011 138 | 24.58590837,599.3649136 139 | 28.54798741,656.6365226 140 | 19.77936855,507.35681 141 | 12.4508333,279.866148 142 | 36.70257212,841.1714271 143 | 19.62265889,483.3330784 144 | 32.40924246,739.3872716 145 | 19.26778542,486.4749845 146 | 19.72807749,456.524341 147 | 8.638075893,241.2785475 148 | 29.43057848,618.2357655 149 | 19.25165371,446.9466515 150 | 24.61523866,603.0913818 151 | 12.44265041,274.0656189 152 | 24.54855656,531.7424848 153 | 12.20968364,297.4991195 154 | 12.26588434,319.4029032 155 | 19.75470829,493.7103332 156 | 23.34903419,586.1387673 157 | 21.14404693,497.7523178 158 | 18.88035599,476.7945251 159 | 28.2717647,625.8046425 160 | 16.40602096,390.4033487 161 | 28.99373705,675.8289158 162 | 10.24505765,273.0733418 163 | 11.07784312,280.5184674 164 | 25.49968375,583.0844489 165 | 27.93134811,648.5546445 166 | 28.4595428,726.2337713 167 | 13.30179635,335.8156867 168 | 25.99599345,570.5778753 169 | 32.80503252,685.6546554 170 | 32.71638067,775.7228577 171 | 32.10707989,773.9247547 172 | 24.77867495,540.9775109 173 | 15.02911176,366.2477143 174 | 23.42464718,539.5277397 175 | 35.21724007,809.7777259 176 | 16.37957324,376.5544719 177 | 20.55667911,477.8417185 178 | 21.32239237,520.4703098 179 | 26.94363797,654.1974057 180 | 22.63473505,518.2161052 181 | 33.25089892,782.0125497 182 | 8.991760111,250.1317278 183 | 26.87495294,634.5847506 184 | 21.35802411,488.1708088 185 | 22.00987431,520.8534562 186 | 29.12912778,652.0054081 187 | 16.19129752,383.9562396 188 | 35.35976059,796.5176848 189 | 11.18775682,293.9263927 190 | 16.55584289,414.423028 191 | 30.33033167,691.9580059 192 | 12.90066587,339.1095829 193 | 19.81463838,471.7015569 194 | 20.93460766,499.4583433 195 | 23.59102806,542.6080704 196 | 16.55794796,401.9247923 197 | 30.66659556,680.0271205 198 | 9.81251047,258.2868099 199 | 31.57998903,715.1246952 200 | 25.42216523,608.9363452 201 | 25.24114819,574.7106486 202 | 26.87358624,615.9266502 203 | 21.42455778,533.3243245 204 | 23.9638795,578.3604354 205 | 10.4471261,278.3098441 206 | 5.822332345,186.4764868 207 | 16.14582413,395.2737497 208 | 23.95931178,537.113833 209 | 9.782380752,228.9010303 210 | 23.97593152,603.2329422 211 | 10.09664458,272.8570213 212 | 22.38760374,493.1154676 213 | 27.32232277,612.8037704 214 | 20.24734584,437.2519927 215 | 23.15300185,506.4937476 216 | 15.75395072,409.4938476 217 | 27.57296049,562.7924633 218 | 18.77682968,402.3984607 219 | 22.65313582,532.05402 220 | 17.99302022,413.9140669 221 | 13.11245224,332.1501054 222 | 24.80257679,563.3012801 223 | 18.60275025,472.5493427 224 | 25.8659433,596.9842407 225 | 26.25074588,596.8891052 226 | 13.36431317,268.9291794 227 | 21.54045905,528.1162401 228 | 27.12812867,627.6508336 229 | 26.9441229,618.1720908 230 | 38.14633277,850.2469822 231 | 4.236464973,118.8121496 232 | 9.403479209,278.0627594 233 | 20.15334527,449.1128688 234 | 19.72133149,448.9304429 235 | 19.19495126,463.0656143 236 | 19.17204498,474.832244 237 | 36.11656147,824.9543567 238 | 23.41086133,553.1196514 239 | 29.91930886,696.6401775 240 | 32.00436506,675.807151 241 | 29.76822349,695.8512979 242 | 11.13270573,288.1581451 243 | 23.38514451,506.4321353 244 | 27.70505923,618.4572771 245 | 15.04792332,367.0523757 246 | 6.352459369,191.6233119 247 | 14.2635406,334.4337199 248 | 25.42294716,583.7597813 249 | 24.72715441,538.1796842 250 | 16.30012497,394.1686196 251 | 18.14895234,473.5681122 252 | 18.57811922,427.1383693 253 | 32.33480808,747.9632701 254 | 7.561124941,212.4835594 255 | 31.47122432,691.5165411 256 | 28.33536277,632.901914 257 | 17.63693676,448.5499609 258 | 21.70395288,521.6728037 259 | 18.46290678,437.8287103 260 | 32.47979434,706.7246037 261 | 17.36073198,405.2503868 262 | 21.00704512,503.0842679 263 | 23.57711325,570.9909316 264 | 30.76273994,706.3649044 265 | 22.6785601,543.9850584 266 | 28.85519147,641.0253891 267 | 9.651495249,274.6789209 268 | 18.50623116,420.9664529 269 | 5.338412673,145.6253019 270 | 35.4581362,828.2960767 271 | 24.77819856,594.8048712 272 | 24.62861149,603.3053386 273 | 28.4917635,651.486741 274 | 24.94971519,607.5421478 275 | 25.44824,625.8464212 276 | 22.24873896,535.8667293 277 | 24.76187643,530.7482251 278 | 22.44803391,535.7089203 279 | 35.03345633,781.9837945 280 | 33.7442087,797.566536 281 | 22.52674945,521.2673794 282 | 28.46493296,607.8391938 283 | 23.49753209,534.3645388 284 | 26.07840506,599.2782774 285 | 28.86558895,662.5589903 286 | 22.14631706,512.5881071 287 | 26.33705248,574.4233102 288 | 25.00237968,550.7014036 289 | 26.4560508,554.7429738 290 | 22.18951589,496.4613625 291 | 15.52116187,350.6290364 292 | 17.6568394,409.4028016 293 | 28.72991474,631.3182368 294 | 27.52923218,661.4675188 295 | 27.18851714,642.3498137 296 | 10.40342267,321.7500343 297 | 17.58837197,412.0650006 298 | 24.52184673,538.31289 299 | 37.99863474,857.5266413 300 | 16.95477783,425.2655958 301 | 7.745285959,198.1215634 302 | 5.858454276,170.2377561 303 | 26.85972289,599.1163601 304 | 24.49347704,558.636932 305 | 21.90251935,550.4412717 306 | 30.02820743,714.5600563 307 | 21.28191601,526.7008607 308 | 32.46497067,759.3774317 309 | 17.09064457,441.5087331 310 | 33.31499762,756.0377021 311 | 23.4125478,542.8391063 312 | 18.97799114,454.1892673 313 | 12.27096675,335.1568558 314 | 25.19142452,575.176896 315 | 27.06860738,594.6510092 316 | 25.72547019,621.9692088 317 | 22.31107869,520.3924054 318 | 25.11606991,587.2212461 319 | 22.15258869,537.7661123 320 | 28.29868977,639.5380115 321 | 21.71200518,467.4023646 322 | 15.11819661,374.9557024 323 | 25.37410906,604.6266727 324 | 18.43998163,463.4805082 325 | 22.87056201,550.055216 326 | 14.36142415,315.6465807 327 | 7.261348397,223.4350161 328 | 25.22777375,563.2509867 329 | 20.97115284,489.3152348 330 | 19.77514871,458.860905 331 | 41.92444647,965.4930396 332 | 28.64919191,689.8516908 333 | 29.24175192,678.7513876 334 | 15.84302201,379.5642675 335 | 20.89871624,508.7204715 336 | 30.45673953,684.8030705 337 | 24.81875357,598.6761971 338 | 19.84924077,416.8486183 339 | 22.11870569,571.7642733 340 | 34.0616734,771.7895369 341 | 9.557275885,235.3646433 342 | 25.55120003,579.3073878 343 | 19.06659113,406.516091 344 | 23.08766377,536.2081816 345 | 8.033152964,249.8842521 346 | 29.70702382,702.9940111 347 | 12.18941793,335.770416 348 | 35.09479555,807.5412872 349 | 24.96044566,564.3105317 350 | 38.18519935,856.3033039 351 | 18.98527518,482.5719881 352 | 18.70847606,436.9513113 353 | 7.223377163,216.183462 354 | 12.70471774,295.3396989 355 | 24.5288527,594.1103517 356 | 39.76412854,935.7172907 357 | 30.24724825,648.4536093 358 | 24.47243216,596.8767502 359 | 20.24414985,498.2521461 360 | 20.22642046,475.5382094 361 | 14.89697237,384.6994155 362 | 28.78743552,633.5040087 363 | 29.70418303,659.8732869 364 | 26.36974685,609.4174787 365 | 6.775206313,199.5552938 366 | 23.24671713,555.245217 367 | 24.30829573,594.3116748 368 | 25.71796257,572.5370483 369 | 21.68442569,478.5985086 370 | 26.19166817,563.3816326 371 | 21.6018917,545.9039291 372 | 18.88371892,444.8268017 373 | 0.267027698,32.54661902 374 | 19.61787546,506.2223794 375 | 20.10390047,491.4304998 376 | 23.98464085,559.1358692 377 | 29.25112258,697.1474728 378 | 34.86070051,798.0597179 379 | 11.17715183,278.7319615 380 | 26.12624136,594.8724701 381 | 39.8593964,875.0193476 382 | 20.41103121,513.8043817 383 | 17.87119907,440.6778286 384 | 18.34681936,410.860905 385 | 9.900293264,256.772593 386 | 25.05628082,583.8552306 387 | 12.08460133,278.4182651 388 | 36.99708394,851.3430963 389 | 14.73182422,322.9839774 390 | 7.107491008,221.400252 391 | 38.09660871,819.1175879 392 | 26.9236056,644.4886327 393 | 12.43313955,283.6796571 394 | 10.11973687,276.3733742 395 | 17.57423462,402.7931738 396 | 12.06247527,300.9322734 397 | 30.40761507,690.7892959 398 | 7.335445007,192.3419961 399 | 22.63297707,546.6938576 400 | 28.04640445,665.6726764 401 | 33.51453981,750.4447328 402 | 24.24037247,569.6187562 403 | 38.62886243,916.648613 404 | 0,10 405 | 24.34910395,572.6720474 406 | 26.16885914,658.6004564 407 | 5.307507347,242.5098553 408 | 17.99701481,441.0029443 409 | 30.96508651,702.6236136 410 | 29.7185162,643.0909437 411 | 32.64993621,818.135393 412 | 21.12912561,493.2266364 413 | 18.55163953,443.1136034 414 | 14.55121221,323.9446718 415 | 41.76658912,969.2916296 416 | 27.11773949,658.5937316 417 | 20.01638438,477.3151879 418 | 20.56301483,425.0120182 419 | 27.51664567,649.729072 420 | 30.22810362,679.7120584 421 | 21.67989703,505.7438672 422 | 20.05018591,473.4996311 423 | 17.29920384,405.9151588 424 | 17.19943002,428.8543561 425 | 35.44454622,800.2024937 426 | 19.11365281,445.7723999 427 | 18.95252067,450.708589 428 | 26.12213765,617.1007232 429 | 19.98286779,541.2936627 430 | 27.72143999,654.8949545 431 | 21.02639814,521.7754452 432 | 25.3804373,603.3246306 433 | 24.11359659,588.5275513 434 | 27.5990664,634.1219776 435 | 8.756004031,246.7871609 436 | 27.54196095,640.1770588 437 | 15.91667782,381.0433769 438 | 17.18894776,390.8791194 439 | 28.79315859,654.1293765 440 | 17.13279538,412.082357 441 | 8.79430294,264.123914 442 | 31.03033279,684.1584437 443 | 19.20297003,459.7353497 444 | 27.12941186,615.1753845 445 | 30.08108934,698.9718063 446 | 45,1000 447 | 18.90848865,449.8133003 448 | 15.10292191,322.888783 449 | 21.89743267,493.4202188 450 | 29.50819457,629.8937918 451 | 19.2786717,452.6263171 452 | 29.87997288,683.5447809 453 | 21.61064376,537.6648006 454 | 24.98851899,608.6299921 455 | 31.16003022,746.9463889 456 | 34.67804648,756.9625616 457 | 20.90057478,491.2306027 458 | 37.12707034,892.9477198 459 | 26.36052065,646.2669458 460 | 35.33120818,804.2600255 461 | 24.48490862,526.5470649 462 | 38.66820248,891.4136462 463 | 28.90019172,636.298374 464 | 12.12301401,297.0254137 465 | 11.5951027,257.078777 466 | 17.45516162,391.7152986 467 | 20.89661921,494.6274373 468 | 40.30376781,926.0671533 469 | 26.53021877,612.2437215 470 | 39.5131548,898.805423 471 | 22.39797728,489.5690899 472 | 9.309345599,291.7230401 473 | 19.49474317,429.4357021 474 | 22.22512228,500.0657787 475 | 18.88716165,475.2133537 476 | 21.75209218,530.356713 477 | 18.15921677,453.7856066 478 | 14.86610347,296.9065323 479 | 28.82975908,682.8085663 480 | 25.33342015,581.2620157 481 | 18.50836039,432.8197952 482 | 22.48279827,507.900282 483 | 30.08518963,691.8554843 484 | 16.99788893,448.3259814 485 | 27.28106383,612.2419632 486 | 4.865873622,188.1513313 487 | 23.40725697,501.34533 488 | 12.30161489,333.3342585 489 | 32.63285815,793.079011 490 | 16.70385182,379.318226 491 | 26.96421749,581.0740052 492 | 23.82492237,584.399945 493 | 34.47216919,809.3525195 494 | 23.05621357,552.8193512 495 | 14.93150577,377.4309279 496 | 25.11206572,571.4342569 497 | 22.27489926,524.7463643 498 | 32.89309211,755.8183987 499 | 12.58815695,306.0907189 500 | 22.36240237,566.2173038 501 | 28.95773632,655.6603879 502 | -------------------------------------------------------------------------------- /traditional_ML_algorithms/SimpleLinearRegression/model.py: -------------------------------------------------------------------------------- 1 | class SimpleLinearRegression: 2 | def fit(self, X, y): 3 | #Preprocessing 4 | X = pd.DataFrame(X) 5 | y = pd.DataFrame(y) 6 | data = pd.concat([X,y], axis=1) 7 | 8 | #Calculating mean for both Target and Feature Variable 9 | meanX = float(X.mean()) 10 | meanY = float(y.mean()) 11 | 12 | #Calculating `x-mean` and `y-mean` for each data point 13 | data['x-mean'] = X - meanX 14 | data['y-mean'] = y - meanY 15 | 16 | #Also calculating product(`x-mean, y-mean`) and square(x-mean) 17 | data['mul'] = data['x-mean'] * data['y-mean'] 18 | data['sq'] = np.power(data['x-mean'], 2) 19 | 20 | #Summation of data['mul'] and data['sq'] 21 | sumXY = data['mul'].sum() 22 | sumX = data['sq'].sum() 23 | 24 | #Calculating the coefficients or weights 25 | global b1, b0 26 | b1 = sumXY/sumX 27 | b0 = meanY - b1 * meanX 28 | 29 | #We will be returning the coefficients/weights from this function 30 | return b0, b1 31 | 32 | def predict(self, X): 33 | predList, y = [], [] 34 | #Creating a list for Feature Variable 35 | for i in range(0, len(X)): 36 | predList.append(X[i]) 37 | #Making predictions over Feature Variable 38 | for i in predList: 39 | itemY = b0 + b1 * i 40 | y.append(itemY) 41 | #Returning the predictions list 42 | return list(y) 43 | 44 | def r2_score(self, y_pred, y_test): 45 | #Calculating r2 using formula 46 | r2 = ((1 - np.sum((y_test - y_pred) * 2) / np.sum((y_test - np.mean(y_test)) * 2)) * 100) 47 | return r2 --------------------------------------------------------------------------------