├── SVD.py
├── Vector_Projection.ipynb
├── Cosine_Dot_Product.ipynb
├── VectorSpace_in_DataScience.ipynb
├── Vector_Operations.ipynb
├── Linear_Algebra_Intution.ipynb
├── Matrix_Inverse.ipynb
├── Coding_1.ipynb
└── README.md
/SVD.py:
--------------------------------------------------------------------------------
1 | # -*- coding: utf-8 -*-
2 | """
3 | Created on Sat May 19 19:45:09 2018
4 |
5 | @author: Kalyan
6 | """
7 |
8 | #Full Singular Value Decomposition
9 | import numpy as np
10 |
11 | A=np.array([[1,1,1,0,0],
12 | [3,3,3,0,0],
13 | [4,4,4,0,0],
14 | [5,5,5,0,0],
15 | [0,2,0,4,4],
16 | [0,0,0,5,5],
17 | [0,1,0,2,2]])
18 |
19 |
20 | np.set_printoptions(suppress=True)
21 | np.set_printoptions(precision=3)
22 |
23 | print ('------FULL SVD EXAMPLE--------')
24 | U,E,VT=np.linalg.svd(A,full_matrices=True)
25 |
26 | print ("U:\n {}".format(U))
27 | print ("E:\n {}".format(E))
28 | print ("VT:\n {}".format(VT))
29 |
30 | #Full Singular Value Decomposition
31 |
32 | print ('------TRUNCATED SVD EXAMPLE--------')
33 | U,E,VT=np.linalg.svd(A,full_matrices=False)
34 |
35 | print ("U:\n {}".format(U))
36 | print ("E:\n {}".format(E))
37 | print ("VT:\n {}".format(VT))
--------------------------------------------------------------------------------
/Vector_Projection.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": null,
6 | "metadata": {},
7 | "outputs": [],
8 | "source": [
9 | "# Projection:\n",
10 | " The dot product of two vectors is defined as :\n",
11 | " * a.b=|a||b|cos(theta)\n",
12 | " * The term |b|cos(theta) is defined as the projection of the vector b on to a\n",
13 | " Now, if we write :\n",
14 | " > (a.b)/|a|=|b|cos(theta), is called the scalar projection\n",
15 | " > a(a.b)/|a||a|= vector projection"
16 | ]
17 | }
18 | ],
19 | "metadata": {
20 | "kernelspec": {
21 | "display_name": "Python 2",
22 | "language": "python",
23 | "name": "python2"
24 | },
25 | "language_info": {
26 | "codemirror_mode": {
27 | "name": "ipython",
28 | "version": 2
29 | },
30 | "file_extension": ".py",
31 | "mimetype": "text/x-python",
32 | "name": "python",
33 | "nbconvert_exporter": "python",
34 | "pygments_lexer": "ipython2",
35 | "version": "2.7.14"
36 | }
37 | },
38 | "nbformat": 4,
39 | "nbformat_minor": 2
40 | }
41 |
--------------------------------------------------------------------------------
/Cosine_Dot_Product.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": null,
6 | "metadata": {},
7 | "outputs": [],
8 | "source": [
9 | " #Modulus and dot product of vectors\n",
10 | " # 1. The dot product or the projection product of two vectors can be explained as :\n",
11 | " r.s = |r||s|cos(theta) , where theta is the angle between the two vectors\n",
12 | " Dot product of two vectors satisfy the following properties:\n",
13 | " * r.s = s.r (Commutative)\n",
14 | " * r.(s+t) = r.s+r.t (Distributive)\n",
15 | " * r.(ks) = k(rs) (Associative over Scalar Multiplication)\n",
16 | " #2. If two vectors are orthogonal to each other , the the value of :\n",
17 | " * r.s = 0 , because the value of cos(90)=0\n",
18 | " If two vectors are parallel to each other, then the value of:\n",
19 | " * r.s = |r||s|, because the value of cos(0)=1"
20 | ]
21 | }
22 | ],
23 | "metadata": {
24 | "kernelspec": {
25 | "display_name": "Python 2",
26 | "language": "python",
27 | "name": "python2"
28 | },
29 | "language_info": {
30 | "codemirror_mode": {
31 | "name": "ipython",
32 | "version": 2
33 | },
34 | "file_extension": ".py",
35 | "mimetype": "text/x-python",
36 | "name": "python",
37 | "nbconvert_exporter": "python",
38 | "pygments_lexer": "ipython2",
39 | "version": "2.7.14"
40 | }
41 | },
42 | "nbformat": 4,
43 | "nbformat_minor": 2
44 | }
45 |
--------------------------------------------------------------------------------
/VectorSpace_in_DataScience.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": null,
6 | "metadata": {},
7 | "outputs": [],
8 | "source": [
9 | "#Vectors in Machine Learning:\n",
10 | "\n",
11 | "Ans: From a Computer Science point of view,A vector can be defined as a list of numbers.\n",
12 | " From a Physical point of view, a vector can be defined as aposition in 3D space.\n",
13 | " If we visualize the parameters or features of an entity, a vector can be visualized spatially as\n",
14 | " a point in the 3D space.The vector can move in the the N dimensional feature space to find the globally\n",
15 | " locally optimum set of features in the feature space.\n",
16 | " So, it is very important to visulize the features as vectors in the N dimensional feature space, so Linear \n",
17 | " Algebra routines, and calculus formulaes can be applied in oder to solve them.\n",
18 | " "
19 | ]
20 | }
21 | ],
22 | "metadata": {
23 | "kernelspec": {
24 | "display_name": "Python 2",
25 | "language": "python",
26 | "name": "python2"
27 | },
28 | "language_info": {
29 | "codemirror_mode": {
30 | "name": "ipython",
31 | "version": 2
32 | },
33 | "file_extension": ".py",
34 | "mimetype": "text/x-python",
35 | "name": "python",
36 | "nbconvert_exporter": "python",
37 | "pygments_lexer": "ipython2",
38 | "version": "2.7.14"
39 | }
40 | },
41 | "nbformat": 4,
42 | "nbformat_minor": 2
43 | }
44 |
--------------------------------------------------------------------------------
/Vector_Operations.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": null,
6 | "metadata": {},
7 | "outputs": [],
8 | "source": [
9 | "#Vector Operations:\n",
10 | "\n",
11 | "# There are two fundamental operations that can be performed on vectors\n",
12 | "\n",
13 | "# 1. Vector addition and Subtraction:\n",
14 | "# Lets us define a co-ordinate space X,Y with two unit vectors X and Y. The space spanned by the \n",
15 | "# unit vectors is called the co-ordinate space.These unit vectors X and Y are called the basis vectors.\n",
16 | "\n",
17 | "# Vector addition and subtraction follows the associativity rule:\n",
18 | "# A+(B+C)=(A+B)+C\n",
19 | "\n",
20 | "# 2. Multiplication by a scalar constant\n",
21 | "# If we multiply a vector with a scalar constant, each of the components of the scalar vector is multiplied\n",
22 | "# by the scalar constant.\n",
23 | "# c[A] = [cA]\n",
24 | "# [B] [cB]"
25 | ]
26 | }
27 | ],
28 | "metadata": {
29 | "kernelspec": {
30 | "display_name": "Python 2",
31 | "language": "python",
32 | "name": "python2"
33 | },
34 | "language_info": {
35 | "codemirror_mode": {
36 | "name": "ipython",
37 | "version": 2
38 | },
39 | "file_extension": ".py",
40 | "mimetype": "text/x-python",
41 | "name": "python",
42 | "nbconvert_exporter": "python",
43 | "pygments_lexer": "ipython2",
44 | "version": "2.7.14"
45 | }
46 | },
47 | "nbformat": 4,
48 | "nbformat_minor": 2
49 | }
50 |
--------------------------------------------------------------------------------
/Linear_Algebra_Intution.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": null,
6 | "metadata": {},
7 | "outputs": [],
8 | "source": [
9 | "#Definition of Linear Algebra:\n",
10 | "\n",
11 | "#Ans: Linear Algebra can be defined as the study of vectors,vector spaces and mapping between vector spaces.\n",
12 | "# It emerged from the study of matrices and the knowledge that they can be solved using system of linear equations.\n",
13 | "\n",
14 | "#Use cases of Linear Algebra:\n",
15 | "\n",
16 | "#Ans: 1. Solving of linear equations like:\n",
17 | " 2a+3b=10\n",
18 | " 7b+5c=20\n",
19 | " The above eqaution can be decomposed into matrices and vectors like:\n",
20 | " [2 3 [a]=[10]\n",
21 | " 7 5][b] [20]\n",
22 | "# 2. Optimization Problem:\n",
23 | "# Another use can be to fit a line that best matches a data distribution.The objective is to find the \n",
24 | "# line or curve that best fits the data distribution by finding the optimal parameters of the line.\n",
25 | "\n",
26 | "# Exercise:\n",
27 | "\n",
28 | "#Solve the system of equations given by:\n",
29 | "\n",
30 | "3x−2y+z=7\n",
31 | "x+y+z=2\n",
32 | "3x−2y−z=3\n"
33 | ]
34 | }
35 | ],
36 | "metadata": {
37 | "kernelspec": {
38 | "display_name": "Python 2",
39 | "language": "python",
40 | "name": "python2"
41 | },
42 | "language_info": {
43 | "codemirror_mode": {
44 | "name": "ipython",
45 | "version": 2
46 | },
47 | "file_extension": ".py",
48 | "mimetype": "text/x-python",
49 | "name": "python",
50 | "nbconvert_exporter": "python",
51 | "pygments_lexer": "ipython2",
52 | "version": "2.7.14"
53 | }
54 | },
55 | "nbformat": 4,
56 | "nbformat_minor": 2
57 | }
58 |
--------------------------------------------------------------------------------
/Matrix_Inverse.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": 4,
6 | "metadata": {},
7 | "outputs": [
8 | {
9 | "name": "stdout",
10 | "output_type": "stream",
11 | "text": [
12 | "('The inverse of matrix ', [[1, 1, 3], [1, 2, 4], [1, 1, 2]], ' is = ', array([[ 0., -1., 2.],\n",
13 | " [-2., 1., 1.],\n",
14 | " [ 1., -0., -1.]]))\n"
15 | ]
16 | }
17 | ],
18 | "source": [
19 | "#Performing matrix inverse\n",
20 | "import numpy as np\n",
21 | "\n",
22 | "A=[[1,1,3],\n",
23 | " [1,2,4],\n",
24 | " [1,1,2]]\n",
25 | "Ainv=np.linalg.inv(A)\n",
26 | "print (\"The inverse of matrix \",A,\" is = \",Ainv)"
27 | ]
28 | },
29 | {
30 | "cell_type": "code",
31 | "execution_count": 7,
32 | "metadata": {},
33 | "outputs": [
34 | {
35 | "name": "stdout",
36 | "output_type": "stream",
37 | "text": [
38 | "('The value of vector r in Ar=s is = ', array([ 3. , -0.5, 0. ]))\n"
39 | ]
40 | }
41 | ],
42 | "source": [
43 | "#Solving the system of linear equations instead of taking inverse which is computationally expensive\n",
44 | "import numpy as np\n",
45 | "A = [[4, 6, 2],\n",
46 | " [3, 4, 1],\n",
47 | " [2, 8, 13]]\n",
48 | "\n",
49 | "s = [9, 7, 2]\n",
50 | "\n",
51 | "r=np.linalg.solve(A,s)\n",
52 | "\n",
53 | "print (\"The value of vector r in Ar=s is = \",r )"
54 | ]
55 | },
56 | {
57 | "cell_type": "code",
58 | "execution_count": 8,
59 | "metadata": {},
60 | "outputs": [
61 | {
62 | "name": "stdout",
63 | "output_type": "stream",
64 | "text": [
65 | "('The value of inverse = ', array([[-1.5, 0.5, 0.5],\n",
66 | " [ 2. , 0. , -1. ],\n",
67 | " [ 0.5, -0.5, 0.5]]))\n"
68 | ]
69 | }
70 | ],
71 | "source": [
72 | "import numpy as np\n",
73 | "A=[[1,1,1],\n",
74 | " [3,2,1],\n",
75 | " [2,1,2]]\n",
76 | "Ainv =np.linalg.inv(A)\n",
77 | "print (\"The value of inverse = \",Ainv)\n"
78 | ]
79 | },
80 | {
81 | "cell_type": "code",
82 | "execution_count": null,
83 | "metadata": {},
84 | "outputs": [],
85 | "source": []
86 | }
87 | ],
88 | "metadata": {
89 | "kernelspec": {
90 | "display_name": "Python 2",
91 | "language": "python",
92 | "name": "python2"
93 | },
94 | "language_info": {
95 | "codemirror_mode": {
96 | "name": "ipython",
97 | "version": 2
98 | },
99 | "file_extension": ".py",
100 | "mimetype": "text/x-python",
101 | "name": "python",
102 | "nbconvert_exporter": "python",
103 | "pygments_lexer": "ipython2",
104 | "version": "2.7.14"
105 | }
106 | },
107 | "nbformat": 4,
108 | "nbformat_minor": 2
109 | }
110 |
--------------------------------------------------------------------------------
/Coding_1.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": null,
6 | "metadata": {},
7 | "outputs": [],
8 | "source": [
9 | "'''\n",
10 | "Identifying special matrices\n",
11 | "Instructions\n",
12 | "In this assignment, you shall write a function that will test if a 4×4 matrix is singular,\n",
13 | "i.e. to determine if an inverse exists, before calculating it.\n",
14 | "\n",
15 | "You shall use the method of converting a matrix to echelon form, and testing if this fails by \n",
16 | "leaving zeros that can’t be removed on the leading diagonal.\n",
17 | "\n",
18 | "\n",
19 | "Matrices in Python\n",
20 | "In the numpy package in Python, matrices are indexed using zero for the top-most column and left-most row. \n",
21 | "I.e., the matrix structure looks like this:\n",
22 | "\n",
23 | "A[0, 0] A[0, 1] A[0, 2] A[0, 3]\n",
24 | "A[1, 0] A[1, 1] A[1, 2] A[1, 3]\n",
25 | "A[2, 0] A[2, 1] A[2, 2] A[2, 3]\n",
26 | "A[3, 0] A[3, 1] A[3, 2] A[3, 3]\n",
27 | "You can access the value of each element individually using,\n",
28 | "\n",
29 | "A[n, m]\n",
30 | "which will give the n'th row and m'th column (starting with zero). You can also access a whole row at a time using,\n",
31 | "\n",
32 | "A[n]\n",
33 | "'''\n",
34 | "\n",
35 | "import numpy as np\n",
36 | "\n",
37 | "# Our function will go through the matrix replacing each row in order turning it into echelon form.\n",
38 | "# If at any point it fails because it can't put a 1 in the leading diagonal,\n",
39 | "# we will return the value True, otherwise, we will return False.\n",
40 | "# There is no need to edit this function.\n",
41 | "def isSingular(A) :\n",
42 | " B = np.array(A, dtype=np.float_) # Make B as a copy of A, since we're going to alter it's values.\n",
43 | " try:\n",
44 | " fixRowZero(B)\n",
45 | " fixRowOne(B)\n",
46 | " fixRowTwo(B)\n",
47 | " fixRowThree(B)\n",
48 | " except MatrixIsSingular:\n",
49 | " return True\n",
50 | " return False\n",
51 | "\n",
52 | "# This next line defines our error flag. For when things go wrong if the matrix is singular.\n",
53 | "# There is no need to edit this line.\n",
54 | "class MatrixIsSingular(Exception): pass\n",
55 | "\n",
56 | "# For Row Zero, all we require is the first element is equal to 1.\n",
57 | "# We'll divide the row by the value of A[0, 0].\n",
58 | "# This will get us in trouble though if A[0, 0] equals 0, so first we'll test for that,\n",
59 | "# and if this is true, we'll add one of the lower rows to the first one before the division.\n",
60 | "# We'll repeat the test going down each lower row until we can do the division.\n",
61 | "# There is no need to edit this function.\n",
62 | "def fixRowZero(A) :\n",
63 | " if A[0,0] == 0 :\n",
64 | " A[0] = A[0] + A[1]\n",
65 | " if A[0,0] == 0 :\n",
66 | " A[0] = A[0] + A[2]\n",
67 | " if A[0,0] == 0 :\n",
68 | " A[0] = A[0] + A[3]\n",
69 | " if A[0,0] == 0 :\n",
70 | " raise MatrixIsSingular()\n",
71 | " A[0] = A[0] / A[0,0]\n",
72 | " return A\n",
73 | "\n",
74 | "# First we'll set the sub-diagonal elements to zero, i.e. A[1,0].\n",
75 | "# Next we want the diagonal element to be equal to one.\n",
76 | "# We'll divide the row by the value of A[1, 1].\n",
77 | "# Again, we need to test if this is zero.\n",
78 | "# If so, we'll add a lower row and repeat setting the sub-diagonal elements to zero.\n",
79 | "# There is no need to edit this function.\n",
80 | "def fixRowOne(A) :\n",
81 | " A[1] = A[1] - A[1,0] * A[0]\n",
82 | " if A[1,1] == 0 :\n",
83 | " A[1] = A[1] + A[2]\n",
84 | " A[1] = A[1] - A[1,0] * A[0]\n",
85 | " if A[1,1] == 0 :\n",
86 | " A[1] = A[1] + A[3]\n",
87 | " A[1] = A[1] - A[1,0] * A[0]\n",
88 | " if A[1,1] == 0 :\n",
89 | " raise MatrixIsSingular()\n",
90 | " A[1] = A[1] / A[1,1]\n",
91 | " return A\n",
92 | "\n",
93 | "# This is the first function that you should complete.\n",
94 | "# Follow the instructions inside the function at each comment.\n",
95 | "def fixRowTwo(A) :\n",
96 | " # Insert code below to set the sub-diagonal elements of row two to zero (there are two of them).\n",
97 | " A[2]=A[2]-A[0]*A[2,0]\n",
98 | " A[2]=A[2]-A[1]*A[2,1]\n",
99 | " \n",
100 | " \n",
101 | " # Next we'll test that the diagonal element is not zero.\n",
102 | " if A[2,2] == 0 :\n",
103 | " # Insert code below that adds a lower row to row 2.\n",
104 | " A[2]=A[2]+A[3]\n",
105 | " \n",
106 | " # Now repeat your code which sets the sub-diagonal elements to zero.\n",
107 | " A[2]=A[2]-A[0]*A[2,0]\n",
108 | " A[2]=A[2]-A[1]*A[2,1]\n",
109 | " if A[2,2] == 0 :\n",
110 | " raise MatrixIsSingular()\n",
111 | " # Finally set the diagonal element to one by dividing the whole row by that element.\n",
112 | " A[2]=A[2]/A[2,2]\n",
113 | " return A\n",
114 | "\n",
115 | "# You should also complete this function\n",
116 | "# Follow the instructions inside the function at each comment.\n",
117 | "def fixRowThree(A) :\n",
118 | " # Insert code below to set the sub-diagonal elements of row three to zero.\n",
119 | " A[3]=A[3]-A[0]*A[3,0]\n",
120 | " A[3]=A[3]-A[1]*A[3,1]\n",
121 | " A[3]=A[3]-A[2]*A[3,2]\n",
122 | " \n",
123 | " # Complete the if statement to test if the diagonal element is zero.\n",
124 | " if A[3,3]==0:\n",
125 | " raise MatrixIsSingular()\n",
126 | " # Transform the row to set the diagonal element to one.\n",
127 | " A[3]=A[3]/A[3,3]\n",
128 | " return A"
129 | ]
130 | }
131 | ],
132 | "metadata": {
133 | "kernelspec": {
134 | "display_name": "Python 2",
135 | "language": "python",
136 | "name": "python2"
137 | },
138 | "language_info": {
139 | "codemirror_mode": {
140 | "name": "ipython",
141 | "version": 2
142 | },
143 | "file_extension": ".py",
144 | "mimetype": "text/x-python",
145 | "name": "python",
146 | "nbconvert_exporter": "python",
147 | "pygments_lexer": "ipython2",
148 | "version": "2.7.14"
149 | }
150 | },
151 | "nbformat": 4,
152 | "nbformat_minor": 2
153 | }
154 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 |
2 |
3 | MATHEMATICS FOR MACHINE LEARNING
4 |
5 | This repository contains code and theory of mathematical concepts required to master Machine Learning.
6 | This repository will give any budding beginner in Machine Learning a solid foundation on the important concepts
7 | in Linear Algebra and Multivariate Calculus.I have included IPython notebooks which has code snippets and important
8 | explanations.
9 |
10 | So, Go ahead, Fork this repo and Happy Machine Learning :)
11 |
12 | TABLE OF CONTENTS:
13 |
14 | 1. LINEAR ALGEBRA:
15 |
16 |
17 | -Linear Algebra Intuition:
18 | ```python
19 | #Definition of Linear Algebra:
20 |
21 | #Ans: Linear Algebra can be defined as the study of vectors,vector spaces and mapping between vector spaces.
22 | # It emerged from the study of matrices and the knowledge that they can be solved using system of linear equations.
23 |
24 | #Use cases of Linear Algebra:
25 |
26 | #Ans: 1. Solving of linear equations like:
27 | 2a+3b=10
28 | 7b+5c=20
29 | The above eqaution can be decomposed into matrices and vectors like:
30 | [2 3 [a]=[10]
31 | 7 5][b] [20]
32 | # 2. Optimization Problem:
33 | # Another use can be to fit a line that best matches a data distribution.The objective is to find the
34 | # line or curve that best fits the data distribution by finding the optimal parameters of the line.
35 |
36 | # Exercise:
37 |
38 | #Solve the system of equations given by:
39 |
40 | 3x−2y+z=7
41 | x+y+z=2
42 | 3x−2y−z=3
43 | ```
44 |
45 |
46 | -Vector Space in DataScience:
47 | ```python
48 | #Vectors in Machine Learning:
49 |
50 | Ans: From a Computer Science point of view,A vector can be defined as a list of numbers.
51 | From a Physical point of view, a vector can be defined as a position in 3D space.
52 | If we visualize the parameters or features of an entity, a vector can be visualized spatially as
53 | a point in the 3D space.The vector can move in the the N dimensional feature space to find the globally
54 | locally optimum set of features in the feature space.
55 | So, it is very important to visulize the features as vectors in the N dimensional feature space, so Linear
56 | Algebra routines, and calculus formulaes can be applied in oder to solve them.
57 | ```
58 |
59 | -Vector_Operations:
60 | ```python
61 | #Vector Operations:
62 |
63 | # There are two fundamental operations that can be performed on vectors
64 |
65 | # 1. Vector addition and Subtraction:
66 | # Lets us define a co-ordinate space X,Y with two unit vectors X and Y. The space spanned by the
67 | # unit vectors is called the co-ordinate space.These unit vectors X and Y are called the basis vectors.
68 |
69 | # Vector addition and subtraction follows the associativity rule:
70 | # A+(B+C)=(A+B)+C
71 |
72 | # 2. Multiplication by a scalar constant
73 | # If we multiply a vector with a scalar constant, each of the components of the scalar vector is multiplied
74 | # by the scalar constant.
75 | # c[A] = [cA]
76 | # [B] [cB]
77 | ```
78 |
79 | -Cosine_Dot_Product:
80 | ```python
81 | #Modulus and dot product of vectors
82 | # 1. The dot product or the projection product of two vectors can be explained as :
83 | r.s = |r||s|cos(theta) , where theta is the angle between the two vectors
84 | Dot product of two vectors satisfy the following properties:
85 | * r.s = s.r (Commutative)
86 | * r.(s+t) = r.s+r.t (Distributive)
87 | * r.(ks) = k(rs) (Associative over Scalar Multiplication)
88 | #2. If two vectors are orthogonal to each other , the the value of :
89 | * r.s = 0 , because the value of cos(90)=0
90 | If two vectors are parallel to each other, then the value of:
91 | * r.s = |r||s|, because the value of cos(0)=1
92 | ```
93 |
94 | -Vector_Projection:
95 | ```python
96 | # Projection:
97 | The dot product of two vectors is defined as :
98 | * a.b=|a||b|cos(theta)
99 | * The term |b|cos(theta) is defined as the projection of the vector b on to a
100 | Now, if we write :
101 | > (a.b)/|a|=|b|cos(theta), is called the scalar projection
102 | > a(a.b)/|a||a|= vector projection
103 | ```
104 |
105 | -Matrix_Inverse:
106 | ```python
107 | #Performing matrix inverse
108 | import numpy as np
109 |
110 | A=[[1,1,3],
111 | [1,2,4],
112 | [1,1,2]]
113 | Ainv=np.linalg.inv(A)
114 | print ("The inverse of matrix ",A," is = ",Ainv)
115 | ```
116 | ```python
117 | #Solving the system of linear equations instead of taking inverse which is computationally expensive
118 | import numpy as np
119 | A = [[4, 6, 2],
120 | [3, 4, 1],
121 | [2, 8, 13]]
122 | s = [9, 7, 2]
123 | r=np.linalg.solve(A,s)
124 | print ("The value of vector r in Ar=s is = ",r )
125 | ```
126 | ```python
127 | import numpy as np
128 | A=[[1,1,1],
129 | [3,2,1],
130 | [2,1,2]]
131 | Ainv =np.linalg.inv(A)
132 | print ("The value of inverse = ",Ainv)
133 | ```
134 |
135 | -Coding Examples:
136 | ```python
137 | '''
138 | Identifying special matrices
139 | Instructions
140 | In this assignment, you shall write a function that will test if a 4×4 matrix is singular,
141 | i.e. to determine if an inverse exists, before calculating it.
142 |
143 | You shall use the method of converting a matrix to echelon form, and testing if this fails by
144 | leaving zeros that can’t be removed on the leading diagonal.
145 |
146 |
147 | Matrices in Python
148 | In the numpy package in Python, matrices are indexed using zero for the top-most column and left-most row.
149 | I.e., the matrix structure looks like this:
150 |
151 | A[0, 0] A[0, 1] A[0, 2] A[0, 3]
152 | A[1, 0] A[1, 1] A[1, 2] A[1, 3]
153 | A[2, 0] A[2, 1] A[2, 2] A[2, 3]
154 | A[3, 0] A[3, 1] A[3, 2] A[3, 3]
155 | You can access the value of each element individually using,
156 |
157 | A[n, m]
158 | which will give the n'th row and m'th column (starting with zero). You can also access a whole row at a time using,
159 |
160 | A[n]
161 | '''
162 |
163 | import numpy as np
164 |
165 | # Our function will go through the matrix replacing each row in order turning it into echelon form.
166 | # If at any point it fails because it can't put a 1 in the leading diagonal,
167 | # we will return the value True, otherwise, we will return False.
168 | # There is no need to edit this function.
169 | def isSingular(A) :
170 | B = np.array(A, dtype=np.float_) # Make B as a copy of A, since we're going to alter it's values.
171 | try:
172 | fixRowZero(B)
173 | fixRowOne(B)
174 | fixRowTwo(B)
175 | fixRowThree(B)
176 | except MatrixIsSingular:
177 | return True
178 | return False
179 |
180 | # This next line defines our error flag. For when things go wrong if the matrix is singular.
181 | # There is no need to edit this line.
182 | class MatrixIsSingular(Exception): pass
183 |
184 | # For Row Zero, all we require is the first element is equal to 1.
185 | # We'll divide the row by the value of A[0, 0].
186 | # This will get us in trouble though if A[0, 0] equals 0, so first we'll test for that,
187 | # and if this is true, we'll add one of the lower rows to the first one before the division.
188 | # We'll repeat the test going down each lower row until we can do the division.
189 | # There is no need to edit this function.
190 | def fixRowZero(A) :
191 | if A[0,0] == 0 :
192 | A[0] = A[0] + A[1]
193 | if A[0,0] == 0 :
194 | A[0] = A[0] + A[2]
195 | if A[0,0] == 0 :
196 | A[0] = A[0] + A[3]
197 | if A[0,0] == 0 :
198 | raise MatrixIsSingular()
199 | A[0] = A[0] / A[0,0]
200 | return A
201 |
202 | # First we'll set the sub-diagonal elements to zero, i.e. A[1,0].
203 | # Next we want the diagonal element to be equal to one.
204 | # We'll divide the row by the value of A[1, 1].
205 | # Again, we need to test if this is zero.
206 | # If so, we'll add a lower row and repeat setting the sub-diagonal elements to zero.
207 | # There is no need to edit this function.
208 | def fixRowOne(A) :
209 | A[1] = A[1] - A[1,0] * A[0]
210 | if A[1,1] == 0 :
211 | A[1] = A[1] + A[2]
212 | A[1] = A[1] - A[1,0] * A[0]
213 | if A[1,1] == 0 :
214 | A[1] = A[1] + A[3]
215 | A[1] = A[1] - A[1,0] * A[0]
216 | if A[1,1] == 0 :
217 | raise MatrixIsSingular()
218 | A[1] = A[1] / A[1,1]
219 | return A
220 |
221 | # This is the first function that you should complete.
222 | # Follow the instructions inside the function at each comment.
223 | def fixRowTwo(A) :
224 | # Insert code below to set the sub-diagonal elements of row two to zero (there are two of them).
225 | A[2]=A[2]-A[0]*A[2,0]
226 | A[2]=A[2]-A[1]*A[2,1]
227 |
228 |
229 | # Next we'll test that the diagonal element is not zero.
230 | if A[2,2] == 0 :
231 | # Insert code below that adds a lower row to row 2.
232 | A[2]=A[2]+A[3]
233 |
234 | # Now repeat your code which sets the sub-diagonal elements to zero.
235 | A[2]=A[2]-A[0]*A[2,0]
236 | A[2]=A[2]-A[1]*A[2,1]
237 | if A[2,2] == 0 :
238 | raise MatrixIsSingular()
239 | # Finally set the diagonal element to one by dividing the whole row by that element.
240 | A[2]=A[2]/A[2,2]
241 | return A
242 |
243 | # You should also complete this function
244 | # Follow the instructions inside the function at each comment.
245 | def fixRowThree(A) :
246 | # Insert code below to set the sub-diagonal elements of row three to zero.
247 | A[3]=A[3]-A[0]*A[3,0]
248 | A[3]=A[3]-A[1]*A[3,1]
249 | A[3]=A[3]-A[2]*A[3,2]
250 |
251 | # Complete the if statement to test if the diagonal element is zero.
252 | if A[3,3]==0:
253 | raise MatrixIsSingular()
254 | # Transform the row to set the diagonal element to one.
255 | A[3]=A[3]/A[3,3]
256 | return A
257 | ```
258 | ## SINGULAR VALUE DECOMPOSITION:
259 | [Best SVD Tutorial Video :](https://www.youtube.com/watch?v=P5mlg91as1c)
260 |
261 |
262 | -Coding Examples:
263 | ```python
264 | '''
265 | # -*- coding: utf-8 -*-
266 | """
267 | Created on Sat May 19 19:45:09 2018
268 |
269 | @author: Kalyan
270 | """
271 |
272 | #Full Singular Value Decomposition
273 | import numpy as np
274 |
275 | A=np.array([[1,1,1,0,0],
276 | [3,3,3,0,0],
277 | [4,4,4,0,0],
278 | [5,5,5,0,0],
279 | [0,2,0,4,4],
280 | [0,0,0,5,5],
281 | [0,1,0,2,2]])
282 |
283 |
284 | np.set_printoptions(suppress=True)
285 | np.set_printoptions(precision=3)
286 |
287 | print ('------FULL SVD EXAMPLE--------')
288 | U,E,VT=np.linalg.svd(A,full_matrices=True)
289 |
290 | print ("U:\n {}".format(U))
291 | print ("E:\n {}".format(E))
292 | print ("VT:\n {}".format(VT))
293 |
294 | #Full Singular Value Decomposition
295 |
296 | print ('------TRUNCATED SVD EXAMPLE--------')
297 | U,E,VT=np.linalg.svd(A,full_matrices=False)
298 |
299 | print ("U:\n {}".format(U))
300 | print ("E:\n {}".format(E))
301 | print ("VT:\n {}".format(VT))
302 | '''
303 |
--------------------------------------------------------------------------------