├── Interpretable-AI ├── reading-course.md └── README.md ├── nn_torch_scratch.py ├── circle_nn.py ├── mult_circle_nn.py ├── circle_exp.py ├── poly_2d_nn.py ├── eye_2d_nn.py └── Linear-Algebra ├── Linear_Algebra_Lesson_1.ipynb └── Linear_Algebra_Lesson_2.ipynb /Interpretable-AI/reading-course.md: -------------------------------------------------------------------------------- 1 | # Interpretability AI Reading Course 2 | 3 | --- 4 | 5 | Please read it sequentially. 6 | 7 | --- 8 | 9 | * https://papers.baulab.info/ 10 | * https://distill.pub/2017/feature-visualization/ 11 | * https://distill.pub/2017/aia/ 12 | * https://distill.pub/2018/building-blocks/ 13 | * https://distill.pub/2018/feature-wise-transformations/ 14 | * https://distill.pub/2018/differentiable-parameterizations/ 15 | * https://distill.pub/2020/attribution-baselines/ 16 | * https://distill.pub/2020/grand-tour/ 17 | * https://distill.pub/2021/multimodal-neurons/ 18 | * https://distill.pub/2019/activation-atlas/ 19 | * (Circuits) https://distill.pub/2020/circuits/ 20 | * https://transformer-circuits.pub/ 21 | * https://transformer-circuits.pub/2023/interpretability-dreams/index.html 22 | * https://www.anthropic.com/news/mapping-mind-language-model 23 | * https://www.anthropic.com/news/golden-gate-claude 24 | * Introduction to Interpretability: https://www.youtube.com/watch?v=TxhhMTOTMDg 25 | * Podcast & Discussion: https://open.spotify.com/episode/5UF79Uu94ia0fwC32a89LU 26 | * Interpretability Challenges: https://www.anthropic.com/research/engineering-challenges-interpretability 27 | * Chris Olah’s Blog: https://colah.github.io/about.html 28 | * Resource from Distill Journal: https://distill.pub/ 29 | * https://www.neelnanda.io/mechanistic-interpretability 30 | * https://www.alignmentforum.org/ 31 | * https://www.alignmentforum.org/w/interpretability-ml-and-ai 32 | -------------------------------------------------------------------------------- /Interpretable-AI/README.md: -------------------------------------------------------------------------------- 1 | # Interpretable AI 2 | 3 | * [Join this WhatsApp community](https://chat.whatsapp.com/Ltny00USikU1JEuVArJHwW) for discussion & access to micro content. 4 | * ⭐ Star this repository if you find enough value, and use it for your work. 5 | * If you want me to add something to it, email me at support@srijitmukherjee.com. 6 | 7 | --- 8 | 9 | ## How to start? 10 | 11 | * Start here with a [vision interpretable ai tutorial](https://github.com/mukherjeesrijit/interpretable-ai/blob/main/vision-interpretable-ai-tutorial-1.ipynb). 12 | * Then, go on to the [reading course](https://github.com/mukherjeesrijit/interpretable-ai/blob/main/reading-course.md). 13 | * More tutorials and reading courses are coming, soon. 14 | * Want specific to medical or domain-specific Interpretable AI? Email me. 15 | 16 | --- 17 | 18 | ## What is the goal? 19 | 20 | This repository is directed toward building tutorials, resources, research papers, and code bases in pytorch for interpretable AI methodologies. My goal is to help everyone build a non-black-box AI and fully understand and look inside the model, one is creating. AI is an artificial brain, that we are striving towards. We don't understand our own brain, but at least we should strive to build the brain, we are building with our own hands, right? 21 | 22 | ## Why does Interpretable AI matter? 23 | 24 | Humans operate in low-dimensional space. Machines operate in high-dimensional space. Humans can adapt to high-dimensional thinking. This is where abstraction becomes essential. Understanding machines is crucial. Not because they will take over, but because they outperform us in many tasks. The problem? We don’t understand why. To evolve, we must bridge this gap. We need to understand how machines think. Machines operate in abstract spaces, without self-awareness or intent. Humans must learn to interpret them. Train their minds. Use AI effectively. Not as a replacement, but as a tool. This is how we push humanity forward. 25 | -------------------------------------------------------------------------------- /nn_torch_scratch.py: -------------------------------------------------------------------------------- 1 | import torch 2 | 3 | # Step 1: Model Creation 4 | # Reminder: No nn.Module is here. 5 | 6 | class MyModel: 7 | def __init__(self): 8 | # Leaf tensors: requires_grad=True, .grad=None 9 | self.W1 = torch.randn(2, 3, requires_grad=True) # input 2 → hidden 3 10 | self.b1 = torch.zeros(3, requires_grad=True) 11 | self.W2 = torch.randn(3, 1, requires_grad=True) # hidden 3 → output 1 12 | self.b2 = torch.zeros(1, requires_grad=True) 13 | # Store parameters in a list for easy optimizer step 14 | self.params = [self.W1, self.b1, self.W2, self.b2] 15 | 16 | # Step 2: Forward Pass 17 | def forward(self, x): 18 | self.z1 = x @ self.W1 + self.b1 # Linear1 19 | self.a1 = torch.relu(self.z1) # Activation 20 | self.z2 = self.a1 @ self.W2 + self.b2 # Linear2 21 | return self.z2 22 | 23 | # Step 3: Input 24 | xb = torch.tensor([[1.0, 2.0]]) # shape (1,2) 25 | yb = torch.tensor([[1.0]]) # target 26 | 27 | # Instantiate model 28 | model = MyModel() 29 | 30 | # Step 4: Loss Computation (custom MSE loss) 31 | def mse_loss(pred, target): 32 | # Returns a scalar tensor connected to graph 33 | return ((pred - target)**2).mean() 34 | 35 | # Step 5: Custom optimizer (SGD) 36 | # No optimizer is called here. 37 | 38 | class SGD: 39 | def __init__(self, params, lr=0.1): 40 | self.params = params 41 | self.lr = lr 42 | 43 | # Update step using .grad 44 | def step(self): 45 | for p in self.params: 46 | if p.grad is not None: 47 | p.data -= self.lr * p.grad 48 | 49 | # Step 6: Zero gradients 50 | def zero_grad(self): 51 | for p in self.params: 52 | if p.grad is not None: 53 | p.grad.zero_() 54 | 55 | optimizer = SGD(model.params, lr=0.1) 56 | 57 | # Training loop for 5 steps 58 | for i in range(5): 59 | # Step 7: Forward Pass 60 | output = model.forward(xb) 61 | 62 | # Step 8: Loss Computation 63 | loss = mse_loss(output, yb) 64 | print(f"Step {i}, Loss: {loss.item()}") 65 | 66 | # Step 9: Backward Pass [The Magic Happens here!!!! 🔴] 67 | loss.backward() # populates .grad for leaf tensors (W1, b1, W2, b2) 68 | 69 | # Step 10: Optimizer Step 70 | optimizer.step() 71 | 72 | # Step 11: Zero Gradients 73 | optimizer.zero_grad() 74 | -------------------------------------------------------------------------------- /circle_nn.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torch.optim as optim 4 | import numpy as np 5 | import matplotlib.pyplot as plt 6 | from scipy.spatial.distance import directed_hausdorff 7 | 8 | # --------------------------- 9 | # 1. Generate training data 10 | # --------------------------- 11 | 12 | # one circle 13 | def generate_circle_data(n_samples=1000, radius=1.0, noise_std=0.1): 14 | X = np.random.uniform(-1.5, 1.5, (n_samples, 2)) 15 | y = np.array([1 if xi**2 + yi**2 <= radius**2 else 0 for xi, yi in X]) 16 | X += np.random.normal(0, noise_std, X.shape) 17 | return torch.tensor(X, dtype=torch.float32), torch.tensor(y, dtype=torch.float32).unsqueeze(1) 18 | 19 | X_train, y_train = generate_circle_data(2000) 20 | 21 | # --------------------------- 22 | # 2. Define flexible MLP 23 | # --------------------------- 24 | class MLP(nn.Module): 25 | def __init__(self, input_dim=2, hidden_layers=[7, 6], output_dim=1): 26 | super().__init__() 27 | layers = [] 28 | in_dim = input_dim 29 | for h in hidden_layers: 30 | layers.append(nn.Linear(in_dim, h)) 31 | layers.append(nn.ReLU()) 32 | in_dim = h 33 | layers.append(nn.Linear(in_dim, output_dim)) 34 | layers.append(nn.Sigmoid()) # Binary classification 35 | self.model = nn.Sequential(*layers) 36 | 37 | def forward(self, x): 38 | return self.model(x) 39 | 40 | # Customize hidden layers here: 41 | hidden_layers = [50, 50] # You can change this to [7], [7,6,5], etc. 42 | model = MLP(hidden_layers=hidden_layers) 43 | 44 | # --------------------------- 45 | # 3. Training 46 | # --------------------------- 47 | criterion = nn.BCELoss() 48 | optimizer = optim.Adam(model.parameters(), lr=0.01) 49 | 50 | epochs = 100 51 | for epoch in range(epochs): 52 | optimizer.zero_grad() 53 | outputs = model(X_train) 54 | loss = criterion(outputs, y_train) 55 | loss.backward() 56 | optimizer.step() 57 | 58 | # if (epoch+1) % 10 == 0: 59 | # print(f"Epoch {epoch+1}/{epochs}, Loss: {loss.item():.4f}") 60 | 61 | # --------------------------- 62 | # 4. Plot decision boundary 63 | # --------------------------- 64 | def plot_decision_boundary(model, X, y, resolution=0.01): 65 | x_min, x_max = X[:,0].min()-0.1, X[:,0].max()+0.1 66 | y_min, y_max = X[:,1].min()-0.1, X[:,1].max()+0.1 67 | 68 | xx, yy = np.meshgrid(np.arange(x_min, x_max, resolution), 69 | np.arange(y_min, y_max, resolution)) 70 | 71 | grid = torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32) 72 | 73 | with torch.no_grad(): 74 | Z = model(grid).reshape(xx.shape).numpy() 75 | 76 | plt.contourf(xx, yy, Z, levels=[0,0.5,1], alpha=0.3, colors=['blue','orange']) 77 | plt.scatter(X[:,0], X[:,1], c=y.squeeze(), cmap='bwr', edgecolor='k', s=20) 78 | 79 | cs = plt.contour(xx, yy, Z, levels=[0.5], colors='black') 80 | 81 | # robust extraction of the contour line (level index = 0) 82 | pred_boundary = cs.allsegs[0][0] # always safe 83 | 84 | # True circle 85 | theta = np.linspace(0, 2*np.pi, 400) 86 | circle_pts = np.vstack([np.cos(theta), np.sin(theta)]).T 87 | 88 | h1 = directed_hausdorff(circle_pts, pred_boundary)[0] 89 | h2 = directed_hausdorff(pred_boundary, circle_pts)[0] 90 | hausdorff_dist = max(h1, h2) 91 | print("Hausdorff distance:", hausdorff_dist) 92 | 93 | circle = plt.Circle((0,0),1, color='green', fill=False, linewidth=2, linestyle='dashed') 94 | plt.gca().add_artist(circle) 95 | 96 | plt.xlabel('x') 97 | plt.ylabel('y') 98 | plt.title('Decision Boundary') 99 | plt.show() 100 | 101 | # Print parameter counts 102 | total_params = sum(p.numel() for p in model.parameters()) 103 | print(f"{hidden_layers} has total parameters:", total_params) 104 | print("loss:", loss.item()) 105 | d = plot_decision_boundary(model, X_train.numpy(), y_train.numpy()) 106 | -------------------------------------------------------------------------------- /mult_circle_nn.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torch.optim as optim 4 | import numpy as np 5 | import matplotlib.pyplot as plt 6 | from scipy.spatial.distance import directed_hausdorff 7 | 8 | # --------------------------- 9 | # 1. Generate training data for N circles 10 | # --------------------------- 11 | def generate_multi_circle_data(n_samples=1000, circle_centers=[(0,0)], radius=1.0, noise_std=0.1): 12 | X = np.random.uniform(-5, 5, (n_samples, 2)) # adjust square bounds as needed 13 | y = np.array([1 if any((xi - cx)**2 + (yi - cy)**2 <= radius**2 for cx, cy in circle_centers) else 0 14 | for xi, yi in X]) 15 | X += np.random.normal(0, noise_std, X.shape) 16 | return torch.tensor(X, dtype=torch.float32), torch.tensor(y, dtype=torch.float32).unsqueeze(1) 17 | 18 | # Example: two disconnected circles 19 | circle_centers = [(-1, 0), (2, 0)] 20 | radius = 1 21 | X_train, y_train = generate_multi_circle_data(2000, circle_centers, radius, noise_std=0.1) 22 | 23 | # --------------------------- 24 | # 2. Define flexible MLP 25 | # --------------------------- 26 | class MLP(nn.Module): 27 | def __init__(self, input_dim=2, hidden_layers=[7, 6], output_dim=1): 28 | super().__init__() 29 | layers = [] 30 | in_dim = input_dim 31 | for h in hidden_layers: 32 | layers.append(nn.Linear(in_dim, h)) 33 | layers.append(nn.ReLU()) 34 | in_dim = h 35 | layers.append(nn.Linear(in_dim, output_dim)) 36 | layers.append(nn.Sigmoid()) 37 | self.model = nn.Sequential(*layers) 38 | 39 | def forward(self, x): 40 | return self.model(x) 41 | 42 | # Customize hidden layers 43 | hidden_layers = [5000] 44 | model = MLP(hidden_layers=hidden_layers) 45 | 46 | # --------------------------- 47 | # 3. Training 48 | # --------------------------- 49 | criterion = nn.BCELoss() 50 | optimizer = optim.Adam(model.parameters(), lr=0.01) 51 | epochs = 100 52 | 53 | for epoch in range(epochs): 54 | optimizer.zero_grad() 55 | outputs = model(X_train) 56 | loss = criterion(outputs, y_train) 57 | loss.backward() 58 | optimizer.step() 59 | 60 | # --------------------------- 61 | # 4. General decision boundary plot 62 | # --------------------------- 63 | def plot_decision_boundary_general(model, X, y, circle_centers=None, radius=1.0, resolution=0.01): 64 | x_min, x_max = X[:,0].min()-0.1, X[:,0].max()+0.1 65 | y_min, y_max = X[:,1].min()-0.1, X[:,1].max()+0.1 66 | 67 | xx, yy = np.meshgrid(np.arange(x_min, x_max, resolution), 68 | np.arange(y_min, y_max, resolution)) 69 | grid = torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32) 70 | 71 | with torch.no_grad(): 72 | Z = model(grid).reshape(xx.shape).numpy() 73 | 74 | plt.contourf(xx, yy, Z, levels=[0,0.5,1], alpha=0.3, colors=['blue','orange']) 75 | plt.scatter(X[:,0], X[:,1], c=y.squeeze(), cmap='bwr', edgecolor='k', s=20) 76 | 77 | cs = plt.contour(xx, yy, Z, levels=[0.5], colors='black') 78 | pred_boundary = cs.allsegs[0][0] 79 | 80 | # Compute Hausdorff distance against all circles 81 | if circle_centers is None: 82 | circle_centers = [(0,0)] 83 | all_circle_pts = [] 84 | theta = np.linspace(0, 2*np.pi, 400) 85 | for cx, cy in circle_centers: 86 | pts = np.vstack([cx + radius*np.cos(theta), cy + radius*np.sin(theta)]).T 87 | all_circle_pts.append(pts) 88 | all_circle_pts = np.vstack(all_circle_pts) 89 | 90 | h1 = directed_hausdorff(all_circle_pts, pred_boundary)[0] 91 | h2 = directed_hausdorff(pred_boundary, all_circle_pts)[0] 92 | hausdorff_dist = max(h1, h2) 93 | print("Hausdorff distance:", hausdorff_dist) 94 | 95 | # Plot true circles 96 | for cx, cy in circle_centers: 97 | circle = plt.Circle((cx, cy), radius, color='green', fill=False, linewidth=2, linestyle='dashed') 98 | plt.gca().add_artist(circle) 99 | 100 | plt.xlabel('x') 101 | plt.ylabel('y') 102 | plt.title('Decision Boundary') 103 | plt.show() 104 | 105 | return hausdorff_dist 106 | 107 | # --------------------------- 108 | # 5. Run plot and print stats 109 | # --------------------------- 110 | total_params = sum(p.numel() for p in model.parameters()) 111 | print(f"{hidden_layers} has total parameters:", total_params) 112 | print("loss:", loss.item()) 113 | 114 | hausdorff_distance = plot_decision_boundary_general(model, X_train.numpy(), y_train.numpy(), 115 | circle_centers=circle_centers, radius=radius) 116 | -------------------------------------------------------------------------------- /circle_exp.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torch.optim as optim 4 | import numpy as np 5 | from scipy.spatial.distance import directed_hausdorff 6 | import csv 7 | import matplotlib.pyplot as plt 8 | 9 | 10 | def generate_circle_data(n_samples=1000, radius=1.0, noise_std=0.05): 11 | X = np.random.uniform(-1.5, 1.5, (n_samples, 2)) 12 | y = np.array([1 if xi**2 + yi**2 <= radius**2 else 0 for xi, yi in X]) 13 | X += np.random.normal(0, noise_std, X.shape) 14 | return torch.tensor(X, dtype=torch.float32), torch.tensor(y, dtype=torch.float32).unsqueeze(1) 15 | 16 | 17 | class MLP(nn.Module): 18 | def __init__(self, input_dim=2, hidden_layers=[10], output_dim=1): 19 | super().__init__() 20 | layers = [] 21 | in_dim = input_dim 22 | for h in hidden_layers: 23 | layers.append(nn.Linear(in_dim, h)) 24 | layers.append(nn.ReLU()) 25 | in_dim = h 26 | layers.append(nn.Linear(in_dim, output_dim)) 27 | layers.append(nn.Sigmoid()) 28 | self.model = nn.Sequential(*layers) 29 | 30 | def forward(self, x): 31 | return self.model(x) 32 | 33 | 34 | def hausdorff_from_model(model, resolution=0.01): 35 | xx, yy = np.meshgrid(np.arange(-1.5, 1.5, resolution), 36 | np.arange(-1.5, 1.5, resolution)) 37 | 38 | grid = torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32) 39 | 40 | with torch.no_grad(): 41 | Z = model(grid).reshape(xx.shape).numpy() 42 | 43 | cs = plt.contour(xx, yy, Z, levels=[0.5]) 44 | pred_boundary = cs.allsegs[0][0] 45 | 46 | theta = np.linspace(0, 2*np.pi, 400) 47 | circle_pts = np.vstack([np.cos(theta), np.sin(theta)]).T 48 | 49 | h1 = directed_hausdorff(circle_pts, pred_boundary)[0] 50 | h2 = directed_hausdorff(pred_boundary, circle_pts)[0] 51 | return max(h1, h2) 52 | 53 | 54 | def train_and_eval(hidden_layers, runs=10, epochs=100): 55 | losses = [] 56 | hausdorffs = [] 57 | 58 | for _ in range(runs): 59 | model = MLP(hidden_layers=hidden_layers) 60 | X, y = generate_circle_data() 61 | 62 | optimizer = optim.Adam(model.parameters(), lr=0.01) 63 | criterion = nn.BCELoss() 64 | 65 | for _ in range(epochs): 66 | optimizer.zero_grad() 67 | pred = model(X) 68 | loss = criterion(pred, y) 69 | loss.backward() 70 | optimizer.step() 71 | 72 | losses.append(loss.item()) 73 | 74 | # compute Hausdorff distance 75 | xx, yy = np.meshgrid(np.linspace(-1.5, 1.5, 300), 76 | np.linspace(-1.5, 1.5, 300)) 77 | grid = torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32) 78 | with torch.no_grad(): 79 | Z = model(grid).reshape(xx.shape).numpy() 80 | 81 | # predicted boundary 82 | cs = plt.contour(xx, yy, Z, levels=[0.5]) 83 | pred_boundary = cs.allsegs[0][0] 84 | 85 | # true circle 86 | theta = np.linspace(0, 2*np.pi, 400) 87 | circle_pts = np.vstack([np.cos(theta), np.sin(theta)]).T 88 | 89 | h1 = directed_hausdorff(circle_pts, pred_boundary)[0] 90 | h2 = directed_hausdorff(pred_boundary, circle_pts)[0] 91 | hd = max(h1, h2) 92 | hausdorffs.append(hd) 93 | 94 | plt.close() # prevent plotting 95 | 96 | total_params = sum(p.numel() for p in model.parameters()) 97 | 98 | return total_params, np.mean(losses), np.mean(hausdorffs) 99 | 100 | 101 | def run_experiments_and_save_csv(): 102 | single_layer_sizes = list(range(1, 80, 5)) 103 | two_layer_sizes = [1, 5, 10, 15, 20] 104 | 105 | rows = [] 106 | 107 | # Single-layer experiments 108 | for h in single_layer_sizes: 109 | params, avg_loss, avg_hd = train_and_eval([h]) 110 | rows.append(([h], params, avg_loss, avg_hd)) 111 | 112 | # Two-layer experiments 113 | for h1 in two_layer_sizes: 114 | for h2 in two_layer_sizes: 115 | params, avg_loss, avg_hd = train_and_eval([h1, h2]) 116 | rows.append(([h1, h2], params, avg_loss, avg_hd)) 117 | 118 | # Write CSV 119 | with open("results.csv", "w", newline="") as f: 120 | writer = csv.writer(f) 121 | writer.writerow(["hidden_layers", "total_params", "avg_loss", "avg_hausdorff"]) 122 | for hidden, params, loss, hd in rows: 123 | writer.writerow([str(hidden), params, loss, hd]) 124 | 125 | 126 | # Run all experiments: 127 | # run_experiments_and_save_csv() 128 | 129 | import matplotlib.pyplot as plt 130 | import ast 131 | import csv 132 | 133 | single_params = [] 134 | single_hd = [] 135 | two_params = [] 136 | two_hd = [] 137 | 138 | with open("results.csv", "r") as f: 139 | reader = csv.DictReader(f) 140 | for row in reader: 141 | hidden = ast.literal_eval(row["hidden_layers"]) 142 | params = int(row["total_params"]) 143 | hd = float(row["avg_hausdorff"]) 144 | if len(hidden) == 1: 145 | single_params.append((params, hd)) 146 | else: 147 | two_params.append((params, hd)) 148 | 149 | # Sort by total parameters 150 | single_params = sorted(single_params, key=lambda x: x[0]) 151 | two_params = sorted(two_params, key=lambda x: x[0]) 152 | 153 | # Unpack 154 | single_params_x, single_hd_y = zip(*single_params) 155 | two_params_x, two_hd_y = zip(*two_params) 156 | 157 | # Plot 158 | plt.figure(figsize=(8,5)) 159 | plt.plot(single_params_x, single_hd_y, 'o-r', label="1 Layer") 160 | plt.plot(two_params_x, two_hd_y, 'o-b', label="2 Layers") 161 | plt.xlabel("Total Parameters") 162 | plt.ylabel("Average Hausdorff Distance") 163 | plt.title("Hausdorff Distance vs Total Parameters") 164 | plt.legend() 165 | plt.grid(True) 166 | plt.show() 167 | -------------------------------------------------------------------------------- /poly_2d_nn.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torch.optim as optim 4 | import numpy as np 5 | import matplotlib.pyplot as plt 6 | from PIL import Image, ImageDraw 7 | import math 8 | import random 9 | from typing import List, Tuple 10 | 11 | # ----------------------------- 12 | # Utility functions 13 | # ----------------------------- 14 | def clip(value, lower, upper): 15 | return min(upper, max(value, lower)) 16 | 17 | def random_angle_steps(steps: int, irregularity: float) -> List[float]: 18 | angles = [] 19 | lower = (2 * math.pi / steps) - irregularity 20 | upper = (2 * math.pi / steps) + irregularity 21 | cumsum = 0 22 | for i in range(steps): 23 | angle = random.uniform(lower, upper) 24 | angles.append(angle) 25 | cumsum += angle 26 | cumsum /= (2 * math.pi) 27 | for i in range(steps): 28 | angles[i] /= cumsum 29 | return angles 30 | 31 | def generate_polygon(center: Tuple[float, float], avg_radius: float, 32 | irregularity: float, spikiness: float, 33 | num_vertices: int) -> List[Tuple[float, float]]: 34 | irregularity *= 2 * math.pi / num_vertices 35 | spikiness *= avg_radius 36 | angle_steps = random_angle_steps(num_vertices, irregularity) 37 | points = [] 38 | angle = random.uniform(0, 2 * math.pi) 39 | for i in range(num_vertices): 40 | radius = clip(random.gauss(avg_radius, spikiness), 0, 2 * avg_radius) 41 | x = center[0] + radius * math.cos(angle) 42 | y = center[1] + radius * math.sin(angle) 43 | points.append((x, y)) 44 | angle += angle_steps[i] 45 | return points 46 | 47 | def generate_polygon_image(img_size=100, num_vertices=12, avg_radius=40, 48 | irregularity=0.35, spikiness=0.2): 49 | label = random.randint(0,1) 50 | center = (img_size//2, img_size//2) 51 | vertices = generate_polygon(center, avg_radius, irregularity, spikiness, num_vertices) 52 | 53 | bg_color = 255 if label==0 else 0 54 | fg_color = 255 if label==1 else 0 55 | img = Image.new('L', (img_size, img_size), bg_color) 56 | draw = ImageDraw.Draw(img) 57 | draw.polygon(vertices, fill=fg_color) 58 | 59 | return np.array(img, dtype=np.float32)/255.0, label, num_vertices 60 | 61 | def generate_polygon_dataset(n_samples=1000, img_size=40, min_vertices=3, max_vertices=12): 62 | X = np.zeros((n_samples, 1, img_size, img_size), dtype=np.float32) 63 | y_color = np.zeros((n_samples, 1), dtype=np.float32) 64 | y_vertices = np.zeros((n_samples, 1), dtype=np.float32) 65 | 66 | for i in range(n_samples): 67 | num_vertices = random.randint(min_vertices, max_vertices) 68 | img, label, num_vertices = generate_polygon_image(img_size=img_size, num_vertices=num_vertices) 69 | X[i,0] = img 70 | y_color[i,0] = label 71 | y_vertices[i,0] = num_vertices 72 | return torch.tensor(X, dtype=torch.float32), \ 73 | torch.tensor(y_color, dtype=torch.float32), \ 74 | torch.tensor(y_vertices, dtype=torch.float32) 75 | 76 | # ----------------------------- 77 | # CUDA device 78 | # ----------------------------- 79 | device = torch.device("cuda" if torch.cuda.is_available() else "cpu") 80 | print("Using device:", device) 81 | 82 | # ----------------------------- 83 | # Dataset 84 | # ----------------------------- 85 | X_train, y_color, y_vertices = generate_polygon_dataset(n_samples=1000, img_size=100, min_vertices=3, max_vertices=12) 86 | X_train, y_color, y_vertices = X_train.to(device), y_color.to(device), y_vertices.to(device) 87 | 88 | # --------------------------- 89 | # Multi-task CNN 90 | # --------------------------- 91 | class MultiTaskCNN(nn.Module): 92 | def __init__(self, input_channels=1, conv_channels=[16,32,64]): 93 | super().__init__() 94 | layers = [] 95 | in_ch = input_channels 96 | for out_ch in conv_channels: 97 | layers.append(nn.Conv2d(in_ch, out_ch, kernel_size=3, padding=1)) 98 | layers.append(nn.ReLU()) 99 | in_ch = out_ch 100 | self.conv_layers = nn.Sequential(*layers) 101 | self.gap = nn.AdaptiveAvgPool2d((1,1)) 102 | self.fc_color = nn.Linear(in_ch, 1) 103 | self.fc_vertices = nn.Linear(in_ch, 1) 104 | 105 | def forward(self, x): 106 | x = self.conv_layers(x) 107 | x = self.gap(x) 108 | x = x.view(x.size(0), -1) 109 | color_out = torch.sigmoid(self.fc_color(x)) 110 | vertices_out = self.fc_vertices(x) 111 | return color_out, vertices_out 112 | 113 | model = MultiTaskCNN().to(device) 114 | 115 | # --------------------------- 116 | # Losses & optimizer 117 | # --------------------------- 118 | criterion_color = nn.BCELoss() 119 | criterion_vertices = nn.MSELoss() 120 | optimizer = optim.Adam(model.parameters(), lr=0.01) 121 | epochs = 50 122 | 123 | # --------------------------- 124 | # Training 125 | # --------------------------- 126 | for epoch in range(epochs): 127 | model.train() 128 | optimizer.zero_grad() 129 | out_color, out_vertices = model(X_train) 130 | loss_color = criterion_color(out_color, y_color) 131 | loss_vertices = criterion_vertices(out_vertices, y_vertices) 132 | loss = loss_color + loss_vertices 133 | loss.backward() 134 | optimizer.step() 135 | if (epoch+1) % 10 == 0: 136 | print(f"Epoch {epoch+1}, Loss: {loss.item():.3f} (Color: {loss_color.item():.3f}, Vertices: {loss_vertices.item():.3f})") 137 | 138 | # --------------------------- 139 | # Show a few training examples 140 | # --------------------------- 141 | X_vis = X_train[:9].cpu().numpy() 142 | y_color_vis = y_color[:9].cpu().numpy() 143 | y_vertices_vis = y_vertices[:9].cpu().numpy() 144 | 145 | plt.figure(figsize=(6,6)) 146 | for i in range(9): 147 | plt.subplot(3,3,i+1) 148 | plt.imshow(X_vis[i,0], cmap='gray') 149 | plt.title(f"Color={int(y_color_vis[i,0])}\nVertices={int(y_vertices_vis[i,0])}") 150 | plt.axis('off') 151 | plt.show() 152 | 153 | # --------------------------- 154 | # Visualize feature maps 155 | # --------------------------- 156 | def visualize_feature_maps(model, X_sample, conv_layers=[0,2,4]): 157 | x = X_sample 158 | for i, layer in enumerate(model.conv_layers): 159 | x = layer(x) 160 | if i in conv_layers: 161 | n_channels = x.shape[1] 162 | plt.figure(figsize=(15,3)) 163 | for c in range(n_channels): 164 | plt.subplot(1, n_channels, c+1) 165 | fmap = x[0,c].detach().cpu().numpy() 166 | plt.imshow(fmap, cmap='gray') 167 | plt.axis('off') 168 | plt.title(f"Ch {c}") 169 | plt.suptitle(f"Feature maps after layer {i+1}") 170 | plt.show() 171 | 172 | sample_img = X_train[0].unsqueeze(0) 173 | visualize_feature_maps(model, sample_img) 174 | 175 | # --------------------------- 176 | # Visualize conv filters 177 | # --------------------------- 178 | def visualize_conv_filters(model): 179 | for i, layer in enumerate(model.conv_layers): 180 | if isinstance(layer, nn.Conv2d): 181 | n_filters = layer.weight.shape[0] 182 | plt.figure(figsize=(15,3)) 183 | for f in range(n_filters): 184 | filt = layer.weight[f,0].detach().cpu().numpy() 185 | plt.subplot(1, n_filters, f+1) 186 | plt.imshow(filt, cmap='gray') 187 | plt.axis('off') 188 | plt.title(f"F {f}") 189 | plt.suptitle(f"Conv layer {i+1} filters") 190 | plt.show() 191 | 192 | visualize_conv_filters(model) 193 | 194 | # --------------------------- 195 | # Predictions 196 | # --------------------------- 197 | model.eval() 198 | with torch.no_grad(): 199 | out_color, out_vertices = model(X_train[:9]) 200 | out_color = (out_color>0.5).cpu().numpy() 201 | out_vertices = out_vertices.cpu().numpy() 202 | 203 | plt.figure(figsize=(6,6)) 204 | for i in range(9): 205 | plt.subplot(3,3,i+1) 206 | plt.imshow(X_vis[i,0], cmap='gray') 207 | plt.title(f"Color True={int(y_color_vis[i,0])}, Pred={int(out_color[i,0])}\nVertices True={int(y_vertices_vis[i,0])}, Pred={int(out_vertices[i,0])}") 208 | plt.axis('off') 209 | plt.show() 210 | -------------------------------------------------------------------------------- /eye_2d_nn.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torch.optim as optim 4 | import numpy as np 5 | import matplotlib.pyplot as plt 6 | from PIL import Image, ImageDraw 7 | import math 8 | import random 9 | from typing import List, Tuple 10 | 11 | # ----------------------------- 12 | # Utility functions 13 | # ----------------------------- 14 | def clip(value, lower, upper): 15 | return min(upper, max(value, lower)) 16 | 17 | def random_angle_steps(steps: int, irregularity: float) -> List[float]: 18 | angles = [] 19 | lower = (2 * math.pi / steps) - irregularity 20 | upper = (2 * math.pi / steps) + irregularity 21 | cumsum = 0 22 | for i in range(steps): 23 | angle = random.uniform(lower, upper) 24 | angles.append(angle) 25 | cumsum += angle 26 | # normalize so that sum is 2*pi 27 | cumsum /= (2 * math.pi) 28 | for i in range(steps): 29 | angles[i] /= cumsum 30 | return angles 31 | 32 | def generate_polygon(center: Tuple[float, float], avg_radius: float, 33 | irregularity: float, spikiness: float, 34 | num_vertices: int) -> List[Tuple[float, float]]: 35 | irregularity *= 2 * math.pi / num_vertices 36 | spikiness *= avg_radius 37 | angle_steps = random_angle_steps(num_vertices, irregularity) 38 | points = [] 39 | angle = random.uniform(0, 2 * math.pi) 40 | for i in range(num_vertices): 41 | radius = clip(random.gauss(avg_radius, spikiness), 0, 2 * avg_radius) 42 | x = center[0] + radius * math.cos(angle) 43 | y = center[1] + radius * math.sin(angle) 44 | points.append((x, y)) 45 | angle += angle_steps[i] 46 | return points 47 | 48 | # ----------------------------- 49 | # Generate a filled polygon image 50 | # ----------------------------- 51 | def generate_polygon_image(img_size=100, num_vertices=12, avg_radius=40, 52 | irregularity=0.35, spikiness=0.2): 53 | """ 54 | Generate a single image with a filled polygon. 55 | Returns: normalized image (0-1) and label (1=white polygon, 0=black polygon) 56 | """ 57 | label = random.randint(0,1) # 1 = white polygon, 0 = black polygon 58 | center = (img_size // 2, img_size // 2) 59 | vertices = generate_polygon(center, avg_radius, irregularity, spikiness, num_vertices) 60 | 61 | bg_color = 255 if label==0 else 0 # background opposite of polygon 62 | fg_color = 255 if label==1 else 0 # polygon fill 63 | 64 | img = Image.new('L', (img_size, img_size), bg_color) 65 | draw = ImageDraw.Draw(img) 66 | draw.polygon(vertices, fill=fg_color) # filled polygon 67 | 68 | return np.array(img, dtype=np.float32)/255.0, label 69 | 70 | # ----------------------------- 71 | # Generate dataset 72 | # ----------------------------- 73 | def generate_polygon_dataset(n_samples=1000, img_size=40, min_vertices=3, max_vertices=12): 74 | X = np.zeros((n_samples, 1, img_size, img_size), dtype=np.float32) 75 | y = np.zeros((n_samples, 1), dtype=np.float32) 76 | 77 | for i in range(n_samples): 78 | num_vertices = random.randint(min_vertices, max_vertices) 79 | img, label = generate_polygon_image(img_size=img_size, num_vertices=num_vertices) 80 | X[i,0] = img 81 | y[i,0] = label 82 | return torch.tensor(X, dtype=torch.float32), torch.tensor(y, dtype=torch.float32) 83 | 84 | # ----------------------------- 85 | # Example usage & visualization 86 | # ----------------------------- 87 | X_train, y_train = generate_polygon_dataset(n_samples=1000, img_size=100, min_vertices=10, max_vertices=25) 88 | 89 | plt.figure(figsize=(6,6)) 90 | for i in range(9): 91 | plt.subplot(3,3,i+1) 92 | plt.imshow(X_train[i,0], cmap='gray') 93 | plt.title(f"Label={int(y_train[i,0])}") 94 | plt.axis('off') 95 | plt.show() 96 | 97 | # --------------------------- 98 | # 2. Flexible CNN 99 | # --------------------------- 100 | class CNN(nn.Module): 101 | def __init__(self, input_channels=1, conv_channels=[8, 16], kernel_size=11, output_dim=1): 102 | super().__init__() 103 | layers = [] 104 | in_ch = input_channels 105 | for out_ch in conv_channels: 106 | layers.append(nn.Conv2d(in_ch, out_ch, kernel_size=kernel_size, padding=1)) 107 | layers.append(nn.ReLU()) 108 | in_ch = out_ch 109 | 110 | self.conv_layers = nn.Sequential(*layers) 111 | # GAP reduces HxW to 1x1 per channel, output size = number of channels 112 | self.gap = nn.AdaptiveAvgPool2d((1,1)) 113 | self.fc = nn.Sequential( 114 | nn.Flatten(), # flatten Cx1x1 to C 115 | nn.Linear(in_ch, output_dim), 116 | nn.Sigmoid() 117 | ) 118 | 119 | def forward(self, x): 120 | x = self.conv_layers(x) 121 | x = self.gap(x) 122 | x = self.fc(x) 123 | return x 124 | 125 | # Check for GPU 126 | device = torch.device("cuda" if torch.cuda.is_available() else "cpu") 127 | print("Using device:", device) 128 | 129 | # ----------------------------- 130 | # Generate dataset (already in CPU) 131 | # ----------------------------- 132 | X_train, y_train = generate_polygon_dataset(n_samples=1000, img_size=100, min_vertices=10, max_vertices=50) 133 | 134 | # Move dataset to GPU 135 | X_train = X_train.to(device) 136 | y_train = y_train.to(device) 137 | 138 | # --------------------------- 139 | # Define CNN and move to GPU 140 | # --------------------------- 141 | conv_channels = [10, 10, 10] 142 | model = CNN(conv_channels=conv_channels).to(device) 143 | 144 | # --------------------------- 145 | # Training 146 | # --------------------------- 147 | criterion = nn.BCELoss() 148 | optimizer = optim.Adam(model.parameters(), lr=0.01) 149 | epochs = 100 150 | 151 | for epoch in range(epochs): 152 | model.train() 153 | optimizer.zero_grad() 154 | outputs = model(X_train) 155 | loss = criterion(outputs, y_train) 156 | loss.backward() 157 | optimizer.step() 158 | 159 | if (epoch+1) % 10 == 0: 160 | print(f"Epoch {epoch+1}/{epochs}, Loss: {loss.item():.4f}") 161 | 162 | if loss == 0: 163 | break 164 | 165 | # --------------------------- 166 | # Visualize a sample 167 | # --------------------------- 168 | sample_img = X_train[0].unsqueeze(0) # shape (1,1,H,W) 169 | model.eval() 170 | with torch.no_grad(): 171 | _ = model(sample_img) 172 | 173 | # --------------------------- 174 | # Visualize feature maps on GPU 175 | # --------------------------- 176 | def visualize_feature_maps(model, X_sample, conv_layers=[0,2]): 177 | x = X_sample 178 | for i, layer in enumerate(model.conv_layers): 179 | x = layer(x) 180 | if i in conv_layers: 181 | n_channels = x.shape[1] 182 | plt.figure(figsize=(15,3)) 183 | for c in range(n_channels): 184 | plt.subplot(1, n_channels, c+1) 185 | fmap = x[0,c].detach().cpu().numpy() # move to CPU for plotting 186 | fmap = (fmap > 0.5).astype(np.float32) 187 | plt.imshow(fmap, cmap='gray') 188 | plt.axis('off') 189 | plt.title(f"Ch {c}") 190 | plt.suptitle(f"Feature maps after layer {i+1}") 191 | plt.show() 192 | 193 | visualize_feature_maps(model, sample_img, conv_layers=[2*i for i in range(len(conv_channels))]) 194 | 195 | # --------------------------- 196 | # Visualize conv filters 197 | # --------------------------- 198 | def visualize_conv_filters(model, threshold=None): 199 | """ 200 | Visualize the filters (weights) of Conv2d layers. 201 | 202 | Parameters: 203 | - model: the CNN model 204 | - threshold: float or None. If set, only values with abs(weight) >= threshold are shown; others set to 0 205 | """ 206 | for i, layer in enumerate(model.conv_layers): 207 | if isinstance(layer, nn.Conv2d): 208 | n_filters = layer.weight.shape[0] 209 | plt.figure(figsize=(15,3)) 210 | for f in range(n_filters): 211 | filt = layer.weight[f,0].detach().cpu().numpy() 212 | if threshold is not None: 213 | filt = np.where(np.abs(filt) >= threshold, filt, 0.0) 214 | plt.subplot(1, n_filters, f+1) 215 | plt.imshow(filt, cmap='gray', vmin=filt.min(), vmax=filt.max()) 216 | plt.axis('off') 217 | plt.title(f"F {f}") 218 | plt.suptitle(f"Conv layer {i+1} filters") 219 | plt.show() 220 | 221 | # Example usage with thresholding 222 | visualize_conv_filters(model, threshold=0.5) 223 | 224 | # --------------------------- 225 | # Parameter count 226 | # --------------------------- 227 | total_params = sum(p.numel() for p in model.parameters()) 228 | print(f"{conv_channels} CNN has total parameters:", total_params) 229 | print("Final loss:", loss.item()) 230 | -------------------------------------------------------------------------------- /Linear-Algebra/Linear_Algebra_Lesson_1.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [], 7 | "collapsed_sections": [ 8 | "7lV7p_S0tXhT" 9 | ] 10 | }, 11 | "kernelspec": { 12 | "name": "python3", 13 | "display_name": "Python 3" 14 | }, 15 | "language_info": { 16 | "name": "python" 17 | } 18 | }, 19 | "cells": [ 20 | { 21 | "cell_type": "markdown", 22 | "source": [ 23 | "# Vectors" 24 | ], 25 | "metadata": { 26 | "id": "7lV7p_S0tXhT" 27 | } 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "source": [ 32 | "## Vector Representation" 33 | ], 34 | "metadata": { 35 | "id": "xDYJBQhA9frh" 36 | } 37 | }, 38 | { 39 | "cell_type": "code", 40 | "source": [ 41 | "v = [1,2,3,4]\n", 42 | "print(v)\n", 43 | "\n", 44 | "d = len(v)\n", 45 | "print(d)" 46 | ], 47 | "metadata": { 48 | "id": "DxmEnMpgX9GL", 49 | "colab": { 50 | "base_uri": "https://localhost:8080/" 51 | }, 52 | "outputId": "f5f13886-3ba5-4714-c104-5beb00c84a8b" 53 | }, 54 | "execution_count": null, 55 | "outputs": [ 56 | { 57 | "output_type": "stream", 58 | "name": "stdout", 59 | "text": [ 60 | "[1, 2, 3, 4]\n", 61 | "4\n" 62 | ] 63 | } 64 | ] 65 | }, 66 | { 67 | "cell_type": "markdown", 68 | "source": [ 69 | "## Vector Summation" 70 | ], 71 | "metadata": { 72 | "id": "oMr5OWIr9h-5" 73 | } 74 | }, 75 | { 76 | "cell_type": "code", 77 | "source": [ 78 | "v1 = ['ab', 'ac']\n", 79 | "v2 = [2,3,4]\n", 80 | "v = v1+v2\n", 81 | "print(v)" 82 | ], 83 | "metadata": { 84 | "colab": { 85 | "base_uri": "https://localhost:8080/" 86 | }, 87 | "id": "O0zBJKVjtnn4", 88 | "outputId": "fb4dc595-b9cb-4b83-b7a5-cdf7350ccd89" 89 | }, 90 | "execution_count": null, 91 | "outputs": [ 92 | { 93 | "output_type": "stream", 94 | "name": "stdout", 95 | "text": [ 96 | "[1, 2, 3, 2, 3, 4]\n" 97 | ] 98 | } 99 | ] 100 | }, 101 | { 102 | "cell_type": "code", 103 | "source": [ 104 | "v = [1,2,3]\n", 105 | "w = v*3\n", 106 | "print(w)" 107 | ], 108 | "metadata": { 109 | "id": "kr_rj1ZPRlJv", 110 | "outputId": "29d0cff8-0f86-4acf-ae35-168e33582456", 111 | "colab": { 112 | "base_uri": "https://localhost:8080/" 113 | } 114 | }, 115 | "execution_count": null, 116 | "outputs": [ 117 | { 118 | "output_type": "stream", 119 | "name": "stdout", 120 | "text": [ 121 | "[1, 2, 3, 1, 2, 3, 1, 2, 3]\n" 122 | ] 123 | } 124 | ] 125 | }, 126 | { 127 | "cell_type": "code", 128 | "source": [ 129 | "v1 = [1,2,3]\n", 130 | "v2 = [2,3,4]\n", 131 | "\n", 132 | "# v = [None]*len(v1)\n", 133 | "# print(v)\n", 134 | "\n", 135 | "def sum(v1,v2):\n", 136 | " v = [None]*len(v1)\n", 137 | " for i in range(len(v1)):\n", 138 | " v[i] = v1[i]-v2[i]\n", 139 | " return v\n", 140 | "\n", 141 | "print(sum(v1,v2))" 142 | ], 143 | "metadata": { 144 | "colab": { 145 | "base_uri": "https://localhost:8080/" 146 | }, 147 | "id": "bqs7C7OJz3nS", 148 | "outputId": "b48e8e4b-3cad-403c-9a35-f80d6a3b3cfb" 149 | }, 150 | "execution_count": null, 151 | "outputs": [ 152 | { 153 | "output_type": "stream", 154 | "name": "stdout", 155 | "text": [ 156 | "[None, None, None]\n", 157 | "[-1, -1, -1]\n" 158 | ] 159 | } 160 | ] 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "source": [ 165 | "## Vector Scaling" 166 | ], 167 | "metadata": { 168 | "id": "zHzFZDbe-JNh" 169 | } 170 | }, 171 | { 172 | "cell_type": "code", 173 | "source": [ 174 | "def scale(v,a):\n", 175 | " w = [None]*len(v)\n", 176 | " for i in range(len(v)):\n", 177 | " w[i] = a*v[i]\n", 178 | " return v\n", 179 | "\n", 180 | "v = [1,4,5]\n", 181 | "a = 5\n", 182 | "\n", 183 | "print(scale(v,a))" 184 | ], 185 | "metadata": { 186 | "colab": { 187 | "base_uri": "https://localhost:8080/" 188 | }, 189 | "id": "6R2Du5y301bn", 190 | "outputId": "bf8c124d-2fdf-41f1-c5e2-20f31f4f2c5c" 191 | }, 192 | "execution_count": null, 193 | "outputs": [ 194 | { 195 | "output_type": "stream", 196 | "name": "stdout", 197 | "text": [ 198 | "[5, 20, 25]\n" 199 | ] 200 | } 201 | ] 202 | }, 203 | { 204 | "cell_type": "markdown", 205 | "source": [ 206 | "## Vector Linear Combination" 207 | ], 208 | "metadata": { 209 | "id": "XnyvRpo_-MBX" 210 | } 211 | }, 212 | { 213 | "cell_type": "code", 214 | "source": [ 215 | "def lin_com(v1,v2,a1,a2):\n", 216 | " v = [None]*len(v1)\n", 217 | " for i in range(len(v1)):\n", 218 | " v[i] = a1*v1[i]+a2*v2[i]\n", 219 | " return v\n", 220 | "\n", 221 | "v1 = [1,2,3]\n", 222 | "v2 = [2,3,4]\n", 223 | "a1 = 1\n", 224 | "a2 = -3\n", 225 | "\n", 226 | "print(lin_com(v1,v2,a1,a2))" 227 | ], 228 | "metadata": { 229 | "colab": { 230 | "base_uri": "https://localhost:8080/" 231 | }, 232 | "id": "i6mQdoiV14Lf", 233 | "outputId": "cdbeee6f-1af1-4bbe-b935-f67c68086700" 234 | }, 235 | "execution_count": null, 236 | "outputs": [ 237 | { 238 | "output_type": "stream", 239 | "name": "stdout", 240 | "text": [ 241 | "[-5, -7, -9]\n" 242 | ] 243 | } 244 | ] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "source": [ 249 | "## Dot Product of Two Vectors" 250 | ], 251 | "metadata": { 252 | "id": "lXC9tINM2uie" 253 | } 254 | }, 255 | { 256 | "cell_type": "code", 257 | "source": [ 258 | "v1 = [1,2,3]\n", 259 | "v2 = [2,3,4]\n", 260 | "\n", 261 | "0\n", 262 | "0 + 1*2\n", 263 | "0 + 1*2 + 2*3\n", 264 | "0 + 1*2 + 2*3 + 3*4\n", 265 | "\n", 266 | "def dot_prod(v1,v2):\n", 267 | "\n", 268 | " if len(v1) != len(v2):\n", 269 | " return -1\n", 270 | "\n", 271 | " dot = 0\n", 272 | " for i in range(len(v1)):\n", 273 | " dot = dot + v1[i]*v2[i]\n", 274 | " return dot\n", 275 | "\n", 276 | "print(dot_prod(v1,v2))" 277 | ], 278 | "metadata": { 279 | "colab": { 280 | "base_uri": "https://localhost:8080/" 281 | }, 282 | "id": "4I_I6_WM2dM4", 283 | "outputId": "e69772d6-4c0a-4289-8bcc-90e9c940aa76" 284 | }, 285 | "execution_count": null, 286 | "outputs": [ 287 | { 288 | "output_type": "stream", 289 | "name": "stdout", 290 | "text": [ 291 | "20\n" 292 | ] 293 | } 294 | ] 295 | }, 296 | { 297 | "cell_type": "markdown", 298 | "source": [ 299 | "## Norm of a Vector" 300 | ], 301 | "metadata": { 302 | "id": "WDXhAQ9C9ulf" 303 | } 304 | }, 305 | { 306 | "cell_type": "code", 307 | "source": [ 308 | "v = [1,2,3]\n", 309 | "\n", 310 | "0\n", 311 | "0 + 1*1\n", 312 | "0 + 1*1 + 2*2\n", 313 | "0 + 1*1 + 2*2 + 3*3\n", 314 | "\n", 315 | "def norm(v):\n", 316 | " n = 0\n", 317 | " for i in range(len(v)):\n", 318 | " n = n + v[i]**2\n", 319 | " n = n**0.5\n", 320 | " return n\n", 321 | "\n", 322 | "print(norm(v))" 323 | ], 324 | "metadata": { 325 | "colab": { 326 | "base_uri": "https://localhost:8080/" 327 | }, 328 | "id": "yz3Shzal3-fp", 329 | "outputId": "9083636b-b76b-4c57-9f7c-fdecee256697" 330 | }, 331 | "execution_count": null, 332 | "outputs": [ 333 | { 334 | "output_type": "stream", 335 | "name": "stdout", 336 | "text": [ 337 | "3.7416573867739413\n" 338 | ] 339 | } 340 | ] 341 | }, 342 | { 343 | "cell_type": "code", 344 | "source": [ 345 | "def norm1(v):\n", 346 | " n = dot_prod(v,v)\n", 347 | " n = n**0.5\n", 348 | " return n\n", 349 | "\n", 350 | "print(norm1(v))" 351 | ], 352 | "metadata": { 353 | "colab": { 354 | "base_uri": "https://localhost:8080/" 355 | }, 356 | "id": "EU4JrTED4qeQ", 357 | "outputId": "e52f31c0-641e-447c-fb9c-72d76d747e0d" 358 | }, 359 | "execution_count": null, 360 | "outputs": [ 361 | { 362 | "output_type": "stream", 363 | "name": "stdout", 364 | "text": [ 365 | "3.7416573867739413\n" 366 | ] 367 | } 368 | ] 369 | }, 370 | { 371 | "cell_type": "markdown", 372 | "source": [ 373 | "# Matrices" 374 | ], 375 | "metadata": { 376 | "id": "zfbjkTNl5gnz" 377 | } 378 | }, 379 | { 380 | "cell_type": "markdown", 381 | "source": [ 382 | "## Matrix Representation" 383 | ], 384 | "metadata": { 385 | "id": "yNytAcKJ91eA" 386 | } 387 | }, 388 | { 389 | "cell_type": "code", 390 | "source": [ 391 | "m = [[1,2,4], [2,3,5], [4,5,6], [7,8,9]]\n", 392 | "print(m[1]) # 2nd row vector ith row\n", 393 | "print(m[1][2])\n", 394 | "#m[i][j] is the element in ith row and jth column" 395 | ], 396 | "metadata": { 397 | "colab": { 398 | "base_uri": "https://localhost:8080/" 399 | }, 400 | "id": "FLtoDOzB8BBo", 401 | "outputId": "b70db502-04a2-4acd-92fe-0f079d8e5f2c" 402 | }, 403 | "execution_count": null, 404 | "outputs": [ 405 | { 406 | "output_type": "stream", 407 | "name": "stdout", 408 | "text": [ 409 | "[2, 3, 5]\n", 410 | "5\n" 411 | ] 412 | } 413 | ] 414 | }, 415 | { 416 | "cell_type": "code", 417 | "source": [ 418 | "m = [[1,2,4], [2,3,5], [4,5,6], [7,8,9]]\n", 419 | "n_rows = len(m)\n", 420 | "n_cols = len(m[0])\n", 421 | "print(n_rows)\n", 422 | "print(n_cols)" 423 | ], 424 | "metadata": { 425 | "colab": { 426 | "base_uri": "https://localhost:8080/" 427 | }, 428 | "id": "NE6WPqqP4_Y7", 429 | "outputId": "72fc7986-62f7-4a3c-c8fe-f068ca47632b" 430 | }, 431 | "execution_count": null, 432 | "outputs": [ 433 | { 434 | "output_type": "stream", 435 | "name": "stdout", 436 | "text": [ 437 | "4\n", 438 | "3\n" 439 | ] 440 | } 441 | ] 442 | }, 443 | { 444 | "cell_type": "markdown", 445 | "source": [ 446 | "## Matrix Summation" 447 | ], 448 | "metadata": { 449 | "id": "RRBQ-Aa994K0" 450 | } 451 | }, 452 | { 453 | "cell_type": "code", 454 | "source": [ 455 | "m1 = [[1,2,4], [2,3,5], [4,5,6], [7,8,9]]\n", 456 | "m2 = [[1,2,5], [2,7,5], [9,5,6], [7,0,9]]\n", 457 | "\n", 458 | "def add(m1,m2):\n", 459 | " m = [[0,0,0], [0,0,0], [0,0,0], [0,0,0]]\n", 460 | " for i in range(len(m1)): # running over rows\n", 461 | " for j in range(len(m1[0])): # running over columns\n", 462 | " m[i][j] = m1[i][j] + m2[i][j]\n", 463 | " return m\n", 464 | "\n", 465 | "print(add(m1,m2))" 466 | ], 467 | "metadata": { 468 | "colab": { 469 | "base_uri": "https://localhost:8080/" 470 | }, 471 | "id": "-WheUtB168Xk", 472 | "outputId": "7f557483-fadd-463f-acf3-e7fbd3f55741" 473 | }, 474 | "execution_count": null, 475 | "outputs": [ 476 | { 477 | "output_type": "stream", 478 | "name": "stdout", 479 | "text": [ 480 | "[[2, 4, 9], [4, 10, 10], [13, 10, 12], [14, 8, 18]]\n" 481 | ] 482 | } 483 | ] 484 | }, 485 | { 486 | "cell_type": "markdown", 487 | "source": [ 488 | "## Next Day\n", 489 | "\n", 490 | "* how can you change the row view to column view?" 491 | ], 492 | "metadata": { 493 | "id": "Ug4bwKNS8l6m" 494 | } 495 | }, 496 | { 497 | "cell_type": "code", 498 | "source": [ 499 | "m_row = [[1,2,4], [2,3,5], [4,5,6], [7,8,9]]\n", 500 | "m_col = [[1,2,4,7], [2,3,5,8], [4,5,6,9]]\n", 501 | "\n", 502 | "# define a fun. that changes m_row to m_col\n", 503 | "# m_t[i][j] = m[j][i]\n", 504 | "\n", 505 | "# why this is important?\n", 506 | "# matrix multiply m = m1*m2\n", 507 | "# m[i][j] = dot product of ith row of m1 and jth column of m2\n", 508 | "# we need to access rows of m1\n", 509 | "# we need to access columns of m2\n", 510 | "# then do dot product" 511 | ], 512 | "metadata": { 513 | "id": "1EB_JIOH8x6D" 514 | }, 515 | "execution_count": null, 516 | "outputs": [] 517 | }, 518 | { 519 | "cell_type": "code", 520 | "source": [ 521 | "# transpose\n", 522 | "\n", 523 | "def transp(m):\n" 524 | ], 525 | "metadata": { 526 | "colab": { 527 | "base_uri": "https://localhost:8080/" 528 | }, 529 | "id": "NFZCgKYiP-Fe", 530 | "outputId": "232bf64b-93e1-44b7-f4b4-81b1e67f6b8d" 531 | }, 532 | "execution_count": null, 533 | "outputs": [ 534 | { 535 | "output_type": "stream", 536 | "name": "stdout", 537 | "text": [ 538 | "[[1, 2, 4], [2, 3, 5], [4, 5, 6], [7, 8, 9]]\n" 539 | ] 540 | } 541 | ] 542 | }, 543 | { 544 | "cell_type": "code", 545 | "source": [ 546 | "def transpose_matrix(matrix):\n", 547 | " \"\"\"Returns the transpose of a given matrix.\"\"\"\n", 548 | " rows = len(matrix)\n", 549 | " cols = len(matrix[0]) if rows > 0 else 0\n", 550 | " return [[matrix[j][i] for j in range(rows)] for i in range(cols)]\n", 551 | "\n", 552 | "# Example usage:\n", 553 | "matrix = [\n", 554 | " [1, 2, 3],\n", 555 | " [4, 5, 6]\n", 556 | "]\n", 557 | "\n", 558 | "transposed = transpose_matrix(matrix)\n", 559 | "print(transposed)" 560 | ], 561 | "metadata": { 562 | "colab": { 563 | "base_uri": "https://localhost:8080/" 564 | }, 565 | "id": "qksOdWWpQF4K", 566 | "outputId": "0e94d045-c4e6-43c0-a8db-a40fc817f633" 567 | }, 568 | "execution_count": null, 569 | "outputs": [ 570 | { 571 | "output_type": "stream", 572 | "name": "stdout", 573 | "text": [ 574 | "[[1, 4], [2, 5], [3, 6]]\n" 575 | ] 576 | } 577 | ] 578 | } 579 | ] 580 | } 581 | -------------------------------------------------------------------------------- /Linear-Algebra/Linear_Algebra_Lesson_2.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [], 7 | "collapsed_sections": [ 8 | "hYfo0KKwtemG", 9 | "F9wqkfMFthF-", 10 | "SDqHrvWotjOc", 11 | "VJKVA3L1tn4H", 12 | "fO4sgfhqtqNj", 13 | "mSLz_ttets3q", 14 | "vbkoWBf_-oKr" 15 | ] 16 | }, 17 | "kernelspec": { 18 | "name": "python3", 19 | "display_name": "Python 3" 20 | }, 21 | "language_info": { 22 | "name": "python" 23 | } 24 | }, 25 | "cells": [ 26 | { 27 | "cell_type": "markdown", 28 | "source": [ 29 | "## List Comprehension" 30 | ], 31 | "metadata": { 32 | "id": "hYfo0KKwtemG" 33 | } 34 | }, 35 | { 36 | "cell_type": "code", 37 | "source": [ 38 | "score_list = [72,88,95,60,83]\n", 39 | "max_score = max(score_list)" 40 | ], 41 | "metadata": { 42 | "id": "vcMuat4Htg4c" 43 | }, 44 | "execution_count": 81, 45 | "outputs": [] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "source": [ 50 | "norm_score = [sc / max_score for sc in score_list]" 51 | ], 52 | "metadata": { 53 | "id": "9CsQEp0n_A4O", 54 | "collapsed": true 55 | }, 56 | "execution_count": 84, 57 | "outputs": [] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "source": [ 62 | "## Generator Expressions" 63 | ], 64 | "metadata": { 65 | "id": "F9wqkfMFthF-" 66 | } 67 | }, 68 | { 69 | "cell_type": "code", 70 | "source": [ 71 | "arr = [1,2,3]\n", 72 | "sq = (a**2 for a in arr)\n", 73 | "print(sq)\n", 74 | "lsq = [a**2 for a in arr]\n", 75 | "print(lsq)\n", 76 | "s1 = min(sq)\n", 77 | "s2 = min(lsq)\n", 78 | "print(s1)" 79 | ], 80 | "metadata": { 81 | "id": "ScyATqOvti-k", 82 | "colab": { 83 | "base_uri": "https://localhost:8080/" 84 | }, 85 | "outputId": "1d013bba-5aa3-4c62-a2bf-8585024a920b" 86 | }, 87 | "execution_count": 21, 88 | "outputs": [ 89 | { 90 | "output_type": "stream", 91 | "name": "stdout", 92 | "text": [ 93 | " at 0x7ce65a01f510>\n", 94 | "[1, 4, 9]\n", 95 | "1\n" 96 | ] 97 | } 98 | ] 99 | }, 100 | { 101 | "cell_type": "markdown", 102 | "source": [ 103 | "## Zipping" 104 | ], 105 | "metadata": { 106 | "id": "SDqHrvWotjOc" 107 | } 108 | }, 109 | { 110 | "cell_type": "code", 111 | "source": [ 112 | "v1 = [1,2,3]\n", 113 | "v2 = [3,4,5]\n", 114 | "v3 = [6,7,5]\n", 115 | "\n", 116 | "print(list(zip(v1,v2,v3)))" 117 | ], 118 | "metadata": { 119 | "id": "H-S0pIuLtnpf", 120 | "colab": { 121 | "base_uri": "https://localhost:8080/" 122 | }, 123 | "outputId": "76603d29-82f4-4988-9f37-14fcb062b70f" 124 | }, 125 | "execution_count": 26, 126 | "outputs": [ 127 | { 128 | "output_type": "stream", 129 | "name": "stdout", 130 | "text": [ 131 | "[(1, 3, 6), (2, 4, 7), (3, 5, 5)]\n" 132 | ] 133 | } 134 | ] 135 | }, 136 | { 137 | "cell_type": "markdown", 138 | "source": [ 139 | "## Argument Unpacking" 140 | ], 141 | "metadata": { 142 | "id": "VJKVA3L1tn4H" 143 | } 144 | }, 145 | { 146 | "cell_type": "code", 147 | "source": [ 148 | "lst = [1,2,3]\n", 149 | "print(*lst)" 150 | ], 151 | "metadata": { 152 | "id": "o1cTzNJWtp7F", 153 | "colab": { 154 | "base_uri": "https://localhost:8080/" 155 | }, 156 | "outputId": "f7b62d00-ccd1-46a3-dd50-a36361e92273" 157 | }, 158 | "execution_count": 27, 159 | "outputs": [ 160 | { 161 | "output_type": "stream", 162 | "name": "stdout", 163 | "text": [ 164 | "1 2 3\n" 165 | ] 166 | } 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "source": [ 172 | "m = [[1,2,3], [3,4,5]]\n", 173 | "print(*m)" 174 | ], 175 | "metadata": { 176 | "colab": { 177 | "base_uri": "https://localhost:8080/" 178 | }, 179 | "id": "Hi0JumCUCToz", 180 | "outputId": "4d9aaad8-c757-4c05-cb8a-618238676317" 181 | }, 182 | "execution_count": 28, 183 | "outputs": [ 184 | { 185 | "output_type": "stream", 186 | "name": "stdout", 187 | "text": [ 188 | "[1, 2, 3] [3, 4, 5]\n" 189 | ] 190 | } 191 | ] 192 | }, 193 | { 194 | "cell_type": "markdown", 195 | "source": [ 196 | "## Nested List Comprehension\n", 197 | "\n", 198 | "* Pattern: [ [ inner-loop ] for outer-loop ]" 199 | ], 200 | "metadata": { 201 | "id": "fO4sgfhqtqNj" 202 | } 203 | }, 204 | { 205 | "cell_type": "code", 206 | "source": [ 207 | "m = [[1,2,3], [3,4,5]]" 208 | ], 209 | "metadata": { 210 | "id": "lnLMp5W7tsnv" 211 | }, 212 | "execution_count": null, 213 | "outputs": [] 214 | }, 215 | { 216 | "cell_type": "code", 217 | "source": [ 218 | "m[0][2]" 219 | ], 220 | "metadata": { 221 | "colab": { 222 | "base_uri": "https://localhost:8080/" 223 | }, 224 | "id": "7nZWXra-Dsnh", 225 | "outputId": "e264a16a-679a-4878-cddb-26aabc8438a9" 226 | }, 227 | "execution_count": 32, 228 | "outputs": [ 229 | { 230 | "output_type": "execute_result", 231 | "data": { 232 | "text/plain": [ 233 | "3" 234 | ] 235 | }, 236 | "metadata": {}, 237 | "execution_count": 32 238 | } 239 | ] 240 | }, 241 | { 242 | "cell_type": "code", 243 | "source": [ 244 | "identity = []\n", 245 | "for i in range(3):\n", 246 | " row = []\n", 247 | " for j in range(3):\n", 248 | " row.append(1 if i == j else 0)\n", 249 | " identity.append(row)\n", 250 | "\n", 251 | "identity = [[1 if i == j else 0 for j in range(3)] for i in range(3)]" 252 | ], 253 | "metadata": { 254 | "id": "Azm329SJDEZn" 255 | }, 256 | "execution_count": 33, 257 | "outputs": [] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "source": [ 262 | "[[1 if i == j else 0 for j in range(3)] for i in range(3)]" 263 | ], 264 | "metadata": { 265 | "id": "UNfL6kNGDOce" 266 | }, 267 | "execution_count": null, 268 | "outputs": [] 269 | }, 270 | { 271 | "cell_type": "markdown", 272 | "source": [ 273 | "## Lambda Function\n", 274 | "\n", 275 | "f: R -> R" 276 | ], 277 | "metadata": { 278 | "id": "mSLz_ttets3q" 279 | } 280 | }, 281 | { 282 | "cell_type": "code", 283 | "source": [ 284 | "def f(x):\n", 285 | " x = x+1\n", 286 | " return x\n", 287 | "\n", 288 | "lambda x: x+1" 289 | ], 290 | "metadata": { 291 | "id": "Egi0RKkttufj", 292 | "colab": { 293 | "base_uri": "https://localhost:8080/" 294 | }, 295 | "outputId": "b8f929ce-5b55-4b67-85a8-b7e0da671d65" 296 | }, 297 | "execution_count": 34, 298 | "outputs": [ 299 | { 300 | "output_type": "execute_result", 301 | "data": { 302 | "text/plain": [ 303 | "(x)>" 304 | ] 305 | }, 306 | "metadata": {}, 307 | "execution_count": 34 308 | } 309 | ] 310 | }, 311 | { 312 | "cell_type": "code", 313 | "source": [ 314 | "(lambda x: x+1)(2)" 315 | ], 316 | "metadata": { 317 | "colab": { 318 | "base_uri": "https://localhost:8080/" 319 | }, 320 | "id": "f-3J2_5xE463", 321 | "outputId": "7296338e-574e-401f-8b51-2db920b6dcb6" 322 | }, 323 | "execution_count": 36, 324 | "outputs": [ 325 | { 326 | "output_type": "execute_result", 327 | "data": { 328 | "text/plain": [ 329 | "3" 330 | ] 331 | }, 332 | "metadata": {}, 333 | "execution_count": 36 334 | } 335 | ] 336 | }, 337 | { 338 | "cell_type": "code", 339 | "source": [ 340 | "nums = [9,15,11,13]\n", 341 | "m = min(nums, key = lambda x:abs(x-10))\n", 342 | "sort = sorted(nums, key = lambda x:abs(x-10))\n", 343 | "print(sort)" 344 | ], 345 | "metadata": { 346 | "colab": { 347 | "base_uri": "https://localhost:8080/" 348 | }, 349 | "id": "T4rrwREXE-PA", 350 | "outputId": "d795006b-cba7-43fa-f3f5-c3dfbd229119" 351 | }, 352 | "execution_count": 39, 353 | "outputs": [ 354 | { 355 | "output_type": "stream", 356 | "name": "stdout", 357 | "text": [ 358 | "[9, 11, 13, 15]\n" 359 | ] 360 | } 361 | ] 362 | }, 363 | { 364 | "cell_type": "markdown", 365 | "source": [ 366 | "# Vectors" 367 | ], 368 | "metadata": { 369 | "id": "7lV7p_S0tXhT" 370 | } 371 | }, 372 | { 373 | "cell_type": "markdown", 374 | "source": [ 375 | "## Cosine Similarity Between Two Vectors\n", 376 | "* list comprehension, zipping, generator expressions" 377 | ], 378 | "metadata": { 379 | "id": "vbkoWBf_-oKr" 380 | } 381 | }, 382 | { 383 | "cell_type": "code", 384 | "source": [ 385 | "v1 = [1,2,3]\n", 386 | "v2 = [2,3,4]\n", 387 | "\n", 388 | "def cos_sim(v1,v2):\n", 389 | " norm1 = sum(a**2 for a in v1)**0.5\n", 390 | " norm2 = sum(b**2 for b in v2)**0.5\n", 391 | " return sum(x*y for x,y in zip(v1,v2))/(norm1*norm2)\n", 392 | "\n", 393 | "print(cos_sim(v1,v2))" 394 | ], 395 | "metadata": { 396 | "id": "hmpIBVFG-rRV", 397 | "colab": { 398 | "base_uri": "https://localhost:8080/" 399 | }, 400 | "outputId": "b358d021-4bfd-487e-c62b-e55251cbcc25" 401 | }, 402 | "execution_count": 41, 403 | "outputs": [ 404 | { 405 | "output_type": "stream", 406 | "name": "stdout", 407 | "text": [ 408 | "0.9925833339709303\n" 409 | ] 410 | } 411 | ] 412 | }, 413 | { 414 | "cell_type": "markdown", 415 | "source": [ 416 | "# Matrices" 417 | ], 418 | "metadata": { 419 | "id": "zfbjkTNl5gnz" 420 | } 421 | }, 422 | { 423 | "cell_type": "markdown", 424 | "source": [ 425 | "## Matrix Addition\n", 426 | "\n", 427 | "* nested list comprehension, zipping" 428 | ], 429 | "metadata": { 430 | "id": "SftjMmGbqVtQ" 431 | } 432 | }, 433 | { 434 | "cell_type": "code", 435 | "source": [ 436 | "m1 = [[1,2,4], [2,3,5], [4,5,6], [7,8,9]]\n", 437 | "m2 = [[1,2,5], [2,7,5], [9,5,6], [7,0,9]]\n", 438 | "m3 = [[1,2,5], [2,7,5], [9,5,6], [7,0,9]]" 439 | ], 440 | "metadata": { 441 | "id": "ZcU4ENImqYUM" 442 | }, 443 | "execution_count": 73, 444 | "outputs": [] 445 | }, 446 | { 447 | "cell_type": "code", 448 | "source": [ 449 | "list(zip(m1,m2)) #accessing the pairwise rows of m1, m2" 450 | ], 451 | "metadata": { 452 | "colab": { 453 | "base_uri": "https://localhost:8080/" 454 | }, 455 | "id": "rY2nj11-JO-r", 456 | "outputId": "7d404eb2-6224-4f62-8986-b3580119d530" 457 | }, 458 | "execution_count": 51, 459 | "outputs": [ 460 | { 461 | "output_type": "execute_result", 462 | "data": { 463 | "text/plain": [ 464 | "[([1, 2, 4], [1, 2, 5]),\n", 465 | " ([2, 3, 5], [2, 7, 5]),\n", 466 | " ([4, 5, 6], [9, 5, 6]),\n", 467 | " ([7, 8, 9], [7, 0, 9])]" 468 | ] 469 | }, 470 | "metadata": {}, 471 | "execution_count": 51 472 | } 473 | ] 474 | }, 475 | { 476 | "cell_type": "code", 477 | "source": [ 478 | "list(zip(m1,m2))[0] #first" 479 | ], 480 | "metadata": { 481 | "colab": { 482 | "base_uri": "https://localhost:8080/" 483 | }, 484 | "id": "BsWU11ZbJacU", 485 | "outputId": "47565a76-2a14-4f3f-e5ac-20f596a55c86" 486 | }, 487 | "execution_count": 50, 488 | "outputs": [ 489 | { 490 | "output_type": "execute_result", 491 | "data": { 492 | "text/plain": [ 493 | "([1, 2, 4], [1, 2, 5])" 494 | ] 495 | }, 496 | "metadata": {}, 497 | "execution_count": 50 498 | } 499 | ] 500 | }, 501 | { 502 | "cell_type": "code", 503 | "source": [ 504 | "[a+b for a,b in zip([1, 2, 4], [1, 2, 5])] #[1, 2, 4], [1, 2, 5] #accessing row elements" 505 | ], 506 | "metadata": { 507 | "colab": { 508 | "base_uri": "https://localhost:8080/" 509 | }, 510 | "id": "OdVYHIh9Jh3F", 511 | "outputId": "c656b857-71e6-408b-d412-d744c9c5aa6a" 512 | }, 513 | "execution_count": 54, 514 | "outputs": [ 515 | { 516 | "output_type": "execute_result", 517 | "data": { 518 | "text/plain": [ 519 | "[2, 4, 9]" 520 | ] 521 | }, 522 | "metadata": {}, 523 | "execution_count": 54 524 | } 525 | ] 526 | }, 527 | { 528 | "cell_type": "code", 529 | "source": [ 530 | "# zip(A,B) = pairs of rows in outer loop\n", 531 | "# zip(rowA, rowB) = pairs of elements in inner loop\n", 532 | "\n", 533 | "def add(A,B):\n", 534 | " return [[a+b for a,b in zip(rowA, robB)] for rowA, robB in zip(A,B)]\n", 535 | "\n", 536 | "print(add(m1,m2))" 537 | ], 538 | "metadata": { 539 | "colab": { 540 | "base_uri": "https://localhost:8080/" 541 | }, 542 | "id": "Hvc7XeTeHZHX", 543 | "outputId": "ba97b003-654c-45b9-efd5-0141f0852f02" 544 | }, 545 | "execution_count": 48, 546 | "outputs": [ 547 | { 548 | "output_type": "stream", 549 | "name": "stdout", 550 | "text": [ 551 | "[[2, 4, 9], [4, 10, 10], [13, 10, 12], [14, 8, 18]]\n" 552 | ] 553 | } 554 | ] 555 | }, 556 | { 557 | "cell_type": "code", 558 | "source": [ 559 | "#multiple matrix addition\n", 560 | "\n", 561 | "def add2(*matrices):\n", 562 | " return [[sum(values) for values in zip(*rows)] for rows in zip(*matrices)]" 563 | ], 564 | "metadata": { 565 | "id": "TlIfpywiPP-A" 566 | }, 567 | "execution_count": 76, 568 | "outputs": [] 569 | }, 570 | { 571 | "cell_type": "code", 572 | "source": [ 573 | "m4 = m1\n", 574 | "add2(m1,m2,m3,m4)" 575 | ], 576 | "metadata": { 577 | "colab": { 578 | "base_uri": "https://localhost:8080/" 579 | }, 580 | "id": "vSBE8OY2QMpC", 581 | "outputId": "b3e675f8-96a0-4ed2-fd95-c5f7e888c1cd" 582 | }, 583 | "execution_count": 78, 584 | "outputs": [ 585 | { 586 | "output_type": "execute_result", 587 | "data": { 588 | "text/plain": [ 589 | "[[4, 8, 18], [8, 20, 20], [26, 20, 24], [28, 16, 36]]" 590 | ] 591 | }, 592 | "metadata": {}, 593 | "execution_count": 78 594 | } 595 | ] 596 | }, 597 | { 598 | "cell_type": "markdown", 599 | "source": [ 600 | "## Matrix Scaling" 601 | ], 602 | "metadata": { 603 | "id": "bAj6qUzq-hED" 604 | } 605 | }, 606 | { 607 | "cell_type": "code", 608 | "source": [], 609 | "metadata": { 610 | "id": "MxraPdOD-kag", 611 | "colab": { 612 | "base_uri": "https://localhost:8080/" 613 | }, 614 | "outputId": "65697638-7d6e-4362-957f-33cf895f24f2" 615 | }, 616 | "execution_count": 56, 617 | "outputs": [ 618 | { 619 | "output_type": "execute_result", 620 | "data": { 621 | "text/plain": [ 622 | "[(1, 2, 4, 7), (2, 3, 5, 8), (4, 5, 6, 9)]" 623 | ] 624 | }, 625 | "metadata": {}, 626 | "execution_count": 56 627 | } 628 | ] 629 | }, 630 | { 631 | "cell_type": "markdown", 632 | "source": [ 633 | "## Tranpose of a Matrix\n", 634 | "\n", 635 | "* Argument Unpacking, Zipping" 636 | ], 637 | "metadata": { 638 | "id": "md1Zdyls-Tkz" 639 | } 640 | }, 641 | { 642 | "cell_type": "code", 643 | "source": [ 644 | "matrix = [[1,2,4], [2,3,5], [4,5,6], [7,8,9]]\n", 645 | "\n", 646 | "list(zip([1,2,4], [2,3,5], [4,5,6], [7,8,9])) # we need the rows to be arguments of zip\n", 647 | "\n", 648 | "print(*matrix) # argument unpacking transforms matrix into the rows argument" 649 | ], 650 | "metadata": { 651 | "id": "B7YMXJd2-YfH", 652 | "colab": { 653 | "base_uri": "https://localhost:8080/" 654 | }, 655 | "outputId": "d5987c6c-bbc4-46f0-93e5-1047bd45a75d" 656 | }, 657 | "execution_count": 59, 658 | "outputs": [ 659 | { 660 | "output_type": "stream", 661 | "name": "stdout", 662 | "text": [ 663 | "[1, 2, 4] [2, 3, 5] [4, 5, 6] [7, 8, 9]\n" 664 | ] 665 | } 666 | ] 667 | }, 668 | { 669 | "cell_type": "code", 670 | "source": [ 671 | "list(zip(*matrix))" 672 | ], 673 | "metadata": { 674 | "colab": { 675 | "base_uri": "https://localhost:8080/" 676 | }, 677 | "id": "A-ce5a6aLfrd", 678 | "outputId": "af329096-01c2-4998-9f7b-bf88fe11c5cf" 679 | }, 680 | "execution_count": 61, 681 | "outputs": [ 682 | { 683 | "output_type": "execute_result", 684 | "data": { 685 | "text/plain": [ 686 | "[(1, 2, 4, 7), (2, 3, 5, 8), (4, 5, 6, 9)]" 687 | ] 688 | }, 689 | "metadata": {}, 690 | "execution_count": 61 691 | } 692 | ] 693 | }, 694 | { 695 | "cell_type": "code", 696 | "source": [ 697 | "transpose = [list(col) for col in zip(*matrix)]\n", 698 | "transpose" 699 | ], 700 | "metadata": { 701 | "colab": { 702 | "base_uri": "https://localhost:8080/" 703 | }, 704 | "id": "uB4tqGwtLLsi", 705 | "outputId": "f62383f8-246d-4a30-d38e-72f2a5571351" 706 | }, 707 | "execution_count": 63, 708 | "outputs": [ 709 | { 710 | "output_type": "execute_result", 711 | "data": { 712 | "text/plain": [ 713 | "[[1, 2, 4, 7], [2, 3, 5, 8], [4, 5, 6, 9]]" 714 | ] 715 | }, 716 | "metadata": {}, 717 | "execution_count": 63 718 | } 719 | ] 720 | }, 721 | { 722 | "cell_type": "markdown", 723 | "source": [ 724 | "## Matrix Vector Dot Product\n", 725 | "\n", 726 | "* zipping, list comprehension, generator expression" 727 | ], 728 | "metadata": { 729 | "id": "cM9jMSg4-eFq" 730 | } 731 | }, 732 | { 733 | "cell_type": "code", 734 | "source": [ 735 | "A = [\n", 736 | " [1, 2, 3],\n", 737 | " [4, 5, 6]\n", 738 | "]\n", 739 | "x = [1, 0, -1]\n", 740 | "\n", 741 | "#[1*1 + 2*0 + 3*(-1), 4*1 + 5*0 + 6*(-1)] = [-2, -2]" 742 | ], 743 | "metadata": { 744 | "id": "2GhuEaCJ-gZr" 745 | }, 746 | "execution_count": 64, 747 | "outputs": [] 748 | }, 749 | { 750 | "cell_type": "code", 751 | "source": [ 752 | "[sum(a*b for a,b in zip(row,x)) for row in A]" 753 | ], 754 | "metadata": { 755 | "colab": { 756 | "base_uri": "https://localhost:8080/" 757 | }, 758 | "id": "TPLBux5mLxke", 759 | "outputId": "afb742ea-3112-4e4d-af1c-2352d030b73b" 760 | }, 761 | "execution_count": 65, 762 | "outputs": [ 763 | { 764 | "output_type": "execute_result", 765 | "data": { 766 | "text/plain": [ 767 | "[-2, -2]" 768 | ] 769 | }, 770 | "metadata": {}, 771 | "execution_count": 65 772 | } 773 | ] 774 | }, 775 | { 776 | "cell_type": "markdown", 777 | "source": [ 778 | "## Matrix Multiplication" 779 | ], 780 | "metadata": { 781 | "id": "KTJlFliM-V3H" 782 | } 783 | }, 784 | { 785 | "cell_type": "code", 786 | "source": [ 787 | "# AB = C\n", 788 | "# C[i][j] = dot product of ith row of A and jth column B\n", 789 | "\n", 790 | "# dot product = generator expression + zipping of ith row of A and jth column B" 791 | ], 792 | "metadata": { 793 | "id": "NEksDO0v-ZGK" 794 | }, 795 | "execution_count": null, 796 | "outputs": [] 797 | }, 798 | { 799 | "cell_type": "code", 800 | "source": [ 801 | "A = [\n", 802 | " [1, 2, 3],\n", 803 | " [4, 5, 6]\n", 804 | "]\n", 805 | "\n", 806 | "B = [\n", 807 | " [7, 8],\n", 808 | " [9, 10],\n", 809 | " [11, 12]\n", 810 | "]" 811 | ], 812 | "metadata": { 813 | "id": "pDT8tESbM2LX" 814 | }, 815 | "execution_count": 66, 816 | "outputs": [] 817 | }, 818 | { 819 | "cell_type": "code", 820 | "source": [ 821 | "cols = zip(*B)\n", 822 | "list(cols)" 823 | ], 824 | "metadata": { 825 | "colab": { 826 | "base_uri": "https://localhost:8080/" 827 | }, 828 | "id": "AoPBiOsKMx95", 829 | "outputId": "eab50607-f856-4379-a694-2c9bd82ec3d5" 830 | }, 831 | "execution_count": 71, 832 | "outputs": [ 833 | { 834 | "output_type": "execute_result", 835 | "data": { 836 | "text/plain": [ 837 | "[(7, 9, 11), (8, 10, 12)]" 838 | ] 839 | }, 840 | "metadata": {}, 841 | "execution_count": 71 842 | } 843 | ] 844 | }, 845 | { 846 | "cell_type": "code", 847 | "source": [ 848 | "# ith row of C = A*B = sequence of dot products of ith row of A and all the columns of B" 849 | ], 850 | "metadata": { 851 | "id": "GEviEFRaN_lv" 852 | }, 853 | "execution_count": null, 854 | "outputs": [] 855 | }, 856 | { 857 | "cell_type": "code", 858 | "source": [ 859 | "def matmul(A,B):\n", 860 | " return [[sum (a*b for a,b in zip(rowA, colB)) # dot product of rowA and colB\n", 861 | " for colB in zip(*B)]\n", 862 | " for rowA in A]" 863 | ], 864 | "metadata": { 865 | "id": "Hjvr-NSxNCUw" 866 | }, 867 | "execution_count": 69, 868 | "outputs": [] 869 | }, 870 | { 871 | "cell_type": "code", 872 | "source": [ 873 | "matmul(A,B)" 874 | ], 875 | "metadata": { 876 | "colab": { 877 | "base_uri": "https://localhost:8080/" 878 | }, 879 | "id": "E7SNapDnOO2s", 880 | "outputId": "882e62fd-986e-401d-e81b-70c7bdd72ec3" 881 | }, 882 | "execution_count": 70, 883 | "outputs": [ 884 | { 885 | "output_type": "execute_result", 886 | "data": { 887 | "text/plain": [ 888 | "[[58, 64], [139, 154]]" 889 | ] 890 | }, 891 | "metadata": {}, 892 | "execution_count": 70 893 | } 894 | ] 895 | }, 896 | { 897 | "cell_type": "markdown", 898 | "source": [ 899 | "## Matrix Mean by Row or Column (HW)" 900 | ], 901 | "metadata": { 902 | "id": "onMX95XJ-ZnU" 903 | } 904 | }, 905 | { 906 | "cell_type": "code", 907 | "source": [], 908 | "metadata": { 909 | "id": "pnerm-oh_KIY" 910 | }, 911 | "execution_count": null, 912 | "outputs": [] 913 | } 914 | ] 915 | } --------------------------------------------------------------------------------