├── LICENSE ├── README.md └── data ├── gaint_panda.bmp ├── gaint_panda.jpg ├── gaint_panda_gray.bmp ├── gaint_panda_rgb.jpg └── sample_speed_time_series.npz /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Xinyu Chen 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 张量计算系列教程 2 | 3 | [![MIT License](https://img.shields.io/badge/license-MIT-green.svg)](https://opensource.org/licenses/MIT) 4 | ![Python 3.7](https://img.shields.io/badge/Python-3.7-blue.svg) 5 | [![GitHub stars](https://img.shields.io/github/stars/xinychen/tensor-book.svg?logo=github&label=Stars&logoColor=white)](https://github.com/xinychen/tensor-book) 6 | 7 |

Made by Xinyu Chen (陈新宇) • :globe_with_meridians: https://xinychen.github.io

8 | 9 | 10 | ## 系列教程 11 | 12 | 1. [《从线性代数到张量计算》(PDF)](https://xinychen.github.io/books/tensor_book.pdf) 13 | 2. [《**机器学习与时空数据建模**》(PDF, 已更新超过100页)](https://xinychen.github.io/books/spatiotemporal_low_rank_models.pdf) 14 | 15 |
16 | 17 | > 该系列教程尚处在更新阶段，欢迎广大读者提供宝贵的反馈意见，如有建议，请在本项目issues区域留言；如需交流，请移步至QQ群（457012422），非诚勿扰。 18 | 19 |
20 | 21 | ## 作者申明 22 | 23 | - 撰写该系列教程的初衷在于传播知识、丰富中文科技文库，为感兴趣的读者提供参考素材。 24 | - 禁止将该系列教程放在其他网站上，唯一下载网址为[https://xinychen.github.io](https://xinychen.github.io)。 25 | - 禁止将该系列教程用于任何形式的商业活动。 26 | 27 |
28 | 29 |

代数结构

30 |

^{▴ 回到顶部}

Kronecker分解与Kronecker分解

53 |

^{▴ 回到顶部}

54 | 55 | **例.** 计算Kronecker积并求Kronecker分解。 56 | 57 | - 计算Kronecker积 58 | 59 | ```python 60 | import numpy as np 61 | 62 | A = np.array([[1, 2], [3, 4]]) 63 | B = np.array([[5, 6, 7], [8, 9, 10]]) 64 | X = np.kron(A, B) 65 | print('X = ') 66 | print(X) 67 | ``` 68 | 69 | - 求Kronecker分解 70 | 71 | ```python 72 | def permute(mat, A_dim1, A_dim2, B_dim1, B_dim2): 73 | ans = np.zeros((A_dim1 * A_dim2, B_dim1 * B_dim2)) 74 | for j in range(A_dim2): 75 | for i in range(A_dim1): 76 | ans[A_dim1 * j + i, :] = mat[i * B_dim1 : (i + 1) * B_dim1, 77 | j * B_dim2 : (j + 1) * B_dim2].reshape(B_dim1 * B_dim2, order = 'F') 78 | return ans 79 | 80 | X_tilde = permute(X, 2, 2, 2, 3) 81 | print('X_tilde = ') 82 | print(X_tilde) 83 | print() 84 | u, s, v = np.linalg.svd(X_tilde, full_matrices = False) 85 | A_hat = np.sqrt(s[0]) * u[:, 0].reshape(2, 2, order = 'F') 86 | B_hat = np.sqrt(s[0]) * v[0, :].reshape(2, 3, order = 'F') 87 | print('A_hat = ') 88 | print(A_hat) 89 | print() 90 | print('B_hat = ') 91 | print(B_hat) 92 | ``` 93 | 94 | **例.** 采用广义 Kronecker 分解重构灰度图像。 95 | 96 | ```python 97 | import numpy as np 98 | 99 | def permute(mat, A_dim1, A_dim2, B_dim1, B_dim2): 100 | ans = np.zeros((A_dim1 * A_dim2, B_dim1 * B_dim2)) 101 | for j in range(A_dim2): 102 | for i in range(A_dim1): 103 | ans[A_dim1 * j + i, :] = mat[i * B_dim1 : (i + 1) * B_dim1, 104 | j * B_dim2 : (j + 1) * B_dim2].reshape(B_dim1 * B_dim2, order = 'F') 105 | return ans 106 | 107 | def kron_decomp(mat, A_dim1, A_dim2, B_dim1, B_dim2, rank): 108 | mat_tilde = permute(mat, A_dim1, A_dim2, B_dim1, B_dim2) 109 | u, s, v = np.linalg.svd(mat_tilde, full_matrices = False) 110 | A_hat = np.zeros((A_dim1, A_dim2, rank)) 111 | B_hat = np.zeros((B_dim1, B_dim2, rank)) 112 | for r in range(rank): 113 | A_hat[:, :, r] = np.sqrt(s[r]) * u[:, r].reshape([A_dim1, A_dim2], order = 'F') 114 | B_hat[:, :, r] = np.sqrt(s[r]) * v[r, :].reshape([B_dim1, B_dim2], order = 'F') 115 | mat_hat = np.zeros(mat.shape) 116 | for r in range(rank): 117 | mat_hat += np.kron(A_hat[:, :, r], B_hat[:, :, r]) 118 | return mat_hat 119 | ``` 120 | 121 | ```python 122 | from skimage import color 123 | from skimage import io 124 | 125 | img = io.imread('data/gaint_panda.bmp') 126 | imgGray = color.rgb2gray(img) 127 | 128 | io.imshow(imgGray) 129 | plt.axis('off') 130 | plt.imsave('gaint_panda_gray.png', imgGray, cmap = plt.cm.gray) 131 | plt.show() 132 | ``` 133 | 134 | ```python 135 | import imageio 136 | import matplotlib.pyplot as plt 137 | 138 | img = io.imread('data/gaint_panda.bmp') 139 | mat = color.rgb2gray(img) 140 | io.imshow(mat) 141 | plt.axis('off') 142 | plt.show() 143 | 144 | A_dim1 = 16 145 | A_dim2 = 32 146 | B_dim1 = 32 147 | B_dim2 = 16 148 | for rank in [5, 10, 50, 100]: 149 | mat_hat = kron_decomp(mat, A_dim1, A_dim2, B_dim1, B_dim2, rank) 150 | mat_hat[mat_hat > 1] = 1 151 | mat_hat[mat_hat < 0] = 0 152 | io.imshow(mat_hat) 153 | plt.axis('off') 154 | plt.imsave('gaint_panda_gray_R{}.png'.format(rank), 155 | mat_hat, cmap = plt.cm.gray) 156 | plt.show() 157 | ``` 158 | 159 |

模态积与Tucker张量分解

160 |

^{▴ 回到顶部}

低秩时序矩阵模型

180 |

^{▴ 回到顶部}

181 | 182 | **例.** 使用时序矩阵分解对流体流动的动态过程进行预测。 183 | 184 | ```python 185 | import numpy as np 186 | import seaborn as sns 187 | import scipy.io 188 | import matplotlib.pyplot as plt 189 | from matplotlib.colors import ListedColormap, LinearSegmentedColormap 190 | color = scipy.io.loadmat('CCcool.mat') 191 | cc = color['CC'] 192 | newcmp = LinearSegmentedColormap.from_list('', cc) 193 | 194 | dense_tensor = np.load('tensor.npz')['arr_0'] 195 | np.random.seed(1) 196 | dense_tensor = dense_tensor[:, :, : 150] 197 | M, N, T = dense_tensor.shape 198 | 199 | plt.rcParams['font.size'] = 13 200 | plt.rcParams['mathtext.fontset'] = 'cm' 201 | fig = plt.figure(figsize = (7, 8)) 202 | id = np.array([5, 10, 15, 20, 25, 30, 35, 40]) 203 | for t in range(8): 204 | ax = fig.add_subplot(4, 2, t + 1) 205 | ax = sns.heatmap(dense_tensor[:, :, id[t] - 1], cmap = newcmp, vmin = -5, vmax = 5, cbar = False) 206 | ax.contour(np.linspace(0, N, N), np.linspace(0, M, M), dense_tensor[:, :, id[t] - 1], 207 | levels = np.linspace(0.15, 15, 30), colors = 'k', linewidths = 0.7) 208 | ax.contour(np.linspace(0, N, N), np.linspace(0, M, M), dense_tensor[:, :, id[t] - 1], 209 | levels = np.linspace(-15, -0.15, 30), colors = 'k', linestyles = 'dashed', linewidths = 0.7) 210 | plt.xticks([]) 211 | plt.yticks([]) 212 | plt.title(r'$t = {}$'.format(id[t])) 213 | for _, spine in ax.spines.items(): 214 | spine.set_visible(True) 215 | plt.show() 216 | fig.savefig("fluid_flow_heatmap.png", bbox_inches = "tight") 217 | ``` 218 | 219 | ```python 220 | import numpy as np 221 | 222 | def update_cg(var, r, q, Aq, rold): 223 | alpha = rold / np.inner(q, Aq) 224 | var = var + alpha * q 225 | r = r - alpha * Aq 226 | rnew = np.inner(r, r) 227 | q = r + (rnew / rold) * q 228 | return var, r, q, rnew 229 | 230 | def ell_w(ind, W, X, rho): 231 | return X @ ((W.T @ X) * ind).T + rho * W 232 | 233 | def conj_grad_w(sparse_mat, ind, W, X, rho, maxiter = 5): 234 | rank, dim1 = W.shape 235 | w = np.reshape(W, -1, order = 'F') 236 | r = np.reshape(X @ sparse_mat.T - ell_w(ind, W, X, rho), -1, order = 'F') 237 | q = r.copy() 238 | rold = np.inner(r, r) 239 | for it in range(maxiter): 240 | Q = np.reshape(q, (rank, dim1), order = 'F') 241 | Aq = np.reshape(ell_w(ind, Q, X, rho), -1, order = 'F') 242 | w, r, q, rold = update_cg(w, r, q, Aq, rold) 243 | return np.reshape(w, (rank, dim1), order = 'F') 244 | 245 | def ell_x(ind, W, X, A, Psi, d, lambda0, rho): 246 | rank, dim2 = X.shape 247 | temp = np.zeros((d * rank, Psi[0].shape[0])) 248 | for k in range(1, d + 1): 249 | temp[(k - 1) * rank : k * rank, :] = X @ Psi[k].T 250 | temp1 = X @ Psi[0].T - A @ temp 251 | temp2 = np.zeros((rank, dim2)) 252 | for k in range(d): 253 | temp2 += A[:, k * rank : (k + 1) * rank].T @ temp1 @ Psi[k + 1] 254 | return W @ ((W.T @ X) * ind) + rho * X + lambda0 * (temp1 @ Psi[0] - temp2) 255 | 256 | def conj_grad_x(sparse_mat, ind, W, X, A, Psi, d, lambda0, rho, maxiter = 5): 257 | rank, dim2 = X.shape 258 | x = np.reshape(X, -1, order = 'F') 259 | r = np.reshape(W @ sparse_mat - ell_x(ind, W, X, A, Psi, d, lambda0, rho), -1, order = 'F') 260 | q = r.copy() 261 | rold = np.inner(r, r) 262 | for it in range(maxiter): 263 | Q = np.reshape(q, (rank, dim2), order = 'F') 264 | Aq = np.reshape(ell_x(ind, W, Q, A, Psi, d, lambda0, rho), -1, order = 'F') 265 | x, r, q, rold = update_cg(x, r, q, Aq, rold) 266 | return np.reshape(x, (rank, dim2), order = 'F') 267 | 268 | def generate_Psi(T, d): 269 | Psi = [] 270 | for k in range(0, d + 1): 271 | if k == 0: 272 | Psi.append(np.append(np.zeros((T - d, d)), np.eye(T - d), axis = 1)) 273 | else: 274 | Psi.append(np.append(np.append(np.zeros((T - d, d - k)), np.eye(T - d), axis = 1), 275 | np.zeros((T - d, k)), axis = 1)) 276 | return Psi 277 | 278 | def tmf(sparse_mat, rank, d, lambda0, rho, maxiter = 50): 279 | dim1, dim2 = sparse_mat.shape 280 | ind = sparse_mat != 0 281 | W = 0.01 * np.random.randn(rank, dim1) 282 | X = 0.01 * np.random.randn(rank, dim2) 283 | A = 0.01 * np.random.randn(rank, d * rank) 284 | Psi = generate_Psi(dim2, d) 285 | temp = np.zeros((d * rank, dim2 - d)) 286 | for it in range(maxiter): 287 | W = conj_grad_w(sparse_mat, ind, W, X, rho) 288 | X = conj_grad_x(sparse_mat, ind, W, X, A, Psi, d, lambda0, rho) 289 | for k in range(1, d + 1): 290 | temp[(k - 1) * rank : k * rank, :] = X @ Psi[k].T 291 | A = X @ Psi[0].T @ np.linalg.pinv(temp) 292 | mat_hat = W.T @ X 293 | return mat_hat, W, X, A 294 | 295 | def var4cast(X, A, d, delta): 296 | dim1, dim2 = X.shape 297 | X_hat = np.append(X, np.zeros((dim1, delta)), axis = 1) 298 | for t in range(delta): 299 | X_hat[:, dim2 + t] = A @ X_hat[:, dim2 + t - np.arange(1, d + 1)].T.reshape(dim1 * d) 300 | return X_hat[:, - delta :] 301 | ``` 302 | 303 | ```python 304 | import numpy as np 305 | np.random.seed(1) 306 | 307 | dense_tensor = np.load('tensor.npz')['arr_0'] 308 | dense_tensor = dense_tensor[:, :, : 150] 309 | M, N, T = dense_tensor.shape 310 | dense_mat = np.reshape(dense_tensor, (M * N, T), order = 'F') 311 | p = 0.5 312 | sparse_mat = dense_mat * np.round(np.random.rand(M * N, T) + 0.5 - p) 313 | 314 | import time 315 | start = time.time() 316 | delta = 3 317 | rank = 10 318 | d = 1 319 | lambda0 = 1 320 | rho = 1 321 | _, W, X, A = tmf(sparse_mat[:, : T - delta], rank, d, lambda0, rho) 322 | mat_hat = W.T @ var4cast(X, A, d, delta) 323 | end = time.time() 324 | print('Running time: %d seconds'%(end - start)) 325 | ``` 326 | 327 | ```python 328 | import seaborn as sns 329 | import scipy.io 330 | import matplotlib.pyplot as plt 331 | from matplotlib.colors import ListedColormap, LinearSegmentedColormap 332 | color = scipy.io.loadmat('CCcool.mat') 333 | cc = color['CC'] 334 | newcmp = LinearSegmentedColormap.from_list('', cc) 335 | 336 | plt.rcParams['font.size'] = 13 337 | plt.rcParams['mathtext.fontset'] = 'cm' 338 | fig = plt.figure(figsize = (11, 1.8)) 339 | i = 1 340 | for t in [147, 148, 149]: 341 | ax = fig.add_subplot(1, 3, i) 342 | ax = sns.heatmap(dense_mat[:, t].reshape((199, 449), order = 'F'), 343 | cmap = newcmp, vmin = -5, vmax = 5, cbar = False) 344 | ax.contour(np.linspace(0, N, N), np.linspace(0, M, M), 345 | dense_mat[:, t].reshape((199, 449), order = 'F'), 346 | levels = np.linspace(0.15, 15, 30), colors = 'k', 347 | linewidths = 0.7) 348 | ax.contour(np.linspace(0, N, N), np.linspace(0, M, M), 349 | dense_mat[:, t].reshape((199, 449), order = 'F'), 350 | levels = np.linspace(-15, -0.15, 30), colors = 'k', 351 | linestyles = 'dashed', linewidths = 0.7) 352 | plt.title(r'$t = {}$'.format(t + 1)) 353 | plt.xticks([]) 354 | plt.yticks([]) 355 | for _, spine in ax.spines.items(): 356 | spine.set_visible(True) 357 | i += 1 358 | plt.show() 359 | fig.savefig("fluid_flow_ground_truth.png", bbox_inches = "tight") 360 | ``` 361 | 362 | ```python 363 | import seaborn as sns 364 | import scipy.io 365 | import matplotlib.pyplot as plt 366 | from matplotlib.colors import ListedColormap, LinearSegmentedColormap 367 | color = scipy.io.loadmat('CCcool.mat') 368 | cc = color['CC'] 369 | newcmp = LinearSegmentedColormap.from_list('', cc) 370 | 371 | plt.rcParams['font.size'] = 13 372 | plt.rcParams['mathtext.fontset'] = 'cm' 373 | fig = plt.figure(figsize = (11, 1.8)) 374 | i = 1 375 | for t in range(3): 376 | ax = fig.add_subplot(1, 3, i) 377 | ax = sns.heatmap(mat_hat[:, t].reshape((199, 449), order = 'F'), 378 | cmap = newcmp, vmin = -5, vmax = 5, cbar = False) 379 | ax.contour(np.linspace(0, N, N), np.linspace(0, M, M), 380 | mat_hat[:, t].reshape((199, 449), order = 'F'), 381 | levels = np.linspace(0.15, 15, 30), colors = 'k', 382 | linewidths = 0.7) 383 | ax.contour(np.linspace(0, N, N), np.linspace(0, M, M), 384 | mat_hat[:, t].reshape((199, 449), order = 'F'), 385 | levels = np.linspace(-15, -0.15, 30), colors = 'k', 386 | linestyles = 'dashed', linewidths = 0.7) 387 | plt.title(r'$t = {}$'.format(148+ t)) 388 | plt.xticks([]) 389 | plt.yticks([]) 390 | for _, spine in ax.spines.items(): 391 | spine.set_visible(True) 392 | i += 1 393 | plt.show() 394 | fig.savefig("fluid_flow_forecasts.png", bbox_inches = "tight") 395 | ``` 396 | 397 | **例.** 使用考虑平滑处理的矩阵分解对灰度图像进行复原。 398 | 399 | ```python 400 | import numpy as np 401 | 402 | def compute_rse(var, var_hat): 403 | return np.linalg.norm(var - var_hat, 2) / np.linalg.norm(var, 2) 404 | 405 | def generate_Psi(n): 406 | mat1 = np.append(np.zeros((n - 1, 1)), np.eye(n - 1), axis = 1) 407 | mat2 = np.append(np.eye(n - 1), np.zeros((n - 1, 1)), axis = 1) 408 | Psi = mat1 - mat2 409 | return Psi 410 | 411 | def update_cg(var, r, q, Aq, rold): 412 | alpha = rold / np.inner(q, Aq) 413 | var = var + alpha * q 414 | r = r - alpha * Aq 415 | rnew = np.inner(r, r) 416 | q = r + (rnew / rold) * q 417 | return var, r, q, rnew 418 | 419 | def ell_w(ind, W, X, Psi1, rho, lmbda): 420 | return X @ ((W.T @ X) * ind).T + rho * W + lmbda * W @ Psi1.T @ Psi1 421 | 422 | def conj_grad_w(sparse_mat, ind, W, X, Psi1, rho, lmbda, maxiter = 5): 423 | rank, dim1 = W.shape 424 | w = np.reshape(W, -1, order = 'F') 425 | r = np.reshape(X @ sparse_mat.T 426 | - ell_w(ind, W, X, Psi1, rho, lmbda), -1, order = 'F') 427 | q = r.copy() 428 | rold = np.inner(r, r) 429 | for it in range(maxiter): 430 | Q = np.reshape(q, (rank, dim1), order = 'F') 431 | Aq = np.reshape(ell_w(ind, Q, X, Psi1, rho, lmbda), -1, order = 'F') 432 | w, r, q, rold = update_cg(w, r, q, Aq, rold) 433 | return np.reshape(w, (rank, dim1), order = 'F') 434 | 435 | def ell_x(ind, W, X, Psi2, rho, lmbda): 436 | return W @ ((W.T @ X) * ind) + rho * X + lmbda * X @ Psi2.T @ Psi2 437 | 438 | def conj_grad_x(sparse_mat, ind, W, X, Psi2, rho, lmbda, maxiter = 5): 439 | rank, dim2 = X.shape 440 | x = np.reshape(X, -1, order = 'F') 441 | r = np.reshape(W @ sparse_mat 442 | - ell_x(ind, W, X, Psi2, rho, lmbda), -1, order = 'F') 443 | q = r.copy() 444 | rold = np.inner(r, r) 445 | for it in range(maxiter): 446 | Q = np.reshape(q, (rank, dim2), order = 'F') 447 | Aq = np.reshape(ell_x(ind, W, Q, Psi2, rho, lmbda), -1, order = 'F') 448 | x, r, q, rold = update_cg(x, r, q, Aq, rold) 449 | return np.reshape(x, (rank, dim2), order = 'F') 450 | 451 | def smoothing_mf(dense_mat, sparse_mat, rank, rho, lmbda, maxiter = 50): 452 | dim1, dim2 = sparse_mat.shape 453 | W = 0.01 * np.random.randn(rank, dim1) 454 | X = 0.01 * np.random.randn(rank, dim2) 455 | ind = sparse_mat != 0 456 | pos_test = np.where((dense_mat != 0) & (sparse_mat == 0)) 457 | dense_test = dense_mat[pos_test] 458 | del dense_mat 459 | Psi1 = generate_Psi(dim1) 460 | Psi2 = generate_Psi(dim2) 461 | show_iter = 10 462 | for it in range(maxiter): 463 | W = conj_grad_w(sparse_mat, ind, W, X, Psi1, rho, lmbda) 464 | X = conj_grad_x(sparse_mat, ind, W, X, Psi2, rho, lmbda) 465 | mat_hat = W.T @ X 466 | if (it + 1) % show_iter == 0: 467 | temp_hat = mat_hat[pos_test] 468 | print('Iter: {}'.format(it + 1)) 469 | print(compute_rse(temp_hat, dense_test)) 470 | print() 471 | return mat_hat, W, X 472 | ``` 473 | 474 | ```python 475 | import numpy as np 476 | np.random.seed(1) 477 | import matplotlib.pyplot as plt 478 | from skimage import color 479 | from skimage import io 480 | 481 | img = io.imread('data/gaint_panda.bmp') 482 | imgGray = color.rgb2gray(img) 483 | M, N = imgGray.shape 484 | missing_rate = 0.9 485 | 486 | sparse_img = imgGray * np.round(np.random.rand(M, N) + 0.5 - missing_rate) 487 | io.imshow(sparse_img) 488 | plt.axis('off') 489 | plt.show() 490 | ``` 491 | 492 | ```python 493 | lmbda = 1e+1 494 | for rank in [5, 10, 50]: 495 | rho = 1e-1 496 | maxiter = 100 497 | mat_hat, W, X = smoothing_mf(imgGray, sparse_img, rank, rho, lmbda, maxiter) 498 | 499 | mat_hat[mat_hat < 0] = 0 500 | mat_hat[mat_hat > 1] = 1 501 | io.imshow(mat_hat) 502 | plt.axis('off') 503 | plt.imsave('gaint_panda_gray_recovery_90_mf_rank_{}_lmbda_10.png'.format(rank), 504 | mat_hat, cmap = plt.cm.gray) 505 | plt.show() 506 | ``` 507 | 508 | ```python 509 | lmbda = 1e-10 510 | for rank in [5, 10, 50]: 511 | rho = 1e-1 512 | maxiter = 100 513 | mat_hat, W, X = smoothing_mf(imgGray, sparse_img, rank, rho, lmbda, maxiter) 514 | 515 | mat_hat[mat_hat < 0] = 0 516 | mat_hat[mat_hat > 1] = 1 517 | io.imshow(mat_hat) 518 | plt.axis('off') 519 | plt.imsave('gaint_panda_gray_recovery_90_mf_rank_{}_lmbda_0.png'.format(rank), 520 | mat_hat, cmap = plt.cm.gray) 521 | plt.show() 522 | ``` 523 | 524 | **例.** 使用考虑空间自回归的矩阵分解对灰度图像进行复原。 525 | 526 | ```python 527 | import numpy as np 528 | 529 | def compute_rse(var, var_hat): 530 | return np.linalg.norm(var - var_hat, 2) / np.linalg.norm(var, 2) 531 | 532 | def generate_Psi(n): 533 | mat1 = np.append(np.zeros((n - 1, 1)), np.eye(n - 1), axis = 1) 534 | mat2 = np.append(np.eye(n - 1), np.zeros((n - 1, 1)), axis = 1) 535 | return mat1, mat2 536 | 537 | def update_cg(var, r, q, Aq, rold): 538 | alpha = rold / np.inner(q, Aq) 539 | var = var + alpha * q 540 | r = r - alpha * Aq 541 | rnew = np.inner(r, r) 542 | q = r + (rnew / rold) * q 543 | return var, r, q, rnew 544 | 545 | def ell_w(ind, W, X, Aw, Phi0, Phi1, rho, lmbda): 546 | temp1 = W @ Phi0.T - Aw @ W @ Phi1.T 547 | temp2 = lmbda * temp1 @ Phi0 - lmbda * Aw.T @ temp1 @ Phi1 548 | return X @ ((W.T @ X) * ind).T + rho * W + temp2 549 | 550 | def conj_grad_w(sparse_mat, ind, W, X, Aw, Phi0, Phi1, rho, lmbda, maxiter = 5): 551 | rank, dim1 = W.shape 552 | w = np.reshape(W, -1, order = 'F') 553 | r = np.reshape(X @ sparse_mat.T 554 | - ell_w(ind, W, X, Aw, Phi0, Phi1, rho, lmbda), -1, order = 'F') 555 | q = r.copy() 556 | rold = np.inner(r, r) 557 | for it in range(maxiter): 558 | Q = np.reshape(q, (rank, dim1), order = 'F') 559 | Aq = np.reshape(ell_w(ind, Q, X, Aw, Phi0, Phi1, rho, lmbda), -1, order = 'F') 560 | w, r, q, rold = update_cg(w, r, q, Aq, rold) 561 | return np.reshape(w, (rank, dim1), order = 'F') 562 | 563 | def ell_x(ind, W, X, Ax, Psi0, Psi1, rho, lmbda): 564 | temp1 = X @ Psi0.T - Ax @ X @ Psi1.T 565 | temp2 = lmbda * temp1 @ Psi0 - lmbda * Ax.T @ temp1 @ Psi1 566 | return W @ ((W.T @ X) * ind) + rho * X + temp2 567 | 568 | def conj_grad_x(sparse_mat, ind, W, X, Ax, Psi0, Psi1, rho, lmbda, maxiter = 5): 569 | rank, dim2 = X.shape 570 | x = np.reshape(X, -1, order = 'F') 571 | r = np.reshape(W @ sparse_mat 572 | - ell_x(ind, W, X, Ax, Psi0, Psi1, rho, lmbda), -1, order = 'F') 573 | q = r.copy() 574 | rold = np.inner(r, r) 575 | for it in range(maxiter): 576 | Q = np.reshape(q, (rank, dim2), order = 'F') 577 | Aq = np.reshape(ell_x(ind, W, Q, Ax, Psi0, Psi1, rho, lmbda), -1, order = 'F') 578 | x, r, q, rold = update_cg(x, r, q, Aq, rold) 579 | return np.reshape(x, (rank, dim2), order = 'F') 580 | 581 | def spatial_autoregressive_mf(dense_mat, sparse_mat, rank, rho, lmbda, maxiter = 50): 582 | dim1, dim2 = sparse_mat.shape 583 | W = 0.01 * np.random.randn(rank, dim1) 584 | X = 0.01 * np.random.randn(rank, dim2) 585 | Aw = 0.01 * np.random.randn(rank, rank) 586 | Ax = 0.01 * np.random.randn(rank, rank) 587 | ind = sparse_mat != 0 588 | pos_test = np.where((dense_mat != 0) & (sparse_mat == 0)) 589 | dense_test = dense_mat[pos_test] 590 | del dense_mat 591 | Phi0, Phi1 = generate_Psi(dim1) 592 | Psi0, Psi1 = generate_Psi(dim2) 593 | show_iter = 10 594 | for it in range(maxiter): 595 | W = conj_grad_w(sparse_mat, ind, W, X, Aw, Phi0, Phi1, rho, lmbda) 596 | X = conj_grad_x(sparse_mat, ind, W, X, Ax, Psi0, Psi1, rho, lmbda) 597 | Aw = W @ Phi0.T @ np.linalg.pinv(W @ Phi1.T) 598 | Ax = X @ Psi0.T @ np.linalg.pinv(X @ Psi1.T) 599 | mat_hat = W.T @ X 600 | if (it + 1) % show_iter == 0: 601 | temp_hat = mat_hat[pos_test] 602 | print('Iter: {}'.format(it + 1)) 603 | print(compute_rse(temp_hat, dense_test)) 604 | print() 605 | return mat_hat, W, X, Aw, Ax 606 | ``` 607 | 608 | ```python 609 | import numpy as np 610 | np.random.seed(1) 611 | import matplotlib.pyplot as plt 612 | from skimage import color 613 | from skimage import io 614 | 615 | img = io.imread('data/gaint_panda.bmp') 616 | imgGray = color.rgb2gray(img) 617 | M, N = imgGray.shape 618 | missing_rate = 0.9 619 | 620 | sparse_img = imgGray * np.round(np.random.rand(M, N) + 0.5 - missing_rate) 621 | io.imshow(sparse_img) 622 | plt.axis('off') 623 | plt.show() 624 | ``` 625 | 626 | ```python 627 | lmbda = 5e-1 628 | for rank in [5, 10, 50]: 629 | rho = 1e-1 630 | maxiter = 100 631 | mat_hat, W, X, Aw, Ax = spatial_autoregressive_mf(imgGray, sparse_img, rank, rho, lmbda, maxiter) 632 | 633 | mat_hat[mat_hat < 0] = 0 634 | mat_hat[mat_hat > 1] = 1 635 | io.imshow(mat_hat) 636 | plt.axis('off') 637 | plt.imsave('gaint_panda_gray_recovery_90_spatial_ar_mf_rank_{}.png'.format(rank), 638 | mat_hat, cmap = plt.cm.gray) 639 | plt.show() 640 | ``` 641 | 642 | **例.** 根据卷积定理计算循环卷积。 643 | 644 | ```python 645 | import numpy as np 646 | 647 | x = np.array([0, 1, 2, 3, 4]) 648 | y = np.array([2, -1, 3]) 649 | fx = np.fft.fft(x) 650 | fy = np.fft.fft(np.append(y, np.zeros(2), axis = 0)) 651 | z = np.fft.ifft(fx * fy).real 652 | ``` 653 | 654 | **例.** 根据卷积定理计算向量**y**。 655 | 656 | ```python 657 | import numpy as np 658 | 659 | x = np.array([0, 1, 2, 3, 4]) 660 | z = np.array([5, 14, 3, 7, 11]) 661 | fx = np.fft.fft(x) 662 | fz = np.fft.fft(z) 663 | y = np.fft.ifft(fz / fx).real 664 | ``` 665 | 666 | **例.** 根据卷积定理计算二维循环卷积。 667 | 668 | ```python 669 | import numpy as np 670 | 671 | X = np.array([[1, 2, 3, 4], [4, 5, 6, 7], 672 | [7, 8, 9, 10], [10, 11, 12, 13]]) 673 | K = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) 674 | pad_K = np.zeros(X.shape) 675 | pad_K[: K.shape[0], : K.shape[1]] = K 676 | Y = np.fft.ifft2(np.fft.fft2(X) * np.fft.fft2(pad_K)).real 677 | ``` 678 | 679 | **例.** 计算矩阵的F范数与傅立叶变换的F范数。 680 | 681 | ```python 682 | import numpy as np 683 | 684 | X = np.array([[5, 6, 7], [8, 9, 10]]) 685 | print('The squared Frobenius norm of the matrix X:') 686 | print(np.linalg.norm(X, 'fro') ** 2) 687 | print() 688 | print('The squared Frobenius norm of the matrix Fx:') 689 | print(np.linalg.norm(np.fft.fft2(X), 'fro') ** 2) 690 | print() 691 | ``` 692 | 693 | **例.** 计算循环矩阵的奇异值与L1范数。 694 | 695 | ```python 696 | import numpy as np 697 | 698 | def circ_mat(vec): 699 | n = vec.shape[0] 700 | mat = np.zeros((n, n)) 701 | mat[:, 0] = vec 702 | for k in range(1, n): 703 | mat[:, k] = np.append(vec[n - k :], vec[: n - k], axis = 0) 704 | return mat 705 | 706 | x = np.array([0, 1, 2, 3, 4]) 707 | Cx = circ_mat(x) 708 | s = np.linalg.svd(Cx, full_matrices = False, compute_uv = False) 709 | print('Singular values of the circulant matrix Cx:') 710 | print(s) 711 | print() 712 | print('Discrete Fourier transform of the vector x:') 713 | print(np.fft.fft(x)) 714 | print() 715 | ``` 716 | 717 | **例.** 对循环矩阵核范数最小化问题进行求解。 718 | 719 | ```python 720 | import numpy as np 721 | 722 | z = np.array([0, 1, 2, 3, 4]) 723 | lmbda = 2 724 | T = z.shape[0] 725 | 726 | h_hat = np.fft.fft(z) 727 | print('h_hat = ') 728 | print(h_hat) 729 | print() 730 | h_hat_abs = np.abs(h_hat) 731 | print('h_hat_abs = ') 732 | print(h_hat_abs) 733 | print() 734 | 735 | temp = h_hat_abs - T / lmbda 736 | temp[temp <= 0] = 0 737 | x = np.fft.ifft(h_hat / h_hat_abs * temp).real 738 | print('x = ') 739 | print(x) 740 | print() 741 | print('The objective function is: ') 742 | print(np.sum(np.abs(np.fft.fft(x))) 743 | + 0.5 * lmbda * np.linalg.norm(x - z, 2) ** 2) 744 | print() 745 | ``` 746 | 747 | **例.** 使用循环矩阵核范数最小化算法对灰度图像进行复原。 748 | 749 | ```python 750 | import numpy as np 751 | 752 | def compute_rse(var, var_hat): 753 | return np.linalg.norm(var - var_hat, 2) / np.linalg.norm(var, 2) 754 | 755 | def prox(z, w, lmbda): 756 | T = z.shape[0] 757 | temp = np.fft.fft(z - w / lmbda) 758 | temp1 = np.abs(temp) - T / lmbda 759 | temp1[temp1 <= 0] = 0 760 | return np.fft.ifft(temp / np.abs(temp) * temp1).real 761 | 762 | def update_z(y_train, pos_train, x, w, lmbda, eta): 763 | z = x + w / lmbda 764 | z[pos_train] = (lmbda / (lmbda + eta) * z[pos_train] 765 | + eta / (lmbda + eta) * y_train) 766 | return z 767 | 768 | def update_w(x, z, w, lmbda): 769 | return w + lmbda * (x - z) 770 | 771 | def circ_nnm(y_true, y, lmbda, eta, maxiter = 50): 772 | pos_train = np.where(y != 0) 773 | y_train = y[pos_train] 774 | pos_test = np.where((y_true != 0) & (y == 0)) 775 | y_test = y_true[pos_test] 776 | z = y.copy() 777 | w = y.copy() 778 | del y_true, y 779 | show_iter = 10 780 | for it in range(maxiter): 781 | x = prox(z, w, lmbda) 782 | z = update_z(y_train, pos_train, x, w, lmbda, eta) 783 | w = update_w(x, z, w, lmbda) 784 | if (it + 1) % show_iter == 0: 785 | print(it + 1) 786 | print(compute_rse(y_test, x[pos_test])) 787 | print() 788 | return x 789 | ``` 790 | 791 | ```python 792 | import numpy as np 793 | np.random.seed(1) 794 | import matplotlib.pyplot as plt 795 | from skimage import color 796 | from skimage import io 797 | 798 | img = io.imread('data/gaint_panda.bmp') 799 | imgGray = color.rgb2gray(img) 800 | M, N = imgGray.shape 801 | missing_rate = 0.9 802 | 803 | sparse_img = imgGray * np.round(np.random.rand(M, N) + 0.5 - missing_rate) 804 | io.imshow(sparse_img) 805 | plt.axis('off') 806 | plt.show() 807 | ``` 808 | 809 | ```python 810 | lmbda = 1e-3 * M * N 811 | eta = 100 * lmbda 812 | maxiter = 100 813 | vec_hat = circ_nnm(imgGray.reshape(M * N, order = 'F'), 814 | sparse_img.reshape(M * N, order = 'F'), 815 | lmbda, eta, maxiter) 816 | 817 | vec_hat[vec_hat < 0] = 0 818 | vec_hat[vec_hat > 1] = 1 819 | io.imshow(vec_hat.reshape([M, N], order = 'F')) 820 | plt.axis('off') 821 | plt.imsave('gaint_panda_gray_recovery_90_circ_nnm.png', 822 | vec_hat.reshape([M, N], order = 'F'), cmap = plt.cm.gray) 823 | plt.show() 824 | ``` 825 | 826 | **例.** 使用一维低秩拉普拉斯卷积模型对车速时间序列进行重构。 827 | 828 | ```python 829 | import numpy as np 830 | 831 | def compute_mape(var, var_hat): 832 | return np.sum(np.abs(var - var_hat) / var) / var.shape[0] 833 | 834 | def compute_rmse(var, var_hat): 835 | return np.sqrt(np.sum((var - var_hat) ** 2) / var.shape[0]) 836 | 837 | def laplacian(n, tau): 838 | ell = np.zeros(n) 839 | ell[0] = 2 * tau 840 | for k in range(tau): 841 | ell[k + 1] = -1 842 | ell[-k - 1] = -1 843 | return ell 844 | 845 | def prox(z, w, lmbda, denominator): 846 | T = z.shape[0] 847 | temp = np.fft.fft(lmbda * z - w) / denominator 848 | temp1 = np.abs(temp) - T / lmbda 849 | temp1[temp1 <= 0] = 0 850 | return np.fft.ifft(temp / np.abs(temp) * temp1).real 851 | 852 | def update_z(y_train, pos_train, x, w, lmbda, eta): 853 | z = x + w / lmbda 854 | z[pos_train] = (lmbda / (lmbda + eta) * z[pos_train] 855 | + eta / (lmbda + eta) * y_train) 856 | return z 857 | 858 | def update_w(x, z, w, lmbda): 859 | return w + lmbda * (x - z) 860 | 861 | def LCR(y_true, y, lmbda, gamma, tau, maxiter = 50): 862 | eta = 100 * lmbda 863 | T = y.shape 864 | pos_train = np.where(y != 0) 865 | y_train = y[pos_train] 866 | pos_test = np.where((y_true != 0) & (y == 0)) 867 | y_test = y_true[pos_test] 868 | z = y.copy() 869 | w = y.copy() 870 | denominator = lmbda + gamma * np.fft.fft(laplacian(T, tau)) ** 2 871 | del y_true, y 872 | show_iter = 10 873 | for it in range(maxiter): 874 | x = prox(z, w, lmbda, denominator) 875 | z = update_z(y_train, pos_train, x, w, lmbda, eta) 876 | w = update_w(x, z, w, lmbda) 877 | if (it + 1) % show_iter == 0: 878 | print(it + 1) 879 | print(compute_mape(y_test, x[pos_test])) 880 | print(compute_rmse(y_test, x[pos_test])) 881 | print() 882 | return x 883 | ``` 884 | 885 | ```python 886 | import numpy as np 887 | np.random.seed(1) 888 | import time 889 | 890 | missing_rate = 0.9 891 | print('Missing rate = {}'.format(missing_rate)) 892 | 893 | dense_vec = np.load('sample_speed_time_series.npz')['arr_0'] 894 | T = dense_vec.shape[0] 895 | sparse_vec = dense_vec * np.round(np.random.rand(T) + 0.5 - missing_rate) 896 | 897 | import time 898 | start = time.time() 899 | lmbda = 5e-3 * T 900 | gamma = 2 * lmbda 901 | tau = 2 902 | maxiter = 100 903 | x = LCR(dense_vec, sparse_vec, lmbda, gamma, tau, maxiter) 904 | end = time.time() 905 | print('Running time: %d seconds.'%(end - start)) 906 | ``` 907 | 908 | ```python 909 | import matplotlib.pyplot as plt 910 | plt.rcParams.update({'font.size': 13}) 911 | 912 | fig = plt.figure(figsize = (7.5, 2.2)) 913 | ax = fig.add_subplot(111) 914 | plt.plot(dense_vec[: T], 'dodgerblue', linewidth = 2) 915 | plt.xlabel('Time') 916 | plt.ylabel('Speed (mph)') 917 | plt.xlim([0, T]) 918 | plt.ylim([54, 65]) 919 | plt.xticks(np.arange(0, T + 1, 24)) 920 | plt.yticks(np.arange(54, 66, 2)) 921 | plt.grid(linestyle = '-.', linewidth = 0.5) 922 | ax.tick_params(direction = 'in') 923 | 924 | plt.savefig('freeway_traffic_speed_obs.pdf', bbox_inches = "tight") 925 | plt.show() 926 | 927 | import matplotlib.pyplot as plt 928 | plt.rcParams.update({'font.size': 13}) 929 | 930 | fig = plt.figure(figsize = (7.5, 2.2)) 931 | ax = fig.add_subplot(111) 932 | plt.plot(np.arange(0, T), sparse_vec[: T], 'o', 933 | markeredgecolor = 'darkblue', alpha = missing_rate, 934 | markerfacecolor = 'deepskyblue', markersize = 10) 935 | plt.xlabel('Time') 936 | plt.ylabel('Speed (mph)') 937 | plt.xlim([0, T]) 938 | plt.ylim([54, 65]) 939 | plt.xticks(np.arange(0, T + 1, 24)) 940 | plt.yticks(np.arange(54, 66, 2)) 941 | plt.grid(linestyle = '-.', linewidth = 0.5) 942 | ax.tick_params(direction = 'in') 943 | 944 | plt.savefig('freeway_traffic_speed_partial_obs.pdf', bbox_inches = "tight") 945 | plt.show() 946 | 947 | import matplotlib.pyplot as plt 948 | plt.rcParams.update({'font.size': 13}) 949 | 950 | fig = plt.figure(figsize = (7.5, 2.2)) 951 | ax = fig.add_subplot(111) 952 | plt.plot(np.arange(0, T), sparse_vec[: T], 'o', 953 | markeredgecolor = 'darkblue', alpha = missing_rate, 954 | markerfacecolor = 'deepskyblue', markersize = 10) 955 | plt.plot(x[: T], 'red', linewidth = 4) 956 | plt.xlabel('Time') 957 | plt.ylabel('Speed (mph)') 958 | plt.xlim([0, T]) 959 | plt.ylim([54, 65]) 960 | plt.xticks(np.arange(0, T + 1, 24)) 961 | plt.yticks(np.arange(54, 66, 2)) 962 | plt.grid(linestyle = '-.', linewidth = 0.5) 963 | ax.tick_params(direction = 'in') 964 | 965 | plt.savefig('freeway_traffic_speed_reconstructed_lap_conv.pdf', 966 | bbox_inches = "tight") 967 | plt.show() 968 | 969 | import matplotlib.pyplot as plt 970 | plt.rcParams.update({'font.size': 13}) 971 | 972 | fig = plt.figure(figsize = (7.5, 2.2)) 973 | ax = fig.add_subplot(111) 974 | plt.plot(dense_vec[: T], 'dodgerblue', linewidth = 1.5) 975 | plt.plot(np.arange(0, T), sparse_vec[: T], 'o', 976 | markeredgecolor = 'darkblue', alpha = missing_rate, 977 | markerfacecolor = 'deepskyblue', markersize = 10) 978 | plt.plot(x[: T], 'red', linewidth = 4) 979 | plt.xlabel('Time') 980 | plt.ylabel('Speed (mph)') 981 | plt.xlim([0, T]) 982 | plt.ylim([54, 65]) 983 | plt.xticks(np.arange(0, T + 1, 24)) 984 | plt.yticks(np.arange(54, 66, 2)) 985 | plt.grid(linestyle = '-.', linewidth = 0.5) 986 | ax.tick_params(direction = 'in') 987 | 988 | plt.savefig('freeway_traffic_speed_reconstructed_vs_true_lap_conv.pdf', 989 | bbox_inches = "tight") 990 | plt.show() 991 | ``` 992 | 993 | **例.** 使用一维低秩拉普拉斯卷积模型对灰度图像进行复原。 994 | 995 | ```python 996 | import numpy as np 997 | 998 | def compute_rse(var, var_hat): 999 | return np.linalg.norm(var - var_hat, 2) / np.linalg.norm(var, 2) 1000 | 1001 | def laplacian(n, tau): 1002 | ell = np.zeros(n) 1003 | ell[0] = 2 * tau 1004 | for k in range(tau): 1005 | ell[k + 1] = -1 1006 | ell[-k - 1] = -1 1007 | return ell 1008 | 1009 | def prox(z, w, lmbda, denominator): 1010 | T = z.shape[0] 1011 | temp = np.fft.fft(lmbda * z - w) / denominator 1012 | temp1 = 1 - T / (lmbda * np.abs(temp)) 1013 | temp1[temp1 <= 0] = 0 1014 | return np.fft.ifft(temp * temp1).real 1015 | 1016 | def update_z(y_train, pos_train, x, w, lmbda, eta): 1017 | z = x + w / lmbda 1018 | z[pos_train] = (lmbda / (lmbda + eta) * z[pos_train] 1019 | + eta / (lmbda + eta) * y_train) 1020 | return z 1021 | 1022 | def update_w(x, z, w, lmbda): 1023 | return w + lmbda * (x - z) 1024 | 1025 | def lap_conv_1d(y_true, y, gamma, lmbda, eta, tau, maxiter = 50): 1026 | T = y.shape 1027 | pos_train = np.where(y != 0) 1028 | y_train = y[pos_train] 1029 | pos_test = np.where((y_true != 0) & (y == 0)) 1030 | y_test = y_true[pos_test] 1031 | z = y.copy() 1032 | w = y.copy() 1033 | denominator = lmbda + gamma * np.fft.fft(laplacian(T, tau)) ** 2 1034 | del y_true, y 1035 | show_iter = 10 1036 | for it in range(maxiter): 1037 | x = prox(z, w, lmbda, denominator) 1038 | z = update_z(y_train, pos_train, x, w, lmbda, eta) 1039 | w = update_w(x, z, w, lmbda) 1040 | if (it + 1) % show_iter == 0: 1041 | print(it + 1) 1042 | print(compute_rse(y_test, x[pos_test])) 1043 | print() 1044 | return x 1045 | ``` 1046 | 1047 | ```python 1048 | import numpy as np 1049 | np.random.seed(1) 1050 | import matplotlib.pyplot as plt 1051 | from skimage import color 1052 | from skimage import io 1053 | 1054 | img = io.imread('data/gaint_panda.bmp') 1055 | imgGray = color.rgb2gray(img) 1056 | M, N = imgGray.shape 1057 | missing_rate = 0.9 1058 | 1059 | sparse_img = imgGray * np.round(np.random.rand(M, N) + 0.5 - missing_rate) 1060 | io.imshow(sparse_img) 1061 | plt.axis('off') 1062 | plt.show() 1063 | ``` 1064 | 1065 | ```python 1066 | lmbda = 5e-3 * M * N 1067 | gamma = 1 * lmbda 1068 | eta = 100 * lmbda 1069 | tau = 1 1070 | maxiter = 100 1071 | vec_hat = lap_conv_1d(imgGray.reshape(M * N, order = 'F'), 1072 | sparse_img.reshape(M * N, order = 'F'), 1073 | gamma, lmbda, eta, tau, maxiter) 1074 | 1075 | vec_hat[vec_hat < 0] = 0 1076 | vec_hat[vec_hat > 1] = 1 1077 | io.imshow(vec_hat.reshape([M, N], order = 'F')) 1078 | plt.axis('off') 1079 | plt.imsave('gaint_panda_gray_recovery_90_lap_conv_1d.png', 1080 | vec_hat.reshape([M, N], order = 'F'), cmap = plt.cm.gray) 1081 | plt.show() 1082 | ``` 1083 | 1084 | **例.** 使用二维低秩拉普拉斯卷积模型对灰度图像进行复原。 1085 | 1086 | ```python 1087 | import numpy as np 1088 | 1089 | def compute_rse(var, var_hat): 1090 | return np.linalg.norm(var - var_hat, 2) / np.linalg.norm(var, 2) 1091 | 1092 | def laplacian(T, tau): 1093 | ell = np.zeros(T) 1094 | ell[0] = 2 * tau 1095 | for k in range(tau): 1096 | ell[k + 1] = -1 1097 | ell[-k - 1] = -1 1098 | return ell 1099 | 1100 | def prox_2d(z, w, lmbda, denominator): 1101 | M, N = z.shape 1102 | temp = np.fft.fft2(lmbda * z - w) / denominator 1103 | temp1 = 1 - M * N / (lmbda * np.abs(temp)) 1104 | temp1[temp1 <= 0] = 0 1105 | return np.fft.ifft2(temp * temp1).real 1106 | 1107 | def update_z(y_train, pos_train, x, w, lmbda, eta): 1108 | z = x + w / lmbda 1109 | z[pos_train] = (lmbda / (lmbda + eta) * z[pos_train] 1110 | + eta / (lmbda + eta) * y_train) 1111 | return z 1112 | 1113 | def update_w(x, z, w, lmbda): 1114 | return w + lmbda * (x - z) 1115 | 1116 | def laplacian_conv_2d(y_true, y, lmbda, gamma, eta, tau, maxiter = 50): 1117 | M, N = y.shape 1118 | pos_train = np.where(y != 0) 1119 | y_train = y[pos_train] 1120 | pos_test = np.where((y_true != 0) & (y == 0)) 1121 | y_test = y_true[pos_test] 1122 | z = y.copy() 1123 | w = y.copy() 1124 | ell_1 = laplacian(M, tau) 1125 | ell_2 = laplacian(N, tau) 1126 | denominator = lmbda + gamma * np.fft.fft2(np.outer(ell_1, ell_2)) ** 2 1127 | del y_true, y 1128 | show_iter = 10 1129 | for it in range(maxiter): 1130 | x = prox_2d(z, w, lmbda, denominator) 1131 | z = update_z(y_train, pos_train, x, w, lmbda, eta) 1132 | w = update_w(x, z, w, lmbda) 1133 | if (it + 1) % show_iter == 0: 1134 | print(it + 1) 1135 | print(compute_rse(y_test, x[pos_test])) 1136 | print() 1137 | return x 1138 | ``` 1139 | 1140 | ```python 1141 | import numpy as np 1142 | np.random.seed(1) 1143 | import matplotlib.pyplot as plt 1144 | from skimage import color 1145 | from skimage import io 1146 | 1147 | img = io.imread('data/gaint_panda.bmp') 1148 | imgGray = color.rgb2gray(img) 1149 | M, N = imgGray.shape 1150 | missing_rate = 0.9 1151 | 1152 | sparse_img = imgGray * np.round(np.random.rand(M, N) + 0.5 - missing_rate) 1153 | io.imshow(sparse_img) 1154 | plt.axis('off') 1155 | plt.imsave('gaint_panda_gray_missing_rate_90.png', 1156 | sparse_img, cmap = plt.cm.gray) 1157 | plt.show() 1158 | ``` 1159 | 1160 | ```python 1161 | lmbda = 5e-3 * M * N 1162 | gamma = 1 * lmbda 1163 | eta = 100 * lmbda 1164 | tau = 2 1165 | maxiter = 100 1166 | mat_hat = laplacian_conv_2d(imgGray, sparse_img, lmbda, gamma, eta, tau, maxiter) 1167 | 1168 | mat_hat[mat_hat < 0] = 0 1169 | mat_hat[mat_hat > 1] = 1 1170 | io.imshow(mat_hat) 1171 | plt.axis('off') 1172 | plt.imsave('gaint_panda_gray_recovery_90_gamm1_tau{}.png'.format(tau), 1173 | mat_hat, cmap = plt.cm.gray) 1174 | plt.show() 1175 | ``` 1176 | 1177 | **例.** 计算延迟嵌入矩阵的奇异值分解、还原向量**x**。 1178 | 1179 | ```python 1180 | import numpy as np 1181 | 1182 | def delay_embedding(vec, kernel_size): 1183 | n = vec.shape[0] 1184 | mat = np.zeros((n, kernel_size)) 1185 | mat[:, 0] = vec 1186 | for k in range(1, kernel_size): 1187 | mat[:, k] = np.append(vec[k :], vec[: k], axis = 0) 1188 | return mat 1189 | 1190 | vec = np.array([0, 1, 2, 3, 4]) 1191 | T = vec.shape[0] 1192 | kernel_size = 3 1193 | mat = delay_embedding(vec, kernel_size) 1194 | 1195 | u, s, v = np.linalg.svd(mat, full_matrices = False) 1196 | x_hat = np.zeros(T) 1197 | for r in range(kernel_size): 1198 | fu = np.fft.fft(u[:, r]) 1199 | fv = np.fft.fft(np.append(v[r, :], np.zeros(T - kernel_size), axis = 0)) 1200 | x_hat += s[r] * np.fft.ifft(fu * fv).real 1201 | x_hat = x_hat / kernel_size 1202 | print(x_hat) 1203 | ``` 1204 | 1205 | 或者 1206 | ```python 1207 | import numpy as np 1208 | 1209 | def CircularConv(x, kernel): 1210 | m = x.shape[0] 1211 | n = kernel.shape[0] 1212 | vec = np.zeros(m) 1213 | for t in range(m): 1214 | temp = 0 1215 | for k in range(n): 1216 | temp += x[t - k] * kernel[k] 1217 | vec[t] = temp 1218 | return vec 1219 | 1220 | def DelayEmbedding(vec, kernel_size): 1221 | n = vec.shape[0] 1222 | mat = np.zeros((n, kernel_size)) 1223 | mat[:, 0] = vec 1224 | for k in range(1, kernel_size): 1225 | mat[:, k] = np.append(vec[k :], vec[: k], axis = 0) 1226 | return mat 1227 | 1228 | x = np.array([0, 1, 2, 3, 4]) 1229 | kernel_size = 3 1230 | mat = DelayEmbedding(x, kernel_size) 1231 | u, s, v = np.linalg.svd(mat, full_matrices = False) 1232 | temp1 = s[0] * CircularConv(u[:, 0], v[0, :]) / 3 1233 | print(temp1) 1234 | print() 1235 | temp2 = s[1] * CircularConv(u[:, 1], v[1, :]) / 3 1236 | print(temp2) 1237 | print() 1238 | temp3 = s[2] * CircularConv(u[:, 2], v[2, :]) / 3 1239 | print(temp3) 1240 | print() 1241 | print(temp1 + temp2 + temp3) 1242 | print() 1243 | ``` 1244 | 1245 |

核范数最小化问题

1246 |

^{▴ 回到顶部}

1247 | 1248 | 1249 | ```python 1250 | import numpy as np 1251 | 1252 | def circ_mat(vec): 1253 | n = vec.shape[0] 1254 | mat = np.zeros((n, n)) 1255 | mat[:, 0] = vec 1256 | for k in range(1, n): 1257 | mat[:, k] = np.append(vec[n - k :], vec[: n - k], axis = 0) 1258 | return mat 1259 | 1260 | def inv_circ_mat(mat): 1261 | n = mat.shape[0] 1262 | vec = mat[:, 0] 1263 | for k in range(1, n): 1264 | vec += np.append(mat[k :, k], mat[: k, k], axis = 0) 1265 | return vec / n 1266 | 1267 | z = np.array([0, 1, 2, 3, 4]) 1268 | T = z.shape[0] 1269 | lmbda = 2 1270 | mat = circ_mat(z) 1271 | u, s, v = np.linalg.svd(mat, full_matrices = False) 1272 | s = s - T / lmbda 1273 | s[s < 0] = 0 1274 | temp = u @ np.diag(s) @ v 1275 | print('The result of singular value thresholding:') 1276 | print(temp) 1277 | print() 1278 | print('The inverse operator of the matrix:') 1279 | x = inv_circ_mat(temp) 1280 | print(x) 1281 | print() 1282 | print('The objective function is: ') 1283 | print(np.sum(np.abs(np.fft.fft(x))) 1284 | + 0.5 * lmbda * np.linalg.norm(x - z, 2) ** 2) 1285 | print() 1286 | ``` 1287 | 1288 | ```python 1289 | def conv_mat(vec, kernel_size): 1290 | n = vec.shape[0] 1291 | mat = np.zeros((n, kernel_size)) 1292 | mat[:, 0] = vec 1293 | for k in range(1, kernel_size): 1294 | mat[:, k] = np.append(vec[n - k :], vec[: n - k], axis = 0) 1295 | return mat 1296 | 1297 | def inv_conv_mat(mat): 1298 | tau = mat.shape[1] 1299 | vec = mat[:, 0] 1300 | for k in range(1, tau): 1301 | vec += np.append(mat[k :, k], mat[: k, k], axis = 0) 1302 | return vec / tau 1303 | ``` 1304 | 1305 | -------------------------------------------------------------------------------- /data/gaint_panda.bmp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xinychen/tensor-book/66b126528b0f33cd75d3c9952b4692a80d8350af/data/gaint_panda.bmp -------------------------------------------------------------------------------- /data/gaint_panda.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xinychen/tensor-book/66b126528b0f33cd75d3c9952b4692a80d8350af/data/gaint_panda.jpg -------------------------------------------------------------------------------- /data/gaint_panda_gray.bmp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xinychen/tensor-book/66b126528b0f33cd75d3c9952b4692a80d8350af/data/gaint_panda_gray.bmp -------------------------------------------------------------------------------- /data/gaint_panda_rgb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xinychen/tensor-book/66b126528b0f33cd75d3c9952b4692a80d8350af/data/gaint_panda_rgb.jpg -------------------------------------------------------------------------------- /data/sample_speed_time_series.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xinychen/tensor-book/66b126528b0f33cd75d3c9952b4692a80d8350af/data/sample_speed_time_series.npz --------------------------------------------------------------------------------