├── fig1.jpg ├── fig2.jpg ├── get_x_sq.m ├── columns.m ├── column_selection.m ├── trajectory_3d.m └── README.md /fig1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geelenr/quad_manifold/HEAD/fig1.jpg -------------------------------------------------------------------------------- /fig2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geelenr/quad_manifold/HEAD/fig2.jpg -------------------------------------------------------------------------------- /get_x_sq.m: -------------------------------------------------------------------------------- 1 | function X2 = get_x_sq(X) 2 | % takes data in X where each column is one variable and generates matrix X, 3 | % where each column contains a quadratic term of the variables in X - this 4 | % is different from a Kronecker product because there is no redundancy 5 | % 6 | % INPUT 7 | % X N-by-w data matrix where N is number of data points, w is number of 8 | % variables 9 | % 10 | % OUTPUT 11 | % X2 N-by-(w*(w+1)/2) matrix of quadratic terms of data in X 12 | % 13 | % AUTHOR 14 | % Elizabeth Qian (elizqian@mit.edu) 17 June 2019 15 | 16 | w = size(X,2); 17 | Xi = cell(w,1); 18 | for j = 1:w 19 | Xi{j} = repmat(X(:,j),1,w-j+1).*X(:,j:w); 20 | end 21 | X2 = cat(2,Xi{:}); 22 | -------------------------------------------------------------------------------- /columns.m: -------------------------------------------------------------------------------- 1 | function cols = columns(W,Eps) 2 | 3 | r2 = size(W,1); 4 | n = size(Eps,1); 5 | max_lambdas=100; 6 | lambda_fac=0.99; 7 | alpha0=1.e-6; % initial alpha, not too important 8 | 9 | % evaluate gradient of the fitting part at Vbar=0. This determines the 10 | % starting value of lambda 11 | G = W*Eps'; 12 | max2norm=0.0; 13 | for i=1:r2 14 | max2norm=max(max2norm, norm(G(i,:),2)); 15 | end 16 | fprintf(1,' max ||W*Eps^T|| row norm is %10.5e\n', max2norm); 17 | 18 | 19 | VbarT=zeros(r2,n); % initialize VbarT to zero the first iteration 20 | lambda = max2norm; % for this lambda, the solution is VbarT=0, so 21 | % choose smaller lambda values than this in the 22 | % loop below; 23 | 24 | for i=1:max_lambdas 25 | lambda=lambda_fac*lambda; 26 | [columns VbarT alpha_end flag]= column_selection(W',Eps',VbarT,lambda,alpha0); 27 | if (flag==1) 28 | fprintf(1,' ERROR in column_selection for lambda = %10.5e\n', lambda); 29 | break; 30 | end 31 | fprintf(1,'\n\n **** lambda=%12.5e, columns selected: %d\n', ... 32 | lambda, length(columns)); 33 | fprintf(1,' Columns selected:\n'); 34 | columns 35 | alpha0=2*alpha_end; 36 | if (length(columns)>1) 37 | break; % quit if we are selecting most of the columns 38 | end 39 | end 40 | 41 | cols = columns; 42 | 43 | end -------------------------------------------------------------------------------- /column_selection.m: -------------------------------------------------------------------------------- 1 | function [columns VbarT alpha flag]= column_selection(WT,EpsT,VbarT,lambda,alpha0) 2 | 3 | % choose regularization parameter lambda, initial steplength alpha0. 4 | 5 | epstol=1.e-6; % terminate when relative change in F drops below epstol 6 | convergence=0; % convergence flag 7 | itmax=1000; % maximum number of iterations 8 | btmax=50; % max number of backtracking steps 9 | alpha_min=1.e-20; % crash out when alpha drops below this threshold 10 | flag=0; % flag=0 means no error on exit 11 | 12 | r2 = size(WT,2); 13 | n = size(EpsT,2); 14 | fprintf(1,' r^2=%d, n=%d\n', r2,n); 15 | 16 | % VbarT = zeros(r2,n); 17 | VnewT=VbarT; 18 | 19 | % evaluate initial function value 20 | Temp=WT*VbarT-EpsT; 21 | Fval=0.5*trace(Temp'*Temp); 22 | for i=1:r2 23 | Fval=Fval+lambda*norm(VbarT(i,:)); 24 | end 25 | fprintf(1,' initial F=%12.7e\n', Fval); 26 | alpha=alpha0; 27 | 28 | for iter=1:itmax 29 | G = WT*VbarT-EpsT; 30 | G = WT'*G; 31 | 32 | for ia=1:btmax 33 | R = VbarT-alpha*G;; 34 | zero_rows=0; 35 | for i=1:r2 36 | nRi=norm(R(i,:),2); 37 | if (nRi<=alpha*lambda) 38 | VnewT(i,:) = zeros(1,n); 39 | zero_rows=zero_rows+1; 40 | % fprintf(1,' %2d ||V(i,:)||=%10.5e ||Vnew(i,:)||=%10.5e\n', i, norm(VbarT(i,:)), norm(VnewT(i,:))); 41 | else 42 | VnewT(i,:) = (1.0-(alpha*lambda)/nRi)*R(i,:); 43 | % fprintf(1,' %2d ||V(i,:)||=%10.5e ||Vnew(i,:)||=%10.5e\n', i, norm(VbarT(i,:)), norm(VnewT(i,:))); 44 | end 45 | end 46 | % fprintf(1,' ||V-Vnew||=%10.4e\n', norm(VbarT-VnewT,'fro')); 47 | % evaluate the function at the proposed new iterate 48 | Temp=WT*VnewT-EpsT; 49 | Fval_new=0.5*trace(Temp'*Temp); 50 | for i=1:r2 51 | Fval_new=Fval_new+lambda*norm(VnewT(i,:),2); 52 | end 53 | fprintf(1,' alpha=%10.3e, F-Fnew=%12.7e, zero rows=%d\n', ... 54 | alpha, Fval-Fval_new, zero_rows); 55 | if (Fval_new < Fval) 56 | fprintf(1,' success with alpha=%10.3e\n', alpha); 57 | break; 58 | else 59 | alpha=0.5*alpha; 60 | if (alpha1.e-6) 101 | % q=q+1; 102 | % Scheck=[Scheck WT(:,i)]; 103 | columns = [columns, i]; 104 | end 105 | end 106 | % VbarT= Scheck\EpsT; 107 | end 108 | 109 | end -------------------------------------------------------------------------------- /trajectory_3d.m: -------------------------------------------------------------------------------- 1 | clc; clear; close all; 2 | 3 | %% problem setup 4 | 5 | % model parameters 6 | s = @(t) [cos(t); sin(t); 0.5*cos(2*t)]; % trajectory 7 | nsamples = 100; % number of data points 8 | r = 2; % reduced basis dimension 9 | sref = s(0); % reference state 10 | lambda = 0; % regularization 11 | 12 | % construct trajectory 13 | t_end = 2*pi; 14 | t = 0:t_end/(nsamples-1):t_end; 15 | trajectory = s(t); 16 | 17 | %% left singular vectors 18 | 19 | [U,S,~] = svd(trajectory-sref,'econ'); % svd 20 | Vr = U(:,1:r); % POD basis matrix 21 | 22 | %% linear dimensionality reduction 23 | 24 | % compute approximated trajectory with linear subspace approach 25 | srec = sref + Vr*Vr'*(trajectory-sref); 26 | 27 | %% data-driven quadratic solution-manifolds 28 | 29 | shat = Vr'*(trajectory-sref); 30 | err = trajectory-sref-Vr*shat; % linear projection error 31 | W = get_x_sq(shat')'; 32 | Wtilde = W([1,2,3],:); % column selection (if applicable) 33 | [q,~] = size(err); 34 | [p,~] = size(Wtilde); 35 | Aplus = [Wtilde'; sqrt(lambda)*eye(p)]; 36 | bplus = [err'; zeros(p,q)]; 37 | Vbar = (Aplus\bplus)'; 38 | srec_quad = sref + Vr*shat + Vbar*Wtilde; 39 | 40 | %% compute error 41 | 42 | err = norm(srec -trajectory, 'fro')/norm(trajectory, 'fro'); 43 | err_quad = norm(srec_quad-trajectory, 'fro')/norm(trajectory, 'fro'); 44 | fprintf("Reconstruction error linear model: \t%s\n", err); 45 | fprintf("Reconstruction error quadratic model: \t%s\n", err_quad); 46 | 47 | %% plotting 48 | 49 | % linear 50 | figure('Position', [10 10 300 350]); 51 | h = scatter3(trajectory(1,:), trajectory(2,:), ... 52 | trajectory(3,:),'filled','k'); hold on; 53 | set(h,'MarkerEdgeAlpha',0.5,'MarkerFaceAlpha',0.7,'SizeData',8) 54 | plot3(srec(1,:), srec(2,:), srec(3,:),'.','color','#006eb8'); 55 | xlabel('$s_1$','interpreter','latex'); 56 | ylabel('$s_2$','interpreter','latex'); 57 | zlabel('$s_3$','interpreter','latex'); 58 | [X,Y] = meshgrid(-2.5:0.1:2.5,-2.5:0.1:2.5); 59 | Z = ones(size(X)); sz = size(X,1)*size(X,2); 60 | surf1 = [reshape(X,1,sz); reshape(Y,1,sz); reshape(Z,1,sz)]; 61 | surf1_rec = sref + Vr*Vr'*(surf1-sref); 62 | surf1_rec_mat1 = reshape(surf1_rec(1,:),size(X,1),size(X,2)); 63 | surf1_rec_mat2 = reshape(surf1_rec(2,:),size(Y,1),size(Y,2)); 64 | surf1_rec_mat3 = reshape(surf1_rec(3,:),size(Z,1),size(Z,2)); 65 | mesh(surf1_rec_mat1,surf1_rec_mat2,surf1_rec_mat3,'FaceAlpha','0.05', ... 66 | 'EdgeColor','#006eb8','FaceColor','#006eb8','LineStyle',':'); 67 | axis([-1.25 1.25 -1.25 1.25 -1 1]); view(-50, 25); 68 | legend('trajectory $\mathbf{s}(t)$','approximated trajectory', ... 69 | 'linear manifold','interpreter','latex','location','southoutside'); 70 | legend boxoff; grid on; set(gca,'GridLineStyle','--'); 71 | 72 | % quadratic 73 | figure('Position', [10 10 300 350]); 74 | h = scatter3(trajectory(1,:), trajectory(2,:), ... 75 | trajectory(3,:),'filled','k'); hold on; 76 | set(h,'MarkerEdgeAlpha',0.5,'MarkerFaceAlpha',0.7,'SizeData',8) 77 | plot3(srec_quad(1,:), srec_quad(2,:), srec_quad(3,:),'.','color','#ee2967'); 78 | xlabel('$s_1$','interpreter','latex'); 79 | ylabel('$s_2$','interpreter','latex'); 80 | zlabel('$s_3$','interpreter','latex'); 81 | W = get_x_sq((Vr'*(surf1-sref))')'; 82 | Wtilde = W([1,2,3],:); 83 | surf2_rec = sref + Vr*Vr'*(surf1-sref) + Vbar*Wtilde; 84 | surf2_rec_mat1 = reshape(surf2_rec(1,:),size(X,1),size(X,2)); 85 | surf2_rec_mat2 = reshape(surf2_rec(2,:),size(Y,1),size(Y,2)); 86 | surf2_rec_mat3 = reshape(surf2_rec(3,:),size(Z,1),size(Z,2)); 87 | mesh(surf2_rec_mat1,surf2_rec_mat2,surf2_rec_mat3,'FaceAlpha','0.1', ... 88 | 'EdgeColor','#ee2967','FaceColor','#ee2967','LineStyle',':'); 89 | axis([-1.25 1.25 -1.25 1.25 -1 1]); view(-50, 25); 90 | grid on; 91 | set(gca,'GridLineStyle','--'); 92 | legend('trajectory $\mathbf{s}(t)$','approximated trajectory', ... 93 | 'quadratic manifold', 'interpreter','latex','location','southoutside'); 94 | legend boxoff; grid on; set(gca,'GridLineStyle','--'); 95 | 96 | % latex stuff 97 | set(gca,'defaulttextinterpreter','latex'); 98 | set(groot,'defaultAxesTickLabelInterpreter','latex', ... 99 | 'defaultLegendInterpreter','latex'); -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Data-driven quadratic solution-manifolds 2 | 3 | This repository contains the source code for the numerical models from Section 2 of the preprint *Operator inference for non-intrusive model reduction with quadratic manifolds* ([arXiv:2205.02304](https://doi.org/10.48550/arXiv.2205.02304)) by Rudy Geelen, Stephen Wright and Karen Willcox. 4 | 5 | ## Abstract 6 | 7 | This paper proposes a novel approach for learning a data-driven quadratic manifold from high-dimensional data, then employing this quadratic manifold to derive efficient physics-based reduced-order models. The key ingredient of the approach is a polynomial mapping between high-dimensional states and a low-dimensional embedding. This mapping consists of two parts: a representation in a linear subspace (computed in this work using the proper orthogonal decomposition) and a quadratic component. The approach can be viewed as a form of data-driven closure modeling, since the quadratic component introduces directions into the approximation that lie in the orthogonal complement of the linear subspace, but without introducing any additional degrees of freedom to the low-dimensional representation. Combining the quadratic manifold approximation with the operator inference method for projection-based model reduction leads to a scalable non-intrusive approach for learning reduced-order models of dynamical systems. Applying the new approach to transport-dominated systems of partial differential equations illustrates the gains in efficiency that can be achieved over approximation in a linear subspace. 8 | 9 | ## Repository 10 | 11 | Our starting point is the quadratic approximation 12 | 13 | $$ \mathbf{s}\_j \approx \mathbf{s}_\text{ref} + \mathbf{V} \widehat{\mathbf{s}}_j + \overline{\mathbf{V}}(\widehat{\mathbf{s}}_j \otimes \widehat{\mathbf{s}}_j), 14 | $$ 15 | 16 | where $\mathbf{s}_j$ is the $j$-th high-dimensional state, $\widehat{\mathbf{s}}_j$ denotes its reduced state representation, $\mathbf{s}\_\text{ref}$ is a given reference state chosen in a problem-specific manner, $(\mathbf{V},\overline{\mathbf{V}})$ are the basis matrices representing the linear and quadratic components of the solution-manifold, and $\otimes$ denotes the Kronecker product. In the following it is assumed that $\mathbf{V}$ and $\widehat{\mathbf{s}}_j$ are computed using the well-known proper orthogonal decomposition (POD), and the operator $\overline{\mathbf{V}}$ is computed from the misfit of the linear dimensionality reduction with basis $\mathbf{V}$ as follows: 17 | 18 | $$ 19 | \overline{\mathbf{V}} = \underset{\overline{\mathbf{V}}}{\operatorname{argmin}} \sum_{j=1}^k \left\| \left\| \mathbf{s}\_j - \mathbf{s}_\text{ref} - \mathbf{V} \widehat{\mathbf{s}}_j - \overline{\mathbf{V}} ( \widehat{\mathbf{s}}_j \otimes \widehat{\mathbf{s}}_j ) \right\| \right\|_2^2 \in \mathbb{R}^{n \times r^2} 20 | $$ 21 | 22 | The following set of codes can be used in the numerical treatment of this approximation: 23 | 24 | - [`trajectory_3d.m`](./trajectory_3d.m) provides the basis for a simple implementation of the above approximation. It reproduces the numerical experiments from Section 2.4 and compares a conventional linear dimensionality reduction (using POD) with the proposed data-driven quadratic manifold approach. The following figures are generated when all columns are selected: 25 |
26 |

27 | 28 | 29 |

30 |

Comparison of the linear (left) and quadratic (right) manifold approaches for a three-dimensional trajectory.

31 |
32 | 33 | - [`columns.m`](./columns.m) is an outer loop script that encloses the column selection problem. The column selection algorithm is generally speaking not applied for a single value of the regularization parameter $\lambda$ but rather a range of distinct values (in decreasing order). Note that the number of columns selected tends to increase as lambda is decreased. 34 | 35 | - [`column_selection.m`](./column_selection.m) considers a basic proximal-gradient algorithm for solving the "sum of $\ell_2$" regularized problem, which is essentially the SpaRSA algorithm described by Wright and co-workers [here](https://doi.org/10.1109/TSP.2009.2016892). The "sum of $\ell_2$" regularizer induces sparsity in a similar way to the $\ell_1$ norm of a vector, except that instead of producing element-wise sparsity, it induces sparsity by groups. 36 | 37 | ## Citation 38 | 39 | If you find this repository useful, please consider citing our paper: 40 | ``` 41 | @article{geelen2022quad_manifolds, 42 | author = {Geelen, Rudy and Wright, Stephen and Willcox, Karen}, 43 | title = {Operator inference for non-intrusive model reduction with quadratic manifolds}, 44 | journal = {arXiv preprint arXiv:2205.02304}, 45 | year = {2022}, 46 | } 47 | ``` 48 | 49 | --- 50 | These codes were developed and tested in MATLAB R2020a. Please contact Rudy Geelen (rudy.geelen@austin.utexas.edu) if you have any questions or encounter issues. Stephen Wright contributed to the development of these codes. 51 | --------------------------------------------------------------------------------