├── .gitignore
├── LICENSE
├── Makefile
├── README.md
├── ai4sci
└── oml
│ ├── __init__.py
│ ├── _online_linear_model.py
│ └── _online_model.py
├── assets
├── linear_time_varying.png
├── linear_time_varying_controlled.png
├── lorenz_control.png
├── lorenz_control_controlled.png
├── lorenz_state.png
└── lorenz_state_controlled.png
├── demo
├── README.md
├── demo_linear_time_varying_system.ipynb
├── demo_lorenz.ipynb
└── requirements.txt
├── setup.py
└── tests
├── README.md
├── requirements.txt
├── test_online_linear_model.py
└── test_online_model.py
/.gitignore:
--------------------------------------------------------------------------------
1 | # Add any directories, files, or patterns you don't want to be tracked by version control
2 |
3 | # matlab
4 | # OSX / *nix default autosave extension
5 | *.m~
6 |
7 | # python
8 | # Byte-compiled / optimized / DLL files
9 | __pycache__/
10 | *.py[cod]
11 | *$py.class
12 |
13 | # Other files
14 | tmp*
15 | .DS_Store
16 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2021 Hao Zhang
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/Makefile:
--------------------------------------------------------------------------------
1 | env:
2 | python3 -m venv ~/my-env
3 | source ~/my-env/bin/activate
4 | file $(which python)
5 | python --version
6 | dev:
7 | python -m pip install -U pip setuptools wheel pip-tools isort black flake8 pylint mypy
8 |
9 | install:
10 | python -m pip install -e .
11 | requirements:
12 | python -m piptools compile --no-emit-index-url
13 | # https://code.visualstudio.com/docs/python/linting
14 | lint:
15 | python -m flake8 .
16 | python -m pylint .
17 | python -m mypy .
18 | test: lint
19 | cd tests
20 | python -m pip install -r requirements.txt
21 | python -m pytest
22 | format:
23 | python -m isort .
24 | python -m black .
25 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # oml
2 | [](https://github.com/haozhg/oml/blob/master/LICENSE)
3 | [](https://docs.python.org/3.8/)
4 | [](https://pypi.org/project/ai4sci.oml/)
5 | [](https://colab.research.google.com/drive/1dWeKuiEsVUjlNaKSFW6b7J-UyyFwov8C?usp=sharing)
6 | [](https://pepy.tech/project/ai4sci.oml)
7 | [](https://github.com/haozhg/oml/pulls)
8 | [](https://github.com/psf/black)
9 |
10 | AI4Science: Efficient data-driven Online Model Learning (OML) / system identification and control. The algorithm is proposed in this [paper](https://arxiv.org/abs/1707.02876.).
11 |
12 | To get started,
13 | ```
14 | pip install ai4sci.oml --upgrade
15 | ```
16 | This python package is based on the online dynamic mode decomposition algorithm, which is also available as a python package `pip install odmd --upgrade`, see [here](https://github.com/haozhg/odmd).
17 |
18 | ## Showcase: Lorenz system control
19 | [Lorenz system](https://en.wikipedia.org/wiki/Lorenz_system) is one of the most classical nonlinear dynamical systems. Here we show how the proposed algorithm can be used to controll that. For more details, see [demo](https://github.com/haozhg/oml/tree/master/demo).
20 |
21 | ### No control
22 | If there is no control, we can see the mysterious butterfly trajectory. It starts close to the bottom plane and enters into the butterfly wing region, then oscillates there.
23 |
24 |
25 | 
26 | 
27 |
28 |
29 | ### With control
30 | If we apply data-driven real-time closed loop control, it can be stabilized at an unstable fixed point (near the center of the butterfly wing).
31 |
32 |
33 | 
34 | 
35 |
36 |
37 | ## Highlights
38 | Here are some hightlights about this algorithm, and for more detail refer to this [paper](https://epubs.siam.org/doi/pdf/10.1137/18M1192329)
39 |
40 | - Efficient data-driven online linear/nonlinear model learning (system identification). Any nonlinear and/or time-varying system is locally linear, as long as the model is updated in real-time wrt to new measurements.
41 | - It finds the exact optimal solution (in the sense of least square error), without any approximation (unlike stochastic gradient descent).
42 | - It achieves theoretical optimal time and space complexity.
43 | - The time complexity (flops for one iteration) is O(n^2), where n is state dimension. This is much faster than standard algorithm O(n^2 * t), where t is the current time step (number of measurements). In online applications, t >> n and essentially will go to infinity.
44 | - The space complexity is O(n^2), which is far more efficient than standard algorithm O(n * t) (t >> n).
45 | - A weighting factor (in (0, 1]) can be used to place more weight on recent data, thus making the model more adaptive.
46 | - This local model can be used for short-horizon prediction and data-driven real-time closed loop control.
47 | - It has been successfully applied to flow separation control problem, and achived real-time closed loop control. See this [paper](https://doi.org/10.1017/jfm.2020.546) for details.
48 |
49 | ## Online model learning algorithm description
50 | This is a brief introduction to the algorithm. For full technical details, see this [paper](https://epubs.siam.org/doi/pdf/10.1137/18M1192329), and chapter 3 and chapter 7 of this [PhD thesis](http://arks.princeton.edu/ark:/88435/dsp0108612r49q).
51 |
52 | ### Unknown dynamical system
53 | Suppose we have a (discrete) nonlinear and/or time-varying [dynamical system](https://en.wikipedia.org/wiki/State-space_representation), and the state space representation is
54 | - x(t+1) = f(t, x(t), u(t))
55 | - y(t) = g(t, x(t), u(t))
56 |
57 | where t is (discrete) time, x(t) is state vector, u(t) is control (input) vector, y(t) is observation (output) vector. f(~, ~, ~) and g(~, ~, ~) are unknown vector-valued nonlinear functions.
58 |
59 | - It is assumed that we have measurements x(i), u(i), y(i) for i = 0,1,...t.
60 | - However, we do not know functions f and g.
61 | - We aim to learn a model for the unknown dynamical system from measurement data up to time t.
62 | - We want to the model to be updated efficiently in real-time as new measurement data becomes available.
63 |
64 | ### Online linear model learning
65 | We would like to learn an adaptive [linear model](https://en.wikipedia.org/wiki/State-space_representation)
66 | - x(t+1) = A x(t) + B u(t)
67 | - y(t) = C x(t) + D u(t)
68 |
69 | that fits/explains the observation optimally. By Taylor expansion approximation, any nonlinear and/or time-varying system is linear locally. There are many powerful tools for linear control, e.g, [Linear Quadratic Regulator](https://en.wikipedia.org/wiki/Linear%E2%80%93quadratic_regulator), [Kalman filter](https://en.wikipedia.org/wiki/Kalman_filter). However, to accurately approximate the original (unknown) dynamical system, we need to update this linear model efficiently in real-time whenever new measurement becomes available.
70 |
71 | This problem can be formulated as an optimization problem, and at each time step t we need to solve a related but slightly different optimization problem. The optimal algorithm is achived through efficient reformulation of the problem.
72 |
73 | - `ai4sci.oml.OnlineLinearModel` class implements the optimal algorithm.
74 |
75 | ### Online nonlinear model learning
76 | If we need to fit a nonlinear model to the observed data, this algorithm also applies in this case. Keep in mind that linear adaptive model is good approximation as long as it is updated in real-time. Also, the choice of nonlinear form can be tricky. Based on Taylor expansion, if we add higher order nonlinearity (e.g., quadratic, cubic), the approximation can be more accurate. However, given the learned nonlinear model, it is still not easy to apply control.
77 |
78 | In particular, we want to fit a nonlinear model of this form
79 | - x(t+1) = F * phi(x(t), u(t))
80 | - y(t) = G * psi(x(t), u(t))
81 |
82 | where phi(~, ~) and psi(~, ~) are known vector-valued nonlinear functions (e.g, quadratic) that we specify, F and G are unknown matrices of proper size.
83 |
84 | - We aim to learn F and G from measurement data.
85 | - Notice that this model form is general, and encompass many systems such as Lorenze attractor, Logistic map, Auto-regressive model, polynomial systems.
86 | - F and G can be updated efficiently in real-time when new data comes in.
87 |
88 | This can also be formulated as the same optimization problem, and the same efficient algorithm works in this case.
89 |
90 | - `ai4sci.oml.OnlineModel` class implements the optimal algorithm.
91 |
92 | ## Use
93 | ### Install
94 | From PyPi
95 | ```
96 | pip install ai4sci.oml --upgrade
97 | ```
98 |
99 | From source
100 | ```
101 | git clone https://github.com/haozhg/oml.git
102 | cd oml/
103 | pip install -e .
104 | ```
105 |
106 | ### Tests
107 | ```
108 | cd tests/
109 | python -m pytest .
110 | ```
111 |
112 | ### Demo
113 | See `./demo` for python notebook to demo the algorithm for data-driven real-time closed loop control.
114 | - `demo_lorenz.ipynb` shows control of [Lorenz attractor](https://en.wikipedia.org/wiki/Lorenz_system).
115 | - `demo_online_linear_model.ipynb` shows control of an unstable linear time-varying system.
116 |
117 | ## Authors:
118 | Hao Zhang
119 |
120 | ## Reference
121 | If you you used these algorithms or this python package in your work, please consider citing
122 |
123 | ```
124 | Zhang, Hao, Clarence W. Rowley, Eric A. Deem, and Louis N. Cattafesta.
125 | "Online dynamic mode decomposition for time-varying systems."
126 | SIAM Journal on Applied Dynamical Systems 18, no. 3 (2019): 1586-1609.
127 | ```
128 |
129 | BibTeX
130 | ```
131 | @article{zhang2019online,
132 | title={Online dynamic mode decomposition for time-varying systems},
133 | author={Zhang, Hao and Rowley, Clarence W and Deem, Eric A and Cattafesta, Louis N},
134 | journal={SIAM Journal on Applied Dynamical Systems},
135 | volume={18},
136 | number={3},
137 | pages={1586--1609},
138 | year={2019},
139 | publisher={SIAM}
140 | }
141 | ```
142 |
143 | ## Date created
144 | April 2017
145 |
146 | ## License
147 | MIT
148 |
149 | If you want to use this package, but find license permission an issue, pls contact me at `haozhang at alumni dot princeton dot edu`.
150 |
151 | ## Issues
152 | If there is any comment/suggestion, or if you find any bug, feel free to
153 | - create an issue [here](https://github.com/haozhg/oml/issues), and/or
154 | - fork this repo, make suggested changes, and create a pull request (merge from your fork to this repo). See [this](https://numpy.org/devdocs/dev/index.html#development-process-summary) as an example guidance for contribution and PRs.
--------------------------------------------------------------------------------
/ai4sci/oml/__init__.py:
--------------------------------------------------------------------------------
1 | from ._online_model import OnlineModel
2 | from ._online_linear_model import OnlineLinearModel
--------------------------------------------------------------------------------
/ai4sci/oml/_online_linear_model.py:
--------------------------------------------------------------------------------
1 | import logging
2 |
3 | import numpy as np
4 |
5 | from ._online_model import OnlineModel
6 |
7 | logger = logging.getLogger(__name__)
8 |
9 |
10 | class OnlineLinearModel:
11 | """
12 | # Unknown dynamical system
13 | Suppose we have a (discrete) nonlinear and/or time-varying [dynamical system](https://en.wikipedia.org/wiki/State-space_representation),
14 | and the state space representation is
15 | - x(t+1) = f(t, x(t), u(t))
16 | - y(t) = g(t, x(t), u(t))
17 |
18 | where t is (discrete) time, x(t) is state vector, u(t) is control (input) vector, y(t) is observation (output) vector.
19 | f(~, ~, ~) and g(~, ~, ~) are unknown vector-valued nonlinear functions.
20 |
21 | - It is assumed that we have measurements x(i), u(i), y(i) for i = 0,1,...t.
22 | - However, we do not know functions f and g.
23 | - We aim to learn a model for the unknown dynamical system from measurement data up to time t.
24 | - We want to the model to be updated efficiently in real-time as new measurement data becomes available.
25 |
26 | # Online linear model learning
27 | We would like to learn an adaptive [linear model](https://en.wikipedia.org/wiki/State-space_representation)
28 | - x(t+1) = A x(t) + B u(t)
29 | - y(t) = C x(t) + D u(t)
30 |
31 | that fits/explains the observation optimally. By Taylor expansion approximation, any nonlinear and/or
32 | time-varying system is linear locally. There are many powerful tools for linear control,
33 | e.g, [Linear Quadratic Regulator](https://en.wikipedia.org/wiki/Linear%E2%80%93quadratic_regulator),
34 | [Kalman filter](https://en.wikipedia.org/wiki/Kalman_filter).
35 | However, to accurately approximate the original (unknown) dynamical system, we need to update this linear
36 | model efficiently in real-time whenever new measurement becomes available.
37 |
38 | This problem can be formulated as an optimization problem, and at each time step t we need to solve a
39 | related but slightly different optimization problem. The optimal algorithm is achived through efficient
40 | reformulation of the problem.
41 | """
42 | def __init__(self, n: int, k: int, m: int = None, alpha: float = 0.9) -> None:
43 | """Online Linear Model
44 | Efficiently learn adaptive linear model from data in real-time
45 |
46 | Args:
47 | n (int): state dimension
48 | k (int): control (input) dimension
49 | m (int): observation (output) dimension
50 | alpha (float, optional): exponentiall weighting factor in system identification. Defaults to 0.9.
51 | """
52 | # input check
53 | assert isinstance(n, int) and n >= 1
54 | assert isinstance(k, int) and k >= 1
55 | assert m is None or (isinstance(m, int) and m >= 1)
56 | alpha = float(alpha)
57 | assert alpha > 0 and alpha <= 1
58 |
59 | # set parameters
60 | self._n = n
61 | self._k = k
62 | self._m = m
63 | self._alpha = alpha
64 |
65 | # additional parameters
66 | self._t = 0
67 | self._tmin = 2 * max(self._n, self._n + self._k)
68 | if self._m:
69 | self._tmin = max(self._tmin, 2 * self._m)
70 | self._ready = False
71 |
72 | # initialize model
73 | self._f = OnlineModel(n + k, n, alpha) # encodes A and B, model for xn=f(x, u)
74 | if self._m:
75 | logger.info("Learn x(t+1) = A * x(t) + B * u(t), y(t) = C * x(t) + D * u(t)")
76 | self._g = OnlineModel(n + k, m, alpha) # encodes C and D, model for y=g(x, u)
77 | else:
78 | logger.info("No output eqution, only learn x(t+1) = A * x(t) + B * u(t)")
79 |
80 | def update(self, x: np.ndarray, u: np.ndarray, xn: np.ndarray, y: np.ndarray =None) -> None:
81 | """Update model wrt new measurement (state x, control u, next state xn, optinal observation y)
82 |
83 | Args:
84 | x (np.ndarray): state x(t), 1D array, shape (n, )
85 | u (np.ndarray): control u(t), 1D array, shape (k, )
86 | xn (np.ndarray): new state x(t+1), 1D array, shape (n, )
87 | y (np.ndarray, optional): observation y(t), 1D array, shape (m, ). Defaults to None.
88 | """
89 | # input check
90 | assert x.shape[0] == self._n
91 | assert u.shape[0] == self._k
92 | assert xn.shape[0] == self._n
93 | if y is None:
94 | assert self._m is None
95 | if self._m:
96 | assert y.shape[0] == self._m
97 |
98 | # update f
99 | w = np.concatenate((x, u))
100 | self._f.update(w, xn)
101 |
102 | # update g if needed
103 | if self._m:
104 | self._g.update(w, y)
105 |
106 | # timestep
107 | self._t += 1
108 |
109 | # mark model as ready
110 | if self._t >= self._tmin:
111 | self._ready = True
112 |
113 | # no setter
114 | @property
115 | def A(self) -> np.ndarray:
116 | """A in state space representation
117 |
118 | Returns:
119 | np.ndarray: [description]
120 | """
121 | if not self._ready:
122 | logger.warning(f"Model not ready (have not seen enough data)!")
123 | return self._f.M[:, : self._n]
124 |
125 | @property
126 | def B(self) -> np.ndarray:
127 | """B in state space representation
128 |
129 | Returns:
130 | np.ndarray: [description]
131 | """
132 | if not self._ready:
133 | logger.warning(f"Model not ready (have not seen enough data)!")
134 | return self._f.M[:, self._n :]
135 |
136 | @property
137 | def C(self) -> np.ndarray:
138 | """C in state space representation
139 |
140 | Raises:
141 | Exception: if no output eqution
142 |
143 | Returns:
144 | np.ndarray: [description]
145 | """
146 | if not self._m:
147 | raise Exception(f"No output eqution!")
148 | if not self._ready:
149 | logger.warning(f"Model not ready (have not seen enough data)!")
150 | return self._g.M[:, : self._n]
151 |
152 | @property
153 | def D(self) -> np.ndarray:
154 | """D in state space representation
155 |
156 | Raises:
157 | Exception: if no output eqution
158 |
159 | Returns:
160 | np.ndarray: [description]
161 | """
162 | if not self._m:
163 | raise Exception(f"No output eqution!")
164 | if not self._ready:
165 | logger.warning(f"Model not ready (have not seen enough data)!")
166 | return self._g.M[:, self._n :]
167 |
168 | @property
169 | def n(self) -> int:
170 | """State x(t) dimension as in state space representation
171 |
172 | Returns:
173 | int: [description]
174 | """
175 | return self._n
176 |
177 | @property
178 | def k(self) -> int:
179 | """Control (input) u(t) dimension as in state space representation
180 |
181 | Returns:
182 | int: [description]
183 | """
184 | return self._k
185 |
186 | @property
187 | def m(self) -> int:
188 | """Observation (output) y(t) dimension as in state space representation
189 |
190 | Returns:
191 | int: [description]
192 | """
193 | return self._m
194 |
195 | @property
196 | def alpha(self) -> float:
197 | """Exponential weighting factor in (0, 1]
198 | Small value allows more adaptive learning and more forgetting
199 |
200 | Returns:
201 | float: [description]
202 | """
203 | return self._alpha
204 |
205 | @property
206 | def t(self) -> int:
207 | """Total number measurements processed
208 |
209 | Returns:
210 | int: [description]
211 | """
212 | return self._t
213 |
214 | @property
215 | def ready(self) -> bool:
216 | """If the model has seen enough data
217 |
218 | Returns:
219 | bool: [description]
220 | """
221 | return self._ready
--------------------------------------------------------------------------------
/ai4sci/oml/_online_model.py:
--------------------------------------------------------------------------------
1 | import logging
2 |
3 | import numpy as np
4 |
5 | logger = logging.getLogger(__name__)
6 |
7 |
8 | class OnlineModel:
9 | """OnlineModel is a class that implements online model learning
10 | The time complexity (multiply–add operation for one iteration) is O(4n^2),
11 | and space complexity is O(2n^2), where m is the state dimension.
12 |
13 | Algorithm description:
14 | If the dynamics is z(t) = f(z(t-1), u(t-1)), then we first choose a nonlinear
15 | observable phi(~, ~), and assume the model can be approximated by
16 | z(t) = M * phi(z(t-1), u(t-1)).
17 |
18 | Let x(t) = phi(z(t-1), u(t-1)), and y(t) = z(t).
19 | We would like to learn an adaptive linear model M (a matrix) st y(t) = M * x(t).
20 | The matrix M is updated recursively by efficient rank-1 updating online algrithm.
21 | An exponential weighting factor (alpha in (0, 1])) can be used to place more weight on
22 | recent data.
23 |
24 | At time step t, define two matrix X(t) = [x(1), x(2), ..., x(t)],
25 | Y(t) = [y(1), y(2),..., y(t)], that contain all the past snapshot pairs,
26 | where x(t), y(t) are the m dimensional state vector, y(t) = f(x(t)) is
27 | the image of x(t), f() is the dynamics.
28 |
29 | Here, if there is no control and dynamics is z(t) = f(z(t-1)),
30 | then x(t), y(t) should be measurements correponding to consecutive
31 | states z(t-1) and z(t).
32 |
33 | Usage:
34 | online_model = OnlineModel(n, m, alpha)
35 | online_model.initialize(X, Y) # optional
36 | online_model.update(x, y)
37 |
38 | properties:
39 | M: the model matrix (adaptive and updated online) as in y(t) = M * x(t)
40 | n: x(t) vector dimension, can be z(t-1) (for z(t) = f(z(t-1))) or phi(z(t-1), u(t-1)) (for z(t) = f(z(t-1), u(t-1)))
41 | state dimension + control dimension for linear system identification
42 | m: y(t) vector dimension, z(t) in z(t) = f(z(t-1), u(t-1)) or z(t) = f(z(t-1))
43 | state dimension for linear system identification
44 | alpha: weighting factor in (0,1]
45 | t: number of snapshot pairs processed (i.e., current time step)
46 |
47 | methods:
48 | initialize(X, Y), initialize online model learning algorithm with first p
49 | snapshot pairs stored in (X, Y), this func call is optional
50 | update(x, y), update online adaptive model when new snapshot pair (x, y)
51 | becomes available
52 |
53 | Authors:
54 | Hao Zhang
55 |
56 | References:
57 | Zhang, Hao, Clarence W. Rowley, Eric A. Deem, and Louis N. Cattafesta.
58 | "Online dynamic mode decomposition for time-varying systems."
59 | SIAM Journal on Applied Dynamical Systems 18, no. 3 (2019): 1586-1609.
60 | """
61 | def __init__(self, n: int, m: int, alpha: float = 0.9) -> None:
62 | """Creat an object for online model learning
63 | Usage: online_model = OnlineModel(n, m, alpha)
64 |
65 | Args:
66 | n (int): x dimension in model y(t) = M * x(t)
67 | m (int): y dimension in model y(t) = M * x(t)
68 | alpha (float, optional): exponential weighting factor in (0, 1], smaller values allows more adaptive learning. Defaults to 0.9.
69 | """
70 | # input check
71 | assert isinstance(n, int) and n >= 1
72 | assert isinstance(m, int) and m >= 1
73 | alpha = float(alpha)
74 | assert alpha > 0 and alpha <= 1
75 |
76 | # initialize parameters
77 | self._n = n
78 | self._m = m
79 | self._alpha = alpha
80 | self._t = 0
81 | self._A = np.zeros([m, n])
82 | self._P = np.zeros([n, n])
83 |
84 | # initialize model
85 | self._initialize()
86 | self._ready = False
87 |
88 | def _initialize(self) -> None:
89 | """Initialize online model with epsilon small (1e-15) ghost snapshot pairs before t=0"""
90 | epsilon = 1e-15
91 | self._A = np.random.randn(self._m, self._n)
92 | self._P = np.identity(self._n) / epsilon
93 |
94 | def initialize(self, X: np.ndarray, Y: np.ndarray) -> None:
95 | """Initialize online model with first p (p >= max(n, m)) snapshot pairs stored in (X, Y)
96 | (x(t), y(t)) as in model y(t) = M * x(t)
97 | Usage: online_model.initialize(X, Y) # this step is optional
98 |
99 | Args:
100 | X (np.ndarray): 2D array, shape (n, p), matrix [x(0), x(1), ..., x(n)]
101 | Y (np.ndarray): 2D array, shape (m, p), matrix [y(0), y(1), ..., y(n)]
102 | """
103 | # input check
104 | assert X is not None and Y is not None
105 | X, Y = np.array(X).reshape(self._n, -1), np.array(Y).reshape(self._m, -1)
106 | assert X.shape[1] == Y.shape[1]
107 |
108 | # necessary condition for over-constrained initialization
109 | p = X.shape[1]
110 | assert p >= max(self._n, self._m)
111 | assert np.linalg.matrix_rank(X) == min(self._n, p)
112 |
113 | # initialize with weighted snapshots
114 | weight = np.sqrt(self._alpha) ** range(p - 1, -1, -1)
115 | Xhat, Yhat = weight * X, weight * Y
116 | self._A = Yhat.dot(np.linalg.pinv(Xhat))
117 | self._P = np.linalg.inv(Xhat.dot(Xhat.T)) / self._alpha
118 | self._t += p
119 |
120 | def update(self, x: np.ndarray, y: np.ndarray) -> None:
121 | """Update the Model with a new pair of snapshots (x, y)
122 | y = f(x) is the dynamics/model
123 | Here, if the (discrete-time) dynamics are given by z(t) = f(z(t-1)),
124 | then (x,y) should be measurements correponding to consecutive states
125 | z(t-1) and z(t).
126 | Usage: online_model.update(x, y)
127 |
128 | Args:
129 | x (np.ndarray): 1D array, shape (n, ), x(t) in model y(t) = M * x(t)
130 | y (np.ndarray): 1D array, shape (m, ), y(t) in model y(t) = M * x(t)
131 | """
132 | # input check
133 | assert x is not None and y is not None
134 | x, y = np.array(x).reshape(-1), np.array(y).reshape(-1)
135 | assert x.shape[0] == self._n
136 | assert y.shape[0] == self._m
137 |
138 | # compute P*x matrix vector product beforehand
139 | Px = self._P.dot(x)
140 | # compute gamma
141 | gamma = 1.0 / (1 + x.T.dot(Px))
142 | # update A
143 | self._A += np.outer(gamma * (y - self._A.dot(x)), Px)
144 | # update P, group Px*Px' to ensure positive definite
145 | self._P = (self._P - gamma * np.outer(Px, Px)) / self._alpha
146 | # ensure P is SPD by taking its symmetric part
147 | self._P = (self._P + self._P.T) / 2
148 |
149 | # time step + 1
150 | self._t += 1
151 |
152 | # mark model as ready
153 | if self._t >= 2 * max(self._m, self._n):
154 | self._ready = True
155 |
156 | # ready only properties
157 | @property
158 | def M(self) -> np.ndarray:
159 | """Matrix in model y(t) = M * x(t)
160 |
161 | Returns:
162 | np.ndarray: [description]
163 | """
164 | if not self._ready:
165 | logger.warning(f"Model not ready (have not seen enough data)!")
166 | return self._A
167 |
168 | @property
169 | def m(self) -> int:
170 | """y dimension in model y(t) = M * x(t)
171 |
172 | Returns:
173 | int: [description]
174 | """
175 | return self._m
176 |
177 | @property
178 | def n(self) -> int:
179 | """x dimension in model y(t) = M * x(t)
180 |
181 | Returns:
182 | int: [description]
183 | """
184 | return self._n
185 |
186 | @property
187 | def alpha(self) -> float:
188 | """Exponential weighting factor in (0, 1]
189 | Small value allows more adaptive learning and more forgetting
190 |
191 | Returns:
192 | float: [description]
193 | """
194 | return self._alpha
195 |
196 | @property
197 | def t(self) -> int:
198 | """Total number measurements processed
199 | y(i) = M * x(i)
200 | (x(i), y(i)), i = 0,1,...,t
201 |
202 | Returns:
203 | int: [description]
204 | """
205 | return self._t
206 |
207 | @property
208 | def ready(self) -> bool:
209 | """If the model has seen enough data
210 |
211 | Returns:
212 | bool: [description]
213 | """
214 | return self._ready
--------------------------------------------------------------------------------
/assets/linear_time_varying.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/haozhg/oml/6008a5d3223ebcce3aff4ea066042d8e23402edb/assets/linear_time_varying.png
--------------------------------------------------------------------------------
/assets/linear_time_varying_controlled.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/haozhg/oml/6008a5d3223ebcce3aff4ea066042d8e23402edb/assets/linear_time_varying_controlled.png
--------------------------------------------------------------------------------
/assets/lorenz_control.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/haozhg/oml/6008a5d3223ebcce3aff4ea066042d8e23402edb/assets/lorenz_control.png
--------------------------------------------------------------------------------
/assets/lorenz_control_controlled.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/haozhg/oml/6008a5d3223ebcce3aff4ea066042d8e23402edb/assets/lorenz_control_controlled.png
--------------------------------------------------------------------------------
/assets/lorenz_state.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/haozhg/oml/6008a5d3223ebcce3aff4ea066042d8e23402edb/assets/lorenz_state.png
--------------------------------------------------------------------------------
/assets/lorenz_state_controlled.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/haozhg/oml/6008a5d3223ebcce3aff4ea066042d8e23402edb/assets/lorenz_state_controlled.png
--------------------------------------------------------------------------------
/demo/README.md:
--------------------------------------------------------------------------------
1 | ## Data-driven closed loop real-time control
2 | This directory contains examples of the application of this algorithm to data-driven real-time closed loop control.
3 | - Linear time-varying system, `demo_linear_time_varying_system.ipynb`, can also run it in [google colab](https://colab.research.google.com/drive/1b4-wEfqXhY4QWeU7M7rmEMXwmJwZvsfU?usp=sharing)
4 | - Lorenz system, `demo_lorenz.ipynb`, can also run it in [google colab](https://colab.research.google.com/drive/1dWeKuiEsVUjlNaKSFW6b7J-UyyFwov8C?usp=sharing)
5 |
--------------------------------------------------------------------------------
/demo/demo_linear_time_varying_system.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "\"\"\"\n",
8 | "An example to demonstrate online linear system identification\n",
9 | "\n",
10 | "We demonstrate the use of OnlineLinearModel class with a simple linear system.\n",
11 | "Take a 2D time-varying system dx/dt=A(t)x(t)+B(t)u(t), where A(t) and B(t)\n",
12 | "are slowly varying with time. In particular, we take A(t)=(1+eps\\*t)\\*A,\n",
13 | "B(t)=(1+eps\\*t)\\*B, and eps = 0.1 is small. It is discretize with\n",
14 | "time step dt = 0.1. Denote the discrete system as x(k+1)=A(k)x(k)+\n",
15 | "B(k)u(k).\n",
16 | "\n",
17 | "At time step t+1, we need to include new snapshot pair x(t), u(t), x(t+1).\n",
18 | "We would like to update the adaptive model in real-time\n",
19 | "\n",
20 | "Authors: \n",
21 | "Hao Zhang\n",
22 | "\n",
23 | "References:\n",
24 | "Zhang, Hao, Clarence W. Rowley, Eric A. Deem, and Louis N. Cattafesta.\n",
25 | "\"Online dynamic mode decomposition for time-varying systems.\"\n",
26 | "SIAM Journal on Applied Dynamical Systems 18, no. 3 (2019): 1586-1609.\n",
27 | "\n",
28 | "Created:\n",
29 | "June 2017.\n",
30 | "\"\"\""
31 | ]
32 | },
33 | {
34 | "cell_type": "code",
35 | "execution_count": 7,
36 | "metadata": {
37 | "ExecuteTime": {
38 | "end_time": "2022-10-20T22:53:15.751498Z",
39 | "start_time": "2022-10-20T22:53:15.587128Z"
40 | }
41 | },
42 | "outputs": [
43 | {
44 | "name": "stdout",
45 | "output_type": "stream",
46 | "text": [
47 | "/Users/haozhang/python-venvs/oml-reorg/bin/python: Mach-O 64-bit executable x86_64\r\n"
48 | ]
49 | }
50 | ],
51 | "source": [
52 | "!file $(which python)"
53 | ]
54 | },
55 | {
56 | "cell_type": "code",
57 | "execution_count": 8,
58 | "metadata": {
59 | "ExecuteTime": {
60 | "end_time": "2022-10-20T22:53:16.348831Z",
61 | "start_time": "2022-10-20T22:53:16.199174Z"
62 | }
63 | },
64 | "outputs": [
65 | {
66 | "name": "stdout",
67 | "output_type": "stream",
68 | "text": [
69 | "Python 3.8.5\r\n"
70 | ]
71 | }
72 | ],
73 | "source": [
74 | "!python --version"
75 | ]
76 | },
77 | {
78 | "cell_type": "code",
79 | "execution_count": 9,
80 | "metadata": {
81 | "ExecuteTime": {
82 | "end_time": "2022-10-20T22:53:18.233275Z",
83 | "start_time": "2022-10-20T22:53:17.151364Z"
84 | }
85 | },
86 | "outputs": [],
87 | "source": [
88 | "!python -m pip install -Uqqq pip setuptools wheel"
89 | ]
90 | },
91 | {
92 | "cell_type": "code",
93 | "execution_count": 15,
94 | "metadata": {
95 | "ExecuteTime": {
96 | "end_time": "2022-10-20T22:54:18.035936Z",
97 | "start_time": "2022-10-20T22:54:17.896542Z"
98 | }
99 | },
100 | "outputs": [
101 | {
102 | "name": "stdout",
103 | "output_type": "stream",
104 | "text": [
105 | "README.md demo_lorenz.ipynb\r\n",
106 | "demo_linear_time_varying_system.ipynb requirements.txt\r\n"
107 | ]
108 | }
109 | ],
110 | "source": [
111 | "!ls"
112 | ]
113 | },
114 | {
115 | "cell_type": "code",
116 | "execution_count": 16,
117 | "metadata": {
118 | "ExecuteTime": {
119 | "end_time": "2022-10-20T22:54:24.872985Z",
120 | "start_time": "2022-10-20T22:54:24.732583Z"
121 | }
122 | },
123 | "outputs": [
124 | {
125 | "name": "stdout",
126 | "output_type": "stream",
127 | "text": [
128 | "ai4sci.oml\r\n",
129 | "numpy\r\n",
130 | "scipy\r\n",
131 | "matplotlib\r\n",
132 | "control\r\n",
133 | "slycot"
134 | ]
135 | }
136 | ],
137 | "source": [
138 | "!cat requirements.txt"
139 | ]
140 | },
141 | {
142 | "cell_type": "code",
143 | "execution_count": 17,
144 | "metadata": {
145 | "ExecuteTime": {
146 | "end_time": "2022-10-20T22:54:26.445102Z",
147 | "start_time": "2022-10-20T22:54:25.697720Z"
148 | }
149 | },
150 | "outputs": [
151 | {
152 | "name": "stdout",
153 | "output_type": "stream",
154 | "text": [
155 | "Requirement already satisfied: ai4sci.oml in /Users/haozhang/Local/open-source/oml (from -r requirements.txt (line 1)) (0.2.2)\n",
156 | "Requirement already satisfied: numpy in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from -r requirements.txt (line 2)) (1.23.4)\n",
157 | "Requirement already satisfied: scipy in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from -r requirements.txt (line 3)) (1.9.3)\n",
158 | "Requirement already satisfied: matplotlib in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from -r requirements.txt (line 4)) (3.6.1)\n",
159 | "Requirement already satisfied: control in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from -r requirements.txt (line 5)) (0.9.2)\n",
160 | "Requirement already satisfied: slycot in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from -r requirements.txt (line 6)) (0.5.0)\n",
161 | "Requirement already satisfied: python-dateutil>=2.7 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (2.8.2)\n",
162 | "Requirement already satisfied: pyparsing>=2.2.1 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (3.0.9)\n",
163 | "Requirement already satisfied: cycler>=0.10 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (0.11.0)\n",
164 | "Requirement already satisfied: pillow>=6.2.0 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (9.2.0)\n",
165 | "Requirement already satisfied: kiwisolver>=1.0.1 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (1.4.4)\n",
166 | "Requirement already satisfied: fonttools>=4.22.0 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (4.37.4)\n",
167 | "Requirement already satisfied: packaging>=20.0 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (21.3)\n",
168 | "Requirement already satisfied: contourpy>=1.0.1 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (1.0.5)\n",
169 | "Requirement already satisfied: six>=1.5 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from python-dateutil>=2.7->matplotlib->-r requirements.txt (line 4)) (1.16.0)\n"
170 | ]
171 | }
172 | ],
173 | "source": [
174 | "!python -m pip install -r requirements.txt"
175 | ]
176 | },
177 | {
178 | "cell_type": "code",
179 | "execution_count": 11,
180 | "metadata": {
181 | "ExecuteTime": {
182 | "end_time": "2022-10-20T22:53:21.040240Z",
183 | "start_time": "2022-10-20T22:53:20.535533Z"
184 | }
185 | },
186 | "outputs": [
187 | {
188 | "name": "stdout",
189 | "output_type": "stream",
190 | "text": [
191 | "ai4sci.oml 0.2.2 /Users/haozhang/Local/open-source/oml\r\n"
192 | ]
193 | }
194 | ],
195 | "source": [
196 | "!pip list | grep ai4sci.oml"
197 | ]
198 | },
199 | {
200 | "cell_type": "code",
201 | "execution_count": 13,
202 | "metadata": {
203 | "ExecuteTime": {
204 | "end_time": "2022-10-20T22:53:39.169234Z",
205 | "start_time": "2022-10-20T22:53:39.139049Z"
206 | }
207 | },
208 | "outputs": [
209 | {
210 | "ename": "ModuleNotFoundError",
211 | "evalue": "No module named 'ai4sci'",
212 | "output_type": "error",
213 | "traceback": [
214 | "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
215 | "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)",
216 | "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 27\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mmatplotlib\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpyplot\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 28\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mnumpy\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 29\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mai4sci\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moml\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mOnlineLinearModel\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 30\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mcontrol\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mlqr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mStateSpace\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mctrb\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 31\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
217 | "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'ai4sci'"
218 | ]
219 | }
220 | ],
221 | "source": [
222 | "\"\"\"\n",
223 | "An example to demonstrate online linear system identification\n",
224 | "\n",
225 | "We demonstrate the use of OnlineLinearModel class with a simple linear system.\n",
226 | "Take a 2D time-varying system dx/dt=A(t)x(t)+B(t)u(t), where A(t) and B(t)\n",
227 | "are slowly varying with time. In particular, we take A(t)=(1+eps*t)*A,\n",
228 | "B(t)=(1+eps*t)*B, and eps = 0.1 is small. It is discretize with\n",
229 | "time step dt = 0.02. Denote the discrete system as x(k+1)=A(k)x(k)+\n",
230 | "B(k)u(k).\n",
231 | "\n",
232 | "At time step k+1, we need to include new snapshot pair x(k), u(k), x(k+1).\n",
233 | "We would like to update the adaptive model in real-time\n",
234 | "\n",
235 | "Authors: \n",
236 | "Hao Zhang\n",
237 | "\n",
238 | "References:\n",
239 | "Zhang, Hao, Clarence W. Rowley, Eric A. Deem, and Louis N. Cattafesta.\n",
240 | "\"Online dynamic mode decomposition for time-varying systems.\"\n",
241 | "SIAM Journal on Applied Dynamical Systems 18, no. 3 (2019): 1586-1609.\n",
242 | "\n",
243 | "Created:\n",
244 | "June 2017.\n",
245 | "\"\"\"\n",
246 | "\n",
247 | "import random\n",
248 | "import matplotlib.pyplot as plt\n",
249 | "import numpy as np\n",
250 | "from ai4sci.oml import OnlineLinearModel\n",
251 | "from control import lqr, StateSpace, ctrb\n",
252 | "\n",
253 | "random.seed(20210220)\n",
254 | "np.random.seed(20210220)\n",
255 | "\n",
256 | "# define dynamics, negative damping -> unstable\n",
257 | "# https://en.wikipedia.org/wiki/Harmonic_oscillator#Damped_harmonic_oscillator\n",
258 | "n = 2 # state dimension\n",
259 | "k = 1 # control dimension\n",
260 | "A = np.array([[0, 1], [-1, 0.2]])\n",
261 | "B = np.array([[0], [1]])\n",
262 | "C = np.eye(n)\n",
263 | "D = np.zeros((n, k))\n",
264 | "\n",
265 | "# check poles\n",
266 | "sys = StateSpace(A, B, np.eye(n), np.zeros((n, k)))\n",
267 | "print(f\"open loop poles: {sys.pole()}\")\n",
268 | "\n",
269 | "# check controllability\n",
270 | "Ctrb = ctrb(A, B)\n",
271 | "assert np.linalg.matrix_rank(Ctrb) == n\n",
272 | "\n",
273 | "# slowly time-varying dynamics\n",
274 | "def dyn(t, x, u, epsilon=1e-3):\n",
275 | " At = (1 + epsilon * t) * A\n",
276 | " Bt = (1 + epsilon * t) * B\n",
277 | " dxdt = At.dot(x) + Bt.dot(u)\n",
278 | " return dxdt\n",
279 | "\n",
280 | "# set up simulation parameter\n",
281 | "dt = 0.02\n",
282 | "tmax, tc = 20, 0.5\n",
283 | "T, kc = int(tmax / dt), int(tc / dt)\n",
284 | "tspan = np.linspace(0, tmax, T + 1)\n",
285 | "\n",
286 | "# online linear system identification setup\n",
287 | "alpha = 0.01 ** (1.0 / kc) # 99% decay after kc samples\n",
288 | "olm = OnlineLinearModel(n, k, None, alpha)\n",
289 | "\n",
290 | "# store data mtrices\n",
291 | "x = np.zeros([n, T])\n",
292 | "u = np.zeros([k, T])\n",
293 | "\n",
294 | "# initial condition, state and control\n",
295 | "x0 = np.array([1, 0])\n",
296 | "u0 = 0\n",
297 | "\n",
298 | "# uncontrolled system\n",
299 | "x[:, 0] = x0\n",
300 | "u[:, 0] = u0\n",
301 | "for t in range(1, T):\n",
302 | " # forward the system for one step\n",
303 | " x[:, t] = x[:, t - 1] + dt * dyn(t * dt, x[:, t - 1], u[:, t - 1])\n",
304 | " u[:, t] = 0\n",
305 | "\n",
306 | "def plot_state_control(x, u):\n",
307 | " plt.rcParams['figure.dpi'] = 100\n",
308 | " fig, axs = plt.subplots(2, figsize=(10, 6))\n",
309 | " fig.suptitle('State and control')\n",
310 | " axs[0].plot(tspan[1:], x[0, :], \"b-\", linewidth=1.0, label=\"State $x_1(t)$\")\n",
311 | " axs[0].plot(tspan[1:], x[1, :], \"g-\", linewidth=1.0, label=\"State $x_2(t)$\")\n",
312 | " axs[0].legend(loc=\"best\", fontsize=12, shadow=True)\n",
313 | " axs[0].grid()\n",
314 | "\n",
315 | " axs[1].plot(tspan[1:], u.reshape(-1), \"r-\", linewidth=1.0, label=\"Control $u(t)$\")\n",
316 | " axs[1].legend(loc=\"best\", fontsize=12, shadow=True)\n",
317 | " axs[1].grid()\n",
318 | " axs[1].set_xlabel(\"Time\", fontsize=12)\n",
319 | "\n",
320 | "plot_state_control(x, u)"
321 | ]
322 | },
323 | {
324 | "cell_type": "code",
325 | "execution_count": 14,
326 | "metadata": {
327 | "ExecuteTime": {
328 | "end_time": "2022-10-20T22:53:39.611008Z",
329 | "start_time": "2022-10-20T22:53:39.581041Z"
330 | }
331 | },
332 | "outputs": [
333 | {
334 | "ename": "NameError",
335 | "evalue": "name 'x' is not defined",
336 | "output_type": "error",
337 | "traceback": [
338 | "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
339 | "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
340 | "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;31m# control system simulation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;31m# initial condition, state and control\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mx\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 4\u001b[0m \u001b[0mu\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mT\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
341 | "\u001b[0;31mNameError\u001b[0m: name 'x' is not defined"
342 | ]
343 | }
344 | ],
345 | "source": [
346 | "# control system simulation\n",
347 | "# initial condition, state and control\n",
348 | "x[:, 0] = np.array([1, 0])\n",
349 | "u[:, 0] = 0\n",
350 | "for t in range(1, T):\n",
351 | " # forward the system for one step\n",
352 | " x[:, t] = x[:, t - 1] + dt * dyn(t * dt, x[:, t - 1], u[:, t - 1])\n",
353 | " # use new measurement to update online system identification\n",
354 | " olm.update(x[:, t - 1], u[:, t - 1], x[:, t])\n",
355 | " # apply control if we have collected enough data\n",
356 | " if t > 4 * max(n, n + k):\n",
357 | " # convert to continuous representation\n",
358 | " Ac = (olm.A - np.eye(n)) / dt\n",
359 | " Bc = olm.B / dt\n",
360 | " # get LQR gain, output = state\n",
361 | " sys = StateSpace(Ac, Bc, C, D, dt=0)\n",
362 | " K, S, E = lqr(sys, np.eye(n), np.eye(k))\n",
363 | " # apply control\n",
364 | " u[:, t] = -1.0 * K.dot(x[:, t])\n",
365 | " # clip control\n",
366 | " u[:, t] = np.clip(u[:, t], -10, 10)\n",
367 | " # smooth control\n",
368 | " w = 0.5\n",
369 | " u[:, t] = (1 - w) * u[:, t - 1] + w * u[:, t]\n",
370 | " # show progress\n",
371 | " if t % 100 == 0:\n",
372 | " print(f\"x(t)={x[:, t]}\")\n",
373 | " print(f\"u(t)={u[:, t]}\")\n",
374 | " print(f\"Ac={Ac}\")\n",
375 | " print(f\"Bc={Bc}\")\n",
376 | " print(f\"K={K}\")\n",
377 | " # random small perturbation if not enough data yet\n",
378 | " else:\n",
379 | " u[:, t] = 1e-1 * np.random.randn()\n",
380 | "\n",
381 | "plot_state_control(x, u)"
382 | ]
383 | },
384 | {
385 | "cell_type": "code",
386 | "execution_count": null,
387 | "metadata": {},
388 | "outputs": [],
389 | "source": []
390 | }
391 | ],
392 | "metadata": {
393 | "kernelspec": {
394 | "display_name": "Python 3",
395 | "language": "python",
396 | "name": "python3"
397 | },
398 | "language_info": {
399 | "codemirror_mode": {
400 | "name": "ipython",
401 | "version": 3
402 | },
403 | "file_extension": ".py",
404 | "mimetype": "text/x-python",
405 | "name": "python",
406 | "nbconvert_exporter": "python",
407 | "pygments_lexer": "ipython3",
408 | "version": "3.8.5"
409 | },
410 | "toc": {
411 | "base_numbering": 1,
412 | "nav_menu": {},
413 | "number_sections": true,
414 | "sideBar": true,
415 | "skip_h1_title": false,
416 | "title_cell": "Table of Contents",
417 | "title_sidebar": "Contents",
418 | "toc_cell": false,
419 | "toc_position": {},
420 | "toc_section_display": true,
421 | "toc_window_display": false
422 | },
423 | "varInspector": {
424 | "cols": {
425 | "lenName": 16,
426 | "lenType": 16,
427 | "lenVar": 40
428 | },
429 | "kernels_config": {
430 | "python": {
431 | "delete_cmd_postfix": "",
432 | "delete_cmd_prefix": "del ",
433 | "library": "var_list.py",
434 | "varRefreshCmd": "print(var_dic_list())"
435 | },
436 | "r": {
437 | "delete_cmd_postfix": ") ",
438 | "delete_cmd_prefix": "rm(",
439 | "library": "var_list.r",
440 | "varRefreshCmd": "cat(var_dic_list()) "
441 | }
442 | },
443 | "types_to_exclude": [
444 | "module",
445 | "function",
446 | "builtin_function_or_method",
447 | "instance",
448 | "_Feature"
449 | ],
450 | "window_display": false
451 | }
452 | },
453 | "nbformat": 4,
454 | "nbformat_minor": 4
455 | }
456 |
--------------------------------------------------------------------------------
/demo/demo_lorenz.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "\"\"\"\n",
8 | "An example to demonstrate online control of lorenz system\n",
9 | "\n",
10 | "Authors: \n",
11 | "Hao Zhang\n",
12 | "\n",
13 | "References:\n",
14 | "Zhang, Hao, Clarence W. Rowley, Eric A. Deem, and Louis N. Cattafesta.\n",
15 | "\"Online dynamic mode decomposition for time-varying systems.\"\n",
16 | "SIAM Journal on Applied Dynamical Systems 18, no. 3 (2019): 1586-1609.\n",
17 | "\n",
18 | "Created:\n",
19 | "June 2017.\n",
20 | "\"\"\""
21 | ]
22 | },
23 | {
24 | "cell_type": "code",
25 | "execution_count": 1,
26 | "metadata": {
27 | "ExecuteTime": {
28 | "end_time": "2022-10-20T22:54:06.411809Z",
29 | "start_time": "2022-10-20T22:54:06.248173Z"
30 | }
31 | },
32 | "outputs": [
33 | {
34 | "name": "stdout",
35 | "output_type": "stream",
36 | "text": [
37 | "/Users/haozhang/python-venvs/oml-reorg/bin/python: Mach-O 64-bit executable x86_64\r\n"
38 | ]
39 | }
40 | ],
41 | "source": [
42 | "!file $(which python)"
43 | ]
44 | },
45 | {
46 | "cell_type": "code",
47 | "execution_count": 2,
48 | "metadata": {
49 | "ExecuteTime": {
50 | "end_time": "2022-10-20T22:54:06.749849Z",
51 | "start_time": "2022-10-20T22:54:06.600938Z"
52 | }
53 | },
54 | "outputs": [
55 | {
56 | "name": "stdout",
57 | "output_type": "stream",
58 | "text": [
59 | "Python 3.8.5\r\n"
60 | ]
61 | }
62 | ],
63 | "source": [
64 | "!python --version"
65 | ]
66 | },
67 | {
68 | "cell_type": "code",
69 | "execution_count": 6,
70 | "metadata": {
71 | "ExecuteTime": {
72 | "end_time": "2022-10-20T22:54:29.821587Z",
73 | "start_time": "2022-10-20T22:54:28.647668Z"
74 | }
75 | },
76 | "outputs": [],
77 | "source": [
78 | "!python -m pip install -Uqqq pip setuptools wheel"
79 | ]
80 | },
81 | {
82 | "cell_type": "code",
83 | "execution_count": 7,
84 | "metadata": {
85 | "ExecuteTime": {
86 | "end_time": "2022-10-20T22:54:29.963607Z",
87 | "start_time": "2022-10-20T22:54:29.829432Z"
88 | }
89 | },
90 | "outputs": [
91 | {
92 | "name": "stdout",
93 | "output_type": "stream",
94 | "text": [
95 | "README.md demo_lorenz.ipynb\r\n",
96 | "demo_linear_time_varying_system.ipynb requirements.txt\r\n"
97 | ]
98 | }
99 | ],
100 | "source": [
101 | "!ls"
102 | ]
103 | },
104 | {
105 | "cell_type": "code",
106 | "execution_count": 8,
107 | "metadata": {
108 | "ExecuteTime": {
109 | "end_time": "2022-10-20T22:54:30.103061Z",
110 | "start_time": "2022-10-20T22:54:29.971794Z"
111 | }
112 | },
113 | "outputs": [
114 | {
115 | "name": "stdout",
116 | "output_type": "stream",
117 | "text": [
118 | "ai4sci.oml\r\n",
119 | "numpy\r\n",
120 | "scipy\r\n",
121 | "matplotlib\r\n",
122 | "control\r\n",
123 | "slycot"
124 | ]
125 | }
126 | ],
127 | "source": [
128 | "!cat requirements.txt"
129 | ]
130 | },
131 | {
132 | "cell_type": "code",
133 | "execution_count": 9,
134 | "metadata": {
135 | "ExecuteTime": {
136 | "end_time": "2022-10-20T22:54:31.074287Z",
137 | "start_time": "2022-10-20T22:54:30.329666Z"
138 | }
139 | },
140 | "outputs": [
141 | {
142 | "name": "stdout",
143 | "output_type": "stream",
144 | "text": [
145 | "Requirement already satisfied: ai4sci.oml in /Users/haozhang/Local/open-source/oml (from -r requirements.txt (line 1)) (0.2.2)\n",
146 | "Requirement already satisfied: numpy in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from -r requirements.txt (line 2)) (1.23.4)\n",
147 | "Requirement already satisfied: scipy in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from -r requirements.txt (line 3)) (1.9.3)\n",
148 | "Requirement already satisfied: matplotlib in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from -r requirements.txt (line 4)) (3.6.1)\n",
149 | "Requirement already satisfied: control in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from -r requirements.txt (line 5)) (0.9.2)\n",
150 | "Requirement already satisfied: slycot in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from -r requirements.txt (line 6)) (0.5.0)\n",
151 | "Requirement already satisfied: fonttools>=4.22.0 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (4.37.4)\n",
152 | "Requirement already satisfied: cycler>=0.10 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (0.11.0)\n",
153 | "Requirement already satisfied: python-dateutil>=2.7 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (2.8.2)\n",
154 | "Requirement already satisfied: pyparsing>=2.2.1 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (3.0.9)\n",
155 | "Requirement already satisfied: contourpy>=1.0.1 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (1.0.5)\n",
156 | "Requirement already satisfied: packaging>=20.0 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (21.3)\n",
157 | "Requirement already satisfied: kiwisolver>=1.0.1 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (1.4.4)\n",
158 | "Requirement already satisfied: pillow>=6.2.0 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 4)) (9.2.0)\n",
159 | "Requirement already satisfied: six>=1.5 in /Users/haozhang/python-venvs/oml-reorg/lib/python3.8/site-packages (from python-dateutil>=2.7->matplotlib->-r requirements.txt (line 4)) (1.16.0)\n"
160 | ]
161 | }
162 | ],
163 | "source": [
164 | "!python -m pip install -r requirements.txt"
165 | ]
166 | },
167 | {
168 | "cell_type": "code",
169 | "execution_count": 10,
170 | "metadata": {
171 | "ExecuteTime": {
172 | "end_time": "2022-10-20T22:54:31.965529Z",
173 | "start_time": "2022-10-20T22:54:31.352735Z"
174 | }
175 | },
176 | "outputs": [
177 | {
178 | "name": "stdout",
179 | "output_type": "stream",
180 | "text": [
181 | "ai4sci.oml 0.2.2 /Users/haozhang/Local/open-source/oml\r\n"
182 | ]
183 | }
184 | ],
185 | "source": [
186 | "!pip list | grep ai4sci.oml"
187 | ]
188 | },
189 | {
190 | "cell_type": "markdown",
191 | "metadata": {},
192 | "source": [
193 | "Uncontrolled system simulation\n",
194 | "- classical butterfly trajectory"
195 | ]
196 | },
197 | {
198 | "cell_type": "code",
199 | "execution_count": 11,
200 | "metadata": {
201 | "ExecuteTime": {
202 | "end_time": "2022-10-20T22:54:38.652620Z",
203 | "start_time": "2022-10-20T22:54:38.373461Z"
204 | }
205 | },
206 | "outputs": [
207 | {
208 | "ename": "ModuleNotFoundError",
209 | "evalue": "No module named 'ai4sci'",
210 | "output_type": "error",
211 | "traceback": [
212 | "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
213 | "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)",
214 | "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 17\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mmatplotlib\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpyplot\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 18\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mnumpy\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 19\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mai4sci\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moml\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mOnlineLinearModel\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 20\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mcontrol\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mlqr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mStateSpace\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mctrb\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 21\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
215 | "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'ai4sci'"
216 | ]
217 | }
218 | ],
219 | "source": [
220 | "\"\"\"\n",
221 | "An example to demonstrate online control of lorenz system\n",
222 | "\n",
223 | "Authors: \n",
224 | "Hao Zhang\n",
225 | "\n",
226 | "References:\n",
227 | "Zhang, Hao, Clarence W. Rowley, Eric A. Deem, and Louis N. Cattafesta.\n",
228 | "\"Online dynamic mode decomposition for time-varying systems.\"\n",
229 | "SIAM Journal on Applied Dynamical Systems 18, no. 3 (2019): 1586-1609.\n",
230 | "\n",
231 | "Created:\n",
232 | "June 2017.\n",
233 | "\"\"\"\n",
234 | "\n",
235 | "import random\n",
236 | "import matplotlib.pyplot as plt\n",
237 | "import numpy as np\n",
238 | "from ai4sci.oml import OnlineLinearModel\n",
239 | "from control import lqr, StateSpace, ctrb\n",
240 | "\n",
241 | "random.seed(20210220)\n",
242 | "np.random.seed(20210220)\n",
243 | "\n",
244 | "# define dynamics\n",
245 | "# https://en.wikipedia.org/wiki/Lorenz_system\n",
246 | "# Lorenz\n",
247 | "n = 3 # state dimension\n",
248 | "k = 3 # input dimension\n",
249 | "# output = state (full state observation)\n",
250 | "sigma = 10\n",
251 | "beta = 8.0 / 3.0\n",
252 | "rho = 28\n",
253 | "def dyn(x, u):\n",
254 | " assert x.shape == (n,)\n",
255 | " assert u.shape == (k,)\n",
256 | " f = np.zeros(n) # dx/dt = f(x, u)\n",
257 | " f[0] = sigma * (x[1] - x[0]) + u[0]\n",
258 | " f[1] = x[0] * (rho - x[2]) - x[1] + u[1]\n",
259 | " f[2] = x[0] * x[1] - beta * x[2] + u[2]\n",
260 | " return f\n",
261 | "\n",
262 | "# fixed point (unstable)\n",
263 | "phi = np.sqrt(beta * (rho - 1))\n",
264 | "xf1 = np.array([phi, phi, rho - 1])\n",
265 | "xf2 = np.array([-phi, -phi, rho - 1])\n",
266 | "xf3 = np.array([0, 0, 0])\n",
267 | "\n",
268 | "# set up simulation parameter\n",
269 | "dt = 0.01\n",
270 | "tmax, tc = 40, 0.4\n",
271 | "T, kc = int(tmax / dt), int(tc / dt)\n",
272 | "tspan = np.linspace(0, tmax, T + 1)\n",
273 | "\n",
274 | "# online linear system identification setup\n",
275 | "alpha = 0.01 ** (1.0 / kc) # 99% decay after kc samples\n",
276 | "olm = OnlineLinearModel(n, k, None, alpha)\n",
277 | "\n",
278 | "# store data mtrices\n",
279 | "x = np.zeros([n, T])\n",
280 | "u = np.zeros([k, T])\n",
281 | "\n",
282 | "# initial condition, state and control\n",
283 | "x0 = np.array([1, 1, 1])\n",
284 | "u0 = np.array([0, 0, 0])\n",
285 | "\n",
286 | "# initial condition\n",
287 | "x[:, 0] = x0\n",
288 | "u[:, 0] = u0\n",
289 | "\n",
290 | "# uncontrolled system simulation\n",
291 | "for t in range(1, T):\n",
292 | " # forward the system for one step\n",
293 | " x[:, t] = x[:, t - 1] + dt * dyn(x[:, t - 1], u[:, t - 1])\n",
294 | " u[:, t] = 0\n",
295 | "\n",
296 | "\n",
297 | "def plot_state(x):\n",
298 | " plt.rcParams['figure.dpi'] = 100\n",
299 | " fig = plt.figure(figsize=(6, 6))\n",
300 | " ax = fig.gca(projection=\"3d\")\n",
301 | " ax.plot(x[0, :], x[1, :], x[2, :])\n",
302 | " plt.title(\"State\")\n",
303 | " plt.draw()\n",
304 | " plt.grid()\n",
305 | " plt.show()\n",
306 | "\n",
307 | "\n",
308 | "def plot_control(u):\n",
309 | " plt.rcParams['figure.dpi'] = 100\n",
310 | " fig, axs = plt.subplots(3, figsize=(6, 6))\n",
311 | "\n",
312 | " axs[0].plot(tspan[1:], u[0, :], \"r-\", linewidth=1.0, label=\"$u_1(t)$\")\n",
313 | " axs[0].legend(loc=\"best\", fontsize=12, shadow=True)\n",
314 | " axs[0].grid()\n",
315 | " axs[0].set_title(\"Control\")\n",
316 | "\n",
317 | " axs[1].plot(tspan[1:], u[1, :], \"g-\", linewidth=1.0, label=\"$u_2(t)$\")\n",
318 | " axs[1].legend(loc=\"best\", fontsize=12, shadow=True)\n",
319 | " axs[1].grid()\n",
320 | "\n",
321 | " axs[2].plot(tspan[1:], u[2, :], \"b-\", linewidth=1.0, label=\"$u_3(t)$\")\n",
322 | " axs[2].legend(loc=\"best\", fontsize=12, shadow=True)\n",
323 | " axs[2].grid()\n",
324 | "\n",
325 | " axs[2].set_xlabel(\"Time\", fontsize=12)\n",
326 | "\n",
327 | "\n",
328 | "# plot state and control\n",
329 | "plot_state(x)\n",
330 | "plot_control(u)"
331 | ]
332 | },
333 | {
334 | "cell_type": "markdown",
335 | "metadata": {},
336 | "source": [
337 | "Controlled system simulation\n",
338 | "- Purely data-driven\n",
339 | "- Real-time model learning\n",
340 | "- Adaptive to new data\n",
341 | "- Closed loop control\n",
342 | "- Stabilizes system at unstable fixed point"
343 | ]
344 | },
345 | {
346 | "cell_type": "code",
347 | "execution_count": 12,
348 | "metadata": {
349 | "ExecuteTime": {
350 | "end_time": "2022-10-20T22:54:41.577450Z",
351 | "start_time": "2022-10-20T22:54:41.521448Z"
352 | }
353 | },
354 | "outputs": [
355 | {
356 | "ename": "NameError",
357 | "evalue": "name 'x0' is not defined",
358 | "output_type": "error",
359 | "traceback": [
360 | "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
361 | "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
362 | "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;31m# initial condition, state and control\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m \u001b[0mx\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mx0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 7\u001b[0m \u001b[0mu\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mu0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
363 | "\u001b[0;31mNameError\u001b[0m: name 'x0' is not defined"
364 | ]
365 | }
366 | ],
367 | "source": [
368 | "# controlled system simulation\n",
369 | "# we want to stabilize system at fixed point xf1\n",
370 | "# need to shift state by -xf1\n",
371 | "\n",
372 | "# initial condition, state and control\n",
373 | "x[:, 0] = x0\n",
374 | "u[:, 0] = u0\n",
375 | "\n",
376 | "# control system simulation\n",
377 | "for t in range(1, T):\n",
378 | " # forward the system for one step\n",
379 | " x[:, t] = x[:, t - 1] + dt * dyn(x[:, t - 1], u[:, t - 1])\n",
380 | " # use new measurement to update online system identification\n",
381 | " # try stabilize at xf1, shift state by -xf1\n",
382 | " olm.update(x[:, t - 1] - xf1, u[:, t - 1], x[:, t] - xf1)\n",
383 | " # apply control if we have collected enough data\n",
384 | " if t > 4 * max(n, n + k):\n",
385 | " # convert to continuous representation\n",
386 | " Ac = (olm.A - np.eye(n)) / dt\n",
387 | " Bc = olm.B / dt\n",
388 | " # get LQR gain, output = state\n",
389 | " sys = StateSpace(Ac, Bc, np.eye(n), np.zeros((n, k)), dt=0)\n",
390 | " K, S, E = lqr(sys, 2 * np.eye(n), np.eye(k))\n",
391 | " # apply control\n",
392 | " u[:, t] = -1.0 * K.dot(x[:, t] - xf1)\n",
393 | " # clip control\n",
394 | " u[:, t] = np.clip(u[:, t], -10, 10)\n",
395 | " # smooth control\n",
396 | " w = 0.5\n",
397 | " u[:, t] = (1 - w) * u[:, t - 1] + w * u[:, t]\n",
398 | " # show progress\n",
399 | " if t % 1000 == 0:\n",
400 | " print(f\"t={t * dt}\")\n",
401 | " print(f\"x(t)={x[:, t]}\")\n",
402 | " print(f\"xf1={xf1}\")\n",
403 | " print(f\"u(t)={u[:, t]}\")\n",
404 | " print(f\"Ac={Ac}\")\n",
405 | " print(f\"Bc={Bc}\")\n",
406 | " print(f\"K={K}\\n\")\n",
407 | " # random small perturbation if not enough data yet\n",
408 | " else:\n",
409 | " u[:, t] = 1e-1 * np.random.randn(k)\n",
410 | "\n",
411 | "plot_state(x)\n",
412 | "plot_control(u)\n",
413 | "assert np.linalg.norm(x[:, -1] - xf1) < 1e-1"
414 | ]
415 | },
416 | {
417 | "cell_type": "code",
418 | "execution_count": null,
419 | "metadata": {},
420 | "outputs": [],
421 | "source": []
422 | }
423 | ],
424 | "metadata": {
425 | "kernelspec": {
426 | "display_name": "Python 3",
427 | "language": "python",
428 | "name": "python3"
429 | },
430 | "language_info": {
431 | "codemirror_mode": {
432 | "name": "ipython",
433 | "version": 3
434 | },
435 | "file_extension": ".py",
436 | "mimetype": "text/x-python",
437 | "name": "python",
438 | "nbconvert_exporter": "python",
439 | "pygments_lexer": "ipython3",
440 | "version": "3.8.5"
441 | },
442 | "toc": {
443 | "base_numbering": 1,
444 | "nav_menu": {},
445 | "number_sections": true,
446 | "sideBar": true,
447 | "skip_h1_title": false,
448 | "title_cell": "Table of Contents",
449 | "title_sidebar": "Contents",
450 | "toc_cell": false,
451 | "toc_position": {},
452 | "toc_section_display": true,
453 | "toc_window_display": false
454 | },
455 | "varInspector": {
456 | "cols": {
457 | "lenName": 16,
458 | "lenType": 16,
459 | "lenVar": 40
460 | },
461 | "kernels_config": {
462 | "python": {
463 | "delete_cmd_postfix": "",
464 | "delete_cmd_prefix": "del ",
465 | "library": "var_list.py",
466 | "varRefreshCmd": "print(var_dic_list())"
467 | },
468 | "r": {
469 | "delete_cmd_postfix": ") ",
470 | "delete_cmd_prefix": "rm(",
471 | "library": "var_list.r",
472 | "varRefreshCmd": "cat(var_dic_list()) "
473 | }
474 | },
475 | "types_to_exclude": [
476 | "module",
477 | "function",
478 | "builtin_function_or_method",
479 | "instance",
480 | "_Feature"
481 | ],
482 | "window_display": false
483 | }
484 | },
485 | "nbformat": 4,
486 | "nbformat_minor": 4
487 | }
488 |
--------------------------------------------------------------------------------
/demo/requirements.txt:
--------------------------------------------------------------------------------
1 | ai4sci.oml
2 | numpy
3 | scipy
4 | matplotlib
5 | control
6 | slycot
--------------------------------------------------------------------------------
/setup.py:
--------------------------------------------------------------------------------
1 | from setuptools import setup, find_namespace_packages
2 |
3 | # read readme as long description
4 | with open("README.md", "r") as f:
5 | long_description = f.read()
6 |
7 | # This call to setup() does all the work
8 | setup(
9 | name="ai4sci.oml",
10 | version="0.2.2",
11 | description="AI4Science: Efficient data-driven Online Model Learning (OML) / system identification and control",
12 | long_description=long_description,
13 | long_description_content_type="text/markdown",
14 | url="https://github.com/haozhg/oml",
15 | author="Hao Zhang",
16 | author_email="haozhang@alumni.princeton.edu",
17 | license="MIT",
18 | keywords=[
19 | "machine-learning",
20 | "AI for Science",
21 | "data-driven modeling",
22 | "reduced-order-modeling",
23 | "dynamical systems",
24 | "control theory",
25 | "system identification",
26 | "online model learning",
27 | ],
28 | classifiers=[
29 | "License :: OSI Approved :: MIT License",
30 | "Programming Language :: Python :: 3",
31 | "Programming Language :: Python :: 3.7",
32 | ],
33 | python_requires='>=3.7',
34 | packages=find_namespace_packages(include=["ai4sci.*"]),
35 | include_package_data=False,
36 | install_requires=["numpy"],
37 | )
38 |
--------------------------------------------------------------------------------
/tests/README.md:
--------------------------------------------------------------------------------
1 | # Tests
2 | ```
3 | pip install ai4sci.oml
4 | pip install -r requirements.txt
5 | python -m pytest .
6 | ```
7 | For demo of these algorithms on data-driven real-time closed loop control, see `/demo` directory.
--------------------------------------------------------------------------------
/tests/requirements.txt:
--------------------------------------------------------------------------------
1 | ai4sci.oml
2 | pytest
--------------------------------------------------------------------------------
/tests/test_online_linear_model.py:
--------------------------------------------------------------------------------
1 | import logging
2 |
3 | import numpy as np
4 | from ai4sci.oml import OnlineLinearModel
5 |
6 | np.random.seed(20210218)
7 |
8 | logger = logging.getLogger(__name__)
9 |
10 |
11 | def update(A, eps=1e-6):
12 | return A + eps * np.random.randn(*A.shape)
13 |
14 |
15 | def test_online_linear_model():
16 | # n is state dimension
17 | for n in range(2, 10):
18 | k = n // 2 # control (input) dimension
19 | m = n // 2 # observation (output) dimension
20 | T = 16 * n # total number of measurements
21 | print(f"n={n}, k={k}, m={m}, T={T}")
22 |
23 | # true model, slowly varying in time
24 | A = np.random.randn(n, n)
25 | B = np.random.randn(n, k)
26 | C = np.random.randn(m, n)
27 | D = np.random.randn(m, k)
28 |
29 | # online linear model learning
30 | # no need to initialize
31 | olm = OnlineLinearModel(n, k, m, alpha=0.5)
32 | for t in range(T):
33 | # initial condition
34 | x = np.random.randn(n)
35 | u = np.random.randn(k)
36 |
37 | # state update
38 | xn = A.dot(x) + B.dot(u)
39 | y = C.dot(x) + D.dot(u)
40 |
41 | # update model est
42 | olm.update(x, u, xn, y)
43 | if olm.ready:
44 | assert np.linalg.norm(olm.A - A) / (n * n) < 1e-3
45 | assert np.linalg.norm(olm.B - B) / (n * k) < 1e-3
46 | assert np.linalg.norm(olm.C - C) / (m * n) < 1e-3
47 | assert np.linalg.norm(olm.D - D) / (m * k) < 1e-3
48 |
49 | # update time-varying model
50 | A = update(A)
51 | B = update(B)
52 | C = update(C)
53 | D = update(D)
54 |
55 |
56 | if __name__ == "__main__":
57 | test_online_linear_model()
58 |
--------------------------------------------------------------------------------
/tests/test_online_model.py:
--------------------------------------------------------------------------------
1 | import numpy as np
2 | from ai4sci.oml import OnlineModel
3 |
4 | np.random.seed(20210218)
5 |
6 |
7 | def update(A, eps=1e-6):
8 | return A + eps * np.random.randn(*A.shape)
9 |
10 |
11 | def test_online_model():
12 | # m is y dimension
13 | for m in range(2, 10):
14 | n = 2 * m # x dimension
15 | T = 16 * m # total number of measurements
16 | print(f"m={m}, n={n}, T={T}")
17 |
18 | # true model, slowly varying in time
19 | A = np.random.randn(m, n)
20 |
21 | # online model learning
22 | # no need to initialize
23 | online_model = OnlineModel(n, m, alpha=0.5)
24 | for t in range(T):
25 | x = np.random.randn(n)
26 | y = A.dot(x)
27 | online_model.update(x, y)
28 | if online_model.ready:
29 | assert np.linalg.norm(online_model.M - A) / (m * n) < 1e-3
30 |
31 | # update time-varying model
32 | A = update(A)
33 |
34 |
35 | if __name__ == "__main__":
36 | test_online_model()
37 |
--------------------------------------------------------------------------------