├── .gitignore ├── BellmanDiffusion.md ├── CITATION.bib ├── DiffEqOperators.md ├── Discretizing Linear Operators.ipynb ├── HeatEquation.md ├── KdV.md ├── README.md ├── Telegrapher_Equations.ipynb ├── linear_operators_overview.tex ├── operator_examples ├── non_zero_dirichlet_heat.jl ├── simple_stationary_HJBE_reflecting.jl └── zero_neumann_heat.jl └── transition_dynamics_examples └── transition_dynamics_example.jl /.gitignore: -------------------------------------------------------------------------------- 1 | *.md.pdf 2 | *-cache.yaml 3 | *.aux 4 | *.fls 5 | *.log 6 | *.pdf 7 | *.gz 8 | _minted* 9 | *.fdb_latexmk 10 | *.bbl 11 | *.blg 12 | .ipynb_checkpoints/ 13 | *.synctex(busy) 14 | *.DS_Store 15 | -------------------------------------------------------------------------------- /BellmanDiffusion.md: -------------------------------------------------------------------------------- 1 | # Solving the Bellman Equation with a Simple Univariate Diffusion 2 | ## Setup 3 | Take the stochastic process 4 | $$ 5 | d x_t = \mu(t, x) dt + \sigma(t, x) d W_t 6 | $$ 7 | where $W_t$ is Brownian motion and reflecting barriers at $x \in (x^{\min},x^{\max})$ 8 | 9 | The partial differential operator (infinitesimal generator) associated with the stochastic process is 10 | 11 | \begin{align} 12 | \tilde{L_1} \equiv \tilde{\mu}(t, x) \partial_x + \frac{\tilde{\sigma}(t, x)^2}{2}\partial_{xx} 13 | \end{align} 14 | 15 | Then, if the payoff in state $x$ is $c(x) = x^2$, and payoffs are discounted at rate $\rho$, then the Bellman equation is, 16 | $$ 17 | \rho \tilde{u}(t, x) = \tilde{c}(t, x) + \tilde{L}_1 \tilde{u}(t, x) + \partial_t \tilde{u}(t,x) 18 | $$ 19 | With boundary values $\partial_x \tilde{u}(t, x^{\min}) = 0$ and $\partial_x \tilde{u}(t, x^{\max}) = 0$ for all $t$ 20 | 21 | We can combine these to form the operator, 22 | \begin{align} 23 | \tilde{L} = \rho - \tilde{L_1} 24 | \end{align} 25 | and the boundary condition operator (using the $|$ for "evaluated at"), 26 | \begin{align} 27 | \tilde{B} = \begin{bmatrix} 28 | \partial_x \Big|_{x=x^{\min},t}\\ 29 | \partial_x \Big|_{x=x^{\max},t} 30 | \end{bmatrix} 31 | \end{align} 32 | 33 | which leads to the PDE, 34 | $$ 35 | \partial_t \tilde{u}(t,x) = \tilde{L}_t \tilde{u}(t,x) - \tilde{c}(t,x) 36 | $$ 37 | and boundary conditions at every $t$, 38 | $$ 39 | \tilde{B} \tilde{u}(t,x) = \begin{bmatrix} 0 \\ 0 \end{bmatrix} 40 | $$ 41 | 42 | 43 | ## Example Functions 44 | As a numerical example, start with something like 45 | - $x^{\min} = 0.01$ 46 | - $x^{\max} = 1.0$ 47 | - $\tilde{\mu}(t,x) = -0.1 + t + .1 x$ 48 | - Note, that this keeps $\tilde{\mu}(t,x) \geq 0$ for all $t,x$. Hence, we know the correct upwind direction. 49 | - $\tilde{\sigma}(t,x) = \bar{\sigma} x$ for $\bar{\sigma} = 0.1$ 50 | - $\tilde{c}(t,x) = e^x$ 51 | - $\rho = 0.05$ 52 | 53 | ## Discretization 54 | Do a discretization of the $\tilde{L}$ operator subject to the $\tilde{B}$, using the standard technique (and knowing that the positive drift ensures we can use a single upwind direction). the value function is then $u(t) \in R^M$, an operator is $L(t) \in R^M$, and a vector of payoffs $c(t) \in R^M$. This leads to the following system of ODEs, 55 | $$ 56 | \partial_t u(t) = L(t) u(t) - c(t) 57 | $$ 58 | 59 | The stationary solution, at a $t=T$ is the solution to the linear system, 60 | $$ 61 | u(T) = L(T) \backslash c(T) 62 | $$ 63 | 64 | Given this solution, we can solve for the transition dynamics by going back in time from the $u(T)$ initial condition. 65 | -------------------------------------------------------------------------------- /CITATION.bib: -------------------------------------------------------------------------------- 1 | @article{DifferentialEquations.jl-2017, 2 | author = {Rackauckas, Christopher and Nie, Qing}, 3 | doi = {10.5334/jors.151}, 4 | journal = {The Journal of Open Research Software}, 5 | keywords = {Applied Mathematics}, 6 | note = {Exported from https://app.dimensions.ai on 2019/05/05}, 7 | number = {1}, 8 | pages = {}, 9 | title = {DifferentialEquations.jl – A Performant and Feature-Rich Ecosystem for Solving Differential Equations in Julia}, 10 | url = {https://app.dimensions.ai/details/publication/pub.1085583166 and http://openresearchsoftware.metajnl.com/articles/10.5334/jors.151/galley/245/download/}, 11 | volume = {5}, 12 | year = {2017} 13 | } 14 | -------------------------------------------------------------------------------- /DiffEqOperators.md: -------------------------------------------------------------------------------- 1 | # Basics of DiffEqOperators 2 | 3 | In this tutorial we will explore the basic functionalities of PDEOperator which is used to obtain the discretizations of PDEs of appropriate derivative and approximation order. 4 | 5 | So an operator API is as follows:- 6 | 7 | A = DerivativeOperator{T} 8 | ( 9 | derivative_order, 10 | approximation_order, 11 | grid_step_size, 12 | grid_size, 13 | :LBC, 14 | :RBC; 15 | BC=(LBV, RBV) 16 | ); 17 | Currently we support the `Dirichlet 0/1`, `Neumann 0/1`, `periodic` and `Robin` boundary conditions. 18 | 19 | Taking a specific example 20 | 21 | A = DerivativeOperator{Float64}(2,2,1/99,10,:Dirichlet,:Dirichlet; BC=(u[1],u[end])) 22 | 23 | this is the time independent Dirichlet BC. You can also specify a time dependent Dirichlet BC as follows:- 24 | 25 | A = DerivativeOperator{Float64}(2,2,1/99,10,:Dirichlet,:Dirichlet; bndry_fn=(t->(u[1]*cos(t)),u[end])) 26 | 27 | We have generated an operator which produces the 2nd order approximation of the Laplacian. We can checkout the stencil as follows:- 28 | 29 | julia> A.stencil_coefs 30 | 3-element SVector{3,Float64}: 31 | 1.0 32 | -2.0 33 | 1.0 34 | 35 | We can get the linear operator as a matrix as follows:- 36 | 37 | julia> full(A) 38 | 10×10 Array{Float64,2}: 39 | -2.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 40 | 1.0 -2.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 41 | 0.0 1.0 -2.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 42 | 0.0 0.0 1.0 -2.0 1.0 0.0 0.0 0.0 0.0 0.0 43 | 0.0 0.0 0.0 1.0 -2.0 1.0 0.0 0.0 0.0 0.0 44 | 0.0 0.0 0.0 0.0 1.0 -2.0 1.0 0.0 0.0 0.0 45 | 0.0 0.0 0.0 0.0 0.0 1.0 -2.0 1.0 0.0 0.0 46 | 0.0 0.0 0.0 0.0 0.0 0.0 1.0 -2.0 1.0 0.0 47 | 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 -2.0 1.0 48 | 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 -2.0 49 | 50 | Note that we **don't** need to define the `BC` only for `:D0` and `:periodic` boundary conditions so you can ignore it. 51 | 52 | 53 | Now coming to the main functionality of DiffEqOperators ie. taking finite difference discretizations of functions. 54 | 55 | julia> x = collect(0 : 1/99 : 1); 56 | julia> u0 = x.^2 -x; 57 | julia> res = A*u0 58 | 100-element Array{Float64,1}: 59 | -98.0 60 | 2.0 61 | 2.0 62 | 2.0 63 | 2.0 64 | 2.0 65 | 2.0 66 | ⋮ 67 | 2.0 68 | 2.0 69 | 2.0 70 | 2.0 71 | 2.0 72 | 2.0 73 | -98.0 74 | 75 | The derivative values at the boundaries are in accordance with the `Dirichlet` boundary condition. 76 | 77 | You can also take derivatives of matrices using `A*M` or `M*A` where the order of multiplication decides the axis along which we want to take derivatives. 78 | 79 | julia> xarr = linspace(0,1,51) 80 | julia> yarr = linspace(0,1,101) 81 | julia> dx = xarr[2]-xarr[1] 82 | julia> dy = yarr[2]-yarr[1] 83 | julia> F = [x^2+y for x = xarr, y = yarr] 84 | julia> A = DerivativeOperator{Float64}(2,2,dx,length(yarr),:None,:None) 85 | julia> B = DerivativeOperator{Float64}(2,2,dy,length(yarr),:None,:None) 86 | 87 | julia> # A*F calculates derivatives along the x axis ie. keeping y constant 88 | julia> A*F 89 | 51×101 Array{Float64,2}: 90 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 … 2.0 2.0 2.0 2.0 2.0 2.0 2.0 91 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 92 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 93 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 94 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 95 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 … 2.0 2.0 2.0 2.0 2.0 2.0 2.0 96 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 97 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 98 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 99 | ⋮ ⋮ ⋱ ⋮ ⋮ 100 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 101 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 102 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 103 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 … 2.0 2.0 2.0 2.0 2.0 2.0 2.0 104 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 105 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 106 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 107 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 108 | 2.0 2.0 2.0 2.0 2.0 2.0 2.0 … 2.0 2.0 2.0 2.0 2.0 2.0 2.0 109 | 110 | julia> # F*B calculates derivatives along the y axis ie. keeping x constant 111 | julia> F*B 112 | 101×51 Array{Float64,2}: 113 | 0.0 1.04083e-13 6.93889e-14 … 2.22045e-12 2.22045e-12 114 | 0.0 3.46945e-14 0.0 0.0 0.0 115 | 0.0 -3.46945e-14 -6.93889e-14 0.0 0.0 116 | 6.93889e-14 0.0 6.93889e-14 0.0 0.0 117 | 0.0 0.0 0.0 0.0 0.0 118 | -6.93889e-14 -6.93889e-14 -6.93889e-14 … 0.0 0.0 119 | 1.38778e-13 1.38778e-13 1.38778e-13 0.0 0.0 120 | -1.38778e-13 -1.38778e-13 -2.77556e-13 0.0 0.0 121 | 0.0 0.0 0.0 0.0 0.0 122 | ⋮ ⋱ ⋮ 123 | 0.0 0.0 0.0 4.44089e-12 4.44089e-12 124 | -1.11022e-12 -1.11022e-12 -1.11022e-12 -4.44089e-12 -4.44089e-12 125 | 1.11022e-12 1.11022e-12 1.11022e-12 2.22045e-12 2.22045e-12 126 | 0.0 0.0 0.0 … 0.0 0.0 127 | 0.0 0.0 0.0 0.0 0.0 128 | 0.0 0.0 0.0 0.0 0.0 129 | 0.0 0.0 0.0 0.0 0.0 130 | 0.0 0.0 0.0 0.0 0.0 131 | 0.0 0.0 0.0 … 8.88178e-12 8.88178e-12 132 | 133 | 134 | 135 | **Note:** Please take care that the boundary values passed to the operator match the initial boundary conditions. The operator with the boundary condition is meant to enforce the boundary condition rather bring the boundaries to that state. ~~Right now we support only **constant** boundaries conditions, time dependent conditions will supported in later versions.~~ 136 | Support for time dependent Dirichlet BC has been added. 137 | 138 | **Note:** If you want to parallelize the operation of PDEOperator, please start Julia by specifying the number of threads using `export JULIA_NUM_THREADS=` 139 | -------------------------------------------------------------------------------- /Discretizing Linear Operators.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Overview\n", 8 | "\n", 9 | "This notebook includes prototypical code for the operator overview write-up. It does not depend on any exisiting code in DiffEqOperators, but it should be easy to modify the current codebase to achieve the same functionality.\n", 10 | "\n", 11 | "To make things simpler, everything is assumed to use `Float64` datatype. I will also always use the out-of-place convention (i.e. `*` instead of `A_mul_B!`).\n", 12 | "\n", 13 | "The naming of different operators will use the following convention, as in the writeup:\n", 14 | "\n", 15 | "- `L` denotes a (quasi)linear operator.\n", 16 | "\n", 17 | "- `A` denotes an operator that can generally (but not necessarily) be affine.\n", 18 | "\n", 19 | "- `b` denotes the bias of an affine operator\n", 20 | "\n", 21 | "- (Not discussed but I guess we may as well make the call?) `M` for raw matrices (`AbstractMatrix`) and `x`, `y`, `u`, etc for vectors." 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": 1, 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [ 30 | "import Base: +, -, *, \\\n", 31 | "\n", 32 | "abstract type DiffEqOperator end\n", 33 | "abstract type DiffEqLinearOperator <: DiffEqOperator end" 34 | ] 35 | }, 36 | { 37 | "cell_type": "markdown", 38 | "metadata": {}, 39 | "source": [ 40 | "# 1. Constant case (i.e. no `update_coefficients!`)\n", 41 | "\n", 42 | "## 1.1 Abstract operator interface\n", 43 | "\n", 44 | "Below is a simplified operator interface used in my new interface draft. Basically we want to express lazy addition and multiplication of linear operators naturally using `+` and `*`. The `as_array` interface returns the most suitable (dense/sparse) representation of the underlying operator as an `AbstractMatrix` (or should we treat this as a type conversion and just use `AbstractMatrix`?).\n", 45 | "\n", 46 | "The affine operator is defined in such a way that arithmetic on them and linear operators always yield an `DiffEqAffineOperator`." 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "execution_count": 2, 52 | "metadata": {}, 53 | "outputs": [], 54 | "source": [ 55 | "# We should improve type stability by including more inferrable fields in the type signature\n", 56 | "# But for the sake of this notebook I'll make things simple\n", 57 | "struct DiffEqArrayOperator <: DiffEqLinearOperator\n", 58 | " M::AbstractMatrix{Float64}\n", 59 | "end\n", 60 | "\n", 61 | "*(L::DiffEqArrayOperator, x::Vector{Float64}) = L.M * x\n", 62 | "as_array(L::DiffEqArrayOperator) = L.M\n", 63 | "\n", 64 | "struct DiffEqOperatorCombination <: DiffEqLinearOperator\n", 65 | " cs::Tuple{Vararg{Float64}} # Coefficients\n", 66 | " Ls::Tuple{Vararg{DiffEqLinearOperator}}\n", 67 | "end\n", 68 | "\n", 69 | "*(L::DiffEqOperatorCombination, x::Vector{Float64}) = sum(ck * (Lk*x) for (ck,Lk) in zip(L.cs,L.Ls))\n", 70 | "as_array(L::DiffEqOperatorCombination) = sum(ck * as_array(Lk) for (ck,Lk) in zip(L.cs,L.Ls))\n", 71 | "\n", 72 | "struct DiffEqOperatorComposition <: DiffEqLinearOperator\n", 73 | " Ls::Tuple{Vararg{DiffEqLinearOperator}}\n", 74 | "end\n", 75 | "\n", 76 | "*(L::DiffEqOperatorComposition, x::Vector{Float64}) = foldl((u, Lk) -> Lk*u, x, L.Ls)\n", 77 | "as_array(L::DiffEqOperatorComposition) = prod(as_array, reverse(L.Ls))\n", 78 | "\n", 79 | "struct DiffEqAffineOperator <: DiffEqOperator\n", 80 | " L::DiffEqLinearOperator\n", 81 | " b::Vector{Float64}\n", 82 | "end\n", 83 | "\n", 84 | "*(A::DiffEqAffineOperator, x::Vector{Float64}) = A.L * x + A.b;" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": 3, 90 | "metadata": {}, 91 | "outputs": [], 92 | "source": [ 93 | "# Operator arithmetic\n", 94 | "# Addition\n", 95 | "+(L1::DiffEqOperatorCombination, L2::DiffEqOperatorCombination) = DiffEqOperatorCombination(\n", 96 | " (L1.cs...,L2.cs...), (L1.Ls...,L2.Ls...))\n", 97 | "+(L1::DiffEqOperatorCombination, L2::DiffEqLinearOperator) = DiffEqOperatorCombination((L1.cs...,1.0), (L1.Ls...,L2))\n", 98 | "+(L1::DiffEqLinearOperator, L2::DiffEqOperatorCombination) = L2 + L1\n", 99 | "+(L1::DiffEqLinearOperator, L2::DiffEqLinearOperator) = DiffEqOperatorCombination((1.0,1.0), (L1,L2))\n", 100 | "\n", 101 | "+(A1::DiffEqAffineOperator, L2::DiffEqLinearOperator) = DiffEqAffineOperator(A1.L + L2, A1.b)\n", 102 | "+(L1::DiffEqLinearOperator, A2::DiffEqAffineOperator) = A2 + L1\n", 103 | "+(A1::DiffEqAffineOperator, A2::DiffEqAffineOperator) = DiffEqAffineOperator(A1.L + A2.L, A1.b + A2.b)\n", 104 | "\n", 105 | "# Scalar multiplication\n", 106 | "*(α::Float64, L::DiffEqOperatorCombination) = DiffEqOperatorCombination(α.*L.cs, L.Ls)\n", 107 | "*(α::Float64, L::DiffEqLinearOperator) = DiffEqOperatorCombination((α,), (L,))\n", 108 | "*(α::Float64, A::DiffEqAffineOperator) = DiffEqAffineOperator(α * A.L, α * A.b)\n", 109 | "*(A::DiffEqOperator, α::Float64) = α * A\n", 110 | "\n", 111 | "# Subtraction/unary minus\n", 112 | "-(A::DiffEqOperator) = (-1.0) * A\n", 113 | "-(A1::DiffEqOperator, A2::DiffEqOperator) = A1 + (-A2)\n", 114 | "\n", 115 | "# Multiplication\n", 116 | "# Note the application order\n", 117 | "*(L1::DiffEqOperatorComposition, L2::DiffEqOperatorComposition) = DiffEqOperatorComposition((L2.Ls..., L1.Ls...))\n", 118 | "*(L1::DiffEqLinearOperator, L2::DiffEqOperatorComposition) = DiffEqOperatorComposition((L2.Ls..., L1))\n", 119 | "*(L1::DiffEqOperatorComposition, L2::DiffEqLinearOperator) = DiffEqOperatorComposition((L2, L1.Ls...))\n", 120 | "*(L1::DiffEqLinearOperator, L2::DiffEqLinearOperator) = DiffEqOperatorComposition((L2, L1))\n", 121 | "\n", 122 | "*(L1::DiffEqLinearOperator, A2::DiffEqAffineOperator) = DiffEqAffineOperator(L1 * A2.L, L1 * A2.b)\n", 123 | "*(A1::DiffEqAffineOperator, L2::DiffEqLinearOperator) = DiffEqAffineOperator(A1.L * L2, A1.b)\n", 124 | "*(A1::DiffEqAffineOperator, A2::DiffEqAffineOperator) = DiffEqAffineOperator(A1.L * A2.L, A1.L * A2.b + A1.b)\n", 125 | "\n", 126 | "# Right division (i.e. linear solve)\n", 127 | "# In the full version we should also include interface to lazy solvers\n", 128 | "\\(L::DiffEqLinearOperator, y::Vector{Float64}) = as_array(L) \\ y\n", 129 | "\\(A::DiffEqAffineOperator, y::Vector{Float64}) = A.L \\ (y - A.b);" 130 | ] 131 | }, 132 | { 133 | "cell_type": "markdown", 134 | "metadata": {}, 135 | "source": [ 136 | "## 1.2 Discretization of the differential operator (stencil convolution)\n", 137 | "\n", 138 | "The 2nd-order central difference approximation to $\\partial_x^2$ and the 1st-order upwind approximation to $\\partial_x$ are included. The general case can be modified from the stencil convolution code in DiffEqOperators.\n", 139 | "\n", 140 | "The upwind operator is implemented differently from DiffEqOperators, where the direction at each point is stored. Here only the left/right operator is constructed and we interpret\n", 141 | "\n", 142 | "$$ \\mu(x)\\partial_x = \\mu^+(x)\\partial_x^+ + \\mu^-(x)\\partial_x^- $$\n", 143 | "\n", 144 | "(Might not be a favorable approach, especially if we wish to extend to multidimensional upwind operators)" 145 | ] 146 | }, 147 | { 148 | "cell_type": "code", 149 | "execution_count": 4, 150 | "metadata": {}, 151 | "outputs": [], 152 | "source": [ 153 | "struct DiffusionOperator <: DiffEqLinearOperator\n", 154 | " dx::Float64\n", 155 | " m::Int # number of interior points\n", 156 | "end\n", 157 | "\n", 158 | "*(L::DiffusionOperator, x::Vector{Float64}) = [x[i] + x[i+2] - 2*x[i+1] for i in 1:L.m] / L.dx^2\n", 159 | "as_array(L::DiffusionOperator) = spdiagm((ones(L.m), -2*ones(L.m), ones(L.m)), (0,1,2)) / L.dx^2;" 160 | ] 161 | }, 162 | { 163 | "cell_type": "code", 164 | "execution_count": 5, 165 | "metadata": {}, 166 | "outputs": [], 167 | "source": [ 168 | "struct DriftOperator <: DiffEqLinearOperator\n", 169 | " dx::Float64\n", 170 | " m::Int # number of interior points\n", 171 | " direction::Bool # true = right, false = left\n", 172 | "end\n", 173 | "\n", 174 | "function *(L::DriftOperator, x::Vector{Float64})\n", 175 | " if L.direction # right drift\n", 176 | " [x[i+1] - x[i] for i in 1:L.m] / L.dx\n", 177 | " else # left drift\n", 178 | " [x[i+2] - x[i+1] for i in 1:L.m] / L.dx\n", 179 | " end\n", 180 | "end\n", 181 | "function as_array(L::DriftOperator)\n", 182 | " if L.direction # right drift\n", 183 | " spdiagm((-ones(L.m), ones(L.m), zeros(L.m)), (0,1,2)) / L.dx\n", 184 | " else # left drift\n", 185 | " spdiagm((zeros(L.m), -ones(L.m), ones(L.m)), (0,1,2)) / L.dx\n", 186 | " end\n", 187 | "end;" 188 | ] 189 | }, 190 | { 191 | "cell_type": "markdown", 192 | "metadata": {}, 193 | "source": [ 194 | "# 1.3 Boundary extrapolation operator $Q$\n", 195 | "\n", 196 | "Question: is it OK to shorthand \"boundary extrapolation operator\" as \"BEOperator\" or simply \"BE\"?\n", 197 | "\n", 198 | "A generic $Q$ from the generic boundary condition $Bu = b$ can be a bit difficult to implement, but the simple case of Dirichlet/Neumann BC is easy to handle.\n", 199 | "\n", 200 | "It should be easy to modify `AbsorbingBoundaryMap` and `ReflectingBoundaryMap` to incorporate boundaries that are absorbing at one end and reflecting at another.\n", 201 | "\n", 202 | "The non-zero Dirichlet/Neumann Qs do not have their own type. Instead we construct the operator as an affine map, with `AbsorbingBoundaryMap` and `ReflectingBoundaryMap` its linear part." 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": 6, 208 | "metadata": {}, 209 | "outputs": [], 210 | "source": [ 211 | "struct AbsorbingBE <: DiffEqLinearOperator\n", 212 | " m::Int # number of interior points\n", 213 | "end\n", 214 | "\n", 215 | "*(Q::AbsorbingBE, x::Vector{Float64}) = [0.0; x; 0.0]\n", 216 | "as_array(Q::AbsorbingBE) = sparse([zeros(Q.m)'; eye(Q.m); zeros(Q.m)'])\n", 217 | "\n", 218 | "struct ReflectingBE <: DiffEqLinearOperator\n", 219 | " m::Int # number of interior points\n", 220 | "end\n", 221 | "\n", 222 | "*(Q::ReflectingBE, x::Vector{Float64}) = [x[1]; x; x[end]]\n", 223 | "as_array(Q::ReflectingBE) = sparse([1.0 zeros(Q.m-1)'; eye(Q.m); zeros(Q.m-1)' 1.0]);" 224 | ] 225 | }, 226 | { 227 | "cell_type": "code", 228 | "execution_count": 7, 229 | "metadata": {}, 230 | "outputs": [], 231 | "source": [ 232 | "function Dirichlet_BE(m::Int, bl::Float64, br::Float64)\n", 233 | " # y[1] = bl, y[end] = br\n", 234 | " L = AbsorbingBE(m)\n", 235 | " b = [bl; zeros(m); br]\n", 236 | " DiffEqAffineOperator(L, b)\n", 237 | "end\n", 238 | "\n", 239 | "function Neumann_BE(m::Int, dx::Float64, bl::Float64, br::Float64)\n", 240 | " # (y[2] - y[1])/dx = bl, (y[end] - y[end-1])/dx = br\n", 241 | " L = ReflectingBE(m)\n", 242 | " b = [-bl*dx; zeros(m); br*dx]\n", 243 | " DiffEqAffineOperator(L, b)\n", 244 | "end;" 245 | ] 246 | }, 247 | { 248 | "cell_type": "markdown", 249 | "metadata": {}, 250 | "source": [ 251 | "Examples of non-zero BC:" 252 | ] 253 | }, 254 | { 255 | "cell_type": "code", 256 | "execution_count": 8, 257 | "metadata": { 258 | "scrolled": true 259 | }, 260 | "outputs": [], 261 | "source": [ 262 | "dx = 1.0\n", 263 | "u = [1.,2.,3.,4.]\n", 264 | "QD = Dirichlet_BE(4, 10.0, 20.0)\n", 265 | "QN = Neumann_BE(4, dx, 1.0, 2.0);" 266 | ] 267 | }, 268 | { 269 | "cell_type": "code", 270 | "execution_count": 9, 271 | "metadata": {}, 272 | "outputs": [ 273 | { 274 | "data": { 275 | "text/plain": [ 276 | "6-element Array{Float64,1}:\n", 277 | " 10.0\n", 278 | " 1.0\n", 279 | " 2.0\n", 280 | " 3.0\n", 281 | " 4.0\n", 282 | " 20.0" 283 | ] 284 | }, 285 | "execution_count": 9, 286 | "metadata": {}, 287 | "output_type": "execute_result" 288 | } 289 | ], 290 | "source": [ 291 | "QD * u" 292 | ] 293 | }, 294 | { 295 | "cell_type": "code", 296 | "execution_count": 10, 297 | "metadata": {}, 298 | "outputs": [ 299 | { 300 | "data": { 301 | "text/plain": [ 302 | "6-element Array{Float64,1}:\n", 303 | " 0.0\n", 304 | " 1.0\n", 305 | " 2.0\n", 306 | " 3.0\n", 307 | " 4.0\n", 308 | " 6.0" 309 | ] 310 | }, 311 | "execution_count": 10, 312 | "metadata": {}, 313 | "output_type": "execute_result" 314 | } 315 | ], 316 | "source": [ 317 | "QN * u" 318 | ] 319 | }, 320 | { 321 | "cell_type": "markdown", 322 | "metadata": {}, 323 | "source": [ 324 | "## 1.4 Constructing operators for the Fokker-Planck equations\n", 325 | "\n", 326 | "The most general case is in section 3.5 of the write-up:\n", 327 | "\n", 328 | "$$ \\mathcal{L} = \\mu(x)\\partial_x + \\frac{\\sigma(x)^2}{2}\\partial_{xx} $$\n", 329 | "\n", 330 | "where the drift term is discretized using upwind operators as described in section 2:\n", 331 | "\n", 332 | "$$ \\mu(x)\\partial_x = \\mu^+(x)\\partial_x^+ + \\mu^-(x)\\partial_x^- $$\n", 333 | "\n", 334 | "The discretized operators are expressed as the composition of an interior stencil convolution operator `L` and boundary extrapolation operator `Q` (they are generally affine). The drift and diffusion coefficients can be expressed as diagonal matrices (wrapped in a `DiffEqArrayOperator`).\n", 335 | "\n", 336 | "The script below can be modified to represent each of the scenarios described in secition 3 of the write-up (except the mixed-BC case)." 337 | ] 338 | }, 339 | { 340 | "cell_type": "code", 341 | "execution_count": 11, 342 | "metadata": {}, 343 | "outputs": [ 344 | { 345 | "data": { 346 | "text/plain": [ 347 | "DiffEqOperatorComposition((AbsorbingBE(10), DiffEqOperatorCombination((1.0, 1.0, 1.0), (DiffEqOperatorComposition((DriftOperator(1.0, 10, true), DiffEqArrayOperator([0.887661 0.0 … 0.0 0.0; 0.0 0.213112 … 0.0 0.0; … ; 0.0 0.0 … 0.436606 0.0; 0.0 0.0 … 0.0 0.569388]))), DiffEqOperatorComposition((DriftOperator(1.0, 10, false), DiffEqArrayOperator([0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0; … ; 0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0]))), DiffEqOperatorComposition((DiffusionOperator(1.0, 10), DiffEqArrayOperator([1.89194 0.0 … 0.0 0.0; 0.0 0.597414 … 0.0 0.0; … ; 0.0 0.0 … 0.539805 0.0; 0.0 0.0 … 0.0 0.94818])))))))" 348 | ] 349 | }, 350 | "execution_count": 11, 351 | "metadata": {}, 352 | "output_type": "execute_result" 353 | } 354 | ], 355 | "source": [ 356 | "# The grid\n", 357 | "N = 10\n", 358 | "dx = 1.0\n", 359 | "xs = collect(1:N) * dx # interior nodes\n", 360 | "\n", 361 | "# Discretization of the differential operators\n", 362 | "L1p = DriftOperator(dx, N, true)\n", 363 | "L1m = DriftOperator(dx, N, false)\n", 364 | "L2 = DiffusionOperator(dx, N)\n", 365 | "\n", 366 | "# Boundary operators\n", 367 | "Q = AbsorbingBE(N)\n", 368 | "# Q = Neumann_BE(N, dx, 1.0, 2.0)\n", 369 | "\n", 370 | "# Coefficients\n", 371 | "mu = rand(N)\n", 372 | "mup = [mu[i] > 0.0 ? mu[i] : 0.0 for i in 1:N]\n", 373 | "mum = [mu[i] < 0.0 ? mu[i] : 0.0 for i in 1:N]\n", 374 | "sigma = rand(N) + 1.0\n", 375 | "\n", 376 | "# Construct the final product\n", 377 | "Ldrift = DiffEqArrayOperator(Diagonal(mup)) * L1p + DiffEqArrayOperator(Diagonal(mum)) * L1m\n", 378 | "Ldiffusion = DiffEqArrayOperator(Diagonal(sigma.^2 / 2)) * L2\n", 379 | "A = (Ldrift + Ldiffusion) * Q" 380 | ] 381 | }, 382 | { 383 | "cell_type": "markdown", 384 | "metadata": {}, 385 | "source": [ 386 | "(The standard printout for the composed operator is a bit messy. Should probably implement `Base.show` for the composition types)" 387 | ] 388 | }, 389 | { 390 | "cell_type": "code", 391 | "execution_count": 12, 392 | "metadata": {}, 393 | "outputs": [ 394 | { 395 | "data": { 396 | "text/plain": [ 397 | "10-element Array{Float64,1}:\n", 398 | " 24.4109\n", 399 | " 37.4853\n", 400 | " 45.6853\n", 401 | " 52.2336\n", 402 | " 51.9707\n", 403 | " 50.2344\n", 404 | " 46.5129\n", 405 | " 37.9023\n", 406 | " 30.9117\n", 407 | " 15.7658" 408 | ] 409 | }, 410 | "execution_count": 12, 411 | "metadata": {}, 412 | "output_type": "execute_result" 413 | } 414 | ], 415 | "source": [ 416 | "# Solve the HJBE (rI - A)u = x\n", 417 | "r = 0.05\n", 418 | "LHS = DiffEqArrayOperator(r*speye(N)) - A\n", 419 | "u = LHS \\ xs" 420 | ] 421 | }, 422 | { 423 | "cell_type": "code", 424 | "execution_count": null, 425 | "metadata": {}, 426 | "outputs": [], 427 | "source": [] 428 | } 429 | ], 430 | "metadata": { 431 | "kernelspec": { 432 | "display_name": "Julia 0.6.2", 433 | "language": "julia", 434 | "name": "julia-0.6" 435 | }, 436 | "language_info": { 437 | "file_extension": ".jl", 438 | "mimetype": "application/julia", 439 | "name": "julia", 440 | "version": "0.6.2" 441 | } 442 | }, 443 | "nbformat": 4, 444 | "nbformat_minor": 2 445 | } 446 | -------------------------------------------------------------------------------- /HeatEquation.md: -------------------------------------------------------------------------------- 1 | # Solving the Heat Equation using DiffEqOperators 2 | 3 | In this tutorial we will solve the famous heat equation using the explicit discretization on a 2D `space x time` grid. The heat equation is:- 4 | 5 | $$\frac{\partial u}{\partial t} - \frac{{\partial}^2 u}{\partial x^2} = 0$$ 6 | 7 | For this example we consider a Dirichlet boundary condition with the initial distribution being parabolic. Since we have fixed the value at boundaries (in this case equal), after a long time we expect the 1D rod to be heated in a linear manner. 8 | 9 | julia> using DiffEqOperators, DifferentialEquations, Plots 10 | julia> x = collect(-pi : 2pi/511 : pi); 11 | julia> u0 = -(x - 0.5).^2 + 1/12; 12 | julia> A = DerivativeOperator{Float64}(2,2,2pi/511,512,:Dirichlet,:Dirichlet;BC=(u0[1],u0[end])); 13 | 14 | Now solving equation as an ODE we have:- 15 | 16 | julia> prob1 = ODEProblem(A, u0, (0.,10.)); 17 | julia> sol1 = solve(prob1, dense=false, tstops=0:0.01:10); 18 | # try to plot the solution at different time points using 19 | julia> plot(x, [sol1(i) for i in 0:1:10]) 20 | 21 | **Note:** Many a times the solver may inform you that the solution is unstable. This problem is usually handled by changing the solver algorithm. Your best bet might be the `CVODE_BDE()` algorithm from the OrdinaryDiffEq.jl suite. -------------------------------------------------------------------------------- /KdV.md: -------------------------------------------------------------------------------- 1 | # Solving the Heat Equation using DiffEqOperators 2 | 3 | In this tutorial we will try to solve the famous **KdV equation** which describes the motion of waves on shallow water surfaces. 4 | The equation is commonly written as 5 | 6 | $$\frac{\partial u}{\partial t} + \frac{{\partial}^3 u}{\partial t^3} - 6*u*\frac{\partial u}{\partial t} = 0$. 7 | 8 | Lets consider the cosine wave as the initial waveform and evolve it using the equation with a [periodic boundary condition](https://journals.aps.org/prl/pdf/10.1103/PhysRevLett.15.240). This example is taken from [here](https://en.wikipedia.org/wiki/Korteweg%E2%80%93de_Vries_equation). 9 | 10 | using DiffEqOperators, DifferentialEquations, Plots 11 | x = collect(0 : 1/99 : 2); 12 | u0 = cos.(π*x); 13 | du3 = zeros(u0); # a container array 14 | function KdV(t, u, du) 15 | C(t,u,du3) 16 | A(t, u, du) 17 | copy!(du, -u.*du .- 0.022^2.*du3) 18 | end 19 | 20 | Now defining our DiffEqOperators 21 | ``` 22 | A = DerivativeOperator{Float64}(1,2,1/99,199,:periodic,:periodic); 23 | C = DerivativeOperator{Float64}(3,2,1/99,199,:periodic,:periodic); 24 | ``` 25 | 26 | Now call the ODE solver as follows:- 27 | 28 | prob1 = ODEProblem(KdV, u0, (0.,5.)); 29 | sol1 = solve(prob1, dense=false, tstops=0:0.01:10); 30 | 31 | and plot the solutions to see the waveform evolve with time. 32 | **Note:** The waveform being solved for here is non-directional unlike the many waves you might see like traveling solitons. In that case you might need to use the Upwind operators. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # PDERoadmap 2 | 3 | A repository for the discussion of PDE tooling 4 | 5 | Release: https://github.com/JuliaDiffEq/PDERoadmap/releases/download/v0.0.2/linear_operators_overview.pdf 6 | -------------------------------------------------------------------------------- /linear_operators_overview.tex: -------------------------------------------------------------------------------- 1 | %!TeX shellEscape = restricted 2 | % !TEX program = pdflatex 3 | %!TeX enableSynctex = true 4 | \documentclass[11pt]{article} 5 | \usepackage{url,amsmath,amsfonts} 6 | \usepackage[capitalise,noabbrev]{cleveref} % 7 | \crefname{equation}{}{} % 8 | %\usepackage{minted} %If require code snippets, turn back on 9 | \usepackage[nohead]{geometry} 10 | %\usepackage{tikz} 11 | %\usetikzlibrary{decorations.pathreplacing,angles,quotes,calligraphy} 12 | \usepackage{graphicx} 13 | 14 | %Some macros 15 | \newcommand{\set}[1]{\ensuremath{\left\{{#1}\right\}}} 16 | \newcommand{\R}{\ensuremath{\mathbb{R}}} 17 | \newcommand{\D}[1][]{\ensuremath{\partial_{#1}}} 18 | 19 | \geometry{left=1in,right=1in,top=0.6in,bottom=1in} 20 | \begin{document} 21 | \title{Discretizing Linear and Affine Operators Overview} 22 | \author{} 23 | \date{} 24 | \maketitle 25 | \section{Overview of Notation} 26 | Some general notation, independent of operators and discretization 27 | \begin{itemize} 28 | \item For a given variable $q$, define the notation $q^{-} \equiv \min\set{q,0}$ and $q^{+} \equiv \max\set{q,0}$, which will be useful for defining finite-differences with an upwind scheme. This can apply to vectors as well. For example, $q_m^{-} = q_m$ if $q_m < 0$ and $0$ if $q_m > 0$, and $q^{-} \equiv \set{q^{-}_m}_{m=1}^{M}$. 29 | \item Let $W_t$ be the Wiener process with the integral defined by the Ito interpretation 30 | \item Derivatives are denoted by the operator $\D$ and univariate derivatives such as $\D[x]\tilde{u}(x) \equiv u'(x)$. 31 | \item We will denote continuous functions, prior to discretization, like $\tilde{u}(x)$. The discretization of $\tilde{u}(x)$ on $x\in \R^M$ is denoted $u \in \R^M$ and the discretization of $\tilde{u}(x)$ on $\bar{x}$ is $\bar{u} \in \R^{\bar{M}}$. 32 | \item Some special matrices to help in the composition notation: 33 | \begin{itemize} 34 | \item $\mathbf{I}_N$ is the $N\times N$ identity matrix. Always drop the subscript when the dimensions are unambiguous, as it would be the same in the code 35 | \item $\mathbf{0}_N$ is the column vector of $N$ $0$s, and $\mathbf{0}_N^{\top}$ a row vector 36 | \item $\mathbf{0}_{N\times M}$ is the $N\times M$ matrix of $0$s 37 | \end{itemize} 38 | \item An \textit{affine} operator $A$ can be decomposed into a \textit{linear operator}, denoted $A_L$, and a \textit{bias} denoted $A_b$, such that for all $x$ in the domain, 39 | $$ 40 | A x = A_L x + A_b 41 | $$ 42 | \item In the case of an affine operator on a discrete space, $A : \R^M \to \R^N$, 43 | \begin{itemize} 44 | \item We can decompose this into the linear operator $A_L \in \R^{N\times M}$ and bias vector $A_b \in \R^{N}$ such that for all $x\in\R^M$, $A x = A_L x + A_b$ 45 | \end{itemize} 46 | \end{itemize} 47 | 48 | The purpose of these notes is to discretize affine or linear operators with finite differences.\footnote{These are often the infinitesimal generator of a stochastic process. See \url{https://en.wikipedia.org/wiki/Infinitesimal_generator_(stochastic_processes)} for some formulas and interpretation for diffusions, and \url{https://en.wikipedia.org/wiki/Transition_rate_matrix}} To set some notation and definitions on operators, 49 | \begin{itemize} 50 | \item Denote a typical affine operator on the space of continuous functions as $\tilde{A}$, and $\tilde{L}$ for a linear operator. When these are discretized on a particular grid, denote them as $A$ and $L$ accordingly. 51 | \item The baseline domain of the operator is on $[x^{\min}, x^{\max}]$. 52 | \item Form a grid on the domain with $M$ points, $\set{x_m}_{m=1}^{M}$ with $x_1 = x^{\min}$ and $x_{M} = x^{\max}$ when. After discretizing, we can sometimes denote the grid with the variable name, i.e. $x \equiv \set{x_m}_{m=1}^M$. In the simple case of a uniform grid, $\Delta \equiv x_{m+1} - x_m$ for all $m < M$. 53 | \item A core part of the discretization process will be to expand the variable onto the \textit{extension} (i.e. including any boundary points required for the boundary conditions). The set of boundary points will be referred to as $S_E$. If there are $M$ points in the grid, and $M_E$ points required for the boundary conditions (i.e. $|S_E| = M_E$), then define $\bar{M} = M + M_E$ as the total set of points on the extended domain. We will denote the extended domain as $\bar{x} \in \R^{\bar{M}}$. 54 | \item For any arbitrary continuous function $\tilde{y}(x)$ defined in the whole space of $x$, we define $\bar{y}$ as its discretization on the whole domain of $x$ and $y$ as the discretization only for interior points of the domain. So while $\bar{y}$ has length $\bar{M}$, $y$ has length $M$. 55 | \end{itemize} 56 | 57 | \section{General Overview of Discretization and Boundary Values}\label{sec:general} 58 | \subsection{Simple Differential Operators}\label{sec:general-simple} 59 | Take a simple linear or affine differential operator $\tilde{A}$, (possibly affine) boundary conditions $\tilde{B}$ and the function of interest $\tilde{u}(x)$. The general problem to solve is to find the $\tilde{u}(x)$ such that. 60 | % \footnote{\textbf{TODO:} Is this correct? Can the $\tilde{L}$ really be a more general $\tilde{A}$? Also, is the $\tilde{B}\tilde{u}(x) = \tilde{b}(x)$ really a good way to write down the possibly affine boundary conditions?} 61 | \begin{align} 62 | \tilde{A} \tilde{u}(x) &= 0\label{eq:A-u-DE}\\ 63 | \tilde{B} \tilde{u}(x) &= 0\label{eq:B-u-DE} 64 | \end{align} 65 | For linear $\tilde{A}$, we will denote it as $\tilde{L}$ to emphasize the fact that it's not affine. The discretization process generates the following objects: 66 | \begin{itemize} 67 | \item $B\in \R^{M_E \times \bar{M}}$ is the (possibly affine) \textit{boundary condition operator}, which satisfies the equation 68 | \begin{align} 69 | B \bar{u} = \mathbf{0}_{M_E}\label{eq:B_operator_block} 70 | \end{align} 71 | for any $\bar{u}$ in the space of functions that satisfy the discretized boundary conditions. \footnote{Notice that $B$ is not necessarily unique. The choice of $B$ is exactly the choice of boundary value discretization. For instance, choosing to do first or second order Neumann border conditions is simply the choice of the operator $B$.} For affine $B$, we can write out \cref{eq:B_operator_block} as 72 | \begin{align} 73 | B_L\bar{u} = -B_b\label{eq:B_operator_block_expanded} 74 | \end{align} 75 | \item $R\in \R^{M\times \bar{M}}$ is the linear \textit{restriction operator} which is defined by the domain. It removes columns which are not in the interior. It fulfills 76 | \begin{align} 77 | R \bar{u} = u \label{eq:R_operator} 78 | \end{align} 79 | 80 | \item $Q^B : \R^M \to \R^{\bar{M}}$ is the (potentially affine) \textit{boundary extrapolation operator} associated with $B$. The operator $Q^B$ is defined as fulfilling the following relationships (keeping in mind that $Q^B$ is affine and $R$ is linear) 81 | \begin{align} 82 | Q^B R\bar{u} &= \bar{u}\label{eq:Q_operator_1}\\ 83 | B Q^B u &= \mathbf{0}_{M_E} \label{eq:Q_operator_2} 84 | \end{align} 85 | To give intuition, for any $\bar{u}$ that satisfies the border conditions:\footnote{Notice that $Q = R^{-1}$ if R is square, and this is only true as maps on functions which satisfy the boundary. Furthermore, in order for \cref{eq:Q_operator_1} to hold on trivial $u$, we need that the interior of $Q$ is identity, so it is defined by its first and last rows.} \cref{eq:Q_operator_1} says that finding the restriction of the function and then extrapolating to extension yields the same function, and \cref{eq:Q_operator_2} says that the boundary extrapolation of the interior of the function, $u$, fulfills the boundary value. 86 | \item $A : \R^{\bar{M}} \to \R^M$ is the (possibly affine) \textit{stencil operator}. It maps the extended domain to the interior by applying a stencil, which is determined by the derivative operator and the numerical differentiation scheme. As with the continous case, we will denote the stencil operator as $L$ if it is linear. 87 | \item The \textit{discretized derivative operator} is $A^B : \R^M \to \R^M$. We use the $B$ superscript to emphasize the operator's dependence on the boundary condition. 88 | 89 | The operator is composed as $A^B = AQ^B$. The intuition is that first $Q^B$ is applied to the interior points to add the ``ghost nodes'' corresponding to the boundary condition, and then the stencil operator $A$ is applied to the whole domain, including the ghost nodes. $A^B$ is in general affine if $A$ and/or $Q^B$ are affine, and linear if both of them are linear, in which case we will denote it as $L^B$ to emphasize the linearity. 90 | \end{itemize} 91 | 92 | \subsection{Composite Differential Operators}\label{sec:general-composite} 93 | The discretization of a composite differential operator follows the same framework as \ref{sec:general-simple}. However, instead of deriving the stencil operator $A$ directly it is customary to think of it as being composed of several component operators. For this document we will only consider the simplest of compositions: linear combinations. For more complex examples, especially high-dimensional ones, we may encounter more advanced form of compositions such as tensor products/sums. 94 | 95 | Let $\tilde{A}$ be the linear combination of several differential operators: $\tilde{A} = \tilde{A}_1 + \tilde{A}_2 + \cdots + \tilde{A}_n$. To get the discretized entities of \ref{sec:general-simple}, we will proceed as follows: 96 | 97 | \begin{itemize} 98 | \item First, discretize each $\tilde{A}_k$ separately and get its $B_k$, $R_k$, $Q^{B_k}$ and $A_k$ along with the extended domain $\bar{u}_k$.\footnote{Even if the physical boundary condition is the same for all components, the required boundary points may still be different because of the order of differentiation and/or numerical approximation. For example, a second-order approximation to $\D[xx]$ requires one boundary point at each end, whereas a fourth-order approximation requires two.} 99 | 100 | \item Let $\bar{u}$ be the union of all the $\bar{u}_k$, i.e. the largest common extended domain. Usually this is just the $\bar{u}_k$ corresponding to the ``biggest'' component operator. We need to work out the $B$, $R$ and $Q_B$ for $\bar{u}$, which is trivial if it coincides with one of the $\bar{u}_k$. 101 | 102 | \item The composed stencil operator defined on $\bar{u}$ is $A = A_1E_1 + A_2E_2 + \cdots + A_nE_n$, with $E_k \in \R^{M_{E,k} \times M_E}$ the \textit{extension operator} for $A_k$. In practice we don't need $E_k$ explicitly as $A_kE_k$ can be constructed easily by padding $A_k$ with zeros at columns corresponding to boundary points that are not used. For example, if $L_1 \in \R^{2 \times 3}$ but the common extended domain include an extra boundary point at the end, then 103 | \begin{align*} 104 | L_kE_k &= \begin{bmatrix} 105 | L_k & \mathbf{0}_2 106 | \end{bmatrix} \in \R^{2 \times 4} 107 | \end{align*} 108 | which extends $L_k$ to the common extended domain.\footnote{For affine $A_k$, the rule of affine algebra applies: $(A_kE_k)_L = A_{k,L}E_k$, $(A_kE_k)_b = A_{k,b}$. In other words, the bias term remains unchanged and the linear part gets padded with zeros.} The extended stencil operators can now be linearly combined as they are of the same shape. 109 | 110 | Alternatively, $E_k$ can be viewed as the \textit{restriction operator} mapping $\bar{u}$ to $\bar{u}_k$ by stripping away unused boundary points. In other words, instead of viewing $A_kE_k\bar{u}$ as $(A_kE_k)\bar{u}$ we can view it as $A_k(E_k\bar{u}) = A_k\bar{u}_k$.\footnote{This can make a big difference in application depending on the details of implemnetation: whether $A_k$ is sparese or not, how the grid is implemented, etc.} 111 | 112 | \item The discretized derivative operator is still $A^B = AQ^B$. 113 | \end{itemize} 114 | 115 | \section{Time-Invariant Stochastic Process Examples}\label{sec:examples} 116 | \subsection{Definitions and Notation for Examples} 117 | Let $x_t$ be a stochastic process for a univariate function defined on a continuous domain $x \in (x^{\min}, x^{\max})$ where $-\infty < x^{\min} < x^{\max} < \infty$. We will assume throughout that the domain is time-invariant. The infinitesimal generator associated with $x_t$ is denoted as $\tilde{L}^s$. 118 | 119 | Let the payoff in state $x$ be $\tilde{p}(x)$ (for a simple example, we can choose $\tilde{p}(x) = x$). If payoffs are discounted at rate $r > 0$, then the simple HJBE for $\tilde{u}(x)$ is 120 | \begin{align} 121 | r \tilde{u}(x) &= \tilde{p}(x) + \tilde{L}^s \tilde{u}(x)\label{eq:general-stationary-HJBE} 122 | \end{align} 123 | We can define another operator $\tilde{L} \equiv r\tilde{I} - \tilde{L}^s$ where $\tilde{I}$ is the identity operator. With this we can rearrange \cref{eq:general-stationary-HJBE} as 124 | \begin{align} 125 | \tilde{L} \tilde{u}(x) &= \tilde{p}(x) 126 | \end{align} 127 | 128 | Common boundary conditions for $\tilde{L}$ are: 129 | \begin{itemize} 130 | \item $\D[x]\tilde{u}(x^{\min}) = b^{\min}$, $\D[x]\tilde{u}(x^{\max}) = b^{\max}$ for \textit{Neumann boundaries}. 131 | \item $\tilde{u}(x^{\min}) = b^{\min}$, $\tilde{u}(x^{\max}) = b^{\max}$ for \textit{Dirichlet boundaries}. 132 | \item Mixed boundaries, e.g. $\tilde{u}(x^{\min}) = b^{\min}$, $\D[x]\tilde{u}(x^{\max}) = b^{\max}$ for Dirichlet boundary at the bottom and Neumann boundary at the top. 133 | \end{itemize} 134 | It is customary to refer to homogeneous (i.e. $b^{\min} = b^{\max} = 0$) Neumann boundaries as \textit{reflecting barriers/boundaries}, and homogeneous Dirichlet boundaries as \textit{absorbing barriers/boundaries}. 135 | 136 | The stencil operators used by the examples are: 137 | \begin{itemize} 138 | \item Second-order approximation to $\D[xx]$ (central differencing): 139 | \begin{align} 140 | L_2 &\equiv \frac{1}{\Delta^2}\begin{bmatrix} 141 | 1&-2&1&\dots&0&0&0\\ 142 | 0&1&-2&\dots&0&0&0\\ 143 | \vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\vdots\\ 144 | 0&0&0&\dots&-2&1&0\\ 145 | 0&0&0&\cdots&1&-2&1 146 | \end{bmatrix}_{M\times(M+2)}\label{eq:L-2} 147 | \end{align} 148 | $L_2$ requires one boundary point at each end of the domain. 149 | 150 | \item First-order approximation to $\D[x]$: 151 | \begin{align} 152 | L_1 &\equiv \frac{1}{\Delta}\begin{bmatrix} 153 | -1&1&0&\dots&0&0&0\\ 154 | 0&-1&1&\dots&0&0&0\\ 155 | \vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\vdots\\ 156 | 0&0&0&\dots&-1&1&0\\ 157 | 0&0&0&\cdots&0&-1&1 158 | \end{bmatrix}_{M\times(M+1)}\label{eq:L-1} 159 | \end{align} 160 | $L_1$ requires one boundary point at the top or bottom end of the domain. For \textit{forward differencing} the extra point is at the top, while for \textit{backward differencing} it is at the bottom. To avoid ambiguity we shall refer to the forward and backward stencils as $L^+_1$ and $L^-_1$ respectively. 161 | \end{itemize} 162 | 163 | In most examples the largest common extended domain is the same one used by $L_2$, which has $M_E = 2$ with $S_E = \{0, M+1\}$. The restriction operator is 164 | \begin{align} 165 | R_2 &\equiv \begin{bmatrix} \mathbf{0}_M & \mathbf{I}_M & \mathbf{0}_M\end{bmatrix}\label{eq:R-2} 166 | \end{align} 167 | 168 | The boundary operators for this domain with the most common boundary conditions are: 169 | \begin{itemize} 170 | \item Neumann boundary at both sides: 171 | \begin{align} 172 | B_{RR,L} &\equiv \begin{bmatrix} 173 | 1&-1&0&\dots&0&0&0\\ 174 | 0&0&0&\cdots&0&-1&1 175 | \end{bmatrix}_{2\times (M+2)}\quad 176 | B_{RR,b} = \begin{bmatrix}b^{\min}\Delta\\-b^{\max}\Delta\end{bmatrix}\label{eq:B-RR} 177 | \end{align} 178 | 179 | \item Dirichlet boundary at both sides: 180 | \begin{align} 181 | B_{AA,L} &\equiv \begin{bmatrix} 182 | 1&0&0&\dots&0&0&0\\ 183 | 0&0&0&\cdots&0&0&1 184 | \end{bmatrix}_{2\times (M+2)}\quad 185 | B_{AA,b} = \begin{bmatrix}-b^{\min}\\-b^{\max}\end{bmatrix}\label{eq:B-AA} 186 | \end{align} 187 | 188 | \item Dirichlet boundary at the bottom and Neumann boundary at the top: 189 | \begin{align} 190 | B_{AR,L} &\equiv \begin{bmatrix} 191 | 1&0&0&\dots&0&0&0\\ 192 | 0&0&0&\cdots&0&-1&1 193 | \end{bmatrix}_{2\times (M+2)}\quad 194 | B_{AR,b} = \begin{bmatrix}-b^{\min}\\-b^{\max}\Delta\end{bmatrix}\label{eq:B-AR} 195 | \end{align} 196 | \end{itemize} 197 | For homogeneous boundaries (i.e. reflecting/absorbing) the $B$ operators are linear. In this case we will slightly abuse the notation and simply drop the $L$ subscript when referring to the linear operator. 198 | 199 | The corresponding boundary extrapolation operators are: 200 | \begin{align} 201 | Q^{B_{RR}}_L &\equiv \begin{bmatrix}1&0&\cdots&0&0\\&&\mathbf{I}_M&&\\0&0&\cdots&0&1\end{bmatrix}\quad 202 | Q^{B_{RR}}_b = \begin{bmatrix}-b^{\min}\Delta\\\mathbf{0}_M\\b^{\max}\Delta\end{bmatrix}\label{eq:Q-RR}\\ 203 | Q^{B_{AA}}_L &\equiv \begin{bmatrix}0&0&\cdots&0&0\\&&\mathbf{I}_M&&\\0&0&\cdots&0&0\end{bmatrix}\quad 204 | Q^{B_{AA}}_b = \begin{bmatrix}b^{\min}\\\mathbf{0}_M\\b^{\max}\end{bmatrix}\label{eq:Q-AA}\\ 205 | Q^{B_{AR}}_L &\equiv \begin{bmatrix}0&0&\cdots&0&0\\&&\mathbf{I}_M&&\\0&0&\cdots&0&1\end{bmatrix}\quad 206 | Q^{B_{AR}}_b = \begin{bmatrix}b^{\min}\\\mathbf{0}_M\\b^{\max}\Delta\end{bmatrix}\label{eq:Q-AR} 207 | \end{align} 208 | Again when the boundaries are homogeneous, we will drop the $L$ subscript. 209 | 210 | \subsection{Stationary HJBE with Reflecting Barriers}\label{sec:simple-reflecting-example} 211 | Take the stochastic process 212 | $$ 213 | d x_t = d W_t 214 | $$ 215 | with reflecting barriers at $x^{\min}$ and $x^{\max}$. The partial differential operator (infinitesimal generator) associated with the stochastic process is 216 | \begin{equation} 217 | \tilde{L}^s \equiv \frac{1}{2}\D[xx]\label{eq:L-s-nodrift} 218 | \end{equation} 219 | 220 | For this process, we derive below all of the matrices of \cref{sec:general} and the system of equations to solve for $\tilde{u}(x)$ in \cref{eq:general-stationary-HJBE}. We still have \textbf{to do}: 221 | \begin{itemize} 222 | \item Check that the code \url{operator_examples\simple_stationary_HJBE_reflecting.jl} is correct 223 | \end{itemize} 224 | Consider 225 | \begin{align} 226 | r \tilde{u}(x) &= \tilde{L}^s \tilde{u}(x) - x\label{HJBE_reflecting_barriers_PDE}\\ 227 | \intertext{Define the operator $\tilde{L}$ and rearrange,} 228 | \tilde{L} \tilde{u}(x) &= x\\ 229 | \tilde{L}&\equiv r - \frac{1}{2}\partial_{xx}\label{HJBE_operator_first_example} 230 | \end{align} 231 | We first consider a one-dimension case where $x\in [x^{\min},x^{\max}]$. Let $M_E = 2$, $S_E = \{1,\bar{M}\}$, thereby $\Delta = \frac{x^{max}-x^{min}}{\bar{M}}$, and $\bar{M} = M+2$. From \cref{HJBE_operator_first_example} and given \cref{eq:L-2} from the previous section, the matrix form of operator $\tilde{L}$ can be defined, given as $A = \frac{L_2}{2}$. 232 | 233 | By reflecting barriers, we can define $B$ just like as $B_{RR}$ in \cref{eq:B-RR} and then we have 234 | \begin{equation} 235 | B\bar{u} = \begin{bmatrix} 236 | 0\\ 237 | 0 238 | \end{bmatrix} \label{B_reflecting} 239 | \end{equation} 240 | Therefore, $\bar{u}(x_0) = \bar{u}(x_1)$ and $\bar{u}(x_{M+1}) = \bar{u}(x_M)$. It is important to notice that our choice for $B$ defines the linear relationship between the interior points and the boundary conditions. 241 | 242 | Moreover, $R$ is again defined as in \cref{eq:R-2} and $Q$ is defined by $Q R\bar{u}\equiv Q_L R\bar{u}+Q_b = \bar{u}$, where 243 | \begin{equation} 244 | Q_L = \begin{bmatrix} 245 | 1& 0&\dots&0&0\\ 246 | & & \mathbf{I}_M & & \\ 247 | 0&0&\dots&0&1 248 | \end{bmatrix}%_{\bar{M}\times M} 249 | \quad , \text{ } Q_b = \begin{matrix} 250 | \mathbf{0}_{\bar{M}} 251 | \end{matrix}\label{Q_reflecting} 252 | \end{equation} 253 | 254 | Then, it is easy to verify that \cref{affine_relation_1} and \cref{affine_relation_2} hold in this case. Additionally, it is worth to note that, since $Q_b = \mathbf{0}_{\bar{M}}$, $Q$ is a linear operator. 255 | 256 | To solve $\bar{u}(x)$, we first solve interiors according to \cref{HJBE_reflecting_barriers_PDE} and the definition of operator $Q$, which provides us with two conditions: 257 | \begin{align} 258 | L \bar{u} &= x\label{solve_u_hat_cond_1}\\ 259 | Q R\bar{u} &= Q u = Q_L u+Q_b = \bar{u}\label{solve_u_hat_cond_2} 260 | \end{align} 261 | where the discretization of the linear operator $\tilde{L}$, defined on the extension, is 262 | \begin{equation} 263 | L \equiv ([\mathbf{0}_{M} \text{ } \mathbf{I}_{M} \text{ } \mathbf{0}_{M}] r - L_2) 264 | \label{L_definition} 265 | \end{equation} 266 | Note that in doing the composition of the operator in \cref{L_definition}, we need to combine the $L_2$ (defined on $M+2$ for the extension) with $r I$, defined on $M$ points. In order to compose these, we need to extend the $r I$ operator with $0$s. 267 | % \begin{align} 268 | % P &= \begin{bmatrix} 269 | % 0&r&0&\cdots&0&0&0\\ 270 | % 0&0&r&\cdots&0&0&0\\ 271 | % \vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\vdots\\ 272 | % 0&0&0&\cdots&r&0&0\\ 273 | % 0&0&0&\cdots&0&r&0\\ 274 | % \end{bmatrix}_{M\times\bar{M}}-A \\ 275 | % &= \begin{bmatrix} 276 | % -1&2(1+r\Delta^2)&-1&\dots&0&0&0\\ 277 | % 0&-1&2(1+r\Delta^2)&\dots&0&0&0\\ 278 | % \vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\vdots\\ 279 | % 0&0&0&\dots&2(1+r\Delta^2)&-1&0\\ 280 | % 0&0&0&\cdots&-1&2(1+r\Delta^2)&-1 281 | % \end{bmatrix}\frac{1}{2\Delta^{2}} 282 | % \end{align} 283 | Substitute \cref{solve_u_hat_cond_2} into \cref{solve_u_hat_cond_1}, we get 284 | \begin{align} 285 | L Q u = L (Q_L u+Q_b) = x 286 | \end{align} 287 | Since $Q_b = 0$ in this case, interiors are solved as 288 | \begin{align} 289 | u = (L Q_L)^{-1}x\label{solve_u_hat_in_terms_of_interiors} 290 | \end{align} 291 | %Then, by \cref{solve_u_hat_cond_2}, we can solve the extended state vector by 292 | % \begin{align} 293 | % \bar{u} = Q_L R\bar{u}+Q_b\label{solve_u_hat_in_terms_of_interiors} 294 | % \end{align} 295 | 296 | \subsection{Stationary HJBE with a Lower Absorbing Barrier} 297 | Take the stochastic process 298 | $$ 299 | d x_t = d W_t 300 | $$ 301 | with an absorbing barrier at $x^{\min}$, and a reflecting barrier at $x^{\max}$. Again, the partial differential operator (infinitesimal generator) associated with the stochastic process is \cref{eq:L-s-nodrift} 302 | 303 | For the absorbing barrier, when solving for the HJBE assume that $u(x^{\min}) = b^{\min}$ and $u'(x^{\max}) = 0$. 304 | 305 | For this process, we derive below all of the matrices of \cref{sec:general} and the system of equations to solve for $\tilde{u}(x)$ in \cref{eq:general-stationary-HJBE}. 306 | % We still have \textbf{to do}: 307 | % \begin{itemize} 308 | % \item Check that the code \url{operator_examples\simple_stationary_HJBE_reflecting.jl} is correct 309 | % \end{itemize} 310 | 311 | Again, we first consider a one-dimension case where $x\in [x^{\min},x^{\max}]$. Let $M_E = 2$, $S_E = \{1,\bar{M}\}$, thereby $\Delta = \frac{x^{\max}-x^{\min}}{\bar{M}}$, and $\bar{M} = M+2$. From \cref{HJBE_operator_first_example} and given \cref{eq:L-2} from the \cref{sec:examples}, the matrix form of operator $\tilde{L}$ is again defined \cref{L_definition} 312 | 313 | According to lower absorbing barrier, we can define $B$ just like as $B_{AR}$ in \cref{eq:B-AR} and then we have 314 | \begin{equation} 315 | B\bar{u} = \begin{bmatrix} 316 | b^{\min}\\ 317 | 0 318 | \end{bmatrix} \equiv b 319 | \end{equation}\\ 320 | Therefore, $\bar{u}(x^{\min}) = x^{\min}$ and $\bar{u}(x_{M+1}) = \bar{u}(x_M)$. 321 | 322 | Moreover, $R$ is again defined as in \cref{eq:R-2} and $Q$ is defined by defined by $Q R\bar{u}\equiv Q_L R\bar{u}+Q_b = \bar{u}$, where 323 | \begin{equation} 324 | Q_L = \begin{bmatrix} 325 | & & \mathbf{0}_{1\times M} & & \\ 326 | & & \mathbf{I}_M & & \\ 327 | 0&0&\dots&0&1 328 | \end{bmatrix}%_{\bar{M}\times M} 329 | \quad, \text{ } Q_b = \begin{bmatrix} 330 | b^{\min}\\ 331 | 0\\ 332 | \vdots\\ 333 | 0 334 | \end{bmatrix}%_{\bar{M}\times 1} 335 | \end{equation} 336 | 337 | Then, it is easy to verify that \cref{affine_relation_1} and \cref{affine_relation_2} hold in this case. Additionally, it is worth to note that, if the absorbing boundary was of Dirichlet$_0$ type, then $b^{\min} = 0$ and $Q_b = \mathbf{0}_{\bar{M}}$ and $Q$ would be a linear operator. 338 | 339 | In this case, $L$ is given by \cref{L_definition}. According to conditions \cref{solve_u_hat_cond_1} and \cref{solve_u_hat_cond_2}, again we get 340 | \begin{align} 341 | L(Q_L R \bar{u}+Q_b) = x 342 | \end{align} 343 | With $Q_b\neq 0$, we can solve interiors as 344 | \begin{align} 345 | R\bar{u} = (L Q_L)^{-1}(x-L Q_b) 346 | \end{align} 347 | Then, the extended state vector $\bar{u}$ again can be similarly solved by \cref{solve_u_hat_in_terms_of_interiors}. 348 | \subsection{Stationary HJBE with Only Drift} 349 | Now, do the same after adding in constant drift (and manually choose the correct upwind direction!) 350 | $$ 351 | d x_t = \mu dt 352 | $$ 353 | With a generator 354 | \begin{equation} 355 | \tilde{L}^s \equiv \mu \D[x]\label{eq:L-s-drift} 356 | \end{equation} 357 | 358 | For this process, we derive below all of the matrices of \cref{sec:general}, paying special attention to the upwind direction, and the system of equations to solve for $\tilde{u}(x)$ in \cref{eq:general-stationary-HJBE}. We still have \textbf{to do}: 359 | \begin{itemize} 360 | \item Write julia code to solve for $\tilde{u}(x)$ with the grid 361 | \item Check these for $\mu < 0$ and $\mu > 0$. 362 | \end{itemize} 363 | 364 | Since the choice of the first difference depends on the sign of drift $\mu$, we define $\mu^- =\min\{\mu, 0\}$ and $\mu^- =\max\{\mu, 0\}$. 365 | Consider 366 | \begin{align} 367 | \tilde{L} \tilde{u}(x) &= x\label{HJBE_PDE_with_drifts}\\ 368 | \text{where }\tilde{L}&\equiv r - \mu\partial_{x} 369 | \end{align} 370 | We first consider a one-dimension case where $x\in [x^{\min},x^{\max}]$. Let $M_E = 2$ and thereby $\Delta = \frac{x^{\max}-x^{\min}}{\bar{M}}$ and $\bar{M} = M+2$. 371 | 372 | Considering $\mu>0$, we must choose the forward first difference, thus the matrix form of operator $\tilde{L}$ can be defined as $L = \mu L_1^+$. Analogously, for $\mu<0$, we must choose the backward first difference, which implies that the matrix form of operator $\tilde{L}$ can be defined as $L = \mu L_1^-$% as in \cref{eq:L-1m}. 373 | 374 | Considering the absorbing barriers, we can define $B$ just like as $B_{AA}$ in \cref{eq:B-AA} and then we have 375 | \begin{equation} 376 | B\bar{u} = \begin{bmatrix} 377 | b^{\min}\\ 378 | b^{\max} 379 | \end{bmatrix} 380 | \end{equation}.\\ 381 | Therefore, $\bar{u}(x_0) = x^{\min}$ and $\bar{u}(x_{M+1}) = x^{\max}$. 382 | 383 | Moreover, $R$ is again defined as in \cref{eq:R-2} and $Q$ is defined by $Q R\bar{u}\equiv Q_L R\bar{u}+Q_b = \bar{u}$, where 384 | \begin{equation} 385 | Q_L = \begin{bmatrix} 386 | \mathbf{0}_{1\times M} \\ 387 | \mathbf{I}_M \\ 388 | \mathbf{0}_{1\times M} 389 | \end{bmatrix}%_{\bar{M}\times M} 390 | \quad, \text{ } Q_b = \begin{bmatrix} 391 | b^{\min}\\ 392 | 0\\ 393 | \vdots\\ 394 | b^{\max} 395 | \end{bmatrix}%_{\bar{M}\times 1} 396 | \end{equation} 397 | 398 | Then, it is easy to verify that \cref{affine_relation_1} and \cref{affine_relation_2} hold in this case. 399 | 400 | 401 | % In this example, the matrix $P$ in \cref{solve_u_hat_cond_1} becomes 402 | % \begin{align} 403 | % P = \frac{1}{\Delta} 404 | % \begin{bmatrix} 405 | % \mu^-&r\Delta-(\mu^--\mu^+)&-\mu^+&\dots&0&0&0\\ 406 | % 0&\mu^-&r\Delta-(\mu^--\mu^+)&\dots&0&0&0\\ 407 | % \vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\vdots\\ 408 | % 0&0&0&\dots&r\Delta-(\mu^--\mu^+)&-\mu^+&0\\ 409 | % 0&0&0&s&\mu^-&r\Delta-(\mu^--\mu^+)&-\mu^+ 410 | % \end{bmatrix}%_{\bar{M}\times\bar{M}} 411 | 412 | Similarly, as $L$ is defined by \cref{L_definition} and the interiors are solved by \cref{solve_u_hat_cond_1} and \cref{solve_u_hat_cond_2}: 413 | \begin{align} 414 | u = (L Q_L)^{-1}(x-L Q_b) 415 | \end{align} 416 | Thus, the extended state vector $\bar{u}$ again can be similarly solved by \cref{solve_u_hat_in_terms_of_interiors}. 417 | 418 | \subsection{Stationary HJBE with Reflecting Barriers and Drift} 419 | Now, do the same after adding in constant drift (and manually choose the correct upwind direction!) 420 | $$ 421 | d x_t = \mu dt + \sigma d W_t 422 | $$ 423 | With a generator 424 | $$ 425 | \tilde{L}^s \equiv \mu \D[x] + \frac{\sigma^2}{2}\D[xx] 426 | $$ 427 | 428 | For this process, we derive below all of the matrices of \cref{sec:general}, paying special attention to the upwind direction, and the system of equations to solve for $\tilde{u}(x)$ in \cref{eq:general-stationary-HJBE}. We still have \textbf{to do}: 429 | \begin{itemize} 430 | \item Write julia code to solve for $\tilde{u}(x)$ with the grid 431 | \item Check these for $\mu < 0$ and $\mu > 0$. 432 | \end{itemize} 433 | 434 | We first consider a one-dimension case where $x\in [x^{\min},x^{\max}]$. Let $M_E = 2$ and thereby $\Delta = \frac{x^{\max}-x^{\min}}{\bar{M}}$ and $\bar{M} = M+2$. 435 | 436 | By combining operators from previous sections, in this case $L^s$ is defined as 437 | \begin{align} 438 | L^s &= \mu L_1^{-} +\frac{\sigma^2}{2}L_2\quad\text{if }\mu<0\\ 439 | L^s &=\mu L_1^{+} +\frac{\sigma^2}{2}L_2\quad\text{if }\mu>0\\ 440 | \intertext{And the composed operator,} 441 | L &\equiv ([\mathbf{0}_{M} \text{ } \mathbf{I}_{M} \text{ } \mathbf{0}_{M}] r - L^s 442 | \end{align} 443 | 444 | Since barriers are reflecting, we can have the same boundary conditions as what we had in the case with reflecting barriers but no drifts. Hence, operators $R$, $B$ and $Q$ are defined by \cref{eq:R-2}, \cref{B_reflecting} and \cref{Q_reflecting}, respectively. Also, with some simple algebras, we can easily verify that \cref{affine_relation_1} and \cref{affine_relation_2} hold in this case. 445 | 446 | 447 | % In this example, the matrix $P$ in \cref{solve_u_hat_cond_1} becomes 448 | % \begin{align} 449 | % P = \frac{1}{\Delta^{2}} 450 | % \begin{bmatrix} 451 | % -X&r\Delta^2-Y&-Z&\dots&0&0&0\\ 452 | % 0&-X&r\Delta^2-Y&\dots&0&0&0\\ 453 | % \vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\vdots\\ 454 | % 0&0&0&\dots&r\Delta^2-Y&-Z&0\\ 455 | % 0&0&0&\cdots&-X&r\Delta^2-Y&-Z 456 | % \end{bmatrix} 457 | % \end{align} 458 | 459 | Given $L$ defined by \cref{L_definition}, the rest steps for solving interiors, $u$, and the extended state vector $\bar{u}$, are similar with what we did for previous examples. 460 | 461 | 462 | \subsection{Stationary Bellman Equation with Reflecting Barriers State Varying Drift/Variance} 463 | Now, do the same after adding in constant drift (and manually choose the correct upwind direction!) 464 | $$ 465 | d x_t = \tilde{\mu}(x_t) dt + \tilde{\sigma}(x_t) d W_t 466 | $$ 467 | With a generator 468 | $$ 469 | \tilde{L}^s \equiv \tilde{\mu}(x) \D[x] + \frac{\tilde{\sigma}(x)^2}{2}\D[xx] 470 | $$ 471 | 472 | For this process, we derive below all of the matrices of \cref{sec:general}, paying special attention to the upwind direction, and the system of equations to solve for $\tilde{u}(x)$ in \cref{eq:general-stationary-HJBE}. We still have \textbf{to do}: 473 | \begin{itemize} 474 | \item Write julia code to solve for $\tilde{u}(x)$ with the grid. 475 | \begin{itemize} 476 | \item Choose a $\tilde{u}(x)$ and $\tilde{\sigma}(x)$ functions, consider using geometric brownian motion as a test. That is: 477 | \begin{align} 478 | \tilde{L}^s &\equiv \bar{\mu} x \D[x] + \frac{\bar{\sigma}^2}{2}x^2\D[xx] 479 | \end{align} 480 | \end{itemize} 481 | \end{itemize} 482 | 483 | We first consider a one-dimension case where $x\in [x^{\min},x^{\max}]$. Let $M_E = 2$ and thereby $\Delta = \frac{x^{\max}-x^{\min}}{\bar{M}}$ and $\bar{M} = M+2$. 484 | 485 | Again, consider $\bar{\mu}^- = \min\{\bar{\mu}, 0\}$ and $\bar{\mu}^+ = \max\{\bar{\mu}, 0\}$. 486 | 487 | This case is similar with the previous one but with variable drift and variance. By combining operators $L$ from previous sections, in this case $L^s, L$ is defined as 488 | \begin{align} 489 | L^s = \mu(x)^- L_1^-+\mu(x)^+L_1^+ +\sigma(x) L_2 490 | \end{align} 491 | where 492 | \begin{align*} 493 | \mu(x)^- &=\begin{bmatrix} 494 | \bar{\mu}^-x_1&0&\cdots&0\\ 495 | 0&\bar{\mu}^-x_2&\cdots&0\\ 496 | \vdots&\vdots&\ddots&\vdots\\ 497 | 0&0&\cdots&\bar{\mu}^-x_M 498 | \end{bmatrix}\\ 499 | \mu(x)^+ &=\begin{bmatrix} 500 | \bar{\mu}^+x_1&0&\cdots&0\\ 501 | 0&\bar{\mu}^+x_2&\cdots&0\\ 502 | \vdots&\vdots&\ddots&\vdots\\ 503 | 0&0&\cdots&\bar{\mu}^+x_M 504 | \end{bmatrix}\\ 505 | \sigma(x)&=\begin{bmatrix} 506 | \frac{(\bar{\sigma}x_1)^2}{2}&0&\cdots&0\\ 507 | 0&\frac{(\bar{\sigma}x_2)^2}{2}&\cdots&0\\ 508 | \vdots&\vdots&\ddots&\vdots\\ 509 | 0&0&\cdots&\frac{(\bar{\sigma}x_M)^2}{2} 510 | \end{bmatrix} 511 | \intertext{And the composed operator,} 512 | L &\equiv ([\mathbf{0}_{M} \text{ } \mathbf{I}_{M} \text{ } \mathbf{0}_{M}] r - L^s 513 | \end{align*} 514 | 515 | Again, since barriers are reflecting, we can have the same boundary conditions as what we had in the case with reflceting barriers but no drifts. Hence, operators $R$, $B$ and $Q$ are defined by \cref{eq:R-2}, \cref{B_reflecting} and \cref{Q_reflecting}, respectively. Also, with some simple algebras, we can easily verify that \cref{affine_relation_1} and \cref{affine_relation_2} hold in this case. 516 | 517 | % In this example, the matrix $P$ in \cref{solve_u_hat_cond_1} becomes 518 | % \begin{align} 519 | % P = \frac{1}{\Delta^{2}} \begin{bmatrix} 520 | % -X_1&r\Delta^2-Y_1&-Z_1&\dots&0&0&0\\ 521 | % 0&-X_2&r\Delta^2-Y_2&\dots&0&0&0\\ 522 | % \vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\vdots\\ 523 | % 0&0&0&\dots&r\Delta^2-Y_{M-1}&-Z_{M-1}&0\\ 524 | % 0&0&0&\cdots&-X_M&r\Delta^2-Y_M&-Z_M 525 | % \end{bmatrix} 526 | % \end{align} 527 | 528 | Again, given $L$ defined by \cref{L_definition}, the remaining steps for solving interiors, $u$, and the extended state vector, $\bar{u}$, are similar with what we did for previous examples. 529 | 530 | \appendix 531 | \section{Extended Equation and Gaussian Elimination} 532 | \subsection{Overview} 533 | The document focuses on solving the discretized equation on the interior $u$, and the information for the boundary condition is encoded in the boundary extrapolation operator $Q^B$. Alternatively, we can consider the discretized equation on the extended domain $\bar{u}$. We wish to demonstrate in this section that the two equations are indeed equivalent. 534 | 535 | The starting point is again the continuous equation \cref{eq:A-u-DE} and \cref{eq:B-u-DE}. We discretize the domain and get the stencil operator $L$ and boundary operator $B$. For simplicity, we shall assume $L$ to be linear in this section, but the boundary conditions need not be homogenous. The discretized equations on the extended domain $\bar{u}$ are then: 536 | \begin{itemize} 537 | \item From \cref{eq:A-u-DE}: $L\bar{u} = p$. 538 | \item From \cref{eq:B-u-DE}: $B_L\bar{u} = -B_b$ (this is just \cref{eq:B_operator_block_expanded}). 539 | \end{itemize} 540 | 541 | The two linear equations have the same number of unknowns, so they can stacked up: 542 | \begin{align} 543 | \begin{bmatrix} L \\ B_L \end{bmatrix} \bar{u} &= 544 | \begin{bmatrix} p \\ -B_b \end{bmatrix}\label{eq:extended-stack} 545 | \end{align} 546 | 547 | In the examples we shall see that the extended equation \cref{eq:extended-stack} can be transformed to an equation on the interior by way of Gaussian elimination. We wish to show that the $Q^B$ matrix can be naturally generated using the elementary matrices associated with the elimination process, however the algebra is not in place yet. Nevertheless, for a known $Q^B$ the equivalence between the equations can be proved easily.\footnote{Note that the equation \cref{eq:extended-stack} does not necessarily have a unique solution. Nevertheless the reduced equation from Gaussian elimination should be the same we get from $LQ^Bu = p$.} 548 | 549 | \subsection{Example 1: Diffusion with Reflecting Boundaries}\label{sec:appendixA-example1} 550 | \subsubsection{Continuous Equation} 551 | We consider a slightly modified example from \cref{sec:simple-reflecting-example}. The stochastic process in question is $dx_t = \sqrt{2}dW_t$ and a reflecting boundary is present at $x^{\min} = 0$ and $x^{\max} = 2$. The infinitesimal generator is then simply $\tilde{L}^s = \D[xx]$. 552 | 553 | For discount rate $r > 0$ and payoff $\tilde{p}(x)$, the stationiary HJBE is then 554 | \begin{align} 555 | \tilde{L}u(x) &= \tilde{p}(x)\\ 556 | \tilde{L} &\equiv r\tilde{I} - \D[xx] \\ 557 | \D[x]\tilde{u}(0) &= \D[x]\tilde{u}(2) = 0 558 | \end{align} 559 | Here $\tilde{I}$ is the (continuous) identity operator. 560 | 561 | \subsubsection{Discretized Equation} 562 | We'll be using a uniform grid with $\Delta x = 1$, in other words $M = 3$ interior nodes. A second-order approximation to $\D[xx]$ is used which require $M_E = 2$ addition nodes, and a total of $\bar{M} = 5$ nodes on the whole domain. The discretized grid entities are: 563 | \begin{align} 564 | p &= \begin{bmatrix} \tilde{p}(0) & \tilde{p}(1) & \tilde{p}(2)\end{bmatrix}^{\top}\\ 565 | u &= \begin{bmatrix} \tilde{u}(0) & \tilde{u}(1) & \tilde{u}(2)\end{bmatrix}^{\top}\\ 566 | \bar{u} &= \begin{bmatrix} \tilde{u}(-1) & \tilde{u}(0) & \tilde{u}(1) & \tilde{u}(2) & \tilde{u}(3)\end{bmatrix}^{\top} 567 | \end{align} 568 | 569 | For the discretization of $\tilde{L} = r\tilde{I} - \D[xx]$, since it is composed of two parts we shall follow the procedure in \ref{sec:general-composite}: 570 | \begin{itemize} 571 | \item The scaling operator $r\tilde{I}$ is discretized simply as $rI_M$, defined on the interior. There's no boundary points associated with it. 572 | 573 | \item The discrete stencil operator for $\D[xx]$ is defined in \cref{eq:L-2}, specifically for this case it is 574 | \begin{align} 575 | L_2 &= \begin{bmatrix} 576 | 1 & -2 & 1 & 0 & 0\\ 577 | 0 & 1 & -2 & 1 & 0\\ 578 | 0 & 0 & 1 & -2 & 1\\ 579 | \end{bmatrix}\in \R^{M \times \bar{M}} 580 | \end{align} 581 | $L_2$ needs an extra boundary point at each end. Its associated boundary operator is given in \cref{eq:B-RR}. In particular, for this case we have a linear $B$ with 582 | \begin{align} 583 | B_L &= \begin{bmatrix} 584 | 1 & -1 & 0 & 0 & 0\\ 585 | 0 & 0 & 0 & -1 & 1 586 | \end{bmatrix}\in\R^{M_E \times \bar{M}},\quad 587 | B_b = \begin{bmatrix}0 \\ 0\end{bmatrix}\label{eq:B-example-A1} 588 | \end{align} 589 | 590 | \item To extend $rI_M$ to $L_2$'s boundary, we simply pad a column of zeros to each end of the identity operator: 591 | \begin{align} 592 | I^E &\equiv \begin{bmatrix} 593 | 0 & 1 & 0 & 0 & 0\\ 594 | 0 & 0 & 1 & 0 & 0\\ 595 | 0 & 0 & 0 & 1 & 0 596 | \end{bmatrix} 597 | \end{align} 598 | 599 | \item The composed stencil is then 600 | \begin{align} 601 | L &= rI^E - L_2\\ 602 | &= \begin{bmatrix} 603 | -1 & 2 + r & -1 & 0 & 0\\ 604 | 0 & -1 & 2+r & -1 & 0\\ 605 | 0 & 0 & -1 & 2+r & -1 606 | \end{bmatrix}\label{eq:L-example1} 607 | \end{align} 608 | \end{itemize} 609 | 610 | \subsubsection{Solving the Stacked Equation} 611 | % Note that if this matrix is singular, then the problem is not well-specified and no solutions exist? Given the solution, we can define the restriction operator to remove one from the first and one from the last as, 612 | % \begin{align} 613 | % R &\equiv E_{11}^{\top} = \begin{bmatrix}\textbf{0}_M & I_ M & \textbf{0}_M\end{bmatrix}\in\R^{M\times \bar{M}} 614 | % \intertext{Then find the appropriate value as,} 615 | % u &= R \bar{u} 616 | % \intertext{These could be combined,} 617 | % u &= R \begin{bmatrix} L \\ B 618 | % \end{bmatrix}^{-1} \begin{bmatrix} p \\ b \end{bmatrix} 619 | % \end{align} 620 | % Note: the extension and restriction operators are transposes in this case? Also, note that some sort of pseudo-inverse of the $R$ matrix is its transpose. That is $R R^{\top} = I$. 621 | 622 | Substituting $L$, $p$ and $B$ into the stacked extended equation \cref{eq:extended-stack}, we get 623 | \begin{align} 624 | \begin{bmatrix} 625 | -1 & 2 + r & -1 & 0 & 0\\ 626 | 0 & -1 & 2+r & -1 & 0\\ 627 | 0 & 0 & -1 & 2+r & -1\\ 628 | 1 & -1 & 0 & 0 & 0\\ 629 | 0 & 0 & 0 & -1 & 1 630 | \end{bmatrix} \bar{u} &= \begin{bmatrix} p_1 \\ p_2 \\ p_3 \\ 0 \\ 0\end{bmatrix}\label{eq:stacked-example-1} 631 | \end{align} 632 | 633 | We will now show that the rows corresponding to $B$ can be used in Gaussian elimination to reduce the system to one defined in the interior. First, add the 4th row to the first row to get 634 | \begin{align} 635 | \begin{bmatrix} 636 | 1-1 & -1+2+r & -1 & 0 & 0\\ 637 | 0 & -1 & 2+r & -1 & 0\\ 638 | 0 & 0 & -1 & 2+r & -1\\ 639 | 1 & -1 & 0 & 0 & 0\\ 640 | 0 & 0 & 0 & -1 & 1 641 | \end{bmatrix} \bar{u} &= \begin{bmatrix} p_1 \\ p_2 \\ p_3 \\ 0 \\ 0\end{bmatrix} 642 | \intertext{Next, add the 5th row to the 3rd row and simplify,} 643 | \begin{bmatrix} 644 | 0 & 1+r & -1 & 0 & 0\\ 645 | 0 & -1 & 2+r & -1 & 0\\ 646 | 0 & 0 & -1 & 1+r & 0\\ 647 | 1 & -1 & 0 & 0 & 0\\ 648 | 0 & 0 & 0 & -1 & 1 649 | \end{bmatrix} \begin{bmatrix}\bar{u}(-1)\\ \bar{u}(0) \\ \bar{u}(1)\\ \bar{u}(2) \\ \bar{u}(3) \end{bmatrix} 650 | &= \begin{bmatrix} p_1 \\ p_2 \\ p_3 \\ 0 \\ 0\end{bmatrix} 651 | \intertext{Notice that we have eliminated the extension nodes from all of the equations involving the $L$. Consequently, can just take out the sub-matrix between columns 2-4 and rows 1-3 to get} 652 | \begin{bmatrix} 653 | 1+r & -1 & 0\\ 654 | -1 & 2+r & -1\\ 655 | 0 & -1 & 1+r 656 | \end{bmatrix}u 657 | &= \begin{bmatrix} p_1 \\ p_2 \\ p_3\end{bmatrix}\label{eq:reduced-example-1} 658 | \end{align} 659 | 660 | On the other hand, for the reflecting boundaries, we know from \cref{eq:Q-RR} the boundary extrapolation operator is 661 | \begin{align} 662 | Q^B &\equiv \begin{bmatrix} 663 | 1 & 0 & 0\\ 664 | 1 & 0 & 0\\ 665 | 0 & 1& 0\\ 666 | 0 & 0& 1\\ 667 | 0 & 0& 1 668 | \end{bmatrix}\label{eq:Q-example} 669 | \end{align} 670 | and the associated discretized equation on the interior is $LQ^Bu = p$. It is easy to check that plugging $L$ from \cref{eq:L-example1} and $Q^B$ from \cref{eq:Q-example} also gives \cref{eq:reduced-example-1}, which proves the equivalence between the two equations. 671 | 672 | \subsection{Example 2: Diffusion and Drift with Reflecting Boundaries}\label{sec:appendixA-example2} 673 | \subsubsection{Continuous Equation} 674 | Let's add a drift term to \ref{sec:appendixA-example1} with constant rate $\mu < 0$ and keep everything else the same. The stochastic process is now $d x_t = \mu d t + \sqrt{2} d W_t$ and the corresponding stationary HJBE becomes 675 | \begin{align} 676 | \tilde{L} u(x) &= \tilde{p}(x)\\ 677 | \tilde{L} &\equiv r\tilde{I} - \D[xx] - \mu \D[x]\\ 678 | \D[x]\tilde{u}(0) &= \D[x]\tilde{u}(2) = 0 679 | \end{align} 680 | 681 | \subsubsection{Discretized Equation} 682 | Again we follow the procedure of \ref{sec:general-composite} for the discretization of $\tilde{L}$. The biggest common extended domain is still the one used by $L_2$ so the parts regarding $rI^E$, $L_2$ and $B$ in \ref{sec:appendixA-example1} remain the same. 683 | 684 | For $\D[x]$, because $\mu < 0$ backward difference should be used and that gives the stencil operator $L^-_1$ \cref{eq:L-1}, which in this case is 685 | \begin{align} 686 | L^-_1 &= \begin{bmatrix} 687 | -1 & 1 & 0 & 0 \\ 688 | 0 & -1 & 1 & 0 \\ 689 | 0 & 0 & -1 & 1 690 | \end{bmatrix} 691 | \end{align} 692 | The extended domain for $L^-_1$ only needs a boundary point at the bottom. We need to extend it to the whole domain, which include one extra node at the top end. This can be achieved by padding a column of zeros to $L^-_1$: 693 | \begin{align} 694 | L^{-E}_1 &= \begin{bmatrix} 695 | -1 & 1 & 0 & 0 & 0\\ 696 | 0 & -1 & 1 & 0 & 0\\ 697 | 0 & 0 & -1 & 1 & 0 698 | \end{bmatrix} 699 | \end{align} 700 | 701 | Finally, the composed operator is 702 | \begin{align} 703 | L &= rI^E - L_2 - \mu L^{-E}_1 \\ 704 | &= \begin{bmatrix} 705 | -1 + \mu & 2 -\mu + r & -1 & 0 & 0\\ 706 | 0 & -1 + \mu & 2 - \mu +r & -1 & 0\\ 707 | 0 & 0 & -1 + \mu & 2 - \mu +r & -1 708 | \end{bmatrix} 709 | \end{align} 710 | and the stacked equation \cref{eq:extended-stack} is 711 | \begin{align} 712 | \begin{bmatrix} 713 | -1 + \mu & 2 -\mu + r & -1 & 0 & 0\\ 714 | 0 & -1 + \mu & 2 - \mu +r & -1 & 0\\ 715 | 0 & 0 & -1 + \mu & 2 - \mu +r & -1\\ 716 | 1 & -1 & 0 & 0 & 0\\ 717 | 0 & 0 & 0 & -1 & 1 718 | \end{bmatrix} \bar{u} &= \begin{bmatrix} p_1 \\ p_2 \\ p_3 \\ 0 \\ 0\end{bmatrix} 719 | \end{align} 720 | 721 | \subsubsection{Solving the Stacked Equation} 722 | Again we solve the stacked equation \cref{eq:extended-stack} using Gaussian elimination on the $B$ rows. First add $(1-\mu)$ times row 4 to row 1, and then add row 5 to row 3. This gives 723 | \begin{align} 724 | \begin{bmatrix} 725 | 0 & 1 + r & -1 & 0 & 0\\ 726 | 0 & -1 + \mu & 2 - \mu +r & -1 & 0\\ 727 | 0 & 0 & -1 + \mu & 1 - \mu +r & 0\\ 728 | 1 & -1 & 0 & 0 & 0\\ 729 | 0 & 0 & 0 & -1 & 1 730 | \end{bmatrix} \bar{u} &= \begin{bmatrix} p_1 \\ p_2 \\ p_3 \\ 0 \\ 0\end{bmatrix} 731 | \end{align} 732 | Extract the interior of the matrix to get 733 | \begin{align} 734 | \begin{bmatrix} 735 | 1 + r & -1 & 0\\ 736 | -1 + \mu & 2 - \mu +r & -1\\ 737 | 0 & -1 + \mu & 1 - \mu +r 738 | \end{bmatrix} u &= \begin{bmatrix} p_1 \\ p_2 \\ p_3\end{bmatrix}\label{eq:reduced-example-2} 739 | \end{align} 740 | 741 | The $Q^B$ in this example is the same as \cref{eq:Q-example}. It is easy to check that the interior equation $LQ^Bu = p$ also gives \cref{eq:reduced-example-2}, again confirming that the extended equation gives the same resuls. 742 | 743 | \subsection{Example 3: Diffusion with Inhomogeneous Boundaries}\label{sec:appendixA-example3} 744 | \subsubsection{Continuous Equation} 745 | Let's consider \ref{sec:appendixA-example1} again but change the boudnary conditions to be inhomogeneous. The stationary HJBE: 746 | \begin{align} 747 | \tilde{L}u(x) &= \tilde{p}(x)\\ 748 | \tilde{L} &\equiv r\tilde{I} - \D[xx] \\ 749 | \D[x]\tilde{u}(0) &= b^{\min}\\ 750 | \D[x]\tilde{u}(2) &= b^{\max} 751 | \end{align} 752 | 753 | \subsubsection{Discretized Equation} 754 | The discretized $L$ and $p$ are the same as \ref{sec:appendixA-example1}. Since the boundary conditions are now inhomogeneous $B$ is no longer linear. Recalling \cref{eq:B-RR}, for this example we have 755 | \begin{align} 756 | B_L = \begin{bmatrix} 757 | 1 & -1 & 0 & 0 & 0\\ 758 | 0 & 0 & 0 & -1 & 1 759 | \end{bmatrix}\in\R^{M_E \times \bar{M}}\quad 760 | B_b = \begin{bmatrix}b^{\min}\\-b^{\max}\end{bmatrix} 761 | \end{align} 762 | and the stacked equation \cref{eq:extended-stack} is now 763 | \begin{align} 764 | \begin{bmatrix} 765 | -1 & 2 + r & -1 & 0 & 0\\ 766 | 0 & -1 & 2+r & -1 & 0\\ 767 | 0 & 0 & -1 & 2+r & -1\\ 768 | 1 & -1 & 0 & 0 & 0\\ 769 | 0 & 0 & 0 & -1 & 1 770 | \end{bmatrix} \bar{u} &= \begin{bmatrix} p_1 \\ p_2 \\ p_3 \\ -b^{\min} \\ b^{\max}\end{bmatrix} 771 | \end{align} 772 | 773 | \subsubsection{Solving the Stacked Equation} 774 | Since the left hand side coefficient matrix is the same, we can use the same Gaussian elimination procedure as \ref{sec:appendixA-example1}. This gives the reduced equation 775 | \begin{align} 776 | \begin{bmatrix} 777 | 1+r & -1 & 0\\ 778 | -1 & 2+r & -1\\ 779 | 0 & -1 & 1+r 780 | \end{bmatrix}u 781 | &= \begin{bmatrix} 782 | p_1 - b^{\min}\\ p_2 \\ p_3 + b^{\max} 783 | \end{bmatrix}\label{eq:reduced-example-3} 784 | \end{align} 785 | 786 | For the $LQ^Bu = p$ route, $Q^B$ is now affine and from \cref{eq:Q-RR} we have 787 | \begin{align} 788 | Q^B_L = \begin{bmatrix} 789 | 1 & 0 & 0 \\ 790 | 1 & 0 & 0 \\ 791 | 0 & 1 & 0 \\ 792 | 0 & 0 & 1 \\ 793 | 0 & 0 & 1 794 | \end{bmatrix}\quad 795 | Q^B_b = \begin{bmatrix} 796 | -b^{\min}\\0\\0\\0\\b^{\max} 797 | \end{bmatrix} 798 | \end{align} 799 | and the equation is 800 | \begin{align} 801 | LQ^B_Lu &= p - LQ^B_b 802 | \end{align} 803 | substituting $L$, $p$, $Q^B_L$ and $Q^B_b$, we get back \cref{eq:reduced-example-3}, again proving the equivalence. 804 | 805 | \section{Affine Relations and Intuition} 806 | The following provides intuition on the relationships above:\footnote{\textbf{TODO: Fernando/Steven} I think you will need to rewrite this with the modified notation and go through it carefully. I don't quite get it, and the notation was slightly different than the rest of the text... Also, I think that abusing notation for the discretized and non-discretized is part of the problem. We might want to rewrite this a little after the expanding operator setup is dine.} 807 | \begin{itemize} 808 | \item Define $S_E$ as the set of non interior grid point indexes, e.g, if we have a univariate problem with just three non-interior point, let's say, the first one and the last two grid points, then $M_E = 3$ and $S_E = \{1,\bar{M}-1,\bar{M}\}$\footnote{Notice that, by construction, the number of elements in $S_E$ is always $M_E$}. 809 | \item Now let's focus on solving an expression like $\bar{u} = A \bar{u}$ where $A$ is affine.\footnote{\textbf{Fernando/Steven}: Is this a particular operator you have in mind from our setup, or a general affine operator you are going through? Point it out from above, and differentiate the $\tilde{A}$ from the discretized $A$} Consider, with some abuse of notation, that such expression means both the discretized and non-discretized forms. Define, for any arbitrary matrix J with at least $M_E$ columns, that $J[:,S_E]$ is a submatrix whose columns are the concatenated vectors $J[:,s]$, $\forall s \in S_E$. 810 | Then we have two relations\footnote{We could not find a way to clearly show two relations above are correct, but some intuitions are provided in the text}: 811 | % 812 | \begin{align} 813 | A [:,S] \left(B[:,S_E]^{-1} b \right) = A Q_b\label{affine_relation_1}\\ 814 | (A -A [:,S_E] (B[:,S_E]^{-1} B)) R^{\top} = A Q_L\label{affine_relation_2} 815 | \end{align} 816 | 817 | Our intuition is that the main idea here is using interiors to recover a relation that boundary conditions should satisfy. Since $Q_b$ is a length $\bar{M}$ vector containing zeros excepts two ends, the two non-zero elements in $Q_b$ capture partial information of boundary nodes (the part that is ``independent'' of interiors).\footnote{\textbf{Typo From Chris} 818 | \cref{affine_relation_1} is actually defined from \cref{affine_relation_2} which is defined from the next one. Looking at it like that, it's clear to see the error 819 | since 89 is just saying 820 | $(A - AQb)*R^T = AQL$ 821 | substitute in 88 for AQb) 822 | you see the substitution was done incorrectly 823 | has a big B instead of a little b. 824 | } 825 | 826 | Recall \cref{eq:B_operator_block}, so $\left(B[:,S_E]^{-1} b \right)$ recovers the ``independent" part of boundary nodes. Then it is reasonable to expect that \cref{affine_relation_1} holds. 827 | 828 | Multiply both sides of \cref{affine_relation_2} by $\bar{u}$, we can roughly rewrite the relation as 829 | \begin{equation} 830 | (A \bar{u}-A Q_b) R^{\top} = A Q_L u 831 | \end{equation} 832 | so $A \bar{u}-A Q_b$ will be a discretized $\bar{u}$ which contains the entire information of interiors and the rest part of boundary information that is not covered by $A Q_b$. 833 | 834 | However, we are not sure if $R$ should exist on the left of \cref{affine_relation_2} since R by definition is a restriction operator and $(A \bar{u}-A Q_b) R^{\top}$ only contains information from interiors. 835 | % % we already solved this by changing R place in the equation. It seems to be right, since Steven reported that now both affine relationships hold for the examples 836 | % Also the size of the LHS of \cref{affine_relation_2} is $M\times \bar{M}$, but the size of the RHS is $\bar{I}\times I$ . 837 | % 838 | For now, while we work on better understanding those expressions, we will take them as given. 839 | 840 | 841 | Given those relationships, in order to solve the differential equation, we now only have to solve for the interior. Otherwise, including the boundary values would imply having more points than there are degrees of freedom in the problem - thus making the numerical solution unstable. Moreover, the boundaries are given directly by the interior $\bar{u} = Q u$. 842 | 843 | Therefore, we actually want to solve $\bar{u} = A Q u$. Notice that the discretized A maps from the full domain to the interior\footnote{Notice that the PDE is only defined on the interior}. Notably, that means it's not square. Additionally, consider that, as described above, since $Q$ is in general affine, thus: 844 | \begin{equation} 845 | \bar{u} = A Q_L u + A Q_b 846 | \end{equation} 847 | Those are the linear equations which define the ODE or whose solution is the solution to the PDE. 848 | \end{itemize} 849 | 850 | \end{document} 851 | 852 | % Algebra has some issues. B, R and Q are wrong. 853 | % \section{The multivariate case} 854 | % \paragraph{Grids}Consider the multivariate function of $x = (x^1,...,x^n )$: 855 | % \begin{itemize} 856 | % \item Consider we have $\bar{I}_j$ points, $\set{x^j_i}_{i=1}^{\bar{I}_j}$%I think here you mean \bar{I}_j instead of \bar{I}. 857 | % with $x^j_1 = x^{\min}^j$ and $x^j_I = x^{\max}^j$ when $x^j \in [x^{\min}^j, x^{\max}^j]$ for each $j \in \{1,...,n\}$. After discretizing, we will often denote the grid with the variable name, i.e. $x^j \equiv \set{x^j_i}_{i=1}^{\bar{I}_j}$ 858 | % In the simple case of a uniform grid, $\Delta^j \equiv x_{i+1}^j - x_i^j$ for all $i < \bar{I}^j$. 859 | % \item When we discretize a function, use the function name without arguments to denote the vector. i.e. $\bar{u}(x)$ discretized on a grid $\set{x_i}_{i=1}^{\bar{I}}$ is $\bar{u} \equiv \set{\bar{u}(x_i)}_{i=1}^{\bar{I}} \in \R^{\bar{I}}$, where $\bar{I} = \sum_{j=1}^n \bar{I}_j$ and $I = \sum_{j = 1}^n I_j$. 860 | % Use $i=1,... I_j$ as the grid in the interior. Moreover, define $I_{j,L}$ and $I_{j,H}$ as the non interior grid points. Therefore, $\bar{I}_j = I_j + I_{j,L} + I_{j,H}$. % Again, I think you mean I_j's in above cases. 861 | % Notice that the solution to the discretized function is $u \in R^{\bar{I}}$ if stationary. 862 | % 863 | % We construct the discretized $\bar{u}$ by column stacking the discretized $\bar{u}_{x^j}$ on each grid $j \in \{1,...,n\}$: 864 | % \begin{equation} 865 | % \bar{u} = \begin{bmatrix} 866 | % \bar{u}_{x^1}\\ 867 | % \bar{u}_{x^2}\\ 868 | % \vdots\\ 869 | % \bar{u}_{x^n}\\ 870 | % \end{bmatrix}\label{u_def} 871 | % \end{equation} 872 | % 873 | % 874 | % 875 | % \item For any arbitrary variable $y(x)$ defined in the whole space of $x$, we define $\bar{y}$ as its discretization on the whole domain of $x$ and $y$ as the discretization only for interior points of the domain. So while $\bar{y}$ has length $\bar{I}$, $y$ has length $I$. 876 | % \end{itemize} 877 | % 878 | % \subsection{General Overview of Discretization and Boundary Values}\label{sec:general} 879 | % \textbf{TODO:} Explain the $R, Q, B, A$ etc. for the general notation from \url{https://github.com/JuliaDiffEq/DifferentialEquations.jl/issues/260}. The idea here is to make sure we understand everything about the ghost nodes, boundary values, etc. 880 | % \begin{itemize} 881 | % \item A is defined as the discretization of the partial differential operator in use. 882 | % 883 | % \item B is the boundary condition operator. For $N$ dimensions, we can define B as a block matrix in the form: 884 | % \begin{equation} 885 | % B\cdot \bar{u} \equiv 886 | % %\begin{bmatrix} 887 | % %\text{B}_{x^{1},L} & ... & \text{B}_{x^{N},L}\\ 888 | % %\text{B}_{x^{2},H} & ... & \text{B}_{x^{N},H} 889 | % %\end{bmatrix} 890 | % \begin{bmatrix} 891 | % \text{B}_{x^{1},L} & \mathbf{0}_{1,L} & ... & \mathbf{0}_{1,L}\\ 892 | % \text{B}_{x^{1},H} & \mathbf{0}_{1,H} & ... & \mathbf{0}_{1,H}\\ 893 | % \mathbf{0}_{2,L} & \text{B}_{x^{2},L} & ... & \mathbf{0}_{2,L}\\ 894 | % \mathbf{0}_{2,H} & \text{B}_{x^{2},H} & ... & \mathbf{0}_{2,H}\\ 895 | % \vdots & \vdots & \ddots & \vdots \\ 896 | % \mathbf{0}_{N,L} & ... & \mathbf{0}_{N,L} & \text{B}_{x^{N},L} & \\ 897 | % \mathbf{0}_{N,H} & ... & \mathbf{0}_{N,H} & \text{B}_{x^{N},H} & \\ 898 | % \end{bmatrix} 899 | % \bar{u} 900 | % = 901 | % \begin{bmatrix} 902 | % \text{z}_{{1},L}\\ 903 | % \text{z}_{{1},H}\\ 904 | % \text{z}_{{2},L}\\ 905 | % \text{z}_{{2},H}\\ 906 | % \vdots\\ 907 | % \text{z}_{{N},L}\\ 908 | % \text{z}_{{N},H}\\ 909 | % \end{bmatrix} 910 | % %\begin{bmatrix} 911 | % %\text{z}_{1, L}&\text{z}_{2, L}&\dots&\text{z}_{n, L}\\ 912 | % %\text{z}_{1, H}&\text{z}_{2, H}&\dots&\text{z}_{n, H} 913 | % %\end{bmatrix} 914 | % \label{eq:B_operator_block} 915 | % \end{equation} 916 | % for any $\bar{u}$ in the space of functions that satisfy the boundary conditions and where B$_{x^j,k}$ is a block matrix satisfying B$_{x^j,k}\cdot \bar{u}_{x^j} = \text{z}_{j,k}$ for $k \in \{L,H\}$ and $j = 1, 2,\dots n$. The size of B$_{x^j,k}$ is $I_{j,k} \times \bar{I}_j$. 917 | % 918 | % Notice that B is not necessarily unique. The choice of B is exactly the choice of boundary value discretization. For instance, choosing to do first or second order Neumann border conditions is simply the choice of the operator B. The size of $B$ is $(I_A + I_H) \times \bar{I}$. 919 | % 920 | % \item R is the restriction operator which is defined by the domain. It removes columns which are not in the interior. 921 | % For an univariate process, we have: 922 | % \begin{equation} 923 | % R\cdot \bar{u} \equiv\set{u(x_j)}_{j \in \{1,...,I\}} \in \R^{I} \label{eq:R_operator} 924 | % \end{equation} 925 | % Notice that R has size $I \times \bar{I}$ and R is defined as 926 | % \begin{equation} 927 | % R_j = %\underbrace{ 928 | % \begin{bmatrix} 929 | % 0&\dots&0&1&0&\dots&0&0&\dots&0\\ 930 | % 0&\dots&0&0&1&\dots&0&0&\dots&0\\ 931 | % \vdots&\ddots&\vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\ddots&\vdots\\ 932 | % 0&\dots&0&0&0&\dots&1&0&\dots&0 933 | % \end{bmatrix}%}_{I_{j, L}\qquad+\qquad I_j\qquad+\qquad I_{j, H}} 934 | % \end{equation} 935 | % \hspace{5.1cm} \begin{tikzpicture}[every text node part/.style={align=center}] 936 | % 937 | % \draw[black, thick, decoration = {brace,mirror,raise=0cm},decorate] (8,0) -- node[below=6pt] {$I_{j,L}$} (9.6,0); 938 | % 939 | % \draw[black, thick, decoration = {brace,mirror,raise=0cm}, decorate] 940 | % (10,0) -- node[below=6pt] {$I_{j}$} (12.1,0); 941 | % 942 | % \draw[black, thick, decoration = {brace,mirror,raise=0cm},decorate] (12.5,0) -- node[below=6pt] {$I_{j,H}$} (14.1,0); 943 | % 944 | % \end{tikzpicture} 945 | % 946 | % \item Q is the operator that is defined as 947 | % \begin{equation} 948 | % Q \cdot R\cdot\bar{u} = \bar{u}\label{eq:Q_operator_1} 949 | % \end{equation} 950 | % for any $\bar{u}$ that satisfies the border conditions. (Notice that $Q = R^{-1}$ if R is square, and this is only true as maps on functions which satisfy the boundary). Formally, we define $Q$ as a matrix of size $\bar{I} \times I$ given by: 951 | % \begin{equation} 952 | % Q = \begin{bmatrix} 953 | % Q_1 & \mathbf{0}_1 & ... & \mathbf{0}_1\\ 954 | % \mathbf{0}_2 & Q_2 & ... & \mathbf{0}_2\\ 955 | % \vdots & \vdots & \ddots & \vdots \\ 956 | % \mathbf{0}_n & \mathbf{0}_n & ... & Q_n\\ 957 | % \end{bmatrix}\label{Q_def} 958 | % \end{equation} 959 | % where, for any arbitrary $j$, $Q_j$ is a partitioned matrix of size $\bar{I_j} \times I_j$ given by: 960 | % \begin{equation} 961 | % Q_j = \begin{bmatrix} 962 | % & & Q_{j,L} & & \\ 963 | % & & \mathbb{I}_{(I_j \times I_j)} \\ 964 | % & & Q_{j,H} & & \\ 965 | % \end{bmatrix}\label{Qj_def} 966 | % \end{equation} 967 | % whose submatrices $Q_{j,L}$ and $Q_{j,H}$ (of size $I_{j,L} \times I_j$ and $I_{j,H} \times I_j$, respectively) elements depend on the border conditions and the submatrix $\mathbb{I}_{(I_j\times I_j)}$ is an identity matrix of dimension $I_j$. 968 | % 969 | % Q also has a second relation: 970 | % \begin{equation} 971 | % B\cdot Q\cdot u = 972 | % %\begin{bmatrix} 973 | % %\text{B}_{x^{1},L} & ... & \text{B}_{x^{N},L}\\ 974 | % %\text{B}_{x^{2},H} & ... & \text{B}_{x^{N},H} 975 | % %\end{bmatrix} 976 | % \begin{bmatrix} 977 | % \text{B}_{x^{1},L} & \mathbf{0}_{1,L} & ... & \mathbf{0}_{1,L}\\ 978 | % \text{B}_{x^{1},H} & \mathbf{0}_{1,H} & ... & \mathbf{0}_{1,H}\\ 979 | % \mathbf{0}_{2,L} & \text{B}_{x^{2},L} & ... & \mathbf{0}_{2,L}\\ 980 | % \mathbf{0}_{2,H} & \text{B}_{x^{2},H} & ... & \mathbf{0}_{2,H}\\ 981 | % \vdots & \vdots & \ddots & \vdots \\ 982 | % \mathbf{0}_{N,L} & ... & \mathbf{0}_{N,L} & \text{B}_{x^{N},L} & \\ 983 | % \mathbf{0}_{N,H} & ... & \mathbf{0}_{N,H} & \text{B}_{x^{N},H} & \\ 984 | % \end{bmatrix} 985 | % \bar{u} 986 | % = 987 | % \begin{bmatrix} 988 | % \text{z}_{{1},L}\\ 989 | % \text{z}_{{1},H}\\ 990 | % \text{z}_{{2},L}\\ 991 | % \text{z}_{{2},H}\\ 992 | % \vdots\\ 993 | % \text{z}_{{N},L}\\ 994 | % \text{z}_{{N},H}\\ 995 | % \end{bmatrix} 996 | % %\begin{bmatrix} 997 | % %\text{z}_{1, L}&\text{z}_{2, L}&\dots&\text{z}_{n, L}\\ 998 | % %\text{z}_{1, H}&\text{z}_{2, H}&\dots&\text{z}_{n, H} 999 | % %\end{bmatrix} 1000 | % %\end{bmatrix} 1001 | % \label{eq:Q_operator_2} 1002 | % \end{equation} 1003 | % 1004 | % basically saying that it's a map from discretizations of functions in the interior to functions which satisfy the border conditions. In order to hold on trivial $u$, we need that the interior of $Q$ is identity, so it is defined by its first and last rows.\\ 1005 | % Q is actually in general affine, meaning that $Q\cdot \bar{u} = Q_A\cdot \bar{u} + Q_b$. For example, the definition given of inner product make $Q\cdot x=b$ in an iterative solver converge to $Q_A\cdot x = b - Q_b$. From the relations, $Q_A$ is $\bar{I}\times I$ and $Q_b$ is of length $\bar{I}$. In order for $Q\cdot R\cdot u = u$ to hold for trivial $\bar{u}$, we need that $Q_b$ is zero except in the boundary rows. 1006 | % 1007 | % \item Now let's focus on solving an expression like $\bar{u}^d = A \bar{u}$, where the superscript $d$ means the discretized version of the variable. Consider, with some abuse of notation, that such expression means both the discretized and non-discretized forms. Then we have two relations\footnote{We could not find a way to clearly show two relations above are correct, but some intuitions are provided in the text}: 1008 | % 1009 | % \begin{align} 1010 | % [A[:, 1:I_L] A[:,\bar{I}-I_H+1:\bar{I}]]\cdot\left([B[:,1:I_L] B[:,\bar{I}-I_H+1:\bar{I}]]^{-1}\begin{bmatrix} 1011 | % \text{z}_{x^{1},L} & ... & \text{z}_{x^{N},L}\\ 1012 | % \text{z}_{x^{2},H} & ... & \text{z}_{x^{N},H} 1013 | % \end{bmatrix}\right) = A\cdot Q_b\label{affine_relation_1} 1014 | % \end{align} 1015 | % \begin{align} 1016 | % (A-[A[:,1:I_L] A[:,\bar{I}-I_H+1:\bar{I}]]\cdot([B[:,1:I_L] B[:,\bar{I}-I_H+1:\bar{I}]]^{-1} B))\cdot R^{\top} = A\cdot Q_A\label{affine_relation_2} 1017 | % \end{align} 1018 | % Our intuition is that the main idea here is using interiors to recover a relation that boundary conditions should satisfy. Since $Q_b$ is a length $\bar{I}$ vector containing zeros excepts two ends, the two non-zero elements in $Q_b$ capture partial information of boundary nodes (the part that is ``independent'' of interiors). Recall \cref{eq:B_operator_block}, so $\left([B[:,1:I_L] B[:,\bar{I}-I_H+1:\bar{I}]]^{-1}\begin{bmatrix} 1019 | % \text{z}_{x^{1},L} & ... & \text{z}_{x^{N},L}\\ 1020 | % \text{z}_{x^{2},H} & ... & \text{z}_{x^{N},H} 1021 | % \end{bmatrix}\right)$ recovers the ``independent" part of boundary nodes. Then it is reasonable to expect that \cref{affine_relation_1} holds. 1022 | % 1023 | % Multiply both sides of \cref{affine_relation_2} by $\bar{u}$, we can roughly rewrite the relation as 1024 | % \begin{equation} 1025 | % R(A\cdot \bar{u}-A\cdot Q_b) = A\cdot Q_A\cdot \bar{u} 1026 | % \end{equation} 1027 | % so $A\cdot \bar{u}-A\cdot Q_b$ will be a discretized $\bar{u}$ which contains the entire information of interiors and the rest part of boundary information that is not covered by $A\cdot Q_b$. 1028 | % 1029 | % However, we are not sure if $R$ should exist on the left of \cref{affine_relation_2} since R by definition is a restriction operator and $R(A\cdot \bar{u}-A\cdot Q_b)$ only contains information from interiors. Also the size of the LHS of \cref{affine_relation_2} is $I\times \bar{I}$, but the size of the RHS is $\bar{I}\times I$. 1030 | % For now, while we work on better understanding those expressions, we will take them as given. 1031 | % 1032 | % 1033 | % Given those relationships, in order to solve the differential equation, we now only have to solve for the interior. Otherwise, including the boundary values would imply having more points than there are degrees of freedom in the problem - thus making the numerical solution unstable. Moreover, the boundaries are given directly by the interior $\bar{u} = Qu$. 1034 | % 1035 | % Therefore, we actually want to solve $\bar{u}^d = AQu$. Notice that the discretized A maps from the full domain to the interior\footnote{Notice that the PDE is only defined on the interior}. Notably, that means it's not square. Additionally, consider that, as described above, since $Q$ is in general affine, thus: 1036 | % \begin{equation} 1037 | % \bar{u} = AQ_Au + AQ_b 1038 | % \end{equation} 1039 | % Those are the linear equations which define the ODE or whose solution is the solution to the PDE. 1040 | % 1041 | % This shows that what's important and non-trivial is $Q$. It contains all of the information about both the boundaries and the domain from the two relations. So what is Q? Here's all of the relevant cases for finite differencing: 1042 | % \begin{enumerate} 1043 | % \item \textbf{Dirichlet$_0$}: If you have any $v$ in the interior, its unique extension to satisfying the border conditions is sticking $0$'s on the ends, so both $Q_{j,L}$ and $Q_{j,H}$ are null matrices, i.e. block matrices in which all elements are zero. Therefore, Q is linear, meaning $Q = Q_A$ and $Q_b$ is a null matrix. 1044 | % 1045 | % \item \textbf{Dirichlet}: The boundary is not dependent on any points in the interior, and therefore $Q_A$ is identical to the previous case ($Q_{j,a} = [\textbf{0}_{(I_{j,L} \times I_j)}; \mathbb{I}_{(I_j \times I_j)} ; \textbf{0}_{(I_{j,H} \times I_j,)} ]$), but now Q is no longer linear and 1046 | % $Q_{j,b}$ is given by 1047 | % $Q_{j,b} = [\text{B}_{j,L}[1,\dots,I_L,:]; \textbf{0}_{(I_j\times I_j)}; 1048 | % \text{B}_{j,H}[\bar{I}_j - I_H,\dots,\bar{I},:]]$. 1049 | % 1050 | % \item \textbf{Neumann}: You have to pick a discretization of B. 1051 | % This defines linear relation between the interior points and the boundary conditions. For example, for the univariate case, the first order discretization of the boundary derivative means that 1052 | % 1053 | % \begin{equation} 1054 | % \frac{db}{dx} = \frac{\bar{u_j}[2] - \bar{u}[1]}{dx} = \text{B}[1,\dots,I_{L},:] 1055 | % \end{equation} 1056 | % 1057 | % and, therefore, $u[1] = u[2] - dx\text{B}[I_L,:]$. You can see that this implies that $Q_b = [dx\text{B}[I_L,:];\{0\}_1^I;dx \text{B}[I_H,:]]$ and that $Q_A$ is the identity matrix for the interior points and $[1,0, 0,...,0]$ for the first $I_L$ rows\footnote{So that $Q_A u = u[1] = \bar{u}[I_L+1]$.} and $[0,...,0,0,1]$ for the last $I_H$ rows.%\footnote{so that $Q_A v = v[1] = u[2]$ and $Q_A v = v[I] = u[I-1]$, respectively.}. 1058 | % 1059 | % \end{enumerate} 1060 | % \end{itemize} 1061 | % \fi 1062 | -------------------------------------------------------------------------------- /operator_examples/non_zero_dirichlet_heat.jl: -------------------------------------------------------------------------------- 1 | using OrdinaryDiffEq, ToeplitzMatrices, Plots 2 | ## Non-zero Dirichlet 3 | 4 | n = 100; 5 | x = linspace(-1,1,n) 6 | Δx = step(x) 7 | 8 | B = [1 0 zeros(1,n-4) 0 0; 9 | 0 0 zeros(1,n-4) 0 1]/Δx 10 | 11 | Δ = Toeplitz([1; zeros(n-3)], [1; -2; 1; zeros(n-3)])/Δx^2 12 | Ã = Δ - Δ[:,[1,n]]*(B[:,[1,n]]\B) 13 | A = Ã[:,2:n-1] 14 | 15 | u₀ = @. exp(x) 16 | r = Δ[:,[1,n]]*(B[:,[1,n]]\(B*u₀)) 17 | 18 | p = (A,r) 19 | 20 | 21 | function heat(du,u,p,t) 22 | (A,r) = p 23 | du .= A*u .+ r 24 | end 25 | 26 | prob = ODEProblem(heat, u₀[2:n-1], (0.0,8.0), p) 27 | @time ũ = solve(prob, Rodas4(autodiff=false); reltol=1e-5,abstol=1e-5) 28 | u_r = [B; eye(n)[2:n-1,:]] \ [B*u₀; ũ(8.0)] 29 | plot(x, u_r; label="Rodas4") # ≈ mean(u₀) 30 | 31 | 32 | @time ũ = solve(prob) 33 | u_s = [B; eye(n)[2:n-1,:]] \ [B*u₀; ũ(8.0)] 34 | plot!(x, u_s; label="solve") # ≈ mean(u₀) 35 | 36 | norm(u_r-u_s) # < 0.0009 37 | -------------------------------------------------------------------------------- /operator_examples/simple_stationary_HJBE_reflecting.jl: -------------------------------------------------------------------------------- 1 | using ToeplitzMatrices 2 | K = 5 #98 3 | K̂ = K + 2 4 | #x = linspace(0.0, 1.0, K̂) 5 | x = linspace(0.0, 1.0, K) 6 | Δx = step(x) 7 | x̂ = [x[1]-Δx; x; x[end]+Δx] #Ended domain 8 | 9 | B = [-1 1 zeros(1,K); 10 | zeros(1,K) -1 1] 11 | 12 | Q = [zeros(K)' ;I; zeros(K)'] 13 | Q[1] = 1 14 | Q[end] = 1 15 | R = [zeros(K) I zeros(K)] 16 | 17 | 18 | A = Toeplitz([1; zeros(K̂-3)], [1; -2; 1; zeros(K̂-3)])/Δx^2 19 | #A2 = A - A[:,[1,K̂]]*(B[:,[1,K̂]]\B) #We think this is useless? 20 | 21 | #Now solve the stationary ODE r u(x) = x^alpha + \sigma * D_xx u(x) 22 | α = 0.5 23 | r = 0.05 24 | σ = 0.1 25 | b(x) = x^α 26 | 27 | B = r * I - σ^2 * A * Q 28 | 29 | u = B \ b.(x) 30 | 31 | # Double check the operators 32 | R*Q*u == u # true 33 | 34 | 35 | 36 | using Plots 37 | #plot(x̂, Q*u) 38 | plot(x, u) 39 | -------------------------------------------------------------------------------- /operator_examples/zero_neumann_heat.jl: -------------------------------------------------------------------------------- 1 | using OrdinaryDiffEq, ToeplitzMatrices, Plots 2 | n = 100; 3 | x = linspace(-1,1,n) 4 | Δx = step(x) 5 | 6 | B = [-1 1 zeros(1,n-4) 0 0; 7 | 0 0 zeros(1,n-4) -1 1]/Δx 8 | 9 | Δ = Toeplitz([1; zeros(n-3)], [1; -2; 1; zeros(n-3)])/Δx^2 10 | Ã = Δ - Δ[:,[1,n]]*(B[:,[1,n]]\B) 11 | A = Ã[:,2:n-1] 12 | 13 | function heat(du,u,A,t) 14 | du .= A*u 15 | end 16 | e 17 | u₀ = @. exp(x) + ((-2 + x)*x)/(4e) - e*x*(2 + x)/4 # satisfies zero Neumann 18 | 19 | prob = ODEProblem(heat, u₀[2:n-1], (0.0,8.0), A) 20 | @time ũ = solve(prob, Rodas4(autodiff=false); reltol=1e-5,abstol=1e-5) 21 | 22 | u = [B; eye(n)[2:n-1,:]] \ [0; 0; ũ(8.0)] 23 | 24 | plot(x, u) # ≈ mean(u₀) 25 | -------------------------------------------------------------------------------- /transition_dynamics_examples/transition_dynamics_example.jl: -------------------------------------------------------------------------------- 1 | using DifferentialEquations, NamedTuples, BenchmarkTools, Plots, BandedMatrices 2 | 3 | #These would be generated automatically by DiffEqOperators.jl 4 | #These are the differential operators for positive drift 0upwind finite differences 5 | function diffusionoperators(x_min, x_max, M) 6 | x = linspace(x_min, x_max, M) 7 | Δ = x[2]-x[1] 8 | 9 | dl_1 = zeros(M-1) 10 | d_1 = -1 * ones(M) 11 | d_1[end] = 0 12 | du_1 = ones(M-1) 13 | L_1_plus = Tridiagonal(dl_1, d_1, du_1)/Δ 14 | 15 | dl_2 = ones(M-1) 16 | d_2 = -2 * ones(M) 17 | d_2[1] = -1 18 | d_2[end] = -1 19 | du_2 = ones(M-1) 20 | L_2 = Tridiagonal(dl_2, d_2, du_2)/(Δ^2) 21 | 22 | #BandedMatrix are much faster, probably because of better specializations in the composition 23 | return (x, BandedMatrix(L_1_plus, (1, 1)), BandedMatrix(L_2, (1, 1))) #The (1,1) are the off-diagonal bandwidths 24 | end 25 | 26 | #Create DiffEq Problem for solving 27 | function createODEproblem(c_tilde, sigma_tilde, mu_tilde, x_min::Float64, x_max::Float64, M::Int64, T::Float64, rho::Float64, reversed = false) 28 | x, L_1_plus, L_2 = diffusionoperators(x_min, x_max, M) #Discretize the operator 29 | 30 | p = @NT(L_1_plus = L_1_plus, L_2 = L_2, x = x, rho = rho, mu_tilde = mu_tilde, sigma_tilde = sigma_tilde, c_tilde = c_tilde, T = T) #Named tuple for parameters. 31 | 32 | #Check upwind direction 33 | @assert minimum(mu_tilde.(T, x)) >= 0.0 34 | @assert minimum(mu_tilde.(0.0, x)) >= 0.0 35 | 36 | #Calculating the stationary solution, 37 | L_T = rho*I - Diagonal(mu_tilde.(T, x)) * L_1_plus - Diagonal(sigma_tilde.(T, x).^2/2.0) * L_2 38 | u_T = L_T \ c_tilde.(T, x) 39 | 40 | @assert(issorted(u_T)) #We are only solving versions that are increasing for now 41 | 42 | function f(du,u,p,t) 43 | L = (p.rho*I - Diagonal(p.mu_tilde.(t, x)) * p.L_1_plus - Diagonal(p.sigma_tilde.(t, p.x).^2/2.0) * p.L_2) 44 | A_mul_B!(du,L,u) 45 | du .-= p.c_tilde.(t, p.x) 46 | end 47 | 48 | function f_reversed(du,u,p,t_tilde) 49 | L = -(p.rho*I - Diagonal(p.mu_tilde.(p.T - t_tilde, x)) * p.L_1_plus - Diagonal(p.sigma_tilde.(p.T - t_tilde, p.x).^2/2.0) * p.L_2) 50 | A_mul_B!(du,L,u) 51 | du .+= p.c_tilde.(p.T - t_tilde, p.x) 52 | end 53 | 54 | #Checks on the residual 55 | du_T = zeros(u_T) 56 | f(du_T, u_T, p, T) 57 | #@show norm(du_T) 58 | @assert norm(du_T) < 1.0E-10 59 | 60 | f_reversed(du_T, u_T, p, 0) 61 | @assert norm(du_T) < 1.0E-10 62 | 63 | if reversed==false 64 | return ODEProblem(f, u_T, (T, 0.0), p) 65 | else 66 | return ODEProblem(f_reversed, u_T, (0.0,T), p) #Backwards in time, starting at T = 0 67 | end 68 | end 69 | 70 | 71 | # mu_tilde is time varying but sigma_tilde is not. 72 | x_min = 0.01 73 | x_max = 1.0 74 | M = 20 75 | T = 10.0 76 | rho = 0.15 77 | sigma_bar = 0.1 78 | c_tilde(t, x) = x + 0.0001*t 79 | sigma_tilde(t, x) = sigma_bar 80 | mu_tilde(t, x) = 0.1*x *(1.0 + 4.0 * t) 81 | basealgorithm = CVODE_BDF() #CVODE_CDF(linear_solver=:GMRES) #ImplicitEuler() #A reasonable alternative. Algorithms which don't work well: Rosenbrock23(), Rodas4(), KenCarp4() 82 | plotevery = 5 83 | 84 | prob = createODEproblem(c_tilde, sigma_tilde, mu_tilde, x_min, x_max, M, T, rho, false) 85 | sol = solve(prob, basealgorithm) 86 | plot(sol, vars=1:plotevery:M) 87 | @assert(issorted(sol[end])) 88 | @benchmark solve($prob, $basealgorithm) #Benchmark 89 | 90 | prob_reversed = createODEproblem(c_tilde, sigma_tilde, mu_tilde, x_min, x_max, M, T, rho, true) 91 | sol_reversed = solve(prob_reversed, basealgorithm) 92 | plot(sol_reversed, vars=1:plotevery:M, xlabel="t_tilde") 93 | 94 | #Compare the timesteps and the terminal/starting values 95 | @show sol.t' 96 | @show T - sol_reversed.t' #i.e. t = T - t_tilde 97 | @show norm(sol(0) - sol_reversed(T)) #i.e. t=0 98 | @show norm(sol(T) - sol_reversed(0)) #i.e. t=T 99 | 100 | #Solve backwards with a DAE and a trivial algebraic equation 101 | #u_ex(1:M) = u(), following the ODE 102 | #u_ex(M+1) = 1.0 #(i.e., a trivial linear function) 103 | function createDAEproblem(c_tilde, sigma_tilde, mu_tilde, x_min, x_max, M, T, rho) 104 | x, L_1_plus, L_2 = diffusionoperators(x_min, x_max, M) #Discretize the operator 105 | 106 | p = @NT(L_1_plus = L_1_plus, L_2 = L_2, x = x, rho = rho, mu_tilde = mu_tilde, sigma_tilde = sigma_tilde, c_tilde = c_tilde, M = M) #Named tuple for parameters. 107 | 108 | #Check upwind direction 109 | @assert minimum(mu_tilde.(T, x)) >= 0.0 110 | @assert minimum(mu_tilde.(0.0, x)) >= 0.0 111 | 112 | #Calculating the stationary solution, 113 | L_T = rho*I - Diagonal(mu_tilde.(T, x)) * L_1_plus - Diagonal(sigma_tilde.(T, x).^2/2.0) * L_2 114 | u_T = L_T \ c_tilde.(T, x) 115 | 116 | u_ex_T = [u_T; 1.0] #Closed form for the trivial linear function we are adding 117 | 118 | @assert(issorted(u_T)) #We are only solving versions that are increasing for now 119 | 120 | 121 | function f(resid,du_ex,u_ex,p,t) 122 | L = (p.rho*I - Diagonal(p.mu_tilde.(t, x)) * p.L_1_plus - Diagonal(p.sigma_tilde.(t, p.x).^2/2.0) * p.L_2) 123 | u = u_ex[1:p.M] 124 | resid[1:M] .= L * u_ex[1:p.M] - p.c_tilde.(t, p.x) 125 | resid[M+1] .= u_ex[M+1] - 1.0 126 | resid .-= du_ex 127 | end 128 | 129 | #Should Verifying that the initial condition is solid 130 | resid_T = zeros(u_ex_T) #preallocation 131 | du_ex_T = zeros(u_ex_T) 132 | f(resid_T, du_ex_T, u_ex_T, p, T) 133 | @show norm(resid_T) 134 | @assert norm(resid_T) < 1.0E-10 135 | 136 | tspan = (T, 0.0) 137 | return DAEProblem(f, zeros(u_ex_T), u_ex_T, tspan, differential_vars = [trues(u_T); false], p) 138 | end 139 | 140 | probDAE = createDAEproblem(c_tilde, sigma_tilde, mu_tilde, x_min, x_max, M, T, rho) 141 | solDAE = solve(probDAE, IDA()) 142 | plot(solDAE, vars=1:plotevery:M) 143 | @show(issorted(solDAE[end][1:M])) 144 | 145 | # @benchmark solve($probDAE, IDA()) 146 | 147 | #Check they are "reasonably" close 148 | @show norm(sol[1] - solDAE[1][1:M]) 149 | @show norm(sol[end] - solDAE[end][1:M]) 150 | --------------------------------------------------------------------------------