$\blacksquare$
26 | -------------------------------------------------------------------------------- /Chapter 06 - Inner Product Spaces/notes.md: -------------------------------------------------------------------------------- 1 | Chapter 6: **Inner Product Spaces** 2 | 3 | **Theorem 1.** 4 | Suppose $V$ is a finite-dimensional inner product space and $u, v \in V$. 5 | Then $u$ and $v$ are orthogonal if and only if their matrices with respect to some orthonormal basis of $V$ are orthogonal when thought as elements of $\mathbb{F}^n$ with the usual Euclidean inner product. 6 | 7 | _Proof_ 8 | 9 | Let $e_1, \dots, e_n$ be an orthonormal basis of $V$. 10 | We have 11 | 12 | $$ 13 | u = a_1 e_1 + \cdots + a_n e_n \text{ and } v = c_1 e_1 + \cdots + c_n e_n 14 | $$ 15 | 16 | where the $a$'s and the $c$'s are the entries of $\mathcal{M}(u, (e_1, \dots, e_n))$ and $\mathcal{M}(v, (e_1, \dots, e_n))$. 17 | Therefore 18 | 19 | $$ 20 | \langle u, v \rangle = a_1\overline{c_1} + \dots + a_n\overline{c_n} = \langle (a_1, \dots, a_n), (c_1, \dots, c_n) \rangle, 21 | $$ 22 | 23 | where the first $\langle \cdot, \cdot \rangle$ is the inner product on $V$ and the second is the usual inner product on $\mathbb{F}^n$. 24 | The equation above yields the desired result. 25 | 26 |$\blacksquare$
27 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Linear Algebra Done Right, 3rd Edition: Solutions 2 | 3 | Most solutions here are of my own authorship, but some were done with the help of this [site](http://linearalgebras.com/). 4 | The `notes.md` in each chapter folder (the ones that do have though) contain some theorems or lemmas. 5 | 6 | To visualize the equations, I recommend Google Chrome with the extension [TeX All the Things](https://chrome.google.com/webstore/detail/tex-all-the-things/cbimabofgmfdkicghcadidpemeenbffn?utm_source=chrome-app-launcher-info-dialog) or [GitHub with MathJax](https://chrome.google.com/webstore/detail/github-with-mathjax/ioemnmodlmafdkllaclgeombjnmnbima). 7 | 8 | _Disclaimer_: do not take for granted that every solution posted here is correct. Everyone is susceptible to errors and even more so when you're just an undergraduate student, like me. In fact, if you see a mistake, or if a solution is not clear enough, post a new issue and I'll look into that. 9 | 10 | **Contributing**: lots of exercises are missing solutions and some could be rewritten, if you want fill some of these gaps, open an issue and I'll give you access to commit. 11 | -------------------------------------------------------------------------------- /Chapter 02 - Finite-Dimensional Vector Spaces/2B - Bases.md: -------------------------------------------------------------------------------- 1 | Chapter 2: **Finite-Dimensional Vector Spaces** 2 | 3 | **2.B** 4 | 5 | - [ ] Exercise 1 6 | - [ ] Exercise 2 7 | - [ ] Exercise 3 8 | - [ ] Exercise 4 9 | - [ ] Exercise 5 10 | - [ ] Exercise 6 11 | - [ ] Exercise 7 12 | - [x] Exercise 8 13 | 14 | _Exercise 8_ 15 | 16 | Let $v \in V$. 17 | Then $v = u + w$ for some $u \in U$ and $w \in W$. 18 | Since $u$ is a linear combination of the $u$'s and $w$ a linear combination of the $w$'s, it follows that 19 | 20 | $$ 21 | u_1, \dots, u_m, w_1, \dots, w_n 22 | $$ 23 | 24 | spans $V$. 25 | 26 | Now let $a_1, \dots, a_m, b_1, \dots, b_n \in \mathbb{F}$ such that 27 | 28 | $$ 29 | a_1 u_1 + \dots + a_m u_m + b_1 w_1 + \dots + b_n w_n = 0. 30 | $$ 31 | 32 | This is a sum of a vector in $U$ and a vector in $W$ (just group the $u$'s and $w$'s in separate parentheses). 33 | Then, 1.44 implies that 34 | 35 | $$ 36 | a_1 u_1 + \dots + a_m u_m = 0 \text{ and } b_1 w_1 + \dots + b_n w_n = 0. 37 | $$ 38 | 39 | Since $u_1, \dots, u_m$ is linearly independent, the first equation above shows that $a_1 = \dots = a_m = 0$. 40 | Similarly, the $b$'s are also $0$. 41 | Therefore, the list 42 | 43 | $$ 44 | u_1, \dots, u_m, w_1, \dots, w_n 45 | $$ 46 | 47 | is linearly independent, that is, a basis of $V$. 48 | -------------------------------------------------------------------------------- /Chapter 02 - Finite-Dimensional Vector Spaces/notes.md: -------------------------------------------------------------------------------- 1 | Chapter 2: **Finite-Dimensional Vector Spaces** 2 | 3 | **Theorem 1.** 4 | Suppose that $V$ is finite-dimensional and $U$ and $W$ are subspaces of $V$. 5 | There is a basis $v_1, \dots, v_n, u_1, \dots, u_m, w_1, \dots, w_p$ of $U + W$ such that $v_1, \dots, v_n$ is a basis of $U \cap W$, $v_1, \dots, v_n, u_1, \dots, u_m$ is a basis of $U$ and $v_1, \dots, v_n, w_1, \dots, w_p$ is a basis of $W$. 6 | 7 | _Proof_ 8 | 9 | Let $v_1, \dots, v_n$ be a basis of $U \cap W$. 10 | Extend it a basis $v_1, \dots, v_n, u_1, \dots, u_m$ of $U$ and a basis $v_1, \dots, v_n, w_1, \dots, w_p$ of $W$. 11 | Suppose there are $a_1, \dots, a_n, b_1, \dots, b_m, c_1, \dots, c_p \in \mathbb{F}$ such that 12 | 13 | $$ 14 | a_1 v_1 + \dots + a_n v_n + b_1 u_1 + \dots + b_m u_m + c_1 w_1 + \dots + c_p w_p = 0 15 | $$ 16 | 17 | We have 18 | 19 | $$ 20 | a_1 v_1 + \dots + a_n v_n + b_1 u_1 + \dots + b_m u_m = - c_1 w_1 - \dots - c_p w_p 21 | $$ 22 | 23 | Which implies that $- c_1 w_1 - \dots - c_p w_p$ is in $U$ and, therefore, in $U \cap W$ too. 24 | Hence, there are scalars $d_1, \dots, d_n \in \mathbb{F}$ such that 25 | 26 | $$ 27 | d_1 v_1 + \dots + d_n v_n = - c_1 w_1 - \dots - c_p w_p 28 | $$ 29 | 30 | Because $v_1, \dots, v_n, w_1, \dots, w_p$ is a basis of $W$, all the $c$'s (and $d$'s) are 0 and the right hand side of the previous equation becomes 0. 31 | It quickly follows that all $a$'s and $b$'s are 0. 32 | Thus $v_1, \dots, v_n, u_1, \dots, u_m, w_1, \dots, w_p$ is linearly independent. 33 |$\blacksquare$
34 | 35 | -------------------------------------------------------------------------------- /Chapter 03 - Linear Maps/notes.md: -------------------------------------------------------------------------------- 1 | Chapter 3: **Linear Maps** 2 | 3 | **Theorem 1.** 4 | Suppose $V$ is a finite-dimensional vector space and $\varphi \in V'$. Then $\operatorname{span}(\varphi) = (\operatorname{null}\varphi)^0$. 5 | 6 | _Proof_ 7 | 8 | Clearly, $\operatorname{span}(\varphi) \subset (\operatorname{null}\varphi)^0$. 9 | 10 | For the inclusion in the other direction, suppose $\psi \notin \operatorname{span}(\varphi)$. 11 | 12 | If $\varphi = 0$, then $(\operatorname{null} \varphi)^0 = V^0 = \\{0\\}$ and then, because $\psi \neq 0$, it follows that $\psi \notin \operatorname{span}(\varphi)$. 13 | 14 | Assume $\varphi$ is non-zero. 15 | Let $v_1, \dots, v_n$ be a basis of $\operatorname{null} \varphi$. 16 | By _Exercise 1_ in section 3F, it follows that $\varphi$ is surjective. 17 | Choose a vector $v$ in $V$ such that $\varphi(v) = 1$. 18 | By the Fundamental Theorem of Linear Maps (3.22), $v, v_1, \dots, v_n$ is a basis of $V$. 19 | Note that $\varphi(v_i) = 0$ for $i = 1, \dots, n$. 20 | Let $\varphi, \psi_1, \dots, \psi_n$ be its dual basis. 21 | 22 | We have $\psi = a \varphi + a_1 \psi_1 + \dots + a_n \psi_n$, for some scalars $a, a_1, \dots, a_n \in \mathbb{F}$, where $a_j \neq 0$ for some $j \in \\{1, \dots, n\\}$ because $\psi \notin \operatorname{span}(\varphi)$. 23 | Then 24 | 25 | $$ 26 | \begin{aligned} 27 | \psi(v_j) &= (a \varphi + a_1 \psi_1 + \dots + a_n \psi_n)(v_j)\\\\ 28 | &= a \varphi(v_j) + a_1 \psi_1(v_j) + \dots + a_n \psi_n(v_j)\\\\ 29 | &= a_j\\\\ 30 | &\neq 0 31 | \end{aligned} 32 | $$ 33 | 34 | Because $v_j \in \operatorname{null} \varphi$ it follows that $\psi \notin (\operatorname{null} \varphi)^0$. 35 | 36 | By modus tollens, $\psi \in (\operatorname{null} \varphi)^0$ implies $\psi \in \operatorname{span}(\varphi)$. 37 | Hence $(\operatorname{null} \varphi)^0 \subset \operatorname{span}(\varphi)$ and $\operatorname{span}(\varphi) = (\operatorname{null}\varphi)^0$, as desired. 38 |$\blacksquare$
39 | 40 | **Theorem 2.** 41 | Suppose $V$ is finite-dimensional and $\varphi, \varphi_1, \dots, \varphi_n \in V'$. 42 | If $(\operatorname{null} \varphi_1) \cap \dots \cap (\operatorname{null} \varphi_n) \subset \operatorname{null} \varphi$, then $\varphi \in \operatorname{span}(\varphi_1, \dots, \varphi_n)$. 43 | 44 | _Proof_ 45 | 46 | We have that 47 | 48 | $$ 49 | \begin{aligned} 50 | \varphi \in \operatorname{span}(\varphi) &= (\operatorname{null} \varphi)^0\\\\ 51 | &\subset ((\operatorname{null} \varphi_1) \cap \dots \cap (\operatorname{null} \varphi_n))^0\\\\ 52 | &= (\operatorname{null} \varphi_1)^0 + \dots + (\operatorname{null} \varphi_n)^0\\\\ 53 | &= \operatorname{span}(\varphi_1) + \dots + \operatorname{span}(\varphi_n)\\\\ 54 | &= \operatorname{span}(\varphi_1, \dots, \varphi_n)\\\\ 55 | \end{aligned} 56 | $$ 57 | 58 | Where the first and fourth lines follow from _Theorem 1_, the second and third from _Exercise 20_ and _Exercise 23_ in section 3F, respectively, and the fifth from the definition of sum of subspaces. 59 |$\blacksquare$
60 | -------------------------------------------------------------------------------- /Chapter 04 - Polynomials/4 - Polynomials.md: -------------------------------------------------------------------------------- 1 | Chapter 4: **Polynomials** 2 | 3 | - [ ] Exercise 1 4 | - [x] Exercise 2 5 | - [x] Exercise 3 6 | - [x] Exercise 4 7 | - [x] Exercise 5 8 | - [x] Exercise 6 9 | - [ ] Exercise 7 10 | - [x] Exercise 8 11 | - [ ] Exercise 9 12 | - [ ] Exercise 10 13 | - [ ] Exercise 11 14 | 15 | _Exercise 2_ 16 | 17 | No. Both $x^m$ and $-x^m + 1$ are in the set, but $x^m + (-x^m + 1) = 1$ has degree 0, therefore the set is not closed under addition. 18 | 19 | _Exercise 3_ 20 | 21 | No, $x^2$ and $-x^2 + x$ are both in the set, but $x^2 + (-x^2 + x) = x$ is not. 22 | 23 | _Exercise 4_ 24 | 25 | Define $p \in \mathcal{P}(\mathbb{F})$ by 26 | 27 | $$ 28 | p(z) = (z - \lambda_1) \dots (z - \lambda_m) (z - \lambda_1)^{n - m} 29 | $$ 30 | 31 | Clearly $p(\lambda_1) = \dots = p(\lambda_m) = 0$, $\deg p = n$ and $p$ has no other zeros, as desired. 32 | 33 | _Exercise 5_ 34 | 35 | Define $\Upsilon: \mathcal{P}_m(\mathbb{F}) \to \mathbb{F}^{m+1}$ by 36 | 37 | $$ 38 | \Upsilon(p) = (p(z_1), \dots, p(z_{m+1})) 39 | $$ 40 | 41 | We have that $\operatorname{null} \Upsilon = \\{0\\}$, because no polynomial in $\mathcal{P}_m(\mathbb{F})$ has $m + 1$ zeros. 42 | Hence $\Upsilon$ is injective, proving the uniqueness part. 43 | By the Fundamental Theorem of Linear Maps, it follows that 44 | 45 | $$ 46 | \operatorname{dim} \operatorname{range} \Upsilon = \operatorname{dim} \mathcal{P}_m(\mathbb{F}) = \operatorname{dim} \mathbb{F}^{m+1} 47 | $$ 48 | 49 | Therefore $\Upsilon$ is surjective, proving the existence part. 50 | 51 | _Exercise 6_ 52 | 53 | Suppose $p$ has $m$ distinct zeros. 54 | Let $\lambda$ be a zero of $p$. 55 | Then 56 | 57 | $$ 58 | p(z) = (z - \lambda)q(z) 59 | $$ 60 | 61 | For some polynomial $q$. 62 | Using the product rule, we get 63 | 64 | $$ 65 | p'(z) = q(z) + (z - \lambda)q'(z) 66 | $$ 67 | 68 | Because the zeros of $p$ are distinct, it follows that $\lambda$ is not a zero of $q$. 69 | Therefore 70 | 71 | $$ 72 | p'(\lambda) = q(\lambda) + (\lambda - \lambda)q'(\lambda) = q(\lambda) \neq 0 73 | $$ 74 | 75 | Thus $\lambda$ is not a zero of $p'$, as desired. 76 | 77 | _Exercise 8_ 78 | 79 | We will show that $Tp = q$ where $q$ is the polynomial such that $p(x) - p(3) = (x - 3)q(x)$. 80 | 81 | Clearly, if $x \neq 3$ then $Tp = q$. 82 | 83 | Suppose $x = 3$. 84 | We have that 3 is a root of $p(x) - p(3)$. 85 | Thus, by the same reasoning used in _Exercise 6_, it follows that 86 | 87 | $$ 88 | (p - p(3))'(3) = q(3) 89 | $$ 90 | 91 | But $p' = (p - p(3))'$, hence $Tp(3) = q(3)$. 92 | Therefore $Tp = q$ and $Tp$ is indeed a polynomial. 93 | 94 | To prove $T$ is linear, let $r, s \in \mathbb{P}(R)$ and $\alpha \in \mathbb{F}$. If $x \neq 3$, then 95 | 96 | $$ 97 | T(r + s) = \frac{r + s - r(3) - s(3)}{x - 3} = \frac{r - r(3)}{x - 3} + \frac{s - s(3)}{x - 3} = Tr + Ts 98 | $$ 99 | 100 | And 101 | 102 | $$ 103 | T(\alpha r) = \frac{\alpha r - \alpha r(3)}{x - 3} = \alpha\frac{r - r(3)}{x - 3} = \alpha Tr 104 | $$ 105 | 106 | If $x = 3$, then $T$ is linear by the properties of derivatives. 107 | -------------------------------------------------------------------------------- /Chapter 07 - Operators on Inner Product Spaces/notes.md: -------------------------------------------------------------------------------- 1 | Chapter 7: **Operators on Inner Product Spaces** 2 | 3 | **Theorem 1.** 4 | Suppose $V$ is a finite-dimensional inner product space and $S \in \mathcal{L}(V)$. 5 | Then $S$ is an isometry if and only if the columns of the matrix of $S$ with respect to every orthonormal basis of $V$ are orthonormal when thought as elements of $\mathbb{F}^n$. 6 | 7 | _Proof_ 8 | 9 | Let $e_1, \dots, e_n$ be an orthonormal basis of $V$ and let $A$ be the matrix of $S$ with respect to this basis. 10 | We have 11 | 12 | $$ 13 | Se_j = A_{1, j} e_1 + \dots + A_{n, j} e_n 14 | $$ 15 | 16 | for each $j = 1, \dots, n$. 17 | Therefore 18 | 19 | $$ 20 | \langle Se_j, Se_k \rangle = A_{1, j}\overline{A_{1, k}} + \cdots + A_{n, j}\overline{A_{n, k}} = \langle (A_{1, j}, \dots, A_{n, j}), (A_{1, k}, \dots, A_{n, k}) \rangle. 21 | $$ 22 | 23 | Then $S$ is an isometry if and only if the list $Se_1, \dots, Se_n$ is orthonormal (see 7.42), which happens if and only if the columns of $\mathcal{M}(S)$ are also orthonormal (by the equation above). 24 | Note that the first $\langle \cdot, \cdot \rangle$ above is the inner product on $V$ and the second is the usual inner product on $\mathbb{F}^n$. 25 |$\blacksquare$
26 | 27 | **Theorem 2.** (Rayleigh's principle) 28 | Suppose $V$ is a finite-dimensional real inner product space and $T \in \mathcal{L}(V)$ is self-adjoint with eigenvalues $\lambda_1 \ge \dots \ge \lambda_n$ (with each eigenvalue $\lambda$ repeated $\dim E(\lambda, T)$ times). 29 | Let $e_1, \dots, e_n$ be a corresponding orthonormal basis of $V$ consisting of eigenvectors of $T$, that is, $Te_j = \lambda_j e_j$. 30 | Then 31 | 32 | $$ 33 | \frac{1}{||u||^2} \langle Tu, u \rangle \ge \lambda_i \text{ if } u \in \operatorname{span}(e_1, \dots, e_i) 34 | $$ 35 | 36 | and 37 | 38 | $$ 39 | \frac{1}{||u||^2} \langle Tu, u \rangle \le \lambda_i \text{ if } u \in \operatorname{span}(e_1, \dots, e_{i-1})^\perp. 40 | $$ 41 | 42 | _Proof_ 43 | 44 | Suppose $u \in \operatorname{span}(e_1, \dots, e_i)$. 45 | We can write $u = a_1 e_1 + \dots + a_i e_i$ for some $a_1, \dots, a_i \in \mathbb{R}$. 46 | Then 47 | 48 | $$ 49 | \begin{aligned} 50 | \frac{1}{||u||^2} \langle Tu, u \rangle &= \frac{1}{||u||^2} (\lambda_1 |a_1|^2 + \dots + \lambda_i |a_i|^2)\\\\ 51 | &\ge \frac{1}{||u||^2} (\lambda_i |a_1|^2 + \dots + \lambda_i |a_i|^2)\\\\ 52 | &= \frac{1}{||u||^2} \lambda_i ||u||^2\\\\ 53 | &= \lambda_i. 54 | \end{aligned} 55 | $$ 56 | 57 | Now suppose $u \in \operatorname{span}(e_1, \dots, e_{i-1})^\perp$. 58 | Then $u \in \operatorname{span}(e_i, \dots, e_n)$ and we can write $u = a_i e_i + \dots + a_n e_n$ for some $a_i, \dots a_n \in \mathbb{R}$. 59 | Thus 60 | 61 | $$ 62 | \begin{aligned} 63 | \frac{1}{||u||^2} \langle Tu, u \rangle &= \frac{1}{||u||^2} (\lambda_i |a_i|^2 + \dots + \lambda_n |a_n|^2)\\\\ 64 | &\le \frac{1}{||u||^2} (\lambda_i |a_i|^2 + \dots + \lambda_i |a_n|^2)\\\\ 65 | &= \frac{1}{||u||^2} \lambda_i ||u||^2\\\\ 66 | &= \lambda_i. 67 | \end{aligned} 68 | $$ 69 |$\blacksquare$
70 | 71 | As a corollary of this we have the following result: 72 | 73 | **Corollary 1.** 74 | 75 | $$ 76 | \min_{u \in V, u \neq 0} \frac{1}{||u||^2} \langle Tu, u \rangle = \lambda_n. 77 | $$ 78 | 79 | Or, equivalently, 80 | 81 | $$ 82 | \min_{u \in V, ||u|| = 1} \langle Tu, u \rangle = \lambda_n. 83 | $$ 84 | -------------------------------------------------------------------------------- /Chapter 02 - Finite-Dimensional Vector Spaces/2A - Span and Linear Independence.md: -------------------------------------------------------------------------------- 1 | Chapter 2: **Finite-Dimensional Vector Spaces** 2 | 3 | **2.A** 4 | 5 | - [ ] Exercise 1 6 | - [ ] Exercise 2 7 | - [ ] Exercise 3 8 | - [ ] Exercise 4 9 | - [ ] Exercise 5 10 | - [ ] Exercise 6 11 | - [ ] Exercise 7 12 | - [ ] Exercise 8 13 | - [ ] Exercise 9 14 | - [x] Exercise 10 15 | - [ ] Exercise 11 16 | - [ ] Exercise 12 17 | - [ ] Exercise 13 18 | - [x] Exercise 14 19 | - [x] Exercise 15 20 | - [x] Exercise 16 21 | - [x] Exercise 17 22 | 23 | _Exercise 10_ 24 | 25 | Suppose that $v_1 + w, \dots, v_m + w$ is linearly dependent. There are $a_1, \dots, a_m \in \mathbb{F}$ not all zero such that 26 | 27 | $$ 28 | \begin{aligned} 29 | a_1 (v_1 + w) + \dots + a_m (v_m + w) &= 0\\\\ 30 | a_1 v_1 + \dots + a_m v_m &= - (a_1 + \dots + a_m) w\\\\ 31 | -\frac{a_1}{a_1 + \dots + a_m} v_1 - \dots -\frac{a_m}{a_1 + \dots + a_m} v_m &= w\\\\ 32 | \end{aligned} 33 | $$ 34 | 35 | Because $v_1, \dots, v_m$ is linearly independent, it follows by the second equation that $a_1 + \dots + a_m \neq 0$. Hence the third equation makes sense and $w \in \operatorname{span}(v_1, \dots, v_m)$. 36 | 37 | _Exercise 14_ 38 | 39 | Suppose $V$ is infinite-dimensional. 40 | Choose any positive integer $m$ and consider the following process 41 | 42 | * Step 1 43 | 44 | Choose a non-zero vector $v_1$ in $V$. 45 | 46 | * Step j 47 | 48 | Because $V$ is infinite-dimensional, it follows that $v_1, \dots, v_{j-1}$ doesn't span $V$. Hence, there is a vector $v_j \in V$ such that $v_j \notin \operatorname{span}(v_1, \dots, v_{j-1})$. Therefore $v_1, \dots, v_j$ is a linearly independent list in $V$. If $j = m$ stop the process. 49 | 50 | After step $m$ the process stops and we have constructed a linearly independent list of length $m$. 51 | 52 | For the converse, we will prove the contrapositive. 53 | Suppose that $V$ is finite-dimensional and $v_1, \dots, v_n$ spans $V$. 54 | By 2.23, we cannot have a linearly independent list of arbitrary length (specifically, it cannot be greater than $n$). 55 | 56 | Therefore, by modus tollens, if we can have a linearly independent list in $V$ of arbitrary length, then $V$ is infinite-dimensional. 57 | 58 | _Exercise 15_ 59 | 60 | Let $m$ be a positive integer. 61 | Consider the list $v_1, \dots, v_m$, where $v_j$ has $0$ in all slots except for a $1$ at the $j$-th slot. 62 | This list is obviously linearly independent. 63 | Since $m$ was arbitrary, by the previous exercise $\mathbb{F}^\infty$ is infinite-dimensional. 64 | 65 | _Exercise 16_ 66 | 67 | Let $C_{\mathbb{R}}[0, 1]$ denote the real vector space of all continuous real-valued functions on the interval $[0, 1]$. 68 | Since every polynomial is continuous on $[0, 1]$, it follows that $\mathcal{P}(\mathbb{R})$ is a subspace of $C_{\mathbb{R}}[0, 1]$. 69 | However, $\mathcal{P}(\mathbb{R})$ is infinite-dimensional (by 2.16). 70 | Thus 2.26 now implies that $C_{\mathbb{R}}[0, 1]$ is infinite-dimensional. 71 | 72 | _Exercise 17_ 73 | 74 | Suppose by contradiction that $p_0, p_1, \dots, p_m$ is linearly indepedent. 75 | For every polynomial $p \in \operatorname{span}(p_0, p_1, \dots, p_m)$ we have $p(2) = 0$. 76 | Therefore $1 \notin \operatorname{span}(p_0, p_1, \dots, p_m)$ and, by the Linear Dependence Lemma, the list $p_0, p_1, \dots, p_m, 1$ is linearly independent. 77 | However, this list has length $m + 2$ and the list $1, x, \dots, x^m$, which spans $\mathcal{P}_m(R)$, has length $m + 1$. 78 | This is a contradiction by 2.23. 79 | -------------------------------------------------------------------------------- /Chapter 02 - Finite-Dimensional Vector Spaces/2C - Dimension.md: -------------------------------------------------------------------------------- 1 | Chapter 2: **Finite-Dimensional Vector Spaces** 2 | 3 | **2.C** 4 | 5 | - [ ] Exercise 1 6 | - [ ] Exercise 2 7 | - [ ] Exercise 3 8 | - [ ] Exercise 4 9 | - [ ] Exercise 5 10 | - [ ] Exercise 6 11 | - [ ] Exercise 7 12 | - [ ] Exercise 8 13 | - [x] Exercise 9 14 | - [ ] Exercise 10 15 | - [ ] Exercise 11 16 | - [ ] Exercise 12 17 | - [ ] Exercise 13 18 | - [x] Exercise 14 19 | - [x] Exercise 15 20 | - [x] Exercise 16 21 | - [x] Exercise 17 22 | 23 | _Exercise 9_ 24 | 25 | The result clearly holds if $v_1 + w, \dots, v_m + w$ is linearly independent, so assume it is not. 26 | By Exercise 10 in section 2A, $w \in \operatorname{span}(v_1, \dots, v_m)$ we can write 27 | $$ 28 | w = a_1 v_1 + \cdots + a_m v_m, 29 | $$ 30 | for some unique scalars $a_1, \dots, a_m \in \mathbb{R}$. 31 | Since $w$ is nonzero, we have $a_j \neq 0$ for at least one $j$. 32 | Removing $v_j + w$, we get a list of length $m-1$ which is linearly independent by previously cited exercise and the uniqueness of the scalars $a_i$. 33 | 34 | _Exercise 13_ 35 | 36 | Let $U \in \mathbb{C}^6$, $W \in \mathbb{C}^6$, and $\operatorname{dim} U = \operatorname{dim} W = 4$. 37 | 38 | We have that $\operatorname{dim} (U + W) = \operatorname{dim} U + \operatorname{dim} V - \operatorname{dim} U \cap V$, which implies that $\operatorname{dim} U \cap V = \operatorname{dim} U + \operatorname{dim} W - \operatorname{dim} (U + W) = 8 - \operatorname{dim} (U + W)$. 39 | 40 | We know, that $\operatorname{dim} (U + W) \le \operatorname{dim} \mathbb{C}^6 = 6$, which in turn means that $\operatorname{dim} (U \cap V) \ge 8 - 6 = 2$, so basis of $U \cap V$ consists of 2 or more vectors, and since basis is linearly list of vectors independent, we can conclude that we have proved the statement. 41 | 42 | _Exercise 14_ 43 | 44 | We will prove by induction on $m$. 45 | For the base case we have $\operatorname{dim} U_1 \le \operatorname{dim} U_1$. 46 | Suppose $m > 1$ and 47 | 48 | $$ 49 | \operatorname{dim} (U_1 + \dots + U_k) \le \operatorname{dim} U_1 + \dots + \operatorname{dim} U_k 50 | $$ 51 | 52 | for all positive integers $k < m$. 53 | Then 54 | 55 | $$ 56 | \begin{aligned} 57 | \operatorname{dim} (U_1 + \dots + U_m) &= \operatorname{dim}(U_1 + \dots + U_{m-1}) + \operatorname{dim}(U_m) - \operatorname{dim}((U_1 + \dots + U_{m-1}) \cap U_m)\\\\ 58 | &\le \operatorname{dim}(U_1 + \dots + U_{m-1}) + \operatorname{dim} U_m\\\\ 59 | &\le \operatorname{dim} U_1 + \dots + \operatorname{dim} U_{m-1} + \operatorname{dim} U_m, 60 | \end{aligned} 61 | $$ 62 | 63 | where first equality follows from 2.43 and third from the induction hypothesis. 64 | 65 | _Exercise 15_ 66 | 67 | Let $u_1, \dots, u_n$ be a basis of $V$. 68 | Define $U_j$ by 69 | 70 | $$ 71 | U_j = \operatorname{span}(u_j) 72 | $$ 73 | 74 | for $j = 1, \dots, n$. 75 | Each $U_j$ is clearly $1$-dimensional. 76 | Moreover, $V = U_1 + \dots + U_n$, because this sum contains all the basis vectors. 77 | Thus, for every $v \in V$, we can write it uniquely in the form 78 | 79 | $$ 80 | v = a_1 u_1 + \dots + a_n u_n 81 | $$ 82 | 83 | for some scalars $a_1, \dots, a_n \in \mathbb{F}$ (by 2.29). 84 | Since each $u_j$ is in $U_j$, by the definition of direct sum, $U_1 + \dots + U_m$ is a direct sum. 85 | 86 | _Exercise 16_ 87 | 88 | Let $V = U_1 \oplus \dots \oplus U_m$. 89 | Because every vector in $V$ can be written as sum $v_1 + \dots + v_m$ where each $v_j \in U_j$ and since each $v_j$ can be written as sum of $u_1 + \dots + u_{\operatorname{dimU_j}}$ for some basis $u_1, \dots, u_{\operatorname{dimU_j}}$ of $U_j$, it follows that the list composed of all such bases spans $V$. 90 | Hence, $V$ is finite-dimensional. 91 | 92 | Moreover, if a linear combination of this list equals $0$, then a linear combination of $v_1, \dots, v_m$ also equals $0$. 93 | But, $U_1 + \dots + U_m$ being a direct sum forces each $u_j$ to equal $0$ and, thus, the coefficients of the basis vectors of $U_j$ must also equal $0$, proving that the list is linear independent. 94 | 95 | Therefore, this list is a basis of $V$ and its length is $\operatorname{dim} U_1 + \dots + \operatorname{dim} U_m$, as desired. 96 | 97 | _Exercise 17_ 98 | 99 | The statement is not true. 100 | 101 | Suppose it was: let us consider 102 | $$ 103 | \begin{aligned} 104 | U_1 &= \{(x, 2x) | x \in \mathbb{R}\}\\ 105 | U_2 &= \{(x,x) | x \in \mathbb{R}\}\\ 106 | U_3 &= \{(x, 0.5x) | x \in \mathbb{R}\} 107 | \end{aligned} 108 | $$ 109 | 110 | We have that $\operatorname{dim} U_1 = \operatorname{dim} U_2 = \operatorname{dim} U_3 = 1$, $\operatorname{dim} (U_1 \cap U_2) = \operatorname{dim} (U_1 \cap U_3) = \operatorname{dim} (U_2 \cap U_3) = 0$, $\operatorname{dim} (U_1 \cap U_2 \cap U_3) = 0$. So, if we apply formula, we get that $\operatorname{dim} (U_1 + U_2 + U_3) = 1 + 1 + 1 - 0 - 0 - 0 + 0 = 3$, but this is wrong, since $\operatorname{dim} (U_1 + U_2 + U_3) = \operatorname{dim} \mathbb{R}^2 = 2$. -------------------------------------------------------------------------------- /Chapter 03 - Linear Maps/3A - The Vector Space of Linear Maps.md: -------------------------------------------------------------------------------- 1 | Chapter 3: **Linear Maps** 2 | 3 | **3.A** 4 | 5 | - [ ] Exercise 1 6 | - [ ] Exercise 2 7 | - [x] Exercise 3 8 | - [x] Exercise 4 9 | - [ ] Exercise 5 10 | - [ ] Exercise 6 11 | - [ ] Exercise 7 12 | - [x] Exercise 8 13 | - [x] Exercise 9 14 | - [x] Exercise 10 15 | - [x] Exercise 11 16 | - [x] Exercise 12 17 | - [x] Exercise 13 18 | - [x] Exercise 14 19 | 20 | _Exercise 3_ 21 | 22 | Let us denote $u_1,...,u_n$ to be canonical basis of $\mathbb{F}^n$, i.e. $u_1 = (1,0,...,0)$, ..., $u_n = (0,0,...,1)$. We can define $T$ fully by defining its effect on this basis: 23 | 24 | $$ 25 | \begin{align} 26 | T(u_1) &= (A_{11},A_{21}...,A_{m1})\\ 27 | T(u_2) &= (A_{12},A_{22}...,A_{m2})\\ 28 | \ldots & \\ 29 | T(u_n) &= (A_{1n},A_{2n}...,A_{mn})\\ 30 | \end{align} 31 | $$ 32 | 33 | Then for arbitrary vector $v \in \mathbb{F}^n, v = (x_1,...,x_n)$ we have 34 | 35 | $$ 36 | \begin{align} 37 | T(v) &= x_1T(u_1) + x_2 T(u_2) + \ldots + x_n T(u_n) \\ 38 | &= (A_{11}x_1 + A_{12}x_2 + \ldots + A_{1n}x_n,\\ 39 | & A_{21}x_1 + A_{22}x_2 + \ldots + A_{2n}x_n,\\ 40 | &\ldots \\ 41 | &A_{m1}x_1 + A_{m2}x_2 + \ldots + A_{mn}x_n) 42 | \end{align} 43 | $$ 44 | 45 | _Exercise 4_ 46 | 47 | Let us consider $a_1v_1 + a_2v_2 + \ldots + a_mv_m = 0$ and apply T to both parts of the equation. We have 48 | 49 | $$ 50 | T(a_1v_1 + a_2v_2 + \ldots + a_mv_m) = T(0) 51 | $$ 52 | 53 | $T(0)$ is 0, as we have proved, and we can use linearity on the left, after which we have 54 | 55 | $$ 56 | a_1T(v_1) + a_2T(v_2) + \ldots + a_mT(v_m) = 0 57 | $$ 58 | 59 | We know that $Tv_1,...,Tv_m$ are linearly independent, which implies that the equation above can hold only when $a_1 = a_2 = ... = a_m$ = 0. But this means, that the original equation holds only under the same conditions, which means that $v_1,...,v_m$ are linearly independent. 60 | 61 | _Exercise 8_ 62 | 63 | I tried to come up with something simple, and I think that $\varphi(x, y) = \operatorname{sign}(x)\sqrt{|xy|}$ is a good choice. We can easily show, that $\varphi(ax,ay) = \operatorname{sign}(ax)\sqrt{|a^2xy|} = \operatorname{sign}(ax)|a|\sqrt{|xy|} = a\operatorname{sign}(x)\sqrt{|xy|} = a\varphi(x,y)$. However, it is obvious that $\varphi$ is not a linear map, since $\varphi(x + 0, y_1 + y_2) = \operatorname{sign}(x)\sqrt{|x(y_1 + y_2)|} \neq \operatorname{sign}(x)\sqrt{|xy_1|} + \operatorname{sign}(0)\sqrt{|0y_2|} = \operatorname{sign}(x)\sqrt{|xy_1|}$ 64 | 65 | _Exercise 9_ 66 | 67 | An example of a function which is additive, but non-homogeneous is $\varphi(x) = x^*$: 68 | 69 | For instance, if we consider $\varphi(i * i) = \varphi(-1) = -1 \neq i\varphi(i) = i\cdot(-i) = 1$. 70 | 71 | _Exercise 10_ 72 | 73 | Let us consider any element $u \in U$ and any element $w \in \{V \backslash U\}$: then $u + w \notin U$, because else we could subtract $u$ from it and get that $w \in U$. Having that, we can write 74 | 75 | $$ 76 | \begin{align} 77 | T(u + v) &= 0 \\ 78 | T(u + v) &= Tu + Tv = Su 79 | \end{align} 80 | $$ 81 | 82 | which means that $Tu = 0$ for any $u \in U$. But this is a contradiction, since S is not 0 map. 83 | 84 | _Exercise 11_ 85 | 86 | Let us denote $u_1,...,u_m$ to be a basis of $U$. We can extend it to a basis of $V$ with some $v_1,...,v_k$. We now define our linear map to be 87 | 88 | $$ 89 | \begin{align} 90 | Tu_i &= Su_i \ \forall i = 1...m \\ 91 | Tv_j &= 0 \ \forall j = 1...k 92 | \end{align} 93 | $$ 94 | 95 | _Exercise 12_ 96 | 97 | Suppose by contradiction that $\mathcal{L}(V, W)$ is finite-dimensional. Let $T_1,\dots,T_n$ be a basis of $\mathcal{L}(V, W)$ and $v_1,\dots,v_m$ of $V$. For every $w \in W$ there exists a linear map $T \in \mathcal{L}(V, W)$ such that 98 | 99 | $$ 100 | Tv_1 = w, Tv_j = 0, \text{ for } j = 2,\dots,m. 101 | $$ 102 | 103 | Because $T \in \mathcal{L}(V, W)$ we have 104 | 105 | $$ 106 | T = a_1 T_1 + \dots + a_n T_n 107 | $$ 108 | 109 | for some $a_1,\dots,a_n \in \mathbb{F}$. Thus 110 | 111 | $$ 112 | a_1 T_1 v_1 + \dots + a_n T_n v_1 = w. 113 | $$ 114 | 115 | Hence, $w \in \operatorname{span}(T_1 v_1,\dots,T_n v_n)$. Because $w$ was arbitrary, we have that $W$ is a subspace of $\operatorname{span}(T_1 v_1,\dots,T_n v_1)$, which is finite-dimensional. By 2.26, $W$ is also finite-dimensional, and we get a contradiction. 116 | 117 | _Exercise 13_ 118 | 119 | If $v_k = 0$ for some $k$ when we can choose any nonzero vector as $w_k$. 120 | So assume $v_k \neq 0$ for each $k$. 121 | Let $n$ be the smallest integer such that $v_1, \dots, v_n$ is linearly independent. 122 | Then we can write 123 | 124 | $$ 125 | v_n = a_1 v_1 + \cdots + a_{n-1}v_{n-1} 126 | $$ 127 | 128 | for some scalars $a_1, \dots, a_{n-1} \in \mathbb{F}$. 129 | Then it suffices to choose any $w_1, \dots, w_n$ (the $w_k$ for $k > n$ won't make a difference) such that 130 | 131 | $$ 132 | w_n \neq a_1 w_1 + \cdots + a_{n-1}w_{n-1} 133 | $$ 134 | 135 | because if $T \in \mathcal{L}(V, W)$, with $Tv_k = w_k$ for each $k$, we must have 136 | 137 | $$ 138 | \begin{align} 139 | w_n 140 | &= Tv_n\\\\ 141 | &= T(a_1 v_1 + \cdots + a_{n-1}v_{n-1})\\\\ 142 | &= a_1 Tv_1 + \cdots + a_{n-1}Tv_{n-1}\\\\ 143 | &= a_1 w_1 + \cdots + a_{n-1}w_{n-1}. 144 | \end{align} 145 | $$ 146 | 147 | _Exercise 14_ 148 | 149 | Let $v_1,\dots,v_n$ be a basis of $V$. Define $S,T \in \mathcal{L}(V, V)$ such that 150 | 151 | $$ 152 | \begin{aligned} 153 | S(a_1v_1 + \dots + a_nv_n) &= a_nv_1 + \dots + a_1v_n\\\\ 154 | T(a_1v_1 + \dots + a_nv_n) &= a_1v_1 + \dots + a_{n-1}v_{n-1}. 155 | \end{aligned} 156 | $$ 157 | 158 | By 3.5 $S$ and $T$ are well-defined linear maps. But 159 | 160 | $$ 161 | \begin{align*} 162 | STv_n &= S0 = 0\\\\ 163 | TSv_n &= Tv_1 = v_1 164 | \end{align*}. 165 | $$ 166 | 167 | Hence, $ST \neq TS$. 168 | -------------------------------------------------------------------------------- /Chapter 08 - Operators on Complex Vector Spaces/8D - Jordan Form.md: -------------------------------------------------------------------------------- 1 | Chapter 8: **Operators on Complex Vector Spaces** 2 | 3 | **8.D** 4 | 5 | - [x] Exercise 1 6 | - [x] Exercise 2 7 | - [x] Exercise 3 8 | - [x] Exercise 4 9 | - [x] Exercise 5 10 | - [x] Exercise 6 11 | - [x] Exercise 7 12 | - [x] Exercise 8 13 | 14 | _Exercise 1_ 15 | 16 | By Exercise 11 in section 8B the characteristic polynomial is $z^4$ and by 8.46 this is a polynomial multiple of the minimal polynomial. 17 | Since $N^3 \neq 0$, it follows that the minimal polynomial of $N$ is $z^4$. 18 | 19 | _Exercise 2_ 20 | 21 | Similarly, the characteristic polynomial is $z^6$ and a quick computation shows that the matrix of $N$ equals $0$ when raised to the third power, hence the minimal polynomial of $N$ is $z^3$. 22 | 23 | _Exercise 3_ 24 | 25 | Let $\mathcal{M}(N)$ denote the matrix of $N$ with respect to some Jordan basis for $N$. 26 | Then $\mathcal{M}(N)$ is a block diagonal matrix of the form 27 | 28 | $$ 29 | \mathcal{M}(N) = 30 | \begin{pmatrix} 31 | A_1 & & 0\\\\ 32 | & \ddots &\\\\ 33 | 0 & & A_p 34 | \end{pmatrix}. 35 | $$ 36 | 37 | Because $N$ is nilpotent, $0$ is the only eigenvalue of $N$ (see Exercise 7 in section 8A). 38 | Hence the diagonal entries of $\mathcal{M}(N)$ are all $0$ and each $A_j$ has the following form 39 | 40 | $$ 41 | A_j = 42 | \begin{pmatrix} 43 | 0 & 1 & & 0\\\\ 44 | & \ddots & \ddots &\\\\ 45 | & & \ddots & 1\\\\ 46 | 0 & & & 0 47 | \end{pmatrix}. 48 | $$ 49 | 50 | Thus every string of consecutive $1$'s corresponds to one of these blocks and its length is the same as the length of the side of the block minus $1$. 51 | It is easy to see that if $A_j$ is $n$-by-$n$, then $A_j^{n-1} \neq 0$ and $A_j^n = 0$ (think of it as the matrix of an operator on a $n$ dimensional vector space with respect some basis, each basis vector is mapped to the previous one, except the first one obviously, and to send the last one to $0$ we have to apply the operator $n$ times). 52 | Exercise 9 in section 8B now implies that $\mathcal{M}(N)^{m} \neq 0$ and $\mathcal{M}(N)^{m+1} = 0$. 53 | Hence the minimal polynomial of $N$ is $z^{m+1}$. 54 | 55 | _Exercise 4_ 56 | 57 | The difference is that the order of the blocks in the diagonal is reversed and the $1$'s appear on the line below the diagonal. 58 | 59 | _Exercise 5_ 60 | 61 | By Exercise 9 in section 8B we just need to square each block on the diagonal of the Jordan form of $T$. 62 | In other words, the matrix of $T^2$ is a block diagonal matrix where each block has the following form 63 | 64 | $$ 65 | \begin{pmatrix} 66 | \lambda_j & 1 & & 0\\\\ 67 | & \ddots & \ddots &\\\\ 68 | & & \ddots & 1\\\\ 69 | 0 & & & \lambda_j 70 | \end{pmatrix}^2 = 71 | \begin{pmatrix} 72 | \lambda_j^2 & 2\lambda_j & 1 & & 0\\\\ 73 | & \ddots & \ddots & \ddots &\\\\ 74 | & & \ddots & \ddots & 2\lambda_j\\\\ 75 | 0 & & & \ddots & \lambda_j^2 76 | \end{pmatrix}. 77 | $$ 78 | 79 | _Exercise 6_ 80 | 81 | No vector in the span of 82 | 83 | $$ 84 | N^{m_1-1}v_1, \dots, Nv_1, v_1, \dots, N^{m_n-1}v_n, \dots, Nv_n, v_n 85 | $$ 86 | 87 | is in $\operatorname{null} N$, because applying $N$ to any such vector we get a linear of combination of 88 | 89 | $$ 90 | N^{m_1}v_1, \dots, Nv_1, \dots, N^{m_n}v_n, \dots, Nv_n, 91 | $$ 92 | 93 | which is linearly independent. 94 | The vectors above are all in $\operatorname{range} N$, therefore, by the Fundamental Theorem of Linear Maps (3.22), the dimension of $\operatorname{null} N$ is at most the dimension of $V$ minus the dimension of the span of the vectors above, that is, at most $n$. 95 | Since $N^{m_1}v_1, \dots, N^{m_n} \in \operatorname{null} N$ is linearly independent and has length $n$, it must be a basis of $\operatorname{null} N$. 96 | 97 | _Exercise 7_ 98 | 99 | Let $\lambda_1, \dots, \lambda_m$ denote the distinct zeros of $p$ and $q$. 100 | Then 101 | 102 | $$ 103 | p(z) = (z - \lambda_1)^{d_1} \cdots (z - \lambda_m)^{d_m} 104 | $$ 105 | 106 | and 107 | 108 | $$ 109 | q(z) = (z - \lambda_1)^{h_1} \cdots (z - \lambda_m)^{h_m} 110 | $$ 111 | 112 | for some positive integers $d_1, \dots, d_m, h_1, \dots, h_m$, where $h_j \ge d_j$ for each $j$ (because $q$ is a polynomial multiple of $p$). 113 | Let $A$ equal the following block diagonal matrix 114 | 115 | $$ 116 | A = 117 | \begin{pmatrix} 118 | A_1 & & 0\\\\ 119 | & \ddots &\\\\ 120 | 0 & & A_m 121 | \end{pmatrix} 122 | $$ 123 | 124 | where each $A_j$ is the $h_j$-by-$h_j$ matrix defiend by 125 | 126 | $$ 127 | A_j = 128 | \begin{pmatrix} 129 | \lambda_j & 1 & & & & &\\\\ 130 | & \ddots & \ddots & & & &\\\\ 131 | & & \ddots & 1 & & &\\\\ 132 | & & & \lambda_j & 0 & &\\\\ 133 | & & & & \lambda_j & \ddots &\\\\ 134 | & & & & & \ddots & 0\\\\ 135 | & & & & & & \lambda_j 136 | \end{pmatrix} 137 | $$ 138 | 139 | where the $1$'s appear up to the $d_j$-th column and the $0$'s fill the rest. 140 | Note that $A$ is a $\deg q$-by-$\deg q$ matrix. 141 | Define $T \in \mathcal{L}(\mathbb{C}^{\deg q})$ such that the matrix of $T$ with respect to the standard basis is $A$. 142 | Then each $\lambda_j$ is an eigenvalue of $T$, because $A$ is upper triangular and $\lambda_j$ appears on the diagonal of $A$. 143 | Moreover, the multiplicity of $\lambda_j$ as an eigenvalue of $T$ is $h_j$, by Exercise 11 in section 8B. 144 | Hence, the characteristic polynomial of $T$ is $q$. 145 | 146 | It is easy to see that $(A_j - \lambda_j I)^{d_j - 1} \neq 0$ but $(A_j - \lambda_j I)^{d_j} = 0$ (we can use a reasoning similar to that of Exercise 3 to show this). 147 | This, together with Exercise 9 from section 8B, shows that the $j$-th block is nonzero in $(A - \lambda_j I)^{d_j-1}$ and is zero in $(A - \lambda_j I)^{d_j}$ (pay attention to this fact). 148 | It follows that 149 | 150 | $$ 151 | (A - \lambda_1 I)^{d_1} \cdots (A - \lambda_m I)^{d_m} = 0. 152 | $$ 153 | 154 | Hence $p(T) = 0$. 155 | We claim now $p$ is the minimal polynomial of $T$. 156 | To see this, suppose that it is not. 157 | Then we can subtract $1$ from one of the exponents in the equation above and it will still hold. 158 | Thus, we can write 159 | 160 | $$ 161 | 0 = p'(T)(T - \lambda_j)^{d_j - 1} 162 | $$ 163 | 164 | for some $j \in \\{1, \dots, m\\}$ and some polynomial $p'$, with $p'(\lambda_j) \neq 0$. 165 | Let $v \in G(\lambda_j, T)$ such that $(T - \lambda_j)^{d_j - 1}v \neq 0$ (this $v$ exists due to the fact we pinpointed before). 166 | We have 167 | 168 | $$ 169 | 0 = p'(T)(T - \lambda_j)^{d_j - 1}v, 170 | $$ 171 | 172 | but this is contradiction because $(T - \lambda_j)^{d_j - 1}v \in G(\lambda_j, T)$ and $\lambda_j$ is not an zero of $p'$. 173 | 174 | _Exercise 8_ 175 | 176 | Suppose thre does not exist a direct sum decomposition of $V$ into two proper subspaces invariant under $T$. 177 | Therefore, the block diagonal matrix of $T$ with respect to some Jordan basis for $T$ only has one block (the span of the basis vectors that correspond to each block is invariant under $T$). 178 | Therefore $T$ only has one eigenvalue, call it $\lambda$. 179 | Exercise 3 now implies that $(z - \lambda)^{\dim V}$ is minimal polynomial of $T$. 180 | 181 | For the other direction, we will prove the contrapositive. 182 | Suppose we can decompose $V$ into two proper subspaces invariant under $T$. 183 | If $T$ has more than one eigenvalue the result is obvious. 184 | Assume $T$ has only one eigenvalue, call it $\lambda$. 185 | Then $V = G(\lambda, T)$ and there are proper subspaces $U_1, U_2$ of $G(\lambda, T)$ invariant under $T$ such that $G(\lambda, T) = U_1 \oplus U_2$ with $1 \le \dim U_1, \dim U_2 < \dim V$. 186 | We have that $(z - \lambda)^{\dim U_1}$ and $(z - \lambda)^{\dim U_2}$ are the characteristic polynomials of $T|\_{U_1}$ and $T|\_{U_2}$. 187 | Thus $(T - \lambda)^{\max\\{\dim U_1, \dim U_2\\}} = 0$ and so the minimal polynomial of $T$ cannot be $z^{\dim V}$. 188 | -------------------------------------------------------------------------------- /Chapter 09 - Operators on Real Vector Spaces/9B - Operators on Real Inner Product Spaces.md: -------------------------------------------------------------------------------- 1 | Chapter 9: **Operators on Real Vector Spaces** 2 | 3 | - [x] Exercise 1 4 | - [x] Exercise 2 5 | - [x] Exercise 3 6 | - [x] Exercise 4 7 | - [x] Exercise 5 8 | - [x] Exercise 6 9 | - [x] Exercise 7 10 | - [x] Exercise 8 11 | 12 | _Exercise 1_ 13 | 14 | Choose an orthonormal basis of $\mathbb{R}^3$ that puts the matrix of $S$ in the form given by 9.36. 15 | Since $\mathcal{M}(S)$ is a $3$-by-$3$ matrix, one of the diagonal blocks is a $1$-by-$1$ matrix containing $1$ or $-1$. 16 | Hence $Sx = x$ or $Sx = -x$ for some nonzero vector $x$ in the chosen basis. 17 | Applying $S$ again in the two cases gives $S^2x = x$, as desired. 18 | 19 | Geometrically speaking, an isometry on $\mathbb{R}^3$ is a rotation about an axis, perhaps with a reflexion through a plane orthogonal to the axis. 20 | Hence an isometry on $\mathbb{R}^3$ either sends the vectors in the axis to themselves or to their reflexion. 21 | If it's the first case, we already have what we wanted to prove, if it's the second, applying the isometry again sends the reflexions of the vectors back to the vectors themselves. 22 | 23 | _Exercise 2_ 24 | 25 | This basically the same as the previous exercise. 26 | Every operator on an odd-dimensional vector space has $n$-by-$n$ matrix for some odd integer $n$, hence one of the diagonal blocks is a $1$-by-$1$ matrix containing $1$ or $-1$. 27 | The basis vector that corresponds to column where this vector appears is an eigenvector corresponding to $1$ or $-1$. 28 | 29 | _Exercise 3_ 30 | 31 | For $u_1, u_2, v_1, v_2 \in V$ we have 32 | 33 | $$ 34 | \begin{aligned} 35 | \langle (u_1 + iv_1) + (u_1 + iv_2), x + iy \rangle &= \langle (u_1 + u_2) + i(v_1 + v_2), x + iy \rangle\\\\ 36 | &= \langle u_1 + u_2, x \rangle + \langle v_1 + v_2, y \rangle + \big(\langle v_1 + v_2, x \rangle - \langle u_1 + u_2, y \rangle\big)i\\\\ 37 | &= \langle u_1, x \rangle + \langle u_2, x \rangle +\langle v_1, y \rangle + \langle v_2, x \rangle + \big(\langle v_1, x \rangle - \langle u_1, y \rangle\big)i + \big(\langle v_2, x \rangle - \langle u_2, y \rangle\big)i\\\\ 38 | &= \langle u_1 + iv_1, x + iy \rangle + \langle u_2 + iv_2, x + iy \rangle. 39 | \end{aligned} 40 | $$ 41 | 42 | Therefore it satisfies additivity in the first slot. 43 | For $a, b \in \mathbb{R}$, we have 44 | 45 | $$ 46 | \begin{aligned} 47 | \langle (a + bi)(u + iv), x + iy \rangle &= \langle (au - bv) + i(av + bu), x + iy \rangle\\\\ 48 | &= \langle au - bv, x \rangle + \langle av + bu, y \rangle + \big(\langle av + bu, x \rangle - \langle au - bv, y \rangle\big)i\\\\ 49 | &= \langle au, x \rangle - \langle bv, x \rangle + \langle av, y \rangle + \langle bu, y \rangle + \big(\langle av, x \rangle - \langle au, y \rangle\big)i + (\langle bu, x \rangle + \langle bv, y \rangle)i\\\\ 50 | &= \langle au, x \rangle + \langle av, y \rangle + \big(\langle av, x \rangle - \langle au, y \rangle\big)i - \langle bv, x \rangle + \langle bu, y \rangle + (\langle bu, x \rangle + \langle bv, y \rangle)i\\\\ 51 | &= a\langle u + iv, x + iy \rangle + bi\big(i\langle v, x \rangle - i\langle u, y \rangle + \langle u, x \rangle + \langle v, y \rangle\big)\\\\ 52 | &= a\langle u + iv, x + iy \rangle + bi\langle u + iv, x + iy \rangle\\\\ 53 | &= (a + bi)\langle u + iv, x + iy \rangle 54 | \end{aligned} 55 | $$ 56 | 57 | Hence homogeneity in the first slot is satisfied. 58 | For positivity, we have 59 | 60 | $$ 61 | \langle u + iv, u + iv \rangle = \langle u, u \rangle + \langle v, v \rangle + \big(\langle v, u \rangle - \langle u, v \rangle\big)i = \langle u, u \rangle + \langle v, v \rangle \ge 0, 62 | $$ 63 | 64 | where the second equality follows because $V$ is a real inner product space. 65 | The equation above also displays definiteness, because the left side equals $0$ if and only if $u = v = 0$, in other words, if $u + iv = 0$. 66 | For conjugate symmetry, we have 67 | 68 | $$ 69 | \begin{aligned} 70 | \langle u + iv, x + iy \rangle &= \overline{\langle u, x \rangle + \langle v, y \rangle - (\langle v, x \rangle - \langle u, y \rangle)i}\\\\ 71 | &= \overline{\langle x, u \rangle + \langle y, v \rangle - (\langle x, v \rangle - \langle y, u \rangle)i}\\\\ 72 | &= \overline{\langle x + iy, u + iv \rangle}. 73 | \end{aligned} 74 | $$ 75 | 76 | _Exercise 4_ 77 | 78 | Choose an orthonormal basis of $V$. 79 | Then this is also a basis of $V_\mathbb{C}$, see (9.4). 80 | We have 81 | 82 | $$ 83 | \mathcal{M}(T_\mathbb{C}) = \mathcal{M}(T) = \mathcal{M}(T^\*) = \mathcal{M}((T^\*)\_\mathbb{C}) = \mathcal{M}((T_\mathbb{C})^\*), 84 | $$ 85 | 86 | where the first and third equalities follow from 9.7, the second from 7.10 and the fourth from 9.30 (c) (take $U = V$). 87 | This implies that $T_\mathbb{C} = (T_\mathbb{C})^\*$. 88 | 89 | _Exercise 5_ 90 | 91 | Suppose $V$ is a real inner product space and $T \in \mathcal{L}(V)$ is self-adjoint. 92 | Then $T_\mathbb{C}$ is a self-adjoint operator on the complex inner product space $V_\mathbb{C}$ defined in Exercise 3. 93 | By the Complex Spectral Theorem, there is an orthonormal basis $e_1 + if_1, \dots, e_n + if_n$ of $V_\mathbb{C}$ consisting of eigenvectors of $T_\mathbb{C}$. 94 | By 7.13, the eigenvalues of $T_\mathbb{C}$ are all real. 95 | Thus, there are $\lambda_1, \dots, \lambda_n \in \mathbb{R}$ such that 96 | 97 | $$ 98 | Te_j + iTf_j = T_\mathbb{C}(e_j + if_j) = \lambda_j e_j + i \lambda_j Tf_j 99 | $$ 100 | 101 | for each $j$. 102 | The equation above shows that the $e$'s and $f$'s are eigenvectors of $T$. 103 | Fix $j \in \\{1, \dots, n\\}$. 104 | Let $e_1' + if_1', \dots, e_{d(\lambda_j)}' + if_{d(\lambda_j)}'$ denote the basis vectors that correspond to $\lambda_j$, where $d(\lambda_j) = \dim E(\lambda_j, T_\mathbb{C})$. 105 | Then the list 106 | 107 | $$ 108 | e_1', f_1', \dots, e_{d(\lambda_j)}', f_{d(\lambda_j)}' \in E(\lambda_j, T) 109 | $$ 110 | 111 | is in $E(\lambda_j, T)$. 112 | Since $E(\lambda, T_\mathbb{C})$ is contained in the complex span of the list above, it follows that the dimension of the compelx span of the list above is at least $d(\lambda_j)$. 113 | Thus, by 9.4, the dimension of the real span of the list above is at least $d(\lambda_j)$. 114 | Hence $\dim E(\lambda_j, T) \ge d(\lambda_j)$. 115 | Because 116 | 117 | $$ 118 | \dim E(\lambda_1, T) + \dots + \dim E(\lambda_n, T) \le \dim V 119 | $$ 120 | 121 | and 122 | 123 | $$ 124 | d(\lambda_1) + \dots + d(\lambda_n) = \dim V 125 | $$ 126 | 127 | we must have $\dim E(\lambda_j, T) = d(\lambda_j)$ for each $j$. 128 | The sum of eigenspaces corresponding to different eigenvalues is a direct sum. 129 | The equation above thus shows that the sum of the eigenspaces of $T$ has the same dimension as $V$. 130 | This means that 131 | 132 | $$ 133 | E(\lambda_1, T) \oplus \dots \oplus E(\lambda_n, T) = V. 134 | $$ 135 | 136 | For each eigenspace of $T$, we can select an orthonormal basis. 137 | Putting these bases together, we get a basis of $V$ (by the equation above) consisting of eigenvectors of $T$. 138 | By 7.22, this basis is orthonormal. 139 | 140 | _Exercise 6_ 141 | 142 | Define $T \in \mathcal{L}(\mathbb{R}^2)$ by 143 | 144 | $$ 145 | T(x, y) = (x + y, y). 146 | $$ 147 | 148 | Then $\operatorname{span}((1, 0))$ is invariant under $T$, however, its orthogonal complement, namely $\operatorname{span}((0, 1))$, is not, because 149 | 150 | $$ 151 | T(0, 1) = (1, 1) \not \in \operatorname{span}((0, 1)). 152 | $$ 153 | 154 | _Exercise 7_ 155 | 156 | We have 157 | 158 | $$ 159 | \mathcal{M}(T_1 \cdots T_m) = \mathcal{M}(T_1) \cdots \mathcal{M}(T_m) = \mathcal{M}(T), 160 | $$ 161 | 162 | where the second equality follows from Exercise 9 in section 8B. 163 | The left and right side of the equation above imply that $T_1 \cdots T_m = T$. 164 | 165 | _Exercise 8_ 166 | 167 | Define 168 | 169 | $$ 170 | e_j = \frac{\cos jx}{\sqrt{\pi}} \text{ and } f_j = \frac{\sin jx}{\sqrt{\pi}} 171 | $$ 172 | 173 | for each $j = 1, \dots, n$. 174 | By Exercise 4 in section 6B, the list $\frac{1}{\sqrt{2\pi}}, f_1, e_1, \dots, f_n, e_n$ is an orthonormal basis of $V$. 175 | Note that $D\frac{1}{\sqrt{2\pi}} = 0$, $Df_j = je_j$ and $De_j = -jf_j$. 176 | Hence, the matrix of $D$ with respect to this basis has the desired form, where the first block is the $1$-by-$1$ matrix $\begin{pmatrix}0\end{pmatrix}$ and others are of the form 177 | 178 | $$ 179 | \begin{pmatrix} 180 | 0 & -j\\\\ 181 | j & 0 182 | \end{pmatrix} 183 | $$ 184 | 185 | for some $j \in \{1, \dots, n\}$. 186 | -------------------------------------------------------------------------------- /Chapter 10 - Trace and Determinant/10B - Determinant.md: -------------------------------------------------------------------------------- 1 | Chapter 10: **Trace and Determinant** 2 | 3 | - [x] Exercise 1 4 | - [x] Exercise 2 5 | - [x] Exercise 3 6 | - [x] Exercise 4 7 | - [x] Exercise 5 8 | - [x] Exercise 6 9 | - [x] Exercise 7 10 | - [x] Exercise 8 11 | - [x] Exercise 9 12 | - [x] Exercise 10 13 | - [x] Exercise 11 14 | - [x] Exercise 12 15 | 16 | _Exercise 1_ 17 | 18 | Because $T_\mathbb{C}$ has no real eigenvalues, if $\lambda$ is an eigenvalue of $T_\mathbb{C}$, then $\overline{\lambda}$ is also an eigenvalue of $T_\mathbb{C}$ with equal multiplicity. 19 | The determinant of $T_\mathbb{C}$, which equals the determinant of $T$, is thus a product of terms of the form $\lambda\overline{\lambda} = |\lambda|^2$, which are all greater than $0$, because $0$ is not eigenvalue of $T$. 20 | Hence $\det T > 0$. 21 | 22 | _Exercise 2_ 23 | 24 | By the previous exercise, $T$ has at least one eigenvalue $\alpha$, which is negative because $\det T < 0$. 25 | Suppose by contradiction that $\alpha$ is the only eigenvalue of $T$. 26 | Then this can't be the only eigenvalue of $T_\mathbb{C}$, otherwise the $\det T$ would equal $\lambda^{\dim V}$ which is positive (because $\dim V$ is even). 27 | The other eigenvalues of $T_\mathbb{C}$ are nonreal, so they come in pairs and won't change sign of $\det T$. 28 | It follows that $\det T$ is a product of absolute values times $\lambda$ raised to its multiplicity. 29 | Thus the multiplicity of $\lambda$ must be odd. 30 | This is a contradiction, because the sum of the multiplicities of the other eigenvalues of $T_\mathbb{C}$ is even (because they come in pairs) and the sum of all multiplicies must equal $\dim V$, which is even. 31 | Hence, our assumption that $\lambda$ was the only eigenvalue of $T$ is false. 32 | 33 | _Exercise 3_ 34 | 35 | _(a)_ 36 | The characteristic polynomial of $T$ is 37 | 38 | $$ 39 | (z - \lambda_1) (z - \lambda_2) \cdots (z - \lambda_n). 40 | $$ 41 | 42 | Expanding this product we see that the coefficient of $z^{n-2}$ is the sum of the products of all pairs of distinct eigenvalues. 43 | In other words, it equals 44 | 45 | $$ 46 | \lambda_1 \sum_{j=2}^n \lambda_j + \lambda_2 \sum_{j=3}^n \lambda_j + \dots + \lambda_{n-1} \sum_{j=n}^n \lambda_j. 47 | $$ 48 | 49 | _(b)_ 50 | It equals the sum of the products of the eigenvalues of $T$ with one term missing: 51 | 52 | $$ 53 | \sum_{j=1}^n \prod_{k \neq j} (-1)^{n-1}\lambda_k. 54 | $$ 55 | 56 | More succinctly, assuming $0$ is not an eigenvalue of $T$: 57 | 58 | $$ 59 | \sum_{j=1}^n (-1)^{n-1}\frac{\det T}{\lambda_j}. 60 | $$ 61 | 62 | _Exercise 4_ 63 | 64 | This is follows easily from the definitions. 65 | If $\lambda$ is an eigenvalue of $T$, or $T_\mathbb{C}$, then $c\lambda$ is an eigenvalue of $cT$, or $(cT)_\mathbb{C}$, with equal multiplicity. 66 | Let $\lambda_1, \dots, \lambda_n$ denote the eigenvalues of $T_\mathbb{C}$, repeated according to its multiplicity. 67 | Then $c\lambda_1, \dots, c\lambda_n$ are the eigenvalues of $(cT)_\mathbb{C}$ and we have 68 | 69 | $$ 70 | \det (cT) = (c \lambda_1) \cdots (c \lambda_n) = c^{\dim V}\lambda_1 \cdots \lambda_n = c^{\dim V} \det T. 71 | $$ 72 | 73 | _Exercise 5_ 74 | 75 | We give a counterexample. 76 | Let $I$ be the identity operator on $\mathbb{R}^2$. 77 | Then 78 | 79 | $$ 80 | \det(I - I) = 0 \neq 2 = \det(I) + \det(-I). 81 | $$ 82 | 83 | _Exercise 6_ 84 | 85 | As we know, 86 | 87 | $$ 88 | \operatorname{det}A = \sum\limits_{(m_1,..,m_n)\in \operatorname{perm}(n)}\operatorname{sign}(m_1,..,m_n)A_{m_1,1}...A_{m_n,n} 89 | $$ 90 | 91 | Denote $n_i$ to be number of rows/columns in $A_i$. Since $A$ is block-diagonal, we can notice, that for every permutation, having $m_i > n_1$ for any $i = 1..n_1$ we have $A_{m_i,i} = 0$. This means, that the sum above has non-zero terms only for permutations which map range $1..n_1$ to range $1..n_1$. We can note, that for such permutations, all the inversions happen with $(i,j)$ which are either both $\le n_1$ or $\ge n_1$, which means that $\operatorname{sign}(m_1,...,m_n) = \operatorname{sign}(m_1,...,m_{n_1})\cdot \operatorname{sign}(m_{n_1+1},...,m_n)$ This means, that we can rewrite equation above as 92 | 93 | $$ 94 | \begin{align} 95 | \begin{split} 96 | \operatorname{det}A &= \sum\limits_{(m_1,..,m_{n_1})\in \operatorname{perm}(n_1)}\sum\limits_{(m_{n_1+1},..,m_{n})\in \operatorname{perm}([n_1+1,...,n])}\operatorname{sign}(m_{1},..,m_{n_1})\operatorname{sign}(m_{n_1 +1},..,m_n)A_{m_1,1}...A_{m_{n_1},n_1} A_{m_{n_1+1},n_1+1}...A_{m_n,n} \\ 97 | &= \sum\limits_{(m_1,..,m_{n_1})\in \operatorname{perm}(n_1)}\operatorname{sign}(m_{1},..,m_{n_1})A_{m_1,1}...A_{m_{n_1},n_1} \sum\limits_{(m_{n_1+1},..,m_{n})\in \operatorname{perm}([n_1+1,...,n])}\operatorname{sign}(m_{n_1 +1},..,m_n) A_{m_{n_1+1},n_1+1}...A_{m_n,n}\\ 98 | &= \operatorname{det}A_1 \sum\limits_{(m_{n_1+1},..,m_{n})\in \operatorname{perm}([n_1+1,...,n])}\operatorname{sign}(m_{n_1 +1},..,m_n) A_{m_{n_1+1},n_1+1}...A_{m_n,n} 99 | \end{split} 100 | \end{align} 101 | $$ 102 | 103 | Continuing in this fashion, we can see that the same argument can be applied to ${m_{n_1+1},...,m_{n_1+n_2}}$: it should be permutation of ${n_1+1,..,n_1+n_2}$, and it goes further until eventually we get 104 | 105 | $$ 106 | \operatorname{det}A = \operatorname{det}A_1...\operatorname{det}A_m 107 | $$ 108 | 109 | _Exercise 7_ 110 | 111 | From 10.16, we have 112 | 113 | $$ 114 | \operatorname{trace} S = \operatorname{trace} A = \operatorname{trace} T 115 | $$ 116 | 117 | and, from 10.42, we have 118 | 119 | $$ 120 | \det S = \det A = \det T. 121 | $$ 122 | 123 | _Exercise 8_ 124 | 125 | Suppose $\mathbb{F} = \mathbb{C}$. 126 | Choose an orthonormal basis of $V$ with respect to which the matrix of $T$ is upper triangular. 127 | Then, by 10.35, the determinant of $\mathcal{M}(T)$ equals the product of the entries on the diagonal. 128 | By the same reasoning used in 10.35, the determinant of $\mathcal{M}(T^*)$ is also the product of the entries on diagonal. 129 | The diagonal entries of $\mathcal{M}(T^*)$ are the conjugates of the diagonal entries of $\mathcal{M}(T)$. 130 | Hence $\det \mathcal{M}(T^*) = \overline{\det \mathcal{M}(T)}$. 131 | Now 10.42 implies that $\det T^* = \overline{\det T}$. 132 | 133 | Form 10.44, we have 134 | 135 | $$ 136 | (\det \sqrt{T^*T})^2 = \det (\sqrt{T^*T}\sqrt{T^*T}) = \det (T^*T) = \det T^* \det T = \overline{\det T} \det T = \left|\det T\right|^2. 137 | $$ 138 | 139 | Taking the square root of each side we get $\det \sqrt{T^*T} = \left|\det T\right|$. 140 | 141 | _Exercise 9_ 142 | 143 | Consider $y = (0,..,\delta x_i, 0, .., 0)$. We can write 144 | 145 | $$ 146 | \begin{align} 147 | \sigma(x + y) - \sigma(x) &= \sigma(x_1, ..., x_i + \delta x_i, ... x_n) - \sigma(x_1,..,x_n) \\ 148 | &= (\sigma'_{x_i}(x_1,..,x_n) + \mathcal{o}(\delta x_i))\delta x_i 149 | \end{align} 150 | $$ 151 | 152 | In the equations above, $T$ is applied to a vector with 1 on $i$th position: 153 | $$ 154 | \begin{align} 155 | \operatorname{lim}_{y\to 0}\frac{||\sigma(x + y) - \sigma(x) - T(y)||}{||y||} &=_{y = (0,...\delta x_i, ...0)} 156 | \operatorname{lim}_{\delta x_i \to 0}\frac{||(\sigma'_{x_i}(x_1,..,x_n) + \mathcal{o}(\delta x_i))\delta x_i - \delta x_iT(0,..,1,..0)||}{|\delta x_i|} \\ 157 | &= ||\sigma'_{x_i}(x_1,..,x_n) - T(0,...,1,...0)|| 158 | \end{align} 159 | $$ 160 | 161 | We know that from inner product definition the only vector that has a zero norm is a zero vector, so $T(0,..1,..0) = \sigma'_{x_i}(x_1,..,x_n)$. This means, that on a standard basis $T$ is defined uniquely, and this means that it is defined uniquely on $\mathbb{R}^n$, so there exists only one such linear operator which satisfies the equation 10.56. 162 | 163 | _Exercise 10_ 164 | 165 | We need to substitute $\sigma(x) = Tx$ into 10.56: 166 | 167 | Suppose $T' = S$, we will prove that $S = T$: 168 | 169 | $$ 170 | \begin{align} 171 | \operatorname{lim}_{y\to 0}\frac{||T(x + y) - T(x) - S(y)||}{||y||} &= \operatorname{lim}_{y\to 0}\frac{||T(x) + T(y) - T(x) - S(y)||}{||y||} \\ 172 | &= \operatorname{lim}_{y\to 0}\frac{||(T-S)(y)||}{||y||} \\ 173 | &= \operatorname{lim}_{y\to 0}\frac{||(T-S)(e_y)|| ||y||}{||y||}\\ 174 | &= \operatorname{lim}_{y\to 0}||(T-S)(e_y)|| = 0 175 | \end{align} 176 | $$ 177 | 178 | Where $y = ||y||e_y$. We see, that $(S-T)e_y = 0$ for arbitrary union vector $e_y$, so taking arbitrary orthonormal basis we conclude that $(S-T)$ is 0 on all vectors of this basis, hence $S-T = 0$, and so $S = T$. 179 | 180 | _Exercise 11_ 181 | 182 | The proof can be derived the same way we proved the uniqueness of $T$ in exercise 9. 183 | 184 | _Exercise 12_ 185 | 186 | We know the volume of a unit ball, which is $\frac{4}{3}\pi$. It is easy to see that described ellipsoid is linearly shrinked unit ball with linear map $T(x,y,z) = (ax, by, cz)$. This linear map has $\operatorname{det}T = abc$, and so ellipsoid volume is $\frac{4}{3}\pi abc$ -------------------------------------------------------------------------------- /Chapter 07 - Operators on Inner Product Spaces/7B - The Spectral Theorem.md: -------------------------------------------------------------------------------- 1 | Chapter 7: **Operators on Inner Product Spaces** 2 | 3 | **7.B** 4 | 5 | - [x] Exercise 1 6 | - [x] Exercise 2 7 | - [x] Exercise 3 8 | - [x] Exercise 4 9 | - [x] Exercise 5 10 | - [x] Exercise 6 11 | - [x] Exercise 7 12 | - [x] Exercise 8 13 | - [x] Exercise 9 14 | - [x] Exercise 10 15 | - [x] Exercise 11 16 | - [x] Exercise 12 17 | - [x] Exercise 13 18 | - [x] Exercise 14 19 | - [x] Exercise 15 20 | 21 | _Exercise 1_ 22 | 23 | False. 24 | Define $T \in \mathcal{L}(\mathbb{R}^3)$ by 25 | 26 | $$ 27 | \begin{aligned} 28 | Te_1 &= e_1\\\\ 29 | Te_2 &= 2e_2\\\\ 30 | T(e_1 + e_2 + e_3) &= 3e_1 + 3e_2 + 3e_3, 31 | \end{aligned} 32 | $$ 33 | 34 | where $e_1, e_2, e_3$ is the standard basis of $\mathbb{R}^3$. 35 | Clearly, $e_1, e_2, e_1 + e_2 + e_3$ is a basis of $\mathbb{R}^3$ consisting of eigenvectors of $T$. 36 | However, the matrix of $T$ with respect to the standard basis (which is orthonormal) is 37 | 38 | $$ 39 | \begin{pmatrix} 40 | 1 & 0 & 3\\\\ 41 | 0 & 2 & 1\\\\ 42 | 0 & 0 & 3 43 | \end{pmatrix}, 44 | $$ 45 | 46 | which does not equal its transpose (the matrix of $T^*$, by 7.10). 47 | Therefore $T$ is not self-adjoint. 48 | 49 | _Exercise 2_ 50 | 51 | Since $T$ is self-adjoint, there exists a basis consisting of eigenvectors of $T$, by either 7.24 or 7.29 (depending on the choice of $\mathbb{F}$). 52 | Because $x^2 - 5x + 6 = (x - 2)(x - 3)$, we have 53 | 54 | $$ 55 | T^2 - 5T + 6I = (T - 2I)(T - 3I). 56 | $$ 57 | 58 | Thus, applying it to any vector in the basis will give $0$ (for the eigenvectors corresponding to $2$ just switch the order to $(T - 3I)(T - 2I)$, which is valid by 5.20). 59 | 60 | _Exercise 3_ 61 | 62 | Define $T \in \mathcal{L}(\mathbb{C}^3)$ by 63 | 64 | $$ 65 | \begin{aligned} 66 | Te_1 &= 2e_1\\\\ 67 | Te_2 &= 3e_2\\\\ 68 | Te_3 &= 3(e_1 + e_2 + e_3), 69 | \end{aligned} 70 | $$ 71 | 72 | where $e_1, e_2, e_3$ is the standard basis of $\mathbb{C}^3$. 73 | The matrix of $T$ with respect to same basis is 74 | 75 | $$ 76 | \mathcal{M}(T) = \begin{pmatrix} 77 | 2 & 0 & 3\\\\ 78 | 0 & 3 & 3\\\\ 79 | 0 & 0 & 3 80 | \end{pmatrix}. 81 | $$ 82 | 83 | Therefore, the only eigenvalues of $T$ are $2$ and $3$ (by 5.32). 84 | Moreover 85 | 86 | $$ 87 | \begin{aligned} 88 | (T^2 - 5T + 6I)e_3 &= T^2e_3 - 5Te_3 + 6Ie_3\\\\ 89 | &= T(3e_1 + 3e_2 + 3e_3) - 15(e_1 + e_2 + e_3) + 6e_3\\\\ 90 | &= 6e_1 + 9e_2 + 9(e_1 + e_2 + e_3) - 15(e_1 + e_2 + e_3) + 6e_3\\\\ 91 | &= 3e_2\\\\ 92 | &\neq 0. 93 | \end{aligned} 94 | $$ 95 | 96 | Therefore, $T^2 - 5T + 6I \neq 0$. 97 | 98 | _Exercise 4_ 99 | 100 | The forward direction is basically the same as 7.22 and 7.24. 101 | 102 | Suppose all pairs of eigenvectors corresponding to distinct eigenvalues of $T$ are orthogonal and 103 | 104 | $$ 105 | V = E(\lambda_1, T) \oplus \dots \oplus E(\lambda_m, T), 106 | $$ 107 | 108 | where $\lambda_1, \dots, \lambda_m$ are distinct eigenvalues of $T$. 109 | Consider the constructed by concatenating orthonormal bases of $E(\lambda_1, T), \dots, E(\lambda_m, T)$. 110 | By 5.41 (e) this list has length $\operatorname{dim} V$. 111 | Moreover, because all pairs of eigenvectors corresponding to distinct eigenvalues of $T$ are orthogonal, by 6.26 this is list is also linearly indepedent. 112 | Therefore $V$ has an orthonormal basis consisting of eigenvectors of $T$ and is normal by 7.24. 113 | 114 | _Exercise 5_ 115 | 116 | Again, the forward direction is basically 7.22 (since $T$ is also normal) and 7.29 and the converse is the same as the previous exercise. 117 | 118 | _Exercise 6_ 119 | 120 | Suppose $T$ is self-adjoint. 121 | Let $\lambda$ be an eigenvalue of $T$ and $v$ a corresponding eigenvector. 122 | Then 123 | 124 | $$ 125 | \lambda \langle v, v \rangle = \langle Tv, v \rangle = \langle v, Tv \rangle = \overline{\lambda} \langle v, v \rangle. 126 | $$ 127 | 128 | Therefore $\lambda = \overline{\lambda}$, that is, $\lambda$ is real. 129 | 130 | Conversely, suppose all eigenvalues of $T$ are real. 131 | Let $e_1, \dots, e_n$ denote an orthonormal basis consisting of eigenvalues of $T$ (which exists by 7.24) and let $\lambda_1, \dots, \lambda_n$ denote their corresponding eigenvalues (which are real). 132 | For any $v \in V$, we can write 133 | 134 | $$ 135 | v = a_1 e_1 + \dots + a_n e_n 136 | $$ 137 | 138 | for some $a_1, \dots, a_n \in \mathbb{C}$. 139 | Then 140 | 141 | $$ 142 | \langle Tv, v \rangle = \langle \lambda_1 a_1 e_1 + \dots + \lambda_n a_n e_n, a_1 e_1 + \dots + a_n e_n \rangle = \lambda_1 |a_1|^2 + \dots + \lambda_n |a_n|^2 \in \mathbb{R}. 143 | $$ 144 | 145 | Thus, by 7.15, $T$ is self-adjoint. 146 | 147 | _Exercise 7_ 148 | 149 | Let $e_1, \dots, e_n$ be an orthonormal basis of $V$ consisting of eigenvalues of $T$ and let $\lambda_1, \dots, \lambda_n$ denote their corresponding eigenvalues. 150 | We have 151 | 152 | $$ 153 | \lambda_j^9 e_j = T^9e_j = T^8e_j = \lambda_j^8 e_j, 154 | $$ 155 | 156 | for each $j = 1, \dots, n$. 157 | This implies that $\lambda_j = 0$ or $\lambda_j = 1$. 158 | Hence, every eigenvalue of $T$ is a real number and by the previous exercise $T$ is self-adjoint. 159 | Moreover, $\lambda_j^2 = \lambda_j$, regardless if $\lambda_j$ is $0$ or $1$. 160 | Therefore $T^2 = T$. 161 | 162 | _Exercise 8_ 163 | 164 | Let $V$ be a $3$-dimensional complex vector space and $v_1, v_2, v_3$ a basis for it. 165 | Define $T \in \mathcal{L}(V)$ by 166 | 167 | $$ 168 | \begin{aligned} 169 | Tv_1 &= v_2\\\\ 170 | Tv_2 &= v_3\\\\ 171 | Tv_3 &= 0. 172 | \end{aligned} 173 | $$ 174 | 175 | Obviously $T^9 = T^8 = 0$, however $T^2e_1 = e_3$ and $Te_1 = e_2$, therefore $T^2 \neq T$. 176 | 177 | _Exercise 9_ 178 | 179 | Suppose that $T$ is a normal operator on $V$. 180 | Let $e_1, \dots, e_n$ be an orthonormal basis of $V$ consisting of eigenvectors of $T$ and let $\lambda_1, \dots, \lambda_n$ denote their corresponding eigenvalues. 181 | Define $S$ by 182 | 183 | $$ 184 | Se_j = \sqrt{\lambda_j}e_j, 185 | $$ 186 | 187 | for each $j = 1, \dots, n$. 188 | Obviously, $S^2e_j = \lambda_j e_j = Te_j$. 189 | Therefore $S^2 = T$. 190 | 191 | _Exercise 10_ 192 | 193 | Define $T \in \mathcal{L}(\mathbb{R}^2)$ by 194 | 195 | $$ 196 | \begin{aligned} 197 | Te_1 &= e_2\\\\ 198 | Te_2 &= -e_1, 199 | \end{aligned} 200 | $$ 201 | 202 | where $e_1, e_2$ is the standard basis of $\mathbb{R}^2$. 203 | Then 204 | 205 | $$ 206 | (T^2 + I)e_1 = T^2e_1 + Ie_1 = Te_2 + e_1 = -e_1 + e_1 = 0. 207 | $$ 208 | 209 | Therefore $T^2 + bT + cI$ is not injective for $b = 0$ and $c = 1$. 210 | 211 | _Exercise 11_ 212 | 213 | This is about the same as Exercise 9. 214 | Just note that is one is valid when $\mathbb{F} = \mathbb{R}$, not only because it just applies to self-adjoint operators, but also because every real number has a cube root, although the same cannot be same about square roots. 215 | 216 | _Exercise 12_ 217 | 218 | Let $e_1, \dots, e_n$ be an orthonormal basis of $V$ consisting of eigenvectors of $T$ and let $\lambda_1, \dots, \lambda_n$ denote their corresponding eigenvalues. 219 | Choose an eigenvalue $\lambda'$ of $T$ such that $|\lambda' - \lambda|^2$ is minimized. 220 | There are $a_1, \dots, a_n \in \mathbb{F}$ such that 221 | 222 | $$ 223 | v = a_1 e_1 + \dots + a_n e_n. 224 | $$ 225 | 226 | Thus, we have 227 | 228 | $$ 229 | \begin{aligned} 230 | \epsilon^2 &> ||Tv - \lambda v||^2\\\\ 231 | &= |\langle Tv - \lambda v, e_1 \rangle|^2 + \dots + |\langle Tv - \lambda v, e_n \rangle|^2\\\\ 232 | &= |\lambda_1a_1 - \lambda a_1|^2 + \dots + |\lambda_n a_n - \lambda a_n|^2\\\\ 233 | &= |a_1|^2|\lambda_1 - \lambda|^2 + \dots + |a_n|^2|\lambda_n - \lambda|^2\\\\ 234 | &\ge |a_1|^2|\lambda' - \lambda|^2 + \dots + |a_n|^2|\lambda' - \lambda|^2\\\\ 235 | &= |\lambda' - \lambda|^2, 236 | \end{aligned} 237 | $$ 238 | 239 | where the second and fifth lines follow from 6.30 (the fifth because $||v|| = 1$). 240 | Taking the square root now yields the desired result. 241 | 242 | _Exercise 13_ 243 | 244 | We will just prove (a) implies (b), since that's the part which uses Schur's Theorem, and we will do so by induction on $\operatorname{dim} V$. 245 | 246 | Suppose (a) holds. Note that if $\operatorname{dim} V = 1$, then (a) trivially implies (b), just take any non-zero vector in $V$ and divide it by its norm. 247 | Now suppose that $\operatorname{dim} V > 1$ and that (a) implies (b) for all complex inner product spaces of smaller dimension. 248 | 249 | Choose an eigenvector $u$ of $T$ with $||u|| = 1$ (the existence of an eigenvalue is guaranteed by 5.21). 250 | Let $U = \operatorname{span}(u)$. 251 | Since $U$ is invariant under $T$, it follows by Exercise 3 in section 7A that $U^\perp$ is invariant under $T^*$. 252 | By 6.50 $\operatorname{dim} U^\perp = \operatorname{dim} V - 1$. 253 | Thus, by the induction hypothesis there exists an orthonormal basis of $U^\perp$ consisting of eigenvectors of $T^*|_{U^\perp}$, which are also eigenvectors of $T$, by 7.21. 254 | Moreover, adjoining $u$ to this basis produces an orthornomal basis of $V$ consisting of eigenvectors of $T$. 255 | Therefore (b) holds. 256 | 257 | _Exercise 14_ 258 | 259 | Suppose $U$ has a basis $e_1, \dots, e_n$ consisting of eigenvectors of $T$. 260 | We can assume without loss of generality that this list is normalized (we can divide each vector by its norm if it isn't). 261 | Define $\langle \cdot, \cdot \rangle: U \times U \to \mathbb{R}$ by 262 | 263 | $$ 264 | \langle a_1 e_1 + \dots + a_n e_n, b_1 e_1 + \dots + b_n e_n \rangle = a_1b_1 + \dots + a_nb_n. 265 | $$ 266 | 267 | Since every vector in $U$ can be uniquely written as linear combination of $e_1, \dots, e_n$, this function is well defined. 268 | Moreover, one easily checks that this function is an inner product. 269 | Note that 270 | 271 | $$ 272 | \langle e_j, e_k \rangle = \langle 0e_1 + \dots + 1e_j + \dots + 0e_n, 0e_1 + \dots + 1e_k + \dots + 0e_n \rangle = 273 | \begin{cases} 274 | 1, \text{ if } j = k\\\\ 275 | 0, \text{ if } j \neq k 276 | \end{cases}. 277 | $$ 278 | 279 | Therefore $e_1, \dots, e_n$ is also orthonormal. 280 | The Real Spectral Theorem (7.29) now implies that $T$ is self-adjoint. 281 | 282 | The converse follows directly from 7.29. 283 | 284 | _Exercise 15_ 285 | 286 | We have 287 | 288 | $$ 289 | \begin{pmatrix} 1 & 1 & 0\\\\ 0 & 1 & 1\\\\ 1 & 0 & x \end{pmatrix} \begin{pmatrix} 1 & 0 & 1\\\\ 1 & 1 & 0\\\\ 0 & 1 & x \end{pmatrix} = \begin{pmatrix} 1 & 0 & 1\\\\ 1 & 1 & 0\\\\ 0 & 1 & x \end{pmatrix} \begin{pmatrix} 1 & 1 & 0\\\\ 0 & 1 & 1\\\\ 1 & 0 & x \end{pmatrix}. 290 | $$ 291 | 292 | Multiplying the third row with the second column on both sides we get 293 | 294 | $$ 295 | 1\cdot0 + 0\cdot1 + x\cdot1 = 0\cdot1 + 1\cdot1 + x\cdot0, 296 | $$ 297 | 298 | therefore $x = 1$. 299 | -------------------------------------------------------------------------------- /Chapter 03 - Linear Maps/3C - Matrices.md: -------------------------------------------------------------------------------- 1 | Chapter 3: **Linear Maps** 2 | 3 | **3.C** 4 | 5 | - [x] Exercise 1 6 | - [x] Exercise 2 7 | - [x] Exercise 3 8 | - [x] Exercise 4 9 | - [x] Exercise 5 10 | - [x] Exercise 6 11 | - [x] Exercise 7 12 | - [x] Exercise 8 13 | - [x] Exercise 9 14 | - [x] Exercise 10 15 | - [x] Exercise 11 16 | - [x] Exercise 12 17 | - [x] Exercise 13 18 | - [x] Exercise 14 19 | - [ ] Exercise 15 20 | 21 | _Exercise 1_ 22 | 23 | Let $v_1, \dots, v_m$ denote a basis of $V$. 24 | The dimension of $\operatorname{range} T$ is the same as the dimension of $\operatorname{span}(Tv_1, \dots, Tv_m)$. 25 | So if the matrix of $T$ has less than $\operatorname{dim} \operatorname{range} T$ nonzero entries, it follows from the definition of $\mathcal{M}(T)$ that the list $Tv_1, \dots, Tv_m$ has less than $\operatorname{dim} \operatorname{range} T$ non-zero vectors, which is not possible, because its span is $\operatorname{range} T$. 26 | 27 | _Exercise 2_ 28 | 29 | Just differentiate any basis of $\mathcal{P}_3(\mathbb{R})$, take whatever is left as basis of $\mathcal{P}_2(\mathbb{\mathbb{R}})$ (except the $0$ vector) and adjust the order. 30 | For example, differentiating the standard basis $1, x, x^2, x^3$ we get $0, 1, 2x, 3x^2$. 31 | Therefore, the matrix of $D$ with respect to the basis $x^3, x^2, x, 1$ of $\mathcal{P}_3(\mathbb{R})$ and the basis $3x^2, 2x, 1$ of $\mathcal{P}_2(\mathbb{R})$ is 32 | 33 | $$ 34 | \begin{pmatrix} 35 | 1 & 0 & 0 & 0\\\\ 36 | 0 & 1 & 0 & 0\\\\ 37 | 0 & 0 & 1 & 0 38 | \end{pmatrix}. 39 | $$ 40 | 41 | _Exercise 3_ 42 | 43 | Use the same notation from 3.22. 44 | Extend $Tv_1, \dots, Tv_n$ to a basis $Tv_1, \dots, Tv_n, w_1, \dots, w_k$ of $W$. 45 | It is easy to see now that the matrix of $T$ with respect to this basis and the basis $v_1, \dots, v_n, u_1, \dots, u_m$ of $V$ satisfies the desired property. 46 | 47 | _Exercise 4_ 48 | 49 | If $Tv_1 = 0$, then all entries in the first column is zero for any basis of $W$. 50 | If $Tv_1 \neq 0$, let $w_1 = Tv_1$ and extend it a basis $w_1, \dots, w_n$ of $W$. 51 | Then 52 | 53 | $$ 54 | Tv_1 = 1w_1 + 0w_2 + \dots + 0w_n, 55 | $$ 56 | 57 | which shows that the entries in the first column are all $0$ except the first one, which is $1$. 58 | 59 | _Exercise 5_ 60 | 61 | I will use facts from section 3F, although this was not what Axler intended. 62 | 63 | Consider the dual basis $\psi_1, \dots, \psi_n$ of $w_1, \dots, w_n$ and the dual map $T'$ of $T$ (note that $T'$ is map from $W'$ to $V'$). 64 | By the previous exercise, there exists a basis of $\varphi_1, \dots, \varphi_m$ of $V'$ such that all entries in the first column of $\mathcal{M}(T')$ (with respect to the bases we have here of $W'$ and $V'$) are $0$ except for possibly a $1$ in the first row, first column. 65 | By Exercise 31 in section 3F, there exists a basis $v_1, \dots, v_m$ of $V$ such that its dual basis is $\varphi_1, \dots, \varphi_m$. 66 | Moreover, by 3.114, we have $\mathcal{M}(T)$ (with respect to the bases shown here of $V$ and $W$) is equal to $(\mathcal{M}(T'))^t$, which shows that $\mathcal{M}(T)$ satisfies the desired property. 67 | 68 | Alternative proof: 69 | 70 | Consider any basis $v_1,...,v_m$ of $V$, and a set of vectors $Tv_1,...,Tv_m$, and denote $Tv_i = \sum\limits_{j=1}^nA_{ji}w_j$. 71 | 72 | Suppose first that all $A_{1i} = 0$ - then we have found the desired basis. 73 | 74 | Suppose now that not all $A_{1i} = 0$. Without loss of generality, let us assume $A_{11} \neq 0$. Then, let us consider set of vectors 75 | 76 | $$ 77 | \begin{align} 78 | v_1' &= \frac{1}{A_{11}}v_1\\ 79 | v_2' &= v_2 - \frac{A_{12}}{A_{11}}v_1\\ 80 | v_3' &= v_3 - \frac{A_{13}}{A_{11}}v_1\\ 81 | \ldots\\ 82 | v_m' &= v_m - \frac{A_{1m}}{A_{11}}v_1 83 | \end{align} 84 | $$ 85 | 86 | It's obvious that $Tv_1' = 1w_1 + \sum\limits_{j=2}^nA_{j1}w_j$, and $Tv_i' = 0w_1 + \sum\limits_{j=2}^nA_{ji}w_j$, so we have the first line of corresponding matrix equal to $(1, 0, 0,..., 0)$. We now need to prove that $v_1',...,v_m'$ is a basis of $V$, which will complete the task. 87 | 88 | Let us consider $\sum\limits_{i=1}^mc_iv_i' = 0$. We can rewrite the left part of this equation 89 | 90 | $$ 91 | \begin{align} 92 | &\frac{c_1}{A_{11}}v_1 + \sum\limits_{i=2}^mc_iv_i - \sum\limits_{i=2}^m\frac{c_i}{A_{11}}v_1 \\ 93 | & = v_1(\frac{c_1}{A_{11}} - \sum\limits_{i=2}^m\frac{c_i}{A_{11}}) + \sum\limits_{i=2}^mc_iv_i 94 | \end{align} 95 | $$ 96 | 97 | Since $v_1,...,v_m$ are linearly independent, if this expression is 0, it must be that all the coefficients in front of $v_1...v_m$ are 0 as well, so $c_2=c_3=...=c_m = 0$, and $\frac{c_1}{A_{11}} - \sum\limits_{i=2}^m\frac{c_i}{A_{11}}$, which implies that $c_1 = 0$. So, this completes the proof that $v_1',...,v_m'$ are linearly independent and hence form a basis of $V$. 98 | 99 | _Exercise 6_ 100 | 101 | $\Rightarrow$: Since $\operatorname{dim}\operatorname{range}T = 1$, its basis consists of a single vector. Denote this vector $w \neq 0$. Next, we can extend this basis to a basis of $W$ with some vectors $w_2,...,w_m$. Now, let $w_1 = w - \sum\limits_{i=2}^mw_i$. It is clear, that $w_1,...,w_m$ are linearly independent: $\sum\limits_{i=1}^mc_iw_i = \sum\limits_{i=2}^mc_iw_i + c_1w - c_1\sum\limits_{i=2}^mw_i = c_1w + \sum\limits_{i=2}^m(c_i - c_1)w_i$. Since $w, w_2,...,w_m$ are linearly independent as a basis, this expression is 0 only if $c_1 = 0$ and $c_i - c_1 = 0$ for all $i=2...m$, which means that all $c_i=0$, so $w_1,...,w_m$ are also linearly independent and hence they form another basis of $W$. 102 | 103 | Let us select $v_1,...,v_{n-1}$ as a basis of $\operatorname{null}T$, and extend it to a basis of $V$ with $v_n$. Since $Tv_n \in \operatorname{range}T$, we have $Tv_n = \lambda w$, $\lambda \neq 0$. We now select a set of vectors 104 | 105 | $$ 106 | \begin{align} 107 | v_1' &= v_1 + \frac{1}{\lambda}v_n \\ 108 | v_2' &= v_2 + \frac{1}{\lambda}v_n \\ 109 | ... \\ 110 | v_{n-1}' &= v_{n-1} + \frac{1}{\lambda}v_n \\ 111 | v_n' &= \frac{1}{\lambda}v_n 112 | \end{align} 113 | $$ 114 | 115 | It is clear that $\forall i Tv_i' = \frac{1}{\lambda}Tv_n = w = w_1 + ... +w_m$. It is also easy to show, that $v_1',...,v_n'$ form a basis of V. 116 | 117 | $\Leftarrow$: suppose there are bases $v_1,...,v_n$ and $w_1,...,w_m$ of $V$ and $W$ such, that $Tv_i = w_1 + ... + w_m$. We know that $\operatorname{range}T = \operatorname{span}(Tv_1, Tv_2, ..., Tv_n)$, but since all $Tv_i$ are equal, $\operatorname{span}(Tv_1, Tv_2, ..., Tv_n) = \operatorname{span}(Tv_1)$, and so $\operatorname{dim}\operatorname{range}T = 1$. 118 | 119 | 120 | _Exercise 7_ 121 | 122 | Use the same choice of basis for $\mathcal{M}(S)$, $\mathcal{M}(T)$ and $\mathcal{M}(S + T)$ and same notation as in 3.32. 123 | 124 | Let $A = \mathcal{M}(S)$, $B = \mathcal{M}(T)$ and $C = \mathcal{M}(S + T)$. 125 | Then 126 | 127 | $$ 128 | \begin{aligned} 129 | \sum\limits_{j=1}^m C_{j,k} w_j &= (S + T)v_k\\\\ 130 | &= Sv_k + Tv_k\\\\ 131 | &= \sum\limits_{j=1}^m A_{j,k} w_j + \sum\limits_{j=1}^m B_{j, k} w_j\\\\ 132 | &= \sum\limits_{j=1}^m (A_{j,k} + B_{j, k}) w_j. 133 | \end{aligned} 134 | $$ 135 | 136 | Since $w_1, \dots, w_m$ is a basis, by 2.29 their coefficients are unique for each vector they determine, therefore it follows that $C_{j, k} = A_{j, k} + B_{j, k}$, as desired. 137 | 138 | _Exercise 8_ 139 | 140 | This is almost the same as _Exercise 7_. 141 | 142 | _Exercise 9_ 143 | 144 | Clearly, $Ac$ is a $m$-by-$1$ matrix. 145 | Then 146 | 147 | $$ 148 | \begin{aligned} 149 | (Ac)_{j, 1} &= \sum\limits_{r=1}^n A_{j, r} c_{r,1}\\\\ 150 | &= \sum\limits_{r=1}^n A_{j, r} c_r\\\\ 151 | &= c_1 A_{j, 1} + \dots + c_n A_{j, n}\\\\ 152 | &= c_1 (A_{.,1})_{j, 1} + \dots + c_n (A_{.,n})_{j, 1}\\\\ 153 | &= (c_1 A_{.,1})_{j, 1} + \dots + (c_n A_{.,n})_{j, 1}\\\\ 154 | &= (c_1 A_{.,1} + \dots + c_n A_{.,n})_{j, 1}. 155 | \end{aligned} 156 | $$ 157 | 158 | _Exercise 10_ 159 | 160 | Use the same notation as in the motivation prior to 3.41. 161 | 162 | Let $Z$ be a $1$-dimensional vector space and $z$ a basis for it. 163 | Define $S_j: V \to Z$ by 164 | 165 | $$ 166 | S_j v_r = A_{j, r} z 167 | $$ 168 | 169 | for $1 \le r \le n$. 170 | Obviously, $\mathcal{M}(S_j) = A_{j,.}$. 171 | Therefore, $A_{j,.}C = \mathcal{M}(S_j) \mathcal{M}(T) = \mathcal{M}(S_j T)$. 172 | Thus 173 | 174 | $$ 175 | \begin{aligned} 176 | S_j Tu_k &= S_j (\sum\limits_{r=1}^n C_{r, k} v_r)\\\\ 177 | &= \sum\limits_{r=1}^n C_{r, k} S_j v_r\\\\ 178 | &= \sum\limits_{r=1}^n C_{r, k} A_{j, r} z\\\\ 179 | &= (AC)_{j, k} z 180 | \end{aligned} 181 | $$ 182 | 183 | for $1 \le k \le p$. 184 | Hence, $(AC)_{j,.} = A_{j,.}C$. 185 | 186 | _Exercise 11_ 187 | 188 | We have 189 | 190 | $$ 191 | \begin{aligned} 192 | (aC)_{1, k} &= \sum\limits_{r=1}^n a_{1, r} C_{r, k}\\\\ 193 | &= \sum\limits_{r=1}^n a_r C_{r, k}\\\\ 194 | &= a_1 C_{1, k} + \dots + a_n C_{n, k}\\\\ 195 | &= a_1 (C_{1,.})_{1, k} + \dots + a_n (C_{n,.})_{1, k}\\\\ 196 | &= (a_1 C_{1,.} + \dots + a_n C_{n,.})_{1, k}\\\\ 197 | \end{aligned} 198 | $$ 199 | 200 | _Exercise 12_ 201 | 202 | We have 203 | 204 | $$ 205 | \begin{pmatrix} 1 & -1\\\\ 1 & -1 \end{pmatrix} \begin{pmatrix} 1 & 1\\\\ -1 & -1 \end{pmatrix} = 206 | \begin{pmatrix} 2 & 2\\\\ 2 & 2 \end{pmatrix}, 207 | $$ 208 | 209 | but 210 | 211 | $$ 212 | \begin{pmatrix} 1 & 1\\\\ -1 & -1 \end{pmatrix} \begin{pmatrix} 1 & -1\\\\ 1 & -1 \end{pmatrix} = 213 | \begin{pmatrix} 2 & -2\\\\ -2 & 2 \end{pmatrix}. 214 | $$ 215 | 216 | _Exercise 13_ 217 | 218 | Since $A(B+C)$ and $(D+E)F$ make sense, $B$ has the same size, as C (denote it $m,n$), and $D$ has the same size as $E$ (denote it also $m,n$). Also, $A$ has the same number of columns, as $B+C$ (as well as $B$ and $C$) has rows, so $A$ has a shape $p,m$ and $F$ has the same number of rows as $D+E$ has columns (which is the same as number of columns in $D$ and $E$), so we can denote the shape of $F$ as $n,p$. So, $AB$ and $AC$ are well-defined and have the same shape $p,n$, so their sum is also well-defined. The same goes for $DF$, $EF$ and their sum, which all have shape $m,p$. 219 | 220 | Consider arbitrary $i,j$: $1 \le i \le p$, $1 \le j \le n$ $A(B+C)_{ij} = \sum\limits_{r=1}^mA_{ir}(B+C)_{rj} = \sum\limits_{r=1}^mA_{ir}(B_{rj}+C_{rj}) = \sum\limits_{r=1}^mA_{ir}B_{rj} + \sum\limits_{r=1}^mA_{ir}C_{rj} = (AB)_{ij} + (AC)_{ij}$. So, all elements of $AB + AC$ and $A(B+C)$ are equal, which means that these matrices are equal. 221 | 222 | The proof for $(D+E)F$ is similar. 223 | 224 | _Exercise 14_ 225 | 226 | If $(AB)C$ makes sense, then if $A$ has shape $m,n$, and $C$ has a shape $p,q$, $C$ should have shape $m,p$, and it means that $A(BC)$ also is well-defined. 227 | 228 | $$ 229 | \begin{align} 230 | ((AB)C)_{ij} &= \sum\limits_{k}(AB)_{ik}C_{kj}\\ 231 | &= \sum\limits_{k}\sum\limits_{r}A_{ir}B_{rk}C_{kj}\\ 232 | &= \sum\limits_{r}\sum\limits_{k}A_{ir}B_{rk}C_{kj}\\ 233 | &= \sum\limits_{r}A_{ir}\sum\limits_{k}B_{rk}C_{kj}\\ 234 | &= \sum\limits_{r}A_{ir}(BC)_{rj} \\ 235 | &= (A(BC))_{ij} 236 | \end{align} 237 | $$ 238 | -------------------------------------------------------------------------------- /Chapter 05 - Eigenvalues, Eigenvectors, and Invariant Subspaces/5B - Eigenvectors and Upper-Triangular Matrices.md: -------------------------------------------------------------------------------- 1 | Chapter 5: **Eigenvalues, Eigenvectors and Invariant Subspaces** 2 | 3 | **5.B** 4 | 5 | - [x] Exercise 1 6 | - [x] Exercise 2 7 | - [x] Exercise 3 8 | - [x] Exercise 4 9 | - [x] Exercise 5 10 | - [x] Exercise 6 11 | - [x] Exercise 7 12 | - [x] Exercise 8 13 | - [x] Exercise 9 14 | - [x] Exercise 10 15 | - [x] Exercise 11 16 | - [x] Exercise 12 17 | - [x] Exercise 13 18 | - [x] Exercise 14 19 | - [x] Exercise 15 20 | - [x] Exercise 16 21 | - [x] Exercise 17 22 | - [x] Exercise 18 23 | - [x] Exercise 19 24 | - [x] Exercise 20 25 | 26 | _Exercise 1_ 27 | 28 | _(a)_ 29 | We will prove the contrapositive. 30 | 31 | Suppose $I - T$ is not invertible. 32 | There exists non-zero $v \in V$ such that $Iv - Tv = 0$, which implies $Tv = v$. 33 | Applying $T$ to both sides of this last equation, yields $TTv = Tv = v$. 34 | Continuing this, we see that for any positive integer $n$, $T^n v = v$. 35 | Therefore $T^n$ can never be $0$. 36 | 37 | Now suppose $T^n = 0$. 38 | We have 39 | 40 | $$ 41 | \begin{aligned} 42 | (I - T)(I + T + \dots + T^{n-1}) &= (I - T) \sum\limits_{k=0}^{n-1} T^k\\\\ 43 | &= \sum\limits_{k=0}^{n-1} T^k - \sum\limits_{k=0}^{n-1} T^{k + 1}\\\\ 44 | &= I + \sum\limits_{k=1}^{n-1} T^k - \sum\limits_{k=1}^n T^k\\\\ 45 | &= I + \sum\limits_{k=1}^{n-1} T^k - T^n - \sum\limits_{k=1}^{n-1} T^k\\\\ 46 | &= I - T^n\\\\ 47 | &= I\\\\ 48 | \end{aligned} 49 | $$ 50 | 51 | Therefore $I - T$ is an inverse of $I + T + \dots + T^{n-1}$, which implies the desired result. 52 | 53 | _(b)_ 54 | I wouldn't. 55 | 56 | We know, that $(1 - x^n) = (1 - x)(1 + x + ... + x^{n-1})$. If we consider this as polynomial over $\mathcal{L}(V)$, and substitute $T$, we get $I = (I - T)(I + T + ... + T^{n-1})$. 57 | 58 | _Exercise 2_ 59 | 60 | Suppose $\lambda$ is an eigenvalue of $T$ and $v$ a corresponding eigenvector. 61 | Then 62 | 63 | $$ 64 | \begin{aligned} 65 | 0 &= (T - 2I)(T - 3I)(T - 4I)v\\\\ 66 | &= (T^3 - 9T^2 + 26T - 24I)v\\\\ 67 | &= T^3 v - 9T^2 v + 26T - 24v\\\\ 68 | &= \lambda^3 v - 9\lambda^2 v + 26\lambda v - 24v\\\\ 69 | &= (\lambda^3 - 9\lambda^2 + 26\lambda - 24)v\\\\ 70 | \end{aligned} 71 | $$ 72 | 73 | Therefore, because $v \neq 0$, $\lambda^3 - 9\lambda^2 + 26\lambda + 24 = 0$. 74 | But we can factor this to $(\lambda - 2)(\lambda - 3)(\lambda - 4) = 0$. 75 | Thus $\lambda$ is either $2$, $3$ or $4$. 76 | 77 | _Exercise 3_ 78 | 79 | Let $v \in V$. 80 | Then 81 | 82 | $$ 83 | 0 = (T^2 - I^2)v = (T + I)(T - I)v 84 | $$ 85 | 86 | which implies $(T - I)v \in \operatorname{null} (T + I)$. 87 | We but need to prove that $T + I$ is injective and we will have $Tv = v$. 88 | Let $u, w \in V$ such that $(T + I)u = (T + I)w$. 89 | Then $Tu + u = Tw + w$ which implies $T(u - w) = w - u = -1(u - w)$. 90 | But $-1$ is not an eigenvalue of $T$, thus $u - w = 0$, implying $u = w$, as desired. 91 | 92 | _Exercise 4_ 93 | 94 | Suppose $v \in V$. 95 | We have $v = (I - P)v + Pv$. 96 | Because $P - P^2 = 0$, it follows that $P(I - P)v = 0$. 97 | Therefore $(I - P)v \in \operatorname{null} P$. 98 | Obviously $Pv \in \operatorname{range} P$, thus $v \in \operatorname{null} P + \operatorname{range} P$ and we have $V \subset \operatorname{null} P + \operatorname{range} P$. 99 | The inclusion in the opposite direction is clearly true, therefore $V = \operatorname{null} P + \operatorname{range} P$. 100 | 101 | To see why this is a direct sum, suppose $u \in \operatorname{null} P \cap \operatorname{range} P$. 102 | Because $u \in \operatorname{range} P$, there exists $w$ such that $Pw = u$. 103 | But $u \in \operatorname{null} P$, so we must have 104 | 105 | $$ 106 | 0 = Pu = P^2 w = Pw = u 107 | $$ 108 | 109 | Therefore $\operatorname{null} P \cap \operatorname{range} P = \{0\}$ and by 1.45 it is direct sum. 110 | 111 | _Exercise 5_ 112 | 113 | Note that, for non-negative $k$, $(STS^{-1})^k = ST^kS^{-1}$, because the $S$'s and $S^{-1}$'s cancel out (this can easily me proven by induction on $k$). 114 | Let $n = \deg p$ and $a_0, a_1, \dots, a_n \in \mathbb{F}$ be the coefficients of $p$. 115 | Then 116 | 117 | $$ 118 | \begin{aligned} 119 | p(STS^{-1}) &= \sum\limits_{k=0}^n a_k(STS^{-1})^k\\\\ 120 | &= \sum\limits_{k=0}^n a_k S T^k S^{-1}\\\\ 121 | &= S(\sum\limits_{k=0}^n a_k T^k)S^{-1}\\\\ 122 | &= Sp(T)S^{-1}\\\\ 123 | \end{aligned} 124 | $$ 125 | 126 | _Exercise 6_ 127 | 128 | It is easy to see that $T^k u \in U$ for $u \in U$ and non-negative $k$ (this can easily be proven by induction on $k$). 129 | Since $p(T)u$ is a sum of terms like this, multiplied by the coefficients of $p$, and $U$ is closed under addition and scalar multiplication, then $p(T)u \in U$. 130 | 131 | _Exercise 7_ 132 | 133 | Suppose $9$ is an eigenvalue of $T^2$. 134 | Let $v$ be a corresponding eigenvector. 135 | Then $(T^2 - 9I)v = 0$, which implies $(T - 3I)(T + 3I)v = 0$. 136 | If $(T + 3I)v = 0$ then $-3$ is an eigenvalue of $T$. 137 | Otherwise, $(T + 3I)v$ is an eigenvector of $T$ and $3$ is a corresponding eigenvalue. 138 | 139 | Conversely, suppose $\pm 3$ is an eigenvalue of $T$. 140 | Let $v$ be a corresponding eigenvector. 141 | Then $T^2 v = T(\pm 3v) = (\pm 3)^2 v = 9v$, showing that $9$ is an eigenvalue of $T^2$. 142 | 143 | _Exercise 8_ 144 | 145 | Define $T \in \mathcal{L}(\mathbb{R}^2)$ by 146 | 147 | $$ 148 | T(x, y) = \frac{1}{\sqrt{2}}(x - y, x + y) 149 | $$ 150 | 151 | Then, for $(a, b) \in \mathbb{R}^2$, we have 152 | 153 | $$ 154 | \begin{aligned} 155 | T^4 (a, b) &= \frac{1}{\sqrt{2}} T^3 (a - b, a + b)\\\\ 156 | &= \frac{1}{2} T^2 (-2b, 2a)\\\\ 157 | &= \frac{1}{2\sqrt{2}} T (-2a - 2b, 2a - 2b)\\\\ 158 | &= \frac{1}{4}(-4a, -4b)\\\\ 159 | &= -(a, b)\\\\ 160 | \end{aligned} 161 | $$ 162 | 163 | _Exercise 9_ 164 | 165 | Let $c(z - \lambda_n) \dotsm (z - \lambda_1)$ be a factorization of $p$. 166 | Then $c(T - \lambda_n I) \dotsm (T - \lambda_1 I)$ is a factorization of $p(T)$. 167 | Since $p$ is the polynomial of smallest degree such that $p(T)v = 0$, it follows that $(T - \lambda_j I) \dotsm (T - \lambda_1 I)v \neq 0$, for $j < n$. 168 | Therefore, we have that $(T - \lambda_{n - 1} I) \dotsm (T - \lambda_1 I)v \neq 0$ is an eigenvector of $T$ and $\lambda_n$ the corresponding eigenvalue. 169 | Note that, by 5.20, the order of factorization can be changed, placing any other factor $(T - \lambda_j)$ in the beginning. This implies that all $\lambda$'s are indeed eigenvalues of $T$. 170 | 171 | _Exercise 10_ 172 | 173 | Note that $T^k v = \lambda^k v$. 174 | Let $n = \deg p$ and $a_0, a_1, \dots, a_n \in \mathbb{F}$ be the coefficients of $p$. 175 | Then 176 | 177 | $$ 178 | \begin{aligned} 179 | p(T)v &= (\sum\limits_{k=0}^n a_k T^k)v\\\\ 180 | &= \sum\limits_{k=0}^n a_k T^k v\\\\ 181 | &= \sum\limits_{k=0}^n a_k \lambda^k v\\\\ 182 | &= (\sum\limits_{k=0}^n a_k \lambda^k) v\\\\ 183 | &= p(\lambda) v\\\\ 184 | \end{aligned} 185 | $$ 186 | 187 | _Exercise 11_ 188 | 189 | Suppose $\alpha$ is an eigenvalue of $p(T)$. 190 | Let $c(z - \lambda_1) \dotsm (z - \lambda_n)$ be a factorization of $p(z) - \alpha$. 191 | We have 192 | 193 | $$ 194 | p(T) - \alpha I = c(T - \lambda_1 I) \dotsm (T - \lambda_n I) 195 | $$ 196 | 197 | Because $p(T) - \alpha I$ is not injective, it follows that, for some $j$, $T - \lambda_j I$ is not injective. 198 | Therefore $\lambda_j$ is an eigenvalue of $T$. 199 | Since $\lambda_j$ is a root of $p(z) - \alpha$, we have that $p(\lambda_j) = \alpha$. 200 | 201 | The converse is the same as _Exercise 10_. 202 | 203 | _Exercise 12_ 204 | 205 | Define $T \in \mathcal{L}(\mathbb{R}^4)$ by 206 | 207 | $$ 208 | T(x_1, x_2, x_3, x_4) = (x_2, x_3, x_4, -x_1) 209 | $$ 210 | 211 | Let $p \in \mathcal{P}(R)$ such that $p(x) = x^4$. 212 | Then $-1$ is an eigenvalue of $p(T)$, but $p$ is always positive, therefore no eigenvalue $\lambda$ of $T$ satisfies $p(\lambda) = -1$. 213 | 214 | _Exercise 13_ 215 | 216 | By 5.21, $W$ is either $\{0\}$ or infinite-dimensional. 217 | Let $U$ be a subspace of $W$ invariant under $T$. 218 | Then $T\rvert_U$ also has no eigenvalues. 219 | But $T\rvert_U$ is also an operator on a complex vector space, therefore $U$ is either $\{0\}$ or infinite-dimensional. 220 | 221 | _Exercise 14_ 222 | 223 | Suppose $V$ is finite-dimensional vector space. 224 | Let $v_1, \dots, v_n$ be a basis of $V$. 225 | Define $T \in \mathcal{L}(V)$ by 226 | 227 | $$ 228 | \begin{aligned} 229 | Tv_j &= v_{j+1}, \text{ for } j = 1, \dots, n - 1\\\\ 230 | Tv_n &= v_1\\\\ 231 | \end{aligned} 232 | $$ 233 | 234 | $T$ is clearly invertible, but $\mathcal{M}(T)$ with respect to same basis only has zeros in the diagonal. 235 | 236 | _Exercise 15_ 237 | 238 | Suppose $V$ is finite-dimensional vector space. 239 | Let $v_1, \dots, v_n$ be a basis of $V$. 240 | Define $T \in \mathcal{L}(V)$ by 241 | 242 | $$ 243 | Tv_j = v_1 + \dots + v_n 244 | $$ 245 | 246 | $\mathcal{M}(T)$ contains $1$'s in all its entries, but $T$ is clearly not inveritible. 247 | 248 | _Exercise 16_ 249 | 250 | Let $n = \operatorname{dim} V$. 251 | Define $\Psi \in \mathcal{L}(\mathcal{P}_n(\mathbb{C}), V)$ by 252 | 253 | $$ 254 | \Psi(p) = (p(T))v 255 | $$ 256 | 257 | for $p \in \mathcal{P}_n(\mathbb{C})$. 258 | One can easily verify that $\Psi$ is linear. 259 | Since $\operatorname{dim} \mathcal{P}_n(\mathbb{C}) > \operatorname{dim} V$, by 3.23, there exists $p \in \mathcal{P}_n(\mathbb{C})$ such that $0 = \Psi(p) = (p(T))v$. 260 | The rest follows exactly as 5.21. 261 | 262 | _Exercise 17_ 263 | 264 | This is almost the same as _Exercise 16_. 265 | 266 | _Exercise 18_ 267 | 268 | Note that $f$ can only output integer values. 269 | Thus, if $f$ is not constant, there will be a jump discontinuity at some point. 270 | We will prove $f$ is not constant. 271 | 272 | If $T$ is invertible, then the existence of an eigenvalue of $T$ (guaranteed by 5.21) implies that $T - \lambda I$ is not surjective for some $\lambda \in \mathbb{F}$. 273 | Hence $f(0) = \operatorname{dim} \operatorname{range} T > \operatorname{dim} \operatorname{range} (T - \lambda I) = f(\lambda)$. 274 | 275 | If $T$ is not invertible, choose $\lambda$ such that it is not an eigenvalue of $T$. 276 | Then, for any non-zero $v \in V$, $(T - \lambda I)v \neq 0$, showing that $T - \lambda I$ is injective and, therefore, surjective. 277 | Hence $f(0) = \operatorname{dim} \operatorname{range} T < \operatorname{dim} \operatorname{range} (T - \lambda I) = f(\lambda)$. 278 | 279 | _Exercise 19_ 280 | 281 | 5.20 implies that any two operators in $\{p(T): p \in \mathcal{P}(\mathbb{F})\}$ commute. 282 | But this is obviously not true for $\mathcal{L}(V)$, because $\operatorname{dim} V > 1$. 283 | For example, let $v_1, \dots, v_n$ be a basis of $V$. 284 | Define $S, R \in L(V)$ by 285 | 286 | $$ 287 | \begin{aligned} 288 | Sv_1 &= v_2, Sv_j = 0 \text{ for } j = 2, \dots, n\\\\ 289 | Rv_1 &= 0, Rv_j = v_j \text{ for } j = 2, \dots, n. 290 | \end{aligned} 291 | $$ 292 | 293 | Then $SRv_1 = 0$ but $RSv_1 = v_2$. 294 | Thus $SR \neq RS$. 295 | 296 | _Exercise 20_ 297 | 298 | This follows directly from 5.27 and 5.26. 299 | -------------------------------------------------------------------------------- /Chapter 07 - Operators on Inner Product Spaces/7C - Positive Operators and Isometries.md: -------------------------------------------------------------------------------- 1 | Chapter 7: **Positive Operators and Isometries** 2 | 3 | **7.C** 4 | 5 | - [x] Exercise 1 6 | - [x] Exercise 2 7 | - [x] Exercise 3 8 | - [x] Exercise 4 9 | - [x] Exercise 5 10 | - [x] Exercise 6 11 | - [x] Exercise 7 12 | - [x] Exercise 8 13 | - [x] Exercise 9 14 | - [x] Exercise 10 15 | - [x] Exercise 11 16 | - [x] Exercise 12 17 | - [x] Exercise 13 18 | - [x] Exercise 14 19 | 20 | _Exercise 1_ 21 | 22 | We give a counterexample. 23 | Define $T \in \mathcal{L}(\mathcal{R}^2)$ by 24 | 25 | $$ 26 | \begin{aligned} 27 | Te_1 = e_1\\\\ 28 | Te_2 = -e_2 29 | \end{aligned} 30 | $$ 31 | 32 | where $e_1, e_2$ is the standard basis of $\mathbb{R}^2$. 33 | The matrix of $T$ with respect to this same basis is 34 | 35 | $$ 36 | \begin{pmatrix}1 & 0\\\\0 & -1\end{pmatrix}, 37 | $$ 38 | 39 | which equals its transpose, therefore $T$ is self-adjoint. 40 | Moreover, the basis $\frac{1}{\sqrt{2}}(e_1 + e_2), \frac{1}{\sqrt{2}}(e_1 - e_2)$ is orthonormal and 41 | 42 | $$ 43 | \begin{aligned} 44 | \left\langle T\left(\frac{1}{\sqrt{2}}(e_1 + e_2)\right), \frac{1}{\sqrt{2}}\left(e_1 + e_2\right) \right\rangle &= 0\\\\ 45 | \left\langle T\left(\frac{1}{\sqrt{2}}(e_1 - e_2)\right), \frac{1}{\sqrt{2}}\left(e_1 - e_2\right) \right\rangle &= 0, 46 | \end{aligned} 47 | $$ 48 | 49 | but $T$ is not positive because 50 | 51 | $$ 52 | \langle Te_2, e_2 \rangle = \langle -e_2, e_2 \rangle = -1. 53 | $$ 54 | 55 | _Exercise 2_ 56 | 57 | If $v = 0$, then $w = 0$ as well, so $v = w$. Suppose now, that $v \neq w$. 58 | 59 | $T$ is the positive square root of $T^2$. 60 | Note that 61 | 62 | $$ 63 | T^2v = TTv = Tw = v 64 | $$ 65 | 66 | hence $v$ is an eigenvector of $T^2$. 67 | From the proof of 7.36, we see that $Tv = \sqrt{1}v$. 68 | But $Tv = w$, therefore $v = w$. 69 | 70 | _Exercise 3_ 71 | 72 | For all $u \in U$, we have 73 | 74 | $$ 75 | \langle T|_U u, u \rangle = \langle Tu, u \rangle = \langle u, Tu \rangle = \langle u, T|_U u \rangle. 76 | $$ 77 | 78 | Thus, $T|_U$ is self-adjoint. 79 | Furthermore, 80 | 81 | $$ 82 | \langle T|_U u, u \rangle = \langle Tu, u \rangle \ge 0, 83 | $$ 84 | 85 | which shows that $T|_U$ is positive. 86 | 87 | _Exercise 4_ 88 | 89 | We have $(T^*T)^* = T^*(T^*)^* = T^*T$, which implies that $T^*T$ is self-adjoint and, for all $v \in V$, 90 | 91 | $$ 92 | \langle T^*Tv, v \rangle = \langle Tv, Tv \rangle \ge 0. 93 | $$ 94 | 95 | Thereofer $T^*T$ is positive. 96 | 97 | It's basically the same thing for $TT^*$. 98 | 99 | _Exercise 5_ 100 | 101 | Suppose $S, T \in \mathcal{L}(V)$ are positive operators. 102 | Then 103 | 104 | $$ 105 | (S + T)^* = S^* + T^* = S + T. 106 | $$ 107 | 108 | Hence $S + T$ is self-adjoint. 109 | Moreover, for all $v \in V$, we have 110 | 111 | $$ 112 | \langle (S + T)v, v \rangle = \langle Sv, v \rangle + \langle Tv, v \rangle \ge 0, 113 | $$ 114 | 115 | showing that $S + T$ is positive. 116 | 117 | _Exercise 6_ 118 | 119 | From 7.6 (e), we have 120 | 121 | $$ 122 | (T^k)^* = (T^*)^k = T^k. 123 | $$ 124 | 125 | Therefore $T^k$ is self-adjoint. 126 | 127 | To prove $T^k$ is positive, there are two cases. 128 | 129 | * If $k$ is odd, we have $k = 2n + 1$ for some nonnegative integer $n$. Then, for all $v \in V$, 130 | 131 | $$ 132 | \langle T^k v, v \rangle = \langle T^nTT^nv, v \rangle = \langle TT^nv, T^nv \rangle \ge 0 133 | $$ 134 | 135 | where the the inequality follows because $T$ is positive. 136 | 137 | * If $k$ is even, we have $k = 2n$ for some nonnegative integer $n$. Then, for all $v \in V$, 138 | 139 | $$ 140 | \langle T^k v, v \rangle = \langle T^nT^nv, v \rangle = \langle T^nv, T^nv \rangle \ge 0. 141 | $$ 142 | 143 | Therefore $T^k$ is positive. 144 | 145 | _Exercise 7_ 146 | 147 | Suppose $T$ is invertible. 148 | Let $R$ be the positive square root of $T$. 149 | Then 150 | 151 | $$ 152 | \langle Tv, v \rangle = \langle R^*Rv, v \rangle = \langle Rv, Rv \rangle > 0, 153 | $$ 154 | 155 | where the inequality follows because $R$ and $T$ have the same eigenvalues (as we saw in the proof of 7.36), but $0$ is not an eigenvalue of $T$, thus $Rv \neq 0$ for all $v$. 156 | 157 | Conversely, suppose $\langle Tv, v \rangle > 0$ for every $v \in V$ with $v \neq 0$. 158 | This directly implies that $\operatorname{null} T = \{0\}$. 159 | Therefore, $T$ is invertible. 160 | 161 | _Exercise 8_ 162 | 163 | Suppose $\langle \cdot, \cdot \rangle _T$ is an inner product on $V$. 164 | Let $v \in \operatorname{null} T$. 165 | Then 166 | 167 | $$ 168 | \langle v, v \rangle_T = \langle Tv, v \rangle = \langle 0, v \rangle = 0. 169 | $$ 170 | 171 | Therefore, by the definiteness property of inner products, $v = 0$. 172 | Thus $\operatorname{null} T = \{0\}$ and so $T$ is invertible. 173 | Suppose now that $v$ is an eigenvector of $T$ with eigenvalue $\lambda$. 174 | Then 175 | 176 | $$ 177 | 0 \le \langle v, v \rangle_T = \langle Tv, v \rangle = \lambda \langle v, v \rangle. 178 | $$ 179 | 180 | Thus $\lambda \ge 0$. 181 | Hence all eigenvalues of $T$ are nonnegative. 182 | We but need to show that $T$ is self-adjoint and then $T$ will be positive by 7.35. 183 | We have 184 | 185 | $$ 186 | \begin{aligned} 187 | \langle u, T^*v \rangle &= \langle Tu, v \rangle\\\\ 188 | &= \langle u, v \rangle_T\\\\ 189 | &= \overline{\langle v, u \rangle_T}\\\\ 190 | &= \overline{\langle Tv, u \rangle}\\\\ 191 | &= \langle u, Tv \rangle. 192 | \end{aligned} 193 | $$ 194 | 195 | Hence $T$ is self-adjoint. 196 | 197 | Conversely, suppose $T$ is an invertible positive operator. 198 | The positive-definiteness property of $\langle \cdot, \cdot \rangle_T$ follows from the forward direction of the previous exercises. 199 | For additivity in the first slot, we have 200 | 201 | $$ 202 | \begin{aligned} 203 | \langle u + v, w \rangle_T &= \langle T(u+v), w \rangle\\\\ 204 | &= \langle Tu, w \rangle + \langle Tv, w \rangle\\\\ 205 | &= \langle u, w \rangle_T + \langle v, w \rangle_T. 206 | \end{aligned} 207 | $$ 208 | 209 | Similarly, $\langle \cdot, \cdot \rangle_T$ satisfies homogeneity in the first slot. 210 | For conjugate symmetry, we have 211 | 212 | $$ 213 | \begin{aligned} 214 | \langle u, v \rangle_T &= \langle Tu, v \rangle\\\\ 215 | &= \langle u, Tv \rangle\\\\ 216 | &= \overline{\langle Tv, u \rangle}\\\\ 217 | &= \overline{\langle v, u \rangle_T}, 218 | \end{aligned} 219 | $$ 220 | 221 | where the second line follows because $T$ is self-adjoint. 222 | Therefore $\langle \cdot, \cdot \rangle_T$ is indeed an inner product on $V$. 223 | 224 | _Exercise 9_ 225 | 226 | It is true. 227 | We shall consider $\mathbb{F} = \mathbb{R}$ for the sake of simplicity and it will be easy to see it generalizes to $\mathbb{C}$ as well. 228 | 229 | Note that if $R$ is a self-adjoint square root of $I$ if and only if the matrix of $R$ is of the form 230 | 231 | $$ 232 | \begin{pmatrix}a & b\\\\b & c\end{pmatrix} 233 | $$ 234 | 235 | and 236 | 237 | $$ 238 | \begin{aligned} 239 | e_1 &= R^2e_1\\\\ 240 | &= R(ae_1 + be_2)\\\\ 241 | &= a(ae_1 + be_2) + b(be_1 + ce_2)\\\\ 242 | &= (a^2 + b^2)e_1 + b(a + c)e_2 243 | \end{aligned} 244 | $$ 245 | 246 | and 247 | 248 | $$ 249 | \begin{aligned} 250 | e_2 &= R^2e_2\\\\ 251 | &= R(be_1 + ce_2)\\\\ 252 | &= b(ae_1 + be_2) + c(be_1 + ce_2)\\\\ 253 | &= b(a+c)e_1 + (b^2 + c^2)e_2. 254 | \end{aligned} 255 | $$ 256 | 257 | Therefore, the problems falls down to finding $a, b, c \in \mathbb{F}$ such that 258 | 259 | $$ 260 | \begin{aligned} 261 | a^2 + b^2 &= 1\\\\ 262 | b^2 + c^2 &= 1\\\\ 263 | b(a + c) &= 0. 264 | \end{aligned} 265 | $$ 266 | 267 | As turns out, there are infinitely many solutions to this (just choose an $a \in [0, 1]$, take $c = -a$ and solve for $b$). 268 | Each of this solutions define a different self-adjoint square root, hence the identity operator has infinitely-many self-adjoint square roots. 269 | 270 | _Exercise 10_ 271 | 272 | Suppose (a) holds, so $S$ is an isometry. 273 | Then 7.42 ("(a) $\Rightarrow$ (f)") implies that $SS^* = I$. 274 | Therefore, for all $u, v \in V$ we have 275 | 276 | $$ 277 | \langle u, v \rangle = \langle SS^*u, v \rangle = \langle S^*u, S^*v \rangle. 278 | $$ 279 | 280 | Thus (b) holds. 281 | Clearly (b) implies (c) and (c) implies (d). 282 | Now suppose (d) holds. 283 | Then 7.42 ("(d) $\Rightarrow$ (g)" with $S$ replaced with $S^*$) implies that $S$ is an isometry. 284 | Thus (a) holds. 285 | 286 | _Exercise 11_ 287 | 288 | Let $e_1, e_2, e_3$ and $f_1, f_2, f_3$ be orthonormal bases of $\mathbb{F}^3$ consisting of eigenvectors of $T_1$ and $T_2$, respectively, corresponding to the eigenvalues $2, 5, 7$. 289 | Define $S$ by 290 | 291 | $$ 292 | Se_j = f_j 293 | $$ 294 | 295 | for $j = 1, 2, 3$. 296 | One easily checks that $S$ is an isometry (using the Pythagorean Theorem). 297 | Then, because $S^{-1} = S^*$ (by 7.42), we have $S^*f_j = e_j$. 298 | Thus 299 | 300 | $$ 301 | T_1e_1 = 2e_1 = S^*(2f_1) = S^*(T_2f_1) = S^*T_2Se_1. 302 | $$ 303 | 304 | Similarly $T_1e_2 = S^*T_2Se_2$ and $T_1e_3 = S^*T_2Se_3$. 305 | Therefore $T_1 = S^*T_2S$. 306 | 307 | _Exercise 12_ 308 | 309 | Let $e_1, e_2, e_3, e_4$ denote an orthonormal basis of $\mathbb{F}^4$. 310 | Define $T_1, T_2 \in \mathcal{L}(\mathbb{F}^4)$ by 311 | 312 | $$ 313 | \begin{aligned} 314 | T_1e_1 &= 2e_1\\\\ 315 | T_1e_2 &= 2e_2\\\\ 316 | T_1e_3 &= 5e_3\\\\ 317 | T_1e_4 &= 7e_4\\\\ 318 | \\\\ 319 | T_2e_1 &= 2e_1\\\\ 320 | T_2e_2 &= 5e_2\\\\ 321 | T_2e_3 &= 5e_3\\\\ 322 | T_2e_4 &= 7e_4. 323 | \end{aligned} 324 | $$ 325 | 326 | Then both $T_1$ and $T_2$ are self-adjoint (the matrices equal their transposes) and $2, 5, 7$ are their eigenvalues. 327 | Suppose by contradiction that $S$ is an isometry on $V$ such that $T_1 = S^*T_2S$. 328 | Let $v \in V$ be the vector that $S$ maps to $e_2$. 329 | Then 330 | 331 | $$ 332 | T_1v = S^*T_2Sv = S^*T_2e_2 = 5S^*e_2 = 5v 333 | $$ 334 | 335 | Therefore $v \in E(T_1, 5) = \operatorname{span}(e_3)$. 336 | Let also $w \in V$ be the vector that $S$ maps to $e_3$. 337 | Note that $v, w$ is linearly independent, because $e_2, e_3$ is linearly independent. 338 | Then 339 | 340 | $$ 341 | T_1w = S^*T_2Sw = S^*T_2e_3 = 5S^*e_3 = 5w. 342 | $$ 343 | 344 | Therefore $w \in E(T_1, 5) = \operatorname{span}(e_3)$. 345 | But this is a contradiction, because we can't have a linearly independent list of length $2$, $v, w$, in a $1$-dimensional vector space, $\operatorname{span}(e_3)$. 346 | Hence, there does not exist such $S$. 347 | 348 | Notice that it wasn't necessary to require $S$ to be an isometry, we just needed to suppose, by contradiction, the existence of an invertible $S$ such that $T_1 = S^{-1}T_2S$. 349 | This $S$ does not exist. 350 | Since the desired isometry must satisfy the same property (because the adjoint of an isometry equals its inverse), it follows that there cannot exist such isometry. 351 | The key idea here is that the eigenspaces of $T_1$ and $T_2$ don't fit. 352 | 353 | _Exercise 13_ 354 | 355 | It is false. 356 | Let $e_1, \dots, e_n$ be an orthonormal basis of $V$ and define $Se_j = e_1$ for $j = 1, \dots, n$. 357 | Then $||Se_j|| = 1$ for each $e_j$, but obviously $S$ is not invertible, therefore $S$ is not an isometry (7.42 requires isometries to be invertible). 358 | 359 | _Exercise 14_ 360 | 361 | In the exercise, $T$ was already shown to be self-adjoint. 362 | So $-T$ is also self-adjoint. 363 | Note that $1, \cos x, \cos 2x, \dots, \cos nx, \sin x, \sin 2x, \dots, \sin nx$ is a basis of $V$ consisting of eigenvectors of $-T$ whose corresponding eigenvalues are all nonnegative. 364 | Thus by 7.35 $-T$ is positive. 365 | -------------------------------------------------------------------------------- /Chapter 08 - Operators on Complex Vector Spaces/8B - Decomposition of an Operator.md: -------------------------------------------------------------------------------- 1 | Chapter 8: **Operators on Complex Vector Spaces** 2 | 3 | **8.B** 4 | 5 | - [x] Exercise 1 6 | - [x] Exercise 2 7 | - [x] Exercise 3 8 | - [x] Exercise 4 9 | - [x] Exercise 5 10 | - [x] Exercise 6 11 | - [x] Exercise 7 12 | - [x] Exercise 8 13 | - [x] Exercise 9 14 | - [x] Exercise 10 15 | - [x] Exercise 11 16 | 17 | _Exercise 1_ 18 | 19 | By 8.21 (a), $V = G(0, N)$. 20 | Since $G(0, N) = \operatorname{null} N^{\operatorname{dim} V}$ (see 8.11), it follows that $N^{\operatorname{dim} V} = 0$ and so $N$ is nilpotent. 21 | 22 | _Exercise 2_ 23 | 24 | Define $T \in \mathcal{L}(\mathbb{R}^3)$ by 25 | 26 | $$ 27 | T(x, y, z) = (0, -z, x). 28 | $$ 29 | 30 | That is, $T$ squashes vectors onto the $yz$ plane and rotates them counterclockwise by $\pi/2$ radians. 31 | So all eigenvectors of $T$ are contained in the $x$-axis and correspond to the eigenvalue $0$. 32 | $T$ obviously is not nilpotent. 33 | 34 | _Exercise 3_ 35 | 36 | Suppose $\lambda$ is an eigenvalue of $T$ and $v \in V$ a corresponding eigenvalue. 37 | $S$ is surject, so there exists $u \in V$ such that $Su = v$. 38 | Then 39 | 40 | $$ 41 | S^{-1}TSu = S^{-1}Tv = \lambda S^{-1}v = \lambda u, 42 | $$ 43 | 44 | which shows that $\lambda$ is an eigenvalue of $S^{-1}TS$. 45 | Hence every eigenvalue of $T$ is an eigenvalue of $S^{-1}TS$. 46 | We will prove these eigenvalues have the same multiplicity and it will follow that $S^{-1}TS$ cannot have other eigenvalues (by 8.26). 47 | 48 | Suppose $\lambda_1, \dots, \lambda_m$ are the distinct eigenvalues of $T$. 49 | Fix $k \in {1, \dots, m}$. 50 | Let $v_1, \dots, v_d$ be a basis of $G(\lambda_k, T)$. 51 | There exist $u_1, \dots, u_d \in V$ such that $Su_j = v_j$ for each $j = 1, \dots, d$. 52 | It easy to check that the $u$'s are linearly independent. 53 | We have 54 | 55 | $$ 56 | G(\lambda_k, S^{-1}TS) = \operatorname{null} (S^{-1}TS - \lambda_k I)^{\operatorname{dim} V} = \operatorname{null} S^{-1}(T - \lambda_k I)^{\operatorname{dim} V}S, 57 | $$ 58 | 59 | where the first equality comes from 8.11 and the second from Exercise 5 in section 5B. 60 | For each $j$, we have 61 | 62 | $$ 63 | S^{-1}(T - \lambda_k I)^{\operatorname{dim} V}Su_j = S^{-1}(T - \lambda_k I)^{\operatorname{dim} V}v_j = 0, 64 | $$ 65 | 66 | where the second equality follows because $v_j \in G(\lambda_k, T)$. 67 | This shows that $u_1, \dots, u_d \in G(\lambda_k, S^{-1}TS)$. 68 | Hence 69 | 70 | $$ 71 | \operatorname{dim} G(\lambda_k, S^{-1}TS) \ge d = \operatorname{dim} G(\lambda_k, T). \tag{1} 72 | $$ 73 | 74 | By 8.26, we must have 75 | 76 | $$ 77 | \operatorname{dim} G(\lambda_1, T) + \dots + \operatorname{dim} G(\lambda_m, T) = \operatorname{dim} V \tag{2} 78 | $$ 79 | 80 | and 81 | 82 | $$ 83 | \operatorname{dim} G(\lambda_1, S^{-1}TS) + \dots + \operatorname{dim} G(\lambda_m, S^{-1}TS) \le \operatorname{dim} V. \tag{3} 84 | $$ 85 | 86 | $(1)$ and $(2)$ imply that $(3)$ is only possible if $\operatorname{dim} G(\lambda_k, S^{-1}TS) = \operatorname{dim} G(\lambda_k, T)$. 87 | Hence, their multiplicieties are the same and $S^{-1}TS$ cannot have other generalized eigenspaces (the ones shown here already eat up the dimension of $V$). 88 | 89 | _Exercise 4_ 90 | 91 | By the same reasoning used in the proof of 8.4, it follows that $\operatorname{dim} \operatorname{null} T^{n-1} \ge n - 1$. 92 | But $\operatorname{null} T^{n-1} \subset \operatorname{null} T^n = G(0, T)$ (see 8.2 and 8.11). 93 | Thus $\operatorname{dim} G(0, T) \ge n - 1$ and $0$ is an eigenvalue of $T$. 94 | If $\operatorname{dim} G(0, T) = n$, 8.26 shows that $0$ is the only eigenvalue of $T$. 95 | If $\operatorname{dim} G(0, T) = n - 1$, there is only space for one more eigenvalue with multiplicty $1$. 96 | 97 | _Exercise 5_ 98 | 99 | Every eigenvector is also a generalized eigenvector, so in the forward direction we can make a similar argument to that of Exercise 3, where the dimensions only fit if each eigenspace equals its corresponding generalized one (because the former is a subset of the latter). 100 | The other direction is obvious from 8.23. 101 | 102 | _Exercise 6_ 103 | 104 | The formula for $N$ doesn't really matter here. 105 | We only care about the dimension of $\mathbb{F}^{5}$, which is $5$. 106 | Using the same reasoning from the proof of 8.31, and because $N^j = 0$ for $j \ge 5$, we have 107 | 108 | $$ 109 | \begin{aligned} 110 | I + N &= (I + a_1N + a_2N^2 + a_3N^3 + a_4N^4)\\\\ 111 | &= I + 2a_1N + (2a_2 + a_1^2)N^2 + (2a_3 + 2a_1a_2)N^3 + (2a_4 + 2a_1a_3 + a_2^2)N^4 112 | \end{aligned} 113 | $$ 114 | 115 | for some $a_1, a_2, a_3, a_4 \in \mathbb{F}$. 116 | Choose 117 | 118 | $$ 119 | \begin{aligned} 120 | a_1 &= \frac{1}{2}\\\\ 121 | a_2 &= \frac{-1}{8}\\\\ 122 | a_3 &= \frac{1}{16}\\\\ 123 | a_4 &= \frac{-5}{128} 124 | \end{aligned} 125 | $$ 126 | 127 | and the terms on the second line will collapse to $N + I$. 128 | Hence 129 | 130 | $$ 131 | I + \frac{1}{2}N - \frac{1}{8}N^2 + \frac{1}{16}N^3 - \frac{5}{128}N^4 132 | $$ 133 | 134 | is a square root of $N + I$. 135 | 136 | _Exercise 7_ 137 | 138 | One can use the same strategy as in the proof of 8.31 to show that $I + N$ has a cube root for any nilpotent $N \in \mathcal{L}(V)$ and the rest of the proof will be same as the proof of 8.33. 139 | 140 | _Exercise 8_ 141 | 142 | If $0$ is not an eigenvalue of $T$, then $T^j$ is injective and surjective for all integers $j$, which gives the desired result (take $j = n-2$). 143 | 144 | Suppose $0$ is an eigenvalue of $T$. 145 | Since $3$ and $8$ are also eigenvalues of $T$, by 8.26 the multiplicity of $0$, namely $\operatorname{dim} G(0, T)$ which equals $\operatorname{dim} \operatorname{null} T^n$, is at most $n - 2$. 146 | By the same reasoning used in the proof of 8.4, we have $\operatorname{null} T^{n-2} = \operatorname{null} T^n$ (because the $\operatorname{null} T^{n-2} \subset \operatorname{null} T^n$). 147 | Exercise 19 of section 8A implies that $\operatorname{range} T^{n-2} = \operatorname{range} T$. 148 | Now 8.5 completes the proof. 149 | 150 | _Exercise 9_ 151 | 152 | Keep in mind that when we mention the size of an $n$-by-$n$ matrix here we mean $n$ and not $n$ times $n$. 153 | 154 | Let $V$ be vector space whose dimension equals the size $A$ (or $B$, since they're the same). 155 | Choose a basis of $V$ and define $S, T \in \mathcal{L}(V)$ such that $\mathcal{M}(S) = A$ and $\mathcal{M}(T) = B$. 156 | Then $\mathcal{M}(ST) = AB$. 157 | 158 | Let $d_j$ equal the size of $A_j$ (or $B_j$, because they're the same). 159 | Consider the list consisting of the first $d_1$ vectors in the chosen basis. 160 | $A$ and $B$ show that the span of these vectors are invariant under $S$ and $T$. 161 | Similarly, the span of the next $d_2$ vectors after this list is also invariant under $T$. 162 | Continuing in this fashion, we see that there are $m$ distinct lists of consecutive vectors, with no intersections, in the chosen basis whose spans are invariant under $S$ and $T$. 163 | 164 | Let $U_1, \dots, U_m$ denote such spans. 165 | Clearly $\mathcal{M}(S|_{U_j}) = A_j$ and $\mathcal{M}(T|_{U_j}) = B_j$ for each $j$. 166 | Hence $\mathcal{M}(S|_{U_j}T|_{U_j}) = A_jB_j$ and so it easy to see that $\mathcal{M}(ST)$ (which equals $AB$) has the desired form. 167 | 168 | _Exercise 10_ 169 | 170 | Let $v_1, \dots, v_n$ denote a basis of $V$ consisting of generalized eigenvectors of $T$ (which exists by 8.23). 171 | Define $\langle \cdot, \cdot \rangle: V \times V \to \mathbb{C}$ by 172 | 173 | $$ 174 | \langle a_1 v_1 + \dots + a_n v_n, b_1 v_1 + \dots + b_n v_n \rangle = a_1\overline{b_1} + \dots + a_n\overline{b_n}, 175 | $$ 176 | 177 | where the $a$'s and $b$'s are complex numbers. 178 | You can check that $\langle \cdot, \cdot \rangle$ is a well defined inner product on $V$. 179 | Thus $v_1, \dots, v_n$ is an orthonormal basis of $V$. 180 | Moreover, the generalized eigenspaces of $T$ are orthogonal to each other. 181 | This implies that, if $v \in G(\beta, T)$, then 182 | 183 | $$ 184 | P_{G(\alpha, T)} v = 185 | \begin{cases} 186 | v, \text{ if } \alpha = \beta\\\\ 187 | 0, \text{ if } \alpha \neq \beta 188 | \end{cases} 189 | \tag{$\*$} 190 | $$ 191 | 192 | where $P_{G(\alpha, T)}$ is the orthogonal projection of $V$ onto $G(\alpha, T)$. 193 | 194 | Let $\lambda_1, \dots, \lambda_m$ denote the distinct eigenvalues of $T$. 195 | We have 196 | 197 | $$ 198 | T = T|_{G(\lambda_1, T)}P_{G(\lambda_1, T)} + \dots + T|_{G(\lambda_m, T)}P_{G(\lambda_m, T)}. 199 | $$ 200 | 201 | For each $j = 1, \dots, m$, we can write $T|_{G(\lambda_j, T)} = \lambda_j I + N_j$ where $N_j$ is a nilpotent operator under which $G(\lambda_j, T)$ is invariant (see 8.21 (c)). 202 | Therefore 203 | 204 | $$ 205 | \begin{aligned} 206 | T &= (\lambda_1 I + N_1)P_{G(\lambda_1, T)} + \dots + (\lambda_m I + N_m)P_{G(\lambda_m, T)}\\\\ 207 | &= \underbrace{\lambda_1 P_{G(\lambda_1, T)} + \dots + \lambda_m P_{G(\lambda_m, T)}}_\text{(4)} + \underbrace{N_1P_{G(\lambda_1, T)} + \dots + N_mP_{G(\lambda_m, T)}}_\text{(5)}. 208 | \end{aligned} 209 | $$ 210 | 211 | Fix $k \in \{1, \dots, n\}$. 212 | Then $v_k \in G(\lambda_j, T)$ for some $j \in \{1, \dots m\}$. 213 | $(*)$ shows that $(4)$ maps $v_k$ to $\lambda_j v_k$. 214 | Hence $v_1, \dots, v_n$ is a basis of eigenvectors of $(4)$ and so $(4)$ is diagonalizable. 215 | $(*)$ also shows that $(5)$ maps $v_k$ to $N_j v_k$. 216 | But $G(\lambda_j, T)$ is invariant under $N_j$, so $(*)$ actually implies that $(5)$ raised to the power of $\operatorname{dim} V$ maps $v_k$ to $N_j^{\dim V}v_k$ which equals $0$. 217 | Therefore $(5)$ is nilpotent. 218 | It is easy to see that $(4)$ and $(5)$ commute (they map $v_k$ to $\lambda_j N_j v_k$, no matter the order), which completes the proof. 219 | 220 | _Exercise 11_ 221 | 222 | Suppose $T$ has an upper-triangular matrix with respect to the basis $v_1, \dots, v_n$. 223 | Suppose also that $\lambda$ appears on the $j$-th diagonal entry of $\mathcal{M}(T)$. 224 | Then 225 | 226 | $$ 227 | Tv_j = a_1 v_1 + \dots + a_{j-1} v_{j-1} + \lambda v_j 228 | $$ 229 | 230 | for some $a_1, \dots, a_{j-1} \in \mathbb{F}$, 231 | and so 232 | 233 | $$ 234 | (T - \lambda I)v_j = a_1 v_1 + \dots + a_{j-1} v_{j-1} \in \operatorname{span}(v_1, \dots, v_{j-1}) 235 | $$ 236 | 237 | We have 238 | 239 | $$ 240 | (T - \lambda I)v_{j-1} = c_1 v_1 + \dots + (c_{j-1} - \lambda) v_{j-1} 241 | $$ 242 | 243 | for some $c_1, \dots, c_{j-1} \in \mathbb{F}$. 244 | If $c_{j-1} - \lambda = 0$, then $(T - \lambda I)^2v_j \in \operatorname{span}(v_1, \dots, v_{j-2})$. 245 | If $c_{j-1} - \lambda \neq 0$, then 246 | 247 | $$ 248 | (T - \lambda I)\left(v_j - \frac{a_{j-1}}{c_{j-1} - \lambda}v_{j-1}\right) \in \operatorname{span}(v_1, \dots, v_{j-2}). 249 | $$ 250 | 251 | We go on, either squaring by squaring $(T - \lambda I)$ or subtracting a vector $u \in \operatorname{span}(v_1, \dots, v_{j-1})$ from $v_j$ in the argument of $(T - \lambda I)$, and we will have $(T - \lambda I)^2(v_j - u)$ in the span of the first $j - 3$ vectors of the basis, then in span of the first $n - 4$, an so on until it will be in the span of an empty list, that is, $\\{0\\}$. 252 | This means that $v_j - u \in G(\lambda, T)$ for some $u \in \operatorname{span}(v_1, \dots, v_{j-1})$. 253 | 254 | Let $\nu_1, \dots, \nu_d$ denote the vectors of the chosen basis of $V$ that correspond to the columns of $\mathcal{M}(T)$ in which $\lambda$ appears and in the order that they appear on the basis. 255 | We can repeat the previous process and find $u_1, \dots, u_d$ such 256 | 257 | $$ 258 | \nu_1 - u_1, \dots, \nu_d - u_d \in G(\lambda, T), \tag{6} 259 | $$ 260 | 261 | where each $u_k$ is in the span of the basis vectors that come before $\nu_k$. 262 | We claim this list is linearly independent. 263 | To see this, fix $k \in \\{1, \dots, d\\}$. 264 | Suppose $\nu_k$ is the $j$-th basis vector, i.e. $\nu_k = v_j$. 265 | This means that 266 | 267 | $$ 268 | \operatorname{span}(\nu_1 - u_1, \dots, \nu_{k-1} - u_{k-1}) \subset \operatorname{span}(v_1, \dots, v_{j-1}). 269 | $$ 270 | 271 | Therefore, we can't have $\nu_k - u_k \in \operatorname{span}(\nu_1 - u_1, \dots, \nu_{k-1} - u_{k-1})$, because that would imply that 272 | 273 | $$ 274 | v_j \in \operatorname{span}(v_1, \dots, v_{j-1}), 275 | $$ 276 | 277 | since $u_k \in \operatorname{span}(v_1, \dots, v_{j-1})$. 278 | This argument can be repeated for each $k$. 279 | Therefore no vector in $(6)$ is in the span of the previous ones. 280 | It follows that the list in $(6)$ is linearly independent. 281 | 282 | Hence $\operatorname{dim} G(\lambda, T) \ge d$. 283 | By 8.26, the dimension of $G(\lambda, T)$ cannot be greater $d$, because we have $\operatorname{dim} V$ diagonal entries and each one adds at least $1$ to the dimension of some generalized eigenspace. 284 | Thus $\operatorname{dim} G(\lambda, T) = d$, completing the proof. 285 | -------------------------------------------------------------------------------- /Chapter 08 - Operators on Complex Vector Spaces/8A - Generalized Eigenvectors and Nilpotent Operators.md: -------------------------------------------------------------------------------- 1 | Chapter 8: **Operators on Complex Vector Spaces** 2 | 3 | **8.A** 4 | 5 | - [x] Exercise 1 6 | - [x] Exercise 2 7 | - [x] Exercise 3 8 | - [x] Exercise 4 9 | - [x] Exercise 5 10 | - [x] Exercise 6 11 | - [x] Exercise 7 12 | - [x] Exercise 8 13 | - [x] Exercise 9 14 | - [x] Exercise 10 15 | - [x] Exercise 11 16 | - [x] Exercise 12 17 | - [x] Exercise 13 18 | - [x] Exercise 14 19 | - [x] Exercise 15 20 | - [x] Exercise 16 21 | - [x] Exercise 17 22 | - [x] Exercise 18 23 | - [x] Exercise 19 24 | - [x] Exercise 20 25 | - [x] Exercise 21 26 | 27 | _Exercise 1_ 28 | 29 | Since 30 | 31 | $$ 32 | T^2(w, z) = T(z, 0) = (0, 0), 33 | $$ 34 | 35 | it follows that $G(0, T) = V$. 36 | Therefore every vector in $\mathbb{C}^2$ is a generalized eigenvector of $T$. 37 | 38 | _Exercise 2_ 39 | 40 | The eigenvalues of $T$ are $i$ and $-i$. 41 | Since $\mathbb{C}^2$ has dimension $2$, the generalized eigenspaces are the eigenspaces themselves. 42 | 43 | _Exercise 3_ 44 | 45 | We will prove $\operatorname{null} (T - \lambda I)^n = \operatorname{null} \left(T^{-1} - \frac{1}{\lambda} I\right)^n$ for all nonnegative integers $n$ by induction on $n$. 46 | 47 | It is easy to check that $\operatorname{null} (T - \lambda I) = \operatorname{null} \left(T^{-1} - \frac{1}{\lambda} I\right)$ (see Exercise 9 in section 5C). 48 | Let $n > 1$ and assume the result holds for all nonnegative integers less than $n$. 49 | Suppose $v \in \operatorname{null}(T - \lambda I)^n$. 50 | Then 51 | 52 | $$ 53 | (T - \lambda I)v \in \operatorname{null}\left(T - \lambda I\right)^{n-1}. 54 | $$ 55 | 56 | By the induction hypothesis 57 | 58 | $$ 59 | (T - \lambda I)v \in \operatorname{null}\left(T^{-1} - \frac{1}{\lambda} I\right)^{n-1}. 60 | $$ 61 | 62 | Thus 63 | 64 | $$ 65 | 0 = \left(T^{-1} - \frac{1}{\lambda} I\right)^{n-1}(T - \lambda I)v = (T - \lambda I)\left(T^{-1} - \frac{1}{\lambda} I\right)^{n-1}v, 66 | $$ 67 | 68 | where the second equality follows from Theorem 1 in Chapter 5 notes. 69 | 70 | Therefore 71 | 72 | $$ 73 | \left(T^{-1} - \frac{1}{\lambda}I\right)^{n-1}v \in \operatorname{null} (T - \lambda I). 74 | $$ 75 | 76 | But 77 | 78 | $$ 79 | \operatorname{null} (T - \lambda I) = \operatorname{null} \left(T^{-1} - \frac{1}{\lambda} I\right). 80 | $$ 81 | 82 | Hence 83 | 84 | $$ 85 | \left(T^{-1} - \frac{1}{\lambda}I\right)^{n-1}v \in \operatorname{null} \left(T^{-1} - \frac{1}{\lambda} I\right) 86 | $$ 87 | 88 | and so 89 | 90 | $$ 91 | 0 = \left(T^{-1} - \frac{1}{\lambda} I\right) \left(T^{-1} - \frac{1}{\lambda}I\right)^{n-1}v = \left(T^{-1} - \frac{1}{\lambda}I\right)^n v, 92 | $$ 93 | 94 | which shows that $v \in \operatorname{null} (T^{-1} - \frac{1}{\lambda}I)$. 95 | Therefore $\operatorname{null} (T - \lambda I)^n \subset \operatorname{null} \left(T^{-1} - \frac{1}{\lambda} I\right)^n$. 96 | To prove the inclusion in the other direction, it suffices to repeat the same thing replacing $\left(T - \lambda I\right)$ with $\left(T^{-1} - \frac{1}{\lambda}I\right)$ and vice versa. 97 | 98 | Now, by 8.11, we have 99 | 100 | $$ 101 | G(\lambda, T) = \operatorname{null}(T - \lambda I)^{\operatorname{dim} V} = \operatorname{null}\left(T^{-1} - \frac{1}{\lambda} I\right)^{\operatorname{dim} V} = G\left(\frac{1}{\lambda}, T^{-1}\right). 102 | $$ 103 | 104 | _Exercise 4_ 105 | 106 | Suppose $v \in G(\alpha, T) \cap G(\beta, T)$ and suppose by contradiction that $v \neq 0$. 107 | Then $v, v$ are generalized eigenvectors corresponding to distinct generalized eigenvalues of $T$. 108 | Now 8.13 implies that $v, v$ is linearly independent, which is clearly a contradiction. 109 | Therefore $v$ must be $0$. 110 | 111 | _Exercise 5_ 112 | 113 | Let $a_0, a_1, \dots, a_{m-1} \in \mathbb{F}$ such that 114 | 115 | $$ 116 | 0 = a_0v + a_1Tv + \dots + a_{m-1}T^{m-1}v 117 | $$ 118 | 119 | Applying $T^{m-1}$ to both sides of the equation above yields 120 | 121 | $$ 122 | 0 = a_0T^{m-1}v, 123 | $$ 124 | 125 | which shows that $a_0 = 0$. 126 | Therefore 127 | 128 | $$ 129 | 0 = a_1Tv + \dots + a_{m-1}T^{m-1}v 130 | $$ 131 | 132 | Applying $T^{m-2}$ yields 133 | 134 | $$ 135 | 0 = a_1T^{m-1}v, 136 | $$ 137 | 138 | which shows that $a_1 = 0$. 139 | Continuing in this fashion, we see that $a_0 = a_1 = \dots = a_m = 0$. 140 | Thus $v, Tv, T^2v, \dots, T^{m-1}v$ is linearly independent. 141 | 142 | _Exercise 6_ 143 | 144 | Suppose by contradiction that $S \in \mathcal{L}(\mathbb{C}^3)$ is a square root of $T$. 145 | Note that $V = \operatorname{null} T^3$. 146 | We have 147 | 148 | $$ 149 | \begin{aligned} 150 | V &= \operatorname{null} T^3\\\\ 151 | &= \operatorname{null} R^6\\\\ 152 | &= \operatorname{null} R^3\\\\ 153 | &= \operatorname{null} RT\\\\ 154 | &\subset \operatorname{null} R^2T\\\\ 155 | &= \operatorname{null} T^2, 156 | \end{aligned} 157 | $$ 158 | 159 | where the third line follows by 8.4. 160 | But this a contradiction, since $T^2(z_1, z_2, z_3) = (z_3, 0, 0)$, we see that $\operatorname{null} T^2 = \\{(0, 0, z): z \in \mathbb{C}\\}$, then we can't have $V \subset \operatorname{null} T^2$. 161 | 162 | _Exercise 7_ 163 | 164 | This follows directly from 8.19 and 5.32. 165 | 166 | _Exercise 8_ 167 | 168 | False. 169 | Let $V = \mathbb{C}^2$. 170 | Define $S, T \in \mathcal{L}(\mathbb{C})$ by 171 | 172 | $$ 173 | \begin{aligned} 174 | S(z_1, z_2) &= (0, z_1)\\\\ 175 | T(z_1, z_2) &= (z_2, 0). 176 | \end{aligned} 177 | $$ 178 | 179 | Both $S$ and $T$ are nilpotent, however $S + T$ is not (its square equals the identity). 180 | 181 | _Exercise 9_ 182 | 183 | We have 184 | 185 | $$ 186 | \operatorname{null} (TS)^{\operatorname{dim} V} = \operatorname{null} TS(TS)^{\operatorname{dim} V} = \operatorname{null} T(ST)^{\operatorname{dim} V}S = V, 187 | $$ 188 | 189 | where the first equality follows from 8.4 and the third because $(ST)^{\operatorname{dim} V} = 0$ (by 8.18). 190 | Thus $(TS)^{\operatorname{dim} V} = 0$ and so $TS$ is nilpotent. 191 | 192 | _Exercise 10_ 193 | 194 | If $T$ is not nilpotent, then $\operatorname{dim} \operatorname{null} T^n < n$ and , by the same reasoning used in 8.4, it follows that $\operatorname{null} T^{n-1} = \operatorname{null} T^n$. 195 | Thus, by 8.5, we have 196 | 197 | $$ 198 | V = \operatorname{null} T^{n-1} + \operatorname{range} T^n. 199 | $$ 200 | 201 | Since $\operatorname{range} T^n \subset \operatorname{range} T^{n-1}$, we must also have 202 | 203 | $$ 204 | V = \operatorname{null} T^{n-1} + \operatorname{range} T^{n-1}. 205 | $$ 206 | 207 | Then, by the Fundamental Theorem of Linear Maps (3.22), 208 | 209 | $$ 210 | \operatorname{dim} (\operatorname{null} T^{n-1} + \operatorname{range} T^{n-1}) = \operatorname{dim} V = \operatorname{dim} \operatorname{null} T^{n-1} + \operatorname{dim} \operatorname{range} T^{n-1}. 211 | $$ 212 | 213 | 3.78 now implies that $\operatorname{null} T^{n-1} + \operatorname{range} T^{n-1}$ is a direct sum. 214 | 215 | _Exercise 11_ 216 | 217 | This is not correct. Indeed, consider $V = \mathbb{C}^2$ and $T$ with matrix 218 | 219 | $$ 220 | \begin{pmatrix} 221 | 1 & x \\ 222 | 0 & 1 223 | \end{pmatrix} 224 | $$ 225 | 226 | It is clear that 1 is the only eigenvalue of $T$, and that $v = (1, 0)$ is the corresponding eigenvector if $x \neq 0$, with all other eigenvectors being linear combinations of this one. 227 | 228 | $T^2$ has a similar matrix: 229 | 230 | $$ 231 | \begin{pmatrix} 232 | 1 & 2x \\ 233 | 0 & 1 234 | \end{pmatrix} 235 | $$ 236 | 237 | ,and so it also has just one linearly independent eigenvector, so there is no basis of $\mathbb{C}^2$ in which its matrix is diagonal. 238 | 239 | _Exercise 12_ 240 | 241 | Suppose $v_1, \dots, v_n$ is such basis. 242 | Then $Nv_1 = 0$, because the the first column of the matrix has $0$ in all its entries. 243 | The definition of matrix of linear map shows that $Nv_2 \in \operatorname{span}(v_1)$. 244 | But this implies that $N^2v_2 = 0$. 245 | Similarly, $Nv_3 \in \operatorname{span}(v_1, v_2)$, so $N^3v_3 = 0$. 246 | Continuing like this, we see that $N^j v_j = 0$, for each $j = 1, \dots, n$. 247 | Therefore $N^n = 0$ and so $N$ is nilpotent. 248 | 249 | _Exercise 13_ 250 | 251 | It is easy when $\mathbb{F} = \mathbb{C}$, because then $V$ has a basis consisting of eigenvectors of $N$ and for each vector $v$ in this basis we have $0 = N^{\operatorname{dim} V} v = \lambda^{\operatorname{dim} V} v$ for the corresponding eigenvalue $\lambda$, which implies that $\lambda = 0$. 252 | 253 | More generally, without restricting $\mathbb{F}$ to $\mathbb{C}$, we will prove $N^{\operatorname{dim} V - 1} = 0$ and this fact can be used to show $N^{\operatorname{dim} V - 2} = 0$, which then can be used to show... and so on until $N^1$. 254 | 255 | Let $\mathcal{N} = N^{\operatorname{dim} V - 1}$. 256 | Note that $\mathcal{N}$ is also normal and that $\mathcal{N}^2 = 0$. 257 | Then, for all $v \in V$, 258 | 259 | $$ 260 | ||\mathcal{N}^*\mathcal{N}v||^2 = ||\mathcal{N}\mathcal{N}v|| = 0, 261 | $$ 262 | 263 | where the first equality comes from 7.20. 264 | Thus $\mathcal{N}^*\mathcal{N} = 0$. 265 | Therefore 266 | 267 | $$ 268 | ||\mathcal{N}v||^2 = \langle \mathcal{N}v, \mathcal{N}v \rangle = \langle v, \mathcal{N}^*\mathcal{N}v \rangle = 0, 269 | $$ 270 | 271 | which shows that $\mathcal{N} = 0$. 272 | 273 | _Exercise 14_ 274 | 275 | This follows directly from 8.19 and 6.37. 276 | 277 | _Exercise 15_ 278 | 279 | By the same reasoning used in the proof of 8.4, it follows that $\operatorname{dim} \operatorname{null} N^{\operatorname{dim} V} \ge \operatorname{dim} V$. 280 | Thus $\operatorname{dim} \operatorname{null} N^{\operatorname{dim} V} = \operatorname{dim} V$ and so $N$ is nilpotent. 281 | We have $\operatorname{dim} V + 1$ null spaces each of different dimension. 282 | Since the sequence 283 | 284 | $$ 285 | \operatorname{dim} \operatorname{null} N^0, \operatorname{dim} \operatorname{null} N^1, \dots, \operatorname{dim} \operatorname{null} N^{\operatorname{dim} V} 286 | $$ 287 | 288 | must be sorted in strictly increasing order, the only way this can fit is if $\operatorname{dim} \operatorname{null} N^j = j$ for each $j$. 289 | 290 | _Exercise 16_ 291 | 292 | Obviously $V = \operatorname{range} T^0 = \operatorname{range} I$. 293 | Let $k$ be a nonnegative integer. 294 | Suppose $v \in \operatorname{range} T^{k+1}$. 295 | Then $v = T^{k+1}u$ for some $u \in V$. 296 | But then $v = T^k(Tu)$. 297 | This implies that $v \in \operatorname{range} T^k$. 298 | 299 | _Exercise 17_ 300 | 301 | By the Fundamental Theorem of Linear Maps (3.22), we have 302 | 303 | $$ 304 | \operatorname{dim} \operatorname{null} T^m + \operatorname{dim} \operatorname{range} T^m = \operatorname{dim} \operatorname{null} T^{m+1} + \operatorname{dim} \operatorname{range} T^{m+1}, 305 | $$ 306 | 307 | which implies that $\operatorname{dim} \operatorname{null} T^m = \operatorname{dim} \operatorname{null} T^{m+1}$ (because $\operatorname{dim} \operatorname{range} T^m = \operatorname{dim} \operatorname{range} T^{m+1}$). 308 | Thus, by 8.3, for all $k > m$, we have 309 | 310 | $$ 311 | \operatorname{dim} \operatorname{null} T^m = \operatorname{dim} \operatorname{null} T^{m+k}. 312 | $$ 313 | 314 | Applying the Fundamental Theorem of Linear Maps again to $T^m$ and $T^{m+k}$ we see that 315 | 316 | $$ 317 | \operatorname{dim} \operatorname{range} T^m = \operatorname{dim} \operatorname{range} T^{m+k}. 318 | $$ 319 | 320 | Since $\operatorname{range} T^{m+k} \subset \operatorname{range} T^m$, it follows that $\operatorname{range} T^{m+k} = \operatorname{range} T^m$. 321 | 322 | _Exercise 18_ 323 | 324 | This follows directly from the previous exercise and 8.4. 325 | 326 | _Exercise 19_ 327 | 328 | This is just a matter of realizing that $\operatorname{null} T^m \subset \operatorname{null} T^{m+1}$ and $\operatorname{range} T^{m+1} \subset \operatorname{range} T^m$ and applying the Fundamental Theorem of Linear Maps. 329 | 330 | _Exercise 20_ 331 | 332 | By Exercise 19, $\operatorname{null} T^4 \neq \operatorname{null} T^5$. 333 | By Exercise 15, this implies that $T$ is nilpotent. 334 | 335 | _Exercise 21_ 336 | 337 | Let $W = \mathbb{F}^\infty \times \mathbb{F}^\infty$ and define $T \in \mathcal{L}(W)$ by 338 | 339 | $$ 340 | T\bigr((x_1, x_2, x_3, \dots), (y_1, y_2, y_3, \dots)\bigl) = \bigr((x_2, x_3, \dots), (0, y_1, y_2, y_3, \dots)\bigl), 341 | $$ 342 | 343 | that is, $T$ applies the backward shift operator (call it $B$) on the first slot and forward shift operator (call it $F$) on the second slot. 344 | Thus, for each positive integer $k$, we have 345 | 346 | $$ 347 | \operatorname{null} B^k = \\{(x_1, x_2, x_3, \dots) \in \mathbb{F}^\infty: x_j = 0 \text{ for all } j > k\\} 348 | $$ 349 | 350 | and 351 | 352 | $$ 353 | \operatorname{range} F^k = \\{(x_1, x_2, x_3, \dots) \in \mathbb{F}^\infty: x_1 = x_2 = \dots = x_k = 0\\}. 354 | $$ 355 | 356 | Moreover $\operatorname{range} B^k = \mathbb{F}^\infty$ and $\operatorname{null} F^k = \\{0\\}$. 357 | Note that $\operatorname{null} B^k \subsetneq \operatorname{null} B^{k+1}$ and $\operatorname{range} F^k \supsetneq \operatorname{range} F^{k+1}$. 358 | Thus 359 | 360 | $$ 361 | \operatorname{null} T^k = \\{(x, 0) \in \mathbb{F}^\infty \times \mathbb{F}^\infty: x \in \operatorname{null} B^k\\} 362 | $$ 363 | 364 | and 365 | 366 | $$ 367 | \operatorname{range} T^k = \\{(x, y) \in \mathbb{F}^\infty \times \mathbb{F}^\infty: y \in \operatorname{range} T^k\\}. 368 | $$ 369 | 370 | Hence $\operatorname{null} T^k \subsetneq \operatorname{null} T^{k+1}$ and $\operatorname{range} T^k \supsetneq \operatorname{range} T^{k+1}$. 371 | -------------------------------------------------------------------------------- /Chapter 06 - Inner Product Spaces/6C - Orthogonal Complements and Minimization Problems.md: -------------------------------------------------------------------------------- 1 | Chapter 6: **Inner Product Spaces** 2 | 3 | **6.C** 4 | 5 | - [x] Exercise 1 6 | - [x] Exercise 2 7 | - [x] Exercise 3 8 | - [ ] Exercise 4 9 | - [x] Exercise 5 10 | - [x] Exercise 6 11 | - [x] Exercise 7 12 | - [x] Exercise 8 13 | - [x] Exercise 9 14 | - [x] Exercise 10 15 | - [x] Exercise 11 16 | - [x] Exercise 12 17 | - [x] Exercise 13 18 | - [x] Exercise 14 19 | 20 | _Exercise 1_ 21 | 22 | Suppose $w \in \{v_1, \dots, v_m\}^\perp$. 23 | Let $v = \in \operatorname{span}(v_1, \dots, v_m)$. 24 | We have that 25 | 26 | $$ 27 | v = a_1 v_1 + \dots a_m v_m 28 | $$ 29 | 30 | for some $a_1, \dots, a_m \in \mathbb{F}$. 31 | Moreover 32 | 33 | $$ 34 | \langle v, w \rangle = \langle a_1 v_1 + \dots a_m v_m, w \rangle = a_1 \langle v_1, w \rangle + \dots + a_m \langle v_m, w \rangle = 0. 35 | $$ 36 | 37 | Thus $w \in (\operatorname{span}(v_1, \dots, v_m))^\perp$ and so $\\{v_1, \dots, v_m\\}^\perp \subset (\operatorname{span}(v_1, \dots, v_m))^\perp$. 38 | 39 | Now suppose $w \in (\operatorname{span}(v_1, \dots, v_m))^\perp$. 40 | Since each $v_j$ is in $\operatorname{span}(v_1, \dots, v_m)$, it follows that $w$ is orthogonal to each $v_j$. 41 | Therefore $w \in \\{v_1, \dots, v_m\\}^\perp$ and thus $(\operatorname{span}(v_1, \dots, v_m))^\perp \subset \\{v_1, \dots, v_m\\}^\perp$. 42 | 43 | _Exercise 2_ 44 | 45 | Suppose $U^\perp = \{0\}$. 46 | Since $V = U \oplus U^\perp$ (by 6.47), it follows that $U = V$. 47 | 48 | Conversely, suppose $U = V$. 49 | Then $U^\perp = V^\perp = \{0\}$. 50 | 51 | _Exercise 3_ 52 | 53 | By 6.31, we have $\operatorname{span}(u_1, \dots, u_m) = \operatorname{span}(e_1, \dots, e_m)$. 54 | Therefore $e_1, \dots, e_m$ is indeed an orthonormal basis of $U$. 55 | Clearly $\operatorname{span}(f_1, \dots, f_n) \subset U^\perp$ (because each of the $f$'s are orthogonal to the basis of $U$). 56 | Moreover, 57 | 58 | $$ 59 | \operatorname{dim} U^\perp = \operatorname{dim} V - \operatorname{dim} U = m + n - m = n. 60 | $$ 61 | 62 | Hence $f_1, \dots, f_n$ is linearly independent list of length $\operatorname{dim} U^\perp$, that is, a basis of $U^\perp$. 63 | 64 | _Exercise 5_ 65 | 66 | Suppose $v \in V$. 67 | By 6.47, we can write $v = u + w$, where $u \in U$ and $w \in U^\perp$. 68 | Note that, by 6.51, $(U^\perp)^\perp = U$. 69 | Thus $P_{U^\perp}v = w$, because $u \in (U^\perp)^\perp$. 70 | Therefore 71 | 72 | $$ 73 | P_{U^\perp}v = w = v - u = Iv - P_Uv = (I - P_U)v. 74 | $$ 75 | 76 | Hence $P_{U^\perp} = I - P_U$. 77 | 78 | _Exercise 6_ 79 | 80 | Suppose $P_U P_W = 0$. 81 | Then $\operatorname{range} P_W \subset \operatorname{null} P_U$. 82 | However, 6.55 shows that $\operatorname{range} P_W = W$ and $\operatorname{null} P_U = U^\perp$. 83 | Thus $W \subset U^\perp$, which implies that $\langle u, w \rangle = 0$ for all $u \in U$ and all $w \in W$. 84 | 85 | Conversely, suppose $\langle u, w \rangle = 0$ for all $u \in U$ and all $w \in W$. 86 | It follows that $W \subset U^\perp$. 87 | Let $v \in V$. 88 | By 6.47, we can write $v = w + u$, where $w \in W$ and $u \in W^\perp$. 89 | Then 90 | 91 | $$ 92 | P_U P_W v = P_Uw = 0. 93 | $$ 94 | 95 | Where the last equality, follows from the fact that $W \subset U^\perp$. 96 | 97 | _Exercise 7_ 98 | 99 | Define $U = \operatorname{range} P$. 100 | Suppose $u \in U$. 101 | There exists $v \in V$ such that $Pv = u$. 102 | Then 103 | 104 | $$ 105 | u = Pv = P^2 v = P(Pv) = Pu. 106 | $$ 107 | 108 | We've shown that $Pu = u$ for any $u \in U$. 109 | Let $v \in V$. 110 | By Exercise 4 in section 5B, we can write $v = u + w$ for some $u \in U$ and $w \in \operatorname{null} P$. 111 | Note that $\operatorname{null} P \subset U^\perp$. 112 | Thus 113 | 114 | $$ 115 | Pv = P(u + w) = Pu = u = P_U(u + w) = P_Uv. 116 | $$ 117 | 118 | Therefore $P = P_U$. 119 | 120 | _Exercise 8_ 121 | 122 | We will try to prove that $U = \operatorname{range}P$ is the desired subspace, such that $P = P_U$. 123 | 124 | Consider any basis $u_1,...,u_k$ of $U$, and extend it to a basis of $V$. Perform Gramm-Schmidt orthogonalization, getting orthonormal basis $e_1,...,e_n$, with $e_1,...,e_k$ still forming basis of $U$. We can notice, that $Pe_i = e_i$ for any $i =1...k$, because since $e_i \in U$, there exists $w: Pv = e_i$, and so $Pe_i = P^2w = Pw = e_i$. 125 | 126 | Consider $j>k$ with $Pe_j = b_1e_1 + ... + b_kc_k$ (since $Pe_j \in U$, it can be represented through its basis), and let us consider vector $v = x_1e_1 + ... x_ke_k + e_{k+1}$. We can easily write its squared norm as 127 | 128 | $$ 129 | ||v||^2 = x_1^2 + ... + x_k^2 + 1 130 | $$ 131 | 132 | Now, applying $P$ to this vector and computing its squared norm, we get 133 | 134 | $$ 135 | ||Pv||^2 = ||P(x_1e_1 + ... x_ke_k + e_{k+1})|| = (x_1 + b_1)^2 + ... + (x_k + b_k)^2 136 | $$ 137 | 138 | We know, that $||Pv|| \le ||v||$, hence $||Pv||^2 \le ||v||^2$, so 139 | 140 | $$ 141 | (x_1 + b_1)^2 + ... + (x_k + b_k)^2 \le x_1^2 + ... + x_k^2 + 1 142 | $$ 143 | 144 | Let us select all $x_i$ to be proportional to $b_i$ with some fixed coefficient $x$: $x_i = xb_i$. Then, inequality simplifies 145 | 146 | $$ 147 | \begin{align} 148 | (1 + x)^2b_1^2 + ... + (x + 1)^2b_k^2 &\le x^2(b_1^2 + ... + b_k^2) + 1 \implies \\ 149 | (2x + 1)\sum\limits_{i=1}^kb_i^2 &\le 1 150 | \end{align} 151 | $$ 152 | 153 | We can now see, that if $\sum\limits_{i=1}^kb_i^2 \neq 0$, we have a condition which can be violated by selecting appropriate $x > \frac{1}{2\sum\limits_{i=1}^kb_i^2} - \frac{1}{2}$. Hence, $\sum\limits_{i=1}^kb_i^2 = 0$, which implies $b_i = 0$ for all $i=1...k$, so $Pe_j = 0$ for any $j > k$, and so since for any $v \in V$ $v = \sum\limits_{i=1}^kx_ie_i + \sum\limits_{i=k+1}^nx_ie_i = u + w$, where $u \in U$ and $w \in U^\perp$, and $Pv = u$, we conclude, that $P = P_U$. 154 | 155 | _Exercise 9_ 156 | 157 | Suppose $U$ is invariant under $T$. 158 | Let $v \in V$. 159 | Then, by part (b) in 6.55, $P_U v \in U$, which implies that $T P_U v \in U$ (since $U$ is invariant under $T$), which implies that $P_U T P_U v = T P_U v$. 160 | Thus $P_U T P_U = T P_U$. 161 | 162 | Conversely, suppose $P_U T P_U = T P_U$. 163 | Let $u \in U$. 164 | Then 165 | 166 | $$ 167 | Tu = T P_U u = P_U T P_U u \in U. 168 | $$ 169 | Therefore $U$ is invariant under $T$. 170 | 171 | _Exercise 10_ 172 | 173 | Suppose $U$ and $U^\perp$ are both invariant under $T$. 174 | Let $v \in V$. 175 | Then $v = u + w$ for some $u \in U$ and $w \in U^\perp$. 176 | Therefore, by parts (b) and (c) from 6.55, we have 177 | 178 | $$ 179 | \begin{aligned} 180 | P_U Tv &= P_U Tu + P_U Tw\\\\ 181 | &= P_U Tu\\\\ 182 | &= Tu\\\\ 183 | &= TP_U u\\\\ 184 | &= TP_U u + TP_U w\\\\ 185 | &= TP_U v. 186 | \end{aligned} 187 | $$ 188 | 189 | Hence $P_U T = TP_U$. 190 | 191 | Conversely, suppose $P_U T = TP_U$. 192 | Let $u \in U$. 193 | Then 194 | 195 | $$ 196 | Tu = TP_U u = P_U Tu \in U, 197 | $$ 198 | 199 | Hence $U$ is invariant under $T$. 200 | Let $w \in U^\perp$. 201 | Then 202 | 203 | $$ 204 | \begin{aligned} 205 | Tw &= Tw - TP_U w = Tw - P_U Tw = (I - P_U)Tw = P_{U^\perp} Tw \in U^{\perp} 206 | \end{aligned} 207 | $$ 208 | 209 | where the last equality follows from Exercise 5. 210 | Therefore $U^\perp$ is invariant under $T$. 211 | 212 | _Exercise 11_ 213 | 214 | Applying the Gram-Schmidt procedure to the basis $(1, 1, 0, 0), (1, 1, 1, 2)$ we get $\frac{1}{\sqrt{2}}(1, 1, 0, 0), \frac{1}{\sqrt{5}}(0, 0, 1, 2)$. 215 | Using the formula from part (i) in 6.55 to calculate $P_U (1, 2, 3, 4)$, we get 216 | 217 | $$ 218 | P_U (1, 2, 3, 4) = \frac{3}{2}(1, 1, 0, 0) + \frac{11}{5}(0, 0, 1, 2). 219 | $$ 220 | 221 | _Exercise 12_ 222 | 223 | Define $U$ by 224 | 225 | $$ 226 | U = \{p \in P_3(\mathbb{R}): p(0) = 0 \text{ and } p'(0)\}. 227 | $$ 228 | 229 | Note that $U$ is a subspace of $\mathcal{P}_3(\mathbb{R})$. 230 | Let $u \in U$. 231 | Since $p \in \mathcal{P}_3(\mathbb{R})$, we have 232 | 233 | $$ 234 | p(x) = ax^3 + bx^2 + cx + d 235 | $$ 236 | 237 | for some $a, b, c, d \in \mathbb{R}$. 238 | $p(0) = 0$ and $p'(0) = 0$ now imply that $c = d = 0$ and so $U \subset \operatorname{span}(x^2, x^3)$. 239 | Because $x^2, x^3 \in U$, it follows that $x^2, x^3$ is a basis of $U$. 240 | Applying the Gram_Schmidt Procedure to this basis, we get 241 | 242 | $$ 243 | \sqrt{5}x^2, 6\sqrt{7}x^3 - 5\sqrt{7}x^2. 244 | $$ 245 | 246 | Now, using the formula from part(i) in 6.55 to get $P_U(2 + 3x)$, we get 247 | 248 | $$ 249 | P_U(2 + 3x) = -\frac{203}{10}x^3 + 24x^2. 250 | $$ 251 | 252 | _Exercise 13_ 253 | 254 | In this Exercise, I used the SymPy Python package to calculate the integrals necessary for the Gram-Schmidt Procedure on the basis $1, x, x^2, x^3, x^4, x^5$ of $\mathcal{P}_5(\mathbb{R})$ and to compute the polynomial $p$ with the formula in 6.55 (i). 255 | I found the following 256 | 257 | $$ 258 | p(x) = x^{5} \left(- \frac{72765}{8 \pi^{8}} + \frac{693}{8 \pi^{6}} + \frac{654885}{8 \pi^{10}}\right) + x^{3} \left(- \frac{363825}{4 \pi^{8}} - \frac{315}{4 \pi^{4}} + \frac{39375}{4 \pi^{6}}\right) + x \left(- \frac{16065}{8 \pi^{4}} + \frac{105}{8 \pi^{2}} + \frac{155925}{8 \pi^{6}}\right), 259 | $$ 260 | 261 | which, if you assume $\pi = 3.14159265359$, is exactly what appers in 6.60. 262 | Here is the code used: 263 | 264 | ```python 265 | from sympy import * 266 | 267 | x = Symbol('x') 268 | 269 | def inner_product(f, g): 270 | return integrate(f*g, (x, -pi, pi)) 271 | 272 | def gs_procedure(basis): 273 | orthonormal_basis = [] 274 | 275 | for j, v_j in enumerate(basis): 276 | e_j = v_j - sum(map(lambda e_k: inner_product(v_j, e_k) * e_k, orthonormal_basis[:j - 1])) 277 | e_j = e_j / sqrt(inner_product(e_j, e_j)) 278 | orthonormal_basis.append(e_j) 279 | 280 | return orthonormal_basis 281 | 282 | def project(func, orthonormal_basis): 283 | return sum(map(lambda e_j: inner_product(func, e_j) * e_j, orthonormal_basis)) 284 | 285 | orthonormal_basis = gs_procedure([1, x, x**2, x**3, x**4, x**5]) 286 | 287 | approx_sin = project(sin(x), orthonormal_basis) 288 | 289 | p = collect(expand(approx_sin), x) 290 | print(latex(p)) 291 | ``` 292 | 293 | _Exercise 14_ 294 | 295 | _(a)_ 296 | 297 | Suppose there exists $v \neq 0: v \in U^\perp$. We can represent it as $v(x) = u(x) + v(0)$, where $u(x) = v(x) - v(0) \in U$, since $u(0) = 0$. 298 | 299 | Since $u \in U$, it follows that $\langle v, u \rangle = 0$: 300 | $$ 301 | \langle v, u \rangle = ||u||^2 + v(0)\langle u, 1 \rangle = 0 302 | $$ 303 | 304 | We have two options here: 305 | 306 | 1. Suppose $\langle u, 1 \rangle = 0$. This implies $||u||^2 = 0$, and this is possible only if $u(x) = 0$, so $v(x) = v(0) = const$, and it is easy to show that this restricts $v(0) = 0$, because $\langle v, x^2 \rangle = v(0)\frac{1}{3}x^3 \Big |_{-1}^1 = \frac{2}{3}v(0)$ should be equal 0 since $x^2 \in U$, which implies $v(x) = v(0) = 0$, and this contradicts our hypothesis that $v(x) \neq 0$. 307 | 2. $\langle u, 1 \rangle \neq 0$. Then, we can express $v(0)$: 308 | 309 | $$ 310 | v(0) = -\frac{||u||^2}{\langle u, 1 \rangle} 311 | $$ 312 | 313 | Consider 314 | 315 | $$ 316 | f_n(x) = 317 | \begin{cases} 318 | 1, |x| > \frac{1}{n}\\ 319 | n|x|, |x| \le \frac{1}{n} 320 | \end{cases} 321 | $$ 322 | 323 | It is clear that $f_n \in C_\mathbb{R}([-1,1])$, and 324 | 325 | $$ 326 | \langle v, f_n \rangle = \langle v, 1 \rangle - \int\limits_{-\frac{1}{n}}^\frac{1}{n}v(x)n|x|dx 327 | $$ 328 | 329 | Since $v$ is continuous, $\forall \varepsilon > 0 \ \exists \delta: |x| < \delta \implies |v(x) - v(0)| < \varepsilon$. 330 | 331 | So, for arbitrary $\varepsilon > 0$ we can find $N_\varepsilon = [\frac{1}{\delta}] + 1$, such that if $n > N_\varepsilon$, we get $|v(x) - v(0)| < \varepsilon$ when $x \in [-\frac{1}{n}, \frac{1}{n}]$. 332 | 333 | This means that when $n > N_\varepsilon$, we have 334 | 335 | $$ 336 | \int\limits_{-\frac{1}{n}}^\frac{1}{n}v(x)n|x|dx \le \int\limits_{-\frac{1}{n}}^\frac{1}{n}(v(0) + \varepsilon)n|x|dx = \frac{(v(0) + \varepsilon)}{n} 337 | $$ 338 | 339 | In the same fashion, 340 | 341 | $$ 342 | \int\limits_{-\frac{1}{n}}^\frac{1}{n}v(x)n|x|dx \ge \int\limits_{-\frac{1}{n}}^\frac{1}{n}(v(0) - \varepsilon)n|x|dx = \frac{(v(0) - \varepsilon)}{n} 343 | $$ 344 | 345 | This means, that $\operatorname{lim}\limits_{n \to \infty}\int\limits_{-\frac{1}{n}}^\frac{1}{n}v(x)n|x|dx = 0$ 346 | 347 | So, taking limit of $\langle f_n, v \rangle$, we get 348 | 349 | $$ 350 | 0 = \langle v, 1 \rangle 351 | $$ 352 | 353 | This implies, that $0 = \langle u + v(0), 1 \rangle = \langle u, 1 \rangle + 2v(0)$, so 354 | 355 | $$ 356 | \langle u, 1 \rangle = - 2v(0) 357 | $$ 358 | 359 | and so, 360 | 361 | $$ 362 | v(0) = -\frac{||u||^2}{\langle u, 1 \rangle} = \frac{||u||^2}{2v(0)} 363 | $$ 364 | 365 | Hence 366 | 367 | $$ 368 | v(0)^2 = \frac{1}{2}||u||^2 369 | $$ 370 | 371 | Now, consider $||v||^2$: 372 | 373 | $$ 374 | ||v||^2 = ||u||^2 + 2v(0)\langle u, 1 \rangle + 2v(0)^2 375 | $$ 376 | 377 | 378 | Substituting $v(0)$ and $v(0)^2$ we get 379 | 380 | $$ 381 | ||v||^2 = ||u||^2 -2\frac{||u||^2}{\langle u, 1 \rangle}\langle u, 1 \rangle + 2 \frac{1}{2}||u||^2 = 0 382 | $$ 383 | 384 | Which implies that $v = 0$, and this is a contradiction with $v \neq 0$. 385 | 386 | So, in both cases we ended up with a contradiction, so $U^\perp = \{0\}$. 387 | 388 | _(b)_ 389 | 390 | We can easily see, that $C_\mathbb{R}([-1, 1]) = U \oplus \{f(x) = c| c \in \mathbb{R}\}$, and so $C_\mathbb{R}([-1, 1]) \neq U \oplus U^\perp$. 391 | 392 | As for the second statement, it is clear that $(U^\perp)^\perp = \{0\}^\perp = C_\mathbb{R}([-1, 1]) \neq U$. 393 | -------------------------------------------------------------------------------- /Chapter 10 - Trace and Determinant/10A - Trace.md: -------------------------------------------------------------------------------- 1 | Chapter 10: **Trace and Determinant** 2 | 3 | **10.A** 4 | 5 | - [x] Exercise 1 6 | - [x] Exercise 2 7 | - [x] Exercise 3 8 | - [x] Exercise 4 9 | - [x] Exercise 5 10 | - [x] Exercise 6 11 | - [x] Exercise 7 12 | - [x] Exercise 8 13 | - [x] Exercise 9 14 | - [x] Exercise 10 15 | - [x] Exercise 11 16 | - [x] Exercise 12 17 | - [x] Exercise 13 18 | - [x] Exercise 14 19 | - [x] Exercise 15 20 | - [x] Exercise 16 21 | - [x] Exercise 17 22 | - [x] Exercise 18 23 | - [x] Exercise 19 24 | - [x] Exercise 20 25 | - [x] Exercise 21 26 | 27 | _Exercise 1_ 28 | 29 | Suppose $\mathcal{M}(T, (v_1, \dots, v_n))$ is invertible. 30 | Then there exists an $n$-by-$n$ matrix $A$ such that 31 | 32 | $$ 33 | I = \mathcal{M}(T, (v_1, \dots, v_n))A. 34 | $$ 35 | 36 | Define $S \in \mathcal{L}(V)$ such that 37 | 38 | $$ 39 | \mathcal{M}(S, (v_1, \dots, v_n)) = A. 40 | $$ 41 | 42 | Then 43 | 44 | $$ 45 | \mathcal{M}(ST, (v_1, \dots, v_n)) = \mathcal{M}(S, (v_1, \dots, v_n)) \mathcal{M}(T, (v_1, \dots, v_n)) = \mathcal{M}(I, (v_1, \dots, v_n)). 46 | $$ 47 | 48 | The equation above shows that $ST = I$. 49 | Exercise 10 in section 3D now implies that $T$ is invertible and $S = T^{-1}$. 50 | 51 | Conversely, suppose $T$ is invertible. 52 | Then 53 | 54 | $$ 55 | I = \mathcal{M}(T^{-1}T, (v_1, \dots, v_n)) = \mathcal{M}(T^{-1}, (v_1, \dots, v_n))\mathcal{M}(T, (v_1, \dots, v_n)) 56 | $$ 57 | 58 | and 59 | 60 | $$ 61 | I = \mathcal{M}(TT^{-1}, (v_1, \dots, v_n)) = \mathcal{M}(T^{-1}, (v_1, \dots, v_n))\mathcal{M}(T^{-1}, (v_1, \dots, v_n)), 62 | $$ 63 | 64 | which shows that $\mathcal{M}(T, (v_1, \dots, v_n))$ is invertible. 65 | 66 | _Exercise 2_ 67 | 68 | Suppose $A$ and $B$ are $n$-by-$n$ matrices. 69 | Let $V$ denote an $n$-dimensional vector space and choose a basis for $V$. 70 | Define $S, T \in \mathcal{L}(\mathbb{F}^n)$ such that $\mathcal{M}(S) = A$ and $\mathcal{M}(T) = B$, where this matrices are with respect to the chosen basis. 71 | We have 72 | 73 | $$ 74 | I = AB = \mathcal{M}(S)\mathcal{M}(T) = \mathcal{M}(ST). 75 | $$ 76 | 77 | Therefore $ST = I$. 78 | Exercise 10 in section 3D shows that $TS = I$. 79 | Thus 80 | 81 | $$ 82 | I = \mathcal{M}(TS) = \mathcal{M}(T)\mathcal{M}(S) = BA. 83 | $$ 84 | 85 | The equation above completes the proof. 86 | 87 | _Exericse 3_ 88 | 89 | Let $A$ denote the matrix of $T$, which is the same with respect to every basis. 90 | Let $v_1, \dots, v_n$ be basis of $V$. 91 | Then 92 | 93 | $$ 94 | Tv_k = A_{1,k}v_1 + \dots + A_{n, k}v_n. \tag{1} 95 | $$ 96 | 97 | for $k = 1, \dots, n$. 98 | We'll asumme $n \ge 2$ here (if $n = 1$ then every operator on $V$ already is a multiple of the identity). 99 | The list $v_1 - v_2, v_2, \dots, v_n$ is also a basis of $V$. 100 | The matrix of $T$ with respect to this basis is the same, therefore we have 101 | 102 | $$ 103 | T(v_1 - v_2) = A_{1,1}(v_1 - v_2) + A_{2,1}v_2 + \dots + A_{n,1}v_n = A_{1,1}v_1 + (A_{2,1} - A_{1,1})v_2 + \dots + A_{n,1}v_n. \tag{2} 104 | $$ 105 | 106 | Calculating $Tv_1 - Tv_2$ using $(1)$ we get 107 | 108 | $$ 109 | T(v_1 - v_2) = Tv_1 - Tv_2 = (A_{1,1} - A_{1,2})v_1 + (A_{2,1} - A_{2,2})v_1 + \dots + (A_{n, 1} - A_{n, 2})v_n. \tag{3} 110 | $$ 111 | 112 | Because $T(v_1 - v_2)$ can be uniquely written as combination of basis vectors, equating $(2)$ and $(3)$ shows that all entries in second column of $A$ equal $0$, except possibly $A_{2,2}$ which equals $A_{1,1}$. 113 | This same argument can be repeated with every pair of the basis vectors. 114 | Hence all nondiagonal entries of $A$ are $0$ and all diagonal entries are equal. 115 | Thus $A$ is a scalar multiple of the identity, which implies that so is $T$. 116 | 117 | _Exercise 4_ 118 | 119 | Let $A = \mathcal{M}(T, (v_1, \dots, v_n))$. 120 | Then 121 | 122 | $$ 123 | Iu_k = Tv_k = A_{1,k}v_1 + \dots + A_{n,k}v_n. 124 | $$ 125 | 126 | The equation above shows that $\mathcal{M}(I, (u_1, \dots, u_n), (v_1, \dots, v_n)) = A$, because each $Iu_k$ can be uniquely written as linear combination of the $v$'s and this combination forces the desired equality. 127 | 128 | _Exercise 5_ 129 | 130 | Suppose $B$ is an $n$-by-$n$ matrix. 131 | Let $V$ denote an $n$-by-$n$ dimensional vector space and let $v_1, \dots, v_n$ be a basis of $V$. 132 | Define $T \in \mathcal{L}(V)$ such that $\mathcal{M}(T, (v_1, \dots, v_n)) = B$. 133 | By 5.27, there is a basis $u_1, \dots, u_n$ of $V$ such that $\mathcal{M}(T, (u_1, \dots, u_n))$ is upper triangular. 134 | The equation written in 10.7 completes the proof. 135 | 136 | _Exercise 6_ 137 | 138 | Choose $V = \mathbb{R}^2$ and define $T \in \mathcal{L}(V)$ by 139 | 140 | $$ 141 | T(x, y) = (-y, x). 142 | $$ 143 | 144 | Then 145 | 146 | $$ 147 | T^2(x, y) = (-x, -y), 148 | $$ 149 | 150 | which shows that $-1$ is the only eigenvalue of $T^2$, with multiplicity $2$. 151 | Thus $\operatorname{trace}(T^2) = -2$. 152 | 153 | _Exercise 7_ 154 | 155 | It is easy to see that every eigenvalue of $T^2$ is nonnegative, by applying $T$ twice to each of basis vectors which are eigenvectors of $T$. 156 | 157 | _Exercise 8_ 158 | 159 | If $w = 0$, then $T = 0$ and $\operatorname{trace} T = 0$. 160 | If $w \neq 0$, then extend $w$ to a basis of $V$. 161 | All entries of the matrix of $T$ with respect to this basis are $0$, except possibly the first row, because $\operatorname{range} T \subset \operatorname{span}(w)$. 162 | Therefore, the trace of this matrix, which equals the trace of $T$, is $\langle w, v \rangle w$. 163 | 164 | _Exercise 9_ 165 | 166 | It is easy to check that $Pw = w$ for every $w \in \operatorname{range} P$. 167 | Let $u_1, \dots, u_n$ be a basis of $\operatorname{null} P$ and let $w_1, \dots, w_m$ be a basis of $\operatorname{range} P$. 168 | Exercise 4 in section 5B implies that joining these two basis we get a basis of $V$. 169 | Moreover this is a basis consisting of eigenvectors of $P$. 170 | Therefore the eigenvalues of $T$ are $0$, with multiplicity $n$, and $1$, with multiplicity $m$. 171 | Thus $\operatorname{trace} T = m$. 172 | 173 | _Exercise 10_ 174 | 175 | This follows directly from Exercise 2 in section 7A. 176 | 177 | _Exercise 11_ 178 | 179 | Because $T$ is self-adjoint, $V$ has a basis consisting of eigenvectors of $T$, by the Spectral Theorems (see 7.24 and 7.29). 180 | By 7.35 (b), all eigenvalues of $T$ are nonnegative. 181 | If their sum equals $0$, then they all equal $0$. 182 | Applying $T$ to each of the basis vectors (which are eigenvectors of $T$), we see that $T = 0$. 183 | 184 | _Exercise 12_ 185 | 186 | Every orthogonal projection is positive (see 7.32) and equal to its square (see 6.55). 187 | We have 188 | 189 | $$ 190 | (PQP)^* = P^*Q^*P\^* = PQP 191 | $$ 192 | 193 | and 194 | 195 | $$ 196 | \langle PQPv, v \rangle = \langle QPv, Pv \rangle \ge 0, 197 | $$ 198 | 199 | where the inequality above follows because $Q$ is positive. 200 | Hence $PQP$ is positive. 201 | Thus 202 | 203 | $$ 204 | \operatorname{trace}(PQ) = \operatorname{trace}(P^2 Q) = \operatorname{trace}(PQP) \ge 0. 205 | $$ 206 | 207 | _Exercise 13_ 208 | 209 | The trace of $T$ is 210 | 211 | $$ 212 | \operatorname{trace} T = 51 - 40 + 1 = 12. 213 | $$ 214 | 215 | If $\lambda$ is the third eigenvalue of $T$, then 216 | 217 | $$ 218 | -48 + 24 + \lambda = 12. 219 | $$ 220 | 221 | Thus $\lambda = 36$. 222 | 223 | _Exercise 14_ 224 | 225 | We have 226 | 227 | $$ 228 | \operatorname{trace} (cT) = \operatorname{trace}(\mathcal{M}(cT)) = \operatorname{trace}(c\mathcal{M}(T)) = c \operatorname{trace}(\mathcal{M}(T)) = c \operatorname{trace} (T). 229 | $$ 230 | 231 | _Exercise 15_ 232 | 233 | $$ 234 | \begin{aligned} 235 | \operatorname{trace} (ST) &= \operatorname{trace} (\mathcal{M}(ST))\\\\ 236 | &= \operatorname{trace} (\mathcal{M}(S)\mathcal{M}(T))\\\\ 237 | &= \operatorname{trace}(\mathcal{M}(T)\mathcal{M}(S))\\\\ 238 | &= \operatorname{trace}(\mathcal{M}(TS))\\\\ 239 | &= \operatorname{trace}(TS) 240 | \end{aligned} 241 | $$ 242 | 243 | _Exercise 16_ 244 | 245 | We give a counterexample. 246 | Define $T \in \mathcal{L}(\mathbb{R}^2)$ by 247 | 248 | $$ 249 | T(x, y) = (-y, x). 250 | $$ 251 | 252 | The matrix of $T$ with respect to standard basis of $\mathbb{R}^2$ is 253 | 254 | $$ 255 | \begin{pmatrix} 256 | 0 & -1\\\\ 257 | -1 & 0 258 | \end{pmatrix}. 259 | $$ 260 | 261 | Hence $\operatorname{trace} T = 0$. 262 | However, $T^2 = -I$, which implies that $\operatorname{trace} T^2 = -2$. 263 | 264 | _Exercise 17_ 265 | 266 | Let $u_1, \dots, u_m$ be a basis of $\operatorname{null} T$. 267 | Extend it to a basis $u_1, \dots, u_m, v_1, \dots, v_n$ of $V$. 268 | Then $Tv_1, \dots, Tv_n$ is a basis of $\operatorname{range} T$ (see the proof of 3.22). 269 | Define a linear map $S': \operatorname{range} T \to V$ by 270 | 271 | $$ 272 | S'(Tv_j) = v_j 273 | $$ 274 | 275 | for $j = 1, \dots, n$. 276 | We can extend $S'$ to an operator $S$ on $V$. 277 | Regardless of how we extend $S'$, it is clear that the matrix of $ST$ will be of the following form 278 | 279 | $$ 280 | \mathcal{M}(ST, (u_1, \dots, u_m, v_1, \dots, v_n)) = 281 | \begin{pmatrix} 282 | 0 & & & &\\\\ 283 | & \ddots & & &\\\\ 284 | & & 0 & & &\\\\ 285 | & & & 1 & &\\\\ 286 | & & & & \ddots &\\\\ 287 | & & & & & 1 288 | \end{pmatrix}, 289 | $$ 290 | 291 | where the $1$'s appear on the last $n$ diagonal entries. 292 | However, the trace of this matrix should equal $0$ (since $\operatorname{trace} (ST) = 0$). 293 | Hence $n = 0$, because the $1$'s should not appear in any of the columns. 294 | Therefore $T = 0$. 295 | 296 | _Exercise 18_ 297 | 298 | Note that $||Te_j||^2$ is the sum of the the squares of the absolute values of the entries in the $j$-th column of the matrix of $T$ with respect to the basis $e_1, \dots, e_n$. 299 | Thus, we're essentially being asked to prove that $\operatorname{trace}(T^*T)$ equals the sum of the squares of the absolute values of the entries in the matrix of $T$ with respect to any orthonormal basis. 300 | We have 301 | 302 | $$ 303 | \operatorname{trace}(T^*T) = \operatorname{trace}(\mathcal{M}(T^*T)) = \operatorname{trace}(\mathcal{M}(T^*)\mathcal{M}(T)), 304 | $$ 305 | 306 | where these matrices are taken with respect to $e_1, \dots, e_n$. 307 | The equation above gives the desired result, because $\mathcal{M}(T^*)$ is the conjugate transpose of $\mathcal{M}(T)$ and so the $j$-th diagonal entry of $\mathcal{M}(T^*)\mathcal{M}(T)$ equals the sum of the squares of the absolute values of the entries in the $j$-th column of $\mathcal{M}(T)$. 308 | 309 | _Exercise 19_ 310 | 311 | The function $\langle \cdot, \cdot \rangle$ satisfies additivity and homogeneity in the first slot by 10.18 and Exercise 14. 312 | For conjugate symmetry we have 313 | 314 | $$ 315 | \langle S, T \rangle = \operatorname{trace}(ST^*) = \operatorname{trace}((TS^*)^*) = \overline{\operatorname{trace}(TS^*)} = \overline{\langle T, S \rangle}, 316 | $$ 317 | 318 | where the third equality follows from Exercise 10. 319 | By Exercise 18, it satisfies positive-definiteness. 320 | 321 | _Exercise 20_ 322 | 323 | As explained in the solution to Exercise 18, the right side of the inequality is equal to $\operatorname{trace} (T^*T)$. 324 | Let $f_1, \dots, f_n$ be an orthonormal basis of $V$ with respect to which the matrix of $T$ is upper triangular (6.37 assures the existence of this basis). 325 | Then the eigenvalues of $T$ appear on the diagonal of this matrix. 326 | Because $\mathcal{M}(T^*)$ is the conjugate transpose of $\mathcal{M}(T)$, where this matrices are with respect to the basis $f_1, \dots, f_n$, a moment's thought shows that 327 | 328 | $$ 329 | |\lambda_1|^2 + \dots + |\lambda_n|^2 \le \operatorname{trace}(\mathcal{M}(T^*)\mathcal{M}(T)). 330 | $$ 331 | 332 | The right side of the inequality above equals $\operatorname{trace}(T^*T)$, which yields the desired result. 333 | 334 | _Exercise 21_ 335 | 336 | We already have proven, that if $e_1,..,e_n$ is an othonormal basis, then $\operatorname{trace}(T^*T) = ||Te_1||^2 + ... + ||Te_n||^2$. 337 | 338 | Since $Te_i = A_{1i}e_1 + ... + A_{ni}e_n$, we have that $||Te_i||^2 = |A_{1i}|^2 + ... + |A_{ni}|^2$, and hence 339 | 340 | $$ 341 | \operatorname{trace}(T^*T) = \sum\limits_{k=1}^n\sum\limits_{j=1}^n|A_{jk}|^2 342 | $$ 343 | 344 | On the other hand, consider a basis $v_1,..,v_n$ of $T$ where it has upper-triangular matrix, which exists for any operator over $\mathbb{C}$. 345 | 346 | $$ 347 | \mathcal{M}(T, (v_1, \dots, v_n)) = 348 | \begin{pmatrix} 349 | a_{11} & a_{12} & & &\\\\ 350 | 0 & a_{22} & \ddots & &\\\\ 351 | & & ... & & \\\\ 352 | & & & & a_{nn}\\\\ 353 | \end{pmatrix} 354 | $$ 355 | 356 | 357 | It is easy to see, that in this basis $T^*T$ has matrix such that $\mathcal{M}(T^*T, (v_1,..,v_n))_{i,i} = |a_{1i}|^2 + ... + |a_{i,i}|^2 \ge |a_{ii}|^2$, with $a_{11}...a_{nn}$ being the list of eigenvalues of $T$. 358 | 359 | Since the trace of an operator is the trace of its matrix in any basis, we conclude that 360 | 361 | $$ 362 | \operatorname{trace}(T^*T) = \sum\limits_{k=1}^n\sum\limits_{j=1}^n|A_{jk}|^2 = |\lambda_1|^2 + ... + |\lambda_n|^2 + c 363 | $$ 364 | 365 | where $c = \sum\limits_{i=1}^n\sum\limits_{j=1}^i|a_{ij}|^2 \ge 0$, so it is clear that 366 | 367 | $$ 368 | |\lambda_1|^2 + ... |\lambda_n|^2 \le \sum\limits_{k=1}^n\sum\limits_{j=1}^n|A_{jk}|^2 369 | $$ 370 | 371 | _Exercise 21_ 372 | 373 | Let $e_1, \dots, e_n$ be an orthonormal basis of $V$. 374 | By Exercise 18, we have 375 | 376 | $$ 377 | \begin{aligned} 378 | \operatorname{trace}(T^*T) &= ||Te_1||^2 + \dots + ||Te_n||^2\\\\ 379 | &\ge ||T^*e_1||^2 + \dots + ||T^*e_n||^2\\\\ 380 | &= \operatorname{trace}(TT^*)\\\\ 381 | &= \operatorname{trace}(T^*T). 382 | \end{aligned} 383 | $$ 384 | 385 | Because the first and last lines are equal, we must have equality throughout. 386 | Thus 387 | 388 | $$ 389 | ||Te_1||^2 + \dots + ||Te_n||^2 = ||T^*e_1||^2 + \dots + ||T^*e_n||^2. 390 | $$ 391 | 392 | Since $||Te_j|| \ge ||T^*e_j||$, the equation above actually implies that $||Te_j|| = ||T^*e_j||$. 393 | Note that the basis $e_1, \dots, e_n$ was arbitrary and for any nonzero vector $v \in V$, we can extend $\frac{v}{||v||}$ to an orthonormal basis of $V$. 394 | Thus 395 | 396 | $$ 397 | \left|\left|T\left(\frac{v}{||v||}\right)\right|\right| = \left|\left|T^*\left(\frac{v}{||v||}\right)\right|\right| 398 | $$ 399 | 400 | for all nonzero $v \in V$. 401 | Multiplying the equation above by $||v||$ we get that $||Tv|| = ||T^*v||$. 402 | Then $T$ is normal by 7.20. 403 | -------------------------------------------------------------------------------- /Chapter 05 - Eigenvalues, Eigenvectors, and Invariant Subspaces/5C - Eigenspaces and Diagonal Matrices.md: -------------------------------------------------------------------------------- 1 | Chapter 5: **Eigenvalues, Eigenvectors and Invariant Subspaces** 2 | 3 | **5.C** 4 | 5 | - [x] Exercise 1 6 | - [x] Exercise 2 7 | - [x] Exercise 3 8 | - [x] Exercise 4 9 | - [x] Exercise 5 10 | - [x] Exercise 6 11 | - [x] Exercise 7 12 | - [x] Exercise 8 13 | - [x] Exercise 9 14 | - [x] Exercise 10 15 | - [x] Exercise 11 16 | - [x] Exercise 12 17 | - [x] Exercise 13 18 | - [x] Exercise 14 19 | - [x] Exercise 15 20 | - [x] Exercise 16 21 | 22 | _Exercise 1_ 23 | 24 | If $\operatorname{null} T = \{0\}$ (because it implies surjectivity) or $\operatorname{range} T = \{0\}$ the result is obvious. 25 | Assume both contain non-zero vectors. 26 | 27 | Since $\operatorname{null} T \neq \{0\}$, we have that $0$ is an eigenvalue of $T$ and that $E(0, T) = \operatorname{null} T$. 28 | Let $\lambda_1, \dots, \lambda_m$ denote the other distinct eigenvalues of $T$. 29 | Now 5.41 implies that $V = \operatorname{null} T \oplus E(\lambda_1, T) \oplus \dots \oplus E(\lambda_m, T)$. 30 | We will prove that $\operatorname{range} T = E(\lambda_1, T) \oplus \dots \oplus E(\lambda_m, T)$. 31 | 32 | Suppose $v \in E(\lambda_1, T) \oplus \dots \oplus E(\lambda_m, T)$. 33 | Then $v = v_1 + \dots + v_m$ for some $v_1, \dots, v_m$, where each $v_j \in E(\lambda_j, T)$. 34 | Moreover, $v = T(\frac{1}{\lambda_1}v_1) + \dots + T(\frac{1}{\lambda_m}v_m)$, which implies that $v \in \operatorname{range}(T)$. 35 | Hence 36 | 37 | $$E(\lambda_1, T) \oplus \dots \oplus E(\lambda_m, T) \subset \operatorname{range} T.$$ 38 | 39 | For the inclusion in the other direction, suppose $v \in \operatorname{range} T$. 40 | Note that $\operatorname{range} T$ stays the same when we restrict $T$ to $E(\lambda_1, T) \oplus \dots \oplus E(\lambda_m, T)$. 41 | Then $v = T(v_1 + \dots + v_m)$ for some $v_1, \dots, v_m$, where each $v_j \in E(\lambda_j, T)$. 42 | Therefore $v = \lambda_1 v_1 + \dots + \lambda_m v_m$ and, because $\lambda_j v_j \in E(\lambda_j, T)$ for each $j$, this proves that $v \in E(\lambda_1, T) \oplus \dots \oplus E(\lambda_m, T)$. 43 | Thus $\operatorname{range} T \subset E(\lambda_1, T) \oplus \dots \oplus E(\lambda_m, T)$, completing the proof. 44 | 45 | _Exercise 2_ 46 | 47 | We give a counterexample. 48 | Let $V = \mathbb{R}^2$ and $T(x, y) = (-y, x)$. 49 | $T$ is invertible, thus $V = \operatorname{null} T \oplus \operatorname{range} T$. 50 | But, as shown in Example 5.8, $T$ has no eigenvalues, therefore $T$ is not diagonalizable. 51 | 52 | Note: If we let $V$ be infinite-dimensional, then any invertible operator $T \in \mathcal{L}(V)$ will prove it to be wrong, because we will have $V = \operatorname{null} T \oplus \operatorname{range} T$, but $T$ being diagonalizable requires $V$ to be finite-dimensional. 53 | 54 | _Exercise 3_ 55 | 56 | Obviously (a) implies (b) and, by 1.45, it also implies (c). 57 | 58 | Suppose (b) holds. 59 | Combining 3.22 and 2.43, we have 60 | 61 | $$ 62 | \begin{aligned} 63 | \operatorname{dim} \operatorname{null} T + \operatorname{dim} \operatorname{range} T &= \operatorname{dim} V\\\\ 64 | &= \operatorname{dim} (\operatorname{null} T + \operatorname{range} T)\\\\ 65 | &= \operatorname{dim} \operatorname{null} T + \operatorname{dim} \operatorname{range} T - \operatorname{dim}(\operatorname{null} T \cap \operatorname{range} T)\\\\ 66 | \end{aligned} 67 | $$ 68 | 69 | Therefore $\operatorname{dim}(\operatorname{null} T \cap \operatorname{range} T) = 0$, implying (c) is true. 70 | 71 | Suppose (c) holds. 72 | By 1.45, $\operatorname{null} T + \operatorname{range} T$ is a direct sum. 73 | Then, by 2.43 and 3.22, we have 74 | 75 | $$ 76 | \operatorname{dim} (\operatorname{null} T \oplus \operatorname{range} T) = \operatorname{dim} \operatorname{null} T + \operatorname{dim} \operatorname{range} T = \operatorname{dim} V 77 | $$ 78 | 79 | Since $\operatorname{null} T \oplus \operatorname{range} T$ is a subspace of $V$, it follows that $\operatorname{null} T \oplus \operatorname{range} T = V$, implying (a) and completing the proof. 80 | 81 | _Exercise 4_ 82 | 83 | Let $T \in \mathcal{L}(\mathbb{F}^\infty)$ be the forward shift operator. 84 | 85 | Then $\operatorname{null} T = \{0\}$, which implies (c). 86 | However $T$ is not surjective, therefore (b) is false. 87 | 88 | _Exercise 5_ 89 | 90 | Suppose $T$ is diagonalizable. 91 | It is easy to see that, for every $\lambda \in \mathbb{C}$, $T - \lambda I$ is also diagonalizable, because $\mathcal{M}(T - \lambda I)$ equals $\mathcal{M}(T)$ with the entries on the diagonal subtracted by $\lambda$. 92 | By _Exercise 1_, this implies that 93 | 94 | $$ 95 | V = \operatorname{null} (T - \lambda I) \oplus \operatorname{range} (T - \lambda I). \tag{1} 96 | $$ 97 | 98 | Conversely, suppose (1) holds. 99 | Denote $\lambda_1,...,\lambda_m$ to be all distinct eigenvalues of $T$. 100 | 101 | We first prove that $\operatorname{null}(T-\lambda_iI) \subset \operatorname{range}(T-\lambda_jI)$, if $i \neq j$. Suppose $v \in \operatorname{null}(T-\lambda_iI)$. We know, that $V = \operatorname{null}(T - \lambda_jI)\oplus \operatorname{range}(T - \lambda_jI)$, so 102 | 103 | $$ 104 | v = u + w\\ 105 | u \in \operatorname{null}(T-\lambda_jI)\\ 106 | w \in \operatorname{range}(T-\lambda_jI) 107 | $$ 108 | 109 | Now, $Tv = \lambda_i v = \lambda_i(u + w) = \lambda_ju + Tw$, hence $(T - \lambda_iI)w = (\lambda_i - \lambda_j)u$, so $(T - \lambda_iI)w \in \operatorname{range}(T - \lambda_iI)$ and $w \in \operatorname{null}(T - \lambda_iI)$, which implies that $w = 0$, because from the definition of direct sum we know, that $\operatorname{null}(T - \lambda_iI) \cap \operatorname{range}(T - \lambda_iI) = \{0\}$. 110 | 111 | We will now prove that $V = \bigcup\limits_{i}\operatorname{range}(T - \lambda_iI)$. Suppose this is not true, so there is $v \notin \operatorname{null}(T-\lambda_iI),\ i = 1...m$. This implies $v \neq 0$, and that $v \in \operatorname{range}(T - \lambda_iI),\ i=1...m$, and so $v \in \bigcap\limits_{i}\operatorname{range}(T - \lambda_iI)$. We can note, that for any $i$ $\operatorname{range}(T - \lambda_iI)$ is invariant under $T$, because if $v \in \operatorname{range}(T - \lambda_iI)$, $\exists w: v = Tw - \lambda_iw$, hence $Tv = T(-\lambda_i)Tw \in \operatorname{range}(T - \lambda_iI)$. So, $U = \bigcap\limits_i\operatorname{range}(T - \lambda_iI)$ is also invariant under $T$. This implies, $T_{|U}$ is an operator on $U$, and since $U \neq \{0\}$ and is finite-dimensional, $T_{|U}$ has an eigenvalue $\lambda$, and this is also an eigenvalue of $T$. But this implies that $\lambda = \lambda_i$, and so the corresponding eigenvector belongs both to $\operatorname{null}(T - \lambda_iI)$ and to $\operatorname{range}(T - \lambda_iI)$, which contradicts with the fact, that $V$ is a direct sum of these two subspaces. 112 | 113 | So, $V = \bigcup\limits_{i}\operatorname{range}(T - \lambda_iI)$. It is clear, that $\bigcup\limits_{i}\operatorname{range}(T - \lambda_iI) \subset \operatorname{range}(T - \lambda_1I) + ... + \operatorname{range}(T - \lambda_mI)$, and the opposite inclusion is correct as well, since $\operatorname{range}(T - \lambda_1I) + ... + \operatorname{range}(T - \lambda_mI) \subset V$. Finally, we have proven, that $\operatorname{range}(T - \lambda_iI) \cap \operatorname{range}(T - \lambda_jI) = \{0\}$, which completes the proof. 114 | 115 | _Exercise 6_ 116 | 117 | Let $\lambda_1, \dots, \lambda_m$ denote the distinct eigenvalues of $T$, $v_1, \dots, v_n$ corresponding eigenvectors and $\alpha_1, \dots, \alpha_n$ the corresponding eigenvalues of $S$. 118 | Then 119 | 120 | $$ 121 | \begin{aligned} 122 | STv_j &= S(\lambda_j v_j)\\\\ 123 | &= \alpha_j \lambda_j v_j\\\\ 124 | &= \alpha_j Tv_j\\\\ 125 | &= T(\alpha_j Tv_j)\\\\ 126 | &= TSv_j\\\\ 127 | \end{aligned} 128 | $$ 129 | 130 | for $j = 1, \dots, n$. 131 | Thus, by 3.5, $ST = TS$. 132 | 133 | _Exercise 7_ 134 | 135 | Suppose $\lambda$ appears $n$ times on the diagonal of $A$. 136 | Let $v_1, \dots, v_n, u_1, \dots, u_m$ be a basis composed of the same basis vectors chosen to represent $A$ (possibly in different order) such that the $v$'s are eigenvectors of $\lambda$ and the remaining are corresponding eigenvectors to $\lambda_1, \dots, \lambda_m$. 137 | Note that $\lambda_k \neq \lambda$ for each $k$, because we have $n$ $v$'s. 138 | 139 | Let $v \in E(\lambda, T)$. 140 | There are $a_1, \dots, a_n, c_1, \dots, c_m \in \mathbb{F}$ such that 141 | 142 | $$ 143 | v = a_1 v_1 + \dots + a_n v_n + c_1 u_1 + \dots + c_m u_m 144 | $$ 145 | 146 | Then 147 | 148 | $$ 149 | \begin{aligned} 150 | 0 &= (T - \lambda I)v\\\\ 151 | &= \sum\limits_{k=1}^n (\lambda - \lambda) a_1 v + \sum\limits_{k=1}^m (\lambda_k - \lambda) c_k u_k\\\\ 152 | &= \sum\limits_{k=1}^m (\lambda_k - \lambda) c_k u_k\\\\ 153 | \end{aligned} 154 | $$ 155 | 156 | Since $\lambda_k - \lambda \neq 0$, all the $c$'s are 0, implying that $v \in \operatorname{span}(v_1, \dots, v_n)$. 157 | Hence $E(\lambda, T) \subset \operatorname{span}(v_1, \dots, v_n)$. 158 | The inclusion in the other directions is obvious, thus it becomes an equality. 159 | Moreover, $v_1, \dots, v_n$ is linearly independent and, hence, a basis of $E(\lambda, T)$. 160 | Thus $\operatorname{dim} E(\lambda, T) = n$. 161 | 162 | _Exercise 8_ 163 | 164 | Assume $T$ has other eigenvalues besides $8$ (otherwise there is nothing to prove). 165 | Then, since $\operatorname{dim} E(8, T) + 1 = \operatorname{dim} F^5$ and by part (d) of 5.41, $T$ has only one eigenvalue besides $8$. 166 | Therefore at least between $T - 2I$ and $T - 6I$ is invertible. 167 | 168 | _Exercise 9_ 169 | 170 | $$ 171 | \begin{aligned} 172 | v \in E(\lambda, T) &\iff Tv = \lambda v\\\\ 173 | &\iff T^{-1}(\lambda v) = v\\\\ 174 | &\iff T^{-1} v = \frac{1}{\lambda} v\\\\ 175 | &\iff v \in E(\frac{1}{\lambda}, T^{-1})\\\\ 176 | \end{aligned} 177 | $$ 178 | 179 | _Exercise 10_ 180 | 181 | If $0$ is an eigenvalue of $T$, then 182 | 183 | $$ 184 | \begin{aligned} 185 | \operatorname{dim} \operatorname{range} T &= \operatorname{dim} V - \operatorname{dim} \operatorname{null} T\\\\ 186 | &= \operatorname{dim} V - \operatorname{dim} E(0, T)\\\\ 187 | &\ge \operatorname{dim}(E(0, T) \oplus E(\lambda_1, T) \oplus \dots \oplus E(\lambda_m, T)) - \operatorname{dim} E(0, T)\\\\ 188 | &= \operatorname{dim} E(0, T) + \operatorname{dim} E(\lambda_1, T) + \dots + \operatorname{dim} E(\lambda_m, T) - \operatorname{dim} E(0, T)\\\\ 189 | &= \operatorname{dim} E(\lambda_1, T) + \dots + \operatorname{dim} E(\lambda_m, T)\\\\ 190 | \end{aligned} 191 | $$ 192 | 193 | Where the third line follows from 5.38. 194 | 195 | If $0$ is not an eigenvalue of $T$, just disregard $E(0, T)$ and $\operatorname{dim} E(0, T)$ in each line. 196 | 197 | _Exercise 11_ 198 | 199 | For the first column we have 200 | 201 | $$ 202 | T(1, 4) = (41 + 28, -20 + 296) = (69, 276) = \mathbf{69}(1, 4) + \mathbf{0}(7,5) 203 | $$ 204 | 205 | and the second 206 | 207 | $$ 208 | T(7, 5) = (287 + 35, - 140 + 370) = (322, 230) = \mathbf{0}(1, 4) + \mathbf{46}(7, 5) 209 | $$ 210 | 211 | _Exercise 12_ 212 | 213 | Let $v_1, v_2, v_3$ and $w_1, w_2, w_3$ be eigenvectors of $R$ and $T$, respectively, corresponding to $2, 6, 7$. 214 | Note that the $v$'s and $w$'s are both bases of $\mathbb{F}^3$. 215 | Define $S \in \mathcal{L}(\mathbb{F}^3)$ by 216 | 217 | $$ 218 | \begin{aligned} 219 | Sv_1 &= w_1\\\\ 220 | Sv_2 &= w_2\\\\ 221 | Sv_3 &= w_3\\\\ 222 | \end{aligned} 223 | $$ 224 | 225 | It is easy to see that $S$ is invertible. 226 | Thus 227 | 228 | $$ 229 | \begin{aligned} 230 | S^{-1}TS(a_1 v_1 + a_2 v_2 + a_3 v_3) &= S^{-1}T(a_1 w_1 + a_2 w_2 + a_3 w_3)\\\\ 231 | &= S^{-1}(2 a_1 w_1 + 6 a_2 w_2 + 7 a_3 w_3)\\\\ 232 | &= 2 a_1 v_1 + 6 a_2 v_2 + 7 a_3 v_3\\\\ 233 | &= R(a_1 v_1 + a_2 v_2 + a_3 v_3)\\\\ 234 | \end{aligned} 235 | $$ 236 | 237 | Therefore $R = S^{-1}TS$. 238 | 239 | _Exercise 13_ 240 | 241 | Suppose $R$ and $T$ are diagonalizable, and $R = \operatorname{diag}(2,6,7,7)$, $T = \operatorname{diag}(2,2,6,7)$ for natural basises. This means that $\operatorname{dim}E(7, T) = 1$ and $\operatorname{dim}E(7, R) = 2$. Denote $e_1,...,e_4$ as basis vectors. Suppose desired invertible $S$ exists, then 242 | 243 | $Re_3 = 7e_3 = S^{-1}TSe_3$, and so $7Se_3 = TSe_3$, which means that $Se_3 \in E(7, T)$. By the same reasoning, $Se_4 \in E(7, T)$. Since $e_3,e_4$ are linearly independent and $S$ is injective, $Se_3, Se_4$ are also linearly independent, which implies that $\operatorname{dim}(E(7, T)) \ge 2$, which is a contradiction. 244 | 245 | _Exercise 14_ 246 | 247 | Consider operator with the following matrix in standard basis 248 | 249 | $$ 250 | A = \begin{pmatrix} 251 | 6 & 0 & 0 \\ 252 | 0 & 7 & 1 \\ 253 | 0 & 0 & 7 254 | \end{pmatrix} 255 | $$ 256 | 257 | Since this is an upper triangular matrix, it is clear that 6 and 7 are the only eigenvalues of $T$. We see that $\operatorname{dim}\operatorname{null}(T - 6I) = 1$, because $u = (T - 6I)e_2 = e_2, v = (T - 6I)e_3 = e_2 + e_3$, and $v,u\in \operatorname{range}(T-6I)$ and $v,u$ are linearly independent, so $\operatorname{dim}\operatorname{range}(T-6I) \ge 2$, so by the fundamental theorem of linear maps $\operatorname{dim}\operatorname{null}(T-6I) \le 1$. 258 | 259 | By the same reasoning, $\operatorname{dim}\operatorname{null}(T-6I) \le 1$. If there were 3 linearly independent eigenvectors of $T$, at least two of them would correspond to the same eigenvalue, but we see that for both eigenspaces this leads to a contradiction. 260 | 261 | _Exercise 15_ 262 | 263 | We are looking for a vector $v: Tv = 8v + w$, where $w = (17, \sqrt 5, w\pi)$. This implies $(T - 8I)v = w$. We know, that 8 is not an eigenvalue of $T$, therefore $T - 8I$ is injective and hence surjective, so, $\forall w \in V \exists v: (T - 8I)v = w$, which completes the proof. 264 | 265 | _Exercise 16_ 266 | 267 | _(a)_ 268 | 269 | We will prove by induction on $n$. When $n = 1$, the result is clearly true. 270 | 271 | Assume it is true for $n$. 272 | Then 273 | 274 | $$ 275 | \begin{aligned} 276 | T^{n+1} (0, 1) &= TT^n (0, 1)\\\\ 277 | &= T(F_n, F_{n+1})\\\\ 278 | &= (F_{n+1}, F_n + F_{n+1})\\\\ 279 | &= (F_{n+1}, F_{n+2})\\\\ 280 | \end{aligned} 281 | $$ 282 | 283 | _(b)_ 284 | 285 | We need to find $(x, y)$ and $\lambda \in \mathbb{R}$ that satisfy $T(x, y) = \lambda(x, y)$. 286 | Thus need to solve the following set of equations 287 | 288 | $$ 289 | \begin{aligned} 290 | \lambda x &= y\\\\ 291 | \lambda y &= x + y\\\\ 292 | \end{aligned} 293 | $$ 294 | 295 | Substituting $y$ in the second equation and dividing both sides by $\lambda$, we get $y = \frac{1 + \lambda}{\lambda} x$. 296 | Therefore, an eigenvector of $T$ has the form $\left(x, \frac{1 + \lambda}{\lambda} x\right)$ and we have 297 | 298 | $$ 299 | T \left(x, \frac{1 + \lambda}{\lambda} x\right) = \left(\frac{1 + \lambda}{\lambda} x, \frac{1 + 2\lambda}{\lambda} x\right) 300 | $$ 301 | 302 | We need then, to solve for $\lambda$ in the following equations 303 | 304 | $$ 305 | \frac{1 + \lambda}{\lambda} = \lambda, \frac{1 + 2\lambda}{\lambda} = 1 + \lambda 306 | $$ 307 | 308 | You can check that both are equivalent and have the solutions $\frac{1 + \sqrt{5}}{2}$ and $\frac{1 - \sqrt{5}}{2}$, which we will denote $\varphi$ and $\hat{\varphi}$. 309 | 310 | _(c)_ 311 | 312 | Substituting the eigenvalues found in (b) on the eigenvector form $\left(x, \frac{1 + \lambda}{\lambda} x\right)$, we get 313 | 314 | $$ 315 | (1, \varphi) \text{ and } (1, \hat{\varphi}). 316 | $$ 317 | 318 | Both are linearly independent, since they are eigenvectors corresponding to distinct eigenvalues. 319 | Thus, they are indeed a basis of $\mathbb{R}^2$. 320 | 321 | _(d)_ 322 | 323 | Here we use the fact that if $\lambda$ is an eigenvalue of $T$ and $v$ a corresponding eigenvector then $T^k v = \lambda^k v$. 324 | Since $(0, 1) = \frac{1}{\sqrt{5}} (0, \varphi - \hat{\varphi})$, we have 325 | 326 | $$ 327 | \begin{aligned} 328 | T^n (0, 1) &= \frac{1}{\sqrt{5}} T^n (0, \varphi - \hat{\varphi})\\\\ 329 | &= \frac{1}{\sqrt{5}} T^n ((1, \varphi) - (1, \hat{\varphi}))\\\\ 330 | &= \frac{1}{\sqrt{5}} \left[T^n (1, \varphi) - T^n (1, \hat{\varphi})\right]\\\\ 331 | &= \frac{1}{\sqrt{5}} \left[\varphi^n (1, \varphi) - \hat{\varphi}^n (1, \hat{\varphi})\right]\\\\ 332 | &= \frac{1}{\sqrt{5}} (\varphi^n - \hat{\varphi}^n, \varphi^{n+1} - \hat{\varphi}^{n+1})\\\\ 333 | \end{aligned} 334 | $$ 335 | 336 | Therefore 337 | 338 | $$ 339 | F^n = \frac{1}{\sqrt{5}} (\varphi^n - \hat{\varphi}^n) = \frac{1}{\sqrt{5}} \left[ \left(\frac{1 + \sqrt{5}}{2}\right)^n - \left(\frac{1 - \sqrt{5}}{2}\right)^n \right] 340 | $$ 341 | 342 | _(e)_ 343 | 344 | This follows directly from the fact that $\frac{1}{\sqrt{5}} \left|\frac{1 - \sqrt{5}}{2}\right|^n < \frac{1}{2}$, for all positive integers $n$, which we will prove by induction. 345 | 346 | When $n = 1$, we have 347 | 348 | $$ 349 | \begin{aligned} 350 | \frac{1}{\sqrt{5}} \left| \frac{1 - \sqrt{5}}{2} \right| &= \frac{1}{\sqrt{5}} \frac{\sqrt{5} - 1}{2}\\\\ 351 | &= \frac{5 - \sqrt{5}}{10}\\\\ 352 | &< \frac{5}{10}\\\\ 353 | &= \frac{1}{2}\\\\ 354 | \end{aligned} 355 | $$ 356 | 357 | Assume the result holds true for $n$. 358 | Then 359 | 360 | $$ 361 | \begin{aligned} 362 | \frac{1}{\sqrt{5}} \left| \frac{1 - \sqrt{5}}{2} \right|^{n + 1} &= 363 | \left| \frac{1 - \sqrt{5}}{2} \right| 364 | \frac{1}{\sqrt{5}} 365 | \left| \frac{1 - \sqrt{5}}{2} \right|^n 366 | \\\\ 367 | &< \left| \frac{1 - \sqrt{5}}{2} \right| \frac{1}{2}\\\\ 368 | &= \frac{\sqrt{5} - 1}{2} \cdot \frac{1}{2}\\\\ 369 | &< \frac{\sqrt{9} - 1}{2} \cdot \frac{1}{2}\\\\ 370 | &= \frac{1}{2}\\\\ 371 | \end{aligned} 372 | $$ 373 | -------------------------------------------------------------------------------- /Chapter 06 - Inner Product Spaces/6A - Inner Products and Norms.md: -------------------------------------------------------------------------------- 1 | Chapter 6: **Inner Product Spaces** 2 | 3 | 4 | **6.A** 5 | 6 | - [x] Exercise 1 7 | - [x] Exercise 2 8 | - [x] Exercise 3 9 | - [x] Exercise 4 10 | - [x] Exercise 5 11 | - [x] Exercise 6 12 | - [x] Exercise 7 13 | - [x] Exercise 8 14 | - [x] Exercise 9 15 | - [x] Exercise 10 16 | - [x] Exercise 11 17 | - [x] Exercise 12 18 | - [x] Exercise 13 19 | - [x] Exercise 14 20 | - [x] Exercise 15 21 | - [x] Exercise 16 22 | - [x] Exercise 17 23 | - [x] Exercise 18 24 | - [x] Exercise 19 25 | - [x] Exercise 20 26 | - [x] Exercise 21 27 | - [x] Exercise 22 28 | - [x] Exercise 23 29 | - [x] Exercise 24 30 | - [x] Exercise 25 31 | - [x] Exercise 26 32 | - [ ] Exercise 27 33 | - [ ] Exercise 28 34 | - [ ] Exercise 29 35 | - [ ] Exercise 30 36 | - [ ] Exercise 31 37 | 38 | _Exercise 1_ 39 | 40 | It does not satisfy homogeneity on the first slot when $\lambda$ is negative. 41 | 42 | _Exercise 2_ 43 | 44 | It does not satisfy the definiteness property if we let the first and third slots equal zero but the second non-zero. 45 | 46 | _Exercise 3_ 47 | 48 | Let $v \in V: \langle v,v\rangle > 0$. Suppose $\exists u \in V: \langle u, u \rangle < 0$. It is clear, that $u,v$ are linearly independent: suppose they are not. Then $u = \lambda v$ for some $\lambda \in \mathbb{F}$, which implies that $\langle u, u \rangle = \lambda^2\langle v, v \rangle \ge 0$, and this is a contradiction. 49 | 50 | Consider vectors of the form $u + \lambda v$ for $\lambda \in \mathbb{F}$. Let us look for such $\lambda$ that $\langle (u + \lambda v), (u + \lambda v) \rangle = 0$. To find such $\lambda$, we need to solve 51 | 52 | $$ 53 | \langle u, u \rangle + 2 \lambda \langle u, v \rangle + \lambda^2\langle v, v \rangle = 0 54 | $$ 55 | 56 | This is quadratic equation with respect to $\lambda$, so 57 | 58 | $$ 59 | \lambda_{1,2} = \frac{-2\langle u, v \rangle \plusmn \sqrt{(4\langle u, v \rangle^2 - 4\langle u, u \rangle\langle v, v \rangle)}}{2\langle v, v \rangle} 60 | $$ 61 | 62 | The value in the denominator is known to be positive, and the value under the root is positive as well, hence $\lambda_{1,2}$ are well-defined. So, there is a vector $u + \lambda_1v$: it's squared norm is 0. However, this vector can't be 0 itself since we proved $u,v$ are linearly independent. So, we got a contradiction, hence $u$ doesn't exist. 63 | 64 | _Exercise 4_ 65 | 66 | _(a)_ 67 | We have 68 | 69 | $$ 70 | \begin{aligned} 71 | \langle u + v, u - v \rangle &= \langle u, u \rangle - \langle u, v \rangle + \langle v, u \rangle - \langle v, v \rangle\\\\ 72 | &= \langle u, u \rangle - \langle v, v \rangle\\\\ 73 | &= ||u||^2 - ||v^2||\\\\ 74 | \end{aligned} 75 | $$ 76 | 77 | _(b)_ 78 | This follows directly from the above. 79 | 80 | _(c)_ 81 | Just look at the picture before 6.22. 82 | 83 | _Exercise 5_ 84 | 85 | Suppose $v \in V$ such that $(T - \sqrt{2}I)v = 0$. 86 | Then $Tv = \sqrt{2}v$. 87 | This implies that $||v|| \ge ||Tv|| = ||\sqrt{2}v|| = \sqrt{2}||v||$. 88 | Since $||v||$ cannot be negative, it follows that $||v|| = 0$. 89 | Hence $v = 0$, showing that $T - \sqrt{2}I$ is injective and, therefore, invertible. 90 | 91 | _Exercise 6_ 92 | 93 | Suppose $\langle u, v \rangle = 0$. 94 | Note that $u$ is orthogonal to any multiple of $v$. 95 | Then, by the Pythagorean Theorem (6.13), we have 96 | 97 | $$ 98 | ||u + av||^2 = ||u||^2 + |a| ||v||^2 \le ||u||^2. 99 | $$ 100 | 101 | Taking the square root of both sides completes the forward direction. 102 | 103 | For the converse, we will prove the contrapositive, that is, if $\langle u, v \rangle \neq 0$, then $||u|| > ||u + av||$ for some $a \in \mathbb{F}$. 104 | 105 | Suppose $\langle u, v \rangle \neq 0$. 106 | Note that neither $u$ nor $v$ can equal $0$. 107 | We have 108 | 109 | $$ 110 | \begin{aligned} 111 | ||u + av||^2 &= \langle u + av, u + av \rangle\\\\ 112 | &= \langle u, u \rangle + \langle u, av \rangle + \langle av, u \rangle + \langle av, av \rangle\\\\ 113 | &= ||u||^2 + \langle u, av \rangle + \langle av, u \rangle + |a|^2 ||v||^2\\\\ 114 | &< ||u||^2 115 | \end{aligned} 116 | $$ 117 | 118 | where the last line follows provided that $\langle u, av \rangle + \langle av, u \rangle + |a|^2 ||v||^2 < 0$. 119 | By 6.14, we can write $u = cv + w$ for some $c \in \mathbb{F}$ and $w \in V$ such that $\langle v, w \rangle = 0$. 120 | Note that $c \neq 0$, because $\langle v, v \rangle \neq 0$ and 121 | 122 | $$ 123 | 0 \neq \langle u, v \rangle = \langle cv + w, v \rangle = \langle cv, v \rangle + \langle w, v \rangle = c \langle v, v \rangle 124 | $$ 125 | 126 | Choose $a = -c$, then 127 | 128 | $$ 129 | \begin{aligned} 130 | \langle u, av \rangle + \langle av, u \rangle + |a|^2 ||v||^2 &= \langle cv + w, av \rangle + \langle av, cv + w \rangle + |a|^2 ||v||^2\\\\ 131 | &= \langle cv, av \rangle + \langle w, av \rangle + \langle av, cv \rangle + \langle av, w \rangle + |a|^2 ||v||^2\\\\ 132 | &= c\bar{a} \langle v, v \rangle + a\bar{c} \langle v, v \rangle + |a|^2 ||v||^2\\\\ 133 | &= (c\bar{a} + a\bar{c} + ||c||^2) ||v||^2\\\\ 134 | &= (-c\bar{c} - c\bar{c} + ||c||^2) ||v||^2\\\\ 135 | &= - |c|^2 ||v||^2\\\\ 136 | &< 0\\\\ 137 | \end{aligned} 138 | $$ 139 | 140 | _Exercise 7_ 141 | 142 | The forward direction is obvious, just choose $a = 1$ and $b = 0$ to get the desired result. 143 | 144 | Conversely, suppose $||u|| = ||v||$. 145 | Then 146 | 147 | $$ 148 | \begin{aligned} 149 | ||au + bv|| &= \langle au + bv, au + bv \rangle\\\\ 150 | &= \langle au, au \rangle + \langle au, bv \rangle + \langle bv, au \rangle + \langle bv, bv \rangle\\\\ 151 | &= |a|^2 ||u||^2 + ab \langle u, v \rangle + ab \langle v, u \rangle + |b|^2 ||v||^2\\\\ 152 | &= |a|^2 ||v||^2 + ab \langle u, v \rangle + ab \langle v, u \rangle + |b|^2 ||u||^2\\\\ 153 | &= \langle av, av \rangle + \langle bu, av \rangle + \langle av, bu \rangle + \langle bu, bu \rangle\\\\ 154 | &= \langle bu + av, bu + av \rangle\\\\ 155 | &= ||bu + av||^2\\\\ 156 | \end{aligned} 157 | $$ 158 | 159 | _Exercise 8_ 160 | 161 | We have 162 | 163 | $$ 164 | \langle u - v, u - v \rangle = \langle u, u \rangle - \langle u, v \rangle - \langle v, u \rangle + \langle v, v \rangle = 1 - 1 - 1 + 1 = 0 165 | $$ 166 | 167 | Thus $u - v = 0$, which implies that $u = v$. 168 | 169 | _Exercise 9_ 170 | 171 | Using Cauchy-Schwartz inequality, we have that $|\langle u, v \rangle| < ||u||||v|| \le 1$, so the right-hand side of the inequality we are interested in is at least positive. We can then apply square to both sides of the inequality, so it is equivalent to 172 | 173 | $$ 174 | (1 - ||u||^2)(1 - ||v||^2) \le (1 - |\langle u, v \rangle|)^2 175 | $$ 176 | 177 | We can strengthen this inequality substituting $||u||||v||$ instead of $|\langle u, v \rangle|$. 178 | 179 | $$ 180 | (1 - ||u||^2)(1 - ||v||^2) \le (1 - ||u||||v||)^2 181 | $$ 182 | 183 | Which is equivalent to 184 | 185 | $$ 186 | -||u||^2 -||v||^2 \le -2||u||||v|| 187 | $$ 188 | 189 | But this is obviously correct, since 190 | 191 | $$ 192 | 0 \le -2||u||||v|| + ||u||^2 + ||v||^2 = (||v|| - ||u||)^2 193 | $$ 194 | 195 | _Exercise 10_ 196 | 197 | $u = \lambda(1,3)$. Since $v$ should be orthogonal to (1,3), $v = \beta(-3, 1)$. Since $u + v = (1,2)$, we get the following system 198 | 199 | $$ 200 | \begin{cases} 201 | \lambda - 3\beta = 1 \\ 202 | 3\lambda + \beta = 2 203 | \end{cases} 204 | $$ 205 | 206 | So, $\lambda = 0.7$, $\beta = -0.1$. 207 | 208 | _Exercise 11_ 209 | 210 | Let $u = \left(\sqrt{a}, \sqrt{b}, \sqrt{c}, \sqrt{d}\right)$ and $v = \left(\frac{1}{\sqrt{a}}, \frac{1}{\sqrt{b}}, \frac{1}{\sqrt{c}}, \frac{1}{\sqrt{d}}\right)$. 211 | Then, by 6.15, we have 212 | 213 | $$ 214 | 4 = |\langle u, v \rangle| \le ||u||\\:||v|| = \sqrt{a + b + c + d} \sqrt{\frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d}}. 215 | $$ 216 | 217 | Taking the square of each side now gives the desired result. 218 | 219 | _Exercise 12_ 220 | 221 | Take $x = (x_1, \dots, x_n)$ and $y = (1, \dots, 1)$ ($n$ ones). 222 | By 6.15, we have 223 | 224 | $$ 225 | |x_1 + \dots + x_n| = |\langle x, y \rangle| \le ||x||\\:||y|| = \sqrt{n}\sqrt{x_1^2 + \dots + x_n^2} 226 | $$ 227 | 228 | Squaring each side now gives the desired result. 229 | 230 | _Exercise 13_ 231 | 232 | From the Law of Cosines, we have 233 | 234 | $$ 235 | \begin{aligned} 236 | ||u - v||^2 &= ||u||^2 + ||v||^2 - 2 ||u||\\:||v|| \cos \theta\\\\ 237 | ||u||^2 - 2 \langle u, v \rangle + ||v||^2 &= ||u||^2 + ||v||^2 - 2||u||\\:||v|| \cos \theta\\\\ 238 | -2 \langle u, v \rangle &= -2 ||u||\\:||v|| \cos \theta\\\\ 239 | \cos \theta &= \frac{\langle u, v \rangle}{||u||\\:||v||}\\\\ 240 | \end{aligned} 241 | $$ 242 | 243 | _Exercise 14_ 244 | 245 | The Cauchy-Schwarz Inequality ensures $\frac{\langle x, y \rangle}{||x||\\:||y||}$ is always a number between $-1$ and $1$. 246 | Moreover it only equals $1$ or $-1$ if one of $x, y$ is a scalar multiple of the other, that is, when the angle is $0$ or $\pi$. 247 | 248 | _Exercise 15_ 249 | 250 | Let $a = (\sqrt{1} a_1, \dots, \sqrt{n} a_n)$ and $b = \left(\frac{b_1}{\sqrt{1}}, \dots, \frac{b_n}{\sqrt{n}}\right)$. 251 | Then 252 | 253 | $$ 254 | \begin{aligned} 255 | \left(\sum\limits_{j=1}^n a_j b_j\right) &= \left(\langle a, b \rangle\right)^2\\\\ 256 | &\le \left(||a||\\:||b||\right)^2\\\\ 257 | &= ||a||^2 ||b||^2\\\\ 258 | &= \left(\sum\limits_{j=1}^n j a_j^2\right) \left(\sum\limits_{j=1}^n \frac{b_j^2}{j}\right)\\\\ 259 | \end{aligned} 260 | $$ 261 | 262 | where the second line follows from the Cauchy-Schwarz Inequality. 263 | 264 | _Exercise 16_ 265 | 266 | We can use the parallelogram equality: 267 | 268 | $$ 269 | 16 + 36 = 2(9 + ||v||^2) 270 | $$ 271 | 272 | So, $||v|| = \sqrt{17}$, since $||v|| \ge 0$. 273 | 274 | _Exercise 17_ 275 | 276 | This is clearly wrong. Suppose $v = (-1,-1)$, then $||v|| = -1 < 0$, which contradicts definition of norm as $||v|| = \sqrt(v,v) > 0$. 277 | 278 | _Exercise 18_ 279 | 280 | First let us notice, that it should be true that $(-1)^p > 0$, because else we can easily construct a vector $(x,-x)$ such that it has zero norm, however it is not zero itself. This implies, that $\forall x \in \mathbb{F}\ x^p = |x|^p = (-x)^p$. 281 | 282 | Now, consider two vectors $u = (x, x)$ and $v = (x, -x)$, with $x > 0$. Using the parallelogram equation, we obtain 283 | 284 | $$ 285 | ||(2x, 0)||^2 + ||(0,2x)||^2 = 2(||(x,x)||^2 + ||(x,-x)||^2) 286 | $$ 287 | 288 | It is clear, that $||(0, 2x)|| = ||(2x, 0)|| = ((2x)^p)^{\frac{1}{p}} = 2x$. On the otehr hand, $||(x,x)|| = ||(x,-x)|| = (2x^p)^{\frac{1}{p}} = 2^{\frac{1}{p}}x$. 289 | 290 | Hence, we end up with equation 291 | 292 | $$ 293 | 8x^2 = 4\cdot 2^{\frac{2}{p}}x^2 294 | $$ 295 | 296 | After contracting $4x^2$, we get $2 = 2^{\frac{2}{p}}$, and $p = 2$ is the only solution to this equation with respect to $p$. 297 | 298 | _Exercise 19_ 299 | 300 | We have 301 | 302 | $$ 303 | \begin{aligned} 304 | ||u + v||^2 - ||u - v||^2 &= \langle u+v, u+v \rangle - \langle u-v, u-v \rangle\\\\ 305 | &= \langle u, u \rangle + 2\langle u, v \rangle + \langle v, v \rangle - \left(\langle u, u \rangle - 2\langle u, v \rangle + \langle v, v \rangle\right)\\\\ 306 | &= 4\langle u, v \rangle\\\\ 307 | \end{aligned} 308 | $$ 309 | 310 | Dividing by $4$ yields the desired result. 311 | 312 | _Exercise 20_ 313 | 314 | We have 315 | 316 | $$ 317 | \begin{aligned} 318 | ||u+v||^2 - ||u-v||^2 + ||u+iv||^2 i - ||u-iv||^2i &= \langle u+v, u+v \rangle - \langle u-v,u-v \rangle + \left( \langle u+iv, u+iv \rangle - \langle u-iv, u-iv \rangle \right)i\\\\ 319 | &= 2\langle u, v \rangle + 2\langle v, u \rangle + \left( 2\langle u, vi \rangle + 2\langle iv, u \rangle \right) i\\\\ 320 | &= 2\langle u, v \rangle + 2\langle v, u \rangle + \left(-2i\langle u, v \rangle + 2i\langle v, u \rangle \right) i\\\\ 321 | &= 4\langle u, v \rangle\\\\ 322 | \end{aligned} 323 | $$ 324 | 325 | Dividing by $4$ yields the desired result. 326 | 327 | _Exercise 21_ 328 | 329 | Suppose $U$ is a complex vector space, then we can use inner product definition through the norm function defined in the previous exercise. All is left is to check that such defined function will in fact satisfy all the properties of inner product. 330 | 331 | _Exercise 22_ 332 | 333 | Let $x = (1, \dots, 1)$ ($n$ ones) and $a = (a_1, \dots, a_n)$. 334 | Then 335 | 336 | $$ 337 | \begin{aligned} 338 | \left( \frac{a_1 + \dots + a_n}{n} \right)^2 &= \frac{|\langle x, a \rangle|^2}{n^2}\\\\ 339 | &\le \frac{||x||^2||a||^2}{n^2}\\\\ 340 | &= \frac{||a||^2}{n}\\\\ 341 | &= \frac{a_1^2 + \dots + a_n^2}{n}\\\\ 342 | \end{aligned} 343 | $$ 344 | 345 | where the second line follows from the Cauchy-Schwarz Inequality. 346 | 347 | _Exercise 23_ 348 | 349 | We have $\langle (v_1, \dots, v_m), (v_1, \dots, v_m) \rangle = \langle v_1, v_1 \rangle + \dots + \langle v_m, v_m \rangle$. 350 | Positivity and definiteness follow directly from the fact that $\langle v_j, v_j \rangle \ge 0$ for each $j$. 351 | The remaining properties follow easily from the definitions. 352 | 353 | _Exercise 24_ 354 | 355 | For positivity, we have $\langle v, v \rangle_1 = \langle Sv, Sv \rangle \ge 0$. 356 | 357 | For definiteness, suppose $\langle v, v \rangle_1 = 0$. 358 | Then $\langle Sv, Sv \rangle = 0$, which implies that $Sv = 0$. 359 | However $S$ is injective, thus $v = 0$. 360 | Conversely, if $v = 0$, then $\langle v, v \rangle_1 = \langle Sv, Sv \rangle = \langle 0, 0 \rangle = 0$. 361 | 362 | For additivity in the first slot, we have $\langle u + v, w \rangle_1 = \langle Su + Sv, Sw \rangle = \langle Su, Sw \rangle + \langle Sv, Sw \rangle = \langle u, w \rangle_1 + \langle v, w \rangle_1$. 363 | 364 | For homogeneity, $\langle \lambda u, v \rangle_1 = \langle \lambda Su, Sv \rangle = \lambda \langle Su, Sv \rangle = \lambda \langle u, v \rangle_1$. 365 | 366 | For conjugate symmetry, we have $\langle u, v \rangle_1 = \langle Su, Sv \rangle = \overline{\langle Sv, Su \rangle} = \overline{\langle v, u \rangle_1}$. 367 | 368 | _Exercise 25_ 369 | 370 | If $S$ is not injective, then there exists a non-zero $v \in V$, such that $Sv = 0$. 371 | But $\langle v, v \rangle_1 = \langle Sv, Sv \rangle = \langle 0, 0 \rangle = 0$, therefore $\langle \cdot, \cdot \rangle_1$ cannot satisfy the definiteness property of inner products. 372 | 373 | _Exercise 26_ 374 | 375 | _(a)_ 376 | 377 | $$ 378 | \begin{aligned} 379 | \langle f(t), g(t) \rangle' &= \lim_{h \to 0} \frac{\langle f(t+h), g(t+h) \rangle - \langle f(t), g(t) \rangle}{h}\\\\ 380 | &= \lim_{h \to 0} \frac{\langle f(t+h), g(t+h) \rangle - \langle f(t), g(t+h) \rangle + \langle f(t), g(t+h) \rangle - \langle f(t), g(t) \rangle}{h}\\\\ 381 | &= \lim_{h \to 0} \frac{\langle f(t+h) - f(t), g(t+h) \rangle + \langle f(t), g(t+h) - g(t) \rangle}{h}\\\\ 382 | &= \lim_{h \to 0} \frac{\langle f(t+h) - f(t), g(t+h) \rangle}{h} + \lim_{h \to 0} \frac{\langle f(t), g(t+h) - g(t) \rangle}{h}\\\\ 383 | &= \lim_{h \to 0} \langle \frac{f(t+h) - f(t)}{h}, g(t+h) \rangle + \lim_{h \to 0} \langle f(t), \frac{g(t+h) - g(t)}{h} \rangle\\\\ 384 | &= \langle f'(t), g(t) \rangle + \langle f(t), g'(t) \rangle\\\\ 385 | \end{aligned} 386 | $$ 387 | 388 | _(b)_ 389 | 390 | We have 391 | 392 | $$ 393 | \langle f(t), f(t) \rangle' = \langle f'(t), f(t) \rangle + \langle f(t), f'(t) \rangle = 2\langle f'(t), f(t) \rangle. 394 | $$ 395 | 396 | However, $\langle f(t), f(t) \rangle'$ is $0$, because $\langle f(t), f(t) \rangle$ is constant (it equals $c^2$ for all $t$). 397 | Thus $\langle f'(t), f(t) \rangle = 0$. 398 | 399 | _(c)_ 400 | 401 | If $f(t)$ describes a trajectory on the surface of a sphere centered at the origin, then $f'(t)$ is perpendicular to $f(t)$. 402 | 403 | _Exercise 27_ 404 | 405 | Let us select $u' = w - \frac{1}{2}(u + v)$, $v' = \frac{1}{2}(v - u)$, then we need to prove that 406 | 407 | $$ 408 | ||u'||^2 = \frac{||u' + v'||^2 + ||u' - v'||^2}{2} - \frac{4||v'||^2}{4} 409 | $$ 410 | 411 | which can be easily regrouped to form parallelogram equality. 412 | 413 | _Exercise 28_ 414 | 415 | Suppose there exist two such points $u,v$. Suppose $||w - u|| = ||w - v|| = c$. Consider $x = \frac{1}{2}(v + u) \in C$. Then, from the previous exercise, we know, that 416 | 417 | $$ 418 | ||w - x||^2 = c^2 - \frac{1}{4}||u-v||^2 < c^2 419 | $$ 420 | 421 | Since $u\neq v$, we end up with $x$ being closer to $w$, then $v,u$, which is a contradiction. 422 | 423 | _Exercise 29_ 424 | 425 | _(a)_ 426 | We need to check 4 properties: 427 | 428 | 1. The distance from point to itself should be zero: $d(x,x) = ||x - x|| = ||0|| = 0$. 429 | 2. Positivity: if $x \neq y$, $d(x,y) = ||x - y|| > 0$, which holds because of definitieness of inner product. 430 | 3. Symmetry is obvious: $d(x,y) = ||x - y|| = |-1|||y - x|| = ||y - x|| = d(y,x)$ 431 | 4. Triangle inequality: $||x - z|| \le ||x - y|| + ||y - z||$, which follows from triangle inequality for norms with $u = y - z$, $v = x - y$. 432 | 433 | _(b)_ 434 | 435 | Suppose $x_1,...,x_n,...$ is a Cauchy sequence, so $\forall \varepsilon >0 \exists N: m,n > N \implies d(x_m,x_n) < \varepsilon$. We will choose orthonormal basis $e_1,...,e_n$, and represent $x_i = \sum\limits_{r=1}^nc_{ir}e_r$. Writing down Cauchy sequence condition, we have that given arbitrary $\varepsilon >0$ we can find $N \in \mathbb{N}$ such, that if $i,j > \mathbb{N}$, we have $||\sum\limits_{r=1}^n(c_{ir} - c_{jr})e_r|| < \varepsilon$, which implies $\sum\limits_{r=1}^n|c_{ir} - c_{jr}| < \varepsilon$, and so $|c_{ir} - c_{jr}| < \varepsilon$ for any $r=1..n$. 436 | 437 | This means, that $\{c_{1r}, c_{2r}, ....\}$ is a Cauchy sequence in $\mathbb{F}$, and if $\mathbb{F}$ is real or complex numbers, then it converges to an element of $\mathbb{F}$, denote it $c_r$. Then, it is easy to show that $d(x_i, c) \rightarrow 0$, with $i \to \infty$, where $c = \sum\limits_{i=1}^nc_ie_i \in V$. 438 | 439 | _(c)_ 440 | 441 | By the same logic, we can operate orthonormal basis of $U$, and hence the limit $c$ will be also in $U$, so it will be closed subset of $V$. 442 | 443 | -------------------------------------------------------------------------------- /Chapter 03 - Linear Maps/3E - Products and Quotients of Vector Spaces.md: -------------------------------------------------------------------------------- 1 | Chapter 3: **Linear Maps** 2 | 3 | **3.E** 4 | 5 | - [x] Exercise 1 6 | - [x] Exercise 2 7 | - [x] Exercise 3 8 | - [x] Exercise 4 9 | - [x] Exercise 5 10 | - [x] Exercise 6 11 | - [x] Exercise 7 12 | - [x] Exercise 8 13 | - [x] Exercise 9 14 | - [x] Exercise 10 15 | - [x] Exercise 11 16 | - [x] Exercise 12 17 | - [x] Exercise 13 18 | - [x] Exercise 14 19 | - [x] Exercise 15 20 | - [x] Exercise 16 21 | - [x] Exercise 17 22 | - [x] Exercise 18 23 | - [x] Exercise 19 24 | - [x] Exercise 20 25 | 26 | _Exercise 1_ 27 | 28 | Suppose $T$ is linear. 29 | Let $(v_1, Tv_1), (v_2, Tv_2) \in \operatorname{graph of} T$. 30 | We have 31 | 32 | $$ 33 | \begin{aligned} 34 | (v_1, Tv_1) + (v_2, Tv_2) &= (v_1 + v_2, Tv_1 + Tv_2)\\\\ 35 | &= (v_1 + v_2, T(v_1 + v_2))\\\\ 36 | &\in \operatorname{graph of} T, 37 | \end{aligned} 38 | $$ 39 | 40 | Where the second line follows because $T$ is linear and the third by definition of $\operatorname{graph of} T$. 41 | Hence, $\operatorname{graph of} T$ is closed under addition. 42 | Similarly, it also closed under scalar multiplication. 43 | Therefore, $\operatorname{graph of} T$ is a subspace of $V \times W$. 44 | 45 | Conversely, suppose $\operatorname{graph of} T$ is a subspace of $V \times W$. 46 | Let $v_1, v_2 \in V$. 47 | Then 48 | 49 | $$ 50 | (v_1, Tv_1), (v_2, Tv_2), (v_1 + v_2, T(v_1 + v_2)) \in \operatorname{graph of} T. \tag{1} 51 | $$ 52 | 53 | Since $\operatorname{graph of} T$ is a subspace of $V \times W$, adding the first two vectors above shows that 54 | 55 | $$ 56 | (v_1 + v_2, Tv_1 + Tv_2) \in \operatorname{graph of} T. \tag{2} 57 | $$ 58 | 59 | Because $T$ is function, if $(v, w), (v, \hat{w}) \in \operatorname{graph of} T$ then $w = \hat{w}$. 60 | This, together with $(1)$ and $(2)$, implies that $T(v_1 + v_2) = Tv_1 + Tv_2$. 61 | Hence, $T$ satisfies the additivity property of linear maps. 62 | Similarly, $T$ satisfies the homogeneity property. 63 | Therefore, $T$ is indeed a linear map. 64 | 65 | _Exercise 2_ 66 | 67 | Define $T: V_1 \times \dots \times V_m \to V_j$ by 68 | 69 | $$ 70 | T(v_1, \dots, v_m) = v_j. 71 | $$ 72 | 73 | $T$ is clearly linear and surjective. 74 | The Fundamental Theorem of Linear Maps (3.22) now implies that $V_j$ is finite-dimensional. 75 | 76 | _Exercise 3_ 77 | 78 | Let $V = U_1 = U_2 = \mathbb{F}^\infty$. 79 | Note that $U_1 + U_2 = \mathbb{F}^\infty$. 80 | Define $T: \mathbb{F}^\infty \times \mathbb{F}^\infty \to \mathbb{F}^\infty$ by 81 | 82 | $$ 83 | T\big((x_1, x_2, x_3, \dots), (y_1, y_2, y_3, \dots)\big) = (x_1, y_1, x_2, y_2, x_3, y_3, \dots). 84 | $$ 85 | 86 | Thus, $T$ is obviously surjective. 87 | To prove injectivity, let 88 | 89 | $$ 90 | \big((x_1, x_2, x_3, \dots), (y_1, y_2, y_3, \dots)\big), \big((z_1, z_2, z_3, \dots), (w_1, w_2, w_3, \dots)\big) \in \mathbb{F}^\infty \times \mathbb{F}^\infty 91 | $$ 92 | 93 | such that 94 | 95 | $$ 96 | T\big((x_1, x_2, x_3, \dots), (y_1, y_2, y_3, \dots)\big) = T\big((z_1, z_2, z_3, \dots), (w_1, w_2, w_3, \dots)\big). 97 | $$ 98 | 99 | This implies that 100 | 101 | $$ 102 | (x_1, y_1, x_2, y_2, x_3, y_3, \dots) = (z_1, w_1, z_2, w_2, z_3, w_3). 103 | $$ 104 | 105 | Therefore, $x_j = z_j$ and $y_j = w_j$ for each $j$. 106 | Hence 107 | 108 | $$ 109 | \big((x_1, x_2, x_3, \dots), (y_1, y_2, y_3, \dots)\big) = \big((z_1, z_2, z_3, \dots), (w_1, w_2, w_3, \dots)\big) 110 | $$ 111 | 112 | and so $T$ is injective. 113 | Thus, $T$ is an isomorphism, as desired. 114 | 115 | _Exercise 4_ 116 | 117 | Consider any $\mathcal{f} \in \mathcal{L(V_1\times V_2 \times ... \times V_m, W)}$. We know that if $v_{11},...,v_{1,n_1}$ is a basis of $V_1$, ..., $v_{m1},...,v_{mn_m}$ is a basis of $V_m$, then $u_{11} = (v_{11}, 0, ... 0), u_{12} = (v_{12}, 0, ... 0), ...., u_{mn_m} = (0, 0, ... v_{mn_m})$ is a basis of $V_1\times V_2 \times ... \times V_m$. Let us also choose any basis $w_1,...,w_k$ in $W$ and denote matrix $A = \mathcal{M}(\mathcal{f})$. Then, $\mathcal{f}u_{ij} = \sum\limits_{k=1}^mA_{k,n_1+n_2+...+n_{i-1}+j}w_k$. We can then define $\varphi_{i} \in \mathcal{L}(V_i,W)$ through submatrix of $A$ corresponding to $V_i$ basis, so that $\mathcal{f}(v_1, v_2, ..., v_m) = \varphi_1(v_1) + ... + \varphi(v_m)$. The equation is clearly correct since we defined $\varphi_i$ through submatrix of $A$, so we have injectivity (obviously, different $A$ matrices will produce different sets of $\varphi_1,...,\varphi_m$, and $A$ matrices are trivially isomorphic to $\mathcal{L(V_1\times V_2 \times ... \times V_m, W)}$). Surjectivity is also obvious, since for every set of $\varphi_i$ we can build corresponding $\mathcal{M}(\mathcal{f})$ and hence find $\mathcal{f}$. So, we have got an isomorphism $\mathcal(f) \to (\varphi_1,...,\varphi_m)$, as desired. 118 | 119 | _Exercise 5_ 120 | 121 | This is a bit simpler. Consider $T \in \mathcal{L}(V,W_1\times ...\times W_m)$ and suppose $Tv = (w_1,...,w_m)$. Define $T_i: T_iv = w_i$ - this will obviously be a linear map from $V$ to $W_i$. Then, we can build an isomorphism $\Psi$: $\Psi T = (T_1,...,T_m)$. It is clearly injective - since the only element in the $\operatorname{null}\Psi$ is $(0,...,0)$, and so it is in fact isomorphism. 122 | 123 | _Exercise 6_ 124 | 125 | Let $u_1,...,u_n$ be a basis of $\mathbb{F}^n$. Let $(v_1,...,v_n) \in V^n$ and construct $T: \mathbb{F}^n \to V$ so, that $Tu_i = v_i$. It is clear that such definition uniquely maps any list of vectors from $V^n$ to a function from $\mathcal{\mathbb{F}^n,V}$. This mapping is also surjective, since we have taken arbitrary list of vectors, so it is isomorphism. 126 | 127 | _Exercise 7_ 128 | 129 | Let $v + U = x + W$. For $u = 0$ there exists $w_0 \in W$: $v + 0 = x + w_0 \implies w_0 = v - x \implies v - x \in W$. We now can subtract $v$ from both parts and get $U = (x - v) + W = -w_0 + W = W$. 130 | 131 | _Exercise 8_ 132 | 133 | $\Leftarrow$: Since $A \neq \emptyset$, there is at least one element. Denote it $v \in A$. We will now prove that $-v + A$ is a subspace. 134 | 135 | 1. Since $v \in A$, $v - v \in A - v$, so $0 \in A - v$. 136 | 2. Let $x \in A - v$. This means that $\exists u \in A: x = u - v$. 137 | By properties of $A$ we know that $\lambda u + (1 - \lambda)v \in A$, which means that $\lambda(u - v) + v \in A \implies \lambda x + v \in A \implies \lambda x \in A - v$. 138 | 3. Let $x_1, x_2 \in A - v$. This means $x_1 = u_1 - v$, $x_2 = u_2 - v$, $u_1,u_2 \in A$. By properties of $A$, $\frac{1}{2}u_1 + \frac{1}{2}u_2 \in A$, so $\frac{1}{2}u_1 + \frac{1}{2}u_2 - v \in A - v$, and using the previous statement we can conclude that $u_1 + u_2 - 2v \in A - v$, so $x_1 + x_2 \in A - v$. 139 | 140 | We have shown that all three conditions of linear space are satisfied, so $A - v$ is in fact a linear space, which means that $A = v + (A - v)$ is an affine subset. 141 | 142 | $\Rightarrow:$ Let $A$ be an affine subset. This means, that $\exists U \subset V$: $U$ is linear space, $v\in V$: $A = v + U$. 143 | 144 | Consider any two elements from $A$: $v_1 = u_1 + v$, $v_2 = u_2 + v$, where $u_1,u_2 \in U$. 145 | 146 | $u_1, u_2 \in U \implies \lambda u_1 + (1 - \lambda)u_2 \in U \implies v + \lambda u_1 + (1 - \lambda)u_2 \in v + U \implies \lambda (v + u_1) + (1 - \lambda)(v + u_2) \in v + U \implies \lambda v_1 + (1-\lambda)v_2 \in v + U$. 147 | 148 | _Exercise 9_ 149 | 150 | Let $A_1,A_2$ be two affine subsets of $V$, and suppose $A_1\cap A_2 \neq \emptyset$. If so, $\exists x \in A_1\cap A_2$. Consider any $y \in A_1 \cap A_2$, and denote $A_1 = v_1 + U_1$, $A_2 = v_2 + U_2$, where $U_1, U_2$ are subspaces of $V$, $v_1,v_2\in V$. After subtracting $y - x$, we get $y - x \in U_1$ and $y - x \in U_2$, and so $y - x \in U_1 \cap U_2$, from which it follows that $y = x + U_1 \cap U_2$. Since $y$ was arbitrary, we conclude, that $A_1 \cap A_2 = x + U_1 \cap U_2$, and since $U_1 \cap U_2$ is a linear subspace itself, $A_1 \cap A_2$ is an affine subset. 151 | 152 | _Exercise 10_ 153 | 154 | This can be easily done through induction. We have already proven that for two subsets it is true. Now, suppose the statement is true for any number of subsets less, than n. Then, we can rewrite $A_1\cap A_2 \cap ... \cap A_n$ as $B \cap A_n$, where $B = A_1\cap A_2 \cap...\cap A_{n-1}$. By induction, $B$ is either empty or is an affine subset. In the first case, the overall intersection is also empty. In the latter case, we can use the base of our induction and conclude, that $B \cap A_n$ is either an affine subset itself, or is empty. 155 | 156 | _Exercise 11_ 157 | 158 | a) We can use the result of the problem #8. To prove that $A$ is an affine subset we need to prove that if $u_1, u_2 \in A$, then $\lambda u_1 + (1 - \lambda)u_2 \in A$. 159 | 160 | Let $u_1 = \lambda_1 v_1 + ... + \lambda_m v_m$ and $u_2 = \mu_1v_1 + ... + \mu_m v_m$, where $\sum\limits_{i=1}^m\lambda_i = \sum\limits_{i=1}^m\mu_i = 1$. 161 | 162 | $$ 163 | \lambda u_1 + (1 - \lambda)u_2 = \lambda\sum\limits_{i=1}^m\lambda_iv_i + (1 - \lambda)\sum\limits_{i=1}^m\mu_iv_i = \sum\limits_{i=1}^m(\lambda\lambda_i + (1 - \lambda)\mu_i)v_i 164 | $$ 165 | 166 | Consider sum of coefficients in front of $v_1$ in the expression above: 167 | 168 | $$ 169 | \sum\limits_{i=1}^m(\lambda\lambda_i + (1 - \lambda)\mu_i) = \lambda\sum\limits_{i=1}^m(\lambda_i) + (1 - \lambda)\sum\limits_{i=1}^m(\\mu_i) = \lambda \cdot 1 + (1 - \lambda) \cdot 1 = 1 170 | $$ 171 | 172 | This implies that by definition of $A$ $\lambda u_1 + (1 - \lambda)u_2 \in A$, so $A$ is indeed an affine subset. 173 | 174 | b) Let $B$ be an affine subset containing all of $v_1,...,v_m$. Suppose $B = v + U$, where $v \in V$ and $U$ is a subspace of $V$. Since $v_1,...,v_m$ are in $B$, we can write 175 | 176 | $$ 177 | v_1 = v + u_1\\ 178 | v_2 = v + u_2\\ 179 | ...\\ 180 | v_m = v + u_m 181 | $$ 182 | 183 | We then have for any $\lambda_1,...,\lambda_m$: $\sum\limits_{i=1}^m\lambda_i = 1$: $\lambda_1 v_1 + ... + \lambda_m v_m = \sum\limits_{i=1}^m\lambda_iv + \sum\limits_{i=1}^m(\lambda_i)u_i = v + \sum\limits_{i=1}^m(\lambda_i)u_i = v + u'$, where $u' \in U$, so $\lambda_1 v_1 + ... + \lambda_m v_m \in B$, and since $\{\lambda_i\}$ were arbitrary, $A \subset B$. 184 | 185 | c) Note that $v_m \in A$, which means that we can write $A = v_m + U$. This implies that $U = A - v_m$, and so any element in $U$ can be written as $u = \sum\limits_{i=1}^m \lambda_iv_i - v_m = \sum\limits_{i=1}^{m-1} \lambda_iv_i + (1 - \sum\limits_{i=1}^{m-1} \lambda_i)v_m - v_m = \sum\limits_{i=1}^{m-1} \lambda_i(v_i - v_m)$, where $\lambda_1...\lambda_{m-1} \in \mathbb{F}$, so $U = \operatorname{span}(v_1 - v_m,..., v_{m-1} - v_m)$, which implies $\operatorname{dim}U = \operatorname{dim}\operatorname{span}(v_1 - v_m,..., v_{m-1} - v_m) \le m - 1$. 186 | 187 | _Exercise 12_ 188 | 189 | Let $v_1 + U, ..., v_k + U$ be a basis of $V/U$. Consider x = $\sum\limits_{i=1}^kc_iv_i$. Suppose $x \in U$, then $x + U = U = 0_{V/U}$, but it is possible only if all $c_i = 0$, since $x + U = \sum\limits_{i=1}^kc_i(v_i + U)$, and this is a linear combination of the $V/U$ bases vectors. So, any linear combination of $v_1,...,v_k$ with not all $c_i=0$ is not in $U$. 190 | 191 | Now, consider any $v \in V$. Let us prove that there $\exists u: v = \sum\limits_{i=1}^kc_iv_i + u$. Indeed, consider $v + u_0: u_0 \in U$. $v + u_0 \in v + U$, so $v + u_0 = \sum\limits_{i=1}^kc_iv_i + u_1$, where $v + U = \sum\limits_{i=1}^kc_i(v_i + U)$ is representation of $v + U$ as linear combination of basis vectors of $V/U$. So, we have that $v + u_0 = \sum\limits_{i=1}^kc_iv_i + u_1$, and so $v = \sum\limits_{i=1}^kc_iv_i + u_1 - u_0 = \sum\limits_{i=1}^kc_iv_i + u$, where $u = u_1 - u_0 \in U$. We have proven, that any vector $v$ can be represented as $\sum\limits_{i=1}^kc_iv_i + u$, where $u \in U$. 192 | 193 | Now, let us show that this representation is unique. Indeed, consider $v \in V$, and let $v = \sum\limits_{i=1}^kc_iv_i + u = \sum\limits_{i=1}^kd_iv_i + u'$. Then, $0 = v - v = \sum\limits_{i=1}^kc_iv_i + u - (\sum\limits_{i=1}^kd_iv_i + u') = \sum\limits_{i=1}^k(c_i - d_i)v_i + u - u'$, hence we get $u' - u = \sum\limits_{i=1}^k(c_i - d_i)v_i$. But the left part of the equation belongs to $U$, and this means that the right part of the equation must be 0, as we have proven earlier. So, $c_i = d_i$, and hence $u = u'$, so the representation is indeed unique. 194 | 195 | We next want to prove that such a mapping $T$: $v = (\sum\limits_{i=1}^kc_iv_i + u) \to (u, \sum\limits_{i=1}^kc_i(v_i + U))$ is isomorphism, and to do this, we need to show that $T$ is linear. 196 | 197 | 1. $T0 = 0$. Indeed, if $v = 0$, then $0 = \sum\limits_{i=1}^kc_iv_i + u \implies u = -\sum\limits_{i=1}^kc_iv_i$. But we already have proven that if linear combination of $v_i$ is in $U$, it implies that $c_i = 0$. So, $c_i = 0$ and $u = 0$, so $T0 = (0, 0 + U)$. By the way, we also have proven that $\operatorname{null}T = \{0\}$, so $T$ is injective. 198 | 199 | 2. $T\lambda v = T\lambda(\sum\limits_{i=1}^kc_iv_i + u) = T(\sum\limits_{i=1}^k\lambda c_iv_i + \lambda u) = (\lambda u, \sum\limits_{i=1}^k\lambda c_iv_i + U) = \lambda (u, \sum\limits_{i=1}^k c_iv_i + U) = \lambda Tv$ 200 | 201 | 3. $T(v + w) = T(\sum\limits_{i=1}^kc_iv_i + u + \sum\limits_{i=1}^kd_iv_i + u') = T(\sum\limits_{i=1}^k(c_i + d_i)v_i + u + u') = (u + u', \sum\limits_{i=1}^k(c_i + d_i)v_i + U) = Tv + Tw$. 202 | 203 | So, $T$ is indeed linear. Since it is also surjective and injective, we can conclude, that it is isomorphism. 204 | 205 | _Exercise 13_ 206 | 207 | We have already proven in the previous task that $v_1,...,v_m$ are linearly independent and $\forall v \in V$ we can represent uniquely $v = \sum\limits_{i=1}^mc_iv_i + u$, where $u \in U$. So, $v = \sum\limits_{i=1}^mc_iv_i + \sum\limits_{i=1}^nd_iu_i$. This representation is unique, so $c_i, d_i$ are unique, which means that $v_1,...v_m,u_1,...,u_n$ indeed form a basis of V. 208 | 209 | _Exercise 14_ 210 | 211 | a) It is clear that if $(0, 0,...) \in U$, $v \in U \implies \lambda v \in U$ and $v_1,v_2 \in U \implies v_1 + v_2 \in U$ since $v_1 + v_2$ will still have a finite number of non-zero coefficients. 212 | 213 | b) Denote $v_i \in \mathbb{F}^\infty$: $v_i[j] = 1$ if $j = p_i^n$, where $p_i$ is the i'th prime number, and $n \in \mathbb{N}$. It is clear, that every $v_i$ has non-zero coefficients on different positions than any other $v_j$, so $\sum\limits_{i=1}^nc_iv_i = 0 \implies c_1=...=c_n0$ for any $n$. It is also clear, that $v_i \notin U$ since there are infinitely many non-zero coefficients in $v_i$. Now, consider $v_1 + U,...,v_n+U$ for some $n \in \mathbb{N}$. If they were linearly independent in $\mathbb{F}^\infty/U$, then there should exist non-zero coefficients $c_1...c_n$: $\sum\limits_{i=1}^nc_iv_i \in U$. But this linear combination will have infinitely many non-zero coefficients, and hence it cannot be in $U$. This means that $v_1 + U,...,v_n+U$ are linearly independent for any $n$, and so $\mathbb{F}^\infty/U$ is infinite-dimensional. 214 | 215 | _Exercise 15_ 216 | 217 | Using fundamental theorem of linear maps, we can write $\operatorname{dim}V = \operatorname{dim}\operatorname{null}\varphi + \operatorname{dim}\operatorname{range}\varphi$. Since $\varphi \neq 0$, we have that $\operatorname{dim}\operatorname{range}\varphi > 0$, and $\operatorname{range}\varphi \subset \mathbb{F}$, so $\operatorname{dim}\operatorname{range}\varphi = 1$. Hence, $\operatorname{dim}\operatorname{null}\varphi = \operatorname{dim}V - 1$. So, the dimension of the quotient space is $\operatorname{dim}V/\operatorname{null}\varphi = \operatorname{dim}V - \operatorname{dim}\operatorname{null}\varphi = 1$. 218 | 219 | _Exercise 16_ 220 | 221 | Since $\operatorname{dim}V/U = 1$, its basis consists of single element $v_0 + U$. This implies, that any vector $v \in V$ can be represented as $\lambda v_0 + u$, where $u \in U$. Define $\varphi$ so that $\varphi v_0 = 1$, and $\forall u \in U \ \varphi u = 0$. It is easy to show that $\varphi$ is linear, and $\operatorname{null}\varphi = U$. 222 | 223 | _Exercise 17_ 224 | 225 | We have proven everything required in exercise 12 226 | 227 | _Exercise 18_ 228 | 229 | $\Rightarrow$: suppose $\exists S: T = S \circ \pi$. Consider $u \in U$. We have $Tu = S \circ \pi u = S(u + U) = S0 = 0$, so indeed $U \subset \operatorname{null}T$. 230 | 231 | $\Leftarrow$: suppose $U \subset \operatorname{null}T$. Let us define $S(v + U) = Tv$ and show that it is linear map $V/U \to W$. 232 | 233 | 1. $S(0 + U) = Tu = 0$ since $U \subset \operatorname{null}T$ 234 | 2. $S((v + w + U)) = T(v + w) = Tv + Tw = S(v + U) + S(w + U)$ 235 | 3. $S(\lambda(v + U)) = S(\lambda v + U) = T\lambda v = \lambda Tv = \lambda S(v + U)$ 236 | 237 | _Exercise 19_ 238 | 239 | The equivalent statement is $U_1$,...,$U_n$: $U_i \in V$ are disjoint if and only if $|U_1 \cup U_2 \cup ... \cup U_n| = |U_1| + ... + |U_n|$. 240 | 241 | _Exercise 20_ 242 | 243 | a) Let's prove $\Gamma$ is linear. 244 | 1. $\Gamma(0) = 0 \circ \pi = 0$ 245 | 2. $\Gamma(S_1 + S_2) = (S_1 + S_2)\circ \pi$. $\forall v \in V \ \implies ((S_1 + S_2)\circ \pi)(v) = (S_1 + S_2)(v + U) = S_1(v + U) + S_2(v + U) = (S_1 \circ \pi) (v) + (S_2 \circ \pi) (v) = \Gamma(S_1)(v) + \Gamma(S_2)(v)$, hence $\Gamma(S_1 + S_2) = \Gamma(S_1) + \Gamma(S_2)$ since these function are equal in every point. 246 | 3. $\Gamma(\lambda S) = (\lambda S) \circ \pi$. $\forall v \in V \ \implies ((\lambda S) \circ \pi)(v) = (\lambda S) (v + U) = \lambda S(v + U) = \lambda \Gamma(S)(v)$, and so $\Gamma(\lambda S) = \lambda \Gamma(S)$ 247 | 248 | b) consider $S_1,S_2 \in \mathcal{L}(V/U,W)$. 249 | 250 | $\Gamma(S_1) - \Gamma(S_2) = (S_1 - S_2) \circ \pi$. If $\Gamma(S_1) = \Gamma(S_2)$, we have that $\forall v \in V$ (S_1 - S_2)(v + U) = 0. But this means, that $S_1 - S_2 = 0$, and hence $S_1 = S_2$, so $\Gamma$ is indeed injective. 251 | 252 | c) We want to prove that $\operatorname{range}\Gamma = \{T \in \mathcal{L}(V, W): Tu = 0 \forall u \in U\}$. 253 | 254 | Consider first $u \in U$. $\forall S \ \Gamma(S)(u) = S(u + U) = S(0 + U) = 0$, which means that $\operatorname{range}\Gamma \subset \{T \in \mathcal{L}(V, W): Tu = 0 \forall u \in U\}$ 255 | 256 | On the other hand, consider any $T: \forall u \in U \ Tu = 0$. Define $S \in \mathcal(L)(V/U, W): S(v + U) = Tv$. It is easy to show, that such defined S will be linear map on $V/U$ and by construction $\Gamma(S) = T$. -------------------------------------------------------------------------------- /Chapter 07 - Operators on Inner Product Spaces/7D - Polar Decomposition and Singular Value Decomposition.md: -------------------------------------------------------------------------------- 1 | Chapter 7: **Operators on Inner Product Spaces** 2 | 3 | - [x] Exercise 1 4 | - [x] Exercise 2 5 | - [x] Exercise 3 6 | - [x] Exercise 4 7 | - [x] Exercise 5 8 | - [x] Exercise 6 9 | - [x] Exercise 7 10 | - [x] Exercise 8 11 | - [x] Exercise 9 12 | - [x] Exercise 10 13 | - [x] Exercise 11 14 | - [x] Exercise 12 15 | - [x] Exercise 13 16 | - [x] Exercise 14 17 | - [x] Exercise 15 18 | - [x] Exercise 16 19 | - [x] Exercise 17 20 | - [x] Exercise 18 21 | - [x] Exercise 19 22 | - [x] Exercise 20 23 | 24 | _Exercise 1_ 25 | 26 | A quick calculation shows that $T^*Tv = ||x||^2\langle v, u \rangle u$ for every $v \in V$. 27 | The map $R \in \mathcal{L}(V)$ defined by 28 | 29 | $$ 30 | Rv = \frac{||x||}{||u||}\langle v, u \rangle u 31 | $$ 32 | 33 | is a square root of $T^*T$. 34 | Moreover, it is easy to check that $\langle Rv, v \rangle \ge 0$ for all $v \in V$. 35 | We but need to prove that $R$ is self-adjoint. 36 | Let $e_1, \dots, e_n$ be an orthonormal basis of $V$. 37 | We can write 38 | 39 | $$ 40 | u = a_1 e_1 + \dots + a_n e_n 41 | $$ 42 | 43 | for some $a_1, \dots, a_n \in \mathbb{F}$. 44 | Note that 45 | 46 | $$ 47 | Re_j = \frac{||x||}{||u||}\langle e_j, u \rangle u = \frac{||x||}{||u||}(a_j \overline{a_1} e_1 + \dots + a_j \overline{a_n} e_n). 48 | $$ 49 | 50 | Therefore, the matrix of $R$ with respect to the basis $e_1, \dots, e_n$ has entries defined by 51 | 52 | $$ 53 | \mathcal{M}(R)_{j, k} = \frac{||x||}{||u||}a_j \overline{a_k}. 54 | $$ 55 | 56 | Thus $\mathcal{M}(R)_{j, k} = \overline{\mathcal{M}(R)_{k, j}}$, that is, $\mathcal{M}(R) = \mathcal{M}(R^*)$. 57 | Hence $R$ is self-adjoint. 58 | 59 | _Exercise 2_ 60 | 61 | Define $T \in \mathcal{L}(\mathbb{C}^2)$ by 62 | 63 | $$ 64 | \begin{aligned} 65 | Te_1 &= 0\\\\ 66 | Te_2 &= 5e_1 67 | \end{aligned} 68 | $$ 69 | 70 | where $e_1, e_2$ is the standard basis of $\mathbb{C}^2$. 71 | Then the matrix of $T$ with respect to this is basis is 72 | 73 | $$ 74 | \mathcal{M}(T) = \begin{pmatrix}0 & 5\\\\0 & 0\end{pmatrix}, 75 | $$ 76 | 77 | which is upper triangular. 78 | Thus $0$ is the only eigenvalue of $T$. 79 | We have 80 | 81 | $$ 82 | \mathcal{M}(T^*T) = \mathcal{M}(T^*)\mathcal{M}(T) = \begin{pmatrix}0 & 0\\\\5 & 0\end{pmatrix}\begin{pmatrix}0 & 5\\\\0 & 0\end{pmatrix} = \begin{pmatrix}0 & 0\\\\0 & 25\end{pmatrix}. 83 | $$ 84 | 85 | Hence the eigenvalues of $T^*T$ are $0$ and $25$. 86 | By 7.52, the singular values of $T$ are $0$ and $5$. 87 | 88 | _Exercise 3_ 89 | 90 | By the Polar Decomposition (7.45), there exists an isometry $S \in \mathcal{L}(V)$ such that $T^* = S\sqrt{TT^*}$. 91 | Taking the adjoint of each side, we get 92 | 93 | $$ 94 | T = (S\sqrt{TT^*})^* = (\sqrt{TT^*})^*S^* = \sqrt{TT^*}S^*, 95 | $$ 96 | 97 | where the last equality follows because $\sqrt{TT^*}$ is self-adjoint. 98 | This yields the desired result, because $S^*$ is also an isometry. 99 | 100 | _Exercise 4_ 101 | 102 | Let $v \in V$ be an eigenvector of $\sqrt{T^*T}$ with $||v|| = 1$ corresponding to $s$ and let $S \in \mathcal{L}(V)$ be an isometry such that $T = S\sqrt{T^*T}$. 103 | Then 104 | 105 | $$ 106 | ||Tv|| = ||S\sqrt{T^*T}v|| = ||\sqrt{T^*T}v|| = |s|||v|| = |s| = s, 107 | $$ 108 | 109 | where the last equality follows because $\sqrt{T^*T}$ is positive. 110 | 111 | _Exercise 5_ 112 | 113 | We have 114 | 115 | $$ 116 | \mathcal{M}(T^*T) = \mathcal{M}(T^*)\mathcal{M}(T) = \begin{pmatrix}0 & 1\\\\-4 & 0\end{pmatrix}\begin{pmatrix}0 & -4\\\\1 & 0\end{pmatrix} = \begin{pmatrix}1 & 0\\\\0 & 16\end{pmatrix}. 117 | $$ 118 | 119 | Therefore, the singular values of $T$ are $1$ and $4$. 120 | 121 | _Exercise 6_ 122 | 123 | We will use the orthonormal basis $\sqrt{\frac{1}{2}}, \sqrt{\frac{3}{2}}x, \sqrt{\frac{45}{8}}\left(x^2 - \frac{1}{3}\right)$ of $\mathcal{P}_2(\mathbb{R})$, which was found in the example. 124 | 125 | We have 126 | 127 | $$ 128 | \begin{aligned} 129 | D\sqrt{\frac{1}{2}} &= 0\\\\ 130 | D\sqrt{\frac{3}{2}}x &= \sqrt{3}\left(\sqrt{\frac{1}{2}}\right)\\\\ 131 | D\sqrt{\frac{45}{8}}\left(x^2 - \frac{1}{3}\right) &= \sqrt{15}\left(\sqrt{\frac{3}{2}}x\right). 132 | \end{aligned} 133 | $$ 134 | 135 | Therefore 136 | 137 | $$ 138 | \mathcal{M}(D) = 139 | \begin{pmatrix} 140 | 0 & \sqrt{3} & 0\\\\ 141 | 0 & 0 & \sqrt{15}\\\\ 142 | 0 & 0 & 0 143 | \end{pmatrix}. 144 | $$ 145 | 146 | Thus 147 | 148 | $$ 149 | \mathcal{M}(D^*D) = 150 | \begin{pmatrix} 151 | 0 & 0 & 0\\\\ 152 | \sqrt{3} & 0 & 0\\\\ 153 | 0 & \sqrt{15} & 0 154 | \end{pmatrix} 155 | \begin{pmatrix} 156 | 0 & \sqrt{3} & 0\\\\ 157 | 0 & 0 & \sqrt{15}\\\\ 158 | 0 & 0 & 0 159 | \end{pmatrix} = 160 | \begin{pmatrix} 161 | 0 & 0 & 0\\\\ 162 | 0 & 3 & 0\\\\ 163 | 0 & 0 & 15 164 | \end{pmatrix}. 165 | $$ 166 | 167 | Thus, the singular values of $D$ are $0, \sqrt{3}, \sqrt{15}$. 168 | 169 | _Exercise 7_ 170 | 171 | A quick calculation, like the ones in the previous exercises, shows that 172 | 173 | $$ 174 | T^*T(z_1, z_2, z_3) = (4z_1, 9z_2, z_3). 175 | $$ 176 | 177 | Thus 178 | 179 | $$ 180 | \sqrt{T^*T}(z_1, z_2, z_3) = (2z_1, 3z_2, z_3). 181 | $$ 182 | 183 | Define $S \in \mathcal{L}(\mathbb{F}^3)$ by 184 | 185 | $$ 186 | S(z_1, z_2, z_3) = (z_3, z_1, z_2). 187 | $$ 188 | 189 | $S$ is clearly an isometry (it maps the standard basis, which is orthonormal, to a permutation of the standard basis) and we have $S\sqrt{T^*T} = T$. 190 | 191 | _Exercise 8_ 192 | 193 | Let $S' \in \mathcal{L}(V)$ be an isometry such that $T = S'\sqrt{T^*T}$. 194 | Then 195 | 196 | $$ 197 | \begin{aligned} 198 | 0 &= ||Tv|| - ||Tv||\\\\ 199 | &= ||S^*Tv|| - ||S'^*Tv||\\\\ 200 | &= ||Rv|| - ||\sqrt{T^*T}v||\\\\ 201 | &= \langle R^*Rv, v \rangle - \langle T^*Tv, v \rangle\\\\ 202 | &= \langle (R^*R - T^*T)v, v \rangle\\\\ 203 | &= \langle (R^2 - T^*T)v, v \rangle 204 | \end{aligned} 205 | $$ 206 | 207 | where the last line follows because $R$ is self-adjoint. 208 | 7.16 now implies that $R^2 = T^*T$. 209 | Since $R$ is positive and the positive square root of $T^*T$ is unique (by 7.36), it follows that $R = \sqrt{T^*T}$. 210 | 211 | _Exercise 9_ 212 | 213 | Consider the proof of 7.45. 214 | $T$ being invertible implies $\operatorname{dim} (\operatorname{range} T)^\perp = 0$, thus $S_2 = 0$ and so $S = S_1$. 215 | Clearly $S_1$ is unique, so $S$ must also be unique. 216 | Conversely, if $S$ is unique then $S_2 = 0$, otherwise we could set $S = S_1 - S_2$ and it would still be an isometry satisfying $T = S\sqrt{T^*T}$. 217 | This implies that $m = 0$, because from the defintion we see that $S_2$ must be invertible for any positive integer $m$. 218 | Thus $\operatorname{dim} (\operatorname{range} T)^\perp = 0$ and so $T$ is invertible. 219 | 220 | _Exercise 10_ 221 | 222 | Suppose $\lambda$ is an eigenvalue of $T$ and $v$ a corresponding eigenvector. 223 | Then 224 | 225 | $$ 226 | T^*Tv = T^2v = \lambda^2 v = |\lambda|^2v, 227 | $$ 228 | 229 | where the last equality follows because the eigenvalues of self-adjoint operators are real (see 7.13). 230 | Therefore, the eigenvectors of $T$ are also eigenvectors of $T^*T$ with corresponding eigenvalues squared. 231 | Since $T$ has a basis consisting of eigenvectors, so does $T^*T$ and thus all eigenvalues of $T^*T$ are squares of the absolute values of eigenvalues of $T$. 232 | 7.52 now implies that the singular values of $T$ are the absolute values of the eigenvalues of $T$. 233 | 234 | _Exercise 11_ 235 | 236 | We need to prove that eigenvalues of $\sqrt{T^*T}$ and $\sqrt{TT^*}$ are the same. 237 | 238 | As we know, eigenvalues of $\sqrt{T^*T}$ are non-negative square roots of $T^*T$, the same goes with eigenvalues of $\sqrt{TT^*}$, so we need to prove, that $T^*T$ and $TT^*$ have the same eigenvalues. 239 | 240 | Suppose $T^*Tv = \lambda v$ with $v \neq 0$, so $\lambda$ is an eigenvalue of $T^*T$. We have two possibilities: either $\lambda = 0$ or not. 241 | 242 | Suppose $\lambda = 0$. Then, $T$ is not injective, which means that $\operatorname{range}T \neq V$, which means that $(\operatorname{null}T^*)^\perp \neq V$, which means that $\operatorname{null}T^* \neq \{0\}$, so $T^*$ is also not injective hence 0 is eigenvalue of $T^*$. 243 | 244 | Now, suppose $\lambda \neq 0$. Then, multiplying this equation from both sides by $T$ we get 245 | 246 | $$ 247 | TT^*(Tv) = \lambda Tv 248 | $$ 249 | 250 | Since $\lambda \neq 0$, it can not be that $Tv = 0$, since else $T^*0 = 0 = \lambda v \neq 0$. This means, that $Tv$ is an eigenvector of $TT^*$ with corresponding eigenvalue $\lambda$. 251 | 252 | We now proved that for any $\lambda$ - eigenvalue of $T^*T$ it follows that $\lambda$ is also an eigenvalue of $TT^*$. Considering the same proof, but starting with $TT^*$, we conclude, that any eigenvalue of $TT^*$ is also an eigenvalue of $T^*T$, which completes the proof. 253 | 254 | _Exercise 12_ 255 | 256 | As a counterexample, take the linear map $T$ from Exercise 5. 257 | We have 258 | 259 | $$ 260 | \mathcal{M}((T^2)^*T^2) = \mathcal{M}(T^*)^2\mathcal{M}(T)^2 = \begin{pmatrix}-4 & 0\\\\0 & -4\end{pmatrix}\begin{pmatrix}-4 & 0\\\\0 & -4\end{pmatrix} 261 | = \begin{pmatrix}16 & 0\\\\0 & 16\end{pmatrix}. 262 | $$ 263 | 264 | Therefore, the singular values of $T^2$ are $4, 4$. 265 | However, the singular values of $T$ are $1, 4$. 266 | 267 | _Exercise 13_ 268 | 269 | Suppose $T$ is invertible. 270 | Then 271 | 272 | $$ 273 | \operatorname{null} T^* = (\operatorname{range} T)^\perp = V^\perp = \{0\}, 274 | $$ 275 | 276 | where the first equality follows from 7.7 and the second because $T$ is surjective. 277 | This shows that $T^*$ is also invertible. 278 | Therefore $0$ is not eigenvalue of $T^*T$ and so, by 7.52, it cannot be a singular value of $T$. 279 | 280 | Conversely, suppose $0$ is not a singular value of $T$. 281 | Then $T^*Tv \neq 0$ for all non-zero $v \in V$. 282 | This implies that $Tv \neq 0$ for all non-zero $v \in V$. 283 | Thus $T$ is invertible. 284 | 285 | _Exercise 14_ 286 | 287 | First we will prove that $\operatorname{range} T = \operatorname{range} T^*T$. 288 | From Exercise 5 in section 7A we see that $\operatorname{range} T = \operatorname{range} T^*$. 289 | 290 | Suppose $w \in \operatorname{range} T$. 291 | Then $w \in \operatorname{range} T^*$. 292 | Thus $w = T^*v$ for some $v \in V$. 293 | We can write $v = v' + v''$ for some $v' \in \operatorname{null} T^*$ and $v'' \in (\operatorname{null} T^*)^\perp$. 294 | Thus $w = T^*v''$. 295 | But 7.7 shows that $(\operatorname{null} T^*)^\perp = \operatorname{range} T$. 296 | Therefore $v'' = Tu$ for some $u \in V$ and so $w = T^*Tu \in \operatorname{range}{T^*T}$. 297 | Hence $\operatorname{range} T \subset \operatorname{range} T^*T$. 298 | 299 | The inclusion in the other direction is easy. 300 | We have 301 | 302 | $$ 303 | \operatorname{range} T^*T \subset \operatorname{range} T^* = \operatorname{range} T. 304 | $$ 305 | 306 | Therefore $\operatorname{range} T = \operatorname{range} T^*T$. 307 | 308 | Since $\sqrt{T^*T}$ is diagonalizable (because it is self-adjoint), it follows that the number of nonzero singular values of $T$ equals the dimension of $\operatorname{range} \sqrt{T^*T}$. 309 | Note that $\operatorname{range} T^*T = \operatorname{range} \sqrt{T^*T}$. 310 | Therefore $\operatorname{dim} \operatorname{range} \sqrt{T^*T} = \operatorname{dim} \operatorname{range} T$, completing the proof. 311 | 312 | _Exercise 15_ 313 | 314 | The forward direction is obvious, because if $S$ is an isometry, then $\sqrt{S^*S}$ equals the identity, whose eigenvalues equal $1$. 315 | 316 | Suppose all singular values of $S$ equal $1$. 317 | This implies that $\sqrt{S^*S}$, and therefore so does $S^*S$, equals the identity. 318 | 7.42 now implies that $S$ is an isometry. 319 | 320 | _Exercise 16_ 321 | 322 | Let $S_1, S_2 \in \mathcal{L}(V)$ be isometries such that $T_1 = S_1\sqrt{T_1^*T_1}$ and $T_2 = S_2\sqrt{T_2^*T_2}$ and let $e_1, \dots, e_n$ and $f_1, \dots, f_n$ be orthonormal basis of $V$ consisting of eigenvectors of $T_1$ and $T_2$, respectively, corresponding to the singular values $s_1, \dots, s_n$. 323 | Defin $S \in \mathcal{L}(V)$ by 324 | 325 | $$ 326 | Se_j = f_j 327 | $$ 328 | 329 | for each $j = 1, \dots, n$. 330 | Then $S$ is also an isometry and we have 331 | 332 | $$ 333 | \begin{aligned} 334 | \sqrt{T_1^*T_1}e_j &= s_je_j\\\\ 335 | &= S^*(s_jf_j)\\\\ 336 | &= S^*\sqrt{T_2^*T_2}f_j\\\\ 337 | &= S^*\sqrt{T_2^*T_2}Se_j. 338 | \end{aligned} 339 | $$ 340 | 341 | So $\sqrt{T_1^*T_1} = S^*\sqrt{T_2^*T_2}S$. 342 | Therefore 343 | 344 | $$ 345 | T_1 = S_1\sqrt{T_1^*T_1} = S_1S^*\sqrt{T_2^*T_2}S = S_1S^*S_2^*T_2S, 346 | $$ 347 | 348 | where the last equality follows by multiplying both sides of the equation $T_2 = S_2\sqrt{T_2^*T_2}$ by $S_2^*$. 349 | This gives the desired result, because the product of isometries is also an isometry. 350 | 351 | Alternative proof: 352 | 353 | Suppose 354 | $$ 355 | \begin{align} 356 | T_1 = S_1T_2S_2 \implies \\ 357 | T_1^* =S_2^*T_2^*S_1^* = S_2^{-1}T_2^*S_1^{-1} 358 | \end{align} 359 | $$ 360 | 361 | Hence, 362 | $$ 363 | T_1^*T_1 = S_2^*T_2^*T_2S_2 364 | $$ 365 | 366 | Suppose $v,\lambda$ are eigen vector and squared singular value of $T_2$. Then, 367 | 368 | $$ 369 | T_2^*T_2v = \lambda v 370 | $$ 371 | 372 | Denote $w = S_2^{-1}v \neq 0$. Then, it follows that 373 | 374 | $$ 375 | T_1^*T_1w = S_2^*T_2^*T_2S_2S_2^{-1}v = S_2^*T_2^*T_2v = \lambda S_2^{-1}v = \lambda w 376 | $$ 377 | 378 | So, $\lambda$ is a squared singular value of $T_1$, hence all the singular values of $T_2$ are also singular values of $T_1$. Replacing $T_1$ and $T_2$ and repeating the reasoning, we conclude that singular values of $T_1$ and $T_2$ are the same. 379 | 380 | Now, let us prove the reverse statement: 381 | 382 | Suppose all the singular values of $T_1$ and $T_2$ are the same, we need to prove, that $T_1 = S_1T_2S_2$ for some isometries $S_1,S_2$. 383 | 384 | Using singular value decomposition, we can write for arbitrary $v \in V$: 385 | 386 | $$ 387 | \begin{align} 388 | T_1v = s_1\langle v, e_1 \rangle f_1 + ... + s_n\langle v, e_n \rangle f_n 389 | T_2v = s_1\langle v, \tilde e_1 \rangle \tilde f_1 + ... + s_n\langle v, \tilde e_n \rangle \tilde f_n 390 | \end{align} 391 | $$ 392 | 393 | Where $e_1,..,e_n$, $f_1,..,f_n$, $\tilde e_1,..,\tilde e_n$, $\tilde f_1,..,\tilde f_n$ are some orhtonormal basises. 394 | 395 | Consider following isometries $S_1,S_2$: 396 | 397 | $$ 398 | S_1e_i = \tilde e_i \\ 399 | S_2\tilde f_i = f_i \\ 400 | $$ 401 | 402 | Such defined linear maps will be isometries, which can be easily seen (using (d) of Characterization of isometries). 403 | 404 | $$ 405 | \begin{align} 406 | S_2v = \langle v, e_1 \rangle \tilde e_1 + ... + \langle v, e_n \rangle \tilde e_n\\ 407 | T_2S_2v = \langle v, e_1 \rangle \tilde f_1 + ... + \langle v, e_n \rangle \tilde f_n \\ 408 | S_1T_2S_2v = \langle v, e_1 \rangle f_1 + ... + \langle v, e_n \rangle f_n \\ 409 | \end{align} 410 | $$ 411 | 412 | So, we see that $S_1T_2S_2v = T_1v$ for any $v$, which means $S_1T_2S_2 = T_1$. 413 | 414 | _Exercise 17_ 415 | 416 | _(a)_ 417 | We have $Te_j = s_jf_j$. 418 | This implies that the matrix of $T$ if respect to the bases $e_1, \dots, e_n$ and $f_1, \dots, f_n$ is the diagonal matrix whose diagonal entries are the singular values of $T$. 419 | By 7.10, we have $T^*f_j = s_je_j$ for each $j$. 420 | If we replace $v$ with $f_j$ in the right hand side of the desired result we get the same thing, therefore 421 | 422 | $$ 423 | T^*v = s_1\langle v, f_1 \rangle e_1 + \dots + s_n \langle v, f_n \rangle e_n 424 | $$ 425 | 426 | by uniqueness of linear maps (see 3.5). 427 | 428 | _(b)_ 429 | Just apply the previous item to the formula given of the singular value decomposition of $T$. 430 | 431 | _(c)_ 432 | Note that the $e_j$'s are eigenvectors of $T^*T$ with corresponding eigenvalue $s_j^2$. 433 | Thus $\sqrt{T^*T}e_j = s_je_j$. 434 | Plugging $e_j$ in the place of $v$ in the right hand side yields the same thing, so the result holds by uniqueness of linear maps again. 435 | 436 | _(d)_ 437 | The given formula satisfies $TT^{-1} = I$ and $T^{-1}T = I$ and it is well defined, because from Exercise 13 we see that none of $s_j$'s are $0$. 438 | 439 | _Exercise 18_ 440 | 441 | _(a)_ 442 | Use the same notation from the Singular Value Decompositon theorem (7.52). 443 | We have 444 | 445 | $$ 446 | \begin{aligned} 447 | ||Tv||^2 448 | &= |s_1\langle v, e_1 \rangle|^2 + \dots + |s_n\langle v, e_n \rangle|^2\\\\ 449 | &= |s_1|^2|\langle v, e_1 \rangle|^2 + \dots + |s_n|^2|\langle v, e_n \rangle|^2\\\\ 450 | &\le |s|^2|\langle v, e_1 \rangle|^2 + \dots + |s|^2|\langle v, e_n \rangle|^2\\\\ 451 | &= |s|^2{|\langle v, e_1 \rangle|^2 + \dots + |\langle v, e_n \rangle|^2}\\\\ 452 | &= s^2||v||^2 453 | \end{aligned} 454 | $$ 455 | 456 | for all $v \in V$, where the first line follows from the Pythagorean Theorem (6.13) and last because $s$ is a positive real value. 457 | Taking the square root of both sides shows that $||Tv|| \le s||v||$ for all $v \in V$. 458 | The proof that $\hat{s}||v|| \le ||Tv||$ is almost the same. 459 | 460 | _(b)_ 461 | Let $v$ be an eigenvector of $T$ corresponding to $\lambda$ with $||v|| = 1$. 462 | Then 463 | 464 | $$ 465 | |\lambda| = |\lambda|||v|| = ||Tv|| \le s||v|| = s. 466 | $$ 467 | 468 | Similarly, we have $\hat{s} \le |\lambda|$. 469 | 470 | 471 | _Exercise 19_ 472 | 473 | To prove uniform continuity we need to show that $\forall \varepsilon > 0 \exists \delta > 0 : \forall u, v \in V ||u - v|| < \delta \implies ||T(u) - T(v)|| < \varepsilon$. 474 | 475 | This is quite easy to do using previous exercise: we already know, that if $s$ is the largest singular value of $T$, then 476 | 477 | $$ 478 | ||T(u - v)|| \le s||u-v|| < s\delta 479 | $$ 480 | 481 | If $s = 0$, then it is obvious that $T = 0$, so any delta will be fine, for instance $\delta = 1$. Suppose $s > 0$, so $T \neq 0$. Then, if we chose $\delta = \frac{\varepsilon}{s}$, we will have $||T(u - v)|| < \varepsilon$ as desired. 482 | 483 | _Exercise 20_ 484 | 485 | Consider $v$ - eigenvector of $T + S$, corresponding to eigenvalue $r$. 486 | 487 | $$ 488 | ||(S+T)v|| = r||v|| 489 | $$ 490 | 491 | On the other hand, using triangle inequality, we have 492 | 493 | $$ 494 | ||(S+T)v|| = ||Sv + Tv|| \le ||Sv|| + ||Tv|| \le s||v|| + t||v|| 495 | $$ 496 | 497 | as was shown in the exercise 18. So, 498 | 499 | $$ 500 | r||v|| \le s||v|| + t||v|| \\ 501 | r \le s + t 502 | $$ 503 | --------------------------------------------------------------------------------