MIT 18.06 Linear Algebra
Eigenvalue, Difference Equation and Differential Equation
update on Oct 20
I finally see a connection between eigenvalue and practice from episode22 and episode23. Take the Fibonacci sequence as an example:
\[F_{k+2} = F_{k+1} + F_{k}\] \[F_{k+1} = F_{k+1}\]let vector \(u_{k} = \left[ \begin{matrix} F_{k+1} \\ F_{k} \end{matrix} \right]\), we can get:
\[u_{k+1} = A u_{k}\]in which \(A = \left[ \begin{matrix} 1 & 1 \\ 1 & 0 \end{matrix} \right]\). And find the eigenvalue $ \lambda = \frac{1 \pm \sqrt{5}}{2} $. So
\[F_{k} \approx c_1 (\frac{1 + \sqrt{5}}{2})^{k}\]since
\[u_{k} = A^ku_0 = c_1 \lambda_1 ^ k x_1 + c_2 \lambda_2 ^ k x_2 + \dots + c_n \lambda_n ^ k x_n = \Lambda ^k S c\]in which \(\Lambda = \left[ \begin{matrix} \lambda_1 \\ & \ddots & \\ & & \lambda _n \end{matrix} \right]\), $ S = \left[ \begin{matrix} x_1 & x_2 & \dots & x_n \end{matrix} \right]$, where $ x $ is the eigenvectors of matrix A.
It is similar with differential equations, say:
\[\frac{du_1}{dt} = -u_1 + 2u_2 \\ \frac{du_2}{dt} = u_1 - 2u_2\]let vector \(u = \left[ \begin{matrix} u_1 \\ u_2 \end{matrix} \right]\), we can get:
\[\frac{du}{dt} = A u\]in which \(A = \left[ \begin{matrix} -1 & 2 \\ 1 & -2 \end{matrix} \right]\). And solution:
\[u(t) = c_1e^{\lambda _1 t}x_1 + c_2e^{\lambda _2 t}x_1\]Another example:
\[y'' + by' + ky = 0\]let vector \(u = \left[ \begin{matrix} y' \\ y \end{matrix} \right]\), we can get:
\[u' = \left[ \begin{matrix} y'' \\ y' \end{matrix} \right]= \left[ \begin{matrix} -b & -k \\ 1 & 0 \end{matrix} \right] \left[ \begin{matrix} y' \\ y \end{matrix} \right]\]in this way, we can easily solve $n$th order diffenrence or diffenrential equations.
Formula of Solving Inverse of Matrixs
\[A^{-1} = \frac{1}{det A} C^T\]update on Oct 18
in which $ C $ means the cofactor of matrixs A. Prof. Strang give a very beautiful proof of this formula: tett beeutiful
to proof
\[A^{-1} = \frac{1}{det A} C^T\]equals to proof:
\[A C^T = det A I\]which means:
\[\left[ \begin{matrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \cdots & a_{nn} \end{matrix} \right] \left[ \begin{matrix} c_{11} & c_{21} & \cdots & c_{n1} \\ c_{12} & c_{22} & \cdots & c_{n2} \\ \vdots & \vdots & \ddots & \vdots \\ c_{1n} & c_{2n} & \cdots & c_{nn} \end{matrix} \right] = \left[ \begin{matrix} detA & 0 & \cdots & 0 \\ 0 & detA & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & detA \end{matrix} \right]\]We can find the $i$th row of $ A $ times the $i$th column of $ C $ equals $ detA $, and the other combinations are all zeros. It’s done!
Big Formula of Determinant
\[det A = \sum_{n!terms}\pm{a_{1\alpha}a_{2\beta}a_{3\gamma}}\dots a_{n\omega}\]update on Oct 17
$ (\alpha, \beta, \gamma \dots \omega) = Permutation \space of \space (1, 2, 3 \dots n) $
关于正规方程(Normal Equation)的理解
\[x = (A^TA)^{-1}A^Tb\]update on Oct 16
对于一个方程 $ Ax = b $ ，其中A是$ m \times n $的长矩阵。这意味着此方程可能无解。换言之，b向量可能不在 $ Ax $ 张成的空间中。为此，我们可以把b向量进行投影(Projection)，也就是方程两边同时乘以矩阵A的转置：
\[A^TAx = A^Tb\]这样就可以得到向量x的最优解：
\[x = (A^TA)^{-1}A^Tb\]