So far we have discussed the math of vectors with real entries, i.e., vectors $(v_1,v_2,v_3)$ where $v_1,v_2,v_3 \in \mathbb{R}$. In fact we can do linear algebra over any field. The term field applies to any mathematical object (think different types of numbers) for which we have defined the operations of addition, subtraction, multiplication and division.

The complex numbers $\mathbb{C}$ are a field. Therefore we can do linear algebra over the complex numbers. We can define complex vectors $\mathbb{C}^n$ and complex matrices $\mathbb{C}^{m \times n}$ which behave similarly to their real counterparts. You will see that complex linear algebra is no more complex than real linear algebra. It is the same, in fact, except for one small difference: instead of matrix transpose $A^T$ we have to use the Hermitian transpose $A^\dagger$ which is the combination of the transpose and an entry-wise complex conjugate operation.

Complex vectors are not just an esoteric mathematical concept intended for specialists. Complex vectors can arise as answers for problems involving ordinary real matrices. For example, the rotation matrix \[ R_\theta = \begin{bmatrix} \cos\theta &-\sin\theta \nl \sin\theta &\cos\theta \end{bmatrix} \] has complex eigenvalues $\lambda_1 = e^{i\theta}$ and $\lambda_2 = e^{-i\theta}$ and eigenvectors with complex coefficients. Thus, if you want to know how to calculate the eigenvalues and eigenvectors of rotation matrices, you need to understand how to do linear algebra calculations with $\mathbb{C}$.

This section will also serve as a review of many of the important concepts in linear algebra so I recommend that you read it even if your class doesn't require you to know about complex matrices. As your linear algebra teacher, I want you to know about linear algebra over the field of complex numbers because I have a hidden agenda, which I'll tell you about at the end of this section.

Recall the basic notions of complex numbers:

• $i$: the unit imaginary number $i \equiv \sqrt{-1}$ or $i^2 = -1$
• $z=a+bi$: a complex number that has both real part and imaginary part
• $\mathbb{C}$: the set of complex numbers $\mathbb{C} = \{ a + bi \ | \ a,b \in \mathbb{R} \}$
• $\textrm{Re}\{ z \}=a$: the real part of $z=a+bi$
• $\textrm{Im}\{ z \}=b$: the imaginary part of $z=a+bi$
• $\bar{z}$: the complex conjugate of $z$. If $z=a+bi$, then $\bar{z}=a-bi$.
• $|z|=\sqrt{ \bar{z}z }=\sqrt{a^2+b^2}$: the magnitude or length of $z=a+bi$

A complex vector $\vec{v} \in \mathbb{C}^n$ is an array of $n$ complex numbers. \[ \vec{v} = (v_1,v_2,v_3) \ \in \ (\mathbb{C},\mathbb{C},\mathbb{C}) \equiv \mathbb{C}^3. \]

A complex matrix $A \in \mathbb{C}^{m\times n}$ is a two-dimensional array of numbers: \[ A = \left[\begin{array}{ccc} a_{11} & a_{12} & a_{13} \nl a_{21} & a_{22} & a_{23} \nl a_{31} & a_{32} & a_{33} \end{array}\right] \ \in \ \left[\begin{array}{ccc} \mathbb{C} & \mathbb{C} & \mathbb{C} \nl \mathbb{C} & \mathbb{C} & \mathbb{C} \nl \mathbb{C} & \mathbb{C} & \mathbb{C} \end{array}\right] \equiv \mathbb{C}^{3\times 3}. \]

The Hermitian transpose operation, complex transpose or “dagger”($\dagger$) operation consists of the combination of the regular transpose ($A \to A^T$) and the complex conjugation of each entry in the matrix ($a_{ij} \to \overline{a_{ij}}$): \[ A^\dagger \equiv \overline{(A^T)}=(\overline{A})^T. \] Expressed in terms of the entries of the matrix $a_{ij}$, the Hermitian transpose corresponds to the transformation $a_{ij} \to \overline{ a_{ji} }$.

For example, the Hermitian conjugation operation applied to a $3\times3$ matrix is \[ A = \begin{bmatrix} a_{11} & a_{12} & a_{13} \nl a_{21} & a_{22} & a_{23} \nl a_{31} & a_{32} & a_{33} \end{bmatrix}, \qquad A^\dagger = \begin{bmatrix} \overline{a_{11}} & \overline{a_{21}} & \overline{a_{31}} \nl \overline{a_{12}} & \overline{a_{22}} & \overline{a_{32}} \nl \overline{a_{13}} & \overline{a_{23}} & \overline{a_{33}} \end{bmatrix}. \]

Recall that a vector is a special case of a matrix: you can identify a vector $\vec{v} \in \mathbb{C}^n$ with a column matrix $\vec{v} \in \mathbb{C}^{n \times 1}$. We can therefore apply the complex conjugation operation on vectors: \[ \vec{v}^\dagger \equiv \overline{(\vec{v}^T)}=(\overline{\vec{v}})^T \] The complex conjugate of a column vector is a row vector in which each of the coefficients have been conjugated: \[ \vec{v} = \begin{bmatrix} \alpha \nl \beta \nl \gamma \end{bmatrix}, \qquad \vec{v}^\dagger = \begin{bmatrix} \alpha \nl \beta \nl \gamma \end{bmatrix}^\dagger = \begin{bmatrix} \overline{\alpha} & \overline{\beta} & \overline{\gamma} \end{bmatrix}. \]

The complex conjugation of vectors is important to understand because it allows us to define an inner product operation for complex vectors.

Recall that the inner product for vectors with complex coefficients ($\vec{u}, \vec{v} \in \mathbb{C}^n$) is defined as the operation: \[ \langle \vec{u}, \vec{v} \rangle \equiv \sum_{i=1}^n \overline{u_i} v_i \equiv \vec{u}^\dagger \vec{v}. \] Note that the complex conjugation is applied to each of the first vector's components in the expression. This corresponds naturally to the notion of applying the Hermitian transpose on the first vector to turn it into a row vector of complex conjugates and then following the general rule for matrix multiplication of a $1 \times n$ matrix $\vec{u}^\dagger$ by an $n \times 1$ matrix $\vec{v}$.

Let us jump right into the heart of the matter. One of the fundamental ideas we learned in this chapter has been how to model linear systems, that is, input-output phenomena in which one vector $\vec{v}$ is related to another vector $\vec{w}$ in a linear way. We can think of this input-output relation as a linear transformation $T:\mathbb{R}^m \to \mathbb{R}^n$. Furthermore, we learned that any linear transformation can be represented as a $m\times n$ matrix with real coefficients with respect to some choice of input basis and output basis.

Linear algebra thinking can also be applied for complex vectors. For example, a linear transformation from $\mathbb{C}^2$ to $\mathbb{C}^2$ can be represented in terms of the matrix product \[ \begin{bmatrix} w_1 \nl w_2 \end{bmatrix} = \begin{bmatrix} \alpha & \beta \nl \gamma & \delta \end{bmatrix} \begin{bmatrix} v_1 \nl v_2 \end{bmatrix}, \] for some $2 \times 2$ matrix $\begin{bmatrix} \alpha & \beta \nl \gamma & \delta \end{bmatrix}$ where $\alpha,\beta, \gamma,\delta \in \mathbb{C}$.

This change from the real numbers to the complex numbers has the effect of doubling the dimensions of the transformation. Indeed, a $2 \times 2$ complex matrix has eight “parameters” not four. Where did you see the eight? Here: \[ \begin{bmatrix} \alpha & \beta \nl \gamma & \delta \end{bmatrix} = \begin{bmatrix} \textrm{Re}\{\alpha\} & \textrm{Re}\{\beta\} \nl \textrm{Re}\{\gamma\} & \textrm{Re}\{\delta\} \end{bmatrix} + \begin{bmatrix} \textrm{Im}\{\alpha\} & \textrm{Im}\{\beta\} \nl \textrm{Im}\{\gamma\} & \textrm{Im}\{\delta\} \end{bmatrix}i \] Each of the four coefficients of the matrix has a real part and an imaginary part $z =\textrm{Re}\{ z \}+\textrm{Im}\{ z \}i$ so there is a total of eight parameters to “pick” when specifying the matrix.

Similarly, to specify a vector $\vec{v}=\mathbb{C}^2$ you need to specify four parameters \[ \begin{bmatrix} v_1 \nl v_2 \end{bmatrix} = \begin{bmatrix} \textrm{Re}\{v_1\} \nl \textrm{Re}\{v_2\} \end{bmatrix} + \begin{bmatrix} \textrm{Im}\{v_1\} \nl \textrm{Im}\{v_2\} \end{bmatrix}i. \]

Suppose you are solving a problem which involves complex numbers and system of two linear equations in two unknowns: \[ \begin{align*} x_1 + 2x_2 & = 3+i, \nl 3x_2 + (9+i)x_2 & = 6+2i. \end{align*} \] You are asked to solve this system, i.e., to find the values of the unknowns $x_1$ and $x_2$.

The solutions $x_1$ and $x_2$ will be complex numbers, but apart from that there is nothing special about this problem: linear algebra with complex numbers is the same as linear algebra with the real numbers. To illustrate this point, we'll now go through the steps we need to solve this system of equations. You will see that all the linear algebra techniques you learned also work for complex numbers.

First observe that the system of equations can be written as a matrix-vector product: \[ \begin{bmatrix} 1 & 2 \nl 3 & 9+i \end{bmatrix} \begin{bmatrix} x_1 \nl x_2 \end{bmatrix} = \begin{bmatrix} 3+i \nl 6+2i \end{bmatrix}, \] or more compactly as $A\vec{x}=\vec{b}$. Here $A$ is a $2 \times 2$ matrix and $\vec{x}$ is the vector of unknowns (a $2 \times 1$ matrix) and $\vec{b}$ is a vector of constants (a $2 \times 1$ matrix).

The solution can easily be obtained if by first finding the inverse matrix $A^{-1}$. We have $\vec{x}=A^{-1}\vec{b}$.

For the above matrix $A$, the inverse matrix $A^{-1}$ is \[ A^{-1} = \begin{bmatrix} 1 + \frac{6}{3 + i} & - \frac{2}{3 + i}\nl - \frac{3}{3 + i} & \frac{1}{3 + i} \end{bmatrix} \] We can now compute the answer $\vec{x}$ using the matrix inverse and the equation $\vec{x}=A^{-1}\vec{b}$. We obtain \[ \begin{bmatrix} x_1 \nl x_2 \end{bmatrix} = \begin{bmatrix} 1 + \frac{6}{3 + i} & - \frac{2}{3 + i}\nl - \frac{3}{3 + i} & \frac{1}{3 + i} \end{bmatrix} \begin{bmatrix} 3+i\nl 6 + 2i \end{bmatrix} = \begin{bmatrix} 3+i + 6 - 4 \nl -3 + 2 \end{bmatrix} = \begin{bmatrix} 5+i \nl -1 \end{bmatrix}. \]

Recall that we learned several different approaches for computing the matrix inverse. Here we will review the general procedure for computing the inverse of a matrix by using row operations.

Given the matrix \[ A = \begin{bmatrix} 1 & 2 \nl 3 & 9+i \end{bmatrix}, \] the first step is to build an augmented array which contains the matrix $A$ and the identity $I$ matrix. \[ \left[ \begin{array}{ccccc} 1 & 2 &|& 1 & 0 \nl 3 & 9+i &|& 0 & 1 \end{array} \right]. \]

We now perform Gauss-Jordan elimination procedure on the resulting $2 \times 4$ matrix.

1. The first step is to subtract three times the first row

from the second row, or written compactly $R_2 \gets R_2 -3R_1$ to obtain:

  \[
  \left[ 
  \begin{array}{ccccc}
  1 & 2  	&|&  1  & 0  \nl
  0 & 3+i  	&|&  -3 & 1  
  \end{array} \right].
  \]
- Second we perform $R_2 \gets \frac{1}{3+i}R_2$ and get:
  \[
  \left[ 
  \begin{array}{ccccc}
  1 & 2  &|&  1  & 0  \nl
  0 & 1  &|&  \frac{-3}{3+i} & \frac{1}{3+i} 
  \end{array} \right].
  \]
- Finally we perform $R_1 \gets R_1 - 2R_2$ to obtain:
  \[
  \left[ 
  \begin{array}{ccccc}
  1 & 0  &|&  1 + \frac{6}{3+i}  & - \frac{2}{3+i}   \nl
  0 & 1  &|&  \frac{-3}{3+i} & \frac{1}{3+i} 
  \end{array} \right].
  \]

The inverse of $A$ can be found on the right-hand side of the above array: \[ A^{-1} = \begin{bmatrix} 1 + \frac{6}{3 + i} & - \frac{2}{3 + i}\nl - \frac{3}{3 + i} & \frac{1}{3 + i} \end{bmatrix}. \]

The effects of multiplying a vector $\vec{v} \in \mathbb{C}^n$ by a matrix $M \in \mathbb{C}^{m\times n}$ has the same effect as a linear transformation $T_M:\mathbb{C}^n \to \mathbb{C}^m$: \[ \vec{w} = M \vec{v} \qquad \Leftrightarrow \qquad \vec{w} = T(\vec{v}). \] The opposite is also true—any linear transformation $T$ can be represented as a matrix product: \[ \vec{w} = T(\vec{v}) \qquad \Leftrightarrow \qquad \vec{w} = M_T \vec{v}, \] for some matrix $M_T$. We will now illustrate the procedure for finding the matrix representation of a linear transformation with a simple example.

Consider the linear transformation $T:\mathbb{C}^2 \to \mathbb{C}^2$ which produces the following input-output pairs: \[ T\!\left( \begin{bmatrix} 1 \nl 0 \end{bmatrix} \right) = \begin{bmatrix} 3 \nl 2i \end{bmatrix}, \quad \textrm{and} \quad T\!\left( \begin{bmatrix} 0 \nl 2 \end{bmatrix} \right) = \begin{bmatrix} 2 \nl 4+4i \end{bmatrix}. \] Do you remember how you can use the information provided above to find the matrix representation $M_T$ of the linear transformation $T$ with respect to the standard basis?

To obtain the matrix representation of $T$ with respect to a given basis we have to combine, as columns, the outputs $T$ for the different elements of the basis: \[ M_T = \begin{bmatrix} | & | & \mathbf{ } & | \nl T(\vec{e}_1) & T(\vec{e}_2) & \dots & T(\vec{e}_n) \nl | & | & \mathbf{ } & | \end{bmatrix}, \] where $\{ \hat{e}_1,\hat{e}_2,\ldots, \hat{e}_n\}$ are the elements of the basis for the input space $\mathbb{R}^n$.

We know the value for the first column $T(\hat{e}_1)$ but we are not given the output of $T$ for the $\hat{e}_2$. This is OK though since we can use the fact that $T$ is a linear transformation ($T(\alpha \vec{v}) = \alpha T(\vec{v})$ which means that \[ T\!\left( 2 \begin{bmatrix} 0 \nl 1 \end{bmatrix} \right) = 2 \begin{bmatrix} 1 \nl 2+2i \end{bmatrix} \quad \Rightarrow \quad T\!\left( \begin{bmatrix} 0 \nl 1 \end{bmatrix} \right) = \begin{bmatrix} 1 \nl 2+2i \end{bmatrix}. \]

Thus we find the final answer \[ M_T= \begin{bmatrix} 3 & 1 \nl 2i & 2+2i \end{bmatrix}. \]

The main reason why I want you, my dear students, to learn about linear algebra with complex vectors is so that we can complete the important task of classifying the basic types of linear transformations in term of their eigenvalues. Recall that

1. projections obey $\Pi=\Pi^2$ and have eigenvalues zero or one
2. reflections have at least one eigenvalue equal to negative one

What kind of eigenvalues do rotations matrices have? The eigenvalues of a matrix $M$ are the roots of its characteristic polynomial $p_M(\lambda)=\textrm{det}(M - \lambda I)$. Thus, to find the eigenvalues of the rotation matrix $R_\theta$ we must solve the following equaiton \[ p_{R_\theta}(\lambda) =\textrm{det}(R_\theta - \lambda I) =\textrm{det}\left( \begin{bmatrix} \cos\theta -\lambda &-\sin\theta \nl \sin\theta &\cos\theta -\lambda \end{bmatrix} \right) =(\cos\theta - \lambda)^2+\sin^2\theta = 0. \]

To solve for $\lambda$ we first move $\sin^2\theta$ to the other side of the equation and then take the square root \[ \cos\theta-\lambda = \pm \sqrt{ - \sin^2 \theta } = \pm \sqrt{ - 1} \sin \theta = \pm i\sin\theta. \] The eigenvalues are $\lambda_1 = \cos\theta + i \sin\theta$ and $\lambda_2 = \cos\theta - i \sin\theta$. Note that by using Euler's equation we can also write the eigenvalues as $\lambda_1 = e^{i\theta}$ and $\lambda_2 =e^{-i\theta}$. All of a sudden, complex numbers show up out of nowhere! This is not a coincidence: complex exponentials are in many ways the natural way to talk about rotations, periodic motion, and waves.

If you pursue a career in math, physics or engineering you will no doubt run into complex numbers and Euler's equation many more times. In this case what is interesting is that complex numbers come out as answers to a problem that was stated strictly in terms of real variables.

We now define some special types of matrices which describe matrices with complex coefficients.

Let $V$ be a complex space on which an inner product is defined. Than a linear transformation $U$ is unitary if $U^\dagger U={11}$. It has determinant $|\det(U)|=1$.

For a $n\times n$ matrix $U$ the following statements are equivalent:

1. $U$ is unitary
2. The columns of $U$ are an orthonormal set
3. The rows of $U$ are an orthonormal set
4. The inverse of $U$ is $U^\dagger$

Unitary matrices are the complex analogues of the orthogonal matrices. Indeed, if a unitary matrix $U$ has real coefficients then $U^\dagger = U^T$ and we have $U^TU={11}$, which is the definition of an orthogonal matrix.

A Hermitian matrix $H$ is the complex analogue of the symmetric matrix: \[ H^T = H, \qquad h_{ij} = \overline{ h_{ji}}, \quad \text{ for all } i,j. \] The eigenvalues of a Hermitian matrix are all real.

A Hermitian matrix $H$ can be freely moved from one side to the other in a dot product calculation: \[ \langle H\vec{x},\vec{y}\rangle =(H\vec{x})^\dagger\vec{y} =\vec{x}^\dagger H^\dagger \vec{y} =\vec{x}^\dagger \: (H\vec{y}) =\langle\vec{x},H\vec{y}\rangle. \]

We defined the set of real normal matrices to be matrices that satisfy $A^TA=AA^T$. For matrices with complex coefficients, the definition of a normal matrix uses the dagger operation instead: $AA^\dagger = A^\dagger A$.

The inner product is defined in terms of the matrix product of the vector $\vec{u}^T$ (a row vector) and the column vector $\vec{v}$. We saw that extending the notion of inner product to work with complex vectors, required that we modify the formula for the inner product slightly. The complex inner product is an operation of the form: \[ \langle \cdot, \cdot \rangle : \mathbb{C}^n \times \mathbb{C}^n \to \mathbb{C}. \] The inner product for vectors $\vec{u},\vec{v} \in \mathbb{C}^n$ is defined by \[ \langle \vec{u},\vec{v}\rangle \equiv \sum_{i=1}^n \overline{u_i} v_i \equiv \vec{u}^\dagger \vec{v}. \] The formula is similar, but we use the Hermitian transpose $\dagger$ on the first vector instead of the regular transpose $^T$.

This dagger thing is very important actually. It is an operation that is close to my heart as it pertains to quantum mechanics, Hilbert space, and probabilities computed as dot products. If we want to preserve the connection between length and dot product we need to use the complex conjugation. For column vectors $\vec{u},\vec{v} \in \mathbb{C}^n$, we have: \[ \vec{u}\cdot \vec{v} = \bar{u}_1v_1 + \bar{u}_2v_2 + \bar{u}_3v_3 = \left[\begin{array}{ccc} \bar{u}_{1} & \bar{u}_{2} & \bar{u}_{3} \nl \end{array}\right] \left[\begin{array}{c} v_1 \nl v_2 \nl v_3 \end{array}\right] = \vec{u}^\dagger\vec{v} \]

Using this definition of the dot product, for $\vec{v} \in \mathbb{R}^n$, we get $|\vec{v}| \equiv \sqrt{\vec{v}\cdot\vec{v}} =\sqrt{ |v_1|^2 + |v_2|^2 + |v_3|^2}$, where $|v_i|^2 = \bar{v}_iv_i$ is the magnitude of the complex coefficient $v_i \in \mathbb{C}$.

The inner product of input vectors induces the following norm for complex vectors: \[ \|\vec{v}\| = \sqrt{ \vec{v}^\dagger\vec{v} } = \sqrt{ \sum_{i=1}^n |v_i|^2 } = \sqrt{ \sum_{i=1}^n \overline{v_i}v_i }. \]

TODO: add an example

Recall that an inner product space is some vector space $V$ for which we have defined an inner product operation $\langle \mathbf{u} , \mathbf{v} \rangle$ which has (1) a symmetric property, (2) a linearity property and (3) a non negativity property.

linear linearity The complex inner product on a complex vector space is defined as follows: for all $\mathbf{u}, \mathbf{v}, \mathbf{v}_1,\mathbf{v}_2\in V$ and $\alpha,\beta \in\mathbb{C}$.

1. $\langle \mathbf{u},\mathbf{v}\rangle =\overline{\langle \mathbf{v},\mathbf{u}\rangle }$,
2. $\langle \mathbf{u},\alpha\mathbf{v}_1+\beta\mathbf{v}_2\rangle =\alpha\langle \mathbf{u},\mathbf{v}_1\rangle +\beta\langle \mathbf{u},\mathbf{v}_2\rangle $
3. $\langle \mathbf{u},\mathbf{u}\rangle \geq0$ for all $\mathbf{u}\in V$, $\langle \mathbf{u},\mathbf{u}\rangle =0$ if and only if $\mathbf{u}=\mathbf{0}$.

Note that, because of the conjugate symmetric property $\langle \mathbf{u},\mathbf{v}\rangle =\overline{\langle \mathbf{v},\mathbf{u}\rangle }$, the inner product of a vector with itself must be a real number: $\langle \mathbf{u},\mathbf{u}\rangle = \overline{\langle \mathbf{u},\mathbf{u}\rangle } \in\mathbb{R}$.

The Hilbert-Schmidt inner product \[ \langle A, B \rangle_{\textrm{HS}} = \textrm{Tr}\!\left[ A^\dagger B \right]. \]

Hilbert-Schmidt norm \[ ||A||_{\textrm{HS}} \equiv \sqrt{ \langle A, A \rangle } = \sqrt{ \textrm{Tr}\!\left[ A^\dagger A \right] } = \left[ \sum_{i,j=1}^{n} |a_{ij}|^2 \right]^2. \]

The matrix decompositions we leaned about in Section~\ref{sec:matrix_decompositions} can be applied to matrices with complex entries. Below we give a

TODO: check others

The singular value decomposition of an $m \times n$ complex matrix $M$ is a way to write $M$ as a diagonal matrix $\Sigma$ surrounded by matrices of left eigenvectors and right eigenvectors: \[ M = U\Sigma V^\dagger. \] where

TODO: copy details from paper, check for consistency with Section~\ref{sec:matrix_decompositions}

The characteristic polynomial of the rotation matrix $R_\theta$ is $p(\lambda)=(\cos\theta - \lambda)^2+\sin^2\theta=0$. The eigenvalues are $\lambda_1 = \cos\theta + i \sin\theta = e^{i\theta}$ and $\lambda_2 = \cos\theta - i \sin\theta=e^{-i\theta}$. What are its eigenvectors?

Before we go into the calculation I want to show you a useful trick for rewriting $\cos$ and $\sin$ expressions in terms of the complex exponential function. Recall Euler's equation $e^{i\theta} = \cos\theta + i \sin\theta$. Using this equation and the analogous expression for $e^{-i\theta}$, we can obtain the following expressions for $\cos\theta$ and $\sin\theta$: \[ \cos\theta = \frac{1}{2}\left( e^{i\theta} + e^{-i\theta} \right), \qquad \sin\theta = \frac{1}{2i}\left( e^{i\theta} - e^{-i\theta} \right). \] Try calculating the right-hand side in each case to verify the accuracy of each expression. These formulas are useful because they allow us to rewrite expression of the form $e^{i\theta}\cos\phi$ as $e^{i\theta}\frac{1}{2}\left( e^{i\phi} + e^{-i\phi} \right) = \frac{1}{2}\left( e^{i(\theta+\phi)} + e^{i(\theta-\phi)} \right)$.

Let us now see how to calculate the eigenvector $\vec{e}_{1}$ which corresponds to the eigenvalue $\lambda_1 = e^{i\theta}$. The eigenvalue equation for the eigenvalue $\lambda_1 = e^{i\theta}$ is \[ R_\theta \vec{e}_1 = e^{i\theta} \vec{e}_1, \qquad \begin{bmatrix} \cos\theta &-\sin\theta \nl \sin\theta &\cos\theta \end{bmatrix} \begin{bmatrix} \alpha \nl \beta \end{bmatrix} = e^{i\theta} \begin{bmatrix} \alpha \nl \beta \end{bmatrix}. \] We are looking for the coefficients $\alpha, \beta$ of the eigenvector $\vec{e}_1$.

Do you remember how to go about finding these coefficients? Wasn't there some sort of algorithm for finding the eigenvector(s) which correspond to a given eigenvalue? Don't worry if you have forgotten. This is why we are having this review chapter. We will go over the problem in details.

The “finding the eigenvector(s) of $M$ for the eigenvalue $\lambda_1$” problem is solved by calculating the null space of the matrix $(M-\lambda_1 I)$. Indeed, we can rewrite the eigenvalue equation stated above as: \[ (R_\theta - e^{i\theta}I) \vec{e}_1 = 0, \qquad \begin{bmatrix} \cos\theta - e^{i\theta} &-\sin\theta \nl \sin\theta &\cos\theta - e^{i\theta} \end{bmatrix} \begin{bmatrix} \alpha \nl \beta \end{bmatrix} = \begin{bmatrix} 0 \nl 0 \end{bmatrix}, \] in which it is clear the finding-the-eigenvectors procedure corresponds to a null space calculation.

We can now use the trick described above and rewrite the expression which appears twice on the main diagonal of the matrix as: \[ \begin{align*} \cos\theta - e^{i\theta} &= \frac{1}{2}\left(e^{i\theta} + e^{-i\theta} \right) \ - e^{i\theta} \nl & = \frac{1}{2}e^{i\theta} + \frac{1}{2}e^{-i\theta} - e^{i\theta} = \frac{-1}{2}e^{i\theta} + \frac{1}{2}e^{-i\theta} = \frac{-1}{2}\left(e^{i\theta} - e^{-i\theta} \right) \nl &= -i \frac{1}{2i}\left(e^{i\theta} - e^{-i\theta} \right) = -i\sin\theta. \end{align*} \]

TODO: finish steps

For matrices with complex entries we define the Hermitian transpose (denoted $\dagger$ by physicists, and $*$ by mathematicians) which, in addition to taking the transpose of a matrix, also takes the complex conjugate of each entry: $a_{ij}^\dagger=\bar{a}_{ji}$.

The Hermitian transpose has the following properties: \[ \begin{align} (A+B)^\dagger &= A^\dagger + B^\dagger \nl (AB)^\dagger &= B^\dagger A^\dagger \nl (ABC)^\dagger &= C^\dagger B^\dagger A^\dagger \nl (A^\dagger)^{-1} &= (A^{-1})^\dagger \end{align} \]

Note that these are the same properties as the regular transpose operation, we just have an extra

We defined the complex inner product as linear in the second component and conjugate-linear in the first component: \[ \begin{align*} \langle\vec{v}, \alpha\vec{a}+ \beta\vec{b} \rangle &= \alpha\langle\vec{v},\vec{a}\rangle+ \beta\langle\vec{v}, \vec{b}\rangle, \nl \langle\alpha\vec{a}+\beta\vec{b}, \vec{w} \rangle &= \overline{\alpha}\langle\vec{a}, \vec{w}\rangle + \overline{\beta}\langle\vec{b}, \vec{w}\rangle. \end{align*} \] You will want to keep that in mind every time you deal with complex inner products. The inner complex inner product is not symmetric since it requires that the complex conjugation be perfumed on the first input. Remember that $\langle\vec{v}, \vec{w} \rangle \neq \langle \vec{w}, \vec{v}\rangle$, instead we have $\langle\vec{v}, \vec{w} \rangle = \overline{ \langle \vec{w}, \vec{v}\rangle}$.

Note that the choice of complex conjugation in the first entry is a matter of convention. In this text we defined the inner product $\langle \cdot, \cdot \rangle$ with the $\dagger$ operation on the first entry, which is known as the physics convention. Some mathematics texts define the inner product of complex vectors using the complex conjugation on the second entry, which would make the inner product linear in the first entry and conjugate-linear in the second entry. That is fine too. The choice of convention doesn't matter so long as one of the entries is conjugated in order to ensure $\langle \vec{u}, \vec{u} \rangle \in \mathbb{R}$.

In the section on inner product spaces we discussed the notion of the vector space of all functions of a real variable $f:\mathbb{R} \to \mathbb{R}$ and the product between two functions was defined as \[ \langle \mathbf{f},\mathbf{g}\rangle =\int_{-\infty}^\infty f(t) g(t)\; dt. \]

We can Given two complex functions $\mathbf{f}=f(t)$ and $\mathbf{g}=g(t)$: \[ f\colon \mathbb{R} \to \mathbb{C}, \qquad g\colon \mathbb{R} \to \mathbb{C}, \] and define their inner product as follows: \[ \langle \mathbf{f},\mathbf{g}\rangle =\int_{-\infty}^\infty \overline{f(t)} g(t)\; dt \] This formula is the complex valued version of the function inner product. The conjugation on one of the entries in the product ensures that the inner product always results in a real number. The function inner product measures the overlap between $\mathbf{f}$ and $\mathbf{g}$.

We can carry out linear algebra calculations over any field. A field is a set of numbers for which an addition, subtraction, multiplication, and division operation are defined. The addition and multiplication operations we define must be and associative and commutative, and multiplication is distributive over addition. Furthermore a field must contain an additive identity element (denoted $0$) and a multiplicative identity element (denoted $1$). The properties of a field are essentially all the properties of the numbers you are familiar.

The focus of our discussion in this section was to show that the linear algebra techniques we learned for manipulating real coefficients work equally well with the complex numbers. This shouldn't be too surprising since, after all, linear algebra manipulations boil down to arithmetic manipulations of the coefficients of vectors and matrices. Since both real numbers and complex numbers can be added, subtracted, multiplied, and divided, we can do linear algebra over both fields.

We can also do linear algebra over finite fields. A finite field is a set $F_q \equiv \{ 0,1,2, \ldots, q-1\}$, where $q$ is prime number or the power of a prime number. All the arithmetic operations in this field are performed modulo the number $q$. If the result of operation is outside the field, you either add or subtract $q$ until the number falls in the range $[0,1,2, \ldots, q-1]$. Consider the finite field $F_5 =\{0,1,2,3,4\}$. To add two numbers in $F_5$ we proceed as follows: $3 + 3 \ \bmod \ 5 = 6 \ \bmod \ 5 = 1 \ \bmod \ 5 = 1$. Similarly for subtraction $1-4 \ \bmod \ 5 = (-3) \ \bmod \ 5 = 2 \ \bmod \ 5 = 2$.

The field of binary numbers $F_2 \equiv \{ 0,1 \}$ of an important finite field which is used in many areas of communication engineering and cryptography. Each data packet that your cellular phone sends over the airwaves is first encoded using an error correcting code. The encoding operation essentially consists of a matrix-vector product where the calculation is carried out over $F_2$.

The field of rational numbers $\mathbb{Q}$ is another example of a field which is often used in practice. Solving systems of equations over using the rational numbers on computers is interesting because the answers obtained are exact—we can avoid many of the numerical accuracy problems associated with floating point arithmetic.

The hidden agenda I had in mind is the following: understanding linear algebra over the complex field means you understand quantum mechanics. Quantum mechanics unfolds in a complex inner product space (Hilbert space) and the “mysterious” quantum effects are not mysterious at all: quantum operations are represented as matrices and quantum measurements are projection operators. Thus, if you understood the material in this section, you should be able to pick up any book on quantum mechanics and you will feel right at home!

Calculate $(2+5i)-(3+4i)$, $(2+5i)(3+4i)$ and $(2+5i)/(3+4i)$.