\documentclass[11pt]{article}

\usepackage{systeme}

\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
%\usepackage{palatino}
\usepackage{setdeck}

\usepackage{xcolor}

%\usepackage[textheight=8.5in, margin=1.6in]{geometry}
\usepackage[total={6in,8in}]{geometry}

\usepackage{tikz}
\usetikzlibrary{cd}
\tikzset{
      labr/.style={anchor=north, rotate=90, inner sep=.7mm}
    }
\tikzset{
      labl/.style={anchor=south, rotate=90, inner sep=.7mm}
    }

%\usepackage{pgfplots}
%\tikzstyle{ball} = [circle,shading=ball, ball color=black,
%    minimum size=1mm,inner sep=1.3pt]
%\usetikzlibrary{arrows.meta}
%\tikzset{>={Latex[length=2mm,width=1.5mm]}}

%\usepackage{graphicx}

\usepackage{needspace}  % <-- to prevent bad pagebreak over environment titles
% Use:  \Needspace{3\baselineskip} before opening an environment

\usepackage[shortlabels]{enumitem}
\setlist[enumerate]{leftmargin=*,label=(\alph*)}

%\usepackage[colorlinks=true,citecolor=black,linkcolor=black,urlcolor=blue]{hyperref}

\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\C}{\mathbb{C}}
\newcommand{\sn}{\mathfrak{S}}
\newcommand{\ve}{\varepsilon}
\newcommand{\ips}[2]{\langle{#1}, {#2}\rangle}
\DeclareMathOperator{\tr}{tr}
\DeclareMathOperator{\Span}{\mathrm{Span}}
\DeclareMathOperator{\rk}{\mathrm{rank}}
\DeclareMathOperator{\rspace}{\mathrm{rowspace}}
\DeclareMathOperator{\cspace}{\mathrm{colspace}}
\DeclareMathOperator{\trace}{\mathrm{tr}}
\DeclareMathOperator{\im}{\mathrm{im}}
\DeclareMathOperator{\Sym}{\mathrm{Sym}}
\DeclareMathOperator{\Hom}{\mathrm{Hom}}
\DeclareMathOperator{\diag}{\mathrm{diag}}
\DeclareMathOperator{\glb}{\mathrm{glb}}
\DeclareMathOperator{\lub}{\mathrm{lub}}

\parskip = 0.05in
\parindent = 0.0in

%\pagestyle{empty}

\begin{document}
\centerline{Math 201 lecture for Monday, Week 7}
\bigskip

\centerline{\textbf{Matrix inversion}}
\medskip

Last time, we defined matrix multiplication: if~$A$ is an~$m\times p$ matrix
and~$B$ is a $p\times n$ matrix, then~$AB$ is the $m\times n$ matrix with
~$i,j$-entry
\[
  (AB)_{ij}:=\sum_{k=1}^{p}A_{ik}B_{kj}.
\]
If~$m=n$, then~$BA$ would also be defined, but it is usually that case
that~$AB\neq BA$.  Another peculiar thing is that for matrices, there are ``zero
divisors'', i.e., matrices~$A,B$ such that~$AB=0$, but neither~$A$ nor~$B$ is a
zero matrix.  For example,
\[
  \left(\begin{array}{cc}
      0&0\\
      0&1
  \end{array} \right)
  \left(\begin{array}{cc}
      0&1\\
      0&0
  \end{array} \right)
  =
  \left(\begin{array}{cc}
      0&0\\
      0&0
  \end{array} \right).
\]

{\bf Diagonal matrices}. The matrix~$A$ is a {\em diagonal matrix} if its only
nonzero entries appear along the diagonal: $A_{ij}=0$ if~$i\neq j$.  This
terminology makes sense regardless of the dimensions of~$A$, but is usually used
in the case of square matrices, i.e., for the case where~$A$ is an~$n\times n$
matrix.  In that case, we write
\[
  A = \mathrm{diag}(a_1,\dots,a_n)
\]
where~$A_{ii}=a_i$ for~$i=1\dots,n$ (and~$A_{ij}=0$, otherwise.).  For instance,
\[
  \mathrm{diag}(1,4,0,6)=
  \left(\begin{array}{cccc}
      1&0&0&0\\
      0&4&0&0\\
      0&0&0&0\\
      0&0&0&6\\
  \end{array} \right).
\]

{\bf Identity matrices.}  The~$n\times n$ {\em identity matrix} is the~$n\times n$ matrix
\[
  I_n=\diag(1,\dots,1).
\]
It has the following property:~$AI_n=A$ and~$I_nB=B$ whenever these products make
sense.  For instance,

\[
  \left(\begin{array}{ccc}
      1&2&3\\
      4&5&6
  \end{array} \right)
  \left(\begin{array}{ccc}
      1&0&0\\
      0&1&0\\
      0&0&1
  \end{array} \right)
  =
  \left(\begin{array}{ccc}
      1&2&3\\
      4&5&6
  \end{array} \right)
\]
and
\[
  \left(\begin{array}{ccc}
      1&0&0\\
      0&1&0\\
      0&0&1
  \end{array} \right)
  \left(\begin{array}{cc}
      1&2\\
      3&4\\
      5&6
  \end{array} \right)
  =
  \left(\begin{array}{cc}
      1&2\\
      3&4\\
      5&6
  \end{array} \right).
\]

{\bf Inverses.}  Let~$A$ be an~$m\times n$ matrix, and let~$B$ be an~$n\times m$
matrix.  If~$AB=I_n$, we say~$A$ is a {\em left-inverse} for~$B$ and~$B$ is a
{\em right-inverse} for~$A$.  For example,
\[
A =
\left(\begin{array}{ccc}
    1&1&1\\
    0&1&1
\end{array} \right)
\quad\text{and}\quad
B=
\left(\begin{array}{rr}
    1&-1\\
    0&0\\
    0&1
\end{array} \right).
\]
Then
\[
AB=
\left(\begin{array}{ccc}
    1&1&1\\
    0&1&1
\end{array} \right)
\left(\begin{array}{rr}
    1&-1\\
    0&0\\
    0&1
\end{array} \right)
=
\left(\begin{array}{cc}
    1&0\\0&1
\end{array} \right).
\]
Hence,~$A$ is a left-inverse for~$B$ and~$B$ is a right-inverse for~$A$.
On the other hand,
\[
BA=
\left(\begin{array}{rr}
    1&-1\\
    0&0\\
    0&1
\end{array} \right)
\left(\begin{array}{ccc}
    1&1&1\\
    0&1&1
\end{array} \right)
=
\left(\begin{array}{ccc}
    1&0&0\\
    0&0&0\\
    0&1&1\\
\end{array} \right).
\]
So~$B$ is not a left-inverse of~$A$ and~$A$ is not a right-inverse of~$B$.  (In
  fact,~$B$ does not have a left-inverse and~$A$ does not have a right-inverse.
  This has to do with their ranks not being high enough. The connection with
solving systems of equations we describe below explains that.)

We will mainly be interested in inverses for square matrices.  Suppose
that~$A$ is an~$n\times n$ matrix.  Suppose~$B$ is a right-inverse.
So~$B$ is an~$n\times n$ matrix such that~$AB=I_n$.  Since matrix multiplication
is not commutative, the value of~$BA$ is not immediately clear.  However, in
fact, we have the following important result:
\medskip

{\bf Theorem.}  Let~$A$ and~$B$ be~$n\times n$ matrices.  The following are
equivalent:
\begin{enumerate}[noitemsep]
  \item $AB=I_n$.
  \item $BA=I_n$.
\end{enumerate}
If~$AB=I_n$, we say~$A$ and~$B$ are {\em invertible} and write~$A^{-1}=B$ and~$B^{-1}=A$.
The following are equivalent:
\begin{enumerate}[noitemsep,label=(\roman*)]
  \item $A$ is invertible.
  \item $\rk(A)=n$.
  \item The reduced echelon form of~$A$ is~$I_n$.
\end{enumerate}
\medskip

The proof of this theorem will follow from an elegant algorithm for computing
the inverse of a matrix which we present below.  The equivalence of the last to
items on the list is something we already know.

\textbf{Calculating the inverse.}  Our problem now is to determine whether an
inverse for a matrix exists, and if so, to calculate that inverse.  The methods
we present here would also be applicable to calculating right- and left-inverses
of non-square matrices---it boils down to solving systems of linear equations,
after all---but we will concentrate on the case of square matrices.

\textbf{Example.} Let
\[
A = 
\left(\begin{array}{rrr}
    0&3&-1\\
    1&0&1\\
    1&-1&0
\end{array} \right).
\]
A right-inverse for~$A$ would satisfy the following:
\[
\left(\begin{array}{rrr}
    0&3&-1\\
    1&0&1\\
    1&-1&0
\end{array} \right)
\left(\begin{array}{ccc}
    a&b&c\\
    d&e&f\\
    g&h&i
\end{array} \right)
=
\left(\begin{array}{ccc}
    1&0&0\\
    0&1&0\\
    0&0&1
\end{array} \right).
\]
So we need to find the entries~$a,b,\dots,i$. We can break this into three
problems:
\begin{align*}
\left(\begin{array}{rrr}
    0&3&-1\\
    1&0&1\\
    1&-1&0
\end{array} \right)
\left(\begin{array}{c}
    a\\
    d\\
    g
\end{array} \right)
&=
\left(\begin{array}{c}
    1\\0\\0
\end{array} \right)\\[15pt]
\left(\begin{array}{rrr}
    0&3&-1\\
    1&0&1\\
    1&-1&0
\end{array} \right)
\left(\begin{array}{c}
    b\\
    e\\
    h
\end{array} \right)
&=
\left(\begin{array}{c}
    0\\1\\0
\end{array} \right)\\[15pt]
\left(\begin{array}{rrr}
    0&3&-1\\
    1&0&1\\
    1&-1&0
\end{array} \right)
\left(\begin{array}{c}
    c\\
    f\\
    i
\end{array} \right)
&=
\left(\begin{array}{c}
    0\\0\\1
\end{array} \right).
\end{align*}
Equivalently, we need to solve three systems of linear equations:
\begin{align*}
  0x+3y-z&=1\\
  x+0y+z&=0\\
  x-y+0z&=0\\[20pt]
  0x+3y-z&=0\\
  x+0y+z&=1\\
  x-y+0z&=0\\[20pt]
  0x+3y-z&=0\\
  x+0y+z&=0\\
  x-y+0z&=1
\end{align*}
Their augmented matrices would like:
\[
 \left(\begin{array}{rrr|c}
      0&3&-1&1\\
      1&0&1&0\\
      1&-1&0&0
  \end{array} \right),
  \quad
 \left(\begin{array}{rrr|c}
      0&3&-1&0\\
      1&0&1&1\\
      1&-1&0&0
  \end{array} \right),
  \quad
 \left(\begin{array}{rrr|c}
      0&3&-1&0\\
      1&0&1&0\\
      1&-1&0&1
  \end{array} \right).
\]
The row operations needed to determine the solvability of this system are the
same in all three cases.  So we can combine all three of these systems at once
in one ``super''-augmented matrix calculation:
\begin{align*}
 \left(\begin{array}{rrr|ccc}
      0&3&-1&1&0&0\\
      1&0&1&0&1&0\\
      1&-1&0&0&0&1
  \end{array} \right)
&\xrightarrow{r_1\leftrightarrow r_2}
 \left(\begin{array}{rrr|ccc}
      1&0&1&0&1&0\\
      0&3&-1&1&0&0\\
      1&-1&0&0&0&1
  \end{array} \right)\\[15pt]
&\xrightarrow{r_3\to r_3-r_1}
 \left(\begin{array}{rrr|ccc}
      1&0&1&0&1&0\\
      0&3&-1&1&0&0\\
      0&-1&-1&0&-1&1
  \end{array} \right)\\[15pt]
  &\xrightarrow[r_3\to-r_3]{r_2\leftrightarrow r_3}
 \left(\begin{array}{rrr|ccc}
      1&0&1&0&1&0\\
      0&1&1&0&1&-1\\
      0&3&-1&1&0&0
  \end{array} \right)\\[15pt]
  &\xrightarrow{r_3\to r_3-3r_2}
 \left(\begin{array}{rrr|ccc}
      1&0&1&0&1&0\\
      0&1&1&0&1&-1\\
      0&0&-4&1&-3&3
  \end{array} \right)\\[15pt]
  &\xrightarrow{r_3\to -r_3/4}
 \left(\begin{array}{rrr|ccc}
      1&0&1&0&1&0\\
      0&1&1&0&1&-1\\
      0&0&1&-1/4&3/4&-3/4
  \end{array} \right)\\[15pt]
  &\xrightarrow[r_2\to r_2-r_3]{r_1\to r_1-r_3}
 \left(\begin{array}{rrr|ccc}
      1&0&0&1/4&1/4&3/4\\
      0&1&0&1/4&1/4&-1/4\\
      0&0&1&-1/4&3/4&-3/4
  \end{array} \right).
\end{align*}
Going back to the original systems of equations, we see that we need
\[
  \left(\begin{array}{c}
      a\\d\\g
  \end{array} \right)
  =
  \left(\begin{array}{r}
      1/4\\1/4\\-1/4
  \end{array} \right),
  \quad
  \left(\begin{array}{c}
      b\\e\\h
  \end{array} \right)
  =
  \left(\begin{array}{c}
      1/4\\1/4\\3/4
  \end{array} \right),
  \quad
  \left(\begin{array}{c}
      c\\d\\i
  \end{array} \right)
  =
  \left(\begin{array}{r}
      3/4\\-1/4\\-3/4
  \end{array} \right).
\]
In other words, the following matrix is a right-inverse for~$A$:
\[
 \left(\begin{array}{rrr}
      1/4&1/4&3/4\\
      1/4&1/4&-1/4\\
      -1/4&3/4&-3/4
  \end{array} \right).
\]

The argument we've just given for a particular matrix easily generalizes to give
the following algorithm.

\textbf{Algorithm for computing the inverse of a matrix.} Let~$A$ be an~$n\times n$
matrix.  Perform row operations on the ``super''-augmented matrix ~$[A\mid I_n]$
to compute the reduced echelon form of~$A$:
\[
  \left( A\mid I_n \right)
  \longrightarrow
  \left( \tilde{A}\mid B \right).
\]
There are two possibilities: either~$\tilde{A}=I_n$ or not.
If~$\tilde{A}=I_n$, then~$B=A^{-1}$.  Next, we consider what happens
when~$\tilde{A}\neq I_n$. Since~$B$ is derived by performing
row operations on~$I_n$, we have~$\rk(B)=\rk(I_n)=n$.  Thus,~$B$ cannot have a
row of zeros.  If~$\tilde{A}\neq I_n$, it must have a row of zeros.  It follows
that the system of equations is inconsistent, and~$A$ has no inverse.  

Now suppose that~$\rk(A)$ so that
\begin{equation}\label{eqn:07monq}
  \left( A\mid I_n \right)
  \longrightarrow
  \left( I_n\mid B \right)
\end{equation}
and~$AB=I_n$.  Consider trying to find~$C$ so that~$BC=I_n$.  In this case,
reverse the row operations in~\eqref{eqn:07monq} to get
\[
  \left( B\mid I_n \right)
  \longrightarrow
  \left( I_n\mid A \right),
\]
and thus, $C=A$, i.e., $BA=I_n$.

In summary:
\begin{itemize}
  \item If~$\tilde{A}=I_n$ (equivalently,~$\rk(A)=n$) then~$AB=BA=I_n$.
    (So~$B=A^{-1}$ and $A=B^{-1}$.)
  \item If~$\tilde{A}\neq I_n$ (equivalently,~$\rk(A)<n$),
    then~$\tilde{A}$ has a row of zeros and~$A$ has no inverse.
\end{itemize}

In particular: $A\in M_{n\times n}$ is invertible if and only if~$\rk(A)=n$.
\end{document}
