A brief primer on block-diagonal matrices

A square matrix is block-diagonal if it has the form

$$\left[\begin{array}{cccc}{B}_1& & & \\ & B_2& & \\ & & \ddots & \\ & & & B_k\end{array}\right]$$

where the blocks $B_i$ are square sub-matrices centered on the main diagonal, and everything outside the blocks is 0. Here is an example of a $5\times 5$ block-diagonal matrix with a $2\times 2$ block and a $3\times 3$ block:

$$\left[\begin{array}{ccccc}1& 2& 0& 0& 0\\ 3& 4& 0& 0& 0\\ 0& 0& 5& 6& 7\\ 0& 0& 8& 9& 10\\ 0& 0& 11& 12& 13\end{array}\right].$$

Block-diagonal matrices are extremely useful because they represent linear transformations that can be decomposed into simpler linear transformations. Consider again the generic block-diagonal matrix

$$B=\left[\begin{array}{cccc}{B}_1& & & \\ & B_2& & \\ & & \ddots & \\ & & & B_k\end{array}\right].$$

For each $1\le i\le k$, let ${\ell }_i$ be the length of the block $B_i$ (i.e. $B_i$ is a ${\ell }_i\times {\ell }_i$ matrix). Then $B$ is an $n\times n$ matrix where $n=\sum _{i=1}^k{\ell }_i$. For each $x\in F^n$ (where $F$ is $\mathbb{R}$ or $\mathbb{C}$ or your favourite field), we can write

$$x=\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_k\end{array}\right]$$

where $x_1\in F^{{\ell }_1}$ consists of the first ${\ell }_1$ components of $x$, $x_2\in F^{{\ell }_2}$ consists of the next ${\ell }_2$ components of $x$, and so on. The key fact is that

$$Bx=\left[\begin{array}{cccc}{B}_1& & & \\ & B_2& & \\ & & \ddots & \\ & & & B_k\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_k\end{array}\right]=\left[\begin{array}{c}{B}_1{x}_1\\ B_2{x}_2\\ ⋮\\ B_k{x}_k\end{array}\right].$$

Each block $B_i$ independently affects its own chunk of components $x_i$, and the final product $Bx$ is equal to the direct sum of the vectors $B_i{x}_i$. Hence, we often say that $B$ is the direct sum of the $B_i$.

Computing all the $B_i{x}_i$ and then “concatenating” the results into one giant vector is easier than computing $Bx$ directly without taking advantage of the blocks. The smaller that each block is, the fewer arithmetic operations are required to compute $Bx$. This is most obvious when each block is a $1\times 1$ matrix (i.e. a single number), in which case $B$ is just a diagonal matrix and $Bx$ is very easy to compute.

Computing $det(B)$ is also easier using the $B_i$—it turns out that $$det(B)=det(B_1)det(B_2)\cdots det(B_k).$$ A short and easy proof can be found on Math StackExchange. This determinant formula also implies that the characteristic polynomial of $B$ is the product of the characteristic polynomials of the $B_i$ (because $B-\lambda$ is also a block-diagonal matrix with blocks $B_i-\lambda$).

Finally, if

$$C=\left[\begin{array}{cccc}{C}_1& & & \\ & C_2& & \\ & & \ddots & \\ & & & C_k\end{array}\right]$$

is a block-diagonal matrix with the same block sizes as $B$ (i.e. $B_i$ and $C_i$ have the same dimensions for all $1\le i\le k$), then the product $BC$ is what you would probably expect:

$$BC=\left[\begin{array}{cccc}{B}_1{C}_1& & & \\ & B_2{C}_2& & \\ & & \ddots & \\ & & & B_k{C}_k\end{array}\right].$$