2.5 Matrix factorizations (矩阵因式分解)

本文为《Linear algebra and its applications》的读书笔记

目录

• The LU Factorization LU分解
• An LU Factorization Algorithm
• A Matrix Factorization in Electrical Engineering
• The QR Factorization QR分解
• Appendix: Permuted LU Factorizations 置换LU分解

A $factorization (因式分解)$factorization(因式分解) of a matrix $A$A is an equation that expresses $A$A as a product of two or more matrices. Whereas matrix multiplication involves a $synthesis$synthesis of data (数据的综合) (combining the effects of two or more linear transformations into a single matrix). In the language of computer science, the expression of $A$A as a product amounts to a preprocessing of the data in $A$A, organizing that data into two or more parts whose structures are more useful in some way, perhaps more accessible for computation(便于计算).

The LU Factorization LU分解

The LU factorization is motivated by the fairly common industrial and business problem of solving a sequence of equations, all with the same coefficient matrix:
$A\boldsymbol x = \boldsymbol b_1,A\boldsymbol x = \boldsymbol b_2,...,A\boldsymbol x = \boldsymbol b_p \ \ \ \ \ (1)$Ax=b1​,Ax=b2​,...,Ax=bp​     (1)When $A$A is invertible, one could compute $A^{-1}$A−1 and then compute $A^{-1}\boldsymbol b_1$A−1b1​, $A^{-1}\boldsymbol b_2$A−1b2​, and so on. However, it is more efficient to solve the first equation in sequence (1) by row reduction and obtain an LU factorization of $A$A at the same time. Thereafter, the remaining equations in sequence (1) are solved with the LU factorization.

At first, assume that $A$A is an $m \times n$m×n matrix that can be row reduced to echelon form, $without\ row\ interchanges$without row interchanges. (Later, we will treat the general case.) Then $A$A can be written in the form $A = LU$A=LU, where $L$L is an $m \times m$m×m lower triangular matrix with 1’s on the diagonal and $U$U is an $m \times n$m×n echelon form of $A$A. For instance, see Figure 1. Such a factorization is called an $LU$LU factorization of $A$A. The matrix $L$L is invertible and is called a $unit$unit lower triangular matrix(单位下三角矩阵).

Before studying how to construct $L$L and $U$U, we should look at why they are so useful. When $A = LU$A=LU, the equation $A\boldsymbol x = \boldsymbol b$Ax=b can be written as $L(U\boldsymbol x)=\boldsymbol b$L(Ux)=b. Writing $\boldsymbol y$y for $U\boldsymbol x$Ux, we can find $\boldsymbol x$x by solving the pair of equations

First solve $L\boldsymbol y = \boldsymbol b$Ly=b for $\boldsymbol y$y; and then solve $U\boldsymbol x =\boldsymbol y$Ux=y for $\boldsymbol x$x. See Figure 2. Each equation is easy to solve because $L$L and $U$U are triangular.

EXAMPLE 1
It can be verified that

Use this LU factorization of $A$A to solve $A\boldsymbol x = \boldsymbol b$Ax=b, where $\boldsymbol b =\begin{bmatrix} -9\\5\\7\\11\end{bmatrix}$b=⎣⎢⎢⎡​−95711​⎦⎥⎥⎤​.
SOLUTION The solution of $L\boldsymbol y = \boldsymbol b$Ly=b needs only 6 multiplications and 6 additions.

Then, for $U\boldsymbol x = \boldsymbol y$Ux=y, the “backward” phase of row reduction requires 4 divisions, 6 multiplications,
and 6 additions. (For instance, creating the zeros in column 4 requires 1 division in row 4 and 3 multiplication–addition pairs to add multiples of row 4 to the rows above.)

To find $\boldsymbol x$x requires 28 arithmetic operations, or “flops” (floating point operations), excluding the cost of finding $L$L and $U$U. In contrast, row reduction of $[ \ \ A\ \ \ \boldsymbol b\ \ ]$[  A   b  ] to $[ \ \ I\ \ \ \boldsymbol x\ \ ]$[  I   x  ] takes 62 operations.

Checkpoint: Section 2.2 shows how to compute $A^{–1}B$A–1B by row reduction. Describe how you could speed up this calculation if you have an LU factorization of $A$A available (and $A$A is invertible).
Answer: If $A$A is an invertible $n\times n$n×n matrix, with an LU factorization $A = LU$A=LU, and if $B$B is $n\times p$n×p, then $A^{–1}B$A–1B can be computed by first row reducing $[\ \ L\ \ \ B\ \ ]$[  L   B  ] to a matrix $[\ \ I\ \ \ Y\ \ ]$[  I   Y  ] for some $Y$Y and then reducing $[\ \ \ U\ \ Y\ \ \ ]$[   U  Y   ] to $[\ \ I\ \ \ A^{–1}B\ \ ]$[  I   A–1B  ]. MATLAB uses this approach to compute $A^{–1}B$A–1B (after first finding L and U).

EXAMPLE
When $A$A is invertible, MATLAB finds $A^{-1}$A−1 by factoring $A =LU$A=LU (where $L$L may be permuted lower triangular), inverting $L$L and $U$U, and then computing $U^{-1}L^{-1}$U−1L−1.

An LU Factorization Algorithm

Suppose $A$A can be reduced to an echelon form $U$U using only row replacements that add a multiple of one row to another row below it. In this case, there exist unit lower triangular elementary matrices $E_1,...,E_p$E1​,...,Ep​ such that
$E_p...E_1A=U\ \ \ \ \ \ (3)$Ep​...E1​A=U      (3)Then
$A=(E_p...E_1)^{-1}U=LU$A=(Ep​...E1​)−1U=LUIt can be shown that products and inverses of unit lower triangular matrices are also unit lower triangular. Thus $L$L is unit lower triangular.

PROOF
Let $A$A be a lower triangular $n \times n$n×n matrix with nonzero entries on the diagonal. Show that $A$A is invertible and $A^{-1}$A−1 is lower triangular.
[Hint: Explain why $A$A can be changed into $I$I using only row replacements and scaling. Also, explain why the row operations that reduce $A$A to $I$I change $I$I into a lower triangular matrix.] 也可以用上一节介绍的分块矩阵+数学归纳法来证明

Note that the row operations in equation (3), which reduce $A$A to $U$U, also reduce the $L$L in equation (4) to $I$I. This observation is the key to constructing $L$L.

Step 1 is not always possible, but when it is, the argument above shows that an LU factorization exists.

EXAMPLE 2
Find an LU factorization of

SOLUTION
Since $A$A has four rows, $L$L should be $4 \times 4$4×4. The first column of $L$L is the first column of $A$A divided by the top pivot entry:

Compare the first columns of $A$A and $L$L. The row operations that create zeros in the first column of $A$A will also create zeros in the first column of $L$L. To make this same correspondence of row operations on $A$A hold for the rest of $L$L, watch a row reduction of $A$A to an echelon form $U$U. That is, highlight the entries in each matrix that are used to determine the sequence of row operations that transform $A$A into $U$U.

The highlighted entries determine the row reduction of $A$A to $U$U. At each pivot column, divide the highlighted entries by the pivot and place the result into $L$L:

In practical work, row interchanges are nearly always needed, because partial pivoting(部分主元法) is used for high accuracy. (Recall that this procedure selects, among the possible choices for a pivot, an entry in the column having the largest absolute value.)

To handle row interchanges, the LU factorization above can be modified easily to produce an $L$L that is $permuted\ lower\ triangular$permuted lower triangular(置换下三角矩阵), in the sense that a rearrangement (called a permutation) of the rows of $L$L can make $L$L (unit) lower triangular.
The resulting $permuted\ LU\ factorization$permuted LU factorization(置换LU分解) solves $A\boldsymbol x = \boldsymbol b$Ax=b in the same way as before, except that the reduction of $[\ \ L\ \ \ \boldsymbol b\ \ ]$[  L   b  ] to $[\ \ I\ \ \ \boldsymbol y\ \ ]$[  I   y  ] follows the order of the pivots in $L$L from left to right, starting with the pivot in the first column. A reference to an “LU factorization” usually includes the possibility that $L$L might be permuted lower triangular. For details, see the Appendix.

A Matrix Factorization in Electrical Engineering

Matrix factorization is intimately related to the problem of constructing an electrical network with specified properties.

Suppose the box in Figure 3 represents some sort of electric circuit, with an input and output.

Record the input voltage and current by $\begin{bmatrix}v_1\\i_1\end{bmatrix}$[v1​i1​​] and record the output voltage and current by $\begin{bmatrix}v_2\\i_2\end{bmatrix}$[v2​i2​​]. Frequently, the transformation $\begin{bmatrix}v_1\\i_1\end{bmatrix}\mapsto\begin{bmatrix}v_2\\i_2\end{bmatrix}$[v1​i1​​]↦[v2​i2​​] is linear. That is, there is a matrix $A$A, called the $transfer\ matrix(传递矩阵)$transfer matrix(传递矩阵), such that
$\begin{bmatrix}v_2\\i_2\end{bmatrix}=A\begin{bmatrix}v_1\\i_1\end{bmatrix}$[v2​i2​​]=A[v1​i1​​]Figure 4 shows a $ladder\ network$ladder network. The left circuit in Figure 4 is called a $series\ circuit(串联电路)$series circuit(串联电路), with resistance $R_1$R1​.

The right circuit in Figure 4 is a $shunt\ circuit(并联电路)$shunt circuit(并联电路), with resistance $R_2$R2​. Using Ohm’s law and Kirchhoff’s laws, one can show that the transfer matrices of the series and shunt circuits, respectively, are

EXAMPLE 3
a. Compute the transfer matrix of the ladder network in Figure 4.
b. Design a ladder network whose transfer matrix is $\begin{bmatrix} 1&-8\\-.5&5 \end{bmatrix}$[1−.5​−85​].
SOLUTION
a. Let $A_1$A1​ and $A_2$A2​ be the transfer matrices of the series and shunt circuits, respectively. Then the transfer matrix of the ladder network corresponds to composition of linear transformations

b. To factor the matrix $\begin{bmatrix} 1&-8\\-.5&5 \end{bmatrix}$[1−.5​−85​] into the product of transfer matrices, as in equation
(6), look for $R_1$R1​ and $R_2$R2​ in Figure 4 to satisfy

A network transfer matrix summarizes the input–output behavior (the design specifications) of the network without reference to the interior circuits. To physically build a network with specified properties, an engineer first determines if such a network can be constructed (or realized). Then the engineer tries to factor the transfer matrix into matrices corresponding to smaller circuits that perhaps are already manufactured and ready for assembly. In the common case of alternating current(交流电), the entries in the transfer matrix are usually rational complex-valued functions. A standard problem is to find a minimal realization that uses the smallest number of electrical components.

The QR Factorization QR分解

Suppose $A = QR$A=QR; where $Q$Q and $R$R are $n \times n$n×n. $R$R is invertible and upper triangular, and $Q$Q has the property that $Q^TQ = I$QTQ=I. It can be shown that for each $\boldsymbol b$b in $\mathbb R^n$Rn, the equation $A\boldsymbol x =\boldsymbol b$Ax=b has a unique solution.

When $A =QR$A=QR, the equation $A\boldsymbol x = \boldsymbol b$Ax=b can be written as $Q(R\boldsymbol x)=\boldsymbol b$Q(Rx)=b. Then $R\boldsymbol x=Q^T\boldsymbol b$Rx=QTb, which is easy to solve.

Appendix: Permuted LU Factorizations 置换LU分解

Any $m\times n$m×n matrix $A$A admits a factorization $A = LU$A=LU, with $U$U in echelon form and $L$L a permuted unit lower triangular matrix. That is, $L$L is a matrix such that a permutation (rearrangement) of its rows (using row interchanges) will produce a lower triangular matrix with 1’s on the diagonal.

The construction of $L$L and $U$U, illustrated below, depends on first using row replacements to reduce $A$A to a permuted echelon form $V$V and then using row interchanges to reduce $V$V to an echelon form $U$U. By watching the reduction of $A$A to $V$V, we can easily construct a permuted unit lower triangular matrix $L$L with the property that the sequence of operations changing $A$A into $U$U also changes $L$L into $I$I. This property will guarantee that $A = LU$A=LU.

The following algorithm reduces any matrix to a permuted echelon form. In the algorithm when a row is covered, we ignore it in later calculations.

1. Begin with the leftmost nonzero column. Choose any nonzero entry as the pivot. Designate the corresponding row as a pivot row.
2. Use row replacements to create zeros above and below the pivot (in all uncovered rows). Then cover that pivot row.
3. Repeat steps 1 and 2 on the uncovered submatrix, if any, until all nonzero entries are
   covered.

As an example, choose any entry in the first column of the following matrix as the first pivot, and use the pivot to create zeros in the rest of column 1. We choose the (3, 1)-entry.

Row 3 is the first pivot row. Choose the (2, 2)-entry as the second pivot, and create zeros in the rest of column 2, excluding the first pivot row.

Cover row 2 and choose the (4, 4)-entry as the pivot. Create zeros in the other rows, excluding the
first two pivot rows.

Let $V$V denote this permuted echelon form, and permute the rows of $V$V to create an echelon form. The resulting echelon matrix $U$U is

The last step is to create $L$L. Go back and watch the reduction of $A$A to $V$V. As each pivot is selected, take the pivot column, and divide the pivot into each entry in the column that is not yet in a pivot row. Place the resulting column into $L$L. At the end, fill the holes in $L$L with zeros.

The next example illustrates what to do when $V$V has one or more rows of zeros. For the reduction of $A$A to $V$V, pivots were chosen to have the largest possible magnitude (the choice used for “partial pivoting”).

The first three columns of $L$L come from the three pivot columns above.

The matrix $L$L needs two more columns. Use columns 1 and 3 of the $5\times5$5×5 identity matrix to place 1’s in the “nonpivot” rows 1 and 3. Fill in the remaining holes with zeros.