Preconditioning and Condition Number

Oct 13, 2025 by wbrum.org 138 views

Introduction

In linear algebra and numerical analysis, the condition number of a matrix is a fundamental concept that measures the sensitivity of the solution of a linear system to changes in the input data. The condition number is defined as the ratio of the largest to the smallest singular value of the matrix. In this article, we will discuss the condition number of a preconditioned positive definite matrix, specifically whether the condition number of the preconditioned matrix is less than or equal to the condition number of the original matrix.

Preconditioning and Condition Number

Preconditioning is a technique used to improve the stability and efficiency of numerical algorithms for solving linear systems. The basic idea is to transform the original matrix into a new matrix with a more favorable condition number. One common preconditioning technique is to multiply the original matrix by a diagonal matrix, which is called diagonal preconditioning. In this article, we will focus on diagonal preconditioning and investigate whether the condition number of the preconditioned matrix is less than or equal to the condition number of the original matrix.

Positive Definite Matrix and Diagonal Preconditioning

A positive definite matrix is a symmetric matrix that has all positive eigenvalues. The diagonal part of a matrix is the matrix obtained by setting all off-diagonal elements to zero. Suppose $A$ is a positive definite matrix and $D$ is its diagonal part. We can write the diagonal preconditioning of $A$ as $D^{-1}A$ , where $D^{-1}$ is the inverse of the diagonal matrix $D$ . The question is whether the condition number of $D^{-1}A$ is less than or equal to the condition number of $A$ .

Condition Number and Singular Values

The condition number of a matrix is defined as the ratio of the largest to the smallest singular value of the matrix. The singular values of a matrix are the square roots of the eigenvalues of the matrix $A^T A$ , where $A^T$ is the transpose of the matrix $A$ . The condition number of a matrix $A$ is denoted by $\kappa(A)$ and is defined as:

\kappa(A) = \frac{\sigma_{max}(A)}{\sigma_{min}(A)}

where $\sigma_{max}(A)$ and $\sigma_{min}(A)$ are the largest and smallest singular values of the matrix $A$ , respectively.

Theorem and Proof

We will now prove that the condition number of the preconditioned matrix $D^{-1}A$ is less than or equal to the condition number of the original matrix $A$ . We will use the following theorem:

Theorem 1: Let $A$ be a positive definite matrix and $D$ be its diagonal part. Then, the condition number of the preconditioned matrix $D^{-1}A$ is less than or equal to the condition number of the original matrix $A$ .

Proof: We will use the following inequality:

\kappa(D^{-1}A) \leq \kappa(A)

To prove this inequality, we will use the following properties of the singular values:

The singular values of a matrix are non-negative.
The singular values of a matrix are invariant under diagonal scaling.

Using these properties, we can write:

\kappa(D^{-1}A) = \frac{\sigma_{max}(D^{-1}A)}{\sigma_{min}(D^{-1}A)}

Since $D$ is a diagonal matrix, we can write:

D^{-1}A = D^{-1}AD