数值计算详细笔记（一）：基础数学知识回顾

文章目录

• 1. Mathematical Preliminaries
• 1.1 Concept introduction
• 1.1.1 Equivalent description
• 1.1.2 Main Contents
• 1.1.3 Reason for learning
• 1.1.4 Book content
• 1.1.5 Steps for solving
• 1.2 Calculus Review
• 1.2.1 Limit of a Function
• 1.2.2 Continuous of a Function
• 1.2.3 Limit of a Sequence
• 1.2.4 Derivative of a Function
• 1.2.5 Mean Value Theorem
• 1.2.6 Extreme Value Theorem
• 1.2.7 Riemann Integral
• 1.2.8 Weighted Mean Value Theorem for the Integral（Importent）
• 1.2.9 Generalized Rolle's Theorem
• 1.2.10 Intermediate Value Theorem
• 1.2.11 Taylor's Theorem
• 1.3 Roundoff Errors in Computer
• 1.3.1 Roundoff Error
• 1.3.2 Measurement of Error
• 1.3.3 Growth of Error
• 1.3.4 Propagation of Error
• 1.4 Algorithms and Convergence
• 1.4.1 Basic Definitions
• 1.4.2 Some Important Concepts
• 1.4.3 Rate of Convergence
• 1.4.4 Well-Posed or Ill-posed Problem
• 1.4.5 Approximation of Solution

1. Mathematical Preliminaries

1.1 Concept introduction

1.1.1 Equivalent description

• Numerical analysis
• Numerical Methods or Computational Methods
• Numerical Mathematics
• Numerical Computing
• Scientific Computing

1.1.2 Main Contents

• Methods using computer to solve mathematical problems in science and engineering, which mostly are continuous.
• design and analysis of algorithms for different mathematical problems.
• Theory and Application of Numerical Approximation Techniques.

1.1.3 Reason for learning

With the fast development of PC techniques, many mathematical problems arising in the science and engineering, such as derivatives (导数)、integrals (积分)、nonlinearities (非线性)、Linear Algebra problems (线性代数问题)、differential equations (微分方程) is possible to be solved.

Deals continuous quantities with numerical approximate techniques and considers effects of approximations, such as error (误差), convergence (收敛), uniqueness (唯一性), existence (存在性).

Assessment on algorithm: efficiency, reliability, accuracy, etc.

1.1.4 Book content

1. Approximation Methods for solving equation(s)
2. Polynomial (多项式) and interpolation approximation (插值逼近)
3. Numerical Differentiation and Integration
4. Numerical methods for ODE and PDE
5. Eigenvalues and Eigenvectors

1.1.5 Steps for solving

1. Mathematical modeling, usually equations (方程).
2. Design algorithms to solve these equations.
3. Implement algorithms in computer software and run.
4. Represent the computed results in forms or graphical visualization.
5. Interpret and validate (解释和验证) the computed results.

summary: MAIRI

1.2 Calculus Review

1.2.1 Limit of a Function

Definition

Let $f$f be a function defined on a set X of real numbers. Then $f$f has the limit L at $x_0$x0​, written as

$\lim_{x \rightarrow x_0} f(x)=L,$x→x0​lim​f(x)=L,

if, given any real number $\varepsilon>0$ε>0, there exists a real number $\xi>0$ξ>0 such that

$|f(x)-L|<\varepsilon$∣f(x)−L∣<ε

whenever $x\in X$x∈X and $0<|x-x_0|<\xi$0<∣x−x0​∣<ξ.

1.2.2 Continuous of a Function

Condition

Let $f$f be a function defined on a set X of real numbers and $x_0 \in X$x0​∈X.

Theorem

Then $f$f is continuous at $x_0$x0​ if

$\lim_{x\rightarrow x_0} f(x)=f(x_0 ).$x→x0​lim​f(x)=f(x0​).

Extended Theorem

The function $f$f is continuous on the set X if it is continuous at each number in X.

Definition of $C(X)$C(X)

Let $C(X)$C(X) denote the set of all functions that are continuous on the set X.

1.2.3 Limit of a Sequence

Difinition

Let $\{x_n\}_{n=1}^{\infty}${xn​}n=1∞​ be an infinite sequence of real or complex number. The sequence converges to a number $x$x (Limit) if, for any $\varepsilon >0 $, there exists a positive integer $N(\varepsilon)$N(ε), such that implies

$|x_n-x|<\varepsilon,$∣xn​−x∣<ε,

whenever $n>N(\epsilon)$n>N(ϵ).

Extended Theorem（Obviously）

If $f$f is a function defined on a set of real numbers and $x_0\in X$x0​∈X, then the following statements are equivalent:

1. f is continuous at $x_0$x0​
2. if $\{x_n\}_{n=1}^\infty${xn​}n=1∞​ is any sequence in X converging to $x_0$x0​, then $\lim_{n\rightarrow\infty}f(x_n)=f(x_0)$limn→∞​f(xn​)=f(x0​).

1.2.4 Derivative of a Function

Definition

If $f$f is a function defined in an open interval (开区间) containing $x_0$x0​, the $f$f is differentiable at $x_0$x0​, if
$f'(x_0) =\lim_{x\rightarrow x_0 }\displaystyle\frac{f(x)-f(x_0) }{x-x_0 }$f′(x0​)=x→x0​lim​x−x0​f(x)−f(x0​)​
exists.

Remark

The number $f’(x_0) $ is called the derivative of $f(x)$f(x) at $x_0$x0​.

$C^n(X)$Cn(X) denote the set of all functions that have $n$n continuous derivatives on $X$X.

Especially $C^\infty(X) $ denote the set of all functions that have derivatives of all orders on X.

Extended Theorem

If the function $f$f is differentiable at $x_0$x0​, the $f$f is continuous at $x_0$x0​.

Rolle’s Theorem（罗尔定理）

Suppose $f\in C[a,b]$f∈C[a,b] and $f$f is differentiable on $(a,b)$(a,b).

If $f(a)=f(b)=0$f(a)=f(b)=0, then a number $c$c in $(a,b)$(a,b) exists with $f'(c)=0$f′(c)=0.

1.2.5 Mean Value Theorem

Suppose $f\in C[a,b]$f∈C[a,b] and $f$f is differentiable on $(a,b)$(a,b), then a number $c$c in $(a,b)$(a,b) exists with

$f'(c)=\displaystyle\frac{f(b)-f(a)}{b-a}$f′(c)=b−af(b)−f(a)​

1.2.6 Extreme Value Theorem

If $f\in C[a,b]$f∈C[a,b], then $c_1,c_2\in[a,b]$c1​,c2​∈[a,b] exist with $f(c_1)\leq f(x)\leq f(c_2)$f(c1​)≤f(x)≤f(c2​) for each $x\in [a,b]$x∈[a,b].

If, in addition, $f$f is differentiable on $(a,b)$(a,b), then the numbers $c_1$c1​ and $c_2$c2​ (Extreme Value) occur either at the endpoints of $[a,b]$[a,b] or where $f'$f′ is zero.

1.2.7 Riemann Integral

Definition

The Riemann Integral of a function on an interval $[a,b]$[a,b] is the following limit, provided it exits:

$\int_a^b f(x)dx=\lim_{max\Delta x_i\rightarrow0}\sum_{i=1}^nf(z_i)\Delta x_i,$∫ab​f(x)dx=maxΔxi​→0lim​i=1∑n​f(zi​)Δxi​,

where the numbers $x_0,x_1,x_2,...,x_n$x0​,x1​,x2​,...,xn​ satisfy $a=x_0\leq x_1\leq x_2\leq ...\leq x_n=b$a=x0​≤x1​≤x2​≤...≤xn​=b, and where $\Delta x_i=x_i-x_{i-1}$Δxi​=xi​−xi−1​ for each $i=1,2,...,n$i=1,2,...,n and $z_i$zi​ is an arbitrarily chosen in the interval $[x_{i-1},x_i]$[xi−1​,xi​].

Remark

Especially, if we choose $z_i=x_i$zi​=xi​ and $\Delta x_i=\displaystyle\frac{b-a}{n}$Δxi​=nb−a​, then in this case

$\int_a^bf(x)dx=\lim_{n\rightarrow\infty}\displaystyle\frac{b-a}{n}\sum_{i=1}^nf(x_i).$∫ab​f(x)dx=n→∞lim​nb−a​i=1∑n​f(xi​).

1.2.8 Weighted Mean Value Theorem for the Integral（Importent）

Definition

If $f\in C[a,b]$f∈C[a,b], the Riemann Integral of $g$g exists on the $[a,b]$[a,b], and $g(x)$g(x) does not change sign on $[a,b]$[a,b], then there exists a number in $(a,b)$(a,b) with
$\int_a^bf(x)g(x)dx=f(c)\int_a^bg(x)dx.$∫ab​f(x)g(x)dx=f(c)∫ab​g(x)dx.

Remark

When $g(x)\equiv 1$g(x)≡1, this theorem give the average value of the function $f$f over the interval $[a,b]$[a,b].

$f(c)=\displaystyle\frac{1}{b-a}\int_a^bf(x)dx$f(c)=b−a1​∫ab​f(x)dx

1.2.9 Generalized Rolle’s Theorem

Definition

Suppose $f\in C[a,b]$f∈C[a,b] is $n$n times differentiable on $(a,b)$(a,b). If $f(x)$f(x) is zero at the $n+1$n+1 distinct numbers $x_0,x_1,x_2,...x_n$x0​,x1​,x2​,...xn​ in the $[a,b]$[a,b], then a number $c$c in the $(a,b)$(a,b) exists with

$f^{(n)}(c)=0.$f(n)(c)=0.

Simple Provement

Applying Rolle’s theorem n times, leaving two zero points in the end.

1.2.10 Intermediate Value Theorem

If $f\in C[a,b]$f∈C[a,b] and $K$K is any number between $f(a)$f(a) and $f(b)$f(b), then there exists a number $c$c in $(a,b)$(a,b) for which $f(c)=K$f(c)=K.

1.2.11 Taylor’s Theorem

Suppose $f\in C^n [a,b]$f∈Cn[a,b], that $f^{(n+1)}$f(n+1) exists on $[a,b]$[a,b], and $x_0\in[a,b]$x0​∈[a,b]. For every $x\in [a,b]$x∈[a,b] there exists a number $\xi(x)$ξ(x) between $x_0$x0​ and $x$x with $f(x)=P_n(x)+R_n(x)$f(x)=Pn​(x)+Rn​(x), where

$P_n(x)=f(x_0)+f'(x_0)(x-x_0)+\displaystyle\frac{f''(x_0)}{2!}(x-x_0)^2+...+\displaystyle\frac{f^n(x_0)}{n!}(x-x_0)^n\\ P_n(x)=\sum_{k=0}^n\displaystyle\frac{f^k(x_0)}{k!}(x-x_0)^k$Pn​(x)=f(x0​)+f′(x0​)(x−x0​)+2!f′′(x0​)​(x−x0​)2+...+n!fn(x0​)​(x−x0​)nPn​(x)=k=0∑n​k!fk(x0​)​(x−x0​)k

and

$R_n(x)=\displaystyle\frac{f^{(n+1)}(\xi(x))}{(n+1)!}(x-x_0)^{n+1}.$Rn​(x)=(n+1)!f(n+1)(ξ(x))​(x−x0​)n+1.

1.3 Roundoff Errors in Computer

1.3.1 Roundoff Error

Binary Floating Arithmetic Standard 754-1985

Format: single, double, or extended precision

Representation: sign(s-1) + characteristic(c-11) + mantissa(f-52)

form: $(-1)^s2^{c-1023}(1+f)$(−1)s2c−1023(1+f)

Example

Underflow

Number less than $2^{-1023}*(1+2^{-52})$2−1023∗(1+2−52), cause to zero.

Overflow

Number greater than $2^{1024}*(2-2^{-52})$21024∗(2−2−52), cause to halt.

Normalized decimal floating-point form

$\pm0.d_1d_2...d_k*10^n$±0.d1​d2​...dk​∗10n

where $1\leq d_1\leq 9$1≤d1​≤9, and $0\leq d_i\leq 9$0≤di​≤9 for each $i=1,2,...,k$i=1,2,...,k.

Numbers of this form are called k-digit decimal machine numbers.

The left digits $d_{k+1}d_{k+2}...$dk+1​dk+2​... can be treated by chopping (截断) or rounding（舍入）methods.

1.3.2 Measurement of Error

Absolute and Relative Error

If $p^*$p∗ is an approximation to $p$p, the absolute error is |p-p^*|, and the relative error is $\displaystyle\frac{|p-p^*|}{|p|}$∣p∣∣p−p∗∣​, provided that $p\not=0$p​=0.

Approximate $p$p to $t$t significant digit

The number $p^*$p∗ is said to approximate $p$p to $t$t significant digit if $t$t is the largest nonnegative integer for which

$\displaystyle\frac{|p-p^*|}{|p|}<5*10^{-t}.$∣p∣∣p−p∗∣​<5∗10−t.

1.3.3 Growth of Error

Prerequisite variables

$E_0$E0​: an initial error

$E_n$En​: the magnitude of an error after $n$n subsequent operations.

Linear Growth

If $E_n\approx C*n*E_0$En​≈C∗n∗E0​, where $C$C is a constant independent of $n$n.

Exponential Growth

If $E_n \approx C^n*E_0$En​≈Cn∗E0​, for some $C>1$C>1.

1.3.4 Propagation of Error

$\hat{p},\hat{q}$p^​,q^​: the approximate value of $p,q$p,q.
$\varepsilon_p,\varepsilon_q$εp​,εq​: the error of $p,q$p,q.

Addition

$p+q=(\hat{p}+\varepsilon_p)+(\hat{q}+\varepsilon_q)=(\hat{p}+\hat{q})+(\varepsilon_p+\varepsilon_q)$p+q=(p^​+εp​)+(q^​+εq​)=(p^​+q^​)+(εp​+εq​)

Multiplication

$p*q=(\hat{p}+\varepsilon_p)*(\hat{q}+\varepsilon_q)=\hat{p}*\hat{q}+\hat{p}*\varepsilon_q+\hat{q}*\varepsilon_q+\varepsilon_p*\varepsilon_q$p∗q=(p^​+εp​)∗(q^​+εq​)=p^​∗q^​+p^​∗εq​+q^​∗εq​+εp​∗εq​

Division

$\displaystyle\frac{p}{q}-\displaystyle\frac{\hat{p}}{\hat{q}}=\displaystyle\frac{\hat{p}+\varepsilon_p}{\hat{q}+\varepsilon_q}-\displaystyle\frac{\hat{p}}{\hat{q}}=\displaystyle\frac{\hat{q}*\varepsilon_p-\hat{p}*\varepsilon_q}{\hat{q}*(\hat{q}+\varepsilon_q)}$qp​−q^​p^​​=q^​+εq​p^​+εp​​−q^​p^​​=q^​∗(q^​+εq​)q^​∗εp​−p^​∗εq​​

1.4 Algorithms and Convergence

1.4.1 Basic Definitions

Algorithm

an algorithm is a procedure that describes, in an unambiguous or clear manner, a finite sequence of steps to be performed in a specified order.

Key Techniques for Algorithm

looping and condition-control method

Description

pseudo-code (伪代码) method.

1.4.2 Some Important Concepts

Stable

An algorithm is said to be stable imply that small changes in the initial data can produce correspondingly small changes in final results.

Conditionally Stable

Algorithm is stable only for certain choices of initial data.

1.4.3 Rate of Convergence

Definition

Suppose that

$\lim_{h\rightarrow 0} G(h) = 0$h→0lim​G(h)=0

and

$\lim_{h\rightarrow 0} F(h) = L.$h→0lim​F(h)=L.

If a positive constant $K$K exists with

$|F(h)-L|\leq K*|G(h)|,$∣F(h)−L∣≤K∗∣G(h)∣,

for sufficient small $h$h, then we write

$F(h)=L+O(G(h)).$F(h)=L+O(G(h)).

Example

Using Taylor formula with sufficient small $h$h, we have

$\cos(h)=1-\displaystyle\frac{1}{2}*h^2+\displaystyle\frac{1}{24}*h^4*\cos(\xi(h))$cos(h)=1−21​∗h2+241​∗h4∗cos(ξ(h))

since

$|\cos(h)+\displaystyle\frac{1}{2}*h^2 -1|=|\displaystyle\frac{1}{24}*h^4*\cos(\xi(h))|\leq \displaystyle\frac{1}{24}*h^4,$∣cos(h)+21​∗h2−1∣=∣241​∗h4∗cos(ξ(h))∣≤241​∗h4,

so

$\cos(h)+\displaystyle\frac{1}{2}*h^2=1+O(h^4).$cos(h)+21​∗h2=1+O(h4).

1.4.4 Well-Posed or Ill-posed Problem

Definition

A mathematical Problem is said to be well-posed if a solution

1. exists,
2. is unique,
3. depends continuously on problem data.

Otherwise, problem is ill-posed.

Remarks

1. Generally, the image of a function $f(x)$f(x) is continuous and the solution for $f(x)=y$f(x)=y exists and is unique.
2. Even if problem is well posed, solution may still be sensitive to input data. (Data changes a little, result changes a lot)
3. Computational algorithm should not make sensitivity worse.

1.4.5 Approximation of Solution

1. True value usually unknown, so we estimate or bound error rather than compute it exactly.

2. Relative error often has taken relative to approximate value, rather than the (unknown) true value.