最小二乘法拟合线性曲线

上一章 采用梯度下降的方法拟合线性方程，经过10次迭代，仍然存在较大误差

本次采用最小二乘法的方法拟合线性方程，目的是提供不同视角，寻找最优解

根据《数值分析》[1] 提供的最小二乘法

设置一个函数 y = S ∗ ( x ) y=S^*(x) y=S∗(x) 与所给出数据 { ( x i , y i ) , i = 0 , 1 , . . . , m } \{(x_i,y_i),i=0,1,...,m\} {(xi​,yi​),i=0,1,...,m}拟合，若记误差 δ i = S ∗ ( x i ) − y i ( i = 0 , 1 , . . . , m ) , δ = ( δ 0 , δ 1 , . . . , δ m ) T δ_i = S^*(x_i)-y_i(i=0,1,...,m),δ = (δ_0,δ_1,...,δ_m)^T δi​=S∗(xi​)−yi​(i=0,1,...,m),δ=(δ0​,δ1​,...,δm​)T， 设 φ 0 ( x ) , φ 1 ( x ) , . . . , φ n ( x ) φ_0(x),φ_1(x),...,φ_n(x) φ0​(x),φ1​(x),...,φn​(x)是$C[a,b]$上线性无关的函数族，在 φ = s p a n { φ 0 ( x ) , φ 1 ( x ) , . . . , φ n ( x ) } φ=span\{φ_0(x),φ_1(x),...,φ_n(x)\} φ=span{φ0​(x),φ1​(x),...,φn​(x)}中找一个函数 S ∗ ( x ) S^*(x) S∗(x)，使误差平方和

∣ ∣ δ ∣ ∣ 2 2 = ∑ i = 0 m δ i 2 = ∑ i = 0 m [ S ∗ ( x i ) − y i ] 2 = m i n S ( x ) ∈ φ ∑ i = 0 m [ S ( x i ) − y i ] 2 (4.1) ||δ||_2^2 = ∑_{i=0}^m δ_i^2 = ∑_{i=0}^m [S^*(x_i) - y_i]^2 = min_{S(x) ∈ φ}∑_{i=0}^m [S(x_i) - y_i]^2 \tag{4.1} ∣∣δ∣∣22​=i=0∑m​δi2​=i=0∑m​[S∗(xi​)−yi​]2=minS(x)∈φ​i=0∑m​[S(xi​)−yi​]2(4.1)
这里
S ( x ) = a 0 φ 0 ( x ) + a 1 φ 1 ( x ) + . . . + a n φ n ( x ) ( n < m ) (4.2) S(x) = a_0φ_0(x) + a_1φ_1(x) +...+ a_nφ_n(x) (nS(x)=a0​φ0​(x)+a1​φ1​(x)+...+an​φn​(x)(n<m)(4.2)

这就是一般的最小二乘逼近，用几何语言说，就称为曲线拟合的最小二乘法。

考虑不同点的比重 ω ( x i ) ω(x_i) ω(xi​)不同，通常最小二乘法中 ∣ ∣ δ ∣ ∣ 2 2 ||δ||_2^2 ∣∣δ∣∣22​都考虑加权平方和

∣ ∣ δ ∣ ∣ 2 2 = ∑ i = 0 m ω ( x i ) [ S ( x i ) − f ( x i ) ] 2 (4.3) ||δ||_2^2 = ∑_{i=0}^m ω(x_i)[S(x_i) - f(x_i)]^2 \tag{4.3} ∣∣δ∣∣22​=i=0∑m​ω(xi​)[S(xi​)−f(xi​)]2(4.3)
转换成函数求极值问题
I ( a 0 , a 1 , . . . , a n ) = ∑ i = 0 m ω ( x i ) [ S ( x i ) − f ( x i ) ] 2 (4.4) I(a_0,a_1,...,a_n) = ∑_{i=0}^m ω(x_i)[S(x_i) - f(x_i)]^2 \tag{4.4} I(a0​,a1​,...,an​)=i=0∑m​ω(xi​)[S(xi​)−f(xi​)]2(4.4)
∂ I ∂ a k = 2 ∑ i = 0 m ω ( x i ) [ ∑ j = 0 m a j φ j ( x i ) − f ( x i ) ] φ k ( x i ) = 0 , k = 0 , 1 , . . . , n \frac{∂I}{∂a_k} = 2∑_{i=0}^m ω(x_i)[∑_{j=0}^m a_jφ_j(x_i)-f(x_i)]φ_k(x_i) = 0, k=0,1,...,n ∂ak​∂I​=2i=0∑m​ω(xi​)[j=0∑m​aj​φj​(xi​)−f(xi​)]φk​(xi​)=0,k=0,1,...,n
( φ j , φ k ) = ∑ i = 0 m ω ( x i ) φ j ( x i ) φ k ( x i ) (4.5) (φ_j,φ_k) = ∑_{i=0}^m ω(x_i)φ_j(x_i)φ_k(x_i) \tag{4.5} (φj​,φk​)=i=0∑m​ω(xi​)φj​(xi​)φk​(xi​)(4.5)
令
( f , φ k ) = ∑ i = 0 m ω ( x i ) f ( x i ) φ k ( x i ) = d k , k = 0 , 1 , . . . , n (f,φ_k) = ∑_{i=0}^m ω(x_i)f(x_i)φ_k(x_i) = d_k,k=0,1,...,n (f,φk​)=i=0∑m​ω(xi​)f(xi​)φk​(xi​)=dk​,k=0,1,...,n
则
∑ j = 0 n ( φ k , φ j ) a j = d k , k = 0 , 1 , . . . , n (4.6) ∑_{j=0}^n(φ_k,φ_j)a_j = d_k, k=0,1,...,n \tag{4.6} j=0∑n​(φk​,φj​)aj​=dk​,k=0,1,...,n(4.6)

写成矩阵的形式

$$Ga=d$$
G = [ ( φ 0 , φ 0 ) ( φ 0 , φ 1 ) . . . ( φ 0 , φ n ) ( φ 1 , φ 0 ) ( φ 1 , φ 1 ) . . . ( φ 1 , φ n ) . . . ( φ n , φ 0 ) ( φ n , φ 1 ) . . . ( φ n , φ n ) ] G = \begin{bmatrix} (φ_0,φ_0) (φ_0,φ_1) ... (φ_0,φ_n) \\ (φ_1,φ_0) (φ_1,φ_1) ... (φ_1,φ_n) \\ ... \\ (φ_n,φ_0) (φ_n,φ_1) ... (φ_n,φ_n) \\ \end{bmatrix} G=⎣⎢⎢⎡​(φ0​,φ0​)(φ0​,φ1​)...(φ0​,φn​)(φ1​,φ0​)(φ1​,φ1​)...(φ1​,φn​)...(φn​,φ0​)(φn​,φ1​)...(φn​,φn​)​⎦⎥⎥⎤​

对于上一个实例，100 离散的点，每个点的权重相同，

m = 100 , n = 1 , φ 0 ( x ) = 1 , φ 1 ( x ) = x , ω i = 1 m=100, n=1, φ_0(x) = 1, φ_1(x)=x, ω_i = 1 m=100,n=1,φ0​(x)=1,φ1​(x)=x,ωi​=1
令
S 1 ( x ) = a 0 + a 1 x S_1(x) = a_0 + a_1 x S1​(x)=a0​+a1​x
( φ 0 , φ 0 ) = ∑ i = 0 100 1 = 100 ( φ 0 , φ 1 ) = ( φ 1 , φ 0 ) = ∑ i = 0 100 x i ( φ 1 , φ 1 ) = ∑ i = 0 100 x i 2 ( φ 0 , f ) = ∑ i = 0 100 f i ( φ 1 , f ) = ∑ i = 0 100 x i f i (φ_0,φ_0) = ∑_{i=0}^{100} 1 = 100 \\ (φ_0,φ_1) = (φ_1,φ_0) = ∑_{i=0}^{100} x_i \\ (φ_1,φ_1) = ∑_{i=0}^{100} x_i^2 \\ (φ_0,f) = ∑_{i=0}^{100} f_i \\ (φ_1,f) = ∑_{i=0}^{100} x_if_i \\ (φ0​,φ0​)=i=0∑100​1=100(φ0​,φ1​)=(φ1​,φ0​)=i=0∑100​xi​(φ1​,φ1​)=i=0∑100​xi2​(φ0​,f)=i=0∑100​fi​(φ1​,f)=i=0∑100​xi​fi​
( φ 0 , φ 0 ) a 0 + ( φ 0 , φ 1 ) a 1 = ( φ 0 , f ) ( φ 1 , φ 0 ) a 0 + ( φ 1 , φ 1 ) a 1 = ( φ 1 , f ) (φ_0,φ_0)a_0 + (φ_0,φ_1)a_1 = (φ_0,f) \\ (φ_1,φ_0)a_0 + (φ_1,φ_1)a_1 = (φ_1,f) (φ0​,φ0​)a0​+(φ0​,φ1​)a1​=(φ0​,f)(φ1​,φ0​)a0​+(φ1​,φ1​)a1​=(φ1​,f)
求解线性方程即可

import numpy as npdef get_line_ratio(points):φ_0_0 = len(points)  # ω =1φ_0_1 = 0.0φ_1_1 = 0.0φ_f_0 = 0.0  # φ0fφ_f_1 = 0.0for i in range(φ_0_0):φ_0_1 += points[i][0]φ_1_1 += points[i][0] * points[i][0]φ_f_0 += points[i][1]φ_f_1 += points[i][0] * points[i][1]return np.linalg.solve(np.array([[φ_0_0, φ_0_1], [φ_0_1, φ_1_1]]), np.array([φ_f_0, φ_f_1]))

绿色为原方程，蓝色线为最小二乘法拟合的线，红色为梯度下降训练得到的线

可以看出，使用最小二程拟合得到线与原来的线几乎重合，拟合效果较好

• [ 1] 数值分析. 李庆扬,王能超,易大义.清华大学出版社.第五版.p73-p75

本文来自互联网用户投稿，文章观点仅代表作者本人，不代表本站立场，不承担相关法律责任。如若转载，请注明出处。 如若内容造成侵权/违法违规/事实不符，请点击【内容举报】进行投诉反馈！