pag.

Simple Linear Regression Formulas

Coefficient's estimators with method of least squares:

SxY (xi − x̄)(Yi − Ȳ ) xi Yi − nx̄Ȳ
b̂1 = = = b̂0 = Y − b̂1 x
Sxx (xi − x̄)2 x2i − nx̄2

Distribution of b̂1 and b̂0

1 2 x2i 2
b̂1 ∼ N (b1 , σ ) b̂0 ∼ N (b0 , σ )
Sxx nSxx
Residuals
Yi − ( b̂0 + b̂1xi) = Yi − Ŷi

Residual variance or sum of squared errors

SSE = (Yi − Ŷi)2

Explained variance or sum of squares due to regression

n 2
SSR = Ŷi − Ȳ
i=1

Y Variance
SY Y = SST = (Yi − Ȳ )2 = Yi2 − nȲ 2
Formulas for SSE e SSR
2
SY Y Sxx − SxY
SSE = = SY Y − b̂1 SxY SSR = b̂1 SxY
Sxx
A formula for SY Y = SST
SY Y = SST = SSR + SSE
Coefficient of determination r 2 and correlation r
2
SSR SSE SxY SxY SxY
r2 = =1− = b̂1 = r= √
SY Y SY Y SY Y Sxx SY Y Sxx SY Y
SSE
Distribution of
σ
(Yi − Ŷi )2 SSE SSE
2
= 2 = MSE ∼ χ2n−2 ; E = σ2
σ σ n−2

Standardized of b̂1
b̂1 − b1
√ ∼ N(0, 1)
σ/ Sxx
Using the symbology of statistical software R, then

SSE SSE
σ 2 Σx2i (n−2) Σx2i σ2 (n−2)
se(b0 ) = = Sxx se(b1 ) = = Sxx
Sxx n n Sxx

σ 2 Σx2i σ2
se(b 0) is the estimation var(b0 ) = and se(b 1) is the estimation of var(b1 ) =
Sxx n Sxx
 pag. 2

Pivotal quantity for b̂0

b̂0 − b0 (n − 2)Sxx n
= (b̂0 − b0 ) ∼ tn−2
se(b0 ) SSE Σx2i

Pivotal quantity for b̂1

b̂1 − b1 (n − 2)Sxx Sxx

= (b̂1 − b1 ) = (b̂1 − b1 ) ∼ tn−2
se(b1 ) SSE M SE

Hypothesis test on b1 of significance 1 − γ

Statistical test
(n − 2)Sxx (b̂1 − β)
(b̂1 − β) ∼ tn−2 OR
SSE se(b1 )
H0 : b1 = β; e Ha : b1 = β; critical region

1+γ SSE 1+γ SSE

b̂1 < β − tn−2 ( ) β + tn−2 ( ) < b̂1
2 (n − 2)Sxx 2 (n − 2)Sxx
or

b̂ β − tn−2(1 + γ ) se(b )
1< β + tn−2 (
1+γ
) se(b1 ) < b̂1
1
2 2
H0 : b1 ≥ β; e Ha : b1 < β; critical region

SSE
b̂1 < β − · tn−2 (γ) OR b̂1 < β − se(b1 ) · tn−2 (γ)
(n − 2)Sxx

H0 : b1 ≤ β; e Ha : b1 > β; critical region

SSE
b̂1 > β + · tn−2 (γ) OR b̂1 > β + se(b1 ) · tn−2 (γ)
(n − 2)Sxx

Confidence Intervals of level γ for b1.

Two-sided:
1+γ SSE 1+γ SSE
b̂1 − tn−2 ( ) < β < b̂1 + tn−2 ( ) .
2 (n − 2)Sxx 2 (n − 2)Sxx
or
1+γ 1+γ
b̂1 − tn−2 ( )se(b1 ) < β < b̂1 + tn−2 ( )se(b1 ).
2 2