Professional Documents
Culture Documents
Chapter 12
Simple Regression
Chap 12-1
Chapter Goals
After completing this chapter, you should be
able to:
Explain the correlation coefficient and perform a
hypothesis test for zero population correlation
Explain the simple linear regression model
Obtain and interpret the simple linear regression
equation for a set of data
Describe R2 as a measure of explanatory power of the
regression model
Understand the assumptions behind regression
analysis
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-2
Chapter Goals
(continued)
Chap 12-3
Correlation Analysis
Correlation analysis is used to measure
strength of the association (linear relationship)
between two variables
Correlation is only concerned with strength of the
relationship
No causal effect is implied with correlation
Correlation was first presented in Chapter 3
Chap 12-4
Correlation Analysis
The population correlation coefficient is
denoted (the Greek letter rho)
The sample correlation coefficient is
r
where
s xy
s xy
sxsy
(x x)(y y)
n 1
Chap 12-5
H0 : 0
r (n 2)
(1 r )
Chap 12-6
Decision Rules
Hypothesis Test for Correlation
Lower-tail test:
Upper-tail test:
Two-tail test:
H0: 0
H1: < 0
H0: 0
H1: > 0
H0: = 0
H1: 0
-t
r (n 2)
(1 r )
2
/2
-t/2
/2
t/2
has n - 2 d.f.
Chap 12-7
Introduction to
Regression Analysis
Regression analysis is used to:
Predict the value of a dependent variable based on
the value of at least one independent variable
Explain the impact of changes in an independent
variable on the dependent variable
Dependent variable: the variable we wish to explain
(also called the endogenous variable)
Chap 12-8
Yi 0 1x i i
Where 0 and 1 are the population model
coefficients and is a random error term.
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-9
Population
Slope
Coefficient
Independent
Variable
Random
Error
term
Yi 0 1Xi i
Linear component
Random Error
component
Chap 12-10
Yi 0 1Xi i
Observed Value
of Y for Xi
Predicted Value
of Y for Xi
Slope = 1
Random Error
for this Xi value
Intercept = 0
Xi
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
X
Chap 12-11
Estimate of
the regression
Estimate of the
regression slope
intercept
y i b0 b1x i
Value of x for
observation i
ei ( y i - y i ) y i - (b0 b1x i )
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-12
Chap 12-13
b1
(x x)(y y)
i 1
x
2
(x
x
)
i
rxy
sY
sX
i 1
b0 y b1x
The regression line always goes through the mean x, y
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-14
Chap 12-15
E[ i ] 0 and E[ i ] 2
for (i 1, , n)
Chap 12-16
Interpretation of the
Slope and the Intercept
b0 is the estimated average value of y
when the value of x is zero (if x = 0 is
in the range of observed x values)
Chap 12-17
Chap 12-18
Square Feet
(X)
245
1400
312
1600
279
1700
308
1875
199
1100
219
1550
405
2350
324
2450
319
1425
255
1700
Chap 12-19
Graphical Presentation
House price model: scatter plot
Chap 12-20
Chap 12-21
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
ANOVA
10
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-22
Graphical Presentation
House price model: scatter plot and
regression line
Slope
= 0.10977
Intercept
= 98.248
Chap 12-23
Interpretation of the
Intercept, b0
house price 98.24833 0.10977 (square feet)
b0 is the estimated average value of Y when the
value of X is zero (if X = 0 is in the range of
observed X values)
Here, no houses had 0 square feet, so b0 = 98.24833
just indicates that, for houses within the range of
sizes observed, $98,248.33 is the portion of the
house price not explained by square feet
Chap 12-24
Interpretation of the
Slope Coefficient, b1
house price 98.24833 0.10977 (square feet)
Chap 12-25
Measures of Variation
Total variation is made up of two parts:
SST
SSR
SSE
Total Sum of
Squares
Regression Sum
of Squares
Error Sum of
Squares
SST (y i y)2
SSR (y i y)2
SSE (y i y i )2
where:
Chap 12-26
Measures of Variation
(continued)
Chap 12-27
Measures of Variation
(continued)
Y
yi
2
SSE = (yi - yi )
_
y
xi
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
_
y
X
Chap 12-28
Coefficient of Determination, R2
The coefficient of determination is the portion
of the total variation in the dependent variable
that is explained by variation in the
independent variable
The coefficient of determination is also called
R-squared and is denoted as R2
SSR regression sum of squares
R
SST
total sum of squares
2
note:
0 R 1
Chap 12-29
Examples of Approximate
r2 Values
Y
r2 = 1
r2 = 1
r =1
2
Chap 12-30
Examples of Approximate
r2 Values
Y
0 < r2 < 1
X
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-31
Examples of Approximate
r2 Values
r2 = 0
No linear relationship
between X and Y:
r2 = 0
Chap 12-32
Excel Output
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
ANOVA
SSR 18934.9348
R
0.58082
SST 32600.5000
2
Regression Statistics
10
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-33
Correlation and R2
The coefficient of determination, R2, for a
simple regression is equal to the simple
correlation squared
R r
2
2
xy
Chap 12-34
Estimation of Model
Error Variance
An estimator for the variance of the population model
error is
n
2
e
i
SSE
s
n2 n2
2
2
e
i1
Chap 12-35
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
ANOVA
s e 41.33032
10
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-36
small se
large se
Chap 12-37
2
2
(xi x) (n 1)s x
where:
sb1
SSE
se
n2
Chap 12-38
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
Observations
ANOVA
sb1 0.03297
41.33032
10
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-39
small Sb1
large Sb1
Chap 12-40
Test statistic
b1 1
t
sb1
d.f. n 2
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
where:
b1 = regression slope
coefficient
1 = hypothesized slope
sb1 = standard
error of the slope
Chap 12-41
Square Feet
(x)
245
1400
312
1600
279
1700
308
1875
199
1100
219
1550
405
2350
324
2450
319
1425
255
1700
Chap 12-42
H1: 1 0
Coefficients
Intercept
Square Feet
b1
Standard Error
sb1
t Stat
P-value
98.24833
58.03348
1.69296
0.12892
0.10977
0.03297
3.32938
0.01039
b1 1 0.10977 0
t
3.32938
t
sb1
0.03297
Chap 12-43
H1: 1 0
Coefficients
Intercept
Square Feet
d.f. = 10-2 = 8
t8,.025 = 2.3060
/2=.025
Reject H0
/2=.025
Do not reject H0
-tn-2,/2
-2.3060
Reject H0
tn-2,/2
2.3060 3.329
b1
Standard Error
sb1
t Stat
P-value
98.24833
58.03348
1.69296
0.12892
0.10977
0.03297
3.32938
0.01039
Decision:
Reject H0
Conclusion:
There is sufficient evidence
that square footage affects
house price
Chap 12-44
P-value = 0.01039
H0: 1 = 0
H1: 1 0
Coefficients
Intercept
Square Feet
P-value
Standard Error
t Stat
P-value
98.24833
58.03348
1.69296
0.12892
0.10977
0.03297
3.32938
0.01039
Chap 12-45
Standard Error
t Stat
P-value
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-46
Standard Error
t Stat
P-value
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-47
Prediction
The regression equation can be used to
predict a value for y, given a particular x
For a specified value, xn+1 , the predicted
value is
y n1 b0 b1x n1
Chap 12-48
Predictions Using
Regression Analysis
Predict the price for a house
with 2000 square feet:
Chap 12-49
Risky to try to
extrapolate far
beyond the range
of observed Xs
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-50
4 51
Where
=
predicted value of Y
b0 = sample intercept (an estimate of 0)
sample coefficient of the ith variable (an estimate
of i)
4 52
where
=
predicted value of dependent variable
(selling price)
b0 =
Y intercept
X1 and X2 =
value of the two independent
variables (square footage and age) respectively
b1 and b2 =
slopes for X1 and X2
respectively
a sample of houses
that have sold
Selects
recently and records the data
Copyright 2015 Pearson
Education, Inc.
4 53
SELLING
PRICE ($)
SQUARE
FOOTAGE
AGE
95,000
1,926
30
Good
119,000
2,069
40
Excellent
124,800
1,720
30
Excellent
135,000
1,396
15
Good
142,000
1,706
32
Mint
145,000
1,847
38
Mint
159,000
1,950
27
Mint
165,000
2,323
30
Excellent
182,000
2,285
26
Mint
183,000
3,752
35
Good
200,000
2,300
18
Good
211,000
2,525
17
Good
215,000
3,800
40
Excellent
1,740
12
Mint
Copyright 2015
Pearson
219,000
Education, Inc.
CONDITION
4 54
PROGRAM 4.2A Input Screen for the Jenny Wilson Realty Multiple Regression Example
4 55
Y b0 b1 X1 b2 X 2
146,630.89 43.82X1 2898.69X 2
Copyright 2015 Pearson
Education, Inc.
4 56
Evaluating Multiple
Regression Models
Similar to simple linear regression models
The p-value for the F test and r2 interpreted
the same
The hypothesis is different because there is
more than one independent variable
The F test is investigating whether all the
coefficients are equal to 0 at the same time
4 57
Evaluating Multiple
Regression Models
To determine which independent variables are
significant, tests are performed for each
variable
H 0 : 1 0
H1 : 1 0
The test statistic is calculated and if the pvalue is lower than the level of significance
(), the null hypothesis is rejected
Copyright 2015 Pearson
Education, Inc.
4 58
r2 = 0.6719
H 0 : 1 0
H1 : 1 0
For X1 (age)
For = 0.05, p-value = 0.0039
Copyright 2015 Pearson
Education, Inc.
r2 = 0.6719
H 0 : 1 0
H1 : 1 0
For X1 age
For = 0.05, p-value = 0.0039
Copyright 2015 Pearson
Education, Inc.
4 61
4 62
4 63
4 64
Coefficient of
determination,
r2 = 0.898
4 65
Nonlinear Regression
In some situations, variables are not linear
Transformations may be used to turn a
nonlinear model into a linear model
*
** *
***
*
Linear relationship
Copyright 2015 Pearson
Education, Inc.
*
* **
* **
*
*
*
Nonlinear relationship
4 66
Colonel Motors
Use regression analysis to improve fuel
efficiency
Study the impact of weight on miles per gallon (MPG)
TABLE 4.6
MPG
WEIGHT
(1,000 LBS.)
MPG
WEIGHT
(1,000 LBS.)
12
4.58
20
3.18
13
4.66
23
2.68
15
4.02
24
2.65
18
2.53
33
1.70
19
3.09
36
1.95
19
3.11
42
1.92
4 67
Colonel Motors
FIGURE 4.6A Linear Model for MPG Data
45
40
35
30
MPG
25
20
15
10
5
0
1.00
2.00
3.00
4.00
5.00
4 68
Colonel Motors
model
PROGRAM 4.6 Excel Output for Linear Regression Model withUseful
MPG Data
Y 47.6 8.2X1
Copyright 2015 Pearson
Education, Inc.
or
Colonel Motors
FIGURE 4.6B Nonlinear Model for MPG Data
45
40
35
30
MPG
25
20
15
10
5
0
1.00
2.00
3.00
4.00
5.00
4 70
Colonel Motors
The nonlinear model is a quadratic model
The easiest approach develop a new variable
X 2 (weight)2
New model
Y b0 b1 X1 b2 X 2
4 71
Colonel Motors
Improved
PROGRAM 4.7 Excel Output for Nonlinear Regression Model
with MPGmodel
Data
4 72