Simple Linear

Regression
Rini Novrianti Sutardjo Tui

Basic Concept
Simple Linear Regression

Basic Concept
Different values of x
will produce different
values of y

The equation of the line

y = o + 1x

Intercept o and slope

1

All means lie on a line

when plotted against x

Both y and x vary

according to normal
distributions

The normal
distributions all have
the same standard
deviation

Population regression
line describes how the
mean response changes
with x

The response y to a given

x is a random variable that
can take different values if
there are several
observations with the
same x-value

Function Population Regression

The population regression model:
Population
y intercept

Dependent
Variable

Population
Slope
Coefficient

Independent
Variable

y 0 1x
Linear component

Random
Error
term, or
residual

Random Error
component

Function Population Regression

Yi 0 1X i i

Observed value

i = random error
E Y 0 1 X i

X
Observed value

Function Population Regression

y 0 1x

Observed Value of
y for xi

Slope = 1

Predicted Value
of y for xi

Random Error
for this x value

Intercept = 0

xi

Function Population Regression (Error)

Error values () are statistically independent
Error values are normally distributed for any
given value of x
The probability distribution of the errors is
normal
The probability distribution of the errors has
constant variance

Function Sample Regression

The sample regression line provides an
of the population regression line
Estimated (or
predicted) y
value

Estimate of the
regression
intercept

estimate

Estimate of the
regression slope

y i b0 b1x

Independent
variable

The individual random error terms, i have a mean of zero

Function Sample Regression

Yi 0 1X i i
^i = random error
Unsampled observation

Yi 0 1X i
Observed value

Calculation
Simple Linear Regression

Least Square Method

60
40
20
0

20

40

X
60

How would you draw a line through the points?

How do you determine which line fits best?

Least Square Method

Slope unchanged

60
40
20
0

20

40

X
60

Slope changed

60
40
20
0

Intercept unchanged

20

40

X
60

60
40
20
0

Intercept changed

60
40
20
0

20

X
60

40

Slope changed

Intercept changed

20

40

X
60

Least Square Method

Best fit means difference between actual y values &
predicted y values are a minimum
The best line is the one that minimizes the sum of squared
vertical differences between the points and the line
The smaller the sum of squared differences, the better the fit
of the line to the data
b0 and b1 are obtained by finding the values of b0 and b1
that minimize the sum of the squared residuals

( x x )( y y )
b1
( x x )2

b1

x y
xy n

( x)
x
2

b0 y b1 x

Estimation Process
Regression Model
y = 0 + 1x +
Regression Equation
E(y) = 0 + 1x
Unknown Parameters
0, 1

b0 and b1
provide estimates of
0 and 1

Sample Data:
x
y
x1
y1
.
.
.
.
xn yn

Estimated
Regression Equation

y b0 b1 x

Sample Statistics
b0, b1

Regression Models
The equation that describes how y is related to x and an error
term is called the regression model
The regression model is

y 0 1 x

Data about x and y are obtained from a sample

From the sample of values of x and y, estimates b0 of 0 and
b1 of 1 are obtained
The resulting estimate of the model is

y b0 b1 x

THANK YOU
Econometrics