REGRESSION

ANALYSIS
ADVANCED RESEARCH METHADOLOGY
Regression Analysis

 Regression analysis is commonly used in research as it

establishes that a correlation exists between variables. But
correlation is not the same as causation. ...
 In Simple linear regression, each observation consists of two
values. One value is for the dependent variable and one value is
for the independent variable.
 In Multiple Regression, it is to learn more about the
relationship between several independent or predictor variables
and a dependent or criterion variable.
Modes of Regression Analysis
 Multiple Regression Analysis:
1.  In Multiple Regression analysis
We have one outcome
(dependent) variable and
multiple predictors.
Linear Regression Analysis:
1.Linear regression analysis
estimates the coefficients of a linear
equation, involving one or more
independent variables, that best
predict the value of the dependent
variable.
What is Multiple Regression
Analysis?
 The general purpose of multiple regression (the term was first
used by Pearson, 1908) is to learn more about the relationship
between several independent or predictor variables and a
dependent or criterion variable. For example, a real estate agent
might record for each listing the size of the house (in square
feet), the number of bedrooms, the average income in the
respective neighborhood according to census data, and a
subjective rating of appeal of the house. Once this information
has been compiled for various houses it would be interesting to
see whether and how these measures relate to the price for
which a house is sold. For example, you might learn that the
number of bedrooms is a better predictor of the price for which a
house sells in a particular neighborhood than how "pretty" the
house is (subjective rating). You may also detect "outliers," that
is, houses that should really sell for more, given their location
and characteristics.
What is Multiple Regression
Analysis?
What is Multiple Regression
Analysis?
Multiple Regression Analysis
How to get started with M.R.A
How to get started with M.R.A (Con’d)
Multiple Regression Analysis Sample
How to get started with M.R.A (Con’d)
Plotting of Graphs
Plotting of Residuals & Graphs
Plotting of Residuals & Graphs
(Con’d)
What is Linear Regression
Analysis
 The overall idea of regression is to examine two things:
 (1) does a set of predictor variables do a good job in predicting an
outcome (dependent) variable?
 (2) Which variables in particular are significant predictors of the
outcome variable, and in what way do they–indicated by the
magnitude and sign of the beta estimates–impact the outcome
variable? These regression estimates are used to explain the
relationship between one dependent variable and one or more
independent variables. The simplest form of the regression
equation with one dependent and one independent variable is
defined by the formula y = c + b*x, where y = estimated
dependent variable score, c = constant, b = regression coefficient,
and x = score on the independent variable
Naming the variables

 Response= Dependent variable (x- coordinate), the data that we

are gonna predict.
 Predicators= Independent variable(y- coordinate), on the basis of
which we are gonna predict.
 Error/Residuals= variation from mean line
Uses of Regression
Three major uses for regression analysis are
Determining the strength of predictors.
Regression might be used to identify the strength of the effect that the
independent variable(s) have on a dependent variable, i.e. strength of relationship
between sales and marketing spending, or age and income.
Forecasting an effect
It can be used to forecast effects or impact of changes. That is, the regression
analysis helps us to understand how much the dependent variable changes with a
change in one or more independent variables, i.e. how much additional sales income
do I get for each additional $1000 spent on marketing?
Trend forecasting.
Regression analysis predicts trends and future values.  The regression analysis
can be used to get point estimates.  A typical question is, “what will the price of gold
be in 6 months?”
Start with the Basic’s

S.No Independent Dependent

1 1 2
2 2 4
3 3 5
4 4 4
5 5 5
Cont. Taking the mean of values

S.No Independent (x) Dependent (y)

1 1 2
2 2 4
3 3 5
4 4 4
5 5 5
Mean 3 4
Relationship b/w dependent
variables and Mean variable
R squared

Independen Dependent
S.No X-xx Y-ȳ
t (x) (y)
1 1 2 -2 -2
2 2 4 -1 0
3 3 5 0 1
4 4 4 1 0
5 5 5 2 1
Mean 3 4
R squared Cont.

Independen Dependent
S.No X-xx Y-ȳ
t (x) (y)
1 1 2 -2 -2
2 2 4 -1 0
3 3 5 0 1
4 4 4 1 0
5 5 5 2 1
Mean 3 4 0 0
R squared Cont.

Indepen Depende (x-

S.No x-xx y-ȳ (y-ȳ)*(x-xx̅ )
dent (x) nt (y) xx̅ )2
1 1 2 -2 -2 4 4
2 2 4 -1 0 1 0
3 3 5 0 1 0 0
4 4 4 1 0 1 0
5 5 5 2 1 4 2
Mean 3 4 10 6
Formula and Calculations
 Indepe Depen Y
(y-ȳ)*(x-
S.No ndent dent x-xx̅ y-ȳ (x-xx̅ )2 pred
xx̅ )
(x) (y) icted
1 1 2 -2 -2 4 4 2.8
2 2 4 -1 0 1 0 3.4
3 3 5 0 1 0 0 4
4 4 4 1 0 1 0 4.6
5 5 5 2 1 4 2 5.2
Mean 3 4 10 6 20
 Inde Y
Depe (y-
pend (y- (x-xx̅ ) pre Error
S.No nden x-xx̅ y-ȳ ȳ)*(x-xx̅
ent ȳ)2 2 dict (y)
t (y) )
(x) ed
1 1 2 -2 -2 4 4 4 2.8 -0.8
2 2 4 -1 0 0 1 0 3.4 0.6
3 3 5 0 1 1 0 0 4 1
4 4 4 1 0 0 1 0 4.6 -0.6
5 5 5 2 1 1 4 2 5.2 -0.2
Mean 3 4 6 10 6 20 2.4
 Indepe Depe (y- Y
S.N x- (y- (x-
ndent nden y-ȳ ȳ)*(x-xx̅ predicte Error SSE
o xx̅ ȳ)2 xx̅ )2
(x) t (y) ) d
1 1 2 -2 -2 4 4 4 2.8 -0.8 0.64
2 2 4 -1 0 0 1 0 3.4 0.6 0.36
3 3 5 0 1 1 0 0 4 1 1
4 4 4 1 0 0 1 0 4.6 -0.6 0.36
5 5 5 2 1 1 4 2 5.2 -0.2 0.04
Me
3 4 6 10 6 20 2.4 2.4
an
Relation b/w Mean, dependent
variable and Regression line
Lets learn
Regression on
Minitab