Analysis of Time Series Data Using R

1
ZONGWU CAI
a,b,c
E-mail address: zcai@uncc.edu
a
Department of Mathematics & Statistics and Department of Economics,
University of North Carolina, Charlotte, NC 28223, U.S.A.
b
Wang Yanan Institute for Studies in Economics, Xiamen University, China
c
College of Economics and Management, Shanghai Jiaotong University, China
July 30, 2006
c 2006, ALL RIGHTS RESERVED by ZONGWU CAI
1
This manuscript may be printed and reproduced for individual or instructional use, but may
not be printed for commercial purposes.
Preface
The purpose of this lecture notes is designed to provide an overview of methods that
are useful for analyzing univariate and multivariate phenomena measured over time. Since
this is a course emphasizing applications with both theory and applications, the reader is
guided through examples involving real time series in the lectures. A collection of simple
theoretical and applied exercises assuming a background that includes a beginning level
course in mathematical statistics and some computing skills follows each chapter. More
importantly, the computer code in Rand datasets are provided for most of examples analyzed
in this lecture notes.
Some materials are based on the lecture notes given by Professor Robert H. Shumway,
Department of Statistics, University of California at Davis and my colleague, Professor
Stanislav Radchenko, Department of Economics, University of North Carolina at Charlotte.
Some datasets are provided by Professor Robert H. Shumway, Department of Statistics, Uni-
versity of California at Davis and Professor Phillips Hans Franses at University of Rotterdam,
Netherland. I am very grateful to them for providing their lecture notes and datasets.
Contents
1 Package R and Simple Applications 1
1.1 Computational Toolkits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 How to Install R ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Data Analysis and Graphics Using R An Introduction (109 pages) . . . . . 4
1.4 CRAN Task View: Empirical Finance . . . . . . . . . . . . . . . . . . . . . . 4
1.5 CRAN Task View: Computational Econometrics . . . . . . . . . . . . . . . . 8
2 Characteristics of Time Series 15
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 Stationary Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2.1 Detrending . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2.2 Dierencing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.3 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.4 Linear Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.3 Other Key Features of Time Series . . . . . . . . . . . . . . . . . . . . . . . 30
2.3.1 Seasonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3.2 Aberrant Observations . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.3.3 Conditional Heteroskedasticity . . . . . . . . . . . . . . . . . . . . . . 34
2.3.4 Nonlinearity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.4 Time Series Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.4.1 Autocorrelation Function . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.4.2 Cross Correlation Function . . . . . . . . . . . . . . . . . . . . . . . . 40
2.4.3 Partial Autocorrelation Function . . . . . . . . . . . . . . . . . . . . 45
2.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.6 Computer Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3 Univariate Time Series Models 69
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.2 Least Squares Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.3 Model Selection Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.3.1 Subset Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.3.2 Sequential Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
3.3.3 Likelihood Based-Criteria . . . . . . . . . . . . . . . . . . . . . . . . 85
3.3.4 Cross-Validation and Generalized Cross-Validation . . . . . . . . . . 87
ii
CONTENTS iii
3.3.5 Penalized Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.4 Integrated Models - I(1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.5 Autoregressive Models - AR(p) . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.5.1 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.5.2 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.6 Moving Average Models MA(q) . . . . . . . . . . . . . . . . . . . . . . . . 102
3.7 Autoregressive Integrated Moving Average Model - ARIMA(p, d, q) . . . . . 106
3.8 Seasonal ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.9 Regression Models With Correlated Errors . . . . . . . . . . . . . . . . . . . 120
3.10 Estimation of Covariance Matrix . . . . . . . . . . . . . . . . . . . . . . . . 130
3.11 Long Memory Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
3.12 Periodicity and Business Cycles . . . . . . . . . . . . . . . . . . . . . . . . . 136
3.13 Impulse Response Function . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
3.13.1 First Order Dierence Equations . . . . . . . . . . . . . . . . . . . . 142
3.13.2 Higher Order Dierence Equations . . . . . . . . . . . . . . . . . . . 146
3.14 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
3.15 Computer Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
3.16 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
4 Non-stationary Processes and Structural Breaks 185
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
4.2 Random Walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
4.2.1 Inappropriate Detrending . . . . . . . . . . . . . . . . . . . . . . . . 189
4.2.2 Spurious (nonsense) Regressions . . . . . . . . . . . . . . . . . . . . . 190
4.3 Unit Root and Stationary Processes . . . . . . . . . . . . . . . . . . . . . . . 190
4.3.1 Comparison of Forecasts of TS and DS Processes . . . . . . . . . . . 191
4.3.2 Random Walk Components and Stochastic Trends . . . . . . . . . . . 193
4.4 Trend Estimation and Forecasting . . . . . . . . . . . . . . . . . . . . . . . . 194
4.4.1 Forecasting a Deterministic Trend . . . . . . . . . . . . . . . . . . . . 194
4.4.2 Forecasting a Stochastic Trend . . . . . . . . . . . . . . . . . . . . . . 195
4.4.3 Forecasting ARMA models with Deterministic Trends . . . . . . . . . 195
4.4.4 Forecasting of ARIMA Models . . . . . . . . . . . . . . . . . . . . . . 196
4.5 Unit Root Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
4.5.1 The Dickey-Fuller and Augmented Dickey-Fuller Tests . . . . . . . . 197
4.5.2 Cautions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
4.6 Structural Breaks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
4.6.1 Testing for Breaks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
4.6.2 Zivot and Andrewss Testing Procedure . . . . . . . . . . . . . . . . . 203
4.6.3 Cautions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
4.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
4.8 Computer Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
4.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
CONTENTS iv
5 Vector Autoregressive Models 215
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
5.1.1 Properties of VAR Models . . . . . . . . . . . . . . . . . . . . . . . . 218
5.1.2 Statistical Inferences . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
5.2 Impulse-Response Function . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
5.3 Variance Decompositions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
5.4 Granger Causality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
5.5 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
5.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
5.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
6 Cointegration 234
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
6.2 Cointegrating Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
6.3 Testing for Cointegration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
6.4 Cointegrated VAR Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
6.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
6.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
7 Nonparametric Density, Distribution & Quantile Estimation 244
7.1 Mixing Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
7.2 Density Estimate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
7.2.1 Asymptotic Properties . . . . . . . . . . . . . . . . . . . . . . . . . . 246
7.2.2 Optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
7.2.3 Boundary Correction . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
7.3 Distribution Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
7.3.1 Smoothed Distribution Estimation . . . . . . . . . . . . . . . . . . . 254
7.3.2 Relative Eciency and Deciency . . . . . . . . . . . . . . . . . . . . 257
7.4 Quantile Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
7.4.1 Value at Risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
7.4.2 Nonparametric Quantile Estimation . . . . . . . . . . . . . . . . . . . 260
7.5 Computer Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
7.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
8 Nonparametric Regression Estimation 267
8.1 Bandwidth Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
8.1.1 Simple Bandwidth Selectors . . . . . . . . . . . . . . . . . . . . . . . 267
8.1.2 Cross-Validation Method . . . . . . . . . . . . . . . . . . . . . . . . . 268
8.2 Multivariate Density Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 270
8.3 Regression Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
8.4 Kernel Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
8.4.1 Asymptotic Properties . . . . . . . . . . . . . . . . . . . . . . . . . . 274
8.4.2 Boundary Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
8.5 Local Polynomial Estimate . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
8.5.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
CONTENTS v
8.5.2 Implementation in R . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
8.5.3 Complexity of Local Polynomial Estimator . . . . . . . . . . . . . . . 281
8.5.4 Properties of Local Polynomial Estimator . . . . . . . . . . . . . . . 284
8.5.5 Bandwidth Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
8.6 Functional Coecient Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
8.6.1 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
8.6.2 Local Linear Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 293
8.6.3 Bandwidth Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
8.6.4 Smoothing Variable Selection . . . . . . . . . . . . . . . . . . . . . . 296
8.6.5 Goodness-of-Fit Test . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
8.6.6 Asymptotic Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
8.6.7 Conditions and Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . 301
8.6.8 Monte Carlo Simulations and Applications . . . . . . . . . . . . . . . 311
8.7 Additive Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
8.7.1 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
8.7.2 Backtting Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 315
8.7.3 Projection Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
8.7.4 Two-Stage Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
8.7.5 Monte Carlo Simulations and Applications . . . . . . . . . . . . . . . 322
8.8 Computer Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
8.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
List of Tables
3.1 AICC values for ten models for the recruits series . . . . . . . . . . . . . . . 98
4.1 Large-sample critical values for the ADF statistic . . . . . . . . . . . . . . . 198
4.2 Summary of DF test for unit roots in the absence of serial correlation . . . . 199
4.3 Critical Values of the QLR statistic with 15% Trimming . . . . . . . . . . . 203
5.1 Sims variance decomposition in three variable VAR model . . . . . . . . . . 228
5.2 Sims variance decomposition including interest rates . . . . . . . . . . . . . . 228
6.1 Critical values for the Engle-Granger ADF statistic . . . . . . . . . . . . . . 238
8.1 Sample sizes required for p-dimensional nonparametric regression to have com-
parable performance with that of 1-dimensional nonparametric regression us-
ing size 100 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
vi
List of Figures
2.1 Monthly SOI (left) and simulated recruitment (right) from a model (n=453
months, 1950-1987). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2 Simulated MA(1) with
1
= 0.9. . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3 Log of annual indices of real national output in China, 1952-1988. . . . . . . 22
2.4 Monthly average temperature in degrees centigrade, January, 1856 - February
2005, n = 1790 months. The straight line (wide and green) is the linear trend
y = 9.037 + 0.0046 t and the curve (wide and red) is the nonparametric
estimated trend. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.5 Detrended monthly global temperatures: left panel (linear) and right panel
(nonlinear). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.6 Dierenced monthly global temperatures. . . . . . . . . . . . . . . . . . . . . 25
2.7 Annual stock of motor cycles in the Netherlands, 1946-1993. . . . . . . . . . 26
2.8 Quarterly earnings for Johnson & Johnson (4th quarter, 1970 to 1st quarter,
1980, left panel) with log transformed earnings (right panel). . . . . . . . . . 27
2.9 The SOI series (black solid line) compared with a 12 point moving average
(red thicker solid line). The top panel: original data and the bottom panel:
ltered series. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.10 US Retail Sales Data from 1967-2000. . . . . . . . . . . . . . . . . . . . . . . 31
2.11 Four-weekly advertising expenditures on radio and television in The Nether-
lands, 1978.01 1994.13. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.12 First dierence in log prices versus the ination rate: the case of Argentina,
1970.1 1989.4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.13 Japanese - U.S. dollar exchange rate return series {y
t
}, from January 1, 1974
to December 31, 2003. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.14 Quarterly unemployment rate in Germany, 1962.1 1991.4 (seasonally ad-
justed and not seasonally adjusted) in the left panel. The scatterplot of un-
employment rate (seasonally adjusted) versus unemployment rate (seasonally
adjusted) one period lagged in the right panel. . . . . . . . . . . . . . . . . . 37
2.15 Multiple lagged scatterplots showing the relationship between SOI and the
present (x
t
) versus the lagged values (x
t+h
) at lags 1 h 16. . . . . . . . . 39
2.16 Autocorrelation functions of SOI and recruitment and cross correlation func-
tion between SOI and recruitment. . . . . . . . . . . . . . . . . . . . . . . . 41
2.17 Multiple lagged scatterplots showing the relationship between the SOI at time
t +h, say x
t+h
(x-axis) versus recruits at time t, say y
t
(y-axis), 0 h 15. 42
vii
LIST OF FIGURES viii
2.18 Multiple lagged scatterplots showing the relationship between the SOI at time
t, say x
t
(x-axis) versus recruits at time t +h, say y
t+h
(y-axis), 0 h 15. 42
2.19 Partial autocorrelation functions for the SOI (left panel) and the recruits
(right panel) series. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.20 Varve data for Problem 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.21 Gas and oil series for Problem 6. . . . . . . . . . . . . . . . . . . . . . . . . 51
2.22 Handgun sales (per 10,000,000) in California and monthly gun death rate (per
100,00) in California (February 2, 1980 -December 31, 1998. . . . . . . . . . 53
3.1 Autocorrelation functions (ACF) for simple (left) and log (right) returns for
IBM (top panels) and for the value-weighted index of US market (bottom
panels), January 1926 to December 1997. . . . . . . . . . . . . . . . . . . . . 72
3.2 Autocorrelation functions (ACF) and partial autocorrelation functions (PACF)
for the detrended (top panel) and dierenced (bottom panel) global temper-
ature series. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.3 A typical realization of the random walk series (left panel) and the rst dif-
ference of the series (right panel). . . . . . . . . . . . . . . . . . . . . . . . . 91
3.4 Autocorrelation functions (ACF) (left) and partial autocorrelation functions
(PACF) (right) for the random walk (top panel) and the rst dierence (bot-
tom panel) series. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.5 Autocorrelation (ACF) of residuals of AR(1) for SOI (left panel) and the plot
of AIC and AICC values (right panel). . . . . . . . . . . . . . . . . . . . . . 99
3.6 Autocorrelation functions (ACF) and partial autocorrelation functions (PACF)
for the log varve series (top panel) and the rst dierence (bottom panel),
showing a peak in the ACF at lag h = 1. . . . . . . . . . . . . . . . . . . . . 104
3.7 Number of live births 1948(1)1979(1) and residuals from models with a rst
dierence, a rst dierence and a seasonal dierence of order 12 and a tted
ARIMA(0, 1, 1) (0, 1, 1)
12
model. . . . . . . . . . . . . . . . . . . . . . . . 111
3.8 Autocorrelation functions (ACF) and partial autocorrelation functions (PACF)
for the birth series (top two panels), the rst dierence (second two panels) an
ARIMA(0, 1, 0)(0, 1, 1)
12
model (third two panels) and an ARIMA(0, 1, 1)
(0, 1, 1)
12
model (last two panels). . . . . . . . . . . . . . . . . . . . . . . . . 112
3.9 Autocorrelation functions (ACF) and partial autocorrelation functions (PACF)
for the log J&J earnings series (top two panels), the rst dierence (sec-
ond two panels), ARIMA(0, 1, 0) (1, 0, 0)
4
model (third two panels), and
ARIMA(0, 1, 1) (1, 0, 0)
4
model (last two panels). . . . . . . . . . . . . . . 115
3.10 Autocorrelation functions (ACF) and partial autocorrelation functions (PACF)
for ARIMA(0, 1, 1)(0, 1, 1)
4
model (top two panels) and the residual plots of
ARIMA(0, 1, 1) (1, 0, 0)
4
(left bottom panel) and ARIMA(0, 1, 1) (0, 1, 1)
4
model (right bottom panel). . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
3.11 Monthly simple return of CRSP Decile 1 index from January 1960 to December
2003: Time series plot of the simple return (left top panel), time series plot
of the simple return after adjusting for January eect (right top panel), the
ACF of the simple return (left bottom panel), and the ACF of the adjusted
simple return. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
LIST OF FIGURES ix
3.12 Autocorrelation functions (ACF) and partial autocorrelation functions (PACF)
for the detrended log J&J earnings series (top two panels)and the tted
ARIMA(0, 0, 0) (1, 0, 0)
4
residuals. . . . . . . . . . . . . . . . . . . . . . . 123
3.13 Time plots of U.S. weekly interest rates (in percentages) from January 5, 1962
to September 10, 1999. The solid line (black) is the Treasury 1-year constant
maturity rate and the dashed line the Treasury 3-year constant maturity rate
(red). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
3.14 Scatterplots of U.S. weekly interest rates from January 5, 1962 to September
10, 1999: the left panel is 3-year rate versus 1-year rate, and the right panel
is changes in 3-year rate versus changes in 1-year rate. . . . . . . . . . . . . 125
3.15 Residual series of linear regression Model I for two U.S. weekly interest rates:
the left panel is time plot and the right panel is ACF. . . . . . . . . . . . . . 126
3.16 Time plots of the change series of U.S. weekly interest rates from January 12,
1962 to September 10, 1999: changes in the Treasury 1-year constant maturity
rate are in denoted by black solid line, and changes in the Treasury 3-year
constant maturity rate are indicated by red dashed line. . . . . . . . . . . . . 127
3.17 Residual series of the linear regression models: Model II (top) and Model III
(bottom) for two change series of U.S. weekly interest rates: time plot (left)
and ACF (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
3.18 Sample autocorrelation function of the absolute series of daily simple returns
for the CRSP value-weighted (left top panel) and equal-weighted (right top
panel) indexes. The log spectral density of the absolute series of daily simple
returns for the CRSP value-weighted (left bottom panel) and equal-weighted
(right bottom panel) indexes. . . . . . . . . . . . . . . . . . . . . . . . . . . 135
3.19 The autocorrelation function of an AR(2) model: (a)
1
= 1.2 and
2
= 0.35,
(b)
1
= 1.0 and
2
= 0.7, (c)
1
= 0.2 and
2
= 0.35, (d)
1
= 0.2 and

2
= 0.35. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
3.20 The growth rate of US quarterly real GNP from 1947.II to 1991.I (seasonally
adjusted and in percentage): the left panel is the time series plot and the right
panel is the ACF. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
3.21 The time-series y
t
is generated with w
t
N(0, 1), y
0
= 5. At period t = 50,
there is an additional impulse to the error term, i.e. w
50
= w
50
+ 1. The
impulse response function is computed as the dierence between the series y
t
without impulse and the series y
t
with the impulse. . . . . . . . . . . . . . . 143
3.22 The time-series y
t
is generated with w
t
N(0, 1), y
0
= 3. At period t = 50,
there is an additional impulse to the error term, i.e. w
50
= w
50
+ 1. The
impulse response function is computed as the dierence between the series y
t
without impulse and the series y
t
with the impulse. . . . . . . . . . . . . . . 144
3.23 Example of impulse response functions for rst order dierence equations. . . 146
3.24 The time series y
t
is generated with w
t
N(0, 1), y
0
= 3. For the transitory
impulse, there is an additional impulse to the error term at period t = 50, i.e.
w
50
= w
50
+1. For the permanent impulse, there is an additional impulse for
period t = 50, , 100, i.e. w
t
= w
t
+ 1, t = 50, 51, , 100. The impulse
response function (IRF) is computed as the dierence between the series y
t
without impulse and the series y
t
with the impulse. . . . . . . . . . . . . . . 147
LIST OF FIGURES x
3.25 Example of impulse response functions for second order dierence equation. . 149
Chapter 1
Package R and Simple Applications
1.1 Computational Toolkits
\hcn you work with arc datascts, mcssy data handin, modcs,
ctc, you nccd to choosc thc comjutationa toos that arc usctu tor
dcain with thcsc kinds ot jrocms Thcrc arc menu driven
systems whcrc you cick somc uttons and ct somc work donc
- ut thcsc arc usccss tor anythin nontrivia To do scrious cco-
nomics and nancc in thc modcrn days, you havc to writc com-
jutcr jrorams And this is truc ot any cd, tor cxamjc, cmjirica
macrocconomics - and not ,ust ot comjutationa nancc which is
a hot uzzword rcccnty
Thc qucstion is how to choosc thc comjutationa toos Accord-
in to A,ay Shah (Lcccmcr 200`), you shoud jay attcntion to tour
ccmcnts price, freedom, elegant and powerful computer
science, and network eects Low jricc is cttcr than hih jricc
lricc 0 is oviousy cst ot a Irccdom hcrc is in many asjccts
A ood sottwarc systcm is onc that docsnt tic you down in tcrms ot
hardwarc,OS, so that you arc ac to kccj movin Anothcr asjcct
ot trccdom is in workin with cocaucs, coaorators and studcnts
\ith commcrcia sottwarc, this ccomcs a jrocm, ccausc your
cocaucs may not havc thc samc sottwarc that you arc usin Lcrc
1
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 2
trcc sottwarc rcay wins sjcctacuary Good jracticc in rcscarch in-
vovcs a rcat acccnt on rcjroduciiity lcjroduciiity is imjortant
oth so as to avoid mistakcs, and ccausc thc ncxt jcrson workin
in your cd shoud c standin on your shoudcrs This rcquircs
an aiity to rccasc codc This is ony jossic with trcc sottwarc
Systcms ikc SAS and Gauss usc archaic comjutcr scicncc Thc
codc is inccant Thc anuac is not jowcrtu ln this day and ac,
writin C or fortran y hand is too ow cvc Lc, with Gauss,
cvcn a minima thin ikc oninc hcj is tawdry Onc jrctcrs a systcm
to c uit y jcojc who know thcir comjutcr scicncc - it shoud c
an ccant, jowcrtu anuac A standard CS knowcdc shoud c
niccy in jay to ivc you a orcous systcm Good comjutcr scicncc
ivcs you morc jroductivc humans Lots ot cconomists usc Gauss,
and ivc out Gauss sourcc codc, so thcrc is a nctwork ccct in tavor
ot Gauss A simiar thin is riht now hajjcnin with statisticians
and R
Lcrc l citc comjarisons amon most commony uscd jackacs (scc
A,ay Shah (Lcccmcr 200`)). scc thc wc sitc at
http://www.mayin.org/ajayshah/
COMPUTING/mytools.html
R is a vcry convcnicnt jrorammin anuac tor doin statistica
anaysis and `ontc Caro simuations as wc as various ajjications
in quantitativc cconomics and nancc lndccd, wc jrctcr to think ot
it ot an cnvironmcnt within which statistica tcchniqucs arc imjc-
mcntcd l wi tcach it at thc introductory cvc, ut `OTlCL that
you wi havc to carn R on your own `otc that aout 9/ ot com-
mands in S-PLUS and Rarc samc ln jarticuar, tor anayzin timc
scrics data, R has a ot ot undcs and jackacs, which can c down-
oadcd tor trcc, tor cxamjc, at http://www.r-project.org/
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 3
R, ikc S, is dcsincd around a truc comjutcr anuac, and it
aows uscrs to add additiona tunctionaity y dcnin ncw tunctions
`uch ot thc systcm is itsct writtcn in thc R diacct ot S, which
makcs it casy tor uscrs to toow thc aorithmic choiccs madc Ior
comjutationay-intcnsivc tasks, C, C++ and Fortran codc can
c inkcd and cacd at run timc Advanccd uscrs can writc C codc
to manijuatc R o,ccts dirccty
1.2 How to Install R ?
(1) o to thc wc sitc http://www.r-project.org/.
(2) cick CRAN.
(3) choosc a sitc tor downoadin, say http://cran.cnr.Berkeley.edu.
(4) cick Windows (95 and later).
(5) cick base.
(6) cick R-2.3.1-win32.exe (\crsion ot 0o-01-200o) to savc this
c rst and thcn run it to insta
Thc asic l is instacd into your comjutcr lt you nccd to insta
othcr jackacs, you nccd to do thc toowins
(7) Attcr it is instacd, thcrc is an icon on thc scrccn Cick thc icon
to ct into l.
(8) Go to thc toj and nd packages and thcn cick it.
(9) Go down to Install package(s)... and cick it.
(10) Thcrc is a ncw window Choosc a ocation to downoad thc
jackacs, say USA(CA1), movc mousc to thcrc and cick OI.
(11) Thcrc is a ncw window istin a jackacs You can sccct any
onc ot jackacs and cick OI, or you can sccct a ot thcm and thcn
cick OI
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 4
1.3 Data Analysis and Graphics Using R An Intro-
duction (109 pages)
Scc thc c r-notes.pdf (109 jacs) which can c downoadcd trom
http://www.math.uncc.edu/ zcai/r-notes.pdf
l cncourac you to downoad this c and carn it y yoursct
1.4 CRAN Task View: Empirical Finance
This ClA` Task \icw contains a ist ot jackacs usctu tor cmjirica
work in Iinancc, roujcd y tojic Lcsidcs thcsc jackacs, a vcry
widc varicty ot tunctions suitac tor cmjirica work in Iinancc is
jrovidcd y oth thc asic R systcm (and its sct ot rccommcndcd
corc jackacs), and a numcr ot othcr jackacs on thc Comjrchcn-
sivc R Archivc `ctwork (ClA`) Conscqucnty, scvcra ot thc othcr
ClA` Task \icws may contain suitac jackacs, in jarticuar thc
Econometrics Task \icw Thc wc sitc is
http://cran.r-project.org/src/contrib/Views/Finance.html
1 Standard regression models: Lincar modcs such as ordi-
nary cast squarcs (OLS) can c cstimatcd y lm() (trom y
thc stats jackac containcd in thc asic l distriution) `ax-
imum Likcihood (`L) cstimation can c undcrtakcn with thc
optim() tunction `on-incar cast squarcs can c cstimatcd
with thc nls() tunction, as wc as with nlme() trom thc nlme
jackac Ior thc incar modc, a varicty ot rcrcssion dianostic
tcsts arc jrovidcd y thc car, lmtest, strucchange, urca,
uroot, and sandwich jackacs Thc Rcmdr and Zelig jack-
acs jrovidc uscr intcrtaccs that may c ot intcrcst as wc
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 5
2 Time series: Cassica timc scrics tunctionaity is jrovidcd
y thc arima() and KalmanLike() commands in thc asic
l distriution Thc dse jackacs jrovidcs a varicty ot morc
advanccd cstimation mcthods. fracdi can cstimatc traction-
ay intcratcd scrics. longmemo covcrs rcatcd matcria Ior
volatily modcin, thc standard GARCH(1,1) modc can c
cstimatcd with thc garch() tunction in thc tseries jackac
nit root and cointcration tcsts arc jrovidcd y tseries, urca
and uroot Thc Rmetrics jackacs fSeries and fMultivar
contain a numcr ot cstimation tunctions tor Al`A, GAlCL,
on mcmory modcs, unit roots and morc Thc ArDec im-
jcmcnts autorcrcssivc timc scrics dccomjosition in a Laycsian
tramcwork Thc dyn and dynlm arc suitac tor dynamic (in-
car) rcrcssion modcs Scvcra jackacs jrovidc wavcct ana-
ysis tunctionaity rwt, wavelets, waveslim, wavethresh
Somc mcthods trom chaos thcory arc jrovidcd y thc jackac
tseriesChaos
3 Finance: Thc Rmetrics undc comjriscd ot thc fBasics,
fCalendar, fSeries, fMultivar, fPortfolio, fOptions and
fExtremes jackacs contains a vcry arc numcr ot rccvant
tunctions tor dicrcnt asjcct ot cmjirica and comjutationa
nancc Thc RQuantLib jackac jrovidcs scvcra ojtion-
jricin tunctions as wc as somc xcd-incomc tunctionaity trom
thc QuantLib jro,cct to R Thc jorttoio jackac contains
casscs tor cquity jorttoio manacmcnt
! Risk Management: Thc VaR jackac cstimatcs \auc-at-
lisk, and scvcra jackacs jrovidc tunctionaity tor Lxtrcmc
\auc Thcory modcs evd, evdbayes, evir, extRremes,
ismec, POT Thc mvtnorm jackac jrovidcs codc tor mu-
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 6
tivariatc `orma and t-distriutions Thc Rmetrics jackacs
fPortfolio and fExtremes aso contain a numcr ot rccvant
tunctions Thc copula and fgac jackacs covcr mutivariatc
dcjcndcncy structurcs usin cojua mcthods
` Data and Date Management: Thc its, zoo and fCalendar
(jart ot Rmetrics) jackacs jrovidc sujjort tor irrcuary-
sjaccd timc scrics fCalendar aso addrcsscs cacndar issucs
such as rccurrin hoidays tor a arc numcr ot nancia ccn-
tcrs, and jrovidcs codc tor hih-trcqucncy data scts
CRAN packages:
ArLcc
car
cojua
dsc
dyn
dynm
cvd
cvdaycs
cvir
cxtlcmcs
tLasics (corc)
tCacndar (corc)
tLxtrcmcs (corc)
tac
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 7
t`utivar (corc)
tOjtions (corc)
tlorttoio (corc)
tracdi
tScrics (corc)
ismcv
its (corc)
mtcst
onmcmo
mvtnorm
jorttoio
lOT
lcmdr
lQuantLi (corc)
rwt
sandwich
strucchanc
tscrics (corc)
tscricsChaos
urca (corc)
uroot
\al
wavccts
wavcsim
wavcthrcsh
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 8
Zci
zoo (corc)
Related links:
ClA` Task \icw Lconomctrics Thc wc sitc is
http://cran.cnr.berkeley.edu/src/
contrib/Views/Econometrics.html
or scc thc ncxt scction
lmctrics y Licthcm \ucrtz contains a wcath ot R
codc tor Iinancc Thc wc sitc is
http://www.itp.phys.ethz.ch/econophysics/R/
Quanti is a C++ irary tor quantitativc nancc Thc
wc sitc is
http://quantlib.org/
`aiin ist l Sjccia lntcrcst Grouj Iinancc
1.5 CRAN Task View: Computational Econometrics
Lasc R shijs with a ot ot tunctionaity usctu tor comjutationa
cconomctrics, in jarticuar in thc stats jackac This tunctionaity
is comjcmcntcd y many jackacs on ClA`, a rict ovcrvicw is
ivcn cow Thcrc is aso a considcrac ovcraj ctwccn thc toos
tor cconomctrics in this vicw and nancc in thc Finance vicw Iur-
thcrmorc, thc nance SIG is a suitac maiin ist tor otainin
hcj and discussin qucstions aout oth comjutationa nancc and
cconomctrics Thc jackacs in this vicw can c rouhy structurcd
into thc toowin tojics Thc wc sitc is
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 9
http://cran.r-project.org/src/contrib/
Views/Econometrics.html
1 Linear regression models: Lincar modcs can c ttcd (via
OLS) with lm() (trom stats) and standard tcsts tor modc com-
jarisons arc avaiac in various mcthods such as summary()
and anova() Anaoous tunctions that aso sujjort asymj-
totic tcsts (z instcad ot t tcsts, and Chi-squarcd instcad ot I
tcsts) and ju-in ot othcr covariancc matriccs arc coeftest()
and waldtest() in lmtest Tcsts ot morc cncra incar hy-
jothcscs arc imjcmcntcd in linear.hypothesis() in car LC
and LAC covariancc matriccs that can c jucd into thcsc
tunctions arc avaiac in sandwich Thc jackacs car and mtcst
aso jrovidc a arc cocction ot turthcr mcthods tor dianostic
chcckin in incar rcrcssion modcs
2 Microeconometrics: `any standard micro-cconomctric mod-
cs con to thc tamiy ot cncraizcd incar modcs (GL`) and
can c ttcd y glm() trom jackac stats This incudcs in
jarticuar oit and jroit modcs tor modcin choicc data and
joisson modcs tor count data `cativc inomia GL`s arc
avaiac via glm.nb() in jackac `ASS trom thc VR undc
Zcro-inatcd count modcs arc jrovidcd in zicounts Iurthcr
ovcr-disjcrscd and inatcd modcs, incudin hurdc modcs, arc
avaiac in jackac pscl Livariatc joisson rcrcssion modcs
arc imjcmcntcd in bivpois Lasic ccnsorcd rcrcssion modcs
(c, toit modcs) can c ttcd y survreg() in survival
Iurthcr morc rcncd toos tor microccnomctrics arc jrovidcd in
micEcon Thc jackac bayesm imjcmcnts a Laycsian aj-
jroach to microcconomctrics and markctin lntcrcncc tor rca-
tivc distriutions is containcd in jackac reldist
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 10
3 Further regression models: \arious cxtcnsions ot thc incar
rcrcssion modc and othcr modc ttin tcchniqucs arc avaiac
in asc l and scvcra ClA` jackacs `onincar cast squarcs
modcin is avaiac in nls() in jackac stats lccvant jack-
acs incudc quantrc (quantic rcrcssion), sem (incar struc-
tura cquation modcs, incudin two-stac cast squarcs), systemt
(simutancous cquation cstimation), betareg (cta rcrcssion),
nlme (nonincar mixcd-ccct modcs), VR (mutinomia oit
modcs in jackac nnet) and MNP(Laycsian mutinomia jro-
it modcs) Thc jackacs Design and Hmisc jrovidc scv-
cra toos tor cxtcndcd handin ot (cncraizcd) incar rcrcssion
modcs
! Basic time series infrastructure: Thc cass ts in jack-
ac stats is ls standard cass tor rcuary sjaccd timc scrics
which can c cocrccd ack and torth without oss ot intorma-
tion to zooreg trom jackac zoo zoo jrovidcs intrastructurc
tor oth rcuary and irrcuary sjaccd timc scrics (thc attcr
via thc cass zoo ) whcrc thc timc intormation can c ot ar-
itrary cass Scvcra othcr imjcmcntations ot irrcuar timc
scrics uidin on thc lOSlXt timc-datc casscs arc avaiac
in its, tseries and fCalendar which arc a aimcd jarticuary
at nancc ajjications (scc thc Finance vicw)
` Time series modelling: Cassica timc scrics modcin toos
arc containcd in thc stats jackac and incudc arima() tor
All`A modcin and Lox-1cnkins-tyjc anaysis Iurthcrmorc
stats jrovidcs StructTS() tor ttin structura timc scrics and
decompose() and HoltWinters() tor timc scrics tcrin and
dccomjosition Ior cstimatin \Al modcs, scvcra mcthods
arc avaiac simjc modcs can c ttcd y ar() in stats, morc
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 11
caoratc modcs arc jrovidcd y estVARXls() in dse and a
Laycsian ajjroach is avaiac in `SL\Al A convcnicnt intcr-
tacc tor ttin dynamic rcrcssion modcs via OLS is avaiac in
dynlm. a dicrcnt ajjroach that aso works with othcr rcrcs-
sion tunctions is imjcmcntcd in dyn `orc advanccd dynamic
systcm cquations can c ttcd usin dse nit root and coin-
tcration tcchniqucs arc avaiac in urca, uroot and tseries
Timc scrics tactor anaysis is avaiac in tsfa
o Matrix manipulations: As a vcctor- and matrix-ascd an-
uac, asc R shijs with many jowcrtu toos tor doin ma-
trix manijuations, which arc comjcmcntcd y thc jackacs
Matrix and SparseM
Inequality: Ior mcasurin incquaity, conccntration and jovcrty
thc jackac ineq jrovidcs somc asic toos such as Lorcnz curvcs,
lcns jaradc, thc Gini coccicnt and many morc
S Structural change: l is jarticuary stron whcn dcain
with structura chancs and changepoints in jaramctric mod-
cs, scc strucchange and segmented
9 Data sets: `any ot thc jackacs in this vicw contain cocc-
tions ot data scts trom thc cconomctric itcraturc and thc jackac
Ecdat contains a comjctc cocction ot data scts trom various
standard cconomctric tcxtooks micEcdat jrovidcs scvcra
data scts trom thc 1ourna ot Ajjicd Lconomctrics and thc
1ourna ot Lusincss : Lconomic Statistics data archivcs lack-
ac CDNmoney jrovidcs Canadian monctary arcatcs and
pwt jrovidcs thc lcnn word tac
CRAN packages:
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 12
aycsm
ctarc
ivjois
car (corc)
CL`moncy
Lcsin
dsc
dyn
dynm
Lcdat
tCacndar
Lmisc
incq
its
mtcst (corc)
`atrix
micLcdat
micLcon
``l
`SL\Al
nmc
jsc
jwt
quantrc
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 13
rcdist
sandwich (corc)
scmcntcd
scm
Sjarsc`
strucchanc
systcmt
tscrics (corc)
tsta
urca (corc)
uroot
\l
zicounts
zoo (corc)
Related links:
ClA` Task \icw Iinancc Thc wc sitc is
http://cran.cnr.berkeley.edu/src/contrib/
Views/Finance.html
or scc thc aovc scction
`aiin ist R Sjccia lntcrcst Grouj Iinancc
A Lrict Guidc to R tor Lcinncrs in Lconomctrics Thc
wc sitc is
http://people.su.se/ ma/R

intro/
CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS 14
l tor Lconomists Thc wc sitc is
http://www.mayin.org/ajayshah/KB/R/
R

for

economists.html
Chapter 2
Characteristics of Time Series
2.1 Introduction
Thc vcry naturc ot data cocctcd in dicrcnt cds as as divcrsc as
cconomics, nancc, iooy, mcdicinc, and cninccrin cads onc nat-
uray to a considcration ot timc scrics modcs Samjcs takcn trom
a ot thcsc discijincs arc tyjicay oscrvcd ovcr a scqucncc ot timc
jcriods Ottcn, tor cxamjc, onc oscrvcs houry or daiy or monthy
or ycary data, cvcn tick-y-tick tradc data, and it is ccar trom cxam-
inin thc historics ot such scrics ovcr a numcr ot timc jcriods that
thc ad,accnt oscrvations arc y no mcans indcjcndcnt Lcncc, thc
usua tcchniqucs trom cassica statistics, dcvcojcd jrimariy tor in-
dependent identically distributed (iid) oscrvations, arc not
ajjicac
Ccary, wc can not hojc to ivc a comjctc accountin ot thc thc-
ory and ajjications ot timc scrics in thc imitcd timc to c dcvotcd
to this coursc Thcrctorc, what wc wi try to accomjish, in this
jrcscntation is a considcray morc modcst sct ot o,cctivcs, with
morc dctaicd rctcrcnccs quotcd tor discussions in dcjth Iirst, wc
wi attcmjt to iustratc thc kinds ot timc scrics anayscs that can
arisc in scicntic contcxts, jarticuary, in cconomics and nancc,
and ivc cxamjcs ot ajjications usin rca data This ncccssariy
15
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 16
wi incudc cxjoratory data anaysis usin rajhica disjays and
numcrica summarics such as thc autocorrcation and cross corrca-
tion tunctions Thc usc ot scattcr diarams and various incar and
nonincar transtormations aso wi c iustratcd \c wi dcnc
cassica timc scrics statistics tor mcasurin thc jattcrns dcscricd
y timc scrics data Ior cxamjc, thc charactcrization ot consistcnt
trcnd jrocs y dynamic incar or quadratic rcrcssion modcs as
wc as thc rcjrcscntation ot jcriodic jattcrns usin sjcctra anaysis
wi c iustratcd \c wi show how onc miht o aout cxaminin
jausic jattcrns ot causc and ccct, oth within and amon timc
scrics Iinay, somc timc scrics modcs that arc jarticuary usctu
such as rcrcssion with corrcatcd crrors as wc as mutivariatc au-
torcrcssivc and statc-sjacc modcs wi c dcvcojcd, tocthcr with
unit root, co-intcration, and nonincar timc scrics modcs, and somc
othcr modcs Iorms ot thcsc modcs that ajjcar to ocr hojc tor
ajjications wi c cmjhasizcd lt is rcconizcd that a discussion ot
thc modcs and tcchniqucs invovcd is not cnouh it onc docs not havc
avaiac thc rcquisitc rcsourccs tor carryin out timc scrics comju-
tations. thcsc can c tormidac Lcncc, wc incudc a comjutin
jackac, cacd R
ln this chajtcr, wc wi try to minimizc thc usc ot mathcmatica
notation throuhout thc discussions and wi not sjcnd timc dcvc-
ojin thc thcorctica jrojcrtics ot any ot thc modcs or jroccdurcs
\hat is imjortant tor this jrcscntation is that you, thc rcadcr, can
ain a modcst undcrstandin as wc as havin acccss to somc ot thc
jrincija tcchniqucs ot timc scrics anaysis Ot coursc, wc wi rctcr
to Lamiton (199!) tor additiona rctcrcnccs or morc comjctc dis-
cussions rcatin to an ajjication or jrincijc and wi discuss thcm
in dctai
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 17
2.2 Stationary Time Series
\c cin y introducin scvcra cnvironmcnta and cconomic as wc
as nancia timc scrics to scrvc as iustrativc data tor timc scrics
mcthodooy Iiurc 21 shows monthy vaucs ot an cnvironmcnta
scrics cacd thc Southcrn Osciation lndcx (SOl) and associatcd
rccruitmcnt (numcr ot ncw sh) comjutcd trom a modc y licrrc
Icicr, Southwcst Iishcrics Ccntcr, La 1oa, Caitornia Loth scrics
arc tor a jcriod ot !`3 months ranin ovcr thc ycars 19`0 19S
Thc SOl mcasurcs chancs in air jrcssurc that arc rcatcd to sca
surtacc tcmjcraturcs in thc ccntra lacic Thc ccntra lacic Occan
warms uj cvcry thrcc to scvcn ycars duc to thc L `i no ccct which
has ccn amcd, in jarticuar, tor toods in thc midwcstcrn jortions
ot thc S
Loth scrics in Iiurc 21 tcnd to cxhiit rcjctitivc chavior, with
rcuary rcjcatin (stochastic) cycles that arc casiy visic This
periodic behavior is ot intcrcst ccausc undcryin jroccsscs ot
intcrcst may c rcuar and thc ratc or trcqucncy ot osciation char-
actcrizin thc chavior ot thc undcryin scrics woud hcj to idcntity
thcm Onc can aso rcmark that thc cyccs ot thc SOl arc rcjcatin
at a tastcr ratc than thosc ot thc rccruitmcnt scrics Thc rccruit scrics
aso shows scvcra kinds ot osciations, a tastcr trcqucncy that sccms
to rcjcat aout cvcry 12 months and a sowcr trcqucncy that sccms
to rcjcat aout cvcry `0 months Thc study ot thc kinds ot cyccs
and thcir strcnths wi c discusscd atcr Thc two scrics aso tcnd
to c somcwhat rcatcd. it is casy to imainc that somchow thc sh
jojuation is dcjcndcnt on thc SOl lcrhajs thcrc is cvcn a acd
rcation, with thc SOl sinain chancs in thc sh jojuation
Thc study ot thc variation in thc dicrcnt kinds ot cycica chav-
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 18
ior in a timc scrics can c aidcd y comjutin thc jowcr sjcctrum
which shows thc variancc as a tunction ot thc trcqucncy ot oscia-
tion Comjarin thc jowcr sjcctra ot thc two scrics woud thcn ivc
vauac intormation rcatin to thc rcativc cyccs drivin cach onc
Onc miht aso want to know whcthcr or not thc cycica variations
ot a jarticuar trcqucncy in onc ot thc scrics, say thc SOl, arc asso-
ciatcd with thc trcqucncics in thc rccruitmcnt scrics This woud c
mcasurcd y comjutin thc corrcation as a tunction ot trcqucncy,
cacd thc coherence Thc study ot systcmatic jcriodic variations
0 100 200 300 400

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
Southern Oscillation Index
0 100 200 300 400
0
2
0
4
0
6
0
8
0
1
0
0
Recruit
Figure 2.1: Monthly SOI (left) and simulated recruitment (right) from a model (n=453
months, 1950-1987).
in timc scrics is cacd spectral analysis Scc Shumway (19SS)
and Shumway and Stocr (2001) tor dctais
\c wi nccd a charactcrization tor thc kind ot staiity that is
cxhiitcd y thc cnvironmcnta and sh scrics Onc can notc that
thc two scrics sccm to osciatc tairy rcuary around ccntra vaucs
(0 tor SOl and o! tor rccruitmcnt) Aso, thc cnths ot thc cyccs and
thcir oricntations rcativc to cach othcr do not sccm to c chanin
drasticay ovcr thc timc historics
ln ordcr to dcscric this in a simjc mathcmatica way, it is con-
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 19
vcnicnt to introducc thc conccjt ot a stationary time series
Sujjosc that wc ct thc vauc ot thc timc scrics at somc timc joint
t c dcnotcd y {x
t
} Thcn, thc oscrvcd vaucs can c rcjrcscntcd
as x
1
, thc initia timc joint, x
2
, thc sccond timc joint and so torth
out to x
n
, thc ast oscrvcd joint A stationary timc scrics is onc
tor which thc statistica chavior ot x
t
1
, x
t
2
, . . . , x
t
k
is idcntica to
that ot thc shittcd sct x
t
1
+h
, x
t
2
+h
, . . . , x
t
k
+h
tor any cocction ot
timc joints t
1
, t
2
, . . ., t
k
and tor any shitt h This mcans that a ot
thc mutivariatc jroaiity dcnsity tunctions tor suscts ot variacs
must arcc with thcir countcrjarts in thc shittcd sct tor a vaucs
ot thc shitt jaramctcr h This is cacd strictly strong station-
ary, which can c rcardcd as a mathematical assumption
Thc aovc vcrsion ot stationarity is too stron tor most ajjications
and is dicut or imjossic to c vcricd statisticay in ajjica-
tions Thcrctorc, to rcax this mathcmatica assumjtion, wc wi usc
a wcakcr vcrsion, cacd weak stationarity or covariance sta-
tionarity, which rcquircs ony that rst and sccond momcnts satisty
thc constraints This imjics that
E(x
t
) and E|(x
t+h
)(x
t
)|
x
(h), (21)
whcrc E dcnotcs cxjcctation or avcrain ovcr thc jojuation dcnsi-
tics and h is thc shitt or a This imjics, rst, that thc mcan vauc
tunction docs not chanc ovcr timc and that
x
(h), thc jojuation
covariancc tunction, is thc samc as on as thc joints arc scjaratcd
y a constant shitt h Lstimators tor thc jojuation covariancc arc
imjortant dianostic toos tor timc corrcation as wc sha scc atcr
\hcn wc usc thc tcrm stationary timc scrics in thc scquc, wc mcan
wcaky stationary as dcncd y (21) Thc autocorrcation tunction
(ACI) is dcncd as a scacd vcrsion ot (21) and is writtcn as

x
(h)
x
(h)/
x
(0), (22)
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 20
which is aways ctwccn 1 and 1 Thc dcnominator ot (22) is thc
mcan squarc crror or variancc ot thc scrics sincc
x
(0) E|(x
t
)
2
|
Exercise: Ior ivcn timc scrics {x
t
}
n
t=1
, how do you chcck whcthcr
thc timc scrics {x
t
} is wcaky or stron stationary Thank aout
this jrocm
Example 1.1: \c introducc in this cxamjc a simjc cxamjc
ot a timc domain modc to c considcrcd in dctai atcr A simjc
movin avcrac modc assumcs that thc scrics x
t
is cncratcd trom
incar cominations ot indcjcndcnt or uncorrcatcd shocks w
t
,
somctimcs cacd white noise
1
(\`), to thc systcm Ior cxamjc,
thc simjc rst ordcr movin avcrac scrics
x
t
w
t
0.9 w
t1
is stationary whcn thc injuts {w
t
} arc assumcd indcjcndcnt with
E(w
t
) 0 and E(w
2
t
) 1 lt can c casiy vcricd that E(x
t
) 0
and
x
(h) 1 + 0.9
2
it h 0, 0.9 it h 1, 0 it h > 1 (jcasc
vcrity this) \c can scc what such a scrics miht ook ikc y drawin
random numcrs w
t
trom a standard norma distriution and thcn
comjutin thc vaucs ot x
t
Onc such simuatcd scrics is shown in
Iiurc 22 tor n 200 vaucs. thc scrics rcscmcs vaucy thc rca
data in thc ottom janc ot Iiurc 21
`any ot our tcchniqucs arc ascd on thc idca that a suitay
modicd timc scrics can c rcardcd as stationary (wcaky) This
rcquircs rst that thc mcan vauc tunction c constant as in (21)
Scvcra simjc commony occurrin nonstationary timc scrics can c
iustratcd y cttin this assumjtion c vioatcd Ior cxamjc, thc
1
white noise is dened as a sequence of uncorrelated ransom variables with mean zero and same variance.
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 21
0 50 100 150 200

1
0
1
2
3
Simulated MA(1)
Figure 2.2: Simulated MA(1) with
1
= 0.9.
scrics y
t
t +x
t
, whcrc x
t
is thc movin avcrac scrics ot Example
1.1, wi c nonstationary ccausc E(x
t
) t and thc constant mcan
assumjtion ot (21) is ccary vioatcd
Iour tcchniqucs tor modityin thc ivcn scrics to imjrovc thc aj-
jroximation to stationarity arc detrending, dierencing, trans-
formations, and linear ltering as discusscd cow A simjc
cxamjc ot a nonstationary scrics is aso ivcn atcr
2.2.1 Detrending
Onc ot thc dominant tcaturcs on many cconomic and usincss timc
scrics is thc trend Such a trcnd can c ujward or downward, it
can c stccj or not, and it can c cxjoncntia or ajjroximatcy
incar Sincc a trcnd shoud c dcnitcy somchow c incorjoratcd
in a timc scrics modc, simjy ccausc it can c cxjoitcd tor out-ot-
samjc torccastin, an anaysis ot trcnd chavior tyjicay rcquircs
quitc somc rcscarch injut Thc discussion atcr wi show that thc
tyjc ot trcnd has an imjortant imjact on torccastin
Thc cncra vcrsion ot thc nonstationary timc scrics ivcn aovc is
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 22
to assumc a cncra trcnd ot thc torm y
t
T
t
+ x
t
, jarticuary, thc
incar trcnd T
t

1
+
2
t lt onc ooks tor a mcthod ot modityin
thc aovc scrics to achicvc stationarity, it is natura to considcr thc
rcsidua

x
t
y
t

T
t
y
t

2
t
as a jausic stationary scrics whcrc

1
and

2
arc thc cstimatcd
intcrccjt and sojc ot thc cast squarcs inc tor y
t
as a tunction ot
t Thc usc ot thc rcsidua or dctrcndcd scrics is common and thc
jroccss ot constructin rcsidua is known as detrending
Example 1.2: To iustratc thc jrcscncc ot trcnds in cconomic data,
considcr thc vc rajhs in Iiurc 23, which arc thc annua indiccs ot
1955 1960 1965 1970 1975 1980 1985
5
6
7
8
agriculture
commerce
consumption
industry
transport
Figure 2.3: Log of annual indices of real national output in China, 1952-1988.
rca nationa outjut (in os) in China in vc dicrcnt scctors tor thc
samjc jcriod 19`2 19SS Thcsc scctors arc aricuturc, industry,
construction, transjortation, and commcrcc
Irom this urc, it can c oscrvcd that thc vc scctors havc rown
ovcr thc ycar at dicrcnt ratcs, and aso that thc vc scctors sccm
to havc ccn acctcd y thc, ikcy, cxocnous shocks to thc Chincsc
cconomy around 19`S and 19oS Thcsc shocks rouhy corrcsjond
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 23
to thc two ma,or joitica movcmcnts in China thc Grcat-Lcaj-
Iorward around 19`S unti 19o2 and thc Cutura lcvoution trom
19oo to 19o lt aso ajjcars trom thc rajhs that thcsc joitica
movcmcnts may not havc acctcd cach ot thc vc scctors in a simiar
tashion Ior cxamjc, thc dccinc ot thc outjut in thc construction
scctor in 19o1 sccms much arcr than that in thc industry scctor
in thc samc ycar lt aso sccms that thc Grcat-Lcaj-Iorward shock
arcady had an imjact on thc outjut in thc aricuturc scctor as
cary as 19`9 To quantity thc trcnds in thc vc Chincsc outjut
scrics, onc miht considcr a simjc rcrcssion modc with a incar
trcnd as mcntioncd caricr or somc morc comjcx modcs
Example 1.3: As anothcr morc intcrcstin cxamjc, considcr thc
oa tcmjcraturc scrics ivcn in Iiurc 2! Thcrc ajjcars to c
an incrcasin trcnd in oa tcmjcraturc which may sina oa
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
ooo
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
oo
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
ooo
o
o
o
oo
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
oo
o
oo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
oo
o
o
o
o
o
oo
o
o
o
o
oo
o
o
o
o
o
o
o
o
ooo
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
oo
o
o
o
o
o
oo
o
o
o
o
o
oooo
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
oo
o
ooo
oo
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
oo
oo
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
ooo
o
o
o
o
o
o
oo
o
oo
o
o
o
o
o
o
o
oo
oo
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
oo
o
o
o
o
o
o
o
o
o
o
o
o
oooo
o
o
oo
o
oo
o
oo
o
o
o
o
o
oo
o
o
oo
o
o
o
o
o
o
o
oo
o
o
oo
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
ooo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
ooo
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
ooo
oo
o
o
o
o
oo
ooo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
ooo
oo
oo
o
o
o
o
o
o
o
o
oo
o
oo
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
oo
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
oo
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
oo
o
o
o
o
oo
oo
o
o
o
o
ooo
oo
o
o
oo
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
oo
o
oo
o
o
o
o
ooo
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
ooo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
ooo
o
o
o
o
o
o
ooo
o
o
o
o
o
o
o
o
o
o
o
oo
oo
o
ooo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
ooo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
oo
oo
o
o
oo
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
oo
o
ooo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
ooo
o
oo
o
o
o
o
o
o
o
o
oo
o
o
o
o
oo
o
o
ooo
o
o
o
o
oo
o
o
o
o
o
o
oo
o
oo
o
o
o
oo
o
oo
o
o
o
oo
o
o
o
o
o
ooo
o
o
o
oo
o
oo
o
o
o
o
o
o
o
o
o
oo
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
oo
o
o
o
o
o
o
o
oooo
o
o
o
o
o
o
oo
o
oo
o
o
o
oo
o
o
o
ooo
o
o
o
o
o
o
o
oo
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
oo
oooo
oo
oo
o
o
o
o
o
oo
o
o
o
o
oo
oo
o
o
o
oo
o
o
oo
o
o
o
oo
oo
o
ooo
o
o
oo
o
o
o
o
oo
o
o
oo
o
o
o
o
o
o
o
o
oo
o
oo
o
o
o
ooo
oo
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
oo
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
oo
o
o
oo
o
o
o
o
oo
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
oo
ooo
oo
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
1900 1950 2000

1
.
0

0
.
5
0
.
0
0
.
5
Original Data with Linear and Nonlinear Trend
Figure 2.4: Monthly average temperature in degrees centigrade, January, 1856 - February
2005, n = 1790 months. The straight line (wide and green) is the linear trend y = 9.037 +
0.0046 t and the curve (wide and red) is the nonparametric estimated trend.
warmin or it may c ,ust a norma uctuation Iittin a straiht inc
rcatin timc t to tcmjcraturc in dcrccs Ccntiradc y simjc cast
squarcs cads to

1
9.03,

2
0.00!o and a dctrcndcd scrics
shown in thc ctt janc ot Iiurc 2` `otc that thc dctrcndcd scrics
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 24
1900 1950 2000

0
.
5
0
.
0
0
.
5
Detrended: Linear
1900 1950 2000

0
.
6

0
.
4

0
.
2
0
.
0
0
.
2
0
.
4
0
.
6
Detrended: Nonlinear
Figure 2.5: Detrended monthly global temperatures: left panel (linear) and right panel
(nonlinear).
sti contains a trcnd ikc uc that is hihcst at aout t o0 ycars
ln this casc thc sojc ot thc inc is ottcn uscd to aruc that thcrc is a
oa warmin trcnd and that thc avcrac incrcasc is ajjroximatcy
0.S3 dcrccs I jcr 100 ycars lt is ccar that thc rcsiduas in Iiurc
2` sti contain sustantia corrcation and thc ordinary cast squarcs
modc may not c ajjrojriatc
Thcrc may aso c othcr tunctiona torms that do a cttcr ,o
ot dctrcndin. tor cxamjc, quadratic or oarithmic rcjrcscntations
arc common or nonparametric approach can c uscd (\c wi
discuss this ajjroach in dctai atcr). scc thc dctrcndcd scrics shown
in thc riht janc ot Iiurc 2` Lctrcndin is jarticuary csscntia
whcn onc is cstimatin thc covariancc tunction and jowcr sjcctrum
2.2.2 Dierencing
A common mcthod tor achicvin stationarity in nonstationary cascs
is with thc rst dicrcncc
y
t
y
t
y
t1
,
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 25
whcrc is cacd thc dierencing operator Thc usc ot dicrcnc-
in as a mcthod tor transtormin to stationarity is common aso tor
scrics with trcnd Ior cxamjc, in thc trcnd in Example 1.3, thc
dicrcnccd scrics woud c y
t
b + x
t
x
t1
, which is stationary
ccausc thc dicrcncc x
t
x
t1
can c shown to c stationary
Example 1.4: Thc rst dicrcncc ot thc oa tcmjcraturc sc-
rics is shown in Iiurc 2o and wc scc that thc ujward incar trcnd
has disajjcarcd as has thc trcnd ikc uc that rcmaincd in thc
1900 1950 2000

0
.
5
0
.
0
0
.
5
1
.
0
Differenced Time Series
Figure 2.6: Dierenced monthly global temperatures.
dctrcndcd scrics Lihcr ordcr dicrcnccs arc dcncd as succcssivc
ajjications ot thc ojcrator Ior cxamjc, thc sccond dicrcncc
is
2
y
t
y
t
so that
2
y
t
y
t
2 y
t1
+ y
t2
lt thc modc aso
contains a quadratic trcnd tcrm c t
2
, it is casy to show that takin
thc sccond dicrcncc rcduccs thc modc to a stationary torm
Thc trcnds in Iiurcs 23 and 2! arc a ot thc tamiiar tyjc, that
is, many cconomic timc scrics disjay an ujward movin trcnd lt
is howcvcr not ncccssary tor a trcnd to movc ujwards to c cacd a
trcnd lt is aso that a trcnd is css smooth and may disjay sowy
chanin tcndcncics which oncc in a whic chanc dircctions
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 26
Example 1.5: An cxamjc ot such a trcndin jattcrn is ivcn in thc
toj ctt janc ot Iiurc 2 and thc rst dicrcncc in thc toj riht
50 60 70 80 90
1
0
0
1
5
0
2
0
0
2
5
0
Data
0 10 20 30 40
0
2
0
4
0
6
0
First difference
0 10 20 30 40

2
0

1
0
0
1
0
2
0
3
0
Second order difference
Figure 2.7: Annual stock of motor cycles in the Netherlands, 1946-1993.
janc ot Iiurc 2 and thc sccond ordcr dicrcncc in thc ottom
ctt janc ot Iiurc 2, whcrc thc annua stock ot motor cyccs in
Thc `cthcrand is disjaycd, tor 19!o 1993, with thc rst ordcr
and sccond ordcr dicrcncin timc scrics Irom thc urcs in thc
riht toj and ottom jancs, wc can scc that thc dicrcncin miht
not work wc tor this cxamjc Onc way to dcscric this chanin
trcnds is to aow thc jaramctcrs to chanc ovcr timc, drivcn y somc
cxocnous shocks (macrocconomic variacs)
2
, tor cxamjc, oi shock
in 19!
2.2.3 Transformations
A transtormation that cuts down thc vaucs ot arcr jcaks ot a timc
scrics and cmjhasizcs thc owcr vaucs may c ccctivc in rcducin
2
see the paper by Cai (2006)
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 27
nonstationary chavior duc to chanin variancc An cxamjc is
thc oarithmic transtormation y
t
o(x
t
), whcrc o dcnotcs thc
cxjoncntia-asc oarithm
Example 1.6: Ior cxamjc, thc data shown in Iiurc 2S rcjrcscnt
quartcry carnins jcr sharc tor thc Amcrican Comjany 1ohnson :
0 20 40 60 80
0
5
1
0
1
5
J&J Earnings
0 20 40 60 80
0
1
2
transformed log(earnings)
Figure 2.8: Quarterly earnings for Johnson & Johnson (4th quarter, 1970 to 1st quarter,
1980, left panel) with log transformed earnings (right panel).
1ohnson trom thc trom thc tourth quartcr ot 190 to thc rst quar-
tcr ot 19S0 lt is casy to notc somc vcry nonstationary chavior in
this scrics that cannot c ciminatcd comjctcy y dicrcncin or
dctrcndin ccausc ot thc arcr uctuations that occur ncar thc cnd
ot thc rccord whcn thc carnins arc hihcr Thc riht janc ot Ii-
urc 2S shows thc o-transtormcd scrics and wc notc that thc attcr
jcaks havc ccn attcnuatcd so that thc variancc ot thc transtormcd
scrics sccms morc stac Onc woud havc to ciminatc thc trcnd sti
rcmainin in thc aovc scrics to otain stationarity Ior morc dctais
on thc currcnt anayscs ot this scrics, scc thc atcr anayscs and thc
jajcrs y Lurman and Shumway (199S) and Cai and Chcn (200o)
A cncra transtormation is thc wc-known Box-Cox transtor-
mation. scc Lamiton (199!, j12o), Shumway (19SS), and Shumway
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 28
and Stocr (2000), dcncd in tcrms ot aritrary jowcr x

t
tor somc
in a ccrtain ranc, which can c choscn ascd on somc ojtima
critcrion such as thc smacst mcan squarcd crror
2.2.4 Linear Filters
Thc rst dicrcncc is a incar comination ot thc vaucs ot thc scrics
at two as, say 0 and 1 and has thc ccct ot rctainin thc tastcr
osciations and attcnuatin or rcducin thc sowcr osciations \c
may dcnc morc cncra incar tcrs to do othcr kinds ot smoothin
or rouhcnin ot a timc scrics to cnhancc sinas and attcnuatc noisc
Considcr thc cncra incar comination ot jast and tuturc vaucs ot
a timc scrics ivcn as
y
t

j=
a
j
x
tj
whcrc a
j
, j 0, 1, 2, . . ., dcnc a sct ot xcd tcr coccicnts to
c ajjicd to thc scrics ot intcrcst An cxamjc is thc rst dicrcncc
whcrc a
0
1, a
1
1, a
j
0 othcrwisc `otc that thc aovc {y
t
}
is aso cacd a incar jroccss in jroaiity itcraturc
Example 1.7: To ivc a simjc iustration, considcr thc twcvc
month movin avcrac a
j
1/12, j 0, 1, 2, 3, !, `,
o and zcro othcrwisc Thc rcsut ot ajjyin this tcr to thc SOl
indcx is shown in Iiurc 29 lt is ccar that this tcr rcmovcs somc
hihcr osciations and jroduccs a smoothcr scrics ln tact, thc ycary
osciations havc ccn tcrcd out (scc thc ottom janc in Iiurc 29)
and a owcr trcqucncy osciation ajjcars with a cycin ratc ot aout
!2 months This is thc so-cacd L `i no ccct that accounts tor a
kinds ot jhcnomcna This tcrin ccct wi c cxamincd turthcr
atcr on sjcctra anaysis sincc it is cxtrcmcy imjortant to know
cxacty how onc is inucncin thc jcriodic osciations y tcrin
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 29
0 100 200 300 400

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
0 100 200 300 400

0
.
5
0
.
0
0
.
5
Figure 2.9: The SOI series (black solid line) compared with a 12 point moving average (red
thicker solid line). The top panel: original data and the bottom panel: ltered series.
To summarizc, thc rajhica cxamination ot timc historics can
joint thc way to turthcr anayscs y notin periodicities and
trends that may c jrcscnt Iurthcrmorc, ookin at timc his-
torics ot transtormcd or tcrcd scrics ottcn ivcs an intuitivc idca as
to whcthcr onc scrics coud aso c associatcd with anothcr Iiurc
21 indicatcs that thc SOl scrics tcnds to jrcccdc or cad thc rccruit
scrics `aturay, onc can ask tor a morc dctaicd sjccication ot
thc cadin ain rcation ln thc toowin scctions, wc wi try to
show how cassica timc scrics mcthods can c uscd to jrovidc jar-
tia answcrs to thcsc kinds ot qucstions Lctorc doin so, wc sjcnd
somc sjacc to introducc somc othcr tcaturcs, such as seasonality,
outliers, nonlinearity, and conditional heteroscedasticity
common sccn in thc cconomic and nancia as wc as cnvironmcnta
data
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 30
2.3 Other Key Features of Time Series
2.3.1 Seasonality
\hcn timc scrics (jarticuary, cconomic and nancia timc scrics)
arc oscrvcd cach day or month or quartcr, it is ottcn thc casc that
such as a scrics disjays a scasona jattcrn (dctcrministic cycica c-
havior) Simiar to thc tcaturc ot trcnd, thcrc is no jrccisc dcnition
ot seasonality suay wc rctcr to scasonaity whcn oscrvations
in ccrtain scasons disjay strikiny dicrcnt tcaturcs to othcr sca-
sons Ior cxamjc, whcn thc rctai sacs arc aways arc in thc tourth
quartcr (ccausc ot thc Christmas sjcndin) and sma in thc rst
quartcr as can c oscrvcd trom Iiurc 210 lt may aso c jossic
that scasonaity is rccctcd in thc variancc ot a timc scrics Ior cx-
amjc, tor daiy oscrvcd stock markct rcturns thc voatiity sccms
ottcn hihcst on `ondays, asicay ccausc invcstors havc to dicst
thrcc days ot ncws instcad ot ony day Ior modc dctais, scc thc
ook y Tayor (200`, !`) and Tsay (200`)
Example 1.8: ln this cxamjc wc considcr thc monthy S rctai
sacs scrics (not scasonay ad,ustcd) trom 1anuary ot 19o to Lcccm-
cr ot 2000 (in iions ot S doars) Thc data can c downoadcd
trom thc wc sitc at http://marketvector.com Thc S rctai sacs
indcx is onc ot thc most imjortant indicators ot thc S cconomy
Thcrc arc vast studics ot thc scasona scrics (ikc this scrics) in thc
itcraturc. scc, c, Iranscs (199o, 199S) and Ghyscs and Osorn
(2001) and Cai and Chcn (200o) Irom Iiurc 210, wc can oscrvc
that thc jcaks occur in Lcccmcr and wc can say that rctai sacs dis-
jay scasonaity Aso, it can c oscrvcd that thc trcnd is asicay
incrcasin ut nonincary Thc samc jhcnomcnon can c oscrvcd
trom Iiurc 2S tor thc quartcry carnins tor 1ohnson : 1ohnson
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 31
0 100 200 300 400
5
0
0
0
0
1
0
0
0
0
0
2
0
0
0
0
0
3
0
0
0
0
0
Figure 2.10: US Retail Sales Data from 1967-2000.
lt simjc rajhs arc not intormativc cnouh to hihiht jossic
scasona variation, a torma rcrcssion modc can c uscd, tor cxam-
jc, onc miht try to considcr thc toowin rcrcssion modc with
scasona dummy variacs
y
t
y
t
y
t1

s

j=1

j
D
j,t
+
t
,
whcrc D
j,t
is a scasona dummy variac and s is thc numcr ot sca-
sons Ot coursc, onc can usc a seasonal ARIMA modc, dcnotcd
y All`A(p, d, q) (Q, D, Q)
s
, which wi c discusscd atcr
Example 1.9: ln this cxamjc, wc considcr a timc scrics with
jronounccd scasonaity disjaycd in Iiurc 211, whcrc os ot tour-
wccky advcrtisin cxjcnditurcs on ratio and tccvision in Thc `cthcr-
ands tor 19S.01 199!.13 Ior thcsc two markctin timc scrics onc
can oscrvc ccary that thc tccvision advcrtisin disjays quitc somc
scasona uctuation throuhout thc cntirc samjc and thc radio ad-
vcrtisin has scasonaity ony tor thc ast vc ycars Aso, thcrc sccms
to c a structura rcak in thc radio scrics around oscrvation `3
This rcak is rcatcd to an incrcasc in radio roadcastin minutcs in
1anuary 19S2 Iurthcrmorc, thcrc is a visua cvidcncc that thc trcnd
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 32
0 50 100 150 200
8
9
1
0
1
1
television
radio
Figure 2.11: Four-weekly advertising expenditures on radio and television in The Nether-
lands, 1978.01 1994.13.
chancs ovcr timc
Gcncray, it ajjcars that many timc scrics scasonay oscrvcd
trom usincss and cconomics as wc as othcr ajjicd cds disjay
scasonaity in thc scnsc that thc oscrvations in ccrtain scasons havc
jrojcrtics that dicr trom thosc data joints in othcr scasons A
sccond tcaturc ot many scasona timc scrics is that thc scasonaity
chancs ovcr timc, ikc what studicd y Cai and Chcn (200o) Somc-
timcs, thcsc chancs ajjcar arujt, as is thc casc tor advcrtisin
on thc radio in Iiurc 211, and somctimcs such chancs occur ony
sowy To cajturc thcsc jhcnomcna, Cai and Chcn (200o) jrojoscd
a morc cncra cxic scasona ccct modc havin thc toowin
torm
y
ij
(t
i
) +
j
(t
i
) + e
ij
, i 1, . . . , n, j 1, . . . , s,
whcrc y
ij
y
(i1)s+j
, t
i
i/n, () is a (smooth) common trcnd
tunction in |0, 1|, {
j
()} arc (smooth) scasona ccct tunctions in
|0, 1|, cithcr xcd or random, su,cct to a sct ot constraints, and thc
crror tcrm e
ij
is assumcd to c stationary Ior morc dctais, scc Cai
and Chcn (200o)
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 33
2.3.2 Aberrant Observations
lossiy distortin oscrvations do not ncccssariy comc in a sc-
qucncc as in thc radio advcrtisin cxamjc (which miht havc so-
cacd rcimc shitts) lt may aso c that ony tcw oscrvations havc
a ma,or imjact on timc scrics modcin and torccastin Such data
joints arc cacd aberrant observations (outliers in statistics)
Example 1.10: As an iustration cxamjc, wc considcr thc dicr-
cnccd y
t
, that is y
t
y
t
y
t1
, whcrc y
t
o(w
t
), with w
t
thc
jricc cvc, and thc ination ratc f
t
(w
t
w
t1
)/w
t1
in Arcntina,
tor thc samjc 190.1 19S9.! in Iiurc 212 Irom thc urc, it
70 75 80 85 90
0
1
2
3
4
5
difference
inflation
Figure 2.12: First dierence in log prices versus the ination rate: the case of Argentina,
1970.1 1989.4.
is ovious that in thc casc whcrc thc quartcry ination ratc is hih
(as is thc casc in 19S93 whcrc it is aout `00 jcrccnt), y
t
scrics
is not a ood ajjroximation to thc ination ratc (sincc thc 19S93
oscrvation woud not now corrcsjond to aout 200 jcrccnt) Aso,
wc can oscrvc that thc data in 19S9 sccm to c quitc dicrcnt trom
thosc oscrvations thc ycar ctorc ln tact, it thcrc is any corrcation
ctwccn y
t
and y
t1
, such a corrcation may c acctcd y thcsc
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 34
oscrvations ln othcr words, it a simjc rcrcssion is uscd to modc
thc corrcation ctwccn y
t
and y
t1
, wc woud cxjcct that an cs-
timatc ot thc coccicnt is inucnccd y thc data joints in thc ast
ycar Thc ordinary cast squarc cstimatcs ot arc

0.`o1(0.09!)
tor thc cntirc samjc and

0.0!(0.0S2) tor thc samjc without
thc data joints in thc ast ycar, whcrc thc cstimatcd standard crror
is in jarcnthcscs
lt now turns to thc qucstion on how to handc with thcsc acrrant
oscrvations Scc Chajtcr o ot Iranscs (199S) tor dctais on discus-
sion ot mcthods to dcctc scvcra tyjcs ot acrrant oscrvations, and
aso mcthods to takc account ot such data tor torccastin
2.3.3 Conditional Heteroskedasticity
An imjortant tcaturc ot cconomic timc scrics, and in jarticuar, ot
nancia timc scrics is that acrrant oscrvations tcnd to cmcrc
in clusters (persistence) Thc intuitivc intcrjrctation is that it
a ccrtain day ncws arrivcs on a stock markct, thc rcaction to this
ncws is to uy or sc many stocks, whic thc day attcr thc ncws has
dicstcd and vaucd jrojcry, thc stock markct rcturns ack to thc
cvc ot ctorc thc arriva ot thc ncws This jattcrn woud c rccctcd
y (a jossiy arc) incrcasin or dccrcasin (usuay, thc ncativc
imjact is arcr than thc jositivc imjact, cacd asymmetric) in thc
rcturns on onc day toowcd y an ojjositc chanc on thc ncxt day
As a rcsut, wc can rcard thcm as acrrant oscrvations in a row
and two suddcn chancs in rcturns arc corrcatcd sincc thc sccond
sharj chanc is causcd y thc rst This is cacd conditional
heteroskedasticity To charactcrizc this jhcnomcnon, onc miht
usc thc so cacd thc autorcrcssivc conditiona hctcrosccdasticity
(AlCL) modc ot Lnc (19S2) and thc cncraizcd autorcrcssivc
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 35
conditiona hctcrosccdasticity (GAlCL) modc ot Locrscv (19So)
or othcr GAlCL tyjc modcs. scc Tayor (200`) and Tsay (200`)
Example 1.11: This cxamjc conccrns thc cosin id jriccs ot thc
1ajancsc Ycn (1lY) in tcrms ot S doar Thcrc is a vast amount
ot itcraturc dcvotcd to thc study ot thc cxchanc ratc timc scrics. scc
Scrcu and jja (2000) and thc rctcrcnccs thcrcin tor dctais Lcrc
wc cxjorc thc jossic nonincarity tcaturc (scc thc ncxt scction),
hctcrosccdasticity, and jrcdictaiity ot thc cxchanc ratc scrics (\c
wi discuss this atcr) Thc data is a wccky scrics trom 1anuary 1,
19! to Lcccmcr 31, 2003 Thc daiy noon uyin ratcs in `cw York
City ccrticd y thc Icdcra lcscrvc Lank ot `cw York tor customs
and cac transtcrs jurjoscs wcrc otaincd trom thc Chicao Icdcra
lcscrvc Loard (www.frbchi.org) Thc wccky scrics is cncratcd y
sccctin thc \cdncsdays scrics (it a \cdncsday is a hoiday thcn thc
toowin Thursday is uscd), which has 1`oo oscrvations Thc usc ot
wccky data avoids thc so-cacd wcckcnd ccct as wc as othcr iascs
associatcd with nontradin, id-ask sjrcad, asynchronous ratcs and
so on, which arc ottcn jrcscnt in hihcr trcqucncy data \c considcr
thc o rcturn scrics y
t
100 o(
t
/
t1
), jottcd in Iiurc 213,
whcrc
t
is an cxchanc ratc cvc on thc t-th wcck Around thc
!!th wcck ot 199S (thc jcriod ot thc Asian ancc crisis), thc rcturns
on thc 1ajancsc and S doar cxchanc ratc dccrcascd y 9./
lmmcdiatcy attcr that oscrvation, wc can nd scvcra data joints
that arc arc in thc asoutc vauc Additionay, in othcr jarts
ot samjc, wc can oscrvc ucs, ic, custcrs ot oscrvations
with arc varianccs This jhcnomcnon is cacd voatiity custcrin
(jcrsistcncc) or, conditiona hctcroskcdasticity ln othcr words, thc
variancc chancs ovcr timc
To aow tor thc jossiiity that hih voatiity is toowcd y hih
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 36
0 500 1000 1500

1
0

5
0
5
Figure 2.13: Japanese - U.S. dollar exchange rate return series {y
t
}, from January 1, 1974
to December 31, 2003.
voatiity, and that ow voatiity wi c toowcd y ow voatiity,
whcrc voatiity is dcncd in tcrms ot rcturns thcmscvcs, onc can
considcr thc jrcscncc ot so-cacd conditiona hctcroskcdasticity or
its variants such as AlCL or GAlCL tyjc modcs y usin (y
t
)
2
as thc variancc ot thc rcturns Aso, oncc can rcjay (y
t
)
2
y |y
t
|,
thc asoutc vauc ot rcturn
Thc main jurjosc ot cxjoitin voatiity custcrin is to torccast
tuturc voatiity Sincc this variac is a mcasurc ot risk, such torccasts
can c usctu to cvauatc invcstmcnt stratcics or jorttoio sccction
or risk manacmcnt Iurthcrmorc, it can c usctu tor dccisions on
uyin or sccin ojtions or dcrivativcs Scc Lamiton (199!, Chajtcr
21), Tayor (200`), and Tsay (200`) tor dctais
2.3.4 Nonlinearity
Thc nonincar tcaturc ot timc scrics can c sccn ottcn in cconomic and
nancia data as wc as othcr ajjicd cds. scc thc jojuar ooks y
Ton (1990), Grancr and Tcrasvirta (1993), Iranscs and van Li,k
(2000), and Ian and Yao (2003) Lcyond incar domain, thcrc arc
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 37
innitc many nonincar torms to c cxjorcd Lary dcvcojmcnt ot
nonincar timc scrics anaysis tocuscd on various nonincar (somc-
timcs non-Gaussian) jaramctric torms Thc succcsstu cxamjcs in-
cudc, amon othcrs, thc AlCL-modcin ot uctuatin structurc
tor nancia timc scrics, and thc threshold modcin tor iooica
and cconomic data, as wc as regime switches or structural
change modcin tor cconomic and nancia timc scrics
Example 1.12: Considcr thc cxamjc ot thc uncmjoymcnt ratc in
Gcrmany tor 19o21 to 1991! in Iiurc 21! Irom thc rajh in thc
65 70 75 80 85 90
0
2
4
6
8
1
0
unadjusted
seasonally adjusted
2 4 6 8
2
4
6
8
Figure 2.14: Quarterly unemployment rate in Germany, 1962.11991.4 (seasonally adjusted
and not seasonally adjusted) in the left panel. The scatterplot of unemployment rate (sea-
sonally adjusted) versus unemployment rate (seasonally adjusted) one period lagged in the
right panel.
ctt janc, it is ccar that uncmjoymcnt ratc somctimcs riscs quitc
rajidy, usuay in thc rcccssion ycars 19o, 19! 19`, and 19S0
19S2, whic it dccrcascs vcry sowy, usuay in timcs ot cxjansions
This asymmctry can c tormaizcd y cstimatin thc jaramctcrs in
thc toowin simjc rcrcssion
y
t
y
t
y
t1

1
I
t
(E) +
2
I
t
(R) +
t
,
whcrc I
t
() is thc indicator variac, which aows thc asoutc vauc
ot thc ratc ot chanc to vary across thc two statc, say, dccrcasin
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 38
y
t
and incrcasin y
t
trom
1
to
2
, whcrc
1
may c dicrcnt
trom
2
Ior thc Gcrman scasonay ad,ustcd uncmjoymcnt ratc,
wc nd that

1
0.0!0 and

2
0.3SS, indicatin that whcn thc
uncmjoymcnt ratc incrcascs (in rcccssions), it riscs tastcr than whcn
it ocs down (in cxjansions)
Iurthcrmorc, trom thc rajh in thc riht janc, thc scattcrjot
ot y
t
vcrsus y
t1
, whcrc y
t
is thc scasonay ad,ustcd uncmjoymcnt
ratc, wc can oscrvc that this scrics disjays cycica chavior around
joints that shitt ovcr timc \hcn thcsc shitts arc cndocnous, ic,
causcd y jast oscrvations on y
t
thcmscvcs, this can c vicwcd as
a tyjica tcaturc ot nonincar timc scrics Ior thc dctaicd anaysis
ot this datasct usin a nonincar mcthods, thc rcadcr is rctcrrcd to
thc ook y Iranscs (199S) tor nonincar jaramctric modc and thc
jajcr y Cai (2002) tor nonjaramctric modc
2.4 Time Series Relationships
Onc can idcntity two asic kinds ot association or correlation
that arc imjortant in timc scrics considcrations Thc rst is thc no-
tion ot sct corrcation or autocorrelation introduccd in (22) Thc
sccond is that scrics arc somchow rcatcd to cach othcr so that onc
miht hyjothcsizc somc causa rcationshij cxistin ctwccn thc jhc-
nomcna cncratin thc scrics Ior cxamjc, onc miht hyjothcsizc
that thc simutancous osciations ot thc SOl and rccruitmcnt scrics
sucst that thcy arc rcatcd \c introducc cow thrcc statistics tor
idcntityin thc sourccs ot timc corrcation Assumc that wc havc two
scrics x
t
and y
t
that arc oscrvcd ovcr somc sct ot timc joints, say
t 1, . . . , n
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 39
2.4.1 Autocorrelation Function
Corrcation at ad,accnt joints ot thc samc scrics is mcasurcd y
thc autocorrcation tunction dcncd in (22) Ior cxamjc, thc SOl
scrics in Iiurc 21 contains rcuar uctuations at intcrvas ot aj-
jroximatcy 12 months An indication ot jossic incar as wc as
nonincar rcations can c intcrrcd y cxaminin thc acd scattcr-
jots, dcncd as jots that jut x
t
on thc horizonta Iiurc 21` `u-
tijc acd scattcrjots showin thc rcations ctwccn SOl and thc
jrcscnt (x
t
) and acd vaucs (x
t+h
) ot SOl at as h 0, 1, . . . , 12
axis and x
t+h
on thc vcrtica axis tor various vaucs ot thc a h 1,
2, 3, . . ., 12
Example 1.13: ln Iiurc 21`, wc havc madc a acd scattcrjot
ot thc SOl scrics at timc t + h aainst thc SOl scrics at timc t and
otaincd a hih corrcation, 0.!12, ctwccn thc scrics x
t+12
and thc
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o oo
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o
o
o o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o oo
o
o
o
o
o
o o
o
o
o o
o
oo
o
o o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
oo
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o oo
o
o
o
o
o
o
o
o ooo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
oo
oo
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
oo
o
o
o
o
o
o
o
o
o
o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
1
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
oo
o
o
o
o o
o
o
o
oo
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
oo
o o o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
oo
o
o
o o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
oo o
o
o
o
o
o
o o
o
o
o o
o
o o
o
o o o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o o oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
oo
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o o
oo
o
o
o
o
o o
o
o
o
o
o
o
o o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
2
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
oo
o
o
o
o o
o
o
o
oo
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o
o
oo
o
o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o oo
o
o
o
o
o
o o
o
o
o o
o
oo
o
oo o
o o
o
o
o
o
o
o
o
o
o
o
o
o
oo o
o
o
o
o o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
oo
o
o
o o o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
oo
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
3
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
oo o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
oo
o
o
oo
o
o
oo
o
o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o o
o o o
o
o
o
o
o
o o
o
o
o o
o
o o
o
ooo
o o
o
o
o
o
o
o
o
o
o
o
o
o
o oo
o
o
o
o o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
oo
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
oo
o
o
o
o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o o
o o
o
o
o
o
oo
o
o
o
o
o
o
o o
o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
4
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o oo
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o
o
o o
o
o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o o o
o
o
o
o
o
o o
o
o
o o
o
o o
o
o oo
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
oo
o
o
o
o
o
o
o
o
o o o
o
o
o
oo
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
oo o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
o o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
oo
o
o
o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
5
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o
o
oo
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o o o
o
o
o
o
o
o o
o
o
o o
o
o o
o
oo o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
oo
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
oo
o
o
o
o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
6
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
oo
o
o
o
o o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o o
o
o
o
o
oo
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
oo
o
o
o o
o
o
o o
o
o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o o o
o
o
o
o
o
o o
o
o
o o
o
o o
o
o oo
oo
o
o
o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
oo o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
o o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
oo
o o
o
o
o
o
o
o
o
o
o
o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
7
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
oo
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
oo
o
o
oo
o
o
o
oo
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
oo
o o o
o
o
o
o
o
oo
o
o
o o
o
o o
o
o o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o oo
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
oo
o o
o
o
o
o
oo
o
o
o
o
o
o
o o
o
o o
o
o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
8
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o
o
o o
o
o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o o o
o
o
o
o
o
o o
o
o
o o
o
o o
o
o o o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
oo
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
oo o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
o o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
oo
o
o
o
o
o
o o
oo
o
o
o
o
o
o
o
o
o
o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o
o
o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
9
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o
o
o o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
oo o
o
o
o
o
o
oo
o
o
o o
o
o o
o
o o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o
o
o
o
o
o
o
o
oo o
o
o
o
o o
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
o o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
ooo o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o o
oo
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
10
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o o
o o o
o
o
o
o
oo
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
oo
o
o
o o
o
o
o
oo
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o oo
o
o
o
o
o
o o
o
o
o o
o
o o
o
o o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
oo o
o
o
o
o o
o
o
o
o
o
o
o
o
ooo
o
o
o
o o
o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o ooo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
oo
o
o
o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
11
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
oo
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o
o
oo
o
o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o o o
o
o
o
o
o
o o
o
o
oo
o
o o
o
o o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o oo
o
o
o
o o
o
o
o
o
o
o
o
o
o oo
o
o
o
o o
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
oo o
o
o
o
o
o
o
o
o o oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
o o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
oo
o o
o
o
o
o
o
o
o
o
o
o
oo
o o
o
o
o
o
o o
o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
12
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
oo
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
oo
o o o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o
o
o o
o
o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o o o
o
o
o
o
o
o o
o
o
o o
o
o o
o
o o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
oo o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o oo
o
o
o
o
o
o
o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
oo
o o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
13
o
o
o
o
o
o
oo
o
o
o
o
o o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o o
o
o
o
o
o o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o
o
o o
o
o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o o o
o
o
o
o
o
o o
o
o
oo
o
o o
o
o o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
oo o
o
o
o
o
o
o
o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
oo
o
o
o
o o
o
o
o
o
o
o o
oo
o
o
o
o
o
o
o
o
o
o
o o
oo
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
14
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
oo
oo o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o
o
oo
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o o o
o
o
o
o
o
o o
o
o
o o
o
oo
o
o o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o o
o
o
o
o
o
o
o
o
oo o
o
o
o
o o
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o oo
o
o
o
o
o
o
o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o o
oo
o
o
o
o
o o
o
o
o
o
o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
15
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o oo
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o
o
o o
o
o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o o o
o
o
o
o
o
o o
o
o
o o
o
o o
o
o o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
oo o
o
o
o
o o
o
o
o
o
o
o
o
o
o oo
o
o
o
oo
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o o o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o o
o
o
o
o
o
oo
o o
o
o
o
o
o
o
o
o
o
o
o o
o o
o
o
o
o
oo
o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o
1.0 0.0 0.5 1.0
1
.
0
0
.
0
1
.
0
16
Figure 2.15: Multiple lagged scatterplots showing the relationship between SOI and the
present (x
t
) versus the lagged values (x
t+h
) at lags 1 h 16.
scrics x
t
shittcd y 12 ycars Lowcr ordcr as at t 1, t 2 aso
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 40
show corrcation Thc scattcrjot shows thc dircction ot thc rcation
which tcnds to c jositivc tor as 1, 2, 11, 12, 13, and tcnds to c
ncativc tor as o, , S Thc scattcrjot can aso show no sinicant
nonincaritics to c jrcscnt ln ordcr to dcvcoj a mcasurc tor this
sct corrcation or autocorrcation, wc utiizc a samjc vcrsion ot thc
scacd autocovariancc tunction (22), say

x
(h)

x
(h)/

x
(0),
whcrc

x
(h)
1
n
nh

t=1
(x
t+h
x)(x
t
x),
which is thc samjc countcrjart ot (22) with x

n
t=1
x
t
/n ndcr
thc assumjtion that thc undcryin jroccss x
t
is whitc noisc, thc
ajjroximatc standard crror ot thc samjc ACI is

n
. (23)
That is,

x
(h) is ajjroximatcy norma with mcan 0 and variancc
1/n
Example 1.14: As an iustration, considcr thc autocorrcation
tunctions comjutcd tor thc cnvironmcnta and rccruitmcnt scrics
shown in thc toj two jancs ot Iiurc 21o Loth ot thc autocor-
rcation tunctions show somc cvidcncc ot jcriodic rcjctition Thc
ACI ot SOl sccms to rcjcat at jcriods ot 12 whic thc rccruitmcnt
has a dominant jcriod that rcjcats at aout 12 to 1o timc joints
Aain, thc maximum vaucs arc wc aovc two standard crrors shown
as dottcd incs aovc and cow thc horizonta axis
2.4.2 Cross Correlation Function
Thc tact that corrcations may occur at somc timc dcay whcn tryin
to rcatc two scrics to onc anothcr at somc a h tor jurjoscs ot
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 41
0 10 20 30 40 50

0
.
5
0
.
0
0
.
5
1
.
0
ACF of SOI Index
0 10 20 30 40 50

0
.
5
0
.
0
0
.
5
1
.
0
ACF of Recruits
40 20 0 20 40

0
.
5
0
.
0
0
.
5
1
.
0
CCF of SOI and Recruits
Figure 2.16: Autocorrelation functions of SOI and recruitment and cross correlation function
between SOI and recruitment.
jrcdiction sucsts that it woud aso c usctu to jot x
t+h
aainst
y
t

Example 1.15: ln ordcr to cxaminc this jossiiity, considcr thc

acd scattcrjot matrix shown in Iiurcs 21 and 21S Iiurc
21 jots thc SOl at timc t + h, x
t+h
, vcrsus thc rccruitmcnt scrics
y
t
at a 0 h 1` in Iiurc 21 Thcrc arc no jarticuary stron
incar rcations ajjarcnt in this jots, ic tuturc vaucs ot SOl arc not
rcatcd to currcnt rccruitmcnt This mcans that thc tcmjcraturcs arc
not rcsjondin to jast rccruitmcnt ln Iiurc 21S, thc currcnt SOl
vaucs, x
t
arc jottcd aainst thc tuturc rccruitmcnt vaucs, y
t+h
tor
0 h 1` lt is ccar trom Iiurc 21S that thc scrics arc corrcatcd
ncativcy tor as h `, . . . , 9 Thc corrcation at a o, tor
cxamjc, is 0.o0 imjyin that incrcascs in thc SOl cad dccrcascs
in numcr ot rccruits y aout o months On thc othcr hand, thc
scrics arc hardy corrcatcd (0.02`) at a in thc convcntiona scnsc,
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 42
o oo o o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
ooo
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o oo oo o
oo
o
o
o
o o
o
oo o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o o o
o
o o
o o o
o
o
o
o
o
o o o
oo
o
o
o
o
o
o oo
o
o
o
o
o o
o
o oo
o
o
o
o
o o o o
o o
o
o
o
o
o o o
o
o
o
o
o
o o
oo
o
o
o
o
o
o o
o
oo
o
o
o
o
o
o o o o
o
o
o
o
o o o
o
o
o
o
o
o oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
oo o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
1
oo o o o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o o
oo o
o
o
o
o o oo
o
o
o
o
o
o
o o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o oo
o
o
o
o
o
oo oo oo
o o
o
o
o
o o
o
o o oo
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
oo
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
o o o
o
o
oo
o
o
oo
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
oo o
o
o
o
o
o o
o
oo o
o
o
o
o
o o oo
o o
o
o
o
o
o o o
o
o
o
o
o
o o
oo
o
o
o
o
o
o o
o
oo
o
o
o
o
o
o o o o
o
o
o
o
o o o
o
o
o
o
o
oo o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
2
o o o o oo
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
oo
o oo
o
o
o
o ooo
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o o o o
o o
o
o
o
o
o
o
oo o
o
o
o
o
o
o oo ooo
o o
o
o
o
o o
o
o oo o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
oo
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
oo
o
o o
o
o
o
oo
o
o
o o o
o
o
o
o
o
oo
o
o o o
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o oo
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o oo
o
o
o
o
o oo o
oo
o
o
o
o
o o o
o
o
o
o
o
oo
o o
o
o
o
o
o
o o
o
oo
o
o
o
o
o
o o o o
o
o
o
o
o o o
o
o
o
o
o
o oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
3
o o o oo o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
oo
oo o
o
o
o
ooo o
o
o
o
o
o
o
o o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
oo ooo o
oo
o
o
o
oo
o
oo o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
oo
o
o
o
oo
o
o
o o o
o
o
o
o
o
o o
o
o oo
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
oo o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
oo o
o
o
o
o
oo o o
o o
o
o
o
o
o o o
o
o
o
o
o
oo
o o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
o o o o
o
o
o
o
o oo
o
o
o
o
o
oo o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o oo
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
4
o o oo o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
oo o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o oo
o
o
o
o
o
o ooo o o
oo
o
o
o
o o
o
o o o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
oo
o
o o
o
o
o
o
o
oo
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
oo o
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
oo
o
o o o
o
o
o
o
o o oo
oo
o
o
o
o
o o o
o
o
o
o
o
o o
o o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
o o o o
o
o
o
o
oo o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o oo o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
oo o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
5
o oo o o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
oo
o o o
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
oo o
o
o
o
o
o
ooo o oo
o o
o
o
o
o o
o
o o o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
o oo
o
o
oo
o
o
o o
o
o
o o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o oo
o
o
o
o
o o
o
o o o
o
o
o
o
o oo o
o o
o
o
o
o
o o o
o
o
o
o
o
o o
o o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
o o o o
o
o
o
o
o o o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
oo o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
6
oo o o o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
oo o ooo
o o
o
o
o
oo
o
o o o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
oo
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
oo o
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
oo o
o
o
o
o
oo
o
o o o
o
o
o
o
oo oo
o o
o
o
o
o
o o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
o oo
o
o
o
o
o
o oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
oo
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
7
o o o o o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o o ooo o
o o
o
o
o
o o
o
o o o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
oo
o
o
o
o o
o
o
o o o
o
o
o
o
o
oo
o
o o o
o
o
o o
o
o
o o
o
o
oo
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o o
o
o
o
o
o oo o
o o
o
o
o
o
o o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
oo o
o
o
o
o
o
oo o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
oo
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
8
o o o o o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o oo
o
o
o
o o o o
o
o
o
o
o
o
oo
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o oo
o
o
o
o
o
o ooo o o
oo
o
o
o
o o
o
o o o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
oo
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
o o
o
o
o o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
oo
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o o
o
o
o
o
oo o o
o o
o
o
o
o
o o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
o oo
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
9
o o o o o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
ooo
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
oo o
o
o
o
o
o
ooo o o o
o o
o
o
o
o o
o
o o o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
oo
o
o
o
o
o
o o
o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o oo
o
o
o
o
o
oo
o
o oo
o
o
o o
o
o
o o
o
o
oo
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o oo
o
o
o
o
o o o o
o o
o
o
o
o
o oo
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
oo o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
oo
o
o
o
o
o
o
o
oo o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
10
o o o o o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
oo o
o
o
o
o o o o
o
o
o
o
o
o
oo
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
oo
o
o
o
o
o
o
o oo
o
o
o
o
o
oo o o oo
o o
o
o
o
o o
o
o o o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o oo
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
oo o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
oo
o
o o
o
o
o
o o
o
o
oo o
o
o
o
o
o
o o
o
oo o
o
o
o o
o
o
o o
o
o
oo
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o oo
o
o
o
o
o
o oo
o o
o
o
o
o
o
o oo
o
o
o
o
o o
o
oo o
o
o
o
o
o o o o
o o
o
o
o
o
ooo
o
o
o
o
o
o o
oo
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o oo
o
o
o
o
o oo
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
oo
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o oo
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
11
o o o o o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
oo
o o o
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
oo o
o
o
o
o
o
o o o oo o
oo
o
o
o
o o
o
o o o o
o
o o o o o
o
o
o
o
o
o
o
o
o
oo o
o
o
o
o
o o
o
oo
o
o
o
o
o
o o
o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
oo
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
oo
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
oo o
o
o
o
o
o
oo o
o o
o
o
o
o
o
oo o
o
o
o
o
o o
o
o o o
o
o
o
o
o o o o
o o
o
o
o
o
oo o
o
o
o
o
o
o o
oo
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o oo o
o
o
o
o
oo o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
oo o
o
o
oo
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
12
o o o o o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
oo
o o o
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o oo
o o
o
o
o
o
o
o
o oo
o
o
o
o
o
o o oo o o
o o
o
o
o
o o
o
o o o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
oo
o
o o
o
o
o
o
o
o o
o
o
o
o
ooo
o
o
oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o oo
o
o
o
o
o o
o
o oo
o
o
o
o
o o o o
o o
o
o
o
o
o o o
o
o
o
o
o
oo
oo
o
o
o
o
o
o o
o
o o
o
o
o
o
o
oo o o
o
o
o
o
o o o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
13
o o o o o o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o oo o
oo
o
o
o
o
o
o
ooo
o
o
o
o
o
o oo o oo
o o
o
o
o
o o
o
o o o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
oo o
o
o
o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
oo
o
o o o
o
o
o o
o
o
oo
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
oo
o o o
o
o
o
o
o
o o o
oo
o
o
o
o
o
oo o
o
o
o
o
oo
o
oo o
o
o
o
o
o o o o
o o
o
o
o
o
o o o
o
o
o
o
o
oo
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o oo
o
o
o
o
o o o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
14
o o o o o o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o o
o o o
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
oo o o
o o
o
o
o
o
o
o
oo o
o
o
o
o
o
oo o oo o
o o
o
o
o
o o
o
o o o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
oo
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
oo
o
o
o oo
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
oo
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o oo
o
o
o
o
o o o o
o o
o
o
o
o
o o o
o
o
o
o
o
oo
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o oo o
o
o
o
o
o oo
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
15
o o o o o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o oo
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o o oo o o
o o
o
o
o
o o
o
o o o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
oo o
o
o
o
o
o
o o
o
o o o
o
o
oo
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o oo
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
oo o
o
o
o
o
o o o o
o o
o
o
o
o
o o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
oo oo
o
o
o
o
oo o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o oo
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
16
Figure 2.17: Multiple lagged scatterplots showing the relationship between the SOI at time
t +h, say x
t+h
(x-axis) versus recruits at time t, say y
t
(y-axis), 0 h 15.
o oo o o o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
ooo
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o oo oo o
oo
o
o
o
o o
o
oo o o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o o o
o
o o
o o o
o
o
o
o
o
o o o
oo
o
o
o
o
o
o oo
o
o
o
o
o o
o
o oo
o
o
o
o
o o o o
o o
o
o
o
o
o o o
o
o
o
o
o
o o
oo
o
o
o
o
o
o o
o
oo
o
o
o
o
o
o o o o
o
o
o
o
o o o
o
o
o
o
o
o oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
oo o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
1
o oo o o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o oo
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o o oo oo
oo
o
o
o
oo
o
o oo o
o
o o o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
oo
o
o
o
o o
o
o
oo o
o
o
o
o
o
o o
o
oo o
o
o
oo
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
oo
o
o o o
o
o
o
o
o o o o
oo
o
o
o
o
o o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
o o o
o
o
o
o
o
oo o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o oo
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
2
o oo o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o o o oo o
o o
o
o
o
oo
o
o o oo
o
oo o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
oo o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
oo
o
o
ooo
o
o
o
o
o
o o
o
o oo
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
oo o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o o
o
o
o
o
o o o o
o o
o
o
o
o
oo o
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
o o o
o
o
o
o
o
o oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
oo o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o o o
o
o
o o
o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
3
o oo
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
oo o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
oo o o oo
oo
o
o
o
o o
o
o o o o
o
o oo o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o oo
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
oo
o
o
oo
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o oo
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o o
o
o
o
o
o o o o
o o
o
o
o
o
o oo
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
o o o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
4
o o
o
o o
o
oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
o oo o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
oo o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o oo o o o
o o
o
o
o
o o
o
oo o o
o
o o oo o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
oo o
o
o
o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
oo o
o
o
o
o
oo o o
o o
o
o
o
o
oo o
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
o o o
o
o
o
o
o
oo o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
oo o
o
o
o o
o
o
o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
5
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
oo oo
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o oo o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o o oo o o
oo
o
o
o
oo
o
ooo o
o
oo o oo
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
oo
o
o
o o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o oo
o
o
o
o
o oo o
o o
o
o
o
o
o oo
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
oo o o
o
o
o
o
o o o
o
o
o
o
o
o oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
6
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
ooo o
o
o
o
o
o
o
oo
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o oo
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
oo o oo o
o o
o
o
o
oo
o
o ooo
o
o oo o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
oo
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
oo
o
o o
o
o
o
oo
o
o
oo o
o
o
o
o
o
o o
o
o o o
o
o
oo
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
oo
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
oo
o
o o o
o
o
o
o
oo oo
o o
o
o
o
o
o o o
o
o
o
o
o
o o
o o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
ooo o
o
o
o
o
o o o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
7
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
oo
o o o
o
o
o
o ooo
o
o
o
o
o
o
oo
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o oo o oo
o o
o
o
o
o o
o
o o oo
o
o o oo o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o oo
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
o o
o
o
oo
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
oo o
o
o
o
o
o o
o
o o o
o
o
o
o
o oo o
o o
o
o
o
o
oo o
o
o
o
o
o
o o
o o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
oooo
o
o
o
o
o o o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
8
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
oo
o o o
o
o
o
o o oo
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
oo o o
oo
o
o
o
o
o
o
o o o
o
o
o
o
o
o o oo o o
o o
o
o
o
oo
o
oo o o
o
o o o oo
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o o
o
o o
oo o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o oo
o
o
o
o
o o
o
o o o
o
o
o
o
o o oo
oo
o
o
o
o
o oo
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o ooo
o
o
o
o
o o o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
oo o
o
o
o o
o
o
o
o
o
o
o
o o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
9
o
oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
oo o
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o oo o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o o o oo o
oo
o
o
o
o o
o
ooo o
o
oo o o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
oo
o
o
o
o o
o
o
o o o
o
o
o
o
o
oo
o
o o o
o
o
o o
o
o
o o
o
o
oo
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o o o
o
o o
o oo
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
oo o
o
o
o
o
o o o o
o o
o
o
o
o
o o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
oo
o
o
o
o
o
o o oo
o
o
o
o
o o o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o oo
o
o
oo
o
o
o
o
o
o
o
o o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
10
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
ooo
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o oo
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o o o o oo
o o
o
o
o
oo
o
o ooo
o
ooo o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
oo o
o
o
o
o
o
oo
o
o o o
o
o
o o
o
o
oo
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o o o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o oo
o
o
o
o
o o o o
oo
o
o
o
o
o o o
o
o
o
o
o
oo
o o
o
o
o
o
o
o o
o
oo
o
o
o
o
o
o o o o
o
o
o
o
o o o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o o
o
o
o
o
o
o
o
oo o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
11
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o oo
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o o o o o o
o o
o
o
o
o o
o
oo oo
o
o ooo o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
oo
o
o o
o
o
o
o o
o
o
o oo
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
o o
o
o
o o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
oo o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o o
o
o
o
o
oo o o
o o
o
o
o
o
o o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
o o o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o o
o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
12
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
oo
o
o
o
o
o
o
o o o
o
o
o
o
o
o o o o o o
oo
o
o
o
o o
o
o oo o
o
o o ooo
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
oo o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
oo o
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
ooo
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o o
o
o
o
o
o oo o
o o
o
o
o
o
o o o
o
o
o
o
o
oo
oo
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
o o o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o o
o
o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
13
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
o o o o
o
o
o
o
o
o
oo
o
o
o
o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o o o o o o
o o
o
o
o
o o
o
oo oo
o
oo o oo
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o oo
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
ooo
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o oo
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o o
o
o
o
o
o o oo
o o
o
o
o
o
o o o
o
o
o
o
o
o o
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
oo o
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
oo
o
o
o
o
o
o
o
o o
o
o
o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
14
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
o o o o
o
o
o
o
o
o
o o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o o o o o o
o o
o
o
o
oo
o
o oo o
o
ooo o o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
oo
o
o
o
oo
o
o
o o o
o
o
o
o
o
oo
o
o oo
o
o
o o
o
o
o o
o
o
oo
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
oo o
o
o o
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
oo
o
o o o
o
o
o
o
o o o o
o o
o
o
o
o
oo o
o
o
o
o
o
o o
oo
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
ooo
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
oo
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
15
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o o o
o
o
o
oo o o
o
o
o
o
o
o
oo
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o o o o
o o
o
o
o
o
o
o
o o o
o
o
o
o
o
o o o o o o
o o
o
o
o
o o
o
o o oo
o
o ooo o
o
o
o
o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
oo
o
o o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o o
o
o o o
o
o
o o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o oo
o
oo
o o o
o
o
o
o
o
o o o
o o
o
o
o
o
o
o o o
o
o
o
o
o o
o
o o o
o
o
o
o
o o o o
oo
o
o
o
o
o oo
o
o
o
o
o
oo
o o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o o o o
o
o
o
o
ooo
o
o
o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o o
o o o o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o o o
o
o
o o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
1.0 0.0 0.5 1.0
0
2
0
6
0
1
0
0
16
Figure 2.18: Multiple lagged scatterplots showing the relationship between the SOI at time
t, say x
t
(x-axis) versus recruits at time t +h, say y
t+h
(y-axis), 0 h 15.
mcasurcd at a h 0 Thc cncra jattcrn sucsts that jrcdictin
rccruits miht c jossic usin thc L `i no at as ot `, o ,, . . .
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 43
months
A mcasurc ot thc corrcation ctwccn scvcra scrics, x
t
and y
t
is thc
cross covariancc tunction (CCI), dcncd in tcrms ot thc countcrjart
covariancc to (22),
E{(x
t+h

x
)(y
t

y
)}
xy
(h).
Thc cross corrcation is thc scacd (to ic ctwccn 1 and 1) vcrsion
ot aovc, say

xy
(h)
xy
(h)/
_

x
(0)
y
(0).
Thc aovc quantitics arc cxjrcsscd in tcrms ot jojuation avcracs
and must cncray c cstimatcd trom samjc data Thc cstimatcd
samjc cross corrcation tunctions can c uscd to invcstiatc thc jos-
siiity ot thc scrics cin rcatcd at dicrcnt as ln ordcr to invcs-
tiatc cross corrcations ctwccn two scrics x
t
, y
t
, t 1, . . . , n, wc
notc that a rcasonac samjc vcrsion ot
xy
(h) miht c comjutcd
as

xy
(h)
1
n
nh

t=1
(x
t+h
x)(y
t
y),
whcrc y is thc samjc mcan ot {y
t
}
n
t=1
Thc cstimatcd samjc cross
corrcation tunction thcn ccomcs

xy
(h)

xy
(h)/
_

x
(0)

y
(0),
whcrc h dcnotcs thc amount that onc scrics is acd rcativc to
thc othcr Gcncray, onc comjutcs thc tunction

xy
(h) tor a numcr
ot jositivc and ncativc vaucs, h 0, 1, 2, uj to aout 0.3 n,
say and disjays thc rcsuts as a tunction ot a h Thc maximum
vauc, attaincd tor h 0, is
xy
(0) 1 and thc tunction takcs vaucs
ctwccn 1 and 1 This tact makcs it casy to comjarc vaucs ot thc
cross corrcation with cach othcr Iurthcrmorc, undcr thc hyjothc-
sis that thcrc is not rcation at timc h and that at cast onc ot thc
two scrics arc indcjcndcnt and idcnticay distriutcd, thc distriu-
tion ot

xy
(h) is ajjroximatcy norma with mcan 0 and standard
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 44
dcviation ivcn aain y (23) Lcncc, onc can comjarc vaucs ot
thc samjc cross corrcation with somc ajjrojriatc numcr ot sam-
jc standard dcviations ascd on norma thcory Gcncray, vaucs
within 1.9o

miht c rcasonac it onc is wiin to ivc with cach

tcst at a sinicancc cvc ot 0.0` Othcrwisc, roadcr imits woud
c ajjrojriatc ln cncra, it m tcsts arc madc, cach at cvc a thc
ovcra cvc ot sinicancc is oundcd y m
Example 1.16: To ivc an cxamjc, considcr thc cross corrcations
ctwccn thc cnvironmcnta scrics and rccruitmcnt shown in thc ot-
tom janc ot Iiurc 21o Thc cross corrcation ctwccn thc SOl,
x
t+h
and rccruitmcnt y
t
shows jcaks at h o, which imjics
that acd jroducts invovin x
t6
and y
t
as wc as thosc invov-
in x
t7
and y
t
match uj coscy Thc vauc shown on thc rajh is

xy
(o) 0.o This mcans, in jracticc that thc vaucs ot thc SOl
scrics tcnd to cad thc rccruitmcnt scrics y o or units Aso, onc
may notc that, sincc thc vauc is ncativc, owcr vaucs ot SOl arc
associatcd with hihcr rccruitmcnt Thc standard crror in this casc is
ajjroximatcy

0.0! and thc 0.o casiy cxcccds two standard

dcviations, shown as incs aovc and cow thc axis in Iiurc 21o
Lcncc, wc can rc,cct thc hyjothcsis that thc corrcation is zcro at
that a
lt is ccar aso that thcrc arc somc jcriodic uctuations ajjarcnt in
thc cross corrcations Ior cxamjc, in thc SOl lccruitmcnt cxamjc,
thcrc sccms to c systcmatic uctuation with a jcriod (1 tu cycc) ot
aout 12 months This jroduccs a numcr ot sccondary jcaks in thc
cross corrcation tunction Thc anaysis ot this jcriodic chavior and
thc accountin tor jcriodicitics in thc scrics is considcrcd in atcr
scctions or thc ooks y Iranscs (199o) and Ghyscs and Osorn
(2001)
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 45
2.4.3 Partial Autocorrelation Function
A third kind ot timc scrics rcationshij cxjrcsscs what is csscntiay
thc sct jrcdictaiity ot a scrics throuh thc jartia autocorrcation
tunction (lACI) Thcrc arc scvcra ways ot thinkin aout this mca-
surc
Onc may rcard thc lACI as thc simjc corrcation ctwccn two
joints scjaratcd y a a h, say x
t
and x
th
, with thc ccct ot thc
intcrvcnin joints x
t1
, x
t2
, . . ., x
th+1
conditioncd out, ic thc
jurc corrcation ctwccn thc two joints This intcrjrctation is ottcn
ivcn in morc jractica statistica scttins Ior cxamjc, onc may ct
siy causa intcrcnccs y quotin corrcations ctwccn two variacs,
c tcachcrs incomc and winc consumjtion, that may occur simjy
ccausc oth arc corrcatcd with somc common drivin tactor, in
this casc, thc ross domcstic jroduct, GLl, or somc othcr torcc
inucncin disjosac incomc
ln timc scrics anaysis wc arc rcay morc intcrcstcd in thc jrc-
diction or torccastin jrocm ln this casc wc miht considcr thc
jrocm ot jrcdictin x
t
ascd on oscrvation oscrvcd h units ack
in thc jast, say x
t1
, x
t2
, . . ., x
th
Sujjosc wc want to jrcdict x
t
trom x
t1
, . . ., x
th
usin somc incar tunction ot thcsc jast vaucs
Considcr minimizin thc mcan squarc jrcdiction crror
MSE E|(x
t

x
t
)
2
|
usin thc jrcdictor

x
t
a
1
x
t1
+ a
2
x
t2
+ + a
h
x
th
ovcr thc
jossic vaucs ot thc wcihtin coccicnts a
1
, . . ., a
h
whcrc wc
assumc, tor convcnicncc that x
t
has ccn ad,ustcd to havc zcro mcan
Considcr thc rcsut ot minimizin thc aovc mcan squarc jrcdic-
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 46
tion crror tor a jarticuar a h Thcn, thc jartia autocorrcation
tunction is dcncd as thc vauc ot thc ast coccicnt

a
h
, ic
hh
a
h

As a jractica mattcr, wc minimizc thc samjc crror sum ot squarcs

SSE
n

t=h+1
_
_
(x
t
x)
h

k=1
a
k
(x
tk
x)
_
_
2
with thc cstimatcd jartia corrcation dcncd as

hh


a
h

Thc coccicnts, as dcncd aovc, arc aso ctwccn 1 and 1 and

havc thc usua jrojcrtics ot corrcation coccicnts ln jarticuar,
its standard crror undcr thc hyjothcsis ot no jartia autocorrcation
is sti (23) Thc intuition ot thc aovc arumcnt is that thc ast
coccicnt wi ct vcry sma oncc thc torccast horizon or a h is
arc cnouh to ivc ood jrcdiction ln jarticuar, thc ordcr ot thc
autorcrcssivc modc in thc ncxt chajtcr wi c cxacty thc a h
cyond which
hh
0
Example 1.17: As an cxamjc, wc show in thc riht jancs janc
ot Iiurc 219 thc jartia autocorrcation tunctions ot thc SOl scrics
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
PACF of SOI
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
PACF of Recruits
Figure 2.19: Partial autocorrelation functions for the SOI (left panel) and the recruits (right
panel) series.
(ctt janc) and thc rccruits scrics (riht janc) `otc that thc lACI
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 47
ot thc SOl has a sinc jcak at a h 1 and thcn rcativcy sma
vaucs This mcans, in ccct, that tairy ood jrcdiction can c
achicvcd y usin thc immcdiatcy jrcccdin joint and that addin
turthcr vaucs docs not rcay imjrovc thc situation Lcncc wc miht
try an autorcrcssivc modc with p 1 Thc rccruits scrics has
two jcaks and thcn sma vaucs, imjyin that thc jurc corrcation
ctwccn joints is summarizcd y thc rst two as
A ma,or ajjication ot thc lACI is to dianosin thc ajjrojriatc
ordcr ot an autorcrcssivc modc tor thc scrics undcr considcration
Autorcrcssivc modcs wi c studicd cxtcnsivcy atcr ut wc notc
hcrc that thcy arc incar modcs cxjrcssin thc jrcscnt vauc ot a
scrics as a incar comination ot a numcr ot jrcvious vaucs, with
an additivc crror Lcncc, an autorcrcssivc modc usin two jrcvious
vaucs tor jrcdiction miht c writtcn in thc torm x
t

1
x
t1
+

2
x
t2
+ w
t
whcrc
1
and
2
arc xcd unknown coccicnts and w
t
arc vaucs trom an indcjcndcnt scrics with zcro mcan and common
variancc
2
w
Ior cxamjc, it thc lACI at a h is rouhy zcro
cyond a xcd vauc, say h 2 as oscrvcd tor thc rccruits scrics in
thc ctt janc ot Iiurc 219, thcn onc miht assumc a modc ot thc
torm aovc tor that rccruits scrics
To nish thc introductory discussion, wc notc that thc cxtcnsion ot
what has ccn said aovc to mutivariatc timc scrics is tairy straiht-
torward it onc rcstricts thc discussion to rcations ctwccn two scrics
at a timc Thc two scrics x
t
and y
t
can c aid out in thc vcctor
(x
t
, y
t
) which ccomcs a mutivariatc scrics ot dimcnsion two (i-
variatc) To cncraizc, considcr thc p scrics (x
t1
, x
t2
, . . . , x
tp
) x
t
as thc row vcctor dcnin thc mutivariatc scrics x
t
Thc autocovari-
ancc matrix ot thc vcctor x
t
can thcn c dcncd as thc p p matrix
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 48
containin as ccmcnts

ij
(h) E{(x
t+h,i

i
)(x
tj

j
)}
and thc anaysis ot thc cross covariancc structurc can jrocccd on two
ccmcnts at a timc Thc discussion ot jossic mutijc rcations is
dctcrrcd unti atcr
To rccajituatc, thc jrimary o,cctivc ot this chajtcr has ccn to
dcnc and iustratc with rca cxamjcs thrcc statistics ot intcrcst
in dcscriin rcationshijs within and amon timc scrics Thc au-
tocorrcation tunction mcasurcs thc corrcation ovcr timc in a sinc
scrics. this corrcation can c cxjoitcd tor jrcdiction jurjoscs or tor
sucstin jcriodicitics Thc cross corrcation tunction mcasurcs thc
corrcation ctwccn scrics ovcr timc. it may c, tor cxamjc, that a
scrics is cttcr rcatcd to thc jast ot anothcr scrics than it is to its
own jast Thc jartia autocorrcation tunction ivcs a dircct mca-
surc ot thc a cnth ncccssary to jrcdict a scrics trom itsct, ic to
torccast tuturc vaucs lt is aso critica in dctcrminin thc ordcr ot
an autorcrcssivc modc satiscd y somc rca data sct
lt shoud c notcd that a thrcc ot thc mcasurcs ivcn in this
scction can c distortcd it thcrc arc sinicant trcnds in thc data lt
is ovious that acd jroducts ot thc torm (x
t+h
x)(x
t
x) wi c
articiay arc it thcrc is a trcnd jrcscnt Sincc thc corrcations ot
intcrcst arc usuay associatcd with thc stationary jart ot thc scrics,
ic, thc jart that can c thouht ot as cin sujcrimjoscd on trcnd,
it is usua to cvauatc thc corrcations ot thc dctrcndcd scrics This
mcans, in ccct, that wc rcjacc x in thc cquations y

a +

b t it
thc trcnd can c considcrcd to c incar lt thc trcnd is quadratic
or oarithmic, thc ajjrojriatc atcrnatc nonincar jrcdictcd vauc
is sutractcd ctorc comjutin thc acd jroducts `otc aso that
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 49
dicrcncin thc scrics, as discusscd in Scction 222, can accomjish
thc samc rcsut
2.5 Problems
1. Considcr a cncraization ot thc modc ivcn in Example 1.1,
namcy, x
t
+w
t
w
t1
, whcrc {w
t
} arc indcjcndcnt zcro-
mcan random variacs with variancc
2
w
lrovc that E(x
t
) ,

x
(0) (1 +
2
)
2
w
,
x
(1)
2
w
,
x
(h) 0 it |h| > 1, and
nay show that x
t
is wcaky stationary
2 Considcr thc timc scrics cncratcd y x
1
+ w
1
and x
t

+ x
t1
+ w
t
tor t 2 Show that x
t
is not stationary no
mattcr 0 or not and nd
x
(h)
3. Sujjosc that x
t
is stationary with mcan
x
and covariancc tunc-
tion ivcn y
x
(h) Iind thc mcan and covariancc tunction ot
(a) y
t
a + b x
t
, whcrc a and b arc constants,
() z
t
x
t
x
t1

4. Considcr thc incar jroccss

y
t

j=
a
j
w
tj
,
whcrc w
t
is a whitc noisc jroccss with variancc
2
w
and a
j
, j 0,
1, 2, ldots, arc constants Thc jroccss y
t
wi cxist (as a
imit in mcan squarc) it

j
|a
j
| < . you do not nccd to jrovc
this Show that thc scrics y
t
is stationary, with autocovariancc
tunction

y
(h)
2
w

j=
a
j+h
a
j
.
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 50
Ajjy thc rcsut to cacuatin thc autocovariancc tunction ot
thc 3-joint movin avcrac (x
t1
+ x
t
+ x
t+1
)/3
Ior thc toowin jrocms, you nccd to usc a comjutcr jackac
5. `ctin acicrs dcjosit ycary aycrs ot sand and sit durin thc
sjrin mctin scasons which can c rcconstructcd ycary ovcr a
jcriod ranin trom thc timc dc-aciation can in `cw Ln-
and (aout 12,o00 ycars ao) to thc timc it cndcd (aout o, 000
ycars ao) Such scdimcntary dcjosits, cacd varvcs, can c
uscd as a jroxy tor jacocimatic jaramctcrs such as tcmjcra-
turc Thc c mass2.dat contains ycary rccords tor o3! ycars
cinnin 11, S3! ycars ao, cocctcd trom onc ocation in `as-
year
v
a
r
v
e

t
h
i
c
k
n
e
s
s
0 100 200 300 400 500 600
0
5
0
1
0
0
1
5
0
Varve thickness from Massachusetts (n=634)
Figure 2.20: Varve data for Problem 5.
sachusctts Ior turthcr intormation, scc Shumway and \crosu
(1992)
(a) lot thc varvc rccords and cxaminc thc autocorrcation and
jartia autocorrcation tunctions tor cvidcncc ot nonstationarity
(b) Aruc that thc transtormation y
t
o x
t
miht c usctu tor
staiizin thc variancc Comjutc
x
(0) and
y
(0) ovcr two timc
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 51
intcrvas tor cach scrics to dctcrminc whcthcr this is rcasonac
lot thc historams ot thc raw and transtormcd scrics
(c) lot thc autocorrcation ot thc scrics y
t
and aruc that a rst
dicrcncc jroduccs a rcasonay stationary scrics Can you think
ot a jractica intcrjrctation tor u
t
y
t
y
t1
o x
t
o x
t1

(d) Comjutc thc autocorrcation tunction ot thc dicrcnccd

transtormcd scrics and aruc that a cncraization ot thc modc
ivcn y Example 1.1 miht c rcasonac Assumc that
u
t
w
t
w
t1
is stationary whcn thc injuts w
t
arc assumcd
indcjcndcnt with E(w
t
) 0 and (w
2
t
)
2
w
sin thc samjc
ACI and thc jrintcd autocovariancc

u
(0), dcrivc cstimators tor
and
2
w

6. Two timc scrics rcjrcscntin avcrac whocsac S asoinc and

oi jriccs ovcr 1S0 months, cinnin in 1uy, 193 and cndin in
Lcccmcr, 19S arc ivcn in thc c oil-gas.dat Anayzc thc
data usin somc ot thc tcchniqucs in this chajtcr with thc idca
that onc shoud c ookin at how chancs in oi jriccs inucncc
month
p
r
ic
e
0 50 100 150
1
0
0
2
0
0
3
0
0
4
0
0
5
0
0
6
0
0
7
0
0
Gas and oil prices (n=180 months)
GAS
OIL
Figure 2.21: Gas and oil series for Problem 6.
chancs in as jriccs Ior turthcr rcadin, scc Liu (1991) ln
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 52
jarticuar, considcr thc toowin ojtions
(a) lot thc raw data and ook at thc autocorrcation tunctions
to aruc that thc untranstormcd data scrics arc nonstationary
(b) lt is ottcn arucd in cconomics that jricc chancs arc im-
jortant, in jarticuar, thc jcrccntac chanc in jriccs trom onc
month to thc ncxt On this asis, aruc that a transtormation
ot thc torm y
t
o x
t
o x
t1
miht c ajjicd to thc data
whcrc x
t
is thc oi or as jricc scrics
(c) sc acd mutijc scattcrjots and thc auto and cross cor-
rcation tunctions ot thc transtormcd oi and as jricc scrics to
invcstiatc thc jrojcrtics ot thcsc scrics ls it jossic to ucss
whcthcr as jriccs arc raiscd morc quicky in rcsjonsc to in-
crcasin oi jriccs than thcy arc dccrcascd whcn oi jriccs arc
dccrcascd Lo you think that it miht c jossic to jrcdict
o jcrccntac chancs in as jriccs trom o jcrccntac chancs
in oi jriccs lot thc two scrics on thc samc scac
7. `onthy Landun Sacs and Iircarms lcatcd Lcaths in Caitor-
nia Lca handun jurchasc intormation tor 22 months sjan-
nin thc timc jcriod Icruary 1, 19S0 throuh Lcccmcr 31,
199S was otaincd trom thc Lcjartmcnt ot 1usticcs automatcd
Iircarms systcms dataasc Caitornia rcsidcnt rcarms dcath
data was otaincd torm thc Caitornia Lcjartmcnt ot Lcath
Scrviccs Thc data arc jottcd in thc urc, with oth ratcs
ivcn in numcrs jcr 100,000 rcsidcnts Sujjosc that thc main
qucstion ot intcrcst tor this data jcrtains to thc jossic rcations
ctwccn handun sacs and dcath ratcs ovcr this timc jcriod
lncudc thc jossiiity ot ain rcations in your anaysis ln
jarticuar, answcr thc qucstions cow
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 53
months
0 50 100 150 200
1
.
0
1
.
5
2
.
0
Gun sales and gun death rate
Hangun sales/100 per 100,000
Gun death rate per 100,000
Figure 2.22: Handgun sales (per 10,000,000) in California and monthly gun death rate (per
100,00) in California (February 2, 1980 -December 31, 1998.
(a) sc scattcrjots to aruc that thcrc is a jotcntia nonincar
rcation ctwccn dcath ratcs and handun sacs and indicatc
whcthcr you think that thcrc miht c a a
(b) Lostcr your arumcnt tor a ain rcationshij y cxamin-
in thc cross corrcation tunction \hat do thc autocorrcation
tunctions indicatc tor this data
(c) Lxaminc thc rst dicrcncc tor thc two jroccsscs and indi-
catc what thc ACIs and CCIs show tor thc dicrcnccd data
(d) Smooth thc two scrics with a 12 joint movin avcrac and
jot thc two scrics on thc samc rajh Sutract thc movin av-
crac trom thc oriina unsmoothcd scrics \hat do thc rcsidua
scrics show in thc ACI and CCI tor this casc
2.6 Computer Code
Thc toowin R commands arc uscd tor makin thc rajhs in this
chajtcr
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 54
# 3-28-2006
graphics.off() # clean the previous graph on the screen
################################################################
# This is Southern Oscillation Index data and Recruits data
##############################################################
y<-read.table("c:\\teaching\\time series\\data\\soi.dat",header=T)
# read data file
x<-read.table("c:\\teaching\\time series\\data\\recruit.dat",header=T)
y=y[,1]
x=x[,1]
postscript(file="c:\\teaching\\time series\\figs\\fig-1.1.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(1,2),mex=0.4)
# save the graph as a postscript file
ts.plot(y,type="l",lty=1,ylab="",xlab="")
# make a time series plot
title(main="Southern Oscillation Index",cex=0.5)
# set up the title of the plot
abline(0,0)
# make a straight line
ts.plot(x,type="l",lty=1,ylab="",xlab="")
abline(mean(x),0)
title(main="Recruit",cex=0.5)
dev.off()
z=arima.sim(n=200,list(ma=c(0.9))) # simulate a MA(1) model
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 55
postscript(file="c:\\teaching\\time series\\figs\\fig-1.2.eps",
horizontal=F,width=6,height=6)
ts.plot(z,type="l",lty=1,ylab="",xlab="")
title(main="Simulated MA(1)",cex=0.5)
abline(0,0)
dev.off()
n=length(y)
n2=n-12
yma=rep(0,n2)
for(i in 1:n2){yma[i]=mean(y[i:(i+12)])} # compute the
yy=y[7:(n2+6)]
yy0=yy-yma
postscript(file="c:\\teaching\\time series\\figs\\fig-1.9.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(1,2),mex=0.4)
ts.plot(yy,type="l",lty=1,ylab="",xlab="")
points(1:n2,yma,type="l",lty=1,lwd=3,col=2)
ts.plot(yy0,type="l",lty=1,ylab="",xlab="")
points(1:n2,yma,lty=1,lwd=3,col=2) # make a point plot
abline(0,0)
dev.off()
m=17
n1=n-m
y.soi=rep(0,n1*m)
dim(y.soi)=c(n1,m)
y.rec=y.soi
for(i in 1:m){
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 56
y.soi[,i]=y[i:(n1+i-1)]
y.rec[,i]=x[i:(n1+i-1)]}
text_soi=c("1","2","3","4","5","6","7","8","9","10","11","12","13"
"14","15","16")
postscript(file="c:\\teaching\\time series\\figs\\fig-1.15.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(4,4),mex=0.4)
for(i in 2:17){
plot(y.soi[,1],y.soi[,i],type="p",pch="o",ylab="",xlab="",
ylim=c(-1,1),xlim=c(-1,1))
text(0.8,-0.8,text_soi[i-1],cex=2)}
dev.off()
text1=c("ACF of SOI Index")
text2=c("ACF of Recruits")
text3=c("CCF of SOI and Recruits")
SOI=y
Recruits=x
postscript(file="c:\\teaching\\time series\\figs\\fig-1.16.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4)
acf(y,ylab="",xlab="",ylim=c(-0.5,1),lag.max=50,main="")
# make an ACF plot
legend(10,0.8, text1) # set up the legend
acf(x,ylab="",xlab="",ylim=c(-0.5,1),lag.max=50,main="")
legend(10,0.8,text2)
ccf(y,x, ylab="",xlab="",ylim=c(-0.5,1),lag.max=50,main="")
legend(-40,0.8,text3)
dev.off()
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 57
postscript(file="c:\\teaching\\time series\\figs\\fig-1.17.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(4,4),mex=0.4)
for(i in 1:16){
plot(y.soi[,i],y.rec[,1],type="p",pch="o",ylab="",xlab="",
ylim=c(0,100),xlim=c(-1,1))
text(-0.8,10,text_soi[i],cex=2)}
dev.off()
postscript(file="c:\\teaching\\time series\\figs\\fig-1.18.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(4,4),mex=0.4)
for(i in 1:16){
plot(y.soi[,1],y.rec[,i],type="p",pch="o",ylab="",xlab="",
ylim=c(0,100),xlim=c(-1,1))
text(-0.8,10,text_soi[i],cex=2)}
dev.off()
postscript(file="c:\\teaching\\time series\\figs\\fig-1.19.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(1,2),mex=0.4)
pacf(y,ylab="",xlab="",lag=30,ylim=c(-0.5,1),main="")
text(10,0.9,"PACF of SOI")
pacf(x,ylab="",xlab="",lag=30,ylim=c(-0.5,1),main="")
text(10,0.9,"PACF of Recruits")
dev.off()
################################################################
###################################################################
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 58
# This is global temperature data
#################################
y1<-matrix(scan("c:\\teaching\\time series\\data\\ngtemp.dat"),
byrow=T,ncol=1)
a<-1:12
a=a/12
y=y1[,1]
n=length(y)
x<-rep(0,n)
for(i in 1:149){
x[((i-1)*12+1):(12*i)]<-1856+i-1+a
}
x[n-1]<-2005+1/12
x[n]=2005+2/13
#########################
# Nonparametric Fitting #
#########################
#########################################################
# Define the Epanechnikov kernel function local estimator
kernel<-function(x){0.75*(1-x^2)*(abs(x)<=1)}
###############################################################
# Define the function for computing the local linear estimation
local<-function(y,x,z,h){
# parameters: y=response, x=design matrix; h=bandwidth; z=
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 59
nz<-length(z)
ny<-length(y)
beta<-rep(0,nz*2)
dim(beta)<-c(nz,2)
for(k in 1:nz){
x0=x-z[k]
w0<-sqrt(kernel(x0/h))
beta[k,]<-glm(y~x0,weight=w0)$coeff
}
return(beta)
}
###################################################################
z=x
h=12 # take a badnwidth
fit=local(y,x,z,h) # fit model y=m(x) + e
mhat=fit[,1] # obtain the nonparametric estimate
resid1=y-(-9.037+0.0046*x)
resid2=y-mhat
postscript(file="c:\\teaching\\time series\\figs\\fig-1.4.eps",
horizontal=F,width=6,height=6)
matplot(x,y,type="p",pch="o",ylab="",xlab="",cex=0.5)
# make multiple plots
points(z,mhat,type="l",lty=1,lwd=3,col=2)
abline(-9.037,0.0046,lty=1,lwd=5,col=3)
# make a stright line with an intercept and slope
title(main="Original Data with Linear and Nonlinear Trend",cex=0.5)
dev.off()
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 60
postscript(file="c:\\teaching\\time series\\figs\\fig-1.5.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(1,2),mex=0.4)
matplot(x,resid1,type="l",lty=1,ylab="",xlab="",cex=0.5)
abline(0,0)
title(main="Detrended: Linear",cex=0.5)
matplot(x,resid2,type="l",lty=1,ylab="",xlab="",cex=0.5)
abline(0,0)
title(main="Detrended: Nonlinear",cex=0.5)
dev.off()
y_diff=diff(y)
postscript(file="c:\\teaching\\time series\\figs\\fig-1.6.eps",
horizontal=F,width=6,height=6)
plot(x[-1],y_diff,type="l",lty=1,ylab="",xlab="",cex=0.5)
abline(0,0)
title(main="Differenced Time Series",cex=0.5)
dev.off()
###################################################################
# This is China data
###################################
data<-read.table("c:/teaching/stat3150/data/data1.txt",header=T)
# read data from a file containing 6 columns of data
y<-data[,1:5]
# put the first 5 columns of data into y
x<-data[,6]
text1<-c("agriculture","commerce","consumption","industry","transp
# set the text for legend in a graph
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 61
postscript(file="c:\\teaching\\time series\\figs\\fig-1.3.eps",
horizontal=F,width=6,height=6)
matplot(x,log(y),type="l",lty=1:5,ylab="",xlab="")
legend(1960,8,text1,lty=1:5,col=1:5)
dev.off()
###################################################################
# This is motor cycles data
###################################
data<-read.table("c:/teaching/stat3150/data/data7.txt",header=T)
# read data from a file containing 6 columns of data
y<-data[,1]
x<-data[,2]-1900
y_diff1=diff(y)
y_diff2=diff(y_diff1)
postscript(file="c:\\teaching\\time series\\figs\\fig-1.7.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4)
matplot(x,y,type="l",lty=1,ylab="",xlab="")
text(60,250,"Data")
ts.plot(y_diff1,type="l",lty=1,ylab="",xlab="")
text(20,40,"First difference")
abline(0,0)
ts.plot(y_diff2,type="l",lty=1,ylab="",xlab="")
text(20,25,"Second order difference")
abline(0,0)
dev.off()
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 62
###################################################################
# This is Johnson and Johnson data
###################################
y<-matrix(scan("c:\\teaching\\time series\\data\\jj.dat"),byrow=T,ncol=1)
n=length(y)
y_log=log(y) # log of data
postscript(file="c:\\teaching\\time series\\figs\\fig-1.8.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(1,2))
ts.plot(y,type="l",lty=1,ylab="",xlab="")
title(main="J&J Earnings",cex=0.5)
ts.plot(y_log,type="l",lty=1,ylab="",xlab="")
title(main="transformed log(earnings)",cex=0.5)
dev.off()
###################################################################
# This is retail sales data
###################################
y=matrix(scan("c:\\res\\0published\\cai-chen\\retail\\retail-sales.dat"),
byrow=T,ncol=1)
postscript(file="c:\\teaching\\time series\\figs\\fig-1.10.eps",
horizontal=F,width=6,height=6)
ts.plot(y,type="l",lty=1,ylab="",xlab="")
dev.off()
###################################################################
# This is marketing data
###################################
text_tv=c("television")
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 63
text_radio=c("radio")
data<-read.table("c:/teaching/stat3150/data/data4.txt",header=T)
TV=log(data[,1])
RADIO=log(data[,2])
postscript(file="c:\\teaching\\time series\\figs\\fig-1.11.eps",
horizontal=F,width=6,height=6)
ts.plot(cbind(TV,RADIO),type="l",lty=c(1,2),col=c(1,2),ylab=
text(20,10.5,text_tv)
text(165,8,text_radio)
dev.off()
###################################################################
# This is Argentina data
###################################
text_ar=c("difference", "inflation")
y<-read.table("c:/teaching/stat3150/data/data8.txt",header=T)
y=y[,1]
n=length(y)
y_t=diff(log(y))
f_t=diff(y)/y[1:(n-1)]
x=seq(70.25,by=0.25,89.75)
postscript(file="c:\\teaching\\time series\\figs\\fig-1.12.eps",
horizontal=F,width=6,height=6)
matplot(x,cbind(y_t,f_t),type="l",lty=c(1,2),col=c(1,2),ylab
legend(72,5,text_ar,lty=c(1,2),col=c(1,2))
dev.off()
###################################################################
# This is exchange rate data
###################################
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 64
x<-matrix(scan(file="c:\\res\\cai-xu\\jpy\\jpy.dat"),byrow=T,ncol=1)
n<-length(x)
nweek<-(n-7)/5
week1<-rep(0,n) # Dates for week
week1[1:4]<-2:5
for(j in 1:nweek){
i1<-4+(j-1)*5+1
i2<-4+j*5
week1[i1:i2]<-c(1,2,3,4,5)
}
i2<-(nweek+1)*5
week1[i2:n]<-1:3
y<-x[week1==3] # Wednsday
x1<-x[week1==4] # Thursday
x1<-append(x1,0)
x1<-(1-(y>0))*x1 # Take value from Thursday if ND on Wednsday
x1<-y+x1 # Wednsday + Thursday
n<-length(x1)
x<-100*(log(x1[2:n])-log(x1[1:(n-1)])) # log return
postscript(file="c:\\teaching\\time series\\figs\\fig-1.13.eps",
horizontal=F,width=6,height=6)
ts.plot(x,type="l",ylab="",xlab="")
abline(0,0)
dev.off()
###################################################################
# This is unemployment data
###################################
text_unemploy=c("unadjusted", "seasonally adjusted")
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 65
data<-read.table("c:/teaching/stat3150/data/data10.txt",header=T)
y1=data[,1]
y2=data[,2]
n=length(y1)
x=seq(62.25,by=0.25,92)
postscript(file="c:\\teaching\\time series\\figs\\fig-1.14.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(1,2),mex=0.4)
matplot(x,cbind(y1,y2),type="l",lty=c(1,2),col=c(1,2),ylab="
legend(66,10,text_unemploy,lty=c(1,2),col=c(1,2))
plot(y2[1:(n-1)],y2[2:n],type="l",lty=c(1),col=c(1),ylab="",
dev.off()
###################################################################
# This is varve data
#####################
x<-matrix(scan("c:\\teaching\\time series\\data\\mass2.dat"),byrow=T,ncol=1)
postscript(file="c:\\teaching\\time series\\figs\\fig-1.20.eps",
horizontal=F,width=6,height=6)
ts.plot(x,type="l",lty=1,ylab="varve thickness",xlab="year")
title(main="Varve thickness from Massachusetts (n=634)",cex=0.5)
dev.off()
###################################################################
# This is oil-gas data
#######################
data<-matrix(scan("c:\\teaching\\time series\\data\\gas-oil.dat"),
byrow=T,ncol=2)
text4=c("GAS","OIL")
postscript(file="c:\\teaching\\time series\\figs\\fig-1.21.eps",
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 66
horizontal=F,width=7,height=7)
ts.plot(data,type="l",lty=c(1,2),col=c(1,2),ylab="price",xlab=
title(main="Gas and oil prices (n=180 months)",cex=0.5)
legend(20,700,text4,lty=c(1,2),col=c(1,2))
dev.off()
###################################################################
# This is handgun data
#####################
y<-matrix(scan("c:\\teaching\\time series\\data\\guns.dat"),byrow=T,ncol=2)
sales=y[,1]
y=cbind(y[,1]/100,y[,2])
text5=c("Hangun sales/100 per 100,000")
text6=c("Gun death rate per 100,000")
postscript(file="c:\\teaching\\time series\\figs\\fig-1.22.eps",
horizontal=F,width=7,height=7)
par(mex=0.4)
ts.plot(y,type="l",lty=c(1,2),col=c(1,2),ylab="",xlab="months"
title(main="Gun sales and gun death rate",cex=0.5)
legend(20,2,lty=1,col=1,text5)
legend(20,0.8,lty=2,col=2,text6)
dev.off()
###################################################################
2.7 References
Bollerslev, T. (1986). Generalized autoregressive conditional heteroskedasticity. Journal of
Econometrics, 31, 307-327.
Burman, P. and R.H. Shumway (1998). Semiparametric modeling of seasonal time series.
Journal of Time Series Analysis, 19, 127-145.
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 67
Cai, Z. (2002). A two-stage approach to additive time series models. Statistica Neerlandica,
56, 415-433.
Cai, Z. (2006). Trending time varying coecient time series models with serially correlated
errors. Forthcoming in Journal of Econometrics.
Cai, Z. and R. Chen (2006). Flexible seasonal time series models. Advances in Economet-
rics, 20B, 63-87.
Engle, R.F. (1982). Autoregressive conditional heteroscedasticity with estimates of the
variance of United Kingdom inations. Econometrica, 50, 987-1007.
Fan, J. and Q. Yao (2003). Nonlinear Time Series: Nonparametric and Parametric Meth-
ods. Springer-Verlag, New York.
Franses, P.H. (1996). Periodicity and Stochastic Trends in Economic Time Series. New
York: Cambridge University Press.
Franses, P.H. (1998). Time Series Models for Business and Economic Forecasting. New
York: Cambridge University Press.
Franses, P.H. and D. van Dijk (2000). Nonlinear Time Series Models for Empirical Finance.
New York: Cambridge University Press.
Ghysels, E. and D.R. Osborn (2001). The Econometric Analysis of Seasonal Time Series.
New York: Cambridge University Press.
Granger, C.W.J. and T. Terasvirta (1993). Modeling Nonlinear Economic Relationships.
Oxford, U.K.: Oxford University Press.
Hamilton, J.D. (1994). Time Series Analysis. Princeton University Press, NJ.
Liu, L.M. (1991). Dynamic relationship analysis of U.S. gasoline and crude oil prices.
Journal of Forecasting, 10, 521-547.
Sercu, P. and R. Uppal (2000). Exchange Rate Volatility, Trade, and Capital Flows under
Alternative Rate Regimes. Cambridge: Cambridge University Press.
Shumway, R.H. (1988). Applied Statistical Time Series Analysis. Englewood Clis, NJ:
Prentice-Hall.
Shumway, R.H. and D.S. Stoer (2000). Time Series Analysis & Its Applications. New
York: Springer-Verlag.
Shumway,R.H. and K.L. Verosub (1992). State space modeling of Paleoclimatic time se-
ries. Proceeding of 5th International Meeting on Statistical Climatology, Toronto, 22-26
June, 1992.
Taylor, S.(2005). Asset Price Dynamics, Volatility, and Prediction. Princeton University
Press, Princeton, NJ.
CHAPTER 2. CHARACTERISTICS OF TIME SERIES 68
Tong, H. (1990). Nonlinear Time Series: A Dynamical System Approach. Oxford Univer-
sity Press, Oxford.
Tsay, R.S. (2005). Analysis of Financial Time Series, 2th Edition. John Wiley & Sons,
New York.
Chapter 3
Univariate Time Series Models
3.1 Introduction
Thc oranization ot this chajtcr is jattcrncd attcr thc andmark aj-
jroach to dcvcojin modcs tor timc scrics data jionccrcd y Lox
and 1cnkins (scc Lox, et al, 199!) This assumcs that thcrc wi c
a rcjrcscntation ot timc scrics data in tcrms ot a dicrcncc cqua-
tion that rcatcs thc currcnt vauc to its jast Such modcs shoud c
cxic cnouh to incudc non-stationary rcaizations ikc thc random
wak ivcn aovc and scasona chavior, whcrc thc currcnt vauc is
rcatcd to jast vaucs at mutijcs ot an undcryin scason. a com-
mon onc miht c mutijcs ot 12 months (1 ycar) tor monthy data
Thc modcs arc constructcd trom dicrcncc cquations drivcn y ran-
dom injut shocks and arc accd in thc most cncra tormuation
as ARMA (Autorcrcssivc `ovin Avcrac) modc or a morc cn-
cra modc ARIMA, ic, Autoregressive Integrated Moving
Average jroccsscs Thc anaoics with dicrcntia cquations, which
modc many jhysica jroccsscs, arc ovious
Ior carity, wc dcvcoj thc scjaratc comjoncnts ot thc modc sc-
qucntiay, considcrin thc intcratcd, autorcrcssivc, and movin
avcrac in ordcr, toowcd y thc scasona modication Thc Lox-
1cnkins ajjroach sucsts thrcc stcjs in a jroccdurc that thcy sum-
69
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 70
marizc as identication, estimation, and forecasting ldcn-
tication uscs modc sccction tcchniqucs, cominin thc ACI and
lACI as dianostics with thc vcrsions ot thc Akaikc lntormation
critcrion (AlC) tyjc modc sccction critcria ivcn cow to nd a
jarsimonious (simjc) modc tor thc data Lstimation ot jaramctcrs
in thc modc wi c thc ncxt stcj Statistica tcchniqucs ascd on
maximum ikcihood and cast squarcs arc jaramount tor this stac
and wi ony c skctchcd in this coursc Lojctuy, wc can discuss
thcm in a rcat cnth it timc jcrmits Iinay, torccastin ot timc
scrics ascd on thc cstimatcd jaramctcrs, with scnsic cstimatcs ot
unccrtainty, is thc ottom inc, tor any assumcd modc
Correlation and Autocorrelation
Thc corrcation coccicnt ctwccn two random variacs x
t
and y
t
is dcncd as
xy
(0), which is thc sjccia casc ot thc cross corrcation
coccicnt
xy
(h) dcncd in Chajtcr 2 Thc corrcation coccicnt
ctwccn x
t
and x
t+h
is cacd thc a h autocorrcation ot x
t
and is
commony dcnotcd y
x
(h), which is undcr thc wcak stationarity
assumjtion Thc dcnition ot
x
(h) is ivcn in Chajtcr 2 Thc
samjc vcrsion ot
x
(h) is ivcn y

x
(h)

x
(h)/

x
(0), whcrc, tor
ivcn data {x
t
}
n
t=1
,

x
(h)
1
n h
nh

t=1
(x
t+h
x)(x
t
x) with x
1
n
n

t=1
x
t
.
ndcr somc cncra conditions,

x
(1) is a consistcnt cstimatc ot
x
(1)
Ior cxamjc, it {x
t
} is an indcjcndcnt and idcnticay distriutcd
(iid) scqucncc and E(x
2
t
) < , thcn

x
(1) is asymjtoticay norma
with mcan zcro and variancc 1/n. scc Lrockwc and Lavis (1991,
Thcorcm 22) This rcsut can c uscd in jracticc to tcst thc
nu hyjothcsis H
0

x
(1) 0 vcrsus thc atcrnativc hyjothcsis
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 71
H
a

x
(1) 0 Thc tcst v statistic is thc usua t-ratio, which is

x
(1) and toows asymjtoticay thc standard norma distriu-
tion ln cncra, tor thc a h samjc autocorrcation ot x
t
, it {x
t
} is
an iid scqucncc satistyin E(x
2
t
) < , thcn,

x
(h) is asymjtoticay
norma with mcan zcro and variancc 1/n tor any xcd jositivc in-
tccr h Ior morc intormation aout thc asymjtotic distriution ot
samjc autocorrcations, scc Lrockwc and Lavis (1991, Chajtcr )
ln nitc samjcs,

x
(h) is a iascd cstimator ot
x
(h) Thc ias is in
thc ordcr ot 1/n, which can c sustantia whcn thc samjc sizc n
is sma ln most cconomic and nancia ajjications, n is rcativcy
arc so that thc ias is not scrious
Portmanteau Test
Lconomic and nancia ajjications ottcn rcquirc to tcst ,ointy that
scvcra autocorrcations ot x
t
arc zcro Lox and licrcc (190) jro-
joscd thc lortmantcau statistic
Q

(m) n
m

h=1

2
x
(h)
as a tcst statistic tor thc nu hyjothcsis H
0

x
(1)
x
(m)
0 aainst thc atcrnativc hyjothcsis H
a

x
(i) 0 tor somc i
|1, . . . , m| ndcr thc assumjtion that {x
t
} is an iid scqucncc with
ccrtain momcnt conditions, Q

(m) is asymjtoticay a chi-squarcd

random variac with m dcrccs ot trccdom L,un and Lox (19S)
modicd thc Q

(m) statistic as cow to incrcasc thc jowcr ot thc

tcst in nitc samjcs,
Q(m) n(n + 2)
m

h=1

2
x
(h)/(n h).
ln jracticc, thc sccction ot m may acct thc jcrtormancc ot thc
Q(m) statistic Scvcra vaucs ot m arc ottcn uscd Simuation stud-
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 72
ics sucst that thc choicc ot m o(n) jrovidcs cttcr jowcr
jcrtormancc
Thc tunction

x
(h) is cacd thc samjc autocorrcation tunction
(ACI) ot x
t
lt jays an imjortant roc in incar timc scrics anaysis
As a mattcr ot tact, a incar timc scrics modc can c charactcrizcd
0 20 40 60 80 100

0
.
2

0
.
1
0
.
0
0
.
1
0
.
2
Simple Returns
IBM
0 20 40 60 80 100

0
.
2

0
.
1
0
.
0
0
.
1
0
.
2
Log Returns
IBM
0 20 40 60 80 100

0
.
2

0
.
1
0
.
0
0
.
1
0
.
2
Simple Returns
valueweighted index
0 20 40 60 80 100

0
.
2

0
.
1
0
.
0
0
.
1
0
.
2
Log Returns
valueweighted index
Figure 3.1: Autocorrelation functions (ACF) for simple (left) and log (right) returns for IBM
(top panels) and for the value-weighted index of US market (bottom panels), January 1926
to December 1997.
y its ACI, and incar timc scrics modcin makcs usc ot thc samjc
ACI to cajturc thc incar dynamic ot thc data Thc toj jancs
ot Iiurc 31 show thc samjc autocorrcation tunctions ot monthy
simjc (ctt toj janc) and o (riht toj janc) rcturns ot lL` stock
trom 1anuary 192o to Lcccmcr 199 Thc two samjc ACIs arc vcry
cosc to cach othcr, and thcy sucst that thc scria corrcations ot
monthy lL` stock rcturns arc vcry sma, it any Thc samjc ACIs
arc a within thcir two standard-crror imits, indicatin that thcy arc
not sinicant at thc `/ cvc ln addition, tor thc simjc rcturns,
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 73
thc L,unLox statistics ivc Q(`) `.! and Q(10) 1!.1, which
corrcsjond to p-vauc ot 0.3 and 0.1, rcsjcctivcy, ascd on chi-
squarcd distriutions with ` and 10 dcrccs ot trccdom Ior thc o
rcturns, wc havc Q(`) `.S and Q(10) 13. with p-vauc 0.33 and
0.19, rcsjcctivcy Thc ,oint tcsts conrm that monthy lL` stock
rcturns havc no sinicant scria corrcations Thc ottom jancs
ot Iiurc 31 show thc samc tor thc monthy rcturns (simjc in thc
ctt janc and o in thc riht janc) ot thc vauc-wcihtcd indcx
trom thc Ccntcr tor lcscarch in Sccurity lriccs (ClSl), nivcrsity
ot Chicao Thcrc arc somc sinicant scria corrcations at thc `/
cvc tor oth rcturn scrics Thc L,unLox statistics ivc Q(`)
2.S and Q(10) 3o.0 tor thc simjc rcturns and Q(`) 2o.9 and
Q(10) 32. tor thc o rcturns Thc p-vaucs ot thcsc tour tcst
statistics arc a css than 0.0003, sucstin that monthy rcturns ot
thc vauc-wcihtcd indcx arc scriay corrcatcd Thus, thc monthy
markct indcx rcturn sccms to havc stroncr scria dcjcndcncc than
individua stock rcturns
ln thc nancc itcraturc, a vcrsion ot thc Cajita Assct lricin
`odc (CAl`) thcory is that thc rcturn {x
t
} ot an assct is not
jrcdictac and shoud havc no auto-corrcations Tcstin tor zcro
autocorrcations has ccn uscd as a too to chcck thc ccicnt markct
assumjtion Lowcvcr, thc way y which stock jriccs arc dctcrmincd
and indcx rcturns arc cacuatcd miht introducc autocorrcations
in thc oscrvcd rcturn scrics This is jarticuary so in anaysis ot
hih-trcqucncy nancia data
Lctorc wc discuss univariatc and mutivariatc timc scrics mcth-
ods, wc rst rcvicw mutijc rcrcssion modcs and modc sccction
mcthods tor oth iid and timc scrics data
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 74
3.2 Least Squares Regression
\c cin our discussion ot univariatc and mutivariatc timc scrics
mcthods y considcrin thc idca ot a simjc rcrcssion modc, which
wc havc mct ctorc in othcr contcxts such as statistics or cconomct-
rics coursc A ot thc mutivariatc mcthods toow, in somc scnsc,
trom thc idcas invovcd in simjc univariatc incar rcrcssion ln this
casc, wc assumc that thcrc is somc cocction ot xcd known tunc-
tions ot timc, say z
t
1
, z
t
2
, . . . , z
t
q
that arc inucncin our outjut y
t
which wc know to c random \c cxjrcss this rcation ctwccn thc
injuts and outjuts as
y
t

1
z
t
1
+
2
z
t
2
+ +
q
z
t
q
+ e
t
(31)
at thc timc joints t 1, 2, . . . , n, whcrc
1
, . . . ,
q
arc unknown
xcd rcrcssion coccicnts and e
t
is a random crror or noisc, assumcd
to c whitc noisc. this mcans that thc oscrvations havc zcro mcans,
cqua varianccs
2
and arc indcjcndcnt \c traditionay assumc aso
that thc whitc noisc scrics, e
t
, is Gaussian or normay distriutcd
Example 2.1: \c havc assumcd imjicity that thc modc
y
t

1
+
2
t + e
t
is rcasonac in our discussion ot dctrcndin in Example 1.2 ot
Chajtcr 2 Iiurc 2! shows thc monthy avcrac oa tcmjcraturc
scrics and it is jausic that a straiht inc is a rcasonac modc
This is in thc torm ot thc rcrcssion modc 31 whcn onc makcs thc
idcntication z
t
1
1 and z
t
2
t Thc jrocm in dctrcndin is to
cstimatc thc coccicnts
1
and
2
in thc aovc cquation and dctrcnd
y constructin thc cstimatcd rcsidua scrics e
t
, which is shown in thc
toj janc ot Iiurc 2! As indicatcd in thc cxamjc, cstimatcs tor
1
and
2
can c takcn as

1
9.03 and

2
0.00!o, rcsjcctivcy
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 75
Thc incar rcrcssion modc dcscricd y Lquation 31 can c con-
vcnicnty writtcn in sihty morc cncra matrix notation y dcnin
thc coumn vcctors z
t
(z
t
1
, . . . , z
t
q
)

and (
1
, . . . ,
q
)

so
that wc writc (21) in thc atcrnatc torm
y
t

z
t
+ e
t
. (32)
To nd cstimators tor and
2
, it is natura to dctcrminc thc co-
ccicnt vcctor minimizin

e
2
t
with rcsjcct to This yicds
cast squarcs or maximum ikcihood cstimator

and thc maximum
ikcihood cstimator tor

2
which is jrojortiona to thc uniascd

1
n q
n1

t=0
_
y
t

z
t
_
2
. (33)
An atcrnatc way ot writin thc modc 32 is as
y Z + e, (3!)
whcrc Z

(z
1
, z
2
, . . . , z
n
) is a q n matrix comjoscd ot thc
vaucs ot thc injut variacs at thc oscrvcd timc joints and y
(y
1
, y
2
, . . . , y
n
) is thc vcctor ot oscrvcd outjuts with thc crrors
stackcd in thc vcctor e (e
1
, e
2
, . . . , e
n
)

Thc ordinary cast

squarcs cstimators

arc thc soutions to thc norma cquations
Z

Z Zy.
You nccd not c conccrncd as to how thc aovc cquation is sovcd in
jracticc as a comjutcr jackacs havc ccicnt sottwarc tor invcrtin
thc q q matrix Z

Z to otain

(Z

Z)
1
Zy. (3`)
An imjortant quantity that a sottwarc jroduccs is a mcasurc ot
unccrtainty tor thc cstimatcd rcrcssion coccicnts, say
Cov
_

_

2
(Z

Z)
1

2
C
2
(c
ij
). (3o)
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 76
Thcn, Cov(
i
,
j
)
2
c
ij
and a 100(1 )/ condcncc intcrva
tor

i
is

i
t
nq
(/2)

c
ii
, (3)
whcrc t
df
(/2) dcnotcs thc ujjcr 100(1 )/ joint on a t distri-
ution with df dcrccs ot trccdom
Example 2.1: Considcr cstimatin thc jossic oa warmin
trcnd audcd to in Scction 221 Thc oa tcmjcraturc scrics,
shown jrcviousy in Iiurc 2! sucsts thc jossiiity ot a raduay
incrcasin avcrac tcmjcraturc ovcr thc 1!9 ycar jcriod covcrcd y
thc and ascd scrics lt wc t thc modc in Example 2.1, rcjacin
t y t/100 to convcrt to a 100 ycar asc so that thc incrcasc wi c
in dcrccs jcr 100 ycars, wc otain

1
9.03 and

2
0.!o0
usin (3`) thc crror variancc, trom (33), is 0.033, with q 2 and
n 190 Thcn, (3o) yicds
Cov(

1
,

2
)
_
_
0.039 0.0020
0.0020 0.0001
_
_
cadin to an cstimatcd standard crror ot

0.001 0.0100S Thc

vauc ot t with n q 190 2 1SS dcrccs ot trccdom tor
0.02` is aout 1.9o, cadin to a narrow condcncc intcrva ot
0.!o0 0.019S tor thc sojc cadin to a condcncc intcrva on thc
onc hundrcd ycar incrcasc ot aout 0.!!09 to 0.!S0` dcrccs \c
woud concudc trom this anaysis that thcrc is a sustantia incrcasc
in oa tcmjcraturc amountin to an incrcasc ot rouhy onc dcrcc
I jcr 100 ycars
lt thc modc is rcasonac, thc rcsiduas e
t
y
t

2
t shoud
c csscntiay indcjcndcnt and idcnticay distriutcd with no corrc-
ation cvidcnt Thc jot that wc havc madc in Iiurc 2` (thc toj
janc) ot thc dctrcndcd oa tcmjcraturc scrics shows that this is
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 77
0 5 10 15 20

0
.
5
0
.
0
0
.
5
1
.
0
Lag
Detrended Temperature
ACF
5 10 15 20

0
.
5
0
.
0
0
.
5
1
.
0
Lag
PACF
0 5 10 15 20

0
.
5
0
.
0
0
.
5
1
.
0
Lag
Differenced Temperature
ACF
5 10 15 20

0
.
5
0
.
0
0
.
5
1
.
0
Lag
PACF
Figure 3.2: Autocorrelation functions (ACF) and partial autocorrelation functions (PACF)
for the detrended (top panel) and dierenced (bottom panel) global temperature series.
jroay not thc casc ccausc ot thc on ow trcqucncy in thc o-
scrvcd rcsiduas Lowcvcr, thc dicrcnccd scrics, aso shown in Iiurc
2o, ajjcars to c morc indcjcndcnt sucstin that jcrhajs thc aj-
jarcnt oa warmin is morc consistcnt with a on tcrm swin in
an undcryin random wak than it is ot a xcd 100 ycar trcnd lt wc
chcck thc autocorrcation tunction ot thc rcrcssion rcsiduas, shown
hcrc in Iiurc 32, it is ccar that thc sinicant vaucs at hihcr
as imjy that thcrc is sinicant corrcation in thc rcsiduas Such
corrcation can c imjortant sincc thc cstimatcd standard crrors ot
thc coccicnts undcr thc assumjtion that thc cast squarcs rcsidu-
as arc uncorrcatcd is ottcn too sma \c can jartiay rcjair thc
damac causcd y thc corrcatcd rcsiduas y ookin at a modc with
corrcatcd crrors Thc jroccdurc and tcchniqucs tor dcain with cor-
rcatcd crrors arc ascd on thc autorcrcssivc movin avcrac modcs
to c considcrcd in thc ncxt scctions Anothcr mcthod ot rcducin
corrcation is to ajjy a rst dicrcncc x
t
x
t
x
t1
to thc oa
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 78
trcnd data Thc ACI ot thc dicrcnccd scrics, aso shown in Iiurc
32, sccms to havc owcr corrcations at thc hihcr as Iiurc 2o
shows quaitativcy that this transtormation aso ciminatcs thc trcnd
in thc oriina scrics
Sincc wc havc aain madc somc rathcr aritrary ookin sjcc-
ications tor thc conuration ot dcjcndcnt variacs in thc aovc
rcrcssion cxamjcs, thc rcadcr may wondcr how to sccct amon var-
ious jausic modcs \c mcntion that two critcria which rcward
rcducin thc squarcd crror and jcnaizc tor additiona jaramctcrs
arc thc Akaikc lntormation Critcrion and thc Schwarz lntormation
Critcrion (SlC) (Schwarz, 19S) with a common torm
o(

2
) + C(K) /n, (3S)
whcrc K is thc numcr ot jaramctcrs ttcd (cxcusivc ot variancc
jaramctcrs),

2
is thc maximum ikcihood cstimator tor thc variancc,
and C(K) 2 K tor AlC and K o(n) tor SlC SlC is somctimcs
tcrmcd thc Laycsian lntormation Critcrion, LlC and wi ottcn yicd
modcs with tcwcr jaramctcrs than thc othcr sccction mcthods A
modication to AlC that is jarticuary wc suitcd tor sma samjcs
was sucstcd y Lurvich and Tsai (19S9) This is thc corrcctcd
AlC, cacd AlCC ivcn y o(

2
) +(n+K)/(nK2) Thc ruc
tor a thrcc mcasurcs aovc is to choosc thc vauc ot K cadin to
thc smacst vauc ot AlC or SlC or AlCC \c wi ivc an cxamjc
atcr comjarin thc aovc simjc cast squarcs modc with a modc
whcrc thc crrors havc a timc scrics corrcation structurc A summary
ot modc sccction mcthods is ivcn in thc ncxt scction `otc that
a mcthods arc cncra jurjoscs
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 79
3.3 Model Selection Methods
Givcn a jossiy arc sct ot jotcntia jrcdictors, which oncs do wc
incudc in our modc Sujjosc |X
1
, X
2
, | is a pool ot jotcntia
jrcdictors Thc modc with a jrcdictors,
Y
0
+
1
X
1
+
2
X
2
+ + ,
is thc most cncra modc lt hods cvcn it somc ot thc individua
j
s
arc zcro Lut it somc
j
s zcro or cosc to zcro, it is cttcr to omit
thosc X
j
s trom thc modc lcasons why you shoud omit variacs
whosc coccicnts arc cosc to zcro
(a) Parsimony principle: Givcn two modcs that jcrtorm cquay
wc in tcrms ot jrcdiction, onc shoud choosc thc modc that is
morc jarsimonious (simjc)
() Prediction principle: Thc modc shoud ivc jrcdictions
that arc as accuratc as jossic, not ,ust tor currcnt oscrvation,
ut tor tuturc oscrvations as wc lncudin unncccssary jrc-
dictors can ajjarcnty imjrovc jrcdiction tor thc currcnt data,
ut can harm jrcdiction tor tuturc data `otc that thc sum ot
squarcd crrors (SSL) ncvcr incrcascs as wc add morc jrcdictors
Thcrctorc, whcn wc uid a statistica modc, wc shoud toow thcsc
jrincijcs
3.3.1 Subset Approaches
Thc a-jossic-rcrcssions jroccdurc cas tor considcrin a jos-
sic suscts ot thc joo ot jotcntia jrcdictors and idcntityin tor
dctaicd cxamination a tcw ood su-scts accordin to somc critc-
rion Thc jurjosc ot a-jossic-rcrcssions ajjroach is idcntityin
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 80
a sma rouj ot rcrcssion modcs that arc good accordin to a sjcc-
icd critcrion (summary statistic) so that a dctaicd cxamination can
c madc ot thcsc modcs cadin to thc sccction ot thc na rcrcs-
sion modc to c cmjoycd Thc main jrocm ot this ajjroach is
comjutationay cxjcnsivc Ior cxamjc, with k 10 jrcdictors, wc
nccd to invcstiatc 2
10
102! jotcntia rcrcssion modcs \ith thc
aid ot modcrn comjutin jowcr, this comjutation is jossic Lut
sti thc numcr ot 102! jossic modcs to cxaminc carctuy woud
c an ovcrwhcmin task tor a data anayst
Licrcnt critcria tor comjarin thc rcrcssion modcs may c uscd
with thc a-jossic-rcrcssions sccction jroccdurc \c discuss scv-
cra summary statistics
(i) R
2
p
(or SSL
p
), (ii) R
2
adj;p
(or `SL
p
), (iii) C
p
, (iv)
llLSS
p
,
(v) Scqucntia `cthods, and (vi) AlC tyjc critcria
\c sha dcnotc thc numcr ot a jotcntia jrcdictors in thc joo
y P 1 Lcncc incudin an intcrccjt jaramctcr
0
, wc havc P
jotcntia jaramctcrs Thc numcr ot jrcdictors in a susct wi
c dcnotcd y p 1, as aways, so that thcrc arc p jaramctcrs in
thc rcrcssion tunction tor this susct ot jrcdictors Thus wc havc
1 p P
1. R
2
p
(or SSL
p
) R
2
p
indicatcs that thcrc arc p jaramctcrs (or, p1
jrcdictors) in thc rcrcssion modc Thc coccicnt ot mutijc
dctcrmination R
2
p
is dcncd as
R
2
p
1 SSE
p
/SSTO,
whcrc SSL
p
is thc sum ot squarcd crrors ot thc modc incudin a
p1 jrcdictors and SSTO is thc sum ot squarcd tota variations
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 81
lt is wc known that R
2
p
mcasurcs thc jrojortion ot variancc ot
Y cxjaincd y p 1 jrcdictors, it aways ocs uj as wc add
a jrcdictor, and it varics invcrscy with SSL
p
ccausc SSTO is
constant tor a jossic rcrcssion modcs That is, choosin thc
modc with thc arcst R
2
p
is cquivacnt to choosin thc modc
with smacst SSL
p

2. R
2
adj;p
(or `SL
p
) Onc ottcn considcrs modcs with a arc R
2
p
vauc Lowcvcr, R
2
p
aways incrcascs with thc numcr ot jrcdic-
tors Lcncc it can not c uscd to comjarc modcs with dicrcnt
sizcs Thc ad,ustcd coccicnt ot mutijc dctcrmination R
2
adj;p
has ccn sucstcd as an atcrnativc critcrion
R
2
adj;p
1
SSE
p
/(n p)
SSTO/(n 1)
1
_
_
n 1
n p
_
_
SSE
p
SSTO
1
MSE
p
SSTO/(n 1)
.
lt is ikc R
2
p
ut with a jcnaty tor addin unncccssary variacs
R
2
adj,p
can o down whcn a usccss jrcdictor is addcd and it
can c cvcn ncativc R
2
adj;p
varics invcrscy with `SL
p
ccausc
SSTO/(n 1) is constant tor a jossic rcrcssion modcs
That is, choosin thc modc with thc arcst R
2
adj;p
is cquivacnt
to choosin thc modc with smacst `SL
p
`otc that R
2
p
is
usctu whcn comjarin modcs ot thc samc sizc, whic R
2
adj;p
(or
C
p
) is uscd to comjarc modcs with dicrcnt sizcs
3. `aows C
p
Thc `aows C
p
is conccrncd with thc tota mcan
squarcd crror ot thc n ttcd vaucs tor cach susct rcrcssion
modc Thc mcan squarcd crror conccjt invovcs thc tota crror
in cach ttcd vauc

Y
i

i


Y
i
E(

Y
i
)
. .
random error
+E(

Y
i
)
i
. .
bias
,
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 82
whcrc
i
is thc truc mcan rcsjonsc at ith oscrvation Thc
mcans squarcd crror tor

Y
i
is dcncd as thc cxjcctcd vauc ot
thc squarc ot thc tota crror in thc aovc lt can c shown that
mse(

Y
i
) E
_
(

Y
i

i
)
2
_
V ar(

Y
i
) +
_
Bias(

Y
i
)
_
2
,
whcrc Lias(

Y
i
) E(

Y
i
)
i
Thc tota mcan squarc crror tor
a n ttcd vaucs

Y
i
is thc sum ovcr thc oscrvation i
n

i=1
mse(

Y
i
)
n

i=1
V ar(

Y
i
) +
n

i=1
_
Bias(

Y
i
)
_
2
.
lt can c shown that
n

i=1
V ar(

Y
i
) p
2
and
n

i=1
_
Bias(

Y
i
)
_
2
(np)|E(S
2
p
)
2
|,
whcrc S
2
p
is thc `SL trom thc currcnt modc sin this, wc
havc
n

i=1
mse(

Y
i
) p
2
+ (n p)|E(S
2
p
)
2
|, (39)
Lividin (39) y
2
, wc makc it scac-trcc
n

i=1
mse(

Y
i
)

2
p + (n p)
E(S
2
p
)
2

2
,
lt thc modc docs not t wc, thcn S
2
p
is a iascd cstimatc ot
2

\c can cstimatc E(S

2
p
) y `SL
p
and cstimatc
2
y thc `SL
trom thc maxima modc (thc arcst modc wc can considcr), ic,

2
MSE
P1
MSE(X
1
, . . . , X
P1
) sin thc cstimators
tor E(S
2
p
) and
2
ivcs
C
p
p+(np)
MSE
p
MSE(X
1
, . . . , X
P1
)
MSE(X
1
, . . . , X
P1
)

SSE
p
MSE(X
1
, . . . , X
P1
)

Sma C
p
is a ood thin A sma vauc ot C
p
indicatcs that
thc modc is rcativcy jrccisc (has sma variancc) in cstimatin
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 83
thc truc rcrcssion coccicnts and jrcdictin tuturc rcsjonscs
This jrccision wi not imjrovc much y addin morc jrcdictors
Look tor modcs with sma C
p
lt wc havc cnouh jrcdictors in
thc rcrcssion modc so that a thc sinicant jrcdictors arc
incudcd, thcn `SL
p
MSE(X
1
, . . . , X
P1
) and it toows
that C
p
p Thus C
p
cosc to p is cvidcncc that thc jrcdictors
in thc joo ot jotcntia jrcdictors (X
1
, . . . , X
P1
) ut not in
thc currcnt modc, arc not imjortant `odcs with considcrac
ack ot t havc vaucs ot C
p
arcr than p Thc C
p
can c uscd to
comjarc modcs with dicrcnt sizcs lt wc usc a thc jotcntia
jrcdictors, thcn C
p
P
4. llLSS
p
Thc llLSS (jrcdiction sum ot squarcs) is dcncd as
PRESS
n

i=1

2
(i)
,
whcrc

(i)
is cacd llLSS (jrcdiction sum ot squarcs) rcsidua
tor thc thc ith oscrvation Thc llLSS rcsidua is dcncd as

(i)
Y
i

Y
(i)
, whcrc

Y
(i)
is thc ttcd vauc otaincd y cavin
thc ith oscrvation `odcs with sma llLSS
p
t wc in thc
scnsc ot havin sma jrcdiction crrors llLSS
p
can c cacu-
atcd without ttin thc modc n timcs, cach timc dcctin onc
ot thc n cascs Onc can show that

(i)

i
/(1 h
ii
),
whcrc h
ii
is thc ith diaona ccmcnt ot H X(X

X)
1
X

3.3.2 Sequential Methods

1. Forward selection
(a) Start with thc null modc
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 84
() Add thc sinicant variac it p-vauc is css than p
enter
, (cquiv-
acnty, F is arcr than F
enter
)
(c) Continuc unti no morc variacs cntcr thc modc
2. Backward elimination
(a) Start with thc full modc
() Liminatc thc cast sinicant variac whosc p-vauc is arcr
than p
remove
, (cquivacnty, F is smacr than F
remove
)
(c) Continuc unti no morc variacs can c discardcd trom thc
modc
3. Stepwise selection
(a) Start with any modc
() Chcck cach jrcdictor that is currcnty in thc modc Sujjosc
thc currcnt modc contains X
1
, . . . , X
k
Thcn F statistic tor X
i
is
F
SSE(X
1
, . . . , X
i1
, X
i+1
, . . . , X
k
) SSE(X
1
, . . . , X
k
)
MSE(X
1
, . . . , X
k
)
F(1. nk
Liminatc thc cast sinicant variac whosc p-vauc is arcr
than p
remove
, (cquivacnty, I is smacr than F
remove
)
(c) Continuc unti no morc variacs can c discardcd trom thc
modc
(d) Add thc sinicant variac it p-vauc is css than p
enter
, (cquiv-
acnty, F is arcr than F
enter
)
(c) Go to stcj (ii)
(t) lcjcat unti no morc jrcdictors can c cntcrcd and no morc can
c discardcd
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 85
3.3.3 Likelihood Based-Criteria
Thc asic idca ot Akaikcs and aikc ajjroachcs can c tound in
Akaikc (193) and suscqucnt jajcrs. scc thc rcccnt ook y Lurn-
ham and Andcrson (2003)
Sujjosc that f(y) truc modc (unknown) ivin risc to data (is a
vcctor ot data) and g(y, ) candidatc modc (jaramctcr vcctor) \c
want to nd a modc g(y, ) cosc to f(y) Thc Iuack-Lcicr
discrcjancy
K(f, g) E
f
_

_o
_
_
_
f(Y )
g(Y, )
_
_
_
_

_ .
This is a mcasurc ot how tar modc g is trom modc f (with rctcr-
cncc to modc f) lrojcrtics
K(f, g) , 0 K(f, g) 0 f g.
Ot coursc, wc can ncvcr know how tar our modc g is trom f Lut
Akaikc (193) showcd that wc miht c ac to cstimatc somcthin
amost as ood
Sujjosc wc havc two modcs undcr considcration g(y, ) and
h(y, ) Akaikc (193) showcd that wc can estimate
K(f, g) K(f, h).
lt turns out that thc dicrcncc ot maximizcd o-ikcihoods, cor-
rcctcd tor a ias, cstimatcs thc dicrcncc ot I-L distanccs Thc
maximizcd ikcihoods arc,

L
g
g(y,

) and

L
h
(y,

), whcrc

and

arc thc `L cstimatcs ot thc jaramctcrs Akaikcs rcsut |o(

L
g
)
q| |o(

L
h
) r| is an asymjtoticay uniascd cstimatc (ic ias
ajjroachcs zcro as samjc sizc incrcascs) ot K(f, g) K(f, h) Lcrc
q is thc numcr ot jaramctcrs cstimatcd in (modc g) and r is
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 86
thc numcr ot jaramctcrs cstimatcd in (modc h) Thc jricc ot
jaramctcrs thc ikcihoods in thc aovc cxjrcssion arc jcnaizcd y
thc numcr ot jaramctcrs
Thc AlC tor modc g is ivcn y
AIC 2 o(

L
g
) + 2 q.
Thc AlC miht not jcrtorm wc tor thc sma samjc sizc casc To
ovcrcomc this shortcomin, a iascd corrcction vcrsion ot AlC was
jrojoscd y Lurvich and Tsai (19S9), dcncd y
AICC 2 o(

L
g
)+2 (q+1)/(nq2) AIC+2 (q+1)(q+2)/(nq2).
Thc AlCC is in thc ctwccn thc AlC (css jcnaty) and thc LlC
(hcavy jcnaty)
Anothcr ajjroach is ivcn y thc much odcr notion ot Laycsian
statistics ln thc Laycsian ajjroach, wc assumc that a jriori unccr-
tainty aout thc vauc ot modc jaramctcrs is rcjrcscntcd y a jrior
distriution jon oscrvin thc data, this jrior is ujdatcd, yicdin
a jostcrior distriution ln ordcr to makc intcrcnccs aout thc modc
(rathcr than its jaramctcrs), wc intcratc across thc jostcrior dis-
triution ndcr thc assumjtion that a modcs arc a jriori cquay
ikcy (ccausc thc Laycsian ajjroach rcquircs modc jriors as wc as
jaramctcr jriors), Laycsian modc sccction chooscs thc modc with
hihcst marina ikcihood Thc ratio ot two marina ikcihoods is
cacd a Laycs tactor (LI), which is a widcy uscd mcthod ot modc
sccction in Laycsian intcrcncc Thc two intcras in thc Laycs tac-
tor arc nontrivia to comjutc uncss thcy torm a con,uatcd tamiy
`ontc Caro mcthods arc usuay rcquircd to comjutc LI, csjcciay
tor hihy jaramctcrizcd modcs A arc samjc ajjroximation ot
LI yicds thc casiy comjutac LlC
BIC 2 o(

L
g
) + q o n.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 87
ln a sum, oth AlC and LlC as wc as thcir cncraizations havc
a simiar torm as
LC 2 o(

L
g
) + q,
whcrc is xcd constant Thc rcccnt dcvcojmcnts sucst thc usc
ot a data adajtivc jcnaty to rcjacc thc xcd jcnatics Scc, Lai,
lao and \u (1999) and Shcn and Yc (2002) That is to cstimatc
y data in a comjcxity torm ascd on a conccjt ot cncraizcd
dcrcc ot trccdom
3.3.4 Cross-Validation and Generalized Cross-Validation
Thc cross vaidation (C\) is thc most commony uscd mcthod tor
modc asscssmcnt and sccction Thc main idca is a dircct cstimatc
ot cxtra-samjc crror Thc cncra vcrsion ot C\ is to sjit data into
I rouhy cqua-sizcd jarts and to t thc modc to thc othcr K 1
jarts and cacuatc jrcdiction crror on thc rcmainin jart
CV
n

i=1
(Y
i

Y
i
)
2
whcrc

Y
i
is thc ttcd vauc comjutcd with k-th jart ot data rc-
movcd
A convcnicnt ajjroximation to C\ tor incar ttin with squarcd
crror oss is cncraizcd cross vaidation (GC\) A incar ttin mcthod
has thc toowin jrojcrty

Y S Y , whcrc

Y
i
is thc ttcd vauc
with thc whoc data Ior many incar ttin mcthods with cavc-
onc-out (k 1), it can c showcd casiy that
CV
n

i=1
(Y
i

Y
i
)
2

i=1
_
_
_
Y
i

Y
i
1 S
ii
_
_
_
2
.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 88
Luc to thc intcnsivc comjutation, thc C\ can c ajjroximatcd y
thc GC\, dcncd y
GCV
n

i=1
_
_
_
Y
i

Y
i
1 tracc(S)/n
_
_
_
2

n
i=1
(Y
i

Y
i
)
2
(1 tracc(S)/n)
2
.
lt has ccn shown that oth thc C\ and GC\ mcthod arc vcry
ajjain to nonjaramctric modcin
lcccnty, thc cavc-onc-out cross-vaidation mcthod was chacncd
y Shao (1993) Shao (1993) caimcd that thc jojuar cavc-onc-out
cross-vaidation mcthod, which is asymjtoticay cquivacnt to many
othcr modc sccction mcthods such as thc AlC, thc C
p
, and thc
ootstraj, is asymjtoticay inconsistcnt in thc scnsc that thc jroa-
iity ot sccctin thc modc with thc cst jrcdictivc aiity docs not
convcrc to 1 as thc tota numcr ot oscrvations n and hc
showcd that thc inconsistcncy ot thc cavc-onc-out cross-vaidation
can c rccticd y usin a cavc-n

-out cross-vaidation with n

, thc
numcr ot oscrvations rcscrvcd tor vaidation, satistyin n

/n 1
as n
3.3.5 Penalized Methods
1. Bridge and Ridge: Irank and Iricdman (1993) jrojoscd thc
L
q
(q > 0) jcnaizcd cast squarcs as
n

i=1
(Y
i

j
X
ij
)
2
+

j
|
j
|
q
,
which rcsuts in thc cstimator which is cacd thc ridc cstima-
tor lt q 2, thc rcsutin cstimator is cacd thc ridc cstimator
ivcn y

(X
T
X + I)
1
X
T
Y
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 89
2. LASSO: Tishirani (199o) jrojoscd thc so-cacd LASSO which
is thc minimizcr ot thc toowin constraincd cast squarcs
n

i=1
(Y
i

j
X
ij
)
2
+

j
|
j
|,
which rcsuts in thc sott thrcshin ruc

j
sin(

0
j
)(|

0
j
| )
+

3. Non-concave Penalized LS: Ian and Li (2001) jrojoscd

thc non-concavc jcnaizcd cast squarcs
n

i=1
(Y
i

j
X
ij
)
2
+

j
p

(|
j
|),
whcrc thc hard thrcshin jcnaty tunction p

(||)
2
(||
)
2
|(|| < ), which rcsuts in thc hard thrcshin ruc

0
j
I(|

0
j
| > ) Iinay, Ian and Li (2001) jrojoscd thc so-
cacd thc smoothy cijjcd asoutc dcviation (SCAL) modc
sccction critcrion with thc jcnaizcd tunction dcncd as
p

()
_

_
I( )
(a )
+
(a 1)
I( > )
_

_
tor somc a > 2 and
which rcsuts in thc cstimator

j

_

_
sin(

0
j
)(|

0
j
| )
+
whcn |

0
j
| 2,
_
(a 1)

0
j
sin(

0
j
) a
_
/(a 2) whcn 2 |

0
j
| a ,

0
j
whcn |

0
j
| > a
Aso, Ian and Li (2001) showcd that thc SCAL cstimator sat-
iscs thrcc jrojcrtics (1) uniascdncss, (2) sjarsity, and (3)
continuity and Ian and lcn (200!) considcrcd thc casc that
thc numcr ot rcrcssors can dcjcnd on thc samjc sizc and
ocs to innity in a ccrtain ratc
Remark: `otc that thc thcory tor thc jcnaizcd mcthods is sti
ojcn tor timc scrics data and it woud c a vcry intcrcstin rcscarch
tojic
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 90
3.4 Integrated Models - I(1)
\c cin our study ot timc corrcation y mcntionin a simjc modc
that wi introducc stron corrcations ovcr timc This is thc ran-
dom walk or unit root modc which dcncs thc currcnt vauc ot
thc timc scrics as ,ust thc immcdiatcy jrcccdin vauc with additivc
noisc Thc modc torms thc asis, tor cxamjc, ot thc random wak
thcory ot stock jricc chavior ln this modc wc dcnc
x
t
x
t1
+ w
t
, (310)
whcrc w
t
is a whitc noisc scrics with mcan zcro and variancc
2
Thc
ctt janc ot Iiurc 33 shows a tyjica rcaization ot such a scrics
(w
t
N(0, 1)) and wc oscrvc that it cars a jassin rcscmancc
to thc oa tcmjcraturc scrics Ajjcain to (310), thc cst jrc-
diction ot thc currcnt vauc woud c cxjcctcd to c ivcn y its
immcdiatcy jrcccdin vauc Thc modc is, in a scnsc, unsatistac-
tory, ccausc onc woud think that cttcr rcsuts woud c jossic
y a morc ccicnt usc ot thc jast Thc ACI ot thc oriina scrics,
shown in Iiurc 3!, cxhiits a sow dccay as as incrcasc ln ordcr
to modc such a scrics without knowin that it is ncccssariy cncr-
atcd y (310), onc miht try ookin at a rst dicrcncc, shown in
thc riht janc ot Iiurc 33, and comjarin thc rcsut to a whitc
noisc or comjctcy indcjcndcnt jroccss lt is ccar trom (310) that
thc rst dicrcncc woud c x
t
x
t
x
t1
w
t
which is ,ust
whitc noisc Thc ACI ot thc dicrcnccd jroccss, in this casc, woud
c cxjcctcd to c zcro at a as h 0 and thc samjc ACI shoud
rccct this chavior Thc rst dicrcncc ot thc random wak in thc
riht janc ot Iiurc 33 is aso shown in thc ottom jancs ot Iiurc
3! and wc notc that it ajjcars to c much morc random Thc ACI,
shown in thc ctt ottom janc ot Iiurc 3!, rcccts this jrcdictcd
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 91
0 50 100 150 200
0
5
1
0 Random Walk
0 50 100 150 200

1
0
1
2
First Difference
Figure 3.3: A typical realization of the random walk series (left panel) and the rst dierence
of the series (right panel).
chavior, with no sinicant vaucs tor as othcr than zcro lt is
ccar that (310) is a rcasonac modc tor this data Thc oriina sc-
rics is nonstationary, with an autocorrcation tunction that dcjcnds
on timc ot thc torm
(x
t+h
, x
t
)
_

_
_
1/(t + h), it h 0,
_
(t + h)/t, it h < 0
Thc aovc cxamjc, usin a dicrcncc transtormation to makc a
random wak stationary, shows a vcry jarticuar casc ot thc modc
idcntication jroccdurc advocatcd y Lox, et al (199!) `amcy,
wc scck a incary tcrcd transtormation ot thc oriina scrics, ascd
stricty on thc jast vaucs, that wi rcducc it to comjctcy random
whitc noisc This ivcs a modc that cnacs jrcdiction to c donc
with a rcsidua noisc that satiscs thc usua statistica assumjtions
aout modc crror
\c wi introducc, in thc toowin discussion, morc cncra vcr-
sions ot this simjc modc that arc usctu tor modcin and torccast-
in scrics with oscrvations that arc corrcatcd in timc Thc notation
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 92
0 5 10 15 20

0
.
5
0
.
0
0
.
5
1
.
0
Random Walk
ACF
5 10 15 20

0
.
5
0
.
0
0
.
5
1
.
0
lag
PACF
0 5 10 15 20

0
.
5
0
.
0
0
.
5
1
.
0
First Difference
ACF
5 10 15 20

0
.
5
0
.
0
0
.
5
1
.
0
lag
PACF
Figure 3.4: Autocorrelation functions (ACF) (left) and partial autocorrelation functions
(PACF) (right) for the random walk (top panel) and the rst dierence (bottom panel)
series.
and tcrminooy wcrc introduccd in thc andmark work y Lox and
1cnkins (190) A rcquircmcnt tor thc Al`A modc ot Lox and
1cnkins is that thc undcryin jroccss c stationary Ccary thc rst
dicrcncc ot thc random wak is stationary ut thc ACI ot thc rst
dicrcncc shows rcativcy ittc dcjcndcncc on thc jast, mcanin
that thc dicrcnccd jroccss is not jrcdictac in tcrms ot its jast
chavior
To introducc a notation that has advantacs tor trcatin morc
cncra modcs, dcnc thc back-shift operator L as thc rcsut ot
shittin thc scrics ack y onc timc unit, ic
Lx
t
x
t1
(311)
and ajjyin succcssivcy hihcr jowcrs, L
k
x
t
x
tk
Thc ojcrator
has many ot thc usua acraic jrojcrtics and aows, tor cxamjc,
writin thc random wak modc (310) as (1 L) x
t
w
t
`otc that
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 93
thc dicrcncc ojcrator discusscd jrcviousy in 122 is ,ust 1L
ldcntityin nonstationarity is an imjortant rst stcj in thc Lox-
1cnkins jroccdurc Irom thc aovc discussion, wc notc that thc
ACI ot a nonstationary jroccss wi tcnd to dccay rathcr sowy as
a tunction ot a h Ior cxamjc, a straihty inc woud c jcrtccty
corrcatcd, rcardcss ot a Lascd on this oscrvation, wc mcntion
thc toowin jrojcrtics that aid in idcntityin non-stationarity
Property 2.1: The ACF of a non-stationary time series decays
very slowly as a function of lag h. The PACF of a non-stationary
time series tends to have a peak very near unity at lag 1, with
other values less than the signicance level.
`otc that sincc l(1) modc is vcry imjortant in modcin cco-
nomic and nancia data, wc wi discuss morc on thc modc and thc
statistica intcrcncc in thc atcr chajtcr
3.5 Autoregressive Models - AR(p)
3.5.1 Model
`ow, cxtcndin thc notions aovc to morc cncra incar comina-
tions ot jast vaucs miht sucst writin
x
t

1
x
t1
+
2
x
t2
+ +
p
x
tp
+ w
t
(312)
as a tunction ot p jast vaucs and an additivc noisc comjoncnt w
t

Thc modc ivcn y (312) is cacd an autorcrcssivc modc ot ordcr

p, sincc it is assumcd that onc nccds p jast vaucs to jrcdict x
t
Thc
coccicnts
1
,
2
, ,
p
arc autorcrcssivc coccicnts, choscn to
jroducc a ood t ctwccn thc oscrvcd x
t
and its jrcdiction ascd
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 94
on x
t1
, x
t2
, , x
tp
lt is convcnicnt to rcwritc (312), usin thc
ack-shitt ojcrator, as
(L) x
t
w
t
, whcrc (L) 1
1
L
2
L
2

p
L
p
,
(313)
is a joynomia with roots (soutions ot (L) 0) outsidc thc unit
circc (|L
j
| > 1)
1
Thc rcstrictions arc ncccssary tor cxjrcssin thc
soution x
t
ot (313) in tcrms ot jrcscnt and jast vaucs ot w
t
, which
is cacd invertibility ot an Al`A scrics That soution has thc
torm
x
t
(L) w
t
, whcrc (L)

k=0

k
L
k
, (31!)
is an innitc joynomia (
0
1), with coccicnts dctcrmincd y
cquatin coccicnts ot B in
(L) (L) 1. (31`)
Lquation (31!) can c otaincd tormay y notin that choosin
(L) satistyin (31`), and mutijyin oth sidcs ot (31!) y (L)
ivcs thc rcjrcscntation (31!) lt is ccar that thc random wak has

1
1 and
k
0 tor a k 2, which docs not satisty thc rcstriction
and thc jroccss is nonstationary x
t
is stationary it

k
|
k
| < . scc
lrojosition 312 in Lrockwc and Lavis (1991, jS!), which can c
wcakcncd y

k

2
k
< . scc Lamiton (199!, j`2)
Example 2.2: Sujjosc that wc havc an autorcrcssivc modc (312)
with p 1, ic, x
t

1
x
t1
(1
1
L)x
t
w
t
Thcn (31`)
ccomcs (1+
1
L+
2
L
2
+ )(1
1
L) 1 Lquatin coccicnts
ot L imjics that
1

1
0 or
1

1
Ior L
2
, wc woud ct

1
0, or
2

2
1
Continuin, wc otain
k

k
1
and thc
1
This restriction is a sucient and necessary condition for an ARMA time series to be invertible; see
Section 3.7 in Hamilton (1994) or Theorem 3.1.2 in Brockwell and Davis (1991, p.86) and the related
discussions.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 95
rcjrcscntation is
(L) 1 +

k=1

k
1
L
k
and wc havc x
t

k=0

k
1
w
tk
Thc rcjrcscntation (31!) is tun-
damcnta tor dcvcojin ajjroximatc torccasts and aso cxhiits thc
scrics as a incar jroccss ot thc torm considcrcd in Chajtcr 2
Ior data invovin such autorcrcssivc (Al) modcs as dcncd
aovc, thc main sccction jrocms arc dccidin that thc autorcrcs-
sivc structurc is ajjrojriatc and thcn in dctcrminin thc vauc ot p
tor thc modc Thc ACI ot thc jroccss is a jotcntia aid tor dctcr-
minin thc ordcr ot thc jroccss as arc thc modc sccction mcasurcs
dcscricd in Scction 33 To dctcrminc thc ACI ot thc pth ordcr Al
in (312), writc thc cquation as
x
t

k=1

k
x
tk
w
t
and mutijy oth sidcs y x
th
tor any h 1 Assumin that
thc mcan E(x
t
) 0, and usin thc dcnition ot thc autocovariancc
tunction cads to thc cquation
E
_
_
(x
t
x
th

k=1

k
x
tk
x
th
_
_
E|w
t
x
th
|.
Thc ctt-hand sidc immcdiatcy ccomcs
x
(h)

p
k=1

k

x
(h k)
Thc rcjrcscntation (31!) imjics that
E|w
t
x
th
| E|w
t
(w
th
+
1
w
th1
+
2
w
th2
+ )|
_
_
_

2
w
, it h 0,
0 othcrwisc
Lcncc, wc may writc thc cquations tor dctcrminin
x
(h) as

x
(0)
p

k=1

k

x
(k)
2
w
(31o)
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 96
and

x
(h)
p

k=1

k

x
(h k) 0 tor h 1. (31)
`otc that onc wi nccd thc jrojcrty
x
(h)
x
(h) in sovin thcsc
cquations Lquations (31o) and (31) arc cacd thc Yuc-\akcr
Lquations (scc Yuc, 192, \akcr, 1931)
Example 2.3: Considcr ndin thc ACI ot thc rst-ordcr autorc-
rcssivc modc Iirst, (31) imjics that
x
(0)
1

x
(1)
2
w

Ior h 1, wc otain
x
(h)
1

x
(h 1) 0 Sovin thcsc
succcssivcy ivcs
x
(h)
x
(0)
h
1
Cominin with (31o) yicds

x
(0)
2
w
/(1
2
1
) lt toows that thc autocovariancc tunction is

x
(h)
2
w

h
1
/(1
2
1
) Takin into account that
x
(h)
x
(h),
wc otain
x
(h)
|h|
1
tor a h
Thc cxjoncntia dccay is tyjica ot autorcrcssivc chavior and
thcrc may aso c somc jcriodic structurc Lowcvcr, thc most ccc-
tivc dianostic ot Al structurc is in thc lACI and is summarizcd
y thc toowin idcntication jrojcrty
Property 2.2: The partial autocorrelation function as a func-
tion of lag h is zero for h > p, the order of the autoregressive
process. This enables one to make a preliminary identication
of the order p of the process using the partial autocorrelation
function PACF. Simply choose the order beyond which most of
the sample values of the PACF are approximately zero.
To vcrity thc aovc, notc that thc lACI (scc Scction 2!3) is
asicay thc ast coccicnt otaincd whcn minimizin thc squarcd
crror
MSE E
_

_
_
_
x
t+h

k=1
a
k
x
t+hk
_
_
2
_

_ .
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 97
Scttin thc dcrivativcs with rcsjcct to a
j
cqua to zcro cads to thc
cquations
E
_

_
_
_
x
t+h

k=1
a
k
x
t+hk
_
_
2
x
t+hj
_

_ 0
This can c writtcn as

x
(j)
h

k=1
a
k

x
(j k) 0
tor 1 j h `ow, trom Lquation and (31), it is ccar that, tor
an Al(p), wc may takc a
k

k
tor k p and a
k
0 tor k > p to
ct a soution tor thc aovc cquation This imjics Property 2.2
aovc
Lavin dccidcd on thc ordcr p ot thc modc, it is ccar that, tor
thc cstimation stcj, onc may writc thc modc (312) in thc rcrcssion
torm
x
t

z
t
+ w
t
, (31S)
whcrc (
1
,
2
, ,
p
)

corrcsjonds to and z
t
(x
t1
, x
t2
, , x
tp
is thc vcctor ot dcjcndcnt variacs in (32) Takin into account thc
tact that x
t
is not oscrvcd tor t 0, wc may run thc rcrcssion
ajjroach in Scction 32 tor t p +1, , n to ct cstimators tor
and tor
2
, thc variancc ot thc whitc noisc jroccss Thcsc so-cacd
conditional maximum likelihood estimators arc commony
uscd ccausc thc cxact maximum ikcihood cstimators invovc sov-
in nonincar cquations. scc Chajtcr ` in Lamiton (199!) tor dctais
and wc wi discuss this issuc atcr
Example 2.4: \c considcr thc simjc jrocm ot modcin thc
rccruit scrics shown in thc riht janc ot Iiurc 21 usin an autorc-
rcssivc modc Thc toj riht janc ot Iiurc 21o and thc toj riht
janc ot Iiurc 219 shows thc autocorrcation and jartia autocorrc-
ation tunctions ot thc rccruit scrics Thc lACI has arc vaucs tor
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 98
Table 3.1: AICC values for ten models for the recruits series
p 1 2 3 4 5 6 7 8 9 10
AICC 5.75 5.52 5.53 5.54 5.54 5.55 5.55 5.56 5.57 5.58
h 1 and 2 and thcn is csscntiay zcro tor hihcr ordcr as This
imjics y Property 2.2 aovc that a sccond ordcr (p 2) Al
modc miht jrovidc a ood t lunnin thc rcrcssion jroram tor
an Al(2) modc with intcrccjt
x
t

0
+
1
x
t1
+
2
x
t2
+ w
t
cads to thc cstimators

0
o1.S!39(!.0121),

1
1.3`12(0.0!1),

2
0.!o12(0.0!1o) and

2
S9.`3, whcrc thc cstimatcd stan-
dard dcviations arc in jarcnthcscs To dctcrminc whcthcr thc aovc
ordcr is thc cst choicc, wc ttcd modcs tor 1 p 10, otain-
in corrcctcd AlCC vaucs summarizcd in Tac 31 usin (3S) with
K 2 This shows that thc minimum AlCC otains tor p 2 and
wc choosc thc sccond ordcr modc
Example 2.5: Thc jrcvious cxamjc uscd various autorcrcssivc
modcs tor thc rccruits scrics, ttin a sccond-ordcr rcrcssion modc
\c may aso usc this rcrcssion idca to t thc modc to othcr scrics
such as a dctrcndcd vcrsion ot thc SOl ivcn in jrcvious discussions
\c havc notcd in our discussions ot Iiurc 219 trom thc jartia
autocorrcation tunction that a jausic modc tor this scrics miht
c a rst ordcr autorcrcssion ot thc torm ivcn aovc with p 1
Aain, juttin thc modc aovc into thc rcrcssion tramcwork (32)
tor a sinc coccicnt cads to thc cstimators

1
0.`9 with stan-
dard crror 0.0!,

2
0.0921S and AICC(1) 1.3` Thc ACI ot
thcsc rcsiduas shown in thc ctt janc ot Iiurc 3`, howcvcr, wi sti
show cycica variation and it is ccar that thcy sti havc a numcr
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 99
ot vaucs cxcccdin thc 1.9o/

n thrcshod A sucstcd jroccdurc

0 5 10 15 20

0
.
5
0
.
0
0
.
5
1
.
0
ACF of residuls of AR(1) for SOI
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o o
o o
o o
o
o
o
o o
o
0 5 10 15 20 25 30

1
.
7
0

1
.
6
5

1
.
6
0

1
.
5
5

1
.
5
0

1
.
4
5

1
.
4
0

1
.
3
5
Lag
o
o
o
o
o o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o o
o o
o
o
o
o
o
o
AIC
AICC
Figure 3.5: Autocorrelation (ACF) of residuals of AR(1) for SOI (left panel) and the plot of
AIC and AICC values (right panel).
is to try hihcr ordcr autorcrcssivc modcs and succcssivc modcs
tor 1 p 30 wcrc ttcd and thc AICC(K) vaucs arc jottcd in
thc riht janc ot Iiurc 3` Thcrc is a ccar minimum tor a p 1o
ordcr modc Thc coccicnt vcctor is with comjoncnts and thcir
standard crrors in thc jarcnthcscs 0.!0`0(0.0!o9), 0.0!0(0.0`0`),
0.1`2(0.0!99), 0.091`(0.0`0`), 0.03(0.0`00), 0.0S03(0.0!93),
0.0!3(0.0!93), 0.0o9(0.0!92), 0.009o(0.0!92), 0.110S (0.0!91),
0.10(0.0!92), 0.1o0o(0.0!99), 0.02S1(0.0`0!), 0.1902(0.0`01), 0.12S3(0.0`10),
0.0!13(0.0!o), and

2
0.01oo
3.5.2 Forecasting
Timc scrics anaysis has jrovcd to c tairy ood way ot jroducin
torccasts lts drawack is that it is tyjicay not conducivc to struc-
tura or cconomic anaysis ot thc torccast Thc modc has torccastin
jowcr ony it thc tuturc variac cin torccastcd is rcatcd to currcnt
vaucs ot thc variacs that wc incudc in thc modc
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 100
Thc oa is to torccast thc variac y
s
ascd on a sct ot variacs
X
t
(X
t
may consist ot thc as ot variac y
t
) Lct y
s
t
dcnotc a torc-
cast ot y
s
ascd on X
t
A quadratic oss tunction is thc samc as in
OLS rcrcssion, ic choosc y
t
s
to minimizc E(y
t
s
y
s
)
2
and thc mcan
squarcd crror (`SL) is dcncd as `SL(y
t
s
) E
_
(y
t
s
y
s
)
2
| X
t
_
lt
can c shown that thc torccast with thc smacst `SL is thc cxjcc-
tation ot y
s
conditiona on X
t
, that is y
t
s
E(y
s
| X
t
) Thcn, thc
`SL ot thc ojtima torccast is thc conditiona variancc ot y
s
ivcn
X
t
, that is \ar(y
s
| X
t
)
\c now considcr thc cass ot torccasts that arc incar jro,cction
Thcsc torccasts arc uscd vcry ottcn in cmjirica anaysis ot timc scrics
data Thcrc arc two conditions tor thc torccast y
t
s
to c a incar
jro,cction (1) Thc torccast y
t
s
nccds to c a incar tunction ot X
t
,
that is y
t
s
E(y
s
| X
t
)

X
t
, and (2) thc coccicnts shoud c
choscn in such a way that E|(y
s

X
t
)X

t
| 0 Thc torccast

X
t
satistyin (1) and (2) is cacd thc incar jro,cction ot y
s
on X
t
Onc
ot thc rcasons incar jro,ccts arc jojuar is that thc incar jro,cction
jroduccs thc smacst `SL amon thc cass ot incar torccastin rucs
Iinay, wc ivc a cncra ajjroach to torccastin tor any jroccss
that can c writtcn in thc torm (31!), a incar jroccss This incudcs
thc AR, MA and ARMA jroccsscs \c cin y dcnin an h-stcj
torccast ot thc jroccss x
t
as
x
t
t+h
E|x
t+h
| x
t
, x
t1
, |
`otc that this is not cxacty riht ccausc wc ony havc x
1
, x
2
,
, x
t
avaiac, so that conditionin on thc innitc jast is ony an
ajjroximation Irom this dcnition, it is rcasonac to intuit that
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 101
x
t
s
x
t
tor s t and
E|w
s
| x
t
, x
t1
, | E|w
s
| w
t
, w
t1
, | w
t
s
w
s
(319)
tor s t Ior s > t, usc x
t
s
and
E|w
s
| x
t
, x
t1
, | E|w
s
| w
t
, w
t1
, | w
t
s
E(w
s
) 0
(320)
sincc w
s
wi c indcjcndcnt ot jast vaucs ot w
t
\c dcnc thc
h-stcj torccast variancc as
P
t
t+h
E|(x
t+h
x
t
t+h
)
2
| x
t
, x
t1
, | (321)
To dcvcoj an cxjrcssion tor this mcan squarc crror, notc that, with

0
1, wc can writc
x
t+h

k=0

k
w
t+hk
.
Thcn, sincc w
t
t+hk
0 tor t + h k > t, ic k < h, wc havc
x
t
t+h

k=0

k
w
t
t+hk

k=h

k
w
t+hk
,
so that thc rcsidua is
x
t+h
x
t
t+h

h1

k=0

k
w
t+hk
.
Lcncc, thc mcan squarc crror (321) is ,ust thc variancc ot a incar
comination ot indcjcndcnt zcro mcan crrors, with common variancc

2
w
P
t
t+h

2
w
h1

k=0

2
k
. (322)
Ior morc discussions, scc Lamiton (199!, Chajtcr !) As an cxam-
jc, wc considcr torccastin thc sccond ordcr modc dcvcojcd tor thc
rccruits scrics in Example 2.4
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 102
Example 2.6: Considcr thc onc-stcj torccast x
t
t+1
rst \ritin
thc dcnin cquation tor t +1 ivcs x
t+1

1
x
t
+
2
x
t1
+w
t+1
. so
that x
t
t+1

1
x
t
t
+
2
x
t
t1
+w
t
t+1

1
x
t
+
2
x
t1
+0 Continuin in
this vcin, wc otain x
t
t+2

1
x
t
t+1
+
2
x
t
t
+w
t
t+2

1
x
t
t+1
+
2
x
t
+0
Thcn, x
t
t+h

1
x
t
t+h1
+
2
x
t
t+h2
+w
t
t+h

1
x
t
t+h1
+
2
x
t
t+h2
+0
tor h > 2 Iorccasts out to a h ! and cyond, it ncccs-
sary, can c tound y sovin (31`) tor
1
,
2
and
3
, and su-
stitutin into (321) Ly cquatin coccicnts ot B, L
2
and L
3
in
(1
1
L
2
L
2
)(1 +
1
L +
2
L
2
+
3
L
3
+ ) 1, wc otain

1

1
,
2

2
+
1

1
0 and
3

1

2

2

1
0 This
ivcs thc coccicnts
1

1
,
2

2

2
1
,
3
2
,
2

1

2
1
Irom Example 2.4, wc havc

1
1.3`,

2

0.!o,

2
w
90.31 and

0
o.! Thc torccasts arc ot thc torm
x
t
t+h
o.! + 1.3` x
t
t+h1
0.!o x
t
t+h2
Ior thc torccast vari-
ancc, wc cvauatc

1
1.3`,

2
2.2S2,

3
3.0o`, cadin
to 90.31, 90.31(2.2SS), 90.31(.!9`) and 90.31(1o.S90) tor torccasts
at h 1, 2, 3, ! Thc standard dcviations ot thc torccasts arc 9.`0,
1!.3, 2o.02 and 39.0o tor thc standard crrors ot thc torccasts Thc
rccruits scrics vaucs ranc trom 20 to 100 so thc torccast unccrtainty
wi c rathcr arc
3.6 Moving Average Models MA(q)
\c may aso considcr jroccsscs that contain incar cominations ot
undcryin unoscrvcd shocks, say, rcjrcscntcd y whitc noisc scrics
w
t
Thcsc movin avcrac comjoncnts cncratc a scrics ot thc torm
x
t
w
t

k=1

k
w
tk
, (323)
whcrc q dcnotcs thc ordcr ot thc movin avcrac comjoncnt and

k
(1 k q) arc jaramctcrs to c cstimatcd sin thc ack-shitt
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 103
notation, thc aovc cquation can c writtcn in thc torm
x
t
(L) w
t
with (L) 1
q

k=1

k
L
k
, (32!)
whcrc (L) is anothcr joynomia in thc shitt ojcrator B lt shoud
c notcd that thc `A jroccss ot ordcr q is a incar jroccss ot thc
torm considcrcd caricr in Problem 4 in Chajtcr 2 with
0
1,

1

1
, ,
q

q
This imjics that thc ACI wi c zcro tor
as arcr than q ccausc tcrms in thc torm ot thc covariancc tunction
ivcn in Problem 4 ot Chajtcr 2 wi a c zcro Sjccicay, thc
cxact torms arc

x
(0)
2
w
_
_
1 +
q

k=1

2
k
_
_
and
x
(h)
2
w
_
_
_
h
+
qh

k=1

k+h

k
_
_
_
(32`)
tor 1 h q 1, with
x
(q)
2
w

q
, and
x
(h) 0 tor h > q
Lcncc, wc wi havc thc jrojcrty ot ACI tor tor `A Scrics
Property 2.3: For a moving average series of order q, note
that the autocorrelation function (ACF) is zero for lags h > q,
i.e.
x
(h) 0 for h > q. Such a result enables us to diagnose
the order of a moving average component by examining
x
(h)
and choosing q as the value beyond which the coecients are
essentially zero.
Example 2.7: Considcr thc varvc thickncsscs in Iiurc 219, which
is dcscricd in Problem 7 ot Chajtcr 2 Iiurc 3o shows thc ACI
and lACI ot thc oriina o-transtormcd varvc scrics {x
t
} and thc
rst dicrcnccs Thc ACI ot thc oriina scrics {x
t
} indicatcs a jos-
sic non-stationary chavior, and sucsts takin a rst dicrcncc
x
t
, intcrjrctcd hcar as thc jcrccntac ycary chanc in dcjosition
Thc ACI ot thc rst dicrcncc x
t
shows a ccar jcak at h 1 and
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 104
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
ACF
log varves
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
PACF
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
First difference
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
Figure 3.6: Autocorrelation functions (ACF) and partial autocorrelation functions (PACF)
for the log varve series (top panel) and the rst dierence (bottom panel), showing a peak
in the ACF at lag h = 1.
no othcr sinicant jcaks, sucstin a rst ordcr movin avcrac
Iittin thc rst ordcr movin avcrac modc x
t
w
t

1
w
t1
to
this data usin thc Gauss-`cwton jroccdurc dcscricd ncxt cads to

1
0. and

2
w
0.23`S
Iittin thc jurc movin avcrac tcrm turns into a nonincar jro-
cm as wc can scc y notin that cithcr maximum ikcihood or rc-
rcssion invovcs sovin (323) or (32!) tor w
t
, and minimizin thc
sum ot thc squarcd crrors Sujjosc that thc roots ot (L) 0 arc a
outsidc thc unit circc, thcn this is jossic y sovin (L) (L) 1,
so that, tor thc vcctor jaramctcr (
1
, ,
q
)

, wc may writc
w
t
() (L) x
t
(32o)
and minimizc SSL()

n
t=q+1
w
2
t
() as a tunction ot thc vcctor
jaramctcr \c do not rcay nccd to nd thc ojcrator (L) ut
can simjy sovc (32o) rccursivcy tor w
t
, with w
1
, w
2
, , w
q
0,
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 105
and w
t
() x
t
+

q
k=1

k
w
tk
tor q +1 t n lt is casy to vcrity
that SSL() wi c a nonincar tunction ot
1
,
2
, ,
q
Lowcvcr,
notc that y thc Tayor cxjansion
w
t
() w
t
(
0
) +
_
_
_
w
t
()

0
_
_
_ (
0
),
whcrc thc dcrivativc is cvauatcd at thc jrcvious ucss
0
lcarran-
in thc aovc cquation cads to
w
t
(
0
)
_
_
_
w
t
()

0
_
_
_ (
0
) + w
t
(),
which is ,ust thc rcrcssion modc (32) Lcncc, wc can cin with an
initia ucss
0
(0.1, 0.1, , 0.1)

, say and succcssivcy minimizc

SSL() unti convcrcncc Scc Chajtcr ` in Lamiton (199!) tor
dctais and wc wi discuss this issuc atcr
Forecasting: ln ordcr to torccast a movin avcrac scrics, notc
that x
t+h
w
t+h

q
k=1

k
w
t+hk
Thc rcsuts cow (319) imjy
that x
t
t+h
0 it h > q and it h q,
x
t
t+h

q

k=h

k
w
t+hk
,
whcrc thc w
t
vaucs nccdcd tor thc aovc arc comjutcd rccursivcy
as ctorc Lccausc ot (31!), it is ccar that
0
1 and
k

k
tor 1 k q and thcsc vaucs can c sustitutcd dirccty into thc
variancc tormua (322) That is,
P
t
t+h

2
w
_
_
1 +
h1

k=1

2
k
_
_
.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 106
3.7 Autoregressive Integrated Moving Average Model
- ARIMA(p, d, q)
`ow cominin thc autorcrcssivc and movin avcrac comjoncnts
cads to thc autorcrcssivc movin avcrac Al`A(p, q) modc, writ-
tcn as (L) x
t
(L) w
t
whcrc thc joynomias in B arc as dcncd
caricr in (313) and (32!), with p autorcrcssivc coccicnts and
q movin avcrac coccicnts ln thc dicrcncc cquation torm, this
ccomcs
x
t

k=1

k
x
tk
w
t

k=1

k
w
tk
.
Thc mixcd jroccsscs do not satisty thc Properties 2.1 - 2.3 any
morc ut thcy tcnd to chavc in ajjroximatcy thc samc way, cvcn
tor thc mixcd cascs Lstimation and torccastin tor such jrocms
arc trcatcd in csscntiay thc samc manncr as tor thc Al and `A
jroccsscs \c notc that wc can tormay dividc oth sidcs ot (320)
y (L) and notc that thc usua rcjrcscntation (31!) hods whcn
(L) (L) (L). (32)
Ior torccastin, wc dctcrminc thc {
k
} y cquatin coccicnts ot
{L
k
} in (32), as ctorc, assumin thc a thc roots ot (L) 0 arc
rcatcr than onc in asoutc vauc Simiary, wc can aways sovc tor
thc rcsiduas, say
w
t
x
t

k=1

k
x
tk
+
q

k=1

k
w
tk
to ct thc tcrms nccdcd tor torccastin and cstimation
Example 2.8: Considcr thc aovc mixcd jroccss with p q 1,
ic Al`A(1, 1) Ly (321), wc may writc
x
t

1
x
t1
+ w
t

1
w
t1
.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 107
`ow, x
t+1

1
x
t
+w
t+1

1
w
t
so that x
t
t+1

1
x
t
+0
1
w
t

1
x
t

1
w
t
and x
t
t+h

1
x
t
t+h1
tor h > 1, cadin to vcry simjc
torccasts in this casc Lquatin coccicnts ot L
k
in (1
1
L)(1 +

1
L +
2
L
2
+ ) (1
1
L) cads to
k
(
1

1
)
k1
1
tor
k 1 sin (322) cads to thc cxjrcssion
P
t
t+h

2
w
_
_
1 + (
1

1
)
2
h1

k=1

2(k1)
1
_
_

2
w
_
1 + (
1

1
)
2
(1
2(h1)
1
)/(1
tor thc torccast variancc
ln thc rst cxamjc ot this chajtcr, it was notcd that nonstationary
jroccsscs arc charactcrizcd y a sow dccay in thc ACI as in Iiurc
3! ln many ot thc cascs whcrc sow dccay is jrcscnt, thc usc ot a
rst ordcr dicrcncc x
t
x
t
x
t1
(1 L) x
t
wi rcducc thc
nonstationary jroccss x
t
to a stationary scrics x
t
On can chcck
to scc whcthcr thc sow dccay has ccn ciminatcd in thc ACI ot thc
transtormcd scrics Lihcr ordcr dicrcnccs,
d
x
t

d1
x
t
arc
jossic and wc ca thc jroccss otaincd whcn thc dth dicrcncc is
an Al`A scrics an All`A(p, d, q) scrics whcrc p is thc ordcr ot
thc autorcrcssivc comjoncnt, d is thc ordcr ot dicrcncin nccdcd
and q is thc ordcr ot thc movin avcrac comjoncnt Symoicay,
thc torm is
(L)
d
x
t
(L) w
t
.
Thc jrincijcs ot modc sccction tor All`A(p, d, q) scrics arc o-
taincd usin thc ikcihood ascd mcthods such as AlC, LlC or AlCC
which rcjacc K y K p +q thc tota numcr ot Al`A jaramc-
tcrs or othcr mcthods such as jcnaizcd mcthods dcscricd in Scction
33
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 108
3.8 Seasonal ARIMA Models
Somc cconomic and nancia as wc as cnvironmcnta timc scrics
such as quartcry carnin jcr sharc ot a comjany cxhiits ccrtain
cyclical or periodic chavior. scc thc atcr chajtcrs on morc dis-
cussions on cycles and periodicity Such a timc scrics is cacd
a seasonal (dctcrministic cycc) timc scrics Iiurc 2S shows thc
timc jot ot quartcry carnin jcr sharc ot 1ohnson and 1ohnson
trom thc rst quartcr ot 19o0 to thc ast quartcr ot 19S0 Thc data
josscss somc sjccia charactcristics ln jarticuar, thc carnin rcw
cxjoncntiay durin thc samjc jcriod and had a stron scasona-
ity Iurthcrmorc, thc variaiity ot carnin incrcascd ovcr timc Thc
cycica jattcrn rcjcats itsct cvcry ycar so that thc jcriodicity ot
thc scrics is ! lt monthy data arc considcrcd (c, monthy sacs
ot \a-`art Storcs), thcn thc jcriodicity is 12 Scasona timc scrics
modcs arc aso usctu in jricin wcathcr-rcatcd dcrivativcs and cn-
cry tuturcs Scc Example 1.8 and Example 1.9 in Chajtcr 2
tor morc cxamjcs with scasonaity
Anaysis ot scasona timc scrics has a on history ln somc ajji-
cations, scasonaity is ot sccondary imjortancc and is rcmovcd trom
thc data, rcsutin in a scasonay ad,ustcd timc scrics that is thcn
uscd to makc intcrcncc Thc jroccdurc to rcmovc scasonaity trom a
timc scrics is rctcrrcd to as seasonal adjustment `ost cconomic
data juishcd y thc S ovcrnmcnt arc scasonay ad,ustcd (c,
thc rowth ratc ot domcstic ross jroduct and thc uncmjoymcnt
ratc) ln othcr ajjications such as torccastin, scasonaity is as im-
jortant as othcr charactcristics ot thc data and must c handcd
accordiny Lccausc torccastin is a ma,or o,cctivc ot cconomic
and nancia timc scrics anaysis, wc tocus on thc attcr ajjroach
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 109
and discuss somc cconomctric modcs that arc usctu in modcin
scasona timc scrics
\hcn thc autorcrcssivc, dicrcncin, or scasona movin avcrac
chavior sccms to occur at mutijcs ot somc undcryin jcriod s,
a scasona All`A scrics may rcsut Thc scasona nonstationarity
is charactcrizcd y sow dccay at mutijcs ot s and can ottcn c
ciminatcd y a seasonal dierencing operator ot thc torm

D
s
x
t
(1 L
s
)
D
x
t
Ior cxamjc, whcn wc havc monthy data, it
is rcasonac that a ycary jhcnomcnon wi inducc s 12 and thc
ACI wi c charactcrizcd y sowy dccayin sjikcs at 12, 2!, 3o, !S,
, and wc can otain a stationary scrics y transtormin with thc
ojcrator (1 L
12
) x
t
x
t
x
t12
which is thc dicrcncc ctwccn
thc currcnt month and thc vauc onc ycar or 12 months ao lt thc
autorcrcssivc or movin avcrac chavior is scasona at jcriod s, wc
dcnc tormay thc ojcrators
(L
s
) 1
1
L
s

2
L
2s

P
L
Ps
(32S)
and
(L
s
) 1
1
L
s

2
L
2s

Q
L
Qs
. (329)
Thc na torm ot thc scasona All`A(p, d, q) (P, D, Q)
s
modc is
(L
s
) (L)
D
s

d
x
t
(L
s
) (L) w
t
. (330)
`otc that onc sjccia modc ot (330) is All`A(0, 1, 1) (0, 1, 1)
s
,
that is
(1 L
s
)(1 L) x
t
(1
1
L)(1
1
L
s
) w
t
.
This modc is rctcrrcd to as thc airline model or multiplicative
seasonal model in thc itcraturc. scc Lox, 1cnkins, and lcinsc
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 110
(199!, Chajtcr 9) lt has ccn tound to c widcy ajjicac in
modcin scasona timc scrics Thc Al jart ot thc modc simjy
consists ot thc rcuar and scasona dicrcnccs, whcrcas thc `A jart
invovcs two jaramctcrs
\c may aso notc thc jrojcrtics cow corrcsjondin to Prop-
erties 2.1 - 2.3
Property 2.1: The ACF of a seasonally non-stationary time
series decays very slowly at lag multiples s, 2s, 3s, , with
zeros in between, where s denotes a seasonal period, usually !
for quarterly data or 12 for monthly data. The PACF of a non-
stationary time series tends to have a peak very near unity at
lag s.
Property 2.2: For a seasonal autoregressive series of order P,
the partial autocorrelation function
hh
as a function of lag h
has nonzero values at s, 2s, 3s, , Ps, with zeros in between,
and is zero for h > Ps, the order of the seasonal autoregressive
process. There should be some exponential decay.
Property 2.3: For a seasonal moving average series of order
Q, note that the autocorrelation function (ACF) has nonzero
values at s, 2s, 3s, , Qs and is zero for h > Qs.
Remark: `otc that thcrc is a uid-in command in Rcacd arima()
which is a jowcrtu too tor cstimatin and makin intcrcncc tor an
All`A modc Thc command is
arima(x,order=c(0,0,0),seasonal=list(order=c(0,0,0),period=NA
xreg=NULL,include.mean=TRUE, transform.pars=TRUE,fixed=NULL,init=NULL,
method=c("CSS-ML","ML","CSS"),n.cond,optim.control=list(),kappa
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 111
Scc thc manuas ot R tor dctais aout this commcnd
Example 2.9: \c iustratc y ttin thc monthy irth scrics
trom 19!S-199 shown in Iiurc 3 Thc jcriod cncomjasscs thc
0 100 200 300
2
5
0
3
0
0
3
5
0
4
0
0
Births
0 100 200 300

4
0

2
0
0
2
0
4
0
First difference
0 50 100 200 300

4
0

2
0
0
2
0
4
0
ARIMA(0,1,0)X(0,1,0)_{12}
0 100 200 300

4
0

2
0
0
2
0
4
0
ARIMA(0,1,1)X(0,1,1)_{12}
Figure 3.7: Number of live births 1948(1) 1979(1) and residuals from models with a rst
dierence, a rst dierence and a seasonal dierence of order 12 and a tted ARIMA(0, 1, 1)
(0, 1, 1)
12
model.
oom that toowcd thc Sccond \ord \ar and thcrc is thc cxjcctcd
risc which jcrsists tor aout 13 ycars toowcd y a dccinc to around
19! Thc scrics ajjcars to havc on-tcrm swins, with scasona
cccts sujcrimjoscd Thc on-tcrm swins indicatc jossic non-
stationarity and wc vcrity that this is thc casc y chcckin thc ACI
and lACI shown in thc toj janc ot Iiurc 3S `otc that y Prop-
erty 2.1, sow dccay ot thc ACI indicatcs non-stationarity and wc
rcsjond y takin a rst dicrcncc Thc rcsuts shown in thc sccond
janc ot Iiurc 2` indicatc that thc rst dicrcncc has ciminatcd
thc stron ow trcqucncy swin Thc ACI, shown in thc sccond
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 112
0 10 20 30 40 50 60

0
.
5
0
.
5
ACF
0 10 20 30 40 50 60

0
.
5
0
.
5
PACF
data
0 10 20 30 40 50 60

0
.
5
0
.
5
0 10 20 30 40 50 60

0
.
5
0
.
5
ARIMA(0,1,0)
0 10 20 30 40 50 60

0
.
5
0
.
5
0 10 20 30 40 50 60

0
.
5
0
.
5
ARIMA(0,1,0)X(0,1,0)_{12}
0 10 20 30 40 50 60

0
.
5
0
.
5
0 10 20 30 40 50 60

0
.
5
0
.
5
ARIMA(0,1,0)X(0,1,1)_{12}
0 10 20 30 40 50 60

0
.
5
0
.
5
0 10 20 30 40 50 60

0
.
5
0
.
5
ARIMA(0,1,1)X(0,1,1)_{12}
Figure 3.8: Autocorrelation functions (ACF) and partial autocorrelation functions (PACF)
for the birth series (top two panels), the rst dierence (second two panels) an
ARIMA(0, 1, 0) (0, 1, 1)
12
model (third two panels) and an ARIMA(0, 1, 1) (0, 1, 1)
12
model (last two panels).
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 113
janc trom thc toj in Iiurc 3S shows jcaks at 12, 2!, 3o, !S, ,
with now dccay This chavior imjics scasona non-stationarity, y
Property 2.1 aovc, with s 12 A scasona dicrcncc ot thc
rst dicrcncc cncratcs an ACI and lACI in Iiurc 3S that wc
cxjcct tor stationary scrics
Takin thc scasona dicrcncc ot thc rst dicrcncc ivcs a scrics
that ooks stationary and has an ACI with jcaks at 1 and 12 and
a lACI with a sustantia jcak at 12 and csscr jcaks at 12, 2!,
This sucsts tryin cithcr a rst ordcr movin avcrac tcrm,
y Property 2.3, or a rst ordcr scasona movin avcrac tcrm
with s 12, y Property 2.3 aovc \c choosc to ciminatc thc
arcst jcak rst y ajjyin a rst-ordcr scasona movin avcrac
modc with s 12 Thc ACI and lACI ot thc rcsidua scrics
trom this modc, ic trom All`A(0, 1, 0) (0, 1, 1)
12
, writtcn as
(1 L)(1 L
12
) x
t
(1
1
L
12
) w
t
, is shown in thc tourth janc
trom thc toj in Iiurc 3S \c notc that thc jcak at a onc is
sti thcrc, with attcndin cxjoncntia dccay in thc lACI This can
c ciminatcd y ttin a rst-ordcr movin avcrac tcrm and wc
considcr thc modc All`A(0, 1, 1) (0, 1, 1)
12
, writtcn as
(1 L)(1 L
12
) x
t
(1
1
L)(1
1
L
12
) w
t
.
Thc ACI ot thc rcsiduas trom this modc arc rcativcy wc chavcd
with a numcr ot jcaks cithcr ncar or cxcccdin thc 9`/ tcst ot no
corrcation Iittin this na All`A(0, 1, 1)(0, 1, 1)
12
modc cads
to thc modc
(1 L)(1 L
12
) x
t
(1 0.!S9o L)(1 0.oS!! L
12
) w
t
with AlCC !.9`, R
2
0.9S0!
2
0.9o1, and thc j-vaucs arc
(0.000, 0.000) Thc All`A scarch cads to thc modc
(1 L)(1 L
12
) x
t
(1 0.!0SS L0.1o!` L
2
)(1 0.o990 L
12
) w
t
,
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 114
yicdin AlCC !.92 and R
2
0.9S1
2
0.9o2, sihty cttcr than
thc All`A(0, 1, 1) (0, 1, 1)
12
modc Lvauatin thcsc attcr mod-
cs cads to thc concusion that thc cxtra jaramctcrs do not add a
jracticay sustantia amount to thc jrcdictaiity Thc modc is
cxjandcd as
x
t
x
t1
+ x
t12
x
t13
+ w
t

1
w
t1

1
w
t12
+
1

1
w
t13
.
Thc torccast is
x
t
t+1
x
t
+ x
t11
x
t12

1
w
t

1
w
t11
+
1

1
w
t12
x
t
t+2
x
t
t+1
+ x
t10
x
t11

1
w
t10
+
1

1
w
t11
.
Continuin in thc samc manncr, wc otain
x
t
t+12
x
t
t+11
+ x
t
x
t1

1
w
t
+
1

1
w
t1
tor thc 12 month torccast
Example 2.10: Iiurc 39 shows thc autocorrcation tunction ot
thc o-transtormcd 1:1 carnins scrics that is jottcd in Iiurc
2S and wc notc thc sow dccay indicatin thc nonstationarity which
has arcady ccn ovious in thc Chajtcr 2 discussion \c may aso
comjarc thc ACI with that ot a random wak, shown in Iiurc 32,
and notc thc cosc simiarity Thc jartia autocorrcation tunction
is vcry hih at a onc which, undcr ordinary circumstanccs, woud
indicatc a rst ordcr autorcrcssivc Al(1) modc, cxccjt that, in
this casc, thc vauc is cosc to unity, indicatin a root cosc to 1 on
thc unit circc Thc ony qucstion woud c whcthcr dicrcncin or
dctrcndin is thc cttcr transtormation to stationarity Ioowin in
thc Lox-1cnkins tradition, dicrcncin cads to thc ACI and lACI
shown in thc sccond janc and no simjc structurc is ajjarcnt To
torcc a ncxt stcj, wc intcrjrct thc jcaks at !, S, 12, 1o, , as
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 115
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
ACF
log(J&J)
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
PACF
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
First Difference
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
ARIMA(0,1,0)X(1,0,0,)_4
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
ARIMA(0,1,1)X(1,0,0,)_4
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
Figure 3.9: Autocorrelation functions (ACF) and partial autocorrelation functions (PACF)
for the log J&J earnings series (top two panels), the rst dierence (second two panels),
ARIMA(0, 1, 0) (1, 0, 0)
4
model (third two panels), and ARIMA(0, 1, 1) (1, 0, 0)
4
model
(last two panels).
contriutin to a jossic scasona autorcrcssivc tcrm, cadin to
a jossic All`A(0, 1, 0) (1, 0, 0)
4
and wc simjy t this modc
and ook at thc ACI and lACI ot thc rcsiduas, shown in thc third
two jancs Thc t imjrovcs somcwhat, with sinicant jcaks sti
rcmainin at a 1 in oth thc ACI and lACI Thc jcak in thc ACI
sccms morc isoatcd and thcrc rcmains somc cxjoncntiay dccayin
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 116
chavior in thc lACI, so wc try a modc with a rst-ordcr movin
avcrac Thc ottom two jancs show thc ACI and lACI ot thc
rcsutin All`A(0, 1, 1)(1, 0, 0)
4
and wc notc ony rcativcy minor
cxcursions aovc and cow thc 9`/ intcrvas undcr thc assumjtion
that thc thcorctica ACI is whitc noisc Thc na modc sucstcd
is (y
t
o x
t
)
(1
1
L
4
)(1 L) y
t
(1
1
L) w
t
, (331)
whcrc

1
0.S20(0.0`S),

1
0.`0S(0.09S), and

2
w
0.00So Thc
modc can c writtcn in torccast torm as
y
t
y
t1
+
1
(y
t4
y
t5
) + w
t

1
w
t1
.
Thc rcsidua jot ot thc aovc is jottcd in thc ctt ottom janc ot
Iiurc 310 To torccast thc oriina scrics tor, say ! quartcrs, wc
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
ACF
ARIMA(0,1,1)X(0,1,1,)_4
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
PACF
0 20 40 60 80

0
.
2

0
.
1
0
.
0
0
.
1
0
.
2
Residual Plot
ARIMA(0,1,1)X(1,0,0,)_4
0 20 40 60 80

0
.
1
0
.
0
0
.
1
0
.
2
Residual Plot
ARIMA(0,1,1)X(0,1,1,)_4
Figure 3.10: Autocorrelation functions (ACF) and partial autocorrelation functions
(PACF) for ARIMA(0, 1, 1) (0, 1, 1)
4
model (top two panels) and the residual plots of
ARIMA(0, 1, 1) (1, 0, 0)
4
(left bottom panel) and ARIMA(0, 1, 1) (0, 1, 1)
4
model (right
bottom panel).
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 117
comjutc thc torccast imits tor y
t
o x
t
and thcn cxjoncntiatc,
ic x
t
t+h
cxj(y
t
t+h
)
Lascd on thc thc cxact ikcihood mcthod, Tsay (200`) considcrcd
thc toowin scasona All`A(0, 1, 1) (0, 1, 1)
4
modc
(1 L)(1 L
4
) y
t
(1 0.oS L)(1 0.31! L
4
) w
t
, (332)
with

2
w
0.0S9, whcrc standard crrors ot thc two `A jaramctcrs
arc 0.0S0 and 0.101, rcsjcctivcy Thc L,un-Lox statistics ot thc
rcsiduas showQ(12) 10.0 with j-vauc 0.!! Thc modc ajjcars to
c adcquatc Thc ACI and lACI ot thc All`A(0, 1, 1) (0, 1, 1)
4
modc arc ivcn in thc toj two jancs ot Iiurc 310 and thc rcsidua
jot is disjaycd in thc riht ottom janc ot Iiurc 310 Lascd on
thc comjarison ot ACI and lACI ot two modc (331) and (332)
|thc ast two jancs ot Iiurc 39 and thc toj two jancs in Iiurc
310|, it sccms that All`A(0, 1, 1) (0, 1, 1)
4
modc in (332) miht
jcrtorm cttcr than All`A(0, 1, 1) (1, 0, 0)
4
modc in (331)
To iustratc thc torccastin jcrtormancc ot thc scasona modc in
(332), wc rc-cstimatc thc modc usin thc rst o oscrvations and
rcscrvc thc ast ciht data joints tor torccastin cvauation \c com-
jutc 1-stcj to S-stcj ahcad torccasts and thcir standard crrors ot thc
ttcd modc at thc torccast oriin t o An anti-o transtormation
is takcn to otain torccasts ot carnin jcr sharc usin thc rcationshij
ctwccn norma and o-norma distriutions Iiurc 21` in Tsay
(200`, j) shows thc torccast jcrtormancc ot thc modc, whcrc thc
oscrvcd data arc in soid inc, joint torccasts arc shown y dots, and
thc dashcd incs show 9`/ intcrva torccasts Thc torccasts show a
stron scasona jattcrn and arc cosc to thc oscrvcd data Ior morc
comjarisons tor torccasts usin dicrcnt modcs incudin scmijara-
mctric and nonjaramctric modcs, thc rcadcr is rctcrrcd to thc ook
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 118
y Shumway (19SS), and Shumway and Stocr (2000) and thc jajcrs
y Lurman and Shummay (199S) and Cai and Chcn (200o)
\hcn thc scasona jattcrn ot a timc scrics is stac ovcr timc (c,
cosc to a dctcrministic tunction), dummy variacs may c uscd to
handc thc scasonaity This ajjroach is takcn y somc anaysts
Lowcvcr, dctcrministic scasonaity is a sjccia casc ot thc mutijica-
tivc scasona modc discusscd ctorc Sjccicay, it
1
1, thcn
modc contains a dctcrministic scasona comjoncnt Conscqucnty,
thc samc torccasts arc otaincd y usin cithcr dummy variacs or
a mutijicativc scasona modc whcn thc scasona jattcrn is dctcr-
ministic Yct usc ot dummy variacs can cad to intcrior torccasts it
thc scasona jattcrn is not dctcrministic ln jracticc, wc rccommcnd
that thc cxact ikcihood mcthod shoud c uscd to cstimatc a mu-
tijicativc scasona modc, csjcciay whcn thc samjc sizc is sma
or whcn thcrc is thc jossiiity ot havin a dctcrministic scasona
comjoncnt
Example 2.11: To dctcrminc dctcrministic chavior, considcr thc
monthy simjc rcturn ot thc ClSl Lccic 1 indcx trom 1anuary
19o0 to Lcccmcr 2003 tor `2S oscrvations Thc scrics is shown in
thc ctt toj janc ot Iiurc 311 and thc timc scrics docs not show
any ccar jattcrn ot scasonaity Lowcvcr, thc samjc ACt ot thc
rcturn scrics shown in thc ctt ottom janc ot Iiurc 311 contains
sinicant as at 12, 2!, and 3o as wc as a 1 lt scasona Al`A
modcs arc cntcrtaincd, a modc in torm
(1
1
L)(1
1
L
12
) x
t
+ (1
1
L
12
) w
t
is idcnticd, whcrc x
t
is thc monthy simjc rcturn sin thc con-
ditiona ikcihood, thc ttcd modc is
(1 0.2` L)(1 0.99 L
12
) x
t
0.000! + (1 0.92 L
12
) w
t
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 119
0 100 200 300 400 500

0
.
2
0
.
0
0
.
2
0
.
4
Simple Returns
0 100 200 300 400 500

0
.
3

0
.
1
0
.
1
0
.
2
0
.
3
0
.
4
Januaryadjusted returns
0 10 20 30 40

0
.
5
0
.
0
0
.
5
1
.
0
ACF
0 10 20 30 40

0
.
5
0
.
0
0
.
5
1
.
0
ACF
Figure 3.11: Monthly simple return of CRSP Decile 1 index from January 1960 to December
2003: Time series plot of the simple return (left top panel), time series plot of the simple
return after adjusting for January eect (right top panel), the ACF of the simple return (left
bottom panel), and the ACF of the adjusted simple return.
with
w
0.01 Thc `A coccicnt is cosc to unity, indicatin
that thc ttcd modc is cosc to cin non-invcrtic lt thc cxact
ikcihood mcthod is uscd, wc havc
(1 0.2o! L)(1 0.99o L
12
) x
t
0.0002 + (1 0.999 L
12
) w
t
with
w
0.0o Canccation ctwccn scasona Al and `A tactors
is ccary This hihihts thc usctuncss ot usin thc cxact ikcihood
mcthod, and thc cstimation rcsut sucsts that thc scasona chavior
miht c dctcrministic To turthcr conrm this asscrtion, wc dcnc
thc dummy variac tor 1anuary, that is
J
t

_
_
_
1 it t is 1anuary,
0 othcrwisc,
and cmjoy thc simjc incar rcrcssion
x
t

0
+
1
J
t
+ e
t
.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 120
Thc riht jancs ot Iiurc 311 show thc timc scrics jot ot and thc
ACI ot thc rcsidua scrics ot thc jrior simjc incar rcrcssion Irom
thc ACI, thcrc arc no sinicant scria corrcation at any mutijcs
ot 12, sucstin that thc scasona jattcrn has ccn succcsstuy rc-
movcd y thc 1anuary dummy variac Conscqucnty, thc scasona
chavior in thc monthy simjc rcturn ot Lccic 1 is duc to thc Jan-
uary eect
3.9 Regression Models With Correlated Errors
ln many ajjications, thc rcationshij ctwccn two timc scrics is ot
ma,or intcrcst Thc markct modc in nancc is an cxamjc that
rcatcs thc rcturn ot an individua stock to thc rcturn ot a markct
indcx Thc tcrm structurc ot intcrcst ratcs is anothcr cxamjc in
which thc timc cvoution ot thc rcationshij ctwccn intcrcst ratcs
with dicrcnt maturitics is invcstiatcd Thcsc cxamjcs cad to thc
considcration ot a incar rcrcssion in thc torm y
t

1
+
2
x
t
+ e
t
,
whcrc y
t
and x
t
arc two timc scrics and e
t
dcnotcs thc crror tcrm Thc
cast squarcs (LS) mcthod is ottcn uscd to cstimatc thc aovc modc
lt {e
t
} is a whitc noisc scrics, thcn thc LS mcthod jroduccs consistcnt
cstimatcs ln jracticc, howcvcr, it is common to scc that thc crror
tcrm e
t
is scriay corrcatcd ln this casc, wc havc a rcrcssion modc
with timc scrics crrors, and thc LS cstimatcs ot
1
and
2
may not
c consistcnt and ccicnt
lcrcssion modc with timc scrics crrors is widcy ajjicac in
cconomics and nancc, ut it is onc ot thc most commony misuscd
cconomctric modcs ccausc thc scria dcjcndcncc in e
t
is ottcn ovcr-
ookcd lt jays to study thc modc carctuy Thc standard mcthod
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 121
tor dcain with corrcatcd crrors e
t
in thc rcrcssion modc
y
t

z
t
+ e
t
is to try to transtorm thc crrors e
t
into uncorrcatcd oncs and thcn
ajjy thc standard cast squarcs ajjroach to thc transtormcd o-
scrvations Ior cxamjc, ct P c an n n matrix that transtorms
thc vcctor e (e
1
, , e
n
)

into a sct ot indcjcndcnt idcnticay

distriutcd variacs with variancc
2
Thcn, transtorm thc matrix
vcrsion (3!) to
Py PZ + Pe
and jrocccd as ctorc Ot coursc, thc ma,or jrocm is dccidin
on what to choosc tor P ut in thc timc scrics casc, hajjiy, thcrc
is a rcasonac soution, ascd aain on timc scrics Al`A modcs
Sujjosc that wc can nd, tor cxamjc, a rcasonac Al`A modc
tor thc rcsiduas, say, tor cxamjc, thc Al`A(p, 0, 0) modc
e
t

p

k=1

k
e
t
+ w
t
,
which dcncs a incar transtormation ot thc corrcatcd e
t
to a sc-
qucncc ot uncorrcatcd w
t
\c can inorc thc jrocms ncar thc
cinnin ot thc scrics y startin at t p ln thc Al`A notation,
usin thc ack-shitt ojcrator B, wc may writc
(L) e
t
w
t
, (333)
whcrc
(L) 1
p

k=1

k
L
k
(33!)
and ajjyin thc ojcrator to oth sidcs ot (32) cads to thc modc
(L) y
t
(L) z
t
+ w
t
, (33`)
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 122
whcrc thc {w
t
}s now satisty thc indcjcndcncc assumjtion Loin
ordinary cast squarcs on thc transtormcd modc is thc samc as do-
in wcihtcd cast squarcs on thc untranstormcd modc Thc ony
jrocm is that wc do not know thc vaucs ot thc coccicnts
k
(1 k p) in thc transtormation (33!) Lowcvcr, it wc kncw thc
rcsiduas e
t
, it woud c casy to cstimatc thc coccicnts, sincc (33!)
can c writtcn in thc torm
e
t

e
t1
+ w
t
, (33o)
which is cxacty thc usua rcrcssion modc (32) with (
1
, ,
p
)

rcjacin and e
t1
(e
t1
, e
t2
, , e
tp
)

rcjacin z
t
Thc
aovc commcnts sucst a cncra ajjroach known as thc Cochran-
Orcutt jroccdurc (Cochranc and Orcutt, 19!9) tor dcain with thc
jrocm ot corrcatcd crrors in thc timc scrics contcxt
1 Lcin y ttin thc oriina rcrcssion modc (32) y cast squarcs,
otainin and thc rcsiduas e
t
y
t

z
t

2 Iit an Al`A to thc cstimatcd rcsiduas, say (L) e

t
(L) w
t

3 Ajjy thc Al`A transtormation tound to oth sidcs ot thc

rcrcssion cquation (32) to otain
(L)
(L)
y
t

(L)
(L)
z
t
+ w
t
.
! lun an ordinary cast squarcs on thc transtormcd vaucs to o-
tain thc ncw
` lcturn to 2 it dcsircd
Ottcn, onc itcration is cnouh to dcvcoj thc cstimators undcr a rca-
sonac corrcation structurc ln cncra, thc Cochran-Orcutt jrocc-
durc convcrcs to thc maximum ikcihood or wcihtcd cast squarcs
cstimators
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 123
Example 2.12: \c miht considcr an atcrnativc ajjroach to
trcatin thc 1ohnson and 1ohnson carnins scrics, assumin that
y
t
o(x
t
)
1
+
2
t+e
t
ln ordcr to anayzc thc data with this aj-
jroach, rst wc t thc modc aovc, otainin

1
0.ooS(0.03!9)
and

2
0.0!1(0.001) Thc comjutcd rcsiduas e
t
y
t

2
t
can c comjutcd casiy, thc ACI and lACI arc shown in thc toj
two jancs ot Iiurc 312 `otc that thc ACI and lACI sucst
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
ACF
detrended
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
PACF
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
ARIMA(1,0,0,)_4
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
Figure 3.12: Autocorrelation functions (ACF) and partial autocorrelation functions (PACF)
for the detrended log J&J earnings series (top two panels)and the tted ARIMA(0, 0, 0)
(1, 0, 0)
4
residuals.
that a scasona Al scrics wi t wc and wc show thc ACI and
lACI ot thcsc rcsiduas in thc ottom jancs ot Iiurc 312 Thc
scasona Al modc is ot thc torm e
t

1
e
t4
+ w
t
and wc otain

1
0.o1!(0.0o39), with

2
w
0.009 sin thcsc vaucs, wc
transtorm y
t
to
y
t

1
y
t4

1
(1

1
) +
2
|t

1
(t !)| + w
t
usin thc cstimatcd vauc

1
0.o1! \ith this transtormcd rc-
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 124
rcssion, wc otain thc ncw cstimators

1
0.!SS(0.110`) and

2
0.0!2!(0.001S) Thc ncw cstimator has thc advantac ot cin
uniascd and havin a smacr cncraizcd variancc
To torccast, wc considcr thc oriina modc, with thc ncwy csti-
matcd

1
and

2
\c otain thc ajjroximatc torccast tor y
t
t+h

1
+

2
(t +h) +e
t
t+h
tor thc o transtormcd scrics, aon with ujjcr
and owcr imits dcjcndin on thc cstimatcd variancc that ony in-
corjoratcs thc jrcdiction variancc ot e
t
t+h
, considcrin thc trcnd and
scasona autorcrcssivc jaramctcrs as xcd Thc narrowcr ujjcr and
owcr imits (Thc urc is not jrcscntcd hcrc) arc mainy a rctcction
ot a sihty cttcr t to thc rcsiduas and thc aiity ot thc trcnd
modc to takc carc ot thc nonstationarity
Example 2:13: \c considcr thc rcationshij ctwccn two S
wccky intcrcst ratc scrics x
t
thc 1-ycar Trcasury constant maturity
ratc and y
t
thc 3-ycar Trcasury constant maturity ratc Loth scrics
havc 19o oscrvations trom 1anuary `, 19o2 to Scjtcmcr 10, 1999
and arc mcasurcd in jcrccntacs Thc scrics arc otaincd trom thc
Icdcra lcscrvc Lank ot St Louis
Iiurc 313 shows thc timc jots ot thc two intcrcst ratcs with
soid inc dcnotin thc 1-ycar ratc and dashcd inc tor thc 3-ycar
ratc Thc ctt janc ot Iiurc 31! jots y
t
vcrsus x
t
, indicatin that,
as cxjcctcd, thc two intcrcst ratcs arc hihy corrcatcd A naivc
way to dcscric thc rcationshij ctwccn thc two intcrcst ratcs is to
usc thc simjc modc, `odc l y
t

1
+
2
x
t
+ e
t
This rcsuts
in a ttcd modc y
t
0.911 + 0.92! x
t
+ e
t
, with

2
e
0.`3S and
R
2
9`.S/, whcrc thc standard crrors ot thc two coccicnts arc
0.032 and 0.00!, rcsjcctivcy This simjc modc (`odc l) conrms
thc hih corrcation ctwccn thc two intcrcst ratcs Lowcvcr, thc
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 125
1970 1980 1990 2000
4
6
8
1
0
1
2
1
4
1
6
Figure 3.13: Time plots of U.S. weekly interest rates (in percentages) from January 5, 1962
to September 10, 1999. The solid line (black) is the Treasury 1-year constant maturity rate
and the dashed line the Treasury 3-year constant maturity rate (red).
oo o oooo o
o o o
o o ooooo oo o o ooo
oooo o o o o o oo o o o oo ooooooo o oooo o oo o oooo oooo ooo o o ooooo oooooo o oooooo oo ooooo ooo ooooooo o o oo ooooo oooo oo o oo oooooo o o o oo ooooooo o oooooo o oo
oo o ooo ooooooooo o oooo ooo o ooo oo o ooooooooooooooo o ooooo ooo
o
ooo o o oooooooo o o o oo ooo oo
ooo o o
oo
oo oo
o
o
oo
oo o
o
o o o ooo o o o o
o
o o o
o
oo oo
oo
o o
o
o o o oo
ooooo
oo
oooo ooo o
o o
oo oooo oooo
o
o oooo ooo
o
o oo oo oooo
o
o o
oo
o
oooo
o
o
oo
o o
o
o o
o o oo o o oo o ooooooooo o
o
o
o
o oo
o o
oo o
ooo o o o oooo
ooo
o
ooo o
o
o o o
oo
o
o
o
o
o
o
oo
o
o
o
o
oo
o
oo o
o
oo
o o
oo
o
o
o
o
o
o oo
o
o o
o
o
o
oo oo
o
oo o
o
o o oooo
o
ooo
o
oo o o o
o
o
o
o
ooo o
ooo
o
o
o o
o
o oo
o
oo
o
o
o
o
o
oo
o
o o
o
oo
o o o
oooo
o o
ooo o
o o
o o
o
oo o
o
o o o o
o o
oo
oo o oooo
oo
o
o o o
o
ooo
oooo
oo o o o o oooo
o
ooooo oo o o o oooooooooo
o
oo o
ooooo
oo
o ooo ooooo oo
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o o oo
oo
o
o
o
o o
oooo
o
o
ooo
o
oo
o
o
oooo
o
o
o
oooo o
o
o
o
o
o
ooo
oo
o
o
o
o o
oo oo
o
o
oo o
o o
o o oo o
o
oo
o
o o o
o
o
o
o o
o o
o
o oo o
o
o
o
ooo
o
o
ooo
o o
o
o
o o
o
o
o
o
oo
ooo
o
o
o
o
o o o oo
ooo
o
oo
oo
o
oo
oo
o
o
oo
o o oo
o
o
o
o
o
o o o o o o oo
o
o
o
o oo
o
o
oo o oo
o
o
o
oo
o o
oo o oo o o
oo
o
ooo o o o
o o ooooo
ooo o ooo ooo
o
o oo o o ooooooo
o o o oooo o o o o
oo oo
ooooo o oo
o
oooo o
o
o
ooo oo
oooo
o
o
oo
oo
oo o
oo oo o
o
ooo
oo oo o o o
o ooo o o
o o
o
oo
oo
ooo ooo
o
o
oo o o
o
o
o
o
oo
o
o
oo
o
o oo oo
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o o o
o
o
o
oooo
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o o
o
o
o
o
o o
o
o
o
oo o
o o
o
o
o
o
o
o
o
o o
o
o
o
oo
o
o
o
o
o
o
o
o
oo
o o
o
o
o
o o o
o o o
oo
o
o
o
o
o
o
o
o o
oo
o oo
o
o
o
oo
o
ooo o
o o
o o o
o
o
o
o
oo
o
o
o
oo
oo
o
oo o
oo
o
o
o
o o
o o
o
o o
o
o
o
o
o
o
o o
o
o o
o
o
ooo o o
o
o
o o o o o
oo ooo
oo
o
o
o o
o
o
oo
o
o
o
o
o
oo
o
o o
o
o
o
o
o oo
o o
o
oo o
o
o
o
o
oo
o o
o o
oo
o oo
oooo
o
o
ooo
o o
o
o oo
o
o
o o
oo
o
o
oo
o
o
oo
o
oo
o
o o
oo
o
oo o
o o
ooo
o
ooo
o
o
o
ooo o
o
o
o o
o
o
o
o
o
o o
o
oo
o
o
o
o
o o
ooo
o
o
o
o
o
oo o o
o
oo ooo
o oooooo
o o ooo
o o ooo o
o
o
o
o
o
o
o
o
o
o o
o
oo o o o
o
o
o oo
o
o
o oo
o
o
o
o
oooooo
o o o oo o
o
o
o oo oo
o o
oo
o o
oo
o
oo
oo
o o
o oo
oooo
ooo
o
o ooo
o ooo o
o
o
ooo
o oo
o o
ooo
o
oo o o
o
o o
oo
o o o
o
o o
o
ooo
o
o
oo
o
o
o
o
o o o
oo
o
o
o o oo o
o oooo o
oo o
oo
oo o
o o
o
o o o oo
o
o o
o
o o o
o oo o oo
o o
o
oo
o o oooo
o
o o
o o
o
o oo
o
o o o
o o o o o
o o
o
oo
o
o o
o oo o oooo o
o
o o oo oo o
o
o
o
oo o
o o
o
o o oo
o
o o
o o
o
o o
o o o
o o
o oo
oo
o
o
o o
o
o o
oo
o
oo
oo
o
oo
o
o
ooo o
o
o
o
o
o oo
oo
o
oo
oooo
o
oo
o o o o o
ooo
o
o
o
o o oo o
o oo oo
o
ooo
oo
oo o
oo o o
o
o o o
oo oooo
o
o oooo oo ooo
o oo
o
oo
o
o
ooo
o
o o
oo
o
o
o
oo
oo
o
ooo
oo
o
oo o o
o
o
oo
o oo
oo
ooo
ooo ooo o o o
o o
o
o
oo
o ooo o
oo
o
o o
o
o o o
o
o o o
o
o oo
o o
o o o o
o
o o o o
o o oo o oo o
o o o
oo o o o
o
oo
oo oo
o
oo
oo
oo
o
o
o o oo
o
oo
o
ooo
o
o
o o
o
o o oo
o o oo oo
oo oooooo
o o o
o
oo
o
ooo oo
o o o oo
o o
oo o
oo o o
oo o
o oo
o o oooo
oooooo o oo o
o ooo ooo
oo o oo o oooo oo o o oo oo o oooo o oo
o
o
oo
o
o o o oo
oooo
oo o
o o o o oo
oo
o
o
o
o oo o
oooo
oo
oo
oo o
o o o
o o
oo
o
o o
o o
4 6 8 10 12 14 16
4
6
8
1
0
1
2
1
4
1
6
o o
o
o
oo o
o
oo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
oo
o o
o oo
o
o
oo
o
o
o
o
o
oo
oo
o o
oo
o
o
o o
o
o
o
o
o
o
ooo
o
o
o
o
o
oooo
o
o
o
o o
o o
o o
o
o
o
o
o o
o
o
oo
ooo oo
o
o
o
o o
o
o o o
ooo
o
o
o
o o
o
o
o
o o
o
ooo
o
o
ooo o
oo o o
o
oo
o
o
o
o o
o
o
oo o
oo o
o
o
o
o
o
o
o o
ooo
ooo
oo o o
oo
o
o o
o
o o
o
o
o
oo o
o
oo ooo
o ooo
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
ooo
o
o o
o o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
oo
o
oo
o
oo
o o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
oo
o
o o
o
o
o o
o o
o
o
o
o
oo
o
o
o
o
o
oo
o o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o o o o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
oo
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o o
o o
o
oo
o
o
o
o
o
o
o
o
o
oo
o
o
oo
o o
o o oo
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o o
o
o
o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o o o
o o
o
oo o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
oo
o
o o
o
o
o
o
o
o oo
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
oo
o
o
oo
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
oo
o
o o
o
o
o o
o
o
o
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o ooo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o o
o
oo
o
o
o
o
o
o
o
o oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o o
o
o
o o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o o
o
o
o
o
o
o oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
oo
o
oo
o
o
o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
oo
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
oo
o
o
o
o
oo
o
o
o
o
o
o
o
oo
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o o
o
o
o
o o
o
o o
o
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
1.5 0.5 0.5 1.0 1.5

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
1
.
5
Figure 3.14: Scatterplots of U.S. weekly interest rates from January 5, 1962 to September
10, 1999: the left panel is 3-year rate versus 1-year rate, and the right panel is changes in
3-year rate versus changes in 1-year rate.
modc is scriousy inadcquatc as shown y Iiurc 31`, which ivcs
thc timc jot and ACI ot its rcsiduas ln jarticuar, thc samjc
ACI ot thc rcsiduas is hihy sinicant and dccays sowy, showin
thc jattcrn ot a unit root nonstationary timc scrics
2
Thc chavior
ot thc rcsiduas sucsts that markcd dicrcnccs cxist ctwccn thc
two intcrcst ratcs sin thc modcrn cconomctric tcrminooy, it onc
2
We will discuss in detail on how to do unit root test later
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 126
1970 1980 1990 2000

1
.
5

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
Figure 3.15: Residual series of linear regression Model I for two U.S. weekly interest rates:
the left panel is time plot and the right panel is ACF.
assumcs that thc two intcrcst ratc scrics arc unit root nonstationary,
thcn thc chavior ot thc rcsiduas indicatcs that thc two intcrcst ratcs
arc not co-integrated. scc atcr chajtcrs tor discussion ot unit
root and co-integration ln othcr words, thc data tai to sujjort
thc hyjothcsis that thcrc cxists a on-tcrm cquiirium ctwccn thc
two intcrcst ratcs ln somc scnsc, this is not surjrisin ccausc thc
jattcrn ot invcrtcd yicd curvc did occur durin thc data sjan Ly
thc invcrtcd yicd curvc, wc mcan thc situation undcr which intcrcst
ratcs arc invcrscy rcatcd to thcir timc to maturitics
Thc unit root chavior ot oth intcrcst ratcs and thc rcsiduas
cads to thc considcration ot thc chanc scrics ot intcrcst ratcs Lct
x
t
y
t
y
t1
(1 L) x
t
c chancs in thc 1-ycar intcrcst
ratc and y
t
y
t
y
t1
(1 L) y
t
dcnotc chancs in thc 3-
ycar intcrcst ratc Considcr thc incar rcrcssion, `odc ll y
t

1
+
2
x
t
+ e
t
Iiurc 31o shows timc jots ot thc two chanc
scrics, whcrcas thc riht janc ot Iiurc 31! jrovidcs a scattcrjot
ctwccn thcm Thc chanc scrics rcmain hihy corrcatcd with a
ttcd incar rcrcssion modc ivcn y y
t
0.0002+0.S11 x
t
+e
t
with

2
e
0.0oS2 and R
2
S!.S/ Thc standard crrors ot thc two
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 127
1970 1980 1990 2000

1
.
5

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
1
.
5
Figure 3.16: Time plots of the change series of U.S. weekly interest rates from January 12,
1962 to September 10, 1999: changes in the Treasury 1-year constant maturity rate are in
denoted by black solid line, and changes in the Treasury 3-year constant maturity rate are
indicated by red dashed line.
coccicnts arc 0.001` and 0.00`, rcsjcctivcy This modc turthcr
conrms thc stron incar dcjcndcncc ctwccn intcrcst ratcs Thc
two toj jancs ot Iiurc 31 show thc timc jot (ctt) and samjc
ACI (riht) ot thc rcsiduas (`odc ll) Oncc aain, thc ACI shows
0 500 1000 1500 2000
0
.
4

0
.
2
0
.
0
0
.
2
0
.
4
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
0 500 1000 1500 2000
0
.
4

0
.
2
0
.
0
0
.
2
0
.
4
0 5 10 15 20 25 30

0
.
5
0
.
0
0
.
5
1
.
0
Figure 3.17: Residual series of the linear regression models: Model II (top) and Model III
(bottom) for two change series of U.S. weekly interest rates: time plot (left) and ACF (right).
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 128
somc sinicant scria corrcation in thc rcsiduas, ut thc manitudc
ot thc corrcation is much smacr This wcak scria dcjcndcncc in
thc rcsiduas can c modccd y usin thc simjc timc scrics modcs
discusscd in thc jrcvious scctions, and wc havc a incar rcrcssion
with timc scrics crrors
Thc main o,cctivc ot this scction is to discuss a simjc ajjroach
tor uidin a incar rcrcssion modc with timc scrics crrors Thc
ajjroach is straihttorward \c cmjoy a simjc timc scrics modc
discusscd in this chajtcr tor thc rcsidua scrics and cstimatc thc whoc
modc ,ointy Ior iustration, considcr thc simjc incar rcrcssion
in `odc ll Lccausc rcsiduas ot thc modc arc scriay corrcatcd, wc
idcntity a simjc Al`A modc tor thc rcsiduas Irom thc samjc
ACI ot thc rcsiduas shown in thc riht toj janc ot Iiurc 31,
wc sjccity an `A(1) modc tor thc rcsiduas and modity thc incar
rcrcssion modc to (`odc lll) y
t

1
+
2
x
t
+ e
t
and e
t

w
t

1
w
t1
, whcrc {w
t
} is assumcd to c a whitc noisc scrics ln
othcr words, wc simjy usc an `A(1) modc, without thc constant
tcrm, to cajturc thc scria dcjcndcncc in thc crror tcrm ot `odc
ll Thc two ottom jancs ot Iiurc 31 show thc timc jot (ctt)
and samjc ACI (riht) ot thc rcsiduas (`odc lll) Thc rcsutin
modc is a simjc cxamjc ot incar rcrcssion with timc scrics crrors
ln jracticc, morc caoratcd timc scrics modcs can c addcd to a
incar rcrcssion cquation to torm a cncra rcrcssion modc with
timc scrics crrors
Lstimatin a rcrcssion modc with timc scrics crrors was not casy
ctorc thc advcnt ot modcrn comjutcrs Sjccia mcthods such as thc
Cochranc-Orcutt cstimator havc ccn jrojoscd to handc thc scria
dcjcndcncc in thc rcsiduas Ly now, thc cstimation is as casy as
that ot othcr timc scrics modcs lt thc timc scrics modc uscd is
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 129
stationary and invcrtic, thcn onc can cstimatc thc modc ,ointy via
thc maximum ikcihood mcthod or conditiona maximum ikcihood
mcthod This is thc ajjroach wc takc y usin thc jackac R with
thc command arima() Ior thc S wccky intcrcst ratc data, thc
ttcd vcrsion ot `odc ll is y
t
0.0002 + 0.S2! x
t
+ e
t
and
e
t
w
t
+ 0.211` w
t1
with

2
w
0.0ooS and R
2
S`.!/ Thc
standard crrors ot thc jaramctcrs arc 0.001S, 0.00, and 0.0221,
rcsjcctivcy Thc modc no oncr has a sinicant a-1 rcsidua
ACI, cvcn thouh somc minor rcsidua scria corrcations rcmain at
as ! and o Thc incrcmcnta imjrovcmcnt ot addin additiona `A
jaramctcrs at as ! and o to thc rcsidua cquation is sma and thc
rcsut is not rcjortcd hcrc
Comjarin thc aovc thrcc modcs, wc makc thc toowin o-
scrvations Iirst, thc hih R
2
and coccicnt 0.92! ot `odc l arc
miscadin ccausc thc rcsiduas ot thc modc show stron scria cor-
rcations Sccond, tor thc chanc scrics, R
2
and thc coccicnt ot
x
t
ot `odc ll and `odc lll arc cosc ln this jarticuar in-
stancc, addin thc `A(1) modc to thc chanc scrics ony jrovidcs
a marina imjrovcmcnt This is not surjrisin ccausc thc csti-
matcd `A coccicnt is sma numcricay, cvcn thouh it is statis-
ticay hihy sinicant Third, thc anaysis dcmonstratcs that it
is imjortant to chcck rcsidua scria dcjcndcncc in incar rcrcssion
anaysis Lccausc thc constant tcrm ot `odc lll is insinicant, thc
modc shows that thc two wccky intcrcst ratc scrics arc rcatcd as
y
t
y
t1
+ 0.S2 (x
t
x
t1
) + w
t
+ 0.212 w
t1
Thc intcrcst ratcs
arc concurrcnty and scriay corrcatcd
Iinay, wc outinc a cncra jroccdurc tor anayzin incar rc-
rcssion modcs with timc scrics crrors Iirst, t thc incar rcrcs-
sion modc and chcck scria corrcations ot thc rcsiduas Sccond,
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 130
it thc rcsidua scrics is unit-root nonstationary, takc thc rst dicr-
cncc ot oth thc dcjcndcnt and cxjanatory variacs Go to stcj
1 lt thc rcsidua scrics ajjcars to c stationary, idcntity an Al`A
modc tor thc rcsiduas and modity thc incar rcrcssion modc ac-
cordiny Third, jcrtorm a ,oint cstimation via thc maximum ikc-
ihood mcthod and chcck thc ttcd modc tor turthcr imjrovcmcnt
To chcck thc scria corrcations ot rcsiduas, wc rccommcnd that
thc L,un-Lox statistics c uscd instcad ot thc Lurin-\atson (L\)
statistic ccausc thc attcr ony considcrs thc a-1 scria corrcation
Thcrc arc cascs in which rcsidua scria dcjcndcncc ajjcars at hihcr
ordcr as This is jarticuary so whcn thc timc scrics invovcd
cxhiits somc scasona chavior
Remark: Ior a rcsidua scrics e
t
with T oscrvations, thc Lurin-
\atson statistic is
DW
T

t=2
(e
t
e
t1
)
2
/
T

t=1
e
2
t
.
Straihttorward cacuation shows that DW 2(1

e
(1)), whcrc

e
(1) is thc a-1 ACI ot {e
t
}
3.10 Estimation of Covariance Matrix
Considcr aain thc rcrcssion modc in (32) Thcrc may cxist sit-
uations which thc crror e
t
has scria corrcations and,or conditiona
hctcrosccdasticity, ut thc main o,cctivc ot thc anaysis is to makc
intcrcncc conccrnin thc rcrcssion coccicnts \hcn e
t
has scria
corrcations, wc discusscd mcthods in Example 2.12 and Exam-
ple 2.13 aovc to ovcrcomc this dicuty Lowcvcr, wc assumc
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 131
that e
t
toows an All`A tyjc modc and this assumjtion miht
not c aways satiscd in somc ajjications Lcrc, wc considcr a
cncra situation without makin this assumjtion ln situations un-
dcr which thc ordinary cast squarcs cstimatcs ot thc coccicnts rc-
main consistcnt, mcthods arc avaiac to jrovidc consistcnt cstimatc
ot thc covariancc matrix ot thc coccicnts Two such mcthods arc
widcy uscd in cconomics and nancc Thc rst mcthod is cacd
hctcrosccdasticity consistcnt (LC) cstimator. scc Lickcr (19o) and
\hitc (19S0) Thc sccond mcthod is cacd hctcrosccdasticity and
autocorrcation consistcnt (LAC) cstimator. scc `cwcy and \cst
(19S)
To casc in discussion, wc sha rc-writc thc rcrcssion modc as
y
t

x
t
+ e
t
,
whcrc y
t
is thc dcjcndcnt variac, x
t
(x
1t
, , x
pt
)

is a p-
dimcnsiona vcctor ot cxjanatory variacs incudin constant and
acd variacs, and (
1
, ,
p
)

is thc jaramctcr vcctor

Thc LS cstimatc ot is ivcn y

_
_
n

t=1
x
t
x

t
_
_
1
n

t=1
x
t
y
t
,
and thc associatcd covariancc matrix has thc so-cacd sandwich
torm as

Cov(

)
_
_
n

t=1
x
t
x

t
_
_
1
C
_
_
n

t=1
x
t
x

t
_
_
1
if e
t
is iid

2
e
_
_
n

t=1
x
t
x

t
_
_
1
,
whcrc C is cacd thc mcat ivcn y
C \ar
_
_
n

t=1
e
t
x
t
_
_
,

2
e
is thc variancc ot e
t
and is cstimatcd y thc variancc ot rcsiduas
ot thc rcrcssion ln thc jrcscncc ot scria corrcations or conditiona
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 132
hctcrosccdasticity, thc jrior covariancc matrix cstimator is inconsis-
tcnt, ottcn rcsutin in inatin thc t-ratios ot

Thc cstimator ot \hitc (19S0) is ascd on toowin

,hc

_
_
n

t=1
x
t
x

t
_
_
1

C
hc
_
_
n

t=1
x
t
x

t
_
_
1
,
whcrc with

e
t
y
t

x
t
cin thc rcsidua at timc t,

C
hc

n
n p
n

t=1

e
2
t
x
t
x

t
.
Thc cstimator ot `cwcy and \cst (19S) is

,hac

_
_
n

t=1
x
t
x

t
_
_
1

C
hac
_
_
n

t=1
x
t
x

t
_
_
1
,
whcrc

C
hac
is ivcn y

C
hac

n

t=1

e
2
t
x
t
x

t
+
l

j=1
w
j
n

t=j+1
_
x
t

e
t

e
tj
x

tj
+ x
tj

e
tj

e
t
x

t
_
with l is a truncation jaramctcr and w
j
is wciht tunction such as
thc Larctt wciht tunction dcncd y w
j
1 j/(l + 1) Othcr
wciht tunction can aso uscd `cwcy and \cst (19S) sucstcd
choosin l to c thc intccr jart ot !(n/100)
2/9
This cstimator
csscntiay uscs a nonjaramctric mcthod to cstimatc thc covariancc
matrix ot

n
t=1
e
t
x
t
and a cass ot kcrnc-ascd hctcroskcdasticity and
autocorrcation consistcnt (LAC) covariancc matrix cstimators was
introduccd y Andrcws (1991)
Example 2.14: (Continuation of Example 2.13) Ior iustration,
wc considcr thc rst dicrcnccd intcrcst ratc scrics in `odc ll in
Example 2.13 Thc t-ratio ot thc coccicnt ot x
t
is 10!.o3 it
oth scria corrcation and conditiona hctcrosccdasticity in rcsiduas
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 133
arc inorcd. it ccomcs !o.3 whcn thc LC cstimator is uscd, and it
rcduccs to !0.0S whcn thc LAC cstimator is cmjoycd To usc LC
or LAC cstimator, wc can usc thc jackac sandwich in R and thc
commands arc vcovHC() or vcovHAC()
3.11 Long Memory Models
\c havc discusscd that tor a stationary timc scrics thc ACI dccays
cxjoncntiay to zcro as a incrcascs Yct tor a unit root nonsta-
tionary timc scrics, it can c shown that thc samjc ACI convcrcs
to 1 tor a xcd as as thc samjc sizc incrcascs. scc Chan and \ci
(19SS) and Tiao and Tsay (19S3) Thcrc cxist somc timc scrics whosc
ACI dccays sowy to zcro at a joynomia ratc as thc a incrcascs
Thcsc jroccsscs arc rctcrrcd to as long memory or long range
dependent timc scrics Onc such an cxamjc is thc fractionally
dierenced process dcncd y
(1 L)
d
x
t
w
t
, |d| < 0.`, (33)
whcrc {w
t
} is a whitc noisc scrics and d is cacd thc on mcm-
ory jaramctcr lrojcrtics ot modc (33) havc ccn widcy studicd
in thc itcraturc (c, Lcran, 199!) \c summarizc somc ot thcsc
jrojcrtics cow
1 lt d < 0.`, thcn x
t
is a wcaky stationary jroccss and has thc
innitc `A rcjrcscntation
x
t
w
t
+

k=1

k
w
tk
with
k
d(d+1) (d+k1)/k'
_
_
_
k + d 1
k
_
_
_
2 lt d > 0.`, thcn x
t
is invcrtic and has thc innitc Al rcjrc-
scntation
x
t
w
t
+

k=1

k
w
tk
with
k
(0d)(1d) (k1d)/k'
_
_
_
k d
k
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 134
3 Ior |d| < 0.`, thc ACI ot x
t
is

x
(h)
d(1 + d) (h 1 + d)
(1 d)(2 d) (h d)
, h 1.
ln jarticuar,
x
(1) d/(1 d) and as h ,

x
(h)
(d)'
(d 1)'
h
2d1
.
! Ior |d| < 0.`, thc lACI ot x
t
is
h,h
d/(h d) tor h 1
` Ior |d| < 0.`, thc sjcctra dcnsity tunction f
x
() ot x
t
, which is
thc Iouricr transtorm ot thc ACI
x
(h) ot x
t
, that is
f
x
()
1
2

h=

x
(h) cxj(i h)
tor |, |, whcrc i

1, satiscs
f
x
()
2d
as 0, (33S)
whcrc |0, | dcnotcs thc trcqucncy Scc Chajtcr o ot Lami-
ton (199!) tor dctais aout thc sjcctra anaysis
Ot jarticuar intcrcst hcrc is thc chavior ot ACI ot x
t
whcn d < 0.`
Thc jrojcrty says that
x
(h) h
2d1
, which dccays at a joyno-
mia, instcad ot cxjoncntia ratc Ior this rcason, such an x
t
jroccss
is cacd a on-mcmory timc scrics A sjccia charactcristic ot thc
sjcctra dcnsity tunction in (33S) is that thc sjcctrum divcrcs to
innity as 0 Lowcvcr, thc sjcctra dcnsity tunction ot a sta-
tionary Al`A jroccss is oundcd tor a |, |
Laricr wc uscd thc inomia thcorcm tor non-intccr jowcrs
(1 L)
d

k=0
(1)
k
_
_
_
d
k
_
_
_ L
k
.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 135
lt thc tractionay dicrcnccd scrics (1L)
d
x
t
toows an Al`A(p, q)
modc, thcn x
t
is cacd an tractionay dicrcnccd autorcrcssivc
movin avcrac (AlIl`A(p, d, q)) jroccss, which is a cncraizcd
All`A modc y aowin tor non-intccr d ln jracticc, it thc sam-
jc ACI ot a timc scrics is not arc in manitudc, ut dccays sowy,
thcn thc scrics may havc on mcmory Ior morc discussions, wc
rctcr to thc ook y Lcran (199!) Ior thc jurc tractionay dicr-
cnccd modc in (33), onc can cstimatc d usin cithcr a maximum
ikcihood mcthod in thc timc domain or thc \hittc ikcihood or
a rcrcssion mcthod with ocd jcriodoram at thc owcr trcqucn-
cics in thc trcqucncy domain Iinay, on-mcmory modcs havc
attractcd somc attcntion in thc nancc itcraturc in jart ccausc
ot thc work on tractiona Lrownian motion in thc continuous timc
modcs
0 100 200 300 400

0
.
1
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
ACF for valueweighted index
0 100 200 300 400

0
.
1
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
ACF for equalweighted index
0.00 0.02 0.04 0.06

1
2

1
1

1
0

6
Log Spectral Density of VW
0.00 0.02 0.04 0.06

1
3

1
1

6
Log Spectral Density of EW
Figure 3.18: Sample autocorrelation function of the absolute series of daily simple returns for
the CRSP value-weighted (left top panel) and equal-weighted (right top panel) indexes. The
log spectral density of the absolute series of daily simple returns for the CRSP value-weighted
(left bottom panel) and equal-weighted (right bottom panel) indexes.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 136
Example 2.15: As an iustration, Iiurc 31S show thc samjc
ACIs ot thc asoutc scrics ot daiy simjc rcturns tor thc ClSl
vauc-wcihtcd (ctt toj janc) and cqua-wcihtcd (riht toj janc)
indcxcs trom 1uy 3, 19o2 to Lcccmcr 31, 199 Thc ACIs arc rc-
ativcy sma in manitudc, ut dccay vcry sowy. thcy ajjcar to c
sinicant at thc `/ cvc cvcn attcr 300 as Ior morc intormation
aout thc chavior ot samjc ACI ot asoutc rcturn scrics, scc Lin,
Grancr, and Lnc (1993) To cstimatc thc on mcmory jaramctcr
cstimatc d, wc can usc thc jackac fracdi in R and rcsuts arc

d 0.1So tor thc asoutc rcturns ot thc vauc-wcihtcd indcx and

d 0.232 tor thc asoutc rcturns ot thc cqua-wcihtcd indcx To

sujjort our concusion aovc, wc jot thc o sjcctra dcnsity ot thc
asoutc scrics ot daiy simjc rcturns tor thc ClSl vauc-wcihtcd
(ctt ottom janc) and cqua-wcihtcd (riht ottom janc) Thcy
show ccary that oth o sjcctra dcnsitics dccay ikc a o tunction
and thcy sujjort thc sjcctra dcnsitics chavior ikc (33S)
3.12 Periodicity and Business Cycles
Lct us rst rcca what wc havc oscrvcd trom trom Iiurc 21 in
Chajtcr 2 Irom Iiurc 21, wc can concudc that oth scrics in tcnd
to cxhiit rcjctitivc chavior, with rcuary rcjcatin (stochastic)
cycles or periodicity that arc casiy visic This jcriodic c-
havior is ot intcrcst ccausc undcryin jroccsscs ot intcrcst may c
rcuar and thc ratc or trcqucncy ot timc scrics charactcrizin thc c-
havior ot thc undcryin scrics woud hcj to idcntity thcm Onc can
aso rcmark that thc cyccs ot thc SOl arc rcjcatin at a tastcr ratc
than thosc ot thc rccruitmcnt scrics Thc rccruits scrics aso shows
scvcra kinds ot osciations, a tastcr trcqucncy that sccms to rcjcat
aout cvcry 12 months and a sowcr trcqucncy that sccms to rcjcat
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 137
aout cvcry `0 months Thc study ot thc kinds ot cyccs and thcir
strcnths arc aso vcry imjortant, jarticuary in macrocconomics to
dctcrminc thc business cycles Ior morc discussions, wc rctcr to
thc ooks y Iranscs (199o, 199S) and Ghyscs and Osorn (2001)
As wc mcntion in Chajtcr 2, onc way to idcntity thc cyccs is to com-
jutc thc power spectrum which shows thc variancc as a tunction
ot thc trcqucncy ot osciation Ior othcr modcin mcthods such as
jcriodic autorcrcssivc modc (PAR), lAl(p) modcin tcchniqucs,
wc rctcr to thc ooks y Iranscs (199o, 199S) and Ghyscs and Os-
orn (2001) tor dctais `cxt, wc introducc onc mcthod to dcscric
thc cycica chavior usin thc ACI
As indicatcd aovc, thcrc cxits cycica chavior tor thc rccruits
Irom Example 2.4, an Al(2) ts this scrics quitc wc Thcrc-
torc, wc considcr thc ACI
x
(h) ot a stationary Al(2) scrics, which
satiscs thc sccond ordcr dicrcncc cquation
(1
1
L
2
L
2
)
x
(h) (L)
x
(h) 0
tor h 2,
x
(0) 1, and
x
(1)
1
/(1
2
) This dicrcncc
cquation dctcrmincs thc jrojcrtics ot thc ACI ot a stationary Al(2)
timc scrics lt aso dctcrmincs thc chavior ot thc torccasts ot x
t

Corrcsjondin to thc jrior dicrcncc cquation, thcrc is a sccond

ordcr joynomia cquation
1
1
x
2
x
2
0.
Soutions ot this cquation arc
x

1

2
1
+ !
2
2
2
.
ln thc timc scrics itcraturc, thc invcrsc ot two soutions arc rctcrrcd
as thc characteristic roots ot thc Al(2) modc Lcnotc thc
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 138
two soutions y
1
and
2
lt oth
i
arc rca vaucd, thcn thc
sccond ordcr dicrcncc cquation ot thc modc can c tactorcd as
(1
1
L)(1
2
L) and thc Al(2) modc can c rcardcd as an
Al(1) modc ojcratcs on toj ot anothcr Al(1) modc Thc ACI ot
x
t
is thcn a mixturc ot two cxjoncntia dccays Yct it
2
1
+!
2
< 0,
thcn
1
and
2
arc comjcx numcrs (cacd a complex conjugate
pair), and thc jot ot ACI ot x
t
woud show a jicturc ot damping
sine and cosine waves. scc thc toj two jancs ot Iiurc 21! tor
thc ACI ot thc SOl and rccruits ln usincss and cconomic ajji-
cations, comjcx charactcristic roots arc imjortant Thcy ivc risc
to thc chavior ot usincss cyccs lt is thcn common tor cconomic
timc scrics modcs to havc comjcx vaucd charactcristic roots Ior
an Al(2) modc with a jair ot comjcx charactcristic roots, thc av-
erage length of the stochastic cycles is
T
0

2
cos
1
|
1
/(2

2
)|
,
whcrc thc cosinc invcrsc is statcd in radians lt onc wishcs thc com-
jcx soutions as a b i, thcn wc havc
1
2 a,
2

a
2
+ b
2
,
and
T
0

2
cos
1
(a/

a
2
+ b
2
)
,
whcrc

a
2
+ b
2
is thc asoutc vauc ot a b i
To iustratc thc aovc idca, Iiurc 319 shows thc ACI ot tour
stationary Al(2) modcs Thc riht toj janc is thc ACI ot thc
Al(2) modc (10.o L+0.! L
2
) x
t
w
t
Lccausc
2
1
+!
2
1.0+
! (0.) 1.S < 0, this jarticuar Al(2) modc contains two
comjcx charactcristic roots, and hcncc its ACI cxhiits damjin
sinc and cosinc wavcs Thc othcr thrcc Al(2) modcs havc rca-
vaucd charactcristic roots Thcir ACIs dccay cxjoncntiay
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 139
5 10 15 20

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
(a)
5 10 15 20

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
(b)
5 10 15 20

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
(c)
5 10 15 20

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
(d)
Figure 3.19: The autocorrelation function of an AR(2) model: (a)
1
= 1.2 and
2
= 0.35,
(b)
1
= 1.0 and
2
= 0.7, (c)
1
= 0.2 and
2
= 0.35, (d)
1
= 0.2 and
2
= 0.35.
Example 2.16: As an iustration, considcr thc quartcry rowth
ratc ot S rca ross nationa jroduct (G`l), scasonay ad,ustcd,
trom thc sccond quartcr ot 19! to thc rst quartcr ot 1991, which
is shown in thc ctt janc ot Iiurc 320 Thc riht janc ot Iiurc
1950 1960 1970 1980 1990

0
.
0
2

0
.
0
1
0
.
0
0
0
.
0
1
0
.
0
2
0
.
0
3
0
.
0
4
0 5 10 15 20 25 30
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
Figure 3.20: The growth rate of US quarterly real GNP from 1947.II to 1991.I (seasonally
adjusted and in percentage): the left panel is the time series plot and the right panel is the
ACF.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 140
320 disjays thc ACI ot this scrics and it shows that a jicturc ot
damjin sinc and cosinc wavcs \c can concudc that cyccs cxist
This scrics can c uscd as an cxamjc ot nonincar cconomic timc
scrics. scc Tsay (200`, Chajtcr !) tor thc dctaicd anayscs usin thc
`arkov switchin modc Lcrc wc simjy cmjoy an Al(3) modc
tor thc data Lcnotin thc rowth ratc y x
t
, wc can usc thc modc
uidin jroccdurc to cstimatc thc modc Thc ttcd modc is
x
t
0.00!+0.3` x
t1
+0.1S x
t2
0.1! x
t3
+w
t
, with

2
w
0.009S.
lcwritin thc modc as x
t
0.3` x
t1
0.1S x
t2
+ 0.1! x
t3

0.00! + w
t
, wc otain a corrcsjondin third-ordcr dicrcncc cqua-
tion (1 0.3` L 0.1S L
2
+ 0.1! L
3
) 0, which can c tactorcd as
(1 + 0.`2 L)(1 0.S L + 0.2 L
2
) 0 Thc rst tactor (1 + 0.`2 L)
shows an cxjoncntiay dccayin tcaturc ot thc G`l rowth ratc
Iocusin on thc sccond ordcr tactor 1 0.S L(0.2) L
2
0, wc
havc
2
1
+ !
2
0.S
2
+ !(0.2) 0.3231 < 0 Thcrctorc, thc
sccond tactor ot thc Al(3) modc conrms thc cxistcncc ot stochas-
tic usincss cyccs in thc quartcry rowth ratc ot S rca G`l
This is rcasonac as thc S cconomy wcnt throuh cxjansion and
contraction jcriods Thc avcrac cnth ot thc stochastic cyccs is
ajjroximatcy
T
0

2
cos
1
|
1
/(2

2
)|
10.S3 quartcrs,
which is aout 3 ycars lt onc uscs a nonincar modc to scjaratc
S cconomy into cxjansion and contraction jcriods, thc data
show that thc avcrac duration ot contraction jcriods is aout thrcc
quartcrs and that ot cxjansion jcriods is aout 3 ycars Thc avcrac
duration ot 10.S3 quartcrs is a comjromisc ctwccn thc two scjaratc
durations Thc jcriodic tcaturc otaincd hcrc is common amon
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 141
rowth ratcs ot nationa cconomics Ior cxamjc, simiar tcaturcs
can c tound tor othcr countrics
Ior a stationary Al(p) scrics, thc ACI satiscs thc dicrcncc
cquation (1
1
L
2
L
2

p
L
p
)
x
(h) 0, tor h > 0 Thc
jot ot ACI ot a stationary Al(p) modc woud thcn show a mixturc
ot damping sine and cosine patterns and cxjoncntia dccays
dcjcndin on thc naturc ot its charactcristic roots
Iinay, wc continuc our anaysis ot thc rccruits scrics as cntcr-
taincd in Example 2.4, trom which an Al(2) modc is ttcd tor
this scrics as
x
t
1.3`12 x
t1
+ 0.!o12 x
t2
o1.S!39 + w
t
.
Ccary,
2
1
+!
2
1.3`12
2
! 0.!o12 0.0191 < 0, which im-
jics thc cxistcncc ot stochastic usincss cyccs in thc rccruits scrics
Thc avcrac cnth ot thc stochastic cyccs ascd on thc aovc ttcd
Al(2) modc is ajjroximatcy
T
0

2
cos
1
|
1
/(2

2
)|
o1.1 months,
which is aout ` ycars `otc that this avcrac cnth ot cyccs is not
cosc to what wc havc oscrvcd (aout `0 months) lcasc urc out
thc rcason why thcrc is a i dicrcncc
3.13 Impulse Response Function
Thc task tacin thc modcrn timc-scrics cconomctrician is to dcvcoj
rcasonay simjc and intuitivc modcs cajac ot torccastin, intcr-
jrctin and hyjothcsis tcstin rcardin cconomic and nancia data
ln this rcsjcct, thc timc scrics cconomctrician is ottcn conccrncd with
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 142
thc cstimation ot dicrcncc cquations containin stochastic comjo-
ncnts
3.13.1 First Order Dierence Equations
Sujjosc wc arc ivcn thc dynamic cquation y
t

1
y
t1
+w
t
, whcrc
y
t
is thc vauc ot variac at jcriod t, w
t
is thc vauc ot variac at
jcriod t, 1 t n lndccd, this is an Al(1) modc Thc cquation
rcatcs a variac y
t
to its jrcvious (acd) vaucs with ony thc rst
a ajjcars on thc riht hand sidc (lLS) ot thc cquation Ior now,
thc injut variac, {w
1
, w
2
, w
3
, . . .}, wi simjy c rcardcd as
a scqucncc ot dctcrministic numcrs which can c cncratcd trom a
distriution Latcr on, wc wi assumc that thcy arc stochastic Ly
sovin a dicrcncc cquation y rccursivc sustitution, assumin that
wc know thc startin vauc ot y
1
, cacd thc initia condition, thcn,
wc havc
y
0

1
y
1
+ w
0
,
y
1

1
y
0
+ w
1

(

1
y
1
+ w
0
) + w
1

2
1
y
1
+
1
w
0
+ w
1
,
y
2

1
y
1
+ w
2

1
(
2
1
y
1
+
1
w
0
+ w
1
) + w
2

3
1
y
1
+
2
1
w
0
+
1
w
1
+

y
t

1
y
t1
+ w
t

t+1
1
y
1
+
t
1
w
0
+
t1
w
1
+ +
1
w
t1
+ w
t

t+1
1
y
1
+
t

k=0

k
1
w
tk
. (339)
Thc jroccdurc to cxjrcss y
t
in tcrm ot thc jast vaucs ot w
t
and thc
startin vauc y
1
is known as recursive substitution Thc ast
tcrm on lLS ot (339) is cacd thc incar jroccss (with nitc tcrms)
cncratcd y {w
t
} it {w
t
} is random
`cxt wc considcr thc comjutation ot dynamic multiplier To
do so, wc considcr onc simjc cxjcrimcnt y assumin that y
1
and
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 143
{w
0
, w
1
, , w
n
} arc ivcn, xcd at this momcnt, and thc vauc
ot
1
is known Thcn wc can comjutc thc timc scrics y
t
y y
t

1
y
t1
+ w
t
tor 1 t n \hat hajjcns to thc scrics y
t
it at thc
jcriod t `0, wc chanc thc vauc ot dctcrministic comjoncnt w
50
and sct thc ncw w to c

w
50
w
50
+ 1 Ot coursc, thc chanc in
w
50
cads to chancs in y
t

y
50

1
y
49
+

w
50

1
y
49
+ w
50
+ 1 y
50
+ 1,

y
51

1
y
50
+ w
51

1
(y
50
+ 1) + w
51
y
51
+
1
,

y
52

1

y
51
+ w
52

1
(y
51
+
1
) + w
52
y
52
+
2
1
,
and so on lt wc ook at thc dicrcncc ctwccn

y
t
and y
t
thcn wc
otain that

y
50
y
50
1,

y
51
y
51

1
,

y
52
y
52

2
1
, and so on
This cxjcrimcnt is iustratcd in Iiurcs 321 and 322 Thcrctorc,
0 20 40 60 80 100

2
0
2
4
No Impulse
With Impulse
phi=0.8
0 20 40 60 80 100

2
0
2
No Impulse
With Impulse
phi= 0.8
0 20 40 60 80 100

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
Impulse Response Function
phi=0.8
0 20 40 60 80 100

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
Impulse Response Function
phi= 0.8
Figure 3.21: The time-series y
t
is generated with w
t
N(0, 1), y
0
= 5. At period t = 50,
there is an additional impulse to the error term, i.e. w
50
= w
50
+ 1. The impulse response
function is computed as the dierence between the series y
t
without impulse and the series
y
t
with the impulse.
onc can say that onc unit incrcasc in w
50
cads to y
50
incrcascd y 1,
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 144
0 20 40 60 80 100
5
1
0
1
5
2
0
2
5
3
0
No Impulse
With Impulse
phi=1.01
0 20 40 60 80 100

2
0

1
0
0
1
0
2
0
No Impulse
With Impulse
phi= 1.01
0 20 40 60 80 100

1
0
1
2
Impulse Response Function
phi=1.01
0 20 40 60 80 100

1
0
1
2
Impulse Response Function
phi= 1.01
Figure 3.22: The time-series y
t
is generated with w
t
N(0, 1), y
0
= 3. At period t = 50,
there is an additional impulse to the error term, i.e. w
50
= w
50
+ 1. The impulse response
function is computed as the dierence between the series y
t
without impulse and the series
y
t
with the impulse.
y
51
incrcascd y
1
, y
52
incrcascd y
2
1
, and so so Thc qucstion is
how y
t+j
chancs it chanc w
t
y onc This is cxacty thc qucstion
that dynamic mutijicrs answcr
Assumc that wc start with y
t1
instcad ot y
1
, ic wc oscrvc thc
vauc ot y
t1
at jcriod t 1 Can wc say somcthin aout y
t+j

\c, ct us answcr this qucstion

y
t

1
y
t1
+ w
t
,
y
t+1

1
y
t
+ w
t+1

1
(
1
y
t1
+ w
t
) + w
t+1

2
1
y
t1
+
1
w
t
+ w
t+1
,

y
t+j

j+1
1
y
t1
+
j
1
w
t
+
j1
w
t+1
+ +
1
w
t+j1
+ w
t+j
Thc dynamic multiplier is dcncd y
y
t+j
w
t

j
1
tor an Al(1) modc.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 145
`otc that thc mutijicr docs not dcjcnd on t Thc dynamic muti-
jicr y
t+j
/w
t
is cacd somctimcs thc impact multiplier
Impulse Response Function
\c can jot dynamic mutijicrs as a tunction ot a j, ic jot
{
y
t+j
w
t
}
J
j=1
Lccausc dynamic mutijicrs cacuatc thc rcsjonsc ot
y
t+j
to a sinc imjusc in w
t
, it is aso rctcrrcd to as thc impulse
response function (IRF) This tunction has many imjortant
ajjications in timc scrics anaysis ccausc it shows how thc cntirc
jath ot a variac is ccctcd y a stochastic shock Oviousy, thc
dynamic ot imjusc rcsjonsc tunction dcjcnds on thc vauc ot
1
tor
an Al(1) modc Lct us ook at what thc rcationshij is ctwccn thc
llI and systcm
(a) 0 <
1
< 1 imjics that thc imjusc rcsjonsc convcrcs to
zcro and thc systcm is stac
() 1 <
1
< 0 ivcs that thc imjusc rcsjonsc osciatcs ut
convcrcs to zcro and thc systcm is stac
(c)
1
> 1 concudcs that thc imjusc rcsjonsc is cxjosivc and
thc systcm is unstac
(d)
1
< 1 imjics that thc imjusc rcsjonsc is cxjosivc (os-
ciationay) and thc systcm is unstac
lmjusc rcsjonsc tunctions tor a jossic cascs arc jrcscntcd in
Iiurc 323
Permanent Change in w
t
ln cacuatin dynamic mutijicrs in Iiurc 323, wc wcrc askin
what woud hajjcn it w
t
wcrc to incrcasc y onc unit with w
t+1
, w
t+2
, , , w
t+
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 146
o
o
o
o
o
o o o o o o o o o o o o o o o
5 10 15 20
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o o o o o o o o o o o o o o o o o o o
phi1=0.5 phi1=0.9
phi1=0.99
o
o
o
o
o
o
o o o o o o o o o o o o o o
5 10 15 20

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
phi1= 0.5 phi1= 0.9
phi1= 0.99
o o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
5 10 15 20
0
1
0
2
0
3
0
phi1=1.2
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
5 10 15 20

3
0

1
0
0
1
0
2
0
3
0
4
0
phi1= 1.2
Figure 3.23: Example of impulse response functions for rst order dierence equations.
unacctcd \c wcrc ndin thc ccct ot a jurcy transitory change
in w
t
lcrmancnt chanc in w
t
mcans that w
t
, w
t+1
, , , w
t+j
woud a incrcasc y onc unit Thc ccct on y
t+j
ot a permanent
change in w
t
cinnin in jcriod t is thcn ivcn y
y
t+j
w
t
+
y
t+j
w
t+1
+ +
y
t+j
w
t+j

j
1
+
j1
1
+ +
1

1
j+1
1
1
1
tor thc Al(1) mo
Thc dicrcnt ctwccn transitory and jcrmancnt chanc in w
t
is i-
ustratcd in Iiurc 32!
3.13.2 Higher Order Dierence Equations
Considcr thc modc A incar pth ordcr dicrcncc cquation has thc
toowin torm y
t

1
y
t1
+
2
y
t2
+ +
p
y
tp
+ w
t
, which
is an Al(p) modc it w
t
is thc whitc noisc lt is casy to drivc thc
jrojcrtics ot thc pth ordcr dicrcncc cquation To iustratc thc
jrojcrtics ot pth ordcr dicrcncc cquation, wc considcr thc sccond
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 147
0 20 40 60 80 100

1
0
1
2
No Impulse
With Impulse
phi=0.8
0 20 40 60 80 100

2
0
2
4
No Impulse
With Impulse
phi=0.8
0 20 40 60 80 100
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
Impulse Response Function
phi=0.8
0 20 40 60 80 100
0
1
2
3
4
5
Impulse Response Function
phi=0.8
Figure 3.24: The time series y
t
is generated with w
t
N(0, 1), y
0
= 3. For the transitory
impulse, there is an additional impulse to the error term at period t = 50, i.e. w
50
= w
50
+1.
For the permanent impulse, there is an additional impulse for period t = 50, , 100, i.e.
w
t
= w
t
+ 1, t = 50, 51, , 100. The impulse response function (IRF) is computed as the
dierence between the series y
t
without impulse and the series y
t
with the impulse.
ordcr dicrcncc cquation y
t

1
y
t1
+
2
y
t2
+w
t
so that p 2
\c writc cquation as a rst ordcr vcctor dicrcncc cquation

t
F
t1
+ v
t
,
whcrc

t

_
_
y
t
y
t1
_
_
, F
_
_

1

2
1 0
_
_
, and v
t

_
_
w
t
0
_
_
.
lt thc startin vauc
1
is known wc usc thc rccursivc sustitution
to otain

t
F
t+1

1
+ F
t
v
0
+ F
t1
v
1
+ + Fv
t1
+ v
t
.
Or it thc startin vauc
t1
is known wc usc thc rccursivc sustitu-
tion to otain

t+j
F
j+1

t1
+ F
j
v
t
+ F
j1
v
t+1
+ + Fv
t+j1
+ v
t+j
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 148
tor any j 0
To comjutc thc dynamic mutijicrs, wc rcca thc rucs ot ot vcctor
and matrix dicrcntiation lt x() is an m1 vcctor that dcjcnds
on thc n 1 vcctor , thcn,
x

mn

_
_
_
_
_
_
_
x
1
()

1

x
1
()

x
m
()

1

x
m
()

n
_
_
_
_
_
_
_
.
lt is casy to vcrity that thc dynamic mutijicrs arc ivcn y

t+j
v

F
j
v
t
v

t
F
j
.
Sincc thc rst ccmcnt in
t+j
is y
t+j
and thc rst ccmcnt in v
t
is
w
t
thcn thc ccmcnt(1, 1) in thc matrix F
j
is y
t+j
/w
t
, ic thc
dynamic mutijicr Ior arcr vaucs ot j, an casy way to otain a
numcrica vaucs tor dynamic mutijicr y
t+j
/w
t
is to simuatc thc
systcm This is donc as toows Sct y
1
y
2
y
p
0 and
w
0
1, and sct thc vaucs ot w tor a othcr datcs to 0 Thcn usc
Al(p) to cacuatc thc vauc tor y
t
tor t n
To iustratc dynamic mutijicrs tor thc pth ordcr dicrcncc cqua-
tion, wc considcr thc sccond ordcr dicrcncc cquation as toows
y
t

1
y
t1
+
2
y
t2
+ w
t
, so that F
_
_

1

2
1 0
_
_
Thc imjusc
rcsjonsc tunctions tor this cxamjc with tour dicrcnt scttins arc
jrcscntcd in Iiurc 32`
(a)
1
0.o and
2
0.2,
()
1
0.S and
2
0.!,
(c)
1
0.9 and
2
0.`, and
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 149
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o o o o o o
0 5 10 15 20
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
(a)
Impulse Response Function
phi1=0.6 and phi2=0.2
o
o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
0 5 10 15 20
0
2
4
6
8
1
0
1
2
(b)
Impulse Response Function
phi1=0.8 and phi2=0.4
o
o
o
o
o
o
o
o
o
o
o
o
o o o o o o o o o
0 5 10 15 20

1
.
0

0
.
5
0
.
0
0
.
5
1
.
0
(c)
Impulse Response Function
phi1= 0.9 and phi2= 0.5
o
o o
o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
0 5 10 15 20

4
0

2
0
0
1
0
2
0
3
0
(d)
Impulse Response Function
phi1= 0.5 and phi2= 1.5
Figure 3.25: Example of impulse response functions for second order dierence equation.
(d)
1
0.` and
2
1.`
Simiar to thc rst ordcr dicrcncc cquation, thc imjusc rcsjonsc
tunction tor thc pth ordcr dicrcncc cquation can c cxjosivc or con-
vcrc to zcro \hat dctcrmincs thc dynamics ot imjusc rcsjonsc
tunction Thc cicnvaucs ot matrix F dctcrminc whcthcr thc im-
jusc rcsjonsc osciatcs, convcrcs or is cxjosivc Thc cicnvaucs
ot a p p matrix A arc thosc numcrs tor which
|AI
p
| 0.
Thc cicnvaucs ot thc cncra matrix F dcncd aovc arc thc vaucs
ot that satisty

p1

p2

p1

p
(
p
) 0.
`otc that thc cicnvaucs arc aso cacd thc characteristic roots
ot Al(p) modc lt thc cicnvaucs arc rca ut at cast onc cicn-
vauc is rcatcr than unity in asoutc vauc, thc systcm is cxjosivc
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 150
\hy do cicnvaucs dctcrminc thc dynamics ot dynamic mutijicrs
lcca that it thc cicnvaucs ot p p matrix A arc distinct, thcrc
cxists a nonsinuar p p matrix T such that A TT
1
, whcrc
is a p p matrix with thc cicnvaucs ot A on thc jrincija di-
aona and zcros cscwhcrc Thcn, F
j
T
j
T
1
and thc vauc ot
cicnvaucs in dctcrmincs whcthcr thc ccmcnts ot F
j
cxjodc or
not lcca that thc dynamic mutijicr is cqua to
t+j
/v

t
F
j

Thcrctorc, thc sizc ot cicnvaucs dctcrmincs whcthcr thc systcm is

stac or not
`ow wc comjutc thc thc cicnvaucs tor cach casc in thc aovc
cxamjc
(a)
1
0.S3S and
2
0.23S so that |
k
| < 1 and thc
systcm is stac
()
1
1.1!S and
2
0.3!S so that |
1
| > 1 and thc
systcm is unstac
(c) 0.!` 0.`!` i so that ||
_
(0.!`)
2
+ 0.`!`
2

0.0o < 1 and thc systcm is stac Sincc cicnvaucs arc

comjcx, thc imjusc rcsjonsc tunction osciatcs
(d) 0.2` 1.19S i so that ||
_
(0.2`)
2
+ 1.19S
2

1.223 > 1 and thc systcm is unstac Sincc cicnvaucs arc

comjcx, thc imjusc rcsjonsc tunction osciatcs
3.14 Problems
1. Considcr thc rcrcssion modc y
t

1
y
t1
+ w
t
, whcrc w
t
is
whitc noisc with zcro-mcan and variancc
2
w
Assumc that wc
oscrvc {y
t
}
n
t=2
Show that thc cast squarcs cstimator ot
1
is

1


n
t=2
y
t
y
t1
/

t=2
y
2
t1
lt wc jrctcnd that y
t1
woud c
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 151
xcd, show that \ar(

1
)
2
e
/

n
t=2
y
2
t1
lcatc your answcr to
a mcthod tor ttin a rst-ordcr Al modc to thc data y
t

2. Considcr thc autorcrcssivc modc Al(1), ic x

t

1
x
t1
w
t

(a) Iind thc ncccssary condition tor {x

t
} to c invcrtic
() Show that x
t
can c cxjrcsscd as a incar jroccss
(c) Show that E|w
t
x
t
|
2
w
and E|w
t
x
t1
| 0, so that tuturc
crrors arc uncorrcatcd with jast data
3. Thc auto-covariancc and autocorrcation tunctions tor Al jro-
ccsscs arc ottcn dcrivcd trom thc Yuc-\akcr cquations, o-
taincd y mutijyin oth sidcs ot thc dcnin cquation, succcs-
sivcy y x
t
, x
t1
, sc thc Yuc-\akcr cquations to drivc

x
(h) tor thc rst-ordcr Al
4. Ior an Al`A scrics wc dcnc thc ojtima torccast ascd on x
t
,
x
t1
, as thc conditiona cxjcctation x
t
t+h
E|x
t+h
| x
t
, x
t1
, |
tor h 1
(a) Show, tor thc cncra Al`A modc that E|w
t+h
| x
t
, x
t1
, |
0 it h > 0, w
t+h
it h 0
() Ior thc Al(1) and Al(2) modcs, dcrivc thc ojtima torccast
x
t
t+h
and thc jrcdiction crror variancc ot thc onc-stcj torccast
5. Sujjosc wc havc thc simjc incar trcnd modc y
t

1
t + x
t
,
1 t n, whcrc x
t

1
x
t1
+w
t
Givc thc cxact torm ot thc
cquations that you woud usc tor cstimatin
1
,
1
and
2
w
usin
thc Cochran-Orcutt jroccdurc
6. Sujjosc that thc simjc rcturn ot a monthy ond indcx toows
thc `A(1) modc
x
t
w
t
+ 0.2 w
t1
,
w
0.02`.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 152
Assumc that w
100
0.01 Comjutc thc 1-stcj (x
t
t+1
) and 2-
stcj (x
t
t+2
) ahcad torccasts ot thc rcturn at thc torccast oriin
t 100 \hat arc thc standard dcviations ot thc associatcd
torccast crrors Aso comjutc thc a-1 ((1)) and a-2 ((2))
autocorrcations ot thc rcturn scrics
7. Sujjosc that thc daiy o rcturn ot a sccurity toows thc modc
x
t
0.01 + 0.2 x
t2
+ w
t
,
whcrc {w
t
} is a Gaussian whitc noisc scrics with mcan zcro and
variancc 0.02 \hat arc thc mcan and variancc ot thc rcturn
scrics x
t
Comjutc thc a-1 ((1)) and a-2 ((2)) autocor-
rcations ot x
t
Assumc that x
100
0.01, and x
99
0.02
Comjutc thc 1-stcj (x
t
t+1
) and 2-stcj (x
t
t+2
) ahcad torccasts ot
thc rcturn scrics at thc torccast oriin t 100 \hat arc thc
associatcd standard dcviations ot thc torccast crrors
8. Considcr thc c la-regr.dat, in thc syaus, which contains
cardiovascuar mortaity, tcmjcraturc vaucs and jarticuatc cv-
cs ovcr o-day jcriods trom Los Anccs County (190-199)
Thc c aso contains two dummy variacs tor rcrcssion jur-
joscs, a coumn ot oncs tor thc constant tcrm and a timc indcx
Thc ordcr is as toows Coumn 1 `0S cardiovascuar mortaity
vaucs (o-day avcracs), Coumn 2 `0S oncs, Coumn 3 thc
intccrs 1, 2, , `0S, Coumn 3 Tcmjcraturc in dcrccs I and
Coumn ! larticuatc cvcs A rctcrcncc is Shumway et. al.
(19SS) Thc joint hcrc is to cxaminc jossic rcations ctwccn
thc tcmjcraturc and mortaity in thc jrcscncc ot a timc trcnd
in cardiovascuar mortaity
(a) sc scattcr diarams to aruc that jarticuatc cvc may c
incary rcatcd to mortaity and that tcmjcraturc has cithcr a
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 153
incar or quadratic rcation Chcck tor acd rcations usin thc
cross corrcation tunction
() Ad,ust tcmjcraturc tor its mcan vauc and t thc modc
M
t

0
+
1
(T
t
T) +
2
(T
t
T)
2
+
3
P
t
+ e
t
,
whcrc M
t
, T
t
and P
t
dcnotc thc mortaity, tcmjcraturc and jar-
ticuatc joution scrics You can usc as injuts Coumns 2 and 3
tor thc trcnd tcrms and run thc rcrcssion anaysis without thc
constant ojtion
(c) lot thc rcsiduas and comjutc thc autocorrcation (ACI)
and jartia autocorrcation (lACI) tunctions Lo thc rcsiduas
ajjcar to c whitc Sucst an All`A modc tor thc rcsiduas
and t thc rcsiduas Thc simjc All`A(2, 0, 0) modc is a ood
comjromisc
(d) Ajjy thc All`A modc otaincd in jart (c) to a ot thc
injut variacs and to cardiovascuar mortaity usin a transtor-
mation lctain thc torccast vaucs tor thc transtormcd mortaity,
say m
t
M
t

1
M
t1

2
M
t2

9. Gcncratc 10 rcaizations ot a (n 200 joints cach) scrics trom an

All`A(1, 0, 1) `odc with
1
0.90,
1
0.20 and
2
w
0.2`
Iit thc All`A modc to cach ot thc scrics and comjarc thc
cstimators to thc truc vaucs y comjutin thc avcrac ot thc
cstimators and thcir standard dcviations
10. Considcr thc ivariatc timc scrics rccord containin monthy
S lroduction as mcasurcd monthy y thc Icdcra lcscrvc
Loard lroduction lndcx and uncmjoymcnt as ivcn in thc c
frb.asd Thc c contains n 32 monthy vaucs tor cach sc-
rics Lctorc you cin, c surc to jot thc scrics Iit a scasona
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 154
All`A modc ot your choicc to thc Icdcra lcscrvc lroduction
lndcx Lcvcoj a 12 month torccast usin thc modc
11. Thc c accd clim-hyd.asd has !`! months ot mcasurcd
vaucs tor thc cimatic variacs Air Tcmjcraturc, Lcw loint,
Coud Covcr, \ind Sjccd, lrccijtation, and lnow at Shasta
Lakc \c woud ikc to ook at jossic rcations ctwccn thc
wcathcr tactors and ctwccn thc wcathcr tactors and thc inow
to Shasta Lakc
(a) Iit thc All`A(0, 0, 0) (0, 1, 1)
12
modc to transtormcd
jrccijitation P
t


p
t
and transtormcd ow i
t
o(i
t
) Savc
thc rcsiduas tor transtormcd jrccijitation tor usc in jart ()
() Ajjy thc All`A modc ttcd in jart (a) tor transtormcd
jrccijitation to thc ow scrics Comjutc thc cross corrcation
ctwccn thc ow rcsiduas usin thc jrccijitation All`A modc
and thc jrccijitation rcsiduas usin thc jrccijitation modc and
intcrjrct
12. Considcr thc daiy simjc rcturn ot ClSl cqua-wcihtcd indcx,
incudin distriutions, trom 1anuary 19S0 to Lcccmcr 1999 in
thc c d-ew8099.txt (day, cw) Thc indicator variacs tor
`ondays, Tucsdays, \cdncsdays, and Thursdays arc in thc rst
tour coumns ot thc c wkdays8099.dat sc a rcrcssion
modc, jossiy with timc scrics crrors, to study thc cccts ot
tradin days on thc indcx rcturn \hat is thc ttcd modc Arc
thc wcckday cccts sinicant in thc rcturns at thc `/ cvc sc
thc LAC cstimator ot thc covariancc matrix to otain thc t-ratios
ot rcrcssion cstimatcs Locs it chanc thc concusion ot wcckday
ccct Arc thcrc scria corrcations in thc rcsiduas sc thc
L,un-Lox tcst to jcrtorm thc tcst Lraw your concusion lt
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 155
ycs, uid a rcrcssion modc with timc scrics crror to study
wcckday cccts
13. This jrocm is conccrncd with thc dynamic rcationshij c-
twccn thc sjot and tuturcs jriccs ot thc S:l`00 indcx Thc
data c sp5may.dat has thrcc coumns o(tuturcs jricc),
o(sjot jricc), and cost-ot-carry (100) Thc data wcrc o-
taincd trom thc Chicao `crcantic Lxchanc tor thc S:l `00
stock indcx in `ay 1993 and its 1unc tuturcs contract Thc timc
intcrva is 1 minutc (intraday) Scvcra authors uscd thc data
to study indcx tuturcs aritrac Lcrc wc tocus on thc rst two
coumns Lct f
t
and s
t
c thc o jriccs ot tuturcs and sjot,
rcsjcctivcy Luid a rcrcssion modc with timc scrics crrors c-
twccn {f
t
} and {s
t
}, with f
t
cin thc dcjcndcnt variac You
nccd to jrovidc a dctais (rcasons and anaysis rcsuts) at cach
stcj
14. Thc quartcry ross domcstic jroduct imjicit jricc dcator is
ottcn uscd as a mcasurc ot ination Thc c q-gdpdef.dat
contains thc data tor S trom thc rst quartcr ot 19! to thc
rst quartcr ot 200! Thc data arc scasonay ad,ustcd and cqua
to 100 tor ycar 2000 Thc data arc otaincd trom thc Icdcra
lcscrvc Lank ot St Louis Luid a (scasona) All`A modc
tor thc scrics and chcck thc vaidity ot thc ttcd modc sc
thc modc to torccast thc ross domcstic jroduct imjicit jricc
dcator tor thc rcst ot 200!
15. Considcr thc monthy simjc rcturns ot thc Lccic 1, Lccic `,
and Lccic 10 ot `YSL,A`LX,`ASLAQ ascd on markct caj-
itaization Thc data sjan is trom 1anuary 19o0 to Lcccmcr
2003, and thc data arc otaincd trom ClSl with thc c namc
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 156
m-decile1510.txt Ior cach scrics, tcst thc nu hyjothcsis
that thc rst 12 as ot autocorrcations arc zcro at thc `/ cvc
Lraw your concusion Luid an Al and `A modc tor thc scrics
Lccic ` sc thc Al and `A modc uit to jroducc 1-stcj and
3-stcj ahcad torccasts ot thc scrics Comjarc thc ttcd Al and
`A modcs
16. Thc data c q-unemrate.txt contains thc S quartcry un-
cmjoymcnt ratc, scasonay ad,ustcd, trom 19!S to thc sccond
quartcr ot 1991 Considcr thc chanc scrics x
t
x
t
x
t1
,
whcrc x
t
is thc quartcry uncmjoymcnt ratc Luid an Al
modc tor thc x
t
scrics Locs thc ttcd modc sucst thc
cxistcncc ot usincss cyccs
17. ln this cxcrcisc, jcasc construct thc imjusc rcsjonsc tunction
tor a a third ordcr dicrcncc cquation y
t

1
y
t1
+
2
y
t2
+

3
y
t3
+w
t
tor 1 t n, whcrc it is assumcd that {w
t
}
n
t=1
is a
scqucncc ot dctcrministic numcrs, say cncratcd trom N(0, 1)
(a) Sct
1
1.1,
2
0.S,
3
0.1, y
0
y
1
y
2
0, and
n 1`0 Gcncratc y
t
usin a third ordcr dicrcncc cquation tor
1 t n
() Chcck cicnvaucs ot this modc to dctcrminc whcthcr llI
arc convcrin or cxjosivc
(c) Construct thc imjusc rcsjonsc tunction tor thc cncratcd
y
t
Sct thc numcr ot jcriods in thc imjusc rcsjonsc tunction
to J 2` Commcnt your rcsuts
(d) Sct
1
1.1 and rcjcat stcjs (a) - (c) Commcnt your
rcsuts
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 157
3.15 Computer Code
Thc toowin R commands arc uscd tor makin thc rajhs in this
chajtcr
# 3-28-2006
graphics.off()
###################################################################
ibm<-matrix(scan("c:\\teaching\\time series\\data\\m-ibm2697.txt"),
byrow=T,ncol=1)
vw<-matrix(scan("c:\\teaching\\time series\\data\\m-vw2697.txt"),
byrow=T,ncol=1)
n=length(ibm)
ibm1=ibm
ibm2=log(ibm1+1)
vw1=vw
vw2=log(1+vw1)
postscript(file="c:\\teaching\\time series\\figs\\fig-2.1.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4)
acf(ibm1, ylab="", xlab="",ylim=c(-0.2,0.2),lag=100,
main="Simple Returns",cex=0.5)
text(50,0.2,"IBM")
acf(ibm2, ylab="", xlab="",ylim=c(-0.2,0.2),lag=100,
main="Log Returns",cex=0.5)
text(50,0.2,"IBM")
acf(vw1, ylab="", xlab="",ylim=c(-0.2,0.2),lag=100,
main="Simple Returns",cex=0.5)
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 158
text(50,0.2,"value-weighted index")
acf(vw2, ylab="", xlab="",ylim=c(-0.2,0.2),lag=100,
main="Log Returns",cex=0.5)
text(50,0.2,"value-weighted index")
dev.off()
###################################################################
###################################################################
y1<-matrix(scan("c:\\teaching\\time series\\data\\ngtemp.dat"),byrow=T,ncol=1)
y=y1[,1]
n=length(y)
a<-1:12
a=a/12
y=y1[,1]
n=length(y)
x<-rep(0,n)
for(i in 1:149){
x[((i-1)*12+1):(12*i)]<-1856+i-1+a
}
x[n-1]<-2005+1/12
x[n]=2005+2/13
x=x/100
x1=cbind(rep(1,n),x)
z=t(x1)%*%x1
fit1=lm(y~x) # fit a regression model
resid1=fit1$resid # obatin residuls
sigma2=mean(resid1^2)
y.diff=diff(y) # compute difference
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 159
var_beta=sigma2*solve(z)
postscript(file="c:\\teaching\\time series\\figs\\fig-2.2.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4)
acf(resid1,lag.max=20,ylab="",ylim=c(-0.5,1),
main="Detrended Temperature",cex=0.5)
text(5,0.8,"ACF")
pacf(resid1,lag.max=20,ylab="",ylim=c(-0.5,1),main="")
text(5,0.8,"PACF")
acf(y.diff,lag.max=20,ylab="",ylim=c(-0.5,1),
main="Differenced Temperature",cex=0.5)
text(5,0.8,"ACF")
pacf(y.diff,lag.max=20,ylab="",ylim=c(-0.5,1),main="")
text(5,0.8,"PACF")
dev.off()
###################################################################
# simulate an I(1) series
n=200
#y=arima.sim(list(order=c(0,1,0)),n=200) # simulate the integradted
x2=rnorm(n) # a white noise series
y=diffinv(x) # simulate I(1) with
postscript(file="c:\\teaching\\time series\\figs\\fig-2.3.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(1,2),mex=0.4)
ts.plot(y,type="l",lty=1,ylab="",xlab="")
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 160
text(100,0.8*max(y),"Random Walk")
ts.plot(x2,type="l",ylab="",xlab="")
text(100,0.8*max(x2),"First Difference")
abline(0,0)
dev.off()
postscript(file="c:\\teaching\\time series\\figs\\fig-2.4.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4)
acf(y,ylab="",xlab="",main="Random Walk",cex=0.5,ylim=c(-0.5,1.0))
text(15,0.8,"ACF")
pacf(y,ylab="",xlab="lag",main="",ylim=c(-0.5,1.0))
text(15,0.8,"PACF")
acf(x2,ylab="",xlab="",main="First Difference",cex=0.5,ylim=c(-0.5,1.0))
text(15,0.8,"ACF")
pacf(x2,ylab="",xlab="lag",main="",ylim=c(-0.5,1.0))
text(15,0.8,"PACF")
dev.off()
################################################################
# This is Example 2.5 in Chapter 2
###################################
x<-read.table("c:\\teaching\\time series\\data\\soi.dat",header=T)
x.soi=x[,1]
n=length(x.soi)
aicc=0
if(aicc==1){
aic.value=rep(0,30) # max.lag=10
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 161
aicc.value=aic.value
sigma.value=rep(0,30)
for(i in 1:30){
fit3=arima(x.soi,order=c(i,0,0)) # fit an AR(i)
aic.value[i]=fit3$aic/n-2 # compute AIC
sigma.value[i]=fit3$sigma2
# obtain the estimated sigma^2
aicc.value[i]=log(sigma.value[i])+(n+i)/(n-i-2) # compute AICC
print(c(i,aic.value[i],aicc.value[i]))}
data=cbind(aic.value,aicc.value)
write(t(data),"c:\\teaching\\time series\\soi_aic.dat",ncol=2)
}else{
data<-matrix(scan("c:\\teaching\\time series\\soi_aic.dat"),byrow=T,ncol=2)
}
text4=c("AIC", "AICC")
postscript(file="c:\\teaching\\time series\\figs\\fig-2.5.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(1,2),mex=0.4)
acf(resid1,ylab="",xlab="",lag.max=20,ylim=c(-0.5,1),main="")
text(10,0.8,"ACF of residuls of AR(1) for SOI")
matplot(1:30,data,type="b",pch="o",col=c(1,2),ylab="",xlab="Lag"
legend(16,-1.40,text4,lty=1,col=c(1,2))
dev.off()
#fit2=arima(x.soi,order=c(16,0,0))
#print(fit2)
###################################################################
# This is Example 2.7 in Chapter 2
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 162
####################################
varve<-read.table("c:\\teaching\\time series\\data\\mass2.dat",header=T)
varve=varve[,1]
n_varve=length(varve)
varve_log=log(varve)
varve_log_diff=diff(varve_log)
postscript(file="c:\\teaching\\time series\\figs\\fig-2.6.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4)
acf(varve_log,ylab="",xlab="",lag.max=30,ylim=c(-0.5,1),main="AC
text(10,0.7,"log varves",cex=0.7)
pacf(varve_log,ylab="",xlab="",lag.max=30,ylim=c(-0.5,1),main="P
acf(varve_log_diff,ylab="",xlab="",lag.max=30,ylim=c(-0.5,1),mai
text(10,0.7,"First difference",cex=0.7)
pacf(varve_log_diff,ylab="",xlab="",lag.max=30,ylim=c(-0.5,1),ma
dev.off()
###################################################################
# This is Example 2.9 in Chapter 2
####################################
x<-matrix(scan("c:\\teaching\\time series\\data\\birth.dat"),byrow=T,ncol=1)
n=length(x)
x_diff=diff(x)
x_diff_12=diff(x_diff,lag=12)
fit1=arima(x,order=c(0,0,0),seasonal=list(order=c(0,0,0)),in
resid_1=fit1$resid
fit2=arima(x,order=c(0,1,0),seasonal=list(order=c(0,0,0)),in
resid_2=fit2$resid
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 163
fit3=arima(x,order=c(0,1,0),seasonal=list(order=c(0,1,0),peri
include.mean=F)
resid_3=fit3$resid
postscript(file="c:\\teaching\\time series\\figs\\fig-2.8.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(5,2),mex=0.4)
acf(resid_1, ylab="", xlab="",ylim=c(-0.5,1),lag=60,main="ACF",cex=0.7)
pacf(resid_1,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="PACF",c
text(20,0.7,"data",cex=1.2)
acf(resid_2, ylab="", xlab="",ylim=c(-0.5,1),lag=60,main="")
# differenced data
pacf(resid_2,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="")
text(30,0.7,"ARIMA(0,1,0)")
acf(resid_3, ylab="", xlab="",ylim=c(-0.5,1),lag=60,main="")
# seasonal difference of differenced data
pacf(resid_3,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="")
text(30,0.7,"ARIMA(0,1,0)X(0,1,0)_{12}",cex=0.8)
fit4=arima(x,order=c(0,1,0),seasonal=list(order=c(0,1,1),
period=12),include.mean=F)
resid_4=fit4$resid
fit5=arima(x,order=c(0,1,1),seasonal=list(order=c(0,1,1),
period=12),include.mean=F)
resid_5=fit5$resid
acf(resid_4, ylab="", xlab="",ylim=c(-0.5,1),lag=60,main="")
# ARIMA(0,1,0)*(0,1,1)_12
pacf(resid_4,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="")
text(30,0.7,"ARIMA(0,1,0)X(0,1,1)_{12}",cex=0.8)
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 164
acf(resid_5, ylab="", xlab="",ylim=c(-0.5,1),lag=60,main="")
# ARIMA(0,1,1)*(0,1,1)_12
pacf(resid_5,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="")
text(30,0.7,"ARIMA(0,1,1)X(0,1,1)_{12}",cex=0.8)
dev.off()
postscript(file="c:\\teaching\\time series\\figs\\fig-2.7.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4)
ts.plot(x,type="l",lty=1,ylab="",xlab="")
text(250,375, "Births")
ts.plot(x_diff,type="l",lty=1,ylab="",xlab="",ylim=c(-50,50))
text(255,45, "First difference")
abline(0,0)
ts.plot(x_diff_12,type="l",lty=1,ylab="",xlab="",ylim=c(-50,50))
# time series plot of the seasonal difference (s=12) of differenced
text(225,40,"ARIMA(0,1,0)X(0,1,0)_{12}")
abline(0,0)
ts.plot(resid_5,type="l",lty=1,ylab="",xlab="",ylim=c(-50,50))
text(225,40, "ARIMA(0,1,1)X(0,1,1)_{12}")
abline(0,0)
dev.off()
###############################################################
###################################################################
# This is Example 2.10 in Chapter 2
#####################################
y<-matrix(scan("c:\\teaching\\time series\\data\\jj.dat"),byrow=T,ncol=1)
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 165
n=length(y)
y_log=log(y) # log of data
y_diff=diff(y_log) # first-order difference
y_diff_4=diff(y_diff,lag=4) # first-order seasonal difference
fit1=ar(y_log,order=1) # fit AR(1) model
#print(fit1)
library(tseries) # call library(tseries)
library(zoo)
fit1_test=adf.test(y_log)
# do Augmented Dicky-Fuller test for testing unit root
#print(fit1_test)
fit1=arima(y_log,order=c(0,0,0),seasonal=list(order=c(0,0,0))
include.mean=F)
resid_21=fit1$resid
fit2=arima(y_log,order=c(0,1,0),seasonal=list(order=c(0,0,0))
include.mean=F)
resid_22=fit2$resid # residual for ARIMA(0,1,0)*(0,0,0)
fit3=arima(y_log,order=c(0,1,0),seasonal=list(order=c(1,0,0),
include.mean=F,method=c("CSS"))
resid_23=fit3$resid # residual for ARIMA(0,1,0)*(1,0,0)_4
# note that this model is non-stationary so that "CSS" is used
postscript(file="c:\\teaching\\time series\\figs\\fig-2.9.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(4,2),mex=0.4)
acf(resid_21, ylab="", xlab="",ylim=c(-0.5,1),lag=30,main="ACF",cex=0.7)
text(16,0.8,"log(J&J)")
pacf(resid_21,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="PACF",
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 166
acf(resid_22, ylab="", xlab="",ylim=c(-0.5,1),lag=30,main="")
text(16,0.8,"First Difference")
pacf(resid_22,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")
acf(resid_23, ylab="", xlab="",ylim=c(-0.5,1),lag=30,main="")
text(16,0.8,"ARIMA(0,1,0)X(1,0,0,)_4",cex=0.8)
pacf(resid_23,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")
fit4=arima(y_log,order=c(0,1,1),seasonal=list(order=c(1,0,0),
period=4),include.mean=F,method=c("CSS"))
resid_24=fit4$resid # residual for ARIMA(0,1,1)*(1,0,0)_4
# note that this model is non-stationary
#print(fit4)
fit4_test=Box.test(resid_4,lag=12, type=c("Ljung-Box"))
#print(fit4_test)
acf(resid_24, ylab="", xlab="",ylim=c(-0.5,1),lag=30,main="")
text(16,0.8,"ARIMA(0,1,1)X(1,0,0,)_4",cex=0.8)
# ARIMA(0,1,1)*(1,0,0)_4
pacf(resid_24,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")
dev.off()
fit5=arima(y_log,order=c(0,1,1),seasonal=list(order=c(0,1,1),
include.mean=F,method=c("ML"))
resid_25=fit5$resid # residual for ARIMA(0,1,1)*(0,1,1)_4
#print(fit5)
fit5_test=Box.test(resid_25,lag=12, type=c("Ljung-Box"))
#print(fit5_test)
postscript(file="c:\\teaching\\time series\\figs\\fig-2.10.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4)
acf(resid_25, ylab="", xlab="",ylim=c(-0.5,1),lag=30,main="ACF")
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 167
text(16,0.8,"ARIMA(0,1,1)X(0,1,1,)_4",cex=0.8)
# ARIMA(0,1,1)*(0,1,1)_4
pacf(resid_25,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="PACF")
ts.plot(resid_24,type="l",lty=1,ylab="",xlab="")
title(main="Residual Plot",cex=0.5)
text(40,0.2,"ARIMA(0,1,1)X(1,0,0,)_4",cex=0.8)
abline(0,0)
ts.plot(resid_25,type="l",lty=1,ylab="",xlab="")
title(main="Residual Plot",cex=0.5)
text(40,0.18,"ARIMA(0,1,1)X(0,1,1,)_4",cex=0.8)
abline(0,0)
dev.off()
###################################################################
###################################################################
# This is Example 2.11 in Chapter 2
z<-matrix(scan("c:\\teaching\\time series\\data\\m-decile1510.txt"),byrow=T,ncol=4)
decile1=z[,2]
# Model 1: an ARIMA(1,0,0)*(1,0,1)_12
fit1=arima(decile1,order=c(1,0,0),seasonal=list(order=c(1,0,1)
period=12),include.mean=T)
#print(fit1)
e1=fit1$resid
n=length(decile1)
m=n/12
jan=rep(c(1,0,0,0,0,0,0,0,0,0,0,0),m)
feb=rep(c(0,1,0,0,0,0,0,0,0,0,0,0),m)
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 168
mar=rep(c(0,0,1,0,0,0,0,0,0,0,0,0),m)
apr=rep(c(0,0,0,1,0,0,0,0,0,0,0,0),m)
may=rep(c(0,0,0,0,1,0,0,0,0,0,0,0),m)
jun=rep(c(0,0,0,0,0,1,0,0,0,0,0,0),m)
jul=rep(c(0,0,0,0,0,0,1,0,0,0,0,0),m)
aug=rep(c(0,0,0,0,0,0,0,1,0,0,0,0),m)
sep=rep(c(0,0,0,0,0,0,0,0,1,0,0,0),m)
oct=rep(c(0,0,0,0,0,0,0,0,0,1,0,0),m)
nov=rep(c(0,0,0,0,0,0,0,0,0,0,1,0),m)
dec=rep(c(0,0,0,0,0,0,0,0,0,0,0,1),m)
de=cbind(decile1[jan==1],decile1[feb==1],decile1[mar==1],decile1[a
decile1[may==1],decile1[jun==1],decile1[jul==1],decile1[aug==1],
decile1[sep==1],decile1[oct==1],decile1[nov==1],decile1[dec==1])
# Model 2: a simple regression model without correlated errors
# to see the effect from January
fit2=lm(decile1~jan)
e2=fit2$resid
#print(summary(fit2))
# Model 3: a regression model with correlated errors
fit3=arima(decile1,xreg=jan,order=c(0,0,1),include.mean=T)
e3=fit3$resid
#print(fit3)
postscript(file="c:\\teaching\\time series\\figs\\fig-2.11.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4)
ts.plot(decile1,type="l",lty=1,col=1,ylab="",xlab="")
title(main="Simple Returns",cex=0.5)
abline(0,0)
ts.plot(e3,type="l",lty=1,col=1,ylab="",xlab="")
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 169
title(main="January-adjusted returns",cex=0.5)
abline(0,0)
acf(decile1, ylab="", xlab="",ylim=c(-0.5,1),lag=40,main="ACF")
acf(e3,ylab="",xlab="",ylim=c(-0.5,1),lag=40,main="ACF")
dev.off()
###################################################################
# This is Example 2.12 in Chapter 2
#####################################
z<-matrix(scan("c:\\teaching\\time series\\data\\jj.dat"),byrow=T,ncol=1)
n=length(z)
z_log=log(z) # log of
# MODEL 1: y_t=beta_0+beta_1 t+ e_t
z1=1:n
fit1=lm(z_log~z1) # fit log(z) versus time
e1=fit1$resid
# Now, we need to re-fit the model using the transformed data
x1=5:n
y_1=z_log[5:n]
y_2=z_log[1:(n-4)]
y_fit=y_1-0.7614*y_2
x2=x1-0.7614*(x1-4)
x1=(1-0.7614)*rep(1,n-4)
fit2=lm(y_fit~-1+x1+x2)
e2=fit2$resid
postscript(file="c:\\teaching\\time series\\figs\\fig-2.12.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4)
acf(e1, ylab="", xlab="",ylim=c(-0.5,1),lag=30,main="ACF")
text(10,0.8,"detrended")
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 170
pacf(e1,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="PACF")
acf(e2, ylab="", xlab="",ylim=c(-0.5,1),lag=30,main="")
text(15,0.8,"ARIMA(1,0,0,)_4")
pacf(e2,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")
dev.off()
#################################################################
##############################################################
# This is Example 2.13 in Chapter 2
#####################################
z<-matrix(scan("c:\\teaching\\time series\\data\\w-gs1n36299.txt"),
byrow=T,ncol=3)
# first column=one year Treasury constant maturity rate;
# second column=three year Treasury constant maturity rate;
# third column=date
x=z[,1]
y=z[,2]
n=length(x)
u=seq(1962+1/52,by=1/52,length=n)
x_diff=diff(x)
y_diff=diff(y)
# Fit a simple regression model and examine the residuals
fit1=lm(y~x) # Model 1
e1=fit1$resid
postscript(file="c:\\teaching\\time series\\figs\\fig-2.13.eps",
horizontal=F,width=6,height=6)
matplot(u,cbind(x,y),type="l",lty=c(1,2),col=c(1,2),ylab="",
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 171
dev.off()
postscript(file="c:\\teaching\\time series\\figs\\fig-2.14.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(1,2),mex=0.4)
plot(x,y,type="p",pch="o",ylab="",xlab="",cex=0.5)
plot(x_diff,y_diff,type="p",pch="o",ylab="",xlab="",cex=0.5)
dev.off()
postscript(file="c:\\teaching\\time series\\figs\\fig-2.15.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(1,2),mex=0.4)
plot(u,e1,type="l",lty=1,ylab="",xlab="")
abline(0,0)
acf(e1,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")
dev.off()
# Take different and fit a simple regression again
fit2=lm(y_diff~x_diff) # Model 2
e2=fit2$resid
postscript(file="c:\\teaching\\time series\\figs\\fig-2.16.eps",
horizontal=F,width=6,height=6)
matplot(u[-1],cbind(x_diff,y_diff),type="l",lty=c(1,2),col=c(
ylab="",xlab="")
abline(0,0)
dev.off()
postscript(file="c:\\teaching\\time series\\figs\\fig-2.17.eps",
horizontal=F,width=6,height=6)
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 172
par(mfrow=c(2,2),mex=0.4)
ts.plot(e2,type="l",lty=1,ylab="",xlab="")
abline(0,0)
acf(e2, ylab="", xlab="",ylim=c(-0.5,1),lag=30,main="")
# fit a model to the differenced data with an MA(1) error
fit3=arima(y_diff,xreg=x_diff, order=c(0,0,1)) # Model
e3=fit3$resid
ts.plot(e3,type="l",lty=1,ylab="",xlab="")
abline(0,0)
acf(e3, ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")
dev.off()
#################################################################
###############################################################
# This is Example 14 of using HC and HAC
##########################################
library(sandwich) # HC and HAC are in the package "sandwich"
library(zoo)
z<-matrix(scan("c:\\teaching\\time series\\data\\w-gs1n36299.txt"),
byrow=T,ncol=3)
x=z[,1]
y=z[,2]
x_diff=diff(x)
y_diff=diff(y)
# Fit a simple regression model and examine the residuals
fit1=lm(y_diff~x_diff)
print(summary(fit1))
e1=fit1$resid
# Heteroskedasticity-Consistent Covariance Matrix Estimation
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 173
#hc0=vcovHC(fit1,type="const")
#print(sqrt(diag(hc0)))
# type=c("const","HC","HC0","HC1","HC2","HC3","HC4")
# HC0 is the White estimator
hc1=vcovHC(fit1,type="HC0")
print(sqrt(diag(hc1)))
#Heteroskedasticity and autocorrelation consistent (HAC) estimation
#of the covariance matrix of the coefficient estimates in a
#(generalized) linear regression model.
hac1=vcovHAC(fit1,sandwich=T)
print(sqrt(diag(hac1)))
###################################################################
# This is the Example 2.15 in Chapter 2
#######################################
z1<-matrix(scan("c:\\teaching\\time series\\data\\d-ibmvwewsp6203.txt"),
byrow=T,ncol=5)
vw=abs(z1[,3])
n_vw=length(vw)
ew=abs(z1[,4])
postscript(file="c:\\teaching\\time series\\figs\\fig-2.18.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4,bg="light green")
acf(vw, ylab="",xlab="",ylim=c(-0.1,0.4),lag=400,main="")
text(200,0.38,"ACF for value-weighted index")
acf(ew, ylab="",xlab="",ylim=c(-0.1,0.4),lag=400,main="")
text(200,0.38,"ACF for equal-weighted index")
library(fracdiff)
d1=fracdiff(vw,ar=0,ma=0)
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 174
d2=fracdiff(ew,ar=0,ma=0)
print(c(d1$d,d2$d))
m1=round(log(n_vw)/log(2)+0.5)
pad1=1-n_vw/2^m1
vw_spec=spec.pgram(vw,spans=c(3,3,3),demean=T,detrend=T,pad=pad1
ew_spec=spec.pgram(ew,spans=c(3,3,3),demean=T,detrend=T,pad=pad1
vw_x=vw_spec$freq[1:1000]
vw_y=vw_spec$spec[1:1000]
ew_x=ew_spec$freq[1:1000]
ew_y=ew_spec$spec[1:1000]
scatter.smooth(vw_x,log(vw_y),span=1/15,ylab="",xlab="",col=6,ce
text(0.04,-7,"Log Spectral Density of VW",cex=0.8)
scatter.smooth(ew_x,log(ew_y),span=1/15,ylab="",xlab="",col=7,ce
text(0.04,-7,"Log Spectral Density of EW",cex=0.8)
dev.off()
###################################################################
# This is the Example 2.16 in Chapter 2
#######################################
phi=c(1.2,-0.35,1.0,-0.70,0.2, 0.35,-0.2,0.35)
dim(phi)=c(2,4)
phi=t(phi)
postscript(file="c:\\teaching\\time series\\figs\\fig-2.19.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4,bg="dark grey")
for(j in 1:4){
rho=rep(0,20)
rho[1]=1
rho[2]=phi[j,1]/(1-phi[j,2])
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 175
for(i in 3:20){rho[i]=phi[j,1]*rho[i-1]+phi[j,2]*rho[i-2]}
plot(1:20,rho,type="h",ylab="",ylim=c(-1,1),xlab="")
if(j==1){title(main="(a)",cex=0.8)}
if(j==2){title(main="(b)",cex=0.8)}
if(j==3){title(main="(c)",cex=0.8)}
if(j==4){title(main="(d)",cex=0.8)}
abline(0,0)
}
dev.off()
z1<-matrix(scan("c:\\teaching\\time series\\data\\q-gnp4791.txt"),
byrow=T,ncol=1)
n=length(z1)
x=1:n
x=x/4+1946.25
postscript(file="c:\\teaching\\time series\\figs\\fig-2.20.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(1,2),mex=0.4,bg="light pink")
plot(x,z1,type="o",ylab="",xlab="")
abline(0,0)
acf(z1,main="",ylab="",xlab="",lag=30)
dev.off()
################################################################
# This is for making graphs for IRF
################################################
n=100
w_t1=rnorm(n,0,1)
w_t2=w_t1
w_t2[50]=w_t1[50]+1
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 176
y1=rep(0,2*(n+1))
dim(y1)=c(n+1,2)
y1[1,]=c(5,5)
y2=y1
phi1=c(0.8,-0.8)
for(i in 2:(n+1)){
y1[i,1]=phi1[1]*y1[(i-1),1]+w_t1[i-1]
y1[i,2]=phi1[1]*y1[(i-1),2]+w_t2[i-1]
y2[i,1]=phi1[2]*y2[(i-1),1]+w_t1[i-1]
y2[i,2]=phi1[2]*y2[(i-1),2]+w_t2[i-1]
}
y1=y1[2:101,]
y2=y2[2:101,]
irf1=y1[,2]-y1[,1]
irf2=y2[,2]-y2[,1]
text1=c("No Impulse","With Impulse")
postscript(file="c:\\teaching\\time series\\figs\\fig-2.21.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4,bg="dark grey")
ts.plot(y1,type="l",lty=1,col=c(1,2),ylab="",xlab="")
abline(0,0)
legend(40,0.9*max(y1[,1]),text1,lty=c(1,2),col=c(1,2),cex=0.
text(40,0.8*min(y1[,1]),"phi=0.8")
ts.plot(y2,type="l",lty=1,col=c(1,2),ylab="",xlab="")
abline(0,0)
legend(40,0.9*max(y2[,1]),text1,lty=c(1,2),col=c(1,2),cex=0.
text(60,0.8*min(y2[,1]),"phi= - 0.8")
plot(1:n,irf1,type="l",ylab="",ylim=c(-1,1),xlab="")
abline(0,0)
text(40,-0.6,"Impulse Response Function",cex=0.8)
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 177
text(20,0.8,"phi=0.8")
plot(1:n,irf2,type="l",ylab="",ylim=c(-1,1),xlab="")
abline(0,0)
text(40,-0.6,"Impulse Response Function",cex=0.8)
text(20,0.8,"phi= - 0.8")
dev.off()
n=100
w_t1=rnorm(n,0,1)
w_t2=w_t1
w_t2[50]=w_t1[50]+1
y1=rep(0,2*(n+1))
dim(y1)=c(n+1,2)
y1[1,]=c(3,3)
y2=y1
phi1=c(1.01,-1.01)
for(i in 2:(n+1)){
y1[i,1]=phi1[1]*y1[(i-1),1]+w_t1[i-1]
y1[i,2]=phi1[1]*y1[(i-1),2]+w_t2[i-1]
y2[i,1]=phi1[2]*y2[(i-1),1]+w_t1[i-1]
y2[i,2]=phi1[2]*y2[(i-1),2]+w_t2[i-1]
}
y1=y1[2:101,]
y2=y2[2:101,]
irf1=y1[,2]-y1[,1]
irf2=y2[,2]-y2[,1]
text1=c("No Impulse","With Impulse")
postscript(file="c:\\teaching\\time series\\figs\\fig-2.22.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4,bg="light pink")
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 178
ts.plot(y1,type="l",lty=1,col=c(1,2),ylab="",xlab="")
abline(0,0)
legend(40,0.9*max(y1[,1]),text1,lty=c(1,2),col=c(1,2),cex=0.
text(40,0.8*min(y1[,1]),"phi=1.01")
ts.plot(y2,type="l",lty=1,col=c(1,2),ylab="",xlab="")
abline(0,0)
legend(40,0.9*max(y2[,1]),text1,lty=c(1,2),col=c(1,2),cex=0.
text(60,0.8*min(y2[,1]),"phi= - 1.01")
plot(1:n,irf1,type="l",ylab="",ylim=c(-2,2),xlab="")
abline(0,0)
text(40,-0.6,"Impulse Response Function",cex=0.8)
text(20,0.8,"phi=1.01")
plot(1:n,irf2,type="l",ylab="",ylim=c(-2,2),xlab="")
abline(0,0)
text(40,-0.6,"Impulse Response Function",cex=0.8)
text(20,0.8,"phi= - 1.01")
dev.off()
x=1:20
phi1=cbind(0.5^x,0.9^x,0.99^x)
phi2=cbind((-0.5)^x,(-0.9)^x,(-0.99)^x)
postscript(file="c:\\teaching\\time series\\figs\\fig-2.23.eps",
horizontal=F,width=6,height=6)
#win.graph()
par(mfrow=c(2,2),mex=0.4,bg="light green")
matplot(x,phi1,type="o",lty=1,pch="o",ylab="",xlab="")
text(4,0.3,"phi1=0.5",cex=0.8)
text(15,0.3,"phi1=0.9",cex=0.8)
text(15,0.9,"phi1=0.99",cex=0.8)
matplot(x,phi2,type="o",lty=1,pch="o",ylab="",xlab="")
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 179
abline(0,0)
text(4,0.3,"phi1= - 0.5",cex=0.8)
text(15,0.3,"phi1= - 0.9",cex=0.8)
text(15,0.9,"phi1= - 0.99",cex=0.8)
matplot(x,1.2^x,type="o",lty=1,pch="o",ylab="",xlab="")
text(14,22,"phi1=1.2",cex=0.8)
matplot(x,(-1.2)^x,type="o",lty=1,pch="o",ylab="",xlab="")
abline(0,0)
text(13,18,"phi1= - 1.2",cex=0.8)
dev.off()
n=100
w_t1=rnorm(n,0,1)
w_t2=w_t1
w_t3=w_t1
w_t2[50]=w_t1[50]+1
w_t3[50:n]=w_t1[50:n]+1
y=rep(0,3*(n+1))
dim(y)=c(n+1,3)
y[1,]=c(3,3,3)
phi1=0.8
for(i in 2:(n+1)){
y[i,1]=phi1*y[(i-1),1]+w_t1[i-1]
y[i,2]=phi1*y[(i-1),2]+w_t2[i-1]
y[i,3]=phi1*y[(i-1),3]+w_t3[i-1]}
y=y[2:101,1:3]
irf1=y[,2]-y[,1]
irf2=y[,3]-y[,1]
text1=c("No Impulse","With Impulse")
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 180
postscript(file="c:\\teaching\\time series\\figs\\fig-2.24.eps",
horizontal=F,width=6,height=6)
#win.graph()
par(mfrow=c(2,2),mex=0.4,bg="light blue")
ts.plot(y[,1:2],type="l",lty=1,col=c(1,2),ylab="",xlab="")
abline(0,0)
legend(40,0.9*max(y[,1]),text1,lty=c(1,2),col=c(1,2),cex=0.8
text(40,0.8*min(y[,1]),"phi=0.8")
ts.plot(cbind(y[,1],y[,3]),type="l",lty=1,col=c(1,2),ylab="",x
abline(0,0)
legend(40,0.9*max(y[,3]),text1,lty=c(1,2),col=c(1,2),cex=0.8
text(10,0.8*min(y[,3]),"phi=0.8")
plot(1:n,irf1,type="l",ylab="",xlab="")
abline(0,0)
text(40,0.6,"Impulse Response Function",cex=0.8)
text(20,0.8,"phi=0.8")
plot(1:n,irf2,type="l",ylab="",xlab="")
abline(0,0)
text(40,3,"Impulse Response Function",cex=0.8)
text(20,0.8,"phi=0.8")
dev.off()
ff=c(0.6,0.2,1,0,0.8,0.4,1,0,-0.9,-0.5,1,0,-0.5,-1.5,1,0)
dim(ff)=c(4,4)
mj=20
x=0:mj
irf=rep(0,(mj+1)*4)
dim(irf)=c(mj+1,4)
irf[1,]=1
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 181
for(j in 1:4){
aa=c(1,0,0,1)
dim(aa)=c(2,2)
ff1=ff[,j]
dim(ff1)=c(2,2)
ff1=t(ff1)
for(i in 1:mj){
aa=aa%*%ff1
irf[i+1,j]=aa[1,1]}}
postscript(file="c:\\teaching\\time series\\figs\\fig-2.25.eps",
horizontal=F,width=6,height=6)
#win.graph()
par(mfrow=c(2,2),mex=0.4,bg="light yellow")
plot(x,irf[,1],type="o",pch="o",ylab="",ylim=c(0,1),xlab="",
main="(a)",cex=0.8)
text(10,0.8,"Impulse Response Function",cex=0.8)
text(12,0.3,"phi1=0.6 and phi2=0.2",cex=0.8)
plot(x,irf[,2],type="o",pch="o",ylab="",ylim=c(0,12),xlab="",
main="(b)",cex=0.8)
text(10,10,"Impulse Response Function",cex=0.8)
text(9,6,"phi1=0.8 and phi2=0.4",cex=0.8)
plot(x,irf[,3],type="o",pch="o",ylab="",ylim=c(-1,1),xlab="",
main="(c)",cex=0.8)
abline(0,0)
text(10,0.8,"Impulse Response Function",cex=0.8)
text(11,-0.3,"phi1= - 0.9 and phi2= - 0.5",cex=0.8)
plot(x,irf[,4],type="o",pch="o",ylab="",ylim=c(-40,30),xlab="",
main="(d)",cex=0.8)
abline(0,0)
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 182
text(10,25,"Impulse Response Function",cex=0.8)
text(9,-16,"phi1= - 0.5 and phi2= - 1.5",cex=0.8)
dev.off()
###################################################################
3.16 References
Akaike, H. (1973). Information theory and an extension of the maximum likelihood princi-
ple. In Proceeding of 2nd International Symposium on Information Theory (V. Petrov
and F. Csaki, eds.) 267281. Akademiai Kiado, Budapest.
Andrews, D.W.K. (1991). Heteroskedasticity and autocorrelation consistent covariance
matrix estimation. Econometrica, 59, 817-858.
Bai, Z., C.R. Rao and Y. Wu (1999). Model selection with data-oriented penalty. Journal
of Statistical Planning and Inferences, 77, 103-117.
Beran, J. (1994). Statistics for Long-Memory Processes. Chapman and Hall, London.
Box, G.E.P. and Jenkins, G.M. (1970). Time Series Analysis, Forecasting, and Control.
Holden Day, San Francisco.
Box, G.E.P., G.M. Jenkins and G.C. Reinsel (1994). Time Series Analysis, Forecasting and
Control. 3th Edn. Englewood Clis, NJ: Prentice-Hall.
Brockwell, P.J. and Davis, R.A. (1991). Time Series Theory and Methods. New York:
Springer.
Burman, P. and R.H. Shumway (1998). Semiparametric modeling of seasonal time series.
Journal of Time Series Analysis, 19, 127-145.
Burnham, K.P. and D. Anderson (2003). Model Selection And Multi-Model Inference: A
Practical Information Theoretic Approach, 2nd edition. New York: Springer-Verlag.
Cai, Z. and R. Chen (2006). Flexible seasonal time series models. Advances in Economet-
rics, 20B, 63-87.
Chan, N.H. and C.Z. Wei (1988). Limiting distributions of least squares estimates of
unstable autoregressive processes. Annals of Statistics, 16, 367-401.
Cochrane, D. and G.H. Orcutt (1949). Applications of least squares regression to relation-
ships containing autocorrelated errors. Journal of the American Statistical Association,
44, 32-61.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 183
Ding, Z., C.W.J. Granger and R.F. Engle (1993). A long memory property of stock returns
and a new model. Journal of Empirical Finance, 1, 83-106.
Eicker, F. (1967). Limit theorems for regression with unequal and dependent errors. In
Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability
(L. LeCam and J. Neyman, eds.), University of California Press, Berkeley.
Fan and Li (2001). Variable selection via nonconcave penalized likelihood and its oracle
properties. Journal of the American Statistical Association, 96, 1348-1360.
Fan and Peng (2004). Nonconcave penalized likelihood with a diverging number of param-
eters. Annals of Statistics, 32, 928-961.
Frank, I.E. and J.H. Friedman (1993). A statistical view of some chemometric regression
tools (with discussion). Technometrics, 35, 109-148.
Franses, P.H. (1996). Periodicity and Stochastic Trends in Economic Time Series. New
York: Cambridge University Press.
Franses, P.H. (1998). Time Series Models for Business and Economic Forecasting. New
York: Cambridge University Press.
Franses, P.H. and D. van Dijk (2000). Nonlinear Time Series Models for Empirical Finance.
New York: Cambridge University Press.
Ghysels, E. and D.R. Osborn (2001). The Econometric Analysis of Seasonal Time Series.
New York: Cambridge University Press.
Hurvich, C.M. and C.-L.Tsai (1989). Regression and time series model selection in small
samples. Biometrika, 76, 297-307.
Newey, W.K. and K.D. West (1987). A simple, positive-denite, heteroskedasticity and
autocorrelation consistent covariance matrix. Econometrica, 55, 703-708.
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6,
461-464.
Shao (1993). Linear model selection by cross-validation. Journal of the American Statistical
Association, 88, 486-494.
Shen, X.T. and J.M. Ye (2002). Adaptive model selection. Journal of the American Sta-
tistical Association, 97, 210-221.
Shumway, R.H. (1988). Applied Statistical Time Series Analysis. Englewood Clis, NJ:
Prentice-Hall.
Shumway, R.H., A.S. Azari and Y. Pawitan (1988). Modeling mortality uctuations in Los
Angeles as functions of pollution and weather eects. Environmental Research, 45,
224-241.
CHAPTER 3. UNIVARIATE TIME SERIES MODELS 184
Shumway, R.H. and D.S. Stoer (2000). Time Series Analysis & Its Applications. New
York: Springer-Verlag.
Tiao, G.C. and R.S. Tsay (1983). Consistency properties of least squares estimates of
autoregressive parameters in ARMA models. Annals of Statistics, 11, 856-871.
Tibshirani, R.J. (1996). Regression shrinkage and selection via the Lasso. Journal of the
Royal Statistical Society, Series B, 58, 267-288.
Tsay, R.S. (2005). Analysis of Financial Time Series, 2th Edition. John Wiley & Sons,
New York.
Walker, G. (1931).On the periodicity in series of related terms. Proceedings of the Royal
Society of London, Series A, 131, 518-532.
Yule, G.U. (1927).On a method of investigating periodicities in disturbed series with special
reference to Wolfers Sun spot numbers. Philosophical Transactions of the Royal Society
of London, Series A, 226, 267-298.
Chapter 4
Non-stationary Processes and
Structural Breaks
4.1 Introduction
ln our anaysis so tar, wc havc assumcd that thc variacs in thc mod-
cs that wc havc anayzcd, univariatc Al`A modcs, arc stationary
y
t
+ (B) e
t
,
whcrc y
t
may c a vcctor ot n variacs at timc jcriod t A ancc at
rajhs ot most cconomic timc scrics succs to rcvca invaidity ot that
assumjtion, ccausc economies evolve, grow, and change
over time in both real and nominal terms and economic
forecasts often are very wrong, athouh that thcy shoud
occur rcativcy intrcqucnty in a stationary jroccss
Thc jractica jrocm that an cconomctrician taccs is to nd any
rcationshijs that survivc tor rcativcy on jcriod ot timc so that
thcy can c uscd tor torccastin and joicy anaysis Lcndry and
1uscius (2000) jointcd out that four issues immcdiatcy arisc in
thc issuc ot nonstationarity
1. Low imjortant is thc assumjtion ot stationarity tor mod-
cin and intcrcncc It is very important \hcn data
185
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 186
mcans and varianccs arc non-constant, oscrvations comc
trom dicrcnt distriutions ovcr timc, crcatin dicut jro-
cms tor cmjirica modcin
2. \hat is thc ccct ot incorrccty assumin it It is poten-
tially hazardous Assumin constant mcans and vari-
anccs whcn that is tasc can inducc scrious statistica mis-
takcs lt thc variacs in y
t
arc not stationary, thcn con-
vcntiona hyjothcsis tcsts, condcncc intcrvas and torccasts
can c unrciac Standard asymjtotic distriution thcory
ottcn docs not ajjy to rcrcssions invovin variacs with
unit roots, and intcrcncc may c miscadin it this is inorcd
3. \hat arc thc sourccs ot non-stationarity There are many
and they are varied `on-stationarity mayc duc to
cvoution ot thc cconomy, cisativc chancs, tcchnooica
chancs, joitica cvcnts, ctc
4. Can cmjirica anaysis c transtormcd so stationarity c-
comcs a vaid assumjtion It is sometimes possible,
depending on the source of non-stationarity Somc
ot non-stationarity can c ciminatcd y transtormations
such as dc-trcndin and dicrcncin, or somc othcr tyjc
transtormations. scc lark and lhiijs (1999) and Chan,
lark and lhiijs (2001) tor morc discussions
Trending Time Series
A non-stationary jroccss is onc which vioatcs thc stationary rc-
quircmcnt, so its mcans and varianccs arc non-constant ovcr timc
A trcnd is a jcrsistcnt on-tcrm movcmcnt ot a variac ovcr timc
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 187
and a timc scrics uctuatcs around its trcnd Thc common tcaturcs
ot a trending time series can c summarizcd as thc toowin
situations
1. Stochastic trends A stochastic trcnd is random and
varics ovcr timc
(1 L)y
t
+ (L)e
t
.
That is, attcr takin thc dicrcncc, thc timc scrics is mod-
ccd as a incar jroccss
2. Deterministic trends A dctcrministic trcnd is a non-
random tunction ot timc Ior cxamjc, a dctcrministic trcnd
miht c incar or quadratic or hihcr ordcr in timc
y
t
a + t + (L)e
t
.
The dierence between a linear stochastic trend and a
deterministic trend is that the changes of a stochastic
trend are random, whereas those of a deterministic trend
are constant over time
3. Permanence of shocks `acrocconomists uscd to dc-
trcnd data and rcardcd usincss cyccs as thc stationary dc-
viations trom that trcnd Lconomists invcstiatcd whcthcr
G`l is cttcr dcscricd as random wak or truc stationary
jroccss
4. Statistical issues \c coud havc mistakcn a timc scrics
with unit roots tor trcnd stationary timc scrics sincc a timc
scrics with unit roots miht disjay a trcndin jhcnomcna
Thc trcndin timc scrics modcs havc aincd a ot ot attcntion
durin thc ast two dccadcs duc to many ajjications in cconomics
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 188
and nancc Scc Cai (200o) tor morc rctcrcnccs Ioowin arc somc
cxamjcs Thc markct modc in nancc is an cxamjc that rcatcs
thc rcturn ot an individua stock to thc rcturn ot a markct indcx or
anothcr individua stock and thc coccicnt usuay is cacd a cta-
coccicnt in thc cajita assct jricin modc (CAl`). scc thc ooks
y Cochranc (2001) and Tsay (200`) tor morc dctais Lowcvcr,
somc rcccnt studics show that thc cta-coccicnts miht vary ovcr
timc Thc tcrm structurc ot intcrcst ratcs is anothcr cxamjc in
which thc timc cvoution ot thc rcationshij ctwccn intcrcst ratcs
with dicrcnt maturitics is invcstiatcd. scc Tsay (200`) Thc ast
cxamjc is thc rcationshij ctwccn thc jrototyjc ccctricity dcmand
and othcr variacs such as thc incomc or jroduction, thc rca jricc ot
ccctricity, and thc tcmjcraturc, and Chan and `artincz-Chomo
(2003) tound that this rcationshij may chanc ovcr timc Athouh
thc itcraturc is arcady vast and continucs to row switty, as jointcd
out y lhiijs (2001), thc rcscarch in this arca is ,ust cinnin
4.2 Random Walks
Thc asic random wak is
y
t
y
t1
+ e
t
with E
t1
(e
t
) 0, whcrc E
t1
() is thc conditiona cxjcctation ivcn
thc jast intormation uj to timc t 1, which imjics that E
t
(y
t+1
)
y
t
landom waks havc a numcr ot intcrcstin jrojcrtics
1. Thc imjusc-rcsjonsc tunction (llI) ot random wak is onc
at a horizons Thc llI ot a stationary jroccsscs dics out
cvcntuay
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 189
2. Thc torccast variancc ot thc random wak rows incary with
thc torccast horizon \ar(y
t+k
| y
t
) \ar(y
t+k
y
t
) k
2
e
3. Thc autocovarianccs ot random wak arc dcncd in Scction
3!
Statistical Issues
Sujjosc a scrics is cncratcd y a random wak
y
t
y
t1
+ e
t
.
You miht tcst tor a random wak y runnin
y
t
+ y
t1
+ e
t
y thc ordinary cast squarc (OLS) and tcstin whcthcr 1 Low-
cvcr, this is not corrcct sincc thc assumjtions undcryin thc usua
asymjtotic thcory tor OLS cstimatcs and tcst statistics arc vioatcd
Exercise: \hy is thc asymjtotic thcory tor OLS cstimatcs not
truc
4.2.1 Inappropriate Detrending
Thins ct cvcn morc comjicatcd with a trcnd in thc modc Suj-
josc thc truc modc is
y
t
+ y
t1
+ e
t
.
Sujjosc you dctrcnd thc modc and thcn t an Al(1) modc, ic
thc ttin modc is
(1 L)(y
t
b t) e
t
.
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 190
Lut thc aovc modc can c writtcn as toows
y
t
+ t + y
t1
+ e
t
,
so you coud aso run dirccty y on a timc trcnd and acd y ln
this casc,

is iascd downward (undcr-cstimatc) and thc standard
OLS crrors arc miscadin
4.2.2 Spurious (nonsense) Regressions
Sujjosc two scrics arc cncratcd y random waks
y
t
y
t1
+ e
t
, x
t
x
t1
+ v
t
, E(e
t
v
s
) 0, tor a t, s.
`ow, sujjosc you run y
t
on x
t
y OLS
y
t
+ x
t
+ u
t
.
Thc assumjtions tor cassica rcrcssion arc vioatcd and wc tcnd to
scc sinicant morc ottcn than OLS tormuas say wc shoud
Exercise: lcasc conduct a `ontc Caro simuation to vcrity thc
aovc concusion
4.3 Unit Root and Stationary Processes
A morc cncra jroccss than jurc random wak may havc thc toow-
in torm
(1 L) y
t
+
1
(L) e
t
.
Thcsc arc cacd unit root or dicrcncc stationary (LS) jroccsscs ln
thc simjcst casc
1
(L) 1, thc LS jroccss ccomcs a random wak
with dritt
y
t
+ y
t1
+ e
t
.
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 191
Atcrnativcy, wc may considcr a jroccss to c stationary around a
incar trcnd
y
t
t +
2
(L) e
t
.
This jroccss is cacd trcndin stationary (TS) jroccss Thc TS
jroccss can c considcrcd as a sjccia casc ot LS modc lndccd, wc
can writc thc TS modc as
(1 L)y
t
(1 L) t + (1 L)
2
(L)e
t
+
1
(L)e
t
.
Thcrctorc, it TS modc is corrcct, thc LS modc is sti vaid and
stationary Ior morc studics on thc TS modcs, wc rctcr thc rcadcr
to thc jajcrs y lhiijs (2001) and Cai (200o)
Onc can think aout unit roots as thc study ot thc imjications
tor cvcs ot a jroccss that is stationary in dicrcnccs Thcrctorc, it
is vcry imjortant to kccj track ot whcthcr you arc thinkin aout
thc cvc ot thc jroccss y
t
or its rst dicrcncc
Lct us cxaminc thc imjusc rcsjonsc tunction (llI) tor thc aovc
two modcs Ior thc TS modc, thc llI is dctcrmincd y `A joy-
nomia
2
(L), ic b
j
is thc j-th jcriod ahcad rcsjonsc Ior thc LS
modc, a
j
ivcs thc rcsjonsc ot thc dicrcncc (1 L)y
t+j
to a shock
at timc t Thc rcsjonsc ot thc cvc ot thc scrics y
t+j
is thc sum ot
thc rcsjonsc ot thc dicrcnccs lcsjonsc ot y
t+j
to shock at jcriod
t is
IRF
j
(y
t
y
t1
)+(y
t+1
y
t
)+ +(y
t+j
y
t+j1
) a
0
+a
1
+ +a
j
.
4.3.1 Comparison of Forecasts of TS and DS Processes
Thc torccast ot a trcnd-stationary jroccss is as toows

y
t
t+s
(t + s) + b
s
e
t
+ b
s+1
e
t1
+ b
s+2
e
t2
+ .
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 192
Thc torccast ot a dicrcncc stationary jroccss can c writtcn as to-
ows

y
t
t+s

y
t
t+s
+

y
t
t+s1
+ +

y
t
t+1
+ y
t
( + b
s
e
t
+ b
s+1
e
t1
+ b
s+2
e
t2
+ )
+( + b
s1
e
t
+ b
s
e
t1
+ b
s+1
e
t2
+ ) +
+( + b
1
e
t
+ b
2
e
t1
+ b
3
e
t2
+ ) + y
t
or

y
t
t+s
s+y
t
+(b
s
+b
s1
+b
1
)e
t
+(b
s+1
+b
s
+ +b
2
)e
t1
+
To scc thc dicrcncc ctwccn torccasts tor TS and LS jroccsscs, wc
considcr a casc in which b
1
b
2
0 Thcn,
TS

y
t
t+s
(t + s)
DS

y
t
t+s
s + y
t
`cxt, comjarc thc torccast crror tor thc TS and LS jroccsscs Ior
thc TS jroccss
y
t+s

y
t
t+s
((t + s) + e
t+s
+ b
1
e
t+s1
+ b
2
e
t+s2
+ + b
s1
e
t+1
+ b
s
e
t
+
((t + s) + b
s
e
t
+ b
s+1
e
t1
+ ) e
t+s
+ b
1
e
t+s1
+ + b
Thc `SL ot this torccast is
E
_
y
t+s

y
t
t+s
_
2
(1 + b
2
1
+ b
2
2
+ + b
2
s1
)
2
.
Ior thc LS jroccss
y
t+s

y
t
t+s
(y
t+s
+ + y
t+1
+ y
t
) (

y
t
t+s
+ +

y
t
t+1
+ y
t
)
e
t+s
+ (1 + b
1
)e
t+s1
+ (1 + b
1
+ b
2
)e
t+s2
+ + (1 + b
1
+ b
2
Thc `SL ot this torccast is
E
_
y
t+s

y
t
t+s
_
2

_
(1 + b
1
)
2
+ (1 + b
1
+ b
2
)
2
+ + (1 + b
1
+ b
2
+ + b
s1
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 193
`otc that tor thc `SL tor a TS jroccss as thc torccastin horizon
incrcascs, thouh as s ccomcs arc, thc addcd unccrtainty trom
torccastin into tuturc ccomcs nciic
im
s
E
_
y
t+s

y
t
t+s
_
2
(1 + b
2
1
+ b
2
2
+ )
2
and thc imitin `SL is ,ust thc unconditiona variancc ot thc sta-
tionary comjoncnt
2
(L) This is not truc tor thc LS jroccss Thc
`SL tor a LS jroccss docs not convcrc to any xcd vauc as s ocs
to innity To summarizc, tor a TS jroccss, thc `SL rcachcs a nitc
ound as thc torccast horizon ccomcs arc, whcrcas tor a unit root
jroccss, thc `SL cvcntuay rows incary with thc torccast horizon
4.3.2 Random Walk Components and Stochastic Trends
lt is wc known that any LS jroccss can c writtcn as a sum ot a
random wak and a stationary comjoncnt A dccomjosition with
a nicc jrojcrty is duc to thc Lcvcridc-`cson (19S1, L`) lt (1
L)y
t
+
1
(L)e
t
, thcn wc can writc y
t
c
t
+ z
t
, whcrc z
t

+z
t1
+
1
(1)e
t
and c
t

1
(L)e
t
with a

j


k>j
a
k
To scc why
this dccomjosition is truc, wc nccd to noticc that any a joynomia

1
(L) can c writtcn as
1
(L)
1
(1) + (1 L)

1
(L), whcrc
a

j


k>j
a
k
To scc this, ,ust writc it out

1
(1) a
0
+a
1
+a
2
+a
3

(1 L)

1
(L) a
1
a
2
a
3

+a
1
L +a
2
L +a
3
L
a
2
L a
3
L

and thc tcrms
1
(L) rcmain whcn you cancc a thc tcrms Thcrc
arc many ways to dccomjosc a unit root into stationary and random
wak comjoncnts Thc L` dccomjosition is a jojuar choicc ccausc
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 194
it has a sjccia jrojcrty thc random wak comjoncnt is a scnsic
dcnition ot thc trcnd in y
t
Thc comjoncnt z
t
is thc imitin
torccast ot tuturc y, ic todays y jus a tuturc cxjcctcd chancs
in y ln thc L` dccomjosition thc innovations to thc stationary
and random wak comjoncnts arc jcrtccty corrcatcd Considcr an
aritrary comination ot stationary and random wak comjoncnts
y
t
z
t
+ c
t
, whcrc z
t
+ z
t1
+ v
t
and c
t

2
(L)e
t
lt can c
shown that in cvcry dccomjosition ot y
t
into stationary and random
wak comjoncnts, thc variancc ot chancs to thc random wak com-
joncnt is thc samc,
1
(1)
2

2
e
Sincc thc unit root jroccss is comjoscd
ot a stationary jus random wak comjoncnt, thc unit root jroccss
has thc samc variancc ot torccasts chavior as thc random wak whcn
thc horizon is on cnouh
4.4 Trend Estimation and Forecasting
4.4.1 Forecasting a Deterministic Trend
Considcr thc incr dctcrministic modc
y
t
+ t + e
t
, t 1, 2, . . . , T.
Thc h-stcj-ahcad torccast is ivcn y y
t
t+h


+

(t + h), whcrc

and

arc thc OLS cstimatcs ot thc jaramctcrs and Thc
torccast variancc may c comjutcd usin thc toowin tormua
E
_
_
y
t+h
y
t
t+h
_
2
_

2
_

_1 +
1
t
+
t + h (t + 1)/2

t
m=1
(m(t + 1)/2)
2
_

_
2
,
whcrc thc ast ajjroximation is vaid it t (thc jcriod at which torccast
is constructcd) is arc rcativc to thc torccast horizon h
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 195
4.4.2 Forecasting a Stochastic Trend
Considcr thc random wak with dritt
y
t
+ y
t1
+ e
t
, t 2, 3, . . . , T.
Lct

c an cstimatc ot a otaincd trom thc toowin rcrcssion
modc
y
t
+ e
t
.
Thc h-stcj-ahcad torccast is ivcn y y
t
t+h
y
t
+

h Thc torccast
variancc may c comjutcd usin thc toowin tormua
E
_
_
y
t+h
y
t
t+h
_
2
_

2
_
_
_h +
h
2
t 1
_
_
_ h
2
,
whcrc thc ast ajjroximation is vaid it t is arc rcativc to h
4.4.3 Forecasting ARMA models with Deterministic Trends
Thc asic modcs tor dctcrministic and stochastic trcnd inorc jossi-
c short-run uctuations in thc scrics Considcr thc toowin Al`A
modc with dctcrministic trcnd
(L)y
t
+ t + (L)e
t
,
whcrc thc joynomia (L) satiscs thc stationarity condition and
(L) satiscs thc invcrtiiity condition
Thc torccast is constructcd as toows
1. Lincar dctrcndin Lstimatc thc toowin rcrcssion modc
y
t

1
+
2
t +

z
t
,
and comjutc z
t
y
t

2
t
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 196
2. Lstimatc an ajjrojriatc ARMA(p, q) modc tor covariancc
stationary variac z
t
(L)z
t
(L)e
t
Thc cstimatcd
ARMA(p, q) modc may c uscd to construct torccasts h-
jcriod-ahcad torccasts ot z
t
, z
t
t+h

3. Construct thc h-jcriod-ahcad torccast ot y

t
as toows y
t
t+h

z
t
t+h
+

1
+

2
(t + h)
Thc `SL ot y
t
t+h
may c ajjroximatcd y thc `SL ot z
t
t+h

4.4.4 Forecasting of ARIMA Models

Considcr thc torccastin ot timc scrics that arc intcratcd ot ordcr 1
that arc dcscricd y thc ARIMA(p, 1, q) modc
(L)(1 L)y
t
+ (L)e
t
,
whcrc thc joynomia (L) satiscs thc stationarity condition and
(L) satiscs thc invcrtiiity condition Thc torccast is constructcd
as toows
1. Comjutc thc rst dicrcncc ot y
t
, ic z
t
y
t

2. Lstimatc an ajjrojriatc ARMA(p, q) modc tor covariancc

stationary variac z
t
(L)z
t
(L)e
t
Thc cstimatcd
ARMA(p, q) modc may c uscd to construct torccasts h-
jcriod-ahcad torccasts ot z
t
, z
t
t+h

3. Construct thc h-jcriod-ahcad torccast ot y

t
as toows
y
t
t+h
y
t
+ z
t
t+h
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 197
4.5 Unit Root Tests
Athouh it miht c intcrcstin to know whcthcr a timc scrics has
a unit root, scvcra jajcrs havc arucd that thc qucstion can not c
answcrcd on thc asis ot a nitc samjc ot oscrvations `cvcrthccss,
you wi havc to conduct tcst ot unit root in doin cmjirica jro,ccts
lt can c donc usin intorma or intorma mcthods Thc intorma
mcthods invovc insjcctin a timc scrics jot ot thc data and com-
jutin thc autocorrcation coccicnts, as what wc did in Chajtcrs
1 and 2 lt a scrics has a stochastic trcnd, thc rst autocorrcation
coccicnt wi c ncar onc A sma rst autocorrcation coccicnt
comincd with a timc scrics jot that has no ajjarcnt trcnd sucsts
that thc scrics docs not havc a trcnd Lickcy-Iucrs (199, LI) tcst
is a most jojuar torma statistica jroccdurc tor unit root tcstin
4.5.1 The Dickey-Fuller and Augmented Dickey-Fuller Tests
Thc startin joint tor thc LI tcst is thc autorcrcssivc modc ot
ordcr onc, Al(1)
y
t
+ y
t1
+ e
t
. (!1)
lt 1, y
t
is nonstationary and contains a stochastic trcnd Thcrc-
torc, within thc Al(1) modc, thc hyjothcsis that y
t
has a trcnd can
c tcstcd y tcstin
H
0
1 vs. H
1
< 1.
This tcst is most casiy imjcmcntcd y cstimatin a modicd vcrsion
ot (!1) Sutract y
t1
trom oth sidcs and ct 1 Thcn,
modc (!1) ccomcs
y
t
+ y
t1
+ e
t
(!2)
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 198
Table 4.1: Large-sample critical values for the ADF statistic
Deterministic regressors 10% 5% 1%
Intercept only -2.57 -2.86 -3.43
Intercept and time trend -3.12 -3.41 -3.96
and thc tcstin hyjothcsis is
H
0
0 vs. H
1
< 0.
Thc OLS t-statistic in (!2) tcstin 0 is known as thc Lickcy-
Iucr statistic
Thc cxtcnsion ot thc LI tcst to thc AR(p) modc is a tcst ot
thc nu hyjothcsis H
0
0 aainst thc onc-sidcd atcrnativc
H
1
< 0 in thc toowin rcrcssion
y
t
+ y
t1
+
1
y
t1
+ +
p
y
tp
+ e
t
. (!3)
ndcr thc nu hyjothcsis, y
t
has a stochastic trcnd and undcr thc
atcrnativc hyjothcsis, y
t
is stationary lt instcad thc atcrnativc
hyjothcsis is that y
t
is stationary around a dctcrministic incar timc
trcnd, thcn this trcnd must c addcd as an additiona rcrcssor in
modc (!3) and thc LI rcrcssion ccomcs
y
t
+ t + y
t1
+
1
y
t1
+ +
p
y
tp
+ e
t
. (!!)
This is cacd thc aumcntcd Lickcy-Iucr (ALI) tcst and thc tcst
statistic is thc OLS t-statistic tcstin that 0 in cquation (!!)
Thc ALI statistic docs not havc a norma distriution, cvcn in
arc samjcs Critica vaucs tor thc onc-sidcd ALI tcst dcjcnd on
whcthcr thc tcst is ascd on cquation (!3) or (!!) and arc ivcn in
Tac !1 Tac 11 ot Lamiton (199!, j`02) jrcscnts a summary
ot LI tcsts tor unit roots in thc ascncc ot scria corrcation tor tcstin
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 199
Table 4.2: Summary of DF test for unit roots in the absence of serial correlation
Case 1:
True process: y
t
= y
t1
+ u
t
, u
t
N(0,
2
) iid.
Estimated regression: y
t
= y
t1
+ u
t
.
T( 1) has the distribution described under the heading Case 1 in Table B.5.
( 1)/
2

has the distribution described under Case 1 in Table B.6.
Case 2:
True process: y
t
= y
t1
+ u
t
, u
t
N(0,
2
) iid.
Estimated regression: y
t
= + y
t1
+ u
t
.
T( 1) has the distribution described under Case 2 in Table B.5.
( 1)/
2

has the distribution described under Case 2 in Table B.6.
OLS F-test of join hypothesis that = 0 and = 1 has the distribution described under Case 2
in Table B.7.
Case 3:
True process: y
t
= + y
t1
+ u
t
, = 0, u
t
N(0,
2
) iid.
Estimated regression: y
t
= + y
t1
+ u
t
.
( 1)/
2

N(0, 1).
Case 4:
True process: y
t
= + y
t1
+ u
t
, = 0, u
t
N(0,
2
) iid.
Estimated regression: y
t
= + y
t1
+ t + u
t
.
T( 1) has the distribution described under Case 4 in Table B.5.
( 1)/
2

has the distribution described under Case 4 in Table B.6.
OLS F-test of join hypothesis that = 1 and = 0 has the distribution described under Case 4
in Table B.7.
thc nu hyjothcsis ot unit root aainst somc dicrcnt atcrnativc
hyjothcsis lt is vcry imjortant tor you to undcrstand what your
atcrnativc hyjothcsis is in conductin unit root tcsts l rcjroducc
this tac hcrc, ut you nccd to chcck Lamitons (199!) ook tor thc
critica vaucs ot LI statistic tor dicrcnt cascs Thc critica vaucs
arc jrcscntcd in thc Ajjcndix ot thc ook
ln thc aovc modcs (! cascs), thc asic assumjtion is that u
t
is iid Lut this assumjtion is vioatcd it u
t
is scriay corrcatcd
and jotcntiay hctcroskcdastic To takc account ot scria corrcation
and jotcntia hctcroskcdasticity, onc way is to usc thc ll tcst jro-
joscd y lhiijs and lcrron (19SS) Ior othcr tcsts tor unit roots,
jcasc rcad thc ook y Lamiton (199!, j`32) Somc rcccnt tcstin
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 200
mcthods havc ccn jrojoscd Ior cxamjc, 1uh (200`) uscd thc
tunctiona coccicnt tyjc modc ot Cai, Ian and Yao (2000) to tcst
unit root and lhiijs and lark (200`) cmjoycd thc nonjaramctric
rcrcssion Iinay, noticc that in R, thcrc arc at cast thrcc jackacs
to jrovidc unit root tcsts such as tseries, urca and uroot
4.5.2 Cautions
Thc most rciac way to handc a trcnd in a scrics is to transtorm
thc scrics so that it docs not havc thc trcnd lt thc scrics has a
stochastic trcnd, unit root, thcn thc rst dicrcncc ot thc scrics docs
not havc a trcnd ln jracticc, you can rarcy c surc whcthcr a scrics
has a stochastic trcnd or not lcca that a taiurc to rc,cct thc nu
hyjothcsis doc not ncccssariy mcan that thc nu hyjothcsis is truc.
it simjy mcans that thcrc arc not cnouh cvidcncc to concudc that
it is tasc Thcrctorc, taiurc to rc,cct thc nu hyjothcsis ot a unit
root usin thc ALI tcst docs not mcan that thc scrics actuay has
a unit root Lavin said that, cvcn thouh taiurc to rc,cct thc nu
hyjothcsis ot a unit root docs not mcan thc scrics has a unit root,
it sti can c rcasonac to ajjroximatc thc truc autorcrcssivc root
as cquain onc and usc thc rst dicrcncc ot thc scrics rathcr than
its cvcs
4.6 Structural Breaks
Anothcr tyjc ot nonstationarity ariscs whcn thc jojuation rcrcs-
sion tunction chancs ovcr thc samjc jcriod This may occur c-
causc ot chancs in cconomic joicy, chancs in thc structurc ot thc
cconomy or industry, cvcnts that chanc thc dynamics ot sjccic
industrics or rm rcatcd quantitics (invcntorics, sacs, jroduction),
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 201
ctc lt such chancs, cacd breaks, occur thcn rcrcssion modcs
that nccct thosc chancs cad to a miscadin intcrcncc or torccast-
in
Lrcaks may rcsut trom a discrctc chanc (or chancs) in thc joj-
uation rcrcssion coccicnts at distinct datcs or trom a radua cvo-
ution ot thc coccicnts ovcr a oncr jcriod ot timc Liscrctc rcaks
may c a rcsut ot somc ma,or chancs in cconomic joicy or in thc
cconomy (oi shocks) whic radua rcaks, jojuation jaramctcrs
cvovc sowy ovcr timc, may c a rcsut ot sow cvoution ot cconomic
joicy
lt a rcak occurs in thc jojuation jaramctcrs durin thc samjc,
thcn thc OLS rcrcssion cstimatcs ovcr thc tu samjc wi cstimatc
a rcationshij that hods on avcrac
4.6.1 Testing for Breaks
Tcsts tor rcaks in thc rcrcssion jaramctcrs dcjcnd on whcthcr thc
rcak datc is know or not lt thc datc ot thc hyjothcsizcd rcak in
thc coccicnts is known, thcn thc nu hyjothcsis ot no rcak can c
tcstin usin a dummy variac
Considcr thc toowin modc
y
t

0
+
1
y
t1
+
1
x
t1
+
0
D
t
() +
1
D
t
() y
t1
+
2
D
t
() x
t1
+ u
t

_
_
_

0
+
1
y
t1
+
1
x
t1
+ u
t
, it t ,
(
0
+
0
) + (
1
+
1
)y
t1
+ (
1
+
2
)x
t1
+ u
t
, it t > ,
whcrc dcnotcs thc hyjothcsizcd rcak datc, D
t
() is a inary
variac that cquas zcro ctorc thc rcak datc and onc attcr, ic
D
t
() 0 it t and D
t
() 1 it t > ndcr thc nu hyjoth-
csis ot no rcak,
0

1

2
0, and thc hyjothcsis ot a rcak
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 202
can c tcstcd usin thc I-statistic This is cacd a Chow test tor
a rcak at a known rcak datc lndccd, thc aovc structura rcak
modc can c rcardcd as a sjccia casc ot thc toowin trcndin
timc scrics modc
y
t

0
(t) +
1
(t) y
t1
+
1
(t) x
t1
+ u
t
.
Ior morc discussions, scc Cai (200o)
lt thcrc arc morc variacs or morc as, this tcst can c cxtcndcd
y constructin inary variac intcraction variacs tor a thc dcjcn-
dcnt variacs This ajjroach can c modicd to chcck tor a rcak
in a susct ot thc coccicnts Thc rcak datc is unknown in most
ot thc ajjications ut you may susjcct that a rcak occurrcd somc-
timc ctwccn two datcs,
0
and
1
Thc Chow tcst can c modicd
to handc this y tcstin tor rcak at a jossic datcs t in ctwccn

0
and
1
, thcn usin thc arcst ot thc rcsutin I-statistics to tcst
tor a rcak at an unknown datc This modicd tcst is ottcn cacd
Quandt likelihood ratio (QLl) statistic or thc sup-Wald statistic
QLR max{F(
0
), F(
0
+ 1), , F(
1
)}.
Sincc thc QLl statistic is thc arcst ot many I-statistics, its distri-
ution is not thc samc as an individua I-statistic Thc critica vaucs
tor QLl statistic must c otaincd trom a sjccia distriution This
distriution dcjcnds on thc numcr ot rcstriction cin tcstcd, m,

0
,
1
, and thc susamjc ovcr which thc I-statistics arc comjutcd
cxjrcsscd as a traction ot thc tota samjc sizc
Ior thc arc-samjc ajjroximation to thc distriution ot thc QLl
statistic to c a ood onc, thc susamjc cndjoints,
0
and
1
, can
not c too cosc to thc cnd ot thc samjc That is why thc QLl
statistic is comjutcd ovcr a trimmcd susct ot thc samjc A
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 203
Table 4.3: Critical Values of the QLR statistic with 15% Trimming
Number of restrictions (m) 10% 5% 1%
1 7.12 8.68 12.16
2 5.00 5.86 7.78
3 4.09 4.71 6.02
4 3.59 4.09 5.12
5 3.26 3.66 4.53
6 3.02 3.37 4.12
7 2.84 3.15 3.82
8 2.69 2.98 3.57
9 2.58 2.84 3.38
10 2.48 2.71 3.23
jojuar choicc is to usc 1`/ trimmin, that is, to sct tor
0
0.1`T
and
1
0.S`T \ith 1`/ trimmin, thc I-statistic is comjutcd
tor rcak datcs in thc ccntra 0/ ot thc samjc Tac !3 jrcscnts
thc critica vaucs tor QLl statistic comjutcd with 1`/ trimmin
This tac is trom Stock and \atson (2003) and you shoud chcck
thc ook tor a comjctc tac Thc QLl tcst can dctcct a sinc
rcak, mutijc discrctc rcaks, and a sow cvoution ot thc rcrcssion
jaramctcrs
lt thcrc is a distinct rcak in thc rcrcssion tunction, thc datc at
which thc arcst Chow statistic occurs is an cstimator ot thc rcak
datc ln R, thc jackacs strucchange and segmented jrovidc
scvcra tcstin mcthods tor tcstin rcaks or you usc thc tunction
StructTS
4.6.2 Zivot and Andrewss Testing Procedure
Somctimcs, you woud susjcct that a scrics may cithcr havc a unit
root or c a trcnd stationary jroccss that has a structura rcak at
somc unknown jcriod ot timc and you woud want to tcst thc nu
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 204
hyjothcsis ot unit root aainst thc atcrnativc ot a trcnd stationary
jroccss with a structura rcak This is cxacty thc hyjothcsis tcstcd
y Zivot and Andrcwss (1992) tcst ln this tcstin jroccdurc, thc
nu hyjothcsis is a unit root jroccss without any structura rcaks
and thc atcrnativc hyjothcsis is a trcnd stationary jroccss with jos-
sic structura chanc occurrin at an unknown joint in timc Zivot
and Andrcws (1992) sucstcd cstimatin thc toowin rcrcssion
x
t
+ DU
t
()+ t+ DT
t
()+x
t1
+
k

i=1
c
i
x
ti
+e
t
, (!`)
whcrc T
B
/T is thc rcak traction. DU
t
() 1 it t > T and
0 othcrwisc. DT
t
() t T it t > T and 0 othcrwisc. and x
t
is thc timc scrics ot intcrcst This rcrcssion aows oth thc sojc
and intcrccjt to chanc at datc T
B
`otc that tor t T (t T
B
)
modc (!`) ccomcs
x
t
+ t + x
t1
+
k

i=1
c
i
x
ti
+ e
t
,
whic tor t > T (t > T
B
) modc (!`) ccomcs
x
t
| + | + | t + (t T
B
)| + x
t1
+
k

i=1
c
i
x
ti
+ e
t
.
`odc (!`) is cstimatcd y OLS with thc rcak joints ranin ovcr
thc samjc and thc t-statistic tor tcstin 1 is comjutcd Thc
minimum t-statistic is rcjortcd Critica vaucs tor 1/, `/ and 10/
critica vaucs arc `.3!, !.S and !.`S, rcsjcctivcy Thc ajjro-
jriatc numcr ot as in dicrcnccs is cstimatcd tor cach vauc ot
lcasc rcad thc jajcr y Sadorsky (1999) tor morc dctais aout this
mcthod and cmjirica ajjications
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 205
4.6.3 Cautions
Thc ajjrojriatc way to ad,ust tor a rcak in thc jojuation jaram-
ctcrs dcjcnds on thc sourccs ot thc rcak lt a distinct rcak occurs
at a sjccic datc, this rcak wi c dctcctcd with hih jroaiity
y thc QLl statistic, and thc rcak datc can c cstimatcd Thc
rcrcssion modc can c cstimatcd usin a dummy variac lt thcrc
is a distinct rcak, thcn intcrcncc on thc rcrcssion coccicnts can
jrocccd as usua usin t-statistics tor hyjothcsis tcstin Iorccasts
can c jroduccd usin thc cstimatcd rcrcssion modc that ajjics
to thc cnd ot thc samjc Thc jrocm is morc dicut it thc rcak
is not distinct and thc jaramctcrs sowy cvovc ovcr timc ln this
casc a statc-sjacc modcin is rcquircd
4.7 Problems
1. You wi uid a timc-scrics modc tor rca oi jricc istcd in thc
tcnth coumn in c MacroData.xls
(a) Construct rajhs ot timc scrics data timc jots and scattcr
jots ot thc cvcs ot rca oi jriccs (Ol) and thc o-dicrcncc
ot oi jriccs Thc o-dicrcncc ot oi jricc is dcncd as toows
o(OP
t
) o(OP
t
) o(OP
t1
) Commcnt your rcsuts
() Try to idcntity a structurc tor cvcs ot rca oi jriccs and thc
o-dicrcncc ot oi jriccs y usin ACI and lACI Commcnt
your rcsuts
(c) \c wi not cstimatc Al`A modcs yct So, cstimatc Al(j)
modc, 1 p S Comjutc Akaikc lntormation Critcria (AlC)
or AlCC and Schwarz lntormation Critcria (SlC) lrcscnt your
rcsuts Choosc thc Al a cnth ascd on thc AlC or AlCC
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 206
\hich a cnth woud you choosc ascd on thc AlC or AlCC
or SlC Commcnt
(d) Lstimatc thc Al(j) modc with thc ojtima a cnth lrcscnt
your rcsuts niccy
(c) Conduct thc dianostic chcckin ot thc cstimatcd Al modc
You nccd do thc toowins and commcnt your rcsuts (i) Con-
struct rajhs ot rcsiduas (timc scrics jots, scattcrjots, squarcd
rcsiduas) (ii) Chcck tor scria corrcation usin samjc ACI
(iii) Chcck tor scria corrcation usin L,un-Lox tcst statistic
(iv) Conduct 1arquc-Lcra tcst ot normaity and makc a Q-Q
jot (v) Lstimatc thc toowin Al(1) modc tor thc cstimatcd
squarcd rcsiduas and tcst thc nu hyjothcsis that thc sojc is
insinicant Low woud intcrjrct your rcsuts \hat docs it
say aout thc constancy ot variancc (vi) Lascd on dianostic
chcckin in thc aovc, can you usc thc modc or you shoud o
ack to idcntication ot a structurc in ()
(t) ls thcrc any structurc chanc tor oi jricc
Note: Thc 1arquc-Lcra (19S0, 19S) tcst cvauatcs thc hyjothcsis that
X has a norma distriution with unsjccicd mcan and variancc,
aainst thc atcrnativc that X docs not havc a norma distriu-
tion Thc tcst is ascd on thc samjc skcwncss and kurtosis ot
X Ior a truc norma distriution, thc samjc skcwncss shoud
c ncar 0 and thc samjc kurtosis shoud c ncar 3 A tcst has
thc toowin cncra torm
JB
T
o
_
_
_S
k
+
(K 3)
2
!
_
_
_
2
2
,
whcrc S
k
and K arc thc mcasurcs ot skcwncss and kurtosis rc-
sjcctivcy To usc thc uid-in tunction tor thc 1arquc-Lcra tcst
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 207
in thc jackac tseries in R, thc command tor thc 1arquc-Lcra
tcst is
library(tseries) # call the package "tseries"
jb=jarque.bera.test(x) # x is the series for the test
print(jb) # print the testing result
2. Lascd on thc cconomic thcory, thc dcscrijtion ot ination su-
csts two tundamcnta causcs, cxccss monctary rowth (tastcr
than rca outjut) and thc dissijation ot cxtcrna shocks Thc
jrccisc mcchanisms at work, ajjrojriatc a structurcs arc not
jcrtccty dcncd ln this cxcrcisc, you wi cstimatc thc toowin
simjc modc
o(P
t
)
1
+
2
| o(M1
t1
) o(Q
t1
)|+
3
o(P
t1
)+u
t
,
whcrc P
t
is thc quartcry jricc cvc (Cll) istcd in thc cihth
coumn in MacroData.xls, Q
t
is thc quartcry rca outjut
(istcd in thc third coumn), and M1
t
is thc quartcry moncy
stock (istcd in thc thirtccnth coumn)
(a) `iccy jrcscnt thc rcsuts ot OLS cstimation Commcnt your
rcsuts
() Lxjain what may c an advantac ot thc aovc modc com-
jarcd to a simjc autorcrcssivc modc ot jriccs To answcr this
qucstion, you miht nccd to do somc statistica anaysis
(c) ls thcrc any structurc chanc tor Cll
(d) Any sucstions to uid a cttcr modc
3. ln this cxcrcisc, you wi uid a timc scrics modc tor thc rca
GLl jroccss istcd in thc sccond coumn in MacroData.xls
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 208
(a) Luid intormativcy and tormay an Al(j) modc with thc oj-
tima a cnth ascd on somc critcrion and conduct thc dia-
nostic chcckin ot thc cstimatcd Al modc
() Lascd on thc data 19`91-19993 construct torccasts tor thc quar-
tcrs 1999! -20023 lot thc constructcd torccasts and thc rca-
izcd vaucs Commcnt your rcsuts
4. You nccd to rcjicatc somc ot thc stcjs in thc anaysis ot oi
jricc shocks and stock markct activity conductcd y Sadorsky
(1999) You shoud writc your rcjort in such a way that an
outsidc rcadcr may undcrstand what thc rcjort is aout and
what you arc doin \ritc a rctcrcc rcjort tor thc jajcr ot
Sadorsky (1999) Onc jossic structurc ot thc rctcrcc rcjort is
(a) Summary ot thc jajcr (This assurcs that you rcay rcad
thc jajcr carctuy) (i) ls thc cconomic,nancia qucstion ot
rccvancc (ii) \hat havc you carncd trom rcadin this jajcr
(iii) \hat contriution docs this jajcr makc to thc itcraturc
() Can you think ot intcrcstin cxtcnsions tor thc jajcr
(c) Lxjository quaity ot thc jajcr (i) ls thc jajcr wc struc-
turcd lt not, sucst and atcrnativc structurc (ii) ls thc jajcr
casy to rcad
5. Anayzc thc stochastic jrojcrtics ot thc toowin intcrcst ratcs
(i) tcdcra tunds ratc, (ii) 90-day T-i ratc, (iii) 1-ycar T-ond
intcrcst ratc, (iv) `-ycar T-ond intcrcst ratc. (v) 10-ycar T-ond
intcrcst ratc Thc intcrcst ratcs may c tound in thc Lxcc c
IntRates.xls
(a) sc thc ALI or ll ajjroach to tcst thc nu hyjothcsis that
vc intcrcst ratc arc dicrcncc stationary aainst thc atcrnativc
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 209
that thcy arc stationary Lxjain carctuy how you conduct thc
tcst
() sc thc ALI or ll ajjroach to tcst thc nu hyjothcsis that
vc intcrcst ratc arc dicrcncc stationary aainst thc atcrnativc
that thcy arc stationary around a dctcrministic trcnd Lxjain
carctuy how you conduct thc tcst
(c) sc thc QLl tcstin jroccdurc to tcst whcthcr thcrc was at cast
onc structura rcak in intcrcst ratcs scrics
4.8 Computer Code
Thc toowin R commands arc uscd tor makin thc rajhs in this
chajtcr
# 5-20-2006
graphics.off()
###################################################################
y=read.csv("c:\\teaching\\time series\\data\\MacroData.csv",header=T,skip=1)
cpi=y[,8]
qt=y[,3]
m0=y[,12]
m1=y[,13]
m2=y[,14]
m3=y[,15]
op=y[,10]
v0=cpi*qt/m0
v1=cpi*qt/m1
v2=cpi*qt/m2
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 210
v3=cpi*qt/m3
vt=cbind(v0,v1,v2,v3)
win.graph()
par(mfrow=c(2,2),mex=0.4,bg="light blue")
ts.plot(cpi,type="l",lty=1,ylab="",xlab="")
title(main="CPI",col.main="red")
ts.plot(qt,type="l",lty=1,ylab="",xlab="")
title(main="Industry Output",col.main="red")
ts.plot(qt,type="l",lty=1,ylab="",xlab="")
title(main="Oil Price",col.main="red")
win.graph()
par(mfrow=c(2,2),mex=0.4,bg="light grey")
ts.plot(m0,type="l",lty=1,ylab="",xlab="")
title(main="Money Aggregate",col.main="red")
ts.plot(m1,type="l",lty=1,ylab="",xlab="")
title(main="Money Aggregate",col.main="red")
ts.plot(m2,type="l",lty=1,ylab="",xlab="")
title(main="Money Aggregate",col.main="red")
ts.plot(m3,type="l",lty=1,ylab="",xlab="")
title(main="Money Aggregate",col.main="red")
win.graph()
par(mfrow=c(2,2),mex=0.4,bg="yellow")
ts.plot(v0,type="l",lty=1,ylab="",xlab="")
title(main="Velocity",col.main="red")
ts.plot(v1,type="l",lty=1,ylab="",xlab="")
title(main="Velocity",col.main="red")
ts.plot(v2,type="l",lty=1,ylab="",xlab="")
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 211
title(main="Velocity",col.main="red")
ts.plot(v3,type="l",lty=1,ylab="",xlab="")
title(main="Velocity",col.main="red")
library(tseries) # call library(tseries)
library(urca) # call library(urca)
library(quadprog)
library(zoo)
adf_test=adf.test(cpi) # Augmented Dickey-Fuller test
print(adf_test)
adf_test=pp.test(cpi) # do Phillips-Perron test
print(adf_test)
#adf_test2=ur.df(y=cpi,lag=5,type=c("drift"))
#print(adf_test2)
adf_test=adf.test(op) # Augmented Dickey-Fuller test
print(adf_test)
adf_test=pp.test(op) # do Phillips-Perron test
print(adf_test)
for(i in 1:4){
adf_test=pp.test(vt[,i])
print(adf_test)
adf_test=adf.test(vt[,i])
print(adf_test)
}
###################################################################
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 212
y=read.csv("c:\\teaching\\time series\\data\\MacroData.csv",header=T,skip=1)
op=y[,10]
library(strucchange)
win.graph()
par(mfrow=c(2,2),mex=0.4,bg="green")
op=ts(op)
fs.op <- Fstats(op ~ 1) # no lags and covariate
plot(op,type="l")
plot(fs.op)
sctest(fs.op)
## visualize the breakpoint implied by the argmax of the F statistics
plot(op,type="l")
lines(breakpoints(fs.op))
#####################################
# The following is the example from R
######################################
win.graph()
par(mfrow=c(2,2),mex=0.4,bg="red")
if(! "package:stats" %in% search()) library(ts)
## Nile data with one breakpoint: the annual flows drop
## because the first Ashwan dam was built
data(Nile)
plot(Nile)
## test whether the annual flow remains constant over the
fs.nile <- Fstats(Nile ~ 1)
plot(fs.nile)
sctest(fs.nile)
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 213
plot(Nile)
lines(breakpoints(fs.nile))
###################################################################
4.9 References
Beveridge, S. and C.R. Nelson (1981). A new approach to decomposition of economic
time series into permanent and transitory components with particular attention to
measurement of the business cycle. Journal of Monetary Economics, 7, 151-174.
Cai, Z. (2006). Trending time varying coecient time series models with serially correlated
errors. Forthcoming in Journal of Econometrics.
Cai, Z., J. Fan, and Q. Yao (2000). Functional-coecient regression models for nonlinear
time series. Journal of the American Statistical Association, 95, 941-956.
Chang, Y., J.Y. Park and P.C.B. Phillips (2001). Nonlinear econometric models with co-
integrated and deterministically treading regressors. Econometrics Journal, 4, 1-36.
Chang, Y. and E. Martinez-Chombo (2003). Electricity demand analysis using cointegration
and error-correction models with time varying parameters: The Mexican case, Working
paper, Department of Economics, Rice University.
Cochrane, J.H. (1997). Time series for macroeconomics and nance. Lecture Notes.
http://gsb.uchicago.edu/fac/john.cochrane/research/Papers/timeser1.pdf
Cochrane, J.H. (2001). Asset Pricing. New Jersey: Princeton University Press.
Dickey, D.A. and W.A. Fuller (1979). Distribution of the estimators for autoregressive time
series with a unit root. Journal of the American Statistical Association, 74, 427-431.
Hamilton, J.D. (1994). Time Series Analysis. Princeton University Press.
Heij, C., P. de Boer, P.H. Franses and H.K. van Dijk (2004). Econometric Methods with
Applications in Business and Economics. Oxford University Press.
Hendry, F.D. and K. Juselius (2000). Explaining cointegration analysis: Part I. Journal of
Energy, 21, 1-42.
Jarque, C.M. and A.K. Bera (1980). Ecient tests for normality, homoscedasticity and
serial independence of regression residuals. Economics Letters, 6, 255-259.
Jarque, C.M. and A.K. Bera (1987). A test for normality of observations and regression
residuals. International Statistical Review, 55, 163-172.
CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS 214
Juhl, T. (2005). Functional coecient models under unit root behavior. Econometrics
Journal, 8, 197-213.
Park, J.Y. and P.C.B. Phillips (1999). Asymptotics for nonlinear transformations of inte-
grated time series. Econometric Theory, 15, 269-298.
Park, J.Y. and P.C.B. Phillips (2002). Nonlinear regressions with integrated time series.
Econometrica, 69, 117-161.
Phillips, P.C.B. (2001). Trending time series and macroeconomic activity: Some present
and future challenges. Journal of Econometrics, 100, 21-27.
Phillips, P.C.B. and J. Park (2005). Non-stationary density and kernel autoregression.
Under revision for Econometrics Theory.
Phillips, P.C.B. and P. Perron (1988). Testing for a unit root in time series regression.
Biometrika, 75, 335-346.
Sadorsky, P. (1999). Oil price shocks and stock market activity. Energy Economics, 21,
449-469.
Stock, J.H. and M.W. Watson (2003). Introduction to Econometrics. Addison-Wesley.
Tsay, R.S. (2005). Analysis of Financial Time Series, 2th Edition. John Wiley & Sons,
New York.
Zivot, E. and D.W.K. Andrews (1992). Further evidence on the great crash, the oil price
shock and the unit root hypothesis. Journal of Business and Economics Statistics, 10,
251-270.
Chapter 5
Vector Autoregressive Models
5.1 Introduction
A univariatc autorcrcssion is a sinc-cquation, sinc-variac incar
modc in which thc currcnt vauc ot a variac is cxjaincd y its own
acd vaucs `utivariatc modcs ook ikc thc univariatc modcs
with thc cttcrs rc-intcrjrctcd as vcctors and matriccs Considcr a
mutivariatc timc scrics
x
t

_
_
y
t
z
t
_
_
.
lcca that y mutivariatc whitc noisc e
t
N(0, ), wc mcan that
e
t

_
_
v
t
u
t
_
_
, E(e
t
) 0, E(e
t
e

t
)
_
_

2
v

vu

uv

2
u
_
_
, E(e
t
e

tj
) 0.
Thc Al(1) modc tor thc random vcctor x
t
is x
t
x
t1
+e
t
which
in mutivariatc tramcwork mcans that
_
_
y
t
z
t
_
_

_
_

yy

yz

zy

zz
_
_
_
_
y
t1
z
t1
_
_
+
_
_
v
t
u
t
_
_
.
`oticc that oth acd y and z ajjcar in cach cquation which mcans
that thc mutivariatc Al(1) jroccss (\Al) cajturcs cross-variac
dynamics (co-movcmcnts)
215
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 216
A cncra \Al is an n-cquation, n-variac incar modc in which
cach variac is cxjaincd y its own acd vaucs, jus currcnt and
jast vaucs ot thc rcmainin n 1 variacs
y
t

1
y
t1
+ +
p
y
t1
+ e
t
, (`1)
whcrc
y
t

_
_
_
_
_
_
_
_
_
_
y
1t
y
2t

y
nt
_
_
_
_
_
_
_
_
_
_
, e
t

_
_
_
_
_
_
_
_
_
_
e
1t
e
2t

e
nt
_
_
_
_
_
_
_
_
_
_
,
i

_
_
_
_
_
_
_
_
_
_

(i)
11

(i)
12

(i)
1n

(i)
21

(i)
22

(i)
2n

(i)
n1

(i)
n2

(i)
nn
_
_
_
_
_
_
_
_
_
_
,
and thc crror tcrms e
t
havc a variancc-covariancc matrix Thc
namc vcctor autorcrcssion is usuay uscd in thc jacc ot vcctor
Al`A ccausc it is vcry uncommon to cstimatc movin avcrac
tcrms Autorcrcssions arc casy to cstimatc ccausc thc OLS as-
sumjtions sti ajjy and cach cquation may c cstimatcd y ordi-
nary cast squarcs rcrcssion Thc `A tcrms havc to c cstimatcd y
maximum ikcihood Lowcvcr, sincc cvcry `A jroccss has Al()
rcjrcscntation, jurc autorcrcssion can ajjroximatc `A jroccss it
cnouh as arc incudcd in Al rcjrcscntation
Thc usctuncss ot \Al modcs is that macro cconomists can do
tour thins (1) dcscric and summarizc macrocconomic data. (2)
makc macrocconomic torccasts. (3) quantity what wc do or do not
know aout thc truc structurc ot thc macro cconomy. (!) advisc
macrocconomic joicymakcrs ln data dcscrijtion and torccastin,
\Als havc jrovcd to c jowcrtu and rciac toos that arc now in
cvcryday usc loicy anaysis is morc dicut in \Al tramcwork c-
causc it rcquircs dicrcntiatin ctwccn corrcation and causation, so
cacd idcntication jrocm Lconomic thcory is rcquircd to sovc
thc idcntication jrocm Standard jracticc in \Al anaysis is
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 217
to rcjort rcsuts trom Granger-causality, impulse responses
and forecast error variance decomposition which wi c dis-
cusscd rcsjcctivcy in thc ncxt scctions Ior morc aout thc history
and rcccnt dcvcojmcnts as wc as ajjication, scc thc jajcr y
Stock and \atson (2001)
\Al modcs comc in thrcc varictics reduced form, recursive
andstructural forms Lcrc, wc ony tocus on thc rst onc and
thc dctais tor thc ast two can c tound in Lamiton (199!, Chajtcr
11) A rcduccd torm \Al jroccss ot ordcr p has thc samc torm as
cquation (`1)
y
t

1
y
t1
+ +
p
y
t1
+ e
t
, (`2)
whcrc y
t
is an n1 vcctor ot thc variacs,
i
is an nn matrix ot
thc jaramctcrs, c is an n1 vcctor ot constants, and e
t
N(0, )
Thc crror tcrms in thcsc rcrcssions arc thc surjrisc movcmcnts in
thc variacs attcr takin thcir jast vaucs into account Thc modc
(`2) can c jrcscntcd in many dicrcnt ways (1) usin thc a
ojcrator notation as
(L) y
t
c + e
t
,
whcrc (L) I
n

1
L
p
L
p
. (2) in thc matrix notation
as
Y c + X + E
whcrc E is a T n matrix ot thc disturanccs and Y is a T n
matrix ot thc oscrvations. (3) in tcrms ot dcviations trom thc mcan
(ccntcrizcd)
Ior cstimatin \Al modcs, scvcra mcthods arc avaiac simjc
modcs can c ttcd y thc tunction ar() in thc jackac stats (uit-
in), morc caoratc modcs arc jrovidcd y estVARXls() in thc
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 218
jackac dse1 and a Laycsian ajjroach is avaiac in thc jackac
MSBVAR
5.1.1 Properties of VAR Models
A vcctor jroccss is said to c a covariancc-stationary (wcaky station-
ary) it its rst and sccond momcnts do not dcjcnd on t Thc \Al(j)
modc in cquation (`1) can c writtcn in thc torm ot \Al(1), cacd
comjanion torm, jroccss as

t
F
t1
+ v
t
, (`3)
whcrc

t

_
_
_
_
_
_
_
_
_
_
y
t
y
t1

y
tp+1
_
_
_
_
_
_
_
_
_
_
, v
t

_
_
_
_
_
_
_
_
_
_
e
t
0

0
_
_
_
_
_
_
_
_
_
_
, F
_
_
_
_
_
_
_
_
_
_
_
_
_

1

2

3

p1

p
I
n
0 0 0 0
0 I
n
0 0 0

0 0 0 I
n
0
_
_
_
_
_
_
_
_
_
_
_
_
_
.
To undcrstand conditions tor stationarity ot a vcctor jroccss, notc
trom thc aovc cquation that

t+s
v
t+s
+ Fv
t+s1
+ F
2
v
t+s2
+ + F
s1
v
t+1
+ F
s

t
.
Proposition 4.1 Thc \Al jroccss is covariancc-stationary it thc
cicnvaucs ot thc matrix F arc css than unity in asoutc vauc
Ior covariancc-stationary n-dimcnsiona vcctor jroccss, thc j-th
auto-covariancc is dcncd to c thc toowin n n matrix

j
E(y
t
)(y
tj
)

.
`otc that
j

j
ut
j

CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 219

A vcctor movin avcrac jroccss ot ordcr q takcs thc toowin
torms
MA(q) y
t
e
t
+
1
e
t1
+ +
q
e
tq
,
whcrc e
t
is a vcctor ot whitc noiscs and e
t
N(0, ) Thc \Al(j)
modc may c jrcscntcd as an `A() modc
y
t

p

j=1

j
y
tj
+ e
t

k=0

k
e
tk
,
whcrc thc scqucncc {
k
} is assumcd to c asoutcy summac
To comjutc thc variancc ot a \Al jroccss, ct us rcwritc a \Al(j)
jroccss in thc torm ot \Al(1) jroccss as in (`3) Assumc that
vcctors and y arc covariancc-stationary, ct dcnotc thc variancc
ot Thcn, is dcncd as toows

_
_
_
_
_
_
_
_
_
_

0

1

p1

1

0

p2

p1

p2

0
_
_
_
_
_
_
_
_
_
_
FF

+ Q,
whcrc Q \ar(v
t
) \c can ajjy \cc ojcrator (lt A
dd
is symmct-
ric, \cc(A) dcnotcs thc d(d + 1)/2 coumn vcctor rcjrcscntin thc
stackcd uj coumns ot A which arc on and cow thc diaona ot A)
to oth sidcs ot thc aovc cquation,
\cc() \cc(FF

) + \cc(Q) (F F) \cc() + \cc(Q),

and with A F F,
\cc() (I
r
2 A)
1
\cc(Q)
jrovidcd that thc matrix I
r
2 A is nonsinuar, r np lt thc
jroccss
t
is covariancc-stationary, thcn I
r
2 A is nonsinuar
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 220
5.1.2 Statistical Inferences
Sujjosc wc havc a samjc ot sizc T , {y
t
}
T
t=1
, drawn trom an n-
dimcnsiona covariancc-stationary jroccss with E(y
t
) and E|(y
t

)(y
tj
)

|
j
As usuay, thc samjc mcan is dcncd
y
T

T

t=1
y
t
.
lt is casy to show that tor a variancc-covariancc stationary jroccss
E(y
T
) , and \ar(y
T
)
1
T
2
_

_T
0
+
T

j=1
(T j){
j
+
j
}
_

_ .
Proposition 4.2 Lct y
t
c a covariancc-stationary jroccss with thc
mcan and thc auto-covariancc
j
and with asoutcy summac
autocovarianccs Thcn thc samjc mcan y
T
satiscs y
T
convcrcs
to in jroaiity and T \ar(y
T
) convcrcs to

j=

j
S
A consistcnt cstimatc ot S can c constructcd ascd on thc `cwcy
and \csts (19S) LAC cstimator as toows (scc Scction 310)

0
+
q

j=1
_
_
1
j
q + 1
_
_
_

j
+

j
_
,
whcrc

j

1
T j
T

v=j+1
(y
t
y
T
)(y
tj
y
T
)

and onc can sct thc vauc ot q as toows q 0.` T

1/3

To cstimatc thc jaramctcrs in a \Al(j) modc, wc considcr thc

toowin \Al(j) modc ivcn in (`2) as
y
t
c +
1
y
t1
+ +
p
y
t1
+ e
t
,
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 221
whcrc y
t
is an n 1 vcctor containin thc vaucs that n variacs
assumc at datc t. e
t
N(0, ) \c assumc that wc havc oscrvcd
cach ot thcsc n variacs tor (T + p) timc jcriods, ic wc oscrvc
thc toowin samjc {y
p+1
, . . . , y
0
, y
1
, y
2
, . . . , y
T
} Thc simjcst
ajjroach to modc cstimation is to condition on thc rst p oscrva-
tions {y
p+1
, y
p+2
, . . . , y
0
} and to do cstimation usin thc ast T
oscrvations {y
1
, y
2
, . . . , y
T
} \c usc thc toowin notation
x
t

_
_
_
_
_
_
_
_
_
_
1
y
1

y
tp
_
_
_
_
_
_
_
_
_
_
,
_
_
_
_
_
_
_
_
_
_
c

p
_
_
_
_
_
_
_
_
_
_
,
whcrc x
t
is a (np + 1) 1 vcctor and is an n (np + 1) matrix
Thcn cquation (`2) can c writtcn as
y
t

x
t
+ e
t
,
whcrc y
t
is an n 1 vcctor ot thc variacs at jcriod t, e
t
is an
n 1 vcctor ot thc disturanccs and e
t
N(0, ) Thc ikcihood
tunction tor thc modc (`2) can c cacuatcd in thc samc way as tor
a univariatc autorcrcssion Thc o ikcihood tor thc cntirc samjc
is as toows
L(, ) C
T
2
n ||
1
2
T

t=1
(y
t

x
t
)

1
(y
t

x
t
),
whcrc C is a constant To nd thc `LL cstimatcs ot thc jaramctcrs
and , wc nd thc dcrivativcs ot thc o-ikcihood in thc aovc
with rcsjcct to and and sct thcm cqua to zcro lt can c shown
that thc `LL cstimatcs ot and arc as toows

(X

X)
1
X

Y, and



E

E

/T with

E Y X

.
Thc OLS cstimator ot is thc samc as thc unrcstrictcd `LL csti-
mator
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 222
To tcst thc hyjothcsis H
0

0
, wc can usc thc jojuar aj-
jroach ikcihood ratio tcst To do so, wc nccd to cacuatc thc
maximum vaucs

and

0
undcr H
0
and H
1
, rcsjcctivcy Thc Likc-
ihood latio tcst statistic is tound as

T
2 |L(

) L(

0
)|.
ndcr thc nu hyjothcsis,
T
asymjtoticay has a
2
distriution
with dcrccs ot trccdom cqua to thc numcr ot rcstrictions imjoscd
undcr H
0

To dcrivc thc asymjtotic jrojcrtics ot thc `LL, ct us dcnc

\cc(

), whcrc

is thc `LL ot `otc that sincc

is an npn
matrix,

is an n
2
p1 vcctor lt can c showcd in lrojosition 111 ot
Lamiton (199!) that (1) (1/T)

T
t=1
x
t
x

t
Qin jroaiity, whcrc
Q E(x
t
x

t
) is an npnp matrix. (2)

in jroaiity. (3)


in jroaiity. (!)

T (

) N
_
0, Q
1
_
Thcrctorc,

can c trcatcd as ajjroximatcy

N
_
, (X

X)
1
_
To tcst
thc hyjothcsis ot thc torm H
0
R r wc can usc thc toowin
torm ot thc \ad tcst

2
(m) (R

r)

_
R
_

(X

X)
1
_
R

_
1
(R

r).
5.2 Impulse-Response Function
lmjusc rcsjonscs tracc out rcsjonscs ot currcnt and tuturc vaucs
ot cach ot thc variacs to a onc-unit incrcasc in thc currcnt vauc ot
onc ot thc \Al crrors, assumin that this crror rcturns to zcro in
suscqucnt jcriod and that a othcr crrors arc cqua to zcro This
tunction is ot intcrcst tor scvcra rcasons lt is anothcr charactcri-
zation ot thc chavior ot modcs and it aows onc to think aout
causcs and cccts
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 223
lcca that tor an Al(1) jroccss, thc modc is x
t
x
t1
+e
t
or
x
t

j=0

j
e
tj
Lascd on thc `A() rcjrcscntation, wc scc trom
Scction 313 that thc imjusc-rcsjonsc tunction is
x
t+j
e
t

j
.
\cctor jroccss works thc samc way Thc covariancc-stationary \Al
modc can c writtcn in `A() torm as
y
t
+

j=0

j
e
tj
.
Thcn,
y
t+j
e

t

j
.
Thc ccmcnt
ij
ot
s
idcntics thc conscqucnccs ot a onc-unit in-
crcasc in thc j-th variacs innovation at datc t (e
jt
) tor thc vauc ot
thc i-th variac at timc t + s (y
i,t+s
), hodin a othcr innovations
at a datcs constant Onc may aso nd thc rcsjonsc ot a sjccic
variac to shocks in a othcr variacs or thc rcsjonsc ot a variacs
to a sjccic shock
y
i,t+s
e

t

(s)
.i
, and
y
t+s
e
jt

(s)
j.
.
lt onc is intcrcstcd in how thc variacs ot thc vcctor y
t+s
arc acctcd
it thc rst ccmcnt ot e
t
chancd y
1
at thc samc timc that thc
sccond ccmcnt chancd y
2
, and thc n-th ccmcnt y
n
, thcn thc
comincd ccct ot thcsc chancs on thc vauc ot y
t+s
is ivcn y
y
t+s

n

j=1
y
t+s
e

jt

j

s
.
A jot ot thc row i, coumn j ccmcnt ot
s
,
_

_
y
i,t+s
e
jt
_

_
S
s=0
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 224
as a tunction ot s is cacd thc orthogonal imjusc-rcsjonsc tunc-
tion lt dcscrics thc rcsjonsc ot y
i,t+s
to a onc-timc imjusc in y
jt
with a othcr variacs datcd t or caricr hcd constant
Sujjosc that thc datc t vauc ot thc rst variac in thc autorc-
rcssion, y
1t
, was hihcr than cxjcctcd Low docs this causc us
to rcvisc thc torccast ot y
i,t+s
To answcr this qucstion, wc dcnc
x

t1
(y

t1
, y

t2
, . . . , y

tp
), whcrc y
ti
is an n 1 vcctor and x
t1
is an np 1 vcctor Thc qucstion ccomcs, what is
E(y
i,t+s
|y
1t
, x
t1
)
y
1t

`otc that
E(y
i,t+s
|y
1t
, x
t1
)
y
1t

E(y
i,t+s
|y
1t
, x
t1
)
E(e

t
| y
1t
, x
t1
)

E(e
t
| y
1t
, x
t1
)
y
1t

(s)
.1
.
Lct us cxaminc thc torccast rcvision rcsutin trom ncw intormation
aout thc sccond variac, y
2t
, cyond that containcd in thc rst
variacs, y
1t
,
E(y
i,t+s
|y
1t
, y
2t
, x
t1
)
y
2t

E(y
i,t+s
|y
1t
, y
2t
, x
t1
)
E(e

t
| y
1t
, y
2t
, x
t1
)

E(e
t
| y
1t
, y
2t
, x
t1
)
y
2t

(
.2
Simiary wc miht nd thc torccast rcvision tor thc third variac
and so on Ior variac y
nt
,
E(y
i,t+s
|y
1t
, , y
nt
, x
t1
)
y
nt

E(y
i,t+s
|y
1t
, , y
nt
, x
t1
)
E(e

t
| y
1t
, , y
n2t
, x
t1
)

E(e
t
| y
1t
, , y
y
nt
Thc toowins arc thrcc imjortant jrojcrtics ot imjusc-rcsjonscs
Iirst, thc `A() rcjrcscntation is thc samc thin as thc imjusc-
rcsjonsc tunction. sccond, thc casicst way to cacuatc `A() rcj-
rcscntation is to simuatc thc imjusc rcsjonsc tunction. nay, thc
imjusc rcsjonsc tunction is thc samc as E
t
(y
t+j
) E
t1
(y
t+j
)
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 225
5.3 Variance Decompositions
ln thc oranizcd systcm, wc can comjutc an accountin ot torccast cr-
ror variancc what jcrccnt ot thc k stcj ahcad torccast crror variancc
is duc to which variac To do this, wc start with `A rcjrcscntation
y
t
(L) e
t
whcrc y
t
(x
t
, z
t
)

, e
t
(e
xt
, e
zt
)

, E(e
t
e

t
) I, and (L)

j=0

j
L
j
Thc onc stcj torccast crror is
e
t+1
y
t1+
E
t
(y
t+1
)
0
e
t
and its variancc is
\ar
t
(x
t+1
)
2
xx,0
+
2
xz,0
.

2
xx,0
ivcs thc amount ot onc-stcj ahcad torccast crror variancc ot
x duc to thc e
x
shock and
2
xz,0
ivcs thc amount duc to e
z
shock
ln jracticc, onc usuay rcjorts tractions
2
xx,0
/(
2
xx,0
+
2
xz,0
) `orc
tormay, wc can writc
\ar
t
(y
t+1
)
0

0
.
Lcnc
I
1

_
_
1 0
0 0
_
_
and I
2

_
_
0 0
0 1
_
_
.
Thcn, thc jart duc ot thc onc stcj ahcad torccast crror variancc duc
to thc rst shock x is
0
I
1

0
and thc jart duc to thc sccond shock
z is
0
I
2

0
Gcncraizin to k stcjs can c donc as toows
\ar
t
(y
t+k
)
k1

j=0

j
.
Thcn,
w
k,

k1

j=0

j
I

j
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 226
is thc variancc ot k stcj ahcad torccast crrors duc to thc -th shock
and thc variancc is sum ot thcsc comjoncnts, c \ar
t
(y
t+k
)

w
k,

5.4 Granger Causality

Thc rst thin that you carn in cconomctrics is a caution that
juttin x on thc riht hand sidc ot y x

+ e docs not mcan

that x causcs y Thcn you carn that causaity is not somcthin
you can tcst tor statisticay, ut must c known a jriori lt turns
out that thcrc is a imitcd scnsc in which wc can tcst whcthcr onc
variac causcs anothcr and vicc vcrsa
Grancr-causaity statistics cxaminc whcthcr acd vaucs ot onc
variac hcj to jrcdict anothcr variac Thc variac y tais to
Grancr-causc thc variac x it tor a s > 0 thc `SL ot a torccast ot
x
t+s
ascd on (x
t
, x
t1
, . . .) is thc samc as thc `SL ot a torccast ot
x
t+s
that uscs oth (x
t
, x
t1
, . . .) and (y
t
, y
t1
, . . .) lt onc considcrs
ony incar tunctions, y tais to Grancr-causc x it
MSE|

E(x
t+s
| x
t
, x
t1
, . . .)| MSE|

E(x
t+s
| x
t
, x
t1
, . . . , y
t
, y
t1
, . . .)|.
Lquivacnty, wc say that x is cxocnous in thc timc scrics scnsc with
rcsjcct to y it thc aovc hods Or, y is not incar intormativc aout
tuturc x
ln a ivariatc \Al modc dcscriin x and y, x docs not Grancr-
causc y it thc coccicnts matriccs
j
arc owcr trianuar tor a j,
ic
_
_
x
t
y
t
_
_

_
_
c
1
c
2
_
_
+
_
_
_

(1)
11
0

(1)
21

(1)
22
_
_
_
_
_
x
t1
y
t1
_
_
+ +
_
_
_

(p)
11
0

(p)
21

(p)
22
_
_
_
_
_
x
tp
y
tp
_
_
+
_
_
e
1t
e
2t
_
_
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 227
Thc Grancr-causaity can c tcstcd y conductin F-tcst ot thc nu
hyjothcsis
H
0

(1)
21

(2)
21

(p)
21
0.
Thc rst and most tamous ajjication ot Grancr causaity was thc
qucstion ot whcthcr moncy rowth causcs chancs in G`l Iricd-
man and Schwartz (19o3) documcntcd a corrcation ctwccn moncy
rowth and G`l Lut Toin (190) arucd that a jhasc cad and
a corrcation may not indicatc causaity Sims (19S0) answcrcd this
criticism and had shown that moncy Grancr causcs G`l and not
vicc vcrsa (hc tound dicrcnt rcsuts atcr) Sims (19S0) anayzcd thc
toowin rcrcssion to study ccct ot moncy on G`l
y
t

j=0
b
j
m
tj
+ u
t
.
This rcrcssion is known as a St Louis Icd cquation Thc coc-
cicnts wcrc intcrjrctcd as thc rcsjonsc ot y to chancs in m, ic it
thc Icd scts m, {b
j
} ivcs thc rcsjonsc ot y Sincc thc coccicnts
wcrc rcativcy i, it imjicd that constant moncy rowth rucs wcrc
dcsirac
Thc ovious o,cction to this statcmcnt is that coccicnts may
rccct rcvcrsc causaity thc Icd scts moncy in anticijation ot su-
scqucnt cconomic rowth, or thc Icd scts moncy in rcsjonsc to jast
y This mcans that thc crror tcrm u is corrcatcd with currcnt and
acd m so OLS cstimatcs ot thc jaramctcrs b arc inconsistcnt duc
to thc cndocncity jrocm \hy is Grancr causaity not causa-
ity Grancr causaity is not causaity ccausc ot thc jossic ccct
ot othcr variacs lt x cads to y with onc a ut to z with two as,
thcn y wi Grancr causc z in ivariatc systcm Thc rcason is that
y wi hcj torccast z ccausc it rcvcas intormation aout thc truc
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 228
Table 5.1: Sims variance decomposition in three variable VAR model
Explained by shocks to
Var. of M1 IP WPI
M1 97 2 1
IP 37 44 18
WPI 14 7 80
Table 5.2: Sims variance decomposition including interest rates
Explained by shocks to
Var. of R M1 IP WPI
R 50 19 4 28
M1 56 42 1 1
IP 2 32 60 6
WPI 30 4 14 52
causc x Lut it docs not toow that it you chanc y thcn a chanc
in z wi toow
This woud not c a jrocm it thc cstimatcd jattcrn ot causaity
in macrocconomic timc scrics was stac ovcr thc incusion ot scvcra
variacs An cxamjc y Sims (19S0) iustratcd that is not thc casc
vcry ottcn Sims (19S0) cstimatcd thrcc variac \Al with moncy,
industria jroduction and whoc sac jricc indcx and tour variac
\Al with intcrcst ratc, moncy, industria jroduction and whocsac
jricc indcx Thc rcsuts arc in Tac `1 Thc rst row in Tac
`1 vcrics that M1 is cxocnous ccausc it docs not rcsjond to
othcr variacs shocks Thc sccond row shows that M1 causcs
chancs in IP, sincc 3/ ot thc !S month ahcad variancc ot IP is
duc to M1 shocks Thc third row is juzzin ccausc it shows that
WPI is cxocnous Tac `2 shows what hajjcns whcn on morc
variac, intcrcst ratc, is addcd to thc modc Thc sccond row shows
sustantia rcsjonsc ot M1 to intcrcst ratc shocks ln this modc M1
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 229
is not cxocnous ln thc third row onc can scc that M1 docs inucncc
IP. thc tourth row shows that M1 docs not inucncc WPI, intcrcst
ratc docs
5.5 Forecasting
To do torccastin, wc nccd to do thc toowin thins Iirst, choosc
thc a cnth ot \Al usin cithcr onc ot thc intormation critcria
(AlC, SlC, AlCC) or onc ot thc torccastin critcria. sccond, cstimatc
thc \Al modc usin thc OLS and otain thc jaramctcr cstimatcs

j
. and nay, thc h-jcriod-ahcad torccasts arc constructcd rccur-
sivcy

y
t
t+1

1
y
t
+

2
y
t1
+ +

p
y
tp+1

y
t
t+2

1

y
t
t+1
+

2
y
t
+ +

p
y
tp+2

y
t
t+3

1

y
t
t+2
+

2

y
t
t+1
+ +

p
y
tp+3
and so on
Low wc do \Als jcrtorm thc tasks Lccausc \Al invovcs
currcnt and acd vaucs ot mutijc timc scrics, thcy cajturc co-
movcmcnts that can not c dctcctcd in univariatc modcs \Al
summary statistics ikc Grancr-causaity tcsts, imjusc rcsjonsc
tunctions and variancc dccomjositions arc wc-acccjtcd mcthods tor
jortrayin co-movcmcnts Sma \Als havc ccomc a cnchmark
aainst which ncw torccastin systcms arc ,udcd Thc jrocm is
that sma \Als arc ottcn unstac and thus joor jrcdictors ot thc
tuturc
5.6 Problems
1. \ritc a rctcrcc rcjort tor thc jajcr y Sims (1992)
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 230
2. sc quartcry data tor rca GLl, GLl dcator, Cll, thc Icdcra
Iunds ratc, `oncy asc mcasurc, M1, and thc indcx ot commod-
ity jriccs tor thc jcriod 19`9l-2002lll (c TEps8data.xls)
Aso, usc thc monthy data tor thc Tota lcscrvcs (c TotRe-
sAdjRR.xls) and `on-Lorrowcd lcscrvcs (c BOGNONBR.xls)
tor thc jcriod 19`91-20029 Transtorm monthy data into quar-
tcry data \c cxaminc thc \Al modc, invcstiatcd y Chris-
tiano, Lichcnaum and Lvans (2000) Thc modc has scvcn
variacs thc o ot rca GLl (Y ), o ot consumcr jricc in-
dcx (CPI), chanc in thc indcx ot scnsitivc commodity jriccs
(Pcom), Icdcra tunds ratc(FF), o ot tota rcscrvcs (TR),
o ot non-orrowcd rcscrvcs (NBR), and o ot M1 monctary
arcatc (M1) A monctary joicy shock in thc modc is rcj-
rcscntcd y a shock to thc Icdcra tunds ratc thc intormation
sct consists ot currcnt and acd vaucs ot Y , CPI and Pcom,
and ony acd vaucs ot FF, TR, NBR and M0 lt imjics
thc toowin ordcrin ot thc variacs in thc modc
x

t
(Y, CPI, Pcom, FF, TR, NBR, M1).
Thc rctcrcncc tor this jajcr is Christiano, Lichcnaum and Lvans
(2000)
(a) Construct rajhs ot a timc scrics data Commcnt your rcsuts
() Lstimatc a \Al modc tor x
t
(i) `iccy jrcscnt thc imjusc rc-
sjonsc tunctions rcjrcscntin thc rcsjonsc ot a variacs in thc
modc to a monctary joicy shock (FF ratc) Carctuy cxjain
your rcsuts (ii) `iccy jrcscnt thc variancc-dccomjosition rc-
suts tor a variacs tor thc torccast horizons k 2, !, 12, 3o
Carctuy cxjain your rcsuts (iii) Conduct Grancr Causa-
ity tcsts tor a variacs in thc modc Carctuy cxjain your
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 231
rcsuts
(c) `ost macro cconomists arcc that thcrc was a shitt ot monctary
joicy toward ination durin thc atc 190s trom accommodat-
in to arcssivc Lstimatc modc tor two jcriods 19`9l-199ll
and 199lll -2002lll `iccy jrcscnt thc imjusc rcsjonsc tunc-
tions rcjrcscntin thc rcsjonsc ot a variacs in thc modc to
a monctary joicy shock (FF ratc) tor oth jcriods Carctuy
cxjain your rcsuts Low do imjusc rcsjonsc tunction rcvca
thc chanc in thc monctary joicy
(d) sc thc \Al modc to construct S-jcriod-ahcad torccasts tor a
thc variacs in thc modc
3. Somc macro cconomists ookcd at thc NBR and NBR/TR
sjccications ot monctary joicy shocks ln thc casc ot a NBR
monctary joicy shock, thc intormation sct is idcntica to a FF
shock, whic in thc casc ot a NBR/TR shock thc intormation sct
incudcs aso thc currcnt vauc ot tota rcscrvcs sc a rccursivc
schcmc tor idcntication and cxaminc thc ccct ot a monctary
joicy shock tor a NBR sjccication (think how you shoud
rcordcr thc variacs in modc)
(a) `iccy jrcscnt thc imjusc rcsjonsc tunctions rcjrcscntin thc
rcsjonsc ot a variacs in thc modc to a monctary joicy shock
(thc cvc ot NBR) Lxjain your rcsuts
() `iccy jrcscnt thc variancc-dccomjosition rcsuts tor a variacs
showin thc contriution ot NBR shock ony tor thc torccast
horizons k 2, !, 12, 3o Lxjain your rcsuts
4. \ritc a jroram tor imjcmcntin IA\Al ajjroach ot Lcrnankc
ct a (200`)
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 232
(a) lun thc jroram and cxjain thc main stcjs in thc cstimation
() sin thc cstimation rcsuts, carctuy cxjain thc ccct ot mon-
ctary joicy on 90-day T-i ratc, 1-ycar T-ond intcrcst ratc,
`-ycar T-ond intcrcst ratc and 10-ycar T-ond intcrcst ratc
(c) Carctuy cxjain your ndin on thc ccct ot monctary joicy on
cmjoymcnt You nccd to usc thc imjusc rcsjonsc tunctions tor
cmjoymcnt scrics, uncmjoymcnt scrics, avcrac hours workcd,
and ncw caims tor uncmjoymcnt
(d) Lxjain thc ccct ot a shock to thc Icdcra Iunds ratc on dicr-
cnt arcatc mcasurcs ot moncy sujjy
(c) Lxjain thc ccct ot a monctary joicy shock on cxchanc ratc
and rca stock jriccs
(t) Lxjain thc ccct ot a monctary joicy shock on dicrcnt mca-
surcs ot GLl (rca GLl, dicrcnt mcasurcs ot lndustria lro-
duction, ctc)
() \hat hajjcns to thc rcsuts it thc numcr ot Liusion lndcxcs
(scc Stock and \atson (2002)) is incrcascd trom thrcc to vc
5. Assumc that you arc taccd with a task ot torccastin dicrcnt
intcrcst ratcs Lxjain how you may ajjy thc Liusion lndcxcs
ajjroach ot Stock and \atson (2002) to usc as many variacs
as jossic in thc torccastin
6. \ritc a rctcrcc rcjort tor thc jajcr y Lachmcicr, Lccahanon
and Li (200`) lcasc think aout any jossic tuturc jro,ccts
CHAPTER 5. VECTOR AUTOREGRESSIVE MODELS 233
5.7 References
Bachmeier, L., S. Leelahanon and Q. Li (2005). Money growth and ination in the United
States. Working Paper, Department of Economics, Texas A&M University.
Bernanke, B.S., J. Boivin and E. Piotr (2005). Measuring the eects of monetary policy: a
factor-augmented vector autoregressive (FAVAR) approach. The Quarterly Journal of
Economics, 120, 387-422.
Christiano, L.J., M. Eichenbaum and C.L. Evans (2000). Monetary policy shocks: what
have we learned and to what end? Handbook of Macroeconomics, Vol. 1A.
Cochrane, J.H. (1994). Shocks. NBER working paper #46984.
Cochrane, J.H. (1997). Time series for macroeconomics and nance. Lecture Notes.
http://www-gsb.uchicago.edu/fac/john.cochrane/research/Papers/timeser1.pdf
Friedman, M. and A.J. Schwartz (1963). A Monetary History of the United States, 1867-
1960. Princeton University Press.
Hamilton, J.D. (1994). Time Series Analysis. Princeton University Press.
Hendry, F.D. and K. Juselius (2000). Explaining cointegration analysis: Part I. Journal of
Energy, 21, 1-42.
Newey, W.K. and K.D. West (1987). A simple, positive-denite, heteroskedasticity and
autocorrelation consistent covariance matrix. Econometrica, 55, 703-708.
Sims, C. (1980). Macroeconomics and reality. Econometrica, 48, 1-48.
Sims, C.A. (1992). Interpreting the macroeconomic time series facts: the eects of monetary
policy. European Economic Review, 36, 975-1000.
Stock, J.H. and M.W. Watson (2001). Vector autoregressions. Journal of Economic Per-
spectives, 15, 101-115.
Stock, J.H. and M.W. Watson (2002). Macroeconomic forecasting using diusion indexes.
Journal of Business and Economic Statistics, 20, 147-162.
Stock, J.H. and M.W. Watson (2003). Introduction to Econometrics. Addison-Wesley.
Tobin, J. (1970). Money and income. Quarterly Journal of Economics.
Chapter 6
Cointegration
6.1 Introduction
Cointcration is a cncraization ot unit root to vcctor modcs as
a sinc scrics can not c cointcratcd Cointcration anaysis is
dcsincd to nd incar cominations ot variacs to rcmovc unit roots
Sujjosc that two scrics arc cach intcratcd with thc toowin `A
rcjrcscntation
(1 L) y
t
a(L) u
t
, and (1 L) x
t
b(L) v
t
.
ln cncra, incar cominations ot y
t
and x
t
wi aso havc a unit
roots Lut it thcrc is somc incar comination, y
t
x
t
, that is
stationary, y
t
and x
t
arc said to c cointcratcd and (1, )

is
thcir cointcratin vcctor Cointcration vcctors arc ot considcrac
intcrcst whcn thcy cxist, sincc thcy dctcrminc I(0) rcations that
hod ctwccn variacs that arc individuay non-stationary
As an cxamjc, wc may ook at thc rca G`l and consumjtion
Lach ot thcsc scrics jroay has a unit root ut thc ratio ot consumj-
tion to rca G`l is stac ovcr thc on jcriods ot timc Thcrctorc,
o consumjtion minus o G`l is stationary, and o G`l and
consumjtion arc cointcratcd Othcr jossic cxamjcs incudc thc
dividcnd,jricc ratio or moncy and jriccs Lowcvcr, cointcration
234
CHAPTER 6. COINTEGRATION 235
docs not say anythin aout thc dircction ot causaity
6.2 Cointegrating Regression
Cointcratin vcctors arc sujcr-consistcnt which mcans that you
can cstimatc thcm y OLS cvcn whcn thc riht hand sidc variacs
arc corrcatcd with thc crror tcrm, and thc cstimatcs convcrc at a
tastcr ratc than usua OLS cstimatcs Sujjosc y
t
and x
t
arc cointc-
ratcd so that y
t
x
t
is stationary Lstimatc thc toowin modc
usin OLS rcrcssion
y
t
x
t
+ e
t
. (o1)
OLS cstimatcs ot convcrc to , cvcn it thc crrors arc corrcatcd
with x
t
`otc that it y
t
and x
t
arc cach individua I(1) ut arc not
cointcratcd thcn thc rcrcssion in cquation (o1) rcsuts in sjuri-
ous rcrcssion Thcrctorc, you havc to chcck whcthcr thc cstimatcd
rcsiduas

e
t
arc I(1) or I(0) \c wi discuss it atcr in thc notcs
Representation of Cointegrating System
Lct y
t
c a rst dicrcncc stationary vcctor timc scrics Thc ccmcnts
ot y
t
arc cointcratcd it thcrc is at cast onc vcctor , cointcratin
vcctor, such that

y
t
is stationary in cvcs Sincc thc dicrcncc ot
y
t
is stationary, it has a movin avcrac rcjrcscntation
(1 L) y
t
A(L) e
t
.
Sincc thc stationarity ot

y
t
is an cxtra rcstriction, it must imjy a
rcstriction on A(L)
Simiar to thc univariatc Lcvcridc-`cson (19S1) dccomjosition,
thc mutivariatc Lcvcridc-`cson dccomjosition can c donc in thc
CHAPTER 6. COINTEGRATION 236
samc way as
y
t
z
t
+ c
t
whcrc (1 L) z
t
A(1) e
t
and c
t
A

(L) e
t
with A

k=j+1
A
k

Thc rcstriction on A(1) imjicd y cointcration thc ccmcnts ot

y
t
arc cointcratcd with cointcratin vcctors it and ony it (i)

A(1) 0 This imjics that thc rank ot A(1) is thc numcr ot

ccmcnts ot y
t
minutcs numcr ot cointcratin vcctors Thcrc arc
thrcc cascs tor A(1) Iirst, A(1) 0 i y
t
is stationary in cvcs
and a incar cominations ot y
t
arc stationary in cvcs Sccond,
A(1) is not tu rank i (1 L) y
t
is stationary and somc incar
cominations

y
t
arc stationary Iinay, A(1) has tu rank i (1
L) y
t
is stationary and no incar cominations ot y
t
arc stationary
Impulse response function
A(1) is thc imitin imjusc-rcsjonsc ot thc cvcs ot thc vcctor y
t

(x
t
, z
t
)

To scc how cointcration accts A(1), considcr a simjc

casc, (1, 1)

Thc rcduccd rank ot A(1) mcans

A(1) 0, or (1 1)
_
_
A(1)
xx
A(1)
zx
A(1)
xz
A(1)
zz
_
_
0.
Thcrctorc, A(1)
xx
A(1)
zx
and A(1)
xz
A(1)
zz
so that cach vari-
acs on-run rcsjonsc to a shock must c thc samc
6.3 Testing for Cointegration
Thcrc arc scvcra ways to dccidc whcthcr variacs can c modccd as
cointcratcd Iirst, usc cxjcrt knowcdc and cconomic thcory Scc-
ond, rajh thc scrics and scc whcthcr thcy ajjcar to havc a common
stochastic trcnd Iinay, jcrtorm statistica tcsts tor cointcration
CHAPTER 6. COINTEGRATION 237
A thrcc mcthods shoud c uscd in jracticc \c wi considcr rcsid-
ua ascd statistica tcst tor cointcration
Testing for cointegration when cointegrating vector is known
Somctimcs, a rcscarchcr may know thc cointcratin vcctor ascd
on thc cconomic thcory Ior cxamjc, thc hyjothcsis ot jurchasin
jowcr jarity imjics that
P
t
S
t
P

t
,
whcrc P
t
is an indcx ot thc jricc cvc in thc S, S
t
is thc cxchanc
ratc (S,Chincsc Yuan), P

t
is a jricc indcx tor China Takin os,
this cquation can c writtcn as
p
t
s
t
+ p

t
A wcakcr vcrsion ot thc hyjothcsis is that thc variac v
t
dcncd y
v
t
p
t
s
t
p

t
(1 1 1)
_
_
_
_
_
_
p
t
s
t
p

t
_
_
_
_
_
_
is stationary, cvcn thouh thc individua ccmcnts (p
t
, s
t
, p

t
) arc
a I(1) ln this casc thc cointcratin vcctor is known to c
(1, 1, 1) Tcstin tor cointcration in this casc consists ot scvcra
stcjs
1. \crity that p
t
, s
t
, p

t
arc cach individuay I(1) This wi
c truc it (a) You tcst tor unit root in cvcs ot thcsc scrics
and can no rc,cct thc nu hyjothcsis ot unit root (ALI or
othcr tcsts tor unit root) () You tcst tor unit root in rst
dicrcnccs ot thcsc scrics and rc,cct thc nu hyjothcsis ot
unit root (ALI or othcr tcsts tor unit root)
2. Tcst that a scrics v
t
is stationary or not
CHAPTER 6. COINTEGRATION 238
Table 6.1: Critical values for the Engle-Granger ADF statistic
Number of X

s in equation (6.1) 10% 5% 1%

1 -3.12 -3.41 -3.96
2 -3.52 -3.80 -4.36
3 -3.84 -4.16 -4.73
4 -4.20 -4.49 -5.07
Testing for cointegration when cointegrating vector is unknown
Considcr an cxamjc in which two scrics y
t
and x
t
arc cointcratcd
with cointcratcd vcctor (1, )

so that v
t
y
t
x
t
is
stationary Lowcvcr, thc cointcratin coccicnt is not known To
cstimatc , wc can usc thc Lnc-Grancr Aumcntcd Lickcy-Iucr
(LG-ALI) tcst tor cointcration which consists ot two stcjs
1. \crity that y
t
and x
t
arc cach individuay I(1)
2. Lstimatc thc cointcratin coccicnt usin OLS cstima-
tion ot thc rcrcssion y
t
+ x
t
+ v
t

3. A Lickcy-Iucr t-tcst (with intcrccjt no timc trcnd) is

uscd to tcst tor a unit root in thc rcsidua trom this rcrcs-
sion,

v
t
Sincc wc cstimatc rcsiduas in thc rst stcj, wc
nccd to usc dicrcnt critica vaucs tor thc unit root tcst
Critica vaucs tor LG-ALI statistic arc ivcn in Tac o1
This tac is takcn trom Stock and \atson (2002)
lt x
t
and y
t
arc cointcratcd, thcn thc OLS cstimator ot thc co-
ccicnt in thc rcrcssion is sujcr-consistcnt Thcrctorc, thc OLS
cstimator has a non-norma distriution, and intcrcnccs ascd on its
t-statistic can c miscadin To avoid this jrocm, Stock and \at-
son (1993) dcvcojcd dynamic OLS (LOLS) cstimator ot trom thc
CHAPTER 6. COINTEGRATION 239
toowin rcrcssion
y
t
+ x
t
+
p

j=p

j
x
tj
+ u
t
.
lt x
t
and y
t
arc cointcratcd, statistica intcrcnccs aout and

s
ascd on LAC standard crrors arc vaid
lt x
t
wcrc stricty cxocnous, thcn thc coccicnt on x
t
, , woud
c thc on-run cumuativc mutijicr, that is, thc on-run ccct
on y ot a chanc in x Scc thc on-run mutijicr ctwccn oi and
asoinc jriccs in thc jajcr y Lorcnstcin et al. (199)
6.4 Cointegrated VAR Models
\c start with thc autorcrcssivc rcjrcscntation ot cvcs ot y
t
, B(L) y
t

e
t

y
t
B
1
y
t1
B
2
y
t2
+ e
t
Ajjyin L` dccomjosition B(L) B(1)+(1L)B

(L), wc otain
|B(1) + (1 L)B

(L)| y
t
B(1) y
t
+ B

(L) y
t
e
t
so that
y
t
|B
1
+ B
2
+ | y
t1

j=1
B

j
y
tj
+ e
t
.
Sutractin y
t1
trom oth sidcs, wc ct
y
t
B(1) y
t1

j=1
B
j
y
tj
+ e
t
.
Thc matrix B(1) contros thc cointcration jrojcrtics
1. lt B(1) is tu rank, any incar comination ot y
t
is stationary
and y
t
is stationary ln this casc, wc run a norma \Al
CHAPTER 6. COINTEGRATION 240
2. lt B(1) has rank ctwccn 0 and tu rank Thcrc arc somc
incar cominations ot y
t
that arc stationary, so y
t
is sta-
tionary Thc \Al in cvcs is consistcnt ut inccicnt it
you know thc cointcratin vcctor and thc \Al in dicr-
cnccs is missjccicd in this casc
3. lt B(1) has rank zcro, so no incar comination ot y
t
is sta-
tionary and y
t
is stationary with no cointcration ln this
casc wc run a norma \Al in dicrcnccs
Error Correction Representation
lt B(1) has css than tu rank, wc can cxjrcss it as
B(1)

.
lt thcrc arc K cointcratin vcctors, thcn thc rank ot B(1) is K and
and cach havc K coumns Thcn thc systcm can c rcwrittcn as
y
t

y
t1

j=1
B

j
y
tj
+ e
t
,
whcrc

is a K N matrix ot cointcratin vcctors Thc aovc

cxjrcssion is thc wc known crror-corrcction modc (LC`) rcjrc-
scntation ot thc intcratcd systcm lt is not casy to cstimatc this
modc whcn a cointcratcd vcctors in arc unknown
Considcr a mutivariatc modc consistin ot two variacs x
t
and
w
t
which arc individuay I(1) Onc may modc thcsc two variacs
usin onc ot thc toowin modcs
1. A \Al modc in cvcs
2. A \Al in thc rst dicrcnccs
CHAPTER 6. COINTEGRATION 241
3. An LC` rcjrcscntation
\ith cointcration, a jurc \Al in dicrcnccs is missjccicd
x
t
a(L) x
t1
+ b(L) z
t1
+ e
t
z
t
c(L) x
t1
+ d(L) z
t1
+ v
t
Lookin at thc crror-corrcction torm, thcrc is a missin rcrcssor,

x
x
t1
+
z
z
t1
This is a jrocm A jurc \Al in cvcs is a ittc
it unconvcntiona sincc thc variacs in thc modc arc nonstationary
Thc \Al in cvcs is not missjccicd and thc cstimatcs arc consistcnt
ut thc coccicnts may havc non-standard distriutions and thcy arc
not ccicnt lt thcrc is cointcration, it imjoscs rcstrictions on B(1)
that arc not imjoscd in a jurc \Al in cvcs Cochranc (199!)
sucstcd that onc way to imjosc cointcration is to run an crror-
corrcction \Al
x
t

x
(
x
x
t1
+
z
z
t1
) + a(L) x
t1
+ b(L) z
t1
+ e
t
z
t

w
(
x
x
t1
+
z
z
t1
) + c(L) x
t1
+ d(L) z
t1
+ v
t
This sjccication imjoscs that x and z arc cointcratcd with coin-
tcratin vcctor This is vcry usctu it you know that thc variacs
arc cointcratcd and you know thc cointcratin vcctor Othcrwisc,
you havc to jrc-tcst tor cointcration and cstimatc thc cointcratin
vcctor in a scjaratc stcj
Anothcr dicuty with thc crror-corrcction torm is that it docs
not t niccy into standard \Al jackacs A way to usc standard
jackacs is to cstimatc comjanion torm
x
t
a(L) x
t1
+ b(L) (
x
x
t1
+
z
z
t1
) + e
t

x
x
t
+
z
z
t
c(L) x
t1
+ d(L)(
x
x
t1
+
z
z
t1
) + v
t
\c nccd to know cointcratin vcctor to usc this jroccdurc Thcrc
is much dcatc as to which ajjroach is cst \hcn you do not
CHAPTER 6. COINTEGRATION 242
rcay know whcthcr thcrc is cointcration or what thc cointcratin
vcctor is, thc \Al in cvcs sccms to c cttcr \hcn you know
that thcrc is cointcration and what thc cointcratin vcctor is, thc
crror-corrcction torm modc or \Al in comjanion torm is cttcr
Somc ot unit root and cointcration tcsts arc jrovidcd y thc jack-
acs tseries, urca and uroot in R Ior cxamjc, in tseries, thc
tunction po.test is tor thc lhiij-Ouiariss (1990) tcst tor tcstin
that x is not cointcratcd Thcrc arc many othcr tcst mcthods avai-
ac in thc jackacs urca and uroot. tor dctais, scc thcir manuas
6.5 Problems
1. Thc intcrcst ratcs may c tound in thc Lxcc c IntRates.xls
Chcck whcthcr 90-day T-i ratc and 10-ycar T-ond intcrcst
ratc arc cointcratcd Carctuy cxjain how you conduct thc
tcst and how you intcrjrct your ndins
2. Lownoad data (y yoursct) trom thc wc sitc to havc data on
consumcr jricc indcx (Cll), jroduccr jricc indcx (lll), thrcc-
month T-i ratc, thc indcx ot industria jroduction, S:l `00
common stock jricc indcx Thc data tor industria jroduction
is scasonay ad,ustcd whic a othcr variacs arc not scasonay
ad,ustcd Conduct a tcst ot cointcration ctwccn thc variacs
Lxjain your rcsuts
3. Cocct thc toowin macrocconomic annua timc scrics ot China
trom China Statistical Yearbook G`l, GLl, GLl1 (GLl ot
jrimary industry), GLl2 (GLl ot sccondary industry), GLl3
(GLl ot tcrtiary industry), and jcr cajita GLl Irom oth thc
nomina and rca tcrms ot thc dcnitions ot nationa jroducts,
dcrivc thc corrcsjondin jricc dcators
CHAPTER 6. COINTEGRATION 243
(a) Lcnc and tcst thc unit roots tor cach ot thc 1S cconomic vari-
acs (nomina, rca, and dcator ot GLls) assumin no struc-
tura rcak in thc scrics
() Lcnc and tcst thc unit roots tor cach ot thc 1S cconomic vari-
acs (nomina, rca, and dcator ot GLls) assumin onc-timc
structura rcak in thc scrics
(c) Conduct cointcration tcsts tor somc ot thc 1S cconomic vari-
acs Lxjain your rcsuts
6.6 References
Bernanke, B.S., J. Boivin and E. Piotr (2005). Measuring the eects of monetary policy: a
factor-augmented vector autoregressive (FAVAR) approach. The Quarterly Journal of
Economics, 120, 387-422.
Borenstein, S., A.C. Cameron and R. Gilbert (1997). Do gasoline prices respond asymmet-
rically to crude oil price changes? The Quarterly Journal of Economics, 112, 305-339.
Christiano, L.J., M. Eichenbaum and C.L. Evans (2000). Monetary policy shocks: what
have we learned and to what end? Handbook of Macroeconomics, Vol. 1A.
Cochrane, J.H. (1994). Shocks. NBER working paper #46984.
Cochrane, J.H. (1997). Time series for macroeconomics and nance. Lecture Notes.
Engle, R.F. and C.W.J. Granger (1987). Cointegration and error correction: Representa-
tion, estimation and testing. Econometrica, 55, 251-276.
Hamilton, J.D. (1994). Time Series Analysis. Princeton University Press.
Hendry, F.D. and K. Juselius (2000). Explaining cointegration analysis: Part I. Journal of
Energy, 21, 1-42.
Phillip, P.C.B. and S. Ouliaris (1990). Asymptotic properties of residual based on tests for
cointegration. Econometrica, 58, 165-193.
Stock, J.H. and M.W. Watson (1993). A simple estimator of cointegrating vectors in higher
order integrated systems. Econometrica, 61, 1097-1107.
Stock, J.H. and M.W. Watson (2003). Introduction to Econometrics. Addison-Wesley.
Chapter 7
Nonparametric Density, Distribution
& Quantile Estimation
7.1 Mixing Conditions
lt is wc known that -mixin incudcs many timc scrics modcs as
a sjccia casc ln tact, undcr vcry mid assumjtions incar autorc-
rcssivc and morc cncray iincar timc scrics modcs arc -mixin
with mixin coccicnts dccayin cxjoncntiay `any nonincar timc
scrics modcs, such as tunctiona coccicnt autorcrcssivc jroccsscs
with,without cxocnous variacs, AlCL and GAlCL tyjc jro-
ccsscs, stochastic voatiity modcs, and nonincar additivc autorc-
rcssivc modcs with,without cxocnous variacs, arc stron mixin
undcr somc mid conditions Scc Cai (2002) and Chcn and Tan
(200`) tor morc dctais
To simjity thc notation, wc ony introducc mixin conditions tor
stricty stationary jroccsscs (in sjitc ot thc tact that a mixin jroccss
is not ncccssariy stationary) Thc idca is to dcnc mixin coccicnts
to mcasurc thc strcnth (in dicrcnt ways) ot dcjcndcncc tor thc two
scmcnts ot a timc scrics which arc ajart trom cach othcr in timc
244
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION245
Lct {X
t
} c a stricty stationary timc scrics Ior n 1, dcnc
(n) suj
AF
0

;BF

n
|P(A)P(B) P(AB)|,
whcrc F
j
i
dcnotcs thc -acra cncratcd y {X
t
. i t j} `otc
that F

n
lt (n) 0 as n , {X
t
} is cacd -mixin or
stron mixin Thcrc arc scvcra othcr mixin conditions such as
-mixin, -mixin, -mixin, and -mixin. scc thc ooks y La
and Lcydc (19S0) and Ian and Yao (2003, jac oS) lt is wc known
that thc rcationshijs amon thc mixin conditions arc -mixin
-mixin -mixin and -mixin -mixin
Lemma 1: (Lavydovs incquaity) (i) lt E|X
i
|
p
+E|X
j
|
q
< tor
somc p 1 and q 1 and 1/p + 1/q < 1, it hods that
|Cov(X
i
, X
j
)| S
1/r
(|j i|){E|X
i
|
p
}
1/p
{E|X
j
|
q
}
1/q
,
whcrc r (1 1/p 1/q)
1

(ii) lt P(|X
i
| C
1
) 1 and P(|X
j
| C
2
) 1 tor somc constants
C
1
and C
2
, it hods that
|Cov(X
i
, X
j
)| ! (|j i|) C
1
C
2
.
`otc that it wc aow X
i
and X
j
to c comjcx-vaucd random vari-
acs, (ii) sti hods with thc coccicnt ! on thc lLS ot thc in-
cquaity rcjaccd y 1o
7.2 Density Estimate
Lct {X
i
} c a random samjc with a (unknown) marina distri-
ution F() (CLI) and its jroaiity dcnsity tunction (lLI) f()
Thc qucstion is how to cstimatc f() and F() Sincc
F(x) P(X
i
x) E|I(X
i
x)|
_
x

f(u)du,
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION246
and
f(x) im
h0
F(x + h) F(x h)
2 h

F(x + h) F(x h)
2 h
it h is vcry sma, y thc mcthod ot momcnt cstimation (``L), F(x)
can c cstimatcd y
F
n
(x)
1
n
n

i=1
I(X
i
x),
which is cacd thc cmjirica cumuativc distriution tunction (ccdt),
so that f(x) can c cstimatcd y
f
n
(x)
F
n
(x + h) F
n
(x h)
2 h

1
n
n

i=1
K
h
(X
i
x),
whcrc K(u) I(|u| 1)/2 and K
h
(u) K(u/h)/h lndccd, thc
kcrnc tunction K(u) can c takcn to c any symmetric dcnsity
tunction
Exercise: lcasc show that F
n
(x) is uniascd cstimatc ot F(x) ut
f
n
(x) is iascd cstimatc ot f(x) Think about intuitively
(1) why f
n
(x) is biased
(2) where the bias comes from
(3) why K() should be symmetric.
7.2.1 Asymptotic Properties
Lct us ook at thc variancc ot cstimators lt {X
i
} is stationary, thcn
n\ar(F
n
(x)) \ar(I(X
i
x)) + 2
n

i=2
_
_
1
i 1
n
_
_
Cov(I(X
1
x), I(X
i

F(x)|1 F(x)| + 2
n

i=2
Cov(I(X
1
x), I(X
i
x))
. .

2
(x) y assumin that
2
(x)<
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION247
2
n

i=2
i 1
n
Cov(I(X
1
x), I(X
i
x))
. .
0 y Ironcckcr Lcmma

2
F
(x) F(x)|1 F(x)| + 2

i=2
Cov(I(X
1
x), I(X
i
x))
.
This tcrm is cacd A
d
Thcrctorc,
n\ar(F
n
(x))
2
F
(x). (1)
lt is ccar that A
d
0 it {X
i
} arc indcjcndcnt lt A
d
0, thc
qucstion is how to cstimatc it \c can usc thc HC cstimator y
\hitc (19S0) or thc HAC cstimator y `cwcy and \cst (19S). scc
Scction 310
`cxt, wc dcrivc thc asymjtotic variancc tor f
n
(x) Iirst, dcnc
Z
i
K
h
(X
i
x) Thcn,
E|Z
1
Z
i
|
_ _
K
h
(u x)K
h
(v x) f
1,i
(u, v)dudv

_ _
K(u)K(v) f
1,i
(x + uh, x + v h)dudv
f
1,i
(x, x),
whcrc f
1,i
(u, v) is thc ,oint dcnsity ot (X
1
, X
i
), so that
Cov(Z
1
, Z
i
) f
1,i
(x, x) f
2
(x).
lt is casy to show that
h\ar(Z
1
)
0
(K) f(x),
whcrc
j
(K)
_
u
j
K
2
(u)du Thcrctorc,
nh\ar(f
n
(x)) \ar(Z
1
) + 2h
n

i=2
_
_
1
i 1
n
_
_
Cov(Z
1
, Z
i
)
. .
A
f
0 undcr somc assumjtions

0
(K) f(x).
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION248
To show that A
f
0, ct d
n
and d
n
h 0 Thcn,
|A
f
| h
d
n

i=2
|Cov(Z
1
, Z
i
)| + h
n

i=d
n
+1
|Cov(Z
1
, Z
i
)|.
Ior thc rst tcrm, it f
1,i
(u, v) M
1
, thcn, it is oundcd y hd
n

o(1) Ior thc sccond tcrm, wc ajjy thc Lavydovs incquaity to
otain
h
n

i=d
n
+1
|Cov(Z
1
, Z
i
)| M
2
n

i=d
n
+1
(i)/h O(d
+1
n
h
1
)
it (n) O(n

) tor somc > 2 lt d

n
O(h
2/
), thcn, thc
sccond tcrm is dominatcd y O(h
12/
) which ocs to 0 as n
Lcncc,
nh\ar(f
n
(x))
0
(K) f(x). (2)
\c can cstaish thc toowin asymjtotic normaity tor f
n
(x) ut
thc jroot wi c discusscd atcr
Theorem 1: Under regularity conditions, we have

nh
_

_f
n
(x) f(x)
h
2
2

2
(K) f

(x) + o
p
(h
2
)
_

_ N (0,
0
(K) f(x)) .
Exercise: Ly comjarin (1) and (2), what can you oscrvc
Example 1: Lct us cxaminc how imjortancc thc choicc ot and-
width is Thc data {X
i
}
n
i=1
arc cncratcd trom N(0, 1) (iid) and
n 300 Thc rid joints arc takcn to c |!, !| with an incrcmcnt
0.1 Landwidth is takcn to c 0.2`, 0.` and 1.0, rcsjcctivcy
and thc kcrnc can c thc Ljancchnikov kcrnc or Gaussian kcrnc
Example 2: `cxt, wc ajjy thc kcrnc dcnsity cstimation to thc
dcnsity ot thc wccky 3-month Trcasury i trom 1anuary 2, 190 to
Lcccmcr 2o, 199
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION249
`otc that thc comjutcr codc in R tor thc aovc two cxamjcs
can c tound in Scction ` R has a uid-in tunction density()
tor comjutin thc nonjaramctric dcnsity cstimation Aso, you can
usc thc command plot(density()) to jot thc cstimatcd dcnsity
Iurthcr, R has a uid-in tunction ecdf() tor comjutin thc cmjir-
ica cumuativc distriution tunction cstimation and plot(ecdf())
tor jottin thc stcj tunction
7.2.2 Optimality
As wc arcady havc shown that
E(f
n
(x)) f(x) +
h
2
2

2
(K) f

(x) + o(h
2
),
and
\ar(f
n
(x))

0
(K) f(x)
nh
+ o((nh)
1
),
so that thc asymjtotic mcan intcratcd squarcs crror (A`lSL) is
A`lSL
h
4
!

2
2
(K)
_
|f

(x)|
2
+

0
(K)
nh
.
`inimizin thc A`lSL ivcs thc
h
opt
C
1
(K) ||f

||
2/5
2
n
2/5
, (3)
whcrc
C
1
(K)
_

0
(K)/
2
2
(K)
_
1/5
.
\ith this asymjtoticay ojtima andwidth, thc ojtima A`lSL is
ivcn y
A`lSL
opt

`
!
C
2
(K) ||f

||
2/5
2
n
4/5
,
whcrc
C
2
(K)
_

2
0
(K)
2
(K)
_
2/5
.
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION250
To choosc thc cst kcrnc, it succs to choosc onc to minimizc C
2
(K)
Proposition 1: The nonnegative probability density function K
minimizing C
2
(K) is a re-scaling of the Epanechnikov kernel:
K
opt
(u)
3
! a
(1 u
2
/a
2
)
+
for any a > 0.
Proof: Iirst ot a, wc notc that C
2
(K
h
) C
2
(K) tor any h >
0 Lct K
0
c thc Ljancchnikov kcrnc Ior any othcr nonncativc
ncativc K, y rc-scain it ncccssary, wc assumc that
2
(K)

2
(K
0
) Thus, wc nccd ony to show that
0
(K)
0
(K) Lct
G K K
0
Thcn,
_
G(u)du 0 and
_
u
2
G(u)du 0,
which imjics that
_
(1 u
2
) G(u)du 0.
sin this and thc tact that K
0
has thc sujjort |1, 1|, wc havc
_
G(u) K
0
(u)du
3
!
_
|u|1
G(u)(1 u
2
)du

3
!
_
|u|>1
G(u)(1 u
2
)du
3
!
_
|u|>1
K(u)(u
2
1)du.
Sincc K is nonncativc, so is thc ast tcrm Thcrctorc,
_
K
2
(u)du
_
K
2
0
(u)du+2
_
K
0
(u)G(u)du+
_
G
2
(u)du
_
K
2
0
(u)du,
which jrovcs that K
0
is thc ojtima kcrnc
Remark: This jrojosition imjics that thc Ljancchnikov kcrnc
shoud c uscd in jracticc
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION251
7.2.3 Boundary Correction
ln many ajjications, thc dcnsity f() has a oundcd sujjort Ior
cxamjc, thc intcrcst ratc can not c css than zcro and thc incomc
is aways nonncativc lt is rcasonac to assumc that thc intcrcst
ratc has sujjort |0, 1) Lowcvcr, ccausc a kcrnc dcnsity cstima-
tor sjrcads smoothy joint masscs around thc oscrvcd data joints,
somc ot thosc ncar thc oundary ot thc sujjort arc distriutcd out-
sidc thc sujjort ot thc dcnsity Thcrctorc, thc kcrnc dcnsity cs-
timator undcr cstimatcs thc dcnsity in thc oundary rcions Thc
jrocm is morc scvcrc tor arc andwidth and tor thc ctt oundary
whcrc thc dcnsity is hih Thcrctorc, somc ad,ustmcnts arc nccdcd
To ain somc turthcr insihts, ct us assumc without oss ot cncraity
that thc dcnsity tunction f() has a oundcd sujjort |0, 1| and wc
dca with thc dcnsity cstimatc at thc ctt oundary Ior simjicity,
sujjosc that K() has a sujjort |1, 1| Ior thc ctt oundary joint
x c h (0 c < 1) , it can casiy c sccn that as h 0,
E(f
n
(ch))
_
1/hc
c
f(ch + hu)K(u)du
f(0+)
0,c
(K) + hf

(0+)|c
0,c
(K) +
1,c
(K)| + o(h), (!)
whcrc f(0+) im
x0
f(x),

j,c

_

c
u
j
K(u)du, and
j,c
(K)
_

c
u
j
K
2
(u)du.
Aso, wc can show that \ar(f
n
(ch)) O(1/nh) Thcrctorc,
f
n
(ch) f(0+)
0,c
(K) + hf

(0+)|c
0,c
(K) +
1,c
(K)| + o
p
(h).
larticuary, it c 0 and K() is symmctric, thcn E(f
n
(0))
f(0)/2 + o(1)
Thcrc arc scvcra mcthods to dca with thc dcnsity cstimation at
oundary joints lossic ajjroachcs incudc thc boundary ker-
nel (scc Gasscr and ` ucr (199) and ` ucr (1993)), reection
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION252
(scc Schustcr (19S`) and La and \chry (1991)), transformation
(scc \and, `arron and lujjcrt (1991) and `arron and lujjcrt
(199!)) and local polynomial tting (scc L,ort and 1oncs (199o)
and Loadcr (199o)), and othcrs
Boundary Kernel
Onc way ot choosin a oundary kcrnc is
K
(c)
(u)
12
(1 + c)
4
(1 + u)
_

_
(1 2c)u +
3c
2
2c + 1
2
_

_
I
[1,c]
.
`otc K
(1)
(t) K(t), thc Ljancchnikov kcrnc as dcncd aovc
`orcovcr, Zhan and Iarunamuni (199S) havc shown that this kcrnc
is ojtima in thc scnsc ot minimizin thc `SL in thc cass ot a
kcrncs ot ordcr (0, 2) with cxacty onc chanc ot sin in thcir sujjort
Thc downsidc to thc oundary kcrnc is that it is not ncccssariy non-
ncativc, as wi c sccn on dcnsitics whcrc f(0) 0
Reection
Thc rccction mcthod is to construct thc kcrnc dcnsity cstimatc
ascd on thc synthctic data {X
t
. 1 t n} whcrc rccctcd
data arc {X
t
. 1 t n} and thc oriina data a rc {X
t
. 1 t
n} This rcsuts in thc cstimatc
f
n
(x)
1
n
_
_
_
n

t=1
K
h
(X
t
x) +
n

t=1
K
h
(X
t
x)
_
_
_
, tor x 0.
`otc that whcn x is away trom thc oundary, thc sccond tcrm in thc
aovc is jracticay nciic Lcncc, it ony corrccts thc cstimatc
in thc oundary rcion This cstimator is twicc thc kcrnc dcnsity
cstimatc ascd on thc synthctic data {X
t
. 1 t n}
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION253
Transformation
Thc transtormation mcthod is to rst transtorm thc data y Y
i

g(X
i
), whcrc g() is a ivcn monotonc incrcasin tunction, ranin
trom to `ow ajjy thc kcrnc dcnsity cstimator to this
transtormcd data sct to otain thc cstimatc f
n
(y) tor Y and ajjy
thc invcrsc transtorm to otain thc dcnsity ot X Thcrctorc,
f
n
(x) g

(x)
1
n
n

t=1
K
h
(g(X
t
) g(x)).
Thc dcnsity at x 0 corrcsjonds to thc tai dcnsity ot thc trans-
tormcd data sincc o(0) , which can not usuay c cstimatcd
wc duc to ack ot thc data at tais Lxccjt at this joint, thc trans-
tormation mcthod docs a tairy ood ,o lt g() is unknown in many
situations, Iarunamuni and Acrts (2003) sucstcd a jaramctric
torm and thcn cstimatcd thc jaramctcr Aso, Iarunamuni and A-
crts (2003) considcrcd othcr tyjcs ot transtormations
Local Likelihood Fitting
Thc main idca is to considcr thc ajjroximation o(f(X
t
)) P(X
t

x), whcrc P(u x)

p
j=0
a
j
(u x)
j
with thc ocaizcd vcrsion ot
o-ikcihood
n

t=1
o(f(X
t
)) K
h
(X
t
x) n
_
K
h
(u x)f(u)du.
\ith this ajjroximation, thc oca ikcihood ccomcs
L(a
0
, , d
p
)
n

t=1
P(X
t
x) K
h
(X
t
x)n
_
K
h
(ux) cxj(P(ux))du.
Lct {

a
j
} c thc maximizcr ot thc aovc oca ikcihood L(a
0
, , d
p
)
Thcn, thc oca ikcihood dcnsity cstimatc is
f
n
(x) cxj(

a
0
).
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION254
Thc maximizcr docs not cxist, thcn f
n
(x) 0 Scc Loadcr (199o)
and L,ort and 1oncs (199o) tor morc dctais lt R is uscd tor thc
oca t tor dcnsity cstimation, jcasc usc thc tunction density.lf()
in thc jackac localt
Exercise: lcasc conduct a `ontc Caro simuation to scc what
thc oundary cccts arc and how thc corrcction mcthods work Ior
cxamjc, you can considcr somc distriution dcnsitics with a nitc
sujjort such as cta-distriution
7.3 Distribution Estimation
7.3.1 Smoothed Distribution Estimation
Thc qucstion is how to otain a smoothcd cstimatc ot CLI F(x)
\c, onc way ot doin so is to intcratc thc cstimatcd lLI f
n
(x),
ivcn y

F
n
(x)
_
x

f
n
(u)du
1
n
n

i=1
K
_
_
x X
i
h
_
_
,
whcrc K(x)
_
x

K(u)du. thc distriution ot K() \hy do wc

nccd this smoothcd cstimatc ot CLI To answcr this qucstion, wc
nccd to considcr thc mcan squarcs crror (`SL)
Iirst, wc dcrivc thc asymjtotic ias Ly thc intcration y jarts,
wc havc
E
_

F
n
(x)
_
E
_
_
K
_
_
x X
i
h
_
_
_
_

_
F(x hu)K(u)du
F(x) +
h
2
2

2
(K) f

(x) + o(h
2
).
`cxt, wc dcrivc thc asymjtotic variancc
E
_
_
K
2
_
_
x X
i
h
_
_
_
_

_
F(x hu)b(u)du F(x) hf(x) + o(h),
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION255
whcrc b(u) 2 K(u) K(u) and
_
ub(u)du Thcn,
\ar
_
_
K
_
_
x X
i
h
_
_
_
_
F(x)|1 F(x)| hf(x) + o(h).
Lcnc I
j
(x) Cov (I(X
1
x), I(X
j+1
t)) F
j
(x, x) F
2
(x)
and
I
nj
(x) Cov
_
_
K
_
_
x X
1
h
_
_
, K
_
_
x X
j+1
h
_
_
_
_
.
Ly mcans ot Lcmma 2 in Lchmann (19oo), thc covariancc I
nj
(x) may
c writtcn as toows
I
nj
(t)
_ _
P
_
_
K
_
_
x X
1
h
_
_
> u, K
_
_
x X
j+1
h
_
_
> v
_
_
P
_
_
K
_
_
x X
1
h
_
_
> u
_
_
P
_
_
K
_
_
x X
j+1
h
_
_
> v
_
_
_
dudv.
lnvcrtin thc CLI K() and makin two chancs ot variacs, thc
aovc rcation ccomcs
I
nj
(x)
_
|F
j
(xhu, xhv)F(xhu)F(xhv)|K(u)K(v)dudv.
Lxjandin thc riht-hand sidc ot thc aovc cquation accordin to
Tayors tormua, wc otain
|I
nj
(x) I
j
(x)| C h
2
.
Ly thc Lavydovs incquaity (scc Lcmma 1), wc havc
|I
nj
(x) I
j
(x)| C (j),
so that tor any 1/2 < < 1,
|I
nj
(x) I
j
(x)| C h
2

1
(j).
Thcrctorc,
1
n
n1

j=1
(n j)|I
nj
(x) I
j
(x)|
n1

j=1
|I
nj
(x) I
j
(x)| C h
2

j=1

1
(j) O
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION256
jrovidcd that

j=1

1
(j) < tor somc 1/2 < < 1 lndccd,
this assumjtion is satiscd it (n) O(n

) tor somc > 2 Ly

thc stationarity, it is ccar that
n\ar
_

F
n
(x)
_
\ar
_
_
K
_
_
x X
1
h
_
_
_
_
+
2
n
n1

j=1
(n j)I
nj
(x).
Thcrctorc,
n\ar
_

F
n
(x)
_
F(x)|1 F(x)| hf(x) + o(h) + 2

j=1
I
j
(x) + O(h
2
)

2
F
(x) hf(x) + o(h).
\c can cstaish thc toowin asymjtotic normaity tor

F
n
(x) ut
thc jroot wi c discusscd atcr
Theorem 2: Under regularity conditions, we have

n
_

_

F
n
(x) F(x)
h
2
2

2
(K) f

(x) + o
p
(h
2
)
_

_ N
_
0,
2
F
(x)
_
.
Simiary, wc havc
nA`SL(

F
n
(x))
nh
4
!

2
2
(K) |f

(x)|
2
+
2
F
(x) hf(x) .
lt > 0, minimizin thc A`SL ivcs thc
h
opt

_
_
_
f(x)

2
2
(K)|f

(x)|
2
_
_
_
1/3
n
1/3
,
and with this asymjtoticay ojtima andwidth, thc ojtima A`SL
is ivcn y
nA`SL
opt
(

F
n
(x))
2
F
(x)
3
!
_
_
_

2
f
2
(x)

2
(K)f

(x)
_
_
_
2/3
n
1/3
.
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION257
Remark: Irom thc atorcmcntioncd cquation, wc can scc that it
> 0, thc A`SL ot

F
n
(x) can c smacr than that tor F
n
(x) in
thc sccond ordcr Aso, it is casy to that it K() is thc Ljancchnikov
kcrnc, > 0
7.3.2 Relative Eciency and Deciency
To mcasurc thc rcativc ccicncy and dccicncy ot

F
n
(x) ovcr F
n
(x),
wc dcnc
i(n) min
_
k {1, 2, . . .}. `SL(F
k
(x)) `SL
_

F
n
(x)
__
.
\c havc thc toowin rcsuts without thc dctaicd jroot which can
c tound in Cai and loussas (199S)
Proposition 2: (i) Under regularity conditions,
i(n)
n
1, if and only if nh
4
n
0.
(ii) Under regularity conditions,
i(n) n
nh
(x), if and only if nh
3
n
0,
where (x) f(x)/
2
F
(x)
Remark: lt is ccar that thc quantity (x) may c ookcd ujon as
a way ot mcasurin thc jcrtormancc ot thc cstimatc

F
n
(x) Sujjosc
that thc kcrnc K() is choscn, so that > 0, which is cquivacnt to
(x) > 0 Thcn, tor sucicnty arc n, i(n) > n + nh
n
((x) )
Thus, i(n) is sustantiay arcr than n, and, indccd, i(n) n tcnds
to Actuay, lciss (19S1) and Iak (19S3) joscd thc qucstion
ot dctcrminin thc cxact vauc ot thc sujcriority ot ovcr a ccrtain
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION258
cass ot kcrncs `orc sjccicay, ct K
m
c thc cass ot kcrncs
K |1, 1| which arc asoutcy continuous and satisty thc
rcquircmcnts K(1) 0, K(1) 1, and
_
1
1
u

K(u)du 0,
1, , m, tor somc m 0, 1, (whcrc thc momcnt condition is
vacuous tor m 0) Sct
m
suj{. K K
m
} Thcn, `ammitzsch
(19S!) answcrcd thc qucstion joscd y showin in an ccant manncr
Scc Cai and loussas (199S) tor morc dctais and simuation rcsuts
7.4 Quantile Estimation
Lct X
(1)
X
(2)
X
(n)
dcnotc thc ordcr statistics ot {X
t
}
n
t=1

Lcnc thc invcrsc ot F(x) as F

1
(p) int{x . F(x) p},
whcrc is thc rca inc Thc traditiona cstimatc ot F(x) has
ccn thc cmjirica distriution tunction F
n
(x) ascd on X
1
, . . . , X
n
,
whic thc cstimatc ot thc p-th quantic
p
F
1
(p), 0 < p < 1, is thc
samjc quantic tunction
pn
F
1
n
(p) X
([np])
, whcrc |x| dcnotcs
thc intccr jart ot x lt is a consistcnt cstimator ot
p
tor -mixin
data (Yoshihara, 199`) Lowcvcr, as statcd in Iak (19S3), F
n
(x)
docs not takc into account thc smoothncss ot F(x). ic, thc cxis-
tcncc ot a jroaiity dcnsity tunction f(x) ln ordcr to incorjoratc
this charactcristic, invcstiators jrojoscd scvcra smoothcd quantic
cstimatcs, onc ot which is ascd on

F
n
(x) otaincd as a convoution
ctwccn F
n
(x) and a jrojcry scacd kcrnc tunction. scc thc jrcvious
scction Iinay, notc that R has a command quantile() which can
c uscd tor comjutin
pn
, thc nonjaramctric cstimatc ot quantic
7.4.1 Value at Risk
\auc at lisk (\al) is a jojuar mcasurc ot markct risk associatcd
with an assct or a jorttoio ot asscts lt has ccn choscn y thc Lasc
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION259
Committcc on Lankin Sujcrvision as a cnchmark risk mcasurc
and has ccn uscd y nancia institutions tor assct manacmcnt and
minimization ot risk Lct {X
t
}
n
t=1
c thc markct vauc ot an assct
ovcr n jcriods ot t 1 a timc unit, and ct Y
t
o(X
t
/X
t1
) c
thc o-rcturns Sujjosc {Y
t
}
n
j=1
is a stricty stationary dcjcndcnt
jroccss with marina distriution tunction F(y) Givcn a jositivc
vauc p cosc to zcro, thc 1 p cvc \al is

p
int{u F(u) p},
which sjccics thc smacst amount ot oss such that thc jroai-
ity ot thc oss in markct vauc cin arcr than
p
is css than p
Comjrchcnsivc discussions on \al arc avaiac in Luc and lan
(199) and 1orion (2001), and rctcrcnccs thcrcin Thcrctorc, \al can
c rcardcd as a sjccia casc ot quantic R has a uid-in jackac
cacd VaR tor a sct ot mcthods tor cacuation ot \al, jarticuary,
tor jaramctric modcs
Anothcr jojuar risk mcasurc is thc cxjcctcd shortta (LS) which
is thc cxjcctcd oss, ivcn that thc oss is at cast as arc as somc
ivcn quantic ot thc oss distriution (c, \al) lt is wc known
trom Artzncr, Lcacn, Lcr and Lcath (1999) that LS is a cohcrcnt
risk mcasurc such as it satiscs thc tour axioms homocncity (in-
crcasin thc sizc ot a jorttoio y a tactor shoud scac its risk mcasurc
y thc samc tactor), monotonicity (a jorttoio must havc rcatcr risk
it it has systcmaticay owcr vaucs than anothcr), risk-trcc condition
or transation invariancc (addin somc amount ot cash to a jorttoio
shoud rcducc its risk y thc samc amount), and suadditivity (thc
risk ot a jorttoio must c css than thc sum ot scjaratc risks or
mcrin jorttoios cannot incrcasc risk) \al satiscs homocncity,
monotonicity, and risk-trcc condition ut is not su-additivc Scc
Artzncr, et al. (1999) tor dctais
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION260
7.4.2 Nonparametric Quantile Estimation
Thc smoothcd samjc quantic cstimatc ot
p
,

p
, ascd on

F
n
(x), is
dcncd y

p


F
1
n
(p) int
_
x .

F
n
(x) p
_
.

p
is rctcrrcd to in itcraturc as thc jcrturcd (smoothcd) samjc
quantic Asymjtotic jrojcrtics ot

p
, oth undcr indcjcndcncc as
wc as undcr ccrtain modcs ot dcjcndcncc, havc ccn invcstiatcd
cxtcnsivcy in itcraturc. scc Cai and loussas (199) and Chcn and
Tan (200`)
Ly thc dicrcntiaiity ot

F
n
(x), wc usc thc Tayor cxjansion and
inorc thc hihcr tcrms to otain

F
n
(

p
) p

F
n
(
p
) f
n
(
p
) (

p
), (`)
thcn,

p
|

F
n
(
p
) p|/f
n
(
p
) |

F
n
(
p
) p|/f(
p
)
sincc f
n
(x) is a consistcnt cstimator ot f(x) As an ajjication ot
Thcorcm 2, wc can cstaish thc toowin thcorcm tor thc asymjtotic
normaity ot

p
ut thc jroot is omittcd sincc it is simiar to that tor
Thcorcm 2
Theorem 3: Under regularity conditions, we have

n
_

h
2
2

2
(K) f

(
p
)/f(
p
) + o
p
(h
2
)
_

_ N
_
0,
2
F
(
p
)/f
2
(
p
)
_
.
`cxt, ct us cxaminc thc A`SL To this ccct, wc can dcrivc thc
asymjtotic ias and variancc Irom thc jrcvious scction, wc havc
E
_

p
_

p
+
h
2
2

2
(K) f

(
p
)/f(
p
) + o
p
(h
2
),
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION261
and
n\ar
_

p
_

2
F
(
p
)/f
2
(
p
) h/f(
p
) + o(h).
Thcrctorc, thc A`SL is
nA`SL(

p
)
nh
4
!

2
2
(K) |f

(
p
)/f(
p
)|
2
+
2
F
(
p
)/f
2
(
p
)h/f(
p
).
lt > 0, minimizin thc A`SL ivcs thc
h
opt

_
_
_
f(
p
)

2
2
(K)|f

(
p
)|
2
_
_
_
1/3
n
1/3
,
and with this asymjtoticay ojtima andwidth, thc ojtima A`SL
is ivcn y
nA`SL
opt
(

p
)
2
F
(
p
)/f
2
(
p
)
3
!
_
_
_

2
(K)f

(
p
)f(
p
)
_
_
_
2/3
n
1/3
,
which indicatcs a rcduction to thc A`SL ot thc sccond ordcr Chcn
and Tan (200`) conductcd an intcnsivc study on simuations to
dcmonstratc thc advantacs ot nonjaramctric cstimation

p
ovcr thc
samjc quantic
pn
undcr thc \al scttin \c rctcr to thc jajcr y
Chcn and Tan (200`) tor simuation rcsuts and cmjirica cxamjcs
Exercise: lcasc usc thc aovc jroccdurcs to cstimatc nonjaramct-
ricay thc LS and discuss its jrojcrtics as wc as conduct simuation
studics and cmjirica ajjications
7.5 Computer Code
# July 20, 2006
graphics.off() # clean the previous garphs on the secreen
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION262
#########################################################
# Define the Epanechnikov kernel function
kernel<-function(x){0.75*(1-x^2)*(abs(x)<=1)}
###############################################################
# Define the kernel density estimator
kernden=function(x,z,h,ker){
# parameters: x=variable; h=bandwidth; z=grid point; ker=kernel
nz<-length(z)
nx<-length(x)
x0=rep(1,nx*nz)
dim(x0)=c(nx,nz)
x1=t(x0)
x0=x*x0
x1=z*x1
x0=x0-t(x1)
if(ker==1){x1=kernel(x0/h)} # Epanechnikov kernel
if(ker==0){x1=dnorm(x0/h)} # normal kernel
f1=apply(x1,2,mean)/h
return(f1)
}
###################################################################
###################################################################
# Simulation for different bandiwidths and different kernels
n=300 # n=300
ker=1 # ker=1 => Epan; ker=0 => Gaussian
h0=c(0.25,0.5,1) # set initial bandwidths
z=seq(-4,4,by=0.1) # grid points
nz=length(z) # number of grid points
x=rnorm(n) # simulate x ~ N(0, 1)
if(ker==1){h_o=2.34*n^{-0.2}} # optimal bandwidth for Epanechnikov
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION263
if(ker==0){h_o=1.06*n^{-0.2}} # optimal bandwidth for normal
f1=kernden(x,z,h0[1],ker)
f2=kernden(x,z,h0[2],ker)
f3=kernden(x,z,h0[3],ker)
f4=kernden(x,z,h_o,ker)
text1=c("True","h=0.25","h=0.5","h=1","h=h_o")
data=cbind(dnorm(z),f1,f2,f3,f4) # combine them
win.graph()
matplot(z,data,type="l",lty=1:5,col=1:5,xlab="",ylab="")
legend(-1,0.2,text1,lty=1:5,col=1:5)
###################################################################
###################################################################
# A Real Example
##################
z1=matrix(scan(file="c:\\teaching\\time series\\data\\w-3mtbs7097.txt"),
byrow=T,ncol=4)
# dada: weekly 3-month Treasury bill from 1970 to 1997
x=z1[,4]/100 # decimal
n=length(x)
y=diff(x) # Delta x_t=x_t-x_{t-1}=change
x=x[1:(n-1)]
n=n-1
x_star=(x-mean(x))/sqrt(var(x)) # standardized
den_3mtb=density(x_star,bw=0.30,kernel=c("epanechnikov"),from=-3
den_est=den_3mtb$y # estimated density values
z_star=seq(-3,3,by=0.1)
text1=c("Estimated Density","Standard Norm")
win.graph()
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION264
par(bg="light green")
plot(den_3mtb,main="Density of 3mtb (Buind-in)",ylab="",xlab="",col.main="red")
points(z_star,dnorm(z_star),type="l",lty=2,col=2,ylab="",xlab=""
legend(0,0.45,text1,lty=c(1,2),col=c(1,2),cex=0.7)
h_den=0.5
f_hat=kernden(x_star,z_star,h_den,1)
ff=cbind(f_hat,dnorm(z_star))
win.graph()
par(bg="light blue")
matplot(z_star,ff,type="l",lty=c(1,2),col=c(1,2),ylab="",xlab=
title(main="Density of 3mtb",col.main="red")
legend(0,0.55,text1,lty=c(1,2),col=c(1,2),cex=0.7)
###################################################################
7.6 References
Artzner, P., F. Delbaen, J.M. Eber, and D. Heath (1999). Coherent measures of risk.
Mathematical Finance, 9, 203-228.
Cai, Z. (2002). Regression quantile for time series. Econometric Theory, 18, 169-192.
Cai, Z. and G.G. Roussas (1997). Smooth estimate of quantiles under association. Statistics
and Probability Letters, 36, 275-287.
Cai, Z. and G.G. Roussas (1998). Ecient estimation of a distribution function under
quadrant dependence. Scandinavian Journal of Statistics, 25, 211-224.
Chen, S.X. and C.Y. Tang (2005). Nonparametric inference of value at risk for dependent
nancial returns. Journal of Financial Econometrics, 3, 227-255.
Due, D. and J. Pan (1997). An overview of value at risk. Journal of Derivatives, 4, 7-49.
Fan, J. and Q. Yao (2003). Nonlinear Time Series: Nonparametric and Parametric Meth-
ods. Springer-Verlag, New York.
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION265
Gasser, T. and H.-G. M uller (1979). Kernel estimation of regression functions. In Smoothing
Techniques for Curve Estimation, Lecture Notes in Mathematics, 757, 23-68. Springer-
Verlag, New York.
Falk, M.(1983). Relative eciency and deciency of kernel type estimators of smooth
distribution functions. Statistica Neerlandica, 37, 73-83.
Hall, P. and C.C. Heyde (1980). Martingale Limit Theory and its Applications. Academic
Press, New York.
Hall, P. and T.E. Wehrly (1991). A geometrical method for removing edge eects from
kernel-type nonparametric regression estimators. Journal of American Statistical As-
sociation, 86, 665-672.
Hjort, N.L. and M.C. Jones (1996). Locally parametric nonparametric density estimation.
The Annals of Statistics, 24,1619-1647.
Jorion, P. (2001). Value at Risk, 2nd Edition. New York: McGraw-Hill.
Karunamuni, R.J. and T. Alberts (2003). On boundary correction in kernel density estima-
tion. Working paper, Department of Mathematical an d Statistical Sciences, University
of Alberta, Canada.
Lehmann, E. (1966). Some concepts of dependence. Annals of Mathematical Statistics, 37,
1137-1153.
Loader, C. R. (1996). Local likelihood density estimation. The Annals of Statistics, 24,
1602-1618.
Mammitzsch, V. (1984). On the asymptotically optimal solution within a certain class of
kernel type estimators. Statistics Decisions, 2, 247-255.
Marron, J.S. and D. Ruppert (1994). Transformations to reduce boundary bias in kernel
density estimation. Journal of the Royal Statistical Society Series B, 56, 653-671.
M uller, H.-G. (1993). On the boundary kernel method for nonparametric curve estimation
near endpoints. Scandinavian Journal of Statistics, 20, 313-328.
Reiss, R.D. (1981). Nonparametric estimation of smooth distribution functions. Scandi-
navia Journal of Statistics, 8, 116-119.
Schuster, E.F. (1985). Incorporating support constraints into nonparametric estimates of
densities. Communications in Statistics Theory and Methods, 14, 1123-1126.
Wand, M.P., J.S. Marron and D. Ruppert (1991). Transformations in density estimation
(with discussion). Journal of the American Statistical Association, 86, 343-361.
Yoshihara, K. (1995). The Bahadur representation of sample quantiles for sequences of
strongly mixing random variables. Statistics and Probability Letters, 24, 299-304.
CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION266
Zhang, S. and R.J. Karunamuni (1998). On Kernel density estimation near endpoints.
Journal of Statistical Planning and Inference, 70, 301-316.
Chapter 8
Nonparametric Regression Estimation
8.1 Bandwidth Selection
8.1.1 Simple Bandwidth Selectors
Thc ojtima andwidth (3) is not dirccty usac sincc it dcjcnds
on thc unknown jaramctcr ||f

||
2
\hcn f(x) is a Gaussian dcnsity
with standard dcviation , it is casy to scc trom (3) that
h
opt
(S

/3)
1/5
C
1
(K) n
1/5
,
which is cacd thc normal reference bandwidth selector in
itcraturc, otaincd y rcjacin thc unknown jaramctcr in thc
aovc cquation y thc samjc standard dcviation s ln jarticuar,
attcr cacuatin thc constant C
1
(K) numcricay, wc havc thc to-
owin norma rctcrcncc andwidth sccctor

h
opt

_

_
1.0o s n
1/5
tor thc Gaussian kcrnc
2.3! s n
1/5
tor thc Ljancchnikov kcrnc
L,ort and 1oncs (199o) jrojoscd an imjrovcd ruc otaincd y usin
an Ldcworth cxjansion tor f(x) around thc Gaussian dcnsity Such
a ruc is ivcn y

opt
h
opt
_
_
1 +
3`
!S

4
+
3`
32

2
3
+
3S`
102!

2
4
_
_
1/5
,
267
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 268
whcrc

3
and

4
arc rcsjcctivcy thc samjc skcwncss and kurtosis
`otc that thc norma rctcrcncc andwidth sccctor is ony a sim-
jc ruc ot thum lt is a ood sccctor whcn thc data arc ncary
Gaussian distriutcd, and is ottcn rcasonac in many ajjications
Lowcvcr, it can cad to ovcr-smooth whcn thc undcryin distriu-
tion is asymmctric or muti-moda ln that casc, onc can cithcr
su,cctivcy tunc thc andwidth, or sccct thc andwidth y morc
sojhisticatcd andwidth sccctors Onc can aso transtorm data rst
to makc thcir distriution coscr to norma, thcn cstimatc thc dcnsity
usin thc norma rctcrcncc andwidth sccctor and ajjy thc invcrsc
transtorm to otain an cstimatcd dcnsity tor thc oriina data Such
a mcthod is cacd thc transtormation mcthod Thcrc arc quitc a
tcw imjortant tcchniqucs tor sccctin thc andwidth such as cross-
vaidation (C\) and ju-in andwidth sccctors A conccjtuay
simjc tcchniquc, with thcorctica ,ustication and ood cmjirica
jcrtormancc, is thc ju-in tcchniquc This tcchniquc rcics on nd-
in an cstimatc ot thc tunctiona ||f

||
2
, which can c otaincd y
usin a jiot andwidth An imjcmcntation ot this ajjroach is jro-
joscd y Shcathcr and 1oncs (1991) and an ovcrvicw on thc jrorcss
ot andwidth sccction can c tound in 1oncs, `arron and Shcathcr
(199o)
Iunction dpik() in thc jackac KernSmooth in R scccts a
andwidth tor cstimatin thc kcrnc dcnsity cstimation usin thc
ju-in mcthod
8.1.2 Cross-Validation Method
Thc intcratcd squarcd crror (lSL) ot f
n
(x) is dcncd y
lSL(h)
_
|f
n
(x) f(x)|
2
dx.
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 269
A commony uscd mcasurc ot discrcjancy ctwccn f
n
(x) and f(x) is
thc mcan intcratcd squarcd crror (`lSL) `lSL(h) E|lSL(h)| lt
can c shown casiy (or scc Chiu, 1991) that `lSL(h) A`lSL(h)
Thc ojtima andwidth minimizin thc A`lSL is ivcn in (3) Thc
cast squarcs cross-vaidation (LSC\) mcthod jrojoscd y ludcmo
(19S2) and Lowman (19S!) is a jojuar mcthod to cstimatc thc
ojtima andwidth h
opt
Cross-vaidation is vcry usctu tor asscssin
thc jcrtormancc ot an cstimator via cstimatin its jrcdiction crror
Thc asic idca is to sct onc ot thc data joint asidc tor vaidation ot
a modc and usc thc rcmainin data to uid thc modc Thc main
idca is to choosc h to minimizc lSL(h) Sincc
lSL(h)
_
f
2
n
(x)dx 2
_
f(x) f
n
(x)dx +
_
f
2
(x)dx,
thc qucstion is how to cstimatc thc sccond tcrm on thc riht hand
sidc \c, ct us considcr thc simjcst casc whcn {X
t
} arc iid lc-
cxjrcss f
n
(x) as
f
n
(x)
n 1
n
f
(s)
n
(x) +
1
n
K
h
(X
s
x)
tor any 1 s n, whcrc
f
(s)
n
(x)
1
n 1
n

t=s
K
h
(X
t
x),
which is thc kcrnc dcnsity cstimatc without thc sth oscrvation,
commony cacd thc jackknife cstimatc or cavc-onc-out cstimatc
lt is casy to scc that tor any 1 s n,
f
n
(x) f
(s)
n
(x).
Lct D
s
{X
1
, , X
s1
, X
s+1
, , X
n
} Thcn,
E
_
f
(s)
n
(X
s
) | D
s
_

_
f
(s)
n
(x)f(x)dx
_
f
n
(x)f(x)dx,
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 270
which, y usin thc mcthod ot momcnt, can c cstimatcd y
1
n

n
s=1
f
(s)
n
(X
s
)
Thcrctorc, thc cross-vaidation is
C\(h)
_
f
2
n
(x)dx
2
n
n

s=1
f
(s)
n
(X
s
)

1
n
2

s,t
K

h
(X
s
X
t
)
2
n(n 1)
n

t=s
K
h
(X
s
X
t
),
whcrc K

h
() is thc convoution ot K
h
() and K
h
() as
K

h
(u)
_
K
h
(v) K
h
(u v)dv.
Lct

h
cv
c thc minimizcr ot C\(h) Thcn, it is cacd thc ojtima
andwidth ascd on thc cross-vaidation Stonc (19S!) showcd that

h
cv
is a consistcnt cstimatc ot thc ojtima andwidth h
opt

Iunction lscv() in thc jackac loct in R scccts a andwidth

tor cstimatin thc kcrnc dcnsity cstimation usin thc cast squarcs
cross-vaidation mcthod
8.2 Multivariate Density Estimation
As wc discusscd in Chajtcr , thc kcrnc dcnsity or distriution csti-
mation is asicay onc-dimcnsiona Ior mutivariatc casc, thc kcrnc
dcnsity cstimatc is ivcn y
f
n
(x)
1
n
n

t=1
K
H
(X
t
x), (S1)
whcrc K
H
(u) K(H
1
u)/ dct(H), K(u) is a mutivariatc kcrnc
tunction, and H is thc andwidth matrix such as tor a 1 i, j p,
nh
ij
and h
ij
0 whcrc h
ij
is thc (i, j)th ccmcnt ot H Thc
andwidth matrix is introduccd to cajturc thc dcjcndcnt structurc
in thc indcjcndcnt variacs larticuary, it H is a diaona matrix
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 271
and K(u)

p
j=1
K
j
(u
j
) whcrc K
j
() is a univariatc kcrnc tunction,
thcn, f
n
(x) ccomcs
f
n
(x)
1
n
n

t=1
p

j=1
K
h
j
(X
jt
x
j
),
which is cacd thc jroduct kcrnc dcnsity cstimation This casc is
commony uscd in jracticc Simiar to thc univariatc casc, it is casy
to dcrivc thc thcorctica rcsuts tor thc mutivariatc casc, which is
ctt as an cxcrcisc Scc \and and 1oncs (199`) tor dctais
Exercise: lcasc dcrivc thc asymjtotic rcsuts ivcn in (S1) tor thc
cncra mutivariatc casc
ln R, thc uid-in tunction density() is ony tor univariatc casc
Ior mutivariatc situations, thcrc arc two jackacs ks and KernS-
mooth Iunction kde() in ks can comjutc thc mutivariatc dcnsity
cstimatc tor 2- to o- dimcnsiona data and Iunction bkde2D() in
KernSmooth comjutcs thc 2L kcrnc dcnsity cstimatc Aso, ks
jrovidcs somc tunctions tor somc andwidth matrix sccction such
as Hbcv() and Hscv tor 2L casc and Hlscv() and Hpi()
8.3 Regression Function
Sujjosc that wc havc thc intormation sct I
t
at timc t and wc want to
torccast thc tuturc vauc, say Y
t+1
(onc stcj-ahcad torccast, or Y
t+s
,
s-stcj ahcad) Thcrc arc scvcra torccastin critcria Thc cncra
torm is
m(I
t
) min
a
E|(Y
t+1
a) | I
t
|,
whcrc () is an o,cctivc (oss) tunction Thcrc arc thrcc ma,or
critcria
(1) lt () is thc quadratic tunction, thcn, m(I
t
) E(Y
t+1
| I
t
),
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 272
cacd thc mcan rcrcssion tunction
(2) lt

(y) y ( I
{y<0}
) cacd thc chcck tunction, whcrc
(0, 1) and I
A
is thc indicator tunction ot any sct A, thcn, m(I
t
)
satiscs
_
m(I
t
)

f(y | I
t
) du F(m(I
t
) | I
t
) ,
whcrc f(y | I
t
) and F(m(I
t
) | I
t
) arc thc conditiona lLI and CLI
ot Y
t+1
, rcsjcctivcy, ivcn I
t
This m(I
t
) ccomcs thc conditiona
quantic or quantic rcrcssion, dcntcd y q

(I
t
) larticuary, it
1/2, thcn, m(I
t
) is thc wc known asoutc dcviation (LAL)
rcrcssion which is roust
(3) lt (x)
1
2
x
2
I
|x|M
+M(|x| M/2) I
|x|>M
, thc so cacd Lucr
tunction in itcraturc, thcn it is thc Lucr roust rcrcssion \c wi
not discuss this tojic lt you havc an intcrcst, jcasc rcad thc ook
y loussccuw and Lcroy (19S) ln R, thc irary MASS has thc
tunction rlm tor roust incar modc Aso, thc irary lqs contains
tunctions tor oundcd-inucncc rcrcssion
Sincc thc intormation sct I
t
contains too many variacs (hih di-
mcnsion), it is ottcn to ajjroximatc I
t
y somc nitc numcrs ot
variacs, say X
t
(X
t1
, . . . , X
tp
)
T
(p 1), incudin thc acd
variacs and cxocnous variacs Iirst, our tocus is on thc mcan
rcrcssion m(X
t
) Ot coursc, y thc samc tokcn, wc can considcr
thc nonjaramctric cstimation ot thc conditiona variancc
2
(x)
\ar(Y
t
|X
t
x) \hy do wc nccd to considcr nonincar (nonjara-
mctric) modcs in cconomic jracticc You can nd thc answcr in thc
ook y Grancr, C\1, and T Tcrasvirta (1993)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 273
8.4 Kernel Estimation
Lct us ook at thc `adaraya-\atson cstimatc ot thc mcan rcrcssion
m(X
t
) Thc main idca is as toows
m(x)
_
y f(y | x)dy
_
y f(x, y)dy
_
f(x, y)dy
,
whcrc f(x, y) is thc ,oint lLI ot X
t
and Y
t
To cstimatc m(x),
wc can ajjy thc ju-in mcthod That is, ju thc nonjaramctric
kcrnc dcnsity cstimatc f
n
(x, y) (douc kcrnc mcthod) into thc riht
hand sidc ot thc aovc cquation to otain

m
nw
(x)
_
y f
n
(x, y)dy
_
f
n
(x, y)dy

1
n
n

t=1
Y
t
K
h
(X
t
x)/f
n
(x)

t=1
W
t
Y
t
,
whcrc f
n
(x) is thc kcrnc dcnsity cstimation ot f(x), dcncd in Chaj-
tcr , and
W
t
K
h
(X
t
x)/
n

t=1
K
h
(X
t
x).

m
nw
(x) is thc wc known `adaraya-\atson (`\) cstimator `otc
that thc wcihts {W
t
} do not dcjcnd on {Y
t
} Thcrctorc,

m
nw
(x)
is cacd a linear estimator, simiar to thc cast squarcs cstimatc
(LSL)
Lct us ook at thc `\ cstimator trom a dicrcnt anc

m
nw
(x)
can c rc-cxjrcsscd as thc minimizcr ot thc weighted locally least
squares. that is,

m
nw
(x) min
a
n

t=1
(Y
t
a)
2
K
h
(X
t
x).
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 274
This mcans that whcn X
t
is in a ncihorhood ot x, m(X
t
) is ajjrox-
imatcd y a constant a (oca ajjroximation) lndccd, wc considcr
thc toowin workin modc
Y
t
m(X
t
) +
t
a +
t
with thc wcihts {K
h
(X
t
x)}, whcrc
t
Y
t
E(Y
t
| X
t
)
ln thc imjcmcntation, tor cach x, wc can t thc toowin trans-
tormcd incar modc
Y

t

1
X

t
+
t
,
whcrc Y

t

_
K
h
(X
t
x) Y
t
and X

t

_
K
h
(X
t
x) Thcrctorc,
thc `adaraya-\atson cstimator is aso cacd thc local constant
estimator ln R, wc can usc tunctions lm() or glm() with wcihts
{K
h
(X
t
x)} to t a wcihtcd cast squarcs or cncraizcd incar
modc Or, you can usc thc wcihtcd cast squarcs thcory (matrix
mutijication)
8.4.1 Asymptotic Properties
\c dcrivc thc asymjtotic jrojcrtics ot thc nonjaramctric cstimator
tor thc timc scrics situations Aso, wc considcr thc simjc casc that
p 1

m
nw
(x)
1
n
n

t=1
m(X
t
) K
h
(X
t
x)/f
n
(x)
. .
I
1
+
n

t=1
W
t

t
. .
I
2
.
\c wi show that I
1
contriutcs ony ias and I
2
ivcs thc asymjtotic
normaity Iirst, wc dcrivc thc asymjtotic ias tor thc intcrior ound-
ary joints Ly thc Tayors cxjansion, whcn X
t
is in (xh, x+h),
wc havc
m(X
t
) m(x) + m

(x)(X
t
x) +
1
2
m

(x
t
)(X
t
x)
2
,
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 275
whcrc x
t
x + (X
t
x) with 1 < < 1 Thcn,
I
11

1
n
n

t=1
m(X
t
) K
h
(X
t
x)
m(x) f
n
(x) + m

(x)
1
n
n

t=1
(X
t
x) K
h
(X
t
x)
. .
J
1
(x)
+
1
2
1
n
n

t=1
m

(x
t
)(X
t
x)
2
K
h
(X
t
x)
. .
J
2
(x)
.
Thcn,
E|J
1
(x)| E|(X
t
x) K
h
(X
t
x)|

_
(u x)K
h
(u x)f(u)du
h
_
uK(u)f(x + hu)du
h
2
f

(x)
2
(K) + o(h
2
).
Simiar to thc dcrivation ot thc variancc ot f
n
(x) in (2), wc can
show that
nh\ar(J
1
(x)) O(1).
Thcrctorc, J
1
(x) h
2
f

(x)
2
(K) + o
p
(h
2
) Ly thc samc tokcn, wc
havc
E|J
2
(x)| E
_
m

(x
t
)(X
t
x)
2
K
h
(X
t
x)
_
h
2
_
m

(x + hu)u
2
K(u)f(x + hu)du
h
2
m

(x)
2
(K) f(x) + o(h
2
)
and \ar(J
2
(x)) O(1/nh) Thcrctorc, J
2
(x) h
2
m

(x)
2
(K) f(x)+
o
p
(h
2
) Lcncc,
I
1
m(x) + m

(x) J
1
(x)/f
n
(x) +
1
2
J
2
(x)/f
n
(x)
m(x) +
h
2
2

2
(K) |m

(x) + 2 m

(x)f

(x)/f(x)| + o
p
(h
2
)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 276
y thc tact that f
n
(x) f(x) + o
p
(1) Thc tcrm
B
nw
(x)
h
2
2

2
(K) |m

(x) + 2 m

(x)f

(x)/f(x)|
is rcardcd as thc asymjtotic ias lt p > 1 (mutivariatc casc),
B
nw
(x) ccomcs
B
nw
(x)
h
2
2
tr
_

2
(K)
_
m

(x) + 2 f

(x)m

(x)
T
/f(x)
__
, (S2)
whcrc
2
(K)
_
uu
T
K(u)du Thc ias tcrm invovcs not ony
curvaturcs ot m(x) ut aso thc unknown dcnsity tunction f(x) and
its dcrivativc f

(x) so that thc dcsin can not c adajtivc

ndcr somc rcuarity conditions, simiar to (2), wc can show
that tor x cin an intcrior rid joint,
nh
p
\ar(I
2
)
0
(K)
2

(x)/f(x)
2
m
(x),
whcrc
2

(x) \ar(
t
| X
t
x) Iurthcr, wc can cstaish thc
asymjtotic normaity (thc jroot is jrovidcd atcr)

nh
p
_

m
nw
(x) m(x) B
nw
(x) + o
p
(h
2
)
_
N
_
0,
2
m
(x)
_
,
whcrc B
nw
(x) is ivcn in (S2)
\hcn p is arc, thcrc cxists thc so cacd curse of dimension-
ality To undcrstand this jrocm quantitativcy, wc can ook at
thc ratc ot convcrcncc Thc ias is ot ordcr O(h
2
) and thc variancc
is ot ordcr O(/nh
p
) This cads to thc ojtima ratc ot convcrcncc
tor `SL O(n
2/(4+p)
) y tradin o thc ratcs ctwccn thc ias and
variancc To havc a comjarac jcrtormancc with onc-dimcnsiona
nonjaramctric rcrcssion with n
1
data joints, tor p-dimcnsiona non-
jaramctric rcrcssion, wc nccd
n
2/(4+p)
O(n
2/5
1
),
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 277
Table 8.1: Sample sizes required for p-dimensional nonparametric regression to have compa-
rable performance with that of 1-dimensional nonparametric regression using size 100
dimension 2 3 4 5 6 7 8 9 10
sample size 252 631 1,585 3,982 10,000 25,119 63,096 158,490 398,108
or n O(n
(p+4)/5
1
) Tac S1 shows thc rcsut with n
1
100 Thc
incrcasc ot rcquircd samjc sizcs is cxjoncntiay tast
8.4.2 Boundary Behavior
As tor thc oundary chavior ot thc `\ cstimator, wc can toow Ian
and Gi,cs (199o) \ithout oss ot cncraity, wc considcr thc ctt
oundary joint x c h, 0 < c < 1 Irom Ian and Gi,cs (199o),
wc takc K() to havc sujjort |1, 1| and m() to havc sujjort
|0, 1| Simiar to (!), it is casy to scc that it x c h,
E|J
1
(ch)|
E|(X
t
ch) K
h
(X
t
ch)|

_
1
0
(u ch)K
h
(u ch)f(u)du
h
_
1/hc
c
uK(u)f(h(u + c))du
hf(0+)
1,c
(K) + h
2
f

(0+)|
2,c
(K) + c
1,c
(K)| + o(h),
and
E|J
2
(ch)| E
_
m

(x
t
)(X
t
ch)
2
K
h
(X
t
ch)
_
h
2
_
1/hc
c
m

(h(c + u))u
2
K(u)f(h(u + c))du
h
2
m

(0+)
2,c
(K) f(0+) + o(h
2
).
Aso, wc can scc that
\ar(J
1
(ch)) O(1/nh) and \ar(J
2
(ch)) O(1/nh),
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 278
which imjy that
J
1
(ch) hf(0+)
1,c
(K) + o
p
(h)
and
J
2
(ch) h
2
m

(0+)
2,c
(K) f(0+) + o(h
2
).
This, in con,unction with (!), ivcs
I
1
m(ch) m

(ch) J
1
(ch)/f
n
(ch) +
1
2
J
2
(ch)/f
n
(ch)
a(c, K) h + b(c, K) h
2
+ o
p
(h
2
)
whcrc
a(c, K)
m

(0+)
1,c
(K)

0,c
(K)
,
and
b(c, K)

2,c
(K) m

(0+)
2
0,c
(K)
+
f

(0+)m

(0+)|
2,c
(K)
0,c
(K)
2
1,c
(K)|
f(0+)
2
0,c
(K)
.
Lcrc, a(c, K) h+b(c, K) h
2
scrvcs as thc asymjtotic ias tcrm, which
is ot thc ordcr O(h)
\c can show that at thc oundary joint, thc asymjtotic variancc
has thc toowin torm
nh
p
\ar(

m
nw
(x))
0,c
(K)
2
m
(0+)/|
0,c
(K) f(0+)|,
which thc samc ordcr as that tor thc intcrior joint athouh thc
scain constant is dicrcnt
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 279
8.5 Local Polynomial Estimate
To ovcrcomc thc aovc shortcomins ot oca constant cstimatc, wc
can usc thc oca joynomia ttin schcmc. scc Ian and Gi,cs
(199o) Thc main idca is dcscricd as toows
8.5.1 Formulation
Assumc that thc rcrcssion tunction m(x) has (q + 1)th ordcr con-
tinuous dcrivativc Ior casc notation, assumc that p 1 \hcn
X
t
(x h, x + h), thcn
m(X
t
)
q

j=0
m
(j)
(x)
j'
(X
t
x)
j

j=0

j
(X
t
x)
j
,
whcrc
j
m
(j)
(x)/j' Thcrctorc, whcn X
t
(x h, x + h), thc
modc ccomcs
Y
t

j=0

j
(X
t
x)
j
+
t
.
Lcncc, wc can ajjy thc wcihtcd cast squarcs mcthod Thc weighted
locally least squares ccomcs
n

t=1
_
_
_Y
t

j=0

j
(X
t
x)
j
_
_
_
2
K
h
(X
t
x). (S3)
`inimizin thc aovc with rcsjcct to (
0
, . . . ,
q
)
T
to otain
thc oca joynomia cstimatc

.

_
X
T
WX
_
1
X
T
WY, (S!)
whcrc W dia{K
h
(X
1
x), , K
h
(X
n
x)},
X
_
_
_
_
_
_
_
_
_
_
1 (X
1
x) (X
1
x)
q
1 (X
2
x) (X
2
x)
q

1 (X
n
x) (X
n
x)
q
_
_
_
_
_
_
_
_
_
_
, and Y
_
_
_
_
_
_
_
_
_
_
Y
1
Y
2

Y
n
_
_
_
_
_
_
_
_
_
_
.
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 280
Thcrctorc, tor 1 j q,

m
(j)
(x) j'

j
.
This mcans that thc oca joynomia mcthod cstimatcs not ony thc
rcrcssion tunction itsct ut aso dcrivativcs ot rcrcssion
8.5.2 Implementation in R
Thcrc arc scvcra ways ot imjcmcntin thc oca joynomia csti-
mator Onc way you can do so is to writc your own codc y us-
in matrix mutijication as in (S!) or cmjoyin tunction lm() or
glm() with wcihts {K
h
(X
t
x)} lcccnty, in R, thcrc arc somc
uid-in jackacs tor imjcmcntin thc oca joynomia cstimatc
Ior cxamjc, thc jackac KernSmooth contains scvcra tunctions
Iunction bkde() comjutcs thc kcrnc dcnsity cstimatc and Iunc-
tion bkde2D() comjutcs thc 2L kcrnc dcnsity cstimatc as wc
as Iunction bkfe() comjutcs thc kcrnc tunctiona (dcrivativc) dcn-
sity cstimatc Iunction dpik() scccts a andwidth tor cstimatin
thc kcrnc dcnsity cstimation usin thc ju-in mcthod and Iunction
dpill() choosc a andwidth tor thc oca incar (q 1) rcrcssion cs-
timation usin thc ju-in ajjroach Iinay, Iunction locpoly() is
tor thc oca joynomia ttin incudin a oca joynomia cstimatc
ot thc dcnsity ot X (or its dcrivativc)
Example: \c ajjy thc kcrnc rcrcssion cstimation mcthod and
oca joynomia ttin mcthod to thc dritt and diusion ot thc
wccky 3-month Trcasury i trom 1anuary 2, 190 to Lcccmcr 2o,
199 Lct x
t
dcnotc thc wccky 3-month Trcasury i lt is ottcn to
modc x
t
y assumin that it satiscs thc continuous-timc stochastic
dicrcntia cquation (Lack-Schocs modc)
d x
t
(x
t
) dt + (x
t
) dW
t
,
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 281
whcrc W
t
is a \icncr jroccss, (x
t
) is cacd thc dritt tunction and
(x
t
) is cacd thc diusion tunction Our intcrcst is to idcntity (x
t
)
and (x
t
) Assumc a timc scrics scqucncc {X
t
, 1 t n} is o-
scrvcd at equally spaced timc joints sin thc innitesimal
generator (Okscnda, 19S`), thc rst-ordcr ajjroximations ot mo-
mcnts ot x
t
, a discrctizcd vcrsion ot thc ltos jroccss, arc ivcn y
Stanton (199)
x
t
(x
t
) t + (x
t
)

,
whcrc x
t
x
t+
x
t
and N(0, 1) Thcrctorc,
(x
t
) im
0
E|x
t
| x
t
|/
and

2
(x
t
) im
0
E
_
(x
t
)
2
| x
t
_
/.
Scc Ian and Zhan (2003) tor thc hihcr ordcrs
8.5.3 Complexity of Local Polynomial Estimator
To imjcmcnt thc oca joynomia cstimator, wc havc to choosc thc
ordcr ot thc joynomia q, thc andwidth h and thc kcrnc tunc-
tion K() Thcsc jaramctcrs arc ot coursc contoundcd cach othcr
Ccary, whcn h , thc oca joynomia ttin ccomcs a oa
joynomia ttin and thc ordcr q dctcrmincs thc modc comjcxity
nikc in thc jaramctric modcs, thc comjcxity ot oca joynomia
ttin is jrimariy controcd y thc andwidth, as shown in Ian and
Gi,cs (199o) and Ian and Yao (2003) Lcncc q is usuay sma and
thc issuc ot choosin q ccomcs css critica \c discuss thosc issucs
in dctai as toows
(1) lt thc o,cctivc is to cstimatc m
(j)
() (j 0), thc oca joyno-
mia ttin corrccts automaticay thc oundary ias whcn q j is
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 282
is odd Iurthcr, whcn q j is odd, comjarin with thc ordcr q 1
t (so that q j 1 is cvcn), thc ordcr q t contains onc cxtra ja-
ramctcr without incrcasin thc variancc tor cstimatin m
(j)
() Lut
this cxtra jaramctcr crcatcs ojjortunitics tor ias rcduction, jartic-
uary in thc oundary rcions. scc thc ncxt scction and thc ooks
y Ian and Gi,cs (199o) and lujjcrt and \and(199!) Ior thcsc
rcasons, thc odd ordcr ts (thc ordcr q is choscn so that q j is
odd) outjcrtorms thc cvcn ordcr ts |thc ordcr (q 1) t so that q
is cvcn| Lascd on thcorctica and jractica considcrations, thc ordcr
q j +1 is rccommcndcd in Ian and Gi,cs (199o) lt thc jrimary
o,cctivc is to cstimatc thc rcrcssion tunction, onc uscs oca incar
t and it thc tarct tunction is thc rst ordcr dcrivativc, onc uscs thc
oca quadratic t and so on
(2) lt is wc known that thc choicc ot thc andwidth h jays an im-
jortant roc in thc oca joynomia ttin A too arc andwidth
causcs ovcr-smoothin, crcatin cxccssivc modcin ias, whic a too
sma andwidth rcsuts in undcr-smoothin, otainin wiy cs-
timatcs Thc andwidth can c su,cctivcy choscn y uscrs via
visuay insjcctin rcsutin cstimatcs, or automaticay choscn y
data via minimizin an cstimatcd thcorctica risk (discusscd atcr)
(3) Sincc thc cstimatc is ascd on thc oca rcrcssion (S3), it is
rcasonac to rcquirc a non-ncativc wciht tunction K() lt can c
shown (scc Ian and Gi,cs (199o)) that tor a choiccs ot q and j,
thc ojtima wciht tunction is K(z) 3/!(1 z
2
)
+
, thc Ljancch-
nikov kcrnc, ascd on minimizin thc asymjtotic variancc ot thc
oca joynomia cstimator Thus, it is a univcrsa wcihtin schcmc
and jrovidcs a usctu cnchmark tor othcr kcrncs to comjarc with
As shown in Ian and Gi,cs (199o) and Ian and Yao (2003), othcr
kcrncs havc ncary thc samc ccicncy tor jractica usc ot q and j
Lcncc thc choicc ot thc kcrnc tunction is not critica
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 283
Thc oca joynomia cstimator comjarcs tavoray with othcr cs-
timators, incudin thc `adaraya-\atson (oca constant) cstimator
and othcr incar cstimators such as thc Gasscr and ` ucr cstimator
ot Gasscr and ` ucr (199) and thc lricstcy and Chao cstimator
ot lricstcy and Chao (192) lndccd, it was shown y Ian (1993)
that thc oca incar ttin is asymjtoticay minimax ascd on thc
quadratic oss tunction amon a incar cstimators and is ncary min-
imax amon a jossic incar cstimators This minimax jrojcrty is
cxtcndcd y Ian, Gasscr, Gi,cs, Lrockmann and Lnc (199`) to
morc cncra oca joynomia ttin Ior thc dctaicd comjarisons
ot thc aovc tour cstimators, scc Ian and Gi,cs (199o)
`otc that thc Gasscr and ` ucr cstimator and thc lricstcy and
Chao cstimator arc jarticuary tor thc xcd dcsin That is, X
t
t
Lct s
t
(2t + 1)/2 (t 1, , n 1) with s
0
and s
n

Thc Gasscr and ` ucr cstimator is

f
gm
(t
0
)
n

t=1
_
s
t
s
t1
K
h
(u t
0
)duY
t
.
nikc thc oca constant cstimator, no dcnominator is nccdcd sincc
thc tota wciht
n

t=1
_
s
t
s
t1
K
h
(u t
0
)du 1.
lndccd, thc Gasscr and ` ucr cstimator is an imjrovcd vcrsion ot
thc lricstcy and Chao cstimator, which is dcncd as

f
pc
(t
0
)
n

t=1
K
h
(t t
0
) Y
t
.
`otc that thc lricstcy and Chao cstimator is ony ajjicac tor thc
cqui-sjacc scttin
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 284
8.5.4 Properties of Local Polynomial Estimator
Lcnc, tor 0 j q,
s
n,j
(x)
n

t=1
(X
t
x)
j
K
h
(X
t
x)
and S
n
(x) X
T
WX Thcn, thc (i + 1, j + 1)th ccmcnt ot S
n
(x)
is s
n,i+j
(x) Simiar to thc cvauation ot I
11
, wc can show casiy that
s
n,j
(x) nh
j

j
(K) f(x){1 + o
p
(1)}.
Lcnc, H dia{1, h, , h
q
} and S (
i+j
(K))
0i,jq
Thcn, it
is not dicut to show that S
n
(x) nf(x) H S H {1 + o
p
(1)}
Iirst ot a, tor 0 j q, ct e
j
c a (q + 1) 1 vcctor with
(j + 1)th ccmcnt cin onc and zcro othcrwisc Thcn,

j
can c
rc-cxjrcsscd as

j
e
T
j

t=1
W
j,n,h
(X
t
x) Y
t
,
whcrc W
j,n,h
(X
t
x) is cacd thc ccctivc kcrnc in Ian and Gi,cs
(199o) and Ian and Yao (2003), ivcn y
W
j,n,h
(X
t
x) e
T
j
S
n
(x)
1
(1, (X
t
x), , (X
t
x)
q
)
T
K
h
(X
t
x).
lt is not dicut to show that W
j,n,h
(X
t
x) satiscs thc toowin
thc so-cacd discrete moment conditions
n

t=1
(X
t
x)
l
W
j,n,h
(X
t
x)
_
_
_
1 it l j,
0 othcrwisc
(S`)
`otc that thc oca constant cstimator docs not havc this jrojcrty.
scc J
1
(x) in Scction S!1 This jrojcrty imjics that thc oca joy-
nomia cstimator is uniascd tor cstimatin
j
, whcn thc truc rcrcs-
sion tunction m(x) is a joynomia ot ordcr q
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 285
To ain morc insihts aout thc oca joynomia cstimator, dcnc
thc equivalent kernel (scc Ian and Gi,cs (199o))
W
j
(u) e
T
j
S
1
(1, u, , u
q
)
T
K(u).
Thcn, it can c shown (scc Ian and Gi,cs (199o)) that
W
j,n,h
(X
t
x)
1
nh
j+1
f(x)
W
j
((X
t
x)/h){1 + o
p
(1)}
and
_
u
l
W
j
(u)du
_
_
_
1 it l j,
0 othcrwisc
Thc imjications ot thcsc rcsuts arc as toows
As jointcd out y Ian and Yao (2003), thc oca joynomia csti-
mator works ikc a kcrnc rcrcssion cstimation with a known dcsin
dcnsity f(x) This cxjains why thc oca joynomia t adajts to
various dcsin dcnsitics ln contrast, thc kcrnc rcrcssion cstima-
tor has arc ias at thc rcion whcrc thc dcrivativc ot f(x) is arc,
namcy it can not adajt to hihy-skcwcd dcsins To scc that, ima-
inc thc truc rcrcssion tunction has arc sojc in this rcion Sincc
thc dcrivativc ot dcsin dcnsity is arc, tor a ivcn x, thcrc arc morc
joints on onc sidc ot x than thc othcr \hcn thc oca avcrac
is takcn, thc `adaraya-\atson cstimatc is iascd towards thc sidc
with morc oca data joints ccausc thc oca data arc asymmct-
ricay distriutcd This issuc is morc jronounccd at thc oundary
rcions, sincc thc oca data arc cvcn morc asymmctric On thc othcr
hand, thc oca joynomia t crcatcs asymmctric wcihts, it nccdcd,
to comjcnsatc tor this kind ot dcsin ias Lcncc, it is adajtivc to
various dcsin dcnsitics and to thc oundary rcions
\c ncxt dcrivc thc asymjtotic ias and variancc cxjrcssion tor
oca joynomia cstimators Ior indcjcndcnt data, wc can otain
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 286
thc ias and variancc cxjrcssion via conditionin on thc dcsin ma-
trix X Lowcvcr, tor timc scrics data, conditionin on X woud
mcan conditionin on ncary thc cntirc scrics Lcncc, wc dcrivc thc
asymjtotic ias and variancc usin thc asymjtotic normaity rathcr
than conditiona cxjcctation As cxjaincd in Chajtcr , ocaizin
in thc statc domain wcakcns thc dcjcndcnt structurc tor thc oca
data Lcncc, onc woud cxjcct that thc rcsut tor thc indcjcndcnt
data continucs to hod tor thc stationary jroccss with ccrtain mix-
in conditions Thc mixin condition and thc andwidth shoud c
rcatcd, which can c sccn atcr
Sct B
n
(x) (b
1
(x), , b
n
(x))
T
, whcrc, tor 0 j q,
b
j+1
(x)
n

t=1
_

_m(X
t
)
q

j=0
m
(j)
(x)
j'
(X
t
x)
j
_

_ (X
t
x)
j
K
h
(X
t
x).
Thcn,

_
X
T
WX
_
1
B
n
(x) +
_
X
T
WX
_
1
X
T
W,
whcrc (
1
, ,
n
)
T
lt is casy to show that it q is odd,
B
n
(x) nh
q+1
H f(x)
m
(q+1)
(x)
(q + 1)'
c
1,q
{1 + o
p
(1)},
whcrc, tor 1 k 3, c
k,q
(
q+k
(K), ,
2q+k
(K))
T
lt q is cvcn,
B
n
(x) nh
q+2
H f(x)
_

_c
2,q
m
(q+1)
(x)f

(x)
f(x)(q + 1)'
+ c
3,q
m
(q+2)
(x)
(q + 2)'
_

_ {1+o
p
(1)}.
`otc that f

(x)/f(x) docs not ajjcar in thc riht hand sidc ot B

n
(x)
whcn q is odd ln cithcr casc, wc can show that
\ar
_
H(

)
_

2
(x)S
1
S

S
1
/f(x) (x),
whcrc S

is a (q +1) (q +1) matrix with thc (i, j)th ccmcnt cin

i+j2
(K)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 287
This shows that thc cadin conditiona ias tcrm dcjcnds on
whcthcr q is odd or cvcn Ly a Tayor scrics cxjansion arumcnt, wc
know that whcn considcrin |X
t
x| < h, thc rcmaindcr tcrm trom
a qth ordcr joynomia cxjansion shoud c ot ordcr O(h
q+1
), so thc
rcsut tor odd q is quitc casy to undcrstand \hcn q is cvcn, (q +1)
is odd hcncc thc tcrm h
q+1
is associatcd with
_
u
l
K(u)du tor l odd,
and this tcrm is zcro ccausc K(u) is a cvcn tunction Thcrctorc, thc
h
q+1
tcrm disajjcars, whic thc rcmaindcr tcrm ccomcs O(h
q+2
)
Sincc q is cithcr odd or cvcn, thcn wc scc that thc ias tcrm is an
cvcn jowcr ot h This is simiar to thc casc whcrc onc uscs hihcr
ordcr kcrnc tunctions ascd ujon a symmctric kcrnc tunction (an
cvcn tunction), whcrc thc ias is aways an cvcn jowcr ot h
Iinay, wc can show that whcn q is odd,

nh
_
H(

) B(x)
_
N(0, (x)),
thc asymjtotic ias tcrm tor thc oca joynomia cstimator is
B(x)
h
q+1
(q + 1)'
m
(q+1)
(x) S
1
c
1,q
{1 + o
p
(1)}.
Or,

nh
2j+1
_

m
(j)
(x) m
(j)
(x) B
j
(x)
_
N(0,
jj
(x)),
whcrc thc asymjtotic ias and variancc tor thc oca joynomia cs-
timator ot m
(j)
(x) arc
B
j
(x)
j' h
q+1j
(q + 1)'
m
(q+1)
(x)
_
u
q+1
W
j
(u)du{1 + o
p
(1)}
and

jj
(x)
(j')
2

2
(x)
f(x)
_
W
2
j
(u)du.
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 288
Simiary, wc can dcrivc thc asymjtotic ias and variancc at ound-
ary joints it thc rcrcssion tunction has a nitc sujjort Ior dctais,
scc Ian and Gi,cs (199o), Ian and Yao (2003), and lujjcrt and
\and (199!) lndccd, dcnc S
c
, S

c
, and c
k,q,c
simiary to S, S

and
c
k,q
with
j
(K) and
j
(K) rcjaccd y
j,c
(K) and
j,c
(K) rcsjcc-
tivcy \c can show that

nh
_
H(

(ch) (ch)) B
c
(0)
_
N(0,
c
(0)), (So)
whcrc thc asymjtotic ias tcrm tor thc oca joynomia cstimator
at thc ctt oundary joint is
B
c
(0)
h
q+1
(q + 1)'
m
(q+1)
(0) S
1
c
c
1,q,c
{1 + o
p
(1)},
and thc asymjtotic variancc is
c
(0)
2
(0)S
1
c
S

c
S
1
c
/f(0) Or,

nh
2j+1
_

m
(j)
(ch) m
(j)
(ch) B
j,c
(0)
_
N(0,
jj,c
(0)),
whcrc with W
j,c
(u) e
T
j
S
1
c
(1, u, , u
q
)
T
K(u),
B
j,c
(0)
j' h
q+1j
(q + 1)'
m
(q+1)
(0)
_

c
u
q+1
W
j,c
(u)du{1 + o
p
(1)}
and

jj,c
(0)
(j')
2

2
(0)
f(0)
_

c
W
2
j,c
(u)du.
Exercise: lcasc dcrivc thc asymjtotic jrojcrtics tor thc oca joy-
nomia cstimator That is to jrovc (So)
Thc aovc concusions show that whcn q j is odd, thc ias at
thc oundary is ot thc samc ordcr as that tor joints on thc intcrior
Lcncc, thc oca joynomia t docs not crcatc cxccssivc oundary
ias whcn q j is odd Thus, thc ajjcain oundary chavior ot
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 289
oca joynomia mcan cstimation cxtcnds to dcrivativc cstimation
Lowcvcr, whcn q j is cvcn, thc ias at thc oundary is arcr than
in thc intcrior, and thc ias can aso c arc at joints whcrc f(x)
is discontinuous This is rctcrrcd to as oundary ccct Ior thcsc
rcasons (and thc minimax ccicncy arumcnts), it is rccommcndcd
that onc stricty sct q j to c odd whcn cstimatin m
(j)
(x) lt is
indccd an odd word'
8.5.5 Bandwidth Selection
As sccn in jrcvious scctions, tor stationary scqucnccs ot data undcr
ccrtain mixin conditions, thc oca joynomia cstimator jcrtorms
vcry much ikc that tor indcjcndcnt data, ccausc ot windowin rc-
duccs dcjcndcncy amon oca data lartiay ccausc ot this, thcrc
arc not many studics on andwidth sccction tor thcsc jrocms
Lowcvcr, it is rcasonac to cxjcct thc andwidth sccctors tor in-
dcjcndcnt data continuc to work tor dcjcndcnt data with ccrtain
mixin conditions Lcow, wc summarizc a tcw ot usctu ajjroachcs
\hcn data do not havc stron cnouh mixin, thc cncra stratcy
is to incrcasc andwidth in ordcr to rcducc thc variancc
As what wc had arcady sccn tor thc nonjaramctric dcnsity csti-
mation, thc cross-vaidation mcthod is vcry usctu tor asscssin thc
jcrtormancc ot an cstimator via cstimatin its jrcdiction crror Thc
asic idca is to sct onc ot thc data joint asidc tor vaidation ot a
modc and usc thc rcmainin data to uid thc modc lt is dcncd
as
C\(h)
n

s=1
|Y
s

m
s
(X
s
)|
2
whcrc

m
s
(X
s
) is thc oca joynomia cstimator with j 0 and
andwidth h, ut without usin thc sth oscrvation Thc aovc sum-
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 290
mand is indccd a squarcd-jrcdiction crror ot thc sth data joint usin
thc trainin sct {(X
t
, Y
t
) t s} This idca ot thc cross-vaidation
mcthod is simjc ut is comjutationay intcnsivc An imjrovcd
vcrsion, in tcrms ot comjutation, is thc cncraizcd cross-vaidation
(GC\), jrojoscd y \aha (19) and Cravcn and \aha (199)
This critcrion can c dcscricd as toows Thc ttcd vaucs

Y
(

m(X
1
), ,

m(X
n
))
T
can c cxjrcsscd as

Y H(h)Y , whcrc H(h)
is an n n hat matrix, dcjcndin on thc X-variatc and andwidth
h, and it is aso cacd a smoothin matrix Thcn thc GC\ ajjroach
scccts thc andwidth h that minimizcs
GC\(h)
_
n
1
tr(I H(h))
_
2
`ASL(h)
whcrc `ASL(h)

n
t=1
(Y
t

m(X
t
))
2
/n is thc avcrac ot squarcd
rcsiduas
A drawack ot thc cross-vaidation tyjc mcthod is its inhcritcd
variaiity (scc La and 1ohnstonc, 1992) Iurthcr, it can not c di-
rccty ajjicd to sccct andwidths tor cstimatin dcrivativc curvcs
As jointcd out y Ian, Lcckman, and \and (199`), thc cross-
vaidation tyjc mcthod jcrtorms joory duc to its arc samjc vari-
ation, cvcn worsc tor dcjcndcnt data lu-in mcthods avoid thcsc
jrocms Thc asic idca is to nd a andwidth h minimizin csti-
matcd mcan intcratcd squarc crror (`lSL) Scc lujjcrt, Shcathcr
and \and (199`) and Ian and Gi,cs (199`) tor dctais
Nonparametric AIC Selector
lnsjircd y thc nonjaramctric vcrsion ot thc Akaikc na jrcdiction
crror critcrion jrojoscd y T,sthcim and Aucstad (199!) tor thc
a sccction in nonjaramctric scttin, Cai (2002) jrojoscd a simjc
and quick mcthod to sccct andwidth tor thc torcoin cstimation
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 291
jroccdurcs, which can c rcardcd as a nonjaramctric vcrsion ot thc
Akaikc intormation critcrion (AlC) to c attcntivc to thc structurc ot
timc scrics data and thc ovcr-ttin or undcr-ttin tcndcncy `otc
that thc idca is aso motivatcd y its anaouc ot Cai and Tiwari
(2000) Thc asic idca is dcscricd as toows
Ly rccain thc cassica AlC tor incar modcs undcr thc ikcihood
scttin
2 (maximizcd o ikcihood)+2 (numcr ot cstimatcd jaramctcrs),
Cai (2002) jrojoscd thc toowin nonjaramctric AlC to sccct h
minimizin
AlC(h) o {`ASL} + (tr(S
h
), n), (S)
whcrc (tr(S
h
), n) is choscn jarticuary to c thc torm ot thc ias-
corrcctcd vcrsion ot thc AlC, duc to Lurvich and Tsai (19S9),
(tr(H(h)), n) 2 {tr(H(h)) + 1}/|n {tr(H(h)) + 2}|, (SS)
and tr(H(h)) is thc tracc ot thc smoothin matrix H(h), rcardcd
as thc nonjaramctric vcrsion ot dcrccs ot trccdom, cacd thc ccc-
tivc numcr ot jaramctcrs Scc thc ook y Lastic and Tishirani
(1990, Scction 3`) tor thc dctaicd discussion on this asjcct tor non-
jaramctric modcs `otc that actuay, (S) is a cncraization ot
thc AlC tor thc jaramctric rcrcssion and autorcrcssivc timc sc-
rics contcxts, in which tr(H(h)) is thc numcr ot rcrcssion (autorc-
rcssivc) jaramctcrs in thc ttin modc ln vicw ot (SS), whcn
(tr(H(h)), n) 2 o(1 tr(H(h))/n), thcn (S) ccomcs thc
cncraizcd cross-vaidation (GC\) critcrion, commony uscd to sc-
cct thc andwidth in thc timc scrics itcraturc cvcn in thc iid scttin,
whcn (tr(H(h)), n) 2 tr(H(h))/n, thcn (S) is thc cassica AlC
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 292
discusscd in Lnc, Grancr, licc, and \ciss (19So) tor timc scrics
data, and whcn (tr(H(h)), n) o(1 2 tr(H(h))/n), (S) is
thc T-critcrion, jrojoscd and studicd y licc (19S!) tor iid samjcs
lt is ccar that whcn tr(H(h))/n 0, thcn thc nonjaramctric AlC,
thc GC\ and thc T-critcrion arc asymjtoticay cquivacnt Lowcvcr,
thc T-critcrion rcquircs tr(H(h))/n < 1/2, and, whcn tr(H(h))/n
is arc, thc GC\ has rcativcy wcak jcnaty This is csjcciay truc
tor thc nonjaramctric scttin Thcrctorc, thc critcrion jrojoscd hcrc
countcracts thc ovcr-ttin tcndcncy ot thc GC\ `otc that Lurvich,
Simono, and Tsai (199S) avc thc dctaicd dcrivation ot thc non-
jaramctric AlC tor thc nonjaramctric rcrcssion jrocms undcr thc
iid Gaussian crror scttin and thcy arucd that thc nonjaramctric
AlC jcrtorms rcasonay wc and cttcr than somc cxistin mcthods
in thc itcraturc
8.6 Functional Coecient Model
8.6.1 Model
As mcntioncd caricr, whcn p is arc, thcrc cxists thc so cacd cursc
ot dimcnsionaity To ovcrcomc this shortcomin, onc way to do so is
to considcr thc tunctiona coccicnt modc as studicd in Cai, Ian and
Yao (2000)c and thc additivc modc discusscd in Scction S Iirst,
wc study thc tunctiona coccicnt modc To usc thc notation trom
Cai, Ian and Yao (2000), wc chanc thc notation trom thc jrcvious
scctions
Lct {U
i
, X
i
, Y
i
}

i=
c ,ointy stricty stationary jroccsscs with
U
i
takin vaucs in
k
and X
i
takin vaucs in
p
Tyjicay, k
is sma Lct E(Y
2
1
) < \c dcnc thc mutivariatc rcrcssion
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 293
tunction
m(u, x) E (Y | U u, X x) , (S9)
whcrc (U, X, Y ) has thc samc distriution as (U
i
, X
i
, Y
i
) ln a
jurc timc scrics contcxt, oth U
i
and X
i
consist ot somc acd
vaucs ot Y
i
Thc tunctiona-coccicnt rcrcssion modc has thc torm
m(u, x)
p

j=1
a
j
(u) x
j
, (S10)
whcrc thc tunctions {a
j
()} arc mcasurac trom
k
to
1
and x
(x
1
, . . . , x
p
)
T
This modc has ccn studicd cxtcnsivcy in thc it-
craturc. scc Cai, Ian and Yao (2000) tor thc dctaicd discussions
Ior simjicity, in what toows, wc considcr ony thc casc k 1
in (S10) Lxtcnsion to thc casc k > 1 invovcs no tundamcntay
ncw idcas `otc that modcs with arc k arc ottcn not jracticay
usctu duc to thc cursc ot dimcnsionaity lt k is arc, to ovcrcomc
thc jrocm, onc way to do so is to considcr an indcx tunctiona
coccicnt modc jrojoscd y Ian, Yao and Cai (2003)
m(u, x)
p

j=1
a
j
(
T
u) x
j
, (S11)
whcrc
1
1 Ian, Yao and Cai (2003) studicd thc cstimation
jroccdurcs, andwidth sccction and ajjications Lon and Lcc
(2003) considcrcd thc ajjications ot modc (S11) to thc cxchanc
ratcs
8.6.2 Local Linear Estimation
As rccommcndcd y Ian and Gi,cs (199o), wc cstimatc thc co-
ccicnt tunctions {a
j
()} usin thc oca incar rcrcssion mcthod
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 294
trom oscrvations {U
i
, X
i
, Y
i
}
n
i=1
, whcrc X
i
(X
i1
, . . . , X
ip
)
T
\c
assumc throuhout that a
j
() has a continuous sccond dcrivativc
`otc that wc may ajjroximatc a
j
() ocay at u
0
y a incar tunc-
tion a
j
(u) a
j
+b
j
(uu
0
). Thc oca incar cstimator is dcncd as

a
j
(u
0
)

a
j
, whcrc {(

a
j
,

b
j
)} minimizc thc sum ot wcihtcd squarcs
n

i=1
_

_Y
i

j=1
{a
j
+ b
j
(U
i
u
0
)} X
ij
_

_
2
K
h
(U
i
u
0
), (S12)
whcrc K
h
() h
1
K(/h), K() is a kcrnc tunction on
1
and h > 0
is a andwidth lt toows trom thc cast squarcs thcory that

a
j
(u
0
)
n

k=1
K
n,j
(U
k
u
0
, X
k
) Y
k
, (S13)
whcrc
K
n,j
(u, x) e
T
j,2p
_

X
T
W

X
_
1
_
_
x
ux
_
_
K
h
(u) (S1!)
e
j,2p
is thc 2p 1 unit vcctor with 1 at thc jth josition,

X dcnotcs
an n 2p matrix with (X
T
i
, X
T
i
(U
i
u
0
)) as its ith row, and W
dia {K
h
(U
1
u
0
), . . . , K
h
(U
n
u
0
)}
8.6.3 Bandwidth Selection
\arious cxistin andwidth sccction tcchniqucs tor nonjaramctric
rcrcssion can c adajtcd tor thc torcoin cstimation. scc, e.g.,
Ian, Yao, and Cai (2003) and thc nonjaramctric AlC as discusscd
in Scction S`` Aso, Ian and Gi,cs (199o) and lujjcrt, Shcathcr,
and \and (199`) dcvcojcd data-drivcn andwidth sccction schcmcs
ascd on asymjtotic tormuas tor thc ojtima andwidths, which arc
css variac and morc ccctivc than thc convcntiona data-drivcn
andwidth sccctors such as thc cross-vaidation andwidth ruc
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 295
Simiar aorithms can c dcvcojcd tor thc cstimation ot tunctiona-
coccicnt modcs ascd on (S2!). howcvcr, this wi c a tuturc
rcscarch tojic
Cai, Ian and Yao (2000) jrojoscd a simjc and quick mcthod tor
sccctin andwidth h lt can c rcardcd as a modicd muti-tod
cross-vaidation critcrion that is attcntivc to thc structurc ot station-
ary timc scrics data Lct m and Q c two ivcn jositivc intccrs
and n > mQ Thc asic idca is rst to usc Q suscrics ot cnths
nqm (q 1, , , Q) to cstimatc thc unknown coccicnt tunc-
tions and thcn comjutc thc onc-stcj torccastin crrors ot thc ncxt
scction ot thc timc scrics ot cnth m ascd on thc cstimatcd mod-
cs `orc jrcciscy, wc choosc h that minimizcs thc avcrac mcan
squarcd (A`S) crror
A`S(h)
Q

q=1
A`S
q
(h), (S1`)
whcrc tor q 1, , Q,
A`S
q
(h)
1
m
nqm+m

i=nqm+1
_

_
Y
i

j=1

a
j,q
(U
i
)X
i,j
_

_
2
,
and {

a
j,q
()} arc comjutcd trom thc samjc {(U
i
, X
i
, Y
i
), 1 i
n qm} with andwidth cqua h|n/(n qm)|
1/5
`otc that wc rc-
scac andwidth h tor dicrcnt samjc sizcs accordin to its ojtima
ratc, ic h n
1/5
ln jractica imjcmcntations, wc may usc
m |0.1n| and Q ! Thc sccctcd andwidth docs not dcjcnd
criticay on thc choicc ot m and Q, as on as mQ is rcasonay
arc so that thc cvauation ot jrcdiction crrors is stac A wcihtcd
vcrsion ot A`S(h) can c uscd, it onc wishcs to down-wciht thc
jrcdiction crrors at an caricr timc \c cicvc that this andwidth
shoud c ood tor modcin and torccastin tor timc scrics
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 296
8.6.4 Smoothing Variable Selection
Ot imjortancc is to choosc an ajjrojriatc smoothin variac U
in ajjyin tunctiona-coccicnt rcrcssion modcs it U is a acd
variac Inowcdc on jhysica ackround ot thc data may c vcry
hcjtu, as Cai, Ian and Yao (2000) discusscd in modcin thc ynx
data \ithout any jrior intormation, it is jcrtincnt to choosc U in
tcrms ot somc data-drivcn mcthods such as thc Akaikc intormation
critcrion (AlC) and its variants, cross-vaidation, and othcr critcria
ldcay, wc woud choosc U as a incar tunction ot ivcn cxjanatory
variacs accordin to somc ojtima critcrion, which can c tuy
cxjorcd in thc work y Ian, Yao and Cai (2003) `cvcrthccss, wc
jrojosc hcrc a simjc and jractica ajjroach ct U c onc ot thc
ivcn cxjanatory variacs such that A`S dcncd in (S1`) otains
its minimum vauc Oviousy, this idca can c aso cxtcndcd to
sccct p (numcr ot as) as wc
8.6.5 Goodness-of-Fit Test
To tcst whcthcr modc (S10) hods with a sjccicd jaramctric torm
which is jojuar in cconomic and nancia ajjications, such as thc
thrcshod autorcrcssivc (TAl) modcs
a
j
(u)
_
_
_
a
j1
, it u
a
j2
, it u > ,
or cncraizcd cxjoncntia autorcrcssivc (LXlAl) modcs
a
j
(u)
j
+ (
j
+
j
u) cxj(
j
u
2
),
or smooth transition autorcrcssivc (STAl) modcs
a
j
(u) |1 cxj(
j
u)|
1
(oistic),
or
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 297
a
j
(u) 1 cxj(
j
u
2
) (cxjoncntia),
or
a
j
(u) |1 cxj(
j
|u|)|
1
(asoutc),
|tor morc discussions on thosc modcs, jcasc scc thc survcy jajcrs y
van Li,k, Tcrasvirta and Iranscs (2002)|, wc jrojosc a oodncss-ot-
t tcst ascd on thc comjarison ot thc rcsidua sum ot squarcs (lSS)
trom oth jaramctric and nonjaramctric ttins This mcthod is
coscy rcatcd to thc sicvc ikcihood mcthod jrojoscd y Ian, Zhan
and Zhan (2001) Thosc authors dcmonstratcd thc ojtimaity ot
this kind ot jroccdurcs tor indcjcndcnt samjcs
Considcr thc nu hyjothcsis
H
0
a
j
(u)
j
(u, ), 1 j p, (S1o)
whcrc
j
(, ) is a ivcn tamiy ot tunctions indcxcd y unknown
jaramctcr vcctor Lct

c an cstimator ot Thc lSS undcr thc
nu hyjothcsis is
lSS
0
n
1
n

i=1
_
Y
i

1
(U
i
,

)X
i1

p
(U
i
,

)X
ip
_
2
.
Anaoousy, thc lSS corrcsjondin to modc (S10) is
lSS
1
n
1
n

i=1
{Y
i

a
1
(U
i
)X
i1


a
p
(U
i
)X
ip
}
2
.
Thc tcst statistic is dcncd as
T
n
(lSS
0
lSS
1
)/lSS
1
lSS
0
/lSS
1
1,
and wc rc,cct thc nu hyjothcsis (S1o) tor arc vauc ot T
n
\c
usc thc toowin nonjaramctric ootstraj ajjroach to cvauatc thc
p vauc ot thc tcst
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 298
1 Gcncratc thc ootstraj rcsiduas {

i
}
n
i=1
trom thc cmjirica dis-
triution ot thc ccntcrcd rcsiduas {

}
n
i=1
, whcrc

i
Y
i

a
1
(U
i
) X
i1


a
p
(U
i
) X
ip
,


1
n
n

i=1

i
,
and dcnc
Y

i

1
(U
i
,

)X
i1
+ +
p
(U
i
,

)X
ip
+

i
.
2 Cacuatc thc ootstraj tcst statistic T

n
ascd on thc samjc
{U
i
, X
i
, Y

i
}
n
i=1

3 lc,cct thc nu hyjothcsis H

0
whcn T
n
is rcatcr than thc ujjcr-
joint ot thc conditiona distriution ot T

n
ivcn {U
i
, X
i
, Y
i
}
n
i=1

Thc p-vauc ot thc tcst is simjy thc rcativc trcqucncy ot thc cvcnt
{T

n
T
n
} in thc rcjications ot thc ootstraj samjin Ior thc
sakc ot simjicity, wc usc thc samc andwidth in cacuatin T

n
as
that in T
n
`otc that wc ootstraj thc ccntraizcd rcsiduas trom thc
nonjaramctric t instcad ot thc jaramctric t, ccausc thc nonjara-
mctric cstimatc ot rcsiduas is aways consistcnt, no mattcr whcthcr
thc nu or thc atcrnativc hyjothcsis is corrcct Thc mcthod shoud
jrovidc a consistcnt cstimator ot thc nu distriution cvcn whcn
thc nu hyjothcsis docs not hod Irciss, `cumann, and Yao (199S)
considcrcd nonjaramctric ootstraj tcsts in a cncra nonjaramctric
rcrcssion scttin Thcy jrovcd that, asymjtoticay, thc conditiona
distriution ot thc ootstraj tcst statistic is indccd thc distriution
ot thc tcst statistic undcr thc nu hyjothcsis lt may c jrovcn that
thc simiar rcsut hods hcrc as on as

convcrcs to at thc ratc
n
1/2

lt is a rcat chacnc to dcrivc thc asymjtotic jrojcrty ot thc tcst-

in statistics T
n
undcr timc scrics contcxt and cncra assumjtions
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 299
That is to show that
b
n
|T
n

n
| N(0,
2
)
tor somc b
n
and
n
, which is a rcat jro,cct tor tuturc rcscarch `otc
that Ian, Zhan and Zhan (2001) dcrivcd thc aovc rcsut tor thc
iid samjc
8.6.6 Asymptotic Results
\c rst jrcscnt a rcsut on mcan squarcd convcrcncc that scrvcs
as a uidin ock tor our main rcsut and is aso ot indcjcndcnt
intcrcst \c now introducc somc notation Lct
S
n
S
n
(u
0
)
_
_
S
n,0
S
n,1
S
n,1
S
n,2
_
_
and
T
n
T
n
(u
0
)
_
_
T
n,0
(u
0
)
T
n,1
(u
0
)
_
_
with
S
n,j
S
n,j
(u
0
)
1
n
n

i=1
X
i
X
T
i
_
_
U
i
u
0
h
_
_
j
K
h
(U
i
u
0
)
and
T
n,j
(u
0
)
1
n
n

i=1
X
i
_
_
U
i
u
0
h
_
_
j
K
h
(U
i
u
0
) Y
i
. (S1)
Thcn, thc soution to (S12) can c cxjrcsscd as

H
1
S
1
n
T
n
, (S1S)
whcrc H dia (1, . . . , 1, h, . . . , h) with p-diaona ccmcnts 1s
and p diaona ccmcnts hs To taciitatc thc notation, wc dcnotc
(
l,m
)
pp
E
_
XX
T
| U u
0
_
. (S19)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 300
Aso, ct f(u, x) dcnotc thc ,oint dcnsity ot (U, X) and f
u
(u) c thc
marina dcnsity ot U \c usc thc toowin convcntion it U X
j
0
tor somc 1 j
0
p, thcn f(u, x) ccomcs f(x) thc ,oint dcnsity
ot X
Theorem 1. Lct condition A1 in hod, and ct f(u, x) c con-
tinuous at thc joint u
0
Lct h
n
0 and nh
n
, as n
Thcn it hods that
E(S
n,j
(u
0
)) f
u
(u
0
) (u
0
)
j
,
and
nh
n
\ar(S
n,j
(u
0
)
l,m
) f
u
(u
0
)
2j

l,m
tor cach 0 j 3 and 1 l, m p
As a conscqucncc ot Thcorcm 1, wc havc
S
n
P
f
u
(u
0
) S, and S
n,3
P

3
f
u
(u
0
)
in thc scnsc that cach ccmcnt convcrcs in jroaiity, whcrc
S
_
_

1

1

2

_
_
.
lut

2
(u, x) \ar(Y | U u, X x) (S20)
and

(u
0
) E
_
XX
T

2
(U, X) | U u
0
_
. (S21)
Lct c
0

2
/
_

2
1
_
and c
1

1
/
_

2
1
_

Theorem 2 Lct
2
(u, x) and f(u, x) c continuous at thc joint
u
0
Thcn undcr conditions A1 and A2,

nh
n
_

a(u
0
) a(u
0
)
h
2
2

2
2

2
1
a

(u
0
)
_

_
D
N
_
0,
2
(u
0
)
_
,
(S22)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 301
jrovidcd that f
u
(u
0
) 0, whcrc

2
(u
0
)
c
2
0

0
+ 2 c
0
c
1

1
+ c
2
1

2
f
u
(u
0
)

1
(u
0
)

(u
0
)
1
(u
0
).
(S23)
Thcorcm 2 indicatcs that thc asymjtotic ias ot

a
j
(u
0
) is
h
2
2

2
2

2
1
a

j
(u
0
)
and thc asymjtotic variancc is (nh
n
)
1

2
j
(u
0
), whcrc

2
j
(u
0
)
c
2
0

0
+ 2 c
0
c
1

1
+ c
2
1

2
f
u
(u
0
)
e
T
j,p

1
(u
0
)

(u
0
)
1
(u
0
) e
j,p
.
\hcn
1
0, thc ias and variancc cxjrcssions can c simjicd as
h
2

2
a

j
(u
0
)/2 and

2
j
(u
0
)

0
f
u
(u
0
)
e
T
j,p

1
(u
0
)

(u
0
)
1
(u
0
) e
j,p
.
Thc ojtima andwidth tor cstimatin a
j
() can c dcncd to c
thc onc that minimizcs thc squarcd ias jus variancc Thc ojtima
andwidth is ivcn y
h
j,ojt

_

2
2

0
2
1

1
+
2
1

2
f
u
(u
0
) (
2
2

3
)
2
e
T
j,p

1
(u
0
)

(u
0
)
1
(u
0
) e
j,p
_
a

j
(u
0
)
_
2
_

_
1/5
n
1/
(S2!)
8.6.7 Conditions and Proofs
\c rst imjosc somc conditions on thc rcrcssion modc ut thcy
miht not c thc wcakcst jossic
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 302
Condition A.1
a Thc kcrnc tunction K() is a oundcd dcnsity with a oundcd
sujjort |1, 1|
|f(u, v | x
0
, x
1
. l)| M < , tor a l 1, whcrc f(u, v, | x
0
, x
1
. l)
is thc conditiona dcnsity ot (U
0
, U
l
)) ivcn (X
0
, X
l
), and f(u| x)
M < , whcrc f(u| x) is thc conditiona dcnsity ot U ivcn
X x
c Thc jroccss {U
i
, X
i
, Y
i
} is -mixin with

k
c
|(k)|
12/
<
tor somc > 2 and c > 1 2/
d E|X|
2
< , whcrc is ivcn in condition A1c
Condition A.2
a Assumc that
E
_
Y
2
0
+ Y
2
l
| U
0
u, X
0
x
0
. U
l
v, X
l
x
1
_
M < ,
(S2`)
tor a l 1, x
0
, x
1

p
, u, and v in a ncihorhood ot u
0

Assumc that h
n
and nh
n
Iurthcr, assumc that thcrc
cxists a scqucncc ot jositivc intccrs s
n
such that s
n
,
s
n
o
_
(nh
n
)
1/2
_
, and (n/h
n
)
1/2
(s
n
) 0, as n
c Thcrc cxists

> , whcrc is ivcn in Condition A1c, such

that
E
_
|Y |

|U u, X x
_
M
4
< (S2o)
tor a x
p
and u in a ncihorhood ot u
0
, and
(n) O
_
n

_
, (S2)
whcrc

/{2(

)}
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 303
d E|X|
2

< , and n
1/2/4
h
/

1/2/4
O(1)
Remark A.1. \c jrovidc a sucicnt condition tor thc mixin
coccicnt (n) to satisty conditions A1c and A2 Sujjosc that
h
n
An

(0 < < 1, A > 0), s

n
(nh
n
/ o n)
1/2
and
(n) O
_
n
d
_
tor somc d > 0 Thcn condition A1c is satiscd
tor d > 2(1 1/)/(1 2/) and condition A2 is satiscd it d >
(1 + )/(1 ) Lcncc oth conditions arc satiscd it
(n) O
_
n
d
_
, d > max
_

_
1 +
1
,
2(1 1/)
1 2/
_

_
.
`otc that this is a tradc-o ctwccn thc ordcr ot thc momcnt ot Y
and thc ratc ot dccay ot thc mixin coccicnt. thc arcr thc ordcr
, thc wcakcr thc dccay ratc ot (n)
To study thc ,oint asymjtotic normaity ot

a(u
0
), wc nccd to ccn-
tcr thc vcctor T
n
(u
0
) y rcjacin Y
i
with Y
i
m(U
i
, X
i
) in thc
cxjrcssion (S1) ot T
n,j
(u
0
) Lct
T

n,j
(u
0
)
1
n
n

i=1
X
i
_
_
U
i
u
0
h
_
_
j
K
h
(U
i
u
0
) |Y
i
m(U
i
, X
i
)|,
and
T

n

_
_
T

n,0
T

n,1
_
_
.
Lccausc thc coccicnt tunctions a
j
(u) arc conductcd in thc ncih-
orhood ot |U
i
u
0
| < h, y Tayors cxjansion,
m(U
i
, X
i
) X
T
i
a(u
0
)+(U
i
u
0
) X
T
i
a

(u
0
)+
h
2
2
_
_
U
i
u
0
h
_
_
2
X
T
i
a

(u
0
)+o
p
(h
2
)
whcrc a

(u
0
) and a

(u
0
) arc thc vcctors consistin ot thc rst and
sccond dcrivativcs ot thc tunctions a
j
() Thcn,
T
n,0
T

n,0
S
n,0
a(u
0
) + hS
n,1
a

(u
0
) +
h
2
2
S
n,2
a

(u
0
) + o
p
(h
2
)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 304
and
T
n,1
T

n,1
S
n,1
a(u
0
) + hS
n,2
a

(u
0
) +
h
2
2
S
n,3
a

(u
0
) + o
p
(h
2
),
so that
T
n
T

n
S
n
H +
h
2
2
_
_
S
n,2
S
n,3
_
_
a

(u
0
) + o
p
(h
2
), (S2S)
whcrc (a(u
0
)
T
, a

(u
0
)
T
)
T
Thus it toows trom (S1S), (S2S),
and Thcorcm 1 that
H
_

_
f
1
u
(u
0
) S
1
T

n
+
h
2
2
S
1
_
_

_
_
a

(u
0
) + o
p
(h
2
),
(S29)
trom which thc ias tcrm ot

(u
0
) is cvidcnt Ccary,

a(u
0
)a(u
0
)

1
f
u
(u
0
) (
2

2
1
)
_

2
T

n,0

1
T

n,1
_
+
h
2
2

2
2

2
1
a

(u
0
)+o
(S30)
Thus (S30) indicatcs that thc asymjtotic ias ot

a(u
0
) is
h
2
2

2
2

2
1
a

(u
0
).
Lct
Q
n

1
n
n

i=1
Z
i
, (S31)
whcrc
Z
i
X
i
_
_
c
0
+ c
1
_
_
U
i
u
0
h
_
_
_
_
K
h
(U
i
u
0
) |Y
i
m(U
i
, X
i
)| (S32)
with c
0

2
/
_

2
1
_
and c
1

1
/
_

2
1
_
lt toows trom
(S30) and (S31) that

nh
n
_

a(u
0
) a(u
0
)
h
2
2

2
2

2
1
a

(u
0
)
_

_

1
f
u
(u
0
)

nh
n
Q
n
+o
p
(1).
(S33)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 305
\c nccd thc toowin cmma, whosc jroot is morc invovcd than
that tor Thcorcm 1 Thcrctorc, wc jrovc ony this cmma Throuh-
out this Ajjcndix, wc ct C dcnotc a cncric constant, which may
takc dicrcnt vaucs at dicrcnt jaccs
Lemma A.1. ndcr conditions A1 and A2 and thc assumjtion
that h
n
0 and nh
n
, as n , it
2
(u, x) and f(u, x)
arc continuous at thc joint u
0
, thcn wc havc
(a) h
n
\ar(Z
1
) f
u
(u
0
)

(u
0
)
_
c
2
0

0
+ 2 c
0
c
1

1
+ c
2
1

2
_
.
() h
n
n1|
l=1
|Cov(Z
1
, Z
l+1
)| o(1). and
(c) nh
n
\ar(Q
n
) f
u
(u
0
)

(u
0
)
_
c
2
0

0
+ 2 c
0
c
1

1
+ c
2
1

2
_

Proof Iirst, y conditionin on (U

1
, X
1
) and usin Thcorcm 1 ot
Sun (19S!), wc havc
\ar(Z
1
) E
_

_X
1
X
T
1

2
(U
1
, X
1
)
_
_
_
c
0
+ c
1
_
_
U
1
u
0
h
_
_
_
_
_
2
K
2
h
(U
1
u
0
)
_

1
h
_
f
u
(u
0
)

(u
0
)
_
c
2
0

0
+ 2 c
0
c
1

1
+ c
2
1

2
_
+ o(1)
_
. (S3!)
Thc rcsut (c) toows in an ovious manncr trom (a) and () aon
with
\ar(Q
n
)
1
n
\ar(Z
1
) +
2
n
n1

l=1
_
_
1
l
n
_
_
Cov(Z
1
, Z
l+1
). (S3`)
lt thus rcmains to jrovc jart () To this cnd, ct d
n
c a
scqucncc ot jositivc intccrs such that d
n
h
n
0 Lcnc
J
1

d
n
1

l=1
|Cov(Z
1
, Z
l+1
)|
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 306
and
J
2

n1

l=d
n
|Cov(Z
1
, Z
l+1
)|.
lt rcmains to show that J
1
o
_
h
1
_
and J
2
o
_
h
1
_

\c rcmark that ccausc K() has a oundcd sujjort |1, 1|,

a
j
(u) is oundcd in thc ncihorhood ot u |u
0
h, u
0
+ h|
Lct B max
1jp
suj
|uu
0
|<h
|a
j
(u)| and g(x)

p
j=1
|x
j
| Thcn
suj
|uu
0
|<h
|m(u, x)| Bg(x) Ly conditionin on (U
1
, X
1
) and
(U
l+1
, X
l+1
), and usin (S2`) and condition A1, wc havc, tor a
l 1,
|Cov(Z
1
, Z
l+1
)|
C E
_
|X
1
X
T
l+1
| {|Y
1
| + Bg(X
1
)}{|Y
l+1
| + Bg(X
l+1
)} K
h
(U
1
u
0
) K
h
(
C E
_
|X
1
X
T
l+1
|
_
M
2
+ B
2
g
2
(X
1
)
_
1/2
_
M
2
+ B
2
g
2
(X
l+1
)
_
1/2
K
h
(U
1
u
0
)
C E
_
|X
1
X
T
l+1
| {1 + g(X
1
)} {1 + g(X
l+1
)}
_
C.
lt toows that
J
1
C d
n
o
_
h
1
_
y thc choicc ot d
n
\c ncxt considcr thc ujjcr ound ot J
2
To this
cnd, usin Lavydovs incquaity (scc La and Lcydc 19S0, Coroary
A2), wc otain, tor a 1 j, m p and l 1,
|Cov(Z
1j
, Z
l+1,m
)| C |(l)|
12/
_
E|Z
j
|

_
1/
_
E|Z
m
|

_
1/
. (S3)
Ly conditionin on (U, X) and usin conditions A1 and A2c, onc
has
E
_
|Z
j
|

_
C E
_
|X
j
|

h
(U u
0
)
_
|Y |

+ B

(X)
__
C E
_
|X
j
|

h
(U u
0
)
_
M
3
+ B

(X)
__
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 307
C h
1
E
_
|X
j
|

_
M
3
+ B

(X)
__
C h
1
. (S3S)
A comination ot (S3) and (S3S) cads to
J
2
C h
2/2

l=d
n
|(l)|
12/
C h
2/2
d
c
n

l=d
n
l
c
|(l)|
12/
o
_
h
1
_
(S39)
y choosin d
n
such that h
12/
d
c
n
C, so thc rcquircmcnt that
d
n
h
n
0 is satiscd
Proof of Theorem 2
\c usc thc sma-ock and arc-ock tcchniquc namcy, jartition
{1, . . . , n} into 2 q
n
+ 1 suscts with arc ock ot sizc r r
n
and
sma ock ot sizc s s
n
Sct
q q
n

_
n
r
n
+ s
n

_
. (S!0)
\c now usc thc Cramcr-\od dcvicc to dcrivc thc asymjtotic nor-
maity ot Q
n
Ior any unit vcctor d
p
, ct Z
n,i

hd
T
Z
i+1
,
i 0, . . . , n 1 Thcn

nh d
T
Q
n

1

n
n1

i=0
Z
n,i
,
and, y Lcmma A1,
\ar(Z
n,0
) f
u
(u
0
) d
T

(u
0
) d
_
c
2
0

0
+ 2 c
0
c
1

1
+ c
2
1

2
_

2
(u
0
) (S!1)
and
n1

l=0
|Cov(Z
n,0
, Z
n,l
)| o(1). (S!2)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 308
Lcnc thc random variacs, tor 0 j q 1,

j

j(r+s)+r1

i=j(r+s)
Z
n,i
,

j

(j+1)(r+s)

i=j(r+s)+r
Z
n,i
,
and

q

n1

i=q(r+s)
Z
n,i
.
Thcn,

nh d
T
Q
n

1

n
_

_
q1

j=0

j
+
q1

j=0

j
+
q
_

n
{Q
n,1
+ Q
n,2
+ Q
n,3
} .
(S!3)
\c show that as n ,
1
n
E |Q
n,2
|
2
0,
1
n
E |Q
n,3
|
2
0, (S!!)

E |cxj(i t Q
n,1
)|
q1

j=0
E |cxj(i t
j
)|

0, (S!`)
1
n
q1

j=0
E
_

2
j
_

2
(u
0
), (S!o)
and
1
n
q1

j=0
E
_

2
j
I
_
|
j
| (u
0
)

n
__
0 (S!)
tor cvcry > 0 (S!!) imjics that Q
n,2
and Q
n,3
arc asymjtoticay
nciic in jroaiity, (S!`) shows that thc summands
j
in Q
n,1
arc asymjtoticay indcjcndcnt and (S!o) and (S!) arc thc stan-
dard Lindccr-Iccr conditions tor asymjtotic normaity ot Q
n,1
tor
thc indcjcndcnt sctuj
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 309
\c rst cstaish (S!!) Ior this jurjosc, wc choosc thc arc
ock sizc Condition A2 imjics that thcrc is a scqucncc ot jositivc
constants
n
such that

n
s
n
o
_

nh
n
_
and

n
(n/h
n
)
1/2
(s
n
) 0. (S!S)
Lcnc thc arc ock sizc r
n
y r
n
(nh
n
)
1/2
/
n
and thc sma
ock sizc s
n
Thcn it can casiy c shown trom (S!S) that as n
,
s
n
/r
n
0, r
n
/n 0, r
n
(nh
n
)
1/2
0, (S!9)
and
(n/r
n
) (s
n
) 0. (S`0)
Oscrvc that
E |Q
n,2
|
2

q1

j=0
\ar(
j
) + 2

0i<jq1
Cov(
i
,
j
) I
1
+ I
2
. (S`1)
lt toows trom stationarity and Lcmma A1 that
I
1
q
n
\ar(
1
) q
n
\ar
_
_
_
s
n

j=1
Z
n,j
_
_
_ q
n
s
n
|
2
(u
0
) + o(1)|. (S`2)
`cxt considcr thc sccond tcrm I
2
in thc riht sidc ot (S`1) Lct
r

j
j(r
n
+ s
n
), thcn r

j
r

i
r
n
tor a j > i, wc thus havc
|I
2
| 2

0i<jq1
s
n

j
1
=1
s
n

j
2
=1
|Cov(Z
n,r

i
+r
n
+j
1
, Z
n,r

j
+r
n
+j
2
)|
2
nr
n

j
1
=1
n

j
2
=j
1
+r
n
|Cov(Z
n,j
1
, Z
n,j
2
)|.
Ly stationarity and Lcmma A1, onc otains
|I
2
| 2n
n

j=r
n
+1
|Cov(Z
n,1
, Z
n,j
)| o(n). (S`3)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 310
Lcncc, y (S!9)-(S`3), wc havc
1
n
E|Q
n,2
|
2
O
_
q
n
s
n
n
1
_
+ o(1) o(1). (S`!)
lt toows trom stationarity, (S!9), and Lcmma A1 that
\ar |Q
n,3
| \ar
_
_
_
nq
n
(r
n
+s
n
)

j=1
Z
n,j
_
_
_ O(n q
n
(r
n
+ s
n
)) o(n).
(S``)
Cominin (S!9), (S`!), and (S``), wc cstaish (S!!) As tor
(S!o), y stationarity, (S!9), (S`0), and Lcmma A1, it is casiy
sccn that
1
n
q
n
1

j=0
E
_

2
j
_

q
n
n
E
_

2
1
_

q
n
r
n
n

1
r
n
\ar
_
_
_
r
n

j=1
Z
n,j
_
_
_
2
(u
0
).
To cstaish (S!`), wc usc Lcmma 11 ot \okonskii and lozanov
(19`9) (scc aso lraimov and Linnik 191, j 33S) to otain

E |cxj(i t Q
n,1
)|
q
n
1

j=0
E |cxj(i t
j
)|

1o (n/r
n
) (s
n
)
tcndin to 0 y (S`0)
lt rcmains to cstaish (S!) Ior this jurjosc, wc usc thcorcm
!1 ot Shao and Yu (199o) and condition A2 to otain
E
_

2
1
I
_
|
1
| (u
0
)

n
__
C n
1/2
E
_
|
1
|

_
C n
1/2
r
/2
n
_
E
_
|Z
n,0
|

__
/

. (S`o)
As in (S3S),
E
_
|Z
n,0
|

_
C h
1

/2
. (S`)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 311
Thcrctorc, y (S`o) and (S`),
E
_

2
1
I
_
|
1
| (u
0
)

n
__
C n
1/2
r
/2
n
h
(2

)/(2

)
. (S`S)
Thus, y (S!0) and thc dcnition ot r
n
, and usin conditions A2c
and A2d, wc otain
1
n
q1

j=0
E
_

2
j
I
_
|
j
| (u
0
)

n
__
C
1/2
n
n
1/2/4
h
/

1/2/4
n
0
(S`9)
ccausc
n
This comjctcs thc jroot ot thc thcorcm
8.6.8 Monte Carlo Simulations and Applications
Scc Cai, Ian and Yao (2000) tor thc dctaicd `ontc Caro simuation
rcsuts and ajjications
8.7 Additive Model
8.7.1 Model
ln this scction, wc usc thc notation trom Cai (2002) Lct {X
t
, Y
t
, Z
t
}

t=
c ,ointy stationary jroccsscs, whcrc X
t
and Y
t
takc vaucs in
p
and
q
with p, q 0, rcsjcctivcy Thc rcrcssion surtacc is dcncd
y
m(x, y) E {Z
t
| X
t
x, Y
t
y} . (So0)
Lcrc, it is assumcd that E|Z
t
| < `otc that thc rcrcssion tunc-
tion m(, ) dcncd in (So0) can idcntity ony thc sum
m(x, y) + g
1
(x) + g
2
(y). (So1)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 312
Such a dccomjosition hods, tor cxamjc, tor thc toowin nonincar
additivc autorcrcssivc modc with cxocnous variacs (AlX)
Y
t
+ g
1
(X
tj
1
, . . . , X
tj
p
) + g
2
(Y
ti
1
, . . . , Y
ti
q
) +
t
,
X
tj
1
g
3
(X
tj
2
, . . . , X
tj
p
) +
t
.
Ior dctaicd discussions on thc AlX modc, thc rcadcr is rctcrrcd
to thc jajcrs y `asry and T,sthcim (199) and Cai and `asry
(2000) Ior idcntiaiity, it is assumcd that E {g
1
(X
t
)} 0 and
E {g
2
(Y
t
)} 0 Thcn, thc jro,cction ot m(x, y) on thc g
1
(x)-
dircction is dcncd y
E{m(x, Y
t
)} + g
1
(x) + E {g
2
(Y
t
)} + g
1
(x). (So2)
Ccary, g
1
() can c idcnticd uj to an additivc constant and g
2
()
can c rctricvcd ikcwisc
A thorouh discussion ot additivc timc scrics modcs dcncd in
(So1) can c tound in Chcn and Tsay (1993) Additivc comjo-
ncnts can c cstimatcd with a onc-dimcnsiona nonjaramctric ratc
ln most jajcrs, to cstimatc additivc comjoncnts, scvcra mcthods
havc ccn jrojoscd Ior cxamjc, Chcn and Tsay (1993) uscd thc
itcrativc ackttin jroccdurcs, such as thc ACL aorithm and thc
LlTO ajjroach. scc Lastic and Tishirani (1990) tor dctais Lut,
thcir asymjtotic jrojcrtics arc not wc undcrstood duc to thc im-
jicit dcnition ot thc rcsutin cstimators To attcnuatc thc draw-
acks ot itcrativc jroccdurcs, Aucstad and T,sthcim (1991) and
T,sthcim and Aucstad (199!a) jrojoscd a dircct mcthod ascd on
an avcrac rcrcssion surtacc idca, rctcrrcd to as jro,cction mcthod in
T,sthcim and Aucstad (199!a) tor timc scrics data As jointcd out
y Cai and Ian (2000), a dircct mcthod has somc advantacs, such
as it docs not rcy on itcrations, it can makc comjutation tast, and
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 313
morc imjortanty, it aows an asymjtotic anaysis Iinay, thc jro-
,cction mcthod is cxtcndcd to nonincar AlX modcs y `asry and
T,sthcim (199) usin thc kcrnc mcthod and Cai and `asry (2000)
coujcd with thc oca joynomia ajjroach lt shoud c rcmarkcd
that thc jro,cction mcthod, undcr thc namc ot marina intcration,
is jrojoscd indcjcndcnty y `cwcy (199!) and Linton and `icscn
(199`) tor iid samjcs, and sincc thcn, somc imjortant jrorcsscs
havc ccn madc y somc authors Ior cxamjc, y cominin thc
marina intcration with onc-stcj ackttin, Linton (199, 2000)
jrcscnts an ccicnt cstimator, `ammcn, Linton, and `icscn (1999)
cstaish riorousy thc asymjtotic thcory ot thc ackttin, Cai and
Ian (2000) considcr cstimatin cach comjoncnt usin thc wcihtcd
jro,cction mcthod coujcd with thc oca incar ttin in an ccicnt
way, and Sjcrich, T,thcim, and Yan (2000) cxtcnd thc ccicnt
mcthod to modcs with simjc intcractions
Thc jro,cction mcthod has somc disadvantacs athouh it has
thc atorcmcntioncd mcrits Thc jro,cction mcthod may not c c-
cicnt it covariatcs (cndocnous or cxocnous variacs) arc strony
corrcatcd, which is jarticuary rccvant tor autorcrcssivc modcs
Thc intuitivc intcrjrctation is that additivc comjoncnts arc not or-
thoona To ovcrcomc this shortcomin, two ccicnt cstimation
mcthods havc ccn jrojoscd in thc itcraturc Thc rst onc is cacd
wciht tunction jroccdurc, jrojoscd y Ian, Lardc, and `ammcn
(199S) tor iid samjcs and cxtcndcd to timc scrics situations y Cai
and Ian (2000) \ith an ajjrojriatc choicc ot thc wciht tunction,
additivc comjoncnts can c ccicnty cstimatcd in thc scnsc that an
additivc comjoncnt can c cstimatcd with thc samc asymjtotic ias
and variancc as it thc rcst ot comjoncnts wcrc known Thc sccond
onc is to cominc thc marina intcration with onc-stcj ackttin,
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 314
introduccd y Linton (199, 2000) tor iid samjcs and cxtcndcd y
Sjcrish, T,sthcim, and Yan (2000) to additivc modcs with sinc
intcractions, ut this mcthod has not ccn advocatcd tor timc scrics
situations Lowcvcr, thcrc has not ccn any attcmjt to discuss thc
andwidth sccction tor thc jro,cction mcthod and its variations in
thc itcraturc duc to thcir comjcxity ln jracticc, onc andwidth is
usuay uscd tor a comjoncnts athouh Cai and Ian (2000) aruc
that dicrcnt andwidths miht c uscd thcorcticay to dca with thc
situation that additivc comjoncnts josscs thc dicrcnt smoothncss
Thcrctorc, thc jro,cction mcthod may not c ojtima in jracticc in
thc scnsc that onc andwidth is uscd
To cstimatc unknown additivc comjoncnts in (So1) ccicnty,
toowin thc sjirit ot thc marina intcration with onc-stcj ack-
ttin jrojoscd y Linton (199) tor iid samjcs, l usc a two-stac
mcthod, duc to Linton (2000), coujcd with thc oca incar (joyno-
mia) mcthod, which has somc attractivc jrojcrtics, such as math-
cmatica ccicncy, ias rcduction and adajtation ot cdc ccct (scc
Ian and Gi,cs, 199o) Thc asic idca ot thc two-stac ajjroach
is dcscricd as toows At thc rst stac, onc otains thc initia
cstimatcd vaucs tor a comjoncnts `orc jrcciscy, thc idca tor
cstimatin any additivc comjoncnt is rst to cstimatc dirccty hih-
dimcnsiona rcrcssion surtacc y thc oca incar mcthod and thcn
to avcrac thc rcrcssion surtacc ovcr thc rcst ot variacs to stai-
izc variancc Such an initia cstimatc, in cncra, is undcr-smoothcd
so that thc ias shoud c asymjtoticay nciic At thc sccond
stac, thc oca incar (joynomia) tcchniquc is uscd aain to csti-
matc any additivc comjoncnt y usin thc initia cstimatcd vaucs
ot thc rcst ot comjoncnts ln such a way, it is shown that thc csti-
matc at thc sccond stac is not ony ccicnt in thc scnsc ot cin
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 315
cquivacnt to a jroccdurc ascd on knowin othcr comjoncnts, ut
aso makin thc andwidth sccction much casicr `otc that this
tcchniquc is not novc to this jajcr sincc thc two-stac mcthod is
rst uscd y Linton (199, 2000) tor iid samjcs, ut many dctais
and insihts arc
Thc rcst ot thc jajcr is oranizcd as toows Scction 2 ivcs a
rict rcvicw ot thc jro,cction mcthod and discusscs its advantacs
and shortcomins Scction 3 jrcscnts thc two-stac ajjroach coujcd
with a ncw andwidth sccctor, ascd on thc nonjaramctric vcrsion ot
thc Akaikc intormation critcrion Aso, thc asymjtotic normaity ot
thc rcsutin cstimator is cstaishcd ln Scction !, a sma simuation
study is carricd out to iustratc thc mcthodooy and thc two-stac
ajjroach is aso ajjicd to a rca cxamjc Iinay, tocthcr with
somc rcuarity conditions, thc tcchnica jroot is rccatcd to thc
Ajjcndix
8.7.2 Backtting Algorithm
Thc uidin ock ot thc cncraizcd additivc modc aorithm is
thc scattcrjot smoothcr \c wi rst dcscric scattcrjot smooth-
in in a simjc scttin, and thcn indicatc how it is uscd in cncraizcd
additivc modcin Lcrc y is a rcsjonsc or outcomc variac, and
x is a jronostic tactor \c wish to t a smooth curvc f(x) that
summarizcs thc dcjcndcncc ot y on x lt wc wcrc to nd thc curvc
that simjy minimizcs

n
i=1
|y
i
f(x
i
)|
2
, thc rcsut woud c an in-
tcrjoatin curvc that woud not c smooth at a Thc cuic sjinc
smoothcr imjoscs smoothncss on f(x) \c scck thc tunction f(x)
that minimizcs
n

i=1
|y
i
f(x
i
)|
2
+
_
|f

(x)|
2
dx (So3)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 316
`oticc that
_
|f

(x)|
2
dx mcasurcs thc wiincss ot thc tunction
f(x) incar f(x)s havc
_
|f

(x)|
2
dx 0, whic non-incar ts jroducc
vaucs icr than zcro is a non-ncativc smoothin jaramc-
tcr that must c choscn y thc data anayst lt ovcrns thc tradco
ctwccn thc oodncss ot t to thc data and (as mcasurcd y and wi-
cncss ot thc tunction Larcr vaucs ot torcc f(x) to c smoothcr
Ior any vauc ot , thc soution to (So3) is a cuic sjinc, ic, a
jicccwisc cuic joynomia with jicccs ,oincd at thc uniquc oscrvcd
vaucs ot x in thc datasct Iast and stac numcrica jroccdurcs arc
avaiac tor comjutation ot thc ttcd curvc \hat vauc ot did wc
usc in jracticc ln tact it is not a convcnicnt to cxjrcss thc dcsircd
smoothncss ot f(x) in tcrms ot , as thc mcanin ot dcjcnds on thc
units ot thc jronostic tactor x lnstcad, it is jossic to dcnc an
ccctivc numcr ot jaramctcrs or dcrccs ot trccdom ot a cuic
sjinc smoothcr, and thcn usc a numcrica scarch to dctcrminc thc
vauc ot to yicd this numcr ln jracticc, it wc chosc thc ccctivc
numcr ot jaramctcrs to c `, rouhy sjcakin, this mcans that thc
comjcxity ot thc curvc is aout thc samc as a joynomia rcrcssion
ot dcrccs ! Lowcvcr, thc cuic sjinc smoothcr sjrcads out its
jaramctcrs in a morc cvcn manncr, and hcncc is much morc cxic
than a joynomia rcrcssion `otc that thc dcrccs ot trccdom ot a
smoothcr nccd not c an intccr
Thc aovc discussion tcs how to t a curvc to a sinc jronostic
tactor \ith mutijc jronostic tactors, it x
ij
dcnotcs thc vauc ot
thc jth jronostic tactor tor thc ith oscrvation, wc t thc additivc
modc
y
i

d

j=1
f
j
(x
ij
) +
i
.
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 317
A critcrion ikc (So3) can c sjccicd tor this jrocm, and a simjc
itcrativc jroccdurc cxists tor cstimatin thc f
j
s \c ajjy a cuic
sjinc smoothcr to thc outcomc y
i

d
j=k

f
j
(x
ij
) as a tunction ot x
ik
,
tor cach jronostic tactor in turn Thc jroccss is continucs unti thc
cstimatcs

f
j
(x) staiizc Thcsc jroccdurc is known as ackttin
and thc rcsutin t is anaoous to a mutijc rcrcssion tor incar
modcs
8.7.3 Projection Method
This scction is dcvotcd to a rict rcvicw ot thc jro,cction mcthod
and discusscs its mcrits and disadvantacs
lt is assumcd that a additivc comjoncnts havc continuous sccond
jartia dcrivativcs, so that m(u, v) can c ocay ajjroximatcd y
a incar tcrm in a ncihorhood ot (x, y), namcy, m(u, v)
0
+

T
1
(u x) +
T
2
(v y) with {
j
} dcjcndin on x and y, whcrc

T
1
dcnotcs thc transjosc ot
1

Lct K() and L() c symmctric kcrnc tunctions in

p
and
q
,
rcsjcctivcy, and h
11
h
11
(n) > 0 and h
12
h
12
(n) > 0 c and-
widths in thc stcj ot cstimatin thc rcrcssion surtacc Lcrc, to
handc various dcrccs ot smoothncss, Cai and Ian (2000) jrojosc
usin h
11
and h
12
dicrcnty athouh thc imjcmcntation may not
c casy in jracticc Thc rcadcr is rctcrrcd to thc jajcr y Cai and
Ian (2000) tor dctais Givcn oscrvations {X
t
, Y
t
, Z
t
}
n
t=1
, ct

j
c thc minimizcr ot thc toowin ocay wcihtcd cast squarcs
n

t=1
_
Z
t

T
1
(X
t
x)
T
2
(Y
t
y)
_
2
K
h
11
(X
t
x) L
h
12
(Y
t
y),
whcrc K
h
() K(/h)/h
p
and L
h
() L(/h)/h
q
Thcn, thc oca
incar cstimator ot thc rcrcssion surtacc m(x, y) is

m(x, y)

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 318

Ly comjutin thc samjc avcrac ot

m(, ) ascd on (So2), thc
jro,cction cstimators ot g
1
() and g
2
() arc dcncd as, rcsjcctivcy,

g
1
(x)
1
n
n

t=1

m(x, Y
t
)

,
and

g
2
(y)
1
n
n

t=1

m(X
t
, y)

,
whcrc

n
1

n
t=1
Z
t
ndcr somc rcuarity conditions, y usin
thc samc arumcnts as thosc cmjoycd in thc jroot ot Thcorcm 3
in Cai and `asry (2000), it can c shown (athouh not casy and
tcdious) that thc asymjtotic ias and asymjtotic variancc ot

g
1
(x)
arc, rcsjcctivcy, h
2
11
tr{
2
(K) g

1
(x)}/2 and v
1
(x)
0
(K) A(x),
whcrc
A(x)
_
p
2
2
(y)
2
(x, y) p
1
(x, y) dy
and

2
(x, y) \ar ( Z
t
| X
t
x, Y
t
y) .
Lcrc, p(x, y) stands tor thc ,oint dcnsity ot X
t
and Y
t
, p
1
(x) dcnotcs
thc marina dcnsity ot X
t
, p
2
(y) is thc marina dcnsity ot Y
t
,

0
(K)
_
K
2
(u)du, and
2
(K)
_
uu
T
K(u) du
Thc torcoin mcthod has somc advantacs, such as it is casy to
undcrstand, it can makc comjutation tast, and it aows an asymj-
totic anaysis Lowcvcr, it can c quitc inccicnt in an asymjtotic
scnsc To dcmonstratc this idca, ct us considcr thc idca situation
that g
2
() and arc known ln such a casc, onc can cstimatc g
1
() y
dirccty rcrcssin thc jartia crror

Z
t
Z
t
g
2
(Y
t
) on X
t
and
such an idca cstimator is ojtima in an asymjtotic minimax scnsc
(scc, c, Ian and Gi,cs, 199o) Thc asymjtotic ias tor thc idca
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 319
cstimator is h
2
11
tr{
2
(K) g

1
(x)}/2 and thc asymjtotic variancc is
v
0
(x)
0
(K) B(x) with B(x) p
1
1
(x) E
_

2
(X
t
, Y
t
) | X
t
x
_
(So!)
(scc, c, `asry and Ian, 199) lt is ccar that v
1
(x) v
0
(x) it
X
t
and Y
t
arc indcjcndcnt lt X
t
and Y
t
arc corrcatcd and whcn

2
(x, y) is a constant, it toows trom thc Cauchy-Schwarz incquaity
that
B(x)

2
p
1
(x)
_
p
1/2
(y|x)
p
2
(y)
p
1/2
(y|x)
dy

2
p
1
(x)
_
p
2
2
(y)
p(y|x)
d y A(x),
which imjics that thc idca cstimator has aways smacr asymjtotic
variancc than thc jro,cction mcthod athouh oth havc thc samc
ias This sucsts that thc jro,cction mcthod coud cad to an in-
ccicnt cstimation ot g
1
() and g
2
() whcn X
t
and Y
t
arc scriay
corrcatcd, which is jarticuary rccvant tor autorcrcssivc modcs
To acviatc this shortcomin, l jrojosc thc two-stac ajjroach dc-
scricd ncxt
8.7.4 Two-Stage Procedure
Thc two-stac mcthod duc to Linton (199, 2000) is introduccd Thc
asic idca is to ct an initia cstimatc tor

g
2
() usin a sma and-
width h
12
Thc initia cstimatc can c otaincd y thc jro,cction
mcthod and h
12
can c choscn so sma that thc ias ot cstimat-
in

g
2
() can c asymjtoticay nciic Thcn, usin thc jartia
rcsiduas Z

t
Z
t

g
2
(Y
t
), wc ajjy thc oca incar rcrcssion
tcchniquc to thc jscudo rcrcssion modc
Z

t
g
1
(X
t
) +

t
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 320
to cstimatc g
1
() This cads naturay to thc wcihtcd cast-squarcs
jrocm
n

t=1
_
Z

t

1

T
2
(X
t
x)
_
2
J
h
2
(X
t
x), (So`)
whcrc J() is thc kcrnc tunction in
p
and h
2
h
2
(n) > 0 is thc
andwidth at thc sccond-stac Thc advantac ot this is twotod thc
andwidth h
2
can now c sccctcd jurjoscy tor cstimatin g
1
() ony
and any andwidth sccction tcchniquc tor nonjaramctric rcrcssion
can c ajjicd hcrc `aximizin (So`) with rcsjcct to
1
and
2
ivcs thc two-stac cstimatc ot g
1
(x), dcnotcd y

g
1
(x)

1
, whcrc

1
and

2
arc thc minimizcr ot (So`)
lt is shown in Thcorcm 1, in which toows, that undcr somc rcu-
arity conditions, thc asymjtotic ias and variancc ot thc two-stac
cstimatc

g
1
(x) arc thc samc as thosc tor thc idca cstimator, jrovidcd
that thc initia andwidth h
12
satiscs h
12
o (h
2
)
Sampling Properties
To cstaish thc asymjtotic normaity ot thc two-stac cstimator, it
is assumcd that thc initia cstimator satiscs a incar ajjroximation.
namcy,

g
2
(Y
t
) g
2
(Y
t
)
1
n
n

i=1
L
h
12
(Y
i
Y
t
)(X
i
, Y
t
)
i
+
1
2
h
2
12
tr{
2
(L) g

2
(Y
t
)}, (Soo)
whcrc
t
Z
t
m(X
t
, Y
t
) and (x, y) p
1
(x)/p(x, y) `otc that
undcr somc rcuarity conditions, y toowin thc samc arumcnts
as in `asry (199o), onc miht show (athouh thc jroot is not casy,
quitc cnthy, and tcdious) that (Soo) hods `otc that this assumj-
tion is aso imjoscd in Linton (2000) tor iid samjcs to simjity thc
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 321
jroot ot thc asymjtotic rcsuts ot thc two-stac cstimator `ow, thc
asymjtotic normaity tor thc two-stac cstimator is statcd hcrc and
its jroot is rccatcd to thc Ajjcndix
THEOREM 1. Under (8.66) and Assumptions A1 A9 stated
in the Appendix, if bandwidths h
12
and h
2
are chosen such that
h
12
0, nh
q
12
, h
2
0, and nh
p
2
as n , then,
_
nh
p
2
_

g
1
(x) g
1
(x) bias(x) + o
p
_
h
2
12
+ h
2
2
__
D
N {0, v
0
(x)} ,
where the asymptotic bias is
bias(x)
h
2
2
2
tr{
2
(J) g

1
(x)}
h
2
12
2
tr {
2
(L) E (g

2
(Y
t
) | X
t
x)}
and the asymptotic variance is v
0
(x)
0
(J) B(x).
\c rcmark that y Thcorcm 1, thc asymjtotic variancc ot thc
two-stac cstimator is indcjcndcnt ot thc initia andwidths Thus,
thc initia andwidths shoud c choscn as sma as jossic This
is anothcr cnct ot usin thc two-stac jroccdurc thc andwidth
sccction jrocm ccomcs rcativcy casy ln jarticuar, whcn h
12

o (h
2
), thc ias trom thc initia cstimation can c asymjtoticay
nciic Ior thc idca situation that g
2
() is known, `asry and
Ian (199) show that undcr somc rcuarity conditions, thc ojtima
cstimatc ot g
1
(x), dcnotcd y

g

1
(x), y usin (So`) in which thc
jartia rcsidua Z

t
is rcjaccd y thc jartia crror

Z
t
Y
t

g
2
(Y
t
), is asymjtoticay normay distriutcd,
_
nh
p
2
_

1
(x) g
1
(x)
h
2
2
2
tr{
2
(J) g

1
(x)} + o
p
(h
2
2
)
_

_
D
N {0, v
0
(x)} .
This, in con,unction with Thcorcm 1, shows that thc two-stac cs-
timator and thc idca cstimator sharc thc samc asymjtotic ias and
variancc it h
12
o (h
2
)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 322
Iinay, it is worth to jointin out that undcr somc rcuarity
conditions, thc nonincar additivc AlX jroccsscs arc stationary and
-mixin with comctric dccay mixin coccicnt, scc `asry and
T,sthcim (199), so that Assumjtions Ao, A, and AS in thc Aj-
jcndix imjoscd on thc mixin coccicnt arc automaticay satis-
cd Thcrctorc, assumin that thc othcr tcchnica assumjtions ot
this jajcr arc satiscd, thc rcsut in Thcorcm 1 can c ajjicd to
thc nonincar additivc AlX modcs
8.7.5 Monte Carlo Simulations and Applications
Scc Cai (2002) tor thc dctaicd `ontc Caro simuation rcsuts and
ajjications
8.8 Computer Code
# 07-31-2006
graphics.off() # clean the previous graphs on the screen
###################################################################
z1=matrix(scan(file="c:\\teaching\\time series\\data\\w-3mtbs7097.txt"),
byrow=T,ncol=4)
# dada: weekly 3-month Treasury bill from 1970 to 1997
x=z1[,4]/100
n=length(x)
y=diff(x) # Delta x_t=x_t-x_{t-1}
x=x[1:(n-1)]
n=n-1
x_star=(x-mean(x))/sqrt(var(x))
z=seq(min(x),max(x),length=50)
#win.graph()
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 323
postscript(file="c:\\teaching\\time series\\figs\\fig-8.1.eps",
horizontal=F,width=6,height=6)
par(mfrow=c(2,2),mex=0.4,bg="light blue")
scatter.smooth(x,y,span=1/10,ylab="",xlab="x(t-1)",evaluation=60
title(main="(a) y(t) vs x(t-1)",col.main="red")
scatter.smooth(x,abs(y),span=1/10,ylab="",xlab="x(t-1)",evalua
title(main="(b) |y(t)| vs x(t-1)",col.main="red")
scatter.smooth(x,y^2,span=1/10,ylab="",xlab="x(t-1)",evaluation=
title(main="(c) y(t)^2 vs x(t-1)",col.main="red")
dev.off()
###################################################################
#########################
# Nonparametric Fitting #
#########################
#########################################################
# Define the Epanechnikov kernel function
kernel<-function(x){0.75*(1-x^2)*(abs(x)<=1)}
###############################################################
# Define the kernel density estimator
kernden=function(x,z,h,ker){
# parameters: x=variable; h=bandwidth; z=grid point; ker=kernel
nz<-length(z)
nx<-length(x)
x0=rep(1,nx*nz)
dim(x0)=c(nx,nz)
x1=t(x0)
x0=x*x0
x1=z*x1
x0=x0-t(x1)
if(ker==1){x1=kernel(x0/h)} # Epanechnikov kernel
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 324
if(ker==0){x1=dnorm(x0/h)} # normal kernel
f1=apply(x1,2,mean)/h
return(f1)
}
###############################################################
# Define the local constant estimator
local.constant=function(y,x,z,h,ker){
# parameters: x=variable; h=bandwidth; z=grid point; ker=kernel
nz<-length(z)
nx<-length(x)
x0=rep(1,nx*nz)
dim(x0)=c(nx,nz)
x1=t(x0)
x0=x*x0
x1=z*x1
x0=x0-t(x1)
if(ker==1){x1=kernel(x0/h)} # Epanechnikov kernel
if(ker==0){x1=dnorm(x0/h)} # normal kernel
x2=y*x1
f1=apply(x1,2,mean)
f2=apply(x2,2,mean)
f3=f2/f1
return(f3)
}
###################################################################
# Define the local linear estimator
local.linear<-function(y,x,z,h){
# parameters: y=response, x=design matrix; h=bandwidth; z=grid
nz<-length(z)
ny<-length(y)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 325
beta<-rep(0,nz*2)
dim(beta)<-c(nz,2)
for(k in 1:nz){
x0=x-z[k]
w0<-kernel(x0/h)
beta[k,]<-glm(y~x0,weight=w0)$coeff
}
return(beta)
}
###################################################################
h=0.02
# Local constant estimate
mu_hat=local.constant(y,x,z,h,1)
sigma_hat=local.constant(abs(y),x,z,h,1)
sigma2_hat=local.constant(y^2,x,z,h,1)
win.graph()
par(mfrow=c(2,2),mex=0.4,bg="light yellow")
scatter.smooth(x,y,span=1/10,ylab="",xlab="x(t-1)")
points(z,mu_hat,type="l",lty=1,lwd=3,col=2)
title(main="(a) y(t) vs x(t-1)",col.main="red")
legend(0.04,0.0175,"Local Constant Estimate")
scatter.smooth(x,abs(y),span=1/10,ylab="",xlab="x(t-1)")
points(z,sigma_hat,type="l",lty=1,lwd=3,col=2)
title(main="(b) |y(t)| vs x(t-1)",col.main="red")
scatter.smooth(x,y^2,span=1/10,ylab="",xlab="x(t-1)")
title(main="(c) y(t)^2 vs x(t-1)",col.main="red")
points(z,sigma2_hat,type="l",lty=1,lwd=3,col=2)
# Local Linear Estimate
fit2=local.linear(y,x,z,h)
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 326
mu_hat=fit2[,1]
fit2=local.linear(abs(y),x,z,h)
sigma_hat=fit2[,1]
fit2=local.linear(y^2,x,z,h)
sigma2_hat=fit2[,1]
win.graph()
par(mfrow=c(2,2),mex=0.4,bg="light green")
scatter.smooth(x,y,span=1/10,ylab="",xlab="x(t-1)")
points(z,mu_hat,type="l",lty=1,lwd=3,col=2)
title(main="(a) y(t) vs x(t-1)",col.main="red")
legend(0.04,0.0175,"Local Linear Estimate")
scatter.smooth(x,abs(y),span=1/10,ylab="",xlab="x(t-1)")
points(z,sigma_hat,type="l",lty=1,lwd=3,col=2)
title(main="(b) |y(t)| vs x(t-1)",col.main="red")
scatter.smooth(x,y^2,span=1/10,ylab="",xlab="x(t-1)")
title(main="(c) y(t)^2 vs x(t-1)",col.main="red")
points(z,sigma2_hat,type="l",lty=1,lwd=3,col=2)
###################################################################
8.9 References
Bowman, A. (1984). An alternative method of cross-validation for the smoothing of density
estimate. Biometrika, 71, 353-360.
Cai, Z. (2002). A two-stage approach to additive time series models. Statistica Neerlandica,
56, 415-433.
CAI, Z. and J. FAN (2000). Average regression surface for dependent data. Journal of
Multivariate Analysis, 75, 112-142.
Cai, Z., J. Fan and Q. Yao (2000). Functional-coecient regression models for nonlinear
time series. Journal of American Statistical Association, 95, 941-956.
CAI, Z. and E. MASRY (2000). Nonparametric estimation of additive nonlinear ARX time
series: Local linear tting and projection. Econometric Theory, 16, 465-501.
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 327
Cai, Z. and R.C. Tiwari (2000). Application of a local linear autoregressive model to BOD
time series. Environmetrics, 11, 341-350.
CHEN, R. and R. TSAY (1993). Nonlinear additive ARX models. Journal of the American
Statistical Association, 88, 310-320.
Chiu, S.T. (1991). Bandwidth selection for kernel density estimation. The Annals of
Statistics, 19, 1883-1905.
Engle, R.F., C.W.J. Grabger, J. Rice, and A. Weiss (1986). Semiparametric estimates of
the relation between weather and electricity sales. Journal of The American Statistical
Association, 81, 310-320.
Fan, J. (1993). Local linear regression smoothers and their minimax eciency. The Annals
of Statistics, 21, 196-216.
Fan, J., N.E. Heckman, and M.P. Wand (1995). Local polynomial kernel regression for
generalized linear models and quasi-likelihood functions. Journal of the American
Statistical Association, 90, 141-150.
Fan, J., T. Gasser, I. Gijbels, M. Brockmann and J. Engel (1996). Local polynomial tting:
optimal kernel and asymptotic minimax eciency. Annals of the Institute of Statistical
Mathematics, 49, 79-99.
Fan, J. and I. Gijbels (1996). Local Polynomial Modeling and Its Applications. London:
Chapman and Hall.
Fan, J. and Q. Yao (2003). Nonlinear Time Series: Nonparametric and Parametric Meth-
ods. New York: Springer-Verlag.
Fan, J., Q. Yao and Z. Cai (2003). Adaptive varying-coecient linear models. Journal of
the Royal Statistical Society, Series B, 65, 57-80.
Fan, J. and C. Zhang (2003). A re-examination of diusion estimators with applications
to nancial model validation. Journal of the American Statistical Association, 98,
118-134.
Fan, J., C. Zhang and J. Zhang (2001). Generalized likelihood test statistic and Wilks
phenomenon. The Annals of Statistics, 29, 153-193.
Gasser, T. and H.-G. M uller (1979). Kernel estimation of regression functions. In Smoothing
Techniques for Curve Estimation, Lecture Notes in Mathematics, 757, 2368. Springer-
Verlag, New York.
Granger, C.W.J., and T. Terasvirta (1993). Modeling Nonlinear Economic Relationships.
Oxford, U.K.: Oxford University Press.
Hall, P., and C.C. Heyde (1980). Martingale Limit Theory and Its Applications. New York:
Academic Press.
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 328
Hall, P., and I. Johnstone (1992). Empirical functional and ecient smoothing parameter
selection (with discussion). Journal of the Royal Statistical Society, Series B, 54,
475-530.
HASTIE, T.J. and R.J. TIBSHIRANI (1990). Generalized Additive Models. Chapman and
Hall, London.
Hjort, N.L. and M.C. Jones (1996). Better rules of thumb for choosing bandwidth in density
estimation. Working paper, Department of Mathematics, University of Oslo, Norway.
Hong, Y. and Lee, T.-H. (2003). Inference on via generalized spectrum and nonlinear time
series models. The Review of Economics and Statistics, 85, 1048-1062.
Hurvich, C.M., J.S. Simono and C.-L. Tsai (1998). Smoothing parameter selection in
nonparametric regression using an improved Akaike information criterion. Journal of
the Royal Statistical, Society B, 60, 271-293.
Jones, M.C., J.S. Marron and S.J. Sheather (1996). A brief survey of bandwidth selection
for density estimation. Journal of American Statistical Association, 91, 401-407.
Kreiss, J.P., M. Neumann and Q. Yao (1998). Bootstrap tests for simple structures in
nonparametric time series regression. Unpublished manuscript.
LINTON, O.B. (1997). Ecient estimation of additive nonparametric regression models.
Biometrika, 84, 469-473.
LINTON, O.B. (2000). Ecient estimation of generalized additive nonparametric regression
models. Econometric Theory, 16, 502-523.
LINTON, O.B. and J.P. NIELSEN (1995). A kernel method of estimating structured
nonparametric regression based on marginal integration. Biometrika, 82, 93-100.
MAMMEN, E., O.B. LINTON, and J.P. NIELSEN (1999). The existence and asymptotic
properties of a backtting projection algorithm under weak conditions. The Annals of
Statistics, 27, 1443-1490.
MASRY, E. and J. FAN (1997). Local polynomial estimation of regression functions for
mixing processes. Scandinavian Journal of Statistics, 24, 165-179.
MASRY, E. and D. TJSTHEIM (1997). Additive nonlinear ARX time series and projec-
tion estimates. Econometric Theory, 13, 214-252.
ksendal, B. (1985). Stochastic Dierential Equations: An Introduction with Applications,
3th edition. New York: Springer-Verlag.
Priestley, M.B. and M.T. Chao (1972). Nonparametric function tting. Journal of the
Royal Statistical Society, Series B, 34, 384-392.
Rice, J. (1984). Bandwidth selection for nonparametric regression. The Annals of Statistics,
12, 1215-1230.
CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION 329
Rudemo, M . (1982). Empirical choice of histograms and kernel density estimators. Scan-
dinavia Journal of Statistics, 9, 65-78 .
Ruppert, D., S.J. Sheather and M.P. Wand (1995). An eective bandwidth selector for local
least squares regression. Journal of American Statistical Association, 90, 1257-1270.
Ruppert, D. and M.P. Wand (1994). Multivariate weighted least squares regression. The
Annals of Statistics, 22, 1346-1370.
Rousseeuw, R.J. and A.M. Leroy (1987). Robust Regression and Outlier Detection. New
York: Wiley.
Shao, Q. and H. Yu (1996). Weak convergence for weighted empirical processes of dependent
sequences. The Annals of Probability, 24, 2098-2127.
Sheather, S.J. and M.C. Jones (1991). A reliable data-based bandwidth selection method
for kernel density estimation. Journal of the Royal Statistical Society, Series B, 53,
683-690.
SPERLISH, S., D. TJSTHEIM, and L. YANG (2000). Nonparametric estimation and
testing of interaction in additive models. Econometric Theory,
Stanton, R. (1997). A nonparametric model of term structure dynamics and the market
price of interest rate risk. Journal of Finance, 52, 1973-2002.
Sun, Z. (1984). Asymptotic unbiased and strong consistency for density function estimator.
Acta Mathematica Sinica, 27, 769-782.
TJSTHEIM, D. and B. AUESTAD (1994a). Nonparametric identication of nonlinear
time series: Projections. Journal of the American Statistical Association, 89, 1398-
1409.
Tjstheim, D. and B. Auestad (1994b). Nonparametric identication of nonlinear time
series: Selecting signicant lags. Journal of the American Statistical Association, 89,
1410-1419.
van Dijk, D., T. Terasvirta, and P.H. Franses (2002). Smooth transition autoregressive
models - a survey of recent developments. Econometric Reviews, 21, 1-47.
Wand, M.P. and M.C. Jones (1995). Kernel Smoothing. London: Chapman and Hall.