Estimation with AR(1) Errors

## Estimation of Models with Autoregressive errors

Economic time series do not adjust instantaneously to changes in the economic environment. One example of a dynamic model is the regression model with first-order autoregressive errors (an AR(1) error model). The equation errors have the form: t =  t`-`1 + vt     with   `-`1 < < 1

where (`RHO`) is the autoregressive parameter and vt is another random error that is assumed to be zero mean, homoskedastic and serially uncorrelated.

Test procedures for detecting the presence of AR(1) errors were discussed earlier in this guide. Users should be reminded that the appearance of autocorrelated errors may reflect misspecification in the structural part of the equation rather than a misspecified error structure.

The SHAZAM `AUTO` command is available for the estimation of models with autoregressive errors. The general command format is:

 `AUTO depvar indeps / options `

where depvar is the dependent variable, indeps is a list of the explanatory variables and options is a list of desired options.

#### Cochrane-Orcutt iterative estimation

By default, SHAZAM assumes an AR(1) error model and implements model estimation by the Cochrane-Orcutt method. An iterative estimation procedure is used. The starting point is to obtain parameter estimates by OLS. The OLS residuals ê are then used to obtain an estimate of from the regression:

êt = êt`-`1 + vt     for   t = 2, ..., N

The estimate of is used to construct transformed observations (the first observation is given a special transformation) and parameter estimates are obtained by applying OLS to the transformed model. A new estimate of is computed and another round of parameter estimates is obtained. The iterations stop when successive estimates of differ by less than `0.001`.

It is worthwhile noting that this is an example of a nonlinear least squares estimator that is derived by minimizing a sum of squared errors function. A specification of the objective function is given in Griffiths, Hill and Judge [1993, Equation (16.4.4), p. 529]. The first observation is included in the objective function by a special transformation. In general, the solution of nonlinear least squares problems requires the use of numerical optimisation algorithms. The Cochrane-Orcutt iterative method is an example of a solution algorithm.

#### Other estimation approaches and more general error structures

Alternative estimation algorithms for the AR(1) error model are available. A number of these are implemented in SHAZAM as options on the `AUTO` command. The interested user can consult the SHAZAM User's Reference Manual.

The SHAZAM `AUTO` command can also estimate models with higher order autoregressive errors and models with moving average errors.

#### Tests for autocorrelation after correcting for AR(1) errors

After estimation with AR(1) errors it is useful to check if the vt errors are serially uncorrelated. The Durbin-Watson test is inappropriate because the transformed model incorporates a lagged dependent variable. A test that can be used is Durbin's h test. SHAZAM reports Durbin's h test when the `RSTAT` option is specified on the `AUTO` command.

#### Example

This example, from Gujarati, examines the relationship between the help-wanted index and the unemployment rate in the United States. The data set contains quarterly data from 1962 to 1967. Gujarati chooses a log-log model for the analysis. The SHAZAM command file (filename: `HWI.SHA`) that follows first estimates the model by OLS and tests for the possibility of AR(1) errors. Cochrane-Orcutt iterative estimation is then implemented.

 ```SAMPLE 1 24 READ (HWI.txt) DATE HWI URATE * Transform to logarithms GENR LHWI=LOG(HWI) GENR LURATE=LOG(URATE) * OLS estimation - test for autocorrelated errors OLS LHWI LURATE / RSTAT DWPVALUE LOGLOG * * Cochrane-Orcutt iterative estimation AUTO LHWI LURATE / RSTAT LOGLOG STOP ```

The SHAZAM output can be viewed. The OLS estimation results report:

 ``` DURBIN-WATSON = .9108 VON NEUMANN RATIO = .9504 RHO = .54571 ```

SHAZAM reports the p-value for the Durbin-Watson test statistic as `.000672`. This gives strong evidence for positive serial correlation in the residuals. The right of the output reports an estimate of the autoregressive parameter `RHO` as `0.54571`. This value is less than 1 in absolute value and so is in the acceptable region for stationarity. Therefore, this model is a candidate for estimation with AR(1) errors.

The iterations in the Cochrane-Orcutt estimation procedure are shown below.

 ``` ITERATION RHO LOG L.F. SSE 1 .00000 -82.8653 .74302E-01 2 .54571 -78.8009 .52180E-01 3 .57223 -78.8066 .52111E-01 4 .57836 -78.8108 .52106E-01 5 .57999 -78.8120 .52106E-01 6 .58044 -78.8124 .52106E-01 ```

The starting point at iteration 1 with `RHO=0` is OLS. Iteration 2 uses the `RHO` estimate computed from the OLS residuals (as reported on the OLS estimation output). The difference in the `RHO` estimate from iteration 5 to 6 is `0.58044-0.57999=0.00045`. This is less than `0.001` and so the iterations stop at iteration 6.

The final column (`SSE`) reports the sum of squared errors (this refers to the `v(t)` error). This is the value of the nonlinear least squares objective function that is being minimized. Each iteration should show a smaller value for the `SSE`. It can be noted that iterations 5 and 6 show little improvement in the `SSE`.

The final estimate for `RHO` is `0.58044`. It is important to take careful note of this value. A result that is not infrequently encountered is a `RHO` value near 1. This indicates non-stationarity and other modelling approaches may need to be investigated.

How successful was the model estimation procedure ? The `RSTAT` option on the `AUTO` command produces the following output after the display of the estimation results.

 ``` DURBIN-WATSON = 1.8594 VON NEUMANN RATIO = 1.9402 RHO = .03757 DURBIN H STATISTIC (ASYMPTOTIC NORMAL) = .31712 MODIFIED FOR AUTO ORDER=1 ```

The `RHO` value reported at the extreme right is now the estimate of the autoregressive parameter for the vt errors. (The previous `RHO` was the estimate of the autoregressive parameter for the t errors.)

Durbin's h statistic is computed to be `0.31712`. This is less than the 5% critical value from the standard normal distribution of 1.96 and so the null hypothesis of no serial correlation is not rejected. However, it can be noted that only 24 observations were available and Durbin's h test may not be reliable in small samples. [SHAZAM Guide home]

#### SHAZAM output : Cochrane-Orcutt Iterative Estimation

This example is from Gujarati [1995, Section 12.7, pp. 433+].

``` |_SAMPLE 1 24

UNIT 88 IS NOW ASSIGNED TO: HWI.txt
3 VARIABLES AND       24 OBSERVATIONS STARTING AT OBS       1

|_* Transform to logarithms
|_GENR LHWI=LOG(HWI)
|_GENR LURATE=LOG(URATE)

|_* OLS estimation - test for autocorrelated errors
|_OLS LHWI LURATE / RSTAT DWPVALUE LOGLOG

OLS ESTIMATION
24 OBSERVATIONS     DEPENDENT VARIABLE = LHWI
...NOTE..SAMPLE RANGE SET TO:    1,   24

DURBIN-WATSON STATISTIC  =   0.91077
DURBIN-WATSON POSITIVE AUTOCORRELATION TEST P-VALUE =    0.000672
NEGATIVE AUTOCORRELATION TEST P-VALUE =    0.999329

R-SQUARE =    .9550     R-SQUARE ADJUSTED =    .9530
VARIANCE OF THE ESTIMATE-SIGMA**2 =   .33773E-02
STANDARD ERROR OF THE ESTIMATE-SIGMA =   .58115E-01
SUM OF SQUARED ERRORS-SSE=   .74302E-01
MEAN OF DEPENDENT VARIABLE =   4.9226
LOG OF THE LIKELIHOOD FUNCTION(IF DEPVAR LOG) = -82.8653

VARIABLE   ESTIMATED  STANDARD   T-RATIO        PARTIAL STANDARDIZED ELASTICITY
NAME    COEFFICIENT   ERROR      22 DF   P-VALUE CORR. COEFFICIENT  AT MEANS
LURATE    -1.5375      .7114E-01  -21.61      .000 -.977     -.9772    -1.5375
CONSTANT   7.3084      .1110       65.82      .000  .997      .0000     7.3084

DURBIN-WATSON =  .9108    VON NEUMANN RATIO =  .9504    RHO =   .54571
RESIDUAL SUM =   .00000      RESIDUAL VARIANCE =   .33773E-02
SUM OF ABSOLUTE ERRORS=   1.1162
R-SQUARE BETWEEN OBSERVED AND PREDICTED =  .9550
R-SQUARE BETWEEN ANTILOGS OBSERVED AND PREDICTED =  .9563
RUNS TEST:    4 RUNS,   13 POS,    0 ZERO,   11 NEG  NORMAL STATISTIC = -3.7492

|_*
|_* Cochrane-Orcutt iterative estimation
|_AUTO LHWI LURATE / RSTAT LOGLOG

DEPENDENT VARIABLE =  LHWI
..NOTE..R-SQUARE,ANOVA,RESIDUALS DONE ON ORIGINAL VARS

LEAST SQUARES ESTIMATION             24 OBSERVATIONS
BY COCHRANE-ORCUTT TYPE PROCEDURE WITH CONVERGENCE =  .00100

ITERATION          RHO               LOG L.F.            SSE
1              .00000        -82.8653              .74302E-01
2              .54571        -78.8009              .52180E-01
3              .57223        -78.8066              .52111E-01
4              .57836        -78.8108              .52106E-01
5              .57999        -78.8120              .52106E-01
6              .58044        -78.8124              .52106E-01

LOG L.F. =   -78.8124       AT RHO =      .58044

ASYMPTOTIC  ASYMPTOTIC  ASYMPTOTIC
ESTIMATE    VARIANCE    ST.ERROR     T-RATIO
RHO         .58044      .02763      .16622     3.49203

R-SQUARE =    .9685     R-SQUARE ADJUSTED =    .9670
VARIANCE OF THE ESTIMATE-SIGMA**2 =   .23684E-02
STANDARD ERROR OF THE ESTIMATE-SIGMA =   .48667E-01
SUM OF SQUARED ERRORS-SSE=   .52106E-01
MEAN OF DEPENDENT VARIABLE =   4.9226
LOG OF THE LIKELIHOOD FUNCTION(IF DEPVAR LOG) = -78.8124

VARIABLE   ESTIMATED  STANDARD   T-RATIO        PARTIAL STANDARDIZED ELASTICITY
NAME    COEFFICIENT   ERROR      22 DF   P-VALUE CORR. COEFFICIENT  AT MEANS
LURATE    -1.4712      .1251      -11.76      .000 -.929     -.9351    -1.4712
CONSTANT   7.2077      .1955       36.87      .000  .992      .0000     7.2077

DURBIN-WATSON = 1.8594    VON NEUMANN RATIO = 1.9402    RHO =   .03757
RESIDUAL SUM =   .84449E-02  RESIDUAL VARIANCE =   .23717E-02
SUM OF ABSOLUTE ERRORS=   .90632
R-SQUARE BETWEEN OBSERVED AND PREDICTED =  .9684
R-SQUARE BETWEEN ANTILOGS OBSERVED AND PREDICTED =  .9710
RUNS TEST:   12 RUNS,   11 POS,    0 ZERO,   13 NEG  NORMAL STATISTIC =  -.3854
DURBIN H STATISTIC (ASYMPTOTIC NORMAL) =   .31712
MODIFIED FOR AUTO ORDER=1
|_STOP
``` [SHAZAM Guide home]