Weighted Least Squares

Weighted Least Squares : 2-step Estimation


It may be the case that the variance functional form contains unknown parameters that must be estimated. An estimation approach is to first estimate the model by OLS and then use the OLS estimated residuals to construct an estimate of the error variance specification. In the second step weighted least squares can be applied. This is a special case of estimated generalized least squares (EGLS) or feasible generalized least squares.

This example returns to the Griffiths, Hill and Judge data set on wheat supply in Australia that was analyzed in the section on testing for heteroskedasticity. Recall that, following OLS estimation, the Goldfeld-Quandt test showed evidence to reject the null hypothesis of equal variance in two groups of observations. That is, the variance assumption is:

      var(et) = sigma12     for   t = 1, ..., N1     and

      var(et) = sigma22     for   t = N1+1, ..., N

where sigma12 is the error variance in the first subset and sigma22 is the error variance in the second subset. The problem is that the error variances are unknown.

The estimation approach is to first obtain estimates of the error variance by separate OLS estimation of each sample partition. This is shown in the SHAZAM commands (filename: WLS2.SHA) below. Note that, immeditately following estimation, the error variance is available in the SHAZAM temporary variable with the special name $SIG2. In the second step the estimated GLS estimates are obtained by using the weighted least squares option of SHAZAM.


SAMPLE 1 26
* Read the Griffiths, Hill and Judge (1993, p.491) wheat supply data set 
READ (WHEAT.txt) Q P
* Generate a time index
GENR T=TIME(0)
* Define a variable to use as the weight variable
GENR WT=0
*
* STEP 1 : Separate OLS regression for the two observation subsets
*   First subset
SAMPLE 1 13
OLS Q P T
* Save the estimated error variance in the weight variable
GENR WT=1/$SIG2
*   Second subset
SAMPLE 14 26
OLS Q P T
GENR WT=1/$SIG2
*
* STEP 2 : Get the weighted least squares (WLS) estimates
*          Griffiths, Hill and Judge (1993 p.499) - Equation (15.2.25)
SAMPLE 1 26
OLS Q P T / WEIGHT=WT NONORM NOMULSIGSQ

* ----------------------- comparison with OLS ---------------------------
* Compare the WLS estimates with OLS - use the HETCOV option to obtain
* standard errors that are adjusted for heteroskedastic errors.
OLS Q P T / HETCOV STDERR=SEHET
* Compare the HETCOV standard errors with the OLS standard errors.
*          Griffiths, Hill and Judge (1993 p.500) - Equation (15.2.29)
OLS Q P T / STDERR=SEOLS
* Obtain standard errors by adjusting for different error variance
* in the 2 sample partitions
GENR ONE=1
COPY P T ONE   X
GENR P=P/SQRT(WT)
GENR T=T/SQRT(WT)
GENR ONE=ONE/SQRT(WT)
COPY P T ONE   XW
*          Griffiths, Hill and Judge (1993 p.499) - Equation (15.2.26)
MATRIX SEWLS=SQRT(DIAG(INV(X'X)*(XW'XW)*INV(X'X)))
* Compare the various OLS standard errors
SAMPLE 1 3 
PRINT SEHET SEOLS SEWLS
STOP

In this application, the error terms of the transformed model are homoskedastic with error variance equal to 1. To make use of this result, the OLS command for the weighted least squares estimation specifies the options NONORM and NOMULSIGSQ. The NONORM option specifies that the weights are not normalized and the NOMULSIGSQ option specifies that the covariance matrix of the parameter estimates is estimated as the inverse of the cross-product matrix of the transformed observations. That is, the matrix is not multiplied by the error variance estimate.

The SHAZAM output can be viewed. The estimation results from the weighted least squares procedure (as reported in Griffiths, Hill and Judge [1993 p.499, Equation 15.2.25]) are as follows:

 
 VARIABLE   ESTIMATED  STANDARD   T-RATIO        PARTIAL STANDARDIZED ELASTICITY
   NAME    COEFFICIENT   ERROR      23 DF   P-VALUE CORR. COEFFICIENT  AT MEANS 
 P          21.720      8.805       2.467      .022  .457      .3661      .2249
 T          3.2834      .8117       4.045      .001  .645      .6003      .2406
 CONSTANT   138.05      12.65       10.91      .000  .915      .0000      .5345

Home [SHAZAM Guide home]

SHAZAM output : 2-Step GLS Estimation

The data set is from Griffiths, Hill and Judge [1993, Table 15.1, p. 491]. The 2-step GLS estimation procedure is described in Section 15.4.2, pp. 505-6 of this text.


 |_SAMPLE 1 26
 |_* Read the Griffiths, Hill and Judge (1993, p.491) wheat supply data set
 |_READ (WHEAT.txt) Q P
 UNIT 88 IS NOW ASSIGNED TO: WHEAT.txt
    2 VARIABLES AND       26 OBSERVATIONS STARTING AT OBS       1
 
 |_* Generate a time index
 |_GENR T=TIME(0)
 |_* Define a variable to use as the weight variable
 |_GENR WT=0
 |_*
 |_* STEP 1 : Separate OLS regression for the two observation subsets
 |_*   First subset
 |_SAMPLE 1 13
 |_OLS Q P T
  OLS ESTIMATION
       13 OBSERVATIONS     DEPENDENT VARIABLE = Q
 ...NOTE..SAMPLE RANGE SET TO:    1,   13
 
  R-SQUARE =    .6639     R-SQUARE ADJUSTED =    .5967
 VARIANCE OF THE ESTIMATE-SIGMA**2 =   641.64
 STANDARD ERROR OF THE ESTIMATE-SIGMA =   25.331
 SUM OF SQUARED ERRORS-SSE=   6416.4
 MEAN OF DEPENDENT VARIABLE =   203.65
 LOG OF THE LIKELIHOOD FUNCTION = -58.7570
 
 VARIABLE   ESTIMATED  STANDARD   T-RATIO        PARTIAL STANDARDIZED ELASTICITY
   NAME    COEFFICIENT   ERROR      10 DF   P-VALUE CORR. COEFFICIENT  AT MEANS 
 P          19.148      32.42       .5906      .568  .184      .1733      .1683
 T          6.8853      3.005       2.291      .045  .587      .6723      .2367
 CONSTANT   121.17      44.20       2.741      .021  .655      .0000      .5950

 |_* Save the estimated error variance in the weight variable
 |_GENR WT=1/$SIG2
 ..NOTE..CURRENT VALUE OF $SIG2=   641.64
 |_*   Second subset
 |_SAMPLE 14 26
 |_OLS Q P T
  OLS ESTIMATION
       13 OBSERVATIONS     DEPENDENT VARIABLE = Q
 ...NOTE..SAMPLE RANGE SET TO:   14,   26
 
  R-SQUARE =    .9010     R-SQUARE ADJUSTED =    .8812
 VARIANCE OF THE ESTIMATE-SIGMA**2 =   57.759
 STANDARD ERROR OF THE ESTIMATE-SIGMA =   7.5999
 SUM OF SQUARED ERRORS-SSE=   577.59
 MEAN OF DEPENDENT VARIABLE =   263.20
 LOG OF THE LIKELIHOOD FUNCTION = -43.1066
 
 VARIABLE   ESTIMATED  STANDARD   T-RATIO        PARTIAL STANDARDIZED ELASTICITY
   NAME    COEFFICIENT   ERROR      10 DF   P-VALUE CORR. COEFFICIENT  AT MEANS 
 P          22.063      9.281       2.377      .039  .601      .4175      .2308
 T          3.2574      .9944       3.276      .008  .719      .5753      .2475
 CONSTANT   137.29      14.68       9.354      .000  .947      .0000      .5216

 |_GENR WT=1/$SIG2
 ..NOTE..CURRENT VALUE OF $SIG2=   57.759
 |_*
 |_* STEP 2 : Get the weighted least squares (WLS) estimates
 |_*          Griffiths, Hill and Judge (1993 p.499) - Equation (15.2.25)
 |_SAMPLE 1 26
 |_OLS Q P T / WEIGHT=WT NONORM NOMULSIGSQ
  OLS ESTIMATION
       26 OBSERVATIONS     DEPENDENT VARIABLE = Q
 ...NOTE..SAMPLE RANGE SET TO:    1,   26
 SUM OF LOG(SQRT(ABS(WEIGHT)))  =  -68.382
 
  R-SQUARE =    .8799     R-SQUARE ADJUSTED =    .8695
 VARIANCE OF THE ESTIMATE-SIGMA**2 =   1.0271
 STANDARD ERROR OF THE ESTIMATE-SIGMA =   1.0135
 SUM OF SQUARED ERRORS-SSE=   23.624
 MEAN OF DEPENDENT VARIABLE =   258.28
 LOG OF THE LIKELIHOOD FUNCTION = -104.029
 
 VARIABLE   ESTIMATED  STANDARD   T-RATIO        PARTIAL STANDARDIZED ELASTICITY
   NAME    COEFFICIENT   ERROR      23 DF   P-VALUE CORR. COEFFICIENT  AT MEANS 
 P          21.720      8.805       2.467      .022  .457      .3661      .2249
 T          3.2834      .8117       4.045      .001  .645      .6003      .2406
 CONSTANT   138.05      12.65       10.91      .000  .915      .0000      .5345

 |_* -----------------------  Optional ------------------------------
 |_* Compare the WLS estimates with OLS - use the HETCOV option to obtain
 |_* standard errors that are adjusted for heteroskedastic errors.
 |_OLS Q P T / HETCOV STDERR=SEHET
  OLS ESTIMATION
       26 OBSERVATIONS     DEPENDENT VARIABLE = Q
 ...NOTE..SAMPLE RANGE SET TO:    1,   26
 
 USING HETEROSKEDASTICITY-CONSISTENT COVARIANCE MATRIX
 
  R-SQUARE =    .8089     R-SQUARE ADJUSTED =    .7923
 VARIANCE OF THE ESTIMATE-SIGMA**2 =   398.68
 STANDARD ERROR OF THE ESTIMATE-SIGMA =   19.967
 SUM OF SQUARED ERRORS-SSE=   9169.5
 MEAN OF DEPENDENT VARIABLE =   233.42
 LOG OF THE LIKELIHOOD FUNCTION = -113.145
 
 VARIABLE   ESTIMATED  STANDARD   T-RATIO        PARTIAL STANDARDIZED ELASTICITY
   NAME    COEFFICIENT   ERROR      23 DF   P-VALUE CORR. COEFFICIENT  AT MEANS 
 P          19.541      19.50       1.002      .327  .205      .2777      .1902
 T          3.6391      1.606       2.265      .033  .427      .6353      .2105
 CONSTANT   139.90      24.58       5.691      .000  .765      .0000      .5993

 |_* Compare the HETCOV standard errors with the OLS standard errors.
 |_*          Griffiths, Hill and Judge (1993 p.500) - Equation (15.2.29)
 |_OLS Q P T / STDERR=SEOLS
  OLS ESTIMATION
       26 OBSERVATIONS     DEPENDENT VARIABLE = Q
 ...NOTE..SAMPLE RANGE SET TO:    1,   26
 
  R-SQUARE =    .8089     R-SQUARE ADJUSTED =    .7923
 VARIANCE OF THE ESTIMATE-SIGMA**2 =   398.68
 STANDARD ERROR OF THE ESTIMATE-SIGMA =   19.967
 SUM OF SQUARED ERRORS-SSE=   9169.5
 MEAN OF DEPENDENT VARIABLE =   233.42
 LOG OF THE LIKELIHOOD FUNCTION = -113.145
 
 VARIABLE   ESTIMATED  STANDARD   T-RATIO        PARTIAL STANDARDIZED ELASTICITY
   NAME    COEFFICIENT   ERROR      23 DF   P-VALUE CORR. COEFFICIENT  AT MEANS 
 P          19.541      17.42       1.122      .273  .228      .2777      .1902
 T          3.6391      1.418       2.567      .017  .472      .6353      .2105
 CONSTANT   139.90      23.22       6.026      .000  .782      .0000      .5993

 |_* Obtain standard errors by adjusting for different error variance
 |_* in the 2 sample partitions
 |_GENR ONE=1
 |_COPY P T ONE   X
 |_GENR P=P/SQRT(WT)
 |_GENR T=T/SQRT(WT)
 |_GENR ONE=ONE/SQRT(WT)
 |_COPY P T ONE   XW
 |_*          Griffiths, Hill and Judge (1993 p.500) - Equation (15.2.26)
 |_MATRIX SEWLS=SQRT(DIAG(INV(X'X)*(XW'XW)*INV(X'X)))

 |_* Compare the various OLS standard errors
 |_SAMPLE 1 3
 |_PRINT SEHET SEOLS SEWLS
       SEHET          SEOLS          SEWLS
    19.50298       17.41501       15.85248
    1.606431       1.417651       1.321047
    24.58077       23.21761       21.55064
 |_STOP

Home [SHAZAM Guide home]