WWW | Community | Store | Account | Contact Us
SHAZAM Homepage
Frequently Asked Questions about SHAZAM

This FAQ accompanies the latest version of SHAZAM. Some answers may not apply to earlier versions of SHAZAM.

Section 1: Working with it
Q1.1: How do I get help?
Q1.2: How do I get examples of SHAZAM command files?
Q1.3: How do I produce and print graphs?
Q1.4: What are the limits on problem size?

Section 2: Using Windows SHAZAM
Q2.1: Problems with loading a data set in repeated runs.

Section 3: Some SHAZAM tricks
Q3.1: Getting the minimum of more than 2 variables.
Q3.2: Getting the inverse of the NCDF.
Q3.3: How to sort with 2 sort variables.
Q3.4: How to compute skewness and kurtosis statistics.
Q3.5: Problem with READ / LIST FORMAT
Q3.6: Problem with GENR INDEX=INDEX/INDEX(3)
Q3.7: Problem with character strings in DO-loops
Q3.8: Last line in data file or command file not read !
Q3.9: Problem with scalar variables in TEST or RESTRICT commands.
Q3.10: Is there a command to compute factorials (that is, n!) ?
Q3.11: How to compute row and column totals of a matrix ?
Q3.12: How to include dummy variables in Box-Cox regression
Q3.13: How to calculate the normal pdf
Q3.14: Problem with divide by zero messages when calculating a matrix Hadamard product
Q3.15: How can a coefficient be restricted to the interval (-1,0) ?
Q3.16: How can a 12-period moving average by calculated ?

Section 4: Numerical problems
Q4.1: Why are random numbers different for different SHAZAM versions ?
Q4.2: ARIMA estimation results different from other packages.

Section 5: Technical Questions
Q5.1: For NL, what is the meaning of GTRANSPOSE*INVERSE(H)*G STATISTIC ?
Q5.2: For the MATRIX command, what algorithm is used to calculate eigenvalues and eigenvectors ?
Q5.3: What solution method is used for solving nonlinear systems of equations ?


Q1.1: How do I get help?

  • Read the manual
  • Consult the extensive Help Resources supplied with SHAZAM
  • View the pages at the SHAZAM web site including Command Reference, Examples, FAQs etc
  • Study the SHAZAM examples at REPEC
  • Visit the SHAZAM Community
  • Use the HELP command
  • Send an E-mail message to help@econometrics.com

Return to Top


Q1.2: How do I get examples of SHAZAM command files?

Return to Top


Q1.3: How do I produce and print graphs?

SHAZAM has an interface to the gnuplot package. The Professional Edition integrates Graph creation, editing and manipulation using menus and dialogs. Because Gnuplot is a command-driven interactive plotting program both the Standard and Professional editions of SHAZAM allow full access to all Gnuplot commands for full customization of your Graphs.

Return to Top


Q1.4: What are the limits on problem size?

SHAZAM sets defaults for memory limits and variable size limits. These can be changed with the PAR and SIZE commands so that there is no effective limit on problem size.

SHAZAM also sets a default for the command line length. This can be changed with the SET COMLEN= command.

Return to Top


Q2.1: Problems with loading a data set in repeated runs.

With Windows SHAZAM Versions 9 and 10 the READ command loads the data set starting at the first record of the data file (that is, the REWIND option is the default). This ensures that, in a Windows session, the data set will be re-loaded in repeated runs of a SHAZAM program. In previous versions of SHAZAM, the READ command set the NOREWIND option as the default.

If multiple READ commands are used to load data sequentially from a data file, then the NOREWIND option must be specified.

Return to Top


Q3.1: Getting the minimum of more than 2 variables.

Suppose you have variables X1, X2, X3, X4 and you would like to obtain the minimum in the variable VARMIN. You can do this with the SHAZAM command:

GENR VARMIN=MIN(X1,MIN(X2,MIN(X3,X4)))

Return to Top


Q3.2: Getting the inverse of the NCDF.

To obtain the inverse of the NCDF and save the result in the variable X use the command:

GENR P=1-P 
DISTRIB P / INV TYPE=NORMAL CRIT=X 

Return to Top


Q3.3: How to sort with 2 sort variables.

The SORT command in SHAZAM implements a fast sort algorithm. You can sort on two variables by the following method. Suppose your data set has variables A, B, C and D and the first sort variable is A and the second sort variable is B. Suppose that B never has values that exceed 1000. Then, the data set can be sorted by constructing a sort variable as follows:

GENR SORTVAR=A*1000+B 
SORT SORTVAR A B C D 

Return to Top


Q3.4: How to compute skewness and kurtosis statistics.

Suppose you have a variable X. You can get skewness and kurtosis with the command:

OLS X / GF

The GF option is generally intended to produce test statistics on the estimated residuals of the OLS regression. But when a single variable is specified you will get summary statistics for this variable. Note that the computations of skewness and excess kurtosis incorporate small sample adjustments and so the formula may be slightly different from formula given in some textbooks.

Return to Top


Q3.5 Problem with READ / LIST FORMAT

The first character is often interpreted as a carriage control character when output is printed to an output file. If a FORMAT command is used that does not allow for a blank as the first character then the data listing may appear with the first character missing. To avoid this the following method should be used to get a data listing of data that is read with the READ command.

FORMAT(A8,2F10.2) 
READ(filename) name var2 var3 / FORMAT 
FORMAT(1X,A8,2F10.2) 
PRINT name var2 var3 / FORMAT 

Instead of the LIST option on the READ command the PRINT command is used following the READ command. The FORMAT command is altered to start with a blank space.

Return to Top


Q3.6 Problem with GENR INDEX=INDEX/INDEX(3)

This example comes from Tim Coelli. Suppose that a price index series is available in the variable INDEX and the base year is to be set at period 3. The following command will give incorrect results.

GENR INDEX=INDEX/INDEX(3)

With the above command SHAZAM does a "dynamic calculation". SHAZAM computes:

   
  1)  INDEX(1) = INDEX(1) / INDEX(3) 
  2)  INDEX(2) = INDEX(2) / INDEX(3) 
  3)  INDEX(3) = INDEX(3) / INDEX(3) = 1 
  4)  INDEX(4) = INDEX(4) / INDEX(3)    (BUT INDEX(3) IS NOW 1 !) 
               = INDEX(4) 
  5)  INDEX(5) = INDEX(5) 
         ETC. 

To keep the calculation "static" the left hand side variable should be a new variable as in the following command:

GENR PRICE=INDEX/INDEX(3)

Return to Top


Q3.7 Problem with character strings in DO-loops

Character string names cannot be referenced with a DO-loop index. This is shown in the next list of SHAZAM commands.

DEMO 
SAMPLE 1 17 
* Specify some character strings 
LIST1: CONSUME PRICE 
LIST2: CONSUME INCOME 
LIST3: CONSUME INCOME PRICE 
* Run some regressions 
OLS [LIST1] 
OLS [LIST2] 
OLS [LIST3] 
* The following commands do NOT work ! 
DO #=1,3 
OLS [LIST#] 
ENDO 
STOP 

Return to Top


Q3.8: Last line in data file or command file not read !

There must be a hard return at the end of a file - otherwise SHAZAM may not recognize the last line. When an excel file is saved as a text file the return character may be placed at the end of the final line. Therefore a text editor must be used to add another return and place the cursor below the final line.

Return to Top


Q3.9: Problem with scalar variables in TEST or RESTRICT commands.

Use the GEN1 command to generate scalar variables for use in TEST or RESTRICT commands. If the GENR command is used to create constants then a syntax error may result when this variable is used in a TEST or RESTRICT command. For example, the following commands can be used:

GEN1 A=-1 
OLS CONSUME INCOME PRICE 
TEST INCOME=A*PRICE 

Return to Top


Q3.10: Is there a command to compute factorials (that is, n!) ?

An example of calculating factorials is available.

This should be used with caution since as n increases the value for n! rapidly becomes extremely large.

Return to Top


Q3.11: How to compute row and column totals of a matrix ?

You can use the STAT command with the SUMS= option to compute row and column totals of a matrix. This is shown in the commands below.

READ M / ROWS=5 COLS=4 
  1800 3000 4200 4800
  2000 3200 4200 5000 
  2000 3500 4500 5700 
  2000 3500 4800 6000 
  2100 3600 5000 6200 
* Save column totals of the matrix in the variable COLTOT 
STAT M / SUMS=COLTOT 
* Save row totals of the matrix in the variable ROWTOT 
MATRIX MT=M' 
STAT MT / SUMS=ROWTOT 
PRINT COLTOT ROWTOT 

Return to Top


Q3.12: How to include dummy variables in Box-Cox regression

0-1 dummy variables may not work in Box-Cox regression that is implemented with the BOX command. The easy solution is to redefine the dummy variables away from 0-1 to 1-2 or 1-e.

Return to Top


Q3.13: How to calculate the normal pdf

The probability density function for a standard normal random variable is:

      formula for normal pdf

The following GENR command can be used:

GENR PDF=EXP(-X**2/2)/SQRT(2*$PI)

To make the order of priority clear, an alternative command is:

GENR PDF2=EXP(-(X**2)/2)/SQRT(2*$PI)

To avoid programming errors when using the GENR command the recommended method for calculating the normal pdf is to use the DISTRIB command. The PDF= option is used to save the pdf in the variable specified. This is shown as follows.

SET NOOUTPUT
DISTRIB X / TYPE=NORMAL PDF=NPDF

Return to Top


Q3.14: Problem with divide by zero messages when calculating a matrix Hadamard product

Before the MATRIX command add the following commands:

SET MISSVALU=0
SET NOWARN NOWARNMISS

An example is shown below.

sample 1 5
read A / rows=5 cols=3 list
1 3 5
1 1 4
1 5 6
1 2 4
1 4 6
read B / rows=5 cols=3 list
1 3 5
1 0 4
0 5 6
1 2 0
0 4 6
SET MISSVALU=0
SET NOWARN NOWARNMISS
MATRIX PROD=A/(1/B)
SET MISSVALU=-99999
SET WARN WARNMISS
PRINT PROD
STOP

Return to Top


Q3.15: How can a coefficient be restricted to the interval (-1,0) ?

With nonlinear least squares, a logit transformation can be used to force a coefficient to be in the (0,1) interval. That is, the coefficient can be expressed as:

        1 / (1 + exp(b))

This is shown in the example below. A linear regression equation is considered. The coefficient on the price variable is restricted to be negative and between -1 and 0.

?DEMO
* Generate some artifical data - the coefficient on price is -0.5
GENR Y=100+0.9*INCOME-0.5*PRICE+NOR(4)
* Estimate with the restriction on the price coefficient.
NL 1 / RSTAT NCOEF=3
EQ Y=B0+B1*INCOME + (-1/(1+EXP(B2)))*PRICE
END
* Get the coefficient on price
TEST -1/(1+EXP(B2))
STOP

Return to Top


Q3.16: How can a 12-period moving average by calculated ?

The SMOOTH command in SHAZAM can generate moving averages. An example is below.

SAMPLE 1 462
READ (X.DAT) X / BYVAR
* Calculate a 12-period moving average
SMOOTH X / NMA=12 NOCENTRAL MAVE=MAVG
* Get some summary statistics
SAMPLE 12 462
STAT MAVG 
STOP

It is useful to note that SHAZAM programming commands can be used to replicate many calculations. The SHAZAM commands below show use the of a DO-loop for obtaining a moving average.

SAMPLE 1 462
READ (X.DAT) X / BYVAR
* Calculate a 12-period moving average
SAMPLE 12 462
GENR MAVG=X
DO #=1,11
GENR MAVG=MAVG+LAG(X,#)
ENDO
GENR MAVG=MAVG/12
STOP

Return to Top


Q4.1: Why are random numbers different for different SHAZAM versions ?

Different compilers may be used to make different SHAZAM versions for various operating systems and therefore, the random numbers generated by SHAZAM will be different.

To guarantee the same random numbers in a repeat run of an installed version of SHAZAM, use the command SET RANFIX at the start of the SHAZAM command file.

Alternatively, to ensure the same random numbers in various runs of SHAZAM, first save a set of random numbers to a data file and then load the random numbers from the file with a READ command.

Return to Top


Q4.2: ARIMA estimation results different from other packages.

The estimation of ARIMA models requires nonlinear algorithms and so there may be numerical differences in results reported by SHAZAM and other packages. The following differences should be noted when comparing results from SHAZAM and RATS.

Algorithm Differences

  • RATS uses a least squares estimation procedure like SHAZAM. The RATS manual indicates that a Gauss-Newton algorithm is used. SHAZAM uses a version of Marquardt's algorithm with numerical derivatives.
  • SHAZAM computes pre-sample values of the residuals by a back-forecasting method as described in Box-Jenkins. However, RATS does not use backcasting. RATS assumes that residuals are zero for all unobservable periods. This can be expected to result in numerical differences between RATS and SHAZAM.

Reporting Differences

  • RATS and SHAZAM use different signs for the MA coefficients. Therefore, the SHAZAM user needs to reverse the sign of the MA coefficient to give a comparison with the results from RATS.
  • RATS and SHAZAM use a different calculation method for the AIC and SC statistics. SHAZAM computes:
    AIC = LOG(SIG2) + 2 * K / N 
    SC =  LOG(SIG2) + K * LOG(N) / N 
    

    RATS computes:

    AIC = N * LOG(SSE) + 2 * K 
    SC = N * LOG(SSE) + K * LOG(N) 
    

A user has pointed out the following differences for ARIMA estimation in SHAZAM and MINITAB.

  • MINITAB has a different convergence criteria. It just uses the relative change in each parameter - SHAZAM will also converge when the relative change in the error sum of squares is "small".
  • MINITAB uses different default starting values. In SHAZAM starting values can be changed with the START= option.

Return to Top


Q5.1: For NL, what is the meaning of GTRANSPOSE*INVERSE(H)*G STATISTIC ?

This statistic has 2 purposes:

  • It is a method of measuring convergence. If the gradient is zero then the statistic will be zero.
  • If starting values are set and the option ITER=0 is used (no iterations) then the statistic will be an LM test against the unrestricted model. The degrees of freedom is the number of parameters.

Return to Top


Q5.2: For the MATRIX command, what algorithm is used to calculate eigenvalues and eigenvectors ?

A description of the algorithm is available.

Return to Top


Q5.3: What solution method is used for solving nonlinear systems of equations ?

Consider the problem of finding an n-dimensional solution vector x such that:

      f(x) = 0

where f is an n-dimensional function that is a system of nonlinear equations. The problem can be solved using iterative algorithms that are available with the NL command in SHAZAM.

An example is shown below. The SOLVE option on the NL command is used to solve a nonlinear set of equations.

An equivalent way of obtaining a solution is to view the problem as a minimization problem where the objective function is the sum of squares of the individual functions in the system. The MINFUNC option on the NL command can then be used to get the answer.

SAMPLE 1 1
* Define the system of 2 equations 
FX1: X1**2 + 3*X1*X2 - 22
FX2: X2**2 + 2*X1*X2 - 21
* Solve the 2 equations
NL 2 / NCOEF=2 SOLVE
EQ [FX1]
EQ [FX2]
COEF X1 1 X2 1 
END
* Repeat using the MINFUNC option 
NL 1 / NCOEF=2 MINFUNC
EQ  ([FX1])**2 + ([FX2])**2
COEF X1 1 X2 1
END
STOP