This FAQ accompanies SHAZAM Version 10.
Some answers may not apply to previous verions of SHAZAM.
Section 1: Working with it
Q1.1: How do I get help?
Q1.2: How do I get examples of SHAZAM command files?
Q1.3: How do I produce and print graphs?
Q1.4: What are the limits on problem size?
Section 2: Using Windows SHAZAM
Q2.1: Problems with loading a data set in repeated
runs.
Section 3: Some SHAZAM tricks
Q3.1: Getting the minimum of more than 2 variables.
Q3.2: Getting the inverse of the NCDF.
Q3.3: How to sort with 2 sort variables.
Q3.4: How to compute skewness and kurtosis statistics.
Q3.5: Problem with READ / LIST FORMAT
Q3.6: Problem with GENR INDEX=INDEX/INDEX(3)
Q3.7: Problem with character strings in DO-loops
Q3.8: Last line in data file or command file not read !
Q3.9: Problem with scalar variables in TEST or
RESTRICT commands.
Q3.10: Is there a command to compute factorials (that is, n!) ?
Q3.11: How to compute row and column totals of a matrix ?
Q3.12: How to include dummy variables in Box-Cox regression
Q3.13: How to calculate the normal pdf
Q3.14:
Problem with divide by zero messages when calculating a matrix Hadamard product
Q3.15:
How can a coefficient be restricted to the interval (-1,0) ?
Q3.16:
How can a 12-period moving average by calculated ?
Section 4: Numerical problems
Q4.1: Why are random numbers different for different
SHAZAM versions ?
Q4.2: ARIMA estimation results different from other packages.
Section 5: Technical Questions
Q5.1: For NL, what is the
meaning of GTRANSPOSE*INVERSE(H)*G STATISTIC ?
Q5.2: For the MATRIX command,
what algorithm is used to calculate eigenvalues and eigenvectors ?
Q5.3:
What solution method is used for solving nonlinear systems of equations ?
[SHAZAM home]
HELP command.
Q1.2: How do I get examples of SHAZAM command files?
Q1.3: How do I produce and print graphs?
SHAZAM has an interface to the gnuplot package. Gnuplot is a command-driven interactive function plotting program and is freely distributable.
Q1.4: What are the limits on problem size?
SHAZAM sets defaults for memory limits and variable size limits.
These can be changed with the PAR and SIZE commands so that there
is no effective limit on problem size.
SHAZAM also sets a default for the command line length. This can be
changed with the SET COMLEN= command.
Q2.1: Problems with loading a data set in repeated
runs.
With Windows SHAZAM Versions 9 and 10 the READ command
loads the data set starting at the first record of the data file
(that is, the REWIND option is the default).
This ensures that, in a Windows session, the data set will be re-loaded
in repeated runs of a SHAZAM program.
In previous versions of SHAZAM, the READ command
set the NOREWIND option as the default.
If multiple READ commands are used to load data
sequentially from a data file, then the NOREWIND option
must be specified.
Q3.1: Getting the minimum of more than 2 variables.
Suppose you have variables X1, X2, X3, X4 and you would
like to obtain the minimum in the variable VARMIN.
You can do this with the SHAZAM command:
GENR VARMIN=MIN(X1,MIN(X2,MIN(X3,X4)))
Q3.2: Getting the inverse of the NCDF.
To obtain the inverse of the NCDF and save the result in the variable X use the command:
GENR P=1-P DISTRIB P / INV TYPE=NORMAL CRIT=X
Q3.3: How to sort with 2 sort variables.
The SORT command in SHAZAM implements a fast sort algorithm. You can
sort on two variables by the following method. Suppose your data set
has variables A, B, C and D and the first sort variable is A and the
second sort variable is B. Suppose that B never has values that exceed
1000. Then, the data set can be sorted by constructing a sort variable
as follows:
GENR SORTVAR=A*1000+B SORT SORTVAR A B C D
Q3.4: How to compute skewness and kurtosis statistics.
Suppose you have a variable X. You can get skewness and kurtosis with the command:
OLS X / GF
The GF option is generally intended to produce test statistics on the
estimated residuals of the OLS regression. But when a single variable
is specified you will get summary statistics for this variable.
Note that the computations of skewness and excess kurtosis incorporate
small sample adjustments and so the formula may be slightly different
from formula given in some textbooks.
Q3.5 Problem with READ / LIST FORMAT
The first character is often interpreted as a carriage control
character when output is printed to an output file. If a FORMAT
command is used that does not allow for a blank as the first character
then the data listing may appear with the first character missing.
To avoid this the following method should be used to get a data listing
of data that is read with the READ command.
FORMAT(A8,2F10.2) READ(filename) name var2 var3 / FORMAT FORMAT(1X,A8,2F10.2) PRINT name var2 var3 / FORMAT
Instead of the LIST option on the READ command the PRINT command is used
following the READ command. The FORMAT command is altered to start with
a blank space.
Q3.6 Problem with GENR INDEX=INDEX/INDEX(3)
This example comes from Tim Coelli. Suppose that a price index series
is available in the variable INDEX and the base year is to be
set at period 3. The following command will give incorrect results.
GENR INDEX=INDEX/INDEX(3)
With the above command SHAZAM does a "dynamic calculation". SHAZAM computes:
1) INDEX(1) = INDEX(1) / INDEX(3)
2) INDEX(2) = INDEX(2) / INDEX(3)
3) INDEX(3) = INDEX(3) / INDEX(3) = 1
4) INDEX(4) = INDEX(4) / INDEX(3) (BUT INDEX(3) IS NOW 1 !)
= INDEX(4)
5) INDEX(5) = INDEX(5)
ETC.
To keep the calculation "static" the left hand side variable should be a new variable as in the following command:
GENR PRICE=INDEX/INDEX(3)
Q3.7 Problem with character strings in DO-loops
Character string names cannot be referenced with a DO-loop index. This is shown in the next list of SHAZAM commands.
DEMO SAMPLE 1 17 * Specify some character strings LIST1: CONSUME PRICE LIST2: CONSUME INCOME LIST3: CONSUME INCOME PRICE * Run some regressions OLS [LIST1] OLS [LIST2] OLS [LIST3] * The following commands do NOT work ! DO #=1,3 OLS [LIST#] ENDO STOP |
Q3.8: Last line in data file or command file not read !
There must be a hard return at the end of a file - otherwise SHAZAM may not recognize the last line. When an excel file is saved as a text file the return character may be placed at the end of the final line. Therefore a text editor must be used to add another return and place the cursor below the final line.
Q3.9: Problem with scalar variables in TEST or RESTRICT commands.
Use the GEN1 command to generate scalar variables for use in TEST or
RESTRICT commands. If the GENR command is used to create constants
then a syntax error may result when this variable is used in a
TEST or RESTRICT command. For example, the following commands can be used:
GEN1 A=-1 OLS CONSUME INCOME PRICE TEST INCOME=A*PRICE
Q3.10: Is there a command to compute factorials (that is, n!) ?
An example of calculating factorials is available.
This should be used with caution since as n increases the value for n! rapidly becomes extremely large.
Q3.11: How to compute row and column totals of a matrix ?
You can use the STAT command with the
SUMS= option to compute row and
column totals of a matrix. This is shown in the commands below.
READ M / ROWS=5 COLS=4 1800 3000 4200 4800 2000 3200 4200 5000 2000 3500 4500 5700 2000 3500 4800 6000 2100 3600 5000 6200 * Save column totals of the matrix in the variable COLTOT STAT M / SUMS=COLTOT * Save row totals of the matrix in the variable ROWTOT MATRIX MT=M' STAT MT / SUMS=ROWTOT PRINT COLTOT ROWTOT |
Q3.12: How to include dummy variables in Box-Cox regression
0-1 dummy variables may not work in Box-Cox regression that is implemented
with the BOX command.
The easy solution is to redefine the dummy variables away
from 0-1 to 1-2 or 1-e.
Q3.13: How to calculate the normal pdf
The probability density function for a standard normal random variable is:
The following GENR command can be used:
GENR PDF=EXP(-X**2/2)/SQRT(2*$PI)
To make the order of priority clear, an alternative command is:
GENR PDF2=EXP(-(X**2)/2)/SQRT(2*$PI)
To avoid programming errors when using the GENR
command the recommended method for calculating the normal pdf is
to use the DISTRIB command. The
PDF= option is used to save the pdf in the variable
specified. This is shown as follows.
SET NOOUTPUT DISTRIB X / TYPE=NORMAL PDF=NPDF
Before the MATRIX command add the following commands:
SET MISSVALU=0 SET NOWARN NOWARNMISS
An example is shown below.
sample 1 5 read A / rows=5 cols=3 list 1 3 5 1 1 4 1 5 6 1 2 4 1 4 6 read B / rows=5 cols=3 list 1 3 5 1 0 4 0 5 6 1 2 0 0 4 6 SET MISSVALU=0 SET NOWARN NOWARNMISS MATRIX PROD=A/(1/B) SET MISSVALU=-99999 SET WARN WARNMISS PRINT PROD STOP |
With nonlinear least squares, a logit transformation can be used to force a coefficient to be in the (0,1) interval. That is, the coefficient can be expressed as:
1 / (1 + exp(b))
This is shown in the example below. A linear regression equation is considered. The coefficient on the price variable is restricted to be negative and between -1 and 0.
?DEMO * Generate some artifical data - the coefficient on price is -0.5 GENR Y=100+0.9*INCOME-0.5*PRICE+NOR(4) * Estimate with the restriction on the price coefficient. NL 1 / RSTAT NCOEF=3 EQ Y=B0+B1*INCOME + (-1/(1+EXP(B2)))*PRICE END * Get the coefficient on price TEST -1/(1+EXP(B2)) STOP |
The SMOOTH command in SHAZAM can generate
moving averages. An example is below.
SAMPLE 1 462 READ (X.DAT) X / BYVAR * Calculate a 12-period moving average SMOOTH X / NMA=12 NOCENTRAL MAVE=MAVG * Get some summary statistics SAMPLE 12 462 STAT MAVG STOP |
It is useful to note that SHAZAM programming commands can
be used to replicate many calculations. The SHAZAM commands below
show use the of a DO-loop for obtaining
a moving average.
SAMPLE 1 462 READ (X.DAT) X / BYVAR * Calculate a 12-period moving average SAMPLE 12 462 GENR MAVG=X DO #=1,11 GENR MAVG=MAVG+LAG(X,#) ENDO GENR MAVG=MAVG/12 STOP |
Q4.1: Why are random numbers different for different
SHAZAM versions ?
Different compilers may be used to make different SHAZAM versions for various operating systems and therefore, the random numbers generated by SHAZAM will be different.
To guarantee the same random numbers in a repeat run of
an installed version of SHAZAM, use the command
SET RANFIX
Alternatively, to ensure the same random numbers in various
runs of SHAZAM, first save a set of random numbers to a data file
and then load the random numbers from the file with a
READ command.
Q4.2: ARIMA estimation results different from other packages.
The estimation of ARIMA models requires nonlinear algorithms and so there may be numerical differences in results reported by SHAZAM and other packages. The following differences should be noted when comparing results from SHAZAM and RATS.
AIC = LOG(SIG2) + 2 * K / N SC = LOG(SIG2) + K * LOG(N) / N
RATS computes:
AIC = N * LOG(SSE) + 2 * K SC = N * LOG(SSE) + K * LOG(N)
A user has pointed out the following differences for ARIMA estimation in SHAZAM and MINITAB.
START= option.
Q5.1: For NL, what is the meaning of GTRANSPOSE*INVERSE(H)*G STATISTIC ?
This statistic has 2 purposes:
ITER=0 is used
(no iterations) then the statistic will be an LM test against
the unrestricted model. The degrees of freedom is the number
of parameters.
Q5.2: For the
MATRIX command, what algorithm is used to calculate eigenvalues and
eigenvectors ?
A description of the algorithm is available.
Consider the problem of finding an n-dimensional solution vector x such that:
f(x) = 0
where f is an n-dimensional function that is a system of
nonlinear equations. The problem can be solved using iterative algorithms
that are available with the NL command in SHAZAM.
An example is shown below. The SOLVE option on
the NL command is used to solve a nonlinear set
of equations.
An equivalent way of obtaining a solution is to
view the problem as a minimization problem where the objective
function is the sum of squares of the individual functions in
the system. The MINFUNC option on the
NL command can then be used to get the answer.
SAMPLE 1 1 * Define the system of 2 equations FX1: X1**2 + 3*X1*X2 - 22 FX2: X2**2 + 2*X1*X2 - 21 * Solve the 2 equations NL 2 / NCOEF=2 SOLVE EQ [FX1] EQ [FX2] COEF X1 1 X2 1 END * Repeat using the MINFUNC option NL 1 / NCOEF=2 MINFUNC EQ ([FX1])**2 + ([FX2])**2 COEF X1 1 X2 1 END STOP |