Outliers and Data Having Undue Influence

In using the powerful analysis techniques we have learned there is a potential problem for which we must be constantly vigilant. We know that SSE and estimators which minimize SSE are extremely sensitive to outliers.

Outliers are extreme observations that for one reason or another do not belong with the other observations in the DATA.

If least square regression estimators are routinely applied to data which contain a few wild observations, then the obtained estimates can be seriously misleading.

It is therefore critically important to investigate the data for the presence of outliers whenever least square regression procedures are used.

Causes of Outliers

1. Data entry errors can produce outliers.^*
2. Sometimes the cases are not a homogeneous set to which a single MODEL will apply, but rather a heterogeneous set of two or more types of cases. One of these types will be far more frequent than the other, forcing the few to be identified as outliers.
3. The third cause of outliers is produced by error distributions with "thick" tails, in which extreme observations occur with greater frequency than expected for a normal distribution. Least squares solutions are fairly robust to violations of the assumption that the errors are normally distributed, except when the violation is that the distribution has thick tails. Ironically, sampling distributions that look quite different from a normal distribution cause little trouble, while these thick tail distributions raise cane with F interpretations.

Scatter Plots

Detecting Outliers

1. The predictor(s) value(s) for the data point can be unusual.
2. The criterion value for the data point can be unusual.
3. The data point is unduly influencing the parameter estimates of the predictions.

If a data point has any of these three characteristics, it can be considered an outlier which requires attention from the researcher.

Unusual Predictor Values?

We do not have formal rules for determining the cut off for the lever, but there are informal guidelines.

According to Judd and McClelland, Velleman & Welsch (1981) suggest that levers two or three times the average ought to be considered large and in need of further attention in the data analysis.

Again according to Judd and McClelland, Huber (1981) suggests that any lever over .2 deserves special attention whenever n is reasonably large.

Statlets Output

Influential points
------------------------------------------------------
                           Mahalanobis
Row           Leverage        Distance           DFITS
------------------------------------------------------
6             0.759815         33.8814          8.2275
------------------------------------------------------
Average leverage = 0.15384615384615385
The table of influential data points lists all observations which
have leverage values greater than 3 times that of an average data
point, or which have an unusually large value of DFITS. Leverage is
a statistic which measures how influential each observation is in
determining the coefficients of the estimated model. DFITS is a
statistic which measures how much the estimated coefficients would
change if each observation was removed from the data set. In this
case, an average data point would have a leverage value equal to
0.15384615384615385. There is one data point with more than 3 times
the average leverage, but none with more than 5 times. There is one
data point with an unusually large value of DFITS. 

Is Yi Unusual?

The first question is unusual with respect to what? Obviously, we want to answer the question of whether a criterion value is unusual with respect to the model we are using.

Residuals look at differences between the model and the criterion value.

Raw residuals have two major problems. Their values depend on the scale of the variables, and, unusual criterion values tend to "grab" the regression line -- producing smaller residuals than when the criterion point is left out of the analysis.

Studentized deleted residuals solve both of these problems. In addition, none of the other commonly reported transformed residual scores detect outliers any better than the studentized deleted residual, and finally, if you square the studentized deleted residual you have the appropriate F statistic (remember student developed the t statistic) for testing an outlier model where a single parameter is used for the case in question, against a model where that parameter is not used for that particular case.

Different Residuals

Blue = All nine, Magenta = first eight values

In general, Judd & McClelland do not recommend that the squared studentized deleted residual or the F that results from evaluating the outlier model be used as a formal statistical test unless one has external information questioning the validity or reliability of a particular observation (don't go on a witch hunt).

Four rules-of-thumb:

1. Studentized deleted residuals with an absolute value less than 2 are not unusual.
2. If the value is greater than 2 it deserves a look.
3. If the value is greater than 4, than all alarm bells out to sound.
4. Accepting a MODEL for which a studentized deleted residual is greater than 4 could be quite misleading.

Statlets output

Unusual residuals
----------------------------------------------------------------------
                             Predicted                     Studentized
Row                  Y               Y       Residuals       Residuals
----------------------------------------------------------------------
6                 86.0         72.6715         13.3285            4.63
----------------------------------------------------------------------

Would Omission of the data point Dramatically Change the Regression Equation?

Cook's D tests to see whether the error in the model changes when a specific data value is either included, or excluded from the model.

1. One recommendation is to consider values to be large which exceed 4/PAn.
2. Another suggested rule is to consider any value greater than 1 or 2 as indicating that an observation requires a careful look.
3. Finally, some researchers look for gaps between the D values.

In summary, Cook's D assesses the global impact of each observation on the parameter estimates and, equivalently, on the predictions for all other observations. Large values of D identify those observations with large impacts. We therefore can use Cook's D to answer this third question.

Statlets Output

DFITS
       A statistic computed when fitting a multiple regression model to measure the change in
       each predicted value which would occur if a single data value was deleted. Large values
       correspond to points which have a big influence on the fitted model. 

The Statlet's application does calculate Cook's D values as shown in the Output tab below.

Judd & McClelland think that it is good practice to omit outliers from the analysis with the explicit admission in the report that there are some observations which are not understood. You can then report a good model for those observations which you do understand.

There is a good Exhibit (9.9) on page 231 in the text which shows what each type of outlier does to the regression research.

Statlet's Predicted Plot

This plot displays observed values of SAT versus values predicted by
the fitted model. The closer the points lie to the diagonal line,
the better the model at predicting the observed data. You should
look for various anomalies, such as increases in variability around
the line as the value of SAT increases (heteroscedasticity), or
individual data points which lie far away from the line (outliers).

Statlet's Residuals Plot

This plot displays the Studentized residuals versus values of
HSRANK. Any non-random pattern could indicate that the selected
model does not adequately describe the observed data. In addition,
any values outside the range of -3 to +3 could well be outliers. 

Influence Plots