KGS Home List of Computer Contributions

Kansas Geological Survey, Computer Contributions 43, originally published in 1970


Minimum Entropy Criterion for Analytic Rotation

by Richard B. McCammon

University of Illinois at Chicago

small image of the cover of the book; red paper with dark blue text.

Originally published in 1970 as Kansas Geological Survey Computer Contributions 43.

Abstract

Minimum entropy is described as an analytic criterion for rotation to simple structure for both principal component and factor analysis data matrices. Minimum entropy rotated matrices come closer to achieving the ideal simple structure than is possible using the varimax method in the sense that a greater proportion of absolute values of the coefficients in the rotated matrix lie closer to zero. This allows greater ease of recognition of the underlying structure in the original data array. The concept of rotation is extended to include rotation of principal components. Numerical examples are given to illustrate the application of minimum entropy rotation in both principal component analysis and factor analysis.

Introduction

Factor analysis is recognized as an effective statistical tool for extracting meaning from large arrays of multivariate data. The computer programs for factor analysis that are available have reduced the necessary calculations to routine operations (Imbrie, 1953; Cooley and Lohnes, 1962; Manson and Imbrie, 1964; Sampson, 1968; Klovan, 1968; IBM, 1968). This has resulted in an increased use of factor analysis in geology (for a list of applications, see Harbaugh and Merriam, 1968). Factor analysis, however, is not as many would think one single operation, but rather a sequence of statistical procedures in which each procedure entering into the calculations is considered on an individual basis in formulating the final result. It is with one of these procedures that the present paper is concerned—analytic rotation.

Analytic rotation has persisted as a challenge to those engaged in developing factor analytic methods. The reason stems from the desire for a simple factor structure which, as a rule, direct factor solutions do not provide. Although the problem has long been recognized (Thurstone, 1947), it has only been since computers became generally available that objective analytic criteria for rotation to simple structure have been devised for practical usage. Of the several analytic criteria proposed, the varimax method due to Kaiser (1958) is by far the best known and most widely used procedure for rotating an initial factor matrix to a position of simple structure. The computer programs that have been made available have incorporated this method.

The purpose of this paper is to make available a computer program for an analytic criterion of factor rotation proposed earlier (McCammon, 1966), which approaches more closely the intuitive concept of simple structure. The method is based on the entropy concept as it is defined in information theory and is used to describe the state of a given rotated factor matrix derived from an initial factor matrix. Although the results are similar to those obtained by the varimax method, the basic difference is that for the minimum entropy criterion, a greater proportion of factor loading values are closer to zero. For large matrices, this can amount to a significant difference.

The second purpose of the paper is to indicate how the concept of analytic rotation can be extended to the method of principal components, a near relative of factor analysis. Principal component analysis differs from factor analysis in that the extracted components explain the total variance of a given set of variables rather than the intercorrelations. Principal component analysis finds useful application in reducing the number of variables in a study and also in problems of classification. The concept of simple structure can be utilized to interpret the principal components of a system of variables in terms of the individual variables. A given subset of principal components can be rotated into a position of simple structure while preserving the total variance. This is analogous in factor analysis to the rotation of initial factors to a position of simple structure while preserving the total correlation. The minimum entropy concept is applicable to both types of rotation.

Acknowledgments

The program was written during my employment with Gulf Research and Development Company. I wish to thank Chester Pelto for suggesting that the entropy function be used to define a criterion for rotation and for the advice offered during the course of the study. Tables 4 and 5 are reproduced with permission of the University of Chicago Press. The subroutine to invert a matrix in the computer program described is reproduced with the permission of IBM.

The complete text of this report is available as an Adobe Acrobat PDF file.

Read the PDF version (4.3 MB)


Kansas Geological Survey
Placed on web Aug. 29, 2019; originally published 1970.
Comments to webadmin@kgs.ku.edu
The URL for this page is http://www.kgs.ku.edu/Publications/Bulletins/CC/43/index.html