A comparison of factor weighting methods in job evaluation: implications for compensation systems.

The concept of pay equity is important not only from the standpoint of employee morale, commitment, and performance (Lawler, 1981), but for compliance with equal employment opportunity laws such as the Equal Pay Act of 1963 and the Civil Rights Act of 1964. Recently, job evaluation has received

considerable attention as part of the continuing concern for sex-based pay equity (Arvey, 1986; Schwab, 1984). In the 1981 Supreme Court ruling, Gunther v. County of Washington, the court held that female dominated jobs were not required to be identical in content (i.e., duties) to jobs held primarily by males in order for federal courts to hear testimony of possible wage discrimination (452 U.S. 161 (1981)). Subsequently, job evaluation methodology and results have become prominent issues in sex-related discrimination cases (Cooper & Barrett, 1984; Milkovich & Newman, 1987), as well as a focal point of pay equity legislation being passed by local and state governments (Reichenberg, 1986).

In addition to legal considerations, important reasons for the use of job evaluation in compensation management are (1) having a rational and communicable basis for explaining different wage rates, (2) maintaining job satisfaction and minimizing grievances, (3) having a flexible basis for revising pay rates and establishing rates for new jobs, and (4) containing the administrative costs of employee compensation (Henderson, 1989; Milkovich & Newman, 1987).

Recently a number of measurement problems cited in a National Academy of Sciences study by Treiman and Hartmann (1981) have been addressed in the personnel literature. For example, research has focused upon such psychometric properties of job evaluation plans as reliability (Doverspike, Carlisi, Barrett, & Alexander, 1983), sex-biasing (Grams & Schwab, 1985; Mount & Ellis, 1987), and validity (Gomez-Mejia, Page, & Tarnow, 1982; Madigan & Hoover, 1986). However, an important methodological issue that has received scant attention in the literature is that of job evaluation factor weighting (Arvey, 1986; Treiman, 1984).

Weighting in Job Evaluation

Most formal job evaluation plans entail the measurement of job worth by ranking or rating jobs on a set of "compensable" factors (Henderson, 1989; Milkovich & Newman, 1987). Attributes of job worth measured by these factors typically fall into the categories of skill, responsibility, effort, and working conditions. (In fact, these 4 criteria are specified in the Equal Pay Act of 1963.) When combining the separate factor scores to form a composite, or when using the individual factors as variables in a prediction model, a decision must be made concerning the weighting of each factor. The determination of job evaluation factor weights is usually done in one of three ways: (1) a priori weights may be chosen and applied to the factors reflecting a subjective notion about worth; (2) weights may be derived empirically, from a statistical regression analysis of the relationship between job evaluation scores and criterion wage rates; or (3) job evaluation factors may be deemed equivalent in value and thus receive "equal" weights (Milkovich & Newman, 1987; Treiman, 1984).

It has been suggested that many job evaluation systems used today could have weighting schemes that reflect a bias against female dominated jobs, although supportive evidence in this regard is incomplete (Milkovich & Broderick, 1982; Remick, 1984; Treiman & Hartmann, 1981). Studies have not been published, for example, that compared the sex-related effects of alternative weighting methods. Weighting bias could occur in a number of ways. First, if a judgmental (a priori) weighting scheme is used, higher weights may be attached to factors that favor male dominated jobs (Remick, 1984; Treiman & Hartmann, 1981). Second, female jobs may be underpriced when the job evaluation plan is used as a "policy-capturing" technique (Schwab, 1984). That is, if female dominated jobs are adversely affected by systematic pay discrimination in the labor market, the use of a job evaluation system to predict (through statistical correlation) existing pay structures would tend to incorporate the wage bias from the market. Finally, weighting bias would occur if job evaluation factors that favored male dominated work tended to correlate highest with the criterion wages. One strategy used to prevent this kind of bias is to select a diverse set of factors for the job evaluation instrument (Doverspike & Barrett, 1984).

Another important consideration in choosing among different weighting approaches is the interpretability of the resulting wage structure; that is, the face validity of the structure in terms of participant understanding and acceptance (Milkovich & Newman, 1987; Tompkins, Brown, & McEwen, 1990). With regard to the issue of acceptance, job evaluation consultants and researchers have noted a trend toward increasing employee demands for participation in compensation design and for greater communication about the technical aspects of compensation methodology, such as job evaluation and market surveying (Mulcahey & Anderson, 1986; Olney, 1987; Pierson, Koziara, & Johannesson, 1984).

As discussed earlier, very little information is available in the job evaluation literature about the comparability of different weighting methods in terms of (1) sex-related pay equity, (2) criterion validity, or (3) pay classification structures. There is also a lack of discussion about the psychometric properties of job evaluation systems that might contribute to differences among weighting models. Therefore, the present research had two primary objectives: First, to review and discuss some of the psychometric parameters that could help to explain how and when different weighting methods might produce divergent results, and second, to empirically examine the effects of different weighting methods in a field study. Comparisons of different weighting methods were made within the framework of policy-capturing (i.e., wage prediction) because of the predominant use of this approach in job evaluation (Belcher & Atchison, 1987; Mahoney, Rosen, & Rynes, 1984).

Psychometric Considerations in Weighting

In discussing the manner in which psychometric characteristics of a measurement instrument may influence different weighting models, the concept of nominal versus effective weights should be clarified. When combining the separate scores of a set of variables into a total score (for each subject or case), the separate variables tend to contribute unequally to this composite. That is, the variables will have different effective (i.e., "true") weights (Richardson, 1941; Wang & Stanley, 1970). In job evaluation, for example, if jobs have very similar ratings on a particular factor (i.e., the factor has low variance) and the factor is highly correlated with other factors (i.e., it measures the same aspects of worth), this factor would obviously contribute very little to the differences among jobs in their total scores. Therefore, the factor would have a low effective weight.

The term "nominal" weighting refers to the transformation of scores for each variable through multiplication by a chosen numerical value (i.e., weight). In job evaluation, the process is variously called a priori or "committee" weighting, depending on how the weights are chosen. However, the important point to understand is that nominal weights will influence but not ultimately determine the effective weights of variables in a composite. Certainly, with regard to the goal of communicating methodological nuances to organizational members, differences in the effects on compensation plans of nominal versus effective weights could be important.

Little evidence is available in the job evaluation literature regarding the differential effects of alternative weighting methods on pay plans, or the psychometric properties of job evaluation systems that may underlie different weighting outcomes. Recently, an extensive review of the psychometric and statistical literature by the present authors (Davis & Sauser, in press) identified four parameters that would affect the relative predictive power of different weighting methods within the policy-capturing framework.

The first important psychometric consideration in choosing among weighting methods is the ratio of sample size (subjects/cases) to the number of predictors employed, or the n to p ratio. Generally, multiple regression weighting will require a larger sample size than other methods because it capitalizes on a larger number of predictors and because the statistical weights (i.e., beta weights) are optimized in the developmental sample (Cattin, 1980; Dawes & Corrigan, 1974; Schmidt, 1971).

A second important parameter in weighting is the level of |R.sup.2~ (i.e., correlation) between the predictors and criterion. It has been demonstrated that regression weighting will tend to provide superior prediction when the underlying model |R.sup.2~ values exceed .70 (Einhorn & Hogarth, 1975). In addition, when predictor-criterion correlations are high, it is possible to use smaller sample sizes to obtain optimal statistical weights in multiple regression. This fact is important because the preponderance of validity studies in job evaluation have consistently found |R.sup.2~ values for policy-capturing models in the range of .78 to .90 (Foster & Gimplin-Poris, 1984; Robinson, Wahlstrom, & Mecham, 1974; Schwab & Heneman, 1986). Therefore, statistical approaches to weighting may be feasible even for small organizations with limited job samples.

A third property of importance related to weighting is the degree of interrelatedness (i.e., multicollinearity) among the predictors (Darlington, 1978; Einhorn & Hogarth, 1975). In multiple regression models, as intercorrelations among the predictors increase, errors of estimating each regression weight will increase causing the weights to fluctuate across different samples. This means that weights developed in one sample (the benchmark sample) may not give high prediction in another sample (Mendenhall & Sincich, 1986). In job evaluation, the issue of cross-validity is important when the benchmark job sample constitutes a small percentage of the total jobs to which the plan will be applied.

A fourth and final parameter related to weighting is the degree of heterogeneity of the measurement instrument. Research by Laughlin (1978), Pruzek and Frederick (1978), and Darlington (1978) found that as "validity concentration" in a set of predictors increased, the less advantage multiple regression weighting had over nonstatistical methods. Validity concentration refers to the degree to which predictive power in a set of predictors is concentrated in a relatively few underlying dimensions.

In summary, we have identified and discussed the potential impact on weighting methods of four relevant psychometric properties. These were: (1) the n-to-p ratio, (2) the level of |R.sup.2~ between predictors and criterion, (3) the degree of multicollinearity among predictors, and (4) the degree of validity concentration in the predictor set. In the present study, four different weighting methods were applied in a policy-capturing approach to job evaluation. They were: (1) an unweighted summation of "raw" job evaluation scores, (2) unit weighting (weights of 1), (3) a priori committee weights, and (4) multiple regression weighting. The intention of our research was to determine if the different weighting methods employed had any differential effects when applied to job evaluation data collected in a public sector organization. In addition, we attempted to explain the results in relation to the underlying psychometric parameters previously discussed.

Methodology

Job Evaluation Instrument and Committee Process

An 8-factor job evaluation instrument was developed by two compensation consultants (the authors). Factors chosen for the instrument were intended to cover a wide range of jobs that are characteristic of municipal employment (e.g., administrative, technical, professional, and a variety of skilled to semi-skilled work such as equipment operation, maintenance, and clerical- office work). In addition, the factors were chosen to cover the four criteria specified in the Equal Pay Act of 1963 (i.e., Skill, Effort, Responsibility, and Work Conditions). The factors were named as follows: (1) Accountability (i.e., impact of decisions and errors); (2) Job Scope (i.e., standardization of duties and closeness of supervision received); (3) Communication Exchange (i.e., frequency, importance, and complexity of interpersonal communication); (4) Job Preparation (i.e., education, training and/or experience required); (5) Task Variety (i.e., diversity of duties performed); (6) Task Complexity (i.e., technical complexity and uncertainty); (7) Work Conditions (i.e., noise, temperature, lighting, and exertion); (8) Job Pressure (i.e., time pacing, deadlines, and hazards.

A job evaluation committee was trained and given the task of evaluating 52 full-time jobs in a small municipality (pop. 28,000). The committee included two department supervisors, a clerical employee, a department manager, and two citizens from the community. The members were asked to develop an a priori weighting model for the eight job evaluation factors, with each weight reflecting the relative importance of that factor to the organization. These numerical weights were expressed as a percent of 100 points.

Detailed and current descriptions of each job were distributed to the committee members and after independently rating each job, the members discussed the ratings and anonymously re-rated every job. A final evaluation score for each job was secured by averaging across the scores of the six evaluators. Three months after the original job evaluation, a random sample of 10 jobs was selected and reevaluated by the committee using the procedure described above. The test-retest reliability coefficient for the ten jobs was .987.

In order to compare the criterion validity (i.e., policy-capturing) of different factor weighting schemes, a published wage survey of fourteen similarly-sized cities was obtained from the state association of municipalities. A "going wage" for each of the 52 jobs was obtained by computing the median wage rate from the sample.

Data Analysis

Psychometric characteristics of the job evaluation instrument were examined by computing intercorrelation coefficients between the factors (based on the job evaluation ratings) and by computing a principal components analysis of the factor ratings. Using the different weighting methods previously described (i.e., unweighted, unit, judgmental, and multiple regression), four separate regression equations were calculated between the market wage rates and the job evaluation ratings (n = 52 jobs). The |R.sup.2~ value of each model was tested for statistical significance, as was the cross-validity of each |R.sup.2~. Differences between the weighting models were examined in three ways. First, significant differences between the |R.sup.2~ values and cross-valid |R.sup.2~ values were tested using a formula presented in Cohen and Cohen (1983) for dependency among predictors (i.e., where the prediction equations share the same criterion). In job evaluation, where the benchmark job sample is a small subset of the total jobs to which the pay policy formula will be applied, cross-validity will be more relevant as an index of predictive power. On the other hand, when the benchmark sample of jobs constitutes a majority of total jobs (as sometimes occurs in small organizations), predictive accuracy in the sample is more important. Second, the weighting models were compared in terms of their effects on male and female dominated jobs. To complete this analysis, we re-computed the prediction equations using only male jobs, which represented a discrimination-free sample (see Milkovich & Newman, 1987; Remick, 1984). Third, we computed intercorrelations between the predicted wage rates (policy rates) of all weighting models and calculated the rates of agreement in classification between all of the models. The predicted wages from each model were transformed into classification systems by establishing a minimum wage class ($6869 annually) and sequentially specifying higher class salaries with 10% intervals. For each weighting model, the 52 jobs were placed into the classes by comparing the class salaries with the predicted policy salaries. Agreement rates between models were based on the number of jobs that "fit" into the same class.

Study Findings

Characteristics of the Instrument

The average intercorrelation in ratings among the 8 job evaluation factors was r = .59. A factor analysis of the data revealed 2 underlying principal components that accounted for 91.1% of the variance in job ratings. The first principal component accounted for 76.9% of the score variance and was labelled "Skill-Responsibility." Six of the eight job evaluation factors had correlations of .89 or above with this component. The second principal component accounted for 14.2% of the variance and was highly correlated to the factors of Working Conditions (r = .98) and Job Pressure (r = .42).

These analyses indicated a great deal of multicollinearity and validity concentration in the set of factors used to evaluate the 52 municipal jobs, which is not uncommon in point-factor job evaluation systems (for example, see Gomez-Mejia, Page, & Tornow, 1982; Grant, 1951; Lawshe & Satter, 1944). Based upon these results, it might be logical to hypothesize that very small differences among the alternative weighting methods would be expected.

Policy-Capturing Accuracy

Table 1 presents results of the four regression analyses and shows that all coefficients were statistically significant. When the 8 female dominated jobs were removed from the regression analyses, the predictive accuracy of all models improved. This occurred because most of the female jobs had market rates substantially below their predicted equitable salary, given the level of job evaluation ratings assigned.

TABULAR DATA OMITTED

We used a formula recommended by Cohen and Cohen (1983) to test for significant differences between the R-coefficients and estimated cross-validities of the weighting models. In the combined male-female job sample, the estimated cross-validity of multiple regression weighting (|R.sup.2~ = .752) was significantly lower than that for all three of the other weighting methods (p. |is less than~ .01). No significant differences in sample |R.sup.2~-coefficients were found among the models. When only male dominated jobs were used, the R-coefficient for multiple regression was significantly higher than that for the other models (p |is less than~ .01). However, no significant differences were found among the estimated cross-validity coefficients of the different models.

Comparability of Models

Intercorrelations in predicted policy salaries among the 4 weighting models ranged from .95 to .99; in other words, the different policy-capturing models produced very similar pay hierarchies. In this study, salaries ranged from approximately $9,500 to $32,000. Given this large a range, it was not surprising that the weighting methods were able to achieve a highly similar ordering of salaries.

However, less agreement occurred among the methods in the placement of jobs into discrete salary classes (with 10% intervals). Agreement rates among the weighting models ranged from 59% to 94%, with the largest difference occurring between multiple regression and the other three models (mean agreement of 64%). Another relevant finding was that higher classification similarity occurred between the "equal" weighting methods (unweighted factors and unit weighting) and between the "differential" weighting methods (committee and regression weighting) than between the equal weighting versus differential weighting models. For example, 94 percent agreement was found between the unweighted and unit weighted systems, and 82 percent agreement was found between committee and regression weighting. In contrast, the mean agreement rate between the equal weighting and differential weighting models was 71%.

Effects on Female and Male Jobs

We calculated the predicted salaries and classifications for 8 female jobs and 12 male jobs that had similar job evaluation ratings. All of these jobs had average total ratings between 3.75 and 4.88 (less than one standard deviation apart). An examination of Table 2 will reveal the following: First, market rates for female jobs were below those for male jobs, which implied possible wage discrimination in the market (Milkovich & Newman, 1987; Remick, 1984). Incidentally, predicted salaries for both male and female jobs were lower when the 8 female jobs were included in the regression sample.

Second, the differential weighting schemes (i.e., committee and multiple regression) produced higher average salaries for the female jobs but lower average salaries for male jobs. A plausible explanation for this result is found in the factor score patterns for male versus female jobs in relation to the specific weights applied. Table 3 reports the actual weights derived by different methods (note that the unweighted raw score method is not shown, obviously because it entailed an unweighted summation of ratings). The female jobs had relatively higher ratings on the factors of Communication, Task Variety, and Task Complexity. As shown in Table 3, these factors received comparatively high positive weights under both the committee and multiple regression weighting schemes. On the other hand, the male jobs tended to have lower ratings on these factors but higher ratings on factors such as Job Preparation and Working Conditions, factors which had smaller and sometimes negative weights.

A third finding reported in Table 2 was the tendency for certain kinds of jobs across the male-female categorization to benefit from different weighting methods. For example, management-oriented jobs (e.g., Senior Citizens Director, Head Mechanic, Animal Control Officer, Parts Manager, and Planning/Codes Assistant) had lower salaries and classifications under the multiple regression model. This was also true of other administrative positions not displayed in Table 2. In contrast, the lower rated jobs in both male and female categories (e.g., most of the clerical jobs and vehicle-equipment operators) tended to fair better under the multiple regression scheme. In multiple regression, multicollinearity (that is, "redundancy") among predictors is handled by computing partial regression coefficients for each predictor. Where there are a number of highly intercorrelated predictors, some of the individual beta weights will be small, or even negative, as the redundant variance is partialled-out. In the present study, administrative positions received comparatively high ratings on the skill-responsibility group of factors (e.g., Accountability, Job Scope, Communication, and Job Preparation); however, because of the high interrelationships among these factors, some of them received small beta weights. In contrast, many of the non-managerial positions had their highest ratings on job evaluation factors with higher beta weights, such as Task Variety, Task Complexity, and Job Pressure.

TABULAR DATA OMITTED

A fourth finding of importance was related to the interpretability of the different pay plans. From this perspective, it would appear that the nonstatistical methods provided more explainable and defensible results. For example, Table 2 indicates that under multiple regression, the highest rated female job (i.e., Senior Citizens Director) would fall into the same class with the Secretary-Receptionist and below the positions of Mayor's Secretary and Administrative Assistant. Among the male dominated jobs, Table 2 indicates substantial fluctuations in pay classes under multiple regression, in contrast with the other weighting models. It might be difficult to explain, for example, how jobs such as Compactor Driver and Radio Dispatcher (with average ratings of 3.88 and 4.13, respectively) could be placed into the same pay class with higher rated positions such as Animal Control Officer and Parts Manager (average ratings of 4.88 and 4.75). In summary, our data indicated that multiple regression weighting produced a rather idiosyncratic pay structure that could be awkward to explain and justify to employees.

TABULAR DATA OMITTED

Implications for Practice

Results from our analyses indicated high levels of predictive accuracy for all four weighting methods, although multiple regression weighting had a slightly higher sample |R.sup.2~ than the other models. There were no significant differences among the models in estimated cross-validity. Still, given the small sample size used in the study and high multicollinearity among the job evaluation factors, these findings suggest that multiple regression analysis may be a viable weighting procedure even for small organizations, such as many city and county governments.

With respect to comparability of wage structures, the different weighting models produced similar rank orders of the jobs from high to low, as reflected in the high intercorrelations among the models. However, as the results indicated, differences occurred in the placement of jobs into discrete pay classes and in terms of the relative impact on male and female dominated jobs. Particularly, the multiple regression model tended to benefit female jobs and non-managerial jobs relative to male jobs and managerial positions. As previously discussed, multiple regression analysis derives predictor weights in a manner that considers redundancy (i.e., intercorrelations) among the variables. Therefore, in job evaluation the beta weights may take on rather unique and often dispersed values that are not directly related to the under lying organizational contributions made by the compensable factors. This kind of weighting scheme tends to incorporate the idiosyncratic score patterns of different jobs more than other models. Consequently, the compensation manager's task of explaining the weighting scheme and interpreting the results for employees may be more complicated.

At the present time in the development and application of point-factor job evaluation plans, no definitive solution has been suggested for dealing with the problem of multicollinearity among job evaluation factors. Past research has consistently reported high factor intercorrelations and has found that most of the variance in ratings can be accounted for by 2 to 3 underlying dimensions (Lawshe & Satter, 1944; Madigan & Hoover, 1986). One answer to the problem might be to evaluate jobs on fewer dimensions; however, this approach has been resisted for a number of reasons. First, employees may not understand or accept abbreviated factor plans that cover only a portion of the many job characteristics that are logically associated with job worth (Arvey, 1986; Thompkins et al., 1990). Second, an abbreviated factor system may not meet the requirements of the Equal Pay Act which specifies that "equal work" among different jobs should be measured in terms of four primary dimensions (i.e., skill, responsibility, effort, and working conditions). A third concern has to do with the need to achieve reliability of measurement, which past psycho metric research has shown to be related to instrument length (Cureton, 1965).

We suggest two possible strategies for handling the multicollinearity problem. One solution is simply to avoid multiple regression weighting when the resulting pay hierarchy appears too complicated to explain and defend. A second approach might be to conduct a factor analysis of the job evaluation scales and compute scores for each principal component. Following this procedure, the principle components could be weighted using multiple regression analysis. This approach allows the use of all of the original factors, but eliminates the problem of applying a set of idiosyncratic beta weights. To illustrate its potential effects, we computed factor scores for the 2 underlying dimensions found through a factor analysis of the present data. As discussed earlier, the two components were labeled "Skill-Responsibility" and "Working Conditions." Applying multiple regression, the two components had beta weights of .863 and .115, respectively. The pay classification hierarchy for this model exhibited greater similarity to the plans under unit and committee weighting than to the multiple regression system. Rates of agreement in salary classification between the principal components method and the other models were 83% with committee, 85% with unit, and 67% with multiple regression.

The present research and findings are mainly relevant within the context of a policy-capturing approach to job evaluation. Milkovich and Newman (1987) have discussed the relationship of job evaluation methodology to a broader range of organizational concerns, including climate and culture, labor relations, and employee participation and acceptance. These issues may at times have higher priorities in the overall job evaluation process than policy-capturing. Overall, our findings indicated that differences in pay structures can result from the application of alternative weighting approaches. The findings were somewhat surprising in view of the high multicollinearity and validity concentration that characterized the 8-factor job evaluation system used here. We would expect and predict even greater divergence between different weighting methods under the following conditions: (1) a larger and more diverse sample of benchmark jobs, (2) a more heterogeneous job evaluation instrument, and (3) lower multicollinearity in the predictor set. The important implication for compensation managers is that it would probably be wise to analyze the effects of a number of different weighting schemes before selecting a final plan. One encouraging finding from this study was the fact that a job evaluation committee comprised of supervisors and non-supervisory employees was able to develop a weighting scheme that performed about as well as unit weighting and multiple regression. Whether or not other committee weighting plans will do as well based on the use of other job evaluation instruments is a relevant question for future research. Within the framework of the present research, the following recommendations are offered to compensation managers concerning a weighting strategy:

* Develop and examine a number of different weighting schemes.

* Avoid commitment to an a priori weighting model until alternative weighting methods have been explored.

* Inform committee members about weighting methodology.

* Inform committee members about potential differences in outcomes.

* Establish a set of criteria, or objectives, for assessing the alternative models (e.g., predictive power, gender effects, and interpretability).

* Select a final weighting model based on the established objectives.

Bibliography

Arvey, R. B. "Sex-Bias in Job Evaluation Procedures." Personnel Psychology 39 (1986): 315-335.

Belcher, D. W. and Atchison, T. J. Compensation Administration. Englewood Cliffs, N.J.: Prentice-Hall, 1987.

Bellak, A. O., Bates, M. W. and Glasner, D. M. "Job Evaluation: Its Role in the Comparable Worth Debate." Public Personnel Management 12 (1983): 418-424.

Cattin, P. "Estimation of the Predictive Power of a Regression Model." Journal of Applied Psychology 65 (1980): 407-414.

Cooper, E. A. and Barrett, G. V. "Equal Pay and Gender: Implications of Court Cases for Personnel Practice." Academy of Management Review 9 (1984): 84-94.

Cohen, J. and Cohen, P. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Hillsdale, N.J.: Erlbaum Associates, 1983.

Cureton, E. E. "Reliability and Validity: Basic Assumptions and Experimental Designs." Educational and Psychological Measurement 25 (1965): 327-346.

Darlington, R. B. "Reduced-variance Regression." Psychological Bulletin 85 (1978): 1238-1255.

Davis, K. R., Jr. and Sauser, W. I., Jr. "Effects of Alternative Weighting Methods in a Policy-Capturing Approach to Job Evaluation: A Review and Empirical Investigation." Personnel Psychology, 1990 (in press).

Dawes, R. M. and Corrigan, B., "Linear Models in Decision Making." Psychological Bulletin 81 (1974): 95-106.

Doverspike, D. and Barrett, G. V. "An Internal Bias Analysis of a Job Evaluation Instrument." Journal of Applied Psychology 69 (1984): 648-662.

Doverspike, D., Carlisi, A. M., Barrett, G. V., and Alexander, R. A. "Generalizability Analysis of a Point-Method Job Evaluation Instrument." Journal of Applied Psychology 68 (1983): 476-483.

Einhorn, H. J. and Hogarth, R. M. "Unit Weighting Schemes for Decision Making." Organizational Behavior and Human Performance 13 (1975): 171-172.

Eyde, L. D. "Evaluating Job Evaluation: Emerging Research Issues for Comparable Worth Analysis." Public Personnel Management 12 (1983): 425-444.

Foster, K. E. and Gimplin-Poris, S. "Job Evaluation: It's Time to Face the Facts." Personnel Administration 29 (1984): 120-125.

Fraser, S. L., Cronshaw, S. F., and Alexander, R. A. "Generalizability Analysis of a Point Method Job Evaluation Instrument: A Field Study." Journal of Applied Psychology 69 (1984): 643-647.

Fulghum, J. B. "The Newest Balancing Act: A Comparable Worth Study." Personnel Journal 63 (1984): 32-38.

Grant, D. L. "An Analysis of a Point Rating Job Evaluation Plan." Journal of Applied Psychology 34 (1951): 236-240.

Grune, J. A. and Reder, N. "Pay Equity: An Innovative Public Policy Approach to Eliminating Sex-based Wage Discrimination." Public Personnel Management 12 (1984): 395-403.

Gomez-Mejia, L. R., Page, R. C. and Tornow, W. W. "A Comparison of the Practical Utility of Traditional, Statistical, and Hybrid Job Evaluation Approaches." Academy of Management Journal 25 (1982): 790-809.

Grams, R. and Schwab, D. R. "An Investigation of Systematic Gender-related Error in Job Evaluation." Academy of Management Journal 28 (1985): 279-290.

Henderson, R. I. Compensation Management. Englewood Cliffs, N.J.: Prentice Hall, 1989.

Laughlin, J. E. "Comment on 'Estimating Coefficients in Linear Models: It Don't Make No Nevermind'." Psychological Bulletin 85 (1978): 247-253.

Lawler, E. E. III. Pay and Organizational Change. Reading, MA: Addison-Wesley, 1981.

Lawshe, C. H. and Satter, G. A. "Studies in Job Evaluation: I. Factor Analysis of Point Ratings for Hourly-Paid Jobs." Journal of Applied Psychology 28 (1944): 189-198.

Leach, D. E. and Werley, E. L. "Comparable Worth, Job Evaluation and Wage Discrimination: The Employer Approaches Wage Gap Issues of the 1980s." Public Personnel Management 12 (1983): 345-357.

Madigan, R. M. and Hoover, D. J. "Effects of Alternative Job Evaluation Methods on Decision Involving Pay Equity." Academy of Management Journal 29 (1986): 84-100.

Mendenhall, W. and Sincich, T. A Second Course in Business Statistics: Regression Analysis. San Francisco: Dellen Publishing Company, 1986.

Milkovich, G. T. and Broderick, R. "Pay Discrimination: Legal Issues and Implications for Research." Industrial Relations. 21 (1982): 309-316.

Milkovich, G. T. and Newman, J. M. Compensation. Plano, TX: Business Publications, Inc., 1987.

Mount, M. K. and Ellis, R. A. "Investigation of Bias in Job Evaluation Ratings of Comparable Worth Study Participants." Personal Psychology 40 (1987): 85-96.

Mulcahey, R. W. and Anderson, J. E. "The Bargaining Battleground Called Comparable Worth." Public Personnel Management 15 (1986): 233-247.

Newman, W. "Statement to the Equal Pay Joint Committee." Public Personnel Management 12 (1983): 381-389.

Olney, P. B. "Meeting the Challenge of Comparable Worth: Part I." Compensation and Benefits Review 19 (1987): 34-44.

Pierson, D. A., Koziara, K. S., and Johannesson, R. "A Policy-capturing Application in a Union Setting." In H. Remick (Ed.), Comparable Worth and Wage Discrimination. Philadelphia, PA: Temple University Press, 1984.

Pruzek, R. M. and Frederick, B. C. "Weighting Predictors in Linear Models: Alternatives to Least Squares and Limitations of Equal Weights." Psychological Bulletin 85 (1978): 254-266.

Reichenberg, N. E. "Pay Equity in Review." Public Personnel Management 15 (1986): 211-231.

Remick, H. "Major Issues in a priori Applications." In Remick (Ed.), Comparable Worth and Wage Discrimination. Philadelphia, PA: Temple University Press, 1984.

Richardson, M. W. "The Combination of Measures." In P. Horst (Ed.), Bulletin Number 48: The Prediction of Personal Adjustment. New York: Social Science Research Council, 1941.

Robinson, D. D., Wahlstrom, O.W., and Mecham, R. C. "Comparison of Job Evaluation Methods: A Policy-Capturing Approach Using the Position Analysis Questionnaire." Journal of Applied Psychology 58 (1974): 633-637.

Schmidt, F. L. "The Relative Efficiency of Regression and Simple Unit Predictor Weights in Applied Differential Psychology." Educational and Psychological Measurement 31 (1971): 699-714.

Schwab, D. P. "Job Evaluation and Pay Setting: Concepts and Practices." In E. R. Livernash (Ed.), Comparable Worth Issues and Alternatives. Washington, D. C.: Equal Employment Advisory Council, 1984.

Straw, R. and Foged, L. E. "Job Evaluation: One Union's Experience." Industrial and Labor Relations Report Spring (1982): 24-26.

Tompkins, J., Brown, J., and McEwen, J. H. "Designing a Comparable Worth Based Job Evaluation System: Failure of an a priori Approach." Public Personnel Management 19 (1990): 31-42.

Treiman, D. J. "Effect of Choice Factors and Factor Weights in Job Evaluation." In Remick (Ed.), Comparable Worth and Wage Discrimination. Philadelphia, PA: Temple University Press, 1984.

Treiman, D. J. and Hartmann, H. J. (Eds), Women, Work, and Wages: Equal Pay for Jobs of Equal Value. Washington, D.C.: National Academy Press, 1981.

Wang, M. W. and Stanley, J. C. "Differential Weighting: A Review of Methods and Empirical Studies." Review of Educational Research 40 (1970): 663-705.

Wherry, R. J. "An Extension of the Doolittle Method to Simple Regression Problems." Journal of Educational Psychology 32 (1941): 459-464, Cases Gunther v. County of Washington, 25 FEP Cases 1521 (1981).

Cases

Gunther v. County of Washington, 25 FEP Cases 1521 (1981).

Kermit Davis is an Associate Professor of Management in the College of Business at Auburn University. His research and consulting interests are in the areas of compensation, employee selection, and performance appraisal.

William Sauser is Associate Vice President for Extension and is involved in the planning, coordination, and implementation of extension activities at Auburn University through the academic departments and specialized extension agencies.