How can an organization determine whether its managers are doing a good job in performance appraisal and that the system is working well?
The single best test is that the organization gets 100 percent uncomplaining compliance with every procedural requirement of its performance appraisal system.
Tell Me More
There are many other checks the organization can make to be sure that the system is operating right. Here are some questions and issues to think about in evaluating the effectiveness of your performance management process:
Is there a significant difference between the ratings received by employees their first year on the job compared with the ratings that they receive in following years? It is logical to assume that performance appraisal ratings will go up in a person’s second and third year on the job as she gains more skill and increases in experience.
Is there a consistent distribution of performance appraisal ratings at various job or salary levels? In other words, for all people at one level, what percent are rated in the middle category, what percent are rated superior, what percent are rated needs improvement?
Is the percentage the same for higher and lower levels? In analyzing this, a reasonable assumption might be that as job level goes up (and therefore, we assume, training, skill, talent, etc.) we would expect to see a positive correlation between the percentage of individuals receiving a superior rating and the level of the job. If the reverse is true (i.e., lower level people are getting a higher percentage of superior ratings), it may indicate that senior managers have higher performance expectations of more highly paid personnel than they do of lower level employees. Whatever the explanation, it would be good to determine the cause. What percentage of people received the same rating in year two of employment as they did in year one?
What percent got a higher rating (moved from the middle rating to superior)? What percent got a lower rating (moved from superior to the middle rating, or from the middle rating down to fair)?
Is the distribution of ratings for employees at various levels reasonably consistent with ratings for employees in other divisions of the company at the same levels? If there is a difference, are there data available to explain the cause?
Are the overall performance appraisal ratings of employees who work in departments that are generally considered to be talent-rich, high-performing operations different from the average ratings of employees in departments that are reputed to harbor large numbers of has-beens and also-rans?
One might expect the average performance appraisal rating to be higher in a talent-rich department than in one that is filled with duds. Frequently, however, the reason high-performing departments and work units are that way is that the manager has high performance expectations and awards stellar ratings only when stellar performance is delivered. Too often, the higher the average performance appraisal rating, the poorer the overall performance of that unit.
Is the face validity of the performance appraisal ratings distribution acceptable to top management? For example, if 60 percent of all exempts are rated in the middle category, 30 percent as superior, and 10 percent spread among the other three categories, does that seem appropriate? Is the organization overall being too tough, too lenient, or about right?
If one-fifth of all employees were to be immediately dismissed because of a corporate mandate to reduce headcount by 20 percent, is senior management willing to rely exclusively on last year’s performance appraisal ratings in making the who goes/who stays decision? If not, why not?
Are there pockets of performance rating skew (positive or negative) within different departments or divisions? Are some managers particularly tough or lenient evaluators?
In addition to getting the answers to these questions, it is a worthwhile exercise to enter all of the data available into an electronic spreadsheet and play with the graphing function. A lot of times displaying data in a graphical format can help you see issues that raw numbers themselves don’t make clear.