Understand manager scoring
An important consideration when looking at manager ratings is to understand any differences in the way that managers are rating their reports. For example, are some managers more lenient or generous than others? To identify such managers, we need an apples-to-apples comparison of all the ratings that managers provide so that we can rank manager ratings and identify outliers. However, in assessments based on job requirements that kind of comparison can be challenging to do.
For example, imagine one manager rated an employee as level 2 in a review, and a second manager rated a different employee as level 4. Is the second manager more generous than the first? The answer, at least in part, would depend on the requirements of the job in question. Is it a more senior job, more likely to require a higher level of proficiency? Or a more junior job, more likely to require a lower level? The job requirements provide the critical context that help you understand the ratings provided.
Manager scoring provides that context and compares the actual ratings provided by managers against the requirements defined on each job and rolls that comparison up into an average. This enables you to easily see which managers are consistently rating their reports as performing above the requirements of their job, or below.
Please note that there can be other factors at play for a manager score that is relatively high or low. For example, a low score might mean that the job requirements are too high or higher than the manager thinks they should be. In other words, manager scoring is not a definitive indicator of leniency or severity, but it does provide a useful data point when it comes to evaluating manager ratings.
How points are awarded
To calculate the score, we first assign points based on all of the manager’s assessment ratings in the review. Keep in mind that one manager might be evaluating multiple employees in a single review. The target is always the level specified on the job description.
Each question rated above target is awarded 2 points.
Each question rated at target is awarded 1 point.
Each question rated below target is awarded 0 points.
Steps used to calculate the manager score
1. Add all the ratings provided by the manager
Sum the points assigned by the manager for each subject in the review.
2. Determine the maximum possible score
Multiply the number of questions by 2 (the highest possible score per question) and then by the number of subjects.
3. Calculate the final manager score
Divide the total score by the maximum possible score, then multiply by 10 to get the final manager score.
Example calculation
Let’s say a review includes 3 subjects, each rated in an assessment with 8 questions. The manager assigns the following scores:
Employee 1 | Employee 2 | Employee 3 |
|---|---|---|
|
|
|
1. Add all employee scores
Total score = 12 + 11 + 14 = 37
2. Calculate the maximum possible score
Maximum possible score = 8 questions × 2 points × 3 employees = 48
3. Calculate the final manager score
Manager score = (37 ÷ 48) × 10 = 7.7
The final manager score is 7.7.
Notes
If the manager rates all subjects below target for every question, their score is 0.
If the manager rates all subjects at target for every question or rates a balanced average of scores below and above target, their score is 5.
If the manager rates all subjects above target for every question, their score is 10.