What you need to know about player raters

Two weeks ago, we covered the innings pace projection as something that fantasy league providers like Yahoo, CBS Sports, and ESPN give to customers, often with the result of creating a psychological imperative towards a goal and shaping the way that those in leagues manage their pitching staffs.

Another tool that fantasy league providers give out is a “player rater”—a mathematical algorithm that takes accumulated statistics to date and spits out how players measure up to each other in overall fantasy value for the current season.

The psychological impact of seeing how one player compares to the next should not be underestimated. Raise your hand if you’ve ever offered or considered a trade in consult with your league provider’s player rater.

We believe there are a few basic things that everybody needs to know about player raters:

First, not all player raters are the same.

For example, Yahoo’s player rater is a pretty dominant feature in its service. You can’t make a trade without seeing it. And if you are using it, you’ll see the top current five batters so far in the 2009 season listed this way:

1. Albert Pujols
2. Ian Kinsler
3. Evan Longoria
4. Raul Ibanez
5. Adam Jones (yes, almost unbelievably, he’s No. 5)

You’ll have to search a bit harder for CBS Sports’ player rater. But once you find it, you’ll see a different order:

1. Ian Kinsler
2. Albert Pujols
3. Carl Crawford
4. Raul Ibanez
5. Kevin Youkilis

Part of the reason for the difference is that the gurus behind CBS Sports, for whatever reason, favor the Head-to-Head points scoring format, and in the past few seasons, has juggled its point calculation formula on “standard leagues.” They’ve constructed their player rater to place more weight on metrics like walks and negative value on metrics like losses and blown saves.

Second, highly scarce statistics like stolen bases and saves tend to make a big impact on player raters.

Obviously, categorical scarcity should play a role in determining value, but at the start of a season, things like a single save or stolen base can be a little deceiving. Take these two players and figure out who is rated as more valuable:

Felix Hernandez: 34 innings, 4 wins, 36 strikeouts, 2.38 ERA, 1.09 WHIP, 0 saves
Jason Frasor: 10.1 innings, 4 wins, 6 strikeouts, 0.0 ERA, 0.48 WHIP, 1 saves

For those who guessed Frasor, congratulations.

It almost goes without saying that player raters measure retrospective value much better than prospective value. A pitcher with a poor strikeout rate may luck his way into a few wins and a good position on the player rater. That’s not to say that luck will continue.

Consider Ramon Ramirez, reliever for the Boston Red Sox, who currently has 2 wins and a 0.0 ERA despite having a strikeout rate of 4.8 per 9 innings pitched and a FIP (a measure of adjusted ERA based on peripherals and minus defense) of 3.1.

A comparative study on an unwritten rule of baseball.

Right now, Yahoo’s player rater judges Ramon Ramirez to be the 15th most valuable pitcher in baseball, ahead of Josh Johnson, among others. Strip away the vulture wins and adjust the ERA and Mr. Ramirez wouldn’t crack the top 200 pitchers.

On the other hand, don’t totally discount the value of a good middle reliever with solid ratios. Year after year, those in fantasy baseball leagues express some disbelief that middle relievers are indeed valuable in leagues that don’t count holds. But in this case, statistics don’t lie.

Yes, in 2008, Grant Balfour computed as a Top 100 fantasy player, ahead of Michael Young, Corey Hart, and Scott Kazmir. Considering the season that Balfour is having now, it may be tough to figure out those rock-solid middle relievers, but they appear high up on player raters for a good reason: A sub-2.0 ERA in 80 innings is indeed quite valuable. That’s a topic for another day, however.

Finally, the longer the season goes on, as sample sizes become larger and larger, player raters gain more credence, but also less worth.

At the start of the season, we all want players who are going to contribute to overall categorical success. As the season moves on, our position in some categories becomes fixed while the opportunity to move ahead in other categories offers promise. Making a player move becomes more dependent on the context of a particular league’s standings.

If there’s a conclusion to be drawn from all of this, it’s that going for the player who is rated as having the highest overall value is not always the smartest move.

Print This Post
Sort by:   newest | oldest | most voted
Briks, Philly
Briks, Philly
I use my own player rater system, and this might be a good place to fund out if people think i’ve used a sound methodology. I’m in a weekly head-to-head 5×5 league with normal cats except OBP instead of average. My league setting have been the same for the past couple years and i have been tracking how many of each stat each team has scored each week over that time. I then take the standard deviation of each category to figure out their value against each other. My idea is that the highest standard deviation would mean the lowest… Read more »
Connecticut Mike
Connecticut Mike
I have always been of the opinion that player raters are very flawed. The yahoo player rater seems to be the same regardless of the scoring system in the league you are in. For instance, if I am in a league with non standard categories (esp walks or OBP) it would stand to reason that some players would be more valuable in that context than the standard.   Further, positional requirements would seem to matter.  Brian McCann would seem to be more valuable in a two catcher league than a one catcher league, and would have very little value in… Read more »
Briks, that method makes sense to me, although there should probably be some adjustment made for plate appearances when determining value for OBP—a guy with 30 PA/week is going to have more of an impact on your team’s OBP than someone with 15 or 20 PA/week. The question, of course, is how you go about determining the correct value for plate appearances. If you just use total PA to date you’ll end up undervaluing players who have been on the DL earlier in the season or who weren’t on the roster. Maybe something like average PA per game spent on… Read more »
Good point, whitty. The most overlooked aspect of influencing rate stats is volume. Ichiro has not only been a huge asset in BA because of his actual performance, but because of his approach as well. He hits atop the order, is durable, and never walks. In short, he’s led the league in Abs in 6 of his 8 MLB seasons. Per 162, he averages nearly 100 more ABs than Pujols (picked him b/c he’s one of the few who can post an Ichiro-in-is-prime like BA). So, if both guys were to hit .330, Ichiro’s would be far more valuable, as… Read more »
Tyson Trautmann
Tyson Trautmann
The other frustrating thing about player raters is that most don’t take into account league specific settings, position scarcity, etc.  For example my baseball league is roto 6×6 (we add Holds and OPS) and folks are still using YRank for deals even though it doesn’t take those categories into account.  Position scarcity can be an even bigger factor, my 12 team baseball league added a second catcher and most people didn’t even stop to consider the enormous impact the change had on catcher values going into the draft. I’ve actually been working for several years now on a Dynamic Fantasy… Read more »