Fantasy Value Above Replacement: Part One
In real-life baseball analysis, we have tools such as WAR to help us boil down a players’ contributions to a single number. Yet, in fantasy baseball, no such tool exists, sans a few “player raters” out there. The goal of this project was to create a number that would allow us to easily compare players’ values to each other in an effort to create accurate rankings before drafts begin. Today I will be rolling out a Fantasy Value Above Replacement metric, henceforth known as FVARz.
Using z-scores as the underpinning of this system can help keep things objective and accurate across the board. Simply start with a projected line in the players’ 5×5 categories and include it among the rest of the players at that position. Then, within each category, take the sum of the z-scores – the number of standard deviations away from the mean a player’s statistic is – and that determines the players’ value above the positional average. We then adjust for replacement level at that position, and you have a final number you can take to the bank. We can then use those overall value numbers to compare across positions, and even convert them into auction dollars.
For the sake of simplicity, all of these numbers are done for 12-team standard roto leagues, but the data can easily be manipulated to reflect a different number of teams, positional requirements, or player pools.
Step One: Comparing Inside the Position
Theory 1: Players can only play the positions that the game engine allows them, so those other players at their position are their only competition for a roster spot (sans UTIL).
All players’ raw numbers are compared only to those in their position grouping when it comes to overall value, because players can only play the position that the game engine allows. It does not make sense to directly compare Albert Pujols’ raw numbers to Robinson Cano’s raw numbers, for example, because they cannot occupy the same spot on your roster. Their raw data needs to be adjusted before we can directly compare them.
This is important because it is the way to calculate “position scarcity,” a key component in fantasy valuation. We can all agree that hitting 25 homers as a catcher is much more valuable than hitting 25 homers as an outfielder, and this will reflect that.
So compare the players’ numbers inside each position by using z-scores. Take the positional average and standard deviation in each 5×5 category, and calculate z-scores for each player’s performance in those 5×5 categories (we’ll talk more about how we properly do batting average and other rate stats later). We then just total up those z-scores, and call that out as our Fantasy Value Above Average, or FVAAz.
It is important to be consistent when deciding which players make it into your positional player pools. It is easiest to set an AB minimum (or IP for pitchers) and use it across the board in order to guarantee consistency. You can pick whatever number you like, but I’d recommend using a minimum around 400 AB’s.
Next, we’ll look at how we set our positional replacement levels, and why we do so.
1
awesome.
that’s all i have.
well the cats out of the bag now, isn’t she.
I’m not sure that using standard deviations from that position only are correct. As an example, let’s use these made-up but reasonable numbers for stolen bases:
OF: avg. 20, sdev 10, Carl Crawford proj. 55
1B: avg. 2, sdev 1, Albert Pujols proj. 12
Crawford is 3.5 standard deviations above average at OF, while Pujols is 10 standard deviations above average at 1B. But certainly Pujols’s 12 SB are not more valuable than Crawford’s 55.
Setting different averages for each position is good, and using standard deviations helps say how 10 SB is more valuable than 10 RBI, for example. But it should be a single sdev value for each stat, not each position, I believe.
To go a little more realistic, answer me this:
Isn’t it harder to find 10 SB from a 1B than 25 from an OF? If so, how else would we reflect that?
The other half of that question is why you’d want to reflect it.
Sure, you’re not going to get 25 SB from your 1B, but that doesn’t make those 10 SB really really valuable. Instead you should choose to spend your money (or draft picks) on the 25 SB OF or on 1Bs who give you other stats.
The value of the 25 SB from the outfield is going to depend on how easily it is to get *some* SB from a worse OF if you take the power-based 1B first. If OF rep level is 20 SBs, then the 25 SB aren’t valuable. But if rep level is 10 SB, they ARE. That’s rep level, which shifts the value of SB by position additively. What you’re talking about is stretching the value of SB multiplicatively.
Sweet…this is what I’ve been doing for the last 2 seasons with my personal rankings. Glad to see I’ll have someone who can maybe correct some of my errors along the way to fine tune my rankings.
In response to jrogers though – Having done this the past 2 years I’ve discovered that despite 1B not being known for steals, my average and standard deviation for 1B the last 3 seasons (2008-2010) had both an average and SD of around 4, so Pujols is really only 2 standard deviations above average.
I don’t recall what numbers I found for OF, but those numbers you have are definitely over shooting the mark, my guess is the average+1SD is around 20, not 30.
The exact numbers are unimportant*; rather, I was trying to illustrate the point that z-scores using different standard deviations for each position can be problematic.
*Though, if we want to get into it, an average and standard deviation of 4 SB for 1B suggests that 32%** of 1B have a SB total outside the range of 0 – 8, which seems unlikely. :)
**Yeah, I know that’s not exact either, and the distribution can’t be a gaussian since you can’t go negative, but that also suggests to me that the standard deviation should be less than 4.
I agree, it doesn’t make much sense to do it this way. Since all offensive players will be contributing to the same statistical categories, regardless of their position, they should all be compared, regardless of position for the sake of obtaining z-scores.
To account for scarcity or plenty at particular positions a positional adjustment should be applied.
I’ll transfer my question here from Twitter — more space for discussion.
I’m concerned that computing variation within positions will introduce false positional differences. For example, let’s assume that HRs for catchers are more tightly bunched than HRs for outfielders (which seems reasonable given that catchers receives less playing time generally.) So maybe the standard deviation for catcher HRs is 7, while it’s 10 for OF. In Zach’s system, an extra 10 HRs from an OF is worth an extra 7 HRs from a CA.
Is that true? I don’t think so. An extra HR is an extra HR. What we care about is the number of extra HRs. That’s what Zach tackles in Part 2, and I agree with that approach.
Am I missing something about what this variation-within-position analysis is trying to do?
That’s a good way of phrasing what I was trying to say, as well. Getting 25 HR from your C is better than getting 25 HR from your 1B. BUT, getting 10 HR over replacement level should be the same value for each.
Wasn’t sure whether to post this here or back with the first comments, so decided to go here. One thing I think was missed is that a homer is a homer is a homer after the season starts but not before. I know this seems counterintutive, but if you think about it, it’ll start to make sense. Drafting and winning are fundamentally different activities. One is about valuing a roster, the other is about a process of arriving at an optimized roster (relative to your projections of choice, of course). Once you’ve drafted, there is no longer any such thing as positional scarcity because all the positions are filled up. So what you really need to realize is that absolute values and draft rankings are a totally different thing.
Calculating absolute value is pretty straight forward and should not consider position, obviously. However, draft rankings are a game theory activity with a range of probabilities and some optimal paths based on the available information at any point in time in the draft, including: your roster construction relative to category targets for victory, the current construction of your opponent rosters, available players and deviation of dollar values at those positions, scarcity of production across stat categories (both absolute and within a category).
Basically you need a supercomputer that recalculates to optimal after every pick. I get around this by trying to do some best guess short cuts. I rate all players absolute dollar value (unadjusted for position) and calculate the standard deviation of dollar value at each position and the distance from the mean of each player (I also kind of like to look at the median as well as the mean) and use that to sort players into tier. Interesting that fantasy experts have settled on tiers as a valuable way to look at the draft as it basically kind of simulates looking at standard deviation without doing so explicitly. It gives you a kind of shorthand for where scarcity begins to override absolute value as a consideration.
You use different averages, but not different standard deviations. 10 SB from a 1B is more helpful than 25 from an OF… but not as valuable as 55 from an OF. I’ll take (55 at OF + 0 at 1B) over (10 at 1B + 20 at OF).
Sky / jrogers, I just asked Tango this question, and he said
‘I don’t see much if any difference between comparing to positional average or last player at position, as you’ll end up getting to the same spot anyway (presuming a continuous kind of distribution).
The choice is:
(HR – posHR) / divider + constant
or
HR/divider + posConstant
You still get the same thing, basically.
However, the standard deviation (the divider) used MUST, ABSOLUTELY MUST be based on the entire pool, not the positional pool.
I don’t know what Zach does.”
http://www.insidethebook.com/ee/index.php/site/comments/economics_of_fantasy_valuation/
Right. That’s what I’m saying.
Another, extreme example: Let’s say there are 10 CA and 10 2B.
CA SB: 5, 5, 5, 10, 10, 10, 10, 15, 15, 20
2B SB: 10, 10, 10, 15, 15, 15, 20, 25, 35, 45
CA mean of SB: 10.5
CA SB above rep: 55
CA s.d. of SB: 4.97
CA total SB s.d.’s above rep: 11.1
2B mean of SB: 20
2B SB above rep: 100
2B s.d. of SB: 11.9
2B total SB s.d.’s above rep: 8.5
So, if you use Zach’s method of s.d.’s within position, CA’s have MORE SB value than 2B. But if you use a simple rep level adjustment, 2Bs provide almost twice as many marginal SBs. If you want SBs, do you want to spend your money on these CAs as a whole, or these 2Bs?
“However, the standard deviation (the divider) used MUST, ABSOLUTELY MUST be based on the entire pool, not the positional pool.”
If this were “real” baseball, sure, that is absolutely correct because any player can play any position at any time, and then we adjust for it. However, this is not the case in fantasy baseball. Players can’t play any position we want, and so it doesn’t make sense to compare their raw numbers to the raw numbers of a player they aren’t really competing with for a roster spot.
They’re not competing for a roster spot, but they ARE competing for a draft pick. (Or, they’re competing for your auction dollars.)
No Zach, Tom is (as usual) absolutely right and this is a big thing you are missing.
The goal (for counting stat categories in roto) is to maximize the number of events, e.g. SB or HR or RBI. If you can increase your team’s HR total by 10, it doesn’t matter whether it comes from catcher or shortstop or OF, your team has 10 more HR. Period.
As demonstrated by a few posters above who noted the same flaw, the z-scores approach can lead to some weird scores when you create a deviation by position. A catcher who is 7 SB above “replacement” could be (according to z-scores) more valuable than an OF who is 18 SB above “replacement”, because the STDEV for catcher’s SB would be much smaller than that for OF. Which is completely nonsensical.
As said eloquently above: “getting 10 HR over replacement level should be the same value for each [position]“.
or to put it simply: the marginal value of 10 extra SB or HR should be identical regardless of position.
Roto scoring is linear — an extra 10 SB or an extra 10 HR absolutely HAS TO have the same point value in your player scoring system.
This cannot happen if you compute z-scores by position, because the denominator of (marginal increase in stat)/(deviation) will be different for each position.
Just a general question, but why use the z-score as a denominator for the counting stats (assuming normality in the distributions)?
I use the average of two systems, one which uses the z-score (a la LastPlayerPicked), and another where I will use the league’s previous year’s standings (or the average of many years) and use the number of units needed to gain one point in the standings.
For instance: 12 team league, best team had 150 SBs, worst 30 SB. On average it takes (150-30)/(12-1) = 10.9 SB’s to gain a point in the standings. I will use 10.9 as the denominator instead of the leaguewise standard deviation.
There are some variation between the two systems, but when normalized they come somewhat close to each other.
YES. This is the correct way to do it, much better than z-scores.
Although I have tried it both ways and, like you, have found that the z-scores system will correlate very strongly with the way you propose.
I agree wholeheartedly with Sky & jrodgers.
Having been doing z-scores myself as a means of creating a single ‘value’, I tried including relative positional valuation within categories up front and it caused the exact mess Sky describes above. It causes players playing positions that have relative extreme weakness in a given category, to have their values inflated just because they do relatively well in the weak category (e.g. Pujols SBs).
Also, z-scores based upon relative positional strength become totally incomparable when it comes to UTIL/Corner Infield/Middle Infield positions. Same with players that have multiple position eligibility – are Youkilis’ home runs really worth more to your team based where you play him (I think not, as it is the same home runs contributing to your totals), or is his value as a player more valuable generally vs other players you could start at a given position?
I have found in my experience that it is far better to do z-scores based upon the expected total universe (or my approximation thereabouts) of fantasy worthy hitters. Then, once these values are established, you can then obtain value-above-replacement numbers at specific positions once you group them that way. This allows you to take the player with multiple position eligibility and see where he ranks at different positions – his main score does not change, though, just his relative value depending upon the other players at a position.
Larry – do you do anything different than what last player picked does?
What I do is quite similar to Last Player Picked. The only (slight) difference would be in the process of getting the category averages – whereas they talk to iterations, I use an auto-updating formula instead. Thus, the iterations happen if players move in/out, but I just do not actively need to repeat the process.
Last Player Picked is a good site – I just came across it this year and recognized its similarities in how I have approached draft prep for a few years. I recommend checking it out, for those that have not.
p.s.
Zach does some great work here and at Roto Hardball – I commend the amount of great content he puts out and his willingness to explain in-depth his thinking
“We can all agree that hitting 25 homers as a catcher is much more valuable than hitting 25 homers as an outfielder, and this will reflect that.”
I like what Sky Kalkman said. All detailed math aside tho, yeah 25 homers is more valuable at catcher than at outfielder in a vacuum from a positional standpoint, but not if it’s at the expense of homers at the outfield slot, especially in regards to roto leagues. If acquiring a 25 homer catcher in the draft was the reason you have a 10 homer guy at your 3rd outfield slot, that’s no different than if acquiring a 25 homer guy at your 3rd outfield slot was the reason you have a 10 homer guy at catcher.
I like the idea and that article its interesting fasho, but I do think that this kinda discredits the idea of why people draft a player like Jacoby ellsbury in a roto league. In most of those situations (and let’s assume that team is 1st in steals) those steals that pujols acquires that is supposed to separate him from the field become unnecessary if your dominating that category with or without those steals.
This is my first year actually constructing my own rankings, and I was doing something slightly different. I went through each category and calculated league wide mean and SD, then calculated z-score for each player based on his projections and the league wide numbers. Then, I added up each player’s z-scores in all categories in order to get a sort of raw ranking of players. Then I took that number for each player, calculated averages by position, and used that number as a positional modifier for each position, which I subtracted from each players raw ranking total. I think this focuses less on which positions contribute to which categories.
Can anybody offer some critique on my method?
This may be the same thing as what Sir Larry does, I’m not completely sure.
You’re doing pretty damn well.
Looks quite good…I agree
Interesting take. In our RotoChamp software we take the replacement player at each position and adjust their z-scores up/down so that the replacement player at each position ends with a zero value, which in turn makes them a $1 auction player when we we calculate values.
In your calculation I suspect that you would have some replacement players with slightly negative values (I’d guess catchers, but not sure) and others with values slightly above $1 (maybe the stronger positions???).
If we get time, I might test your approach to see the differences.
Thanks for the feedback. One thing that I’m kind of unsure of is whether to use the z-score of the replacement level player (at each position), or the mean of all above replacement level players (also at each position) for my positional modifier. I’m currently using the latter, as basing the system on a single replacement level player seems kind of arbitrary, I’d rather base it on the average of what the other teams in the league are getting. Any thoughts on this?
Nick,
I’ve been doing what you describe for a few years and I’ve always used the replacement player at that position, rather than the mean. I do this based on what I’ve read at Tango’s blog and at Last Player Picked. In a 12-team league, I want the #12 catcher (or 1B or 2B…) to be worth $1, so that all the catchers who are more valuable are higher in value and the catchers below replacement level are worth negative-value.
If you’re doing this in Excel, maybe you should try it both ways and see how your values come out and see what looks better to you.
Agree with using STDEV on the population as a whole, not position by position. Been stated so I won’t re-hash.
Couple of questions – first is how do you treat bench players if you bid on them with your auction dollars? Essentially extra utility men? Do you back them out and back out $1 per bench spot so you’re only calculating values on starters? Something else?
Second, I’m in a 12 team mixed with 2 C, CI, MI, 5 OF, 2 UT, 5 Bench spots. Based on my projections (which I don’t think are far off from most), I’m ending up with similar replacement values for all positions except Catcher. Part of it could be that I am treating bench players the same as utility in conjunction with the above paragraph. Otherwise, is this conceivable or have I errored greatly? And if conceivable, does that mean absolute values are pretty much the same as positional adjusted values (with the exception of catcher)?
Definitely confused here and could use some help.
I think this is what happens in a “deep” league. The more players you have to start, the lower replacement value gets and the more the full-time players will converge.
With a CI and 2 UT spots you could potentially play FOUR first-baseman on your team. So instead of a standard league (one CI, one UT) setup where you might see 25-30 first basemen rostered, in your league anybody who can hit a lick is going to be on a roster somewhere.
So it doesn’t surprise me that replacement level is similar for 1B, 2B, etc. Either way, with 13 non-catcher starters per team, you are using up at minimum 260 non-catcher position players. So replacement level is going to be terrible at EVERY position!
Thanks, batpig. Is that how you would approach bench spots, as well? Basically treat them similarly to the utility spots? Or would you back them out somehow so that you’re spending more $ on starters?
Since replacement level is equal everywhere, is it fair to say that position scarcity is close to non-existent and I should just be looking at the best player available.