A Retrospective Look at the Price of a Win
Barry Zito got paid $18.5 million last year and produced -0.4 Wins Above Replacement. You and I checked in at about 0 WAR — and by staying off the field, we out-produced Zito. I don’t know about you, but my paycheck didn’t look anything like Zito’s last year. And while my employer might be happy about that, the Giants are wondering what they got for their millions. In the everyday business world, an employer could find a way to get rid of a guy like Zito — an under-performing, overpaid employee. The Giants can’t. In fact, to get another team to take him, they’d need to pay most — if not all — of his salary. Regardless of his performance, Zito’s getting his money.
Then there’s Albert Pujols. He was paid $16 million last year and he produced 5.1 WAR. There’s nothing wrong with $16 million, if you were the St. Louis Cardinals. In fact, Pujols was worth a lot more. Teams fell over themselves trying to give Pujols a 50% raise this off-season, and that was no surprise.
So we know that the Giants had a bad deal with Zito, and the Cardinals had a good deal with Pujols. Those are extremes, and they’re obvious. But baseball, like life, is usually lived in a gray area — like the players who got the medium deals. But to figure out who those guys are, a more sophisticated analysis is required.
In today’s and tomorrow’s articles, I’ll determine the average price paid for a win in the past five years. To do this, I’ll first explain the difference between the expected price of wins before a season begins and the actual price paid after a season is finished. There are several important differences between expected price and actual price, and they’ll matter when we take a retrospective look at these contracts.
Most businesses try to get the most bang for their buck on labor, and baseball’s no different. Sabermetricians for years have been figuring the price of a win — which helps them sort the good deals from the bad. FanGraphs’ “Dollars” metric estimates the value that teams place on production in the free-agent market. The formula is straightforward: Take the average annual value of contracts given out to free agents, then use Marcel projections to match the contract values with the players’ expected production.
The FanGraph’s method is basic and gives us apples-to-apples comparisons of contracts going into the season. Naturally, by using the total salary paid to free agents — divided by their total expected WAR — roughly half the contracts rate above average, while the other half rate below. Simple, right? Maybe not: When we look at past season, things don’t exactly add up.
Take the past six seasons as an example. From 2007 to 2011, there were 1,436 players with at least six years of service time. If we take the retrospective value of those contracts using “Dollars” and the actual salaries paid for all of those guys put together, we might expect these two to be equivalent. The “Dollars” statistic is supposed to represent the average expenditure per win on players eligible for free agency. But look what happened:
“Dollars” Value: $7.41 billion
Actual Salaries: $8.46 billion
This means that players get paid more, on average, than we expected the average player to be paid. This happened because players under-produced their expected WAR by 16%. That doesn’t mean using projections to develop a market value is useless, but it does mean that evaluating deals in retrospect requires some sort of adjustment.
Why did these players under-produce relative to expectations? There are a few reasons:
1) Playing time projections for free agents are too high
Ask anyone who developed a top projection systems about playing time, and they’ll tell you that’s not what they do. They only attempt to estimate rate statistics, but they do a pretty good job. Still, they generally come in way too high on playing time. Taking all 797 players and 638 pitchers with six years of service time or more, I checked all the Marcel, Oliver and ZiPS projections that I could find, and I discovered a common trend:
| N | PA | AVG | OBP | SLG | |
|---|---|---|---|---|---|
| Marcel | 782 | 461 | .272 | .344 | .438 |
| Actual | 782 | 384 | .269 | .339 | .424 |
| N | PA | AVG | OBP | SLG | |
|---|---|---|---|---|---|
| Oliver | 755 | 491 | .271 | .339 | .433 |
| Actual | 755 | 391 | .269 | .339 | .424 |
| N | PA | AVG | OBP | SLG | |
|---|---|---|---|---|---|
| ZiPS | 776 | 441 | .271 | .344 | .432 |
| Actual | 776 | 384 | .269 | .339 | .424 |
The three rate statistics for batters were projected pretty well. They came in a little high, but that was probably due to the recently declining run environment in baseball. The playing time was over-projected by about 20%, which means that WAR was over-projected for hitters.
The same result held for pitchers:
| N | IP | ERA | |
|---|---|---|---|
| Marcel | 611 | 103 | 4.25 |
| Actual | 611 | 91 | 4.20 |
| N | IP | ERA | |
|---|---|---|---|
| Oliver | 541 | 113 | 4.16 |
| Actual | 541 | 94 | 4.28 |
| N | IP | ERA | |
|---|---|---|---|
| ZiPS | 596 | 110 | 4.24 |
| Actual | 596 | 93 | 4.19 |
Pitchers actually had better ERAs than their projections suggested, but this was also probably due to the decreasing run environment. But much like the hitters, pitchers played less than expected. In fact, they pitched about 17% fewer innings than the estimations.
None of this is an indictment of projection systems. But the result of over-projecting playing time means an over-projection of WAR. The end result of that means under projection of the $/WAR paid on the free-agent market.
Over-projecting playing time isn’t entirely unique to free agents. In last week’s article on testing projection systems, I evaluated how close projections were to wOBA and ERA, which are rate stats. What I didn’t show is playing-time statistics. Using all systems for only 2011, we see that over-projecting playing time was common, but it wasn’t an epidemic.
| Projection | PA Over-projection | IP Over-projection |
|---|---|---|
| PECOTA | 18% | 5% |
| ZiPS | 15% | 5% |
| Oliver | 18% | 8% |
| Marcel | -2% | -11% |
| Cairo | 13% | -3% |
| Steamer | -2% | -6% |
The plate-appearance over-projection is even more surprising, because I only included players who got at least 200 PAs. The most egregious over-projections are excluded from this. And even though only some projection systems look high for pitchers, the 40-innings minimum also excluded the worst over-projections. These numbers are all somewhat under-estimated.
Notice that not all projection systems over-estimate playing time. In fact, when looking at projections across the board, Marcel doesn’t over-project playing time, but it does over-project playing time for players who are eligible for free agency. Interestingly, the systems seemingly under-project playing time for players who aren’t eligible for free agency.
The main lesson to take away from this is that playing time matters when we’re evaluating free agents. If we’re only looking to evaluate deals relative to each other, we can get away with over-projecting playing time across the board. But if we want to make a statement about what Albert Pujols does for the Angels, we need to take an additional step and ask whether we are over-estimating his WAR, even if we correctly estimate his wOBA.
2) Players decline throughout contracts, making $/WAR lowest in the first year
To estimate the value of free-agent contracts, we generally look at just at a contract’s first year, because contracts signed several years ago have little to do with the current market. Unsurprisingly, players decline over time. As a result, the $/WAR in a deal’s first year is generally higher than the $/WAR in later years.
In fact, if you look at the $/WAR paid for players with at least six years of service time throughout the 2007 to 2011 seasons, you’d get the following ratios:
First year of contracts: $4.25 million/WAR
Later years of contracts: $5.5 million/WAR
Combining all contracts: $4.92 million/WAR
Players in the first year of their contracts get paid almost $1 million less per WAR than all free-agency eligible players, and that causes a retrospective over-estimates of $/WAR. Contract evaluation needs to take into account the fact that the most recent contracts are the most relevant, and the fact that the first year is usually the one that gives the best bang for a team’s buck.
3) Free agents who sign elsewhere generally give their new teams less bang for their buck.
This was detailed in my recent article in The 2012 Hardball Times Annual, but a look at a couple results from that piece will tell an important story here.
The $/rWAR* for all 196 multi-year deals that ended in between 2007 to 2011:
Re-signed players: $4.9 million/WAR
Players who switched teams: $6.8 million/WAR
Difference: 39%
Looking only at deals between two and four years long, I found the following differences:
Re-signed players who signed before the end of the season: $4.6 million/rWAR*
Re-signed players who signed after the end of the season: $6.6 million/rWAR*
Players who switched teams after the season ended: $8.9 million/rWAR*
*Note that rWAR is different (and usually lower — than WAR used at FanGraphs.
There’s a clear bias if we look only at the contracts for players who reached free agency; these players are more likely to be overpaid, which also throws more measurement error into the data.
4) The amount surrendered for a free agent includes both the dollars and the draft picks that are given up.
In fact, when I say that “Dollars” is limited in retrospective because of the under-production of free agents, I’m not even taking into account an additional cost that teams consider when signing free agents.
In this article, I showed how to estimate the costs of draft-pick compensation when signing free agents. Specifically, I explained that teams would pay 10% more for free agents overall if they didn’t also have to surrender draft picks. I also gave the following estimates of the dollar value that teams place on draft picks:
Surrendering 16th overall pick: $8.3 million
Surrendering 20th overall pick: $7.3 million
Surrendering 30th overall pick: $5.7 million
Surrendering 50th overall pick: $4.1 million
Surrendering 60th overall pick: $3.7 million
Surrendering 99th overall pick: $2.6 million
Re-signing Type A Free Agents: $9.6 million
Re-signing Type B Free Agents: $4.8 million
Going Forward
Coming up with a well-reasoned retrospective estimate of the average dollars paid per WAR for free agents is tricky. Obviously, we now know that playing-time projections are too high, but dealing with “Other People’s Player Bias” and early extension bias is something that we’ll need to understand better.
If you come away from this piece with more questions than answers, you’re reading correctly. In the next few days, I’ll be writing more articles that explain how the free-agent market has behaved in the past five years. I’ll also show some interesting idiosyncrasies about how teams pay players. After we’ve established how the market has worked in the past, we’ll have a better sense of how it might function in the future and how teams could better approach free agency in the future.
1
With regards to point 3, surely this is an artefact of the Winner’s Curse: http://en.wikipedia.org/wiki/Winner%27s_curse – and part of the reason that these players are not getting re-signed may be that their original club has more complete information on them (e.g. they expect them to age badly) in the first place?
Additionally, there’s also the fact that players may be giving discounts to their current club because they’re deriving utility from staying put – both in terms of their personal lives (not moving family etc.) and also from a personal-marketability perspective.
I explored this in a lot of detail in this year’s THT Annual. I don’t know that the Winner’s Curse is the whole explanation, because if players knew this, they would hold out more and switch. So at best it’s a coupling of Winner’s Curses plus hometown discounts.
But a very important finding I had was that pitchers actually fell short of projections significantly when they switched teams and beat them pretty handily if they stayed put. The best market imperfection explanation is probably that players who switch teams are Akerlof Lemons: http://en.wikipedia.org/wiki/The_Market_for_Lemons
If you want to get technical, we probably didn’t outproduce Zito. Speaking only for myself I know that I haven’t been a replacement level baseball player since high school jv.
I like the article though, and am wondering if you could go a step further with point number three: break the sample into players who have not switched teams since team control expired, and players who have. See what kind of “hometown discount” there actually is.
I recommend you get yourself a copy of the 2012 THT Annual where you can find Matt’s article on the topic.
full disclosure – I write for THT and have an article in that Annual.
I also think it’s kind of ridiculous that somebody thinks $18.5 million is a lot of money. That’s only about a third of what you’d need to buy a decent townhouse in Manhattan.
In the THT Annual piece, I break it down in a lot of detail, but basically I think the hometown discount affect seems like it would at best explain half of the effect, and probably a good deal less.
Good article – I look forward to more on the free agent market.
Has anyone done research into allocation of total payroll to free agents vs. players not yet eligible? It seems odd to say that a 3 WAR season is “worth” $15 million when plenty of pre-free agency players contribute as much war, but much cheaper.
Agreed. Excellent article with lots of food for thought.
WRT the $5M per WAR figure, it only applies to free agents because you can’t price players who are not available on the market.
This is 100% true and a very essential fact to know about understanding the market for baseball players. From Marvin Miller’s wikipedia:
“As an economist, Miller understood that too many free agents could actually drive down player salaries. Miller agreed to limit free agency to players with more than six years of service, knowing that restricting the supply of labor would drive up salaries as owners bid for an annual, finite pool of free agents.”
That’s one reason it’s not really “$ per win”, but rather “$ per win that we can’t get out of cost-controlled options”. An imaginary awesome team with cost-controlled players (second/third year guys, all at least good players) might only be able to upgrade a couple of wins with 20 million a year to one premium free agent. A terrible team full of useless players might only need to spend 10 million/year to get the same benefit in wins spread out over 3-4 roster spots. It comes out in the wash in the end, I suppose, but team-specific dynamics make it a blunt tool for single acquistion analyses.
The fraction of dollars above league minimum that goes to free agents is under 70%, but it was closer to 80% a few years ago. The flat economy plus arbitration raises continuing the grow in a non-market environment is the culprit there.
Why can’t all articles be like Matt Swartz articles?
Amen!
Good stuff…
Good article but unfortunately sabermatics does not equate economics. A players contract is for a players worth, not WAR (or ability). A players worth can be skewed a thousand different ways (merchandise, attendance, etc, etc). Though this good start for pricing a win, should also look into how much of a players contract is for peripherals?
I would really like to see this. I’ve read Dave Cameron chats where he alludes to studies indicating that the non-baseball value of superstars is negligible, but I’d like to see them.
Very good idea. Of course, without free entry and exit, Major League Baseball will not have anywhere near the same market pressures as other sports, so you can get mispriced assets in the baseball labor market like you’d never imagine in markets where bad businesses disappear.
Just a question, not sure what the methodology is, on point number 4:
Does the value of the draft picks considered present value and time value of money? I just ask because on a non-analytical thought track the first thing that came to mind was yes these draft picks may offer more value over their lifetime, but you have to wait as many as 4 years or so for them to start producing. I would rather have a 2 win player for this season than a 5 win player 4 seasons from now (im sure there is a more exact valuing if you did the math (ill try it out) but this is my rough offering). Especially when you consider that the people making the moves know they have their job now but not in 4-5 years you would think this would add a little more value to the FA who can produce sooner…
I believe all the rest of us are interested in the player’s value to the team, not his value to the employee who signs him.
Value, much like beauty, is in the eye of the beholder. While it seems ideal to assume that all decisions should be made in the best interests of the “team” (however you define that word) its simply non-sensical to ignore that fact that real, live human beings, with unique emotions and motivations, are the ones making the decisions. It is certainly likely that a GM, team president, or team owner will apply their own personal perspectives (biases, maybe) when determing the value of a prospect or a FA.
Bottom line: while we’d like value to be spit out of an exacting formula; value is not a one-size fits all commodity. Its ruled not by our formulas, but by the individual perspectives of those attempting to behold it.
It really shouldn’t make a difference who you are measuring value for, the team or the management, Wins in Baseball are the same as Dollars in Finance: one today is worth more than one tomorrow…
It really comes down to the fact that you can’t measure value to a team in such a general sense. What is the goal of the “team”? Would anyone argue that you would prefer to win the World Series this year vs winning it 10 years from now?
Check out the link on the present value of draft picks linked above. Teams appear to act as though they value wins and dollars about 10% less next year than this year.
can we PLEASE banish the notion (however joking it is) that by not playing the game, us basement occupying bloggers and mouth breathing blog readers have accumulated 0 WAR? i’d be fucking thrilled if i were as good of a ballplayer as kosuke fukudome or edinson volquez were in 2011.
dude, the point is not that we’re *as good a ballplayer* as, or better than, a player posting a negative or zero WAR…. the point is that *by not playing at all* we have helped the team more than a player posting a negative WAR.
The term is “above replacement”… that should not be confuse with NEGATIVE value. Replacement level does produce some value, similarly a replacement player is not paid $0; yet because the way the construct is done 0WAR = $0.
An all replacement team does not generate 0 wins. It generates something in the 40′s… so to say 0 WAR players are worth zero dollars is just another reason why baseball stat folks should not play internet economist or internet CPA. Last I checked even replacement players get paid more than 0 (or at least that’s what the CBA says)
Similarly 4.5MIl is the PRICE (or if you prefer, cost) of a win on the free agent market (assuming you agree with how this is calculated). It is not the VALUE of a win…. yet this constantly is used interchangeably. Is the value of a win from Strasburg next year less than the value of a win from Werth. The price is obviously different and you can look at “ROI” , but people continue to use a free agent cost of a win as the generic value of a win.
You are complete correct Joe. What they really mean to say is “Average cost of a win on the open market.” I think it is kind of comical to really use that as the basis to say contract A is bad, contract B is good. After all if everyone started paying only the average cost of a win to FAs the average cost would start to drop. The problem is that player talent is not perfectly distributed into the same amount of 1,2,3,4,5 WAR players. Therefore the higher WAR players are very likely to get more than their fair share of the FA dollars available because generally speaking teams perfer WAR density vs. WAR balance.
As to the value of a win if you took the reported MLB revenues for 2009 and divided by the total number of regular season wins you would come up with a number very close to $3mil. Two caveats with that figure.
1) That is the REPORTED revenues so who knows how much those numbers have been fudged and it doesnt take into account the sweetheart deals with sister companies where vendors and other services are banking money and paying the team pennies on the dollar all while being own by the same owner as the team.
2) The total above assumes that all wins are equal and doesnt include the post season at all. So regular season wins are gonna be worth less. Also on a team basis vs a league basis we all know that the extra wins that get you into the postseason are much more valuable in terms of revenue than going from 50 wins to 60 wins. So 5 wins for the Tigers might equal 50 mil, while 5 wins for the Astros might equal 10 mil.
Joe, I must have liked the idea of playing an economist on the internet so much that I spent a decade at Penn getting a bachelors, masters, and Ph.D. in economics. Try playing a guy who uses Google on the internet and you might have checked out my background a little.
So it shouldn’t surprise you that I valued all contracts is by considering opportunity cost– relative to using the roster spot on a 0 WAR player playing for to the league minimum. Since we’re dealing with a labor market with finite vacancies, we need to focus on the opportunity cost of employing a worker at the expense of employing another.
And of course, the reason that the market price of a win is treated as the value of a win is that we are making a rational expectations assumption. Of course, without free entry and exit, this assumption can be seen as limited, but if one assumes an increasing marginal revenue product of wins, the approximation seems more than defensible. Add in the fact that teams that increase their spending appear to stay just as profitable and teams that decrease their spending don’t seem to become more profitable, it seems quite likely that the marginal revenue product of free agent labor approximately equals the marginal cost. The value of a win for teams that spend money on them (disproportionally on the cusp of playoff contention) tends to be particularly high.
Ronin, what you’re missing in just dividing wins per dollar is that you’re explicitly assuming that the marginal revenue product of a win is equal to the average revenue product of a win. This is particularly unlikely when you consider that the marginal revenue product of a win increases as you approach the playoff cusp, and because players actually signed represent the highest value that any team placed on them.
This is a very good example of the difference between zero and null. A replacement level ballplayer has 0 WAR; you and I have a NULL instead.
Matt – not sure you noticed but I was responding to Richard.
Sorry if you took the stat guys line as a shot directed at you it was directed at a lot of these commenters who shout dumb move or great move because they hear 4.5mil = 1 win and maybe are ‘advanced’ enough to apply a 5% inflation rate and an aging curve and think that’s all there is to it. Or think a guy off the street provided more value than a below replacement level player.
Though I do have issues with your rational expectations assumption and using pretty much unknown/unconfirmable profitability vs payroll to validate a marginal revenue assumption… it may be true, but we really have no way of knowing what the profitability of these teams are. We just don’t have enough insight into their books and detailed cash flow analysis to understand where money is going/not going and what the true profitability of these teams are.
I think the market is too small (and it gets even smaller with some players) to make some of your market assumptions. I’m not sure you can apply a rationale expectations market assumption when this may not be a market at or near equilibrium, has different levels of knowledge among the participants and the motivation of the market members are not all the same.
Is each owner trying to max profitability? Are some trying to win at all costs? Are some trying to get a stadium built? Are some trying to assuage the commish’s office and MLBPA and take on enough payroll so they stop threatening them? Similarly players’ motivations may also not be about max money as well and lead to a less than rational expectations market. If Oswalt signs for 5-6 mil in Stl or Tex is that the real rational market demand or is it a somewhat artificial one as he has made clear there are only a few places he wants to play. Are some players applying location, team strength, personal concerns (wife/family), etc to the decision process which might significantly impact what that player signs for and how much he gets paid…
Anyone with 0 PA has accumulated 0 WAR, no matter how talented they are, right? I accumulated 0 doctor WAR, 0 lawyer WAR, and 0 teacher WAR in 2011, because I spent no time as a doctor, lawyer, or teacher. This is not to say I’d be better at surgery than the worst doctor in 2011 was.
What it means is the patient would be better off with your as doctor (and saying, “I’m not really a doctor! Get any real doctor to do this”) than having the world’s worst surgeon, who would kill them on the operating table.
I wonder how much of point 4 is an artefact of “hometown discounts”
*point 3 sorry
Do you think you could show a comparison to the “fan’s” projections (as well as Tango’s survey’s). I’m hoping to use those for my play time projections.
I’ll look into this. Thanks.
This article is the Albert Pujols of FG pieces.
I think the playing time issue is complicated by injuries. Over a 5 year contract, at least 1 year is likely to be a down year due to injury. That’s one reason multi-year deals are discounted, but maybe they are not discounted enough.
There are a number of high profile players who have had free agent years bought out with extensions with below free market rates, so the over pay for free agents is even worse.
I think the reason for the overall overpay is that for those teams who spend the most of free agents, a win is worth more than those who don’t. So while the average incremental W might be valued at 5 million or less by the Nationals, that W’s value is worth more to the Red Sox, Yankees and Phillies with higher potential revenues and more to lose by not fielding a contender. And worth more to teams who are close to being a playoff team but need a bit more like the Tigers (post V-Mart) and Angels
The other reason for the “apparent” overpay is revenue figures are too low. With many teams owning regional stations like the Yankees and Red Sox, revenue from NESN and YES are part of the equation, and indeed, revenues for the Red Sox and Yankees may be understated to minimize revenue sharing..
Matt, did you look at the pitching numbers split by starters and relievers?
Coming soon!
Look at this article.. it seems all you need is a copy of THE CREATIVE MIND” and a bunch of signs in your room to tell you how wonderful you are! Ask Barry what the hell happened to him?
Did his creative mind get sidetracked and loose focus?
http://www.mitchhorowitz.com/barry-zito.html
Could the playing time overestimates also be an artifact of the decreasing run environment? Playing time for hitters = PA, so fewer runs (probably) derives mostly from fewer men on base, which by extension means fewer PAs.
That effect wouldn’t be as pronounced with have much effect with pitchers, at least league-wide, since IP is pretty much fixed but for extra inning games, and pitcher IP is generally overprojected much less than hitters…
Apologies for the butchered grammar, im commenting from a phone.
An interesting idea, but the average change in PA vs. the previous year is only 0.3% less than the previous year. Creative thinking though.
Hypothetically assuming 4.5mil/win cost)
3 WAR player is paid 13.5 mil
2 WAR player is paid 9mil
1 WAR player is paid 4.5mil
0 WAR player is paid 0.5 (rounding from 480K league min)
–> from this hypothetical we conclude all players are paid fairly
However
- the upgrade from 0 to 1 is at a cost of 4mil (4mil/win)
- the upgrade from 1 to 2 is at a cost of 4.5mil (4.5mil/win)
- the upgrade from 0 to 2 is 8.5mil (4.25mil/win)
- the upgrade from 0 to 3 is 13mil ( 4.33mil/win)
Or to put it differently if a team wants to upgrade by a win and has the option to upgrade a 0WAR player or a 1 WAR the net cost is different. Going from a 0WAR player to a 1WAR player is an extra 4mil in payroll (assuming the players are priced efficiently); going from a 1 WAR player to a 2 WAR player is an extra 4.5 mil (again assuming players are priced efficiently). For a given team that 1 extra win has the same value but it comes at 2 different costs.
rWAR and fWAR have different replacement level. Team full of rWAR replacement players would produce several more wins than team of fWAR replacement level players (I believe it’s 54 to 50, but I’m not sure). So, there is less less WAR, around 4 per team, in rWAR than in fWAR. So, $/WAR should be higher, maybe 10% or so.
Players on the free market often sign for the highest bidder and almost always for one of 2-3 highest (they may decline PIT or someone like that, but most of players don’t take way smaller contract than 2nd best). So, team that has signed them is generally most optimistic. When you pay people according to 60th or 70th percentile projection (specially considering injuries) instead of 50th, over large enough sample you have to lose. For example, NYY valued Burnett at something like 4/60. But when ATl started bidding, they had a choice between paying him 5/82.5 or losing him, even though they knew that 5th year was lost money, maybe even 4th. But they got WS win out of that.
Quick clarification, someone. Is rWAR Baseball Reference WAR?
Playing time using Tango’s community forecasts is over-projected too. For 2011:
IP over-projected 7.5% (103.7 vs. 96.5)
PA over-projected11.9% (415.5 vs. 371.3)