A Retrospective Look at the Price of a Win

Barry Zito got paid $18.5 million last year and produced -0.4 Wins Above Replacement. You and I checked in at about 0 WAR — and by staying off the field, we out-produced Zito. I don’t know about you, but my paycheck didn’t look anything like Zito’s last year. And while my employer might be happy about that, the Giants are wondering what they got for their millions. In the everyday business world, an employer could find a way to get rid of a guy like Zito — an under-performing, overpaid employee. The Giants can’t. In fact, to get another team to take him, they’d need to pay most — if not all — of his salary. Regardless of his performance, Zito’s getting his money.

Then there’s Albert Pujols. He was paid $16 million last year and he produced 5.1 WAR. There’s nothing wrong with $16 million, if you were the St. Louis Cardinals. In fact, Pujols was worth a lot more. Teams fell over themselves trying to give Pujols a 50% raise this off-season, and that was no surprise.

So we know that the Giants had a bad deal with Zito, and the Cardinals had a good deal with Pujols. Those are extremes, and they’re obvious. But baseball, like life, is usually lived in a gray area — like the players who got the medium deals. But to figure out who those guys are, a more sophisticated analysis is required.

In today’s and tomorrow’s articles, I’ll determine the average price paid for a win in the past five years. To do this, I’ll first explain the difference between the expected price of wins before a season begins and the actual price paid after a season is finished. There are several important differences between expected price and actual price, and they’ll matter when we take a retrospective look at these contracts.

Most businesses try to get the most bang for their buck on labor, and baseball’s no different. Sabermetricians for years have been figuring the price of a win — which helps them sort the good deals from the bad. FanGraphs’ “Dollars” metric estimates the value that teams place on production in the free-agent market. The formula is straightforward: Take the average annual value of contracts given out to free agents, then use Marcel projections to match the contract values with the players’ expected production.

The FanGraph’s method is basic and gives us apples-to-apples comparisons of contracts going into the season. Naturally, by using the total salary paid to free agents — divided by their total expected WAR — roughly half the contracts rate above average, while the other half rate below. Simple, right? Maybe not: When we look at past season, things don’t exactly add up.

Take the past six seasons as an example. From 2007 to 2011, there were 1,436 players with at least six years of service time. If we take the retrospective value of those contracts using “Dollars” and the actual salaries paid for all of those guys put together, we might expect these two to be equivalent. The “Dollars” statistic is supposed to represent the average expenditure per win on players eligible for free agency. But look what happened:

“Dollars” Value: $7.41 billion
Actual Salaries: $8.46 billion

This means that players get paid more, on average, than we expected the average player to be paid. This happened because players under-produced their expected WAR by 16%. That doesn’t mean using projections to develop a market value is useless, but it does mean that evaluating deals in retrospect requires some sort of adjustment.

Why did these players under-produce relative to expectations? There are a few reasons:

1) Playing time projections for free agents are too high

Ask anyone who developed a top projection systems about playing time, and they’ll tell you that’s not what they do. They only attempt to estimate rate statistics, but they do a pretty good job. Still, they generally come in way too high on playing time. Taking all 797 players and 638 pitchers with six years of service time or more, I checked all the Marcel, Oliver and ZiPS projections that I could find, and I discovered a common trend:

N PA AVG OBP SLG
Marcel 782 461 .272 .344 .438
Actual 782 384 .269 .339 .424

 

N PA AVG OBP SLG
Oliver 755 491 .271 .339 .433
Actual 755 391 .269 .339 .424

 

N PA AVG OBP SLG
ZiPS 776 441 .271 .344 .432
Actual 776 384 .269 .339 .424

The three rate statistics for batters were projected pretty well. They came in a little high, but that was probably due to the recently declining run environment in baseball. The playing time was over-projected by about 20%, which means that WAR was over-projected for hitters.

The same result held for pitchers:

N IP ERA
Marcel 611 103 4.25
Actual 611 91 4.20

 

N IP ERA
Oliver 541 113 4.16
Actual 541 94 4.28

 

N IP ERA
ZiPS 596 110 4.24
Actual 596 93 4.19

Pitchers actually had better ERAs than their projections suggested, but this was also probably due to the decreasing run environment. But much like the hitters, pitchers played less than expected. In fact, they pitched about 17% fewer innings than the estimations.

None of this is an indictment of projection systems. But the result of over-projecting playing time means an over-projection of WAR. The end result of that means under projection of the $/WAR paid on the free-agent market.

Over-projecting playing time isn’t entirely unique to free agents. In last week’s article on testing projection systems, I evaluated how close projections were to wOBA and ERA, which are rate stats. What I didn’t show is playing-time statistics. Using all systems for only 2011, we see that over-projecting playing time was common, but it wasn’t an epidemic.

Projection PA Over-projection IP Over-projection
PECOTA 18% 5%
ZiPS 15% 5%
Oliver 18% 8%
Marcel -2% -11%
Cairo 13% -3%
Steamer -2% -6%

The plate-appearance over-projection is even more surprising, because I only included players who got at least 200 PAs. The most egregious over-projections are excluded from this. And even though only some projection systems look high for pitchers, the 40-innings minimum also excluded the worst over-projections. These numbers are all somewhat under-estimated.

Notice that not all projection systems over-estimate playing time. In fact, when looking at projections across the board, Marcel doesn’t over-project playing time, but it does over-project playing time for players who are eligible for free agency. Interestingly, the systems seemingly under-project playing time for players who aren’t eligible for free agency.

The main lesson to take away from this is that playing time matters when we’re evaluating free agents. If we’re only looking to evaluate deals relative to each other, we can get away with over-projecting playing time across the board. But if we want to make a statement about what Albert Pujols does for the Angels, we need to take an additional step and ask whether we are over-estimating his WAR, even if we correctly estimate his wOBA.

2) Players decline throughout contracts, making $/WAR lowest in the first year

To estimate the value of free-agent contracts, we generally look at just at a contract’s first year, because contracts signed several years ago have little to do with the current market. Unsurprisingly, players decline over time. As a result, the $/WAR in a deal’s first year is generally higher than the $/WAR in later years.

In fact, if you look at the $/WAR paid for players with at least six years of service time throughout the 2007 to 2011 seasons, you’d get the following ratios:

First year of contracts: $4.25 million/WAR
Later years of contracts: $5.5 million/WAR
Combining all contracts: $4.92 million/WAR

Players in the first year of their contracts get paid almost $1 million less per WAR than all free-agency eligible players, and that causes a retrospective over-estimates of $/WAR. Contract evaluation needs to take into account the fact that the most recent contracts are the most relevant, and the fact that the first year is usually the one that gives the best bang for a team’s buck.

3) Free agents who sign elsewhere generally give their new teams less bang for their buck.

This was detailed in my recent article in The 2012 Hardball Times Annual, but a look at a couple results from that piece will tell an important story here.

The $/rWAR* for all 196 multi-year deals that ended in between 2007 to 2011:

Re-signed players: $4.9 million/WAR
Players who switched teams: $6.8 million/WAR
Difference: 39%

Looking only at deals between two and four years long, I found the following differences:

Re-signed players who signed before the end of the season: $4.6 million/rWAR*
Re-signed players who signed after the end of the season: $6.6 million/rWAR*
Players who switched teams after the season ended: $8.9 million/rWAR*

*Note that rWAR is different (and usually lower — than WAR used at FanGraphs.

There’s a clear bias if we look only at the contracts for players who reached free agency; these players are more likely to be overpaid, which also throws more measurement error into the data.

4) The amount surrendered for a free agent includes both the dollars and the draft picks that are given up.

In fact, when I say that “Dollars” is limited in retrospective because of the under-production of free agents, I’m not even taking into account an additional cost that teams consider when signing free agents.

In this article, I showed how to estimate the costs of draft-pick compensation when signing free agents. Specifically, I explained that teams would pay 10% more for free agents overall if they didn’t also have to surrender draft picks. I also gave the following estimates of the dollar value that teams place on draft picks:

Surrendering 16th overall pick: $8.3 million
Surrendering 20th overall pick: $7.3 million
Surrendering 30th overall pick: $5.7 million
Surrendering 50th overall pick: $4.1 million
Surrendering 60th overall pick: $3.7 million
Surrendering 99th overall pick: $2.6 million

Re-signing Type A Free Agents: $9.6 million
Re-signing Type B Free Agents: $4.8 million

Going Forward

Coming up with a well-reasoned retrospective estimate of the average dollars paid per WAR for free agents is tricky. Obviously, we now know that playing-time projections are too high, but dealing with “Other People’s Player Bias” and early extension bias is something that we’ll need to understand better.

If you come away from this piece with more questions than answers, you’re reading correctly. In the next few days, I’ll be writing more articles that explain how the free-agent market has behaved in the past five years. I’ll also show some interesting idiosyncrasies about how teams pay players. After we’ve established how the market has worked in the past, we’ll have a better sense of how it might function in the future and how teams could better approach free agency in the future.



Print This Post



Matt writes for FanGraphs and The Hardball Times, and models arbitration salaries for MLB Trade Rumors. Follow him on Twitter @Matt_Swa.


Sort by:   newest | oldest | most voted
Aaron (UK)
Guest
Aaron (UK)
4 years 6 months ago

With regards to point 3, surely this is an artefact of the Winner’s Curse: http://en.wikipedia.org/wiki/Winner%27s_curse – and part of the reason that these players are not getting re-signed may be that their original club has more complete information on them (e.g. they expect them to age badly) in the first place?

Additionally, there’s also the fact that players may be giving discounts to their current club because they’re deriving utility from staying put – both in terms of their personal lives (not moving family etc.) and also from a personal-marketability perspective.

ussdavidprice
Guest
ussdavidprice
4 years 6 months ago

If you want to get technical, we probably didn’t outproduce Zito. Speaking only for myself I know that I haven’t been a replacement level baseball player since high school jv.

I like the article though, and am wondering if you could go a step further with point number three: break the sample into players who have not switched teams since team control expired, and players who have. See what kind of “hometown discount” there actually is.

Brad Johnson
Member
Member
4 years 6 months ago

I recommend you get yourself a copy of the 2012 THT Annual where you can find Matt’s article on the topic.

full disclosure – I write for THT and have an article in that Annual.

JimNYC
Guest
JimNYC
4 years 6 months ago

I also think it’s kind of ridiculous that somebody thinks $18.5 million is a lot of money. That’s only about a third of what you’d need to buy a decent townhouse in Manhattan.

tz
Guest
tz
4 years 6 months ago

Good article – I look forward to more on the free agent market.

Has anyone done research into allocation of total payroll to free agents vs. players not yet eligible? It seems odd to say that a 3 WAR season is “worth” $15 million when plenty of pre-free agency players contribute as much war, but much cheaper.

Craig
Guest
Craig
4 years 6 months ago

Agreed. Excellent article with lots of food for thought.

WRT the $5M per WAR figure, it only applies to free agents because you can’t price players who are not available on the market.

test
Guest
test
4 years 6 months ago

That’s one reason it’s not really “$ per win”, but rather “$ per win that we can’t get out of cost-controlled options”. An imaginary awesome team with cost-controlled players (second/third year guys, all at least good players) might only be able to upgrade a couple of wins with 20 million a year to one premium free agent. A terrible team full of useless players might only need to spend 10 million/year to get the same benefit in wins spread out over 3-4 roster spots. It comes out in the wash in the end, I suppose, but team-specific dynamics make it a blunt tool for single acquistion analyses.

Matt H
Guest
Matt H
4 years 6 months ago

Why can’t all articles be like Matt Swartz articles?

Baltar
Guest
Baltar
4 years 6 months ago

Amen!

AL Eastbound
Guest
4 years 6 months ago

Good stuff…

Thomsawyer
Guest
Thomsawyer
4 years 6 months ago

Good article but unfortunately sabermatics does not equate economics. A players contract is for a players worth, not WAR (or ability). A players worth can be skewed a thousand different ways (merchandise, attendance, etc, etc). Though this good start for pricing a win, should also look into how much of a players contract is for peripherals?

Chris
Guest
Chris
4 years 6 months ago

I would really like to see this. I’ve read Dave Cameron chats where he alludes to studies indicating that the non-baseball value of superstars is negligible, but I’d like to see them.

zbelair
Member
zbelair
4 years 6 months ago

Just a question, not sure what the methodology is, on point number 4:
Does the value of the draft picks considered present value and time value of money? I just ask because on a non-analytical thought track the first thing that came to mind was yes these draft picks may offer more value over their lifetime, but you have to wait as many as 4 years or so for them to start producing. I would rather have a 2 win player for this season than a 5 win player 4 seasons from now (im sure there is a more exact valuing if you did the math (ill try it out) but this is my rough offering). Especially when you consider that the people making the moves know they have their job now but not in 4-5 years you would think this would add a little more value to the FA who can produce sooner…

Baltar
Guest
Baltar
4 years 6 months ago

I believe all the rest of us are interested in the player’s value to the team, not his value to the employee who signs him.

SeaWolf
Guest
SeaWolf
4 years 6 months ago

Value, much like beauty, is in the eye of the beholder. While it seems ideal to assume that all decisions should be made in the best interests of the “team” (however you define that word) its simply non-sensical to ignore that fact that real, live human beings, with unique emotions and motivations, are the ones making the decisions. It is certainly likely that a GM, team president, or team owner will apply their own personal perspectives (biases, maybe) when determing the value of a prospect or a FA.

Bottom line: while we’d like value to be spit out of an exacting formula; value is not a one-size fits all commodity. Its ruled not by our formulas, but by the individual perspectives of those attempting to behold it.

zbelair
Member
zbelair
4 years 6 months ago

It really shouldn’t make a difference who you are measuring value for, the team or the management, Wins in Baseball are the same as Dollars in Finance: one today is worth more than one tomorrow…
It really comes down to the fact that you can’t measure value to a team in such a general sense. What is the goal of the “team”? Would anyone argue that you would prefer to win the World Series this year vs winning it 10 years from now?

gcm
Guest
gcm
4 years 6 months ago

can we PLEASE banish the notion (however joking it is) that by not playing the game, us basement occupying bloggers and mouth breathing blog readers have accumulated 0 WAR? i’d be fucking thrilled if i were as good of a ballplayer as kosuke fukudome or edinson volquez were in 2011.

Richard
Guest
Richard
4 years 6 months ago

dude, the point is not that we’re *as good a ballplayer* as, or better than, a player posting a negative or zero WAR…. the point is that *by not playing at all* we have helped the team more than a player posting a negative WAR.

Joe
Guest
Joe
4 years 6 months ago

The term is “above replacement”… that should not be confuse with NEGATIVE value. Replacement level does produce some value, similarly a replacement player is not paid $0; yet because the way the construct is done 0WAR = $0.

An all replacement team does not generate 0 wins. It generates something in the 40’s… so to say 0 WAR players are worth zero dollars is just another reason why baseball stat folks should not play internet economist or internet CPA. Last I checked even replacement players get paid more than 0 (or at least that’s what the CBA says)

Similarly 4.5MIl is the PRICE (or if you prefer, cost) of a win on the free agent market (assuming you agree with how this is calculated). It is not the VALUE of a win…. yet this constantly is used interchangeably. Is the value of a win from Strasburg next year less than the value of a win from Werth. The price is obviously different and you can look at “ROI” , but people continue to use a free agent cost of a win as the generic value of a win.

Ronin
Guest
Ronin
4 years 6 months ago

You are complete correct Joe. What they really mean to say is “Average cost of a win on the open market.” I think it is kind of comical to really use that as the basis to say contract A is bad, contract B is good. After all if everyone started paying only the average cost of a win to FAs the average cost would start to drop. The problem is that player talent is not perfectly distributed into the same amount of 1,2,3,4,5 WAR players. Therefore the higher WAR players are very likely to get more than their fair share of the FA dollars available because generally speaking teams perfer WAR density vs. WAR balance.

As to the value of a win if you took the reported MLB revenues for 2009 and divided by the total number of regular season wins you would come up with a number very close to $3mil. Two caveats with that figure.

1) That is the REPORTED revenues so who knows how much those numbers have been fudged and it doesnt take into account the sweetheart deals with sister companies where vendors and other services are banking money and paying the team pennies on the dollar all while being own by the same owner as the team.

2) The total above assumes that all wins are equal and doesnt include the post season at all. So regular season wins are gonna be worth less. Also on a team basis vs a league basis we all know that the extra wins that get you into the postseason are much more valuable in terms of revenue than going from 50 wins to 60 wins. So 5 wins for the Tigers might equal 50 mil, while 5 wins for the Astros might equal 10 mil.

Colin Wyers
Guest
4 years 6 months ago

This is a very good example of the difference between zero and null. A replacement level ballplayer has 0 WAR; you and I have a NULL instead.

Joe
Guest
Joe
4 years 6 months ago

Matt – not sure you noticed but I was responding to Richard.

Sorry if you took the stat guys line as a shot directed at you it was directed at a lot of these commenters who shout dumb move or great move because they hear 4.5mil = 1 win and maybe are ‘advanced’ enough to apply a 5% inflation rate and an aging curve and think that’s all there is to it. Or think a guy off the street provided more value than a below replacement level player.

Though I do have issues with your rational expectations assumption and using pretty much unknown/unconfirmable profitability vs payroll to validate a marginal revenue assumption… it may be true, but we really have no way of knowing what the profitability of these teams are. We just don’t have enough insight into their books and detailed cash flow analysis to understand where money is going/not going and what the true profitability of these teams are.

I think the market is too small (and it gets even smaller with some players) to make some of your market assumptions. I’m not sure you can apply a rationale expectations market assumption when this may not be a market at or near equilibrium, has different levels of knowledge among the participants and the motivation of the market members are not all the same.

Is each owner trying to max profitability? Are some trying to win at all costs? Are some trying to get a stadium built? Are some trying to assuage the commish’s office and MLBPA and take on enough payroll so they stop threatening them? Similarly players’ motivations may also not be about max money as well and lead to a less than rational expectations market. If Oswalt signs for 5-6 mil in Stl or Tex is that the real rational market demand or is it a somewhat artificial one as he has made clear there are only a few places he wants to play. Are some players applying location, team strength, personal concerns (wife/family), etc to the decision process which might significantly impact what that player signs for and how much he gets paid…

Chris
Guest
Chris
4 years 6 months ago

Anyone with 0 PA has accumulated 0 WAR, no matter how talented they are, right? I accumulated 0 doctor WAR, 0 lawyer WAR, and 0 teacher WAR in 2011, because I spent no time as a doctor, lawyer, or teacher. This is not to say I’d be better at surgery than the worst doctor in 2011 was.

Rally
Guest
Rally
4 years 6 months ago

What it means is the patient would be better off with your as doctor (and saying, “I’m not really a doctor! Get any real doctor to do this”) than having the world’s worst surgeon, who would kill them on the operating table.

Dave
Guest
Dave
4 years 6 months ago

I wonder how much of point 4 is an artefact of “hometown discounts”

Dave
Guest
Dave
4 years 6 months ago

*point 3 sorry

bcrossman
Member
bcrossman
4 years 6 months ago

Do you think you could show a comparison to the “fan’s” projections (as well as Tango’s survey’s). I’m hoping to use those for my play time projections.

david
Guest
david
4 years 6 months ago

This article is the Albert Pujols of FG pieces.

pft
Guest
pft
4 years 6 months ago

I think the playing time issue is complicated by injuries. Over a 5 year contract, at least 1 year is likely to be a down year due to injury. That’s one reason multi-year deals are discounted, but maybe they are not discounted enough.

There are a number of high profile players who have had free agent years bought out with extensions with below free market rates, so the over pay for free agents is even worse.

I think the reason for the overall overpay is that for those teams who spend the most of free agents, a win is worth more than those who don’t. So while the average incremental W might be valued at 5 million or less by the Nationals, that W’s value is worth more to the Red Sox, Yankees and Phillies with higher potential revenues and more to lose by not fielding a contender. And worth more to teams who are close to being a playoff team but need a bit more like the Tigers (post V-Mart) and Angels

The other reason for the “apparent” overpay is revenue figures are too low. With many teams owning regional stations like the Yankees and Red Sox, revenue from NESN and YES are part of the equation, and indeed, revenues for the Red Sox and Yankees may be understated to minimize revenue sharing..

Colin Wyers
Guest
4 years 6 months ago

Matt, did you look at the pitching numbers split by starters and relievers?

Zito's Conscience
Guest
Zito's Conscience
4 years 6 months ago

Look at this article.. it seems all you need is a copy of THE CREATIVE MIND” and a bunch of signs in your room to tell you how wonderful you are! Ask Barry what the hell happened to him?
Did his creative mind get sidetracked and loose focus?
http://www.mitchhorowitz.com/barry-zito.html

Pat Folz
Guest
Pat Folz
4 years 6 months ago

Could the playing time overestimates also be an artifact of the decreasing run environment? Playing time for hitters = PA, so fewer runs (probably) derives mostly from fewer men on base, which by extension means fewer PAs.

That effect wouldn’t be as pronounced with have much effect with pitchers, at least league-wide, since IP is pretty much fixed but for extra inning games, and pitcher IP is generally overprojected much less than hitters…

Pat Folz
Guest
Pat Folz
4 years 6 months ago

Apologies for the butchered grammar, im commenting from a phone.

Tom
Guest
Tom
4 years 6 months ago

Hypothetically assuming 4.5mil/win cost)
3 WAR player is paid 13.5 mil
2 WAR player is paid 9mil
1 WAR player is paid 4.5mil
0 WAR player is paid 0.5 (rounding from 480K league min)

–> from this hypothetical we conclude all players are paid fairly

However
– the upgrade from 0 to 1 is at a cost of 4mil (4mil/win)
– the upgrade from 1 to 2 is at a cost of 4.5mil (4.5mil/win)
– the upgrade from 0 to 2 is 8.5mil (4.25mil/win)
– the upgrade from 0 to 3 is 13mil ( 4.33mil/win)

Or to put it differently if a team wants to upgrade by a win and has the option to upgrade a 0WAR player or a 1 WAR the net cost is different. Going from a 0WAR player to a 1WAR player is an extra 4mil in payroll (assuming the players are priced efficiently); going from a 1 WAR player to a 2 WAR player is an extra 4.5 mil (again assuming players are priced efficiently). For a given team that 1 extra win has the same value but it comes at 2 different costs.

Davor
Guest
Davor
4 years 6 months ago

rWAR and fWAR have different replacement level. Team full of rWAR replacement players would produce several more wins than team of fWAR replacement level players (I believe it’s 54 to 50, but I’m not sure). So, there is less less WAR, around 4 per team, in rWAR than in fWAR. So, $/WAR should be higher, maybe 10% or so.
Players on the free market often sign for the highest bidder and almost always for one of 2-3 highest (they may decline PIT or someone like that, but most of players don’t take way smaller contract than 2nd best). So, team that has signed them is generally most optimistic. When you pay people according to 60th or 70th percentile projection (specially considering injuries) instead of 50th, over large enough sample you have to lose. For example, NYY valued Burnett at something like 4/60. But when ATl started bidding, they had a choice between paying him 5/82.5 or losing him, even though they knew that 5th year was lost money, maybe even 4th. But they got WS win out of that.

Ben Hall
Member
Member
Ben Hall
4 years 6 months ago

Quick clarification, someone. Is rWAR Baseball Reference WAR?

wpDiscuz