Criminals of WAR

I’m not going to single anyone out, since we’re all guilty of abusing FanGraphs’ Wins Above Replacement metric. But I’ve been seeing cases pop up where it’s getting out of hand. So I’ve set up a few guidelines for how to go about using WAR responsibly. Do not break these rules, or I may call you out.

1. Do not exclude baserunning from a position player’s WAR. I’m sure David Appelman will include baserunning in the next edition of WAR, since it’s so easy to calculate, but the numbers are already out there, so please take the time to go to BP, B-Ref, or BJOL to look up the numbers and tack them on.

2. Do not place undue trust in WAR for catchers. How much of a catcher’s value do you think is in his defense? I’ll give you a hint: it’s a lot. FanGraphs has unfortunately yet to give an effort to quantifying this vital aspect of the game, other than with the positional adjustment. In fact, catchers should possibly be considered a separate group of players with a separate replacement level and therefore be treated as different from all other position players.

3. Do not place undue trust in WAR for pitchers. First off, pitcher defense and hitting aren’t included. This should be righted ASAP. Then there are the more nuanced issues like how leverage is accounted for and the conversion of FIP to runs. Personally, I’d trust the calculations of David Gassko’s pitching runs created or StatCorner’s WAR well before I would FanGraphs’ WAR.

4. Do not cite WAR as a measure of skill. WAR measures production. FanGraphs has a lot more granular data if you’re trying to assess skill. And if you’re going to try to make a projection of WAR, regress each component individually. Also, players with negative WAR still may have value if they excel at a certain skill that can be leveraged.

5. Do not use the linear conversion of WAR to salary to determine what a team should be willing to pay a free agent. Every team has a different scale, depending on that team’s market and where the team is on the win curve. Few teams should pay $5 million for a single win.

I’m sure there are other commandments I’m missing, so feel free to add your own.


Print This Post

Leave a Reply

24 Comments on "Criminals of WAR"

Notify of
avatar
Sort by:   newest | oldest | most voted
Dave Studeman
Guest
Dave Studeman

But FIP is production too: strikeouts, walks and home runs allowed.  It’s a subset of ERA and it’s more constant, or more predictable, than ERA, but it’s still production.

#5 is my personal favorite.  I think the salary figures on Fangraphs are good guidelines, but that’s all they are.  They’re the beginning of a good discussion about what a team should pay a player, not the end.

Think Blue Crew
Guest
Think Blue Crew

Regarding #5, Vince Gennaro’s Diamond Dollars explains brilliantly why some teams would pay different $ depending on their current and expected win total. People tend to forget (I’m a culprit as well) that the marginal win changes for teams, even for the same team in different years. A middle-rotation starter is not worth much to the Pirates, as it will not mean much in terms of increasing playoff odds, but a wildcard team may pay much more for the same pitcher, depending how many (or few) wins the team needs in order to make the playoffs.

dkappelman
Guest
dkappelman
<> I don’t mean to get snippy, but this was a completely unsupported statement.  tRA has been shown to be barely better than FIP.  Neither PRAA or tRA WAR on stat corner adjust relievers for leverage to my knowledge.  They don’t have dynamic run converters for starting pitchers as far as I know either.  Also, none of the pitcher WAR include defense for pitchers, which is going to be pretty minimal, and you can always look up a pitcher’s WAR on offense (which we do calculate) and add it to their defense. What exactly is there not to “trust” about… Read more »
dkappelman
Guest
dkappelman

“Personally, I’d trust the calculations of David Gassko’s pitching runs created or StatCorner’s WAR well before I would FanGraphs’ WAR.”  was the statement I was trying to quote for the post above.

MikeS
Guest
MikeS

Thank you ever so much for point number 5.  Beyond the obvious that the Yankees will pay more for one WAR than the Pirates, there is so much more to the economics.  A decent shortstop or a middle of the rotation starter is worth much more to the Twins who may see it as the one piece they are missing than it is to the Royals who need so much more to even be worth noticing.  It makes sense for the Twins to overpay for that, but not the Royals.

Adam W.
Guest
Adam W.

@dkappelman: I think he’s referring to issues like the one raised in this article: http://mobile.beyondtheboxscore.com/2009/10/28/1104776/ricky-nolasco-4-war-or-1-war

Jeremy Greenhouse
Guest
Jeremy Greenhouse
Dave, thanks for commenting. The entire way that we’re thinking about adjusting relievers for leverage might be flawed. The purpose is to find out the value of the pitcher, isolated from the context in which he pitches. The two best ways to do this are to either not account for leverage or to assign every single pitcher a “deserved” leverage index, including starters, based on the optimal average LI he should pitch in, independent of his actual LI. StatCorner doesn’t account for leverage, which I’m fine with, and PRC does pretty much what you need by adjusting the pitcher’s run… Read more »
Hecubot
Guest
Hecubot

The number one argument for the value of catcher’s defense is probably just the defensive spectrum.  It’s clear that we haven’t got a handle on how to measure a catcher’s defensive production, but there are a lot of little clues coming forward.

I particularly liked the study that showed that Piazza was a plus defender at blocking pitches in the dirt and was able to reclaim some of his defensive worth for his poor throwing.  Stuff like that helps make clear why he was kept at catcher for so long.

dkappelman
Guest
dkappelman
Jeremy, thanks for clarifying. Starting pitchers on FanGraphs are not leverage adjusted on FanGraphs, because except for some strange cases, they’re all going to have an average leverage of 1 anyway.  So there’s really nothing to complain about here, they are completely context neutral. Relief pitchers use a regressed gmLI leverage adjustment.  So it only accounts for the situations they were used in.  I see what you’re saying about optimal average LI, but WAR, like you said is not really predictive (with FIP maybe a little more so), so I’d say adjusting for leverage in the situation they actually pitched… Read more »
Jeremy Greenhouse
Guest
Jeremy Greenhouse
I understand starters have the same average LI. My point is that all pitchers are from the same group of players. They (starters and relievers) shouldn’t be treated differently. I’m not smart enough to come up with the correct metric, but I’d imagine the replacement level (or whatever you want to call it) would be fluid, based on the expected outs per outing, and the deserved leverage index would be fluid, based on the expected outs per outing as well as the pitcher’s run environment. The regressed version of tRA tries to account for everything the pitcher controls, and dismiss… Read more »
Nick Steiner
Guest
Nick Steiner

Statcorner calculates WAR using the straight park adjusted tRA.  The regressed tRA is “just for show”.

I agree that PRC is probably the best, mainly because it uses a dynamic run estimator I believe.

The regressed version of tRA tries to account for everything the pitcher controls, and dismiss everything he can’t control.

Regressed tRA just regresses tRA to league average based off of the sample size.  So a 6.00 tRA will be around a 5.2 tRA*, or something.

Jeremy Greenhouse
Guest
Jeremy Greenhouse

Nick, I don’t know where you’re getting any of that. If you have an explanation of StatCorner’s pWAR, please pass that along. You’re off on your assessment of tRA*. It’s park adjusted and every component that goes into tRA is regressed individually. Homer Bailey had a tRA of 7.64 and tRA* of 5.03 while Miguel Batista had a tRA of 7.93 and tRA* of 6.58.

dkappelman
Guest
dkappelman
“My point is that all pitchers are from the same group of players. They (starters and relievers) shouldn’t be treated differently. I’m not smart enough to come up with the correct metric, but I’d imagine the replacement level (or whatever you want to call it) would be fluid, based on the expected outs per outing, and the deserved leverage index would be fluid, based on the expected outs per outing as well as the pitcher’s run environment.” Well, back to the original point, which is your complaint about leverage not being applied properly or on a sliding scale, if you’re… Read more »
CH
Guest
CH

“Like all these stats, none are perfect or should be used in a vacuum, but I guess if you’re going to pick one, you could do a lot worse than WAR.”

Well said.  So many flame wars could be avoided if people would follow that simple logic.

Firpo
Guest
Firpo

Not to be a noob, but what is BJOL? And where are the baserunning stats at B-Ref?

Nick Steiner
Guest
Nick Steiner

Jeremy – I’m saying the version of WAR that’s shown at StatCorner is calculated using tRA, which is park adjusted by NOT regressed.  tRA* is the regressed version, and that’s not used for anything in particular.  That’s why the statement I quoted from you above is confusing.

Jeremy Greenhouse
Guest
Jeremy Greenhouse

Firpo, Bill James Online, and if you go to a hitter’s page on Baseball Reference and scroll down, there’s a section with baserunning stats.

Jeremy Greenhouse
Guest
Jeremy Greenhouse
Dave, I appreciate you taking the time. I’m realizing I overstated my case when it came to pitcher’s WAR. The other points were all pretty much fact, and the non-defense/fielding arguments against pitcher WAR are based on theory.  However, I still don’t think you should treat relievers and starters differently because they’re from the same group of players. Think of it in terms of positional adjustments. A one inning pitcher (think of innings in terms of expected outs) gets a negative positional adjustment because of the lack of scarcity and lack of difficulty at the position. A six inning pitcher… Read more »
WY
Guest
WY

Thanks for writing this. A lot of this needed to be said. It drives me nuts to see people say “So-and-so was worth $13.6M last year” as if that is an etched-in-stone fact. That is really sloppy thinking.

Also, to Dave Appelman: I read and appreciate Fangraphs, but I didn’t interpret the main thrust of this article as anti-Fangraphs as much as I read it as a plea for people to think more critically about and be more careful with what they read there.

Jack Moore
Guest
Jack Moore
Re: the point on catcher defense, Tango said that the difference between Piazza/Pudge (worst and best of all time) was only 20 runs.  RJ Anderson cites it here ( http://www.beyondtheboxscore.com/2009/1/30/740437/rambling-on-catcher-defens ), but I don’t remember the original source nor do I have time to find it now.  The point being, if the difference between the best and worst of all time is only 20 runs, chances are we’re not missing by a whole lot.  We should be able to do pretty well as long as we make a mental adjustment based on what we know about the player, his age,… Read more »
Red Sox Talk
Guest
Red Sox Talk

Hear, hear. Took a stab at starting the salary discussion here:
http://saberrattling.wordpress.com/2009/11/12/money-matters-what-do-teams-pay-per-win/

Would appreciate input. Thanks!

B
Guest
B
I’m unsure of #5.  I should probably read the book someone else mentioned before commenting, but oh well.  Basically, we’re not concerned about the marginal value of a win to a specific team as much as what the market will pay for them.  If the Pirates won’t pay ~$4.5M for a win…well, the Pirates won’t be landing any free agents, other teams that will pay more will get them.  I guess I could see a price discrimination type supply and demand setup, though, where some teams can pay more and some teams can pay less, and with a limited number… Read more »
Harold
Guest
Harold

99 times out of 100, the stat nerd and the grizzled veteran manager will come to the same conclusion as to who are the best players. Where it really gets interesting is when you’re talking about the mediocre to worst (yet still essential) players. The problem then becomes which stat or set of stats to value.

acerimusdux
Guest
acerimusdux
1. wOBA/WAR does include SB and CS, which is what most people think of first when they think of baserunning. There are subtler aspects, such as taking an extra base, that are missed for now, but they tend to not amount to a large run value. I do recommend B-Ref though for the XBT% (% extra base taken) and RS% (% runner scored). Also, in the situational stats, look at BRS% (% of base runners scored by the hitter) and advances relative to average for for hints of other possible skills not captured by WAR. 2. Agree a useful measure… Read more »
wpDiscuz