Using FIP to evaluate pitchers? I wouldn’t

image
Is Derek Lowe the most overvalued pitcher in baseball this year? If you’re looking at FIP he is. (Icon/SMI)

I’ve seen a lot of articles popping up recently that attempt to evaluate pitchers using the FIP statistic. While the original, underlying premise for FIP is sound, and while it’s absolutely better to use than simple ERA, and while there are certainly uses for FIP in some circumstances, for 99 percent of fantasy purposes, I ignore FIP completely and absolutely.

What is FIP?

FIP is a statistic that attempts to estimate what a pitcher’s ERA would be based on his peripheral statistics—or rather, the peripheral statistics originally suggested by Voros McCracken when he introduced DIPS Theory. This includes strikeouts, walks, hit-by-pitches, and home runs. While we know that strikeouts and walks are extremely important, eight years after Voros’s work was originally published, we can definitively say that home runs aren’t entirely under a pitcher’s control.

Here’s how things work: a pitcher can influence the rate of fly balls he gives up. By this logic, the more fly balls allowed, the more total balls will clear the fences for home runs (all else being equal). However, while a starting pitcher can control the rate of fly balls allowed, he cannot do a very good job of controlling the rate at which those fly balls become home runs (with very few exceptions).

To put it more simply, starting pitchers don’t have any underlying ability to prevent home runs—the best they can do is prevent fly balls. If those fly balls are clearing the fence at too high a rate (or too low), we say that the pitcher has been unlucky (or lucky).

And therein lies the problem with FIP.

Pitchers undervalued by FIP

Here is a list of pitchers that have been undervalued by FIP so far in 2009. If you were to look at the FIP of these pitchers, you would believe that they have pitched worse than they actually have.

To explain the table below, we’re seeing the pitcher’s LIPS ERA (which also estimates ERA but normalizes HR/FB, in addition to some other things), his FIP, and the difference between the two. I’ve also included each pitcher’s HR/FB rate so you can see just how much this stat influences FIP (for the worse). League average is around 11 percent, and you’ll notice that just about every player on this list is well above that number.

+-------------+------------+----+----+-------+------+----------+------+----------+-------+
| LAST        | FIRST      | G  | GS | IP    | ERA  | LIPS ERA | FIP  | LIPS-FIP | HR/FB |
+-------------+------------+----+----+-------+------+----------+------+----------+-------+
| Geer        | Joshua B   | 11 |  9 |  58.7 | 6.14 |     4.48 | 5.97 |    -1.49 |   18% |
| Guthrie     | Jeremy     | 13 | 13 |  73.3 | 5.52 |     4.39 | 5.78 |    -1.39 |   17% |
| Colon       | Bartolo    | 11 | 11 |  55.3 | 4.23 |     4.21 | 5.60 |    -1.39 |   19% |
| Harden      | Rich       |  9 |  9 |  49.7 | 4.53 |     3.18 | 4.47 |    -1.29 |   23% |
| Bush        | David T    | 13 | 12 |  74.7 | 4.58 |     4.34 | 5.52 |    -1.18 |   17% |
| Baker       | Scott S    | 11 | 11 |  67.7 | 5.59 |     3.76 | 4.93 |    -1.17 |   17% |
| Young       | Chris      | 13 | 13 |  73.7 | 4.76 |     4.16 | 5.13 |    -0.97 |   11% |
| Johnson     | Randy      | 13 | 13 |  70.0 | 4.89 |     3.61 | 4.55 |    -0.94 |   22% |
| Galarraga   | Armando    | 13 | 13 |  69.7 | 5.56 |     5.08 | 6.02 |    -0.94 |   17% |
| Blanton     | Joe M      | 12 | 12 |  71.3 | 5.17 |     3.97 | 4.86 |    -0.89 |   19% |
| Suppan      | Jeff       | 13 | 13 |  70.3 | 4.48 |     4.88 | 5.72 |    -0.84 |   16% |
| Moyer       | Jamie      | 12 | 12 |  66.3 | 6.11 |     4.97 | 5.81 |    -0.84 |   16% |
| Parra       | Manny      | 13 | 13 |  64.7 | 7.52 |     4.39 | 5.12 |    -0.73 |   15% |
| Davies      | Kyle K     | 13 | 13 |  77.0 | 5.14 |     4.67 | 5.34 |    -0.67 |   14% |
| Eaton       | Adam       |  8 |  8 |  41.0 | 8.56 |     5.29 | 5.96 |    -0.67 |   16% |
| Looper      | Braden     | 12 | 12 |  68.0 | 4.50 |     4.58 | 5.23 |    -0.65 |   18% |
| Oswalt      | Roy        | 14 | 14 |  82.3 | 4.37 |     3.96 | 4.56 |    -0.60 |   15% |
| Slowey      | Kevin      | 13 | 13 |  78.7 | 4.23 |     3.57 | 4.09 |    -0.52 |   11% |
| Sonnanstine | Andy       | 13 | 13 |  70.3 | 6.65 |     4.97 | 5.48 |    -0.51 |   16% |
| Carmona     | Fausto C   | 12 | 12 |  60.7 | 7.42 |     5.58 | 6.08 |    -0.50 |   18% |
+-------------+------------+----+----+-------+------+----------+------+----------+-------+

While true for nearly all players to some extent, if you see analysis done on these players using FIP, take note that the conclusions should probably be much more positive.

Pitchers overvalued by FIP

Here is a list of pitchers that have been overvalued by FIP so far in 2009. If you were to look at the FIP of these pitchers, you would believe that they have pitched better than they actually have.

+-------------+------------+----+----+-------+------+----------+------+----------+-------+
| LAST        | FIRST      | G  | GS | IP    | ERA  | LIPS ERA | FIP  | LIPS-FIP | HR/FB |
+-------------+------------+----+----+-------+------+----------+------+----------+-------+
| Lowe        | Derek      | 13 | 13 |  83.7 | 3.44 |     4.75 | 3.20 |     1.55 |    3% |
| Billingsley | Chad R     | 13 | 13 |  85.7 | 2.73 |     4.07 | 2.73 |     1.34 |    3% |
| Stults      | Eric W     |  9 |  9 |  45.0 | 4.80 |     5.47 | 4.14 |     1.33 |    3% |
| Jurrjens    | Jair       | 13 | 13 |  79.0 | 2.85 |     4.83 | 3.53 |     1.30 |    5% |
| Garland     | Jon        | 13 | 13 |  76.0 | 5.45 |     6.42 | 5.13 |     1.29 |    9% |
| Blackburn   | Nick N     | 13 | 13 |  84.3 | 3.31 |     5.40 | 4.22 |     1.18 |    7% |
| Greinke     | Zack Z     | 13 | 13 |  94.3 | 1.72 |     3.01 | 1.92 |     1.09 |    2% |
| Pelfrey     | Mike A     | 11 | 11 |  65.3 | 4.68 |     5.48 | 4.40 |     1.08 |    8% |
| Lincecum    | Tim        | 13 | 13 |  88.0 | 2.66 |     3.24 | 2.22 |     1.02 |    4% |
| Rodriguez   | Wandy      | 13 | 13 |  79.7 | 2.82 |     4.16 | 3.16 |     1.00 |    7% |
| Hammel      | Jason A    | 12 |  9 |  54.0 | 4.33 |     4.94 | 3.99 |     0.95 |   11% |
| Carpenter   | Chris      |  7 |  7 |  44.0 | 1.23 |     3.34 | 2.39 |     0.95 |    3% |
| Jimenez     | Ubaldo     | 13 | 13 |  82.7 | 3.92 |     4.21 | 3.27 |     0.94 |    6% |
| Meche       | Gil        | 13 | 13 |  75.3 | 3.70 |     4.00 | 3.08 |     0.92 |    3% |
| Happ        | J.A.       | 16 |  4 |  45.3 | 2.98 |     5.34 | 4.42 |     0.92 |    9% |
| Pineiro     | Joel       | 12 | 12 |  76.7 | 3.99 |     4.01 | 3.09 |     0.92 |    4% |
| Verlander   | Justin B   | 13 | 13 |  86.3 | 3.02 |     3.26 | 2.36 |     0.90 |    6% |
| Lee         | Cliff      | 13 | 13 |  88.0 | 3.17 |     4.03 | 3.15 |     0.88 |    6% |
| Miller      | Andrew M   | 10 |  8 |  46.0 | 4.30 |     4.45 | 3.62 |     0.83 |    5% |
| Johnson     | Josh       | 13 | 13 |  89.0 | 2.73 |     3.55 | 2.72 |     0.83 |    6% |
| Washburn    | Jarrod     | 12 | 12 |  76.3 | 3.30 |     4.53 | 3.71 |     0.82 |    7% |
| Padilla     | Vicente    | 11 | 11 |  65.3 | 4.82 |     5.70 | 4.89 |     0.81 |    8% |
| Wakefield   | Tim        | 12 | 12 |  76.0 | 4.50 |     5.50 | 4.75 |     0.75 |    6% |
+-------------+------------+----+----+-------+------+----------+------+----------+-------+

While true for nearly all players to some extent, if you see analysis done on these players using FIP, take note that the conclusions should probably be much more negative.

Evaluating on your own

If you’re looking for an alternative to FIP for evaluating players on your own, you have a few options. LIPS ERA is my favorite, and it’s now available in Heater Magazine. You could also simply go with xFIP, which appears on every THT player page and is sortable on the leaders pages. Other free options include stats like tRA and QERA.

Concluding thoughts

Hopefully this cleared some things up for some of you, as I know I’ve gotten a few questions about FIP this season. For those of you who were already aware of its shortcomings, well, hopefully you at least enjoyed chewing on the leaderboards presented. As always, if you have any questions, feel free to e-mail me or comment.


Print This Post
Sort by:   newest | oldest | most voted
Mike Ketchen
Guest
Mike Ketchen
Derek, coming from a guy who has been very successful playing fantasy, I use FIP a lot and I have enjoyed a lot of success with it. Your post here slighting it just seems lacking to me. I am a fan of your work and usually come away learning something new each time you write. However something just seems off here. Do you use LIPS and have seen its value from year to year? For example I was big on Vazquez and Lowe this season and both have paid off big for me. Did Lips nail any guys coming into… Read more »
The Real Neal
Guest
The Real Neal

When evaluating a pitcher’s ‘true’ talent level this may be useful to know.  But in a given season, which is the range we’re talking about for most fantasy purposes, the HR/FB ratio is going to be influenced by the park the pitcher is playing in,  opposing hitters etc.  It doesn’t make much sense to dismiss it as a vagary of sample size when we know that pitchers in Philadelphia and Colorado are going to tend to yield a higher ratio than those in Petco and Safeco.

David Gassko
Guest
David Gassko

The Real Neal,

LIPS actually takes into account a pitcher’s home park so that a pitcher’s LIPS can be compared directly to his ERA. In other words, LIPS “knows” that a pitcher in PETCO will have a lower HR/F than a pitcher at Coors and adjusts accordingly.

The Real Neal
Guest
The Real Neal

Well, color me confused.  It’s difficult to understand what LIPS is because you haven’t given a definition of it.  The other confusing thing is that when I look at the ERA, the LIPS and the FIP, assuming all three are supposed to represent runs/game – LIPS is the least accurate predictor.  Unless your fantasy league uses LIPS… what is the point of the article again?

David Gassko
Guest
David Gassko

The Real Neal,

Both Derek and I have written about LIPS on THT before, but I have been thinking that it might be time for me to write a primer on THT Fantasy. I’ll work on it, and try to have it posted this week. The point of this article (and of LIPS) is that LIPS is the single-best predictor of ERA based on a season’s worth of stats (or less). Better than FIP or ERA itself or anything else that’s out there. So if your league uses ERA, you want to know a pitcher’s LIPS.

Drew
Guest
Drew

How do you explain a guy like ubaldo jimenez, who consistently has a low hr/fb?
Also, Lips era has joe blanton having a lower era than chad billingsley?? I know it is just one example, but it raises my suspicions on the method

John Burnson
Guest
John Burnson
Jimenez’s HR/FB the last four seasons are .10, .13, .10, and .08. I don’t think that the first three numbers are unduly low, even for Coors. His low HR/FB this year may be related to his rising GB/FB: from 1.5 in 2007 to 2.8 last year to 3.4 this year. Research indicates that a higher GB/FB is correlated with a lower HR/FB. As for Blanton/Billingsley, Billingsley has the better K/9 (9.5 vs. 8.1) but Blanton has the better K/BB (2.8 vs. 2.4). The reason you’re suspicious is because Billingsley has HR/FB of 5% (and an ERA under 3.00) whereas Blanton… Read more »
Kampfer
Guest
Kampfer
Just by looking at your table, it seems that FIP is closer to ERA than LIPS. I am also not sold on the idea that pitcher’s stuff or whatever he has does not have any relationship to HR/FB. HR/FB of pitchers are rather steady throughout one’s career. Could it be that if those flyballs are hit better and end up flying longer than they should be? Some had research on sinker and find out that a faster sinker is better than a diving sinker in generating GB. Could it be that pitcher with high velocity and simultaneously generates a lot… Read more »
John Burnson
Guest
John Burnson

FIP will always be better backward-looking than LIPS, because FIPS uses the pitcher’s actual HR whereas LIPS (and xFIP) use predicted numbers.

Andrew
Guest
Andrew

Good stuff, Derek. I like the philosophy behind LIPS ERA and Baseball HQ’s xERA best.

David Gassko
Guest
David Gassko
Kampfer, John’s explanation is exactly right. We don’t care about looking backwards, since we already know what a pitcher’s ERA was. What we’re interested in is what his ERA will be. If you compare how good FIP and LIPS are at predicting that, you’ll see that LIPS takes the cake and is therefore more useful. As for pitchers controlling their HR/F, I’ve done a lot of work on this, the definitive piece being in the Hardball Times Annual 2007. Pick it up if you’d like to see all the gory details, but the basic conclusion is that pitchers have a… Read more »
Eric/OR
Guest
Eric/OR

Honest question – if one replaces “FIP” with “xFIP” above, does this article make sense?  Wouldn’t that *precisely* address the complaint with FIP?

David Gassko
Guest
David Gassko

xFIP is close to LIPS, but it does not take into account as much as LIPS – things like park effects, batted ball types beyond OF flies, etc. xFIP is good, but LIPS is better.

Derek Carty
Guest
Derek Carty

Hey guys,
I’ve got a full plate today, but I’ll be around tonight to address all of these comments.

Thanks!

Colin Wyers
Guest
Colin Wyers

Has anyone done a rigorous test of xFIP/LIPS/xERA/QERA/tRA and whether it actually reflects true talent better than FIP? I’m not doubting that someone has, I just don’t know where to find it.

And Fangraphs now publishes projections updated in-season – shouldn’t that be a better estimate of true talent than any of those?

Eric/OR
Guest
Eric/OR

David – I suspected as much, thank you.

John Burnson
Guest
John Burnson

It’s a question of what question you’re asking. There are two questions, “What should the pitcher’s ERA be, given his exhibited skills?” and “What should the pitcher’s ERA be, given his predicted skills?”, and both are legitimate. That’s why Heater includes both LIPS ERA and True Talent ERA.

David Gassko
Guest
David Gassko

Colin,

What John said. Also, I have my doubts as to the quality of the updated projections on Fangraphs, though I’m not going to expound on them in this forum.

Sky Kalkman
Guest
Sky Kalkman

Also, for fantasy, we don’t really care about a lot of the neutralization, right?  We WANT to know what ERA a pitcher’s skills combined with park and defense will produce.  You want to use something like DIPS, but using actual team BABIP rates and ballpark rates for HR/FB, BABIP, etc.

DT
Guest
DT

excellent article and comments by everyone. 
I would love a detailed explanation of LIPS vs xFip if possible.

Colin Wyers
Guest
Colin Wyers
“Also, for fantasy purposes, it’s sometimes useful to know how a pitcher is performing *now*, not necessarily his true talent level (or our estimation of it, rather).” If that’s true – and I don’t know that it is, necessarily – then we certainly should be FIP, shouldn’t we? If what we’re after is actual value, not true talent level, it’s wrong to regress the HR/FB. The reason to use FIP or DIPS in that case is not to regress the pitcher’s performance to the mean, but to regress the pitcher’s FIELDER’S performance. In other words, we use FIP to measure… Read more »
David Gassko
Guest
David Gassko
Colin, What Derek meant (I believe) was that sometimes a pitcher’s performance is not in-line with his track record. What we want in that case is a statistic that can tell us whether or not his performance thus far has been “real,” in so far as his peripherals backing it up. LIPS is a more useful tool here than FIP. Yes, an updated projection is best but (1) There aren’t any top-notch regularly updated projections out there as far as I’m concerned, and (2) An updated projection can’t give you the same kind of scope that LIPS might. It tells… Read more »
John Burnson
Guest
John Burnson

To rely on FIP, you have to believe that a pitcher deserves his HR/FB, often wacky in only 2+ months of play. If you want something simple, swap out HR for 0.11*FB (or whatever), which is xFIP.

SharksRog
Guest
SharksRog
The numbers here indeed make Tim Lincecum seem overvalue by a LOT—but is he? I agree that Tim’s 2.22 FIP is too low—although his ERA after his first two of what may have been weakened starts is a surprisingly close 2.15. What I don’t agree is that his LIP is closer to how he has really pitched than his FIP. First of all, Tim’s ERA of 2.66 is actually closer to the 2.22 FIP than the 3.24 LIP.  Second, that 2.66 ERA includes a BABIP against of .338.  Tim’s bases per hit are among the lowest of all pitchers and… Read more »
Cyril Morong
Guest
Cyril Morong

I think Colin asked about year-to-year correlation of HR/FB rates. Did anyone address? Maybe I missed it or mis-understood something.

Dave Studeman
Guest
Dave Studeman

One thing to remember is that xFIP is a park-neutral stat (at least on the home run line), so it won’t predict future ERA well.  It isn’t constructed to do that.

Derek Carty
Guest
Derek Carty
Mike Ketchen, As David and John have already pointed out, LIPS does indeed have better than value than FIP year-to-year.  The Real Neal, I think you might be overestimating the impact of park factors.  It’s much sounder to assume that the player will post a normal HR/FB (we’ll say 10%) and then park adjust from there.  For the most extreme HR park in baseball (U.S. Cellular Field – PF = 1.26), we’d expect a pitcher to post a 12.6% HR/FB while pitching at home.  If we assume a league average mix of road parks though (and an even playing time… Read more »
Colin Wyers
Guest
Colin Wyers

…can someone put a correlation coefficient on it? An RMSE? Average Absolute Error? I’m willing to buy into these more advanced FIP-like measures, but I just want some evidence that they actually do what they say on the label.

Derek Carty
Guest
Derek Carty
Drew, I think John covered your question pretty well.  As I qualified in the article, some pitchers do indeed vary from this general rule, but not very many and not by very much.  Also, because HR/FB is so unstable, we really need to see several years of a pitcher beating the rule for us to say he has some sort of knack for it.  As to Ubaldo, he’s only really bested a league average HR/FB for one and a quarter years now, so I don’t think we can call him an exception yet.  Check out Brett Myers, though.  There’s very… Read more »
Derek Carty
Guest
Derek Carty
Eric/OR, It absolutely would, which is why it’s frustrating to see analysts continue to use FIP!  I do prefer LIPS, though, because it’s methodology is sounder, accounts for more things, and tests I’ve run have shown it to have a higher predictive value. Colin, I would have addressed you the same way John and David did.  It depends upon the question you’re asking, and I too have some reservations with using the in-season ZiPS projections.  A few I’ve seen have looked a bit strange. Also, for fantasy purposes, it’s sometimes useful to know how a pitcher is performing *now*, not… Read more »
David Gassko
Guest
David Gassko

Cyril,

It was answered. The year-to-year correlation of HR/OF is very, very low, so much so that it’s better to ignore home runs completely unless you have many years of data.

Cyril Morong
Guest
Cyril Morong

Thanks. I was just wondering if there was an actual number like a correlation of .01. I found a correlation of .30 for HR/BFP, taking out IBBs, year-to-year from 2007-2008, for pitchers with 500+ BFP in each year. That seems a little too high for luck. But it is way less than it was for strikeout rate (.78) or walk rate (.67).

Maybe the .30 is due to pitchers tending to be flyball pitchers year in and year out. Do pitchers ever change or do they tend to stay flyball or goundball pitchers throughout their careers?

David Gassko
Guest
David Gassko

Taking pitchers with 350+ BFP, I found a y-t-y “r” of 0.84 for GB/Batted Ball and a 0.17 “r” for HR/OF in the 2007 Annual.

Cyril Morong
Guest
Cyril Morong

Thanks, I appreciate it.

David Gassko
Guest
David Gassko

Exactly, James. The range of “true” HR/OF is probably around 9-13%.

James
Guest
James

Thanks! I will be sticking to xFIP since I don’t have access to LIPS. However, I’ll make sure to keep an eye on the stats that xFIP doesn’t take into account, which I normally do anyway!

James
Guest
James

Will a pitcher not go far from 11% HR/FB since pitchers cannot control how many HR fly out of the park? If so, what is the normal variance?

Matt
Guest
Matt

Derek,
I am just looking back on this article now and noticing that the “overvalued” guys are almost all pitchers that ended the season as players I would have wanted, but the “undervalued” guys are almost guys that were not that good.

Have you done further analysis on this?

wpDiscuz