FanGraphs Baseball


RSS feed for comments on this post.

  1. Matt,

    I’m really liking this series, and I’m excited that SIERA is at Fangraphs. I do think you should be a bit more cautious in some of your statements.

    “SIERA saw Tommy Hanson’s high strikeout rate in 2010 and knew that his low BABIP wasn’t a fluke.” Seems like “suspected” would be better than “knew.” While I’m trusting your research has shown that high strikeout pitchers have lower BABIP, there certainly have been high strikeout pitchers with average or high BABIP.

    “Both these pitchers generate weak contact because of their high strikeout rates — and both of their high fly ball rates mean they generate shorter, more catchable balls.” Again, these ideas may generally be true, but I don’t think we should state them as fact.

    That said, on second glance most of the time you did state these ideas as likelihoods rather than absolute truths, so I’m probably nitpicking. Looking forward to tomorrow’s entry.

    Comment by Ben Hall — July 20, 2011 @ 11:19 am

  2. The more I see, the more in love I become.

    Comment by Stormin' Norman — July 20, 2011 @ 11:20 am

  3. Great stuff, is this Sierra equation something that is/will be made public?

    Comment by slash12 — July 20, 2011 @ 11:42 am

  4. Yes. It was in yesterday’s article in that table with coefficients and variables. I gave the formula’s initial BP version and the new FanGraphs version.

    Comment by Matt Swartz — July 20, 2011 @ 11:46 am

  5. In reading this series it seems as though SIERA is the most effective tool in evaluating a pitcher’s true talent level. If this is the case would it make more sense to use SIERA to calculate a pitcher’s WAR? I’m not saying it is or isn’t just curious what the more enlightened readers, or the author thinks. Seems like if we’re looking for true talent level then the most accurate measure of a pitcher’s ability would make the most sense.

    Comment by The Only Nolan — July 20, 2011 @ 11:49 am

  6. Why are you looking at difference in rankings and not just difference between xFIP and SIERA themselves? You’re going to have some bias because of the inconsistent spacing in the rankings.

    Also, given that these are the pitchers with the least agreement between xFIP and SIERA and you’re only showing a few with gaps wider than .15 runs, it implies that xFIP and SIERA are really damn similar. Maybe you could tease their y-t-y correlation for us?

    Comment by Sky Kalkman — July 20, 2011 @ 11:51 am

  7. Great series so far. Just one question. How do you pronounce it? I’ve always said it like Sierra Mountains. Is it supposed to be S I ERA?

    Comment by Matty — July 20, 2011 @ 12:17 pm

  8. I think this is partly a philosophical question, but the argument for using FIP over xFIP, despite the latter being a better measure of skill, is that you need team WAR to go down when a HR is hit against them. Removing BABIP makes sense because it can be credited/debited from fielder’s WAR. There are some things that don’t get factored into team WAR but this at least lets a HR help the hitting team’s WAR while it hurts the pitching team’s WAR. You ask a fair question, though.

    Comment by Matt Swartz — July 20, 2011 @ 12:20 pm

  9. The correlation between xFIP & SIERA is .94, as opposed to SIERA and FIP which is .80, and just .62 between SIERA & ERA. That’s all pitchers with at least 40 IP between 2002-10.

    The logic behind using rankings was just that if the average xFIP and the average SIERA in a given season did not match, there could be too many pitchers on the SIERA>>xFIP or too many on the SIERA<

    Comment by Matt Swartz — July 20, 2011 @ 12:22 pm

  10. I pronounce it. I assume most people do, too, but I guess I wouldn’t know. Hmm…

    Comment by Matt Swartz — July 20, 2011 @ 12:25 pm

  11. since SIERA is so high on Ks does it factor in how much easier pitchers are to strike out?

    Comment by pbjsandwich — July 20, 2011 @ 12:29 pm

  12. So you’re not setting league-average SIERA to league-average ERA? I thought that’s what the coefficients from yesterday were about?

    Thanks for the correlations. Pretty damn close. Any chance you have FIP/xFIP handy?

    Comment by Sky Kalkman — July 20, 2011 @ 12:30 pm

  13. I love that we are getting better and better with our measures, but this methodology still undervalues greatly pitchers who actually show a skill in preventing hits: the guys who are able to attain a significantly sub-.300 BABIP.

    I realize that exceptions makes the process less mechanical, less easy to do, but we are basically penalizing pitchers like Matt Cain, Barry Zito, knuckleballers, for being good at what they do.

    Comment by obsessivegiantscompulsive — July 20, 2011 @ 12:34 pm

  14. The coefficients were set to minimize the mean square error for pitchers with 40 IP or more, weighting on IP, so they should be very close league average ERA but not quite. SIERA should probably be a little lower because it’s not trying to model the MLB BABIP of pitchers who throw 5 innnings and get demoted right away, because they generally aren’t MLB talent level. In practice, this won’t make a huge bit of a difference but rankings seemed easier to show a few on each side each year.

    xFIP & FIP correlation is .82.

    Comment by Matt Swartz — July 20, 2011 @ 12:36 pm

  15. As long as they keep their walks down and their HRs down, I think it won’t make much difference. I, for one, will say there’s value in Tim Wakefield doing what he does, just very little value.

    Comment by My echo and bunnymen (Dodgers Fan) — July 20, 2011 @ 1:36 pm

  16. Matt, I’m curious… If having a variable for pitcher handedness lowered your MSE, would you include it in SIERA?

    Comment by jason k — July 20, 2011 @ 2:12 pm

  17. Probably not, but it depends why. If I really only wanted to minimize MSE using next-year ERA, I would regress on next-year ERA. I want to pick up same year skill effects, so I regress on same-year ERA. The reason the %IP as SP was included was specifically because I noticed that SIERA/xFIP/FIP always are lower than ERA on average for SP and higher than ERA on average for RP. Since I’m focused now on the concept of SIERA as picking up BABIP on HR/FB skill levels, it only makes sense to allow RP to have lower BABIPs.

    As to why handedness probably doesn’t belong, I can’t think of a good reason why any effect would actually improve the relationship on out-of-sample data. In other words, is there a reason why LHP would have different BABIP or HR/FB than RHP, conditional on their peripherals? I can’t think of any, but maybe I’m missing something. It sounds like the type of thing that would only work on finding a best fit line, and would fall apart when correlating with next-year ERA or previous-year ERA or anything like that. I’m curious if I’m not thinking of anything though?

    Comment by Matt Swartz — July 20, 2011 @ 2:23 pm

  18. I actually like the idea of using SIERA for pitcher WAR. This is probably in the distant future at best, though, because it would complicate matters. To compensate, you would have to credit/debit fielders based on who was pitching.
    For example, if I understand this right, SIERA credits pitchers like Verlander because his flyballs are “more catchable.” So, it would make sense that Austin Jackson would get less credit for catching a Verlander flyball than an average one. I understand this adds yet another coefficient to each runs saved calculation (and would be a ton of work), but I think the idea is sound.

    Comment by Sitting Curveball — July 20, 2011 @ 3:03 pm

  19. Matt — Perhaps you will cover this in one of the next two parts of the series, but with respect to SIERA being valuable as a “predictor” for future ERA: have you tested how many IP it takes for SIERA to become “useful”?

    For example:

    - it’s one thing to look at a full-season’s worth of data and conclude that a pitcher is unlikely to repeat that performance next year (e.g. Clay Bucholz in 2010).

    - however, what I’m asking about is IN-SEASON predictive power. For example, let’s say a guy with a long history of 4.50 SIERA performance (Edwin Jackson type) comes out and puts up a 3.50 SIERA for the first two months of the season. If I’m projecting him going forward for the rest of the season, should I pay more attention to the smaller sample of data (3.50 ERA for the first two monts) with the idea that some underlying skills / approach have changed, or is it more likely that he will perform like his career numbers (which has a much larger sample size)? or would you do some regressed amalgam (weighted avg) of the two?

    Comment by batpig — July 20, 2011 @ 3:14 pm

  20. forget something?

    Comment by Max — July 20, 2011 @ 3:57 pm

  21. By “pronounce it”, I mean I sound it out. Like Sierra Mountains.

    Comment by Matt Swartz — July 20, 2011 @ 4:00 pm

  22. That’s a really good question. In general, the average baseball stat should be weighted about 50% for events that happened this year, 30% for events that happened last year, and 20% for events that happened the year before. But obviously if you have 3/5 as much data this year as last, you effectively are weighting last year and this year’s totals similarly. However, statistics that stabilize more quickly, you can do better than 50/30/20. Since the year-to-year correlation for BB%, SO%, and GB% are all around 70%, I guess maybe weight this year’s events as 2x as much as last years (as in weight this year’s SIERA through 81 games about as much as last year’s through 162 games). But there’s probably a more thorough way to do this. I’ll think about it. Really good question. Thanks.

    Comment by Matt Swartz — July 20, 2011 @ 4:05 pm

  23. Good “true” BABIP reducers can prevent .5 WAR per season. Great ones can prevent 1 WAR per season. Not a huge deal over the course of a season…but over a career, it is a huge deal. Of course, you don’t come to FG WAR if you are worried about career pitcher WAR anyway. People use rWAR for that.

    Comment by Matthew Cornwell — July 20, 2011 @ 4:52 pm

  24. (pats self on back)

    Comment by batpig — July 20, 2011 @ 6:22 pm

  25. matt i gotta say, the best part of this SIERA series is that you come back with timely responses to reader questions, something fangraphs typically is very lacking in

    Comment by jim — July 20, 2011 @ 7:29 pm

  26. Am I the only one that noticed that Jaime Garcia and Tommy Hanson have the same xFIP in 2010 but are ranked 20 spots apart? I’m not sure where the error is there…

    Comment by ryan — July 20, 2011 @ 11:19 pm

  27. Left-handers are known for being able to reduce stolen bases, get more pickoffs, and keep runners closer to first, limiting baserunning advances. That’s not usually considered a “pitching” skill, but it certainly influences how proficient a pitcher is at preventing runs.

    That’s probably a whole ‘nother ball of wax, though, as that skill varies greatly among pitchers, and it can’t be considered an inherent skill, as there are right-handers that are very good at this as well. I’m not even sure a metric exists right now that addresses this.

    Perhaps that belongs on the defensive side of a pitchers ledger, which presently isn’t factored into pitcher WAR.

    Comment by Nathaniel Dawson — July 21, 2011 @ 1:11 am

  28. Yes, big props to Matt for taking the time to read and respond to all of our questions and comments. A lot of time and effort on his part that’s very appreciated.

    Comment by Nathaniel Dawson — July 21, 2011 @ 1:17 am

  29. So, not to be a constant nag about this, but I do want to point out that FG uses all FB in its xFIP calculation, while the original intent was to use just outfield fly balls. Matt, do you have an opinion about which approach is better?

    Comment by studes — July 21, 2011 @ 7:17 am

  30. None consider it specifically, but Sean Smith’s WAR does not weed it out. Kinda the same thing.

    Comment by Matthew Cornwell — July 21, 2011 @ 7:22 am

  31. Good point. I would guess that xFIP with outfield fly balls only would probably perform better, because HR/FB is lower for fly ball pitchers overall, and IFFB/FB is higher for fly ball pitchers. Since that’s not really the direct reason it was created, I would guess the question is whether HR/FB (net of team) has more persistence than HR/OFFB (net of team). The one with the least persistence would be the way to go. I’d bet it’s OFFB and I’m guessing that improves xFIP too. I’ll see if I can check out this stuff. Thanks.

    Comment by Matt Swartz — July 21, 2011 @ 7:32 am

  32. Jaime Garcia is 3.62. It was a copy/paste problem. Sorry and thanks.

    Comment by Matt Swartz — July 21, 2011 @ 7:36 am

  33. Yet another metric that makes Nolasco look really solid.

    /frustrated fantasy owner

    Comment by Feeding the Abscess — July 21, 2011 @ 8:44 am

  34. Well, it helps that this is technical enough that it seems to be keeping the rude trolls away. Considering the tone of some of the comments you see on Fangraphs, I’m not surprised when the authors choose not to respond.

    Comment by joser — July 21, 2011 @ 11:52 am

  35. Yup, I’ve just started Nolasco and Vazquez back-to-back against the Padres. What could possibly go wrong, twice, I thought. :-P

    Comment by jrogers — July 21, 2011 @ 1:58 pm

  36. Thanks, Matt. Maybe it’s the wording, but isn’t the factor with the MOST persistence the way to go?

    Comment by studes — July 21, 2011 @ 2:27 pm

  37. Well, I mean that if HR/OFFB has less persistence than HR/FB, then you’d want to regress xHR to OFFB*(league average HR/FB). Basically, figure out which number is less likely to be a skill and make it league average.

    Comment by Matt Swartz — July 21, 2011 @ 2:36 pm

  38. Gotcha

    Comment by studes — July 21, 2011 @ 2:48 pm

  39. Sorry if this was brought up in another post but any chance you move SIERA to the dashboard right next to ERA, xFIP and FIP for a quick comparison? Great work

    Comment by russel58 — July 23, 2011 @ 12:19 am

  40. Good question!

    Comment by brad — July 25, 2011 @ 12:36 am

Leave a comment

Line and paragraph breaks automatic, e-mail address never displayed, HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current day month ye@r *

Close this window.

0.119 Powered by WordPress