Predicting Shutdowns and Saves

“I heard the jury was still out on… Science…”

–G.O.B. Bluth, Arrested Development

Saunter over to the Shutdown and Meltdown leaderboard and you will find a curious leader: The San Diego Padres.

Yes, the Friars have somehow amassed an absurd 31 shutdowns (SD) while winning a sport-worst nine games. This seems somewhat surprising, because experience has taught me — among many things about cats — that bad teams typically do not need great bullpens. They might have them (*cough* Joakim Soria *cough cough*), but they do not need them because they get destroyed early and often.

Take my hand and follow me down Logic Avenue: The worst teams will not often participate in three-run, save-opportunity games — or even one-run games. Instead they will presumably watch this and employ a slew of mop-up long men who do not affect the game’s already-decided outcome. I mean, c’mon, three-fifths of the Royals rotation is likely to allow five runs before finishing the 6th inning. What can a bulllpen possibly do when the score is 5 to -1?

In the same stroke of logic, wouldn’t we expect the best teams to have fewer save opportunities? Unlike impressively mediocre teams, like the Chicago Cubs, the New York Yankees spend a good deal of time slapping homers and trouncing weaklings. As a result, we should expect they play fewer close games than the Cubs, who must crawl, snarl, and curse their way into every victory and loss.

Well, that may be logical, but it’s not entirely correct.

What do we know?
Any fantasy baseball nerd will tell you: Don’t waste a high pick on a Mariners closer. Why? Well, the Mariners are not going to win a lot of games, and so, by default, their closers — regardless of how good they may be — will not rack up many saves. From 2000 through 2010, we can see there is a direct and easy correlation between how good a team is (as measured by winning percentage) and how many 3-runs-or-less victories they have:

However, we should expect the same thing to be true for the aforementioned San Diego Padres, whose relievers already have the bards singing of their majesty. The team has amassed only five saves, but the bullpen has been pivotal in almost every game. How can that be?

Well, the first truth we encounter is that my previous assumption was wrong: Great teams don’t avoid three-run and one-run games. If we again refer to those 2000 through 2010 seasons, we see a big, fat shotgun distribution of results:

Yeah, the regression line has a frowny face, meaning my assumption appears kinda correct, but we can clearly see any relationship here is sketchy at best (the low R-squared confirms this).

Whattabout blowouts?
Surely the Yankees form more congo lines than the Royals, don’t they? Looking at the distribution of blowout games (here defined as games decided by 4 or more runs) from the aughties, we again see a shotgun plot:

This means the mediocre teams — as well as the good and bad teams — are playing in an equalish number of blowout games. The regression says great and bad teams play in more blowouts, but frankly the correlation is not strong enough for any predictive power. However, we can see with certainty who benefits from these blowout games, the excellent teams:

Putting It All Together
The Padres aren’t a terrible team, but they’re likely to finish below .500 — so say projection systems and gurus and such. The fact that their bullpen has played the hero in these early weeks becomes even more surprising when we consider what we just learned:

  • Every team plays in about 100 three-runs or less games, 60 blowout games, and 45 one-run games — and the distribution is about equal, regardless of how good a team is.
  • The good teams win a lot of blowouts, and the bad teams lose a lot of blowouts. The mediocre have a little bit of both.
  • Lastly, (and this we didn’t discuss above, but it is widely understood) one-run games are a flip of the coin, both good and bad teams tend to play near .500 unless lady luck frowns on them.

    In other words, the Padres are a team predisposed towards losing blowouts, but their bullpen has instead received an unlikely large amount of Shutdown and Meltdown possibilities (in a blowout, the losing team’s reliever rarely can affect a win probability in a positive and significant manner).

    At the same time, the Rays, who are at .500 presently and appear to be an above-average team, have only a combined 15 SD and MDs, meaning their wins and losses have been blowouts or decided by their starters. Even when their team is in a three-run-or-less game, the Padres should tend to be on the losing end.

    Anyway, it is interesting stuff to watch as the season ambles forward.




  • Print This Post



    Bradley writes for FanGraphs and The Hardball Times. Follow him on Twitter @BradleyWoodrum.


    40 Responses to “Predicting Shutdowns and Saves”

    You can follow any responses to this entry through the RSS 2.0 feed.
    1. fantasystud1305 says:

      Great stuff. Really enjoyed it.

      Vote -1 Vote +1

    2. Tom says:

      Maybe I’m missing the point, but is this not all common sense? The reason for the frowns and smiles simply show that baseball outcomes (close games and blowouts) are evenly distributed. But the “winning” teams are the ones who come out with the…well, wins. The curves are simply due to the fact that bad teams lose games and good teams win them. Simple enough. Put even more simply, the Yankees have a much greater chance than the Royals to be on the winning side of a blowout.

      I think it’s better to look at winning % compared with save opportunities. I had looked at the relationship over a number of years and found a distinct, positive correlation (i.e. team like the Phillies have more save opps than the Pirates over time). All of this is common sense, however. There are exceptions of course. Soria “somehow” gets about 40 saves a year on a bad team because a) he is the exclusive closer who gets nearly all opps and b) he has a higher conversion rate than other closers on bad teams.

      Vote -1 Vote +1

      • An excellent point, Tom. I will admit my findings here are certainly not earth-shattering, but — all the same — it is an interesting survey of the “demographics” of wins. Also, it offers a good deal of illumination to Padres unlikely early season SDs.

        Could I have said all that with much less? Yeah, probably.

        Vote -1 Vote +1

    3. Big Van Vader says:

      “In the same stroke of logic, wouldn’t we expect the best teams to have fewer save opportunities?…Well, that may be logical, but it’s not entirely correct.”

      This claim, while somewhat popular, never seemed logical to me. I don’t know why so many people think a good team won’t play many many close games. The difference in runs scored per game for the best and worst teams in a league is never more than 3 runs. I would guess that it is rarely ever more than 2-runs. The only cases I could find since 2000 are the 2010 Mariners, 2003 Tigers, and 2003 Dodgers. These teams all averaged at least 2 fewer runs per game compared to the team that led their league in runs per game. But for the most part, especially in the NL, it seems that the biggest differences in runs per game are roughly 1.5 runs, and sometimes much closer to 1.0 runs per game.

      So the much more logical claim to me seems to be that on average we should expect a great team will win roughly 60% of the time and that margin of victory will be within three runs.

      Also, a 60% rate comes out to 97 wins. So that is a rather high estimate for where most good teams will finish. There are some years where no teams reach 97 wins. Many good teams will finish with win totals in the high 80′s and low to mid 90′s, and there is no reason to conclude that the closers of these teams will have fewer save opportunities than closers on terrible teams. In fact, that is backwards thinking. You should expect these closers to have more save opps than the closers on losing teams simply because their teams are more likely to win a given game, and you can’t get a save if your team doesn’t win.

      Vote -1 Vote +1

      • Interesting stuff, Vader.

        “You should expect these closers to have more save opps than the closers on losing teams simply because their teams are more likely to win a given game, and you can’t get a save if your team doesn’t win.”

        Yeah, even accounting for blowouts, we should expect “winning closers” to have more save opportunities.

        Vote -1 Vote +1

    4. Lewie Pollis says:

      This would be a fantastic study if you hadn’t misspelled G.O.B. Bluth’s name. In that typo, you “may have committed some light treason.”

      Vote -1 Vote +1

    5. Mike Savino says:

      I have this ugly feeling that PETCO might have some obnoxious factor in this. WPA isn’t park adjusted–it just looks at the average winning percentage of teams in a certain situation. If we were to look closer at teams in lower run-scoring environments, a reliever’s contribution might have a lower WPA and thus the Padres might have less shutdowns.

      Right? Logic: A pitcher pitching a scoreless inning in a 6 run environment is worth far more towards adding win probability than a pitcher pitching in a 3 run environment. PETCO might be a 3 run environment. (or whatever it is.)

      Also, the Padres have pretty good starters, good relievers and so far crappy run production. They’ve played 17 games decided by two runs or less–giving more opportunities for their relievers to play a larger role. They’re exacerbating PETCO with the roster they’ve created and that’s allowing run production to be even lower. Like I claimed above, I think PETCO has a serious affect on this statistic.

      Vote -1 Vote +1

      • Excellent observations. I haven’t really followed the Padres too closely this year, but I assumed there had to be some confluence of elements, such as PETCO, their starters, and such.

        Although, even when you do consider PETCO, you would think it wouldn’t matter if the deficit is what you might expect from a losing team. For instance, say a Padre reliever throws a quiet 7th inning, but the team is down 4-1. I’m not sure, but I suspect that would be around .002 WPA added.

        So, we should kind of expect the relievers are either coming in with the bases jammed, but the game still close, or putting up solid innings in a really low run environment.

        Vote -1 Vote +1

    6. jakesprecher says:

      Look at the R^2 in your second and third graphs… the values are both barely greater than .01. Unless my statistics knowledge has failed me, an R^2 of close to zero implies that there is almost zero correlation between the two variables.

      Vote -1 Vote +1

      • Yes. Correct. That’s kind of what I was trying to say when I said: “…frankly the correlation is not strong enough for any predictive power.”

        A .01 R-squared is essentially saying 1% of the variation in the regression is explained by the presented relationship. That’s kinda why I said: “…we can clearly see any relationship here is sketchy at best (the low R-squared confirms this).”

        …Did you just look at my pretty pictures and then leave a comment? It’s okay if you did — I do the same thing all the time.

        Vote -1 Vote +1

        • D4P says:

          Try adding a negative winning percentage squared variable to the regression model for Figure 2 and see what happens.

          Vote -1 Vote +1

    7. mattinm says:

      Forgive me if my knowledge of the Shutdown statistic (well, more correctly WPA) is flawed, but where was this particular nugget proven — “(in a blowout, the losing team’s reliever rarely can affect a win probability in a positive and significant manner)”?

      The title lead me to believe there would be some statistical method for predicting both Shutdowns and Saves; however, it seems the entire focus was on Saves and blowouts. Yes, it does make sense that the ability of a relief corps to affect games is lowered within Blowouts (for which you showed poor teams are overwhelmingly on the negative side). However, it also made sense, as declared by you in the article, that better teams would be involved in fewer close games (and therefore fewer save opps), which you convincingly disproved.

      I guess, in short, I’m still left wondering the correlation between Shutdown, Meltdown, and winning percentage (i.e. “goodness” of team).

      Vote -1 Vote +1

      • “…where was this particular nugget proven — ‘(in a blowout, the losing team’s reliever rarely can affect a win probability in a positive and significant manner)’?”

        Well, if it’s proven somewhere, it’s not here. :)

        It’s more of a logical claim than anything. If the WE for the Royals if .100 at the end of 5th inning, a reliever putting up a silent 6th inning for the Royals will do almost nothing for the Royals WE.

        Consider this game here:

        http://www.fangraphs.com/livewins.aspx?date=2011-04-27&team=Diamondbacks&dh=0&season=2011

        Collmenter had a perfect inning for the Diamondbacks, but they were losing big at that time, so he added only 0.010 WPA. At that point in the game, the only people capable of adding serious WPA for the Diamondbacks would be the hitters.

        Hopes this clears it up a little.

        Vote -1 Vote +1

    8. MikeS says:

      Great article. Love the scatterplots. they really help visualize this.

      Small typo – the Yankees are probably forming a conga line, not a congo line. Unless you meant they are hitting the ball to Africa or losing it in the jungle or They hit so well the other team might as well just read Michael Crichton books or something.

      Vote -1 Vote +1

    9. MatManz says:

      Best part about saves is that you simply CAN’T predict them. That’s what makes baseball so great, the UNpredictability.

      Vote -1 Vote +1

    10. ang says:

      You really should have plotted “win % in blowouts” vs “win % EX-blowouts”, since “win %” is the sum of these two components, so the correlation you find really tells us nothing. Similarly for “win % 3-runs or less” .

      Vote -1 Vote +1

      • Good points Ang.

        However, I do disagree with the assertion my plots tell us “nothing.” I’m really looking to see the source of winning teams’ winningness. It may seem like the winning teams would naturally have a positive correlation with blowout games, but consider the 1-Run Games chart (hyperlinked in the bulleted section). Here we see no relationship with W% and one of its inside components.

        Vote -1 Vote +1

    11. What did you use to make the graphs? Some things look like Excel, but if so you did a really nice job of making it NOT look like Excel.

      Vote -1 Vote +1

    12. DominicanRepublican says:

      I would venture that teams with lower total runs (both for and against) should have the most save opportunities. By definition, in most cases a lower-scoring team will have a smaller standard deviation of wins and losses, leading to more 3-run games. This is why you should continue to expect San Diego to continue to get high leverage reliever situations. They score little and, especially in their home park, they allow few runs.

      Teams like the Yankees score a lot of runs and their pitchers allow a lot of runs, meaning you should expect fewer high leverage situations for their relievers (they’ll win and lose a lot of blowouts because of the high-standard deviation offense/pitching they have).

      Vote -1 Vote +1

      • This is an excellent suggestion, but — and, who knows, it’s early in the season — how do we reconcile the Rays with this conundrum? Their offense is anything but prolific or fearsome; their home park depresses power; their starting pitchers keeps runs to a minimum; but their bullpen has had nary a chance to shine or stink.

        I dunno, maybe that will change as the season goes on.

        Vote -1 Vote +1

        • DominicanRepublican says:

          I didn’t mean to suggest that this trend should show itself in this small sample size about 15% of the way through the season, but I’d expect it will be true at the end of this season and has been true in past seasons.

          Vote -1 Vote +1

    13. Joltin' Joe says:

      So the moral of the story is to avoid 1-run games if you’re good and try for them if you suck?

      Vote -1 Vote +1

    14. Joakim says:

      yeah but…didn’t Royals lead the AL in save opps last year?

      What say you now?

      Vote -1 Vote +1

      • Did they? Hmm… I would have to think it’s either (1) random variation or (2) symbolic of Datyon Moore’s false belief that he’s built an adequate team, so he goes out and gets a quality bullpen that keeps games closer than they normally would be.

        I’m not sure though. This is just me theorizing.

        Vote -1 Vote +1

      • DominicanRepublican says:

        It’s the low runs scored. They had a bad offense last year. That’s also why the all-time record holder in saves was a member of the Angels – they were a pitching-heavy team without a dominate offense. This results in a low standard deviation in runs scored which makes for more save opps.

        Vote -1 Vote +1

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

    Current ye@r *