Fun with Shutdowns and Meltdowns

Yesterday David introduced a couple of new stats, Shutdowns and Meltdowns, to the site.  It’s fashioned after saves/blown saves but is vastly superior, because it’s a metric that uses WPA as a substitute of the brainless, archaic save stat and the rules that guide it.

A team essentially has something like a 98% likelihood of winning the game with a three-run lead with no outs, yet a manager will trot out his ace reliever in that situation about 98% of the time for the sake of save. But when the game is on the line and it’s non-save situation, we often see managers make some of the most bizarre choices in their bullpen usage.

Take for instance Tuesday night’s Phillies–Cardinals game that ended in the 10th on a walk-off homer by Carlos Ruiz. While there’s no real “ace” in the Cardinal bullpen, the inexperienced Blake Hawksworth isn’t the guy you normally would want on the mound against the Phillies in such a high leverage situation, but it appears Tony La Russa held back his closer because it wasn’t a save situation.

Anyway, with any luck this catches on. Just to recap:

Shutdown is when a reliever accumulates greater than or equal to 0.06 WPA in any individual game.

Meltdown is when a reliever’s WPA is less than or equal to -0.06 in any individual game.

What I thought would be interesting is to look at the “Meltdowniest” pitchers of the past three seasons, as well as the ones who we could say have ice water in their veins. The pitchers with the most meltdowns are usually the ones fans want to ride out of town on a rail, along with their manager, while the pitchers with the smallest meltdown rate we tend to feel pretty comfortable with, even in the highest leverage situations.

These are the pitchers with the highest percentage of relief appearances that resulted in a Meltdown:

Name G SD MD Meltdown%
Aaron Heilman 229 55 52 22.7%
Luis Ayala 163 33 37 22.7%
Shawn Camp 149 36 33 22.1%
Brian Bass 93 18 20 21.5%
Juan Rincon 143 27 30 21.0%
Scott Linebrink 178 61 37 20.8%
Sean Green 215 52 43 20.0%
Mark Lowe 136 47 27 19.9%
Zach Miner 111 39 22 19.8%
Cla Meredith 217 45 43 19.8%

Aaron Heilman is our King of Catastrophe. Cub fans are glad to see him take his act elsewhere. That four-year, $19 million deal given to Scott Linebrink was given by the White Sox is looking like one the more awful contracts given to a middle reliever in recent memory.

So what pitchers have ice water in their veins, or in other words, the pitchers who have experienced the fewest rate of blowups per appearance? I excluded anyone who didn’t average at least an average pLI of 1.2.

Name G SD MD SD% Meltdown%
Joe Nathan 206 104 18 50.5% 8.7%
Jonathan Papelbon 192 85 17 44.3% 8.9%
Joakim Soria 172 86 16 50.0% 9.3%
Mariano Rivera 197 104 20 52.8% 10.2%
Takashi Saito 164 57 17 34.8% 10.4%
Jose Valverde 191 89 21 46.6% 11.0%
Francisco Cordero 206 101 23 49.0% 11.2%
David Aardsma 145 63 17 43.4% 11.7%
Rafael Soriano 162 63 19 38.9% 11.7%
Bobby Jenks 175 76 21 43.4% 12.0%

No major surprises here, especially the from the top four.

What about the highest Meltdown % by pitchers with 30 or more saves the past three seasons?

Name G SD MD SD% Meltdown%
Chad Qualls 207 88 36 42.5% 17.4%
David Weathers 210 73 36 34.8% 17.1%
Kevin Gregg 218 89 35 40.8% 16.1%
Brian Wilson 155 84 24 54.2% 15.5%
C.J. Wilson 190 74 29 38.9% 15.3%
Fernando Rodney 159 66 24 41.5% 15.1%
Matt Capps 182 75 26 41.2% 14.3%
Brad Lidge 205 83 29 40.5% 14.1%
Brandon Lyon 199 78 28 39.2% 14.1%
Trevor Hoffman 164 86 23 52.4% 14.0%

The unpopular trio of Chad Qualls, Kevin Gregg and David “Stormy” Weathers lead the list. We also see the Phillies favorite bipolar pitcher, Brad Lidge make this list. Interestingly enough, Brian Wilson had the highest percentage of his outings end in either a Shutdown or a Meltdown of any reliever.

Again, I hope this catches on. Shutdowns and Meltdowns are a more practical and intuitive way at evaluating reliever performance, and I think a more fun one, too.




Print This Post



Erik Manning is the founder of Future Redbirds and covers the Cardinals for Heater Magazine. You can get more of his analysis and rantings in bite-sized bits by following him on twitter.


37 Responses to “Fun with Shutdowns and Meltdowns”

You can follow any responses to this entry through the RSS 2.0 feed.
  1. nothingxs says:

    This is actually kind of interesting. I sort of expected Kevin Gregg to show up in your third list, but I’m actually kind of surprised to see Trevor Hoffman in there too. What about a SD to MD ratio? Maybe it’s not just important to see how many times in general someone “shuts down”, but it’s also important to see what the ratio of great games to bad games is.

    Like I suggested elsewhere, though, I’d like to scale WPA to outs to be able to better estimate the impact of the usage of lesser relievers in smaller situations. How much does a LOOGY contribute to the bullpen when used specifically in those situations? Perhaps a secondary statistic based on SD can be devised that measures the effectiveness of relievers when used in lower-out situations, or SD can be expanded.

    So really, why not scale the amount of WPA added to the amount of outs the pitcher gets? If you get a positive WPA in just one out, that means you’re doing your job as a LOOGY, right? You can set the tolerance for this stat pretty low because it wants to measure minute changes.

    We know that the +/- .06 WPA is essentially over three outs. What about +/- .04 WPA for a 2 out situation and +/- .02 WPA for a 1 out situation? This is obviously just throwing an idea out there, because I’m not (yet) a statistician.

    Vote -1 Vote +1

    • Boots says:

      A LOOGY should be able to still get a shutdown if he comes in for one or two outs in certain situations, especially if it’s a close game and there are runners on base. The WPA changes enough in close games in the later innings that if the LOOGY comes in, gets one or two outs, he should still be able to get a shutdown even though he didn’t pitch three outs.

      But if a LOOGY comes in to get a single out earlier in the game when the score isn’t close, Im not sure that reliever has earned a shutdown, as I’m not sure how much that really helped the team get closer to winning

      Vote -1 Vote +1

    • Boots says:

      However, I do think the SD/MD ratio could be very useful.

      Vote -1 Vote +1

      • nothingxs says:

        It might be interesting because it would be good to quickly rate the consistency — and value — of people with decent shutdown percentages at a glance.

        Case in point: In the past three seasons, Kevin Gregg has an SD/MD of 2.54, Joe Nathan has an SD/MD of 5.78 and Trevor Hoffman has an SD/MD of 3.74. You already knew Nathan was better than Kevin Gregg anyway, but it’s nice to have a quick metric that tells you what’s up. You can also obtain an average league SD/MD and quickly determine who is above or below average in this aspect.

        Vote -1 Vote +1

      • Erik Manning says:

        Honestly, I originally thought of looking at it this way, but for some reason or another I settled the percentages of their games that were either shutdowns/meltdowns or neutral. Looking back, I think I’d just do the SD/MD for all the reasons stated. Maybe for another post I can span SD/MD ratios over the decades, see who was the king of each categories for aughts, 90′s, 80′s…

        Vote -1 Vote +1

      • Erik Manning says:

        Of course, I listed the SD/MDs in the tables so you can still eyeball it.

        Vote -1 Vote +1

      • nothingxs says:

        Erik, it’s not like you can’t have both. It’s not a bad idea to be able to see how many games a reliever came into in which they didn’t really wildly affect the outcome of the game, so maybe having SD, MD, NE (neutral effect) and SD/MD could be the four things we’re looking to determine.

        Vote -1 Vote +1

      • Boots says:

        I agree having both would be good to see, as the SD% and MD% show how often the pitcher is used during important situations in the game. The SD/MD could be a better indication of performance overall.

        Vote -1 Vote +1

    • tangotiger says:

      The win value of an out, when LI = 1, is +.027 wins. So, if you can get two outs when the LI is 1.2, you get your shutdown. To get it for one out, you need the LI to be 2.3.

      So, definitely possible to be a LOOGY and earn a shutdown under the current construction.

      Vote -1 Vote +1

  2. My Grate Friend, Peason says:

    Man, those graphs are long! ;)

    Vote -1 Vote +1

  3. rmtx97 says:

    Wow, just awesome work FG!

    Vote -1 Vote +1

    • BDanahy14 says:

      Pretty sure it was Tango and some commentators over at his site that put the wrapping paper on this one. The conversation on naming the stat was pretty great – like trying to figure out what to call a new dog

      Vote -1 Vote +1

  4. AK707 says:

    The Wilson thing might be that he can’t get a save without putting a couple guys on first. This makes for some super-high leverage strikeouts with runners on in the ninth (shutdown), and when it goes south, he gets a meltdown.

    Vote -1 Vote +1

    • tangotiger says:

      The WPA will come out to be the same thing if you start the ninth with a 2 run lead and allow no runs, whether it’s 1-2-3, or you load the bases first and then get 1-2-3.

      Vote -1 Vote +1

      • Boots says:

        To expand, this is because it is the NET change in WPA. If he allows some baserunners in the inning, his WPA will be negative at that time, but if he gets through the inning without anyone scoring his WPA will always be positive. It only matters in someone comes in during the inning to replace him.

        Vote -1 Vote +1

  5. Adam D says:

    after being at last night’s Rangers/Royals game, my gut is very familiar with the Meltdown statistic. see Nippert, Dustin; Ray, Chris; and Francisco, Frankie

    of course, Mr. Soria wasn’t so hot last night either :)

    Vote -1 Vote +1

  6. cje says:

    Regarding the Hawksworth example, I think LaRussa left him in because memories of the twenty inning Mets game were still fresh, and he didn’t want to burn through relievers if they were still pitching well, as Hawksworth had in the 9th.

    Vote -1 Vote +1

  7. wpe says:

    I’m new to the game so I apologize if this is already out there, but is there a stat that improves upon the “quality start” like this does for saves?

    Vote -1 Vote +1

    • SF 55 for life says:

      they have something called game score but isn’t used here.

      Vote -1 Vote +1

    • tangotiger says:

      I’m working on it…

      Vote -1 Vote +1

      • vtsoxfan says:

        It would be nice to have both descriptive and predictive evaluations for starting pitchers. Descriptive evaluations–trying to record how they actually did– would do something with WPA or WPA/LI. Predictive evaluations–looking at how pitchers “should” have done–might combine FIP with Innings Pitched. If Justin Verlander has a 12 K, 1 BB, 1 HR, 120-pitch, 6-inning start, and Roy Halladay has a 7 K, 1 BB, 1 HR, 120-pitch, 8-inning start, Verlander has a better FIP and xFIP than Halladay (I think, right?). But Halladay has had a better start, because you should credit Verlander with 2 IP of average relief work that Halladay avoided. Now, that may end up judging pitchers somewhat on their manager’s decisions, but it’s better than throwing our hands up and saying, “We can’t evaluate this!”

        Vote -1 Vote +1

  8. scott says:

    If the WE actually decreases even though the pitcher got an out, who is the WPA increase/decrease actually attributed to? If its negative to the pitcher I think the stastic can be unfair to a 7th inning reliever where the win probabilities can actually decrease even with an increase in outs

    For example, the home starter leaves a mess with the home team up by 1 in the bottom of the 7th with no outs and men on first and second. Reliever comes in and the WE prob is .885 at that point. He K’s the next guy for the 1st out: WE=.859. Next guy pops out: WE .809. Lets assume the manager puts in a loogy to face the final hitter of the inning. The 7th inning guy just got a blowup @ -.076 for shutting down 2 hitters in a row.

    Vote -1 Vote +1

    • tangotiger says:

      You are highly misinformed. In no way does your example make sense. You are flipping the signs, or looking at the WE from the batting team’s perspective, or something.

      If you make an out, and no runner advances a base, the batting team ALWAYS (without exception) reduces its chances of winning and the fielding teams ALWAYS increases its chances of winning.

      Vote -1 Vote +1

      • scott says:

        I figured out what I was doing wrong – I was looking at the bottom of the 7th from the perspective of the home team – obviously the home team is hitting – not pitching! I knew something was severely odd. Thanks!

        Vote -1 Vote +1

  9. Franco says:

    The fact that Heilman is ranked #1 on Meltdowns confirms that this is the most accurate reliever stat of all time. Even with his solid traditional stats in 07, you could hear the collective groans of 50k fans when he came into close games.

    Vote -1 Vote +1

  10. payday0023 says:

    I love the Shutodwn/Metldown stat. Love the name too.
    Just thinking out loud…
    -Can this be translated or applied into a predictive nature?
    -How dependent would this be on the management (or mis-management) of the bull pen or is that an unanswerable question?
    -Does x number of Shutdowns/Meltdowns correlate to a pitchers WAR?

    Vote -1 Vote +1

  11. Franco says:

    Rivera’s postseason Shutdown/Meltdown ratio has to be absurd.

    Vote -1 Vote +1

  12. Andrew Martinez says:

    I used Shutdowns and Meltdowns to come up with a stat called ROL (short for Rolaids because Rolaids spells relief). The formula is (IP*SD*gmLI)/(GM*MD*xFIP), where IP is innings, SD is Shutdowns, gmLI is leverage index entering a game, GM is games, MD is Meltdowns, and xFIP is expected field independent pitching. I ran it for 2009. You can check it out if you want… http://www.themartinezreport.com/2010/05/rolaids-spells-relief-2009.html I have no idea if it has any merit.

    Vote -1 Vote +1

  13. Shawn says:

    Erik, I think this is a great stat. This is definitely one i want to see catch on.

    Vote -1 Vote +1

  14. Dan says:

    Brian Wilson probably has the highest shutdown/meltdown frequency because he’s completely insane and his pitching reflects whichever bipolar mood he’s in that day.

    Vote -1 Vote +1

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current ye@r *