Fun with Shutdowns and Meltdowns

Yesterday David introduced a couple of new stats, Shutdowns and Meltdowns, to the site.  It’s fashioned after saves/blown saves but is vastly superior, because it’s a metric that uses WPA as a substitute of the brainless, archaic save stat and the rules that guide it.

A team essentially has something like a 98% likelihood of winning the game with a three-run lead with no outs, yet a manager will trot out his ace reliever in that situation about 98% of the time for the sake of save. But when the game is on the line and it’s non-save situation, we often see managers make some of the most bizarre choices in their bullpen usage.

Take for instance Tuesday night’s Phillies–Cardinals game that ended in the 10th on a walk-off homer by Carlos Ruiz. While there’s no real “ace” in the Cardinal bullpen, the inexperienced Blake Hawksworth isn’t the guy you normally would want on the mound against the Phillies in such a high leverage situation, but it appears Tony La Russa held back his closer because it wasn’t a save situation.

Anyway, with any luck this catches on. Just to recap:

Shutdown is when a reliever accumulates greater than or equal to 0.06 WPA in any individual game.

Meltdown is when a reliever’s WPA is less than or equal to -0.06 in any individual game.

What I thought would be interesting is to look at the “Meltdowniest” pitchers of the past three seasons, as well as the ones who we could say have ice water in their veins. The pitchers with the most meltdowns are usually the ones fans want to ride out of town on a rail, along with their manager, while the pitchers with the smallest meltdown rate we tend to feel pretty comfortable with, even in the highest leverage situations.

These are the pitchers with the highest percentage of relief appearances that resulted in a Meltdown:

Name G SD MD Meltdown%
Aaron Heilman 229 55 52 22.7%
Luis Ayala 163 33 37 22.7%
Shawn Camp 149 36 33 22.1%
Brian Bass 93 18 20 21.5%
Juan Rincon 143 27 30 21.0%
Scott Linebrink 178 61 37 20.8%
Sean Green 215 52 43 20.0%
Mark Lowe 136 47 27 19.9%
Zach Miner 111 39 22 19.8%
Cla Meredith 217 45 43 19.8%

Aaron Heilman is our King of Catastrophe. Cub fans are glad to see him take his act elsewhere. That four-year, $19 million deal given to Scott Linebrink was given by the White Sox is looking like one the more awful contracts given to a middle reliever in recent memory.

So what pitchers have ice water in their veins, or in other words, the pitchers who have experienced the fewest rate of blowups per appearance? I excluded anyone who didn’t average at least an average pLI of 1.2.

Name G SD MD SD% Meltdown%
Joe Nathan 206 104 18 50.5% 8.7%
Jonathan Papelbon 192 85 17 44.3% 8.9%
Joakim Soria 172 86 16 50.0% 9.3%
Mariano Rivera 197 104 20 52.8% 10.2%
Takashi Saito 164 57 17 34.8% 10.4%
Jose Valverde 191 89 21 46.6% 11.0%
Francisco Cordero 206 101 23 49.0% 11.2%
David Aardsma 145 63 17 43.4% 11.7%
Rafael Soriano 162 63 19 38.9% 11.7%
Bobby Jenks 175 76 21 43.4% 12.0%

No major surprises here, especially the from the top four.

What about the highest Meltdown % by pitchers with 30 or more saves the past three seasons?

Name G SD MD SD% Meltdown%
Chad Qualls 207 88 36 42.5% 17.4%
David Weathers 210 73 36 34.8% 17.1%
Kevin Gregg 218 89 35 40.8% 16.1%
Brian Wilson 155 84 24 54.2% 15.5%
C.J. Wilson 190 74 29 38.9% 15.3%
Fernando Rodney 159 66 24 41.5% 15.1%
Matt Capps 182 75 26 41.2% 14.3%
Brad Lidge 205 83 29 40.5% 14.1%
Brandon Lyon 199 78 28 39.2% 14.1%
Trevor Hoffman 164 86 23 52.4% 14.0%

The unpopular trio of Chad Qualls, Kevin Gregg and David “Stormy” Weathers lead the list. We also see the Phillies favorite bipolar pitcher, Brad Lidge make this list. Interestingly enough, Brian Wilson had the highest percentage of his outings end in either a Shutdown or a Meltdown of any reliever.

Again, I hope this catches on. Shutdowns and Meltdowns are a more practical and intuitive way at evaluating reliever performance, and I think a more fun one, too.

We hoped you liked reading Fun with Shutdowns and Meltdowns by Erik Manning!

Please support FanGraphs by becoming a member. We publish thousands of articles a year, host multiple podcasts, and have an ever growing database of baseball stats.

FanGraphs does not have a paywall. With your membership, we can continue to offer the content you've come to rely on and add to our unique baseball coverage.

Support FanGraphs




Erik Manning is the founder of Future Redbirds and covers the Cardinals for Heater Magazine. You can get more of his analysis and rantings in bite-sized bits by following him on twitter.

newest oldest most voted
nothingxs
Guest

This is actually kind of interesting. I sort of expected Kevin Gregg to show up in your third list, but I’m actually kind of surprised to see Trevor Hoffman in there too. What about a SD to MD ratio? Maybe it’s not just important to see how many times in general someone “shuts down”, but it’s also important to see what the ratio of great games to bad games is.

Like I suggested elsewhere, though, I’d like to scale WPA to outs to be able to better estimate the impact of the usage of lesser relievers in smaller situations. How much does a LOOGY contribute to the bullpen when used specifically in those situations? Perhaps a secondary statistic based on SD can be devised that measures the effectiveness of relievers when used in lower-out situations, or SD can be expanded.

So really, why not scale the amount of WPA added to the amount of outs the pitcher gets? If you get a positive WPA in just one out, that means you’re doing your job as a LOOGY, right? You can set the tolerance for this stat pretty low because it wants to measure minute changes.

We know that the +/- .06 WPA is essentially over three outs. What about +/- .04 WPA for a 2 out situation and +/- .02 WPA for a 1 out situation? This is obviously just throwing an idea out there, because I’m not (yet) a statistician.

Boots
Guest
Boots

A LOOGY should be able to still get a shutdown if he comes in for one or two outs in certain situations, especially if it’s a close game and there are runners on base. The WPA changes enough in close games in the later innings that if the LOOGY comes in, gets one or two outs, he should still be able to get a shutdown even though he didn’t pitch three outs.

But if a LOOGY comes in to get a single out earlier in the game when the score isn’t close, Im not sure that reliever has earned a shutdown, as I’m not sure how much that really helped the team get closer to winning

Steve Sommer
Guest

Yeah, take for example the Cards. Their LOOGYs are 2nd and 3rd in SDs because they have a decent amount of appearances in high lev situations

Boots
Guest
Boots

However, I do think the SD/MD ratio could be very useful.

nothingxs
Guest

It might be interesting because it would be good to quickly rate the consistency — and value — of people with decent shutdown percentages at a glance.

Case in point: In the past three seasons, Kevin Gregg has an SD/MD of 2.54, Joe Nathan has an SD/MD of 5.78 and Trevor Hoffman has an SD/MD of 3.74. You already knew Nathan was better than Kevin Gregg anyway, but it’s nice to have a quick metric that tells you what’s up. You can also obtain an average league SD/MD and quickly determine who is above or below average in this aspect.

Erik Manning
Guest

Honestly, I originally thought of looking at it this way, but for some reason or another I settled the percentages of their games that were either shutdowns/meltdowns or neutral. Looking back, I think I’d just do the SD/MD for all the reasons stated. Maybe for another post I can span SD/MD ratios over the decades, see who was the king of each categories for aughts, 90’s, 80’s…

Erik Manning
Guest

Of course, I listed the SD/MDs in the tables so you can still eyeball it.

nothingxs
Guest

Erik, it’s not like you can’t have both. It’s not a bad idea to be able to see how many games a reliever came into in which they didn’t really wildly affect the outcome of the game, so maybe having SD, MD, NE (neutral effect) and SD/MD could be the four things we’re looking to determine.

Boots
Guest
Boots

I agree having both would be good to see, as the SD% and MD% show how often the pitcher is used during important situations in the game. The SD/MD could be a better indication of performance overall.

tangotiger
Guest
tangotiger

The win value of an out, when LI = 1, is +.027 wins. So, if you can get two outs when the LI is 1.2, you get your shutdown. To get it for one out, you need the LI to be 2.3.

So, definitely possible to be a LOOGY and earn a shutdown under the current construction.

nothingxs
Guest

So in other words, Renyel Pinto’s SD/MD must be like .1 or something