# Using WPA to grade bullpen management, part two

In my last article, I outlined a method of analyzing bullpens by making a small tweak to the inside of the WPA Clutch metric. To quickly recap, Clutch gives the difference between how a player performed and how that player would have performed in a luck-neutral context-independent environment. Clutch was designed for batters who cannot choose when and where to bat, so the first term has a pLI denominator that puts every batter on an equal footing regarding the number of high leverage plate appearances they participated in.

But since we’re measuring relievers, who don’t get equal amounts of leverage by design, the pLI denominator excises some pretty useful information. By stripping away the pLI denominator from the first term, the Clutch metric changes what the stat is measuring. By taking it out, we arrive at a number that is a sum of how a player performed in the clutch added to how efficiently a manager deployed his relievers. To put it in mathematical terms:

If we want to remove player effects and isolate the manager’s contribution, all it takes is some simple algebra.

Which finally leaves us with this.

It’s really quite simple. WPA gives a total picture of a player’s contributions to a team’s win. Subtracting context-independent WPA/LI leaves behind only the effects from the timing of game events. Subtracting away Clutch from that total pares WPA down even further, so all that remains is how a manager helped or hurt his team with his bullpen management decisions.

And now, of course, the fun part. Here’s 2012’s leaderboard.

Team | Manager WPA |
---|---|

Orioles | 5.06 |

Athletics | 4.73 |

Cubs | 4.51 |

Angels | 4.09 |

Padres | 4.08 |

Blue Jays | 3.45 |

Giants | 3.06 |

Braves | 2.65 |

Mets | 2.43 |

Royals | 2.25 |

Cardinals | 2.24 |

Mariners | 1.90 |

Reds | 1.75 |

Rangers | 1.53 |

Rockies | 1.52 |

Marlins | 1.50 |

Rays | 1.49 |

Yankees | 1.46 |

Twins | 1.42 |

Tigers | 1.20 |

White Sox | 1.17 |

Pirates | 1.04 |

Astros | 0.80 |

Indians | 0.54 |

Diamondbacks | 0.42 |

Phillies | 0.37 |

Brewers | -0.41 |

Dodgers | -0.46 |

Red Sox | -0.73 |

Nationals | -1.01 |

I find this absolutely fascinating. Two teams that surprised everyone by making the playoffs are ranked one and two, with bullpen management responsible for five wins over the course of the season. Keep in mind that this doesn’t mean that Buck Showalter and Bob Melvin were five wins better than the average manager. Zero wins would mean that a manager’s bullpen management was no better than drawing names out of a hat, and as you might imagine, most managers are better than that. The average, as counted from the beginning of WPA data on Fangraphs, is a hair shy of two wins.

So what does it all mean? Managers who use their bullpens efficiently generally add somewhere around two or three wins to a team’s season total, as compared to an average two-win manager. By the same coin, managers with inefficient bullpen management can cost their teams a win or two by using the wrong pitchers in critical situations.

**References & Resources**

All data from Fangraphs (1974-2012).

My full data set is here. It contains WPA, Clutch and Manager figures for every team from 1974 to the present.

Print This Post

I’m genuinely surprised by the evenness of the results, especially at either end of the curve. I’d always believed that a bad manager could cost his team more wins than a good manager could add, but it looks as though (at least in terms of bullpen management) that’s not really the case.

I am guessing, but I would assume that Farrell used them in low leverage situations (ie: blowouts) where their lack of talent would not effect the game as much. The pLI of these players other than Janssen appear very low. Further, while he didnt have to use them at all, he saves his other bullpen arms for closer more meaningful situations.

Right Darren, but who you use in a blowout doesn’t matter. There is no win impact at all from who you use in a blowout going either way (although guys like Pauley and Carpenter are bad enough to make any lead unsafe).

LOL Don Mattingly and Davey Johnson!

Oh, and LOL Bobby Valentine too!

Do the same managers do well every year?

If so, this is exciting.

If not it does paint a picture for a season as RBIs do, but it may not be useful in evaluating managers.

I have trouble assigning these scores to the managers because over the course of the season anagers use Prior information about each pitcher, collect New Information, and combine the prior and new information to use in the next game. The list appears to give high or low scores to teams that were presented with lots of New Information (A’s, Nationals), and near zero scores for teams that were presented with less New Information and just stuck with a formula that worked (Rays, Yankees). I don’t think that managers should be penalized for sticking with a formula that works.

I would love to see whether the results for managers showed any consistency over time.