Using WPA to grade bullpen management, part two

In my last article, I outlined a method of analyzing bullpens by making a small tweak to the inside of the WPA Clutch metric. To quickly recap, Clutch gives the difference between how a player performed and how that player would have performed in a luck-neutral context-independent environment. Clutch was designed for batters who cannot choose when and where to bat, so the first term has a pLI denominator that puts every batter on an equal footing regarding the number of high leverage plate appearances they participated in.

But since we’re measuring relievers, who don’t get equal amounts of leverage by design, the pLI denominator excises some pretty useful information. By stripping away the pLI denominator from the first term, the Clutch metric changes what the stat is measuring. By taking it out, we arrive at a number that is a sum of how a player performed in the clutch added to how efficiently a manager deployed his relievers. To put it in mathematical terms:

image

If we want to remove player effects and isolate the manager’s contribution, all it takes is some simple algebra.

image

Which finally leaves us with this.

image

It’s really quite simple. WPA gives a total picture of a player’s contributions to a team’s win. Subtracting context-independent WPA/LI leaves behind only the effects from the timing of game events. Subtracting away Clutch from that total pares WPA down even further, so all that remains is how a manager helped or hurt his team with his bullpen management decisions.

And now, of course, the fun part. Here’s 2012’s leaderboard.

Tebow or Not Tebow, a Visualization
When it comes to the Mets' famous minor leaguer, it's not just will he get major league time, but should he.

Team Manager WPA
Orioles
5.06
Athletics 4.73
Cubs 4.51
Angels 4.09
Padres 4.08
Blue Jays 3.45
Giants 3.06
Braves 2.65
Mets 2.43
Royals 2.25
Cardinals 2.24
Mariners 1.90
Reds 1.75
Rangers 1.53
Rockies 1.52
Marlins 1.50
Rays 1.49
Yankees 1.46
Twins 1.42
Tigers 1.20
White Sox 1.17
Pirates 1.04
Astros 0.80
Indians 0.54
Diamondbacks 0.42
Phillies 0.37
Brewers -0.41
Dodgers -0.46
Red Sox -0.73
Nationals -1.01

I find this absolutely fascinating. Two teams that surprised everyone by making the playoffs are ranked one and two, with bullpen management responsible for five wins over the course of the season. Keep in mind that this doesn’t mean that Buck Showalter and Bob Melvin were five wins better than the average manager. Zero wins would mean that a manager’s bullpen management was no better than drawing names out of a hat, and as you might imagine, most managers are better than that. The average, as counted from the beginning of WPA data on Fangraphs, is a hair shy of two wins.

image

So what does it all mean? Managers who use their bullpens efficiently generally add somewhere around two or three wins to a team’s season total, as compared to an average two-win manager. By the same coin, managers with inefficient bullpen management can cost their teams a win or two by using the wrong pitchers in critical situations.

References & Resources
All data from Fangraphs (1974-2012).

My full data set is here. It contains WPA, Clutch and Manager figures for every team from 1974 to the present.


Print This Post
Sort by:   newest | oldest | most voted
Ian R.
Guest
Ian R.

I’m genuinely surprised by the evenness of the results, especially at either end of the curve. I’d always believed that a bad manager could cost his team more wins than a good manager could add, but it looks as though (at least in terms of bullpen management) that’s not really the case.

Rob
Guest
Rob
Interesting idea, but I think small sample size quirks are skewing the results. Take the Blue Jays, for example. Applying the WPA – WPA/pLI formula to each individual pitcher gives us this top five: 0.72   David Carpenter 0.66   David Pauley 0.50   Casey Janssen 0.34   Darren Oliver 0.32   Bobby Korecky So John Farrell, at +3.45 overall, somehow got half of that from Carpenter, Pauley, and Korecky, who together pitched ten innings and weren’t any good when doing so? Also, 8th on the list is Ryota Igarashi (+0.25), which is just a cruel joke for anyone who… Read more »
Darren
Guest
Darren

I am guessing, but I would assume that Farrell used them in low leverage situations (ie: blowouts) where their lack of talent would not effect the game as much. The pLI of these players other than Janssen appear very low. Further, while he didnt have to use them at all, he saves his other bullpen arms for closer more meaningful situations.

Craig Burley
Guest
Craig Burley

Right Darren, but who you use in a blowout doesn’t matter. There is no win impact at all from who you use in a blowout going either way (although guys like Pauley and Carpenter are bad enough to make any lead unsafe).

DrBGiantsfan
Guest
DrBGiantsfan

LOL Don Mattingly and Davey Johnson!

Obsessivegiantscompulsive
Guest
Obsessivegiantscompulsive
Really nice analysis if the results hold up re the complaint about the individual results.  Re Donny, he had avg in 2011, DrB.  But yeah, lol on 2012. Wow, Bochy shows up really nicely in the data.  First year with Pads and Giants, he had a very low win, but for Pads afterward, he was much above average, and over 8 in one season.  He has not done great wiith the Giants, but he has at least been above avg, over two, other than his first season.  I will need to add it up, but he has been above average… Read more »
DrBGiantsfan
Guest
DrBGiantsfan

Oh, and LOL Bobby Valentine too!

Jorgenswest
Guest
Jorgenswest

Do the same managers do well every year?

If so, this is exciting.

If not it does paint a picture for a season as RBIs do, but it may not be useful in evaluating managers.

JKB
Guest
JKB

I have trouble assigning these scores to the managers because over the course of the season anagers use Prior information about each pitcher, collect New Information, and combine the prior and new information to use in the next game. The list appears to give high or low scores to teams that were presented with lots of New Information (A’s, Nationals), and near zero scores for teams that were presented with less New Information and just stuck with a formula that worked (Rays, Yankees). I don’t think that managers should be penalized for sticking with a formula that works.

studes
Guest
studes
Dan, this is really interesting. What you’re capturing however, is how well the pitcher pitched to the leverage of the situation.  You could claim that that is the manager’s job, but it seems to me you could more easily claim that that was the pitcher’s job. In other words, haven’t you really just designed a different way of defining clutch?  Why assign the difference between this and the other definition of clutch to the manager? Jorgenswest points to the next level of analysis, I think. If there is some predictability from year to year, particularly across different bullpens, than you’re… Read more »
Jonathan Hale
Guest
Jonathan Hale
“So John Farrell, at +3.45 overall, somehow got half of that from Carpenter, Pauley, and Korecky, who together pitched ten innings and weren’t any good when doing so?” I think this is because using this technique, credit is given for good pitchers pitching in high leverage situations as well as vice versa (since – WPA/LI will be larger when LI is less than 0). Also, when LI is approaching zero (as for Carpenter) then the values will get really large. Doesn’t it make also sense that one more side of the coin is much more important? I don’t think it… Read more »
Pizza Cutter
Guest
Pizza Cutter

I would love to see whether the results for managers showed any consistency over time.

wpDiscuz