Leveraging the Orioles Bullpen

Last night the Orioles won another one-run game — no surprise there. Darren O’Day, Pedro Strop, and Jim Johnson combined for a scoreless 7th, 8th, and 9th — also no surprise. The Orioles bullpen is a major reason the team is tied atop the AL East. On Tuesday Jeff Zimmerman argued that successfully leveraging these relievers could partially account for the O’s performance in one run games. The theory is that an inconsistent rotation and streaky offense put the O’s in a disproportionate number of low leverage situations and preserving the best relievers hurts run differential in these games, while allowing the team to consistently use its best arms in close games. The result? Bigger blowout losses, fewer blowout wins, and more wins than typical run differential expectations would dictate.

Testing the theory that there is a method to the O’s 2012 run requires analysis of three questions. First, do the O’s find themselves in enough low leverage situations for such a strategy to impact their run differential on a significant scale? As Jeff pointed out on Tuesday, relative to other similarly successful teams the O’s seem to have a disproportionate number of blowout losses—games lost by more than six runs. In fact, the O’s have the 12th highest number of low leverage relief appearances in baseball.

Second, do the O’s actually employ a unique strategy in low leverage situations that suggests they are preserving the best relievers for higher leverage situations? Finally, does this strategy actually mean that the O’s best relievers appear in fewer low leverage situations and more high leverage situations than do the best relievers for other teams? The phrasing of this final question is very important, because every team tries to avoid using the top relievers in blowouts. The real question is do the O’s do so more successfully than other teams and, more importantly, is the result that these relievers consistently are available and successful in high leverage situations. The rest of the analysis will focus on these two important questions.

Surviving Blowouts: Low Leverage Extraordinaire Kevin Gregg

The first part of the theory that must be tested is whether or not the Orioles do in fact manage to avoid using the best relievers in low leverage situations. One method for doing so is to force starters to suck it up and push through bad starts to eat innings, but eventually this takes its toll and has long-term consequences. The table below shows all 11 of the Orioles losses by more than six runs (hereafter referred to as blowouts) this year in order of occurrence by the starting pitcher, the number of innings they threw, and the earned runs they allowed.

Pitcher Innings Runs
Brian Matusz 5.2 5
Tommy Hunter 5.2 8
Brian Matusz 5.0 7
Jake Arrieta 6.1 6
Jason Hammel 3.1 8
Chris Tillman 0.1 7
Miguel Gonzalez 2.2 7
Zach Britton 2.2 7
Tommy Hunter 3.0 8
Joe Saunders 5.1 6
Zach Britton 3.1 5

While there are some cases of pitchers gutting it out for a while, in all of these blowouts there were obviously a significant number of low leverage innings thrown by relievers. I don’t think we can say definitely that any blowout strategy employed involves hanging starters out to dry longer than other teams normally do.

So where do the Orioles’ low leverage relief innings go? To Kevin Gregg, of course. At .49, Kevin Gregg has the second lowest gmLI (average leverage index upon entering the game) of any reliever with at least 40 appearances in baseball. He has thrown in nine of the Orioles 11 blowout losses and two of their four blowout wins. In the process he has posted a very pedestrian 4.74 ERA, but has managed to grind through 43.2 innings in 40 appearances. He is not alone in the mop up ranks within elite bullpens, though. JP Howell for the Rays, Alfredo Simon for the Reds, Cristhian Martinez for the Braves, and Louis Coleman for the Royals have all made 40 or more relief appearances and have a gmLI less than .70 pitching for bullpens in the top ten in WAR.

Kevin Gregg is not the end of the story, however, and in fact it is the myriad of other pitchers the Orioles have used in low leverage situations that is most impressive. The team has a combined 54 relief appearances from 12 different pitchers including normal starter Tommy Hunter, acquisitions Randy Wolf, Miguel Socolovich, and J.C. Romero, and journeymen call ups like Dana Eveland. Of these 54 appearances, 37 (69%) of them have been in low leverage situations. All told the Orioles have used 19 relievers this year (if you include Chris Davis’s emergency extra inning appearance). Only ten teams have used at least 18 relievers this season and of these teams, only the Orioles and Red Sox rank in the top 15 in baseball in reliever WAR, implying that for the other teams it has a lot more to do with ineffectiveness than with strategy.

Leveraging O’Day, Strop, and Johnson

So the Orioles have been creative in constructing the relief corps, but how does that translate into actual usage of the team’s best relievers? First, a quick return to the 15 blowouts the Orioles have taken part in. Below is a table of the Orioles seven most used relievers (including Kevin Gregg) with their appearances in blowout losses, blowout wins, and all blowout games.

Pitcher Blowout Loss Blowout Win Total
Jim Johnson 1 1
Pedro Strop 1 1
Darren O’Day 4 4
Luis Ayala 3 1 4
Troy Patton 5 1 5
Matt Lindstrom 5 1 6
Kevin Gregg 9 2 11

The Orioles thought enough of Matt Lindstrom to trade him to Arizona and a fair number of Troy Patton’s low leverage appearances came early in the season when he was somewhat less established. That leaves 10 total appearances in blowouts by Jim Johnson, Pedro Strop, Darren O’Day, and Luis Ayala in the eleven blowouts. However, that actually overstates the case, as half of those 10 appearances came during the stretch immediately after the All-Star break, where the Orioles played 20 games 20 days. During this stretch, the Orioles were at some points forced out of necessity to get innings out of these pitchers simply due to the lack of off days. Even including these games Jim Johnson and Pedro Strop have only pitched in one blowout each. Strop’s came after three straight rest days when he needed work. Johnson’s was a strange departure from the strategy on July 16th when he gave up 5 ER in 1/3 of an inning in a throwaway loss to Minnesota.

Now, the important point of all this is that the Orioles should be able to use their top pitchers in the most high leverage situations and in the least low leverage situations. I very subjectively chose the top three relievers from each team and looked at the percentage of each reliever’s appearances that were low leverage and the percentage that were high leverage. I also looked at the proportion of a team’s high and low leverage appearances accounted for by these top relievers.

The Orioles seem to do pretty well within this framework, but there are a handful of other teams that leverage their best relievers similarly. The average proportion of total appearances that were in low leverage situations among O’Day, Strop, and Johnson was the 9th lowest in the league. These three relievers made up 28% of the team’s total relief appearances in low leverage situations, good for the 11th fewest in the league. More importantly, the average proportion of total appearances that were in high leverage situations for the Orioles relievers was second highest only to the Royals (for whom I included Jonathan Broxton, despite being traded, because without including the team’s closer for much of the season it skews the results). These three relievers made up 63% of the Orioles total high leverage appearances, good for 6th in the league.

As far as minimizing appearances in low leverage situations and maximizing appearances in high leverage situations for the top three relievers, the Giants, Indians, Reds, and Royals are the only teams that seem to do as well or better than the Orioles across the board. As they did last night, the Rays rely heavily on Joel Peralta, Jake McGee, and Fernando Rodney in high leverage situations, but also use them in a fair amount of low leverage situations.

If we restrict our analysis to only the team’s top two relievers the combination of Jim Johnson and Pedro Strop rank 3rd and 30th in the league in gmLI, respectively. Here only the Rockies, Reds, Indians, and Giants have a higher combined ranking among their top two relievers. In fact, the three headed monster of Javier Lopez, Santiago Casilla, and Sergio Romo in San Francisco has a combined higher ranking with gmLI’s of 6, 10, and 11, respectively.

Strategy or Luck?

In a way this is circular logic, but baseball’s best bullpens all do a good job of leveraging their best relievers and the Orioles seem to fit squarely into this group. What appears to be different is the manner in which the Orioles have creatively found ways to avoid using these relievers in blowout games, which in turn has caused the team’s run differential to take a hit.

I do not think this is the whole story — certainly a combination of randomness and offensive performance has a significant impact on winning close games as well. I do think, however, that Dan Duquette and Buck Showalter deserve credit for being creative and sticking to a potentially unpopular method for utilizing an already great bullpen. Having been a part of teams that felt like the coaches quit on games that got out of hand, I cannot emphasize enough the importance of Showalter’s ability to communicate this strategy with players to prevent it from becoming a negative.

As Jeff Zimmerman pointed out on Tuesday, whatever the true reason for the Orioles unlikely success, barring a major collapse in the final weeks we could be witnessing the greatest “over-performance” in baseball history — or perhaps one of the greatest strokes of managerial genius.



Print This Post





Sort by:   newest | oldest | most voted
The Sauce
Guest
The Sauce
3 years 10 months ago

I was curious as to whether Buck Showalter’s teams have a tendency to outperform their WAR, given his (absurdly consistent) history of turning losing teams into playoff teams in his second season.

Overall, Buck’s teams have actually underperformed their WAR by 5.8 wins. Disclaimer: I was too lazy to figure out actual replacement level win totals, so I used 43 wins and prorated the strike season. Also, I ignored the 50 some odd games he coached in 2010, because, honestly, does anyone care about the 2010 Orioles? Figuring out accurate replacement team numbers would almost certainly result in worse numbers for Buck.

Before 2012, that number turns into -16.8 wins over 12 years. So I guess if we decide that the O’s outperforming their true talent is Buck’s doing, then we probably have to come to terms with the fact that Buck has been a bad manager when we survey his entire career.

Curiously, of the five seasons out of 13 that Buck’s teams have outperformed their WAR, three have been his second year with a team.

When you look at his Pythag versus his actual wins, he comes out at 5 wins above expected (-6 before this season).

My conclusion? Managers are overrated. Also, there are too many baseball stats, which makes my head hurt.

jpg
Guest
jpg
3 years 10 months ago

Very interesting article, thanks. Where/when did you play? Just curious.

MC
Guest
MC
3 years 10 months ago

The O’s have been a big puzzle for me, I haven’t been able to figure this team out at all. If you look at their offense and their starting pitching, it’s pretty mediocre. OK the pitching is better than the offense.

To me there is just no way that this team should have this many wins.

VeveJones007
Guest
VeveJones007
3 years 10 months ago

The Orioles are 64-0 when leading after 7 innings–so they condense the game. They are 3rd in MLB in home runs–so they can change the complexion of a game on one swing. In a condensed game, any runs (especially home runs) are like gold.

MC
Guest
MC
3 years 10 months ago

OK terrific my question is why are they 64-0 and those situations

MC
Guest
MC
3 years 10 months ago

* in * those situations (Typo)

MC
Guest
MC
3 years 10 months ago

In other words just clarify why does a basically mediocre team put itself in the position of leading after 7 innings so often

Matt P
Guest
Matt P
3 years 10 months ago

I’m not sure having a lead after the seventh in 64 out of 143 games is that good. That means in 79 out of 143 games, they’ve been either tied or behind after the seventh.

My guess is that it’s the starting pitching. Either the starters get rocked or they put together a good outing (but not necessarily going six innings). The offense is designed to score less than the average but not get shut out (lots of power and bad OBP). If you can put together a good pitching outing and consistantly get a few runs each game, you’ll win a lot of low scoring games.

I could be wrong though. I’m not sure how to test it with the data that I have.

Matt P
Guest
Matt P
3 years 10 months ago

I think you missed the most important question:

Is there enough of a difference between the Orioles “bad” relievers and the Orioles “good” relievers for it to matter which ones pitch in low leverage situations?

Given that Patton, Lindstrom, Strop, JJ, O’Day and Ayala all have ERAs under 3 it means that it doesn’t matter which ones are pitching in high-leverage or low-leverage situations because they’re all effective. The only way leveraging matters is if Gregg, Eveland and the Os bad relievers are so bad that they have a substantial impact on bullpen performance.

To be fair, it seems that they do. The Orioles high leverage relievers have an average ERA of 2.5 while the low leverage guys have an average ERA of 4.23.

But there’s only one small problem. The Rays also use their relievers well. Their high leverage guys have an ERA of 2.16 while their low guys have an ERA of 3.32. Given your argument, you’d expect the Rays to be awesome in one run games. Unfortunately, they’re 20-26.

The difficulty of arguing that the Orioles bullpen is helping them win one run games is the fact that the Rays bullpen (with a better performance) isn’t.

jsolid
Guest
jsolid
3 years 10 months ago

What the Orioles are doing is historically unprecedented. I dont think people fully grasp that.

Using baseball-reference.com, I looked at every single MLB team from every season from 1901-2011. that’s 2,302 team-seasons. Exactly 14 teams finished with a 1-run game winning percentage of .700+ The all-time record is held by the 1981 Orioles at .750. if you think that a strike-shortened season shouldnt count, your team is the 1908 Pirates, at .733. [data here] And how are the 2012 Orioles doing? They currently have a 27-7 record, for a winning percentage of .794

The 2012 Orioles are about to set the all-time record for 1-run game winning percentage – and by a lot.

That tells me that everything is going right, including a ton of luck. The fact that on the season the Orioles have scored fewer runs than they have allowed makes it even more jaw-droppingly absurd.

MC
Guest
MC
3 years 10 months ago

Yes exactly. On paper they shouldn’t be winning this many games but they somehow manage to scrape by, and actually take series from the Rays, Yankees, etc., who we would all agree have much better teams on paper.

wpDiscuz