Relief Pitching in Context

If you recall, last week, I talked about one approach that we can take for evaluating starting pitcher performance. Today, I’d like to continue on that vein, this time taking a look at relief pitching.

With regards to evaluating both player performance and player talent, relief pitching is one of the least understood aspects of baseball. There are a few factors that lead me to believe this, but the only one I’d like to talk about today is the problem of mid-inning pitching changes.

You’ve probably noticed the ridiculousness of the following situation: A starting pitcher grooves through the first six innings of the game, allowing just one run. However, he loses his control with two outs in the seventh and walks the bases loaded, forcing the manager to call on the bullpen to get the last out. The reliever proceeds to allow a three-run double, followed by the third out.

Of course, in this situation, the starting pitcher is held responsible, and subsequently penalized for, all three runners that he left on base when the manager removed him from the game. Conversely, the reliever, despite being the active pitcher while all three runs scored, is rewarded for pitching a scoreless third of an inning. In the box score for the game, there is no indication that those three runs allotted to the starting pitcher in fact scored while said pitcher was sitting on the bench.

Anyone will agree that this is not only unfair to the relief pitcher, whose salary and legacy is strongly tied to his ERA, but a significant flaw in the way that we allocate runs allowed. It is particularly important for relief pitchers given their small number of innings pitched each season, and subsequently the larger importance of each run allowed.

On the surface, there is not a simple solution to this issue. For obvious reasons, we cannot assign all inherited runners to the reliever, nor can we assign none, and any whole number in between will likely be an arbitrary number.

Luckily for us, there is a fantastically simple solution: RE24. While it has a complicated name, RE24 takes the team’s expected runs scored in the inning before the play and subtracts that from their expected runs scored in the inning after the play. In other words, how much did the hitter, or conversely the pitcher, help his team maximize runs scored?

Let’s apply that to the case of inherited runners, specifically the one presented above. When the manager brings in the reliever with the bases loaded and two outs, the batting team was expected to score about 0.7 runs for the rest of the inning given an average offense and defense. However, the reliever then gives up a three-run double followed by the third out. At the end of the inning, the batting team obviously has a run expectancy of zero. Therefore, the run expectancy dropped from 0.7 to 0 from when the reliever entered the game, but three runs scored. The 0.7 run change in run expectancy minus the three runs scored means that the reliever’s RE24 for the inning was -2.3. In contrast to the official Runs Allowed, in which the reliever is assigned zero runs, RE24 assigns him almost all of the runs that scored, minus the runs that were expected to score when the starter left the game.

Make sense? Good. Let’s look at some actual numbers.

Top ten reliever seasons by RE24, since 1974:

Num	Name	Season	Team	RE24
1	Mark Eichhorn	1986	Blue Jays	52.88
2	Doug Corbett	1980	Twins	43.81
3	Bill Campbell	1977	Red Sox	42.97
4	Mariano Rivera	1996	Yankees	40.48
5	Jim Kern	1979	Rangers	40.38
6	Rich Gossage	1977	Pirates	40.1
7	Willie Hernandez	1984	Tigers	37.91
8	Rich Gossage	1975	White Sox	35.8
9	Keith Foulke	1999	White Sox	35.14
10	B.J. Ryan	2006	Blue Jays	35.13

Mark Eichhorn’s epic season in which he pitched 157 innings with a 1.72 ERA all in relief leads the way, and it’s not close. There are some other big names on this list, as well as some not-so-big names, but regardless, these are ten of the best relief seasons in the past 40 years.

While the above list is somewhat interesting, it doesn’t really tell us everything we want to know. First of all, RE24 is a cumulative metric, as evidenced by the fact that almost every one of those relievers pitched over 100 innings, a feat rarely seen in recent years. Secondly, the scale means very little for most people. There is nothing to compare that number to as far as we normally measure pitching, making it difficult to practically use in conversation and discussion.

These two concerns can be solved in one simple way: convert RE24 to a “runs-per-nine-innings” scale similar to ERA and RA. At this point, I must give credit where credit is due, as Tom Tango provided the method for converting RE24 to an RA9-scale yesterday:

[W]e can recast RE24 into an RA9 scale (i.e., similar to ERA) as follows.  Say the league average is .48 runs per inning.  Say you have a pitcher that has an RE24 of +40 runs and has pitched 200 innings.  That means the league average is .48 x 200 = 96 runs, and our pitcher here is 40 runs better than that, or 56 runs allowed.  So, his (RE24-based) RA9 is simply 56/200*9 = 2.52.

Don’t get bogged down in the details of the calculation. The result is a metric which measures the pitcher’s runs allowed per nine innings using RE24 instead of actual runs allowed. We’ll call this Context RA9, or cRA9, going forward.

The great thing about cRA9 is not just that we have a more accurate measure of a relief pitcher’s run prevention, but that we can compare it to their RA9 in order to identify the best and worst pitchers at preventing inhereted runners from scoring.

First of all, the leaders in reliever Context RA9 since 1974:

Num	Name	Season	Team	RA9	cRA9	Diff
1	David Robertson	2011	Yankees	1.21	0.38	-0.84
2	Al Alburquerque	2011	Tigers	1.87	0.40	-1.47
3	B.J. Ryan	2006	Blue Jays	1.49	0.54	-0.95
4	Joaquin Benoit	2005	Rangers	1.30	0.55	-0.74
5	Mike Jackson	1998	Indians	1.55	0.70	-0.85
6	Jonathan Papelbon	2006	Red Sox	1.05	0.70	-0.35
7	Rob Murphy	1986	Reds	0.72	0.81	0.09
8	Cla Meredith	2006	Padres	1.07	0.84	-0.22
9	Neal Cotts	2013	Rangers	1.41	0.86	-0.55
10	Greg Holland	2011	Royals	1.95	0.89	-1.06

Wow. I remember David Robertson’s 2011 season as being very good, but based on RE24/cRA9, it was historically elite. He is known as “Houdini” for his ability to pitch out of seemingly impossible jams, and these numbers are just further evidence of that. Amazingly enough, Al Alberquerque had an equally impressive season the same year, albeit with fewer innings pitched.

A noted above, it may also be interesting to look at the relief pitchers with the largest positive differences between their Context RA9 and their normal RA9 — that is, the pitchers who received more credit than they deserved from RA9 and ERA:

Num	Name	Season	Team	RA9	cRA9	Diff
1	Dan Quisenberry	1987	Royals	2.76	5.66	2.90
2	Joe Smith	2007	Mets	3.65	6.23	2.57
3	Mike Duvall	1999	Devil Rays	4.73	7.24	2.51
4	Tim Crews	1989	Dodgers	3.94	6.19	2.25
5	John Franco	1996	Mets	2.50	4.73	2.23
6	Luis Aquino	1995	– – –	7.23	9.43	2.21
7	Scott Radinsky	1996	Dodgers	3.27	5.44	2.18
8	Scott Radinsky	1998	Dodgers	3.06	5.23	2.16
9	Joe Beckwith	1980	Dodgers	2.56	4.67	2.11
10	Matt Kinney	2004	– – –	4.53	6.63	2.10

This list, above everything else, is evidence that reliever ERA or RA9 can be, frankly, horrible indicators of a pitcher’s actual performance. By RA9, pitchers like Quisenberry and Franco look fantastic, but by cRA9, we might think them deserving of demotion to the minors.

On the other side of the coin, here are the ten largest negative differences in cRA9 and RA9 since 1974, or the pitchers who were better than their RA9 and ERA indicated:

Num	Name	Season	Team	RA9	cRA9	Diff
1	Javier Lopez	2004	Rockies	7.52	4.13	-3.40
2	Mike Munoz	1997	Rockies	4.93	2.38	-2.54
3	Mike Holtz	2000	Angels	5.71	3.20	-2.51
4	Steve Frey	1992	Angels	3.57	1.37	-2.20
5	Dennis Cook	1996	Rangers	4.35	2.21	-2.14
6	Lee Smith	1981	Cubs	4.30	2.42	-1.88
7	Dennis Powell	1992	Mariners	4.74	2.88	-1.86
8	Steve Reed	1993	Rockies	5.02	3.18	-1.83
9	Geoff Geary	2007	Phillies	5.88	4.09	-1.79
10	Andrew Miller	2012	Red Sox	3.35	1.56	-1.79

Again, these numbers are indicators of the unreliability of conventional run prevention metrics with regards to relievers, especially relievers that often come into games in the middle of an inning. By cRA9, we see a list of mostly very good, possibly elite, relief pitchers,  but by RA9, we see a list of pitchers only worth appearing in blowouts.

The issue presented at the beginning of this piece should not be a controversial one. The idea that starting pitchers should be responsible for all runners left on base is ridiculous, and the idea that relievers should not be responsible for said runners is just as ridiculous, but more important. As the results above show, a relief pitcher’s performance when brought in mid-inning can be the difference between one of the best reliever seasons in baseball and one of the worst.

So next time you see a relief pitcher’s ERA, consider the context of their appearances and their RE24 compared to their peers before you come to any conclusions.