Johnny B. Goode

Controlling the run game, pitcher fielding and ERA

Run & Glove

Johnny Cueto has been mocking his peripherals ever since his big league debut.  For the most part FIP serves as a terrific gauge for pitcher performance, but in 2011 Cueto made FIP look like a heart monitor trying to explain the weather.  On what most consider a separate note, base runners have a healthy and robust fear of Cueto’s pickoff move, which is one of the best in the show.

FIP measures outcomes a pitcher can control (home runs, walks and strikeouts) and chalks the rest up to random variation.  Studies have shown that stolen bases contribute relatively little to run creation and perhaps on that basis the ability to control the run game has generally been ignored or deemed overrated.

It is difficult, however, to ignore the six runs Cueto saved the Reds via his contributions to controlling the run game in 2012.  By contrast, A.J. Burnett’s inability to control runners cost the Pirates four runs. The typical scale is that 10 runs amount to one team win – and teams will pay about $5 million per win.

Acknowledging run game control cannot fully explain how Cueto has routinely outperformed his peripherals, just as it cannot wholly explain Pittsburgh’s inability to keep pace with Cincinnati in the NL Central last season.  It does, however, get us closer.

Incorporating a pitcher’s fielding ability proved of comparative importance in explaining and predicting performance.  Here we’ll turn to Mark Buehrle, whose glove has saved four runs per year since 2004, and among fellow hurlers the fast-working lefty has been one of the decade’s most steadily superb fielders.  FIP underestimated Buehrle in eight of the past nine seasons, slighting his ERA by an average of .30 per year over that span.

Numbers

 The numbers indicate that a pitcher’s defense and ability to control the run game should both be considered in assessing and forecasting the pitcher’s value.

Focusing on seasons in which pitchers hurled 100 same-league (AL or NL) innings from 2003-2012 (n=1400), I ran a multiple linear regression to create a formula (“MBRA”) incorporating run control (rSB) and pitcher fielding (rPM) on top of line drive and infield fly ball percentages (credit to BABIP guru Steve Staude) and a regressed take on FIP.

MBRA = (55.25*HR + 14.05*BB – 8.57*K)/TBF – .041*rPM – .056*rSB + (5.71*LD – 8.27*IFFB)/(LD+GB+FB) + 2.34 

Correlation

Mean Absolute Error

MBRA

.7750

.4570

FIP

.7647

.4697

BERA

.7477

.4922

tERA

.7472

.5616

MBRAT

.7216

.5394

xFIP

.6451

.5649

SIERA

.6290

.5768

 

MBRA is engineered to properly credit pitchers who can field and control the run game.  When I subtracted MBRA from FIP to locate the pitcher-seasons that benifitted most from my formula, I was encouraged seeing Buehrle show up twice in the top ten, and five times in the top 100 (again, that is out of 1400).

Next, I looked at seasons in which pitchers threw 100 same-league innings in consecutive seasons from 2003 to 2012 (n=791).  This time I ran a regression to create a model suited to predict a pitcher’s ERA based on his previous year’s statistics.

MBRAT = (20.12*HR + 7.13*BB – 6.7*K)/TBF -.025*rPM -.034*rSB + 2.37*ZC% + 2.22

 

Correlation

Mean Absolute Error

MBRAT

.4526

.6498

SBERA

.4398

.6582

BERA

.4347

.6634

xFIP

.4220

.6803

MBRA

.4198

.6987

FIP

.4162

.7024

ERA

.3630

.7920

MBRAT stands tall on the lofty pinnacle of public forward-looking ERA estimators, and if you factor in the percentage at which pitchers throw over the edge of the plate (EDGE%) its correlation jumps even higher (.4621).  Unfortunately, I only have Edge% data from 2008 to 2012 (n=362) and cannot yet justify its inclusion.

On Deck

I will create expectations for pitchers with fewer innings pitched and convert my findings to a WAR measure that may serve as a middle-ground between fWAR and rWAR.  I also stumbled on a potentially significant relationship between pick-off attempts and strand rates that may work its way into future formulas.



Print This Post

Cleveland native, University of Minnesota Law School alum, grand slam enthusiast and the last of the hardcore troubadours.


Sort by:   newest | oldest | most voted
Beni
Guest
Beni

Fascinating article, thanks for putting this together.

Glenn DuPaul
Guest

You need to test whether the components you are adding in (rPM, rSB, etc.) are significant. Obviously the more factors you throw in the model the higher the overall correlation. However, if they’re not adding significant predictive value then there is no reason to put them in your model. When you keep adding predictors you’ll artificially inflate the correlation without actually doing a better job of explaining or predicting.. For instance, if you look at the start of your MBRAT, it looks remarkably like my pFiP equation (http://www.hardballtimes.com/main/article/more-on-standard-deviation-and-era-estimators/):

20*HR + 10*BB – 10*K/TBF + 4.60

And I”m sure the correlation between pFIP and MBRAT would be very close despite using way less factors.

You would also need to test this equation out of sample, as you’re actually explaining future runs with MBRAT rather than predicting it. If you use the entire population and do not test outside of it, you’ll risk overfittting your model.

Dan Rozenson
Guest
Dan Rozenson

Glenn is right. Overfit is a risk if you use an explanatory formula for prediction. I’m also cautious of using LD%, which is more volatile than GB%.

Glenn DuPaul
Guest

I may have come off as a little too harsh in my original comment. I really do like the idea. Cueto’s ability to keep runners from stealing is actually absurd. No one runs on him, he picks people off and when they run they get caught. I do believe he deserves credit for that in his run prevention ability.

The same can be said for Buehrle’s defensive. Defensive Runs Saved for pitchers is a very real thing and they also deserve credit for it. I think MBRAT is a really good starting point, I just think the overall model could be improved if you split the sample instead of using the entirety of 2003-12.

Keep up the good work!

Tom Glavine
Guest
Tom Glavine

Brilliant work Mr. Greenlee. As the poster boy for FIP overachievers, it is perhaps no coincidence that I was also one of the best fielders to ever take the mound.

Glenn DuPaul
Guest

Greenlee,

Again great work and I figured that you had found p-values that were less than .05 for the predictors that you left, I just was not sure because you didn’t explicitly say it in the piece.

You still run the risk of an overfit even if you only have significant predictors though. The weights for the coefficients of all of the estimators you tested (well at least of SIERA, xFIP and FIP) were tested out of sample rather than being fit specifically for this sample; which inflates MBRAT’s correlation.

I know you would like to use as much of the sample as possible, but I can almost promise you that using 4-5 seasons will be a big enough sample to find similar weights, and that your six predictors will still be significant in those seasons. Once you do that you can test the correlation on the other four-five seasons. If MBRAT comes out ahead of SIERA, xFIP, FIP, pFIP, etc. Then you’ll know that this new estimator may, in fact, be the way to go.

I really like the first effort and like I said a pitcher’s defense and ability to control the running game do have an effect on their ERA/and future ERA.

Keep up the good work

BrianK384
Member
BrianK384

Very cool article, I’ve actually been hoping to see something like this for a while. Glenn, I disagree with your criticism to a certain extent. The author’s correlation will certainly be higher than the competing formulas if he incorporates their variables into his data-specific regression, which he has. Yet, so long as the author’s contributions are significant (rPM & rSB) there is added value beyond the “artificial inflation.” I think it boils down to — which time frame do you want to use to explain or predict data? If you want to use the last decade to do so, I would use the author’s formula for its advantage in considering pitcher fielding and run control. Nice work.

Jimbo
Guest
Jimbo

MBRA is named after Mark Buehrle then I take it? Very solid article. Eager to see if edge% can join pitcher-fielding and steal-prevention in future models.

J. Cross
Guest

I think this is a great idea. Why not just make a rSB or rPM adjustment to kwERA, FIP, xFIP, BERA or SIERA though? Is there reason to think that there’s an association between the stats in those metrics and defense or stolen base prevention such that they already take them into account in some accidental or indirect way. Maybe there is, I’m just posing the question.

Jimmy Ro
Guest
Jimmy Ro

Great article! Loved your analysis.

HAHarris
Member
HAHarris

Awesome work Mr. Greenlee! Glad I stumbled upon this before my fantasy draft. This looks even better than the predictor I used last season. It just may provide me with the edge I need to defend my championship.

Steve Staude
Member

Wow, great writing and outstanding research! Thanks for the shout-outs, by the way.

Anyway, were you able to find many details on what goes into rSB and rPM? I’m just wondering if they actually include run data within them, which would be a bit circular (unless you’re just using them to guess the next year’s ERA).

Steve Staude
Member

Very nice — great idea to include those.

Just double checking a couple of your numbers, I found that the same-year correlations for FIP averaged only 0.716, while BERA’s were 0.741 (though I didn’t go so far as to only look at same-league numbers there). FBERA was at 0.756, by the way. So your BERA sounds about right, but would you mind re-checking the 0.7647 you got for FIP?

beckett19
Member
beckett19

To Greenlee

Fantastic article. Definitely want to use MBRA for my fantasy league. I just though I’d make a quick suggestion. Have you considered an interactive term between GB% and pitcher fielding? It would seem to me that the more ground balls a pitcher produces, the more likely he is to get fielding chances, and thus his fielding would increase in importance for predictions.

JakeZilla
Guest
JakeZilla

This is sweet. Do you think ZiPS, Steamers and PECOTA will pick up on this?

Steve Staude
Member

FYI, everybody, I was off in my FIP calculation, somehow, and Greenlee’s looks about right. It does make a big difference whether you’re looking at a group of each player’s individual seasons or all of those seasons averaged together, though — FIP beats BERA in the former but not the latter (FBERA is tied in the former, and even better in the latter).

You know MBRA and MBRAT are not really defense-independent like their competitors here, but I think only accounting for a pitcher’s defense makes perfect sense. I think the category should be redefined as something like “non-pitcher-defense-independent”…

Bill but not Ted
Member

Excellent stuff, glad I have been witness to the great work coming out of community research over the last couple months

Big Bo Peep
Guest
Big Bo Peep

One more reason to steer clear of AJ Burnett. Cool idea, it definitely makes sense for pitchers to be held accountable for their defense.

jdm
Member
jdm

Just curious, at what quantities does edge%, rPM, rSB, and ZC% stabilize? How consistent are they year to year?

wpDiscuz