In my last post on explaining pitchers’ BABIPs by way of their batted ball rates, I was very careful to say that it was applicable in the long run, as it’s hard to be accurate over a short number of innings pitched, due to all the “noise” in BABIP (Batting Average on Balls In Play). I only used pitchers with a qualifying number of innings pitched (IP) in the calculations, for that reason. After writing the post, I did some messing around with the data, to find out just how much of an effect IP had on the predictability of BABIP.
Hold on to your propeller beanies, fellow stat geeks: the correlation between xBABIP and BABIP went from 0.805 when the minimum IP was set to 1500, to 0.632 at a 200 IP minimum, down to 0.518 at 50 IP. OK, maybe it’s not that surprising. Still, I thought I’d better show you how confident you can be in my xBABIP formula’s accuracy when you take the pitcher’s innings pitched into account.
The formula, again: xBABIP = 0.4*LD% – 0.6*FB%*IFFB% + 0.235
And remember, that formula is primarily meant to be a backwards-looking estimator of “true,” defense-neutral BABIP. My next article will (probably) discuss another formula I’ve come up with that’s more forward-looking.
Here’s the raw scatter plot of BABIP against IP over my main 2002-2012 sample (using the overall numbers for each pitcher):
As you can see, towards the lower innings pitched, there is massive variation, but even with plenty of innings, there are still noteworthy differences between them that remain. Also notice the outliers in the lower IP area tend towards high BABIPs — they probably didn’t get much time in the MLB due to their horrible BABIPs (whether that was due to bad luck or incompetence, who knows).
Next, we have a comparison of the Mean Absolute Error (MAE) between xBABIP and actual BABIP, according to innings pitched. This was done by grouping pitchers into one of several IP categories, which you’ll see specified in the table below the graph. You can interpret the MAE as the average amount that xBABIP will be off the mark from actual BABIP, either plus or minus. It’s similar to RMSE (Root Mean Squared Error), except that RMSE basically gives an extra penalty for being further off the mark (which makes RMSE arguably more useful for comparing formulas, but more difficult to interpret and apply).
So, the formula xMAE = 0.163 * IP^-0.436 for xBABIP pretty much nails it with a 0.9961 r-squared (an r^2 of 1 is a perfect fit). That’s the overall trend, anyway — it’s a lot noisier when applied to individual pitchers instead of averages, of course. But I guess you could say the unpredictability is very predictable. Here are the average MAEs for each range of IPs, and their expected MAEs according to the formula:
|IP range||MAE||xMAE||Standard Deviation of BABIP||# of Pitchers|
|1500 to 2500||0.00591||0.00593||0.01204||37|
|1000 to 1500||0.00764||0.00728||0.01073||70|
|500 to 1000||0.00918||0.00909||0.01541||182|
|250 to 500||0.01201||0.01230||0.01841||279|
|100 to 250||0.01595||0.01715||0.02357||365|
|50 to 100||0.02483||0.02481||0.03419||267|
|10 to 50||0.03869||0.03700||0.05219||459|
And here are some confidence measures:
|IP range||Percent of the time MAE is less than:|
|1500 to 2500||55.3%||76.3%||97.4%||100.0%||100.0%||100.0%||100.0%||100.0%|
|1000 to 1500||40.0%||70.0%||87.1%||95.7%||100.0%||100.0%||100.0%||100.0%|
|500 to 1000||29.7%||59.9%||84.6%||95.6%||96.2%||98.4%||99.5%||100.0%|
|250 to 500||26.2%||48.4%||65.2%||81.4%||91.0%||95.7%||98.6%||99.6%|
|100 to 250||21.1%||39.7%||58.4%||68.8%||80.0%||85.5%||91.0%||94.2%|
|50 to 100||12.7%||24.3%||36.7%||49.1%||57.3%||66.3%||71.5%||79.4%|
|10 to 50||6.5%||14.2%||19.4%||24.6%||29.1%||34.6%||40.5%||47.1%|
So, you could say the standard 95% confidence interval can be achieved to within 0.020 points BABIP when there are over 500 IP. However, when there are fewer than 100 IP of data to work with, there’s not much you can be very sure of.
If you read my last article, you know that I see popups (defined as FB%*IFFB%) as the main key to distinguishing pitchers’ BABIPs. I took a little time to see if some of the factors I uncovered as being correlated to infield fly balls could be used to come up with an estimation of the rate. Specifically, I went for factors that are no-doubt-about-it controlled by the pitcher. Here are the variables:
xPU% = expected popup percentage (also known as xFB%*IFFB%)
FA% = percentage of 4-seam fastballs thrown
FAZ = vertical movement on 4-seamers (positive values means it rises relative to a spinless ball)
SI% = percentage of sinkers thrown
FAvCH = the difference in speed between the pitcher’s 4-seamer and changeup (=vFA – vCH)
Zone% = Percentage of pitches thrown in the zone
I won’t worry about trying to simplify the formula down, because I figure hardly anybody is actually going to try to use this:
xPU% = 0.00289*FA%*FAZ + 0.00189*FAZ – 0.01815*FA% -0.00589*SI% + 0.00111*FAvCH + 0.05398*Zone% – 0.02040
The results: a 0.627 correlation to FB%*IFFB%, and a MAE of 0.789% (the mean and standard deviation of actual FB%*IFFB% are 3.616% and 1.308%, respectively). Considering this doesn’t even take into account pitch locations (other than the vague Zone%), and that 5 variables are a pretty small piece of the overall puzzle, this seems pretty good to me.
If you’re curious, xPU% according to this formula has a -0.172 correlation to BABIP, -0.199 to HR/FB, and 0.388 to K% (which means pitchers with high xPU% tend to look better in all these areas), but on the downside, a 0.219 correlation to LD%. It has almost no apparent connection to ERA (-0.045 correlation).
Updates to My Previous Article
I had included the stat Pace (which is the time a pitcher takes between pitches) as an interesting correlate to some factors (especially K%), but as J. Cross of Steamer Projections pointed out to me in a conversation on the Inside the Book site, Pace is mainly explained by % of innings as a starter. For whatever reason, relievers tend to wait longer between pitches, but they also tend to strike out more hitters and generally have lower BABIPs as well. I suspect the latter two points have a lot to do with my sample — the worse (or unluckier) relievers weren’t used enough to meet the IP requirements, so were excluded from consideration.
Matthew Cornwell, in that same discussion, brought up how extreme groundball pitchers tend to have lower than expected BABIPs. I made a scatter plot and found that there appears to be some truth to that:
The quadratic equation that dips a bit towards the extreme GB% legitimately does work better than the linear one, as the r^2 numbers show. Apparently, the new version of SIERA employs something along those lines as part of its formula. It does make some sense — the extreme GB% pitchers are probably getting more of the ground ball equivalent of a popup: the chopper. A chopper is far from an automatic out when the runner is fast, but it’s still a desirable thing overall, I’m sure.
A similar trend exists for OFFB%, or outfield fly ball percentage, which is FB% – FB%*IFFB%. Actually, it’s an even stronger effect, although apparently IFFB%*FB% itself captures a lot of it.
I applied all this new knowledge to coming up with a new xBABIP formula, and the effects were pretty modest, I have to say. Here it is:
xBABIP = 0.59*LD% – 0.31*FB%*IFFB% + 0.39*LD% – 0.22*LD%^2 + 0.21*OFFB% – 0.08*OFFB%^2
Over the same sample I used in the last article (qualified IP = 300+), there was only a 0.005 improvement in correlation, and a 0.00014 improvement in MAE, compared to the simple version. Not a big upgrade for all that extra complexity, but there you have it.
Thanks again to J. Cross for providing me with the pitcher handedness data. Unfortunately, there’s not a lot I can say about it, other than that lefties had a BABIP a little under 0.002 higher than righties over my data set.
It’s now almost a couple weeks after I initially submitted this article (I’m waiting for it to be published), and Eno Sarris just posted an article today that compares my formula (the simple version) to SIERA’s BABIP component, which uses the formula: 0.295 + 0.045*GB% – 0.103*K%. I thought I’d put them head-to-head according to different criteria. A couple I’ll introduce now: wMAE and wRMSE, which are the weighted versions of MAE and RMSE (in this case, weighted by innings pitched). I’m also including results for J Cross’ formula, which is: 0.67*LD% + 0.24*GB% + 0.17*OFFB% (OFFB% means outfield fly ball percentage, which is the fly balls that aren’t infield fly balls, of course). This comparison, as usual, is for qualified pitchers (>300 IP) over the span of 2002-2012:
|xBABIP (simple)||xBABIP (complex)||SIERA BABIP||J Cross|
|Correlation to BABIP||0.62800||0.63434||0.40811||0.60725|
As Eno correctly points out, putting a lot of weight on line drives does hurt the value for forecasting future BABIPs, since line drives are hard to predict. But an article with a new formula tailored to predicting future BABIPs is in the works. I’ll compare that formula to all of these.
I also have to say, as you can see from the effect of IP, don’t use only one season worth of batted ball data if you’re trying to project the next season — you’re much better off using several years’ worth.
I’ll leave you now with a couple of lists — the pitchers whose BABIPs from 2002-2012 are worst explained by the new, improved(?) formula, either due to good luck or misfortune, their defenses, or something else the formula just can’t explain. They were selected for having the highest combinations of innings pitched and deviations from actual BABIP (but then listed according to the deviation afterwards):
|IP||BABIP||xBABIP (simple)||xBABIP (complex)||Expected – Actual|
|IP||BABIP||xBABIP (simple)||xBABIP (complex)||Expected – Actual|
Lowest xBABIPs according to complex formula (new addition)