Love it – I’ve always believed that the two things that you can’t fake are raw power (a player who can hit 5 HRs against MLB talent, even in spring training, meets the power threshold) and the ability to strike MLB hitters out. Those skills can change over time, but they are definitely skills, and spring performance can be analyzed for changes in skill.

Matt, not COMPLETLEY random. .02 or .04 is still something. As well, what is the P-value or confidence interval? It is possible that you made a Type II error, no, and that the true values are .1 or .2?

In any case, we expect a much lower correlation with ERA, but since ERA is explicitly related to BB and K rates, of course it is not going to be completely random if K and BB rates are not.

I think also we can infer the ERA correlation, at least an estimate of it, from the K and BB correlations, no? The only thing missing would be the HR correlation, since we know that the BABIP correlation between any two samples, let alone 10 IP of spring training and a whole season, is going to be very near zero.

Oh, and really good stuff, BTW! Although I am not surprised that there is some correlation and predictive effect. Why shouldn’t there be? Sure, pitchers work on stuff, they are not in mid-season form, and their competition is not quite the same as the regular reason, but still, I would think that pitching is pitching. The principal limiting factor in terms of the meaning/value of ST stats is sample size. 10 or 20 IP is not much of a sample to me that predictive regardless of whether it is ST or the regular season. I would love to compare the predictive value/correlation of what you got to the same number of IP during the regular season (of course you have to exclude that sample from the regular season stats). I suspect that the ST stats are a little less predictive than the same number of IP during the reg season for the obvious reasons.

The study did not look at ERA so why is that part of the conclusion?.

The study showed that BB and K rates in ST are somewhat correlated with BB and K rates in regular season. However, most people want to know if there is any meaning in terms of seasons runs allowed (or ERA), and BB and K rates are only part of the equation there.

Furthermore, while the data may be significant for the population of pitchers, it is still rather meaningless for individual pitchers whose results may be driven by weaker competition, low arm strength, working on new pitches/mechanics, etc.

Any chance that adjusting for location (Ariz. vs. Flo.) would help the predictiveness of the spring stats? I’m sure it’s not worth doing exact park factors, but I wonder if K and BB rates differ materially between the two ST leagues.

My long-ago college stats courses taught me that adding any variable will always increase R squared. You need to also look at adjusted R squared and the p-stat of each of the variables separately. (As a start.)

The study did look at ERA, but I didn’t publish the correlations because they were so low. My last sentence though stated “Last, I figured I would also test spring ERA one more time to see if we can glean anything from it. As expected, the results proved that itâ€™s all noise.” The correlations I posted in a comment above:

Spring to season ERA: 0.035
Spring to Marcel ERA: .02

This is a good question and would require a lot more work. I am pretty sure the stadiums in Ariz are much more hitter friendly, so park adjusting may help. But i’m not sure the effect they have on just K% and BB%

I’ll try to get Matt in here to respond to the rest of the math related questions. So much for my college stats class…guess that knowledge disappeared from my brain soon after.

The standard errors are about .02, so the coefficients’ confidence intervals are something like (.14,.22) for Spring K% and something like (.08,.16) for Spring BB%.

Comment by Matt Swartz — March 27, 2012 @ 11:46 am

Are you concerned about normality?

Comment by Matt Swartz — March 27, 2012 @ 11:47 am

Adjusted R^2 was decidedly up. The p-stats were <.001, so usually that increases Adj R^2.

Comment by Matt Swartz — March 27, 2012 @ 11:49 am

Less interested in the normality of the errors, and more interested in the assumption that the observational data used is i.i.d. – more specifically, interested in statistics which test that assumption.

This is a very nice start. I have a couple of suggestions:

1. One potential problem is that essentially you are fitting these observational data and then trying to use them to predict future performances. All these really tell is is that your model fits *these* data quite well, but we don’t really have any sense of how the model plays out going forward. It would be more compelling if you randomly took half your cases to fit the original model and then used the coefficients from that model to predict other half of the cases. We could then see what the R2 would be on this other half to see if it retains its predictive power.

2. It could be the case that once you control for K% BB% becomes insignificant. so you may want to run a model with both included.

3. The magnitude of the BB% coefficient is about 2/3 of the size of the K% variable suggesting that K% has an effect about 50% larger than BB%.

4. Finally, it seems that these effects are relatively small. If I understand your metrics correctly, a 1% increase in K% corresponds to a .18% increase in regular season K% (with Marcel held constant). So a 10% increase in Spring K% corresponds to a 1.8% increase regular season K%.

Despite my suggestions, this is a very interesting first step. Well done!

I guess this question is for Matt – any goodness of fit tests performed to test the i.i.d. assumption?

Comment by Chris — March 26, 2012 @ 9:20 am

Love it – I’ve always believed that the two things that you can’t fake are raw power (a player who can hit 5 HRs against MLB talent, even in spring training, meets the power threshold) and the ability to strike MLB hitters out. Those skills can change over time, but they are definitely skills, and spring performance can be analyzed for changes in skill.

Comment by V — March 26, 2012 @ 9:21 am

What is the correlation between Spring ERA and Season/Marcel ERA?

(I’m looking for a number here, to back up your assertion that “it’s all noise.”)

Comment by max — March 26, 2012 @ 4:54 pm

To season ERA: 0.035

To Marcel ERA: .02

Yup, completely random. Yet Rotoworld and the media continue to talk about these numbers like they have any meaning at all.

Comment by Mike Podhorzer — March 26, 2012 @ 6:03 pm

Matt, not COMPLETLEY random. .02 or .04 is still something. As well, what is the P-value or confidence interval? It is possible that you made a Type II error, no, and that the true values are .1 or .2?

In any case, we expect a much lower correlation with ERA, but since ERA is explicitly related to BB and K rates, of course it is not going to be completely random if K and BB rates are not.

I think also we can infer the ERA correlation, at least an estimate of it, from the K and BB correlations, no? The only thing missing would be the HR correlation, since we know that the BABIP correlation between any two samples, let alone 10 IP of spring training and a whole season, is going to be very near zero.

Comment by MGL — March 27, 2012 @ 1:43 am

Oh, and really good stuff, BTW! Although I am not surprised that there is some correlation and predictive effect. Why shouldn’t there be? Sure, pitchers work on stuff, they are not in mid-season form, and their competition is not quite the same as the regular reason, but still, I would think that pitching is pitching. The principal limiting factor in terms of the meaning/value of ST stats is sample size. 10 or 20 IP is not much of a sample to me that predictive regardless of whether it is ST or the regular season. I would love to compare the predictive value/correlation of what you got to the same number of IP during the regular season (of course you have to exclude that sample from the regular season stats). I suspect that the ST stats are a little less predictive than the same number of IP during the reg season for the obvious reasons.

Comment by MGL — March 27, 2012 @ 1:47 am

The study did not look at ERA so why is that part of the conclusion?.

The study showed that BB and K rates in ST are somewhat correlated with BB and K rates in regular season. However, most people want to know if there is any meaning in terms of seasons runs allowed (or ERA), and BB and K rates are only part of the equation there.

Furthermore, while the data may be significant for the population of pitchers, it is still rather meaningless for individual pitchers whose results may be driven by weaker competition, low arm strength, working on new pitches/mechanics, etc.

Comment by pft — March 27, 2012 @ 2:12 am

Any chance that adjusting for location (Ariz. vs. Flo.) would help the predictiveness of the spring stats? I’m sure it’s not worth doing exact park factors, but I wonder if K and BB rates differ materially between the two ST leagues.

Comment by MP — March 27, 2012 @ 3:27 am

My long-ago college stats courses taught me that adding any variable will always increase R squared. You need to also look at adjusted R squared and the p-stat of each of the variables separately. (As a start.)

Comment by mulkowsky — March 27, 2012 @ 8:08 am

The study did look at ERA, but I didn’t publish the correlations because they were so low. My last sentence though stated “Last, I figured I would also test spring ERA one more time to see if we can glean anything from it. As expected, the results proved that itâ€™s all noise.” The correlations I posted in a comment above:

Spring to season ERA: 0.035

Spring to Marcel ERA: .02

Comment by Mike Podhorzer — March 27, 2012 @ 10:44 am

This is a good question and would require a lot more work. I am pretty sure the stadiums in Ariz are much more hitter friendly, so park adjusting may help. But i’m not sure the effect they have on just K% and BB%

Comment by Mike Podhorzer — March 27, 2012 @ 10:45 am

I’ll try to get Matt in here to respond to the rest of the math related questions. So much for my college stats class…guess that knowledge disappeared from my brain soon after.

Comment by Mike Podhorzer — March 27, 2012 @ 10:46 am

The standard errors are about .02, so the coefficients’ confidence intervals are something like (.14,.22) for Spring K% and something like (.08,.16) for Spring BB%.

Comment by Matt Swartz — March 27, 2012 @ 11:46 am

Are you concerned about normality?

Comment by Matt Swartz — March 27, 2012 @ 11:47 am

Adjusted R^2 was decidedly up. The p-stats were <.001, so usually that increases Adj R^2.

Comment by Matt Swartz — March 27, 2012 @ 11:49 am

Less interested in the normality of the errors, and more interested in the assumption that the observational data used is i.i.d. – more specifically, interested in statistics which test that assumption.

Comment by Chris — March 27, 2012 @ 5:39 pm

This is a very nice start. I have a couple of suggestions:

1. One potential problem is that essentially you are fitting these observational data and then trying to use them to predict future performances. All these really tell is is that your model fits *these* data quite well, but we don’t really have any sense of how the model plays out going forward. It would be more compelling if you randomly took half your cases to fit the original model and then used the coefficients from that model to predict other half of the cases. We could then see what the R2 would be on this other half to see if it retains its predictive power.

2. It could be the case that once you control for K% BB% becomes insignificant. so you may want to run a model with both included.

3. The magnitude of the BB% coefficient is about 2/3 of the size of the K% variable suggesting that K% has an effect about 50% larger than BB%.

4. Finally, it seems that these effects are relatively small. If I understand your metrics correctly, a 1% increase in K% corresponds to a .18% increase in regular season K% (with Marcel held constant). So a 10% increase in Spring K% corresponds to a 1.8% increase regular season K%.

Despite my suggestions, this is a very interesting first step. Well done!

Comment by Ben Bishin — April 2, 2012 @ 11:13 am

“So a 10% increase in Spring K% corresponds to a 1.8% increase regular season K%.”

A 1.8% increase in expected K rate is huge — whether you are a baseball GM or a fantasy player.

Comment by evo34 — April 7, 2015 @ 5:25 am