Love it – I’ve always believed that the two things that you can’t fake are raw power (a player who can hit 5 HRs against MLB talent, even in spring training, meets the power threshold) and the ability to strike MLB hitters out. Those skills can change over time, but they are definitely skills, and spring performance can be analyzed for changes in skill.
Matt, not COMPLETLEY random. .02 or .04 is still something. As well, what is the P-value or confidence interval? It is possible that you made a Type II error, no, and that the true values are .1 or .2?
In any case, we expect a much lower correlation with ERA, but since ERA is explicitly related to BB and K rates, of course it is not going to be completely random if K and BB rates are not.
I think also we can infer the ERA correlation, at least an estimate of it, from the K and BB correlations, no? The only thing missing would be the HR correlation, since we know that the BABIP correlation between any two samples, let alone 10 IP of spring training and a whole season, is going to be very near zero.
Oh, and really good stuff, BTW! Although I am not surprised that there is some correlation and predictive effect. Why shouldn’t there be? Sure, pitchers work on stuff, they are not in mid-season form, and their competition is not quite the same as the regular reason, but still, I would think that pitching is pitching. The principal limiting factor in terms of the meaning/value of ST stats is sample size. 10 or 20 IP is not much of a sample to me that predictive regardless of whether it is ST or the regular season. I would love to compare the predictive value/correlation of what you got to the same number of IP during the regular season (of course you have to exclude that sample from the regular season stats). I suspect that the ST stats are a little less predictive than the same number of IP during the reg season for the obvious reasons.
The study did not look at ERA so why is that part of the conclusion?.
The study showed that BB and K rates in ST are somewhat correlated with BB and K rates in regular season. However, most people want to know if there is any meaning in terms of seasons runs allowed (or ERA), and BB and K rates are only part of the equation there.
Furthermore, while the data may be significant for the population of pitchers, it is still rather meaningless for individual pitchers whose results may be driven by weaker competition, low arm strength, working on new pitches/mechanics, etc.
Any chance that adjusting for location (Ariz. vs. Flo.) would help the predictiveness of the spring stats? I’m sure it’s not worth doing exact park factors, but I wonder if K and BB rates differ materially between the two ST leagues.
My long-ago college stats courses taught me that adding any variable will always increase R squared. You need to also look at adjusted R squared and the p-stat of each of the variables separately. (As a start.)
The study did look at ERA, but I didn’t publish the correlations because they were so low. My last sentence though stated “Last, I figured I would also test spring ERA one more time to see if we can glean anything from it. As expected, the results proved that it’s all noise.” The correlations I posted in a comment above:
Spring to season ERA: 0.035
Spring to Marcel ERA: .02
This is a good question and would require a lot more work. I am pretty sure the stadiums in Ariz are much more hitter friendly, so park adjusting may help. But i’m not sure the effect they have on just K% and BB%
The standard errors are about .02, so the coefficients’ confidence intervals are something like (.14,.22) for Spring K% and something like (.08,.16) for Spring BB%.
Comment by Matt Swartz — March 27, 2012 @ 11:46 am
Are you concerned about normality?
Comment by Matt Swartz — March 27, 2012 @ 11:47 am
Adjusted R^2 was decidedly up. The p-stats were <.001, so usually that increases Adj R^2.
Comment by Matt Swartz — March 27, 2012 @ 11:49 am
Less interested in the normality of the errors, and more interested in the assumption that the observational data used is i.i.d. – more specifically, interested in statistics which test that assumption.
This is a very nice start. I have a couple of suggestions:
1. One potential problem is that essentially you are fitting these observational data and then trying to use them to predict future performances. All these really tell is is that your model fits *these* data quite well, but we don’t really have any sense of how the model plays out going forward. It would be more compelling if you randomly took half your cases to fit the original model and then used the coefficients from that model to predict other half of the cases. We could then see what the R2 would be on this other half to see if it retains its predictive power.
2. It could be the case that once you control for K% BB% becomes insignificant. so you may want to run a model with both included.
3. The magnitude of the BB% coefficient is about 2/3 of the size of the K% variable suggesting that K% has an effect about 50% larger than BB%.
4. Finally, it seems that these effects are relatively small. If I understand your metrics correctly, a 1% increase in K% corresponds to a .18% increase in regular season K% (with Marcel held constant). So a 10% increase in Spring K% corresponds to a 1.8% increase regular season K%.
Despite my suggestions, this is a very interesting first step. Well done!