(Note: I noticed a coding issue in the data, which resulted in three parks having a different classification. The data has been re-run to reflect the new results and the article updated to reflect the findings.)

Researchers have gone to great pains to highlight and account for factors outside of an individual player’s control when evaluating their performance and value. The standard for this is of course Voros McCracken’s seminal research into defense independent pitching and Tom Tango’s fielding independent pitching (FIP). While baseball is arguably the most “individualistic” of the major team sports, players do not perform in isolation from each other or from their environment.

Lately I’ve become more interested in how the physical environment of a team and its players affects their outcomes on the field. My initial research led me to look at whether a team’s home park and the degree to which it inflated or suppressed run scoring put the team at a fundamental advantage or disadvantage in terms of winning. The results suggested that hitter-friendly parks do, in fact, put a team at a fundamental disadvantage, likely due to the stress that playing 81 games a year in that environment places on the pitching staff.

In this article, I am concerned with how park factors may affect the various constructs we’ve developed to help us better evaluate a player’s talent and likely performance in the future. Specifically, to what extent to do park factors affect the usefulness of various ERA estimators? It seems reasonable to assume that while much of what happens when a ball is put in play is not controlled by a pitcher. However, given that some extreme parks are likely to exercise their own environmental force over the outcome of batted balls it stands to reason that ERA estimators that factor in a pitcher’s batted ball profile may do a better job in certain types of parks than others.

To test this hypothesis I collected data on all pitchers that threw 40 or more innings in successive seasons from 2002 through 2010 for the same team. For each season pair I collected the pitcher’s ERA, FIP, xFIP, tERA, and SIERA in Year 1 as well as their ERA in Year 2. I then set about calculating the correlation between each Year 1 metric and Year 2 ERA as well as the Root Mean Square Error (RMSE) for each (which is simply a measure for how accurately each predicted measure was to the actual value–the lower the RMSE, the better the predictor).

Now, let’s see what we see:

In aggregate, we find that each of the estimators does a better job of predicting the following year’s ERA than the previous year’s ERA. Each estimator has a higher correlation with YR2_ERA and shows less variance in terms of the RMSE. This is consistent with previous research.

However, when we segment the Year1 and Year2 pairs by type of park we see some interesting differences.

When we look at ERA estimator performance in hitter-friendly parks we find that the advantage of the estimators relative to YR1_ERA generally increases. The easiest way to visualize this is to look at the amount of variance each estimator alone explains (i.e. R-squared, or RSQR in the tables). For all parks, each estimator explains at least 4% more variation in YR2_ERA than YR1_ERA. However, when we focus just on hitter-friendly parks, the difference increases to at least 6% (xFIP) and a maximum of 11% (tERA–more than double the explanatory power of YR1_ERA).

SIERA and tERA do provide additional insight above just YR1_ERA, but FIP manages to outperform every other metric except tERA. That’s interesting, given that tERA and SIERA take batted ball profile into account.

For pitcher-friendly parks, the pattern is more familiar. Compared to all parks, the predictive power of the estimators actually decreases, but the relative ranking of each metric is about what we would expect. What’s more interesting is that a clear advantage emerges for SIERA in terms of R-squared and RMSE relative to both YR1_ERA and all other estimators.

What about neutral parks?

By definition, neutral park pairs make up the largest part of the sample. Compared to all parks, the results look quite similar. In terms of pure correlation, each estimator looks about the same (not surprising, since they are driving the overall sample).

So how should we interpret these results?

Well, first of all, we should note this is a first step. All this study looked at was how ERA and it’s estimators fared when segmented by park type when pitchers threw in the same park in consecutive seasons. It also included both relief pitchers and starting pitchers. Follow on articles in this series will look to segment by pitcher type (starters versus relievers).

Second, estimators that take into account batted ball profile fare do not show a clear advantage in hitter-friendly environments. This goes against my initial hypothesis. While tERA had the strongest showing, plain old FIP did nearly as well as better than SIERA. Since FIP takes into account actual HR/FB it appears able to pick up the effects of playing in a high run-producing environment. At this point, I am not sure why SIERA essentially performed the same as xFIP.

Third, the analysis suggests that different estimators will be more applicable to pitchers with different batted ball profiles. Like park factors, it may be that different estimators do a better job of predicting next year’s ERA for extreme ground ball or flyball pitchers. More importantly, it may also be that combinations of park factors and batted ball profiles lend themselves to some ERA estimators over others.

Fourth, it is not clear from this analysis which is the best estimator to use when trying to get a handle on how a pitcher will perform when moving from one park to another. Originally, I thought that predicting the future performance of someone like Michael Pineda, who is moving from Safeco to Yankees Stadium next year, the safest route would be to rely on estimators that take into account batted ball profile. However, the initial analysis here suggests that while tERA performed the best, relying on FIP might be just as useful. When it comes to pitcher-friendly parks, however, initial SIERA appears to be the best bet. This will also be the focus of a future article.

——————–

*In order to segment the pairs I took 3-year regressed park factors (courtesy of Seamheads) for each season since 2002 and calculated the standard deviation park factor for the league each year. For each season, a park was considered hitter-friendly if its park factor was greater than one standard deviation from the league average for that season. For pitcher-friendly parks, their park factor needed to be less than one standard deviation away from league average. The balance of parks were considered neutral. This coding scheme ensured that, on average, 76% of all parks each year were neutral, 14% were hitter-friendly, and 10% were pitcher-friendly. Season pairs were coded using the park factors from the first year of the pair. The average difference between Year1 and Year2 park factors was only .05 (i.e. a Year1 factor of 95 would have, on average, a Year2 factor of 95.5).