I will say I largely agree with mcbrown’s points though Chad does make good points as well. I am naturally skeptical of anyone who claims to have an ability to outperform FIP at this point. However, I will say that for fantasy purposes, team defense (such as the Rangers and now Angels here) should be factored into any decision in a positive favor for the pitcher. When playing fantasy baseball, I do not care if Wilson is getting good fortune by the strength of his teammates gloves as long as it figure to keep happening.

]]>We’re not even considering defense here. The entire Rangers staff outperformed by 0.24 runs in 2010 and 0.19 runs in 2011. The Angels so far have outperformed by 0.33 runs. Part of that is attributable to Wilson himself, but remove Wilson and the teams still outperformed overall. How much of the outperformance is attributable to the pitchers and how much to the defense? If we’re going to claim a given pitcher has this skill, we definitely have to factor this in.

]]>As you say, survivor bias is a massive problem with the historical data in this kind of analysis. To account for it will require a much more rigorous analysis than I have the time to do, and may well be impossible as we have no reasonable way on estimating the performance of people who fell out of the data set.

]]>In 2011, the standard deviation of ERA-FIP among all pitchers who threw 100 innings or more was 0.59. So let’s estimate the implied probability of outperforming FIP by 0.2 runs as 0.367, assuming this metric is normally distributed. The probability of outperforming by 0.2 runs for 3 consecutive years is 0.05, and so we should expect something like 3-7 pitchers to do this by luck in a given three year period.

While we’re on the subject of rigor, we’re not factoring in the inherent bias in the data. Starters that have outperformed will tend to be more likely to continue starting in the future, while starters that underperform will tend to be less likely to continue starting and thus disappear from the data set. Thus we should expect the number of pitchers who display this historical tendency to be overrepresented in the data.

My point remains. This may indeed be a real skill, but to differentiate between who has the skill and who has just been lucky, with any reasonable confidence, requires more than three years of data.

]]>For example, my dataset has only 22 pitchers who qualified in at least 6 of the past 7 years. Out of that set, if we used a 50-50 shot to be over or under FIP, we would expect 1.5% of pitchers to go 6 years beating their FIP by at least .00001. I have 2 of 22 beating it every year by at least .2. I’d say that at least for those two (Lilly, Santana) there is something going on there beyond pure chance and statistics. There is some selection bias, as pitcher’s performing worse than their FIP are less likely to keep pitching.

But, again, my point isn’t that Wilson is definitively better than his FIP, just that I think we can’t assume he is NOT definitively better than his FIP.

]]>