Lex, that’s only true of the random causes of variance. What you’re describing would only be true if baseball were entirely a game of chance – if all players were equally talented, in other words.

]]>.424 +/- 1.28 sqrt(.424(.576)/139, or .370 to .477 .

The advantage of confidence intervals are that as you acquire more data, you are dividing by a larger “n”, so the confidence interval gets tighter; you don’t need a fuzzy “small sample size” disclaimer. However, does a GB% of between 37% and 48% tell us anything meaningful? And if a batter gets 11 hits in his first 50 AB’s, do we really think his true Batting Average is between .145 and .295? More likely he’s just been unlucky so far, and a BA over .300 is probably more likely than one below the Mendoza line for an established or talented hitter. So I think a better approach would employ “regression to the mean”, but also produce a range rather than a point estimate.

]]>I admit that the simple confidence intervals taught in introductory stats classes might not be the complete answer; but I suspect the literature has examples of forecasting a range. In project management, for example, times to complete a project are often estimated using a “best case–most likely–worst case” estimate for each component, weighted 1-4-1.

]]>There are two problems with using that here.

1) PC overstates his case. 50 PA is not the point at which swing % “go[es] from being garbage to being meaningful,” it is the point (rounded to the nearest 50 PA) where swing % goes from having a split-half correlation of .7.

In other words, 50 PA is not some kind of talismanic number where a light bulb suddenly goes on. It is simply the point at which you can say that a guy with a swing % of N, in a league where the average swing % is M, has a “true” swing % of (N+M)/2. Before that point, you can still say that his true level is closer to N than M, and after that point, you can continue to build evidence that his true level is closer to N than (N+M)/2.

2) PC’s research was done in a vaccum, where he was simply looking at a player’s performance in a single season. It is not directly attributable to the question here, which is that N PA tells you when someone’s skill level has **changed**. If you want to determine that, the size of the sample isn’t the only thing that matters — how much the sample differs from expectations matters as well.

Lance Berkman isn’t anywhere near the threshold where his HR/FB rate “stabilizes” (300 PA), yet ZiPS has gone from projecting him to hit 17 HR over 504 PA before the season to projecting 17 HR over 413 remaining PA as of today.

Why? Because Berkman’s performance to date is already statistically significant. He is multiple standard deviations above his projected performance to the point where it is more likely than not that his projection is no longer accurate going forward.

Calculating exactly how much significance his performance to date has — and thus how much to adjust his projection — will relate to similar concepts as “stabilization,” but you cannot simply treat any PA/BF threshold as a magic number where things suddenly become meaningful. Every PA and BF carries some tiny bit of significance, and if they consistently point in the same direction, they can add up to real meaning long before a statistic “stabilizes.”

]]>