Hank, yeah, I’m with you and Nate Silver on that one (haven’t picked up his book yet, though). It doesn’t make sense to view a p-value of .049 much differently than a 0.051. The 0.05 threshold is indeed arbitrary.

Anyway, if you use the 95% confidence level as your guide, then you can say there’s a about a 5% chance you’ll incorrectly say that a team isn’t legitimately on the level of their preseason expectations. Drawn out over 30 teams, that would mean there’s about a 78.5% chance you’ll come to at least one incorrect conclusion in the bunch.

Like Konoldo says, this isn’t about how likely they are to eventually reach their expectations this season, if that’s unclear. The White Sox have practically no chance of going 39-13 the rest of the way. If their true win rate is 0.494, they’d have a 0.015% chance of doing that; if their true rate is their current 0.367, their chance is 0.000002% (according to the binomial distribution). By the way, Konoldo, I made a simulator last week that’s based on the binomial distribution. I’ll hopefully be sharing that with you all in the near future.

]]>Basically, the way you performed the test is actually better then what I did. It directly tests the hypothesis in question.

Hank – I agree that crossing some arbitrary chosen level of p is silly in many cases (including this one). But p-values themselves give you information on the probability of being correct (or incorrect). I would say, (Using Steve’s correct p-values) there is 99.3% probability they will not hit their preseason expectations.

]]>Angels 0.055

Astros 0.171

Athletics 0.129

Blue Jays 0.183

Braves 0.255

Brewers 0.129

Cardinals 0.127

Cubs 0.447

Diamondbacks 0.704

Dodgers 0.567

Giants 0.127

Indians 0.036

Mariners 0.849

Marlins 0.562

Mets 0.848

Nationals 0.253

Orioles 0.088

Padres 0.850

Phillies 0.184

Pirates 0.023

Rangers 1.000

Rays 0.128

Red Sox 0.090

Reds 0.705

Rockies 0.452

Royals 0.564

Tigers 0.846

Twins 0.846

White Sox 0.007

Yankees 0.849

There’s a discrepancy between mine and Konoldo’s findings on the White Sox… can anybody weigh in on this? I haven’t done any hypothesis testing in way too long…

By the way, everybody, the values listed above [are supposed to] relate to the chances that the expected records are no different from the actual records. So, the Angels, for example, just barely make the typical 95% confidence cut (1 – 0.055 = 94.5%), and therefore we can’t quite say that the Angels aren’t really a 0.556 team with 95% confidence (only with 94.5% confidence…).

]]>For example, take two simple elements of the Jose Reyes Steamer projection:

657 ABs

.342 wOBA

So, if I’m correct in how a projection system is set up (and please please, correct me if I’m wrong)–this indicates that the probability of Reyes exceeding .342 wOBA is 49ish% and the probability of missing .342 is 49ish% with a small chance he hits it exactly.

Likewise, it’s also saying that there is a 49ish% chance Reyes hits 657 ABs and a 49ish% chance he misses.

So to find the offense part of Reyes total production we’d multiply his production times his playing time.

Doing so, we get 4 outcomes with this method, each of equal likelihood:

>.342, >657 ABs

657 ABs

>.342, <657 ABs

<.342, <657 ABs

Now, this would be ok if all players were the same. But veterans get PA benefits more than rookies do in projection systems, right? Except, I suppose, in the Bill James projection system? But we see again and again that younger players–even especially gifted younger players–don't improve linearly. Their performance is, simply, more highly variable.

Plus, are injuries functionally random? Based on history, it seems more likely that Reyes plays 120 or 160 games–think of this probability distribution as almost slightly bimodal. For someone like Prince Fielder, it will likely be a tighter, more normal distribution.

That's why (at least to me) without knowing what the 25th or 75th percentiles for a performance measure (wOBA) AND PAs…these projections aren't really providing a good picture. Which is generally what happens when we just consider the means and not both the means and variances of a data set, right?

]]>What you’re asking for is just the result of adding “Chance of Fewer Wins” and “Chance of Exact Wins,” though.

]]>