FanGraphs Baseball - Comments on Reviewing the Preseason Standings Projections
RSS feed for comments on this post.

## Leave a comment

Line and paragraph breaks automatic, e-mail address never displayed, HTML allowed: `<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> `

0.164 Powered by **WordPress**

“They’ve only won 51 out of 110 games; a legit 0.556 team would be expected to win somewhere between 48 and 54 games out of 110 somewhere around 9.8% of the time…”

I absolutely don’t know why you decided to present it in this way. Why not go with the standard statistical convention, which is to report the %chance that a .556 team would win less than or equal to 51 games? Then it’s comparable to standard hypothesis testing.

Comment by Roger — August 5, 2013 @ 3:12 pm

Roger, it’s a confidence interval, which is, to me at least, a much better way of presenting the data than a 1 -tailed hypothesis test.

Comment by Steven Ellingson — August 5, 2013 @ 3:16 pm

Nevermind, I read that wrong. You’re right, that is an odd way of presenting the data. A confidence interval would be centered around the expected value (61), not the actual value (51). I’ve never seen anyone present data in this way before.

Comment by Steven Ellingson — August 5, 2013 @ 3:21 pm

i agree, i’d like to see which preseason hypotheses we can reject with 95% confidence.

Comment by kdm628496 — August 5, 2013 @ 3:30 pm

Yeah, I thought this was an interesting take on it… maybe not. I thought the chance that a win total would be somewhere in the current vicinity would be a relatable concept.

What you’re asking for is just the result of adding “Chance of Fewer Wins” and “Chance of Exact Wins,” though.

Comment by Steve Staude. — August 5, 2013 @ 3:35 pm

Using today’s numbers and using Chi-square tests of independence (no continuity correction) on 2×2 tables for Expected Wins and Losses by Actual Wins and Losses, none of the current preseason expectations can be rejected with a 95% level of confidence. However the White Sox are awfully close with a p-value of 0.0555 (Expected Record:54-55, Actual Record: 40-69).

Comment by Konoldo — August 5, 2013 @ 3:44 pm

Awesome, thank you.

Comment by Steve Staude. — August 5, 2013 @ 3:54 pm

The question I have is how projection systems consider both PAs and Performance. Are they set up so the mean projection for both is the spit out stat line?

For example, take two simple elements of the Jose Reyes Steamer projection:

657 ABs

.342 wOBA

So, if I’m correct in how a projection system is set up (and please please, correct me if I’m wrong)–this indicates that the probability of Reyes exceeding .342 wOBA is 49ish% and the probability of missing .342 is 49ish% with a small chance he hits it exactly.

Likewise, it’s also saying that there is a 49ish% chance Reyes hits 657 ABs and a 49ish% chance he misses.

So to find the offense part of Reyes total production we’d multiply his production times his playing time.

Doing so, we get 4 outcomes with this method, each of equal likelihood:

>.342, >657 ABs

657 ABs

>.342, <657 ABs

<.342, <657 ABs

Now, this would be ok if all players were the same. But veterans get PA benefits more than rookies do in projection systems, right? Except, I suppose, in the Bill James projection system? But we see again and again that younger players–even especially gifted younger players–don't improve linearly. Their performance is, simply, more highly variable.

Plus, are injuries functionally random? Based on history, it seems more likely that Reyes plays 120 or 160 games–think of this probability distribution as almost slightly bimodal. For someone like Prince Fielder, it will likely be a tighter, more normal distribution.

That's why (at least to me) without knowing what the 25th or 75th percentiles for a performance measure (wOBA) AND PAs…these projections aren't really providing a good picture. Which is generally what happens when we just consider the means and not both the means and variances of a data set, right?

Comment by RA — August 5, 2013 @ 4:29 pm

i think you have your means and medians confused. the 50th percentile is the median, not the mean. i can’t speak to how projection systems arrive at their numbers, but i can make you more confused by putting forth the possibility that they actually show the mode, or most likely outcome, after some sort of monte carlo simulation.

Comment by kdm628496 — August 5, 2013 @ 4:40 pm

I just performed my own chi-square tests on the rounded off expected and current wins and losses and here’s what I came up with:

Angels 0.055

Astros 0.171

Athletics 0.129

Blue Jays 0.183

Braves 0.255

Brewers 0.129

Cardinals 0.127

Cubs 0.447

Diamondbacks 0.704

Dodgers 0.567

Giants 0.127

Indians 0.036

Mariners 0.849

Marlins 0.562

Mets 0.848

Nationals 0.253

Orioles 0.088

Padres 0.850

Phillies 0.184

Pirates 0.023

Rangers 1.000

Rays 0.128

Red Sox 0.090

Reds 0.705

Rockies 0.452

Royals 0.564

Tigers 0.846

Twins 0.846

White Sox 0.007

Yankees 0.849

There’s a discrepancy between mine and Konoldo’s findings on the White Sox… can anybody weigh in on this? I haven’t done any hypothesis testing in way too long…

By the way, everybody, the values listed above [are supposed to] relate to the chances that the expected records are no different from the actual records. So, the Angels, for example, just barely make the typical 95% confidence cut (1 – 0.055 = 94.5%), and therefore we can’t quite say that the Angels aren’t really a 0.556 team with 95% confidence (only with 94.5% confidence…).

Comment by Steve Staude. — August 5, 2013 @ 8:38 pm

you should read the new book by nate silver, the signal and the noise. confidence levels and rejection thresholds are bunk. don’t talk about almost crossing an arbitrarily chosen p=.05, talk about being more or less likely. it’s highly likely the white sox will not hit their preseason expectations.

Comment by Hank — August 6, 2013 @ 9:37 am

Steve – After doing a little research, I replicated your results in excel. The difference between our results was you used the chi-square goodness of fit test directly testing against expected win rate. I used 2×2 contingency tables to test whether there was difference in the two groups (actual vs. expected). The null hypothesis assumes both actual and expected comes from the same distribution and calculates an expected 2×2 table. It compares the original 2×2 with the expected 2×2 table. Sorry if this isn’t making much sense.

Basically, the way you performed the test is actually better then what I did. It directly tests the hypothesis in question.

Hank – I agree that crossing some arbitrary chosen level of p is silly in many cases (including this one). But p-values themselves give you information on the probability of being correct (or incorrect). I would say, (Using Steve’s correct p-values) there is 99.3% probability they will not hit their preseason expectations.

Comment by Konoldo — August 6, 2013 @ 11:53 am

Sorry correction on my last statement to Hank. Steve said it better in his comment. There is 99.3% chance that White Sox are not a .494 team. There is even a higher chance that they will not meet there preseason win-loss record given that we are 110 games into the season and they would need to go 39-13 in there last 52 games to reach .494 winning percentage. You could probably do some kind of simulations to estimate the probability of that occurring.

Comment by Konoldo — August 6, 2013 @ 12:07 pm

Phew, thanks Konoldo.

Hank, yeah, I’m with you and Nate Silver on that one (haven’t picked up his book yet, though). It doesn’t make sense to view a p-value of .049 much differently than a 0.051. The 0.05 threshold is indeed arbitrary.

Anyway, if you use the 95% confidence level as your guide, then you can say there’s a about a 5% chance you’ll incorrectly say that a team isn’t legitimately on the level of their preseason expectations. Drawn out over 30 teams, that would mean there’s about a 78.5% chance you’ll come to at least one incorrect conclusion in the bunch.

Like Konoldo says, this isn’t about how likely they are to eventually reach their expectations this season, if that’s unclear. The White Sox have practically no chance of going 39-13 the rest of the way. If their true win rate is 0.494, they’d have a 0.015% chance of doing that; if their true rate is their current 0.367, their chance is 0.000002% (according to the binomial distribution). By the way, Konoldo, I made a simulator last week that’s based on the binomial distribution. I’ll hopefully be sharing that with you all in the near future.

Comment by Steve Staude. — August 6, 2013 @ 2:09 pm