CHONE Projections
The CHONE Projections are now up in the player pages, the projections section, and you can get a customized player list of projections using the MyTeam feature.
Huge thanks to Sean Smith for allowing us to post the projections again this year and don’t forget to check out baseballprojection.com to see any player’s projection in more detail.
David Appelman is the creator of FanGraphs.
9
Has any site (or anyone) ever taken the time to compare the accuracy and reliability of various projection systems (CHONE, Marcel, Forcaster, Prospectus, CBS, etc.)? For example, what projection system was the most accurate in predicting the breakout MVP year Jimmy Rollins had in 2007? It would be a tough task indeed but seems like it would be extremely valuable information if some of baseball’s best fantasy sites were compiled and then weighted into a formula that could determine the “margin of error” found in player projections from year to year. It would be interesting to see which systems are best and rank them each year based on accuracy.
Comparing a few of Prince Fielder’s 2008 stats (R:86, 2B:30, HR:34, RBI:102, BB:84, SB:3) to two 2008 projection systems:
Baseball Prospectus 2008: (R:106, 2B:34, HR:38, RBI:111, BB:82, SB:7)
Marcel 2008: (R:86, 2B:33, HR:47, RBI:92, BB:68, SB:5)
Adding the margin of error from each category above yields the following totals:
BP 2008: 43
Marcel: 44
So, in this example, BP was more accurate than Marcel, but not by much. Not bad for a monkey.
There are obviously much better ways to compare these systems and weight their accuracy, but I haven’t seen it anywhere and it would be an interesting resource….
Here:
http://www.baseballprospectus.com/unfiltered/?p=564
http://www.baseballprospectus.com/unfiltered/?p=569
http://www.hardballtimes.com/main/article/so-how-did-tht-projections-do/
And here, which points out some problems with the THT one.
http://www.insidethebook.com/ee/index.php/site/comments/evaluating_the_2008_forecasting_systems/#15
Here are my problems with the CHONE projections:
no hitter has >190 hits
13 players have >= 100 RBIs (08 had 26, 07 had 30, 06 had 33)
10 players have >= 100 runs (08 had 26, 07 had 27, 06 had 34)
16 players have >= 30 HRs (08 had 26, 07 had 23, 06 had 29)
5 players with >200 IP (08 had 31, 07 had 33, 06 had 31)
only 1 player with >15 wins
no pitcher with >200 K
I highly doubt either of these [lack of] stats have ever occured once in MLB history yet CHONE is saying all three will happen this year? give me a break
If your projections are across the board lowly projected, why would you beleive them, and why would you publish them. If some random yahoo published these projections on a message board, theyd be laughed right off the site and dismissed as someone who didnt know the first thing about baseball. I mean you project Ichiro with 188 hits, Dunn 33 HRs, CC 185 K, its just absurd that CHONE is given one ounce of credibility.
I can’t tell you why exactly those numbers come out the way they are (you will have to ask Sean Smith) but it seems to me these are somewhat cherry picked.
Wins, RBIs, Runs, Playing Time: they’re all pretty unpredictable by nature, and knowing that most of the projection systems are based on some sort of regression, a player would have to have a real history of having 15+ win seasons, 100+ RBIs, etc…
CHONE is actually pretty realistic when it comes to real world run environments so, I don’t believe the projections are low across the board.
CHONE also does an excellent if not the best job at predicting rate stats and really no one is telling you to blindly follow any projections. Use them as a guide. You could probably win using any projection system, they’re all within a similar margin of error. This is why we don’t just provide one projection system.
cherry picked??? I used 7 mundane stat categories to demonstate that the CHONE projections were low across the board for everyone. Those happened to be the ones that stood out the most at me right away, as they are the most commonly known stats, Im sure I could go through all the counting stats and find this to be the case.
Upon further investigation I found that CHONE projects no position player to play more than 152 games or no pitcher to start more than 31, thats why all the numbers are low, he has everyone going on the DL at least once.
I think I might multiply his counting stats by a multiple of around 1.1 to account for the lack of projected playing time.
What does “real world run environments” mean? Is that the total runs scored/allowed for everyone is right on par with MLB averages?
I will somewhat agree with Steve here. Failing to see that no player will reach 15 wins and that no player will play over 152 games are signs that a system is flawed. However, it shouldn’t be presumed that projection systems are to be used to really try to predict player performance. They are just tools to be used (and compared against) to try and identify trends and patterns that can then in turn be used to evaluate players.
Macro statistical analysis is pretty accurate. Micro statistical analysis is a lot of luck. In the case of nailing Fielder’s lowered production in 2008 I think the projection systems simply saw a trend and got lucky at the same time.
I’ll take a step back for a second on my “cherry picked” comment. I see what you’re both saying and I guess there’s a couple ways of looking at this.
From a fantasy perspective, you’re basically ranking players based on stats, so as long as the projection system reasonably ranks the players for various stats, then it’s not a big deal if only 13 players reach 100 RBIs as opposed to 28.
All projection systems can only be so accurate anyway, so they’re all more or less “flawed”: http://www.insidethebook.com/ee/index.php/site/comments/forecasters_how_accurate_can_they_possibly_be/
On that note, taking a look at the Wins/Losses in CHONE, they do seem rather conservative and at first glance a bit strange ranking wise, but Wins/Losses are ridiculously unpredictable anyway. This is the first year I think there have been Win/Loss projections for CHONE, I don’t remember them being included last year.
I only included the comment on 15 W and not much more on WL bc I know they are very hard to predict year to year even though a pitchers skill set reamains somewhat consistent.
That being said, Im still tinkering with an exact factor (~1.1) to multiply the counting stats by to make the projections more appealling to me, and I can wiegh the injury risk on each player individually, and adjust accordingly. Obviously if you project a player like chipper jones to play 150 games, his numbers will be amazing but its a fact that he cant stay healthy and for me, I assume hell only play <130 games.
My very first thought of the CHONE numbers was something Bill Simmons said about Hollingers team rating system, something he developed this year. Bills point was, if your system doesnt have the celtics, who were something like 25-2 at the time, as the best team, you need to reword your method, bc obviously they are the best team, to that point in the season.
As we discuss the CHONE numbers, I feel they do a great job of ranking players with respect to one another, just that the overall numbers arent aesthetically pleasing.
Side note, is there a site that compiles the record of past suprising projections? ie when someone projects a player to have a “career year” or to have a very bad season, how often are these right.
Only they weren’t the best team at that point. I’m a regular visitor of Basketball-Reference.com and the Cavaliers have unyieldingly had a better Pythagorean record and SRS than the Celtics since mid-November.