How Optimistic Are Fans?
Fangraphs shows 220 non-pitchers with Fan forecasts. The total WAR of those players forecasted is 685 wins. With my personal forecasts, I have those exact players at 496 wins.
Seeing that all non-pitchers in 2009 came in at just under 600 wins, and seeing that there were nearly 700 non-pitchers in 2009 (meaning that there are plenty of players still left to be forecasted), my personal forecast probably serves as a reasonable baseline.
On that basis, Fans are expecting 38% more marginal wins than will be generated. That works out so that the average team wins about 94 games.
Hope truly springs eternal, as everyone thinks they are a playoff contender. Not to worry though. At some point, the Fan forecasts will be recalibrated to knock out the optimism, so that the total WAR (nonpitchers + pitchers) will come in around 1000.
How generous are fans compared to other projection systems with regards to playing time?
My thoughts exactly, though CHONE is the only one on this site projecting WAR totals.
CHONE currently has 1367 players forecasted at 379.7 WAR over 489,515 PAs. Those 685 wins came in 125,778 PA. So both tango and CHONE are showing the fans are overly optimistic.
In doing this same sort of project last year for a blog for my favorite team, I noticed the exact same thing. My team’s fans were supremely optimistic about everyone. Fan is, after all, short for “fanatic.” I wonder why we, as fans, do pretty well when evaluating defense — a la the fans’ scouting report — and so poorly with projections. Is it because one is measuring what is or was and the other is measuring what we hope will be? Is it because one is offense and one is defense and because defense has been seen as less important, or less sexy, than offense it’s less consequential if our favorite player is a poor defensive player?
No idea why you would get 2 thumbs down for your comment. Are people disagreeing with your conclusion, or with the content of your post? The content itself is fine. I would have hoped that people would judge the comments based on quality not on whether they agree with it.
Anwyay, the same phenomenon happens with the Fielding Scouting Report I run every year. The range is from 1 to 5, with a mean of 3. But the Fans mean is 3.3. So, I have to recalibrate downward.
It’s about the same range when you think about it. If you were to convert a 1-5 range into a 0-162 range, you would do: (Fans-1)*40.5. So, when the Fans rated fielders as 3.3, that would be the equivalent of…. 93.
See? Same kind of optimism.
Some people can’t resist the “thumb down” button. Poor wretched souls.
On a complete side note, I’m trying out a new comment rating system. Usually I wouldn’t even show a +/- rating until it hit -5 or +5 collectively, but now you can see everything from the very start.
Probably not worth taking any of these seriously until you see at least a cumulative +/- 5.
David, I’d much rather seeing people being asked if the comment is worth reading or not, rather than up/down.
In Time, they have a thing where the reviewer of a book ends with:
- Read / Skim / Toss
That’s how I would think the commenting system should work. The up/down is like Ebert’s system of “Agree / Disagree”.
Ideally, we’d have a page at some point with the top comments, so we can see those posts judged as “must-read”. Lord knows there’s lots of great comments that are simply impossible to follow all the time.
“No idea why you would get 2 thumbs down for your comment. Are people disagreeing with your conclusion, or with the content of your post?”
They just read your article about how overly positive most fans are and they probably felt the need to adjust their expectations, in order to keep the overall ratio of thumbs up to thumbs down at 1. It’s not the the post was bad, but you just talked them into raising their standards. :)
economist.com and nytimes.com both let you recommend a post, so that you can sort by most recommended if you want to. it eliminates the need for thumbs down by sorting out the best posts if the reader so desires.
i agree with tango, the thumbs up/down is ineffective because it becomes a proxy for whether you agree/disagree with the post or the commenter, rather than a means of sorting out the best posts.
I might change it back to the old system, but in general, what comment rating system we use is up to whatever the best the WordPress development community can come up with.
Dinged for hyperbole. VEB commenters are somewhat optimistic about most Cardinals, not supremely optimistic about all Cardinals. See for yourself: http://www.vivaelbirdos.com/2009/2/7/752369/cards-community-war-projec
Why even have a rating system? I’m not sure if I see anything about it that’s suppose to help anyone out.
Yeah a rating system! At least it isn’t like BPro’s where snarky comments are rated below the threshold for viewing (and actually get minimized!) for their snark rather than their content (also weird that BPro doesn’t like snark – I would assume everyone on the internet does).
To be fair, when you’re bored at work, sometimes the ones with a -5 or more are the most fun to read.
Get you riled up because someone probably said something stupid about the FIP methodology or how they see someone play and don’t need UZR or something.
A small part of the problem is the way the stats split. It’s like playing a video game with too few pixels to render a nice image. But that’s only a small part of the problem. I think don might be on to something, I’ve done about 60 projections and The Fans project a remarkably higher playing time than me on about 90% of those. I don’t think injury/collapse risk is being properly weighted by The Fans. If only everyone had projections, then we could see how many games or pa/season would be required to accommodate the projections.
An extreme example of the first point is Adam Dunn. The worst UZR We can choose is -18. I happen to think Dunn will repeat his butchery in the -25 to -30 range. That’s 7 to 12 runs I can’t access in my assessment.
I had the same thought as your first point. For example, there was a player I was rating who I thought would hit exactly 10 home runs. I chose the 10-14 category, since it fell in there, but the way that splits it looks I projected 12 home runs instead.
Of course, as you said, it’s only part of the problem. I’m almost certainly guilty of extreme optimism as well, as my projections for home-town Chipper Jones at nearly 6 WAR coming off a season where he posted 2.8. But of course, that followed two seasons where he was over 7, so I can’t bring myself to change it.
Playing time probably has a lot to do with it. There is a lot of wiggle room in a span of 120-149 games. When I choose that option I am usually thinking in the 130-135 range. I’d be curious to see the WAR/gm or WAR/100 numbers for the fans as well as for tango and see if the spread is still almost 20%.
Not that we wouldn’t see the same thing if you allowed everyone to specify an exact projection, ie 147 games, 588 ab, 27 HR, .268 average, etc. There is definitely a selection bias at work as well.
What’s the likelihood those who read this post will lower their expectations on future fan forecasts? If they are all recalibrated at some point, the ones that occur after this post may receive an undue decrease.
I had to adjust my process upwards after projecting John Lannon to be out of the major leagues (probably still sore about him breaking Utley’s hand :) and Reyes Beltran to rack up just over 800 PA combined.
David captures a timestamp for every forecast. So, we’d be able to see the differences in post pre-Jan 19 and post-jan 19.
I wonder how much of an impact health has on this problem as playing time directly impacts WAR because fans are overly optimistic about how healthy players will be as well as well they will perform (they also, understandably, omit the unexpected injuries that hit players every year).
How many projected PAs are coming from the fans? Perhaps that is something that needs to be normalized before anything else.
There are two things to normalize:
1 – Make sure that league PA for nonpitchers totals around 180,000 PA (probably make a better case for 170,000 PA, to account for the 5% PA that goes to players that are not on anyone’s radar). And, if need be, further normalize it by team.
2 – Make sure that the population average is used, rather than last year’s average.
Once you do that, you don’t have to worry about too much more.
You *could* try to ensure that the total WAR for each team matches the Vegas forecasts, so that if a team is way too optimistic, they are cut down more than a team that is pessmistic.
Fun stuff…
The thumbs up thumbs down i guess doesnt apply to older posts even though it was an option?
But while fans are always going to be optimistic about their teams more than most, i would argue that the extremity can go both ways. Players that were sucktacular and not fan favorites get hit extra hard in 2010 (probably partly because of generous PT assumptions which might be harmful for a below replacement level player).
I’d also say there is some confirmation bias. People who think Yuniesky Betancourt is really the man won’t stick around here for long, and all that is left are the haters (read: 99% of us here), hence he gets hammered extra hard.
Anyway, i don’t think it’s so one sided, but perhaps i am biased (I only had chase for 7.2 WAR, which was below what my fellow Philadelphians seemed to think.
as an aside, i know i had offered a loser league (draft only, no lineup setting or roster moves post draft) with traditional roto categories. If I did one that was WAR based would that be more fun for people?
i’d like to get 9-19 other likeminded dorks on a sunday and bang this out.
I’d like to join that league, I’ve never been in one before. If you do end up starting that league and you need managers, my email is chadam189@gmail.com. Thanks!
Expanding on what the unrepentant gunner touched on above, with regards to his Yuni example:
I wonder if using the “wisdom of the crowds” to collect projections is very effective since we only have the submissions of FanGraphs readers. 99% of the people here are sabr-minded, and we view baseball very similarly. Most of us also follow the authors here, and if those authors have an opinion on a certain player (Yuni was a great example), the people who submit projections are more likely to be too negative in his projections. I know that after reading Dave’s disdain for Yuni on USSM, I find that I don’t like him as a player, and it probably would taint my projection on Yuni if I had made one because I don’t like him. Do you guys think we’re getting a wide enough range of opinions?
I agree with those who have brought up the playing time issue. Of that 38% marginal difference, what amount is due to increased playing time and what is due to superior performance?
I actually like the Bill James approach. With 10 players projected to reach 600 at bats (in an injury-free season), you know there will be some missed time. Let’s say 1 of the ten only gets 100 ab.
Do you:
A – project all ten @ 600 ab, since you don’t know which player will lose time
B – reduce the ab to 550 for all 10
“A” gives you a better idea of what each player is capable of, in a vaccuum. You’ll be closer to the right answer 90% of the time. Option “B” will sell short the (majority) who actually acheive their ab projection.
Of course, if you assume anyone with good health is going to play a full season…then you’ll automatically overstate their production. I’d rather have that than the constant reconciling of “conservative” stats versus assumed playing time.
Yes, it is “pick your poison”. I would think of it in terms of a Vegas over/under on PA with even odds. With B, you’d just take the over every time and be right 8 or 9 out of 10 bets. With “A”, you are more likely to get a 50/50 split on the over/under outcomes.
vr, Xei
I’ve been trolling the projections in preparation for my fantasy season, and I can say that in general the fans are optimistic on playing time, more Runs and RBIs, and rate statistics (OBP, SLG) for most batters. None of the fan projections seem unreasonable for any given player, but taken in aggregate they’re off. Personally, I think it’s hard to estimate the weighted average of a set of possible outcomes; instead the fan projections look to me like a likely outcome, but one that has not been dragged down by the long left-hand side of the distribution – filled with injuries, regression, and the vagaries of playing time.
This would probably need to be looked at over the longer term, but I would be extremely interested to see how fan optimism in terms of projecting players is affected by overall team success (or other factors).
It’d be interesting to see which teams have overvalued players.
It’d also be interesting to see how free agents do compared to signed players.
As a Mariner fan, I noticed that Franklin Gutierrez, Ichiro, and Chone Figgins are significantly lower than where they were last year. Also remaining free agents like Orlando Hudson, Felipe Lopez, and Johnny Damon are fairly low.
Injuries have to be huge in this overstatement, right? I mean, how many people projected Peavy’s ’09 (or Beltran’s or Reyes or any Mets’). When you’re not sure who is going to get injured, you don’t randomly “project” for them. A lot of those injuries are replaced with near-replacement level talent, which would account for the overstatement.
It’s also hard since the fans in aggregate don’t project the same way as something like CHONE. I could project maybe 10% of players (the ones I am excited about and optimistic about) but if I were to project every one, it would probably be more reasonable than the small sample I chose.
Plus, playing time is hard to project anyway for some players.
It’s been been touched on to a degree already, but to continue the thought, I think the main problem isn’t so much the playing time issue as much as that each person predicts a likely outcome for each player – which limits the projections to a pretty limited range. The chances of any given player having a huge collapse is pretty small…but some do every year. Thus, the optimistic projects(at least to a degree).
If you are asking the fans for projections, why would you adjust them? It would be nice to see how the fans fare against the so-called experts.
It might not be that the fans are optimistic about any given player, but it might be that fans only rate players they are interested in, which translates to star players and good prospects. For example out of 220 non-pitchers rated by the fans only 1 has a negative WAR, CHONE has 657 non-pitchers with a negative WAR. Therefore the fans aggregate WAR is not weighed down by sub-replacement level players, most likely because there is less interest in rating them.
If the fans actually rated all of the players, rather than just highly touted prospects and stars, we might actually have an aggregate WAR closer to what is expected.
I’m not saying it wouldn’t be overly optimistic to some degree, given the issues with playing time, but it would probably be less optimistic on the whole.
I didn’t notice before I posted, but I’m really just agreeing with Bill above.
Actually, I baseline the exact same players against my WAR for those players. So, it’s not a question of the sample of players being above average. That’s been accounted for.
***
As for adjusting them: suppose the fans come out with an average UZR of +3 runs. Well, we know what the average UZR is going to be: 0. Should we leave it at +3 because that’s what fans said it would be?
That’s not exactly what we’re saying…
The implication was, fans will tend to rate players they are more optimistic about, and ignore players they are less optimistic about. As everyone’s sample isn’t the same (it’s not a survey of the same players, it’s basically a survey of whatever players the rater wants), there will be a tendency to bias upward. As in, it’s not that all fans rate the same players, but rather fans tend to view the players they choose to rate more favorably. If it were a paid or forced exercise (not saying it should be, obviously), the ratings might be toned down. If I were *forced* to rate every player on say, the Royals (a team I don’t follow very closely, with the exception of Zack Greinke) I’d probably rate them less favorably than players I already liked.
Professional projections are more likely to underrate better players, making the fans more accurate in my opinion. The good players will be the ones projections are already done for (this group).
Also injuries have to be mostly projected not to happen. So that most people will meet fan projections less the injury time. Ignoring injuries, I’d take fan projects over any other system for good hitters. Professional systems way over-regress toward the mean.
On fault I’ve noticed is unrealistic babips for pitchers. Many are given babips around .350 which is “unsustainable”
It’s the blowout seasons, like those of K Greene and Burrell, that mop-up all the excess WAR. It’s somewhat unlikely that any given player with 3+ war potential will suddenly be replacement level for a season, so no one predicts that.
I agree. We shouldn’t underproject all players by a win to make up for the wasted seasons of the Glaus types.
Glaus rightly should have been projected several WAR, but was worth none (or very little.)
So taking a half win away from Cliff Lee to get the total WAR accurate isn’t worth a damn.
What about situations when a player moves to a potentially more favorable situation. Take Roy Halladay for instance. He gets a ton more run support, no dh, etc etc (fans will overate this for sure because of the dream situation). And the replacement in Toronto (I don’t know who it is) is probably a young, high-upside pitcher. Locals will likely overrate him based on homer-mentality and possibly local hype vs more realistic models.
Would it be possible to produce some stats on the sit that were redistributed/adjusted to reflect reality? So we can see how close the fans are at the end of the season?
There a lot of players who end up with negative hitting WAR. If you look at the past team totals, you will see that every single team had negative WAR from their hitters (more so than pitchers, but true there also). You must take into account all the negative hitting WAR from pitchers, all the negative hitting WAR from major leaguers, and all the negative hitting WAR from AAA players. It is difficult to impossible to project where the negative WAR will come from, but you must allow for it, as it is real and tangible. I think the “Fans” are allowed to go over the so-called “limit” due to this fact.
vr, Xei
I’ve yet to make any projections, but I promise to be very pessimistic.
I expect that the over-projection is 2/3-3/4 playing time and the remainder performance issues (and if you took out the home fans, the projected performance would probably be in a reasonable zone).
When I projected players, I sometimes would forecast a serious injury based on history and what I knew of the player. And you do have to be realistic, a 29 year old second baseman who has played 145, 153, 155, and 157 games the last 4 years ought to be projected in the 120-149 game range, but most fans will have him in the 150+.
i’d like to see projection overstatement by team. i know brewer fans would like to reconcile some opinions held regarding the fans of certain other nl central teams.