Why Fans Do a Worse Job Projecting Their Favorite Team?

Yesterday in my Fan Projections 2010 recap I reported that fans do a worse job projecting players on their favorite team than other fans do at projecting those players. This is an interesting finding: these fans probably have more information about those players, but in spite of that do a worse job projecting them. Why is that?

Commenters suggested a couple of reasons that I will look at here. Samuel E. Giddins writes:

Any chance it’s a selection bias? Only better players get rated more often, and people are less likely to predict a collapse than a breakout?

Samuel is right that better players get ranked more often, but I am not sure this leads to any selection bias. If you look at the graphs from yesterday you can see that across the spectrum players were over-projected, not just the good ones (the blob is below the red line through out, not just for high-WAR players). That people are more likely to predict a breakout than a collapse is probably a fundamental reason for the over projection, but I am not sure that it is a selection bias.

J. Cross writes:

One possible source of bias: what if forecasters who project a lot of non-favorite team players are simply more accurate forecasters than those who only project players on their favorite team

This is an interesting idea, and could very well be possible. I know that David Appelman has the identity of the individual projectors, but all I have is the composite data. Anyway J. Cross could be right that the pool of hometown projectors (I think hometown is a better way to describe this group than favorite-team, which I was using yesterday) might be very different from the pool of non-hometown projectors. Tango suggests a way of addressing this potential bias.

Along the same lines for any player there were more non-hometown projectors than hometown projectors. For my post I look players with at least 10 hometown projectors. This left 206 players who had, on average, 27 hometown projectors and 51 non-hometown projectors. More projectors could mean more accurate projections.

Craig Glaser asks:

I wonder if this might have something to do with playing time projections. When I was looking at some of the fangraphs PT projections I noticed that there were bigger differences there than in rate stats, usually.

Perhaps the fans of the team are more “sure” of who is going to start and award them a lot of PAs and that means that they overestimate playing time on average and that is why they are going over on WAR? … I wonder how this would look using WAR/PA or something else which could account for playing time.

A couple other commenters, Newcomer and AustinRHL, also asked about playing time. The idea being you could over project because you assigned too much playing time or because you assigned too high a rate (or both). It turns out the hometown fans do a better job of predicting playing time than others (RMSE of 186 versus 192 and average absolute error of 133 PAs versus 138). Hometown fans predict a lower number of PAs compared to others (5 PAs fewer on average). But they do a worse job of predicting rates, projecting too high wOBA and UZR/150. It looks like the over-projected rates outweighs the better playing time projections for the hometown fans.

So it seems like it is the opposite of what Craig suggests: hometown fans do well at predicting playing time, but are overly optimistic with rates. This suggests that it would be best to take a hybrid projection of playing time from hometown fans (who probably have a better idea of the depth chart) and rate stats from everyone else.

Theo:

I’d put this down to the simple trait of most (all?) sports fans: optimism. No one wants to see their team’s stars regress, or their prospects not pan out, and will focus more on positives than on negatives.

However, it is very interesting in the context of the argument that fans of a certain team are higher on that team’s players because they know more about their abilities, rather than being biased. With these numbers, it certainly seems that bias is a great influence than any kind of special insight they might have.

My guess is the reason hometown fans come out worse is a combination of this optimism and, maybe, the selection bias in projectors that J. Cross explained.

And for a slightly different topic, Lou Struble asks:

Any chance you’re going to do a post on which team’s fans had the most unrealistic projections from last year? I know it might be difficult because not all the players received enough votes for projections, but it’d be interesting and I’d bet it’d generate a lot of discussion.

Lou is right that there is some much variation in the number of players ranked by at least 10 hometown fans: the Rockies, Marlins and Astros didn’t have any players who reached the cutoff while the Red Sox, Yankees and Mets each had 12 and the Mariners had 17. Here are the 19 teams with at least five players who reached the cutoff ranked by mean error (a measure of over/under projection), but also with the mean absolute error (a measure of accuracy).

Team Players Mean Error Mean Absolute Error
Orioles 10 1.8 2.1
Mariners 17 1.5 1.6
Braves 9 1.4 1.9
Rays 10 1.2 1.7
Phillies 9 1.1 1.6
White Sox 7 1.0 1.6
Cardinals 9 1.0 1.3
Mets 12 1.0 1.6
Giants 9 0.9 1.6
Yankees 12 0.9 1.8
Red Sox 12 0.9 1.6
Rangers 6 0.9 2.3
Twins 8 0.7 1.5
Royals 7 0.7 1.8
Nationals 5 0.6 1.2
Brewers 7 0.2 1.5
Blue Jays 10 0.2 2.2
Reds 10 -0.2 1.1


Print This Post



Dave Allen's other baseball work can be found at Baseball Analysts.


Sort by:   newest | oldest | most voted
bill
Guest
bill
5 years 4 months ago

Very interesting post. I guess it makes a kind of intuitive sense – fans have a better knowledge of depth chart, who regularly gets dinged up, who is injury-prone on their own team than on other teams. And vice versa, they tend to be more optimistic about their own favorite players rate stats than they are about other players.

jfish26101
Member
5 years 4 months ago

As fantasy sports continue to grow, I get the feeling that people often overvalue the players they own (which for some tend to be their favorite players). I think bias and optimism are definitely still at play, but I also feel that could be a small factor. People are competitive by nature, wouldn’t surprise me at all if people are trying to bump the value of players they own on various sites they know their competitors go to in the hopes of bringing back a better package.

Not the biggest or most obvious reason but I’ve seen it and some may even say I’ve done it myself.

Uday
Guest
Uday
5 years 4 months ago

What about the mean absolute error vs. the actual projection systems? Like what if the orioles sucked so much last year that even the computers were fooled? Then you can hardly blame the fans. Make it a % of a composite computer projection and then you could see which fans were more off than the computers.

AustinRHL
Member
AustinRHL
5 years 4 months ago

Yeah, that’s what I immediately think of, too. Unsurprisingly, the teams with lots of players that “overperformed” are all at the bottom of this list while the huge underperformers, like the Mariners and Orioles, are at the top. I’m not sure that there will be meaningful differences at all between different fanbases.

brendan
Guest
brendan
5 years 4 months ago

maybe dave can clarify: does the chart show which teams’ fans were the worst at projecting their hometown team in absolute terms? or does the chart show which teams’ fans had the worst ‘spread’ between the accuracy of their predictions and the predictions of other fans?

I think the second question is more interesting, for the reasons given by Austin. i.e. I want to know which teams’ fans have the most hometown-bias.

Bo
Guest
Bo
5 years 4 months ago

My immediate reaction to seeing that Baltimore fans had the highest mean error leads me to believe that perhaps they all overvalued a single player way too much, which in this case would be Matt Wieters. While your criteria call for the fans to over/under predict at least 5 players, it appears that this happened for more teams than not.

Jason B
Guest
Jason B
5 years 4 months ago

I thought the mean absolute error team-by-team was pretty enlightening too. Just looking at the largest deviations – O’s fans like too optimistic on Wieters, whereas the Rangers and Jays probably had players outperform their preseason expectations, hence both were better in ’10 than most expected.

Kevin
Guest
Kevin
5 years 4 months ago

Am I correct in how I’m interpreting this chart? Were the Reds fans the only fans to underestimate their team’s players while everyone else overestimated? What does that say about the psychology of the Cincinnati fans?

The Nicker
Guest
The Nicker
5 years 4 months ago

It says Cincinnati was the breakout team of the year, so the fans overshot the least. Likwise the O’s and M’s had the most disappointing years so their fans overshot the most.

Actually, without an adjustment to see how much worse each fan’s bases did in comparison to the other projection systems the data is kind of useless.

Ratwar
Member
Ratwar
5 years 4 months ago

I think Samuel E. Giddins is onto something with his selection bias. I think pretty logical that people tend to fill out more projections for their favorite players, and that many people have favorite players on their own team, and that people tend to overrate their favorite players.

I’d also note that in a teams media market, coverage of prospects/stars is almost uniformly positive, making people more likely to predict a break out than if they were in a market that wasn’t being bombarded with such coverage.

I think an interesting idea would be to look at how team fans rate players that recently left their team.

R
Guest
R
5 years 4 months ago

“I’d also note that in a teams media market, coverage of prospects/stars is almost uniformly positive, making people more likely to predict a break out than if they were in a market that wasn’t being bombarded with such coverage.”

Umm, what? What media market is this?

Resolution
Guest
Resolution
5 years 4 months ago

I’d like to shift the selection bias from not only overestimation by people filling out projections for their fav/hometown players, but also people being more likely to project players who they think are going to do something different. Personally, I’m much more interested in filling out a projection for a player that I think is going to take a step forward rather than a player who I think is going to do exactly what he did last season.

Assuming I (like everyone else) is wrong more often than right, this may lead to some inflation – not that everyone is unrealistic about all players, but rather we’re more motivated to fill out projections for those we are most unrealistic about – which may or may not coincide with hometown bias.

Craig Glaser
Guest
Craig Glaser
5 years 4 months ago

Awesome job (in every way except the extra s in my name) Dave. I’m glad that the PT comment helped in some way and I was never sure of which way it would go.

Mike
Guest
Mike
5 years 4 months ago

I’m just spit-balling here, and have absolutely no idea if this is even remotely supported by data, but what if the fans weren’t wrong?

The fans tended to be higher than what actually transpired, according to WAR, which factors in defense. (I think I have this right.) But, we’ve seen that the defensive metrics are still evolving. What if the fans who watch the players everyday have an advantage as far as more accurately evaluating home players defense? Perhaps the difference between “reality” and “fan projections” is accounted for in the fans having a better grasp on the defensive aspect of WAR…?

Theo
Member
Theo
5 years 4 months ago

Talking strictly from a Blue Jays standpoint, it’s pretty hard to say that the overall projections were so good because we’re smarter than fans of, say, the Orioles or Mariners. For the Jays, while Lind and Hill vastly underperformed, so many players overperformed that it all kind of balances out, whereas teams like the Orioles and Mariners both performed well below pre-season expectations, and so you’d expect the projections to have overshot the mark by a lot.

This seems to be a small sample size thing: it’s very easy for players (or even teams) to have outlying seasons that throw off the overall results of studies such as these; even the best projection can be undone by sheer chance and randomness. It’ll be interesting to see how (or if) these things normalize over larger sample sizes.

Lou
Member
Lou
5 years 4 months ago

What about making the fan projections for each player available only after you’ve voted on that player? I’ve caught myself wanting to vote for a certain player in a certain way because I’ve felt that their fan projections seemed low or high. If it’s one of your favorite players on your favorite team and you feel other people have been undervaluing them then you might make your projection more optimistic than you would have without that information.

wpDiscuz