Wisdom Of The Crowd

Today, David announced the newest addition to the site, and one we’re all pretty excited about it – Fan Projections. We’ve hosted the forecasts of most of the various top projection systems over the last few years, and you’ve probably become accustomed to hearing various writers quote CHONE, ZiPS, or Marcel. With FanGraphs now offering the ability to aggregate projections from various sources, we’ll have a new set of projections to offer this winter – those of the crowd.

If you’ve read James Surowiecki’s “The Wisdom of Crowds”, you’ll no doubt recognize the theory behind the endeavor. As Surowiecki suggests, there is evidence that certain groups of lay people can give better estimates than any single expert, due to the unique experiences we all have in life. By blending our understandings together, we can eliminate some of our individual biases and enhance the shared wisdom of the population.

Tom Tango has done some research on this as it pertains to projections in baseball, and Surowiecki’s theory holds up pretty well. In a four part series that matched up six projection systems against 165 fans, the aggregate projections of the fans was essentially the equal of the complex statistical models. Individual fans by themselves didn’t fare so well, but when all fans were combined into a single projection, they held their own.

This is, essentially, Surowiecki’s argument in a nutshell. We all have our limitations of understanding, but the whole can be greater than the sum of its parts.

So, in that spirit, we offer you the opportunity to make your voice heard. Use the Fan Projections to add your personal wisdom the crowd, based on your insights and experiences. Despite the fact that this is site is statistically inclined, we have a fairly broad base of readers, offering a wide variety of opinions and views – the kind of crowd where the wisdom of many can really shine through. Try projecting some players every day, putting real thought into what you expect from each player in 2010.

If we have the diverse, intelligent crowd that I think reads this site, don’t be too surprised if the Fan Projections end up hanging with the big boys. Let’s put the wisdom of the FanGraphs crowd to use and see what happens.




Print This Post



Dave is a co-founder of USSMariner.com and contributes to the Wall Street Journal.

21 Responses to “Wisdom Of The Crowd”

You can follow any responses to this entry through the RSS 2.0 feed.
  1. JDSussman says:

    Dave,

    I have not read “Wisdom of the Crowds” but is that because the fans tend to have a natural regression in place that is similar to those in the statistical models?

    Vote -1 Vote +1

    • JoeR43 says:

      It’s just essentially the theory that when enough people use simple past performance and observation to judge a player, eventually you get a real consensus on said player.

      And that a sample size of just 165 people can come to about the same conclusion as complex analytical processes, it shows the power of simple objectivity.

      Vote -1 Vote +1

    • philosofool says:

      The phenomenon of the wisdom of the crowds is well known but not well understood. There are many hypotheses about why it works and none are perfect. Some people offer a “cancelling out” explanation: the idea is that our personal biases each other out (for every pessimist, there’s an optimist). I find this explanation partial at best, since it leaves out why the remainder after all our biases cancel tends toward reality.

      Another explanation appeals to the Condorcet Jury Theorem, which is just a theorem of probability mathematics; it says that given a set of independent trials with a greater than .5 chance of getting the right answer to a yes/no question, the probability that the majority of trials is correct gets arbitrarily close to 1 as the number of trials approaches infinity. (There are ways to show that any complex question that isn’t a yes/no question can be reduced to a series of yes/no questions, so the theorem can be applied to a complex question like “How many home runs will Jeter hit?”)

      There are cases where the Wisdom goes completely unwise. Here’s a fun example: if you ask a room full of people to secretly write the number of beans in a jar on a piece of paper, the average answer tends to be very close to the number of beans in the jar. If instead you ask each person, in sequence, to say out loud how many they think are in the jar so everyone hears everyone else’s answer, the average answer usually converges on the first answer that everyone heard. Crowds can be very stupid and sheepish under certain conditions. An explanation of this sort is apt to make the fans scouting report a lot like projection systems because many people will simply look at a few projection systems before offering their forecast. What is interesting is whether enough fans can beat the projection systems by knowing things that the systems don’t, e.g., injury history.

      Vote -1 Vote +1

    • Toffer Peak says:

      The Wikipedia Article has a surprisingly good break down of the phenomenon and the criteria necessary for it to be successful. – http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds

      My biggest concern would be bias (Independence). Since this site already lists Bill James’ projections, the fan’s and in the following weeks, CHONE, ZiPS, etc. a lot of fans will probably be biased by those numbers. To be most effective WOTC predictions need to be “blind”. Unfortunately I don’t think there is any way to realistically fix this. Fortunately those projection systems are pretty good so this variable shouldn’t too negatively affect the results.

      The other concern I have that has been mentioned is ensuring that everyone has good intentions. On the Internet their are always jerks who are willing to ruin a great thing. An “easy” way to correct this would be to put your money where your mouth is. Submissions could cost say $.25-$1 with the person whose projection was closest at the end of the year getting all of the winnings. This would incentivize people to only submit their true beliefs. Of course this would severely reduce the number of submissions as well as be a huge burden on Fangraphs to create such a system.

      Overall I think this is an awesome idea (something that I unfortunately had been hoping to implement myself in the future) and as long as they can control for crazy submissions I think the final results should be quite accurate.

      Vote -1 Vote +1

      • philosofool says:

        When it comes to jerks, the best you can hope for is that jerks washout by contradicting one another or simply get swamped by the vast majority of people that are just giving their best opinion.

        Vote -1 Vote +1

  2. Adam D says:

    Are we not concerned that we fans will rate players on our favorite teams and not on others, therefore inducing the bias you are trying to avoid? As a Rangers fan, I feel I could give good estimations of numbers for next year for my team, and maybe some division foes, but I wouldn’t dare make those estimations of teams, say, in the NL Central with no knowledge of them whatsoever. So, if all Ranger fans (with their bias) estimate numbers for the Rangers only, this does not represent a diverse enough sample group for the crowd theory to work.

    Vote -1 Vote +1

    • Sky Kalkman says:

      My guess is that David and Tom will present not only the raw results of the survey, but some “processed” results. You already have to pick your favorite team in order to rate players, which will present the opportunity for adjustment. Plus you can easily compare the overall stat lines of hitters and pitchers to make sure things even out.

      Vote -1 Vote +1

    • Adam, this is why we ask you to enter your favorite team. Behind the scenes there’s a lot more going on than just aggregating all the projections and I think we’ll probably have a decent sample size for non team fans. You can actually see the breakout of fans/non fans if you go any of the detailed projection pages.

      http://www.fangraphs.com/fanpdetails.aspx?playerid=8370&position=2B

      Vote -1 Vote +1

  3. Jimbo says:

    How much of a crowd do you expect/hope for? 50, 100, 200?

    Vote -1 Vote +1

    • It’d be nice to get at least 100 for each player. FanGraphs has a large enough audience where I think it’ll be pretty doable and we may end up getting a lot more for some of the more popular players.

      Vote -1 Vote +1

  4. tangotiger says:

    In my experience of running the Fans Scouting Report, the Community Playing Time survey, the Clutch Project, and god-knows what else I’ve done, all you need is 20 votes per player. Even 10 votes is pretty good. Things have a tendency to settle down pretty fast.

    And yes, the entire reasoning behind asking for the Favorite team is to handle the issue of home-team bias.

    Vote -1 Vote +1

    • JoeR43 says:

      And so far, even home team doesn’t look all that relevant.
      Examples:
      Pujols, Cards fan vote: .331/.440/.633, 42 HR
      Pujols, other fan vote: .330/.441/.630, 41 HR

      Pedroia, Sox fan vote: .311/.383/.467, 15 HR
      Pedroia, other fan vote: .310/.379/.463, 15 HR

      Ichiro, Mariners vote: .336/.382/.434, 116 R
      Ichiro, other fan vote: .332/.377/.422, 110 R

      I’m sure some guys (most noteably young players and guys in decline) will get some home crowd bias, but so far so good.

      Vote -1 Vote +1

  5. Logan says:

    Dave-

    Challenge time. You and the Fangraphs writers should make your own projections and match them up against ours. I’m guessing you’ll hit the Mariner ones square on the head, but you’ll be too cynical towards the high RBI 1B’s on big market teams, Matt K will have the most accurate figures, but you’ll have to sort through 3×10^872938676209875029879 links to find out what they are, and Carson will have some really long post loaded with humorous observations on his picks, and everyone will bitch and moan about how his writing made them laugh too much.

    …and go!

    Vote -1 Vote +1

  6. Shizane says:

    This is awesome…very user-friendly. One suggestion: can the actual value of the projection show up right under the actual stats (perhaps in a different color) so I don’t have to go to the “view my projections” screen to see the nominal output of my choices?

    Vote -1 Vote +1

  7. Rudy Gamble says:

    Awesome stuff. Will look forward to the inevitable comparisons by Tango of these results vs. the other projected estimates.

    Vote -1 Vote +1

  8. philosofool says:

    The thing I’m finding awesome is the obviously large number of M’s fans that read this site. Jack Wilson was the second SS to receive a projection, after Derek Jeter.

    Vote -1 Vote +1

  9. Patrick says:

    This looks like such fun. I hardly feel qualified to have an opinion on such matters… But it looks like a lot of fun is to be had with this.

    Vote -1 Vote +1

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current day month ye@r *