Fans Scouting Report, Part 2

Following up David’s announcement, let me give you the lowdown on a project that is near and dear to my heart. You know how saberists and statheads are accused of being all about the numbers, and ignoring the human component? This project is the antithesis of that. This is all about the human component.

The idea behind it goes all the way back to the mid-1980s when Bill James in his Baseball Abstracts asked his fans to rate each player by position, 1-30. He compiled their results, and it became the rankings in at least one of his annuals. I was sick and tired of how the media would tell us the fans about how good and bad newly-traded fielders were, when invariably, what they said did not matchup to reality. Really, Kaz Matsui was such a good-fielding shortstop that he could displace the young Jose Reyes? I believed that stuff each time, even though we kept getting confirmation that it wasn’t true.

Put the two things together (James’ crowdsourcing plus distrust of media’s objectivity), and you get The Scouting Report, By the Fans, For the Fans, of which I’ve ran it for the last eight years. I can’t tell you how incredibly insightful you guys are. Well, individually, you aren’t. Indeed, individually, you are as useless as I am, and any other individual. That’s just the way it is. The power is when you get just a few of you guys together. That coalescing is where the real brains of the operations lie. All I do is provide the brawns to bring you guys together under one voice (and remove any obvious party-crashers).

The end result is that you have a bunch of Giants fans evaluate Giants players, and a bunch of Rangers fans evaluate Rangers players, and if a Giants fan is interested in the fielding talent traits of Elvis Andus or Vladimir Guerrero, all he has to do is ask 20 or 50 Rangers fans. He does that asking simply by looking at the results of the Fans Scouting Report.

And, the power of crowdsourcing is really making waves. A few years ago, just days before Opening Day, I started collecting your views on the depth chart of your team (expected games played, expected innings). Once again, rather than try to aggregate from 30 media sources on the latest depth for the 30 teams, I instead pooled the power you guys provide. And you continue to impress me with how much, collectively, you know. Fangraphs last year expanded on that idea even more, by including forecasts of performance as well.

I’ve done here-and-there crowdsourcing on contracts. And Fangraphs has really expanded on that idea too this year. I’ve crowdsourced for favorite movies, most outstanding players, among other things. Really, there is so much that, individually, I would never listen to any single person (myself included, because, after all, who am I?), but that collectively, it trumps anything and everyone out there. It seems like a paradox.

Anyway, so here we are. I’ve provided to Fangraphs the results for the 2009 and 2010 seasons, and am working on preparing the other six seasons back to 2003. There are seven traits that the fans are asked to evaluate, on the idea of capturing the entire spectrum of fielding talents on display (from crack of bat, to last out). Furthermore, by focusing on the particulars, it removes from the fan the impulse to give an overall evaluation. When I compile it, I weight them a certain way to give that overall evaluation. The end-result is a score from 0-100, with 50 as the average. One standard deviation is 20, meaning that for each trait, 16% of the players will exceed a score of 70, and the same number will be worse than 30. Furthermore, I convert those scores to runs, so they are directly comparable to MGL’s UZR and Dewan’s plus/minus.

The pinnacle of sabermetrics is the convergence of performance analysis and scouting observations. And I think that the voice you guys provide, as a group, is part of that convergence.




Print This Post



30 Responses to “Fans Scouting Report, Part 2”

You can follow any responses to this entry through the RSS 2.0 feed.
  1. kbertling353 says:

    The problem is that I know who the numbers say are good fielders. I don’t see how I can be objective.

    Vote -1 Vote +1

    • tangotiger says:

      Have you actually looked at the ballot to fill out? Are you presupposing you can’t be objective, or did you fill out the ballot and then surmised that you weren’t objective?

      Vote -1 Vote +1

  2. Choo says:

    You can be objective without conflicting your favorite statistics. The numbers might say a fielder is X, but not necessarily why a fielder is X. The “why” (instincts, first step, speed, hands, release, strength, accuracy) is answered by the fans.

    It would be cool to see a breakdown of the relative value of each category by position in a way that could meld fan ratings to UZR/150.

    Vote -1 Vote +1

    • tangotiger says:

      Right. I’m not suggesting that it’s completely unbiased. Let’s say that you know that UZR loves Chase Utley, but when you see him, you don’t think he’s hot stuff. And, you would normally give him “3 – Average” across the board, but your knowledge of his UZR might bias some of those numbers to be a “4 – Good.” It’s possible.

      Before Fangraphs started publishing UZR on a week-by-week basis, we had no knowledge of the rookie numbers. Evan Longoria and Ryan Zimmerman had no “numbers”. And Zimm’s numbers weren’t that hot anyway. This did not stop the fans from loving their fielding from year 1.

      So, the complaint being issued is one that you would think might be an issue in terms of predicting how thousands of fans are going to behave voting, but practically speaking, it’s really not that big of an issue.

      Vote -1 Vote +1

      • Brad Johnson says:

        Chase Utley’s a very good example of that. If you watch him for any one game, you have to wonder how he gets by in the field. Watch him for a full season and you’ll still be wondering about how ugly he looks fielding. But the nice thing about the FSR is that it asks you the pertinent questions that lie beneath that ugliness. What is his range (who can say anything but ‘very good’), what is his arm (crappy), how are his hands, etc. etc.

        Vote -1 Vote +1

  3. Xeifrank says:

    Thanks for doing this Tom and Dave. Can’t wait for the leaderboards to have this stuff.

    Couple of questions.

    1. In the past, have you picked a consistent point in the season/pre-season where you collect your FSR data? If not, it might be a good idea to have this once a year measurement taken at the same time period each year.

    2. Any hints on how you convert from SD to runs saved for each position. Is there a different factor involved for a position that has more opportunities? If so, how big of a difference?

    Vote -1 Vote +1

    • jinaz says:

      As a comparison, based on my home brew stuff converting FSR to runs (completed just two nights ago!), I had Pujols at +17/yr in 2009 and +14/yr in 2010. Tango has him as +8 runs each year, and I’m guessing the reason has to do with down-weighting his opportunities b/c he’s a 1B. I could be wrong.

      Also, are the data runs/season, runs/150G, or just runs/inning played? And it’s RAA at their primary position?
      -j

      Vote -1 Vote +1

      • Xeifrank says:

        Jinaz, how many runs saved per standard deviation did you have? I have Pujol’s 2010 FSR at 1.5 SD. The highest was Teixeira at 1.57 SD. Pujols was second.

        Vote -1 Vote +1

      • tangotiger says:

        The runs are for the season, just like UZR and DRS. They are directly comparable.

        The runs data is listed at a position-by-position basis, not “primary” position.

        Vote -1 Vote +1

      • jinaz says:

        Xei, I can’t seem to reply to your question under your question. But I’m using ~10 runs per SD. Tango defines 20 units FSR as one STD, so it works out to ~.5 runs per unit FSR

        Pujols is a 74 at 1B based on the weightings Tango reported in his poll thread a few weeks back, while the position average is 48. So he’s 26 units above average, which would be 13 runs per season (my spreadsheet is saying 14 runs because I’m using slightly more than 10 runs per SD, which was based on distribution of qualified players UZR). Tex comes out as +15 runs/season in my spreadsheet.

        Like I said, I’m not positive why I’m missing high vs Tango. Part of it’s probably opportunities. Part of it’s that Pujols didn’t play a full season (1380 innings).
        -j

        Vote -1 Vote +1

    • tangotiger says:

      Except for year 1, every other year, the project started in mid-to-late August. I pick that point, because I want to capture those guys who were traded by the Aug 1 deadline, and accumulated at least 10 games with their new teams. So, pretty much, I run on a Mon or Tue, around Aug 20something.

      I use a factor of “5/4″ for SS, “4/4″ (or 1) for 2B, 3B, CF, C, “3/4″ for LF, RF, and “2/4″ for 1B. Those numbers are multiplied by some constant, which I think I use is 0.7. And further multiplied by the percentage of time he’s played that position.

      As an example, if someone is an 85 SS, and the average SS is 65, and he has 90% of the innings, then he’s +20 times 0.7 times 5/4 x .90 = +16 runs.

      Something like that.

      Normally, I would have had the C as 6/4 as the weight, but I was getting farout numbers for Molina. Basically, I don’t think the FSR is necessarily doing justice to the catchers, so, rather than have a skewed view, I decided to temper the results.

      Vote -1 Vote +1

      • jinaz says:

        And…that would be why I’m missing. I’m using a smaller constant than Tango, but he’s halving the effect for 1B’s and I’m not.

        Tango, where does the 0.7 coefficient come from? StDev for fielders of 14 runs/20 units? I’ve been using 10.6 runs, which I got empirically last year iirc.
        -j

        Vote -1 Vote +1

      • tangotiger says:

        While each trait is 1 SD = 20, the average of the 7 traits is 1 SD = 15.

        Vote -1 Vote +1

      • jinaz says:

        Ahhh, makes sense. So I guess you lose some variation because of correlations among the different variables. Thanks.
        -j

        Vote -1 Vote +1

  4. VM says:

    Nice to have this all in one place – thanks.

    I was scanning the Rangers numbers and noticed Chris Davis at a 54 for “first few steps” vs David Murphy at 52. Comparatively, Davis is at 44 and Murphy at 55 when it comes to sprint speed.

    And Josh Hamilton’s arm is an 88, while Nelson Cruz’s is an 87.

    This got me thinking… there’s no record anywhere of players 60-yard dash time and/or mph on outfield/infield throws, is there? It would be great if we could see just how accurate the fans are…. Would Chris Davis edge out David Murphy in, say, a 20-yard dash – but then get smoked in a 60? Can Josh really throw ever so slightly harder than Cruz?

    It’d be interesting to know how precisely the fans can really judge speed or arm strength – two things that we could actually measure….

    Vote -1 Vote +1

    • tangotiger says:

      Right, that would be good to have.

      Vote -1 Vote +1

    • Brad Johnson says:

      “I was scanning the Rangers numbers and noticed Chris Davis at a 54 for “first few steps” vs David Murphy at 52. Comparatively, Davis is at 44 and Murphy at 55 when it comes to sprint speed”

      I suspect you’re just seeing a natural bias here. Corner infielders rely much more on that first jab step to either side while in television broadcasts we don’t even see the outfielders start to move. I suspect the fans might unconsciously be over crediting Davis or under crediting Murphy. Or perhaps Davis’ reaction times really are better than Murphy’s. After all, first step doesn’t have too much to do with speed. Not that 54 and 52 are really different.

      Vote -1 Vote +1

  5. Xeifrank says:

    It would be nice if Dave could compute and add a FSR/150 to go along with the FSR raw totals. You could use innings played on defense (div by 8.75) as a proxy for games played then scale the FSR to FSR/150. Just a thought.

    Vote -1 Vote +1

    • tangotiger says:

      The FSR/150 would simply look like the overall 0-100 scale, right? So, I’m not sure that buys you anything.

      Vote -1 Vote +1

      • Xeifrank says:

        It buys you another set of units to compare apples to apples with UZR. The end unit that I like to see is runs saved per game, and to get this I need UZR/150. It is fairly easy to compute yourself, but would be “nice” if it was already done. A number like 85, where the average is 50 tells the average fan very little. Number of runs saved (raw total) is good and so is runs saved per X (X = some unit of games, UZR uses 150).

        Vote -1 Vote +1

      • tangotiger says:

        I disagree Xei. It only means nothing because he hasn’t seen it in action. Once he’s used to it, that he sees an “80″ as a positional leader, then he’s got a good sense.

        I think the UZR/150 is dangerous on any level other than 300+ games. Is +11 runs at 2B positional leader? How about at SS? 1B?

        Vote -1 Vote +1

      • Xeifrank says:

        I think the per 150 is helpful because you are staying in the same realm units wise.

        UZR:
        Runs saved: Units runs
        Runs saved/150: Units runs/X# of games

        FSR
        Runs saved: Units runs
        Overall: Units standard deviation times constant above average.

        FSR needs a runs saved/# of games
        and
        UZR needs a standard deviation times constant above average.

        The second would line up with what you are doing in FSR. As they play on Sesame Street every morning, one of these things doesn’t belong with the other. Units-wise the “overall” of FSR does not belong with the others more so than a FSR/150 number.

        In my opinion there should be all (FSR Runs saved, FSR “overall”, FSR/150). Let people regress the FSR/150 on their own. They are use to doing so with small sample size batting averages etc… Perhaps I am of the minority, but that is just my opinion.

        Vote -1 Vote +1

  6. nucchemist says:

    @tangotiger

    I was wondering if you could help me out. Recently I brought up a question to RiverAve Blues about Cano and his subpar advance fielding stats. Their conclusion was that we as fans usually don’t see the other great fielders in the league on a regular basis and thus can’t adequately assess how the player we watch regularly compares to them. Cano finished 11th in the league in UZR amongst qualifying 2B (19 qualified), but as a fan I rate his defense much higher. Looking at the components of UZR (DPR, RngR and ErrR) this is how Cano ranks: 4th, 18th and 1st. His RngR is comparable to a converted OF (Schumaker) and a converted 3B (Walker) while the other two are comparable to Utley, Ellis and the other elite defensive 2B. This is where the fan in me won’t allow me to swallow this statistic pill. Sure I remember the great diving stops in the hole more than I do the ones that just get under his glove, but watching Cano play, you don’t notice him not getting to balls that he should. That screams to me that he at least league average, which for arguments sake is 0.0. Insert that number into UZR and you got an entirely different view. Instead of a UZR of -0.6, the number would increase to 6.9. With the new FSR, Cano receives a 10, which is comparable with Utley (9) and Ellis (8) and not Schumaker (-9) and Walker (-1). So I guess what I’m getting at is that I believe that there is an outlier in the group, which is causing Cano UZR to rate him as a below average fielder and was wondering what your thoughts on the matter is. Thanks.

    Vote -1 Vote +1

    • tangotiger says:

      There is uncertainty in all the data.

      Aubrey Huff, with a career 6000 PA, has a career .350 wOBA. In 2009, it was .297, and in 2010 it was .388. (Or if you prefer wRC+: 115, 77, 145 respectively.)

      What is he really? Well, no one is going to pay like he was really a 145. And no one believes he’s actually a 77. We’re going to think he’s somewhere in-between, but we’ll have alot of uncertainty.

      I think the fans have alot to add to fielding. No one is going to argue that Cano REALLY put up a -1 UZR in 2010, or whatever it was that he did. It’s more likely that he put up a +5 +/- 6 (i.e., -7 to +11), or something like that. We have a big uncertainty, and anyone who argues that Cano was -1 +/- 0 doesn’t know what he’s talking about.

      Vote -1 Vote +1

  7. Choo says:

    “…but watching Cano play, you don’t notice him not getting to balls that he should.”

    I notice Cano not getting to balls the average 2B would get to all the time, particularly mid-to-short range plays, and primarily sharply-hit balls to his backhand side that require two or three steps at an ideal angle. I’ll say “Jeezus, Cano. Really?” And then I’ll think about Tango’s Fan Scouting report and make a mental note to remember my reaction when I grade Cano’s “acceleration.”

    Same with Jeter. Both players do a lot of things well defensively, but the initial “burst” to cover ground an get on line is not one of them. That’s just one opinion of course.

    Vote -1 Vote +1

    • nucchemist says:

      I understand where you are coming from, but the what the “average player” is going to vary from one fan to another. The average fan probably doesn’t have the access or the time to watch enough games to get a good enough sample size to be able to develop an image of what is average and what is not. That’s why I went the route of comparing players. Where did Cano rank on a scale of Gold Glove to converted OF in terms of range? Certainly not Gold Glove, but definetly not converted OF either. Mostly like slightly below league average, which would jump his UZR up to amongst the better defensive 2B. The FSR backed what I was thinking because thats where the common fan ranked him. Now is FSR necessarily accurate now? Probably not, but I think the FSR will get more accurate over time as now Fangraph readers like you and me have catergories to look for and judge. I’m excited to see where this goes.

      Vote -1 Vote +1

  8. bill says:

    I still have trouble with this because the fans are necessarily biased by what they read/see/hear, and not necessarily great at noticing subtle “nuances” that might make a player better than others.

    As a fan I notice if a player is slow to a ball, but I don’t *really* notice if a player positions himself better than another player (partially because it’s just not possible – TV just does not show where fielders stand on a batter-to-batter basis, unless there’s a shift, and it’s hard to judge based on seeing maybe 10-15 live games a year).

    Vote -1 Vote +1

    • Choo says:

      That’s the most difficult thing to judge: Why was the defender in THAT spot when the ball was hit and can we credit the player for being there? Did the bench coach waive him there? Did he cheat over a step or two after he saw the catcher’s signal and did the pitcher actually hit his spot? Did the hitter’s body language give him a clue? Did the specific base/out state dictate his alignment?

      Vote -1 Vote +1

    • tangotiger says:

      The point here is when someone says (an individual) about his “lying eyes”, then what are we to make of it? Any single person should always be ignored.

      But, get 4 or 5 like that, and you’ve got something. Get 50 of them, and you’ve got even more.

      Rather than leaving it to some anecdotes that we informally collect in a non-systematic fashion, what I have here instead of a systematic way to collect that data.

      Whether it means something is a different question. At the very least, we can now proceed to step B in terms of understanding what “lying eyes” means.

      Vote -1 Vote +1

  9. tangotiger says:

    The reason why the units work is because it puts all fielders at all positions on the same single scale.

    Vote -1 Vote +1

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current day month ye@r *