The Relative Importance of Fielding Metrics

Ultimate Zone Rating, created by Mitchel Lichtman several years ago and available free of charge on this very site, is a fantastic achievement in the world of baseball analytics, quantifying the logical intuition of what constitutes good or bad fielding by measuring the BIP-outs conversion rate in various different zones for each position. The data even breaks down into several different components, evaluating a player’s arm, ability to turn double plays, his ability to prevent runs by not making errors, as well as his range. One area of the data that gets overlooked far too frequently, though, that deserves to be mentioned more often, is that UZR calculates fielding abilities relative to the league average.

Sure, many of us know and understand that a +5 UZR means five runs better than the average player at that position, but grasping that the data is relative should force us to ask a secondary question upon glancing at a player’s results: did the league itself get better?

For instance, if a player turned 50% of the balls hit his way in a particular zone into outs in Year #1, when the league converted 40%, and then held stagnant at 50% in Year #2, when the league increased its conversion rate to 50%, the player didn’t change but his UZR would decrease. His actual overall ability to convert outs in that zone did not erode in any way, shape or form, per se, but his skills no longer looked as shiny because the talent level of the league at this specific position increased.

I like to refer to this as The Jimmy Rollins Conundrum, when his UZR marks hovered right around the league average in 2002 and 2005 despite playing a very, very solid shortstop. It could very well be that the eyes of myself and many other Phillies fans deceived us, with our scouting overrating Rollins’ fielding, but it seems that very few ever wonder if the fluctuations for a player in a given season are direct results of an improved league. Looking at Rollins in 2005, it is a bit tough to tell why the UZR fell to 0.8 from 4-5 runs above average the previous two years, before increasing to 6-7 runs over the next two seasons.

His double play runs were down as were his runs prevented by not making errors, but his overall number of errors in opportunities were consistent with the sandwiching seasons. Add in that his range was identical and that he would revert back to previously established norms in the double play and error runs departments and it stands to reason that perhaps one major reason for the lower UZR dealt with shortstops across the league improving in this area. I’m not suggesting this is the only reason, as we have seen fielders have down years before for one reason or another, but rather shedding light on a question we should be asking when looking at these numbers.

SO, moral of the story: don’t always assume that an increase or decrease in UZR is solely on the player, as the skill level of the league may have something to do with fluctuations as well.

Print This Post

Eric is an accountant and statistical analyst from Philadelphia. He also covers the Phillies at Phillies Nation and can be found here on Twitter.

13 Responses to “The Relative Importance of Fielding Metrics”

You can follow any responses to this entry through the RSS 2.0 feed.
  1. Garrett says:

    Interesting stuff — I admit I don’t know as much as I should about fielding metrics, but wouldn’t a league-wide shift that large in one year suggest something’s missing from UZR? I can’t imagine any other offensive or pitching stat swinging wildly enough to make a pitcher’s league adjusted numbers change so greatly when his base numbers are practically the same.

    Or is it because its position-comparisoned meaning a few better or worse shortstops would change the baseline by that much?

    Vote -1 Vote +1

    • Eric Seidman says:

      Second one. If a few SS played much better or much worse, converting more outs on balls in specific zones, the league itself would convert a higher percentage, meaning that a +5 the year before might be a +2 now, even though the fielder himself isn’t doing anything differently.

      Vote -1 Vote +1

  2. Bill says:

    I love the UZR stat and defensive stats available here in general. Great work guys. Wish we had historical ratings for all of our all time faves. Would love to see Robbie’s (Alomar) UZR in his hey-day.

    Vote -1 Vote +1

    • Kincaid says:

      Sean Smith’s Total Zone is designed to emulate advanced fielding metrics with only PBP data, so there are ratings back to the mid-50s or so, although they are not as precise as UZR (TZ is published at Baseball-Reference). Colin Wyers has also worked on a metric that uses only traditional stats so pre-Retrosheet years can be calculated as well, but I think it’s still under development.

      Vote -1 Vote +1

  3. BenJ says:

    This is why John Dewan calculates his Plus/Minus numbers based only on a single season of data. Both guys had the same idea in creating their systems, with a few subtle differences.

    Vote -1 Vote +1

    • Colin Wyers says:

      That exacerabates the problem, BenJ. If we want to know who was a better fielder when comparing across seasons, we may not always be interested in who was the better fielder relative to average; we may want to know who was the better fielder relative to the other. Without adjustment, we can’t necessarily do that with Dewan +/- (or UZR), although for short time periods we shouldn’t see a significant difference.

      Vote -1 Vote +1

  4. Steve C says:

    If wishes were fishes we would have a large enough sample of data to create a replacement level UZR to compare fielders to as opposed to average. That would probablt remove some of the relative noise.

    Vote -1 Vote +1

    • Eric Seidman says:

      This is assuming you consider the relativity to be a problem. I don’t, at all, because that’s what we want to know – quality in relation to the league. I merely wanted to highlight that when players fluctuate, it might not always be solely based on their own skills or results; the league could also improve/weaken and we need to take this into account. Would it be great to know that Rollins had an absolute +5 or higher in every season? Yes, but it’s probably more important to know that in a couple of years he was only a run or so above average, perhaps because the league improved.

      Vote -1 Vote +1

  5. KJOK says:

    So, is there some way we can tell whether or not this is actually the case? I’m somewhat skeptical that quality would change that drastically in any one season.

    Vote -1 Vote +1

    • Colin Wyers says:

      Sure there is. We can look at all players who played in consecutive seasons and see what the average change was. Here’s a weighted average in the change of UZR_150 for players who played in consecutive seasons, compared to the previous year:

      Year AVG_DIFF
      2003 0.37
      2004 -1.07
      2005 0.30
      2006 0.64
      2007 0.74
      2008 -0.01

      I’m not seeing any evidence of a Jimmy Rollins Effect here.

      Vote -1 Vote +1

  6. Sean says:

    1) do we have minor league UZR?

    2) I think it’s generally perceived that major league level hitting is more rare than major league level fielding. Is this true? How rare is the MVP-level hitter/fielder? Average hitter/fielder?

    Vote -1 Vote +1

  7. MGL says:

    The baseline for each “bucket” (for example, hard hit ground balls hit within 5 feet of the second base bag) is based on 4 years of data. That is to smooth out the fluctuations you get because some of those “buckets” have so few balls in them (small sample size).

    However, once everyone’s UZR is calculated relative to those 4 year samples, each year is “zero’d out” per position. So, as Eric says, a player’s UZR is always relative to the pool of players in both leagues for that year only.

    I did not have to do that. I would have left everyone’s UZR as relative to the 4 years that I use as the baselines for each bucket. That actually would be much better if the data did not have biases in some years as compared to others. But I am afraid it does, so that I am forced to make a “Sophie’s Choice” (not quite as serious of course) and I chose the zero everything out per year. Remember that each position is zero’d out so that for a “Rollins effect” to exist for a particular SS (like, say, Rollins) only the SS overall talent has to change from one year to another. And that is easy to do because we are only dealing with one position out of 7 or so.

    And it is certainly possible that one can estimate the relative strength of each position from year to year. One can do as Colin suggests for each position, making the appropriate age adjustments of course (some positions lose as much as 1 to 2 runs a year per player per season with age). One can also look at who arrives and who leaves. For example, if for SS, every player had the same number of chances in year X+1 as in year X, and nothing else changed, we could assume that the SS position got worse overall since everyone aged by a year. Let’s say that Jeter retired after this year and some slick fielding SS from the minors “took his place” and those were the only changes. Other than the aforementioned age considerations for all the other players, we could infer that the SS position got better. Etc.

    So, yes, Eric makes a very good point that one should be aware that all UZRs you see are relative to league average (both NL and AL combined) at each position separately. Because of that, one has to be careful when comparing players from different years and even when combining years for the same player.

    We actually have the same problem with other metrics, but the chances of an entire league changing from one year to the next are not nearly as great as one position or another. On the other hand, how do you think the AL got so much better than the NL, pitching and hitting-wise, just a few years ago? What I mean is that even overall league-wide hitting and pitching talent can change fairly significantly from year to year.

    Just to make one more thing clear. You are going to see much more random fluctuation on a player’s UZR from year to year than you will from a change in league-wide talent, so when you see a player with +5, +3, +7, -6, while you can probably infer a league-wide talent increase in that -6 year, on the average, don’t assume THAT much of a talent change. Most of that change is likely to be random fluctuation, secondly, some of that change is likely a change in that player’s true talent, and lastly, the smallest amount of that change is likely to be a change in league-wide talent at that position (an increase in this case).

    Vote -1 Vote +1

  8. I very delighted to find this internet internet site on bing, just what I was looking for : D besides saved to bookmarks .

    Vote -1 Vote +1

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current day month ye@r *