I am constantly hearing how UZR is an unreliable way to measure fielders because there are “wild fluctuations” from year to year. I decided to put that to the test doing an apples to apples comparison of UZR from 2008 to the nearly completed 2009 season. I only compared players that played the same position and had the qualified number of UZR Chances. Here are the results:
more than 50 chances in both years = r^2 of .15
more than 100 chances in both years = r^2 of .19
more than 150 chances in both years = r^2 of .24
more than 200 chances in both years = r^2 of .28
Quite simply, the more of a fielding sample we have for a particular player, the great the correlation from year to year.
For comparisons sake, if we look at wOBA from 2008 to 2009 you get this:
more than 300 PA in both years = r^2 of .24
more than 500 PA in both years = r^2 of .30
So the lesson is, when there’s not a lot of UZR data on a player, there will be a lot of noise, but as the sample size increases, the data (at least from 2008 to 2009) actually becomes almost as highly correlated year to year as the stats that are considered to be the most reliable.