Lee

For instance, it’s not uncommon for a fielder to post a +5 UZR one year and a -5 UZR the next. It’s also not uncommon for a hitter to give you 45 batting RAR one year and 55 the next. Because of the way the two stats are scaled, the difference in UZR jumps out at you way more than the difference in batting RAR does.

Quick question: has anyone looked into UZR and park factors? For instance, if you put Adam Dunn in Fenway’s left field, would his UZR still be historically bad?

So, there is a reason NOT to suspect bias: he handles the parameters in an intelligent fashion.

And, even by some reason you want to pull out of thin air that there is bias, how much of an effect can there by, once you handled all the big ones? One run?

We have no way of knowing how biased UZR is at the moment. It may have very little bias in it which would mean we have a reliable and unbiased measure – the gold standard in statistics. Or it could be reliable and very biased which would be simply fools gold.

“Vote -1 Vote +1DC Stack says:
October 3, 2009 at 5:44 pm

Wow! Major “my bad” on my part. This is what happens when you get too cocky about your own abilities to discern fact from fiction. Right after I read your reply I realized I was doing what so many amateurs do. I was making a statement of fact based on a hand selected few players. I knew at that point I needed to come back and say I was wrong. But before doing that I wanted to see if I was wrong in my methods but right in my conclusions. Well I was wrong in both.

I decided to do a simple test about the serial reliability of UZR. I used UZR/150 for all eligible players from 2008 and 2009. There were 83 players eligible from both years. I ran a simple correlation between their 2008 and 2009 numbers. It came back with a fat Pearson’s r of .729. I could already feel the egg on my face. I then wanted to see if this is bigger/smaller/same as other stats that are far less controversial. I did the same procedure for OPS (unadjusted). The correlation came back in the low .5s. Not only is UZR consistent from year to year, it is more consistent than OPS – at least in the two years I looked at.

HUGE CAVEAT: This is a very small sample size. To get a better feel for the reliability of this metric this really should be done across multiple years. My analysis took me about 15 minutes and that was all I was willing to dedicate to it. If someone wants to take an hour or so to do the full proper tests I say go for it.”

Second, even after that correction, changing the sample of players fundamentally changes the analysis. An increasing correlation could be the result of increasing sample size, or it could be the group of players that play frequently are simply more consistent than those that play infrequently.

He should actually do it for UZR/150, since that’s a rate stat like wOBA. No biggie.

Anyway, they both show an r-squared of .24 (roughly r=.50) when UZR chances is a minimum of 150 (I’d like to know the mean) and when PA for wOBA is a minimum of 300 (again, I’d like to know the mean). Say the mean is 250 UZR chances and 450 PA. THAT IS THE EQUIVALENCY.

That is, whatever you think of someone with 450 PA, that’s how you think of someone with 250 UZR chances. They are equally reliable.

I also can’t believe the 450 PA (or whatever it would be). Historically r=.50 when PA is roughly 200 to 300. Perhaps it’s just this year (2008 to 2009) that shows the inconsistency. Obviously, we really need to do this for many years.

When I’ve done the correlation, I get r=.50 for UZR at a mean of around 100 games. (And, as I said about 50 games for wOBA.)

That’s the equivalency that I’ve always been using.