It might be the biggest debate in baseball, statistically speaking. We’re well past RBI and pitcher wins, by now. WAR is a big debate, but not so much because of the offensive statistics, or the baserunning figures. WAR is debated largely due to the thing I had in mind when I wrote that first sentence, the one about the biggest debate in baseball, statistically speaking: defense.
There’s still a strong “eye test” contingent. Folks who believe you just can’t put a number on defense. On the other side, there’s a staunch numbers crowd. The crowd that argues, well, you can’t see every play from every defender, and you also can’t ignore or probably even be aware of your own internal biases; I’ll stick with the numbers. Where it gets real tricky is that, even within the numbers-oriented crowd, there’s some skepticism of those very numbers. There’s some concerns with the methodology. Defensive shifts make things extra tough.
So for the most part, we shrug our shoulders and accept that, for as far as these things have come over the years, we’ve still got to do some leg work. If we really want to gain an idea of a player’s defensive ability, we’ve got to just take it all in, and look for clues along the way. What does each defensive metric say? When they agree on one thing or another, we’ve got ourselves a clue. How about errors? They’re not the best, but they’re not worthless. Do they line up with what we saw in the advanced stats? Clue. Check out some spray charts, or Inside Edge. Watch some film, and read some scouting reports. Plenty of clues to be found in there, especially given all you’ve learned along the way. Do all this, and you’ll have a pretty good idea. Even if one number or one play or one quote goes against what you’ve concluded, that’s the point; your body of research holds more weight than that one thing that purports to invalidate your findings.
* * *
Each year, Tom Tango does a fun little project called the Fans Scouting Report. The nature of the project, essentially, is to crowdsource the eye test. There’s plenty of ways to use the data, and I’ve settled on one, for now. I wanted to look for improvement, and I wanted to look for agreement, using both the eye test, and the advanced numbers. I used three sources of defensive metrics (UZR, DRS, FRAA) for fielders with at least 500 innings in 2014, and 2015. I averaged those to get component defensive runs above average figures, and then, I compared against the Fans Scouting Report’s numbers. Using some z-scores, I could come up with an overall ranking of agreed-upon improvement.
Read the rest of this entry »