How To Build A Hall of Fame*

ESPN just started a new project, Hall of 100, and the basic intention is to create a list of the 100 best players regardless of steroid use. It’s right there in the introductory sentence: “With some big PED-era names facing judgment day next month in the Baseball Hall of Fame voting and with the everlasting cacophony over who belongs in Cooperstown and who doesn’t, we decided to take a fresh look at the greats of the game.” The idea is to remove the asterisk.

Of course, lists of any kind are pretty much SEO gold, so ESPN is not doing this out of the goodness of their hearts. They’re doing it because it’s clickbait, and they can own the hashtag #Hallof100, and so on. But the methodology is quite interesting, and I’m fascinated by Halls of Fame, from the mythology to the difficulty of expressing something subjective — greatness — using objective criteria. Greatness is subjective because it has to do with how we feel about a player; it is separate from a simple ordinal statistical ranking, which you could call “bestness.” So what’s the best way to make a Hall of Fame?

As I wrote a couple of weeks ago regarding lists of worst owners, I’m interested in understanding intellectually defensible reasons for coming up with rankings, as opposed to the bankrupt clickbait that populates much of the internet. I’m also fascinated by the Hall of Fame itself, the logjam of deserving players that contrasts with the wide variety of patently undeserving players already enshrined, the Rabbit Maranvilles and High Pockets Kellys staring out at the Tim Raineses and Bobby Griches looking in.

So what is an appropriate methodology to follow? You need to account for both qualitative and quantitative qualifications, both greatness and bestness. (You will note that I am mapping qualitative data to subjective description, and quantitative data to objective description. I have done that because, while there may be such a thing as objective qualitative data, it is surpassingly hard to measure objectively.)

In his book The Politics of Glory (now known as “Whatever Happened to the Hall of Fame?”), Bill James developed several metrics that he believed measured a player’s worthiness for the Hall given the apparent criteria for induction, as well as their likelihood of getting in. He also developed a list of questions, the “Keltner List,” to help a voter decide whether a player deserved entry or, like Ken Keltner, happened to be a good player who simply did not deserve it.

The Keltner List consists of 15 questions designed to get at the player’s greatness on his own teams, during pennant runs, at his position, in the league, relative to all other Hall-eligible players, and as measured by All-Star and awards voting. It also includes an important qualitative grab-bag question: “Is there any evidence to suggest that the player was significantly better or worse than is suggested by his statistics?”

The questions combine with the metrics to make a case for a player that is both qualitative and quantitative. Baseball Think Factory’s Hall of Merit works very similarly. This is how Joe Dimino set out his criteria:

As far as criteria, numbers aren’t everything: there are things we cannot account for in the numbers. But since we have them, we are going to make them available to help you with your ballots. Players’ contributions on the field are to be the main criteria for selection; off-field actions should only be taken into account for the effect they had on the players’ teams on the field of play. The language may be tweaked so as many people are comfortable with the criteria as possible. We want to make the criteria reasonably broad so that each voter is able to interpret them according to his own tastes.

ESPN’s methodology, partly developed by Baseball Think Factory’s Dan Szymborski, has a similar idea.

First, they developed a modified WAR called GAR (Greatness Above Replacement) that overweights a player’s peak value by giving them extra credit for their best years, and picked the 150 best hitters and pitchers in history by that measure. And it is calculated by position, where replacement value is calculated as the average of the 20th to the 30th best player at each position. Then, they asked a panel of ESPN baseball experts to assign them scores on a 100-point scale, and players were ranked according to average score.

Any Hall of Fame has to have both qualitative and quantitative criteria. At my blog Braves Journal, we’ve had Keltner Lists for years, to analyze the candidacies of retired Braves from Dale Murphy to Kenny Lofton. And because the Roll Hall of Fame just announced its newest inductees, from Public Enemy to Heart, I thought it would be fun to develop a Rock Keltner List and see whether Devo is Hall-worthy. (Spoiler: I love Devo, but they’re not.)

ESPN’s list is timed to coincide with Barry Bonds and Roger Clemens’s debuts on the Hall ballot, as many writers have proclaimed that they will snub them to punish them for their drug use. And they are surrounded by several other admitted or suspected PED users, including Sammy Sosa, Rafael Palmeiro, Mark McGwire, Mike Piazza, and Jeff Bagwell, who was undoubtedly punished by some voters last year despite no evidence that he ever used a banned substance.

Of course, neither WAR nor GAR can discount for possible PED use. We have no statistical metrics that can account for how usage of banned substances affects player performance, and there are plenty of skeptics who are uncertain that they have a measurable effect. So accounting for it is purely left to the qualitative decisions of the voters. Moreover, as David Schoenfield explained during the chat about the list, PEDs were “Not a factor. Voters were told to only consider on-field accomplishments.” That’s even more restrictive than the Hall of Merit’s criteria.

I don’t think that players should be disqualified for using a banned substance, but I don’t think it should be ignored, either. As I wrote a year and a half ago, “Ultimately, drug users are people too, and when someone connected to PEDs retires, their entire career should be considered, including but not exclusively focusing on their alleged PED usage.” There’s a lot of moral sanctimony around “cheating,” but that doesn’t mean that it’s immaterial, just that there should be explicit standards.

The Hall of Fame is ultimately what you make of it. As Bill James writes in his book on the Hall of Fame, “This is how the Hall of Fame argument progresses: cacophony, leading to confusion… I’m not trying to serve any candidate for the Hall of Fame. I’m trying to serve the argument itself.”

I respect ESPN’s efforts to create a greatest-players list that moves past the poisonous steroid controversy that clogs up all current Hall of Fame arguments, and I respect Dan Szymborski, David Schoenfield, and the other lead writers who developed it. Obviously, there’s no best way to do it: there’s no way to come up with an objective best methodology for a subjective ranking.

ESPN’s Hall of 100 was clearly conceived as a response to the steroid ballot, and so its methodology instructed voters to absolutely ignore PED usage and solely focus on on-field accomplishments. As a result of that focus, they didn’t just rank steroid users, they also ranked Pete Rose #37 and put Joe Jackson on their Honorable Mention list at #102 overall. You can quibble with the overall rankings — should #10 Honus Wagner really be behind #8 Stan Musial? — but they aren’t outlandish. (Unlike, say, the Hall of Fame candidacy of Freddie Lindstrom.)

There’s a Hall of Fame in Cooperstown, and there’s a Hall in each of our heads. None of us will support the exact same list of players. As long as we can outline the criteria we use, we’ll be able to defend our choices. Even if we won’t be able to convince everyone else of their merits, we’ll avoid cacophony and confusion.