How To Build A Hall of Fame*

ESPN just started a new project, Hall of 100, and the basic intention is to create a list of the 100 best players regardless of steroid use. It’s right there in the introductory sentence: “With some big PED-era names facing judgment day next month in the Baseball Hall of Fame voting and with the everlasting cacophony over who belongs in Cooperstown and who doesn’t, we decided to take a fresh look at the greats of the game.” The idea is to remove the asterisk.

Of course, lists of any kind are pretty much SEO gold, so ESPN is not doing this out of the goodness of their hearts. They’re doing it because it’s clickbait, and they can own the hashtag #Hallof100, and so on. But the methodology is quite interesting, and I’m fascinated by Halls of Fame, from the mythology to the difficulty of expressing something subjective — greatness — using objective criteria. Greatness is subjective because it has to do with how we feel about a player; it is separate from a simple ordinal statistical ranking, which you could call “bestness.” So what’s the best way to make a Hall of Fame?

As I wrote a couple of weeks ago regarding lists of worst owners, I’m interested in understanding intellectually defensible reasons for coming up with rankings, as opposed to the bankrupt clickbait that populates much of the internet. I’m also fascinated by the Hall of Fame itself, the logjam of deserving players that contrasts with the wide variety of patently undeserving players already enshrined, the Rabbit Maranvilles and High Pockets Kellys staring out at the Tim Raineses and Bobby Griches looking in.

So what is an appropriate methodology to follow? You need to account for both qualitative and quantitative qualifications, both greatness and bestness. (You will note that I am mapping qualitative data to subjective description, and quantitative data to objective description. I have done that because, while there may be such a thing as objective qualitative data, it is surpassingly hard to measure objectively.)

In his book The Politics of Glory (now known as “Whatever Happened to the Hall of Fame?”), Bill James developed several metrics that he believed measured a player’s worthiness for the Hall given the apparent criteria for induction, as well as their likelihood of getting in. He also developed a list of questions, the “Keltner List,” to help a voter decide whether a player deserved entry or, like Ken Keltner, happened to be a good player who simply did not deserve it.

The Keltner List consists of 15 questions designed to get at the player’s greatness on his own teams, during pennant runs, at his position, in the league, relative to all other Hall-eligible players, and as measured by All-Star and awards voting. It also includes an important qualitative grab-bag question: “Is there any evidence to suggest that the player was significantly better or worse than is suggested by his statistics?”

The questions combine with the metrics to make a case for a player that is both qualitative and quantitative. Baseball Think Factory’s Hall of Merit works very similarly. This is how Joe Dimino set out his criteria:

As far as criteria, numbers aren’t everything: there are things we cannot account for in the numbers. But since we have them, we are going to make them available to help you with your ballots. Players’ contributions on the field are to be the main criteria for selection; off-field actions should only be taken into account for the effect they had on the players’ teams on the field of play. The language may be tweaked so as many people are comfortable with the criteria as possible. We want to make the criteria reasonably broad so that each voter is able to interpret them according to his own tastes.

ESPN’s methodology, partly developed by Baseball Think Factory’s Dan Szymborski, has a similar idea.

First, they developed a modified WAR called GAR (Greatness Above Replacement) that overweights a player’s peak value by giving them extra credit for their best years, and picked the 150 best hitters and pitchers in history by that measure. And it is calculated by position, where replacement value is calculated as the average of the 20th to the 30th best player at each position. Then, they asked a panel of ESPN baseball experts to assign them scores on a 100-point scale, and players were ranked according to average score.

Any Hall of Fame has to have both qualitative and quantitative criteria. At my blog Braves Journal, we’ve had Keltner Lists for years, to analyze the candidacies of retired Braves from Dale Murphy to Kenny Lofton. And because the Roll Hall of Fame just announced its newest inductees, from Public Enemy to Heart, I thought it would be fun to develop a Rock Keltner List and see whether Devo is Hall-worthy. (Spoiler: I love Devo, but they’re not.)

ESPN’s list is timed to coincide with Barry Bonds and Roger Clemens’s debuts on the Hall ballot, as many writers have proclaimed that they will snub them to punish them for their drug use. And they are surrounded by several other admitted or suspected PED users, including Sammy Sosa, Rafael Palmeiro, Mark McGwire, Mike Piazza, and Jeff Bagwell, who was undoubtedly punished by some voters last year despite no evidence that he ever used a banned substance.

Of course, neither WAR nor GAR can discount for possible PED use. We have no statistical metrics that can account for how usage of banned substances affects player performance, and there are plenty of skeptics who are uncertain that they have a measurable effect. So accounting for it is purely left to the qualitative decisions of the voters. Moreover, as David Schoenfield explained during the chat about the list, PEDs were “Not a factor. Voters were told to only consider on-field accomplishments.” That’s even more restrictive than the Hall of Merit’s criteria.

I don’t think that players should be disqualified for using a banned substance, but I don’t think it should be ignored, either. As I wrote a year and a half ago, “Ultimately, drug users are people too, and when someone connected to PEDs retires, their entire career should be considered, including but not exclusively focusing on their alleged PED usage.” There’s a lot of moral sanctimony around “cheating,” but that doesn’t mean that it’s immaterial, just that there should be explicit standards.

The Hall of Fame is ultimately what you make of it. As Bill James writes in his book on the Hall of Fame, “This is how the Hall of Fame argument progresses: cacophony, leading to confusion… I’m not trying to serve any candidate for the Hall of Fame. I’m trying to serve the argument itself.”

I respect ESPN’s efforts to create a greatest-players list that moves past the poisonous steroid controversy that clogs up all current Hall of Fame arguments, and I respect Dan Szymborski, David Schoenfield, and the other lead writers who developed it. Obviously, there’s no best way to do it: there’s no way to come up with an objective best methodology for a subjective ranking.

ESPN’s Hall of 100 was clearly conceived as a response to the steroid ballot, and so its methodology instructed voters to absolutely ignore PED usage and solely focus on on-field accomplishments. As a result of that focus, they didn’t just rank steroid users, they also ranked Pete Rose #37 and put Joe Jackson on their Honorable Mention list at #102 overall. You can quibble with the overall rankings — should #10 Honus Wagner really be behind #8 Stan Musial? — but they aren’t outlandish. (Unlike, say, the Hall of Fame candidacy of Freddie Lindstrom.)

There’s a Hall of Fame in Cooperstown, and there’s a Hall in each of our heads. None of us will support the exact same list of players. As long as we can outline the criteria we use, we’ll be able to defend our choices. Even if we won’t be able to convince everyone else of their merits, we’ll avoid cacophony and confusion.



Print This Post



Alex is a writer for FanGraphs and The Hardball Times, and is a product manager for The Washington Post. Follow him on Twitter @alexremington.


Sort by:   newest | oldest | most voted
Ryan Braun
Guest
Ryan Braun
3 years 7 months ago

My advice: always use FedEx and get tested on Friday so the sample sits over the weekend.

HMK
Guest
HMK
3 years 7 months ago

do you post this on every article that might be related to PED’s? like your piss, ryan, it’s getting stale.

Bear Grylls
Guest
Bear Grylls
3 years 7 months ago

I’ll be the judge of that.

tired of advertisements
Guest
tired of advertisements
3 years 7 months ago

I appreciate the thoughtful insight.. but I feel like this was just a plug for espn.. maybe not, but with fangraphs fairly recent and growing connection I couldn’t help but wonder..

Jason
Guest
Jason
3 years 7 months ago

Its going to be really interesting to see how this develops over the next several years. I don’t want to be a PED’s apologist, but the Hall itself is going to start losing legitimacy given the inconsistency involved in its selection process which is likely to result in many of the all time greats on the outside looking in. Ultimately I think the Hall has more at stake in terms of its historical legacy than does Bonds or Clemens. Everyone knows Babe Ruth was great, even though very few of us have been to the hall of fame. The same will be true of Bonds & Clemens in 50 years regardless of their Hall of Fame fates.

Izzy Hechkoff
Guest
Izzy Hechkoff
3 years 7 months ago

Putting Jeter over Collins is pretty stupid, unless you make huge deductions for quality of competition, which given their other rankings, they didn’t do. Collins is closer to the top 5 than he is to Derek Jeter.

algionfriddo
Member
algionfriddo
3 years 7 months ago

“…see whether Devo is Hall-worthy. (Spoiler: I love Devo, but they’re not.)”

Since opened up the digression… I agree, Devo in not. But where the hell are the Moody Blues?

Personally, Rock Raines was the greatest leadoff hitter in history not named Rickey Henderson. And why was Ted Simmons given such short shrift?

Neelsen Ratings
Guest
Neelsen Ratings
3 years 7 months ago

I am probably the only one in the room for whom this is the case, but when I saw “Devo” my most immediate thought was of Devon White and I was puzzled. Moments later my hand had a rather abrupt encounter with my face.

Jon L.
Guest
Jon L.
3 years 7 months ago

I still think it’s Devon White.

My echo and bunnymen
Guest
3 years 7 months ago

Heart doesn’t deserve to be in the Hall either, but whatever.

My echo and bunnymen
Guest
3 years 7 months ago

besteveralbums.com may not be a perfect statistical website but they do good at rating which albums are the best of a band.
Ftr they have the album you listed as their best, and Duty now for the Future.
You have great taste in music and yes Talking Heads were among the best new wave bands, though I’d sumbit strong competition from The Cure.

My echo and bunnymen
Guest
3 years 7 months ago

Yeah me too, on The Cure. I lumped them in with new wave because they so frequently get thrown in there. I thought you were one of those who may throw them in there, they do have songs that can be considered such. I don’t use IMDB, but if your argument is used then why isn’t there a more blurry middle? I won’t say it’s perfect, but that’s a weak argument. I also submit that the best albums on besteveralbums are OK Computer, Dark Side of the Moon, and Revolver. If you wish to keep going down the list, than so be it, but those are some of the best CDs and they are all deserving of that title. Even if my love of The Beatles is more like than love.

My echo and bunnymen
Guest
3 years 7 months ago

Oh and besteveralbums is not a website who’s point totals are based on a critics average score. They take lists of great albums from Critics (and to a far lesser extent users) and assign point values for where the CD placed. I’m not sure how to word it so hopefully the following makes sense. Bestever uses a baynesian average where the more lists that have a certain CD on it, the higher the value of the votes for that CD. There’s probably a weakness to that calculation somewhere, but it’s not what you described.

My echo and bunnymen
Guest
3 years 7 months ago

Major evidence? Wouldn’t the odds say that the best albums be from different decades, being new doesn’t matter as far as the greatness of the bum and Radiohead has been releasing albums for almost 20 years now, I think it’s safe to say that present is now able to be tagged to some of their albums such as The Bends and OK Computer which I assume are two of the albums you were talking about. The other, Kid A, is 12 years old. It’s more like someone looking back at the end of ballplayers career and realizing that they were and should be placed among the greats, This is It is 11 years. In Rainbows, Funeral, The Suburbs do qualify as present but the list is pretty well represented by all decades 1960s forward, with one 1950s album by Miles Davis. Since albums didn’t start becoming an art themselves until around the 1960s.

My echo and bunnymen
Guest
3 years 7 months ago

You can make art objective as long you understand the difference between a talented band and a band you enjoy.

My echo and bunnymen
Guest
3 years 7 months ago

Also the breakdown of decades of music is as follows for the websites, you may find it at ten bottom of the top 10 webpage. 50s 1% 60s 10%, 70s 24%, 80s 16%, 90s 22%, 00s 22%, 10s 5%.

My echo and bunnymen
Guest
3 years 7 months ago

Heart is about the same, in terms of musical excellence, compared to Devo. Neither one of them had great records, for me great is either in, A. the top 10 in any given year according to besteveralbums.com, or above at least 2,000.
Think MCR’s The Black Parade.

My echo and bunnymen
Guest
3 years 7 months ago

Or Adele 21 is a closer match to the 2000 point total. Greatness is subjective but at the same, it’s sad that music is even worse at figuring out the best artists. When Arcade Fire and Radiohead are darkhorses to win Album of the Year in music’s biggest awards event, something wrong. I’m glad they get it write once in awhile, for Arcade Fire only once. But it’s still sad.

Baltar
Guest
Baltar
3 years 7 months ago

I love Heart, but they don’t belong in the Hall. Neither do 90% of the bands/individuals who are in there. That Hall is way too big.

Cidron
Member
Cidron
3 years 7 months ago

The MLB Hall of Fame may ultimately have some of the same flaws as the Rock and Roll HOF (earlier mentioned) has.. Who is in, and who is not in.

MLB probably wont have the Hit King (Rose), Homer King (Bonds), Cy Young King (Clemens), only player with 60+ homers in three seasons (Sosa), one of four with 500 homer/3000 hits (palmiero), one of two with 160+ RBI (again, sosa).. In short, alot of the kings/record holders (and most certainly, above average players) arent there..

Rock and Roll HOF lacks Jan & Dean, Peter Paul and Mary, Deep Purple, Thin Lizzy, KISS, Iron Maiden and many more.. maybe not the biggest names to you all, but very influential in what became today’s rock.

If MLB takes the same eventual route, not bringing in those that may belong, but .. arent quite “palatable” to the voters, the hall will lose credibility alltogether.

My echo and bunnymen
Guest
3 years 7 months ago

Iron Maiden is excluded, wow, that’s disgraceful.

Ken
Guest
Ken
3 years 7 months ago

What’s disgraceful is that the phrase “Iron Maiden” causes me to think of Keanu Reeves several second before it causes me to think of any actual music.

Bill and Ted
Guest
3 years 7 months ago

Banananear!

bstar
Guest
bstar
3 years 7 months ago

That comment above, echo, pretty much excludes you from talking anymore about music and greatness.

My echo and bunnymen
Guest
3 years 7 months ago

Haha, at least the first movie was funny.

monkey business
Member
monkey business
3 years 7 months ago

Walter Johnson at 12? In 1918 and 1919 he had an ERA- of 46 and 47, both lower than Mariano Rivera’s career average of 49, and he pitched 600 innings in those two years. We are talking about a starter that pitched like the best closer ever for complete games every third day. In those two years he put in 1/2 of the IP Mo put in for his career but maintained a similar ERA-. How is he 12th?

What about Clemens? Well, he pitched 1,000 fewer innings and had a career ERA- three points higher. There are some differences where Clemens comes out better, like Clemens was more recent and won a pitching award many times that didn’t exist when Johnson pitched.

Cidron
Member
Cidron
3 years 7 months ago

comparing era’s always had its pitfalls. In this case, and in defense of Clemens, he pitched in a) the liveball era, and b) right thru the steroid era, and in alot of the newer smaller stadiums. Not only that, but there was a philosophical difference. Pitchers were pitching in a five man rotation, as well as not usually ending their games (even though Clemens did it alot).

monkey business
Member
monkey business
3 years 7 months ago

I complete agree that ERA cannot be compared across years, an especially across decades. That’s why I quoted ERA-.

Interestingly, I basically nailed David Schoenfield’s reason for preferring Clemens too: more CYA, more recent.

B N
Guest
B N
3 years 7 months ago

I also have no idea how Clemens ranks as the #1 pitcher. I grew up as a Red Sox fan and The Rocket wasn’t even the best pitcher I’ve seen (on a per-inning basis). Obviously Pedro had that distinction. Clemens’ greatness was in being great AND very durable.

Combine that with the obvious steroid use and he’s somehow the best of all time? As the durability might have been partially artificial, there are good reasons to partially discount his numbers from 96 or 97 on. There’s no doubt he belongs on the list somewhere, but Walter Johnson has superior numbers and no such question marks. Heck, I put more weight on peak performance than longevity also, and Clemens is much farther down the list when you look at ERA+ rather than WAR also. In that light, Clemens could be even farther down the list.

rubesandbabes
Guest
rubesandbabes
3 years 7 months ago

Eckersley is way too low on the ESPN list.

Kraftwerk is most definitely not one of the most influential bands of all time.

My echo and bunnymen
Guest
3 years 7 months ago

Transfer Europe Express and their early showing to the New Wave party says otherwise.

YABooble
Guest
YABooble
3 years 7 months ago

Eckersley is definitely the equivalent of Aerosmith – showed major chops in the early years, fizzled mid-career due to substance abuse, but returned in a different role as a mega-star.

And on the new-wave front, New Order is the equivalent of Bruce Sutter. They turned a sour start into a great early career by dusting off an out-of-favor specialty (white dance music and the forkball, respectively). So they should get some props as well.

My echo and bunnymen
Guest
3 years 7 months ago

Sour start indeed…

TKDC
Guest
TKDC
3 years 7 months ago

“Voters were told to only consider on-field accomplishments.”

Yet you will never convince me that Joe DiMaggio makes #21 if he did not:

a) play for the Yankees
b) have relations with Marilyn Monroe
c) miss three years at war
d) achieve a statistical anomoly that was almost meaningless but is somehow regarded as one of the five most important baseball records, due to a, b, and c.

Rod
Guest
Rod
3 years 7 months ago

Joe DiMaggio put up 91.9 WAR and missed three years of his prime during the war, he easily could have had another 15-20 WAR during those years. That doesn’t put you in the conversation for one of the 20 greatest players ever?

TKDC
Guest
TKDC
3 years 7 months ago

Maybe missing time to war should be considered in making a top 100 list. Maybe drug use should. Maybe being an asshole should. But if you say “on field performance” then war service is meaningless.

Michael
Guest
Michael
3 years 7 months ago

I’ll admit I clicked through the Top 25 on their list, and fed the SEO machine, and I was amazed that they put Pujols in the top 10 even. He’s been a great player, and all signs point to him continuing that well into his 30s, but he’s not already one of the 10 greatest players ever. If his career were to end tomorrow, I don’t see any chance of him making the hall. You can’t base that kind of stuff on projected future performance. I didn’t even bother to look at the rest of the list after that.

Cidron
Member
Cidron
3 years 7 months ago

on stats he is in pretty easily.. ok, may not be the supercharismatic or the “notice me” type that alot are, but, the numbers say he is very hall worthy.

Not sure about others, but as for me, I have some kneejerk reaction when someone says a current player, or recently retired player is hallworthy. Its “Naa, he isnt among the best ever.. Why I remember hearing about (player a) and his accomplishments, and HE is hall worthy, but this guy that is still playing… not a chance”. And, in hindsight, I find its just usually the “newness” by comparison to the rest in the hall. Miriano Rivera, easy example. Not a chance on kneejerk reaction.. then, upon actually looking at stats.. Surefire hall of famer. Its the newness and recentness of his stuff. He is new, not history. Hall of fame is about history.

B N
Guest
B N
3 years 7 months ago

Eh, I don’t know. I’ve been considering Mo a living hall of famer for years now. Heck, I considered Bonds and almost sure-fire HOF guy before he even started juicing. Not top 10 at that point, but definitely HOF. Other living HOF? Pujols is nearly a lock. Chipper Jones was a lock for a couple years now and you can make a good case that he’s the 2nd or 3rd best 3B ever.

I think one issue is that an active player can still wreck his ratio stats (OBP, ERA, etc). If a guy has too much time, sometimes they can dilute a great start with a bunch of terrible years. These years add minimally to counting stats, but can really hide a player’s peak.

monkey business
Member
monkey business
3 years 7 months ago

He didn’t say the player was/wasn’t HOF material, just that if his career ended today he wouldn’t make it. Other players, they would.

Gareth White
Guest
Gareth White
3 years 7 months ago

Are you sure you read the list? Because they put Pujols at #19.

ben
Guest
ben
3 years 7 months ago

re: BB HOF compared to RR HOF

Problem with the BB HOF is sometimes it’s too steeped in tradition.

Problem with the RR HOF is it’s not steeped in tradition enough.

Dan G
Guest
Dan G
3 years 7 months ago

The HOF has its fingers crossed that everything will work out and they won’t have to take any action regarding PED witch hunts.

At the least, the HOF should make a statement offering direction to the voters on how they should assess the players who chose to use illegal PED’s to maximize their playing contribution. It needs to be a strong statement telling voters either to put a lot or a little of emphasis on that behavior; either that it should be a major disqualifier or pretty much disregarded. Unfortunately, the HOF board of directors is doing its usual act of ducking for cover and hoping it blows over; which it may.

The BBWAA’s long-established rate of electing players is 1.55 per year (73 players in 47 years since annual elections resumed in 1966). This is easily seen in recent history: in the past eight years, they’ve elected 12 players in a perfectly alternating pattern, 2-1-2-1-2-1-2-1. The HOF is praying that the number of “clean” candidates coming up will enable the electors to maintain something close to this modest rate of electees, especially if the reformed VC can be counted on to provide an inductee or two every year. Then they can ignore the issue and proclaim “the electorate has spoken, having again done an excellent job of identifying the deserving players, blah blah blah.”

Even if the HOF gets its wish, it won’t work. There will be an outcry if the BBWAA fails to elect the best of the PED-tainted candidates. And that is what has always motivated the Hall’s board to change the election process: reaction to outrage among a large enough segment of the fans or the writers.

Once a few of the known users are elected, it cracks the door open. It will lead to less and less regard for considering PED’s a disqualifying indicator of lack of integrity, and more and more as a condition of the game as it was played in that era.

wpDiscuz