Fun with Low Reliability Scores

It’s been a crazy day. I’m busy working, writing various things. Then, because I’m a real genius when it comes to hitting “reply all,” I realized that I didn’t let people know I wanted to post, and what I wanted to write on got (rightly) taken. To top it off, there’s big happenings afoot in Rumorland that may dramatically affect the outcome of The Contest, but I don’t want to jinx it (whatever “jinxing it” would involve).

So today, I want to do something different and look at some “low reliability” offensive projections for 2010. From time to time, I reference “my” projections for a particular player. Believe me, I’m no statistical or programming guru. I simply started with Colin Wyers’ MySQL script for a Marcel-type offensive projections, then added in some tweaks for different aging curves and amounts of regression for different components. Like Marcel, it doesn’t adjust for parks or leagues or include minor-league data. Marcel isn’t intended to compete with the “big boys” (e.g., CHONE, ZiPS, PECOTA), but rather to be a baseline against which they are judged — in fact, Marcel does pretty well. I just hope I’m not doing something worse. In any case, it’s just a toy for my personal enjoyment at this point; it would take a fair bit more tweaking before I published the projections as a whole or even gave the “system” a name (although I admit having some ideas for the latter).

For each player projected, my system, like Marcel, generates a “reliability score,” which, in Tango’s words,

shows how much of the forecast is based on his performance, and how much was regression towards the mean…. [for example] Bobby Abreu shows a .87. That means that I regressed towards the mean 13%. Using that, it should be easy enough to figure out a confidence interval for each of the stats. If I show a reliability of .00, this means that it is an absolute pure guess on my part.

Generally, you want to look elsewhere than Marcel’s or my system when the reliability score is under .72 or so.

But that’s where today’s “fun “starts, because the way the projection system is set up with the Baseball Databank, if a player (pitchers are excluded) even has 1 plate appearance in last three years, it generates a projection for him. So let’s take a look at four goodies my mystery projection system has in store for in 2010. I’ll give the “three slash,” then runs created above average per 150 games, and then the reliability score for each player and some commentary. At the very least, this might provide a weird corollary to Dave Cameron’s point about regression “fixing” problems.

Morgan Ensberg, .250/.332/.408, -2/150, .505 r. I just read that he retired last spring due to lack of interest from major league teams. It’s not surprising, I guess, but I have to say that it seems like just yet yesterday he was a decent-fielding, three true outcomes machine for Houston. Then, after 2006… nothing. But my system says he’s not a bad bat if he can field decently.

Neifi Perez, .251/.315/.402, -8/150 .141 r. Perez was the return for one of the worst trades in recent history, but at least the team that got him never repeated that sort of mistake again. That’s not a bad line for a shortstop. Indeed, Neifi’s due for a career year at the plate (regression!). I mean, you wouldn’t hit him first or anything.

With the signings of Vladimir Guerrero and Jack Cust this weekend, here are two DH candidates.

Jay Gibbons, .254/.305/.403 -12/150, .412 r. Last seen playing for the Newark Bears, Gibbons, like Ensberg, retired in 2009 after being unable to find a job for a major-league team. I used to regularly take a certain well-known fantasy guru’s advice and put Gibbons on my roto team. It never worked, not even once. I was never really not sure what the guru or the Orioles saw in him. I guess he put up a decent season in 2005, and even played the outfield pretty well. But he never really seemed to have the patience or contact skills to back up his lack of defense.

Barry Bonds, .261/.410/.478, +16/150, .536 r. You had to know it was coming, right? Not bad for a man who hasn’t played the past two seasons. Perhaps you’d expect better linear weights for a guy with a .410 OBP, but keep in mind that the lwts don’t include intentional walks, which Bonds had a lot of (like Marcel, my system current projects iBBs). Still, there are plenty of teams that could use a +16 hitter at DH. I wonder how permanent that retirement is?

Until the next installment!

We hoped you liked reading Fun with Low Reliability Scores by Matt Klaassen!

Please support FanGraphs by becoming a member. We publish thousands of articles a year, host multiple podcasts, and have an ever growing database of baseball stats.

FanGraphs does not have a paywall. With your membership, we can continue to offer the content you've come to rely on and add to our unique baseball coverage.

Support FanGraphs

Matt Klaassen reads and writes obituaries in the Greater Toronto Area. If you can't get enough of him, follow him on Twitter.

newest oldest most voted

You win the “WTF” prize for the week, I think. And it’s only Monday.


The WTF prize is a platypus wearing fighter goggles and playing a pretty goddamn good kazoo, if you ask me.


And on this site the platypus’ name is normally Carson Cistulli.