2008 Fan Save Values
One of the main reasons I got involved with sabermetrics was to show fans intimidated by statistical analysis that not all effective evaluative methods consist of complex mathematical formulas. My favorite statistics are ones like FIP, wOBA, and EqA: better indicators of skill and performance but scaled to look similar to the more commonly accepted barometers. In using similar scales more of a chance exists for widespread acceptance.
Through surveys I found that a major reason fans shy away from better evaluative methods is that they are unfamiliar with the baselines. They don’t necessarily know what a good VORP is, or a good WPA; however, they definitely know what constitutes a good or bad ERA or batting average.
One of these stats not mentioned too much is called “Fan Save Value,” and it was created by analyst Ari Kaplan while working for the Orioles in 1990. Essentially, as described in the book Baseball Hacks by Joseph Adler, FSV measures the difficulty level of each save by taking into account the lead with which the closer enters as well as the number of outs he must record to secure a win for his team. When all of the results are added together we are left with a number similar to the saves total but more indicative of how hard a closer had to work.
The formula for FSV is (X/Y)/2, where X=the amount of outs to record and Y=the lead of his team. For instance, recording a one-inning save with a two-run lead would result in an FSV of 0.75; 3 outs divided by 2 runs ahead, then divided by 2. Using this statistic I decided to look at the current saves leaders and determine how hard each has had to work:
Francisco Rodriguez, LAA: 12 saves, 11.3 FSV
George Sherrill, Bal: 11 saves, 7.6 FSV
Joe Nathan, Min: 10 saves, 10.8 FSV
Jonathan Papelbon, Bos: 8 saves, 10.3 FSV
Mariano Rivera, NYY: 8 saves, 9.8 FSV
Huston Street, Oak: 8 saves, 8.3 FSV
Brian Wilson, SF: 10 saves, 10.5 FSV
Eric Gagne, Mil: 9 saves, 8.0 FSV
Jason Isringhausen, StL: 9 saves, 6.5 FSV
Brandon Lyon, Ari: 9 saves, 9.5 FSV
Brad Lidge, Phi: 7 saves, 7.5 FSV
Jon Rauch, Was: 7 saves, 6.1 FSV
The saves leader in each league remains on top when using FSV but the rest of the leaderboard shifts. The biggest dropoffs come from Isringhausen and Sherrill: Both of these pitchers have entered save situations in which their teams led by large margins and/or recorded a couple of 0.2 IP saves.
Papelbon, on the other hand, has pitched more than one inning in a few of his saves. Since the overall result looks similar to the saves total, and saves are commonly used as an end-all when evaluating closers, the FSV is an easy to use and better evaluative tool because it adds context to a normally context-free statistic.
Print This Post
Thank you for this post – beyond being interesting, it addresses an issue that doesn’t get brought up nearly often enough.
All these new stats are great, but you’re right, without baseline levels of comparison it’s kinda hard to do anything with them or use them to convert new fans to saber.
Thanks again for bringing this up.
P.S. How does the 0.2 IP save figure in to the math? You use 0.666 when doing the conversion though, right? I’m sure you get where I’m going with this.
Eli, it’s not the 0.666 that gets used. The formula is -
(Outs To Record/Runs Ahead)/2
So if a pitcher goes 0.2 innings with a 3-run lead, it would be (2/3)/2, or .033. You wouldn’t do 0.666 but rather 2 because there are two outs left to record.
I have a book coming out towards the end of this month, beginning of June, titled “Bridging the Statistical Gap,” that addresses this issue more fully by breaking things down for fans unaccustomed to or intimidated by statistical analysis and explaining baselines as well as showing not-too complicated methods to evaluate players.
Don’t know why I overlooked that… Thanks for redirecting me ;)
Will your book be available on Amazon?
Yep, no problem. Amazon and Barnes n Noble.com among others. I’ll post something in a bunch of places when it gets closer.
Kaplan also came up with “Save Value” which is a bit more complex, but measures not just the outs and lead but also the difference between expected runs scored with the bases loaded and no outs and expected runs scored based on the situation in which the closer entered.
It looks like this:
= 1.12 * X/ (L+Y-E)
where,
X = outs left to go
Y= runs ahead
L = 2.27 (which is the expected runs with no outs and bases loaded)
E = expected runs based in the situation you entered
So, if a closer pitches the ninth inning, and he pitched the whole inning with a 1-run lead, his Fan Save Value would be 1.50, however his Save Value would be:
(1.12 * 3) / (2.27+1-0.54) = 1.23
Save Value is going to be more accurate but in terms of integrating all fans, FSV does a pretty good job.
Good stuff — I actually hadn’t heard about FSV before.
Are you actually from Philly? I’m in the suburbs just outside, in Delaware County.
I wonder if that formula could be adjusted to get an idea of the value of all relief appearances. Instead of “outs to record to finish the game”, there would be a value for “outs produced”. The value for run differential stays, maybe with adjustment if the reliever gives something up. I think it’d be interesting to see what kind of mid-game “saves” a Scot Shields or Tom Gordon type acquires.
Yep, I live in Northeast Philly, about a 25 minute drive from the stadiums. My father was the Phillies/Sixers/Flyers TV producer for PRISM for about 13 years and he was programming director for Sportschannel when it was around.
TC, that is something I’m actually working on this summer, as I try to figure out a system for evaluating relievers effectively. I’m separating the relivers into different groups, because we shouldn’t look at 1-out specialists the same as mopup guys and so forth, but yeah, manipulating stats like this is one of the key components.
For specialists I have in mind something similar to the LWTS Runs for each pitch thrown; since they face so few batters each pitch is very important. Factoring in that for the pitches thrown as well as an ongoing additive component of each out recorded and the difference in expected runs.
That could then be applied to non-specialist relievers, but without the pitch lwt-runs.
The one issue to work around is that Holds are one of the only statistics that really measure non-closer efforts in relief and, in my opinion, that stat stinks. In relation to having a similar baseline with evaluating relievers it becomes tough because we would have to make it known what is good and what isn’t.
It would likely result in using a baseline similar to ERA+ and OPS+ where 100 is average and above that is that specific percent better than the average.
So instead of looking at JC Romero and seeing he gets a score of +24.1, which would be vague unless looked at in the scope of a league leaderboard, we could say his Relief Score is 154; his results are 54% better than the average short-specialist.
I know this simply because I was perusing his game logs not too long ago, but J.C. Romero’s performance this year is misleading. He’s allowed quite a few inherited runners to score already, and he’d blown two saves all the while keeping his scoreless innings streak intact (until Aaron Rowand broke it Friday night).
Bill, my brother Corey covers the Phillies over at MVN and I had given him a stat early on that Romero allowed just 6 of 26 inherited runners to score with the Phillies last year, and as of the time I gave him the stat he had already allowed 5 of his first 8 this year to score. This is one of the reasons ERA is bad for a reliever. Another is because they pitch such a small sample of innings that the results are not stable.
If Romero pitches 0.2 IP in 9 games, it comes out to 6 IP. Now, if he is perfect in 8 of those 9 games, but in Game #9 he gives up a walk, single, and three run homer, he is going to end up with a 4.50 ERA, an ERA we don’t consider too good for a reliever.
Despite this, he would have done his job a remarkable 89% of the time. I’m still confident in him but definitely agree his ERA is misleading because what he is asked to do is strand runners and get lefties out. He’s gotten lefties out as they are currently just 2-20 with 1 BB off of him, but 5 of his 11 inherited runners have scored.
Can you try comparing to WPA, LI, and WPA/LI ?
Well, in terms of definitions, WPA, LI, and WPA/LI are a constantly updating stat whereas FSV is only dependant on the situation in which the Closer enters.
If a Closer enters into the top of the ninth, with a 1-run lead, he would get an FSV of 1.50 for that game because it’s just the three outs and one-run lead that matters.
If we look at that with LI, he would be entering into an LI of +2.9. Looking at WPA, getting a 1-2-3 save in the ninth, with a 1-run lead, would result in a WPA of roughly 1.34.
So, using the FSV it would be 1.50, using the SV it would be 1.23, and using WPA it would be 1.34, though WPA takes everything that occurs in the inning into account. The Save Value stats look solely at the entering situation.
In terms of contribution to a team’s wins, WPA and WPA/LI are going to be more indicative, but the SV and FSV make the Saves statistic useful by gauging the difficulty level in recording said saves while still representing it with a number close in appearance to the Saves total.
My point is that the function itself may not be valid. The gmLI tells you exactly the impact of his appearance. Does “outs left divided by runs lead” do that? That’s what I’m asking. How well does that function correlate to gmLI. And, it will clearly fail somewhere (like coming in with men on base), so perhaps it can be tweaked somewhat.
Just because you get a number that seems reasonable doesn’t mean it is reasonable. Look at BP’s LEV as an example of a good idea gone bad.
Yeah, it can definitely be tweaked. I’m working on that right now. I’d like it to examine runners on base as well because a 0.2 IP save with nobody on and a 1-run lead is much different than a 0.2 IP save with bases loaded.
The latter example would result in a lower FSV even though it is a much tougher situation.
I’d also like to somehow work in the type of batters faced. Though it is ultimately left to chance it is something I’m very curious about. Recording a 1-run save in the ninth against guys like Adam Everett and Nick Punto is much different than facing Utley-Howard-Burrell.
Eric, if you look at these players WPA only in the games where they got saves (same subset you are using), you should see that WPA roughly correlates with FSV. FSV shows a very rough approximation for WPA by using outs, runs, and the number 2 to come up with a leverage index. Let’s just look at 3 out saves, you could expand this table to show all save situations if you wanted…
from tangotiger’s page
http://www.insidethebook.com/li.shtml
just bottom of the 9th, 3 outs to go, bases empty.
up by 1, 3.6 LI, 1.5 FSV
up by 2, 2.0 LI, 0.8 FSV
up by 3, 1.0 LI, 0.5 FSV
FSV is a nice idea in that it tries to take some of the save context into consideration and put the results on the saves scale. What might be a fun project is to use WPA (which already has the real context FSV tries to approximate, and more, like base state, home vs road) and put that on a saves scale to make it more accessible.
Jacob, that’s definitely something to explore. And, as I wrote in the comment above, something I’d like to also add in is the quality of batters faced.