# Evaluating catchers: Quantifying the framing pitches skill

Catchers’ ability at framing pitches is a skill that, since the advent of PITCHf/x data, seems so close to being understood and measured. Several analysts have given first passes at the issue (Dan Turkenkopf in 2008, Matthew Carruth and Bill Letson in 2010).

Following is another contribution on the subject, which will give a rough estimate of the runs a catcher might save in a season thanks to this particular ability. A file with catchers’ ranking is available at the end of the article.

### Definition and problems

The framing skill will be measured here as the increased (or decreased) probability of a pitch to be called a strike due to the presence of a particular catcher behind the plate. Obviously it is not easy to isolate the receiver effect from the rest of factors influencing the final umpire’s decision. Otherwise, a framing measure would have been around for some time now.

Several components concur and interact in the outcome of a pitch not swung at. **John Walsh** has shown how umpires tend to be “compassionate”, give the benefit of doubt to the batter on two-strike counts, while helping the pitchers when they are one ball away from issuing a walk. Other than their merciful nature, umpires are individuals and each has his own peculiar strike zone. Some of them are known for being pitcher friendly, while others are particularly stingy at calling strikes.

Pitchers also have different abilities of getting a favorable call on borderline pitches. Tom Glavine, for one, was able to expand the strike zone inch after inch (at least that was his reputation). Also, different kind of pitches, due to their particular movement, have unequal chances of being declared strikes.

Finally it is possible that the batters themselves can influence the call, because of their stance or their established reputation as selective hitters or free swingers.

### Tools and technique

Since the nature of the subject is full of complications, this analysis focuses on a selected part of the problem. Only outside pitches to right-handed batters will be considered. Let’s go step by step in selecting our subset of data.

Pitches are selected such as their height is unlikely to influence the strike call. Mike Fast has recently shown that nearly every pitch delivered between 2.2 and 3.0 feet of height is called for a strike if it’s thrown to the fat part of the plate. The charts below show the chance of pitches at that height of being called strikes, as a function of their horizontal location.

Focusing our attention to the outside part of the plate (thick blue line in the left chart), we see the probability of a called strike has sort of a sigmoidal trend: It’s constantly close to 100 percent in the fat part of the plate, and constantly close to zero from two feet outside on. The uncertainty is in the zone in between, which has been zoomed and marked with a thick red line in the right chart: Here is where the factors mentioned before, including the catcher’s skill, come into play. Thus pitches (every one recorded by PITCHf/x from 2007 to the end of May 2011) in that zone are considered for this analysis.

To deal with pitcher-catcher simultaneous effects on wild pitches and passed balls, **Tom Tango** has widely used a technique he dubbed With-Or-Without-You, shortened to WOWY. In a previous article I proposed an advanced statistical technique which makes possible to take care of more-than-two-ways interactions. In that case it was possible to measure the base stealing game giving appropriate credit to pitchers, catchers and baserunners.

Here the *multilevel logistic regression with crossed random effects* (that’s the name of the technique for those who want to check) allows us to apportion the difference in called strikes probability among umpires, pitchers, catchers and batters.

### Results

Before ranking the catchers, it’s useful to check if the estimated effects of other (more known) factors make sense. According to the model, the chance of a borderline pitch to be called a strike decreases of about 20 percent for every inch it gets further from home plate. So far so good.

Below is an estimate of the change in probability according to the ball-strike count (The 0-0 count is set as the reference).

count prob change0-1 -16 0-2 -19 1-0 4 1-1 -7 1-2 -12 2-0 8 2-1 -4 2-2 -11 3-0 2 3-1 -7 3-2 -7

The results are quite in line with the previous knowledge, as the player who is behind in the count usually gets the favorable call. The 3-0 value is probably lower than one would expect after having read Walsh’s work.

The umpire ranking resembles the one by Jeff Zimmermann at Beyond the Boxscore”>Jeff Zimmermann at Beyond the Boxscore (circa 2010), with Brian O’Nora, Brian Runge and Wally Bell outlined as pitcher friendly and Paul Schrieber, Gerry Davis and Adrian Johnson among the stingiest.

Regarding pitchers, I did not know who to expect at the top and at the bottom of the list. Since Tom Glavine is always cited as a pitcher who could get the call on borderline pitches, it’s reassuring to find his name in the above-average part of the ranking (though not at the extreme).

The most likely to get a questionable call their way are, according to the model, Francisco Cordero, Jake Peavy and Chris Young. The least likely, on the other side, are Max Scherzer, Jonathon Niese and Jonny Venters.

What to expect in the batters’ table? A great hitter like Albert Pujols being among the ones getting the benefit of doubt should not be a surprise, perpetrating a decades long tradition best symbolized by Hall of Fame umpire Bill Klem’s address to a rookie pitcher: “Son, when you throw a strike, Mr. (Rogers) Hornsby will let you know it”.

Finding a selective hitter like Marco Scutaro on one side (getting the benign call) and noted free swingers like Vladimir Guerrero and Jeff Francoeur on the other is also expected (however there doesn’t seem to be a correlation between hacking tendencies and rank on the list).

Since the model seems to give sensible results for the other factors, it’s quite likely that what we will see for the catchers is a decent approximation of the truth as well.

### Catchers

Since logistic regression results might not be of immediate interpretation, what you will find in the Excel file below has gone through a sort of translation to make it easier to understand.

Thus the four sheets in the Excel file (80 Kb) report the difference in probability due to the particular player (or umpire) involved.

The magnitude of variation is highest for pitchers and lowest for batters, with umpires and catchers in between and quite close between them.

It makes sense that batters are less likely to influence the call since, other than their stance, there should be nothing they can do to play a role in the outcome. It also makes sense that the pitchers have the most power on the call, as they are the ones delivering the ball and imparting the (possibly deceiving) trajectory.

The fact that umpires and catchers have a similar range of variation implies that playing with a receiver who is good at framing pitches is the equivalent of having a pitcher friendly umpire calling the game.

### Games won thanks to framing pitches

Let’s do some back-of-the-envelope calculations. A top catcher at framing pitches, such as Russell Martin, improves the chances of a borderline pitch to be called a strike of roughly 20 percent. Since the difference in run value between a ball and a strike has been estimated around 0.13 runs (see the References and Resources section at the end), a skilled catcher might be worth 0.026 runs on a single borderline pitch.

In the data used for this analysis, each game contributes on average four pitches. That is, there are four borderline pitches per game on the outside corner. Let’s conservatively guesstimate a total of eight when we add the inside part and the upper and lower border of the strike zone. Then divide by two, to apportion the borderline pitches between the two teams playing the game. That makes four uncertain pitches a game where the catcher can make the difference.

Martin caught 97 games in 2010. Multiply that by four pitches and by 0.026 runs and you get 10 runs in a limited number of games.

According to this analysis **the top catchers can win a ballgame per season** (even playing fewer than 100 games) only **with the skill of framing pitches**.

If you think that’s a lot, I’m with you.

Anyway, let’s look at that from a different perspective. Please re-read the last sentence of the previous section. **The fact that umpires and catchers have a similar range of variation implies that playing with a receiver who is good at framing pitches is the equivalent of having a pitcher friendly umpire calling the game.** Now, suppose you are allowed to have every game called by an average umpire when your team is at the plate and by the most pitcher-riendly umpire when your team is pitching. Does an extra win per season seem an acceptable effect of having such an advantage?

### Reliability, assumptions and limitations

Replicating the analysis on two subsets of the data (even-numbered caught pitches and odd-numbered caught pitches for every catcher) we get pretty stable results: The correlations of the catchers’ effect calculated in the two subsets of pitches is a solid 0.76.

The pitchers’ effect shows a similar stability (0.75), while the batters’ expected effect returns a lower correlation (0.40). Finally, the umpire effect lends the highest correlation (0.89) between the two subgroups, a good indicator of individual consistency.

As it always happens when building models, some simplifications have been made.

Here the “compassionate effect” is modeled as constant among umpires, while probably each man in blue has his own peculiar way of expanding and shrinking the zone according to the count. Also, while the probabilities are adjusted by the type of the pitch, again it is assumed that umpires react uniformly to the different behavior of fastballs, curves, and the rest.

Similarly the catchers’ skill is measured across the board; in other words, the model does not consider the possibility that one backstopper can be awful at framing fastballs but very competent on offspeed pitches. Plus, when doing the back-of-the-envelope calculation of saved runs, we implicitly assumed that catchers have equal skills on pitches outside and inside, high or low.

While the aforementioned assumptions do somewhat simplify the reality, it would be very difficult and computing intensive to add those individual variations in a model, as the one used for this article runs for hours.

### Final remarks

According to the analysis presented here, the best catchers at framing pitches can add something like one or two wins per season, which is the equivalent of trading Alex Rodriguez‘s 2010 bat (.270/.341/.506, 30 HR, 125 RBI, 19 runs above replacement) with Alex Rios‘ lumber in the same year (.284/.334/.457, 11 HR, 45 RBI, nine runs above replacement).

The number could even be a conservative estimate. In fact, as soon as a pitcher realizes his catcher gives him an edge on borderline pitches, he should immediately begin to exploit the advantage.

If the magnitude of the framing effect measured in this study is confirmed, major league teams should not neglect this factor when they go hunting for a catcher in the market, especially those with pitching staffs that make their living on the black.

**References & Resources**

PITCHf/x data by MLBAM, corrected using the author’s algorithm.

Dan Turkenkopf on the value of switching a ball to a strike.

John Walsh on pitches run value.

Craig Burley on the benefit of throwing a strike, by count.

Nice job Max!

I don’t know if you saw Mike Fast’s article from February that looked at framing and attempted to correct for a pitcher’s tendency to throw to the edge of the zone (http://www.baseballprospectus.com/article.php?articleid=12965).

He found about the same range of value as you did – +/- 10 runs from average, which is certainly more believable than what Bill and I found.

Peter, he actually conservatively doubled, not quadrupled:

You were probably as excited as I was in reading this article to have missed the nuance!

Wow, Max. What a great article.

Great work, Max.

It’s comforting to see Livan Hernandez near the top of the pitchers list.

Terrific work, Max. I even understood it all. Thanks.

Peter, you’re right, I neglected the effect on lefties.

Dan, I had missed Mike’s article.

All, thank you for your kind comments.

Just checked my unpublished research on a Catcher WOWY for runs saved by catchers by framing and pitch selection. It doesn’t take into account the many factors that your analysis does, relying instead on the aggregate career stats to even out everything except the pitchers the catchers worked with. I have Russel Martin as the 5th best catcher at 12.8 runs saved per 130 games.

Max – There is an alternative explanation to the compassionate umpire theory of why batters seem to get favorable calls from umpires in pitcher’s counts. Your study is lumping all balls between point .8 and 1.2 px as a single category. It is possible that behind in the count a batter will be more likely to swing at the balls within that interval that are closer to the plate in order to to protect against the called strikeout, leaving more balls farther away from the plate which are correctly called “balls” by the umpire at a higher rate.

Holy Moly, this is fantastic work Max. I need to sit down and think really hard about this – I think the findings cohere with some of the basic “a priori” analysis of catcher framing that I had done in the past – in particular, I think four pitches is an exactly right ballpark estimate, and I think relying on Dan Turkenkopf’s run value estimates is a good move, as they smell right to me.

Sweet work.

Great article Max

MGL, the compassionate umpire effect is harder to explain away than that.

First of all, John dealt with some of your concerns in his original article.

http://www.hardballtimes.com/main/article/the-compassionate-umpire/

J-Doug addressed a number of them in his follow-up research.

http://www.beyondtheboxscore.com/2010/12/24/1892898/benefit-of-the-doubt-odd-patterns-in-umpire-compensation

http://www.beyondtheboxscore.com/2011/1/5/1915896/benefit-of-the-doubt-the-cruel-umpire

I also looked into some of them in my article, which Dan linked in the third comment up above.

I agree we’re not to the point of understanding why it’s happening yet, but the things you suggest don’t seem to explain it.

Pigged=pitches. Autocorrect on iPhone…..

Peter + MGL,

I have selected pitches from .8 to 1.2 ft from the middle of the plate, but I have not lumped them together. I retained the actual pitch location, so whatever effect came out (for example the pitch count effect) should not be a byproduct of different locations of pitches in different counts.

Guy, as far as batters becoming familiar with the strike zone and/or umpires shrinking the zone (which is easily verifiable with the pitch f/x data), that would imply that relievers would not have any advantage when they come in late in a game. That clearly is not the case so I don’t think those things come into play. I think it is simply batters being familiar with a pitcher’s repertoire, etc.