Is Joe Mauer a good defensive catcher?

When Mauer won the gold glove this year, I realized I had no idea whether he deserved it.  I mean, my impression was that he’s not a defensive liability, but I didn’t really know if he was average, elite, or somewhere in between.  I’m certainly no scout, so the fact that he seems pretty good when I watch TV doesn’t mean a whole lot.  Judging by the awards, his reputation in the league is quite good, but Gold Gloves are often handed out to the less-than deserving and Mauer would certainly qualify as a player who might get collateral respect for his defense since his all-around game is so amazing—the dreaded "Jeter effect."

At the same time, catcher defense is sort of the red-headed stepchild of sabermetrics.  No one seems to know what to do with it, most just ignore it, and those that don’t usually find the experience equal parts baffling and regrettable.  The so-called advanced metrics for the other defensive positions are mind-numbing enough.

But undaunted I remain.  This is my attempt to see if I could slog through the tall weeds and find out what, if any, reliable information the numbers had to offer about catcher defense and Joe Mauer. 

I started with some of the good-old-fashioned stats that are easy to understand.  We’ll leave the really gnarly stuff for later.

Fielding percentage.  Avoiding errors is only a small piece of the puzzle, but still, errors are bad.  Yes, the scoring of errors is a subjective process prone to scorer bias, but again, it’s clearly better to have a high fielding percentage than a low one.

We have a pretty nice selection of fielding stats for catchers from 1956 on so let’s look at that group.  To avoid oddities due to small sample sizes, we will look only at players who have started at least 520 games at catcher (about 4 full seasons worth).  That gives us a group of 150 of the most accomplished catchers of the last 55 years (spreadsheet link at the bottom).

Out of those 150, Mauer’s fielding percentage of .996 comes in second overall.  (He’s tied with Mike Redmond, oddly enough.)  The average fielding percentage of the last 55 years for catchers is .989.  That may not seem like a huge difference, but then, someone with a .989 fielding percentage would commit almost three times as many errors as Mauer over the same span.  Mauer’s 21 errors over his career would instead be 54 or so.

Passed balls.  A close cousin of the error, the passed ball suffers from some of the same scoring vagaries that fielding percentage does, but again, you clearly want to keep these off your baseball card.

Looking at that same group of 150 catchers, Mauer is 16th in fewest passed balls per inning caught.  The average catcher of the last 55 years would allow about 16 passed balls for every 10 that get by the Chairman.  So if Mauer were merely average on this score, he’d have 52 career passed balls instead of the 32 he actually has. 

Wild pitches.  Some argue that wild pitches should be counted along with passed balls when evaluating catcher defense.  For one, catchers try to prevent both.  For another, the difference between a passed ball and a wild pitch is often as much due to the scorer’s discretion as to anything else.   

Side note: If you care to share my pain, I’ll fill you in on an odd problem with the wild-pitch data.  It seems no one has the data for 2009.  Fangraphs, bizarrely, has essentially no data on wild pitches for catchers after 2005.  Equally weird, Baseball Reference has wild-pitch data for every year except for 2009.  As far as I can tell, no one else has wild-pitch data for catchers.  Not MLB.com, ESPN, cbssportsline, Baseball Prospectus, or Yahoo.  So I’ve got this great spreadsheet with wild-pitch information on 150 catchers going back to 1956, but somehow 2009 got lost.

The effect of this when I look at wild-pitch rates is that it tends to give catchers who caught in 2009 a "free" year and therefore understate their true wild-pitch rate.  It’s really annoying, but we can make a rough adjustment to compensate.  Figure the rate at which the catcher allowed wild pitches in non-2009 years, and then figure how many they would have allowed in 2009 given the number of innings they caught (which we know).  The numbers below reflect this adjustment.  But it was annoying.

Out of the 150 catchers on our list, Mauer is slightly above average when it comes to preventing wild pitches.  Fifty-ninth out of 150.  Not his strongest point, but no shame there.  You could speculate that the Baby Jesus gets the benefit of the doubt more than his pitchers, resulting in fewer passed balls for Mauer and more wild pitches for the mere mortals who throw to him.

But on the whole, Mauer is clearly quite good at controlling pitches.  Looking at wild pitches plus passed balls per innings caught, Mauer is 35th out of 150.  Just for some context, some of the great defensive catchers by reputation are ahead of Mauer on this list, like Gary Carter (16th) and Yadier Molina (27th), but so too are some catchers with somewhat poor defensive reps, like Victor Martinez (24th) and Mike Piazza (25th).  Johnny Bench lags Mauer here (48th) and Ivan Rodriguez is way down at 121st!  Mike Redmond, it must be noted, is a impressive 4th.

Stolen bases.  Perhaps the most visible aspect of catcher defense is throwing out base runners.  Mauer has thrown out 36% of would-be base stealers compared to an average of about 34%.  That slightly-above-average showing is good for 61st on our list.

But you can prevent stolen bases in other ways.  Perhaps most significantly, opposing managers and base runners should know better than to run on the best catchers unless the situation is very favorable.   This "reputation effect" will tend to limit a well-thought-of catcher’s ability to throw out a very high percentage of runners, but will simultaneously drive down the total number of bases stolen (and of course the opposite will be true for catchers with the opposite reputation).  Aside from reputation effect, catchers can call throw-overs, manage pitch selection, and do other subtle little things that discourage attempts in the first place.

So instead of looking simply at stolen-base success rates, we can look more broadly at how many bases were successfully stolen per catcher inning.  On this measure, Mauer is near the top, 15th out of 150, one slot ahead of the great Johnny Bench.  (Stolen-bases allowed per inning seems to line up pretty closely with my broad perceptions of catcher reputations.  Yadier Molina is 1st and Ivan Rodriguez 13th.  V-Mart is 113th and Piazza is 147th.)  Mauer has allowed 257 stolen bases in his career, but if he had been stolen on at an average rate, he’d have allowed 419.

An interesting side-note: It’s generally not worth stealing unless you can do it successfully 75% of the time or more (link 1, link 2).  The fact that the average catcher of the last 55 years only allowed successful steals 66% of the time suggests that managers and players simply steal far too much (paging Ron Gardenhire!).  While still below the break-even rate, the stolen-base success rate is somewhat higher today than it was in the ’50s and ‘60s, which makes Mauer’s numbers a bit more impressive compared to those of the golden oldies.

Ok, I think that exhausts the easy stuff.  Here’s a nice little chart of Mauer on these measures against the 21 active players with at least 520 games started at catcher.  It’s in order of stolen bases allowed per 1,000 innings caught (about one season’s worth), and also shows rankings for stolen-base percentage, fielding percentage, and passed balls plus wild pitches per 1,000 innings: 

Maybe it’s just me, but it’s pretty hard to eyeball that data and not conclude that Mauer is something like the 2nd-, 3rd-, or 4th-best defensive catcher in the group.   He’s 3rd and 6th on the stolen-base measures, 2nd in fielding percentage, and 8th in controlling pitches.  Pudge and some others might get allowances for having logged a lot of innings while in the decline phase of their careers; their rate numbers probably looked a touch sharper when they were Mauer’s age.  And of course, this isn't looking at the harder-to-quantify stuff.  But still, Mauer’s clearly in pretty rarified air here.

Ok.  But do the so-called advanced metrics see it that way?  This is where it gets a little nasty, so hang in there.  (There’s no shame in taking a little break if you need to.  Don’t be a hero.)

Fangraphs and rSB.   Fangraphs doesn’t seem to give Mauer much credit.  To the extent Fangraphs puts catcher defense into its WAR stat for modern-day catchers, Mauer is factored in at one run below the average catcher for his career.  Because Fangraphs is so fantastically accessible, that’s the number I’ve seen most often.  While I’ve always known to take catcher-defense stats with a grain of salt, I just sort of took for granted that that number was at least a good college try at determining backstop value based on whatever solid data there was.

But when you look into the matter, you find that Fangraphs’ attempt to quantify catcher defense is at best half-baked.  There is no UZR for catcher defense like there is for other positions, so in its WAR stat Fangraphs instead just substitutes something called rSB, Catcher Stolen Bases Runs Saved.  Here is a definition of rSB, to the extent I could find one:

How do you measure Catcher Stolen Base Runs Saved?
Catchers also have a hand in controlling the running game. We also control for the pitcher’s historical stolen base rates in order to isolate the catcher’s contributions. (italics added, link)

So it’s attempting to quantify just the ability to prevent stolen bases; it’s not trying to gauge pitch blocking, errors, or anything else.  Yet still, how does Mauer end up a shade in the negative since by the basic measures we looked at above he seems at least above average if not quite good at preventing steals?

I’d speculate that the italicized sentence provides the answer.  They control for pitchers’ historical stolen-base rates.  So if Mauer caught pitchers with very good rates of preventing stolen bases, he would effectively be penalized.  Which would make sense, except...doesn’t Mauer have a heck of a lot to do with his pitchers’ historical stolen-base rates?  Most of the guys Mauer has caught have pitched much of their careers to him.  If you strip out these pitchers’ career stolen-base rates, you are simultaneously stripping out Mauer’s own contributions.

Unless there’s a subtlety in rSB that I’m missing, just looking at the unadjusted stolen-base data seems like a far better (if hardly perfect) approach.  While catchers are going to catch a fairly wide variety of pitchers, which will tend to smooth out the data over time, pitchers may throw a very large percentage of their innings to a particular catcher or two, making their stolen base rates very much a function of who those catchers happen to be.  Thus, using the pitcher rate data to "fix" the catcher data seems like a big step backward to me.  (I know, you too right?)

DRS.  Defensive Runs Saved (DRS), a product of Baseball Info Solutions that’s available at both Fangraphs and Baseball Reference, simply takes rSB and adds to that a factor that attempts to quantify a catcher’s pitch-calling skill.  I’m not sure how much stock we can put into this tricky enterprise, but Mauer gets solidly above-average marks on pitch calling (+13 runs), turning his -1 by rSB into a career score of +12 DRS.

I’m unsure if this is much more than a look at catcher ERA, which is sort of a dubious measure.  But for what it’s worth, Mauer’s 3.94 catcher ERA is quite good and it makes sense he’d get points when someone tried to quantify pitch-calling ability.  Still, like the rSB stat on which it is partially based, DRS doesn’t attempt to be a comprehensive measure of catcher defense and seems to be a fairly questionable measure of even the stolen-base and pitch-calling skills that it sets out to quantify.

FRAA at Baseball Prospectus.  Even as a long-time subscriber, digging through Baseball Prospectus’s alphabet soup of proprietary stats can be a tough shovel.  Although currently in the process of a complete overhaul, Fielding Runs Above Average (FRAA) is their current go-to defense stat and is used in their WARP calculation. 

I certainly couldn’t explain to you exactly how it works for catchers (and their glossary provides precious little help), but this article seems to say that FRAA attempts to measure throwing out base runners, blocking pitches, and as a bonus they throw in a measurement for the fielding of balls in play.  (The later measure doesn’t vary much among catchers and is a somewhat different approach than looking at fielding percentage.)  Anywho, Mauer’s a full 34 runs above average on his career in FRAA, which despite its murkiness and impending obsolescence appears to be a more comprehensive, less problematic metric than rSB and DRS.

How does that stack up to other catchers?  Unfortunately, Baseball Prospectus makes digging up FRAA data pretty damn difficult.  To get Mauer’s +34 FRAA on his career I had to pull up a WARP report for each year in which he played and do several tedious rounds of addition(!) to get his career numbers; something I don’t especially care to do for a bunch of other catchers.  But I made an exception for these Mauer contemporaries:  On a per-1000-innings basis, you get V-Mart at -6 FRAA, McCann at -4, Mauer +5, Laird +10, and Yadier +15.

Total Zone.  This is the one I like.  Total Zone (TZ) is the defense component of Baseball Reference’s WAR stat (rWAR) and of Fangraph’s WAR stat (fWAR) for years prior to 2003.  (Confusingly, it’s abbreviated "TZ" at Fangraphs and "Rtot" and "Rfield" at Baseball Reference.)  Here’ s an explanation of how Total Zone does defense for catchers:

Catcher data looks at stolen bases allowed, caught stealing, errors, pickoffs, passed balls, and wild pitches. I split the data by pitcher handedness (otherwise a catcher will look better if he catches more lefthanders than normal.) Once again, everything is compared to league averages and converted to runs.  (link)

Sing it sister!  This sounds like what I’d expect from a fair attempt to quantify what is quantifiable about catcher defense.  All the basic catcher stats are accounted for, there’s a sensible adjustment based on pitcher handedness, and then everything is converted into runs compared to league average so you can compare pretty well across eras.  As a bonus, unlike rSB and DRS, which aren’t available prior to 2003, and FRAA, which is available, but only at the expense of a full weekend of stat gathering, career TZ numbers are easily accessible at Fangraphs and Baseball Reference and go all the way back to 1956.

So how does Mauer compare by Total Zone?  First, here’s Mauer with the 21 currently active catchers with at least 520 games started at catcher, ranked in order of most runs above average per 1000 catcher innings:

So Mauer’s fourth on this list, which jives pretty nicely with what we figured by eyeballing his performance on the basic catcher stats we looked at above.  The other catchers also seem to line up about as you’d expect.  (And if you care to see another ranking by someone who independently followed a similar method to calculate defensive value for catchers from 2002-2009, take a peek at this post at this post over at Bless You Boys.)

Now going back to the list of 150 catchers, here are the top 25 by TZ per 1000 innings:

(* stats are only partial career stats because Crandall and Berra started playing before 1956)

So there’s our guy, snuggled right in cozy-like with the greatest defensive catchers of the last 55 years.  Ahead of Bench, Carter, late-career Berra, and Carlton Fisk (69th).

There’s nothing gospel about the TZ numbers—I’m sure they’re imperfect in all sorts of ways and no one is going to accurately capture all the significant intangibles that go into catching nor control for all the potentially confounding variables.  But comparing it to the other advanced stats that are widely available, it’s relatively straightforward, relatively comprehensive, and it seems to be backed up pretty nicely by our look at the good-old-fashioned stats everyone understands.

As I said above, I really didn’t know going into this if Mauer was anything much more than a solid average on defense.  But I’m pretty convinced now that he’s a really excellent catcher.  Forget his bat, based on his defense he could start in MLB even if he were as bad a hitter as, say, Gerald Laird!  His performance will probably tail off some as he ages, but he can decline considerably and still be a really solid MLB catcher.  Something to think about whenever the next round of move-Mauer-to-third comes up. 

(But is he a good hitter for a catcher?  That one’s easy.  So far he’s been the third-best hitting catcher of all time.  Go here and sort by wRC+--a great stat for comparing overall offensive performance across leagues and across eras.  And no one’s arguing that Mike Piazza and Gene Tenace were great shakes on defense.)

Here’s  a spreadsheet with the data on the 150 catchers if you care to look in more detail.  (It reflects the adjustment I discussed above for the 2009 wild-pitch data.)

The end.