On Productive Outs

by Jon Marthaler on Nov 16, 2009 9:00 AM EST in By the Numbers Comment 30 comments

A couple of weeks ago, I read this blog post from Joe Posnanski, in which he discusses comments made by Red Sox GM Theo Epstein about outfielder J.D. Drew. The whole thing's interesting, but what caught my eye was the following quote from Epstein:

He [Drew] does the single most important thing you can do in baseball as an offensive player. And that’s not make outs.

Well, this really hit me, because it's about the most concise possible argument you could make about hitting - just to simply state, "We get only three outs an inning, and when we reach three, we have to start over. Wasting outs for any reason does not help us." And yes, I realize that slugging percentage plays a big part in scoring runs, too, and a single is sometimes more valuable than a walk, and all of that. But ultimately good hitters that help their teams simply don't make outs.

This, however, got me thinking about "productive outs," and sacrifice bunting and all the rest of the things that managers and announcers talk about. How many times have you heard the Twins broadcast crew laud Nick Punto for grounding out to the correct side of the infield, thus moving the runner from second to third?

With all this in mind, I went over to Baseball Prospectus to grab the Run Expectancy Matrix for 2009. If you're not familiar with this graph, simply put, it tells you how many runs a team can be expected to score, given a certain situation and all else being equal. Here's the 2009 version:

Situation / Outs	0	1	2
Bases Empty	0.5173	0.2789	0.1064
Runner on First	0.8834	0.5330	0.2234
Second	1.1415	0.6884	0.3219
Third	1.3146	0.9655	0.3701
First and Second	1.4837	0.9217	0.4556
First and Third	1.7686	1.2018	0.5220
Second and Third	2.0137	1.4138	0.5623
Bases Loaded	2.2790	1.5580	0.7499

If you're new to reading this table, examine with me the second row ("Runner on first"). This row says that, in 2009, a team could be expected to score .8834 runs, on average, if it had a runner on first with nobody out. With one out and a runner on first, it was expected to score .5330 runs, and so on.

What I thought I'd find, from this table, was the ability to state a hard-and-fast rule about outs. I assumed that we could all live by the following statement: "Making an out ALWAYS decreases the number of runs a team can be expected to score in an inning."

What I found was close to that statement... but not quite.

Of course, the value of an out always depends on the situation, the batter at the plate, the runners on base, and so forth. (For example: it's a better idea to have Alexi Casilla try to bunt a run home than it is to have Joe Mauer try to bunt a run home, because Casilla is far less likely to get a hit or otherwise drive the run home.) However, with all else being equal, I was surprised to find that there are productive outs.

By this, I mean outs that actually increase the expected number of runs that a team would score. Here's the list of plausible productive outs I've found; perhaps you can find more.

With the bases loaded and fewer than two out, an out that advances all three runners is a productive out. (Just as an example, here's the math: With nobody out and the bases full, a team is expected to score 2.2790 runs. If the batter advances all three runners - say, with a sacrifice bunt - a run comes home, and the team has second and third with one out, an expectancy of 1.4138. In other words, the run expectancy has gone from 2.2790 to 2.4138. With one out, the expectancy goes from 1.5580 to 1.5623.)
With a runner on third base and one out, getting the run home via a sacrifice or simple ground-out is productive (the expectancy goes from .9655 to 1). Interestingly, this isn't true if no one is out (decreases from 1.3146 to 1.2789).
With runners on first and third and one out, moving both runners along is productive (1.2018 to 1.3219). This can plausibly be done via sacrifice, or possibly a deep fly ball to Johnny Damon or any other outfielder with a terrible arm.

And that's it. In all other situations, all else being equal, making an out - whether the announcers think it's productive, or not - doesn't help the team score more runs.

Of course, productive outs are better than non-productive outs. It's better for Nick Punto to move a runner over than strike out. But these are Pyrrhic victories; in most situations, the team is worse off.

I thought, when I looked at the numbers, we'd end up with a strict rule. Instead, we're stuck with more of the ambiguity that baseball will always be famous for: "For the most part, making an out decreases the number of runs a team can be expected to score, all else being equal."

0 recs | Comment 30 comments | Share

Story-email Email Printer Print

Read More: Minnesota Twins, Boston Red Sox

Comments

Nice work Jon.

I like this. Of course it all depends on actually having that runner on third with less than two outs.

Where I get tied up is that once the run scores, it no longer counts against run expectancy. If it’s situation-by-situation, can we really add that run to the expectancy total in order to produce a “productive” out? Or are we allowed to add it because it counts toward the whole?

Whatever. I’m buying what you’re selling.

by Jesse on Nov 16, 2009 9:39 AM EST reply actions 0 recs

I think it makes sense to add it.

If you’ve got the bases loaded with nobody out and no runs yet in, the chart says you’re expected to score 2.2790 runs. If the batter at the plate walks, driving in a run, the expectancy doesn’t change, but there’s already a run on the board. Hence, the run expectancy for this inning is now 3.2790.

Well, it makes sense to me.

by Jon Marthaler on Nov 16, 2009 9:43 AM EST up reply actions 0 recs

That is right

you have to add in the run that all ready scored.

by lookatthosetwins on Nov 16, 2009 11:13 AM EST up reply actions 0 recs

Exactly

That’s what I do with my own analysis. If it’s a walk with the bases loaded, the batter is assigned 1 run = runs scored + expectancy_after – expectancy_before.

by Adam Peterson on Nov 16, 2009 8:20 PM EST up reply actions 0 recs

Great analysis.

As a general rule, what you want is to not get out, of course. That’s just sense. But I’m glad you found that there are exceptions.

I love sabermetrics, but it does have limitations. Depending on the mindset of the guy on the mound and the guy in the batter’s box, a sac can make a big difference. (At some point, I really need to do some research on which pitchers are more likely to give up hits when they’ve got a known base stealer on first. In theory, base stealing is always too risky; but stats don’t automatically reflect that some pitchers simply melt down when Gomez dances around out there. I’ll miss that.)

http://www.realityfish.com

by Robin G on Nov 16, 2009 9:50 AM EST via mobile reply actions 0 recs

Sure, that's

your total runs. But in that next PA after the bases loaded walk, the run expectancy for that situation is still 2.2790 runs, right? The chart doesn’t care how many runs have already scored, it’s looking at every occurance as its own event. Sort of like flipping a quarter 10 times and getting heads every time, on the 11th flip there’s still a 50% chance it comes up heads.

I’m only arguing for arguments’ sake. I have no vested emotional right or wrong. Also I’m at work and for some reason my reply button doesn’t work. So Jon I’m still talking to you. JON, DOWN HERE!

by Jesse on Nov 16, 2009 9:53 AM EST reply actions 0 recs

Let me see if I understand what you're saying.

You’re making this very correct point: “Making an out ALWAYS decreases the number of runs a team is expected to score, in the future.” This is true.

However, if said out also scores a run, we need to take that into account too. The run expectancy chart is past-agnostic. It doesn’t “know” the run has already scored. All it can say is, “given this situation, teams average scoring this number of runs.” Even if three batters in a row have hit two-out solo home runs, the run expectancy chart still says that the team, on average, will score .1064 more runs.

by Jon Marthaler on Nov 16, 2009 10:02 AM EST up reply actions 0 recs

I like that you said 50%

Because the run expectancy chart is the average number of runs scored in that situation. Standard deviation is also very important to consider.

For example – with runners on 1st and 2nd with nobody out it gives a number of 1.4837. 2nd and 3rd with 1 out (bunt the runners over) is 1.4138, or virtually the same number. However, the probability that at least 1 run scores (versus not scoring at all) goes UP in that situation. In that case you’ve decreased your downside and not changed your upside much at all. Also, if the batter strikes out, your run expectancy drops to 0.9217 and a GIDP leaving the runner at 3B means 0.3701.

As an aside, I love/hate the 4 decimal places of precision on these numbers. There is no way the 4 decimal places are exactly the same if you look at run expectancy 2008 to 2009. For my example above the numbers in 2008 are 1.49811, 1.42246, 0.96937, and 0.34549. Those numbers are really close but calling them 1.5, 1.4, 0.95 and 0.35 is a better way to think about them. Those numbers do change if overall run scoring changes but the situation (pitcher, runner, batter) always makes more sense to consider. This is why Delmon Young should bunt more often, he GIDPs a lot.

by DJL44 on Nov 16, 2009 10:35 AM EST up reply actions 0 recs

upside

However, the probability that at least 1 run scores (versus not scoring at all) goes UP in that situation. In that case you’ve decreased your downside and not changed your upside much at all.

Actually, the upside goes down. That’s what balances things and keeps the total run expectancy the same.

by DavidRF on Nov 16, 2009 11:14 AM EST up reply actions 0 recs

Right

Shoulda said “average”, not “upside”.

by DJL44 on Nov 16, 2009 11:15 AM EST up reply actions 0 recs

I'm surprised that the classic case--runnner on second and no outs--is not productive

The other one I expected to be productive is first and second and no outs. I guess I tend to focus on the failures in those cases—guys who try to do too much and make an out that does not advance the runners. Of course, all things considered, an out that advances the runners is better than one that does not. Whether an out has any value is not black and white. There is a range of values you could assign to an out based on the circumstance. This is the blind spot in previous sabermetric discussions of productive outs.

The one thing that most discussions of productive outs misses is the relative ease of making an out vs getting a hit. If you have Nick Punto up and a runner on second and no out, it makes sense to ask him to hit the ball on the ground on the right side. His chance of success—getting a hit—is so low that the risk of having him try for a hit is greater than the rewards of him getting that hit. If it’s JD Drew, you don’t want to waste his out on advancing the runner because he has a very good chance of doing more than just making an out that advances the runner.

I once asked Tom Kelly on a Sid Hartman show why he doesn’t like bunting. The question was in reference to having Gene Larkin hit away with runners on first and second and no outs. Larkin grounded into a double play. He said what you said: Outs are precious. You don’t waste them except in special circumstances. So he asked me detailed questions about a circumstance where a bunt makes sense to me: the hitters, game situations, etc. I described a case where bunting made sense to him (Pat Meares hitting and Leius and McCarty running) when behind by at least one run. The upshot: The best managers don’t view productive outs as some sort of automatic formula, but on a case-by-case basis depending on the situation.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Nov 16, 2009 11:00 AM EST reply actions 0 recs

It could be productive

Runner on 2nd no out – 1.15, Runner on 3rd one out – 0.95. You’re giving up 0.2 runs on average but you also have to consider that the distribution of runs scored shifted away from 0’s and 2+ to more 1’s. In other words, you’re more likely to score 1 run but you’ve given up scoring 2+. That’s why they call them “one run” strategies. Any time you’re playing for one run by giving up an out you’re giving up a multiple run inning. That’s why no manager does this when down 3 or ahead 3.

by DJL44 on Nov 16, 2009 11:14 AM EST up reply actions 0 recs

All about the game situation

There are times when one run is all the difference, tie game late, etc. Jon’s analysis is really focused on the long term strategy, which will result in more runs scored over a season.

This is also why the first inning bunt with a runner on first (net -0.2 runs) drives me crazy. But as you note, it probably increases the chances of scoring at least one run, and at times getting that first run on the board can be critical. How critical is a topic for another day.

by Adam Peterson on Nov 16, 2009 8:25 PM EST up reply actions 0 recs

"At least one run" would result in a completely different chart

That’s the difference here. If you don’t care about the second, third, fourth runs of the inning, then all the probabilities are different. For example in the bottom of the 9th with the game tied. You move the lead runner over and it doesn’t matter if the trailing runner or batter is less likely to also score.

by DavidRF on Nov 16, 2009 11:17 AM EST up reply actions 0 recs

Yes

Does anyone know where to find that chart? The standard run expectancy matrix is pretty easy to find, but I don’t know where the “just one run” chart lives on the internets.

"There are only two things that are infinite, the universe and human stupidity, and I'm not sure about the former." - Albert Einstein

by BeefMaster on Nov 16, 2009 11:34 AM EST up reply actions 0 recs

Its a bit hard to read. For “1+” you have to sum the probabilities of 1,2,3,4,5+.

by DavidRF on Nov 16, 2009 11:37 AM EST up reply actions 0 recs

... or you could just do 1 - "Prob0"

That makes more sense.

by DavidRF on Nov 16, 2009 11:38 AM EST up reply actions 0 recs

One of the things to remember is that its not always about the total amount of runs expected to score

in some situations, the probability of scoring ONE run is very important. Early in games, your rule of not making outs in any situation is about right. Late in games, there are some exceptions. In order to look at this, you’d need to look at the WPA (win probability added) for the situation.

If its a tie game in the bottom of the ninth, we obviously don’t care about the chances of scoring a second run, since the game will be over then. In this case, we look solely at the chances of scoring one run. Even in the 8th in a tie game, the first run is much more important than any subsequent runs. Again, looking at WPA will help.

One other thing to consider is the amount of times that a bunt fails – turns into an 0-2 count or doesn’t advance the runner. This makes bunting even less of a good idea.

by lookatthosetwins on Nov 16, 2009 11:20 AM EST reply actions 0 recs

I guess we'd call this "run leverage index"

or the WPA value at any given point of scoring a single run. Late in the game, tied, it’s probably quite large. Down 10 runs, it’s very low.

by Adam Peterson on Nov 16, 2009 8:26 PM EST up reply actions 0 recs

Other things to consider

The run environment – with our offense, especially against a mediocre pitcher, “productive” outs are even less productive. If its an intersquad scrimmage of the Royals facing Greinke, there may be many situations where bunting is a good idea.

Making the corners play in. If you NEVER bunt early in games, the infield can play back, and do shifts that the can’t do if they are protecting for the bunt.

As I said above, the chances of the bunt not advancing the runner need to be taken into account, but I forgot to mention that so do the chances of not creating an out at all, via hit or error. For almost anyone, the chances of the bunt failing will be higher than the chances of a hit. Guys like Gomez and Punto are fast enough to beat some bunts, even when the defense knows you will bunt, but they also pop the bunts up a lot or foul 2 off and end up striking out.

by lookatthosetwins on Nov 16, 2009 11:26 AM EST reply actions 0 recs

“With a runner on third base and one out, getting the run home via a sacrifice or simple ground-out is productive (the expectancy goes from .9655 to 1). Interestingly, this isn’t true if no one is out (decreases from 1.3146 to 1.2789).”

I assume this is just a typo, but you mean .9655 to 1.1064

by Tweety on Nov 16, 2009 1:10 PM EST reply actions 0 recs

Defensive standpoint

I was doing some napkin math looking at how to take in the runs expected and run probability chart that DavidRF posted above, but then work exploded at me and I just wanted to get some thoughts out there.

I was wondering if you could take a statistical approach to see the percentage chance change of letting in a run vs. not letting any in any when you attempt to get a force out through an intentional walk. What is the risk if you had a guy on 2nd with 2 outs, and were considering walking the current batter to face the next one and have a force at 2 bases? (What’s the percentage chance increase of runs scored?) As people have mentioned, obviously this changes depending on who is being walked, who is next up to bat, what point in the game it is, etc. Walking Mauer to get to Punto (yes, I know that’s not the lineup order…) is definitely going to sway the scale slightly. I’m curious how much of a change there would be, though. These stats are just league averages as well. Can we calculate these for the Twins in 2009? Are any individual teams that far off league average? And would there be a way to calculate runs expected based on the actual batter at the plate; ie Mauer, and not Mr. League Average Batter of Averages? (I realize this would lead to smaller sample sizes and skewed results, but you could come up with something if you really, really liked math and wanted to run simulations and used something like BP’s “comparable hitter” stats. Where’s Nate Silver when we need him?)

Other situations that may be interesting to look at:
2nd and 3rd, 2 outs, an IBB to make it Loaded with 2 outs
2nd and 1 out, an IBB to make it 1st and 2nd with 1 out
etc.

Hopefully I’m making sense, and maybe I’m missing something, but I think it’d be interesting to figure out how much an intentional walk helps/hurts your chances. I assume any sort of answer would be “It depends…” since we’re dealing with real pitchers and hitters in situational circumstances and not ROBOTs. Maybe the simplest way to see if people are making the correct choice would be to get the charts above minus intentional walks?

Work beckons…

by dlfedie on Nov 16, 2009 2:14 PM EST reply actions 0 recs

Great question, Fedie.

Get on that one and get back to me tomorrow by lunch.

by Jesse on Nov 16, 2009 2:24 PM EST up reply actions 0 recs

Check DavidRF's link above

According to TangoTiger’s numbers there, there are a few situations where an IBB improves the odds of allowing 0 runs:

Runner at 3rd, 1 out (.338 to .345) – That strikes me as intuitive (setting up a double play), although it’s a really small improvement. I’d also note that the IBB with a runner on second and one out has the opposite effect, slightly more pronounced.

Runners at 2nd & 3rd, 1 out (.305 to .330) – Easily the largest increase in odds of a scoreless inning, and another predictable improvement, setting up the double play with one out.

Runners at 1st & 3rd, 0 outs (.124 to .128) – The raw numbers seem to say that there’s a tiny bit of value in setting up the force at home. In reality, I’d guess that this is largely a product of strategy – in the early innings or with a multi-run lead, a team will concede the run at home to get a double play even on a grounder to the corner infielders, inflating the nonzero results for that situation. In a tie game situation, the infield would be more likely to attempt a play at home plate with runners at the corners, which might increase the chance of a shutout inning.

I’d note that there are many of these that are very close, to the extent that I don’t know useful their results actually are.

"There are only two things that are infinite, the universe and human stupidity, and I'm not sure about the former." - Albert Einstein

by BeefMaster on Nov 16, 2009 2:36 PM EST up reply actions 0 recs

Maybe if you got the results for those at-bats after an IBB and compare them to TangoTiger’s chart, which is for all at-bats? That would at least tell us how well managers are at guessing to walk someone in order to put a force out at 1 or more bases, compared to the league average of getting guys on 1st and 2nd with 1 out, etc, the “old-fashioned way”.

I guess getting more data on all of the random variables involved (Was a reliever brought in?, Who is up to bat?, Is it a Tuesday?) just adds to the complexity of the whole thing and leads to more “who really cares?” moments. Also, as you pointed out, in early innings you may just be going for damage control rather than stopping the game-tying or game-winning run. It could be interesting to see how an inning by inning spread would look. Then again, maybe not.

I realize this is just nit-picky and probably not useful at all, but some people sleep better at night knowing that this information is at least out there.

by dlfedie on Nov 16, 2009 4:43 PM EST up reply actions 0 recs

Breaking it down for an individual batter

What you need to be able to calculate this is the odds of all outcomes (1B, 2B, 3B, HR, BB, K, out, GIDP, etc) based on the pitcher and the batter versus the odds in general. For example, the odds Mauer makes an out are < 0.6 versus an average batter is 0.66.

by DJL44 on Nov 16, 2009 5:27 PM EST up reply actions 0 recs

Earl Weaver

was another manager who understood the value of outs. That’s why he virtually never sacrificed.

Of course, sacrificing is different from bunting. With good bunting leadoff type guys (Brett Butler comes to mind), it can be a weapon just getting on base.

by Adam Peterson on Nov 16, 2009 8:29 PM EST reply actions 0 recs

Conceptual issue

Jon – you touched on this topic ever so briefly, but I want to stress it: Your comparison is between making a productive out and “not making an out” (as in nothing happens, somehow your PA is erased and the next persion comes up). This is flawed logic. It is comparable to saying that pitchers shouldn’t give up hits (or walks), because they will be worse off for it. Agreed, but they still have to pitch.

The question, in the case of LNP is this:
Hypothetically (my numbers are bogus), LNP’s outcomes when he’s not sacrificing are the following: 5% walk, 20% single, 4% double, 1% HR, 30% productive out, 30% non-productive out.
When trying to make a productive out, his outcomes are: 5% walk, 5% single, 70% productive out, 20% non-productive out.
Now let’s run this through the matrix, and see what comes out, to determine best game strategy.

Another point, if you have two players who both hit .275/.350/.450, and one makes a productive out in 5% of at-bats, while the other makes a productive out in 10% of at-bats, then who is more valuable? My math says that the productive out is worth about .2 runs over the unproductive out, so over 500 PAs, the second player will produce .05*.2*500 = 5 more runs.

by snolls on Nov 18, 2009 8:46 AM EST reply actions 0 recs

Gotta mix DP's in there too.

Yes, all other things being equal, its best to make your outs productively than non-productively. You just have be careful because all other things usually aren’t equal. You go for more bat control, you might not hit the ball as hard. You avoid strikeouts and and you might ground into more DP’s. Too often productive outs are used to rationalize the performance of an unproductive hitter.

by DavidRF on Nov 18, 2009 8:46 PM EST up reply actions 0 recs

Too often productive outs are used to rationalize the performance of an unproductive hitter.

Yes. But given that you have unproductive hitters in your line-up, what do you do? You try to minimize the impact of their outs. Teams like Boston and New York don’t have to worry about this as much because there aren’t many easy outs in their line-ups. Over the last decade, the Twins have overachieved with three easy outs in their line-ups by minimizing the impact of their outs.

Aside: One of the reasons Gardenhire puts an easy out in the second slot is to break up the easy outs. If you bunch them together at the back of the line-up, you have fewer innings with run potential. For this to work, the number 2 hitter needs to be able to bunt and hit it the other way. But he need not be a .300 hitter or anything. I don’t know if it works better than the alternative, but that’s his rationale.

That’s not to say that they’ve done a perfect job in this area, or that they haven’t overused productive outs on productive hitters like Span. But they tend to do better than the rest of the league in this area.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Nov 19, 2009 12:56 PM EST up reply actions 0 recs